2024-03-23T09:28:59.463 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-23T09:28:59.463 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi --owner scheduled_yuriw@teuthology --verbose --archive /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} -- /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427/orig.config.yaml 2024-03-23T09:28:59.504 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T09:28:59.589 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427 branch: wip-yuri11-testing-2024-03-21-0851-reef description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: yweinste@redhat.com first_in_suite: false job_id: '7617427' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri11-testing-2024-03-21-0851-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 82316b04821a6c2c1d458a881920454bbadb8837 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 82316b04821a6c2c1d458a881920454bbadb8837 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri11-testing-2024-03-21-0851-reef sha1: 82316b04821a6c2c1d458a881920454bbadb8837 owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1919 sha1: 82316b04821a6c2c1d458a881920454bbadb8837 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri11-testing-2024-03-21-0851-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 82316b04821a6c2c1d458a881920454bbadb8837 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN4NZx+BuJhDznyGKe/diyX8WDTP3EKrADaXXu6wf534XL6d+2TjpBMY3IxKxezfWgv0a/reLAHWTfEnXFwD10w= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIdxuUty99rrpzpD84B71aZ9IUxYu6J/FBGUB3ehkllQxBfNgtQAITD6ApkLFMCuIWn2rUen4iMnaOezzkYrt+4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_13:15:24 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-23T09:28:59.589 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa; will attempt to use it 2024-03-23T09:28:59.590 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks 2024-03-23T09:28:59.590 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-23T09:28:59.592 INFO:teuthology.task.internal:Checking packages... 2024-03-23T09:28:59.614 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '82316b04821a6c2c1d458a881920454bbadb8837' 2024-03-23T09:28:59.614 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T09:28:59.614 INFO:teuthology.packaging:ref: None 2024-03-23T09:28:59.614 INFO:teuthology.packaging:tag: None 2024-03-23T09:28:59.614 INFO:teuthology.packaging:branch: wip-yuri11-testing-2024-03-21-0851-reef 2024-03-23T09:28:59.614 INFO:teuthology.packaging:sha1: 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T09:28:59.615 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri11-testing-2024-03-21-0851-reef 2024-03-23T09:28:59.758 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-954.g82316b04 2024-03-23T09:28:59.759 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-23T09:28:59.766 INFO:teuthology.task.internal:no buildpackages task found 2024-03-23T09:28:59.766 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-23T09:28:59.808 INFO:teuthology.task.internal:Saving configuration 2024-03-23T09:28:59.822 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-23T09:28:59.829 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T09:28:59.854 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 09:24:09.621044', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN4NZx+BuJhDznyGKe/diyX8WDTP3EKrADaXXu6wf534XL6d+2TjpBMY3IxKxezfWgv0a/reLAHWTfEnXFwD10w='} 2024-03-23T09:28:59.875 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 09:24:09.622540', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIdxuUty99rrpzpD84B71aZ9IUxYu6J/FBGUB3ehkllQxBfNgtQAITD6ApkLFMCuIWn2rUen4iMnaOezzkYrt+4='} 2024-03-23T09:28:59.876 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-23T09:28:59.882 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-23T09:28:59.882 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-23T09:28:59.882 INFO:teuthology.run_tasks:Running task console_log... 2024-03-23T09:28:59.949 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdc67141160>, signals=[15]) 2024-03-23T09:28:59.949 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-23T09:28:59.955 INFO:teuthology.task.internal:Opening connections... 2024-03-23T09:28:59.955 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:28:59.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T09:29:00.060 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:29:00.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T09:29:00.150 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-23T09:29:00.157 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-03-23T09:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-03-23T09:29:00.175 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-03-23T09:29:00.231 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-03-23T09:29:00.231 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-03-23T09:29:00.231 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-03-23T09:29:00.231 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-03-23T09:29:00.232 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-03-23T09:29:00.232 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-03-23T09:29:00.232 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-23T09:29:00.232 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-03-23T09:29:00.232 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-03-23T09:29:00.232 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-23T09:29:00.232 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-03-23T09:29:00.232 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T09:29:00.232 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-23T09:29:00.232 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T09:29:00.233 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-03-23T09:29:00.259 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-03-23T09:29:00.276 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-03-23T09:29:00.276 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-03-23T09:29:00.332 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-03-23T09:29:00.332 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-03-23T09:29:00.333 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-03-23T09:29:00.333 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-03-23T09:29:00.333 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-03-23T09:29:00.333 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-03-23T09:29:00.333 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-23T09:29:00.333 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-03-23T09:29:00.333 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-03-23T09:29:00.333 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-23T09:29:00.333 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-03-23T09:29:00.333 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T09:29:00.333 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-23T09:29:00.333 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T09:29:00.333 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-03-23T09:29:00.354 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-23T09:29:00.364 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-23T09:29:00.370 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-23T09:29:00.370 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-03-23T09:29:00.372 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-03-23T09:29:00.389 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-23T09:29:00.394 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-23T09:29:00.395 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-03-23T09:29:00.429 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-03-23T09:29:00.461 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-23T09:29:00.516 INFO:teuthology.run_tasks:Running task kernel... 2024-03-23T09:29:00.529 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-23T09:29:00.529 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-23T09:29:00.529 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T09:29:00.530 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T09:29:00.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:29:00.549 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-03-23T09:29:00.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:29:00.550 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-03-23T09:29:00.607 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-361.el9.x86_64 2024-03-23T09:29:00.608 INFO:teuthology.task.kernel:Running kernel on smithi175: 5.14.0-361.el9.x86_64 2024-03-23T09:29:00.608 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum install -y kernel 2024-03-23T09:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-361.el9.x86_64 2024-03-23T09:29:00.609 INFO:teuthology.task.kernel:Running kernel on smithi047: 5.14.0-361.el9.x86_64 2024-03-23T09:29:00.609 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum install -y kernel 2024-03-23T09:29:03.356 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - AppStream 6.0 MB/s | 6.7 MB 00:01 2024-03-23T09:29:03.363 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS-9 - AppStream 5.8 MB/s | 6.7 MB 00:01 2024-03-23T09:29:05.313 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS-9 - BaseOS 2.4 MB/s | 2.0 MB 00:00 2024-03-23T09:29:05.356 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - BaseOS 2.4 MB/s | 2.0 MB 00:00 2024-03-23T09:29:06.438 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-03-23T09:29:07.012 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.0 MB 00:01 2024-03-23T09:29:11.021 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 19 MB 00:02 2024-03-23T09:29:14.413 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 19 MB 00:06 2024-03-23T09:29:16.554 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.1 MB 00:02 2024-03-23T09:29:18.463 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 16 kB 00:00 2024-03-23T09:29:20.247 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.1 MB 00:02 2024-03-23T09:29:20.739 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 9.6 MB/s | 21 MB 00:02 2024-03-23T09:29:22.124 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 16 kB 00:00 2024-03-23T09:29:23.099 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-03-23T09:29:25.630 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-03-23T09:29:27.662 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-23T09:29:27.663 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-23T09:29:27.757 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-03-23T09:29:27.764 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-03-23T09:29:27.764 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-03-23T09:29:27.764 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-03-23T09:29:27.764 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-03-23T09:29:27.764 INFO:teuthology.orchestra.run.smithi175.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-23T09:29:27.764 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-03-23T09:29:27.765 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-23T09:29:27.765 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-23T09:29:27.765 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-23T09:29:27.765 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:29:27.765 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-03-23T09:29:27.765 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-03-23T09:29:27.765 INFO:teuthology.orchestra.run.smithi175.stdout:Install 4 Packages 2024-03-23T09:29:27.765 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:29:27.766 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 76 M 2024-03-23T09:29:27.766 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 123 M 2024-03-23T09:29:27.766 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-03-23T09:29:28.116 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-03-23T09:29:28.965 INFO:teuthology.orchestra.run.smithi175.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 46 kB/s | 44 kB 00:00 2024-03-23T09:29:30.289 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-23T09:29:30.289 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-23T09:29:30.389 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-03-23T09:29:30.406 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-03-23T09:29:30.406 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-03-23T09:29:30.406 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-03-23T09:29:30.407 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-03-23T09:29:30.407 INFO:teuthology.orchestra.run.smithi047.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-23T09:29:30.407 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-03-23T09:29:30.407 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-23T09:29:30.407 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-23T09:29:30.407 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-23T09:29:30.407 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:29:30.407 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-03-23T09:29:30.407 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-03-23T09:29:30.407 INFO:teuthology.orchestra.run.smithi047.stdout:Install 4 Packages 2024-03-23T09:29:30.407 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:29:30.408 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 76 M 2024-03-23T09:29:30.408 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 123 M 2024-03-23T09:29:30.408 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-03-23T09:29:30.892 INFO:teuthology.orchestra.run.smithi047.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 142 kB/s | 44 kB 00:00 2024-03-23T09:29:38.425 INFO:teuthology.orchestra.run.smithi047.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 1.9 MB/s | 15 MB 00:07 2024-03-23T09:29:45.875 INFO:teuthology.orchestra.run.smithi047.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 1.8 MB/s | 27 MB 00:14 2024-03-23T09:29:51.291 INFO:teuthology.orchestra.run.smithi047.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 1.6 MB/s | 33 MB 00:20 2024-03-23T09:29:51.291 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-03-23T09:29:51.292 INFO:teuthology.orchestra.run.smithi047.stdout:Total 3.6 MB/s | 76 MB 00:20 2024-03-23T09:29:51.983 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-03-23T09:29:52.420 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-03-23T09:29:52.420 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-03-23T09:29:53.473 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-03-23T09:29:53.474 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-03-23T09:29:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-03-23T09:29:56.697 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-23T09:29:56.776 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-23T09:29:57.602 INFO:teuthology.orchestra.run.smithi175.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 516 kB/s | 15 MB 00:29 2024-03-23T09:29:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-23T09:29:59.853 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-23T09:30:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-23T09:30:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-23T09:30:10.059 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-23T09:30:23.473 INFO:teuthology.orchestra.run.smithi175.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 516 kB/s | 27 MB 00:54 2024-03-23T09:30:40.856 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-23T09:30:40.856 INFO:teuthology.orchestra.run.smithi047.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-23T09:30:40.856 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:30:40.902 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-23T09:30:41.295 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-23T09:30:41.295 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-23T09:30:41.295 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-23T09:30:41.295 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-23T09:30:41.612 INFO:teuthology.orchestra.run.smithi175.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 465 kB/s | 33 MB 01:13 2024-03-23T09:30:41.613 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-03-23T09:30:41.613 INFO:teuthology.orchestra.run.smithi175.stdout:Total 1.0 MB/s | 76 MB 01:13 2024-03-23T09:30:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-23T09:30:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:30:41.706 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-03-23T09:30:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-23T09:30:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-23T09:30:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-23T09:30:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-23T09:30:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:30:41.707 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-03-23T09:30:42.042 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-03-23T09:30:42.151 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-23T09:30:42.151 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-23T09:30:42.151 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-23T09:30:42.151 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-23T09:30:42.151 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-23T09:30:42.151 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-23T09:30:42.151 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=None, version=distro) 2024-03-23T09:30:42.152 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-03-23T09:30:42.235 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-23T09:30:42.235 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-23T09:30:42.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -qi grub2-tools 2024-03-23T09:30:42.276 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-03-23T09:30:42.278 INFO:teuthology.orchestra.run.smithi047.stdout:Name : grub2-tools 2024-03-23T09:30:42.278 INFO:teuthology.orchestra.run.smithi047.stdout:Epoch : 1 2024-03-23T09:30:42.278 INFO:teuthology.orchestra.run.smithi047.stdout:Version : 2.06 2024-03-23T09:30:42.279 INFO:teuthology.orchestra.run.smithi047.stdout:Release : 68.el9 2024-03-23T09:30:42.279 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: x86_64 2024-03-23T09:30:42.279 INFO:teuthology.orchestra.run.smithi047.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-23T09:30:42.279 INFO:teuthology.orchestra.run.smithi047.stdout:Group : Unspecified 2024-03-23T09:30:42.279 INFO:teuthology.orchestra.run.smithi047.stdout:Size : 8264254 2024-03-23T09:30:42.279 INFO:teuthology.orchestra.run.smithi047.stdout:License : GPLv3+ 2024-03-23T09:30:42.279 INFO:teuthology.orchestra.run.smithi047.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-23T09:30:42.279 INFO:teuthology.orchestra.run.smithi047.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-23T09:30:42.279 INFO:teuthology.orchestra.run.smithi047.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-23T09:30:42.279 INFO:teuthology.orchestra.run.smithi047.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-23T09:30:42.279 INFO:teuthology.orchestra.run.smithi047.stdout:Packager : builder@centos.org 2024-03-23T09:30:42.279 INFO:teuthology.orchestra.run.smithi047.stdout:Vendor : CentOS 2024-03-23T09:30:42.279 INFO:teuthology.orchestra.run.smithi047.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-23T09:30:42.280 INFO:teuthology.orchestra.run.smithi047.stdout:Summary : Support tools for GRUB. 2024-03-23T09:30:42.280 INFO:teuthology.orchestra.run.smithi047.stdout:Description : 2024-03-23T09:30:42.280 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:30:42.280 INFO:teuthology.orchestra.run.smithi047.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-23T09:30:42.280 INFO:teuthology.orchestra.run.smithi047.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-23T09:30:42.280 INFO:teuthology.orchestra.run.smithi047.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-23T09:30:42.280 INFO:teuthology.orchestra.run.smithi047.stdout:hardware devices. 2024-03-23T09:30:42.280 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:30:42.280 INFO:teuthology.orchestra.run.smithi047.stdout:This subpackage provides tools for support of all platforms. 2024-03-23T09:30:42.282 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-23T09:30:42.282 INFO:teuthology.task.kernel:Updating grub on smithi047 to boot 5.14.0-430.el9.x86_64 2024-03-23T09:30:42.282 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-23T09:30:42.636 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-03-23T09:30:42.636 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-03-23T09:30:42.909 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2024-03-23T09:30:43.661 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-03-23T09:30:43.662 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-03-23T09:30:44.286 INFO:teuthology.orchestra.run.smithi047.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-23T09:30:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:done 2024-03-23T09:30:44.309 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-03-23T09:30:44.324 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.b0SxoUA8Lf 2024-03-23T09:30:44.324 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.b0SxoUA8Lf 2024-03-23T09:30:44.396 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.b0SxoUA8Lf 2024-03-23T09:30:44.540 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.b0SxoUA8Lf is 6KB 2024-03-23T09:30:44.588 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.b0SxoUA8Lf 2024-03-23T09:30:44.603 DEBUG:teuthology.orchestra.run.smithi047:> sudo /bin/ls /boot/loader/entries || true 2024-03-23T09:30:44.671 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-23T09:30:44.672 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-23T09:30:44.672 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-23T09:30:44.672 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-23T09:30:44.673 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-23T09:30:44.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2024-03-23T09:30:46.137 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-03-23T09:30:46.768 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-23T09:30:46.853 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-23T09:30:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-23T09:30:49.736 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-23T09:30:54.437 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-23T09:30:54.478 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-23T09:30:59.195 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-23T09:31:14.837 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-23T09:31:14.838 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-23T09:31:14.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T09:31:28.544 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-23T09:31:28.544 INFO:teuthology.orchestra.run.smithi175.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-23T09:31:28.544 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:31:28.589 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-23T09:31:28.957 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-23T09:31:28.958 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-23T09:31:28.958 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-23T09:31:28.958 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-23T09:31:29.412 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-23T09:31:29.412 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:31:29.412 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-03-23T09:31:29.412 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-23T09:31:29.412 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-23T09:31:29.412 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-23T09:31:29.412 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-23T09:31:29.412 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:31:29.412 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-03-23T09:31:29.768 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2024-03-23T09:31:29.865 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-23T09:31:29.866 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-23T09:31:29.866 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-23T09:31:29.866 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-23T09:31:29.866 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-23T09:31:29.866 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-23T09:31:29.866 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=None, version=distro) 2024-03-23T09:31:29.866 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2024-03-23T09:31:29.946 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-23T09:31:29.946 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-23T09:31:29.946 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -qi grub2-tools 2024-03-23T09:31:29.980 INFO:teuthology.orchestra.run.smithi175.stdout:Name : grub2-tools 2024-03-23T09:31:29.980 INFO:teuthology.orchestra.run.smithi175.stdout:Epoch : 1 2024-03-23T09:31:29.980 INFO:teuthology.orchestra.run.smithi175.stdout:Version : 2.06 2024-03-23T09:31:29.980 INFO:teuthology.orchestra.run.smithi175.stdout:Release : 68.el9 2024-03-23T09:31:29.980 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: x86_64 2024-03-23T09:31:29.980 INFO:teuthology.orchestra.run.smithi175.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-23T09:31:29.980 INFO:teuthology.orchestra.run.smithi175.stdout:Group : Unspecified 2024-03-23T09:31:29.980 INFO:teuthology.orchestra.run.smithi175.stdout:Size : 8264254 2024-03-23T09:31:29.980 INFO:teuthology.orchestra.run.smithi175.stdout:License : GPLv3+ 2024-03-23T09:31:29.980 INFO:teuthology.orchestra.run.smithi175.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-23T09:31:29.980 INFO:teuthology.orchestra.run.smithi175.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout:Packager : builder@centos.org 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout:Vendor : CentOS 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout:Summary : Support tools for GRUB. 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout:Description : 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout:hardware devices. 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:31:29.981 INFO:teuthology.orchestra.run.smithi175.stdout:This subpackage provides tools for support of all platforms. 2024-03-23T09:31:29.982 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-23T09:31:29.982 INFO:teuthology.task.kernel:Updating grub on smithi175 to boot 5.14.0-430.el9.x86_64 2024-03-23T09:31:29.983 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-23T09:31:30.646 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2024-03-23T09:31:32.042 INFO:teuthology.orchestra.run.smithi175.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-23T09:31:32.062 INFO:teuthology.orchestra.run.smithi175.stderr:done 2024-03-23T09:31:32.065 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-03-23T09:31:32.081 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.YsJnmrKYSb 2024-03-23T09:31:32.081 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.YsJnmrKYSb 2024-03-23T09:31:32.149 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.YsJnmrKYSb 2024-03-23T09:31:32.293 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.YsJnmrKYSb is 6KB 2024-03-23T09:31:32.341 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.YsJnmrKYSb 2024-03-23T09:31:32.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo /bin/ls /boot/loader/entries || true 2024-03-23T09:31:32.424 INFO:teuthology.orchestra.run.smithi175.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-23T09:31:32.424 INFO:teuthology.orchestra.run.smithi175.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-23T09:31:32.424 INFO:teuthology.orchestra.run.smithi175.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-23T09:31:32.424 INFO:teuthology.orchestra.run.smithi175.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-23T09:31:32.426 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-23T09:31:32.558 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2024-03-23T09:31:33.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-03-23T09:31:42.288 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-23T09:31:42.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T09:32:02.580 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-23T09:32:02.581 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-23T09:32:02.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T09:32:21.155 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-03-23T09:32:30.157 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-23T09:32:30.158 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T09:32:42.302 DEBUG:teuthology.orchestra.remote:timed out 2024-03-23T09:32:54.315 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-23T09:32:54.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T09:32:54.637 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-03-23T09:32:55.025 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-23T09:32:55.025 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-430.el9.x86_64"... 2024-03-23T09:32:55.025 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-03-23T09:32:55.040 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-430.el9.x86_64 2024-03-23T09:32:55.040 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-23T09:32:55.041 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-23T09:32:55.041 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-23T09:32:56.041 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-23T09:32:56.042 DEBUG:teuthology.orchestra.run.smithi047:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-23T09:32:56.099 INFO:teuthology.orchestra.run.smithi047.stdout:ttyS1 2024-03-23T09:32:56.133 DEBUG:teuthology.parallel:result is None 2024-03-23T09:33:30.161 DEBUG:teuthology.orchestra.remote:timed out 2024-03-23T09:33:42.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-23T09:33:42.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T09:33:42.503 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-03-23T09:33:42.904 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-23T09:33:42.904 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-430.el9.x86_64"... 2024-03-23T09:33:42.904 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-03-23T09:33:42.920 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-430.el9.x86_64 2024-03-23T09:33:42.920 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-23T09:33:42.920 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-23T09:33:42.920 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-23T09:33:43.922 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-23T09:33:43.922 DEBUG:teuthology.orchestra.run.smithi175:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-23T09:33:43.997 INFO:teuthology.orchestra.run.smithi175.stdout:ttyS1 2024-03-23T09:33:44.028 DEBUG:teuthology.parallel:result is None 2024-03-23T09:33:44.028 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-23T09:33:44.036 INFO:teuthology.task.internal:Creating test directory... 2024-03-23T09:33:44.036 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T09:33:44.039 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T09:33:44.056 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-23T09:33:44.062 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-23T09:33:44.071 INFO:teuthology.task.internal:Creating archive directory... 2024-03-23T09:33:44.071 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T09:33:44.096 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T09:33:44.127 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-23T09:33:44.133 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-23T09:33:44.133 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T09:33:44.164 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T09:33:44.201 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T09:33:44.206 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T09:33:44.215 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T09:33:44.219 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T09:33:44.221 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-23T09:33:44.227 INFO:teuthology.task.internal:Configuring sudo... 2024-03-23T09:33:44.227 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T09:33:44.258 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T09:33:44.290 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-23T09:33:44.298 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-23T09:33:44.298 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T09:33:44.328 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T09:33:44.345 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T09:33:44.424 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T09:33:44.471 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T09:33:44.556 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T09:33:44.601 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:33:44.601 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T09:33:44.671 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T09:33:44.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T09:33:44.766 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T09:33:44.848 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T09:33:44.893 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:33:44.893 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T09:33:44.961 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-03-23T09:33:44.964 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-03-23T09:33:45.010 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T09:33:45.051 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T09:33:45.475 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-23T09:33:45.483 INFO:teuthology.task.internal:Starting timer... 2024-03-23T09:33:45.483 INFO:teuthology.run_tasks:Running task pcp... 2024-03-23T09:33:45.559 INFO:teuthology.run_tasks:Running task selinux... 2024-03-23T09:33:45.567 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-03-23T09:33:45.633 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-03-23T09:33:45.635 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-03-23T09:33:45.706 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-03-23T09:33:45.710 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-23T09:33:45.710 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-03-23T09:33:45.730 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-03-23T09:33:45.731 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-03-23T09:33:45.758 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-03-23T09:33:45.758 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2024-03-23T09:33:45.758 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T09:33:45.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:33:45.799 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T09:33:45.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:33:45.827 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-23T09:33:45.827 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-03-23T09:33:45.874 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-03-23T09:33:45.910 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-23T09:33:45.918 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-23T09:33:45.925 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-23T09:33:46.049 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-23T09:33:46.072 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-23T09:33:46.073 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-23T09:37:55.134 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-03-23T09:37:55.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-23T09:37:55.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T09:37:55.225 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-03-23T09:37:55.309 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-23T09:37:55.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-23T09:37:55.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T09:37:55.396 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-03-23T09:37:55.473 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-23T09:37:55.474 INFO:teuthology.run_tasks:Running task clock... 2024-03-23T09:37:55.484 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-23T09:37:55.484 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T09:37:55.485 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T09:37:55.487 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T09:37:55.487 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T09:37:55.517 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T09:37:55.535 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T09:37:55.543 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T09:37:55.561 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T09:37:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-03-23T09:37:55.578 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-03-23T09:37:55.590 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-03-23T09:37:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T09:37:55.603 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-03-23T09:37:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T09:37:55.619 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T09:37:55.636 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T09:37:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-03-23T09:37:55.665 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T09:37:55.666 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-03-23T09:37:55.666 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T09:37:55.666 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T09:37:55.666 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T09:37:55.666 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T09:37:55.691 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-03-23T09:37:55.694 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T09:37:55.694 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-03-23T09:37:55.694 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T09:37:55.694 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T09:37:55.694 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T09:37:55.694 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T09:37:55.695 INFO:teuthology.run_tasks:Running task install... 2024-03-23T09:37:55.704 DEBUG:teuthology.task.install:project ceph 2024-03-23T09:37:55.704 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'}} 2024-03-23T09:37:55.704 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'} 2024-03-23T09:37:55.704 INFO:teuthology.task.install:Using flavor: default 2024-03-23T09:37:55.710 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-23T09:37:55.711 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-23T09:37:55.711 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'tag': None, 'wait_for_package': False} 2024-03-23T09:37:55.711 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T09:37:55.713 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'tag': None, 'wait_for_package': False} 2024-03-23T09:37:55.714 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T09:37:55.862 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/ 2024-03-23T09:37:55.862 INFO:teuthology.task.install.rpm:Package version is 18.2.2-954.g82316b04 2024-03-23T09:37:55.931 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/ 2024-03-23T09:37:55.931 INFO:teuthology.task.install.rpm:Package version is 18.2.2-954.g82316b04 2024-03-23T09:37:55.973 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T09:37:55.974 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:37:55.974 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T09:37:56.001 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T09:37:56.002 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/82316b04821a6c2c1d458a881920454bbadb8837/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T09:37:56.044 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T09:37:56.044 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:37:56.044 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T09:37:56.072 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T09:37:56.073 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T09:37:56.073 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/82316b04821a6c2c1d458a881920454bbadb8837/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T09:37:56.141 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T09:37:56.151 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T09:37:56.218 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-03-23T09:37:56.220 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-03-23T09:37:56.220 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T09:37:56.287 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-03-23T09:37:56.289 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-03-23T09:37:56.471 INFO:teuthology.orchestra.run.smithi175.stdout:74 files removed 2024-03-23T09:37:56.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T09:37:56.538 INFO:teuthology.orchestra.run.smithi047.stdout:74 files removed 2024-03-23T09:37:56.566 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T09:37:57.210 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 189 kB/s | 77 kB 00:00 2024-03-23T09:37:57.243 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 207 kB/s | 77 kB 00:00 2024-03-23T09:37:57.627 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 30 kB/s | 9.9 kB 00:00 2024-03-23T09:37:57.635 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 26 kB/s | 9.9 kB 00:00 2024-03-23T09:37:58.019 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-03-23T09:37:58.044 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-03-23T09:37:58.496 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-23T09:37:58.796 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS-9 - AppStream 9.4 MB/s | 6.7 MB 00:00 2024-03-23T09:38:00.064 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2024-03-23T09:38:00.357 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS-9 - BaseOS 4.9 MB/s | 2.0 MB 00:00 2024-03-23T09:38:01.539 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 8.0 MB 00:01 2024-03-23T09:38:01.606 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.0 MB 00:00 2024-03-23T09:38:05.450 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 19 MB 00:02 2024-03-23T09:38:16.698 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 817 kB/s | 6.1 MB 00:07 2024-03-23T09:38:18.298 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-03-23T09:38:19.721 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-03-23T09:38:19.907 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-23T09:38:25.049 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-03-23T09:38:27.496 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T09:38:27.497 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T09:38:27.511 INFO:teuthology.orchestra.run.smithi047.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-23T09:38:27.512 INFO:teuthology.orchestra.run.smithi047.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-23T09:38:27.513 INFO:teuthology.orchestra.run.smithi047.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-23T09:38:27.514 INFO:teuthology.orchestra.run.smithi047.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-23T09:38:27.516 INFO:teuthology.orchestra.run.smithi047.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-23T09:38:27.519 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-23T09:38:27.520 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-23T09:38:27.555 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.1 MB 00:04 2024-03-23T09:38:27.602 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-03-23T09:38:27.613 INFO:teuthology.orchestra.run.smithi047.stdout:=========================================================================================== 2024-03-23T09:38:27.613 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-03-23T09:38:27.613 INFO:teuthology.orchestra.run.smithi047.stdout:=========================================================================================== 2024-03-23T09:38:27.613 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-03-23T09:38:27.614 INFO:teuthology.orchestra.run.smithi047.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-23T09:38:27.614 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:18.2.2-954.g82316b04.el9 ceph 6.5 k 2024-03-23T09:38:27.614 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:18.2.2-954.g82316b04.el9 ceph 5.1 M 2024-03-23T09:38:27.614 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:18.2.2-954.g82316b04.el9 ceph 843 k 2024-03-23T09:38:27.614 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-954.g82316b04.el9 ceph 143 k 2024-03-23T09:38:27.614 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:18.2.2-954.g82316b04.el9 ceph 1.4 M 2024-03-23T09:38:27.614 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 138 k 2024-03-23T09:38:27.614 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 1.7 M 2024-03-23T09:38:27.614 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 7.4 M 2024-03-23T09:38:27.614 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 49 k 2024-03-23T09:38:27.614 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:18.2.2-954.g82316b04.el9 ceph 7.7 M 2024-03-23T09:38:27.614 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:18.2.2-954.g82316b04.el9 ceph 40 M 2024-03-23T09:38:27.615 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 264 k 2024-03-23T09:38:27.615 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 224 k 2024-03-23T09:38:27.615 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror x86_64 2:18.2.2-954.g82316b04.el9 ceph 221 k 2024-03-23T09:38:27.615 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 18 k 2024-03-23T09:38:27.615 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-23T09:38:27.615 INFO:teuthology.orchestra.run.smithi047.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-23T09:38:27.615 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-23T09:38:27.615 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:18.2.2-954.g82316b04.el9 ceph 31 k 2024-03-23T09:38:27.615 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:18.2.2-954.g82316b04.el9 ceph 704 k 2024-03-23T09:38:27.615 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:18.2.2-954.g82316b04.el9 ceph 127 k 2024-03-23T09:38:27.615 INFO:teuthology.orchestra.run.smithi047.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-23T09:38:27.615 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-23T09:38:27.615 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:18.2.2-954.g82316b04.el9 ceph 161 k 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:18.2.2-954.g82316b04.el9 ceph 320 k 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:18.2.2-954.g82316b04.el9 ceph 297 k 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:18.2.2-954.g82316b04.el9 ceph 99 k 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:18.2.2-954.g82316b04.el9 ceph 87 k 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:18.2.2-954.g82316b04.el9 ceph 3.0 M 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:18.2.2-954.g82316b04.el9 ceph 172 k 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-23T09:38:27.616 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-23T09:38:27.617 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-23T09:38:27.617 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-23T09:38:27.617 INFO:teuthology.orchestra.run.smithi047.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-23T09:38:27.617 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-23T09:38:27.617 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-23T09:38:27.617 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-23T09:38:27.617 INFO:teuthology.orchestra.run.smithi047.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-23T09:38:27.617 INFO:teuthology.orchestra.run.smithi047.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-23T09:38:27.617 INFO:teuthology.orchestra.run.smithi047.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-23T09:38:27.617 INFO:teuthology.orchestra.run.smithi047.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-23T09:38:27.617 INFO:teuthology.orchestra.run.smithi047.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-23T09:38:27.617 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:18.2.2-954.g82316b04.el9 ceph 3.3 M 2024-03-23T09:38:27.617 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:18.2.2-954.g82316b04.el9 ceph 3.0 M 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:18.2.2-954.g82316b04.el9 ceph 18 M 2024-03-23T09:38:27.618 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 23 k 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:18.2.2-954.g82316b04.el9 ceph 2.1 M 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 245 k 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:18.2.2-954.g82316b04.el9 ceph 4.7 M 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:18.2.2-954.g82316b04.el9 ceph 16 M 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 15 k 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:18.2.2-954.g82316b04.el9 ceph 25 k 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-23T09:38:27.619 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-23T09:38:27.620 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-23T09:38:27.620 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:18.2.2-954.g82316b04.el9 ceph 166 k 2024-03-23T09:38:27.620 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-23T09:38:27.620 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-23T09:38:27.620 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-23T09:38:27.620 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-23T09:38:27.620 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-23T09:38:27.620 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:18.2.2-954.g82316b04.el9 ceph 474 k 2024-03-23T09:38:27.620 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-23T09:38:27.620 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:18.2.2-954.g82316b04.el9 ceph 4.5 M 2024-03-23T09:38:27.620 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-23T09:38:27.620 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-23T09:38:27.620 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:18.2.2-954.g82316b04.el9 ceph 45 k 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:18.2.2-954.g82316b04.el9 ceph 127 k 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-23T09:38:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-23T09:38:27.622 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-23T09:38:27.622 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-23T09:38:27.622 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-23T09:38:27.622 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-23T09:38:27.622 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-23T09:38:27.622 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-23T09:38:27.622 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-23T09:38:27.622 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-23T09:38:27.622 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-23T09:38:27.622 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-23T09:38:27.622 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-23T09:38:27.622 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-23T09:38:27.622 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-23T09:38:27.623 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-23T09:38:27.623 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-23T09:38:27.623 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-23T09:38:27.623 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-23T09:38:27.623 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-23T09:38:27.623 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-23T09:38:27.623 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-23T09:38:27.623 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-23T09:38:27.623 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-23T09:38:27.623 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-23T09:38:27.623 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-23T09:38:27.623 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-23T09:38:27.623 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-23T09:38:27.624 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout:=========================================================================================== 2024-03-23T09:38:27.625 INFO:teuthology.orchestra.run.smithi047.stdout:Install 132 Packages 2024-03-23T09:38:27.626 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 24 Packages 2024-03-23T09:38:27.626 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:38:27.626 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 204 M 2024-03-23T09:38:27.626 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-03-23T09:38:28.301 INFO:teuthology.orchestra.run.smithi047.stdout:(1/156): ceph-18.2.2-954.g82316b04.el9.x86_64.r 35 kB/s | 6.5 kB 00:00 2024-03-23T09:38:28.592 INFO:teuthology.orchestra.run.smithi047.stdout:(2/156): ceph-fuse-18.2.2-954.g82316b04.el9.x86 2.8 MB/s | 843 kB 00:00 2024-03-23T09:38:28.668 INFO:teuthology.orchestra.run.smithi047.stdout:(3/156): ceph-immutable-object-cache-18.2.2-954 1.9 MB/s | 143 kB 00:00 2024-03-23T09:38:28.751 INFO:teuthology.orchestra.run.smithi047.stdout:(4/156): ceph-base-18.2.2-954.g82316b04.el9.x86 8.0 MB/s | 5.1 MB 00:00 2024-03-23T09:38:28.918 INFO:teuthology.orchestra.run.smithi047.stdout:(5/156): ceph-mgr-18.2.2-954.g82316b04.el9.x86_ 8.7 MB/s | 1.4 MB 00:00 2024-03-23T09:38:29.118 INFO:teuthology.orchestra.run.smithi047.stdout:(6/156): ceph-mds-18.2.2-954.g82316b04.el9.x86_ 4.7 MB/s | 2.1 MB 00:00 2024-03-23T09:38:29.419 INFO:teuthology.orchestra.run.smithi047.stdout:(7/156): ceph-mon-18.2.2-954.g82316b04.el9.x86_ 9.3 MB/s | 4.7 MB 00:00 2024-03-23T09:38:29.836 INFO:teuthology.orchestra.run.smithi047.stdout:(8/156): ceph-common-18.2.2-954.g82316b04.el9.x 11 MB/s | 18 MB 00:01 2024-03-23T09:38:29.903 INFO:teuthology.orchestra.run.smithi047.stdout:(9/156): ceph-selinux-18.2.2-954.g82316b04.el9. 378 kB/s | 25 kB 00:00 2024-03-23T09:38:30.005 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-03-23T09:38:30.128 INFO:teuthology.orchestra.run.smithi047.stdout:(10/156): ceph-radosgw-18.2.2-954.g82316b04.el9 11 MB/s | 7.7 MB 00:00 2024-03-23T09:38:30.212 INFO:teuthology.orchestra.run.smithi047.stdout:(11/156): cephfs-mirror-18.2.2-954.g82316b04.el 2.6 MB/s | 221 kB 00:00 2024-03-23T09:38:30.278 INFO:teuthology.orchestra.run.smithi047.stdout:(12/156): libcephfs-devel-18.2.2-954.g82316b04. 467 kB/s | 31 kB 00:00 2024-03-23T09:38:30.370 INFO:teuthology.orchestra.run.smithi047.stdout:(13/156): libcephfs2-18.2.2-954.g82316b04.el9.x 7.5 MB/s | 704 kB 00:00 2024-03-23T09:38:30.446 INFO:teuthology.orchestra.run.smithi047.stdout:(14/156): libcephsqlite-18.2.2-954.g82316b04.el 2.2 MB/s | 166 kB 00:00 2024-03-23T09:38:30.521 INFO:teuthology.orchestra.run.smithi047.stdout:(15/156): librados-devel-18.2.2-954.g82316b04.e 1.7 MB/s | 127 kB 00:00 2024-03-23T09:38:30.604 INFO:teuthology.orchestra.run.smithi047.stdout:(16/156): libradosstriper1-18.2.2-954.g82316b04 5.6 MB/s | 474 kB 00:00 2024-03-23T09:38:31.080 INFO:teuthology.orchestra.run.smithi047.stdout:(17/156): librgw2-18.2.2-954.g82316b04.el9.x86_ 9.4 MB/s | 4.5 MB 00:00 2024-03-23T09:38:31.148 INFO:teuthology.orchestra.run.smithi047.stdout:(18/156): python3-ceph-argparse-18.2.2-954.g823 678 kB/s | 45 kB 00:00 2024-03-23T09:38:31.222 INFO:teuthology.orchestra.run.smithi047.stdout:(19/156): python3-ceph-common-18.2.2-954.g82316 1.7 MB/s | 127 kB 00:00 2024-03-23T09:38:31.289 INFO:teuthology.orchestra.run.smithi047.stdout:(20/156): python3-cephfs-18.2.2-954.g82316b04.e 2.4 MB/s | 161 kB 00:00 2024-03-23T09:38:31.373 INFO:teuthology.orchestra.run.smithi047.stdout:(21/156): python3-rados-18.2.2-954.g82316b04.el 3.8 MB/s | 320 kB 00:00 2024-03-23T09:38:31.456 INFO:teuthology.orchestra.run.smithi047.stdout:(22/156): python3-rbd-18.2.2-954.g82316b04.el9. 3.5 MB/s | 297 kB 00:00 2024-03-23T09:38:31.532 INFO:teuthology.orchestra.run.smithi047.stdout:(23/156): python3-rgw-18.2.2-954.g82316b04.el9. 1.3 MB/s | 99 kB 00:00 2024-03-23T09:38:31.607 INFO:teuthology.orchestra.run.smithi047.stdout:(24/156): rbd-fuse-18.2.2-954.g82316b04.el9.x86 1.1 MB/s | 87 kB 00:00 2024-03-23T09:38:31.965 INFO:teuthology.orchestra.run.smithi047.stdout:(25/156): ceph-osd-18.2.2-954.g82316b04.el9.x86 5.8 MB/s | 16 MB 00:02 2024-03-23T09:38:32.024 INFO:teuthology.orchestra.run.smithi047.stdout:(26/156): rbd-mirror-18.2.2-954.g82316b04.el9.x 7.2 MB/s | 3.0 MB 00:00 2024-03-23T09:38:32.124 INFO:teuthology.orchestra.run.smithi047.stdout:(27/156): rbd-nbd-18.2.2-954.g82316b04.el9.x86_ 1.1 MB/s | 172 kB 00:00 2024-03-23T09:38:32.158 INFO:teuthology.orchestra.run.smithi047.stdout:(28/156): ceph-grafana-dashboards-18.2.2-954.g8 175 kB/s | 23 kB 00:00 2024-03-23T09:38:32.183 INFO:teuthology.orchestra.run.smithi047.stdout:(29/156): ceph-mgr-cephadm-18.2.2-954.g82316b04 2.3 MB/s | 138 kB 00:00 2024-03-23T09:38:32.317 INFO:teuthology.orchestra.run.smithi047.stdout:(30/156): ceph-mgr-dashboard-18.2.2-954.g82316b 11 MB/s | 1.7 MB 00:00 2024-03-23T09:38:32.725 INFO:teuthology.orchestra.run.smithi047.stdout:(31/156): ceph-test-18.2.2-954.g82316b04.el9.x8 14 MB/s | 40 MB 00:02 2024-03-23T09:38:32.751 INFO:teuthology.orchestra.run.smithi047.stdout:(32/156): ceph-mgr-modules-core-18.2.2-954.g823 567 kB/s | 245 kB 00:00 2024-03-23T09:38:32.767 INFO:teuthology.orchestra.run.smithi047.stdout:(33/156): ceph-mgr-rook-18.2.2-954.g82316b04.el 1.2 MB/s | 49 kB 00:00 2024-03-23T09:38:32.792 INFO:teuthology.orchestra.run.smithi047.stdout:(34/156): ceph-prometheus-alerts-18.2.2-954.g82 357 kB/s | 15 kB 00:00 2024-03-23T09:38:32.818 INFO:teuthology.orchestra.run.smithi047.stdout:(35/156): ceph-volume-18.2.2-954.g82316b04.el9. 5.2 MB/s | 264 kB 00:00 2024-03-23T09:38:32.843 INFO:teuthology.orchestra.run.smithi047.stdout:(36/156): cephadm-18.2.2-954.g82316b04.el9.noar 4.4 MB/s | 224 kB 00:00 2024-03-23T09:38:32.860 INFO:teuthology.orchestra.run.smithi047.stdout:(37/156): cephfs-top-18.2.2-954.g82316b04.el9.n 436 kB/s | 18 kB 00:00 2024-03-23T09:38:33.035 INFO:teuthology.orchestra.run.smithi047.stdout:(38/156): bison-3.7.4-5.el9.x86_64.rpm 5.1 MB/s | 920 kB 00:00 2024-03-23T09:38:33.060 INFO:teuthology.orchestra.run.smithi047.stdout:(39/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 662 kB 00:00 2024-03-23T09:38:33.086 INFO:teuthology.orchestra.run.smithi047.stdout:(40/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-03-23T09:38:33.120 INFO:teuthology.orchestra.run.smithi047.stdout:(41/156): flex-2.6.4-9.el9.x86_64.rpm 5.1 MB/s | 307 kB 00:00 2024-03-23T09:38:33.136 INFO:teuthology.orchestra.run.smithi047.stdout:(42/156): flexiblas-3.0.4-8.el9.x86_64.rpm 635 kB/s | 32 kB 00:00 2024-03-23T09:38:33.229 INFO:teuthology.orchestra.run.smithi047.stdout:(43/156): ceph-mgr-diskprediction-local-18.2.2- 7.1 MB/s | 7.4 MB 00:01 2024-03-23T09:38:33.296 INFO:teuthology.orchestra.run.smithi047.stdout:(44/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-03-23T09:38:33.312 INFO:teuthology.orchestra.run.smithi047.stdout:(45/156): flexiblas-openblas-openmp-3.0.4-8.el9 98 kB/s | 17 kB 00:00 2024-03-23T09:38:33.329 INFO:teuthology.orchestra.run.smithi047.stdout:(46/156): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 10 kB 00:00 2024-03-23T09:38:33.354 INFO:teuthology.orchestra.run.smithi047.stdout:(47/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-23T09:38:33.371 INFO:teuthology.orchestra.run.smithi047.stdout:(48/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-23T09:38:33.413 INFO:teuthology.orchestra.run.smithi047.stdout:(49/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 661 kB 00:00 2024-03-23T09:38:33.438 INFO:teuthology.orchestra.run.smithi047.stdout:(50/156): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-03-23T09:38:33.464 INFO:teuthology.orchestra.run.smithi047.stdout:(51/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 282 kB 00:00 2024-03-23T09:38:33.489 INFO:teuthology.orchestra.run.smithi047.stdout:(52/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 624 kB/s | 46 kB 00:00 2024-03-23T09:38:33.514 INFO:teuthology.orchestra.run.smithi047.stdout:(53/156): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-23T09:38:33.539 INFO:teuthology.orchestra.run.smithi047.stdout:(54/156): openblas-0.3.21-2.el9.x86_64.rpm 456 kB/s | 34 kB 00:00 2024-03-23T09:38:33.590 INFO:teuthology.orchestra.run.smithi047.stdout:(55/156): postgresql-13.11-1.el9.x86_64.rpm 20 MB/s | 1.5 MB 00:00 2024-03-23T09:38:33.673 INFO:teuthology.orchestra.run.smithi047.stdout:(56/156): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-03-23T09:38:33.709 INFO:teuthology.orchestra.run.smithi047.stdout:(57/156): postgresql-contrib-13.11-1.el9.x86_64 4.8 MB/s | 811 kB 00:00 2024-03-23T09:38:33.732 INFO:teuthology.orchestra.run.smithi047.stdout:(58/156): postgresql-private-libs-13.11-1.el9.x 962 kB/s | 134 kB 00:00 2024-03-23T09:38:33.832 INFO:teuthology.orchestra.run.smithi047.stdout:(59/156): postgresql-server-13.11-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-03-23T09:38:33.857 INFO:teuthology.orchestra.run.smithi047.stdout:(60/156): python3-jsonpatch-1.21-16.el9.noarch. 200 kB/s | 24 kB 00:00 2024-03-23T09:38:33.958 INFO:teuthology.orchestra.run.smithi047.stdout:(61/156): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2024-03-23T09:38:33.983 INFO:teuthology.orchestra.run.smithi047.stdout:(62/156): python3-jsonpointer-2.0-4.el9.noarch. 120 kB/s | 18 kB 00:00 2024-03-23T09:38:34.008 INFO:teuthology.orchestra.run.smithi047.stdout:(63/156): python3-mako-1.1.4-6.el9.noarch.rpm 977 kB/s | 144 kB 00:00 2024-03-23T09:38:34.033 INFO:teuthology.orchestra.run.smithi047.stdout:(64/156): python3-markupsafe-1.1.1-12.el9.x86_6 422 kB/s | 32 kB 00:00 2024-03-23T09:38:34.058 INFO:teuthology.orchestra.run.smithi047.stdout:(65/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 217 kB 00:00 2024-03-23T09:38:34.134 INFO:teuthology.orchestra.run.smithi047.stdout:(66/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-23T09:38:34.159 INFO:teuthology.orchestra.run.smithi047.stdout:(67/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-03-23T09:38:34.184 INFO:teuthology.orchestra.run.smithi047.stdout:(68/156): python3-packaging-20.9-5.el9.noarch.r 560 kB/s | 69 kB 00:00 2024-03-23T09:38:34.201 INFO:teuthology.orchestra.run.smithi047.stdout:(69/156): python3-prettytable-0.7.2-27.el9.noar 620 kB/s | 41 kB 00:00 2024-03-23T09:38:34.226 INFO:teuthology.orchestra.run.smithi047.stdout:(70/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 132 kB 00:00 2024-03-23T09:38:34.251 INFO:teuthology.orchestra.run.smithi047.stdout:(71/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-03-23T09:38:34.268 INFO:teuthology.orchestra.run.smithi047.stdout:(72/156): python3-pytz-2021.1-5.el9.noarch.rpm 714 kB/s | 47 kB 00:00 2024-03-23T09:38:34.285 INFO:teuthology.orchestra.run.smithi047.stdout:(73/156): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-03-23T09:38:34.377 INFO:teuthology.orchestra.run.smithi047.stdout:(74/156): python3-toml-0.10.2-6.el9.noarch.rpm 360 kB/s | 39 kB 00:00 2024-03-23T09:38:34.686 INFO:teuthology.orchestra.run.smithi047.stdout:(75/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-03-23T09:38:34.711 INFO:teuthology.orchestra.run.smithi047.stdout:(76/156): socat-1.7.4.1-5.el9.x86_64.rpm 708 kB/s | 300 kB 00:00 2024-03-23T09:38:34.729 INFO:teuthology.orchestra.run.smithi047.stdout:(77/156): uuid-1.6.2-55.el9.x86_64.rpm 162 kB/s | 56 kB 00:00 2024-03-23T09:38:34.745 INFO:teuthology.orchestra.run.smithi047.stdout:(78/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2024-03-23T09:38:34.762 INFO:teuthology.orchestra.run.smithi047.stdout:(79/156): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-23T09:38:34.787 INFO:teuthology.orchestra.run.smithi047.stdout:(80/156): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-23T09:38:34.812 INFO:teuthology.orchestra.run.smithi047.stdout:(81/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-23T09:38:34.838 INFO:teuthology.orchestra.run.smithi047.stdout:(82/156): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-03-23T09:38:34.871 INFO:teuthology.orchestra.run.smithi047.stdout:(83/156): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-23T09:38:34.897 INFO:teuthology.orchestra.run.smithi047.stdout:(84/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2024-03-23T09:38:34.922 INFO:teuthology.orchestra.run.smithi047.stdout:(85/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-03-23T09:38:35.764 INFO:teuthology.orchestra.run.smithi047.stdout:(86/156): jq-1.6-16.el9.x86_64.rpm 216 kB/s | 187 kB 00:00 2024-03-23T09:38:35.814 INFO:teuthology.orchestra.run.smithi047.stdout:(87/156): libquadmath-11.4.1-3.el9.x86_64.rpm 219 kB/s | 195 kB 00:00 2024-03-23T09:38:36.081 INFO:teuthology.orchestra.run.smithi047.stdout:(88/156): elfutils-libelf-devel-0.190-2.el9.x86 81 kB/s | 21 kB 00:00 2024-03-23T09:38:36.181 INFO:teuthology.orchestra.run.smithi047.stdout:(89/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-03-23T09:38:36.506 INFO:teuthology.orchestra.run.smithi047.stdout:(90/156): libgfortran-11.4.1-3.el9.x86_64.rpm 499 kB/s | 803 kB 00:01 2024-03-23T09:38:36.548 INFO:teuthology.orchestra.run.smithi047.stdout:(91/156): python3-requests-2.25.1-8.el9.noarch. 159 kB/s | 125 kB 00:00 2024-03-23T09:38:36.999 INFO:teuthology.orchestra.run.smithi047.stdout:(92/156): openssl-devel-3.0.7-27.el9.x86_64.rpm 5.1 MB/s | 4.1 MB 00:00 2024-03-23T09:38:37.040 INFO:teuthology.orchestra.run.smithi047.stdout:(93/156): python3-devel-3.9.17-2.el9.x86_64.rpm 460 kB/s | 245 kB 00:00 2024-03-23T09:38:37.149 INFO:teuthology.orchestra.run.smithi047.stdout:(94/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 624 kB/s | 94 kB 00:00 2024-03-23T09:38:37.258 INFO:teuthology.orchestra.run.smithi047.stdout:(95/156): python3-jinja2-2.11.3-5.el9.noarch.rp 350 kB/s | 248 kB 00:00 2024-03-23T09:38:37.441 INFO:teuthology.orchestra.run.smithi047.stdout:(96/156): fmt-8.1.1-5.el9.x86_64.rpm 276 kB/s | 111 kB 00:00 2024-03-23T09:38:37.516 INFO:teuthology.orchestra.run.smithi047.stdout:(97/156): libarrow-doc-9.0.0-10.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-03-23T09:38:37.592 INFO:teuthology.orchestra.run.smithi047.stdout:(98/156): liboath-2.6.7-2.el9.x86_64.rpm 649 kB/s | 49 kB 00:00 2024-03-23T09:38:37.617 INFO:teuthology.orchestra.run.smithi047.stdout:(99/156): gperftools-libs-2.9.1-2.el9.x86_64.rp 661 kB/s | 309 kB 00:00 2024-03-23T09:38:37.700 INFO:teuthology.orchestra.run.smithi047.stdout:(100/156): libunwind-1.6.2-1.el9.x86_64.rpm 622 kB/s | 67 kB 00:00 2024-03-23T09:38:37.776 INFO:teuthology.orchestra.run.smithi047.stdout:(101/156): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 838 kB 00:00 2024-03-23T09:38:37.868 INFO:teuthology.orchestra.run.smithi047.stdout:(102/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 474 kB/s | 43 kB 00:00 2024-03-23T09:38:37.901 INFO:teuthology.orchestra.run.smithi047.stdout:(103/156): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-03-23T09:38:37.960 INFO:teuthology.orchestra.run.smithi047.stdout:(104/156): libarrow-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-03-23T09:38:37.985 INFO:teuthology.orchestra.run.smithi047.stdout:(105/156): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-03-23T09:38:38.010 INFO:teuthology.orchestra.run.smithi047.stdout:(106/156): python3-cachetools-4.2.4-1.el9.noarc 226 kB/s | 32 kB 00:00 2024-03-23T09:38:38.036 INFO:teuthology.orchestra.run.smithi047.stdout:(107/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-03-23T09:38:38.069 INFO:teuthology.orchestra.run.smithi047.stdout:(108/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-23T09:38:38.095 INFO:teuthology.orchestra.run.smithi047.stdout:(109/156): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-03-23T09:38:38.120 INFO:teuthology.orchestra.run.smithi047.stdout:(110/156): python3-jaraco-8.2.1-3.el9.noarch.rp 127 kB/s | 11 kB 00:00 2024-03-23T09:38:38.145 INFO:teuthology.orchestra.run.smithi047.stdout:(111/156): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-03-23T09:38:38.170 INFO:teuthology.orchestra.run.smithi047.stdout:(112/156): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-03-23T09:38:38.195 INFO:teuthology.orchestra.run.smithi047.stdout:(113/156): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-03-23T09:38:38.221 INFO:teuthology.orchestra.run.smithi047.stdout:(114/156): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-03-23T09:38:38.263 INFO:teuthology.orchestra.run.smithi047.stdout:(115/156): python3-logutils-0.3.5-21.el9.noarch 693 kB/s | 46 kB 00:00 2024-03-23T09:38:38.288 INFO:teuthology.orchestra.run.smithi047.stdout:(116/156): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-03-23T09:38:38.322 INFO:teuthology.orchestra.run.smithi047.stdout:(117/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-03-23T09:38:38.347 INFO:teuthology.orchestra.run.smithi047.stdout:(118/156): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-03-23T09:38:38.372 INFO:teuthology.orchestra.run.smithi047.stdout:(119/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-23T09:38:38.398 INFO:teuthology.orchestra.run.smithi047.stdout:(120/156): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-03-23T09:38:38.423 INFO:teuthology.orchestra.run.smithi047.stdout:(121/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-23T09:38:38.448 INFO:teuthology.orchestra.run.smithi047.stdout:(122/156): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-03-23T09:38:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:(123/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-23T09:38:38.498 INFO:teuthology.orchestra.run.smithi047.stdout:(124/156): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-03-23T09:38:38.515 INFO:teuthology.orchestra.run.smithi047.stdout:(125/156): python3-tempora-5.0.0-2.el9.noarch.r 534 kB/s | 36 kB 00:00 2024-03-23T09:38:38.541 INFO:teuthology.orchestra.run.smithi047.stdout:(126/156): python3-typing-extensions-4.4.0-2.el 759 kB/s | 51 kB 00:00 2024-03-23T09:38:38.574 INFO:teuthology.orchestra.run.smithi047.stdout:(127/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-23T09:38:38.599 INFO:teuthology.orchestra.run.smithi047.stdout:(128/156): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-23T09:38:38.633 INFO:teuthology.orchestra.run.smithi047.stdout:(129/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-03-23T09:38:38.650 INFO:teuthology.orchestra.run.smithi047.stdout:(130/156): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-03-23T09:38:38.675 INFO:teuthology.orchestra.run.smithi047.stdout:(131/156): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-23T09:38:38.742 INFO:teuthology.orchestra.run.smithi047.stdout:(132/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-03-23T09:38:38.784 INFO:teuthology.orchestra.run.smithi047.stdout:(133/156): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-23T09:38:39.029 INFO:teuthology.orchestra.run.smithi047.stdout:(134/156): librbd1-18.2.2-954.g82316b04.el9.x86 8.6 MB/s | 3.0 MB 00:00 2024-03-23T09:38:39.110 INFO:teuthology.orchestra.run.smithi047.stdout:(135/156): librados2-18.2.2-954.g82316b04.el9.x 7.1 MB/s | 3.3 MB 00:00 2024-03-23T09:38:39.719 INFO:teuthology.orchestra.run.smithi047.stdout:(136/156): NetworkManager-team-1.47.2-1.el9.x86 69 kB/s | 42 kB 00:00 2024-03-23T09:38:40.677 INFO:teuthology.orchestra.run.smithi047.stdout:(137/156): NetworkManager-tui-1.47.2-1.el9.x86_ 259 kB/s | 248 kB 00:00 2024-03-23T09:38:41.344 INFO:teuthology.orchestra.run.smithi047.stdout:(138/156): acl-2.3.1-4.el9.x86_64.rpm 107 kB/s | 71 kB 00:00 2024-03-23T09:38:41.935 INFO:teuthology.orchestra.run.smithi047.stdout:(139/156): NetworkManager-libnm-1.47.2-1.el9.x8 651 kB/s | 1.8 MB 00:02 2024-03-23T09:38:41.961 INFO:teuthology.orchestra.run.smithi047.stdout:(140/156): elfutils-debuginfod-client-0.190-2.e 59 kB/s | 36 kB 00:00 2024-03-23T09:38:42.270 INFO:teuthology.orchestra.run.smithi047.stdout:(141/156): NetworkManager-1.47.2-1.el9.x86_64.r 677 kB/s | 2.3 MB 00:03 2024-03-23T09:38:42.828 INFO:teuthology.orchestra.run.smithi047.stdout:(142/156): elfutils-libelf-0.190-2.el9.x86_64.r 216 kB/s | 192 kB 00:00 2024-03-23T09:38:42.870 INFO:teuthology.orchestra.run.smithi047.stdout:(143/156): libacl-2.3.1-4.el9.x86_64.rpm 38 kB/s | 23 kB 00:00 2024-03-23T09:38:42.904 INFO:teuthology.orchestra.run.smithi047.stdout:(144/156): elfutils-libs-0.190-2.el9.x86_64.rpm 271 kB/s | 256 kB 00:00 2024-03-23T09:38:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:(145/156): libblkid-2.37.4-18.el9.x86_64.rpm 143 kB/s | 108 kB 00:00 2024-03-23T09:38:43.662 INFO:teuthology.orchestra.run.smithi047.stdout:(146/156): libfdisk-2.37.4-18.el9.x86_64.rpm 195 kB/s | 155 kB 00:00 2024-03-23T09:38:43.688 INFO:teuthology.orchestra.run.smithi047.stdout:(147/156): libmount-2.37.4-18.el9.x86_64.rpm 173 kB/s | 136 kB 00:00 2024-03-23T09:38:44.247 INFO:teuthology.orchestra.run.smithi047.stdout:(148/156): libuuid-2.37.4-18.el9.x86_64.rpm 49 kB/s | 29 kB 00:00 2024-03-23T09:38:44.297 INFO:teuthology.orchestra.run.smithi047.stdout:(149/156): libsmartcols-2.37.4-18.el9.x86_64.rp 90 kB/s | 64 kB 00:00 2024-03-23T09:38:45.838 INFO:teuthology.orchestra.run.smithi047.stdout:(150/156): openssl-3.0.7-27.el9.x86_64.rpm 576 kB/s | 1.2 MB 00:02 2024-03-23T09:38:46.989 INFO:teuthology.orchestra.run.smithi047.stdout:(151/156): util-linux-core-2.37.4-18.el9.x86_64 405 kB/s | 465 kB 00:01 2024-03-23T09:38:47.472 INFO:teuthology.orchestra.run.smithi047.stdout:(152/156): openssl-libs-3.0.7-27.el9.x86_64.rpm 684 kB/s | 2.2 MB 00:03 2024-03-23T09:38:47.580 INFO:teuthology.orchestra.run.smithi047.stdout:(153/156): util-linux-user-2.37.4-18.el9.x86_64 53 kB/s | 31 kB 00:00 2024-03-23T09:38:47.781 INFO:teuthology.orchestra.run.smithi047.stdout:(154/156): util-linux-2.37.4-18.el9.x86_64.rpm 671 kB/s | 2.3 MB 00:03 2024-03-23T09:38:47.923 INFO:teuthology.orchestra.run.smithi047.stdout:(155/156): libacl-devel-2.3.1-4.el9.x86_64.rpm 217 kB/s | 74 kB 00:00 2024-03-23T09:38:49.381 INFO:teuthology.orchestra.run.smithi047.stdout:(156/156): xfsprogs-6.3.0-1.el9.x86_64.rpm 576 kB/s | 1.1 MB 00:01 2024-03-23T09:38:49.386 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-03-23T09:38:49.386 INFO:teuthology.orchestra.run.smithi047.stdout:Total 9.4 MB/s | 204 MB 00:21 2024-03-23T09:38:49.815 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-03-23T09:38:50.104 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-03-23T09:38:50.104 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-03-23T09:38:50.316 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-03-23T09:38:52.029 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-03-23T09:38:52.030 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-03-23T09:38:55.145 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-03-23T09:38:57.568 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T09:38:57.568 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T09:38:57.581 INFO:teuthology.orchestra.run.smithi175.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-23T09:38:57.582 INFO:teuthology.orchestra.run.smithi175.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-23T09:38:57.583 INFO:teuthology.orchestra.run.smithi175.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-23T09:38:57.584 INFO:teuthology.orchestra.run.smithi175.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-23T09:38:57.586 INFO:teuthology.orchestra.run.smithi175.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-23T09:38:57.589 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-23T09:38:57.590 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-23T09:38:57.666 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-03-23T09:38:57.684 INFO:teuthology.orchestra.run.smithi175.stdout:=========================================================================================== 2024-03-23T09:38:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-03-23T09:38:57.684 INFO:teuthology.orchestra.run.smithi175.stdout:=========================================================================================== 2024-03-23T09:38:57.685 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-03-23T09:38:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-23T09:38:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:18.2.2-954.g82316b04.el9 ceph 6.5 k 2024-03-23T09:38:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:18.2.2-954.g82316b04.el9 ceph 5.1 M 2024-03-23T09:38:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:18.2.2-954.g82316b04.el9 ceph 843 k 2024-03-23T09:38:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-954.g82316b04.el9 ceph 143 k 2024-03-23T09:38:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:18.2.2-954.g82316b04.el9 ceph 1.4 M 2024-03-23T09:38:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 138 k 2024-03-23T09:38:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 1.7 M 2024-03-23T09:38:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 7.4 M 2024-03-23T09:38:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 49 k 2024-03-23T09:38:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:18.2.2-954.g82316b04.el9 ceph 7.7 M 2024-03-23T09:38:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:18.2.2-954.g82316b04.el9 ceph 40 M 2024-03-23T09:38:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 264 k 2024-03-23T09:38:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 224 k 2024-03-23T09:38:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror x86_64 2:18.2.2-954.g82316b04.el9 ceph 221 k 2024-03-23T09:38:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 18 k 2024-03-23T09:38:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-23T09:38:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-23T09:38:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-23T09:38:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:18.2.2-954.g82316b04.el9 ceph 31 k 2024-03-23T09:38:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:18.2.2-954.g82316b04.el9 ceph 704 k 2024-03-23T09:38:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:18.2.2-954.g82316b04.el9 ceph 127 k 2024-03-23T09:38:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-23T09:38:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-23T09:38:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-23T09:38:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-23T09:38:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-23T09:38:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-23T09:38:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:18.2.2-954.g82316b04.el9 ceph 161 k 2024-03-23T09:38:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:18.2.2-954.g82316b04.el9 ceph 320 k 2024-03-23T09:38:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:18.2.2-954.g82316b04.el9 ceph 297 k 2024-03-23T09:38:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:18.2.2-954.g82316b04.el9 ceph 99 k 2024-03-23T09:38:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:18.2.2-954.g82316b04.el9 ceph 87 k 2024-03-23T09:38:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:18.2.2-954.g82316b04.el9 ceph 3.0 M 2024-03-23T09:38:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:18.2.2-954.g82316b04.el9 ceph 172 k 2024-03-23T09:38:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-23T09:38:57.688 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-03-23T09:38:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-23T09:38:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-23T09:38:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-23T09:38:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-23T09:38:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-23T09:38:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-23T09:38:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-23T09:38:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-23T09:38:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-23T09:38:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-23T09:38:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:18.2.2-954.g82316b04.el9 ceph 3.3 M 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:18.2.2-954.g82316b04.el9 ceph 3.0 M 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-23T09:38:57.689 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-03-23T09:38:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-23T09:38:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-23T09:38:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:18.2.2-954.g82316b04.el9 ceph 18 M 2024-03-23T09:38:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 23 k 2024-03-23T09:38:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:18.2.2-954.g82316b04.el9 ceph 2.1 M 2024-03-23T09:38:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 245 k 2024-03-23T09:38:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:18.2.2-954.g82316b04.el9 ceph 4.7 M 2024-03-23T09:38:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:18.2.2-954.g82316b04.el9 ceph 16 M 2024-03-23T09:38:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 15 k 2024-03-23T09:38:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:18.2.2-954.g82316b04.el9 ceph 25 k 2024-03-23T09:38:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-23T09:38:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-23T09:38:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:18.2.2-954.g82316b04.el9 ceph 166 k 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:18.2.2-954.g82316b04.el9 ceph 474 k 2024-03-23T09:38:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-23T09:38:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:18.2.2-954.g82316b04.el9 ceph 4.5 M 2024-03-23T09:38:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-23T09:38:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-23T09:38:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-23T09:38:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-23T09:38:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-23T09:38:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-23T09:38:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-23T09:38:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-23T09:38:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-23T09:38:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-23T09:38:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-23T09:38:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:18.2.2-954.g82316b04.el9 ceph 45 k 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:18.2.2-954.g82316b04.el9 ceph 127 k 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-23T09:38:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-23T09:38:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-23T09:38:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-23T09:38:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-23T09:38:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-23T09:38:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-23T09:38:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-23T09:38:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-23T09:38:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-23T09:38:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-23T09:38:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-23T09:38:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-23T09:38:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-23T09:38:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-23T09:38:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-23T09:38:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-23T09:38:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-23T09:38:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:38:57.697 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-03-23T09:38:57.697 INFO:teuthology.orchestra.run.smithi175.stdout:=========================================================================================== 2024-03-23T09:38:57.697 INFO:teuthology.orchestra.run.smithi175.stdout:Install 132 Packages 2024-03-23T09:38:57.697 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 24 Packages 2024-03-23T09:38:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:38:57.697 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 204 M 2024-03-23T09:38:57.698 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-03-23T09:38:58.263 INFO:teuthology.orchestra.run.smithi175.stdout:(1/156): ceph-18.2.2-954.g82316b04.el9.x86_64.r 32 kB/s | 6.5 kB 00:00 2024-03-23T09:38:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-03-23T09:38:58.797 INFO:teuthology.orchestra.run.smithi175.stdout:(2/156): ceph-fuse-18.2.2-954.g82316b04.el9.x86 1.5 MB/s | 843 kB 00:00 2024-03-23T09:38:58.878 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/180 2024-03-23T09:38:58.897 INFO:teuthology.orchestra.run.smithi175.stdout:(3/156): ceph-immutable-object-cache-18.2.2-954 1.4 MB/s | 143 kB 00:00 2024-03-23T09:38:58.920 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/180 2024-03-23T09:38:58.958 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-23T09:38:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-23T09:38:59.012 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/180 2024-03-23T09:38:59.064 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/180 2024-03-23T09:38:59.104 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/180 2024-03-23T09:38:59.129 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/180 2024-03-23T09:38:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/180 2024-03-23T09:38:59.231 INFO:teuthology.orchestra.run.smithi175.stdout:(4/156): ceph-common-18.2.2-954.g82316b04.el9.x 16 MB/s | 18 MB 00:01 2024-03-23T09:38:59.339 INFO:teuthology.orchestra.run.smithi175.stdout:(5/156): ceph-mgr-18.2.2-954.g82316b04.el9.x86_ 13 MB/s | 1.4 MB 00:00 2024-03-23T09:38:59.514 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/180 2024-03-23T09:38:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:18.2.2-954.g82316b04.el9.x86_64 10/180 2024-03-23T09:38:59.648 INFO:teuthology.orchestra.run.smithi175.stdout:(6/156): ceph-mon-18.2.2-954.g82316b04.el9.x86_ 15 MB/s | 4.7 MB 00:00 2024-03-23T09:38:59.676 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:18.2.2-954.g82316b04.el9.x86_64 10/180 2024-03-23T09:38:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 11/180 2024-03-23T09:38:59.715 INFO:teuthology.orchestra.run.smithi175.stdout:(7/156): ceph-base-18.2.2-954.g82316b04.el9.x86 3.1 MB/s | 5.1 MB 00:01 2024-03-23T09:38:59.765 INFO:teuthology.orchestra.run.smithi175.stdout:(8/156): ceph-mds-18.2.2-954.g82316b04.el9.x86_ 2.4 MB/s | 2.1 MB 00:00 2024-03-23T09:38:59.798 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 11/180 2024-03-23T09:38:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:18.2.2-954.g82316b04.el9.x86_64 12/180 2024-03-23T09:38:59.857 INFO:teuthology.orchestra.run.smithi175.stdout:(9/156): ceph-selinux-18.2.2-954.g82316b04.el9. 274 kB/s | 25 kB 00:00 2024-03-23T09:38:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/180 2024-03-23T09:39:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/180 2024-03-23T09:39:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/180 2024-03-23T09:39:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 16/180 2024-03-23T09:39:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 16/180 2024-03-23T09:39:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_ 17/180 2024-03-23T09:39:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_ 17/180 2024-03-23T09:39:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/180 2024-03-23T09:39:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/180 2024-03-23T09:39:00.591 INFO:teuthology.orchestra.run.smithi175.stdout:(10/156): ceph-osd-18.2.2-954.g82316b04.el9.x86 18 MB/s | 16 MB 00:00 2024-03-23T09:39:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/180 2024-03-23T09:39:00.649 INFO:teuthology.orchestra.run.smithi175.stdout:(11/156): cephfs-mirror-18.2.2-954.g82316b04.el 3.7 MB/s | 221 kB 00:00 2024-03-23T09:39:00.699 INFO:teuthology.orchestra.run.smithi175.stdout:(12/156): libcephfs-devel-18.2.2-954.g82316b04. 621 kB/s | 31 kB 00:00 2024-03-23T09:39:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 21/180 2024-03-23T09:39:00.791 INFO:teuthology.orchestra.run.smithi175.stdout:(13/156): ceph-radosgw-18.2.2-954.g82316b04.el9 7.2 MB/s | 7.7 MB 00:01 2024-03-23T09:39:00.820 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 21/180 2024-03-23T09:39:00.825 INFO:teuthology.orchestra.run.smithi175.stdout:(14/156): libcephfs2-18.2.2-954.g82316b04.el9.x 5.5 MB/s | 704 kB 00:00 2024-03-23T09:39:00.850 INFO:teuthology.orchestra.run.smithi175.stdout:(15/156): libcephsqlite-18.2.2-954.g82316b04.el 2.8 MB/s | 166 kB 00:00 2024-03-23T09:39:00.875 INFO:teuthology.orchestra.run.smithi175.stdout:(16/156): librados-devel-18.2.2-954.g82316b04.e 2.5 MB/s | 127 kB 00:00 2024-03-23T09:39:00.912 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-03-23T09:39:00.917 INFO:teuthology.orchestra.run.smithi175.stdout:(17/156): libradosstriper1-18.2.2-954.g82316b04 6.9 MB/s | 474 kB 00:00 2024-03-23T09:39:00.972 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-03-23T09:39:01.017 INFO:teuthology.orchestra.run.smithi175.stdout:(18/156): python3-ceph-argparse-18.2.2-954.g823 449 kB/s | 45 kB 00:00 2024-03-23T09:39:01.126 INFO:teuthology.orchestra.run.smithi175.stdout:(19/156): librgw2-18.2.2-954.g82316b04.el9.x86_ 18 MB/s | 4.5 MB 00:00 2024-03-23T09:39:01.152 INFO:teuthology.orchestra.run.smithi175.stdout:(20/156): python3-ceph-common-18.2.2-954.g82316 948 kB/s | 127 kB 00:00 2024-03-23T09:39:01.177 INFO:teuthology.orchestra.run.smithi175.stdout:(21/156): python3-cephfs-18.2.2-954.g82316b04.e 3.1 MB/s | 161 kB 00:00 2024-03-23T09:39:01.219 INFO:teuthology.orchestra.run.smithi175.stdout:(22/156): python3-rados-18.2.2-954.g82316b04.el 4.7 MB/s | 320 kB 00:00 2024-03-23T09:39:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-03-23T09:39:01.261 INFO:teuthology.orchestra.run.smithi175.stdout:(23/156): python3-rbd-18.2.2-954.g82316b04.el9. 3.5 MB/s | 297 kB 00:00 2024-03-23T09:39:01.284 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/180 2024-03-23T09:39:01.286 INFO:teuthology.orchestra.run.smithi175.stdout:(24/156): python3-rgw-18.2.2-954.g82316b04.el9. 1.4 MB/s | 99 kB 00:00 2024-03-23T09:39:01.312 INFO:teuthology.orchestra.run.smithi175.stdout:(25/156): rbd-fuse-18.2.2-954.g82316b04.el9.x86 1.7 MB/s | 87 kB 00:00 2024-03-23T09:39:01.313 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/180 2024-03-23T09:39:01.358 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/180 2024-03-23T09:39:01.362 INFO:teuthology.orchestra.run.smithi175.stdout:(26/156): rbd-nbd-18.2.2-954.g82316b04.el9.x86_ 3.4 MB/s | 172 kB 00:00 2024-03-23T09:39:01.412 INFO:teuthology.orchestra.run.smithi175.stdout:(27/156): ceph-grafana-dashboards-18.2.2-954.g8 469 kB/s | 23 kB 00:00 2024-03-23T09:39:01.434 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/180 2024-03-23T09:39:01.479 INFO:teuthology.orchestra.run.smithi175.stdout:(28/156): ceph-mgr-cephadm-18.2.2-954.g82316b04 2.0 MB/s | 138 kB 00:00 2024-03-23T09:39:01.486 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/180 2024-03-23T09:39:01.523 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/180 2024-03-23T09:39:01.537 INFO:teuthology.orchestra.run.smithi175.stdout:(29/156): rbd-mirror-18.2.2-954.g82316b04.el9.x 12 MB/s | 3.0 MB 00:00 2024-03-23T09:39:01.591 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/180 2024-03-23T09:39:01.596 INFO:teuthology.orchestra.run.smithi175.stdout:(30/156): ceph-mgr-dashboard-18.2.2-954.g82316b 15 MB/s | 1.7 MB 00:00 2024-03-23T09:39:01.631 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/180 2024-03-23T09:39:01.658 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/180 2024-03-23T09:39:01.671 INFO:teuthology.orchestra.run.smithi175.stdout:(31/156): ceph-mgr-modules-core-18.2.2-954.g823 3.2 MB/s | 245 kB 00:00 2024-03-23T09:39:01.721 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 32/180 2024-03-23T09:39:01.722 INFO:teuthology.orchestra.run.smithi175.stdout:(32/156): ceph-mgr-rook-18.2.2-954.g82316b04.el 986 kB/s | 49 kB 00:00 2024-03-23T09:39:01.754 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:18.2.2-954.g82316b04.el9.x 33/180 2024-03-23T09:39:01.772 INFO:teuthology.orchestra.run.smithi175.stdout:(33/156): ceph-prometheus-alerts-18.2.2-954.g82 299 kB/s | 15 kB 00:00 2024-03-23T09:39:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:18.2.2-954.g82316b04.el9 34/180 2024-03-23T09:39:01.839 INFO:teuthology.orchestra.run.smithi175.stdout:(34/156): ceph-volume-18.2.2-954.g82316b04.el9. 3.9 MB/s | 264 kB 00:00 2024-03-23T09:39:01.875 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:18.2.2-954.g82316b04.el9.x86_64 35/180 2024-03-23T09:39:01.906 INFO:teuthology.orchestra.run.smithi175.stdout:(35/156): cephadm-18.2.2-954.g82316b04.el9.noar 3.3 MB/s | 224 kB 00:00 2024-03-23T09:39:01.940 INFO:teuthology.orchestra.run.smithi175.stdout:(36/156): cephfs-top-18.2.2-954.g82316b04.el9.n 549 kB/s | 18 kB 00:00 2024-03-23T09:39:02.012 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/180 2024-03-23T09:39:02.023 INFO:teuthology.orchestra.run.smithi175.stdout:(37/156): ceph-mgr-diskprediction-local-18.2.2- 15 MB/s | 7.4 MB 00:00 2024-03-23T09:39:02.068 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/180 2024-03-23T09:39:02.182 INFO:teuthology.orchestra.run.smithi175.stdout:(38/156): bison-3.7.4-5.el9.x86_64.rpm 5.7 MB/s | 920 kB 00:00 2024-03-23T09:39:02.207 INFO:teuthology.orchestra.run.smithi175.stdout:(39/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-03-23T09:39:02.232 INFO:teuthology.orchestra.run.smithi175.stdout:(40/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-03-23T09:39:02.258 INFO:teuthology.orchestra.run.smithi175.stdout:(41/156): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-23T09:39:02.283 INFO:teuthology.orchestra.run.smithi175.stdout:(42/156): flexiblas-3.0.4-8.el9.x86_64.rpm 636 kB/s | 32 kB 00:00 2024-03-23T09:39:02.367 INFO:teuthology.orchestra.run.smithi175.stdout:(43/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-03-23T09:39:02.392 INFO:teuthology.orchestra.run.smithi175.stdout:(44/156): flexiblas-openblas-openmp-3.0.4-8.el9 160 kB/s | 17 kB 00:00 2024-03-23T09:39:02.417 INFO:teuthology.orchestra.run.smithi175.stdout:(45/156): libaio-devel-0.3.111-13.el9.x86_64.rp 201 kB/s | 10 kB 00:00 2024-03-23T09:39:02.442 INFO:teuthology.orchestra.run.smithi175.stdout:(46/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-23T09:39:02.468 INFO:teuthology.orchestra.run.smithi175.stdout:(47/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 882 kB/s | 44 kB 00:00 2024-03-23T09:39:02.552 INFO:teuthology.orchestra.run.smithi175.stdout:(48/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 661 kB 00:00 2024-03-23T09:39:02.577 INFO:teuthology.orchestra.run.smithi175.stdout:(49/156): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 578 kB 00:00 2024-03-23T09:39:02.602 INFO:teuthology.orchestra.run.smithi175.stdout:(50/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 936 kB/s | 46 kB 00:00 2024-03-23T09:39:02.621 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/180 2024-03-23T09:39:02.627 INFO:teuthology.orchestra.run.smithi175.stdout:(51/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-23T09:39:02.653 INFO:teuthology.orchestra.run.smithi175.stdout:(52/156): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-23T09:39:02.670 INFO:teuthology.orchestra.run.smithi175.stdout:(53/156): openblas-0.3.21-2.el9.x86_64.rpm 821 kB/s | 34 kB 00:00 2024-03-23T09:39:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/180 2024-03-23T09:39:02.770 INFO:teuthology.orchestra.run.smithi175.stdout:(54/156): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-03-23T09:39:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/180 2024-03-23T09:39:02.821 INFO:teuthology.orchestra.run.smithi175.stdout:(55/156): postgresql-13.11-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-03-23T09:39:02.823 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-23T09:39:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-23T09:39:02.854 INFO:teuthology.orchestra.run.smithi175.stdout:(56/156): postgresql-contrib-13.11-1.el9.x86_64 9.8 MB/s | 811 kB 00:00 2024-03-23T09:39:02.879 INFO:teuthology.orchestra.run.smithi175.stdout:(57/156): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-23T09:39:02.883 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/180 2024-03-23T09:39:02.925 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:18.2.2-954.g82316b04.el9.x86_64 43/180 2024-03-23T09:39:02.956 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/180 2024-03-23T09:39:02.988 INFO:teuthology.orchestra.run.smithi175.stdout:(58/156): postgresql-server-13.11-1.el9.x86_64. 43 MB/s | 5.7 MB 00:00 2024-03-23T09:39:03.080 INFO:teuthology.orchestra.run.smithi175.stdout:(59/156): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 5.8 MB 00:00 2024-03-23T09:39:03.097 INFO:teuthology.orchestra.run.smithi175.stdout:(60/156): python3-jsonpatch-1.21-16.el9.noarch. 230 kB/s | 24 kB 00:00 2024-03-23T09:39:03.114 INFO:teuthology.orchestra.run.smithi175.stdout:(61/156): python3-jsonpointer-2.0-4.el9.noarch. 541 kB/s | 18 kB 00:00 2024-03-23T09:39:03.139 INFO:teuthology.orchestra.run.smithi175.stdout:(62/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-23T09:39:03.156 INFO:teuthology.orchestra.run.smithi175.stdout:(63/156): python3-markupsafe-1.1.1-12.el9.x86_6 755 kB/s | 32 kB 00:00 2024-03-23T09:39:03.250 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 45/180 2024-03-23T09:39:03.265 INFO:teuthology.orchestra.run.smithi175.stdout:(64/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-23T09:39:03.290 INFO:teuthology.orchestra.run.smithi175.stdout:(65/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-23T09:39:03.315 INFO:teuthology.orchestra.run.smithi175.stdout:(66/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-23T09:39:03.341 INFO:teuthology.orchestra.run.smithi175.stdout:(67/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-23T09:39:03.366 INFO:teuthology.orchestra.run.smithi175.stdout:(68/156): python3-prettytable-0.7.2-27.el9.noar 819 kB/s | 41 kB 00:00 2024-03-23T09:39:03.391 INFO:teuthology.orchestra.run.smithi175.stdout:(69/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-23T09:39:03.415 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 46/180 2024-03-23T09:39:03.416 INFO:teuthology.orchestra.run.smithi175.stdout:(70/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-23T09:39:03.433 INFO:teuthology.orchestra.run.smithi175.stdout:(71/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-23T09:39:03.459 INFO:teuthology.orchestra.run.smithi175.stdout:(72/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2024-03-23T09:39:03.490 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 47/180 2024-03-23T09:39:03.534 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 48/180 2024-03-23T09:39:03.601 INFO:teuthology.orchestra.run.smithi175.stdout:(73/156): python3-toml-0.10.2-6.el9.noarch.rpm 276 kB/s | 39 kB 00:00 2024-03-23T09:39:03.684 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/180 2024-03-23T09:39:03.726 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 50/180 2024-03-23T09:39:03.760 INFO:teuthology.orchestra.run.smithi175.stdout:(74/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 48 MB/s | 16 MB 00:00 2024-03-23T09:39:03.785 INFO:teuthology.orchestra.run.smithi175.stdout:(75/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 300 kB 00:00 2024-03-23T09:39:03.818 INFO:teuthology.orchestra.run.smithi175.stdout:(76/156): uuid-1.6.2-55.el9.x86_64.rpm 969 kB/s | 56 kB 00:00 2024-03-23T09:39:03.838 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : re2-1:20211101-3.el9.x86_64 51/180 2024-03-23T09:39:03.844 INFO:teuthology.orchestra.run.smithi175.stdout:(77/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2024-03-23T09:39:03.869 INFO:teuthology.orchestra.run.smithi175.stdout:(78/156): zlib-devel-1.2.11-40.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-03-23T09:39:03.871 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 52/180 2024-03-23T09:39:03.894 INFO:teuthology.orchestra.run.smithi175.stdout:(79/156): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-23T09:39:03.919 INFO:teuthology.orchestra.run.smithi175.stdout:(80/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-23T09:39:03.928 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/180 2024-03-23T09:39:03.953 INFO:teuthology.orchestra.run.smithi175.stdout:(81/156): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-23T09:39:03.978 INFO:teuthology.orchestra.run.smithi175.stdout:(82/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-23T09:39:04.000 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/180 2024-03-23T09:39:04.004 INFO:teuthology.orchestra.run.smithi175.stdout:(83/156): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-23T09:39:04.029 INFO:teuthology.orchestra.run.smithi175.stdout:(84/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-23T09:39:04.030 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/180 2024-03-23T09:39:04.067 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/180 2024-03-23T09:39:04.115 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/180 2024-03-23T09:39:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/180 2024-03-23T09:39:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/180 2024-03-23T09:39:04.246 INFO:teuthology.orchestra.run.smithi175.stdout:(85/156): jq-1.6-16.el9.x86_64.rpm 774 kB/s | 187 kB 00:00 2024-03-23T09:39:04.296 INFO:teuthology.orchestra.run.smithi175.stdout:(86/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-03-23T09:39:04.339 INFO:teuthology.orchestra.run.smithi175.stdout:(87/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-03-23T09:39:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/180 2024-03-23T09:39:04.363 INFO:teuthology.orchestra.run.smithi175.stdout:(88/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-23T09:39:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/180 2024-03-23T09:39:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/180 2024-03-23T09:39:04.422 INFO:teuthology.orchestra.run.smithi175.stdout:(89/156): elfutils-libelf-devel-0.190-2.el9.x86 258 kB/s | 21 kB 00:00 2024-03-23T09:39:04.447 INFO:teuthology.orchestra.run.smithi175.stdout:(90/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 304 kB/s | 25 kB 00:00 2024-03-23T09:39:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/180 2024-03-23T09:39:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 64/180 2024-03-23T09:39:04.564 INFO:teuthology.orchestra.run.smithi175.stdout:(91/156): python3-devel-3.9.17-2.el9.x86_64.rpm 2.1 MB/s | 245 kB 00:00 2024-03-23T09:39:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/180 2024-03-23T09:39:04.614 INFO:teuthology.orchestra.run.smithi175.stdout:(92/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-03-23T09:39:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/180 2024-03-23T09:39:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 67/180 2024-03-23T09:39:04.690 INFO:teuthology.orchestra.run.smithi175.stdout:(93/156): openssl-devel-3.0.7-27.el9.x86_64.rpm 15 MB/s | 4.1 MB 00:00 2024-03-23T09:39:04.715 INFO:teuthology.orchestra.run.smithi175.stdout:(94/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 934 kB/s | 94 kB 00:00 2024-03-23T09:39:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fmt-8.1.1-5.el9.x86_64 68/180 2024-03-23T09:39:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 69/180 2024-03-23T09:39:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ply-3.11-14.el9.noarch 70/180 2024-03-23T09:39:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 71/180 2024-03-23T09:39:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 72/180 2024-03-23T09:39:05.149 INFO:teuthology.orchestra.run.smithi175.stdout:(95/156): fmt-8.1.1-5.el9.x86_64.rpm 241 kB/s | 111 kB 00:00 2024-03-23T09:39:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 73/180 2024-03-23T09:39:05.257 INFO:teuthology.orchestra.run.smithi175.stdout:(96/156): gperftools-libs-2.9.1-2.el9.x86_64.rp 570 kB/s | 309 kB 00:00 2024-03-23T09:39:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 74/180 2024-03-23T09:39:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 75/180 2024-03-23T09:39:05.366 INFO:teuthology.orchestra.run.smithi175.stdout:(97/156): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-03-23T09:39:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/180 2024-03-23T09:39:05.404 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.9-15.el9.x86_64 77/180 2024-03-23T09:39:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 78/180 2024-03-23T09:39:05.466 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/180 2024-03-23T09:39:05.475 INFO:teuthology.orchestra.run.smithi175.stdout:(98/156): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-03-23T09:39:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/180 2024-03-23T09:39:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/180 2024-03-23T09:39:05.592 INFO:teuthology.orchestra.run.smithi175.stdout:(99/156): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-03-23T09:39:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/180 2024-03-23T09:39:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/180 2024-03-23T09:39:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/180 2024-03-23T09:39:05.700 INFO:teuthology.orchestra.run.smithi175.stdout:(100/156): libarrow-9.0.0-10.el9.x86_64.rpm 8.0 MB/s | 4.4 MB 00:00 2024-03-23T09:39:05.743 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/180 2024-03-23T09:39:05.777 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/180 2024-03-23T09:39:05.809 INFO:teuthology.orchestra.run.smithi175.stdout:(101/156): python3-asyncssh-2.13.2-5.el9.noarch 5.0 MB/s | 548 kB 00:00 2024-03-23T09:39:05.842 INFO:teuthology.orchestra.run.smithi175.stdout:(102/156): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2024-03-23T09:39:05.871 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/180 2024-03-23T09:39:05.906 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/180 2024-03-23T09:39:05.918 INFO:teuthology.orchestra.run.smithi175.stdout:(103/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 399 kB/s | 43 kB 00:00 2024-03-23T09:39:05.935 INFO:teuthology.orchestra.run.smithi175.stdout:(104/156): python3-cachetools-4.2.4-1.el9.noarc 348 kB/s | 32 kB 00:00 2024-03-23T09:39:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/180 2024-03-23T09:39:06.010 INFO:teuthology.orchestra.run.smithi175.stdout:(105/156): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-03-23T09:39:06.054 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-20.9-5.el9.noarch 90/180 2024-03-23T09:39:06.111 INFO:teuthology.orchestra.run.smithi175.stdout:(106/156): python3-cheroot-8.6.0-4.el9.noarch.r 980 kB/s | 172 kB 00:00 2024-03-23T09:39:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 91/180 2024-03-23T09:39:06.144 INFO:teuthology.orchestra.run.smithi175.stdout:(107/156): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-03-23T09:39:06.236 INFO:teuthology.orchestra.run.smithi175.stdout:(108/156): python3-google-auth-2.28.2-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-23T09:39:06.261 INFO:teuthology.orchestra.run.smithi175.stdout:(109/156): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-03-23T09:39:06.353 INFO:teuthology.orchestra.run.smithi175.stdout:(110/156): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-03-23T09:39:06.379 INFO:teuthology.orchestra.run.smithi175.stdout:(111/156): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-03-23T09:39:06.446 INFO:teuthology.orchestra.run.smithi175.stdout:(112/156): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-03-23T09:39:06.471 INFO:teuthology.orchestra.run.smithi175.stdout:(113/156): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-03-23T09:39:06.571 INFO:teuthology.orchestra.run.smithi175.stdout:(114/156): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-03-23T09:39:06.622 INFO:teuthology.orchestra.run.smithi175.stdout:(115/156): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-03-23T09:39:06.672 INFO:teuthology.orchestra.run.smithi175.stdout:(116/156): python3-more-itertools-8.12.0-2.el9. 785 kB/s | 79 kB 00:00 2024-03-23T09:39:06.714 INFO:teuthology.orchestra.run.smithi175.stdout:(117/156): python3-natsort-7.1.1-5.el9.noarch.r 626 kB/s | 58 kB 00:00 2024-03-23T09:39:06.806 INFO:teuthology.orchestra.run.smithi175.stdout:(118/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-03-23T09:39:06.823 INFO:teuthology.orchestra.run.smithi175.stdout:(119/156): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-03-23T09:39:06.882 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/180 2024-03-23T09:39:06.898 INFO:teuthology.orchestra.run.smithi175.stdout:(120/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 978 kB/s | 90 kB 00:00 2024-03-23T09:39:06.915 INFO:teuthology.orchestra.run.smithi175.stdout:(121/156): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-03-23T09:39:06.937 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 93/180 2024-03-23T09:39:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 94/180 2024-03-23T09:39:07.003 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 95/180 2024-03-23T09:39:07.015 INFO:teuthology.orchestra.run.smithi175.stdout:(122/156): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-03-23T09:39:07.032 INFO:teuthology.orchestra.run.smithi175.stdout:(123/156): python3-rsa-4.9-2.el9.noarch.rpm 505 kB/s | 59 kB 00:00 2024-03-23T09:39:07.124 INFO:teuthology.orchestra.run.smithi175.stdout:(124/156): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-03-23T09:39:07.150 INFO:teuthology.orchestra.run.smithi175.stdout:(125/156): python3-typing-extensions-4.4.0-2.el 436 kB/s | 51 kB 00:00 2024-03-23T09:39:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/180 2024-03-23T09:39:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/180 2024-03-23T09:39:07.242 INFO:teuthology.orchestra.run.smithi175.stdout:(126/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-03-23T09:39:07.267 INFO:teuthology.orchestra.run.smithi175.stdout:(127/156): python3-websocket-client-1.2.3-2.el9 768 kB/s | 90 kB 00:00 2024-03-23T09:39:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/180 2024-03-23T09:39:07.342 INFO:teuthology.orchestra.run.smithi175.stdout:(128/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-23T09:39:07.367 INFO:teuthology.orchestra.run.smithi175.stdout:(129/156): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-03-23T09:39:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:(130/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-03-23T09:39:07.568 INFO:teuthology.orchestra.run.smithi175.stdout:(131/156): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-03-23T09:39:07.979 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/180 2024-03-23T09:39:08.094 INFO:teuthology.orchestra.run.smithi175.stdout:(132/156): ceph-test-18.2.2-954.g82316b04.el9.x 4.8 MB/s | 40 MB 00:08 2024-03-23T09:39:08.103 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/180 2024-03-23T09:39:08.161 INFO:teuthology.orchestra.run.smithi175.stdout:(133/156): librados2-18.2.2-954.g82316b04.el9.x 4.6 MB/s | 3.3 MB 00:00 2024-03-23T09:39:08.186 INFO:teuthology.orchestra.run.smithi175.stdout:(134/156): xfsdump-3.1.12-4.el9.x86_64.rpm 3.4 MB/s | 321 kB 00:00 2024-03-23T09:39:08.394 INFO:teuthology.orchestra.run.smithi175.stdout:(135/156): NetworkManager-1.47.2-1.el9.x86_64.r 9.9 MB/s | 2.3 MB 00:00 2024-03-23T09:39:08.453 INFO:teuthology.orchestra.run.smithi175.stdout:(136/156): NetworkManager-libnm-1.47.2-1.el9.x8 6.9 MB/s | 1.8 MB 00:00 2024-03-23T09:39:08.470 INFO:teuthology.orchestra.run.smithi175.stdout:(137/156): NetworkManager-team-1.47.2-1.el9.x86 559 kB/s | 42 kB 00:00 2024-03-23T09:39:08.528 INFO:teuthology.orchestra.run.smithi175.stdout:(138/156): librbd1-18.2.2-954.g82316b04.el9.x86 3.1 MB/s | 3.0 MB 00:00 2024-03-23T09:39:08.579 INFO:teuthology.orchestra.run.smithi175.stdout:(139/156): NetworkManager-tui-1.47.2-1.el9.x86_ 1.9 MB/s | 248 kB 00:00 2024-03-23T09:39:08.629 INFO:teuthology.orchestra.run.smithi175.stdout:(140/156): acl-2.3.1-4.el9.x86_64.rpm 449 kB/s | 71 kB 00:00 2024-03-23T09:39:08.679 INFO:teuthology.orchestra.run.smithi175.stdout:(141/156): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-03-23T09:39:08.704 INFO:teuthology.orchestra.run.smithi175.stdout:(142/156): elfutils-debuginfod-client-0.190-2.e 207 kB/s | 36 kB 00:00 2024-03-23T09:39:08.730 INFO:teuthology.orchestra.run.smithi175.stdout:(143/156): elfutils-libs-0.190-2.el9.x86_64.rpm 2.5 MB/s | 256 kB 00:00 2024-03-23T09:39:08.755 INFO:teuthology.orchestra.run.smithi175.stdout:(144/156): libacl-2.3.1-4.el9.x86_64.rpm 305 kB/s | 23 kB 00:00 2024-03-23T09:39:08.780 INFO:teuthology.orchestra.run.smithi175.stdout:(145/156): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-03-23T09:39:08.805 INFO:teuthology.orchestra.run.smithi175.stdout:(146/156): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-23T09:39:08.831 INFO:teuthology.orchestra.run.smithi175.stdout:(147/156): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 136 kB 00:00 2024-03-23T09:39:08.856 INFO:teuthology.orchestra.run.smithi175.stdout:(148/156): libsmartcols-2.37.4-18.el9.x86_64.rp 848 kB/s | 64 kB 00:00 2024-03-23T09:39:08.881 INFO:teuthology.orchestra.run.smithi175.stdout:(149/156): libuuid-2.37.4-18.el9.x86_64.rpm 381 kB/s | 29 kB 00:00 2024-03-23T09:39:08.923 INFO:teuthology.orchestra.run.smithi175.stdout:(150/156): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-03-23T09:39:08.999 INFO:teuthology.orchestra.run.smithi175.stdout:(151/156): openssl-libs-3.0.7-27.el9.x86_64.rpm 15 MB/s | 2.2 MB 00:00 2024-03-23T09:39:09.049 INFO:teuthology.orchestra.run.smithi175.stdout:(152/156): util-linux-2.37.4-18.el9.x86_64.rpm 14 MB/s | 2.3 MB 00:00 2024-03-23T09:39:09.091 INFO:teuthology.orchestra.run.smithi175.stdout:(153/156): util-linux-user-2.37.4-18.el9.x86_64 339 kB/s | 31 kB 00:00 2024-03-23T09:39:09.116 INFO:teuthology.orchestra.run.smithi175.stdout:(154/156): util-linux-core-2.37.4-18.el9.x86_64 2.4 MB/s | 465 kB 00:00 2024-03-23T09:39:09.158 INFO:teuthology.orchestra.run.smithi175.stdout:(155/156): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.9 MB/s | 1.1 MB 00:00 2024-03-23T09:39:09.191 INFO:teuthology.orchestra.run.smithi175.stdout:(156/156): libacl-devel-2.3.1-4.el9.x86_64.rpm 743 kB/s | 74 kB 00:00 2024-03-23T09:39:09.198 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-03-23T09:39:09.198 INFO:teuthology.orchestra.run.smithi175.stdout:Total 18 MB/s | 204 MB 00:11 2024-03-23T09:39:09.409 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/180 2024-03-23T09:39:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/180 2024-03-23T09:39:09.559 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 103/180 2024-03-23T09:39:09.589 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : jq-1.6-16.el9.x86_64 104/180 2024-03-23T09:39:09.609 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-03-23T09:39:09.617 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 105/180 2024-03-23T09:39:09.877 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-03-23T09:39:09.877 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-03-23T09:39:09.931 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 106/180 2024-03-23T09:39:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:18.2.2-954.g82316b04.el9.x86_64 107/180 2024-03-23T09:39:10.038 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:18.2.2-954.g82316b04.el9.x86_64 107/180 2024-03-23T09:39:10.068 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:18.2.2-954.g82316b04.el9.x86_64 108/180 2024-03-23T09:39:10.105 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 109/180 2024-03-23T09:39:10.527 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 110/180 2024-03-23T09:39:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:18.2.2-954.g82316b04.el9.x86_64 111/180 2024-03-23T09:39:10.621 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:18.2.2-954.g82316b04.el9.x86_64 111/180 2024-03-23T09:39:10.643 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:18.2.2-954.g82316b04.el9.x86_64 112/180 2024-03-23T09:39:11.744 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-03-23T09:39:11.744 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-03-23T09:39:12.790 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 113/180 2024-03-23T09:39:12.823 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 113/180 2024-03-23T09:39:13.378 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 113/180 2024-03-23T09:39:13.409 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 114/180 2024-03-23T09:39:13.500 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 114/180 2024-03-23T09:39:13.500 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-23T09:39:13.500 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-23T09:39:13.500 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:39:13.536 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 115/180 2024-03-23T09:39:18.002 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-03-23T09:39:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/180 2024-03-23T09:39:18.147 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/180 2024-03-23T09:39:18.185 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-23T09:39:18.195 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-23T09:39:18.235 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/180 2024-03-23T09:39:18.289 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/180 2024-03-23T09:39:18.330 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/180 2024-03-23T09:39:18.355 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/180 2024-03-23T09:39:18.406 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/180 2024-03-23T09:39:18.738 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/180 2024-03-23T09:39:18.798 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:18.2.2-954.g82316b04.el9.x86_64 10/180 2024-03-23T09:39:18.900 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:18.2.2-954.g82316b04.el9.x86_64 10/180 2024-03-23T09:39:18.932 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 11/180 2024-03-23T09:39:19.008 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 11/180 2024-03-23T09:39:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:18.2.2-954.g82316b04.el9.x86_64 12/180 2024-03-23T09:39:19.090 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/180 2024-03-23T09:39:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/180 2024-03-23T09:39:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/180 2024-03-23T09:39:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 16/180 2024-03-23T09:39:19.431 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 16/180 2024-03-23T09:39:19.456 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_ 17/180 2024-03-23T09:39:19.505 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_ 17/180 2024-03-23T09:39:19.545 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/180 2024-03-23T09:39:19.585 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/180 2024-03-23T09:39:19.784 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/180 2024-03-23T09:39:19.895 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 21/180 2024-03-23T09:39:19.955 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 21/180 2024-03-23T09:39:20.057 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-03-23T09:39:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-03-23T09:39:20.371 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-03-23T09:39:20.432 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/180 2024-03-23T09:39:20.462 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/180 2024-03-23T09:39:20.509 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/180 2024-03-23T09:39:20.583 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/180 2024-03-23T09:39:20.637 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/180 2024-03-23T09:39:20.683 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/180 2024-03-23T09:39:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/180 2024-03-23T09:39:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/180 2024-03-23T09:39:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/180 2024-03-23T09:39:20.878 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 32/180 2024-03-23T09:39:20.915 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:18.2.2-954.g82316b04.el9.x 33/180 2024-03-23T09:39:20.959 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:18.2.2-954.g82316b04.el9 34/180 2024-03-23T09:39:21.032 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:18.2.2-954.g82316b04.el9.x86_64 35/180 2024-03-23T09:39:21.163 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/180 2024-03-23T09:39:21.210 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/180 2024-03-23T09:39:21.728 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/180 2024-03-23T09:39:21.807 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/180 2024-03-23T09:39:21.887 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/180 2024-03-23T09:39:21.935 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-23T09:39:21.948 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-23T09:39:22.014 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/180 2024-03-23T09:39:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:18.2.2-954.g82316b04.el9.x86_64 43/180 2024-03-23T09:39:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/180 2024-03-23T09:39:22.376 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 45/180 2024-03-23T09:39:22.581 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 46/180 2024-03-23T09:39:22.660 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 47/180 2024-03-23T09:39:22.703 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 48/180 2024-03-23T09:39:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/180 2024-03-23T09:39:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 50/180 2024-03-23T09:39:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : re2-1:20211101-3.el9.x86_64 51/180 2024-03-23T09:39:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 52/180 2024-03-23T09:39:23.079 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/180 2024-03-23T09:39:23.143 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/180 2024-03-23T09:39:23.175 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/180 2024-03-23T09:39:23.212 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/180 2024-03-23T09:39:23.258 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/180 2024-03-23T09:39:23.303 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/180 2024-03-23T09:39:23.345 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/180 2024-03-23T09:39:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/180 2024-03-23T09:39:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/180 2024-03-23T09:39:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/180 2024-03-23T09:39:23.792 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/180 2024-03-23T09:39:23.824 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 64/180 2024-03-23T09:39:23.864 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/180 2024-03-23T09:39:23.889 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/180 2024-03-23T09:39:23.916 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 67/180 2024-03-23T09:39:24.043 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fmt-8.1.1-5.el9.x86_64 68/180 2024-03-23T09:39:24.084 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 69/180 2024-03-23T09:39:24.133 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 70/180 2024-03-23T09:39:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 71/180 2024-03-23T09:39:24.387 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 72/180 2024-03-23T09:39:24.438 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 73/180 2024-03-23T09:39:24.566 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 74/180 2024-03-23T09:39:24.603 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 75/180 2024-03-23T09:39:24.635 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/180 2024-03-23T09:39:24.666 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-15.el9.x86_64 77/180 2024-03-23T09:39:24.702 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 78/180 2024-03-23T09:39:24.736 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/180 2024-03-23T09:39:24.773 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/180 2024-03-23T09:39:24.803 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/180 2024-03-23T09:39:24.878 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/180 2024-03-23T09:39:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/180 2024-03-23T09:39:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/180 2024-03-23T09:39:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/180 2024-03-23T09:39:25.023 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/180 2024-03-23T09:39:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/180 2024-03-23T09:39:25.151 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/180 2024-03-23T09:39:25.197 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/180 2024-03-23T09:39:25.293 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 90/180 2024-03-23T09:39:25.368 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 91/180 2024-03-23T09:39:25.444 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 115/180 2024-03-23T09:39:25.444 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-03-23T09:39:25.444 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-03-23T09:39:25.445 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-03-23T09:39:25.445 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-03-23T09:39:25.445 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-03-23T09:39:25.445 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /root 2024-03-23T09:39:25.445 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-03-23T09:39:25.445 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:39:25.607 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 116/180 2024-03-23T09:39:25.622 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 116/180 2024-03-23T09:39:25.622 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:25.622 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-23T09:39:25.622 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:39:26.081 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/180 2024-03-23T09:39:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 93/180 2024-03-23T09:39:26.508 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 94/180 2024-03-23T09:39:26.540 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 95/180 2024-03-23T09:39:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/180 2024-03-23T09:39:26.746 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/180 2024-03-23T09:39:26.833 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/180 2024-03-23T09:39:27.118 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 117/180 2024-03-23T09:39:27.143 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 117/180 2024-03-23T09:39:27.144 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:27.144 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-23T09:39:27.144 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T09:39:27.144 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T09:39:27.144 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:39:27.360 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 118/180 2024-03-23T09:39:27.383 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 118/180 2024-03-23T09:39:27.383 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:27.383 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-23T09:39:27.383 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T09:39:27.383 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T09:39:27.383 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:39:27.484 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/180 2024-03-23T09:39:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/180 2024-03-23T09:39:27.853 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 119/180 2024-03-23T09:39:27.875 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 119/180 2024-03-23T09:39:27.875 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:27.876 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-23T09:39:27.876 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T09:39:27.876 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T09:39:27.876 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:39:27.971 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : automake-1.16.2-8.el9.noarch 120/180 2024-03-23T09:39:28.384 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:18.2.2-954.g82316b04.el9.noarch 121/180 2024-03-23T09:39:28.403 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:18.2.2-954.g82316b04.el9.noarch 121/180 2024-03-23T09:39:28.432 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 122/180 2024-03-23T09:39:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 123/180 2024-03-23T09:39:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noar 124/180 2024-03-23T09:39:28.761 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noar 124/180 2024-03-23T09:39:28.793 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.no 125/180 2024-03-23T09:39:28.810 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/180 2024-03-23T09:39:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/180 2024-03-23T09:39:28.927 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 103/180 2024-03-23T09:39:28.957 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : jq-1.6-16.el9.x86_64 104/180 2024-03-23T09:39:28.984 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 105/180 2024-03-23T09:39:29.285 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 106/180 2024-03-23T09:39:29.314 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:18.2.2-954.g82316b04.el9.x86_64 107/180 2024-03-23T09:39:29.386 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:18.2.2-954.g82316b04.el9.x86_64 107/180 2024-03-23T09:39:29.418 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:18.2.2-954.g82316b04.el9.x86_64 108/180 2024-03-23T09:39:29.455 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 109/180 2024-03-23T09:39:29.806 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.no 125/180 2024-03-23T09:39:29.836 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 126/180 2024-03-23T09:39:29.861 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 110/180 2024-03-23T09:39:29.892 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:18.2.2-954.g82316b04.el9.x86_64 111/180 2024-03-23T09:39:29.942 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 126/180 2024-03-23T09:39:29.962 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:18.2.2-954.g82316b04.el9.x86_64 111/180 2024-03-23T09:39:29.985 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:18.2.2-954.g82316b04.el9.x86_64 112/180 2024-03-23T09:39:30.098 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el9 127/180 2024-03-23T09:39:30.129 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 128/180 2024-03-23T09:39:30.152 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 128/180 2024-03-23T09:39:30.152 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:30.152 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-23T09:39:30.153 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T09:39:30.153 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T09:39:30.153 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:39:30.196 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 129/180 2024-03-23T09:39:30.208 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 129/180 2024-03-23T09:39:30.259 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:18.2.2-954.g82316b04.el9.x86_64 130/180 2024-03-23T09:39:31.004 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libtool-2.4.6-45.el9.x86_64 131/180 2024-03-23T09:39:31.032 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 132/180 2024-03-23T09:39:31.054 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 132/180 2024-03-23T09:39:31.054 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:31.055 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-23T09:39:31.055 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T09:39:31.055 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T09:39:31.055 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:39:31.099 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-954.g82316b 133/180 2024-03-23T09:39:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-954.g82316b 133/180 2024-03-23T09:39:31.121 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:31.121 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-23T09:39:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:39:31.175 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 134/180 2024-03-23T09:39:31.197 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 134/180 2024-03-23T09:39:31.197 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:31.197 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-23T09:39:31.198 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T09:39:31.198 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T09:39:31.198 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:39:31.508 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 135/180 2024-03-23T09:39:31.531 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 135/180 2024-03-23T09:39:31.531 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:31.531 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-23T09:39:31.531 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T09:39:31.531 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T09:39:31.531 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:39:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 113/180 2024-03-23T09:39:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 113/180 2024-03-23T09:39:32.661 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 113/180 2024-03-23T09:39:32.694 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 114/180 2024-03-23T09:39:32.771 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 114/180 2024-03-23T09:39:32.771 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-23T09:39:32.771 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-23T09:39:32.772 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:39:32.812 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 115/180 2024-03-23T09:39:35.429 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:18.2.2-954.g82316b04.el9.x86_64 136/180 2024-03-23T09:39:35.471 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:18.2.2-954.g82316b04.el9.x86_64 137/180 2024-03-23T09:39:35.501 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:18.2.2-954.g82316b04.el9.x86_64 138/180 2024-03-23T09:39:35.705 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/180 2024-03-23T09:39:35.819 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 140/180 2024-03-23T09:39:35.860 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:18.2.2-954.g82316b04.el9.x86_64 141/180 2024-03-23T09:39:35.917 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/180 2024-03-23T09:39:35.967 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 143/180 2024-03-23T09:39:36.011 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 144/180 2024-03-23T09:39:36.043 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:18.2.2-954.g82316b04.el9.x86_6 145/180 2024-03-23T09:39:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 146/180 2024-03-23T09:39:36.105 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 147/180 2024-03-23T09:39:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 148/180 2024-03-23T09:39:36.240 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/180 2024-03-23T09:39:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 150/180 2024-03-23T09:39:36.890 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-23T09:39:36.927 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-23T09:39:36.947 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-23T09:39:37.819 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-top-2:18.2.2-954.g82316b04.el9.noarch 152/180 2024-03-23T09:39:37.907 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 153/180 2024-03-23T09:39:37.949 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/180 2024-03-23T09:39:38.044 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/180 2024-03-23T09:39:38.077 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/180 2024-03-23T09:39:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 157/180 2024-03-23T09:39:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 158/180 2024-03-23T09:39:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 159/180 2024-03-23T09:39:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-23T09:39:38.242 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-23T09:39:38.271 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/180 2024-03-23T09:39:38.271 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-23T09:39:38.356 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-23T09:39:38.400 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 163/180 2024-03-23T09:39:38.432 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 164/180 2024-03-23T09:39:38.458 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 165/180 2024-03-23T09:39:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 166/180 2024-03-23T09:39:38.518 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 167/180 2024-03-23T09:39:38.555 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 168/180 2024-03-23T09:39:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 169/180 2024-03-23T09:39:38.621 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 170/180 2024-03-23T09:39:38.646 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 171/180 2024-03-23T09:39:38.671 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 172/180 2024-03-23T09:39:38.705 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/180 2024-03-23T09:39:38.713 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-23T09:39:38.713 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-23T09:39:38.892 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-23T09:39:38.892 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-23T09:39:38.991 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-23T09:39:39.016 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 176/180 2024-03-23T09:39:39.041 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 177/180 2024-03-23T09:39:39.067 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 178/180 2024-03-23T09:39:39.092 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/180 2024-03-23T09:39:39.160 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-23T09:39:43.646 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-23T09:39:43.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:18.2.2-954.g82316b04.el9.x86_64 1/180 2024-03-23T09:39:43.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 2/180 2024-03-23T09:39:43.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 3/180 2024-03-23T09:39:43.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:18.2.2-954.g82316b04.el9.x86_64 4/180 2024-03-23T09:39:43.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-954.g82316b 5/180 2024-03-23T09:39:43.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 6/180 2024-03-23T09:39:43.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 7/180 2024-03-23T09:39:43.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 8/180 2024-03-23T09:39:43.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 9/180 2024-03-23T09:39:43.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 10/180 2024-03-23T09:39:43.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 11/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:18.2.2-954.g82316b04.el9.x86_64 12/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 13/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:18.2.2-954.g82316b04.el9.x86_6 14/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 15/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 16/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:18.2.2-954.g82316b04.el9.x86_64 17/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_ 18/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:18.2.2-954.g82316b04.el9.x86_64 19/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:18.2.2-954.g82316b04.el9 20/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:18.2.2-954.g82316b04.el9.x 21/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:18.2.2-954.g82316b04.el9.x86_64 22/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:18.2.2-954.g82316b04.el9.x86_64 23/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:18.2.2-954.g82316b04.el9.x86_64 24/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:18.2.2-954.g82316b04.el9.x86_64 25/180 2024-03-23T09:39:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:18.2.2-954.g82316b04.el9.x86_64 26/180 2024-03-23T09:39:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 27/180 2024-03-23T09:39:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:18.2.2-954.g82316b04.el9.x86_64 28/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 29/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noar 30/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.no 31/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 32/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el9 33/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 34/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 35/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 36/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:18.2.2-954.g82316b04.el9.noarch 37/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-top-2:18.2.2-954.g82316b04.el9.noarch 38/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flex-2.6.4-9.el9.x86_64 42/180 2024-03-23T09:39:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 43/180 2024-03-23T09:39:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 44/180 2024-03-23T09:39:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 45/180 2024-03-23T09:39:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/180 2024-03-23T09:39:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/180 2024-03-23T09:39:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 48/180 2024-03-23T09:39:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 49/180 2024-03-23T09:39:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 50/180 2024-03-23T09:39:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 51/180 2024-03-23T09:39:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/180 2024-03-23T09:39:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 53/180 2024-03-23T09:39:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 54/180 2024-03-23T09:39:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 55/180 2024-03-23T09:39:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-13.11-1.el9.x86_64 56/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 57/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 58/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 59/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/180 2024-03-23T09:39:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 72/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 74/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 75/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 76/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 77/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 78/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 79/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 80/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 82/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ply-3.11-14.el9.noarch 83/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 84/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 85/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : jq-1.6-16.el9.x86_64 86/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 87/180 2024-03-23T09:39:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 88/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 89/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 90/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 91/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 92/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 93/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 94/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 95/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 96/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 102/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 103/180 2024-03-23T09:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 104/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 105/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 106/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 107/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 108/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 109/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 110/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 111/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 112/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 113/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 114/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/180 2024-03-23T09:39:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:18.2.2-954.g82316b04.el9.x86_64 133/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:18.2.2-954.g82316b04.el9.x86_64 135/180 2024-03-23T09:39:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 137/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 138/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 139/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 141/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 143/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 145/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 149/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 151/180 2024-03-23T09:39:43.668 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 153/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 157/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 158/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 159/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 160/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 161/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 162/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 163/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 164/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 165/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 166/180 2024-03-23T09:39:43.669 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 167/180 2024-03-23T09:39:43.670 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 168/180 2024-03-23T09:39:43.670 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 169/180 2024-03-23T09:39:43.670 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 170/180 2024-03-23T09:39:43.670 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 171/180 2024-03-23T09:39:43.670 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 172/180 2024-03-23T09:39:43.670 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 173/180 2024-03-23T09:39:43.670 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 174/180 2024-03-23T09:39:43.670 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 175/180 2024-03-23T09:39:43.670 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 176/180 2024-03-23T09:39:43.670 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 177/180 2024-03-23T09:39:43.670 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 178/180 2024-03-23T09:39:43.670 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 179/180 2024-03-23T09:39:44.210 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 115/180 2024-03-23T09:39:44.210 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-03-23T09:39:44.210 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-03-23T09:39:44.210 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-03-23T09:39:44.210 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-03-23T09:39:44.210 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-03-23T09:39:44.211 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-03-23T09:39:44.211 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-03-23T09:39:44.211 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:39:44.371 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 116/180 2024-03-23T09:39:44.387 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 116/180 2024-03-23T09:39:44.387 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:44.387 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-23T09:39:44.387 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:39:45.866 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 117/180 2024-03-23T09:39:45.892 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 117/180 2024-03-23T09:39:45.892 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:45.892 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-23T09:39:45.892 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T09:39:45.892 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T09:39:45.892 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:39:46.108 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 118/180 2024-03-23T09:39:46.130 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 118/180 2024-03-23T09:39:46.131 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:46.131 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-23T09:39:46.131 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T09:39:46.131 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T09:39:46.131 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:39:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 180/180 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: acl-2.3.1-4.el9.x86_64 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-23T09:39:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: automake-1.16.2-8.el9.noarch 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: bison-3.7.4-5.el9.x86_64 2024-03-23T09:39:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: flex-2.6.4-9.el9.x86_64 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-23T09:39:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-23T09:39:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: jq-1.6-16.el9.x86_64 2024-03-23T09:39:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-23T09:39:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-23T09:39:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-23T09:39:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-23T09:39:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-23T09:39:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-23T09:39:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-23T09:39:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-23T09:39:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-23T09:39:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-13.11-1.el9.x86_64 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-23T09:39:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-23T09:39:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply-3.11-14.el9.noarch 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-23T09:39:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:39:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: re2-1:20211101-3.el9.x86_64 2024-03-23T09:39:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-23T09:39:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-23T09:39:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-23T09:39:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-23T09:39:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-23T09:39:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-23T09:39:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:39:46.448 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-03-23T09:39:46.567 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 119/180 2024-03-23T09:39:46.590 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 119/180 2024-03-23T09:39:46.590 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:46.590 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-23T09:39:46.590 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T09:39:46.590 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T09:39:46.590 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:39:46.677 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : automake-1.16.2-8.el9.noarch 120/180 2024-03-23T09:39:46.914 DEBUG:teuthology.parallel:result is None 2024-03-23T09:39:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:18.2.2-954.g82316b04.el9.noarch 121/180 2024-03-23T09:39:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:18.2.2-954.g82316b04.el9.noarch 121/180 2024-03-23T09:39:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 122/180 2024-03-23T09:39:47.201 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 123/180 2024-03-23T09:39:47.223 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noar 124/180 2024-03-23T09:39:47.446 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noar 124/180 2024-03-23T09:39:47.482 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.no 125/180 2024-03-23T09:39:48.444 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.no 125/180 2024-03-23T09:39:48.483 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 126/180 2024-03-23T09:39:48.582 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 126/180 2024-03-23T09:39:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el9 127/180 2024-03-23T09:39:48.768 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 128/180 2024-03-23T09:39:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 128/180 2024-03-23T09:39:48.792 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:48.792 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-23T09:39:48.792 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T09:39:48.792 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T09:39:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:39:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 129/180 2024-03-23T09:39:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 129/180 2024-03-23T09:39:48.897 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:18.2.2-954.g82316b04.el9.x86_64 130/180 2024-03-23T09:39:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libtool-2.4.6-45.el9.x86_64 131/180 2024-03-23T09:39:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 132/180 2024-03-23T09:39:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 132/180 2024-03-23T09:39:49.668 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:49.668 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-23T09:39:49.668 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T09:39:49.668 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T09:39:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:39:49.713 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-954.g82316b 133/180 2024-03-23T09:39:49.735 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-954.g82316b 133/180 2024-03-23T09:39:49.736 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:49.736 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-23T09:39:49.736 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:39:49.788 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 134/180 2024-03-23T09:39:49.810 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 134/180 2024-03-23T09:39:49.811 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:49.811 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-23T09:39:49.811 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T09:39:49.811 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T09:39:49.811 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:39:50.106 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 135/180 2024-03-23T09:39:50.128 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 135/180 2024-03-23T09:39:50.129 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T09:39:50.129 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-23T09:39:50.129 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T09:39:50.129 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T09:39:50.129 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:39:53.850 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:18.2.2-954.g82316b04.el9.x86_64 136/180 2024-03-23T09:39:53.892 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:18.2.2-954.g82316b04.el9.x86_64 137/180 2024-03-23T09:39:53.922 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:18.2.2-954.g82316b04.el9.x86_64 138/180 2024-03-23T09:39:54.114 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/180 2024-03-23T09:39:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 140/180 2024-03-23T09:39:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:18.2.2-954.g82316b04.el9.x86_64 141/180 2024-03-23T09:39:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/180 2024-03-23T09:39:54.389 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 143/180 2024-03-23T09:39:54.432 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 144/180 2024-03-23T09:39:54.465 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:18.2.2-954.g82316b04.el9.x86_6 145/180 2024-03-23T09:39:54.497 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 146/180 2024-03-23T09:39:54.526 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 147/180 2024-03-23T09:39:54.599 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 148/180 2024-03-23T09:39:54.649 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/180 2024-03-23T09:39:54.691 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 150/180 2024-03-23T09:39:55.282 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-23T09:39:55.331 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-23T09:39:55.351 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-23T09:39:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-top-2:18.2.2-954.g82316b04.el9.noarch 152/180 2024-03-23T09:39:56.243 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 153/180 2024-03-23T09:39:56.284 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/180 2024-03-23T09:39:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/180 2024-03-23T09:39:56.404 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/180 2024-03-23T09:39:56.429 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 157/180 2024-03-23T09:39:56.455 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 158/180 2024-03-23T09:39:56.490 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 159/180 2024-03-23T09:39:56.490 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-23T09:39:56.586 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-23T09:39:56.615 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/180 2024-03-23T09:39:56.615 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-23T09:39:56.700 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-23T09:39:56.727 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 163/180 2024-03-23T09:39:56.759 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 164/180 2024-03-23T09:39:56.785 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 165/180 2024-03-23T09:39:56.809 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 166/180 2024-03-23T09:39:56.845 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 167/180 2024-03-23T09:39:56.882 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 168/180 2024-03-23T09:39:56.914 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 169/180 2024-03-23T09:39:56.939 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 170/180 2024-03-23T09:39:56.964 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 171/180 2024-03-23T09:39:56.990 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 172/180 2024-03-23T09:39:57.015 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/180 2024-03-23T09:39:57.022 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-23T09:39:57.022 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-23T09:39:57.201 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-23T09:39:57.202 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-23T09:39:57.284 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-23T09:39:57.309 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 176/180 2024-03-23T09:39:57.334 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 177/180 2024-03-23T09:39:57.360 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 178/180 2024-03-23T09:39:57.385 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/180 2024-03-23T09:39:57.450 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-23T09:40:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-23T09:40:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:18.2.2-954.g82316b04.el9.x86_64 1/180 2024-03-23T09:40:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 2/180 2024-03-23T09:40:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 3/180 2024-03-23T09:40:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:18.2.2-954.g82316b04.el9.x86_64 4/180 2024-03-23T09:40:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-954.g82316b 5/180 2024-03-23T09:40:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 6/180 2024-03-23T09:40:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 7/180 2024-03-23T09:40:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 8/180 2024-03-23T09:40:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 9/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 10/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 11/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:18.2.2-954.g82316b04.el9.x86_64 12/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 13/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:18.2.2-954.g82316b04.el9.x86_6 14/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 15/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 16/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:18.2.2-954.g82316b04.el9.x86_64 17/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_ 18/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:18.2.2-954.g82316b04.el9.x86_64 19/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:18.2.2-954.g82316b04.el9 20/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:18.2.2-954.g82316b04.el9.x 21/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:18.2.2-954.g82316b04.el9.x86_64 22/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:18.2.2-954.g82316b04.el9.x86_64 23/180 2024-03-23T09:40:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:18.2.2-954.g82316b04.el9.x86_64 24/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:18.2.2-954.g82316b04.el9.x86_64 25/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:18.2.2-954.g82316b04.el9.x86_64 26/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 27/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:18.2.2-954.g82316b04.el9.x86_64 28/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 29/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noar 30/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.no 31/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 32/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el9 33/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 34/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 35/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 36/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:18.2.2-954.g82316b04.el9.noarch 37/180 2024-03-23T09:40:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-top-2:18.2.2-954.g82316b04.el9.noarch 38/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flex-2.6.4-9.el9.x86_64 42/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 43/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 44/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 45/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 48/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 49/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 50/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 51/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/180 2024-03-23T09:40:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 53/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 54/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 55/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-13.11-1.el9.x86_64 56/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 57/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 58/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 59/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/180 2024-03-23T09:40:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 72/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 74/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 75/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 76/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 77/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 78/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 79/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 80/180 2024-03-23T09:40:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/180 2024-03-23T09:40:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 82/180 2024-03-23T09:40:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 83/180 2024-03-23T09:40:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 84/180 2024-03-23T09:40:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 85/180 2024-03-23T09:40:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : jq-1.6-16.el9.x86_64 86/180 2024-03-23T09:40:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 87/180 2024-03-23T09:40:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 88/180 2024-03-23T09:40:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 89/180 2024-03-23T09:40:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 90/180 2024-03-23T09:40:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 91/180 2024-03-23T09:40:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 92/180 2024-03-23T09:40:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 93/180 2024-03-23T09:40:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 94/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 95/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 96/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 102/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 103/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 104/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 105/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 106/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 107/180 2024-03-23T09:40:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 108/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 109/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 110/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 111/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 112/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 113/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 114/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/180 2024-03-23T09:40:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/180 2024-03-23T09:40:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/180 2024-03-23T09:40:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/180 2024-03-23T09:40:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/180 2024-03-23T09:40:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/180 2024-03-23T09:40:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/180 2024-03-23T09:40:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/180 2024-03-23T09:40:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/180 2024-03-23T09:40:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/180 2024-03-23T09:40:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/180 2024-03-23T09:40:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:18.2.2-954.g82316b04.el9.x86_64 133/180 2024-03-23T09:40:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/180 2024-03-23T09:40:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:18.2.2-954.g82316b04.el9.x86_64 135/180 2024-03-23T09:40:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 137/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 138/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 139/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 141/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 143/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 145/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 149/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/180 2024-03-23T09:40:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 151/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 153/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 157/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 158/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 159/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 160/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 161/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 162/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 163/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 164/180 2024-03-23T09:40:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 165/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 166/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 167/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 168/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 169/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 170/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 171/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 172/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 173/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 174/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 175/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 176/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 177/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 178/180 2024-03-23T09:40:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 179/180 2024-03-23T09:40:04.438 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 180/180 2024-03-23T09:40:04.438 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout: acl-2.3.1-4.el9.x86_64 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-23T09:40:04.439 INFO:teuthology.orchestra.run.smithi175.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout: automake-1.16.2-8.el9.noarch 2024-03-23T09:40:04.440 INFO:teuthology.orchestra.run.smithi175.stdout: bison-3.7.4-5.el9.x86_64 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:40:04.441 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:40:04.442 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.442 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.442 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:40:04.442 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.442 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.442 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.442 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:40:04.442 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:40:04.442 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.442 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T09:40:04.442 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-23T09:40:04.442 INFO:teuthology.orchestra.run.smithi175.stdout: flex-2.6.4-9.el9.x86_64 2024-03-23T09:40:04.442 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: jq-1.6-16.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-23T09:40:04.443 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-23T09:40:04.444 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-13.11-1.el9.x86_64 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.445 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-23T09:40:04.446 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-23T09:40:04.446 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-23T09:40:04.446 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-23T09:40:04.446 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-23T09:40:04.446 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-23T09:40:04.446 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-23T09:40:04.446 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-23T09:40:04.446 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-23T09:40:04.446 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-23T09:40:04.446 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-23T09:40:04.446 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-23T09:40:04.446 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-23T09:40:04.446 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-03-23T09:40:04.447 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-23T09:40:04.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: re2-1:20211101-3.el9.x86_64 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-23T09:40:04.449 INFO:teuthology.orchestra.run.smithi175.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-23T09:40:04.450 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-23T09:40:04.450 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-23T09:40:04.450 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-23T09:40:04.450 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:04.450 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-03-23T09:40:05.064 DEBUG:teuthology.parallel:result is None 2024-03-23T09:40:05.065 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T09:40:05.283 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T09:40:05.309 INFO:teuthology.orchestra.run.smithi047.stdout:18.2.2-954.g82316b04.el9 2024-03-23T09:40:05.309 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-954.g82316b04.el9 2024-03-23T09:40:05.309 INFO:teuthology.task.install:The correct ceph version 18.2.2-954.g82316b04 is installed. 2024-03-23T09:40:05.311 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T09:40:05.455 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T09:40:05.478 INFO:teuthology.orchestra.run.smithi175.stdout:18.2.2-954.g82316b04.el9 2024-03-23T09:40:05.478 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-954.g82316b04.el9 2024-03-23T09:40:05.478 INFO:teuthology.task.install:The correct ceph version 18.2.2-954.g82316b04 is installed. 2024-03-23T09:40:05.480 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-23T09:40:05.480 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:05.480 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T09:40:05.523 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:05.523 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T09:40:05.552 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-23T09:40:05.553 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:05.553 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T09:40:05.594 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T09:40:05.662 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:05.663 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T09:40:05.690 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T09:40:05.757 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-23T09:40:05.757 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:05.757 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T09:40:05.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T09:40:05.855 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:05.855 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T09:40:05.881 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T09:40:05.948 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-23T09:40:05.948 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:05.948 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T09:40:05.974 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T09:40:06.041 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:06.041 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T09:40:06.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T09:40:06.134 INFO:teuthology.run_tasks:Running task ceph... 2024-03-23T09:40:06.190 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-23T09:40:06.191 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /var/log/ceph 2024-03-23T09:40:06.193 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /var/log/ceph 2024-03-23T09:40:06.235 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-23T09:40:06.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-23T09:40:06.238 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-23T09:40:06.286 INFO:tasks.ceph:Creating extra log directories... 2024-03-23T09:40:06.286 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-23T09:40:06.305 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-23T09:40:06.359 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-23T09:40:06.360 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-23T09:40:06.360 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-23T09:40:06.360 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:06.360 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-23T09:40:06.386 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-23T09:40:06.507 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-23T09:40:06.533 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:06.533 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-23T09:40:06.548 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-23T09:40:06.666 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-23T09:40:06.697 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-23T09:40:06.697 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-23T09:40:06.697 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427', 'branch': 'wip-yuri11-testing-2024-03-21-0851-reef', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7617427', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri11-testing-2024-03-21-0851-reef'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri11-testing-2024-03-21-0851-reef', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1919, 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri11-testing-2024-03-21-0851-reef', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi047.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN4NZx+BuJhDznyGKe/diyX8WDTP3EKrADaXXu6wf534XL6d+2TjpBMY3IxKxezfWgv0a/reLAHWTfEnXFwD10w=', 'smithi175.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIdxuUty99rrpzpD84B71aZ9IUxYu6J/FBGUB3ehkllQxBfNgtQAITD6ApkLFMCuIWn2rUen4iMnaOezzkYrt+4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-22_13:15:24', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-23T09:40:06.698 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-23T09:40:06.700 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-23T09:40:06.748 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/run/ceph 2024-03-23T09:40:06.754 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/run/ceph 2024-03-23T09:40:06.815 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:06.815 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T09:40:06.832 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T09:40:06.833 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-03-23T09:40:06.887 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T09:40:06.887 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T09:40:06.887 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-23T09:40:06.887 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T09:40:06.887 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T09:40:06.887 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-23 09:39:39.510102962 +0000 2024-03-23T09:40:06.887 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-23 09:37:33.411620446 +0000 2024-03-23T09:40:06.887 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-23 09:37:33.411620446 +0000 2024-03-23T09:40:06.888 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-03-23 09:37:33.411620446 +0000 2024-03-23T09:40:06.888 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T09:40:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-23T09:40:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-23T09:40:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.2747e-05 s, 5.5 MB/s 2024-03-23T09:40:06.953 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T09:40:07.008 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-03-23T09:40:07.063 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T09:40:07.064 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T09:40:07.064 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-23T09:40:07.064 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T09:40:07.064 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T09:40:07.064 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-23 09:39:39.510102962 +0000 2024-03-23T09:40:07.064 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-23 09:37:33.742613457 +0000 2024-03-23T09:40:07.064 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-23 09:37:33.742613457 +0000 2024-03-23T09:40:07.064 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-03-23 09:37:33.742613457 +0000 2024-03-23T09:40:07.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T09:40:07.129 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-23T09:40:07.129 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-23T09:40:07.129 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.3504e-05 s, 5.5 MB/s 2024-03-23T09:40:07.130 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T09:40:07.185 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-03-23T09:40:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T09:40:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T09:40:07.239 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-23T09:40:07.239 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T09:40:07.239 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T09:40:07.240 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-23 09:39:39.511102940 +0000 2024-03-23T09:40:07.240 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-23 09:37:34.081606299 +0000 2024-03-23T09:40:07.240 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-23 09:37:34.081606299 +0000 2024-03-23T09:40:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-03-23 09:37:34.081606299 +0000 2024-03-23T09:40:07.240 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T09:40:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-23T09:40:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-23T09:40:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.3057e-05 s, 5.5 MB/s 2024-03-23T09:40:07.306 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T09:40:07.361 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-03-23T09:40:07.415 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T09:40:07.415 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T09:40:07.415 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-23T09:40:07.415 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T09:40:07.415 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T09:40:07.415 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-23 09:39:39.511102940 +0000 2024-03-23T09:40:07.415 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-23 09:37:34.432598887 +0000 2024-03-23T09:40:07.415 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-23 09:37:34.432598887 +0000 2024-03-23T09:40:07.416 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-03-23 09:37:34.432598887 +0000 2024-03-23T09:40:07.416 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T09:40:07.480 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-23T09:40:07.480 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-23T09:40:07.480 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.2574e-05 s, 5.5 MB/s 2024-03-23T09:40:07.481 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T09:40:07.536 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T09:40:07.536 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:07.536 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T09:40:07.555 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T09:40:07.555 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-03-23T09:40:07.611 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T09:40:07.611 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T09:40:07.611 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-23T09:40:07.611 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T09:40:07.611 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T09:40:07.611 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-03-23 09:39:57.795273165 +0000 2024-03-23T09:40:07.612 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-03-23 09:37:32.417854388 +0000 2024-03-23T09:40:07.612 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-03-23 09:37:32.417854388 +0000 2024-03-23T09:40:07.612 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-03-23 09:37:32.417854388 +0000 2024-03-23T09:40:07.612 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T09:40:07.678 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-03-23T09:40:07.678 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-03-23T09:40:07.678 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000190634 s, 2.7 MB/s 2024-03-23T09:40:07.679 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T09:40:07.736 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-03-23T09:40:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T09:40:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T09:40:07.792 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-23T09:40:07.792 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T09:40:07.792 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T09:40:07.792 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-03-23 09:39:57.795273165 +0000 2024-03-23T09:40:07.792 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-03-23 09:37:32.757849132 +0000 2024-03-23T09:40:07.792 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-03-23 09:37:32.757849132 +0000 2024-03-23T09:40:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-03-23 09:37:32.757849132 +0000 2024-03-23T09:40:07.792 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T09:40:07.859 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-03-23T09:40:07.859 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-03-23T09:40:07.859 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000202749 s, 2.5 MB/s 2024-03-23T09:40:07.860 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T09:40:07.917 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-03-23T09:40:07.972 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T09:40:07.973 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T09:40:07.973 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-23T09:40:07.973 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T09:40:07.973 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T09:40:07.973 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-03-23 09:39:57.795273165 +0000 2024-03-23T09:40:07.973 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-03-23 09:37:33.089844019 +0000 2024-03-23T09:40:07.973 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-03-23 09:37:33.089844019 +0000 2024-03-23T09:40:07.973 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-03-23 09:37:33.089844019 +0000 2024-03-23T09:40:07.973 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T09:40:08.039 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-03-23T09:40:08.039 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-03-23T09:40:08.039 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000170002 s, 3.0 MB/s 2024-03-23T09:40:08.040 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T09:40:08.096 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-03-23T09:40:08.151 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T09:40:08.151 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T09:40:08.151 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-23T09:40:08.151 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T09:40:08.151 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T09:40:08.152 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-03-23 09:39:57.796273150 +0000 2024-03-23T09:40:08.152 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-03-23 09:37:33.438838646 +0000 2024-03-23T09:40:08.152 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-03-23 09:37:33.438838646 +0000 2024-03-23T09:40:08.152 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-03-23 09:37:33.438838646 +0000 2024-03-23T09:40:08.152 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T09:40:08.216 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-03-23T09:40:08.216 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-03-23T09:40:08.216 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000221708 s, 2.3 MB/s 2024-03-23T09:40:08.218 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T09:40:08.273 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T09:40:08.274 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-23T09:40:08.274 INFO:tasks.ceph:Generating config... 2024-03-23T09:40:08.275 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-23T09:40:08.275 INFO:tasks.ceph:[client] debug client = 20 2024-03-23T09:40:08.275 INFO:tasks.ceph:[client] debug ms = 1 2024-03-23T09:40:08.275 INFO:tasks.ceph:[client] pid file = 2024-03-23T09:40:08.275 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-23T09:40:08.275 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-23T09:40:08.276 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-23T09:40:08.277 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-23T09:40:08.277 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-23T09:40:08.277 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-23T09:40:08.277 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-23T09:40:08.277 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-23T09:40:08.277 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-23T09:40:08.277 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-23T09:40:08.277 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-23T09:40:08.277 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-23T09:40:08.277 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-23T09:40:08.277 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-23T09:40:08.277 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-23T09:40:08.277 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-23T09:40:08.278 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-23T09:40:08.278 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-23T09:40:08.278 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-23T09:40:08.278 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-23T09:40:08.278 INFO:tasks.ceph:Setting up mon.a... 2024-03-23T09:40:08.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-23T09:40:08.329 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.keyring 2024-03-23T09:40:08.332 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-23T09:40:08.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-23T09:40:08.403 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.47'), ('mon.b', '172.21.15.175'), ('mon.c', '[v2:172.21.15.175:3301,v1:172.21.15.175:6790]')] 2024-03-23T09:40:08.404 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-23T09:40:08.404 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:08.404 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-23T09:40:08.458 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.47 --add b 172.21.15.175 --addv c '[v2:172.21.15.175:3301,v1:172.21.15.175:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:08.542 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:08.543 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: generated fsid f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:40:08.543 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = pacific 2024-03-23T09:40:08.543 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 0 2024-03-23T09:40:08.543 INFO:teuthology.orchestra.run.smithi047.stdout:fsid f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:40:08.543 INFO:teuthology.orchestra.run.smithi047.stdout:last_changed 2024-03-23T09:40:08.544322+0000 2024-03-23T09:40:08.543 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-03-23T09:40:08.544322+0000 2024-03-23T09:40:08.543 INFO:teuthology.orchestra.run.smithi047.stdout:min_mon_release 16 (pacific) 2024-03-23T09:40:08.543 INFO:teuthology.orchestra.run.smithi047.stdout:election_strategy: 1 2024-03-23T09:40:08.543 INFO:teuthology.orchestra.run.smithi047.stdout:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-23T09:40:08.543 INFO:teuthology.orchestra.run.smithi047.stdout:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-03-23T09:40:08.543 INFO:teuthology.orchestra.run.smithi047.stdout:2: [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] mon.c 2024-03-23T09:40:08.543 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-23T09:40:08.544 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-23T09:40:08.559 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f31804be-685c-4b3b-b2be-155afa9f5d13... 2024-03-23T09:40:08.559 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T09:40:08.601 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T09:40:08.649 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-03-23T09:40:08.649 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2024-03-23T09:40:08.649 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T09:40:08.649 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2024-03-23T09:40:08.649 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.649 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2024-03-23T09:40:08.649 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.649 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2024-03-23T09:40:08.649 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.649 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2024-03-23T09:40:08.649 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2024-03-23T09:40:08.649 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.649 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2024-03-23T09:40:08.650 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2024-03-23T09:40:08.651 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock profile = high_recovery_ops 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2024-03-23T09:40:08.652 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2024-03-23T09:40:08.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: bdev async discard = True 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: bdev enable discard = True 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore allocator = bitmap 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 1/20 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 1/20 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 4/10 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-03-23T09:40:08.654 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2024-03-23T09:40:08.655 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: # 1m isn't quite enough 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: mon_down_mkfs_grace = 2m 2024-03-23T09:40:08.656 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_filestore_osds = false 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = "" 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2024-03-23T09:40:08.657 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds balancer = 20 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = True 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = True 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-03-23T09:40:08.658 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-03-23T09:40:08.659 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2024-03-23T09:40:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2024-03-23T09:40:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T09:40:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2024-03-23T09:40:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2024-03-23T09:40:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2024-03-23T09:40:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2024-03-23T09:40:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2024-03-23T09:40:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-03-23T09:40:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:40:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.662 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2024-03-23T09:40:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2024-03-23T09:40:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2024-03-23T09:40:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 600 2024-03-23T09:40:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock profile = high_recovery_ops 2024-03-23T09:40:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2024-03-23T09:40:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2024-03-23T09:40:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2024-03-23T09:40:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2024-03-23T09:40:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: bdev async discard = True 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: bdev enable discard = True 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore allocator = bitmap 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore block size = 96636764160 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore fsck on mount = True 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluefs = 1/20 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluestore = 1/20 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 20 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: debug rocksdb = 4/10 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd full ratio = 0.9 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd nearfull ratio = 0.8 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: osd failsafe full ratio = 0.95 2024-03-23T09:40:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = bluestore 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-03-23T09:40:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: # 1m isn't quite enough 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: mon_down_mkfs_grace = 2m 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_filestore_osds = false 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: mon op complaint time = 120 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: client mount timeout = 600 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: debug client = 20 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = "" 2024-03-23T09:40:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout:[mds] 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds = 20 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds balancer = 20 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug frag = True 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug scatterstat = True 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: mds op complaint time = 180 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: mds verify scatter = True 2024-03-23T09:40:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-03-23T09:40:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-03-23T09:40:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-03-23T09:40:08.670 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-23T09:40:08.670 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-23T09:40:08.764 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-23T09:40:08.765 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:08.765 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-23T09:40:08.779 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:08.779 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-23T09:40:08.833 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:40:08.833 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:08.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-23T09:40:08.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-23T09:40:08.912 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:08.912 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:08.967 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:40:08.967 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:08.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-23T09:40:08.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-23T09:40:09.007 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:09.007 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:09.062 INFO:tasks.ceph:Setting up mon nodes... 2024-03-23T09:40:09.062 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-23T09:40:09.062 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-23T09:40:09.116 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-23T09:40:09.118 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-23T09:40:09.183 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-23T09:40:09.186 INFO:tasks.ceph:Setting up mds nodes... 2024-03-23T09:40:09.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-23T09:40:09.240 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-23T09:40:09.242 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-23T09:40:09.268 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-23T09:40:09.362 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-23T09:40:09.364 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-23T09:40:09.391 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-23T09:40:09.445 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-23T09:40:09.447 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-23T09:40:09.473 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-23T09:40:09.568 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-23T09:40:09.570 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-23T09:40:09.597 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-23T09:40:09.597 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-23T09:40:09.639 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-23T09:40:09.654 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-23T09:40:09.735 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-23T09:40:09.750 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-23T09:40:09.792 INFO:teuthology.orchestra.run.smithi175.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-23T09:40:09.807 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-23T09:40:09.889 INFO:teuthology.orchestra.run.smithi175.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-23T09:40:09.905 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-23T09:40:09.905 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-23T09:40:09.905 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-23T09:40:09.932 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T09:40:09.932 INFO:tasks.ceph:role: osd.0 2024-03-23T09:40:09.932 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:40:09.932 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-23T09:40:09.998 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T09:40:09.998 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T09:40:09.999 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T09:40:09.999 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T09:40:09.999 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T09:40:09.999 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-03-23T09:40:09.999 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T09:40:09.999 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T09:40:09.999 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T09:40:09.999 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T09:40:10.030 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-03-23T09:40:10.031 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-03-23T09:40:10.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-23T09:40:10.102 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-23T09:40:10.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-23T09:40:10.239 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T09:40:10.239 INFO:tasks.ceph:role: osd.1 2024-03-23T09:40:10.239 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:40:10.239 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-23T09:40:10.305 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T09:40:10.305 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T09:40:10.305 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T09:40:10.305 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T09:40:10.305 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T09:40:10.305 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-03-23T09:40:10.305 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T09:40:10.305 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T09:40:10.305 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T09:40:10.305 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T09:40:10.338 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-03-23T09:40:10.340 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-03-23T09:40:10.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-23T09:40:10.415 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-23T09:40:10.485 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-23T09:40:10.551 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T09:40:10.551 INFO:tasks.ceph:role: osd.2 2024-03-23T09:40:10.552 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:40:10.552 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-23T09:40:10.617 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T09:40:10.617 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T09:40:10.617 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T09:40:10.617 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T09:40:10.617 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T09:40:10.617 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-03-23T09:40:10.617 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T09:40:10.617 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T09:40:10.617 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T09:40:10.617 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T09:40:10.649 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-03-23T09:40:10.651 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-03-23T09:40:10.651 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-23T09:40:10.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-23T09:40:10.791 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-23T09:40:10.858 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T09:40:10.858 INFO:tasks.ceph:role: osd.3 2024-03-23T09:40:10.858 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:40:10.858 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-23T09:40:10.924 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T09:40:10.924 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T09:40:10.924 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T09:40:10.924 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T09:40:10.924 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T09:40:10.924 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-03-23T09:40:10.924 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T09:40:10.924 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T09:40:10.925 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T09:40:10.925 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T09:40:10.956 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-03-23T09:40:10.958 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-03-23T09:40:10.958 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-23T09:40:11.028 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-23T09:40:11.098 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:11.200+0000 7fc4ec0494c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-23T09:40:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:11.200+0000 7fc4ec0494c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-23T09:40:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:11.200+0000 7fc4ec0494c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-23T09:40:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:11.200+0000 7fc4ec0494c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-23T09:40:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:11.200+0000 7fc4ec0494c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-23T09:40:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:11.201+0000 7fc4ec0494c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-23T09:40:14.041 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-23T09:40:14.072 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:14.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:14.163+0000 7ff37061c4c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-23T09:40:14.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:14.163+0000 7ff37061c4c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-23T09:40:14.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:14.163+0000 7ff37061c4c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-23T09:40:14.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:14.163+0000 7ff37061c4c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-23T09:40:14.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:14.163+0000 7ff37061c4c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-23T09:40:14.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:14.163+0000 7ff37061c4c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-23T09:40:17.011 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-23T09:40:17.042 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:17.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:17.131+0000 7f173271a4c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-23T09:40:17.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:17.131+0000 7f173271a4c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-23T09:40:17.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:17.131+0000 7f173271a4c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-23T09:40:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:17.131+0000 7f173271a4c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-23T09:40:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:17.131+0000 7f173271a4c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-23T09:40:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:17.132+0000 7f173271a4c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-23T09:40:19.984 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-23T09:40:20.016 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:20.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:20.106+0000 7f409e7bc4c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-23T09:40:20.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:20.107+0000 7f409e7bc4c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-23T09:40:20.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:20.107+0000 7f409e7bc4c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-23T09:40:20.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:20.107+0000 7f409e7bc4c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-23T09:40:20.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:20.107+0000 7f409e7bc4c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-23T09:40:20.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:20.107+0000 7f409e7bc4c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-23T09:40:22.963 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-23T09:40:22.994 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-23T09:40:23.026 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T09:40:23.026 INFO:tasks.ceph:role: osd.4 2024-03-23T09:40:23.026 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:40:23.026 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-23T09:40:23.105 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T09:40:23.105 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T09:40:23.105 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T09:40:23.105 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T09:40:23.105 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T09:40:23.105 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-03-23T09:40:23.105 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T09:40:23.105 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T09:40:23.106 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T09:40:23.106 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T09:40:23.599 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-03-23T09:40:23.601 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-03-23T09:40:23.602 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-23T09:40:23.737 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-23T09:40:23.773 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-23T09:40:23.842 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T09:40:23.842 INFO:tasks.ceph:role: osd.5 2024-03-23T09:40:23.842 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:40:23.842 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-23T09:40:23.917 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T09:40:23.917 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T09:40:23.918 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T09:40:23.918 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T09:40:23.918 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T09:40:23.918 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-03-23T09:40:23.918 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T09:40:23.918 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T09:40:23.918 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T09:40:23.918 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T09:40:24.376 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-03-23T09:40:24.378 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-03-23T09:40:24.379 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-23T09:40:24.479 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-23T09:40:24.512 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-23T09:40:24.582 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T09:40:24.582 INFO:tasks.ceph:role: osd.6 2024-03-23T09:40:24.582 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:40:24.582 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-23T09:40:24.658 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T09:40:24.658 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T09:40:24.658 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T09:40:24.659 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T09:40:24.659 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T09:40:24.659 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-03-23T09:40:24.659 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T09:40:24.659 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T09:40:24.659 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T09:40:24.659 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T09:40:25.158 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-03-23T09:40:25.160 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-03-23T09:40:25.161 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-23T09:40:25.270 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-23T09:40:25.304 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-23T09:40:25.373 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T09:40:25.373 INFO:tasks.ceph:role: osd.7 2024-03-23T09:40:25.373 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:40:25.373 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-23T09:40:25.449 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T09:40:25.449 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T09:40:25.449 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T09:40:25.449 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T09:40:25.449 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T09:40:25.449 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-03-23T09:40:25.449 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T09:40:25.449 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T09:40:25.449 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T09:40:25.450 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T09:40:25.970 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-03-23T09:40:25.973 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-03-23T09:40:25.973 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-23T09:40:26.097 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-23T09:40:26.132 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:26.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:26.228+0000 7f44a5a0d4c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-23T09:40:26.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:26.229+0000 7f44a5a0d4c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-23T09:40:26.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:26.229+0000 7f44a5a0d4c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-23T09:40:26.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:26.229+0000 7f44a5a0d4c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-23T09:40:26.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:26.229+0000 7f44a5a0d4c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-23T09:40:26.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:26.229+0000 7f44a5a0d4c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-23T09:40:29.079 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-23T09:40:29.109 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:29.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:29.198+0000 7f1cfc8d84c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-23T09:40:29.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:29.198+0000 7f1cfc8d84c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-23T09:40:29.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:29.198+0000 7f1cfc8d84c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-23T09:40:29.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:29.198+0000 7f1cfc8d84c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-23T09:40:29.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:29.198+0000 7f1cfc8d84c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-23T09:40:29.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:29.199+0000 7f1cfc8d84c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-23T09:40:32.048 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-23T09:40:32.079 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:32.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:32.170+0000 7f78228174c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-23T09:40:32.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:32.170+0000 7f78228174c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-23T09:40:32.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:32.170+0000 7f78228174c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-23T09:40:32.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:32.170+0000 7f78228174c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-23T09:40:32.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:32.170+0000 7f78228174c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-23T09:40:32.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:32.171+0000 7f78228174c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-23T09:40:35.025 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-23T09:40:35.057 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:35.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:35.148+0000 7f886171c4c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-23T09:40:35.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:35.148+0000 7f886171c4c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-23T09:40:35.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:35.148+0000 7f886171c4c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-23T09:40:35.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:35.148+0000 7f886171c4c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-23T09:40:35.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:35.148+0000 7f886171c4c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-23T09:40:35.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:35.149+0000 7f886171c4c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-23T09:40:36.700 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:40:36.703 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:40:37.998 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-23T09:40:38.029 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-23T09:40:38.029 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:38.030 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-23T09:40:38.056 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:38.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-23T09:40:38.122 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:38.123 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-23T09:40:38.188 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:38.189 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-23T09:40:38.254 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:38.254 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-23T09:40:38.320 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:38.320 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-23T09:40:38.386 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:38.387 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-23T09:40:38.453 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:38.453 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-23T09:40:38.484 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:38.484 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-23T09:40:38.554 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:38.554 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-23T09:40:38.623 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:38.623 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-23T09:40:38.692 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:38.692 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-23T09:40:38.760 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:38.760 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-23T09:40:38.827 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:38.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-23T09:40:38.894 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:38.894 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-23T09:40:38.909 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:38.909 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-23T09:40:38.965 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:38.965 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-23T09:40:38.980 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:38.980 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-23T09:40:39.034 INFO:tasks.ceph:Adding keys to all mons... 2024-03-23T09:40:39.035 DEBUG:teuthology.orchestra.run.smithi047:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-23T09:40:39.037 DEBUG:teuthology.orchestra.run.smithi175:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-23T09:40:39.078 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.y] 2024-03-23T09:40:39.078 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQB5o/5lywMFBxAAIVwjCe6B9EtBMAG2n7cZJw== 2024-03-23T09:40:39.078 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2024-03-23T09:40:39.078 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQB5o/5lo+xoDhAAyxqr1H6eiINwt2X7LUlrSg== 2024-03-23T09:40:39.078 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2024-03-23T09:40:39.078 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQB5o/5lFwyzFRAAKldMOA7oSblA8YMrvntY8w== 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.0] 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQB7o/5lLvoADBAAgdixkm+jHwQilYw0IAn6EA== 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.1] 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQB+o/5lOBfGCRAApglzalO1ztnJzOKg/UAyWA== 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.2] 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCBo/5lRBjlBxAAd45w8/kTMPqxmNyfCS3XYA== 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.3] 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCEo/5l+JxrBhAAbR/XqDyPDnN8eLpmvBZ7Mg== 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.x] 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQB5o/5l/0EOCxAAIrgIXRiFSQ1G7Jat5hNm4A== 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQB5o/5lZ5OlGhAAuUElzRghNRFGKREdCpxKvA== 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2024-03-23T09:40:39.079 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQB5o/5lpiD3IRAApXAdUQ4AkxWHzsFV0qcBng== 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.4] 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCKo/5ltsqwDRAAkqcZ/sEf2oozWFN4AXg/+w== 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.5] 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCNo/5lpLHiCxAAB2axUv9gjcYVlij9f+AEHg== 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.6] 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCQo/5l2io3ChAAu+lfYcWWBQf43aYzWjCBNg== 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.7] 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCTo/5lA7jjCBAAjJgBkp89H8lx6fidl/Z7IA== 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQB5o/5lZFc3JhAAZWmDZOnkW9eo9hqEDYynQw== 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout:[client.2] 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQB5o/5l1+DyKxAAZLKZNbUpUJ0fWPIYNvWK+g== 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout:[client.1] 2024-03-23T09:40:39.080 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQB5o/5lIrJSLxAAPJer3ZP3h8VkImkiJmnKVw== 2024-03-23T09:40:39.081 INFO:teuthology.orchestra.run.smithi047.stdout:[client.3] 2024-03-23T09:40:39.081 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQB5o/5lWqkiNRAAL1q0lEcM/NKW3Pf9LcP21g== 2024-03-23T09:40:39.100 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.y] 2024-03-23T09:40:39.100 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5o/5lywMFBxAAIVwjCe6B9EtBMAG2n7cZJw== 2024-03-23T09:40:39.100 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-03-23T09:40:39.100 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5o/5lo+xoDhAAyxqr1H6eiINwt2X7LUlrSg== 2024-03-23T09:40:39.100 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-03-23T09:40:39.100 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5o/5lFwyzFRAAKldMOA7oSblA8YMrvntY8w== 2024-03-23T09:40:39.100 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.0] 2024-03-23T09:40:39.100 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB7o/5lLvoADBAAgdixkm+jHwQilYw0IAn6EA== 2024-03-23T09:40:39.100 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.1] 2024-03-23T09:40:39.100 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB+o/5lOBfGCRAApglzalO1ztnJzOKg/UAyWA== 2024-03-23T09:40:39.100 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.2] 2024-03-23T09:40:39.100 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCBo/5lRBjlBxAAd45w8/kTMPqxmNyfCS3XYA== 2024-03-23T09:40:39.100 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.3] 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCEo/5l+JxrBhAAbR/XqDyPDnN8eLpmvBZ7Mg== 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.x] 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5o/5l/0EOCxAAIrgIXRiFSQ1G7Jat5hNm4A== 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5o/5lZ5OlGhAAuUElzRghNRFGKREdCpxKvA== 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5o/5lpiD3IRAApXAdUQ4AkxWHzsFV0qcBng== 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.4] 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCKo/5ltsqwDRAAkqcZ/sEf2oozWFN4AXg/+w== 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.5] 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCNo/5lpLHiCxAAB2axUv9gjcYVlij9f+AEHg== 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.6] 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCQo/5l2io3ChAAu+lfYcWWBQf43aYzWjCBNg== 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.7] 2024-03-23T09:40:39.101 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCTo/5lA7jjCBAAjJgBkp89H8lx6fidl/Z7IA== 2024-03-23T09:40:39.102 INFO:teuthology.orchestra.run.smithi175.stdout:[client.0] 2024-03-23T09:40:39.102 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5o/5lZFc3JhAAZWmDZOnkW9eo9hqEDYynQw== 2024-03-23T09:40:39.102 INFO:teuthology.orchestra.run.smithi175.stdout:[client.2] 2024-03-23T09:40:39.102 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5o/5l1+DyKxAAZLKZNbUpUJ0fWPIYNvWK+g== 2024-03-23T09:40:39.102 INFO:teuthology.orchestra.run.smithi175.stdout:[client.1] 2024-03-23T09:40:39.102 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5o/5lIrJSLxAAPJer3ZP3h8VkImkiJmnKVw== 2024-03-23T09:40:39.102 INFO:teuthology.orchestra.run.smithi175.stdout:[client.3] 2024-03-23T09:40:39.102 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5o/5lWqkiNRAAL1q0lEcM/NKW3Pf9LcP21g== 2024-03-23T09:40:39.103 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-23T09:40:39.124 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-23T09:40:39.194 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T09:40:39.197 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T09:40:39.251 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T09:40:39.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T09:40:39.307 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.363 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.414 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.417 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.468 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.470 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.521 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-23T09:40:39.524 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-23T09:40:39.575 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T09:40:39.577 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T09:40:39.632 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T09:40:39.634 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T09:40:39.688 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.690 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.742 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.744 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.796 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.799 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.850 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.853 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T09:40:39.905 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T09:40:39.907 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T09:40:39.961 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T09:40:39.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T09:40:40.018 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T09:40:40.020 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T09:40:40.075 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T09:40:40.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T09:40:40.131 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-23T09:40:40.131 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-23T09:40:40.157 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-23T09:40:40.253 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-23T09:40:40.279 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-23T09:40:40.305 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-23T09:40:40.402 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-23T09:40:40.429 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-23T09:40:40.496 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-23T09:40:40.593 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-23T09:40:40.620 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:40.622 DEBUG:teuthology.orchestra.run.smithi175:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-23T09:40:40.674 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-23T09:40:40.675 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-23T09:40:40.675 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-23T09:40:40.677 INFO:tasks.ceph.mon.a:Started 2024-03-23T09:40:40.677 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-23T09:40:40.678 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-23T09:40:40.718 INFO:tasks.ceph.mon.b:Started 2024-03-23T09:40:40.718 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-23T09:40:40.718 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-23T09:40:40.720 INFO:tasks.ceph.mon.c:Started 2024-03-23T09:40:40.720 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-23T09:40:40.720 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-23T09:40:40.720 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-23T09:40:40.722 INFO:tasks.ceph.mgr.y:Started 2024-03-23T09:40:40.722 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-23T09:40:40.722 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-23T09:40:40.724 INFO:tasks.ceph.mgr.x:Started 2024-03-23T09:40:40.725 INFO:tasks.ceph:Setting crush tunables to default 2024-03-23T09:40:40.725 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd crush tunables default 2024-03-23T09:40:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.825+0000 7f01fb355640 1 Processor -- start 2024-03-23T09:40:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.826+0000 7f01fb355640 1 -- start start 2024-03-23T09:40:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.826+0000 7f01fb355640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.826+0000 7f01fb355640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.826+0000 7f01fb355640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f4160d10 con 0x7f01f415c8b0 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.826+0000 7f01fb355640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f406a5c0 con 0x7f01f415cc90 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.826+0000 7f01fb355640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f41606c0 con 0x7f01f40aeb70 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.826+0000 7f01f90f1640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f01f415c8b0 0x7f01f4069ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58044/0 (socket says 172.21.15.47:58044) 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.826+0000 7f01f88f0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.826+0000 7f01f90f1640 1 -- 172.21.15.47:0/664876770 learned_addr learned my addr 172.21.15.47:0/664876770 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.826+0000 7f01f88f0640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 msgr2=0x7f01f41611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.826+0000 7f01f88f0640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 msgr2=0x7f01f41611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.826+0000 7f01f88f0640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.827+0000 7f01f88f0640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.827+0000 7f01f98f2640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.827+0000 7f01f98f2640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 msgr2=0x7f01f40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.827+0000 7f01f98f2640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 msgr2=0x7f01f40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T09:40:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.827+0000 7f01f98f2640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T09:40:40.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:40.827+0000 7f01f98f2640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T09:40:41.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.028+0000 7f01f88f0640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:41.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.028+0000 7f01f88f0640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 msgr2=0x7f01f41611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-23T09:40:41.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.028+0000 7f01f88f0640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 msgr2=0x7f01f41611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T09:40:41.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.028+0000 7f01f88f0640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T09:40:41.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.028+0000 7f01f88f0640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T09:40:41.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.028+0000 7f01f98f2640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:41.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.028+0000 7f01f98f2640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 msgr2=0x7f01f40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-23T09:40:41.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.028+0000 7f01f98f2640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 msgr2=0x7f01f40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T09:40:41.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.028+0000 7f01f98f2640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T09:40:41.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.028+0000 7f01f98f2640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T09:40:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.429+0000 7f01f88f0640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.429+0000 7f01f88f0640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 msgr2=0x7f01f41611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-23T09:40:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.429+0000 7f01f88f0640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 msgr2=0x7f01f41611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T09:40:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.429+0000 7f01f88f0640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T09:40:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.429+0000 7f01f88f0640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T09:40:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.429+0000 7f01f98f2640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.429+0000 7f01f98f2640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 msgr2=0x7f01f40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-23T09:40:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.429+0000 7f01f98f2640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 msgr2=0x7f01f40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T09:40:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.429+0000 7f01f98f2640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T09:40:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:41.429+0000 7f01f98f2640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T09:40:42.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:42.230+0000 7f01f88f0640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:42.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:42.230+0000 7f01f88f0640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 msgr2=0x7f01f41611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-23T09:40:42.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:42.230+0000 7f01f88f0640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 msgr2=0x7f01f41611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T09:40:42.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:42.230+0000 7f01f88f0640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T09:40:42.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:42.230+0000 7f01f88f0640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T09:40:42.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:42.230+0000 7f01f98f2640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:42.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:42.231+0000 7f01f98f2640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 msgr2=0x7f01f40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-23T09:40:42.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:42.231+0000 7f01f98f2640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 msgr2=0x7f01f40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T09:40:42.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:42.231+0000 7f01f98f2640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T09:40:42.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:42.231+0000 7f01f98f2640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T09:40:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:43.826+0000 7f01eaffd640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 msgr2=0x7f01f40ce850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T09:40:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:43.826+0000 7f01eaffd640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.175:3300/0 conn(0x7f01f40aeb70 0x7f01f40ce850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:43.826+0000 7f01eaffd640 1 -- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 msgr2=0x7f01f41611a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T09:40:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:43.826+0000 7f01eaffd640 1 --2- 172.21.15.47:0/664876770 >> v2:172.21.15.47:3300/0 conn(0x7f01f415cc90 0x7f01f41611a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:43.826+0000 7f01eaffd640 1 -- 172.21.15.47:0/664876770 >> v1:172.21.15.47:6789/0 conn(0x7f01f415c8b0 legacy=0x7f01f4069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:43.827+0000 7f01eaffd640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:43.827+0000 7f01eaffd640 1 -- 172.21.15.47:0/664876770 --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f41609a0 con 0x7f01d8006170 2024-03-23T09:40:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:43.827+0000 7f01eaffd640 1 -- 172.21.15.47:0/664876770 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f406a5c0 con 0x7f01d80036b0 2024-03-23T09:40:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:43.827+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:43.827+0000 7f01f90f1640 1 -- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 msgr2=0x7f01d8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-23T09:40:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:43.827+0000 7f01f90f1640 1 -- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 msgr2=0x7f01d8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T09:40:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:43.827+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T09:40:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:43.827+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T09:40:44.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:44.028+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:44.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:44.028+0000 7f01f90f1640 1 -- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 msgr2=0x7f01d8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-23T09:40:44.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:44.028+0000 7f01f90f1640 1 -- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 msgr2=0x7f01d8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T09:40:44.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:44.028+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T09:40:44.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:44.028+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T09:40:44.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:44.429+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:44.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:44.429+0000 7f01f90f1640 1 -- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 msgr2=0x7f01d8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-23T09:40:44.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:44.429+0000 7f01f90f1640 1 -- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 msgr2=0x7f01d8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T09:40:44.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:44.429+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T09:40:44.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:44.429+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T09:40:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:45.230+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:45.230+0000 7f01f90f1640 1 -- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 msgr2=0x7f01d8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-23T09:40:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:45.230+0000 7f01f90f1640 1 -- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 msgr2=0x7f01d8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T09:40:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:45.230+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T09:40:45.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:45.230+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T09:40:46.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.040+0000 7f01ebfff640 1 -- 172.21.15.47:0/664876770 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982276094 0 0) 0x7f01f41609a0 con 0x7f01d8006170 2024-03-23T09:40:46.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.040+0000 7f01ebfff640 1 -- 172.21.15.47:0/664876770 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01e0003520 con 0x7f01d8006170 2024-03-23T09:40:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01ebfff640 1 -- 172.21.15.47:0/664876770 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4091616654 0 0) 0x7f01e0003520 con 0x7f01d8006170 2024-03-23T09:40:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01ebfff640 1 -- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 msgr2=0x7f01d8005ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T09:40:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01ebfff640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01d80036b0 0x7f01d8005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01ebfff640 1 -- 172.21.15.47:0/664876770 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d800af50 con 0x7f01d8006170 2024-03-23T09:40:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01ebfff640 1 -- 172.21.15.47:0/664876770 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f01e400d580 con 0x7f01d8006170 2024-03-23T09:40:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01ebfff640 1 -- 172.21.15.47:0/664876770 >> v1:172.21.15.175:6789/0 conn(0x7f01d8006170 legacy=0x7f01d8008610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01f98f2640 1 -- 172.21.15.47:0/664876770 reap_dead start 2024-03-23T09:40:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01ebfff640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e0003970 0x7f01e0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01ebfff640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01e00047c0 0x7f01e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01ebfff640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e00052a0 0x7f01e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01ebfff640 1 -- 172.21.15.47:0/664876770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f406a5c0 con 0x7f01e00047c0 2024-03-23T09:40:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01ebfff640 1 -- 172.21.15.47:0/664876770 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e000c1f0 con 0x7f01e0003970 2024-03-23T09:40:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01ebfff640 1 -- 172.21.15.47:0/664876770 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e000c3b0 con 0x7f01e00052a0 2024-03-23T09:40:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e0003970 0x7f01e0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01f88f0640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01e00047c0 0x7f01e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.041+0000 7f01f98f2640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e00052a0 0x7f01e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.042+0000 7f01f90f1640 1 -- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e00052a0 msgr2=0x7f01e000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:40:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.042+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e00052a0 0x7f01e000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.042+0000 7f01f90f1640 1 -- 172.21.15.47:0/664876770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01e00047c0 msgr2=0x7f01e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:40:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.042+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01e00047c0 0x7f01e0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.042+0000 7f01f90f1640 1 -- 172.21.15.47:0/664876770 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e0003520 con 0x7f01e0003970 2024-03-23T09:40:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.042+0000 7f01f88f0640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01e00047c0 0x7f01e0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:40:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.043+0000 7f01f90f1640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e0003970 0x7f01e0003d70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01f0007fd0 tx=0x7f01e00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13f2f99699f47f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:46.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.047+0000 7f01ebfff640 1 -- 172.21.15.47:0/664876770 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0035410 con 0x7f01e0003970 2024-03-23T09:40:46.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.047+0000 7f01ebfff640 1 -- 172.21.15.47:0/664876770 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f002f030 con 0x7f01e0003970 2024-03-23T09:40:46.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.047+0000 7f01ebfff640 1 -- 172.21.15.47:0/664876770 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f00357d0 con 0x7f01e0003970 2024-03-23T09:40:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.048+0000 7f01fb355640 1 -- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e0003970 msgr2=0x7f01e0003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.048+0000 7f01fb355640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e0003970 0x7f01e0003d70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01f0007fd0 tx=0x7f01e00057f0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.048+0000 7f01fb355640 1 -- 172.21.15.47:0/664876770 shutdown_connections 2024-03-23T09:40:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.048+0000 7f01fb355640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e00052a0 0x7f01e000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.048+0000 7f01fb355640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01e00047c0 0x7f01e0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.048+0000 7f01fb355640 1 --2- 172.21.15.47:0/664876770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e0003970 0x7f01e0003d70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.048+0000 7f01fb355640 1 -- 172.21.15.47:0/664876770 >> 172.21.15.47:0/664876770 conn(0x7f01f4098a00 msgr2=0x7f01f40cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.049+0000 7f01fb355640 1 -- 172.21.15.47:0/664876770 shutdown_connections 2024-03-23T09:40:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.049+0000 7f01fb355640 1 -- 172.21.15.47:0/664876770 wait complete. 2024-03-23T09:40:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.050+0000 7f01fb355640 1 Processor -- start 2024-03-23T09:40:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.050+0000 7f01fb355640 1 -- start start 2024-03-23T09:40:46.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.051+0000 7f01fb355640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e0003970 0x7f01f40b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.051+0000 7f01fb355640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e00047c0 0x7f01f415d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.051+0000 7f01fb355640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01e00052a0 0x7f01f414c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.051+0000 7f01fb355640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40b3c50 con 0x7f01e00052a0 2024-03-23T09:40:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.051+0000 7f01fb355640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4157f80 con 0x7f01e0003970 2024-03-23T09:40:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.051+0000 7f01fb355640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40b2fd0 con 0x7f01e00047c0 2024-03-23T09:40:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.051+0000 7f01f98f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01e00052a0 0x7f01f414c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.051+0000 7f01f98f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01e00052a0 0x7f01f414c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36284/0 (socket says 172.21.15.47:36284) 2024-03-23T09:40:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.051+0000 7f01f88f0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e00047c0 0x7f01f415d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.051+0000 7f01f98f2640 1 -- 172.21.15.47:0/2706339820 learned_addr learned my addr 172.21.15.47:0/2706339820 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:40:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.051+0000 7f01f90f1640 1 --2- 172.21.15.47:0/2706339820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e0003970 0x7f01f40b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.052+0000 7f01f98f2640 1 -- 172.21.15.47:0/2706339820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e00047c0 msgr2=0x7f01f415d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.052+0000 7f01f98f2640 1 --2- 172.21.15.47:0/2706339820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e00047c0 0x7f01f415d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.052+0000 7f01f98f2640 1 -- 172.21.15.47:0/2706339820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e0003970 msgr2=0x7f01f40b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.052+0000 7f01f98f2640 1 --2- 172.21.15.47:0/2706339820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e0003970 0x7f01f40b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.052+0000 7f01f98f2640 1 -- 172.21.15.47:0/2706339820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40b32b0 con 0x7f01e00052a0 2024-03-23T09:40:46.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.052+0000 7f01f98f2640 1 --2- 172.21.15.47:0/2706339820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01e00052a0 0x7f01f414c5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009610 tx=0x7f01ec015700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:46.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.053+0000 7f01e9ffb640 1 -- 172.21.15.47:0/2706339820 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec009ca0 con 0x7f01e00052a0 2024-03-23T09:40:46.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.053+0000 7f01fb355640 1 -- 172.21.15.47:0/2706339820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f40b3590 con 0x7f01e00052a0 2024-03-23T09:40:46.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.053+0000 7f01e9ffb640 1 -- 172.21.15.47:0/2706339820 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec00f040 con 0x7f01e00052a0 2024-03-23T09:40:46.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.053+0000 7f01e9ffb640 1 -- 172.21.15.47:0/2706339820 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec021410 con 0x7f01e00052a0 2024-03-23T09:40:46.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.054+0000 7f01fb355640 1 -- 172.21.15.47:0/2706339820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f40b2bf0 con 0x7f01e00052a0 2024-03-23T09:40:46.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.055+0000 7f01fb355640 1 -- 172.21.15.47:0/2706339820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f40b3d90 con 0x7f01e00052a0 2024-03-23T09:40:46.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.055+0000 7f01e9ffb640 1 -- 172.21.15.47:0/2706339820 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f01ec021910 con 0x7f01e00052a0 2024-03-23T09:40:46.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.055+0000 7f01e9ffb640 1 -- 172.21.15.47:0/2706339820 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f01ec017d60 con 0x7f01e00052a0 2024-03-23T09:40:46.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.058+0000 7f01e9ffb640 1 -- 172.21.15.47:0/2706339820 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f01f40b3d90 con 0x7f01e00052a0 2024-03-23T09:40:46.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.126+0000 7f01fb355640 1 -- 172.21.15.47:0/2706339820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f01f40672f0 con 0x7f01e00052a0 2024-03-23T09:40:46.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.128+0000 7f01e9ffb640 1 -- 172.21.15.47:0/2706339820 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f01ec01c030 con 0x7f01e00052a0 2024-03-23T09:40:46.128 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-03-23T09:40:46.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.129+0000 7f01fb355640 1 -- 172.21.15.47:0/2706339820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01e00052a0 msgr2=0x7f01f414c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.129+0000 7f01fb355640 1 --2- 172.21.15.47:0/2706339820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01e00052a0 0x7f01f414c5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009610 tx=0x7f01ec015700 comp rx=0 tx=0).stop 2024-03-23T09:40:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.129+0000 7f01fb355640 1 -- 172.21.15.47:0/2706339820 shutdown_connections 2024-03-23T09:40:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.129+0000 7f01fb355640 1 --2- 172.21.15.47:0/2706339820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01e00052a0 0x7f01f414c5b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.129+0000 7f01fb355640 1 --2- 172.21.15.47:0/2706339820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e00047c0 0x7f01f415d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.129+0000 7f01fb355640 1 --2- 172.21.15.47:0/2706339820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e0003970 0x7f01f40b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.129+0000 7f01fb355640 1 -- 172.21.15.47:0/2706339820 >> 172.21.15.47:0/2706339820 conn(0x7f01f4098a00 msgr2=0x7f01f415bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.129+0000 7f01fb355640 1 -- 172.21.15.47:0/2706339820 shutdown_connections 2024-03-23T09:40:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:46.130+0000 7f01fb355640 1 -- 172.21.15.47:0/2706339820 wait complete. 2024-03-23T09:40:46.140 INFO:tasks.ceph:check_enable_crimson: False 2024-03-23T09:40:46.140 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-23T09:40:46.140 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:46.141 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-23T09:40:46.168 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:46.168 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-23T09:40:46.235 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:46.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-23T09:40:46.302 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:40:46.302 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-23T09:40:46.370 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:46.370 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-23T09:40:46.397 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:46.397 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-23T09:40:46.464 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:46.464 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-23T09:40:46.508 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:46.508+0000 7fa059e3e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T09:40:46.524 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:46.525+0000 7ff054a10040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T09:40:46.531 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:40:46.531 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-23T09:40:46.563 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:46.564+0000 7fa059e3e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T09:40:46.579 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:46.580+0000 7ff054a10040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T09:40:46.589 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b 0 2024-03-23T09:40:46.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.730+0000 7f7f2eb06640 1 Processor -- start 2024-03-23T09:40:46.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.730+0000 7f7f2eb06640 1 -- start start 2024-03-23T09:40:46.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.731+0000 7f7f2eb06640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f2815a610 0x7f7f2815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.731+0000 7f7f2eb06640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2806ae60 con 0x7f7f280cd020 2024-03-23T09:40:46.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.731+0000 7f7f2eb06640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2806a250 con 0x7f7f28069ce0 2024-03-23T09:40:46.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.731+0000 7f7f2eb06640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2815a9f0 con 0x7f7f2815a610 2024-03-23T09:40:46.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.731+0000 7f7f27fff640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f7f28069ce0 0x7f7f280cc910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:38100/0 (socket says 172.21.15.175:38100) 2024-03-23T09:40:46.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.731+0000 7f7f27fff640 1 -- 172.21.15.175:0/3992451274 learned_addr learned my addr 172.21.15.175:0/3992451274 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-03-23T09:40:46.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.731+0000 7f7f2c8a2640 1 --2- 172.21.15.175:0/3992451274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f2815a610 0x7f7f2815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.732+0000 7f7f277fe640 1 -- 172.21.15.175:0/3992451274 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443477752 0 0) 0x7f7f2806a250 con 0x7f7f28069ce0 2024-03-23T09:40:46.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.732+0000 7f7f277fe640 1 -- 172.21.15.175:0/3992451274 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f18003520 con 0x7f7f28069ce0 2024-03-23T09:40:46.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.732+0000 7f7f2c8a2640 1 -- 172.21.15.175:0/3992451274 >> v1:172.21.15.175:6789/0 conn(0x7f7f28069ce0 legacy=0x7f7f280cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.732+0000 7f7f2c8a2640 1 -- 172.21.15.175:0/3992451274 >> v1:172.21.15.47:6789/0 conn(0x7f7f280cd020 legacy=0x7f7f2806c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.732+0000 7f7f2c8a2640 1 -- 172.21.15.175:0/3992451274 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2806a530 con 0x7f7f2815a610 2024-03-23T09:40:46.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.732+0000 7f7f2c8a2640 1 --2- 172.21.15.175:0/3992451274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f2815a610 0x7f7f2815cfa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0097d0 tx=0x7f7f1c02b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=156bf291ee552bd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:46.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.733+0000 7f7f277fe640 1 -- 172.21.15.175:0/3992451274 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c031770 con 0x7f7f2815a610 2024-03-23T09:40:46.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.733+0000 7f7f277fe640 1 -- 172.21.15.175:0/3992451274 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c0318d0 con 0x7f7f2815a610 2024-03-23T09:40:46.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.733+0000 7f7f277fe640 1 -- 172.21.15.175:0/3992451274 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c031c60 con 0x7f7f2815a610 2024-03-23T09:40:46.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.733+0000 7f7f2eb06640 1 -- 172.21.15.175:0/3992451274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f2815a610 msgr2=0x7f7f2815cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.733+0000 7f7f2eb06640 1 --2- 172.21.15.175:0/3992451274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f2815a610 0x7f7f2815cfa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0097d0 tx=0x7f7f1c02b810 comp rx=0 tx=0).stop 2024-03-23T09:40:46.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.733+0000 7f7f2eb06640 1 -- 172.21.15.175:0/3992451274 shutdown_connections 2024-03-23T09:40:46.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.733+0000 7f7f2eb06640 1 --2- 172.21.15.175:0/3992451274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f2815a610 0x7f7f2815cfa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.733+0000 7f7f2eb06640 1 -- 172.21.15.175:0/3992451274 >> 172.21.15.175:0/3992451274 conn(0x7f7f280cb7c0 msgr2=0x7f7f280cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:46.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.733+0000 7f7f2eb06640 1 -- 172.21.15.175:0/3992451274 shutdown_connections 2024-03-23T09:40:46.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.733+0000 7f7f2eb06640 1 -- 172.21.15.175:0/3992451274 wait complete. 2024-03-23T09:40:46.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.734+0000 7f7f2eb06640 1 Processor -- start 2024-03-23T09:40:46.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.734+0000 7f7f2eb06640 1 -- start start 2024-03-23T09:40:46.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.735+0000 7f7f2eb06640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f28069ce0 0x7f7f280c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.735+0000 7f7f2eb06640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f280cd020 0x7f7f28146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.735+0000 7f7f2eb06640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f2815a610 0x7f7f281467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.735+0000 7f7f2eb06640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f280c6a30 con 0x7f7f28069ce0 2024-03-23T09:40:46.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.735+0000 7f7f2eb06640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f281611d0 con 0x7f7f2815a610 2024-03-23T09:40:46.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.735+0000 7f7f2eb06640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f28146cf0 con 0x7f7f280cd020 2024-03-23T09:40:46.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.735+0000 7f7f2d0a3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f2815a610 0x7f7f281467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.735+0000 7f7f27fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f280cd020 0x7f7f28146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.735+0000 7f7f2d0a3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f2815a610 0x7f7f281467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:49976/0 (socket says 172.21.15.175:49976) 2024-03-23T09:40:46.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.735+0000 7f7f2d0a3640 1 -- 172.21.15.175:0/4090228468 learned_addr learned my addr 172.21.15.175:0/4090228468 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.735+0000 7f7f2c8a2640 1 --2- 172.21.15.175:0/4090228468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f28069ce0 0x7f7f280c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f2d0a3640 1 -- 172.21.15.175:0/4090228468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f280cd020 msgr2=0x7f7f28146270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f2d0a3640 1 --2- 172.21.15.175:0/4090228468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f280cd020 0x7f7f28146270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f2d0a3640 1 -- 172.21.15.175:0/4090228468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f28069ce0 msgr2=0x7f7f280c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f2d0a3640 1 --2- 172.21.15.175:0/4090228468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f28069ce0 0x7f7f280c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f2d0a3640 1 -- 172.21.15.175:0/4090228468 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c009400 con 0x7f7f2815a610 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f27fff640 1 --2- 172.21.15.175:0/4090228468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f280cd020 0x7f7f28146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f2d0a3640 1 --2- 172.21.15.175:0/4090228468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f2815a610 0x7f7f281467b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c7d0 tx=0x7f7f2000cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f257fa640 1 -- 172.21.15.175:0/4090228468 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20003b70 con 0x7f7f2815a610 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f257fa640 1 -- 172.21.15.175:0/4090228468 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f20003cf0 con 0x7f7f2815a610 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f2eb06640 1 -- 172.21.15.175:0/4090228468 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f28160ef0 con 0x7f7f2815a610 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f257fa640 1 -- 172.21.15.175:0/4090228468 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2000aa90 con 0x7f7f2815a610 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f2eb06640 1 -- 172.21.15.175:0/4090228468 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f280c6cd0 con 0x7f7f2815a610 2024-03-23T09:40:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f257fa640 1 -- 172.21.15.175:0/4090228468 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7f200050d0 con 0x7f7f2815a610 2024-03-23T09:40:46.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.736+0000 7f7f257fa640 1 -- 172.21.15.175:0/4090228468 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7f20012880 con 0x7f7f2815a610 2024-03-23T09:40:46.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.737+0000 7f7f2eb06640 1 -- 172.21.15.175:0/4090228468 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f2806e970 con 0x7f7f2815a610 2024-03-23T09:40:46.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.739+0000 7f7f257fa640 1 -- 172.21.15.175:0/4090228468 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f7f20005560 con 0x7f7f2815a610 2024-03-23T09:40:46.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.810+0000 7f7f2eb06640 1 -- 172.21.15.175:0/4090228468 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b", "id": 0} v 0) v1 -- 0x7f7f2806de30 con 0x7f7f2815a610 2024-03-23T09:40:46.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.813+0000 7f7f257fa640 1 -- 172.21.15.175:0/4090228468 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7f20005700 con 0x7f7f2815a610 2024-03-23T09:40:46.812 INFO:teuthology.orchestra.run.smithi175.stdout:0 2024-03-23T09:40:46.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.814+0000 7f7f2eb06640 1 -- 172.21.15.175:0/4090228468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f2815a610 msgr2=0x7f7f281467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.814+0000 7f7f2eb06640 1 --2- 172.21.15.175:0/4090228468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f2815a610 0x7f7f281467b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c7d0 tx=0x7f7f2000cc90 comp rx=0 tx=0).stop 2024-03-23T09:40:46.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.814+0000 7f7f2eb06640 1 -- 172.21.15.175:0/4090228468 shutdown_connections 2024-03-23T09:40:46.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.814+0000 7f7f2eb06640 1 --2- 172.21.15.175:0/4090228468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f2815a610 0x7f7f281467b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.814+0000 7f7f2eb06640 1 --2- 172.21.15.175:0/4090228468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f280cd020 0x7f7f28146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.814+0000 7f7f2eb06640 1 --2- 172.21.15.175:0/4090228468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f28069ce0 0x7f7f280c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.814+0000 7f7f2eb06640 1 -- 172.21.15.175:0/4090228468 >> 172.21.15.175:0/4090228468 conn(0x7f7f280cb7c0 msgr2=0x7f7f280ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:46.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.814+0000 7f7f2eb06640 1 -- 172.21.15.175:0/4090228468 shutdown_connections 2024-03-23T09:40:46.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.814+0000 7f7f2eb06640 1 -- 172.21.15.175:0/4090228468 wait complete. 2024-03-23T09:40:46.824 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 49f7a329-d653-43d1-9e27-54b0e93f2eec 1 2024-03-23T09:40:46.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.922+0000 7f4e4c248640 1 Processor -- start 2024-03-23T09:40:46.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.922+0000 7f4e4c248640 1 -- start start 2024-03-23T09:40:46.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.923+0000 7f4e4c248640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e4415d0c0 0x7f4e4415d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.923+0000 7f4e4c248640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e441575c0 con 0x7f4e440c9dc0 2024-03-23T09:40:46.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.923+0000 7f4e4c248640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4415dbf0 con 0x7f4e441571e0 2024-03-23T09:40:46.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.923+0000 7f4e4c248640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4406f190 con 0x7f4e4415d0c0 2024-03-23T09:40:46.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.923+0000 7f4e49fe4640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f4e441571e0 0x7f4e4406f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:38114/0 (socket says 172.21.15.175:38114) 2024-03-23T09:40:46.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.923+0000 7f4e49fe4640 1 -- 172.21.15.175:0/4249860256 learned_addr learned my addr 172.21.15.175:0/4249860256 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-03-23T09:40:46.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.923+0000 7f4e497e3640 1 --2- 172.21.15.175:0/4249860256 >> v2:172.21.15.47:3300/0 conn(0x7f4e4415d0c0 0x7f4e4415d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.923+0000 7f4e48fe2640 1 -- 172.21.15.175:0/4249860256 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843931822 0 0) 0x7f4e4415dbf0 con 0x7f4e441571e0 2024-03-23T09:40:46.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.923+0000 7f4e48fe2640 1 -- 172.21.15.175:0/4249860256 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e30003520 con 0x7f4e441571e0 2024-03-23T09:40:46.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.923+0000 7f4e48fe2640 1 -- 172.21.15.175:0/4249860256 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287709062 0 0) 0x7f4e441575c0 con 0x7f4e440c9dc0 2024-03-23T09:40:46.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.923+0000 7f4e48fe2640 1 -- 172.21.15.175:0/4249860256 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e4415dbf0 con 0x7f4e440c9dc0 2024-03-23T09:40:46.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.923+0000 7f4e497e3640 1 -- 172.21.15.175:0/4249860256 >> v1:172.21.15.175:6789/0 conn(0x7f4e441571e0 legacy=0x7f4e4406f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.924+0000 7f4e48fe2640 1 -- 172.21.15.175:0/4249860256 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2202357797 0 0) 0x7f4e30003520 con 0x7f4e441571e0 2024-03-23T09:40:46.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.924+0000 7f4e497e3640 1 -- 172.21.15.175:0/4249860256 >> v1:172.21.15.47:6789/0 conn(0x7f4e440c9dc0 legacy=0x7f4e44069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.924+0000 7f4e497e3640 1 -- 172.21.15.175:0/4249860256 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4415ded0 con 0x7f4e4415d0c0 2024-03-23T09:40:46.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.924+0000 7f4e497e3640 1 --2- 172.21.15.175:0/4249860256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e4415d0c0 0x7f4e4415d550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009720 tx=0x7f4e4002b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aab00e096e27595 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:46.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.924+0000 7f4e48fe2640 1 -- 172.21.15.175:0/4249860256 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34001670 con 0x7f4e4415d0c0 2024-03-23T09:40:46.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.924+0000 7f4e48fe2640 1 -- 172.21.15.175:0/4249860256 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e40031520 con 0x7f4e4415d0c0 2024-03-23T09:40:46.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.924+0000 7f4e48fe2640 1 -- 172.21.15.175:0/4249860256 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e400316b0 con 0x7f4e4415d0c0 2024-03-23T09:40:46.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.925+0000 7f4e4c248640 1 -- 172.21.15.175:0/4249860256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e4415d0c0 msgr2=0x7f4e4415d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.925+0000 7f4e4c248640 1 --2- 172.21.15.175:0/4249860256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e4415d0c0 0x7f4e4415d550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009720 tx=0x7f4e4002b7c0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.925+0000 7f4e4c248640 1 -- 172.21.15.175:0/4249860256 shutdown_connections 2024-03-23T09:40:46.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.925+0000 7f4e4c248640 1 --2- 172.21.15.175:0/4249860256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e4415d0c0 0x7f4e4415d550 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.925+0000 7f4e4c248640 1 -- 172.21.15.175:0/4249860256 >> 172.21.15.175:0/4249860256 conn(0x7f4e44071400 msgr2=0x7f4e44071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:46.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.925+0000 7f4e4c248640 1 -- 172.21.15.175:0/4249860256 shutdown_connections 2024-03-23T09:40:46.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.925+0000 7f4e4c248640 1 -- 172.21.15.175:0/4249860256 wait complete. 2024-03-23T09:40:46.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.926+0000 7f4e4c248640 1 Processor -- start 2024-03-23T09:40:46.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.926+0000 7f4e4c248640 1 -- start start 2024-03-23T09:40:46.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e4c248640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e440c9dc0 0x7f4e4414b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e4c248640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e441571e0 0x7f4e440c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e4c248640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e4415d0c0 0x7f4e440c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:46.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e4c248640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440b2490 con 0x7f4e440c9dc0 2024-03-23T09:40:46.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e4c248640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440b2730 con 0x7f4e4415d0c0 2024-03-23T09:40:46.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e4c248640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440b2a10 con 0x7f4e441571e0 2024-03-23T09:40:46.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e497e3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e441571e0 0x7f4e440c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e4a7e5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e4415d0c0 0x7f4e440c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e497e3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e441571e0 0x7f4e440c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:46808/0 (socket says 172.21.15.175:46808) 2024-03-23T09:40:46.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e497e3640 1 -- 172.21.15.175:0/1470310255 learned_addr learned my addr 172.21.15.175:0/1470310255 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-23T09:40:46.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e49fe4640 1 --2- 172.21.15.175:0/1470310255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e440c9dc0 0x7f4e4414b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:46.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e497e3640 1 -- 172.21.15.175:0/1470310255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e4415d0c0 msgr2=0x7f4e440c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e497e3640 1 --2- 172.21.15.175:0/1470310255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e4415d0c0 0x7f4e440c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e497e3640 1 -- 172.21.15.175:0/1470310255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e440c9dc0 msgr2=0x7f4e4414b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:46.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e497e3640 1 --2- 172.21.15.175:0/1470310255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e440c9dc0 0x7f4e4414b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:46.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e497e3640 1 -- 172.21.15.175:0/1470310255 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e40009400 con 0x7f4e441571e0 2024-03-23T09:40:46.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e4a7e5640 1 --2- 172.21.15.175:0/1470310255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e4415d0c0 0x7f4e440c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:40:46.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.927+0000 7f4e49fe4640 1 --2- 172.21.15.175:0/1470310255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e440c9dc0 0x7f4e4414b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:40:46.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.928+0000 7f4e497e3640 1 --2- 172.21.15.175:0/1470310255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e441571e0 0x7f4e440c6840 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e40036040 tx=0x7f4e400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:46.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.928+0000 7f4e3a7fc640 1 -- 172.21.15.175:0/1470310255 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e400338e0 con 0x7f4e441571e0 2024-03-23T09:40:46.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.928+0000 7f4e3a7fc640 1 -- 172.21.15.175:0/1470310255 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e40033a40 con 0x7f4e441571e0 2024-03-23T09:40:46.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.928+0000 7f4e3a7fc640 1 -- 172.21.15.175:0/1470310255 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40033dd0 con 0x7f4e441571e0 2024-03-23T09:40:46.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.928+0000 7f4e4c248640 1 -- 172.21.15.175:0/1470310255 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e440b2cf0 con 0x7f4e441571e0 2024-03-23T09:40:46.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.928+0000 7f4e4c248640 1 -- 172.21.15.175:0/1470310255 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4415f0a0 con 0x7f4e441571e0 2024-03-23T09:40:46.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.928+0000 7f4e3a7fc640 1 -- 172.21.15.175:0/1470310255 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4e4003a040 con 0x7f4e441571e0 2024-03-23T09:40:46.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.928+0000 7f4e3a7fc640 1 -- 172.21.15.175:0/1470310255 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f4e4003c840 con 0x7f4e441571e0 2024-03-23T09:40:46.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.929+0000 7f4e4c248640 1 -- 172.21.15.175:0/1470310255 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e440672f0 con 0x7f4e441571e0 2024-03-23T09:40:46.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:46.931+0000 7f4e3a7fc640 1 -- 172.21.15.175:0/1470310255 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f4e40067b50 con 0x7f4e441571e0 2024-03-23T09:40:47.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.051+0000 7f4e4c248640 1 -- 172.21.15.175:0/1470310255 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "49f7a329-d653-43d1-9e27-54b0e93f2eec", "id": 1} v 0) v1 -- 0x7f4e440cc1f0 con 0x7f4e441571e0 2024-03-23T09:40:47.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.054+0000 7f4e3a7fc640 1 -- 172.21.15.175:0/1470310255 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "49f7a329-d653-43d1-9e27-54b0e93f2eec", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4e40056070 con 0x7f4e441571e0 2024-03-23T09:40:47.054 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-03-23T09:40:47.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.055+0000 7f4e4c248640 1 -- 172.21.15.175:0/1470310255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e441571e0 msgr2=0x7f4e440c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.055+0000 7f4e4c248640 1 --2- 172.21.15.175:0/1470310255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e441571e0 0x7f4e440c6840 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e40036040 tx=0x7f4e400046c0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.055+0000 7f4e4c248640 1 -- 172.21.15.175:0/1470310255 shutdown_connections 2024-03-23T09:40:47.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.055+0000 7f4e4c248640 1 --2- 172.21.15.175:0/1470310255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e4415d0c0 0x7f4e440c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.055+0000 7f4e4c248640 1 --2- 172.21.15.175:0/1470310255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e441571e0 0x7f4e440c6840 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.055+0000 7f4e4c248640 1 --2- 172.21.15.175:0/1470310255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e440c9dc0 0x7f4e4414b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.055+0000 7f4e4c248640 1 -- 172.21.15.175:0/1470310255 >> 172.21.15.175:0/1470310255 conn(0x7f4e44071400 msgr2=0x7f4e44069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:47.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.055+0000 7f4e4c248640 1 -- 172.21.15.175:0/1470310255 shutdown_connections 2024-03-23T09:40:47.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.055+0000 7f4e4c248640 1 -- 172.21.15.175:0/1470310255 wait complete. 2024-03-23T09:40:47.065 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 8d3fec55-1be5-429f-888f-5ae197d8ad96 2 2024-03-23T09:40:47.112 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:47.113+0000 7ff054a10040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T09:40:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.165+0000 7f969e777640 1 Processor -- start 2024-03-23T09:40:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.165+0000 7f969e777640 1 -- start start 2024-03-23T09:40:47.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.166+0000 7f969e777640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980cdee0 0x7f96981645a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.166+0000 7f969e777640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96980ca600 con 0x7f969815ce60 2024-03-23T09:40:47.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.166+0000 7f969e777640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96980ca8e0 con 0x7f96980ca000 2024-03-23T09:40:47.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.166+0000 7f969e777640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96980cb280 con 0x7f96980cdee0 2024-03-23T09:40:47.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.166+0000 7f969cd14640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980cdee0 0x7f96981645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.166+0000 7f969cd14640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980cdee0 0x7f96981645a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:46816/0 (socket says 172.21.15.175:46816) 2024-03-23T09:40:47.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.166+0000 7f96977fe640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f96980ca000 0x7f96980cd7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:38118/0 (socket says 172.21.15.175:38118) 2024-03-23T09:40:47.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.166+0000 7f969cd14640 1 -- 172.21.15.175:0/3167587488 learned_addr learned my addr 172.21.15.175:0/3167587488 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-23T09:40:47.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.167+0000 7f969cd14640 1 -- 172.21.15.175:0/3167587488 >> v1:172.21.15.175:6789/0 conn(0x7f96980ca000 legacy=0x7f96980cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.167+0000 7f9696ffd640 1 -- 172.21.15.175:0/3167587488 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 639881359 0 0) 0x7f96980ca8e0 con 0x7f96980ca000 2024-03-23T09:40:47.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.167+0000 7f969cd14640 1 -- 172.21.15.175:0/3167587488 >> v1:172.21.15.47:6789/0 conn(0x7f969815ce60 legacy=0x7f969806ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.167+0000 7f969cd14640 1 -- 172.21.15.175:0/3167587488 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980cabc0 con 0x7f96980cdee0 2024-03-23T09:40:47.168 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:47.167+0000 7ff054a10040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T09:40:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.167+0000 7f969cd14640 1 --2- 172.21.15.175:0/3167587488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980cdee0 0x7f96981645a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9690009e80 tx=0x7f969002c060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=198bade35c5c0a1e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.167+0000 7f9696ffd640 1 -- 172.21.15.175:0/3167587488 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969002cc00 con 0x7f96980cdee0 2024-03-23T09:40:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.167+0000 7f9696ffd640 1 -- 172.21.15.175:0/3167587488 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969002cd60 con 0x7f96980cdee0 2024-03-23T09:40:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.167+0000 7f9696ffd640 1 -- 172.21.15.175:0/3167587488 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690034610 con 0x7f96980cdee0 2024-03-23T09:40:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.168+0000 7f969e777640 1 -- 172.21.15.175:0/3167587488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980cdee0 msgr2=0x7f96981645a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.168+0000 7f969e777640 1 --2- 172.21.15.175:0/3167587488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980cdee0 0x7f96981645a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9690009e80 tx=0x7f969002c060 comp rx=0 tx=0).stop 2024-03-23T09:40:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.168+0000 7f969e777640 1 -- 172.21.15.175:0/3167587488 shutdown_connections 2024-03-23T09:40:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.168+0000 7f969e777640 1 --2- 172.21.15.175:0/3167587488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980cdee0 0x7f96981645a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.168+0000 7f969e777640 1 -- 172.21.15.175:0/3167587488 >> 172.21.15.175:0/3167587488 conn(0x7f9698098a00 msgr2=0x7f96980cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.168+0000 7f969e777640 1 -- 172.21.15.175:0/3167587488 shutdown_connections 2024-03-23T09:40:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.168+0000 7f969e777640 1 -- 172.21.15.175:0/3167587488 wait complete. 2024-03-23T09:40:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.169+0000 7f969e777640 1 Processor -- start 2024-03-23T09:40:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.169+0000 7f969e777640 1 -- start start 2024-03-23T09:40:47.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.170+0000 7f969e777640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980ca000 0x7f9698155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.170+0000 7f969e777640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96980cdee0 0x7f96980b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.170+0000 7f969e777640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969815ce60 0x7f96980b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.170+0000 7f969e777640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9698157370 con 0x7f969815ce60 2024-03-23T09:40:47.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.170+0000 7f969e777640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96981566f0 con 0x7f96980cdee0 2024-03-23T09:40:47.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.170+0000 7f969e777640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96981569d0 con 0x7f96980ca000 2024-03-23T09:40:47.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.170+0000 7f9697fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980ca000 0x7f9698155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.170+0000 7f96977fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96980cdee0 0x7f96980b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.170+0000 7f9697fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980ca000 0x7f9698155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:46830/0 (socket says 172.21.15.175:46830) 2024-03-23T09:40:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.170+0000 7f9697fff640 1 -- 172.21.15.175:0/2923822161 learned_addr learned my addr 172.21.15.175:0/2923822161 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-23T09:40:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.170+0000 7f969cd14640 1 --2- 172.21.15.175:0/2923822161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969815ce60 0x7f96980b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f96977fe640 1 -- 172.21.15.175:0/2923822161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980ca000 msgr2=0x7f9698155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f96977fe640 1 --2- 172.21.15.175:0/2923822161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980ca000 0x7f9698155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f96977fe640 1 -- 172.21.15.175:0/2923822161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969815ce60 msgr2=0x7f96980b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f96977fe640 1 --2- 172.21.15.175:0/2923822161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969815ce60 0x7f96980b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f96977fe640 1 -- 172.21.15.175:0/2923822161 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9690009b30 con 0x7f96980cdee0 2024-03-23T09:40:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f9697fff640 1 --2- 172.21.15.175:0/2923822161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980ca000 0x7f9698155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:40:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f96977fe640 1 --2- 172.21.15.175:0/2923822161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96980cdee0 0x7f96980b3e60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f968800c3e0 tx=0x7f968800c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f9694ff9640 1 -- 172.21.15.175:0/2923822161 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96880074e0 con 0x7f96980cdee0 2024-03-23T09:40:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f9694ff9640 1 -- 172.21.15.175:0/2923822161 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9688007640 con 0x7f96980cdee0 2024-03-23T09:40:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f969e777640 1 -- 172.21.15.175:0/2923822161 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9698156cb0 con 0x7f96980cdee0 2024-03-23T09:40:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f9694ff9640 1 -- 172.21.15.175:0/2923822161 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688004a20 con 0x7f96980cdee0 2024-03-23T09:40:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f969e777640 1 -- 172.21.15.175:0/2923822161 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96981563e0 con 0x7f96980cdee0 2024-03-23T09:40:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f9694ff9640 1 -- 172.21.15.175:0/2923822161 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f968801d3f0 con 0x7f96980cdee0 2024-03-23T09:40:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.171+0000 7f9694ff9640 1 -- 172.21.15.175:0/2923822161 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f96880137e0 con 0x7f96980cdee0 2024-03-23T09:40:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.172+0000 7f969e777640 1 -- 172.21.15.175:0/2923822161 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9698073560 con 0x7f96980cdee0 2024-03-23T09:40:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.174+0000 7f9694ff9640 1 -- 172.21.15.175:0/2923822161 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f96880133c0 con 0x7f96980cdee0 2024-03-23T09:40:47.196 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:47.196+0000 7fa059e3e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T09:40:47.212 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:47.213+0000 7ff054a10040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T09:40:47.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.246+0000 7f969e777640 1 -- 172.21.15.175:0/2923822161 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8d3fec55-1be5-429f-888f-5ae197d8ad96", "id": 2} v 0) v1 -- 0x7f96980cf240 con 0x7f96980cdee0 2024-03-23T09:40:47.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.249+0000 7f9694ff9640 1 -- 172.21.15.175:0/2923822161 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8d3fec55-1be5-429f-888f-5ae197d8ad96", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9688013a80 con 0x7f96980cdee0 2024-03-23T09:40:47.249 INFO:teuthology.orchestra.run.smithi175.stdout:2 2024-03-23T09:40:47.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.250+0000 7f969e777640 1 -- 172.21.15.175:0/2923822161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96980cdee0 msgr2=0x7f96980b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.250+0000 7f969e777640 1 --2- 172.21.15.175:0/2923822161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96980cdee0 0x7f96980b3e60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f968800c3e0 tx=0x7f968800c8a0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.250+0000 7f969e777640 1 -- 172.21.15.175:0/2923822161 shutdown_connections 2024-03-23T09:40:47.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.250+0000 7f969e777640 1 --2- 172.21.15.175:0/2923822161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969815ce60 0x7f96980b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.250+0000 7f969e777640 1 --2- 172.21.15.175:0/2923822161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96980cdee0 0x7f96980b3e60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.250+0000 7f969e777640 1 --2- 172.21.15.175:0/2923822161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96980ca000 0x7f9698155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.250+0000 7f969e777640 1 -- 172.21.15.175:0/2923822161 >> 172.21.15.175:0/2923822161 conn(0x7f9698098a00 msgr2=0x7f9698069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:47.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.250+0000 7f969e777640 1 -- 172.21.15.175:0/2923822161 shutdown_connections 2024-03-23T09:40:47.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.250+0000 7f969e777640 1 -- 172.21.15.175:0/2923822161 wait complete. 2024-03-23T09:40:47.251 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:47.250+0000 7fa059e3e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T09:40:47.260 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 07e2cb27-0df4-4a86-8e29-3dd1bc03072f 3 2024-03-23T09:40:47.269 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:47.269+0000 7ff054a10040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T09:40:47.297 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:47.298+0000 7fa059e3e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T09:40:47.312 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:47.313+0000 7ff054a10040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T09:40:47.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.361+0000 7f887e070640 1 Processor -- start 2024-03-23T09:40:47.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.361+0000 7f887e070640 1 -- start start 2024-03-23T09:40:47.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.361+0000 7f887e070640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f887815a8d0 0x7f88780cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.361+0000 7f887e070640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88780ce510 con 0x7f88780993c0 2024-03-23T09:40:47.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.361+0000 7f887e070640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88780ce7d0 con 0x7f887815ebd0 2024-03-23T09:40:47.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.361+0000 7f887e070640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88780ce320 con 0x7f887815a8d0 2024-03-23T09:40:47.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.362+0000 7f88777fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f887815a8d0 0x7f88780cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.362+0000 7f8877fff640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f887815ebd0 0x7f88780c42a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:38130/0 (socket says 172.21.15.175:38130) 2024-03-23T09:40:47.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.362+0000 7f8877fff640 1 -- 172.21.15.175:0/1093070712 learned_addr learned my addr 172.21.15.175:0/1093070712 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-03-23T09:40:47.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.362+0000 7f88767fc640 1 -- 172.21.15.175:0/1093070712 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519612522 0 0) 0x7f88780ce7d0 con 0x7f887815ebd0 2024-03-23T09:40:47.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.362+0000 7f88777fe640 1 -- 172.21.15.175:0/1093070712 >> v1:172.21.15.175:6789/0 conn(0x7f887815ebd0 legacy=0x7f88780c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.362+0000 7f88777fe640 1 -- 172.21.15.175:0/1093070712 >> v1:172.21.15.47:6789/0 conn(0x7f88780993c0 legacy=0x7f887815e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.362+0000 7f88777fe640 1 -- 172.21.15.175:0/1093070712 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88780ceab0 con 0x7f887815a8d0 2024-03-23T09:40:47.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.363+0000 7f88777fe640 1 --2- 172.21.15.175:0/1093070712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f887815a8d0 0x7f88780cf2b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f88680097d0 tx=0x7f886802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15980de80d1214dd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:47.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.363+0000 7f88767fc640 1 -- 172.21.15.175:0/1093070712 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868031800 con 0x7f887815a8d0 2024-03-23T09:40:47.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.363+0000 7f88767fc640 1 -- 172.21.15.175:0/1093070712 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8868031960 con 0x7f887815a8d0 2024-03-23T09:40:47.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.363+0000 7f88767fc640 1 -- 172.21.15.175:0/1093070712 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868031cf0 con 0x7f887815a8d0 2024-03-23T09:40:47.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.363+0000 7f887e070640 1 -- 172.21.15.175:0/1093070712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f887815a8d0 msgr2=0x7f88780cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.363+0000 7f887e070640 1 --2- 172.21.15.175:0/1093070712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f887815a8d0 0x7f88780cf2b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f88680097d0 tx=0x7f886802b8a0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.363+0000 7f887e070640 1 -- 172.21.15.175:0/1093070712 shutdown_connections 2024-03-23T09:40:47.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.363+0000 7f887e070640 1 --2- 172.21.15.175:0/1093070712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f887815a8d0 0x7f88780cf2b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.363+0000 7f887e070640 1 -- 172.21.15.175:0/1093070712 >> 172.21.15.175:0/1093070712 conn(0x7f88780b2240 msgr2=0x7f88780b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:47.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.363+0000 7f887e070640 1 -- 172.21.15.175:0/1093070712 shutdown_connections 2024-03-23T09:40:47.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.363+0000 7f887e070640 1 -- 172.21.15.175:0/1093070712 wait complete. 2024-03-23T09:40:47.367 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:47.364+0000 7fa059e3e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T09:40:47.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.364+0000 7f887e070640 1 Processor -- start 2024-03-23T09:40:47.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.365+0000 7f887e070640 1 -- start start 2024-03-23T09:40:47.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.365+0000 7f887e070640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88780993c0 0x7f887815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.365+0000 7f887e070640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f887815a8d0 0x7f88780b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.365+0000 7f887e070640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f887815ebd0 0x7f8878150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.365+0000 7f887e070640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88780b4a70 con 0x7f887815a8d0 2024-03-23T09:40:47.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.365+0000 7f887e070640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88780c8a10 con 0x7f88780993c0 2024-03-23T09:40:47.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.365+0000 7f887e070640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88780b3880 con 0x7f887815ebd0 2024-03-23T09:40:47.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.365+0000 7f88777fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88780993c0 0x7f887815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.365+0000 7f8877fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f887815ebd0 0x7f8878150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f8877fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f887815ebd0 0x7f8878150570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:46838/0 (socket says 172.21.15.175:46838) 2024-03-23T09:40:47.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f88777fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88780993c0 0x7f887815cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50000/0 (socket says 172.21.15.175:50000) 2024-03-23T09:40:47.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f8877fff640 1 -- 172.21.15.175:0/549537651 learned_addr learned my addr 172.21.15.175:0/549537651 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-23T09:40:47.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f8876ffd640 1 --2- 172.21.15.175:0/549537651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f887815a8d0 0x7f88780b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f88777fe640 1 -- 172.21.15.175:0/549537651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f887815ebd0 msgr2=0x7f8878150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f88777fe640 1 --2- 172.21.15.175:0/549537651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f887815ebd0 0x7f8878150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f88777fe640 1 -- 172.21.15.175:0/549537651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f887815a8d0 msgr2=0x7f88780b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f88777fe640 1 --2- 172.21.15.175:0/549537651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f887815a8d0 0x7f88780b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f88777fe640 1 -- 172.21.15.175:0/549537651 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8868009400 con 0x7f88780993c0 2024-03-23T09:40:47.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f8877fff640 1 --2- 172.21.15.175:0/549537651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f887815ebd0 0x7f8878150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:40:47.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f88777fe640 1 --2- 172.21.15.175:0/549537651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88780993c0 0x7f887815cb60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f88780d0720 tx=0x7f88680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:47.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f885ffff640 1 -- 172.21.15.175:0/549537651 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886803be00 con 0x7f88780993c0 2024-03-23T09:40:47.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f885ffff640 1 -- 172.21.15.175:0/549537651 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8868045400 con 0x7f88780993c0 2024-03-23T09:40:47.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f887e070640 1 -- 172.21.15.175:0/549537651 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88780b3b20 con 0x7f88780993c0 2024-03-23T09:40:47.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f885ffff640 1 -- 172.21.15.175:0/549537651 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88680457b0 con 0x7f88780993c0 2024-03-23T09:40:47.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f887e070640 1 -- 172.21.15.175:0/549537651 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88780b40e0 con 0x7f88780993c0 2024-03-23T09:40:47.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f885ffff640 1 -- 172.21.15.175:0/549537651 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8868045c10 con 0x7f88780993c0 2024-03-23T09:40:47.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.366+0000 7f885ffff640 1 -- 172.21.15.175:0/549537651 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f886802e070 con 0x7f88780993c0 2024-03-23T09:40:47.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.367+0000 7f887e070640 1 -- 172.21.15.175:0/549537651 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88780672f0 con 0x7f88780993c0 2024-03-23T09:40:47.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.369+0000 7f885ffff640 1 -- 172.21.15.175:0/549537651 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f8868035020 con 0x7f88780993c0 2024-03-23T09:40:47.408 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:47.409+0000 7fa059e3e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T09:40:47.431 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:47.432+0000 7ff054a10040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T09:40:47.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.441+0000 7f887e070640 1 -- 172.21.15.175:0/549537651 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "07e2cb27-0df4-4a86-8e29-3dd1bc03072f", "id": 3} v 0) v1 -- 0x7f88780b30e0 con 0x7f88780993c0 2024-03-23T09:40:47.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.444+0000 7f885ffff640 1 -- 172.21.15.175:0/549537651 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "07e2cb27-0df4-4a86-8e29-3dd1bc03072f", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f886804e440 con 0x7f88780993c0 2024-03-23T09:40:47.444 INFO:teuthology.orchestra.run.smithi175.stdout:3 2024-03-23T09:40:47.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.446+0000 7f887e070640 1 -- 172.21.15.175:0/549537651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88780993c0 msgr2=0x7f887815cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.446+0000 7f887e070640 1 --2- 172.21.15.175:0/549537651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88780993c0 0x7f887815cb60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f88780d0720 tx=0x7f88680046c0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.446+0000 7f887e070640 1 -- 172.21.15.175:0/549537651 shutdown_connections 2024-03-23T09:40:47.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.446+0000 7f887e070640 1 --2- 172.21.15.175:0/549537651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f887815ebd0 0x7f8878150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.446+0000 7f887e070640 1 --2- 172.21.15.175:0/549537651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f887815a8d0 0x7f88780b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.446+0000 7f887e070640 1 --2- 172.21.15.175:0/549537651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88780993c0 0x7f887815cb60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.446+0000 7f887e070640 1 -- 172.21.15.175:0/549537651 >> 172.21.15.175:0/549537651 conn(0x7f88780b2240 msgr2=0x7f88780cb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:47.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.446+0000 7f887e070640 1 -- 172.21.15.175:0/549537651 shutdown_connections 2024-03-23T09:40:47.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.446+0000 7f887e070640 1 -- 172.21.15.175:0/549537651 wait complete. 2024-03-23T09:40:47.456 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new ee55dc92-68e7-472b-8953-1e67644f8e63 4 2024-03-23T09:40:47.504 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:47.504+0000 7ff054a10040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T09:40:47.531 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:47.532+0000 7fa059e3e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T09:40:47.551 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:47.552+0000 7ff054a10040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T09:40:47.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.598+0000 7f6d88e5c640 1 Processor -- start 2024-03-23T09:40:47.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.598+0000 7f6d88e5c640 1 -- start start 2024-03-23T09:40:47.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d88e5c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6d840ce2f0 0x7f6d8415dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d88e5c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d840cc4b0 0x7f6d840ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d88e5c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8406ea80 con 0x7f6d840cb420 2024-03-23T09:40:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d88e5c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d8415da80 con 0x7f6d840cc4b0 2024-03-23T09:40:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d88e5c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d840659d0 con 0x7f6d840ce2f0 2024-03-23T09:40:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d8259c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6d840ce2f0 0x7f6d8415dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d8259c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6d840ce2f0 0x7f6d8415dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50006/0 (socket says 172.21.15.175:50006) 2024-03-23T09:40:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d8259c640 1 -- 172.21.15.175:0/3237277283 learned_addr learned my addr 172.21.15.175:0/3237277283 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-23T09:40:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d82d9d640 1 --1- 172.21.15.175:0/3237277283 >> v1:172.21.15.175:6789/0 conn(0x7f6d840cb420 0x7f6d8415c520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:38132/0 (socket says 172.21.15.175:38132) 2024-03-23T09:40:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d81d9b640 1 --2- 172.21.15.175:0/3237277283 >> v2:172.21.15.47:3300/0 conn(0x7f6d840cc4b0 0x7f6d840ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d8259c640 1 -- 172.21.15.175:0/3237277283 >> v2:172.21.15.47:3300/0 conn(0x7f6d840cc4b0 msgr2=0x7f6d840ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:40:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d8259c640 1 --2- 172.21.15.175:0/3237277283 >> v2:172.21.15.47:3300/0 conn(0x7f6d840cc4b0 0x7f6d840ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d8259c640 1 -- 172.21.15.175:0/3237277283 >> v1:172.21.15.175:6789/0 conn(0x7f6d840cb420 legacy=0x7f6d8415c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.599+0000 7f6d8259c640 1 -- 172.21.15.175:0/3237277283 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d840cec10 con 0x7f6d840ce2f0 2024-03-23T09:40:47.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.600+0000 7f6d8259c640 1 --2- 172.21.15.175:0/3237277283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d840ce2f0 0x7f6d8415dea0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6d8404e8f0 tx=0x7f6d7402b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0a0a769241ed7e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:47.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.600+0000 7f6d8159a640 1 -- 172.21.15.175:0/3237277283 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d740317e0 con 0x7f6d840ce2f0 2024-03-23T09:40:47.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.600+0000 7f6d8159a640 1 -- 172.21.15.175:0/3237277283 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d74031940 con 0x7f6d840ce2f0 2024-03-23T09:40:47.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.600+0000 7f6d8159a640 1 -- 172.21.15.175:0/3237277283 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74031cd0 con 0x7f6d840ce2f0 2024-03-23T09:40:47.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.600+0000 7f6d88e5c640 1 -- 172.21.15.175:0/3237277283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d840ce2f0 msgr2=0x7f6d8415dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.600+0000 7f6d88e5c640 1 --2- 172.21.15.175:0/3237277283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d840ce2f0 0x7f6d8415dea0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6d8404e8f0 tx=0x7f6d7402b880 comp rx=0 tx=0).stop 2024-03-23T09:40:47.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.601+0000 7f6d88e5c640 1 -- 172.21.15.175:0/3237277283 shutdown_connections 2024-03-23T09:40:47.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.601+0000 7f6d88e5c640 1 --2- 172.21.15.175:0/3237277283 >> v2:172.21.15.47:3300/0 conn(0x7f6d840cc4b0 0x7f6d840ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.601+0000 7f6d88e5c640 1 --2- 172.21.15.175:0/3237277283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d840ce2f0 0x7f6d8415dea0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.601+0000 7f6d88e5c640 1 -- 172.21.15.175:0/3237277283 >> 172.21.15.175:0/3237277283 conn(0x7f6d840cb990 msgr2=0x7f6d840cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:47.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.601+0000 7f6d88e5c640 1 -- 172.21.15.175:0/3237277283 shutdown_connections 2024-03-23T09:40:47.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.601+0000 7f6d88e5c640 1 -- 172.21.15.175:0/3237277283 wait complete. 2024-03-23T09:40:47.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.602+0000 7f6d88e5c640 1 Processor -- start 2024-03-23T09:40:47.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.602+0000 7f6d88e5c640 1 -- start start 2024-03-23T09:40:47.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d88e5c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d840cb420 0x7f6d84158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d88e5c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d840cc4b0 0x7f6d840b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d88e5c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d840ce2f0 0x7f6d840b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d88e5c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84160820 con 0x7f6d840cb420 2024-03-23T09:40:47.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d88e5c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d840b38c0 con 0x7f6d840cc4b0 2024-03-23T09:40:47.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d88e5c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d840b3ba0 con 0x7f6d840ce2f0 2024-03-23T09:40:47.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d82d9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d840ce2f0 0x7f6d840b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d81d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d840cc4b0 0x7f6d840b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d82d9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d840ce2f0 0x7f6d840b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:46844/0 (socket says 172.21.15.175:46844) 2024-03-23T09:40:47.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d82d9d640 1 -- 172.21.15.175:0/2409040563 learned_addr learned my addr 172.21.15.175:0/2409040563 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d8259c640 1 --2- 172.21.15.175:0/2409040563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d840cb420 0x7f6d84158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d81d9b640 1 -- 172.21.15.175:0/2409040563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d840ce2f0 msgr2=0x7f6d840b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d81d9b640 1 --2- 172.21.15.175:0/2409040563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d840ce2f0 0x7f6d840b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d81d9b640 1 -- 172.21.15.175:0/2409040563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d840cb420 msgr2=0x7f6d84158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d81d9b640 1 --2- 172.21.15.175:0/2409040563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d840cb420 0x7f6d84158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d81d9b640 1 -- 172.21.15.175:0/2409040563 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d740094d0 con 0x7f6d840cc4b0 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d82d9d640 1 --2- 172.21.15.175:0/2409040563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d840ce2f0 0x7f6d840b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d8259c640 1 --2- 172.21.15.175:0/2409040563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d840cb420 0x7f6d84158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d81d9b640 1 --2- 172.21.15.175:0/2409040563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d840cc4b0 0x7f6d840b2a60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800e820 tx=0x7f6d7800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.603+0000 7f6d72ffd640 1 -- 172.21.15.175:0/2409040563 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7800c940 con 0x7f6d840cc4b0 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.604+0000 7f6d72ffd640 1 -- 172.21.15.175:0/2409040563 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7800caa0 con 0x7f6d840cc4b0 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.604+0000 7f6d88e5c640 1 -- 172.21.15.175:0/2409040563 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d84158570 con 0x7f6d840cc4b0 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.604+0000 7f6d72ffd640 1 -- 172.21.15.175:0/2409040563 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7800ce30 con 0x7f6d840cc4b0 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.604+0000 7f6d88e5c640 1 -- 172.21.15.175:0/2409040563 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d84158f30 con 0x7f6d840cc4b0 2024-03-23T09:40:47.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.604+0000 7f6d72ffd640 1 -- 172.21.15.175:0/2409040563 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6d78005c90 con 0x7f6d840cc4b0 2024-03-23T09:40:47.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.604+0000 7f6d72ffd640 1 -- 172.21.15.175:0/2409040563 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f6d78013980 con 0x7f6d840cc4b0 2024-03-23T09:40:47.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.604+0000 7f6d88e5c640 1 -- 172.21.15.175:0/2409040563 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8415d660 con 0x7f6d840cc4b0 2024-03-23T09:40:47.607 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:47.606+0000 7fa059e3e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T09:40:47.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.606+0000 7f6d72ffd640 1 -- 172.21.15.175:0/2409040563 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f6d78021060 con 0x7f6d840cc4b0 2024-03-23T09:40:47.621 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:47.622+0000 7ff054a10040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T09:40:47.652 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:47.653+0000 7fa059e3e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T09:40:47.670 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:47.671+0000 7ff054a10040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T09:40:47.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.676+0000 7f6d88e5c640 1 -- 172.21.15.175:0/2409040563 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ee55dc92-68e7-472b-8953-1e67644f8e63", "id": 4} v 0) v1 -- 0x7f6d84158700 con 0x7f6d840cc4b0 2024-03-23T09:40:47.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.680+0000 7f6d72ffd640 1 -- 172.21.15.175:0/2409040563 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ee55dc92-68e7-472b-8953-1e67644f8e63", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6d84158700 con 0x7f6d840cc4b0 2024-03-23T09:40:47.679 INFO:teuthology.orchestra.run.smithi175.stdout:4 2024-03-23T09:40:47.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.681+0000 7f6d88e5c640 1 -- 172.21.15.175:0/2409040563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d840cc4b0 msgr2=0x7f6d840b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.681+0000 7f6d88e5c640 1 --2- 172.21.15.175:0/2409040563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d840cc4b0 0x7f6d840b2a60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800e820 tx=0x7f6d7800ece0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.681+0000 7f6d88e5c640 1 -- 172.21.15.175:0/2409040563 shutdown_connections 2024-03-23T09:40:47.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.681+0000 7f6d88e5c640 1 --2- 172.21.15.175:0/2409040563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d840ce2f0 0x7f6d840b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.681+0000 7f6d88e5c640 1 --2- 172.21.15.175:0/2409040563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d840cc4b0 0x7f6d840b2a60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.681+0000 7f6d88e5c640 1 --2- 172.21.15.175:0/2409040563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d840cb420 0x7f6d84158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.681+0000 7f6d88e5c640 1 -- 172.21.15.175:0/2409040563 >> 172.21.15.175:0/2409040563 conn(0x7f6d840cb990 msgr2=0x7f6d8415c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:47.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.681+0000 7f6d88e5c640 1 -- 172.21.15.175:0/2409040563 shutdown_connections 2024-03-23T09:40:47.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.681+0000 7f6d88e5c640 1 -- 172.21.15.175:0/2409040563 wait complete. 2024-03-23T09:40:47.690 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new b6112e8d-e0eb-4b16-bff7-465d2013a7ee 5 2024-03-23T09:40:47.706 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:47.706+0000 7fa059e3e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T09:40:47.755 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:47.756+0000 7fa059e3e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T09:40:47.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.806+0000 7f91c1be7640 1 Processor -- start 2024-03-23T09:40:47.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.806+0000 7f91c1be7640 1 -- start start 2024-03-23T09:40:47.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.807+0000 7f91c1be7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 0x7f91bc06f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.807+0000 7f91c1be7640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91bc0cf2f0 con 0x7f91bc0ce780 2024-03-23T09:40:47.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.807+0000 7f91c1be7640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91bc06fb30 con 0x7f91bc0b28e0 2024-03-23T09:40:47.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.807+0000 7f91c1be7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0d01c0 con 0x7f91bc06f210 2024-03-23T09:40:47.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.807+0000 7f91bb7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 0x7f91bc06f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.807+0000 7f91bb7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 0x7f91bc06f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:46860/0 (socket says 172.21.15.175:46860) 2024-03-23T09:40:47.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.807+0000 7f91bb7fe640 1 -- 172.21.15.175:0/1921253204 learned_addr learned my addr 172.21.15.175:0/1921253204 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-23T09:40:47.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.808+0000 7f91ba7fc640 1 -- 172.21.15.175:0/1921253204 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136911764 0 0) 0x7f91bc06fb30 con 0x7f91bc0b28e0 2024-03-23T09:40:47.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.808+0000 7f91bb7fe640 1 -- 172.21.15.175:0/1921253204 >> v1:172.21.15.175:6789/0 conn(0x7f91bc0b28e0 legacy=0x7f91bc0b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.808+0000 7f91bb7fe640 1 -- 172.21.15.175:0/1921253204 >> v1:172.21.15.47:6789/0 conn(0x7f91bc0ce780 legacy=0x7f91bc0cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.808+0000 7f91bb7fe640 1 -- 172.21.15.175:0/1921253204 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc0cf540 con 0x7f91bc06f210 2024-03-23T09:40:47.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.809+0000 7f91bb7fe640 1 --2- 172.21.15.175:0/1921253204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 0x7f91bc06f5f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f91b00098a0 tx=0x7f91b002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5066240981f8a050 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:47.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.809+0000 7f91ba7fc640 1 -- 172.21.15.175:0/1921253204 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0030840 con 0x7f91bc06f210 2024-03-23T09:40:47.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.809+0000 7f91ba7fc640 1 -- 172.21.15.175:0/1921253204 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b00309a0 con 0x7f91bc06f210 2024-03-23T09:40:47.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.809+0000 7f91ba7fc640 1 -- 172.21.15.175:0/1921253204 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0030d30 con 0x7f91bc06f210 2024-03-23T09:40:47.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.809+0000 7f91c1be7640 1 -- 172.21.15.175:0/1921253204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 msgr2=0x7f91bc06f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.809+0000 7f91c1be7640 1 --2- 172.21.15.175:0/1921253204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 0x7f91bc06f5f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f91b00098a0 tx=0x7f91b002b980 comp rx=0 tx=0).stop 2024-03-23T09:40:47.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.809+0000 7f91c1be7640 1 -- 172.21.15.175:0/1921253204 shutdown_connections 2024-03-23T09:40:47.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.809+0000 7f91c1be7640 1 --2- 172.21.15.175:0/1921253204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 0x7f91bc06f5f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.809+0000 7f91c1be7640 1 -- 172.21.15.175:0/1921253204 >> 172.21.15.175:0/1921253204 conn(0x7f91bc071400 msgr2=0x7f91bc071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:47.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.810+0000 7f91c1be7640 1 -- 172.21.15.175:0/1921253204 shutdown_connections 2024-03-23T09:40:47.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.810+0000 7f91c1be7640 1 -- 172.21.15.175:0/1921253204 wait complete. 2024-03-23T09:40:47.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.811+0000 7f91c1be7640 1 Processor -- start 2024-03-23T09:40:47.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.811+0000 7f91c1be7640 1 -- start start 2024-03-23T09:40:47.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.811+0000 7f91c1be7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 0x7f91bc151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.811+0000 7f91c1be7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91bc0b28e0 0x7f91bc154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.811+0000 7f91c1be7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91bc0ce780 0x7f91bc155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:47.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91c1be7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc1524d0 con 0x7f91bc0b28e0 2024-03-23T09:40:47.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91c1be7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc06ab40 con 0x7f91bc0ce780 2024-03-23T09:40:47.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91c1be7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc15aef0 con 0x7f91bc06f210 2024-03-23T09:40:47.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91bb7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 0x7f91bc151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91bb7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 0x7f91bc151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:46866/0 (socket says 172.21.15.175:46866) 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91bb7fe640 1 -- 172.21.15.175:0/1152525302 learned_addr learned my addr 172.21.15.175:0/1152525302 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91bbfff640 1 --2- 172.21.15.175:0/1152525302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91bc0ce780 0x7f91bc155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91baffd640 1 --2- 172.21.15.175:0/1152525302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91bc0b28e0 0x7f91bc154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91bb7fe640 1 -- 172.21.15.175:0/1152525302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91bc0ce780 msgr2=0x7f91bc155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91bb7fe640 1 --2- 172.21.15.175:0/1152525302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91bc0ce780 0x7f91bc155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91bb7fe640 1 -- 172.21.15.175:0/1152525302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91bc0b28e0 msgr2=0x7f91bc154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91bb7fe640 1 --2- 172.21.15.175:0/1152525302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91bc0b28e0 0x7f91bc154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91bb7fe640 1 -- 172.21.15.175:0/1152525302 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b00094d0 con 0x7f91bc06f210 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91bbfff640 1 --2- 172.21.15.175:0/1152525302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91bc0ce780 0x7f91bc155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91bb7fe640 1 --2- 172.21.15.175:0/1152525302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 0x7f91bc151f90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f91b0000c00 tx=0x7f91b0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91a3fff640 1 -- 172.21.15.175:0/1152525302 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b00307f0 con 0x7f91bc06f210 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91a3fff640 1 -- 172.21.15.175:0/1152525302 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b0044480 con 0x7f91bc06f210 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91c1be7640 1 -- 172.21.15.175:0/1152525302 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91bc0cb1f0 con 0x7f91bc06f210 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91a3fff640 1 -- 172.21.15.175:0/1152525302 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0044830 con 0x7f91bc06f210 2024-03-23T09:40:47.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91c1be7640 1 -- 172.21.15.175:0/1152525302 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc0cb7b0 con 0x7f91bc06f210 2024-03-23T09:40:47.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.812+0000 7f91a3fff640 1 -- 172.21.15.175:0/1152525302 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f91b0044c90 con 0x7f91bc06f210 2024-03-23T09:40:47.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.813+0000 7f91a3fff640 1 -- 172.21.15.175:0/1152525302 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f91b003e070 con 0x7f91bc06f210 2024-03-23T09:40:47.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.814+0000 7f91c1be7640 1 -- 172.21.15.175:0/1152525302 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91bc073530 con 0x7f91bc06f210 2024-03-23T09:40:47.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.816+0000 7f91a3fff640 1 -- 172.21.15.175:0/1152525302 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f91b0049070 con 0x7f91bc06f210 2024-03-23T09:40:47.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.910+0000 7f91c1be7640 1 -- 172.21.15.175:0/1152525302 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b6112e8d-e0eb-4b16-bff7-465d2013a7ee", "id": 5} v 0) v1 -- 0x7f91bc0672f0 con 0x7f91bc06f210 2024-03-23T09:40:47.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.913+0000 7f91a3fff640 1 -- 172.21.15.175:0/1152525302 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b6112e8d-e0eb-4b16-bff7-465d2013a7ee", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f91b0034070 con 0x7f91bc06f210 2024-03-23T09:40:47.912 INFO:teuthology.orchestra.run.smithi175.stdout:5 2024-03-23T09:40:47.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.914+0000 7f91c1be7640 1 -- 172.21.15.175:0/1152525302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 msgr2=0x7f91bc151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:47.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.914+0000 7f91c1be7640 1 --2- 172.21.15.175:0/1152525302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 0x7f91bc151f90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f91b0000c00 tx=0x7f91b0004730 comp rx=0 tx=0).stop 2024-03-23T09:40:47.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.914+0000 7f91c1be7640 1 -- 172.21.15.175:0/1152525302 shutdown_connections 2024-03-23T09:40:47.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.914+0000 7f91c1be7640 1 --2- 172.21.15.175:0/1152525302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91bc0ce780 0x7f91bc155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.914+0000 7f91c1be7640 1 --2- 172.21.15.175:0/1152525302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91bc0b28e0 0x7f91bc154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.914+0000 7f91c1be7640 1 --2- 172.21.15.175:0/1152525302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91bc06f210 0x7f91bc151f90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:47.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.914+0000 7f91c1be7640 1 -- 172.21.15.175:0/1152525302 >> 172.21.15.175:0/1152525302 conn(0x7f91bc071400 msgr2=0x7f91bc0c54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:47.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.914+0000 7f91c1be7640 1 -- 172.21.15.175:0/1152525302 shutdown_connections 2024-03-23T09:40:47.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:47.914+0000 7f91c1be7640 1 -- 172.21.15.175:0/1152525302 wait complete. 2024-03-23T09:40:47.923 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 4770eeee-358c-49ba-bf13-f4d5ef6c9e5b 6 2024-03-23T09:40:48.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.024+0000 7f86bb873640 1 Processor -- start 2024-03-23T09:40:48.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.024+0000 7f86bb873640 1 -- start start 2024-03-23T09:40:48.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86bb873640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86b415cc50 0x7f86b40d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86bb873640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f86b406eb50 0x7f86b40cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86bb873640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b40cdd40 con 0x7f86b406ef30 2024-03-23T09:40:48.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86bb873640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b40cafe0 con 0x7f86b415cc50 2024-03-23T09:40:48.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86bb873640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b406e590 con 0x7f86b406eb50 2024-03-23T09:40:48.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86b8e0e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f86b406eb50 0x7f86b40cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86b8e0e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f86b406eb50 0x7f86b40cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50042/0 (socket says 172.21.15.175:50042) 2024-03-23T09:40:48.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86b8e0e640 1 -- 172.21.15.175:0/3855008842 learned_addr learned my addr 172.21.15.175:0/3855008842 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-23T09:40:48.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86b960f640 1 --2- 172.21.15.175:0/3855008842 >> v2:172.21.15.47:3300/0 conn(0x7f86b415cc50 0x7f86b40d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86b8e0e640 1 -- 172.21.15.175:0/3855008842 >> v2:172.21.15.47:3300/0 conn(0x7f86b415cc50 msgr2=0x7f86b40d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:40:48.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86b8e0e640 1 --2- 172.21.15.175:0/3855008842 >> v2:172.21.15.47:3300/0 conn(0x7f86b415cc50 0x7f86b40d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86b8e0e640 1 -- 172.21.15.175:0/3855008842 >> v1:172.21.15.47:6789/0 conn(0x7f86b406ef30 legacy=0x7f86b406d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86abfff640 1 -- 172.21.15.175:0/3855008842 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626761314 0 0) 0x7f86b40cdd40 con 0x7f86b406ef30 2024-03-23T09:40:48.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.025+0000 7f86b8e0e640 1 -- 172.21.15.175:0/3855008842 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b406e870 con 0x7f86b406eb50 2024-03-23T09:40:48.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.026+0000 7f86b8e0e640 1 --2- 172.21.15.175:0/3855008842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86b406eb50 0x7f86b40cbbc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f86b40cd8b0 tx=0x7f86a402ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f5819f124c8b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:48.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.026+0000 7f86abfff640 1 -- 172.21.15.175:0/3855008842 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a402f850 con 0x7f86b406eb50 2024-03-23T09:40:48.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.026+0000 7f86abfff640 1 -- 172.21.15.175:0/3855008842 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a402f9b0 con 0x7f86b406eb50 2024-03-23T09:40:48.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.026+0000 7f86abfff640 1 -- 172.21.15.175:0/3855008842 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a402fd40 con 0x7f86b406eb50 2024-03-23T09:40:48.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.027+0000 7f86bb873640 1 -- 172.21.15.175:0/3855008842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86b406eb50 msgr2=0x7f86b40cbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.027+0000 7f86bb873640 1 --2- 172.21.15.175:0/3855008842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86b406eb50 0x7f86b40cbbc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f86b40cd8b0 tx=0x7f86a402ba30 comp rx=0 tx=0).stop 2024-03-23T09:40:48.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.027+0000 7f86bb873640 1 -- 172.21.15.175:0/3855008842 shutdown_connections 2024-03-23T09:40:48.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.027+0000 7f86bb873640 1 --2- 172.21.15.175:0/3855008842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86b406eb50 0x7f86b40cbbc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.027+0000 7f86bb873640 1 --2- 172.21.15.175:0/3855008842 >> v2:172.21.15.47:3300/0 conn(0x7f86b415cc50 0x7f86b40d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.027+0000 7f86bb873640 1 -- 172.21.15.175:0/3855008842 >> 172.21.15.175:0/3855008842 conn(0x7f86b4098a00 msgr2=0x7f86b40cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:48.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.027+0000 7f86bb873640 1 -- 172.21.15.175:0/3855008842 shutdown_connections 2024-03-23T09:40:48.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.027+0000 7f86bb873640 1 -- 172.21.15.175:0/3855008842 wait complete. 2024-03-23T09:40:48.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86bb873640 1 Processor -- start 2024-03-23T09:40:48.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86bb873640 1 -- start start 2024-03-23T09:40:48.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86bb873640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86b406eb50 0x7f86b40c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86bb873640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86b406ef30 0x7f86b40c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86bb873640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86b415cc50 0x7f86b40b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86bb873640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b40c3c90 con 0x7f86b415cc50 2024-03-23T09:40:48.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86bb873640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b40b4150 con 0x7f86b406ef30 2024-03-23T09:40:48.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86bb873640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b40b43f0 con 0x7f86b406eb50 2024-03-23T09:40:48.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86b8e0e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86b406ef30 0x7f86b40c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86b960f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86b406eb50 0x7f86b40c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86b8e0e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86b406ef30 0x7f86b40c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50048/0 (socket says 172.21.15.175:50048) 2024-03-23T09:40:48.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86b960f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86b406eb50 0x7f86b40c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:46878/0 (socket says 172.21.15.175:46878) 2024-03-23T09:40:48.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86b8e0e640 1 -- 172.21.15.175:0/122080312 learned_addr learned my addr 172.21.15.175:0/122080312 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-23T09:40:48.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.029+0000 7f86b9e10640 1 --2- 172.21.15.175:0/122080312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86b415cc50 0x7f86b40b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86b960f640 1 -- 172.21.15.175:0/122080312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86b406ef30 msgr2=0x7f86b40c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86b960f640 1 --2- 172.21.15.175:0/122080312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86b406ef30 0x7f86b40c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86b960f640 1 -- 172.21.15.175:0/122080312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86b415cc50 msgr2=0x7f86b40b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86b960f640 1 --2- 172.21.15.175:0/122080312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86b415cc50 0x7f86b40b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86b960f640 1 -- 172.21.15.175:0/122080312 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a40094d0 con 0x7f86b406eb50 2024-03-23T09:40:48.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86b8e0e640 1 --2- 172.21.15.175:0/122080312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86b406ef30 0x7f86b40c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:40:48.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86b9e10640 1 --2- 172.21.15.175:0/122080312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86b415cc50 0x7f86b40b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:40:48.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86b960f640 1 --2- 172.21.15.175:0/122080312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86b406eb50 0x7f86b40c5e20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f86b000e870 tx=0x7f86b000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:48.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86a9ffb640 1 -- 172.21.15.175:0/122080312 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b000ee60 con 0x7f86b406eb50 2024-03-23T09:40:48.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86a9ffb640 1 -- 172.21.15.175:0/122080312 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b000c940 con 0x7f86b406eb50 2024-03-23T09:40:48.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86bb873640 1 -- 172.21.15.175:0/122080312 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b40ce350 con 0x7f86b406eb50 2024-03-23T09:40:48.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86a9ffb640 1 -- 172.21.15.175:0/122080312 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b000ccd0 con 0x7f86b406eb50 2024-03-23T09:40:48.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86bb873640 1 -- 172.21.15.175:0/122080312 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b40b49b0 con 0x7f86b406eb50 2024-03-23T09:40:48.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.030+0000 7f86a9ffb640 1 -- 172.21.15.175:0/122080312 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f86b0005c90 con 0x7f86b406eb50 2024-03-23T09:40:48.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.031+0000 7f86a9ffb640 1 -- 172.21.15.175:0/122080312 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f86b00067a0 con 0x7f86b406eb50 2024-03-23T09:40:48.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.031+0000 7f86bb873640 1 -- 172.21.15.175:0/122080312 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b40c3a60 con 0x7f86b406eb50 2024-03-23T09:40:48.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.033+0000 7f86a9ffb640 1 -- 172.21.15.175:0/122080312 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f86b40c3a60 con 0x7f86b406eb50 2024-03-23T09:40:48.063 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:48.064+0000 7ff054a10040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T09:40:48.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.105+0000 7f86bb873640 1 -- 172.21.15.175:0/122080312 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4770eeee-358c-49ba-bf13-f4d5ef6c9e5b", "id": 6} v 0) v1 -- 0x7f86b40672f0 con 0x7f86b406eb50 2024-03-23T09:40:48.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.108+0000 7f86a9ffb640 1 -- 172.21.15.175:0/122080312 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4770eeee-358c-49ba-bf13-f4d5ef6c9e5b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f86b0021060 con 0x7f86b406eb50 2024-03-23T09:40:48.108 INFO:teuthology.orchestra.run.smithi175.stdout:6 2024-03-23T09:40:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.109+0000 7f86bb873640 1 -- 172.21.15.175:0/122080312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86b406eb50 msgr2=0x7f86b40c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.109+0000 7f86bb873640 1 --2- 172.21.15.175:0/122080312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86b406eb50 0x7f86b40c5e20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f86b000e870 tx=0x7f86b000ed30 comp rx=0 tx=0).stop 2024-03-23T09:40:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.109+0000 7f86bb873640 1 -- 172.21.15.175:0/122080312 shutdown_connections 2024-03-23T09:40:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.109+0000 7f86bb873640 1 --2- 172.21.15.175:0/122080312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86b415cc50 0x7f86b40b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.109+0000 7f86bb873640 1 --2- 172.21.15.175:0/122080312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86b406ef30 0x7f86b40c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.109+0000 7f86bb873640 1 --2- 172.21.15.175:0/122080312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86b406eb50 0x7f86b40c5e20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.109+0000 7f86bb873640 1 -- 172.21.15.175:0/122080312 >> 172.21.15.175:0/122080312 conn(0x7f86b4098a00 msgr2=0x7f86b406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.109+0000 7f86bb873640 1 -- 172.21.15.175:0/122080312 shutdown_connections 2024-03-23T09:40:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.109+0000 7f86bb873640 1 -- 172.21.15.175:0/122080312 wait complete. 2024-03-23T09:40:48.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new fb4de25d-5bae-4e6e-9ece-54f206914efa 7 2024-03-23T09:40:48.176 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:48.177+0000 7fa059e3e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T09:40:48.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.219+0000 7f5a5dac4640 1 Processor -- start 2024-03-23T09:40:48.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.220+0000 7f5a5dac4640 1 -- start start 2024-03-23T09:40:48.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.220+0000 7f5a5dac4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a5815a8f0 0x7f5a58098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.220+0000 7f5a5dac4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5a5815e3c0 0x7f5a58098f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.220+0000 7f5a5dac4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a58099ce0 con 0x7f5a580ce2b0 2024-03-23T09:40:48.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.220+0000 7f5a5dac4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5815acd0 con 0x7f5a5815e3c0 2024-03-23T09:40:48.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.220+0000 7f5a5dac4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a580cc5c0 con 0x7f5a5815a8f0 2024-03-23T09:40:48.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.220+0000 7f5a577fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a5815a8f0 0x7f5a58098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.220+0000 7f5a57fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5a5815e3c0 0x7f5a58098f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.220+0000 7f5a577fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a5815a8f0 0x7f5a58098a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:46888/0 (socket says 172.21.15.175:46888) 2024-03-23T09:40:48.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.220+0000 7f5a57fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5a5815e3c0 0x7f5a58098f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50050/0 (socket says 172.21.15.175:50050) 2024-03-23T09:40:48.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.220+0000 7f5a577fe640 1 -- 172.21.15.175:0/4201532645 learned_addr learned my addr 172.21.15.175:0/4201532645 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-23T09:40:48.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.221+0000 7f5a577fe640 1 -- 172.21.15.175:0/4201532645 >> v2:172.21.15.175:3300/0 conn(0x7f5a5815e3c0 msgr2=0x7f5a58098f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:40:48.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.221+0000 7f5a577fe640 1 --2- 172.21.15.175:0/4201532645 >> v2:172.21.15.175:3300/0 conn(0x7f5a5815e3c0 0x7f5a58098f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.221+0000 7f5a577fe640 1 -- 172.21.15.175:0/4201532645 >> v1:172.21.15.175:6789/0 conn(0x7f5a580ce2b0 legacy=0x7f5a580ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.221+0000 7f5a577fe640 1 -- 172.21.15.175:0/4201532645 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a580cb520 con 0x7f5a5815a8f0 2024-03-23T09:40:48.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.221+0000 7f5a577fe640 1 --2- 172.21.15.175:0/4201532645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a5815a8f0 0x7f5a58098a00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a480098a0 tx=0x7f5a4802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c18cf9e116be6db server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:48.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.222+0000 7f5a567fc640 1 -- 172.21.15.175:0/4201532645 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a48030840 con 0x7f5a5815a8f0 2024-03-23T09:40:48.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.222+0000 7f5a567fc640 1 -- 172.21.15.175:0/4201532645 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a480309a0 con 0x7f5a5815a8f0 2024-03-23T09:40:48.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.222+0000 7f5a567fc640 1 -- 172.21.15.175:0/4201532645 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a48030d30 con 0x7f5a5815a8f0 2024-03-23T09:40:48.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.222+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4201532645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a5815a8f0 msgr2=0x7f5a58098a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.222+0000 7f5a5dac4640 1 --2- 172.21.15.175:0/4201532645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a5815a8f0 0x7f5a58098a00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a480098a0 tx=0x7f5a4802b980 comp rx=0 tx=0).stop 2024-03-23T09:40:48.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.222+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4201532645 shutdown_connections 2024-03-23T09:40:48.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.222+0000 7f5a5dac4640 1 --2- 172.21.15.175:0/4201532645 >> v2:172.21.15.175:3300/0 conn(0x7f5a5815e3c0 0x7f5a58098f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.222+0000 7f5a5dac4640 1 --2- 172.21.15.175:0/4201532645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a5815a8f0 0x7f5a58098a00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.222+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4201532645 >> 172.21.15.175:0/4201532645 conn(0x7f5a580b21d0 msgr2=0x7f5a580b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:48.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.222+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4201532645 shutdown_connections 2024-03-23T09:40:48.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.222+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4201532645 wait complete. 2024-03-23T09:40:48.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.223+0000 7f5a5dac4640 1 Processor -- start 2024-03-23T09:40:48.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.224+0000 7f5a5dac4640 1 -- start start 2024-03-23T09:40:48.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.224+0000 7f5a5dac4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a580ce2b0 0x7f5a5815c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.224+0000 7f5a5dac4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a5815a8f0 0x7f5a5815c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.224+0000 7f5a5dac4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a5815e3c0 0x7f5a580c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.224+0000 7f5a5dac4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a580ca140 con 0x7f5a5815e3c0 2024-03-23T09:40:48.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.224+0000 7f5a5dac4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5815cd50 con 0x7f5a5815a8f0 2024-03-23T09:40:48.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.224+0000 7f5a5dac4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a580c4340 con 0x7f5a580ce2b0 2024-03-23T09:40:48.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.224+0000 7f5a577fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a580ce2b0 0x7f5a5815c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.224+0000 7f5a56ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a5815a8f0 0x7f5a5815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.224+0000 7f5a577fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a580ce2b0 0x7f5a5815c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:46902/0 (socket says 172.21.15.175:46902) 2024-03-23T09:40:48.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a577fe640 1 -- 172.21.15.175:0/4017751421 learned_addr learned my addr 172.21.15.175:0/4017751421 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-23T09:40:48.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a57fff640 1 --2- 172.21.15.175:0/4017751421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a5815e3c0 0x7f5a580c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a577fe640 1 -- 172.21.15.175:0/4017751421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a5815a8f0 msgr2=0x7f5a5815c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a577fe640 1 --2- 172.21.15.175:0/4017751421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a5815a8f0 0x7f5a5815c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a577fe640 1 -- 172.21.15.175:0/4017751421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a5815e3c0 msgr2=0x7f5a580c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a577fe640 1 --2- 172.21.15.175:0/4017751421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a5815e3c0 0x7f5a580c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a577fe640 1 -- 172.21.15.175:0/4017751421 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a480094d0 con 0x7f5a580ce2b0 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a577fe640 1 --2- 172.21.15.175:0/4017751421 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a580ce2b0 0x7f5a5815c2d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5a480097c0 tx=0x7f5a48004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a3ffff640 1 -- 172.21.15.175:0/4017751421 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a48039a90 con 0x7f5a580ce2b0 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a3ffff640 1 -- 172.21.15.175:0/4017751421 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a48039bf0 con 0x7f5a580ce2b0 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4017751421 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a580c4620 con 0x7f5a580ce2b0 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a3ffff640 1 -- 172.21.15.175:0/4017751421 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a48043410 con 0x7f5a580ce2b0 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4017751421 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a580c4be0 con 0x7f5a580ce2b0 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a3ffff640 1 -- 172.21.15.175:0/4017751421 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5a48043870 con 0x7f5a580ce2b0 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.225+0000 7f5a3ffff640 1 -- 172.21.15.175:0/4017751421 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f5a4804ca10 con 0x7f5a580ce2b0 2024-03-23T09:40:48.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.226+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4017751421 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a580cbac0 con 0x7f5a580ce2b0 2024-03-23T09:40:48.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.230+0000 7f5a3ffff640 1 -- 172.21.15.175:0/4017751421 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f5a580cbac0 con 0x7f5a580ce2b0 2024-03-23T09:40:48.233 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:48.233+0000 7ff054a10040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T09:40:48.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.301+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4017751421 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fb4de25d-5bae-4e6e-9ece-54f206914efa", "id": 7} v 0) v1 -- 0x7f5a580672f0 con 0x7f5a580ce2b0 2024-03-23T09:40:48.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.309+0000 7f5a3ffff640 1 -- 172.21.15.175:0/4017751421 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fb4de25d-5bae-4e6e-9ece-54f206914efa", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5a48048070 con 0x7f5a580ce2b0 2024-03-23T09:40:48.309 INFO:teuthology.orchestra.run.smithi175.stdout:7 2024-03-23T09:40:48.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.310+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4017751421 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a580ce2b0 msgr2=0x7f5a5815c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.310+0000 7f5a5dac4640 1 --2- 172.21.15.175:0/4017751421 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a580ce2b0 0x7f5a5815c2d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5a480097c0 tx=0x7f5a48004730 comp rx=0 tx=0).stop 2024-03-23T09:40:48.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.310+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4017751421 shutdown_connections 2024-03-23T09:40:48.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.310+0000 7f5a5dac4640 1 --2- 172.21.15.175:0/4017751421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a5815e3c0 0x7f5a580c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.310+0000 7f5a5dac4640 1 --2- 172.21.15.175:0/4017751421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a5815a8f0 0x7f5a5815c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.310+0000 7f5a5dac4640 1 --2- 172.21.15.175:0/4017751421 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a580ce2b0 0x7f5a5815c2d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.310+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4017751421 >> 172.21.15.175:0/4017751421 conn(0x7f5a580b21d0 msgr2=0x7f5a5806e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:48.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.310+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4017751421 shutdown_connections 2024-03-23T09:40:48.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-23T09:40:48.310+0000 7f5a5dac4640 1 -- 172.21.15.175:0/4017751421 wait complete. 2024-03-23T09:40:48.318 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:48.319+0000 7fa059e3e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T09:40:48.320 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-23T09:40:48.320 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-23T09:40:48.322 INFO:tasks.ceph.osd.0:Started 2024-03-23T09:40:48.322 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-23T09:40:48.323 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-23T09:40:48.325 INFO:tasks.ceph.osd.1:Started 2024-03-23T09:40:48.325 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-23T09:40:48.325 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-23T09:40:48.327 INFO:tasks.ceph.osd.2:Started 2024-03-23T09:40:48.327 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-23T09:40:48.327 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-23T09:40:48.329 INFO:tasks.ceph.osd.3:Started 2024-03-23T09:40:48.329 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-23T09:40:48.329 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-23T09:40:48.331 INFO:tasks.ceph.osd.4:Started 2024-03-23T09:40:48.332 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-23T09:40:48.332 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-23T09:40:48.334 INFO:tasks.ceph.osd.5:Started 2024-03-23T09:40:48.334 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-23T09:40:48.334 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-23T09:40:48.336 INFO:tasks.ceph.osd.6:Started 2024-03-23T09:40:48.336 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-23T09:40:48.336 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-23T09:40:48.338 INFO:tasks.ceph.osd.7:Started 2024-03-23T09:40:48.338 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:40:48.361 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:48.362+0000 7ff054a10040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T09:40:48.428 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:48.429+0000 7ff054a10040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T09:40:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.458+0000 7febee484640 1 Processor -- start 2024-03-23T09:40:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.458+0000 7febee484640 1 -- start start 2024-03-23T09:40:48.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.458+0000 7febee484640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe815a8d0 0x7febe80cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.459+0000 7febee484640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7febe80cf360 0x7febe80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.459+0000 7febee484640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe80cc610 con 0x7febe80cb730 2024-03-23T09:40:48.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.459+0000 7febee484640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe80c41c0 con 0x7febe80cf360 2024-03-23T09:40:48.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.459+0000 7febee484640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe80b0700 con 0x7febe815a8d0 2024-03-23T09:40:48.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.459+0000 7febe7fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe815a8d0 0x7febe80cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.459+0000 7febeca21640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7febe80cf360 0x7febe80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.459+0000 7febeca21640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7febe80cf360 0x7febe80b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:43328/0 (socket says 172.21.15.47:43328) 2024-03-23T09:40:48.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.459+0000 7febeca21640 1 -- 172.21.15.47:0/1949631838 learned_addr learned my addr 172.21.15.47:0/1949631838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:40:48.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.459+0000 7febeca21640 1 -- 172.21.15.47:0/1949631838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe815a8d0 msgr2=0x7febe80cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:40:48.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.459+0000 7febeca21640 1 --2- 172.21.15.47:0/1949631838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe815a8d0 0x7febe80cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.459+0000 7febeca21640 1 -- 172.21.15.47:0/1949631838 >> v1:172.21.15.175:6789/0 conn(0x7febe80cb730 legacy=0x7febe80cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.459+0000 7febeca21640 1 -- 172.21.15.47:0/1949631838 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe80cbd70 con 0x7febe80cf360 2024-03-23T09:40:48.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.459+0000 7febe7fff640 1 --2- 172.21.15.47:0/1949631838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe815a8d0 0x7febe80cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:40:48.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.460+0000 7febeca21640 1 --2- 172.21.15.47:0/1949631838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe80cf360 0x7febe80b0b90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7febe0009e50 tx=0x7febe002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3eae10cb3128118 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:48.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.460+0000 7febe6ffd640 1 -- 172.21.15.47:0/1949631838 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe002ca70 con 0x7febe80cf360 2024-03-23T09:40:48.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.460+0000 7febe6ffd640 1 -- 172.21.15.47:0/1949631838 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe002cbd0 con 0x7febe80cf360 2024-03-23T09:40:48.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.460+0000 7febe6ffd640 1 -- 172.21.15.47:0/1949631838 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe00343f0 con 0x7febe80cf360 2024-03-23T09:40:48.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.461+0000 7febee484640 1 -- 172.21.15.47:0/1949631838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe80cf360 msgr2=0x7febe80b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.461+0000 7febee484640 1 --2- 172.21.15.47:0/1949631838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe80cf360 0x7febe80b0b90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7febe0009e50 tx=0x7febe002bed0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.461+0000 7febee484640 1 -- 172.21.15.47:0/1949631838 shutdown_connections 2024-03-23T09:40:48.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.461+0000 7febee484640 1 --2- 172.21.15.47:0/1949631838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe80cf360 0x7febe80b0b90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.461+0000 7febee484640 1 --2- 172.21.15.47:0/1949631838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe815a8d0 0x7febe80cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.461+0000 7febee484640 1 -- 172.21.15.47:0/1949631838 >> 172.21.15.47:0/1949631838 conn(0x7febe80d0670 msgr2=0x7febe80d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:48.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.461+0000 7febee484640 1 -- 172.21.15.47:0/1949631838 shutdown_connections 2024-03-23T09:40:48.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.461+0000 7febee484640 1 -- 172.21.15.47:0/1949631838 wait complete. 2024-03-23T09:40:48.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.463+0000 7febee484640 1 Processor -- start 2024-03-23T09:40:48.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.463+0000 7febee484640 1 -- start start 2024-03-23T09:40:48.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.464+0000 7febee484640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe80cb730 0x7febe80c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.464+0000 7febee484640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febe80cf360 0x7febe8158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.464+0000 7febee484640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe815a8d0 0x7febe806bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:48.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.464+0000 7febee484640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe80af2b0 con 0x7febe80cf360 2024-03-23T09:40:48.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.464+0000 7febee484640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe80af590 con 0x7febe80cb730 2024-03-23T09:40:48.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.464+0000 7febee484640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe80af870 con 0x7febe815a8d0 2024-03-23T09:40:48.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.464+0000 7febe77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febe80cf360 0x7febe8158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.464+0000 7febe77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febe80cf360 0x7febe8158b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36324/0 (socket says 172.21.15.47:36324) 2024-03-23T09:40:48.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.464+0000 7febe77fe640 1 -- 172.21.15.47:0/1197504751 learned_addr learned my addr 172.21.15.47:0/1197504751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:40:48.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.464+0000 7febe7fff640 1 --2- 172.21.15.47:0/1197504751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe80cb730 0x7febe80c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.464+0000 7febeca21640 1 --2- 172.21.15.47:0/1197504751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe815a8d0 0x7febe806bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:48.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febe7fff640 1 -- 172.21.15.47:0/1197504751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe815a8d0 msgr2=0x7febe806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febe7fff640 1 --2- 172.21.15.47:0/1197504751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe815a8d0 0x7febe806bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febe7fff640 1 -- 172.21.15.47:0/1197504751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febe80cf360 msgr2=0x7febe8158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febe7fff640 1 --2- 172.21.15.47:0/1197504751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febe80cf360 0x7febe8158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febe7fff640 1 -- 172.21.15.47:0/1197504751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febdc00d8a0 con 0x7febe80cb730 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febe77fe640 1 --2- 172.21.15.47:0/1197504751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febe80cf360 0x7febe8158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febe7fff640 1 --2- 172.21.15.47:0/1197504751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe80cb730 0x7febe80c9910 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7febdc00bed0 tx=0x7febdc00f480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febe4ff9640 1 -- 172.21.15.47:0/1197504751 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc00fd90 con 0x7febe80cb730 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febe4ff9640 1 -- 172.21.15.47:0/1197504751 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc00d2e0 con 0x7febe80cb730 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febe4ff9640 1 -- 172.21.15.47:0/1197504751 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc00dd80 con 0x7febe80cb730 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febee484640 1 -- 172.21.15.47:0/1197504751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe0009b30 con 0x7febe80cb730 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febee484640 1 -- 172.21.15.47:0/1197504751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe815bca0 con 0x7febe80cb730 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febe4ff9640 1 -- 172.21.15.47:0/1197504751 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7febdc014070 con 0x7febe80cb730 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.465+0000 7febe4ff9640 1 -- 172.21.15.47:0/1197504751 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7febdc005c90 con 0x7febe80cb730 2024-03-23T09:40:48.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.466+0000 7febee484640 1 -- 172.21.15.47:0/1197504751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe815c210 con 0x7febe80cb730 2024-03-23T09:40:48.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.469+0000 7febe4ff9640 1 -- 172.21.15.47:0/1197504751 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7febe815c210 con 0x7febe80cb730 2024-03-23T09:40:48.469 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:48.470+0000 7fa059e3e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T09:40:48.514 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:48.515+0000 7fa059e3e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T09:40:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.530+0000 7febee484640 1 -- 172.21.15.47:0/1197504751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febe80672f0 con 0x7febe80cb730 2024-03-23T09:40:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.531+0000 7febe4ff9640 1 -- 172.21.15.47:0/1197504751 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7febdc01b070 con 0x7febe80cb730 2024-03-23T09:40:48.530 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:48.531 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":10,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:40:48.303352+0000","last_up_change":"0.000000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:40:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.531+0000 7febee484640 1 -- 172.21.15.47:0/1197504751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe80cb730 msgr2=0x7febe80c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.531+0000 7febee484640 1 --2- 172.21.15.47:0/1197504751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe80cb730 0x7febe80c9910 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7febdc00bed0 tx=0x7febdc00f480 comp rx=0 tx=0).stop 2024-03-23T09:40:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.532+0000 7febee484640 1 -- 172.21.15.47:0/1197504751 shutdown_connections 2024-03-23T09:40:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.532+0000 7febee484640 1 --2- 172.21.15.47:0/1197504751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe815a8d0 0x7febe806bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.532+0000 7febee484640 1 --2- 172.21.15.47:0/1197504751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febe80cf360 0x7febe8158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.532+0000 7febee484640 1 --2- 172.21.15.47:0/1197504751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe80cb730 0x7febe80c9910 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.532+0000 7febee484640 1 -- 172.21.15.47:0/1197504751 >> 172.21.15.47:0/1197504751 conn(0x7febe80d0670 msgr2=0x7febe80cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.532+0000 7febee484640 1 -- 172.21.15.47:0/1197504751 shutdown_connections 2024-03-23T09:40:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:48.532+0000 7febee484640 1 -- 172.21.15.47:0/1197504751 wait complete. 2024-03-23T09:40:48.542 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-23T09:40:48.542 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-23T09:40:48.548 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:48.549+0000 7ff054a10040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T09:40:48.624 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:48.625+0000 7fa059e3e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T09:40:48.646 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:48.647+0000 7ff054a10040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T09:40:48.719 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:48.720+0000 7fa059e3e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T09:40:48.805 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:48.806+0000 7ff054a10040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T09:40:48.849 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:48.849+0000 7ff054a10040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T09:40:48.870 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:48.871+0000 7fa059e3e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T09:40:48.911 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:48.912+0000 7fa059e3e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T09:40:48.931 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-03-23T09:40:48.931+0000 7fec18c144c0 -1 Falling back to public interface 2024-03-23T09:40:48.933 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-03-23T09:40:48.933+0000 7f71d4a1f4c0 -1 Falling back to public interface 2024-03-23T09:40:48.936 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-03-23T09:40:48.937+0000 7f54ebc124c0 -1 Falling back to public interface 2024-03-23T09:40:48.943 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-03-23T09:40:48.943+0000 7f07d24224c0 -1 Falling back to public interface 2024-03-23T09:40:48.944 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-03-23T09:40:48.945+0000 7f23f34134c0 -1 Falling back to public interface 2024-03-23T09:40:48.953 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-03-23T09:40:48.953+0000 7f17aa40d4c0 -1 Falling back to public interface 2024-03-23T09:40:48.954 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-03-23T09:40:48.955+0000 7fce2961d4c0 -1 Falling back to public interface 2024-03-23T09:40:48.956 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-03-23T09:40:48.957+0000 7fbc041824c0 -1 Falling back to public interface 2024-03-23T09:40:49.010 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:49.010+0000 7ff054a10040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T09:40:49.073 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:49.074+0000 7fa059e3e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T09:40:49.348 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-23T09:40:49.438 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.438+0000 7f413f48f640 1 Processor -- start 2024-03-23T09:40:49.438 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.438+0000 7f413f48f640 1 -- start start 2024-03-23T09:40:49.439 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.439+0000 7f413f48f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4138099110 0x7f413806c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:49.439 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.439+0000 7f413f48f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41380cfb10 con 0x7f413815ce80 2024-03-23T09:40:49.439 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.439+0000 7f413f48f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41380d0300 con 0x7f413806b000 2024-03-23T09:40:49.439 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.439+0000 7f413f48f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41380cb240 con 0x7f4138099110 2024-03-23T09:40:49.439 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.439+0000 7f413ca2a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4138099110 0x7f413806c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:49.439 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.439+0000 7f413d22b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f413815ce80 0x7f4138098a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58464/0 (socket says 172.21.15.47:58464) 2024-03-23T09:40:49.439 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.439+0000 7f413d22b640 1 -- 172.21.15.47:0/1053615996 learned_addr learned my addr 172.21.15.47:0/1053615996 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:40:49.440 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.440+0000 7f413ca2a640 1 -- 172.21.15.47:0/1053615996 >> v1:172.21.15.175:6789/0 conn(0x7f413806b000 legacy=0x7f41380cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:49.440 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.440+0000 7f413ca2a640 1 -- 172.21.15.47:0/1053615996 >> v1:172.21.15.47:6789/0 conn(0x7f413815ce80 legacy=0x7f4138098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:49.440 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.440+0000 7f413ca2a640 1 -- 172.21.15.47:0/1053615996 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41380cfd40 con 0x7f4138099110 2024-03-23T09:40:49.441 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.440+0000 7f413ca2a640 1 --2- 172.21.15.47:0/1053615996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4138099110 0x7f413806c0f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f41280097a0 tx=0x7f412802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e443037846d2a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:49.441 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.440+0000 7f412ffff640 1 -- 172.21.15.47:0/1053615996 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128031830 con 0x7f4138099110 2024-03-23T09:40:49.441 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.441+0000 7f412ffff640 1 -- 172.21.15.47:0/1053615996 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4128031990 con 0x7f4138099110 2024-03-23T09:40:49.441 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.441+0000 7f412ffff640 1 -- 172.21.15.47:0/1053615996 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128031d20 con 0x7f4138099110 2024-03-23T09:40:49.441 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.441+0000 7f413f48f640 1 -- 172.21.15.47:0/1053615996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4138099110 msgr2=0x7f413806c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:49.441 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.441+0000 7f413f48f640 1 --2- 172.21.15.47:0/1053615996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4138099110 0x7f413806c0f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f41280097a0 tx=0x7f412802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:40:49.441 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.441+0000 7f413f48f640 1 -- 172.21.15.47:0/1053615996 shutdown_connections 2024-03-23T09:40:49.442 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.441+0000 7f413f48f640 1 --2- 172.21.15.47:0/1053615996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4138099110 0x7f413806c0f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:49.442 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.441+0000 7f413f48f640 1 -- 172.21.15.47:0/1053615996 >> 172.21.15.47:0/1053615996 conn(0x7f41380b21f0 msgr2=0x7f41380b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:49.442 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.441+0000 7f413f48f640 1 -- 172.21.15.47:0/1053615996 shutdown_connections 2024-03-23T09:40:49.442 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.441+0000 7f413f48f640 1 -- 172.21.15.47:0/1053615996 wait complete. 2024-03-23T09:40:49.444 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.442+0000 7f413f48f640 1 Processor -- start 2024-03-23T09:40:49.444 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.442+0000 7f413f48f640 1 -- start start 2024-03-23T09:40:49.444 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.443+0000 7f413f48f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f413806b000 0x7f413814ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:49.445 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.443+0000 7f413f48f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4138099110 0x7f41380c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:49.445 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.443+0000 7f413f48f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f413815ce80 0x7f41380b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:49.445 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.443+0000 7f413f48f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41380c91e0 con 0x7f413806b000 2024-03-23T09:40:49.445 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.443+0000 7f413f48f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41380c8140 con 0x7f4138099110 2024-03-23T09:40:49.445 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.443+0000 7f413f48f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41380c8420 con 0x7f413815ce80 2024-03-23T09:40:49.445 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.443+0000 7f413d22b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f413806b000 0x7f413814ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:49.445 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.443+0000 7f413d22b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f413806b000 0x7f413814ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36346/0 (socket says 172.21.15.47:36346) 2024-03-23T09:40:49.445 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.443+0000 7f413d22b640 1 -- 172.21.15.47:0/2479310175 learned_addr learned my addr 172.21.15.47:0/2479310175 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:40:49.445 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.443+0000 7f413ca2a640 1 --2- 172.21.15.47:0/2479310175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4138099110 0x7f41380c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:49.445 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.443+0000 7f413da2c640 1 --2- 172.21.15.47:0/2479310175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f413815ce80 0x7f41380b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:49.445 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.444+0000 7f413d22b640 1 -- 172.21.15.47:0/2479310175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f413815ce80 msgr2=0x7f41380b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:49.446 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.444+0000 7f413d22b640 1 --2- 172.21.15.47:0/2479310175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f413815ce80 0x7f41380b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:49.446 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.444+0000 7f413d22b640 1 -- 172.21.15.47:0/2479310175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4138099110 msgr2=0x7f41380c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:49.446 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.444+0000 7f413d22b640 1 --2- 172.21.15.47:0/2479310175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4138099110 0x7f41380c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:49.446 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.444+0000 7f413d22b640 1 -- 172.21.15.47:0/2479310175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4128009400 con 0x7f413806b000 2024-03-23T09:40:49.446 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.444+0000 7f413d22b640 1 --2- 172.21.15.47:0/2479310175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f413806b000 0x7f413814ca30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f413400c480 tx=0x7f413400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:49.446 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.444+0000 7f412dffb640 1 -- 172.21.15.47:0/2479310175 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134005ce0 con 0x7f413806b000 2024-03-23T09:40:49.446 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.444+0000 7f412dffb640 1 -- 172.21.15.47:0/2479310175 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4134005e40 con 0x7f413806b000 2024-03-23T09:40:49.446 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.444+0000 7f413f48f640 1 -- 172.21.15.47:0/2479310175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41380c8700 con 0x7f413806b000 2024-03-23T09:40:49.446 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.444+0000 7f412dffb640 1 -- 172.21.15.47:0/2479310175 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41340072f0 con 0x7f413806b000 2024-03-23T09:40:49.446 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.444+0000 7f412dffb640 1 -- 172.21.15.47:0/2479310175 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4134012400 con 0x7f413806b000 2024-03-23T09:40:49.446 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.444+0000 7f413f48f640 1 -- 172.21.15.47:0/2479310175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41380c8cc0 con 0x7f413806b000 2024-03-23T09:40:49.446 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.444+0000 7f412dffb640 1 -- 172.21.15.47:0/2479310175 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4134012d80 con 0x7f413806b000 2024-03-23T09:40:49.446 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.445+0000 7f413f48f640 1 -- 172.21.15.47:0/2479310175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41380cb660 con 0x7f413806b000 2024-03-23T09:40:49.447 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.447+0000 7f412dffb640 1 -- 172.21.15.47:0/2479310175 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f4134021060 con 0x7f413806b000 2024-03-23T09:40:49.508 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.509+0000 7f413f48f640 1 -- 172.21.15.47:0/2479310175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4138073ef0 con 0x7f413806b000 2024-03-23T09:40:49.508 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.509+0000 7f412dffb640 1 -- 172.21.15.47:0/2479310175 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4134017070 con 0x7f413806b000 2024-03-23T09:40:49.509 INFO:teuthology.misc.health.smithi047.stdout: 2024-03-23T09:40:49.509 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":10,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:40:48.303352+0000","last_up_change":"0.000000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:40:49.510 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.510+0000 7f413f48f640 1 -- 172.21.15.47:0/2479310175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f413806b000 msgr2=0x7f413814ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:49.510 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.510+0000 7f413f48f640 1 --2- 172.21.15.47:0/2479310175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f413806b000 0x7f413814ca30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f413400c480 tx=0x7f413400c940 comp rx=0 tx=0).stop 2024-03-23T09:40:49.511 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.510+0000 7f413f48f640 1 -- 172.21.15.47:0/2479310175 shutdown_connections 2024-03-23T09:40:49.511 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.510+0000 7f413f48f640 1 --2- 172.21.15.47:0/2479310175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f413815ce80 0x7f41380b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:49.511 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.510+0000 7f413f48f640 1 --2- 172.21.15.47:0/2479310175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4138099110 0x7f41380c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:49.511 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.510+0000 7f413f48f640 1 --2- 172.21.15.47:0/2479310175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f413806b000 0x7f413814ca30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:49.511 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.510+0000 7f413f48f640 1 -- 172.21.15.47:0/2479310175 >> 172.21.15.47:0/2479310175 conn(0x7f41380b21f0 msgr2=0x7f41380ad3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:49.511 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.510+0000 7f413f48f640 1 -- 172.21.15.47:0/2479310175 shutdown_connections 2024-03-23T09:40:49.511 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:49.510+0000 7f413f48f640 1 -- 172.21.15.47:0/2479310175 wait complete. 2024-03-23T09:40:49.518 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-23T09:40:49.877 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:49.878+0000 7fa059e3e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T09:40:49.891 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:49.879+0000 7ff054a10040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T09:40:50.149 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:50.149+0000 7fa059e3e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T09:40:50.160 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:50.161+0000 7ff054a10040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T09:40:50.198 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:50.199+0000 7fa059e3e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T09:40:50.213 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:50.214+0000 7ff054a10040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T09:40:50.313 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:50.313+0000 7fa059e3e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T09:40:50.333 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-23T09:40:50.334+0000 7ff054a10040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T09:40:51.340 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:51.341+0000 7f9ffac50640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-23T09:40:51.341 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-03-23T09:40:51.341+0000 7f9ffac50640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-23T09:40:52.428 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-03-23T09:40:52.429+0000 7f71d4a1f4c0 -1 osd.4 0 log_to_monitors true 2024-03-23T09:40:52.444 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-03-23T09:40:52.445+0000 7f54ebc124c0 -1 osd.6 0 log_to_monitors true 2024-03-23T09:40:52.449 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-03-23T09:40:52.450+0000 7f17aa40d4c0 -1 osd.5 0 log_to_monitors true 2024-03-23T09:40:52.460 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-03-23T09:40:52.461+0000 7fbc041824c0 -1 osd.7 0 log_to_monitors true 2024-03-23T09:40:52.467 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-03-23T09:40:52.467+0000 7fec18c144c0 -1 osd.2 0 log_to_monitors true 2024-03-23T09:40:52.472 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-03-23T09:40:52.473+0000 7f07d24224c0 -1 osd.3 0 log_to_monitors true 2024-03-23T09:40:52.477 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-03-23T09:40:52.478+0000 7fce2961d4c0 -1 osd.1 0 log_to_monitors true 2024-03-23T09:40:52.479 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-03-23T09:40:52.480+0000 7f23f34134c0 -1 osd.0 0 log_to_monitors true 2024-03-23T09:40:55.765 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-03-23T09:40:55.765+0000 7f23ee5b8640 -1 osd.0 0 waiting for initial osdmap 2024-03-23T09:40:55.778 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-03-23T09:40:55.779+0000 7fce247e6640 -1 osd.1 0 waiting for initial osdmap 2024-03-23T09:40:55.779 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-03-23T09:40:55.779+0000 7fec13dfb640 -1 osd.2 0 waiting for initial osdmap 2024-03-23T09:40:55.788 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-03-23T09:40:55.788+0000 7f07cd5ca640 -1 osd.3 0 waiting for initial osdmap 2024-03-23T09:40:55.790 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-03-23T09:40:55.791+0000 7f23e6beb640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T09:40:55.790 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-03-23T09:40:55.791+0000 7fce1ce19640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T09:40:55.791 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-03-23T09:40:55.791+0000 7fec0c42e640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T09:40:55.797 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-03-23T09:40:55.798+0000 7f07c5bfd640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T09:40:56.326 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-23T09:40:56.419 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.419+0000 7f6953f7d640 1 Processor -- start 2024-03-23T09:40:56.419 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.419+0000 7f6953f7d640 1 -- start start 2024-03-23T09:40:56.420 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.420+0000 7f6953f7d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f694c070cb0 0x7f694c0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:56.421 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.420+0000 7f6953f7d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694c157570 con 0x7f694c071e40 2024-03-23T09:40:56.421 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.420+0000 7f6953f7d640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694c071630 con 0x7f694c0ce9a0 2024-03-23T09:40:56.421 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.420+0000 7f6953f7d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694c15d710 con 0x7f694c070cb0 2024-03-23T09:40:56.421 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.420+0000 7f695251a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f694c070cb0 0x7f694c0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:56.421 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.420+0000 7f6951518640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f694c071e40 0x7f694c160e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34130/0 (socket says 172.21.15.47:34130) 2024-03-23T09:40:56.421 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.420+0000 7f6951518640 1 -- 172.21.15.47:0/18338401 learned_addr learned my addr 172.21.15.47:0/18338401 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:40:56.421 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.421+0000 7f695251a640 1 -- 172.21.15.47:0/18338401 >> v1:172.21.15.175:6789/0 conn(0x7f694c0ce9a0 legacy=0x7f694c15d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:56.421 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.421+0000 7f695251a640 1 -- 172.21.15.47:0/18338401 >> v1:172.21.15.47:6789/0 conn(0x7f694c071e40 legacy=0x7f694c160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:56.421 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.421+0000 7f695251a640 1 -- 172.21.15.47:0/18338401 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c0ced80 con 0x7f694c070cb0 2024-03-23T09:40:56.423 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.421+0000 7f695251a640 1 --2- 172.21.15.47:0/18338401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694c070cb0 0x7f694c0710f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6944009d90 tx=0x7f694402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaa0d6c1a087eff server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:56.423 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.422+0000 7f6950d17640 1 -- 172.21.15.47:0/18338401 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694402cab0 con 0x7f694c070cb0 2024-03-23T09:40:56.423 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.422+0000 7f6950d17640 1 -- 172.21.15.47:0/18338401 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694402cc10 con 0x7f694c070cb0 2024-03-23T09:40:56.424 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.422+0000 7f6950d17640 1 -- 172.21.15.47:0/18338401 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6944034440 con 0x7f694c070cb0 2024-03-23T09:40:56.424 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.422+0000 7f6953f7d640 1 -- 172.21.15.47:0/18338401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694c070cb0 msgr2=0x7f694c0710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:56.424 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.422+0000 7f6953f7d640 1 --2- 172.21.15.47:0/18338401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694c070cb0 0x7f694c0710f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6944009d90 tx=0x7f694402bd80 comp rx=0 tx=0).stop 2024-03-23T09:40:56.424 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.422+0000 7f6953f7d640 1 -- 172.21.15.47:0/18338401 shutdown_connections 2024-03-23T09:40:56.424 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.422+0000 7f6953f7d640 1 --2- 172.21.15.47:0/18338401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694c070cb0 0x7f694c0710f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.424 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.422+0000 7f6953f7d640 1 -- 172.21.15.47:0/18338401 >> 172.21.15.47:0/18338401 conn(0x7f694c0cbb30 msgr2=0x7f694c0cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:56.424 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.422+0000 7f6953f7d640 1 -- 172.21.15.47:0/18338401 shutdown_connections 2024-03-23T09:40:56.424 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.422+0000 7f6953f7d640 1 -- 172.21.15.47:0/18338401 wait complete. 2024-03-23T09:40:56.424 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.423+0000 7f6953f7d640 1 Processor -- start 2024-03-23T09:40:56.424 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.423+0000 7f6953f7d640 1 -- start start 2024-03-23T09:40:56.427 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6953f7d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694c070cb0 0x7f694c14e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:56.428 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6953f7d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f694c071e40 0x7f694c152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:56.428 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6953f7d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f694c0ce9a0 0x7f694c1e5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:56.428 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6953f7d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c154b60 con 0x7f694c070cb0 2024-03-23T09:40:56.428 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6953f7d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c14eb20 con 0x7f694c0ce9a0 2024-03-23T09:40:56.428 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6953f7d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694c152e80 con 0x7f694c071e40 2024-03-23T09:40:56.428 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6951d19640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694c070cb0 0x7f694c14e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:56.428 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6951d19640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694c070cb0 0x7f694c14e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56756/0 (socket says 172.21.15.47:56756) 2024-03-23T09:40:56.428 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6951d19640 1 -- 172.21.15.47:0/1231104130 learned_addr learned my addr 172.21.15.47:0/1231104130 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:40:56.428 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f695251a640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f694c0ce9a0 0x7f694c1e5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:56.428 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6951518640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f694c071e40 0x7f694c152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:56.428 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6951d19640 1 -- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f694c071e40 msgr2=0x7f694c152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:56.429 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6951d19640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f694c071e40 0x7f694c152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.429 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6951d19640 1 -- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f694c0ce9a0 msgr2=0x7f694c1e5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:56.429 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6951d19640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f694c0ce9a0 0x7f694c1e5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.429 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.424+0000 7f6951d19640 1 -- 172.21.15.47:0/1231104130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6944009a70 con 0x7f694c070cb0 2024-03-23T09:40:56.429 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.425+0000 7f6951d19640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694c070cb0 0x7f694c14e5e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f694800c400 tx=0x7f694800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:56.429 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.425+0000 7f69427fc640 1 -- 172.21.15.47:0/1231104130 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69480052d0 con 0x7f694c070cb0 2024-03-23T09:40:56.429 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.425+0000 7f69427fc640 1 -- 172.21.15.47:0/1231104130 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6948005430 con 0x7f694c070cb0 2024-03-23T09:40:56.429 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.425+0000 7f6953f7d640 1 -- 172.21.15.47:0/1231104130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694c0ac7b0 con 0x7f694c070cb0 2024-03-23T09:40:56.429 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.425+0000 7f69427fc640 1 -- 172.21.15.47:0/1231104130 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6948003830 con 0x7f694c070cb0 2024-03-23T09:40:56.429 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.425+0000 7f6953f7d640 1 -- 172.21.15.47:0/1231104130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694c154780 con 0x7f694c070cb0 2024-03-23T09:40:56.430 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.426+0000 7f6953f7d640 1 -- 172.21.15.47:0/1231104130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694c14c7e0 con 0x7f694c070cb0 2024-03-23T09:40:56.430 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.426+0000 7f69427fc640 1 -- 172.21.15.47:0/1231104130 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7f6948003990 con 0x7f694c070cb0 2024-03-23T09:40:56.430 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.427+0000 7f69427fc640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f693006ab70 0x7f693006d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:56.430 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.427+0000 7f69427fc640 1 -- 172.21.15.47:0/1231104130 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f69480878e0 con 0x7f694c070cb0 2024-03-23T09:40:56.430 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.427+0000 7f6951518640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f693006ab70 0x7f693006d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:56.430 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.427+0000 7f6951518640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f693006ab70 0x7f693006d030 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f693c007fd0 tx=0x7f693c007c20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:56.431 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.431+0000 7f69427fc640 1 -- 172.21.15.47:0/1231104130 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f694c14c7e0 con 0x7f694c070cb0 2024-03-23T09:40:56.454 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-03-23T09:40:56.455+0000 7f71d1376640 -1 osd.4 0 waiting for initial osdmap 2024-03-23T09:40:56.469 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-03-23T09:40:56.469+0000 7f54e6dd8640 -1 osd.6 0 waiting for initial osdmap 2024-03-23T09:40:56.469 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-03-23T09:40:56.470+0000 7f17a5474640 -1 osd.5 0 waiting for initial osdmap 2024-03-23T09:40:56.471 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-03-23T09:40:56.472+0000 7f71c816e640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T09:40:56.479 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-03-23T09:40:56.480+0000 7f179daa7640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T09:40:56.479 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-03-23T09:40:56.480+0000 7fbbff8ec640 -1 osd.7 0 waiting for initial osdmap 2024-03-23T09:40:56.480 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-03-23T09:40:56.480+0000 7f54df40b640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T09:40:56.495 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-03-23T09:40:56.496+0000 7fbbf7f1f640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T09:40:56.619 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.620+0000 7f6953f7d640 1 -- 172.21.15.47:0/1231104130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f694c06b7a0 con 0x7f694c070cb0 2024-03-23T09:40:56.620 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.621+0000 7f69427fc640 1 -- 172.21.15.47:0/1231104130 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10440 (secure 0 0 0) 0x7f6948017020 con 0x7f694c070cb0 2024-03-23T09:40:56.620 INFO:teuthology.misc.health.smithi047.stdout: 2024-03-23T09:40:56.620 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":13,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:40:56.508007+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:40:56.622 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.622+0000 7f6953f7d640 1 -- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f693006ab70 msgr2=0x7f693006d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:56.622 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.622+0000 7f6953f7d640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f693006ab70 0x7f693006d030 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f693c007fd0 tx=0x7f693c007c20 comp rx=0 tx=0).stop 2024-03-23T09:40:56.622 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.622+0000 7f6953f7d640 1 -- 172.21.15.47:0/1231104130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694c070cb0 msgr2=0x7f694c14e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:56.623 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.622+0000 7f6953f7d640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694c070cb0 0x7f694c14e5e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f694800c400 tx=0x7f694800c8c0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.623 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.622+0000 7f6953f7d640 1 -- 172.21.15.47:0/1231104130 shutdown_connections 2024-03-23T09:40:56.623 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.622+0000 7f6953f7d640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f693006ab70 0x7f693006d030 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.623 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.622+0000 7f6953f7d640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f694c0ce9a0 0x7f694c1e5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.623 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.622+0000 7f6953f7d640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f694c071e40 0x7f694c152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.623 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.622+0000 7f6953f7d640 1 --2- 172.21.15.47:0/1231104130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694c070cb0 0x7f694c14e5e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.623 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.622+0000 7f6953f7d640 1 -- 172.21.15.47:0/1231104130 >> 172.21.15.47:0/1231104130 conn(0x7f694c0cbb30 msgr2=0x7f694c069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:56.623 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.622+0000 7f6953f7d640 1 -- 172.21.15.47:0/1231104130 shutdown_connections 2024-03-23T09:40:56.623 INFO:teuthology.misc.health.smithi047.stderr:2024-03-23T09:40:56.622+0000 7f6953f7d640 1 -- 172.21.15.47:0/1231104130 wait complete. 2024-03-23T09:40:56.632 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-23T09:40:56.632 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-23T09:40:56.632 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-23T09:40:56.632 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-23T09:40:56.634 INFO:tasks.ceph.mds.a:Started 2024-03-23T09:40:56.635 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-23T09:40:56.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-23T09:40:56.637 INFO:tasks.ceph.mds.c:Started 2024-03-23T09:40:56.637 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-23T09:40:56.637 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-23T09:40:56.639 INFO:tasks.ceph.mds.b:Started 2024-03-23T09:40:56.640 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-23T09:40:56.640 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-23T09:40:56.642 INFO:tasks.ceph.mds.d:Started 2024-03-23T09:40:56.642 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-23T09:40:56.642 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:40:56.709 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-03-23T09:40:56.711 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:40:56.714 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-03-23T09:40:56.714 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-03-23T09:40:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.748+0000 7f1a3173c640 1 Processor -- start 2024-03-23T09:40:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.748+0000 7f1a3173c640 1 -- start start 2024-03-23T09:40:56.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a3173c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1a2c15a8d0 0x7f1a2c0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a3173c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a2c0cb730 0x7f1a2c0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a3173c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a2c0cf190 0x7f1a2c098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a3173c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2c0c41c0 con 0x7f1a2c0cf190 2024-03-23T09:40:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a3173c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2c0cf600 con 0x7f1a2c15a8d0 2024-03-23T09:40:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a3173c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0cc610 con 0x7f1a2c0cb730 2024-03-23T09:40:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a2b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a2c0cf190 0x7f1a2c098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a2b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a2c0cf190 0x7f1a2c098a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56798/0 (socket says 172.21.15.47:56798) 2024-03-23T09:40:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a2b7fe640 1 -- 172.21.15.47:0/429440928 learned_addr learned my addr 172.21.15.47:0/429440928 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:40:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a2a7fc640 1 --2- 172.21.15.47:0/429440928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a2c0cb730 0x7f1a2c0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a2affd640 1 --2- 172.21.15.47:0/429440928 >> v2:172.21.15.175:3300/0 conn(0x7f1a2c15a8d0 0x7f1a2c0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a2b7fe640 1 -- 172.21.15.47:0/429440928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a2c0cb730 msgr2=0x7f1a2c0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:40:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a2b7fe640 1 --2- 172.21.15.47:0/429440928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a2c0cb730 0x7f1a2c0cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a2b7fe640 1 -- 172.21.15.47:0/429440928 >> v2:172.21.15.175:3300/0 conn(0x7f1a2c15a8d0 msgr2=0x7f1a2c0cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:40:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a2b7fe640 1 --2- 172.21.15.47:0/429440928 >> v2:172.21.15.175:3300/0 conn(0x7f1a2c15a8d0 0x7f1a2c0cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.749+0000 7f1a2b7fe640 1 -- 172.21.15.47:0/429440928 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c0b0700 con 0x7f1a2c0cf190 2024-03-23T09:40:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.750+0000 7f1a2b7fe640 1 --2- 172.21.15.47:0/429440928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a2c0cf190 0x7f1a2c098a00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0d1410 tx=0x7f1a2402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fec79d27cd6dbb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.750+0000 7f1a29ffb640 1 -- 172.21.15.47:0/429440928 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2402c9e0 con 0x7f1a2c0cf190 2024-03-23T09:40:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.750+0000 7f1a29ffb640 1 -- 172.21.15.47:0/429440928 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2402cb40 con 0x7f1a2c0cf190 2024-03-23T09:40:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.750+0000 7f1a29ffb640 1 -- 172.21.15.47:0/429440928 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a240343f0 con 0x7f1a2c0cf190 2024-03-23T09:40:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.750+0000 7f1a3173c640 1 -- 172.21.15.47:0/429440928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a2c0cf190 msgr2=0x7f1a2c098a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.750+0000 7f1a3173c640 1 --2- 172.21.15.47:0/429440928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a2c0cf190 0x7f1a2c098a00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0d1410 tx=0x7f1a2402be40 comp rx=0 tx=0).stop 2024-03-23T09:40:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.750+0000 7f1a3173c640 1 -- 172.21.15.47:0/429440928 shutdown_connections 2024-03-23T09:40:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.750+0000 7f1a3173c640 1 --2- 172.21.15.47:0/429440928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a2c0cf190 0x7f1a2c098a00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.750+0000 7f1a3173c640 1 --2- 172.21.15.47:0/429440928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a2c0cb730 0x7f1a2c0cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.750+0000 7f1a3173c640 1 --2- 172.21.15.47:0/429440928 >> v2:172.21.15.175:3300/0 conn(0x7f1a2c15a8d0 0x7f1a2c0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.750+0000 7f1a3173c640 1 -- 172.21.15.47:0/429440928 >> 172.21.15.47:0/429440928 conn(0x7f1a2c0d0670 msgr2=0x7f1a2c0d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.751+0000 7f1a3173c640 1 -- 172.21.15.47:0/429440928 shutdown_connections 2024-03-23T09:40:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.751+0000 7f1a3173c640 1 -- 172.21.15.47:0/429440928 wait complete. 2024-03-23T09:40:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.752+0000 7f1a3173c640 1 Processor -- start 2024-03-23T09:40:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.752+0000 7f1a3173c640 1 -- start start 2024-03-23T09:40:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a3173c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a2c0cb730 0x7f1a2c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a3173c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a2c0cf190 0x7f1a2c0ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a3173c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a2c15a8d0 0x7f1a2c158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a3173c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0b4cf0 con 0x7f1a2c0cf190 2024-03-23T09:40:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a3173c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0b35e0 con 0x7f1a2c0cb730 2024-03-23T09:40:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a3173c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0c9650 con 0x7f1a2c15a8d0 2024-03-23T09:40:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a2a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a2c0cf190 0x7f1a2c0ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a2a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a2c0cf190 0x7f1a2c0ad380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56804/0 (socket says 172.21.15.47:56804) 2024-03-23T09:40:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a2a7fc640 1 -- 172.21.15.47:0/2071733669 learned_addr learned my addr 172.21.15.47:0/2071733669 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:40:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a2affd640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a2c0cb730 0x7f1a2c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a2b7fe640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a2c15a8d0 0x7f1a2c158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a2a7fc640 1 -- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a2c15a8d0 msgr2=0x7f1a2c158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a2a7fc640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a2c15a8d0 0x7f1a2c158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a2a7fc640 1 -- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a2c0cb730 msgr2=0x7f1a2c156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a2a7fc640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a2c0cb730 0x7f1a2c156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a2a7fc640 1 -- 172.21.15.47:0/2071733669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24009b30 con 0x7f1a2c0cf190 2024-03-23T09:40:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a2a7fc640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a2c0cf190 0x7f1a2c0ad380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000e9b0 tx=0x7f1a2000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a137fe640 1 -- 172.21.15.47:0/2071733669 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20009670 con 0x7f1a2c0cf190 2024-03-23T09:40:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a137fe640 1 -- 172.21.15.47:0/2071733669 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2000c9a0 con 0x7f1a2c0cf190 2024-03-23T09:40:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a3173c640 1 -- 172.21.15.47:0/2071733669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c0b4f00 con 0x7f1a2c0cf190 2024-03-23T09:40:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a137fe640 1 -- 172.21.15.47:0/2071733669 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2000cd50 con 0x7f1a2c0cf190 2024-03-23T09:40:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.753+0000 7f1a3173c640 1 -- 172.21.15.47:0/2071733669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c0b51a0 con 0x7f1a2c0cf190 2024-03-23T09:40:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.754+0000 7f1a3173c640 1 -- 172.21.15.47:0/2071733669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2c0672f0 con 0x7f1a2c0cf190 2024-03-23T09:40:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.755+0000 7f1a137fe640 1 -- 172.21.15.47:0/2071733669 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7f1a20013470 con 0x7f1a2c0cf190 2024-03-23T09:40:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.755+0000 7f1a137fe640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1a1806ab70 0x7f1a1806d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.755+0000 7f1a137fe640 1 -- 172.21.15.47:0/2071733669 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f1a20088ec0 con 0x7f1a2c0cf190 2024-03-23T09:40:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.755+0000 7f1a2affd640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1a1806ab70 0x7f1a1806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.755+0000 7f1a2affd640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1a1806ab70 0x7f1a1806d030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009a20 tx=0x7f1a1c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:56.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.759+0000 7f1a137fe640 1 -- 172.21.15.47:0/2071733669 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1a20054700 con 0x7f1a2c0cf190 2024-03-23T09:40:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.940+0000 7f1a3173c640 1 -- 172.21.15.47:0/2071733669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a2c159d20 con 0x7f1a2c0cf190 2024-03-23T09:40:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.941+0000 7f1a137fe640 1 -- 172.21.15.47:0/2071733669 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10440 (secure 0 0 0) 0x7f1a20017090 con 0x7f1a2c0cf190 2024-03-23T09:40:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:56.941 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":13,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:40:56.508007+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:40:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.943+0000 7f1a3173c640 1 -- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1a1806ab70 msgr2=0x7f1a1806d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.943+0000 7f1a3173c640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1a1806ab70 0x7f1a1806d030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009a20 tx=0x7f1a1c009190 comp rx=0 tx=0).stop 2024-03-23T09:40:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.943+0000 7f1a3173c640 1 -- 172.21.15.47:0/2071733669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a2c0cf190 msgr2=0x7f1a2c0ad380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.943+0000 7f1a3173c640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a2c0cf190 0x7f1a2c0ad380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000e9b0 tx=0x7f1a2000ee70 comp rx=0 tx=0).stop 2024-03-23T09:40:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.943+0000 7f1a3173c640 1 -- 172.21.15.47:0/2071733669 shutdown_connections 2024-03-23T09:40:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.943+0000 7f1a3173c640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1a1806ab70 0x7f1a1806d030 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.943+0000 7f1a3173c640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a2c15a8d0 0x7f1a2c158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.943+0000 7f1a3173c640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a2c0cf190 0x7f1a2c0ad380 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.943+0000 7f1a3173c640 1 --2- 172.21.15.47:0/2071733669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a2c0cb730 0x7f1a2c156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.943+0000 7f1a3173c640 1 -- 172.21.15.47:0/2071733669 >> 172.21.15.47:0/2071733669 conn(0x7f1a2c0d0670 msgr2=0x7f1a2c157970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.943+0000 7f1a3173c640 1 -- 172.21.15.47:0/2071733669 shutdown_connections 2024-03-23T09:40:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:56.943+0000 7f1a3173c640 1 -- 172.21.15.47:0/2071733669 wait complete. 2024-03-23T09:40:56.953 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-03-23T09:40:56.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:40:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.054+0000 7f0d46106640 1 Processor -- start 2024-03-23T09:40:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.054+0000 7f0d46106640 1 -- start start 2024-03-23T09:40:57.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.055+0000 7f0d46106640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d4015ca90 0x7f0d40160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.055+0000 7f0d46106640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0d400cde10 0x7f0d400d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.055+0000 7f0d46106640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d400d11e0 con 0x7f0d400b8350 2024-03-23T09:40:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.055+0000 7f0d46106640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d40160f60 con 0x7f0d400cde10 2024-03-23T09:40:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.055+0000 7f0d46106640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40161240 con 0x7f0d4015ca90 2024-03-23T09:40:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.055+0000 7f0d3ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0d400cde10 0x7f0d400d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.055+0000 7f0d3ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0d400cde10 0x7f0d400d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56812/0 (socket says 172.21.15.47:56812) 2024-03-23T09:40:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.055+0000 7f0d3ffff640 1 -- 172.21.15.47:0/3823947776 learned_addr learned my addr 172.21.15.47:0/3823947776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:40:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.055+0000 7f0d3f7fe640 1 --2- 172.21.15.47:0/3823947776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d4015ca90 0x7f0d40160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.055+0000 7f0d3ffff640 1 -- 172.21.15.47:0/3823947776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d4015ca90 msgr2=0x7f0d40160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:40:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.055+0000 7f0d3ffff640 1 --2- 172.21.15.47:0/3823947776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d4015ca90 0x7f0d40160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.055+0000 7f0d3ffff640 1 -- 172.21.15.47:0/3823947776 >> v1:172.21.15.175:6789/0 conn(0x7f0d400b8350 legacy=0x7f0d400cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:40:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.056+0000 7f0d3ffff640 1 -- 172.21.15.47:0/3823947776 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d40161520 con 0x7f0d400cde10 2024-03-23T09:40:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.056+0000 7f0d3f7fe640 1 --2- 172.21.15.47:0/3823947776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d4015ca90 0x7f0d40160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:40:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.056+0000 7f0d3ffff640 1 --2- 172.21.15.47:0/3823947776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d400cde10 0x7f0d400d1670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009e50 tx=0x7f0d3802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2f1430faf34794 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.056+0000 7f0d3e7fc640 1 -- 172.21.15.47:0/3823947776 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3802ca70 con 0x7f0d400cde10 2024-03-23T09:40:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.056+0000 7f0d3e7fc640 1 -- 172.21.15.47:0/3823947776 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3802cbd0 con 0x7f0d400cde10 2024-03-23T09:40:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.056+0000 7f0d3e7fc640 1 -- 172.21.15.47:0/3823947776 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d380343f0 con 0x7f0d400cde10 2024-03-23T09:40:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.057+0000 7f0d46106640 1 -- 172.21.15.47:0/3823947776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d400cde10 msgr2=0x7f0d400d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.057+0000 7f0d46106640 1 --2- 172.21.15.47:0/3823947776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d400cde10 0x7f0d400d1670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009e50 tx=0x7f0d3802bed0 comp rx=0 tx=0).stop 2024-03-23T09:40:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.057+0000 7f0d46106640 1 -- 172.21.15.47:0/3823947776 shutdown_connections 2024-03-23T09:40:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.057+0000 7f0d46106640 1 --2- 172.21.15.47:0/3823947776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d400cde10 0x7f0d400d1670 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.057+0000 7f0d46106640 1 --2- 172.21.15.47:0/3823947776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d4015ca90 0x7f0d40160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.057+0000 7f0d46106640 1 -- 172.21.15.47:0/3823947776 >> 172.21.15.47:0/3823947776 conn(0x7f0d400659d0 msgr2=0x7f0d40065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.057+0000 7f0d46106640 1 -- 172.21.15.47:0/3823947776 shutdown_connections 2024-03-23T09:40:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.057+0000 7f0d46106640 1 -- 172.21.15.47:0/3823947776 wait complete. 2024-03-23T09:40:57.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.058+0000 7f0d46106640 1 Processor -- start 2024-03-23T09:40:57.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.058+0000 7f0d46106640 1 -- start start 2024-03-23T09:40:57.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d46106640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d400b8350 0x7f0d400c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:57.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d46106640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d400cde10 0x7f0d400cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:57.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d46106640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d4015ca90 0x7f0d4014bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:57.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d46106640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400ca1b0 con 0x7f0d400cde10 2024-03-23T09:40:57.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d46106640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400cc6c0 con 0x7f0d400b8350 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d46106640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d401f28d0 con 0x7f0d4015ca90 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d3effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d400cde10 0x7f0d400cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d3effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d400cde10 0x7f0d400cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56814/0 (socket says 172.21.15.47:56814) 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d3effd640 1 -- 172.21.15.47:0/222592467 learned_addr learned my addr 172.21.15.47:0/222592467 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d3f7fe640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d400b8350 0x7f0d400c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d3ffff640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d4015ca90 0x7f0d4014bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d3effd640 1 -- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d4015ca90 msgr2=0x7f0d4014bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d3effd640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d4015ca90 0x7f0d4014bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d3effd640 1 -- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d400b8350 msgr2=0x7f0d400c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d3effd640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d400b8350 0x7f0d400c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d3effd640 1 -- 172.21.15.47:0/222592467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d38009b30 con 0x7f0d400cde10 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d3ffff640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d4015ca90 0x7f0d4014bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.059+0000 7f0d3effd640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d400cde10 0x7f0d400cc180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400c400 tx=0x7f0d3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.060+0000 7f0d27fff640 1 -- 172.21.15.47:0/222592467 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340052d0 con 0x7f0d400cde10 2024-03-23T09:40:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.060+0000 7f0d27fff640 1 -- 172.21.15.47:0/222592467 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d34005430 con 0x7f0d400cde10 2024-03-23T09:40:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.060+0000 7f0d46106640 1 -- 172.21.15.47:0/222592467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4014c310 con 0x7f0d400cde10 2024-03-23T09:40:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.060+0000 7f0d27fff640 1 -- 172.21.15.47:0/222592467 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34003830 con 0x7f0d400cde10 2024-03-23T09:40:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.060+0000 7f0d46106640 1 -- 172.21.15.47:0/222592467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d400ccbf0 con 0x7f0d400cde10 2024-03-23T09:40:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.061+0000 7f0d46106640 1 -- 172.21.15.47:0/222592467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d40072360 con 0x7f0d400cde10 2024-03-23T09:40:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.061+0000 7f0d27fff640 1 -- 172.21.15.47:0/222592467 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7f0d34003990 con 0x7f0d400cde10 2024-03-23T09:40:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.061+0000 7f0d27fff640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d2c06ab70 0x7f0d2c06d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.061+0000 7f0d27fff640 1 -- 172.21.15.47:0/222592467 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f0d34088230 con 0x7f0d400cde10 2024-03-23T09:40:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.062+0000 7f0d3f7fe640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d2c06ab70 0x7f0d2c06d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.062+0000 7f0d3f7fe640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d2c06ab70 0x7f0d2c06d030 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d400b68c0 tx=0x7f0d30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:57.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.066+0000 7f0d27fff640 1 -- 172.21.15.47:0/222592467 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0d34053a70 con 0x7f0d400cde10 2024-03-23T09:40:57.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.283+0000 7f0d46106640 1 -- 172.21.15.47:0/222592467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d40066be0 con 0x7f0d400cde10 2024-03-23T09:40:57.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.283+0000 7f0d27fff640 1 -- 172.21.15.47:0/222592467 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1256 (secure 0 0 0) 0x7f0d34017500 con 0x7f0d400cde10 2024-03-23T09:40:57.283 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:57.283 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4162785911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4162785911},{"type":"v1","addr":"172.21.15.47:6835","nonce":4162785911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-23T09:40:57.283 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 2 2024-03-23T09:40:57.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.285+0000 7f0d46106640 1 -- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d2c06ab70 msgr2=0x7f0d2c06d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.285+0000 7f0d46106640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d2c06ab70 0x7f0d2c06d030 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d400b68c0 tx=0x7f0d30008040 comp rx=0 tx=0).stop 2024-03-23T09:40:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.285+0000 7f0d46106640 1 -- 172.21.15.47:0/222592467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d400cde10 msgr2=0x7f0d400cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.285+0000 7f0d46106640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d400cde10 0x7f0d400cc180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400c400 tx=0x7f0d3400c8c0 comp rx=0 tx=0).stop 2024-03-23T09:40:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.285+0000 7f0d46106640 1 -- 172.21.15.47:0/222592467 shutdown_connections 2024-03-23T09:40:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.285+0000 7f0d46106640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d2c06ab70 0x7f0d2c06d030 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.285+0000 7f0d46106640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d4015ca90 0x7f0d4014bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.285+0000 7f0d46106640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d400cde10 0x7f0d400cc180 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.285+0000 7f0d46106640 1 --2- 172.21.15.47:0/222592467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d400b8350 0x7f0d400c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.285+0000 7f0d46106640 1 -- 172.21.15.47:0/222592467 >> 172.21.15.47:0/222592467 conn(0x7f0d400659d0 msgr2=0x7f0d400b75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.285+0000 7f0d46106640 1 -- 172.21.15.47:0/222592467 shutdown_connections 2024-03-23T09:40:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:57.285+0000 7f0d46106640 1 -- 172.21.15.47:0/222592467 wait complete. 2024-03-23T09:40:59.295 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:40:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.410+0000 7fc83095f640 1 Processor -- start 2024-03-23T09:40:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.410+0000 7fc83095f640 1 -- start start 2024-03-23T09:40:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.410+0000 7fc83095f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc82c069ce0 0x7fc82c06a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.410+0000 7fc83095f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82c06a680 con 0x7fc82c15a7d0 2024-03-23T09:40:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.410+0000 7fc83095f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82c06ef60 con 0x7fc82c0ca170 2024-03-23T09:40:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.410+0000 7fc83095f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c06f240 con 0x7fc82c069ce0 2024-03-23T09:40:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.411+0000 7fc82a59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc82c15a7d0 0x7fc82c071d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34156/0 (socket says 172.21.15.47:34156) 2024-03-23T09:40:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.411+0000 7fc82a59c640 1 -- 172.21.15.47:0/215021201 learned_addr learned my addr 172.21.15.47:0/215021201 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:40:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.411+0000 7fc823fff640 1 --2- 172.21.15.47:0/215021201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc82c069ce0 0x7fc82c06a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.411+0000 7fc829d9b640 1 -- 172.21.15.47:0/215021201 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987119534 0 0) 0x7fc82c06a680 con 0x7fc82c15a7d0 2024-03-23T09:40:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.411+0000 7fc829d9b640 1 -- 172.21.15.47:0/215021201 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc818003520 con 0x7fc82c15a7d0 2024-03-23T09:40:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.411+0000 7fc829d9b640 1 -- 172.21.15.47:0/215021201 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339123028 0 0) 0x7fc82c06ef60 con 0x7fc82c0ca170 2024-03-23T09:40:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.411+0000 7fc829d9b640 1 -- 172.21.15.47:0/215021201 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc82c06a680 con 0x7fc82c0ca170 2024-03-23T09:40:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.411+0000 7fc823fff640 1 -- 172.21.15.47:0/215021201 >> v1:172.21.15.175:6789/0 conn(0x7fc82c0ca170 legacy=0x7fc82c0ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.411+0000 7fc823fff640 1 -- 172.21.15.47:0/215021201 >> v1:172.21.15.47:6789/0 conn(0x7fc82c15a7d0 legacy=0x7fc82c071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.411+0000 7fc823fff640 1 -- 172.21.15.47:0/215021201 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82c06f520 con 0x7fc82c069ce0 2024-03-23T09:40:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.412+0000 7fc823fff640 1 --2- 172.21.15.47:0/215021201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc82c069ce0 0x7fc82c06a140 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8140097d0 tx=0x7fc81402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2f7a96fd74dbd15 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.412+0000 7fc829d9b640 1 -- 172.21.15.47:0/215021201 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814031890 con 0x7fc82c069ce0 2024-03-23T09:40:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.412+0000 7fc829d9b640 1 -- 172.21.15.47:0/215021201 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8140319f0 con 0x7fc82c069ce0 2024-03-23T09:40:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.412+0000 7fc829d9b640 1 -- 172.21.15.47:0/215021201 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814031d60 con 0x7fc82c069ce0 2024-03-23T09:40:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.412+0000 7fc83095f640 1 -- 172.21.15.47:0/215021201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc82c069ce0 msgr2=0x7fc82c06a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.412+0000 7fc83095f640 1 --2- 172.21.15.47:0/215021201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc82c069ce0 0x7fc82c06a140 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8140097d0 tx=0x7fc81402b930 comp rx=0 tx=0).stop 2024-03-23T09:40:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.413+0000 7fc83095f640 1 -- 172.21.15.47:0/215021201 shutdown_connections 2024-03-23T09:40:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.413+0000 7fc83095f640 1 --2- 172.21.15.47:0/215021201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc82c069ce0 0x7fc82c06a140 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.413+0000 7fc83095f640 1 -- 172.21.15.47:0/215021201 >> 172.21.15.47:0/215021201 conn(0x7fc82c0b1410 msgr2=0x7fc82c0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.413+0000 7fc83095f640 1 -- 172.21.15.47:0/215021201 shutdown_connections 2024-03-23T09:40:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.413+0000 7fc83095f640 1 -- 172.21.15.47:0/215021201 wait complete. 2024-03-23T09:40:59.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.414+0000 7fc83095f640 1 Processor -- start 2024-03-23T09:40:59.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.414+0000 7fc83095f640 1 -- start start 2024-03-23T09:40:59.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc83095f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc82c069ce0 0x7fc82c0b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:59.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc83095f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc82c0ca170 0x7fc82c150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:59.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc83095f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc82c15a7d0 0x7fc82c0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:59.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc83095f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c15ace0 con 0x7fc82c0ca170 2024-03-23T09:40:59.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc83095f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c150b70 con 0x7fc82c069ce0 2024-03-23T09:40:59.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc83095f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c0b2820 con 0x7fc82c15a7d0 2024-03-23T09:40:59.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc823fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc82c0ca170 0x7fc82c150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:59.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc823fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc82c0ca170 0x7fc82c150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56828/0 (socket says 172.21.15.47:56828) 2024-03-23T09:40:59.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc823fff640 1 -- 172.21.15.47:0/2825750627 learned_addr learned my addr 172.21.15.47:0/2825750627 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:40:59.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc82a59c640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc82c069ce0 0x7fc82c0b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc82ad9d640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc82c15a7d0 0x7fc82c0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc823fff640 1 -- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc82c15a7d0 msgr2=0x7fc82c0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc823fff640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc82c15a7d0 0x7fc82c0b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc823fff640 1 -- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc82c069ce0 msgr2=0x7fc82c0b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc823fff640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc82c069ce0 0x7fc82c0b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc823fff640 1 -- 172.21.15.47:0/2825750627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc814009400 con 0x7fc82c0ca170 2024-03-23T09:40:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.415+0000 7fc823fff640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc82c0ca170 0x7fc82c150630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc814000c00 tx=0x7fc8140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.416+0000 7fc822ffd640 1 -- 172.21.15.47:0/2825750627 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81403bbf0 con 0x7fc82c0ca170 2024-03-23T09:40:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.416+0000 7fc822ffd640 1 -- 172.21.15.47:0/2825750627 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81403bd50 con 0x7fc82c0ca170 2024-03-23T09:40:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.416+0000 7fc83095f640 1 -- 172.21.15.47:0/2825750627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82c0b3340 con 0x7fc82c0ca170 2024-03-23T09:40:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.416+0000 7fc822ffd640 1 -- 172.21.15.47:0/2825750627 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814045630 con 0x7fc82c0ca170 2024-03-23T09:40:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.416+0000 7fc83095f640 1 -- 172.21.15.47:0/2825750627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82c0b2d80 con 0x7fc82c0ca170 2024-03-23T09:40:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.417+0000 7fc83095f640 1 -- 172.21.15.47:0/2825750627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc82c15ae50 con 0x7fc82c0ca170 2024-03-23T09:40:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.417+0000 7fc822ffd640 1 -- 172.21.15.47:0/2825750627 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7fc81404e450 con 0x7fc82c0ca170 2024-03-23T09:40:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.417+0000 7fc822ffd640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc80c06ab70 0x7fc80c06d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.418+0000 7fc822ffd640 1 -- 172.21.15.47:0/2825750627 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4846+0+0 (secure 0 0 0) 0x7fc8140ab400 con 0x7fc82c0ca170 2024-03-23T09:40:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.418+0000 7fc82a59c640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc80c06ab70 0x7fc80c06d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.418+0000 7fc82a59c640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc80c06ab70 0x7fc80c06d030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc81c000f60 tx=0x7fc81c007d70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.422+0000 7fc822ffd640 1 -- 172.21.15.47:0/2825750627 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc814076a50 con 0x7fc82c0ca170 2024-03-23T09:40:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.640+0000 7fc83095f640 1 -- 172.21.15.47:0/2825750627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc82c0672f0 con 0x7fc82c0ca170 2024-03-23T09:40:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.640+0000 7fc822ffd640 1 -- 172.21.15.47:0/2825750627 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7fc814035de0 con 0x7fc82c0ca170 2024-03-23T09:40:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:59.640 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1970049669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1970049669},{"type":"v1","addr":"172.21.15.47:6833","nonce":1970049669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4162785911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4162785911},{"type":"v1","addr":"172.21.15.47:6835","nonce":4162785911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1036174787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1036174787},{"type":"v1","addr":"172.21.15.175:6836","nonce":1036174787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1633245139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1633245139},{"type":"v1","addr":"172.21.15.175:6837","nonce":1633245139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-23T09:40:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 3 2024-03-23T09:40:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.642+0000 7fc83095f640 1 -- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc80c06ab70 msgr2=0x7fc80c06d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.642+0000 7fc83095f640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc80c06ab70 0x7fc80c06d030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc81c000f60 tx=0x7fc81c007d70 comp rx=0 tx=0).stop 2024-03-23T09:40:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.642+0000 7fc83095f640 1 -- 172.21.15.47:0/2825750627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc82c0ca170 msgr2=0x7fc82c150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.642+0000 7fc83095f640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc82c0ca170 0x7fc82c150630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc814000c00 tx=0x7fc8140046c0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.642+0000 7fc83095f640 1 -- 172.21.15.47:0/2825750627 shutdown_connections 2024-03-23T09:40:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.642+0000 7fc83095f640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc80c06ab70 0x7fc80c06d030 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.642+0000 7fc83095f640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc82c15a7d0 0x7fc82c0b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.642+0000 7fc83095f640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc82c0ca170 0x7fc82c150630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.642+0000 7fc83095f640 1 --2- 172.21.15.47:0/2825750627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc82c069ce0 0x7fc82c0b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.642+0000 7fc83095f640 1 -- 172.21.15.47:0/2825750627 >> 172.21.15.47:0/2825750627 conn(0x7fc82c0b1410 msgr2=0x7fc82c072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.642+0000 7fc83095f640 1 -- 172.21.15.47:0/2825750627 shutdown_connections 2024-03-23T09:40:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.642+0000 7fc83095f640 1 -- 172.21.15.47:0/2825750627 wait complete. 2024-03-23T09:40:59.653 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:40:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.754+0000 7f14333b9640 1 Processor -- start 2024-03-23T09:40:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.754+0000 7f14333b9640 1 -- start start 2024-03-23T09:40:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.755+0000 7f14333b9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f142c0cd090 0x7f142c0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.755+0000 7f14333b9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f142c15ab50 0x7f142c0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.755+0000 7f14333b9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f142c0cd9b0 con 0x7f142c0cb6f0 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.755+0000 7f14333b9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f142c0cbf50 con 0x7f142c0cd090 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.755+0000 7f14333b9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142c0cc230 con 0x7f142c15ab50 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.756+0000 7f1431155640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f142c0cd090 0x7f142c0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.756+0000 7f1430954640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f142c15ab50 0x7f142c0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.756+0000 7f1431956640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f142c0cb6f0 0x7f142c098a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:48432/0 (socket says 172.21.15.47:48432) 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.756+0000 7f1430954640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f142c15ab50 0x7f142c0cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:49716/0 (socket says 172.21.15.47:49716) 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.756+0000 7f1431155640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f142c0cd090 0x7f142c0cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:38646/0 (socket says 172.21.15.47:38646) 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.756+0000 7f1431956640 1 -- 172.21.15.47:0/3991417007 learned_addr learned my addr 172.21.15.47:0/3991417007 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.756+0000 7f1431155640 1 -- 172.21.15.47:0/3991417007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f142c15ab50 msgr2=0x7f142c0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.756+0000 7f1431155640 1 --2- 172.21.15.47:0/3991417007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f142c15ab50 0x7f142c0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.756+0000 7f1431155640 1 -- 172.21.15.47:0/3991417007 >> v1:172.21.15.175:6789/0 conn(0x7f142c0cb6f0 legacy=0x7f142c098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.756+0000 7f142bfff640 1 -- 172.21.15.47:0/3991417007 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2349932730 0 0) 0x7f142c0cd9b0 con 0x7f142c0cb6f0 2024-03-23T09:40:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.756+0000 7f1431155640 1 -- 172.21.15.47:0/3991417007 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c0cc510 con 0x7f142c0cd090 2024-03-23T09:40:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.757+0000 7f1431155640 1 --2- 172.21.15.47:0/3991417007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f142c0cd090 0x7f142c0cd470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f14240097a0 tx=0x7f142402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a308730d73539d2d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.757+0000 7f142bfff640 1 -- 172.21.15.47:0/3991417007 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424030610 con 0x7f142c0cd090 2024-03-23T09:40:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.757+0000 7f142bfff640 1 -- 172.21.15.47:0/3991417007 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1424030770 con 0x7f142c0cd090 2024-03-23T09:40:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.757+0000 7f142bfff640 1 -- 172.21.15.47:0/3991417007 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424030b00 con 0x7f142c0cd090 2024-03-23T09:40:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.757+0000 7f14333b9640 1 -- 172.21.15.47:0/3991417007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f142c0cd090 msgr2=0x7f142c0cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.757+0000 7f14333b9640 1 --2- 172.21.15.47:0/3991417007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f142c0cd090 0x7f142c0cd470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f14240097a0 tx=0x7f142402b960 comp rx=0 tx=0).stop 2024-03-23T09:40:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.757+0000 7f14333b9640 1 -- 172.21.15.47:0/3991417007 shutdown_connections 2024-03-23T09:40:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.757+0000 7f14333b9640 1 --2- 172.21.15.47:0/3991417007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f142c15ab50 0x7f142c0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.757+0000 7f14333b9640 1 --2- 172.21.15.47:0/3991417007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f142c0cd090 0x7f142c0cd470 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.757+0000 7f14333b9640 1 -- 172.21.15.47:0/3991417007 >> 172.21.15.47:0/3991417007 conn(0x7f142c0d06a0 msgr2=0x7f142c0d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.758+0000 7f14333b9640 1 -- 172.21.15.47:0/3991417007 shutdown_connections 2024-03-23T09:40:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.758+0000 7f14333b9640 1 -- 172.21.15.47:0/3991417007 wait complete. 2024-03-23T09:40:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.759+0000 7f14333b9640 1 Processor -- start 2024-03-23T09:40:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.759+0000 7f14333b9640 1 -- start start 2024-03-23T09:40:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.759+0000 7f14333b9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f142c0cb6f0 0x7f142c0c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.759+0000 7f14333b9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f142c0cd090 0x7f142c15d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.759+0000 7f14333b9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f142c15ab50 0x7f142c1ef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.759+0000 7f14333b9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c1591e0 con 0x7f142c0cb6f0 2024-03-23T09:40:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.759+0000 7f14333b9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c144b00 con 0x7f142c0cd090 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.759+0000 7f14333b9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142c0add00 con 0x7f142c15ab50 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1431155640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f142c0cb6f0 0x7f142c0c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1431155640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f142c0cb6f0 0x7f142c0c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56842/0 (socket says 172.21.15.47:56842) 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1431155640 1 -- 172.21.15.47:0/2314471011 learned_addr learned my addr 172.21.15.47:0/2314471011 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1430954640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f142c0cd090 0x7f142c15d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1431956640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f142c15ab50 0x7f142c1ef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1431155640 1 -- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f142c15ab50 msgr2=0x7f142c1ef1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1431155640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f142c15ab50 0x7f142c1ef1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1431155640 1 -- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f142c0cd090 msgr2=0x7f142c15d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1431155640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f142c0cd090 0x7f142c15d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1431155640 1 -- 172.21.15.47:0/2314471011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1424009400 con 0x7f142c0cb6f0 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1430954640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f142c0cd090 0x7f142c15d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1431956640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f142c15ab50 0x7f142c1ef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1431155640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f142c0cb6f0 0x7f142c0c7770 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f14240098c0 tx=0x7f14240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1429ffb640 1 -- 172.21.15.47:0/2314471011 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424039c90 con 0x7f142c0cb6f0 2024-03-23T09:40:59.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1429ffb640 1 -- 172.21.15.47:0/2314471011 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142403f040 con 0x7f142c0cb6f0 2024-03-23T09:40:59.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f1429ffb640 1 -- 172.21.15.47:0/2314471011 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424006e10 con 0x7f142c0cb6f0 2024-03-23T09:40:59.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.760+0000 7f14333b9640 1 -- 172.21.15.47:0/2314471011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c0ae5a0 con 0x7f142c0cb6f0 2024-03-23T09:40:59.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.761+0000 7f14333b9640 1 -- 172.21.15.47:0/2314471011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c0adf50 con 0x7f142c0cb6f0 2024-03-23T09:40:59.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.761+0000 7f14333b9640 1 -- 172.21.15.47:0/2314471011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c0672f0 con 0x7f142c0cb6f0 2024-03-23T09:40:59.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.762+0000 7f1429ffb640 1 -- 172.21.15.47:0/2314471011 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7f1424004030 con 0x7f142c0cb6f0 2024-03-23T09:40:59.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.762+0000 7f1429ffb640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f141806ab70 0x7f141806d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:40:59.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.762+0000 7f1429ffb640 1 -- 172.21.15.47:0/2314471011 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f14240aae10 con 0x7f142c0cb6f0 2024-03-23T09:40:59.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.762+0000 7f1430954640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f141806ab70 0x7f141806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:40:59.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.763+0000 7f1430954640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f141806ab70 0x7f141806d030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f142c1f3520 tx=0x7f1420008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:40:59.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.767+0000 7f1429ffb640 1 -- 172.21.15.47:0/2314471011 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f14240764b0 con 0x7f142c0cb6f0 2024-03-23T09:40:59.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.946+0000 7f14333b9640 1 -- 172.21.15.47:0/2314471011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f142c099740 con 0x7f142c0cb6f0 2024-03-23T09:40:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.947+0000 7f1429ffb640 1 -- 172.21.15.47:0/2314471011 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12395 (secure 0 0 0) 0x7f1424079ad0 con 0x7f142c0cb6f0 2024-03-23T09:40:59.947 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:40:59.947 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":16,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:40:59.729105+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:40:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.949+0000 7f14333b9640 1 -- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f141806ab70 msgr2=0x7f141806d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.949+0000 7f14333b9640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f141806ab70 0x7f141806d030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f142c1f3520 tx=0x7f1420008040 comp rx=0 tx=0).stop 2024-03-23T09:40:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.949+0000 7f14333b9640 1 -- 172.21.15.47:0/2314471011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f142c0cb6f0 msgr2=0x7f142c0c7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:40:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.949+0000 7f14333b9640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f142c0cb6f0 0x7f142c0c7770 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f14240098c0 tx=0x7f14240046c0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.949+0000 7f14333b9640 1 -- 172.21.15.47:0/2314471011 shutdown_connections 2024-03-23T09:40:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.949+0000 7f14333b9640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f141806ab70 0x7f141806d030 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.949+0000 7f14333b9640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f142c15ab50 0x7f142c1ef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.949+0000 7f14333b9640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f142c0cd090 0x7f142c15d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.949+0000 7f14333b9640 1 --2- 172.21.15.47:0/2314471011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f142c0cb6f0 0x7f142c0c7770 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:40:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.949+0000 7f14333b9640 1 -- 172.21.15.47:0/2314471011 >> 172.21.15.47:0/2314471011 conn(0x7f142c0d06a0 msgr2=0x7f142c14ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:40:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.949+0000 7f14333b9640 1 -- 172.21.15.47:0/2314471011 shutdown_connections 2024-03-23T09:40:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:40:59.949+0000 7f14333b9640 1 -- 172.21.15.47:0/2314471011 wait complete. 2024-03-23T09:40:59.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:40:59.960 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:00.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.063+0000 7fd72c789640 1 Processor -- start 2024-03-23T09:41:00.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.063+0000 7fd72c789640 1 -- start start 2024-03-23T09:41:00.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.064+0000 7fd72c789640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd72415c990 0x7fd724160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.064+0000 7fd72c789640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7240cde10 0x7fd7240d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.064+0000 7fd72c789640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd724161630 con 0x7fd7240b8350 2024-03-23T09:41:00.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.064+0000 7fd72c789640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7240d1030 con 0x7fd72415c990 2024-03-23T09:41:00.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.064+0000 7fd72c789640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240cfe10 con 0x7fd7240cde10 2024-03-23T09:41:00.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.065+0000 7fd72a525640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd72415c990 0x7fd724160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.065+0000 7fd72ad26640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7240cde10 0x7fd7240d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.065+0000 7fd72a525640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd72415c990 0x7fd724160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:38662/0 (socket says 172.21.15.47:38662) 2024-03-23T09:41:00.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.065+0000 7fd72a525640 1 -- 172.21.15.47:0/1440491524 learned_addr learned my addr 172.21.15.47:0/1440491524 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:00.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.065+0000 7fd72a525640 1 -- 172.21.15.47:0/1440491524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7240cde10 msgr2=0x7fd7240d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:00.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.065+0000 7fd72a525640 1 --2- 172.21.15.47:0/1440491524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7240cde10 0x7fd7240d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.065+0000 7fd72a525640 1 -- 172.21.15.47:0/1440491524 >> v1:172.21.15.175:6789/0 conn(0x7fd7240b8350 legacy=0x7fd7240cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.065+0000 7fd72a525640 1 -- 172.21.15.47:0/1440491524 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7240b5b60 con 0x7fd72415c990 2024-03-23T09:41:00.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.066+0000 7fd72a525640 1 --2- 172.21.15.47:0/1440491524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd72415c990 0x7fd724160690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009870 tx=0x7fd71c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=741c12baedb4da66 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.066+0000 7fd729523640 1 -- 172.21.15.47:0/1440491524 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c031640 con 0x7fd72415c990 2024-03-23T09:41:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.066+0000 7fd729523640 1 -- 172.21.15.47:0/1440491524 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71c0317a0 con 0x7fd72415c990 2024-03-23T09:41:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.066+0000 7fd729523640 1 -- 172.21.15.47:0/1440491524 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c031b30 con 0x7fd72415c990 2024-03-23T09:41:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.066+0000 7fd72c789640 1 -- 172.21.15.47:0/1440491524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd72415c990 msgr2=0x7fd724160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.066+0000 7fd72c789640 1 --2- 172.21.15.47:0/1440491524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd72415c990 0x7fd724160690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009870 tx=0x7fd71c02b8f0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.067+0000 7fd72c789640 1 -- 172.21.15.47:0/1440491524 shutdown_connections 2024-03-23T09:41:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.067+0000 7fd72c789640 1 --2- 172.21.15.47:0/1440491524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7240cde10 0x7fd7240d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.067+0000 7fd72c789640 1 --2- 172.21.15.47:0/1440491524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd72415c990 0x7fd724160690 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.067+0000 7fd72c789640 1 -- 172.21.15.47:0/1440491524 >> 172.21.15.47:0/1440491524 conn(0x7fd7240659d0 msgr2=0x7fd724065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.067+0000 7fd72c789640 1 -- 172.21.15.47:0/1440491524 shutdown_connections 2024-03-23T09:41:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.067+0000 7fd72c789640 1 -- 172.21.15.47:0/1440491524 wait complete. 2024-03-23T09:41:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.068+0000 7fd72c789640 1 Processor -- start 2024-03-23T09:41:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.068+0000 7fd72c789640 1 -- start start 2024-03-23T09:41:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.068+0000 7fd72c789640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7240b8350 0x7fd7241f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.068+0000 7fd72c789640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7240cde10 0x7fd7240b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.068+0000 7fd72c789640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd72415c990 0x7fd7240b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.068+0000 7fd72c789640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7241f5a30 con 0x7fd7240cde10 2024-03-23T09:41:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.068+0000 7fd72c789640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7241f2c60 con 0x7fd72415c990 2024-03-23T09:41:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.068+0000 7fd72c789640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7241f2f40 con 0x7fd7240b8350 2024-03-23T09:41:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd729d24640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7240cde10 0x7fd7240b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd729d24640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7240cde10 0x7fd7240b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56852/0 (socket says 172.21.15.47:56852) 2024-03-23T09:41:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd729d24640 1 -- 172.21.15.47:0/3632663982 learned_addr learned my addr 172.21.15.47:0/3632663982 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd72ad26640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd72415c990 0x7fd7240b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd72a525640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7240b8350 0x7fd7241f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd729d24640 1 -- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7240b8350 msgr2=0x7fd7241f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd729d24640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7240b8350 0x7fd7241f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd729d24640 1 -- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd72415c990 msgr2=0x7fd7240b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd729d24640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd72415c990 0x7fd7240b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd729d24640 1 -- 172.21.15.47:0/3632663982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c0094d0 con 0x7fd7240cde10 2024-03-23T09:41:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd729d24640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7240cde10 0x7fd7240b4180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd71800c3c0 tx=0x7fd71800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd722ffd640 1 -- 172.21.15.47:0/3632663982 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7180052d0 con 0x7fd7240cde10 2024-03-23T09:41:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd722ffd640 1 -- 172.21.15.47:0/3632663982 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd718005450 con 0x7fd7240cde10 2024-03-23T09:41:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd72c789640 1 -- 172.21.15.47:0/3632663982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7241f3220 con 0x7fd7240cde10 2024-03-23T09:41:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd722ffd640 1 -- 172.21.15.47:0/3632663982 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718007260 con 0x7fd7240cde10 2024-03-23T09:41:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.069+0000 7fd72c789640 1 -- 172.21.15.47:0/3632663982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7241f66e0 con 0x7fd7240cde10 2024-03-23T09:41:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.070+0000 7fd72c789640 1 -- 172.21.15.47:0/3632663982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd724072360 con 0x7fd7240cde10 2024-03-23T09:41:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.071+0000 7fd722ffd640 1 -- 172.21.15.47:0/3632663982 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7fd7180073c0 con 0x7fd7240cde10 2024-03-23T09:41:00.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.071+0000 7fd722ffd640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd71006ab70 0x7fd71006d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.071+0000 7fd722ffd640 1 -- 172.21.15.47:0/3632663982 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd718088160 con 0x7fd7240cde10 2024-03-23T09:41:00.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.071+0000 7fd72a525640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd71006ab70 0x7fd71006d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.072+0000 7fd72a525640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd71006ab70 0x7fd71006d030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd71c036000 tx=0x7fd71c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:00.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.076+0000 7fd722ffd640 1 -- 172.21.15.47:0/3632663982 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd7180538b0 con 0x7fd7240cde10 2024-03-23T09:41:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.260+0000 7fd72c789640 1 -- 172.21.15.47:0/3632663982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd724066be0 con 0x7fd7240cde10 2024-03-23T09:41:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.261+0000 7fd722ffd640 1 -- 172.21.15.47:0/3632663982 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd718056ed0 con 0x7fd7240cde10 2024-03-23T09:41:00.260 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.263+0000 7fd72c789640 1 -- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd71006ab70 msgr2=0x7fd71006d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.263+0000 7fd72c789640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd71006ab70 0x7fd71006d030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd71c036000 tx=0x7fd71c03c040 comp rx=0 tx=0).stop 2024-03-23T09:41:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.263+0000 7fd72c789640 1 -- 172.21.15.47:0/3632663982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7240cde10 msgr2=0x7fd7240b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.263+0000 7fd72c789640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7240cde10 0x7fd7240b4180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd71800c3c0 tx=0x7fd71800c880 comp rx=0 tx=0).stop 2024-03-23T09:41:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.263+0000 7fd72c789640 1 -- 172.21.15.47:0/3632663982 shutdown_connections 2024-03-23T09:41:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.263+0000 7fd72c789640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd71006ab70 0x7fd71006d030 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.263+0000 7fd72c789640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd72415c990 0x7fd7240b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.263+0000 7fd72c789640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7240cde10 0x7fd7240b4180 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.263+0000 7fd72c789640 1 --2- 172.21.15.47:0/3632663982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7240b8350 0x7fd7241f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.263+0000 7fd72c789640 1 -- 172.21.15.47:0/3632663982 >> 172.21.15.47:0/3632663982 conn(0x7fd7240659d0 msgr2=0x7fd7240d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.263+0000 7fd72c789640 1 -- 172.21.15.47:0/3632663982 shutdown_connections 2024-03-23T09:41:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.263+0000 7fd72c789640 1 -- 172.21.15.47:0/3632663982 wait complete. 2024-03-23T09:41:00.275 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T09:41:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.378+0000 7f0cd3f88640 1 Processor -- start 2024-03-23T09:41:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.378+0000 7f0cd3f88640 1 -- start start 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.379+0000 7f0cd3f88640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0ccc099110 0x7f0ccc0ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.379+0000 7f0cd3f88640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ccc0ce8f0 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.379+0000 7f0cd3f88640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ccc0cc300 con 0x7f0ccc0cbdf0 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.379+0000 7f0cd3f88640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ccc0cc5c0 con 0x7f0ccc099110 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.380+0000 7f0cd1d24640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0ccc0b10f0 0x7f0ccc0affe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34160/0 (socket says 172.21.15.47:34160) 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.380+0000 7f0cd2525640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0ccc099110 0x7f0ccc0ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.380+0000 7f0cd1d24640 1 -- 172.21.15.47:0/532301718 learned_addr learned my addr 172.21.15.47:0/532301718 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.380+0000 7f0cd2525640 1 -- 172.21.15.47:0/532301718 >> v1:172.21.15.175:6789/0 conn(0x7f0ccc0cbdf0 legacy=0x7f0ccc098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.380+0000 7f0cd2525640 1 -- 172.21.15.47:0/532301718 >> v1:172.21.15.47:6789/0 conn(0x7f0ccc0b10f0 legacy=0x7f0ccc0affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.380+0000 7f0cd2525640 1 -- 172.21.15.47:0/532301718 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc0ce2a0 con 0x7f0ccc099110 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.381+0000 7f0cd2525640 1 --2- 172.21.15.47:0/532301718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc099110 0x7f0ccc0ced80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009dc0 tx=0x7f0cc402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfe688247055b97 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.381+0000 7f0cd0d22640 1 -- 172.21.15.47:0/532301718 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc402cc40 con 0x7f0ccc099110 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.381+0000 7f0cd0d22640 1 -- 172.21.15.47:0/532301718 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc402cda0 con 0x7f0ccc099110 2024-03-23T09:41:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.381+0000 7f0cd0d22640 1 -- 172.21.15.47:0/532301718 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4034660 con 0x7f0ccc099110 2024-03-23T09:41:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.381+0000 7f0cd3f88640 1 -- 172.21.15.47:0/532301718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc099110 msgr2=0x7f0ccc0ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.381+0000 7f0cd3f88640 1 --2- 172.21.15.47:0/532301718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc099110 0x7f0ccc0ced80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009dc0 tx=0x7f0cc402be90 comp rx=0 tx=0).stop 2024-03-23T09:41:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.381+0000 7f0cd3f88640 1 -- 172.21.15.47:0/532301718 shutdown_connections 2024-03-23T09:41:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.381+0000 7f0cd3f88640 1 --2- 172.21.15.47:0/532301718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc099110 0x7f0ccc0ced80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.381+0000 7f0cd3f88640 1 -- 172.21.15.47:0/532301718 >> 172.21.15.47:0/532301718 conn(0x7f0ccc0b21c0 msgr2=0x7f0ccc0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.382+0000 7f0cd3f88640 1 -- 172.21.15.47:0/532301718 shutdown_connections 2024-03-23T09:41:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.382+0000 7f0cd3f88640 1 -- 172.21.15.47:0/532301718 wait complete. 2024-03-23T09:41:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.383+0000 7f0cd3f88640 1 Processor -- start 2024-03-23T09:41:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.383+0000 7f0cd3f88640 1 -- start start 2024-03-23T09:41:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd3f88640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ccc099110 0x7f0ccc06e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd3f88640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc0b10f0 0x7f0ccc1e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd3f88640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ccc0cbdf0 0x7f0ccc1d1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd3f88640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc1e3660 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd3f88640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc1e3920 con 0x7f0ccc099110 2024-03-23T09:41:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd3f88640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc1e3c00 con 0x7f0ccc0cbdf0 2024-03-23T09:41:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd1523640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc0b10f0 0x7f0ccc1e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd1523640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc0b10f0 0x7f0ccc1e4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56858/0 (socket says 172.21.15.47:56858) 2024-03-23T09:41:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd1523640 1 -- 172.21.15.47:0/1241903750 learned_addr learned my addr 172.21.15.47:0/1241903750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd2525640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ccc0cbdf0 0x7f0ccc1d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd1d24640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ccc099110 0x7f0ccc06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd1523640 1 -- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ccc0cbdf0 msgr2=0x7f0ccc1d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd1523640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ccc0cbdf0 0x7f0ccc1d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd1523640 1 -- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ccc099110 msgr2=0x7f0ccc06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd1523640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ccc099110 0x7f0ccc06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd1523640 1 -- 172.21.15.47:0/1241903750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc4009a70 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cd1523640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc0b10f0 0x7f0ccc1e4c60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800c450 tx=0x7f0cc800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.384+0000 7f0cc27fc640 1 -- 172.21.15.47:0/1241903750 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc80052d0 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.385+0000 7f0cc27fc640 1 -- 172.21.15.47:0/1241903750 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc8005430 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.385+0000 7f0cd3f88640 1 -- 172.21.15.47:0/1241903750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ccc1e3ee0 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.385+0000 7f0cc27fc640 1 -- 172.21.15.47:0/1241903750 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8012870 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.385+0000 7f0cd3f88640 1 -- 172.21.15.47:0/1241903750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ccc1e44a0 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.386+0000 7f0cd3f88640 1 -- 172.21.15.47:0/1241903750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ccc066be0 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.386+0000 7f0cc27fc640 1 -- 172.21.15.47:0/1241903750 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7f0cc80129d0 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.386+0000 7f0cc27fc640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cb006ab70 0x7f0cb006d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.386+0000 7f0cc27fc640 1 -- 172.21.15.47:0/1241903750 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0cc80889c0 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.387+0000 7f0cd1d24640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cb006ab70 0x7f0cb006d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.387+0000 7f0cd1d24640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cb006ab70 0x7f0cb006d030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc005880 tx=0x7f0cbc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.393+0000 7f0cc27fc640 1 -- 172.21.15.47:0/1241903750 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0cc8053f70 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.580+0000 7f0cd3f88640 1 -- 172.21.15.47:0/1241903750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0ccc0b6650 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.580+0000 7f0cc27fc640 1 -- 172.21.15.47:0/1241903750 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0cc8057590 con 0x7f0ccc0b10f0 2024-03-23T09:41:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:00.580 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-23T09:41:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-03-23T09:41:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-03-23T09:41:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:00.580 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-23T09:41:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.582+0000 7f0cd3f88640 1 -- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cb006ab70 msgr2=0x7f0cb006d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.582+0000 7f0cd3f88640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cb006ab70 0x7f0cb006d030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc005880 tx=0x7f0cbc009040 comp rx=0 tx=0).stop 2024-03-23T09:41:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.582+0000 7f0cd3f88640 1 -- 172.21.15.47:0/1241903750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc0b10f0 msgr2=0x7f0ccc1e4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.582+0000 7f0cd3f88640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc0b10f0 0x7f0ccc1e4c60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800c450 tx=0x7f0cc800c910 comp rx=0 tx=0).stop 2024-03-23T09:41:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.582+0000 7f0cd3f88640 1 -- 172.21.15.47:0/1241903750 shutdown_connections 2024-03-23T09:41:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.582+0000 7f0cd3f88640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cb006ab70 0x7f0cb006d030 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.582+0000 7f0cd3f88640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ccc0cbdf0 0x7f0ccc1d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.582+0000 7f0cd3f88640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc0b10f0 0x7f0ccc1e4c60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.582+0000 7f0cd3f88640 1 --2- 172.21.15.47:0/1241903750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ccc099110 0x7f0ccc06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.582+0000 7f0cd3f88640 1 -- 172.21.15.47:0/1241903750 >> 172.21.15.47:0/1241903750 conn(0x7f0ccc0b21c0 msgr2=0x7f0ccc0ad260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.582+0000 7f0cd3f88640 1 -- 172.21.15.47:0/1241903750 shutdown_connections 2024-03-23T09:41:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.582+0000 7f0cd3f88640 1 -- 172.21.15.47:0/1241903750 wait complete. 2024-03-23T09:41:00.595 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T09:41:00.595 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T09:41:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.699+0000 7f9488eaf640 1 Processor -- start 2024-03-23T09:41:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.699+0000 7f9488eaf640 1 -- start start 2024-03-23T09:41:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.700+0000 7f9488eaf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f948415a860 0x7f9484161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.700+0000 7f9488eaf640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94840cf8a0 0x7f948415d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.700+0000 7f9488eaf640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9484160bd0 con 0x7f948415d0f0 2024-03-23T09:41:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.700+0000 7f9488eaf640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94840d2790 con 0x7f948415a860 2024-03-23T09:41:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.700+0000 7f9488eaf640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948415dbb0 con 0x7f94840cf8a0 2024-03-23T09:41:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.700+0000 7f948259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f948415a860 0x7f9484161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.700+0000 7f9481d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f948415d0f0 0x7f94840cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34172/0 (socket says 172.21.15.47:34172) 2024-03-23T09:41:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.700+0000 7f9481d9b640 1 -- 172.21.15.47:0/1916009442 learned_addr learned my addr 172.21.15.47:0/1916009442 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.700+0000 7f9482d9d640 1 --2- 172.21.15.47:0/1916009442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94840cf8a0 0x7f948415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.701+0000 7f948259c640 1 -- 172.21.15.47:0/1916009442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94840cf8a0 msgr2=0x7f948415d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.701+0000 7f948259c640 1 --2- 172.21.15.47:0/1916009442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94840cf8a0 0x7f948415d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.701+0000 7f948259c640 1 -- 172.21.15.47:0/1916009442 >> v1:172.21.15.47:6789/0 conn(0x7f948415d0f0 legacy=0x7f94840cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.701+0000 7f948159a640 1 -- 172.21.15.47:0/1916009442 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142766733 0 0) 0x7f9484160bd0 con 0x7f948415d0f0 2024-03-23T09:41:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.701+0000 7f948259c640 1 -- 172.21.15.47:0/1916009442 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948415de90 con 0x7f948415a860 2024-03-23T09:41:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.701+0000 7f948259c640 1 --2- 172.21.15.47:0/1916009442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948415a860 0x7f9484161060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9474009d90 tx=0x7f947402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f20a4add3b557f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.701+0000 7f948159a640 1 -- 172.21.15.47:0/1916009442 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947402cc50 con 0x7f948415a860 2024-03-23T09:41:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.701+0000 7f948159a640 1 -- 172.21.15.47:0/1916009442 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947402cdb0 con 0x7f948415a860 2024-03-23T09:41:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.701+0000 7f948159a640 1 -- 172.21.15.47:0/1916009442 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474034660 con 0x7f948415a860 2024-03-23T09:41:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.702+0000 7f9488eaf640 1 -- 172.21.15.47:0/1916009442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948415a860 msgr2=0x7f9484161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.702+0000 7f9488eaf640 1 --2- 172.21.15.47:0/1916009442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948415a860 0x7f9484161060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9474009d90 tx=0x7f947402bea0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.702+0000 7f9488eaf640 1 -- 172.21.15.47:0/1916009442 shutdown_connections 2024-03-23T09:41:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.702+0000 7f9488eaf640 1 --2- 172.21.15.47:0/1916009442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94840cf8a0 0x7f948415d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.702+0000 7f9488eaf640 1 --2- 172.21.15.47:0/1916009442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948415a860 0x7f9484161060 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.702+0000 7f9488eaf640 1 -- 172.21.15.47:0/1916009442 >> 172.21.15.47:0/1916009442 conn(0x7f94840659d0 msgr2=0x7f9484065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.702+0000 7f9488eaf640 1 -- 172.21.15.47:0/1916009442 shutdown_connections 2024-03-23T09:41:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.702+0000 7f9488eaf640 1 -- 172.21.15.47:0/1916009442 wait complete. 2024-03-23T09:41:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.703+0000 7f9488eaf640 1 Processor -- start 2024-03-23T09:41:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.703+0000 7f9488eaf640 1 -- start start 2024-03-23T09:41:00.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f9488eaf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94840cf8a0 0x7f9484149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f9488eaf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f948415a860 0x7f94840c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f9488eaf640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f948415d0f0 0x7f94840b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f9488eaf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94840ce0d0 con 0x7f94840cf8a0 2024-03-23T09:41:00.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f9488eaf640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948414a380 con 0x7f948415a860 2024-03-23T09:41:00.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f9488eaf640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94840c8040 con 0x7f948415d0f0 2024-03-23T09:41:00.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f948259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94840cf8a0 0x7f9484149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f948259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94840cf8a0 0x7f9484149e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56874/0 (socket says 172.21.15.47:56874) 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f948259c640 1 -- 172.21.15.47:0/1993437436 learned_addr learned my addr 172.21.15.47:0/1993437436 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f9481d9b640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f948415a860 0x7f94840c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f9482d9d640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f948415d0f0 0x7f94840b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f948259c640 1 -- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f948415d0f0 msgr2=0x7f94840b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f948259c640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f948415d0f0 0x7f94840b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f948259c640 1 -- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f948415a860 msgr2=0x7f94840c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f948259c640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f948415a860 0x7f94840c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.704+0000 7f948259c640 1 -- 172.21.15.47:0/1993437436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474009a70 con 0x7f94840cf8a0 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.705+0000 7f948259c640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94840cf8a0 0x7f9484149e40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9474009d90 tx=0x7f9474004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.705+0000 7f947affd640 1 -- 172.21.15.47:0/1993437436 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474040650 con 0x7f94840cf8a0 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.705+0000 7f947affd640 1 -- 172.21.15.47:0/1993437436 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94740407b0 con 0x7f94840cf8a0 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.705+0000 7f9488eaf640 1 -- 172.21.15.47:0/1993437436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94840c6a60 con 0x7f94840cf8a0 2024-03-23T09:41:00.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.705+0000 7f947affd640 1 -- 172.21.15.47:0/1993437436 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474040b60 con 0x7f94840cf8a0 2024-03-23T09:41:00.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.705+0000 7f9488eaf640 1 -- 172.21.15.47:0/1993437436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94840c7020 con 0x7f94840cf8a0 2024-03-23T09:41:00.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.706+0000 7f9488eaf640 1 -- 172.21.15.47:0/1993437436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9484072360 con 0x7f94840cf8a0 2024-03-23T09:41:00.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.706+0000 7f947affd640 1 -- 172.21.15.47:0/1993437436 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7f9474040cc0 con 0x7f94840cf8a0 2024-03-23T09:41:00.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.706+0000 7f947affd640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f946406ab70 0x7f946406d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:00.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.707+0000 7f947affd640 1 -- 172.21.15.47:0/1993437436 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f94740abc20 con 0x7f94840cf8a0 2024-03-23T09:41:00.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.707+0000 7f9481d9b640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f946406ab70 0x7f946406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:00.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.707+0000 7f9481d9b640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f946406ab70 0x7f946406d030 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f94840c6080 tx=0x7f946c007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:00.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.711+0000 7f947affd640 1 -- 172.21.15.47:0/1993437436 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9474077400 con 0x7f94840cf8a0 2024-03-23T09:41:00.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:00.892+0000 7f9488eaf640 1 -- 172.21.15.47:0/1993437436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9484066be0 con 0x7f94840cf8a0 2024-03-23T09:41:01.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.343+0000 7f947affd640 1 -- 172.21.15.47:0/1993437436 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f94740777d0 con 0x7f94840cf8a0 2024-03-23T09:41:01.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.766+0000 7f947affd640 1 -- 172.21.15.47:0/1993437436 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f947407e8f0 con 0x7f94840cf8a0 2024-03-23T09:41:01.766 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-03-23T09:41:01.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.771+0000 7f9488eaf640 1 -- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f946406ab70 msgr2=0x7f946406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:01.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.771+0000 7f9488eaf640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f946406ab70 0x7f946406d030 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f94840c6080 tx=0x7f946c007de0 comp rx=0 tx=0).stop 2024-03-23T09:41:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.771+0000 7f9488eaf640 1 -- 172.21.15.47:0/1993437436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94840cf8a0 msgr2=0x7f9484149e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.771+0000 7f9488eaf640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94840cf8a0 0x7f9484149e40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9474009d90 tx=0x7f9474004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.771+0000 7f9488eaf640 1 -- 172.21.15.47:0/1993437436 shutdown_connections 2024-03-23T09:41:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.771+0000 7f9488eaf640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f946406ab70 0x7f946406d030 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.771+0000 7f9488eaf640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f948415d0f0 0x7f94840b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.771+0000 7f9488eaf640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f948415a860 0x7f94840c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.771+0000 7f9488eaf640 1 --2- 172.21.15.47:0/1993437436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94840cf8a0 0x7f9484149e40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.771+0000 7f9488eaf640 1 -- 172.21.15.47:0/1993437436 >> 172.21.15.47:0/1993437436 conn(0x7f94840659d0 msgr2=0x7f948415ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:01.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.773+0000 7f9488eaf640 1 -- 172.21.15.47:0/1993437436 shutdown_connections 2024-03-23T09:41:01.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.773+0000 7f9488eaf640 1 -- 172.21.15.47:0/1993437436 wait complete. 2024-03-23T09:41:01.785 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T09:41:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.898+0000 7f56eafdc640 1 Processor -- start 2024-03-23T09:41:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.898+0000 7f56eafdc640 1 -- start start 2024-03-23T09:41:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.899+0000 7f56eafdc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f56e415cbb0 0x7f56e40b8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.899+0000 7f56eafdc640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e415cf90 con 0x7f56e40d1df0 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.899+0000 7f56eafdc640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e40d0d50 con 0x7f56e40b4ac0 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.899+0000 7f56eafdc640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e40b5210 con 0x7f56e415cbb0 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.899+0000 7f56e8d78640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f56e415cbb0 0x7f56e40b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.899+0000 7f56e8d78640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f56e415cbb0 0x7f56e40b8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56884/0 (socket says 172.21.15.47:56884) 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.899+0000 7f56e8d78640 1 -- 172.21.15.47:0/3075835297 learned_addr learned my addr 172.21.15.47:0/3075835297 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.899+0000 7f56dbfff640 1 --1- 172.21.15.47:0/3075835297 >> v1:172.21.15.47:6789/0 conn(0x7f56e40d1df0 0x7f56e40b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34180/0 (socket says 172.21.15.47:34180) 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.899+0000 7f56e8d78640 1 -- 172.21.15.47:0/3075835297 >> v1:172.21.15.175:6789/0 conn(0x7f56e40b4ac0 legacy=0x7f56e40b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.899+0000 7f56e8d78640 1 -- 172.21.15.47:0/3075835297 >> v1:172.21.15.47:6789/0 conn(0x7f56e40d1df0 legacy=0x7f56e40b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.899+0000 7f56db7fe640 1 -- 172.21.15.47:0/3075835297 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124968263 0 0) 0x7f56e415cf90 con 0x7f56e40d1df0 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.899+0000 7f56e8d78640 1 -- 172.21.15.47:0/3075835297 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e40d1030 con 0x7f56e415cbb0 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.900+0000 7f56e8d78640 1 --2- 172.21.15.47:0/3075835297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e415cbb0 0x7f56e40b8250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009870 tx=0x7f56e002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd49adeb6b4e97ed server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.900+0000 7f56db7fe640 1 -- 172.21.15.47:0/3075835297 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e002f840 con 0x7f56e415cbb0 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.900+0000 7f56db7fe640 1 -- 172.21.15.47:0/3075835297 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e002f9a0 con 0x7f56e415cbb0 2024-03-23T09:41:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.900+0000 7f56db7fe640 1 -- 172.21.15.47:0/3075835297 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e002fd30 con 0x7f56e415cbb0 2024-03-23T09:41:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.901+0000 7f56eafdc640 1 -- 172.21.15.47:0/3075835297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e415cbb0 msgr2=0x7f56e40b8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.901+0000 7f56eafdc640 1 --2- 172.21.15.47:0/3075835297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e415cbb0 0x7f56e40b8250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009870 tx=0x7f56e002ba20 comp rx=0 tx=0).stop 2024-03-23T09:41:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.901+0000 7f56eafdc640 1 -- 172.21.15.47:0/3075835297 shutdown_connections 2024-03-23T09:41:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.901+0000 7f56eafdc640 1 --2- 172.21.15.47:0/3075835297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e415cbb0 0x7f56e40b8250 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.901+0000 7f56eafdc640 1 -- 172.21.15.47:0/3075835297 >> 172.21.15.47:0/3075835297 conn(0x7f56e4065dc0 msgr2=0x7f56e40661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.901+0000 7f56eafdc640 1 -- 172.21.15.47:0/3075835297 shutdown_connections 2024-03-23T09:41:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.901+0000 7f56eafdc640 1 -- 172.21.15.47:0/3075835297 wait complete. 2024-03-23T09:41:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.902+0000 7f56eafdc640 1 Processor -- start 2024-03-23T09:41:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.902+0000 7f56eafdc640 1 -- start start 2024-03-23T09:41:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56eafdc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56e40b4ac0 0x7f56e41f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56eafdc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e40d1df0 0x7f56e40ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56eafdc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56e415cbb0 0x7f56e415b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56eafdc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41f6560 con 0x7f56e40d1df0 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56eafdc640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41f23c0 con 0x7f56e415cbb0 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56eafdc640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41f26a0 con 0x7f56e40b4ac0 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56dbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e40d1df0 0x7f56e40ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56dbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e40d1df0 0x7f56e40ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56892/0 (socket says 172.21.15.47:56892) 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56dbfff640 1 -- 172.21.15.47:0/589400269 learned_addr learned my addr 172.21.15.47:0/589400269 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56e8d78640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56e40b4ac0 0x7f56e41f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56e9579640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56e415cbb0 0x7f56e415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56dbfff640 1 -- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56e40b4ac0 msgr2=0x7f56e41f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56dbfff640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56e40b4ac0 0x7f56e41f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56dbfff640 1 -- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56e415cbb0 msgr2=0x7f56e415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56dbfff640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56e415cbb0 0x7f56e415b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56dbfff640 1 -- 172.21.15.47:0/589400269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e00094d0 con 0x7f56e40d1df0 2024-03-23T09:41:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56dbfff640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e40d1df0 0x7f56e40ccd80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f56d400c4a0 tx=0x7f56d400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56d97fa640 1 -- 172.21.15.47:0/589400269 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d40074e0 con 0x7f56e40d1df0 2024-03-23T09:41:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56d97fa640 1 -- 172.21.15.47:0/589400269 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d4007640 con 0x7f56e40d1df0 2024-03-23T09:41:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56d97fa640 1 -- 172.21.15.47:0/589400269 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4004a00 con 0x7f56e40d1df0 2024-03-23T09:41:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56eafdc640 1 -- 172.21.15.47:0/589400269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e41f2980 con 0x7f56e40d1df0 2024-03-23T09:41:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.903+0000 7f56eafdc640 1 -- 172.21.15.47:0/589400269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e41f2f40 con 0x7f56e40d1df0 2024-03-23T09:41:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.904+0000 7f56eafdc640 1 -- 172.21.15.47:0/589400269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e4072360 con 0x7f56e40d1df0 2024-03-23T09:41:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.905+0000 7f56d97fa640 1 -- 172.21.15.47:0/589400269 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f56d4004b60 con 0x7f56e40d1df0 2024-03-23T09:41:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.905+0000 7f56d97fa640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f56d006ac00 0x7f56d006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.905+0000 7f56d97fa640 1 -- 172.21.15.47:0/589400269 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f56d40882c0 con 0x7f56e40d1df0 2024-03-23T09:41:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.905+0000 7f56e8d78640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f56d006ac00 0x7f56d006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.906+0000 7f56e8d78640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f56d006ac00 0x7f56d006d0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f56e0036000 tx=0x7f56e0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:01.910+0000 7f56d97fa640 1 -- 172.21.15.47:0/589400269 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f56d4053730 con 0x7f56e40d1df0 2024-03-23T09:41:02.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.088+0000 7f56eafdc640 1 -- 172.21.15.47:0/589400269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f56e4066be0 con 0x7f56e40d1df0 2024-03-23T09:41:02.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.771+0000 7f56d97fa640 1 -- 172.21.15.47:0/589400269 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f56d4056d50 con 0x7f56e40d1df0 2024-03-23T09:41:02.771 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-03-23T09:41:02.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.776+0000 7f56eafdc640 1 -- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f56d006ac00 msgr2=0x7f56d006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:02.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.776+0000 7f56eafdc640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f56d006ac00 0x7f56d006d0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f56e0036000 tx=0x7f56e0008030 comp rx=0 tx=0).stop 2024-03-23T09:41:02.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.776+0000 7f56eafdc640 1 -- 172.21.15.47:0/589400269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e40d1df0 msgr2=0x7f56e40ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:02.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.776+0000 7f56eafdc640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e40d1df0 0x7f56e40ccd80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f56d400c4a0 tx=0x7f56d400c960 comp rx=0 tx=0).stop 2024-03-23T09:41:02.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.778+0000 7f56eafdc640 1 -- 172.21.15.47:0/589400269 shutdown_connections 2024-03-23T09:41:02.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.778+0000 7f56eafdc640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f56d006ac00 0x7f56d006d0c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:02.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.778+0000 7f56eafdc640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56e415cbb0 0x7f56e415b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:02.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.779+0000 7f56eafdc640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e40d1df0 0x7f56e40ccd80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:02.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.779+0000 7f56eafdc640 1 --2- 172.21.15.47:0/589400269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56e40b4ac0 0x7f56e41f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:02.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.779+0000 7f56eafdc640 1 -- 172.21.15.47:0/589400269 >> 172.21.15.47:0/589400269 conn(0x7f56e4065dc0 msgr2=0x7f56e40b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:02.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.781+0000 7f56eafdc640 1 -- 172.21.15.47:0/589400269 shutdown_connections 2024-03-23T09:41:02.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.782+0000 7f56eafdc640 1 -- 172.21.15.47:0/589400269 wait complete. 2024-03-23T09:41:02.799 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T09:41:02.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.908+0000 7fdff1854640 1 Processor -- start 2024-03-23T09:41:02.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.908+0000 7fdff1854640 1 -- start start 2024-03-23T09:41:02.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.909+0000 7fdff1854640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b8350 0x7fdfec160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.909+0000 7fdff1854640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec161630 con 0x7fdfec0cde10 2024-03-23T09:41:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.909+0000 7fdff1854640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec0d1030 con 0x7fdfec15c990 2024-03-23T09:41:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.909+0000 7fdff1854640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec0cfe10 con 0x7fdfec0b8350 2024-03-23T09:41:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.909+0000 7fdfea7fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdfec0cde10 0x7fdfec0d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34188/0 (socket says 172.21.15.47:34188) 2024-03-23T09:41:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.909+0000 7fdfea7fc640 1 -- 172.21.15.47:0/1405333911 learned_addr learned my addr 172.21.15.47:0/1405333911 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.909+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67930435 0 0) 0x7fdfec161630 con 0x7fdfec0cde10 2024-03-23T09:41:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.909+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd4003520 con 0x7fdfec0cde10 2024-03-23T09:41:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.909+0000 7fdfeb7fe640 1 --2- 172.21.15.47:0/1405333911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b8350 0x7fdfec160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 756199381 0 0) 0x7fdfd4003520 con 0x7fdfec0cde10 2024-03-23T09:41:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b8350 msgr2=0x7fdfec160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfe9ffb640 1 --2- 172.21.15.47:0/1405333911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b8350 0x7fdfec160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 >> v1:172.21.15.175:6789/0 conn(0x7fdfec15c990 legacy=0x7fdfec0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec0b5b60 con 0x7fdfec0cde10 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fdfe0001670 con 0x7fdfec0cde10 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfeb7fe640 1 --2- 172.21.15.47:0/1405333911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b8350 0x7fdfec160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 >> v1:172.21.15.47:6789/0 conn(0x7fdfec0cde10 legacy=0x7fdfec0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfe9ffb640 1 --2- 172.21.15.47:0/1405333911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfd40039a0 0x7fdfd4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfe9ffb640 1 --2- 172.21.15.47:0/1405333911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b8350 0x7fdfd40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfe9ffb640 1 --2- 172.21.15.47:0/1405333911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfd4004d00 0x7fdfd4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec0cfe10 con 0x7fdfd4004d00 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd400bb10 con 0x7fdfd40039a0 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd400bcd0 con 0x7fdfec0b8350 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfeb7fe640 1 --2- 172.21.15.47:0/1405333911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfd4004d00 0x7fdfd4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfea7fc640 1 --2- 172.21.15.47:0/1405333911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b8350 0x7fdfd40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfeaffd640 1 --2- 172.21.15.47:0/1405333911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfd40039a0 0x7fdfd4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfeb7fe640 1 -- 172.21.15.47:0/1405333911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b8350 msgr2=0x7fdfd40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfeb7fe640 1 --2- 172.21.15.47:0/1405333911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b8350 0x7fdfd40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:02.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfeb7fe640 1 -- 172.21.15.47:0/1405333911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfd40039a0 msgr2=0x7fdfd4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:02.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfeb7fe640 1 --2- 172.21.15.47:0/1405333911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfd40039a0 0x7fdfd4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:02.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.910+0000 7fdfeb7fe640 1 -- 172.21.15.47:0/1405333911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd4003520 con 0x7fdfd4004d00 2024-03-23T09:41:02.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.911+0000 7fdfeb7fe640 1 --2- 172.21.15.47:0/1405333911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfd4004d00 0x7fdfd4005180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40098a0 tx=0x7fdfd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25876a3ca48254c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:02.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.911+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4004230 con 0x7fdfd4004d00 2024-03-23T09:41:02.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.911+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe4004390 con 0x7fdfd4004d00 2024-03-23T09:41:02.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.911+0000 7fdfe9ffb640 1 -- 172.21.15.47:0/1405333911 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4003830 con 0x7fdfd4004d00 2024-03-23T09:41:02.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.911+0000 7fdff1854640 1 -- 172.21.15.47:0/1405333911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfd4004d00 msgr2=0x7fdfd4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:02.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.911+0000 7fdff1854640 1 --2- 172.21.15.47:0/1405333911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfd4004d00 0x7fdfd4005180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40098a0 tx=0x7fdfd40057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:02.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.911+0000 7fdfeb7fe640 1 -- 172.21.15.47:0/1405333911 reap_dead start 2024-03-23T09:41:02.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.912+0000 7fdff1854640 1 -- 172.21.15.47:0/1405333911 shutdown_connections 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.912+0000 7fdff1854640 1 -- 172.21.15.47:0/1405333911 >> 172.21.15.47:0/1405333911 conn(0x7fdfec0659d0 msgr2=0x7fdfec065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.912+0000 7fdff1854640 1 -- 172.21.15.47:0/1405333911 shutdown_connections 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.912+0000 7fdff1854640 1 -- 172.21.15.47:0/1405333911 wait complete. 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.913+0000 7fdff1854640 1 Processor -- start 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.913+0000 7fdff1854640 1 -- start start 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.913+0000 7fdff1854640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfd4004d00 0x7fdfec15f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.913+0000 7fdff1854640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfec150e00 0x7fdfec0b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdff1854640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b7530 0x7fdfec0c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdff1854640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec0b79b0 con 0x7fdfd4004d00 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdff1854640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec15fc90 con 0x7fdfec150e00 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdff1854640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec1f2440 con 0x7fdfec0b7530 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfeaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfd4004d00 0x7fdfec15f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfeaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfd4004d00 0x7fdfec15f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56902/0 (socket says 172.21.15.47:56902) 2024-03-23T09:41:02.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfeaffd640 1 -- 172.21.15.47:0/888200802 learned_addr learned my addr 172.21.15.47:0/888200802 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfeb7fe640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b7530 0x7fdfec0c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfeaffd640 1 -- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b7530 msgr2=0x7fdfec0c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfeaffd640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b7530 0x7fdfec0c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfea7fc640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfec150e00 0x7fdfec0b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfeaffd640 1 -- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfec150e00 msgr2=0x7fdfec0b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfeaffd640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfec150e00 0x7fdfec0b6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfeaffd640 1 -- 172.21.15.47:0/888200802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec1f2720 con 0x7fdfd4004d00 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfeb7fe640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b7530 0x7fdfec0c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfeaffd640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfd4004d00 0x7fdfec15f750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00fbc0 tx=0x7fdfdc00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfd3fff640 1 -- 172.21.15.47:0/888200802 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc00fec0 con 0x7fdfd4004d00 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfd3fff640 1 -- 172.21.15.47:0/888200802 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfdc00dbd0 con 0x7fdfd4004d00 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdff1854640 1 -- 172.21.15.47:0/888200802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfec1f2a00 con 0x7fdfd4004d00 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdfd3fff640 1 -- 172.21.15.47:0/888200802 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc003970 con 0x7fdfd4004d00 2024-03-23T09:41:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.914+0000 7fdff1854640 1 -- 172.21.15.47:0/888200802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec1f2fc0 con 0x7fdfd4004d00 2024-03-23T09:41:02.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.915+0000 7fdff1854640 1 -- 172.21.15.47:0/888200802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfec072360 con 0x7fdfd4004d00 2024-03-23T09:41:02.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.916+0000 7fdfd3fff640 1 -- 172.21.15.47:0/888200802 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdfdc003ad0 con 0x7fdfd4004d00 2024-03-23T09:41:02.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.916+0000 7fdfd3fff640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdfd806ac00 0x7fdfd806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:02.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.916+0000 7fdfd3fff640 1 -- 172.21.15.47:0/888200802 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fdfdc0894a0 con 0x7fdfd4004d00 2024-03-23T09:41:02.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.916+0000 7fdfea7fc640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdfd806ac00 0x7fdfd806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:02.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.917+0000 7fdfea7fc640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdfd806ac00 0x7fdfd806d0c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0b8250 tx=0x7fdfe0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:02.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:02.920+0000 7fdfd3fff640 1 -- 172.21.15.47:0/888200802 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdfdc054770 con 0x7fdfd4004d00 2024-03-23T09:41:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.136+0000 7fdff1854640 1 -- 172.21.15.47:0/888200802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdfec066be0 con 0x7fdfd4004d00 2024-03-23T09:41:03.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.141+0000 7fdfd3fff640 1 -- 172.21.15.47:0/888200802 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdfdc057d90 con 0x7fdfd4004d00 2024-03-23T09:41:03.140 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 2 and data pool 3 2024-03-23T09:41:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.144+0000 7fdff1854640 1 -- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdfd806ac00 msgr2=0x7fdfd806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.144+0000 7fdff1854640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdfd806ac00 0x7fdfd806d0c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0b8250 tx=0x7fdfe0009040 comp rx=0 tx=0).stop 2024-03-23T09:41:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.144+0000 7fdff1854640 1 -- 172.21.15.47:0/888200802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfd4004d00 msgr2=0x7fdfec15f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.144+0000 7fdff1854640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfd4004d00 0x7fdfec15f750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00fbc0 tx=0x7fdfdc00d520 comp rx=0 tx=0).stop 2024-03-23T09:41:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.145+0000 7fdff1854640 1 -- 172.21.15.47:0/888200802 shutdown_connections 2024-03-23T09:41:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.145+0000 7fdff1854640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdfd806ac00 0x7fdfd806d0c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.145+0000 7fdff1854640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfec0b7530 0x7fdfec0c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.145+0000 7fdff1854640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfec150e00 0x7fdfec0b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.146+0000 7fdff1854640 1 --2- 172.21.15.47:0/888200802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfd4004d00 0x7fdfec15f750 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.146+0000 7fdff1854640 1 -- 172.21.15.47:0/888200802 >> 172.21.15.47:0/888200802 conn(0x7fdfec0659d0 msgr2=0x7fdfec0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:03.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.146+0000 7fdff1854640 1 -- 172.21.15.47:0/888200802 shutdown_connections 2024-03-23T09:41:03.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.147+0000 7fdff1854640 1 -- 172.21.15.47:0/888200802 wait complete. 2024-03-23T09:41:03.166 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.267+0000 7f8cc5aaf640 1 Processor -- start 2024-03-23T09:41:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.267+0000 7f8cc5aaf640 1 -- start start 2024-03-23T09:41:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.268+0000 7f8cc5aaf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8cc015ca90 0x7f8cc0160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.268+0000 7f8cc5aaf640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cc00b8350 0x7f8cc00cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.268+0000 7f8cc5aaf640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc00d11e0 con 0x7f8cc00cdc40 2024-03-23T09:41:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.268+0000 7f8cc5aaf640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc00ce0d0 con 0x7f8cc015ca90 2024-03-23T09:41:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.268+0000 7f8cc5aaf640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00d2680 con 0x7f8cc00b8350 2024-03-23T09:41:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.268+0000 7f8cbf7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8cc015ca90 0x7f8cc0160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.268+0000 7f8cbf7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8cc015ca90 0x7f8cc0160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56910/0 (socket says 172.21.15.47:56910) 2024-03-23T09:41:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.268+0000 7f8cbf7fe640 1 -- 172.21.15.47:0/2535328892 learned_addr learned my addr 172.21.15.47:0/2535328892 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.268+0000 7f8cbffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8cc00cdc40 0x7f8cc00b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34194/0 (socket says 172.21.15.47:34194) 2024-03-23T09:41:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.268+0000 7f8cbeffd640 1 --2- 172.21.15.47:0/2535328892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cc00b8350 0x7f8cc00cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.268+0000 7f8cbf7fe640 1 -- 172.21.15.47:0/2535328892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cc00b8350 msgr2=0x7f8cc00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.269+0000 7f8cbf7fe640 1 --2- 172.21.15.47:0/2535328892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cc00b8350 0x7f8cc00cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.269+0000 7f8cbf7fe640 1 -- 172.21.15.47:0/2535328892 >> v1:172.21.15.47:6789/0 conn(0x7f8cc00cdc40 legacy=0x7f8cc00b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.269+0000 7f8cbe7fc640 1 -- 172.21.15.47:0/2535328892 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862153183 0 0) 0x7f8cc00d11e0 con 0x7f8cc00cdc40 2024-03-23T09:41:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.269+0000 7f8cbf7fe640 1 -- 172.21.15.47:0/2535328892 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0160f60 con 0x7f8cc015ca90 2024-03-23T09:41:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.269+0000 7f8cbeffd640 1 --2- 172.21.15.47:0/2535328892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cc00b8350 0x7f8cc00cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.269+0000 7f8cbf7fe640 1 --2- 172.21.15.47:0/2535328892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cc015ca90 0x7f8cc0160710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009870 tx=0x7f8cb002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18dacf65279ecaa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.269+0000 7f8cbe7fc640 1 -- 172.21.15.47:0/2535328892 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0031640 con 0x7f8cc015ca90 2024-03-23T09:41:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.269+0000 7f8cbe7fc640 1 -- 172.21.15.47:0/2535328892 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb00317a0 con 0x7f8cc015ca90 2024-03-23T09:41:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.269+0000 7f8cbe7fc640 1 -- 172.21.15.47:0/2535328892 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0031b30 con 0x7f8cc015ca90 2024-03-23T09:41:03.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.270+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/2535328892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cc015ca90 msgr2=0x7f8cc0160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.270+0000 7f8cc5aaf640 1 --2- 172.21.15.47:0/2535328892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cc015ca90 0x7f8cc0160710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009870 tx=0x7f8cb002b8f0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.270+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/2535328892 shutdown_connections 2024-03-23T09:41:03.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.270+0000 7f8cc5aaf640 1 --2- 172.21.15.47:0/2535328892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cc00b8350 0x7f8cc00cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.270+0000 7f8cc5aaf640 1 --2- 172.21.15.47:0/2535328892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cc015ca90 0x7f8cc0160710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.270+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/2535328892 >> 172.21.15.47:0/2535328892 conn(0x7f8cc00659d0 msgr2=0x7f8cc0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.270+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/2535328892 shutdown_connections 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.270+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/2535328892 wait complete. 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.271+0000 7f8cc5aaf640 1 Processor -- start 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.271+0000 7f8cc5aaf640 1 -- start start 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.272+0000 7f8cc5aaf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cc00b8350 0x7f8cc0150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.272+0000 7f8cc5aaf640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cc00cdc40 0x7f8cc01529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.272+0000 7f8cc5aaf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cc015ca90 0x7f8cc00b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.272+0000 7f8cc5aaf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0153e40 con 0x7f8cc015ca90 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.272+0000 7f8cc5aaf640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc01532c0 con 0x7f8cc00b8350 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.272+0000 7f8cc5aaf640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc01535a0 con 0x7f8cc00cdc40 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.272+0000 7f8cbffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cc015ca90 0x7f8cc00b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.272+0000 7f8cbffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cc015ca90 0x7f8cc00b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56916/0 (socket says 172.21.15.47:56916) 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.272+0000 7f8cbffff640 1 -- 172.21.15.47:0/3814799984 learned_addr learned my addr 172.21.15.47:0/3814799984 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.272+0000 7f8cbf7fe640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cc00b8350 0x7f8cc0150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.272+0000 7f8cbffff640 1 -- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cc00cdc40 msgr2=0x7f8cc01529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.272+0000 7f8cbeffd640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cc00cdc40 0x7f8cc01529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.272+0000 7f8cbffff640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cc00cdc40 0x7f8cc01529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.273+0000 7f8cbffff640 1 -- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cc00b8350 msgr2=0x7f8cc0150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.273+0000 7f8cbffff640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cc00b8350 0x7f8cc0150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.273+0000 7f8cbffff640 1 -- 172.21.15.47:0/3814799984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb00094d0 con 0x7f8cc015ca90 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.273+0000 7f8cbf7fe640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cc00b8350 0x7f8cc0150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.273+0000 7f8cbffff640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cc015ca90 0x7f8cc00b6680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800c780 tx=0x7f8cb800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.273+0000 7f8ca7fff640 1 -- 172.21.15.47:0/3814799984 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800ce90 con 0x7f8cc015ca90 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.273+0000 7f8ca7fff640 1 -- 172.21.15.47:0/3814799984 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb800a640 con 0x7f8cc015ca90 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.273+0000 7f8ca7fff640 1 -- 172.21.15.47:0/3814799984 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800a9d0 con 0x7f8cc015ca90 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.273+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/3814799984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc0153880 con 0x7f8cc015ca90 2024-03-23T09:41:03.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.273+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/3814799984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc01512f0 con 0x7f8cc015ca90 2024-03-23T09:41:03.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.274+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/3814799984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc0072360 con 0x7f8cc015ca90 2024-03-23T09:41:03.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.274+0000 7f8ca7fff640 1 -- 172.21.15.47:0/3814799984 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8cb800ab30 con 0x7f8cc015ca90 2024-03-23T09:41:03.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.275+0000 7f8ca7fff640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8cac06ac00 0x7f8cac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.275+0000 7f8ca7fff640 1 -- 172.21.15.47:0/3814799984 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8cb8088830 con 0x7f8cc015ca90 2024-03-23T09:41:03.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.275+0000 7f8cbf7fe640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8cac06ac00 0x7f8cac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.275+0000 7f8cbf7fe640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8cac06ac00 0x7f8cac06d0c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0036000 tx=0x7f8cb003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:03.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.279+0000 7f8ca7fff640 1 -- 172.21.15.47:0/3814799984 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8cb8053aa0 con 0x7f8cc015ca90 2024-03-23T09:41:03.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.459+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/3814799984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cc0066be0 con 0x7f8cc015ca90 2024-03-23T09:41:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.460+0000 7f8ca7fff640 1 -- 172.21.15.47:0/3814799984 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16050 (secure 0 0 0) 0x7f8cb8018070 con 0x7f8cc015ca90 2024-03-23T09:41:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:03.460 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:03.137658+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:03.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.462+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8cac06ac00 msgr2=0x7f8cac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.462+0000 7f8cc5aaf640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8cac06ac00 0x7f8cac06d0c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0036000 tx=0x7f8cb003c040 comp rx=0 tx=0).stop 2024-03-23T09:41:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.462+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/3814799984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cc015ca90 msgr2=0x7f8cc00b6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.462+0000 7f8cc5aaf640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cc015ca90 0x7f8cc00b6680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800c780 tx=0x7f8cb800cc40 comp rx=0 tx=0).stop 2024-03-23T09:41:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.462+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/3814799984 shutdown_connections 2024-03-23T09:41:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.462+0000 7f8cc5aaf640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8cac06ac00 0x7f8cac06d0c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.462+0000 7f8cc5aaf640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cc015ca90 0x7f8cc00b6680 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.462+0000 7f8cc5aaf640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cc00cdc40 0x7f8cc01529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.462+0000 7f8cc5aaf640 1 --2- 172.21.15.47:0/3814799984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cc00b8350 0x7f8cc0150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.462+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/3814799984 >> 172.21.15.47:0/3814799984 conn(0x7f8cc00659d0 msgr2=0x7f8cc015bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.463+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/3814799984 shutdown_connections 2024-03-23T09:41:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.463+0000 7f8cc5aaf640 1 -- 172.21.15.47:0/3814799984 wait complete. 2024-03-23T09:41:03.473 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:03.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.574+0000 7f4eadf63640 1 Processor -- start 2024-03-23T09:41:03.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.574+0000 7f4eadf63640 1 -- start start 2024-03-23T09:41:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.575+0000 7f4eadf63640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4ea815ca90 0x7f4ea8160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.575+0000 7f4eadf63640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ea80b8350 0x7f4ea80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.575+0000 7f4eadf63640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea80ce090 con 0x7f4ea80cdc40 2024-03-23T09:41:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.575+0000 7f4eadf63640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea80d11e0 con 0x7f4ea80b8350 2024-03-23T09:41:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.575+0000 7f4eadf63640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea80d2680 con 0x7f4ea815ca90 2024-03-23T09:41:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.575+0000 7f4ea6ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ea80b8350 0x7f4ea80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.575+0000 7f4ea6ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ea80b8350 0x7f4ea80cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56922/0 (socket says 172.21.15.47:56922) 2024-03-23T09:41:03.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.575+0000 7f4ea6ffd640 1 -- 172.21.15.47:0/500409714 learned_addr learned my addr 172.21.15.47:0/500409714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:03.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.575+0000 7f4ea6ffd640 1 -- 172.21.15.47:0/500409714 >> v2:172.21.15.175:3300/0 conn(0x7f4ea815ca90 msgr2=0x7f4ea8160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:03.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.575+0000 7f4ea6ffd640 1 --2- 172.21.15.47:0/500409714 >> v2:172.21.15.175:3300/0 conn(0x7f4ea815ca90 0x7f4ea8160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.575+0000 7f4ea6ffd640 1 -- 172.21.15.47:0/500409714 >> v1:172.21.15.175:6789/0 conn(0x7f4ea80cdc40 legacy=0x7f4ea80b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:03.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.575+0000 7f4ea6ffd640 1 -- 172.21.15.47:0/500409714 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea8160f60 con 0x7f4ea80b8350 2024-03-23T09:41:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.576+0000 7f4ea6ffd640 1 --2- 172.21.15.47:0/500409714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ea80b8350 0x7f4ea80cd700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8066680 tx=0x7f4e9c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df013e4e1941a36e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.576+0000 7f4ea67fc640 1 -- 172.21.15.47:0/500409714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0317e0 con 0x7f4ea80b8350 2024-03-23T09:41:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.576+0000 7f4ea67fc640 1 -- 172.21.15.47:0/500409714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c031940 con 0x7f4ea80b8350 2024-03-23T09:41:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.576+0000 7f4ea67fc640 1 -- 172.21.15.47:0/500409714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c031cd0 con 0x7f4ea80b8350 2024-03-23T09:41:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.577+0000 7f4eadf63640 1 -- 172.21.15.47:0/500409714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ea80b8350 msgr2=0x7f4ea80cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.577+0000 7f4eadf63640 1 --2- 172.21.15.47:0/500409714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ea80b8350 0x7f4ea80cd700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8066680 tx=0x7f4e9c02b880 comp rx=0 tx=0).stop 2024-03-23T09:41:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.577+0000 7f4eadf63640 1 -- 172.21.15.47:0/500409714 shutdown_connections 2024-03-23T09:41:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.577+0000 7f4eadf63640 1 --2- 172.21.15.47:0/500409714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ea80b8350 0x7f4ea80cd700 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.577+0000 7f4eadf63640 1 --2- 172.21.15.47:0/500409714 >> v2:172.21.15.175:3300/0 conn(0x7f4ea815ca90 0x7f4ea8160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.577+0000 7f4eadf63640 1 -- 172.21.15.47:0/500409714 >> 172.21.15.47:0/500409714 conn(0x7f4ea80659d0 msgr2=0x7f4ea8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.577+0000 7f4eadf63640 1 -- 172.21.15.47:0/500409714 shutdown_connections 2024-03-23T09:41:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.577+0000 7f4eadf63640 1 -- 172.21.15.47:0/500409714 wait complete. 2024-03-23T09:41:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.578+0000 7f4eadf63640 1 Processor -- start 2024-03-23T09:41:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.578+0000 7f4eadf63640 1 -- start start 2024-03-23T09:41:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4eadf63640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ea80b8350 0x7f4ea81509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4eadf63640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ea80cdc40 0x7f4ea80cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4eadf63640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ea815ca90 0x7f4ea815ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4eadf63640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea81f22e0 con 0x7f4ea815ca90 2024-03-23T09:41:03.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4eadf63640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea80ca4a0 con 0x7f4ea80cdc40 2024-03-23T09:41:03.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4eadf63640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea80ca780 con 0x7f4ea80b8350 2024-03-23T09:41:03.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ea815ca90 0x7f4ea815ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ea815ca90 0x7f4ea815ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56926/0 (socket says 172.21.15.47:56926) 2024-03-23T09:41:03.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4ea7fff640 1 -- 172.21.15.47:0/2955037431 learned_addr learned my addr 172.21.15.47:0/2955037431 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:03.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4ea6ffd640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ea80cdc40 0x7f4ea80cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4ea77fe640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ea80b8350 0x7f4ea81509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4ea7fff640 1 -- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ea80b8350 msgr2=0x7f4ea81509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4ea7fff640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ea80b8350 0x7f4ea81509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4ea7fff640 1 -- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ea80cdc40 msgr2=0x7f4ea80cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4ea7fff640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ea80cdc40 0x7f4ea80cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4ea7fff640 1 -- 172.21.15.47:0/2955037431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c0094d0 con 0x7f4ea815ca90 2024-03-23T09:41:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4ea77fe640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ea80b8350 0x7f4ea81509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.579+0000 7f4ea7fff640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ea815ca90 0x7f4ea815ef10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000c780 tx=0x7f4ea000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.580+0000 7f4e8ffff640 1 -- 172.21.15.47:0/2955037431 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0003d80 con 0x7f4ea815ca90 2024-03-23T09:41:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.580+0000 7f4e8ffff640 1 -- 172.21.15.47:0/2955037431 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea000ce90 con 0x7f4ea815ca90 2024-03-23T09:41:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.580+0000 7f4e8ffff640 1 -- 172.21.15.47:0/2955037431 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00052b0 con 0x7f4ea815ca90 2024-03-23T09:41:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.580+0000 7f4eadf63640 1 -- 172.21.15.47:0/2955037431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea80caa60 con 0x7f4ea815ca90 2024-03-23T09:41:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.580+0000 7f4eadf63640 1 -- 172.21.15.47:0/2955037431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea80cb020 con 0x7f4ea815ca90 2024-03-23T09:41:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.581+0000 7f4eadf63640 1 -- 172.21.15.47:0/2955037431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea8072360 con 0x7f4ea815ca90 2024-03-23T09:41:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.581+0000 7f4e8ffff640 1 -- 172.21.15.47:0/2955037431 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4ea0005410 con 0x7f4ea815ca90 2024-03-23T09:41:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.581+0000 7f4e8ffff640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4e9406ac30 0x7f4e9406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.582+0000 7f4e8ffff640 1 -- 172.21.15.47:0/2955037431 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ea0088700 con 0x7f4ea815ca90 2024-03-23T09:41:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.582+0000 7f4ea77fe640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4e9406ac30 0x7f4e9406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.582+0000 7f4ea77fe640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4e9406ac30 0x7f4e9406d0f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009930 tx=0x7f4e98008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:03.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.586+0000 7f4e8ffff640 1 -- 172.21.15.47:0/2955037431 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4ea0053900 con 0x7f4ea815ca90 2024-03-23T09:41:03.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.766+0000 7f4eadf63640 1 -- 172.21.15.47:0/2955037431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ea80cb210 con 0x7f4ea815ca90 2024-03-23T09:41:03.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.767+0000 7f4e8ffff640 1 -- 172.21.15.47:0/2955037431 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16050 (secure 0 0 0) 0x7f4ea0056f20 con 0x7f4ea815ca90 2024-03-23T09:41:03.767 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:03.767 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:03.137658+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:03.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.770+0000 7f4eadf63640 1 -- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4e9406ac30 msgr2=0x7f4e9406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.770+0000 7f4eadf63640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4e9406ac30 0x7f4e9406d0f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009930 tx=0x7f4e98008040 comp rx=0 tx=0).stop 2024-03-23T09:41:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.770+0000 7f4eadf63640 1 -- 172.21.15.47:0/2955037431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ea815ca90 msgr2=0x7f4ea815ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.770+0000 7f4eadf63640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ea815ca90 0x7f4ea815ef10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000c780 tx=0x7f4ea000cc40 comp rx=0 tx=0).stop 2024-03-23T09:41:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.770+0000 7f4eadf63640 1 -- 172.21.15.47:0/2955037431 shutdown_connections 2024-03-23T09:41:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.770+0000 7f4eadf63640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4e9406ac30 0x7f4e9406d0f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.770+0000 7f4eadf63640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ea815ca90 0x7f4ea815ef10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.770+0000 7f4eadf63640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ea80cdc40 0x7f4ea80cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.770+0000 7f4eadf63640 1 --2- 172.21.15.47:0/2955037431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ea80b8350 0x7f4ea81509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.770+0000 7f4eadf63640 1 -- 172.21.15.47:0/2955037431 >> 172.21.15.47:0/2955037431 conn(0x7f4ea80659d0 msgr2=0x7f4ea80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.770+0000 7f4eadf63640 1 -- 172.21.15.47:0/2955037431 shutdown_connections 2024-03-23T09:41:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.770+0000 7f4eadf63640 1 -- 172.21.15.47:0/2955037431 wait complete. 2024-03-23T09:41:03.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T09:41:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.884+0000 7f03a4500640 1 Processor -- start 2024-03-23T09:41:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.884+0000 7f03a4500640 1 -- start start 2024-03-23T09:41:03.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.885+0000 7f03a4500640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f039c15c990 0x7f039c160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.885+0000 7f03a4500640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039c161630 con 0x7f039c0cde10 2024-03-23T09:41:03.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.885+0000 7f03a4500640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039c0d1030 con 0x7f039c0b8350 2024-03-23T09:41:03.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.885+0000 7f03a4500640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c0cfe10 con 0x7f039c15c990 2024-03-23T09:41:03.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.885+0000 7f03a2a9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f039c0cde10 0x7f039c0d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34210/0 (socket says 172.21.15.47:34210) 2024-03-23T09:41:03.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.885+0000 7f03a2a9d640 1 -- 172.21.15.47:0/978690246 learned_addr learned my addr 172.21.15.47:0/978690246 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.885+0000 7f03a229c640 1 --2- 172.21.15.47:0/978690246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f039c15c990 0x7f039c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.885+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571127798 0 0) 0x7f039c161630 con 0x7f039c0cde10 2024-03-23T09:41:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.885+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0384003520 con 0x7f039c0cde10 2024-03-23T09:41:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.885+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406063821 0 0) 0x7f039c0d1030 con 0x7f039c0b8350 2024-03-23T09:41:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.885+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f039c161630 con 0x7f039c0b8350 2024-03-23T09:41:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 294557083 0 0) 0x7f0384003520 con 0x7f039c0cde10 2024-03-23T09:41:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f039c15c990 msgr2=0x7f039c160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a129a640 1 --2- 172.21.15.47:0/978690246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f039c15c990 0x7f039c160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 >> v1:172.21.15.175:6789/0 conn(0x7f039c0b8350 legacy=0x7f039c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0b5b60 con 0x7f039c0cde10 2024-03-23T09:41:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f038c0019f0 con 0x7f039c0cde10 2024-03-23T09:41:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 >> v1:172.21.15.47:6789/0 conn(0x7f039c0cde10 legacy=0x7f039c0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a129a640 1 --2- 172.21.15.47:0/978690246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0384003b30 0x7f0384003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a129a640 1 --2- 172.21.15.47:0/978690246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f039c15c990 0x7f03840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a129a640 1 --2- 172.21.15.47:0/978690246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0384004d00 0x7f0384005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c0cfe10 con 0x7f0384003b30 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038400bb10 con 0x7f0384004d00 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038400bca0 con 0x7f039c15c990 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a229c640 1 --2- 172.21.15.47:0/978690246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0384003b30 0x7f0384003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a2a9d640 1 --2- 172.21.15.47:0/978690246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0384004d00 0x7f0384005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a1a9b640 1 --2- 172.21.15.47:0/978690246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f039c15c990 0x7f03840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a229c640 1 -- 172.21.15.47:0/978690246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f039c15c990 msgr2=0x7f03840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a229c640 1 --2- 172.21.15.47:0/978690246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f039c15c990 0x7f03840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a229c640 1 -- 172.21.15.47:0/978690246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0384004d00 msgr2=0x7f0384005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a229c640 1 --2- 172.21.15.47:0/978690246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0384004d00 0x7f0384005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.886+0000 7f03a229c640 1 -- 172.21.15.47:0/978690246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0384003520 con 0x7f0384003b30 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.887+0000 7f03a229c640 1 --2- 172.21.15.47:0/978690246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0384003b30 0x7f0384003f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0394009af0 tx=0x7f03840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca7d614eb116296 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.887+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039402c6b0 con 0x7f0384003b30 2024-03-23T09:41:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.887+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039402c810 con 0x7f0384003b30 2024-03-23T09:41:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.887+0000 7f03a129a640 1 -- 172.21.15.47:0/978690246 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039402cba0 con 0x7f0384003b30 2024-03-23T09:41:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.887+0000 7f03a4500640 1 -- 172.21.15.47:0/978690246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0384003b30 msgr2=0x7f0384003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.887+0000 7f03a4500640 1 --2- 172.21.15.47:0/978690246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0384003b30 0x7f0384003f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0394009af0 tx=0x7f03840057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.887+0000 7f03a2a9d640 1 -- 172.21.15.47:0/978690246 reap_dead start 2024-03-23T09:41:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.888+0000 7f03a4500640 1 -- 172.21.15.47:0/978690246 shutdown_connections 2024-03-23T09:41:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.888+0000 7f03a4500640 1 -- 172.21.15.47:0/978690246 >> 172.21.15.47:0/978690246 conn(0x7f039c0659d0 msgr2=0x7f039c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.888+0000 7f03a4500640 1 -- 172.21.15.47:0/978690246 shutdown_connections 2024-03-23T09:41:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.888+0000 7f03a4500640 1 -- 172.21.15.47:0/978690246 wait complete. 2024-03-23T09:41:03.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.889+0000 7f03a4500640 1 Processor -- start 2024-03-23T09:41:03.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.889+0000 7f03a4500640 1 -- start start 2024-03-23T09:41:03.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.889+0000 7f03a4500640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0384003b30 0x7f039c0c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.889+0000 7f03a4500640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039c0b6750 0x7f039c0c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.889+0000 7f03a4500640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c0c8430 0x7f039c0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.889+0000 7f03a4500640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c0c8df0 con 0x7f039c0c8430 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.889+0000 7f03a4500640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c0b7280 con 0x7f039c0b6750 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.889+0000 7f03a4500640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c0b6cc0 con 0x7f0384003b30 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a2a9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c0c8430 0x7f039c0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a2a9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c0c8430 0x7f039c0c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56950/0 (socket says 172.21.15.47:56950) 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a2a9d640 1 -- 172.21.15.47:0/4037139809 learned_addr learned my addr 172.21.15.47:0/4037139809 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a1a9b640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039c0b6750 0x7f039c0c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a229c640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0384003b30 0x7f039c0c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a2a9d640 1 -- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0384003b30 msgr2=0x7f039c0c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a2a9d640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0384003b30 0x7f039c0c6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a2a9d640 1 -- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039c0b6750 msgr2=0x7f039c0c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a2a9d640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039c0b6750 0x7f039c0c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a2a9d640 1 -- 172.21.15.47:0/4037139809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0b6fa0 con 0x7f039c0c8430 2024-03-23T09:41:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a1a9b640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039c0b6750 0x7f039c0c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a229c640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0384003b30 0x7f039c0c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a2a9d640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c0c8430 0x7f039c0c88b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f039c0b87b0 tx=0x7f038c00a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f039affd640 1 -- 172.21.15.47:0/4037139809 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c001db0 con 0x7f039c0c8430 2024-03-23T09:41:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f039affd640 1 -- 172.21.15.47:0/4037139809 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c00ab80 con 0x7f039c0c8430 2024-03-23T09:41:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a4500640 1 -- 172.21.15.47:0/4037139809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c158650 con 0x7f039c0c8430 2024-03-23T09:41:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f039affd640 1 -- 172.21.15.47:0/4037139809 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0133f0 con 0x7f039c0c8430 2024-03-23T09:41:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.890+0000 7f03a4500640 1 -- 172.21.15.47:0/4037139809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c0c6940 con 0x7f039c0c8430 2024-03-23T09:41:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.891+0000 7f03a4500640 1 -- 172.21.15.47:0/4037139809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c072360 con 0x7f039c0c8430 2024-03-23T09:41:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.892+0000 7f039affd640 1 -- 172.21.15.47:0/4037139809 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f038c013550 con 0x7f039c0c8430 2024-03-23T09:41:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.892+0000 7f039affd640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f038806ac00 0x7f038806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.892+0000 7f039affd640 1 -- 172.21.15.47:0/4037139809 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f038c089920 con 0x7f039c0c8430 2024-03-23T09:41:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.892+0000 7f03a229c640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f038806ac00 0x7f038806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.893+0000 7f03a229c640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f038806ac00 0x7f038806d0c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0394036000 tx=0x7f0394030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:03.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:03.896+0000 7f039affd640 1 -- 172.21.15.47:0/4037139809 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f038c054b00 con 0x7f039c0c8430 2024-03-23T09:41:04.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.110+0000 7f03a4500640 1 -- 172.21.15.47:0/4037139809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f039c066be0 con 0x7f039c0c8430 2024-03-23T09:41:04.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.147+0000 7f039affd640 1 -- 172.21.15.47:0/4037139809 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f038c058120 con 0x7f039c0c8430 2024-03-23T09:41:04.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.149+0000 7f03a4500640 1 -- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f038806ac00 msgr2=0x7f038806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:04.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.149+0000 7f03a4500640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f038806ac00 0x7f038806d0c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0394036000 tx=0x7f0394030000 comp rx=0 tx=0).stop 2024-03-23T09:41:04.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.149+0000 7f03a4500640 1 -- 172.21.15.47:0/4037139809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c0c8430 msgr2=0x7f039c0c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:04.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.149+0000 7f03a4500640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c0c8430 0x7f039c0c88b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f039c0b87b0 tx=0x7f038c00a600 comp rx=0 tx=0).stop 2024-03-23T09:41:04.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.149+0000 7f03a4500640 1 -- 172.21.15.47:0/4037139809 shutdown_connections 2024-03-23T09:41:04.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.149+0000 7f03a4500640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f038806ac00 0x7f038806d0c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:04.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.149+0000 7f03a4500640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c0c8430 0x7f039c0c88b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:04.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.149+0000 7f03a4500640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039c0b6750 0x7f039c0c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:04.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.149+0000 7f03a4500640 1 --2- 172.21.15.47:0/4037139809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0384003b30 0x7f039c0c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:04.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.149+0000 7f03a4500640 1 -- 172.21.15.47:0/4037139809 >> 172.21.15.47:0/4037139809 conn(0x7f039c0659d0 msgr2=0x7f039c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:04.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.149+0000 7f03a4500640 1 -- 172.21.15.47:0/4037139809 shutdown_connections 2024-03-23T09:41:04.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.149+0000 7f03a4500640 1 -- 172.21.15.47:0/4037139809 wait complete. 2024-03-23T09:41:04.159 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-23T09:41:04.159 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-23T09:41:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.263+0000 7f2a4fb89640 1 Processor -- start 2024-03-23T09:41:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.263+0000 7f2a4fb89640 1 -- start start 2024-03-23T09:41:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.263+0000 7f2a4fb89640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a480cde10 0x7f2a48160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4fb89640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a48161630 con 0x7f2a4815c990 2024-03-23T09:41:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4fb89640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a480d1030 con 0x7f2a480b8350 2024-03-23T09:41:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4fb89640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a480cfe10 con 0x7f2a480cde10 2024-03-23T09:41:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4d925640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2a4815c990 0x7f2a480cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34218/0 (socket says 172.21.15.47:34218) 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4d925640 1 -- 172.21.15.47:0/3397954185 learned_addr learned my addr 172.21.15.47:0/3397954185 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4d124640 1 --2- 172.21.15.47:0/3397954185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a480cde10 0x7f2a48160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249091070 0 0) 0x7f2a48161630 con 0x7f2a4815c990 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a34003520 con 0x7f2a4815c990 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743810795 0 0) 0x7f2a480d1030 con 0x7f2a480b8350 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a48161630 con 0x7f2a480b8350 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 320298471 0 0) 0x7f2a34003520 con 0x7f2a4815c990 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a480cde10 msgr2=0x7f2a48160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 --2- 172.21.15.47:0/3397954185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a480cde10 0x7f2a48160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 >> v1:172.21.15.175:6789/0 conn(0x7f2a480b8350 legacy=0x7f2a480d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a480b5b60 con 0x7f2a4815c990 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f2a44001670 con 0x7f2a4815c990 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 >> v1:172.21.15.47:6789/0 conn(0x7f2a4815c990 legacy=0x7f2a480cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 --2- 172.21.15.47:0/3397954185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a34003b30 0x7f2a34003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 --2- 172.21.15.47:0/3397954185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a340047c0 0x7f2a34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 --2- 172.21.15.47:0/3397954185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a480cde10 0x7f2a34005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a480cfe10 con 0x7f2a34003b30 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3400bb80 con 0x7f2a340047c0 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3400bd40 con 0x7f2a480cde10 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4d925640 1 --2- 172.21.15.47:0/3397954185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a34003b30 0x7f2a34003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4d124640 1 --2- 172.21.15.47:0/3397954185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a340047c0 0x7f2a34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.264+0000 7f2a4e126640 1 --2- 172.21.15.47:0/3397954185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a480cde10 0x7f2a34005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.265+0000 7f2a4d925640 1 -- 172.21.15.47:0/3397954185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a480cde10 msgr2=0x7f2a34005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.265+0000 7f2a4d925640 1 --2- 172.21.15.47:0/3397954185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a480cde10 0x7f2a34005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.265+0000 7f2a4d925640 1 -- 172.21.15.47:0/3397954185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a340047c0 msgr2=0x7f2a34004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.265+0000 7f2a4d925640 1 --2- 172.21.15.47:0/3397954185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a340047c0 0x7f2a34004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.265+0000 7f2a4d925640 1 -- 172.21.15.47:0/3397954185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a34003520 con 0x7f2a34003b30 2024-03-23T09:41:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.265+0000 7f2a4d925640 1 --2- 172.21.15.47:0/3397954185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a34003b30 0x7f2a34003f30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2a44007f90 tx=0x7f2a340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bfe40e272bce6d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.266+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44005ad0 con 0x7f2a34003b30 2024-03-23T09:41:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.266+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a44005c30 con 0x7f2a34003b30 2024-03-23T09:41:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.266+0000 7f2a4c923640 1 -- 172.21.15.47:0/3397954185 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4402c790 con 0x7f2a34003b30 2024-03-23T09:41:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.266+0000 7f2a4fb89640 1 -- 172.21.15.47:0/3397954185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a34003b30 msgr2=0x7f2a34003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.266+0000 7f2a4fb89640 1 --2- 172.21.15.47:0/3397954185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a34003b30 0x7f2a34003f30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2a44007f90 tx=0x7f2a340057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.266+0000 7f2a4e126640 1 -- 172.21.15.47:0/3397954185 reap_dead start 2024-03-23T09:41:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.266+0000 7f2a4fb89640 1 -- 172.21.15.47:0/3397954185 shutdown_connections 2024-03-23T09:41:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.266+0000 7f2a4fb89640 1 -- 172.21.15.47:0/3397954185 >> 172.21.15.47:0/3397954185 conn(0x7f2a480659d0 msgr2=0x7f2a48065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.266+0000 7f2a4fb89640 1 -- 172.21.15.47:0/3397954185 shutdown_connections 2024-03-23T09:41:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.266+0000 7f2a4fb89640 1 -- 172.21.15.47:0/3397954185 wait complete. 2024-03-23T09:41:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.267+0000 7f2a4fb89640 1 Processor -- start 2024-03-23T09:41:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.267+0000 7f2a4fb89640 1 -- start start 2024-03-23T09:41:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4fb89640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a34003b30 0x7f2a480cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4fb89640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a481f2ee0 0x7f2a4815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4fb89640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a4815fc00 0x7f2a480c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4fb89640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a480cbdb0 con 0x7f2a4815fc00 2024-03-23T09:41:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4fb89640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a480cb480 con 0x7f2a481f2ee0 2024-03-23T09:41:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4fb89640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a480cb760 con 0x7f2a34003b30 2024-03-23T09:41:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4e126640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a4815fc00 0x7f2a480c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4e126640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a4815fc00 0x7f2a480c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56962/0 (socket says 172.21.15.47:56962) 2024-03-23T09:41:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4e126640 1 -- 172.21.15.47:0/1061008492 learned_addr learned my addr 172.21.15.47:0/1061008492 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4d124640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a481f2ee0 0x7f2a4815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4e126640 1 -- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a34003b30 msgr2=0x7f2a480cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4e126640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a34003b30 0x7f2a480cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4e126640 1 -- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a481f2ee0 msgr2=0x7f2a4815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4e126640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a481f2ee0 0x7f2a4815b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4e126640 1 -- 172.21.15.47:0/1061008492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a480cba40 con 0x7f2a4815fc00 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4d124640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a481f2ee0 0x7f2a4815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.268+0000 7f2a4e126640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a4815fc00 0x7f2a480c9a90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2a480cd210 tx=0x7f2a4000ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.269+0000 7f2a3e7fc640 1 -- 172.21.15.47:0/1061008492 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40006e20 con 0x7f2a4815fc00 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.269+0000 7f2a3e7fc640 1 -- 172.21.15.47:0/1061008492 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a40003970 con 0x7f2a4815fc00 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.269+0000 7f2a4fb89640 1 -- 172.21.15.47:0/1061008492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a481600b0 con 0x7f2a4815fc00 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.269+0000 7f2a3e7fc640 1 -- 172.21.15.47:0/1061008492 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40003d60 con 0x7f2a4815fc00 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.269+0000 7f2a4fb89640 1 -- 172.21.15.47:0/1061008492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a480ca1b0 con 0x7f2a4815fc00 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.270+0000 7f2a4fb89640 1 -- 172.21.15.47:0/1061008492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a480ca430 con 0x7f2a4815fc00 2024-03-23T09:41:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.270+0000 7f2a3e7fc640 1 -- 172.21.15.47:0/1061008492 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2a4000d740 con 0x7f2a4815fc00 2024-03-23T09:41:04.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.270+0000 7f2a3e7fc640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a2c06ac30 0x7f2a2c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:04.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.271+0000 7f2a3e7fc640 1 -- 172.21.15.47:0/1061008492 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a400888a0 con 0x7f2a4815fc00 2024-03-23T09:41:04.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.271+0000 7f2a4d925640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a2c06ac30 0x7f2a2c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:04.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.271+0000 7f2a4d925640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a2c06ac30 0x7f2a2c06d0f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2a44034000 tx=0x7f2a44007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:04.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.275+0000 7f2a3e7fc640 1 -- 172.21.15.47:0/1061008492 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2a400539e0 con 0x7f2a4815fc00 2024-03-23T09:41:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:04.491+0000 7f2a4fb89640 1 -- 172.21.15.47:0/1061008492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2a48072360 con 0x7f2a4815fc00 2024-03-23T09:41:05.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.152+0000 7f2a3e7fc640 1 -- 172.21.15.47:0/1061008492 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2a400190c0 con 0x7f2a4815fc00 2024-03-23T09:41:05.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.154+0000 7f2a4fb89640 1 -- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a2c06ac30 msgr2=0x7f2a2c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.154+0000 7f2a4fb89640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a2c06ac30 0x7f2a2c06d0f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2a44034000 tx=0x7f2a44007000 comp rx=0 tx=0).stop 2024-03-23T09:41:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.154+0000 7f2a4fb89640 1 -- 172.21.15.47:0/1061008492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a4815fc00 msgr2=0x7f2a480c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.154+0000 7f2a4fb89640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a4815fc00 0x7f2a480c9a90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2a480cd210 tx=0x7f2a4000ff70 comp rx=0 tx=0).stop 2024-03-23T09:41:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.154+0000 7f2a4fb89640 1 -- 172.21.15.47:0/1061008492 shutdown_connections 2024-03-23T09:41:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.154+0000 7f2a4fb89640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a2c06ac30 0x7f2a2c06d0f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.154+0000 7f2a4fb89640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a4815fc00 0x7f2a480c9a90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.154+0000 7f2a4fb89640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a481f2ee0 0x7f2a4815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.154+0000 7f2a4fb89640 1 --2- 172.21.15.47:0/1061008492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a34003b30 0x7f2a480cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.154+0000 7f2a4fb89640 1 -- 172.21.15.47:0/1061008492 >> 172.21.15.47:0/1061008492 conn(0x7f2a480659d0 msgr2=0x7f2a480d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.155+0000 7f2a4fb89640 1 -- 172.21.15.47:0/1061008492 shutdown_connections 2024-03-23T09:41:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.155+0000 7f2a4fb89640 1 -- 172.21.15.47:0/1061008492 wait complete. 2024-03-23T09:41:05.167 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-23T09:41:05.167 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:41:05.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.275+0000 7fa19f248640 1 Processor -- start 2024-03-23T09:41:05.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.275+0000 7fa19f248640 1 -- start start 2024-03-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19f248640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19815ca90 0x7fa198160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19f248640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa1980b8350 0x7fa1980cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19f248640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1980cdc40 0x7fa1980b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19f248640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1980ce0d0 con 0x7fa1980cdc40 2024-03-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19f248640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1980d11e0 con 0x7fa1980b8350 2024-03-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19f248640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa198160f60 con 0x7fa19815ca90 2024-03-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19d7e5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1980cdc40 0x7fa1980b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19d7e5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1980cdc40 0x7fa1980b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45774/0 (socket says 172.21.15.47:45774) 2024-03-23T09:41:05.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19d7e5640 1 -- 172.21.15.47:0/175788899 learned_addr learned my addr 172.21.15.47:0/175788899 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:05.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19cfe4640 1 --2- 172.21.15.47:0/175788899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19815ca90 0x7fa198160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa18ffff640 1 --2- 172.21.15.47:0/175788899 >> v2:172.21.15.175:3300/0 conn(0x7fa1980b8350 0x7fa1980cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19d7e5640 1 -- 172.21.15.47:0/175788899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19815ca90 msgr2=0x7fa198160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:05.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19d7e5640 1 --2- 172.21.15.47:0/175788899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19815ca90 0x7fa198160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19d7e5640 1 -- 172.21.15.47:0/175788899 >> v2:172.21.15.175:3300/0 conn(0x7fa1980b8350 msgr2=0x7fa1980cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:05.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19d7e5640 1 --2- 172.21.15.47:0/175788899 >> v2:172.21.15.175:3300/0 conn(0x7fa1980b8350 0x7fa1980cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19d7e5640 1 -- 172.21.15.47:0/175788899 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198161240 con 0x7fa1980cdc40 2024-03-23T09:41:05.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa19cfe4640 1 --2- 172.21.15.47:0/175788899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19815ca90 0x7fa198160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:05.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.276+0000 7fa18ffff640 1 --2- 172.21.15.47:0/175788899 >> v2:172.21.15.175:3300/0 conn(0x7fa1980b8350 0x7fa1980cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.277+0000 7fa19d7e5640 1 --2- 172.21.15.47:0/175788899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1980cdc40 0x7fa1980b4440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa1980b5540 tx=0x7fa19002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8317127cb478f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.277+0000 7fa18f7fe640 1 -- 172.21.15.47:0/175788899 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19002c9e0 con 0x7fa1980cdc40 2024-03-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.277+0000 7fa18f7fe640 1 -- 172.21.15.47:0/175788899 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19002cb40 con 0x7fa1980cdc40 2024-03-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.277+0000 7fa18f7fe640 1 -- 172.21.15.47:0/175788899 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1900343f0 con 0x7fa1980cdc40 2024-03-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.278+0000 7fa19f248640 1 -- 172.21.15.47:0/175788899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1980cdc40 msgr2=0x7fa1980b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.278+0000 7fa19f248640 1 --2- 172.21.15.47:0/175788899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1980cdc40 0x7fa1980b4440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa1980b5540 tx=0x7fa19002be40 comp rx=0 tx=0).stop 2024-03-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.278+0000 7fa19f248640 1 -- 172.21.15.47:0/175788899 shutdown_connections 2024-03-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.278+0000 7fa19f248640 1 --2- 172.21.15.47:0/175788899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1980cdc40 0x7fa1980b4440 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.278+0000 7fa19f248640 1 --2- 172.21.15.47:0/175788899 >> v2:172.21.15.175:3300/0 conn(0x7fa1980b8350 0x7fa1980cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.278+0000 7fa19f248640 1 --2- 172.21.15.47:0/175788899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19815ca90 0x7fa198160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.278+0000 7fa19f248640 1 -- 172.21.15.47:0/175788899 >> 172.21.15.47:0/175788899 conn(0x7fa1980659d0 msgr2=0x7fa198065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.278+0000 7fa19f248640 1 -- 172.21.15.47:0/175788899 shutdown_connections 2024-03-23T09:41:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.278+0000 7fa19f248640 1 -- 172.21.15.47:0/175788899 wait complete. 2024-03-23T09:41:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.279+0000 7fa19f248640 1 Processor -- start 2024-03-23T09:41:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.279+0000 7fa19f248640 1 -- start start 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19f248640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1980b8350 0x7fa1981532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19f248640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1980cdc40 0x7fa198151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19f248640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19815ca90 0x7fa198153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19f248640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980b2550 con 0x7fa1980b8350 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19f248640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1981545e0 con 0x7fa19815ca90 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19f248640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19814d640 con 0x7fa1980cdc40 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19cfe4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1980b8350 0x7fa1981532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19cfe4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1980b8350 0x7fa1981532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45776/0 (socket says 172.21.15.47:45776) 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19cfe4640 1 -- 172.21.15.47:0/334995665 learned_addr learned my addr 172.21.15.47:0/334995665 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19d7e5640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19815ca90 0x7fa198153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa18ffff640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1980cdc40 0x7fa198151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19cfe4640 1 -- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1980cdc40 msgr2=0x7fa198151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19cfe4640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1980cdc40 0x7fa198151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19cfe4640 1 -- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19815ca90 msgr2=0x7fa198153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19cfe4640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19815ca90 0x7fa198153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19cfe4640 1 -- 172.21.15.47:0/334995665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa190009b30 con 0x7fa1980b8350 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19d7e5640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19815ca90 0x7fa198153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa19cfe4640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1980b8350 0x7fa1981532d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c7f0 tx=0x7fa19400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.280+0000 7fa18d7fa640 1 -- 172.21.15.47:0/334995665 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194007a30 con 0x7fa1980b8350 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.281+0000 7fa18d7fa640 1 -- 172.21.15.47:0/334995665 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa194007b90 con 0x7fa1980b8350 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.281+0000 7fa19f248640 1 -- 172.21.15.47:0/334995665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1981522a0 con 0x7fa1980b8350 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.281+0000 7fa18d7fa640 1 -- 172.21.15.47:0/334995665 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19400ce40 con 0x7fa1980b8350 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.281+0000 7fa19f248640 1 -- 172.21.15.47:0/334995665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1981542f0 con 0x7fa1980b8350 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.282+0000 7fa19f248640 1 -- 172.21.15.47:0/334995665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa198072360 con 0x7fa1980b8350 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.282+0000 7fa18d7fa640 1 -- 172.21.15.47:0/334995665 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa1940113f0 con 0x7fa1980b8350 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.282+0000 7fa18d7fa640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa18406ac30 0x7fa18406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.282+0000 7fa18d7fa640 1 -- 172.21.15.47:0/334995665 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa194089440 con 0x7fa1980b8350 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.282+0000 7fa18ffff640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa18406ac30 0x7fa18406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.283+0000 7fa18ffff640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa18406ac30 0x7fa18406d0f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1980b7a20 tx=0x7fa188008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:05.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.286+0000 7fa18d7fa640 1 -- 172.21.15.47:0/334995665 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa1940546b0 con 0x7fa1980b8350 2024-03-23T09:41:05.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.501+0000 7fa19f248640 1 -- 172.21.15.47:0/334995665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1980b2690 con 0x7fa1980b8350 2024-03-23T09:41:05.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.502+0000 7fa18d7fa640 1 -- 172.21.15.47:0/334995665 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4734 (secure 0 0 0) 0x7fa1980b2690 con 0x7fa1980b8350 2024-03-23T09:41:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:05.502 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1970049669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1970049669},{"type":"v1","addr":"172.21.15.47:6833","nonce":1970049669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4162785911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4162785911},{"type":"v1","addr":"172.21.15.47:6835","nonce":4162785911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1036174787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1036174787},{"type":"v1","addr":"172.21.15.175:6836","nonce":1036174787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:03.137957+0000","modified":"2024-03-23T09:41:05.147528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1633245139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1633245139},{"type":"v1","addr":"172.21.15.175:6837","nonce":1633245139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T09:41:05.502 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2024-03-23T09:41:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.504+0000 7fa19f248640 1 -- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa18406ac30 msgr2=0x7fa18406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.504+0000 7fa19f248640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa18406ac30 0x7fa18406d0f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1980b7a20 tx=0x7fa188008040 comp rx=0 tx=0).stop 2024-03-23T09:41:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.504+0000 7fa19f248640 1 -- 172.21.15.47:0/334995665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1980b8350 msgr2=0x7fa1981532d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.504+0000 7fa19f248640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1980b8350 0x7fa1981532d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c7f0 tx=0x7fa19400cbf0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.504+0000 7fa19f248640 1 -- 172.21.15.47:0/334995665 shutdown_connections 2024-03-23T09:41:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.504+0000 7fa19f248640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa18406ac30 0x7fa18406d0f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.504+0000 7fa19f248640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19815ca90 0x7fa198153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.504+0000 7fa19f248640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1980cdc40 0x7fa198151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.504+0000 7fa19f248640 1 --2- 172.21.15.47:0/334995665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1980b8350 0x7fa1981532d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.504+0000 7fa19f248640 1 -- 172.21.15.47:0/334995665 >> 172.21.15.47:0/334995665 conn(0x7fa1980659d0 msgr2=0x7fa19815d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.504+0000 7fa19f248640 1 -- 172.21.15.47:0/334995665 shutdown_connections 2024-03-23T09:41:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.504+0000 7fa19f248640 1 -- 172.21.15.47:0/334995665 wait complete. 2024-03-23T09:41:05.514 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:05.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.615+0000 7f6fcd1cb640 1 Processor -- start 2024-03-23T09:41:05.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.615+0000 7f6fcd1cb640 1 -- start start 2024-03-23T09:41:05.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.616+0000 7f6fcd1cb640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6fc815ca90 0x7f6fc8160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.616+0000 7f6fcd1cb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc80b8350 0x7f6fc80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.616+0000 7f6fcd1cb640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc80d11e0 con 0x7f6fc80cdc40 2024-03-23T09:41:05.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.616+0000 7f6fcd1cb640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc80ce0d0 con 0x7f6fc815ca90 2024-03-23T09:41:05.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.616+0000 7f6fcd1cb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80d2680 con 0x7f6fc80b8350 2024-03-23T09:41:05.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.616+0000 7f6fc759e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6fc80cdc40 0x7f6fc80b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57196/0 (socket says 172.21.15.47:57196) 2024-03-23T09:41:05.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.616+0000 7f6fc759e640 1 -- 172.21.15.47:0/2598899748 learned_addr learned my addr 172.21.15.47:0/2598899748 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:05.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.616+0000 7f6fc6d9d640 1 --2- 172.21.15.47:0/2598899748 >> v2:172.21.15.175:3300/0 conn(0x7f6fc815ca90 0x7f6fc8160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.616+0000 7f6fc659c640 1 --2- 172.21.15.47:0/2598899748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc80b8350 0x7f6fc80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.616+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031156803 0 0) 0x7f6fc80d11e0 con 0x7f6fc80cdc40 2024-03-23T09:41:05.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.616+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb4003520 con 0x7f6fc80cdc40 2024-03-23T09:41:05.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2228342608 0 0) 0x7f6fb4003520 con 0x7f6fc80cdc40 2024-03-23T09:41:05.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc80b8350 msgr2=0x7f6fc80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:05.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 --2- 172.21.15.47:0/2598899748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc80b8350 0x7f6fc80cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 >> v2:172.21.15.175:3300/0 conn(0x7f6fc815ca90 msgr2=0x7f6fc8160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:05.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 --2- 172.21.15.47:0/2598899748 >> v2:172.21.15.175:3300/0 conn(0x7f6fc815ca90 0x7f6fc8160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc8160f60 con 0x7f6fc80cdc40 2024-03-23T09:41:05.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f6fb80019f0 con 0x7f6fc80cdc40 2024-03-23T09:41:05.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 >> v1:172.21.15.47:6789/0 conn(0x7f6fc80cdc40 legacy=0x7f6fc80b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 --2- 172.21.15.47:0/2598899748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fb4003970 0x7f6fb4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 --2- 172.21.15.47:0/2598899748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fb40047c0 0x7f6fb4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 --2- 172.21.15.47:0/2598899748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc80b8350 0x7f6fb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80ce0d0 con 0x7f6fb4003970 2024-03-23T09:41:05.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80d2680 con 0x7f6fb40047c0 2024-03-23T09:41:05.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb400b980 con 0x7f6fc80b8350 2024-03-23T09:41:05.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc6d9d640 1 --2- 172.21.15.47:0/2598899748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fb4003970 0x7f6fb4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc659c640 1 --2- 172.21.15.47:0/2598899748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fb40047c0 0x7f6fb4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc759e640 1 --2- 172.21.15.47:0/2598899748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc80b8350 0x7f6fb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc6d9d640 1 -- 172.21.15.47:0/2598899748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc80b8350 msgr2=0x7f6fb4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:05.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc6d9d640 1 --2- 172.21.15.47:0/2598899748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc80b8350 0x7f6fb4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc6d9d640 1 -- 172.21.15.47:0/2598899748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fb40047c0 msgr2=0x7f6fb4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc6d9d640 1 --2- 172.21.15.47:0/2598899748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fb40047c0 0x7f6fb4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.617+0000 7f6fc6d9d640 1 -- 172.21.15.47:0/2598899748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb4003520 con 0x7f6fb4003970 2024-03-23T09:41:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.618+0000 7f6fc6d9d640 1 --2- 172.21.15.47:0/2598899748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fb4003970 0x7f6fb4003d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009af0 tx=0x7f6fb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b25247bba6f7b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.618+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc002c710 con 0x7f6fb4003970 2024-03-23T09:41:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.618+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc002c870 con 0x7f6fb4003970 2024-03-23T09:41:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.618+0000 7f6fc5d9b640 1 -- 172.21.15.47:0/2598899748 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc002cc00 con 0x7f6fb4003970 2024-03-23T09:41:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.618+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/2598899748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fb4003970 msgr2=0x7f6fb4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.618+0000 7f6fcd1cb640 1 --2- 172.21.15.47:0/2598899748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fb4003970 0x7f6fb4003d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009af0 tx=0x7f6fb40057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.618+0000 7f6fc759e640 1 -- 172.21.15.47:0/2598899748 reap_dead start 2024-03-23T09:41:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.618+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/2598899748 shutdown_connections 2024-03-23T09:41:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.618+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/2598899748 >> 172.21.15.47:0/2598899748 conn(0x7f6fc80659d0 msgr2=0x7f6fc8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.618+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/2598899748 shutdown_connections 2024-03-23T09:41:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.619+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/2598899748 wait complete. 2024-03-23T09:41:05.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.620+0000 7f6fcd1cb640 1 Processor -- start 2024-03-23T09:41:05.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.620+0000 7f6fcd1cb640 1 -- start start 2024-03-23T09:41:05.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fcd1cb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fb4003970 0x7f6fc815ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fcd1cb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc8150bd0 0x7f6fc80cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fcd1cb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fc8151030 0x7f6fc80cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fcd1cb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc81f6760 con 0x7f6fc8151030 2024-03-23T09:41:05.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fcd1cb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc8159dc0 con 0x7f6fc8150bd0 2024-03-23T09:41:05.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fcd1cb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc815a0a0 con 0x7f6fb4003970 2024-03-23T09:41:05.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fc759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fc8151030 0x7f6fc80cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fc759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fc8151030 0x7f6fc80cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45804/0 (socket says 172.21.15.47:45804) 2024-03-23T09:41:05.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fc759e640 1 -- 172.21.15.47:0/4142324203 learned_addr learned my addr 172.21.15.47:0/4142324203 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:05.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fc659c640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc8150bd0 0x7f6fc80cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fc6d9d640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fb4003970 0x7f6fc815ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fc759e640 1 -- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fb4003970 msgr2=0x7f6fc815ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fc759e640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fb4003970 0x7f6fc815ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fc759e640 1 -- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc8150bd0 msgr2=0x7f6fc80cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fc759e640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc8150bd0 0x7f6fc80cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fc759e640 1 -- 172.21.15.47:0/4142324203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc815a380 con 0x7f6fc8151030 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fc759e640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fc8151030 0x7f6fc80cc3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80b5510 tx=0x7f6fb8007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fbf7fe640 1 -- 172.21.15.47:0/4142324203 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb800ad40 con 0x7f6fc8151030 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fbf7fe640 1 -- 172.21.15.47:0/4142324203 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb80133f0 con 0x7f6fc8151030 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fbf7fe640 1 -- 172.21.15.47:0/4142324203 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb80137a0 con 0x7f6fc8151030 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.621+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/4142324203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc815a660 con 0x7f6fc8151030 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.622+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/4142324203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc81f5bd0 con 0x7f6fc8151030 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.622+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/4142324203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc8072360 con 0x7f6fc8151030 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.623+0000 7f6fbf7fe640 1 -- 172.21.15.47:0/4142324203 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6fb8013900 con 0x7f6fc8151030 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.623+0000 7f6fbf7fe640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6fa806ac00 0x7f6fa806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.623+0000 7f6fbf7fe640 1 -- 172.21.15.47:0/4142324203 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fb8089b20 con 0x7f6fc8151030 2024-03-23T09:41:05.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.624+0000 7f6fc6d9d640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6fa806ac00 0x7f6fa806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.624+0000 7f6fc6d9d640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6fa806ac00 0x7f6fa806d0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0036000 tx=0x7f6fc0030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:05.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.628+0000 7f6fbf7fe640 1 -- 172.21.15.47:0/4142324203 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6fb8054d00 con 0x7f6fc8151030 2024-03-23T09:41:05.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.808+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/4142324203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fc815a850 con 0x7f6fc8151030 2024-03-23T09:41:05.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.809+0000 7f6fbf7fe640 1 -- 172.21.15.47:0/4142324203 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f6fb8058320 con 0x7f6fc8151030 2024-03-23T09:41:05.809 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:05.809 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:05.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.812+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6fa806ac00 msgr2=0x7f6fa806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.812+0000 7f6fcd1cb640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6fa806ac00 0x7f6fa806d0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0036000 tx=0x7f6fc0030000 comp rx=0 tx=0).stop 2024-03-23T09:41:05.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.812+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/4142324203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fc8151030 msgr2=0x7f6fc80cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.812+0000 7f6fcd1cb640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fc8151030 0x7f6fc80cc3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80b5510 tx=0x7f6fb8007bd0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.812+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/4142324203 shutdown_connections 2024-03-23T09:41:05.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.812+0000 7f6fcd1cb640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6fa806ac00 0x7f6fa806d0c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.812+0000 7f6fcd1cb640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fc8151030 0x7f6fc80cc3c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.812+0000 7f6fcd1cb640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc8150bd0 0x7f6fc80cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.812+0000 7f6fcd1cb640 1 --2- 172.21.15.47:0/4142324203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fb4003970 0x7f6fc815ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.812+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/4142324203 >> 172.21.15.47:0/4142324203 conn(0x7f6fc80659d0 msgr2=0x7f6fc80d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:05.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.812+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/4142324203 shutdown_connections 2024-03-23T09:41:05.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.812+0000 7f6fcd1cb640 1 -- 172.21.15.47:0/4142324203 wait complete. 2024-03-23T09:41:05.822 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T09:41:05.822 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:41:05.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.923+0000 7feb75aab640 1 Processor -- start 2024-03-23T09:41:05.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.923+0000 7feb75aab640 1 -- start start 2024-03-23T09:41:05.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.924+0000 7feb75aab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7015a7d0 0x7feb700cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.924+0000 7feb75aab640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb700cb9d0 con 0x7feb7015abb0 2024-03-23T09:41:05.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.924+0000 7feb75aab640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb700cbcb0 con 0x7feb70098a00 2024-03-23T09:41:05.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.924+0000 7feb75aab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb700cbf90 con 0x7feb7015a7d0 2024-03-23T09:41:05.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.924+0000 7feb6effd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feb7015abb0 0x7feb700cf1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57204/0 (socket says 172.21.15.47:57204) 2024-03-23T09:41:05.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.924+0000 7feb6effd640 1 -- 172.21.15.47:0/1106627080 learned_addr learned my addr 172.21.15.47:0/1106627080 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:05.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6f7fe640 1 --2- 172.21.15.47:0/1106627080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7015a7d0 0x7feb700cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389367070 0 0) 0x7feb700cb9d0 con 0x7feb7015abb0 2024-03-23T09:41:05.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb58003520 con 0x7feb7015abb0 2024-03-23T09:41:05.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276307555 0 0) 0x7feb700cbcb0 con 0x7feb70098a00 2024-03-23T09:41:05.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb700cb9d0 con 0x7feb70098a00 2024-03-23T09:41:05.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 499980944 0 0) 0x7feb58003520 con 0x7feb7015abb0 2024-03-23T09:41:05.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7015a7d0 msgr2=0x7feb700cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:05.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 --2- 172.21.15.47:0/1106627080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7015a7d0 0x7feb700cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 >> v1:172.21.15.175:6789/0 conn(0x7feb70098a00 legacy=0x7feb700b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb700cc270 con 0x7feb7015abb0 2024-03-23T09:41:05.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6f7fe640 1 --2- 172.21.15.47:0/1106627080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7015a7d0 0x7feb700cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:05.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7feb64001670 con 0x7feb7015abb0 2024-03-23T09:41:05.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 >> v1:172.21.15.47:6789/0 conn(0x7feb7015abb0 legacy=0x7feb700cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 --2- 172.21.15.47:0/1106627080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb58003b30 0x7feb58003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 --2- 172.21.15.47:0/1106627080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7015a7d0 0x7feb580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 --2- 172.21.15.47:0/1106627080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb58004d00 0x7feb58005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb700cbf90 con 0x7feb58003b30 2024-03-23T09:41:05.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5800ba90 con 0x7feb58004d00 2024-03-23T09:41:05.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5800bc50 con 0x7feb7015a7d0 2024-03-23T09:41:05.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6f7fe640 1 --2- 172.21.15.47:0/1106627080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb58003b30 0x7feb58003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6effd640 1 --2- 172.21.15.47:0/1106627080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7015a7d0 0x7feb580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6ffff640 1 --2- 172.21.15.47:0/1106627080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb58004d00 0x7feb58005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6f7fe640 1 -- 172.21.15.47:0/1106627080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7015a7d0 msgr2=0x7feb580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:05.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6f7fe640 1 --2- 172.21.15.47:0/1106627080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7015a7d0 0x7feb580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6f7fe640 1 -- 172.21.15.47:0/1106627080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb58004d00 msgr2=0x7feb58005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:05.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6f7fe640 1 --2- 172.21.15.47:0/1106627080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb58004d00 0x7feb58005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.925+0000 7feb6f7fe640 1 -- 172.21.15.47:0/1106627080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb60009400 con 0x7feb58003b30 2024-03-23T09:41:05.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.926+0000 7feb6f7fe640 1 --2- 172.21.15.47:0/1106627080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb58003b30 0x7feb58003f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feb60009c90 tx=0x7feb580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c47b4bade385ae7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:05.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.926+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6002c8f0 con 0x7feb58003b30 2024-03-23T09:41:05.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.926+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6002ca50 con 0x7feb58003b30 2024-03-23T09:41:05.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.926+0000 7feb6e7fc640 1 -- 172.21.15.47:0/1106627080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6002ce00 con 0x7feb58003b30 2024-03-23T09:41:05.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.927+0000 7feb75aab640 1 -- 172.21.15.47:0/1106627080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb58003b30 msgr2=0x7feb58003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.927+0000 7feb75aab640 1 --2- 172.21.15.47:0/1106627080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb58003b30 0x7feb58003f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feb60009c90 tx=0x7feb580057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.927+0000 7feb6ffff640 1 -- 172.21.15.47:0/1106627080 reap_dead start 2024-03-23T09:41:05.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.927+0000 7feb75aab640 1 -- 172.21.15.47:0/1106627080 shutdown_connections 2024-03-23T09:41:05.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.927+0000 7feb75aab640 1 -- 172.21.15.47:0/1106627080 >> 172.21.15.47:0/1106627080 conn(0x7feb700b21d0 msgr2=0x7feb700b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:05.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.927+0000 7feb75aab640 1 -- 172.21.15.47:0/1106627080 shutdown_connections 2024-03-23T09:41:05.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.927+0000 7feb75aab640 1 -- 172.21.15.47:0/1106627080 wait complete. 2024-03-23T09:41:05.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.928+0000 7feb75aab640 1 Processor -- start 2024-03-23T09:41:05.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.928+0000 7feb75aab640 1 -- start start 2024-03-23T09:41:05.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb75aab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb58003b30 0x7feb7015b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb75aab640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7014f9f0 0x7feb701f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb75aab640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb701f6090 0x7feb700cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb75aab640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7015b570 con 0x7feb701f6090 2024-03-23T09:41:05.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb75aab640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb700b47d0 con 0x7feb7014f9f0 2024-03-23T09:41:05.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb75aab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb701f6530 con 0x7feb58003b30 2024-03-23T09:41:05.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb6ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb701f6090 0x7feb700cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb6ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb701f6090 0x7feb700cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45824/0 (socket says 172.21.15.47:45824) 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb6ffff640 1 -- 172.21.15.47:0/2193633645 learned_addr learned my addr 172.21.15.47:0/2193633645 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb6f7fe640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb58003b30 0x7feb7015b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb6effd640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7014f9f0 0x7feb701f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb6ffff640 1 -- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb58003b30 msgr2=0x7feb7015b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb6ffff640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb58003b30 0x7feb7015b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb6ffff640 1 -- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7014f9f0 msgr2=0x7feb701f5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb6ffff640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7014f9f0 0x7feb701f5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb6ffff640 1 -- 172.21.15.47:0/2193633645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb701f6810 con 0x7feb701f6090 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb6effd640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7014f9f0 0x7feb701f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb6ffff640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb701f6090 0x7feb700cd130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb70071cf0 tx=0x7feb6800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.929+0000 7feb57fff640 1 -- 172.21.15.47:0/2193633645 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68017070 con 0x7feb701f6090 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.930+0000 7feb57fff640 1 -- 172.21.15.47:0/2193633645 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6800ee50 con 0x7feb701f6090 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.930+0000 7feb75aab640 1 -- 172.21.15.47:0/2193633645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7015b190 con 0x7feb701f6090 2024-03-23T09:41:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.930+0000 7feb57fff640 1 -- 172.21.15.47:0/2193633645 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68003b80 con 0x7feb701f6090 2024-03-23T09:41:05.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.930+0000 7feb75aab640 1 -- 172.21.15.47:0/2193633645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb700b5df0 con 0x7feb701f6090 2024-03-23T09:41:05.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.930+0000 7feb75aab640 1 -- 172.21.15.47:0/2193633645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7015b6b0 con 0x7feb701f6090 2024-03-23T09:41:05.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.931+0000 7feb57fff640 1 -- 172.21.15.47:0/2193633645 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7feb68003ce0 con 0x7feb701f6090 2024-03-23T09:41:05.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.931+0000 7feb57fff640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feb5c06ac00 0x7feb5c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:05.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.931+0000 7feb57fff640 1 -- 172.21.15.47:0/2193633645 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb68088880 con 0x7feb701f6090 2024-03-23T09:41:05.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.932+0000 7feb6f7fe640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feb5c06ac00 0x7feb5c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:05.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.932+0000 7feb6f7fe640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feb5c06ac00 0x7feb5c06d0c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feb60036000 tx=0x7feb60030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:05.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:05.936+0000 7feb57fff640 1 -- 172.21.15.47:0/2193633645 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7feb7015b6b0 con 0x7feb701f6090 2024-03-23T09:41:06.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.112+0000 7feb75aab640 1 -- 172.21.15.47:0/2193633645 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb7015b010 con 0x7feb5c06ac00 2024-03-23T09:41:06.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.120+0000 7feb57fff640 1 -- 172.21.15.47:0/2193633645 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253964 (secure 0 0 0) 0x7feb6002cbc0 con 0x7feb5c06ac00 2024-03-23T09:41:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:06.126 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:41:06.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.123+0000 7feb75aab640 1 -- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feb5c06ac00 msgr2=0x7feb5c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.123+0000 7feb75aab640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feb5c06ac00 0x7feb5c06d0c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feb60036000 tx=0x7feb60030000 comp rx=0 tx=0).stop 2024-03-23T09:41:06.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.123+0000 7feb75aab640 1 -- 172.21.15.47:0/2193633645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb701f6090 msgr2=0x7feb700cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.123+0000 7feb75aab640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb701f6090 0x7feb700cd130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb70071cf0 tx=0x7feb6800ec00 comp rx=0 tx=0).stop 2024-03-23T09:41:06.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.123+0000 7feb75aab640 1 -- 172.21.15.47:0/2193633645 shutdown_connections 2024-03-23T09:41:06.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.123+0000 7feb75aab640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feb5c06ac00 0x7feb5c06d0c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.123+0000 7feb75aab640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb701f6090 0x7feb700cd130 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.123+0000 7feb75aab640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7014f9f0 0x7feb701f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.123+0000 7feb75aab640 1 --2- 172.21.15.47:0/2193633645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb58003b30 0x7feb7015b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.123+0000 7feb75aab640 1 -- 172.21.15.47:0/2193633645 >> 172.21.15.47:0/2193633645 conn(0x7feb700b21d0 msgr2=0x7feb7006ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:06.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.123+0000 7feb75aab640 1 -- 172.21.15.47:0/2193633645 shutdown_connections 2024-03-23T09:41:06.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.123+0000 7feb75aab640 1 -- 172.21.15.47:0/2193633645 wait complete. 2024-03-23T09:41:06.133 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-03-23T09:41:04.337441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2500,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":2560000,"data_stored":1600165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.566670"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771175+0000","last_change":"2024-03-23T09:41:01.771175+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771175+0000","last_undegraded":"2024-03-23T09:41:01.771175+0000","last_fullsized":"2024-03-23T09:41:01.771175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772653+0000","last_change":"2024-03-23T09:41:01.772653+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772653+0000","last_undegraded":"2024-03-23T09:41:01.772653+0000","last_fullsized":"2024-03-23T09:41:01.772653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771073+0000","last_change":"2024-03-23T09:41:01.771073+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771073+0000","last_undegraded":"2024-03-23T09:41:01.771073+0000","last_fullsized":"2024-03-23T09:41:01.771073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772119+0000","last_change":"2024-03-23T09:41:01.772119+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772119+0000","last_undegraded":"2024-03-23T09:41:01.772119+0000","last_fullsized":"2024-03-23T09:41:01.772119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.773013+0000","last_change":"2024-03-23T09:41:01.773013+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.773013+0000","last_undegraded":"2024-03-23T09:41:01.773013+0000","last_fullsized":"2024-03-23T09:41:01.773013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.773380+0000","last_change":"2024-03-23T09:41:01.773380+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.773380+0000","last_undegraded":"2024-03-23T09:41:01.773380+0000","last_fullsized":"2024-03-23T09:41:01.773380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772996+0000","last_change":"2024-03-23T09:41:01.772996+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772996+0000","last_undegraded":"2024-03-23T09:41:01.772996+0000","last_fullsized":"2024-03-23T09:41:01.772996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771811+0000","last_change":"2024-03-23T09:41:01.771811+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771811+0000","last_undegraded":"2024-03-23T09:41:01.771811+0000","last_fullsized":"2024-03-23T09:41:01.771811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771848+0000","last_change":"2024-03-23T09:41:01.771848+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771848+0000","last_undegraded":"2024-03-23T09:41:01.771848+0000","last_fullsized":"2024-03-23T09:41:01.771848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.773388+0000","last_change":"2024-03-23T09:41:01.773388+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.773388+0000","last_undegraded":"2024-03-23T09:41:01.773388+0000","last_fullsized":"2024-03-23T09:41:01.773388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771803+0000","last_change":"2024-03-23T09:41:01.771803+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771803+0000","last_undegraded":"2024-03-23T09:41:01.771803+0000","last_fullsized":"2024-03-23T09:41:01.771803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:09:59.090234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.767949+0000","last_change":"2024-03-23T09:41:01.767949+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.767949+0000","last_undegraded":"2024-03-23T09:41:01.767949+0000","last_fullsized":"2024-03-23T09:41:01.767949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768108+0000","last_change":"2024-03-23T09:41:01.768108+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768108+0000","last_undegraded":"2024-03-23T09:41:01.768108+0000","last_fullsized":"2024-03-23T09:41:01.768108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768080+0000","last_change":"2024-03-23T09:41:01.768080+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768080+0000","last_undegraded":"2024-03-23T09:41:01.768080+0000","last_fullsized":"2024-03-23T09:41:01.768080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772436+0000","last_change":"2024-03-23T09:41:01.772436+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772436+0000","last_undegraded":"2024-03-23T09:41:01.772436+0000","last_fullsized":"2024-03-23T09:41:01.772436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.767977+0000","last_change":"2024-03-23T09:41:01.767977+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.767977+0000","last_undegraded":"2024-03-23T09:41:01.767977+0000","last_fullsized":"2024-03-23T09:41:01.767977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-03-23T09:41:01.769360+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:41:01.769360+0000","last_peered":"2024-03-23T09:41:01.769360+0000","last_clean":"2024-03-23T09:41:01.769360+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:41:01.769360+0000","last_undegraded":"2024-03-23T09:41:01.769360+0000","last_fullsized":"2024-03-23T09:41:01.769360+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772612+0000","last_change":"2024-03-23T09:41:01.772612+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772612+0000","last_undegraded":"2024-03-23T09:41:01.772612+0000","last_fullsized":"2024-03-23T09:41:01.772612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768113+0000","last_change":"2024-03-23T09:41:01.768113+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768113+0000","last_undegraded":"2024-03-23T09:41:01.768113+0000","last_fullsized":"2024-03-23T09:41:01.768113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.767991+0000","last_change":"2024-03-23T09:41:01.767991+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.767991+0000","last_undegraded":"2024-03-23T09:41:01.767991+0000","last_fullsized":"2024-03-23T09:41:01.767991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772041+0000","last_change":"2024-03-23T09:41:01.772041+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772041+0000","last_undegraded":"2024-03-23T09:41:01.772041+0000","last_fullsized":"2024-03-23T09:41:01.772041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771158+0000","last_change":"2024-03-23T09:41:01.771158+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771158+0000","last_undegraded":"2024-03-23T09:41:01.771158+0000","last_fullsized":"2024-03-23T09:41:01.771158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768125+0000","last_change":"2024-03-23T09:41:01.768125+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768125+0000","last_undegraded":"2024-03-23T09:41:01.768125+0000","last_fullsized":"2024-03-23T09:41:01.768125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.770996+0000","last_change":"2024-03-23T09:41:01.770996+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.770996+0000","last_undegraded":"2024-03-23T09:41:01.770996+0000","last_fullsized":"2024-03-23T09:41:01.770996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772497+0000","last_change":"2024-03-23T09:41:01.772497+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772497+0000","last_undegraded":"2024-03-23T09:41:01.772497+0000","last_fullsized":"2024-03-23T09:41:01.772497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.769148+0000","last_change":"2024-03-23T09:41:01.769148+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.769148+0000","last_undegraded":"2024-03-23T09:41:01.769148+0000","last_fullsized":"2024-03-23T09:41:01.769148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.770857+0000","last_change":"2024-03-23T09:41:01.770857+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.770857+0000","last_undegraded":"2024-03-23T09:41:01.770857+0000","last_fullsized":"2024-03-23T09:41:01.770857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771065+0000","last_change":"2024-03-23T09:41:01.771065+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771065+0000","last_undegraded":"2024-03-23T09:41:01.771065+0000","last_fullsized":"2024-03-23T09:41:01.771065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768126+0000","last_change":"2024-03-23T09:41:01.768126+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768126+0000","last_undegraded":"2024-03-23T09:41:01.768126+0000","last_fullsized":"2024-03-23T09:41:01.768126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768127+0000","last_change":"2024-03-23T09:41:01.768127+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768127+0000","last_undegraded":"2024-03-23T09:41:01.768127+0000","last_fullsized":"2024-03-23T09:41:01.768127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768856+0000","last_change":"2024-03-23T09:41:01.768856+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768856+0000","last_undegraded":"2024-03-23T09:41:01.768856+0000","last_fullsized":"2024-03-23T09:41:01.768856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:11.278420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771267+0000","last_change":"2024-03-23T09:41:01.771267+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771267+0000","last_undegraded":"2024-03-23T09:41:01.771267+0000","last_fullsized":"2024-03-23T09:41:01.771267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.770977+0000","last_change":"2024-03-23T09:41:01.770977+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.770977+0000","last_undegraded":"2024-03-23T09:41:01.770977+0000","last_fullsized":"2024-03-23T09:41:01.770977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:02:19.284328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28324,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343516,"statfs":{"total":96636764160,"available":96607760384,"internally_reserved":0,"allocated":761856,"data_stored":640770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":55600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":180224,"data_stored":56219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574850,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":757760,"data_stored":640151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:41:06.138 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:41:06.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.275+0000 7f6cecb2e640 1 Processor -- start 2024-03-23T09:41:06.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.275+0000 7f6cecb2e640 1 -- start start 2024-03-23T09:41:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.275+0000 7f6cecb2e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6ce815ca90 0x7f6ce8160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.276+0000 7f6cecb2e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ce80cde10 0x7f6ce80d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.276+0000 7f6cecb2e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce80d11e0 con 0x7f6ce80b8350 2024-03-23T09:41:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.276+0000 7f6cecb2e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce8160f60 con 0x7f6ce80cde10 2024-03-23T09:41:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.276+0000 7f6cecb2e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce8161240 con 0x7f6ce815ca90 2024-03-23T09:41:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.276+0000 7f6ce6d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ce80cde10 0x7f6ce80d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.276+0000 7f6ce6d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ce80cde10 0x7f6ce80d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45836/0 (socket says 172.21.15.47:45836) 2024-03-23T09:41:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.276+0000 7f6ce6d9d640 1 -- 172.21.15.47:0/3769968140 learned_addr learned my addr 172.21.15.47:0/3769968140 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.276+0000 7f6ce6d9d640 1 -- 172.21.15.47:0/3769968140 >> v2:172.21.15.175:3300/0 conn(0x7f6ce815ca90 msgr2=0x7f6ce8160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.276+0000 7f6ce6d9d640 1 --2- 172.21.15.47:0/3769968140 >> v2:172.21.15.175:3300/0 conn(0x7f6ce815ca90 0x7f6ce8160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.276+0000 7f6ce6d9d640 1 -- 172.21.15.47:0/3769968140 >> v1:172.21.15.175:6789/0 conn(0x7f6ce80b8350 legacy=0x7f6ce80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.276+0000 7f6ce6d9d640 1 -- 172.21.15.47:0/3769968140 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce8161520 con 0x7f6ce80cde10 2024-03-23T09:41:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.277+0000 7f6ce6d9d640 1 --2- 172.21.15.47:0/3769968140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce80cde10 0x7f6ce80d1670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8066680 tx=0x7f6ce002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171f8033cdb60467 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.277+0000 7f6ce5d9b640 1 -- 172.21.15.47:0/3769968140 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce002c9f0 con 0x7f6ce80cde10 2024-03-23T09:41:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.277+0000 7f6ce5d9b640 1 -- 172.21.15.47:0/3769968140 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce002cb50 con 0x7f6ce80cde10 2024-03-23T09:41:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.277+0000 7f6ce5d9b640 1 -- 172.21.15.47:0/3769968140 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce00343f0 con 0x7f6ce80cde10 2024-03-23T09:41:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.277+0000 7f6cecb2e640 1 -- 172.21.15.47:0/3769968140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce80cde10 msgr2=0x7f6ce80d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.277+0000 7f6cecb2e640 1 --2- 172.21.15.47:0/3769968140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce80cde10 0x7f6ce80d1670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8066680 tx=0x7f6ce002be50 comp rx=0 tx=0).stop 2024-03-23T09:41:06.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.277+0000 7f6cecb2e640 1 -- 172.21.15.47:0/3769968140 shutdown_connections 2024-03-23T09:41:06.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.277+0000 7f6cecb2e640 1 --2- 172.21.15.47:0/3769968140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce80cde10 0x7f6ce80d1670 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.277+0000 7f6cecb2e640 1 --2- 172.21.15.47:0/3769968140 >> v2:172.21.15.175:3300/0 conn(0x7f6ce815ca90 0x7f6ce8160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.277+0000 7f6cecb2e640 1 -- 172.21.15.47:0/3769968140 >> 172.21.15.47:0/3769968140 conn(0x7f6ce80659d0 msgr2=0x7f6ce8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:06.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.277+0000 7f6cecb2e640 1 -- 172.21.15.47:0/3769968140 shutdown_connections 2024-03-23T09:41:06.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.277+0000 7f6cecb2e640 1 -- 172.21.15.47:0/3769968140 wait complete. 2024-03-23T09:41:06.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.279+0000 7f6cecb2e640 1 Processor -- start 2024-03-23T09:41:06.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.279+0000 7f6cecb2e640 1 -- start start 2024-03-23T09:41:06.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.279+0000 7f6cecb2e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80b8350 0x7f6ce80b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.279+0000 7f6cecb2e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce80cde10 0x7f6ce81f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.279+0000 7f6cecb2e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce815ca90 0x7f6ce80b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.279+0000 7f6cecb2e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce8154560 con 0x7f6ce815ca90 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.279+0000 7f6cecb2e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce8154280 con 0x7f6ce80b8350 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.279+0000 7f6cecb2e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce80b2550 con 0x7f6ce80cde10 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.279+0000 7f6ce6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce815ca90 0x7f6ce80b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.279+0000 7f6ce6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce815ca90 0x7f6ce80b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45848/0 (socket says 172.21.15.47:45848) 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.279+0000 7f6ce6d9d640 1 -- 172.21.15.47:0/2291985351 learned_addr learned my addr 172.21.15.47:0/2291985351 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6ce659c640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80b8350 0x7f6ce80b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6cdffff640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce80cde10 0x7f6ce81f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6ce6d9d640 1 -- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce80cde10 msgr2=0x7f6ce81f1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6ce6d9d640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce80cde10 0x7f6ce81f1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6ce6d9d640 1 -- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80b8350 msgr2=0x7f6ce80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6ce6d9d640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80b8350 0x7f6ce80b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6ce6d9d640 1 -- 172.21.15.47:0/2291985351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce0009b30 con 0x7f6ce815ca90 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6cdffff640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce80cde10 0x7f6ce81f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6ce6d9d640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce815ca90 0x7f6ce80b3030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0036040 tx=0x7f6ce0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6cdeffd640 1 -- 172.21.15.47:0/2291985351 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce003e450 con 0x7f6ce815ca90 2024-03-23T09:41:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6cdeffd640 1 -- 172.21.15.47:0/2291985351 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce003e5d0 con 0x7f6ce815ca90 2024-03-23T09:41:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6cecb2e640 1 -- 172.21.15.47:0/2291985351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce80b2830 con 0x7f6ce815ca90 2024-03-23T09:41:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6cdeffd640 1 -- 172.21.15.47:0/2291985351 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce003e980 con 0x7f6ce815ca90 2024-03-23T09:41:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.280+0000 7f6cecb2e640 1 -- 172.21.15.47:0/2291985351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce81f22b0 con 0x7f6ce815ca90 2024-03-23T09:41:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.281+0000 7f6cecb2e640 1 -- 172.21.15.47:0/2291985351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce8072360 con 0x7f6ce815ca90 2024-03-23T09:41:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.282+0000 7f6cdeffd640 1 -- 172.21.15.47:0/2291985351 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6ce003eae0 con 0x7f6ce815ca90 2024-03-23T09:41:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.282+0000 7f6cdeffd640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6cc806ac00 0x7f6cc806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.282+0000 7f6cdeffd640 1 -- 172.21.15.47:0/2291985351 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ce00ac190 con 0x7f6ce815ca90 2024-03-23T09:41:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.282+0000 7f6ce659c640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6cc806ac00 0x7f6cc806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.283+0000 7f6ce659c640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6cc806ac00 0x7f6cc806d0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009890 tx=0x7f6cd8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:06.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.286+0000 7f6cdeffd640 1 -- 172.21.15.47:0/2291985351 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6ce0077400 con 0x7f6ce815ca90 2024-03-23T09:41:06.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.462+0000 7f6cecb2e640 1 -- 172.21.15.47:0/2291985351 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ce8160c50 con 0x7f6cc806ac00 2024-03-23T09:41:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.469+0000 7f6cdeffd640 1 -- 172.21.15.47:0/2291985351 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253964 (secure 0 0 0) 0x7f6cd8009480 con 0x7f6cc806ac00 2024-03-23T09:41:06.471 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:06.473 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:41:06.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.472+0000 7f6cecb2e640 1 -- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6cc806ac00 msgr2=0x7f6cc806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.472+0000 7f6cecb2e640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6cc806ac00 0x7f6cc806d0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009890 tx=0x7f6cd8008040 comp rx=0 tx=0).stop 2024-03-23T09:41:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.472+0000 7f6cecb2e640 1 -- 172.21.15.47:0/2291985351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce815ca90 msgr2=0x7f6ce80b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.472+0000 7f6cecb2e640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce815ca90 0x7f6ce80b3030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0036040 tx=0x7f6ce0004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.472+0000 7f6cecb2e640 1 -- 172.21.15.47:0/2291985351 shutdown_connections 2024-03-23T09:41:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.472+0000 7f6cecb2e640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6cc806ac00 0x7f6cc806d0c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.472+0000 7f6cecb2e640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce815ca90 0x7f6ce80b3030 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.472+0000 7f6cecb2e640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce80cde10 0x7f6ce81f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.472+0000 7f6cecb2e640 1 --2- 172.21.15.47:0/2291985351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80b8350 0x7f6ce80b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.472+0000 7f6cecb2e640 1 -- 172.21.15.47:0/2291985351 >> 172.21.15.47:0/2291985351 conn(0x7f6ce80659d0 msgr2=0x7f6ce80b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.472+0000 7f6cecb2e640 1 -- 172.21.15.47:0/2291985351 shutdown_connections 2024-03-23T09:41:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.472+0000 7f6cecb2e640 1 -- 172.21.15.47:0/2291985351 wait complete. 2024-03-23T09:41:06.482 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-03-23T09:41:06.337929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2500,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":2560000,"data_stored":1600165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.567159"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771175+0000","last_change":"2024-03-23T09:41:01.771175+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771175+0000","last_undegraded":"2024-03-23T09:41:01.771175+0000","last_fullsized":"2024-03-23T09:41:01.771175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772653+0000","last_change":"2024-03-23T09:41:01.772653+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772653+0000","last_undegraded":"2024-03-23T09:41:01.772653+0000","last_fullsized":"2024-03-23T09:41:01.772653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771073+0000","last_change":"2024-03-23T09:41:01.771073+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771073+0000","last_undegraded":"2024-03-23T09:41:01.771073+0000","last_fullsized":"2024-03-23T09:41:01.771073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772119+0000","last_change":"2024-03-23T09:41:01.772119+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772119+0000","last_undegraded":"2024-03-23T09:41:01.772119+0000","last_fullsized":"2024-03-23T09:41:01.772119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.773013+0000","last_change":"2024-03-23T09:41:01.773013+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.773013+0000","last_undegraded":"2024-03-23T09:41:01.773013+0000","last_fullsized":"2024-03-23T09:41:01.773013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.773380+0000","last_change":"2024-03-23T09:41:01.773380+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.773380+0000","last_undegraded":"2024-03-23T09:41:01.773380+0000","last_fullsized":"2024-03-23T09:41:01.773380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772996+0000","last_change":"2024-03-23T09:41:01.772996+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772996+0000","last_undegraded":"2024-03-23T09:41:01.772996+0000","last_fullsized":"2024-03-23T09:41:01.772996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771811+0000","last_change":"2024-03-23T09:41:01.771811+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771811+0000","last_undegraded":"2024-03-23T09:41:01.771811+0000","last_fullsized":"2024-03-23T09:41:01.771811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771848+0000","last_change":"2024-03-23T09:41:01.771848+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771848+0000","last_undegraded":"2024-03-23T09:41:01.771848+0000","last_fullsized":"2024-03-23T09:41:01.771848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.773388+0000","last_change":"2024-03-23T09:41:01.773388+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.773388+0000","last_undegraded":"2024-03-23T09:41:01.773388+0000","last_fullsized":"2024-03-23T09:41:01.773388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771803+0000","last_change":"2024-03-23T09:41:01.771803+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771803+0000","last_undegraded":"2024-03-23T09:41:01.771803+0000","last_fullsized":"2024-03-23T09:41:01.771803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:09:59.090234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.767949+0000","last_change":"2024-03-23T09:41:01.767949+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.767949+0000","last_undegraded":"2024-03-23T09:41:01.767949+0000","last_fullsized":"2024-03-23T09:41:01.767949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768108+0000","last_change":"2024-03-23T09:41:01.768108+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768108+0000","last_undegraded":"2024-03-23T09:41:01.768108+0000","last_fullsized":"2024-03-23T09:41:01.768108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768080+0000","last_change":"2024-03-23T09:41:01.768080+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768080+0000","last_undegraded":"2024-03-23T09:41:01.768080+0000","last_fullsized":"2024-03-23T09:41:01.768080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772436+0000","last_change":"2024-03-23T09:41:01.772436+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772436+0000","last_undegraded":"2024-03-23T09:41:01.772436+0000","last_fullsized":"2024-03-23T09:41:01.772436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.767977+0000","last_change":"2024-03-23T09:41:01.767977+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.767977+0000","last_undegraded":"2024-03-23T09:41:01.767977+0000","last_fullsized":"2024-03-23T09:41:01.767977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-03-23T09:41:01.769360+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:41:01.769360+0000","last_peered":"2024-03-23T09:41:01.769360+0000","last_clean":"2024-03-23T09:41:01.769360+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:41:01.769360+0000","last_undegraded":"2024-03-23T09:41:01.769360+0000","last_fullsized":"2024-03-23T09:41:01.769360+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772612+0000","last_change":"2024-03-23T09:41:01.772612+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772612+0000","last_undegraded":"2024-03-23T09:41:01.772612+0000","last_fullsized":"2024-03-23T09:41:01.772612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768113+0000","last_change":"2024-03-23T09:41:01.768113+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768113+0000","last_undegraded":"2024-03-23T09:41:01.768113+0000","last_fullsized":"2024-03-23T09:41:01.768113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.767991+0000","last_change":"2024-03-23T09:41:01.767991+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.767991+0000","last_undegraded":"2024-03-23T09:41:01.767991+0000","last_fullsized":"2024-03-23T09:41:01.767991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772041+0000","last_change":"2024-03-23T09:41:01.772041+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772041+0000","last_undegraded":"2024-03-23T09:41:01.772041+0000","last_fullsized":"2024-03-23T09:41:01.772041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771158+0000","last_change":"2024-03-23T09:41:01.771158+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771158+0000","last_undegraded":"2024-03-23T09:41:01.771158+0000","last_fullsized":"2024-03-23T09:41:01.771158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768125+0000","last_change":"2024-03-23T09:41:01.768125+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768125+0000","last_undegraded":"2024-03-23T09:41:01.768125+0000","last_fullsized":"2024-03-23T09:41:01.768125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.770996+0000","last_change":"2024-03-23T09:41:01.770996+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.770996+0000","last_undegraded":"2024-03-23T09:41:01.770996+0000","last_fullsized":"2024-03-23T09:41:01.770996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772497+0000","last_change":"2024-03-23T09:41:01.772497+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772497+0000","last_undegraded":"2024-03-23T09:41:01.772497+0000","last_fullsized":"2024-03-23T09:41:01.772497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.769148+0000","last_change":"2024-03-23T09:41:01.769148+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.769148+0000","last_undegraded":"2024-03-23T09:41:01.769148+0000","last_fullsized":"2024-03-23T09:41:01.769148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.770857+0000","last_change":"2024-03-23T09:41:01.770857+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.770857+0000","last_undegraded":"2024-03-23T09:41:01.770857+0000","last_fullsized":"2024-03-23T09:41:01.770857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771065+0000","last_change":"2024-03-23T09:41:01.771065+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771065+0000","last_undegraded":"2024-03-23T09:41:01.771065+0000","last_fullsized":"2024-03-23T09:41:01.771065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768126+0000","last_change":"2024-03-23T09:41:01.768126+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768126+0000","last_undegraded":"2024-03-23T09:41:01.768126+0000","last_fullsized":"2024-03-23T09:41:01.768126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768127+0000","last_change":"2024-03-23T09:41:01.768127+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768127+0000","last_undegraded":"2024-03-23T09:41:01.768127+0000","last_fullsized":"2024-03-23T09:41:01.768127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768856+0000","last_change":"2024-03-23T09:41:01.768856+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768856+0000","last_undegraded":"2024-03-23T09:41:01.768856+0000","last_fullsized":"2024-03-23T09:41:01.768856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:11.278420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771267+0000","last_change":"2024-03-23T09:41:01.771267+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771267+0000","last_undegraded":"2024-03-23T09:41:01.771267+0000","last_fullsized":"2024-03-23T09:41:01.771267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.770977+0000","last_change":"2024-03-23T09:41:01.770977+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.770977+0000","last_undegraded":"2024-03-23T09:41:01.770977+0000","last_fullsized":"2024-03-23T09:41:01.770977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:02:19.284328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28324,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343516,"statfs":{"total":96636764160,"available":96607760384,"internally_reserved":0,"allocated":761856,"data_stored":640770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":55600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":180224,"data_stored":56219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574850,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":757760,"data_stored":640151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:41:06.486 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:41:06.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.623+0000 7f0d9a1fb640 1 Processor -- start 2024-03-23T09:41:06.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.623+0000 7f0d9a1fb640 1 -- start start 2024-03-23T09:41:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.624+0000 7f0d9a1fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d9415ca90 0x7f0d94160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.624+0000 7f0d9a1fb640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d940d11e0 con 0x7f0d940b8350 2024-03-23T09:41:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.624+0000 7f0d9a1fb640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d94160f60 con 0x7f0d940cde10 2024-03-23T09:41:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.624+0000 7f0d9a1fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d94161240 con 0x7f0d9415ca90 2024-03-23T09:41:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.624+0000 7f0d92ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0d940b8350 0x7f0d940cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57212/0 (socket says 172.21.15.47:57212) 2024-03-23T09:41:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.624+0000 7f0d92ffd640 1 -- 172.21.15.47:0/1142087262 learned_addr learned my addr 172.21.15.47:0/1142087262 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.624+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858592183 0 0) 0x7f0d940d11e0 con 0x7f0d940b8350 2024-03-23T09:41:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d7c003520 con 0x7f0d940b8350 2024-03-23T09:41:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d937fe640 1 --2- 172.21.15.47:0/1142087262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d9415ca90 0x7f0d94160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4059784578 0 0) 0x7f0d7c003520 con 0x7f0d940b8350 2024-03-23T09:41:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d9415ca90 msgr2=0x7f0d94160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 --2- 172.21.15.47:0/1142087262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d9415ca90 0x7f0d94160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 >> v1:172.21.15.175:6789/0 conn(0x7f0d940cde10 legacy=0x7f0d940d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d94161520 con 0x7f0d940b8350 2024-03-23T09:41:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0d88001670 con 0x7f0d940b8350 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 >> v1:172.21.15.47:6789/0 conn(0x7f0d940b8350 legacy=0x7f0d940cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 --2- 172.21.15.47:0/1142087262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d7c0039a0 0x7f0d7c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 --2- 172.21.15.47:0/1142087262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0047c0 0x7f0d7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 --2- 172.21.15.47:0/1142087262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d9415ca90 0x7f0d7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d94161240 con 0x7f0d7c0039a0 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c00bab0 con 0x7f0d7c0047c0 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c00bc70 con 0x7f0d9415ca90 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d937fe640 1 --2- 172.21.15.47:0/1142087262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d7c0039a0 0x7f0d7c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d92ffd640 1 --2- 172.21.15.47:0/1142087262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0047c0 0x7f0d7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d93fff640 1 --2- 172.21.15.47:0/1142087262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d9415ca90 0x7f0d7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d937fe640 1 -- 172.21.15.47:0/1142087262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d9415ca90 msgr2=0x7f0d7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d937fe640 1 --2- 172.21.15.47:0/1142087262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d9415ca90 0x7f0d7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d937fe640 1 -- 172.21.15.47:0/1142087262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0047c0 msgr2=0x7f0d7c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d937fe640 1 --2- 172.21.15.47:0/1142087262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0047c0 0x7f0d7c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d937fe640 1 -- 172.21.15.47:0/1142087262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c003520 con 0x7f0d7c0039a0 2024-03-23T09:41:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.625+0000 7f0d93fff640 1 --2- 172.21.15.47:0/1142087262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d9415ca90 0x7f0d7c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.626+0000 7f0d937fe640 1 --2- 172.21.15.47:0/1142087262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d7c0039a0 0x7f0d7c003dc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009ac0 tx=0x7f0d7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8e567c6d6a793f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.626+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8402c6b0 con 0x7f0d7c0039a0 2024-03-23T09:41:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.626+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8402c810 con 0x7f0d7c0039a0 2024-03-23T09:41:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.626+0000 7f0d927fc640 1 -- 172.21.15.47:0/1142087262 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8402cba0 con 0x7f0d7c0039a0 2024-03-23T09:41:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.627+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/1142087262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d7c0039a0 msgr2=0x7f0d7c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.627+0000 7f0d9a1fb640 1 --2- 172.21.15.47:0/1142087262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d7c0039a0 0x7f0d7c003dc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009ac0 tx=0x7f0d7c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.627+0000 7f0d93fff640 1 -- 172.21.15.47:0/1142087262 reap_dead start 2024-03-23T09:41:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.627+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/1142087262 shutdown_connections 2024-03-23T09:41:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.627+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/1142087262 >> 172.21.15.47:0/1142087262 conn(0x7f0d940659d0 msgr2=0x7f0d94065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.627+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/1142087262 shutdown_connections 2024-03-23T09:41:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.627+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/1142087262 wait complete. 2024-03-23T09:41:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.628+0000 7f0d9a1fb640 1 Processor -- start 2024-03-23T09:41:06.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.628+0000 7f0d9a1fb640 1 -- start start 2024-03-23T09:41:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d9a1fb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0039a0 0x7f0d940c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d9a1fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d941f2230 0x7f0d941f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d9a1fb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d940c9a90 0x7f0d940c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d9a1fb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d940ca450 con 0x7f0d940c9a90 2024-03-23T09:41:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d9a1fb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d941f3570 con 0x7f0d7c0039a0 2024-03-23T09:41:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d9a1fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d941f2cf0 con 0x7f0d941f2230 2024-03-23T09:41:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d93fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d940c9a90 0x7f0d940c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d93fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d940c9a90 0x7f0d940c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45864/0 (socket says 172.21.15.47:45864) 2024-03-23T09:41:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d93fff640 1 -- 172.21.15.47:0/2816505051 learned_addr learned my addr 172.21.15.47:0/2816505051 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d937fe640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0039a0 0x7f0d940c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d92ffd640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d941f2230 0x7f0d941f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d93fff640 1 -- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d941f2230 msgr2=0x7f0d941f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d93fff640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d941f2230 0x7f0d941f26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d93fff640 1 -- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0039a0 msgr2=0x7f0d940c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d93fff640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0039a0 0x7f0d940c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d93fff640 1 -- 172.21.15.47:0/2816505051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d941f2fd0 con 0x7f0d940c9a90 2024-03-23T09:41:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d92ffd640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d941f2230 0x7f0d941f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d93fff640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d940c9a90 0x7f0d940c9f10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c000f80 tx=0x7f0d8c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2816505051 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c017070 con 0x7f0d940c9a90 2024-03-23T09:41:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2816505051 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8c006b50 con 0x7f0d940c9a90 2024-03-23T09:41:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/2816505051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d941f3830 con 0x7f0d940c9a90 2024-03-23T09:41:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.629+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2816505051 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c006d00 con 0x7f0d940c9a90 2024-03-23T09:41:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.630+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/2816505051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d94145510 con 0x7f0d940c9a90 2024-03-23T09:41:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.631+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/2816505051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d94072360 con 0x7f0d940c9a90 2024-03-23T09:41:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.631+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2816505051 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0d8c003970 con 0x7f0d940c9a90 2024-03-23T09:41:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.631+0000 7f0d7bfff640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d8006ac30 0x7f0d8006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.632+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2816505051 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d8c0889c0 con 0x7f0d940c9a90 2024-03-23T09:41:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.632+0000 7f0d937fe640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d8006ac30 0x7f0d8006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.632+0000 7f0d937fe640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d8006ac30 0x7f0d8006d0f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0d84036000 tx=0x7f0d84030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.636+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2816505051 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0d8c053c30 con 0x7f0d940c9a90 2024-03-23T09:41:06.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.812+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/2816505051 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d940b7fe0 con 0x7f0d8006ac30 2024-03-23T09:41:06.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.819+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2816505051 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253964 (secure 0 0 0) 0x7f0d940b7fe0 con 0x7f0d8006ac30 2024-03-23T09:41:06.821 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:41:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.822+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d8006ac30 msgr2=0x7f0d8006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.822+0000 7f0d9a1fb640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d8006ac30 0x7f0d8006d0f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0d84036000 tx=0x7f0d84030000 comp rx=0 tx=0).stop 2024-03-23T09:41:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.822+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/2816505051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d940c9a90 msgr2=0x7f0d940c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.822+0000 7f0d9a1fb640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d940c9a90 0x7f0d940c9f10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c000f80 tx=0x7f0d8c00eb90 comp rx=0 tx=0).stop 2024-03-23T09:41:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.822+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/2816505051 shutdown_connections 2024-03-23T09:41:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.822+0000 7f0d9a1fb640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d8006ac30 0x7f0d8006d0f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.822+0000 7f0d9a1fb640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d940c9a90 0x7f0d940c9f10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.822+0000 7f0d9a1fb640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d941f2230 0x7f0d941f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.822+0000 7f0d9a1fb640 1 --2- 172.21.15.47:0/2816505051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0039a0 0x7f0d940c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.822+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/2816505051 >> 172.21.15.47:0/2816505051 conn(0x7f0d940659d0 msgr2=0x7f0d9415d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.822+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/2816505051 shutdown_connections 2024-03-23T09:41:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.822+0000 7f0d9a1fb640 1 -- 172.21.15.47:0/2816505051 wait complete. 2024-03-23T09:41:06.832 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-03-23T09:41:06.337929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2500,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":2560000,"data_stored":1600165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.567159"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771175+0000","last_change":"2024-03-23T09:41:01.771175+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771175+0000","last_undegraded":"2024-03-23T09:41:01.771175+0000","last_fullsized":"2024-03-23T09:41:01.771175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772653+0000","last_change":"2024-03-23T09:41:01.772653+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772653+0000","last_undegraded":"2024-03-23T09:41:01.772653+0000","last_fullsized":"2024-03-23T09:41:01.772653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771073+0000","last_change":"2024-03-23T09:41:01.771073+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771073+0000","last_undegraded":"2024-03-23T09:41:01.771073+0000","last_fullsized":"2024-03-23T09:41:01.771073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772119+0000","last_change":"2024-03-23T09:41:01.772119+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772119+0000","last_undegraded":"2024-03-23T09:41:01.772119+0000","last_fullsized":"2024-03-23T09:41:01.772119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.773013+0000","last_change":"2024-03-23T09:41:01.773013+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.773013+0000","last_undegraded":"2024-03-23T09:41:01.773013+0000","last_fullsized":"2024-03-23T09:41:01.773013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.773380+0000","last_change":"2024-03-23T09:41:01.773380+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.773380+0000","last_undegraded":"2024-03-23T09:41:01.773380+0000","last_fullsized":"2024-03-23T09:41:01.773380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772996+0000","last_change":"2024-03-23T09:41:01.772996+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772996+0000","last_undegraded":"2024-03-23T09:41:01.772996+0000","last_fullsized":"2024-03-23T09:41:01.772996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771811+0000","last_change":"2024-03-23T09:41:01.771811+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771811+0000","last_undegraded":"2024-03-23T09:41:01.771811+0000","last_fullsized":"2024-03-23T09:41:01.771811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771848+0000","last_change":"2024-03-23T09:41:01.771848+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771848+0000","last_undegraded":"2024-03-23T09:41:01.771848+0000","last_fullsized":"2024-03-23T09:41:01.771848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.773388+0000","last_change":"2024-03-23T09:41:01.773388+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.773388+0000","last_undegraded":"2024-03-23T09:41:01.773388+0000","last_fullsized":"2024-03-23T09:41:01.773388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771803+0000","last_change":"2024-03-23T09:41:01.771803+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771803+0000","last_undegraded":"2024-03-23T09:41:01.771803+0000","last_fullsized":"2024-03-23T09:41:01.771803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:09:59.090234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.767949+0000","last_change":"2024-03-23T09:41:01.767949+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.767949+0000","last_undegraded":"2024-03-23T09:41:01.767949+0000","last_fullsized":"2024-03-23T09:41:01.767949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768108+0000","last_change":"2024-03-23T09:41:01.768108+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768108+0000","last_undegraded":"2024-03-23T09:41:01.768108+0000","last_fullsized":"2024-03-23T09:41:01.768108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768080+0000","last_change":"2024-03-23T09:41:01.768080+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768080+0000","last_undegraded":"2024-03-23T09:41:01.768080+0000","last_fullsized":"2024-03-23T09:41:01.768080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772436+0000","last_change":"2024-03-23T09:41:01.772436+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772436+0000","last_undegraded":"2024-03-23T09:41:01.772436+0000","last_fullsized":"2024-03-23T09:41:01.772436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.767977+0000","last_change":"2024-03-23T09:41:01.767977+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.767977+0000","last_undegraded":"2024-03-23T09:41:01.767977+0000","last_fullsized":"2024-03-23T09:41:01.767977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-03-23T09:41:01.769360+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:41:01.769360+0000","last_peered":"2024-03-23T09:41:01.769360+0000","last_clean":"2024-03-23T09:41:01.769360+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:41:01.769360+0000","last_undegraded":"2024-03-23T09:41:01.769360+0000","last_fullsized":"2024-03-23T09:41:01.769360+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772612+0000","last_change":"2024-03-23T09:41:01.772612+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772612+0000","last_undegraded":"2024-03-23T09:41:01.772612+0000","last_fullsized":"2024-03-23T09:41:01.772612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768113+0000","last_change":"2024-03-23T09:41:01.768113+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768113+0000","last_undegraded":"2024-03-23T09:41:01.768113+0000","last_fullsized":"2024-03-23T09:41:01.768113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.767991+0000","last_change":"2024-03-23T09:41:01.767991+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.767991+0000","last_undegraded":"2024-03-23T09:41:01.767991+0000","last_fullsized":"2024-03-23T09:41:01.767991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772041+0000","last_change":"2024-03-23T09:41:01.772041+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772041+0000","last_undegraded":"2024-03-23T09:41:01.772041+0000","last_fullsized":"2024-03-23T09:41:01.772041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771158+0000","last_change":"2024-03-23T09:41:01.771158+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771158+0000","last_undegraded":"2024-03-23T09:41:01.771158+0000","last_fullsized":"2024-03-23T09:41:01.771158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768125+0000","last_change":"2024-03-23T09:41:01.768125+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768125+0000","last_undegraded":"2024-03-23T09:41:01.768125+0000","last_fullsized":"2024-03-23T09:41:01.768125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.770996+0000","last_change":"2024-03-23T09:41:01.770996+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.770996+0000","last_undegraded":"2024-03-23T09:41:01.770996+0000","last_fullsized":"2024-03-23T09:41:01.770996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772497+0000","last_change":"2024-03-23T09:41:01.772497+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772497+0000","last_undegraded":"2024-03-23T09:41:01.772497+0000","last_fullsized":"2024-03-23T09:41:01.772497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.769148+0000","last_change":"2024-03-23T09:41:01.769148+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.769148+0000","last_undegraded":"2024-03-23T09:41:01.769148+0000","last_fullsized":"2024-03-23T09:41:01.769148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.770857+0000","last_change":"2024-03-23T09:41:01.770857+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.770857+0000","last_undegraded":"2024-03-23T09:41:01.770857+0000","last_fullsized":"2024-03-23T09:41:01.770857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771065+0000","last_change":"2024-03-23T09:41:01.771065+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771065+0000","last_undegraded":"2024-03-23T09:41:01.771065+0000","last_fullsized":"2024-03-23T09:41:01.771065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768126+0000","last_change":"2024-03-23T09:41:01.768126+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768126+0000","last_undegraded":"2024-03-23T09:41:01.768126+0000","last_fullsized":"2024-03-23T09:41:01.768126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768127+0000","last_change":"2024-03-23T09:41:01.768127+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768127+0000","last_undegraded":"2024-03-23T09:41:01.768127+0000","last_fullsized":"2024-03-23T09:41:01.768127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768856+0000","last_change":"2024-03-23T09:41:01.768856+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768856+0000","last_undegraded":"2024-03-23T09:41:01.768856+0000","last_fullsized":"2024-03-23T09:41:01.768856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:11.278420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771267+0000","last_change":"2024-03-23T09:41:01.771267+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771267+0000","last_undegraded":"2024-03-23T09:41:01.771267+0000","last_fullsized":"2024-03-23T09:41:01.771267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.770977+0000","last_change":"2024-03-23T09:41:01.770977+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.770977+0000","last_undegraded":"2024-03-23T09:41:01.770977+0000","last_fullsized":"2024-03-23T09:41:01.770977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:02:19.284328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28324,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343516,"statfs":{"total":96636764160,"available":96607760384,"internally_reserved":0,"allocated":761856,"data_stored":640770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":55600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":180224,"data_stored":56219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574850,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":757760,"data_stored":640151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:41:06.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2024-03-23T09:41:06.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-03-23T09:41:06.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-03-23T09:41:06.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-03-23T09:41:06.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2024-03-23T09:41:06.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-03-23T09:41:06.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2024-03-23T09:41:06.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-03-23T09:41:06.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2024-03-23T09:41:06.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2024-03-23T09:41:06.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2024-03-23T09:41:06.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-03-23T09:41:06.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-03-23T09:41:06.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-03-23T09:41:06.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-03-23T09:41:06.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-03-23T09:41:06.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-03-23T09:41:06.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2024-03-23T09:41:06.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2024-03-23T09:41:06.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2024-03-23T09:41:06.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-03-23T09:41:06.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-03-23T09:41:06.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-03-23T09:41:06.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-03-23T09:41:06.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-03-23T09:41:06.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2024-03-23T09:41:06.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-03-23T09:41:06.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-03-23T09:41:06.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-03-23T09:41:06.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-03-23T09:41:06.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-03-23T09:41:06.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2024-03-23T09:41:06.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-03-23T09:41:06.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2024-03-23T09:41:06.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.771175+0000', 'last_change': '2024-03-23T09:41:01.771175+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.771175+0000', 'last_undegraded': '2024-03-23T09:41:01.771175+0000', 'last_fullsized': '2024-03-23T09:41:01.771175+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:41:06.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-03-23T09:41:06.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2024-03-23T09:41:06.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.772653+0000', 'last_change': '2024-03-23T09:41:01.772653+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.772653+0000', 'last_undegraded': '2024-03-23T09:41:01.772653+0000', 'last_fullsized': '2024-03-23T09:41:01.772653+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:41:06.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2024-03-23T09:41:06.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.771073+0000', 'last_change': '2024-03-23T09:41:01.771073+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.771073+0000', 'last_undegraded': '2024-03-23T09:41:01.771073+0000', 'last_fullsized': '2024-03-23T09:41:01.771073+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:41:06.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-03-23T09:41:06.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2024-03-23T09:41:06.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.772119+0000', 'last_change': '2024-03-23T09:41:01.772119+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.772119+0000', 'last_undegraded': '2024-03-23T09:41:01.772119+0000', 'last_fullsized': '2024-03-23T09:41:01.772119+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:41:06.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-03-23T09:41:06.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2024-03-23T09:41:06.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.773013+0000', 'last_change': '2024-03-23T09:41:01.773013+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.773013+0000', 'last_undegraded': '2024-03-23T09:41:01.773013+0000', 'last_fullsized': '2024-03-23T09:41:01.773013+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:41:06.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-03-23T09:41:06.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2024-03-23T09:41:06.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.773380+0000', 'last_change': '2024-03-23T09:41:01.773380+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.773380+0000', 'last_undegraded': '2024-03-23T09:41:01.773380+0000', 'last_fullsized': '2024-03-23T09:41:01.773380+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:41:06.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-03-23T09:41:06.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2024-03-23T09:41:06.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.772996+0000', 'last_change': '2024-03-23T09:41:01.772996+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.772996+0000', 'last_undegraded': '2024-03-23T09:41:01.772996+0000', 'last_fullsized': '2024-03-23T09:41:01.772996+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:41:06.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-03-23T09:41:06.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2024-03-23T09:41:06.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.771811+0000', 'last_change': '2024-03-23T09:41:01.771811+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.771811+0000', 'last_undegraded': '2024-03-23T09:41:01.771811+0000', 'last_fullsized': '2024-03-23T09:41:01.771811+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:41:06.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-03-23T09:41:06.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2024-03-23T09:41:06.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.771848+0000', 'last_change': '2024-03-23T09:41:01.771848+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.771848+0000', 'last_undegraded': '2024-03-23T09:41:01.771848+0000', 'last_fullsized': '2024-03-23T09:41:01.771848+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:41:06.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-03-23T09:41:06.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2024-03-23T09:41:06.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.773388+0000', 'last_change': '2024-03-23T09:41:01.773388+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.773388+0000', 'last_undegraded': '2024-03-23T09:41:01.773388+0000', 'last_fullsized': '2024-03-23T09:41:01.773388+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:41:06.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-03-23T09:41:06.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2024-03-23T09:41:06.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.771803+0000', 'last_change': '2024-03-23T09:41:01.771803+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.771803+0000', 'last_undegraded': '2024-03-23T09:41:01.771803+0000', 'last_fullsized': '2024-03-23T09:41:01.771803+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:09:59.090234+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:41:06.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-03-23T09:41:06.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2024-03-23T09:41:06.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.767949+0000', 'last_change': '2024-03-23T09:41:01.767949+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.767949+0000', 'last_undegraded': '2024-03-23T09:41:01.767949+0000', 'last_fullsized': '2024-03-23T09:41:01.767949+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:41:06.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-03-23T09:41:06.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2024-03-23T09:41:06.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.768108+0000', 'last_change': '2024-03-23T09:41:01.768108+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.768108+0000', 'last_undegraded': '2024-03-23T09:41:01.768108+0000', 'last_fullsized': '2024-03-23T09:41:01.768108+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:41:06.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-03-23T09:41:06.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2024-03-23T09:41:06.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.768080+0000', 'last_change': '2024-03-23T09:41:01.768080+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.768080+0000', 'last_undegraded': '2024-03-23T09:41:01.768080+0000', 'last_fullsized': '2024-03-23T09:41:01.768080+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:41:06.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-03-23T09:41:06.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-03-23T09:41:06.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.772436+0000', 'last_change': '2024-03-23T09:41:01.772436+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.772436+0000', 'last_undegraded': '2024-03-23T09:41:01.772436+0000', 'last_fullsized': '2024-03-23T09:41:01.772436+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:41:06.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-03-23T09:41:06.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2024-03-23T09:41:06.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.767977+0000', 'last_change': '2024-03-23T09:41:01.767977+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.767977+0000', 'last_undegraded': '2024-03-23T09:41:01.767977+0000', 'last_fullsized': '2024-03-23T09:41:01.767977+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:41:06.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-03-23T09:41:06.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2024-03-23T09:41:06.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.772612+0000', 'last_change': '2024-03-23T09:41:01.772612+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.772612+0000', 'last_undegraded': '2024-03-23T09:41:01.772612+0000', 'last_fullsized': '2024-03-23T09:41:01.772612+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:41:06.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-03-23T09:41:06.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2024-03-23T09:41:06.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.768113+0000', 'last_change': '2024-03-23T09:41:01.768113+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.768113+0000', 'last_undegraded': '2024-03-23T09:41:01.768113+0000', 'last_fullsized': '2024-03-23T09:41:01.768113+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:41:06.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-03-23T09:41:06.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2024-03-23T09:41:06.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.767991+0000', 'last_change': '2024-03-23T09:41:01.767991+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.767991+0000', 'last_undegraded': '2024-03-23T09:41:01.767991+0000', 'last_fullsized': '2024-03-23T09:41:01.767991+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:41:06.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-03-23T09:41:06.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2024-03-23T09:41:06.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.772041+0000', 'last_change': '2024-03-23T09:41:01.772041+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.772041+0000', 'last_undegraded': '2024-03-23T09:41:01.772041+0000', 'last_fullsized': '2024-03-23T09:41:01.772041+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:41:06.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-03-23T09:41:06.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-03-23T09:41:06.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.771158+0000', 'last_change': '2024-03-23T09:41:01.771158+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.771158+0000', 'last_undegraded': '2024-03-23T09:41:01.771158+0000', 'last_fullsized': '2024-03-23T09:41:01.771158+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:41:06.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-03-23T09:41:06.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2024-03-23T09:41:06.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.768125+0000', 'last_change': '2024-03-23T09:41:01.768125+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.768125+0000', 'last_undegraded': '2024-03-23T09:41:01.768125+0000', 'last_fullsized': '2024-03-23T09:41:01.768125+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:41:06.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-03-23T09:41:06.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2024-03-23T09:41:06.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.770996+0000', 'last_change': '2024-03-23T09:41:01.770996+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.770996+0000', 'last_undegraded': '2024-03-23T09:41:01.770996+0000', 'last_fullsized': '2024-03-23T09:41:01.770996+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:41:06.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-03-23T09:41:06.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2024-03-23T09:41:06.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.772497+0000', 'last_change': '2024-03-23T09:41:01.772497+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.772497+0000', 'last_undegraded': '2024-03-23T09:41:01.772497+0000', 'last_fullsized': '2024-03-23T09:41:01.772497+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:41:06.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-03-23T09:41:06.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2024-03-23T09:41:06.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.769148+0000', 'last_change': '2024-03-23T09:41:01.769148+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.769148+0000', 'last_undegraded': '2024-03-23T09:41:01.769148+0000', 'last_fullsized': '2024-03-23T09:41:01.769148+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:41:06.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-03-23T09:41:06.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2024-03-23T09:41:06.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.770857+0000', 'last_change': '2024-03-23T09:41:01.770857+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.770857+0000', 'last_undegraded': '2024-03-23T09:41:01.770857+0000', 'last_fullsized': '2024-03-23T09:41:01.770857+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:41:06.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-03-23T09:41:06.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2024-03-23T09:41:06.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.771065+0000', 'last_change': '2024-03-23T09:41:01.771065+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.771065+0000', 'last_undegraded': '2024-03-23T09:41:01.771065+0000', 'last_fullsized': '2024-03-23T09:41:01.771065+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:41:06.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-03-23T09:41:06.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2024-03-23T09:41:06.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.768126+0000', 'last_change': '2024-03-23T09:41:01.768126+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.768126+0000', 'last_undegraded': '2024-03-23T09:41:01.768126+0000', 'last_fullsized': '2024-03-23T09:41:01.768126+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:41:06.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-03-23T09:41:06.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-03-23T09:41:06.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2024-03-23T09:41:06.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.768127+0000', 'last_change': '2024-03-23T09:41:01.768127+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.768127+0000', 'last_undegraded': '2024-03-23T09:41:01.768127+0000', 'last_fullsized': '2024-03-23T09:41:01.768127+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:41:06.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-03-23T09:41:06.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2024-03-23T09:41:06.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.768856+0000', 'last_change': '2024-03-23T09:41:01.768856+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.768856+0000', 'last_undegraded': '2024-03-23T09:41:01.768856+0000', 'last_fullsized': '2024-03-23T09:41:01.768856+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:56:11.278420+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:41:06.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-03-23T09:41:06.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2024-03-23T09:41:06.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.771267+0000', 'last_change': '2024-03-23T09:41:01.771267+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.771267+0000', 'last_undegraded': '2024-03-23T09:41:01.771267+0000', 'last_fullsized': '2024-03-23T09:41:01.771267+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:41:06.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-03-23T09:41:06.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:02.765916+0000', 'last_change': '2024-03-23T09:41:02.765916+0000', 'last_active': '2024-03-23T09:41:02.765916+0000', 'last_peered': '2024-03-23T09:41:02.765916+0000', 'last_clean': '2024-03-23T09:41:02.765916+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:02.765916+0000', 'last_undegraded': '2024-03-23T09:41:02.765916+0000', 'last_fullsized': '2024-03-23T09:41:02.765916+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:02.765916+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:41:06.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2024-03-23T09:41:06.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:41:01.770977+0000', 'last_change': '2024-03-23T09:41:01.770977+0000', 'last_active': '2024-03-23T09:41:01.764185+0000', 'last_peered': '2024-03-23T09:41:01.764185+0000', 'last_clean': '2024-03-23T09:41:01.764185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:01.770977+0000', 'last_undegraded': '2024-03-23T09:41:01.770977+0000', 'last_fullsized': '2024-03-23T09:41:01.770977+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:01.764185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:02:19.284328+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:41:06.854 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-23T09:41:06.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:41:06.875 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:41:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.973+0000 7f00f9c84640 1 Processor -- start 2024-03-23T09:41:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.973+0000 7f00f9c84640 1 -- start start 2024-03-23T09:41:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.973+0000 7f00f9c84640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f00f40b8360 0x7f00f415d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.973+0000 7f00f9c84640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00f40b8740 0x7f00f40cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.973+0000 7f00f9c84640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00f40cf960 0x7f00f40cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.973+0000 7f00f9c84640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00f40cfe10 con 0x7f00f40cf960 2024-03-23T09:41:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.973+0000 7f00f9c84640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00f40b45e0 con 0x7f00f40b8360 2024-03-23T09:41:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.973+0000 7f00f9c84640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f40b48c0 con 0x7f00f40b8740 2024-03-23T09:41:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.974+0000 7f00f3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00f40cf960 0x7f00f40cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.974+0000 7f00f3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00f40cf960 0x7f00f40cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45876/0 (socket says 172.21.15.47:45876) 2024-03-23T09:41:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.974+0000 7f00f3fff640 1 -- 172.21.15.47:0/2233924759 learned_addr learned my addr 172.21.15.47:0/2233924759 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:06.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.974+0000 7f00f37fe640 1 --2- 172.21.15.47:0/2233924759 >> v2:172.21.15.175:3300/0 conn(0x7f00f40b8360 0x7f00f415d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.974+0000 7f00f2ffd640 1 --2- 172.21.15.47:0/2233924759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00f40b8740 0x7f00f40cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.974+0000 7f00f3fff640 1 -- 172.21.15.47:0/2233924759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00f40b8740 msgr2=0x7f00f40cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:06.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.974+0000 7f00f3fff640 1 --2- 172.21.15.47:0/2233924759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00f40b8740 0x7f00f40cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.974+0000 7f00f3fff640 1 -- 172.21.15.47:0/2233924759 >> v2:172.21.15.175:3300/0 conn(0x7f00f40b8360 msgr2=0x7f00f415d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:06.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.974+0000 7f00f3fff640 1 --2- 172.21.15.47:0/2233924759 >> v2:172.21.15.175:3300/0 conn(0x7f00f40b8360 0x7f00f415d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.974+0000 7f00f3fff640 1 -- 172.21.15.47:0/2233924759 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f40b4ba0 con 0x7f00f40cf960 2024-03-23T09:41:06.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.975+0000 7f00f3fff640 1 --2- 172.21.15.47:0/2233924759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40cf960 0x7f00f40cd800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009e50 tx=0x7f00ec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ed3d1f9edb8124 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:06.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.975+0000 7f00f27fc640 1 -- 172.21.15.47:0/2233924759 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec02c9e0 con 0x7f00f40cf960 2024-03-23T09:41:06.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.975+0000 7f00f27fc640 1 -- 172.21.15.47:0/2233924759 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec02cb40 con 0x7f00f40cf960 2024-03-23T09:41:06.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.975+0000 7f00f27fc640 1 -- 172.21.15.47:0/2233924759 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0343f0 con 0x7f00f40cf960 2024-03-23T09:41:06.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.975+0000 7f00f9c84640 1 -- 172.21.15.47:0/2233924759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40cf960 msgr2=0x7f00f40cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.975+0000 7f00f9c84640 1 --2- 172.21.15.47:0/2233924759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40cf960 0x7f00f40cd800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009e50 tx=0x7f00ec02be40 comp rx=0 tx=0).stop 2024-03-23T09:41:06.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.975+0000 7f00f9c84640 1 -- 172.21.15.47:0/2233924759 shutdown_connections 2024-03-23T09:41:06.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.975+0000 7f00f9c84640 1 --2- 172.21.15.47:0/2233924759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40cf960 0x7f00f40cd800 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.975+0000 7f00f9c84640 1 --2- 172.21.15.47:0/2233924759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00f40b8740 0x7f00f40cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.975+0000 7f00f9c84640 1 --2- 172.21.15.47:0/2233924759 >> v2:172.21.15.175:3300/0 conn(0x7f00f40b8360 0x7f00f415d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.975+0000 7f00f9c84640 1 -- 172.21.15.47:0/2233924759 >> 172.21.15.47:0/2233924759 conn(0x7f00f40659d0 msgr2=0x7f00f4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:06.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.975+0000 7f00f9c84640 1 -- 172.21.15.47:0/2233924759 shutdown_connections 2024-03-23T09:41:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.975+0000 7f00f9c84640 1 -- 172.21.15.47:0/2233924759 wait complete. 2024-03-23T09:41:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.976+0000 7f00f9c84640 1 Processor -- start 2024-03-23T09:41:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.977+0000 7f00f9c84640 1 -- start start 2024-03-23T09:41:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.977+0000 7f00f9c84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40b8360 0x7f00f415d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.977+0000 7f00f9c84640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00f40b8740 0x7f00f40a9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.977+0000 7f00f9c84640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f40cf960 0x7f00f4156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.977+0000 7f00f9c84640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f415d320 con 0x7f00f40b8360 2024-03-23T09:41:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.977+0000 7f00f9c84640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f40a9db0 con 0x7f00f40cf960 2024-03-23T09:41:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.977+0000 7f00f9c84640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f40a9ad0 con 0x7f00f40b8740 2024-03-23T09:41:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.977+0000 7f00f37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40b8360 0x7f00f415d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.977+0000 7f00f37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40b8360 0x7f00f415d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45880/0 (socket says 172.21.15.47:45880) 2024-03-23T09:41:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.977+0000 7f00f37fe640 1 -- 172.21.15.47:0/4234834343 learned_addr learned my addr 172.21.15.47:0/4234834343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00f3fff640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f40cf960 0x7f00f4156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00f2ffd640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00f40b8740 0x7f00f40a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00f37fe640 1 -- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00f40b8740 msgr2=0x7f00f40a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00f37fe640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00f40b8740 0x7f00f40a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00f37fe640 1 -- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f40cf960 msgr2=0x7f00f4156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00f37fe640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f40cf960 0x7f00f4156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00f37fe640 1 -- 172.21.15.47:0/4234834343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec009b30 con 0x7f00f40b8360 2024-03-23T09:41:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00f3fff640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f40cf960 0x7f00f4156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00f37fe640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40b8360 0x7f00f415d7b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f00e400d800 tx=0x7f00e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00dbfff640 1 -- 172.21.15.47:0/4234834343 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e40096a0 con 0x7f00f40b8360 2024-03-23T09:41:06.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00dbfff640 1 -- 172.21.15.47:0/4234834343 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e4009800 con 0x7f00f40b8360 2024-03-23T09:41:06.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00f9c84640 1 -- 172.21.15.47:0/4234834343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f415ccd0 con 0x7f00f40b8360 2024-03-23T09:41:06.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00dbfff640 1 -- 172.21.15.47:0/4234834343 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e4009990 con 0x7f00f40b8360 2024-03-23T09:41:06.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.978+0000 7f00f9c84640 1 -- 172.21.15.47:0/4234834343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f40a97c0 con 0x7f00f40b8360 2024-03-23T09:41:06.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.979+0000 7f00f9c84640 1 -- 172.21.15.47:0/4234834343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f4072360 con 0x7f00f40b8360 2024-03-23T09:41:06.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.979+0000 7f00dbfff640 1 -- 172.21.15.47:0/4234834343 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f00e4009af0 con 0x7f00f40b8360 2024-03-23T09:41:06.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.980+0000 7f00dbfff640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f00e006ac00 0x7f00e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:06.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.980+0000 7f00dbfff640 1 -- 172.21.15.47:0/4234834343 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00e4089750 con 0x7f00f40b8360 2024-03-23T09:41:06.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.980+0000 7f00f2ffd640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f00e006ac00 0x7f00e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:06.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.981+0000 7f00f2ffd640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f00e006ac00 0x7f00e006d0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f00e8006fd0 tx=0x7f00e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:06.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:06.984+0000 7f00dbfff640 1 -- 172.21.15.47:0/4234834343 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f00e40549c0 con 0x7f00f40b8360 2024-03-23T09:41:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.230+0000 7f00f9c84640 1 -- 172.21.15.47:0/4234834343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f00f4066be0 con 0x7f00f40b8360 2024-03-23T09:41:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.230+0000 7f00dbfff640 1 -- 172.21.15.47:0/4234834343 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1106 (secure 0 0 0) 0x7f00e400b3a0 con 0x7f00f40b8360 2024-03-23T09:41:07.230 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:07.230 INFO:teuthology.orchestra.run.smithi047.stdout:{"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":21,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1711186856,"num_in_osds":8,"osd_in_since":1711186848,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":241410048,"bytes_avail":772852703232,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4285}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-03-23T09:40:46.027182+0000","services":{}},"progress_events":{}} 2024-03-23T09:41:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.232+0000 7f00f9c84640 1 -- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f00e006ac00 msgr2=0x7f00e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.232+0000 7f00f9c84640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f00e006ac00 0x7f00e006d0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f00e8006fd0 tx=0x7f00e8008040 comp rx=0 tx=0).stop 2024-03-23T09:41:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.232+0000 7f00f9c84640 1 -- 172.21.15.47:0/4234834343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40b8360 msgr2=0x7f00f415d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.232+0000 7f00f9c84640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40b8360 0x7f00f415d7b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f00e400d800 tx=0x7f00e400dcc0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.232+0000 7f00f9c84640 1 -- 172.21.15.47:0/4234834343 shutdown_connections 2024-03-23T09:41:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.232+0000 7f00f9c84640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f00e006ac00 0x7f00e006d0c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.232+0000 7f00f9c84640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f40cf960 0x7f00f4156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.232+0000 7f00f9c84640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00f40b8740 0x7f00f40a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.232+0000 7f00f9c84640 1 --2- 172.21.15.47:0/4234834343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40b8360 0x7f00f415d7b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.232+0000 7f00f9c84640 1 -- 172.21.15.47:0/4234834343 >> 172.21.15.47:0/4234834343 conn(0x7f00f40659d0 msgr2=0x7f00f40d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.232+0000 7f00f9c84640 1 -- 172.21.15.47:0/4234834343 shutdown_connections 2024-03-23T09:41:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.232+0000 7f00f9c84640 1 -- 172.21.15.47:0/4234834343 wait complete. 2024-03-23T09:41:07.242 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-23T09:41:07.242 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:41:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.343+0000 7f585ae41640 1 Processor -- start 2024-03-23T09:41:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.343+0000 7f585ae41640 1 -- start start 2024-03-23T09:41:07.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.344+0000 7f585ae41640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f585415ca90 0x7f5854160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.344+0000 7f585ae41640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58540d11e0 con 0x7f58540b8350 2024-03-23T09:41:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.344+0000 7f585ae41640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5854160f60 con 0x7f58540cde10 2024-03-23T09:41:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.344+0000 7f585ae41640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5854161240 con 0x7f585415ca90 2024-03-23T09:41:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.344+0000 7f5853fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f58540b8350 0x7f58540cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57216/0 (socket says 172.21.15.47:57216) 2024-03-23T09:41:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.344+0000 7f5853fff640 1 -- 172.21.15.47:0/4206990951 learned_addr learned my addr 172.21.15.47:0/4206990951 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.344+0000 7f5858bdd640 1 --2- 172.21.15.47:0/4206990951 >> v2:172.21.15.175:3300/0 conn(0x7f585415ca90 0x7f5854160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.344+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856577902 0 0) 0x7f58540d11e0 con 0x7f58540b8350 2024-03-23T09:41:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.344+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f583c003520 con 0x7f58540b8350 2024-03-23T09:41:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 852343534 0 0) 0x7f583c003520 con 0x7f58540b8350 2024-03-23T09:41:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 >> v2:172.21.15.175:3300/0 conn(0x7f585415ca90 msgr2=0x7f5854160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58537fe640 1 --2- 172.21.15.47:0/4206990951 >> v2:172.21.15.175:3300/0 conn(0x7f585415ca90 0x7f5854160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 >> v1:172.21.15.175:6789/0 conn(0x7f58540cde10 legacy=0x7f58540d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5854161520 con 0x7f58540b8350 2024-03-23T09:41:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f5848001670 con 0x7f58540b8350 2024-03-23T09:41:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 >> v1:172.21.15.47:6789/0 conn(0x7f58540b8350 legacy=0x7f58540cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58537fe640 1 --2- 172.21.15.47:0/4206990951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f583c003970 0x7f583c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58537fe640 1 --2- 172.21.15.47:0/4206990951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f583c0047c0 0x7f583c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58537fe640 1 --2- 172.21.15.47:0/4206990951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f583c005180 0x7f583c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5854161240 con 0x7f583c005180 2024-03-23T09:41:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c00c120 con 0x7f583c003970 2024-03-23T09:41:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583c00c2e0 con 0x7f583c0047c0 2024-03-23T09:41:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58593de640 1 --2- 172.21.15.47:0/4206990951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f583c005180 0x7f583c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f5858bdd640 1 --2- 172.21.15.47:0/4206990951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f583c003970 0x7f583c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f5853fff640 1 --2- 172.21.15.47:0/4206990951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f583c0047c0 0x7f583c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58593de640 1 -- 172.21.15.47:0/4206990951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f583c0047c0 msgr2=0x7f583c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58593de640 1 --2- 172.21.15.47:0/4206990951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f583c0047c0 0x7f583c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58593de640 1 -- 172.21.15.47:0/4206990951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f583c003970 msgr2=0x7f583c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58593de640 1 --2- 172.21.15.47:0/4206990951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f583c003970 0x7f583c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.345+0000 7f58593de640 1 -- 172.21.15.47:0/4206990951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c003520 con 0x7f583c005180 2024-03-23T09:41:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.346+0000 7f58593de640 1 -- 172.21.15.47:0/4206990951 reap_dead start 2024-03-23T09:41:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.346+0000 7f58593de640 1 --2- 172.21.15.47:0/4206990951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f583c005180 0x7f583c00b980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f584c001e20 tx=0x7f583c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3e93d22fbcd791 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.346+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c003b70 con 0x7f583c005180 2024-03-23T09:41:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.346+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c003cd0 con 0x7f583c005180 2024-03-23T09:41:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.346+0000 7f58537fe640 1 -- 172.21.15.47:0/4206990951 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c005310 con 0x7f583c005180 2024-03-23T09:41:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.346+0000 7f585ae41640 1 -- 172.21.15.47:0/4206990951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f583c005180 msgr2=0x7f583c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.346+0000 7f585ae41640 1 --2- 172.21.15.47:0/4206990951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f583c005180 0x7f583c00b980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f584c001e20 tx=0x7f583c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.347+0000 7f585ae41640 1 -- 172.21.15.47:0/4206990951 shutdown_connections 2024-03-23T09:41:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.347+0000 7f585ae41640 1 --2- 172.21.15.47:0/4206990951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f583c005180 0x7f583c00b980 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.347+0000 7f585ae41640 1 -- 172.21.15.47:0/4206990951 >> 172.21.15.47:0/4206990951 conn(0x7f58540659d0 msgr2=0x7f5854065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.347+0000 7f585ae41640 1 -- 172.21.15.47:0/4206990951 shutdown_connections 2024-03-23T09:41:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.347+0000 7f585ae41640 1 -- 172.21.15.47:0/4206990951 wait complete. 2024-03-23T09:41:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.348+0000 7f585ae41640 1 Processor -- start 2024-03-23T09:41:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.348+0000 7f585ae41640 1 -- start start 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f585ae41640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f583c005180 0x7f58540b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f585ae41640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5854148980 0x7f585415f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f585ae41640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58541f1620 0x7f58541f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f585ae41640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58540c9ff0 con 0x7f583c005180 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f585ae41640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58540b43c0 con 0x7f5854148980 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f585ae41640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58540b4640 con 0x7f58541f1620 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f5858bdd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f583c005180 0x7f58540b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f5858bdd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f583c005180 0x7f58540b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45900/0 (socket says 172.21.15.47:45900) 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f5858bdd640 1 -- 172.21.15.47:0/2103509542 learned_addr learned my addr 172.21.15.47:0/2103509542 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f5853fff640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5854148980 0x7f585415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f58593de640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58541f1620 0x7f58541f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f5858bdd640 1 -- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58541f1620 msgr2=0x7f58541f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f5858bdd640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58541f1620 0x7f58541f1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f5858bdd640 1 -- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5854148980 msgr2=0x7f585415f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f5858bdd640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5854148980 0x7f585415f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f5858bdd640 1 -- 172.21.15.47:0/2103509542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58540b4920 con 0x7f583c005180 2024-03-23T09:41:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f5858bdd640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f583c005180 0x7f58540b3e80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58440118e0 tx=0x7f5844011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f58517fa640 1 -- 172.21.15.47:0/2103509542 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584400fa30 con 0x7f583c005180 2024-03-23T09:41:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f58517fa640 1 -- 172.21.15.47:0/2103509542 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584400fb90 con 0x7f583c005180 2024-03-23T09:41:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f585ae41640 1 -- 172.21.15.47:0/2103509542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58540c9c10 con 0x7f583c005180 2024-03-23T09:41:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f58517fa640 1 -- 172.21.15.47:0/2103509542 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844014440 con 0x7f583c005180 2024-03-23T09:41:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.349+0000 7f585ae41640 1 -- 172.21.15.47:0/2103509542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585415fe80 con 0x7f583c005180 2024-03-23T09:41:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.350+0000 7f585ae41640 1 -- 172.21.15.47:0/2103509542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5854072360 con 0x7f583c005180 2024-03-23T09:41:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.351+0000 7f58517fa640 1 -- 172.21.15.47:0/2103509542 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f58440145a0 con 0x7f583c005180 2024-03-23T09:41:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.351+0000 7f58517fa640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f584006ac00 0x7f584006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.351+0000 7f58517fa640 1 -- 172.21.15.47:0/2103509542 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5844089790 con 0x7f583c005180 2024-03-23T09:41:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.351+0000 7f5853fff640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f584006ac00 0x7f584006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.352+0000 7f5853fff640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f584006ac00 0x7f584006d0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f58541579b0 tx=0x7f5848000ef0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.355+0000 7f58517fa640 1 -- 172.21.15.47:0/2103509542 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5844054a00 con 0x7f583c005180 2024-03-23T09:41:07.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.530+0000 7f585ae41640 1 -- 172.21.15.47:0/2103509542 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58540b5b60 con 0x7f584006ac00 2024-03-23T09:41:07.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.536+0000 7f58517fa640 1 -- 172.21.15.47:0/2103509542 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253964 (secure 0 0 0) 0x7f5848001650 con 0x7f584006ac00 2024-03-23T09:41:07.538 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:07.539 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:41:07.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.539+0000 7f585ae41640 1 -- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f584006ac00 msgr2=0x7f584006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:07.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.539+0000 7f585ae41640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f584006ac00 0x7f584006d0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f58541579b0 tx=0x7f5848000ef0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.539+0000 7f585ae41640 1 -- 172.21.15.47:0/2103509542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f583c005180 msgr2=0x7f58540b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:07.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.539+0000 7f585ae41640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f583c005180 0x7f58540b3e80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58440118e0 tx=0x7f5844011ce0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.539+0000 7f585ae41640 1 -- 172.21.15.47:0/2103509542 shutdown_connections 2024-03-23T09:41:07.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.539+0000 7f585ae41640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f584006ac00 0x7f584006d0c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.539+0000 7f585ae41640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58541f1620 0x7f58541f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.539+0000 7f585ae41640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5854148980 0x7f585415f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.539+0000 7f585ae41640 1 --2- 172.21.15.47:0/2103509542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f583c005180 0x7f58540b3e80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:07.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.539+0000 7f585ae41640 1 -- 172.21.15.47:0/2103509542 >> 172.21.15.47:0/2103509542 conn(0x7f58540659d0 msgr2=0x7f5854154020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:07.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.539+0000 7f585ae41640 1 -- 172.21.15.47:0/2103509542 shutdown_connections 2024-03-23T09:41:07.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:07.539+0000 7f585ae41640 1 -- 172.21.15.47:0/2103509542 wait complete. 2024-03-23T09:41:07.549 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-03-23T09:41:06.337929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2500,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":2560000,"data_stored":1600165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.567159"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771175+0000","last_change":"2024-03-23T09:41:01.771175+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771175+0000","last_undegraded":"2024-03-23T09:41:01.771175+0000","last_fullsized":"2024-03-23T09:41:01.771175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772653+0000","last_change":"2024-03-23T09:41:01.772653+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772653+0000","last_undegraded":"2024-03-23T09:41:01.772653+0000","last_fullsized":"2024-03-23T09:41:01.772653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771073+0000","last_change":"2024-03-23T09:41:01.771073+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771073+0000","last_undegraded":"2024-03-23T09:41:01.771073+0000","last_fullsized":"2024-03-23T09:41:01.771073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772119+0000","last_change":"2024-03-23T09:41:01.772119+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772119+0000","last_undegraded":"2024-03-23T09:41:01.772119+0000","last_fullsized":"2024-03-23T09:41:01.772119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.773013+0000","last_change":"2024-03-23T09:41:01.773013+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.773013+0000","last_undegraded":"2024-03-23T09:41:01.773013+0000","last_fullsized":"2024-03-23T09:41:01.773013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.773380+0000","last_change":"2024-03-23T09:41:01.773380+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.773380+0000","last_undegraded":"2024-03-23T09:41:01.773380+0000","last_fullsized":"2024-03-23T09:41:01.773380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772996+0000","last_change":"2024-03-23T09:41:01.772996+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772996+0000","last_undegraded":"2024-03-23T09:41:01.772996+0000","last_fullsized":"2024-03-23T09:41:01.772996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771811+0000","last_change":"2024-03-23T09:41:01.771811+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771811+0000","last_undegraded":"2024-03-23T09:41:01.771811+0000","last_fullsized":"2024-03-23T09:41:01.771811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771848+0000","last_change":"2024-03-23T09:41:01.771848+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771848+0000","last_undegraded":"2024-03-23T09:41:01.771848+0000","last_fullsized":"2024-03-23T09:41:01.771848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.773388+0000","last_change":"2024-03-23T09:41:01.773388+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.773388+0000","last_undegraded":"2024-03-23T09:41:01.773388+0000","last_fullsized":"2024-03-23T09:41:01.773388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771803+0000","last_change":"2024-03-23T09:41:01.771803+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771803+0000","last_undegraded":"2024-03-23T09:41:01.771803+0000","last_fullsized":"2024-03-23T09:41:01.771803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:09:59.090234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.767949+0000","last_change":"2024-03-23T09:41:01.767949+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.767949+0000","last_undegraded":"2024-03-23T09:41:01.767949+0000","last_fullsized":"2024-03-23T09:41:01.767949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768108+0000","last_change":"2024-03-23T09:41:01.768108+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768108+0000","last_undegraded":"2024-03-23T09:41:01.768108+0000","last_fullsized":"2024-03-23T09:41:01.768108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768080+0000","last_change":"2024-03-23T09:41:01.768080+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768080+0000","last_undegraded":"2024-03-23T09:41:01.768080+0000","last_fullsized":"2024-03-23T09:41:01.768080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772436+0000","last_change":"2024-03-23T09:41:01.772436+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772436+0000","last_undegraded":"2024-03-23T09:41:01.772436+0000","last_fullsized":"2024-03-23T09:41:01.772436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.767977+0000","last_change":"2024-03-23T09:41:01.767977+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.767977+0000","last_undegraded":"2024-03-23T09:41:01.767977+0000","last_fullsized":"2024-03-23T09:41:01.767977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-03-23T09:41:01.769360+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:41:01.769360+0000","last_peered":"2024-03-23T09:41:01.769360+0000","last_clean":"2024-03-23T09:41:01.769360+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:41:01.769360+0000","last_undegraded":"2024-03-23T09:41:01.769360+0000","last_fullsized":"2024-03-23T09:41:01.769360+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772612+0000","last_change":"2024-03-23T09:41:01.772612+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772612+0000","last_undegraded":"2024-03-23T09:41:01.772612+0000","last_fullsized":"2024-03-23T09:41:01.772612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768113+0000","last_change":"2024-03-23T09:41:01.768113+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768113+0000","last_undegraded":"2024-03-23T09:41:01.768113+0000","last_fullsized":"2024-03-23T09:41:01.768113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.767991+0000","last_change":"2024-03-23T09:41:01.767991+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.767991+0000","last_undegraded":"2024-03-23T09:41:01.767991+0000","last_fullsized":"2024-03-23T09:41:01.767991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772041+0000","last_change":"2024-03-23T09:41:01.772041+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772041+0000","last_undegraded":"2024-03-23T09:41:01.772041+0000","last_fullsized":"2024-03-23T09:41:01.772041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771158+0000","last_change":"2024-03-23T09:41:01.771158+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771158+0000","last_undegraded":"2024-03-23T09:41:01.771158+0000","last_fullsized":"2024-03-23T09:41:01.771158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768125+0000","last_change":"2024-03-23T09:41:01.768125+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768125+0000","last_undegraded":"2024-03-23T09:41:01.768125+0000","last_fullsized":"2024-03-23T09:41:01.768125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.770996+0000","last_change":"2024-03-23T09:41:01.770996+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.770996+0000","last_undegraded":"2024-03-23T09:41:01.770996+0000","last_fullsized":"2024-03-23T09:41:01.770996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.772497+0000","last_change":"2024-03-23T09:41:01.772497+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.772497+0000","last_undegraded":"2024-03-23T09:41:01.772497+0000","last_fullsized":"2024-03-23T09:41:01.772497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.769148+0000","last_change":"2024-03-23T09:41:01.769148+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.769148+0000","last_undegraded":"2024-03-23T09:41:01.769148+0000","last_fullsized":"2024-03-23T09:41:01.769148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.770857+0000","last_change":"2024-03-23T09:41:01.770857+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.770857+0000","last_undegraded":"2024-03-23T09:41:01.770857+0000","last_fullsized":"2024-03-23T09:41:01.770857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771065+0000","last_change":"2024-03-23T09:41:01.771065+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771065+0000","last_undegraded":"2024-03-23T09:41:01.771065+0000","last_fullsized":"2024-03-23T09:41:01.771065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768126+0000","last_change":"2024-03-23T09:41:01.768126+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768126+0000","last_undegraded":"2024-03-23T09:41:01.768126+0000","last_fullsized":"2024-03-23T09:41:01.768126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768127+0000","last_change":"2024-03-23T09:41:01.768127+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768127+0000","last_undegraded":"2024-03-23T09:41:01.768127+0000","last_fullsized":"2024-03-23T09:41:01.768127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.768856+0000","last_change":"2024-03-23T09:41:01.768856+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.768856+0000","last_undegraded":"2024-03-23T09:41:01.768856+0000","last_fullsized":"2024-03-23T09:41:01.768856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:11.278420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.771267+0000","last_change":"2024-03-23T09:41:01.771267+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.771267+0000","last_undegraded":"2024-03-23T09:41:01.771267+0000","last_fullsized":"2024-03-23T09:41:01.771267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:02.765916+0000","last_change":"2024-03-23T09:41:02.765916+0000","last_active":"2024-03-23T09:41:02.765916+0000","last_peered":"2024-03-23T09:41:02.765916+0000","last_clean":"2024-03-23T09:41:02.765916+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:02.765916+0000","last_undegraded":"2024-03-23T09:41:02.765916+0000","last_fullsized":"2024-03-23T09:41:02.765916+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-23T09:41:01.770977+0000","last_change":"2024-03-23T09:41:01.770977+0000","last_active":"2024-03-23T09:41:01.764185+0000","last_peered":"2024-03-23T09:41:01.764185+0000","last_clean":"2024-03-23T09:41:01.764185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:01.770977+0000","last_undegraded":"2024-03-23T09:41:01.770977+0000","last_fullsized":"2024-03-23T09:41:01.770977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:02:19.284328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28324,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343516,"statfs":{"total":96636764160,"available":96607760384,"internally_reserved":0,"allocated":761856,"data_stored":640770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":55600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":180224,"data_stored":56219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574850,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":757760,"data_stored":640151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:41:07.724 INFO:tasks.ceph.mds.d.smithi175.stderr:2024-03-23T09:41:07.725+0000 7fb6071c6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:41:10.554 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:41:10.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.668+0000 7ff3353a4640 1 Processor -- start 2024-03-23T09:41:10.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.668+0000 7ff3353a4640 1 -- start start 2024-03-23T09:41:10.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.668+0000 7ff3353a4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff3300b8350 0x7ff3300cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:10.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.668+0000 7ff3353a4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3300ce090 con 0x7ff33015ca90 2024-03-23T09:41:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.668+0000 7ff3353a4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3300d11e0 con 0x7ff3300cdc40 2024-03-23T09:41:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.668+0000 7ff3353a4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff330160f60 con 0x7ff3300b8350 2024-03-23T09:41:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.669+0000 7ff3267fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff3300b8350 0x7ff3300cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.669+0000 7ff32effd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff33015ca90 0x7ff330160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57230/0 (socket says 172.21.15.47:57230) 2024-03-23T09:41:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.669+0000 7ff32effd640 1 -- 172.21.15.47:0/1208670787 learned_addr learned my addr 172.21.15.47:0/1208670787 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.669+0000 7ff3267fc640 1 -- 172.21.15.47:0/1208670787 >> v1:172.21.15.175:6789/0 conn(0x7ff3300cdc40 legacy=0x7ff3300d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.669+0000 7ff3267fc640 1 -- 172.21.15.47:0/1208670787 >> v1:172.21.15.47:6789/0 conn(0x7ff33015ca90 legacy=0x7ff330160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.669+0000 7ff3267fc640 1 -- 172.21.15.47:0/1208670787 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff330161240 con 0x7ff3300b8350 2024-03-23T09:41:10.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.669+0000 7ff32e7fc640 1 -- 172.21.15.47:0/1208670787 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975818860 0 0) 0x7ff320001110 con 0x7ff33015ca90 2024-03-23T09:41:10.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.670+0000 7ff3267fc640 1 --2- 172.21.15.47:0/1208670787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3300b8350 0x7ff3300cd700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3180097a0 tx=0x7ff31802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78903748a635ccf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:10.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.670+0000 7ff32e7fc640 1 -- 172.21.15.47:0/1208670787 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31802e070 con 0x7ff3300b8350 2024-03-23T09:41:10.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.670+0000 7ff32e7fc640 1 -- 172.21.15.47:0/1208670787 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3180315a0 con 0x7ff3300b8350 2024-03-23T09:41:10.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.670+0000 7ff32e7fc640 1 -- 172.21.15.47:0/1208670787 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318031930 con 0x7ff3300b8350 2024-03-23T09:41:10.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.670+0000 7ff3353a4640 1 -- 172.21.15.47:0/1208670787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3300b8350 msgr2=0x7ff3300cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:10.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.670+0000 7ff3353a4640 1 --2- 172.21.15.47:0/1208670787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3300b8350 0x7ff3300cd700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3180097a0 tx=0x7ff31802b840 comp rx=0 tx=0).stop 2024-03-23T09:41:10.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.670+0000 7ff3353a4640 1 -- 172.21.15.47:0/1208670787 shutdown_connections 2024-03-23T09:41:10.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.670+0000 7ff3353a4640 1 --2- 172.21.15.47:0/1208670787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3300b8350 0x7ff3300cd700 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:10.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.670+0000 7ff3353a4640 1 -- 172.21.15.47:0/1208670787 >> 172.21.15.47:0/1208670787 conn(0x7ff3300659d0 msgr2=0x7ff330065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:10.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.670+0000 7ff3353a4640 1 -- 172.21.15.47:0/1208670787 shutdown_connections 2024-03-23T09:41:10.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.670+0000 7ff3353a4640 1 -- 172.21.15.47:0/1208670787 wait complete. 2024-03-23T09:41:10.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.672+0000 7ff3353a4640 1 Processor -- start 2024-03-23T09:41:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.672+0000 7ff3353a4640 1 -- start start 2024-03-23T09:41:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.672+0000 7ff3353a4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3300b8350 0x7ff330155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.672+0000 7ff3353a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3300cdc40 0x7ff3300ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.672+0000 7ff3353a4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff33015ca90 0x7ff3301f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff3353a4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3301f6590 con 0x7ff3300b8350 2024-03-23T09:41:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff3353a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3301f6870 con 0x7ff3300cdc40 2024-03-23T09:41:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff3353a4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3300c9c10 con 0x7ff33015ca90 2024-03-23T09:41:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff32effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3300b8350 0x7ff330155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff32effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3300b8350 0x7ff330155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45918/0 (socket says 172.21.15.47:45918) 2024-03-23T09:41:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff32effd640 1 -- 172.21.15.47:0/2006134116 learned_addr learned my addr 172.21.15.47:0/2006134116 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff3267fc640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3300cdc40 0x7ff3300ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff32f7fe640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff33015ca90 0x7ff3301f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff32effd640 1 -- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff33015ca90 msgr2=0x7ff3301f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff32effd640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff33015ca90 0x7ff3301f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff32effd640 1 -- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3300cdc40 msgr2=0x7ff3300ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff32effd640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3300cdc40 0x7ff3300ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff32effd640 1 -- 172.21.15.47:0/2006134116 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff318009400 con 0x7ff3300b8350 2024-03-23T09:41:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff32effd640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3300b8350 0x7ff330155e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff32000c780 tx=0x7ff32000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff327fff640 1 -- 172.21.15.47:0/2006134116 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3200072d0 con 0x7ff3300b8350 2024-03-23T09:41:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff327fff640 1 -- 172.21.15.47:0/2006134116 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff320007450 con 0x7ff3300b8350 2024-03-23T09:41:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff3353a4640 1 -- 172.21.15.47:0/2006134116 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3300c9ef0 con 0x7ff3300b8350 2024-03-23T09:41:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff327fff640 1 -- 172.21.15.47:0/2006134116 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3200047e0 con 0x7ff3300b8350 2024-03-23T09:41:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.673+0000 7ff3353a4640 1 -- 172.21.15.47:0/2006134116 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3300cb8b0 con 0x7ff3300b8350 2024-03-23T09:41:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.674+0000 7ff3353a4640 1 -- 172.21.15.47:0/2006134116 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff330072360 con 0x7ff3300b8350 2024-03-23T09:41:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.675+0000 7ff327fff640 1 -- 172.21.15.47:0/2006134116 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff320004960 con 0x7ff3300b8350 2024-03-23T09:41:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.675+0000 7ff327fff640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff31006ac00 0x7ff31006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.675+0000 7ff327fff640 1 -- 172.21.15.47:0/2006134116 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3200886e0 con 0x7ff3300b8350 2024-03-23T09:41:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.675+0000 7ff3267fc640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff31006ac00 0x7ff31006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.676+0000 7ff3267fc640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff31006ac00 0x7ff31006d0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff318036000 tx=0x7ff31803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.680+0000 7ff327fff640 1 -- 172.21.15.47:0/2006134116 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff320053950 con 0x7ff3300b8350 2024-03-23T09:41:10.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.858+0000 7ff3353a4640 1 -- 172.21.15.47:0/2006134116 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3300d0e70 con 0x7ff31006ac00 2024-03-23T09:41:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.865+0000 7ff327fff640 1 -- 172.21.15.47:0/2006134116 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264618 (secure 0 0 0) 0x7ff318031b30 con 0x7ff31006ac00 2024-03-23T09:41:10.869 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:41:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.868+0000 7ff3353a4640 1 -- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff31006ac00 msgr2=0x7ff31006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.868+0000 7ff3353a4640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff31006ac00 0x7ff31006d0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff318036000 tx=0x7ff31803d040 comp rx=0 tx=0).stop 2024-03-23T09:41:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.868+0000 7ff3353a4640 1 -- 172.21.15.47:0/2006134116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3300b8350 msgr2=0x7ff330155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.868+0000 7ff3353a4640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3300b8350 0x7ff330155e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff32000c780 tx=0x7ff32000cc40 comp rx=0 tx=0).stop 2024-03-23T09:41:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.868+0000 7ff3353a4640 1 -- 172.21.15.47:0/2006134116 shutdown_connections 2024-03-23T09:41:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.868+0000 7ff3353a4640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff31006ac00 0x7ff31006d0c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.868+0000 7ff3353a4640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff33015ca90 0x7ff3301f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.868+0000 7ff3353a4640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3300cdc40 0x7ff3300ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.868+0000 7ff3353a4640 1 --2- 172.21.15.47:0/2006134116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3300b8350 0x7ff330155e00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.868+0000 7ff3353a4640 1 -- 172.21.15.47:0/2006134116 >> 172.21.15.47:0/2006134116 conn(0x7ff3300659d0 msgr2=0x7ff33015d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.868+0000 7ff3353a4640 1 -- 172.21.15.47:0/2006134116 shutdown_connections 2024-03-23T09:41:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:10.868+0000 7ff3353a4640 1 -- 172.21.15.47:0/2006134116 wait complete. 2024-03-23T09:41:10.879 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-23T09:41:10.339075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236068,"kb_used_data":2912,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738652,"statfs":{"total":773094113280,"available":772852379648,"internally_reserved":0,"allocated":2981888,"data_stored":1788774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.568305"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.147121+0000","last_change":"2024-03-23T09:41:03.164352+0000","last_active":"2024-03-23T09:41:04.147121+0000","last_peered":"2024-03-23T09:41:04.147121+0000","last_clean":"2024-03-23T09:41:04.147121+0000","last_became_active":"2024-03-23T09:41:03.164058+0000","last_became_peered":"2024-03-23T09:41:03.164058+0000","last_unstale":"2024-03-23T09:41:04.147121+0000","last_undegraded":"2024-03-23T09:41:04.147121+0000","last_fullsized":"2024-03-23T09:41:04.147121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:54:59.125701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142356+0000","last_change":"2024-03-23T09:41:03.159612+0000","last_active":"2024-03-23T09:41:04.142356+0000","last_peered":"2024-03-23T09:41:04.142356+0000","last_clean":"2024-03-23T09:41:04.142356+0000","last_became_active":"2024-03-23T09:41:03.159341+0000","last_became_peered":"2024-03-23T09:41:03.159341+0000","last_unstale":"2024-03-23T09:41:04.142356+0000","last_undegraded":"2024-03-23T09:41:04.142356+0000","last_fullsized":"2024-03-23T09:41:04.142356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586632+0000","last_change":"2024-03-23T09:41:03.156325+0000","last_active":"2024-03-23T09:41:04.586632+0000","last_peered":"2024-03-23T09:41:04.586632+0000","last_clean":"2024-03-23T09:41:04.586632+0000","last_became_active":"2024-03-23T09:41:03.156061+0000","last_became_peered":"2024-03-23T09:41:03.156061+0000","last_unstale":"2024-03-23T09:41:04.586632+0000","last_undegraded":"2024-03-23T09:41:04.586632+0000","last_fullsized":"2024-03-23T09:41:04.586632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:06:51.567637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155051+0000","last_change":"2024-03-23T09:41:03.158679+0000","last_active":"2024-03-23T09:41:04.155051+0000","last_peered":"2024-03-23T09:41:04.155051+0000","last_clean":"2024-03-23T09:41:04.155051+0000","last_became_active":"2024-03-23T09:41:03.158151+0000","last_became_peered":"2024-03-23T09:41:03.158151+0000","last_unstale":"2024-03-23T09:41:04.155051+0000","last_undegraded":"2024-03-23T09:41:04.155051+0000","last_fullsized":"2024-03-23T09:41:04.155051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:29.855586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146283+0000","last_change":"2024-03-23T09:41:03.167599+0000","last_active":"2024-03-23T09:41:04.146283+0000","last_peered":"2024-03-23T09:41:04.146283+0000","last_clean":"2024-03-23T09:41:04.146283+0000","last_became_active":"2024-03-23T09:41:03.167332+0000","last_became_peered":"2024-03-23T09:41:03.167332+0000","last_unstale":"2024-03-23T09:41:04.146283+0000","last_undegraded":"2024-03-23T09:41:04.146283+0000","last_fullsized":"2024-03-23T09:41:04.146283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:32:17.579489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.154914+0000","last_change":"2024-03-23T09:41:03.162714+0000","last_active":"2024-03-23T09:41:04.154914+0000","last_peered":"2024-03-23T09:41:04.154914+0000","last_clean":"2024-03-23T09:41:04.154914+0000","last_became_active":"2024-03-23T09:41:03.162461+0000","last_became_peered":"2024-03-23T09:41:03.162461+0000","last_unstale":"2024-03-23T09:41:04.154914+0000","last_undegraded":"2024-03-23T09:41:04.154914+0000","last_fullsized":"2024-03-23T09:41:04.154914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:30:48.469383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146273+0000","last_change":"2024-03-23T09:41:03.167591+0000","last_active":"2024-03-23T09:41:04.146273+0000","last_peered":"2024-03-23T09:41:04.146273+0000","last_clean":"2024-03-23T09:41:04.146273+0000","last_became_active":"2024-03-23T09:41:03.167320+0000","last_became_peered":"2024-03-23T09:41:03.167320+0000","last_unstale":"2024-03-23T09:41:04.146273+0000","last_undegraded":"2024-03-23T09:41:04.146273+0000","last_fullsized":"2024-03-23T09:41:04.146273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:29.855586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665874+0000","last_change":"2024-03-23T09:41:03.153495+0000","last_active":"2024-03-23T09:41:04.665874+0000","last_peered":"2024-03-23T09:41:04.665874+0000","last_clean":"2024-03-23T09:41:04.665874+0000","last_became_active":"2024-03-23T09:41:03.153257+0000","last_became_peered":"2024-03-23T09:41:03.153257+0000","last_unstale":"2024-03-23T09:41:04.665874+0000","last_undegraded":"2024-03-23T09:41:04.665874+0000","last_fullsized":"2024-03-23T09:41:04.665874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:46:38.667131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.147216+0000","last_change":"2024-03-23T09:41:03.170688+0000","last_active":"2024-03-23T09:41:04.147216+0000","last_peered":"2024-03-23T09:41:04.147216+0000","last_clean":"2024-03-23T09:41:04.147216+0000","last_became_active":"2024-03-23T09:41:03.170446+0000","last_became_peered":"2024-03-23T09:41:03.170446+0000","last_unstale":"2024-03-23T09:41:04.147216+0000","last_undegraded":"2024-03-23T09:41:04.147216+0000","last_fullsized":"2024-03-23T09:41:04.147216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:22:15.797821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666694+0000","last_change":"2024-03-23T09:41:03.162169+0000","last_active":"2024-03-23T09:41:04.666694+0000","last_peered":"2024-03-23T09:41:04.666694+0000","last_clean":"2024-03-23T09:41:04.666694+0000","last_became_active":"2024-03-23T09:41:03.161587+0000","last_became_peered":"2024-03-23T09:41:03.161587+0000","last_unstale":"2024-03-23T09:41:04.666694+0000","last_undegraded":"2024-03-23T09:41:04.666694+0000","last_fullsized":"2024-03-23T09:41:04.666694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666339+0000","last_change":"2024-03-23T09:41:03.160438+0000","last_active":"2024-03-23T09:41:04.666339+0000","last_peered":"2024-03-23T09:41:04.666339+0000","last_clean":"2024-03-23T09:41:04.666339+0000","last_became_active":"2024-03-23T09:41:03.160175+0000","last_became_peered":"2024-03-23T09:41:03.160175+0000","last_unstale":"2024-03-23T09:41:04.666339+0000","last_undegraded":"2024-03-23T09:41:04.666339+0000","last_fullsized":"2024-03-23T09:41:04.666339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146726+0000","last_change":"2024-03-23T09:41:03.163213+0000","last_active":"2024-03-23T09:41:04.146726+0000","last_peered":"2024-03-23T09:41:04.146726+0000","last_clean":"2024-03-23T09:41:04.146726+0000","last_became_active":"2024-03-23T09:41:03.162954+0000","last_became_peered":"2024-03-23T09:41:03.162954+0000","last_unstale":"2024-03-23T09:41:04.146726+0000","last_undegraded":"2024-03-23T09:41:04.146726+0000","last_fullsized":"2024-03-23T09:41:04.146726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:28.007092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666209+0000","last_change":"2024-03-23T09:41:03.161818+0000","last_active":"2024-03-23T09:41:04.666209+0000","last_peered":"2024-03-23T09:41:04.666209+0000","last_clean":"2024-03-23T09:41:04.666209+0000","last_became_active":"2024-03-23T09:41:03.159384+0000","last_became_peered":"2024-03-23T09:41:03.159384+0000","last_unstale":"2024-03-23T09:41:04.666209+0000","last_undegraded":"2024-03-23T09:41:04.666209+0000","last_fullsized":"2024-03-23T09:41:04.666209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586410+0000","last_change":"2024-03-23T09:41:03.159776+0000","last_active":"2024-03-23T09:41:04.586410+0000","last_peered":"2024-03-23T09:41:04.586410+0000","last_clean":"2024-03-23T09:41:04.586410+0000","last_became_active":"2024-03-23T09:41:03.159466+0000","last_became_peered":"2024-03-23T09:41:03.159466+0000","last_unstale":"2024-03-23T09:41:04.586410+0000","last_undegraded":"2024-03-23T09:41:04.586410+0000","last_fullsized":"2024-03-23T09:41:04.586410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142102+0000","last_change":"2024-03-23T09:41:03.154187+0000","last_active":"2024-03-23T09:41:04.142102+0000","last_peered":"2024-03-23T09:41:04.142102+0000","last_clean":"2024-03-23T09:41:04.142102+0000","last_became_active":"2024-03-23T09:41:03.153946+0000","last_became_peered":"2024-03-23T09:41:03.153946+0000","last_unstale":"2024-03-23T09:41:04.142102+0000","last_undegraded":"2024-03-23T09:41:04.142102+0000","last_fullsized":"2024-03-23T09:41:04.142102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.154801+0000","last_change":"2024-03-23T09:41:03.160290+0000","last_active":"2024-03-23T09:41:04.154801+0000","last_peered":"2024-03-23T09:41:04.154801+0000","last_clean":"2024-03-23T09:41:04.154801+0000","last_became_active":"2024-03-23T09:41:03.160010+0000","last_became_peered":"2024-03-23T09:41:03.160010+0000","last_unstale":"2024-03-23T09:41:04.154801+0000","last_undegraded":"2024-03-23T09:41:04.154801+0000","last_fullsized":"2024-03-23T09:41:04.154801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155450+0000","last_change":"2024-03-23T09:41:03.165189+0000","last_active":"2024-03-23T09:41:04.155450+0000","last_peered":"2024-03-23T09:41:04.155450+0000","last_clean":"2024-03-23T09:41:04.155450+0000","last_became_active":"2024-03-23T09:41:03.164956+0000","last_became_peered":"2024-03-23T09:41:03.164956+0000","last_unstale":"2024-03-23T09:41:04.155450+0000","last_undegraded":"2024-03-23T09:41:04.155450+0000","last_fullsized":"2024-03-23T09:41:04.155450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665970+0000","last_change":"2024-03-23T09:41:03.163535+0000","last_active":"2024-03-23T09:41:04.665970+0000","last_peered":"2024-03-23T09:41:04.665970+0000","last_clean":"2024-03-23T09:41:04.665970+0000","last_became_active":"2024-03-23T09:41:03.163271+0000","last_became_peered":"2024-03-23T09:41:03.163271+0000","last_unstale":"2024-03-23T09:41:04.665970+0000","last_undegraded":"2024-03-23T09:41:04.665970+0000","last_fullsized":"2024-03-23T09:41:04.665970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:12.280151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146986+0000","last_change":"2024-03-23T09:41:03.168829+0000","last_active":"2024-03-23T09:41:04.146986+0000","last_peered":"2024-03-23T09:41:04.146986+0000","last_clean":"2024-03-23T09:41:04.146986+0000","last_became_active":"2024-03-23T09:41:03.167734+0000","last_became_peered":"2024-03-23T09:41:03.167734+0000","last_unstale":"2024-03-23T09:41:04.146986+0000","last_undegraded":"2024-03-23T09:41:04.146986+0000","last_fullsized":"2024-03-23T09:41:04.146986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:11:48.328619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666441+0000","last_change":"2024-03-23T09:41:03.161820+0000","last_active":"2024-03-23T09:41:04.666441+0000","last_peered":"2024-03-23T09:41:04.666441+0000","last_clean":"2024-03-23T09:41:04.666441+0000","last_became_active":"2024-03-23T09:41:03.161310+0000","last_became_peered":"2024-03-23T09:41:03.161310+0000","last_unstale":"2024-03-23T09:41:04.666441+0000","last_undegraded":"2024-03-23T09:41:04.666441+0000","last_fullsized":"2024-03-23T09:41:04.666441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:42:19.838774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586549+0000","last_change":"2024-03-23T09:41:03.171444+0000","last_active":"2024-03-23T09:41:04.586549+0000","last_peered":"2024-03-23T09:41:04.586549+0000","last_clean":"2024-03-23T09:41:04.586549+0000","last_became_active":"2024-03-23T09:41:03.171114+0000","last_became_peered":"2024-03-23T09:41:03.171114+0000","last_unstale":"2024-03-23T09:41:04.586549+0000","last_undegraded":"2024-03-23T09:41:04.586549+0000","last_fullsized":"2024-03-23T09:41:04.586549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.154962+0000","last_change":"2024-03-23T09:41:03.166192+0000","last_active":"2024-03-23T09:41:04.154962+0000","last_peered":"2024-03-23T09:41:04.154962+0000","last_clean":"2024-03-23T09:41:04.154962+0000","last_became_active":"2024-03-23T09:41:03.165944+0000","last_became_peered":"2024-03-23T09:41:03.165944+0000","last_unstale":"2024-03-23T09:41:04.154962+0000","last_undegraded":"2024-03-23T09:41:04.154962+0000","last_fullsized":"2024-03-23T09:41:04.154962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:51:26.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666043+0000","last_change":"2024-03-23T09:41:03.149122+0000","last_active":"2024-03-23T09:41:04.666043+0000","last_peered":"2024-03-23T09:41:04.666043+0000","last_clean":"2024-03-23T09:41:04.666043+0000","last_became_active":"2024-03-23T09:41:03.148885+0000","last_became_peered":"2024-03-23T09:41:03.148885+0000","last_unstale":"2024-03-23T09:41:04.666043+0000","last_undegraded":"2024-03-23T09:41:04.666043+0000","last_fullsized":"2024-03-23T09:41:04.666043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:06.444696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.154922+0000","last_change":"2024-03-23T09:41:03.155719+0000","last_active":"2024-03-23T09:41:04.154922+0000","last_peered":"2024-03-23T09:41:04.154922+0000","last_clean":"2024-03-23T09:41:04.154922+0000","last_became_active":"2024-03-23T09:41:03.155436+0000","last_became_peered":"2024-03-23T09:41:03.155436+0000","last_unstale":"2024-03-23T09:41:04.154922+0000","last_undegraded":"2024-03-23T09:41:04.154922+0000","last_fullsized":"2024-03-23T09:41:04.154922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666163+0000","last_change":"2024-03-23T09:41:03.148314+0000","last_active":"2024-03-23T09:41:04.666163+0000","last_peered":"2024-03-23T09:41:04.666163+0000","last_clean":"2024-03-23T09:41:04.666163+0000","last_became_active":"2024-03-23T09:41:03.148048+0000","last_became_peered":"2024-03-23T09:41:03.148048+0000","last_unstale":"2024-03-23T09:41:04.666163+0000","last_undegraded":"2024-03-23T09:41:04.666163+0000","last_fullsized":"2024-03-23T09:41:04.666163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:09.777921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666670+0000","last_change":"2024-03-23T09:41:03.169146+0000","last_active":"2024-03-23T09:41:04.666670+0000","last_peered":"2024-03-23T09:41:04.666670+0000","last_clean":"2024-03-23T09:41:04.666670+0000","last_became_active":"2024-03-23T09:41:03.168877+0000","last_became_peered":"2024-03-23T09:41:03.168877+0000","last_unstale":"2024-03-23T09:41:04.666670+0000","last_undegraded":"2024-03-23T09:41:04.666670+0000","last_fullsized":"2024-03-23T09:41:04.666670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:10.603669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142270+0000","last_change":"2024-03-23T09:41:03.147239+0000","last_active":"2024-03-23T09:41:04.142270+0000","last_peered":"2024-03-23T09:41:04.142270+0000","last_clean":"2024-03-23T09:41:04.142270+0000","last_became_active":"2024-03-23T09:41:03.146990+0000","last_became_peered":"2024-03-23T09:41:03.146990+0000","last_unstale":"2024-03-23T09:41:04.142270+0000","last_undegraded":"2024-03-23T09:41:04.142270+0000","last_fullsized":"2024-03-23T09:41:04.142270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666449+0000","last_change":"2024-03-23T09:41:03.149202+0000","last_active":"2024-03-23T09:41:04.666449+0000","last_peered":"2024-03-23T09:41:04.666449+0000","last_clean":"2024-03-23T09:41:04.666449+0000","last_became_active":"2024-03-23T09:41:03.148958+0000","last_became_peered":"2024-03-23T09:41:03.148958+0000","last_unstale":"2024-03-23T09:41:04.666449+0000","last_undegraded":"2024-03-23T09:41:04.666449+0000","last_fullsized":"2024-03-23T09:41:04.666449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:10.603669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586617+0000","last_change":"2024-03-23T09:41:03.157714+0000","last_active":"2024-03-23T09:41:04.586617+0000","last_peered":"2024-03-23T09:41:04.586617+0000","last_clean":"2024-03-23T09:41:04.586617+0000","last_became_active":"2024-03-23T09:41:03.157315+0000","last_became_peered":"2024-03-23T09:41:03.157315+0000","last_unstale":"2024-03-23T09:41:04.586617+0000","last_undegraded":"2024-03-23T09:41:04.586617+0000","last_fullsized":"2024-03-23T09:41:04.586617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:06.444696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155043+0000","last_change":"2024-03-23T09:41:03.155373+0000","last_active":"2024-03-23T09:41:04.155043+0000","last_peered":"2024-03-23T09:41:04.155043+0000","last_clean":"2024-03-23T09:41:04.155043+0000","last_became_active":"2024-03-23T09:41:03.155063+0000","last_became_peered":"2024-03-23T09:41:03.155063+0000","last_unstale":"2024-03-23T09:41:04.155043+0000","last_undegraded":"2024-03-23T09:41:04.155043+0000","last_fullsized":"2024-03-23T09:41:04.155043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:33:41.605578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665826+0000","last_change":"2024-03-23T09:41:03.154532+0000","last_active":"2024-03-23T09:41:04.665826+0000","last_peered":"2024-03-23T09:41:04.665826+0000","last_clean":"2024-03-23T09:41:04.665826+0000","last_became_active":"2024-03-23T09:41:03.154277+0000","last_became_peered":"2024-03-23T09:41:03.154277+0000","last_unstale":"2024-03-23T09:41:04.665826+0000","last_undegraded":"2024-03-23T09:41:04.665826+0000","last_fullsized":"2024-03-23T09:41:04.665826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:49.353786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666387+0000","last_change":"2024-03-23T09:41:03.166933+0000","last_active":"2024-03-23T09:41:04.666387+0000","last_peered":"2024-03-23T09:41:04.666387+0000","last_clean":"2024-03-23T09:41:04.666387+0000","last_became_active":"2024-03-23T09:41:03.166359+0000","last_became_peered":"2024-03-23T09:41:03.166359+0000","last_unstale":"2024-03-23T09:41:04.666387+0000","last_undegraded":"2024-03-23T09:41:04.666387+0000","last_fullsized":"2024-03-23T09:41:04.666387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:09.777921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146129+0000","last_change":"2024-03-23T09:41:03.149144+0000","last_active":"2024-03-23T09:41:04.146129+0000","last_peered":"2024-03-23T09:41:04.146129+0000","last_clean":"2024-03-23T09:41:04.146129+0000","last_became_active":"2024-03-23T09:41:03.148842+0000","last_became_peered":"2024-03-23T09:41:03.148842+0000","last_unstale":"2024-03-23T09:41:04.146129+0000","last_undegraded":"2024-03-23T09:41:04.146129+0000","last_fullsized":"2024-03-23T09:41:04.146129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:12.280151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.167252+0000","last_change":"2024-03-23T09:41:02.789724+0000","last_active":"2024-03-23T09:41:03.167252+0000","last_peered":"2024-03-23T09:41:03.167252+0000","last_clean":"2024-03-23T09:41:03.167252+0000","last_became_active":"2024-03-23T09:41:02.789485+0000","last_became_peered":"2024-03-23T09:41:02.789485+0000","last_unstale":"2024-03-23T09:41:03.167252+0000","last_undegraded":"2024-03-23T09:41:03.167252+0000","last_fullsized":"2024-03-23T09:41:03.167252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586658+0000","last_change":"2024-03-23T09:41:03.157560+0000","last_active":"2024-03-23T09:41:04.586658+0000","last_peered":"2024-03-23T09:41:04.586658+0000","last_clean":"2024-03-23T09:41:04.586658+0000","last_became_active":"2024-03-23T09:41:03.157046+0000","last_became_peered":"2024-03-23T09:41:03.157046+0000","last_unstale":"2024-03-23T09:41:04.586658+0000","last_undegraded":"2024-03-23T09:41:04.586658+0000","last_fullsized":"2024-03-23T09:41:04.586658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:04:04.702736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.161341+0000","last_change":"2024-03-23T09:41:02.785765+0000","last_active":"2024-03-23T09:41:03.161341+0000","last_peered":"2024-03-23T09:41:03.161341+0000","last_clean":"2024-03-23T09:41:03.161341+0000","last_became_active":"2024-03-23T09:41:02.785483+0000","last_became_peered":"2024-03-23T09:41:02.785483+0000","last_unstale":"2024-03-23T09:41:03.161341+0000","last_undegraded":"2024-03-23T09:41:03.161341+0000","last_fullsized":"2024-03-23T09:41:03.161341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.145332+0000","last_change":"2024-03-23T09:41:02.790132+0000","last_active":"2024-03-23T09:41:03.145332+0000","last_peered":"2024-03-23T09:41:03.145332+0000","last_clean":"2024-03-23T09:41:03.145332+0000","last_became_active":"2024-03-23T09:41:02.789893+0000","last_became_peered":"2024-03-23T09:41:02.789893+0000","last_unstale":"2024-03-23T09:41:03.145332+0000","last_undegraded":"2024-03-23T09:41:03.145332+0000","last_fullsized":"2024-03-23T09:41:03.145332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665949+0000","last_change":"2024-03-23T09:41:03.154684+0000","last_active":"2024-03-23T09:41:04.665949+0000","last_peered":"2024-03-23T09:41:04.665949+0000","last_clean":"2024-03-23T09:41:04.665949+0000","last_became_active":"2024-03-23T09:41:03.154441+0000","last_became_peered":"2024-03-23T09:41:03.154441+0000","last_unstale":"2024-03-23T09:41:04.665949+0000","last_undegraded":"2024-03-23T09:41:04.665949+0000","last_fullsized":"2024-03-23T09:41:04.665949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.149205+0000","last_change":"2024-03-23T09:41:02.792603+0000","last_active":"2024-03-23T09:41:03.149205+0000","last_peered":"2024-03-23T09:41:03.149205+0000","last_clean":"2024-03-23T09:41:03.149205+0000","last_became_active":"2024-03-23T09:41:02.792316+0000","last_became_peered":"2024-03-23T09:41:02.792316+0000","last_unstale":"2024-03-23T09:41:03.149205+0000","last_undegraded":"2024-03-23T09:41:03.149205+0000","last_fullsized":"2024-03-23T09:41:03.149205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142096+0000","last_change":"2024-03-23T09:41:03.147255+0000","last_active":"2024-03-23T09:41:04.142096+0000","last_peered":"2024-03-23T09:41:04.142096+0000","last_clean":"2024-03-23T09:41:04.142096+0000","last_became_active":"2024-03-23T09:41:03.147013+0000","last_became_peered":"2024-03-23T09:41:03.147013+0000","last_unstale":"2024-03-23T09:41:04.142096+0000","last_undegraded":"2024-03-23T09:41:04.142096+0000","last_fullsized":"2024-03-23T09:41:04.142096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:16.974803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.152789+0000","last_change":"2024-03-23T09:41:02.791718+0000","last_active":"2024-03-23T09:41:03.152789+0000","last_peered":"2024-03-23T09:41:03.152789+0000","last_clean":"2024-03-23T09:41:03.152789+0000","last_became_active":"2024-03-23T09:41:02.791442+0000","last_became_peered":"2024-03-23T09:41:02.791442+0000","last_unstale":"2024-03-23T09:41:03.152789+0000","last_undegraded":"2024-03-23T09:41:03.152789+0000","last_fullsized":"2024-03-23T09:41:03.152789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155272+0000","last_change":"2024-03-23T09:41:03.171999+0000","last_active":"2024-03-23T09:41:04.155272+0000","last_peered":"2024-03-23T09:41:04.155272+0000","last_clean":"2024-03-23T09:41:04.155272+0000","last_became_active":"2024-03-23T09:41:03.171731+0000","last_became_peered":"2024-03-23T09:41:03.171731+0000","last_unstale":"2024-03-23T09:41:04.155272+0000","last_undegraded":"2024-03-23T09:41:04.155272+0000","last_fullsized":"2024-03-23T09:41:04.155272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:16.974803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.150565+0000","last_change":"2024-03-23T09:41:02.791445+0000","last_active":"2024-03-23T09:41:03.150565+0000","last_peered":"2024-03-23T09:41:03.150565+0000","last_clean":"2024-03-23T09:41:03.150565+0000","last_became_active":"2024-03-23T09:41:02.791075+0000","last_became_peered":"2024-03-23T09:41:02.791075+0000","last_unstale":"2024-03-23T09:41:03.150565+0000","last_undegraded":"2024-03-23T09:41:03.150565+0000","last_fullsized":"2024-03-23T09:41:03.150565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586653+0000","last_change":"2024-03-23T09:41:03.159171+0000","last_active":"2024-03-23T09:41:04.586653+0000","last_peered":"2024-03-23T09:41:04.586653+0000","last_clean":"2024-03-23T09:41:04.586653+0000","last_became_active":"2024-03-23T09:41:03.158869+0000","last_became_peered":"2024-03-23T09:41:03.158869+0000","last_unstale":"2024-03-23T09:41:04.586653+0000","last_undegraded":"2024-03-23T09:41:04.586653+0000","last_fullsized":"2024-03-23T09:41:04.586653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:42:19.838774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.160077+0000","last_change":"2024-03-23T09:41:02.786747+0000","last_active":"2024-03-23T09:41:03.160077+0000","last_peered":"2024-03-23T09:41:03.160077+0000","last_clean":"2024-03-23T09:41:03.160077+0000","last_became_active":"2024-03-23T09:41:02.786499+0000","last_became_peered":"2024-03-23T09:41:02.786499+0000","last_unstale":"2024-03-23T09:41:03.160077+0000","last_undegraded":"2024-03-23T09:41:03.160077+0000","last_fullsized":"2024-03-23T09:41:03.160077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146832+0000","last_change":"2024-03-23T09:41:03.165917+0000","last_active":"2024-03-23T09:41:04.146832+0000","last_peered":"2024-03-23T09:41:04.146832+0000","last_clean":"2024-03-23T09:41:04.146832+0000","last_became_active":"2024-03-23T09:41:03.165617+0000","last_became_peered":"2024-03-23T09:41:03.165617+0000","last_unstale":"2024-03-23T09:41:04.146832+0000","last_undegraded":"2024-03-23T09:41:04.146832+0000","last_fullsized":"2024-03-23T09:41:04.146832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:49:00.143163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.169412+0000","last_change":"2024-03-23T09:41:02.787570+0000","last_active":"2024-03-23T09:41:03.169412+0000","last_peered":"2024-03-23T09:41:03.169412+0000","last_clean":"2024-03-23T09:41:03.169412+0000","last_became_active":"2024-03-23T09:41:02.785815+0000","last_became_peered":"2024-03-23T09:41:02.785815+0000","last_unstale":"2024-03-23T09:41:03.169412+0000","last_undegraded":"2024-03-23T09:41:03.169412+0000","last_fullsized":"2024-03-23T09:41:03.169412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146644+0000","last_change":"2024-03-23T09:41:03.167089+0000","last_active":"2024-03-23T09:41:04.146644+0000","last_peered":"2024-03-23T09:41:04.146644+0000","last_clean":"2024-03-23T09:41:04.146644+0000","last_became_active":"2024-03-23T09:41:03.166788+0000","last_became_peered":"2024-03-23T09:41:03.166788+0000","last_unstale":"2024-03-23T09:41:04.146644+0000","last_undegraded":"2024-03-23T09:41:04.146644+0000","last_fullsized":"2024-03-23T09:41:04.146644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:22:09.627571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.161339+0000","last_change":"2024-03-23T09:41:02.785499+0000","last_active":"2024-03-23T09:41:03.161339+0000","last_peered":"2024-03-23T09:41:03.161339+0000","last_clean":"2024-03-23T09:41:03.161339+0000","last_became_active":"2024-03-23T09:41:02.785228+0000","last_became_peered":"2024-03-23T09:41:02.785228+0000","last_unstale":"2024-03-23T09:41:03.161339+0000","last_undegraded":"2024-03-23T09:41:03.161339+0000","last_fullsized":"2024-03-23T09:41:03.161339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586476+0000","last_change":"2024-03-23T09:41:03.164565+0000","last_active":"2024-03-23T09:41:04.586476+0000","last_peered":"2024-03-23T09:41:04.586476+0000","last_clean":"2024-03-23T09:41:04.586476+0000","last_became_active":"2024-03-23T09:41:03.164285+0000","last_became_peered":"2024-03-23T09:41:03.164285+0000","last_unstale":"2024-03-23T09:41:04.586476+0000","last_undegraded":"2024-03-23T09:41:04.586476+0000","last_fullsized":"2024-03-23T09:41:04.586476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.149928+0000","last_change":"2024-03-23T09:41:02.790603+0000","last_active":"2024-03-23T09:41:03.149928+0000","last_peered":"2024-03-23T09:41:03.149928+0000","last_clean":"2024-03-23T09:41:03.149928+0000","last_became_active":"2024-03-23T09:41:02.790337+0000","last_became_peered":"2024-03-23T09:41:02.790337+0000","last_unstale":"2024-03-23T09:41:03.149928+0000","last_undegraded":"2024-03-23T09:41:03.149928+0000","last_fullsized":"2024-03-23T09:41:03.149928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146310+0000","last_change":"2024-03-23T09:41:03.170711+0000","last_active":"2024-03-23T09:41:04.146310+0000","last_peered":"2024-03-23T09:41:04.146310+0000","last_clean":"2024-03-23T09:41:04.146310+0000","last_became_active":"2024-03-23T09:41:03.170418+0000","last_became_peered":"2024-03-23T09:41:03.170418+0000","last_unstale":"2024-03-23T09:41:04.146310+0000","last_undegraded":"2024-03-23T09:41:04.146310+0000","last_fullsized":"2024-03-23T09:41:04.146310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:16.974803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.153343+0000","last_change":"2024-03-23T09:41:02.788294+0000","last_active":"2024-03-23T09:41:03.153343+0000","last_peered":"2024-03-23T09:41:03.153343+0000","last_clean":"2024-03-23T09:41:03.153343+0000","last_became_active":"2024-03-23T09:41:02.787911+0000","last_became_peered":"2024-03-23T09:41:02.787911+0000","last_unstale":"2024-03-23T09:41:03.153343+0000","last_undegraded":"2024-03-23T09:41:03.153343+0000","last_fullsized":"2024-03-23T09:41:03.153343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:09:59.090234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666362+0000","last_change":"2024-03-23T09:41:03.166733+0000","last_active":"2024-03-23T09:41:04.666362+0000","last_peered":"2024-03-23T09:41:04.666362+0000","last_clean":"2024-03-23T09:41:04.666362+0000","last_became_active":"2024-03-23T09:41:03.165156+0000","last_became_peered":"2024-03-23T09:41:03.165156+0000","last_unstale":"2024-03-23T09:41:04.666362+0000","last_undegraded":"2024-03-23T09:41:04.666362+0000","last_fullsized":"2024-03-23T09:41:04.666362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:06.444696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.163373+0000","last_change":"2024-03-23T09:41:02.784571+0000","last_active":"2024-03-23T09:41:03.163373+0000","last_peered":"2024-03-23T09:41:03.163373+0000","last_clean":"2024-03-23T09:41:03.163373+0000","last_became_active":"2024-03-23T09:41:02.784282+0000","last_became_peered":"2024-03-23T09:41:02.784282+0000","last_unstale":"2024-03-23T09:41:03.163373+0000","last_undegraded":"2024-03-23T09:41:03.163373+0000","last_fullsized":"2024-03-23T09:41:03.163373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146163+0000","last_change":"2024-03-23T09:41:03.168817+0000","last_active":"2024-03-23T09:41:04.146163+0000","last_peered":"2024-03-23T09:41:04.146163+0000","last_clean":"2024-03-23T09:41:04.146163+0000","last_became_active":"2024-03-23T09:41:03.168491+0000","last_became_peered":"2024-03-23T09:41:03.168491+0000","last_unstale":"2024-03-23T09:41:04.146163+0000","last_undegraded":"2024-03-23T09:41:04.146163+0000","last_fullsized":"2024-03-23T09:41:04.146163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:51:26.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.162936+0000","last_change":"2024-03-23T09:41:02.787278+0000","last_active":"2024-03-23T09:41:03.162936+0000","last_peered":"2024-03-23T09:41:03.162936+0000","last_clean":"2024-03-23T09:41:03.162936+0000","last_became_active":"2024-03-23T09:41:02.786760+0000","last_became_peered":"2024-03-23T09:41:02.786760+0000","last_unstale":"2024-03-23T09:41:03.162936+0000","last_undegraded":"2024-03-23T09:41:03.162936+0000","last_fullsized":"2024-03-23T09:41:03.162936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146844+0000","last_change":"2024-03-23T09:41:03.168702+0000","last_active":"2024-03-23T09:41:04.146844+0000","last_peered":"2024-03-23T09:41:04.146844+0000","last_clean":"2024-03-23T09:41:04.146844+0000","last_became_active":"2024-03-23T09:41:03.168425+0000","last_became_peered":"2024-03-23T09:41:03.168425+0000","last_unstale":"2024-03-23T09:41:04.146844+0000","last_undegraded":"2024-03-23T09:41:04.146844+0000","last_fullsized":"2024-03-23T09:41:04.146844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.158014+0000","last_change":"2024-03-23T09:41:02.785601+0000","last_active":"2024-03-23T09:41:03.158014+0000","last_peered":"2024-03-23T09:41:03.158014+0000","last_clean":"2024-03-23T09:41:03.158014+0000","last_became_active":"2024-03-23T09:41:02.785347+0000","last_became_peered":"2024-03-23T09:41:02.785347+0000","last_unstale":"2024-03-23T09:41:03.158014+0000","last_undegraded":"2024-03-23T09:41:03.158014+0000","last_fullsized":"2024-03-23T09:41:03.158014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.147217+0000","last_change":"2024-03-23T09:41:03.170126+0000","last_active":"2024-03-23T09:41:04.147217+0000","last_peered":"2024-03-23T09:41:04.147217+0000","last_clean":"2024-03-23T09:41:04.147217+0000","last_became_active":"2024-03-23T09:41:03.169191+0000","last_became_peered":"2024-03-23T09:41:03.169191+0000","last_unstale":"2024-03-23T09:41:04.147217+0000","last_undegraded":"2024-03-23T09:41:04.147217+0000","last_fullsized":"2024-03-23T09:41:04.147217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:33:41.605578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.168081+0000","last_change":"2024-03-23T09:41:02.781849+0000","last_active":"2024-03-23T09:41:03.168081+0000","last_peered":"2024-03-23T09:41:03.168081+0000","last_clean":"2024-03-23T09:41:03.168081+0000","last_became_active":"2024-03-23T09:41:02.781582+0000","last_became_peered":"2024-03-23T09:41:02.781582+0000","last_unstale":"2024-03-23T09:41:03.168081+0000","last_undegraded":"2024-03-23T09:41:03.168081+0000","last_fullsized":"2024-03-23T09:41:03.168081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155217+0000","last_change":"2024-03-23T09:41:03.156001+0000","last_active":"2024-03-23T09:41:04.155217+0000","last_peered":"2024-03-23T09:41:04.155217+0000","last_clean":"2024-03-23T09:41:04.155217+0000","last_became_active":"2024-03-23T09:41:03.155692+0000","last_became_peered":"2024-03-23T09:41:03.155692+0000","last_unstale":"2024-03-23T09:41:04.155217+0000","last_undegraded":"2024-03-23T09:41:04.155217+0000","last_fullsized":"2024-03-23T09:41:04.155217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:54:59.125701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.144394+0000","last_change":"2024-03-23T09:41:02.786345+0000","last_active":"2024-03-23T09:41:03.144394+0000","last_peered":"2024-03-23T09:41:03.144394+0000","last_clean":"2024-03-23T09:41:03.144394+0000","last_became_active":"2024-03-23T09:41:02.786070+0000","last_became_peered":"2024-03-23T09:41:02.786070+0000","last_unstale":"2024-03-23T09:41:03.144394+0000","last_undegraded":"2024-03-23T09:41:03.144394+0000","last_fullsized":"2024-03-23T09:41:03.144394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146543+0000","last_change":"2024-03-23T09:41:03.171409+0000","last_active":"2024-03-23T09:41:04.146543+0000","last_peered":"2024-03-23T09:41:04.146543+0000","last_clean":"2024-03-23T09:41:04.146543+0000","last_became_active":"2024-03-23T09:41:03.170992+0000","last_became_peered":"2024-03-23T09:41:03.170992+0000","last_unstale":"2024-03-23T09:41:04.146543+0000","last_undegraded":"2024-03-23T09:41:04.146543+0000","last_fullsized":"2024-03-23T09:41:04.146543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:40:45.654649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.159124+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:41:03.159124+0000","last_peered":"2024-03-23T09:41:03.159124+0000","last_clean":"2024-03-23T09:41:03.159124+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:41:03.159124+0000","last_undegraded":"2024-03-23T09:41:03.159124+0000","last_fullsized":"2024-03-23T09:41:03.159124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.149309+0000","last_change":"2024-03-23T09:41:02.780681+0000","last_active":"2024-03-23T09:41:03.149309+0000","last_peered":"2024-03-23T09:41:03.149309+0000","last_clean":"2024-03-23T09:41:03.149309+0000","last_became_active":"2024-03-23T09:41:02.780040+0000","last_became_peered":"2024-03-23T09:41:02.780040+0000","last_unstale":"2024-03-23T09:41:03.149309+0000","last_undegraded":"2024-03-23T09:41:03.149309+0000","last_fullsized":"2024-03-23T09:41:03.149309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666144+0000","last_change":"2024-03-23T09:41:03.159071+0000","last_active":"2024-03-23T09:41:04.666144+0000","last_peered":"2024-03-23T09:41:04.666144+0000","last_clean":"2024-03-23T09:41:04.666144+0000","last_became_active":"2024-03-23T09:41:03.158637+0000","last_became_peered":"2024-03-23T09:41:03.158637+0000","last_unstale":"2024-03-23T09:41:04.666144+0000","last_undegraded":"2024-03-23T09:41:04.666144+0000","last_fullsized":"2024-03-23T09:41:04.666144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:33:41.605578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.159038+0000","last_change":"2024-03-23T09:41:02.787080+0000","last_active":"2024-03-23T09:41:03.159038+0000","last_peered":"2024-03-23T09:41:03.159038+0000","last_clean":"2024-03-23T09:41:03.159038+0000","last_became_active":"2024-03-23T09:41:02.786573+0000","last_became_peered":"2024-03-23T09:41:02.786573+0000","last_unstale":"2024-03-23T09:41:03.159038+0000","last_undegraded":"2024-03-23T09:41:03.159038+0000","last_fullsized":"2024-03-23T09:41:03.159038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586381+0000","last_change":"2024-03-23T09:41:03.159217+0000","last_active":"2024-03-23T09:41:04.586381+0000","last_peered":"2024-03-23T09:41:04.586381+0000","last_clean":"2024-03-23T09:41:04.586381+0000","last_became_active":"2024-03-23T09:41:03.157721+0000","last_became_peered":"2024-03-23T09:41:03.157721+0000","last_unstale":"2024-03-23T09:41:04.586381+0000","last_undegraded":"2024-03-23T09:41:04.586381+0000","last_fullsized":"2024-03-23T09:41:04.586381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:28.007092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.145411+0000","last_change":"2024-03-23T09:41:02.785777+0000","last_active":"2024-03-23T09:41:03.145411+0000","last_peered":"2024-03-23T09:41:03.145411+0000","last_clean":"2024-03-23T09:41:03.145411+0000","last_became_active":"2024-03-23T09:41:02.784961+0000","last_became_peered":"2024-03-23T09:41:02.784961+0000","last_unstale":"2024-03-23T09:41:03.145411+0000","last_undegraded":"2024-03-23T09:41:03.145411+0000","last_fullsized":"2024-03-23T09:41:03.145411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142138+0000","last_change":"2024-03-23T09:41:03.146739+0000","last_active":"2024-03-23T09:41:04.142138+0000","last_peered":"2024-03-23T09:41:04.142138+0000","last_clean":"2024-03-23T09:41:04.142138+0000","last_became_active":"2024-03-23T09:41:03.146310+0000","last_became_peered":"2024-03-23T09:41:03.146310+0000","last_unstale":"2024-03-23T09:41:04.142138+0000","last_undegraded":"2024-03-23T09:41:04.142138+0000","last_fullsized":"2024-03-23T09:41:04.142138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:10.603669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.150027+0000","last_change":"2024-03-23T09:41:02.789860+0000","last_active":"2024-03-23T09:41:03.150027+0000","last_peered":"2024-03-23T09:41:03.150027+0000","last_clean":"2024-03-23T09:41:03.150027+0000","last_became_active":"2024-03-23T09:41:02.787110+0000","last_became_peered":"2024-03-23T09:41:02.787110+0000","last_unstale":"2024-03-23T09:41:03.150027+0000","last_undegraded":"2024-03-23T09:41:03.150027+0000","last_fullsized":"2024-03-23T09:41:03.150027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146588+0000","last_change":"2024-03-23T09:41:03.162623+0000","last_active":"2024-03-23T09:41:04.146588+0000","last_peered":"2024-03-23T09:41:04.146588+0000","last_clean":"2024-03-23T09:41:04.146588+0000","last_became_active":"2024-03-23T09:41:03.162237+0000","last_became_peered":"2024-03-23T09:41:03.162237+0000","last_unstale":"2024-03-23T09:41:04.146588+0000","last_undegraded":"2024-03-23T09:41:04.146588+0000","last_fullsized":"2024-03-23T09:41:04.146588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:49.353786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.141570+0000","last_change":"2024-03-23T09:41:02.785671+0000","last_active":"2024-03-23T09:41:03.141570+0000","last_peered":"2024-03-23T09:41:03.141570+0000","last_clean":"2024-03-23T09:41:03.141570+0000","last_became_active":"2024-03-23T09:41:02.785397+0000","last_became_peered":"2024-03-23T09:41:02.785397+0000","last_unstale":"2024-03-23T09:41:03.141570+0000","last_undegraded":"2024-03-23T09:41:03.141570+0000","last_fullsized":"2024-03-23T09:41:03.141570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155275+0000","last_change":"2024-03-23T09:41:03.158343+0000","last_active":"2024-03-23T09:41:04.155275+0000","last_peered":"2024-03-23T09:41:04.155275+0000","last_clean":"2024-03-23T09:41:04.155275+0000","last_became_active":"2024-03-23T09:41:03.158079+0000","last_became_peered":"2024-03-23T09:41:03.158079+0000","last_unstale":"2024-03-23T09:41:04.155275+0000","last_undegraded":"2024-03-23T09:41:04.155275+0000","last_fullsized":"2024-03-23T09:41:04.155275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:46:38.667131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.161554+0000","last_change":"2024-03-23T09:41:02.788728+0000","last_active":"2024-03-23T09:41:03.161554+0000","last_peered":"2024-03-23T09:41:03.161554+0000","last_clean":"2024-03-23T09:41:03.161554+0000","last_became_active":"2024-03-23T09:41:02.788461+0000","last_became_peered":"2024-03-23T09:41:02.788461+0000","last_unstale":"2024-03-23T09:41:03.161554+0000","last_undegraded":"2024-03-23T09:41:03.161554+0000","last_fullsized":"2024-03-23T09:41:03.161554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142267+0000","last_change":"2024-03-23T09:41:03.147531+0000","last_active":"2024-03-23T09:41:04.142267+0000","last_peered":"2024-03-23T09:41:04.142267+0000","last_clean":"2024-03-23T09:41:04.142267+0000","last_became_active":"2024-03-23T09:41:03.147207+0000","last_became_peered":"2024-03-23T09:41:03.147207+0000","last_unstale":"2024-03-23T09:41:04.142267+0000","last_undegraded":"2024-03-23T09:41:04.142267+0000","last_fullsized":"2024-03-23T09:41:04.142267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:42:19.838774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142336+0000","last_change":"2024-03-23T09:41:02.789060+0000","last_active":"2024-03-23T09:41:04.142336+0000","last_peered":"2024-03-23T09:41:04.142336+0000","last_clean":"2024-03-23T09:41:04.142336+0000","last_became_active":"2024-03-23T09:41:02.788772+0000","last_became_peered":"2024-03-23T09:41:02.788772+0000","last_unstale":"2024-03-23T09:41:04.142336+0000","last_undegraded":"2024-03-23T09:41:04.142336+0000","last_fullsized":"2024-03-23T09:41:04.142336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666270+0000","last_change":"2024-03-23T09:41:03.169569+0000","last_active":"2024-03-23T09:41:04.666270+0000","last_peered":"2024-03-23T09:41:04.666270+0000","last_clean":"2024-03-23T09:41:04.666270+0000","last_became_active":"2024-03-23T09:41:03.169275+0000","last_became_peered":"2024-03-23T09:41:03.169275+0000","last_unstale":"2024-03-23T09:41:04.666270+0000","last_undegraded":"2024-03-23T09:41:04.666270+0000","last_fullsized":"2024-03-23T09:41:04.666270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:30:48.469383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.168962+0000","last_change":"2024-03-23T09:41:02.788239+0000","last_active":"2024-03-23T09:41:03.168962+0000","last_peered":"2024-03-23T09:41:03.168962+0000","last_clean":"2024-03-23T09:41:03.168962+0000","last_became_active":"2024-03-23T09:41:02.787874+0000","last_became_peered":"2024-03-23T09:41:02.787874+0000","last_unstale":"2024-03-23T09:41:03.168962+0000","last_undegraded":"2024-03-23T09:41:03.168962+0000","last_fullsized":"2024-03-23T09:41:03.168962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586705+0000","last_change":"2024-03-23T09:41:03.156480+0000","last_active":"2024-03-23T09:41:04.586705+0000","last_peered":"2024-03-23T09:41:04.586705+0000","last_clean":"2024-03-23T09:41:04.586705+0000","last_became_active":"2024-03-23T09:41:03.155683+0000","last_became_peered":"2024-03-23T09:41:03.155683+0000","last_unstale":"2024-03-23T09:41:04.586705+0000","last_undegraded":"2024-03-23T09:41:04.586705+0000","last_fullsized":"2024-03-23T09:41:04.586705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:09.777921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.160846+0000","last_change":"2024-03-23T09:41:02.789413+0000","last_active":"2024-03-23T09:41:03.160846+0000","last_peered":"2024-03-23T09:41:03.160846+0000","last_clean":"2024-03-23T09:41:03.160846+0000","last_became_active":"2024-03-23T09:41:02.788397+0000","last_became_peered":"2024-03-23T09:41:02.788397+0000","last_unstale":"2024-03-23T09:41:03.160846+0000","last_undegraded":"2024-03-23T09:41:03.160846+0000","last_fullsized":"2024-03-23T09:41:03.160846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146940+0000","last_change":"2024-03-23T09:41:03.148447+0000","last_active":"2024-03-23T09:41:04.146940+0000","last_peered":"2024-03-23T09:41:04.146940+0000","last_clean":"2024-03-23T09:41:04.146940+0000","last_became_active":"2024-03-23T09:41:03.148194+0000","last_became_peered":"2024-03-23T09:41:03.148194+0000","last_unstale":"2024-03-23T09:41:04.146940+0000","last_undegraded":"2024-03-23T09:41:04.146940+0000","last_fullsized":"2024-03-23T09:41:04.146940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:21.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.141503+0000","last_change":"2024-03-23T09:41:02.782038+0000","last_active":"2024-03-23T09:41:03.141503+0000","last_peered":"2024-03-23T09:41:03.141503+0000","last_clean":"2024-03-23T09:41:03.141503+0000","last_became_active":"2024-03-23T09:41:02.781322+0000","last_became_peered":"2024-03-23T09:41:02.781322+0000","last_unstale":"2024-03-23T09:41:03.141503+0000","last_undegraded":"2024-03-23T09:41:03.141503+0000","last_fullsized":"2024-03-23T09:41:03.141503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665970+0000","last_change":"2024-03-23T09:41:03.148500+0000","last_active":"2024-03-23T09:41:04.665970+0000","last_peered":"2024-03-23T09:41:04.665970+0000","last_clean":"2024-03-23T09:41:04.665970+0000","last_became_active":"2024-03-23T09:41:03.148263+0000","last_became_peered":"2024-03-23T09:41:03.148263+0000","last_unstale":"2024-03-23T09:41:04.665970+0000","last_undegraded":"2024-03-23T09:41:04.665970+0000","last_fullsized":"2024-03-23T09:41:04.665970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:51:26.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.141242+0000","last_change":"2024-03-23T09:41:02.785162+0000","last_active":"2024-03-23T09:41:03.141242+0000","last_peered":"2024-03-23T09:41:03.141242+0000","last_clean":"2024-03-23T09:41:03.141242+0000","last_became_active":"2024-03-23T09:41:02.784744+0000","last_became_peered":"2024-03-23T09:41:02.784744+0000","last_unstale":"2024-03-23T09:41:03.141242+0000","last_undegraded":"2024-03-23T09:41:03.141242+0000","last_fullsized":"2024-03-23T09:41:03.141242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666237+0000","last_change":"2024-03-23T09:41:03.160780+0000","last_active":"2024-03-23T09:41:04.666237+0000","last_peered":"2024-03-23T09:41:04.666237+0000","last_clean":"2024-03-23T09:41:04.666237+0000","last_became_active":"2024-03-23T09:41:03.159686+0000","last_became_peered":"2024-03-23T09:41:03.159686+0000","last_unstale":"2024-03-23T09:41:04.666237+0000","last_undegraded":"2024-03-23T09:41:04.666237+0000","last_fullsized":"2024-03-23T09:41:04.666237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:16.974803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.170285+0000","last_change":"2024-03-23T09:41:02.786708+0000","last_active":"2024-03-23T09:41:03.170285+0000","last_peered":"2024-03-23T09:41:03.170285+0000","last_clean":"2024-03-23T09:41:03.170285+0000","last_became_active":"2024-03-23T09:41:02.786306+0000","last_became_peered":"2024-03-23T09:41:02.786306+0000","last_unstale":"2024-03-23T09:41:03.170285+0000","last_undegraded":"2024-03-23T09:41:03.170285+0000","last_fullsized":"2024-03-23T09:41:03.170285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666270+0000","last_change":"2024-03-23T09:41:03.157669+0000","last_active":"2024-03-23T09:41:04.666270+0000","last_peered":"2024-03-23T09:41:04.666270+0000","last_clean":"2024-03-23T09:41:04.666270+0000","last_became_active":"2024-03-23T09:41:03.157206+0000","last_became_peered":"2024-03-23T09:41:03.157206+0000","last_unstale":"2024-03-23T09:41:04.666270+0000","last_undegraded":"2024-03-23T09:41:04.666270+0000","last_fullsized":"2024-03-23T09:41:04.666270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:53:01.847138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155328+0000","last_change":"2024-03-23T09:41:03.158925+0000","last_active":"2024-03-23T09:41:04.155328+0000","last_peered":"2024-03-23T09:41:04.155328+0000","last_clean":"2024-03-23T09:41:04.155328+0000","last_became_active":"2024-03-23T09:41:03.158643+0000","last_became_peered":"2024-03-23T09:41:03.158643+0000","last_unstale":"2024-03-23T09:41:04.155328+0000","last_undegraded":"2024-03-23T09:41:04.155328+0000","last_fullsized":"2024-03-23T09:41:04.155328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:32:17.579489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.170957+0000","last_change":"2024-03-23T09:41:02.786654+0000","last_active":"2024-03-23T09:41:03.170957+0000","last_peered":"2024-03-23T09:41:03.170957+0000","last_clean":"2024-03-23T09:41:03.170957+0000","last_became_active":"2024-03-23T09:41:02.786387+0000","last_became_peered":"2024-03-23T09:41:02.786387+0000","last_unstale":"2024-03-23T09:41:03.170957+0000","last_undegraded":"2024-03-23T09:41:03.170957+0000","last_fullsized":"2024-03-23T09:41:03.170957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155110+0000","last_change":"2024-03-23T09:41:03.158846+0000","last_active":"2024-03-23T09:41:04.155110+0000","last_peered":"2024-03-23T09:41:04.155110+0000","last_clean":"2024-03-23T09:41:04.155110+0000","last_became_active":"2024-03-23T09:41:03.158433+0000","last_became_peered":"2024-03-23T09:41:03.158433+0000","last_unstale":"2024-03-23T09:41:04.155110+0000","last_undegraded":"2024-03-23T09:41:04.155110+0000","last_fullsized":"2024-03-23T09:41:04.155110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:11:48.328619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.168704+0000","last_change":"2024-03-23T09:41:02.788509+0000","last_active":"2024-03-23T09:41:03.168704+0000","last_peered":"2024-03-23T09:41:03.168704+0000","last_clean":"2024-03-23T09:41:03.168704+0000","last_became_active":"2024-03-23T09:41:02.788072+0000","last_became_peered":"2024-03-23T09:41:02.788072+0000","last_unstale":"2024-03-23T09:41:03.168704+0000","last_undegraded":"2024-03-23T09:41:03.168704+0000","last_fullsized":"2024-03-23T09:41:03.168704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:11.278420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586632+0000","last_change":"2024-03-23T09:41:03.159417+0000","last_active":"2024-03-23T09:41:04.586632+0000","last_peered":"2024-03-23T09:41:04.586632+0000","last_clean":"2024-03-23T09:41:04.586632+0000","last_became_active":"2024-03-23T09:41:03.159146+0000","last_became_peered":"2024-03-23T09:41:03.159146+0000","last_unstale":"2024-03-23T09:41:04.586632+0000","last_undegraded":"2024-03-23T09:41:04.586632+0000","last_fullsized":"2024-03-23T09:41:04.586632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:10.603669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.163045+0000","last_change":"2024-03-23T09:41:02.789237+0000","last_active":"2024-03-23T09:41:03.163045+0000","last_peered":"2024-03-23T09:41:03.163045+0000","last_clean":"2024-03-23T09:41:03.163045+0000","last_became_active":"2024-03-23T09:41:02.788896+0000","last_became_peered":"2024-03-23T09:41:02.788896+0000","last_unstale":"2024-03-23T09:41:03.163045+0000","last_undegraded":"2024-03-23T09:41:03.163045+0000","last_fullsized":"2024-03-23T09:41:03.163045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666186+0000","last_change":"2024-03-23T09:41:03.163326+0000","last_active":"2024-03-23T09:41:04.666186+0000","last_peered":"2024-03-23T09:41:04.666186+0000","last_clean":"2024-03-23T09:41:04.666186+0000","last_became_active":"2024-03-23T09:41:03.163069+0000","last_became_peered":"2024-03-23T09:41:03.163069+0000","last_unstale":"2024-03-23T09:41:04.666186+0000","last_undegraded":"2024-03-23T09:41:04.666186+0000","last_fullsized":"2024-03-23T09:41:04.666186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.171686+0000","last_change":"2024-03-23T09:41:02.787414+0000","last_active":"2024-03-23T09:41:03.171686+0000","last_peered":"2024-03-23T09:41:03.171686+0000","last_clean":"2024-03-23T09:41:03.171686+0000","last_became_active":"2024-03-23T09:41:02.787074+0000","last_became_peered":"2024-03-23T09:41:02.787074+0000","last_unstale":"2024-03-23T09:41:03.171686+0000","last_undegraded":"2024-03-23T09:41:03.171686+0000","last_fullsized":"2024-03-23T09:41:03.171686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:02:19.284328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28392,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343448,"statfs":{"total":96636764160,"available":96607690752,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":76514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":819200,"data_stored":665956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:41:10.886 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T09:41:10.886 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-23T09:41:10.887 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-23T09:41:10.887 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-23T09:41:10.887 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-23T09:41:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.021+0000 7fd70c15c640 1 Processor -- start 2024-03-23T09:41:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.021+0000 7fd70c15c640 1 -- start start 2024-03-23T09:41:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd70c15c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd70415e110 0x7fd7040b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd70c15c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd7040cee40 0x7fd7040c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd70c15c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704098a00 0x7fd704098e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd70c15c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7040993d0 con 0x7fd70415e110 2024-03-23T09:41:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd70c15c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7040cf390 con 0x7fd7040cee40 2024-03-23T09:41:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd70c15c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7040cf670 con 0x7fd704098a00 2024-03-23T09:41:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd709ef8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd70415e110 0x7fd7040b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd709ef8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd70415e110 0x7fd7040b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45934/0 (socket says 172.21.15.47:45934) 2024-03-23T09:41:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd709ef8640 1 -- 172.21.15.47:0/910891464 learned_addr learned my addr 172.21.15.47:0/910891464 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd7096f7640 1 --2- 172.21.15.47:0/910891464 >> v2:172.21.15.175:3300/0 conn(0x7fd7040cee40 0x7fd7040c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd709ef8640 1 -- 172.21.15.47:0/910891464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704098a00 msgr2=0x7fd704098e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd709ef8640 1 --2- 172.21.15.47:0/910891464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704098a00 0x7fd704098e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd709ef8640 1 -- 172.21.15.47:0/910891464 >> v2:172.21.15.175:3300/0 conn(0x7fd7040cee40 msgr2=0x7fd7040c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd709ef8640 1 --2- 172.21.15.47:0/910891464 >> v2:172.21.15.175:3300/0 conn(0x7fd7040cee40 0x7fd7040c41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.022+0000 7fd709ef8640 1 -- 172.21.15.47:0/910891464 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7040cb4b0 con 0x7fd70415e110 2024-03-23T09:41:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.023+0000 7fd709ef8640 1 --2- 172.21.15.47:0/910891464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd70415e110 0x7fd7040b1640 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7040b6700 tx=0x7fd6f402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8896ebda365312aa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.023+0000 7fd708ef6640 1 -- 172.21.15.47:0/910891464 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4031860 con 0x7fd70415e110 2024-03-23T09:41:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.023+0000 7fd708ef6640 1 -- 172.21.15.47:0/910891464 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f40319c0 con 0x7fd70415e110 2024-03-23T09:41:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.023+0000 7fd708ef6640 1 -- 172.21.15.47:0/910891464 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4031d50 con 0x7fd70415e110 2024-03-23T09:41:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.023+0000 7fd70c15c640 1 -- 172.21.15.47:0/910891464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd70415e110 msgr2=0x7fd7040b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.023+0000 7fd70c15c640 1 --2- 172.21.15.47:0/910891464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd70415e110 0x7fd7040b1640 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7040b6700 tx=0x7fd6f402b900 comp rx=0 tx=0).stop 2024-03-23T09:41:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.023+0000 7fd70c15c640 1 -- 172.21.15.47:0/910891464 shutdown_connections 2024-03-23T09:41:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.023+0000 7fd70c15c640 1 --2- 172.21.15.47:0/910891464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704098a00 0x7fd704098e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.023+0000 7fd70c15c640 1 --2- 172.21.15.47:0/910891464 >> v2:172.21.15.175:3300/0 conn(0x7fd7040cee40 0x7fd7040c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.023+0000 7fd70c15c640 1 --2- 172.21.15.47:0/910891464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd70415e110 0x7fd7040b1640 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.023+0000 7fd70c15c640 1 -- 172.21.15.47:0/910891464 >> 172.21.15.47:0/910891464 conn(0x7fd7040b2140 msgr2=0x7fd7040b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.023+0000 7fd70c15c640 1 -- 172.21.15.47:0/910891464 shutdown_connections 2024-03-23T09:41:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.023+0000 7fd70c15c640 1 -- 172.21.15.47:0/910891464 wait complete. 2024-03-23T09:41:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.025+0000 7fd70c15c640 1 Processor -- start 2024-03-23T09:41:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.025+0000 7fd70c15c640 1 -- start start 2024-03-23T09:41:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.025+0000 7fd70c15c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704098a00 0x7fd7041f0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.025+0000 7fd70c15c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7040cee40 0x7fd7040b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.025+0000 7fd70c15c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd70415e110 0x7fd704156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.025+0000 7fd70c15c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7040c78e0 con 0x7fd70415e110 2024-03-23T09:41:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.025+0000 7fd70c15c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7041f13a0 con 0x7fd7040cee40 2024-03-23T09:41:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.025+0000 7fd70c15c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7040b64b0 con 0x7fd704098a00 2024-03-23T09:41:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.025+0000 7fd70a6f9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd70415e110 0x7fd704156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.025+0000 7fd70a6f9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd70415e110 0x7fd704156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45948/0 (socket says 172.21.15.47:45948) 2024-03-23T09:41:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.025+0000 7fd70a6f9640 1 -- 172.21.15.47:0/730096360 learned_addr learned my addr 172.21.15.47:0/730096360 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd7096f7640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7040cee40 0x7fd7040b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd70a6f9640 1 -- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704098a00 msgr2=0x7fd7041f0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd709ef8640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704098a00 0x7fd7041f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd70a6f9640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704098a00 0x7fd7041f0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd70a6f9640 1 -- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7040cee40 msgr2=0x7fd7040b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd70a6f9640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7040cee40 0x7fd7040b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd70a6f9640 1 -- 172.21.15.47:0/730096360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f40094d0 con 0x7fd70415e110 2024-03-23T09:41:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd7096f7640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7040cee40 0x7fd7040b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd70a6f9640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd70415e110 0x7fd704156ac0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00b9a0 tx=0x7fd6fc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd7027fc640 1 -- 172.21.15.47:0/730096360 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc00cba0 con 0x7fd70415e110 2024-03-23T09:41:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd7027fc640 1 -- 172.21.15.47:0/730096360 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6fc00cd20 con 0x7fd70415e110 2024-03-23T09:41:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd70c15c640 1 -- 172.21.15.47:0/730096360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7041f0b00 con 0x7fd70415e110 2024-03-23T09:41:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd7027fc640 1 -- 172.21.15.47:0/730096360 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc014680 con 0x7fd70415e110 2024-03-23T09:41:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.026+0000 7fd70c15c640 1 -- 172.21.15.47:0/730096360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7041f10c0 con 0x7fd70415e110 2024-03-23T09:41:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.027+0000 7fd70c15c640 1 -- 172.21.15.47:0/730096360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7040d02d0 con 0x7fd70415e110 2024-03-23T09:41:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.027+0000 7fd7027fc640 1 -- 172.21.15.47:0/730096360 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd6fc01d3b0 con 0x7fd70415e110 2024-03-23T09:41:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.028+0000 7fd7027fc640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd6f006ac00 0x7fd6f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.028+0000 7fd7027fc640 1 -- 172.21.15.47:0/730096360 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6fc089260 con 0x7fd70415e110 2024-03-23T09:41:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.028+0000 7fd709ef8640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd6f006ac00 0x7fd6f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.029+0000 7fd709ef8640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd6f006ac00 0x7fd6f006d0c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4036000 tx=0x7fd6f403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.032+0000 7fd7027fc640 1 -- 172.21.15.47:0/730096360 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd6fc0544d0 con 0x7fd70415e110 2024-03-23T09:41:11.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.263+0000 7fd70c15c640 1 -- 172.21.15.47:0/730096360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd7040c7a90 con 0x7fd70415e110 2024-03-23T09:41:11.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.266+0000 7fd7027fc640 1 -- 172.21.15.47:0/730096360 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170932 (secure 0 0 0) 0x7fd6fc057af0 con 0x7fd70415e110 2024-03-23T09:41:11.270 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.269+0000 7fd70c15c640 1 -- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd6f006ac00 msgr2=0x7fd6f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.269+0000 7fd70c15c640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd6f006ac00 0x7fd6f006d0c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4036000 tx=0x7fd6f403c040 comp rx=0 tx=0).stop 2024-03-23T09:41:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.269+0000 7fd70c15c640 1 -- 172.21.15.47:0/730096360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd70415e110 msgr2=0x7fd704156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.269+0000 7fd70c15c640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd70415e110 0x7fd704156ac0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00b9a0 tx=0x7fd6fc00be60 comp rx=0 tx=0).stop 2024-03-23T09:41:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.269+0000 7fd70c15c640 1 -- 172.21.15.47:0/730096360 shutdown_connections 2024-03-23T09:41:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.269+0000 7fd70c15c640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd6f006ac00 0x7fd6f006d0c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.269+0000 7fd70c15c640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd70415e110 0x7fd704156ac0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.269+0000 7fd70c15c640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7040cee40 0x7fd7040b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.269+0000 7fd70c15c640 1 --2- 172.21.15.47:0/730096360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704098a00 0x7fd7041f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.269+0000 7fd70c15c640 1 -- 172.21.15.47:0/730096360 >> 172.21.15.47:0/730096360 conn(0x7fd7040b2140 msgr2=0x7fd7040cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:11.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.269+0000 7fd70c15c640 1 -- 172.21.15.47:0/730096360 shutdown_connections 2024-03-23T09:41:11.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.269+0000 7fd70c15c640 1 -- 172.21.15.47:0/730096360 wait complete. 2024-03-23T09:41:11.279 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":5,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":37590},{"type":"v1","addr":"172.21.15.175:6833","nonce":37590}]},"active_addr":"172.21.15.175:6833/37590","active_change":"2024-03-23T09:40:50.328604+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4098,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":726881713}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":142292507}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":3861548356}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2672890158}]}]} 2024-03-23T09:41:11.286 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-23T09:41:11.286 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-23T09:41:11.286 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:11.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.386+0000 7f1da80e9640 1 Processor -- start 2024-03-23T09:41:11.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.387+0000 7f1da80e9640 1 -- start start 2024-03-23T09:41:11.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.387+0000 7f1da80e9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00d09e0 0x7f1da00d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.387+0000 7f1da80e9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da015d0a0 con 0x7f1da00cf830 2024-03-23T09:41:11.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.387+0000 7f1da80e9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da00d14a0 con 0x7f1da00b5a00 2024-03-23T09:41:11.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.387+0000 7f1da80e9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00d2120 con 0x7f1da00d09e0 2024-03-23T09:41:11.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.387+0000 7f1da5e85640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1da00cf830 0x7f1da00cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57232/0 (socket says 172.21.15.47:57232) 2024-03-23T09:41:11.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.387+0000 7f1da5e85640 1 -- 172.21.15.47:0/1249425695 learned_addr learned my addr 172.21.15.47:0/1249425695 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da5684640 1 --2- 172.21.15.47:0/1249425695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00d09e0 0x7f1da00d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517250671 0 0) 0x7f1d9c001110 con 0x7f1da00cf830 2024-03-23T09:41:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d8c003520 con 0x7f1da00cf830 2024-03-23T09:41:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845779788 0 0) 0x7f1da00d14a0 con 0x7f1da00b5a00 2024-03-23T09:41:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d9c001110 con 0x7f1da00b5a00 2024-03-23T09:41:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3182188592 0 0) 0x7f1d8c003520 con 0x7f1da00cf830 2024-03-23T09:41:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00d09e0 msgr2=0x7f1da00d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 --2- 172.21.15.47:0/1249425695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00d09e0 0x7f1da00d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 >> v1:172.21.15.175:6789/0 conn(0x7f1da00b5a00 legacy=0x7f1da00b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da00d1780 con 0x7f1da00cf830 2024-03-23T09:41:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f1d9c001800 con 0x7f1da00cf830 2024-03-23T09:41:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 >> v1:172.21.15.47:6789/0 conn(0x7f1da00cf830 legacy=0x7f1da00cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 --2- 172.21.15.47:0/1249425695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00d09e0 0x7f1d8c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 --2- 172.21.15.47:0/1249425695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d8c004090 0x7f1d8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 --2- 172.21.15.47:0/1249425695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d8c004d00 0x7f1d8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00d2120 con 0x7f1d8c004d00 2024-03-23T09:41:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c00ba60 con 0x7f1d8c004090 2024-03-23T09:41:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c00bc20 con 0x7f1da00d09e0 2024-03-23T09:41:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da6686640 1 --2- 172.21.15.47:0/1249425695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d8c004d00 0x7f1d8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da5684640 1 --2- 172.21.15.47:0/1249425695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d8c004090 0x7f1d8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.388+0000 7f1da5e85640 1 --2- 172.21.15.47:0/1249425695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00d09e0 0x7f1d8c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.389+0000 7f1da6686640 1 -- 172.21.15.47:0/1249425695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00d09e0 msgr2=0x7f1d8c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.389+0000 7f1da6686640 1 --2- 172.21.15.47:0/1249425695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00d09e0 0x7f1d8c003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.389+0000 7f1da6686640 1 -- 172.21.15.47:0/1249425695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d8c004090 msgr2=0x7f1d8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.389+0000 7f1da6686640 1 --2- 172.21.15.47:0/1249425695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d8c004090 0x7f1d8c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.389+0000 7f1da6686640 1 -- 172.21.15.47:0/1249425695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c003520 con 0x7f1d8c004d00 2024-03-23T09:41:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.389+0000 7f1da6686640 1 --2- 172.21.15.47:0/1249425695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d8c004d00 0x7f1d8c005180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d98001fd0 tx=0x7f1d8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c558a2df670e09b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.389+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98003b70 con 0x7f1d8c004d00 2024-03-23T09:41:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.389+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d98003cd0 con 0x7f1d8c004d00 2024-03-23T09:41:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.389+0000 7f1da4e83640 1 -- 172.21.15.47:0/1249425695 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d980052f0 con 0x7f1d8c004d00 2024-03-23T09:41:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.390+0000 7f1da80e9640 1 -- 172.21.15.47:0/1249425695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d8c004d00 msgr2=0x7f1d8c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.390+0000 7f1da80e9640 1 --2- 172.21.15.47:0/1249425695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d8c004d00 0x7f1d8c005180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d98001fd0 tx=0x7f1d8c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.390+0000 7f1da6686640 1 -- 172.21.15.47:0/1249425695 reap_dead start 2024-03-23T09:41:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.390+0000 7f1da80e9640 1 -- 172.21.15.47:0/1249425695 shutdown_connections 2024-03-23T09:41:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.390+0000 7f1da80e9640 1 -- 172.21.15.47:0/1249425695 >> 172.21.15.47:0/1249425695 conn(0x7f1da0065dc0 msgr2=0x7f1da00661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.390+0000 7f1da80e9640 1 -- 172.21.15.47:0/1249425695 shutdown_connections 2024-03-23T09:41:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.390+0000 7f1da80e9640 1 -- 172.21.15.47:0/1249425695 wait complete. 2024-03-23T09:41:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.391+0000 7f1da80e9640 1 Processor -- start 2024-03-23T09:41:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.391+0000 7f1da80e9640 1 -- start start 2024-03-23T09:41:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da80e9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d8c004d00 0x7f1da014ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da80e9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00b6910 0x7f1da00b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da80e9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1da00b2c40 0x7f1da00b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da80e9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00b72b0 con 0x7f1d8c004d00 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da80e9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da014f810 con 0x7f1da00b2c40 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da80e9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da01f30c0 con 0x7f1da00b6910 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da5e85640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d8c004d00 0x7f1da014ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da5e85640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d8c004d00 0x7f1da014ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45968/0 (socket says 172.21.15.47:45968) 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da5e85640 1 -- 172.21.15.47:0/538489705 learned_addr learned my addr 172.21.15.47:0/538489705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da6686640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1da00b2c40 0x7f1da00b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da5684640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00b6910 0x7f1da00b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da5e85640 1 -- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00b6910 msgr2=0x7f1da00b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da5e85640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00b6910 0x7f1da00b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da5e85640 1 -- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1da00b2c40 msgr2=0x7f1da00b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da5e85640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1da00b2c40 0x7f1da00b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da5e85640 1 -- 172.21.15.47:0/538489705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da01f33a0 con 0x7f1d8c004d00 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da5684640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00b6910 0x7f1da00b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.392+0000 7f1da5e85640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d8c004d00 0x7f1da014ecf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c007320 tx=0x7f1d9c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.393+0000 7f1d967fc640 1 -- 172.21.15.47:0/538489705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c00fa20 con 0x7f1d8c004d00 2024-03-23T09:41:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.393+0000 7f1d967fc640 1 -- 172.21.15.47:0/538489705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c00fbc0 con 0x7f1d8c004d00 2024-03-23T09:41:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.393+0000 7f1da80e9640 1 -- 172.21.15.47:0/538489705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da01f3680 con 0x7f1d8c004d00 2024-03-23T09:41:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.393+0000 7f1d967fc640 1 -- 172.21.15.47:0/538489705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0133f0 con 0x7f1d8c004d00 2024-03-23T09:41:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.393+0000 7f1da80e9640 1 -- 172.21.15.47:0/538489705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da01f3c40 con 0x7f1d8c004d00 2024-03-23T09:41:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.393+0000 7f1da80e9640 1 -- 172.21.15.47:0/538489705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da00672f0 con 0x7f1d8c004d00 2024-03-23T09:41:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.394+0000 7f1d967fc640 1 -- 172.21.15.47:0/538489705 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1d9c013550 con 0x7f1d8c004d00 2024-03-23T09:41:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.394+0000 7f1d967fc640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1d8406ac00 0x7f1d8406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.395+0000 7f1d967fc640 1 -- 172.21.15.47:0/538489705 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d9c089440 con 0x7f1d8c004d00 2024-03-23T09:41:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.395+0000 7f1da5684640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1d8406ac00 0x7f1d8406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.395+0000 7f1da5684640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1d8406ac00 0x7f1d8406d0c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1da00b6340 tx=0x7f1d90008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.399+0000 7f1d967fc640 1 -- 172.21.15.47:0/538489705 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1d9c054620 con 0x7f1d8c004d00 2024-03-23T09:41:11.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.578+0000 7f1da80e9640 1 -- 172.21.15.47:0/538489705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1da015d0a0 con 0x7f1d8c004d00 2024-03-23T09:41:11.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.579+0000 7f1d967fc640 1 -- 172.21.15.47:0/538489705 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f1da015d0a0 con 0x7f1d8c004d00 2024-03-23T09:41:11.578 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:11.578 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:11.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.580+0000 7f1da80e9640 1 -- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1d8406ac00 msgr2=0x7f1d8406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.580+0000 7f1da80e9640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1d8406ac00 0x7f1d8406d0c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1da00b6340 tx=0x7f1d90008040 comp rx=0 tx=0).stop 2024-03-23T09:41:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.580+0000 7f1da80e9640 1 -- 172.21.15.47:0/538489705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d8c004d00 msgr2=0x7f1da014ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.580+0000 7f1da80e9640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d8c004d00 0x7f1da014ecf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c007320 tx=0x7f1d9c0077e0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.580+0000 7f1da80e9640 1 -- 172.21.15.47:0/538489705 shutdown_connections 2024-03-23T09:41:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.580+0000 7f1da80e9640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1d8406ac00 0x7f1d8406d0c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.580+0000 7f1da80e9640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1da00b2c40 0x7f1da00b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.581+0000 7f1da80e9640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da00b6910 0x7f1da00b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.581+0000 7f1da80e9640 1 --2- 172.21.15.47:0/538489705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d8c004d00 0x7f1da014ecf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.581+0000 7f1da80e9640 1 -- 172.21.15.47:0/538489705 >> 172.21.15.47:0/538489705 conn(0x7f1da0065dc0 msgr2=0x7f1da00b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.581+0000 7f1da80e9640 1 -- 172.21.15.47:0/538489705 shutdown_connections 2024-03-23T09:41:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.581+0000 7f1da80e9640 1 -- 172.21.15.47:0/538489705 wait complete. 2024-03-23T09:41:11.591 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-23T09:41:11.591 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.692+0000 7f91c97bf640 1 Processor -- start 2024-03-23T09:41:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.692+0000 7f91c97bf640 1 -- start start 2024-03-23T09:41:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c97bf640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91c40cde10 0x7f91c4167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c97bf640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c40d11e0 con 0x7f91c415ca90 2024-03-23T09:41:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c97bf640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c40d2680 con 0x7f91c40b8350 2024-03-23T09:41:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c97bf640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c4160f60 con 0x7f91c40cde10 2024-03-23T09:41:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c2ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f91c415ca90 0x7f91c4160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57240/0 (socket says 172.21.15.47:57240) 2024-03-23T09:41:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c2ffd640 1 -- 172.21.15.47:0/3075670006 learned_addr learned my addr 172.21.15.47:0/3075670006 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969198207 0 0) 0x7f91c40d11e0 con 0x7f91c415ca90 2024-03-23T09:41:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c37fe640 1 --2- 172.21.15.47:0/3075670006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91c40cde10 0x7f91c4167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b0003520 con 0x7f91c415ca90 2024-03-23T09:41:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3463908081 0 0) 0x7f91b0003520 con 0x7f91c415ca90 2024-03-23T09:41:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91c40cde10 msgr2=0x7f91c4167aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c1ffb640 1 --2- 172.21.15.47:0/3075670006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91c40cde10 0x7f91c4167aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 >> v1:172.21.15.175:6789/0 conn(0x7f91c40b8350 legacy=0x7f91c40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.693+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c4161240 con 0x7f91c415ca90 2024-03-23T09:41:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f91bc001670 con 0x7f91c415ca90 2024-03-23T09:41:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 >> v1:172.21.15.47:6789/0 conn(0x7f91c415ca90 legacy=0x7f91c4160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c1ffb640 1 --2- 172.21.15.47:0/3075670006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b00039a0 0x7f91b0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c1ffb640 1 --2- 172.21.15.47:0/3075670006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91c40cde10 0x7f91b00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c1ffb640 1 --2- 172.21.15.47:0/3075670006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b0004d00 0x7f91b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c4160f60 con 0x7f91b00039a0 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b000bae0 con 0x7f91b0004d00 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b000bca0 con 0x7f91c40cde10 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c2ffd640 1 --2- 172.21.15.47:0/3075670006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b00039a0 0x7f91b0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c27fc640 1 --2- 172.21.15.47:0/3075670006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91c40cde10 0x7f91b00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c37fe640 1 --2- 172.21.15.47:0/3075670006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b0004d00 0x7f91b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c2ffd640 1 -- 172.21.15.47:0/3075670006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91c40cde10 msgr2=0x7f91b00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c2ffd640 1 --2- 172.21.15.47:0/3075670006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91c40cde10 0x7f91b00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c2ffd640 1 -- 172.21.15.47:0/3075670006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b0004d00 msgr2=0x7f91b0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c2ffd640 1 --2- 172.21.15.47:0/3075670006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b0004d00 0x7f91b0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.694+0000 7f91c2ffd640 1 -- 172.21.15.47:0/3075670006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b0003520 con 0x7f91b00039a0 2024-03-23T09:41:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.695+0000 7f91c2ffd640 1 --2- 172.21.15.47:0/3075670006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b00039a0 0x7f91b0003dc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c240 tx=0x7f91b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bda9233a600a652 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.695+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc005ad0 con 0x7f91b00039a0 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.695+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc005c30 con 0x7f91b00039a0 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.695+0000 7f91c1ffb640 1 -- 172.21.15.47:0/3075670006 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc02c810 con 0x7f91b00039a0 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.695+0000 7f91c97bf640 1 -- 172.21.15.47:0/3075670006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b00039a0 msgr2=0x7f91b0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.695+0000 7f91c97bf640 1 --2- 172.21.15.47:0/3075670006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b00039a0 0x7f91b0003dc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c240 tx=0x7f91b00057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.695+0000 7f91c37fe640 1 -- 172.21.15.47:0/3075670006 reap_dead start 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.695+0000 7f91c97bf640 1 -- 172.21.15.47:0/3075670006 shutdown_connections 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.695+0000 7f91c97bf640 1 -- 172.21.15.47:0/3075670006 >> 172.21.15.47:0/3075670006 conn(0x7f91c40659d0 msgr2=0x7f91c4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.695+0000 7f91c97bf640 1 -- 172.21.15.47:0/3075670006 shutdown_connections 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.695+0000 7f91c97bf640 1 -- 172.21.15.47:0/3075670006 wait complete. 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.697+0000 7f91c97bf640 1 Processor -- start 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.697+0000 7f91c97bf640 1 -- start start 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.697+0000 7f91c97bf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b00039a0 0x7f91c41f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.697+0000 7f91c97bf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91c41f5b00 0x7f91c41f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.697+0000 7f91c97bf640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91c40b3270 0x7f91c40b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.697+0000 7f91c97bf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c40cc380 con 0x7f91c41f5b00 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.697+0000 7f91c97bf640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c41f64c0 con 0x7f91b00039a0 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.697+0000 7f91c97bf640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c40b3c30 con 0x7f91c40b3270 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.697+0000 7f91c27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91c41f5b00 0x7f91c41f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.697+0000 7f91c27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91c41f5b00 0x7f91c41f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45994/0 (socket says 172.21.15.47:45994) 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.697+0000 7f91c27fc640 1 -- 172.21.15.47:0/3065855722 learned_addr learned my addr 172.21.15.47:0/3065855722 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.698+0000 7f91c27fc640 1 -- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91c40b3270 msgr2=0x7f91c40b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.698+0000 7f91c27fc640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91c40b3270 0x7f91c40b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.698+0000 7f91c2ffd640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b00039a0 0x7f91c41f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.698+0000 7f91c27fc640 1 -- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b00039a0 msgr2=0x7f91c41f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.698+0000 7f91c27fc640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b00039a0 0x7f91c41f1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.698+0000 7f91c27fc640 1 -- 172.21.15.47:0/3065855722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c40b3ef0 con 0x7f91c41f5b00 2024-03-23T09:41:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.698+0000 7f91c2ffd640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b00039a0 0x7f91c41f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.698+0000 7f91c27fc640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91c41f5b00 0x7f91c41f5f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00f7a0 tx=0x7f91ac00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.698+0000 7f91bb7fe640 1 -- 172.21.15.47:0/3065855722 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac0050d0 con 0x7f91c41f5b00 2024-03-23T09:41:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.698+0000 7f91bb7fe640 1 -- 172.21.15.47:0/3065855722 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac005230 con 0x7f91c41f5b00 2024-03-23T09:41:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.698+0000 7f91c97bf640 1 -- 172.21.15.47:0/3065855722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c40b41d0 con 0x7f91c41f5b00 2024-03-23T09:41:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.698+0000 7f91bb7fe640 1 -- 172.21.15.47:0/3065855722 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac00d730 con 0x7f91c41f5b00 2024-03-23T09:41:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.698+0000 7f91c97bf640 1 -- 172.21.15.47:0/3065855722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c41f2b80 con 0x7f91c41f5b00 2024-03-23T09:41:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.699+0000 7f91c97bf640 1 -- 172.21.15.47:0/3065855722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c41f2d40 con 0x7f91c41f5b00 2024-03-23T09:41:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.699+0000 7f91bb7fe640 1 -- 172.21.15.47:0/3065855722 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f91ac00d890 con 0x7f91c41f5b00 2024-03-23T09:41:11.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.700+0000 7f91bb7fe640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f91a406ac30 0x7f91a406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:11.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.700+0000 7f91bb7fe640 1 -- 172.21.15.47:0/3065855722 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91ac089260 con 0x7f91c41f5b00 2024-03-23T09:41:11.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.700+0000 7f91c2ffd640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f91a406ac30 0x7f91a406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:11.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.700+0000 7f91c2ffd640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f91a406ac30 0x7f91a406d0f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91bc034000 tx=0x7f91bc009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:11.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.704+0000 7f91bb7fe640 1 -- 172.21.15.47:0/3065855722 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f91ac054440 con 0x7f91c41f5b00 2024-03-23T09:41:11.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.886+0000 7f91c97bf640 1 -- 172.21.15.47:0/3065855722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91c4072360 con 0x7f91c41f5b00 2024-03-23T09:41:11.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.887+0000 7f91bb7fe640 1 -- 172.21.15.47:0/3065855722 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f91ac057a60 con 0x7f91c41f5b00 2024-03-23T09:41:11.886 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:11.887 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:11.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.889+0000 7f91c97bf640 1 -- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f91a406ac30 msgr2=0x7f91a406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.889+0000 7f91c97bf640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f91a406ac30 0x7f91a406d0f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91bc034000 tx=0x7f91bc009030 comp rx=0 tx=0).stop 2024-03-23T09:41:11.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.889+0000 7f91c97bf640 1 -- 172.21.15.47:0/3065855722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91c41f5b00 msgr2=0x7f91c41f5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:11.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.889+0000 7f91c97bf640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91c41f5b00 0x7f91c41f5f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00f7a0 tx=0x7f91ac00fc60 comp rx=0 tx=0).stop 2024-03-23T09:41:11.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.889+0000 7f91c97bf640 1 -- 172.21.15.47:0/3065855722 shutdown_connections 2024-03-23T09:41:11.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.889+0000 7f91c97bf640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f91a406ac30 0x7f91a406d0f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.889+0000 7f91c97bf640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91c40b3270 0x7f91c40b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.889+0000 7f91c97bf640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91c41f5b00 0x7f91c41f5f80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.889+0000 7f91c97bf640 1 --2- 172.21.15.47:0/3065855722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b00039a0 0x7f91c41f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:11.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.889+0000 7f91c97bf640 1 -- 172.21.15.47:0/3065855722 >> 172.21.15.47:0/3065855722 conn(0x7f91c40659d0 msgr2=0x7f91c415d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:11.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.889+0000 7f91c97bf640 1 -- 172.21.15.47:0/3065855722 shutdown_connections 2024-03-23T09:41:11.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:11.889+0000 7f91c97bf640 1 -- 172.21.15.47:0/3065855722 wait complete. 2024-03-23T09:41:11.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-23T09:41:11.900 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-23T09:41:11.900 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-23T09:41:11.901 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-23T09:41:11.901 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-23T09:41:11.901 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-23T09:41:11.902 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-23T09:41:11.902 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-23T09:41:12.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7faf2228b640 1 Processor -- start 2024-03-23T09:41:12.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7faf2228b640 1 -- start start 2024-03-23T09:41:12.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.074+0000 7f9944b37640 1 Processor -- start 2024-03-23T09:41:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.074+0000 7f9944b37640 1 -- start start 2024-03-23T09:41:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7f9944b37640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9940150810 0x7f99400cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7f9944b37640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99400ce500 0x7f994015a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.074+0000 7fd327fff640 1 Processor -- start 2024-03-23T09:41:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.074+0000 7fd327fff640 1 -- start start 2024-03-23T09:41:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7f0ce647e640 1 Processor -- start 2024-03-23T09:41:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7f0ce647e640 1 -- start start 2024-03-23T09:41:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.074+0000 7faf28fb1640 1 Processor -- start 2024-03-23T09:41:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.074+0000 7faf28fb1640 1 -- start start 2024-03-23T09:41:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7faf28fb1640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faf240cb860 0x7faf240afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7f9944b37640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f994006ebe0 con 0x7f99400b2cb0 2024-03-23T09:41:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7fd327fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd320053790 0x7fd320007130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7fd327fff640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3200548d0 con 0x7fd32004f8d0 2024-03-23T09:41:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7fd327fff640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd320037bf0 con 0x7fd3200380f0 2024-03-23T09:41:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7f9944b37640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f994006eec0 con 0x7f99400ce500 2024-03-23T09:41:12.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7f9944b37640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994015a080 con 0x7f9940150810 2024-03-23T09:41:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7fd327fff640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd320053d50 con 0x7fd320053790 2024-03-23T09:41:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7f40abfff640 1 Processor -- start 2024-03-23T09:41:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7f40abfff640 1 -- start start 2024-03-23T09:41:12.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7faf28fb1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf24159dc0 con 0x7faf24156920 2024-03-23T09:41:12.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7faf28fb1640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2415d910 con 0x7faf240b0330 2024-03-23T09:41:12.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.075+0000 7faf28fb1640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf240b0780 con 0x7faf240cb860 2024-03-23T09:41:12.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7faf2259c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7faf24156920 0x7faf240cb150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57244/0 (socket says 172.21.15.47:57244) 2024-03-23T09:41:12.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7faf2259c640 1 -- 172.21.15.47:0/592053738 learned_addr learned my addr 172.21.15.47:0/592053738 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:12.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7faf21d9b640 1 --2- 172.21.15.47:0/592053738 >> v2:172.21.15.175:3300/0 conn(0x7faf240cb860 0x7faf240afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878640092 0 0) 0x7faf24159dc0 con 0x7faf24156920 2024-03-23T09:41:12.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf10003520 con 0x7faf24156920 2024-03-23T09:41:12.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7faf2228b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faf1c15a8f0 0x7faf1c098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7faf2228b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf1c0ce2b0 0x7faf1c098f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7faf2228b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf1c099ce0 con 0x7faf1c15e3c0 2024-03-23T09:41:12.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7faf2228b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf1c15acd0 con 0x7faf1c15a8f0 2024-03-23T09:41:12.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7faf2228b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c0cc5c0 con 0x7faf1c0ce2b0 2024-03-23T09:41:12.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7f993dd9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99400ce500 0x7f994015a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7f993ed9d640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f99400b2cb0 0x7f994006c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47822/0 (socket says 172.21.15.47:47822) 2024-03-23T09:41:12.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7f403bc38640 1 Processor -- start 2024-03-23T09:41:12.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7f403bc38640 1 -- start start 2024-03-23T09:41:12.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7f993ed9d640 1 -- 172.21.15.47:0/504341036 learned_addr learned my addr 172.21.15.47:0/504341036 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:12.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308369400 0 0) 0x7faf2415d910 con 0x7faf240b0330 2024-03-23T09:41:12.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf24159dc0 con 0x7faf240b0330 2024-03-23T09:41:12.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7f0ce647e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0cd80df1f0 0x7f0cd8038050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7f993e59c640 1 --2- 172.21.15.47:0/504341036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9940150810 0x7f99400cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f0ce647e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd8054490 con 0x7f0cd8053c40 2024-03-23T09:41:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf20828640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7faf1c15e3c0 0x7faf1c0ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57258/0 (socket says 172.21.15.47:57258) 2024-03-23T09:41:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf20828640 1 -- 172.21.15.47:0/3282274715 learned_addr learned my addr 172.21.15.47:0/3282274715 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:12.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7f993e59c640 1 -- 172.21.15.47:0/504341036 >> v2:172.21.15.47:3300/0 conn(0x7f99400ce500 msgr2=0x7f994015a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7f993e59c640 1 --2- 172.21.15.47:0/504341036 >> v2:172.21.15.47:3300/0 conn(0x7f99400ce500 0x7f994015a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 118047546 0 0) 0x7faf10003520 con 0x7faf24156920 2024-03-23T09:41:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 >> v2:172.21.15.175:3300/0 conn(0x7faf240cb860 msgr2=0x7faf240afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 --2- 172.21.15.47:0/592053738 >> v2:172.21.15.175:3300/0 conn(0x7faf240cb860 0x7faf240afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 >> v1:172.21.15.175:6789/0 conn(0x7faf240b0330 legacy=0x7faf2415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240c9830 con 0x7faf24156920 2024-03-23T09:41:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7faf0c001670 con 0x7faf24156920 2024-03-23T09:41:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f0ce647e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd80541b0 con 0x7f0cd8050170 2024-03-23T09:41:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f0ce647e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd8005f00 con 0x7f0cd80df1f0 2024-03-23T09:41:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 >> v1:172.21.15.47:6789/0 conn(0x7faf24156920 legacy=0x7faf240cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 --2- 172.21.15.47:0/592053738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf10003b00 0x7faf10003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 --2- 172.21.15.47:0/592053738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf100047c0 0x7faf10004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f0ce5c7d640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f0cd8050170 0x7f0cd80069e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47856/0 (socket says 172.21.15.47:47856) 2024-03-23T09:41:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f0ce5c7d640 1 -- 172.21.15.47:0/1300069441 learned_addr learned my addr 172.21.15.47:0/1300069441 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 --2- 172.21.15.47:0/592053738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf10005180 0x7faf1000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f40abfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40a40e76e0 0x7f40a404ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f40abfff640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a4057d60 con 0x7f40a4058990 2024-03-23T09:41:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f40abfff640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a4006230 con 0x7f40a4053c10 2024-03-23T09:41:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f40abfff640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a4006510 con 0x7f40a40e76e0 2024-03-23T09:41:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7fd3267fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd320053790 0x7fd320007130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7fd3267fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd320053790 0x7fd320007130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46012/0 (socket says 172.21.15.47:46012) 2024-03-23T09:41:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7fd3267fc640 1 -- 172.21.15.47:0/1213871117 learned_addr learned my addr 172.21.15.47:0/1213871117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7fd3267fc640 1 -- 172.21.15.47:0/1213871117 >> v1:172.21.15.175:6789/0 conn(0x7fd3200380f0 legacy=0x7fd320038580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7fd3267fc640 1 -- 172.21.15.47:0/1213871117 >> v1:172.21.15.47:6789/0 conn(0x7fd32004f8d0 legacy=0x7fd32004fcb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T09:41:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.076+0000 7fd3267fc640 1 -- 172.21.15.47:0/1213871117 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320054030 con 0x7fd320053790 2024-03-23T09:41:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf240b0780 con 0x7faf100047c0 2024-03-23T09:41:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2259c640 1 --2- 172.21.15.47:0/592053738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf10003b00 0x7faf10003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf21d9b640 1 --2- 172.21.15.47:0/592053738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf100047c0 0x7faf10004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf1bfff640 1 --2- 172.21.15.47:0/3282274715 >> v2:172.21.15.175:3300/0 conn(0x7faf1c15a8f0 0x7faf1c098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1000c000 con 0x7faf10003b00 2024-03-23T09:41:12.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f40ab7fe640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f40a4053c10 0x7f40a4006ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47868/0 (socket says 172.21.15.47:47868) 2024-03-23T09:41:12.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f40ab7fe640 1 -- 172.21.15.47:0/2019432741 learned_addr learned my addr 172.21.15.47:0/2019432741 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:12.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1000c1c0 con 0x7faf10005180 2024-03-23T09:41:12.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf1b7fe640 1 --2- 172.21.15.47:0/3282274715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf1c0ce2b0 0x7faf1c098f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3254799774 0 0) 0x7f0cd80541b0 con 0x7f0cd8050170 2024-03-23T09:41:12.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ccc003520 con 0x7f0cd8050170 2024-03-23T09:41:12.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f993e59c640 1 -- 172.21.15.47:0/504341036 >> v1:172.21.15.175:6789/0 conn(0x7f99400b2cb0 legacy=0x7f994006c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f993e59c640 1 -- 172.21.15.47:0/504341036 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994015a360 con 0x7f9940150810 2024-03-23T09:41:12.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f40aaffd640 1 --2- 172.21.15.47:0/2019432741 >> v2:172.21.15.47:3300/0 conn(0x7f40a40e76e0 0x7f40a404ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f40a9ffb640 1 -- 172.21.15.47:0/2019432741 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375490704 0 0) 0x7f40a4006230 con 0x7f40a4053c10 2024-03-23T09:41:12.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f40a9ffb640 1 -- 172.21.15.47:0/2019432741 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4094003520 con 0x7f40a4053c10 2024-03-23T09:41:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f0ce547c640 1 --2- 172.21.15.47:0/1300069441 >> v2:172.21.15.47:3300/0 conn(0x7f0cd80df1f0 0x7f0cd8038050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7f993dd9b640 1 --2- 172.21.15.47:0/504341036 >> v2:172.21.15.47:3300/0 conn(0x7f99400ce500 0x7f994015a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.077+0000 7faf22d9d640 1 --2- 172.21.15.47:0/592053738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf10005180 0x7faf1000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f403bc38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402c050160 0x7f402c038040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7faf1affd640 1 -- 172.21.15.47:0/3282274715 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527562588 0 0) 0x7faf1c099ce0 con 0x7faf1c15e3c0 2024-03-23T09:41:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7faf1affd640 1 -- 172.21.15.47:0/3282274715 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf04003520 con 0x7faf1c15e3c0 2024-03-23T09:41:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2043783317 0 0) 0x7f0ccc003520 con 0x7f0cd8050170 2024-03-23T09:41:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 >> v2:172.21.15.47:3300/0 conn(0x7f0cd80df1f0 msgr2=0x7f0cd8038050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7fcc34cd5640 1 Processor -- start 2024-03-23T09:41:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f403bc38640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f402c053c30 0x7f402c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f403bc38640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402c054480 con 0x7f402c0df1e0 2024-03-23T09:41:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f403bc38640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402c0077a0 con 0x7f402c053c30 2024-03-23T09:41:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f40391d3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402c050160 0x7f402c038040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f40391d3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402c050160 0x7f402c038040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:46582/0 (socket says 172.21.15.47:46582) 2024-03-23T09:41:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f40399d4640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f402c0df1e0 0x7f402c04fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57290/0 (socket says 172.21.15.47:57290) 2024-03-23T09:41:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f40391d3640 1 -- 172.21.15.47:0/701300662 learned_addr learned my addr 172.21.15.47:0/701300662 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f403bc38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c0541a0 con 0x7f402c050160 2024-03-23T09:41:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7faf1bfff640 1 -- 172.21.15.47:0/3282274715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf1c0ce2b0 msgr2=0x7faf1c098f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7faf1bfff640 1 --2- 172.21.15.47:0/3282274715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf1c0ce2b0 0x7faf1c098f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7faf1bfff640 1 -- 172.21.15.47:0/3282274715 >> v1:172.21.15.47:6789/0 conn(0x7faf1c15e3c0 legacy=0x7faf1c0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7faf1bfff640 1 -- 172.21.15.47:0/3282274715 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1c0cb520 con 0x7faf1c15a8f0 2024-03-23T09:41:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7fd3267fc640 1 --2- 172.21.15.47:0/1213871117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd320053790 0x7fd320007130 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd318009870 tx=0x7fd31802b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45ee270d1505fef server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0cd7fff640 1 --2- 172.21.15.47:0/1300069441 >> v2:172.21.15.47:3300/0 conn(0x7f0cd80df1f0 0x7f0cd8038050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 >> v1:172.21.15.47:6789/0 conn(0x7f0cd8053c40 legacy=0x7f0cd804fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd80061e0 con 0x7f0cd8050170 2024-03-23T09:41:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0ce547c640 1 --2- 172.21.15.47:0/1300069441 >> v2:172.21.15.47:3300/0 conn(0x7f0cd80df1f0 0x7f0cd8038050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0ce004e550 con 0x7f0cd8050170 2024-03-23T09:41:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 >> v1:172.21.15.175:6789/0 conn(0x7f0cd8050170 legacy=0x7f0cd80069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0cd7fff640 1 --2- 172.21.15.47:0/1300069441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc003970 0x7f0ccc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7faf22d9d640 1 -- 172.21.15.47:0/592053738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf10003b00 msgr2=0x7faf10003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7faf22d9d640 1 --2- 172.21.15.47:0/592053738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf10003b00 0x7faf10003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7faf22d9d640 1 -- 172.21.15.47:0/592053738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf100047c0 msgr2=0x7faf10004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7faf22d9d640 1 --2- 172.21.15.47:0/592053738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf100047c0 0x7faf10004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7faf22d9d640 1 -- 172.21.15.47:0/592053738 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240c9830 con 0x7faf10005180 2024-03-23T09:41:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0cd7fff640 1 --2- 172.21.15.47:0/1300069441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ccc004310 0x7f0ccc0047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205013808 0 0) 0x7f402c054480 con 0x7f402c0df1e0 2024-03-23T09:41:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4020003520 con 0x7f402c0df1e0 2024-03-23T09:41:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f403a1d5640 1 --2- 172.21.15.47:0/701300662 >> v2:172.21.15.47:3300/0 conn(0x7f402c053c30 0x7f402c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0cd7fff640 1 --2- 172.21.15.47:0/1300069441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ccc005080 0x7f0ccc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f993e59c640 1 --2- 172.21.15.47:0/504341036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9940150810 0x7f99400cbe30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f993000b850 tx=0x7f993002bca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bdf8cdc13b3fbe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd8005f00 con 0x7f0ccc003970 2024-03-23T09:41:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc00bfd0 con 0x7f0ccc005080 2024-03-23T09:41:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc0105a0 con 0x7f0ccc004310 2024-03-23T09:41:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f40aaffd640 1 -- 172.21.15.47:0/2019432741 >> v1:172.21.15.175:6789/0 conn(0x7f40a4053c10 legacy=0x7f40a4006ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f40aaffd640 1 -- 172.21.15.47:0/2019432741 >> v1:172.21.15.47:6789/0 conn(0x7f40a4058990 legacy=0x7f40a4053500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7f40aaffd640 1 -- 172.21.15.47:0/2019432741 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a40067f0 con 0x7f40a40e76e0 2024-03-23T09:41:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f0ce547c640 1 --2- 172.21.15.47:0/1300069441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc003970 0x7f0ccc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f993d59a640 1 -- 172.21.15.47:0/504341036 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993002c9d0 con 0x7f9940150810 2024-03-23T09:41:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7fd325ffb640 1 -- 172.21.15.47:0/1213871117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318030880 con 0x7fd320053790 2024-03-23T09:41:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7fd325ffb640 1 -- 172.21.15.47:0/1213871117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3180309e0 con 0x7fd320053790 2024-03-23T09:41:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7fd325ffb640 1 -- 172.21.15.47:0/1213871117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318030db0 con 0x7fd320053790 2024-03-23T09:41:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7faf22d9d640 1 -- 172.21.15.47:0/592053738 reap_dead start 2024-03-23T09:41:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 30149553 0 0) 0x7f4020003520 con 0x7f402c0df1e0 2024-03-23T09:41:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402c050160 msgr2=0x7f402c038040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 --2- 172.21.15.47:0/701300662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402c050160 0x7f402c038040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 >> v2:172.21.15.47:3300/0 conn(0x7f402c053c30 msgr2=0x7f402c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 --2- 172.21.15.47:0/701300662 >> v2:172.21.15.47:3300/0 conn(0x7f402c053c30 0x7f402c0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c005ef0 con 0x7f402c0df1e0 2024-03-23T09:41:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f4030001670 con 0x7f402c0df1e0 2024-03-23T09:41:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 >> v1:172.21.15.47:6789/0 conn(0x7f402c0df1e0 legacy=0x7f402c04fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 --2- 172.21.15.47:0/701300662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4020003970 0x7f4020003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 --2- 172.21.15.47:0/701300662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40200047c0 0x7f4020004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 --2- 172.21.15.47:0/701300662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4020005180 0x7f402000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.078+0000 7fcc34cd5640 1 -- start start 2024-03-23T09:41:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c0077a0 con 0x7f40200047c0 2024-03-23T09:41:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c0541a0 con 0x7f4020003970 2024-03-23T09:41:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4020010460 con 0x7f4020005180 2024-03-23T09:41:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f0ce4c7b640 1 --2- 172.21.15.47:0/1300069441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ccc004310 0x7f0ccc0047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7faf1bfff640 1 --2- 172.21.15.47:0/3282274715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf1c15a8f0 0x7faf1c098a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faf100097d0 tx=0x7faf1002fc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fad33697160121b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7faf22d9d640 1 --2- 172.21.15.47:0/592053738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf10005180 0x7faf1000b980 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faf1c001fd0 tx=0x7faf100057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c15a8f6b3b948cc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7fcc34cd5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcc280eacc0 0x7fcc2804fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f403a1d5640 1 --2- 172.21.15.47:0/701300662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4020005180 0x7f402000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f0ce5c7d640 1 --2- 172.21.15.47:0/1300069441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ccc005080 0x7f0ccc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40aaffd640 1 --2- 172.21.15.47:0/2019432741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a40e76e0 0x7f40a404ca00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f40ac000b60 tx=0x7f40ac0a54e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4843ba390be05033 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c003b70 con 0x7faf10005180 2024-03-23T09:41:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f40399d4640 1 --2- 172.21.15.47:0/701300662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4020003970 0x7f4020003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c003cd0 con 0x7faf10005180 2024-03-23T09:41:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7fcc34cd5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc28006e60 0x7fcc280072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7fcc34cd5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc280542f0 con 0x7fcc2804ff90 2024-03-23T09:41:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7fcc34cd5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc28007830 con 0x7fcc280eacc0 2024-03-23T09:41:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7fcc34cd5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280536e0 con 0x7fcc28006e60 2024-03-23T09:41:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7faf1affd640 1 -- 172.21.15.47:0/3282274715 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf100309e0 con 0x7faf1c15a8f0 2024-03-23T09:41:12.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7faf2159a640 1 -- 172.21.15.47:0/592053738 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c005310 con 0x7faf10005180 2024-03-23T09:41:12.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7faf1affd640 1 -- 172.21.15.47:0/3282274715 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf10030b40 con 0x7faf1c15a8f0 2024-03-23T09:41:12.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f40a9ffb640 1 -- 172.21.15.47:0/2019432741 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0aa7d0 con 0x7f40a40e76e0 2024-03-23T09:41:12.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7fd327fff640 1 -- 172.21.15.47:0/1213871117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd320053790 msgr2=0x7fd320007130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7fd327fff640 1 --2- 172.21.15.47:0/1213871117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd320053790 0x7fd320007130 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd318009870 tx=0x7fd31802b9c0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f40a9ffb640 1 -- 172.21.15.47:0/2019432741 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac0a8050 con 0x7f40a40e76e0 2024-03-23T09:41:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f40a9ffb640 1 -- 172.21.15.47:0/2019432741 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0aac30 con 0x7f40a40e76e0 2024-03-23T09:41:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f993d59a640 1 -- 172.21.15.47:0/504341036 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993002cb30 con 0x7f9940150810 2024-03-23T09:41:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.079+0000 7f993d59a640 1 -- 172.21.15.47:0/504341036 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930034440 con 0x7f9940150810 2024-03-23T09:41:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f9944b37640 1 -- 172.21.15.47:0/504341036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9940150810 msgr2=0x7f99400cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f9944b37640 1 --2- 172.21.15.47:0/504341036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9940150810 0x7f99400cbe30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f993000b850 tx=0x7f993002bca0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f403a1d5640 1 -- 172.21.15.47:0/701300662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4020003970 msgr2=0x7f4020003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f403a1d5640 1 --2- 172.21.15.47:0/701300662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4020003970 0x7f4020003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f403a1d5640 1 -- 172.21.15.47:0/701300662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40200047c0 msgr2=0x7f4020004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f403a1d5640 1 --2- 172.21.15.47:0/701300662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40200047c0 0x7f4020004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f403a1d5640 1 -- 172.21.15.47:0/701300662 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c005ef0 con 0x7f4020005180 2024-03-23T09:41:12.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf28fb1640 1 -- 172.21.15.47:0/592053738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf10005180 msgr2=0x7faf1000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7fcc2e59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcc280eacc0 0x7fcc2804fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7fcc2ed9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc28006e60 0x7fcc280072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7fcc2ed9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc28006e60 0x7fcc280072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:46604/0 (socket says 172.21.15.47:46604) 2024-03-23T09:41:12.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7fcc2ed9d640 1 -- 172.21.15.47:0/251624940 learned_addr learned my addr 172.21.15.47:0/251624940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7fcc2ed9d640 1 -- 172.21.15.47:0/251624940 >> v2:172.21.15.47:3300/0 conn(0x7fcc280eacc0 msgr2=0x7fcc2804fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7fcc2ed9d640 1 --2- 172.21.15.47:0/251624940 >> v2:172.21.15.47:3300/0 conn(0x7fcc280eacc0 0x7fcc2804fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7fcc2ed9d640 1 -- 172.21.15.47:0/251624940 >> v1:172.21.15.175:6789/0 conn(0x7fcc2804ff90 legacy=0x7fcc28037b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7fcc2ed9d640 1 -- 172.21.15.47:0/251624940 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc280539c0 con 0x7fcc28006e60 2024-03-23T09:41:12.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7fcc2e59c640 1 --2- 172.21.15.47:0/251624940 >> v2:172.21.15.47:3300/0 conn(0x7fcc280eacc0 0x7fcc2804fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:12.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf28fb1640 1 --2- 172.21.15.47:0/592053738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf10005180 0x7faf1000b980 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faf1c001fd0 tx=0x7faf100057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f0ce5c7d640 1 -- 172.21.15.47:0/1300069441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ccc004310 msgr2=0x7f0ccc0047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f0ce5c7d640 1 --2- 172.21.15.47:0/1300069441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ccc004310 0x7f0ccc0047e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f0ce5c7d640 1 -- 172.21.15.47:0/1300069441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc003970 msgr2=0x7f0ccc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f0ce5c7d640 1 --2- 172.21.15.47:0/1300069441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc003970 0x7f0ccc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7f0ce5c7d640 1 -- 172.21.15.47:0/1300069441 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc003520 con 0x7f0ccc005080 2024-03-23T09:41:12.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.080+0000 7faf1affd640 1 -- 172.21.15.47:0/3282274715 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10038440 con 0x7faf1c15a8f0 2024-03-23T09:41:12.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf2228b640 1 -- 172.21.15.47:0/3282274715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf1c15a8f0 msgr2=0x7faf1c098a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf2228b640 1 --2- 172.21.15.47:0/3282274715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf1c15a8f0 0x7faf1c098a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faf100097d0 tx=0x7faf1002fc30 comp rx=0 tx=0).stop 2024-03-23T09:41:12.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f0ce5c7d640 1 -- 172.21.15.47:0/1300069441 reap_dead start 2024-03-23T09:41:12.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf2228b640 1 -- 172.21.15.47:0/3282274715 shutdown_connections 2024-03-23T09:41:12.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf2228b640 1 --2- 172.21.15.47:0/3282274715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf1c0ce2b0 0x7faf1c098f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf2228b640 1 --2- 172.21.15.47:0/3282274715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf1c15a8f0 0x7faf1c098a00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf2228b640 1 -- 172.21.15.47:0/3282274715 >> 172.21.15.47:0/3282274715 conn(0x7faf1c0b21d0 msgr2=0x7faf1c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f9944b37640 1 -- 172.21.15.47:0/504341036 shutdown_connections 2024-03-23T09:41:12.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f9944b37640 1 --2- 172.21.15.47:0/504341036 >> v2:172.21.15.47:3300/0 conn(0x7f99400ce500 0x7f994015a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f9944b37640 1 --2- 172.21.15.47:0/504341036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9940150810 0x7f99400cbe30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f9944b37640 1 -- 172.21.15.47:0/504341036 >> 172.21.15.47:0/504341036 conn(0x7f994006a310 msgr2=0x7f99400b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf28fb1640 1 -- 172.21.15.47:0/592053738 shutdown_connections 2024-03-23T09:41:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf28fb1640 1 --2- 172.21.15.47:0/592053738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf10005180 0x7faf1000b980 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf28fb1640 1 -- 172.21.15.47:0/592053738 >> 172.21.15.47:0/592053738 conn(0x7faf240659d0 msgr2=0x7faf24065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f40abfff640 1 -- 172.21.15.47:0/2019432741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a40e76e0 msgr2=0x7f40a404ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f0ce5c7d640 1 --2- 172.21.15.47:0/1300069441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ccc005080 0x7f0ccc00b980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00741c0 tx=0x7f0ccc0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177feacd402161a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f40abfff640 1 --2- 172.21.15.47:0/2019432741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a40e76e0 0x7f40a404ca00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f40ac000b60 tx=0x7f40ac0a54e0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf28fb1640 1 -- 172.21.15.47:0/592053738 shutdown_connections 2024-03-23T09:41:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf2228b640 1 -- 172.21.15.47:0/3282274715 shutdown_connections 2024-03-23T09:41:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f403a1d5640 1 --2- 172.21.15.47:0/701300662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4020005180 0x7f402000b980 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f402400ba90 tx=0x7f40200057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2686ea6e2e6675 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf28fb1640 1 -- 172.21.15.47:0/592053738 wait complete. 2024-03-23T09:41:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7fd327fff640 1 -- 172.21.15.47:0/1213871117 shutdown_connections 2024-03-23T09:41:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7fd327fff640 1 --2- 172.21.15.47:0/1213871117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd320053790 0x7fd320007130 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7fd327fff640 1 -- 172.21.15.47:0/1213871117 >> 172.21.15.47:0/1213871117 conn(0x7fd320009f80 msgr2=0x7fd32000a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f40abfff640 1 -- 172.21.15.47:0/2019432741 shutdown_connections 2024-03-23T09:41:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f40abfff640 1 --2- 172.21.15.47:0/2019432741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a40e76e0 0x7f40a404ca00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f40abfff640 1 -- 172.21.15.47:0/2019432741 >> 172.21.15.47:0/2019432741 conn(0x7f40a4058eb0 msgr2=0x7f40a4059290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024032660 con 0x7f4020005180 2024-03-23T09:41:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f40abfff640 1 -- 172.21.15.47:0/2019432741 shutdown_connections 2024-03-23T09:41:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40240327c0 con 0x7f4020005180 2024-03-23T09:41:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00a6a60 con 0x7f0ccc005080 2024-03-23T09:41:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce00a6c00 con 0x7f0ccc005080 2024-03-23T09:41:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f40389d2640 1 -- 172.21.15.47:0/701300662 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024032b90 con 0x7f4020005180 2024-03-23T09:41:12.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f0cd7fff640 1 -- 172.21.15.47:0/1300069441 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00af470 con 0x7f0ccc005080 2024-03-23T09:41:12.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.082+0000 7f9944b37640 1 -- 172.21.15.47:0/504341036 shutdown_connections 2024-03-23T09:41:12.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7fd327fff640 1 -- 172.21.15.47:0/1213871117 shutdown_connections 2024-03-23T09:41:12.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.082+0000 7f9944b37640 1 -- 172.21.15.47:0/504341036 wait complete. 2024-03-23T09:41:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.082+0000 7fcc2ed9d640 1 --2- 172.21.15.47:0/251624940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc28006e60 0x7fcc280072f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009870 tx=0x7fcc2402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e81af048e0c29052 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7faf2228b640 1 -- 172.21.15.47:0/3282274715 wait complete. 2024-03-23T09:41:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.081+0000 7f40abfff640 1 -- 172.21.15.47:0/2019432741 wait complete. 2024-03-23T09:41:12.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.082+0000 7fd327fff640 1 -- 172.21.15.47:0/1213871117 wait complete. 2024-03-23T09:41:12.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.082+0000 7f403bc38640 1 -- 172.21.15.47:0/701300662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4020005180 msgr2=0x7f402000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.082+0000 7f403bc38640 1 --2- 172.21.15.47:0/701300662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4020005180 0x7f402000b980 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f402400ba90 tx=0x7f40200057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.082+0000 7f403a1d5640 1 -- 172.21.15.47:0/701300662 reap_dead start 2024-03-23T09:41:12.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.082+0000 7f403bc38640 1 -- 172.21.15.47:0/701300662 shutdown_connections 2024-03-23T09:41:12.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.082+0000 7f403bc38640 1 -- 172.21.15.47:0/701300662 >> 172.21.15.47:0/701300662 conn(0x7f402c009f70 msgr2=0x7f402c00a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.082+0000 7f0ce647e640 1 -- 172.21.15.47:0/1300069441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ccc005080 msgr2=0x7f0ccc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.082+0000 7f0ce647e640 1 --2- 172.21.15.47:0/1300069441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ccc005080 0x7f0ccc00b980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00741c0 tx=0x7f0ccc0057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7fcc2d59a640 1 -- 172.21.15.47:0/251624940 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24030840 con 0x7fcc28006e60 2024-03-23T09:41:12.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7fcc2d59a640 1 -- 172.21.15.47:0/251624940 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc240309a0 con 0x7fcc28006e60 2024-03-23T09:41:12.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7fcc2d59a640 1 -- 172.21.15.47:0/251624940 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24030d30 con 0x7fcc28006e60 2024-03-23T09:41:12.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7f0ce647e640 1 -- 172.21.15.47:0/1300069441 shutdown_connections 2024-03-23T09:41:12.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7f403bc38640 1 -- 172.21.15.47:0/701300662 shutdown_connections 2024-03-23T09:41:12.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7f403bc38640 1 -- 172.21.15.47:0/701300662 wait complete. 2024-03-23T09:41:12.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7f0ce647e640 1 --2- 172.21.15.47:0/1300069441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ccc005080 0x7f0ccc00b980 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7f0ce647e640 1 -- 172.21.15.47:0/1300069441 >> 172.21.15.47:0/1300069441 conn(0x7f0cd8009f80 msgr2=0x7f0cd800a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7faf2228b640 1 Processor -- start 2024-03-23T09:41:12.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7faf28fb1640 1 Processor -- start 2024-03-23T09:41:12.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7f40abfff640 1 Processor -- start 2024-03-23T09:41:12.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7faf2228b640 1 -- start start 2024-03-23T09:41:12.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7f40abfff640 1 -- start start 2024-03-23T09:41:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.083+0000 7faf28fb1640 1 -- start start 2024-03-23T09:41:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7f9944b37640 1 Processor -- start 2024-03-23T09:41:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7fd327fff640 1 Processor -- start 2024-03-23T09:41:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7fcc34cd5640 1 -- 172.21.15.47:0/251624940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc28006e60 msgr2=0x7fcc280072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7fcc34cd5640 1 --2- 172.21.15.47:0/251624940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc28006e60 0x7fcc280072f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009870 tx=0x7fcc2402b980 comp rx=0 tx=0).stop 2024-03-23T09:41:12.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7f9944b37640 1 -- start start 2024-03-23T09:41:12.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7f0ce647e640 1 -- 172.21.15.47:0/1300069441 shutdown_connections 2024-03-23T09:41:12.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7fd327fff640 1 -- start start 2024-03-23T09:41:12.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7f0ce647e640 1 -- 172.21.15.47:0/1300069441 wait complete. 2024-03-23T09:41:12.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7fcc34cd5640 1 -- 172.21.15.47:0/251624940 shutdown_connections 2024-03-23T09:41:12.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7faf28fb1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf10005180 0x7faf240cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7f403bc38640 1 Processor -- start 2024-03-23T09:41:12.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7faf28fb1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf24158700 0x7faf24156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7fcc34cd5640 1 --2- 172.21.15.47:0/251624940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc28006e60 0x7fcc280072f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7f403bc38640 1 -- start start 2024-03-23T09:41:12.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7fcc34cd5640 1 --2- 172.21.15.47:0/251624940 >> v2:172.21.15.47:3300/0 conn(0x7fcc280eacc0 0x7fcc2804fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7f40abfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a4053c10 0x7f40a403d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7f40abfff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4058990 0x7f40a417f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7f40abfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a40e76e0 0x7f40a4051c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7f40abfff640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a4052b60 con 0x7f40a4053c10 2024-03-23T09:41:12.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.084+0000 7fcc34cd5640 1 -- 172.21.15.47:0/251624940 >> 172.21.15.47:0/251624940 conn(0x7fcc28009f70 msgr2=0x7fcc2800a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7f40abfff640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a417fc60 con 0x7f40a40e76e0 2024-03-23T09:41:12.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7f40abfff640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a403dc10 con 0x7f40a4058990 2024-03-23T09:41:12.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf21d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf24158700 0x7faf24156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf21d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf24158700 0x7faf24156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46092/0 (socket says 172.21.15.47:46092) 2024-03-23T09:41:12.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf2228b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf1c0ce2b0 0x7faf1c0b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf28fb1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf24158b80 0x7faf241574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf28fb1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2415a3f0 con 0x7faf24158700 2024-03-23T09:41:12.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf28fb1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf240c40b0 con 0x7faf10005180 2024-03-23T09:41:12.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf28fb1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf240c3810 con 0x7faf24158b80 2024-03-23T09:41:12.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf2228b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf1c15a8f0 0x7faf1c0b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf21d9b640 1 -- 172.21.15.47:0/1412648872 learned_addr learned my addr 172.21.15.47:0/1412648872 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf2228b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf1c15e3c0 0x7faf1c156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf2228b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c0b3c00 con 0x7faf1c15e3c0 2024-03-23T09:41:12.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7f40aa7fc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4058990 0x7f40a417f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf2228b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c0b3ee0 con 0x7faf1c0ce2b0 2024-03-23T09:41:12.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7f40aa7fc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4058990 0x7f40a417f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:46618/0 (socket says 172.21.15.47:46618) 2024-03-23T09:41:12.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7f40aa7fc640 1 -- 172.21.15.47:0/2917251019 learned_addr learned my addr 172.21.15.47:0/2917251019 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf2228b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c0b41c0 con 0x7faf1c15a8f0 2024-03-23T09:41:12.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7f40ab7fe640 1 --2- 172.21.15.47:0/2917251019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a40e76e0 0x7f40a4051c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7f9944b37640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99400b2cb0 0x7f9940152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf20828640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf1c15e3c0 0x7faf1c156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7f9944b37640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99400ce500 0x7f99400b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf20828640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf1c15e3c0 0x7faf1c156320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46100/0 (socket says 172.21.15.47:46100) 2024-03-23T09:41:12.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7fd327fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3200380f0 0x7fd3200498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf20828640 1 -- 172.21.15.47:0/1589273956 learned_addr learned my addr 172.21.15.47:0/1589273956 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7faf1bfff640 1 --2- 172.21.15.47:0/1589273956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf1c0ce2b0 0x7faf1c0b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f993e59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99400b2cb0 0x7f9940152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf21d9b640 1 -- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf24158b80 msgr2=0x7faf241574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.085+0000 7faf21d9b640 1 --2- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf24158b80 0x7faf241574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7faf21d9b640 1 -- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf10005180 msgr2=0x7faf240cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7faf21d9b640 1 --2- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf10005180 0x7faf240cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7faf21d9b640 1 -- 172.21.15.47:0/1412648872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240c3af0 con 0x7faf24158700 2024-03-23T09:41:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f993e59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99400b2cb0 0x7f9940152ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:46500/0 (socket says 172.21.15.47:46500) 2024-03-23T09:41:12.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f40aa7fc640 1 -- 172.21.15.47:0/2917251019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a40e76e0 msgr2=0x7f40a4051c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f40aa7fc640 1 --2- 172.21.15.47:0/2917251019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a40e76e0 0x7f40a4051c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7fd327fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd32004f8d0 0x7fd3200e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f993e59c640 1 -- 172.21.15.47:0/3059052544 learned_addr learned my addr 172.21.15.47:0/3059052544 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f9944b37640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9940150810 0x7f99400ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f40aa7fc640 1 -- 172.21.15.47:0/2917251019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a4053c10 msgr2=0x7f40a403d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T09:41:12.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f40aa7fc640 1 --2- 172.21.15.47:0/2917251019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a4053c10 0x7f40a403d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f40aa7fc640 1 -- 172.21.15.47:0/2917251019 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac06be70 con 0x7f40a4058990 2024-03-23T09:41:12.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7fd327fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd320053790 0x7fd3200e8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f403bc38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4020005180 0x7f402c0e1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7faf20828640 1 -- 172.21.15.47:0/1589273956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf1c15a8f0 msgr2=0x7faf1c0b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7faf20828640 1 --2- 172.21.15.47:0/1589273956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf1c15a8f0 0x7faf1c0b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7faf20828640 1 -- 172.21.15.47:0/1589273956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf1c0ce2b0 msgr2=0x7faf1c0b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7faf20828640 1 --2- 172.21.15.47:0/1589273956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf1c0ce2b0 0x7faf1c0b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7faf21d9b640 1 --2- 172.21.15.47:0/1412648872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf24158700 0x7faf24156f70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faf14009fd0 tx=0x7faf14011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f403bc38640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f402c0e0610 0x7f402c0344a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7faf20828640 1 -- 172.21.15.47:0/1589273956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf10009400 con 0x7faf1c15e3c0 2024-03-23T09:41:12.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99400b0d30 con 0x7f99400ce500 2024-03-23T09:41:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f993ed9d640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9940150810 0x7f99400ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7fd3267fc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd32004f8d0 0x7fd3200e7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7fd3267fc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd32004f8d0 0x7fd3200e7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:46656/0 (socket says 172.21.15.47:46656) 2024-03-23T09:41:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7fd3267fc640 1 -- 172.21.15.47:0/3620664449 learned_addr learned my addr 172.21.15.47:0/3620664449 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7fd327fff640 1 -- 172.21.15.47:0/3620664449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd320048340 con 0x7fd3200380f0 2024-03-23T09:41:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f403bc38640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f402c02d6f0 0x7f402c02dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7fd3277fe640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd320053790 0x7fd3200e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7fd326ffd640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3200380f0 0x7fd3200498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7fd327fff640 1 -- 172.21.15.47:0/3620664449 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3200485e0 con 0x7fd320053790 2024-03-23T09:41:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f40399d4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4020005180 0x7f402c0e1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f40399d4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4020005180 0x7f402c0e1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:46670/0 (socket says 172.21.15.47:46670) 2024-03-23T09:41:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7fd327fff640 1 -- 172.21.15.47:0/3620664449 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3200488c0 con 0x7fd32004f8d0 2024-03-23T09:41:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f0ce647e640 1 Processor -- start 2024-03-23T09:41:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f403bc38640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c0e1670 con 0x7f402c02d6f0 2024-03-23T09:41:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f403bc38640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c02e0e0 con 0x7f402c0e0610 2024-03-23T09:41:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f403bc38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c0e0af0 con 0x7f4020005180 2024-03-23T09:41:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f993dd9b640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99400ce500 0x7f99400b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f40aa7fc640 1 --2- 172.21.15.47:0/2917251019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4058990 0x7f40a417f720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f409c00c430 tx=0x7f409c00c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f40399d4640 1 -- 172.21.15.47:0/332622576 learned_addr learned my addr 172.21.15.47:0/332622576 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f0ce647e640 1 -- start start 2024-03-23T09:41:12.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7faf20828640 1 --2- 172.21.15.47:0/1589273956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf1c15e3c0 0x7faf1c156320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faf1400c7e0 tx=0x7faf1400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99400af030 con 0x7f99400b2cb0 2024-03-23T09:41:12.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f403a1d5640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f402c02d6f0 0x7f402c02dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f40391d3640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f402c0e0610 0x7f402c0344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99400af310 con 0x7f9940150810 2024-03-23T09:41:12.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.086+0000 7fcc34cd5640 1 -- 172.21.15.47:0/251624940 shutdown_connections 2024-03-23T09:41:12.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7fcc34cd5640 1 -- 172.21.15.47:0/251624940 wait complete. 2024-03-23T09:41:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7faf18ff9640 1 -- 172.21.15.47:0/1589273956 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1400a640 con 0x7faf1c15e3c0 2024-03-23T09:41:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7faf1affd640 1 -- 172.21.15.47:0/1412648872 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf14015870 con 0x7faf24158700 2024-03-23T09:41:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7faf1affd640 1 -- 172.21.15.47:0/1412648872 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf140159d0 con 0x7faf24158700 2024-03-23T09:41:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7faf1affd640 1 -- 172.21.15.47:0/1412648872 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf14015d40 con 0x7faf24158700 2024-03-23T09:41:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7faf28fb1640 1 -- 172.21.15.47:0/1412648872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf240c3dd0 con 0x7faf24158700 2024-03-23T09:41:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7faf28fb1640 1 -- 172.21.15.47:0/1412648872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf240ccd70 con 0x7faf24158700 2024-03-23T09:41:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7fd3267fc640 1 -- 172.21.15.47:0/3620664449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd320053790 msgr2=0x7fd3200e8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7fd3267fc640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd320053790 0x7fd3200e8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7fd3267fc640 1 -- 172.21.15.47:0/3620664449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3200380f0 msgr2=0x7fd3200498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7fd3267fc640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3200380f0 0x7fd3200498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7fd3267fc640 1 -- 172.21.15.47:0/3620664449 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3180094d0 con 0x7fd32004f8d0 2024-03-23T09:41:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7fd326ffd640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3200380f0 0x7fd3200498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7fd3277fe640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd320053790 0x7fd3200e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:12.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7faf2228b640 1 -- 172.21.15.47:0/1589273956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1c0b44a0 con 0x7faf1c15e3c0 2024-03-23T09:41:12.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7faf2228b640 1 -- 172.21.15.47:0/1589273956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1c0b3840 con 0x7faf1c15e3c0 2024-03-23T09:41:12.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7f993ed9d640 1 -- 172.21.15.47:0/3059052544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99400b2cb0 msgr2=0x7f9940152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7f403a1d5640 1 -- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4020005180 msgr2=0x7f402c0e1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7f403a1d5640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4020005180 0x7f402c0e1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7f403a1d5640 1 -- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f402c0e0610 msgr2=0x7f402c0344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7f403a1d5640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f402c0e0610 0x7f402c0344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7f403a1d5640 1 -- 172.21.15.47:0/332622576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c0e0dd0 con 0x7f402c02d6f0 2024-03-23T09:41:12.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7f993ed9d640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99400b2cb0 0x7f9940152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7f993ed9d640 1 -- 172.21.15.47:0/3059052544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99400ce500 msgr2=0x7f99400b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7f993ed9d640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99400ce500 0x7f99400b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7f993ed9d640 1 -- 172.21.15.47:0/3059052544 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993400d8a0 con 0x7f9940150810 2024-03-23T09:41:12.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7f0ce647e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc005080 0x7f0cd80d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7f40937fe640 1 -- 172.21.15.47:0/2917251019 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c011790 con 0x7f40a4058990 2024-03-23T09:41:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7f0ce647e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd81692d0 0x7f0cd8177af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7f0ce647e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cd8169750 0x7f0cd8178030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7f0ce647e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd804bb60 con 0x7f0ccc005080 2024-03-23T09:41:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7f0ce647e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd8168790 con 0x7f0cd81692d0 2024-03-23T09:41:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7f0ce647e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd8168a70 con 0x7f0cd8169750 2024-03-23T09:41:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7f40abfff640 1 -- 172.21.15.47:0/2917251019 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a403def0 con 0x7f40a4058990 2024-03-23T09:41:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7f40abfff640 1 -- 172.21.15.47:0/2917251019 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a403e4b0 con 0x7f40a4058990 2024-03-23T09:41:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7f40937fe640 1 -- 172.21.15.47:0/2917251019 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c0118f0 con 0x7f40a4058990 2024-03-23T09:41:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7faf18ff9640 1 -- 172.21.15.47:0/1589273956 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1400a7a0 con 0x7faf1c15e3c0 2024-03-23T09:41:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7f40937fe640 1 -- 172.21.15.47:0/2917251019 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c011cc0 con 0x7f40a4058990 2024-03-23T09:41:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7f0ce4c7b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd81692d0 0x7f0cd8177af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7f0ce5c7d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cd8169750 0x7f0cd8178030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.087+0000 7fd3267fc640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd32004f8d0 0x7fd3200e7c50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd3180090b0 tx=0x7fd3180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7fd31f7fe640 1 -- 172.21.15.47:0/3620664449 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31803e070 con 0x7fd32004f8d0 2024-03-23T09:41:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7fd31f7fe640 1 -- 172.21.15.47:0/3620664449 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd318039b00 con 0x7fd32004f8d0 2024-03-23T09:41:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7fd31f7fe640 1 -- 172.21.15.47:0/3620664449 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318039e90 con 0x7fd32004f8d0 2024-03-23T09:41:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7f0ce4c7b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd81692d0 0x7f0cd8177af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:46518/0 (socket says 172.21.15.47:46518) 2024-03-23T09:41:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7fd327fff640 1 -- 172.21.15.47:0/3620664449 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd320048ba0 con 0x7fd32004f8d0 2024-03-23T09:41:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7f0ce4c7b640 1 -- 172.21.15.47:0/2190116685 learned_addr learned my addr 172.21.15.47:0/2190116685 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7fcc34cd5640 1 Processor -- start 2024-03-23T09:41:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7fcc34cd5640 1 -- start start 2024-03-23T09:41:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.088+0000 7f403a1d5640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f402c02d6f0 0x7f402c02dba0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f40200105f0 tx=0x7f4024004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7f0ce547c640 1 --2- 172.21.15.47:0/2190116685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc005080 0x7f0cd80d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7fd327fff640 1 -- 172.21.15.47:0/3620664449 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd320049de0 con 0x7fd32004f8d0 2024-03-23T09:41:12.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7f993ed9d640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9940150810 0x7f99400ae4c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f993800c8f0 tx=0x7f993800cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7f0ce4c7b640 1 -- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cd8169750 msgr2=0x7f0cd8178030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7faf18ff9640 1 -- 172.21.15.47:0/1589273956 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1400ab90 con 0x7faf1c15e3c0 2024-03-23T09:41:12.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7faf027fc640 1 -- 172.21.15.47:0/1589273956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faefc000fc0 con 0x7faf1c15e3c0 2024-03-23T09:41:12.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7f40abfff640 1 -- 172.21.15.47:0/2917251019 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f408c000fc0 con 0x7f40a4058990 2024-03-23T09:41:12.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7f0ce4c7b640 1 --2- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cd8169750 0x7f0cd8178030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7f0ce4c7b640 1 -- 172.21.15.47:0/2190116685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc005080 msgr2=0x7f0cd80d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7f0ce4c7b640 1 --2- 172.21.15.47:0/2190116685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ccc005080 0x7f0cd80d1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.089+0000 7faf18ff9640 1 -- 172.21.15.47:0/1412648872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf24159a50 con 0x7faf24158700 2024-03-23T09:41:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7f0ce4c7b640 1 -- 172.21.15.47:0/2190116685 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd8168d50 con 0x7f0cd81692d0 2024-03-23T09:41:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7f992effd640 1 -- 172.21.15.47:0/3059052544 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993800a640 con 0x7f9940150810 2024-03-23T09:41:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7f402a7fc640 1 -- 172.21.15.47:0/332622576 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024004220 con 0x7f402c02d6f0 2024-03-23T09:41:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7f992effd640 1 -- 172.21.15.47:0/3059052544 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993800a7a0 con 0x7f9940150810 2024-03-23T09:41:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993000b4b0 con 0x7f9940150810 2024-03-23T09:41:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc34cd5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc28006e60 0x7fcc280e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc34cd5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc2804ff90 0x7fcc280dae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc34cd5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc280eacc0 0x7fcc28037c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc34cd5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280e6310 con 0x7fcc2804ff90 2024-03-23T09:41:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc34cd5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280e4d00 con 0x7fcc280eacc0 2024-03-23T09:41:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc34cd5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2804ec50 con 0x7fcc28006e60 2024-03-23T09:41:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc2e59c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc28006e60 0x7fcc280e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc2e59c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc28006e60 0x7fcc280e47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:46688/0 (socket says 172.21.15.47:46688) 2024-03-23T09:41:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc2e59c640 1 -- 172.21.15.47:0/606731553 learned_addr learned my addr 172.21.15.47:0/606731553 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7f402a7fc640 1 -- 172.21.15.47:0/332622576 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4024004380 con 0x7f402c02d6f0 2024-03-23T09:41:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7f403bc38640 1 -- 172.21.15.47:0/332622576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402c0e10b0 con 0x7f402c02d6f0 2024-03-23T09:41:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7faf1affd640 1 -- 172.21.15.47:0/1412648872 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7faf14013400 con 0x7faf24158700 2024-03-23T09:41:12.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99400afff0 con 0x7f9940150810 2024-03-23T09:41:12.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc2ed9d640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc280eacc0 0x7fcc28037c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7f403bc38640 1 -- 172.21.15.47:0/332622576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c0e22a0 con 0x7f402c02d6f0 2024-03-23T09:41:12.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7f992effd640 1 -- 172.21.15.47:0/3059052544 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993800ab50 con 0x7f9940150810 2024-03-23T09:41:12.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc2e59c640 1 -- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc280eacc0 msgr2=0x7fcc28037c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc2e59c640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc280eacc0 0x7fcc28037c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc2e59c640 1 -- 172.21.15.47:0/606731553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc2804ff90 msgr2=0x7fcc280dae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc2e59c640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc2804ff90 0x7fcc280dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc2e59c640 1 -- 172.21.15.47:0/606731553 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc240094d0 con 0x7fcc28006e60 2024-03-23T09:41:12.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7f40937fe640 1 -- 172.21.15.47:0/2917251019 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f409c01c460 con 0x7f40a4058990 2024-03-23T09:41:12.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fd327fff640 1 -- 172.21.15.47:0/3620664449 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd320048750 con 0x7fd32004f8d0 2024-03-23T09:41:12.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.090+0000 7fcc2ed9d640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc280eacc0 0x7fcc28037c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:12.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7faf18ff9640 1 -- 172.21.15.47:0/1589273956 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7faf1400acf0 con 0x7faf1c15e3c0 2024-03-23T09:41:12.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7f40937fe640 1 --2- 172.21.15.47:0/2917251019 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806acd0 0x7f409806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7f40937fe640 1 -- 172.21.15.47:0/2917251019 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f409c089150 con 0x7f40a4058990 2024-03-23T09:41:12.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7faf1affd640 1 --2- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf0406ac30 0x7faf0406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7faf1affd640 1 -- 172.21.15.47:0/1412648872 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf140892d0 con 0x7faf24158700 2024-03-23T09:41:12.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7faf1affd640 1 --2- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x7faf04072dd0 0x7faf040751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7faf1affd640 1 -- 172.21.15.47:0/1412648872 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf040758a0 con 0x7faf04072dd0 2024-03-23T09:41:12.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7faf1affd640 1 -- 172.21.15.47:0/1412648872 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf1408b8b0 con 0x7faf24158700 2024-03-23T09:41:12.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7f402a7fc640 1 -- 172.21.15.47:0/332622576 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402403cb70 con 0x7f402c02d6f0 2024-03-23T09:41:12.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7f40937fe640 1 --2- 172.21.15.47:0/2917251019 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x7f4098072ef0 0x7f4098075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7f40937fe640 1 -- 172.21.15.47:0/2917251019 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40980759c0 con 0x7f4098072ef0 2024-03-23T09:41:12.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7f40937fe640 1 -- 172.21.15.47:0/2917251019 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f409c08b770 con 0x7f40a4058990 2024-03-23T09:41:12.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7faf2259c640 1 --2- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf0406ac30 0x7faf0406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7faf22d9d640 1 --2- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x7faf04072dd0 0x7faf040751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9924000fc0 con 0x7f9940150810 2024-03-23T09:41:12.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.091+0000 7fd31f7fe640 1 -- 172.21.15.47:0/3620664449 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd318042070 con 0x7fd32004f8d0 2024-03-23T09:41:12.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7faf18ff9640 1 --2- 172.21.15.47:0/1589273956 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf0806ac30 0x7faf0806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7fd31f7fe640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd30806ad00 0x7fd30806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7f0ce4c7b640 1 --2- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd81692d0 0x7f0cd8177af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000f570 tx=0x7f0cd000fa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7fcc2e59c640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc28006e60 0x7fcc280e47c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000c7f0 tx=0x7fcc2000ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7faf18ff9640 1 -- 172.21.15.47:0/1589273956 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf14088d20 con 0x7faf1c15e3c0 2024-03-23T09:41:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7f40aaffd640 1 --2- 172.21.15.47:0/2917251019 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806acd0 0x7f409806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7fcc1effd640 1 -- 172.21.15.47:0/606731553 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20007820 con 0x7fcc28006e60 2024-03-23T09:41:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7fcc1effd640 1 -- 172.21.15.47:0/606731553 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc20007980 con 0x7fcc28006e60 2024-03-23T09:41:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7fcc1effd640 1 -- 172.21.15.47:0/606731553 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20007cf0 con 0x7fcc28006e60 2024-03-23T09:41:12.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7f0cd5ffb640 1 -- 172.21.15.47:0/2190116685 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd00052e0 con 0x7f0cd81692d0 2024-03-23T09:41:12.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7fd31f7fe640 1 -- 172.21.15.47:0/3620664449 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3180ab850 con 0x7fd32004f8d0 2024-03-23T09:41:12.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7fd31f7fe640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x7fd308072d30 0x7fd308075150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7fd31f7fe640 1 -- 172.21.15.47:0/3620664449 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd308075800 con 0x7fd308072d30 2024-03-23T09:41:12.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7fd31f7fe640 1 -- 172.21.15.47:0/3620664449 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd318076aa0 con 0x7fd32004f8d0 2024-03-23T09:41:12.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7f0cd5ffb640 1 -- 172.21.15.47:0/2190116685 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0005440 con 0x7f0cd81692d0 2024-03-23T09:41:12.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7f992effd640 1 -- 172.21.15.47:0/3059052544 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f993800acb0 con 0x7f9940150810 2024-03-23T09:41:12.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7f0ce647e640 1 -- 172.21.15.47:0/2190116685 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd804bdb0 con 0x7f0cd81692d0 2024-03-23T09:41:12.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7faf2259c640 1 --2- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf0406ac30 0x7faf0406d0f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0059e0 tx=0x7faf0c000ef0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7f40aaffd640 1 --2- 172.21.15.47:0/2917251019 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806acd0 0x7f409806d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f40a400f910 tx=0x7f40ac06d030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.093+0000 7f40ab7fe640 1 --2- 172.21.15.47:0/2917251019 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x7f4098072ef0 0x7f4098075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.093+0000 7f0ce647e640 1 -- 172.21.15.47:0/2190116685 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd80d2420 con 0x7f0cd81692d0 2024-03-23T09:41:12.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.093+0000 7f0cd5ffb640 1 -- 172.21.15.47:0/2190116685 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0003810 con 0x7f0cd81692d0 2024-03-23T09:41:12.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7fcc34cd5640 1 -- 172.21.15.47:0/606731553 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2804dbb0 con 0x7fcc28006e60 2024-03-23T09:41:12.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7fcc34cd5640 1 -- 172.21.15.47:0/606731553 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2804e170 con 0x7fcc28006e60 2024-03-23T09:41:12.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.093+0000 7f992effd640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f992006ad00 0x7f992006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.093+0000 7fd326ffd640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd30806ad00 0x7fd30806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.093+0000 7f992effd640 1 -- 172.21.15.47:0/3059052544 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9938088c90 con 0x7f9940150810 2024-03-23T09:41:12.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.092+0000 7faf1bfff640 1 --2- 172.21.15.47:0/1589273956 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf0806ac30 0x7faf0806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.093+0000 7faf18ff9640 1 --2- 172.21.15.47:0/1589273956 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x7faf08072dd0 0x7faf080751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.093+0000 7faf1b7fe640 1 --2- 172.21.15.47:0/1589273956 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x7faf08072dd0 0x7faf080751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.093+0000 7f40ab7fe640 1 --2- 172.21.15.47:0/2917251019 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x7f4098072ef0 0x7f4098075310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.093+0000 7f992effd640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x7f9920072f20 0x7f9920075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.093+0000 7f403bc38640 1 -- 172.21.15.47:0/332622576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f402c0064b0 con 0x7f402c02d6f0 2024-03-23T09:41:12.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7f993dd9b640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x7f9920072f20 0x7f9920075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7fcc34cd5640 1 -- 172.21.15.47:0/606731553 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc28053f80 con 0x7fcc28006e60 2024-03-23T09:41:12.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7faf22d9d640 1 --2- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x7faf04072dd0 0x7faf040751f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7fd3277fe640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x7fd308072d30 0x7fd308075150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7faf1bfff640 1 --2- 172.21.15.47:0/1589273956 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf0806ac30 0x7faf0806d0f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf1003a000 tx=0x7faf10034000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7f993e59c640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f992006ad00 0x7f992006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7f992effd640 1 -- 172.21.15.47:0/3059052544 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99200759f0 con 0x7f9920072f20 2024-03-23T09:41:12.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7faf18ff9640 1 -- 172.21.15.47:0/1589273956 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf080758a0 con 0x7faf08072dd0 2024-03-23T09:41:12.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7faf1b7fe640 1 --2- 172.21.15.47:0/1589273956 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x7faf08072dd0 0x7faf080751f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7faf18ff9640 1 -- 172.21.15.47:0/1589273956 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf14053e60 con 0x7faf1c15e3c0 2024-03-23T09:41:12.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7f993dd9b640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x7f9920072f20 0x7f9920075340 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7fd3277fe640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x7fd308072d30 0x7fd308075150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7f402a7fc640 1 -- 172.21.15.47:0/332622576 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f402403ccd0 con 0x7f402c02d6f0 2024-03-23T09:41:12.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.095+0000 7fcc1effd640 1 -- 172.21.15.47:0/606731553 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcc2001a660 con 0x7fcc28006e60 2024-03-23T09:41:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7f993e59c640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f992006ad00 0x7f992006d1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f993002bce0 tx=0x7f9930030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.094+0000 7f0ce647e640 1 -- 172.21.15.47:0/2190116685 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0cc0000fc0 con 0x7f0cd81692d0 2024-03-23T09:41:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.095+0000 7f992effd640 1 -- 172.21.15.47:0/3059052544 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f993808e2a0 con 0x7f9940150810 2024-03-23T09:41:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.095+0000 7f402a7fc640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f401806ad00 0x7f401806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.095+0000 7f40937fe640 1 -- 172.21.15.47:0/2917251019 <== osd.2 v2:172.21.15.47:6800/37678 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f40980759c0 con 0x7f4098072ef0 2024-03-23T09:41:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.095+0000 7faf1affd640 1 -- 172.21.15.47:0/1412648872 <== osd.5 v2:172.21.15.175:6816/38161 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7faf040758a0 con 0x7faf04072dd0 2024-03-23T09:41:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.095+0000 7f402a7fc640 1 -- 172.21.15.47:0/332622576 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40240ab9f0 con 0x7f402c02d6f0 2024-03-23T09:41:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.095+0000 7f0cd5ffb640 1 -- 172.21.15.47:0/2190116685 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0cd0003970 con 0x7f0cd81692d0 2024-03-23T09:41:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.095+0000 7f402a7fc640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x7f4018072db0 0x7f40180751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.095+0000 7fd31f7fe640 1 -- 172.21.15.47:0/3620664449 <== osd.3 v2:172.21.15.47:6808/37683 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fd308075800 con 0x7fd308072d30 2024-03-23T09:41:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7fd326ffd640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd30806ad00 0x7fd30806d1c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd310009870 tx=0x7fd310008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7f402a7fc640 1 -- 172.21.15.47:0/332622576 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4018075880 con 0x7f4018072db0 2024-03-23T09:41:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.095+0000 7fcc1effd640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcc1006ac00 0x7fcc1006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7fcc1effd640 1 -- 172.21.15.47:0/606731553 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc20089a30 con 0x7fcc28006e60 2024-03-23T09:41:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7fcc1effd640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x7fcc10072da0 0x7fcc100751c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7fcc1effd640 1 -- 172.21.15.47:0/606731553 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc10075870 con 0x7fcc10072da0 2024-03-23T09:41:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7fcc1effd640 1 -- 172.21.15.47:0/606731553 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc2003c020 con 0x7fcc28006e60 2024-03-23T09:41:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7f0cd5ffb640 1 --2- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cc406acd0 0x7f0cc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7f40391d3640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x7f4018072db0 0x7f40180751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.095+0000 7f992effd640 1 -- 172.21.15.47:0/3059052544 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f99200759f0 con 0x7f9920072f20 2024-03-23T09:41:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7f0cd5ffb640 1 -- 172.21.15.47:0/2190116685 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cd0089410 con 0x7f0cd81692d0 2024-03-23T09:41:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7fcc2dd9b640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcc1006ac00 0x7fcc1006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7fcc2ed9d640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x7fcc10072da0 0x7fcc100751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7f0ce547c640 1 --2- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cc406acd0 0x7f0cc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7f0cd5ffb640 1 --2- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x7f0cc4072ef0 0x7f0cc4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7f0cd5ffb640 1 -- 172.21.15.47:0/2190116685 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cc40759c0 con 0x7f0cc4072ef0 2024-03-23T09:41:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7f0cd5ffb640 1 -- 172.21.15.47:0/2190116685 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd00897c0 con 0x7f0cd81692d0 2024-03-23T09:41:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7faf18ff9640 1 -- 172.21.15.47:0/1589273956 <== osd.0 v2:172.21.15.47:6816/37679 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7faf080758a0 con 0x7faf08072dd0 2024-03-23T09:41:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7f40399d4640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f401806ad00 0x7f401806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.096+0000 7f0ce5c7d640 1 --2- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x7f0cc4072ef0 0x7f0cc4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.097+0000 7fcc2dd9b640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcc1006ac00 0x7fcc1006d0c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcc2804d2a0 tx=0x7fcc30061990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.097+0000 7fcc2ed9d640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x7fcc10072da0 0x7fcc100751c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.097+0000 7f402a7fc640 1 -- 172.21.15.47:0/332622576 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40240b02a0 con 0x7f402c02d6f0 2024-03-23T09:41:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.097+0000 7f0ce5c7d640 1 --2- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x7f0cc4072ef0 0x7f0cc4075310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.097+0000 7f0ce547c640 1 --2- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cc406acd0 0x7f0cc406d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00bc80 tx=0x7f0cdc00f040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.097+0000 7f40391d3640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x7f4018072db0 0x7f40180751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.097+0000 7f40399d4640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f401806ad00 0x7f401806d1c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40300017e0 tx=0x7f4030007830 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.098+0000 7fcc1effd640 1 -- 172.21.15.47:0/606731553 <== osd.6 v2:172.21.15.175:6808/38160 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fcc10075870 con 0x7fcc10072da0 2024-03-23T09:41:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.098+0000 7f0cd5ffb640 1 -- 172.21.15.47:0/2190116685 <== osd.4 v2:172.21.15.175:6800/38159 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f0cc40759c0 con 0x7f0cc4072ef0 2024-03-23T09:41:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.098+0000 7f402a7fc640 1 -- 172.21.15.47:0/332622576 <== osd.7 v2:172.21.15.175:6824/38174 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f4018075880 con 0x7f4018072db0 2024-03-23T09:41:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.127+0000 7fd327fff640 1 -- 172.21.15.47:0/3620664449 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd3200545f0 con 0x7fd308072d30 2024-03-23T09:41:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.127+0000 7faf18ff9640 1 -- 172.21.15.47:0/1412648872 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faf24159770 con 0x7faf04072dd0 2024-03-23T09:41:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7fd31f7fe640 1 -- 172.21.15.47:0/3620664449 <== osd.3 v2:172.21.15.47:6808/37683 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd32804e5d0 con 0x7fd308072d30 2024-03-23T09:41:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9924002cb0 con 0x7f9920072f20 2024-03-23T09:41:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.127+0000 7f40abfff640 1 -- 172.21.15.47:0/2917251019 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f408c002ca0 con 0x7f4098072ef0 2024-03-23T09:41:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7f40937fe640 1 -- 172.21.15.47:0/2917251019 <== osd.2 v2:172.21.15.47:6800/37678 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f408c002ca0 con 0x7f4098072ef0 2024-03-23T09:41:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7f40abfff640 1 -- 172.21.15.47:0/2917251019 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x7f4098072ef0 msgr2=0x7f4098075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7f40abfff640 1 --2- 172.21.15.47:0/2917251019 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x7f4098072ef0 0x7f4098075310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7fd31d7fa640 1 -- 172.21.15.47:0/3620664449 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x7fd308072d30 msgr2=0x7fd308075150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7fd31d7fa640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x7fd308072d30 0x7fd308075150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7faf1affd640 1 -- 172.21.15.47:0/1412648872 <== osd.5 v2:172.21.15.175:6816/38161 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faf1c003970 con 0x7faf04072dd0 2024-03-23T09:41:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7faf027fc640 1 -- 172.21.15.47:0/1589273956 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faefc002cb0 con 0x7faf08072dd0 2024-03-23T09:41:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7f992effd640 1 -- 172.21.15.47:0/3059052544 <== osd.1 v2:172.21.15.47:6824/37680 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9924002cb0 con 0x7f9920072f20 2024-03-23T09:41:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7f40abfff640 1 -- 172.21.15.47:0/2917251019 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806acd0 msgr2=0x7f409806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7f40abfff640 1 --2- 172.21.15.47:0/2917251019 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806acd0 0x7f409806d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f40a400f910 tx=0x7f40ac06d030 comp rx=0 tx=0).stop 2024-03-23T09:41:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7f40abfff640 1 -- 172.21.15.47:0/2917251019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4058990 msgr2=0x7f40a417f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7f40abfff640 1 --2- 172.21.15.47:0/2917251019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4058990 0x7f40a417f720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f409c00c430 tx=0x7f409c00c8f0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7fd31d7fa640 1 -- 172.21.15.47:0/3620664449 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd30806ad00 msgr2=0x7fd30806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7fd31d7fa640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd30806ad00 0x7fd30806d1c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd310009870 tx=0x7fd310008040 comp rx=0 tx=0).stop 2024-03-23T09:41:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7fd31d7fa640 1 -- 172.21.15.47:0/3620664449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd32004f8d0 msgr2=0x7fd3200e7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7fd31d7fa640 1 --2- 172.21.15.47:0/3620664449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd32004f8d0 0x7fd3200e7c50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd3180090b0 tx=0x7fd3180046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7fd3277fe640 1 -- 172.21.15.47:0/3620664449 reap_dead start 2024-03-23T09:41:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7fd31d7fa640 1 -- 172.21.15.47:0/3620664449 shutdown_connections 2024-03-23T09:41:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.128+0000 7fd31d7fa640 1 -- 172.21.15.47:0/3620664449 >> 172.21.15.47:0/3620664449 conn(0x7fd320009f80 msgr2=0x7fd320052db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7f40ab7fe640 1 -- 172.21.15.47:0/2917251019 reap_dead start 2024-03-23T09:41:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7fd31d7fa640 1 -- 172.21.15.47:0/3620664449 shutdown_connections 2024-03-23T09:41:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf18ff9640 1 -- 172.21.15.47:0/1589273956 <== osd.0 v2:172.21.15.47:6816/37679 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faefc002cb0 con 0x7faf08072dd0 2024-03-23T09:41:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7fd31d7fa640 1 -- 172.21.15.47:0/3620664449 wait complete. 2024-03-23T09:41:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf18ff9640 1 -- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x7faf04072dd0 msgr2=0x7faf040751f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf18ff9640 1 --2- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x7faf04072dd0 0x7faf040751f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7f40abfff640 1 -- 172.21.15.47:0/2917251019 shutdown_connections 2024-03-23T09:41:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7f40abfff640 1 -- 172.21.15.47:0/2917251019 >> 172.21.15.47:0/2917251019 conn(0x7f40a4058eb0 msgr2=0x7f40a4039ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf18ff9640 1 -- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf0406ac30 msgr2=0x7faf0406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf18ff9640 1 --2- 172.21.15.47:0/1412648872 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf0406ac30 0x7faf0406d0f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0059e0 tx=0x7faf0c000ef0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf18ff9640 1 -- 172.21.15.47:0/1412648872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf24158700 msgr2=0x7faf24156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf18ff9640 1 --2- 172.21.15.47:0/1412648872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf24158700 0x7faf24156f70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faf14009fd0 tx=0x7faf14011920 comp rx=0 tx=0).stop 2024-03-23T09:41:12.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf027fc640 1 -- 172.21.15.47:0/1589273956 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x7faf08072dd0 msgr2=0x7faf080751f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf22d9d640 1 -- 172.21.15.47:0/1412648872 reap_dead start 2024-03-23T09:41:12.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf18ff9640 1 -- 172.21.15.47:0/1412648872 shutdown_connections 2024-03-23T09:41:12.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf18ff9640 1 -- 172.21.15.47:0/1412648872 >> 172.21.15.47:0/1412648872 conn(0x7faf240659d0 msgr2=0x7faf241479f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7f40abfff640 1 -- 172.21.15.47:0/2917251019 shutdown_connections 2024-03-23T09:41:12.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7f40abfff640 1 -- 172.21.15.47:0/2917251019 wait complete. 2024-03-23T09:41:12.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf18ff9640 1 -- 172.21.15.47:0/1412648872 shutdown_connections 2024-03-23T09:41:12.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf18ff9640 1 -- 172.21.15.47:0/1412648872 wait complete. 2024-03-23T09:41:12.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7faf027fc640 1 --2- 172.21.15.47:0/1589273956 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x7faf08072dd0 0x7faf080751f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x7f9920072f20 msgr2=0x7f9920075340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7f9944b37640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x7f9920072f20 0x7f9920075340 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f992006ad00 msgr2=0x7f992006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7f9944b37640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f992006ad00 0x7f992006d1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f993002bce0 tx=0x7f9930030000 comp rx=0 tx=0).stop 2024-03-23T09:41:12.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9940150810 msgr2=0x7f99400ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7f9944b37640 1 --2- 172.21.15.47:0/3059052544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9940150810 0x7f99400ae4c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f993800c8f0 tx=0x7f993800cdb0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.129+0000 7f993ed9d640 1 -- 172.21.15.47:0/3059052544 reap_dead start 2024-03-23T09:41:12.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 shutdown_connections 2024-03-23T09:41:12.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 >> 172.21.15.47:0/3059052544 conn(0x7f994006a310 msgr2=0x7f994015c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 shutdown_connections 2024-03-23T09:41:12.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7f9944b37640 1 -- 172.21.15.47:0/3059052544 wait complete. 2024-03-23T09:41:12.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7f403bc38640 1 -- 172.21.15.47:0/332622576 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f402c005350 con 0x7f4018072db0 2024-03-23T09:41:12.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7faf027fc640 1 -- 172.21.15.47:0/1589273956 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf0806ac30 msgr2=0x7faf0806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7faf027fc640 1 --2- 172.21.15.47:0/1589273956 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf0806ac30 0x7faf0806d0f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf1003a000 tx=0x7faf10034000 comp rx=0 tx=0).stop 2024-03-23T09:41:12.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7faf027fc640 1 -- 172.21.15.47:0/1589273956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf1c15e3c0 msgr2=0x7faf1c156320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7faf027fc640 1 --2- 172.21.15.47:0/1589273956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf1c15e3c0 0x7faf1c156320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faf1400c7e0 tx=0x7faf1400cca0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7faf20828640 1 -- 172.21.15.47:0/1589273956 reap_dead start 2024-03-23T09:41:12.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7faf027fc640 1 -- 172.21.15.47:0/1589273956 shutdown_connections 2024-03-23T09:41:12.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7faf027fc640 1 -- 172.21.15.47:0/1589273956 >> 172.21.15.47:0/1589273956 conn(0x7faf1c0b21d0 msgr2=0x7faf1c06f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7faf027fc640 1 -- 172.21.15.47:0/1589273956 shutdown_connections 2024-03-23T09:41:12.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7faf027fc640 1 -- 172.21.15.47:0/1589273956 wait complete. 2024-03-23T09:41:12.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7fcc34cd5640 1 -- 172.21.15.47:0/606731553 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc280525a0 con 0x7fcc10072da0 2024-03-23T09:41:12.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7f402a7fc640 1 -- 172.21.15.47:0/332622576 <== osd.7 v2:172.21.15.175:6824/38174 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f402c005350 con 0x7f4018072db0 2024-03-23T09:41:12.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.130+0000 7f0ce647e640 1 -- 172.21.15.47:0/2190116685 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0cc0002cb0 con 0x7f0cc4072ef0 2024-03-23T09:41:12.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.131+0000 7f0cd5ffb640 1 -- 172.21.15.47:0/2190116685 <== osd.4 v2:172.21.15.175:6800/38159 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0cc0002cb0 con 0x7f0cc4072ef0 2024-03-23T09:41:12.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.131+0000 7fcc1effd640 1 -- 172.21.15.47:0/606731553 <== osd.6 v2:172.21.15.175:6808/38160 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcc280525a0 con 0x7fcc10072da0 2024-03-23T09:41:12.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.131+0000 7f403bc38640 1 -- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x7f4018072db0 msgr2=0x7f40180751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.131+0000 7f403bc38640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x7f4018072db0 0x7f40180751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.131+0000 7f403bc38640 1 -- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f401806ad00 msgr2=0x7f401806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.131+0000 7f403bc38640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f401806ad00 0x7f401806d1c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40300017e0 tx=0x7f4030007830 comp rx=0 tx=0).stop 2024-03-23T09:41:12.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.131+0000 7f403bc38640 1 -- 172.21.15.47:0/332622576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f402c02d6f0 msgr2=0x7f402c02dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.131+0000 7f403bc38640 1 --2- 172.21.15.47:0/332622576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f402c02d6f0 0x7f402c02dba0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f40200105f0 tx=0x7f4024004730 comp rx=0 tx=0).stop 2024-03-23T09:41:12.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.131+0000 7f403a1d5640 1 -- 172.21.15.47:0/332622576 reap_dead start 2024-03-23T09:41:12.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.131+0000 7f0ce647e640 1 -- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x7f0cc4072ef0 msgr2=0x7f0cc4075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.131+0000 7f0ce647e640 1 --2- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x7f0cc4072ef0 0x7f0cc4075310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.131+0000 7fcc34cd5640 1 -- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x7fcc10072da0 msgr2=0x7fcc100751c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.131+0000 7fcc34cd5640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x7fcc10072da0 0x7fcc100751c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7f0ce647e640 1 -- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cc406acd0 msgr2=0x7f0cc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7f0ce647e640 1 --2- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cc406acd0 0x7f0cc406d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00bc80 tx=0x7f0cdc00f040 comp rx=0 tx=0).stop 2024-03-23T09:41:12.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7f0ce647e640 1 -- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd81692d0 msgr2=0x7f0cd8177af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7f0ce647e640 1 --2- 172.21.15.47:0/2190116685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd81692d0 0x7f0cd8177af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000f570 tx=0x7f0cd000fa30 comp rx=0 tx=0).stop 2024-03-23T09:41:12.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7f0ce5c7d640 1 -- 172.21.15.47:0/2190116685 reap_dead start 2024-03-23T09:41:12.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7f0ce647e640 1 -- 172.21.15.47:0/2190116685 shutdown_connections 2024-03-23T09:41:12.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7fcc34cd5640 1 -- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcc1006ac00 msgr2=0x7fcc1006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7f0ce647e640 1 -- 172.21.15.47:0/2190116685 >> 172.21.15.47:0/2190116685 conn(0x7f0cd8009f80 msgr2=0x7f0cd80068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7fcc34cd5640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcc1006ac00 0x7fcc1006d0c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcc2804d2a0 tx=0x7fcc30061990 comp rx=0 tx=0).stop 2024-03-23T09:41:12.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7fcc34cd5640 1 -- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc28006e60 msgr2=0x7fcc280e47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7fcc34cd5640 1 --2- 172.21.15.47:0/606731553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc28006e60 0x7fcc280e47c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000c7f0 tx=0x7fcc2000ccb0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7f403bc38640 1 -- 172.21.15.47:0/332622576 shutdown_connections 2024-03-23T09:41:12.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7f403bc38640 1 -- 172.21.15.47:0/332622576 >> 172.21.15.47:0/332622576 conn(0x7f402c009f70 msgr2=0x7f402c04db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7fcc2ed9d640 1 -- 172.21.15.47:0/606731553 reap_dead start 2024-03-23T09:41:12.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7fcc34cd5640 1 -- 172.21.15.47:0/606731553 shutdown_connections 2024-03-23T09:41:12.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7fcc34cd5640 1 -- 172.21.15.47:0/606731553 >> 172.21.15.47:0/606731553 conn(0x7fcc28009f70 msgr2=0x7fcc28037230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7fcc34cd5640 1 -- 172.21.15.47:0/606731553 shutdown_connections 2024-03-23T09:41:12.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7f403bc38640 1 -- 172.21.15.47:0/332622576 shutdown_connections 2024-03-23T09:41:12.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7fcc34cd5640 1 -- 172.21.15.47:0/606731553 wait complete. 2024-03-23T09:41:12.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.132+0000 7f403bc38640 1 -- 172.21.15.47:0/332622576 wait complete. 2024-03-23T09:41:12.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.133+0000 7f0ce647e640 1 -- 172.21.15.47:0/2190116685 shutdown_connections 2024-03-23T09:41:12.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.133+0000 7f0ce647e640 1 -- 172.21.15.47:0/2190116685 wait complete. 2024-03-23T09:41:12.214 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-23T09:41:12.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-23T09:41:12.215 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-23T09:41:12.216 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-23T09:41:12.216 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-23T09:41:12.217 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-23T09:41:12.218 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-23T09:41:12.218 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-23T09:41:12.218 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-23T09:41:12.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-23T09:41:12.219 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-23T09:41:12.220 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-23T09:41:12.221 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-23T09:41:12.221 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-23T09:41:12.222 INFO:teuthology.orchestra.run.smithi047.stdout:55834574854 2024-03-23T09:41:12.222 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-23T09:41:12.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.428+0000 7fc684911640 1 Processor -- start 2024-03-23T09:41:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.429+0000 7ffbb8f20640 1 Processor -- start 2024-03-23T09:41:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.428+0000 7f2a8bdc4640 1 Processor -- start 2024-03-23T09:41:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.428+0000 7ff673fff640 1 Processor -- start 2024-03-23T09:41:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.428+0000 7ff673fff640 1 -- start start 2024-03-23T09:41:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.429+0000 7ff673fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff674070de0 0x7ff6740caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.429+0000 7ff673fff640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6740cceb0 con 0x7ff674160a80 2024-03-23T09:41:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.429+0000 7ff673fff640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6740cc2a0 con 0x7ff674160e60 2024-03-23T09:41:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.429+0000 7ff673fff640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740cc580 con 0x7ff674070de0 2024-03-23T09:41:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.429+0000 7f9b78755640 1 Processor -- start 2024-03-23T09:41:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.429+0000 7f9b78755640 1 -- start start 2024-03-23T09:41:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7fe76bfff640 1 Processor -- start 2024-03-23T09:41:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.429+0000 7ffbb8f20640 1 -- start start 2024-03-23T09:41:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.428+0000 7fc684911640 1 -- start start 2024-03-23T09:41:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7fc684911640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6800ceaa0 0x7fc6801611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7fc684911640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6801618c0 con 0x7fc6800b2450 2024-03-23T09:41:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7fc684911640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc680161ba0 con 0x7fc68006cf40 2024-03-23T09:41:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7fc684911640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc680161e80 con 0x7fc6800ceaa0 2024-03-23T09:41:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7fc67ed9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc6800b2450 0x7fc680162960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57316/0 (socket says 172.21.15.47:57316) 2024-03-23T09:41:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7fc67ed9d640 1 -- 172.21.15.47:0/595579080 learned_addr learned my addr 172.21.15.47:0/595579080 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:12.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f1b6b54c640 1 Processor -- start 2024-03-23T09:41:12.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.428+0000 7f2a8bdc4640 1 -- start start 2024-03-23T09:41:12.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7fe76bfff640 1 -- start start 2024-03-23T09:41:12.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7ff672ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff674160a80 0x7ff6740b1eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57300/0 (socket says 172.21.15.47:57300) 2024-03-23T09:41:12.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7ff672ffd640 1 -- 172.21.15.47:0/815436565 learned_addr learned my addr 172.21.15.47:0/815436565 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:12.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7ff6737fe640 1 --2- 172.21.15.47:0/815436565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff674070de0 0x7ff6740caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f9b78755640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9b7006b420 0x7f9b700add10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f9b78755640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9b7006e6b0 0x7f9b70069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f9b764f1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9b7006b420 0x7f9b700add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f9b78755640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b700ce6b0 con 0x7f9b70165770 2024-03-23T09:41:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f9b78755640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b70161720 con 0x7f9b7006b420 2024-03-23T09:41:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f9b78755640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b70165ce0 con 0x7f9b7006e6b0 2024-03-23T09:41:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f9b764f1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9b7006b420 0x7f9b700add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46152/0 (socket says 172.21.15.47:46152) 2024-03-23T09:41:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f9b764f1640 1 -- 172.21.15.47:0/609217128 learned_addr learned my addr 172.21.15.47:0/609217128 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f2a8bdc4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2a8415ff60 0x7f2a840cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f2a8bdc4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2a840b1b20 0x7f2a840cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f2a8bdc4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a84160d60 con 0x7f2a840b81f0 2024-03-23T09:41:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f2a8bdc4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a84160340 con 0x7f2a840b1b20 2024-03-23T09:41:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f2a8bdc4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a840b8680 con 0x7f2a8415ff60 2024-03-23T09:41:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.429+0000 7f7b88bfd640 1 Processor -- start 2024-03-23T09:41:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.429+0000 7f7b88bfd640 1 -- start start 2024-03-23T09:41:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7fc67e59c640 1 --2- 172.21.15.47:0/595579080 >> v2:172.21.15.47:3300/0 conn(0x7fc6800ceaa0 0x7fc6801611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f9b76cf2640 1 --2- 172.21.15.47:0/609217128 >> v2:172.21.15.175:3300/0 conn(0x7f9b7006e6b0 0x7f9b70069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307423262 0 0) 0x7fc6801618c0 con 0x7fc6800b2450 2024-03-23T09:41:12.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc668003520 con 0x7fc6800b2450 2024-03-23T09:41:12.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7f2a89b60640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2a8415ff60 0x7f2a840cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7f2a89b60640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2a8415ff60 0x7f2a840cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:46548/0 (socket says 172.21.15.47:46548) 2024-03-23T09:41:12.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7f2a89b60640 1 -- 172.21.15.47:0/3214600583 learned_addr learned my addr 172.21.15.47:0/3214600583 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7ffbb8f20640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffbb415de00 0x7ffbb40ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7ffbb8f20640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffbb4099110 0x7ffbb40b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7ffbb8f20640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb40cbea0 con 0x7ffbb40ce5f0 2024-03-23T09:41:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7ffbb8f20640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb415e1e0 con 0x7ffbb4099110 2024-03-23T09:41:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7ffbb8f20640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb40cb290 con 0x7ffbb415de00 2024-03-23T09:41:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.430+0000 7f1b6b54c640 1 -- start start 2024-03-23T09:41:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7ffbb1d9b640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7ffbb40ce5f0 0x7ffbb4098a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47902/0 (socket says 172.21.15.47:47902) 2024-03-23T09:41:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7ffbb1d9b640 1 -- 172.21.15.47:0/3460973216 learned_addr learned my addr 172.21.15.47:0/3460973216 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7ffbb2d9d640 1 --2- 172.21.15.47:0/3460973216 >> v2:172.21.15.47:3300/0 conn(0x7ffbb4099110 0x7ffbb40b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7f2a8a361640 1 --2- 172.21.15.47:0/3214600583 >> v2:172.21.15.47:3300/0 conn(0x7f2a840b1b20 0x7f2a840cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3347688684 0 0) 0x7fc668003520 con 0x7fc6800b2450 2024-03-23T09:41:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 >> v2:172.21.15.47:3300/0 conn(0x7fc6800ceaa0 msgr2=0x7fc6801611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f1b6b54c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b640b5540 0x7f1b640d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f1b6b54c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b640b4bc0 con 0x7f1b640d1eb0 2024-03-23T09:41:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f1b6b54c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b640b2050 con 0x7f1b640b40d0 2024-03-23T09:41:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f1b6b54c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b64071ae0 con 0x7f1b640b5540 2024-03-23T09:41:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7fe76bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe76c0ccd30 0x7fe76c161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7fe76bfff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c1651f0 0x7fe76c165640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7fe76bfff640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76c162460 con 0x7fe76c0b4160 2024-03-23T09:41:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7fe76bfff640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76c0d05d0 con 0x7fe76c0ccd30 2024-03-23T09:41:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7fe76bfff640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c164ac0 con 0x7fe76c1651f0 2024-03-23T09:41:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f1b6a54a640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f1b640b40d0 0x7f1b640b44b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47904/0 (socket says 172.21.15.47:47904) 2024-03-23T09:41:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7f7b88bfd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b7c0069a0 0x7f7b7c006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7fe76a7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe76c0ccd30 0x7fe76c161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7fe76a7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe76c0ccd30 0x7fe76c161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46200/0 (socket says 172.21.15.47:46200) 2024-03-23T09:41:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7fe76a7fc640 1 -- 172.21.15.47:0/1952538669 learned_addr learned my addr 172.21.15.47:0/1952538669 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7fe76affd640 1 --1- 172.21.15.47:0/1952538669 >> v1:172.21.15.47:6789/0 conn(0x7fe76c0b4160 0x7fe76c0b4540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57340/0 (socket says 172.21.15.47:57340) 2024-03-23T09:41:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7f7b88bfd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7c0542f0 con 0x7f7b7c005e50 2024-03-23T09:41:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7f7b88bfd640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7c0536e0 con 0x7f7b7c0eacc0 2024-03-23T09:41:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7f7b88bfd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c0539c0 con 0x7f7b7c0069a0 2024-03-23T09:41:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7f7b82d9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b7c0069a0 0x7f7b7c006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7f7b82d9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b7c0069a0 0x7f7b7c006e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:46698/0 (socket says 172.21.15.47:46698) 2024-03-23T09:41:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7f7b81d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7b7c005e50 0x7f7b7c006290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57322/0 (socket says 172.21.15.47:57322) 2024-03-23T09:41:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7f7b82d9d640 1 -- 172.21.15.47:0/1446612046 learned_addr learned my addr 172.21.15.47:0/1446612046 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f7b82d9d640 1 -- 172.21.15.47:0/1446612046 >> v1:172.21.15.175:6789/0 conn(0x7f7b7c0eacc0 legacy=0x7f7b7c04fa50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T09:41:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f7b82d9d640 1 -- 172.21.15.47:0/1446612046 >> v1:172.21.15.47:6789/0 conn(0x7f7b7c005e50 legacy=0x7f7b7c006290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 --2- 172.21.15.47:0/595579080 >> v2:172.21.15.47:3300/0 conn(0x7fc6800ceaa0 0x7fc6801611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 >> v1:172.21.15.175:6789/0 conn(0x7fc68006cf40 legacy=0x7fc6800b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc680162160 con 0x7fc6800b2450 2024-03-23T09:41:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fc678001a20 con 0x7fc6800b2450 2024-03-23T09:41:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f9b76cf2640 1 -- 172.21.15.47:0/609217128 >> v2:172.21.15.47:3300/0 conn(0x7f9b7006b420 msgr2=0x7f9b700add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f9b76cf2640 1 --2- 172.21.15.47:0/609217128 >> v2:172.21.15.47:3300/0 conn(0x7f9b7006b420 0x7f9b700add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 >> v1:172.21.15.47:6789/0 conn(0x7fc6800b2450 legacy=0x7fc680162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f9b76cf2640 1 -- 172.21.15.47:0/609217128 >> v1:172.21.15.175:6789/0 conn(0x7f9b70165770 legacy=0x7f9b7006dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f9b76cf2640 1 -- 172.21.15.47:0/609217128 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b70065b70 con 0x7f9b7006e6b0 2024-03-23T09:41:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 --2- 172.21.15.47:0/595579080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc668003970 0x7fc668003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 --2- 172.21.15.47:0/595579080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6680047c0 0x7fc668004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 --2- 172.21.15.47:0/595579080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc668005180 0x7fc66800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc680161e80 con 0x7fc668003970 2024-03-23T09:41:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66800c0d0 con 0x7fc668005180 2024-03-23T09:41:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66800c290 con 0x7fc6680047c0 2024-03-23T09:41:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7fc67e59c640 1 --2- 172.21.15.47:0/595579080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc668003970 0x7fc668003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7ff671ffb640 1 -- 172.21.15.47:0/815436565 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485959801 0 0) 0x7ff6740cceb0 con 0x7ff674160a80 2024-03-23T09:41:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7ff671ffb640 1 -- 172.21.15.47:0/815436565 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff65c003520 con 0x7ff674160a80 2024-03-23T09:41:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7ff6737fe640 1 -- 172.21.15.47:0/815436565 >> v1:172.21.15.175:6789/0 conn(0x7ff674160e60 legacy=0x7ff6740706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7fc67e59c640 1 -- 172.21.15.47:0/595579080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6680047c0 msgr2=0x7fc668004c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T09:41:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7fc67e59c640 1 --2- 172.21.15.47:0/595579080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6680047c0 0x7fc668004c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7fc67e59c640 1 -- 172.21.15.47:0/595579080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc668005180 msgr2=0x7fc66800b980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T09:41:12.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7fc67e59c640 1 --2- 172.21.15.47:0/595579080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc668005180 0x7fc66800b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7fc67e59c640 1 -- 172.21.15.47:0/595579080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc680162160 con 0x7fc668003970 2024-03-23T09:41:12.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7ff6737fe640 1 -- 172.21.15.47:0/815436565 >> v1:172.21.15.47:6789/0 conn(0x7ff674160a80 legacy=0x7ff6740b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7ff6737fe640 1 -- 172.21.15.47:0/815436565 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740cc860 con 0x7ff674070de0 2024-03-23T09:41:12.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7fc67ed9d640 1 -- 172.21.15.47:0/595579080 reap_dead start 2024-03-23T09:41:12.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f2a8a361640 1 -- 172.21.15.47:0/3214600583 >> v2:172.21.15.175:3300/0 conn(0x7f2a8415ff60 msgr2=0x7f2a840cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f2a8a361640 1 --2- 172.21.15.47:0/3214600583 >> v2:172.21.15.175:3300/0 conn(0x7f2a8415ff60 0x7f2a840cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f2a8a361640 1 -- 172.21.15.47:0/3214600583 >> v1:172.21.15.175:6789/0 conn(0x7f2a840b81f0 legacy=0x7f2a840d2860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T09:41:12.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f2a8a361640 1 -- 172.21.15.47:0/3214600583 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a84160710 con 0x7f2a840b1b20 2024-03-23T09:41:12.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7f9b76cf2640 1 --2- 172.21.15.47:0/609217128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b7006e6b0 0x7f9b70069ce0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b700701e0 tx=0x7f9b6802bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4583434fc3647161 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f7b82d9d640 1 -- 172.21.15.47:0/1446612046 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c053ca0 con 0x7f7b7c0069a0 2024-03-23T09:41:12.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.431+0000 7ffbb159a640 1 -- 172.21.15.47:0/3460973216 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912325268 0 0) 0x7ffbb40cbea0 con 0x7ffbb40ce5f0 2024-03-23T09:41:12.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7ffbb159a640 1 -- 172.21.15.47:0/3460973216 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb9c003520 con 0x7ffbb40ce5f0 2024-03-23T09:41:12.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f1b6ad4b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1b640d1eb0 0x7f1b640d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57324/0 (socket says 172.21.15.47:57324) 2024-03-23T09:41:12.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f1b6a54a640 1 -- 172.21.15.47:0/501956883 learned_addr learned my addr 172.21.15.47:0/501956883 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:12.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7f1b69d49640 1 --2- 172.21.15.47:0/501956883 >> v2:172.21.15.47:3300/0 conn(0x7f1b640b5540 0x7f1b640d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7ffbb2d9d640 1 -- 172.21.15.47:0/3460973216 >> v2:172.21.15.175:3300/0 conn(0x7ffbb415de00 msgr2=0x7ffbb40ce0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T09:41:12.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7ffbb2d9d640 1 --2- 172.21.15.47:0/3460973216 >> v2:172.21.15.175:3300/0 conn(0x7ffbb415de00 0x7ffbb40ce0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7ffbb2d9d640 1 -- 172.21.15.47:0/3460973216 >> v1:172.21.15.175:6789/0 conn(0x7ffbb40ce5f0 legacy=0x7ffbb4098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.432+0000 7ffbb2d9d640 1 -- 172.21.15.47:0/3460973216 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb40cb570 con 0x7ffbb4099110 2024-03-23T09:41:12.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7f7b82d9d640 1 --2- 172.21.15.47:0/1446612046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b7c0069a0 0x7f7b7c006e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b84066ac0 tx=0x7f7b840a54d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3827f38b2d2d85ba server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7f1b69d49640 1 -- 172.21.15.47:0/501956883 >> v1:172.21.15.175:6789/0 conn(0x7f1b640b40d0 legacy=0x7f1b640b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7f9b754ef640 1 -- 172.21.15.47:0/609217128 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6802cb90 con 0x7f9b7006e6b0 2024-03-23T09:41:12.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7f9b754ef640 1 -- 172.21.15.47:0/609217128 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6802ccf0 con 0x7f9b7006e6b0 2024-03-23T09:41:12.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7f9b754ef640 1 -- 172.21.15.47:0/609217128 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6802cea0 con 0x7f9b7006e6b0 2024-03-23T09:41:12.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7f1b69d49640 1 -- 172.21.15.47:0/501956883 >> v1:172.21.15.47:6789/0 conn(0x7f1b640d1eb0 legacy=0x7f1b640d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7f2a8a361640 1 --2- 172.21.15.47:0/3214600583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a840b1b20 0x7f2a840cfd60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2a840b4e10 tx=0x7f2a8002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b0fd1478aafa40 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7fe76b7fe640 1 --2- 172.21.15.47:0/1952538669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c1651f0 0x7fe76c165640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7fe76a7fc640 1 -- 172.21.15.47:0/1952538669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c1651f0 msgr2=0x7fe76c165640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7fe76a7fc640 1 --2- 172.21.15.47:0/1952538669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c1651f0 0x7fe76c165640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7fe76a7fc640 1 -- 172.21.15.47:0/1952538669 >> v1:172.21.15.47:6789/0 conn(0x7fe76c0b4160 legacy=0x7fe76c0b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7fe76a7fc640 1 -- 172.21.15.47:0/1952538669 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c0cd260 con 0x7fe76c0ccd30 2024-03-23T09:41:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7f1b69d49640 1 -- 172.21.15.47:0/501956883 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b640b22a0 con 0x7f1b640b5540 2024-03-23T09:41:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7ff6737fe640 1 --2- 172.21.15.47:0/815436565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff674070de0 0x7ff6740caac0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009df0 tx=0x7ff66c02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fedb3eb3dc9e3d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.433+0000 7f1b69548640 1 -- 172.21.15.47:0/501956883 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159947350 0 0) 0x7f1b640b2050 con 0x7f1b640b40d0 2024-03-23T09:41:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7f2a88b5e640 1 -- 172.21.15.47:0/3214600583 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80031740 con 0x7f2a840b1b20 2024-03-23T09:41:12.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7f7b8159a640 1 -- 172.21.15.47:0/1446612046 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b840aa7d0 con 0x7f7b7c0069a0 2024-03-23T09:41:12.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7f7b8159a640 1 -- 172.21.15.47:0/1446612046 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b840a8050 con 0x7f7b7c0069a0 2024-03-23T09:41:12.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7f7b8159a640 1 -- 172.21.15.47:0/1446612046 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b840aabb0 con 0x7f7b7c0069a0 2024-03-23T09:41:12.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7f2a88b5e640 1 -- 172.21.15.47:0/3214600583 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a800318a0 con 0x7f2a840b1b20 2024-03-23T09:41:12.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7f2a88b5e640 1 -- 172.21.15.47:0/3214600583 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80031c70 con 0x7f2a840b1b20 2024-03-23T09:41:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7ff671ffb640 1 -- 172.21.15.47:0/815436565 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c02cc40 con 0x7ff674070de0 2024-03-23T09:41:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7ff671ffb640 1 -- 172.21.15.47:0/815436565 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c02cda0 con 0x7ff674070de0 2024-03-23T09:41:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7ff671ffb640 1 -- 172.21.15.47:0/815436565 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c034660 con 0x7ff674070de0 2024-03-23T09:41:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7fc67e59c640 1 --2- 172.21.15.47:0/595579080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc668003970 0x7fc668003d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc67400b5a0 tx=0x7fc6680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60911a9faa92fbe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7f9b78755640 1 -- 172.21.15.47:0/609217128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b7006e6b0 msgr2=0x7f9b70069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7f9b78755640 1 --2- 172.21.15.47:0/609217128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b7006e6b0 0x7f9b70069ce0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b700701e0 tx=0x7f9b6802bde0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7fe76a7fc640 1 --2- 172.21.15.47:0/1952538669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0ccd30 0x7fe76c161f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe764009e50 tx=0x7fe76402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d827ca8e582ae6e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7f9b78755640 1 -- 172.21.15.47:0/609217128 shutdown_connections 2024-03-23T09:41:12.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7f9b78755640 1 --2- 172.21.15.47:0/609217128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b7006e6b0 0x7f9b70069ce0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7f9b78755640 1 --2- 172.21.15.47:0/609217128 >> v2:172.21.15.47:3300/0 conn(0x7f9b7006b420 0x7f9b700add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7f9b78755640 1 -- 172.21.15.47:0/609217128 >> 172.21.15.47:0/609217128 conn(0x7f9b700cb990 msgr2=0x7f9b700cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7f1b69d49640 1 --2- 172.21.15.47:0/501956883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b640b5540 0x7f1b640d1970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009790 tx=0x7f1b6002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed48e644e50923 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f9b78755640 1 -- 172.21.15.47:0/609217128 shutdown_connections 2024-03-23T09:41:12.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f9b78755640 1 -- 172.21.15.47:0/609217128 wait complete. 2024-03-23T09:41:12.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc674043650 con 0x7fc668003970 2024-03-23T09:41:12.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6740437b0 con 0x7fc668003970 2024-03-23T09:41:12.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7fc67d59a640 1 -- 172.21.15.47:0/595579080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc674043b20 con 0x7fc668003970 2024-03-23T09:41:12.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f7b88bfd640 1 -- 172.21.15.47:0/1446612046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b7c0069a0 msgr2=0x7f7b7c006e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f7b88bfd640 1 --2- 172.21.15.47:0/1446612046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b7c0069a0 0x7f7b7c006e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b84066ac0 tx=0x7f7b840a54d0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7fe769ffb640 1 -- 172.21.15.47:0/1952538669 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76402cb00 con 0x7fe76c0ccd30 2024-03-23T09:41:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f7b88bfd640 1 -- 172.21.15.47:0/1446612046 shutdown_connections 2024-03-23T09:41:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f7b88bfd640 1 --2- 172.21.15.47:0/1446612046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b7c0069a0 0x7f7b7c006e30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f7b88bfd640 1 -- 172.21.15.47:0/1446612046 >> 172.21.15.47:0/1446612046 conn(0x7f7b7c009f70 msgr2=0x7f7b7c00a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f2a8bdc4640 1 -- 172.21.15.47:0/3214600583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a840b1b20 msgr2=0x7f2a840cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f2a8bdc4640 1 --2- 172.21.15.47:0/3214600583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a840b1b20 0x7f2a840cfd60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2a840b4e10 tx=0x7f2a8002b7e0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7fe769ffb640 1 -- 172.21.15.47:0/1952538669 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76402cc60 con 0x7fe76c0ccd30 2024-03-23T09:41:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f1b69548640 1 -- 172.21.15.47:0/501956883 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6002d040 con 0x7f1b640b5540 2024-03-23T09:41:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f2a8bdc4640 1 -- 172.21.15.47:0/3214600583 shutdown_connections 2024-03-23T09:41:12.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7ffbb2d9d640 1 --2- 172.21.15.47:0/3460973216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb4099110 0x7ffbb40b3330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4071a30 tx=0x7ffbac0302f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c4bb18eaa002a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7ffbb159a640 1 -- 172.21.15.47:0/3460973216 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac030ea0 con 0x7ffbb4099110 2024-03-23T09:41:12.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.434+0000 7ffbb159a640 1 -- 172.21.15.47:0/3460973216 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac038640 con 0x7ffbb4099110 2024-03-23T09:41:12.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7ffbb159a640 1 -- 172.21.15.47:0/3460973216 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac0389b0 con 0x7ffbb4099110 2024-03-23T09:41:12.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f1b69548640 1 -- 172.21.15.47:0/501956883 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b60031810 con 0x7f1b640b5540 2024-03-23T09:41:12.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f1b69548640 1 -- 172.21.15.47:0/501956883 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60031ba0 con 0x7f1b640b5540 2024-03-23T09:41:12.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7ff673fff640 1 -- 172.21.15.47:0/815436565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff674070de0 msgr2=0x7ff6740caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7ff673fff640 1 --2- 172.21.15.47:0/815436565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff674070de0 0x7ff6740caac0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009df0 tx=0x7ff66c02be90 comp rx=0 tx=0).stop 2024-03-23T09:41:12.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f2a8bdc4640 1 --2- 172.21.15.47:0/3214600583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a840b1b20 0x7f2a840cfd60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f2a8bdc4640 1 --2- 172.21.15.47:0/3214600583 >> v2:172.21.15.175:3300/0 conn(0x7f2a8415ff60 0x7f2a840cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f2a8bdc4640 1 -- 172.21.15.47:0/3214600583 >> 172.21.15.47:0/3214600583 conn(0x7f2a84065df0 msgr2=0x7f2a84066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f2a8bdc4640 1 -- 172.21.15.47:0/3214600583 shutdown_connections 2024-03-23T09:41:12.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.435+0000 7f2a8bdc4640 1 -- 172.21.15.47:0/3214600583 wait complete. 2024-03-23T09:41:12.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7f7b88bfd640 1 -- 172.21.15.47:0/1446612046 shutdown_connections 2024-03-23T09:41:12.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7fe769ffb640 1 -- 172.21.15.47:0/1952538669 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7640343f0 con 0x7fe76c0ccd30 2024-03-23T09:41:12.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7ff673fff640 1 -- 172.21.15.47:0/815436565 shutdown_connections 2024-03-23T09:41:12.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7ff673fff640 1 --2- 172.21.15.47:0/815436565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff674070de0 0x7ff6740caac0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7ff673fff640 1 -- 172.21.15.47:0/815436565 >> 172.21.15.47:0/815436565 conn(0x7ff674069ec0 msgr2=0x7ff67406a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7fe76bfff640 1 -- 172.21.15.47:0/1952538669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0ccd30 msgr2=0x7fe76c161f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7fc684911640 1 -- 172.21.15.47:0/595579080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc668003970 msgr2=0x7fc668003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7fc684911640 1 --2- 172.21.15.47:0/595579080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc668003970 0x7fc668003d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc67400b5a0 tx=0x7fc6680057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7fe76bfff640 1 --2- 172.21.15.47:0/1952538669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0ccd30 0x7fe76c161f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe764009e50 tx=0x7fe76402bf60 comp rx=0 tx=0).stop 2024-03-23T09:41:12.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7fc684911640 1 -- 172.21.15.47:0/595579080 shutdown_connections 2024-03-23T09:41:12.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7fc684911640 1 --2- 172.21.15.47:0/595579080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc668003970 0x7fc668003d90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7fc684911640 1 -- 172.21.15.47:0/595579080 >> 172.21.15.47:0/595579080 conn(0x7fc680066000 msgr2=0x7fc680069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7f1b6b54c640 1 -- 172.21.15.47:0/501956883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b640b5540 msgr2=0x7f1b640d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7f1b6b54c640 1 --2- 172.21.15.47:0/501956883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b640b5540 0x7f1b640d1970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009790 tx=0x7f1b6002b8b0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7f9b78755640 1 Processor -- start 2024-03-23T09:41:12.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7f1b6b54c640 1 -- 172.21.15.47:0/501956883 shutdown_connections 2024-03-23T09:41:12.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7f1b6b54c640 1 --2- 172.21.15.47:0/501956883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b640b5540 0x7f1b640d1970 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7f1b6b54c640 1 -- 172.21.15.47:0/501956883 >> 172.21.15.47:0/501956883 conn(0x7f1b640cf170 msgr2=0x7f1b640cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7fc684911640 1 -- 172.21.15.47:0/595579080 shutdown_connections 2024-03-23T09:41:12.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7ffbb8f20640 1 -- 172.21.15.47:0/3460973216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb4099110 msgr2=0x7ffbb40b3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7ffbb8f20640 1 --2- 172.21.15.47:0/3460973216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb4099110 0x7ffbb40b3330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4071a30 tx=0x7ffbac0302f0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7ffbb8f20640 1 -- 172.21.15.47:0/3460973216 shutdown_connections 2024-03-23T09:41:12.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7ffbb8f20640 1 --2- 172.21.15.47:0/3460973216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb4099110 0x7ffbb40b3330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7ffbb8f20640 1 --2- 172.21.15.47:0/3460973216 >> v2:172.21.15.175:3300/0 conn(0x7ffbb415de00 0x7ffbb40ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7ffbb8f20640 1 -- 172.21.15.47:0/3460973216 >> 172.21.15.47:0/3460973216 conn(0x7ffbb40b1fd0 msgr2=0x7ffbb40b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7ffbb8f20640 1 -- 172.21.15.47:0/3460973216 shutdown_connections 2024-03-23T09:41:12.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7f9b78755640 1 -- start start 2024-03-23T09:41:12.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7fe76bfff640 1 -- 172.21.15.47:0/1952538669 shutdown_connections 2024-03-23T09:41:12.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7fe76bfff640 1 --2- 172.21.15.47:0/1952538669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c1651f0 0x7fe76c165640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7fe76bfff640 1 --2- 172.21.15.47:0/1952538669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0ccd30 0x7fe76c161f20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7fe76bfff640 1 -- 172.21.15.47:0/1952538669 >> 172.21.15.47:0/1952538669 conn(0x7fe76c0cf160 msgr2=0x7fe76c0cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7ff673fff640 1 -- 172.21.15.47:0/815436565 shutdown_connections 2024-03-23T09:41:12.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.437+0000 7ff673fff640 1 -- 172.21.15.47:0/815436565 wait complete. 2024-03-23T09:41:12.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.437+0000 7fe76bfff640 1 -- 172.21.15.47:0/1952538669 shutdown_connections 2024-03-23T09:41:12.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.437+0000 7fe76bfff640 1 -- 172.21.15.47:0/1952538669 wait complete. 2024-03-23T09:41:12.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.437+0000 7f2a8bdc4640 1 Processor -- start 2024-03-23T09:41:12.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.437+0000 7f2a8bdc4640 1 -- start start 2024-03-23T09:41:12.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.437+0000 7fc684911640 1 -- 172.21.15.47:0/595579080 wait complete. 2024-03-23T09:41:12.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f9b78755640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b7006b420 0x7f9b700c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7f7b88bfd640 1 -- 172.21.15.47:0/1446612046 wait complete. 2024-03-23T09:41:12.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f1b6b54c640 1 -- 172.21.15.47:0/501956883 shutdown_connections 2024-03-23T09:41:12.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f7b88bfd640 1 Processor -- start 2024-03-23T09:41:12.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f9b78755640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b7006e6b0 0x7f9b701f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f9b78755640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b70165770 0x7f9b700c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f9b764f1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b7006b420 0x7f9b700c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f9b764f1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b7006b420 0x7f9b700c3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:46568/0 (socket says 172.21.15.47:46568) 2024-03-23T09:41:12.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.436+0000 7ffbb8f20640 1 -- 172.21.15.47:0/3460973216 wait complete. 2024-03-23T09:41:12.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7ffbb8f20640 1 Processor -- start 2024-03-23T09:41:12.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f9b75cf0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b7006e6b0 0x7f9b701f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7ffbb8f20640 1 -- start start 2024-03-23T09:41:12.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f9b75cf0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b7006e6b0 0x7f9b701f6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:46724/0 (socket says 172.21.15.47:46724) 2024-03-23T09:41:12.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7fc684911640 1 Processor -- start 2024-03-23T09:41:12.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7fe76bfff640 1 Processor -- start 2024-03-23T09:41:12.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f9b764f1640 1 -- 172.21.15.47:0/3673696600 learned_addr learned my addr 172.21.15.47:0/3673696600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f9b78755640 1 -- 172.21.15.47:0/3673696600 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700c45a0 con 0x7f9b70165770 2024-03-23T09:41:12.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f9b78755640 1 -- 172.21.15.47:0/3673696600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700c4880 con 0x7f9b7006b420 2024-03-23T09:41:12.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f9b78755640 1 -- 172.21.15.47:0/3673696600 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700c4b60 con 0x7f9b7006e6b0 2024-03-23T09:41:12.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7fe76bfff640 1 -- start start 2024-03-23T09:41:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f7b88bfd640 1 -- start start 2024-03-23T09:41:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f7b88bfd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b7c005e50 0x7f7b7c17a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f9b76cf2640 1 --2- 172.21.15.47:0/3673696600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b70165770 0x7f9b700c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7fc684911640 1 -- start start 2024-03-23T09:41:12.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f7b88bfd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b7c0069a0 0x7f7b7c17bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f1b6b54c640 1 -- 172.21.15.47:0/501956883 wait complete. 2024-03-23T09:41:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f7b88bfd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b7c0eacc0 0x7f7b7c04ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f7b8259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b7c005e50 0x7f7b7c17a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f7b8259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b7c005e50 0x7f7b7c17a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46232/0 (socket says 172.21.15.47:46232) 2024-03-23T09:41:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f7b8259c640 1 -- 172.21.15.47:0/3859860669 learned_addr learned my addr 172.21.15.47:0/3859860669 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f7b88bfd640 1 -- 172.21.15.47:0/3859860669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c048410 con 0x7f7b7c005e50 2024-03-23T09:41:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f7b88bfd640 1 -- 172.21.15.47:0/3859860669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c17a770 con 0x7f7b7c0069a0 2024-03-23T09:41:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f7b88bfd640 1 -- 172.21.15.47:0/3859860669 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c17aa50 con 0x7f7b7c0eacc0 2024-03-23T09:41:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f2a8bdc4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a840b1b20 0x7f2a841562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f2a8bdc4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a8415ff60 0x7f2a841557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f2a8bdc4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a84153d60 0x7f2a8415db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f2a8bdc4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84155cf0 con 0x7f2a8415ff60 2024-03-23T09:41:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f2a8bdc4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84158230 con 0x7f2a840b1b20 2024-03-23T09:41:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7f2a8bdc4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a841569f0 con 0x7f2a84153d60 2024-03-23T09:41:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f2a8935f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a8415ff60 0x7f2a841557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f2a8935f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a8415ff60 0x7f2a841557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46222/0 (socket says 172.21.15.47:46222) 2024-03-23T09:41:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7f2a8935f640 1 -- 172.21.15.47:0/2531438232 learned_addr learned my addr 172.21.15.47:0/2531438232 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f2a89b60640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a840b1b20 0x7f2a841562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f1b6b54c640 1 Processor -- start 2024-03-23T09:41:12.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f2a8a361640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a84153d60 0x7f2a8415db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f1b6b54c640 1 -- start start 2024-03-23T09:41:12.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.438+0000 7ff673fff640 1 Processor -- start 2024-03-23T09:41:12.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.439+0000 7ff673fff640 1 -- start start 2024-03-23T09:41:12.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ff673fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff674070de0 0x7ff674202d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ff673fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff674160a80 0x7ff6741feb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f9b764f1640 1 -- 172.21.15.47:0/3673696600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b7006e6b0 msgr2=0x7f9b701f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f9b764f1640 1 --2- 172.21.15.47:0/3673696600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b7006e6b0 0x7f9b701f6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f9b764f1640 1 -- 172.21.15.47:0/3673696600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b70165770 msgr2=0x7f9b700c28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f9b764f1640 1 --2- 172.21.15.47:0/3673696600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b70165770 0x7f9b700c28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f9b764f1640 1 -- 172.21.15.47:0/3673696600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b68009a70 con 0x7f9b7006b420 2024-03-23T09:41:12.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ff673fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff674160e60 0x7ff67406cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ff673fff640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67406c5e0 con 0x7ff674160a80 2024-03-23T09:41:12.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ff673fff640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6741ff430 con 0x7ff674070de0 2024-03-23T09:41:12.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ff673fff640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6741ff710 con 0x7ff674160e60 2024-03-23T09:41:12.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ffbb8f20640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbb4099110 0x7ffbb415b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7fe76bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76c0b4160 0x7fe76c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7fe76bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0ccd30 0x7fe76c157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7fe76bfff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c1651f0 0x7fe76c157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ff6727fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff674160a80 0x7ff6741feb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ffbb8f20640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbb40ce5f0 0x7ffbb41568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7fe76bfff640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c1587e0 con 0x7fe76c0ccd30 2024-03-23T09:41:12.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7fe76bfff640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c158c70 con 0x7fe76c0b4160 2024-03-23T09:41:12.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ff672ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff674070de0 0x7ff674202d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7fe76bfff640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c157bd0 con 0x7fe76c1651f0 2024-03-23T09:41:12.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f2a89b60640 1 -- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a84153d60 msgr2=0x7f2a8415db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f2a89b60640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a84153d60 0x7f2a8415db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f2a89b60640 1 -- 172.21.15.47:0/2531438232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a8415ff60 msgr2=0x7f2a841557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f2a89b60640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a8415ff60 0x7f2a841557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f2a89b60640 1 -- 172.21.15.47:0/2531438232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a80009400 con 0x7f2a840b1b20 2024-03-23T09:41:12.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ff6737fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff674160e60 0x7ff67406cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ff6737fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff674160e60 0x7ff67406cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:46758/0 (socket says 172.21.15.47:46758) 2024-03-23T09:41:12.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ffbb8f20640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb415de00 0x7ffbb40c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ff6737fe640 1 -- 172.21.15.47:0/1869133121 learned_addr learned my addr 172.21.15.47:0/1869133121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ffbb8f20640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40c7db0 con 0x7ffbb415de00 2024-03-23T09:41:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ffbb1d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbb40ce5f0 0x7ffbb41568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ffbb8f20640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb415bcb0 con 0x7ffbb40ce5f0 2024-03-23T09:41:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ffbb259c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbb4099110 0x7ffbb415b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ffbb8f20640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb4156df0 con 0x7ffbb4099110 2024-03-23T09:41:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7ffbb1d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbb40ce5f0 0x7ffbb41568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:46608/0 (socket says 172.21.15.47:46608) 2024-03-23T09:41:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7ffbb1d9b640 1 -- 172.21.15.47:0/1540270545 learned_addr learned my addr 172.21.15.47:0/1540270545 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f7b8259c640 1 -- 172.21.15.47:0/3859860669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b7c0eacc0 msgr2=0x7f7b7c04ddd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-23T09:41:12.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f7b8259c640 1 --2- 172.21.15.47:0/3859860669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b7c0eacc0 0x7f7b7c04ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f7b8259c640 1 -- 172.21.15.47:0/3859860669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b7c0069a0 msgr2=0x7f7b7c17bda0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-23T09:41:12.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f7b8259c640 1 --2- 172.21.15.47:0/3859860669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b7c0069a0 0x7f7b7c17bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.440+0000 7f7b8259c640 1 -- 172.21.15.47:0/3859860669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8406be70 con 0x7f7b7c005e50 2024-03-23T09:41:12.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f2a89b60640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a840b1b20 0x7f2a841562f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00ebc0 tx=0x7f2a7c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7fe76affd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76c0b4160 0x7fe76c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7fe76affd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76c0b4160 0x7fe76c0c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:46612/0 (socket says 172.21.15.47:46612) 2024-03-23T09:41:12.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f9b764f1640 1 --2- 172.21.15.47:0/3673696600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b7006b420 0x7f9b700c3d50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009fd0 tx=0x7f9b6c00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7fe76b7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c1651f0 0x7fe76c157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7fe76a7fc640 1 --2- 172.21.15.47:0/1715343552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0ccd30 0x7fe76c157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7fe76affd640 1 -- 172.21.15.47:0/1715343552 learned_addr learned my addr 172.21.15.47:0/1715343552 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7ffbb2d9d640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb415de00 0x7ffbb40c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f7b8259c640 1 --2- 172.21.15.47:0/3859860669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b7c005e50 0x7f7b7c17a090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400b3e0 tx=0x7f7b7400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f1b6b54c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b640b40d0 0x7f1b64070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f1b6b54c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b640d1eb0 0x7f1b64070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f1b6b54c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b641fe750 0x7f1b641febe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f1b6b54c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b641ff120 con 0x7f1b640d1eb0 2024-03-23T09:41:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f1b6b54c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b640a0320 con 0x7f1b641fe750 2024-03-23T09:41:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f1b6b54c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b64070ef0 con 0x7f1b640b40d0 2024-03-23T09:41:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7ffbb259c640 1 -- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbb40ce5f0 msgr2=0x7ffbb41568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7ffbb259c640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbb40ce5f0 0x7ffbb41568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7ffbb259c640 1 -- 172.21.15.47:0/1540270545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb415de00 msgr2=0x7ffbb40c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7ffbb259c640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb415de00 0x7ffbb40c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7ffbb259c640 1 -- 172.21.15.47:0/1540270545 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac009a70 con 0x7ffbb4099110 2024-03-23T09:41:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7ffbb1d9b640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbb40ce5f0 0x7ffbb41568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f1b69d49640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b640d1eb0 0x7f1b64070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f1b6a54a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b640b40d0 0x7f1b64070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f1b69d49640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b640d1eb0 0x7f1b64070590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46270/0 (socket says 172.21.15.47:46270) 2024-03-23T09:41:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f1b6a54a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b640b40d0 0x7f1b64070050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:46784/0 (socket says 172.21.15.47:46784) 2024-03-23T09:41:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f1b69d49640 1 -- 172.21.15.47:0/1134432027 learned_addr learned my addr 172.21.15.47:0/1134432027 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f7b72ffd640 1 -- 172.21.15.47:0/3859860669 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74011830 con 0x7f7b7c005e50 2024-03-23T09:41:12.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7fe76affd640 1 -- 172.21.15.47:0/1715343552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c1651f0 msgr2=0x7fe76c157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7fe76affd640 1 --2- 172.21.15.47:0/1715343552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c1651f0 0x7fe76c157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7fe76affd640 1 -- 172.21.15.47:0/1715343552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0ccd30 msgr2=0x7fe76c157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7fe76affd640 1 --2- 172.21.15.47:0/1715343552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0ccd30 0x7fe76c157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7fe76affd640 1 -- 172.21.15.47:0/1715343552 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe764009b30 con 0x7fe76c0b4160 2024-03-23T09:41:12.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7fc684911640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc680149df0 0x7fc68015a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f7b88bfd640 1 -- 172.21.15.47:0/3859860669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7c17ad30 con 0x7f7b7c005e50 2024-03-23T09:41:12.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f7b88bfd640 1 -- 172.21.15.47:0/3859860669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7c185a60 con 0x7f7b7c005e50 2024-03-23T09:41:12.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7fc684911640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc68014d9e0 0x7fc680150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7fc684911640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc680150ea0 0x7fc68014f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7fc684911640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68014a1d0 con 0x7fc680150ea0 2024-03-23T09:41:12.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7fc684911640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc680151330 con 0x7fc680149df0 2024-03-23T09:41:12.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7fc684911640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68015aea0 con 0x7fc68014d9e0 2024-03-23T09:41:12.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7fe76affd640 1 --2- 172.21.15.47:0/1715343552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76c0b4160 0x7fe76c0c8f40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c3e0 tx=0x7fe75c00c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7fc67ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc680150ea0 0x7fc68014f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f7b72ffd640 1 -- 172.21.15.47:0/3859860669 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b740119b0 con 0x7f7b7c005e50 2024-03-23T09:41:12.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f7b72ffd640 1 -- 172.21.15.47:0/3859860669 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74011da0 con 0x7f7b7c005e50 2024-03-23T09:41:12.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7fc67ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc680150ea0 0x7fc68014f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46278/0 (socket says 172.21.15.47:46278) 2024-03-23T09:41:12.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7fc67ed9d640 1 -- 172.21.15.47:0/171730519 learned_addr learned my addr 172.21.15.47:0/171730519 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:12.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7ff6727fc640 1 -- 172.21.15.47:0/1869133121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff674160e60 msgr2=0x7ff67406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7ff6727fc640 1 --2- 172.21.15.47:0/1869133121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff674160e60 0x7ff67406cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7ff6727fc640 1 -- 172.21.15.47:0/1869133121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff674070de0 msgr2=0x7ff674202d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7ff6727fc640 1 --2- 172.21.15.47:0/1869133121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff674070de0 0x7ff674202d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7ff6727fc640 1 -- 172.21.15.47:0/1869133121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c009a70 con 0x7ff674160a80 2024-03-23T09:41:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f1b6ad4b640 1 --2- 172.21.15.47:0/1134432027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b641fe750 0x7f1b641febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7fc67ed9d640 1 -- 172.21.15.47:0/171730519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc68014d9e0 msgr2=0x7fc680150960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T09:41:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7fc67ed9d640 1 --2- 172.21.15.47:0/171730519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc68014d9e0 0x7fc680150960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7fc67ed9d640 1 -- 172.21.15.47:0/171730519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc680149df0 msgr2=0x7fc68015a530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T09:41:12.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7fc67ed9d640 1 --2- 172.21.15.47:0/171730519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc680149df0 0x7fc68015a530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7fc67ed9d640 1 -- 172.21.15.47:0/171730519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6801f1740 con 0x7fc680150ea0 2024-03-23T09:41:12.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f2a7a7fc640 1 -- 172.21.15.47:0/2531438232 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c0092d0 con 0x7f2a840b1b20 2024-03-23T09:41:12.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.441+0000 7f2a7a7fc640 1 -- 172.21.15.47:0/2531438232 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c00f040 con 0x7f2a840b1b20 2024-03-23T09:41:12.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f2a7a7fc640 1 -- 172.21.15.47:0/2531438232 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00cda0 con 0x7f2a840b1b20 2024-03-23T09:41:12.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f2a8bdc4640 1 -- 172.21.15.47:0/2531438232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a841584a0 con 0x7f2a840b1b20 2024-03-23T09:41:12.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f2a8bdc4640 1 -- 172.21.15.47:0/2531438232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a84156cd0 con 0x7f2a840b1b20 2024-03-23T09:41:12.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f9b66ffd640 1 -- 172.21.15.47:0/3673696600 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c00c940 con 0x7f9b7006b420 2024-03-23T09:41:12.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f9b66ffd640 1 -- 172.21.15.47:0/3673696600 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c00caa0 con 0x7f9b7006b420 2024-03-23T09:41:12.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f9b66ffd640 1 -- 172.21.15.47:0/3673696600 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c00ce30 con 0x7f9b7006b420 2024-03-23T09:41:12.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f1b69d49640 1 -- 172.21.15.47:0/1134432027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b640b40d0 msgr2=0x7f1b64070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f1b69d49640 1 --2- 172.21.15.47:0/1134432027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b640b40d0 0x7f1b64070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f1b69d49640 1 -- 172.21.15.47:0/1134432027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b641fe750 msgr2=0x7f1b641febe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f1b69d49640 1 --2- 172.21.15.47:0/1134432027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b641fe750 0x7f1b641febe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.442+0000 7f1b69d49640 1 -- 172.21.15.47:0/1134432027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b600093a0 con 0x7f1b640d1eb0 2024-03-23T09:41:12.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7fe753fff640 1 -- 172.21.15.47:0/1715343552 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0074e0 con 0x7fe76c0b4160 2024-03-23T09:41:12.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7fc67ed9d640 1 --2- 172.21.15.47:0/171730519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc680150ea0 0x7fc68014f760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc6801647a0 tx=0x7fc67800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7f1b69d49640 1 --2- 172.21.15.47:0/1134432027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b640d1eb0 0x7f1b64070590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1b6002bec0 tx=0x7f1b600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7fe76bfff640 1 -- 172.21.15.47:0/1715343552 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76c157eb0 con 0x7fe76c0b4160 2024-03-23T09:41:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7fe76bfff640 1 -- 172.21.15.47:0/1715343552 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76c158470 con 0x7fe76c0b4160 2024-03-23T09:41:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7f9b78755640 1 -- 172.21.15.47:0/3673696600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7009d2d0 con 0x7f9b7006b420 2024-03-23T09:41:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7ffbb259c640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbb4099110 0x7ffbb415b770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffba400b410 tx=0x7ffba400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7ff6727fc640 1 --2- 172.21.15.47:0/1869133121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff674160a80 0x7ff6741feb20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff66400c390 tx=0x7ff66400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7fc66effd640 1 -- 172.21.15.47:0/171730519 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67800ce80 con 0x7fc680150ea0 2024-03-23T09:41:12.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7f7b88bfd640 1 -- 172.21.15.47:0/3859860669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b68005250 con 0x7f7b7c005e50 2024-03-23T09:41:12.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7f1b5affd640 1 -- 172.21.15.47:0/1134432027 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6003ab50 con 0x7f1b640d1eb0 2024-03-23T09:41:12.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7fc66effd640 1 -- 172.21.15.47:0/171730519 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67800a5b0 con 0x7fc680150ea0 2024-03-23T09:41:12.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7fc684911640 1 -- 172.21.15.47:0/171730519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6801f1a20 con 0x7fc680150ea0 2024-03-23T09:41:12.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7f1b5affd640 1 -- 172.21.15.47:0/1134432027 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6003acb0 con 0x7f1b640d1eb0 2024-03-23T09:41:12.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7fe753fff640 1 -- 172.21.15.47:0/1715343552 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c007640 con 0x7fe76c0b4160 2024-03-23T09:41:12.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7fe753fff640 1 -- 172.21.15.47:0/1715343552 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0074e0 con 0x7fe76c0b4160 2024-03-23T09:41:12.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.444+0000 7f2a67fff640 1 -- 172.21.15.47:0/2531438232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a60005250 con 0x7f2a840b1b20 2024-03-23T09:41:12.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.444+0000 7ffba2ffd640 1 -- 172.21.15.47:0/1540270545 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4011830 con 0x7ffbb4099110 2024-03-23T09:41:12.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.444+0000 7f9b78755640 1 -- 172.21.15.47:0/3673696600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b700c4290 con 0x7f9b7006b420 2024-03-23T09:41:12.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.444+0000 7f1b5affd640 1 -- 172.21.15.47:0/1134432027 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60005b90 con 0x7f1b640d1eb0 2024-03-23T09:41:12.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.444+0000 7f1b6b54c640 1 -- 172.21.15.47:0/1134432027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b64070c10 con 0x7f1b640d1eb0 2024-03-23T09:41:12.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.445+0000 7f1b6b54c640 1 -- 172.21.15.47:0/1134432027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6406d3f0 con 0x7f1b640d1eb0 2024-03-23T09:41:12.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.444+0000 7f7b72ffd640 1 -- 172.21.15.47:0/3859860669 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7b7401a400 con 0x7f7b7c005e50 2024-03-23T09:41:12.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.444+0000 7f7b72ffd640 1 --2- 172.21.15.47:0/3859860669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b6406acd0 0x7f7b6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.444+0000 7f7b72ffd640 1 -- 172.21.15.47:0/3859860669 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b740893b0 con 0x7f7b7c005e50 2024-03-23T09:41:12.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.444+0000 7fe76bfff640 1 -- 172.21.15.47:0/1715343552 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74c005250 con 0x7fe76c0b4160 2024-03-23T09:41:12.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.445+0000 7ff65b7fe640 1 -- 172.21.15.47:0/1869133121 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6640052d0 con 0x7ff674160a80 2024-03-23T09:41:12.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.443+0000 7fc684911640 1 -- 172.21.15.47:0/171730519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6801f1fe0 con 0x7fc680150ea0 2024-03-23T09:41:12.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.444+0000 7ffbb8f20640 1 -- 172.21.15.47:0/1540270545 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb40cc4b0 con 0x7ffbb4099110 2024-03-23T09:41:12.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.444+0000 7ffbb8f20640 1 -- 172.21.15.47:0/1540270545 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb41582a0 con 0x7ffbb4099110 2024-03-23T09:41:12.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.445+0000 7f7b81d9b640 1 --2- 172.21.15.47:0/3859860669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b6406acd0 0x7f7b6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.445+0000 7f9b64ff9640 1 -- 172.21.15.47:0/3673696600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4c005250 con 0x7f9b7006b420 2024-03-23T09:41:12.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.445+0000 7fc66effd640 1 -- 172.21.15.47:0/171730519 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67800a960 con 0x7fc680150ea0 2024-03-23T09:41:12.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.445+0000 7ffbb8f20640 1 -- 172.21.15.47:0/1540270545 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb40cc310 con 0x7ffbb4099110 2024-03-23T09:41:12.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.446+0000 7ffba2ffd640 1 -- 172.21.15.47:0/1540270545 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba4011990 con 0x7ffbb4099110 2024-03-23T09:41:12.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.446+0000 7fc684911640 1 -- 172.21.15.47:0/171730519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc664005250 con 0x7fc680150ea0 2024-03-23T09:41:12.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.445+0000 7ff673fff640 1 -- 172.21.15.47:0/1869133121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6741ff9f0 con 0x7ff674160a80 2024-03-23T09:41:12.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.445+0000 7ff673fff640 1 -- 172.21.15.47:0/1869133121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6741ff060 con 0x7ff674160a80 2024-03-23T09:41:12.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.445+0000 7ff65b7fe640 1 -- 172.21.15.47:0/1869133121 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff664005430 con 0x7ff674160a80 2024-03-23T09:41:12.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.445+0000 7ff65b7fe640 1 -- 172.21.15.47:0/1869133121 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff664003830 con 0x7ff674160a80 2024-03-23T09:41:12.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.446+0000 7fe753fff640 1 -- 172.21.15.47:0/1715343552 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe75c0047e0 con 0x7fe76c0b4160 2024-03-23T09:41:12.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.446+0000 7fe753fff640 1 --2- 172.21.15.47:0/1715343552 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe75806ac90 0x7fe75806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.446+0000 7fe753fff640 1 -- 172.21.15.47:0/1715343552 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe75c057e00 con 0x7fe76c0b4160 2024-03-23T09:41:12.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.446+0000 7fe76a7fc640 1 --2- 172.21.15.47:0/1715343552 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe75806ac90 0x7fe75806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.446+0000 7f7b81d9b640 1 --2- 172.21.15.47:0/3859860669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b6406acd0 0x7f7b6406d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b78000f60 tx=0x7f7b78007a00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.446+0000 7ff673fff640 1 -- 172.21.15.47:0/1869133121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff654005250 con 0x7ff674160a80 2024-03-23T09:41:12.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.447+0000 7fc66effd640 1 -- 172.21.15.47:0/171730519 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc67800aac0 con 0x7fc680150ea0 2024-03-23T09:41:12.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.447+0000 7ffba2ffd640 1 -- 172.21.15.47:0/1540270545 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4011d80 con 0x7ffbb4099110 2024-03-23T09:41:12.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.447+0000 7fe76a7fc640 1 --2- 172.21.15.47:0/1715343552 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe75806ac90 0x7fe75806d150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe764035000 tx=0x7fe764030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.447+0000 7fc66effd640 1 --2- 172.21.15.47:0/171730519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc66006ac50 0x7fc66006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.447+0000 7f2a7a7fc640 1 -- 172.21.15.47:0/2531438232 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2a7c014480 con 0x7f2a840b1b20 2024-03-23T09:41:12.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.447+0000 7f2a7a7fc640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a6806acd0 0x7f2a6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.447+0000 7fc66effd640 1 -- 172.21.15.47:0/171730519 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6780889a0 con 0x7fc680150ea0 2024-03-23T09:41:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.447+0000 7f2a7a7fc640 1 -- 172.21.15.47:0/2531438232 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a7c08a2f0 con 0x7f2a840b1b20 2024-03-23T09:41:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.447+0000 7f2a8935f640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a6806acd0 0x7f2a6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.447+0000 7f9b66ffd640 1 -- 172.21.15.47:0/3673696600 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9b6c013470 con 0x7f9b7006b420 2024-03-23T09:41:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.448+0000 7f9b66ffd640 1 --2- 172.21.15.47:0/3673696600 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b5406ac50 0x7f9b5406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.448+0000 7fc67e59c640 1 --2- 172.21.15.47:0/171730519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc66006ac50 0x7fc66006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.448+0000 7f2a8935f640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a6806acd0 0x7f2a6806d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2a840b76f0 tx=0x7f2a74009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.448+0000 7f9b75cf0640 1 --2- 172.21.15.47:0/3673696600 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b5406ac50 0x7f9b5406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.448+0000 7f1b6b54c640 1 -- 172.21.15.47:0/1134432027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b64073ef0 con 0x7f1b640d1eb0 2024-03-23T09:41:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.448+0000 7ffba2ffd640 1 -- 172.21.15.47:0/1540270545 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ffba401a400 con 0x7ffbb4099110 2024-03-23T09:41:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.448+0000 7fc67e59c640 1 --2- 172.21.15.47:0/171730519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc66006ac50 0x7fc66006d110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc67400b9e0 tx=0x7fc674041000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.448+0000 7f9b75cf0640 1 --2- 172.21.15.47:0/3673696600 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b5406ac50 0x7f9b5406d110 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b60000f60 tx=0x7f9b60009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.449+0000 7ffba2ffd640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffb9406acd0 0x7ffb9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.449+0000 7ffba2ffd640 1 -- 172.21.15.47:0/1540270545 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffba4089720 con 0x7ffbb4099110 2024-03-23T09:41:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.449+0000 7ffbb1d9b640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffb9406acd0 0x7ffb9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.449+0000 7ffbb1d9b640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffb9406acd0 0x7ffb9406d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffbb41f12f0 tx=0x7ffba8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.450+0000 7f9b66ffd640 1 -- 172.21.15.47:0/3673696600 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b6c089e40 con 0x7f9b7006b420 2024-03-23T09:41:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.450+0000 7f2a7a7fc640 1 -- 172.21.15.47:0/2531438232 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2a7c055540 con 0x7f2a840b1b20 2024-03-23T09:41:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.451+0000 7f7b72ffd640 1 -- 172.21.15.47:0/3859860669 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7b740546c0 con 0x7f7b7c005e50 2024-03-23T09:41:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.452+0000 7f1b5affd640 1 -- 172.21.15.47:0/1134432027 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1b60042030 con 0x7f1b640d1eb0 2024-03-23T09:41:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.452+0000 7ffba2ffd640 1 -- 172.21.15.47:0/1540270545 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ffba4054990 con 0x7ffbb4099110 2024-03-23T09:41:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.452+0000 7f1b5affd640 1 --2- 172.21.15.47:0/1134432027 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b4806ac50 0x7f1b4806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.453+0000 7f1b6a54a640 1 --2- 172.21.15.47:0/1134432027 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b4806ac50 0x7f1b4806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.453+0000 7f1b6a54a640 1 --2- 172.21.15.47:0/1134432027 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b4806ac50 0x7f1b4806d110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1b54000f60 tx=0x7f1b54007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.456+0000 7fe753fff640 1 -- 172.21.15.47:0/1715343552 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe75c0541d0 con 0x7fe76c0b4160 2024-03-23T09:41:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.456+0000 7f1b5affd640 1 -- 172.21.15.47:0/1134432027 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b6003ae20 con 0x7f1b640d1eb0 2024-03-23T09:41:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.456+0000 7fc66effd640 1 -- 172.21.15.47:0/171730519 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc678053c60 con 0x7fc680150ea0 2024-03-23T09:41:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.457+0000 7ff65b7fe640 1 -- 172.21.15.47:0/1869133121 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff664003990 con 0x7ff674160a80 2024-03-23T09:41:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.458+0000 7ff65b7fe640 1 --2- 172.21.15.47:0/1869133121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff66006ad00 0x7ff66006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.458+0000 7ff65b7fe640 1 -- 172.21.15.47:0/1869133121 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff664088bb0 con 0x7ff674160a80 2024-03-23T09:41:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.458+0000 7ff672ffd640 1 --2- 172.21.15.47:0/1869133121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff66006ad00 0x7ff66006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.459+0000 7ff672ffd640 1 --2- 172.21.15.47:0/1869133121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff66006ad00 0x7ff66006d1c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6680011e0 tx=0x7ff668009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:12.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.460+0000 7f9b66ffd640 1 -- 172.21.15.47:0/3673696600 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9b6c054730 con 0x7f9b7006b420 2024-03-23T09:41:12.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.466+0000 7f1b5affd640 1 -- 172.21.15.47:0/1134432027 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1b60076cc0 con 0x7f1b640d1eb0 2024-03-23T09:41:12.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.475+0000 7ff65b7fe640 1 -- 172.21.15.47:0/1869133121 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff664053eb0 con 0x7ff674160a80 2024-03-23T09:41:12.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.870+0000 7f2a67fff640 1 -- 172.21.15.47:0/2531438232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2a600057d0 con 0x7f2a840b1b20 2024-03-23T09:41:12.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.871+0000 7f2a7a7fc640 1 -- 172.21.15.47:0/2531438232 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2a7c058b60 con 0x7f2a840b1b20 2024-03-23T09:41:12.871 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-23T09:41:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.873+0000 7f7b88bfd640 1 -- 172.21.15.47:0/3859860669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7b68005500 con 0x7f7b7c005e50 2024-03-23T09:41:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.874+0000 7f7b72ffd640 1 -- 172.21.15.47:0/3859860669 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b74057ce0 con 0x7f7b7c005e50 2024-03-23T09:41:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.874+0000 7f2a8bdc4640 1 -- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a6806acd0 msgr2=0x7f2a6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.874+0000 7f2a8bdc4640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a6806acd0 0x7f2a6806d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2a840b76f0 tx=0x7f2a74009190 comp rx=0 tx=0).stop 2024-03-23T09:41:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.874+0000 7f2a8bdc4640 1 -- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a840b1b20 msgr2=0x7f2a841562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.874+0000 7f2a8bdc4640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a840b1b20 0x7f2a841562f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00ebc0 tx=0x7f2a7c00c490 comp rx=0 tx=0).stop 2024-03-23T09:41:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.875+0000 7f2a8bdc4640 1 -- 172.21.15.47:0/2531438232 shutdown_connections 2024-03-23T09:41:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.875+0000 7f2a8bdc4640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a6806acd0 0x7f2a6806d190 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.875+0000 7f2a8bdc4640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a84153d60 0x7f2a8415db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.875+0000 7f2a8bdc4640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a8415ff60 0x7f2a841557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.875+0000 7f2a8bdc4640 1 --2- 172.21.15.47:0/2531438232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a840b1b20 0x7f2a841562f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.875+0000 7f2a8bdc4640 1 -- 172.21.15.47:0/2531438232 >> 172.21.15.47:0/2531438232 conn(0x7f2a84065df0 msgr2=0x7f2a8406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.876 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-23T09:41:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.875+0000 7f2a8bdc4640 1 -- 172.21.15.47:0/2531438232 shutdown_connections 2024-03-23T09:41:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.875+0000 7f2a8bdc4640 1 -- 172.21.15.47:0/2531438232 wait complete. 2024-03-23T09:41:12.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.878+0000 7f7b88bfd640 1 -- 172.21.15.47:0/3859860669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b6406acd0 msgr2=0x7f7b6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.878+0000 7f7b88bfd640 1 --2- 172.21.15.47:0/3859860669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b6406acd0 0x7f7b6406d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b78000f60 tx=0x7f7b78007a00 comp rx=0 tx=0).stop 2024-03-23T09:41:12.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.878+0000 7f7b88bfd640 1 -- 172.21.15.47:0/3859860669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b7c005e50 msgr2=0x7f7b7c17a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.878+0000 7f7b88bfd640 1 --2- 172.21.15.47:0/3859860669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b7c005e50 0x7f7b7c17a090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400b3e0 tx=0x7f7b7400b8a0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.879+0000 7fe76bfff640 1 -- 172.21.15.47:0/1715343552 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe74c0057d0 con 0x7fe76c0b4160 2024-03-23T09:41:12.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.879+0000 7f7b88bfd640 1 -- 172.21.15.47:0/3859860669 shutdown_connections 2024-03-23T09:41:12.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.879+0000 7f7b88bfd640 1 --2- 172.21.15.47:0/3859860669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b6406acd0 0x7f7b6406d190 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.879+0000 7f7b88bfd640 1 --2- 172.21.15.47:0/3859860669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b7c0eacc0 0x7f7b7c04ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.879+0000 7f7b88bfd640 1 --2- 172.21.15.47:0/3859860669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b7c0069a0 0x7f7b7c17bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.879+0000 7ffbb8f20640 1 -- 172.21.15.47:0/1540270545 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ffbb40cc310 con 0x7ffbb4099110 2024-03-23T09:41:12.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.879+0000 7f7b88bfd640 1 --2- 172.21.15.47:0/3859860669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b7c005e50 0x7f7b7c17a090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.879+0000 7f7b88bfd640 1 -- 172.21.15.47:0/3859860669 >> 172.21.15.47:0/3859860669 conn(0x7f7b7c009f70 msgr2=0x7f7b7c037ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.879+0000 7f7b88bfd640 1 -- 172.21.15.47:0/3859860669 shutdown_connections 2024-03-23T09:41:12.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.879+0000 7f7b88bfd640 1 -- 172.21.15.47:0/3859860669 wait complete. 2024-03-23T09:41:12.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.879+0000 7fe753fff640 1 -- 172.21.15.47:0/1715343552 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe75c0541d0 con 0x7fe76c0b4160 2024-03-23T09:41:12.881 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-23T09:41:12.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.879+0000 7ffba2ffd640 1 -- 172.21.15.47:0/1540270545 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffba4015020 con 0x7ffbb4099110 2024-03-23T09:41:12.882 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-23T09:41:12.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.881+0000 7f9b64ff9640 1 -- 172.21.15.47:0/3673696600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9b4c005500 con 0x7f9b7006b420 2024-03-23T09:41:12.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.882+0000 7f9b66ffd640 1 -- 172.21.15.47:0/3673696600 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9b6c021090 con 0x7f9b7006b420 2024-03-23T09:41:12.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7ffbb8f20640 1 -- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffb9406acd0 msgr2=0x7ffb9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7ffbb8f20640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffb9406acd0 0x7ffb9406d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffbb41f12f0 tx=0x7ffba8009040 comp rx=0 tx=0).stop 2024-03-23T09:41:12.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7ffbb8f20640 1 -- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbb4099110 msgr2=0x7ffbb415b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7ffbb8f20640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbb4099110 0x7ffbb415b770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffba400b410 tx=0x7ffba400b8d0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7fe76bfff640 1 -- 172.21.15.47:0/1715343552 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe75806ac90 msgr2=0x7fe75806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7fe76bfff640 1 --2- 172.21.15.47:0/1715343552 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe75806ac90 0x7fe75806d150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe764035000 tx=0x7fe764030000 comp rx=0 tx=0).stop 2024-03-23T09:41:12.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7fe76bfff640 1 -- 172.21.15.47:0/1715343552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76c0b4160 msgr2=0x7fe76c0c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7fe76bfff640 1 --2- 172.21.15.47:0/1715343552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76c0b4160 0x7fe76c0c8f40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c3e0 tx=0x7fe75c00c8a0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7fe76bfff640 1 -- 172.21.15.47:0/1715343552 shutdown_connections 2024-03-23T09:41:12.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7fe76bfff640 1 --2- 172.21.15.47:0/1715343552 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe75806ac90 0x7fe75806d150 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe764035000 tx=0x7fe764030000 comp rx=0 tx=0).stop 2024-03-23T09:41:12.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7fe76bfff640 1 --2- 172.21.15.47:0/1715343552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c1651f0 0x7fe76c157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7fe76bfff640 1 --2- 172.21.15.47:0/1715343552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0ccd30 0x7fe76c157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7fe76bfff640 1 --2- 172.21.15.47:0/1715343552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76c0b4160 0x7fe76c0c8f40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7fe76bfff640 1 -- 172.21.15.47:0/1715343552 >> 172.21.15.47:0/1715343552 conn(0x7fe76c0cf160 msgr2=0x7fe76c0cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7ffbb8f20640 1 -- 172.21.15.47:0/1540270545 shutdown_connections 2024-03-23T09:41:12.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7ffbb8f20640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffb9406acd0 0x7ffb9406d190 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7ffbb8f20640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb415de00 0x7ffbb40c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7ffbb8f20640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbb40ce5f0 0x7ffbb41568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7ffbb8f20640 1 --2- 172.21.15.47:0/1540270545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbb4099110 0x7ffbb415b770 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7ffbb8f20640 1 -- 172.21.15.47:0/1540270545 >> 172.21.15.47:0/1540270545 conn(0x7ffbb40b1fd0 msgr2=0x7ffbb4098a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.885 INFO:teuthology.orchestra.run.smithi047.stdout:55834574854 2024-03-23T09:41:12.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.884+0000 7ffbb8f20640 1 -- 172.21.15.47:0/1540270545 shutdown_connections 2024-03-23T09:41:12.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.884+0000 7ffbb8f20640 1 -- 172.21.15.47:0/1540270545 wait complete. 2024-03-23T09:41:12.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7fe76bfff640 1 -- 172.21.15.47:0/1715343552 shutdown_connections 2024-03-23T09:41:12.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.883+0000 7fe76bfff640 1 -- 172.21.15.47:0/1715343552 wait complete. 2024-03-23T09:41:12.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.886+0000 7f1b6b54c640 1 -- 172.21.15.47:0/1134432027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1b640b4bc0 con 0x7f1b640d1eb0 2024-03-23T09:41:12.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.887+0000 7f9b64ff9640 1 -- 172.21.15.47:0/3673696600 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b5406ac50 msgr2=0x7f9b5406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.887+0000 7f9b64ff9640 1 --2- 172.21.15.47:0/3673696600 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b5406ac50 0x7f9b5406d110 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b60000f60 tx=0x7f9b60009040 comp rx=0 tx=0).stop 2024-03-23T09:41:12.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.887+0000 7f9b64ff9640 1 -- 172.21.15.47:0/3673696600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b7006b420 msgr2=0x7f9b700c3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.887+0000 7f9b64ff9640 1 --2- 172.21.15.47:0/3673696600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b7006b420 0x7f9b700c3d50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009fd0 tx=0x7f9b6c00ed40 comp rx=0 tx=0).stop 2024-03-23T09:41:12.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.887+0000 7f9b64ff9640 1 -- 172.21.15.47:0/3673696600 shutdown_connections 2024-03-23T09:41:12.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.887+0000 7f9b64ff9640 1 --2- 172.21.15.47:0/3673696600 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b5406ac50 0x7f9b5406d110 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.887+0000 7f9b64ff9640 1 --2- 172.21.15.47:0/3673696600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b70165770 0x7f9b700c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.887+0000 7f9b64ff9640 1 --2- 172.21.15.47:0/3673696600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b7006e6b0 0x7f9b701f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.887+0000 7f9b64ff9640 1 --2- 172.21.15.47:0/3673696600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b7006b420 0x7f9b700c3d50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.887+0000 7f9b64ff9640 1 -- 172.21.15.47:0/3673696600 >> 172.21.15.47:0/3673696600 conn(0x7f9b700cb990 msgr2=0x7f9b70069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.888 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-23T09:41:12.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.887+0000 7f1b5affd640 1 -- 172.21.15.47:0/1134432027 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b640b4bc0 con 0x7f1b640d1eb0 2024-03-23T09:41:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.887+0000 7f9b64ff9640 1 -- 172.21.15.47:0/3673696600 shutdown_connections 2024-03-23T09:41:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.887+0000 7f9b64ff9640 1 -- 172.21.15.47:0/3673696600 wait complete. 2024-03-23T09:41:12.890 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-23T09:41:12.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.889+0000 7fc684911640 1 -- 172.21.15.47:0/171730519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc6640057d0 con 0x7fc680150ea0 2024-03-23T09:41:12.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.890+0000 7fc66effd640 1 -- 172.21.15.47:0/171730519 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc678018070 con 0x7fc680150ea0 2024-03-23T09:41:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.891+0000 7f1b6b54c640 1 -- 172.21.15.47:0/1134432027 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b4806ac50 msgr2=0x7f1b4806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.891+0000 7f1b6b54c640 1 --2- 172.21.15.47:0/1134432027 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b4806ac50 0x7f1b4806d110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1b54000f60 tx=0x7f1b54007d50 comp rx=0 tx=0).stop 2024-03-23T09:41:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.891+0000 7f1b6b54c640 1 -- 172.21.15.47:0/1134432027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b640d1eb0 msgr2=0x7f1b64070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.891+0000 7f1b6b54c640 1 --2- 172.21.15.47:0/1134432027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b640d1eb0 0x7f1b64070590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1b6002bec0 tx=0x7f1b600046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.891+0000 7f1b6b54c640 1 -- 172.21.15.47:0/1134432027 shutdown_connections 2024-03-23T09:41:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.891+0000 7f1b6b54c640 1 --2- 172.21.15.47:0/1134432027 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b4806ac50 0x7f1b4806d110 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.891+0000 7f1b6b54c640 1 --2- 172.21.15.47:0/1134432027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b641fe750 0x7f1b641febe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.891+0000 7f1b6b54c640 1 --2- 172.21.15.47:0/1134432027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b640d1eb0 0x7f1b64070590 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.891+0000 7f1b6b54c640 1 --2- 172.21.15.47:0/1134432027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b640b40d0 0x7f1b64070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.891+0000 7f1b6b54c640 1 -- 172.21.15.47:0/1134432027 >> 172.21.15.47:0/1134432027 conn(0x7f1b640cf170 msgr2=0x7f1b640cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.891+0000 7f1b6b54c640 1 -- 172.21.15.47:0/1134432027 shutdown_connections 2024-03-23T09:41:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.891+0000 7f1b6b54c640 1 -- 172.21.15.47:0/1134432027 wait complete. 2024-03-23T09:41:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.893+0000 7fc684911640 1 -- 172.21.15.47:0/171730519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc66006ac50 msgr2=0x7fc66006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.893+0000 7fc684911640 1 --2- 172.21.15.47:0/171730519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc66006ac50 0x7fc66006d110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc67400b9e0 tx=0x7fc674041000 comp rx=0 tx=0).stop 2024-03-23T09:41:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.893+0000 7fc684911640 1 -- 172.21.15.47:0/171730519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc680150ea0 msgr2=0x7fc68014f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.893+0000 7fc684911640 1 --2- 172.21.15.47:0/171730519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc680150ea0 0x7fc68014f760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc6801647a0 tx=0x7fc67800cc30 comp rx=0 tx=0).stop 2024-03-23T09:41:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.893+0000 7fc684911640 1 -- 172.21.15.47:0/171730519 shutdown_connections 2024-03-23T09:41:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.893+0000 7fc684911640 1 --2- 172.21.15.47:0/171730519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc66006ac50 0x7fc66006d110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.894+0000 7fc684911640 1 --2- 172.21.15.47:0/171730519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc680150ea0 0x7fc68014f760 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.894+0000 7fc684911640 1 --2- 172.21.15.47:0/171730519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc68014d9e0 0x7fc680150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.894+0000 7fc684911640 1 --2- 172.21.15.47:0/171730519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc680149df0 0x7fc68015a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.894+0000 7fc684911640 1 -- 172.21.15.47:0/171730519 >> 172.21.15.47:0/171730519 conn(0x7fc680066000 msgr2=0x7fc6800af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.895 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2024-03-23T09:41:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.895+0000 7fc684911640 1 -- 172.21.15.47:0/171730519 shutdown_connections 2024-03-23T09:41:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.895+0000 7fc684911640 1 -- 172.21.15.47:0/171730519 wait complete. 2024-03-23T09:41:12.896 DEBUG:teuthology.parallel:result is None 2024-03-23T09:41:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.895+0000 7ff673fff640 1 -- 172.21.15.47:0/1869133121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff6540057d0 con 0x7ff674160a80 2024-03-23T09:41:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.896+0000 7ff65b7fe640 1 -- 172.21.15.47:0/1869133121 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff664017090 con 0x7ff674160a80 2024-03-23T09:41:12.896 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-23T09:41:12.900 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2024-03-23T09:41:12.900 DEBUG:teuthology.parallel:result is None 2024-03-23T09:41:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.900+0000 7ff673fff640 1 -- 172.21.15.47:0/1869133121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff66006ad00 msgr2=0x7ff66006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.900+0000 7ff673fff640 1 --2- 172.21.15.47:0/1869133121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff66006ad00 0x7ff66006d1c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6680011e0 tx=0x7ff668009040 comp rx=0 tx=0).stop 2024-03-23T09:41:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.900+0000 7ff673fff640 1 -- 172.21.15.47:0/1869133121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff674160a80 msgr2=0x7ff6741feb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.900+0000 7ff673fff640 1 --2- 172.21.15.47:0/1869133121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff674160a80 0x7ff6741feb20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff66400c390 tx=0x7ff66400c850 comp rx=0 tx=0).stop 2024-03-23T09:41:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.900+0000 7ff673fff640 1 -- 172.21.15.47:0/1869133121 shutdown_connections 2024-03-23T09:41:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.900+0000 7ff673fff640 1 --2- 172.21.15.47:0/1869133121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff66006ad00 0x7ff66006d1c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.900+0000 7ff673fff640 1 --2- 172.21.15.47:0/1869133121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff674160e60 0x7ff67406cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.900+0000 7ff673fff640 1 --2- 172.21.15.47:0/1869133121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff674160a80 0x7ff6741feb20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.900+0000 7ff673fff640 1 --2- 172.21.15.47:0/1869133121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff674070de0 0x7ff674202d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:12.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.900+0000 7ff673fff640 1 -- 172.21.15.47:0/1869133121 >> 172.21.15.47:0/1869133121 conn(0x7ff674069ec0 msgr2=0x7ff6740cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:12.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.900+0000 7ff673fff640 1 -- 172.21.15.47:0/1869133121 shutdown_connections 2024-03-23T09:41:12.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:12.900+0000 7ff673fff640 1 -- 172.21.15.47:0/1869133121 wait complete. 2024-03-23T09:41:12.907 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-03-23T09:41:12.907 DEBUG:teuthology.parallel:result is None 2024-03-23T09:41:12.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-03-23T09:41:12.908 DEBUG:teuthology.parallel:result is None 2024-03-23T09:41:12.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-03-23T09:41:12.908 DEBUG:teuthology.parallel:result is None 2024-03-23T09:41:12.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-03-23T09:41:12.911 DEBUG:teuthology.parallel:result is None 2024-03-23T09:41:12.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2024-03-23T09:41:12.912 DEBUG:teuthology.parallel:result is None 2024-03-23T09:41:12.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-03-23T09:41:12.916 DEBUG:teuthology.parallel:result is None 2024-03-23T09:41:12.916 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-23T09:41:12.917 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:41:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.057+0000 7f1b29057640 1 Processor -- start 2024-03-23T09:41:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.057+0000 7f1b29057640 1 -- start start 2024-03-23T09:41:13.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b29057640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b2415ca90 0x7f1b24160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b29057640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b240b8350 0x7f1b240cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b29057640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1b240cdc40 0x7f1b240b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b29057640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b240ce0d0 con 0x7f1b2415ca90 2024-03-23T09:41:13.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b29057640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b240d11e0 con 0x7f1b240cdc40 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b29057640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b24160f60 con 0x7f1b240b8350 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b22d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b2415ca90 0x7f1b24160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b22d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b2415ca90 0x7f1b24160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46280/0 (socket says 172.21.15.47:46280) 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b22d9d640 1 -- 172.21.15.47:0/3749807988 learned_addr learned my addr 172.21.15.47:0/3749807988 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b22d9d640 1 -- 172.21.15.47:0/3749807988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b240b8350 msgr2=0x7f1b240cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b22d9d640 1 --2- 172.21.15.47:0/3749807988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b240b8350 0x7f1b240cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b22d9d640 1 -- 172.21.15.47:0/3749807988 >> v2:172.21.15.175:3300/0 conn(0x7f1b240cdc40 msgr2=0x7f1b240b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b2359e640 1 --2- 172.21.15.47:0/3749807988 >> v2:172.21.15.175:3300/0 conn(0x7f1b240cdc40 0x7f1b240b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b22d9d640 1 --2- 172.21.15.47:0/3749807988 >> v2:172.21.15.175:3300/0 conn(0x7f1b240cdc40 0x7f1b240b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b22d9d640 1 -- 172.21.15.47:0/3749807988 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b24161240 con 0x7f1b2415ca90 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.058+0000 7f1b2359e640 1 --2- 172.21.15.47:0/3749807988 >> v2:172.21.15.175:3300/0 conn(0x7f1b240cdc40 0x7f1b240b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.059+0000 7f1b22d9d640 1 --2- 172.21.15.47:0/3749807988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b2415ca90 0x7f1b24160710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1b24161800 tx=0x7f1b1402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d6b1a60efb1736 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.059+0000 7f1b2259c640 1 -- 172.21.15.47:0/3749807988 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b140317e0 con 0x7f1b2415ca90 2024-03-23T09:41:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.059+0000 7f1b2259c640 1 -- 172.21.15.47:0/3749807988 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b14031940 con 0x7f1b2415ca90 2024-03-23T09:41:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.059+0000 7f1b2259c640 1 -- 172.21.15.47:0/3749807988 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b14031cd0 con 0x7f1b2415ca90 2024-03-23T09:41:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.059+0000 7f1b29057640 1 -- 172.21.15.47:0/3749807988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b2415ca90 msgr2=0x7f1b24160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.059+0000 7f1b29057640 1 --2- 172.21.15.47:0/3749807988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b2415ca90 0x7f1b24160710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1b24161800 tx=0x7f1b1402b880 comp rx=0 tx=0).stop 2024-03-23T09:41:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.060+0000 7f1b29057640 1 -- 172.21.15.47:0/3749807988 shutdown_connections 2024-03-23T09:41:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.060+0000 7f1b29057640 1 --2- 172.21.15.47:0/3749807988 >> v2:172.21.15.175:3300/0 conn(0x7f1b240cdc40 0x7f1b240b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.060+0000 7f1b29057640 1 --2- 172.21.15.47:0/3749807988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b240b8350 0x7f1b240cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.060+0000 7f1b29057640 1 --2- 172.21.15.47:0/3749807988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b2415ca90 0x7f1b24160710 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.060+0000 7f1b29057640 1 -- 172.21.15.47:0/3749807988 >> 172.21.15.47:0/3749807988 conn(0x7f1b240659d0 msgr2=0x7f1b24065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.060+0000 7f1b29057640 1 -- 172.21.15.47:0/3749807988 shutdown_connections 2024-03-23T09:41:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.060+0000 7f1b29057640 1 -- 172.21.15.47:0/3749807988 wait complete. 2024-03-23T09:41:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.061+0000 7f1b29057640 1 Processor -- start 2024-03-23T09:41:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.061+0000 7f1b29057640 1 -- start start 2024-03-23T09:41:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b29057640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b240b8350 0x7f1b24151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b29057640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b240cdc40 0x7f1b24150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b29057640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b2415ca90 0x7f1b24150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b29057640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240b4c70 con 0x7f1b240cdc40 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b29057640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240b40f0 con 0x7f1b240b8350 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b29057640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240b43d0 con 0x7f1b2415ca90 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b1bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b240cdc40 0x7f1b24150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b1bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b240cdc40 0x7f1b24150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46292/0 (socket says 172.21.15.47:46292) 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b1bfff640 1 -- 172.21.15.47:0/2294734020 learned_addr learned my addr 172.21.15.47:0/2294734020 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b22d9d640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b240b8350 0x7f1b24151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b2359e640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b2415ca90 0x7f1b24150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b1bfff640 1 -- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b2415ca90 msgr2=0x7f1b24150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b1bfff640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b2415ca90 0x7f1b24150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b1bfff640 1 -- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b240b8350 msgr2=0x7f1b24151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b1bfff640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b240b8350 0x7f1b24151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b1bfff640 1 -- 172.21.15.47:0/2294734020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b140094d0 con 0x7f1b240cdc40 2024-03-23T09:41:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b2359e640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b2415ca90 0x7f1b24150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b1bfff640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b240cdc40 0x7f1b24150aa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00b410 tx=0x7f1b0c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b1b7fe640 1 -- 172.21.15.47:0/2294734020 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c011830 con 0x7f1b240cdc40 2024-03-23T09:41:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b1b7fe640 1 -- 172.21.15.47:0/2294734020 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c0119b0 con 0x7f1b240cdc40 2024-03-23T09:41:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b29057640 1 -- 172.21.15.47:0/2294734020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b240b46b0 con 0x7f1b240cdc40 2024-03-23T09:41:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b1b7fe640 1 -- 172.21.15.47:0/2294734020 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c011d60 con 0x7f1b240cdc40 2024-03-23T09:41:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.062+0000 7f1b29057640 1 -- 172.21.15.47:0/2294734020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b241521d0 con 0x7f1b240cdc40 2024-03-23T09:41:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.063+0000 7f1b29057640 1 -- 172.21.15.47:0/2294734020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b24072360 con 0x7f1b240cdc40 2024-03-23T09:41:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.064+0000 7f1b1b7fe640 1 -- 172.21.15.47:0/2294734020 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1b0c01a400 con 0x7f1b240cdc40 2024-03-23T09:41:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.064+0000 7f1b1b7fe640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b0406ac00 0x7f1b0406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.064+0000 7f1b1b7fe640 1 -- 172.21.15.47:0/2294734020 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b0c0894b0 con 0x7f1b240cdc40 2024-03-23T09:41:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.064+0000 7f1b22d9d640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b0406ac00 0x7f1b0406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.065+0000 7f1b22d9d640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b0406ac00 0x7f1b0406d0c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b14036000 tx=0x7f1b1403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:13.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.068+0000 7f1b1b7fe640 1 -- 172.21.15.47:0/2294734020 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1b0c054690 con 0x7f1b240cdc40 2024-03-23T09:41:13.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.242+0000 7f1b29057640 1 -- 172.21.15.47:0/2294734020 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b24160c50 con 0x7f1b0406ac00 2024-03-23T09:41:13.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.250+0000 7f1b1b7fe640 1 -- 172.21.15.47:0/2294734020 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264564 (secure 0 0 0) 0x7f1b14031c30 con 0x7f1b0406ac00 2024-03-23T09:41:13.252 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:13.253 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:41:13.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.252+0000 7f1b29057640 1 -- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b0406ac00 msgr2=0x7f1b0406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.252+0000 7f1b29057640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b0406ac00 0x7f1b0406d0c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b14036000 tx=0x7f1b1403d040 comp rx=0 tx=0).stop 2024-03-23T09:41:13.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.252+0000 7f1b29057640 1 -- 172.21.15.47:0/2294734020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b240cdc40 msgr2=0x7f1b24150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.252+0000 7f1b29057640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b240cdc40 0x7f1b24150aa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00b410 tx=0x7f1b0c00b8d0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.253+0000 7f1b29057640 1 -- 172.21.15.47:0/2294734020 shutdown_connections 2024-03-23T09:41:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.253+0000 7f1b29057640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b0406ac00 0x7f1b0406d0c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.253+0000 7f1b29057640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b2415ca90 0x7f1b24150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.253+0000 7f1b29057640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b240cdc40 0x7f1b24150aa0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.253+0000 7f1b29057640 1 --2- 172.21.15.47:0/2294734020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b240b8350 0x7f1b24151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.253+0000 7f1b29057640 1 -- 172.21.15.47:0/2294734020 >> 172.21.15.47:0/2294734020 conn(0x7f1b240659d0 msgr2=0x7f1b2415f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.253+0000 7f1b29057640 1 -- 172.21.15.47:0/2294734020 shutdown_connections 2024-03-23T09:41:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.253+0000 7f1b29057640 1 -- 172.21.15.47:0/2294734020 wait complete. 2024-03-23T09:41:13.263 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-23T09:41:12.339464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236068,"kb_used_data":2912,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738652,"statfs":{"total":773094113280,"available":772852379648,"internally_reserved":0,"allocated":2981888,"data_stored":1788774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.196352"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.147121+0000","last_change":"2024-03-23T09:41:03.164352+0000","last_active":"2024-03-23T09:41:04.147121+0000","last_peered":"2024-03-23T09:41:04.147121+0000","last_clean":"2024-03-23T09:41:04.147121+0000","last_became_active":"2024-03-23T09:41:03.164058+0000","last_became_peered":"2024-03-23T09:41:03.164058+0000","last_unstale":"2024-03-23T09:41:04.147121+0000","last_undegraded":"2024-03-23T09:41:04.147121+0000","last_fullsized":"2024-03-23T09:41:04.147121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:54:59.125701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142356+0000","last_change":"2024-03-23T09:41:03.159612+0000","last_active":"2024-03-23T09:41:04.142356+0000","last_peered":"2024-03-23T09:41:04.142356+0000","last_clean":"2024-03-23T09:41:04.142356+0000","last_became_active":"2024-03-23T09:41:03.159341+0000","last_became_peered":"2024-03-23T09:41:03.159341+0000","last_unstale":"2024-03-23T09:41:04.142356+0000","last_undegraded":"2024-03-23T09:41:04.142356+0000","last_fullsized":"2024-03-23T09:41:04.142356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586632+0000","last_change":"2024-03-23T09:41:03.156325+0000","last_active":"2024-03-23T09:41:04.586632+0000","last_peered":"2024-03-23T09:41:04.586632+0000","last_clean":"2024-03-23T09:41:04.586632+0000","last_became_active":"2024-03-23T09:41:03.156061+0000","last_became_peered":"2024-03-23T09:41:03.156061+0000","last_unstale":"2024-03-23T09:41:04.586632+0000","last_undegraded":"2024-03-23T09:41:04.586632+0000","last_fullsized":"2024-03-23T09:41:04.586632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:06:51.567637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155051+0000","last_change":"2024-03-23T09:41:03.158679+0000","last_active":"2024-03-23T09:41:04.155051+0000","last_peered":"2024-03-23T09:41:04.155051+0000","last_clean":"2024-03-23T09:41:04.155051+0000","last_became_active":"2024-03-23T09:41:03.158151+0000","last_became_peered":"2024-03-23T09:41:03.158151+0000","last_unstale":"2024-03-23T09:41:04.155051+0000","last_undegraded":"2024-03-23T09:41:04.155051+0000","last_fullsized":"2024-03-23T09:41:04.155051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:29.855586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146283+0000","last_change":"2024-03-23T09:41:03.167599+0000","last_active":"2024-03-23T09:41:04.146283+0000","last_peered":"2024-03-23T09:41:04.146283+0000","last_clean":"2024-03-23T09:41:04.146283+0000","last_became_active":"2024-03-23T09:41:03.167332+0000","last_became_peered":"2024-03-23T09:41:03.167332+0000","last_unstale":"2024-03-23T09:41:04.146283+0000","last_undegraded":"2024-03-23T09:41:04.146283+0000","last_fullsized":"2024-03-23T09:41:04.146283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:32:17.579489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.154914+0000","last_change":"2024-03-23T09:41:03.162714+0000","last_active":"2024-03-23T09:41:04.154914+0000","last_peered":"2024-03-23T09:41:04.154914+0000","last_clean":"2024-03-23T09:41:04.154914+0000","last_became_active":"2024-03-23T09:41:03.162461+0000","last_became_peered":"2024-03-23T09:41:03.162461+0000","last_unstale":"2024-03-23T09:41:04.154914+0000","last_undegraded":"2024-03-23T09:41:04.154914+0000","last_fullsized":"2024-03-23T09:41:04.154914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:30:48.469383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146273+0000","last_change":"2024-03-23T09:41:03.167591+0000","last_active":"2024-03-23T09:41:04.146273+0000","last_peered":"2024-03-23T09:41:04.146273+0000","last_clean":"2024-03-23T09:41:04.146273+0000","last_became_active":"2024-03-23T09:41:03.167320+0000","last_became_peered":"2024-03-23T09:41:03.167320+0000","last_unstale":"2024-03-23T09:41:04.146273+0000","last_undegraded":"2024-03-23T09:41:04.146273+0000","last_fullsized":"2024-03-23T09:41:04.146273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:29.855586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665874+0000","last_change":"2024-03-23T09:41:03.153495+0000","last_active":"2024-03-23T09:41:04.665874+0000","last_peered":"2024-03-23T09:41:04.665874+0000","last_clean":"2024-03-23T09:41:04.665874+0000","last_became_active":"2024-03-23T09:41:03.153257+0000","last_became_peered":"2024-03-23T09:41:03.153257+0000","last_unstale":"2024-03-23T09:41:04.665874+0000","last_undegraded":"2024-03-23T09:41:04.665874+0000","last_fullsized":"2024-03-23T09:41:04.665874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:46:38.667131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.147216+0000","last_change":"2024-03-23T09:41:03.170688+0000","last_active":"2024-03-23T09:41:04.147216+0000","last_peered":"2024-03-23T09:41:04.147216+0000","last_clean":"2024-03-23T09:41:04.147216+0000","last_became_active":"2024-03-23T09:41:03.170446+0000","last_became_peered":"2024-03-23T09:41:03.170446+0000","last_unstale":"2024-03-23T09:41:04.147216+0000","last_undegraded":"2024-03-23T09:41:04.147216+0000","last_fullsized":"2024-03-23T09:41:04.147216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:22:15.797821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666694+0000","last_change":"2024-03-23T09:41:03.162169+0000","last_active":"2024-03-23T09:41:04.666694+0000","last_peered":"2024-03-23T09:41:04.666694+0000","last_clean":"2024-03-23T09:41:04.666694+0000","last_became_active":"2024-03-23T09:41:03.161587+0000","last_became_peered":"2024-03-23T09:41:03.161587+0000","last_unstale":"2024-03-23T09:41:04.666694+0000","last_undegraded":"2024-03-23T09:41:04.666694+0000","last_fullsized":"2024-03-23T09:41:04.666694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666339+0000","last_change":"2024-03-23T09:41:03.160438+0000","last_active":"2024-03-23T09:41:04.666339+0000","last_peered":"2024-03-23T09:41:04.666339+0000","last_clean":"2024-03-23T09:41:04.666339+0000","last_became_active":"2024-03-23T09:41:03.160175+0000","last_became_peered":"2024-03-23T09:41:03.160175+0000","last_unstale":"2024-03-23T09:41:04.666339+0000","last_undegraded":"2024-03-23T09:41:04.666339+0000","last_fullsized":"2024-03-23T09:41:04.666339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146726+0000","last_change":"2024-03-23T09:41:03.163213+0000","last_active":"2024-03-23T09:41:04.146726+0000","last_peered":"2024-03-23T09:41:04.146726+0000","last_clean":"2024-03-23T09:41:04.146726+0000","last_became_active":"2024-03-23T09:41:03.162954+0000","last_became_peered":"2024-03-23T09:41:03.162954+0000","last_unstale":"2024-03-23T09:41:04.146726+0000","last_undegraded":"2024-03-23T09:41:04.146726+0000","last_fullsized":"2024-03-23T09:41:04.146726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:28.007092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666209+0000","last_change":"2024-03-23T09:41:03.161818+0000","last_active":"2024-03-23T09:41:04.666209+0000","last_peered":"2024-03-23T09:41:04.666209+0000","last_clean":"2024-03-23T09:41:04.666209+0000","last_became_active":"2024-03-23T09:41:03.159384+0000","last_became_peered":"2024-03-23T09:41:03.159384+0000","last_unstale":"2024-03-23T09:41:04.666209+0000","last_undegraded":"2024-03-23T09:41:04.666209+0000","last_fullsized":"2024-03-23T09:41:04.666209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586410+0000","last_change":"2024-03-23T09:41:03.159776+0000","last_active":"2024-03-23T09:41:04.586410+0000","last_peered":"2024-03-23T09:41:04.586410+0000","last_clean":"2024-03-23T09:41:04.586410+0000","last_became_active":"2024-03-23T09:41:03.159466+0000","last_became_peered":"2024-03-23T09:41:03.159466+0000","last_unstale":"2024-03-23T09:41:04.586410+0000","last_undegraded":"2024-03-23T09:41:04.586410+0000","last_fullsized":"2024-03-23T09:41:04.586410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142102+0000","last_change":"2024-03-23T09:41:03.154187+0000","last_active":"2024-03-23T09:41:04.142102+0000","last_peered":"2024-03-23T09:41:04.142102+0000","last_clean":"2024-03-23T09:41:04.142102+0000","last_became_active":"2024-03-23T09:41:03.153946+0000","last_became_peered":"2024-03-23T09:41:03.153946+0000","last_unstale":"2024-03-23T09:41:04.142102+0000","last_undegraded":"2024-03-23T09:41:04.142102+0000","last_fullsized":"2024-03-23T09:41:04.142102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.154801+0000","last_change":"2024-03-23T09:41:03.160290+0000","last_active":"2024-03-23T09:41:04.154801+0000","last_peered":"2024-03-23T09:41:04.154801+0000","last_clean":"2024-03-23T09:41:04.154801+0000","last_became_active":"2024-03-23T09:41:03.160010+0000","last_became_peered":"2024-03-23T09:41:03.160010+0000","last_unstale":"2024-03-23T09:41:04.154801+0000","last_undegraded":"2024-03-23T09:41:04.154801+0000","last_fullsized":"2024-03-23T09:41:04.154801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155450+0000","last_change":"2024-03-23T09:41:03.165189+0000","last_active":"2024-03-23T09:41:04.155450+0000","last_peered":"2024-03-23T09:41:04.155450+0000","last_clean":"2024-03-23T09:41:04.155450+0000","last_became_active":"2024-03-23T09:41:03.164956+0000","last_became_peered":"2024-03-23T09:41:03.164956+0000","last_unstale":"2024-03-23T09:41:04.155450+0000","last_undegraded":"2024-03-23T09:41:04.155450+0000","last_fullsized":"2024-03-23T09:41:04.155450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665970+0000","last_change":"2024-03-23T09:41:03.163535+0000","last_active":"2024-03-23T09:41:04.665970+0000","last_peered":"2024-03-23T09:41:04.665970+0000","last_clean":"2024-03-23T09:41:04.665970+0000","last_became_active":"2024-03-23T09:41:03.163271+0000","last_became_peered":"2024-03-23T09:41:03.163271+0000","last_unstale":"2024-03-23T09:41:04.665970+0000","last_undegraded":"2024-03-23T09:41:04.665970+0000","last_fullsized":"2024-03-23T09:41:04.665970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:12.280151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146986+0000","last_change":"2024-03-23T09:41:03.168829+0000","last_active":"2024-03-23T09:41:04.146986+0000","last_peered":"2024-03-23T09:41:04.146986+0000","last_clean":"2024-03-23T09:41:04.146986+0000","last_became_active":"2024-03-23T09:41:03.167734+0000","last_became_peered":"2024-03-23T09:41:03.167734+0000","last_unstale":"2024-03-23T09:41:04.146986+0000","last_undegraded":"2024-03-23T09:41:04.146986+0000","last_fullsized":"2024-03-23T09:41:04.146986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:11:48.328619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666441+0000","last_change":"2024-03-23T09:41:03.161820+0000","last_active":"2024-03-23T09:41:04.666441+0000","last_peered":"2024-03-23T09:41:04.666441+0000","last_clean":"2024-03-23T09:41:04.666441+0000","last_became_active":"2024-03-23T09:41:03.161310+0000","last_became_peered":"2024-03-23T09:41:03.161310+0000","last_unstale":"2024-03-23T09:41:04.666441+0000","last_undegraded":"2024-03-23T09:41:04.666441+0000","last_fullsized":"2024-03-23T09:41:04.666441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:42:19.838774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586549+0000","last_change":"2024-03-23T09:41:03.171444+0000","last_active":"2024-03-23T09:41:04.586549+0000","last_peered":"2024-03-23T09:41:04.586549+0000","last_clean":"2024-03-23T09:41:04.586549+0000","last_became_active":"2024-03-23T09:41:03.171114+0000","last_became_peered":"2024-03-23T09:41:03.171114+0000","last_unstale":"2024-03-23T09:41:04.586549+0000","last_undegraded":"2024-03-23T09:41:04.586549+0000","last_fullsized":"2024-03-23T09:41:04.586549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.154962+0000","last_change":"2024-03-23T09:41:03.166192+0000","last_active":"2024-03-23T09:41:04.154962+0000","last_peered":"2024-03-23T09:41:04.154962+0000","last_clean":"2024-03-23T09:41:04.154962+0000","last_became_active":"2024-03-23T09:41:03.165944+0000","last_became_peered":"2024-03-23T09:41:03.165944+0000","last_unstale":"2024-03-23T09:41:04.154962+0000","last_undegraded":"2024-03-23T09:41:04.154962+0000","last_fullsized":"2024-03-23T09:41:04.154962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:51:26.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666043+0000","last_change":"2024-03-23T09:41:03.149122+0000","last_active":"2024-03-23T09:41:04.666043+0000","last_peered":"2024-03-23T09:41:04.666043+0000","last_clean":"2024-03-23T09:41:04.666043+0000","last_became_active":"2024-03-23T09:41:03.148885+0000","last_became_peered":"2024-03-23T09:41:03.148885+0000","last_unstale":"2024-03-23T09:41:04.666043+0000","last_undegraded":"2024-03-23T09:41:04.666043+0000","last_fullsized":"2024-03-23T09:41:04.666043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:06.444696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.154922+0000","last_change":"2024-03-23T09:41:03.155719+0000","last_active":"2024-03-23T09:41:04.154922+0000","last_peered":"2024-03-23T09:41:04.154922+0000","last_clean":"2024-03-23T09:41:04.154922+0000","last_became_active":"2024-03-23T09:41:03.155436+0000","last_became_peered":"2024-03-23T09:41:03.155436+0000","last_unstale":"2024-03-23T09:41:04.154922+0000","last_undegraded":"2024-03-23T09:41:04.154922+0000","last_fullsized":"2024-03-23T09:41:04.154922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666163+0000","last_change":"2024-03-23T09:41:03.148314+0000","last_active":"2024-03-23T09:41:04.666163+0000","last_peered":"2024-03-23T09:41:04.666163+0000","last_clean":"2024-03-23T09:41:04.666163+0000","last_became_active":"2024-03-23T09:41:03.148048+0000","last_became_peered":"2024-03-23T09:41:03.148048+0000","last_unstale":"2024-03-23T09:41:04.666163+0000","last_undegraded":"2024-03-23T09:41:04.666163+0000","last_fullsized":"2024-03-23T09:41:04.666163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:09.777921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666670+0000","last_change":"2024-03-23T09:41:03.169146+0000","last_active":"2024-03-23T09:41:04.666670+0000","last_peered":"2024-03-23T09:41:04.666670+0000","last_clean":"2024-03-23T09:41:04.666670+0000","last_became_active":"2024-03-23T09:41:03.168877+0000","last_became_peered":"2024-03-23T09:41:03.168877+0000","last_unstale":"2024-03-23T09:41:04.666670+0000","last_undegraded":"2024-03-23T09:41:04.666670+0000","last_fullsized":"2024-03-23T09:41:04.666670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:10.603669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142270+0000","last_change":"2024-03-23T09:41:03.147239+0000","last_active":"2024-03-23T09:41:04.142270+0000","last_peered":"2024-03-23T09:41:04.142270+0000","last_clean":"2024-03-23T09:41:04.142270+0000","last_became_active":"2024-03-23T09:41:03.146990+0000","last_became_peered":"2024-03-23T09:41:03.146990+0000","last_unstale":"2024-03-23T09:41:04.142270+0000","last_undegraded":"2024-03-23T09:41:04.142270+0000","last_fullsized":"2024-03-23T09:41:04.142270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666449+0000","last_change":"2024-03-23T09:41:03.149202+0000","last_active":"2024-03-23T09:41:04.666449+0000","last_peered":"2024-03-23T09:41:04.666449+0000","last_clean":"2024-03-23T09:41:04.666449+0000","last_became_active":"2024-03-23T09:41:03.148958+0000","last_became_peered":"2024-03-23T09:41:03.148958+0000","last_unstale":"2024-03-23T09:41:04.666449+0000","last_undegraded":"2024-03-23T09:41:04.666449+0000","last_fullsized":"2024-03-23T09:41:04.666449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:10.603669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586617+0000","last_change":"2024-03-23T09:41:03.157714+0000","last_active":"2024-03-23T09:41:04.586617+0000","last_peered":"2024-03-23T09:41:04.586617+0000","last_clean":"2024-03-23T09:41:04.586617+0000","last_became_active":"2024-03-23T09:41:03.157315+0000","last_became_peered":"2024-03-23T09:41:03.157315+0000","last_unstale":"2024-03-23T09:41:04.586617+0000","last_undegraded":"2024-03-23T09:41:04.586617+0000","last_fullsized":"2024-03-23T09:41:04.586617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:06.444696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155043+0000","last_change":"2024-03-23T09:41:03.155373+0000","last_active":"2024-03-23T09:41:04.155043+0000","last_peered":"2024-03-23T09:41:04.155043+0000","last_clean":"2024-03-23T09:41:04.155043+0000","last_became_active":"2024-03-23T09:41:03.155063+0000","last_became_peered":"2024-03-23T09:41:03.155063+0000","last_unstale":"2024-03-23T09:41:04.155043+0000","last_undegraded":"2024-03-23T09:41:04.155043+0000","last_fullsized":"2024-03-23T09:41:04.155043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:33:41.605578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665826+0000","last_change":"2024-03-23T09:41:03.154532+0000","last_active":"2024-03-23T09:41:04.665826+0000","last_peered":"2024-03-23T09:41:04.665826+0000","last_clean":"2024-03-23T09:41:04.665826+0000","last_became_active":"2024-03-23T09:41:03.154277+0000","last_became_peered":"2024-03-23T09:41:03.154277+0000","last_unstale":"2024-03-23T09:41:04.665826+0000","last_undegraded":"2024-03-23T09:41:04.665826+0000","last_fullsized":"2024-03-23T09:41:04.665826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:49.353786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666387+0000","last_change":"2024-03-23T09:41:03.166933+0000","last_active":"2024-03-23T09:41:04.666387+0000","last_peered":"2024-03-23T09:41:04.666387+0000","last_clean":"2024-03-23T09:41:04.666387+0000","last_became_active":"2024-03-23T09:41:03.166359+0000","last_became_peered":"2024-03-23T09:41:03.166359+0000","last_unstale":"2024-03-23T09:41:04.666387+0000","last_undegraded":"2024-03-23T09:41:04.666387+0000","last_fullsized":"2024-03-23T09:41:04.666387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:09.777921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146129+0000","last_change":"2024-03-23T09:41:03.149144+0000","last_active":"2024-03-23T09:41:04.146129+0000","last_peered":"2024-03-23T09:41:04.146129+0000","last_clean":"2024-03-23T09:41:04.146129+0000","last_became_active":"2024-03-23T09:41:03.148842+0000","last_became_peered":"2024-03-23T09:41:03.148842+0000","last_unstale":"2024-03-23T09:41:04.146129+0000","last_undegraded":"2024-03-23T09:41:04.146129+0000","last_fullsized":"2024-03-23T09:41:04.146129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:12.280151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.167252+0000","last_change":"2024-03-23T09:41:02.789724+0000","last_active":"2024-03-23T09:41:03.167252+0000","last_peered":"2024-03-23T09:41:03.167252+0000","last_clean":"2024-03-23T09:41:03.167252+0000","last_became_active":"2024-03-23T09:41:02.789485+0000","last_became_peered":"2024-03-23T09:41:02.789485+0000","last_unstale":"2024-03-23T09:41:03.167252+0000","last_undegraded":"2024-03-23T09:41:03.167252+0000","last_fullsized":"2024-03-23T09:41:03.167252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586658+0000","last_change":"2024-03-23T09:41:03.157560+0000","last_active":"2024-03-23T09:41:04.586658+0000","last_peered":"2024-03-23T09:41:04.586658+0000","last_clean":"2024-03-23T09:41:04.586658+0000","last_became_active":"2024-03-23T09:41:03.157046+0000","last_became_peered":"2024-03-23T09:41:03.157046+0000","last_unstale":"2024-03-23T09:41:04.586658+0000","last_undegraded":"2024-03-23T09:41:04.586658+0000","last_fullsized":"2024-03-23T09:41:04.586658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:04:04.702736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.161341+0000","last_change":"2024-03-23T09:41:02.785765+0000","last_active":"2024-03-23T09:41:03.161341+0000","last_peered":"2024-03-23T09:41:03.161341+0000","last_clean":"2024-03-23T09:41:03.161341+0000","last_became_active":"2024-03-23T09:41:02.785483+0000","last_became_peered":"2024-03-23T09:41:02.785483+0000","last_unstale":"2024-03-23T09:41:03.161341+0000","last_undegraded":"2024-03-23T09:41:03.161341+0000","last_fullsized":"2024-03-23T09:41:03.161341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.145332+0000","last_change":"2024-03-23T09:41:02.790132+0000","last_active":"2024-03-23T09:41:03.145332+0000","last_peered":"2024-03-23T09:41:03.145332+0000","last_clean":"2024-03-23T09:41:03.145332+0000","last_became_active":"2024-03-23T09:41:02.789893+0000","last_became_peered":"2024-03-23T09:41:02.789893+0000","last_unstale":"2024-03-23T09:41:03.145332+0000","last_undegraded":"2024-03-23T09:41:03.145332+0000","last_fullsized":"2024-03-23T09:41:03.145332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665949+0000","last_change":"2024-03-23T09:41:03.154684+0000","last_active":"2024-03-23T09:41:04.665949+0000","last_peered":"2024-03-23T09:41:04.665949+0000","last_clean":"2024-03-23T09:41:04.665949+0000","last_became_active":"2024-03-23T09:41:03.154441+0000","last_became_peered":"2024-03-23T09:41:03.154441+0000","last_unstale":"2024-03-23T09:41:04.665949+0000","last_undegraded":"2024-03-23T09:41:04.665949+0000","last_fullsized":"2024-03-23T09:41:04.665949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.149205+0000","last_change":"2024-03-23T09:41:02.792603+0000","last_active":"2024-03-23T09:41:03.149205+0000","last_peered":"2024-03-23T09:41:03.149205+0000","last_clean":"2024-03-23T09:41:03.149205+0000","last_became_active":"2024-03-23T09:41:02.792316+0000","last_became_peered":"2024-03-23T09:41:02.792316+0000","last_unstale":"2024-03-23T09:41:03.149205+0000","last_undegraded":"2024-03-23T09:41:03.149205+0000","last_fullsized":"2024-03-23T09:41:03.149205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142096+0000","last_change":"2024-03-23T09:41:03.147255+0000","last_active":"2024-03-23T09:41:04.142096+0000","last_peered":"2024-03-23T09:41:04.142096+0000","last_clean":"2024-03-23T09:41:04.142096+0000","last_became_active":"2024-03-23T09:41:03.147013+0000","last_became_peered":"2024-03-23T09:41:03.147013+0000","last_unstale":"2024-03-23T09:41:04.142096+0000","last_undegraded":"2024-03-23T09:41:04.142096+0000","last_fullsized":"2024-03-23T09:41:04.142096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:16.974803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.152789+0000","last_change":"2024-03-23T09:41:02.791718+0000","last_active":"2024-03-23T09:41:03.152789+0000","last_peered":"2024-03-23T09:41:03.152789+0000","last_clean":"2024-03-23T09:41:03.152789+0000","last_became_active":"2024-03-23T09:41:02.791442+0000","last_became_peered":"2024-03-23T09:41:02.791442+0000","last_unstale":"2024-03-23T09:41:03.152789+0000","last_undegraded":"2024-03-23T09:41:03.152789+0000","last_fullsized":"2024-03-23T09:41:03.152789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155272+0000","last_change":"2024-03-23T09:41:03.171999+0000","last_active":"2024-03-23T09:41:04.155272+0000","last_peered":"2024-03-23T09:41:04.155272+0000","last_clean":"2024-03-23T09:41:04.155272+0000","last_became_active":"2024-03-23T09:41:03.171731+0000","last_became_peered":"2024-03-23T09:41:03.171731+0000","last_unstale":"2024-03-23T09:41:04.155272+0000","last_undegraded":"2024-03-23T09:41:04.155272+0000","last_fullsized":"2024-03-23T09:41:04.155272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:16.974803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.150565+0000","last_change":"2024-03-23T09:41:02.791445+0000","last_active":"2024-03-23T09:41:03.150565+0000","last_peered":"2024-03-23T09:41:03.150565+0000","last_clean":"2024-03-23T09:41:03.150565+0000","last_became_active":"2024-03-23T09:41:02.791075+0000","last_became_peered":"2024-03-23T09:41:02.791075+0000","last_unstale":"2024-03-23T09:41:03.150565+0000","last_undegraded":"2024-03-23T09:41:03.150565+0000","last_fullsized":"2024-03-23T09:41:03.150565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586653+0000","last_change":"2024-03-23T09:41:03.159171+0000","last_active":"2024-03-23T09:41:04.586653+0000","last_peered":"2024-03-23T09:41:04.586653+0000","last_clean":"2024-03-23T09:41:04.586653+0000","last_became_active":"2024-03-23T09:41:03.158869+0000","last_became_peered":"2024-03-23T09:41:03.158869+0000","last_unstale":"2024-03-23T09:41:04.586653+0000","last_undegraded":"2024-03-23T09:41:04.586653+0000","last_fullsized":"2024-03-23T09:41:04.586653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:42:19.838774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.160077+0000","last_change":"2024-03-23T09:41:02.786747+0000","last_active":"2024-03-23T09:41:03.160077+0000","last_peered":"2024-03-23T09:41:03.160077+0000","last_clean":"2024-03-23T09:41:03.160077+0000","last_became_active":"2024-03-23T09:41:02.786499+0000","last_became_peered":"2024-03-23T09:41:02.786499+0000","last_unstale":"2024-03-23T09:41:03.160077+0000","last_undegraded":"2024-03-23T09:41:03.160077+0000","last_fullsized":"2024-03-23T09:41:03.160077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146832+0000","last_change":"2024-03-23T09:41:03.165917+0000","last_active":"2024-03-23T09:41:04.146832+0000","last_peered":"2024-03-23T09:41:04.146832+0000","last_clean":"2024-03-23T09:41:04.146832+0000","last_became_active":"2024-03-23T09:41:03.165617+0000","last_became_peered":"2024-03-23T09:41:03.165617+0000","last_unstale":"2024-03-23T09:41:04.146832+0000","last_undegraded":"2024-03-23T09:41:04.146832+0000","last_fullsized":"2024-03-23T09:41:04.146832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:49:00.143163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.169412+0000","last_change":"2024-03-23T09:41:02.787570+0000","last_active":"2024-03-23T09:41:03.169412+0000","last_peered":"2024-03-23T09:41:03.169412+0000","last_clean":"2024-03-23T09:41:03.169412+0000","last_became_active":"2024-03-23T09:41:02.785815+0000","last_became_peered":"2024-03-23T09:41:02.785815+0000","last_unstale":"2024-03-23T09:41:03.169412+0000","last_undegraded":"2024-03-23T09:41:03.169412+0000","last_fullsized":"2024-03-23T09:41:03.169412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146644+0000","last_change":"2024-03-23T09:41:03.167089+0000","last_active":"2024-03-23T09:41:04.146644+0000","last_peered":"2024-03-23T09:41:04.146644+0000","last_clean":"2024-03-23T09:41:04.146644+0000","last_became_active":"2024-03-23T09:41:03.166788+0000","last_became_peered":"2024-03-23T09:41:03.166788+0000","last_unstale":"2024-03-23T09:41:04.146644+0000","last_undegraded":"2024-03-23T09:41:04.146644+0000","last_fullsized":"2024-03-23T09:41:04.146644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:22:09.627571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.161339+0000","last_change":"2024-03-23T09:41:02.785499+0000","last_active":"2024-03-23T09:41:03.161339+0000","last_peered":"2024-03-23T09:41:03.161339+0000","last_clean":"2024-03-23T09:41:03.161339+0000","last_became_active":"2024-03-23T09:41:02.785228+0000","last_became_peered":"2024-03-23T09:41:02.785228+0000","last_unstale":"2024-03-23T09:41:03.161339+0000","last_undegraded":"2024-03-23T09:41:03.161339+0000","last_fullsized":"2024-03-23T09:41:03.161339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586476+0000","last_change":"2024-03-23T09:41:03.164565+0000","last_active":"2024-03-23T09:41:04.586476+0000","last_peered":"2024-03-23T09:41:04.586476+0000","last_clean":"2024-03-23T09:41:04.586476+0000","last_became_active":"2024-03-23T09:41:03.164285+0000","last_became_peered":"2024-03-23T09:41:03.164285+0000","last_unstale":"2024-03-23T09:41:04.586476+0000","last_undegraded":"2024-03-23T09:41:04.586476+0000","last_fullsized":"2024-03-23T09:41:04.586476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.149928+0000","last_change":"2024-03-23T09:41:02.790603+0000","last_active":"2024-03-23T09:41:03.149928+0000","last_peered":"2024-03-23T09:41:03.149928+0000","last_clean":"2024-03-23T09:41:03.149928+0000","last_became_active":"2024-03-23T09:41:02.790337+0000","last_became_peered":"2024-03-23T09:41:02.790337+0000","last_unstale":"2024-03-23T09:41:03.149928+0000","last_undegraded":"2024-03-23T09:41:03.149928+0000","last_fullsized":"2024-03-23T09:41:03.149928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146310+0000","last_change":"2024-03-23T09:41:03.170711+0000","last_active":"2024-03-23T09:41:04.146310+0000","last_peered":"2024-03-23T09:41:04.146310+0000","last_clean":"2024-03-23T09:41:04.146310+0000","last_became_active":"2024-03-23T09:41:03.170418+0000","last_became_peered":"2024-03-23T09:41:03.170418+0000","last_unstale":"2024-03-23T09:41:04.146310+0000","last_undegraded":"2024-03-23T09:41:04.146310+0000","last_fullsized":"2024-03-23T09:41:04.146310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:16.974803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.153343+0000","last_change":"2024-03-23T09:41:02.788294+0000","last_active":"2024-03-23T09:41:03.153343+0000","last_peered":"2024-03-23T09:41:03.153343+0000","last_clean":"2024-03-23T09:41:03.153343+0000","last_became_active":"2024-03-23T09:41:02.787911+0000","last_became_peered":"2024-03-23T09:41:02.787911+0000","last_unstale":"2024-03-23T09:41:03.153343+0000","last_undegraded":"2024-03-23T09:41:03.153343+0000","last_fullsized":"2024-03-23T09:41:03.153343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:09:59.090234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666362+0000","last_change":"2024-03-23T09:41:03.166733+0000","last_active":"2024-03-23T09:41:04.666362+0000","last_peered":"2024-03-23T09:41:04.666362+0000","last_clean":"2024-03-23T09:41:04.666362+0000","last_became_active":"2024-03-23T09:41:03.165156+0000","last_became_peered":"2024-03-23T09:41:03.165156+0000","last_unstale":"2024-03-23T09:41:04.666362+0000","last_undegraded":"2024-03-23T09:41:04.666362+0000","last_fullsized":"2024-03-23T09:41:04.666362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:06.444696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.163373+0000","last_change":"2024-03-23T09:41:02.784571+0000","last_active":"2024-03-23T09:41:03.163373+0000","last_peered":"2024-03-23T09:41:03.163373+0000","last_clean":"2024-03-23T09:41:03.163373+0000","last_became_active":"2024-03-23T09:41:02.784282+0000","last_became_peered":"2024-03-23T09:41:02.784282+0000","last_unstale":"2024-03-23T09:41:03.163373+0000","last_undegraded":"2024-03-23T09:41:03.163373+0000","last_fullsized":"2024-03-23T09:41:03.163373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146163+0000","last_change":"2024-03-23T09:41:03.168817+0000","last_active":"2024-03-23T09:41:04.146163+0000","last_peered":"2024-03-23T09:41:04.146163+0000","last_clean":"2024-03-23T09:41:04.146163+0000","last_became_active":"2024-03-23T09:41:03.168491+0000","last_became_peered":"2024-03-23T09:41:03.168491+0000","last_unstale":"2024-03-23T09:41:04.146163+0000","last_undegraded":"2024-03-23T09:41:04.146163+0000","last_fullsized":"2024-03-23T09:41:04.146163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:51:26.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.162936+0000","last_change":"2024-03-23T09:41:02.787278+0000","last_active":"2024-03-23T09:41:03.162936+0000","last_peered":"2024-03-23T09:41:03.162936+0000","last_clean":"2024-03-23T09:41:03.162936+0000","last_became_active":"2024-03-23T09:41:02.786760+0000","last_became_peered":"2024-03-23T09:41:02.786760+0000","last_unstale":"2024-03-23T09:41:03.162936+0000","last_undegraded":"2024-03-23T09:41:03.162936+0000","last_fullsized":"2024-03-23T09:41:03.162936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146844+0000","last_change":"2024-03-23T09:41:03.168702+0000","last_active":"2024-03-23T09:41:04.146844+0000","last_peered":"2024-03-23T09:41:04.146844+0000","last_clean":"2024-03-23T09:41:04.146844+0000","last_became_active":"2024-03-23T09:41:03.168425+0000","last_became_peered":"2024-03-23T09:41:03.168425+0000","last_unstale":"2024-03-23T09:41:04.146844+0000","last_undegraded":"2024-03-23T09:41:04.146844+0000","last_fullsized":"2024-03-23T09:41:04.146844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.158014+0000","last_change":"2024-03-23T09:41:02.785601+0000","last_active":"2024-03-23T09:41:03.158014+0000","last_peered":"2024-03-23T09:41:03.158014+0000","last_clean":"2024-03-23T09:41:03.158014+0000","last_became_active":"2024-03-23T09:41:02.785347+0000","last_became_peered":"2024-03-23T09:41:02.785347+0000","last_unstale":"2024-03-23T09:41:03.158014+0000","last_undegraded":"2024-03-23T09:41:03.158014+0000","last_fullsized":"2024-03-23T09:41:03.158014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.147217+0000","last_change":"2024-03-23T09:41:03.170126+0000","last_active":"2024-03-23T09:41:04.147217+0000","last_peered":"2024-03-23T09:41:04.147217+0000","last_clean":"2024-03-23T09:41:04.147217+0000","last_became_active":"2024-03-23T09:41:03.169191+0000","last_became_peered":"2024-03-23T09:41:03.169191+0000","last_unstale":"2024-03-23T09:41:04.147217+0000","last_undegraded":"2024-03-23T09:41:04.147217+0000","last_fullsized":"2024-03-23T09:41:04.147217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:33:41.605578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.168081+0000","last_change":"2024-03-23T09:41:02.781849+0000","last_active":"2024-03-23T09:41:03.168081+0000","last_peered":"2024-03-23T09:41:03.168081+0000","last_clean":"2024-03-23T09:41:03.168081+0000","last_became_active":"2024-03-23T09:41:02.781582+0000","last_became_peered":"2024-03-23T09:41:02.781582+0000","last_unstale":"2024-03-23T09:41:03.168081+0000","last_undegraded":"2024-03-23T09:41:03.168081+0000","last_fullsized":"2024-03-23T09:41:03.168081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155217+0000","last_change":"2024-03-23T09:41:03.156001+0000","last_active":"2024-03-23T09:41:04.155217+0000","last_peered":"2024-03-23T09:41:04.155217+0000","last_clean":"2024-03-23T09:41:04.155217+0000","last_became_active":"2024-03-23T09:41:03.155692+0000","last_became_peered":"2024-03-23T09:41:03.155692+0000","last_unstale":"2024-03-23T09:41:04.155217+0000","last_undegraded":"2024-03-23T09:41:04.155217+0000","last_fullsized":"2024-03-23T09:41:04.155217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:54:59.125701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.144394+0000","last_change":"2024-03-23T09:41:02.786345+0000","last_active":"2024-03-23T09:41:03.144394+0000","last_peered":"2024-03-23T09:41:03.144394+0000","last_clean":"2024-03-23T09:41:03.144394+0000","last_became_active":"2024-03-23T09:41:02.786070+0000","last_became_peered":"2024-03-23T09:41:02.786070+0000","last_unstale":"2024-03-23T09:41:03.144394+0000","last_undegraded":"2024-03-23T09:41:03.144394+0000","last_fullsized":"2024-03-23T09:41:03.144394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146543+0000","last_change":"2024-03-23T09:41:03.171409+0000","last_active":"2024-03-23T09:41:04.146543+0000","last_peered":"2024-03-23T09:41:04.146543+0000","last_clean":"2024-03-23T09:41:04.146543+0000","last_became_active":"2024-03-23T09:41:03.170992+0000","last_became_peered":"2024-03-23T09:41:03.170992+0000","last_unstale":"2024-03-23T09:41:04.146543+0000","last_undegraded":"2024-03-23T09:41:04.146543+0000","last_fullsized":"2024-03-23T09:41:04.146543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:40:45.654649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.159124+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:41:03.159124+0000","last_peered":"2024-03-23T09:41:03.159124+0000","last_clean":"2024-03-23T09:41:03.159124+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:41:03.159124+0000","last_undegraded":"2024-03-23T09:41:03.159124+0000","last_fullsized":"2024-03-23T09:41:03.159124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.149309+0000","last_change":"2024-03-23T09:41:02.780681+0000","last_active":"2024-03-23T09:41:03.149309+0000","last_peered":"2024-03-23T09:41:03.149309+0000","last_clean":"2024-03-23T09:41:03.149309+0000","last_became_active":"2024-03-23T09:41:02.780040+0000","last_became_peered":"2024-03-23T09:41:02.780040+0000","last_unstale":"2024-03-23T09:41:03.149309+0000","last_undegraded":"2024-03-23T09:41:03.149309+0000","last_fullsized":"2024-03-23T09:41:03.149309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666144+0000","last_change":"2024-03-23T09:41:03.159071+0000","last_active":"2024-03-23T09:41:04.666144+0000","last_peered":"2024-03-23T09:41:04.666144+0000","last_clean":"2024-03-23T09:41:04.666144+0000","last_became_active":"2024-03-23T09:41:03.158637+0000","last_became_peered":"2024-03-23T09:41:03.158637+0000","last_unstale":"2024-03-23T09:41:04.666144+0000","last_undegraded":"2024-03-23T09:41:04.666144+0000","last_fullsized":"2024-03-23T09:41:04.666144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:33:41.605578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.159038+0000","last_change":"2024-03-23T09:41:02.787080+0000","last_active":"2024-03-23T09:41:03.159038+0000","last_peered":"2024-03-23T09:41:03.159038+0000","last_clean":"2024-03-23T09:41:03.159038+0000","last_became_active":"2024-03-23T09:41:02.786573+0000","last_became_peered":"2024-03-23T09:41:02.786573+0000","last_unstale":"2024-03-23T09:41:03.159038+0000","last_undegraded":"2024-03-23T09:41:03.159038+0000","last_fullsized":"2024-03-23T09:41:03.159038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586381+0000","last_change":"2024-03-23T09:41:03.159217+0000","last_active":"2024-03-23T09:41:04.586381+0000","last_peered":"2024-03-23T09:41:04.586381+0000","last_clean":"2024-03-23T09:41:04.586381+0000","last_became_active":"2024-03-23T09:41:03.157721+0000","last_became_peered":"2024-03-23T09:41:03.157721+0000","last_unstale":"2024-03-23T09:41:04.586381+0000","last_undegraded":"2024-03-23T09:41:04.586381+0000","last_fullsized":"2024-03-23T09:41:04.586381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:28.007092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.145411+0000","last_change":"2024-03-23T09:41:02.785777+0000","last_active":"2024-03-23T09:41:03.145411+0000","last_peered":"2024-03-23T09:41:03.145411+0000","last_clean":"2024-03-23T09:41:03.145411+0000","last_became_active":"2024-03-23T09:41:02.784961+0000","last_became_peered":"2024-03-23T09:41:02.784961+0000","last_unstale":"2024-03-23T09:41:03.145411+0000","last_undegraded":"2024-03-23T09:41:03.145411+0000","last_fullsized":"2024-03-23T09:41:03.145411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142138+0000","last_change":"2024-03-23T09:41:03.146739+0000","last_active":"2024-03-23T09:41:04.142138+0000","last_peered":"2024-03-23T09:41:04.142138+0000","last_clean":"2024-03-23T09:41:04.142138+0000","last_became_active":"2024-03-23T09:41:03.146310+0000","last_became_peered":"2024-03-23T09:41:03.146310+0000","last_unstale":"2024-03-23T09:41:04.142138+0000","last_undegraded":"2024-03-23T09:41:04.142138+0000","last_fullsized":"2024-03-23T09:41:04.142138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:10.603669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.150027+0000","last_change":"2024-03-23T09:41:02.789860+0000","last_active":"2024-03-23T09:41:03.150027+0000","last_peered":"2024-03-23T09:41:03.150027+0000","last_clean":"2024-03-23T09:41:03.150027+0000","last_became_active":"2024-03-23T09:41:02.787110+0000","last_became_peered":"2024-03-23T09:41:02.787110+0000","last_unstale":"2024-03-23T09:41:03.150027+0000","last_undegraded":"2024-03-23T09:41:03.150027+0000","last_fullsized":"2024-03-23T09:41:03.150027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146588+0000","last_change":"2024-03-23T09:41:03.162623+0000","last_active":"2024-03-23T09:41:04.146588+0000","last_peered":"2024-03-23T09:41:04.146588+0000","last_clean":"2024-03-23T09:41:04.146588+0000","last_became_active":"2024-03-23T09:41:03.162237+0000","last_became_peered":"2024-03-23T09:41:03.162237+0000","last_unstale":"2024-03-23T09:41:04.146588+0000","last_undegraded":"2024-03-23T09:41:04.146588+0000","last_fullsized":"2024-03-23T09:41:04.146588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:49.353786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.141570+0000","last_change":"2024-03-23T09:41:02.785671+0000","last_active":"2024-03-23T09:41:03.141570+0000","last_peered":"2024-03-23T09:41:03.141570+0000","last_clean":"2024-03-23T09:41:03.141570+0000","last_became_active":"2024-03-23T09:41:02.785397+0000","last_became_peered":"2024-03-23T09:41:02.785397+0000","last_unstale":"2024-03-23T09:41:03.141570+0000","last_undegraded":"2024-03-23T09:41:03.141570+0000","last_fullsized":"2024-03-23T09:41:03.141570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155275+0000","last_change":"2024-03-23T09:41:03.158343+0000","last_active":"2024-03-23T09:41:04.155275+0000","last_peered":"2024-03-23T09:41:04.155275+0000","last_clean":"2024-03-23T09:41:04.155275+0000","last_became_active":"2024-03-23T09:41:03.158079+0000","last_became_peered":"2024-03-23T09:41:03.158079+0000","last_unstale":"2024-03-23T09:41:04.155275+0000","last_undegraded":"2024-03-23T09:41:04.155275+0000","last_fullsized":"2024-03-23T09:41:04.155275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:46:38.667131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.161554+0000","last_change":"2024-03-23T09:41:02.788728+0000","last_active":"2024-03-23T09:41:03.161554+0000","last_peered":"2024-03-23T09:41:03.161554+0000","last_clean":"2024-03-23T09:41:03.161554+0000","last_became_active":"2024-03-23T09:41:02.788461+0000","last_became_peered":"2024-03-23T09:41:02.788461+0000","last_unstale":"2024-03-23T09:41:03.161554+0000","last_undegraded":"2024-03-23T09:41:03.161554+0000","last_fullsized":"2024-03-23T09:41:03.161554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142267+0000","last_change":"2024-03-23T09:41:03.147531+0000","last_active":"2024-03-23T09:41:04.142267+0000","last_peered":"2024-03-23T09:41:04.142267+0000","last_clean":"2024-03-23T09:41:04.142267+0000","last_became_active":"2024-03-23T09:41:03.147207+0000","last_became_peered":"2024-03-23T09:41:03.147207+0000","last_unstale":"2024-03-23T09:41:04.142267+0000","last_undegraded":"2024-03-23T09:41:04.142267+0000","last_fullsized":"2024-03-23T09:41:04.142267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:42:19.838774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142336+0000","last_change":"2024-03-23T09:41:02.789060+0000","last_active":"2024-03-23T09:41:04.142336+0000","last_peered":"2024-03-23T09:41:04.142336+0000","last_clean":"2024-03-23T09:41:04.142336+0000","last_became_active":"2024-03-23T09:41:02.788772+0000","last_became_peered":"2024-03-23T09:41:02.788772+0000","last_unstale":"2024-03-23T09:41:04.142336+0000","last_undegraded":"2024-03-23T09:41:04.142336+0000","last_fullsized":"2024-03-23T09:41:04.142336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666270+0000","last_change":"2024-03-23T09:41:03.169569+0000","last_active":"2024-03-23T09:41:04.666270+0000","last_peered":"2024-03-23T09:41:04.666270+0000","last_clean":"2024-03-23T09:41:04.666270+0000","last_became_active":"2024-03-23T09:41:03.169275+0000","last_became_peered":"2024-03-23T09:41:03.169275+0000","last_unstale":"2024-03-23T09:41:04.666270+0000","last_undegraded":"2024-03-23T09:41:04.666270+0000","last_fullsized":"2024-03-23T09:41:04.666270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:30:48.469383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.168962+0000","last_change":"2024-03-23T09:41:02.788239+0000","last_active":"2024-03-23T09:41:03.168962+0000","last_peered":"2024-03-23T09:41:03.168962+0000","last_clean":"2024-03-23T09:41:03.168962+0000","last_became_active":"2024-03-23T09:41:02.787874+0000","last_became_peered":"2024-03-23T09:41:02.787874+0000","last_unstale":"2024-03-23T09:41:03.168962+0000","last_undegraded":"2024-03-23T09:41:03.168962+0000","last_fullsized":"2024-03-23T09:41:03.168962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586705+0000","last_change":"2024-03-23T09:41:03.156480+0000","last_active":"2024-03-23T09:41:04.586705+0000","last_peered":"2024-03-23T09:41:04.586705+0000","last_clean":"2024-03-23T09:41:04.586705+0000","last_became_active":"2024-03-23T09:41:03.155683+0000","last_became_peered":"2024-03-23T09:41:03.155683+0000","last_unstale":"2024-03-23T09:41:04.586705+0000","last_undegraded":"2024-03-23T09:41:04.586705+0000","last_fullsized":"2024-03-23T09:41:04.586705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:09.777921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.160846+0000","last_change":"2024-03-23T09:41:02.789413+0000","last_active":"2024-03-23T09:41:03.160846+0000","last_peered":"2024-03-23T09:41:03.160846+0000","last_clean":"2024-03-23T09:41:03.160846+0000","last_became_active":"2024-03-23T09:41:02.788397+0000","last_became_peered":"2024-03-23T09:41:02.788397+0000","last_unstale":"2024-03-23T09:41:03.160846+0000","last_undegraded":"2024-03-23T09:41:03.160846+0000","last_fullsized":"2024-03-23T09:41:03.160846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146940+0000","last_change":"2024-03-23T09:41:03.148447+0000","last_active":"2024-03-23T09:41:04.146940+0000","last_peered":"2024-03-23T09:41:04.146940+0000","last_clean":"2024-03-23T09:41:04.146940+0000","last_became_active":"2024-03-23T09:41:03.148194+0000","last_became_peered":"2024-03-23T09:41:03.148194+0000","last_unstale":"2024-03-23T09:41:04.146940+0000","last_undegraded":"2024-03-23T09:41:04.146940+0000","last_fullsized":"2024-03-23T09:41:04.146940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:21.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.141503+0000","last_change":"2024-03-23T09:41:02.782038+0000","last_active":"2024-03-23T09:41:03.141503+0000","last_peered":"2024-03-23T09:41:03.141503+0000","last_clean":"2024-03-23T09:41:03.141503+0000","last_became_active":"2024-03-23T09:41:02.781322+0000","last_became_peered":"2024-03-23T09:41:02.781322+0000","last_unstale":"2024-03-23T09:41:03.141503+0000","last_undegraded":"2024-03-23T09:41:03.141503+0000","last_fullsized":"2024-03-23T09:41:03.141503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665970+0000","last_change":"2024-03-23T09:41:03.148500+0000","last_active":"2024-03-23T09:41:04.665970+0000","last_peered":"2024-03-23T09:41:04.665970+0000","last_clean":"2024-03-23T09:41:04.665970+0000","last_became_active":"2024-03-23T09:41:03.148263+0000","last_became_peered":"2024-03-23T09:41:03.148263+0000","last_unstale":"2024-03-23T09:41:04.665970+0000","last_undegraded":"2024-03-23T09:41:04.665970+0000","last_fullsized":"2024-03-23T09:41:04.665970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:51:26.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.141242+0000","last_change":"2024-03-23T09:41:02.785162+0000","last_active":"2024-03-23T09:41:03.141242+0000","last_peered":"2024-03-23T09:41:03.141242+0000","last_clean":"2024-03-23T09:41:03.141242+0000","last_became_active":"2024-03-23T09:41:02.784744+0000","last_became_peered":"2024-03-23T09:41:02.784744+0000","last_unstale":"2024-03-23T09:41:03.141242+0000","last_undegraded":"2024-03-23T09:41:03.141242+0000","last_fullsized":"2024-03-23T09:41:03.141242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666237+0000","last_change":"2024-03-23T09:41:03.160780+0000","last_active":"2024-03-23T09:41:04.666237+0000","last_peered":"2024-03-23T09:41:04.666237+0000","last_clean":"2024-03-23T09:41:04.666237+0000","last_became_active":"2024-03-23T09:41:03.159686+0000","last_became_peered":"2024-03-23T09:41:03.159686+0000","last_unstale":"2024-03-23T09:41:04.666237+0000","last_undegraded":"2024-03-23T09:41:04.666237+0000","last_fullsized":"2024-03-23T09:41:04.666237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:16.974803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.170285+0000","last_change":"2024-03-23T09:41:02.786708+0000","last_active":"2024-03-23T09:41:03.170285+0000","last_peered":"2024-03-23T09:41:03.170285+0000","last_clean":"2024-03-23T09:41:03.170285+0000","last_became_active":"2024-03-23T09:41:02.786306+0000","last_became_peered":"2024-03-23T09:41:02.786306+0000","last_unstale":"2024-03-23T09:41:03.170285+0000","last_undegraded":"2024-03-23T09:41:03.170285+0000","last_fullsized":"2024-03-23T09:41:03.170285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666270+0000","last_change":"2024-03-23T09:41:03.157669+0000","last_active":"2024-03-23T09:41:04.666270+0000","last_peered":"2024-03-23T09:41:04.666270+0000","last_clean":"2024-03-23T09:41:04.666270+0000","last_became_active":"2024-03-23T09:41:03.157206+0000","last_became_peered":"2024-03-23T09:41:03.157206+0000","last_unstale":"2024-03-23T09:41:04.666270+0000","last_undegraded":"2024-03-23T09:41:04.666270+0000","last_fullsized":"2024-03-23T09:41:04.666270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:53:01.847138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155328+0000","last_change":"2024-03-23T09:41:03.158925+0000","last_active":"2024-03-23T09:41:04.155328+0000","last_peered":"2024-03-23T09:41:04.155328+0000","last_clean":"2024-03-23T09:41:04.155328+0000","last_became_active":"2024-03-23T09:41:03.158643+0000","last_became_peered":"2024-03-23T09:41:03.158643+0000","last_unstale":"2024-03-23T09:41:04.155328+0000","last_undegraded":"2024-03-23T09:41:04.155328+0000","last_fullsized":"2024-03-23T09:41:04.155328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:32:17.579489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.170957+0000","last_change":"2024-03-23T09:41:02.786654+0000","last_active":"2024-03-23T09:41:03.170957+0000","last_peered":"2024-03-23T09:41:03.170957+0000","last_clean":"2024-03-23T09:41:03.170957+0000","last_became_active":"2024-03-23T09:41:02.786387+0000","last_became_peered":"2024-03-23T09:41:02.786387+0000","last_unstale":"2024-03-23T09:41:03.170957+0000","last_undegraded":"2024-03-23T09:41:03.170957+0000","last_fullsized":"2024-03-23T09:41:03.170957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155110+0000","last_change":"2024-03-23T09:41:03.158846+0000","last_active":"2024-03-23T09:41:04.155110+0000","last_peered":"2024-03-23T09:41:04.155110+0000","last_clean":"2024-03-23T09:41:04.155110+0000","last_became_active":"2024-03-23T09:41:03.158433+0000","last_became_peered":"2024-03-23T09:41:03.158433+0000","last_unstale":"2024-03-23T09:41:04.155110+0000","last_undegraded":"2024-03-23T09:41:04.155110+0000","last_fullsized":"2024-03-23T09:41:04.155110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:11:48.328619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.168704+0000","last_change":"2024-03-23T09:41:02.788509+0000","last_active":"2024-03-23T09:41:03.168704+0000","last_peered":"2024-03-23T09:41:03.168704+0000","last_clean":"2024-03-23T09:41:03.168704+0000","last_became_active":"2024-03-23T09:41:02.788072+0000","last_became_peered":"2024-03-23T09:41:02.788072+0000","last_unstale":"2024-03-23T09:41:03.168704+0000","last_undegraded":"2024-03-23T09:41:03.168704+0000","last_fullsized":"2024-03-23T09:41:03.168704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:11.278420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586632+0000","last_change":"2024-03-23T09:41:03.159417+0000","last_active":"2024-03-23T09:41:04.586632+0000","last_peered":"2024-03-23T09:41:04.586632+0000","last_clean":"2024-03-23T09:41:04.586632+0000","last_became_active":"2024-03-23T09:41:03.159146+0000","last_became_peered":"2024-03-23T09:41:03.159146+0000","last_unstale":"2024-03-23T09:41:04.586632+0000","last_undegraded":"2024-03-23T09:41:04.586632+0000","last_fullsized":"2024-03-23T09:41:04.586632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:10.603669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.163045+0000","last_change":"2024-03-23T09:41:02.789237+0000","last_active":"2024-03-23T09:41:03.163045+0000","last_peered":"2024-03-23T09:41:03.163045+0000","last_clean":"2024-03-23T09:41:03.163045+0000","last_became_active":"2024-03-23T09:41:02.788896+0000","last_became_peered":"2024-03-23T09:41:02.788896+0000","last_unstale":"2024-03-23T09:41:03.163045+0000","last_undegraded":"2024-03-23T09:41:03.163045+0000","last_fullsized":"2024-03-23T09:41:03.163045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666186+0000","last_change":"2024-03-23T09:41:03.163326+0000","last_active":"2024-03-23T09:41:04.666186+0000","last_peered":"2024-03-23T09:41:04.666186+0000","last_clean":"2024-03-23T09:41:04.666186+0000","last_became_active":"2024-03-23T09:41:03.163069+0000","last_became_peered":"2024-03-23T09:41:03.163069+0000","last_unstale":"2024-03-23T09:41:04.666186+0000","last_undegraded":"2024-03-23T09:41:04.666186+0000","last_fullsized":"2024-03-23T09:41:04.666186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.171686+0000","last_change":"2024-03-23T09:41:02.787414+0000","last_active":"2024-03-23T09:41:03.171686+0000","last_peered":"2024-03-23T09:41:03.171686+0000","last_clean":"2024-03-23T09:41:03.171686+0000","last_became_active":"2024-03-23T09:41:02.787074+0000","last_became_peered":"2024-03-23T09:41:02.787074+0000","last_unstale":"2024-03-23T09:41:03.171686+0000","last_undegraded":"2024-03-23T09:41:03.171686+0000","last_fullsized":"2024-03-23T09:41:03.171686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:02:19.284328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28392,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343448,"statfs":{"total":96636764160,"available":96607690752,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":76514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":819200,"data_stored":665956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:41:13.267 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:41:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.404+0000 7fb34af00640 1 Processor -- start 2024-03-23T09:41:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.404+0000 7fb34af00640 1 -- start start 2024-03-23T09:41:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.405+0000 7fb34af00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3440cec50 0x7fb3440cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.405+0000 7fb34af00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3440cc610 con 0x7fb34415a8d0 2024-03-23T09:41:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.405+0000 7fb34af00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3440b0700 con 0x7fb3440b0b90 2024-03-23T09:41:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.405+0000 7fb34af00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c41c0 con 0x7fb3440cec50 2024-03-23T09:41:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.405+0000 7fb348c9c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb34415a8d0 0x7fb3440cb1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57354/0 (socket says 172.21.15.47:57354) 2024-03-23T09:41:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.405+0000 7fb348c9c640 1 -- 172.21.15.47:0/4234629849 learned_addr learned my addr 172.21.15.47:0/4234629849 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.405+0000 7fb343fff640 1 --2- 172.21.15.47:0/4234629849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3440cec50 0x7fb3440cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.405+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59564800 0 0) 0x7fb3440cc610 con 0x7fb34415a8d0 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.405+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb32c003520 con 0x7fb34415a8d0 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700388020 0 0) 0x7fb3440b0700 con 0x7fb3440b0b90 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3440cc610 con 0x7fb3440b0b90 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4045258121 0 0) 0x7fb32c003520 con 0x7fb34415a8d0 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3440cec50 msgr2=0x7fb3440cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 --2- 172.21.15.47:0/4234629849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3440cec50 0x7fb3440cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 >> v1:172.21.15.175:6789/0 conn(0x7fb3440b0b90 legacy=0x7fb3440b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440cbd70 con 0x7fb34415a8d0 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fb33c001970 con 0x7fb34415a8d0 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 >> v1:172.21.15.47:6789/0 conn(0x7fb34415a8d0 legacy=0x7fb3440cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 --2- 172.21.15.47:0/4234629849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb32c003b30 0x7fb32c003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 --2- 172.21.15.47:0/4234629849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3440cec50 0x7fb32c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 --2- 172.21.15.47:0/4234629849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb32c004d00 0x7fb32c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c00bb10 con 0x7fb32c003b30 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c00bc50 con 0x7fb32c004d00 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c00bde0 con 0x7fb3440cec50 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb348c9c640 1 --2- 172.21.15.47:0/4234629849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb32c003b30 0x7fb32c003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb343fff640 1 --2- 172.21.15.47:0/4234629849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3440cec50 0x7fb32c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb34949d640 1 --2- 172.21.15.47:0/4234629849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb32c004d00 0x7fb32c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb348c9c640 1 -- 172.21.15.47:0/4234629849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3440cec50 msgr2=0x7fb32c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb348c9c640 1 --2- 172.21.15.47:0/4234629849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3440cec50 0x7fb32c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb348c9c640 1 -- 172.21.15.47:0/4234629849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb32c004d00 msgr2=0x7fb32c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb348c9c640 1 --2- 172.21.15.47:0/4234629849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb32c004d00 0x7fb32c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.406+0000 7fb348c9c640 1 -- 172.21.15.47:0/4234629849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c003520 con 0x7fb32c003b30 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.407+0000 7fb348c9c640 1 --2- 172.21.15.47:0/4234629849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb32c003b30 0x7fb32c003f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00a6e0 tx=0x7fb32c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429f27a033499f2f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.407+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c007330 con 0x7fb32c003b30 2024-03-23T09:41:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.407+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c02f070 con 0x7fb32c003b30 2024-03-23T09:41:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.407+0000 7fb3437fe640 1 -- 172.21.15.47:0/4234629849 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c007730 con 0x7fb32c003b30 2024-03-23T09:41:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.408+0000 7fb34af00640 1 -- 172.21.15.47:0/4234629849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb32c003b30 msgr2=0x7fb32c003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.408+0000 7fb34af00640 1 --2- 172.21.15.47:0/4234629849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb32c003b30 0x7fb32c003f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00a6e0 tx=0x7fb32c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.408+0000 7fb34949d640 1 -- 172.21.15.47:0/4234629849 reap_dead start 2024-03-23T09:41:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.408+0000 7fb34af00640 1 -- 172.21.15.47:0/4234629849 shutdown_connections 2024-03-23T09:41:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.408+0000 7fb34af00640 1 -- 172.21.15.47:0/4234629849 >> 172.21.15.47:0/4234629849 conn(0x7fb3440d0670 msgr2=0x7fb3440d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.408+0000 7fb34af00640 1 -- 172.21.15.47:0/4234629849 shutdown_connections 2024-03-23T09:41:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.408+0000 7fb34af00640 1 -- 172.21.15.47:0/4234629849 wait complete. 2024-03-23T09:41:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.409+0000 7fb34af00640 1 Processor -- start 2024-03-23T09:41:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.409+0000 7fb34af00640 1 -- start start 2024-03-23T09:41:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb34af00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb32c003b30 0x7fb34415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb34af00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3440c4160 0x7fb344146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb34af00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb344146a80 0x7fb3441ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb34af00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb344147480 con 0x7fb3440c4160 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb34af00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb344146f30 con 0x7fb344146a80 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb34af00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb344155ee0 con 0x7fb32c003b30 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb343fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3440c4160 0x7fb344146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb343fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3440c4160 0x7fb344146540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46298/0 (socket says 172.21.15.47:46298) 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb343fff640 1 -- 172.21.15.47:0/254887758 learned_addr learned my addr 172.21.15.47:0/254887758 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb348c9c640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb32c003b30 0x7fb34415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb34949d640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb344146a80 0x7fb3441ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb343fff640 1 -- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb32c003b30 msgr2=0x7fb34415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb343fff640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb32c003b30 0x7fb34415b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb343fff640 1 -- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb344146a80 msgr2=0x7fb3441ef8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb343fff640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb344146a80 0x7fb3441ef8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb343fff640 1 -- 172.21.15.47:0/254887758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440c48d0 con 0x7fb3440c4160 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb343fff640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3440c4160 0x7fb344146540 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb3380119d0 tx=0x7fb338011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb3417fa640 1 -- 172.21.15.47:0/254887758 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800fb50 con 0x7fb3440c4160 2024-03-23T09:41:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb3417fa640 1 -- 172.21.15.47:0/254887758 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33800fcb0 con 0x7fb3440c4160 2024-03-23T09:41:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb34af00640 1 -- 172.21.15.47:0/254887758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3441558d0 con 0x7fb3440c4160 2024-03-23T09:41:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb3417fa640 1 -- 172.21.15.47:0/254887758 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338014440 con 0x7fb3440c4160 2024-03-23T09:41:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.410+0000 7fb34af00640 1 -- 172.21.15.47:0/254887758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb344155b70 con 0x7fb3440c4160 2024-03-23T09:41:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.411+0000 7fb34af00640 1 -- 172.21.15.47:0/254887758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34415b5d0 con 0x7fb3440c4160 2024-03-23T09:41:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.412+0000 7fb3417fa640 1 -- 172.21.15.47:0/254887758 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb3380145a0 con 0x7fb3440c4160 2024-03-23T09:41:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.412+0000 7fb3417fa640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb33006ac00 0x7fb33006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.412+0000 7fb3417fa640 1 -- 172.21.15.47:0/254887758 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb338089b00 con 0x7fb3440c4160 2024-03-23T09:41:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.412+0000 7fb348c9c640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb33006ac00 0x7fb33006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.413+0000 7fb348c9c640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb33006ac00 0x7fb33006d0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00a6b0 tx=0x7fb33c02f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.416+0000 7fb3417fa640 1 -- 172.21.15.47:0/254887758 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb338054d90 con 0x7fb3440c4160 2024-03-23T09:41:13.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.591+0000 7fb34af00640 1 -- 172.21.15.47:0/254887758 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3440ce670 con 0x7fb33006ac00 2024-03-23T09:41:13.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.598+0000 7fb3417fa640 1 -- 172.21.15.47:0/254887758 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264564 (secure 0 0 0) 0x7fb33c001870 con 0x7fb33006ac00 2024-03-23T09:41:13.600 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:13.601 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:41:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.600+0000 7fb34af00640 1 -- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb33006ac00 msgr2=0x7fb33006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.600+0000 7fb34af00640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb33006ac00 0x7fb33006d0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00a6b0 tx=0x7fb33c02f000 comp rx=0 tx=0).stop 2024-03-23T09:41:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.600+0000 7fb34af00640 1 -- 172.21.15.47:0/254887758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3440c4160 msgr2=0x7fb344146540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.600+0000 7fb34af00640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3440c4160 0x7fb344146540 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb3380119d0 tx=0x7fb338011e90 comp rx=0 tx=0).stop 2024-03-23T09:41:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.601+0000 7fb34af00640 1 -- 172.21.15.47:0/254887758 shutdown_connections 2024-03-23T09:41:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.601+0000 7fb34af00640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb33006ac00 0x7fb33006d0c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.601+0000 7fb34af00640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb344146a80 0x7fb3441ef8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.601+0000 7fb34af00640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3440c4160 0x7fb344146540 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.601+0000 7fb34af00640 1 --2- 172.21.15.47:0/254887758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb32c003b30 0x7fb34415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.601+0000 7fb34af00640 1 -- 172.21.15.47:0/254887758 >> 172.21.15.47:0/254887758 conn(0x7fb3440d0670 msgr2=0x7fb344157580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:13.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.601+0000 7fb34af00640 1 -- 172.21.15.47:0/254887758 shutdown_connections 2024-03-23T09:41:13.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.601+0000 7fb34af00640 1 -- 172.21.15.47:0/254887758 wait complete. 2024-03-23T09:41:13.611 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-23T09:41:12.339464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236068,"kb_used_data":2912,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738652,"statfs":{"total":773094113280,"available":772852379648,"internally_reserved":0,"allocated":2981888,"data_stored":1788774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.196352"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.147121+0000","last_change":"2024-03-23T09:41:03.164352+0000","last_active":"2024-03-23T09:41:04.147121+0000","last_peered":"2024-03-23T09:41:04.147121+0000","last_clean":"2024-03-23T09:41:04.147121+0000","last_became_active":"2024-03-23T09:41:03.164058+0000","last_became_peered":"2024-03-23T09:41:03.164058+0000","last_unstale":"2024-03-23T09:41:04.147121+0000","last_undegraded":"2024-03-23T09:41:04.147121+0000","last_fullsized":"2024-03-23T09:41:04.147121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:54:59.125701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142356+0000","last_change":"2024-03-23T09:41:03.159612+0000","last_active":"2024-03-23T09:41:04.142356+0000","last_peered":"2024-03-23T09:41:04.142356+0000","last_clean":"2024-03-23T09:41:04.142356+0000","last_became_active":"2024-03-23T09:41:03.159341+0000","last_became_peered":"2024-03-23T09:41:03.159341+0000","last_unstale":"2024-03-23T09:41:04.142356+0000","last_undegraded":"2024-03-23T09:41:04.142356+0000","last_fullsized":"2024-03-23T09:41:04.142356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586632+0000","last_change":"2024-03-23T09:41:03.156325+0000","last_active":"2024-03-23T09:41:04.586632+0000","last_peered":"2024-03-23T09:41:04.586632+0000","last_clean":"2024-03-23T09:41:04.586632+0000","last_became_active":"2024-03-23T09:41:03.156061+0000","last_became_peered":"2024-03-23T09:41:03.156061+0000","last_unstale":"2024-03-23T09:41:04.586632+0000","last_undegraded":"2024-03-23T09:41:04.586632+0000","last_fullsized":"2024-03-23T09:41:04.586632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:06:51.567637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155051+0000","last_change":"2024-03-23T09:41:03.158679+0000","last_active":"2024-03-23T09:41:04.155051+0000","last_peered":"2024-03-23T09:41:04.155051+0000","last_clean":"2024-03-23T09:41:04.155051+0000","last_became_active":"2024-03-23T09:41:03.158151+0000","last_became_peered":"2024-03-23T09:41:03.158151+0000","last_unstale":"2024-03-23T09:41:04.155051+0000","last_undegraded":"2024-03-23T09:41:04.155051+0000","last_fullsized":"2024-03-23T09:41:04.155051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:29.855586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146283+0000","last_change":"2024-03-23T09:41:03.167599+0000","last_active":"2024-03-23T09:41:04.146283+0000","last_peered":"2024-03-23T09:41:04.146283+0000","last_clean":"2024-03-23T09:41:04.146283+0000","last_became_active":"2024-03-23T09:41:03.167332+0000","last_became_peered":"2024-03-23T09:41:03.167332+0000","last_unstale":"2024-03-23T09:41:04.146283+0000","last_undegraded":"2024-03-23T09:41:04.146283+0000","last_fullsized":"2024-03-23T09:41:04.146283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:32:17.579489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.154914+0000","last_change":"2024-03-23T09:41:03.162714+0000","last_active":"2024-03-23T09:41:04.154914+0000","last_peered":"2024-03-23T09:41:04.154914+0000","last_clean":"2024-03-23T09:41:04.154914+0000","last_became_active":"2024-03-23T09:41:03.162461+0000","last_became_peered":"2024-03-23T09:41:03.162461+0000","last_unstale":"2024-03-23T09:41:04.154914+0000","last_undegraded":"2024-03-23T09:41:04.154914+0000","last_fullsized":"2024-03-23T09:41:04.154914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:30:48.469383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146273+0000","last_change":"2024-03-23T09:41:03.167591+0000","last_active":"2024-03-23T09:41:04.146273+0000","last_peered":"2024-03-23T09:41:04.146273+0000","last_clean":"2024-03-23T09:41:04.146273+0000","last_became_active":"2024-03-23T09:41:03.167320+0000","last_became_peered":"2024-03-23T09:41:03.167320+0000","last_unstale":"2024-03-23T09:41:04.146273+0000","last_undegraded":"2024-03-23T09:41:04.146273+0000","last_fullsized":"2024-03-23T09:41:04.146273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:29.855586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665874+0000","last_change":"2024-03-23T09:41:03.153495+0000","last_active":"2024-03-23T09:41:04.665874+0000","last_peered":"2024-03-23T09:41:04.665874+0000","last_clean":"2024-03-23T09:41:04.665874+0000","last_became_active":"2024-03-23T09:41:03.153257+0000","last_became_peered":"2024-03-23T09:41:03.153257+0000","last_unstale":"2024-03-23T09:41:04.665874+0000","last_undegraded":"2024-03-23T09:41:04.665874+0000","last_fullsized":"2024-03-23T09:41:04.665874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:46:38.667131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.147216+0000","last_change":"2024-03-23T09:41:03.170688+0000","last_active":"2024-03-23T09:41:04.147216+0000","last_peered":"2024-03-23T09:41:04.147216+0000","last_clean":"2024-03-23T09:41:04.147216+0000","last_became_active":"2024-03-23T09:41:03.170446+0000","last_became_peered":"2024-03-23T09:41:03.170446+0000","last_unstale":"2024-03-23T09:41:04.147216+0000","last_undegraded":"2024-03-23T09:41:04.147216+0000","last_fullsized":"2024-03-23T09:41:04.147216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:22:15.797821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666694+0000","last_change":"2024-03-23T09:41:03.162169+0000","last_active":"2024-03-23T09:41:04.666694+0000","last_peered":"2024-03-23T09:41:04.666694+0000","last_clean":"2024-03-23T09:41:04.666694+0000","last_became_active":"2024-03-23T09:41:03.161587+0000","last_became_peered":"2024-03-23T09:41:03.161587+0000","last_unstale":"2024-03-23T09:41:04.666694+0000","last_undegraded":"2024-03-23T09:41:04.666694+0000","last_fullsized":"2024-03-23T09:41:04.666694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666339+0000","last_change":"2024-03-23T09:41:03.160438+0000","last_active":"2024-03-23T09:41:04.666339+0000","last_peered":"2024-03-23T09:41:04.666339+0000","last_clean":"2024-03-23T09:41:04.666339+0000","last_became_active":"2024-03-23T09:41:03.160175+0000","last_became_peered":"2024-03-23T09:41:03.160175+0000","last_unstale":"2024-03-23T09:41:04.666339+0000","last_undegraded":"2024-03-23T09:41:04.666339+0000","last_fullsized":"2024-03-23T09:41:04.666339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146726+0000","last_change":"2024-03-23T09:41:03.163213+0000","last_active":"2024-03-23T09:41:04.146726+0000","last_peered":"2024-03-23T09:41:04.146726+0000","last_clean":"2024-03-23T09:41:04.146726+0000","last_became_active":"2024-03-23T09:41:03.162954+0000","last_became_peered":"2024-03-23T09:41:03.162954+0000","last_unstale":"2024-03-23T09:41:04.146726+0000","last_undegraded":"2024-03-23T09:41:04.146726+0000","last_fullsized":"2024-03-23T09:41:04.146726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:28.007092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666209+0000","last_change":"2024-03-23T09:41:03.161818+0000","last_active":"2024-03-23T09:41:04.666209+0000","last_peered":"2024-03-23T09:41:04.666209+0000","last_clean":"2024-03-23T09:41:04.666209+0000","last_became_active":"2024-03-23T09:41:03.159384+0000","last_became_peered":"2024-03-23T09:41:03.159384+0000","last_unstale":"2024-03-23T09:41:04.666209+0000","last_undegraded":"2024-03-23T09:41:04.666209+0000","last_fullsized":"2024-03-23T09:41:04.666209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586410+0000","last_change":"2024-03-23T09:41:03.159776+0000","last_active":"2024-03-23T09:41:04.586410+0000","last_peered":"2024-03-23T09:41:04.586410+0000","last_clean":"2024-03-23T09:41:04.586410+0000","last_became_active":"2024-03-23T09:41:03.159466+0000","last_became_peered":"2024-03-23T09:41:03.159466+0000","last_unstale":"2024-03-23T09:41:04.586410+0000","last_undegraded":"2024-03-23T09:41:04.586410+0000","last_fullsized":"2024-03-23T09:41:04.586410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142102+0000","last_change":"2024-03-23T09:41:03.154187+0000","last_active":"2024-03-23T09:41:04.142102+0000","last_peered":"2024-03-23T09:41:04.142102+0000","last_clean":"2024-03-23T09:41:04.142102+0000","last_became_active":"2024-03-23T09:41:03.153946+0000","last_became_peered":"2024-03-23T09:41:03.153946+0000","last_unstale":"2024-03-23T09:41:04.142102+0000","last_undegraded":"2024-03-23T09:41:04.142102+0000","last_fullsized":"2024-03-23T09:41:04.142102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.154801+0000","last_change":"2024-03-23T09:41:03.160290+0000","last_active":"2024-03-23T09:41:04.154801+0000","last_peered":"2024-03-23T09:41:04.154801+0000","last_clean":"2024-03-23T09:41:04.154801+0000","last_became_active":"2024-03-23T09:41:03.160010+0000","last_became_peered":"2024-03-23T09:41:03.160010+0000","last_unstale":"2024-03-23T09:41:04.154801+0000","last_undegraded":"2024-03-23T09:41:04.154801+0000","last_fullsized":"2024-03-23T09:41:04.154801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155450+0000","last_change":"2024-03-23T09:41:03.165189+0000","last_active":"2024-03-23T09:41:04.155450+0000","last_peered":"2024-03-23T09:41:04.155450+0000","last_clean":"2024-03-23T09:41:04.155450+0000","last_became_active":"2024-03-23T09:41:03.164956+0000","last_became_peered":"2024-03-23T09:41:03.164956+0000","last_unstale":"2024-03-23T09:41:04.155450+0000","last_undegraded":"2024-03-23T09:41:04.155450+0000","last_fullsized":"2024-03-23T09:41:04.155450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665970+0000","last_change":"2024-03-23T09:41:03.163535+0000","last_active":"2024-03-23T09:41:04.665970+0000","last_peered":"2024-03-23T09:41:04.665970+0000","last_clean":"2024-03-23T09:41:04.665970+0000","last_became_active":"2024-03-23T09:41:03.163271+0000","last_became_peered":"2024-03-23T09:41:03.163271+0000","last_unstale":"2024-03-23T09:41:04.665970+0000","last_undegraded":"2024-03-23T09:41:04.665970+0000","last_fullsized":"2024-03-23T09:41:04.665970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:12.280151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146986+0000","last_change":"2024-03-23T09:41:03.168829+0000","last_active":"2024-03-23T09:41:04.146986+0000","last_peered":"2024-03-23T09:41:04.146986+0000","last_clean":"2024-03-23T09:41:04.146986+0000","last_became_active":"2024-03-23T09:41:03.167734+0000","last_became_peered":"2024-03-23T09:41:03.167734+0000","last_unstale":"2024-03-23T09:41:04.146986+0000","last_undegraded":"2024-03-23T09:41:04.146986+0000","last_fullsized":"2024-03-23T09:41:04.146986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:11:48.328619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666441+0000","last_change":"2024-03-23T09:41:03.161820+0000","last_active":"2024-03-23T09:41:04.666441+0000","last_peered":"2024-03-23T09:41:04.666441+0000","last_clean":"2024-03-23T09:41:04.666441+0000","last_became_active":"2024-03-23T09:41:03.161310+0000","last_became_peered":"2024-03-23T09:41:03.161310+0000","last_unstale":"2024-03-23T09:41:04.666441+0000","last_undegraded":"2024-03-23T09:41:04.666441+0000","last_fullsized":"2024-03-23T09:41:04.666441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:42:19.838774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586549+0000","last_change":"2024-03-23T09:41:03.171444+0000","last_active":"2024-03-23T09:41:04.586549+0000","last_peered":"2024-03-23T09:41:04.586549+0000","last_clean":"2024-03-23T09:41:04.586549+0000","last_became_active":"2024-03-23T09:41:03.171114+0000","last_became_peered":"2024-03-23T09:41:03.171114+0000","last_unstale":"2024-03-23T09:41:04.586549+0000","last_undegraded":"2024-03-23T09:41:04.586549+0000","last_fullsized":"2024-03-23T09:41:04.586549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.154962+0000","last_change":"2024-03-23T09:41:03.166192+0000","last_active":"2024-03-23T09:41:04.154962+0000","last_peered":"2024-03-23T09:41:04.154962+0000","last_clean":"2024-03-23T09:41:04.154962+0000","last_became_active":"2024-03-23T09:41:03.165944+0000","last_became_peered":"2024-03-23T09:41:03.165944+0000","last_unstale":"2024-03-23T09:41:04.154962+0000","last_undegraded":"2024-03-23T09:41:04.154962+0000","last_fullsized":"2024-03-23T09:41:04.154962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:51:26.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666043+0000","last_change":"2024-03-23T09:41:03.149122+0000","last_active":"2024-03-23T09:41:04.666043+0000","last_peered":"2024-03-23T09:41:04.666043+0000","last_clean":"2024-03-23T09:41:04.666043+0000","last_became_active":"2024-03-23T09:41:03.148885+0000","last_became_peered":"2024-03-23T09:41:03.148885+0000","last_unstale":"2024-03-23T09:41:04.666043+0000","last_undegraded":"2024-03-23T09:41:04.666043+0000","last_fullsized":"2024-03-23T09:41:04.666043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:06.444696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.154922+0000","last_change":"2024-03-23T09:41:03.155719+0000","last_active":"2024-03-23T09:41:04.154922+0000","last_peered":"2024-03-23T09:41:04.154922+0000","last_clean":"2024-03-23T09:41:04.154922+0000","last_became_active":"2024-03-23T09:41:03.155436+0000","last_became_peered":"2024-03-23T09:41:03.155436+0000","last_unstale":"2024-03-23T09:41:04.154922+0000","last_undegraded":"2024-03-23T09:41:04.154922+0000","last_fullsized":"2024-03-23T09:41:04.154922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666163+0000","last_change":"2024-03-23T09:41:03.148314+0000","last_active":"2024-03-23T09:41:04.666163+0000","last_peered":"2024-03-23T09:41:04.666163+0000","last_clean":"2024-03-23T09:41:04.666163+0000","last_became_active":"2024-03-23T09:41:03.148048+0000","last_became_peered":"2024-03-23T09:41:03.148048+0000","last_unstale":"2024-03-23T09:41:04.666163+0000","last_undegraded":"2024-03-23T09:41:04.666163+0000","last_fullsized":"2024-03-23T09:41:04.666163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:09.777921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666670+0000","last_change":"2024-03-23T09:41:03.169146+0000","last_active":"2024-03-23T09:41:04.666670+0000","last_peered":"2024-03-23T09:41:04.666670+0000","last_clean":"2024-03-23T09:41:04.666670+0000","last_became_active":"2024-03-23T09:41:03.168877+0000","last_became_peered":"2024-03-23T09:41:03.168877+0000","last_unstale":"2024-03-23T09:41:04.666670+0000","last_undegraded":"2024-03-23T09:41:04.666670+0000","last_fullsized":"2024-03-23T09:41:04.666670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:10.603669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142270+0000","last_change":"2024-03-23T09:41:03.147239+0000","last_active":"2024-03-23T09:41:04.142270+0000","last_peered":"2024-03-23T09:41:04.142270+0000","last_clean":"2024-03-23T09:41:04.142270+0000","last_became_active":"2024-03-23T09:41:03.146990+0000","last_became_peered":"2024-03-23T09:41:03.146990+0000","last_unstale":"2024-03-23T09:41:04.142270+0000","last_undegraded":"2024-03-23T09:41:04.142270+0000","last_fullsized":"2024-03-23T09:41:04.142270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666449+0000","last_change":"2024-03-23T09:41:03.149202+0000","last_active":"2024-03-23T09:41:04.666449+0000","last_peered":"2024-03-23T09:41:04.666449+0000","last_clean":"2024-03-23T09:41:04.666449+0000","last_became_active":"2024-03-23T09:41:03.148958+0000","last_became_peered":"2024-03-23T09:41:03.148958+0000","last_unstale":"2024-03-23T09:41:04.666449+0000","last_undegraded":"2024-03-23T09:41:04.666449+0000","last_fullsized":"2024-03-23T09:41:04.666449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:10.603669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586617+0000","last_change":"2024-03-23T09:41:03.157714+0000","last_active":"2024-03-23T09:41:04.586617+0000","last_peered":"2024-03-23T09:41:04.586617+0000","last_clean":"2024-03-23T09:41:04.586617+0000","last_became_active":"2024-03-23T09:41:03.157315+0000","last_became_peered":"2024-03-23T09:41:03.157315+0000","last_unstale":"2024-03-23T09:41:04.586617+0000","last_undegraded":"2024-03-23T09:41:04.586617+0000","last_fullsized":"2024-03-23T09:41:04.586617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:06.444696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155043+0000","last_change":"2024-03-23T09:41:03.155373+0000","last_active":"2024-03-23T09:41:04.155043+0000","last_peered":"2024-03-23T09:41:04.155043+0000","last_clean":"2024-03-23T09:41:04.155043+0000","last_became_active":"2024-03-23T09:41:03.155063+0000","last_became_peered":"2024-03-23T09:41:03.155063+0000","last_unstale":"2024-03-23T09:41:04.155043+0000","last_undegraded":"2024-03-23T09:41:04.155043+0000","last_fullsized":"2024-03-23T09:41:04.155043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:33:41.605578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665826+0000","last_change":"2024-03-23T09:41:03.154532+0000","last_active":"2024-03-23T09:41:04.665826+0000","last_peered":"2024-03-23T09:41:04.665826+0000","last_clean":"2024-03-23T09:41:04.665826+0000","last_became_active":"2024-03-23T09:41:03.154277+0000","last_became_peered":"2024-03-23T09:41:03.154277+0000","last_unstale":"2024-03-23T09:41:04.665826+0000","last_undegraded":"2024-03-23T09:41:04.665826+0000","last_fullsized":"2024-03-23T09:41:04.665826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:49.353786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666387+0000","last_change":"2024-03-23T09:41:03.166933+0000","last_active":"2024-03-23T09:41:04.666387+0000","last_peered":"2024-03-23T09:41:04.666387+0000","last_clean":"2024-03-23T09:41:04.666387+0000","last_became_active":"2024-03-23T09:41:03.166359+0000","last_became_peered":"2024-03-23T09:41:03.166359+0000","last_unstale":"2024-03-23T09:41:04.666387+0000","last_undegraded":"2024-03-23T09:41:04.666387+0000","last_fullsized":"2024-03-23T09:41:04.666387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:09.777921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146129+0000","last_change":"2024-03-23T09:41:03.149144+0000","last_active":"2024-03-23T09:41:04.146129+0000","last_peered":"2024-03-23T09:41:04.146129+0000","last_clean":"2024-03-23T09:41:04.146129+0000","last_became_active":"2024-03-23T09:41:03.148842+0000","last_became_peered":"2024-03-23T09:41:03.148842+0000","last_unstale":"2024-03-23T09:41:04.146129+0000","last_undegraded":"2024-03-23T09:41:04.146129+0000","last_fullsized":"2024-03-23T09:41:04.146129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:12.280151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.167252+0000","last_change":"2024-03-23T09:41:02.789724+0000","last_active":"2024-03-23T09:41:03.167252+0000","last_peered":"2024-03-23T09:41:03.167252+0000","last_clean":"2024-03-23T09:41:03.167252+0000","last_became_active":"2024-03-23T09:41:02.789485+0000","last_became_peered":"2024-03-23T09:41:02.789485+0000","last_unstale":"2024-03-23T09:41:03.167252+0000","last_undegraded":"2024-03-23T09:41:03.167252+0000","last_fullsized":"2024-03-23T09:41:03.167252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586658+0000","last_change":"2024-03-23T09:41:03.157560+0000","last_active":"2024-03-23T09:41:04.586658+0000","last_peered":"2024-03-23T09:41:04.586658+0000","last_clean":"2024-03-23T09:41:04.586658+0000","last_became_active":"2024-03-23T09:41:03.157046+0000","last_became_peered":"2024-03-23T09:41:03.157046+0000","last_unstale":"2024-03-23T09:41:04.586658+0000","last_undegraded":"2024-03-23T09:41:04.586658+0000","last_fullsized":"2024-03-23T09:41:04.586658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:04:04.702736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.161341+0000","last_change":"2024-03-23T09:41:02.785765+0000","last_active":"2024-03-23T09:41:03.161341+0000","last_peered":"2024-03-23T09:41:03.161341+0000","last_clean":"2024-03-23T09:41:03.161341+0000","last_became_active":"2024-03-23T09:41:02.785483+0000","last_became_peered":"2024-03-23T09:41:02.785483+0000","last_unstale":"2024-03-23T09:41:03.161341+0000","last_undegraded":"2024-03-23T09:41:03.161341+0000","last_fullsized":"2024-03-23T09:41:03.161341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.145332+0000","last_change":"2024-03-23T09:41:02.790132+0000","last_active":"2024-03-23T09:41:03.145332+0000","last_peered":"2024-03-23T09:41:03.145332+0000","last_clean":"2024-03-23T09:41:03.145332+0000","last_became_active":"2024-03-23T09:41:02.789893+0000","last_became_peered":"2024-03-23T09:41:02.789893+0000","last_unstale":"2024-03-23T09:41:03.145332+0000","last_undegraded":"2024-03-23T09:41:03.145332+0000","last_fullsized":"2024-03-23T09:41:03.145332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665949+0000","last_change":"2024-03-23T09:41:03.154684+0000","last_active":"2024-03-23T09:41:04.665949+0000","last_peered":"2024-03-23T09:41:04.665949+0000","last_clean":"2024-03-23T09:41:04.665949+0000","last_became_active":"2024-03-23T09:41:03.154441+0000","last_became_peered":"2024-03-23T09:41:03.154441+0000","last_unstale":"2024-03-23T09:41:04.665949+0000","last_undegraded":"2024-03-23T09:41:04.665949+0000","last_fullsized":"2024-03-23T09:41:04.665949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.149205+0000","last_change":"2024-03-23T09:41:02.792603+0000","last_active":"2024-03-23T09:41:03.149205+0000","last_peered":"2024-03-23T09:41:03.149205+0000","last_clean":"2024-03-23T09:41:03.149205+0000","last_became_active":"2024-03-23T09:41:02.792316+0000","last_became_peered":"2024-03-23T09:41:02.792316+0000","last_unstale":"2024-03-23T09:41:03.149205+0000","last_undegraded":"2024-03-23T09:41:03.149205+0000","last_fullsized":"2024-03-23T09:41:03.149205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142096+0000","last_change":"2024-03-23T09:41:03.147255+0000","last_active":"2024-03-23T09:41:04.142096+0000","last_peered":"2024-03-23T09:41:04.142096+0000","last_clean":"2024-03-23T09:41:04.142096+0000","last_became_active":"2024-03-23T09:41:03.147013+0000","last_became_peered":"2024-03-23T09:41:03.147013+0000","last_unstale":"2024-03-23T09:41:04.142096+0000","last_undegraded":"2024-03-23T09:41:04.142096+0000","last_fullsized":"2024-03-23T09:41:04.142096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:16.974803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.152789+0000","last_change":"2024-03-23T09:41:02.791718+0000","last_active":"2024-03-23T09:41:03.152789+0000","last_peered":"2024-03-23T09:41:03.152789+0000","last_clean":"2024-03-23T09:41:03.152789+0000","last_became_active":"2024-03-23T09:41:02.791442+0000","last_became_peered":"2024-03-23T09:41:02.791442+0000","last_unstale":"2024-03-23T09:41:03.152789+0000","last_undegraded":"2024-03-23T09:41:03.152789+0000","last_fullsized":"2024-03-23T09:41:03.152789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155272+0000","last_change":"2024-03-23T09:41:03.171999+0000","last_active":"2024-03-23T09:41:04.155272+0000","last_peered":"2024-03-23T09:41:04.155272+0000","last_clean":"2024-03-23T09:41:04.155272+0000","last_became_active":"2024-03-23T09:41:03.171731+0000","last_became_peered":"2024-03-23T09:41:03.171731+0000","last_unstale":"2024-03-23T09:41:04.155272+0000","last_undegraded":"2024-03-23T09:41:04.155272+0000","last_fullsized":"2024-03-23T09:41:04.155272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:16.974803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.150565+0000","last_change":"2024-03-23T09:41:02.791445+0000","last_active":"2024-03-23T09:41:03.150565+0000","last_peered":"2024-03-23T09:41:03.150565+0000","last_clean":"2024-03-23T09:41:03.150565+0000","last_became_active":"2024-03-23T09:41:02.791075+0000","last_became_peered":"2024-03-23T09:41:02.791075+0000","last_unstale":"2024-03-23T09:41:03.150565+0000","last_undegraded":"2024-03-23T09:41:03.150565+0000","last_fullsized":"2024-03-23T09:41:03.150565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586653+0000","last_change":"2024-03-23T09:41:03.159171+0000","last_active":"2024-03-23T09:41:04.586653+0000","last_peered":"2024-03-23T09:41:04.586653+0000","last_clean":"2024-03-23T09:41:04.586653+0000","last_became_active":"2024-03-23T09:41:03.158869+0000","last_became_peered":"2024-03-23T09:41:03.158869+0000","last_unstale":"2024-03-23T09:41:04.586653+0000","last_undegraded":"2024-03-23T09:41:04.586653+0000","last_fullsized":"2024-03-23T09:41:04.586653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:42:19.838774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.160077+0000","last_change":"2024-03-23T09:41:02.786747+0000","last_active":"2024-03-23T09:41:03.160077+0000","last_peered":"2024-03-23T09:41:03.160077+0000","last_clean":"2024-03-23T09:41:03.160077+0000","last_became_active":"2024-03-23T09:41:02.786499+0000","last_became_peered":"2024-03-23T09:41:02.786499+0000","last_unstale":"2024-03-23T09:41:03.160077+0000","last_undegraded":"2024-03-23T09:41:03.160077+0000","last_fullsized":"2024-03-23T09:41:03.160077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146832+0000","last_change":"2024-03-23T09:41:03.165917+0000","last_active":"2024-03-23T09:41:04.146832+0000","last_peered":"2024-03-23T09:41:04.146832+0000","last_clean":"2024-03-23T09:41:04.146832+0000","last_became_active":"2024-03-23T09:41:03.165617+0000","last_became_peered":"2024-03-23T09:41:03.165617+0000","last_unstale":"2024-03-23T09:41:04.146832+0000","last_undegraded":"2024-03-23T09:41:04.146832+0000","last_fullsized":"2024-03-23T09:41:04.146832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:49:00.143163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.169412+0000","last_change":"2024-03-23T09:41:02.787570+0000","last_active":"2024-03-23T09:41:03.169412+0000","last_peered":"2024-03-23T09:41:03.169412+0000","last_clean":"2024-03-23T09:41:03.169412+0000","last_became_active":"2024-03-23T09:41:02.785815+0000","last_became_peered":"2024-03-23T09:41:02.785815+0000","last_unstale":"2024-03-23T09:41:03.169412+0000","last_undegraded":"2024-03-23T09:41:03.169412+0000","last_fullsized":"2024-03-23T09:41:03.169412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146644+0000","last_change":"2024-03-23T09:41:03.167089+0000","last_active":"2024-03-23T09:41:04.146644+0000","last_peered":"2024-03-23T09:41:04.146644+0000","last_clean":"2024-03-23T09:41:04.146644+0000","last_became_active":"2024-03-23T09:41:03.166788+0000","last_became_peered":"2024-03-23T09:41:03.166788+0000","last_unstale":"2024-03-23T09:41:04.146644+0000","last_undegraded":"2024-03-23T09:41:04.146644+0000","last_fullsized":"2024-03-23T09:41:04.146644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:22:09.627571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.161339+0000","last_change":"2024-03-23T09:41:02.785499+0000","last_active":"2024-03-23T09:41:03.161339+0000","last_peered":"2024-03-23T09:41:03.161339+0000","last_clean":"2024-03-23T09:41:03.161339+0000","last_became_active":"2024-03-23T09:41:02.785228+0000","last_became_peered":"2024-03-23T09:41:02.785228+0000","last_unstale":"2024-03-23T09:41:03.161339+0000","last_undegraded":"2024-03-23T09:41:03.161339+0000","last_fullsized":"2024-03-23T09:41:03.161339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586476+0000","last_change":"2024-03-23T09:41:03.164565+0000","last_active":"2024-03-23T09:41:04.586476+0000","last_peered":"2024-03-23T09:41:04.586476+0000","last_clean":"2024-03-23T09:41:04.586476+0000","last_became_active":"2024-03-23T09:41:03.164285+0000","last_became_peered":"2024-03-23T09:41:03.164285+0000","last_unstale":"2024-03-23T09:41:04.586476+0000","last_undegraded":"2024-03-23T09:41:04.586476+0000","last_fullsized":"2024-03-23T09:41:04.586476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:33.315317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.149928+0000","last_change":"2024-03-23T09:41:02.790603+0000","last_active":"2024-03-23T09:41:03.149928+0000","last_peered":"2024-03-23T09:41:03.149928+0000","last_clean":"2024-03-23T09:41:03.149928+0000","last_became_active":"2024-03-23T09:41:02.790337+0000","last_became_peered":"2024-03-23T09:41:02.790337+0000","last_unstale":"2024-03-23T09:41:03.149928+0000","last_undegraded":"2024-03-23T09:41:03.149928+0000","last_fullsized":"2024-03-23T09:41:03.149928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146310+0000","last_change":"2024-03-23T09:41:03.170711+0000","last_active":"2024-03-23T09:41:04.146310+0000","last_peered":"2024-03-23T09:41:04.146310+0000","last_clean":"2024-03-23T09:41:04.146310+0000","last_became_active":"2024-03-23T09:41:03.170418+0000","last_became_peered":"2024-03-23T09:41:03.170418+0000","last_unstale":"2024-03-23T09:41:04.146310+0000","last_undegraded":"2024-03-23T09:41:04.146310+0000","last_fullsized":"2024-03-23T09:41:04.146310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:16.974803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.153343+0000","last_change":"2024-03-23T09:41:02.788294+0000","last_active":"2024-03-23T09:41:03.153343+0000","last_peered":"2024-03-23T09:41:03.153343+0000","last_clean":"2024-03-23T09:41:03.153343+0000","last_became_active":"2024-03-23T09:41:02.787911+0000","last_became_peered":"2024-03-23T09:41:02.787911+0000","last_unstale":"2024-03-23T09:41:03.153343+0000","last_undegraded":"2024-03-23T09:41:03.153343+0000","last_fullsized":"2024-03-23T09:41:03.153343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:09:59.090234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666362+0000","last_change":"2024-03-23T09:41:03.166733+0000","last_active":"2024-03-23T09:41:04.666362+0000","last_peered":"2024-03-23T09:41:04.666362+0000","last_clean":"2024-03-23T09:41:04.666362+0000","last_became_active":"2024-03-23T09:41:03.165156+0000","last_became_peered":"2024-03-23T09:41:03.165156+0000","last_unstale":"2024-03-23T09:41:04.666362+0000","last_undegraded":"2024-03-23T09:41:04.666362+0000","last_fullsized":"2024-03-23T09:41:04.666362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:06.444696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.163373+0000","last_change":"2024-03-23T09:41:02.784571+0000","last_active":"2024-03-23T09:41:03.163373+0000","last_peered":"2024-03-23T09:41:03.163373+0000","last_clean":"2024-03-23T09:41:03.163373+0000","last_became_active":"2024-03-23T09:41:02.784282+0000","last_became_peered":"2024-03-23T09:41:02.784282+0000","last_unstale":"2024-03-23T09:41:03.163373+0000","last_undegraded":"2024-03-23T09:41:03.163373+0000","last_fullsized":"2024-03-23T09:41:03.163373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146163+0000","last_change":"2024-03-23T09:41:03.168817+0000","last_active":"2024-03-23T09:41:04.146163+0000","last_peered":"2024-03-23T09:41:04.146163+0000","last_clean":"2024-03-23T09:41:04.146163+0000","last_became_active":"2024-03-23T09:41:03.168491+0000","last_became_peered":"2024-03-23T09:41:03.168491+0000","last_unstale":"2024-03-23T09:41:04.146163+0000","last_undegraded":"2024-03-23T09:41:04.146163+0000","last_fullsized":"2024-03-23T09:41:04.146163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:51:26.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.162936+0000","last_change":"2024-03-23T09:41:02.787278+0000","last_active":"2024-03-23T09:41:03.162936+0000","last_peered":"2024-03-23T09:41:03.162936+0000","last_clean":"2024-03-23T09:41:03.162936+0000","last_became_active":"2024-03-23T09:41:02.786760+0000","last_became_peered":"2024-03-23T09:41:02.786760+0000","last_unstale":"2024-03-23T09:41:03.162936+0000","last_undegraded":"2024-03-23T09:41:03.162936+0000","last_fullsized":"2024-03-23T09:41:03.162936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:14.040281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146844+0000","last_change":"2024-03-23T09:41:03.168702+0000","last_active":"2024-03-23T09:41:04.146844+0000","last_peered":"2024-03-23T09:41:04.146844+0000","last_clean":"2024-03-23T09:41:04.146844+0000","last_became_active":"2024-03-23T09:41:03.168425+0000","last_became_peered":"2024-03-23T09:41:03.168425+0000","last_unstale":"2024-03-23T09:41:04.146844+0000","last_undegraded":"2024-03-23T09:41:04.146844+0000","last_fullsized":"2024-03-23T09:41:04.146844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:47.109633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.158014+0000","last_change":"2024-03-23T09:41:02.785601+0000","last_active":"2024-03-23T09:41:03.158014+0000","last_peered":"2024-03-23T09:41:03.158014+0000","last_clean":"2024-03-23T09:41:03.158014+0000","last_became_active":"2024-03-23T09:41:02.785347+0000","last_became_peered":"2024-03-23T09:41:02.785347+0000","last_unstale":"2024-03-23T09:41:03.158014+0000","last_undegraded":"2024-03-23T09:41:03.158014+0000","last_fullsized":"2024-03-23T09:41:03.158014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.147217+0000","last_change":"2024-03-23T09:41:03.170126+0000","last_active":"2024-03-23T09:41:04.147217+0000","last_peered":"2024-03-23T09:41:04.147217+0000","last_clean":"2024-03-23T09:41:04.147217+0000","last_became_active":"2024-03-23T09:41:03.169191+0000","last_became_peered":"2024-03-23T09:41:03.169191+0000","last_unstale":"2024-03-23T09:41:04.147217+0000","last_undegraded":"2024-03-23T09:41:04.147217+0000","last_fullsized":"2024-03-23T09:41:04.147217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:33:41.605578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.168081+0000","last_change":"2024-03-23T09:41:02.781849+0000","last_active":"2024-03-23T09:41:03.168081+0000","last_peered":"2024-03-23T09:41:03.168081+0000","last_clean":"2024-03-23T09:41:03.168081+0000","last_became_active":"2024-03-23T09:41:02.781582+0000","last_became_peered":"2024-03-23T09:41:02.781582+0000","last_unstale":"2024-03-23T09:41:03.168081+0000","last_undegraded":"2024-03-23T09:41:03.168081+0000","last_fullsized":"2024-03-23T09:41:03.168081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155217+0000","last_change":"2024-03-23T09:41:03.156001+0000","last_active":"2024-03-23T09:41:04.155217+0000","last_peered":"2024-03-23T09:41:04.155217+0000","last_clean":"2024-03-23T09:41:04.155217+0000","last_became_active":"2024-03-23T09:41:03.155692+0000","last_became_peered":"2024-03-23T09:41:03.155692+0000","last_unstale":"2024-03-23T09:41:04.155217+0000","last_undegraded":"2024-03-23T09:41:04.155217+0000","last_fullsized":"2024-03-23T09:41:04.155217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:54:59.125701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.144394+0000","last_change":"2024-03-23T09:41:02.786345+0000","last_active":"2024-03-23T09:41:03.144394+0000","last_peered":"2024-03-23T09:41:03.144394+0000","last_clean":"2024-03-23T09:41:03.144394+0000","last_became_active":"2024-03-23T09:41:02.786070+0000","last_became_peered":"2024-03-23T09:41:02.786070+0000","last_unstale":"2024-03-23T09:41:03.144394+0000","last_undegraded":"2024-03-23T09:41:03.144394+0000","last_fullsized":"2024-03-23T09:41:03.144394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:51.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146543+0000","last_change":"2024-03-23T09:41:03.171409+0000","last_active":"2024-03-23T09:41:04.146543+0000","last_peered":"2024-03-23T09:41:04.146543+0000","last_clean":"2024-03-23T09:41:04.146543+0000","last_became_active":"2024-03-23T09:41:03.170992+0000","last_became_peered":"2024-03-23T09:41:03.170992+0000","last_unstale":"2024-03-23T09:41:04.146543+0000","last_undegraded":"2024-03-23T09:41:04.146543+0000","last_fullsized":"2024-03-23T09:41:04.146543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:40:45.654649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.159124+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:41:03.159124+0000","last_peered":"2024-03-23T09:41:03.159124+0000","last_clean":"2024-03-23T09:41:03.159124+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:41:03.159124+0000","last_undegraded":"2024-03-23T09:41:03.159124+0000","last_fullsized":"2024-03-23T09:41:03.159124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.149309+0000","last_change":"2024-03-23T09:41:02.780681+0000","last_active":"2024-03-23T09:41:03.149309+0000","last_peered":"2024-03-23T09:41:03.149309+0000","last_clean":"2024-03-23T09:41:03.149309+0000","last_became_active":"2024-03-23T09:41:02.780040+0000","last_became_peered":"2024-03-23T09:41:02.780040+0000","last_unstale":"2024-03-23T09:41:03.149309+0000","last_undegraded":"2024-03-23T09:41:03.149309+0000","last_fullsized":"2024-03-23T09:41:03.149309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666144+0000","last_change":"2024-03-23T09:41:03.159071+0000","last_active":"2024-03-23T09:41:04.666144+0000","last_peered":"2024-03-23T09:41:04.666144+0000","last_clean":"2024-03-23T09:41:04.666144+0000","last_became_active":"2024-03-23T09:41:03.158637+0000","last_became_peered":"2024-03-23T09:41:03.158637+0000","last_unstale":"2024-03-23T09:41:04.666144+0000","last_undegraded":"2024-03-23T09:41:04.666144+0000","last_fullsized":"2024-03-23T09:41:04.666144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:33:41.605578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.159038+0000","last_change":"2024-03-23T09:41:02.787080+0000","last_active":"2024-03-23T09:41:03.159038+0000","last_peered":"2024-03-23T09:41:03.159038+0000","last_clean":"2024-03-23T09:41:03.159038+0000","last_became_active":"2024-03-23T09:41:02.786573+0000","last_became_peered":"2024-03-23T09:41:02.786573+0000","last_unstale":"2024-03-23T09:41:03.159038+0000","last_undegraded":"2024-03-23T09:41:03.159038+0000","last_fullsized":"2024-03-23T09:41:03.159038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:12.910390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586381+0000","last_change":"2024-03-23T09:41:03.159217+0000","last_active":"2024-03-23T09:41:04.586381+0000","last_peered":"2024-03-23T09:41:04.586381+0000","last_clean":"2024-03-23T09:41:04.586381+0000","last_became_active":"2024-03-23T09:41:03.157721+0000","last_became_peered":"2024-03-23T09:41:03.157721+0000","last_unstale":"2024-03-23T09:41:04.586381+0000","last_undegraded":"2024-03-23T09:41:04.586381+0000","last_fullsized":"2024-03-23T09:41:04.586381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:28.007092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.145411+0000","last_change":"2024-03-23T09:41:02.785777+0000","last_active":"2024-03-23T09:41:03.145411+0000","last_peered":"2024-03-23T09:41:03.145411+0000","last_clean":"2024-03-23T09:41:03.145411+0000","last_became_active":"2024-03-23T09:41:02.784961+0000","last_became_peered":"2024-03-23T09:41:02.784961+0000","last_unstale":"2024-03-23T09:41:03.145411+0000","last_undegraded":"2024-03-23T09:41:03.145411+0000","last_fullsized":"2024-03-23T09:41:03.145411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142138+0000","last_change":"2024-03-23T09:41:03.146739+0000","last_active":"2024-03-23T09:41:04.142138+0000","last_peered":"2024-03-23T09:41:04.142138+0000","last_clean":"2024-03-23T09:41:04.142138+0000","last_became_active":"2024-03-23T09:41:03.146310+0000","last_became_peered":"2024-03-23T09:41:03.146310+0000","last_unstale":"2024-03-23T09:41:04.142138+0000","last_undegraded":"2024-03-23T09:41:04.142138+0000","last_fullsized":"2024-03-23T09:41:04.142138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:10.603669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.150027+0000","last_change":"2024-03-23T09:41:02.789860+0000","last_active":"2024-03-23T09:41:03.150027+0000","last_peered":"2024-03-23T09:41:03.150027+0000","last_clean":"2024-03-23T09:41:03.150027+0000","last_became_active":"2024-03-23T09:41:02.787110+0000","last_became_peered":"2024-03-23T09:41:02.787110+0000","last_unstale":"2024-03-23T09:41:03.150027+0000","last_undegraded":"2024-03-23T09:41:03.150027+0000","last_fullsized":"2024-03-23T09:41:03.150027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146588+0000","last_change":"2024-03-23T09:41:03.162623+0000","last_active":"2024-03-23T09:41:04.146588+0000","last_peered":"2024-03-23T09:41:04.146588+0000","last_clean":"2024-03-23T09:41:04.146588+0000","last_became_active":"2024-03-23T09:41:03.162237+0000","last_became_peered":"2024-03-23T09:41:03.162237+0000","last_unstale":"2024-03-23T09:41:04.146588+0000","last_undegraded":"2024-03-23T09:41:04.146588+0000","last_fullsized":"2024-03-23T09:41:04.146588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:49.353786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.141570+0000","last_change":"2024-03-23T09:41:02.785671+0000","last_active":"2024-03-23T09:41:03.141570+0000","last_peered":"2024-03-23T09:41:03.141570+0000","last_clean":"2024-03-23T09:41:03.141570+0000","last_became_active":"2024-03-23T09:41:02.785397+0000","last_became_peered":"2024-03-23T09:41:02.785397+0000","last_unstale":"2024-03-23T09:41:03.141570+0000","last_undegraded":"2024-03-23T09:41:03.141570+0000","last_fullsized":"2024-03-23T09:41:03.141570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155275+0000","last_change":"2024-03-23T09:41:03.158343+0000","last_active":"2024-03-23T09:41:04.155275+0000","last_peered":"2024-03-23T09:41:04.155275+0000","last_clean":"2024-03-23T09:41:04.155275+0000","last_became_active":"2024-03-23T09:41:03.158079+0000","last_became_peered":"2024-03-23T09:41:03.158079+0000","last_unstale":"2024-03-23T09:41:04.155275+0000","last_undegraded":"2024-03-23T09:41:04.155275+0000","last_fullsized":"2024-03-23T09:41:04.155275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:46:38.667131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.161554+0000","last_change":"2024-03-23T09:41:02.788728+0000","last_active":"2024-03-23T09:41:03.161554+0000","last_peered":"2024-03-23T09:41:03.161554+0000","last_clean":"2024-03-23T09:41:03.161554+0000","last_became_active":"2024-03-23T09:41:02.788461+0000","last_became_peered":"2024-03-23T09:41:02.788461+0000","last_unstale":"2024-03-23T09:41:03.161554+0000","last_undegraded":"2024-03-23T09:41:03.161554+0000","last_fullsized":"2024-03-23T09:41:03.161554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142267+0000","last_change":"2024-03-23T09:41:03.147531+0000","last_active":"2024-03-23T09:41:04.142267+0000","last_peered":"2024-03-23T09:41:04.142267+0000","last_clean":"2024-03-23T09:41:04.142267+0000","last_became_active":"2024-03-23T09:41:03.147207+0000","last_became_peered":"2024-03-23T09:41:03.147207+0000","last_unstale":"2024-03-23T09:41:04.142267+0000","last_undegraded":"2024-03-23T09:41:04.142267+0000","last_fullsized":"2024-03-23T09:41:04.142267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:42:19.838774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.142336+0000","last_change":"2024-03-23T09:41:02.789060+0000","last_active":"2024-03-23T09:41:04.142336+0000","last_peered":"2024-03-23T09:41:04.142336+0000","last_clean":"2024-03-23T09:41:04.142336+0000","last_became_active":"2024-03-23T09:41:02.788772+0000","last_became_peered":"2024-03-23T09:41:02.788772+0000","last_unstale":"2024-03-23T09:41:04.142336+0000","last_undegraded":"2024-03-23T09:41:04.142336+0000","last_fullsized":"2024-03-23T09:41:04.142336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666270+0000","last_change":"2024-03-23T09:41:03.169569+0000","last_active":"2024-03-23T09:41:04.666270+0000","last_peered":"2024-03-23T09:41:04.666270+0000","last_clean":"2024-03-23T09:41:04.666270+0000","last_became_active":"2024-03-23T09:41:03.169275+0000","last_became_peered":"2024-03-23T09:41:03.169275+0000","last_unstale":"2024-03-23T09:41:04.666270+0000","last_undegraded":"2024-03-23T09:41:04.666270+0000","last_fullsized":"2024-03-23T09:41:04.666270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:30:48.469383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.168962+0000","last_change":"2024-03-23T09:41:02.788239+0000","last_active":"2024-03-23T09:41:03.168962+0000","last_peered":"2024-03-23T09:41:03.168962+0000","last_clean":"2024-03-23T09:41:03.168962+0000","last_became_active":"2024-03-23T09:41:02.787874+0000","last_became_peered":"2024-03-23T09:41:02.787874+0000","last_unstale":"2024-03-23T09:41:03.168962+0000","last_undegraded":"2024-03-23T09:41:03.168962+0000","last_fullsized":"2024-03-23T09:41:03.168962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586705+0000","last_change":"2024-03-23T09:41:03.156480+0000","last_active":"2024-03-23T09:41:04.586705+0000","last_peered":"2024-03-23T09:41:04.586705+0000","last_clean":"2024-03-23T09:41:04.586705+0000","last_became_active":"2024-03-23T09:41:03.155683+0000","last_became_peered":"2024-03-23T09:41:03.155683+0000","last_unstale":"2024-03-23T09:41:04.586705+0000","last_undegraded":"2024-03-23T09:41:04.586705+0000","last_fullsized":"2024-03-23T09:41:04.586705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:09.777921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.160846+0000","last_change":"2024-03-23T09:41:02.789413+0000","last_active":"2024-03-23T09:41:03.160846+0000","last_peered":"2024-03-23T09:41:03.160846+0000","last_clean":"2024-03-23T09:41:03.160846+0000","last_became_active":"2024-03-23T09:41:02.788397+0000","last_became_peered":"2024-03-23T09:41:02.788397+0000","last_unstale":"2024-03-23T09:41:03.160846+0000","last_undegraded":"2024-03-23T09:41:03.160846+0000","last_fullsized":"2024-03-23T09:41:03.160846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:20.350813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.146940+0000","last_change":"2024-03-23T09:41:03.148447+0000","last_active":"2024-03-23T09:41:04.146940+0000","last_peered":"2024-03-23T09:41:04.146940+0000","last_clean":"2024-03-23T09:41:04.146940+0000","last_became_active":"2024-03-23T09:41:03.148194+0000","last_became_peered":"2024-03-23T09:41:03.148194+0000","last_unstale":"2024-03-23T09:41:04.146940+0000","last_undegraded":"2024-03-23T09:41:04.146940+0000","last_fullsized":"2024-03-23T09:41:04.146940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:21.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.141503+0000","last_change":"2024-03-23T09:41:02.782038+0000","last_active":"2024-03-23T09:41:03.141503+0000","last_peered":"2024-03-23T09:41:03.141503+0000","last_clean":"2024-03-23T09:41:03.141503+0000","last_became_active":"2024-03-23T09:41:02.781322+0000","last_became_peered":"2024-03-23T09:41:02.781322+0000","last_unstale":"2024-03-23T09:41:03.141503+0000","last_undegraded":"2024-03-23T09:41:03.141503+0000","last_fullsized":"2024-03-23T09:41:03.141503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:41.536190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.665970+0000","last_change":"2024-03-23T09:41:03.148500+0000","last_active":"2024-03-23T09:41:04.665970+0000","last_peered":"2024-03-23T09:41:04.665970+0000","last_clean":"2024-03-23T09:41:04.665970+0000","last_became_active":"2024-03-23T09:41:03.148263+0000","last_became_peered":"2024-03-23T09:41:03.148263+0000","last_unstale":"2024-03-23T09:41:04.665970+0000","last_undegraded":"2024-03-23T09:41:04.665970+0000","last_fullsized":"2024-03-23T09:41:04.665970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:51:26.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.141242+0000","last_change":"2024-03-23T09:41:02.785162+0000","last_active":"2024-03-23T09:41:03.141242+0000","last_peered":"2024-03-23T09:41:03.141242+0000","last_clean":"2024-03-23T09:41:03.141242+0000","last_became_active":"2024-03-23T09:41:02.784744+0000","last_became_peered":"2024-03-23T09:41:02.784744+0000","last_unstale":"2024-03-23T09:41:03.141242+0000","last_undegraded":"2024-03-23T09:41:03.141242+0000","last_fullsized":"2024-03-23T09:41:03.141242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:48:59.141432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666237+0000","last_change":"2024-03-23T09:41:03.160780+0000","last_active":"2024-03-23T09:41:04.666237+0000","last_peered":"2024-03-23T09:41:04.666237+0000","last_clean":"2024-03-23T09:41:04.666237+0000","last_became_active":"2024-03-23T09:41:03.159686+0000","last_became_peered":"2024-03-23T09:41:03.159686+0000","last_unstale":"2024-03-23T09:41:04.666237+0000","last_undegraded":"2024-03-23T09:41:04.666237+0000","last_fullsized":"2024-03-23T09:41:04.666237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:16.974803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.170285+0000","last_change":"2024-03-23T09:41:02.786708+0000","last_active":"2024-03-23T09:41:03.170285+0000","last_peered":"2024-03-23T09:41:03.170285+0000","last_clean":"2024-03-23T09:41:03.170285+0000","last_became_active":"2024-03-23T09:41:02.786306+0000","last_became_peered":"2024-03-23T09:41:02.786306+0000","last_unstale":"2024-03-23T09:41:03.170285+0000","last_undegraded":"2024-03-23T09:41:03.170285+0000","last_fullsized":"2024-03-23T09:41:03.170285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:55.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666270+0000","last_change":"2024-03-23T09:41:03.157669+0000","last_active":"2024-03-23T09:41:04.666270+0000","last_peered":"2024-03-23T09:41:04.666270+0000","last_clean":"2024-03-23T09:41:04.666270+0000","last_became_active":"2024-03-23T09:41:03.157206+0000","last_became_peered":"2024-03-23T09:41:03.157206+0000","last_unstale":"2024-03-23T09:41:04.666270+0000","last_undegraded":"2024-03-23T09:41:04.666270+0000","last_fullsized":"2024-03-23T09:41:04.666270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:53:01.847138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155328+0000","last_change":"2024-03-23T09:41:03.158925+0000","last_active":"2024-03-23T09:41:04.155328+0000","last_peered":"2024-03-23T09:41:04.155328+0000","last_clean":"2024-03-23T09:41:04.155328+0000","last_became_active":"2024-03-23T09:41:03.158643+0000","last_became_peered":"2024-03-23T09:41:03.158643+0000","last_unstale":"2024-03-23T09:41:04.155328+0000","last_undegraded":"2024-03-23T09:41:04.155328+0000","last_fullsized":"2024-03-23T09:41:04.155328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:32:17.579489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.170957+0000","last_change":"2024-03-23T09:41:02.786654+0000","last_active":"2024-03-23T09:41:03.170957+0000","last_peered":"2024-03-23T09:41:03.170957+0000","last_clean":"2024-03-23T09:41:03.170957+0000","last_became_active":"2024-03-23T09:41:02.786387+0000","last_became_peered":"2024-03-23T09:41:02.786387+0000","last_unstale":"2024-03-23T09:41:03.170957+0000","last_undegraded":"2024-03-23T09:41:03.170957+0000","last_fullsized":"2024-03-23T09:41:03.170957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:40:48.352055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.155110+0000","last_change":"2024-03-23T09:41:03.158846+0000","last_active":"2024-03-23T09:41:04.155110+0000","last_peered":"2024-03-23T09:41:04.155110+0000","last_clean":"2024-03-23T09:41:04.155110+0000","last_became_active":"2024-03-23T09:41:03.158433+0000","last_became_peered":"2024-03-23T09:41:03.158433+0000","last_unstale":"2024-03-23T09:41:04.155110+0000","last_undegraded":"2024-03-23T09:41:04.155110+0000","last_fullsized":"2024-03-23T09:41:04.155110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:11:48.328619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.168704+0000","last_change":"2024-03-23T09:41:02.788509+0000","last_active":"2024-03-23T09:41:03.168704+0000","last_peered":"2024-03-23T09:41:03.168704+0000","last_clean":"2024-03-23T09:41:03.168704+0000","last_became_active":"2024-03-23T09:41:02.788072+0000","last_became_peered":"2024-03-23T09:41:02.788072+0000","last_unstale":"2024-03-23T09:41:03.168704+0000","last_undegraded":"2024-03-23T09:41:03.168704+0000","last_fullsized":"2024-03-23T09:41:03.168704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:11.278420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.586632+0000","last_change":"2024-03-23T09:41:03.159417+0000","last_active":"2024-03-23T09:41:04.586632+0000","last_peered":"2024-03-23T09:41:04.586632+0000","last_clean":"2024-03-23T09:41:04.586632+0000","last_became_active":"2024-03-23T09:41:03.159146+0000","last_became_peered":"2024-03-23T09:41:03.159146+0000","last_unstale":"2024-03-23T09:41:04.586632+0000","last_undegraded":"2024-03-23T09:41:04.586632+0000","last_fullsized":"2024-03-23T09:41:04.586632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:10.603669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.163045+0000","last_change":"2024-03-23T09:41:02.789237+0000","last_active":"2024-03-23T09:41:03.163045+0000","last_peered":"2024-03-23T09:41:03.163045+0000","last_clean":"2024-03-23T09:41:03.163045+0000","last_became_active":"2024-03-23T09:41:02.788896+0000","last_became_peered":"2024-03-23T09:41:02.788896+0000","last_unstale":"2024-03-23T09:41:03.163045+0000","last_undegraded":"2024-03-23T09:41:03.163045+0000","last_fullsized":"2024-03-23T09:41:03.163045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:18.436284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-23T09:41:04.666186+0000","last_change":"2024-03-23T09:41:03.163326+0000","last_active":"2024-03-23T09:41:04.666186+0000","last_peered":"2024-03-23T09:41:04.666186+0000","last_clean":"2024-03-23T09:41:04.666186+0000","last_became_active":"2024-03-23T09:41:03.163069+0000","last_became_peered":"2024-03-23T09:41:03.163069+0000","last_unstale":"2024-03-23T09:41:04.666186+0000","last_undegraded":"2024-03-23T09:41:04.666186+0000","last_fullsized":"2024-03-23T09:41:04.666186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:02.765916+0000","last_clean_scrub_stamp":"2024-03-23T09:41:02.765916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:32.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T09:41:03.171686+0000","last_change":"2024-03-23T09:41:02.787414+0000","last_active":"2024-03-23T09:41:03.171686+0000","last_peered":"2024-03-23T09:41:03.171686+0000","last_clean":"2024-03-23T09:41:03.171686+0000","last_became_active":"2024-03-23T09:41:02.787074+0000","last_became_peered":"2024-03-23T09:41:02.787074+0000","last_unstale":"2024-03-23T09:41:03.171686+0000","last_undegraded":"2024-03-23T09:41:03.171686+0000","last_fullsized":"2024-03-23T09:41:03.171686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:01.764185+0000","last_clean_scrub_stamp":"2024-03-23T09:41:01.764185+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:02:19.284328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28392,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343448,"statfs":{"total":96636764160,"available":96607690752,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":76514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":819200,"data_stored":665956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:41:13.615 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-23T09:41:13.615 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-23T09:41:13.615 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-23T09:41:13.615 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-23T09:41:13.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.751+0000 7f08f49c9640 1 Processor -- start 2024-03-23T09:41:13.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.751+0000 7f08f49c9640 1 -- start start 2024-03-23T09:41:13.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.752+0000 7f08f49c9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f08f00b8740 0x7f08f015d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.752+0000 7f08f49c9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08f00cfb30 0x7f08f00cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.752+0000 7f08f49c9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f00b45e0 con 0x7f08f00b8360 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.752+0000 7f08f49c9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f00b48c0 con 0x7f08f00b8740 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.752+0000 7f08f49c9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f00b4ba0 con 0x7f08f00cfb30 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.752+0000 7f08ee59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f08f00b8360 0x7f08f00cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57366/0 (socket says 172.21.15.47:57366) 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.752+0000 7f08ee59c640 1 -- 172.21.15.47:0/1517175554 learned_addr learned my addr 172.21.15.47:0/1517175554 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426559905 0 0) 0x7f08f00b45e0 con 0x7f08f00b8360 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08dc003520 con 0x7f08f00b8360 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08edd9b640 1 --2- 172.21.15.47:0/1517175554 >> v2:172.21.15.175:3300/0 conn(0x7f08f00b8740 0x7f08f015d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08eed9d640 1 --2- 172.21.15.47:0/1517175554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08f00cfb30 0x7f08f00cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2168773157 0 0) 0x7f08dc003520 con 0x7f08f00b8360 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08f00cfb30 msgr2=0x7f08f00cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 --2- 172.21.15.47:0/1517175554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08f00cfb30 0x7f08f00cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 >> v2:172.21.15.175:3300/0 conn(0x7f08f00b8740 msgr2=0x7f08f015d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 --2- 172.21.15.47:0/1517175554 >> v2:172.21.15.175:3300/0 conn(0x7f08f00b8740 0x7f08f015d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f00b4e80 con 0x7f08f00b8360 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f08e8001670 con 0x7f08f00b8360 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08edd9b640 1 --2- 172.21.15.47:0/1517175554 >> v2:172.21.15.175:3300/0 conn(0x7f08f00b8740 0x7f08f015d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 >> v1:172.21.15.47:6789/0 conn(0x7f08f00b8360 legacy=0x7f08f00cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 --2- 172.21.15.47:0/1517175554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc003970 0x7f08dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 --2- 172.21.15.47:0/1517175554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08f00cfb30 0x7f08dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 --2- 172.21.15.47:0/1517175554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08dc004d00 0x7f08dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f00b48c0 con 0x7f08dc003970 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f00b4ba0 con 0x7f08dc004d00 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc00bae0 con 0x7f08f00cfb30 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ee59c640 1 --2- 172.21.15.47:0/1517175554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc003970 0x7f08dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08edd9b640 1 --2- 172.21.15.47:0/1517175554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08f00cfb30 0x7f08dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08eed9d640 1 --2- 172.21.15.47:0/1517175554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08dc004d00 0x7f08dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ee59c640 1 -- 172.21.15.47:0/1517175554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08f00cfb30 msgr2=0x7f08dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:13.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ee59c640 1 --2- 172.21.15.47:0/1517175554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08f00cfb30 0x7f08dc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ee59c640 1 -- 172.21.15.47:0/1517175554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08dc004d00 msgr2=0x7f08dc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:13.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ee59c640 1 --2- 172.21.15.47:0/1517175554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08dc004d00 0x7f08dc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08ee59c640 1 -- 172.21.15.47:0/1517175554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc003520 con 0x7f08dc003970 2024-03-23T09:41:13.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.753+0000 7f08eed9d640 1 --2- 172.21.15.47:0/1517175554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08dc004d00 0x7f08dc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:13.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.754+0000 7f08ee59c640 1 --2- 172.21.15.47:0/1517175554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc003970 0x7f08dc003d90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f08e800c290 tx=0x7f08dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99abf3b2696b3245 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:13.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.754+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8005ad0 con 0x7f08dc003970 2024-03-23T09:41:13.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.754+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e8005c30 con 0x7f08dc003970 2024-03-23T09:41:13.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.754+0000 7f08ed59a640 1 -- 172.21.15.47:0/1517175554 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e802c8d0 con 0x7f08dc003970 2024-03-23T09:41:13.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.755+0000 7f08f49c9640 1 -- 172.21.15.47:0/1517175554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc003970 msgr2=0x7f08dc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.755+0000 7f08f49c9640 1 --2- 172.21.15.47:0/1517175554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc003970 0x7f08dc003d90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f08e800c290 tx=0x7f08dc0057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.755+0000 7f08eed9d640 1 -- 172.21.15.47:0/1517175554 reap_dead start 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.755+0000 7f08f49c9640 1 -- 172.21.15.47:0/1517175554 shutdown_connections 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.755+0000 7f08f49c9640 1 -- 172.21.15.47:0/1517175554 >> 172.21.15.47:0/1517175554 conn(0x7f08f00659d0 msgr2=0x7f08f0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.755+0000 7f08f49c9640 1 -- 172.21.15.47:0/1517175554 shutdown_connections 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.755+0000 7f08f49c9640 1 -- 172.21.15.47:0/1517175554 wait complete. 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.756+0000 7f08f49c9640 1 Processor -- start 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.756+0000 7f08f49c9640 1 -- start start 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08f49c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc003970 0x7f08f015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08f49c9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08f01f2f90 0x7f08f01f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08f49c9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f00d0960 0x7f08f00d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08f49c9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f015cfb0 con 0x7f08dc003970 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08f49c9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f00d1320 con 0x7f08f00d0960 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08f49c9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f01f3410 con 0x7f08f01f2f90 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08ee59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc003970 0x7f08f015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08ee59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc003970 0x7f08f015bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46322/0 (socket says 172.21.15.47:46322) 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08ee59c640 1 -- 172.21.15.47:0/37281990 learned_addr learned my addr 172.21.15.47:0/37281990 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08edd9b640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08f01f2f90 0x7f08f01f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08eed9d640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f00d0960 0x7f08f00d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08ee59c640 1 -- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08f01f2f90 msgr2=0x7f08f01f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08ee59c640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08f01f2f90 0x7f08f01f1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08ee59c640 1 -- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f00d0960 msgr2=0x7f08f00d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08ee59c640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f00d0960 0x7f08f00d0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08ee59c640 1 -- 172.21.15.47:0/37281990 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f015c330 con 0x7f08dc003970 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08eed9d640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f00d0960 0x7f08f00d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08ee59c640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc003970 0x7f08f015bc70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f08e8001930 tx=0x7f08e8000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.757+0000 7f08e6ffd640 1 -- 172.21.15.47:0/37281990 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8039070 con 0x7f08dc003970 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.758+0000 7f08e6ffd640 1 -- 172.21.15.47:0/37281990 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e802dd90 con 0x7f08dc003970 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.758+0000 7f08f49c9640 1 -- 172.21.15.47:0/37281990 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f015c610 con 0x7f08dc003970 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.758+0000 7f08e6ffd640 1 -- 172.21.15.47:0/37281990 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e80072b0 con 0x7f08dc003970 2024-03-23T09:41:13.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.758+0000 7f08f49c9640 1 -- 172.21.15.47:0/37281990 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f015cbd0 con 0x7f08dc003970 2024-03-23T09:41:13.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.759+0000 7f08f49c9640 1 -- 172.21.15.47:0/37281990 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f0072360 con 0x7f08dc003970 2024-03-23T09:41:13.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.759+0000 7f08e6ffd640 1 -- 172.21.15.47:0/37281990 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f08e8032070 con 0x7f08dc003970 2024-03-23T09:41:13.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.759+0000 7f08e6ffd640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f08d006ac30 0x7f08d006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:13.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.760+0000 7f08e6ffd640 1 -- 172.21.15.47:0/37281990 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08e803c040 con 0x7f08dc003970 2024-03-23T09:41:13.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.760+0000 7f08edd9b640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f08d006ac30 0x7f08d006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:13.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.760+0000 7f08edd9b640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f08d006ac30 0x7f08d006d0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f08f00cbc80 tx=0x7f08d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:13.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.764+0000 7f08e6ffd640 1 -- 172.21.15.47:0/37281990 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f08e8075700 con 0x7f08dc003970 2024-03-23T09:41:13.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.982+0000 7f08f49c9640 1 -- 172.21.15.47:0/37281990 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f08f008a420 con 0x7f08dc003970 2024-03-23T09:41:13.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.982+0000 7f08e6ffd640 1 -- 172.21.15.47:0/37281990 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f08e8045040 con 0x7f08dc003970 2024-03-23T09:41:13.982 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:13.982 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-23T09:41:13.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.984+0000 7f08f49c9640 1 -- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f08d006ac30 msgr2=0x7f08d006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.984+0000 7f08f49c9640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f08d006ac30 0x7f08d006d0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f08f00cbc80 tx=0x7f08d8008040 comp rx=0 tx=0).stop 2024-03-23T09:41:13.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.984+0000 7f08f49c9640 1 -- 172.21.15.47:0/37281990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc003970 msgr2=0x7f08f015bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:13.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.984+0000 7f08f49c9640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc003970 0x7f08f015bc70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f08e8001930 tx=0x7f08e8000fe0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.984+0000 7f08f49c9640 1 -- 172.21.15.47:0/37281990 shutdown_connections 2024-03-23T09:41:13.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.984+0000 7f08f49c9640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f08d006ac30 0x7f08d006d0f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.984+0000 7f08f49c9640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f00d0960 0x7f08f00d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.984+0000 7f08f49c9640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08f01f2f90 0x7f08f01f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.984+0000 7f08f49c9640 1 --2- 172.21.15.47:0/37281990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc003970 0x7f08f015bc70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:13.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.984+0000 7f08f49c9640 1 -- 172.21.15.47:0/37281990 >> 172.21.15.47:0/37281990 conn(0x7f08f00659d0 msgr2=0x7f08f00cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:13.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.984+0000 7f08f49c9640 1 -- 172.21.15.47:0/37281990 shutdown_connections 2024-03-23T09:41:13.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:13.984+0000 7f08f49c9640 1 -- 172.21.15.47:0/37281990 wait complete. 2024-03-23T09:41:13.994 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-23T09:41:13.995 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:14.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.095+0000 7f9cb1763640 1 Processor -- start 2024-03-23T09:41:14.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.095+0000 7f9cb1763640 1 -- start start 2024-03-23T09:41:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9cb1763640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9cac15ca90 0x7f9cac160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9cb1763640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cac0d11e0 con 0x7f9cac0cde10 2024-03-23T09:41:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9cb1763640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cac160f60 con 0x7f9cac0b8350 2024-03-23T09:41:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9cb1763640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cac161240 con 0x7f9cac15ca90 2024-03-23T09:41:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9cab7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9cac0cde10 0x7f9cac0d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57380/0 (socket says 172.21.15.47:57380) 2024-03-23T09:41:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9cab7fe640 1 -- 172.21.15.47:0/348768924 learned_addr learned my addr 172.21.15.47:0/348768924 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9caaffd640 1 --2- 172.21.15.47:0/348768924 >> v2:172.21.15.175:3300/0 conn(0x7f9cac15ca90 0x7f9cac160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9ca9ffb640 1 -- 172.21.15.47:0/348768924 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752728865 0 0) 0x7f9cac0d11e0 con 0x7f9cac0cde10 2024-03-23T09:41:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9ca9ffb640 1 -- 172.21.15.47:0/348768924 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c94003520 con 0x7f9cac0cde10 2024-03-23T09:41:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9ca9ffb640 1 -- 172.21.15.47:0/348768924 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222452750 0 0) 0x7f9cac160f60 con 0x7f9cac0b8350 2024-03-23T09:41:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9ca9ffb640 1 -- 172.21.15.47:0/348768924 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cac0d11e0 con 0x7f9cac0b8350 2024-03-23T09:41:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9caaffd640 1 -- 172.21.15.47:0/348768924 >> v1:172.21.15.175:6789/0 conn(0x7f9cac0b8350 legacy=0x7f9cac0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9caaffd640 1 -- 172.21.15.47:0/348768924 >> v1:172.21.15.47:6789/0 conn(0x7f9cac0cde10 legacy=0x7f9cac0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.096+0000 7f9caaffd640 1 -- 172.21.15.47:0/348768924 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac161520 con 0x7f9cac15ca90 2024-03-23T09:41:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.097+0000 7f9caaffd640 1 --2- 172.21.15.47:0/348768924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac15ca90 0x7f9cac160710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40097a0 tx=0x7f9ca402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3184acddb4d0069e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.097+0000 7f9ca9ffb640 1 -- 172.21.15.47:0/348768924 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4031830 con 0x7f9cac15ca90 2024-03-23T09:41:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.097+0000 7f9ca9ffb640 1 -- 172.21.15.47:0/348768924 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca4031990 con 0x7f9cac15ca90 2024-03-23T09:41:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.097+0000 7f9ca9ffb640 1 -- 172.21.15.47:0/348768924 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4031d00 con 0x7f9cac15ca90 2024-03-23T09:41:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.098+0000 7f9cb1763640 1 -- 172.21.15.47:0/348768924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac15ca90 msgr2=0x7f9cac160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.098+0000 7f9cb1763640 1 --2- 172.21.15.47:0/348768924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac15ca90 0x7f9cac160710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40097a0 tx=0x7f9ca402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.098+0000 7f9cb1763640 1 -- 172.21.15.47:0/348768924 shutdown_connections 2024-03-23T09:41:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.098+0000 7f9cb1763640 1 --2- 172.21.15.47:0/348768924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac15ca90 0x7f9cac160710 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.098+0000 7f9cb1763640 1 -- 172.21.15.47:0/348768924 >> 172.21.15.47:0/348768924 conn(0x7f9cac0659d0 msgr2=0x7f9cac065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.098+0000 7f9cb1763640 1 -- 172.21.15.47:0/348768924 shutdown_connections 2024-03-23T09:41:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.098+0000 7f9cb1763640 1 -- 172.21.15.47:0/348768924 wait complete. 2024-03-23T09:41:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.099+0000 7f9cb1763640 1 Processor -- start 2024-03-23T09:41:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.099+0000 7f9cb1763640 1 -- start start 2024-03-23T09:41:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cb1763640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac0b8350 0x7f9cac1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cb1763640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac0cde10 0x7f9cac0caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cb1763640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cac15ca90 0x7f9cac155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cb1763640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0cc590 con 0x7f9cac15ca90 2024-03-23T09:41:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cb1763640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0cb980 con 0x7f9cac0cde10 2024-03-23T09:41:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cb1763640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0cbc60 con 0x7f9cac0b8350 2024-03-23T09:41:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cac15ca90 0x7f9cac155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cac15ca90 0x7f9cac155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46328/0 (socket says 172.21.15.47:46328) 2024-03-23T09:41:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cab7fe640 1 -- 172.21.15.47:0/1362754123 learned_addr learned my addr 172.21.15.47:0/1362754123 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9caaffd640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac0b8350 0x7f9cac1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9caa7fc640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac0cde10 0x7f9cac0caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cab7fe640 1 -- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac0b8350 msgr2=0x7f9cac1571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cab7fe640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac0b8350 0x7f9cac1571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cab7fe640 1 -- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac0cde10 msgr2=0x7f9cac0caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cab7fe640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac0cde10 0x7f9cac0caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.100+0000 7f9cab7fe640 1 -- 172.21.15.47:0/1362754123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca4009400 con 0x7f9cac15ca90 2024-03-23T09:41:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.101+0000 7f9cab7fe640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cac15ca90 0x7f9cac155c70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00cb70 tx=0x7f9c9c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.101+0000 7f9c937fe640 1 -- 172.21.15.47:0/1362754123 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c00ace0 con 0x7f9cac15ca90 2024-03-23T09:41:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.101+0000 7f9c937fe640 1 -- 172.21.15.47:0/1362754123 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c00ae40 con 0x7f9cac15ca90 2024-03-23T09:41:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.101+0000 7f9cb1763640 1 -- 172.21.15.47:0/1362754123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac0cbf40 con 0x7f9cac15ca90 2024-03-23T09:41:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.101+0000 7f9c937fe640 1 -- 172.21.15.47:0/1362754123 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c012620 con 0x7f9cac15ca90 2024-03-23T09:41:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.101+0000 7f9cb1763640 1 -- 172.21.15.47:0/1362754123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac15b420 con 0x7f9cac15ca90 2024-03-23T09:41:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.102+0000 7f9cb1763640 1 -- 172.21.15.47:0/1362754123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac072360 con 0x7f9cac15ca90 2024-03-23T09:41:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.102+0000 7f9c937fe640 1 -- 172.21.15.47:0/1362754123 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9c9c007070 con 0x7f9cac15ca90 2024-03-23T09:41:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.102+0000 7f9c937fe640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9c9806ac00 0x7f9c9806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.103+0000 7f9c937fe640 1 -- 172.21.15.47:0/1362754123 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c9c0893d0 con 0x7f9cac15ca90 2024-03-23T09:41:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.103+0000 7f9caaffd640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9c9806ac00 0x7f9c9806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.103+0000 7f9caaffd640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9c9806ac00 0x7f9c9806d0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4036000 tx=0x7f9ca403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.107+0000 7f9c937fe640 1 -- 172.21.15.47:0/1362754123 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9c9c057fa0 con 0x7f9cac15ca90 2024-03-23T09:41:14.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.285+0000 7f9cb1763640 1 -- 172.21.15.47:0/1362754123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cac066be0 con 0x7f9cac15ca90 2024-03-23T09:41:14.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.286+0000 7f9c937fe640 1 -- 172.21.15.47:0/1362754123 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f9c9c057b30 con 0x7f9cac15ca90 2024-03-23T09:41:14.286 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:14.286 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:14.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.288+0000 7f9cb1763640 1 -- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9c9806ac00 msgr2=0x7f9c9806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.288+0000 7f9cb1763640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9c9806ac00 0x7f9c9806d0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4036000 tx=0x7f9ca403c040 comp rx=0 tx=0).stop 2024-03-23T09:41:14.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.288+0000 7f9cb1763640 1 -- 172.21.15.47:0/1362754123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cac15ca90 msgr2=0x7f9cac155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.288+0000 7f9cb1763640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cac15ca90 0x7f9cac155c70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00cb70 tx=0x7f9c9c00a430 comp rx=0 tx=0).stop 2024-03-23T09:41:14.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.288+0000 7f9cb1763640 1 -- 172.21.15.47:0/1362754123 shutdown_connections 2024-03-23T09:41:14.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.288+0000 7f9cb1763640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9c9806ac00 0x7f9c9806d0c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.288+0000 7f9cb1763640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cac15ca90 0x7f9cac155c70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.288+0000 7f9cb1763640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac0cde10 0x7f9cac0caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.288+0000 7f9cb1763640 1 --2- 172.21.15.47:0/1362754123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac0b8350 0x7f9cac1571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.288+0000 7f9cb1763640 1 -- 172.21.15.47:0/1362754123 >> 172.21.15.47:0/1362754123 conn(0x7f9cac0659d0 msgr2=0x7f9cac0b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:14.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.288+0000 7f9cb1763640 1 -- 172.21.15.47:0/1362754123 shutdown_connections 2024-03-23T09:41:14.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.288+0000 7f9cb1763640 1 -- 172.21.15.47:0/1362754123 wait complete. 2024-03-23T09:41:14.298 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:14.299 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:14.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.403+0000 7f402740b640 1 Processor -- start 2024-03-23T09:41:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.403+0000 7f402740b640 1 -- start start 2024-03-23T09:41:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f402740b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402015c990 0x7f4020160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f402740b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f40200b8350 0x7f40200cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f402740b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4020161630 con 0x7f40200cdc40 2024-03-23T09:41:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f402740b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40200ce0b0 con 0x7f40200b8350 2024-03-23T09:41:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f402740b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40200d1030 con 0x7f402015c990 2024-03-23T09:41:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f40251a7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402015c990 0x7f4020160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f40249a6640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f40200b8350 0x7f40200cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f40251a7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402015c990 0x7f4020160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:46866/0 (socket says 172.21.15.47:46866) 2024-03-23T09:41:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f40251a7640 1 -- 172.21.15.47:0/638775263 learned_addr learned my addr 172.21.15.47:0/638775263 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f40259a8640 1 --1- 172.21.15.47:0/638775263 >> v1:172.21.15.175:6789/0 conn(0x7f40200cdc40 0x7f40200b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47930/0 (socket says 172.21.15.47:47930) 2024-03-23T09:41:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f40251a7640 1 -- 172.21.15.47:0/638775263 >> v2:172.21.15.175:3300/0 conn(0x7f40200b8350 msgr2=0x7f40200cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f40251a7640 1 --2- 172.21.15.47:0/638775263 >> v2:172.21.15.175:3300/0 conn(0x7f40200b8350 0x7f40200cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f40251a7640 1 -- 172.21.15.47:0/638775263 >> v1:172.21.15.175:6789/0 conn(0x7f40200cdc40 legacy=0x7f40200b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.404+0000 7f40251a7640 1 -- 172.21.15.47:0/638775263 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40200cfe10 con 0x7f402015c990 2024-03-23T09:41:14.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.405+0000 7f40251a7640 1 --2- 172.21.15.47:0/638775263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402015c990 0x7f4020160690 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4018009870 tx=0x7f401802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e71f3b03ca3d98b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.405+0000 7f401ffff640 1 -- 172.21.15.47:0/638775263 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018031640 con 0x7f402015c990 2024-03-23T09:41:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.405+0000 7f401ffff640 1 -- 172.21.15.47:0/638775263 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40180317a0 con 0x7f402015c990 2024-03-23T09:41:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.405+0000 7f401ffff640 1 -- 172.21.15.47:0/638775263 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018031b30 con 0x7f402015c990 2024-03-23T09:41:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.406+0000 7f402740b640 1 -- 172.21.15.47:0/638775263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402015c990 msgr2=0x7f4020160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.406+0000 7f402740b640 1 --2- 172.21.15.47:0/638775263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402015c990 0x7f4020160690 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4018009870 tx=0x7f401802b8f0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.406+0000 7f402740b640 1 -- 172.21.15.47:0/638775263 shutdown_connections 2024-03-23T09:41:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.406+0000 7f402740b640 1 --2- 172.21.15.47:0/638775263 >> v2:172.21.15.175:3300/0 conn(0x7f40200b8350 0x7f40200cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.406+0000 7f402740b640 1 --2- 172.21.15.47:0/638775263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402015c990 0x7f4020160690 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.406+0000 7f402740b640 1 -- 172.21.15.47:0/638775263 >> 172.21.15.47:0/638775263 conn(0x7f40200659d0 msgr2=0x7f4020065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.406+0000 7f402740b640 1 -- 172.21.15.47:0/638775263 shutdown_connections 2024-03-23T09:41:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.406+0000 7f402740b640 1 -- 172.21.15.47:0/638775263 wait complete. 2024-03-23T09:41:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.407+0000 7f402740b640 1 Processor -- start 2024-03-23T09:41:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.407+0000 7f402740b640 1 -- start start 2024-03-23T09:41:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f402740b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40200b8350 0x7f40200c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f402740b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40200cdc40 0x7f402015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f402740b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f402015c990 0x7f402014d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f402740b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40200c6d90 con 0x7f40200cdc40 2024-03-23T09:41:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f402740b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40201513f0 con 0x7f402015c990 2024-03-23T09:41:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f402740b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40200c6430 con 0x7f40200b8350 2024-03-23T09:41:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f40249a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40200cdc40 0x7f402015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f40249a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40200cdc40 0x7f402015ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46330/0 (socket says 172.21.15.47:46330) 2024-03-23T09:41:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f40249a6640 1 -- 172.21.15.47:0/2405385349 learned_addr learned my addr 172.21.15.47:0/2405385349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f40259a8640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f402015c990 0x7f402014d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f40251a7640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40200b8350 0x7f40200c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f40249a6640 1 -- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40200b8350 msgr2=0x7f40200c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f40249a6640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40200b8350 0x7f40200c71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f40249a6640 1 -- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f402015c990 msgr2=0x7f402014d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f40249a6640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f402015c990 0x7f402014d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f40249a6640 1 -- 172.21.15.47:0/2405385349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40180094d0 con 0x7f40200cdc40 2024-03-23T09:41:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.408+0000 7f40251a7640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40200b8350 0x7f40200c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.409+0000 7f40249a6640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40200cdc40 0x7f402015ec70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f401400e8a0 tx=0x7f401400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.409+0000 7f401dffb640 1 -- 172.21.15.47:0/2405385349 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40140094f0 con 0x7f40200cdc40 2024-03-23T09:41:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.409+0000 7f401dffb640 1 -- 172.21.15.47:0/2405385349 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401400c940 con 0x7f40200cdc40 2024-03-23T09:41:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.409+0000 7f402740b640 1 -- 172.21.15.47:0/2405385349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40200c66d0 con 0x7f40200cdc40 2024-03-23T09:41:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.409+0000 7f401dffb640 1 -- 172.21.15.47:0/2405385349 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401400ccf0 con 0x7f40200cdc40 2024-03-23T09:41:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.409+0000 7f402740b640 1 -- 172.21.15.47:0/2405385349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402015f370 con 0x7f40200cdc40 2024-03-23T09:41:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.410+0000 7f402740b640 1 -- 172.21.15.47:0/2405385349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4020072360 con 0x7f40200cdc40 2024-03-23T09:41:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.410+0000 7f401dffb640 1 -- 172.21.15.47:0/2405385349 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4014013470 con 0x7f40200cdc40 2024-03-23T09:41:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.411+0000 7f401dffb640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f400c06ac00 0x7f400c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.411+0000 7f401dffb640 1 -- 172.21.15.47:0/2405385349 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4014089390 con 0x7f40200cdc40 2024-03-23T09:41:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.411+0000 7f40251a7640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f400c06ac00 0x7f400c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.411+0000 7f40251a7640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f400c06ac00 0x7f400c06d0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4018036000 tx=0x7f4018008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.415+0000 7f401dffb640 1 -- 172.21.15.47:0/2405385349 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4014054600 con 0x7f40200cdc40 2024-03-23T09:41:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.595+0000 7f402740b640 1 -- 172.21.15.47:0/2405385349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4020066be0 con 0x7f40200cdc40 2024-03-23T09:41:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.595+0000 7f401dffb640 1 -- 172.21.15.47:0/2405385349 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4014057c20 con 0x7f40200cdc40 2024-03-23T09:41:14.595 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.597+0000 7f402740b640 1 -- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f400c06ac00 msgr2=0x7f400c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.597+0000 7f402740b640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f400c06ac00 0x7f400c06d0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4018036000 tx=0x7f4018008030 comp rx=0 tx=0).stop 2024-03-23T09:41:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.597+0000 7f402740b640 1 -- 172.21.15.47:0/2405385349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40200cdc40 msgr2=0x7f402015ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.597+0000 7f402740b640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40200cdc40 0x7f402015ec70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f401400e8a0 tx=0x7f401400ed60 comp rx=0 tx=0).stop 2024-03-23T09:41:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.597+0000 7f402740b640 1 -- 172.21.15.47:0/2405385349 shutdown_connections 2024-03-23T09:41:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.597+0000 7f402740b640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f400c06ac00 0x7f400c06d0c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.597+0000 7f402740b640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f402015c990 0x7f402014d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.597+0000 7f402740b640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40200cdc40 0x7f402015ec70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.597+0000 7f402740b640 1 --2- 172.21.15.47:0/2405385349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40200b8350 0x7f40200c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.597+0000 7f402740b640 1 -- 172.21.15.47:0/2405385349 >> 172.21.15.47:0/2405385349 conn(0x7f40200659d0 msgr2=0x7f40200cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.597+0000 7f402740b640 1 -- 172.21.15.47:0/2405385349 shutdown_connections 2024-03-23T09:41:14.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.597+0000 7f402740b640 1 -- 172.21.15.47:0/2405385349 wait complete. 2024-03-23T09:41:14.607 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:14.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.710+0000 7feee0341640 1 Processor -- start 2024-03-23T09:41:14.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.710+0000 7feee0341640 1 -- start start 2024-03-23T09:41:14.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.711+0000 7feee0341640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feed80cec50 0x7feed80cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.711+0000 7feee0341640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80b05f0 0x7feed80b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.711+0000 7feee0341640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed80cb210 con 0x7feed815ac40 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.711+0000 7feee0341640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed80cbac0 con 0x7feed80cec50 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.711+0000 7feee0341640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed80cf610 con 0x7feed80b05f0 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.711+0000 7feede0dd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feed815ac40 0x7feed80cbee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33136/0 (socket says 172.21.15.47:33136) 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.711+0000 7feede0dd640 1 -- 172.21.15.47:0/1125976011 learned_addr learned my addr 172.21.15.47:0/1125976011 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feede8de640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80b05f0 0x7feed80b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586021466 0 0) 0x7feed80cb210 con 0x7feed815ac40 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd8dc640 1 --2- 172.21.15.47:0/1125976011 >> v2:172.21.15.175:3300/0 conn(0x7feed80cec50 0x7feed80cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec0003520 con 0x7feed815ac40 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 151999082 0 0) 0x7feec0003520 con 0x7feed815ac40 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80b05f0 msgr2=0x7feed80b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80b05f0 0x7feed80b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 >> v2:172.21.15.175:3300/0 conn(0x7feed80cec50 msgr2=0x7feed80cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 --2- 172.21.15.47:0/1125976011 >> v2:172.21.15.175:3300/0 conn(0x7feed80cec50 0x7feed80cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed80cc5f0 con 0x7feed815ac40 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7feed0001670 con 0x7feed815ac40 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 >> v1:172.21.15.47:6789/0 conn(0x7feed815ac40 legacy=0x7feed80cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80b05f0 0x7feec0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec0003ef0 0x7feec00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0004d00 0x7feec0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed80cbac0 con 0x7feec0003ef0 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed80cf610 con 0x7feec0004d00 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec000b980 con 0x7feed80b05f0 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd8dc640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec0003ef0 0x7feec00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feede0dd640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80b05f0 0x7feec0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feede8de640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0004d00 0x7feec0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd8dc640 1 -- 172.21.15.47:0/1125976011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80b05f0 msgr2=0x7feec0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd8dc640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80b05f0 0x7feec0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd8dc640 1 -- 172.21.15.47:0/1125976011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0004d00 msgr2=0x7feec0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd8dc640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0004d00 0x7feec0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feedd8dc640 1 -- 172.21.15.47:0/1125976011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec0003520 con 0x7feec0003ef0 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feede0dd640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80b05f0 0x7feec0003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.712+0000 7feede8de640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0004d00 0x7feec0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.713+0000 7feedd8dc640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec0003ef0 0x7feec00047c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feecc009ac0 tx=0x7feec00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e1cd3a374a54c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.713+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc02c6d0 con 0x7feec0003ef0 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.713+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc02c830 con 0x7feec0003ef0 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.713+0000 7feedd0db640 1 -- 172.21.15.47:0/1125976011 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc02cbc0 con 0x7feec0003ef0 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.714+0000 7feee0341640 1 -- 172.21.15.47:0/1125976011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec0003ef0 msgr2=0x7feec00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.714+0000 7feee0341640 1 --2- 172.21.15.47:0/1125976011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec0003ef0 0x7feec00047c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feecc009ac0 tx=0x7feec00057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.714+0000 7feede8de640 1 -- 172.21.15.47:0/1125976011 reap_dead start 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.714+0000 7feee0341640 1 -- 172.21.15.47:0/1125976011 shutdown_connections 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.714+0000 7feee0341640 1 -- 172.21.15.47:0/1125976011 >> 172.21.15.47:0/1125976011 conn(0x7feed80d0670 msgr2=0x7feed80d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.714+0000 7feee0341640 1 -- 172.21.15.47:0/1125976011 shutdown_connections 2024-03-23T09:41:14.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.714+0000 7feee0341640 1 -- 172.21.15.47:0/1125976011 wait complete. 2024-03-23T09:41:14.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.715+0000 7feee0341640 1 Processor -- start 2024-03-23T09:41:14.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.715+0000 7feee0341640 1 -- start start 2024-03-23T09:41:14.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feee0341640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0003ef0 0x7feed814db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feee0341640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80ca330 0x7feed80c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feee0341640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feed80c8df0 0x7feed80b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feee0341640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed806c510 con 0x7feed80c8df0 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feee0341640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed80c9f10 con 0x7feec0003ef0 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feee0341640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed80c9290 con 0x7feed80ca330 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feede8de640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feed80c8df0 0x7feed80b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feede8de640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feed80c8df0 0x7feed80b0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41478/0 (socket says 172.21.15.47:41478) 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feede8de640 1 -- 172.21.15.47:0/2237466113 learned_addr learned my addr 172.21.15.47:0/2237466113 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feede0dd640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0003ef0 0x7feed814db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feedd8dc640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80ca330 0x7feed80c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feede8de640 1 -- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80ca330 msgr2=0x7feed80c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feede8de640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80ca330 0x7feed80c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feede8de640 1 -- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0003ef0 msgr2=0x7feed814db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feede8de640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0003ef0 0x7feed814db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feede8de640 1 -- 172.21.15.47:0/2237466113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed80c9570 con 0x7feed80c8df0 2024-03-23T09:41:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feede0dd640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0003ef0 0x7feed814db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feede8de640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feed80c8df0 0x7feed80b0fe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feed80d1250 tx=0x7feec8010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feed6ffd640 1 -- 172.21.15.47:0/2237466113 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec800e050 con 0x7feed80c8df0 2024-03-23T09:41:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feed6ffd640 1 -- 172.21.15.47:0/2237466113 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec800f040 con 0x7feed80c8df0 2024-03-23T09:41:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feee0341640 1 -- 172.21.15.47:0/2237466113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed80c9850 con 0x7feed80c8df0 2024-03-23T09:41:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feed6ffd640 1 -- 172.21.15.47:0/2237466113 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8003990 con 0x7feed80c8df0 2024-03-23T09:41:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.716+0000 7feee0341640 1 -- 172.21.15.47:0/2237466113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed8155a90 con 0x7feed80c8df0 2024-03-23T09:41:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.717+0000 7feee0341640 1 -- 172.21.15.47:0/2237466113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed80ca050 con 0x7feed80c8df0 2024-03-23T09:41:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.718+0000 7feed6ffd640 1 -- 172.21.15.47:0/2237466113 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7feec8015440 con 0x7feed80c8df0 2024-03-23T09:41:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.718+0000 7feed6ffd640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feec406ac00 0x7feec406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.718+0000 7feed6ffd640 1 -- 172.21.15.47:0/2237466113 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feec8089830 con 0x7feed80c8df0 2024-03-23T09:41:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.718+0000 7feede0dd640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feec406ac00 0x7feec406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.719+0000 7feede0dd640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feec406ac00 0x7feec406d0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feed0001050 tx=0x7feed0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:14.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.722+0000 7feed6ffd640 1 -- 172.21.15.47:0/2237466113 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7feed80ca050 con 0x7feed80c8df0 2024-03-23T09:41:14.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.903+0000 7feee0341640 1 -- 172.21.15.47:0/2237466113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feed80ca050 con 0x7feed80c8df0 2024-03-23T09:41:14.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.904+0000 7feed6ffd640 1 -- 172.21.15.47:0/2237466113 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7feed80ca050 con 0x7feed80c8df0 2024-03-23T09:41:14.903 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.906+0000 7feee0341640 1 -- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feec406ac00 msgr2=0x7feec406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.906+0000 7feee0341640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feec406ac00 0x7feec406d0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feed0001050 tx=0x7feed0009040 comp rx=0 tx=0).stop 2024-03-23T09:41:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.906+0000 7feee0341640 1 -- 172.21.15.47:0/2237466113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feed80c8df0 msgr2=0x7feed80b0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.906+0000 7feee0341640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feed80c8df0 0x7feed80b0fe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feed80d1250 tx=0x7feec8010590 comp rx=0 tx=0).stop 2024-03-23T09:41:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.906+0000 7feee0341640 1 -- 172.21.15.47:0/2237466113 shutdown_connections 2024-03-23T09:41:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.906+0000 7feee0341640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feec406ac00 0x7feec406d0c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.906+0000 7feee0341640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feed80c8df0 0x7feed80b0fe0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.906+0000 7feee0341640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed80ca330 0x7feed80c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.906+0000 7feee0341640 1 --2- 172.21.15.47:0/2237466113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0003ef0 0x7feed814db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.906+0000 7feee0341640 1 -- 172.21.15.47:0/2237466113 >> 172.21.15.47:0/2237466113 conn(0x7feed80d0670 msgr2=0x7feed8159e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.906+0000 7feee0341640 1 -- 172.21.15.47:0/2237466113 shutdown_connections 2024-03-23T09:41:14.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:14.906+0000 7feee0341640 1 -- 172.21.15.47:0/2237466113 wait complete. 2024-03-23T09:41:14.916 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.020+0000 7f188200b640 1 Processor -- start 2024-03-23T09:41:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.020+0000 7f188200b640 1 -- start start 2024-03-23T09:41:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.021+0000 7f188200b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f187c0cde10 0x7f187c160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.021+0000 7f188200b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f187c0b8350 0x7f187c0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.021+0000 7f188200b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187c161630 con 0x7f187c15c990 2024-03-23T09:41:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.021+0000 7f188200b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187c0d1030 con 0x7f187c0b8350 2024-03-23T09:41:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.021+0000 7f188200b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0cfe10 con 0x7f187c0cde10 2024-03-23T09:41:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.021+0000 7f187b7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f187c15c990 0x7f187c0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33150/0 (socket says 172.21.15.47:33150) 2024-03-23T09:41:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.021+0000 7f187b7fe640 1 -- 172.21.15.47:0/3036251488 learned_addr learned my addr 172.21.15.47:0/3036251488 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.021+0000 7f187affd640 1 --2- 172.21.15.47:0/3036251488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f187c0cde10 0x7f187c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.021+0000 7f187bfff640 1 --2- 172.21.15.47:0/3036251488 >> v2:172.21.15.175:3300/0 conn(0x7f187c0b8350 0x7f187c0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.021+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103412676 0 0) 0x7f187c161630 con 0x7f187c15c990 2024-03-23T09:41:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.021+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1864003520 con 0x7f187c15c990 2024-03-23T09:41:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 951937195 0 0) 0x7f1864003520 con 0x7f187c15c990 2024-03-23T09:41:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f187c0cde10 msgr2=0x7f187c160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 --2- 172.21.15.47:0/3036251488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f187c0cde10 0x7f187c160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 >> v2:172.21.15.175:3300/0 conn(0x7f187c0b8350 msgr2=0x7f187c0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 --2- 172.21.15.47:0/3036251488 >> v2:172.21.15.175:3300/0 conn(0x7f187c0b8350 0x7f187c0d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c0b5b60 con 0x7f187c15c990 2024-03-23T09:41:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f1870001670 con 0x7f187c15c990 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 >> v1:172.21.15.47:6789/0 conn(0x7f187c15c990 legacy=0x7f187c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 --2- 172.21.15.47:0/3036251488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1864003970 0x7f1864003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 --2- 172.21.15.47:0/3036251488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18640047c0 0x7f1864004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 --2- 172.21.15.47:0/3036251488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1864005180 0x7f186400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0d1030 con 0x7f1864005180 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0cfe10 con 0x7f1864003970 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186400bfa0 con 0x7f18640047c0 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187affd640 1 --2- 172.21.15.47:0/3036251488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f187c0cde10 0x7f187c160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187bfff640 1 --2- 172.21.15.47:0/3036251488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1864005180 0x7f186400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187b7fe640 1 --2- 172.21.15.47:0/3036251488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1864003970 0x7f1864003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187affd640 1 --2- 172.21.15.47:0/3036251488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18640047c0 0x7f1864004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187bfff640 1 -- 172.21.15.47:0/3036251488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18640047c0 msgr2=0x7f1864004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187bfff640 1 --2- 172.21.15.47:0/3036251488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18640047c0 0x7f1864004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187bfff640 1 -- 172.21.15.47:0/3036251488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1864003970 msgr2=0x7f1864003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187bfff640 1 --2- 172.21.15.47:0/3036251488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1864003970 0x7f1864003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.022+0000 7f187bfff640 1 -- 172.21.15.47:0/3036251488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1864003520 con 0x7f1864005180 2024-03-23T09:41:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.023+0000 7f187bfff640 1 --2- 172.21.15.47:0/3036251488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1864005180 0x7f186400b980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1874009fc0 tx=0x7f18640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5daf42b9c60b38 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.023+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187402cc60 con 0x7f1864005180 2024-03-23T09:41:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.023+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187400f040 con 0x7f1864005180 2024-03-23T09:41:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.023+0000 7f187a7fc640 1 -- 172.21.15.47:0/3036251488 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874034440 con 0x7f1864005180 2024-03-23T09:41:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.023+0000 7f188200b640 1 -- 172.21.15.47:0/3036251488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1864005180 msgr2=0x7f186400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.023+0000 7f188200b640 1 --2- 172.21.15.47:0/3036251488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1864005180 0x7f186400b980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1874009fc0 tx=0x7f18640057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.023+0000 7f187bfff640 1 -- 172.21.15.47:0/3036251488 reap_dead start 2024-03-23T09:41:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.023+0000 7f188200b640 1 -- 172.21.15.47:0/3036251488 shutdown_connections 2024-03-23T09:41:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.023+0000 7f188200b640 1 -- 172.21.15.47:0/3036251488 >> 172.21.15.47:0/3036251488 conn(0x7f187c0659d0 msgr2=0x7f187c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.024+0000 7f188200b640 1 -- 172.21.15.47:0/3036251488 shutdown_connections 2024-03-23T09:41:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.024+0000 7f188200b640 1 -- 172.21.15.47:0/3036251488 wait complete. 2024-03-23T09:41:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.025+0000 7f188200b640 1 Processor -- start 2024-03-23T09:41:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.025+0000 7f188200b640 1 -- start start 2024-03-23T09:41:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.025+0000 7f188200b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f187c1507c0 0x7f187c15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f188200b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f187c150ba0 0x7f187c0b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f188200b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f187c0b7480 0x7f187c151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f188200b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c160460 con 0x7f187c150ba0 2024-03-23T09:41:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f188200b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0b7910 con 0x7f187c0b7480 2024-03-23T09:41:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f188200b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187c15eac0 con 0x7f187c1507c0 2024-03-23T09:41:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f187affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f187c150ba0 0x7f187c0b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f187affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f187c150ba0 0x7f187c0b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41484/0 (socket says 172.21.15.47:41484) 2024-03-23T09:41:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f187affd640 1 -- 172.21.15.47:0/594669485 learned_addr learned my addr 172.21.15.47:0/594669485 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f187b7fe640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f187c1507c0 0x7f187c15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f187bfff640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f187c0b7480 0x7f187c151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f187affd640 1 -- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f187c1507c0 msgr2=0x7f187c15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f187affd640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f187c1507c0 0x7f187c15ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f187affd640 1 -- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f187c0b7480 msgr2=0x7f187c151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f187affd640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f187c0b7480 0x7f187c151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f187affd640 1 -- 172.21.15.47:0/594669485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c15ef50 con 0x7f187c150ba0 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f187affd640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f187c150ba0 0x7f187c0b6f40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f186c011830 tx=0x7f186c011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f1863fff640 1 -- 172.21.15.47:0/594669485 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c00f9c0 con 0x7f187c150ba0 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f1863fff640 1 -- 172.21.15.47:0/594669485 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186c00fb20 con 0x7f187c150ba0 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f188200b640 1 -- 172.21.15.47:0/594669485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187c1536e0 con 0x7f187c150ba0 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f1863fff640 1 -- 172.21.15.47:0/594669485 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c00fed0 con 0x7f187c150ba0 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.026+0000 7f188200b640 1 -- 172.21.15.47:0/594669485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187c15e470 con 0x7f187c150ba0 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.027+0000 7f188200b640 1 -- 172.21.15.47:0/594669485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187c072360 con 0x7f187c150ba0 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.028+0000 7f1863fff640 1 -- 172.21.15.47:0/594669485 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f186c015460 con 0x7f187c150ba0 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.028+0000 7f1863fff640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f186806ac00 0x7f186806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.028+0000 7f1863fff640 1 -- 172.21.15.47:0/594669485 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f186c089d30 con 0x7f187c150ba0 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.028+0000 7f187b7fe640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f186806ac00 0x7f186806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.029+0000 7f187b7fe640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f186806ac00 0x7f186806d0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1870004840 tx=0x7f1870009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.032+0000 7f1863fff640 1 -- 172.21.15.47:0/594669485 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f186c054f10 con 0x7f187c150ba0 2024-03-23T09:41:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.210+0000 7f188200b640 1 -- 172.21.15.47:0/594669485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f187c15ec30 con 0x7f187c150ba0 2024-03-23T09:41:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.211+0000 7f1863fff640 1 -- 172.21.15.47:0/594669485 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f186c058530 con 0x7f187c150ba0 2024-03-23T09:41:15.211 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.212+0000 7f188200b640 1 -- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f186806ac00 msgr2=0x7f186806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.212+0000 7f188200b640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f186806ac00 0x7f186806d0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1870004840 tx=0x7f1870009040 comp rx=0 tx=0).stop 2024-03-23T09:41:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.212+0000 7f188200b640 1 -- 172.21.15.47:0/594669485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f187c150ba0 msgr2=0x7f187c0b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.212+0000 7f188200b640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f187c150ba0 0x7f187c0b6f40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f186c011830 tx=0x7f186c011cf0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.213+0000 7f188200b640 1 -- 172.21.15.47:0/594669485 shutdown_connections 2024-03-23T09:41:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.213+0000 7f188200b640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f186806ac00 0x7f186806d0c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.213+0000 7f188200b640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f187c0b7480 0x7f187c151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.213+0000 7f188200b640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f187c150ba0 0x7f187c0b6f40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.213+0000 7f188200b640 1 --2- 172.21.15.47:0/594669485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f187c1507c0 0x7f187c15ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.213+0000 7f188200b640 1 -- 172.21.15.47:0/594669485 >> 172.21.15.47:0/594669485 conn(0x7f187c0659d0 msgr2=0x7f187c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.213+0000 7f188200b640 1 -- 172.21.15.47:0/594669485 shutdown_connections 2024-03-23T09:41:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.213+0000 7f188200b640 1 -- 172.21.15.47:0/594669485 wait complete. 2024-03-23T09:41:15.223 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:41:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.323+0000 7f8054da4640 1 Processor -- start 2024-03-23T09:41:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.323+0000 7f8054da4640 1 -- start start 2024-03-23T09:41:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f8054da4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f805015ca90 0x7f8050160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f8054da4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80500cde10 0x7f80500d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f8054da4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80500d11e0 con 0x7f80500b8350 2024-03-23T09:41:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f8054da4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8050160f60 con 0x7f805015ca90 2024-03-23T09:41:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f8054da4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8050161240 con 0x7f80500cde10 2024-03-23T09:41:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f804ed9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80500cde10 0x7f80500d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f804e59c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f805015ca90 0x7f8050160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f804ed9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80500cde10 0x7f80500d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:44788/0 (socket says 172.21.15.47:44788) 2024-03-23T09:41:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f804ed9d640 1 -- 172.21.15.47:0/3786338777 learned_addr learned my addr 172.21.15.47:0/3786338777 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f804d59a640 1 -- 172.21.15.47:0/3786338777 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941771461 0 0) 0x7f80500d11e0 con 0x7f80500b8350 2024-03-23T09:41:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f804ed9d640 1 -- 172.21.15.47:0/3786338777 >> v2:172.21.15.175:3300/0 conn(0x7f805015ca90 msgr2=0x7f8050160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f804ed9d640 1 --2- 172.21.15.47:0/3786338777 >> v2:172.21.15.175:3300/0 conn(0x7f805015ca90 0x7f8050160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f804ed9d640 1 -- 172.21.15.47:0/3786338777 >> v1:172.21.15.175:6789/0 conn(0x7f80500b8350 legacy=0x7f80500cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.324+0000 7f804ed9d640 1 -- 172.21.15.47:0/3786338777 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8050161520 con 0x7f80500cde10 2024-03-23T09:41:15.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.325+0000 7f804ed9d640 1 --2- 172.21.15.47:0/3786338777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80500cde10 0x7f80500d1670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8040009e50 tx=0x7f804002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f85ece184ee1ed7c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:15.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.325+0000 7f804d59a640 1 -- 172.21.15.47:0/3786338777 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804002ca70 con 0x7f80500cde10 2024-03-23T09:41:15.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.325+0000 7f804d59a640 1 -- 172.21.15.47:0/3786338777 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804002cbd0 con 0x7f80500cde10 2024-03-23T09:41:15.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.325+0000 7f804d59a640 1 -- 172.21.15.47:0/3786338777 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400343f0 con 0x7f80500cde10 2024-03-23T09:41:15.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.326+0000 7f8054da4640 1 -- 172.21.15.47:0/3786338777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80500cde10 msgr2=0x7f80500d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.326+0000 7f8054da4640 1 --2- 172.21.15.47:0/3786338777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80500cde10 0x7f80500d1670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8040009e50 tx=0x7f804002bed0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.326+0000 7f8054da4640 1 -- 172.21.15.47:0/3786338777 shutdown_connections 2024-03-23T09:41:15.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.326+0000 7f8054da4640 1 --2- 172.21.15.47:0/3786338777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80500cde10 0x7f80500d1670 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.326+0000 7f8054da4640 1 --2- 172.21.15.47:0/3786338777 >> v2:172.21.15.175:3300/0 conn(0x7f805015ca90 0x7f8050160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.326+0000 7f8054da4640 1 -- 172.21.15.47:0/3786338777 >> 172.21.15.47:0/3786338777 conn(0x7f80500659d0 msgr2=0x7f8050065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:15.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.326+0000 7f8054da4640 1 -- 172.21.15.47:0/3786338777 shutdown_connections 2024-03-23T09:41:15.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.326+0000 7f8054da4640 1 -- 172.21.15.47:0/3786338777 wait complete. 2024-03-23T09:41:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.327+0000 7f8054da4640 1 Processor -- start 2024-03-23T09:41:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.327+0000 7f8054da4640 1 -- start start 2024-03-23T09:41:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f8054da4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80500b8350 0x7f80501533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f8054da4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80500cde10 0x7f805015f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f8054da4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f805015ca90 0x7f805015f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f8054da4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80500b4ba0 con 0x7f80500cde10 2024-03-23T09:41:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f8054da4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80500b3f90 con 0x7f805015ca90 2024-03-23T09:41:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f8054da4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80500b4270 con 0x7f80500b8350 2024-03-23T09:41:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f804dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80500cde10 0x7f805015f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f804dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80500cde10 0x7f805015f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41496/0 (socket says 172.21.15.47:41496) 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f804dd9b640 1 -- 172.21.15.47:0/3005383600 learned_addr learned my addr 172.21.15.47:0/3005383600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f804e59c640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80500b8350 0x7f80501533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f804ed9d640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f805015ca90 0x7f805015f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f804dd9b640 1 -- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80500b8350 msgr2=0x7f80501533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f804dd9b640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80500b8350 0x7f80501533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f804dd9b640 1 -- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f805015ca90 msgr2=0x7f805015f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f804dd9b640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f805015ca90 0x7f805015f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.328+0000 7f804dd9b640 1 -- 172.21.15.47:0/3005383600 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040009b30 con 0x7f80500cde10 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.329+0000 7f804dd9b640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80500cde10 0x7f805015f390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f804400c410 tx=0x7f804400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.329+0000 7f803effd640 1 -- 172.21.15.47:0/3005383600 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80440035f0 con 0x7f80500cde10 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.329+0000 7f803effd640 1 -- 172.21.15.47:0/3005383600 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8044003770 con 0x7f80500cde10 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.329+0000 7f8054da4640 1 -- 172.21.15.47:0/3005383600 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80500b4550 con 0x7f80500cde10 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.329+0000 7f803effd640 1 -- 172.21.15.47:0/3005383600 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8044003b20 con 0x7f80500cde10 2024-03-23T09:41:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.329+0000 7f8054da4640 1 -- 172.21.15.47:0/3005383600 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80500b5030 con 0x7f80500cde10 2024-03-23T09:41:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.330+0000 7f8054da4640 1 -- 172.21.15.47:0/3005383600 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8050072360 con 0x7f80500cde10 2024-03-23T09:41:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.330+0000 7f803effd640 1 -- 172.21.15.47:0/3005383600 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8044003c80 con 0x7f80500cde10 2024-03-23T09:41:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.331+0000 7f803effd640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f803006ac00 0x7f803006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.331+0000 7f803effd640 1 -- 172.21.15.47:0/3005383600 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80440888f0 con 0x7f80500cde10 2024-03-23T09:41:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.331+0000 7f804e59c640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f803006ac00 0x7f803006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.331+0000 7f804e59c640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f803006ac00 0x7f803006d0c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8048009b40 tx=0x7f8048009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.335+0000 7f803effd640 1 -- 172.21.15.47:0/3005383600 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8044053b70 con 0x7f80500cde10 2024-03-23T09:41:15.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.550+0000 7f8054da4640 1 -- 172.21.15.47:0/3005383600 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8050066be0 con 0x7f80500cde10 2024-03-23T09:41:15.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.551+0000 7f803effd640 1 -- 172.21.15.47:0/3005383600 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4734 (secure 0 0 0) 0x7f8044017020 con 0x7f80500cde10 2024-03-23T09:41:15.550 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:15.551 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1970049669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1970049669},{"type":"v1","addr":"172.21.15.47:6833","nonce":1970049669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4162785911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4162785911},{"type":"v1","addr":"172.21.15.47:6835","nonce":4162785911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1036174787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1036174787},{"type":"v1","addr":"172.21.15.175:6836","nonce":1036174787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:03.137957+0000","modified":"2024-03-23T09:41:05.147528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1633245139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1633245139},{"type":"v1","addr":"172.21.15.175:6837","nonce":1633245139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T09:41:15.551 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2024-03-23T09:41:15.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.552+0000 7f8054da4640 1 -- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f803006ac00 msgr2=0x7f803006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.552+0000 7f8054da4640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f803006ac00 0x7f803006d0c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8048009b40 tx=0x7f8048009190 comp rx=0 tx=0).stop 2024-03-23T09:41:15.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.552+0000 7f8054da4640 1 -- 172.21.15.47:0/3005383600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80500cde10 msgr2=0x7f805015f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.552+0000 7f8054da4640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80500cde10 0x7f805015f390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f804400c410 tx=0x7f804400c8d0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.552+0000 7f8054da4640 1 -- 172.21.15.47:0/3005383600 shutdown_connections 2024-03-23T09:41:15.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.552+0000 7f8054da4640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f803006ac00 0x7f803006d0c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.552+0000 7f8054da4640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f805015ca90 0x7f805015f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.552+0000 7f8054da4640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80500cde10 0x7f805015f390 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.552+0000 7f8054da4640 1 --2- 172.21.15.47:0/3005383600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80500b8350 0x7f80501533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.552+0000 7f8054da4640 1 -- 172.21.15.47:0/3005383600 >> 172.21.15.47:0/3005383600 conn(0x7f80500659d0 msgr2=0x7f8050156520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:15.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.552+0000 7f8054da4640 1 -- 172.21.15.47:0/3005383600 shutdown_connections 2024-03-23T09:41:15.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.553+0000 7f8054da4640 1 -- 172.21.15.47:0/3005383600 wait complete. 2024-03-23T09:41:15.563 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.663+0000 7f6869241640 1 Processor -- start 2024-03-23T09:41:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.663+0000 7f6869241640 1 -- start start 2024-03-23T09:41:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.664+0000 7f6869241640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f68640b8350 0x7f68640cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.664+0000 7f6869241640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68640cdc40 0x7f68640d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.664+0000 7f6869241640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68640d11e0 con 0x7f686415ca90 2024-03-23T09:41:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.664+0000 7f6869241640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68640ce0d0 con 0x7f68640b8350 2024-03-23T09:41:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.664+0000 7f6869241640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6864160f60 con 0x7f68640cdc40 2024-03-23T09:41:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.664+0000 7f686259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f68640b8350 0x7f68640cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.664+0000 7f686259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f68640b8350 0x7f68640cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41504/0 (socket says 172.21.15.47:41504) 2024-03-23T09:41:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.664+0000 7f686259c640 1 -- 172.21.15.47:0/260481165 learned_addr learned my addr 172.21.15.47:0/260481165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.664+0000 7f686359e640 1 --2- 172.21.15.47:0/260481165 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68640cdc40 0x7f68640d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.664+0000 7f686259c640 1 -- 172.21.15.47:0/260481165 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68640cdc40 msgr2=0x7f68640d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.664+0000 7f686259c640 1 --2- 172.21.15.47:0/260481165 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68640cdc40 0x7f68640d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.664+0000 7f686259c640 1 -- 172.21.15.47:0/260481165 >> v1:172.21.15.175:6789/0 conn(0x7f686415ca90 legacy=0x7f6864160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.664+0000 7f686259c640 1 -- 172.21.15.47:0/260481165 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6864161240 con 0x7f68640b8350 2024-03-23T09:41:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.665+0000 7f686259c640 1 --2- 172.21.15.47:0/260481165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68640b8350 0x7f68640cd700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f684c009870 tx=0x7f684c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e1ebaed6e96c42 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.665+0000 7f6861d9b640 1 -- 172.21.15.47:0/260481165 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c0317c0 con 0x7f68640b8350 2024-03-23T09:41:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.665+0000 7f6861d9b640 1 -- 172.21.15.47:0/260481165 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c031920 con 0x7f68640b8350 2024-03-23T09:41:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.665+0000 7f6861d9b640 1 -- 172.21.15.47:0/260481165 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c031cb0 con 0x7f68640b8350 2024-03-23T09:41:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.666+0000 7f6869241640 1 -- 172.21.15.47:0/260481165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68640b8350 msgr2=0x7f68640cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.666+0000 7f6869241640 1 --2- 172.21.15.47:0/260481165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68640b8350 0x7f68640cd700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f684c009870 tx=0x7f684c02b860 comp rx=0 tx=0).stop 2024-03-23T09:41:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.666+0000 7f6869241640 1 -- 172.21.15.47:0/260481165 shutdown_connections 2024-03-23T09:41:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.666+0000 7f6869241640 1 --2- 172.21.15.47:0/260481165 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68640cdc40 0x7f68640d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.666+0000 7f6869241640 1 --2- 172.21.15.47:0/260481165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68640b8350 0x7f68640cd700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.666+0000 7f6869241640 1 -- 172.21.15.47:0/260481165 >> 172.21.15.47:0/260481165 conn(0x7f68640659d0 msgr2=0x7f6864065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:15.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.666+0000 7f6869241640 1 -- 172.21.15.47:0/260481165 shutdown_connections 2024-03-23T09:41:15.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.666+0000 7f6869241640 1 -- 172.21.15.47:0/260481165 wait complete. 2024-03-23T09:41:15.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.667+0000 7f6869241640 1 Processor -- start 2024-03-23T09:41:15.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.667+0000 7f6869241640 1 -- start start 2024-03-23T09:41:15.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f6869241640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68640b8350 0x7f68640c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f6869241640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68640cdc40 0x7f68640b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f6869241640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f686415ca90 0x7f68640b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f6869241640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68640c9370 con 0x7f68640cdc40 2024-03-23T09:41:15.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f6869241640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68640b2420 con 0x7f68640b8350 2024-03-23T09:41:15.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f6869241640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68640b3430 con 0x7f686415ca90 2024-03-23T09:41:15.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f686259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68640cdc40 0x7f68640b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f686259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68640cdc40 0x7f68640b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41508/0 (socket says 172.21.15.47:41508) 2024-03-23T09:41:15.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f686259c640 1 -- 172.21.15.47:0/3972753036 learned_addr learned my addr 172.21.15.47:0/3972753036 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:15.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f686359e640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f686415ca90 0x7f68640b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f6862d9d640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68640b8350 0x7f68640c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f686259c640 1 -- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f686415ca90 msgr2=0x7f68640b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f686259c640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f686415ca90 0x7f68640b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f686259c640 1 -- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68640b8350 msgr2=0x7f68640c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f686259c640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68640b8350 0x7f68640c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f686259c640 1 -- 172.21.15.47:0/3972753036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c0094d0 con 0x7f68640cdc40 2024-03-23T09:41:15.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.668+0000 7f686259c640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68640cdc40 0x7f68640b4050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f684c000c00 tx=0x7f684c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:15.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.669+0000 7f685b7fe640 1 -- 172.21.15.47:0/3972753036 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c0329a0 con 0x7f68640cdc40 2024-03-23T09:41:15.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.669+0000 7f685b7fe640 1 -- 172.21.15.47:0/3972753036 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c032b20 con 0x7f68640cdc40 2024-03-23T09:41:15.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.669+0000 7f6869241640 1 -- 172.21.15.47:0/3972753036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68640b2670 con 0x7f68640cdc40 2024-03-23T09:41:15.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.669+0000 7f685b7fe640 1 -- 172.21.15.47:0/3972753036 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c0453f0 con 0x7f68640cdc40 2024-03-23T09:41:15.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.669+0000 7f6869241640 1 -- 172.21.15.47:0/3972753036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68640b2c30 con 0x7f68640cdc40 2024-03-23T09:41:15.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.670+0000 7f6869241640 1 -- 172.21.15.47:0/3972753036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6864072360 con 0x7f68640cdc40 2024-03-23T09:41:15.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.670+0000 7f685b7fe640 1 -- 172.21.15.47:0/3972753036 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f684c04e450 con 0x7f68640cdc40 2024-03-23T09:41:15.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.670+0000 7f685b7fe640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f684406ac00 0x7f684406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.671+0000 7f685b7fe640 1 -- 172.21.15.47:0/3972753036 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f684c0ab6f0 con 0x7f68640cdc40 2024-03-23T09:41:15.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.671+0000 7f6862d9d640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f684406ac00 0x7f684406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.671+0000 7f6862d9d640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f684406ac00 0x7f684406d0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6864160b70 tx=0x7f6854009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:15.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.675+0000 7f685b7fe640 1 -- 172.21.15.47:0/3972753036 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f684c0768d0 con 0x7f68640cdc40 2024-03-23T09:41:15.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.858+0000 7f6869241640 1 -- 172.21.15.47:0/3972753036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6864066be0 con 0x7f68640cdc40 2024-03-23T09:41:15.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.859+0000 7f685b7fe640 1 -- 172.21.15.47:0/3972753036 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f684c079ef0 con 0x7f68640cdc40 2024-03-23T09:41:15.859 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:15.859 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:15.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.861+0000 7f6869241640 1 -- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f684406ac00 msgr2=0x7f684406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.861+0000 7f6869241640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f684406ac00 0x7f684406d0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6864160b70 tx=0x7f6854009040 comp rx=0 tx=0).stop 2024-03-23T09:41:15.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.861+0000 7f6869241640 1 -- 172.21.15.47:0/3972753036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68640cdc40 msgr2=0x7f68640b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.861+0000 7f6869241640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68640cdc40 0x7f68640b4050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f684c000c00 tx=0x7f684c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.861+0000 7f6869241640 1 -- 172.21.15.47:0/3972753036 shutdown_connections 2024-03-23T09:41:15.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.861+0000 7f6869241640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f684406ac00 0x7f684406d0c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.861+0000 7f6869241640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f686415ca90 0x7f68640b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.861+0000 7f6869241640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68640cdc40 0x7f68640b4050 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.861+0000 7f6869241640 1 --2- 172.21.15.47:0/3972753036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68640b8350 0x7f68640c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.861+0000 7f6869241640 1 -- 172.21.15.47:0/3972753036 >> 172.21.15.47:0/3972753036 conn(0x7f68640659d0 msgr2=0x7f686415e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:15.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.861+0000 7f6869241640 1 -- 172.21.15.47:0/3972753036 shutdown_connections 2024-03-23T09:41:15.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.861+0000 7f6869241640 1 -- 172.21.15.47:0/3972753036 wait complete. 2024-03-23T09:41:15.872 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:15.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:15.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.975+0000 7f5247077640 1 Processor -- start 2024-03-23T09:41:15.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.976+0000 7f5247077640 1 -- start start 2024-03-23T09:41:15.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.976+0000 7f5247077640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f52400cd090 0x7f52400cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.976+0000 7f5247077640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f524015ab50 0x7f52400cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.976+0000 7f5247077640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52400cd9b0 con 0x7f52400cb6f0 2024-03-23T09:41:15.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.976+0000 7f5247077640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52400cbf50 con 0x7f52400cd090 2024-03-23T09:41:15.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.976+0000 7f5247077640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52400cc230 con 0x7f524015ab50 2024-03-23T09:41:15.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.976+0000 7f5244e13640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f52400cd090 0x7f52400cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.976+0000 7f5244e13640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f52400cd090 0x7f52400cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41522/0 (socket says 172.21.15.47:41522) 2024-03-23T09:41:15.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.976+0000 7f5244e13640 1 -- 172.21.15.47:0/3762551254 learned_addr learned my addr 172.21.15.47:0/3762551254 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:15.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.977+0000 7f523ffff640 1 --2- 172.21.15.47:0/3762551254 >> v2:172.21.15.175:3300/0 conn(0x7f524015ab50 0x7f52400cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.977+0000 7f5244e13640 1 -- 172.21.15.47:0/3762551254 >> v2:172.21.15.175:3300/0 conn(0x7f524015ab50 msgr2=0x7f52400cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:15.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.977+0000 7f5244e13640 1 --2- 172.21.15.47:0/3762551254 >> v2:172.21.15.175:3300/0 conn(0x7f524015ab50 0x7f52400cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.977+0000 7f5244e13640 1 -- 172.21.15.47:0/3762551254 >> v1:172.21.15.175:6789/0 conn(0x7f52400cb6f0 legacy=0x7f5240098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.977+0000 7f5244e13640 1 -- 172.21.15.47:0/3762551254 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52400cc510 con 0x7f52400cd090 2024-03-23T09:41:15.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.977+0000 7f5244e13640 1 --2- 172.21.15.47:0/3762551254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52400cd090 0x7f52400cd470 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f52400cdd10 tx=0x7f523402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de40042e00a833c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:15.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.978+0000 7f523f7fe640 1 -- 172.21.15.47:0/3762551254 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523402f850 con 0x7f52400cd090 2024-03-23T09:41:15.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.978+0000 7f523f7fe640 1 -- 172.21.15.47:0/3762551254 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523402f9b0 con 0x7f52400cd090 2024-03-23T09:41:15.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.978+0000 7f523f7fe640 1 -- 172.21.15.47:0/3762551254 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523402fd40 con 0x7f52400cd090 2024-03-23T09:41:15.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.978+0000 7f5247077640 1 -- 172.21.15.47:0/3762551254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52400cd090 msgr2=0x7f52400cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.978+0000 7f5247077640 1 --2- 172.21.15.47:0/3762551254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52400cd090 0x7f52400cd470 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f52400cdd10 tx=0x7f523402ba30 comp rx=0 tx=0).stop 2024-03-23T09:41:15.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.978+0000 7f5247077640 1 -- 172.21.15.47:0/3762551254 shutdown_connections 2024-03-23T09:41:15.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.978+0000 7f5247077640 1 --2- 172.21.15.47:0/3762551254 >> v2:172.21.15.175:3300/0 conn(0x7f524015ab50 0x7f52400cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.978+0000 7f5247077640 1 --2- 172.21.15.47:0/3762551254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52400cd090 0x7f52400cd470 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.978+0000 7f5247077640 1 -- 172.21.15.47:0/3762551254 >> 172.21.15.47:0/3762551254 conn(0x7f52400d06a0 msgr2=0x7f52400d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:15.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.978+0000 7f5247077640 1 -- 172.21.15.47:0/3762551254 shutdown_connections 2024-03-23T09:41:15.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.978+0000 7f5247077640 1 -- 172.21.15.47:0/3762551254 wait complete. 2024-03-23T09:41:15.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.979+0000 7f5247077640 1 Processor -- start 2024-03-23T09:41:15.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.979+0000 7f5247077640 1 -- start start 2024-03-23T09:41:15.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.980+0000 7f5247077640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52400cb6f0 0x7f52400c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.980+0000 7f5247077640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52400cd090 0x7f52400c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.980+0000 7f5247077640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524015ab50 0x7f5240140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.980+0000 7f5247077640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52400c9b50 con 0x7f52400cd090 2024-03-23T09:41:15.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.980+0000 7f5247077640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52400c7cb0 con 0x7f524015ab50 2024-03-23T09:41:15.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.980+0000 7f5247077640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52401418f0 con 0x7f52400cb6f0 2024-03-23T09:41:15.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.980+0000 7f523ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52400cd090 0x7f52400c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.980+0000 7f523ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52400cd090 0x7f52400c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41528/0 (socket says 172.21.15.47:41528) 2024-03-23T09:41:15.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.980+0000 7f523ffff640 1 -- 172.21.15.47:0/1141548654 learned_addr learned my addr 172.21.15.47:0/1141548654 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f5244e13640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52400cb6f0 0x7f52400c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f5245614640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524015ab50 0x7f5240140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f523ffff640 1 -- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52400cb6f0 msgr2=0x7f52400c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f523ffff640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52400cb6f0 0x7f52400c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f523ffff640 1 -- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524015ab50 msgr2=0x7f5240140d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f523ffff640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524015ab50 0x7f5240140d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f523ffff640 1 -- 172.21.15.47:0/1141548654 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52340094d0 con 0x7f52400cd090 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f5245614640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524015ab50 0x7f5240140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f5244e13640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52400cb6f0 0x7f52400c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f523ffff640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52400cd090 0x7f52400c9f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f522c00e840 tx=0x7f522c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f523d7fa640 1 -- 172.21.15.47:0/1141548654 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c0094f0 con 0x7f52400cd090 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f523d7fa640 1 -- 172.21.15.47:0/1141548654 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c00c940 con 0x7f52400cd090 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f5247077640 1 -- 172.21.15.47:0/1141548654 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52400c9490 con 0x7f52400cd090 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f523d7fa640 1 -- 172.21.15.47:0/1141548654 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c00ccd0 con 0x7f52400cd090 2024-03-23T09:41:15.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.981+0000 7f5247077640 1 -- 172.21.15.47:0/1141548654 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52400c9140 con 0x7f52400cd090 2024-03-23T09:41:15.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.982+0000 7f5247077640 1 -- 172.21.15.47:0/1141548654 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52400672f0 con 0x7f52400cd090 2024-03-23T09:41:15.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.982+0000 7f523d7fa640 1 -- 172.21.15.47:0/1141548654 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f522c013470 con 0x7f52400cd090 2024-03-23T09:41:15.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.983+0000 7f523d7fa640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f522406ac00 0x7f522406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:15.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.983+0000 7f523d7fa640 1 -- 172.21.15.47:0/1141548654 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f522c089370 con 0x7f52400cd090 2024-03-23T09:41:15.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.983+0000 7f5244e13640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f522406ac00 0x7f522406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:15.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.984+0000 7f5244e13640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f522406ac00 0x7f522406d0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5234036000 tx=0x7f5234039040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:15.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:15.987+0000 7f523d7fa640 1 -- 172.21.15.47:0/1141548654 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f522c0545e0 con 0x7f52400cd090 2024-03-23T09:41:16.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.165+0000 7f5247077640 1 -- 172.21.15.47:0/1141548654 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5240159c40 con 0x7f52400cd090 2024-03-23T09:41:16.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.165+0000 7f523d7fa640 1 -- 172.21.15.47:0/1141548654 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f522c057c00 con 0x7f52400cd090 2024-03-23T09:41:16.165 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:16.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.167+0000 7f5247077640 1 -- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f522406ac00 msgr2=0x7f522406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.167+0000 7f5247077640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f522406ac00 0x7f522406d0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5234036000 tx=0x7f5234039040 comp rx=0 tx=0).stop 2024-03-23T09:41:16.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.167+0000 7f5247077640 1 -- 172.21.15.47:0/1141548654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52400cd090 msgr2=0x7f52400c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.167+0000 7f5247077640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52400cd090 0x7f52400c9f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f522c00e840 tx=0x7f522c00ed00 comp rx=0 tx=0).stop 2024-03-23T09:41:16.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.167+0000 7f5247077640 1 -- 172.21.15.47:0/1141548654 shutdown_connections 2024-03-23T09:41:16.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.167+0000 7f5247077640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f522406ac00 0x7f522406d0c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.167+0000 7f5247077640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524015ab50 0x7f5240140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.167+0000 7f5247077640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52400cd090 0x7f52400c9f70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.167+0000 7f5247077640 1 --2- 172.21.15.47:0/1141548654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52400cb6f0 0x7f52400c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.167+0000 7f5247077640 1 -- 172.21.15.47:0/1141548654 >> 172.21.15.47:0/1141548654 conn(0x7f52400d06a0 msgr2=0x7f524006f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:16.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.167+0000 7f5247077640 1 -- 172.21.15.47:0/1141548654 shutdown_connections 2024-03-23T09:41:16.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.167+0000 7f5247077640 1 -- 172.21.15.47:0/1141548654 wait complete. 2024-03-23T09:41:16.178 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:16.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.282+0000 7fb13f1a2640 1 Processor -- start 2024-03-23T09:41:16.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.282+0000 7fb13f1a2640 1 -- start start 2024-03-23T09:41:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13f1a2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb13815c860 0x7fb1380d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13f1a2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1381611a0 0x7fb1380cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13f1a2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1380ce6e0 con 0x7fb1380ce260 2024-03-23T09:41:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13f1a2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1380d0d10 con 0x7fb13815c860 2024-03-23T09:41:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13f1a2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1380d0370 con 0x7fb1381611a0 2024-03-23T09:41:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13e1a0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb13815c860 0x7fb1380d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13e9a1640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb1380ce260 0x7fb1380d2460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33152/0 (socket says 172.21.15.47:33152) 2024-03-23T09:41:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13e1a0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb13815c860 0x7fb1380d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41544/0 (socket says 172.21.15.47:41544) 2024-03-23T09:41:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13e9a1640 1 -- 172.21.15.47:0/1601940085 learned_addr learned my addr 172.21.15.47:0/1601940085 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:16.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13e1a0640 1 -- 172.21.15.47:0/1601940085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1381611a0 msgr2=0x7fb1380cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:16.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13e1a0640 1 --2- 172.21.15.47:0/1601940085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1381611a0 0x7fb1380cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13e1a0640 1 -- 172.21.15.47:0/1601940085 >> v1:172.21.15.47:6789/0 conn(0x7fb1380ce260 legacy=0x7fb1380d2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13e1a0640 1 -- 172.21.15.47:0/1601940085 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1380d0650 con 0x7fb13815c860 2024-03-23T09:41:16.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.283+0000 7fb13d19e640 1 -- 172.21.15.47:0/1601940085 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409065549 0 0) 0x7fb1380ce6e0 con 0x7fb1380ce260 2024-03-23T09:41:16.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.284+0000 7fb13e1a0640 1 --2- 172.21.15.47:0/1601940085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb13815c860 0x7fb1380d1130 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb1340097a0 tx=0x7fb13402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e3199c81c22964 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:16.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.284+0000 7fb13d19e640 1 -- 172.21.15.47:0/1601940085 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13402e070 con 0x7fb13815c860 2024-03-23T09:41:16.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.284+0000 7fb13d19e640 1 -- 172.21.15.47:0/1601940085 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1340315a0 con 0x7fb13815c860 2024-03-23T09:41:16.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.284+0000 7fb13d19e640 1 -- 172.21.15.47:0/1601940085 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134031930 con 0x7fb13815c860 2024-03-23T09:41:16.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.284+0000 7fb13f1a2640 1 -- 172.21.15.47:0/1601940085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb13815c860 msgr2=0x7fb1380d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.284+0000 7fb13f1a2640 1 --2- 172.21.15.47:0/1601940085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb13815c860 0x7fb1380d1130 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb1340097a0 tx=0x7fb13402b840 comp rx=0 tx=0).stop 2024-03-23T09:41:16.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.284+0000 7fb13f1a2640 1 -- 172.21.15.47:0/1601940085 shutdown_connections 2024-03-23T09:41:16.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.284+0000 7fb13f1a2640 1 --2- 172.21.15.47:0/1601940085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1381611a0 0x7fb1380cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.284+0000 7fb13f1a2640 1 --2- 172.21.15.47:0/1601940085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb13815c860 0x7fb1380d1130 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.285+0000 7fb13f1a2640 1 -- 172.21.15.47:0/1601940085 >> 172.21.15.47:0/1601940085 conn(0x7fb13809b9e0 msgr2=0x7fb138089f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:16.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.285+0000 7fb13f1a2640 1 -- 172.21.15.47:0/1601940085 shutdown_connections 2024-03-23T09:41:16.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.285+0000 7fb13f1a2640 1 -- 172.21.15.47:0/1601940085 wait complete. 2024-03-23T09:41:16.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.286+0000 7fb13f1a2640 1 Processor -- start 2024-03-23T09:41:16.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.286+0000 7fb13f1a2640 1 -- start start 2024-03-23T09:41:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.286+0000 7fb13f1a2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1380ce260 0x7fb13815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.286+0000 7fb13f1a2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13815c860 0x7fb138149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.286+0000 7fb13f1a2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1381611a0 0x7fb138149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.286+0000 7fb13f1a2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb138149d80 con 0x7fb1380ce260 2024-03-23T09:41:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.286+0000 7fb13f1a2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb138149690 con 0x7fb13815c860 2024-03-23T09:41:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.286+0000 7fb13f1a2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13815d610 con 0x7fb1381611a0 2024-03-23T09:41:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.286+0000 7fb13e1a0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1380ce260 0x7fb13815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb13e1a0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1380ce260 0x7fb13815cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41556/0 (socket says 172.21.15.47:41556) 2024-03-23T09:41:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb13e1a0640 1 -- 172.21.15.47:0/3704035489 learned_addr learned my addr 172.21.15.47:0/3704035489 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb13e9a1640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1381611a0 0x7fb138149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb13d99f640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13815c860 0x7fb138149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb13e1a0640 1 -- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1381611a0 msgr2=0x7fb138149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb13e1a0640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1381611a0 0x7fb138149150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb13e1a0640 1 -- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13815c860 msgr2=0x7fb138149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb13e1a0640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13815c860 0x7fb138149840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb13e1a0640 1 -- 172.21.15.47:0/3704035489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb134009400 con 0x7fb1380ce260 2024-03-23T09:41:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb13e1a0640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1380ce260 0x7fb13815cfa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb134034040 tx=0x7fb1340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb12effd640 1 -- 172.21.15.47:0/3704035489 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134032900 con 0x7fb1380ce260 2024-03-23T09:41:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb12effd640 1 -- 172.21.15.47:0/3704035489 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb134032a60 con 0x7fb1380ce260 2024-03-23T09:41:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb13f1a2640 1 -- 172.21.15.47:0/3704035489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1380d17f0 con 0x7fb1380ce260 2024-03-23T09:41:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb12effd640 1 -- 172.21.15.47:0/3704035489 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134032e10 con 0x7fb1380ce260 2024-03-23T09:41:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.287+0000 7fb13f1a2640 1 -- 172.21.15.47:0/3704035489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1380cb760 con 0x7fb1380ce260 2024-03-23T09:41:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.288+0000 7fb13f1a2640 1 -- 172.21.15.47:0/3704035489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb138073ef0 con 0x7fb1380ce260 2024-03-23T09:41:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.289+0000 7fb12effd640 1 -- 172.21.15.47:0/3704035489 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb13403b3f0 con 0x7fb1380ce260 2024-03-23T09:41:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.289+0000 7fb12effd640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb11c06ac00 0x7fb11c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.289+0000 7fb12effd640 1 -- 172.21.15.47:0/3704035489 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1340ab420 con 0x7fb1380ce260 2024-03-23T09:41:16.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.289+0000 7fb13d99f640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb11c06ac00 0x7fb11c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.290+0000 7fb13d99f640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb11c06ac00 0x7fb11c06d0c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb1380ce8d0 tx=0x7fb128008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:16.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.293+0000 7fb12effd640 1 -- 172.21.15.47:0/3704035489 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb134076600 con 0x7fb1380ce260 2024-03-23T09:41:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.470+0000 7fb13f1a2640 1 -- 172.21.15.47:0/3704035489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1380b2280 con 0x7fb1380ce260 2024-03-23T09:41:16.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.470+0000 7fb12effd640 1 -- 172.21.15.47:0/3704035489 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb134079c20 con 0x7fb1380ce260 2024-03-23T09:41:16.470 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:16.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.472+0000 7fb13f1a2640 1 -- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb11c06ac00 msgr2=0x7fb11c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.472+0000 7fb13f1a2640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb11c06ac00 0x7fb11c06d0c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb1380ce8d0 tx=0x7fb128008040 comp rx=0 tx=0).stop 2024-03-23T09:41:16.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.473+0000 7fb13f1a2640 1 -- 172.21.15.47:0/3704035489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1380ce260 msgr2=0x7fb13815cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.473+0000 7fb13f1a2640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1380ce260 0x7fb13815cfa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb134034040 tx=0x7fb1340046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.473+0000 7fb13f1a2640 1 -- 172.21.15.47:0/3704035489 shutdown_connections 2024-03-23T09:41:16.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.473+0000 7fb13f1a2640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb11c06ac00 0x7fb11c06d0c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.473+0000 7fb13f1a2640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1381611a0 0x7fb138149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.473+0000 7fb13f1a2640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13815c860 0x7fb138149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.473+0000 7fb13f1a2640 1 --2- 172.21.15.47:0/3704035489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1380ce260 0x7fb13815cfa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.473+0000 7fb13f1a2640 1 -- 172.21.15.47:0/3704035489 >> 172.21.15.47:0/3704035489 conn(0x7fb13809b9e0 msgr2=0x7fb13815a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:16.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.473+0000 7fb13f1a2640 1 -- 172.21.15.47:0/3704035489 shutdown_connections 2024-03-23T09:41:16.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.473+0000 7fb13f1a2640 1 -- 172.21.15.47:0/3704035489 wait complete. 2024-03-23T09:41:16.483 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:16.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.586+0000 7ff5eb851640 1 Processor -- start 2024-03-23T09:41:16.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.586+0000 7ff5eb851640 1 -- start start 2024-03-23T09:41:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.587+0000 7ff5eb851640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff5e415c990 0x7ff5e4160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.587+0000 7ff5eb851640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff5e40cde10 0x7ff5e40d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.587+0000 7ff5eb851640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e4161630 con 0x7ff5e40b8350 2024-03-23T09:41:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.587+0000 7ff5eb851640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e40d1030 con 0x7ff5e40cde10 2024-03-23T09:41:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.587+0000 7ff5eb851640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e40cfe10 con 0x7ff5e415c990 2024-03-23T09:41:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.587+0000 7ff5e9dee640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff5e40cde10 0x7ff5e40d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.587+0000 7ff5e8dec640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff5e40b8350 0x7ff5e40cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33164/0 (socket says 172.21.15.47:33164) 2024-03-23T09:41:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.587+0000 7ff5e8dec640 1 -- 172.21.15.47:0/2545910266 learned_addr learned my addr 172.21.15.47:0/2545910266 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.588+0000 7ff5e95ed640 1 --2- 172.21.15.47:0/2545910266 >> v2:172.21.15.175:3300/0 conn(0x7ff5e415c990 0x7ff5e4160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.588+0000 7ff5dbfff640 1 -- 172.21.15.47:0/2545910266 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993397097 0 0) 0x7ff5e4161630 con 0x7ff5e40b8350 2024-03-23T09:41:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.588+0000 7ff5dbfff640 1 -- 172.21.15.47:0/2545910266 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5d0003520 con 0x7ff5e40b8350 2024-03-23T09:41:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.588+0000 7ff5e9dee640 1 -- 172.21.15.47:0/2545910266 >> v2:172.21.15.175:3300/0 conn(0x7ff5e415c990 msgr2=0x7ff5e4160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.588+0000 7ff5e9dee640 1 --2- 172.21.15.47:0/2545910266 >> v2:172.21.15.175:3300/0 conn(0x7ff5e415c990 0x7ff5e4160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.588+0000 7ff5e9dee640 1 -- 172.21.15.47:0/2545910266 >> v1:172.21.15.47:6789/0 conn(0x7ff5e40b8350 legacy=0x7ff5e40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.588+0000 7ff5e9dee640 1 -- 172.21.15.47:0/2545910266 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e40b5b60 con 0x7ff5e40cde10 2024-03-23T09:41:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.588+0000 7ff5e95ed640 1 --2- 172.21.15.47:0/2545910266 >> v2:172.21.15.175:3300/0 conn(0x7ff5e415c990 0x7ff5e4160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.588+0000 7ff5e9dee640 1 --2- 172.21.15.47:0/2545910266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e40cde10 0x7ff5e40d14c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40b8b30 tx=0x7ff5e002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829ec386fff4eb8d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.589+0000 7ff5dbfff640 1 -- 172.21.15.47:0/2545910266 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e002cbf0 con 0x7ff5e40cde10 2024-03-23T09:41:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.589+0000 7ff5dbfff640 1 -- 172.21.15.47:0/2545910266 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e002cd50 con 0x7ff5e40cde10 2024-03-23T09:41:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.589+0000 7ff5dbfff640 1 -- 172.21.15.47:0/2545910266 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0034660 con 0x7ff5e40cde10 2024-03-23T09:41:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.589+0000 7ff5eb851640 1 -- 172.21.15.47:0/2545910266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e40cde10 msgr2=0x7ff5e40d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.589+0000 7ff5eb851640 1 --2- 172.21.15.47:0/2545910266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e40cde10 0x7ff5e40d14c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40b8b30 tx=0x7ff5e002be40 comp rx=0 tx=0).stop 2024-03-23T09:41:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.589+0000 7ff5eb851640 1 -- 172.21.15.47:0/2545910266 shutdown_connections 2024-03-23T09:41:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.589+0000 7ff5eb851640 1 --2- 172.21.15.47:0/2545910266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e40cde10 0x7ff5e40d14c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.589+0000 7ff5eb851640 1 --2- 172.21.15.47:0/2545910266 >> v2:172.21.15.175:3300/0 conn(0x7ff5e415c990 0x7ff5e4160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.589+0000 7ff5eb851640 1 -- 172.21.15.47:0/2545910266 >> 172.21.15.47:0/2545910266 conn(0x7ff5e40659d0 msgr2=0x7ff5e4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.589+0000 7ff5eb851640 1 -- 172.21.15.47:0/2545910266 shutdown_connections 2024-03-23T09:41:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.589+0000 7ff5eb851640 1 -- 172.21.15.47:0/2545910266 wait complete. 2024-03-23T09:41:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.590+0000 7ff5eb851640 1 Processor -- start 2024-03-23T09:41:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.590+0000 7ff5eb851640 1 -- start start 2024-03-23T09:41:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.591+0000 7ff5eb851640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e40b8350 0x7ff5e41f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.591+0000 7ff5eb851640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5e40cde10 0x7ff5e41f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.591+0000 7ff5eb851640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5e415c990 0x7ff5e41594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.591+0000 7ff5eb851640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e41599f0 con 0x7ff5e40b8350 2024-03-23T09:41:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.591+0000 7ff5eb851640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4159c00 con 0x7ff5e415c990 2024-03-23T09:41:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.591+0000 7ff5eb851640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e41f5af0 con 0x7ff5e40cde10 2024-03-23T09:41:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.591+0000 7ff5e95ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e40b8350 0x7ff5e41f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.591+0000 7ff5e95ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e40b8350 0x7ff5e41f65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41564/0 (socket says 172.21.15.47:41564) 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.591+0000 7ff5e95ed640 1 -- 172.21.15.47:0/1974605786 learned_addr learned my addr 172.21.15.47:0/1974605786 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.591+0000 7ff5e9dee640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5e415c990 0x7ff5e41594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.591+0000 7ff5e8dec640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5e40cde10 0x7ff5e41f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.592+0000 7ff5e95ed640 1 -- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5e40cde10 msgr2=0x7ff5e41f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.592+0000 7ff5e95ed640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5e40cde10 0x7ff5e41f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.592+0000 7ff5e95ed640 1 -- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5e415c990 msgr2=0x7ff5e41594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.592+0000 7ff5e95ed640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5e415c990 0x7ff5e41594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.592+0000 7ff5e95ed640 1 -- 172.21.15.47:0/1974605786 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e0009a70 con 0x7ff5e40b8350 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.592+0000 7ff5e8dec640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5e40cde10 0x7ff5e41f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.592+0000 7ff5e95ed640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e40b8350 0x7ff5e41f65d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00d7d0 tx=0x7ff5dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.592+0000 7ff5d9ffb640 1 -- 172.21.15.47:0/1974605786 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc004030 con 0x7ff5e40b8350 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.592+0000 7ff5d9ffb640 1 -- 172.21.15.47:0/1974605786 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5dc004190 con 0x7ff5e40b8350 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.592+0000 7ff5eb851640 1 -- 172.21.15.47:0/1974605786 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e41f5dd0 con 0x7ff5e40b8350 2024-03-23T09:41:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.592+0000 7ff5d9ffb640 1 -- 172.21.15.47:0/1974605786 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc004520 con 0x7ff5e40b8350 2024-03-23T09:41:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.592+0000 7ff5eb851640 1 -- 172.21.15.47:0/1974605786 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e4159ee0 con 0x7ff5e40b8350 2024-03-23T09:41:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.593+0000 7ff5eb851640 1 -- 172.21.15.47:0/1974605786 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e4072360 con 0x7ff5e40b8350 2024-03-23T09:41:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.593+0000 7ff5d9ffb640 1 -- 172.21.15.47:0/1974605786 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff5dc0096a0 con 0x7ff5e40b8350 2024-03-23T09:41:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.594+0000 7ff5d9ffb640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5c806ac00 0x7ff5c806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.594+0000 7ff5d9ffb640 1 -- 172.21.15.47:0/1974605786 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5dc089460 con 0x7ff5e40b8350 2024-03-23T09:41:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.594+0000 7ff5e8dec640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5c806ac00 0x7ff5c806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.594+0000 7ff5e8dec640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5c806ac00 0x7ff5c806d0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff5e415b840 tx=0x7ff5d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.598+0000 7ff5d9ffb640 1 -- 172.21.15.47:0/1974605786 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff5dc0546d0 con 0x7ff5e40b8350 2024-03-23T09:41:16.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.776+0000 7ff5eb851640 1 -- 172.21.15.47:0/1974605786 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5e4066be0 con 0x7ff5e40b8350 2024-03-23T09:41:16.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.776+0000 7ff5d9ffb640 1 -- 172.21.15.47:0/1974605786 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5dc057cf0 con 0x7ff5e40b8350 2024-03-23T09:41:16.776 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:16.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.778+0000 7ff5eb851640 1 -- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5c806ac00 msgr2=0x7ff5c806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.778+0000 7ff5eb851640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5c806ac00 0x7ff5c806d0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff5e415b840 tx=0x7ff5d4009040 comp rx=0 tx=0).stop 2024-03-23T09:41:16.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.778+0000 7ff5eb851640 1 -- 172.21.15.47:0/1974605786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e40b8350 msgr2=0x7ff5e41f65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.778+0000 7ff5eb851640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e40b8350 0x7ff5e41f65d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00d7d0 tx=0x7ff5dc00dc90 comp rx=0 tx=0).stop 2024-03-23T09:41:16.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.778+0000 7ff5eb851640 1 -- 172.21.15.47:0/1974605786 shutdown_connections 2024-03-23T09:41:16.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.778+0000 7ff5eb851640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5c806ac00 0x7ff5c806d0c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.778+0000 7ff5eb851640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5e415c990 0x7ff5e41594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.778+0000 7ff5eb851640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5e40cde10 0x7ff5e41f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.778+0000 7ff5eb851640 1 --2- 172.21.15.47:0/1974605786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e40b8350 0x7ff5e41f65d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.778+0000 7ff5eb851640 1 -- 172.21.15.47:0/1974605786 >> 172.21.15.47:0/1974605786 conn(0x7ff5e40659d0 msgr2=0x7ff5e41566d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:16.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.778+0000 7ff5eb851640 1 -- 172.21.15.47:0/1974605786 shutdown_connections 2024-03-23T09:41:16.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.778+0000 7ff5eb851640 1 -- 172.21.15.47:0/1974605786 wait complete. 2024-03-23T09:41:16.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:41:16.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.890+0000 7ff621327640 1 Processor -- start 2024-03-23T09:41:16.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.890+0000 7ff621327640 1 -- start start 2024-03-23T09:41:16.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.890+0000 7ff621327640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff61c0b8350 0x7ff61c0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.890+0000 7ff621327640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff61c0cdc40 0x7ff61c0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.890+0000 7ff621327640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c0d11e0 con 0x7ff61c15ca90 2024-03-23T09:41:16.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.890+0000 7ff621327640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c0ce0d0 con 0x7ff61c0cdc40 2024-03-23T09:41:16.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.890+0000 7ff621327640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c160f60 con 0x7ff61c0b8350 2024-03-23T09:41:16.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61affd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff61c15ca90 0x7ff61c160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33174/0 (socket says 172.21.15.47:33174) 2024-03-23T09:41:16.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61affd640 1 -- 172.21.15.47:0/1561084600 learned_addr learned my addr 172.21.15.47:0/1561084600 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:16.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61b7fe640 1 --2- 172.21.15.47:0/1561084600 >> v2:172.21.15.175:3300/0 conn(0x7ff61c0cdc40 0x7ff61c0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff613fff640 1 --2- 172.21.15.47:0/1561084600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff61c0b8350 0x7ff61c0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025592967 0 0) 0x7ff61c0d11e0 con 0x7ff61c15ca90 2024-03-23T09:41:16.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff608003520 con 0x7ff61c15ca90 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 956568276 0 0) 0x7ff608003520 con 0x7ff61c15ca90 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff61c0b8350 msgr2=0x7ff61c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 --2- 172.21.15.47:0/1561084600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff61c0b8350 0x7ff61c0cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 >> v2:172.21.15.175:3300/0 conn(0x7ff61c0cdc40 msgr2=0x7ff61c0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 --2- 172.21.15.47:0/1561084600 >> v2:172.21.15.175:3300/0 conn(0x7ff61c0cdc40 0x7ff61c0d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c161240 con 0x7ff61c15ca90 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7ff60c001670 con 0x7ff61c15ca90 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 >> v1:172.21.15.47:6789/0 conn(0x7ff61c15ca90 legacy=0x7ff61c160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 --2- 172.21.15.47:0/1561084600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff61c0b8350 0x7ff608003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 --2- 172.21.15.47:0/1561084600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff608003ef0 0x7ff6080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 --2- 172.21.15.47:0/1561084600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff608004d00 0x7ff608005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0ce0d0 con 0x7ff608003ef0 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c160f60 con 0x7ff608004d00 2024-03-23T09:41:16.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60800b9d0 con 0x7ff61c0b8350 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff613fff640 1 --2- 172.21.15.47:0/1561084600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff608003ef0 0x7ff6080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61affd640 1 --2- 172.21.15.47:0/1561084600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff61c0b8350 0x7ff608003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.891+0000 7ff61b7fe640 1 --2- 172.21.15.47:0/1561084600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff608004d00 0x7ff608005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.892+0000 7ff613fff640 1 -- 172.21.15.47:0/1561084600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff61c0b8350 msgr2=0x7ff608003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.892+0000 7ff613fff640 1 --2- 172.21.15.47:0/1561084600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff61c0b8350 0x7ff608003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.892+0000 7ff613fff640 1 -- 172.21.15.47:0/1561084600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff608004d00 msgr2=0x7ff608005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.892+0000 7ff613fff640 1 --2- 172.21.15.47:0/1561084600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff608004d00 0x7ff608005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.892+0000 7ff613fff640 1 -- 172.21.15.47:0/1561084600 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff608003520 con 0x7ff608003ef0 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.892+0000 7ff61b7fe640 1 --2- 172.21.15.47:0/1561084600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff608004d00 0x7ff608005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.892+0000 7ff613fff640 1 --2- 172.21.15.47:0/1561084600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff608003ef0 0x7ff6080047c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff604009af0 tx=0x7ff6080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9763e5a3e197327 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.892+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60402c6b0 con 0x7ff608003ef0 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.892+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60402c810 con 0x7ff608003ef0 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.892+0000 7ff61a7fc640 1 -- 172.21.15.47:0/1561084600 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60402cba0 con 0x7ff608003ef0 2024-03-23T09:41:16.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.893+0000 7ff621327640 1 -- 172.21.15.47:0/1561084600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff608003ef0 msgr2=0x7ff6080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.893+0000 7ff621327640 1 --2- 172.21.15.47:0/1561084600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff608003ef0 0x7ff6080047c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff604009af0 tx=0x7ff6080057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.893+0000 7ff61b7fe640 1 -- 172.21.15.47:0/1561084600 reap_dead start 2024-03-23T09:41:16.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.893+0000 7ff621327640 1 -- 172.21.15.47:0/1561084600 shutdown_connections 2024-03-23T09:41:16.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.893+0000 7ff621327640 1 -- 172.21.15.47:0/1561084600 >> 172.21.15.47:0/1561084600 conn(0x7ff61c0659d0 msgr2=0x7ff61c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:16.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.893+0000 7ff621327640 1 -- 172.21.15.47:0/1561084600 shutdown_connections 2024-03-23T09:41:16.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.893+0000 7ff621327640 1 -- 172.21.15.47:0/1561084600 wait complete. 2024-03-23T09:41:16.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.894+0000 7ff621327640 1 Processor -- start 2024-03-23T09:41:16.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.894+0000 7ff621327640 1 -- start start 2024-03-23T09:41:16.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff621327640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff608003ef0 0x7ff61c155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff621327640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff61c0b2420 0x7ff61c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff621327640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c0b28a0 0x7ff61c159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff621327640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0b3a10 con 0x7ff61c0b28a0 2024-03-23T09:41:16.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff621327640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c156460 con 0x7ff608003ef0 2024-03-23T09:41:16.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff621327640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0b2e00 con 0x7ff61c0b2420 2024-03-23T09:41:16.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff61b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c0b28a0 0x7ff61c159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff61b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c0b28a0 0x7ff61c159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41582/0 (socket says 172.21.15.47:41582) 2024-03-23T09:41:16.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff61b7fe640 1 -- 172.21.15.47:0/3008837732 learned_addr learned my addr 172.21.15.47:0/3008837732 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:16.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff61affd640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff608003ef0 0x7ff61c155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff613fff640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff61c0b2420 0x7ff61c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff61b7fe640 1 -- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff61c0b2420 msgr2=0x7ff61c158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff61b7fe640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff61c0b2420 0x7ff61c158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff61b7fe640 1 -- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff608003ef0 msgr2=0x7ff61c155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff61b7fe640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff608003ef0 0x7ff61c155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff61b7fe640 1 -- 172.21.15.47:0/3008837732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c0b30e0 con 0x7ff61c0b28a0 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.895+0000 7ff61b7fe640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c0b28a0 0x7ff61c159500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0b54c0 tx=0x7ff614010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.896+0000 7ff6137fe640 1 -- 172.21.15.47:0/3008837732 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61400e050 con 0x7ff61c0b28a0 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.896+0000 7ff6137fe640 1 -- 172.21.15.47:0/3008837732 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61400f040 con 0x7ff61c0b28a0 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.896+0000 7ff6137fe640 1 -- 172.21.15.47:0/3008837732 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614003990 con 0x7ff61c0b28a0 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.896+0000 7ff621327640 1 -- 172.21.15.47:0/3008837732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c0b33c0 con 0x7ff61c0b28a0 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.896+0000 7ff621327640 1 -- 172.21.15.47:0/3008837732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c15b270 con 0x7ff61c0b28a0 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.897+0000 7ff621327640 1 -- 172.21.15.47:0/3008837732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c072360 con 0x7ff61c0b28a0 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.897+0000 7ff6137fe640 1 -- 172.21.15.47:0/3008837732 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff614015440 con 0x7ff61c0b28a0 2024-03-23T09:41:16.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.897+0000 7ff6137fe640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5fc06ac30 0x7ff5fc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:16.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.898+0000 7ff6137fe640 1 -- 172.21.15.47:0/3008837732 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6140897e0 con 0x7ff61c0b28a0 2024-03-23T09:41:16.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.898+0000 7ff61affd640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5fc06ac30 0x7ff5fc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:16.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.898+0000 7ff61affd640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5fc06ac30 0x7ff5fc06d0f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff60c001550 tx=0x7ff60c0078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:16.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:16.902+0000 7ff6137fe640 1 -- 172.21.15.47:0/3008837732 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff614054b00 con 0x7ff61c0b28a0 2024-03-23T09:41:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.117+0000 7ff621327640 1 -- 172.21.15.47:0/3008837732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff61c0b3bc0 con 0x7ff61c0b28a0 2024-03-23T09:41:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.118+0000 7ff6137fe640 1 -- 172.21.15.47:0/3008837732 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4734 (secure 0 0 0) 0x7ff61c0b3bc0 con 0x7ff61c0b28a0 2024-03-23T09:41:17.117 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:17.118 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1970049669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1970049669},{"type":"v1","addr":"172.21.15.47:6833","nonce":1970049669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4162785911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4162785911},{"type":"v1","addr":"172.21.15.47:6835","nonce":4162785911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1036174787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1036174787},{"type":"v1","addr":"172.21.15.175:6836","nonce":1036174787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:03.137957+0000","modified":"2024-03-23T09:41:05.147528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1633245139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1633245139},{"type":"v1","addr":"172.21.15.175:6837","nonce":1633245139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T09:41:17.118 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2024-03-23T09:41:17.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.119+0000 7ff621327640 1 -- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5fc06ac30 msgr2=0x7ff5fc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.119+0000 7ff621327640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5fc06ac30 0x7ff5fc06d0f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff60c001550 tx=0x7ff60c0078f0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.119+0000 7ff621327640 1 -- 172.21.15.47:0/3008837732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c0b28a0 msgr2=0x7ff61c159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.119+0000 7ff621327640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c0b28a0 0x7ff61c159500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0b54c0 tx=0x7ff614010590 comp rx=0 tx=0).stop 2024-03-23T09:41:17.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.119+0000 7ff621327640 1 -- 172.21.15.47:0/3008837732 shutdown_connections 2024-03-23T09:41:17.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.119+0000 7ff621327640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5fc06ac30 0x7ff5fc06d0f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.119+0000 7ff621327640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c0b28a0 0x7ff61c159500 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.119+0000 7ff621327640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff61c0b2420 0x7ff61c158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.119+0000 7ff621327640 1 --2- 172.21.15.47:0/3008837732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff608003ef0 0x7ff61c155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.119+0000 7ff621327640 1 -- 172.21.15.47:0/3008837732 >> 172.21.15.47:0/3008837732 conn(0x7ff61c0659d0 msgr2=0x7ff61c072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:17.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.119+0000 7ff621327640 1 -- 172.21.15.47:0/3008837732 shutdown_connections 2024-03-23T09:41:17.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.119+0000 7ff621327640 1 -- 172.21.15.47:0/3008837732 wait complete. 2024-03-23T09:41:17.130 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:17.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.230+0000 7f9f9bb00640 1 Processor -- start 2024-03-23T09:41:17.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.230+0000 7f9f9bb00640 1 -- start start 2024-03-23T09:41:17.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.231+0000 7f9f9bb00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f9415ca90 0x7f9f94160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:17.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.231+0000 7f9f9bb00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f940d11e0 con 0x7f9f940b8350 2024-03-23T09:41:17.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.231+0000 7f9f9bb00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f94160f60 con 0x7f9f940cde10 2024-03-23T09:41:17.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.231+0000 7f9f9bb00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f94161240 con 0x7f9f9415ca90 2024-03-23T09:41:17.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.231+0000 7f9f9989c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f9415ca90 0x7f9f94160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:17.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.231+0000 7f9f9909b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9f940b8350 0x7f9f940cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33180/0 (socket says 172.21.15.47:33180) 2024-03-23T09:41:17.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.231+0000 7f9f9909b640 1 -- 172.21.15.47:0/2514116597 learned_addr learned my addr 172.21.15.47:0/2514116597 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:17.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.231+0000 7f9f9989c640 1 -- 172.21.15.47:0/2514116597 >> v1:172.21.15.175:6789/0 conn(0x7f9f940cde10 legacy=0x7f9f940d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:17.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.231+0000 7f9f9989c640 1 -- 172.21.15.47:0/2514116597 >> v1:172.21.15.47:6789/0 conn(0x7f9f940b8350 legacy=0x7f9f940cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.231+0000 7f9f9989c640 1 -- 172.21.15.47:0/2514116597 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f94161520 con 0x7f9f9415ca90 2024-03-23T09:41:17.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.231+0000 7f9f9889a640 1 -- 172.21.15.47:0/2514116597 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295051015 0 0) 0x7f9f940d11e0 con 0x7f9f940b8350 2024-03-23T09:41:17.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.232+0000 7f9f9989c640 1 --2- 172.21.15.47:0/2514116597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f9415ca90 0x7f9f94160710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0097a0 tx=0x7f9f8c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b1e68c386a073d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:17.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.232+0000 7f9f9889a640 1 -- 172.21.15.47:0/2514116597 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c02e070 con 0x7f9f9415ca90 2024-03-23T09:41:17.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.232+0000 7f9f9889a640 1 -- 172.21.15.47:0/2514116597 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c0315a0 con 0x7f9f9415ca90 2024-03-23T09:41:17.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.232+0000 7f9f9889a640 1 -- 172.21.15.47:0/2514116597 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c031930 con 0x7f9f9415ca90 2024-03-23T09:41:17.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.233+0000 7f9f9bb00640 1 -- 172.21.15.47:0/2514116597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f9415ca90 msgr2=0x7f9f94160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.233+0000 7f9f9bb00640 1 --2- 172.21.15.47:0/2514116597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f9415ca90 0x7f9f94160710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0097a0 tx=0x7f9f8c02b840 comp rx=0 tx=0).stop 2024-03-23T09:41:17.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.233+0000 7f9f9bb00640 1 -- 172.21.15.47:0/2514116597 shutdown_connections 2024-03-23T09:41:17.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.233+0000 7f9f9bb00640 1 --2- 172.21.15.47:0/2514116597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f9415ca90 0x7f9f94160710 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.233+0000 7f9f9bb00640 1 -- 172.21.15.47:0/2514116597 >> 172.21.15.47:0/2514116597 conn(0x7f9f940659d0 msgr2=0x7f9f94065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:17.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.233+0000 7f9f9bb00640 1 -- 172.21.15.47:0/2514116597 shutdown_connections 2024-03-23T09:41:17.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.233+0000 7f9f9bb00640 1 -- 172.21.15.47:0/2514116597 wait complete. 2024-03-23T09:41:17.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.234+0000 7f9f9bb00640 1 Processor -- start 2024-03-23T09:41:17.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.234+0000 7f9f9bb00640 1 -- start start 2024-03-23T09:41:17.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9bb00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f940b8350 0x7f9f94152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:17.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9bb00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f940cde10 0x7f9f941515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:17.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9bb00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f9415ca90 0x7f9f9414e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:17.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9bb00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940b2420 con 0x7f9f9415ca90 2024-03-23T09:41:17.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9bb00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940b2fa0 con 0x7f9f940b8350 2024-03-23T09:41:17.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9bb00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940b2670 con 0x7f9f940cde10 2024-03-23T09:41:17.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9a09d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f9415ca90 0x7f9f9414e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:17.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9a09d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f9415ca90 0x7f9f9414e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41592/0 (socket says 172.21.15.47:41592) 2024-03-23T09:41:17.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9a09d640 1 -- 172.21.15.47:0/1623524026 learned_addr learned my addr 172.21.15.47:0/1623524026 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:17.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9989c640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f940b8350 0x7f9f94152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:17.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9909b640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f940cde10 0x7f9f941515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9a09d640 1 -- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f940cde10 msgr2=0x7f9f941515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9a09d640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f940cde10 0x7f9f941515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9a09d640 1 -- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f940b8350 msgr2=0x7f9f94152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9a09d640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f940b8350 0x7f9f94152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9a09d640 1 -- 172.21.15.47:0/1623524026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c009400 con 0x7f9f9415ca90 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9909b640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f940cde10 0x7f9f941515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9a09d640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f9415ca90 0x7f9f9414e3a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400c700 tx=0x7f9f8400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f927fc640 1 -- 172.21.15.47:0/1623524026 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8400aa50 con 0x7f9f9415ca90 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f927fc640 1 -- 172.21.15.47:0/1623524026 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8400abb0 con 0x7f9f9415ca90 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f9bb00640 1 -- 172.21.15.47:0/1623524026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f940b2950 con 0x7f9f9415ca90 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.235+0000 7f9f927fc640 1 -- 172.21.15.47:0/1623524026 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8400ce10 con 0x7f9f9415ca90 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.236+0000 7f9f9bb00640 1 -- 172.21.15.47:0/1623524026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f940b2ce0 con 0x7f9f9415ca90 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.236+0000 7f9f9bb00640 1 -- 172.21.15.47:0/1623524026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f94072360 con 0x7f9f9415ca90 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.237+0000 7f9f927fc640 1 -- 172.21.15.47:0/1623524026 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9f84012420 con 0x7f9f9415ca90 2024-03-23T09:41:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.237+0000 7f9f927fc640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f8006ac00 0x7f9f8006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:17.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.237+0000 7f9f927fc640 1 -- 172.21.15.47:0/1623524026 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f84089740 con 0x7f9f9415ca90 2024-03-23T09:41:17.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.237+0000 7f9f9989c640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f8006ac00 0x7f9f8006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:17.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.238+0000 7f9f9989c640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f8006ac00 0x7f9f8006d0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c036000 tx=0x7f9f8c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:17.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.241+0000 7f9f927fc640 1 -- 172.21.15.47:0/1623524026 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9f84054910 con 0x7f9f9415ca90 2024-03-23T09:41:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.420+0000 7f9f9bb00640 1 -- 172.21.15.47:0/1623524026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f940b3110 con 0x7f9f9415ca90 2024-03-23T09:41:17.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.421+0000 7f9f927fc640 1 -- 172.21.15.47:0/1623524026 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f9f84016090 con 0x7f9f9415ca90 2024-03-23T09:41:17.420 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:17.421 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.423+0000 7f9f9bb00640 1 -- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f8006ac00 msgr2=0x7f9f8006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.423+0000 7f9f9bb00640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f8006ac00 0x7f9f8006d0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c036000 tx=0x7f9f8c03d040 comp rx=0 tx=0).stop 2024-03-23T09:41:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.423+0000 7f9f9bb00640 1 -- 172.21.15.47:0/1623524026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f9415ca90 msgr2=0x7f9f9414e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.423+0000 7f9f9bb00640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f9415ca90 0x7f9f9414e3a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400c700 tx=0x7f9f8400cbc0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.423+0000 7f9f9bb00640 1 -- 172.21.15.47:0/1623524026 shutdown_connections 2024-03-23T09:41:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.423+0000 7f9f9bb00640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f8006ac00 0x7f9f8006d0c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.423+0000 7f9f9bb00640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f9415ca90 0x7f9f9414e3a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.423+0000 7f9f9bb00640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f940cde10 0x7f9f941515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.423+0000 7f9f9bb00640 1 --2- 172.21.15.47:0/1623524026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f940b8350 0x7f9f94152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.423+0000 7f9f9bb00640 1 -- 172.21.15.47:0/1623524026 >> 172.21.15.47:0/1623524026 conn(0x7f9f940659d0 msgr2=0x7f9f9415d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.423+0000 7f9f9bb00640 1 -- 172.21.15.47:0/1623524026 shutdown_connections 2024-03-23T09:41:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.423+0000 7f9f9bb00640 1 -- 172.21.15.47:0/1623524026 wait complete. 2024-03-23T09:41:17.433 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:41:17.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.534+0000 7f4d7bcb9640 1 Processor -- start 2024-03-23T09:41:17.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.534+0000 7f4d7bcb9640 1 -- start start 2024-03-23T09:41:17.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.535+0000 7f4d7bcb9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d7415a9e0 0x7f4d7415b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:17.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.535+0000 7f4d7bcb9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7415efc0 con 0x7f4d7406a3f0 2024-03-23T09:41:17.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.535+0000 7f4d7bcb9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7415f2a0 con 0x7f4d7415e4e0 2024-03-23T09:41:17.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.535+0000 7f4d7bcb9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7415b590 con 0x7f4d7415a9e0 2024-03-23T09:41:17.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.535+0000 7f4d79a55640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d7415a9e0 0x7f4d7415b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:17.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.535+0000 7f4d7a256640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4d7406a3f0 0x7f4d7415c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33182/0 (socket says 172.21.15.47:33182) 2024-03-23T09:41:17.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.535+0000 7f4d7a256640 1 -- 172.21.15.47:0/796410046 learned_addr learned my addr 172.21.15.47:0/796410046 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:17.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.535+0000 7f4d79a55640 1 -- 172.21.15.47:0/796410046 >> v1:172.21.15.175:6789/0 conn(0x7f4d7415e4e0 legacy=0x7f4d74069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:17.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.535+0000 7f4d79a55640 1 -- 172.21.15.47:0/796410046 >> v1:172.21.15.47:6789/0 conn(0x7f4d7406a3f0 legacy=0x7f4d7415c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.535+0000 7f4d78a53640 1 -- 172.21.15.47:0/796410046 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721139799 0 0) 0x7f4d7415efc0 con 0x7f4d7406a3f0 2024-03-23T09:41:17.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.535+0000 7f4d79a55640 1 -- 172.21.15.47:0/796410046 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7415b1b0 con 0x7f4d7415a9e0 2024-03-23T09:41:17.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.536+0000 7f4d79a55640 1 --2- 172.21.15.47:0/796410046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d7415a9e0 0x7f4d7415b9b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0097a0 tx=0x7f4d6c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fae4c21b95c15a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:17.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.536+0000 7f4d78a53640 1 -- 172.21.15.47:0/796410046 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c031830 con 0x7f4d7415a9e0 2024-03-23T09:41:17.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.536+0000 7f4d78a53640 1 -- 172.21.15.47:0/796410046 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c031990 con 0x7f4d7415a9e0 2024-03-23T09:41:17.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.536+0000 7f4d78a53640 1 -- 172.21.15.47:0/796410046 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c031d20 con 0x7f4d7415a9e0 2024-03-23T09:41:17.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.537+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/796410046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d7415a9e0 msgr2=0x7f4d7415b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.537+0000 7f4d7bcb9640 1 --2- 172.21.15.47:0/796410046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d7415a9e0 0x7f4d7415b9b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0097a0 tx=0x7f4d6c02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.537+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/796410046 shutdown_connections 2024-03-23T09:41:17.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.537+0000 7f4d7bcb9640 1 --2- 172.21.15.47:0/796410046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d7415a9e0 0x7f4d7415b9b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.537+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/796410046 >> 172.21.15.47:0/796410046 conn(0x7f4d74098a00 msgr2=0x7f4d740cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:17.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.537+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/796410046 shutdown_connections 2024-03-23T09:41:17.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.537+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/796410046 wait complete. 2024-03-23T09:41:17.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.538+0000 7f4d7bcb9640 1 Processor -- start 2024-03-23T09:41:17.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.538+0000 7f4d7bcb9640 1 -- start start 2024-03-23T09:41:17.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d7bcb9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d7406a3f0 0x7f4d7414fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:17.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d7bcb9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d7415a9e0 0x7f4d740acad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:17.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d7bcb9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d7415e4e0 0x7f4d740ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:17.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d7bcb9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740c6ce0 con 0x7f4d7406a3f0 2024-03-23T09:41:17.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d7bcb9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7415cff0 con 0x7f4d7415e4e0 2024-03-23T09:41:17.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d7bcb9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7415d2d0 con 0x7f4d7415a9e0 2024-03-23T09:41:17.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d79a55640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d7406a3f0 0x7f4d7414fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:17.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d79a55640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d7406a3f0 0x7f4d7414fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41614/0 (socket says 172.21.15.47:41614) 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d79a55640 1 -- 172.21.15.47:0/2352638545 learned_addr learned my addr 172.21.15.47:0/2352638545 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d7a256640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d7415e4e0 0x7f4d740ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d79254640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d7415a9e0 0x7f4d740acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d79a55640 1 -- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d7415a9e0 msgr2=0x7f4d740acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d79a55640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d7415a9e0 0x7f4d740acad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d79a55640 1 -- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d7415e4e0 msgr2=0x7f4d740ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d79a55640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d7415e4e0 0x7f4d740ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d79a55640 1 -- 172.21.15.47:0/2352638545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c009400 con 0x7f4d7406a3f0 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d79254640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d7415a9e0 0x7f4d740acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.539+0000 7f4d79a55640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d7406a3f0 0x7f4d7414fb10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0098c0 tx=0x7f4d6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.540+0000 7f4d727fc640 1 -- 172.21.15.47:0/2352638545 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c03ba70 con 0x7f4d7406a3f0 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.540+0000 7f4d727fc640 1 -- 172.21.15.47:0/2352638545 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c03bbd0 con 0x7f4d7406a3f0 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.540+0000 7f4d727fc640 1 -- 172.21.15.47:0/2352638545 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0453f0 con 0x7f4d7406a3f0 2024-03-23T09:41:17.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.540+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/2352638545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7415d5b0 con 0x7f4d7406a3f0 2024-03-23T09:41:17.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.540+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/2352638545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d74155780 con 0x7f4d7406a3f0 2024-03-23T09:41:17.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.540+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/2352638545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d740672f0 con 0x7f4d7406a3f0 2024-03-23T09:41:17.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.541+0000 7f4d727fc640 1 -- 172.21.15.47:0/2352638545 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4d6c04e450 con 0x7f4d7406a3f0 2024-03-23T09:41:17.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.541+0000 7f4d727fc640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4d6006ac00 0x7f4d6006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:17.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.541+0000 7f4d727fc640 1 -- 172.21.15.47:0/2352638545 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d6c0ab590 con 0x7f4d7406a3f0 2024-03-23T09:41:17.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.542+0000 7f4d79254640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4d6006ac00 0x7f4d6006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:17.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.542+0000 7f4d79254640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4d6006ac00 0x7f4d6006d0c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4d74154560 tx=0x7f4d68009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:17.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.546+0000 7f4d727fc640 1 -- 172.21.15.47:0/2352638545 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4d6c076800 con 0x7f4d7406a3f0 2024-03-23T09:41:17.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.759+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/2352638545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d740aeda0 con 0x7f4d7406a3f0 2024-03-23T09:41:17.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.759+0000 7f4d727fc640 1 -- 172.21.15.47:0/2352638545 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4734 (secure 0 0 0) 0x7f4d6c079e20 con 0x7f4d7406a3f0 2024-03-23T09:41:17.759 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:17.759 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1970049669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1970049669},{"type":"v1","addr":"172.21.15.47:6833","nonce":1970049669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4162785911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4162785911},{"type":"v1","addr":"172.21.15.47:6835","nonce":4162785911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1036174787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1036174787},{"type":"v1","addr":"172.21.15.175:6836","nonce":1036174787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:03.137957+0000","modified":"2024-03-23T09:41:05.147528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1633245139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1633245139},{"type":"v1","addr":"172.21.15.175:6837","nonce":1633245139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T09:41:17.760 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2024-03-23T09:41:17.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.761+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4d6006ac00 msgr2=0x7f4d6006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.761+0000 7f4d7bcb9640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4d6006ac00 0x7f4d6006d0c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4d74154560 tx=0x7f4d68009040 comp rx=0 tx=0).stop 2024-03-23T09:41:17.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.761+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/2352638545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d7406a3f0 msgr2=0x7f4d7414fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:17.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.761+0000 7f4d7bcb9640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d7406a3f0 0x7f4d7414fb10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0098c0 tx=0x7f4d6c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.761+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/2352638545 shutdown_connections 2024-03-23T09:41:17.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.761+0000 7f4d7bcb9640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4d6006ac00 0x7f4d6006d0c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.761+0000 7f4d7bcb9640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d7415e4e0 0x7f4d740ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.761+0000 7f4d7bcb9640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d7415a9e0 0x7f4d740acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.761+0000 7f4d7bcb9640 1 --2- 172.21.15.47:0/2352638545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d7406a3f0 0x7f4d7414fb10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:17.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.761+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/2352638545 >> 172.21.15.47:0/2352638545 conn(0x7f4d74098a00 msgr2=0x7f4d74069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:17.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.761+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/2352638545 shutdown_connections 2024-03-23T09:41:17.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:17.761+0000 7f4d7bcb9640 1 -- 172.21.15.47:0/2352638545 wait complete. 2024-03-23T09:41:18.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:41:03.137957+0000', 'modified': '2024-03-23T09:41:05.147528+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6837/1633245139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6835', 'nonce': 1633245139}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 1633245139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:41:18.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:41:18.175 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-23T09:41:18.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.281+0000 7f60b2e3e640 1 Processor -- start 2024-03-23T09:41:18.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.281+0000 7f60b2e3e640 1 -- start start 2024-03-23T09:41:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.282+0000 7f60b2e3e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60ac15cd00 0x7f60ac0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.282+0000 7f60b2e3e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f60ac0d1020 0x7f60ac0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.282+0000 7f60b2e3e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ac0cfba0 con 0x7f60ac0cf750 2024-03-23T09:41:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.282+0000 7f60b2e3e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ac15d0e0 con 0x7f60ac15cd00 2024-03-23T09:41:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.282+0000 7f60b2e3e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ac0d1540 con 0x7f60ac0d1020 2024-03-23T09:41:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.282+0000 7f60b0bda640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60ac15cd00 0x7f60ac0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.282+0000 7f60b0bda640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60ac15cd00 0x7f60ac0d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41616/0 (socket says 172.21.15.47:41616) 2024-03-23T09:41:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.282+0000 7f60b0bda640 1 -- 172.21.15.47:0/1244306201 learned_addr learned my addr 172.21.15.47:0/1244306201 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.282+0000 7f60abfff640 1 --2- 172.21.15.47:0/1244306201 >> v2:172.21.15.175:3300/0 conn(0x7f60ac0d1020 0x7f60ac0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.282+0000 7f60b0bda640 1 -- 172.21.15.47:0/1244306201 >> v2:172.21.15.175:3300/0 conn(0x7f60ac0d1020 msgr2=0x7f60ac0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.282+0000 7f60b0bda640 1 --2- 172.21.15.47:0/1244306201 >> v2:172.21.15.175:3300/0 conn(0x7f60ac0d1020 0x7f60ac0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.282+0000 7f60b0bda640 1 -- 172.21.15.47:0/1244306201 >> v1:172.21.15.175:6789/0 conn(0x7f60ac0cf750 legacy=0x7f60ac0b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.282+0000 7f60b0bda640 1 -- 172.21.15.47:0/1244306201 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac0d1820 con 0x7f60ac15cd00 2024-03-23T09:41:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.283+0000 7f60b0bda640 1 --2- 172.21.15.47:0/1244306201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ac15cd00 0x7f60ac0d0ae0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f60ac06c1a0 tx=0x7f60a402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8bcdfbe8d5571b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.283+0000 7f60ab7fe640 1 -- 172.21.15.47:0/1244306201 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a40317c0 con 0x7f60ac15cd00 2024-03-23T09:41:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.283+0000 7f60ab7fe640 1 -- 172.21.15.47:0/1244306201 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a4031920 con 0x7f60ac15cd00 2024-03-23T09:41:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.283+0000 7f60ab7fe640 1 -- 172.21.15.47:0/1244306201 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a4031cb0 con 0x7f60ac15cd00 2024-03-23T09:41:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.284+0000 7f60b2e3e640 1 -- 172.21.15.47:0/1244306201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ac15cd00 msgr2=0x7f60ac0d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.284+0000 7f60b2e3e640 1 --2- 172.21.15.47:0/1244306201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ac15cd00 0x7f60ac0d0ae0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f60ac06c1a0 tx=0x7f60a402b860 comp rx=0 tx=0).stop 2024-03-23T09:41:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.284+0000 7f60b2e3e640 1 -- 172.21.15.47:0/1244306201 shutdown_connections 2024-03-23T09:41:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.284+0000 7f60b2e3e640 1 --2- 172.21.15.47:0/1244306201 >> v2:172.21.15.175:3300/0 conn(0x7f60ac0d1020 0x7f60ac0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.284+0000 7f60b2e3e640 1 --2- 172.21.15.47:0/1244306201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ac15cd00 0x7f60ac0d0ae0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.284+0000 7f60b2e3e640 1 -- 172.21.15.47:0/1244306201 >> 172.21.15.47:0/1244306201 conn(0x7f60ac06b160 msgr2=0x7f60ac06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.284+0000 7f60b2e3e640 1 -- 172.21.15.47:0/1244306201 shutdown_connections 2024-03-23T09:41:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.284+0000 7f60b2e3e640 1 -- 172.21.15.47:0/1244306201 wait complete. 2024-03-23T09:41:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.285+0000 7f60b2e3e640 1 Processor -- start 2024-03-23T09:41:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.285+0000 7f60b2e3e640 1 -- start start 2024-03-23T09:41:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b2e3e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ac0cf750 0x7f60ac0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b2e3e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60ac0d1020 0x7f60ac0a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b2e3e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ac15cd00 0x7f60ac14beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b2e3e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac14e630 con 0x7f60ac0cf750 2024-03-23T09:41:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b2e3e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac0b5360 con 0x7f60ac15cd00 2024-03-23T09:41:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b2e3e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac0a98e0 con 0x7f60ac0d1020 2024-03-23T09:41:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b0bda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ac0cf750 0x7f60ac0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b0bda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ac0cf750 0x7f60ac0b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41630/0 (socket says 172.21.15.47:41630) 2024-03-23T09:41:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b0bda640 1 -- 172.21.15.47:0/2920211733 learned_addr learned my addr 172.21.15.47:0/2920211733 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60abfff640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60ac0d1020 0x7f60ac0a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b13db640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ac15cd00 0x7f60ac14beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b0bda640 1 -- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60ac0d1020 msgr2=0x7f60ac0a93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b0bda640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60ac0d1020 0x7f60ac0a93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b0bda640 1 -- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ac15cd00 msgr2=0x7f60ac14beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b0bda640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ac15cd00 0x7f60ac14beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60b0bda640 1 -- 172.21.15.47:0/2920211733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a40094d0 con 0x7f60ac0cf750 2024-03-23T09:41:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.286+0000 7f60abfff640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60ac0d1020 0x7f60ac0a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.287+0000 7f60b0bda640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ac0cf750 0x7f60ac0b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60a402b8a0 tx=0x7f60a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.287+0000 7f60a97fa640 1 -- 172.21.15.47:0/2920211733 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a4032860 con 0x7f60ac0cf750 2024-03-23T09:41:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.287+0000 7f60a97fa640 1 -- 172.21.15.47:0/2920211733 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a40329e0 con 0x7f60ac0cf750 2024-03-23T09:41:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.287+0000 7f60b2e3e640 1 -- 172.21.15.47:0/2920211733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ac14c3f0 con 0x7f60ac0cf750 2024-03-23T09:41:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.287+0000 7f60a97fa640 1 -- 172.21.15.47:0/2920211733 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a4032d90 con 0x7f60ac0cf750 2024-03-23T09:41:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.287+0000 7f60b2e3e640 1 -- 172.21.15.47:0/2920211733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac14dc90 con 0x7f60ac0cf750 2024-03-23T09:41:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.288+0000 7f60a97fa640 1 -- 172.21.15.47:0/2920211733 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f60a404e450 con 0x7f60ac0cf750 2024-03-23T09:41:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.289+0000 7f60a97fa640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f608c06ac00 0x7f608c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.289+0000 7f60a97fa640 1 -- 172.21.15.47:0/2920211733 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60a40ab620 con 0x7f60ac0cf750 2024-03-23T09:41:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.289+0000 7f60abfff640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f608c06ac00 0x7f608c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.289+0000 7f60abfff640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f608c06ac00 0x7f608c06d0c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f60ac06b890 tx=0x7f609c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.290+0000 7f60b59b5740 1 Processor -- start 2024-03-23T09:41:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.290+0000 7f60b59b5740 1 -- start start 2024-03-23T09:41:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.290+0000 7f60b59b5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113428750 0x5601133dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.290+0000 7f60b59b5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601133de7d0 0x5601133e02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.290+0000 7f60b59b5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601133dac70 0x56011340aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.290+0000 7f60b59b5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011334b9b0 con 0x560113428750 2024-03-23T09:41:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.290+0000 7f60b59b5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560113349140 con 0x5601133dac70 2024-03-23T09:41:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.290+0000 7f60b59b5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x560113344a30 con 0x5601133de7d0 2024-03-23T09:41:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.290+0000 7f60b0bda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113428750 0x5601133dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.291+0000 7f60b0bda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113428750 0x5601133dfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41642/0 (socket says 172.21.15.47:41642) 2024-03-23T09:41:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.291+0000 7f60b0bda640 1 -- 172.21.15.47:0/1721943277 learned_addr learned my addr 172.21.15.47:0/1721943277 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.291+0000 7f60abfff640 1 --2- 172.21.15.47:0/1721943277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601133de7d0 0x5601133e02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.291+0000 7f60b13db640 1 --2- 172.21.15.47:0/1721943277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601133dac70 0x56011340aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.291+0000 7f60b0bda640 1 -- 172.21.15.47:0/1721943277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601133de7d0 msgr2=0x5601133e02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.291+0000 7f60b0bda640 1 --2- 172.21.15.47:0/1721943277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601133de7d0 0x5601133e02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.291+0000 7f60b0bda640 1 -- 172.21.15.47:0/1721943277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601133dac70 msgr2=0x56011340aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.291+0000 7f60b0bda640 1 --2- 172.21.15.47:0/1721943277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601133dac70 0x56011340aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.291+0000 7f60b0bda640 1 -- 172.21.15.47:0/1721943277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601132d12f0 con 0x560113428750 2024-03-23T09:41:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.291+0000 7f60b0bda640 1 --2- 172.21.15.47:0/1721943277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113428750 0x5601133dfd80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f60a40769b0 tx=0x7f60a4009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66d210cb3784f23 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.291+0000 7f60a27fc640 1 -- 172.21.15.47:0/1721943277 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a407e2e0 con 0x560113428750 2024-03-23T09:41:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.291+0000 7f60a27fc640 1 -- 172.21.15.47:0/1721943277 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a40828a0 con 0x560113428750 2024-03-23T09:41:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.291+0000 7f60a27fc640 1 -- 172.21.15.47:0/1721943277 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a4078850 con 0x560113428750 2024-03-23T09:41:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.292+0000 7f60b59b5740 1 -- 172.21.15.47:0/1721943277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113428750 msgr2=0x5601133dfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.292+0000 7f60b59b5740 1 --2- 172.21.15.47:0/1721943277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113428750 0x5601133dfd80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f60a40769b0 tx=0x7f60a4009240 comp rx=0 tx=0).stop 2024-03-23T09:41:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.292+0000 7f60b59b5740 1 -- 172.21.15.47:0/1721943277 shutdown_connections 2024-03-23T09:41:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.292+0000 7f60b59b5740 1 --2- 172.21.15.47:0/1721943277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601133dac70 0x56011340aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.292+0000 7f60b59b5740 1 --2- 172.21.15.47:0/1721943277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601133de7d0 0x5601133e02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.292+0000 7f60b59b5740 1 --2- 172.21.15.47:0/1721943277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113428750 0x5601133dfd80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.292+0000 7f60b59b5740 1 -- 172.21.15.47:0/1721943277 >> 172.21.15.47:0/1721943277 conn(0x5601133dcee0 msgr2=0x560113425a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.292+0000 7f60b59b5740 1 -- 172.21.15.47:0/1721943277 shutdown_connections 2024-03-23T09:41:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.292+0000 7f60b59b5740 1 -- 172.21.15.47:0/1721943277 wait complete. 2024-03-23T09:41:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.292+0000 7f60b59b5740 1 -- start start 2024-03-23T09:41:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.293+0000 7f60b59b5740 1 Processor -- start 2024-03-23T09:41:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.293+0000 7f60b59b5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 0x560113402a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.293+0000 7f60b59b5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560113428750 0x56011342ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.293+0000 7f60b59b5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113408b00 0x560113405d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.293+0000 7f60b59b5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560113303090 con 0x5601133de7d0 2024-03-23T09:41:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.293+0000 7f60b59b5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560113302d30 con 0x560113408b00 2024-03-23T09:41:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.293+0000 7f60b59b5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601132c82a0 con 0x560113428750 2024-03-23T09:41:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.293+0000 7f60b59b5740 10 client.0 _ll_register_callbacks cb 0x5601132fe050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:41:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.293+0000 7f60b0bda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 0x560113402a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.293+0000 7f60b0bda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 0x560113402a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41658/0 (socket says 172.21.15.47:41658) 2024-03-23T09:41:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.293+0000 7f60b0bda640 1 -- 172.21.15.47:0/2759680804 learned_addr learned my addr 172.21.15.47:0/2759680804 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.293+0000 7f60b13db640 1 --2- 172.21.15.47:0/2759680804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113408b00 0x560113405d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.293+0000 7f60abfff640 1 --2- 172.21.15.47:0/2759680804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560113428750 0x56011342ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b0bda640 1 -- 172.21.15.47:0/2759680804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560113428750 msgr2=0x56011342ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b0bda640 1 --2- 172.21.15.47:0/2759680804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560113428750 0x56011342ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b0bda640 1 -- 172.21.15.47:0/2759680804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113408b00 msgr2=0x560113405d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b0bda640 1 --2- 172.21.15.47:0/2759680804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113408b00 0x560113405d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b0bda640 1 -- 172.21.15.47:0/2759680804 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac14dc90 con 0x5601133de7d0 2024-03-23T09:41:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b0bda640 1 --2- 172.21.15.47:0/2759680804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 0x560113402a10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f60a40769b0 tx=0x7f60a4046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf463c26db18bd60 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f608b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:41:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f608b7fe640 1 -- 172.21.15.47:0/2759680804 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a4079ba0 con 0x5601133de7d0 2024-03-23T09:41:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f608b7fe640 1 -- 172.21.15.47:0/2759680804 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60a407e2e0 con 0x5601133de7d0 2024-03-23T09:41:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b59b5740 1 -- 172.21.15.47:0/2759680804 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601130383c0 con 0x5601133de7d0 2024-03-23T09:41:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f608b7fe640 1 -- 172.21.15.47:0/2759680804 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60a407e0e0 con 0x5601133de7d0 2024-03-23T09:41:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b59b5740 10 client.4638 fetch_fsmap learned FSMap version 6 2024-03-23T09:41:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b59b5740 1 -- 172.21.15.47:0/2759680804 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x5601133430b0 con 0x5601133de7d0 2024-03-23T09:41:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f608b7fe640 1 -- 172.21.15.47:0/2759680804 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a4086080 con 0x5601133de7d0 2024-03-23T09:41:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f608b7fe640 1 -- 172.21.15.47:0/2759680804 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 6) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f60a4082410 con 0x5601133de7d0 2024-03-23T09:41:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b59b5740 10 client.4638 fetch_fsmap finished waiting for FSMap version 6 2024-03-23T09:41:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b59b5740 10 client.4638 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T09:41:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b59b5740 20 client.4638 populate_metadata read hostname 'smithi047' 2024-03-23T09:41:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b59b5740 1 --2- 172.21.15.47:0/2759680804 >> [v2:172.21.15.175:6835/1633245139,v1:172.21.15.175:6837/1633245139] conn(0x5601133eeff0 0x5601133ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b59b5740 4 client.4638 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:41:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.294+0000 7f60b59b5740 1 -- 172.21.15.47:0/2759680804 --> [v2:172.21.15.175:6835/1633245139,v1:172.21.15.175:6837/1633245139] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601130496c0 con 0x5601133eeff0 2024-03-23T09:41:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.295+0000 7f60abfff640 1 --2- 172.21.15.47:0/2759680804 >> [v2:172.21.15.175:6835/1633245139,v1:172.21.15.175:6837/1633245139] conn(0x5601133eeff0 0x5601133ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.295+0000 7f608b7fe640 10 client.4638 ms_handle_connect on v2:172.21.15.175:6835/1633245139 2024-03-23T09:41:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.295+0000 7f60abfff640 1 --2- 172.21.15.47:0/2759680804 >> [v2:172.21.15.175:6835/1633245139,v1:172.21.15.175:6837/1633245139] conn(0x5601133eeff0 0x5601133ef3d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ab67eb6db9d618 server_cookie=678e28a72de6e7bd in_seq=0 out_seq=0 2024-03-23T09:41:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f608b7fe640 1 -- 172.21.15.47:0/2759680804 <== mds.0 v2:172.21.15.175:6835/1633245139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5601130496c0 con 0x5601133eeff0 2024-03-23T09:41:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f608b7fe640 10 client.4638 handle_command_reply: tid=0 2024-03-23T09:41:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f60b59b5740 1 client.4638 shutdown 2024-03-23T09:41:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f60b59b5740 1 -- 172.21.15.47:0/2759680804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 msgr2=0x560113402a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f60b59b5740 1 --2- 172.21.15.47:0/2759680804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 0x560113402a10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f60a40769b0 tx=0x7f60a4046000 comp rx=0 tx=0).stop 2024-03-23T09:41:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f60b59b5740 1 -- 172.21.15.47:0/2759680804 shutdown_connections 2024-03-23T09:41:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f60b59b5740 1 --2- 172.21.15.47:0/2759680804 >> [v2:172.21.15.175:6835/1633245139,v1:172.21.15.175:6837/1633245139] conn(0x5601133eeff0 0x5601133ef3d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f60b59b5740 1 --2- 172.21.15.47:0/2759680804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113408b00 0x560113405d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f608b7fe640 0 client.4638 ms_handle_reset on v2:172.21.15.175:6835/1633245139 2024-03-23T09:41:18.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f60b59b5740 1 --2- 172.21.15.47:0/2759680804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560113428750 0x56011342ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f60b59b5740 1 --2- 172.21.15.47:0/2759680804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 0x560113402a10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f60b59b5740 1 -- 172.21.15.47:0/2759680804 >> 172.21.15.47:0/2759680804 conn(0x5601133dac70 msgr2=0x560113408100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:18.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f60b59b5740 1 -- 172.21.15.47:0/2759680804 shutdown_connections 2024-03-23T09:41:18.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f60b59b5740 1 -- 172.21.15.47:0/2759680804 wait complete. 2024-03-23T09:41:18.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.296+0000 7f60b59b5740 20 client.4638 trim_cache size 0 max 16384 2024-03-23T09:41:18.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.311+0000 7f60b59b5740 1 Processor -- start 2024-03-23T09:41:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.311+0000 7f60b59b5740 1 -- start start 2024-03-23T09:41:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60b59b5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601133de7d0 0x560113424ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60b59b5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113408b00 0x5601133db1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60b59b5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113428750 0x5601133e0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60b59b5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601133ddaf0 con 0x560113408b00 2024-03-23T09:41:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60b59b5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601132fe440 con 0x560113428750 2024-03-23T09:41:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60b59b5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x56011343f800 con 0x5601133de7d0 2024-03-23T09:41:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60abfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113408b00 0x5601133db1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60abfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113408b00 0x5601133db1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41662/0 (socket says 172.21.15.47:41662) 2024-03-23T09:41:18.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60abfff640 1 -- 172.21.15.47:0/3330012053 learned_addr learned my addr 172.21.15.47:0/3330012053 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:18.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60abfff640 1 -- 172.21.15.47:0/3330012053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601133de7d0 msgr2=0x560113424ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:18.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60abfff640 1 --2- 172.21.15.47:0/3330012053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601133de7d0 0x560113424ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60abfff640 1 -- 172.21.15.47:0/3330012053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113428750 msgr2=0x5601133e0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:18.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60abfff640 1 --2- 172.21.15.47:0/3330012053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113428750 0x5601133e0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60abfff640 1 -- 172.21.15.47:0/3330012053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601132e5430 con 0x560113408b00 2024-03-23T09:41:18.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60abfff640 1 --2- 172.21.15.47:0/3330012053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113408b00 0x5601133db1e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0b4cf0 tx=0x7f609c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359e103ee201a982 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:18.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60a2ffd640 1 -- 172.21.15.47:0/3330012053 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c02a1b0 con 0x560113408b00 2024-03-23T09:41:18.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.312+0000 7f60a2ffd640 1 -- 172.21.15.47:0/3330012053 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c02a040 con 0x560113408b00 2024-03-23T09:41:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.313+0000 7f60a2ffd640 1 -- 172.21.15.47:0/3330012053 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c003990 con 0x560113408b00 2024-03-23T09:41:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.313+0000 7f60b59b5740 1 -- 172.21.15.47:0/3330012053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113408b00 msgr2=0x5601133db1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.313+0000 7f60b59b5740 1 --2- 172.21.15.47:0/3330012053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113408b00 0x5601133db1e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0b4cf0 tx=0x7f609c015000 comp rx=0 tx=0).stop 2024-03-23T09:41:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.313+0000 7f60b59b5740 1 -- 172.21.15.47:0/3330012053 shutdown_connections 2024-03-23T09:41:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.313+0000 7f60b59b5740 1 --2- 172.21.15.47:0/3330012053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113428750 0x5601133e0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.313+0000 7f60b59b5740 1 --2- 172.21.15.47:0/3330012053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560113408b00 0x5601133db1e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.313+0000 7f60b59b5740 1 --2- 172.21.15.47:0/3330012053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601133de7d0 0x560113424ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.313+0000 7f60b59b5740 1 -- 172.21.15.47:0/3330012053 >> 172.21.15.47:0/3330012053 conn(0x5601133dac70 msgr2=0x560113408100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.313+0000 7f60b59b5740 1 -- 172.21.15.47:0/3330012053 shutdown_connections 2024-03-23T09:41:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.313+0000 7f60b59b5740 1 -- 172.21.15.47:0/3330012053 wait complete. 2024-03-23T09:41:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.314+0000 7f60b59b5740 1 -- start start 2024-03-23T09:41:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.314+0000 7f60b59b5740 1 Processor -- start 2024-03-23T09:41:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.314+0000 7f60b59b5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 0x5601134858c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.314+0000 7f60b59b5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560113408b00 0x5601133e0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.314+0000 7f60b59b5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113428750 0x5601133e0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.314+0000 7f60b59b5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601134269b0 con 0x5601133de7d0 2024-03-23T09:41:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.314+0000 7f60b59b5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601133db6e0 con 0x560113428750 2024-03-23T09:41:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.314+0000 7f60b59b5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x56011341b890 con 0x560113408b00 2024-03-23T09:41:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.314+0000 7f60b59b5740 10 client.0 _ll_register_callbacks cb 0x7f60a4032ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:41:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b0bda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 0x5601134858c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b0bda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 0x5601134858c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41672/0 (socket says 172.21.15.47:41672) 2024-03-23T09:41:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b0bda640 1 -- 172.21.15.47:0/1999676244 learned_addr learned my addr 172.21.15.47:0/1999676244 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b13db640 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113428750 0x5601133e0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60abfff640 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560113408b00 0x5601133e0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b0bda640 1 -- 172.21.15.47:0/1999676244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560113408b00 msgr2=0x5601133e0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b0bda640 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560113408b00 0x5601133e0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b0bda640 1 -- 172.21.15.47:0/1999676244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113428750 msgr2=0x5601133e0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b0bda640 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113428750 0x5601133e0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b0bda640 1 -- 172.21.15.47:0/1999676244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609c016520 con 0x5601133de7d0 2024-03-23T09:41:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b13db640 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113428750 0x5601133e0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b0bda640 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 0x5601134858c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f60a407a1b0 tx=0x7f60a40a9a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6486f1191c632b96 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f608b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:41:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f608b7fe640 1 -- 172.21.15.47:0/1999676244 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a4006b40 con 0x5601133de7d0 2024-03-23T09:41:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f608b7fe640 1 -- 172.21.15.47:0/1999676244 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60a407e2e0 con 0x5601133de7d0 2024-03-23T09:41:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b59b5740 1 -- 172.21.15.47:0/1999676244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60a40828a0 con 0x5601133de7d0 2024-03-23T09:41:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f608b7fe640 1 -- 172.21.15.47:0/1999676244 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60a407e0e0 con 0x5601133de7d0 2024-03-23T09:41:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b59b5740 10 client.4644 fetch_fsmap learned FSMap version 6 2024-03-23T09:41:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b59b5740 1 -- 172.21.15.47:0/1999676244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x56011330ab40 con 0x5601133de7d0 2024-03-23T09:41:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f608b7fe640 1 -- 172.21.15.47:0/1999676244 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a40828a0 con 0x5601133de7d0 2024-03-23T09:41:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f608b7fe640 1 -- 172.21.15.47:0/1999676244 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 6) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f60a4080270 con 0x5601133de7d0 2024-03-23T09:41:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b59b5740 10 client.4644 fetch_fsmap finished waiting for FSMap version 6 2024-03-23T09:41:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b59b5740 10 client.4644 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T09:41:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.315+0000 7f60b59b5740 20 client.4644 populate_metadata read hostname 'smithi047' 2024-03-23T09:41:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f60b59b5740 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.175:6835/1633245139,v1:172.21.15.175:6837/1633245139] conn(0x56011348ee60 0x56011348f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f60b59b5740 4 client.4644 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:41:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f60b59b5740 1 -- 172.21.15.47:0/1999676244 --> [v2:172.21.15.175:6835/1633245139,v1:172.21.15.175:6837/1633245139] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56011309cd10 con 0x56011348ee60 2024-03-23T09:41:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f60abfff640 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.175:6835/1633245139,v1:172.21.15.175:6837/1633245139] conn(0x56011348ee60 0x56011348f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f60abfff640 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.175:6835/1633245139,v1:172.21.15.175:6837/1633245139] conn(0x56011348ee60 0x56011348f240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2a42cd646935b1 server_cookie=7a3c113e9231bd31 in_seq=0 out_seq=0 2024-03-23T09:41:18.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f608b7fe640 10 client.4644 ms_handle_connect on v2:172.21.15.175:6835/1633245139 2024-03-23T09:41:18.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f608b7fe640 1 -- 172.21.15.47:0/1999676244 <== mds.0 v2:172.21.15.175:6835/1633245139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x56011309cd10 con 0x56011348ee60 2024-03-23T09:41:18.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f608b7fe640 10 client.4644 handle_command_reply: tid=0 2024-03-23T09:41:18.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f60b59b5740 1 client.4644 shutdown 2024-03-23T09:41:18.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f60b59b5740 1 -- 172.21.15.47:0/1999676244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 msgr2=0x5601134858c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f60b59b5740 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 0x5601134858c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f60a407a1b0 tx=0x7f60a40a9a10 comp rx=0 tx=0).stop 2024-03-23T09:41:18.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f60b59b5740 1 -- 172.21.15.47:0/1999676244 shutdown_connections 2024-03-23T09:41:18.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f60b59b5740 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.175:6835/1633245139,v1:172.21.15.175:6837/1633245139] conn(0x56011348ee60 0x56011348f240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f60b59b5740 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560113428750 0x5601133e0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f608b7fe640 0 client.4644 ms_handle_reset on v2:172.21.15.175:6835/1633245139 2024-03-23T09:41:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f60b59b5740 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560113408b00 0x5601133e0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.316+0000 7f60b59b5740 1 --2- 172.21.15.47:0/1999676244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601133de7d0 0x5601134858c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b59b5740 1 -- 172.21.15.47:0/1999676244 >> 172.21.15.47:0/1999676244 conn(0x5601133dac70 msgr2=0x5601133df8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b59b5740 1 -- 172.21.15.47:0/1999676244 shutdown_connections 2024-03-23T09:41:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b59b5740 1 -- 172.21.15.47:0/1999676244 wait complete. 2024-03-23T09:41:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b59b5740 20 client.4644 trim_cache size 0 max 16384 2024-03-23T09:41:18.333 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:41:18.333 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:41:18.333 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:41:18.333 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4285, 2024-03-23T09:41:18.333 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:41:18.333 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:41:18.333 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:41:18.333 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 15.176932039, 2024-03-23T09:41:18.333 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 6, 2024-03-23T09:41:18.334 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 21, 2024-03-23T09:41:18.334 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 21, 2024-03-23T09:41:18.334 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 21.600210228000002 2024-03-23T09:41:18.334 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:41:18.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b2e3e640 1 -- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f608c06ac00 msgr2=0x7f608c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b2e3e640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f608c06ac00 0x7f608c06d0c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f60ac06b890 tx=0x7f609c00b040 comp rx=0 tx=0).stop 2024-03-23T09:41:18.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b2e3e640 1 -- 172.21.15.47:0/2920211733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ac0cf750 msgr2=0x7f60ac0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b2e3e640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ac0cf750 0x7f60ac0b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60a402b8a0 tx=0x7f60a40046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b2e3e640 1 -- 172.21.15.47:0/2920211733 shutdown_connections 2024-03-23T09:41:18.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b2e3e640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f608c06ac00 0x7f608c06d0c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b2e3e640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ac15cd00 0x7f60ac14beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b2e3e640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60ac0d1020 0x7f60ac0a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b2e3e640 1 --2- 172.21.15.47:0/2920211733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ac0cf750 0x7f60ac0b4e20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b2e3e640 1 -- 172.21.15.47:0/2920211733 >> 172.21.15.47:0/2920211733 conn(0x7f60ac06b160 msgr2=0x7f60ac0c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:18.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b2e3e640 1 -- 172.21.15.47:0/2920211733 shutdown_connections 2024-03-23T09:41:18.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.317+0000 7f60b2e3e640 1 -- 172.21.15.47:0/2920211733 wait complete. 2024-03-23T09:41:18.338 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-23T09:41:18.363 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-23T09:41:18.364 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-23T09:41:18.364 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-23T09:41:18.364 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T09:41:18.364 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:41:18.364 INFO:tasks.ceph_fuse:client.2 config is {} 2024-03-23T09:41:18.364 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T09:41:18.364 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-23T09:41:18.365 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-23T09:41:18.365 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T09:41:18.365 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-23T09:41:18.365 INFO:tasks.ceph_fuse:client.3 config is {} 2024-03-23T09:41:18.365 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T09:41:18.365 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-23T09:41:18.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:18.365 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:41:18.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:18.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-03-23T09:41:18.448 INFO:teuthology.orchestra.run.smithi047.stderr:Cannot find device "ceph-brx" 2024-03-23T09:41:18.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:41:18.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:18.450 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-03-23T09:41:18.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:18.469 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-03-23T09:41:18.541 INFO:teuthology.orchestra.run.smithi175.stderr:Cannot find device "ceph-brx" 2024-03-23T09:41:18.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:41:18.542 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-23T09:41:18.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:41:18.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.645+0000 7f1f05dfb640 1 Processor -- start 2024-03-23T09:41:18.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.645+0000 7f1f05dfb640 1 -- start start 2024-03-23T09:41:18.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.646+0000 7f1f05dfb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f000cf210 0x7f1f000cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.646+0000 7f1f05dfb640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1f000b5a90 0x7f1f000b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.646+0000 7f1f05dfb640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f000cfbd0 con 0x7f1f0015cd00 2024-03-23T09:41:18.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.646+0000 7f1f05dfb640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0015d0e0 con 0x7f1f000b5a90 2024-03-23T09:41:18.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.646+0000 7f1f05dfb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f000d1540 con 0x7f1f000cf210 2024-03-23T09:41:18.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.646+0000 7f1efffff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1f000b5a90 0x7f1f000b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.646+0000 7f1efeffd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f000cf210 0x7f1f000cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.646+0000 7f1eff7fe640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f1f0015cd00 0x7f1f000d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:46332/0 (socket says 172.21.15.47:46332) 2024-03-23T09:41:18.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.646+0000 7f1eff7fe640 1 -- 172.21.15.47:0/1809514553 learned_addr learned my addr 172.21.15.47:0/1809514553 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:18.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.647+0000 7f1efeffd640 1 -- 172.21.15.47:0/1809514553 >> v2:172.21.15.175:3300/0 conn(0x7f1f000b5a90 msgr2=0x7f1f000b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:18.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.647+0000 7f1efeffd640 1 --2- 172.21.15.47:0/1809514553 >> v2:172.21.15.175:3300/0 conn(0x7f1f000b5a90 0x7f1f000b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.647+0000 7f1efeffd640 1 -- 172.21.15.47:0/1809514553 >> v1:172.21.15.175:6789/0 conn(0x7f1f0015cd00 legacy=0x7f1f000d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.647+0000 7f1efeffd640 1 -- 172.21.15.47:0/1809514553 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f000d1820 con 0x7f1f000cf210 2024-03-23T09:41:18.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.647+0000 7f1efffff640 1 --2- 172.21.15.47:0/1809514553 >> v2:172.21.15.175:3300/0 conn(0x7f1f000b5a90 0x7f1f000b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:18.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.647+0000 7f1efeffd640 1 --2- 172.21.15.47:0/1809514553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f000cf210 0x7f1f000cf690 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40097a0 tx=0x7f1ef402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56d7b3d9fb63eab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:18.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.648+0000 7f1efe7fc640 1 -- 172.21.15.47:0/1809514553 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef402e070 con 0x7f1f000cf210 2024-03-23T09:41:18.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.648+0000 7f1efe7fc640 1 -- 172.21.15.47:0/1809514553 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef40315a0 con 0x7f1f000cf210 2024-03-23T09:41:18.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.648+0000 7f1efe7fc640 1 -- 172.21.15.47:0/1809514553 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4031930 con 0x7f1f000cf210 2024-03-23T09:41:18.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.648+0000 7f1f05dfb640 1 -- 172.21.15.47:0/1809514553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f000cf210 msgr2=0x7f1f000cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.648+0000 7f1f05dfb640 1 --2- 172.21.15.47:0/1809514553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f000cf210 0x7f1f000cf690 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40097a0 tx=0x7f1ef402b840 comp rx=0 tx=0).stop 2024-03-23T09:41:18.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.648+0000 7f1f05dfb640 1 -- 172.21.15.47:0/1809514553 shutdown_connections 2024-03-23T09:41:18.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.648+0000 7f1f05dfb640 1 --2- 172.21.15.47:0/1809514553 >> v2:172.21.15.175:3300/0 conn(0x7f1f000b5a90 0x7f1f000b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.648+0000 7f1f05dfb640 1 --2- 172.21.15.47:0/1809514553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f000cf210 0x7f1f000cf690 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.648+0000 7f1f05dfb640 1 -- 172.21.15.47:0/1809514553 >> 172.21.15.47:0/1809514553 conn(0x7f1f0006b160 msgr2=0x7f1f0006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:18.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.648+0000 7f1f05dfb640 1 -- 172.21.15.47:0/1809514553 shutdown_connections 2024-03-23T09:41:18.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.648+0000 7f1f05dfb640 1 -- 172.21.15.47:0/1809514553 wait complete. 2024-03-23T09:41:18.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.649+0000 7f1f05dfb640 1 Processor -- start 2024-03-23T09:41:18.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.650+0000 7f1f05dfb640 1 -- start start 2024-03-23T09:41:18.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.650+0000 7f1f05dfb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5a90 0x7f1f000cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.650+0000 7f1f05dfb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f000cf210 0x7f1f000b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.650+0000 7f1f05dfb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f0015cd00 0x7f1f000b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.650+0000 7f1f05dfb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f000cbb70 con 0x7f1f000b5a90 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.650+0000 7f1f05dfb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f00151f20 con 0x7f1f0015cd00 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.650+0000 7f1f05dfb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f000caf60 con 0x7f1f000cf210 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.650+0000 7f1eff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5a90 0x7f1f000cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.650+0000 7f1eff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5a90 0x7f1f000cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41678/0 (socket says 172.21.15.47:41678) 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.650+0000 7f1eff7fe640 1 -- 172.21.15.47:0/3322005321 learned_addr learned my addr 172.21.15.47:0/3322005321 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1efeffd640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f000cf210 0x7f1f000b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1efffff640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f0015cd00 0x7f1f000b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1eff7fe640 1 -- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f000cf210 msgr2=0x7f1f000b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1eff7fe640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f000cf210 0x7f1f000b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1eff7fe640 1 -- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f0015cd00 msgr2=0x7f1f000b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1eff7fe640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f0015cd00 0x7f1f000b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1eff7fe640 1 -- 172.21.15.47:0/3322005321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef8009a70 con 0x7f1f000b5a90 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1efeffd640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f000cf210 0x7f1f000b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1efffff640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f0015cd00 0x7f1f000b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:18.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1eff7fe640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5a90 0x7f1f000cc000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000c4a0 tx=0x7f1ef000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:18.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1ee7fff640 1 -- 172.21.15.47:0/3322005321 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0005ce0 con 0x7f1f000b5a90 2024-03-23T09:41:18.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1ee7fff640 1 -- 172.21.15.47:0/3322005321 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef0005e40 con 0x7f1f000b5a90 2024-03-23T09:41:18.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1ee7fff640 1 -- 172.21.15.47:0/3322005321 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0007270 con 0x7f1f000b5a90 2024-03-23T09:41:18.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1f05dfb640 1 -- 172.21.15.47:0/3322005321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef4009400 con 0x7f1f000b5a90 2024-03-23T09:41:18.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.651+0000 7f1f05dfb640 1 -- 172.21.15.47:0/3322005321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f000cb520 con 0x7f1f000b5a90 2024-03-23T09:41:18.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.652+0000 7f1f05dfb640 1 -- 172.21.15.47:0/3322005321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f00072360 con 0x7f1f000b5a90 2024-03-23T09:41:18.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.652+0000 7f1ee7fff640 1 -- 172.21.15.47:0/3322005321 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1ef0007490 con 0x7f1f000b5a90 2024-03-23T09:41:18.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.653+0000 7f1ee7fff640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1eec06ac00 0x7f1eec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:18.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.653+0000 7f1ee7fff640 1 -- 172.21.15.47:0/3322005321 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ef00887a0 con 0x7f1f000b5a90 2024-03-23T09:41:18.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.653+0000 7f1efeffd640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1eec06ac00 0x7f1eec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:18.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.653+0000 7f1efeffd640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1eec06ac00 0x7f1eec06d0c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4036000 tx=0x7f1ef403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:18.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.657+0000 7f1ee7fff640 1 -- 172.21.15.47:0/3322005321 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1ef0053a30 con 0x7f1f000b5a90 2024-03-23T09:41:18.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.874+0000 7f1f05dfb640 1 -- 172.21.15.47:0/3322005321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1f00066be0 con 0x7f1f000b5a90 2024-03-23T09:41:18.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.874+0000 7f1ee7fff640 1 -- 172.21.15.47:0/3322005321 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1ef0017070 con 0x7f1f000b5a90 2024-03-23T09:41:18.874 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:41:18.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.876+0000 7f1f05dfb640 1 -- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1eec06ac00 msgr2=0x7f1eec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.876+0000 7f1f05dfb640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1eec06ac00 0x7f1eec06d0c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4036000 tx=0x7f1ef403d040 comp rx=0 tx=0).stop 2024-03-23T09:41:18.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.876+0000 7f1f05dfb640 1 -- 172.21.15.47:0/3322005321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5a90 msgr2=0x7f1f000cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:18.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.876+0000 7f1f05dfb640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5a90 0x7f1f000cc000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000c4a0 tx=0x7f1ef000c960 comp rx=0 tx=0).stop 2024-03-23T09:41:18.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.876+0000 7f1f05dfb640 1 -- 172.21.15.47:0/3322005321 shutdown_connections 2024-03-23T09:41:18.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.876+0000 7f1f05dfb640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1eec06ac00 0x7f1eec06d0c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.876+0000 7f1f05dfb640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f0015cd00 0x7f1f000b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.876+0000 7f1f05dfb640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f000cf210 0x7f1f000b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.876+0000 7f1f05dfb640 1 --2- 172.21.15.47:0/3322005321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5a90 0x7f1f000cc000 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:18.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.876+0000 7f1f05dfb640 1 -- 172.21.15.47:0/3322005321 >> 172.21.15.47:0/3322005321 conn(0x7f1f0006b160 msgr2=0x7f1f0015d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:18.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.876+0000 7f1f05dfb640 1 -- 172.21.15.47:0/3322005321 shutdown_connections 2024-03-23T09:41:18.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:18.876+0000 7f1f05dfb640 1 -- 172.21.15.47:0/3322005321 wait complete. 2024-03-23T09:41:18.886 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:41:18.886 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:41:18.886 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:41:18.886 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:41:18.886 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:41:18.886 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:41:18.886 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:41:18.887 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:41:18.887 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:41:18.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:18.887 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:41:18.901 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:41:18.903 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:41:18.903 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42660sec preferred_lft 42660sec 2024-03-23T09:41:18.903 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:41:18.903 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:18.903 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:41:18.903 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:18.903 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:41:18.903 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-23T09:41:18.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:18.904 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:41:18.904 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:41:18.904 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add name ceph-brx type bridge 2024-03-23T09:41:18.904 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr flush dev ceph-brx 2024-03-23T09:41:18.904 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set ceph-brx up 2024-03-23T09:41:18.904 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-23T09:41:18.904 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:41:19.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:19.052 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:41:19.077 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:41:19.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:19.079 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:41:19.150 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:41:19.150 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:41:19.150 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:41:19.150 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:41:19.151 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:41:19.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:19.151 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:41:19.152 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:41:19.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:41:19.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:41:19.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:41:19.152 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:41:19.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:19.413 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:41:19.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:19.427 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-03-23T09:41:19.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:19.481 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:41:19.481 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:41:19.482 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:41:19.482 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-23T09:41:19.482 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:41:19.563 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-23T09:41:19.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:19.564 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:41:19.564 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:41:19.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-23T09:41:19.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-23T09:41:19.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-23T09:41:19.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-23T09:41:19.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-23T09:41:19.564 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:41:19.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:19.699 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:41:19.699 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:41:19.699 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:41:19.699 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.0 master ceph-brx 2024-03-23T09:41:19.699 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:41:19.740 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:41:19.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:41:19.740 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:41:19.795 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:41:19.795 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:41:19.795 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:41:19.849 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:41:19.916 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:41:19.916 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:41:19.982 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:41:19.983 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:41:19.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:19.984 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:41:20.037 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:41:20.038 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:41:20.079 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:41:20.105 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:41:20.105 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:41:20.164 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:41:20.164+0000 7efcb38fa480 -1 init, newargv = 0x56245607fee0 newargc=15 2024-03-23T09:41:20.164 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[45603]: starting ceph client 2024-03-23T09:41:20.169 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[45603]: starting fuse 2024-03-23T09:41:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:41:20.177 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:41:20.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:20.177 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:41:20.232 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:41:20.232 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:41:20.232 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:41:20.232 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:41:20.232 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:41:20.232 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:41:20.232 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:41:20.232 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:41:20.232 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:41:20.233 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:41:20.234 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:41:20.234 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:41:20.348 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.45603.asok 2024-03-23T09:41:20.535 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45603.asok 2024-03-23T09:41:20.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:20.535 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45603.asok status 2024-03-23T09:41:20.671 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:41:20.671 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:41:20.671 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:41:20.671 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:41:20.671 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:41:20.671 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:41:20.671 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:41:20.671 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "45603", 2024-03-23T09:41:20.671 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:41:20.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:20.672 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:41:20.672 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:41:20.672 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4653, 2024-03-23T09:41:20.672 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:41:20.672 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:41:20.672 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:41:20.672 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4653 2024-03-23T09:41:20.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:20.672 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:41:20.672 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:41:20.672 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:41:20.672 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2832928664 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2832928664 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.4653 192.168.0.1:0/2832928664", 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/2832928664", 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 6, 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 21, 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:41:20.673 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:41:20.679 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:41:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.779+0000 7fe3a4e8e640 1 Processor -- start 2024-03-23T09:41:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.779+0000 7fe3a4e8e640 1 -- start start 2024-03-23T09:41:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.780+0000 7fe3a4e8e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe3a015cc40 0x7fe3a0160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.780+0000 7fe3a4e8e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a0160c90 0x7fe3a00cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.780+0000 7fe3a4e8e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3a00cdbf0 0x7fe3a00b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.780+0000 7fe3a4e8e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a00ce0a0 con 0x7fe3a00cdbf0 2024-03-23T09:41:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.780+0000 7fe3a4e8e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a00d09d0 con 0x7fe3a015cc40 2024-03-23T09:41:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.780+0000 7fe3a4e8e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00b5a80 con 0x7fe3a0160c90 2024-03-23T09:41:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.780+0000 7fe39ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3a00cdbf0 0x7fe3a00b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.780+0000 7fe39ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3a00cdbf0 0x7fe3a00b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41692/0 (socket says 172.21.15.47:41692) 2024-03-23T09:41:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.780+0000 7fe39ed9d640 1 -- 172.21.15.47:0/1096590974 learned_addr learned my addr 172.21.15.47:0/1096590974 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.780+0000 7fe39e59c640 1 --2- 172.21.15.47:0/1096590974 >> v2:172.21.15.175:3300/0 conn(0x7fe3a015cc40 0x7fe3a0160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.780+0000 7fe39dd9b640 1 --2- 172.21.15.47:0/1096590974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a0160c90 0x7fe3a00cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.780+0000 7fe39ed9d640 1 -- 172.21.15.47:0/1096590974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a0160c90 msgr2=0x7fe3a00cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.780+0000 7fe39ed9d640 1 --2- 172.21.15.47:0/1096590974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a0160c90 0x7fe3a00cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.781+0000 7fe39ed9d640 1 -- 172.21.15.47:0/1096590974 >> v2:172.21.15.175:3300/0 conn(0x7fe3a015cc40 msgr2=0x7fe3a0160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.781+0000 7fe39ed9d640 1 --2- 172.21.15.47:0/1096590974 >> v2:172.21.15.175:3300/0 conn(0x7fe3a015cc40 0x7fe3a0160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.781+0000 7fe39ed9d640 1 -- 172.21.15.47:0/1096590974 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a015d020 con 0x7fe3a00cdbf0 2024-03-23T09:41:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.781+0000 7fe39dd9b640 1 --2- 172.21.15.47:0/1096590974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a0160c90 0x7fe3a00cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.781+0000 7fe39ed9d640 1 --2- 172.21.15.47:0/1096590974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a00cdbf0 0x7fe3a00b4360 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3a008b280 tx=0x7fe39002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428636276882e71b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.781+0000 7fe39d59a640 1 -- 172.21.15.47:0/1096590974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39002ca70 con 0x7fe3a00cdbf0 2024-03-23T09:41:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.781+0000 7fe39d59a640 1 -- 172.21.15.47:0/1096590974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39002cbd0 con 0x7fe3a00cdbf0 2024-03-23T09:41:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.781+0000 7fe39d59a640 1 -- 172.21.15.47:0/1096590974 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3900343f0 con 0x7fe3a00cdbf0 2024-03-23T09:41:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.782+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/1096590974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a00cdbf0 msgr2=0x7fe3a00b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.782+0000 7fe3a4e8e640 1 --2- 172.21.15.47:0/1096590974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a00cdbf0 0x7fe3a00b4360 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3a008b280 tx=0x7fe39002bed0 comp rx=0 tx=0).stop 2024-03-23T09:41:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.782+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/1096590974 shutdown_connections 2024-03-23T09:41:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.782+0000 7fe3a4e8e640 1 --2- 172.21.15.47:0/1096590974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a00cdbf0 0x7fe3a00b4360 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.782+0000 7fe3a4e8e640 1 --2- 172.21.15.47:0/1096590974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a0160c90 0x7fe3a00cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.782+0000 7fe3a4e8e640 1 --2- 172.21.15.47:0/1096590974 >> v2:172.21.15.175:3300/0 conn(0x7fe3a015cc40 0x7fe3a0160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.782+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/1096590974 >> 172.21.15.47:0/1096590974 conn(0x7fe3a006b160 msgr2=0x7fe3a006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.782+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/1096590974 shutdown_connections 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.782+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/1096590974 wait complete. 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.783+0000 7fe3a4e8e640 1 Processor -- start 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.783+0000 7fe3a4e8e640 1 -- start start 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe3a4e8e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cdbf0 0x7fe3a015fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe3a4e8e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a015cc40 0x7fe3a00b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe3a4e8e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a0160c90 0x7fe3a00b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe3a4e8e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00a92b0 con 0x7fe3a0160c90 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe3a4e8e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0160180 con 0x7fe3a015cc40 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe3a4e8e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0159ae0 con 0x7fe3a00cdbf0 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe39ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a0160c90 0x7fe3a00b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe39ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a0160c90 0x7fe3a00b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41696/0 (socket says 172.21.15.47:41696) 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe39ed9d640 1 -- 172.21.15.47:0/3633120082 learned_addr learned my addr 172.21.15.47:0/3633120082 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe39e59c640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cdbf0 0x7fe3a015fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe39dd9b640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a015cc40 0x7fe3a00b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe39ed9d640 1 -- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cdbf0 msgr2=0x7fe3a015fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe39ed9d640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cdbf0 0x7fe3a015fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe39ed9d640 1 -- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a015cc40 msgr2=0x7fe3a00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.784+0000 7fe39ed9d640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a015cc40 0x7fe3a00b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.785+0000 7fe39ed9d640 1 -- 172.21.15.47:0/3633120082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe390009b30 con 0x7fe3a0160c90 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.785+0000 7fe39e59c640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cdbf0 0x7fe3a015fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.785+0000 7fe39dd9b640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a015cc40 0x7fe3a00b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.785+0000 7fe39ed9d640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a0160c90 0x7fe3a00b4580 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe390009f40 tx=0x7fe390004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.785+0000 7fe396ffd640 1 -- 172.21.15.47:0/3633120082 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390009510 con 0x7fe3a0160c90 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.785+0000 7fe396ffd640 1 -- 172.21.15.47:0/3633120082 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39003e450 con 0x7fe3a0160c90 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.785+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/3633120082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a00a94c0 con 0x7fe3a0160c90 2024-03-23T09:41:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.785+0000 7fe396ffd640 1 -- 172.21.15.47:0/3633120082 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39003e7e0 con 0x7fe3a0160c90 2024-03-23T09:41:20.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.785+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/3633120082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a0159700 con 0x7fe3a0160c90 2024-03-23T09:41:20.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.786+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/3633120082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a0072360 con 0x7fe3a0160c90 2024-03-23T09:41:20.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.786+0000 7fe396ffd640 1 -- 172.21.15.47:0/3633120082 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe39003e940 con 0x7fe3a0160c90 2024-03-23T09:41:20.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.787+0000 7fe396ffd640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe38006ac00 0x7fe38006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:20.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.787+0000 7fe396ffd640 1 -- 172.21.15.47:0/3633120082 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3900ac190 con 0x7fe3a0160c90 2024-03-23T09:41:20.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.787+0000 7fe39e59c640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe38006ac00 0x7fe38006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:20.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.787+0000 7fe39e59c640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe38006ac00 0x7fe38006d0c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe398006fd0 tx=0x7fe398008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:20.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:20.791+0000 7fe396ffd640 1 -- 172.21.15.47:0/3633120082 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe390077360 con 0x7fe3a0160c90 2024-03-23T09:41:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.009+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/3633120082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe3a0066be0 con 0x7fe3a0160c90 2024-03-23T09:41:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.009+0000 7fe396ffd640 1 -- 172.21.15.47:0/3633120082 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe39007a980 con 0x7fe3a0160c90 2024-03-23T09:41:21.010 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:41:21.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.011+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe38006ac00 msgr2=0x7fe38006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:21.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.011+0000 7fe3a4e8e640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe38006ac00 0x7fe38006d0c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe398006fd0 tx=0x7fe398008040 comp rx=0 tx=0).stop 2024-03-23T09:41:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.011+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/3633120082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a0160c90 msgr2=0x7fe3a00b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.011+0000 7fe3a4e8e640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a0160c90 0x7fe3a00b4580 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe390009f40 tx=0x7fe390004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.011+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/3633120082 shutdown_connections 2024-03-23T09:41:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.011+0000 7fe3a4e8e640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe38006ac00 0x7fe38006d0c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.011+0000 7fe3a4e8e640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a0160c90 0x7fe3a00b4580 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.011+0000 7fe3a4e8e640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a015cc40 0x7fe3a00b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.011+0000 7fe3a4e8e640 1 --2- 172.21.15.47:0/3633120082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cdbf0 0x7fe3a015fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.011+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/3633120082 >> 172.21.15.47:0/3633120082 conn(0x7fe3a006b160 msgr2=0x7fe3a00b5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.011+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/3633120082 shutdown_connections 2024-03-23T09:41:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:21.011+0000 7fe3a4e8e640 1 -- 172.21.15.47:0/3633120082 wait complete. 2024-03-23T09:41:21.022 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:41:21.022 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:41:21.022 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:41:21.022 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-23T09:41:21.023 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-23T09:41:21.023 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:41:21.023 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:41:21.023 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:41:21.023 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-23T09:41:21.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:21.023 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:41:21.038 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:41:21.038 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:41:21.038 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:41:21.038 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.038 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:41:21.038 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.038 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:41:21.038 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:21.038 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:41:21.038 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:41:21.038 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:21.038 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:41:21.038 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42657sec preferred_lft 42657sec 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.039 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:41:21.040 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.040 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:41:21.040 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:41:21.040 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link tentative 2024-03-23T09:41:21.040 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:21.040 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:41:21.103 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:41:21.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:21.105 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:41:21.160 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:41:21.161 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:41:21.161 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:41:21.161 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:41:21.161 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:41:21.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:21.161 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:41:21.161 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:41:21.161 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:41:21.161 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:41:21.162 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:41:21.162 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:41:21.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:21.255 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:41:21.269 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:41:21.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:21.270 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-03-23T09:41:21.323 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-23T09:41:21.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:21.324 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:41:21.324 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:41:21.324 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:41:21.324 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-23T09:41:21.324 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:41:21.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:21.404 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-23T09:41:21.470 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:41:21.470 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:41:21.470 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:41:21.470 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.470 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:41:21.470 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.470 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:41:21.470 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether f6:c6:4b:21:81:12 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-23T09:41:21.471 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-23T09:41:21.471 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.471 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::f4c6:4bff:fe21:8112/64 scope link 2024-03-23T09:41:21.471 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:21.472 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-23T09:41:21.538 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:41:21.539 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:41:21.539 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:41:21.539 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.539 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:41:21.539 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.539 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:41:21.539 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether f6:c6:4b:21:81:12 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-23T09:41:21.539 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-23T09:41:21.539 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.539 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::f4c6:4bff:fe21:8112/64 scope link 2024-03-23T09:41:21.539 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:21.540 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-23T09:41:21.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:21.541 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:41:21.541 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:41:21.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-23T09:41:21.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-23T09:41:21.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-23T09:41:21.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-23T09:41:21.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-23T09:41:21.541 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:41:21.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:21.664 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:41:21.664 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:41:21.664 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.1 up 2024-03-23T09:41:21.664 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.1 master ceph-brx 2024-03-23T09:41:21.664 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:41:21.705 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-03-23T09:41:21.705 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:41:21.705 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-23T09:41:21.760 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-23T09:41:21.760 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:41:21.760 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-23T09:41:21.814 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:41:21.881 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:41:21.881 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:41:21.947 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:41:21.948 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:41:21.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:21.949 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:41:22.003 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:41:22.003 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-03-23T09:41:22.003 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-03-23T09:41:22.045 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:41:22.071 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:41:22.071 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:41:22.131 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi047.stderr:2024-03-23T09:41:22.130+0000 7fd2dec91480 -1 init, newargv = 0x56218678d1c0 newargc=15 2024-03-23T09:41:22.131 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi047.stderr:ceph-fuse[46098]: starting ceph client 2024-03-23T09:41:22.135 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi047.stderr:ceph-fuse[46098]: starting fuse 2024-03-23T09:41:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:41:22.143 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:41:22.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:22.146 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:41:22.200 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:41:22.200 INFO:teuthology.orchestra.run.smithi047.stdout:44 2024-03-23T09:41:22.200 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-03-23T09:41:22.200 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:41:22.200 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:41:22.200 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:41:22.200 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:41:22.201 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:41:22.202 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:41:22.202 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:41:22.202 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:41:22.202 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:41:22.202 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:41:22.202 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.2")) 2024-03-23T09:41:22.202 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:41:22.316 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.2.46098.asok 2024-03-23T09:41:22.502 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.46098.asok 2024-03-23T09:41:22.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:22.503 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.46098.asok status 2024-03-23T09:41:22.640 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:41:22.640 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:41:22.640 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:41:22.640 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:41:22.640 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "2", 2024-03-23T09:41:22.640 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:41:22.640 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-03-23T09:41:22.640 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "46098", 2024-03-23T09:41:22.640 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:41:22.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4665, 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4665 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1039480553 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1039480553 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.4665 192.168.0.2:0/1039480553", 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.2:0/1039480553", 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 6, 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 21, 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:41:22.642 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:41:22.648 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:41:22.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.748+0000 7f0eb195c640 1 Processor -- start 2024-03-23T09:41:22.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.748+0000 7f0eb195c640 1 -- start start 2024-03-23T09:41:22.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.749+0000 7f0eb195c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0eac0cf210 0x7f0eac0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:22.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.749+0000 7f0eb195c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eac0b5a90 0x7f0eac0b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:22.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.749+0000 7f0eb195c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eac0cfbd0 con 0x7f0eac15cd00 2024-03-23T09:41:22.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.749+0000 7f0eb195c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eac15d0e0 con 0x7f0eac0cf210 2024-03-23T09:41:22.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.749+0000 7f0eb195c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0d1540 con 0x7f0eac0b5a90 2024-03-23T09:41:22.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.749+0000 7f0eaa7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0eac0cf210 0x7f0eac0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:22.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.749+0000 7f0eaa7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0eac0cf210 0x7f0eac0cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41710/0 (socket says 172.21.15.47:41710) 2024-03-23T09:41:22.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.749+0000 7f0eaa7fc640 1 -- 172.21.15.47:0/590650090 learned_addr learned my addr 172.21.15.47:0/590650090 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:22.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.750+0000 7f0eab7fe640 1 --2- 172.21.15.47:0/590650090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eac0b5a90 0x7f0eac0b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:22.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.750+0000 7f0eaa7fc640 1 -- 172.21.15.47:0/590650090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eac0b5a90 msgr2=0x7f0eac0b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:22.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.750+0000 7f0eaa7fc640 1 --2- 172.21.15.47:0/590650090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eac0b5a90 0x7f0eac0b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:22.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.750+0000 7f0eaa7fc640 1 -- 172.21.15.47:0/590650090 >> v1:172.21.15.175:6789/0 conn(0x7f0eac15cd00 legacy=0x7f0eac0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:22.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.750+0000 7f0eaa7fc640 1 -- 172.21.15.47:0/590650090 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac0d1820 con 0x7f0eac0cf210 2024-03-23T09:41:22.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.750+0000 7f0eaa7fc640 1 --2- 172.21.15.47:0/590650090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0cf210 0x7f0eac0cf690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009870 tx=0x7f0e9402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f73653c242aa17f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:22.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.750+0000 7f0ea9ffb640 1 -- 172.21.15.47:0/590650090 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e940317c0 con 0x7f0eac0cf210 2024-03-23T09:41:22.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.751+0000 7f0ea9ffb640 1 -- 172.21.15.47:0/590650090 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e94031920 con 0x7f0eac0cf210 2024-03-23T09:41:22.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.751+0000 7f0ea9ffb640 1 -- 172.21.15.47:0/590650090 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e94031cb0 con 0x7f0eac0cf210 2024-03-23T09:41:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.751+0000 7f0eb195c640 1 -- 172.21.15.47:0/590650090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0cf210 msgr2=0x7f0eac0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.751+0000 7f0eb195c640 1 --2- 172.21.15.47:0/590650090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0cf210 0x7f0eac0cf690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009870 tx=0x7f0e9402b860 comp rx=0 tx=0).stop 2024-03-23T09:41:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.751+0000 7f0eb195c640 1 -- 172.21.15.47:0/590650090 shutdown_connections 2024-03-23T09:41:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.751+0000 7f0eb195c640 1 --2- 172.21.15.47:0/590650090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eac0b5a90 0x7f0eac0b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.751+0000 7f0eb195c640 1 --2- 172.21.15.47:0/590650090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0cf210 0x7f0eac0cf690 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.751+0000 7f0eb195c640 1 -- 172.21.15.47:0/590650090 >> 172.21.15.47:0/590650090 conn(0x7f0eac06b160 msgr2=0x7f0eac06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.751+0000 7f0eb195c640 1 -- 172.21.15.47:0/590650090 shutdown_connections 2024-03-23T09:41:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.751+0000 7f0eb195c640 1 -- 172.21.15.47:0/590650090 wait complete. 2024-03-23T09:41:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.752+0000 7f0eb195c640 1 Processor -- start 2024-03-23T09:41:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.752+0000 7f0eb195c640 1 -- start start 2024-03-23T09:41:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.753+0000 7f0eb195c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eac0b5a90 0x7f0eac155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.753+0000 7f0eb195c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0cf210 0x7f0eac157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.753+0000 7f0eb195c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eac15cd00 0x7f0eac158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.753+0000 7f0eb195c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0ca680 con 0x7f0eac0cf210 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.753+0000 7f0eb195c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0ccae0 con 0x7f0eac15cd00 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.753+0000 7f0eb195c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac1586f0 con 0x7f0eac0b5a90 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.753+0000 7f0eaa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0cf210 0x7f0eac157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.753+0000 7f0eaa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0cf210 0x7f0eac157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41712/0 (socket says 172.21.15.47:41712) 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.753+0000 7f0eaa7fc640 1 -- 172.21.15.47:0/2153758926 learned_addr learned my addr 172.21.15.47:0/2153758926 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.753+0000 7f0eab7fe640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eac15cd00 0x7f0eac158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.753+0000 7f0eaa7fc640 1 -- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eac0b5a90 msgr2=0x7f0eac155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.753+0000 7f0eaaffd640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eac0b5a90 0x7f0eac155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.754+0000 7f0eaa7fc640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eac0b5a90 0x7f0eac155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.754+0000 7f0eaa7fc640 1 -- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eac15cd00 msgr2=0x7f0eac158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.754+0000 7f0eaa7fc640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eac15cd00 0x7f0eac158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:22.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.754+0000 7f0eaa7fc640 1 -- 172.21.15.47:0/2153758926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e940094d0 con 0x7f0eac0cf210 2024-03-23T09:41:22.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.754+0000 7f0eab7fe640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eac15cd00 0x7f0eac158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:22.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.754+0000 7f0eaaffd640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eac0b5a90 0x7f0eac155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:22.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.754+0000 7f0eaa7fc640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0cf210 0x7f0eac157b50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0e94000c00 tx=0x7f0e940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:22.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.754+0000 7f0ea3fff640 1 -- 172.21.15.47:0/2153758926 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e940329a0 con 0x7f0eac0cf210 2024-03-23T09:41:22.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.754+0000 7f0ea3fff640 1 -- 172.21.15.47:0/2153758926 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e94032b20 con 0x7f0eac0cf210 2024-03-23T09:41:22.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.754+0000 7f0eb195c640 1 -- 172.21.15.47:0/2153758926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eac1589d0 con 0x7f0eac0cf210 2024-03-23T09:41:22.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.754+0000 7f0ea3fff640 1 -- 172.21.15.47:0/2153758926 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e940453f0 con 0x7f0eac0cf210 2024-03-23T09:41:22.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.754+0000 7f0eb195c640 1 -- 172.21.15.47:0/2153758926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eac148570 con 0x7f0eac0cf210 2024-03-23T09:41:22.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.755+0000 7f0eb195c640 1 -- 172.21.15.47:0/2153758926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eac072360 con 0x7f0eac0cf210 2024-03-23T09:41:22.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.755+0000 7f0ea3fff640 1 -- 172.21.15.47:0/2153758926 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0e9404d3f0 con 0x7f0eac0cf210 2024-03-23T09:41:22.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.756+0000 7f0ea3fff640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0e8c06ac00 0x7f0e8c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:22.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.756+0000 7f0ea3fff640 1 -- 172.21.15.47:0/2153758926 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e940ab3f0 con 0x7f0eac0cf210 2024-03-23T09:41:22.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.756+0000 7f0eaaffd640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0e8c06ac00 0x7f0e8c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:22.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.756+0000 7f0eaaffd640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0e8c06ac00 0x7f0e8c06d0c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c007e90 tx=0x7f0e9c0073a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:22.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.760+0000 7f0ea3fff640 1 -- 172.21.15.47:0/2153758926 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0e94076540 con 0x7f0eac0cf210 2024-03-23T09:41:22.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.979+0000 7f0eb195c640 1 -- 172.21.15.47:0/2153758926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0eac148760 con 0x7f0eac0cf210 2024-03-23T09:41:22.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.979+0000 7f0ea3fff640 1 -- 172.21.15.47:0/2153758926 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0eac148760 con 0x7f0eac0cf210 2024-03-23T09:41:22.980 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:41:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.981+0000 7f0eb195c640 1 -- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0e8c06ac00 msgr2=0x7f0e8c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.981+0000 7f0eb195c640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0e8c06ac00 0x7f0e8c06d0c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c007e90 tx=0x7f0e9c0073a0 comp rx=0 tx=0).stop 2024-03-23T09:41:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.981+0000 7f0eb195c640 1 -- 172.21.15.47:0/2153758926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0cf210 msgr2=0x7f0eac157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.981+0000 7f0eb195c640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0cf210 0x7f0eac157b50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0e94000c00 tx=0x7f0e940046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.981+0000 7f0eb195c640 1 -- 172.21.15.47:0/2153758926 shutdown_connections 2024-03-23T09:41:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.981+0000 7f0eb195c640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0e8c06ac00 0x7f0e8c06d0c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.981+0000 7f0eb195c640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eac15cd00 0x7f0eac158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.981+0000 7f0eb195c640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0cf210 0x7f0eac157b50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.981+0000 7f0eb195c640 1 --2- 172.21.15.47:0/2153758926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eac0b5a90 0x7f0eac155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.981+0000 7f0eb195c640 1 -- 172.21.15.47:0/2153758926 >> 172.21.15.47:0/2153758926 conn(0x7f0eac06b160 msgr2=0x7f0eac15d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.981+0000 7f0eb195c640 1 -- 172.21.15.47:0/2153758926 shutdown_connections 2024-03-23T09:41:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:22.981+0000 7f0eb195c640 1 -- 172.21.15.47:0/2153758926 wait complete. 2024-03-23T09:41:22.992 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:41:22.992 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:41:22.992 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-03-23T09:41:22.992 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-23T09:41:22.992 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-23T09:41:22.992 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:41:22.992 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:41:22.992 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:41:22.993 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-23T09:41:22.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:22.993 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-03-23T09:41:23.012 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:41:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:41:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:41:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-03-23T09:41:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:23.012 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:41:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-03-23T09:41:23.012 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:41:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-03-23T09:41:23.013 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:41:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:23.013 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:41:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-23T09:41:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42706sec preferred_lft 42706sec 2024-03-23T09:41:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-03-23T09:41:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:23.013 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-23T09:41:23.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:23.014 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec bash -c ' 2024-03-23T09:41:23.014 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-03-23T09:41:23.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add name ceph-brx type bridge 2024-03-23T09:41:23.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip addr flush dev ceph-brx 2024-03-23T09:41:23.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set ceph-brx up 2024-03-23T09:41:23.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-23T09:41:23.014 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-03-23T09:41:23.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:23.177 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:41:23.204 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-03-23T09:41:23.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:23.207 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-03-23T09:41:23.278 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-03-23T09:41:23.278 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:41:23.278 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T09:41:23.278 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T09:41:23.278 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:41:23.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:23.279 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec bash -c ' 2024-03-23T09:41:23.279 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-03-23T09:41:23.279 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T09:41:23.279 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T09:41:23.279 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T09:41:23.279 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-03-23T09:41:23.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:23.538 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-03-23T09:41:23.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:23.553 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list-id 2024-03-23T09:41:23.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:23.607 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec bash -c ' 2024-03-23T09:41:23.607 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-03-23T09:41:23.607 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-23T09:41:23.607 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-23T09:41:23.607 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-03-23T09:41:23.690 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-23T09:41:23.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:23.691 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec bash -c ' 2024-03-23T09:41:23.691 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-03-23T09:41:23.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-23T09:41:23.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-23T09:41:23.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-23T09:41:23.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-23T09:41:23.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-23T09:41:23.691 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-03-23T09:41:23.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:23.859 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec bash -c ' 2024-03-23T09:41:23.859 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-03-23T09:41:23.859 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-03-23T09:41:23.859 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set dev brx.0 master ceph-brx 2024-03-23T09:41:23.859 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-03-23T09:41:23.900 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-23T09:41:23.900 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:41:23.900 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-23T09:41:23.955 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-23T09:41:23.955 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:41:23.955 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-23T09:41:24.010 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-03-23T09:41:24.078 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:41:24.078 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:41:24.144 INFO:teuthology.orchestra.run.smithi175.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:41:24.146 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:41:24.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:24.146 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-03-23T09:41:24.201 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:41:24.201 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-23T09:41:24.244 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-03-23T09:41:24.270 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:41:24.270 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:41:24.329 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-03-23T09:41:24.328+0000 7fc6516bb480 -1 init, newargv = 0x55bc2026cf50 newargc=15 2024-03-23T09:41:24.329 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[43923]: starting ceph client 2024-03-23T09:41:24.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[43923]: starting fuse 2024-03-23T09:41:24.338 INFO:teuthology.orchestra.run.smithi175.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:41:24.339 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:41:24.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:24.340 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-03-23T09:41:24.395 INFO:teuthology.orchestra.run.smithi175.stdout:43 2024-03-23T09:41:24.395 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:41:24.395 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:41:24.395 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-03-23T09:41:24.395 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-03-23T09:41:24.395 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-03-23T09:41:24.395 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:41:24.396 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-03-23T09:41:24.397 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-03-23T09:41:24.397 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-03-23T09:41:24.397 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:41:24.397 DEBUG:teuthology.orchestra.run.smithi175:> 2024-03-23T09:41:24.397 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-03-23T09:41:24.397 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-03-23T09:41:24.511 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.43923.asok 2024-03-23T09:41:24.699 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.43923.asok 2024-03-23T09:41:24.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:24.699 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.43923.asok status 2024-03-23T09:41:24.835 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "43923", 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 4609, 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-03-23T09:41:24.836 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 4609 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3485840004 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3485840004 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.4609 192.168.0.1:0/3485840004", 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/3485840004", 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 6, 2024-03-23T09:41:24.837 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 21, 2024-03-23T09:41:24.838 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:41:24.838 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-03-23T09:41:24.838 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-03-23T09:41:24.838 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-03-23T09:41:24.844 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:41:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.956+0000 7ff47184a640 1 Processor -- start 2024-03-23T09:41:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.956+0000 7ff47184a640 1 -- start start 2024-03-23T09:41:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.957+0000 7ff47184a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff46c160070 0x7ff46c098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.957+0000 7ff47184a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff46c0b1580 0x7ff46c098f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.957+0000 7ff47184a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff46c0cee20 con 0x7ff46c0cf2b0 2024-03-23T09:41:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.957+0000 7ff47184a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff46c0b31b0 con 0x7ff46c160070 2024-03-23T09:41:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.957+0000 7ff47184a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff46c0ceb10 con 0x7ff46c0b1580 2024-03-23T09:41:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.958+0000 7ff46affd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff46c160070 0x7ff46c098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.958+0000 7ff46affd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff46c160070 0x7ff46c098a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56346/0 (socket says 172.21.15.47:56346) 2024-03-23T09:41:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.958+0000 7ff46affd640 1 -- 172.21.15.47:0/1437739024 learned_addr learned my addr 172.21.15.47:0/1437739024 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.958+0000 7ff46a7fc640 1 --2- 172.21.15.47:0/1437739024 >> v2:172.21.15.175:3300/0 conn(0x7ff46c0b1580 0x7ff46c098f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.958+0000 7ff46affd640 1 -- 172.21.15.47:0/1437739024 >> v2:172.21.15.175:3300/0 conn(0x7ff46c0b1580 msgr2=0x7ff46c098f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.958+0000 7ff46affd640 1 --2- 172.21.15.47:0/1437739024 >> v2:172.21.15.175:3300/0 conn(0x7ff46c0b1580 0x7ff46c098f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.958+0000 7ff46affd640 1 -- 172.21.15.47:0/1437739024 >> v1:172.21.15.47:6789/0 conn(0x7ff46c0cf2b0 legacy=0x7ff46c0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.958+0000 7ff46affd640 1 -- 172.21.15.47:0/1437739024 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c160450 con 0x7ff46c160070 2024-03-23T09:41:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.958+0000 7ff469ffb640 1 -- 172.21.15.47:0/1437739024 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466664027 0 0) 0x7ff46c0cee20 con 0x7ff46c0cf2b0 2024-03-23T09:41:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.959+0000 7ff46affd640 1 --2- 172.21.15.47:0/1437739024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff46c160070 0x7ff46c098a00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0c4250 tx=0x7ff46402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e393d10b6cb6c9c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.959+0000 7ff469ffb640 1 -- 172.21.15.47:0/1437739024 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464030860 con 0x7ff46c160070 2024-03-23T09:41:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.959+0000 7ff469ffb640 1 -- 172.21.15.47:0/1437739024 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4640309c0 con 0x7ff46c160070 2024-03-23T09:41:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.959+0000 7ff469ffb640 1 -- 172.21.15.47:0/1437739024 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464030d50 con 0x7ff46c160070 2024-03-23T09:41:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.959+0000 7ff47184a640 1 -- 172.21.15.47:0/1437739024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff46c160070 msgr2=0x7ff46c098a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.959+0000 7ff47184a640 1 --2- 172.21.15.47:0/1437739024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff46c160070 0x7ff46c098a00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0c4250 tx=0x7ff46402b9a0 comp rx=0 tx=0).stop 2024-03-23T09:41:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.959+0000 7ff47184a640 1 -- 172.21.15.47:0/1437739024 shutdown_connections 2024-03-23T09:41:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.959+0000 7ff47184a640 1 --2- 172.21.15.47:0/1437739024 >> v2:172.21.15.175:3300/0 conn(0x7ff46c0b1580 0x7ff46c098f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.959+0000 7ff47184a640 1 --2- 172.21.15.47:0/1437739024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff46c160070 0x7ff46c098a00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.959+0000 7ff47184a640 1 -- 172.21.15.47:0/1437739024 >> 172.21.15.47:0/1437739024 conn(0x7ff46c0b2240 msgr2=0x7ff46c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.960+0000 7ff47184a640 1 -- 172.21.15.47:0/1437739024 shutdown_connections 2024-03-23T09:41:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.960+0000 7ff47184a640 1 -- 172.21.15.47:0/1437739024 wait complete. 2024-03-23T09:41:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.961+0000 7ff47184a640 1 Processor -- start 2024-03-23T09:41:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.961+0000 7ff47184a640 1 -- start start 2024-03-23T09:41:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.961+0000 7ff47184a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff46c0b1580 0x7ff46c14bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.961+0000 7ff47184a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff46c0cf2b0 0x7ff46c14b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff47184a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff46c160070 0x7ff46c151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff47184a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c14e1c0 con 0x7ff46c0b1580 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff47184a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c14e4a0 con 0x7ff46c160070 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff47184a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c14e780 con 0x7ff46c0cf2b0 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff46affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff46c0b1580 0x7ff46c14bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff46affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff46c0b1580 0x7ff46c14bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56362/0 (socket says 172.21.15.47:56362) 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff46affd640 1 -- 172.21.15.47:0/3567925349 learned_addr learned my addr 172.21.15.47:0/3567925349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff46a7fc640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff46c0cf2b0 0x7ff46c14b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff46b7fe640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff46c160070 0x7ff46c151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff46affd640 1 -- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff46c0cf2b0 msgr2=0x7ff46c14b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff46affd640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff46c0cf2b0 0x7ff46c14b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff46affd640 1 -- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff46c160070 msgr2=0x7ff46c151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff46affd640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff46c160070 0x7ff46c151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff46affd640 1 -- 172.21.15.47:0/3567925349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4640094d0 con 0x7ff46c0b1580 2024-03-23T09:41:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff46affd640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff46c0b1580 0x7ff46c14bbe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff46402b9e0 tx=0x7ff464004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:24.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff463fff640 1 -- 172.21.15.47:0/3567925349 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464039a90 con 0x7ff46c0b1580 2024-03-23T09:41:24.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff463fff640 1 -- 172.21.15.47:0/3567925349 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff464039c10 con 0x7ff46c0b1580 2024-03-23T09:41:24.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff47184a640 1 -- 172.21.15.47:0/3567925349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46c14ea60 con 0x7ff46c0b1580 2024-03-23T09:41:24.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff463fff640 1 -- 172.21.15.47:0/3567925349 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464043410 con 0x7ff46c0b1580 2024-03-23T09:41:24.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.962+0000 7ff47184a640 1 -- 172.21.15.47:0/3567925349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46c151ea0 con 0x7ff46c0b1580 2024-03-23T09:41:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.964+0000 7ff47184a640 1 -- 172.21.15.47:0/3567925349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46c06cbe0 con 0x7ff46c0b1580 2024-03-23T09:41:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.964+0000 7ff463fff640 1 -- 172.21.15.47:0/3567925349 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff46402f3b0 con 0x7ff46c0b1580 2024-03-23T09:41:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.964+0000 7ff463fff640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff44c06ac00 0x7ff44c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.964+0000 7ff463fff640 1 -- 172.21.15.47:0/3567925349 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4640ab500 con 0x7ff46c0b1580 2024-03-23T09:41:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.965+0000 7ff46a7fc640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff44c06ac00 0x7ff44c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.965+0000 7ff46a7fc640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff44c06ac00 0x7ff44c06d0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff46c15df90 tx=0x7ff454009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:24.969+0000 7ff463fff640 1 -- 172.21.15.47:0/3567925349 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff4640768a0 con 0x7ff46c0b1580 2024-03-23T09:41:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.187+0000 7ff47184a640 1 -- 172.21.15.47:0/3567925349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff46c0672f0 con 0x7ff46c0b1580 2024-03-23T09:41:25.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.187+0000 7ff463fff640 1 -- 172.21.15.47:0/3567925349 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff464048060 con 0x7ff46c0b1580 2024-03-23T09:41:25.188 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:41:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.189+0000 7ff47184a640 1 -- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff44c06ac00 msgr2=0x7ff44c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.189+0000 7ff47184a640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff44c06ac00 0x7ff44c06d0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff46c15df90 tx=0x7ff454009190 comp rx=0 tx=0).stop 2024-03-23T09:41:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.189+0000 7ff47184a640 1 -- 172.21.15.47:0/3567925349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff46c0b1580 msgr2=0x7ff46c14bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.189+0000 7ff47184a640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff46c0b1580 0x7ff46c14bbe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff46402b9e0 tx=0x7ff464004730 comp rx=0 tx=0).stop 2024-03-23T09:41:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.189+0000 7ff47184a640 1 -- 172.21.15.47:0/3567925349 shutdown_connections 2024-03-23T09:41:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.189+0000 7ff47184a640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff44c06ac00 0x7ff44c06d0c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.189+0000 7ff47184a640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff46c160070 0x7ff46c151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.189+0000 7ff47184a640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff46c0cf2b0 0x7ff46c14b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.189+0000 7ff47184a640 1 --2- 172.21.15.47:0/3567925349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff46c0b1580 0x7ff46c14bbe0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.189+0000 7ff47184a640 1 -- 172.21.15.47:0/3567925349 >> 172.21.15.47:0/3567925349 conn(0x7ff46c0b2240 msgr2=0x7ff46c0af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:25.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.189+0000 7ff47184a640 1 -- 172.21.15.47:0/3567925349 shutdown_connections 2024-03-23T09:41:25.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:25.189+0000 7ff47184a640 1 -- 172.21.15.47:0/3567925349 wait complete. 2024-03-23T09:41:25.200 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:41:25.200 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:41:25.200 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-03-23T09:41:25.200 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-23T09:41:25.200 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-23T09:41:25.200 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:41:25.200 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:41:25.200 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:41:25.200 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-23T09:41:25.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:25.201 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-03-23T09:41:25.217 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:41:25.217 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:41:25.217 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:41:25.218 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.218 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-03-23T09:41:25.218 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.218 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:41:25.218 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:25.218 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-03-23T09:41:25.218 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:41:25.218 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:25.218 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-03-23T09:41:25.218 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:41:25.218 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:25.218 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:41:25.219 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:25.219 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-23T09:41:25.219 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42704sec preferred_lft 42704sec 2024-03-23T09:41:25.219 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-03-23T09:41:25.219 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.219 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:41:25.219 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 9e:01:e6:65:b5:66 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:41:25.219 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:41:25.219 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.219 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::6c25:6cff:fe07:890d/64 scope link 2024-03-23T09:41:25.219 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.219 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:41:25.219 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 9e:01:e6:65:b5:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-23T09:41:25.220 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::9c01:e6ff:fe65:b566/64 scope link tentative 2024-03-23T09:41:25.220 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:25.220 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:41:25.283 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-03-23T09:41:25.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:25.285 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-03-23T09:41:25.340 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-03-23T09:41:25.340 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:41:25.340 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T09:41:25.340 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T09:41:25.340 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:41:25.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:25.341 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec bash -c ' 2024-03-23T09:41:25.341 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-03-23T09:41:25.341 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T09:41:25.341 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T09:41:25.341 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T09:41:25.341 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-03-23T09:41:25.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:25.437 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-03-23T09:41:25.452 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-23T09:41:25.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:25.452 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list-id 2024-03-23T09:41:25.507 INFO:teuthology.orchestra.run.smithi175.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-23T09:41:25.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:25.507 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec bash -c ' 2024-03-23T09:41:25.507 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-03-23T09:41:25.508 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-23T09:41:25.508 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-23T09:41:25.508 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-03-23T09:41:25.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:25.589 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-23T09:41:25.656 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:41:25.656 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:41:25.656 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:41:25.656 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.656 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-03-23T09:41:25.656 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.656 INFO:teuthology.orchestra.run.smithi175.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:41:25.657 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 1e:9c:51:2c:92:cb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-23T09:41:25.657 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-23T09:41:25.657 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.657 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::1c9c:51ff:fe2c:92cb/64 scope link 2024-03-23T09:41:25.657 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:25.658 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-23T09:41:25.725 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:41:25.732 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:41:25.732 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:41:25.732 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.732 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-03-23T09:41:25.732 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.732 INFO:teuthology.orchestra.run.smithi175.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:41:25.732 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 1e:9c:51:2c:92:cb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-23T09:41:25.732 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-23T09:41:25.732 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.732 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::1c9c:51ff:fe2c:92cb/64 scope link 2024-03-23T09:41:25.732 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:41:25.733 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-23T09:41:25.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:25.734 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec bash -c ' 2024-03-23T09:41:25.734 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-03-23T09:41:25.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-23T09:41:25.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-23T09:41:25.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-23T09:41:25.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-23T09:41:25.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-23T09:41:25.734 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-03-23T09:41:25.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:25.861 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec bash -c ' 2024-03-23T09:41:25.861 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-03-23T09:41:25.861 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.1 up 2024-03-23T09:41:25.861 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set dev brx.1 master ceph-brx 2024-03-23T09:41:25.861 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-03-23T09:41:25.903 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-03-23T09:41:25.903 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:41:25.903 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-23T09:41:25.958 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-23T09:41:25.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:41:25.958 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-23T09:41:26.012 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-03-23T09:41:26.080 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:41:26.080 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:41:26.146 INFO:teuthology.orchestra.run.smithi175.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:41:26.147 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:41:26.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:26.148 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-03-23T09:41:26.203 INFO:teuthology.orchestra.run.smithi175.stdout:43 2024-03-23T09:41:26.203 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-03-23T09:41:26.203 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-03-23T09:41:26.245 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-03-23T09:41:26.271 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:41:26.271 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:41:26.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi175.stderr:2024-03-23T09:41:26.329+0000 7f6b5ac38480 -1 init, newargv = 0x55a083ca2110 newargc=15 2024-03-23T09:41:26.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi175.stderr:ceph-fuse[44376]: starting ceph client 2024-03-23T09:41:26.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi175.stderr:ceph-fuse[44376]: starting fuse 2024-03-23T09:41:26.341 INFO:teuthology.orchestra.run.smithi175.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:41:26.342 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:41:26.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:26.342 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-03-23T09:41:26.397 INFO:teuthology.orchestra.run.smithi175.stdout:43 2024-03-23T09:41:26.397 INFO:teuthology.orchestra.run.smithi175.stdout:44 2024-03-23T09:41:26.397 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-03-23T09:41:26.397 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:41:26.397 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:41:26.398 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:41:26.399 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:41:26.399 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-03-23T09:41:26.399 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-03-23T09:41:26.399 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-03-23T09:41:26.399 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:41:26.399 DEBUG:teuthology.orchestra.run.smithi175:> 2024-03-23T09:41:26.399 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.3")) 2024-03-23T09:41:26.399 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-03-23T09:41:26.513 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.3.44376.asok 2024-03-23T09:41:26.699 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.44376.asok 2024-03-23T09:41:26.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:26.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.44376.asok status 2024-03-23T09:41:26.836 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-03-23T09:41:26.836 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-03-23T09:41:26.836 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:41:26.836 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:41:26.836 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "3", 2024-03-23T09:41:26.836 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-03-23T09:41:26.836 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-03-23T09:41:26.836 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "44376", 2024-03-23T09:41:26.836 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 4463, 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 4463 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.2:0", 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3037399047 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-23T09:41:26.837 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-03-23T09:41:26.838 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-03-23T09:41:26.838 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.2:0", 2024-03-23T09:41:26.838 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3037399047 2024-03-23T09:41:26.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-23T09:41:26.838 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.4463 192.168.0.2:0/3037399047", 2024-03-23T09:41:26.838 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.2:0/3037399047", 2024-03-23T09:41:26.838 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-03-23T09:41:26.838 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 6, 2024-03-23T09:41:26.838 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 21, 2024-03-23T09:41:26.838 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:41:26.838 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-03-23T09:41:26.838 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-03-23T09:41:26.838 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-03-23T09:41:26.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:26.845 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:41:26.845 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:41:26.863 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:41:26.865 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:41:26.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:26.866 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:41:26.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:26.937 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:41:26.937 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.2 2024-03-23T09:41:26.994 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:41:26.994 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-03-23T09:41:26.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:26.995 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-23T09:41:27.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:27.070 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-03-23T09:41:27.070 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-23T09:41:27.086 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-03-23T09:41:27.086 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-23T09:41:27.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:27.086 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-23T09:41:27.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:27.153 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-03-23T09:41:27.153 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.3 2024-03-23T09:41:27.209 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-03-23T09:41:27.209 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-03-23T09:41:27.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:27.209 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-23T09:41:27.276 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-23T09:41:27.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:27.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.427+0000 7fabbbc77640 1 Processor -- start 2024-03-23T09:41:27.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.428+0000 7fabbbc77640 1 -- start start 2024-03-23T09:41:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.428+0000 7fabbbc77640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fabb4069ce0 0x7fabb40ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.428+0000 7fabbbc77640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fabb406d3a0 0x7fabb415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.428+0000 7fabbbc77640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb415d210 con 0x7fabb40ce830 2024-03-23T09:41:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.428+0000 7fabbbc77640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb406a2c0 con 0x7fabb406d3a0 2024-03-23T09:41:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.428+0000 7fabbbc77640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb406a5a0 con 0x7fabb4069ce0 2024-03-23T09:41:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.428+0000 7fabba214640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fabb406d3a0 0x7fabb415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.429+0000 7fabba214640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fabb406d3a0 0x7fabb415d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56370/0 (socket says 172.21.15.47:56370) 2024-03-23T09:41:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.429+0000 7fabba214640 1 -- 172.21.15.47:0/2665512705 learned_addr learned my addr 172.21.15.47:0/2665512705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.429+0000 7fabb9212640 1 --2- 172.21.15.47:0/2665512705 >> v2:172.21.15.175:3300/0 conn(0x7fabb4069ce0 0x7fabb40ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.429+0000 7fabba214640 1 -- 172.21.15.47:0/2665512705 >> v2:172.21.15.175:3300/0 conn(0x7fabb4069ce0 msgr2=0x7fabb40ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.429+0000 7fabba214640 1 --2- 172.21.15.47:0/2665512705 >> v2:172.21.15.175:3300/0 conn(0x7fabb4069ce0 0x7fabb40ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.429+0000 7fabba214640 1 -- 172.21.15.47:0/2665512705 >> v1:172.21.15.175:6789/0 conn(0x7fabb40ce830 legacy=0x7fabb40cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.429+0000 7fabba214640 1 -- 172.21.15.47:0/2665512705 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb406a880 con 0x7fabb406d3a0 2024-03-23T09:41:27.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.430+0000 7fabba214640 1 --2- 172.21.15.47:0/2665512705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb406d3a0 0x7fabb415d6a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fabb40cd010 tx=0x7fabb002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5d4effc8685152 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:27.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.430+0000 7fabb8a11640 1 -- 172.21.15.47:0/2665512705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb002cb20 con 0x7fabb406d3a0 2024-03-23T09:41:27.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.430+0000 7fabb8a11640 1 -- 172.21.15.47:0/2665512705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb002cc80 con 0x7fabb406d3a0 2024-03-23T09:41:27.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.430+0000 7fabb8a11640 1 -- 172.21.15.47:0/2665512705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb00343f0 con 0x7fabb406d3a0 2024-03-23T09:41:27.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.430+0000 7fabbbc77640 1 -- 172.21.15.47:0/2665512705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb406d3a0 msgr2=0x7fabb415d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:27.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.430+0000 7fabbbc77640 1 --2- 172.21.15.47:0/2665512705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb406d3a0 0x7fabb415d6a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fabb40cd010 tx=0x7fabb002bf80 comp rx=0 tx=0).stop 2024-03-23T09:41:27.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.430+0000 7fabbbc77640 1 -- 172.21.15.47:0/2665512705 shutdown_connections 2024-03-23T09:41:27.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.430+0000 7fabbbc77640 1 --2- 172.21.15.47:0/2665512705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb406d3a0 0x7fabb415d6a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.430+0000 7fabbbc77640 1 --2- 172.21.15.47:0/2665512705 >> v2:172.21.15.175:3300/0 conn(0x7fabb4069ce0 0x7fabb40ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.430+0000 7fabbbc77640 1 -- 172.21.15.47:0/2665512705 >> 172.21.15.47:0/2665512705 conn(0x7fabb4098a00 msgr2=0x7fabb40b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:27.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.431+0000 7fabbbc77640 1 -- 172.21.15.47:0/2665512705 shutdown_connections 2024-03-23T09:41:27.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.431+0000 7fabbbc77640 1 -- 172.21.15.47:0/2665512705 wait complete. 2024-03-23T09:41:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.432+0000 7fabbbc77640 1 Processor -- start 2024-03-23T09:41:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.432+0000 7fabbbc77640 1 -- start start 2024-03-23T09:41:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.432+0000 7fabbbc77640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb4069ce0 0x7fabb40b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.432+0000 7fabbbc77640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb406d3a0 0x7fabb41528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.432+0000 7fabbbc77640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb40ce830 0x7fabb4152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.432+0000 7fabbbc77640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb414abc0 con 0x7fabb40ce830 2024-03-23T09:41:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.432+0000 7fabbbc77640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb41f5530 con 0x7fabb4069ce0 2024-03-23T09:41:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.432+0000 7fabbbc77640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40b3290 con 0x7fabb406d3a0 2024-03-23T09:41:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabba214640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb40ce830 0x7fabb4152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabba214640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb40ce830 0x7fabb4152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56382/0 (socket says 172.21.15.47:56382) 2024-03-23T09:41:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabba214640 1 -- 172.21.15.47:0/4171271655 learned_addr learned my addr 172.21.15.47:0/4171271655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:27.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabb9a13640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb4069ce0 0x7fabb40b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:27.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabb9212640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb406d3a0 0x7fabb41528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:27.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabba214640 1 -- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb406d3a0 msgr2=0x7fabb41528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:27.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabba214640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb406d3a0 0x7fabb41528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabba214640 1 -- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb4069ce0 msgr2=0x7fabb40b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:27.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabba214640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb4069ce0 0x7fabb40b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabba214640 1 -- 172.21.15.47:0/4171271655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb0009b30 con 0x7fabb40ce830 2024-03-23T09:41:27.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabba214640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb40ce830 0x7fabb4152df0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009f40 tx=0x7fabb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:27.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabaa7fc640 1 -- 172.21.15.47:0/4171271655 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0007e70 con 0x7fabb40ce830 2024-03-23T09:41:27.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabaa7fc640 1 -- 172.21.15.47:0/4171271655 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb0009510 con 0x7fabb40ce830 2024-03-23T09:41:27.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabbbc77640 1 -- 172.21.15.47:0/4171271655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb40b34e0 con 0x7fabb40ce830 2024-03-23T09:41:27.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabaa7fc640 1 -- 172.21.15.47:0/4171271655 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb003e690 con 0x7fabb40ce830 2024-03-23T09:41:27.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.433+0000 7fabbbc77640 1 -- 172.21.15.47:0/4171271655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb40b3aa0 con 0x7fabb40ce830 2024-03-23T09:41:27.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.434+0000 7fabbbc77640 1 -- 172.21.15.47:0/4171271655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb40672f0 con 0x7fabb40ce830 2024-03-23T09:41:27.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.435+0000 7fabaa7fc640 1 -- 172.21.15.47:0/4171271655 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fabb003e7f0 con 0x7fabb40ce830 2024-03-23T09:41:27.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.435+0000 7fabaa7fc640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fab9806ac00 0x7fab9806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:27.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.435+0000 7fabaa7fc640 1 -- 172.21.15.47:0/4171271655 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabb00ac030 con 0x7fabb40ce830 2024-03-23T09:41:27.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.436+0000 7fabb9a13640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fab9806ac00 0x7fab9806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:27.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.436+0000 7fabb9a13640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fab9806ac00 0x7fab9806d0c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fabac000f60 tx=0x7fabac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:27.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.440+0000 7fabaa7fc640 1 -- 172.21.15.47:0/4171271655 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fabb0077470 con 0x7fabb40ce830 2024-03-23T09:41:27.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.620+0000 7fabbbc77640 1 -- 172.21.15.47:0/4171271655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabb415a430 con 0x7fabb40ce830 2024-03-23T09:41:27.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.621+0000 7fabaa7fc640 1 -- 172.21.15.47:0/4171271655 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fabb007aa90 con 0x7fabb40ce830 2024-03-23T09:41:27.621 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:27.621 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.623+0000 7fabbbc77640 1 -- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fab9806ac00 msgr2=0x7fab9806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.623+0000 7fabbbc77640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fab9806ac00 0x7fab9806d0c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fabac000f60 tx=0x7fabac009040 comp rx=0 tx=0).stop 2024-03-23T09:41:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.623+0000 7fabbbc77640 1 -- 172.21.15.47:0/4171271655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb40ce830 msgr2=0x7fabb4152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.623+0000 7fabbbc77640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb40ce830 0x7fabb4152df0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009f40 tx=0x7fabb0004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.623+0000 7fabbbc77640 1 -- 172.21.15.47:0/4171271655 shutdown_connections 2024-03-23T09:41:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.623+0000 7fabbbc77640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fab9806ac00 0x7fab9806d0c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.623+0000 7fabbbc77640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb40ce830 0x7fabb4152df0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.623+0000 7fabbbc77640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb406d3a0 0x7fabb41528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.623+0000 7fabbbc77640 1 --2- 172.21.15.47:0/4171271655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb4069ce0 0x7fabb40b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.623+0000 7fabbbc77640 1 -- 172.21.15.47:0/4171271655 >> 172.21.15.47:0/4171271655 conn(0x7fabb4098a00 msgr2=0x7fabb40acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.623+0000 7fabbbc77640 1 -- 172.21.15.47:0/4171271655 shutdown_connections 2024-03-23T09:41:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.623+0000 7fabbbc77640 1 -- 172.21.15.47:0/4171271655 wait complete. 2024-03-23T09:41:27.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:27.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:27.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.738+0000 7fdcd531b640 1 Processor -- start 2024-03-23T09:41:27.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.738+0000 7fdcd531b640 1 -- start start 2024-03-23T09:41:27.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.739+0000 7fdcd531b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdcd00c9aa0 0x7fdcd006abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:27.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.739+0000 7fdcd531b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd015ac30 con 0x7fdcd015a850 2024-03-23T09:41:27.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.739+0000 7fdcd531b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd00ae640 con 0x7fdcd0069ce0 2024-03-23T09:41:27.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.739+0000 7fdcd531b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd006d150 con 0x7fdcd00c9aa0 2024-03-23T09:41:27.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.739+0000 7fdcceffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdcd015a850 0x7fdcd00c9390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33920/0 (socket says 172.21.15.47:33920) 2024-03-23T09:41:27.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.739+0000 7fdcceffd640 1 -- 172.21.15.47:0/979692439 learned_addr learned my addr 172.21.15.47:0/979692439 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:27.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.739+0000 7fdcce7fc640 1 --2- 172.21.15.47:0/979692439 >> v2:172.21.15.47:3300/0 conn(0x7fdcd00c9aa0 0x7fdcd006abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:27.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.740+0000 7fdcce7fc640 1 -- 172.21.15.47:0/979692439 >> v1:172.21.15.175:6789/0 conn(0x7fdcd0069ce0 legacy=0x7fdcd006d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:27.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.740+0000 7fdcce7fc640 1 -- 172.21.15.47:0/979692439 >> v1:172.21.15.47:6789/0 conn(0x7fdcd015a850 legacy=0x7fdcd00c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:27.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.740+0000 7fdcce7fc640 1 -- 172.21.15.47:0/979692439 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd006b270 con 0x7fdcd00c9aa0 2024-03-23T09:41:27.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.740+0000 7fdcce7fc640 1 --2- 172.21.15.47:0/979692439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00c9aa0 0x7fdcd006abd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80097a0 tx=0x7fdcb802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ef30b9651464f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:27.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.740+0000 7fdccdffb640 1 -- 172.21.15.47:0/979692439 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb802e070 con 0x7fdcd00c9aa0 2024-03-23T09:41:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.741+0000 7fdccdffb640 1 -- 172.21.15.47:0/979692439 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb80315a0 con 0x7fdcd00c9aa0 2024-03-23T09:41:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.741+0000 7fdccdffb640 1 -- 172.21.15.47:0/979692439 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8031930 con 0x7fdcd00c9aa0 2024-03-23T09:41:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.741+0000 7fdcd531b640 1 -- 172.21.15.47:0/979692439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00c9aa0 msgr2=0x7fdcd006abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.741+0000 7fdcd531b640 1 --2- 172.21.15.47:0/979692439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00c9aa0 0x7fdcd006abd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80097a0 tx=0x7fdcb802b840 comp rx=0 tx=0).stop 2024-03-23T09:41:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.741+0000 7fdcd531b640 1 -- 172.21.15.47:0/979692439 shutdown_connections 2024-03-23T09:41:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.741+0000 7fdcd531b640 1 --2- 172.21.15.47:0/979692439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00c9aa0 0x7fdcd006abd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.741+0000 7fdcd531b640 1 -- 172.21.15.47:0/979692439 >> 172.21.15.47:0/979692439 conn(0x7fdcd00cb7c0 msgr2=0x7fdcd00cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.741+0000 7fdcd531b640 1 -- 172.21.15.47:0/979692439 shutdown_connections 2024-03-23T09:41:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.741+0000 7fdcd531b640 1 -- 172.21.15.47:0/979692439 wait complete. 2024-03-23T09:41:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.742+0000 7fdcd531b640 1 Processor -- start 2024-03-23T09:41:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.742+0000 7fdcd531b640 1 -- start start 2024-03-23T09:41:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.743+0000 7fdcd531b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcd0069ce0 0x7fdcd014be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.743+0000 7fdcd531b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcd00c9aa0 0x7fdcd00b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.743+0000 7fdcd531b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd015a850 0x7fdcd00b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.743+0000 7fdcd531b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd00b1f00 con 0x7fdcd015a850 2024-03-23T09:41:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.743+0000 7fdcd531b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd00b3da0 con 0x7fdcd00c9aa0 2024-03-23T09:41:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.743+0000 7fdcd531b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd014c4d0 con 0x7fdcd0069ce0 2024-03-23T09:41:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.743+0000 7fdccf7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd015a850 0x7fdcd00b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.743+0000 7fdccf7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd015a850 0x7fdcd00b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56392/0 (socket says 172.21.15.47:56392) 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.743+0000 7fdccf7fe640 1 -- 172.21.15.47:0/955018146 learned_addr learned my addr 172.21.15.47:0/955018146 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdcce7fc640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcd00c9aa0 0x7fdcd00b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdcceffd640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcd0069ce0 0x7fdcd014be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdccf7fe640 1 -- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcd0069ce0 msgr2=0x7fdcd014be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdccf7fe640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcd0069ce0 0x7fdcd014be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdccf7fe640 1 -- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcd00c9aa0 msgr2=0x7fdcd00b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdccf7fe640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcd00c9aa0 0x7fdcd00b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdccf7fe640 1 -- 172.21.15.47:0/955018146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb8009400 con 0x7fdcd015a850 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdcceffd640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcd0069ce0 0x7fdcd014be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdccf7fe640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd015a850 0x7fdcd00b3860 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000c780 tx=0x7fdcc000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdcc77fe640 1 -- 172.21.15.47:0/955018146 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0003d80 con 0x7fdcd015a850 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdcc77fe640 1 -- 172.21.15.47:0/955018146 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc000ce90 con 0x7fdcd015a850 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdcd531b640 1 -- 172.21.15.47:0/955018146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd014c7b0 con 0x7fdcd015a850 2024-03-23T09:41:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdcc77fe640 1 -- 172.21.15.47:0/955018146 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc00052b0 con 0x7fdcd015a850 2024-03-23T09:41:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.744+0000 7fdcd531b640 1 -- 172.21.15.47:0/955018146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd00c5f90 con 0x7fdcd015a850 2024-03-23T09:41:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.745+0000 7fdcd531b640 1 -- 172.21.15.47:0/955018146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd014ccd0 con 0x7fdcd015a850 2024-03-23T09:41:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.746+0000 7fdcc77fe640 1 -- 172.21.15.47:0/955018146 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdcc0005410 con 0x7fdcd015a850 2024-03-23T09:41:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.746+0000 7fdcc77fe640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdcb006ac00 0x7fdcb006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.746+0000 7fdcc77fe640 1 -- 172.21.15.47:0/955018146 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdcc0088b10 con 0x7fdcd015a850 2024-03-23T09:41:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.746+0000 7fdcceffd640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdcb006ac00 0x7fdcb006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.747+0000 7fdcceffd640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdcb006ac00 0x7fdcb006d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8005880 tx=0x7fdcc8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.750+0000 7fdcc77fe640 1 -- 172.21.15.47:0/955018146 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdcc0053d80 con 0x7fdcd015a850 2024-03-23T09:41:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.932+0000 7fdcd531b640 1 -- 172.21.15.47:0/955018146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcd00672f0 con 0x7fdcd015a850 2024-03-23T09:41:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.933+0000 7fdcc77fe640 1 -- 172.21.15.47:0/955018146 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdcc0017070 con 0x7fdcd015a850 2024-03-23T09:41:27.933 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.935+0000 7fdcd531b640 1 -- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdcb006ac00 msgr2=0x7fdcb006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.935+0000 7fdcd531b640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdcb006ac00 0x7fdcb006d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8005880 tx=0x7fdcc8009040 comp rx=0 tx=0).stop 2024-03-23T09:41:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.935+0000 7fdcd531b640 1 -- 172.21.15.47:0/955018146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd015a850 msgr2=0x7fdcd00b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.935+0000 7fdcd531b640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd015a850 0x7fdcd00b3860 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000c780 tx=0x7fdcc000cc40 comp rx=0 tx=0).stop 2024-03-23T09:41:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.935+0000 7fdcd531b640 1 -- 172.21.15.47:0/955018146 shutdown_connections 2024-03-23T09:41:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.935+0000 7fdcd531b640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdcb006ac00 0x7fdcb006d0c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.935+0000 7fdcd531b640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd015a850 0x7fdcd00b3860 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.935+0000 7fdcd531b640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcd00c9aa0 0x7fdcd00b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.935+0000 7fdcd531b640 1 --2- 172.21.15.47:0/955018146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcd0069ce0 0x7fdcd014be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.935+0000 7fdcd531b640 1 -- 172.21.15.47:0/955018146 >> 172.21.15.47:0/955018146 conn(0x7fdcd00cb7c0 msgr2=0x7fdcd0159340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:27.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.935+0000 7fdcd531b640 1 -- 172.21.15.47:0/955018146 shutdown_connections 2024-03-23T09:41:27.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:27.935+0000 7fdcd531b640 1 -- 172.21.15.47:0/955018146 wait complete. 2024-03-23T09:41:27.945 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.050+0000 7f44c21b9640 1 Processor -- start 2024-03-23T09:41:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.050+0000 7f44c21b9640 1 -- start start 2024-03-23T09:41:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44c21b9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0cde10 0x7f44bc160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44c21b9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f44bc0b8350 0x7f44bc0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44c21b9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc161630 con 0x7f44bc15c990 2024-03-23T09:41:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44c21b9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44bc0d1030 con 0x7f44bc0b8350 2024-03-23T09:41:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44c21b9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc0cfe10 con 0x7f44bc0cde10 2024-03-23T09:41:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44bbfff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f44bc0b8350 0x7f44bc0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44baffd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0cde10 0x7f44bc160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44bbfff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f44bc0b8350 0x7f44bc0d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:33274/0 (socket says 172.21.15.47:33274) 2024-03-23T09:41:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44bbfff640 1 -- 172.21.15.47:0/1088531902 learned_addr learned my addr 172.21.15.47:0/1088531902 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44bb7fe640 1 --1- 172.21.15.47:0/1088531902 >> v1:172.21.15.175:6789/0 conn(0x7f44bc15c990 0x7f44bc0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:44328/0 (socket says 172.21.15.47:44328) 2024-03-23T09:41:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44bbfff640 1 -- 172.21.15.47:0/1088531902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0cde10 msgr2=0x7f44bc160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44bbfff640 1 --2- 172.21.15.47:0/1088531902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0cde10 0x7f44bc160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44bbfff640 1 -- 172.21.15.47:0/1088531902 >> v1:172.21.15.175:6789/0 conn(0x7f44bc15c990 legacy=0x7f44bc0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44bbfff640 1 -- 172.21.15.47:0/1088531902 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc0b5b60 con 0x7f44bc0b8350 2024-03-23T09:41:28.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.051+0000 7f44ba7fc640 1 -- 172.21.15.47:0/1088531902 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724270069 0 0) 0x7f44bc161630 con 0x7f44bc15c990 2024-03-23T09:41:28.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.052+0000 7f44bbfff640 1 --2- 172.21.15.47:0/1088531902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc0b8350 0x7f44bc0d14c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009e50 tx=0x7f44b402be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9370fd29f0380eec server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:28.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.052+0000 7f44ba7fc640 1 -- 172.21.15.47:0/1088531902 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b402c9e0 con 0x7f44bc0b8350 2024-03-23T09:41:28.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.052+0000 7f44ba7fc640 1 -- 172.21.15.47:0/1088531902 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b402cb40 con 0x7f44bc0b8350 2024-03-23T09:41:28.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.052+0000 7f44ba7fc640 1 -- 172.21.15.47:0/1088531902 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b40343f0 con 0x7f44bc0b8350 2024-03-23T09:41:28.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.053+0000 7f44c21b9640 1 -- 172.21.15.47:0/1088531902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc0b8350 msgr2=0x7f44bc0d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.053+0000 7f44c21b9640 1 --2- 172.21.15.47:0/1088531902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc0b8350 0x7f44bc0d14c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009e50 tx=0x7f44b402be40 comp rx=0 tx=0).stop 2024-03-23T09:41:28.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.053+0000 7f44c21b9640 1 -- 172.21.15.47:0/1088531902 shutdown_connections 2024-03-23T09:41:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.053+0000 7f44c21b9640 1 --2- 172.21.15.47:0/1088531902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc0b8350 0x7f44bc0d14c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.053+0000 7f44c21b9640 1 --2- 172.21.15.47:0/1088531902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0cde10 0x7f44bc160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.053+0000 7f44c21b9640 1 -- 172.21.15.47:0/1088531902 >> 172.21.15.47:0/1088531902 conn(0x7f44bc0659d0 msgr2=0x7f44bc065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.053+0000 7f44c21b9640 1 -- 172.21.15.47:0/1088531902 shutdown_connections 2024-03-23T09:41:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.053+0000 7f44c21b9640 1 -- 172.21.15.47:0/1088531902 wait complete. 2024-03-23T09:41:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.054+0000 7f44c21b9640 1 Processor -- start 2024-03-23T09:41:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.054+0000 7f44c21b9640 1 -- start start 2024-03-23T09:41:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44c21b9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc0b8350 0x7f44bc153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44c21b9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0cde10 0x7f44bc0c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44c21b9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44bc15c990 0x7f44bc15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44c21b9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc0c7930 con 0x7f44bc15c990 2024-03-23T09:41:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44c21b9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc0c7dc0 con 0x7f44bc0b8350 2024-03-23T09:41:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44c21b9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc0c6d20 con 0x7f44bc0cde10 2024-03-23T09:41:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44bbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44bc15c990 0x7f44bc15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44bbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44bc15c990 0x7f44bc15e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56406/0 (socket says 172.21.15.47:56406) 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44bbfff640 1 -- 172.21.15.47:0/4191475385 learned_addr learned my addr 172.21.15.47:0/4191475385 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44bb7fe640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc0b8350 0x7f44bc153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44baffd640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0cde10 0x7f44bc0c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44bbfff640 1 -- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0cde10 msgr2=0x7f44bc0c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44bbfff640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0cde10 0x7f44bc0c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44bbfff640 1 -- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc0b8350 msgr2=0x7f44bc153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44bbfff640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc0b8350 0x7f44bc153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.055+0000 7f44bbfff640 1 -- 172.21.15.47:0/4191475385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b4009b30 con 0x7f44bc15c990 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.056+0000 7f44bbfff640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44bc15c990 0x7f44bc15e9d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009e50 tx=0x7f44b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.056+0000 7f44a3fff640 1 -- 172.21.15.47:0/4191475385 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4009510 con 0x7f44bc15c990 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.056+0000 7f44a3fff640 1 -- 172.21.15.47:0/4191475385 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b403e450 con 0x7f44bc15c990 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.056+0000 7f44c21b9640 1 -- 172.21.15.47:0/4191475385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44bc0c7000 con 0x7f44bc15c990 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.056+0000 7f44a3fff640 1 -- 172.21.15.47:0/4191475385 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b403e620 con 0x7f44bc15c990 2024-03-23T09:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.056+0000 7f44c21b9640 1 -- 172.21.15.47:0/4191475385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44bc0c75c0 con 0x7f44bc15c990 2024-03-23T09:41:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.057+0000 7f44c21b9640 1 -- 172.21.15.47:0/4191475385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44bc072360 con 0x7f44bc15c990 2024-03-23T09:41:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.057+0000 7f44a3fff640 1 -- 172.21.15.47:0/4191475385 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f44b403e780 con 0x7f44bc15c990 2024-03-23T09:41:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.058+0000 7f44a3fff640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f44a806ac00 0x7f44a806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.058+0000 7f44a3fff640 1 -- 172.21.15.47:0/4191475385 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44b40abc90 con 0x7f44bc15c990 2024-03-23T09:41:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.058+0000 7f44bb7fe640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f44a806ac00 0x7f44a806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.058+0000 7f44bb7fe640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f44a806ac00 0x7f44a806d0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f44ac000f60 tx=0x7f44ac007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.062+0000 7f44a3fff640 1 -- 172.21.15.47:0/4191475385 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f44b4076e70 con 0x7f44bc15c990 2024-03-23T09:41:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.241+0000 7f44c21b9640 1 -- 172.21.15.47:0/4191475385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f44bc066be0 con 0x7f44bc15c990 2024-03-23T09:41:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.242+0000 7f44a3fff640 1 -- 172.21.15.47:0/4191475385 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f44b407a490 con 0x7f44bc15c990 2024-03-23T09:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:28.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.244+0000 7f44c21b9640 1 -- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f44a806ac00 msgr2=0x7f44a806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.244+0000 7f44c21b9640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f44a806ac00 0x7f44a806d0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f44ac000f60 tx=0x7f44ac007b50 comp rx=0 tx=0).stop 2024-03-23T09:41:28.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.244+0000 7f44c21b9640 1 -- 172.21.15.47:0/4191475385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44bc15c990 msgr2=0x7f44bc15e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.244+0000 7f44c21b9640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44bc15c990 0x7f44bc15e9d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009e50 tx=0x7f44b4004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:28.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.244+0000 7f44c21b9640 1 -- 172.21.15.47:0/4191475385 shutdown_connections 2024-03-23T09:41:28.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.244+0000 7f44c21b9640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f44a806ac00 0x7f44a806d0c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.244+0000 7f44c21b9640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44bc15c990 0x7f44bc15e9d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.244+0000 7f44c21b9640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0cde10 0x7f44bc0c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.244+0000 7f44c21b9640 1 --2- 172.21.15.47:0/4191475385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc0b8350 0x7f44bc153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.244+0000 7f44c21b9640 1 -- 172.21.15.47:0/4191475385 >> 172.21.15.47:0/4191475385 conn(0x7f44bc0659d0 msgr2=0x7f44bc0b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:28.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.244+0000 7f44c21b9640 1 -- 172.21.15.47:0/4191475385 shutdown_connections 2024-03-23T09:41:28.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.244+0000 7f44c21b9640 1 -- 172.21.15.47:0/4191475385 wait complete. 2024-03-23T09:41:28.254 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.358+0000 7fa0a4482640 1 Processor -- start 2024-03-23T09:41:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.358+0000 7fa0a4482640 1 -- start start 2024-03-23T09:41:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.359+0000 7fa0a4482640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0caf80 0x7fa09c0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.359+0000 7fa0a4482640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa09c0b0810 0x7fa09c0b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.359+0000 7fa0a4482640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09c0b11c0 con 0x7fa09c0cb360 2024-03-23T09:41:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.359+0000 7fa0a4482640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09c0d0810 con 0x7fa09c0b0810 2024-03-23T09:41:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.359+0000 7fa0a4482640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c160b70 con 0x7fa09c0caf80 2024-03-23T09:41:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.359+0000 7fa0a1a1d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa09c0cb360 0x7fa09c161000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33936/0 (socket says 172.21.15.47:33936) 2024-03-23T09:41:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.359+0000 7fa0a1a1d640 1 -- 172.21.15.47:0/2520180042 learned_addr learned my addr 172.21.15.47:0/2520180042 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a2a1f640 1 --2- 172.21.15.47:0/2520180042 >> v2:172.21.15.175:3300/0 conn(0x7fa09c0b0810 0x7fa09c0b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328499603 0 0) 0x7fa09c0b11c0 con 0x7fa09c0cb360 2024-03-23T09:41:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a221e640 1 --2- 172.21.15.47:0/2520180042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0caf80 0x7fa09c0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa088003520 con 0x7fa09c0cb360 2024-03-23T09:41:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2573475002 0 0) 0x7fa088003520 con 0x7fa09c0cb360 2024-03-23T09:41:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0caf80 msgr2=0x7fa09c0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 --2- 172.21.15.47:0/2520180042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0caf80 0x7fa09c0cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 >> v2:172.21.15.175:3300/0 conn(0x7fa09c0b0810 msgr2=0x7fa09c0b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 --2- 172.21.15.47:0/2520180042 >> v2:172.21.15.175:3300/0 conn(0x7fa09c0b0810 0x7fa09c0b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c161710 con 0x7fa09c0cb360 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fa08c001670 con 0x7fa09c0cb360 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 >> v1:172.21.15.47:6789/0 conn(0x7fa09c0cb360 legacy=0x7fa09c161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 --2- 172.21.15.47:0/2520180042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088003970 0x7fa088003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 --2- 172.21.15.47:0/2520180042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0caf80 0x7fa0880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 --2- 172.21.15.47:0/2520180042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa088004d00 0x7fa088005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0d0810 con 0x7fa088003970 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c160b70 con 0x7fa088004d00 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08800b980 con 0x7fa09c0caf80 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a221e640 1 --2- 172.21.15.47:0/2520180042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088003970 0x7fa088003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a1a1d640 1 --2- 172.21.15.47:0/2520180042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0caf80 0x7fa0880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a2a1f640 1 --2- 172.21.15.47:0/2520180042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa088004d00 0x7fa088005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a221e640 1 -- 172.21.15.47:0/2520180042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0caf80 msgr2=0x7fa0880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a221e640 1 --2- 172.21.15.47:0/2520180042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0caf80 0x7fa0880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a221e640 1 -- 172.21.15.47:0/2520180042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa088004d00 msgr2=0x7fa088005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a221e640 1 --2- 172.21.15.47:0/2520180042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa088004d00 0x7fa088005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a221e640 1 -- 172.21.15.47:0/2520180042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa088003520 con 0x7fa088003970 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.360+0000 7fa0a1a1d640 1 --2- 172.21.15.47:0/2520180042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0caf80 0x7fa0880047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.361+0000 7fa0a221e640 1 --2- 172.21.15.47:0/2520180042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088003970 0x7fa088003d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa098009ac0 tx=0x7fa0880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e524e2941e44d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.361+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09802c6b0 con 0x7fa088003970 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.361+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09802c810 con 0x7fa088003970 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.361+0000 7fa0a121c640 1 -- 172.21.15.47:0/2520180042 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09802cba0 con 0x7fa088003970 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.362+0000 7fa0a4482640 1 -- 172.21.15.47:0/2520180042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088003970 msgr2=0x7fa088003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.362+0000 7fa0a4482640 1 --2- 172.21.15.47:0/2520180042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088003970 0x7fa088003d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa098009ac0 tx=0x7fa0880057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.362+0000 7fa0a2a1f640 1 -- 172.21.15.47:0/2520180042 reap_dead start 2024-03-23T09:41:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.362+0000 7fa0a4482640 1 -- 172.21.15.47:0/2520180042 shutdown_connections 2024-03-23T09:41:28.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.362+0000 7fa0a4482640 1 -- 172.21.15.47:0/2520180042 >> 172.21.15.47:0/2520180042 conn(0x7fa09c098a00 msgr2=0x7fa09c0cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:28.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.362+0000 7fa0a4482640 1 -- 172.21.15.47:0/2520180042 shutdown_connections 2024-03-23T09:41:28.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.362+0000 7fa0a4482640 1 -- 172.21.15.47:0/2520180042 wait complete. 2024-03-23T09:41:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.363+0000 7fa0a4482640 1 Processor -- start 2024-03-23T09:41:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.363+0000 7fa0a4482640 1 -- start start 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a4482640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa088003970 0x7fa09c1593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a4482640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0b2fb0 0x7fa09c14ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a4482640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09c0ac530 0x7fa09c0b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a4482640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0b34b0 con 0x7fa09c0ac530 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a4482640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0b3790 con 0x7fa088003970 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a4482640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0b3a70 con 0x7fa09c0b2fb0 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a2a1f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09c0ac530 0x7fa09c0b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a2a1f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09c0ac530 0x7fa09c0b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56426/0 (socket says 172.21.15.47:56426) 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a2a1f640 1 -- 172.21.15.47:0/3792705322 learned_addr learned my addr 172.21.15.47:0/3792705322 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a1a1d640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0b2fb0 0x7fa09c14ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a221e640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa088003970 0x7fa09c1593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a2a1f640 1 -- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0b2fb0 msgr2=0x7fa09c14ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a2a1f640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0b2fb0 0x7fa09c14ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a2a1f640 1 -- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa088003970 msgr2=0x7fa09c1593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a2a1f640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa088003970 0x7fa09c1593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a2a1f640 1 -- 172.21.15.47:0/3792705322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c0b3d50 con 0x7fa09c0ac530 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a1a1d640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0b2fb0 0x7fa09c14ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a221e640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa088003970 0x7fa09c1593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a2a1f640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09c0ac530 0x7fa09c0b4550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa09c06fc40 tx=0x7fa0940115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa092ffd640 1 -- 172.21.15.47:0/3792705322 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09400e070 con 0x7fa09c0ac530 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa092ffd640 1 -- 172.21.15.47:0/3792705322 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa094011e10 con 0x7fa09c0ac530 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa0a4482640 1 -- 172.21.15.47:0/3792705322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c0b4030 con 0x7fa09c0ac530 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.364+0000 7fa092ffd640 1 -- 172.21.15.47:0/3792705322 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094003990 con 0x7fa09c0ac530 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.365+0000 7fa0a4482640 1 -- 172.21.15.47:0/3792705322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c1591f0 con 0x7fa09c0ac530 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.366+0000 7fa0a4482640 1 -- 172.21.15.47:0/3792705322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09c0672f0 con 0x7fa09c0ac530 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.366+0000 7fa092ffd640 1 -- 172.21.15.47:0/3792705322 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa094003af0 con 0x7fa09c0ac530 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.366+0000 7fa092ffd640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa08006ac00 0x7fa08006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.366+0000 7fa092ffd640 1 -- 172.21.15.47:0/3792705322 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa094089860 con 0x7fa09c0ac530 2024-03-23T09:41:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.367+0000 7fa0a221e640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa08006ac00 0x7fa08006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.367+0000 7fa0a221e640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa08006ac00 0x7fa08006d0c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa098036000 tx=0x7fa098030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:28.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.371+0000 7fa092ffd640 1 -- 172.21.15.47:0/3792705322 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa094054ad0 con 0x7fa09c0ac530 2024-03-23T09:41:28.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.548+0000 7fa0a4482640 1 -- 172.21.15.47:0/3792705322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa09c15c3e0 con 0x7fa09c0ac530 2024-03-23T09:41:28.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.548+0000 7fa092ffd640 1 -- 172.21.15.47:0/3792705322 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0940580f0 con 0x7fa09c0ac530 2024-03-23T09:41:28.548 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:28.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.550+0000 7fa0a4482640 1 -- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa08006ac00 msgr2=0x7fa08006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.550+0000 7fa0a4482640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa08006ac00 0x7fa08006d0c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa098036000 tx=0x7fa098030000 comp rx=0 tx=0).stop 2024-03-23T09:41:28.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.550+0000 7fa0a4482640 1 -- 172.21.15.47:0/3792705322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09c0ac530 msgr2=0x7fa09c0b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.550+0000 7fa0a4482640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09c0ac530 0x7fa09c0b4550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa09c06fc40 tx=0x7fa0940115a0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.550+0000 7fa0a4482640 1 -- 172.21.15.47:0/3792705322 shutdown_connections 2024-03-23T09:41:28.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.550+0000 7fa0a4482640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa08006ac00 0x7fa08006d0c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.550+0000 7fa0a4482640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09c0ac530 0x7fa09c0b4550 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.550+0000 7fa0a4482640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa09c0b2fb0 0x7fa09c14ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.550+0000 7fa0a4482640 1 --2- 172.21.15.47:0/3792705322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa088003970 0x7fa09c1593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.550+0000 7fa0a4482640 1 -- 172.21.15.47:0/3792705322 >> 172.21.15.47:0/3792705322 conn(0x7fa09c098a00 msgr2=0x7fa09c0c9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:28.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.550+0000 7fa0a4482640 1 -- 172.21.15.47:0/3792705322 shutdown_connections 2024-03-23T09:41:28.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.550+0000 7fa0a4482640 1 -- 172.21.15.47:0/3792705322 wait complete. 2024-03-23T09:41:28.560 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:28.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.662+0000 7fd06d5b1640 1 Processor -- start 2024-03-23T09:41:28.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.662+0000 7fd06d5b1640 1 -- start start 2024-03-23T09:41:28.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd06d5b1640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd0680cde10 0x7fd068167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd06d5b1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0680d11e0 con 0x7fd06815ca90 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd06d5b1640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0680d2680 con 0x7fd0680b8350 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd06d5b1640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd068160f60 con 0x7fd0680cde10 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd066ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd06815ca90 0x7fd068160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33938/0 (socket says 172.21.15.47:33938) 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd066ffd640 1 -- 172.21.15.47:0/2883085829 learned_addr learned my addr 172.21.15.47:0/2883085829 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872775259 0 0) 0x7fd0680d11e0 con 0x7fd06815ca90 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd054003520 con 0x7fd06815ca90 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0677fe640 1 --2- 172.21.15.47:0/2883085829 >> v2:172.21.15.175:3300/0 conn(0x7fd0680cde10 0x7fd068167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1952008221 0 0) 0x7fd054003520 con 0x7fd06815ca90 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 >> v2:172.21.15.175:3300/0 conn(0x7fd0680cde10 msgr2=0x7fd068167aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 --2- 172.21.15.47:0/2883085829 >> v2:172.21.15.175:3300/0 conn(0x7fd0680cde10 0x7fd068167aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 >> v1:172.21.15.175:6789/0 conn(0x7fd0680b8350 legacy=0x7fd0680cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd068161240 con 0x7fd06815ca90 2024-03-23T09:41:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fd058001670 con 0x7fd06815ca90 2024-03-23T09:41:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0677fe640 1 --2- 172.21.15.47:0/2883085829 >> v2:172.21.15.175:3300/0 conn(0x7fd0680cde10 0x7fd068167aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 >> v1:172.21.15.47:6789/0 conn(0x7fd06815ca90 legacy=0x7fd068160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 --2- 172.21.15.47:0/2883085829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd054003970 0x7fd054003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 --2- 172.21.15.47:0/2883085829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0540047c0 0x7fd054004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 --2- 172.21.15.47:0/2883085829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0540052a0 0x7fd05400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd068160f60 con 0x7fd0540047c0 2024-03-23T09:41:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05400c170 con 0x7fd0540052a0 2024-03-23T09:41:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.663+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05400c330 con 0x7fd054003970 2024-03-23T09:41:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.664+0000 7fd05ffff640 1 --2- 172.21.15.47:0/2883085829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0540047c0 0x7fd054004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.664+0000 7fd066ffd640 1 --2- 172.21.15.47:0/2883085829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd054003970 0x7fd054003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.664+0000 7fd0677fe640 1 --2- 172.21.15.47:0/2883085829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0540052a0 0x7fd05400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.664+0000 7fd05ffff640 1 -- 172.21.15.47:0/2883085829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd054003970 msgr2=0x7fd054003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.664+0000 7fd05ffff640 1 --2- 172.21.15.47:0/2883085829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd054003970 0x7fd054003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.664+0000 7fd05ffff640 1 -- 172.21.15.47:0/2883085829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0540052a0 msgr2=0x7fd05400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.664+0000 7fd05ffff640 1 --2- 172.21.15.47:0/2883085829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0540052a0 0x7fd05400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.664+0000 7fd05ffff640 1 -- 172.21.15.47:0/2883085829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd054003520 con 0x7fd0540047c0 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.664+0000 7fd066ffd640 1 --2- 172.21.15.47:0/2883085829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd054003970 0x7fd054003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.664+0000 7fd0677fe640 1 --2- 172.21.15.47:0/2883085829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0540052a0 0x7fd05400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.664+0000 7fd0677fe640 1 -- 172.21.15.47:0/2883085829 reap_dead start 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.665+0000 7fd05ffff640 1 --2- 172.21.15.47:0/2883085829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0540047c0 0x7fd054004c40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd050001720 tx=0x7fd0540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5052aa1084ecf342 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.665+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500052d0 con 0x7fd0540047c0 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.665+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05002e050 con 0x7fd0540047c0 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.665+0000 7fd0667fc640 1 -- 172.21.15.47:0/2883085829 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050003830 con 0x7fd0540047c0 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.665+0000 7fd06d5b1640 1 -- 172.21.15.47:0/2883085829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0540047c0 msgr2=0x7fd054004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.665+0000 7fd06d5b1640 1 --2- 172.21.15.47:0/2883085829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0540047c0 0x7fd054004c40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd050001720 tx=0x7fd0540057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.665+0000 7fd06d5b1640 1 -- 172.21.15.47:0/2883085829 shutdown_connections 2024-03-23T09:41:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.665+0000 7fd06d5b1640 1 --2- 172.21.15.47:0/2883085829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0540047c0 0x7fd054004c40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.665+0000 7fd06d5b1640 1 -- 172.21.15.47:0/2883085829 >> 172.21.15.47:0/2883085829 conn(0x7fd0680659d0 msgr2=0x7fd068065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.665+0000 7fd06d5b1640 1 -- 172.21.15.47:0/2883085829 shutdown_connections 2024-03-23T09:41:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.665+0000 7fd06d5b1640 1 -- 172.21.15.47:0/2883085829 wait complete. 2024-03-23T09:41:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.666+0000 7fd06d5b1640 1 Processor -- start 2024-03-23T09:41:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.666+0000 7fd06d5b1640 1 -- start start 2024-03-23T09:41:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.667+0000 7fd06d5b1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0540047c0 0x7fd0681f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.667+0000 7fd06d5b1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0680cb1a0 0x7fd068150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.667+0000 7fd06d5b1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0680b3470 0x7fd068155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.667+0000 7fd06d5b1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680cb940 con 0x7fd0540047c0 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.667+0000 7fd06d5b1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680cbc20 con 0x7fd0680cb1a0 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.667+0000 7fd06d5b1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680c9b30 con 0x7fd0680b3470 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.667+0000 7fd066ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0540047c0 0x7fd0681f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.667+0000 7fd066ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0540047c0 0x7fd0681f3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56442/0 (socket says 172.21.15.47:56442) 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.667+0000 7fd066ffd640 1 -- 172.21.15.47:0/3061927131 learned_addr learned my addr 172.21.15.47:0/3061927131 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.668+0000 7fd066ffd640 1 -- 172.21.15.47:0/3061927131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0680b3470 msgr2=0x7fd068155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.668+0000 7fd066ffd640 1 --2- 172.21.15.47:0/3061927131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0680b3470 0x7fd068155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.668+0000 7fd066ffd640 1 -- 172.21.15.47:0/3061927131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0680cb1a0 msgr2=0x7fd068150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.668+0000 7fd066ffd640 1 --2- 172.21.15.47:0/3061927131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0680cb1a0 0x7fd068150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.668+0000 7fd066ffd640 1 -- 172.21.15.47:0/3061927131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0680c9e10 con 0x7fd0540047c0 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.668+0000 7fd066ffd640 1 --2- 172.21.15.47:0/3061927131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0540047c0 0x7fd0681f3210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd0580017a0 tx=0x7fd05800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.668+0000 7fd05f7fe640 1 -- 172.21.15.47:0/3061927131 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05800ee60 con 0x7fd0540047c0 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.668+0000 7fd05f7fe640 1 -- 172.21.15.47:0/3061927131 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05800c850 con 0x7fd0540047c0 2024-03-23T09:41:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.668+0000 7fd06d5b1640 1 -- 172.21.15.47:0/3061927131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0680ca0f0 con 0x7fd0540047c0 2024-03-23T09:41:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.668+0000 7fd05f7fe640 1 -- 172.21.15.47:0/3061927131 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05800cbe0 con 0x7fd0540047c0 2024-03-23T09:41:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.668+0000 7fd06d5b1640 1 -- 172.21.15.47:0/3061927131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd068151050 con 0x7fd0540047c0 2024-03-23T09:41:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.669+0000 7fd06d5b1640 1 -- 172.21.15.47:0/3061927131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd068072360 con 0x7fd0540047c0 2024-03-23T09:41:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.669+0000 7fd05f7fe640 1 -- 172.21.15.47:0/3061927131 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd05800cd40 con 0x7fd0540047c0 2024-03-23T09:41:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.670+0000 7fd05f7fe640 1 --2- 172.21.15.47:0/3061927131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd04806ac30 0x7fd04806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.670+0000 7fd05f7fe640 1 -- 172.21.15.47:0/3061927131 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd058089290 con 0x7fd0540047c0 2024-03-23T09:41:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.670+0000 7fd05ffff640 1 --2- 172.21.15.47:0/3061927131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd04806ac30 0x7fd04806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.671+0000 7fd05ffff640 1 --2- 172.21.15.47:0/3061927131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd04806ac30 0x7fd04806d0f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd050033000 tx=0x7fd05002f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.674+0000 7fd05f7fe640 1 -- 172.21.15.47:0/3061927131 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd058054470 con 0x7fd0540047c0 2024-03-23T09:41:28.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.856+0000 7fd06d5b1640 1 -- 172.21.15.47:0/3061927131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0680ca330 con 0x7fd0540047c0 2024-03-23T09:41:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.857+0000 7fd05f7fe640 1 -- 172.21.15.47:0/3061927131 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fd0680ca330 con 0x7fd0540047c0 2024-03-23T09:41:28.856 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:28.857 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.859+0000 7fd06d5b1640 1 -- 172.21.15.47:0/3061927131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd04806ac30 msgr2=0x7fd04806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.859+0000 7fd06d5b1640 1 --2- 172.21.15.47:0/3061927131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd04806ac30 0x7fd04806d0f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd050033000 tx=0x7fd05002f000 comp rx=0 tx=0).stop 2024-03-23T09:41:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.859+0000 7fd06d5b1640 1 -- 172.21.15.47:0/3061927131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0540047c0 msgr2=0x7fd0681f3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.859+0000 7fd06d5b1640 1 --2- 172.21.15.47:0/3061927131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0540047c0 0x7fd0681f3210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd0580017a0 tx=0x7fd05800ec10 comp rx=0 tx=0).stop 2024-03-23T09:41:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.859+0000 7fd06d5b1640 1 -- 172.21.15.47:0/3061927131 shutdown_connections 2024-03-23T09:41:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.859+0000 7fd06d5b1640 1 --2- 172.21.15.47:0/3061927131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd04806ac30 0x7fd04806d0f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.859+0000 7fd06d5b1640 1 --2- 172.21.15.47:0/3061927131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0680b3470 0x7fd068155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.859+0000 7fd06d5b1640 1 --2- 172.21.15.47:0/3061927131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0680cb1a0 0x7fd068150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.859+0000 7fd06d5b1640 1 --2- 172.21.15.47:0/3061927131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0540047c0 0x7fd0681f3210 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.859+0000 7fd06d5b1640 1 -- 172.21.15.47:0/3061927131 >> 172.21.15.47:0/3061927131 conn(0x7fd0680659d0 msgr2=0x7fd06815d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.859+0000 7fd06d5b1640 1 -- 172.21.15.47:0/3061927131 shutdown_connections 2024-03-23T09:41:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.859+0000 7fd06d5b1640 1 -- 172.21.15.47:0/3061927131 wait complete. 2024-03-23T09:41:28.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:28.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:28.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.974+0000 7f0b210a0640 1 Processor -- start 2024-03-23T09:41:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.974+0000 7f0b210a0640 1 -- start start 2024-03-23T09:41:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b210a0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0b1c15ca30 0x7f0b1c160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b210a0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1c161600 con 0x7f0b1c0cde40 2024-03-23T09:41:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b210a0640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1c161040 con 0x7f0b1c15ce10 2024-03-23T09:41:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b210a0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b1c161320 con 0x7f0b1c15ca30 2024-03-23T09:41:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b1b59e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0b1c0cde40 0x7f0b1c0d15d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33950/0 (socket says 172.21.15.47:33950) 2024-03-23T09:41:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b1b59e640 1 -- 172.21.15.47:0/203040837 learned_addr learned my addr 172.21.15.47:0/203040837 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b1ad9d640 1 --2- 172.21.15.47:0/203040837 >> v2:172.21.15.175:3300/0 conn(0x7f0b1c15ca30 0x7f0b1c160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630680921 0 0) 0x7f0b1c161600 con 0x7f0b1c0cde40 2024-03-23T09:41:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b04003520 con 0x7f0b1c0cde40 2024-03-23T09:41:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 360340646 0 0) 0x7f0b04003520 con 0x7f0b1c0cde40 2024-03-23T09:41:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 >> v2:172.21.15.175:3300/0 conn(0x7f0b1c15ca30 msgr2=0x7f0b1c160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 --2- 172.21.15.47:0/203040837 >> v2:172.21.15.175:3300/0 conn(0x7f0b1c15ca30 0x7f0b1c160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 >> v1:172.21.15.175:6789/0 conn(0x7f0b1c15ce10 legacy=0x7f0b1c0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c0d0dd0 con 0x7f0b1c0cde40 2024-03-23T09:41:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0b14001670 con 0x7f0b1c0cde40 2024-03-23T09:41:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 >> v1:172.21.15.47:6789/0 conn(0x7f0b1c0cde40 legacy=0x7f0b1c0d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 --2- 172.21.15.47:0/203040837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b04003970 0x7f0b04003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 --2- 172.21.15.47:0/203040837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b040047c0 0x7f0b04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 --2- 172.21.15.47:0/203040837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b04005180 0x7f0b0400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c161320 con 0x7f0b04003970 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0400c120 con 0x7f0b040047c0 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0400c2e0 con 0x7f0b04005180 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.975+0000 7f0b1ad9d640 1 --2- 172.21.15.47:0/203040837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b04003970 0x7f0b04003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.976+0000 7f0b1a59c640 1 --2- 172.21.15.47:0/203040837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b040047c0 0x7f0b04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.976+0000 7f0b1b59e640 1 --2- 172.21.15.47:0/203040837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b04005180 0x7f0b0400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.976+0000 7f0b1ad9d640 1 -- 172.21.15.47:0/203040837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b04005180 msgr2=0x7f0b0400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.976+0000 7f0b1ad9d640 1 --2- 172.21.15.47:0/203040837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b04005180 0x7f0b0400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.976+0000 7f0b1ad9d640 1 -- 172.21.15.47:0/203040837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b040047c0 msgr2=0x7f0b04004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.976+0000 7f0b1ad9d640 1 --2- 172.21.15.47:0/203040837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b040047c0 0x7f0b04004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.976+0000 7f0b1ad9d640 1 -- 172.21.15.47:0/203040837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b04003520 con 0x7f0b04003970 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.976+0000 7f0b1a59c640 1 --2- 172.21.15.47:0/203040837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b040047c0 0x7f0b04004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.976+0000 7f0b1b59e640 1 -- 172.21.15.47:0/203040837 reap_dead start 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.976+0000 7f0b1ad9d640 1 --2- 172.21.15.47:0/203040837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b04003970 0x7f0b04003d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009fc0 tx=0x7f0b040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e3d5e388530fb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.977+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c02cc20 con 0x7f0b04003970 2024-03-23T09:41:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.977+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c00f040 con 0x7f0b04003970 2024-03-23T09:41:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.977+0000 7f0b19d9b640 1 -- 172.21.15.47:0/203040837 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c034440 con 0x7f0b04003970 2024-03-23T09:41:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.977+0000 7f0b210a0640 1 -- 172.21.15.47:0/203040837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b04003970 msgr2=0x7f0b04003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.977+0000 7f0b210a0640 1 --2- 172.21.15.47:0/203040837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b04003970 0x7f0b04003d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009fc0 tx=0x7f0b040057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.977+0000 7f0b210a0640 1 -- 172.21.15.47:0/203040837 shutdown_connections 2024-03-23T09:41:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.977+0000 7f0b210a0640 1 --2- 172.21.15.47:0/203040837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b04003970 0x7f0b04003d90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.977+0000 7f0b210a0640 1 -- 172.21.15.47:0/203040837 >> 172.21.15.47:0/203040837 conn(0x7f0b1c06b160 msgr2=0x7f0b1c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.977+0000 7f0b210a0640 1 -- 172.21.15.47:0/203040837 shutdown_connections 2024-03-23T09:41:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.977+0000 7f0b210a0640 1 -- 172.21.15.47:0/203040837 wait complete. 2024-03-23T09:41:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.979+0000 7f0b210a0640 1 Processor -- start 2024-03-23T09:41:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.979+0000 7f0b210a0640 1 -- start start 2024-03-23T09:41:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.979+0000 7f0b210a0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b04003970 0x7f0b1c1f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.979+0000 7f0b210a0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b1c0bb3f0 0x7f0b1c14be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.979+0000 7f0b210a0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1c0bae30 0x7f0b1c14c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.979+0000 7f0b210a0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c15efd0 con 0x7f0b1c0bae30 2024-03-23T09:41:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.979+0000 7f0b210a0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c0b9910 con 0x7f0b1c0bb3f0 2024-03-23T09:41:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.979+0000 7f0b210a0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c0b8d00 con 0x7f0b04003970 2024-03-23T09:41:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.979+0000 7f0b1b59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1c0bae30 0x7f0b1c14c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.979+0000 7f0b1b59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1c0bae30 0x7f0b1c14c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56448/0 (socket says 172.21.15.47:56448) 2024-03-23T09:41:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.979+0000 7f0b1b59e640 1 -- 172.21.15.47:0/653970841 learned_addr learned my addr 172.21.15.47:0/653970841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.980+0000 7f0b1ad9d640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b04003970 0x7f0b1c1f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.980+0000 7f0b1b59e640 1 -- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b04003970 msgr2=0x7f0b1c1f52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.980+0000 7f0b1a59c640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b1c0bb3f0 0x7f0b1c14be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.980+0000 7f0b1b59e640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b04003970 0x7f0b1c1f52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.980+0000 7f0b1b59e640 1 -- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b1c0bb3f0 msgr2=0x7f0b1c14be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.980+0000 7f0b1b59e640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b1c0bb3f0 0x7f0b1c14be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.980+0000 7f0b1b59e640 1 -- 172.21.15.47:0/653970841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c0b8fe0 con 0x7f0b1c0bae30 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.980+0000 7f0b1b59e640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1c0bae30 0x7f0b1c14c3d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0ceb70 tx=0x7f0b1400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.980+0000 7f0b037fe640 1 -- 172.21.15.47:0/653970841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1400eed0 con 0x7f0b1c0bae30 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.980+0000 7f0b037fe640 1 -- 172.21.15.47:0/653970841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1400c5e0 con 0x7f0b1c0bae30 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.980+0000 7f0b210a0640 1 -- 172.21.15.47:0/653970841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1c0b92c0 con 0x7f0b1c0bae30 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.980+0000 7f0b037fe640 1 -- 172.21.15.47:0/653970841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1400c990 con 0x7f0b1c0bae30 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.980+0000 7f0b210a0640 1 -- 172.21.15.47:0/653970841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1c1f57e0 con 0x7f0b1c0bae30 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.981+0000 7f0b210a0640 1 -- 172.21.15.47:0/653970841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1c072360 con 0x7f0b1c0bae30 2024-03-23T09:41:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.982+0000 7f0b037fe640 1 -- 172.21.15.47:0/653970841 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0b1400caf0 con 0x7f0b1c0bae30 2024-03-23T09:41:28.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.982+0000 7f0b037fe640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b0806ac00 0x7f0b0806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:28.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.982+0000 7f0b037fe640 1 -- 172.21.15.47:0/653970841 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b14088a80 con 0x7f0b1c0bae30 2024-03-23T09:41:28.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.982+0000 7f0b1ad9d640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b0806ac00 0x7f0b0806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:28.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.983+0000 7f0b1ad9d640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b0806ac00 0x7f0b0806d0c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c036000 tx=0x7f0b0c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:28.986+0000 7f0b037fe640 1 -- 172.21.15.47:0/653970841 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0b14057780 con 0x7f0b1c0bae30 2024-03-23T09:41:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.165+0000 7f0b210a0640 1 -- 172.21.15.47:0/653970841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b1c066be0 con 0x7f0b1c0bae30 2024-03-23T09:41:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.165+0000 7f0b037fe640 1 -- 172.21.15.47:0/653970841 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b14019070 con 0x7f0b1c0bae30 2024-03-23T09:41:29.165 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:29.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.167+0000 7f0b210a0640 1 -- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b0806ac00 msgr2=0x7f0b0806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.167+0000 7f0b210a0640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b0806ac00 0x7f0b0806d0c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c036000 tx=0x7f0b0c02e000 comp rx=0 tx=0).stop 2024-03-23T09:41:29.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.167+0000 7f0b210a0640 1 -- 172.21.15.47:0/653970841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1c0bae30 msgr2=0x7f0b1c14c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.167+0000 7f0b210a0640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1c0bae30 0x7f0b1c14c3d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0ceb70 tx=0x7f0b1400eb90 comp rx=0 tx=0).stop 2024-03-23T09:41:29.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.167+0000 7f0b210a0640 1 -- 172.21.15.47:0/653970841 shutdown_connections 2024-03-23T09:41:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.167+0000 7f0b210a0640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b0806ac00 0x7f0b0806d0c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.167+0000 7f0b210a0640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1c0bae30 0x7f0b1c14c3d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.167+0000 7f0b210a0640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b1c0bb3f0 0x7f0b1c14be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.167+0000 7f0b210a0640 1 --2- 172.21.15.47:0/653970841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b04003970 0x7f0b1c1f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.167+0000 7f0b210a0640 1 -- 172.21.15.47:0/653970841 >> 172.21.15.47:0/653970841 conn(0x7f0b1c06b160 msgr2=0x7f0b1c14d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.167+0000 7f0b210a0640 1 -- 172.21.15.47:0/653970841 shutdown_connections 2024-03-23T09:41:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.167+0000 7f0b210a0640 1 -- 172.21.15.47:0/653970841 wait complete. 2024-03-23T09:41:29.178 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:29.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.281+0000 7f509421e640 1 Processor -- start 2024-03-23T09:41:29.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.281+0000 7f509421e640 1 -- start start 2024-03-23T09:41:29.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.282+0000 7f509421e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c06b690 0x7f508c06ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.282+0000 7f509421e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c0cffc0 con 0x7f508c161450 2024-03-23T09:41:29.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.282+0000 7f509421e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c06bfb0 con 0x7f508c0b1c60 2024-03-23T09:41:29.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.282+0000 7f509421e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0cf3b0 con 0x7f508c06b690 2024-03-23T09:41:29.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.282+0000 7f50927bb640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f508c161450 0x7f508c0cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33956/0 (socket says 172.21.15.47:33956) 2024-03-23T09:41:29.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.282+0000 7f50927bb640 1 -- 172.21.15.47:0/2859306064 learned_addr learned my addr 172.21.15.47:0/2859306064 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5091fba640 1 --2- 172.21.15.47:0/2859306064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c06b690 0x7f508c06ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814377392 0 0) 0x7f508c0cffc0 con 0x7f508c161450 2024-03-23T09:41:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5074003520 con 0x7f508c161450 2024-03-23T09:41:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2645068005 0 0) 0x7f5074003520 con 0x7f508c161450 2024-03-23T09:41:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c06b690 msgr2=0x7f508c06ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 --2- 172.21.15.47:0/2859306064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c06b690 0x7f508c06ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 >> v1:172.21.15.175:6789/0 conn(0x7f508c0b1c60 legacy=0x7f508c0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c0cf690 con 0x7f508c161450 2024-03-23T09:41:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f507c0019f0 con 0x7f508c161450 2024-03-23T09:41:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 >> v1:172.21.15.47:6789/0 conn(0x7f508c161450 legacy=0x7f508c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 --2- 172.21.15.47:0/2859306064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50740039a0 0x7f5074003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 --2- 172.21.15.47:0/2859306064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c06b690 0x7f50740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 --2- 172.21.15.47:0/2859306064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074004d00 0x7f5074005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0cf3b0 con 0x7f5074004d00 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507400bac0 con 0x7f50740039a0 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507400bc80 con 0x7f508c06b690 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f50927bb640 1 --2- 172.21.15.47:0/2859306064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074004d00 0x7f5074005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f5091fba640 1 --2- 172.21.15.47:0/2859306064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50740039a0 0x7f5074003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f50917b9640 1 --2- 172.21.15.47:0/2859306064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c06b690 0x7f50740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f50927bb640 1 -- 172.21.15.47:0/2859306064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c06b690 msgr2=0x7f50740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f50927bb640 1 --2- 172.21.15.47:0/2859306064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c06b690 0x7f50740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f50927bb640 1 -- 172.21.15.47:0/2859306064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50740039a0 msgr2=0x7f5074003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f50927bb640 1 --2- 172.21.15.47:0/2859306064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50740039a0 0x7f5074003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f50927bb640 1 -- 172.21.15.47:0/2859306064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5074003520 con 0x7f5074004d00 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.283+0000 7f50917b9640 1 --2- 172.21.15.47:0/2859306064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c06b690 0x7f50740047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.284+0000 7f50927bb640 1 --2- 172.21.15.47:0/2859306064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074004d00 0x7f5074005180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f507c00a800 tx=0x7f50740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5fb386ea89520a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:29.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.284+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c0072d0 con 0x7f5074004d00 2024-03-23T09:41:29.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.284+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c02f070 con 0x7f5074004d00 2024-03-23T09:41:29.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.284+0000 7f5090fb8640 1 -- 172.21.15.47:0/2859306064 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c007690 con 0x7f5074004d00 2024-03-23T09:41:29.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.285+0000 7f509421e640 1 -- 172.21.15.47:0/2859306064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074004d00 msgr2=0x7f5074005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.285+0000 7f509421e640 1 --2- 172.21.15.47:0/2859306064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074004d00 0x7f5074005180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f507c00a800 tx=0x7f50740057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.285+0000 7f50927bb640 1 -- 172.21.15.47:0/2859306064 reap_dead start 2024-03-23T09:41:29.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.285+0000 7f509421e640 1 -- 172.21.15.47:0/2859306064 shutdown_connections 2024-03-23T09:41:29.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.285+0000 7f509421e640 1 -- 172.21.15.47:0/2859306064 >> 172.21.15.47:0/2859306064 conn(0x7f508c0b21d0 msgr2=0x7f508c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:29.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.285+0000 7f509421e640 1 -- 172.21.15.47:0/2859306064 shutdown_connections 2024-03-23T09:41:29.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.285+0000 7f509421e640 1 -- 172.21.15.47:0/2859306064 wait complete. 2024-03-23T09:41:29.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.286+0000 7f509421e640 1 Processor -- start 2024-03-23T09:41:29.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.286+0000 7f509421e640 1 -- start start 2024-03-23T09:41:29.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f509421e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074004d00 0x7f508c15ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f509421e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c1f29b0 0x7f508c157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f509421e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f508c1f2e30 0x7f508c158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f509421e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c15bc50 con 0x7f5074004d00 2024-03-23T09:41:29.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f509421e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c155630 con 0x7f508c1f2e30 2024-03-23T09:41:29.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f509421e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c155910 con 0x7f508c1f29b0 2024-03-23T09:41:29.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f5091fba640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074004d00 0x7f508c15ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f5091fba640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074004d00 0x7f508c15ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56452/0 (socket says 172.21.15.47:56452) 2024-03-23T09:41:29.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f5091fba640 1 -- 172.21.15.47:0/449469111 learned_addr learned my addr 172.21.15.47:0/449469111 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:29.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f50917b9640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c1f29b0 0x7f508c157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f50927bb640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f508c1f2e30 0x7f508c158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f5091fba640 1 -- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c1f29b0 msgr2=0x7f508c157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f5091fba640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c1f29b0 0x7f508c157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f5091fba640 1 -- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f508c1f2e30 msgr2=0x7f508c158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f5091fba640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f508c1f2e30 0x7f508c158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f5091fba640 1 -- 172.21.15.47:0/449469111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c155bf0 con 0x7f5074004d00 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f50927bb640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f508c1f2e30 0x7f508c158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f50917b9640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c1f29b0 0x7f508c157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f5091fba640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074004d00 0x7f508c15ab90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f50840118e0 tx=0x7f5084011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f508a7fc640 1 -- 172.21.15.47:0/449469111 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508400faf0 con 0x7f5074004d00 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f508a7fc640 1 -- 172.21.15.47:0/449469111 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508400fc50 con 0x7f5074004d00 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f509421e640 1 -- 172.21.15.47:0/449469111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c155ed0 con 0x7f5074004d00 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f508a7fc640 1 -- 172.21.15.47:0/449469111 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084014440 con 0x7f5074004d00 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.287+0000 7f509421e640 1 -- 172.21.15.47:0/449469111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c15b0d0 con 0x7f5074004d00 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.288+0000 7f509421e640 1 -- 172.21.15.47:0/449469111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c0672f0 con 0x7f5074004d00 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.289+0000 7f508a7fc640 1 -- 172.21.15.47:0/449469111 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f50840145a0 con 0x7f5074004d00 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.289+0000 7f508a7fc640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f507806ac00 0x7f507806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.289+0000 7f508a7fc640 1 -- 172.21.15.47:0/449469111 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5084089c40 con 0x7f5074004d00 2024-03-23T09:41:29.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.290+0000 7f50917b9640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f507806ac00 0x7f507806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.290+0000 7f50917b9640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f507806ac00 0x7f507806d0c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f508c0b6230 tx=0x7f5080009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:29.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.293+0000 7f508a7fc640 1 -- 172.21.15.47:0/449469111 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5084054e20 con 0x7f5074004d00 2024-03-23T09:41:29.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.473+0000 7f509421e640 1 -- 172.21.15.47:0/449469111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f508c06b470 con 0x7f5074004d00 2024-03-23T09:41:29.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.473+0000 7f508a7fc640 1 -- 172.21.15.47:0/449469111 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5084058440 con 0x7f5074004d00 2024-03-23T09:41:29.473 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:29.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.475+0000 7f509421e640 1 -- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f507806ac00 msgr2=0x7f507806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.475+0000 7f509421e640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f507806ac00 0x7f507806d0c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f508c0b6230 tx=0x7f5080009040 comp rx=0 tx=0).stop 2024-03-23T09:41:29.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.475+0000 7f509421e640 1 -- 172.21.15.47:0/449469111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074004d00 msgr2=0x7f508c15ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.475+0000 7f509421e640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074004d00 0x7f508c15ab90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f50840118e0 tx=0x7f5084011da0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.475+0000 7f509421e640 1 -- 172.21.15.47:0/449469111 shutdown_connections 2024-03-23T09:41:29.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.475+0000 7f509421e640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f507806ac00 0x7f507806d0c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.475+0000 7f509421e640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f508c1f2e30 0x7f508c158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.475+0000 7f509421e640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c1f29b0 0x7f508c157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.475+0000 7f509421e640 1 --2- 172.21.15.47:0/449469111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074004d00 0x7f508c15ab90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.475+0000 7f509421e640 1 -- 172.21.15.47:0/449469111 >> 172.21.15.47:0/449469111 conn(0x7f508c0b21d0 msgr2=0x7f508c0ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:29.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.475+0000 7f509421e640 1 -- 172.21.15.47:0/449469111 shutdown_connections 2024-03-23T09:41:29.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.475+0000 7f509421e640 1 -- 172.21.15.47:0/449469111 wait complete. 2024-03-23T09:41:29.485 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:29.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.589+0000 7f151aeb6640 1 Processor -- start 2024-03-23T09:41:29.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.589+0000 7f151aeb6640 1 -- start start 2024-03-23T09:41:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.590+0000 7f151aeb6640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f15140cffa0 0x7f15140ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.590+0000 7f151aeb6640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15140cfb10 con 0x7f1514098a00 2024-03-23T09:41:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.590+0000 7f151aeb6640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15140cef60 con 0x7f151415caa0 2024-03-23T09:41:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.590+0000 7f151aeb6640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15140cf200 con 0x7f15140cffa0 2024-03-23T09:41:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.590+0000 7f1519453640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1514098a00 0x7f15140cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33960/0 (socket says 172.21.15.47:33960) 2024-03-23T09:41:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.590+0000 7f1519453640 1 -- 172.21.15.47:0/1190923856 learned_addr learned my addr 172.21.15.47:0/1190923856 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.590+0000 7f1513fff640 1 --2- 172.21.15.47:0/1190923856 >> v2:172.21.15.175:3300/0 conn(0x7f15140cffa0 0x7f15140ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332396010 0 0) 0x7f15140cfb10 con 0x7f1514098a00 2024-03-23T09:41:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1500003520 con 0x7f1514098a00 2024-03-23T09:41:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303307432 0 0) 0x7f15140cef60 con 0x7f151415caa0 2024-03-23T09:41:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15140cfb10 con 0x7f151415caa0 2024-03-23T09:41:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2833992023 0 0) 0x7f1500003520 con 0x7f1514098a00 2024-03-23T09:41:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 >> v2:172.21.15.175:3300/0 conn(0x7f15140cffa0 msgr2=0x7f15140ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 --2- 172.21.15.47:0/1190923856 >> v2:172.21.15.175:3300/0 conn(0x7f15140cffa0 0x7f15140ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 >> v1:172.21.15.175:6789/0 conn(0x7f151415caa0 legacy=0x7f151406bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15140cf4c0 con 0x7f1514098a00 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f14fc001670 con 0x7f1514098a00 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 >> v1:172.21.15.47:6789/0 conn(0x7f1514098a00 legacy=0x7f15140cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 --2- 172.21.15.47:0/1190923856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1500003b00 0x7f1500003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 --2- 172.21.15.47:0/1190923856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15000047c0 0x7f1500004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 --2- 172.21.15.47:0/1190923856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1500005180 0x7f150000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15140cf200 con 0x7f1500003b00 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150000c000 con 0x7f1500005180 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150000c1c0 con 0x7f15000047c0 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f1518c52640 1 --2- 172.21.15.47:0/1190923856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1500003b00 0x7f1500003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f1513fff640 1 --2- 172.21.15.47:0/1190923856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15000047c0 0x7f1500004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f1519453640 1 --2- 172.21.15.47:0/1190923856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1500005180 0x7f150000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f1518c52640 1 -- 172.21.15.47:0/1190923856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15000047c0 msgr2=0x7f1500004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f1518c52640 1 --2- 172.21.15.47:0/1190923856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15000047c0 0x7f1500004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f1518c52640 1 -- 172.21.15.47:0/1190923856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1500005180 msgr2=0x7f150000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f1518c52640 1 --2- 172.21.15.47:0/1190923856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1500005180 0x7f150000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f1519453640 1 -- 172.21.15.47:0/1190923856 reap_dead start 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.591+0000 7f1518c52640 1 -- 172.21.15.47:0/1190923856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1500003520 con 0x7f1500003b00 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.592+0000 7f1518c52640 1 --2- 172.21.15.47:0/1190923856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1500003b00 0x7f1500003f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f150c001fd0 tx=0x7f15000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e52783ab785f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.592+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c003b70 con 0x7f1500003b00 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.592+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c003cd0 con 0x7f1500003b00 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.592+0000 7f15137fe640 1 -- 172.21.15.47:0/1190923856 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c0052f0 con 0x7f1500003b00 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.593+0000 7f151aeb6640 1 -- 172.21.15.47:0/1190923856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1500003b00 msgr2=0x7f1500003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.593+0000 7f151aeb6640 1 --2- 172.21.15.47:0/1190923856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1500003b00 0x7f1500003f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f150c001fd0 tx=0x7f15000057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.593+0000 7f151aeb6640 1 -- 172.21.15.47:0/1190923856 shutdown_connections 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.593+0000 7f151aeb6640 1 --2- 172.21.15.47:0/1190923856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1500003b00 0x7f1500003f00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.593+0000 7f151aeb6640 1 -- 172.21.15.47:0/1190923856 >> 172.21.15.47:0/1190923856 conn(0x7f15140b21d0 msgr2=0x7f15140b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:29.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.593+0000 7f151aeb6640 1 -- 172.21.15.47:0/1190923856 shutdown_connections 2024-03-23T09:41:29.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.593+0000 7f151aeb6640 1 -- 172.21.15.47:0/1190923856 wait complete. 2024-03-23T09:41:29.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.594+0000 7f151aeb6640 1 Processor -- start 2024-03-23T09:41:29.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.594+0000 7f151aeb6640 1 -- start start 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f151aeb6640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1500003b00 0x7f15140cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f151aeb6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15140c6d90 0x7f151415b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f151aeb6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15140b5b10 0x7f15140cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f151aeb6640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15140cc690 con 0x7f15140c6d90 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f151aeb6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15140cc930 con 0x7f15140b5b10 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f151aeb6640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15140ccc10 con 0x7f1500003b00 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f1513fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15140c6d90 0x7f151415b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f1513fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15140c6d90 0x7f151415b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56464/0 (socket says 172.21.15.47:56464) 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f1513fff640 1 -- 172.21.15.47:0/2886554854 learned_addr learned my addr 172.21.15.47:0/2886554854 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f1518c52640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1500003b00 0x7f15140cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f1513fff640 1 -- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1500003b00 msgr2=0x7f15140cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f1513fff640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1500003b00 0x7f15140cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f1513fff640 1 -- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15140b5b10 msgr2=0x7f15140cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f1513fff640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15140b5b10 0x7f15140cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f1513fff640 1 -- 172.21.15.47:0/2886554854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15140ccef0 con 0x7f15140c6d90 2024-03-23T09:41:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f1518c52640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1500003b00 0x7f15140cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.595+0000 7f1513fff640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15140c6d90 0x7f151415b1e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1504011a70 tx=0x7f1504011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.596+0000 7f15117fa640 1 -- 172.21.15.47:0/2886554854 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1504009680 con 0x7f15140c6d90 2024-03-23T09:41:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.596+0000 7f15117fa640 1 -- 172.21.15.47:0/2886554854 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150400fad0 con 0x7f15140c6d90 2024-03-23T09:41:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.596+0000 7f15117fa640 1 -- 172.21.15.47:0/2886554854 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150400fe60 con 0x7f15140c6d90 2024-03-23T09:41:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.596+0000 7f151aeb6640 1 -- 172.21.15.47:0/2886554854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15140cd1d0 con 0x7f15140c6d90 2024-03-23T09:41:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.596+0000 7f151aeb6640 1 -- 172.21.15.47:0/2886554854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15140cdc30 con 0x7f15140c6d90 2024-03-23T09:41:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.597+0000 7f151aeb6640 1 -- 172.21.15.47:0/2886554854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15140cdeb0 con 0x7f15140c6d90 2024-03-23T09:41:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.597+0000 7f15117fa640 1 -- 172.21.15.47:0/2886554854 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1504014440 con 0x7f15140c6d90 2024-03-23T09:41:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.597+0000 7f15117fa640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f14f406ac30 0x7f14f406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.597+0000 7f15117fa640 1 -- 172.21.15.47:0/2886554854 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1504089bd0 con 0x7f15140c6d90 2024-03-23T09:41:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.598+0000 7f1518c52640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f14f406ac30 0x7f14f406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.598+0000 7f1518c52640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f14f406ac30 0x7f14f406d0f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f150c034000 tx=0x7f150c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:29.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.601+0000 7f15117fa640 1 -- 172.21.15.47:0/2886554854 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1504054db0 con 0x7f15140c6d90 2024-03-23T09:41:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.779+0000 7f151aeb6640 1 -- 172.21.15.47:0/2886554854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15140672f0 con 0x7f15140c6d90 2024-03-23T09:41:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.779+0000 7f15117fa640 1 -- 172.21.15.47:0/2886554854 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1504022020 con 0x7f15140c6d90 2024-03-23T09:41:29.779 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.781+0000 7f151aeb6640 1 -- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f14f406ac30 msgr2=0x7f14f406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.781+0000 7f151aeb6640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f14f406ac30 0x7f14f406d0f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f150c034000 tx=0x7f150c009030 comp rx=0 tx=0).stop 2024-03-23T09:41:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.781+0000 7f151aeb6640 1 -- 172.21.15.47:0/2886554854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15140c6d90 msgr2=0x7f151415b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.781+0000 7f151aeb6640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15140c6d90 0x7f151415b1e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1504011a70 tx=0x7f1504011f30 comp rx=0 tx=0).stop 2024-03-23T09:41:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.781+0000 7f151aeb6640 1 -- 172.21.15.47:0/2886554854 shutdown_connections 2024-03-23T09:41:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.781+0000 7f151aeb6640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f14f406ac30 0x7f14f406d0f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.781+0000 7f151aeb6640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15140b5b10 0x7f15140cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.781+0000 7f151aeb6640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15140c6d90 0x7f151415b1e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.781+0000 7f151aeb6640 1 --2- 172.21.15.47:0/2886554854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1500003b00 0x7f15140cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.781+0000 7f151aeb6640 1 -- 172.21.15.47:0/2886554854 >> 172.21.15.47:0/2886554854 conn(0x7f15140b21d0 msgr2=0x7f15140ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.781+0000 7f151aeb6640 1 -- 172.21.15.47:0/2886554854 shutdown_connections 2024-03-23T09:41:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.781+0000 7f151aeb6640 1 -- 172.21.15.47:0/2886554854 wait complete. 2024-03-23T09:41:29.791 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:29.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.892+0000 7f0ebbc4a640 1 Processor -- start 2024-03-23T09:41:29.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.892+0000 7f0ebbc4a640 1 -- start start 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.893+0000 7f0ebbc4a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb406a3f0 0x7f0eb40cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.893+0000 7f0ebbc4a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb40c9ed0 con 0x7f0eb415cbc0 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.893+0000 7f0ebbc4a640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb40b0d10 con 0x7f0eb4098e10 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.893+0000 7f0ebbc4a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40b0ff0 con 0x7f0eb406a3f0 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.893+0000 7f0eb99e6640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0eb415cbc0 0x7f0eb4069ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33966/0 (socket says 172.21.15.47:33966) 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.893+0000 7f0eb99e6640 1 -- 172.21.15.47:0/2311422970 learned_addr learned my addr 172.21.15.47:0/2311422970 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb91e5640 1 --2- 172.21.15.47:0/2311422970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb406a3f0 0x7f0eb40cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009171289 0 0) 0x7f0eb40c9ed0 con 0x7f0eb415cbc0 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e9c003520 con 0x7f0eb415cbc0 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167440682 0 0) 0x7f0eb40b0d10 con 0x7f0eb4098e10 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eb40c9ed0 con 0x7f0eb4098e10 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1119505819 0 0) 0x7f0e9c003520 con 0x7f0eb415cbc0 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb406a3f0 msgr2=0x7f0eb40cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 --2- 172.21.15.47:0/2311422970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb406a3f0 0x7f0eb40cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 >> v1:172.21.15.175:6789/0 conn(0x7f0eb4098e10 legacy=0x7f0eb40ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40ca120 con 0x7f0eb415cbc0 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0eac001670 con 0x7f0eb415cbc0 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb91e5640 1 --2- 172.21.15.47:0/2311422970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb406a3f0 0x7f0eb40cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 >> v1:172.21.15.47:6789/0 conn(0x7f0eb415cbc0 legacy=0x7f0eb4069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 --2- 172.21.15.47:0/2311422970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9c003b30 0x7f0e9c003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 --2- 172.21.15.47:0/2311422970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb406a3f0 0x7f0e9c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 --2- 172.21.15.47:0/2311422970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c004d00 0x7f0e9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40b0ff0 con 0x7f0e9c004d00 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c00ba30 con 0x7f0e9c003b30 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c00bb70 con 0x7f0eb406a3f0 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eba1e7640 1 --2- 172.21.15.47:0/2311422970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c004d00 0x7f0e9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb99e6640 1 --2- 172.21.15.47:0/2311422970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9c003b30 0x7f0e9c003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eb91e5640 1 --2- 172.21.15.47:0/2311422970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb406a3f0 0x7f0e9c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eba1e7640 1 -- 172.21.15.47:0/2311422970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb406a3f0 msgr2=0x7f0e9c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eba1e7640 1 --2- 172.21.15.47:0/2311422970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb406a3f0 0x7f0e9c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eba1e7640 1 -- 172.21.15.47:0/2311422970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9c003b30 msgr2=0x7f0e9c003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.894+0000 7f0eba1e7640 1 --2- 172.21.15.47:0/2311422970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9c003b30 0x7f0e9c003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.895+0000 7f0eba1e7640 1 -- 172.21.15.47:0/2311422970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea8009400 con 0x7f0e9c004d00 2024-03-23T09:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.895+0000 7f0eba1e7640 1 --2- 172.21.15.47:0/2311422970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c004d00 0x7f0e9c005180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4002000 tx=0x7f0e9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7cc6cc3fce37a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.895+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4003b70 con 0x7f0e9c004d00 2024-03-23T09:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.895+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea4003cd0 con 0x7f0e9c004d00 2024-03-23T09:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.895+0000 7f0eb89e4640 1 -- 172.21.15.47:0/2311422970 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40052f0 con 0x7f0e9c004d00 2024-03-23T09:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.896+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/2311422970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c004d00 msgr2=0x7f0e9c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.896+0000 7f0ebbc4a640 1 --2- 172.21.15.47:0/2311422970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c004d00 0x7f0e9c005180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4002000 tx=0x7f0e9c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.896+0000 7f0eba1e7640 1 -- 172.21.15.47:0/2311422970 reap_dead start 2024-03-23T09:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.896+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/2311422970 shutdown_connections 2024-03-23T09:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.896+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/2311422970 >> 172.21.15.47:0/2311422970 conn(0x7f0eb4098a00 msgr2=0x7f0eb40cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:29.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.896+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/2311422970 shutdown_connections 2024-03-23T09:41:29.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.896+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/2311422970 wait complete. 2024-03-23T09:41:29.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.897+0000 7f0ebbc4a640 1 Processor -- start 2024-03-23T09:41:29.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.897+0000 7f0ebbc4a640 1 -- start start 2024-03-23T09:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0ebbc4a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb40cb510 0x7f0eb40cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0ebbc4a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb414b8b0 0x7f0eb414bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0ebbc4a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb40c6ca0 0x7f0eb40c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0ebbc4a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb414c270 con 0x7f0eb414b8b0 2024-03-23T09:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0ebbc4a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40c8650 con 0x7f0eb40cb510 2024-03-23T09:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0ebbc4a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40c7a40 con 0x7f0eb40c6ca0 2024-03-23T09:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0eb91e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb414b8b0 0x7f0eb414bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0eb91e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb414b8b0 0x7f0eb414bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56496/0 (socket says 172.21.15.47:56496) 2024-03-23T09:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0eb91e5640 1 -- 172.21.15.47:0/1378192279 learned_addr learned my addr 172.21.15.47:0/1378192279 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0eba1e7640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb40c6ca0 0x7f0eb40c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0eb99e6640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb40cb510 0x7f0eb40cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0eb91e5640 1 -- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb40c6ca0 msgr2=0x7f0eb40c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0eb91e5640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb40c6ca0 0x7f0eb40c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0eb91e5640 1 -- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb40cb510 msgr2=0x7f0eb40cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0eb91e5640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb40cb510 0x7f0eb40cb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.898+0000 7f0eb91e5640 1 -- 172.21.15.47:0/1378192279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40c7d20 con 0x7f0eb414b8b0 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.899+0000 7f0eb91e5640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb414b8b0 0x7f0eb414bd30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8011cd0 tx=0x7f0ea800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.899+0000 7f0eb27fc640 1 -- 172.21.15.47:0/1378192279 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea800fd20 con 0x7f0eb414b8b0 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.899+0000 7f0eb27fc640 1 -- 172.21.15.47:0/1378192279 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea800fe80 con 0x7f0eb414b8b0 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.899+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/1378192279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb40c8000 con 0x7f0eb414b8b0 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.899+0000 7f0eb27fc640 1 -- 172.21.15.47:0/1378192279 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8004240 con 0x7f0eb414b8b0 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.899+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/1378192279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb40c7690 con 0x7f0eb414b8b0 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.900+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/1378192279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb40672f0 con 0x7f0eb414b8b0 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.900+0000 7f0eb27fc640 1 -- 172.21.15.47:0/1378192279 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0ea80043a0 con 0x7f0eb414b8b0 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.900+0000 7f0eb27fc640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ea006ac00 0x7f0ea006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.901+0000 7f0eb27fc640 1 -- 172.21.15.47:0/1378192279 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ea8089a10 con 0x7f0eb414b8b0 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.901+0000 7f0eb99e6640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ea006ac00 0x7f0ea006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.901+0000 7f0eb99e6640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ea006ac00 0x7f0ea006d0c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0eac004840 tx=0x7f0eac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:29.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:29.905+0000 7f0eb27fc640 1 -- 172.21.15.47:0/1378192279 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0ea8054c80 con 0x7f0eb414b8b0 2024-03-23T09:41:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.084+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/1378192279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0eb406ab00 con 0x7f0eb414b8b0 2024-03-23T09:41:30.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.085+0000 7f0eb27fc640 1 -- 172.21.15.47:0/1378192279 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f0ea8022070 con 0x7f0eb414b8b0 2024-03-23T09:41:30.084 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:30.085 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.086+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ea006ac00 msgr2=0x7f0ea006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.086+0000 7f0ebbc4a640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ea006ac00 0x7f0ea006d0c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0eac004840 tx=0x7f0eac009040 comp rx=0 tx=0).stop 2024-03-23T09:41:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.086+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/1378192279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb414b8b0 msgr2=0x7f0eb414bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.086+0000 7f0ebbc4a640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb414b8b0 0x7f0eb414bd30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8011cd0 tx=0x7f0ea800f610 comp rx=0 tx=0).stop 2024-03-23T09:41:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.087+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/1378192279 shutdown_connections 2024-03-23T09:41:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.087+0000 7f0ebbc4a640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ea006ac00 0x7f0ea006d0c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.087+0000 7f0ebbc4a640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb40c6ca0 0x7f0eb40c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.087+0000 7f0ebbc4a640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb414b8b0 0x7f0eb414bd30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.087+0000 7f0ebbc4a640 1 --2- 172.21.15.47:0/1378192279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb40cb510 0x7f0eb40cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.087+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/1378192279 >> 172.21.15.47:0/1378192279 conn(0x7f0eb4098a00 msgr2=0x7f0eb406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.087+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/1378192279 shutdown_connections 2024-03-23T09:41:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.087+0000 7f0ebbc4a640 1 -- 172.21.15.47:0/1378192279 wait complete. 2024-03-23T09:41:30.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:30.097 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:30.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.201+0000 7f6711b0a640 1 Processor -- start 2024-03-23T09:41:30.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.201+0000 7f6711b0a640 1 -- start start 2024-03-23T09:41:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f6711b0a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f670c0cde10 0x7f670c160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f6711b0a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f670c0b8350 0x7f670c0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f6711b0a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670c161630 con 0x7f670c15c990 2024-03-23T09:41:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f6711b0a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670c0d1030 con 0x7f670c0cde10 2024-03-23T09:41:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f6711b0a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670c0cfe10 con 0x7f670c0b8350 2024-03-23T09:41:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f670affd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f670c0cde10 0x7f670c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f670b7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f670c15c990 0x7f670c0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33982/0 (socket says 172.21.15.47:33982) 2024-03-23T09:41:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f670b7fe640 1 -- 172.21.15.47:0/2581038577 learned_addr learned my addr 172.21.15.47:0/2581038577 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f670affd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f670c0cde10 0x7f670c160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56510/0 (socket says 172.21.15.47:56510) 2024-03-23T09:41:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f670bfff640 1 --2- 172.21.15.47:0/2581038577 >> v2:172.21.15.175:3300/0 conn(0x7f670c0b8350 0x7f670c0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f670affd640 1 -- 172.21.15.47:0/2581038577 >> v2:172.21.15.175:3300/0 conn(0x7f670c0b8350 msgr2=0x7f670c0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f670affd640 1 --2- 172.21.15.47:0/2581038577 >> v2:172.21.15.175:3300/0 conn(0x7f670c0b8350 0x7f670c0d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f670affd640 1 -- 172.21.15.47:0/2581038577 >> v1:172.21.15.47:6789/0 conn(0x7f670c15c990 legacy=0x7f670c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f670affd640 1 -- 172.21.15.47:0/2581038577 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c0b5b60 con 0x7f670c0cde10 2024-03-23T09:41:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.202+0000 7f670bfff640 1 --2- 172.21.15.47:0/2581038577 >> v2:172.21.15.175:3300/0 conn(0x7f670c0b8350 0x7f670c0d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.203+0000 7f670affd640 1 --2- 172.21.15.47:0/2581038577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f670c0cde10 0x7f670c160690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f670c0d0fa0 tx=0x7f66fc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7078a9ed41d762c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.203+0000 7f670a7fc640 1 -- 172.21.15.47:0/2581038577 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc02e070 con 0x7f670c0cde10 2024-03-23T09:41:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.203+0000 7f670a7fc640 1 -- 172.21.15.47:0/2581038577 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66fc0315a0 con 0x7f670c0cde10 2024-03-23T09:41:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.203+0000 7f670a7fc640 1 -- 172.21.15.47:0/2581038577 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc031930 con 0x7f670c0cde10 2024-03-23T09:41:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.204+0000 7f6711b0a640 1 -- 172.21.15.47:0/2581038577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f670c0cde10 msgr2=0x7f670c160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.204+0000 7f6711b0a640 1 --2- 172.21.15.47:0/2581038577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f670c0cde10 0x7f670c160690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f670c0d0fa0 tx=0x7f66fc02b840 comp rx=0 tx=0).stop 2024-03-23T09:41:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.204+0000 7f6711b0a640 1 -- 172.21.15.47:0/2581038577 shutdown_connections 2024-03-23T09:41:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.204+0000 7f6711b0a640 1 --2- 172.21.15.47:0/2581038577 >> v2:172.21.15.175:3300/0 conn(0x7f670c0b8350 0x7f670c0d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.204+0000 7f6711b0a640 1 --2- 172.21.15.47:0/2581038577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f670c0cde10 0x7f670c160690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.204+0000 7f6711b0a640 1 -- 172.21.15.47:0/2581038577 >> 172.21.15.47:0/2581038577 conn(0x7f670c0659d0 msgr2=0x7f670c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.204+0000 7f6711b0a640 1 -- 172.21.15.47:0/2581038577 shutdown_connections 2024-03-23T09:41:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.204+0000 7f6711b0a640 1 -- 172.21.15.47:0/2581038577 wait complete. 2024-03-23T09:41:30.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.205+0000 7f6711b0a640 1 Processor -- start 2024-03-23T09:41:30.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.205+0000 7f6711b0a640 1 -- start start 2024-03-23T09:41:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f6711b0a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670c0b8350 0x7f670c153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f6711b0a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0cde10 0x7f670c15eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f6711b0a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f670c15c990 0x7f670c0b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f6711b0a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c1543b0 con 0x7f670c15c990 2024-03-23T09:41:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f6711b0a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c0cc2e0 con 0x7f670c0b8350 2024-03-23T09:41:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f6711b0a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670c0caf60 con 0x7f670c0cde10 2024-03-23T09:41:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f670bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f670c15c990 0x7f670c0b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f670bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f670c15c990 0x7f670c0b6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56512/0 (socket says 172.21.15.47:56512) 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f670bfff640 1 -- 172.21.15.47:0/655152415 learned_addr learned my addr 172.21.15.47:0/655152415 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f670b7fe640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670c0b8350 0x7f670c153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f670bfff640 1 -- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0cde10 msgr2=0x7f670c15eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f670bfff640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0cde10 0x7f670c15eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f670bfff640 1 -- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670c0b8350 msgr2=0x7f670c153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f670bfff640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670c0b8350 0x7f670c153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f670bfff640 1 -- 172.21.15.47:0/655152415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc009400 con 0x7f670c15c990 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f670b7fe640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670c0b8350 0x7f670c153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.206+0000 7f670bfff640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f670c15c990 0x7f670c0b6ea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f670400cd50 tx=0x7f670400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.207+0000 7f66f3fff640 1 -- 172.21.15.47:0/655152415 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670400f680 con 0x7f670c15c990 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.207+0000 7f66f3fff640 1 -- 172.21.15.47:0/655152415 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670400f800 con 0x7f670c15c990 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.207+0000 7f6711b0a640 1 -- 172.21.15.47:0/655152415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670c0cb240 con 0x7f670c15c990 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.207+0000 7f66f3fff640 1 -- 172.21.15.47:0/655152415 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670400fbb0 con 0x7f670c15c990 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.207+0000 7f6711b0a640 1 -- 172.21.15.47:0/655152415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670c0cb800 con 0x7f670c15c990 2024-03-23T09:41:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.207+0000 7f6711b0a640 1 -- 172.21.15.47:0/655152415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670c072360 con 0x7f670c15c990 2024-03-23T09:41:30.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.208+0000 7f66f3fff640 1 -- 172.21.15.47:0/655152415 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f670400b040 con 0x7f670c15c990 2024-03-23T09:41:30.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.208+0000 7f66f3fff640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f66f806ac00 0x7f66f806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.208+0000 7f66f3fff640 1 -- 172.21.15.47:0/655152415 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6704089e50 con 0x7f670c15c990 2024-03-23T09:41:30.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.209+0000 7f670b7fe640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f66f806ac00 0x7f66f806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.209+0000 7f670b7fe640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f66f806ac00 0x7f66f806d0c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6700007fd0 tx=0x7f6700009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:30.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.212+0000 7f66f3fff640 1 -- 172.21.15.47:0/655152415 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6704054f90 con 0x7f670c15c990 2024-03-23T09:41:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.392+0000 7f6711b0a640 1 -- 172.21.15.47:0/655152415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f670c066be0 con 0x7f670c15c990 2024-03-23T09:41:30.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.392+0000 7f66f3fff640 1 -- 172.21.15.47:0/655152415 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67040585b0 con 0x7f670c15c990 2024-03-23T09:41:30.392 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:30.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.394+0000 7f6711b0a640 1 -- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f66f806ac00 msgr2=0x7f66f806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.394+0000 7f6711b0a640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f66f806ac00 0x7f66f806d0c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6700007fd0 tx=0x7f6700009040 comp rx=0 tx=0).stop 2024-03-23T09:41:30.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.394+0000 7f6711b0a640 1 -- 172.21.15.47:0/655152415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f670c15c990 msgr2=0x7f670c0b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.394+0000 7f6711b0a640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f670c15c990 0x7f670c0b6ea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f670400cd50 tx=0x7f670400a650 comp rx=0 tx=0).stop 2024-03-23T09:41:30.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.394+0000 7f6711b0a640 1 -- 172.21.15.47:0/655152415 shutdown_connections 2024-03-23T09:41:30.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.394+0000 7f6711b0a640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f66f806ac00 0x7f66f806d0c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.394+0000 7f6711b0a640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f670c15c990 0x7f670c0b6ea0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.394+0000 7f6711b0a640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0cde10 0x7f670c15eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.394+0000 7f6711b0a640 1 --2- 172.21.15.47:0/655152415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670c0b8350 0x7f670c153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.394+0000 7f6711b0a640 1 -- 172.21.15.47:0/655152415 >> 172.21.15.47:0/655152415 conn(0x7f670c0659d0 msgr2=0x7f670c0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.394+0000 7f6711b0a640 1 -- 172.21.15.47:0/655152415 shutdown_connections 2024-03-23T09:41:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.394+0000 7f6711b0a640 1 -- 172.21.15.47:0/655152415 wait complete. 2024-03-23T09:41:30.404 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:30.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.508+0000 7ff1c0454640 1 Processor -- start 2024-03-23T09:41:30.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.508+0000 7ff1c0454640 1 -- start start 2024-03-23T09:41:30.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.509+0000 7ff1c0454640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff1b80cde10 0x7ff1b8160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.509+0000 7ff1c0454640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8350 0x7ff1b80d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.509+0000 7ff1c0454640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b8161630 con 0x7ff1b815c990 2024-03-23T09:41:30.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.509+0000 7ff1c0454640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b80d1030 con 0x7ff1b80cde10 2024-03-23T09:41:30.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.509+0000 7ff1c0454640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b80cfe10 con 0x7ff1b80b8350 2024-03-23T09:41:30.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.509+0000 7ff1bd9ef640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff1b80cde10 0x7ff1b8160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.509+0000 7ff1bd9ef640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff1b80cde10 0x7ff1b8160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56520/0 (socket says 172.21.15.47:56520) 2024-03-23T09:41:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.509+0000 7ff1bd9ef640 1 -- 172.21.15.47:0/2849679474 learned_addr learned my addr 172.21.15.47:0/2849679474 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.509+0000 7ff1be1f0640 1 --1- 172.21.15.47:0/2849679474 >> v1:172.21.15.47:6789/0 conn(0x7ff1b815c990 0x7ff1b80cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33998/0 (socket says 172.21.15.47:33998) 2024-03-23T09:41:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.509+0000 7ff1be9f1640 1 --2- 172.21.15.47:0/2849679474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8350 0x7ff1b80d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.510+0000 7ff1bd9ef640 1 -- 172.21.15.47:0/2849679474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8350 msgr2=0x7ff1b80d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.510+0000 7ff1bd9ef640 1 --2- 172.21.15.47:0/2849679474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8350 0x7ff1b80d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.510+0000 7ff1bd9ef640 1 -- 172.21.15.47:0/2849679474 >> v1:172.21.15.47:6789/0 conn(0x7ff1b815c990 legacy=0x7ff1b80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.510+0000 7ff1bd9ef640 1 -- 172.21.15.47:0/2849679474 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b80b5b60 con 0x7ff1b80cde10 2024-03-23T09:41:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.510+0000 7ff1bd1ee640 1 -- 172.21.15.47:0/2849679474 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566468858 0 0) 0x7ff1b8161630 con 0x7ff1b815c990 2024-03-23T09:41:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.510+0000 7ff1bd9ef640 1 --2- 172.21.15.47:0/2849679474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b80cde10 0x7ff1b8160690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009870 tx=0x7ff1ac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b752a7984cf323 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.510+0000 7ff1bd1ee640 1 -- 172.21.15.47:0/2849679474 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac031640 con 0x7ff1b80cde10 2024-03-23T09:41:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.510+0000 7ff1bd1ee640 1 -- 172.21.15.47:0/2849679474 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac0317a0 con 0x7ff1b80cde10 2024-03-23T09:41:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.510+0000 7ff1bd1ee640 1 -- 172.21.15.47:0/2849679474 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac031b30 con 0x7ff1b80cde10 2024-03-23T09:41:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.511+0000 7ff1c0454640 1 -- 172.21.15.47:0/2849679474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b80cde10 msgr2=0x7ff1b8160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.511+0000 7ff1c0454640 1 --2- 172.21.15.47:0/2849679474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b80cde10 0x7ff1b8160690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009870 tx=0x7ff1ac02b8f0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.511+0000 7ff1c0454640 1 -- 172.21.15.47:0/2849679474 shutdown_connections 2024-03-23T09:41:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.511+0000 7ff1c0454640 1 --2- 172.21.15.47:0/2849679474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8350 0x7ff1b80d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.511+0000 7ff1c0454640 1 --2- 172.21.15.47:0/2849679474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b80cde10 0x7ff1b8160690 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.511+0000 7ff1c0454640 1 -- 172.21.15.47:0/2849679474 >> 172.21.15.47:0/2849679474 conn(0x7ff1b80659d0 msgr2=0x7ff1b8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.511+0000 7ff1c0454640 1 -- 172.21.15.47:0/2849679474 shutdown_connections 2024-03-23T09:41:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.511+0000 7ff1c0454640 1 -- 172.21.15.47:0/2849679474 wait complete. 2024-03-23T09:41:30.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.512+0000 7ff1c0454640 1 Processor -- start 2024-03-23T09:41:30.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.512+0000 7ff1c0454640 1 -- start start 2024-03-23T09:41:30.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1c0454640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8350 0x7ff1b815a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1c0454640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b80cde10 0x7ff1b8153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1c0454640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1b815c990 0x7ff1b80b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1c0454640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b815a550 con 0x7ff1b80cde10 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1c0454640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b815f8b0 con 0x7ff1b815c990 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1c0454640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b8154440 con 0x7ff1b80b8350 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1bd9ef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b80cde10 0x7ff1b8153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1bd9ef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b80cde10 0x7ff1b8153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56532/0 (socket says 172.21.15.47:56532) 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1bd9ef640 1 -- 172.21.15.47:0/2737701907 learned_addr learned my addr 172.21.15.47:0/2737701907 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1bd9ef640 1 -- 172.21.15.47:0/2737701907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8350 msgr2=0x7ff1b815a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1bd9ef640 1 --2- 172.21.15.47:0/2737701907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8350 0x7ff1b815a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1bd9ef640 1 -- 172.21.15.47:0/2737701907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1b815c990 msgr2=0x7ff1b80b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1bd9ef640 1 --2- 172.21.15.47:0/2737701907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1b815c990 0x7ff1b80b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.513+0000 7ff1bd9ef640 1 -- 172.21.15.47:0/2737701907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac0094d0 con 0x7ff1b80cde10 2024-03-23T09:41:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.514+0000 7ff1bd9ef640 1 --2- 172.21.15.47:0/2737701907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b80cde10 0x7ff1b8153f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac000c00 tx=0x7ff1ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.514+0000 7ff1b6ffd640 1 -- 172.21.15.47:0/2737701907 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac004230 con 0x7ff1b80cde10 2024-03-23T09:41:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.514+0000 7ff1b6ffd640 1 -- 172.21.15.47:0/2737701907 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac004390 con 0x7ff1b80cde10 2024-03-23T09:41:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.514+0000 7ff1b6ffd640 1 -- 172.21.15.47:0/2737701907 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac03baa0 con 0x7ff1b80cde10 2024-03-23T09:41:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.514+0000 7ff1c0454640 1 -- 172.21.15.47:0/2737701907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b815abd0 con 0x7ff1b80cde10 2024-03-23T09:41:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.514+0000 7ff1c0454640 1 -- 172.21.15.47:0/2737701907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b815ef80 con 0x7ff1b80cde10 2024-03-23T09:41:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.515+0000 7ff1c0454640 1 -- 172.21.15.47:0/2737701907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b8072360 con 0x7ff1b80cde10 2024-03-23T09:41:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.515+0000 7ff1b6ffd640 1 -- 172.21.15.47:0/2737701907 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff1ac043070 con 0x7ff1b80cde10 2024-03-23T09:41:30.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.515+0000 7ff1b6ffd640 1 --2- 172.21.15.47:0/2737701907 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff1a406ac00 0x7ff1a406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.516+0000 7ff1b6ffd640 1 -- 172.21.15.47:0/2737701907 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1ac0ab020 con 0x7ff1b80cde10 2024-03-23T09:41:30.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.516+0000 7ff1be1f0640 1 --2- 172.21.15.47:0/2737701907 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff1a406ac00 0x7ff1a406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.516+0000 7ff1be1f0640 1 --2- 172.21.15.47:0/2737701907 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff1a406ac00 0x7ff1a406d0c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0007fd0 tx=0x7ff1b0007960 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:30.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.520+0000 7ff1b6ffd640 1 -- 172.21.15.47:0/2737701907 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff1ac076320 con 0x7ff1b80cde10 2024-03-23T09:41:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.702+0000 7ff1c0454640 1 -- 172.21.15.47:0/2737701907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1b8066be0 con 0x7ff1b80cde10 2024-03-23T09:41:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.702+0000 7ff1b6ffd640 1 -- 172.21.15.47:0/2737701907 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1ac079940 con 0x7ff1b80cde10 2024-03-23T09:41:30.702 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:30.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.704+0000 7ff1c0454640 1 -- 172.21.15.47:0/2737701907 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff1a406ac00 msgr2=0x7ff1a406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.704+0000 7ff1c0454640 1 --2- 172.21.15.47:0/2737701907 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff1a406ac00 0x7ff1a406d0c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0007fd0 tx=0x7ff1b0007960 comp rx=0 tx=0).stop 2024-03-23T09:41:30.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.704+0000 7ff1c0454640 1 -- 172.21.15.47:0/2737701907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b80cde10 msgr2=0x7ff1b8153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.704+0000 7ff1c0454640 1 --2- 172.21.15.47:0/2737701907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b80cde10 0x7ff1b8153f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac000c00 tx=0x7ff1ac0046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.704+0000 7ff1c0454640 1 -- 172.21.15.47:0/2737701907 shutdown_connections 2024-03-23T09:41:30.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.704+0000 7ff1c0454640 1 --2- 172.21.15.47:0/2737701907 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff1a406ac00 0x7ff1a406d0c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.704+0000 7ff1c0454640 1 --2- 172.21.15.47:0/2737701907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1b815c990 0x7ff1b80b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.704+0000 7ff1c0454640 1 --2- 172.21.15.47:0/2737701907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b80cde10 0x7ff1b8153f00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.704+0000 7ff1c0454640 1 --2- 172.21.15.47:0/2737701907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8350 0x7ff1b815a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.704+0000 7ff1c0454640 1 -- 172.21.15.47:0/2737701907 >> 172.21.15.47:0/2737701907 conn(0x7ff1b80659d0 msgr2=0x7ff1b80cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:30.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.704+0000 7ff1c0454640 1 -- 172.21.15.47:0/2737701907 shutdown_connections 2024-03-23T09:41:30.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.704+0000 7ff1c0454640 1 -- 172.21.15.47:0/2737701907 wait complete. 2024-03-23T09:41:30.714 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:30.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.818+0000 7fbcda1c9640 1 Processor -- start 2024-03-23T09:41:30.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.818+0000 7fbcda1c9640 1 -- start start 2024-03-23T09:41:30.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.819+0000 7fbcda1c9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbcd415ce60 0x7fbcd40ca000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.819+0000 7fbcda1c9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbcd406ef10 0x7fbcd40cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.819+0000 7fbcda1c9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd40ca600 con 0x7fbcd406f330 2024-03-23T09:41:30.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.819+0000 7fbcda1c9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd40cb280 con 0x7fbcd406ef10 2024-03-23T09:41:30.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.819+0000 7fbcda1c9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd40ca8e0 con 0x7fbcd415ce60 2024-03-23T09:41:30.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.819+0000 7fbcd2ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbcd406ef10 0x7fbcd40cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.819+0000 7fbcd2ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbcd406ef10 0x7fbcd40cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56544/0 (socket says 172.21.15.47:56544) 2024-03-23T09:41:30.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.819+0000 7fbcd2ffd640 1 -- 172.21.15.47:0/2503060785 learned_addr learned my addr 172.21.15.47:0/2503060785 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:30.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.819+0000 7fbcd37fe640 1 --2- 172.21.15.47:0/2503060785 >> v2:172.21.15.175:3300/0 conn(0x7fbcd415ce60 0x7fbcd40ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.819+0000 7fbcd2ffd640 1 -- 172.21.15.47:0/2503060785 >> v2:172.21.15.175:3300/0 conn(0x7fbcd415ce60 msgr2=0x7fbcd40ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:30.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.819+0000 7fbcd2ffd640 1 --2- 172.21.15.47:0/2503060785 >> v2:172.21.15.175:3300/0 conn(0x7fbcd415ce60 0x7fbcd40ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.819+0000 7fbcd2ffd640 1 -- 172.21.15.47:0/2503060785 >> v1:172.21.15.175:6789/0 conn(0x7fbcd406f330 legacy=0x7fbcd40cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:30.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.819+0000 7fbcd2ffd640 1 -- 172.21.15.47:0/2503060785 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd40cabc0 con 0x7fbcd406ef10 2024-03-23T09:41:30.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.820+0000 7fbcd2ffd640 1 --2- 172.21.15.47:0/2503060785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd406ef10 0x7fbcd40cd7d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4070dd0 tx=0x7fbcc802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebcee26307e1001 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:30.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.820+0000 7fbcd27fc640 1 -- 172.21.15.47:0/2503060785 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8030860 con 0x7fbcd406ef10 2024-03-23T09:41:30.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.820+0000 7fbcd27fc640 1 -- 172.21.15.47:0/2503060785 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc80309c0 con 0x7fbcd406ef10 2024-03-23T09:41:30.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.820+0000 7fbcd27fc640 1 -- 172.21.15.47:0/2503060785 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8030d50 con 0x7fbcd406ef10 2024-03-23T09:41:30.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.821+0000 7fbcda1c9640 1 -- 172.21.15.47:0/2503060785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd406ef10 msgr2=0x7fbcd40cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.821+0000 7fbcda1c9640 1 --2- 172.21.15.47:0/2503060785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd406ef10 0x7fbcd40cd7d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4070dd0 tx=0x7fbcc802b9a0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.821+0000 7fbcda1c9640 1 -- 172.21.15.47:0/2503060785 shutdown_connections 2024-03-23T09:41:30.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.821+0000 7fbcda1c9640 1 --2- 172.21.15.47:0/2503060785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd406ef10 0x7fbcd40cd7d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.821+0000 7fbcda1c9640 1 --2- 172.21.15.47:0/2503060785 >> v2:172.21.15.175:3300/0 conn(0x7fbcd415ce60 0x7fbcd40ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.821+0000 7fbcda1c9640 1 -- 172.21.15.47:0/2503060785 >> 172.21.15.47:0/2503060785 conn(0x7fbcd4098a00 msgr2=0x7fbcd40cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:30.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.821+0000 7fbcda1c9640 1 -- 172.21.15.47:0/2503060785 shutdown_connections 2024-03-23T09:41:30.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.821+0000 7fbcda1c9640 1 -- 172.21.15.47:0/2503060785 wait complete. 2024-03-23T09:41:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.822+0000 7fbcda1c9640 1 Processor -- start 2024-03-23T09:41:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.822+0000 7fbcda1c9640 1 -- start start 2024-03-23T09:41:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcda1c9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd406ef10 0x7fbcd40c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcda1c9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcd406f330 0x7fbcd40b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcda1c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd415ce60 0x7fbcd40c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcda1c9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40c55a0 con 0x7fbcd415ce60 2024-03-23T09:41:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcda1c9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40c4650 con 0x7fbcd406ef10 2024-03-23T09:41:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcda1c9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40c4920 con 0x7fbcd406f330 2024-03-23T09:41:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcd3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd415ce60 0x7fbcd40c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcd3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd415ce60 0x7fbcd40c4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56552/0 (socket says 172.21.15.47:56552) 2024-03-23T09:41:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcd3fff640 1 -- 172.21.15.47:0/3558083684 learned_addr learned my addr 172.21.15.47:0/3558083684 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcd37fe640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd406ef10 0x7fbcd40c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcd2ffd640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcd406f330 0x7fbcd40b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcd3fff640 1 -- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcd406f330 msgr2=0x7fbcd40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcd3fff640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcd406f330 0x7fbcd40b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcd3fff640 1 -- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd406ef10 msgr2=0x7fbcd40c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcd3fff640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd406ef10 0x7fbcd40c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcd3fff640 1 -- 172.21.15.47:0/3558083684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc80094d0 con 0x7fbcd415ce60 2024-03-23T09:41:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.823+0000 7fbcd37fe640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd406ef10 0x7fbcd40c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.824+0000 7fbcd3fff640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd415ce60 0x7fbcd40c4110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00b9e0 tx=0x7fbccc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.824+0000 7fbcbbfff640 1 -- 172.21.15.47:0/3558083684 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00cb20 con 0x7fbcd415ce60 2024-03-23T09:41:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.824+0000 7fbcbbfff640 1 -- 172.21.15.47:0/3558083684 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc00cc80 con 0x7fbcd415ce60 2024-03-23T09:41:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.824+0000 7fbcda1c9640 1 -- 172.21.15.47:0/3558083684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd40c4c00 con 0x7fbcd415ce60 2024-03-23T09:41:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.824+0000 7fbcbbfff640 1 -- 172.21.15.47:0/3558083684 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc014680 con 0x7fbcd415ce60 2024-03-23T09:41:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.824+0000 7fbcda1c9640 1 -- 172.21.15.47:0/3558083684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd40c51c0 con 0x7fbcd415ce60 2024-03-23T09:41:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.825+0000 7fbcda1c9640 1 -- 172.21.15.47:0/3558083684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd40672f0 con 0x7fbcd415ce60 2024-03-23T09:41:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.825+0000 7fbcbbfff640 1 -- 172.21.15.47:0/3558083684 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbccc01d3b0 con 0x7fbcd415ce60 2024-03-23T09:41:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.826+0000 7fbcbbfff640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbcc006ac00 0x7fbcc006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.826+0000 7fbcbbfff640 1 -- 172.21.15.47:0/3558083684 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbccc089280 con 0x7fbcd415ce60 2024-03-23T09:41:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.826+0000 7fbcd37fe640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbcc006ac00 0x7fbcc006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.827+0000 7fbcd37fe640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbcc006ac00 0x7fbcc006d0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400a9d0 tx=0x7fbcc4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:30.830+0000 7fbcbbfff640 1 -- 172.21.15.47:0/3558083684 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbccc054470 con 0x7fbcd415ce60 2024-03-23T09:41:31.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.010+0000 7fbcda1c9640 1 -- 172.21.15.47:0/3558083684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbcd40d0440 con 0x7fbcd415ce60 2024-03-23T09:41:31.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.010+0000 7fbcbbfff640 1 -- 172.21.15.47:0/3558083684 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbccc057a90 con 0x7fbcd415ce60 2024-03-23T09:41:31.010 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:31.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.012+0000 7fbcda1c9640 1 -- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbcc006ac00 msgr2=0x7fbcc006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.012+0000 7fbcda1c9640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbcc006ac00 0x7fbcc006d0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400a9d0 tx=0x7fbcc4008040 comp rx=0 tx=0).stop 2024-03-23T09:41:31.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.012+0000 7fbcda1c9640 1 -- 172.21.15.47:0/3558083684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd415ce60 msgr2=0x7fbcd40c4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.012+0000 7fbcda1c9640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd415ce60 0x7fbcd40c4110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00b9e0 tx=0x7fbccc00bea0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.012+0000 7fbcda1c9640 1 -- 172.21.15.47:0/3558083684 shutdown_connections 2024-03-23T09:41:31.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.012+0000 7fbcda1c9640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbcc006ac00 0x7fbcc006d0c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.012+0000 7fbcda1c9640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd415ce60 0x7fbcd40c4110 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.012+0000 7fbcda1c9640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcd406f330 0x7fbcd40b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.012+0000 7fbcda1c9640 1 --2- 172.21.15.47:0/3558083684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd406ef10 0x7fbcd40c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.012+0000 7fbcda1c9640 1 -- 172.21.15.47:0/3558083684 >> 172.21.15.47:0/3558083684 conn(0x7fbcd4098a00 msgr2=0x7fbcd40cbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:31.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.012+0000 7fbcda1c9640 1 -- 172.21.15.47:0/3558083684 shutdown_connections 2024-03-23T09:41:31.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.012+0000 7fbcda1c9640 1 -- 172.21.15.47:0/3558083684 wait complete. 2024-03-23T09:41:31.023 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:31.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.123+0000 7f16973c9640 1 Processor -- start 2024-03-23T09:41:31.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.124+0000 7f16973c9640 1 -- start start 2024-03-23T09:41:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.124+0000 7f16973c9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f169015a8d0 0x7f16900cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.124+0000 7f16973c9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f16900cb730 0x7f16900cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.124+0000 7f16973c9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16900cc610 con 0x7f16900cf190 2024-03-23T09:41:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.124+0000 7f16973c9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16900c41c0 con 0x7f16900cb730 2024-03-23T09:41:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.124+0000 7f16973c9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16900cf600 con 0x7f169015a8d0 2024-03-23T09:41:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.125+0000 7f1695165640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f169015a8d0 0x7f16900cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.125+0000 7f1694964640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f16900cb730 0x7f16900cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.125+0000 7f1695966640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f16900cf190 0x7f1690098a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:44394/0 (socket says 172.21.15.47:44394) 2024-03-23T09:41:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.125+0000 7f1695165640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f169015a8d0 0x7f16900cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:59312/0 (socket says 172.21.15.47:59312) 2024-03-23T09:41:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.125+0000 7f1695966640 1 -- 172.21.15.47:0/3233065796 learned_addr learned my addr 172.21.15.47:0/3233065796 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.125+0000 7f1694964640 1 -- 172.21.15.47:0/3233065796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f169015a8d0 msgr2=0x7f16900cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.125+0000 7f1694964640 1 --2- 172.21.15.47:0/3233065796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f169015a8d0 0x7f16900cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.125+0000 7f1694964640 1 -- 172.21.15.47:0/3233065796 >> v1:172.21.15.175:6789/0 conn(0x7f16900cf190 legacy=0x7f1690098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.125+0000 7f1687fff640 1 -- 172.21.15.47:0/3233065796 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646910281 0 0) 0x7f16900cc610 con 0x7f16900cf190 2024-03-23T09:41:31.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.125+0000 7f1694964640 1 -- 172.21.15.47:0/3233065796 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16900b0700 con 0x7f16900cb730 2024-03-23T09:41:31.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.125+0000 7f1695165640 1 --2- 172.21.15.47:0/3233065796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f169015a8d0 0x7f16900cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:31.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.126+0000 7f1694964640 1 --2- 172.21.15.47:0/3233065796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16900cb730 0x7f16900cec50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16800097d0 tx=0x7f168002b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fabb395c4886ff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:31.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.126+0000 7f1687fff640 1 -- 172.21.15.47:0/3233065796 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680031770 con 0x7f16900cb730 2024-03-23T09:41:31.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.126+0000 7f1687fff640 1 -- 172.21.15.47:0/3233065796 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16800318d0 con 0x7f16900cb730 2024-03-23T09:41:31.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.126+0000 7f1687fff640 1 -- 172.21.15.47:0/3233065796 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680031c60 con 0x7f16900cb730 2024-03-23T09:41:31.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.127+0000 7f16973c9640 1 -- 172.21.15.47:0/3233065796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16900cb730 msgr2=0x7f16900cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.127+0000 7f16973c9640 1 --2- 172.21.15.47:0/3233065796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16900cb730 0x7f16900cec50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16800097d0 tx=0x7f168002b810 comp rx=0 tx=0).stop 2024-03-23T09:41:31.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.127+0000 7f16973c9640 1 -- 172.21.15.47:0/3233065796 shutdown_connections 2024-03-23T09:41:31.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.127+0000 7f16973c9640 1 --2- 172.21.15.47:0/3233065796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16900cb730 0x7f16900cec50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.127+0000 7f16973c9640 1 --2- 172.21.15.47:0/3233065796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f169015a8d0 0x7f16900cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.127+0000 7f16973c9640 1 -- 172.21.15.47:0/3233065796 >> 172.21.15.47:0/3233065796 conn(0x7f16900d0670 msgr2=0x7f16900d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:31.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.127+0000 7f16973c9640 1 -- 172.21.15.47:0/3233065796 shutdown_connections 2024-03-23T09:41:31.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.127+0000 7f16973c9640 1 -- 172.21.15.47:0/3233065796 wait complete. 2024-03-23T09:41:31.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.128+0000 7f16973c9640 1 Processor -- start 2024-03-23T09:41:31.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.128+0000 7f16973c9640 1 -- start start 2024-03-23T09:41:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f16973c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16900cb730 0x7f16900b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f16973c9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16900cf190 0x7f16900b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f16973c9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f169015a8d0 0x7f1690147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f16973c9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16900c77e0 con 0x7f16900cb730 2024-03-23T09:41:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f16973c9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16900c7ac0 con 0x7f169015a8d0 2024-03-23T09:41:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f16973c9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16900c7da0 con 0x7f16900cf190 2024-03-23T09:41:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1695165640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16900cb730 0x7f16900b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1695165640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16900cb730 0x7f16900b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56556/0 (socket says 172.21.15.47:56556) 2024-03-23T09:41:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1695165640 1 -- 172.21.15.47:0/2046790125 learned_addr learned my addr 172.21.15.47:0/2046790125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1695966640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f169015a8d0 0x7f1690147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1694964640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16900cf190 0x7f16900b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1695165640 1 -- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16900cf190 msgr2=0x7f16900b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1695165640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16900cf190 0x7f16900b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1695165640 1 -- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f169015a8d0 msgr2=0x7f1690147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1695165640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f169015a8d0 0x7f1690147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1695165640 1 -- 172.21.15.47:0/2046790125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c009400 con 0x7f16900cb730 2024-03-23T09:41:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1694964640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16900cf190 0x7f16900b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1695165640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16900cb730 0x7f16900b4760 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e960 tx=0x7f168c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1685ffb640 1 -- 172.21.15.47:0/2046790125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c00ca00 con 0x7f16900cb730 2024-03-23T09:41:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1685ffb640 1 -- 172.21.15.47:0/2046790125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168c00cb60 con 0x7f16900cb730 2024-03-23T09:41:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f16973c9640 1 -- 172.21.15.47:0/2046790125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1680009400 con 0x7f16900cb730 2024-03-23T09:41:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.129+0000 7f1685ffb640 1 -- 172.21.15.47:0/2046790125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c013470 con 0x7f16900cb730 2024-03-23T09:41:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.130+0000 7f16973c9640 1 -- 172.21.15.47:0/2046790125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16900ca830 con 0x7f16900cb730 2024-03-23T09:41:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.131+0000 7f16973c9640 1 -- 172.21.15.47:0/2046790125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16900b4ca0 con 0x7f16900cb730 2024-03-23T09:41:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.131+0000 7f1685ffb640 1 -- 172.21.15.47:0/2046790125 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f168c0135d0 con 0x7f16900cb730 2024-03-23T09:41:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.131+0000 7f1685ffb640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f167406ac00 0x7f167406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.131+0000 7f1685ffb640 1 -- 172.21.15.47:0/2046790125 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f168c089510 con 0x7f16900cb730 2024-03-23T09:41:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.132+0000 7f1694964640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f167406ac00 0x7f167406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.132+0000 7f1694964640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f167406ac00 0x7f167406d0c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1680036000 tx=0x7f1680008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.136+0000 7f1685ffb640 1 -- 172.21.15.47:0/2046790125 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f168c054830 con 0x7f16900cb730 2024-03-23T09:41:31.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.317+0000 7f16973c9640 1 -- 172.21.15.47:0/2046790125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16900672f0 con 0x7f16900cb730 2024-03-23T09:41:31.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.318+0000 7f1685ffb640 1 -- 172.21.15.47:0/2046790125 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f168c057e50 con 0x7f16900cb730 2024-03-23T09:41:31.318 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:31.318 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:31.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.320+0000 7f16973c9640 1 -- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f167406ac00 msgr2=0x7f167406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.320+0000 7f16973c9640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f167406ac00 0x7f167406d0c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1680036000 tx=0x7f1680008030 comp rx=0 tx=0).stop 2024-03-23T09:41:31.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.320+0000 7f16973c9640 1 -- 172.21.15.47:0/2046790125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16900cb730 msgr2=0x7f16900b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.320+0000 7f16973c9640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16900cb730 0x7f16900b4760 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e960 tx=0x7f168c00ee20 comp rx=0 tx=0).stop 2024-03-23T09:41:31.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.320+0000 7f16973c9640 1 -- 172.21.15.47:0/2046790125 shutdown_connections 2024-03-23T09:41:31.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.320+0000 7f16973c9640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f167406ac00 0x7f167406d0c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.320+0000 7f16973c9640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f169015a8d0 0x7f1690147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.320+0000 7f16973c9640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16900cf190 0x7f16900b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.320+0000 7f16973c9640 1 --2- 172.21.15.47:0/2046790125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16900cb730 0x7f16900b4760 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.320+0000 7f16973c9640 1 -- 172.21.15.47:0/2046790125 >> 172.21.15.47:0/2046790125 conn(0x7f16900d0670 msgr2=0x7f1690157a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:31.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.320+0000 7f16973c9640 1 -- 172.21.15.47:0/2046790125 shutdown_connections 2024-03-23T09:41:31.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.320+0000 7f16973c9640 1 -- 172.21.15.47:0/2046790125 wait complete. 2024-03-23T09:41:31.331 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:31.331 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:31.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.435+0000 7fe2b4457640 1 Processor -- start 2024-03-23T09:41:31.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.435+0000 7fe2b4457640 1 -- start start 2024-03-23T09:41:31.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.436+0000 7fe2b4457640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2ac15c990 0x7fe2ac160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.436+0000 7fe2b4457640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe2ac0cde10 0x7fe2ac0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.436+0000 7fe2b4457640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ac161630 con 0x7fe2ac0b8350 2024-03-23T09:41:31.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.436+0000 7fe2b4457640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ac0d1030 con 0x7fe2ac15c990 2024-03-23T09:41:31.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.436+0000 7fe2b4457640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ac0cfe10 con 0x7fe2ac0cde10 2024-03-23T09:41:31.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.436+0000 7fe2b21f3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2ac15c990 0x7fe2ac160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.436+0000 7fe2b19f2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe2ac0b8350 0x7fe2ac0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34008/0 (socket says 172.21.15.47:34008) 2024-03-23T09:41:31.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.436+0000 7fe2b19f2640 1 -- 172.21.15.47:0/648269459 learned_addr learned my addr 172.21.15.47:0/648269459 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:31.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.436+0000 7fe2b29f4640 1 --2- 172.21.15.47:0/648269459 >> v2:172.21.15.175:3300/0 conn(0x7fe2ac0cde10 0x7fe2ac0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.436+0000 7fe2b21f3640 1 -- 172.21.15.47:0/648269459 >> v2:172.21.15.175:3300/0 conn(0x7fe2ac0cde10 msgr2=0x7fe2ac0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:31.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.436+0000 7fe2b21f3640 1 --2- 172.21.15.47:0/648269459 >> v2:172.21.15.175:3300/0 conn(0x7fe2ac0cde10 0x7fe2ac0d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.436+0000 7fe2b21f3640 1 -- 172.21.15.47:0/648269459 >> v1:172.21.15.47:6789/0 conn(0x7fe2ac0b8350 legacy=0x7fe2ac0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.436+0000 7fe2b21f3640 1 -- 172.21.15.47:0/648269459 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac0b5b60 con 0x7fe2ac15c990 2024-03-23T09:41:31.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.437+0000 7fe2b21f3640 1 --2- 172.21.15.47:0/648269459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2ac15c990 0x7fe2ac160690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0b87e0 tx=0x7fe29c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf065665f60dfc9e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:31.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.437+0000 7fe2b11f1640 1 -- 172.21.15.47:0/648269459 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c02cbf0 con 0x7fe2ac15c990 2024-03-23T09:41:31.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.437+0000 7fe2b11f1640 1 -- 172.21.15.47:0/648269459 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c02cd50 con 0x7fe2ac15c990 2024-03-23T09:41:31.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.437+0000 7fe2b11f1640 1 -- 172.21.15.47:0/648269459 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c034660 con 0x7fe2ac15c990 2024-03-23T09:41:31.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.438+0000 7fe2b4457640 1 -- 172.21.15.47:0/648269459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2ac15c990 msgr2=0x7fe2ac160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.438+0000 7fe2b4457640 1 --2- 172.21.15.47:0/648269459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2ac15c990 0x7fe2ac160690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0b87e0 tx=0x7fe29c02be40 comp rx=0 tx=0).stop 2024-03-23T09:41:31.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.438+0000 7fe2b4457640 1 -- 172.21.15.47:0/648269459 shutdown_connections 2024-03-23T09:41:31.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.438+0000 7fe2b4457640 1 --2- 172.21.15.47:0/648269459 >> v2:172.21.15.175:3300/0 conn(0x7fe2ac0cde10 0x7fe2ac0d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.438+0000 7fe2b4457640 1 --2- 172.21.15.47:0/648269459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2ac15c990 0x7fe2ac160690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.438+0000 7fe2b4457640 1 -- 172.21.15.47:0/648269459 >> 172.21.15.47:0/648269459 conn(0x7fe2ac0659d0 msgr2=0x7fe2ac065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:31.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.438+0000 7fe2b4457640 1 -- 172.21.15.47:0/648269459 shutdown_connections 2024-03-23T09:41:31.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.438+0000 7fe2b4457640 1 -- 172.21.15.47:0/648269459 wait complete. 2024-03-23T09:41:31.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.439+0000 7fe2b4457640 1 Processor -- start 2024-03-23T09:41:31.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.439+0000 7fe2b4457640 1 -- start start 2024-03-23T09:41:31.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b4457640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2ac0b8350 0x7fe2ac150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b4457640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2ac0cde10 0x7fe2ac1515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b4457640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2ac15c990 0x7fe2ac151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b4457640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0c7cd0 con 0x7fe2ac15c990 2024-03-23T09:41:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b4457640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0c70c0 con 0x7fe2ac0b8350 2024-03-23T09:41:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b4457640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0c73a0 con 0x7fe2ac0cde10 2024-03-23T09:41:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b29f4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2ac15c990 0x7fe2ac151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b29f4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2ac15c990 0x7fe2ac151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56568/0 (socket says 172.21.15.47:56568) 2024-03-23T09:41:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b29f4640 1 -- 172.21.15.47:0/3097614444 learned_addr learned my addr 172.21.15.47:0/3097614444 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b19f2640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2ac0cde10 0x7fe2ac1515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b21f3640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2ac0b8350 0x7fe2ac150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b29f4640 1 -- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2ac0cde10 msgr2=0x7fe2ac1515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b29f4640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2ac0cde10 0x7fe2ac1515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b29f4640 1 -- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2ac0b8350 msgr2=0x7fe2ac150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b29f4640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2ac0b8350 0x7fe2ac150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b29f4640 1 -- 172.21.15.47:0/3097614444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c009a70 con 0x7fe2ac15c990 2024-03-23T09:41:31.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.440+0000 7fe2b21f3640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2ac0b8350 0x7fe2ac150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:31.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.441+0000 7fe2b29f4640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2ac15c990 0x7fe2ac151b30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400d800 tx=0x7fe2a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:31.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.441+0000 7fe2aaffd640 1 -- 172.21.15.47:0/3097614444 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400b6c0 con 0x7fe2ac15c990 2024-03-23T09:41:31.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.441+0000 7fe2aaffd640 1 -- 172.21.15.47:0/3097614444 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a400b840 con 0x7fe2ac15c990 2024-03-23T09:41:31.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.441+0000 7fe2b4457640 1 -- 172.21.15.47:0/3097614444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ac0c7680 con 0x7fe2ac15c990 2024-03-23T09:41:31.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.441+0000 7fe2aaffd640 1 -- 172.21.15.47:0/3097614444 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400bbf0 con 0x7fe2ac15c990 2024-03-23T09:41:31.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.441+0000 7fe2b4457640 1 -- 172.21.15.47:0/3097614444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac150fc0 con 0x7fe2ac15c990 2024-03-23T09:41:31.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.442+0000 7fe2b4457640 1 -- 172.21.15.47:0/3097614444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ac072360 con 0x7fe2ac15c990 2024-03-23T09:41:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.442+0000 7fe2aaffd640 1 -- 172.21.15.47:0/3097614444 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe2a400bd50 con 0x7fe2ac15c990 2024-03-23T09:41:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.443+0000 7fe2aaffd640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe29806ac00 0x7fe29806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.443+0000 7fe2aaffd640 1 -- 172.21.15.47:0/3097614444 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2a4089800 con 0x7fe2ac15c990 2024-03-23T09:41:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.443+0000 7fe2b21f3640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe29806ac00 0x7fe29806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.443+0000 7fe2b21f3640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe29806ac00 0x7fe29806d0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe29c036000 tx=0x7fe29c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:31.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.447+0000 7fe2aaffd640 1 -- 172.21.15.47:0/3097614444 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe2a4054a70 con 0x7fe2ac15c990 2024-03-23T09:41:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.623+0000 7fe2b4457640 1 -- 172.21.15.47:0/3097614444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2ac066be0 con 0x7fe2ac15c990 2024-03-23T09:41:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.624+0000 7fe2aaffd640 1 -- 172.21.15.47:0/3097614444 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2a4058090 con 0x7fe2ac15c990 2024-03-23T09:41:31.624 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.626+0000 7fe2b4457640 1 -- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe29806ac00 msgr2=0x7fe29806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.626+0000 7fe2b4457640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe29806ac00 0x7fe29806d0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe29c036000 tx=0x7fe29c030000 comp rx=0 tx=0).stop 2024-03-23T09:41:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.626+0000 7fe2b4457640 1 -- 172.21.15.47:0/3097614444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2ac15c990 msgr2=0x7fe2ac151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.626+0000 7fe2b4457640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2ac15c990 0x7fe2ac151b30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400d800 tx=0x7fe2a400dcc0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.626+0000 7fe2b4457640 1 -- 172.21.15.47:0/3097614444 shutdown_connections 2024-03-23T09:41:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.626+0000 7fe2b4457640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe29806ac00 0x7fe29806d0c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.626+0000 7fe2b4457640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2ac15c990 0x7fe2ac151b30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.626+0000 7fe2b4457640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2ac0cde10 0x7fe2ac1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.626+0000 7fe2b4457640 1 --2- 172.21.15.47:0/3097614444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2ac0b8350 0x7fe2ac150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.626+0000 7fe2b4457640 1 -- 172.21.15.47:0/3097614444 >> 172.21.15.47:0/3097614444 conn(0x7fe2ac0659d0 msgr2=0x7fe2ac0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.626+0000 7fe2b4457640 1 -- 172.21.15.47:0/3097614444 shutdown_connections 2024-03-23T09:41:31.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.626+0000 7fe2b4457640 1 -- 172.21.15.47:0/3097614444 wait complete. 2024-03-23T09:41:31.636 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:31.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.740+0000 7f5d6b496640 1 Processor -- start 2024-03-23T09:41:31.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.740+0000 7f5d6b496640 1 -- start start 2024-03-23T09:41:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.741+0000 7f5d6b496640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5d6415c990 0x7f5d64160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.741+0000 7f5d6b496640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d640b8350 0x7f5d640cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.741+0000 7f5d6b496640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d64161630 con 0x7f5d640cdc40 2024-03-23T09:41:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.741+0000 7f5d6b496640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d640d1030 con 0x7f5d640b8350 2024-03-23T09:41:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.741+0000 7f5d6b496640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d640ce070 con 0x7f5d6415c990 2024-03-23T09:41:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.741+0000 7f5d68a31640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d640b8350 0x7f5d640cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.741+0000 7f5d68a31640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d640b8350 0x7f5d640cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56576/0 (socket says 172.21.15.47:56576) 2024-03-23T09:41:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.741+0000 7f5d68a31640 1 -- 172.21.15.47:0/3978397488 learned_addr learned my addr 172.21.15.47:0/3978397488 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.741+0000 7f5d69232640 1 --2- 172.21.15.47:0/3978397488 >> v2:172.21.15.175:3300/0 conn(0x7f5d6415c990 0x7f5d64160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.741+0000 7f5d68a31640 1 -- 172.21.15.47:0/3978397488 >> v2:172.21.15.175:3300/0 conn(0x7f5d6415c990 msgr2=0x7f5d64160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.741+0000 7f5d68a31640 1 --2- 172.21.15.47:0/3978397488 >> v2:172.21.15.175:3300/0 conn(0x7f5d6415c990 0x7f5d64160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.742+0000 7f5d68a31640 1 -- 172.21.15.47:0/3978397488 >> v1:172.21.15.175:6789/0 conn(0x7f5d640cdc40 legacy=0x7f5d640b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.742+0000 7f5d68a31640 1 -- 172.21.15.47:0/3978397488 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d640cfe10 con 0x7f5d640b8350 2024-03-23T09:41:31.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.742+0000 7f5d68a31640 1 --2- 172.21.15.47:0/3978397488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640b8350 0x7f5d640cd700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5d640b8b30 tx=0x7f5d5802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c069b08db59a6b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.742+0000 7f5d63fff640 1 -- 172.21.15.47:0/3978397488 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d580317e0 con 0x7f5d640b8350 2024-03-23T09:41:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.742+0000 7f5d63fff640 1 -- 172.21.15.47:0/3978397488 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d58031940 con 0x7f5d640b8350 2024-03-23T09:41:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.743+0000 7f5d63fff640 1 -- 172.21.15.47:0/3978397488 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58031cd0 con 0x7f5d640b8350 2024-03-23T09:41:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.743+0000 7f5d6b496640 1 -- 172.21.15.47:0/3978397488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640b8350 msgr2=0x7f5d640cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.743+0000 7f5d6b496640 1 --2- 172.21.15.47:0/3978397488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640b8350 0x7f5d640cd700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5d640b8b30 tx=0x7f5d5802b880 comp rx=0 tx=0).stop 2024-03-23T09:41:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.743+0000 7f5d6b496640 1 -- 172.21.15.47:0/3978397488 shutdown_connections 2024-03-23T09:41:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.743+0000 7f5d6b496640 1 --2- 172.21.15.47:0/3978397488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640b8350 0x7f5d640cd700 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.743+0000 7f5d6b496640 1 --2- 172.21.15.47:0/3978397488 >> v2:172.21.15.175:3300/0 conn(0x7f5d6415c990 0x7f5d64160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.743+0000 7f5d6b496640 1 -- 172.21.15.47:0/3978397488 >> 172.21.15.47:0/3978397488 conn(0x7f5d640659d0 msgr2=0x7f5d64065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.743+0000 7f5d6b496640 1 -- 172.21.15.47:0/3978397488 shutdown_connections 2024-03-23T09:41:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.743+0000 7f5d6b496640 1 -- 172.21.15.47:0/3978397488 wait complete. 2024-03-23T09:41:31.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.744+0000 7f5d6b496640 1 Processor -- start 2024-03-23T09:41:31.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.744+0000 7f5d6b496640 1 -- start start 2024-03-23T09:41:31.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.745+0000 7f5d6b496640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640b8350 0x7f5d641563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.745+0000 7f5d6b496640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d640cdc40 0x7f5d6414bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.745+0000 7f5d6b496640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d6415c990 0x7f5d640cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.745+0000 7f5d6b496640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6414c2c0 con 0x7f5d640b8350 2024-03-23T09:41:31.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.745+0000 7f5d6b496640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d64156160 con 0x7f5d6415c990 2024-03-23T09:41:31.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.745+0000 7f5d6b496640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d640caa40 con 0x7f5d640cdc40 2024-03-23T09:41:31.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.745+0000 7f5d69232640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640b8350 0x7f5d641563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.745+0000 7f5d69232640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640b8350 0x7f5d641563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56586/0 (socket says 172.21.15.47:56586) 2024-03-23T09:41:31.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.745+0000 7f5d69232640 1 -- 172.21.15.47:0/3019514495 learned_addr learned my addr 172.21.15.47:0/3019514495 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:31.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.745+0000 7f5d69a33640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d6415c990 0x7f5d640cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.745+0000 7f5d68a31640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d640cdc40 0x7f5d6414bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.746+0000 7f5d69232640 1 -- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d640cdc40 msgr2=0x7f5d6414bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.746+0000 7f5d69232640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d640cdc40 0x7f5d6414bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.746+0000 7f5d69232640 1 -- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d6415c990 msgr2=0x7f5d640cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.746+0000 7f5d69232640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d6415c990 0x7f5d640cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.746+0000 7f5d69232640 1 -- 172.21.15.47:0/3019514495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d580094d0 con 0x7f5d640b8350 2024-03-23T09:41:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.746+0000 7f5d69a33640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d6415c990 0x7f5d640cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.746+0000 7f5d69232640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640b8350 0x7f5d641563f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00d5f0 tx=0x7f5d5c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.746+0000 7f5d61ffb640 1 -- 172.21.15.47:0/3019514495 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c0125f0 con 0x7f5d640b8350 2024-03-23T09:41:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.746+0000 7f5d61ffb640 1 -- 172.21.15.47:0/3019514495 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5c012750 con 0x7f5d640b8350 2024-03-23T09:41:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.746+0000 7f5d6b496640 1 -- 172.21.15.47:0/3019514495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d640cad20 con 0x7f5d640b8350 2024-03-23T09:41:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.746+0000 7f5d61ffb640 1 -- 172.21.15.47:0/3019514495 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c012b00 con 0x7f5d640b8350 2024-03-23T09:41:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.746+0000 7f5d6b496640 1 -- 172.21.15.47:0/3019514495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d640cb2e0 con 0x7f5d640b8350 2024-03-23T09:41:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.747+0000 7f5d6b496640 1 -- 172.21.15.47:0/3019514495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d64072360 con 0x7f5d640b8350 2024-03-23T09:41:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.747+0000 7f5d61ffb640 1 -- 172.21.15.47:0/3019514495 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5d5c012c60 con 0x7f5d640b8350 2024-03-23T09:41:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.748+0000 7f5d61ffb640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5d5006ac00 0x7f5d5006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.748+0000 7f5d61ffb640 1 -- 172.21.15.47:0/3019514495 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d5c089840 con 0x7f5d640b8350 2024-03-23T09:41:31.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.748+0000 7f5d68a31640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5d5006ac00 0x7f5d5006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:31.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.748+0000 7f5d68a31640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5d5006ac00 0x7f5d5006d0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d58036000 tx=0x7f5d5803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.752+0000 7f5d61ffb640 1 -- 172.21.15.47:0/3019514495 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5d5c054a20 con 0x7f5d640b8350 2024-03-23T09:41:31.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.930+0000 7f5d6b496640 1 -- 172.21.15.47:0/3019514495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d64066be0 con 0x7f5d640b8350 2024-03-23T09:41:31.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.930+0000 7f5d61ffb640 1 -- 172.21.15.47:0/3019514495 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d5c058040 con 0x7f5d640b8350 2024-03-23T09:41:31.930 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:31.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.932+0000 7f5d6b496640 1 -- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5d5006ac00 msgr2=0x7f5d5006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.932+0000 7f5d6b496640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5d5006ac00 0x7f5d5006d0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d58036000 tx=0x7f5d5803d040 comp rx=0 tx=0).stop 2024-03-23T09:41:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.932+0000 7f5d6b496640 1 -- 172.21.15.47:0/3019514495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640b8350 msgr2=0x7f5d641563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.932+0000 7f5d6b496640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640b8350 0x7f5d641563f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00d5f0 tx=0x7f5d5c00dab0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.932+0000 7f5d6b496640 1 -- 172.21.15.47:0/3019514495 shutdown_connections 2024-03-23T09:41:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.932+0000 7f5d6b496640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5d5006ac00 0x7f5d5006d0c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.932+0000 7f5d6b496640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d6415c990 0x7f5d640cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.932+0000 7f5d6b496640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d640cdc40 0x7f5d6414bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.932+0000 7f5d6b496640 1 --2- 172.21.15.47:0/3019514495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640b8350 0x7f5d641563f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.932+0000 7f5d6b496640 1 -- 172.21.15.47:0/3019514495 >> 172.21.15.47:0/3019514495 conn(0x7f5d640659d0 msgr2=0x7f5d640cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.933+0000 7f5d6b496640 1 -- 172.21.15.47:0/3019514495 shutdown_connections 2024-03-23T09:41:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:31.933+0000 7f5d6b496640 1 -- 172.21.15.47:0/3019514495 wait complete. 2024-03-23T09:41:31.943 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.047+0000 7fc4e23d9640 1 Processor -- start 2024-03-23T09:41:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.047+0000 7fc4e23d9640 1 -- start start 2024-03-23T09:41:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.048+0000 7fc4e23d9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4dc0cde10 0x7fc4dc160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.048+0000 7fc4e23d9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4dc0b8350 0x7fc4dc0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.048+0000 7fc4e23d9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4dc161630 con 0x7fc4dc15c990 2024-03-23T09:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.048+0000 7fc4e23d9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4dc0d1030 con 0x7fc4dc0b8350 2024-03-23T09:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.048+0000 7fc4e23d9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc0cfe10 con 0x7fc4dc0cde10 2024-03-23T09:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.048+0000 7fc4e0976640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4dc0b8350 0x7fc4dc0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.048+0000 7fc4e0976640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4dc0b8350 0x7fc4dc0d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56592/0 (socket says 172.21.15.47:56592) 2024-03-23T09:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.048+0000 7fc4e0976640 1 -- 172.21.15.47:0/730136345 learned_addr learned my addr 172.21.15.47:0/730136345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.048+0000 7fc4db7fe640 1 --2- 172.21.15.47:0/730136345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4dc0cde10 0x7fc4dc160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.048+0000 7fc4e0976640 1 -- 172.21.15.47:0/730136345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4dc0cde10 msgr2=0x7fc4dc160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.048+0000 7fc4e0976640 1 --2- 172.21.15.47:0/730136345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4dc0cde10 0x7fc4dc160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.048+0000 7fc4e0976640 1 -- 172.21.15.47:0/730136345 >> v1:172.21.15.175:6789/0 conn(0x7fc4dc15c990 legacy=0x7fc4dc0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.048+0000 7fc4e0976640 1 -- 172.21.15.47:0/730136345 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc0b5b60 con 0x7fc4dc0b8350 2024-03-23T09:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.049+0000 7fc4e0976640 1 --2- 172.21.15.47:0/730136345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4dc0b8350 0x7fc4dc0d14c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009e50 tx=0x7fc4d402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d03b950ab230cf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:32.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.049+0000 7fc4daffd640 1 -- 172.21.15.47:0/730136345 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d402c950 con 0x7fc4dc0b8350 2024-03-23T09:41:32.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.049+0000 7fc4daffd640 1 -- 172.21.15.47:0/730136345 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d402cab0 con 0x7fc4dc0b8350 2024-03-23T09:41:32.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.049+0000 7fc4daffd640 1 -- 172.21.15.47:0/730136345 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d402ce40 con 0x7fc4dc0b8350 2024-03-23T09:41:32.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.049+0000 7fc4e23d9640 1 -- 172.21.15.47:0/730136345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4dc0b8350 msgr2=0x7fc4dc0d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.049+0000 7fc4e23d9640 1 --2- 172.21.15.47:0/730136345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4dc0b8350 0x7fc4dc0d14c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009e50 tx=0x7fc4d402bdb0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.049+0000 7fc4e23d9640 1 -- 172.21.15.47:0/730136345 shutdown_connections 2024-03-23T09:41:32.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.049+0000 7fc4e23d9640 1 --2- 172.21.15.47:0/730136345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4dc0b8350 0x7fc4dc0d14c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.049+0000 7fc4e23d9640 1 --2- 172.21.15.47:0/730136345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4dc0cde10 0x7fc4dc160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.049+0000 7fc4e23d9640 1 -- 172.21.15.47:0/730136345 >> 172.21.15.47:0/730136345 conn(0x7fc4dc0659d0 msgr2=0x7fc4dc065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.049+0000 7fc4e23d9640 1 -- 172.21.15.47:0/730136345 shutdown_connections 2024-03-23T09:41:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.049+0000 7fc4e23d9640 1 -- 172.21.15.47:0/730136345 wait complete. 2024-03-23T09:41:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.051+0000 7fc4e23d9640 1 Processor -- start 2024-03-23T09:41:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.051+0000 7fc4e23d9640 1 -- start start 2024-03-23T09:41:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.051+0000 7fc4e23d9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4dc0b8350 0x7fc4dc0c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.051+0000 7fc4e23d9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4dc0cde10 0x7fc4dc0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.051+0000 7fc4e23d9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4dc15c990 0x7fc4dc0c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.051+0000 7fc4e23d9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc0c89b0 con 0x7fc4dc15c990 2024-03-23T09:41:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.051+0000 7fc4e23d9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc1f2670 con 0x7fc4dc0cde10 2024-03-23T09:41:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.051+0000 7fc4e23d9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc1f2950 con 0x7fc4dc0b8350 2024-03-23T09:41:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4e0976640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4dc15c990 0x7fc4dc0c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4e0976640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4dc15c990 0x7fc4dc0c7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56602/0 (socket says 172.21.15.47:56602) 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4e0976640 1 -- 172.21.15.47:0/1500700868 learned_addr learned my addr 172.21.15.47:0/1500700868 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4db7fe640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4dc0cde10 0x7fc4dc0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4dbfff640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4dc0b8350 0x7fc4dc0c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4e0976640 1 -- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4dc0b8350 msgr2=0x7fc4dc0c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4e0976640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4dc0b8350 0x7fc4dc0c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4e0976640 1 -- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4dc0cde10 msgr2=0x7fc4dc0c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4e0976640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4dc0cde10 0x7fc4dc0c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4e0976640 1 -- 172.21.15.47:0/1500700868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d4009b30 con 0x7fc4dc15c990 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4e0976640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4dc15c990 0x7fc4dc0c7400 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009e50 tx=0x7fc4d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4d8ff9640 1 -- 172.21.15.47:0/1500700868 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4009510 con 0x7fc4dc15c990 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4d8ff9640 1 -- 172.21.15.47:0/1500700868 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d403e450 con 0x7fc4dc15c990 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4e23d9640 1 -- 172.21.15.47:0/1500700868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4dc1f2c30 con 0x7fc4dc15c990 2024-03-23T09:41:32.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4d8ff9640 1 -- 172.21.15.47:0/1500700868 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d403e7e0 con 0x7fc4dc15c990 2024-03-23T09:41:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.052+0000 7fc4e23d9640 1 -- 172.21.15.47:0/1500700868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4dc15ed00 con 0x7fc4dc15c990 2024-03-23T09:41:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.053+0000 7fc4e23d9640 1 -- 172.21.15.47:0/1500700868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4dc072360 con 0x7fc4dc15c990 2024-03-23T09:41:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.054+0000 7fc4d8ff9640 1 -- 172.21.15.47:0/1500700868 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc4d403e940 con 0x7fc4dc15c990 2024-03-23T09:41:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.054+0000 7fc4d8ff9640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4c806ac00 0x7fc4c806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.054+0000 7fc4d8ff9640 1 -- 172.21.15.47:0/1500700868 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4d40abf00 con 0x7fc4dc15c990 2024-03-23T09:41:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.054+0000 7fc4dbfff640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4c806ac00 0x7fc4c806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.055+0000 7fc4dbfff640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4c806ac00 0x7fc4c806d0c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc004f60 tx=0x7fc4cc007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:32.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.058+0000 7fc4d8ff9640 1 -- 172.21.15.47:0/1500700868 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc4d4077220 con 0x7fc4dc15c990 2024-03-23T09:41:32.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.236+0000 7fc4e23d9640 1 -- 172.21.15.47:0/1500700868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4dc066be0 con 0x7fc4dc15c990 2024-03-23T09:41:32.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.237+0000 7fc4d8ff9640 1 -- 172.21.15.47:0/1500700868 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4d407a840 con 0x7fc4dc15c990 2024-03-23T09:41:32.237 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:32.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.239+0000 7fc4e23d9640 1 -- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4c806ac00 msgr2=0x7fc4c806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.239+0000 7fc4e23d9640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4c806ac00 0x7fc4c806d0c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc004f60 tx=0x7fc4cc007e30 comp rx=0 tx=0).stop 2024-03-23T09:41:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.239+0000 7fc4e23d9640 1 -- 172.21.15.47:0/1500700868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4dc15c990 msgr2=0x7fc4dc0c7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.239+0000 7fc4e23d9640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4dc15c990 0x7fc4dc0c7400 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009e50 tx=0x7fc4d4004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.239+0000 7fc4e23d9640 1 -- 172.21.15.47:0/1500700868 shutdown_connections 2024-03-23T09:41:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.239+0000 7fc4e23d9640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4c806ac00 0x7fc4c806d0c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.239+0000 7fc4e23d9640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4dc15c990 0x7fc4dc0c7400 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.239+0000 7fc4e23d9640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4dc0cde10 0x7fc4dc0c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.239+0000 7fc4e23d9640 1 --2- 172.21.15.47:0/1500700868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4dc0b8350 0x7fc4dc0c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.239+0000 7fc4e23d9640 1 -- 172.21.15.47:0/1500700868 >> 172.21.15.47:0/1500700868 conn(0x7fc4dc0659d0 msgr2=0x7fc4dc0b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.239+0000 7fc4e23d9640 1 -- 172.21.15.47:0/1500700868 shutdown_connections 2024-03-23T09:41:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.239+0000 7fc4e23d9640 1 -- 172.21.15.47:0/1500700868 wait complete. 2024-03-23T09:41:32.273 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-23T09:41:32.274 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-03-23T09:41:32.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.375+0000 7fd2cdfa4640 1 Processor -- start 2024-03-23T09:41:32.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.375+0000 7fd2cdfa4640 1 -- start start 2024-03-23T09:41:32.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.376+0000 7fd2cdfa4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd2c815cfd0 0x7fd2c80cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.376+0000 7fd2cdfa4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c8071920 con 0x7fd2c8157150 2024-03-23T09:41:32.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.376+0000 7fd2cdfa4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c815d4d0 con 0x7fd2c8069ce0 2024-03-23T09:41:32.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.376+0000 7fd2cdfa4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c8157530 con 0x7fd2c815cfd0 2024-03-23T09:41:32.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.376+0000 7fd2c77fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd2c8157150 0x7fd2c8071c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34022/0 (socket says 172.21.15.47:34022) 2024-03-23T09:41:32.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.376+0000 7fd2c77fe640 1 -- 172.21.15.47:0/142947658 learned_addr learned my addr 172.21.15.47:0/142947658 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:32.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.376+0000 7fd2c6ffd640 1 --2- 172.21.15.47:0/142947658 >> v2:172.21.15.47:3300/0 conn(0x7fd2c815cfd0 0x7fd2c80cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.377+0000 7fd2c6ffd640 1 -- 172.21.15.47:0/142947658 >> v1:172.21.15.175:6789/0 conn(0x7fd2c8069ce0 legacy=0x7fd2c80c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:32.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.377+0000 7fd2c6ffd640 1 -- 172.21.15.47:0/142947658 >> v1:172.21.15.47:6789/0 conn(0x7fd2c8157150 legacy=0x7fd2c8071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.377+0000 7fd2c67fc640 1 -- 172.21.15.47:0/142947658 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937626179 0 0) 0x7fd2c8071920 con 0x7fd2c8157150 2024-03-23T09:41:32.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.377+0000 7fd2c6ffd640 1 -- 172.21.15.47:0/142947658 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c815d7b0 con 0x7fd2c815cfd0 2024-03-23T09:41:32.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.377+0000 7fd2c6ffd640 1 --2- 172.21.15.47:0/142947658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2c815cfd0 0x7fd2c80cd690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097a0 tx=0x7fd2bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b79f578d9de3dd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:32.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.378+0000 7fd2c67fc640 1 -- 172.21.15.47:0/142947658 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc031830 con 0x7fd2c815cfd0 2024-03-23T09:41:32.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.378+0000 7fd2c67fc640 1 -- 172.21.15.47:0/142947658 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc031990 con 0x7fd2c815cfd0 2024-03-23T09:41:32.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.378+0000 7fd2c67fc640 1 -- 172.21.15.47:0/142947658 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc031d20 con 0x7fd2c815cfd0 2024-03-23T09:41:32.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.378+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/142947658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2c815cfd0 msgr2=0x7fd2c80cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.378+0000 7fd2cdfa4640 1 --2- 172.21.15.47:0/142947658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2c815cfd0 0x7fd2c80cd690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097a0 tx=0x7fd2bc02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.378+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/142947658 shutdown_connections 2024-03-23T09:41:32.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.378+0000 7fd2cdfa4640 1 --2- 172.21.15.47:0/142947658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2c815cfd0 0x7fd2c80cd690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.378+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/142947658 >> 172.21.15.47:0/142947658 conn(0x7fd2c80b1410 msgr2=0x7fd2c80b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:32.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.378+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/142947658 shutdown_connections 2024-03-23T09:41:32.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.378+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/142947658 wait complete. 2024-03-23T09:41:32.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.379+0000 7fd2cdfa4640 1 Processor -- start 2024-03-23T09:41:32.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.379+0000 7fd2cdfa4640 1 -- start start 2024-03-23T09:41:32.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.380+0000 7fd2cdfa4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2c8069ce0 0x7fd2c81423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.380+0000 7fd2cdfa4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2c8157150 0x7fd2c8142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.380+0000 7fd2cdfa4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2c815cfd0 0x7fd2c8145490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.380+0000 7fd2cdfa4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c815a120 con 0x7fd2c8157150 2024-03-23T09:41:32.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.380+0000 7fd2cdfa4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80b3e10 con 0x7fd2c815cfd0 2024-03-23T09:41:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.380+0000 7fd2cdfa4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80b40f0 con 0x7fd2c8069ce0 2024-03-23T09:41:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.380+0000 7fd2c6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2c8157150 0x7fd2c8142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.380+0000 7fd2c6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2c8157150 0x7fd2c8142a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56624/0 (socket says 172.21.15.47:56624) 2024-03-23T09:41:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.380+0000 7fd2c6ffd640 1 -- 172.21.15.47:0/863479141 learned_addr learned my addr 172.21.15.47:0/863479141 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.381+0000 7fd2c6ffd640 1 -- 172.21.15.47:0/863479141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2c8069ce0 msgr2=0x7fd2c81423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.381+0000 7fd2c6ffd640 1 --2- 172.21.15.47:0/863479141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2c8069ce0 0x7fd2c81423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.381+0000 7fd2c6ffd640 1 -- 172.21.15.47:0/863479141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2c815cfd0 msgr2=0x7fd2c8145490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.381+0000 7fd2c6ffd640 1 --2- 172.21.15.47:0/863479141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2c815cfd0 0x7fd2c8145490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.381+0000 7fd2c6ffd640 1 -- 172.21.15.47:0/863479141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc009400 con 0x7fd2c8157150 2024-03-23T09:41:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.381+0000 7fd2c6ffd640 1 --2- 172.21.15.47:0/863479141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2c8157150 0x7fd2c8142a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0096f0 tx=0x7fd2bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.381+0000 7fd2affff640 1 -- 172.21.15.47:0/863479141 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc03ca80 con 0x7fd2c8157150 2024-03-23T09:41:32.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.381+0000 7fd2affff640 1 -- 172.21.15.47:0/863479141 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc03cbe0 con 0x7fd2c8157150 2024-03-23T09:41:32.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.381+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/863479141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c80b43d0 con 0x7fd2c8157150 2024-03-23T09:41:32.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.381+0000 7fd2affff640 1 -- 172.21.15.47:0/863479141 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc03a450 con 0x7fd2c8157150 2024-03-23T09:41:32.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.381+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/863479141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c815a400 con 0x7fd2c8157150 2024-03-23T09:41:32.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.382+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/863479141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c80b3c70 con 0x7fd2c8157150 2024-03-23T09:41:32.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.382+0000 7fd2affff640 1 -- 172.21.15.47:0/863479141 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd2bc03a5b0 con 0x7fd2c8157150 2024-03-23T09:41:32.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.383+0000 7fd2affff640 1 --2- 172.21.15.47:0/863479141 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd2b406ac00 0x7fd2b406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.383+0000 7fd2affff640 1 -- 172.21.15.47:0/863479141 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2bc0ab430 con 0x7fd2c8157150 2024-03-23T09:41:32.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.383+0000 7fd2c77fe640 1 --2- 172.21.15.47:0/863479141 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd2b406ac00 0x7fd2b406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.383+0000 7fd2c77fe640 1 --2- 172.21.15.47:0/863479141 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd2b406ac00 0x7fd2b406d0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8007fd0 tx=0x7fd2b8007960 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:32.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.387+0000 7fd2affff640 1 -- 172.21.15.47:0/863479141 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd2c80b3c70 con 0x7fd2c8157150 2024-03-23T09:41:32.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.638+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/863479141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fd2c80b3c70 con 0x7fd2c8157150 2024-03-23T09:41:32.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.841+0000 7fd2affff640 1 -- 172.21.15.47:0/863479141 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v42) v1 ==== 145+0+0 (secure 0 0 0) 0x7fd2c80b3c70 con 0x7fd2c8157150 2024-03-23T09:41:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.843+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/863479141 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd2b406ac00 msgr2=0x7fd2b406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.843+0000 7fd2cdfa4640 1 --2- 172.21.15.47:0/863479141 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd2b406ac00 0x7fd2b406d0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8007fd0 tx=0x7fd2b8007960 comp rx=0 tx=0).stop 2024-03-23T09:41:32.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.843+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/863479141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2c8157150 msgr2=0x7fd2c8142a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.843+0000 7fd2cdfa4640 1 --2- 172.21.15.47:0/863479141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2c8157150 0x7fd2c8142a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0096f0 tx=0x7fd2bc0046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.844+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/863479141 shutdown_connections 2024-03-23T09:41:32.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.844+0000 7fd2cdfa4640 1 --2- 172.21.15.47:0/863479141 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd2b406ac00 0x7fd2b406d0c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.844+0000 7fd2cdfa4640 1 --2- 172.21.15.47:0/863479141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2c815cfd0 0x7fd2c8145490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.844+0000 7fd2cdfa4640 1 --2- 172.21.15.47:0/863479141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2c8157150 0x7fd2c8142a50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.844+0000 7fd2cdfa4640 1 --2- 172.21.15.47:0/863479141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2c8069ce0 0x7fd2c81423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.844+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/863479141 >> 172.21.15.47:0/863479141 conn(0x7fd2c80b1410 msgr2=0x7fd2c80c4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:32.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.844+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/863479141 shutdown_connections 2024-03-23T09:41:32.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.844+0000 7fd2cdfa4640 1 -- 172.21.15.47:0/863479141 wait complete. 2024-03-23T09:41:32.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-23T09:41:32.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.959+0000 7fd63af69640 1 Processor -- start 2024-03-23T09:41:32.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.959+0000 7fd63af69640 1 -- start start 2024-03-23T09:41:32.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.960+0000 7fd63af69640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd6340cde10 0x7fd634160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.960+0000 7fd63af69640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6340b8350 0x7fd6340d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.960+0000 7fd63af69640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd634161630 con 0x7fd63415c990 2024-03-23T09:41:32.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.960+0000 7fd63af69640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6340d1030 con 0x7fd6340cde10 2024-03-23T09:41:32.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.960+0000 7fd63af69640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6340cfe10 con 0x7fd6340b8350 2024-03-23T09:41:32.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.960+0000 7fd638d05640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd63415c990 0x7fd6340cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34036/0 (socket says 172.21.15.47:34036) 2024-03-23T09:41:32.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.960+0000 7fd638d05640 1 -- 172.21.15.47:0/1049609128 learned_addr learned my addr 172.21.15.47:0/1049609128 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:32.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.960+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761482182 0 0) 0x7fd634161630 con 0x7fd63415c990 2024-03-23T09:41:32.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.960+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd620003520 con 0x7fd63415c990 2024-03-23T09:41:32.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.960+0000 7fd633fff640 1 --2- 172.21.15.47:0/1049609128 >> v2:172.21.15.175:3300/0 conn(0x7fd6340cde10 0x7fd634160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.960+0000 7fd639506640 1 --2- 172.21.15.47:0/1049609128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6340b8350 0x7fd6340d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.960+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1894569709 0 0) 0x7fd620003520 con 0x7fd63415c990 2024-03-23T09:41:32.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6340b8350 msgr2=0x7fd6340d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:32.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd6337fe640 1 --2- 172.21.15.47:0/1049609128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6340b8350 0x7fd6340d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 >> v2:172.21.15.175:3300/0 conn(0x7fd6340cde10 msgr2=0x7fd634160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:32.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd6337fe640 1 --2- 172.21.15.47:0/1049609128 >> v2:172.21.15.175:3300/0 conn(0x7fd6340cde10 0x7fd634160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6340b5b60 con 0x7fd63415c990 2024-03-23T09:41:32.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fd62c001670 con 0x7fd63415c990 2024-03-23T09:41:32.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 >> v1:172.21.15.47:6789/0 conn(0x7fd63415c990 legacy=0x7fd6340cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd6337fe640 1 --2- 172.21.15.47:0/1049609128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd620003970 0x7fd620003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd6337fe640 1 --2- 172.21.15.47:0/1049609128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6340b8350 0x7fd6200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd6337fe640 1 --2- 172.21.15.47:0/1049609128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd620004d00 0x7fd620005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6340d1030 con 0x7fd620003970 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6340cfe10 con 0x7fd620004d00 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62000ba60 con 0x7fd6340b8350 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd638d05640 1 --2- 172.21.15.47:0/1049609128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd620003970 0x7fd620003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd633fff640 1 --2- 172.21.15.47:0/1049609128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6340b8350 0x7fd6200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd639506640 1 --2- 172.21.15.47:0/1049609128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd620004d00 0x7fd620005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd638d05640 1 -- 172.21.15.47:0/1049609128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6340b8350 msgr2=0x7fd6200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd638d05640 1 --2- 172.21.15.47:0/1049609128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6340b8350 0x7fd6200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd638d05640 1 -- 172.21.15.47:0/1049609128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd620004d00 msgr2=0x7fd620005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd638d05640 1 --2- 172.21.15.47:0/1049609128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd620004d00 0x7fd620005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd638d05640 1 -- 172.21.15.47:0/1049609128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd620003520 con 0x7fd620003970 2024-03-23T09:41:32.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.961+0000 7fd639506640 1 --2- 172.21.15.47:0/1049609128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd620004d00 0x7fd620005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:32.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.962+0000 7fd638d05640 1 --2- 172.21.15.47:0/1049609128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd620003970 0x7fd620003d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00c0e0 tx=0x7fd6200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e6d09c9378b7c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:32.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.962+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c005ad0 con 0x7fd620003970 2024-03-23T09:41:32.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.962+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c005c30 con 0x7fd620003970 2024-03-23T09:41:32.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.962+0000 7fd6337fe640 1 -- 172.21.15.47:0/1049609128 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c02c810 con 0x7fd620003970 2024-03-23T09:41:32.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.962+0000 7fd63af69640 1 -- 172.21.15.47:0/1049609128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd620003970 msgr2=0x7fd620003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.962+0000 7fd63af69640 1 --2- 172.21.15.47:0/1049609128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd620003970 0x7fd620003d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00c0e0 tx=0x7fd6200057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.962+0000 7fd639506640 1 -- 172.21.15.47:0/1049609128 reap_dead start 2024-03-23T09:41:32.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.962+0000 7fd63af69640 1 -- 172.21.15.47:0/1049609128 shutdown_connections 2024-03-23T09:41:32.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.962+0000 7fd63af69640 1 -- 172.21.15.47:0/1049609128 >> 172.21.15.47:0/1049609128 conn(0x7fd6340659d0 msgr2=0x7fd634065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:32.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.962+0000 7fd63af69640 1 -- 172.21.15.47:0/1049609128 shutdown_connections 2024-03-23T09:41:32.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.962+0000 7fd63af69640 1 -- 172.21.15.47:0/1049609128 wait complete. 2024-03-23T09:41:32.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.964+0000 7fd63af69640 1 Processor -- start 2024-03-23T09:41:32.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.964+0000 7fd63af69640 1 -- start start 2024-03-23T09:41:32.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.964+0000 7fd63af69640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd620003970 0x7fd634155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.964+0000 7fd63af69640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd634158fb0 0x7fd6340b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.964+0000 7fd63af69640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6340c8d60 0x7fd6340b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.964+0000 7fd63af69640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63415ec60 con 0x7fd634158fb0 2024-03-23T09:41:32.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.964+0000 7fd63af69640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6341f2190 con 0x7fd6340c8d60 2024-03-23T09:41:32.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.964+0000 7fd63af69640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6341594e0 con 0x7fd620003970 2024-03-23T09:41:32.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd633fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd634158fb0 0x7fd6340b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd633fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd634158fb0 0x7fd6340b6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56656/0 (socket says 172.21.15.47:56656) 2024-03-23T09:41:32.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd633fff640 1 -- 172.21.15.47:0/1734427935 learned_addr learned my addr 172.21.15.47:0/1734427935 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:32.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd639506640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6340c8d60 0x7fd6340b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd638d05640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd620003970 0x7fd634155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd633fff640 1 -- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd620003970 msgr2=0x7fd634155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd633fff640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd620003970 0x7fd634155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd633fff640 1 -- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6340c8d60 msgr2=0x7fd6340b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd633fff640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6340c8d60 0x7fd6340b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd633fff640 1 -- 172.21.15.47:0/1734427935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6341597c0 con 0x7fd634158fb0 2024-03-23T09:41:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd633fff640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd634158fb0 0x7fd6340b6150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd624011e30 tx=0x7fd62400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd6317fa640 1 -- 172.21.15.47:0/1734427935 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62400e040 con 0x7fd634158fb0 2024-03-23T09:41:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd6317fa640 1 -- 172.21.15.47:0/1734427935 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62400f040 con 0x7fd634158fb0 2024-03-23T09:41:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd63af69640 1 -- 172.21.15.47:0/1734427935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63415e8f0 con 0x7fd634158fb0 2024-03-23T09:41:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd6317fa640 1 -- 172.21.15.47:0/1734427935 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624009c00 con 0x7fd634158fb0 2024-03-23T09:41:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.965+0000 7fd63af69640 1 -- 172.21.15.47:0/1734427935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd634158790 con 0x7fd634158fb0 2024-03-23T09:41:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.966+0000 7fd63af69640 1 -- 172.21.15.47:0/1734427935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd634072360 con 0x7fd634158fb0 2024-03-23T09:41:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.966+0000 7fd6317fa640 1 -- 172.21.15.47:0/1734427935 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd624015850 con 0x7fd634158fb0 2024-03-23T09:41:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.967+0000 7fd6317fa640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd61406ac30 0x7fd61406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.967+0000 7fd6317fa640 1 -- 172.21.15.47:0/1734427935 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd62408a090 con 0x7fd634158fb0 2024-03-23T09:41:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.967+0000 7fd638d05640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd61406ac30 0x7fd61406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.968+0000 7fd638d05640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd61406ac30 0x7fd61406d0f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd62c034000 tx=0x7fd62c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:32.971+0000 7fd6317fa640 1 -- 172.21.15.47:0/1734427935 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd624055270 con 0x7fd634158fb0 2024-03-23T09:41:33.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.149+0000 7fd63af69640 1 -- 172.21.15.47:0/1734427935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd634066be0 con 0x7fd634158fb0 2024-03-23T09:41:33.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.153+0000 7fd6317fa640 1 -- 172.21.15.47:0/1734427935 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd624058890 con 0x7fd634158fb0 2024-03-23T09:41:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.154+0000 7fd63af69640 1 -- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd61406ac30 msgr2=0x7fd61406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.154+0000 7fd63af69640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd61406ac30 0x7fd61406d0f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd62c034000 tx=0x7fd62c007000 comp rx=0 tx=0).stop 2024-03-23T09:41:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.154+0000 7fd63af69640 1 -- 172.21.15.47:0/1734427935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd634158fb0 msgr2=0x7fd6340b6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.154+0000 7fd63af69640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd634158fb0 0x7fd6340b6150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd624011e30 tx=0x7fd62400f580 comp rx=0 tx=0).stop 2024-03-23T09:41:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.155+0000 7fd63af69640 1 -- 172.21.15.47:0/1734427935 shutdown_connections 2024-03-23T09:41:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.155+0000 7fd63af69640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd61406ac30 0x7fd61406d0f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.155+0000 7fd63af69640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6340c8d60 0x7fd6340b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.155+0000 7fd63af69640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd634158fb0 0x7fd6340b6150 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.155+0000 7fd63af69640 1 --2- 172.21.15.47:0/1734427935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd620003970 0x7fd634155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.155+0000 7fd63af69640 1 -- 172.21.15.47:0/1734427935 >> 172.21.15.47:0/1734427935 conn(0x7fd6340659d0 msgr2=0x7fd6340b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.155+0000 7fd63af69640 1 -- 172.21.15.47:0/1734427935 shutdown_connections 2024-03-23T09:41:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.155+0000 7fd63af69640 1 -- 172.21.15.47:0/1734427935 wait complete. 2024-03-23T09:41:33.165 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:41:33.165 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:41:33.208 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:41:33.262 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:41:33.262 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:41:33.262 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:41:33.278 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:41:33.278 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:41:33.332 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.473+0000 7fd6dc973640 1 Processor -- start 2024-03-23T09:41:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.473+0000 7fd6dc973640 1 -- start start 2024-03-23T09:41:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.474+0000 7fd6dc973640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd6d815ca90 0x7fd6d8160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.474+0000 7fd6dc973640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d80b8350 0x7fd6d80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.474+0000 7fd6dc973640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d80d11e0 con 0x7fd6d80cdc40 2024-03-23T09:41:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.474+0000 7fd6dc973640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d80ce0d0 con 0x7fd6d815ca90 2024-03-23T09:41:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.474+0000 7fd6dc973640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80d2680 con 0x7fd6d80b8350 2024-03-23T09:41:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.475+0000 7fd6d659c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd6d815ca90 0x7fd6d8160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.475+0000 7fd6d5d9b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d80b8350 0x7fd6d80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.475+0000 7fd6d659c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd6d815ca90 0x7fd6d8160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:33556/0 (socket says 172.21.15.47:33556) 2024-03-23T09:41:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.475+0000 7fd6d659c640 1 -- 172.21.15.47:0/2232175720 learned_addr learned my addr 172.21.15.47:0/2232175720 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.475+0000 7fd6d5d9b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d80b8350 0x7fd6d80cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:59402/0 (socket says 172.21.15.47:59402) 2024-03-23T09:41:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.475+0000 7fd6d6d9d640 1 --1- 172.21.15.47:0/2232175720 >> v1:172.21.15.175:6789/0 conn(0x7fd6d80cdc40 0x7fd6d80b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:44430/0 (socket says 172.21.15.47:44430) 2024-03-23T09:41:33.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.475+0000 7fd6d5d9b640 1 -- 172.21.15.47:0/2232175720 >> v2:172.21.15.175:3300/0 conn(0x7fd6d815ca90 msgr2=0x7fd6d8160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:33.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.475+0000 7fd6d5d9b640 1 --2- 172.21.15.47:0/2232175720 >> v2:172.21.15.175:3300/0 conn(0x7fd6d815ca90 0x7fd6d8160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.475+0000 7fd6d5d9b640 1 -- 172.21.15.47:0/2232175720 >> v1:172.21.15.175:6789/0 conn(0x7fd6d80cdc40 legacy=0x7fd6d80b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:33.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.475+0000 7fd6d5d9b640 1 -- 172.21.15.47:0/2232175720 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d8160f60 con 0x7fd6d80b8350 2024-03-23T09:41:33.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.475+0000 7fd6d659c640 1 --2- 172.21.15.47:0/2232175720 >> v2:172.21.15.175:3300/0 conn(0x7fd6d815ca90 0x7fd6d8160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:33.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.476+0000 7fd6d5d9b640 1 --2- 172.21.15.47:0/2232175720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d80b8350 0x7fd6d80cd700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00097a0 tx=0x7fd6c002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb5107d17c703a92 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:33.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.476+0000 7fd6d559a640 1 -- 172.21.15.47:0/2232175720 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c002e070 con 0x7fd6d80b8350 2024-03-23T09:41:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.476+0000 7fd6d559a640 1 -- 172.21.15.47:0/2232175720 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c00315a0 con 0x7fd6d80b8350 2024-03-23T09:41:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.476+0000 7fd6d559a640 1 -- 172.21.15.47:0/2232175720 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0031930 con 0x7fd6d80b8350 2024-03-23T09:41:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.477+0000 7fd6dc973640 1 -- 172.21.15.47:0/2232175720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d80b8350 msgr2=0x7fd6d80cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.477+0000 7fd6dc973640 1 --2- 172.21.15.47:0/2232175720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d80b8350 0x7fd6d80cd700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00097a0 tx=0x7fd6c002b840 comp rx=0 tx=0).stop 2024-03-23T09:41:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.477+0000 7fd6dc973640 1 -- 172.21.15.47:0/2232175720 shutdown_connections 2024-03-23T09:41:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.477+0000 7fd6dc973640 1 --2- 172.21.15.47:0/2232175720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d80b8350 0x7fd6d80cd700 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.477+0000 7fd6dc973640 1 --2- 172.21.15.47:0/2232175720 >> v2:172.21.15.175:3300/0 conn(0x7fd6d815ca90 0x7fd6d8160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.477+0000 7fd6dc973640 1 -- 172.21.15.47:0/2232175720 >> 172.21.15.47:0/2232175720 conn(0x7fd6d80659d0 msgr2=0x7fd6d8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.477+0000 7fd6dc973640 1 -- 172.21.15.47:0/2232175720 shutdown_connections 2024-03-23T09:41:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.477+0000 7fd6dc973640 1 -- 172.21.15.47:0/2232175720 wait complete. 2024-03-23T09:41:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.478+0000 7fd6dc973640 1 Processor -- start 2024-03-23T09:41:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.478+0000 7fd6dc973640 1 -- start start 2024-03-23T09:41:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6dc973640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6d80b8350 0x7fd6d815e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6dc973640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d80cdc40 0x7fd6d80c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6dc973640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d815ca90 0x7fd6d80cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6dc973640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80c91e0 con 0x7fd6d80cdc40 2024-03-23T09:41:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6dc973640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d815afa0 con 0x7fd6d80b8350 2024-03-23T09:41:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6dc973640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d815dca0 con 0x7fd6d815ca90 2024-03-23T09:41:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6d5d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d80cdc40 0x7fd6d80c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6d5d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d80cdc40 0x7fd6d80c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56662/0 (socket says 172.21.15.47:56662) 2024-03-23T09:41:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6d5d9b640 1 -- 172.21.15.47:0/3329624121 learned_addr learned my addr 172.21.15.47:0/3329624121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6d6d9d640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d815ca90 0x7fd6d80cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6d659c640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6d80b8350 0x7fd6d815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6d5d9b640 1 -- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d815ca90 msgr2=0x7fd6d80cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6d5d9b640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d815ca90 0x7fd6d80cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6d5d9b640 1 -- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6d80b8350 msgr2=0x7fd6d815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6d5d9b640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6d80b8350 0x7fd6d815e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6d5d9b640 1 -- 172.21.15.47:0/3329624121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d00094d0 con 0x7fd6d80cdc40 2024-03-23T09:41:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.479+0000 7fd6d5d9b640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d80cdc40 0x7fd6d80c8ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0034040 tx=0x7fd6c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.480+0000 7fd6ceffd640 1 -- 172.21.15.47:0/3329624121 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0032630 con 0x7fd6d80cdc40 2024-03-23T09:41:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.480+0000 7fd6ceffd640 1 -- 172.21.15.47:0/3329624121 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c0032790 con 0x7fd6d80cdc40 2024-03-23T09:41:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.480+0000 7fd6dc973640 1 -- 172.21.15.47:0/3329624121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c0009400 con 0x7fd6d80cdc40 2024-03-23T09:41:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.480+0000 7fd6ceffd640 1 -- 172.21.15.47:0/3329624121 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0032b40 con 0x7fd6d80cdc40 2024-03-23T09:41:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.480+0000 7fd6dc973640 1 -- 172.21.15.47:0/3329624121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d815e260 con 0x7fd6d80cdc40 2024-03-23T09:41:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.481+0000 7fd6dc973640 1 -- 172.21.15.47:0/3329624121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d8072360 con 0x7fd6d80cdc40 2024-03-23T09:41:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.481+0000 7fd6ceffd640 1 -- 172.21.15.47:0/3329624121 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd6c0032ca0 con 0x7fd6d80cdc40 2024-03-23T09:41:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.481+0000 7fd6ceffd640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd6b806ac00 0x7fd6b806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.481+0000 7fd6ceffd640 1 -- 172.21.15.47:0/3329624121 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6c00ab0c0 con 0x7fd6d80cdc40 2024-03-23T09:41:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.482+0000 7fd6d659c640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd6b806ac00 0x7fd6b806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.482+0000 7fd6d659c640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd6b806ac00 0x7fd6b806d0c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0006ca0 tx=0x7fd6d0009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:33.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.486+0000 7fd6ceffd640 1 -- 172.21.15.47:0/3329624121 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd6c00763e0 con 0x7fd6d80cdc40 2024-03-23T09:41:33.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.668+0000 7fd6dc973640 1 -- 172.21.15.47:0/3329624121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6d8066be0 con 0x7fd6d80cdc40 2024-03-23T09:41:33.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.669+0000 7fd6ceffd640 1 -- 172.21.15.47:0/3329624121 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fd6c0079a00 con 0x7fd6d80cdc40 2024-03-23T09:41:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:33.669 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:33.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.671+0000 7fd6dc973640 1 -- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd6b806ac00 msgr2=0x7fd6b806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:33.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.671+0000 7fd6dc973640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd6b806ac00 0x7fd6b806d0c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0006ca0 tx=0x7fd6d0009240 comp rx=0 tx=0).stop 2024-03-23T09:41:33.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.671+0000 7fd6dc973640 1 -- 172.21.15.47:0/3329624121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d80cdc40 msgr2=0x7fd6d80c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:33.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.671+0000 7fd6dc973640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d80cdc40 0x7fd6d80c8ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0034040 tx=0x7fd6c00046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.671+0000 7fd6dc973640 1 -- 172.21.15.47:0/3329624121 shutdown_connections 2024-03-23T09:41:33.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.671+0000 7fd6dc973640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd6b806ac00 0x7fd6b806d0c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.671+0000 7fd6dc973640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d815ca90 0x7fd6d80cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.671+0000 7fd6dc973640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d80cdc40 0x7fd6d80c8ca0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.671+0000 7fd6dc973640 1 --2- 172.21.15.47:0/3329624121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6d80b8350 0x7fd6d815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.671+0000 7fd6dc973640 1 -- 172.21.15.47:0/3329624121 >> 172.21.15.47:0/3329624121 conn(0x7fd6d80659d0 msgr2=0x7fd6d80b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:33.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.671+0000 7fd6dc973640 1 -- 172.21.15.47:0/3329624121 shutdown_connections 2024-03-23T09:41:33.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.671+0000 7fd6dc973640 1 -- 172.21.15.47:0/3329624121 wait complete. 2024-03-23T09:41:33.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:33.682 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:33.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.786+0000 7f27a661e640 1 Processor -- start 2024-03-23T09:41:33.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.786+0000 7f27a661e640 1 -- start start 2024-03-23T09:41:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f27a661e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27a015c990 0x7f27a0160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f27a661e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00b8350 0x7f27a00cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f27a661e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f27a00cdc40 0x7f27a00b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f27a661e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a00ce0b0 con 0x7f27a015c990 2024-03-23T09:41:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f27a661e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a0161630 con 0x7f27a00cdc40 2024-03-23T09:41:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f27a661e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a00d1030 con 0x7f27a00b8350 2024-03-23T09:41:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f279ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27a015c990 0x7f27a0160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f279ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27a015c990 0x7f27a0160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56668/0 (socket says 172.21.15.47:56668) 2024-03-23T09:41:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f279ffff640 1 -- 172.21.15.47:0/492389530 learned_addr learned my addr 172.21.15.47:0/492389530 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f27a4bbb640 1 --2- 172.21.15.47:0/492389530 >> v2:172.21.15.175:3300/0 conn(0x7f27a00cdc40 0x7f27a00b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f279f7fe640 1 --2- 172.21.15.47:0/492389530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00b8350 0x7f27a00cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f279ffff640 1 -- 172.21.15.47:0/492389530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00b8350 msgr2=0x7f27a00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f279ffff640 1 --2- 172.21.15.47:0/492389530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00b8350 0x7f27a00cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f279ffff640 1 -- 172.21.15.47:0/492389530 >> v2:172.21.15.175:3300/0 conn(0x7f27a00cdc40 msgr2=0x7f27a00b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f279ffff640 1 --2- 172.21.15.47:0/492389530 >> v2:172.21.15.175:3300/0 conn(0x7f27a00cdc40 0x7f27a00b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f279ffff640 1 -- 172.21.15.47:0/492389530 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a00cfe10 con 0x7f27a015c990 2024-03-23T09:41:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f27a4bbb640 1 --2- 172.21.15.47:0/492389530 >> v2:172.21.15.175:3300/0 conn(0x7f27a00cdc40 0x7f27a00b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.787+0000 7f279f7fe640 1 --2- 172.21.15.47:0/492389530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00b8350 0x7f27a00cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:33.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.788+0000 7f279ffff640 1 --2- 172.21.15.47:0/492389530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27a015c990 0x7f27a0160690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f27a00b88c0 tx=0x7f279002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec95b1a1fd2f03a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:33.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.788+0000 7f279effd640 1 -- 172.21.15.47:0/492389530 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27900317e0 con 0x7f27a015c990 2024-03-23T09:41:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.788+0000 7f279effd640 1 -- 172.21.15.47:0/492389530 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2790031940 con 0x7f27a015c990 2024-03-23T09:41:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.788+0000 7f279effd640 1 -- 172.21.15.47:0/492389530 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790031cd0 con 0x7f27a015c990 2024-03-23T09:41:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.789+0000 7f27a661e640 1 -- 172.21.15.47:0/492389530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27a015c990 msgr2=0x7f27a0160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.789+0000 7f27a661e640 1 --2- 172.21.15.47:0/492389530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27a015c990 0x7f27a0160690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f27a00b88c0 tx=0x7f279002b880 comp rx=0 tx=0).stop 2024-03-23T09:41:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.789+0000 7f27a661e640 1 -- 172.21.15.47:0/492389530 shutdown_connections 2024-03-23T09:41:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.789+0000 7f27a661e640 1 --2- 172.21.15.47:0/492389530 >> v2:172.21.15.175:3300/0 conn(0x7f27a00cdc40 0x7f27a00b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.789+0000 7f27a661e640 1 --2- 172.21.15.47:0/492389530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00b8350 0x7f27a00cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.789+0000 7f27a661e640 1 --2- 172.21.15.47:0/492389530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27a015c990 0x7f27a0160690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.789+0000 7f27a661e640 1 -- 172.21.15.47:0/492389530 >> 172.21.15.47:0/492389530 conn(0x7f27a00659d0 msgr2=0x7f27a0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.789+0000 7f27a661e640 1 -- 172.21.15.47:0/492389530 shutdown_connections 2024-03-23T09:41:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.789+0000 7f27a661e640 1 -- 172.21.15.47:0/492389530 wait complete. 2024-03-23T09:41:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.790+0000 7f27a661e640 1 Processor -- start 2024-03-23T09:41:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.790+0000 7f27a661e640 1 -- start start 2024-03-23T09:41:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f27a661e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27a00b8350 0x7f27a01f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f27a661e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27a00cdc40 0x7f27a015bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f27a661e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a015c990 0x7f27a00c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f27a661e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a014d5b0 con 0x7f27a00cdc40 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f27a661e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a01f3e40 con 0x7f27a00b8350 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f27a661e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a01504b0 con 0x7f27a015c990 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f279f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27a00cdc40 0x7f27a015bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f279f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27a00cdc40 0x7f27a015bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56672/0 (socket says 172.21.15.47:56672) 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f279f7fe640 1 -- 172.21.15.47:0/2039895445 learned_addr learned my addr 172.21.15.47:0/2039895445 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f279f7fe640 1 -- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a015c990 msgr2=0x7f27a00c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f279f7fe640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a015c990 0x7f27a00c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f279f7fe640 1 -- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27a00b8350 msgr2=0x7f27a01f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f279ffff640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27a00b8350 0x7f27a01f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f279f7fe640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27a00b8350 0x7f27a01f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f279f7fe640 1 -- 172.21.15.47:0/2039895445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27900094d0 con 0x7f27a00cdc40 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.791+0000 7f279ffff640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27a00b8350 0x7f27a01f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.792+0000 7f279f7fe640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27a00cdc40 0x7f27a015bbb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f278800d830 tx=0x7f278800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.792+0000 7f279cff9640 1 -- 172.21.15.47:0/2039895445 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278800be10 con 0x7f27a00cdc40 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.792+0000 7f279cff9640 1 -- 172.21.15.47:0/2039895445 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2788004030 con 0x7f27a00cdc40 2024-03-23T09:41:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.792+0000 7f27a661e640 1 -- 172.21.15.47:0/2039895445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a00c6430 con 0x7f27a00cdc40 2024-03-23T09:41:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.792+0000 7f279cff9640 1 -- 172.21.15.47:0/2039895445 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27880043c0 con 0x7f27a00cdc40 2024-03-23T09:41:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.792+0000 7f27a661e640 1 -- 172.21.15.47:0/2039895445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a00c67c0 con 0x7f27a00cdc40 2024-03-23T09:41:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.793+0000 7f27a661e640 1 -- 172.21.15.47:0/2039895445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a0072360 con 0x7f27a00cdc40 2024-03-23T09:41:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.793+0000 7f279cff9640 1 -- 172.21.15.47:0/2039895445 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f27880098b0 con 0x7f27a00cdc40 2024-03-23T09:41:33.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.794+0000 7f279cff9640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f278006ac00 0x7f278006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:33.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.794+0000 7f279cff9640 1 -- 172.21.15.47:0/2039895445 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2788089530 con 0x7f27a00cdc40 2024-03-23T09:41:33.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.794+0000 7f279ffff640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f278006ac00 0x7f278006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:33.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.794+0000 7f279ffff640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f278006ac00 0x7f278006d0c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2790036000 tx=0x7f279003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.798+0000 7f279cff9640 1 -- 172.21.15.47:0/2039895445 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2788054710 con 0x7f27a00cdc40 2024-03-23T09:41:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.977+0000 7f27a661e640 1 -- 172.21.15.47:0/2039895445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27a00c65c0 con 0x7f27a00cdc40 2024-03-23T09:41:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.978+0000 7f279cff9640 1 -- 172.21.15.47:0/2039895445 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f278800b3a0 con 0x7f27a00cdc40 2024-03-23T09:41:33.977 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:33.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.980+0000 7f27a661e640 1 -- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f278006ac00 msgr2=0x7f278006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.980+0000 7f27a661e640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f278006ac00 0x7f278006d0c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2790036000 tx=0x7f279003d040 comp rx=0 tx=0).stop 2024-03-23T09:41:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.980+0000 7f27a661e640 1 -- 172.21.15.47:0/2039895445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27a00cdc40 msgr2=0x7f27a015bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.980+0000 7f27a661e640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27a00cdc40 0x7f27a015bbb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f278800d830 tx=0x7f278800dcf0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.980+0000 7f27a661e640 1 -- 172.21.15.47:0/2039895445 shutdown_connections 2024-03-23T09:41:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.980+0000 7f27a661e640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f278006ac00 0x7f278006d0c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.980+0000 7f27a661e640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a015c990 0x7f27a00c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.980+0000 7f27a661e640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27a00cdc40 0x7f27a015bbb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.980+0000 7f27a661e640 1 --2- 172.21.15.47:0/2039895445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27a00b8350 0x7f27a01f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.980+0000 7f27a661e640 1 -- 172.21.15.47:0/2039895445 >> 172.21.15.47:0/2039895445 conn(0x7f27a00659d0 msgr2=0x7f27a00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.980+0000 7f27a661e640 1 -- 172.21.15.47:0/2039895445 shutdown_connections 2024-03-23T09:41:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:33.980+0000 7f27a661e640 1 -- 172.21.15.47:0/2039895445 wait complete. 2024-03-23T09:41:33.990 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:34.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.094+0000 7f9697a41640 1 Processor -- start 2024-03-23T09:41:34.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.094+0000 7f9697a41640 1 -- start start 2024-03-23T09:41:34.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9697a41640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969015c990 0x7f9690160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9697a41640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f96900b8350 0x7f96900cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9697a41640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f96900cdc40 0x7f96900b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9697a41640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96900ce0b0 con 0x7f96900cdc40 2024-03-23T09:41:34.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9697a41640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9690161630 con 0x7f96900b8350 2024-03-23T09:41:34.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9697a41640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96900d1030 con 0x7f969015c990 2024-03-23T09:41:34.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9695fde640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f96900cdc40 0x7f96900b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9695fde640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f96900cdc40 0x7f96900b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56678/0 (socket says 172.21.15.47:56678) 2024-03-23T09:41:34.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9695fde640 1 -- 172.21.15.47:0/1614580490 learned_addr learned my addr 172.21.15.47:0/1614580490 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:34.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9694fdc640 1 --2- 172.21.15.47:0/1614580490 >> v2:172.21.15.175:3300/0 conn(0x7f96900b8350 0x7f96900cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f96957dd640 1 --2- 172.21.15.47:0/1614580490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969015c990 0x7f9690160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9695fde640 1 -- 172.21.15.47:0/1614580490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969015c990 msgr2=0x7f9690160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:34.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9695fde640 1 --2- 172.21.15.47:0/1614580490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969015c990 0x7f9690160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9695fde640 1 -- 172.21.15.47:0/1614580490 >> v2:172.21.15.175:3300/0 conn(0x7f96900b8350 msgr2=0x7f96900cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:34.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9695fde640 1 --2- 172.21.15.47:0/1614580490 >> v2:172.21.15.175:3300/0 conn(0x7f96900b8350 0x7f96900cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9695fde640 1 -- 172.21.15.47:0/1614580490 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96900cfe10 con 0x7f96900cdc40 2024-03-23T09:41:34.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f9694fdc640 1 --2- 172.21.15.47:0/1614580490 >> v2:172.21.15.175:3300/0 conn(0x7f96900b8350 0x7f96900cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:34.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.095+0000 7f96957dd640 1 --2- 172.21.15.47:0/1614580490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969015c990 0x7f9690160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:34.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.096+0000 7f9695fde640 1 --2- 172.21.15.47:0/1614580490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96900cdc40 0x7f96900b4440 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f96900b5540 tx=0x7f968802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8da3f0b9d83648 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:34.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.096+0000 7f9687fff640 1 -- 172.21.15.47:0/1614580490 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968802c9e0 con 0x7f96900cdc40 2024-03-23T09:41:34.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.096+0000 7f9687fff640 1 -- 172.21.15.47:0/1614580490 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968802cb40 con 0x7f96900cdc40 2024-03-23T09:41:34.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.096+0000 7f9687fff640 1 -- 172.21.15.47:0/1614580490 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96880343f0 con 0x7f96900cdc40 2024-03-23T09:41:34.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.097+0000 7f9697a41640 1 -- 172.21.15.47:0/1614580490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96900cdc40 msgr2=0x7f96900b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.097+0000 7f9697a41640 1 --2- 172.21.15.47:0/1614580490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96900cdc40 0x7f96900b4440 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f96900b5540 tx=0x7f968802be40 comp rx=0 tx=0).stop 2024-03-23T09:41:34.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.097+0000 7f9697a41640 1 -- 172.21.15.47:0/1614580490 shutdown_connections 2024-03-23T09:41:34.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.097+0000 7f9697a41640 1 --2- 172.21.15.47:0/1614580490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96900cdc40 0x7f96900b4440 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.097+0000 7f9697a41640 1 --2- 172.21.15.47:0/1614580490 >> v2:172.21.15.175:3300/0 conn(0x7f96900b8350 0x7f96900cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.097+0000 7f9697a41640 1 --2- 172.21.15.47:0/1614580490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969015c990 0x7f9690160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.097+0000 7f9697a41640 1 -- 172.21.15.47:0/1614580490 >> 172.21.15.47:0/1614580490 conn(0x7f96900659d0 msgr2=0x7f9690065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:34.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.097+0000 7f9697a41640 1 -- 172.21.15.47:0/1614580490 shutdown_connections 2024-03-23T09:41:34.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.097+0000 7f9697a41640 1 -- 172.21.15.47:0/1614580490 wait complete. 2024-03-23T09:41:34.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.098+0000 7f9697a41640 1 Processor -- start 2024-03-23T09:41:34.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.098+0000 7f9697a41640 1 -- start start 2024-03-23T09:41:34.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9697a41640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96900b8350 0x7f969015dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9697a41640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96900cdc40 0x7f9690155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9697a41640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f969015c990 0x7f96900b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9697a41640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900c9340 con 0x7f96900b8350 2024-03-23T09:41:34.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9697a41640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969015cd70 con 0x7f969015c990 2024-03-23T09:41:34.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9697a41640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969015d8f0 con 0x7f96900cdc40 2024-03-23T09:41:34.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f96957dd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96900b8350 0x7f969015dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f96957dd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96900b8350 0x7f969015dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56688/0 (socket says 172.21.15.47:56688) 2024-03-23T09:41:34.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f96957dd640 1 -- 172.21.15.47:0/1718440913 learned_addr learned my addr 172.21.15.47:0/1718440913 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:34.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9695fde640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f969015c990 0x7f96900b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9694fdc640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96900cdc40 0x7f9690155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f96957dd640 1 -- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96900cdc40 msgr2=0x7f9690155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f96957dd640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96900cdc40 0x7f9690155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f96957dd640 1 -- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f969015c990 msgr2=0x7f96900b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f96957dd640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f969015c990 0x7f96900b7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f96957dd640 1 -- 172.21.15.47:0/1718440913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9688009b30 con 0x7f96900b8350 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9695fde640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f969015c990 0x7f96900b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9694fdc640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96900cdc40 0x7f9690155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f96957dd640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96900b8350 0x7f969015dd80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c7f0 tx=0x7f968c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9685ffb640 1 -- 172.21.15.47:0/1718440913 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c007a30 con 0x7f96900b8350 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9685ffb640 1 -- 172.21.15.47:0/1718440913 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c007b90 con 0x7f96900b8350 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9697a41640 1 -- 172.21.15.47:0/1718440913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969015e2c0 con 0x7f96900b8350 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.099+0000 7f9685ffb640 1 -- 172.21.15.47:0/1718440913 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c00ce40 con 0x7f96900b8350 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.100+0000 7f9697a41640 1 -- 172.21.15.47:0/1718440913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969015d2a0 con 0x7f96900b8350 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.100+0000 7f9697a41640 1 -- 172.21.15.47:0/1718440913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9690072360 con 0x7f96900b8350 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.101+0000 7f9685ffb640 1 -- 172.21.15.47:0/1718440913 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f968c0113f0 con 0x7f96900b8350 2024-03-23T09:41:34.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.101+0000 7f9685ffb640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f967406ac00 0x7f967406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.101+0000 7f9685ffb640 1 -- 172.21.15.47:0/1718440913 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f968c089480 con 0x7f96900b8350 2024-03-23T09:41:34.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.101+0000 7f9694fdc640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f967406ac00 0x7f967406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.102+0000 7f9694fdc640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f967406ac00 0x7f967406d0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9680006fd0 tx=0x7f9680008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:34.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.106+0000 7f9685ffb640 1 -- 172.21.15.47:0/1718440913 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f968c0546f0 con 0x7f96900b8350 2024-03-23T09:41:34.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.284+0000 7f9697a41640 1 -- 172.21.15.47:0/1718440913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9690066be0 con 0x7f96900b8350 2024-03-23T09:41:34.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.284+0000 7f9685ffb640 1 -- 172.21.15.47:0/1718440913 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f968c057d10 con 0x7f96900b8350 2024-03-23T09:41:34.284 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:34.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.286+0000 7f9697a41640 1 -- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f967406ac00 msgr2=0x7f967406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.286+0000 7f9697a41640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f967406ac00 0x7f967406d0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9680006fd0 tx=0x7f9680008040 comp rx=0 tx=0).stop 2024-03-23T09:41:34.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.286+0000 7f9697a41640 1 -- 172.21.15.47:0/1718440913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96900b8350 msgr2=0x7f969015dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.286+0000 7f9697a41640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96900b8350 0x7f969015dd80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c7f0 tx=0x7f968c00cbf0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.286+0000 7f9697a41640 1 -- 172.21.15.47:0/1718440913 shutdown_connections 2024-03-23T09:41:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.286+0000 7f9697a41640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f967406ac00 0x7f967406d0c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.286+0000 7f9697a41640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f969015c990 0x7f96900b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.286+0000 7f9697a41640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96900cdc40 0x7f9690155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.286+0000 7f9697a41640 1 --2- 172.21.15.47:0/1718440913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96900b8350 0x7f969015dd80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.286+0000 7f9697a41640 1 -- 172.21.15.47:0/1718440913 >> 172.21.15.47:0/1718440913 conn(0x7f96900659d0 msgr2=0x7f96900b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.286+0000 7f9697a41640 1 -- 172.21.15.47:0/1718440913 shutdown_connections 2024-03-23T09:41:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.286+0000 7f9697a41640 1 -- 172.21.15.47:0/1718440913 wait complete. 2024-03-23T09:41:34.297 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:34.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.402+0000 7fe25ee07640 1 Processor -- start 2024-03-23T09:41:34.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.402+0000 7fe25ee07640 1 -- start start 2024-03-23T09:41:34.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.403+0000 7fe25ee07640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2580cde10 0x7fe258160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.403+0000 7fe25ee07640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe258161630 con 0x7fe25815c990 2024-03-23T09:41:34.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.403+0000 7fe25ee07640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2580d1030 con 0x7fe2580b8350 2024-03-23T09:41:34.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.403+0000 7fe25ee07640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2580cfe10 con 0x7fe2580cde10 2024-03-23T09:41:34.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.403+0000 7fe257fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2580cde10 0x7fe258160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.403+0000 7fe25cba3640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe25815c990 0x7fe2580cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34048/0 (socket says 172.21.15.47:34048) 2024-03-23T09:41:34.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.403+0000 7fe257fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2580cde10 0x7fe258160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56694/0 (socket says 172.21.15.47:56694) 2024-03-23T09:41:34.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.403+0000 7fe25cba3640 1 -- 172.21.15.47:0/3476394231 learned_addr learned my addr 172.21.15.47:0/3476394231 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:34.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.404+0000 7fe257fff640 1 -- 172.21.15.47:0/3476394231 >> v1:172.21.15.175:6789/0 conn(0x7fe2580b8350 legacy=0x7fe2580d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:34.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.404+0000 7fe257fff640 1 -- 172.21.15.47:0/3476394231 >> v1:172.21.15.47:6789/0 conn(0x7fe25815c990 legacy=0x7fe2580cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.404+0000 7fe257fff640 1 -- 172.21.15.47:0/3476394231 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2580b5b60 con 0x7fe2580cde10 2024-03-23T09:41:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.404+0000 7fe257fff640 1 --2- 172.21.15.47:0/3476394231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2580cde10 0x7fe258160690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe248009d90 tx=0x7fe24802bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a078cf4634ff6e07 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.404+0000 7fe2577fe640 1 -- 172.21.15.47:0/3476394231 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24802c990 con 0x7fe2580cde10 2024-03-23T09:41:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.404+0000 7fe2577fe640 1 -- 172.21.15.47:0/3476394231 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24802caf0 con 0x7fe2580cde10 2024-03-23T09:41:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.404+0000 7fe2577fe640 1 -- 172.21.15.47:0/3476394231 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24802ce80 con 0x7fe2580cde10 2024-03-23T09:41:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.405+0000 7fe25ee07640 1 -- 172.21.15.47:0/3476394231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2580cde10 msgr2=0x7fe258160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.405+0000 7fe25ee07640 1 --2- 172.21.15.47:0/3476394231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2580cde10 0x7fe258160690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe248009d90 tx=0x7fe24802bc60 comp rx=0 tx=0).stop 2024-03-23T09:41:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.405+0000 7fe25ee07640 1 -- 172.21.15.47:0/3476394231 shutdown_connections 2024-03-23T09:41:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.405+0000 7fe25ee07640 1 --2- 172.21.15.47:0/3476394231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2580cde10 0x7fe258160690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.405+0000 7fe25ee07640 1 -- 172.21.15.47:0/3476394231 >> 172.21.15.47:0/3476394231 conn(0x7fe2580659d0 msgr2=0x7fe258065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.405+0000 7fe25ee07640 1 -- 172.21.15.47:0/3476394231 shutdown_connections 2024-03-23T09:41:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.405+0000 7fe25ee07640 1 -- 172.21.15.47:0/3476394231 wait complete. 2024-03-23T09:41:34.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.406+0000 7fe25ee07640 1 Processor -- start 2024-03-23T09:41:34.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.406+0000 7fe25ee07640 1 -- start start 2024-03-23T09:41:34.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe25ee07640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580b8350 0x7fe258150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe25ee07640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2580cde10 0x7fe25815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe25ee07640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe25815c990 0x7fe25815add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe25ee07640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25815e120 con 0x7fe2580cde10 2024-03-23T09:41:34.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe25ee07640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe258153500 con 0x7fe25815c990 2024-03-23T09:41:34.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe25ee07640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe258153fd0 con 0x7fe2580b8350 2024-03-23T09:41:34.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe257fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2580cde10 0x7fe25815a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe257fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2580cde10 0x7fe25815a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56696/0 (socket says 172.21.15.47:56696) 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe257fff640 1 -- 172.21.15.47:0/3179347664 learned_addr learned my addr 172.21.15.47:0/3179347664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe25cba3640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580b8350 0x7fe258150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe25d3a4640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe25815c990 0x7fe25815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe257fff640 1 -- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580b8350 msgr2=0x7fe258150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe257fff640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580b8350 0x7fe258150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe257fff640 1 -- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe25815c990 msgr2=0x7fe25815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.407+0000 7fe257fff640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe25815c990 0x7fe25815add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.408+0000 7fe257fff640 1 -- 172.21.15.47:0/3179347664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe248009a70 con 0x7fe2580cde10 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.408+0000 7fe257fff640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2580cde10 0x7fe25815a890 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe248037040 tx=0x7fe248004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.408+0000 7fe2557fa640 1 -- 172.21.15.47:0/3179347664 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24803e610 con 0x7fe2580cde10 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.408+0000 7fe2557fa640 1 -- 172.21.15.47:0/3179347664 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24803e770 con 0x7fe2580cde10 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.408+0000 7fe25ee07640 1 -- 172.21.15.47:0/3179347664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe258153730 con 0x7fe2580cde10 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.408+0000 7fe2557fa640 1 -- 172.21.15.47:0/3179347664 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24803eb20 con 0x7fe2580cde10 2024-03-23T09:41:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.408+0000 7fe25ee07640 1 -- 172.21.15.47:0/3179347664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe258153cf0 con 0x7fe2580cde10 2024-03-23T09:41:34.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.409+0000 7fe25ee07640 1 -- 172.21.15.47:0/3179347664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe258072360 con 0x7fe2580cde10 2024-03-23T09:41:34.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.409+0000 7fe2557fa640 1 -- 172.21.15.47:0/3179347664 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe24803b050 con 0x7fe2580cde10 2024-03-23T09:41:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.410+0000 7fe2557fa640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe23806ac00 0x7fe23806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.410+0000 7fe2557fa640 1 -- 172.21.15.47:0/3179347664 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2480abe20 con 0x7fe2580cde10 2024-03-23T09:41:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.410+0000 7fe25cba3640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe23806ac00 0x7fe23806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.410+0000 7fe25cba3640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe23806ac00 0x7fe23806d0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe250000f60 tx=0x7fe250009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:34.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.414+0000 7fe2557fa640 1 -- 172.21.15.47:0/3179347664 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe2480771e0 con 0x7fe2580cde10 2024-03-23T09:41:34.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.594+0000 7fe25ee07640 1 -- 172.21.15.47:0/3179347664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe258072360 con 0x7fe2580cde10 2024-03-23T09:41:34.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.595+0000 7fe2557fa640 1 -- 172.21.15.47:0/3179347664 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe24804e020 con 0x7fe2580cde10 2024-03-23T09:41:34.594 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:34.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.596+0000 7fe25ee07640 1 -- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe23806ac00 msgr2=0x7fe23806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.596+0000 7fe25ee07640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe23806ac00 0x7fe23806d0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe250000f60 tx=0x7fe250009040 comp rx=0 tx=0).stop 2024-03-23T09:41:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.596+0000 7fe25ee07640 1 -- 172.21.15.47:0/3179347664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2580cde10 msgr2=0x7fe25815a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.596+0000 7fe25ee07640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2580cde10 0x7fe25815a890 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe248037040 tx=0x7fe248004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.597+0000 7fe25ee07640 1 -- 172.21.15.47:0/3179347664 shutdown_connections 2024-03-23T09:41:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.597+0000 7fe25ee07640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe23806ac00 0x7fe23806d0c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.597+0000 7fe25ee07640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe25815c990 0x7fe25815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.597+0000 7fe25ee07640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2580cde10 0x7fe25815a890 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.597+0000 7fe25ee07640 1 --2- 172.21.15.47:0/3179347664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580b8350 0x7fe258150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.597+0000 7fe25ee07640 1 -- 172.21.15.47:0/3179347664 >> 172.21.15.47:0/3179347664 conn(0x7fe2580659d0 msgr2=0x7fe2580cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.597+0000 7fe25ee07640 1 -- 172.21.15.47:0/3179347664 shutdown_connections 2024-03-23T09:41:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.597+0000 7fe25ee07640 1 -- 172.21.15.47:0/3179347664 wait complete. 2024-03-23T09:41:34.607 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:41:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.709+0000 7fdb68137640 1 Processor -- start 2024-03-23T09:41:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.709+0000 7fdb68137640 1 -- start start 2024-03-23T09:41:34.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.710+0000 7fdb68137640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdb600cf210 0x7fdb600cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.710+0000 7fdb68137640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb600cfbd0 con 0x7fdb600b5a90 2024-03-23T09:41:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.710+0000 7fdb68137640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb6015d0e0 con 0x7fdb6015cd00 2024-03-23T09:41:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.710+0000 7fdb68137640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb600d1540 con 0x7fdb600cf210 2024-03-23T09:41:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.710+0000 7fdb666d4640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdb600b5a90 0x7fdb600b5ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51044/0 (socket says 172.21.15.47:51044) 2024-03-23T09:41:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.710+0000 7fdb666d4640 1 -- 172.21.15.47:0/2961241375 learned_addr learned my addr 172.21.15.47:0/2961241375 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.711+0000 7fdb656d2640 1 --2- 172.21.15.47:0/2961241375 >> v2:172.21.15.175:3300/0 conn(0x7fdb600cf210 0x7fdb600cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.711+0000 7fdb64ed1640 1 -- 172.21.15.47:0/2961241375 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987905301 0 0) 0x7fdb600cfbd0 con 0x7fdb600b5a90 2024-03-23T09:41:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.711+0000 7fdb64ed1640 1 -- 172.21.15.47:0/2961241375 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb48003520 con 0x7fdb600b5a90 2024-03-23T09:41:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.711+0000 7fdb656d2640 1 -- 172.21.15.47:0/2961241375 >> v1:172.21.15.175:6789/0 conn(0x7fdb6015cd00 legacy=0x7fdb600d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.711+0000 7fdb656d2640 1 -- 172.21.15.47:0/2961241375 >> v1:172.21.15.47:6789/0 conn(0x7fdb600b5a90 legacy=0x7fdb600b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.711+0000 7fdb656d2640 1 -- 172.21.15.47:0/2961241375 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb600d1820 con 0x7fdb600cf210 2024-03-23T09:41:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.712+0000 7fdb656d2640 1 --2- 172.21.15.47:0/2961241375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb600cf210 0x7fdb600cf690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdb540097a0 tx=0x7fdb5402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=661b8a301bae32db server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.712+0000 7fdb64ed1640 1 -- 172.21.15.47:0/2961241375 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54031830 con 0x7fdb600cf210 2024-03-23T09:41:34.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.712+0000 7fdb64ed1640 1 -- 172.21.15.47:0/2961241375 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb54031990 con 0x7fdb600cf210 2024-03-23T09:41:34.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.712+0000 7fdb64ed1640 1 -- 172.21.15.47:0/2961241375 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54031d20 con 0x7fdb600cf210 2024-03-23T09:41:34.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.712+0000 7fdb68137640 1 -- 172.21.15.47:0/2961241375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb600cf210 msgr2=0x7fdb600cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.712+0000 7fdb68137640 1 --2- 172.21.15.47:0/2961241375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb600cf210 0x7fdb600cf690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdb540097a0 tx=0x7fdb5402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.712+0000 7fdb68137640 1 -- 172.21.15.47:0/2961241375 shutdown_connections 2024-03-23T09:41:34.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.712+0000 7fdb68137640 1 --2- 172.21.15.47:0/2961241375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb600cf210 0x7fdb600cf690 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.712+0000 7fdb68137640 1 -- 172.21.15.47:0/2961241375 >> 172.21.15.47:0/2961241375 conn(0x7fdb6006b160 msgr2=0x7fdb6006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:34.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.713+0000 7fdb68137640 1 -- 172.21.15.47:0/2961241375 shutdown_connections 2024-03-23T09:41:34.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.713+0000 7fdb68137640 1 -- 172.21.15.47:0/2961241375 wait complete. 2024-03-23T09:41:34.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.714+0000 7fdb68137640 1 Processor -- start 2024-03-23T09:41:34.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.714+0000 7fdb68137640 1 -- start start 2024-03-23T09:41:34.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.714+0000 7fdb68137640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb600b5a90 0x7fdb601f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.714+0000 7fdb68137640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb600cf210 0x7fdb601f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.714+0000 7fdb68137640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb6015cd00 0x7fdb600cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.714+0000 7fdb68137640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb601f6250 con 0x7fdb6015cd00 2024-03-23T09:41:34.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.714+0000 7fdb68137640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb601f6530 con 0x7fdb600cf210 2024-03-23T09:41:34.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.714+0000 7fdb68137640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb601f6810 con 0x7fdb600b5a90 2024-03-23T09:41:34.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.714+0000 7fdb666d4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb6015cd00 0x7fdb600cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb666d4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb6015cd00 0x7fdb600cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50020/0 (socket says 172.21.15.47:50020) 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb666d4640 1 -- 172.21.15.47:0/2645091953 learned_addr learned my addr 172.21.15.47:0/2645091953 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb656d2640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb600cf210 0x7fdb601f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb65ed3640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb600b5a90 0x7fdb601f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb666d4640 1 -- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb600b5a90 msgr2=0x7fdb601f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb666d4640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb600b5a90 0x7fdb601f5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb666d4640 1 -- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb600cf210 msgr2=0x7fdb601f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb666d4640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb600cf210 0x7fdb601f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb666d4640 1 -- 172.21.15.47:0/2645091953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb54009400 con 0x7fdb6015cd00 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb666d4640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb6015cd00 0x7fdb600cd1e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000cb40 tx=0x7fdb5000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb5e7fc640 1 -- 172.21.15.47:0/2645091953 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5000aca0 con 0x7fdb6015cd00 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb5e7fc640 1 -- 172.21.15.47:0/2645091953 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5000ae00 con 0x7fdb6015cd00 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb68137640 1 -- 172.21.15.47:0/2645091953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb600a9330 con 0x7fdb6015cd00 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb5e7fc640 1 -- 172.21.15.47:0/2645091953 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50012620 con 0x7fdb6015cd00 2024-03-23T09:41:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.715+0000 7fdb68137640 1 -- 172.21.15.47:0/2645091953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb601f5f60 con 0x7fdb6015cd00 2024-03-23T09:41:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.716+0000 7fdb68137640 1 -- 172.21.15.47:0/2645091953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb60072360 con 0x7fdb6015cd00 2024-03-23T09:41:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.717+0000 7fdb5e7fc640 1 -- 172.21.15.47:0/2645091953 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdb50007070 con 0x7fdb6015cd00 2024-03-23T09:41:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.717+0000 7fdb5e7fc640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdb4c06ac00 0x7fdb4c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.717+0000 7fdb5e7fc640 1 -- 172.21.15.47:0/2645091953 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb500892c0 con 0x7fdb6015cd00 2024-03-23T09:41:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.717+0000 7fdb65ed3640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdb4c06ac00 0x7fdb4c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.718+0000 7fdb65ed3640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdb4c06ac00 0x7fdb4c06d0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdb58000f60 tx=0x7fdb58009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:34.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.721+0000 7fdb5e7fc640 1 -- 172.21.15.47:0/2645091953 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdb50054560 con 0x7fdb6015cd00 2024-03-23T09:41:34.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.917+0000 7fdb68137640 1 -- 172.21.15.47:0/2645091953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdb600a9850 con 0x7fdb6015cd00 2024-03-23T09:41:34.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.917+0000 7fdb5e7fc640 1 -- 172.21.15.47:0/2645091953 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdb600a9850 con 0x7fdb6015cd00 2024-03-23T09:41:34.917 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:41:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.919+0000 7fdb68137640 1 -- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdb4c06ac00 msgr2=0x7fdb4c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.919+0000 7fdb68137640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdb4c06ac00 0x7fdb4c06d0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdb58000f60 tx=0x7fdb58009040 comp rx=0 tx=0).stop 2024-03-23T09:41:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.919+0000 7fdb68137640 1 -- 172.21.15.47:0/2645091953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb6015cd00 msgr2=0x7fdb600cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.919+0000 7fdb68137640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb6015cd00 0x7fdb600cd1e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000cb40 tx=0x7fdb5000a060 comp rx=0 tx=0).stop 2024-03-23T09:41:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.919+0000 7fdb68137640 1 -- 172.21.15.47:0/2645091953 shutdown_connections 2024-03-23T09:41:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.919+0000 7fdb68137640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdb4c06ac00 0x7fdb4c06d0c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.919+0000 7fdb68137640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb6015cd00 0x7fdb600cd1e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.919+0000 7fdb68137640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb600cf210 0x7fdb601f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.919+0000 7fdb68137640 1 --2- 172.21.15.47:0/2645091953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb600b5a90 0x7fdb601f5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.919+0000 7fdb68137640 1 -- 172.21.15.47:0/2645091953 >> 172.21.15.47:0/2645091953 conn(0x7fdb6006b160 msgr2=0x7fdb600b7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.919+0000 7fdb68137640 1 -- 172.21.15.47:0/2645091953 shutdown_connections 2024-03-23T09:41:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:34.919+0000 7fdb68137640 1 -- 172.21.15.47:0/2645091953 wait complete. 2024-03-23T09:41:34.929 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:41:34.929 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:41:34.944 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:35.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.085+0000 7f8065181640 1 Processor -- start 2024-03-23T09:41:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.085+0000 7f8065181640 1 -- start start 2024-03-23T09:41:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f8065181640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f80600b8350 0x7f80600cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f8065181640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80600cdc40 0x7f80600d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f8065181640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80600d11e0 con 0x7f806015ca90 2024-03-23T09:41:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f8065181640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80600ce0d0 con 0x7f80600b8350 2024-03-23T09:41:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f8065181640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8060160f60 con 0x7f80600cdc40 2024-03-23T09:41:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f805f59e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80600cdc40 0x7f80600d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f805659c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f80600b8350 0x7f80600cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f805f59e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80600cdc40 0x7f80600d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:59240/0 (socket says 172.21.15.47:59240) 2024-03-23T09:41:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f805659c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f80600b8350 0x7f80600cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40658/0 (socket says 172.21.15.47:40658) 2024-03-23T09:41:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f805f59e640 1 -- 172.21.15.47:0/2470376689 learned_addr learned my addr 172.21.15.47:0/2470376689 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f805ed9d640 1 --1- 172.21.15.47:0/2470376689 >> v1:172.21.15.175:6789/0 conn(0x7f806015ca90 0x7f8060160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:60950/0 (socket says 172.21.15.47:60950) 2024-03-23T09:41:35.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f805659c640 1 -- 172.21.15.47:0/2470376689 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80600cdc40 msgr2=0x7f80600d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:35.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f805659c640 1 --2- 172.21.15.47:0/2470376689 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80600cdc40 0x7f80600d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f805659c640 1 -- 172.21.15.47:0/2470376689 >> v1:172.21.15.175:6789/0 conn(0x7f806015ca90 legacy=0x7f8060160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f805659c640 1 -- 172.21.15.47:0/2470376689 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8060161240 con 0x7f80600b8350 2024-03-23T09:41:35.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.086+0000 7f805e59c640 1 -- 172.21.15.47:0/2470376689 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102887795 0 0) 0x7f80600d11e0 con 0x7f806015ca90 2024-03-23T09:41:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.087+0000 7f805659c640 1 --2- 172.21.15.47:0/2470376689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80600b8350 0x7f80600cd700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f80480097a0 tx=0x7f804802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510b55b33a22b878 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.087+0000 7f805e59c640 1 -- 172.21.15.47:0/2470376689 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804802e070 con 0x7f80600b8350 2024-03-23T09:41:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.087+0000 7f805e59c640 1 -- 172.21.15.47:0/2470376689 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80480315a0 con 0x7f80600b8350 2024-03-23T09:41:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.087+0000 7f805e59c640 1 -- 172.21.15.47:0/2470376689 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048031930 con 0x7f80600b8350 2024-03-23T09:41:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.088+0000 7f8065181640 1 -- 172.21.15.47:0/2470376689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80600b8350 msgr2=0x7f80600cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.088+0000 7f8065181640 1 --2- 172.21.15.47:0/2470376689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80600b8350 0x7f80600cd700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f80480097a0 tx=0x7f804802b840 comp rx=0 tx=0).stop 2024-03-23T09:41:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.088+0000 7f8065181640 1 -- 172.21.15.47:0/2470376689 shutdown_connections 2024-03-23T09:41:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.088+0000 7f8065181640 1 --2- 172.21.15.47:0/2470376689 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80600cdc40 0x7f80600d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.088+0000 7f8065181640 1 --2- 172.21.15.47:0/2470376689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80600b8350 0x7f80600cd700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.088+0000 7f8065181640 1 -- 172.21.15.47:0/2470376689 >> 172.21.15.47:0/2470376689 conn(0x7f80600659d0 msgr2=0x7f8060065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.088+0000 7f8065181640 1 -- 172.21.15.47:0/2470376689 shutdown_connections 2024-03-23T09:41:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.088+0000 7f8065181640 1 -- 172.21.15.47:0/2470376689 wait complete. 2024-03-23T09:41:35.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.089+0000 7f8065181640 1 Processor -- start 2024-03-23T09:41:35.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.089+0000 7f8065181640 1 -- start start 2024-03-23T09:41:35.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.090+0000 7f8065181640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80600b8350 0x7f8060153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.090+0000 7f8065181640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80600cdc40 0x7f80600b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.090+0000 7f8065181640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f806015ca90 0x7f80600cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.090+0000 7f8065181640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80601f6950 con 0x7f80600cdc40 2024-03-23T09:41:35.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.090+0000 7f8065181640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80600b5360 con 0x7f806015ca90 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.090+0000 7f8065181640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80601f5cd0 con 0x7f80600b8350 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.090+0000 7f805659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80600cdc40 0x7f80600b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.090+0000 7f805659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80600cdc40 0x7f80600b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50024/0 (socket says 172.21.15.47:50024) 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.090+0000 7f805659c640 1 -- 172.21.15.47:0/2374068764 learned_addr learned my addr 172.21.15.47:0/2374068764 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f805ed9d640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80600b8350 0x7f8060153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f805f59e640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f806015ca90 0x7f80600cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f805659c640 1 -- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80600b8350 msgr2=0x7f8060153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f805659c640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80600b8350 0x7f8060153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f805659c640 1 -- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f806015ca90 msgr2=0x7f80600cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f805659c640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f806015ca90 0x7f80600cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f805659c640 1 -- 172.21.15.47:0/2374068764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8048009400 con 0x7f80600cdc40 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f805ed9d640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80600b8350 0x7f8060153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f805f59e640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f806015ca90 0x7f80600cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f805659c640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80600cdc40 0x7f80600b4e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8048034040 tx=0x7f80480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f8057fff640 1 -- 172.21.15.47:0/2374068764 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048032840 con 0x7f80600cdc40 2024-03-23T09:41:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f8057fff640 1 -- 172.21.15.47:0/2374068764 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8048031f00 con 0x7f80600cdc40 2024-03-23T09:41:35.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f8065181640 1 -- 172.21.15.47:0/2374068764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80601f5fb0 con 0x7f80600cdc40 2024-03-23T09:41:35.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f8057fff640 1 -- 172.21.15.47:0/2374068764 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048032be0 con 0x7f80600cdc40 2024-03-23T09:41:35.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.091+0000 7f8065181640 1 -- 172.21.15.47:0/2374068764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80601f6570 con 0x7f80600cdc40 2024-03-23T09:41:35.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.092+0000 7f8065181640 1 -- 172.21.15.47:0/2374068764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8060072360 con 0x7f80600cdc40 2024-03-23T09:41:35.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.092+0000 7f8057fff640 1 -- 172.21.15.47:0/2374068764 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8048032d40 con 0x7f80600cdc40 2024-03-23T09:41:35.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.093+0000 7f8057fff640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f804006ac00 0x7f804006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.093+0000 7f8057fff640 1 -- 172.21.15.47:0/2374068764 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80480ab230 con 0x7f80600cdc40 2024-03-23T09:41:35.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.094+0000 7f805ed9d640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f804006ac00 0x7f804006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.094+0000 7f805ed9d640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f804006ac00 0x7f804006d0c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8050000f60 tx=0x7f8050009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:35.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.097+0000 7f8057fff640 1 -- 172.21.15.47:0/2374068764 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8048076550 con 0x7f80600cdc40 2024-03-23T09:41:35.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.281+0000 7f8065181640 1 -- 172.21.15.47:0/2374068764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80601f6a90 con 0x7f80600cdc40 2024-03-23T09:41:35.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.282+0000 7f8057fff640 1 -- 172.21.15.47:0/2374068764 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f80601f6a90 con 0x7f80600cdc40 2024-03-23T09:41:35.282 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:35.282 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.284+0000 7f8065181640 1 -- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f804006ac00 msgr2=0x7f804006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.284+0000 7f8065181640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f804006ac00 0x7f804006d0c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8050000f60 tx=0x7f8050009040 comp rx=0 tx=0).stop 2024-03-23T09:41:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.284+0000 7f8065181640 1 -- 172.21.15.47:0/2374068764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80600cdc40 msgr2=0x7f80600b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.284+0000 7f8065181640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80600cdc40 0x7f80600b4e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8048034040 tx=0x7f80480046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.284+0000 7f8065181640 1 -- 172.21.15.47:0/2374068764 shutdown_connections 2024-03-23T09:41:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.284+0000 7f8065181640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f804006ac00 0x7f804006d0c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.284+0000 7f8065181640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f806015ca90 0x7f80600cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.284+0000 7f8065181640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80600cdc40 0x7f80600b4e20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.284+0000 7f8065181640 1 --2- 172.21.15.47:0/2374068764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80600b8350 0x7f8060153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.284+0000 7f8065181640 1 -- 172.21.15.47:0/2374068764 >> 172.21.15.47:0/2374068764 conn(0x7f80600659d0 msgr2=0x7f806015fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.284+0000 7f8065181640 1 -- 172.21.15.47:0/2374068764 shutdown_connections 2024-03-23T09:41:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.285+0000 7f8065181640 1 -- 172.21.15.47:0/2374068764 wait complete. 2024-03-23T09:41:35.295 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:35.295 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.399+0000 7f6a3cd99640 1 Processor -- start 2024-03-23T09:41:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.399+0000 7f6a3cd99640 1 -- start start 2024-03-23T09:41:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.400+0000 7f6a3cd99640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6a3806d3a0 0x7f6a3815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.400+0000 7f6a3cd99640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3815d210 con 0x7f6a380ce830 2024-03-23T09:41:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.400+0000 7f6a3cd99640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3806a2c0 con 0x7f6a38069ce0 2024-03-23T09:41:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.400+0000 7f6a3cd99640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3806a5a0 con 0x7f6a3806d3a0 2024-03-23T09:41:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.400+0000 7f6a36d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6a3806d3a0 0x7f6a3815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.400+0000 7f6a3659c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6a380ce830 0x7f6a380cec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51048/0 (socket says 172.21.15.47:51048) 2024-03-23T09:41:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.400+0000 7f6a3659c640 1 -- 172.21.15.47:0/1806300350 learned_addr learned my addr 172.21.15.47:0/1806300350 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:35.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.401+0000 7f6a36d9d640 1 -- 172.21.15.47:0/1806300350 >> v1:172.21.15.175:6789/0 conn(0x7f6a38069ce0 legacy=0x7f6a380ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.401+0000 7f6a36d9d640 1 -- 172.21.15.47:0/1806300350 >> v1:172.21.15.47:6789/0 conn(0x7f6a380ce830 legacy=0x7f6a380cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.401+0000 7f6a3559a640 1 -- 172.21.15.47:0/1806300350 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 560957496 0 0) 0x7f6a3815d210 con 0x7f6a380ce830 2024-03-23T09:41:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.401+0000 7f6a36d9d640 1 -- 172.21.15.47:0/1806300350 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3806a880 con 0x7f6a3806d3a0 2024-03-23T09:41:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.401+0000 7f6a36d9d640 1 --2- 172.21.15.47:0/1806300350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a3806d3a0 0x7f6a3815d6a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009d90 tx=0x7f6a2802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6f2304cb0f7528 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.402+0000 7f6a3559a640 1 -- 172.21.15.47:0/1806300350 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2802cc60 con 0x7f6a3806d3a0 2024-03-23T09:41:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.402+0000 7f6a3559a640 1 -- 172.21.15.47:0/1806300350 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2802cdc0 con 0x7f6a3806d3a0 2024-03-23T09:41:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.402+0000 7f6a3559a640 1 -- 172.21.15.47:0/1806300350 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28034660 con 0x7f6a3806d3a0 2024-03-23T09:41:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.402+0000 7f6a3cd99640 1 -- 172.21.15.47:0/1806300350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a3806d3a0 msgr2=0x7f6a3815d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.402+0000 7f6a3cd99640 1 --2- 172.21.15.47:0/1806300350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a3806d3a0 0x7f6a3815d6a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009d90 tx=0x7f6a2802beb0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.402+0000 7f6a3cd99640 1 -- 172.21.15.47:0/1806300350 shutdown_connections 2024-03-23T09:41:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.402+0000 7f6a3cd99640 1 --2- 172.21.15.47:0/1806300350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a3806d3a0 0x7f6a3815d6a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.402+0000 7f6a3cd99640 1 -- 172.21.15.47:0/1806300350 >> 172.21.15.47:0/1806300350 conn(0x7f6a38098a00 msgr2=0x7f6a380b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.402+0000 7f6a3cd99640 1 -- 172.21.15.47:0/1806300350 shutdown_connections 2024-03-23T09:41:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.402+0000 7f6a3cd99640 1 -- 172.21.15.47:0/1806300350 wait complete. 2024-03-23T09:41:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.403+0000 7f6a3cd99640 1 Processor -- start 2024-03-23T09:41:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.404+0000 7f6a3cd99640 1 -- start start 2024-03-23T09:41:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.404+0000 7f6a3cd99640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a38069ce0 0x7f6a38148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.404+0000 7f6a3cd99640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a3806d3a0 0x7f6a380c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.404+0000 7f6a3cd99640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a380ce830 0x7f6a380b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.404+0000 7f6a3cd99640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a380c43b0 con 0x7f6a38069ce0 2024-03-23T09:41:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.404+0000 7f6a3cd99640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a380b3480 con 0x7f6a3806d3a0 2024-03-23T09:41:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.404+0000 7f6a3cd99640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a380b3760 con 0x7f6a380ce830 2024-03-23T09:41:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.404+0000 7f6a3659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a38069ce0 0x7f6a38148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.404+0000 7f6a3659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a38069ce0 0x7f6a38148890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50042/0 (socket says 172.21.15.47:50042) 2024-03-23T09:41:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.404+0000 7f6a3659c640 1 -- 172.21.15.47:0/3074210130 learned_addr learned my addr 172.21.15.47:0/3074210130 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.405+0000 7f6a36d9d640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a380ce830 0x7f6a380b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.405+0000 7f6a35d9b640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a3806d3a0 0x7f6a380c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.405+0000 7f6a3659c640 1 -- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a380ce830 msgr2=0x7f6a380b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.405+0000 7f6a3659c640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a380ce830 0x7f6a380b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.405+0000 7f6a3659c640 1 -- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a3806d3a0 msgr2=0x7f6a380c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.405+0000 7f6a3659c640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a3806d3a0 0x7f6a380c3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.405+0000 7f6a3659c640 1 -- 172.21.15.47:0/3074210130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a28009a70 con 0x7f6a38069ce0 2024-03-23T09:41:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.405+0000 7f6a3659c640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a38069ce0 0x7f6a38148890 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000c430 tx=0x7f6a3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.405+0000 7f6a2effd640 1 -- 172.21.15.47:0/3074210130 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a300074e0 con 0x7f6a38069ce0 2024-03-23T09:41:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.405+0000 7f6a2effd640 1 -- 172.21.15.47:0/3074210130 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a30007660 con 0x7f6a38069ce0 2024-03-23T09:41:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.405+0000 7f6a3cd99640 1 -- 172.21.15.47:0/3074210130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a380b3a40 con 0x7f6a38069ce0 2024-03-23T09:41:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.405+0000 7f6a2effd640 1 -- 172.21.15.47:0/3074210130 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30004a20 con 0x7f6a38069ce0 2024-03-23T09:41:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.405+0000 7f6a3cd99640 1 -- 172.21.15.47:0/3074210130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a38156290 con 0x7f6a38069ce0 2024-03-23T09:41:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.406+0000 7f6a3cd99640 1 -- 172.21.15.47:0/3074210130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a380b3f60 con 0x7f6a38069ce0 2024-03-23T09:41:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.407+0000 7f6a2effd640 1 -- 172.21.15.47:0/3074210130 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6a30004b80 con 0x7f6a38069ce0 2024-03-23T09:41:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.407+0000 7f6a2effd640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6a1806ac00 0x7f6a1806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.407+0000 7f6a2effd640 1 -- 172.21.15.47:0/3074210130 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a30089240 con 0x7f6a38069ce0 2024-03-23T09:41:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.407+0000 7f6a35d9b640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6a1806ac00 0x7f6a1806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.408+0000 7f6a35d9b640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6a1806ac00 0x7f6a1806d0c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6a380b13e0 tx=0x7f6a20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.411+0000 7f6a2effd640 1 -- 172.21.15.47:0/3074210130 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6a380b3f60 con 0x7f6a38069ce0 2024-03-23T09:41:35.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.589+0000 7f6a3cd99640 1 -- 172.21.15.47:0/3074210130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a380b3f60 con 0x7f6a38069ce0 2024-03-23T09:41:35.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.590+0000 7f6a2effd640 1 -- 172.21.15.47:0/3074210130 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a30054430 con 0x7f6a38069ce0 2024-03-23T09:41:35.590 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:35.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.592+0000 7f6a3cd99640 1 -- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6a1806ac00 msgr2=0x7f6a1806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.592+0000 7f6a3cd99640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6a1806ac00 0x7f6a1806d0c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6a380b13e0 tx=0x7f6a20009040 comp rx=0 tx=0).stop 2024-03-23T09:41:35.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.592+0000 7f6a3cd99640 1 -- 172.21.15.47:0/3074210130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a38069ce0 msgr2=0x7f6a38148890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.592+0000 7f6a3cd99640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a38069ce0 0x7f6a38148890 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000c430 tx=0x7f6a3000c8f0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.592+0000 7f6a3cd99640 1 -- 172.21.15.47:0/3074210130 shutdown_connections 2024-03-23T09:41:35.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.592+0000 7f6a3cd99640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6a1806ac00 0x7f6a1806d0c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.592+0000 7f6a3cd99640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a380ce830 0x7f6a380b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.592+0000 7f6a3cd99640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a3806d3a0 0x7f6a380c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.592+0000 7f6a3cd99640 1 --2- 172.21.15.47:0/3074210130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a38069ce0 0x7f6a38148890 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.592+0000 7f6a3cd99640 1 -- 172.21.15.47:0/3074210130 >> 172.21.15.47:0/3074210130 conn(0x7f6a38098a00 msgr2=0x7f6a3815a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:35.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.592+0000 7f6a3cd99640 1 -- 172.21.15.47:0/3074210130 shutdown_connections 2024-03-23T09:41:35.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.592+0000 7f6a3cd99640 1 -- 172.21.15.47:0/3074210130 wait complete. 2024-03-23T09:41:35.602 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.706+0000 7f7883111640 1 Processor -- start 2024-03-23T09:41:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.706+0000 7f7883111640 1 -- start start 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f7883111640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f787c15c990 0x7f787c160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f7883111640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f787c0cde10 0x7f787c0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f7883111640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c161630 con 0x7f787c0b8350 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f7883111640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787c0d1030 con 0x7f787c15c990 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f7883111640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787c0cfe10 con 0x7f787c0cde10 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f7880ead640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f787c15c990 0x7f787c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f787bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f787c0b8350 0x7f787c0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51056/0 (socket says 172.21.15.47:51056) 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f7880ead640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f787c15c990 0x7f787c160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50056/0 (socket says 172.21.15.47:50056) 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f787bfff640 1 -- 172.21.15.47:0/3146351796 learned_addr learned my addr 172.21.15.47:0/3146351796 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f7880ead640 1 -- 172.21.15.47:0/3146351796 >> v2:172.21.15.175:3300/0 conn(0x7f787c0cde10 msgr2=0x7f787c0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f7880ead640 1 --2- 172.21.15.47:0/3146351796 >> v2:172.21.15.175:3300/0 conn(0x7f787c0cde10 0x7f787c0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f7880ead640 1 -- 172.21.15.47:0/3146351796 >> v1:172.21.15.47:6789/0 conn(0x7f787c0b8350 legacy=0x7f787c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f7880ead640 1 -- 172.21.15.47:0/3146351796 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c0b5b60 con 0x7f787c15c990 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.707+0000 7f787b7fe640 1 -- 172.21.15.47:0/3146351796 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642035826 0 0) 0x7f787c161630 con 0x7f787c0b8350 2024-03-23T09:41:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.708+0000 7f7880ead640 1 --2- 172.21.15.47:0/3146351796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c15c990 0x7f787c160690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f787c0b87e0 tx=0x7f786c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8d0865ca130eb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.708+0000 7f787b7fe640 1 -- 172.21.15.47:0/3146351796 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c02e070 con 0x7f787c15c990 2024-03-23T09:41:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.708+0000 7f787b7fe640 1 -- 172.21.15.47:0/3146351796 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c0315a0 con 0x7f787c15c990 2024-03-23T09:41:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.708+0000 7f787b7fe640 1 -- 172.21.15.47:0/3146351796 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c031930 con 0x7f787c15c990 2024-03-23T09:41:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.709+0000 7f7883111640 1 -- 172.21.15.47:0/3146351796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c15c990 msgr2=0x7f787c160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.709+0000 7f7883111640 1 --2- 172.21.15.47:0/3146351796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c15c990 0x7f787c160690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f787c0b87e0 tx=0x7f786c02b840 comp rx=0 tx=0).stop 2024-03-23T09:41:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.709+0000 7f7883111640 1 -- 172.21.15.47:0/3146351796 shutdown_connections 2024-03-23T09:41:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.709+0000 7f7883111640 1 --2- 172.21.15.47:0/3146351796 >> v2:172.21.15.175:3300/0 conn(0x7f787c0cde10 0x7f787c0d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.709+0000 7f7883111640 1 --2- 172.21.15.47:0/3146351796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c15c990 0x7f787c160690 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.709+0000 7f7883111640 1 -- 172.21.15.47:0/3146351796 >> 172.21.15.47:0/3146351796 conn(0x7f787c0659d0 msgr2=0x7f787c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.709+0000 7f7883111640 1 -- 172.21.15.47:0/3146351796 shutdown_connections 2024-03-23T09:41:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.709+0000 7f7883111640 1 -- 172.21.15.47:0/3146351796 wait complete. 2024-03-23T09:41:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.710+0000 7f7883111640 1 Processor -- start 2024-03-23T09:41:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.710+0000 7f7883111640 1 -- start start 2024-03-23T09:41:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7883111640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c0b8350 0x7f787c153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7883111640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f787c0cde10 0x7f787c15de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7883111640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f787c15c990 0x7f787c1f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7883111640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c15e8d0 con 0x7f787c0b8350 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7883111640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c15ebb0 con 0x7f787c15c990 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7883111640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c15ee90 con 0x7f787c0cde10 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7880ead640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c0b8350 0x7f787c153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7880ead640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c0b8350 0x7f787c153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50062/0 (socket says 172.21.15.47:50062) 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7880ead640 1 -- 172.21.15.47:0/95113805 learned_addr learned my addr 172.21.15.47:0/95113805 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f787bfff640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f787c0cde10 0x7f787c15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7880ead640 1 -- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f787c0cde10 msgr2=0x7f787c15de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f78816ae640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f787c15c990 0x7f787c1f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7880ead640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f787c0cde10 0x7f787c15de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7880ead640 1 -- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f787c15c990 msgr2=0x7f787c1f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7880ead640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f787c15c990 0x7f787c1f2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7880ead640 1 -- 172.21.15.47:0/95113805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c009400 con 0x7f787c0b8350 2024-03-23T09:41:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f787bfff640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f787c0cde10 0x7f787c15de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f78816ae640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f787c15c990 0x7f787c1f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.711+0000 7f7880ead640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c0b8350 0x7f787c153580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f786c000c00 tx=0x7f786c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.712+0000 7f78797fa640 1 -- 172.21.15.47:0/95113805 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c032900 con 0x7f787c0b8350 2024-03-23T09:41:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.712+0000 7f78797fa640 1 -- 172.21.15.47:0/95113805 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c032a80 con 0x7f787c0b8350 2024-03-23T09:41:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.712+0000 7f7883111640 1 -- 172.21.15.47:0/95113805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c15f170 con 0x7f787c0b8350 2024-03-23T09:41:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.712+0000 7f78797fa640 1 -- 172.21.15.47:0/95113805 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c032e30 con 0x7f787c0b8350 2024-03-23T09:41:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.712+0000 7f7883111640 1 -- 172.21.15.47:0/95113805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c153ac0 con 0x7f787c0b8350 2024-03-23T09:41:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.713+0000 7f7883111640 1 -- 172.21.15.47:0/95113805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c072360 con 0x7f787c0b8350 2024-03-23T09:41:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.713+0000 7f78797fa640 1 -- 172.21.15.47:0/95113805 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f786c04d3f0 con 0x7f787c0b8350 2024-03-23T09:41:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.714+0000 7f78797fa640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f786806ac00 0x7f786806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.714+0000 7f78797fa640 1 -- 172.21.15.47:0/95113805 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f786c0ab0a0 con 0x7f787c0b8350 2024-03-23T09:41:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.714+0000 7f787bfff640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f786806ac00 0x7f786806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.714+0000 7f787bfff640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f786806ac00 0x7f786806d0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7870000f60 tx=0x7f7870009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.718+0000 7f78797fa640 1 -- 172.21.15.47:0/95113805 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f786c076280 con 0x7f787c0b8350 2024-03-23T09:41:35.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.896+0000 7f7883111640 1 -- 172.21.15.47:0/95113805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f787c071f40 con 0x7f787c0b8350 2024-03-23T09:41:35.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.896+0000 7f78797fa640 1 -- 172.21.15.47:0/95113805 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f786c0798a0 con 0x7f787c0b8350 2024-03-23T09:41:35.896 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:35.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.898+0000 7f7883111640 1 -- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f786806ac00 msgr2=0x7f786806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.898+0000 7f7883111640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f786806ac00 0x7f786806d0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7870000f60 tx=0x7f7870009040 comp rx=0 tx=0).stop 2024-03-23T09:41:35.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.898+0000 7f7883111640 1 -- 172.21.15.47:0/95113805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c0b8350 msgr2=0x7f787c153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:35.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.898+0000 7f7883111640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c0b8350 0x7f787c153580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f786c000c00 tx=0x7f786c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.898+0000 7f7883111640 1 -- 172.21.15.47:0/95113805 shutdown_connections 2024-03-23T09:41:35.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.898+0000 7f7883111640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f786806ac00 0x7f786806d0c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.898+0000 7f7883111640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f787c15c990 0x7f787c1f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.898+0000 7f7883111640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f787c0cde10 0x7f787c15de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.898+0000 7f7883111640 1 --2- 172.21.15.47:0/95113805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c0b8350 0x7f787c153580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:35.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.898+0000 7f7883111640 1 -- 172.21.15.47:0/95113805 >> 172.21.15.47:0/95113805 conn(0x7f787c0659d0 msgr2=0x7f787c15bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:35.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.898+0000 7f7883111640 1 -- 172.21.15.47:0/95113805 shutdown_connections 2024-03-23T09:41:35.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:35.898+0000 7f7883111640 1 -- 172.21.15.47:0/95113805 wait complete. 2024-03-23T09:41:35.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:36.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.012+0000 7f6501696640 1 Processor -- start 2024-03-23T09:41:36.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.012+0000 7f6501696640 1 -- start start 2024-03-23T09:41:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.013+0000 7f6501696640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f64fc15ac40 0x7f64fc0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.013+0000 7f6501696640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc0cb210 con 0x7f64fc0cec50 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.013+0000 7f6501696640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc0cb440 con 0x7f64fc0b0d00 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.013+0000 7f6501696640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64fc0cbac0 con 0x7f64fc15ac40 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.013+0000 7f64fa7fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f64fc0cec50 0x7f64fc0b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51066/0 (socket says 172.21.15.47:51066) 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.013+0000 7f64fa7fc640 1 -- 172.21.15.47:0/296751489 learned_addr learned my addr 172.21.15.47:0/296751489 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592087696 0 0) 0x7f64fc0cb210 con 0x7f64fc0cec50 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e4003520 con 0x7f64fc0cec50 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64faffd640 1 --2- 172.21.15.47:0/296751489 >> v2:172.21.15.175:3300/0 conn(0x7f64fc15ac40 0x7f64fc0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2246334407 0 0) 0x7f64e4003520 con 0x7f64fc0cec50 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 >> v2:172.21.15.175:3300/0 conn(0x7f64fc15ac40 msgr2=0x7f64fc0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 --2- 172.21.15.47:0/296751489 >> v2:172.21.15.175:3300/0 conn(0x7f64fc15ac40 0x7f64fc0cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 >> v1:172.21.15.175:6789/0 conn(0x7f64fc0b0d00 legacy=0x7f64fc098ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc099b30 con 0x7f64fc0cec50 2024-03-23T09:41:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f64f0001670 con 0x7f64fc0cec50 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 >> v1:172.21.15.47:6789/0 conn(0x7f64fc0cec50 legacy=0x7f64fc0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 --2- 172.21.15.47:0/296751489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e4003970 0x7f64e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 --2- 172.21.15.47:0/296751489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40047c0 0x7f64e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 --2- 172.21.15.47:0/296751489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64e4005180 0x7f64e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc0cbac0 con 0x7f64e4003970 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e400c020 con 0x7f64e4005180 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e400c1e0 con 0x7f64e40047c0 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64faffd640 1 --2- 172.21.15.47:0/296751489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e4003970 0x7f64e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64fa7fc640 1 --2- 172.21.15.47:0/296751489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40047c0 0x7f64e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64fb7fe640 1 --2- 172.21.15.47:0/296751489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64e4005180 0x7f64e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64faffd640 1 -- 172.21.15.47:0/296751489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40047c0 msgr2=0x7f64e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64faffd640 1 --2- 172.21.15.47:0/296751489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40047c0 0x7f64e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64faffd640 1 -- 172.21.15.47:0/296751489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64e4005180 msgr2=0x7f64e400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64faffd640 1 --2- 172.21.15.47:0/296751489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64e4005180 0x7f64e400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64faffd640 1 -- 172.21.15.47:0/296751489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e4003520 con 0x7f64e4003970 2024-03-23T09:41:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.014+0000 7f64fb7fe640 1 -- 172.21.15.47:0/296751489 reap_dead start 2024-03-23T09:41:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.015+0000 7f64faffd640 1 --2- 172.21.15.47:0/296751489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e4003970 0x7f64e4003d90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009ac0 tx=0x7f64e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae71b6809d587e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.015+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f402c6b0 con 0x7f64e4003970 2024-03-23T09:41:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.015+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f402c810 con 0x7f64e4003970 2024-03-23T09:41:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.015+0000 7f64f9ffb640 1 -- 172.21.15.47:0/296751489 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f402cba0 con 0x7f64e4003970 2024-03-23T09:41:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.016+0000 7f6501696640 1 -- 172.21.15.47:0/296751489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e4003970 msgr2=0x7f64e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.016+0000 7f6501696640 1 --2- 172.21.15.47:0/296751489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e4003970 0x7f64e4003d90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009ac0 tx=0x7f64e40057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.016+0000 7f6501696640 1 -- 172.21.15.47:0/296751489 shutdown_connections 2024-03-23T09:41:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.016+0000 7f6501696640 1 --2- 172.21.15.47:0/296751489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e4003970 0x7f64e4003d90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.016+0000 7f6501696640 1 -- 172.21.15.47:0/296751489 >> 172.21.15.47:0/296751489 conn(0x7f64fc0d0670 msgr2=0x7f64fc0d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.016+0000 7f6501696640 1 -- 172.21.15.47:0/296751489 shutdown_connections 2024-03-23T09:41:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.016+0000 7f6501696640 1 -- 172.21.15.47:0/296751489 wait complete. 2024-03-23T09:41:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.017+0000 7f6501696640 1 Processor -- start 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.017+0000 7f6501696640 1 -- start start 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f6501696640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e4003970 0x7f64fc1ed8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f6501696640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64fc15b020 0x7f64fc1f04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f6501696640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64fc1f0a00 0x7f64fc0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f6501696640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1f14d0 con 0x7f64e4003970 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f6501696640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1f0f10 con 0x7f64fc1f0a00 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f6501696640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1f11f0 con 0x7f64fc15b020 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f64faffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e4003970 0x7f64fc1ed8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f64faffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e4003970 0x7f64fc1ed8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50086/0 (socket says 172.21.15.47:50086) 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f64faffd640 1 -- 172.21.15.47:0/361772846 learned_addr learned my addr 172.21.15.47:0/361772846 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f64faffd640 1 -- 172.21.15.47:0/361772846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64fc15b020 msgr2=0x7f64fc1f04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f64faffd640 1 --2- 172.21.15.47:0/361772846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64fc15b020 0x7f64fc1f04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f64faffd640 1 -- 172.21.15.47:0/361772846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64fc1f0a00 msgr2=0x7f64fc0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f64faffd640 1 --2- 172.21.15.47:0/361772846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64fc1f0a00 0x7f64fc0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f64faffd640 1 -- 172.21.15.47:0/361772846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc0c9070 con 0x7f64e4003970 2024-03-23T09:41:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f64faffd640 1 --2- 172.21.15.47:0/361772846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e4003970 0x7f64fc1ed8f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009ac0 tx=0x7f64f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f64e37fe640 1 -- 172.21.15.47:0/361772846 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4003810 con 0x7f64e4003970 2024-03-23T09:41:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f64e37fe640 1 -- 172.21.15.47:0/361772846 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f4003970 con 0x7f64e4003970 2024-03-23T09:41:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f64e37fe640 1 -- 172.21.15.47:0/361772846 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4003d00 con 0x7f64e4003970 2024-03-23T09:41:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.018+0000 7f6501696640 1 -- 172.21.15.47:0/361772846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc0c9350 con 0x7f64e4003970 2024-03-23T09:41:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.019+0000 7f6501696640 1 -- 172.21.15.47:0/361772846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc1f3c00 con 0x7f64e4003970 2024-03-23T09:41:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.019+0000 7f6501696640 1 -- 172.21.15.47:0/361772846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64fc0c8ed0 con 0x7f64e4003970 2024-03-23T09:41:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.020+0000 7f64e37fe640 1 -- 172.21.15.47:0/361772846 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f64f4040450 con 0x7f64e4003970 2024-03-23T09:41:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.020+0000 7f64e37fe640 1 --2- 172.21.15.47:0/361772846 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f64e806ac30 0x7f64e806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.020+0000 7f64e37fe640 1 -- 172.21.15.47:0/361772846 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64f40ab1d0 con 0x7f64e4003970 2024-03-23T09:41:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.021+0000 7f64fa7fc640 1 --2- 172.21.15.47:0/361772846 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f64e806ac30 0x7f64e806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.021+0000 7f64fa7fc640 1 --2- 172.21.15.47:0/361772846 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f64e806ac30 0x7f64e806d0f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f64fc069520 tx=0x7f64f0000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.024+0000 7f64e37fe640 1 -- 172.21.15.47:0/361772846 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f64fc0c8ed0 con 0x7f64e4003970 2024-03-23T09:41:36.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.206+0000 7f6501696640 1 -- 172.21.15.47:0/361772846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64fc0c8ed0 con 0x7f64e4003970 2024-03-23T09:41:36.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.207+0000 7f64e37fe640 1 -- 172.21.15.47:0/361772846 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64fc0c8ed0 con 0x7f64e4003970 2024-03-23T09:41:36.206 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:36.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.208+0000 7f6501696640 1 -- 172.21.15.47:0/361772846 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f64e806ac30 msgr2=0x7f64e806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.208+0000 7f6501696640 1 --2- 172.21.15.47:0/361772846 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f64e806ac30 0x7f64e806d0f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f64fc069520 tx=0x7f64f0000ed0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.208+0000 7f6501696640 1 -- 172.21.15.47:0/361772846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e4003970 msgr2=0x7f64fc1ed8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.208+0000 7f6501696640 1 --2- 172.21.15.47:0/361772846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e4003970 0x7f64fc1ed8f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009ac0 tx=0x7f64f4004730 comp rx=0 tx=0).stop 2024-03-23T09:41:36.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.208+0000 7f6501696640 1 -- 172.21.15.47:0/361772846 shutdown_connections 2024-03-23T09:41:36.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.208+0000 7f6501696640 1 --2- 172.21.15.47:0/361772846 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f64e806ac30 0x7f64e806d0f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.208+0000 7f6501696640 1 --2- 172.21.15.47:0/361772846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64fc1f0a00 0x7f64fc0b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.208+0000 7f6501696640 1 --2- 172.21.15.47:0/361772846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64fc15b020 0x7f64fc1f04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.208+0000 7f6501696640 1 --2- 172.21.15.47:0/361772846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e4003970 0x7f64fc1ed8f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.208+0000 7f6501696640 1 -- 172.21.15.47:0/361772846 >> 172.21.15.47:0/361772846 conn(0x7f64fc0d0670 msgr2=0x7f64fc06e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:36.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.209+0000 7f6501696640 1 -- 172.21.15.47:0/361772846 shutdown_connections 2024-03-23T09:41:36.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.209+0000 7f6501696640 1 -- 172.21.15.47:0/361772846 wait complete. 2024-03-23T09:41:36.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:41:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.322+0000 7ff66cb77640 1 Processor -- start 2024-03-23T09:41:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.322+0000 7ff66cb77640 1 -- start start 2024-03-23T09:41:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff66cb77640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff668153b80 0x7ff668159c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff66cb77640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6680b0a00 con 0x7ff6680b38b0 2024-03-23T09:41:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff66cb77640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff66813ce20 con 0x7ff668153f60 2024-03-23T09:41:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff66cb77640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66815a2e0 con 0x7ff668153b80 2024-03-23T09:41:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff666d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff6680b38b0 0x7ff66813d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51070/0 (socket says 172.21.15.47:51070) 2024-03-23T09:41:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff666d9d640 1 -- 172.21.15.47:0/3846001058 learned_addr learned my addr 172.21.15.47:0/3846001058 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52949175 0 0) 0x7ff6680b0a00 con 0x7ff6680b38b0 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff654003520 con 0x7ff6680b38b0 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff66659c640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff668153b80 0x7ff668159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1796370730 0 0) 0x7ff654003520 con 0x7ff6680b38b0 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff668153b80 msgr2=0x7ff668159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff665d9b640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff668153b80 0x7ff668159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 >> v1:172.21.15.175:6789/0 conn(0x7ff668153f60 legacy=0x7ff6680b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66815a5c0 con 0x7ff6680b38b0 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7ff6600019f0 con 0x7ff6680b38b0 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff66659c640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff668153b80 0x7ff668159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 >> v1:172.21.15.47:6789/0 conn(0x7ff6680b38b0 legacy=0x7ff66813d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff665d9b640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6540039a0 0x7ff654003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.323+0000 7ff665d9b640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff668153b80 0x7ff6540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff665d9b640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff654004d00 0x7ff654005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66815a2e0 con 0x7ff6540039a0 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65400bb10 con 0x7ff654004d00 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65400bcd0 con 0x7ff668153b80 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff66659c640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6540039a0 0x7ff654003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff65ffff640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff668153b80 0x7ff6540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff666d9d640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff654004d00 0x7ff654005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff66659c640 1 -- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff668153b80 msgr2=0x7ff6540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff66659c640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff668153b80 0x7ff6540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff66659c640 1 -- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff654004d00 msgr2=0x7ff654005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff66659c640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff654004d00 0x7ff654005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff66659c640 1 -- 172.21.15.47:0/3846001058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff654003520 con 0x7ff6540039a0 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff666d9d640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff654004d00 0x7ff654005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.324+0000 7ff65ffff640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff668153b80 0x7ff6540047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.325+0000 7ff66659c640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6540039a0 0x7ff654003dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff6580098a0 tx=0x7ff6540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca83a985e9d1f2f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.325+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658004230 con 0x7ff6540039a0 2024-03-23T09:41:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.325+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff658004390 con 0x7ff6540039a0 2024-03-23T09:41:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.325+0000 7ff665d9b640 1 -- 172.21.15.47:0/3846001058 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658003830 con 0x7ff6540039a0 2024-03-23T09:41:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.325+0000 7ff66cb77640 1 -- 172.21.15.47:0/3846001058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6540039a0 msgr2=0x7ff654003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.325+0000 7ff66cb77640 1 --2- 172.21.15.47:0/3846001058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6540039a0 0x7ff654003dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff6580098a0 tx=0x7ff6540057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.325+0000 7ff666d9d640 1 -- 172.21.15.47:0/3846001058 reap_dead start 2024-03-23T09:41:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.325+0000 7ff66cb77640 1 -- 172.21.15.47:0/3846001058 shutdown_connections 2024-03-23T09:41:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.325+0000 7ff66cb77640 1 -- 172.21.15.47:0/3846001058 >> 172.21.15.47:0/3846001058 conn(0x7ff668066000 msgr2=0x7ff668069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.325+0000 7ff66cb77640 1 -- 172.21.15.47:0/3846001058 shutdown_connections 2024-03-23T09:41:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.325+0000 7ff66cb77640 1 -- 172.21.15.47:0/3846001058 wait complete. 2024-03-23T09:41:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.327+0000 7ff66cb77640 1 Processor -- start 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.327+0000 7ff66cb77640 1 -- start start 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.327+0000 7ff66cb77640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6681f1d50 0x7ff6680c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.327+0000 7ff66cb77640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6680c8490 0x7ff6680c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.327+0000 7ff66cb77640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6680c5b50 0x7ff6680c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff66cb77640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680c74e0 con 0x7ff6680c5b50 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff66cb77640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680c8e50 con 0x7ff6680c8490 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff66cb77640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680c6860 con 0x7ff6681f1d50 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff666d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6680c5b50 0x7ff6680c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff666d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6680c5b50 0x7ff6680c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50102/0 (socket says 172.21.15.47:50102) 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff666d9d640 1 -- 172.21.15.47:0/3371779946 learned_addr learned my addr 172.21.15.47:0/3371779946 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff65ffff640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6680c8490 0x7ff6680c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff66659c640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6681f1d50 0x7ff6680c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff666d9d640 1 -- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6681f1d50 msgr2=0x7ff6680c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff666d9d640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6681f1d50 0x7ff6680c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff666d9d640 1 -- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6680c8490 msgr2=0x7ff6680c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff666d9d640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6680c8490 0x7ff6680c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff666d9d640 1 -- 172.21.15.47:0/3371779946 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6680c6b40 con 0x7ff6680c5b50 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff66659c640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6681f1d50 0x7ff6680c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff666d9d640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6680c5b50 0x7ff6680c6000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff6680ce150 tx=0x7ff66000e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff65effd640 1 -- 172.21.15.47:0/3371779946 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66000eaf0 con 0x7ff6680c5b50 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff65effd640 1 -- 172.21.15.47:0/3371779946 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66000ec70 con 0x7ff6680c5b50 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff65effd640 1 -- 172.21.15.47:0/3371779946 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660013420 con 0x7ff6680c5b50 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff66cb77640 1 -- 172.21.15.47:0/3371779946 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6680c6e20 con 0x7ff6680c5b50 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.328+0000 7ff66cb77640 1 -- 172.21.15.47:0/3371779946 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6680c6540 con 0x7ff6680c5b50 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.329+0000 7ff66cb77640 1 -- 172.21.15.47:0/3371779946 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6680c7620 con 0x7ff6680c5b50 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.330+0000 7ff65effd640 1 -- 172.21.15.47:0/3371779946 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff660013580 con 0x7ff6680c5b50 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.330+0000 7ff65effd640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff64806ac00 0x7ff64806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.330+0000 7ff65effd640 1 -- 172.21.15.47:0/3371779946 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6600895c0 con 0x7ff6680c5b50 2024-03-23T09:41:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.330+0000 7ff66659c640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff64806ac00 0x7ff64806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.331+0000 7ff66659c640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff64806ac00 0x7ff64806d0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff658035000 tx=0x7ff65803a040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.339+0000 7ff65effd640 1 -- 172.21.15.47:0/3371779946 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff6680c7620 con 0x7ff6680c5b50 2024-03-23T09:41:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.532+0000 7ff66cb77640 1 -- 172.21.15.47:0/3371779946 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff6680c7620 con 0x7ff6680c5b50 2024-03-23T09:41:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.532+0000 7ff65effd640 1 -- 172.21.15.47:0/3371779946 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff6680c7620 con 0x7ff6680c5b50 2024-03-23T09:41:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:41:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.534+0000 7ff66cb77640 1 -- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff64806ac00 msgr2=0x7ff64806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.534+0000 7ff66cb77640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff64806ac00 0x7ff64806d0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff658035000 tx=0x7ff65803a040 comp rx=0 tx=0).stop 2024-03-23T09:41:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.534+0000 7ff66cb77640 1 -- 172.21.15.47:0/3371779946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6680c5b50 msgr2=0x7ff6680c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.534+0000 7ff66cb77640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6680c5b50 0x7ff6680c6000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff6680ce150 tx=0x7ff66000e160 comp rx=0 tx=0).stop 2024-03-23T09:41:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.534+0000 7ff66cb77640 1 -- 172.21.15.47:0/3371779946 shutdown_connections 2024-03-23T09:41:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.534+0000 7ff66cb77640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff64806ac00 0x7ff64806d0c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.534+0000 7ff66cb77640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6680c5b50 0x7ff6680c6000 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.534+0000 7ff66cb77640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6680c8490 0x7ff6680c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.534+0000 7ff66cb77640 1 --2- 172.21.15.47:0/3371779946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6681f1d50 0x7ff6680c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.534+0000 7ff66cb77640 1 -- 172.21.15.47:0/3371779946 >> 172.21.15.47:0/3371779946 conn(0x7ff668066000 msgr2=0x7ff668071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.534+0000 7ff66cb77640 1 -- 172.21.15.47:0/3371779946 shutdown_connections 2024-03-23T09:41:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.535+0000 7ff66cb77640 1 -- 172.21.15.47:0/3371779946 wait complete. 2024-03-23T09:41:36.546 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:41:36.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:36.546 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:41:36.574 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:41:36.574 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:41:36.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:36.574 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:41:36.640 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:41:36.641 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:41:36.659 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:41:36.659 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:41:36.718 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:36.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.820+0000 7f5316e8b640 1 Processor -- start 2024-03-23T09:41:36.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.820+0000 7f5316e8b640 1 -- start start 2024-03-23T09:41:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.821+0000 7f5316e8b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f53100cd7f0 0x7f53100ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.821+0000 7f5316e8b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531015ae50 con 0x7f53100ce800 2024-03-23T09:41:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.821+0000 7f5316e8b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53100cf010 con 0x7f531015aa70 2024-03-23T09:41:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.821+0000 7f5316e8b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53100cf2f0 con 0x7f53100cd7f0 2024-03-23T09:41:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.821+0000 7f5315428640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f53100ce800 0x7f53100cbb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51072/0 (socket says 172.21.15.47:51072) 2024-03-23T09:41:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.821+0000 7f5315428640 1 -- 172.21.15.47:0/844139201 learned_addr learned my addr 172.21.15.47:0/844139201 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.821+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959959543 0 0) 0x7f531015ae50 con 0x7f53100ce800 2024-03-23T09:41:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5300003520 con 0x7f53100ce800 2024-03-23T09:41:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530ffff640 1 --2- 172.21.15.47:0/844139201 >> v2:172.21.15.175:3300/0 conn(0x7f53100cd7f0 0x7f53100ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3903474600 0 0) 0x7f5300003520 con 0x7f53100ce800 2024-03-23T09:41:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 >> v2:172.21.15.175:3300/0 conn(0x7f53100cd7f0 msgr2=0x7f53100ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 --2- 172.21.15.47:0/844139201 >> v2:172.21.15.175:3300/0 conn(0x7f53100cd7f0 0x7f53100ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 >> v1:172.21.15.175:6789/0 conn(0x7f531015aa70 legacy=0x7f53100cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53100cf5d0 con 0x7f53100ce800 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f53080019f0 con 0x7f53100ce800 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 >> v1:172.21.15.47:6789/0 conn(0x7f53100ce800 legacy=0x7f53100cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 --2- 172.21.15.47:0/844139201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5300003970 0x7f5300003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 --2- 172.21.15.47:0/844139201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53000047c0 0x7f5300004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 --2- 172.21.15.47:0/844139201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5300005180 0x7f530000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53100cf2f0 con 0x7f53000047c0 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530000c030 con 0x7f5300005180 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530000c1f0 con 0x7f5300003970 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530ffff640 1 --2- 172.21.15.47:0/844139201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53000047c0 0x7f5300004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f5315428640 1 --2- 172.21.15.47:0/844139201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5300005180 0x7f530000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f5314c27640 1 --2- 172.21.15.47:0/844139201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5300003970 0x7f5300003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530ffff640 1 -- 172.21.15.47:0/844139201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5300003970 msgr2=0x7f5300003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530ffff640 1 --2- 172.21.15.47:0/844139201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5300003970 0x7f5300003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530ffff640 1 -- 172.21.15.47:0/844139201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5300005180 msgr2=0x7f530000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530ffff640 1 --2- 172.21.15.47:0/844139201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5300005180 0x7f530000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f530ffff640 1 -- 172.21.15.47:0/844139201 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5300003520 con 0x7f53000047c0 2024-03-23T09:41:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.822+0000 7f5315428640 1 -- 172.21.15.47:0/844139201 reap_dead start 2024-03-23T09:41:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.823+0000 7f530ffff640 1 --2- 172.21.15.47:0/844139201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53000047c0 0x7f5300004c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009ac0 tx=0x7f53000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3567e09497f3a281 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.823+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc02c6b0 con 0x7f53000047c0 2024-03-23T09:41:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.823+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc02c810 con 0x7f53000047c0 2024-03-23T09:41:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.823+0000 7f530f7fe640 1 -- 172.21.15.47:0/844139201 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc02cba0 con 0x7f53000047c0 2024-03-23T09:41:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.824+0000 7f5316e8b640 1 -- 172.21.15.47:0/844139201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53000047c0 msgr2=0x7f5300004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.824+0000 7f5316e8b640 1 --2- 172.21.15.47:0/844139201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53000047c0 0x7f5300004c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009ac0 tx=0x7f53000057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.824+0000 7f5316e8b640 1 -- 172.21.15.47:0/844139201 shutdown_connections 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.824+0000 7f5316e8b640 1 --2- 172.21.15.47:0/844139201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53000047c0 0x7f5300004c40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.824+0000 7f5316e8b640 1 -- 172.21.15.47:0/844139201 >> 172.21.15.47:0/844139201 conn(0x7f53100d06f0 msgr2=0x7f53100d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.824+0000 7f5316e8b640 1 -- 172.21.15.47:0/844139201 shutdown_connections 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.824+0000 7f5316e8b640 1 -- 172.21.15.47:0/844139201 wait complete. 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.825+0000 7f5316e8b640 1 Processor -- start 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.825+0000 7f5316e8b640 1 -- start start 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5316e8b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53000047c0 0x7f53100b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5316e8b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f531014cce0 0x7f53100b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5316e8b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53100cbfa0 0x7f53100cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5316e8b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53100cc960 con 0x7f53100cbfa0 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5316e8b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531014d4e0 con 0x7f53000047c0 2024-03-23T09:41:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5316e8b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531014d7c0 con 0x7f531014cce0 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5315428640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53100cbfa0 0x7f53100cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5315428640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53100cbfa0 0x7f53100cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50114/0 (socket says 172.21.15.47:50114) 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5315428640 1 -- 172.21.15.47:0/4104877218 learned_addr learned my addr 172.21.15.47:0/4104877218 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5314c27640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53000047c0 0x7f53100b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f530ffff640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f531014cce0 0x7f53100b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5315428640 1 -- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f531014cce0 msgr2=0x7f53100b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5315428640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f531014cce0 0x7f53100b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5315428640 1 -- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53000047c0 msgr2=0x7f53100b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5315428640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53000047c0 0x7f53100b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.826+0000 7f5315428640 1 -- 172.21.15.47:0/4104877218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531014daa0 con 0x7f53100cbfa0 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.827+0000 7f5315428640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53100cbfa0 0x7f53100cc420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f531006a740 tx=0x7f530800a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.827+0000 7f530d7fa640 1 -- 172.21.15.47:0/4104877218 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308001ec0 con 0x7f53100cbfa0 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.827+0000 7f530d7fa640 1 -- 172.21.15.47:0/4104877218 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530800ac60 con 0x7f53100cbfa0 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.827+0000 7f530d7fa640 1 -- 172.21.15.47:0/4104877218 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53080133f0 con 0x7f53100cbfa0 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.827+0000 7f5316e8b640 1 -- 172.21.15.47:0/4104877218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531014dd80 con 0x7f53100cbfa0 2024-03-23T09:41:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.827+0000 7f5316e8b640 1 -- 172.21.15.47:0/4104877218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531014d1b0 con 0x7f53100cbfa0 2024-03-23T09:41:36.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.828+0000 7f5316e8b640 1 -- 172.21.15.47:0/4104877218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5310066be0 con 0x7f53100cbfa0 2024-03-23T09:41:36.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.828+0000 7f530d7fa640 1 -- 172.21.15.47:0/4104877218 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5308013550 con 0x7f53100cbfa0 2024-03-23T09:41:36.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.829+0000 7f530d7fa640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f52f406ac30 0x7f52f406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:36.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.829+0000 7f530d7fa640 1 -- 172.21.15.47:0/4104877218 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5308089960 con 0x7f53100cbfa0 2024-03-23T09:41:36.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.829+0000 7f5314c27640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f52f406ac30 0x7f52f406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:36.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.830+0000 7f5314c27640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f52f406ac30 0x7f52f406d0f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5304000c00 tx=0x7f5304009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:36.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:36.837+0000 7f530d7fa640 1 -- 172.21.15.47:0/4104877218 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5308054bd0 con 0x7f53100cbfa0 2024-03-23T09:41:36.940 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:41:36.942 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:41:37.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.019+0000 7f5316e8b640 1 -- 172.21.15.47:0/4104877218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53100c6e80 con 0x7f53100cbfa0 2024-03-23T09:41:37.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.020+0000 7f530d7fa640 1 -- 172.21.15.47:0/4104877218 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f53080581f0 con 0x7f53100cbfa0 2024-03-23T09:41:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:37.020 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:37.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.022+0000 7f5316e8b640 1 -- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f52f406ac30 msgr2=0x7f52f406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.022+0000 7f5316e8b640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f52f406ac30 0x7f52f406d0f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5304000c00 tx=0x7f5304009040 comp rx=0 tx=0).stop 2024-03-23T09:41:37.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.022+0000 7f5316e8b640 1 -- 172.21.15.47:0/4104877218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53100cbfa0 msgr2=0x7f53100cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.022+0000 7f5316e8b640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53100cbfa0 0x7f53100cc420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f531006a740 tx=0x7f530800a6e0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.022+0000 7f5316e8b640 1 -- 172.21.15.47:0/4104877218 shutdown_connections 2024-03-23T09:41:37.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.022+0000 7f5316e8b640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f52f406ac30 0x7f52f406d0f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.022+0000 7f5316e8b640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53100cbfa0 0x7f53100cc420 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.022+0000 7f5316e8b640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f531014cce0 0x7f53100b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.022+0000 7f5316e8b640 1 --2- 172.21.15.47:0/4104877218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53000047c0 0x7f53100b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.022+0000 7f5316e8b640 1 -- 172.21.15.47:0/4104877218 >> 172.21.15.47:0/4104877218 conn(0x7f53100d06f0 msgr2=0x7f531006b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:37.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.022+0000 7f5316e8b640 1 -- 172.21.15.47:0/4104877218 shutdown_connections 2024-03-23T09:41:37.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.022+0000 7f5316e8b640 1 -- 172.21.15.47:0/4104877218 wait complete. 2024-03-23T09:41:37.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:37.033 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.176+0000 7f17bd1c9640 1 Processor -- start 2024-03-23T09:41:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.176+0000 7f17bd1c9640 1 -- start start 2024-03-23T09:41:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.177+0000 7f17bd1c9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b815c990 0x7f17b8160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.177+0000 7f17bd1c9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f17b80b8350 0x7f17b80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.177+0000 7f17bd1c9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b8161630 con 0x7f17b80cdc40 2024-03-23T09:41:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.177+0000 7f17bd1c9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b80ce0b0 con 0x7f17b80b8350 2024-03-23T09:41:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.177+0000 7f17bd1c9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b80d1030 con 0x7f17b815c990 2024-03-23T09:41:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.178+0000 7f17b6d9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b815c990 0x7f17b8160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.178+0000 7f17affff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f17b80b8350 0x7f17b80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.178+0000 7f17b6d9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b815c990 0x7f17b8160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:59328/0 (socket says 172.21.15.47:59328) 2024-03-23T09:41:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.178+0000 7f17b759e640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f17b80cdc40 0x7f17b80b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:60988/0 (socket says 172.21.15.47:60988) 2024-03-23T09:41:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.178+0000 7f17b6d9d640 1 -- 172.21.15.47:0/3234622566 learned_addr learned my addr 172.21.15.47:0/3234622566 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.178+0000 7f17b6d9d640 1 -- 172.21.15.47:0/3234622566 >> v2:172.21.15.175:3300/0 conn(0x7f17b80b8350 msgr2=0x7f17b80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:37.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.178+0000 7f17b6d9d640 1 --2- 172.21.15.47:0/3234622566 >> v2:172.21.15.175:3300/0 conn(0x7f17b80b8350 0x7f17b80cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.178+0000 7f17b6d9d640 1 -- 172.21.15.47:0/3234622566 >> v1:172.21.15.175:6789/0 conn(0x7f17b80cdc40 legacy=0x7f17b80b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.178+0000 7f17b6d9d640 1 -- 172.21.15.47:0/3234622566 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b80cfe10 con 0x7f17b815c990 2024-03-23T09:41:37.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.178+0000 7f17affff640 1 --2- 172.21.15.47:0/3234622566 >> v2:172.21.15.175:3300/0 conn(0x7f17b80b8350 0x7f17b80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:37.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.179+0000 7f17b6d9d640 1 --2- 172.21.15.47:0/3234622566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b815c990 0x7f17b8160690 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f17a8009870 tx=0x7f17a802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a311b755366780 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:37.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.179+0000 7f17b659c640 1 -- 172.21.15.47:0/3234622566 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8031640 con 0x7f17b815c990 2024-03-23T09:41:37.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.179+0000 7f17b659c640 1 -- 172.21.15.47:0/3234622566 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a80317a0 con 0x7f17b815c990 2024-03-23T09:41:37.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.179+0000 7f17b659c640 1 -- 172.21.15.47:0/3234622566 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8031b30 con 0x7f17b815c990 2024-03-23T09:41:37.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.180+0000 7f17bd1c9640 1 -- 172.21.15.47:0/3234622566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b815c990 msgr2=0x7f17b8160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.180+0000 7f17bd1c9640 1 --2- 172.21.15.47:0/3234622566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b815c990 0x7f17b8160690 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f17a8009870 tx=0x7f17a802b8f0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.180+0000 7f17bd1c9640 1 -- 172.21.15.47:0/3234622566 shutdown_connections 2024-03-23T09:41:37.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.180+0000 7f17bd1c9640 1 --2- 172.21.15.47:0/3234622566 >> v2:172.21.15.175:3300/0 conn(0x7f17b80b8350 0x7f17b80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.180+0000 7f17bd1c9640 1 --2- 172.21.15.47:0/3234622566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b815c990 0x7f17b8160690 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.180+0000 7f17bd1c9640 1 -- 172.21.15.47:0/3234622566 >> 172.21.15.47:0/3234622566 conn(0x7f17b80659d0 msgr2=0x7f17b8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:37.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.180+0000 7f17bd1c9640 1 -- 172.21.15.47:0/3234622566 shutdown_connections 2024-03-23T09:41:37.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.180+0000 7f17bd1c9640 1 -- 172.21.15.47:0/3234622566 wait complete. 2024-03-23T09:41:37.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.181+0000 7f17bd1c9640 1 Processor -- start 2024-03-23T09:41:37.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.181+0000 7f17bd1c9640 1 -- start start 2024-03-23T09:41:37.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17bd1c9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b80b8350 0x7f17b80c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17bd1c9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b80cdc40 0x7f17b80c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17bd1c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17b815c990 0x7f17b8155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17bd1c9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b815ee20 con 0x7f17b815c990 2024-03-23T09:41:37.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17bd1c9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b815fc80 con 0x7f17b80b8350 2024-03-23T09:41:37.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17bd1c9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b815f070 con 0x7f17b80cdc40 2024-03-23T09:41:37.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17b759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17b815c990 0x7f17b8155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17b759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17b815c990 0x7f17b8155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50128/0 (socket says 172.21.15.47:50128) 2024-03-23T09:41:37.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17b759e640 1 -- 172.21.15.47:0/1059455761 learned_addr learned my addr 172.21.15.47:0/1059455761 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:37.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17b6d9d640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b80b8350 0x7f17b80c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17affff640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b80cdc40 0x7f17b80c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17b759e640 1 -- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b80cdc40 msgr2=0x7f17b80c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17b759e640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b80cdc40 0x7f17b80c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17b759e640 1 -- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b80b8350 msgr2=0x7f17b80c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17b759e640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b80b8350 0x7f17b80c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17b759e640 1 -- 172.21.15.47:0/1059455761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a0009400 con 0x7f17b815c990 2024-03-23T09:41:37.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17affff640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b80cdc40 0x7f17b80c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:37.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.182+0000 7f17b759e640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17b815c990 0x7f17b8155dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f17b000c7b0 tx=0x7f17b000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:37.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.183+0000 7f17af7fe640 1 -- 172.21.15.47:0/1059455761 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000a640 con 0x7f17b815c990 2024-03-23T09:41:37.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.183+0000 7f17af7fe640 1 -- 172.21.15.47:0/1059455761 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b000a7a0 con 0x7f17b815c990 2024-03-23T09:41:37.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.183+0000 7f17af7fe640 1 -- 172.21.15.47:0/1059455761 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000ab10 con 0x7f17b815c990 2024-03-23T09:41:37.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.183+0000 7f17bd1c9640 1 -- 172.21.15.47:0/1059455761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a80094d0 con 0x7f17b815c990 2024-03-23T09:41:37.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.183+0000 7f17bd1c9640 1 -- 172.21.15.47:0/1059455761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b815f630 con 0x7f17b815c990 2024-03-23T09:41:37.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.184+0000 7f17bd1c9640 1 -- 172.21.15.47:0/1059455761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b8072360 con 0x7f17b815c990 2024-03-23T09:41:37.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.185+0000 7f17af7fe640 1 -- 172.21.15.47:0/1059455761 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f17b000ac70 con 0x7f17b815c990 2024-03-23T09:41:37.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.185+0000 7f17af7fe640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f179806ac00 0x7f179806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.185+0000 7f17af7fe640 1 -- 172.21.15.47:0/1059455761 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17b0088a90 con 0x7f17b815c990 2024-03-23T09:41:37.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.185+0000 7f17b6d9d640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f179806ac00 0x7f179806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.186+0000 7f17b6d9d640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f179806ac00 0x7f179806d0c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f17a8036000 tx=0x7f17a803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:37.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.189+0000 7f17af7fe640 1 -- 172.21.15.47:0/1059455761 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f17b0053d30 con 0x7f17b815c990 2024-03-23T09:41:37.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.369+0000 7f17bd1c9640 1 -- 172.21.15.47:0/1059455761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17b814e7c0 con 0x7f17b815c990 2024-03-23T09:41:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.369+0000 7f17af7fe640 1 -- 172.21.15.47:0/1059455761 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17b0057350 con 0x7f17b815c990 2024-03-23T09:41:37.369 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.371+0000 7f17bd1c9640 1 -- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f179806ac00 msgr2=0x7f179806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.371+0000 7f17bd1c9640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f179806ac00 0x7f179806d0c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f17a8036000 tx=0x7f17a803c040 comp rx=0 tx=0).stop 2024-03-23T09:41:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.371+0000 7f17bd1c9640 1 -- 172.21.15.47:0/1059455761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17b815c990 msgr2=0x7f17b8155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.371+0000 7f17bd1c9640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17b815c990 0x7f17b8155dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f17b000c7b0 tx=0x7f17b000cc70 comp rx=0 tx=0).stop 2024-03-23T09:41:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.371+0000 7f17bd1c9640 1 -- 172.21.15.47:0/1059455761 shutdown_connections 2024-03-23T09:41:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.371+0000 7f17bd1c9640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f179806ac00 0x7f179806d0c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.371+0000 7f17bd1c9640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17b815c990 0x7f17b8155dd0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.371+0000 7f17bd1c9640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b80cdc40 0x7f17b80c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.371+0000 7f17bd1c9640 1 --2- 172.21.15.47:0/1059455761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b80b8350 0x7f17b80c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.371+0000 7f17bd1c9640 1 -- 172.21.15.47:0/1059455761 >> 172.21.15.47:0/1059455761 conn(0x7f17b80659d0 msgr2=0x7f17b814d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.371+0000 7f17bd1c9640 1 -- 172.21.15.47:0/1059455761 shutdown_connections 2024-03-23T09:41:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.371+0000 7f17bd1c9640 1 -- 172.21.15.47:0/1059455761 wait complete. 2024-03-23T09:41:37.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.487+0000 7f03b7937640 1 Processor -- start 2024-03-23T09:41:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.487+0000 7f03b7937640 1 -- start start 2024-03-23T09:41:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03b7937640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b0143010 0x7f03b00b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03b7937640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b00ccd00 con 0x7f03b00b40e0 2024-03-23T09:41:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03b7937640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b00ccfe0 con 0x7f03b0142a90 2024-03-23T09:41:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03b7937640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b00b27c0 con 0x7f03b0143010 2024-03-23T09:41:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03b5ed4640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f03b00b40e0 0x7f03b006c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51074/0 (socket says 172.21.15.47:51074) 2024-03-23T09:41:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03b5ed4640 1 -- 172.21.15.47:0/3238869362 learned_addr learned my addr 172.21.15.47:0/3238869362 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236964595 0 0) 0x7f03b00ccd00 con 0x7f03b00b40e0 2024-03-23T09:41:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03b4ed2640 1 --2- 172.21.15.47:0/3238869362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b0143010 0x7f03b00b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f039c003520 con 0x7f03b00b40e0 2024-03-23T09:41:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026014564 0 0) 0x7f03b00ccfe0 con 0x7f03b0142a90 2024-03-23T09:41:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03b00ccd00 con 0x7f03b0142a90 2024-03-23T09:41:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 310922611 0 0) 0x7f039c003520 con 0x7f03b00b40e0 2024-03-23T09:41:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b0143010 msgr2=0x7f03b00b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03a7fff640 1 --2- 172.21.15.47:0/3238869362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b0143010 0x7f03b00b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 >> v1:172.21.15.175:6789/0 conn(0x7f03b0142a90 legacy=0x7f03b00cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.488+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b00cd2c0 con 0x7f03b00b40e0 2024-03-23T09:41:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f03a8001970 con 0x7f03b00b40e0 2024-03-23T09:41:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03b4ed2640 1 --2- 172.21.15.47:0/3238869362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b0143010 0x7f03b00b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 >> v1:172.21.15.47:6789/0 conn(0x7f03b00b40e0 legacy=0x7f03b006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03a7fff640 1 --2- 172.21.15.47:0/3238869362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c003b30 0x7f039c003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03a7fff640 1 --2- 172.21.15.47:0/3238869362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039c0047c0 0x7f039c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03a7fff640 1 --2- 172.21.15.47:0/3238869362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b0143010 0x7f039c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b00b27c0 con 0x7f039c003b30 2024-03-23T09:41:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c00ba30 con 0x7f039c0047c0 2024-03-23T09:41:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c00bbf0 con 0x7f03b0143010 2024-03-23T09:41:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03b56d3640 1 --2- 172.21.15.47:0/3238869362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c003b30 0x7f039c003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03b5ed4640 1 --2- 172.21.15.47:0/3238869362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b0143010 0x7f039c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03b4ed2640 1 --2- 172.21.15.47:0/3238869362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039c0047c0 0x7f039c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03b56d3640 1 -- 172.21.15.47:0/3238869362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b0143010 msgr2=0x7f039c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03b56d3640 1 --2- 172.21.15.47:0/3238869362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b0143010 0x7f039c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03b56d3640 1 -- 172.21.15.47:0/3238869362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039c0047c0 msgr2=0x7f039c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03b56d3640 1 --2- 172.21.15.47:0/3238869362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039c0047c0 0x7f039c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.489+0000 7f03b56d3640 1 -- 172.21.15.47:0/3238869362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a0009400 con 0x7f039c003b30 2024-03-23T09:41:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.490+0000 7f03b56d3640 1 --2- 172.21.15.47:0/3238869362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c003b30 0x7f039c003f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0018a0 tx=0x7f039c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27ef02bad9b0711 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.490+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac0052d0 con 0x7f039c003b30 2024-03-23T09:41:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.490+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac005430 con 0x7f039c003b30 2024-03-23T09:41:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.490+0000 7f03a7fff640 1 -- 172.21.15.47:0/3238869362 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac003810 con 0x7f039c003b30 2024-03-23T09:41:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.490+0000 7f03b7937640 1 -- 172.21.15.47:0/3238869362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c003b30 msgr2=0x7f039c003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.490+0000 7f03b7937640 1 --2- 172.21.15.47:0/3238869362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c003b30 0x7f039c003f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0018a0 tx=0x7f039c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.490+0000 7f03b5ed4640 1 -- 172.21.15.47:0/3238869362 reap_dead start 2024-03-23T09:41:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.490+0000 7f03b7937640 1 -- 172.21.15.47:0/3238869362 shutdown_connections 2024-03-23T09:41:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.490+0000 7f03b7937640 1 -- 172.21.15.47:0/3238869362 >> 172.21.15.47:0/3238869362 conn(0x7f03b006a310 msgr2=0x7f03b00b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.490+0000 7f03b7937640 1 -- 172.21.15.47:0/3238869362 shutdown_connections 2024-03-23T09:41:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.490+0000 7f03b7937640 1 -- 172.21.15.47:0/3238869362 wait complete. 2024-03-23T09:41:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.492+0000 7f03b7937640 1 Processor -- start 2024-03-23T09:41:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.492+0000 7f03b7937640 1 -- start start 2024-03-23T09:41:37.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.492+0000 7f03b7937640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c003b30 0x7f03b013cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.492+0000 7f03b7937640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03b00a8d00 0x7f03b0070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.492+0000 7f03b7937640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b00aa570 0x7f03b00aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.492+0000 7f03b7937640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b013d510 con 0x7f039c003b30 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.492+0000 7f03b7937640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b013df60 con 0x7f03b00a8d00 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.492+0000 7f03b7937640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b013e240 con 0x7f03b00aa570 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.492+0000 7f03b56d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c003b30 0x7f03b013cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.492+0000 7f03b56d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c003b30 0x7f03b013cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50148/0 (socket says 172.21.15.47:50148) 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.492+0000 7f03b56d3640 1 -- 172.21.15.47:0/1191457743 learned_addr learned my addr 172.21.15.47:0/1191457743 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.493+0000 7f03b56d3640 1 -- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b00aa570 msgr2=0x7f03b00aa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.493+0000 7f03b56d3640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b00aa570 0x7f03b00aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.493+0000 7f03b56d3640 1 -- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03b00a8d00 msgr2=0x7f03b0070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.493+0000 7f03b4ed2640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03b00a8d00 0x7f03b0070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.493+0000 7f03b56d3640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03b00a8d00 0x7f03b0070460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.493+0000 7f03b56d3640 1 -- 172.21.15.47:0/1191457743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b013e520 con 0x7f039c003b30 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.493+0000 7f03b4ed2640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03b00a8d00 0x7f03b0070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.493+0000 7f03b56d3640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c003b30 0x7f03b013cc50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f03ac035040 tx=0x7f03ac0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:37.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.493+0000 7f03a5ffb640 1 -- 172.21.15.47:0/1191457743 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac007450 con 0x7f039c003b30 2024-03-23T09:41:37.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.493+0000 7f03a5ffb640 1 -- 172.21.15.47:0/1191457743 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac0075b0 con 0x7f039c003b30 2024-03-23T09:41:37.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.493+0000 7f03b7937640 1 -- 172.21.15.47:0/1191457743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b013e800 con 0x7f039c003b30 2024-03-23T09:41:37.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.493+0000 7f03a5ffb640 1 -- 172.21.15.47:0/1191457743 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac007720 con 0x7f039c003b30 2024-03-23T09:41:37.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.493+0000 7f03b7937640 1 -- 172.21.15.47:0/1191457743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b00709a0 con 0x7f039c003b30 2024-03-23T09:41:37.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.494+0000 7f03b7937640 1 -- 172.21.15.47:0/1191457743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b0073480 con 0x7f039c003b30 2024-03-23T09:41:37.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.495+0000 7f03a5ffb640 1 -- 172.21.15.47:0/1191457743 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f03ac0037f0 con 0x7f039c003b30 2024-03-23T09:41:37.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.495+0000 7f03a5ffb640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f039406ac00 0x7f039406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.495+0000 7f03a5ffb640 1 -- 172.21.15.47:0/1191457743 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03ac0aaed0 con 0x7f039c003b30 2024-03-23T09:41:37.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.495+0000 7f03b4ed2640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f039406ac00 0x7f039406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.496+0000 7f03b4ed2640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f039406ac00 0x7f039406d0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f03b00cb790 tx=0x7f03a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:37.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.499+0000 7f03a5ffb640 1 -- 172.21.15.47:0/1191457743 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f03ac0761f0 con 0x7f039c003b30 2024-03-23T09:41:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.677+0000 7f03b7937640 1 -- 172.21.15.47:0/1191457743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03b013e380 con 0x7f039c003b30 2024-03-23T09:41:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.677+0000 7f03a5ffb640 1 -- 172.21.15.47:0/1191457743 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03b013e380 con 0x7f039c003b30 2024-03-23T09:41:37.677 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:37.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.679+0000 7f03b7937640 1 -- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f039406ac00 msgr2=0x7f039406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.679+0000 7f03b7937640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f039406ac00 0x7f039406d0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f03b00cb790 tx=0x7f03a0008040 comp rx=0 tx=0).stop 2024-03-23T09:41:37.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.679+0000 7f03b7937640 1 -- 172.21.15.47:0/1191457743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c003b30 msgr2=0x7f03b013cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.679+0000 7f03b7937640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c003b30 0x7f03b013cc50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f03ac035040 tx=0x7f03ac0012d0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.679+0000 7f03b7937640 1 -- 172.21.15.47:0/1191457743 shutdown_connections 2024-03-23T09:41:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.679+0000 7f03b7937640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f039406ac00 0x7f039406d0c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.679+0000 7f03b7937640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b00aa570 0x7f03b00aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.679+0000 7f03b7937640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03b00a8d00 0x7f03b0070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.679+0000 7f03b7937640 1 --2- 172.21.15.47:0/1191457743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f039c003b30 0x7f03b013cc50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.679+0000 7f03b7937640 1 -- 172.21.15.47:0/1191457743 >> 172.21.15.47:0/1191457743 conn(0x7f03b006a310 msgr2=0x7f03b00b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.679+0000 7f03b7937640 1 -- 172.21.15.47:0/1191457743 shutdown_connections 2024-03-23T09:41:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.679+0000 7f03b7937640 1 -- 172.21.15.47:0/1191457743 wait complete. 2024-03-23T09:41:37.689 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.793+0000 7fef4f9a4640 1 Processor -- start 2024-03-23T09:41:37.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.793+0000 7fef4f9a4640 1 -- start start 2024-03-23T09:41:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.794+0000 7fef4f9a4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef480d23a0 0x7fef480d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.794+0000 7fef4f9a4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef480d10a0 con 0x7fef480b8300 2024-03-23T09:41:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.794+0000 7fef4f9a4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef480d1300 con 0x7fef4815cee0 2024-03-23T09:41:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.794+0000 7fef4f9a4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef480d15e0 con 0x7fef480d23a0 2024-03-23T09:41:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.794+0000 7fef4cf3f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef480d23a0 0x7fef480d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.794+0000 7fef4df41640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fef480b8300 0x7fef480b5a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51082/0 (socket says 172.21.15.47:51082) 2024-03-23T09:41:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.794+0000 7fef4df41640 1 -- 172.21.15.47:0/2718851005 learned_addr learned my addr 172.21.15.47:0/2718851005 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.794+0000 7fef4cf3f640 1 -- 172.21.15.47:0/2718851005 >> v1:172.21.15.175:6789/0 conn(0x7fef4815cee0 legacy=0x7fef480d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.794+0000 7fef4cf3f640 1 -- 172.21.15.47:0/2718851005 >> v1:172.21.15.47:6789/0 conn(0x7fef480b8300 legacy=0x7fef480b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.794+0000 7fef4cf3f640 1 -- 172.21.15.47:0/2718851005 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef480d18c0 con 0x7fef480d23a0 2024-03-23T09:41:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.794+0000 7fef3ffff640 1 -- 172.21.15.47:0/2718851005 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841388635 0 0) 0x7fef480d10a0 con 0x7fef480b8300 2024-03-23T09:41:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.795+0000 7fef4cf3f640 1 --2- 172.21.15.47:0/2718851005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef480d23a0 0x7fef480d2870 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef380097a0 tx=0x7fef3802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c6132b6f117b39 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.795+0000 7fef3ffff640 1 -- 172.21.15.47:0/2718851005 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef38031830 con 0x7fef480d23a0 2024-03-23T09:41:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.795+0000 7fef3ffff640 1 -- 172.21.15.47:0/2718851005 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef38031990 con 0x7fef480d23a0 2024-03-23T09:41:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.795+0000 7fef3ffff640 1 -- 172.21.15.47:0/2718851005 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef38031d20 con 0x7fef480d23a0 2024-03-23T09:41:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.796+0000 7fef4f9a4640 1 -- 172.21.15.47:0/2718851005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef480d23a0 msgr2=0x7fef480d2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.796+0000 7fef4f9a4640 1 --2- 172.21.15.47:0/2718851005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef480d23a0 0x7fef480d2870 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef380097a0 tx=0x7fef3802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.796+0000 7fef4f9a4640 1 -- 172.21.15.47:0/2718851005 shutdown_connections 2024-03-23T09:41:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.796+0000 7fef4f9a4640 1 --2- 172.21.15.47:0/2718851005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef480d23a0 0x7fef480d2870 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.796+0000 7fef4f9a4640 1 -- 172.21.15.47:0/2718851005 >> 172.21.15.47:0/2718851005 conn(0x7fef48065df0 msgr2=0x7fef48066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.796+0000 7fef4f9a4640 1 -- 172.21.15.47:0/2718851005 shutdown_connections 2024-03-23T09:41:37.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.796+0000 7fef4f9a4640 1 -- 172.21.15.47:0/2718851005 wait complete. 2024-03-23T09:41:37.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.797+0000 7fef4f9a4640 1 Processor -- start 2024-03-23T09:41:37.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.797+0000 7fef4f9a4640 1 -- start start 2024-03-23T09:41:37.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4f9a4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef480b8300 0x7fef480c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4f9a4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef480d23a0 0x7fef481f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4f9a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef4815cee0 0x7fef4813e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4f9a4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef480c7e50 con 0x7fef480b8300 2024-03-23T09:41:37.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4f9a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef480cc730 con 0x7fef4815cee0 2024-03-23T09:41:37.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4f9a4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef480c8060 con 0x7fef480d23a0 2024-03-23T09:41:37.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4d740640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef480b8300 0x7fef480c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4d740640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef480b8300 0x7fef480c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50152/0 (socket says 172.21.15.47:50152) 2024-03-23T09:41:37.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4d740640 1 -- 172.21.15.47:0/1167435980 learned_addr learned my addr 172.21.15.47:0/1167435980 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:37.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4d740640 1 -- 172.21.15.47:0/1167435980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef480d23a0 msgr2=0x7fef481f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4d740640 1 --2- 172.21.15.47:0/1167435980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef480d23a0 0x7fef481f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4d740640 1 -- 172.21.15.47:0/1167435980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef4815cee0 msgr2=0x7fef4813e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4d740640 1 --2- 172.21.15.47:0/1167435980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef4815cee0 0x7fef4813e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4d740640 1 -- 172.21.15.47:0/1167435980 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef38009400 con 0x7fef480b8300 2024-03-23T09:41:37.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4d740640 1 --2- 172.21.15.47:0/1167435980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef480b8300 0x7fef480c8b40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fef4000b3f0 tx=0x7fef4000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:37.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef3dffb640 1 -- 172.21.15.47:0/1167435980 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40011830 con 0x7fef480b8300 2024-03-23T09:41:37.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef3dffb640 1 -- 172.21.15.47:0/1167435980 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef40011990 con 0x7fef480b8300 2024-03-23T09:41:37.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef4f9a4640 1 -- 172.21.15.47:0/1167435980 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef480c8340 con 0x7fef480b8300 2024-03-23T09:41:37.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.798+0000 7fef3dffb640 1 -- 172.21.15.47:0/1167435980 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40011d40 con 0x7fef480b8300 2024-03-23T09:41:37.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.799+0000 7fef4f9a4640 1 -- 172.21.15.47:0/1167435980 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef480c9080 con 0x7fef480b8300 2024-03-23T09:41:37.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.800+0000 7fef4f9a4640 1 -- 172.21.15.47:0/1167435980 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef48072360 con 0x7fef480b8300 2024-03-23T09:41:37.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.800+0000 7fef3dffb640 1 -- 172.21.15.47:0/1167435980 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fef4001a400 con 0x7fef480b8300 2024-03-23T09:41:37.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.800+0000 7fef3dffb640 1 --2- 172.21.15.47:0/1167435980 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef2c06ac00 0x7fef2c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:37.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.800+0000 7fef3dffb640 1 -- 172.21.15.47:0/1167435980 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef400891a0 con 0x7fef480b8300 2024-03-23T09:41:37.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.800+0000 7fef4cf3f640 1 --2- 172.21.15.47:0/1167435980 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef2c06ac00 0x7fef2c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:37.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.801+0000 7fef4cf3f640 1 --2- 172.21.15.47:0/1167435980 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef2c06ac00 0x7fef2c06d0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fef38036000 tx=0x7fef38030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:37.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.804+0000 7fef3dffb640 1 -- 172.21.15.47:0/1167435980 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fef40054380 con 0x7fef480b8300 2024-03-23T09:41:37.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.985+0000 7fef4f9a4640 1 -- 172.21.15.47:0/1167435980 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fef480c8860 con 0x7fef480b8300 2024-03-23T09:41:37.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.985+0000 7fef3dffb640 1 -- 172.21.15.47:0/1167435980 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fef400579a0 con 0x7fef480b8300 2024-03-23T09:41:37.985 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:37.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.987+0000 7fef4f9a4640 1 -- 172.21.15.47:0/1167435980 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef2c06ac00 msgr2=0x7fef2c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.987+0000 7fef4f9a4640 1 --2- 172.21.15.47:0/1167435980 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef2c06ac00 0x7fef2c06d0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fef38036000 tx=0x7fef38030000 comp rx=0 tx=0).stop 2024-03-23T09:41:37.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.987+0000 7fef4f9a4640 1 -- 172.21.15.47:0/1167435980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef480b8300 msgr2=0x7fef480c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:37.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.987+0000 7fef4f9a4640 1 --2- 172.21.15.47:0/1167435980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef480b8300 0x7fef480c8b40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fef4000b3f0 tx=0x7fef4000b8b0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.987+0000 7fef4f9a4640 1 -- 172.21.15.47:0/1167435980 shutdown_connections 2024-03-23T09:41:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.988+0000 7fef4f9a4640 1 --2- 172.21.15.47:0/1167435980 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef2c06ac00 0x7fef2c06d0c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.988+0000 7fef4f9a4640 1 --2- 172.21.15.47:0/1167435980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef4815cee0 0x7fef4813e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.988+0000 7fef4f9a4640 1 --2- 172.21.15.47:0/1167435980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef480d23a0 0x7fef481f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.988+0000 7fef4f9a4640 1 --2- 172.21.15.47:0/1167435980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef480b8300 0x7fef480c8b40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.988+0000 7fef4f9a4640 1 -- 172.21.15.47:0/1167435980 >> 172.21.15.47:0/1167435980 conn(0x7fef48065df0 msgr2=0x7fef480b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.988+0000 7fef4f9a4640 1 -- 172.21.15.47:0/1167435980 shutdown_connections 2024-03-23T09:41:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:37.988+0000 7fef4f9a4640 1 -- 172.21.15.47:0/1167435980 wait complete. 2024-03-23T09:41:37.998 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:41:38.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.100+0000 7f1538700640 1 Processor -- start 2024-03-23T09:41:38.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.100+0000 7f1538700640 1 -- start start 2024-03-23T09:41:38.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f1538700640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f15300cd6b0 0x7f15300cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f1538700640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15300d09d0 con 0x7f15300d1420 2024-03-23T09:41:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f1538700640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15300b5a80 con 0x7f153015cc40 2024-03-23T09:41:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f1538700640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15300ce080 con 0x7f15300cd6b0 2024-03-23T09:41:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f1536c9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f15300d1420 0x7f15300d1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51096/0 (socket says 172.21.15.47:51096) 2024-03-23T09:41:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f1536c9d640 1 -- 172.21.15.47:0/1632180664 learned_addr learned my addr 172.21.15.47:0/1632180664 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f1535c9b640 1 --2- 172.21.15.47:0/1632180664 >> v2:172.21.15.175:3300/0 conn(0x7f15300cd6b0 0x7f15300cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722862156 0 0) 0x7f15300d09d0 con 0x7f15300d1420 2024-03-23T09:41:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f151c003520 con 0x7f15300d1420 2024-03-23T09:41:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276344680 0 0) 0x7f15300b5a80 con 0x7f153015cc40 2024-03-23T09:41:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15300d09d0 con 0x7f153015cc40 2024-03-23T09:41:38.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3452201111 0 0) 0x7f151c003520 con 0x7f15300d1420 2024-03-23T09:41:38.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 >> v2:172.21.15.175:3300/0 conn(0x7f15300cd6b0 msgr2=0x7f15300cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:38.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f153549a640 1 --2- 172.21.15.47:0/1632180664 >> v2:172.21.15.175:3300/0 conn(0x7f15300cd6b0 0x7f15300cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 >> v1:172.21.15.175:6789/0 conn(0x7f153015cc40 legacy=0x7f1530160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.101+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153015d020 con 0x7f15300d1420 2024-03-23T09:41:38.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f15280019f0 con 0x7f15300d1420 2024-03-23T09:41:38.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 >> v1:172.21.15.47:6789/0 conn(0x7f15300d1420 legacy=0x7f15300d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f153549a640 1 --2- 172.21.15.47:0/1632180664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f151c003b00 0x7f151c003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f153549a640 1 --2- 172.21.15.47:0/1632180664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151c0047c0 0x7f151c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f153549a640 1 --2- 172.21.15.47:0/1632180664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f151c005180 0x7f151c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15300ce080 con 0x7f151c0047c0 2024-03-23T09:41:38.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c00c000 con 0x7f151c003b00 2024-03-23T09:41:38.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c00c1c0 con 0x7f151c005180 2024-03-23T09:41:38.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f1535c9b640 1 --2- 172.21.15.47:0/1632180664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151c0047c0 0x7f151c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f153649c640 1 --2- 172.21.15.47:0/1632180664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f151c003b00 0x7f151c003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f1536c9d640 1 --2- 172.21.15.47:0/1632180664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f151c005180 0x7f151c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f1535c9b640 1 -- 172.21.15.47:0/1632180664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f151c005180 msgr2=0x7f151c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:38.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f1535c9b640 1 --2- 172.21.15.47:0/1632180664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f151c005180 0x7f151c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f1535c9b640 1 -- 172.21.15.47:0/1632180664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f151c003b00 msgr2=0x7f151c003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:38.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f1535c9b640 1 --2- 172.21.15.47:0/1632180664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f151c003b00 0x7f151c003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f1535c9b640 1 -- 172.21.15.47:0/1632180664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c003520 con 0x7f151c0047c0 2024-03-23T09:41:38.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.102+0000 7f1536c9d640 1 -- 172.21.15.47:0/1632180664 reap_dead start 2024-03-23T09:41:38.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.103+0000 7f1535c9b640 1 --2- 172.21.15.47:0/1632180664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151c0047c0 0x7f151c004c40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1520009ac0 tx=0x7f151c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b8f753f0098a44 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:38.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.103+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152002c6b0 con 0x7f151c0047c0 2024-03-23T09:41:38.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.103+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152002c810 con 0x7f151c0047c0 2024-03-23T09:41:38.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.103+0000 7f153549a640 1 -- 172.21.15.47:0/1632180664 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152002cba0 con 0x7f151c0047c0 2024-03-23T09:41:38.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.103+0000 7f1538700640 1 -- 172.21.15.47:0/1632180664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151c0047c0 msgr2=0x7f151c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.103+0000 7f1538700640 1 --2- 172.21.15.47:0/1632180664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151c0047c0 0x7f151c004c40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1520009ac0 tx=0x7f151c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.103+0000 7f1538700640 1 -- 172.21.15.47:0/1632180664 shutdown_connections 2024-03-23T09:41:38.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.103+0000 7f1538700640 1 --2- 172.21.15.47:0/1632180664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151c0047c0 0x7f151c004c40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.103+0000 7f1538700640 1 -- 172.21.15.47:0/1632180664 >> 172.21.15.47:0/1632180664 conn(0x7f153006b160 msgr2=0x7f153006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:38.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.104+0000 7f1538700640 1 -- 172.21.15.47:0/1632180664 shutdown_connections 2024-03-23T09:41:38.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.104+0000 7f1538700640 1 -- 172.21.15.47:0/1632180664 wait complete. 2024-03-23T09:41:38.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.105+0000 7f1538700640 1 Processor -- start 2024-03-23T09:41:38.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.105+0000 7f1538700640 1 -- start start 2024-03-23T09:41:38.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.105+0000 7f1538700640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f151c0047c0 0x7f15300b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.105+0000 7f1538700640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f153015a2d0 0x7f153015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.105+0000 7f1538700640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15301587e0 0x7f1530158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.105+0000 7f1538700640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153015bc50 con 0x7f15301587e0 2024-03-23T09:41:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.105+0000 7f1538700640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153015b040 con 0x7f151c0047c0 2024-03-23T09:41:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.105+0000 7f1538700640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153015b320 con 0x7f153015a2d0 2024-03-23T09:41:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.105+0000 7f1536c9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15301587e0 0x7f1530158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f1536c9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15301587e0 0x7f1530158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50174/0 (socket says 172.21.15.47:50174) 2024-03-23T09:41:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f1536c9d640 1 -- 172.21.15.47:0/2508568932 learned_addr learned my addr 172.21.15.47:0/2508568932 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f1536c9d640 1 -- 172.21.15.47:0/2508568932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f153015a2d0 msgr2=0x7f153015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f1536c9d640 1 --2- 172.21.15.47:0/2508568932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f153015a2d0 0x7f153015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f1536c9d640 1 -- 172.21.15.47:0/2508568932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f151c0047c0 msgr2=0x7f15300b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f153649c640 1 --2- 172.21.15.47:0/2508568932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f151c0047c0 0x7f15300b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f1536c9d640 1 --2- 172.21.15.47:0/2508568932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f151c0047c0 0x7f15300b6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f1536c9d640 1 -- 172.21.15.47:0/2508568932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153015b600 con 0x7f15301587e0 2024-03-23T09:41:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f1536c9d640 1 --2- 172.21.15.47:0/2508568932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15301587e0 0x7f1530158c40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f153006cb40 tx=0x7f152800a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f1526ffd640 1 -- 172.21.15.47:0/2508568932 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528007af0 con 0x7f15301587e0 2024-03-23T09:41:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f1526ffd640 1 -- 172.21.15.47:0/2508568932 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15280133f0 con 0x7f15301587e0 2024-03-23T09:41:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f1538700640 1 -- 172.21.15.47:0/2508568932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153015b8e0 con 0x7f15301587e0 2024-03-23T09:41:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f1526ffd640 1 -- 172.21.15.47:0/2508568932 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528013780 con 0x7f15301587e0 2024-03-23T09:41:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.106+0000 7f1538700640 1 -- 172.21.15.47:0/2508568932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15300b7530 con 0x7f15301587e0 2024-03-23T09:41:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.107+0000 7f1538700640 1 -- 172.21.15.47:0/2508568932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1530072360 con 0x7f15301587e0 2024-03-23T09:41:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.108+0000 7f1526ffd640 1 -- 172.21.15.47:0/2508568932 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f15280138e0 con 0x7f15301587e0 2024-03-23T09:41:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.108+0000 7f1526ffd640 1 --2- 172.21.15.47:0/2508568932 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f151406ac00 0x7f151406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.108+0000 7f1526ffd640 1 -- 172.21.15.47:0/2508568932 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1528089e40 con 0x7f15301587e0 2024-03-23T09:41:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.108+0000 7f153649c640 1 --2- 172.21.15.47:0/2508568932 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f151406ac00 0x7f151406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.109+0000 7f153649c640 1 --2- 172.21.15.47:0/2508568932 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f151406ac00 0x7f151406d0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1530067290 tx=0x7f152c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:38.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.112+0000 7f1526ffd640 1 -- 172.21.15.47:0/2508568932 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1528055010 con 0x7f15301587e0 2024-03-23T09:41:38.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.303+0000 7f1538700640 1 -- 172.21.15.47:0/2508568932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f15300b7720 con 0x7f15301587e0 2024-03-23T09:41:38.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.303+0000 7f1526ffd640 1 -- 172.21.15.47:0/2508568932 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1528017070 con 0x7f15301587e0 2024-03-23T09:41:38.303 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:41:38.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.305+0000 7f1538700640 1 -- 172.21.15.47:0/2508568932 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f151406ac00 msgr2=0x7f151406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.305+0000 7f1538700640 1 --2- 172.21.15.47:0/2508568932 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f151406ac00 0x7f151406d0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1530067290 tx=0x7f152c009040 comp rx=0 tx=0).stop 2024-03-23T09:41:38.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.305+0000 7f1538700640 1 -- 172.21.15.47:0/2508568932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15301587e0 msgr2=0x7f1530158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.305+0000 7f1538700640 1 --2- 172.21.15.47:0/2508568932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15301587e0 0x7f1530158c40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f153006cb40 tx=0x7f152800a6a0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.305+0000 7f1538700640 1 -- 172.21.15.47:0/2508568932 shutdown_connections 2024-03-23T09:41:38.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.305+0000 7f1538700640 1 --2- 172.21.15.47:0/2508568932 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f151406ac00 0x7f151406d0c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.305+0000 7f1538700640 1 --2- 172.21.15.47:0/2508568932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15301587e0 0x7f1530158c40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.305+0000 7f1538700640 1 --2- 172.21.15.47:0/2508568932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f153015a2d0 0x7f153015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.305+0000 7f1538700640 1 --2- 172.21.15.47:0/2508568932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f151c0047c0 0x7f15300b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.305+0000 7f1538700640 1 -- 172.21.15.47:0/2508568932 >> 172.21.15.47:0/2508568932 conn(0x7f153006b160 msgr2=0x7f153015bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:38.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.305+0000 7f1538700640 1 -- 172.21.15.47:0/2508568932 shutdown_connections 2024-03-23T09:41:38.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.305+0000 7f1538700640 1 -- 172.21.15.47:0/2508568932 wait complete. 2024-03-23T09:41:38.315 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:41:38.315 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:41:38.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:38.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.435+0000 7fc474c5c640 1 Processor -- start 2024-03-23T09:41:38.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.435+0000 7fc474c5c640 1 -- start start 2024-03-23T09:41:38.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.436+0000 7fc474c5c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4700b8300 0x7fc4700cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.436+0000 7fc474c5c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc4700cf6d0 0x7fc4700d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.436+0000 7fc474c5c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4700d0ee0 con 0x7fc47015cb40 2024-03-23T09:41:38.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.436+0000 7fc474c5c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4700b5a10 con 0x7fc4700b8300 2024-03-23T09:41:38.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.436+0000 7fc474c5c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4700cfb60 con 0x7fc4700cf6d0 2024-03-23T09:41:38.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.436+0000 7fc46dd9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4700b8300 0x7fc4700cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.436+0000 7fc46e59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc47015cb40 0x7fc470160770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51098/0 (socket says 172.21.15.47:51098) 2024-03-23T09:41:38.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.436+0000 7fc46dd9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4700b8300 0x7fc4700cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50186/0 (socket says 172.21.15.47:50186) 2024-03-23T09:41:38.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.436+0000 7fc46e59c640 1 -- 172.21.15.47:0/1874360976 learned_addr learned my addr 172.21.15.47:0/1874360976 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:38.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.436+0000 7fc46ed9d640 1 --2- 172.21.15.47:0/1874360976 >> v2:172.21.15.175:3300/0 conn(0x7fc4700cf6d0 0x7fc4700d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.436+0000 7fc46dd9b640 1 -- 172.21.15.47:0/1874360976 >> v2:172.21.15.175:3300/0 conn(0x7fc4700cf6d0 msgr2=0x7fc4700d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:38.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.436+0000 7fc46dd9b640 1 --2- 172.21.15.47:0/1874360976 >> v2:172.21.15.175:3300/0 conn(0x7fc4700cf6d0 0x7fc4700d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.436+0000 7fc46dd9b640 1 -- 172.21.15.47:0/1874360976 >> v1:172.21.15.47:6789/0 conn(0x7fc47015cb40 legacy=0x7fc470160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.437+0000 7fc46dd9b640 1 -- 172.21.15.47:0/1874360976 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4701616f0 con 0x7fc4700b8300 2024-03-23T09:41:38.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.437+0000 7fc46d59a640 1 -- 172.21.15.47:0/1874360976 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864939577 0 0) 0x7fc4700d0ee0 con 0x7fc47015cb40 2024-03-23T09:41:38.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.437+0000 7fc46dd9b640 1 --2- 172.21.15.47:0/1874360976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4700b8300 0x7fc4700cf190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc4640097a0 tx=0x7fc46402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762cee74a7044ca3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:38.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.437+0000 7fc46d59a640 1 -- 172.21.15.47:0/1874360976 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc464031830 con 0x7fc4700b8300 2024-03-23T09:41:38.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.437+0000 7fc46d59a640 1 -- 172.21.15.47:0/1874360976 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc464031990 con 0x7fc4700b8300 2024-03-23T09:41:38.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.437+0000 7fc46d59a640 1 -- 172.21.15.47:0/1874360976 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc464031d20 con 0x7fc4700b8300 2024-03-23T09:41:38.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.438+0000 7fc474c5c640 1 -- 172.21.15.47:0/1874360976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4700b8300 msgr2=0x7fc4700cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.438+0000 7fc474c5c640 1 --2- 172.21.15.47:0/1874360976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4700b8300 0x7fc4700cf190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc4640097a0 tx=0x7fc46402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.438+0000 7fc474c5c640 1 -- 172.21.15.47:0/1874360976 shutdown_connections 2024-03-23T09:41:38.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.438+0000 7fc474c5c640 1 --2- 172.21.15.47:0/1874360976 >> v2:172.21.15.175:3300/0 conn(0x7fc4700cf6d0 0x7fc4700d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.438+0000 7fc474c5c640 1 --2- 172.21.15.47:0/1874360976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4700b8300 0x7fc4700cf190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.438+0000 7fc474c5c640 1 -- 172.21.15.47:0/1874360976 >> 172.21.15.47:0/1874360976 conn(0x7fc47006b160 msgr2=0x7fc47006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:38.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.438+0000 7fc474c5c640 1 -- 172.21.15.47:0/1874360976 shutdown_connections 2024-03-23T09:41:38.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.438+0000 7fc474c5c640 1 -- 172.21.15.47:0/1874360976 wait complete. 2024-03-23T09:41:38.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.439+0000 7fc474c5c640 1 Processor -- start 2024-03-23T09:41:38.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.439+0000 7fc474c5c640 1 -- start start 2024-03-23T09:41:38.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc474c5c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4700b8300 0x7fc470150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc474c5c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4700cf6d0 0x7fc470151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc474c5c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc47015cb40 0x7fc470153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc474c5c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc470152060 con 0x7fc4700b8300 2024-03-23T09:41:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc474c5c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc470152340 con 0x7fc47015cb40 2024-03-23T09:41:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc474c5c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc470152620 con 0x7fc4700cf6d0 2024-03-23T09:41:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc46e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4700b8300 0x7fc470150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc46e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4700b8300 0x7fc470150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50192/0 (socket says 172.21.15.47:50192) 2024-03-23T09:41:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc46e59c640 1 -- 172.21.15.47:0/767527312 learned_addr learned my addr 172.21.15.47:0/767527312 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc46e59c640 1 -- 172.21.15.47:0/767527312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4700cf6d0 msgr2=0x7fc470151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc46e59c640 1 --2- 172.21.15.47:0/767527312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4700cf6d0 0x7fc470151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc46e59c640 1 -- 172.21.15.47:0/767527312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc47015cb40 msgr2=0x7fc470153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc46e59c640 1 --2- 172.21.15.47:0/767527312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc47015cb40 0x7fc470153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.440+0000 7fc46e59c640 1 -- 172.21.15.47:0/767527312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc464009400 con 0x7fc4700b8300 2024-03-23T09:41:38.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.441+0000 7fc46e59c640 1 --2- 172.21.15.47:0/767527312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4700b8300 0x7fc470150910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc46000c7c0 tx=0x7fc46000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:38.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.441+0000 7fc45effd640 1 -- 172.21.15.47:0/767527312 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46000a640 con 0x7fc4700b8300 2024-03-23T09:41:38.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.441+0000 7fc45effd640 1 -- 172.21.15.47:0/767527312 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46000a7c0 con 0x7fc4700b8300 2024-03-23T09:41:38.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.441+0000 7fc474c5c640 1 -- 172.21.15.47:0/767527312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc470152900 con 0x7fc4700b8300 2024-03-23T09:41:38.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.441+0000 7fc45effd640 1 -- 172.21.15.47:0/767527312 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46000ab70 con 0x7fc4700b8300 2024-03-23T09:41:38.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.441+0000 7fc474c5c640 1 -- 172.21.15.47:0/767527312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc470153100 con 0x7fc4700b8300 2024-03-23T09:41:38.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.442+0000 7fc474c5c640 1 -- 172.21.15.47:0/767527312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc470072360 con 0x7fc4700b8300 2024-03-23T09:41:38.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.442+0000 7fc45effd640 1 -- 172.21.15.47:0/767527312 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc46000acd0 con 0x7fc4700b8300 2024-03-23T09:41:38.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.443+0000 7fc45effd640 1 --2- 172.21.15.47:0/767527312 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc45006ac00 0x7fc45006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.443+0000 7fc45effd640 1 -- 172.21.15.47:0/767527312 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc460088be0 con 0x7fc4700b8300 2024-03-23T09:41:38.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.443+0000 7fc46dd9b640 1 --2- 172.21.15.47:0/767527312 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc45006ac00 0x7fc45006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.443+0000 7fc46dd9b640 1 --2- 172.21.15.47:0/767527312 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc45006ac00 0x7fc45006d0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc464036000 tx=0x7fc464030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:38.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.447+0000 7fc45effd640 1 -- 172.21.15.47:0/767527312 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc460053db0 con 0x7fc4700b8300 2024-03-23T09:41:38.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.630+0000 7fc474c5c640 1 -- 172.21.15.47:0/767527312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc470152e20 con 0x7fc4700b8300 2024-03-23T09:41:38.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.631+0000 7fc45effd640 1 -- 172.21.15.47:0/767527312 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fc470152e20 con 0x7fc4700b8300 2024-03-23T09:41:38.631 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:38.631 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:38.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.633+0000 7fc474c5c640 1 -- 172.21.15.47:0/767527312 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc45006ac00 msgr2=0x7fc45006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.633+0000 7fc474c5c640 1 --2- 172.21.15.47:0/767527312 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc45006ac00 0x7fc45006d0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc464036000 tx=0x7fc464030000 comp rx=0 tx=0).stop 2024-03-23T09:41:38.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.633+0000 7fc474c5c640 1 -- 172.21.15.47:0/767527312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4700b8300 msgr2=0x7fc470150910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.633+0000 7fc474c5c640 1 --2- 172.21.15.47:0/767527312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4700b8300 0x7fc470150910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc46000c7c0 tx=0x7fc46000cc80 comp rx=0 tx=0).stop 2024-03-23T09:41:38.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.633+0000 7fc474c5c640 1 -- 172.21.15.47:0/767527312 shutdown_connections 2024-03-23T09:41:38.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.633+0000 7fc474c5c640 1 --2- 172.21.15.47:0/767527312 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc45006ac00 0x7fc45006d0c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.633+0000 7fc474c5c640 1 --2- 172.21.15.47:0/767527312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc47015cb40 0x7fc470153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.633+0000 7fc474c5c640 1 --2- 172.21.15.47:0/767527312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4700cf6d0 0x7fc470151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.633+0000 7fc474c5c640 1 --2- 172.21.15.47:0/767527312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4700b8300 0x7fc470150910 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.633+0000 7fc474c5c640 1 -- 172.21.15.47:0/767527312 >> 172.21.15.47:0/767527312 conn(0x7fc47006b160 msgr2=0x7fc470159090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:38.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.633+0000 7fc474c5c640 1 -- 172.21.15.47:0/767527312 shutdown_connections 2024-03-23T09:41:38.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.633+0000 7fc474c5c640 1 -- 172.21.15.47:0/767527312 wait complete. 2024-03-23T09:41:38.644 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:38.645 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:38.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.748+0000 7fda373d2640 1 Processor -- start 2024-03-23T09:41:38.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.748+0000 7fda373d2640 1 -- start start 2024-03-23T09:41:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.749+0000 7fda373d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda30160de0 0x7fda300c4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.749+0000 7fda373d2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fda300aedd0 0x7fda300cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.749+0000 7fda373d2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda300c54e0 con 0x7fda300c4d60 2024-03-23T09:41:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.749+0000 7fda373d2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda300c5200 con 0x7fda300aedd0 2024-03-23T09:41:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.749+0000 7fda373d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda30164c70 con 0x7fda30160de0 2024-03-23T09:41:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.749+0000 7fda3496d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fda300aedd0 0x7fda300cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.749+0000 7fda3496d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fda300aedd0 0x7fda300cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50198/0 (socket says 172.21.15.47:50198) 2024-03-23T09:41:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.749+0000 7fda3496d640 1 -- 172.21.15.47:0/757057595 learned_addr learned my addr 172.21.15.47:0/757057595 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.749+0000 7fda3596f640 1 --1- 172.21.15.47:0/757057595 >> v1:172.21.15.47:6789/0 conn(0x7fda300c4d60 0x7fda3006eba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51106/0 (socket says 172.21.15.47:51106) 2024-03-23T09:41:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.750+0000 7fda3496d640 1 -- 172.21.15.47:0/757057595 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda30160de0 msgr2=0x7fda300c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.750+0000 7fda3496d640 1 --2- 172.21.15.47:0/757057595 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda30160de0 0x7fda300c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.750+0000 7fda3496d640 1 -- 172.21.15.47:0/757057595 >> v1:172.21.15.47:6789/0 conn(0x7fda300c4d60 legacy=0x7fda3006eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.750+0000 7fda3496d640 1 -- 172.21.15.47:0/757057595 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda30164f50 con 0x7fda300aedd0 2024-03-23T09:41:38.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.750+0000 7fda3496d640 1 --2- 172.21.15.47:0/757057595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda300aedd0 0x7fda300cc410 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda24009870 tx=0x7fda2402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189874e31fe28f25 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:38.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.751+0000 7fda2ffff640 1 -- 172.21.15.47:0/757057595 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24031640 con 0x7fda300aedd0 2024-03-23T09:41:38.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.751+0000 7fda2ffff640 1 -- 172.21.15.47:0/757057595 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda240317a0 con 0x7fda300aedd0 2024-03-23T09:41:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.751+0000 7fda2ffff640 1 -- 172.21.15.47:0/757057595 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24031b30 con 0x7fda300aedd0 2024-03-23T09:41:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.751+0000 7fda373d2640 1 -- 172.21.15.47:0/757057595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda300aedd0 msgr2=0x7fda300cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.751+0000 7fda373d2640 1 --2- 172.21.15.47:0/757057595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda300aedd0 0x7fda300cc410 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda24009870 tx=0x7fda2402b8f0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.751+0000 7fda373d2640 1 -- 172.21.15.47:0/757057595 shutdown_connections 2024-03-23T09:41:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.751+0000 7fda373d2640 1 --2- 172.21.15.47:0/757057595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda300aedd0 0x7fda300cc410 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.751+0000 7fda373d2640 1 --2- 172.21.15.47:0/757057595 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda30160de0 0x7fda300c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.751+0000 7fda373d2640 1 -- 172.21.15.47:0/757057595 >> 172.21.15.47:0/757057595 conn(0x7fda30071400 msgr2=0x7fda30071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.751+0000 7fda373d2640 1 -- 172.21.15.47:0/757057595 shutdown_connections 2024-03-23T09:41:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.751+0000 7fda373d2640 1 -- 172.21.15.47:0/757057595 wait complete. 2024-03-23T09:41:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.752+0000 7fda373d2640 1 Processor -- start 2024-03-23T09:41:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.752+0000 7fda373d2640 1 -- start start 2024-03-23T09:41:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.753+0000 7fda373d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda300aedd0 0x7fda3014a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.753+0000 7fda373d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda300c4d60 0x7fda301614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.753+0000 7fda373d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda30160de0 0x7fda3015f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.753+0000 7fda373d2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda300b2c30 con 0x7fda300c4d60 2024-03-23T09:41:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.753+0000 7fda373d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda30161a00 con 0x7fda30160de0 2024-03-23T09:41:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.753+0000 7fda373d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda30164690 con 0x7fda300aedd0 2024-03-23T09:41:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.753+0000 7fda3496d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda300c4d60 0x7fda301614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.753+0000 7fda3496d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda300c4d60 0x7fda301614c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50208/0 (socket says 172.21.15.47:50208) 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.753+0000 7fda3496d640 1 -- 172.21.15.47:0/2651211716 learned_addr learned my addr 172.21.15.47:0/2651211716 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.753+0000 7fda3596f640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda30160de0 0x7fda3015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.753+0000 7fda3516e640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda300aedd0 0x7fda3014a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.754+0000 7fda3496d640 1 -- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda300aedd0 msgr2=0x7fda3014a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.754+0000 7fda3496d640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda300aedd0 0x7fda3014a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.754+0000 7fda3496d640 1 -- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda30160de0 msgr2=0x7fda3015f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.754+0000 7fda3496d640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda30160de0 0x7fda3015f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.754+0000 7fda3496d640 1 -- 172.21.15.47:0/2651211716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda240094d0 con 0x7fda300c4d60 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.754+0000 7fda3596f640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda30160de0 0x7fda3015f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.754+0000 7fda3496d640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda300c4d60 0x7fda301614c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fda24009870 tx=0x7fda240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.754+0000 7fda2dffb640 1 -- 172.21.15.47:0/2651211716 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2403ac60 con 0x7fda300c4d60 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.754+0000 7fda2dffb640 1 -- 172.21.15.47:0/2651211716 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2403ade0 con 0x7fda300c4d60 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.754+0000 7fda373d2640 1 -- 172.21.15.47:0/2651211716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda300b2950 con 0x7fda300c4d60 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.754+0000 7fda2dffb640 1 -- 172.21.15.47:0/2651211716 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24044630 con 0x7fda300c4d60 2024-03-23T09:41:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.754+0000 7fda373d2640 1 -- 172.21.15.47:0/2651211716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda301643b0 con 0x7fda300c4d60 2024-03-23T09:41:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.755+0000 7fda373d2640 1 -- 172.21.15.47:0/2651211716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3015ee70 con 0x7fda300c4d60 2024-03-23T09:41:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.755+0000 7fda2dffb640 1 -- 172.21.15.47:0/2651211716 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fda2402f050 con 0x7fda300c4d60 2024-03-23T09:41:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.756+0000 7fda2dffb640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda1c06ac00 0x7fda1c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.756+0000 7fda2dffb640 1 -- 172.21.15.47:0/2651211716 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda240ab4d0 con 0x7fda300c4d60 2024-03-23T09:41:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.756+0000 7fda3516e640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda1c06ac00 0x7fda1c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.757+0000 7fda3516e640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda1c06ac00 0x7fda1c06d0c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fda20009f20 tx=0x7fda200094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:38.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.760+0000 7fda2dffb640 1 -- 172.21.15.47:0/2651211716 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fda3015ee70 con 0x7fda300c4d60 2024-03-23T09:41:38.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.939+0000 7fda373d2640 1 -- 172.21.15.47:0/2651211716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda3015ee70 con 0x7fda300c4d60 2024-03-23T09:41:38.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.939+0000 7fda2dffb640 1 -- 172.21.15.47:0/2651211716 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda3015ee70 con 0x7fda300c4d60 2024-03-23T09:41:38.939 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.941+0000 7fda373d2640 1 -- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda1c06ac00 msgr2=0x7fda1c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.941+0000 7fda373d2640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda1c06ac00 0x7fda1c06d0c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fda20009f20 tx=0x7fda200094f0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.941+0000 7fda373d2640 1 -- 172.21.15.47:0/2651211716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda300c4d60 msgr2=0x7fda301614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.941+0000 7fda373d2640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda300c4d60 0x7fda301614c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fda24009870 tx=0x7fda240046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.941+0000 7fda373d2640 1 -- 172.21.15.47:0/2651211716 shutdown_connections 2024-03-23T09:41:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.941+0000 7fda373d2640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda1c06ac00 0x7fda1c06d0c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.941+0000 7fda373d2640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda30160de0 0x7fda3015f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.941+0000 7fda373d2640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda300c4d60 0x7fda301614c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.941+0000 7fda373d2640 1 --2- 172.21.15.47:0/2651211716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda300aedd0 0x7fda3014a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.941+0000 7fda373d2640 1 -- 172.21.15.47:0/2651211716 >> 172.21.15.47:0/2651211716 conn(0x7fda30071400 msgr2=0x7fda300cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.941+0000 7fda373d2640 1 -- 172.21.15.47:0/2651211716 shutdown_connections 2024-03-23T09:41:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:38.941+0000 7fda373d2640 1 -- 172.21.15.47:0/2651211716 wait complete. 2024-03-23T09:41:38.951 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:39.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.055+0000 7fc2df6ce640 1 Processor -- start 2024-03-23T09:41:39.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.055+0000 7fc2df6ce640 1 -- start start 2024-03-23T09:41:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.056+0000 7fc2df6ce640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2d815c990 0x7fc2d8160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.056+0000 7fc2df6ce640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d8161630 con 0x7fc2d80cde10 2024-03-23T09:41:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.056+0000 7fc2df6ce640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d80d1030 con 0x7fc2d80b8350 2024-03-23T09:41:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.056+0000 7fc2df6ce640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d80cfe10 con 0x7fc2d815c990 2024-03-23T09:41:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.056+0000 7fc2dd46a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2d815c990 0x7fc2d8160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.056+0000 7fc2ddc6b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc2d80cde10 0x7fc2d80d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51114/0 (socket says 172.21.15.47:51114) 2024-03-23T09:41:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.056+0000 7fc2ddc6b640 1 -- 172.21.15.47:0/3549294333 learned_addr learned my addr 172.21.15.47:0/3549294333 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.056+0000 7fc2dd46a640 1 -- 172.21.15.47:0/3549294333 >> v1:172.21.15.175:6789/0 conn(0x7fc2d80b8350 legacy=0x7fc2d80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.056+0000 7fc2dd46a640 1 -- 172.21.15.47:0/3549294333 >> v1:172.21.15.47:6789/0 conn(0x7fc2d80cde10 legacy=0x7fc2d80d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.056+0000 7fc2dd46a640 1 -- 172.21.15.47:0/3549294333 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d80b5b60 con 0x7fc2d815c990 2024-03-23T09:41:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.056+0000 7fc2cffff640 1 -- 172.21.15.47:0/3549294333 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472605060 0 0) 0x7fc2d8161630 con 0x7fc2d80cde10 2024-03-23T09:41:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.057+0000 7fc2dd46a640 1 --2- 172.21.15.47:0/3549294333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d815c990 0x7fc2d8160690 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40097a0 tx=0x7fc2d402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b90e879fedbfbf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.057+0000 7fc2cffff640 1 -- 172.21.15.47:0/3549294333 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d402e070 con 0x7fc2d815c990 2024-03-23T09:41:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.057+0000 7fc2cffff640 1 -- 172.21.15.47:0/3549294333 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d40315a0 con 0x7fc2d815c990 2024-03-23T09:41:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.057+0000 7fc2cffff640 1 -- 172.21.15.47:0/3549294333 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4031930 con 0x7fc2d815c990 2024-03-23T09:41:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.057+0000 7fc2df6ce640 1 -- 172.21.15.47:0/3549294333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d815c990 msgr2=0x7fc2d8160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.057+0000 7fc2df6ce640 1 --2- 172.21.15.47:0/3549294333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d815c990 0x7fc2d8160690 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40097a0 tx=0x7fc2d402b840 comp rx=0 tx=0).stop 2024-03-23T09:41:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.057+0000 7fc2df6ce640 1 -- 172.21.15.47:0/3549294333 shutdown_connections 2024-03-23T09:41:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.057+0000 7fc2df6ce640 1 --2- 172.21.15.47:0/3549294333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d815c990 0x7fc2d8160690 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.057+0000 7fc2df6ce640 1 -- 172.21.15.47:0/3549294333 >> 172.21.15.47:0/3549294333 conn(0x7fc2d80659d0 msgr2=0x7fc2d8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.058+0000 7fc2df6ce640 1 -- 172.21.15.47:0/3549294333 shutdown_connections 2024-03-23T09:41:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.058+0000 7fc2df6ce640 1 -- 172.21.15.47:0/3549294333 wait complete. 2024-03-23T09:41:39.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.059+0000 7fc2df6ce640 1 Processor -- start 2024-03-23T09:41:39.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.059+0000 7fc2df6ce640 1 -- start start 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.059+0000 7fc2df6ce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2d80b8350 0x7fc2d8153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.059+0000 7fc2df6ce640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2d80cde10 0x7fc2d80c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.059+0000 7fc2df6ce640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d815c990 0x7fc2d815fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.059+0000 7fc2df6ce640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d80c7280 con 0x7fc2d815c990 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.059+0000 7fc2df6ce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d8154080 con 0x7fc2d80b8350 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.059+0000 7fc2df6ce640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d80c63b0 con 0x7fc2d80cde10 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2ddc6b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d815c990 0x7fc2d815fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2ddc6b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d815c990 0x7fc2d815fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50222/0 (socket says 172.21.15.47:50222) 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2ddc6b640 1 -- 172.21.15.47:0/2354364670 learned_addr learned my addr 172.21.15.47:0/2354364670 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2dd46a640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2d80b8350 0x7fc2d8153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2dcc69640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2d80cde10 0x7fc2d80c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2ddc6b640 1 -- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2d80cde10 msgr2=0x7fc2d80c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2ddc6b640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2d80cde10 0x7fc2d80c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2ddc6b640 1 -- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2d80b8350 msgr2=0x7fc2d8153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2ddc6b640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2d80b8350 0x7fc2d8153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2ddc6b640 1 -- 172.21.15.47:0/2354364670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d4009400 con 0x7fc2d815c990 2024-03-23T09:41:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2dcc69640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2d80cde10 0x7fc2d80c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2dd46a640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2d80b8350 0x7fc2d8153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2ddc6b640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d815c990 0x7fc2d815fce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000c810 tx=0x7fc2d000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2cdffb640 1 -- 172.21.15.47:0/2354364670 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000a5e0 con 0x7fc2d815c990 2024-03-23T09:41:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2cdffb640 1 -- 172.21.15.47:0/2354364670 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d000a760 con 0x7fc2d815c990 2024-03-23T09:41:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2df6ce640 1 -- 172.21.15.47:0/2354364670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d80c91d0 con 0x7fc2d815c990 2024-03-23T09:41:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.060+0000 7fc2cdffb640 1 -- 172.21.15.47:0/2354364670 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000ab10 con 0x7fc2d815c990 2024-03-23T09:41:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.061+0000 7fc2df6ce640 1 -- 172.21.15.47:0/2354364670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d80c68e0 con 0x7fc2d815c990 2024-03-23T09:41:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.062+0000 7fc2df6ce640 1 -- 172.21.15.47:0/2354364670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d8072360 con 0x7fc2d815c990 2024-03-23T09:41:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.062+0000 7fc2cdffb640 1 -- 172.21.15.47:0/2354364670 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc2d000ac70 con 0x7fc2d815c990 2024-03-23T09:41:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.062+0000 7fc2cdffb640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc2bc06ac00 0x7fc2bc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.062+0000 7fc2cdffb640 1 -- 172.21.15.47:0/2354364670 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2d0088770 con 0x7fc2d815c990 2024-03-23T09:41:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.062+0000 7fc2dd46a640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc2bc06ac00 0x7fc2bc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.063+0000 7fc2dd46a640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc2bc06ac00 0x7fc2bc06d0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4036000 tx=0x7fc2d403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.066+0000 7fc2cdffb640 1 -- 172.21.15.47:0/2354364670 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc2d00539e0 con 0x7fc2d815c990 2024-03-23T09:41:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.245+0000 7fc2df6ce640 1 -- 172.21.15.47:0/2354364670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2d80c73c0 con 0x7fc2d815c990 2024-03-23T09:41:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.245+0000 7fc2cdffb640 1 -- 172.21.15.47:0/2354364670 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2d0018020 con 0x7fc2d815c990 2024-03-23T09:41:39.245 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.247+0000 7fc2df6ce640 1 -- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc2bc06ac00 msgr2=0x7fc2bc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.247+0000 7fc2df6ce640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc2bc06ac00 0x7fc2bc06d0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4036000 tx=0x7fc2d403d040 comp rx=0 tx=0).stop 2024-03-23T09:41:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.247+0000 7fc2df6ce640 1 -- 172.21.15.47:0/2354364670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d815c990 msgr2=0x7fc2d815fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.247+0000 7fc2df6ce640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d815c990 0x7fc2d815fce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000c810 tx=0x7fc2d000ccd0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.247+0000 7fc2df6ce640 1 -- 172.21.15.47:0/2354364670 shutdown_connections 2024-03-23T09:41:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.247+0000 7fc2df6ce640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc2bc06ac00 0x7fc2bc06d0c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.247+0000 7fc2df6ce640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d815c990 0x7fc2d815fce0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.247+0000 7fc2df6ce640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2d80cde10 0x7fc2d80c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.247+0000 7fc2df6ce640 1 --2- 172.21.15.47:0/2354364670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2d80b8350 0x7fc2d8153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.247+0000 7fc2df6ce640 1 -- 172.21.15.47:0/2354364670 >> 172.21.15.47:0/2354364670 conn(0x7fc2d80659d0 msgr2=0x7fc2d815bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.247+0000 7fc2df6ce640 1 -- 172.21.15.47:0/2354364670 shutdown_connections 2024-03-23T09:41:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.247+0000 7fc2df6ce640 1 -- 172.21.15.47:0/2354364670 wait complete. 2024-03-23T09:41:39.257 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:39.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.361+0000 7efdc2706640 1 Processor -- start 2024-03-23T09:41:39.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.361+0000 7efdc2706640 1 -- start start 2024-03-23T09:41:39.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdc2706640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efdbc15c990 0x7efdbc160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdc2706640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdbc0cde10 0x7efdbc0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdc2706640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdbc161630 con 0x7efdbc0b8350 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdc2706640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdbc0d1030 con 0x7efdbc15c990 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdc2706640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc0cfe10 con 0x7efdbc0cde10 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdbbfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efdbc15c990 0x7efdbc160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdbb7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efdbc0b8350 0x7efdbc0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51124/0 (socket says 172.21.15.47:51124) 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdbbfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efdbc15c990 0x7efdbc160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50230/0 (socket says 172.21.15.47:50230) 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdbb7fe640 1 -- 172.21.15.47:0/3204772440 learned_addr learned my addr 172.21.15.47:0/3204772440 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdc0ca3640 1 --2- 172.21.15.47:0/3204772440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdbc0cde10 0x7efdbc0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdbbfff640 1 -- 172.21.15.47:0/3204772440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdbc0cde10 msgr2=0x7efdbc0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdbbfff640 1 --2- 172.21.15.47:0/3204772440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdbc0cde10 0x7efdbc0d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdbbfff640 1 -- 172.21.15.47:0/3204772440 >> v1:172.21.15.47:6789/0 conn(0x7efdbc0b8350 legacy=0x7efdbc0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdbbfff640 1 -- 172.21.15.47:0/3204772440 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc0b5b60 con 0x7efdbc15c990 2024-03-23T09:41:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdbaffd640 1 -- 172.21.15.47:0/3204772440 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844430852 0 0) 0x7efdbc161630 con 0x7efdbc0b8350 2024-03-23T09:41:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.362+0000 7efdc0ca3640 1 --2- 172.21.15.47:0/3204772440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdbc0cde10 0x7efdbc0d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.363+0000 7efdbbfff640 1 --2- 172.21.15.47:0/3204772440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdbc15c990 0x7efdbc160690 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efdac0097a0 tx=0x7efdac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8043cb9d44cc817e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.363+0000 7efdbaffd640 1 -- 172.21.15.47:0/3204772440 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac02e070 con 0x7efdbc15c990 2024-03-23T09:41:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.363+0000 7efdbaffd640 1 -- 172.21.15.47:0/3204772440 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdac0315a0 con 0x7efdbc15c990 2024-03-23T09:41:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.363+0000 7efdbaffd640 1 -- 172.21.15.47:0/3204772440 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac031930 con 0x7efdbc15c990 2024-03-23T09:41:39.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.363+0000 7efdc2706640 1 -- 172.21.15.47:0/3204772440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdbc15c990 msgr2=0x7efdbc160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.363+0000 7efdc2706640 1 --2- 172.21.15.47:0/3204772440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdbc15c990 0x7efdbc160690 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efdac0097a0 tx=0x7efdac02b840 comp rx=0 tx=0).stop 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.363+0000 7efdc2706640 1 -- 172.21.15.47:0/3204772440 shutdown_connections 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.363+0000 7efdc2706640 1 --2- 172.21.15.47:0/3204772440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdbc0cde10 0x7efdbc0d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.363+0000 7efdc2706640 1 --2- 172.21.15.47:0/3204772440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdbc15c990 0x7efdbc160690 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.363+0000 7efdc2706640 1 -- 172.21.15.47:0/3204772440 >> 172.21.15.47:0/3204772440 conn(0x7efdbc0659d0 msgr2=0x7efdbc065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.364+0000 7efdc2706640 1 -- 172.21.15.47:0/3204772440 shutdown_connections 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.364+0000 7efdc2706640 1 -- 172.21.15.47:0/3204772440 wait complete. 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.365+0000 7efdc2706640 1 Processor -- start 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.365+0000 7efdc2706640 1 -- start start 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.365+0000 7efdc2706640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdbc0b8350 0x7efdbc153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.365+0000 7efdc2706640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdbc0cde10 0x7efdbc0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.365+0000 7efdc2706640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdbc15c990 0x7efdbc15e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.365+0000 7efdc2706640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc0c7130 con 0x7efdbc0cde10 2024-03-23T09:41:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.365+0000 7efdc2706640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc0c7410 con 0x7efdbc0b8350 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.365+0000 7efdc2706640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc0c76f0 con 0x7efdbc15c990 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdbb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdbc0cde10 0x7efdbc0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdbb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdbc0cde10 0x7efdbc0c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50232/0 (socket says 172.21.15.47:50232) 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdbb7fe640 1 -- 172.21.15.47:0/140749319 learned_addr learned my addr 172.21.15.47:0/140749319 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdc0ca3640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdbc15c990 0x7efdbc15e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdbbfff640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdbc0b8350 0x7efdbc153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdbb7fe640 1 -- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdbc15c990 msgr2=0x7efdbc15e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdbb7fe640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdbc15c990 0x7efdbc15e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdbb7fe640 1 -- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdbc0b8350 msgr2=0x7efdbc153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdbb7fe640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdbc0b8350 0x7efdbc153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdbb7fe640 1 -- 172.21.15.47:0/140749319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac009400 con 0x7efdbc0cde10 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdbb7fe640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdbc0cde10 0x7efdbc0c8120 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c430 tx=0x7efdb000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdb8ff9640 1 -- 172.21.15.47:0/140749319 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb00074e0 con 0x7efdbc0cde10 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdb8ff9640 1 -- 172.21.15.47:0/140749319 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb0007660 con 0x7efdbc0cde10 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdc2706640 1 -- 172.21.15.47:0/140749319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc0c79d0 con 0x7efdbc0cde10 2024-03-23T09:41:39.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdb8ff9640 1 -- 172.21.15.47:0/140749319 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0004a20 con 0x7efdbc0cde10 2024-03-23T09:41:39.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.366+0000 7efdc2706640 1 -- 172.21.15.47:0/140749319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc0c6dc0 con 0x7efdbc0cde10 2024-03-23T09:41:39.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.367+0000 7efdc2706640 1 -- 172.21.15.47:0/140749319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc072360 con 0x7efdbc0cde10 2024-03-23T09:41:39.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.368+0000 7efdb8ff9640 1 -- 172.21.15.47:0/140749319 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efdb0004b80 con 0x7efdbc0cde10 2024-03-23T09:41:39.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.368+0000 7efdb8ff9640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efda806ac00 0x7efda806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.368+0000 7efdb8ff9640 1 -- 172.21.15.47:0/140749319 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efdb00891f0 con 0x7efdbc0cde10 2024-03-23T09:41:39.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.368+0000 7efdbbfff640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efda806ac00 0x7efda806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.369+0000 7efdbbfff640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efda806ac00 0x7efda806d0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efdac036000 tx=0x7efdac03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:39.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.372+0000 7efdb8ff9640 1 -- 172.21.15.47:0/140749319 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efdb00543d0 con 0x7efdbc0cde10 2024-03-23T09:41:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.552+0000 7efdc2706640 1 -- 172.21.15.47:0/140749319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efdbc066be0 con 0x7efdbc0cde10 2024-03-23T09:41:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.552+0000 7efdb8ff9640 1 -- 172.21.15.47:0/140749319 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7efdb00579f0 con 0x7efdbc0cde10 2024-03-23T09:41:39.552 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.554+0000 7efdc2706640 1 -- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efda806ac00 msgr2=0x7efda806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.554+0000 7efdc2706640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efda806ac00 0x7efda806d0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efdac036000 tx=0x7efdac03d040 comp rx=0 tx=0).stop 2024-03-23T09:41:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.554+0000 7efdc2706640 1 -- 172.21.15.47:0/140749319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdbc0cde10 msgr2=0x7efdbc0c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.554+0000 7efdc2706640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdbc0cde10 0x7efdbc0c8120 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c430 tx=0x7efdb000c8f0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.555+0000 7efdc2706640 1 -- 172.21.15.47:0/140749319 shutdown_connections 2024-03-23T09:41:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.555+0000 7efdc2706640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efda806ac00 0x7efda806d0c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.555+0000 7efdc2706640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdbc15c990 0x7efdbc15e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.555+0000 7efdc2706640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdbc0cde10 0x7efdbc0c8120 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.555+0000 7efdc2706640 1 --2- 172.21.15.47:0/140749319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdbc0b8350 0x7efdbc153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.555+0000 7efdc2706640 1 -- 172.21.15.47:0/140749319 >> 172.21.15.47:0/140749319 conn(0x7efdbc0659d0 msgr2=0x7efdbc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.555+0000 7efdc2706640 1 -- 172.21.15.47:0/140749319 shutdown_connections 2024-03-23T09:41:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.555+0000 7efdc2706640 1 -- 172.21.15.47:0/140749319 wait complete. 2024-03-23T09:41:39.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:41:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.667+0000 7f6a12c84640 1 Processor -- start 2024-03-23T09:41:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.667+0000 7f6a12c84640 1 -- start start 2024-03-23T09:41:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a12c84640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a0c159930 0x7f6a0c0ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a12c84640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6a0c0b44a0 0x7f6a0c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a12c84640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a0c0cff50 con 0x7f6a0c0ce660 2024-03-23T09:41:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a12c84640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a0c0cf340 con 0x7f6a0c0b44a0 2024-03-23T09:41:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a12c84640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c0cf620 con 0x7f6a0c159930 2024-03-23T09:41:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a11221640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6a0c0b44a0 0x7f6a0c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a0bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6a0c0ce660 0x7f6a0c0b3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51140/0 (socket says 172.21.15.47:51140) 2024-03-23T09:41:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a0bfff640 1 -- 172.21.15.47:0/1170977123 learned_addr learned my addr 172.21.15.47:0/1170977123 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a11221640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6a0c0b44a0 0x7f6a0c0b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50240/0 (socket says 172.21.15.47:50240) 2024-03-23T09:41:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a10a20640 1 --2- 172.21.15.47:0/1170977123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a0c159930 0x7f6a0c0ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a11221640 1 -- 172.21.15.47:0/1170977123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a0c159930 msgr2=0x7f6a0c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a11221640 1 --2- 172.21.15.47:0/1170977123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a0c159930 0x7f6a0c0ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a11221640 1 -- 172.21.15.47:0/1170977123 >> v1:172.21.15.47:6789/0 conn(0x7f6a0c0ce660 legacy=0x7f6a0c0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a11221640 1 -- 172.21.15.47:0/1170977123 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c0cf900 con 0x7f6a0c0b44a0 2024-03-23T09:41:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.668+0000 7f6a0b7fe640 1 -- 172.21.15.47:0/1170977123 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386128130 0 0) 0x7f6a0c0cff50 con 0x7f6a0c0ce660 2024-03-23T09:41:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.669+0000 7f6a11221640 1 --2- 172.21.15.47:0/1170977123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a0c0b44a0 0x7f6a0c0b2850 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400bfa0 tx=0x7f6a0402e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fceda9febd1582 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.669+0000 7f6a0b7fe640 1 -- 172.21.15.47:0/1170977123 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0402ea40 con 0x7f6a0c0b44a0 2024-03-23T09:41:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.669+0000 7f6a0b7fe640 1 -- 172.21.15.47:0/1170977123 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0402eba0 con 0x7f6a0c0b44a0 2024-03-23T09:41:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.669+0000 7f6a0b7fe640 1 -- 172.21.15.47:0/1170977123 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04009540 con 0x7f6a0c0b44a0 2024-03-23T09:41:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.670+0000 7f6a12c84640 1 -- 172.21.15.47:0/1170977123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a0c0b44a0 msgr2=0x7f6a0c0b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.670+0000 7f6a12c84640 1 --2- 172.21.15.47:0/1170977123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a0c0b44a0 0x7f6a0c0b2850 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400bfa0 tx=0x7f6a0402e0a0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.670+0000 7f6a12c84640 1 -- 172.21.15.47:0/1170977123 shutdown_connections 2024-03-23T09:41:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.670+0000 7f6a12c84640 1 --2- 172.21.15.47:0/1170977123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a0c0b44a0 0x7f6a0c0b2850 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.670+0000 7f6a12c84640 1 --2- 172.21.15.47:0/1170977123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a0c159930 0x7f6a0c0ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.670+0000 7f6a12c84640 1 -- 172.21.15.47:0/1170977123 >> 172.21.15.47:0/1170977123 conn(0x7f6a0c0b13a0 msgr2=0x7f6a0c0b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.670+0000 7f6a12c84640 1 -- 172.21.15.47:0/1170977123 shutdown_connections 2024-03-23T09:41:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.670+0000 7f6a12c84640 1 -- 172.21.15.47:0/1170977123 wait complete. 2024-03-23T09:41:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.671+0000 7f6a12c84640 1 Processor -- start 2024-03-23T09:41:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.671+0000 7f6a12c84640 1 -- start start 2024-03-23T09:41:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a12c84640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a0c0b44a0 0x7f6a0c1535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a12c84640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a0c0ce660 0x7f6a0c157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a12c84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a0c159930 0x7f6a0c0b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a12c84640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c0a7570 con 0x7f6a0c159930 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a12c84640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c1f2210 con 0x7f6a0c0b44a0 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a12c84640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c1f24f0 con 0x7f6a0c0ce660 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a11221640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a0c159930 0x7f6a0c0b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a11221640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a0c159930 0x7f6a0c0b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50244/0 (socket says 172.21.15.47:50244) 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a11221640 1 -- 172.21.15.47:0/2234697450 learned_addr learned my addr 172.21.15.47:0/2234697450 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a0bfff640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a0c0ce660 0x7f6a0c157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a10a20640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a0c0b44a0 0x7f6a0c1535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a11221640 1 -- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a0c0ce660 msgr2=0x7f6a0c157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a11221640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a0c0ce660 0x7f6a0c157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a11221640 1 -- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a0c0b44a0 msgr2=0x7f6a0c1535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a11221640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a0c0b44a0 0x7f6a0c1535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a11221640 1 -- 172.21.15.47:0/2234697450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0400bc80 con 0x7f6a0c159930 2024-03-23T09:41:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a11221640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a0c159930 0x7f6a0c0b03e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6a04038040 tx=0x7f6a04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:39.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a097fa640 1 -- 172.21.15.47:0/2234697450 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04044070 con 0x7f6a0c159930 2024-03-23T09:41:39.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a097fa640 1 -- 172.21.15.47:0/2234697450 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a040092b0 con 0x7f6a0c159930 2024-03-23T09:41:39.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a12c84640 1 -- 172.21.15.47:0/2234697450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0c1f27d0 con 0x7f6a0c159930 2024-03-23T09:41:39.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a097fa640 1 -- 172.21.15.47:0/2234697450 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04036600 con 0x7f6a0c159930 2024-03-23T09:41:39.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.672+0000 7f6a12c84640 1 -- 172.21.15.47:0/2234697450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0c158100 con 0x7f6a0c159930 2024-03-23T09:41:39.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.673+0000 7f6a12c84640 1 -- 172.21.15.47:0/2234697450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0c158350 con 0x7f6a0c159930 2024-03-23T09:41:39.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.674+0000 7f6a097fa640 1 -- 172.21.15.47:0/2234697450 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6a04036760 con 0x7f6a0c159930 2024-03-23T09:41:39.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.674+0000 7f6a097fa640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f69f006ac00 0x7f69f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:39.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.674+0000 7f6a097fa640 1 -- 172.21.15.47:0/2234697450 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a040abc60 con 0x7f6a0c159930 2024-03-23T09:41:39.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.674+0000 7f6a10a20640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f69f006ac00 0x7f69f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:39.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.675+0000 7f6a10a20640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f69f006ac00 0x7f69f006d0c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a00006fd0 tx=0x7f6a00008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:39.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.678+0000 7f6a097fa640 1 -- 172.21.15.47:0/2234697450 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6a04076e40 con 0x7f6a0c159930 2024-03-23T09:41:39.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.871+0000 7f6a12c84640 1 -- 172.21.15.47:0/2234697450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6a0c1f2cf0 con 0x7f6a0c159930 2024-03-23T09:41:39.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.871+0000 7f6a097fa640 1 -- 172.21.15.47:0/2234697450 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6a0407a460 con 0x7f6a0c159930 2024-03-23T09:41:39.871 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:41:39.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.873+0000 7f6a12c84640 1 -- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f69f006ac00 msgr2=0x7f69f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.873+0000 7f6a12c84640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f69f006ac00 0x7f69f006d0c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a00006fd0 tx=0x7f6a00008040 comp rx=0 tx=0).stop 2024-03-23T09:41:39.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.873+0000 7f6a12c84640 1 -- 172.21.15.47:0/2234697450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a0c159930 msgr2=0x7f6a0c0b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:39.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.873+0000 7f6a12c84640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a0c159930 0x7f6a0c0b03e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6a04038040 tx=0x7f6a04004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:39.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.873+0000 7f6a12c84640 1 -- 172.21.15.47:0/2234697450 shutdown_connections 2024-03-23T09:41:39.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.873+0000 7f6a12c84640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f69f006ac00 0x7f69f006d0c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.873+0000 7f6a12c84640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a0c159930 0x7f6a0c0b03e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.873+0000 7f6a12c84640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a0c0ce660 0x7f6a0c157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.873+0000 7f6a12c84640 1 --2- 172.21.15.47:0/2234697450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a0c0b44a0 0x7f6a0c1535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:39.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.873+0000 7f6a12c84640 1 -- 172.21.15.47:0/2234697450 >> 172.21.15.47:0/2234697450 conn(0x7f6a0c0b13a0 msgr2=0x7f6a0c0cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:39.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.873+0000 7f6a12c84640 1 -- 172.21.15.47:0/2234697450 shutdown_connections 2024-03-23T09:41:39.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:39.873+0000 7f6a12c84640 1 -- 172.21.15.47:0/2234697450 wait complete. 2024-03-23T09:41:39.883 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-03-23T09:41:39.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:39.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-03-23T09:41:39.916 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:41:39.916 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:41:39.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:39.916 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-23T09:41:39.984 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:41:39.984 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:41:40.000 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:41:40.000 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:41:40.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:40.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.196+0000 7f1fd060b640 1 Processor -- start 2024-03-23T09:41:40.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.197+0000 7f1fd060b640 1 -- start start 2024-03-23T09:41:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.197+0000 7f1fd060b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1fc80b42b0 0x7f1fc8164450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.197+0000 7f1fd060b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc80ccd00 con 0x7f1fc8142a90 2024-03-23T09:41:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.197+0000 7f1fd060b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc80ccfe0 con 0x7f1fc8143010 2024-03-23T09:41:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.197+0000 7f1fd060b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc80b27c0 con 0x7f1fc80b42b0 2024-03-23T09:41:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fce3a7640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1fc8142a90 0x7f1fc80cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51150/0 (socket says 172.21.15.47:51150) 2024-03-23T09:41:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fce3a7640 1 -- 172.21.15.47:0/369483081 learned_addr learned my addr 172.21.15.47:0/369483081 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fceba8640 1 --2- 172.21.15.47:0/369483081 >> v2:172.21.15.175:3300/0 conn(0x7f1fc80b42b0 0x7f1fc8164450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323605438 0 0) 0x7f1fc80ccd00 con 0x7f1fc8142a90 2024-03-23T09:41:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fb4003520 con 0x7f1fc8142a90 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 928852405 0 0) 0x7f1fb4003520 con 0x7f1fc8142a90 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 >> v2:172.21.15.175:3300/0 conn(0x7f1fc80b42b0 msgr2=0x7f1fc8164450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 --2- 172.21.15.47:0/369483081 >> v2:172.21.15.175:3300/0 conn(0x7f1fc80b42b0 0x7f1fc8164450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 >> v1:172.21.15.175:6789/0 conn(0x7f1fc8143010 legacy=0x7f1fc80b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc80cd2c0 con 0x7f1fc8142a90 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f1fc4001670 con 0x7f1fc8142a90 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 >> v1:172.21.15.47:6789/0 conn(0x7f1fc8142a90 legacy=0x7f1fc80cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 --2- 172.21.15.47:0/369483081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fb4003970 0x7f1fb4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 --2- 172.21.15.47:0/369483081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb40047c0 0x7f1fb4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 --2- 172.21.15.47:0/369483081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fb4005180 0x7f1fb400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc80b27c0 con 0x7f1fb4005180 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb400c020 con 0x7f1fb40047c0 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb400c1e0 con 0x7f1fb4003970 2024-03-23T09:41:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fceba8640 1 --2- 172.21.15.47:0/369483081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fb4005180 0x7f1fb400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fce3a7640 1 --2- 172.21.15.47:0/369483081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fb4003970 0x7f1fb4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.198+0000 7f1fcdba6640 1 --2- 172.21.15.47:0/369483081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb40047c0 0x7f1fb4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.199+0000 7f1fceba8640 1 -- 172.21.15.47:0/369483081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fb4003970 msgr2=0x7f1fb4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.199+0000 7f1fceba8640 1 --2- 172.21.15.47:0/369483081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fb4003970 0x7f1fb4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.199+0000 7f1fceba8640 1 -- 172.21.15.47:0/369483081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb40047c0 msgr2=0x7f1fb4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.199+0000 7f1fceba8640 1 --2- 172.21.15.47:0/369483081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb40047c0 0x7f1fb4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.199+0000 7f1fceba8640 1 -- 172.21.15.47:0/369483081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb4003520 con 0x7f1fb4005180 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.199+0000 7f1fceba8640 1 -- 172.21.15.47:0/369483081 reap_dead start 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.199+0000 7f1fceba8640 1 --2- 172.21.15.47:0/369483081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fb4005180 0x7f1fb400b980 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009fc0 tx=0x7f1fb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4b0bce9ebe1f9a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.199+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc002cc60 con 0x7f1fb4005180 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.199+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc000f040 con 0x7f1fb4005180 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.199+0000 7f1fcd3a5640 1 -- 172.21.15.47:0/369483081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0003990 con 0x7f1fb4005180 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.200+0000 7f1fd060b640 1 -- 172.21.15.47:0/369483081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fb4005180 msgr2=0x7f1fb400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.200+0000 7f1fd060b640 1 --2- 172.21.15.47:0/369483081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fb4005180 0x7f1fb400b980 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009fc0 tx=0x7f1fb40057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.200+0000 7f1fd060b640 1 -- 172.21.15.47:0/369483081 shutdown_connections 2024-03-23T09:41:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.200+0000 7f1fd060b640 1 --2- 172.21.15.47:0/369483081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fb4005180 0x7f1fb400b980 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.200+0000 7f1fd060b640 1 -- 172.21.15.47:0/369483081 >> 172.21.15.47:0/369483081 conn(0x7f1fc806a310 msgr2=0x7f1fc80b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.200+0000 7f1fd060b640 1 -- 172.21.15.47:0/369483081 shutdown_connections 2024-03-23T09:41:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.200+0000 7f1fd060b640 1 -- 172.21.15.47:0/369483081 wait complete. 2024-03-23T09:41:40.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.201+0000 7f1fd060b640 1 Processor -- start 2024-03-23T09:41:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.201+0000 7f1fd060b640 1 -- start start 2024-03-23T09:41:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fd060b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb4005180 0x7f1fc80701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fd060b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fc8159430 0x7f1fc8070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fd060b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fc80a84b0 0x7f1fc80a8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fd060b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc812e5b0 con 0x7f1fc8159430 2024-03-23T09:41:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fd060b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc806d500 con 0x7f1fb4005180 2024-03-23T09:41:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fd060b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc80a8e70 con 0x7f1fc80a84b0 2024-03-23T09:41:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fcdba6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fc8159430 0x7f1fc8070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fcdba6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fc8159430 0x7f1fc8070730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50260/0 (socket says 172.21.15.47:50260) 2024-03-23T09:41:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fcdba6640 1 -- 172.21.15.47:0/3525629933 learned_addr learned my addr 172.21.15.47:0/3525629933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fce3a7640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb4005180 0x7f1fc80701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fceba8640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fc80a84b0 0x7f1fc80a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fcdba6640 1 -- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fc80a84b0 msgr2=0x7f1fc80a8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fcdba6640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fc80a84b0 0x7f1fc80a8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fcdba6640 1 -- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb4005180 msgr2=0x7f1fc80701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fcdba6640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb4005180 0x7f1fc80701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fcdba6640 1 -- 172.21.15.47:0/3525629933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc80a9100 con 0x7f1fc8159430 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fce3a7640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb4005180 0x7f1fc80701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fceba8640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fc80a84b0 0x7f1fc80a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.202+0000 7f1fcdba6640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fc8159430 0x7f1fc8070730 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800f5f0 tx=0x7f1fb800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.203+0000 7f1fbeffd640 1 -- 172.21.15.47:0/3525629933 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80052e0 con 0x7f1fc8159430 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.203+0000 7f1fbeffd640 1 -- 172.21.15.47:0/3525629933 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb8005440 con 0x7f1fc8159430 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.203+0000 7f1fd060b640 1 -- 172.21.15.47:0/3525629933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc80a93e0 con 0x7f1fc8159430 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.203+0000 7f1fbeffd640 1 -- 172.21.15.47:0/3525629933 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8003810 con 0x7f1fc8159430 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.203+0000 7f1fd060b640 1 -- 172.21.15.47:0/3525629933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc812eac0 con 0x7f1fc8159430 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.203+0000 7f1fd060b640 1 -- 172.21.15.47:0/3525629933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc806acb0 con 0x7f1fc8159430 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.204+0000 7f1fbeffd640 1 -- 172.21.15.47:0/3525629933 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1fb8003970 con 0x7f1fc8159430 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.204+0000 7f1fbeffd640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1fac06ac30 0x7f1fac06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.204+0000 7f1fbeffd640 1 -- 172.21.15.47:0/3525629933 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fb8089160 con 0x7f1fc8159430 2024-03-23T09:41:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.205+0000 7f1fce3a7640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1fac06ac30 0x7f1fac06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.205+0000 7f1fce3a7640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1fac06ac30 0x7f1fac06d0f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4007c60 tx=0x7f1fc4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.209+0000 7f1fbeffd640 1 -- 172.21.15.47:0/3525629933 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1fb80543d0 con 0x7f1fc8159430 2024-03-23T09:41:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.389+0000 7f1fd060b640 1 -- 172.21.15.47:0/3525629933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fc8073480 con 0x7f1fc8159430 2024-03-23T09:41:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.390+0000 7f1fbeffd640 1 -- 172.21.15.47:0/3525629933 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f1fb80579f0 con 0x7f1fc8159430 2024-03-23T09:41:40.390 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:40.390 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:40.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.392+0000 7f1fd060b640 1 -- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1fac06ac30 msgr2=0x7f1fac06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.392+0000 7f1fd060b640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1fac06ac30 0x7f1fac06d0f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4007c60 tx=0x7f1fc4009040 comp rx=0 tx=0).stop 2024-03-23T09:41:40.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.392+0000 7f1fd060b640 1 -- 172.21.15.47:0/3525629933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fc8159430 msgr2=0x7f1fc8070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.392+0000 7f1fd060b640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fc8159430 0x7f1fc8070730 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800f5f0 tx=0x7f1fb800fab0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.392+0000 7f1fd060b640 1 -- 172.21.15.47:0/3525629933 shutdown_connections 2024-03-23T09:41:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.392+0000 7f1fd060b640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1fac06ac30 0x7f1fac06d0f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.392+0000 7f1fd060b640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fc80a84b0 0x7f1fc80a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.392+0000 7f1fd060b640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fc8159430 0x7f1fc8070730 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.392+0000 7f1fd060b640 1 --2- 172.21.15.47:0/3525629933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb4005180 0x7f1fc80701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.392+0000 7f1fd060b640 1 -- 172.21.15.47:0/3525629933 >> 172.21.15.47:0/3525629933 conn(0x7f1fc806a310 msgr2=0x7f1fc815ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.392+0000 7f1fd060b640 1 -- 172.21.15.47:0/3525629933 shutdown_connections 2024-03-23T09:41:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.392+0000 7f1fd060b640 1 -- 172.21.15.47:0/3525629933 wait complete. 2024-03-23T09:41:40.403 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:40.403 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:40.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.507+0000 7f671443e640 1 Processor -- start 2024-03-23T09:41:40.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.507+0000 7f671443e640 1 -- start start 2024-03-23T09:41:40.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.508+0000 7f671443e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f670c0d23a0 0x7f670c0d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.508+0000 7f671443e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0b8300 0x7f670c0d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.508+0000 7f671443e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670c0d10a0 con 0x7f670c15cee0 2024-03-23T09:41:40.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.508+0000 7f671443e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670c0d1300 con 0x7f670c0d23a0 2024-03-23T09:41:40.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.508+0000 7f671443e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670c0d15e0 con 0x7f670c0b8300 2024-03-23T09:41:40.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.508+0000 7f67121da640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f670c15cee0 0x7f670c0d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51154/0 (socket says 172.21.15.47:51154) 2024-03-23T09:41:40.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.508+0000 7f67121da640 1 -- 172.21.15.47:0/3305714437 learned_addr learned my addr 172.21.15.47:0/3305714437 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:40.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.508+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709831385 0 0) 0x7f670c0d10a0 con 0x7f670c15cee0 2024-03-23T09:41:40.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.508+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66f8003520 con 0x7f670c15cee0 2024-03-23T09:41:40.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.508+0000 7f67119d9640 1 --2- 172.21.15.47:0/3305714437 >> v2:172.21.15.175:3300/0 conn(0x7f670c0d23a0 0x7f670c0d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.508+0000 7f67129db640 1 --2- 172.21.15.47:0/3305714437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0b8300 0x7f670c0d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3364106776 0 0) 0x7f66f8003520 con 0x7f670c15cee0 2024-03-23T09:41:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0b8300 msgr2=0x7f670c0d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 --2- 172.21.15.47:0/3305714437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0b8300 0x7f670c0d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 >> v2:172.21.15.175:3300/0 conn(0x7f670c0d23a0 msgr2=0x7f670c0d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 --2- 172.21.15.47:0/3305714437 >> v2:172.21.15.175:3300/0 conn(0x7f670c0d23a0 0x7f670c0d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c0d18c0 con 0x7f670c15cee0 2024-03-23T09:41:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f6704001670 con 0x7f670c15cee0 2024-03-23T09:41:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 >> v1:172.21.15.47:6789/0 conn(0x7f670c15cee0 legacy=0x7f670c0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 --2- 172.21.15.47:0/3305714437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0b8300 0x7f66f8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 --2- 172.21.15.47:0/3305714437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f8003fa0 0x7f66f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 --2- 172.21.15.47:0/3305714437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66f8004d00 0x7f66f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c0d1300 con 0x7f66f8003fa0 2024-03-23T09:41:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c0d15e0 con 0x7f66f8004d00 2024-03-23T09:41:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f800ba80 con 0x7f670c0b8300 2024-03-23T09:41:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67119d9640 1 --2- 172.21.15.47:0/3305714437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f8003fa0 0x7f66f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67121da640 1 --2- 172.21.15.47:0/3305714437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0b8300 0x7f66f8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67129db640 1 --2- 172.21.15.47:0/3305714437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66f8004d00 0x7f66f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67119d9640 1 -- 172.21.15.47:0/3305714437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0b8300 msgr2=0x7f66f8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67119d9640 1 --2- 172.21.15.47:0/3305714437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0b8300 0x7f66f8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67119d9640 1 -- 172.21.15.47:0/3305714437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66f8004d00 msgr2=0x7f66f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:40.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67119d9640 1 --2- 172.21.15.47:0/3305714437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66f8004d00 0x7f66f8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67119d9640 1 -- 172.21.15.47:0/3305714437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f8003520 con 0x7f66f8003fa0 2024-03-23T09:41:40.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.509+0000 7f67121da640 1 --2- 172.21.15.47:0/3305714437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0b8300 0x7f66f8003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.510+0000 7f67119d9640 1 --2- 172.21.15.47:0/3305714437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f8003fa0 0x7f66f80047c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00ba00 tx=0x7f66f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f5d0794cbf8a8c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.510+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc032640 con 0x7f66f8003fa0 2024-03-23T09:41:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.510+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66fc0327a0 con 0x7f66f8003fa0 2024-03-23T09:41:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.510+0000 7f67111d8640 1 -- 172.21.15.47:0/3305714437 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc032b30 con 0x7f66f8003fa0 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.510+0000 7f671443e640 1 -- 172.21.15.47:0/3305714437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f8003fa0 msgr2=0x7f66f80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.510+0000 7f671443e640 1 --2- 172.21.15.47:0/3305714437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f8003fa0 0x7f66f80047c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00ba00 tx=0x7f66f80057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.510+0000 7f67129db640 1 -- 172.21.15.47:0/3305714437 reap_dead start 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.511+0000 7f671443e640 1 -- 172.21.15.47:0/3305714437 shutdown_connections 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.511+0000 7f671443e640 1 -- 172.21.15.47:0/3305714437 >> 172.21.15.47:0/3305714437 conn(0x7f670c065df0 msgr2=0x7f670c066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.511+0000 7f671443e640 1 -- 172.21.15.47:0/3305714437 shutdown_connections 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.511+0000 7f671443e640 1 -- 172.21.15.47:0/3305714437 wait complete. 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.512+0000 7f671443e640 1 Processor -- start 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.512+0000 7f671443e640 1 -- start start 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.512+0000 7f671443e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f8003fa0 0x7f670c15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f671443e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0cc4f0 0x7f670c15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f671443e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670c148e60 0x7f670c0cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f671443e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c15db10 con 0x7f66f8003fa0 2024-03-23T09:41:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f671443e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c15ddf0 con 0x7f670c148e60 2024-03-23T09:41:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f671443e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670c15e0d0 con 0x7f670c0cc4f0 2024-03-23T09:41:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f67121da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f8003fa0 0x7f670c15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f67121da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f8003fa0 0x7f670c15ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50278/0 (socket says 172.21.15.47:50278) 2024-03-23T09:41:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f67121da640 1 -- 172.21.15.47:0/2024001889 learned_addr learned my addr 172.21.15.47:0/2024001889 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f67119d9640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0cc4f0 0x7f670c15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f67129db640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670c148e60 0x7f670c0cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f67121da640 1 -- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0cc4f0 msgr2=0x7f670c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f67121da640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0cc4f0 0x7f670c15d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f67121da640 1 -- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670c148e60 msgr2=0x7f670c0cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f67121da640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670c148e60 0x7f670c0cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f67121da640 1 -- 172.21.15.47:0/2024001889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c15e3b0 con 0x7f66f8003fa0 2024-03-23T09:41:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f67121da640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f8003fa0 0x7f670c15ebb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6704001c10 tx=0x7f670400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f6702ffd640 1 -- 172.21.15.47:0/2024001889 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670400c840 con 0x7f66f8003fa0 2024-03-23T09:41:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f6702ffd640 1 -- 172.21.15.47:0/2024001889 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670400c9a0 con 0x7f66f8003fa0 2024-03-23T09:41:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f671443e640 1 -- 172.21.15.47:0/2024001889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670c15e690 con 0x7f66f8003fa0 2024-03-23T09:41:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f6702ffd640 1 -- 172.21.15.47:0/2024001889 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670400cd30 con 0x7f66f8003fa0 2024-03-23T09:41:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.513+0000 7f671443e640 1 -- 172.21.15.47:0/2024001889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670c1492e0 con 0x7f66f8003fa0 2024-03-23T09:41:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.514+0000 7f671443e640 1 -- 172.21.15.47:0/2024001889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670c072360 con 0x7f66f8003fa0 2024-03-23T09:41:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.515+0000 7f6702ffd640 1 -- 172.21.15.47:0/2024001889 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f670401d460 con 0x7f66f8003fa0 2024-03-23T09:41:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.515+0000 7f6702ffd640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f66f006ac30 0x7f66f006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.515+0000 7f6702ffd640 1 -- 172.21.15.47:0/2024001889 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67040891d0 con 0x7f66f8003fa0 2024-03-23T09:41:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.515+0000 7f67119d9640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f66f006ac30 0x7f66f006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.516+0000 7f67119d9640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f66f006ac30 0x7f66f006d0f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f670c0cfb90 tx=0x7f66fc03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.519+0000 7f6702ffd640 1 -- 172.21.15.47:0/2024001889 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f67040544f0 con 0x7f66f8003fa0 2024-03-23T09:41:40.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.698+0000 7f671443e640 1 -- 172.21.15.47:0/2024001889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f670c066be0 con 0x7f66f8003fa0 2024-03-23T09:41:40.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.698+0000 7f6702ffd640 1 -- 172.21.15.47:0/2024001889 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6704057b10 con 0x7f66f8003fa0 2024-03-23T09:41:40.698 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:40.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.700+0000 7f671443e640 1 -- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f66f006ac30 msgr2=0x7f66f006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.700+0000 7f671443e640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f66f006ac30 0x7f66f006d0f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f670c0cfb90 tx=0x7f66fc03d040 comp rx=0 tx=0).stop 2024-03-23T09:41:40.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.700+0000 7f671443e640 1 -- 172.21.15.47:0/2024001889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f8003fa0 msgr2=0x7f670c15ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.700+0000 7f671443e640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f8003fa0 0x7f670c15ebb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6704001c10 tx=0x7f670400ed50 comp rx=0 tx=0).stop 2024-03-23T09:41:40.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.700+0000 7f671443e640 1 -- 172.21.15.47:0/2024001889 shutdown_connections 2024-03-23T09:41:40.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.700+0000 7f671443e640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f66f006ac30 0x7f66f006d0f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.700+0000 7f671443e640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670c148e60 0x7f670c0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.700+0000 7f671443e640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f670c0cc4f0 0x7f670c15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.700+0000 7f671443e640 1 --2- 172.21.15.47:0/2024001889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f8003fa0 0x7f670c15ebb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.700+0000 7f671443e640 1 -- 172.21.15.47:0/2024001889 >> 172.21.15.47:0/2024001889 conn(0x7f670c065df0 msgr2=0x7f670c0c83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:40.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.700+0000 7f671443e640 1 -- 172.21.15.47:0/2024001889 shutdown_connections 2024-03-23T09:41:40.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.700+0000 7f671443e640 1 -- 172.21.15.47:0/2024001889 wait complete. 2024-03-23T09:41:40.711 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:40.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.814+0000 7f049871b640 1 Processor -- start 2024-03-23T09:41:40.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.814+0000 7f049871b640 1 -- start start 2024-03-23T09:41:40.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.815+0000 7f049871b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049015ff60 0x7f04900cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.815+0000 7f049871b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f04900b81f0 0x7f04900cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.815+0000 7f049871b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0490160d60 con 0x7f04900b1b20 2024-03-23T09:41:40.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.815+0000 7f049871b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0490160340 con 0x7f04900b81f0 2024-03-23T09:41:40.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.815+0000 7f049871b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04900b8680 con 0x7f049015ff60 2024-03-23T09:41:40.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f0496cb8640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f04900b1b20 0x7f04900d2860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51168/0 (socket says 172.21.15.47:51168) 2024-03-23T09:41:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f0496cb8640 1 -- 172.21.15.47:0/1639857577 learned_addr learned my addr 172.21.15.47:0/1639857577 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f0495cb6640 1 --2- 172.21.15.47:0/1639857577 >> v2:172.21.15.175:3300/0 conn(0x7f04900b81f0 0x7f04900cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04964b7640 1 --2- 172.21.15.47:0/1639857577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049015ff60 0x7f04900cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493010249 0 0) 0x7f0490160d60 con 0x7f04900b1b20 2024-03-23T09:41:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f047c003520 con 0x7f04900b1b20 2024-03-23T09:41:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3545451194 0 0) 0x7f047c003520 con 0x7f04900b1b20 2024-03-23T09:41:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049015ff60 msgr2=0x7f04900cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 --2- 172.21.15.47:0/1639857577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049015ff60 0x7f04900cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 >> v2:172.21.15.175:3300/0 conn(0x7f04900b81f0 msgr2=0x7f04900cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 --2- 172.21.15.47:0/1639857577 >> v2:172.21.15.175:3300/0 conn(0x7f04900b81f0 0x7f04900cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0490160710 con 0x7f04900b1b20 2024-03-23T09:41:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f04880019f0 con 0x7f04900b1b20 2024-03-23T09:41:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 >> v1:172.21.15.47:6789/0 conn(0x7f04900b1b20 legacy=0x7f04900d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 --2- 172.21.15.47:0/1639857577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f047c003970 0x7f047c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 --2- 172.21.15.47:0/1639857577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047c0047c0 0x7f047c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 --2- 172.21.15.47:0/1639857577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049015ff60 0x7f047c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0490160340 con 0x7f047c0047c0 2024-03-23T09:41:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04900b8680 con 0x7f047c003970 2024-03-23T09:41:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c00b980 con 0x7f049015ff60 2024-03-23T09:41:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f0495cb6640 1 --2- 172.21.15.47:0/1639857577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047c0047c0 0x7f047c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f0496cb8640 1 --2- 172.21.15.47:0/1639857577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049015ff60 0x7f047c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.816+0000 7f04964b7640 1 --2- 172.21.15.47:0/1639857577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f047c003970 0x7f047c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.817+0000 7f0495cb6640 1 -- 172.21.15.47:0/1639857577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049015ff60 msgr2=0x7f047c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.817+0000 7f0495cb6640 1 --2- 172.21.15.47:0/1639857577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049015ff60 0x7f047c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.817+0000 7f0495cb6640 1 -- 172.21.15.47:0/1639857577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f047c003970 msgr2=0x7f047c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.817+0000 7f0495cb6640 1 --2- 172.21.15.47:0/1639857577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f047c003970 0x7f047c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.817+0000 7f0495cb6640 1 -- 172.21.15.47:0/1639857577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c003520 con 0x7f047c0047c0 2024-03-23T09:41:40.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.817+0000 7f0495cb6640 1 --2- 172.21.15.47:0/1639857577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047c0047c0 0x7f047c004c40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0480009af0 tx=0x7f047c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba19957762e42720 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:40.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.817+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048002c710 con 0x7f047c0047c0 2024-03-23T09:41:40.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.817+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048002c870 con 0x7f047c0047c0 2024-03-23T09:41:40.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.817+0000 7f04954b5640 1 -- 172.21.15.47:0/1639857577 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048002cc00 con 0x7f047c0047c0 2024-03-23T09:41:40.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.818+0000 7f049871b640 1 -- 172.21.15.47:0/1639857577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047c0047c0 msgr2=0x7f047c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.818+0000 7f049871b640 1 --2- 172.21.15.47:0/1639857577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047c0047c0 0x7f047c004c40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0480009af0 tx=0x7f047c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.818+0000 7f0496cb8640 1 -- 172.21.15.47:0/1639857577 reap_dead start 2024-03-23T09:41:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.818+0000 7f049871b640 1 -- 172.21.15.47:0/1639857577 shutdown_connections 2024-03-23T09:41:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.818+0000 7f049871b640 1 -- 172.21.15.47:0/1639857577 >> 172.21.15.47:0/1639857577 conn(0x7f0490065df0 msgr2=0x7f0490066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.818+0000 7f049871b640 1 -- 172.21.15.47:0/1639857577 shutdown_connections 2024-03-23T09:41:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.818+0000 7f049871b640 1 -- 172.21.15.47:0/1639857577 wait complete. 2024-03-23T09:41:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.819+0000 7f049871b640 1 Processor -- start 2024-03-23T09:41:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.819+0000 7f049871b640 1 -- start start 2024-03-23T09:41:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f049871b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f047c0047c0 0x7f04900c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f049871b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04900ca100 0x7f04900d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f049871b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04900c8d40 0x7f04900c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f049871b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04901543a0 con 0x7f04900c8d40 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f049871b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04900c7a70 con 0x7f047c0047c0 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f049871b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04900c7ca0 con 0x7f04900ca100 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f0496cb8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04900c8d40 0x7f04900c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f0496cb8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04900c8d40 0x7f04900c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50306/0 (socket says 172.21.15.47:50306) 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f0496cb8640 1 -- 172.21.15.47:0/2769007468 learned_addr learned my addr 172.21.15.47:0/2769007468 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f04964b7640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f047c0047c0 0x7f04900c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f0495cb6640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04900ca100 0x7f04900d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f0496cb8640 1 -- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04900ca100 msgr2=0x7f04900d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f0496cb8640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04900ca100 0x7f04900d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f0496cb8640 1 -- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f047c0047c0 msgr2=0x7f04900c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f0496cb8640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f047c0047c0 0x7f04900c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f0496cb8640 1 -- 172.21.15.47:0/2769007468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04900c7f80 con 0x7f04900c8d40 2024-03-23T09:41:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f04964b7640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f047c0047c0 0x7f04900c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.820+0000 7f0495cb6640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04900ca100 0x7f04900d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.821+0000 7f0496cb8640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04900c8d40 0x7f04900c6230 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f04900b4e10 tx=0x7f0488007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.821+0000 7f0486ffd640 1 -- 172.21.15.47:0/2769007468 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048800ad40 con 0x7f04900c8d40 2024-03-23T09:41:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.821+0000 7f0486ffd640 1 -- 172.21.15.47:0/2769007468 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04880133f0 con 0x7f04900c8d40 2024-03-23T09:41:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.821+0000 7f049871b640 1 -- 172.21.15.47:0/2769007468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04900c8260 con 0x7f04900c8d40 2024-03-23T09:41:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.821+0000 7f0486ffd640 1 -- 172.21.15.47:0/2769007468 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04880137a0 con 0x7f04900c8d40 2024-03-23T09:41:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.821+0000 7f049871b640 1 -- 172.21.15.47:0/2769007468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04900c8820 con 0x7f04900c8d40 2024-03-23T09:41:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.822+0000 7f049871b640 1 -- 172.21.15.47:0/2769007468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04900672f0 con 0x7f04900c8d40 2024-03-23T09:41:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.822+0000 7f0486ffd640 1 -- 172.21.15.47:0/2769007468 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0488013900 con 0x7f04900c8d40 2024-03-23T09:41:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.823+0000 7f0486ffd640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f047406ac00 0x7f047406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.823+0000 7f0486ffd640 1 -- 172.21.15.47:0/2769007468 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0488089bc0 con 0x7f04900c8d40 2024-03-23T09:41:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.823+0000 7f04964b7640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f047406ac00 0x7f047406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.823+0000 7f04964b7640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f047406ac00 0x7f047406d0c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f048c006d40 tx=0x7f048c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:40.827+0000 7f0486ffd640 1 -- 172.21.15.47:0/2769007468 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0488054d00 con 0x7f04900c8d40 2024-03-23T09:41:41.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.006+0000 7f049871b640 1 -- 172.21.15.47:0/2769007468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04900666c0 con 0x7f04900c8d40 2024-03-23T09:41:41.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.007+0000 7f0486ffd640 1 -- 172.21.15.47:0/2769007468 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0488058320 con 0x7f04900c8d40 2024-03-23T09:41:41.006 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.009+0000 7f049871b640 1 -- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f047406ac00 msgr2=0x7f047406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.009+0000 7f049871b640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f047406ac00 0x7f047406d0c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f048c006d40 tx=0x7f048c008040 comp rx=0 tx=0).stop 2024-03-23T09:41:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.009+0000 7f049871b640 1 -- 172.21.15.47:0/2769007468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04900c8d40 msgr2=0x7f04900c6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.009+0000 7f049871b640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04900c8d40 0x7f04900c6230 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f04900b4e10 tx=0x7f0488007bd0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.009+0000 7f049871b640 1 -- 172.21.15.47:0/2769007468 shutdown_connections 2024-03-23T09:41:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.009+0000 7f049871b640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f047406ac00 0x7f047406d0c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.009+0000 7f049871b640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04900c8d40 0x7f04900c6230 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.009+0000 7f049871b640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04900ca100 0x7f04900d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.009+0000 7f049871b640 1 --2- 172.21.15.47:0/2769007468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f047c0047c0 0x7f04900c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.009+0000 7f049871b640 1 -- 172.21.15.47:0/2769007468 >> 172.21.15.47:0/2769007468 conn(0x7f0490065df0 msgr2=0x7f049015e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.009+0000 7f049871b640 1 -- 172.21.15.47:0/2769007468 shutdown_connections 2024-03-23T09:41:41.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.009+0000 7f049871b640 1 -- 172.21.15.47:0/2769007468 wait complete. 2024-03-23T09:41:41.019 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:41.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.123+0000 7fe4aea69640 1 Processor -- start 2024-03-23T09:41:41.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.123+0000 7fe4aea69640 1 -- start start 2024-03-23T09:41:41.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.124+0000 7fe4aea69640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a80cde10 0x7fe4a8160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.124+0000 7fe4aea69640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe4a80b8350 0x7fe4a80d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.124+0000 7fe4aea69640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a8161630 con 0x7fe4a815c990 2024-03-23T09:41:41.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.124+0000 7fe4aea69640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a80d1030 con 0x7fe4a80b8350 2024-03-23T09:41:41.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.124+0000 7fe4aea69640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a80cfe10 con 0x7fe4a80cde10 2024-03-23T09:41:41.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.124+0000 7fe4ad006640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe4a80b8350 0x7fe4a80d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.124+0000 7fe4ac805640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe4a815c990 0x7fe4a80cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51184/0 (socket says 172.21.15.47:51184) 2024-03-23T09:41:41.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.124+0000 7fe4ac805640 1 -- 172.21.15.47:0/3071744378 learned_addr learned my addr 172.21.15.47:0/3071744378 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:41.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.124+0000 7fe49ffff640 1 --2- 172.21.15.47:0/3071744378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a80cde10 0x7fe4a8160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.124+0000 7fe4ad006640 1 -- 172.21.15.47:0/3071744378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a80cde10 msgr2=0x7fe4a8160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:41.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.124+0000 7fe4ad006640 1 --2- 172.21.15.47:0/3071744378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a80cde10 0x7fe4a8160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.124+0000 7fe4ad006640 1 -- 172.21.15.47:0/3071744378 >> v1:172.21.15.47:6789/0 conn(0x7fe4a815c990 legacy=0x7fe4a80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.124+0000 7fe4ad006640 1 -- 172.21.15.47:0/3071744378 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a80b5b60 con 0x7fe4a80b8350 2024-03-23T09:41:41.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.125+0000 7fe4ad006640 1 --2- 172.21.15.47:0/3071744378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a80b8350 0x7fe4a80d14c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009d90 tx=0x7fe4a002bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5683dae27c9191 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:41.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.125+0000 7fe49f7fe640 1 -- 172.21.15.47:0/3071744378 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a002cb30 con 0x7fe4a80b8350 2024-03-23T09:41:41.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.125+0000 7fe49f7fe640 1 -- 172.21.15.47:0/3071744378 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a002cc90 con 0x7fe4a80b8350 2024-03-23T09:41:41.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.125+0000 7fe49f7fe640 1 -- 172.21.15.47:0/3071744378 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0034440 con 0x7fe4a80b8350 2024-03-23T09:41:41.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.126+0000 7fe4aea69640 1 -- 172.21.15.47:0/3071744378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a80b8350 msgr2=0x7fe4a80d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.126+0000 7fe4aea69640 1 --2- 172.21.15.47:0/3071744378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a80b8350 0x7fe4a80d14c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009d90 tx=0x7fe4a002bd80 comp rx=0 tx=0).stop 2024-03-23T09:41:41.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.126+0000 7fe4aea69640 1 -- 172.21.15.47:0/3071744378 shutdown_connections 2024-03-23T09:41:41.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.126+0000 7fe4aea69640 1 --2- 172.21.15.47:0/3071744378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a80b8350 0x7fe4a80d14c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.126+0000 7fe4aea69640 1 --2- 172.21.15.47:0/3071744378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a80cde10 0x7fe4a8160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.126+0000 7fe4aea69640 1 -- 172.21.15.47:0/3071744378 >> 172.21.15.47:0/3071744378 conn(0x7fe4a80659d0 msgr2=0x7fe4a8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:41.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.126+0000 7fe4aea69640 1 -- 172.21.15.47:0/3071744378 shutdown_connections 2024-03-23T09:41:41.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.126+0000 7fe4aea69640 1 -- 172.21.15.47:0/3071744378 wait complete. 2024-03-23T09:41:41.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.127+0000 7fe4aea69640 1 Processor -- start 2024-03-23T09:41:41.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.127+0000 7fe4aea69640 1 -- start start 2024-03-23T09:41:41.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4aea69640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a80b8350 0x7fe4a815dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4aea69640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a80cde10 0x7fe4a80c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4aea69640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a815c990 0x7fe4a8153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4aea69640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a815e670 con 0x7fe4a80b8350 2024-03-23T09:41:41.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4aea69640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a80c84f0 con 0x7fe4a815c990 2024-03-23T09:41:41.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4aea69640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a80c7c50 con 0x7fe4a80cde10 2024-03-23T09:41:41.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4ac805640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a80b8350 0x7fe4a815dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4ac805640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a80b8350 0x7fe4a815dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50316/0 (socket says 172.21.15.47:50316) 2024-03-23T09:41:41.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4ac805640 1 -- 172.21.15.47:0/4092760249 learned_addr learned my addr 172.21.15.47:0/4092760249 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4ad006640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a815c990 0x7fe4a8153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe49ffff640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a80cde10 0x7fe4a80c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4ac805640 1 -- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a80cde10 msgr2=0x7fe4a80c7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4ac805640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a80cde10 0x7fe4a80c7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4ac805640 1 -- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a815c990 msgr2=0x7fe4a8153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4ac805640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a815c990 0x7fe4a8153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.128+0000 7fe4ac805640 1 -- 172.21.15.47:0/4092760249 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a0009a70 con 0x7fe4a80b8350 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.129+0000 7fe4ac805640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a80b8350 0x7fe4a815dcf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400c4a0 tx=0x7fe4a400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.129+0000 7fe49d7fa640 1 -- 172.21.15.47:0/4092760249 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4005ce0 con 0x7fe4a80b8350 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.129+0000 7fe49d7fa640 1 -- 172.21.15.47:0/4092760249 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a4005e40 con 0x7fe4a80b8350 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.129+0000 7fe4aea69640 1 -- 172.21.15.47:0/4092760249 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a80c7f30 con 0x7fe4a80b8350 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.129+0000 7fe49d7fa640 1 -- 172.21.15.47:0/4092760249 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4007270 con 0x7fe4a80b8350 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.129+0000 7fe4aea69640 1 -- 172.21.15.47:0/4092760249 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a80c7940 con 0x7fe4a80b8350 2024-03-23T09:41:41.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.130+0000 7fe4aea69640 1 -- 172.21.15.47:0/4092760249 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a8072360 con 0x7fe4a80b8350 2024-03-23T09:41:41.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.130+0000 7fe49d7fa640 1 -- 172.21.15.47:0/4092760249 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe4a4007490 con 0x7fe4a80b8350 2024-03-23T09:41:41.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.130+0000 7fe49d7fa640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe49406ac00 0x7fe49406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.131+0000 7fe49d7fa640 1 -- 172.21.15.47:0/4092760249 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4a4088700 con 0x7fe4a80b8350 2024-03-23T09:41:41.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.131+0000 7fe49ffff640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe49406ac00 0x7fe49406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.131+0000 7fe49ffff640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe49406ac00 0x7fe49406d0c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80c9950 tx=0x7fe498008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:41.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.135+0000 7fe49d7fa640 1 -- 172.21.15.47:0/4092760249 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe4a4053a20 con 0x7fe4a80b8350 2024-03-23T09:41:41.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.313+0000 7fe4aea69640 1 -- 172.21.15.47:0/4092760249 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4a8066be0 con 0x7fe4a80b8350 2024-03-23T09:41:41.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.314+0000 7fe49d7fa640 1 -- 172.21.15.47:0/4092760249 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4a4057040 con 0x7fe4a80b8350 2024-03-23T09:41:41.313 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.315+0000 7fe4aea69640 1 -- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe49406ac00 msgr2=0x7fe49406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.315+0000 7fe4aea69640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe49406ac00 0x7fe49406d0c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80c9950 tx=0x7fe498008040 comp rx=0 tx=0).stop 2024-03-23T09:41:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.315+0000 7fe4aea69640 1 -- 172.21.15.47:0/4092760249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a80b8350 msgr2=0x7fe4a815dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.315+0000 7fe4aea69640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a80b8350 0x7fe4a815dcf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400c4a0 tx=0x7fe4a400c960 comp rx=0 tx=0).stop 2024-03-23T09:41:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.316+0000 7fe4aea69640 1 -- 172.21.15.47:0/4092760249 shutdown_connections 2024-03-23T09:41:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.316+0000 7fe4aea69640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe49406ac00 0x7fe49406d0c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.316+0000 7fe4aea69640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a815c990 0x7fe4a8153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.316+0000 7fe4aea69640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a80cde10 0x7fe4a80c7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.316+0000 7fe4aea69640 1 --2- 172.21.15.47:0/4092760249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a80b8350 0x7fe4a815dcf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.316+0000 7fe4aea69640 1 -- 172.21.15.47:0/4092760249 >> 172.21.15.47:0/4092760249 conn(0x7fe4a80659d0 msgr2=0x7fe4a80b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.316+0000 7fe4aea69640 1 -- 172.21.15.47:0/4092760249 shutdown_connections 2024-03-23T09:41:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.316+0000 7fe4aea69640 1 -- 172.21.15.47:0/4092760249 wait complete. 2024-03-23T09:41:41.326 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:41:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.428+0000 7f1471c77640 1 Processor -- start 2024-03-23T09:41:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.429+0000 7f1471c77640 1 -- start start 2024-03-23T09:41:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.429+0000 7f1471c77640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f146c15c8c0 0x7f146c0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.429+0000 7f1471c77640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f146c06b400 0x7f146c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.429+0000 7f1471c77640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146c0ccea0 con 0x7f146c15cca0 2024-03-23T09:41:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.429+0000 7f1471c77640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146c0719a0 con 0x7f146c06b400 2024-03-23T09:41:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.429+0000 7f1471c77640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0cb650 con 0x7f146c15c8c0 2024-03-23T09:41:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.429+0000 7f146affd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f146c15cca0 0x7f146c06acf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51194/0 (socket says 172.21.15.47:51194) 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.429+0000 7f146affd640 1 -- 172.21.15.47:0/2533022359 learned_addr learned my addr 172.21.15.47:0/2533022359 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146b7fe640 1 --2- 172.21.15.47:0/2533022359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f146c15c8c0 0x7f146c0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146bfff640 1 --2- 172.21.15.47:0/2533022359 >> v2:172.21.15.175:3300/0 conn(0x7f146c06b400 0x7f146c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902301026 0 0) 0x7f1460001110 con 0x7f146c15cca0 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1454003520 con 0x7f146c15cca0 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2072500002 0 0) 0x7f1454003520 con 0x7f146c15cca0 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f146c15c8c0 msgr2=0x7f146c0cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 --2- 172.21.15.47:0/2533022359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f146c15c8c0 0x7f146c0cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 >> v2:172.21.15.175:3300/0 conn(0x7f146c06b400 msgr2=0x7f146c06cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 --2- 172.21.15.47:0/2533022359 >> v2:172.21.15.175:3300/0 conn(0x7f146c06b400 0x7f146c06cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c06d070 con 0x7f146c15cca0 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f1460001800 con 0x7f146c15cca0 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146bfff640 1 --2- 172.21.15.47:0/2533022359 >> v2:172.21.15.175:3300/0 conn(0x7f146c06b400 0x7f146c06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 >> v1:172.21.15.47:6789/0 conn(0x7f146c15cca0 legacy=0x7f146c06acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 --2- 172.21.15.47:0/2533022359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f146c15c8c0 0x7f1454003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 --2- 172.21.15.47:0/2533022359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1454003ef0 0x7f14540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 --2- 172.21.15.47:0/2533022359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1454004d00 0x7f1454005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0719a0 con 0x7f1454003ef0 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0cb650 con 0x7f1454004d00 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145400b980 con 0x7f146c15c8c0 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146affd640 1 --2- 172.21.15.47:0/2533022359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1454003ef0 0x7f14540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146b7fe640 1 --2- 172.21.15.47:0/2533022359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f146c15c8c0 0x7f1454003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146bfff640 1 --2- 172.21.15.47:0/2533022359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1454004d00 0x7f1454005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.430+0000 7f146affd640 1 -- 172.21.15.47:0/2533022359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f146c15c8c0 msgr2=0x7f1454003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.431+0000 7f146affd640 1 --2- 172.21.15.47:0/2533022359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f146c15c8c0 0x7f1454003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.431+0000 7f146affd640 1 -- 172.21.15.47:0/2533022359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1454004d00 msgr2=0x7f1454005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.431+0000 7f146affd640 1 --2- 172.21.15.47:0/2533022359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1454004d00 0x7f1454005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.431+0000 7f146affd640 1 -- 172.21.15.47:0/2533022359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1454003520 con 0x7f1454003ef0 2024-03-23T09:41:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.431+0000 7f146affd640 1 --2- 172.21.15.47:0/2533022359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1454003ef0 0x7f14540047c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f146c15d080 tx=0x7f14540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b958b8f3fa58666 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:41.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.431+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460005ad0 con 0x7f1454003ef0 2024-03-23T09:41:41.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.431+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1460005c30 con 0x7f1454003ef0 2024-03-23T09:41:41.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.431+0000 7f146a7fc640 1 -- 172.21.15.47:0/2533022359 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146002c880 con 0x7f1454003ef0 2024-03-23T09:41:41.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.432+0000 7f1471c77640 1 -- 172.21.15.47:0/2533022359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1454003ef0 msgr2=0x7f14540047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.432+0000 7f1471c77640 1 --2- 172.21.15.47:0/2533022359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1454003ef0 0x7f14540047c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f146c15d080 tx=0x7f14540057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.432+0000 7f146bfff640 1 -- 172.21.15.47:0/2533022359 reap_dead start 2024-03-23T09:41:41.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.432+0000 7f1471c77640 1 -- 172.21.15.47:0/2533022359 shutdown_connections 2024-03-23T09:41:41.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.432+0000 7f1471c77640 1 -- 172.21.15.47:0/2533022359 >> 172.21.15.47:0/2533022359 conn(0x7f146c070ad0 msgr2=0x7f146c070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:41.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.432+0000 7f1471c77640 1 -- 172.21.15.47:0/2533022359 shutdown_connections 2024-03-23T09:41:41.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.432+0000 7f1471c77640 1 -- 172.21.15.47:0/2533022359 wait complete. 2024-03-23T09:41:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.433+0000 7f1471c77640 1 Processor -- start 2024-03-23T09:41:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.433+0000 7f1471c77640 1 -- start start 2024-03-23T09:41:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f1471c77640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1454003ef0 0x7f146c15dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f1471c77640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f146c15f960 0x7f146c15fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f1471c77640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f146c1f1540 0x7f146c1f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f1471c77640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c160320 con 0x7f1454003ef0 2024-03-23T09:41:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f1471c77640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0add40 con 0x7f146c1f1540 2024-03-23T09:41:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f1471c77640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c1578a0 con 0x7f146c15f960 2024-03-23T09:41:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f146b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1454003ef0 0x7f146c15dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f146b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1454003ef0 0x7f146c15dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50330/0 (socket says 172.21.15.47:50330) 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f146b7fe640 1 -- 172.21.15.47:0/3028340359 learned_addr learned my addr 172.21.15.47:0/3028340359 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f146bfff640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f146c1f1540 0x7f146c1f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f146affd640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f146c15f960 0x7f146c15fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f146b7fe640 1 -- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f146c15f960 msgr2=0x7f146c15fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f146b7fe640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f146c15f960 0x7f146c15fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f146b7fe640 1 -- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f146c1f1540 msgr2=0x7f146c1f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f146b7fe640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f146c1f1540 0x7f146c1f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f146b7fe640 1 -- 172.21.15.47:0/3028340359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c1f1f00 con 0x7f1454003ef0 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.434+0000 7f146bfff640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f146c1f1540 0x7f146c1f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.435+0000 7f146b7fe640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1454003ef0 0x7f146c15dd00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f145c0119a0 tx=0x7f145c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.435+0000 7f1453fff640 1 -- 172.21.15.47:0/3028340359 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c00faf0 con 0x7f1454003ef0 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.435+0000 7f1453fff640 1 -- 172.21.15.47:0/3028340359 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c00fc50 con 0x7f1454003ef0 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.435+0000 7f1471c77640 1 -- 172.21.15.47:0/3028340359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c0ae510 con 0x7f1454003ef0 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.435+0000 7f1453fff640 1 -- 172.21.15.47:0/3028340359 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c014440 con 0x7f1454003ef0 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.435+0000 7f1471c77640 1 -- 172.21.15.47:0/3028340359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c0ae230 con 0x7f1454003ef0 2024-03-23T09:41:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.436+0000 7f1471c77640 1 -- 172.21.15.47:0/3028340359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c0672f0 con 0x7f1454003ef0 2024-03-23T09:41:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.436+0000 7f1453fff640 1 -- 172.21.15.47:0/3028340359 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f145c0145a0 con 0x7f1454003ef0 2024-03-23T09:41:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.436+0000 7f1453fff640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f145806ac30 0x7f145806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.437+0000 7f1453fff640 1 -- 172.21.15.47:0/3028340359 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f145c089b10 con 0x7f1454003ef0 2024-03-23T09:41:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.437+0000 7f146affd640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f145806ac30 0x7f145806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.437+0000 7f146affd640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f145806ac30 0x7f145806d0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1460034000 tx=0x7f1460007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.441+0000 7f1453fff640 1 -- 172.21.15.47:0/3028340359 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f145c054cf0 con 0x7f1454003ef0 2024-03-23T09:41:41.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.635+0000 7f1471c77640 1 -- 172.21.15.47:0/3028340359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f146c065de0 con 0x7f1454003ef0 2024-03-23T09:41:41.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.635+0000 7f1453fff640 1 -- 172.21.15.47:0/3028340359 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f145c058310 con 0x7f1454003ef0 2024-03-23T09:41:41.635 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:41:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.637+0000 7f1471c77640 1 -- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f145806ac30 msgr2=0x7f145806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.637+0000 7f1471c77640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f145806ac30 0x7f145806d0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1460034000 tx=0x7f1460007000 comp rx=0 tx=0).stop 2024-03-23T09:41:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.637+0000 7f1471c77640 1 -- 172.21.15.47:0/3028340359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1454003ef0 msgr2=0x7f146c15dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.637+0000 7f1471c77640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1454003ef0 0x7f146c15dd00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f145c0119a0 tx=0x7f145c011e60 comp rx=0 tx=0).stop 2024-03-23T09:41:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.638+0000 7f1471c77640 1 -- 172.21.15.47:0/3028340359 shutdown_connections 2024-03-23T09:41:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.638+0000 7f1471c77640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f145806ac30 0x7f145806d0f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.638+0000 7f1471c77640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f146c1f1540 0x7f146c1f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.638+0000 7f1471c77640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f146c15f960 0x7f146c15fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.638+0000 7f1471c77640 1 --2- 172.21.15.47:0/3028340359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1454003ef0 0x7f146c15dd00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.638+0000 7f1471c77640 1 -- 172.21.15.47:0/3028340359 >> 172.21.15.47:0/3028340359 conn(0x7f146c070ad0 msgr2=0x7f146c069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.638+0000 7f1471c77640 1 -- 172.21.15.47:0/3028340359 shutdown_connections 2024-03-23T09:41:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.638+0000 7f1471c77640 1 -- 172.21.15.47:0/3028340359 wait complete. 2024-03-23T09:41:41.648 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:41:41.648 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:41:41.663 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:41.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.804+0000 7f1209541640 1 Processor -- start 2024-03-23T09:41:41.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.804+0000 7f1209541640 1 -- start start 2024-03-23T09:41:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f1209541640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f12040b8350 0x7f12040cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f1209541640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12040cdc40 0x7f12040d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f1209541640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12040d11e0 con 0x7f120415ca90 2024-03-23T09:41:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f1209541640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12040ce0d0 con 0x7f12040b8350 2024-03-23T09:41:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f1209541640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1204160f60 con 0x7f12040cdc40 2024-03-23T09:41:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f11fbfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f12040b8350 0x7f12040cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f1202ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f120415ca90 0x7f1204160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51200/0 (socket says 172.21.15.47:51200) 2024-03-23T09:41:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f1202ffd640 1 -- 172.21.15.47:0/946946761 learned_addr learned my addr 172.21.15.47:0/946946761 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f12037fe640 1 --2- 172.21.15.47:0/946946761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12040cdc40 0x7f12040d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f11fbfff640 1 -- 172.21.15.47:0/946946761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12040cdc40 msgr2=0x7f12040d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f11fbfff640 1 --2- 172.21.15.47:0/946946761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12040cdc40 0x7f12040d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f11fbfff640 1 -- 172.21.15.47:0/946946761 >> v1:172.21.15.47:6789/0 conn(0x7f120415ca90 legacy=0x7f1204160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f12027fc640 1 -- 172.21.15.47:0/946946761 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510237513 0 0) 0x7f12040d11e0 con 0x7f120415ca90 2024-03-23T09:41:41.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.805+0000 7f11fbfff640 1 -- 172.21.15.47:0/946946761 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1204161240 con 0x7f12040b8350 2024-03-23T09:41:41.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.806+0000 7f11fbfff640 1 --2- 172.21.15.47:0/946946761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12040b8350 0x7f12040cd700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0097a0 tx=0x7f11ec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d11dcb68b58981 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:41.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.806+0000 7f12027fc640 1 -- 172.21.15.47:0/946946761 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec02e070 con 0x7f12040b8350 2024-03-23T09:41:41.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.806+0000 7f12027fc640 1 -- 172.21.15.47:0/946946761 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec0315a0 con 0x7f12040b8350 2024-03-23T09:41:41.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.806+0000 7f12027fc640 1 -- 172.21.15.47:0/946946761 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec031930 con 0x7f12040b8350 2024-03-23T09:41:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.807+0000 7f1209541640 1 -- 172.21.15.47:0/946946761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12040b8350 msgr2=0x7f12040cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.807+0000 7f1209541640 1 --2- 172.21.15.47:0/946946761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12040b8350 0x7f12040cd700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0097a0 tx=0x7f11ec02b840 comp rx=0 tx=0).stop 2024-03-23T09:41:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.807+0000 7f1209541640 1 -- 172.21.15.47:0/946946761 shutdown_connections 2024-03-23T09:41:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.807+0000 7f1209541640 1 --2- 172.21.15.47:0/946946761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12040cdc40 0x7f12040d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.807+0000 7f1209541640 1 --2- 172.21.15.47:0/946946761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12040b8350 0x7f12040cd700 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.807+0000 7f1209541640 1 -- 172.21.15.47:0/946946761 >> 172.21.15.47:0/946946761 conn(0x7f12040659d0 msgr2=0x7f1204065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.807+0000 7f1209541640 1 -- 172.21.15.47:0/946946761 shutdown_connections 2024-03-23T09:41:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.807+0000 7f1209541640 1 -- 172.21.15.47:0/946946761 wait complete. 2024-03-23T09:41:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.808+0000 7f1209541640 1 Processor -- start 2024-03-23T09:41:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.808+0000 7f1209541640 1 -- start start 2024-03-23T09:41:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1209541640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12040b8350 0x7f1204153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1209541640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12040cdc40 0x7f12040b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1209541640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f120415ca90 0x7f12040cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1209541640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12041f6950 con 0x7f12040b8350 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1209541640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12040b5360 con 0x7f12040cdc40 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1209541640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12041f5cd0 con 0x7f120415ca90 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1202ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12040b8350 0x7f1204153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1202ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12040b8350 0x7f1204153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50344/0 (socket says 172.21.15.47:50344) 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1202ffd640 1 -- 172.21.15.47:0/4108815244 learned_addr learned my addr 172.21.15.47:0/4108815244 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f11fbfff640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12040cdc40 0x7f12040b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f12037fe640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f120415ca90 0x7f12040cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1202ffd640 1 -- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f120415ca90 msgr2=0x7f12040cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1202ffd640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f120415ca90 0x7f12040cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1202ffd640 1 -- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12040cdc40 msgr2=0x7f12040b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1202ffd640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12040cdc40 0x7f12040b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1202ffd640 1 -- 172.21.15.47:0/4108815244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec009400 con 0x7f12040b8350 2024-03-23T09:41:41.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f12037fe640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f120415ca90 0x7f12040cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:41.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1202ffd640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12040b8350 0x7f1204153db0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f11f400c4d0 tx=0x7f11f400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:41.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f11fb7fe640 1 -- 172.21.15.47:0/4108815244 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f40074e0 con 0x7f12040b8350 2024-03-23T09:41:41.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f11fb7fe640 1 -- 172.21.15.47:0/4108815244 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4007660 con 0x7f12040b8350 2024-03-23T09:41:41.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1209541640 1 -- 172.21.15.47:0/4108815244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12041f5fb0 con 0x7f12040b8350 2024-03-23T09:41:41.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f11fb7fe640 1 -- 172.21.15.47:0/4108815244 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4004a20 con 0x7f12040b8350 2024-03-23T09:41:41.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.809+0000 7f1209541640 1 -- 172.21.15.47:0/4108815244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12041f6570 con 0x7f12040b8350 2024-03-23T09:41:41.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.811+0000 7f1209541640 1 -- 172.21.15.47:0/4108815244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1204072360 con 0x7f12040b8350 2024-03-23T09:41:41.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.811+0000 7f11fb7fe640 1 -- 172.21.15.47:0/4108815244 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f11f4004b80 con 0x7f12040b8350 2024-03-23T09:41:41.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.811+0000 7f11fb7fe640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f11e406ac00 0x7f11e406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:41.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.811+0000 7f11fb7fe640 1 -- 172.21.15.47:0/4108815244 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11f40887c0 con 0x7f12040b8350 2024-03-23T09:41:41.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.812+0000 7f11fbfff640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f11e406ac00 0x7f11e406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:41.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.812+0000 7f11fbfff640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f11e406ac00 0x7f11e406d0c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11ec036000 tx=0x7f11ec03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:41.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.816+0000 7f11fb7fe640 1 -- 172.21.15.47:0/4108815244 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f11f4053ae0 con 0x7f12040b8350 2024-03-23T09:41:41.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.996+0000 7f1209541640 1 -- 172.21.15.47:0/4108815244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12041f6a90 con 0x7f12040b8350 2024-03-23T09:41:41.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.997+0000 7f11fb7fe640 1 -- 172.21.15.47:0/4108815244 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f12041f6a90 con 0x7f12040b8350 2024-03-23T09:41:41.997 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:41.997 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.999+0000 7f1209541640 1 -- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f11e406ac00 msgr2=0x7f11e406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.999+0000 7f1209541640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f11e406ac00 0x7f11e406d0c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11ec036000 tx=0x7f11ec03d040 comp rx=0 tx=0).stop 2024-03-23T09:41:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.999+0000 7f1209541640 1 -- 172.21.15.47:0/4108815244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12040b8350 msgr2=0x7f1204153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.999+0000 7f1209541640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12040b8350 0x7f1204153db0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f11f400c4d0 tx=0x7f11f400c990 comp rx=0 tx=0).stop 2024-03-23T09:41:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.999+0000 7f1209541640 1 -- 172.21.15.47:0/4108815244 shutdown_connections 2024-03-23T09:41:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.999+0000 7f1209541640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f11e406ac00 0x7f11e406d0c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.999+0000 7f1209541640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f120415ca90 0x7f12040cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.999+0000 7f1209541640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12040cdc40 0x7f12040b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.999+0000 7f1209541640 1 --2- 172.21.15.47:0/4108815244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12040b8350 0x7f1204153db0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.999+0000 7f1209541640 1 -- 172.21.15.47:0/4108815244 >> 172.21.15.47:0/4108815244 conn(0x7f12040659d0 msgr2=0x7f120415fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.999+0000 7f1209541640 1 -- 172.21.15.47:0/4108815244 shutdown_connections 2024-03-23T09:41:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:41.999+0000 7f1209541640 1 -- 172.21.15.47:0/4108815244 wait complete. 2024-03-23T09:41:42.009 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:42.010 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.114+0000 7f6affdc9640 1 Processor -- start 2024-03-23T09:41:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.114+0000 7f6affdc9640 1 -- start start 2024-03-23T09:41:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.114+0000 7f6affdc9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6af815c990 0x7f6af8160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.114+0000 7f6affdc9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6af80b8350 0x7f6af80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.114+0000 7f6affdc9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af8161630 con 0x7f6af80cdc40 2024-03-23T09:41:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.115+0000 7f6affdc9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af80d1030 con 0x7f6af80b8350 2024-03-23T09:41:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.115+0000 7f6affdc9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af80ce070 con 0x7f6af815c990 2024-03-23T09:41:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.115+0000 7f6afd364640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6af80b8350 0x7f6af80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.115+0000 7f6afd364640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6af80b8350 0x7f6af80cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50346/0 (socket says 172.21.15.47:50346) 2024-03-23T09:41:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.115+0000 7f6afd364640 1 -- 172.21.15.47:0/1427947483 learned_addr learned my addr 172.21.15.47:0/1427947483 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.115+0000 7f6afdb65640 1 --2- 172.21.15.47:0/1427947483 >> v2:172.21.15.175:3300/0 conn(0x7f6af815c990 0x7f6af8160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.115+0000 7f6afd364640 1 -- 172.21.15.47:0/1427947483 >> v2:172.21.15.175:3300/0 conn(0x7f6af815c990 msgr2=0x7f6af8160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:42.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.115+0000 7f6afd364640 1 --2- 172.21.15.47:0/1427947483 >> v2:172.21.15.175:3300/0 conn(0x7f6af815c990 0x7f6af8160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.115+0000 7f6afd364640 1 -- 172.21.15.47:0/1427947483 >> v1:172.21.15.175:6789/0 conn(0x7f6af80cdc40 legacy=0x7f6af80b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.115+0000 7f6afd364640 1 -- 172.21.15.47:0/1427947483 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af80cfe10 con 0x7f6af80b8350 2024-03-23T09:41:42.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.115+0000 7f6afdb65640 1 --2- 172.21.15.47:0/1427947483 >> v2:172.21.15.175:3300/0 conn(0x7f6af815c990 0x7f6af8160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:42.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.116+0000 7f6afd364640 1 --2- 172.21.15.47:0/1427947483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af80b8350 0x7f6af80cd700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6af80b8b30 tx=0x7f6ae802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb68c1b047681bf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:42.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.116+0000 7f6afcb63640 1 -- 172.21.15.47:0/1427947483 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8031870 con 0x7f6af80b8350 2024-03-23T09:41:42.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.116+0000 7f6afcb63640 1 -- 172.21.15.47:0/1427947483 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae80319d0 con 0x7f6af80b8350 2024-03-23T09:41:42.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.116+0000 7f6afcb63640 1 -- 172.21.15.47:0/1427947483 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8031d60 con 0x7f6af80b8350 2024-03-23T09:41:42.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.116+0000 7f6affdc9640 1 -- 172.21.15.47:0/1427947483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af80b8350 msgr2=0x7f6af80cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.116+0000 7f6affdc9640 1 --2- 172.21.15.47:0/1427947483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af80b8350 0x7f6af80cd700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6af80b8b30 tx=0x7f6ae802b910 comp rx=0 tx=0).stop 2024-03-23T09:41:42.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.116+0000 7f6affdc9640 1 -- 172.21.15.47:0/1427947483 shutdown_connections 2024-03-23T09:41:42.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.116+0000 7f6affdc9640 1 --2- 172.21.15.47:0/1427947483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af80b8350 0x7f6af80cd700 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.116+0000 7f6affdc9640 1 --2- 172.21.15.47:0/1427947483 >> v2:172.21.15.175:3300/0 conn(0x7f6af815c990 0x7f6af8160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.116+0000 7f6affdc9640 1 -- 172.21.15.47:0/1427947483 >> 172.21.15.47:0/1427947483 conn(0x7f6af80659d0 msgr2=0x7f6af8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:42.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.117+0000 7f6affdc9640 1 -- 172.21.15.47:0/1427947483 shutdown_connections 2024-03-23T09:41:42.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.117+0000 7f6affdc9640 1 -- 172.21.15.47:0/1427947483 wait complete. 2024-03-23T09:41:42.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.118+0000 7f6affdc9640 1 Processor -- start 2024-03-23T09:41:42.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.118+0000 7f6affdc9640 1 -- start start 2024-03-23T09:41:42.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.118+0000 7f6affdc9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af80b8350 0x7f6af81f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.118+0000 7f6affdc9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af80cdc40 0x7f6af80c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.118+0000 7f6affdc9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6af815c990 0x7f6af81586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.118+0000 7f6affdc9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af81f2590 con 0x7f6af80b8350 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.118+0000 7f6affdc9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af81f2870 con 0x7f6af80cdc40 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.118+0000 7f6affdc9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af81f2b50 con 0x7f6af815c990 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.118+0000 7f6afdb65640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af80b8350 0x7f6af81f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.118+0000 7f6afdb65640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af80b8350 0x7f6af81f1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50360/0 (socket says 172.21.15.47:50360) 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.118+0000 7f6afdb65640 1 -- 172.21.15.47:0/3923733651 learned_addr learned my addr 172.21.15.47:0/3923733651 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.119+0000 7f6afdb65640 1 -- 172.21.15.47:0/3923733651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6af815c990 msgr2=0x7f6af81586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.119+0000 7f6afdb65640 1 --2- 172.21.15.47:0/3923733651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6af815c990 0x7f6af81586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.119+0000 7f6afdb65640 1 -- 172.21.15.47:0/3923733651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af80cdc40 msgr2=0x7f6af80c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.119+0000 7f6afdb65640 1 --2- 172.21.15.47:0/3923733651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af80cdc40 0x7f6af80c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.119+0000 7f6afdb65640 1 -- 172.21.15.47:0/3923733651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae80094d0 con 0x7f6af80b8350 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.119+0000 7f6afdb65640 1 --2- 172.21.15.47:0/3923733651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af80b8350 0x7f6af81f1f40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6af400d7d0 tx=0x7f6af400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.119+0000 7f6aee7fc640 1 -- 172.21.15.47:0/3923733651 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4004030 con 0x7f6af80b8350 2024-03-23T09:41:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.119+0000 7f6aee7fc640 1 -- 172.21.15.47:0/3923733651 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af4004190 con 0x7f6af80b8350 2024-03-23T09:41:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.119+0000 7f6affdc9640 1 -- 172.21.15.47:0/3923733651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af81f2e30 con 0x7f6af80b8350 2024-03-23T09:41:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.119+0000 7f6aee7fc640 1 -- 172.21.15.47:0/3923733651 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4004520 con 0x7f6af80b8350 2024-03-23T09:41:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.119+0000 7f6affdc9640 1 -- 172.21.15.47:0/3923733651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af80c90f0 con 0x7f6af80b8350 2024-03-23T09:41:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.120+0000 7f6affdc9640 1 -- 172.21.15.47:0/3923733651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af80c92e0 con 0x7f6af80b8350 2024-03-23T09:41:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.121+0000 7f6aee7fc640 1 -- 172.21.15.47:0/3923733651 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6af40096a0 con 0x7f6af80b8350 2024-03-23T09:41:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.121+0000 7f6aee7fc640 1 --2- 172.21.15.47:0/3923733651 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6adc06ac00 0x7f6adc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.121+0000 7f6aee7fc640 1 -- 172.21.15.47:0/3923733651 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6af4089460 con 0x7f6af80b8350 2024-03-23T09:41:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.121+0000 7f6afd364640 1 --2- 172.21.15.47:0/3923733651 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6adc06ac00 0x7f6adc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.122+0000 7f6afd364640 1 --2- 172.21.15.47:0/3923733651 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6adc06ac00 0x7f6adc06d0c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8036000 tx=0x7f6ae8030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:42.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.125+0000 7f6aee7fc640 1 -- 172.21.15.47:0/3923733651 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6af40546d0 con 0x7f6af80b8350 2024-03-23T09:41:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.304+0000 7f6affdc9640 1 -- 172.21.15.47:0/3923733651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6af8072360 con 0x7f6af80b8350 2024-03-23T09:41:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.305+0000 7f6aee7fc640 1 -- 172.21.15.47:0/3923733651 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6af4057cf0 con 0x7f6af80b8350 2024-03-23T09:41:42.306 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.307+0000 7f6affdc9640 1 -- 172.21.15.47:0/3923733651 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6adc06ac00 msgr2=0x7f6adc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.307+0000 7f6affdc9640 1 --2- 172.21.15.47:0/3923733651 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6adc06ac00 0x7f6adc06d0c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8036000 tx=0x7f6ae8030000 comp rx=0 tx=0).stop 2024-03-23T09:41:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.307+0000 7f6affdc9640 1 -- 172.21.15.47:0/3923733651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af80b8350 msgr2=0x7f6af81f1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.307+0000 7f6affdc9640 1 --2- 172.21.15.47:0/3923733651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af80b8350 0x7f6af81f1f40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6af400d7d0 tx=0x7f6af400dc90 comp rx=0 tx=0).stop 2024-03-23T09:41:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.307+0000 7f6affdc9640 1 -- 172.21.15.47:0/3923733651 shutdown_connections 2024-03-23T09:41:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.307+0000 7f6affdc9640 1 --2- 172.21.15.47:0/3923733651 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6adc06ac00 0x7f6adc06d0c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.307+0000 7f6affdc9640 1 --2- 172.21.15.47:0/3923733651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6af815c990 0x7f6af81586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.307+0000 7f6affdc9640 1 --2- 172.21.15.47:0/3923733651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af80cdc40 0x7f6af80c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.307+0000 7f6affdc9640 1 --2- 172.21.15.47:0/3923733651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af80b8350 0x7f6af81f1f40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.307+0000 7f6affdc9640 1 -- 172.21.15.47:0/3923733651 >> 172.21.15.47:0/3923733651 conn(0x7f6af80659d0 msgr2=0x7f6af80b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.307+0000 7f6affdc9640 1 -- 172.21.15.47:0/3923733651 shutdown_connections 2024-03-23T09:41:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.307+0000 7f6affdc9640 1 -- 172.21.15.47:0/3923733651 wait complete. 2024-03-23T09:41:42.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:42.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.421+0000 7fa2dbd88640 1 Processor -- start 2024-03-23T09:41:42.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.421+0000 7fa2dbd88640 1 -- start start 2024-03-23T09:41:42.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2dbd88640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa2d415c990 0x7fa2d4160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2dbd88640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d4161630 con 0x7fa2d40b8350 2024-03-23T09:41:42.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2dbd88640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d40d1030 con 0x7fa2d40cde10 2024-03-23T09:41:42.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2dbd88640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d40cfe10 con 0x7fa2d415c990 2024-03-23T09:41:42.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2d9323640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa2d40b8350 0x7fa2d40cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51210/0 (socket says 172.21.15.47:51210) 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2d9323640 1 -- 172.21.15.47:0/2096541732 learned_addr learned my addr 172.21.15.47:0/2096541732 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2d9b24640 1 --2- 172.21.15.47:0/2096541732 >> v2:172.21.15.175:3300/0 conn(0x7fa2d415c990 0x7fa2d4160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 465030942 0 0) 0x7fa2d4161630 con 0x7fa2d40b8350 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c0003520 con 0x7fa2d40b8350 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3450237229 0 0) 0x7fa2c0003520 con 0x7fa2d40b8350 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 >> v2:172.21.15.175:3300/0 conn(0x7fa2d415c990 msgr2=0x7fa2d4160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2d8b22640 1 --2- 172.21.15.47:0/2096541732 >> v2:172.21.15.175:3300/0 conn(0x7fa2d415c990 0x7fa2d4160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 >> v1:172.21.15.175:6789/0 conn(0x7fa2d40cde10 legacy=0x7fa2d40d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d40b5b60 con 0x7fa2d40b8350 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fa2c4001670 con 0x7fa2d40b8350 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.422+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 >> v1:172.21.15.47:6789/0 conn(0x7fa2d40b8350 legacy=0x7fa2d40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2d8b22640 1 --2- 172.21.15.47:0/2096541732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2c0003970 0x7fa2c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2d8b22640 1 --2- 172.21.15.47:0/2096541732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c00047c0 0x7fa2c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2d8b22640 1 --2- 172.21.15.47:0/2096541732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2c0005180 0x7fa2c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d40cfe10 con 0x7fa2c0005180 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c000c120 con 0x7fa2c00047c0 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c000c2e0 con 0x7fa2c0003970 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2da325640 1 --2- 172.21.15.47:0/2096541732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2c0005180 0x7fa2c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2d9323640 1 --2- 172.21.15.47:0/2096541732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c00047c0 0x7fa2c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2d9b24640 1 --2- 172.21.15.47:0/2096541732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2c0003970 0x7fa2c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2da325640 1 -- 172.21.15.47:0/2096541732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2c0003970 msgr2=0x7fa2c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2da325640 1 --2- 172.21.15.47:0/2096541732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2c0003970 0x7fa2c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2da325640 1 -- 172.21.15.47:0/2096541732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c00047c0 msgr2=0x7fa2c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2da325640 1 --2- 172.21.15.47:0/2096541732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c00047c0 0x7fa2c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.423+0000 7fa2da325640 1 -- 172.21.15.47:0/2096541732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c0003520 con 0x7fa2c0005180 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.424+0000 7fa2da325640 1 -- 172.21.15.47:0/2096541732 reap_dead start 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.424+0000 7fa2da325640 1 --2- 172.21.15.47:0/2096541732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2c0005180 0x7fa2c000b980 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc001e50 tx=0x7fa2c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da1229d654cf6d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.424+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc003b70 con 0x7fa2c0005180 2024-03-23T09:41:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.424+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc003d10 con 0x7fa2c0005180 2024-03-23T09:41:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.424+0000 7fa2d8b22640 1 -- 172.21.15.47:0/2096541732 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc005310 con 0x7fa2c0005180 2024-03-23T09:41:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.424+0000 7fa2dbd88640 1 -- 172.21.15.47:0/2096541732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2c0005180 msgr2=0x7fa2c000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.424+0000 7fa2dbd88640 1 --2- 172.21.15.47:0/2096541732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2c0005180 0x7fa2c000b980 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc001e50 tx=0x7fa2c00057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.424+0000 7fa2dbd88640 1 -- 172.21.15.47:0/2096541732 shutdown_connections 2024-03-23T09:41:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.424+0000 7fa2dbd88640 1 --2- 172.21.15.47:0/2096541732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2c0005180 0x7fa2c000b980 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.424+0000 7fa2dbd88640 1 -- 172.21.15.47:0/2096541732 >> 172.21.15.47:0/2096541732 conn(0x7fa2d40659d0 msgr2=0x7fa2d4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.425+0000 7fa2dbd88640 1 -- 172.21.15.47:0/2096541732 shutdown_connections 2024-03-23T09:41:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.425+0000 7fa2dbd88640 1 -- 172.21.15.47:0/2096541732 wait complete. 2024-03-23T09:41:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.426+0000 7fa2dbd88640 1 Processor -- start 2024-03-23T09:41:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.426+0000 7fa2dbd88640 1 -- start start 2024-03-23T09:41:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.426+0000 7fa2dbd88640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2c0005180 0x7fa2d4150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.426+0000 7fa2dbd88640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2d415be80 0x7fa2d40caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.426+0000 7fa2dbd88640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d40cb020 0x7fa2d40cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.426+0000 7fa2dbd88640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d4158740 con 0x7fa2c0005180 2024-03-23T09:41:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.426+0000 7fa2dbd88640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d41545d0 con 0x7fa2d40cb020 2024-03-23T09:41:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.426+0000 7fa2dbd88640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d41548b0 con 0x7fa2d415be80 2024-03-23T09:41:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.426+0000 7fa2d9b24640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2c0005180 0x7fa2d4150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.427+0000 7fa2d9b24640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2c0005180 0x7fa2d4150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50378/0 (socket says 172.21.15.47:50378) 2024-03-23T09:41:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.427+0000 7fa2d9b24640 1 -- 172.21.15.47:0/1366609282 learned_addr learned my addr 172.21.15.47:0/1366609282 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.427+0000 7fa2d9b24640 1 -- 172.21.15.47:0/1366609282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2d415be80 msgr2=0x7fa2d40caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.427+0000 7fa2d9b24640 1 --2- 172.21.15.47:0/1366609282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2d415be80 0x7fa2d40caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.427+0000 7fa2d9b24640 1 -- 172.21.15.47:0/1366609282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d40cb020 msgr2=0x7fa2d40cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.427+0000 7fa2d9b24640 1 --2- 172.21.15.47:0/1366609282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d40cb020 0x7fa2d40cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.427+0000 7fa2d9b24640 1 -- 172.21.15.47:0/1366609282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d4158990 con 0x7fa2c0005180 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.427+0000 7fa2d9b24640 1 --2- 172.21.15.47:0/1366609282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2c0005180 0x7fa2d4150e20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00118e0 tx=0x7fa2d0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.427+0000 7fa2ca7fc640 1 -- 172.21.15.47:0/1366609282 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000fa70 con 0x7fa2c0005180 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.427+0000 7fa2ca7fc640 1 -- 172.21.15.47:0/1366609282 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d000fbd0 con 0x7fa2c0005180 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.427+0000 7fa2dbd88640 1 -- 172.21.15.47:0/1366609282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d4158c70 con 0x7fa2c0005180 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.427+0000 7fa2ca7fc640 1 -- 172.21.15.47:0/1366609282 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0014440 con 0x7fa2c0005180 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.427+0000 7fa2dbd88640 1 -- 172.21.15.47:0/1366609282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d4153130 con 0x7fa2c0005180 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.428+0000 7fa2dbd88640 1 -- 172.21.15.47:0/1366609282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d4154a70 con 0x7fa2c0005180 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.428+0000 7fa2ca7fc640 1 -- 172.21.15.47:0/1366609282 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa2d00145a0 con 0x7fa2c0005180 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.429+0000 7fa2ca7fc640 1 --2- 172.21.15.47:0/1366609282 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa2b806ac30 0x7fa2b806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.429+0000 7fa2ca7fc640 1 -- 172.21.15.47:0/1366609282 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2d0089aa0 con 0x7fa2c0005180 2024-03-23T09:41:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.429+0000 7fa2d9323640 1 --2- 172.21.15.47:0/1366609282 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa2b806ac30 0x7fa2b806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.430+0000 7fa2d9323640 1 --2- 172.21.15.47:0/1366609282 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa2b806ac30 0x7fa2b806d0f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40cc690 tx=0x7fa2c4001490 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.433+0000 7fa2ca7fc640 1 -- 172.21.15.47:0/1366609282 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa2d0054d10 con 0x7fa2c0005180 2024-03-23T09:41:42.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.610+0000 7fa2dbd88640 1 -- 172.21.15.47:0/1366609282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa2d4072360 con 0x7fa2c0005180 2024-03-23T09:41:42.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.611+0000 7fa2ca7fc640 1 -- 172.21.15.47:0/1366609282 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa2d0058330 con 0x7fa2c0005180 2024-03-23T09:41:42.610 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.613+0000 7fa2dbd88640 1 -- 172.21.15.47:0/1366609282 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa2b806ac30 msgr2=0x7fa2b806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.613+0000 7fa2dbd88640 1 --2- 172.21.15.47:0/1366609282 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa2b806ac30 0x7fa2b806d0f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40cc690 tx=0x7fa2c4001490 comp rx=0 tx=0).stop 2024-03-23T09:41:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.613+0000 7fa2dbd88640 1 -- 172.21.15.47:0/1366609282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2c0005180 msgr2=0x7fa2d4150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.613+0000 7fa2dbd88640 1 --2- 172.21.15.47:0/1366609282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2c0005180 0x7fa2d4150e20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00118e0 tx=0x7fa2d0011da0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.613+0000 7fa2dbd88640 1 -- 172.21.15.47:0/1366609282 shutdown_connections 2024-03-23T09:41:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.613+0000 7fa2dbd88640 1 --2- 172.21.15.47:0/1366609282 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa2b806ac30 0x7fa2b806d0f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.613+0000 7fa2dbd88640 1 --2- 172.21.15.47:0/1366609282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d40cb020 0x7fa2d40cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.613+0000 7fa2dbd88640 1 --2- 172.21.15.47:0/1366609282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2d415be80 0x7fa2d40caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.613+0000 7fa2dbd88640 1 --2- 172.21.15.47:0/1366609282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2c0005180 0x7fa2d4150e20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.613+0000 7fa2dbd88640 1 -- 172.21.15.47:0/1366609282 >> 172.21.15.47:0/1366609282 conn(0x7fa2d40659d0 msgr2=0x7fa2d4157440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.613+0000 7fa2dbd88640 1 -- 172.21.15.47:0/1366609282 shutdown_connections 2024-03-23T09:41:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.613+0000 7fa2dbd88640 1 -- 172.21.15.47:0/1366609282 wait complete. 2024-03-23T09:41:42.623 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:42.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.727+0000 7ff7837c3640 1 Processor -- start 2024-03-23T09:41:42.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.727+0000 7ff7837c3640 1 -- start start 2024-03-23T09:41:42.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff7837c3640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff77c15c990 0x7ff77c160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff7837c3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff77c0cde10 0x7ff77c0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff7837c3640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77c161630 con 0x7ff77c0b8350 2024-03-23T09:41:42.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff7837c3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77c0d1030 con 0x7ff77c0cde10 2024-03-23T09:41:42.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff7837c3640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77c0cfe10 con 0x7ff77c15c990 2024-03-23T09:41:42.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff781d60640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff77c0cde10 0x7ff77c0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff780d5e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff77c0b8350 0x7ff77c0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51214/0 (socket says 172.21.15.47:51214) 2024-03-23T09:41:42.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff780d5e640 1 -- 172.21.15.47:0/1422476782 learned_addr learned my addr 172.21.15.47:0/1422476782 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:42.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff78155f640 1 --2- 172.21.15.47:0/1422476782 >> v2:172.21.15.175:3300/0 conn(0x7ff77c15c990 0x7ff77c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff781d60640 1 -- 172.21.15.47:0/1422476782 >> v2:172.21.15.175:3300/0 conn(0x7ff77c15c990 msgr2=0x7ff77c160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:42.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff781d60640 1 --2- 172.21.15.47:0/1422476782 >> v2:172.21.15.175:3300/0 conn(0x7ff77c15c990 0x7ff77c160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff781d60640 1 -- 172.21.15.47:0/1422476782 >> v1:172.21.15.47:6789/0 conn(0x7ff77c0b8350 legacy=0x7ff77c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff781d60640 1 -- 172.21.15.47:0/1422476782 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c0b5b60 con 0x7ff77c0cde10 2024-03-23T09:41:42.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.728+0000 7ff78155f640 1 --2- 172.21.15.47:0/1422476782 >> v2:172.21.15.175:3300/0 conn(0x7ff77c15c990 0x7ff77c160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.729+0000 7ff781d60640 1 --2- 172.21.15.47:0/1422476782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff77c0cde10 0x7ff77c0d14c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0b8b30 tx=0x7ff76c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adb427f36693707 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.729+0000 7ff77bfff640 1 -- 172.21.15.47:0/1422476782 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c02e070 con 0x7ff77c0cde10 2024-03-23T09:41:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.729+0000 7ff77bfff640 1 -- 172.21.15.47:0/1422476782 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c0315a0 con 0x7ff77c0cde10 2024-03-23T09:41:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.729+0000 7ff77bfff640 1 -- 172.21.15.47:0/1422476782 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c031930 con 0x7ff77c0cde10 2024-03-23T09:41:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.730+0000 7ff7837c3640 1 -- 172.21.15.47:0/1422476782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff77c0cde10 msgr2=0x7ff77c0d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.730+0000 7ff7837c3640 1 --2- 172.21.15.47:0/1422476782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff77c0cde10 0x7ff77c0d14c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0b8b30 tx=0x7ff76c02b840 comp rx=0 tx=0).stop 2024-03-23T09:41:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.730+0000 7ff7837c3640 1 -- 172.21.15.47:0/1422476782 shutdown_connections 2024-03-23T09:41:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.730+0000 7ff7837c3640 1 --2- 172.21.15.47:0/1422476782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff77c0cde10 0x7ff77c0d14c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.730+0000 7ff7837c3640 1 --2- 172.21.15.47:0/1422476782 >> v2:172.21.15.175:3300/0 conn(0x7ff77c15c990 0x7ff77c160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.730+0000 7ff7837c3640 1 -- 172.21.15.47:0/1422476782 >> 172.21.15.47:0/1422476782 conn(0x7ff77c0659d0 msgr2=0x7ff77c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.730+0000 7ff7837c3640 1 -- 172.21.15.47:0/1422476782 shutdown_connections 2024-03-23T09:41:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.730+0000 7ff7837c3640 1 -- 172.21.15.47:0/1422476782 wait complete. 2024-03-23T09:41:42.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.731+0000 7ff7837c3640 1 Processor -- start 2024-03-23T09:41:42.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.731+0000 7ff7837c3640 1 -- start start 2024-03-23T09:41:42.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff7837c3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff77c0b8350 0x7ff77c155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff7837c3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77c0cde10 0x7ff77c0c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff7837c3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff77c15c990 0x7ff77c157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff7837c3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c156440 con 0x7ff77c15c990 2024-03-23T09:41:42.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff7837c3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c1f5eb0 con 0x7ff77c0b8350 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff7837c3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c1f31a0 con 0x7ff77c0cde10 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff781d60640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff77c15c990 0x7ff77c157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff781d60640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff77c15c990 0x7ff77c157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50400/0 (socket says 172.21.15.47:50400) 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff781d60640 1 -- 172.21.15.47:0/1221172748 learned_addr learned my addr 172.21.15.47:0/1221172748 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff78155f640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff77c0b8350 0x7ff77c155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff780d5e640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77c0cde10 0x7ff77c0c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff781d60640 1 -- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77c0cde10 msgr2=0x7ff77c0c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff781d60640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77c0cde10 0x7ff77c0c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff781d60640 1 -- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff77c0b8350 msgr2=0x7ff77c155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff781d60640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff77c0b8350 0x7ff77c155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff781d60640 1 -- 172.21.15.47:0/1221172748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c009400 con 0x7ff77c15c990 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff78155f640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff77c0b8350 0x7ff77c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.732+0000 7ff781d60640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff77c15c990 0x7ff77c157050 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff76c034040 tx=0x7ff76c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.733+0000 7ff779ffb640 1 -- 172.21.15.47:0/1221172748 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c032840 con 0x7ff77c15c990 2024-03-23T09:41:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.733+0000 7ff779ffb640 1 -- 172.21.15.47:0/1221172748 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c0329a0 con 0x7ff77c15c990 2024-03-23T09:41:42.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.733+0000 7ff7837c3640 1 -- 172.21.15.47:0/1221172748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c1f3480 con 0x7ff77c15c990 2024-03-23T09:41:42.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.733+0000 7ff779ffb640 1 -- 172.21.15.47:0/1221172748 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c032d50 con 0x7ff77c15c990 2024-03-23T09:41:42.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.733+0000 7ff7837c3640 1 -- 172.21.15.47:0/1221172748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c0c6710 con 0x7ff77c15c990 2024-03-23T09:41:42.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.734+0000 7ff7837c3640 1 -- 172.21.15.47:0/1221172748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77c072360 con 0x7ff77c15c990 2024-03-23T09:41:42.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.734+0000 7ff779ffb640 1 -- 172.21.15.47:0/1221172748 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff76c0453f0 con 0x7ff77c15c990 2024-03-23T09:41:42.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.735+0000 7ff779ffb640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff76806ac00 0x7ff76806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:42.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.735+0000 7ff779ffb640 1 -- 172.21.15.47:0/1221172748 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff76c0ab680 con 0x7ff77c15c990 2024-03-23T09:41:42.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.735+0000 7ff78155f640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff76806ac00 0x7ff76806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:42.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.735+0000 7ff78155f640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff76806ac00 0x7ff76806d0c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff77400e3d0 tx=0x7ff774007a10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:42.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.739+0000 7ff779ffb640 1 -- 172.21.15.47:0/1221172748 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff76c0767c0 con 0x7ff77c15c990 2024-03-23T09:41:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.918+0000 7ff7837c3640 1 -- 172.21.15.47:0/1221172748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff77c0c6260 con 0x7ff77c15c990 2024-03-23T09:41:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.918+0000 7ff779ffb640 1 -- 172.21.15.47:0/1221172748 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff76c079de0 con 0x7ff77c15c990 2024-03-23T09:41:42.918 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:42.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.920+0000 7ff7837c3640 1 -- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff76806ac00 msgr2=0x7ff76806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.920+0000 7ff7837c3640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff76806ac00 0x7ff76806d0c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff77400e3d0 tx=0x7ff774007a10 comp rx=0 tx=0).stop 2024-03-23T09:41:42.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.920+0000 7ff7837c3640 1 -- 172.21.15.47:0/1221172748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff77c15c990 msgr2=0x7ff77c157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:42.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.920+0000 7ff7837c3640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff77c15c990 0x7ff77c157050 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff76c034040 tx=0x7ff76c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.920+0000 7ff7837c3640 1 -- 172.21.15.47:0/1221172748 shutdown_connections 2024-03-23T09:41:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.920+0000 7ff7837c3640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff76806ac00 0x7ff76806d0c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.920+0000 7ff7837c3640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff77c15c990 0x7ff77c157050 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.920+0000 7ff7837c3640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77c0cde10 0x7ff77c0c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.920+0000 7ff7837c3640 1 --2- 172.21.15.47:0/1221172748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff77c0b8350 0x7ff77c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.920+0000 7ff7837c3640 1 -- 172.21.15.47:0/1221172748 >> 172.21.15.47:0/1221172748 conn(0x7ff77c0659d0 msgr2=0x7ff77c0d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.920+0000 7ff7837c3640 1 -- 172.21.15.47:0/1221172748 shutdown_connections 2024-03-23T09:41:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:42.920+0000 7ff7837c3640 1 -- 172.21.15.47:0/1221172748 wait complete. 2024-03-23T09:41:42.931 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:41:43.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.033+0000 7f9bb2fd2640 1 Processor -- start 2024-03-23T09:41:43.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.033+0000 7f9bb2fd2640 1 -- start start 2024-03-23T09:41:43.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.034+0000 7f9bb2fd2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bac0cf690 0x7f9bac0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.034+0000 7f9bb2fd2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac15d0e0 con 0x7f9bac15cd00 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.034+0000 7f9bb2fd2640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac0d1540 con 0x7f9bac0cf210 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.034+0000 7f9bb2fd2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0d1820 con 0x7f9bac0cf690 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.034+0000 7f9bb0d6e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9bac15cd00 0x7f9bac0d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51224/0 (socket says 172.21.15.47:51224) 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.034+0000 7f9bb0d6e640 1 -- 172.21.15.47:0/1679349085 learned_addr learned my addr 172.21.15.47:0/1679349085 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.034+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119427485 0 0) 0x7f9bac15d0e0 con 0x7f9bac15cd00 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.034+0000 7f9bb156f640 1 --2- 172.21.15.47:0/1679349085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bac0cf690 0x7f9bac0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.034+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b94003520 con 0x7f9bac15cd00 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3121937465 0 0) 0x7f9b94003520 con 0x7f9bac15cd00 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bac0cf690 msgr2=0x7f9bac0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bab7fe640 1 --2- 172.21.15.47:0/1679349085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bac0cf690 0x7f9bac0b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 >> v1:172.21.15.175:6789/0 conn(0x7f9bac0cf210 legacy=0x7f9bac0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac0d1b00 con 0x7f9bac15cd00 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f9b9c001670 con 0x7f9bac15cd00 2024-03-23T09:41:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 >> v1:172.21.15.47:6789/0 conn(0x7f9bac15cd00 legacy=0x7f9bac0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bab7fe640 1 --2- 172.21.15.47:0/1679349085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b940039a0 0x7f9b94003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bab7fe640 1 --2- 172.21.15.47:0/1679349085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b94004350 0x7f9b94004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bab7fe640 1 --2- 172.21.15.47:0/1679349085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bac0cf690 0x7f9b94005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0d1820 con 0x7f9b940039a0 2024-03-23T09:41:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9400ba30 con 0x7f9b94004350 2024-03-23T09:41:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9400bbf0 con 0x7f9bac0cf690 2024-03-23T09:41:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bb0d6e640 1 --2- 172.21.15.47:0/1679349085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b940039a0 0x7f9b94003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9babfff640 1 --2- 172.21.15.47:0/1679349085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b94004350 0x7f9b94004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bb156f640 1 --2- 172.21.15.47:0/1679349085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bac0cf690 0x7f9b94005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bb0d6e640 1 -- 172.21.15.47:0/1679349085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bac0cf690 msgr2=0x7f9b94005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bb0d6e640 1 --2- 172.21.15.47:0/1679349085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bac0cf690 0x7f9b94005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bb0d6e640 1 -- 172.21.15.47:0/1679349085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b94004350 msgr2=0x7f9b94004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bb0d6e640 1 --2- 172.21.15.47:0/1679349085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b94004350 0x7f9b94004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9bb0d6e640 1 -- 172.21.15.47:0/1679349085 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b94003520 con 0x7f9b940039a0 2024-03-23T09:41:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.035+0000 7f9babfff640 1 --2- 172.21.15.47:0/1679349085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b94004350 0x7f9b94004820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.036+0000 7f9bb0d6e640 1 --2- 172.21.15.47:0/1679349085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b940039a0 0x7f9b94003d80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00c210 tx=0x7f9b940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e40c54903e9d4f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.036+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c02cae0 con 0x7f9b940039a0 2024-03-23T09:41:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.036+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c02cc60 con 0x7f9b940039a0 2024-03-23T09:41:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.036+0000 7f9bab7fe640 1 -- 172.21.15.47:0/1679349085 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c0058d0 con 0x7f9b940039a0 2024-03-23T09:41:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.036+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/1679349085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b940039a0 msgr2=0x7f9b94003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.036+0000 7f9bb2fd2640 1 --2- 172.21.15.47:0/1679349085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b940039a0 0x7f9b94003d80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00c210 tx=0x7f9b940057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.037+0000 7f9bb156f640 1 -- 172.21.15.47:0/1679349085 reap_dead start 2024-03-23T09:41:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.037+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/1679349085 shutdown_connections 2024-03-23T09:41:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.037+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/1679349085 >> 172.21.15.47:0/1679349085 conn(0x7f9bac06b160 msgr2=0x7f9bac06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.037+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/1679349085 shutdown_connections 2024-03-23T09:41:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.037+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/1679349085 wait complete. 2024-03-23T09:41:43.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.038+0000 7f9bb2fd2640 1 Processor -- start 2024-03-23T09:41:43.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.038+0000 7f9bb2fd2640 1 -- start start 2024-03-23T09:41:43.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb2fd2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b940039a0 0x7f9bac155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb2fd2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bac0cc800 0x7f9bac158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb2fd2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bac1f5910 0x7f9bac1f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb2fd2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac155fd0 con 0x7f9b940039a0 2024-03-23T09:41:43.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb2fd2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac1f62d0 con 0x7f9bac0cc800 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb2fd2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0ca050 con 0x7f9bac1f5910 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb0d6e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b940039a0 0x7f9bac155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb0d6e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b940039a0 0x7f9bac155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50418/0 (socket says 172.21.15.47:50418) 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb0d6e640 1 -- 172.21.15.47:0/2443193654 learned_addr learned my addr 172.21.15.47:0/2443193654 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9babfff640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bac0cc800 0x7f9bac158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb156f640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bac1f5910 0x7f9bac1f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb0d6e640 1 -- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bac1f5910 msgr2=0x7f9bac1f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb0d6e640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bac1f5910 0x7f9bac1f5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb0d6e640 1 -- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bac0cc800 msgr2=0x7f9bac158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb0d6e640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bac0cc800 0x7f9bac158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb0d6e640 1 -- 172.21.15.47:0/2443193654 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac158090 con 0x7f9b940039a0 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9babfff640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bac0cc800 0x7f9bac158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb0d6e640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b940039a0 0x7f9bac155a90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c02c480 tx=0x7f9b9c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9ba97fa640 1 -- 172.21.15.47:0/2443193654 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c039070 con 0x7f9b940039a0 2024-03-23T09:41:43.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9ba97fa640 1 -- 172.21.15.47:0/2443193654 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c02caf0 con 0x7f9b940039a0 2024-03-23T09:41:43.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/2443193654 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bac0ca2a0 con 0x7f9b940039a0 2024-03-23T09:41:43.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9ba97fa640 1 -- 172.21.15.47:0/2443193654 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c03c040 con 0x7f9b940039a0 2024-03-23T09:41:43.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.039+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/2443193654 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bac1582e0 con 0x7f9b940039a0 2024-03-23T09:41:43.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.041+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/2443193654 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bac072360 con 0x7f9b940039a0 2024-03-23T09:41:43.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.041+0000 7f9ba97fa640 1 -- 172.21.15.47:0/2443193654 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9b9c032070 con 0x7f9b940039a0 2024-03-23T09:41:43.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.041+0000 7f9ba97fa640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b9806ac00 0x7f9b9806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.041+0000 7f9ba97fa640 1 -- 172.21.15.47:0/2443193654 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b9c0aa300 con 0x7f9b940039a0 2024-03-23T09:41:43.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.042+0000 7f9babfff640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b9806ac00 0x7f9b9806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.042+0000 7f9babfff640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b9806ac00 0x7f9b9806d0c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9bac15ff70 tx=0x7f9ba0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:43.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.045+0000 7f9ba97fa640 1 -- 172.21.15.47:0/2443193654 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9b9c0756b0 con 0x7f9b940039a0 2024-03-23T09:41:43.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.239+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/2443193654 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9bac0b8150 con 0x7f9b940039a0 2024-03-23T09:41:43.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.240+0000 7f9ba97fa640 1 -- 172.21.15.47:0/2443193654 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9b9c001630 con 0x7f9b940039a0 2024-03-23T09:41:43.239 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:41:43.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.242+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b9806ac00 msgr2=0x7f9b9806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.242+0000 7f9bb2fd2640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b9806ac00 0x7f9b9806d0c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9bac15ff70 tx=0x7f9ba0009040 comp rx=0 tx=0).stop 2024-03-23T09:41:43.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.242+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/2443193654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b940039a0 msgr2=0x7f9bac155a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.242+0000 7f9bb2fd2640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b940039a0 0x7f9bac155a90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c02c480 tx=0x7f9b9c001a00 comp rx=0 tx=0).stop 2024-03-23T09:41:43.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.242+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/2443193654 shutdown_connections 2024-03-23T09:41:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.242+0000 7f9bb2fd2640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b9806ac00 0x7f9b9806d0c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.242+0000 7f9bb2fd2640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bac1f5910 0x7f9bac1f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.242+0000 7f9bb2fd2640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bac0cc800 0x7f9bac158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.242+0000 7f9bb2fd2640 1 --2- 172.21.15.47:0/2443193654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b940039a0 0x7f9bac155a90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.242+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/2443193654 >> 172.21.15.47:0/2443193654 conn(0x7f9bac06b160 msgr2=0x7f9bac15d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.242+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/2443193654 shutdown_connections 2024-03-23T09:41:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.242+0000 7f9bb2fd2640 1 -- 172.21.15.47:0/2443193654 wait complete. 2024-03-23T09:41:43.252 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:41:43.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:43.252 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-03-23T09:41:43.278 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:41:43.279 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:41:43.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:43.279 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-23T09:41:43.334 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:41:43.334 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:41:43.353 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:41:43.353 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:41:43.412 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:43.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.513+0000 7f8a8b4ad640 1 Processor -- start 2024-03-23T09:41:43.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.513+0000 7f8a8b4ad640 1 -- start start 2024-03-23T09:41:43.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.514+0000 7f8a8b4ad640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a8406c350 0x7f8a8406d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.514+0000 7f8a8b4ad640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8a84142aa0 0x7f8a840cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.514+0000 7f8a8b4ad640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8406c8c0 con 0x7f8a840b3c30 2024-03-23T09:41:43.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.514+0000 7f8a8b4ad640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8406cba0 con 0x7f8a84142aa0 2024-03-23T09:41:43.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.514+0000 7f8a8b4ad640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8406ce80 con 0x7f8a8406c350 2024-03-23T09:41:43.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.514+0000 7f8a88a48640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a8406c350 0x7f8a8406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.514+0000 7f8a89a4a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8a84142aa0 0x7f8a840cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.514+0000 7f8a89249640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f8a840b3c30 0x7f8a840b4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:32850/0 (socket says 172.21.15.47:32850) 2024-03-23T09:41:43.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.514+0000 7f8a89249640 1 -- 172.21.15.47:0/3775324809 learned_addr learned my addr 172.21.15.47:0/3775324809 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:43.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.515+0000 7f8a83fff640 1 -- 172.21.15.47:0/3775324809 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182911683 0 0) 0x7f8a8406c8c0 con 0x7f8a840b3c30 2024-03-23T09:41:43.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.515+0000 7f8a83fff640 1 -- 172.21.15.47:0/3775324809 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a6c003520 con 0x7f8a840b3c30 2024-03-23T09:41:43.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.515+0000 7f8a89a4a640 1 -- 172.21.15.47:0/3775324809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a8406c350 msgr2=0x7f8a8406d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:43.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.515+0000 7f8a89a4a640 1 --2- 172.21.15.47:0/3775324809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a8406c350 0x7f8a8406d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.515+0000 7f8a89a4a640 1 -- 172.21.15.47:0/3775324809 >> v1:172.21.15.175:6789/0 conn(0x7f8a840b3c30 legacy=0x7f8a840b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.515+0000 7f8a89a4a640 1 -- 172.21.15.47:0/3775324809 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8406d160 con 0x7f8a84142aa0 2024-03-23T09:41:43.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.515+0000 7f8a88a48640 1 --2- 172.21.15.47:0/3775324809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a8406c350 0x7f8a8406d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:43.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.516+0000 7f8a89a4a640 1 --2- 172.21.15.47:0/3775324809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a84142aa0 0x7f8a840cc4b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0097a0 tx=0x7f8a7c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c81149d4addf383 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:43.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.516+0000 7f8a83fff640 1 -- 172.21.15.47:0/3775324809 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c031830 con 0x7f8a84142aa0 2024-03-23T09:41:43.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.516+0000 7f8a83fff640 1 -- 172.21.15.47:0/3775324809 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c031990 con 0x7f8a84142aa0 2024-03-23T09:41:43.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.516+0000 7f8a83fff640 1 -- 172.21.15.47:0/3775324809 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c031d00 con 0x7f8a84142aa0 2024-03-23T09:41:43.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.516+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/3775324809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a84142aa0 msgr2=0x7f8a840cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.516+0000 7f8a8b4ad640 1 --2- 172.21.15.47:0/3775324809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a84142aa0 0x7f8a840cc4b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0097a0 tx=0x7f8a7c02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.516+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/3775324809 shutdown_connections 2024-03-23T09:41:43.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.516+0000 7f8a8b4ad640 1 --2- 172.21.15.47:0/3775324809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a84142aa0 0x7f8a840cc4b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.516+0000 7f8a8b4ad640 1 --2- 172.21.15.47:0/3775324809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a8406c350 0x7f8a8406d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.516+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/3775324809 >> 172.21.15.47:0/3775324809 conn(0x7f8a8406a310 msgr2=0x7f8a840b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:43.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.517+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/3775324809 shutdown_connections 2024-03-23T09:41:43.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.517+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/3775324809 wait complete. 2024-03-23T09:41:43.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.518+0000 7f8a8b4ad640 1 Processor -- start 2024-03-23T09:41:43.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.518+0000 7f8a8b4ad640 1 -- start start 2024-03-23T09:41:43.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.518+0000 7f8a8b4ad640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8406c350 0x7f8a8415be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.518+0000 7f8a8b4ad640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a840b3c30 0x7f8a84158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.518+0000 7f8a8b4ad640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a84142aa0 0x7f8a8413e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.518+0000 7f8a8b4ad640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8415c340 con 0x7f8a840b3c30 2024-03-23T09:41:43.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.518+0000 7f8a8b4ad640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8413f100 con 0x7f8a8406c350 2024-03-23T09:41:43.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.518+0000 7f8a8b4ad640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8415b040 con 0x7f8a84142aa0 2024-03-23T09:41:43.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a88a48640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a840b3c30 0x7f8a84158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a88a48640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a840b3c30 0x7f8a84158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50420/0 (socket says 172.21.15.47:50420) 2024-03-23T09:41:43.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a88a48640 1 -- 172.21.15.47:0/2923299590 learned_addr learned my addr 172.21.15.47:0/2923299590 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:43.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a89249640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8406c350 0x7f8a8415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a89a4a640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a84142aa0 0x7f8a8413e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a88a48640 1 -- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a84142aa0 msgr2=0x7f8a8413e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a88a48640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a84142aa0 0x7f8a8413e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a88a48640 1 -- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8406c350 msgr2=0x7f8a8415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a88a48640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8406c350 0x7f8a8415be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a88a48640 1 -- 172.21.15.47:0/2923299590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a78009400 con 0x7f8a840b3c30 2024-03-23T09:41:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a88a48640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a840b3c30 0x7f8a84158620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800ebe0 tx=0x7f8a7800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a81ffb640 1 -- 172.21.15.47:0/2923299590 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78009760 con 0x7f8a840b3c30 2024-03-23T09:41:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a81ffb640 1 -- 172.21.15.47:0/2923299590 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7800cc30 con 0x7f8a840b3c30 2024-03-23T09:41:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a81ffb640 1 -- 172.21.15.47:0/2923299590 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800eee0 con 0x7f8a840b3c30 2024-03-23T09:41:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.519+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/2923299590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7c009400 con 0x7f8a840b3c30 2024-03-23T09:41:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.520+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/2923299590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8415b600 con 0x7f8a840b3c30 2024-03-23T09:41:43.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.520+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/2923299590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a84066be0 con 0x7f8a840b3c30 2024-03-23T09:41:43.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.521+0000 7f8a81ffb640 1 -- 172.21.15.47:0/2923299590 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8a78013470 con 0x7f8a840b3c30 2024-03-23T09:41:43.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.521+0000 7f8a81ffb640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8a7006ac00 0x7f8a7006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.521+0000 7f8a81ffb640 1 -- 172.21.15.47:0/2923299590 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a78089610 con 0x7f8a840b3c30 2024-03-23T09:41:43.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.521+0000 7f8a89249640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8a7006ac00 0x7f8a7006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.522+0000 7f8a89249640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8a7006ac00 0x7f8a7006d0c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8a74005c00 tx=0x7f8a74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.525+0000 7f8a81ffb640 1 -- 172.21.15.47:0/2923299590 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8a78054800 con 0x7f8a840b3c30 2024-03-23T09:41:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.705+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/2923299590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a8415c740 con 0x7f8a840b3c30 2024-03-23T09:41:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.706+0000 7f8a81ffb640 1 -- 172.21.15.47:0/2923299590 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f8a78057e20 con 0x7f8a840b3c30 2024-03-23T09:41:43.706 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:43.706 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.708+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8a7006ac00 msgr2=0x7f8a7006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.708+0000 7f8a8b4ad640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8a7006ac00 0x7f8a7006d0c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8a74005c00 tx=0x7f8a74009040 comp rx=0 tx=0).stop 2024-03-23T09:41:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.709+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/2923299590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a840b3c30 msgr2=0x7f8a84158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.709+0000 7f8a8b4ad640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a840b3c30 0x7f8a84158620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800ebe0 tx=0x7f8a7800c520 comp rx=0 tx=0).stop 2024-03-23T09:41:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.709+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/2923299590 shutdown_connections 2024-03-23T09:41:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.709+0000 7f8a8b4ad640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8a7006ac00 0x7f8a7006d0c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.709+0000 7f8a8b4ad640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a84142aa0 0x7f8a8413e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.709+0000 7f8a8b4ad640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a840b3c30 0x7f8a84158620 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.709+0000 7f8a8b4ad640 1 --2- 172.21.15.47:0/2923299590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8406c350 0x7f8a8415be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.709+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/2923299590 >> 172.21.15.47:0/2923299590 conn(0x7f8a8406a310 msgr2=0x7f8a840ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:43.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.709+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/2923299590 shutdown_connections 2024-03-23T09:41:43.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.709+0000 7f8a8b4ad640 1 -- 172.21.15.47:0/2923299590 wait complete. 2024-03-23T09:41:43.719 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:43.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.824+0000 7f43b0abd640 1 Processor -- start 2024-03-23T09:41:43.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.824+0000 7f43b0abd640 1 -- start start 2024-03-23T09:41:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.825+0000 7f43b0abd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f43ac0b8350 0x7f43ac160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.825+0000 7f43b0abd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ac161630 con 0x7f43ac15c990 2024-03-23T09:41:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.825+0000 7f43b0abd640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ac0d1030 con 0x7f43ac0cde10 2024-03-23T09:41:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.825+0000 7f43b0abd640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ac0cfe10 con 0x7f43ac0b8350 2024-03-23T09:41:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.825+0000 7f43aad9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f43ac0b8350 0x7f43ac160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.825+0000 7f43aa59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f43ac15c990 0x7f43ac0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51240/0 (socket says 172.21.15.47:51240) 2024-03-23T09:41:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.825+0000 7f43aa59c640 1 -- 172.21.15.47:0/342172882 learned_addr learned my addr 172.21.15.47:0/342172882 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.825+0000 7f43aad9d640 1 -- 172.21.15.47:0/342172882 >> v1:172.21.15.175:6789/0 conn(0x7f43ac0cde10 legacy=0x7f43ac0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.825+0000 7f43aad9d640 1 -- 172.21.15.47:0/342172882 >> v1:172.21.15.47:6789/0 conn(0x7f43ac15c990 legacy=0x7f43ac0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.825+0000 7f43aad9d640 1 -- 172.21.15.47:0/342172882 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac0b5b60 con 0x7f43ac0b8350 2024-03-23T09:41:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.825+0000 7f43a9d9b640 1 -- 172.21.15.47:0/342172882 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343239534 0 0) 0x7f43ac161630 con 0x7f43ac15c990 2024-03-23T09:41:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.826+0000 7f43aad9d640 1 --2- 172.21.15.47:0/342172882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ac0b8350 0x7f43ac160690 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009d90 tx=0x7f43a402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616533ee70d578f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.826+0000 7f43a9d9b640 1 -- 172.21.15.47:0/342172882 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a402cc60 con 0x7f43ac0b8350 2024-03-23T09:41:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.826+0000 7f43a9d9b640 1 -- 172.21.15.47:0/342172882 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a402cdc0 con 0x7f43ac0b8350 2024-03-23T09:41:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.826+0000 7f43a9d9b640 1 -- 172.21.15.47:0/342172882 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4034660 con 0x7f43ac0b8350 2024-03-23T09:41:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.827+0000 7f43b0abd640 1 -- 172.21.15.47:0/342172882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ac0b8350 msgr2=0x7f43ac160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.827+0000 7f43b0abd640 1 --2- 172.21.15.47:0/342172882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ac0b8350 0x7f43ac160690 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009d90 tx=0x7f43a402beb0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.827+0000 7f43b0abd640 1 -- 172.21.15.47:0/342172882 shutdown_connections 2024-03-23T09:41:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.827+0000 7f43b0abd640 1 --2- 172.21.15.47:0/342172882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ac0b8350 0x7f43ac160690 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.827+0000 7f43b0abd640 1 -- 172.21.15.47:0/342172882 >> 172.21.15.47:0/342172882 conn(0x7f43ac0659d0 msgr2=0x7f43ac065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.827+0000 7f43b0abd640 1 -- 172.21.15.47:0/342172882 shutdown_connections 2024-03-23T09:41:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.827+0000 7f43b0abd640 1 -- 172.21.15.47:0/342172882 wait complete. 2024-03-23T09:41:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.828+0000 7f43b0abd640 1 Processor -- start 2024-03-23T09:41:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.828+0000 7f43b0abd640 1 -- start start 2024-03-23T09:41:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43b0abd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43ac0b8350 0x7f43ac15a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43b0abd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43ac0cde10 0x7f43ac0c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43b0abd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ac15c990 0x7f43ac158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43b0abd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac1f30c0 con 0x7f43ac15c990 2024-03-23T09:41:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43b0abd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac0c8da0 con 0x7f43ac0b8350 2024-03-23T09:41:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43b0abd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac159320 con 0x7f43ac0cde10 2024-03-23T09:41:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43aad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ac15c990 0x7f43ac158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43aad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ac15c990 0x7f43ac158de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50430/0 (socket says 172.21.15.47:50430) 2024-03-23T09:41:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43aad9d640 1 -- 172.21.15.47:0/2612158643 learned_addr learned my addr 172.21.15.47:0/2612158643 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43aa59c640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43ac0b8350 0x7f43ac15a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43a3fff640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43ac0cde10 0x7f43ac0c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43aad9d640 1 -- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43ac0cde10 msgr2=0x7f43ac0c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43aad9d640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43ac0cde10 0x7f43ac0c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43aad9d640 1 -- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43ac0b8350 msgr2=0x7f43ac15a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43aad9d640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43ac0b8350 0x7f43ac15a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43aad9d640 1 -- 172.21.15.47:0/2612158643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a4009a70 con 0x7f43ac15c990 2024-03-23T09:41:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43aad9d640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ac15c990 0x7f43ac158de0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009d90 tx=0x7f43a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43a2ffd640 1 -- 172.21.15.47:0/2612158643 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4040450 con 0x7f43ac15c990 2024-03-23T09:41:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43a2ffd640 1 -- 172.21.15.47:0/2612158643 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a40405d0 con 0x7f43ac15c990 2024-03-23T09:41:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43a2ffd640 1 -- 172.21.15.47:0/2612158643 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4040980 con 0x7f43ac15c990 2024-03-23T09:41:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43b0abd640 1 -- 172.21.15.47:0/2612158643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ac1f2440 con 0x7f43ac15c990 2024-03-23T09:41:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.829+0000 7f43b0abd640 1 -- 172.21.15.47:0/2612158643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ac1f2a00 con 0x7f43ac15c990 2024-03-23T09:41:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.831+0000 7f43b0abd640 1 -- 172.21.15.47:0/2612158643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ac072360 con 0x7f43ac15c990 2024-03-23T09:41:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.831+0000 7f43a2ffd640 1 -- 172.21.15.47:0/2612158643 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f43a4040ae0 con 0x7f43ac15c990 2024-03-23T09:41:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.831+0000 7f43a2ffd640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f438c06ac00 0x7f438c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.832+0000 7f43a2ffd640 1 -- 172.21.15.47:0/2612158643 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43a40abe60 con 0x7f43ac15c990 2024-03-23T09:41:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.832+0000 7f43aa59c640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f438c06ac00 0x7f438c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.832+0000 7f43aa59c640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f438c06ac00 0x7f438c06d0c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f439c007f70 tx=0x7f439c007f00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:43.836+0000 7f43a2ffd640 1 -- 172.21.15.47:0/2612158643 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f43a4077100 con 0x7f43ac15c990 2024-03-23T09:41:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.018+0000 7f43b0abd640 1 -- 172.21.15.47:0/2612158643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43ac066be0 con 0x7f43ac15c990 2024-03-23T09:41:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.018+0000 7f43a2ffd640 1 -- 172.21.15.47:0/2612158643 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43a407a720 con 0x7f43ac15c990 2024-03-23T09:41:44.018 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.020+0000 7f43b0abd640 1 -- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f438c06ac00 msgr2=0x7f438c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.020+0000 7f43b0abd640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f438c06ac00 0x7f438c06d0c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f439c007f70 tx=0x7f439c007f00 comp rx=0 tx=0).stop 2024-03-23T09:41:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.020+0000 7f43b0abd640 1 -- 172.21.15.47:0/2612158643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ac15c990 msgr2=0x7f43ac158de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.020+0000 7f43b0abd640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ac15c990 0x7f43ac158de0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009d90 tx=0x7f43a4004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.021+0000 7f43b0abd640 1 -- 172.21.15.47:0/2612158643 shutdown_connections 2024-03-23T09:41:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.021+0000 7f43b0abd640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f438c06ac00 0x7f438c06d0c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.021+0000 7f43b0abd640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ac15c990 0x7f43ac158de0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.021+0000 7f43b0abd640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43ac0cde10 0x7f43ac0c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.021+0000 7f43b0abd640 1 --2- 172.21.15.47:0/2612158643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43ac0b8350 0x7f43ac15a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.021+0000 7f43b0abd640 1 -- 172.21.15.47:0/2612158643 >> 172.21.15.47:0/2612158643 conn(0x7f43ac0659d0 msgr2=0x7f43ac0d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.021+0000 7f43b0abd640 1 -- 172.21.15.47:0/2612158643 shutdown_connections 2024-03-23T09:41:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.021+0000 7f43b0abd640 1 -- 172.21.15.47:0/2612158643 wait complete. 2024-03-23T09:41:44.031 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:44.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.134+0000 7f6166aab640 1 Processor -- start 2024-03-23T09:41:44.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.134+0000 7f6166aab640 1 -- start start 2024-03-23T09:41:44.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.135+0000 7f6166aab640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f616015c990 0x7f6160160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.135+0000 7f6166aab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61600b8350 0x7f61600cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.135+0000 7f6166aab640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6160161630 con 0x7f61600cdc40 2024-03-23T09:41:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.135+0000 7f6166aab640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61600ce0b0 con 0x7f616015c990 2024-03-23T09:41:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.135+0000 7f6166aab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61600d1030 con 0x7f61600b8350 2024-03-23T09:41:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.135+0000 7f6164847640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f616015c990 0x7f6160160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.135+0000 7f6164847640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f616015c990 0x7f6160160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50442/0 (socket says 172.21.15.47:50442) 2024-03-23T09:41:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.135+0000 7f6164847640 1 -- 172.21.15.47:0/1789298251 learned_addr learned my addr 172.21.15.47:0/1789298251 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.136+0000 7f615ffff640 1 --2- 172.21.15.47:0/1789298251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61600b8350 0x7f61600cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.136+0000 7f6164847640 1 -- 172.21.15.47:0/1789298251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61600b8350 msgr2=0x7f61600cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.136+0000 7f6164847640 1 --2- 172.21.15.47:0/1789298251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61600b8350 0x7f61600cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.136+0000 7f6164847640 1 -- 172.21.15.47:0/1789298251 >> v1:172.21.15.175:6789/0 conn(0x7f61600cdc40 legacy=0x7f61600b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.136+0000 7f6164847640 1 -- 172.21.15.47:0/1789298251 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61600cfe10 con 0x7f616015c990 2024-03-23T09:41:44.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.136+0000 7f6164847640 1 --2- 172.21.15.47:0/1789298251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f616015c990 0x7f6160160690 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6158009e50 tx=0x7f615802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf64c7c96560ae42 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:44.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.137+0000 7f615f7fe640 1 -- 172.21.15.47:0/1789298251 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615802ca70 con 0x7f616015c990 2024-03-23T09:41:44.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.137+0000 7f615f7fe640 1 -- 172.21.15.47:0/1789298251 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615802cbd0 con 0x7f616015c990 2024-03-23T09:41:44.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.137+0000 7f615f7fe640 1 -- 172.21.15.47:0/1789298251 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61580343f0 con 0x7f616015c990 2024-03-23T09:41:44.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.137+0000 7f6166aab640 1 -- 172.21.15.47:0/1789298251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f616015c990 msgr2=0x7f6160160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.137+0000 7f6166aab640 1 --2- 172.21.15.47:0/1789298251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f616015c990 0x7f6160160690 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6158009e50 tx=0x7f615802bed0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.137+0000 7f6166aab640 1 -- 172.21.15.47:0/1789298251 shutdown_connections 2024-03-23T09:41:44.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.137+0000 7f6166aab640 1 --2- 172.21.15.47:0/1789298251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61600b8350 0x7f61600cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.137+0000 7f6166aab640 1 --2- 172.21.15.47:0/1789298251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f616015c990 0x7f6160160690 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.137+0000 7f6166aab640 1 -- 172.21.15.47:0/1789298251 >> 172.21.15.47:0/1789298251 conn(0x7f61600659d0 msgr2=0x7f6160065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:44.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.137+0000 7f6166aab640 1 -- 172.21.15.47:0/1789298251 shutdown_connections 2024-03-23T09:41:44.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.137+0000 7f6166aab640 1 -- 172.21.15.47:0/1789298251 wait complete. 2024-03-23T09:41:44.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.138+0000 7f6166aab640 1 Processor -- start 2024-03-23T09:41:44.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.138+0000 7f6166aab640 1 -- start start 2024-03-23T09:41:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.139+0000 7f6166aab640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61600b8350 0x7f61601585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.139+0000 7f6166aab640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61600cdc40 0x7f6160155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.139+0000 7f6166aab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f616015c990 0x7f61600c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.139+0000 7f6166aab640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601561a0 con 0x7f61600b8350 2024-03-23T09:41:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.139+0000 7f6166aab640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61600c7570 con 0x7f61600cdc40 2024-03-23T09:41:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.139+0000 7f6166aab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61600c7830 con 0x7f616015c990 2024-03-23T09:41:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.139+0000 7f6164847640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61600b8350 0x7f61601585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.139+0000 7f6164847640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61600b8350 0x7f61601585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50452/0 (socket says 172.21.15.47:50452) 2024-03-23T09:41:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.139+0000 7f6164847640 1 -- 172.21.15.47:0/2444278547 learned_addr learned my addr 172.21.15.47:0/2444278547 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f615ffff640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61600cdc40 0x7f6160155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f6165048640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f616015c990 0x7f61600c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f6164847640 1 -- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f616015c990 msgr2=0x7f61600c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f6164847640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f616015c990 0x7f61600c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f6164847640 1 -- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61600cdc40 msgr2=0x7f6160155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f6164847640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61600cdc40 0x7f6160155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f6164847640 1 -- 172.21.15.47:0/2444278547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6158009b30 con 0x7f61600b8350 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f615ffff640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61600cdc40 0x7f6160155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f6164847640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61600b8350 0x7f61601585b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6158000f80 tx=0x7f6158004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f615d7fa640 1 -- 172.21.15.47:0/2444278547 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158009510 con 0x7f61600b8350 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f615d7fa640 1 -- 172.21.15.47:0/2444278547 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615803e450 con 0x7f61600b8350 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f6166aab640 1 -- 172.21.15.47:0/2444278547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61600c7b10 con 0x7f61600b8350 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f615d7fa640 1 -- 172.21.15.47:0/2444278547 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615803e800 con 0x7f61600b8350 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.140+0000 7f6166aab640 1 -- 172.21.15.47:0/2444278547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61600c80d0 con 0x7f61600b8350 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.141+0000 7f6166aab640 1 -- 172.21.15.47:0/2444278547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6160072360 con 0x7f61600b8350 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.141+0000 7f615d7fa640 1 -- 172.21.15.47:0/2444278547 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f615803e960 con 0x7f61600b8350 2024-03-23T09:41:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.142+0000 7f615d7fa640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f614006ac30 0x7f614006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.142+0000 7f615d7fa640 1 -- 172.21.15.47:0/2444278547 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61580ac0a0 con 0x7f61600b8350 2024-03-23T09:41:44.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.142+0000 7f615ffff640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f614006ac30 0x7f614006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.143+0000 7f615ffff640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f614006ac30 0x7f614006d0f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6150006fd0 tx=0x7f6150008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:44.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.146+0000 7f615d7fa640 1 -- 172.21.15.47:0/2444278547 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6158077310 con 0x7f61600b8350 2024-03-23T09:41:44.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.326+0000 7f6166aab640 1 -- 172.21.15.47:0/2444278547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6160066be0 con 0x7f61600b8350 2024-03-23T09:41:44.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.326+0000 7f615d7fa640 1 -- 172.21.15.47:0/2444278547 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f615807a930 con 0x7f61600b8350 2024-03-23T09:41:44.327 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.328+0000 7f6166aab640 1 -- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f614006ac30 msgr2=0x7f614006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.328+0000 7f6166aab640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f614006ac30 0x7f614006d0f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6150006fd0 tx=0x7f6150008040 comp rx=0 tx=0).stop 2024-03-23T09:41:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.328+0000 7f6166aab640 1 -- 172.21.15.47:0/2444278547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61600b8350 msgr2=0x7f61601585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.328+0000 7f6166aab640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61600b8350 0x7f61601585b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6158000f80 tx=0x7f6158004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.328+0000 7f6166aab640 1 -- 172.21.15.47:0/2444278547 shutdown_connections 2024-03-23T09:41:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.328+0000 7f6166aab640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f614006ac30 0x7f614006d0f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.328+0000 7f6166aab640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f616015c990 0x7f61600c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.328+0000 7f6166aab640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61600cdc40 0x7f6160155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.328+0000 7f6166aab640 1 --2- 172.21.15.47:0/2444278547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61600b8350 0x7f61601585b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.328+0000 7f6166aab640 1 -- 172.21.15.47:0/2444278547 >> 172.21.15.47:0/2444278547 conn(0x7f61600659d0 msgr2=0x7f6160157bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.328+0000 7f6166aab640 1 -- 172.21.15.47:0/2444278547 shutdown_connections 2024-03-23T09:41:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.328+0000 7f6166aab640 1 -- 172.21.15.47:0/2444278547 wait complete. 2024-03-23T09:41:44.339 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:44.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.442+0000 7fa457fc3640 1 Processor -- start 2024-03-23T09:41:44.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.442+0000 7fa457fc3640 1 -- start start 2024-03-23T09:41:44.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.443+0000 7fa457fc3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa45015c990 0x7fa450160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.443+0000 7fa457fc3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4500cde10 0x7fa4500d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.443+0000 7fa457fc3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa450161630 con 0x7fa4500b8350 2024-03-23T09:41:44.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.443+0000 7fa457fc3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4500d1030 con 0x7fa45015c990 2024-03-23T09:41:44.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.443+0000 7fa457fc3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500cfe10 con 0x7fa4500cde10 2024-03-23T09:41:44.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.443+0000 7fa455d5f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa45015c990 0x7fa450160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.443+0000 7fa455d5f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa45015c990 0x7fa450160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50468/0 (socket says 172.21.15.47:50468) 2024-03-23T09:41:44.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.443+0000 7fa455d5f640 1 -- 172.21.15.47:0/1885419728 learned_addr learned my addr 172.21.15.47:0/1885419728 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:44.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.443+0000 7fa455d5f640 1 -- 172.21.15.47:0/1885419728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4500cde10 msgr2=0x7fa4500d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:44.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.443+0000 7fa456560640 1 --2- 172.21.15.47:0/1885419728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4500cde10 0x7fa4500d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.443+0000 7fa455d5f640 1 --2- 172.21.15.47:0/1885419728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4500cde10 0x7fa4500d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.443+0000 7fa455d5f640 1 -- 172.21.15.47:0/1885419728 >> v1:172.21.15.175:6789/0 conn(0x7fa4500b8350 legacy=0x7fa4500cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:44.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.444+0000 7fa455d5f640 1 -- 172.21.15.47:0/1885419728 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500b5b60 con 0x7fa45015c990 2024-03-23T09:41:44.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.444+0000 7fa456560640 1 --2- 172.21.15.47:0/1885419728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4500cde10 0x7fa4500d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:44.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.444+0000 7fa455d5f640 1 --2- 172.21.15.47:0/1885419728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45015c990 0x7fa450160690 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa448009870 tx=0x7fa44802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1c75f53c993380 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:44.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.444+0000 7fa454d5d640 1 -- 172.21.15.47:0/1885419728 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480317c0 con 0x7fa45015c990 2024-03-23T09:41:44.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.444+0000 7fa454d5d640 1 -- 172.21.15.47:0/1885419728 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa448031920 con 0x7fa45015c990 2024-03-23T09:41:44.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.444+0000 7fa454d5d640 1 -- 172.21.15.47:0/1885419728 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448031cb0 con 0x7fa45015c990 2024-03-23T09:41:44.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.445+0000 7fa457fc3640 1 -- 172.21.15.47:0/1885419728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45015c990 msgr2=0x7fa450160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.445+0000 7fa457fc3640 1 --2- 172.21.15.47:0/1885419728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45015c990 0x7fa450160690 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa448009870 tx=0x7fa44802b860 comp rx=0 tx=0).stop 2024-03-23T09:41:44.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.445+0000 7fa457fc3640 1 -- 172.21.15.47:0/1885419728 shutdown_connections 2024-03-23T09:41:44.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.445+0000 7fa457fc3640 1 --2- 172.21.15.47:0/1885419728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4500cde10 0x7fa4500d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.445+0000 7fa457fc3640 1 --2- 172.21.15.47:0/1885419728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45015c990 0x7fa450160690 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.445+0000 7fa457fc3640 1 -- 172.21.15.47:0/1885419728 >> 172.21.15.47:0/1885419728 conn(0x7fa4500659d0 msgr2=0x7fa450065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:44.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.445+0000 7fa457fc3640 1 -- 172.21.15.47:0/1885419728 shutdown_connections 2024-03-23T09:41:44.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.445+0000 7fa457fc3640 1 -- 172.21.15.47:0/1885419728 wait complete. 2024-03-23T09:41:44.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.446+0000 7fa457fc3640 1 Processor -- start 2024-03-23T09:41:44.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.446+0000 7fa457fc3640 1 -- start start 2024-03-23T09:41:44.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa457fc3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4500b8350 0x7fa4501f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa457fc3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4500cde10 0x7fa4501560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa457fc3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa45015c990 0x7fa45015fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa457fc3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45015e2f0 con 0x7fa4500cde10 2024-03-23T09:41:44.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa457fc3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45015d6e0 con 0x7fa45015c990 2024-03-23T09:41:44.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa457fc3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45015d9c0 con 0x7fa4500b8350 2024-03-23T09:41:44.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa45555e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4500cde10 0x7fa4501560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa45555e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4500cde10 0x7fa4501560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50476/0 (socket says 172.21.15.47:50476) 2024-03-23T09:41:44.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa45555e640 1 -- 172.21.15.47:0/2679767570 learned_addr learned my addr 172.21.15.47:0/2679767570 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:44.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa45555e640 1 -- 172.21.15.47:0/2679767570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4500b8350 msgr2=0x7fa4501f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa45555e640 1 --2- 172.21.15.47:0/2679767570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4500b8350 0x7fa4501f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa456560640 1 --2- 172.21.15.47:0/2679767570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa45015c990 0x7fa45015fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa45555e640 1 -- 172.21.15.47:0/2679767570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa45015c990 msgr2=0x7fa45015fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa45555e640 1 --2- 172.21.15.47:0/2679767570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa45015c990 0x7fa45015fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa45555e640 1 -- 172.21.15.47:0/2679767570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4480094d0 con 0x7fa4500cde10 2024-03-23T09:41:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.447+0000 7fa45555e640 1 --2- 172.21.15.47:0/2679767570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4500cde10 0x7fa4501560b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa44400b3f0 tx=0x7fa44400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.448+0000 7fa44e7fc640 1 -- 172.21.15.47:0/2679767570 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444011830 con 0x7fa4500cde10 2024-03-23T09:41:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.448+0000 7fa44e7fc640 1 -- 172.21.15.47:0/2679767570 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444011990 con 0x7fa4500cde10 2024-03-23T09:41:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.448+0000 7fa457fc3640 1 -- 172.21.15.47:0/2679767570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45015dca0 con 0x7fa4500cde10 2024-03-23T09:41:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.448+0000 7fa44e7fc640 1 -- 172.21.15.47:0/2679767570 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444011d40 con 0x7fa4500cde10 2024-03-23T09:41:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.448+0000 7fa457fc3640 1 -- 172.21.15.47:0/2679767570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa450160010 con 0x7fa4500cde10 2024-03-23T09:41:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.449+0000 7fa457fc3640 1 -- 172.21.15.47:0/2679767570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa450072360 con 0x7fa4500cde10 2024-03-23T09:41:44.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.449+0000 7fa44e7fc640 1 -- 172.21.15.47:0/2679767570 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa44401a400 con 0x7fa4500cde10 2024-03-23T09:41:44.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.449+0000 7fa44e7fc640 1 --2- 172.21.15.47:0/2679767570 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa43c06ac00 0x7fa43c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.449+0000 7fa44e7fc640 1 -- 172.21.15.47:0/2679767570 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4440891f0 con 0x7fa4500cde10 2024-03-23T09:41:44.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.450+0000 7fa455d5f640 1 --2- 172.21.15.47:0/2679767570 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa43c06ac00 0x7fa43c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.450+0000 7fa455d5f640 1 --2- 172.21.15.47:0/2679767570 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa43c06ac00 0x7fa43c06d0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa448036000 tx=0x7fa448030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:44.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.454+0000 7fa44e7fc640 1 -- 172.21.15.47:0/2679767570 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa4440543d0 con 0x7fa4500cde10 2024-03-23T09:41:44.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.631+0000 7fa457fc3640 1 -- 172.21.15.47:0/2679767570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa450066be0 con 0x7fa4500cde10 2024-03-23T09:41:44.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.631+0000 7fa44e7fc640 1 -- 172.21.15.47:0/2679767570 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4440579f0 con 0x7fa4500cde10 2024-03-23T09:41:44.631 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:44.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.633+0000 7fa457fc3640 1 -- 172.21.15.47:0/2679767570 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa43c06ac00 msgr2=0x7fa43c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.633+0000 7fa457fc3640 1 --2- 172.21.15.47:0/2679767570 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa43c06ac00 0x7fa43c06d0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa448036000 tx=0x7fa448030000 comp rx=0 tx=0).stop 2024-03-23T09:41:44.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.633+0000 7fa457fc3640 1 -- 172.21.15.47:0/2679767570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4500cde10 msgr2=0x7fa4501560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.633+0000 7fa457fc3640 1 --2- 172.21.15.47:0/2679767570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4500cde10 0x7fa4501560b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa44400b3f0 tx=0x7fa44400b8b0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.633+0000 7fa457fc3640 1 -- 172.21.15.47:0/2679767570 shutdown_connections 2024-03-23T09:41:44.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.633+0000 7fa457fc3640 1 --2- 172.21.15.47:0/2679767570 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa43c06ac00 0x7fa43c06d0c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.633+0000 7fa457fc3640 1 --2- 172.21.15.47:0/2679767570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa45015c990 0x7fa45015fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.633+0000 7fa457fc3640 1 --2- 172.21.15.47:0/2679767570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4500cde10 0x7fa4501560b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.633+0000 7fa457fc3640 1 --2- 172.21.15.47:0/2679767570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4500b8350 0x7fa4501f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.633+0000 7fa457fc3640 1 -- 172.21.15.47:0/2679767570 >> 172.21.15.47:0/2679767570 conn(0x7fa4500659d0 msgr2=0x7fa4500b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:44.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.633+0000 7fa457fc3640 1 -- 172.21.15.47:0/2679767570 shutdown_connections 2024-03-23T09:41:44.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.633+0000 7fa457fc3640 1 -- 172.21.15.47:0/2679767570 wait complete. 2024-03-23T09:41:44.643 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:41:44.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.745+0000 7faeafe70640 1 Processor -- start 2024-03-23T09:41:44.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.746+0000 7faeafe70640 1 -- start start 2024-03-23T09:41:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.746+0000 7faeafe70640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faea806b400 0x7faea80cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.746+0000 7faeafe70640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea80ccea0 con 0x7faea815c8c0 2024-03-23T09:41:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.746+0000 7faeafe70640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea80c9550 con 0x7faea815cca0 2024-03-23T09:41:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.746+0000 7faeafe70640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea80719a0 con 0x7faea806b400 2024-03-23T09:41:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.746+0000 7faeadc0c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7faea815c8c0 0x7faea806acf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47936/0 (socket says 172.21.15.47:47936) 2024-03-23T09:41:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeadc0c640 1 -- 172.21.15.47:0/154306848 learned_addr learned my addr 172.21.15.47:0/154306848 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeae40d640 1 --2- 172.21.15.47:0/154306848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faea806b400 0x7faea80cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058172311 0 0) 0x7faea4001110 con 0x7faea815c8c0 2024-03-23T09:41:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae94003520 con 0x7faea815c8c0 2024-03-23T09:41:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1468142225 0 0) 0x7fae94003520 con 0x7faea815c8c0 2024-03-23T09:41:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faea806b400 msgr2=0x7faea80cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 --2- 172.21.15.47:0/154306848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faea806b400 0x7faea80cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 >> v1:172.21.15.175:6789/0 conn(0x7faea815cca0 legacy=0x7faea80ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea80c9830 con 0x7faea815c8c0 2024-03-23T09:41:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7faea4001800 con 0x7faea815c8c0 2024-03-23T09:41:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 >> v1:172.21.15.47:6789/0 conn(0x7faea815c8c0 legacy=0x7faea806acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 --2- 172.21.15.47:0/154306848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae940039a0 0x7fae94003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 --2- 172.21.15.47:0/154306848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faea806b400 0x7fae940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 --2- 172.21.15.47:0/154306848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae94004d00 0x7fae94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea80719a0 con 0x7fae94004d00 2024-03-23T09:41:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9400b9b0 con 0x7fae940039a0 2024-03-23T09:41:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9400bb70 con 0x7faea806b400 2024-03-23T09:41:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeae40d640 1 --2- 172.21.15.47:0/154306848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae94004d00 0x7fae94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faeadc0c640 1 --2- 172.21.15.47:0/154306848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae940039a0 0x7fae94003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.747+0000 7faead40b640 1 --2- 172.21.15.47:0/154306848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faea806b400 0x7fae940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.748+0000 7faeae40d640 1 -- 172.21.15.47:0/154306848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faea806b400 msgr2=0x7fae940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.748+0000 7faeae40d640 1 --2- 172.21.15.47:0/154306848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faea806b400 0x7fae940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.748+0000 7faeae40d640 1 -- 172.21.15.47:0/154306848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae940039a0 msgr2=0x7fae94003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.748+0000 7faeae40d640 1 --2- 172.21.15.47:0/154306848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae940039a0 0x7fae94003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.748+0000 7faeae40d640 1 -- 172.21.15.47:0/154306848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94003520 con 0x7fae94004d00 2024-03-23T09:41:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.748+0000 7faeae40d640 1 --2- 172.21.15.47:0/154306848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae94004d00 0x7fae94005180 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faea8065af0 tx=0x7fae940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899fa6bf7a7c8a91 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.748+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea002cb90 con 0x7fae94004d00 2024-03-23T09:41:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.748+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea000f040 con 0x7fae94004d00 2024-03-23T09:41:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.748+0000 7faeacc0a640 1 -- 172.21.15.47:0/154306848 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0034440 con 0x7fae94004d00 2024-03-23T09:41:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.749+0000 7faeafe70640 1 -- 172.21.15.47:0/154306848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae94004d00 msgr2=0x7fae94005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.749+0000 7faeafe70640 1 --2- 172.21.15.47:0/154306848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae94004d00 0x7fae94005180 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faea8065af0 tx=0x7fae940057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.749+0000 7faeae40d640 1 -- 172.21.15.47:0/154306848 reap_dead start 2024-03-23T09:41:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.749+0000 7faeafe70640 1 -- 172.21.15.47:0/154306848 shutdown_connections 2024-03-23T09:41:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.749+0000 7faeafe70640 1 -- 172.21.15.47:0/154306848 >> 172.21.15.47:0/154306848 conn(0x7faea8070ad0 msgr2=0x7faea8070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.749+0000 7faeafe70640 1 -- 172.21.15.47:0/154306848 shutdown_connections 2024-03-23T09:41:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.749+0000 7faeafe70640 1 -- 172.21.15.47:0/154306848 wait complete. 2024-03-23T09:41:44.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.750+0000 7faeafe70640 1 Processor -- start 2024-03-23T09:41:44.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.750+0000 7faeafe70640 1 -- start start 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeafe70640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faea814dc80 0x7faea80acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeafe70640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faea80ad8c0 0x7faea815a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeafe70640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faea815ab40 0x7faea81f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeafe70640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea815afc0 con 0x7faea815ab40 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeafe70640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea80b2720 con 0x7faea814dc80 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeafe70640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea80b1b10 con 0x7faea80ad8c0 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeae40d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faea815ab40 0x7faea81f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeae40d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faea815ab40 0x7faea81f5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48560/0 (socket says 172.21.15.47:48560) 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeae40d640 1 -- 172.21.15.47:0/94246184 learned_addr learned my addr 172.21.15.47:0/94246184 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeadc0c640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faea814dc80 0x7faea80acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faead40b640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faea80ad8c0 0x7faea815a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeae40d640 1 -- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faea80ad8c0 msgr2=0x7faea815a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeae40d640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faea80ad8c0 0x7faea815a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeae40d640 1 -- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faea814dc80 msgr2=0x7faea80acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeae40d640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faea814dc80 0x7faea80acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeae40d640 1 -- 172.21.15.47:0/94246184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea80b1df0 con 0x7faea815ab40 2024-03-23T09:41:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.751+0000 7faeadc0c640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faea814dc80 0x7faea80acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.752+0000 7faeae40d640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faea815ab40 0x7faea81f5d50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faea0037040 tx=0x7faea0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.752+0000 7fae9e7fc640 1 -- 172.21.15.47:0/94246184 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0034cd0 con 0x7faea815ab40 2024-03-23T09:41:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.752+0000 7fae9e7fc640 1 -- 172.21.15.47:0/94246184 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea0034e30 con 0x7faea815ab40 2024-03-23T09:41:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.752+0000 7faeafe70640 1 -- 172.21.15.47:0/94246184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea80b20d0 con 0x7faea815ab40 2024-03-23T09:41:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.752+0000 7fae9e7fc640 1 -- 172.21.15.47:0/94246184 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea003e680 con 0x7faea815ab40 2024-03-23T09:41:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.752+0000 7faeafe70640 1 -- 172.21.15.47:0/94246184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea80ad020 con 0x7faea815ab40 2024-03-23T09:41:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.753+0000 7faeafe70640 1 -- 172.21.15.47:0/94246184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea80672f0 con 0x7faea815ab40 2024-03-23T09:41:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.753+0000 7fae9e7fc640 1 -- 172.21.15.47:0/94246184 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7faea003e7e0 con 0x7faea815ab40 2024-03-23T09:41:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.754+0000 7fae9e7fc640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae8c06ac00 0x7fae8c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.754+0000 7fae9e7fc640 1 -- 172.21.15.47:0/94246184 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faea00ab3e0 con 0x7faea815ab40 2024-03-23T09:41:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.754+0000 7faeadc0c640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae8c06ac00 0x7fae8c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.755+0000 7faeadc0c640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae8c06ac00 0x7fae8c06d0c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faea4001270 tx=0x7faea4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:44.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.758+0000 7fae9e7fc640 1 -- 172.21.15.47:0/94246184 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7faea00765b0 con 0x7faea815ab40 2024-03-23T09:41:44.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.950+0000 7faeafe70640 1 -- 172.21.15.47:0/94246184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faea80ae740 con 0x7faea815ab40 2024-03-23T09:41:44.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.950+0000 7fae9e7fc640 1 -- 172.21.15.47:0/94246184 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7faea0079bd0 con 0x7faea815ab40 2024-03-23T09:41:44.950 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:41:44.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.952+0000 7faeafe70640 1 -- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae8c06ac00 msgr2=0x7fae8c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.952+0000 7faeafe70640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae8c06ac00 0x7fae8c06d0c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faea4001270 tx=0x7faea4009040 comp rx=0 tx=0).stop 2024-03-23T09:41:44.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.952+0000 7faeafe70640 1 -- 172.21.15.47:0/94246184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faea815ab40 msgr2=0x7faea81f5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:44.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.952+0000 7faeafe70640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faea815ab40 0x7faea81f5d50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faea0037040 tx=0x7faea0009750 comp rx=0 tx=0).stop 2024-03-23T09:41:44.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.952+0000 7faeafe70640 1 -- 172.21.15.47:0/94246184 shutdown_connections 2024-03-23T09:41:44.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.952+0000 7faeafe70640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae8c06ac00 0x7fae8c06d0c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.952+0000 7faeafe70640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faea815ab40 0x7faea81f5d50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.952+0000 7faeafe70640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faea80ad8c0 0x7faea815a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.952+0000 7faeafe70640 1 --2- 172.21.15.47:0/94246184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faea814dc80 0x7faea80acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:44.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.952+0000 7faeafe70640 1 -- 172.21.15.47:0/94246184 >> 172.21.15.47:0/94246184 conn(0x7faea8070ad0 msgr2=0x7faea81589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:44.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.952+0000 7faeafe70640 1 -- 172.21.15.47:0/94246184 shutdown_connections 2024-03-23T09:41:44.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:44.952+0000 7faeafe70640 1 -- 172.21.15.47:0/94246184 wait complete. 2024-03-23T09:41:44.963 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:41:44.963 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:41:44.981 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:45.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.082+0000 7fcab8438640 1 Processor -- start 2024-03-23T09:41:45.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.082+0000 7fcab8438640 1 -- start start 2024-03-23T09:41:45.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.083+0000 7fcab8438640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcab00b0670 0x7fcab0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.083+0000 7fcab8438640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcab00d01a0 0x7fcab00b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.083+0000 7fcab8438640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab006a370 con 0x7fcab015ac10 2024-03-23T09:41:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.083+0000 7fcab8438640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab006a650 con 0x7fcab00d01a0 2024-03-23T09:41:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.083+0000 7fcab8438640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab015b150 con 0x7fcab00b0670 2024-03-23T09:41:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.083+0000 7fcab69d5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcab00d01a0 0x7fcab00b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.083+0000 7fcab59d3640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcab015ac10 0x7fcab015b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47938/0 (socket says 172.21.15.47:47938) 2024-03-23T09:41:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.083+0000 7fcab59d3640 1 -- 172.21.15.47:0/2044233316 learned_addr learned my addr 172.21.15.47:0/2044233316 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.084+0000 7fcab61d4640 1 --2- 172.21.15.47:0/2044233316 >> v2:172.21.15.175:3300/0 conn(0x7fcab00b0670 0x7fcab0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.084+0000 7fcab69d5640 1 -- 172.21.15.47:0/2044233316 >> v2:172.21.15.175:3300/0 conn(0x7fcab00b0670 msgr2=0x7fcab0069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.084+0000 7fcab69d5640 1 --2- 172.21.15.47:0/2044233316 >> v2:172.21.15.175:3300/0 conn(0x7fcab00b0670 0x7fcab0069ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.084+0000 7fcab69d5640 1 -- 172.21.15.47:0/2044233316 >> v1:172.21.15.47:6789/0 conn(0x7fcab015ac10 legacy=0x7fcab015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.084+0000 7fcab51d2640 1 -- 172.21.15.47:0/2044233316 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365106084 0 0) 0x7fcab006a370 con 0x7fcab015ac10 2024-03-23T09:41:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.084+0000 7fcab69d5640 1 -- 172.21.15.47:0/2044233316 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab015b430 con 0x7fcab00d01a0 2024-03-23T09:41:45.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.084+0000 7fcab61d4640 1 --2- 172.21.15.47:0/2044233316 >> v2:172.21.15.175:3300/0 conn(0x7fcab00b0670 0x7fcab0069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:45.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.084+0000 7fcab69d5640 1 --2- 172.21.15.47:0/2044233316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcab00d01a0 0x7fcab00b0a50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcab00cc260 tx=0x7fcaa802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8610252683afc5b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:45.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.085+0000 7fcab51d2640 1 -- 172.21.15.47:0/2044233316 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa802cc80 con 0x7fcab00d01a0 2024-03-23T09:41:45.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.085+0000 7fcab51d2640 1 -- 172.21.15.47:0/2044233316 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa802cde0 con 0x7fcab00d01a0 2024-03-23T09:41:45.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.085+0000 7fcab51d2640 1 -- 172.21.15.47:0/2044233316 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8034660 con 0x7fcab00d01a0 2024-03-23T09:41:45.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.085+0000 7fcab8438640 1 -- 172.21.15.47:0/2044233316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcab00d01a0 msgr2=0x7fcab00b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.085+0000 7fcab8438640 1 --2- 172.21.15.47:0/2044233316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcab00d01a0 0x7fcab00b0a50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcab00cc260 tx=0x7fcaa802bed0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.085+0000 7fcab8438640 1 -- 172.21.15.47:0/2044233316 shutdown_connections 2024-03-23T09:41:45.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.085+0000 7fcab8438640 1 --2- 172.21.15.47:0/2044233316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcab00d01a0 0x7fcab00b0a50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.085+0000 7fcab8438640 1 --2- 172.21.15.47:0/2044233316 >> v2:172.21.15.175:3300/0 conn(0x7fcab00b0670 0x7fcab0069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.085+0000 7fcab8438640 1 -- 172.21.15.47:0/2044233316 >> 172.21.15.47:0/2044233316 conn(0x7fcab0098a00 msgr2=0x7fcab00cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:45.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.085+0000 7fcab8438640 1 -- 172.21.15.47:0/2044233316 shutdown_connections 2024-03-23T09:41:45.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.085+0000 7fcab8438640 1 -- 172.21.15.47:0/2044233316 wait complete. 2024-03-23T09:41:45.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.086+0000 7fcab8438640 1 Processor -- start 2024-03-23T09:41:45.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.086+0000 7fcab8438640 1 -- start start 2024-03-23T09:41:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.087+0000 7fcab8438640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcab00b0670 0x7fcab0154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.087+0000 7fcab8438640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcab00d01a0 0x7fcab014eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.087+0000 7fcab8438640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcab015ac10 0x7fcab0157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.087+0000 7fcab8438640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab0154590 con 0x7fcab00b0670 2024-03-23T09:41:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.087+0000 7fcab8438640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab01517a0 con 0x7fcab00d01a0 2024-03-23T09:41:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.087+0000 7fcab8438640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00b1c00 con 0x7fcab015ac10 2024-03-23T09:41:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.087+0000 7fcab61d4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcab00b0670 0x7fcab0154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.087+0000 7fcab61d4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcab00b0670 0x7fcab0154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48578/0 (socket says 172.21.15.47:48578) 2024-03-23T09:41:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.087+0000 7fcab61d4640 1 -- 172.21.15.47:0/4008902136 learned_addr learned my addr 172.21.15.47:0/4008902136 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.087+0000 7fcab59d3640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcab00d01a0 0x7fcab014eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.087+0000 7fcab69d5640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcab015ac10 0x7fcab0157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.087+0000 7fcab61d4640 1 -- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcab015ac10 msgr2=0x7fcab0157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.088+0000 7fcab61d4640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcab015ac10 0x7fcab0157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.088+0000 7fcab61d4640 1 -- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcab00d01a0 msgr2=0x7fcab014eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.088+0000 7fcab61d4640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcab00d01a0 0x7fcab014eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.088+0000 7fcab61d4640 1 -- 172.21.15.47:0/4008902136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa8009a70 con 0x7fcab00b0670 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.088+0000 7fcab59d3640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcab00d01a0 0x7fcab014eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.088+0000 7fcab69d5640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcab015ac10 0x7fcab0157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.088+0000 7fcab61d4640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcab00b0670 0x7fcab0154050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00c7f0 tx=0x7fcaac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.088+0000 7fcaa6ffd640 1 -- 172.21.15.47:0/4008902136 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac007a30 con 0x7fcab00b0670 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.088+0000 7fcaa6ffd640 1 -- 172.21.15.47:0/4008902136 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac007b90 con 0x7fcab00b0670 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.088+0000 7fcab8438640 1 -- 172.21.15.47:0/4008902136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab0151cf0 con 0x7fcab00b0670 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.088+0000 7fcaa6ffd640 1 -- 172.21.15.47:0/4008902136 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00ce40 con 0x7fcab00b0670 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.088+0000 7fcab8438640 1 -- 172.21.15.47:0/4008902136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab0151a10 con 0x7fcab00b0670 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.089+0000 7fcab8438640 1 -- 172.21.15.47:0/4008902136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab0153d70 con 0x7fcab00b0670 2024-03-23T09:41:45.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.089+0000 7fcaa6ffd640 1 -- 172.21.15.47:0/4008902136 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcaac0113f0 con 0x7fcab00b0670 2024-03-23T09:41:45.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.090+0000 7fcaa6ffd640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca9406ac30 0x7fca9406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.090+0000 7fcaa6ffd640 1 -- 172.21.15.47:0/4008902136 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcaac0895d0 con 0x7fcab00b0670 2024-03-23T09:41:45.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.090+0000 7fcab59d3640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca9406ac30 0x7fca9406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.091+0000 7fcab59d3640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca9406ac30 0x7fca9406d0f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcab0150db0 tx=0x7fcaa0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:45.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.094+0000 7fcaa6ffd640 1 -- 172.21.15.47:0/4008902136 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcab0153d70 con 0x7fcab00b0670 2024-03-23T09:41:45.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.276+0000 7fcab8438640 1 -- 172.21.15.47:0/4008902136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcab0153d70 con 0x7fcab00b0670 2024-03-23T09:41:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.277+0000 7fcaa6ffd640 1 -- 172.21.15.47:0/4008902136 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fcab0153d70 con 0x7fcab00b0670 2024-03-23T09:41:45.277 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:45.277 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:45.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.279+0000 7fcab8438640 1 -- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca9406ac30 msgr2=0x7fca9406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.279+0000 7fcab8438640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca9406ac30 0x7fca9406d0f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcab0150db0 tx=0x7fcaa0009040 comp rx=0 tx=0).stop 2024-03-23T09:41:45.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.279+0000 7fcab8438640 1 -- 172.21.15.47:0/4008902136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcab00b0670 msgr2=0x7fcab0154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.279+0000 7fcab8438640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcab00b0670 0x7fcab0154050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00c7f0 tx=0x7fcaac00cbf0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.279+0000 7fcab8438640 1 -- 172.21.15.47:0/4008902136 shutdown_connections 2024-03-23T09:41:45.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.279+0000 7fcab8438640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca9406ac30 0x7fca9406d0f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.279+0000 7fcab8438640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcab015ac10 0x7fcab0157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.279+0000 7fcab8438640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcab00d01a0 0x7fcab014eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.279+0000 7fcab8438640 1 --2- 172.21.15.47:0/4008902136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcab00b0670 0x7fcab0154050 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.279+0000 7fcab8438640 1 -- 172.21.15.47:0/4008902136 >> 172.21.15.47:0/4008902136 conn(0x7fcab0098a00 msgr2=0x7fcab006d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.279+0000 7fcab8438640 1 -- 172.21.15.47:0/4008902136 shutdown_connections 2024-03-23T09:41:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.279+0000 7fcab8438640 1 -- 172.21.15.47:0/4008902136 wait complete. 2024-03-23T09:41:45.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:45.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.394+0000 7efddd810640 1 Processor -- start 2024-03-23T09:41:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.394+0000 7efddd810640 1 -- start start 2024-03-23T09:41:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.394+0000 7efddd810640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efdd815c990 0x7efdd8160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.394+0000 7efddd810640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd80b8350 0x7efdd80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.394+0000 7efddd810640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efdd80cdc40 0x7efdd80b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.394+0000 7efddd810640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd80ce0b0 con 0x7efdd815c990 2024-03-23T09:41:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.394+0000 7efddd810640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd8161630 con 0x7efdd80cdc40 2024-03-23T09:41:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.394+0000 7efddd810640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd80d1030 con 0x7efdd80b8350 2024-03-23T09:41:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.395+0000 7efdd6ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efdd815c990 0x7efdd8160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.395+0000 7efdd6ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efdd815c990 0x7efdd8160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48586/0 (socket says 172.21.15.47:48586) 2024-03-23T09:41:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.395+0000 7efdd6ffd640 1 -- 172.21.15.47:0/1806442474 learned_addr learned my addr 172.21.15.47:0/1806442474 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.395+0000 7efdd77fe640 1 --2- 172.21.15.47:0/1806442474 >> v2:172.21.15.175:3300/0 conn(0x7efdd80cdc40 0x7efdd80b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.395+0000 7efdd6ffd640 1 -- 172.21.15.47:0/1806442474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd80b8350 msgr2=0x7efdd80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.395+0000 7efdd67fc640 1 --2- 172.21.15.47:0/1806442474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd80b8350 0x7efdd80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.395+0000 7efdd6ffd640 1 --2- 172.21.15.47:0/1806442474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd80b8350 0x7efdd80cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.395+0000 7efdd6ffd640 1 -- 172.21.15.47:0/1806442474 >> v2:172.21.15.175:3300/0 conn(0x7efdd80cdc40 msgr2=0x7efdd80b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.395+0000 7efdd6ffd640 1 --2- 172.21.15.47:0/1806442474 >> v2:172.21.15.175:3300/0 conn(0x7efdd80cdc40 0x7efdd80b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.395+0000 7efdd6ffd640 1 -- 172.21.15.47:0/1806442474 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd80cfe10 con 0x7efdd815c990 2024-03-23T09:41:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.395+0000 7efdd77fe640 1 --2- 172.21.15.47:0/1806442474 >> v2:172.21.15.175:3300/0 conn(0x7efdd80cdc40 0x7efdd80b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.395+0000 7efdd67fc640 1 --2- 172.21.15.47:0/1806442474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd80b8350 0x7efdd80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.396+0000 7efdd6ffd640 1 --2- 172.21.15.47:0/1806442474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd815c990 0x7efdd8160690 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efdd80b88c0 tx=0x7efdc802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102a8873701ede1f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.396+0000 7efdd5ffb640 1 -- 172.21.15.47:0/1806442474 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc80317e0 con 0x7efdd815c990 2024-03-23T09:41:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.396+0000 7efdd5ffb640 1 -- 172.21.15.47:0/1806442474 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc8031940 con 0x7efdd815c990 2024-03-23T09:41:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.396+0000 7efdd5ffb640 1 -- 172.21.15.47:0/1806442474 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8031cd0 con 0x7efdd815c990 2024-03-23T09:41:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.396+0000 7efddd810640 1 -- 172.21.15.47:0/1806442474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd815c990 msgr2=0x7efdd8160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.396+0000 7efddd810640 1 --2- 172.21.15.47:0/1806442474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd815c990 0x7efdd8160690 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efdd80b88c0 tx=0x7efdc802b880 comp rx=0 tx=0).stop 2024-03-23T09:41:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.396+0000 7efddd810640 1 -- 172.21.15.47:0/1806442474 shutdown_connections 2024-03-23T09:41:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.396+0000 7efddd810640 1 --2- 172.21.15.47:0/1806442474 >> v2:172.21.15.175:3300/0 conn(0x7efdd80cdc40 0x7efdd80b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.396+0000 7efddd810640 1 --2- 172.21.15.47:0/1806442474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd80b8350 0x7efdd80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.396+0000 7efddd810640 1 --2- 172.21.15.47:0/1806442474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd815c990 0x7efdd8160690 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.396+0000 7efddd810640 1 -- 172.21.15.47:0/1806442474 >> 172.21.15.47:0/1806442474 conn(0x7efdd80659d0 msgr2=0x7efdd8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.396+0000 7efddd810640 1 -- 172.21.15.47:0/1806442474 shutdown_connections 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.396+0000 7efddd810640 1 -- 172.21.15.47:0/1806442474 wait complete. 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.398+0000 7efddd810640 1 Processor -- start 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.398+0000 7efddd810640 1 -- start start 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.398+0000 7efddd810640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd80b8350 0x7efdd81f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.398+0000 7efddd810640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd80cdc40 0x7efdd80c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.398+0000 7efddd810640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd815c990 0x7efdd814e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.398+0000 7efddd810640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd815d0e0 con 0x7efdd815c990 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.398+0000 7efddd810640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd815d3c0 con 0x7efdd80b8350 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.398+0000 7efddd810640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd815d6a0 con 0x7efdd80cdc40 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdd77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd815c990 0x7efdd814e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdd77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd815c990 0x7efdd814e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48590/0 (socket says 172.21.15.47:48590) 2024-03-23T09:41:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdd77fe640 1 -- 172.21.15.47:0/3898834578 learned_addr learned my addr 172.21.15.47:0/3898834578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdd67fc640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd80cdc40 0x7efdd80c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdd6ffd640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd80b8350 0x7efdd81f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdd77fe640 1 -- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd80cdc40 msgr2=0x7efdd80c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdd77fe640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd80cdc40 0x7efdd80c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdd77fe640 1 -- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd80b8350 msgr2=0x7efdd81f1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdd77fe640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd80b8350 0x7efdd81f1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdd77fe640 1 -- 172.21.15.47:0/3898834578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc80094d0 con 0x7efdd815c990 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdd77fe640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd815c990 0x7efdd814e040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efdd000cef0 tx=0x7efdd000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdbffff640 1 -- 172.21.15.47:0/3898834578 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd000d840 con 0x7efdd815c990 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdbffff640 1 -- 172.21.15.47:0/3898834578 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd000d9c0 con 0x7efdd815c990 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efdbffff640 1 -- 172.21.15.47:0/3898834578 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd000dd70 con 0x7efdd815c990 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efddd810640 1 -- 172.21.15.47:0/3898834578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd815d980 con 0x7efdd815c990 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.399+0000 7efddd810640 1 -- 172.21.15.47:0/3898834578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd81f2080 con 0x7efdd815c990 2024-03-23T09:41:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.400+0000 7efddd810640 1 -- 172.21.15.47:0/3898834578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd8072360 con 0x7efdd815c990 2024-03-23T09:41:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.401+0000 7efdbffff640 1 -- 172.21.15.47:0/3898834578 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efdd001e3b0 con 0x7efdd815c990 2024-03-23T09:41:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.401+0000 7efdbffff640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efdc406ac00 0x7efdc406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.401+0000 7efdbffff640 1 -- 172.21.15.47:0/3898834578 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efdd0089d70 con 0x7efdd815c990 2024-03-23T09:41:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.401+0000 7efdd6ffd640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efdc406ac00 0x7efdc406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.402+0000 7efdd6ffd640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efdc406ac00 0x7efdc406d0c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efdc8036000 tx=0x7efdc803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:45.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.405+0000 7efdbffff640 1 -- 172.21.15.47:0/3898834578 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efdd00550a0 con 0x7efdd815c990 2024-03-23T09:41:45.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.584+0000 7efddd810640 1 -- 172.21.15.47:0/3898834578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efdd815dea0 con 0x7efdd815c990 2024-03-23T09:41:45.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.585+0000 7efdbffff640 1 -- 172.21.15.47:0/3898834578 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7efdd815dea0 con 0x7efdd815c990 2024-03-23T09:41:45.585 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.587+0000 7efddd810640 1 -- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efdc406ac00 msgr2=0x7efdc406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.587+0000 7efddd810640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efdc406ac00 0x7efdc406d0c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efdc8036000 tx=0x7efdc803d040 comp rx=0 tx=0).stop 2024-03-23T09:41:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.587+0000 7efddd810640 1 -- 172.21.15.47:0/3898834578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd815c990 msgr2=0x7efdd814e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.587+0000 7efddd810640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd815c990 0x7efdd814e040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efdd000cef0 tx=0x7efdd000a710 comp rx=0 tx=0).stop 2024-03-23T09:41:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.587+0000 7efddd810640 1 -- 172.21.15.47:0/3898834578 shutdown_connections 2024-03-23T09:41:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.587+0000 7efddd810640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efdc406ac00 0x7efdc406d0c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.587+0000 7efddd810640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd815c990 0x7efdd814e040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.587+0000 7efddd810640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd80cdc40 0x7efdd80c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.587+0000 7efddd810640 1 --2- 172.21.15.47:0/3898834578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd80b8350 0x7efdd81f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.587+0000 7efddd810640 1 -- 172.21.15.47:0/3898834578 >> 172.21.15.47:0/3898834578 conn(0x7efdd80659d0 msgr2=0x7efdd8065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.587+0000 7efddd810640 1 -- 172.21.15.47:0/3898834578 shutdown_connections 2024-03-23T09:41:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.587+0000 7efddd810640 1 -- 172.21.15.47:0/3898834578 wait complete. 2024-03-23T09:41:45.598 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:45.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.701+0000 7fb149789640 1 Processor -- start 2024-03-23T09:41:45.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.702+0000 7fb149789640 1 -- start start 2024-03-23T09:41:45.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.702+0000 7fb149789640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb14415cbf0 0x7fb1440b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.702+0000 7fb149789640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb1440cf9c0 0x7fb1440d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.702+0000 7fb149789640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb144161570 con 0x7fb1440cd140 2024-03-23T09:41:45.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.702+0000 7fb149789640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb144160960 con 0x7fb14415cbf0 2024-03-23T09:41:45.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.702+0000 7fb149789640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb144160c40 con 0x7fb1440cf9c0 2024-03-23T09:41:45.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.702+0000 7fb142ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb14415cbf0 0x7fb1440b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.702+0000 7fb1427fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb1440cd140 0x7fb1440cf2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47942/0 (socket says 172.21.15.47:47942) 2024-03-23T09:41:45.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.702+0000 7fb1427fc640 1 -- 172.21.15.47:0/2594506682 learned_addr learned my addr 172.21.15.47:0/2594506682 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:45.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.703+0000 7fb1437fe640 1 --2- 172.21.15.47:0/2594506682 >> v2:172.21.15.175:3300/0 conn(0x7fb1440cf9c0 0x7fb1440d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.703+0000 7fb142ffd640 1 -- 172.21.15.47:0/2594506682 >> v2:172.21.15.175:3300/0 conn(0x7fb1440cf9c0 msgr2=0x7fb1440d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:45.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.703+0000 7fb142ffd640 1 --2- 172.21.15.47:0/2594506682 >> v2:172.21.15.175:3300/0 conn(0x7fb1440cf9c0 0x7fb1440d0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.703+0000 7fb142ffd640 1 -- 172.21.15.47:0/2594506682 >> v1:172.21.15.47:6789/0 conn(0x7fb1440cd140 legacy=0x7fb1440cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.703+0000 7fb141ffb640 1 -- 172.21.15.47:0/2594506682 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889769180 0 0) 0x7fb144161570 con 0x7fb1440cd140 2024-03-23T09:41:45.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.703+0000 7fb142ffd640 1 -- 172.21.15.47:0/2594506682 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb144160f20 con 0x7fb14415cbf0 2024-03-23T09:41:45.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.704+0000 7fb142ffd640 1 --2- 172.21.15.47:0/2594506682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb14415cbf0 0x7fb1440b1df0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb1440b58e0 tx=0x7fb13c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc89cf5404f1bf75 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:45.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.704+0000 7fb141ffb640 1 -- 172.21.15.47:0/2594506682 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c031830 con 0x7fb14415cbf0 2024-03-23T09:41:45.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.704+0000 7fb141ffb640 1 -- 172.21.15.47:0/2594506682 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13c031990 con 0x7fb14415cbf0 2024-03-23T09:41:45.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.704+0000 7fb141ffb640 1 -- 172.21.15.47:0/2594506682 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c031d20 con 0x7fb14415cbf0 2024-03-23T09:41:45.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.704+0000 7fb149789640 1 -- 172.21.15.47:0/2594506682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb14415cbf0 msgr2=0x7fb1440b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.704+0000 7fb149789640 1 --2- 172.21.15.47:0/2594506682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb14415cbf0 0x7fb1440b1df0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb1440b58e0 tx=0x7fb13c02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.704+0000 7fb149789640 1 -- 172.21.15.47:0/2594506682 shutdown_connections 2024-03-23T09:41:45.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.704+0000 7fb149789640 1 --2- 172.21.15.47:0/2594506682 >> v2:172.21.15.175:3300/0 conn(0x7fb1440cf9c0 0x7fb1440d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.704+0000 7fb149789640 1 --2- 172.21.15.47:0/2594506682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb14415cbf0 0x7fb1440b1df0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.704+0000 7fb149789640 1 -- 172.21.15.47:0/2594506682 >> 172.21.15.47:0/2594506682 conn(0x7fb14406b160 msgr2=0x7fb14406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:45.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.704+0000 7fb149789640 1 -- 172.21.15.47:0/2594506682 shutdown_connections 2024-03-23T09:41:45.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.704+0000 7fb149789640 1 -- 172.21.15.47:0/2594506682 wait complete. 2024-03-23T09:41:45.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.706+0000 7fb149789640 1 Processor -- start 2024-03-23T09:41:45.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.706+0000 7fb149789640 1 -- start start 2024-03-23T09:41:45.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.706+0000 7fb149789640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1440cd140 0x7fb144153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.706+0000 7fb149789640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440cf9c0 0x7fb1440b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.706+0000 7fb149789640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb14415cbf0 0x7fb1440b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.706+0000 7fb149789640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440ccb60 con 0x7fb1440cf9c0 2024-03-23T09:41:45.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.706+0000 7fb149789640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1441f5fc0 con 0x7fb1440cd140 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.706+0000 7fb149789640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440b2c50 con 0x7fb14415cbf0 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.706+0000 7fb1427fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440cf9c0 0x7fb1440b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.706+0000 7fb1427fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440cf9c0 0x7fb1440b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48622/0 (socket says 172.21.15.47:48622) 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.706+0000 7fb1427fc640 1 -- 172.21.15.47:0/2411539636 learned_addr learned my addr 172.21.15.47:0/2411539636 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb1437fe640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb14415cbf0 0x7fb1440b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb142ffd640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1440cd140 0x7fb144153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb1427fc640 1 -- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb14415cbf0 msgr2=0x7fb1440b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb1427fc640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb14415cbf0 0x7fb1440b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb1427fc640 1 -- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1440cd140 msgr2=0x7fb144153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb1427fc640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1440cd140 0x7fb144153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb1427fc640 1 -- 172.21.15.47:0/2411539636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c009400 con 0x7fb1440cf9c0 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb1437fe640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb14415cbf0 0x7fb1440b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb142ffd640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1440cd140 0x7fb144153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:45.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb1427fc640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440cf9c0 0x7fb1440b2200 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00c4d0 tx=0x7fb12c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:45.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb13b7fe640 1 -- 172.21.15.47:0/2411539636 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0074e0 con 0x7fb1440cf9c0 2024-03-23T09:41:45.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb13b7fe640 1 -- 172.21.15.47:0/2411539636 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c007660 con 0x7fb1440cf9c0 2024-03-23T09:41:45.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb13b7fe640 1 -- 172.21.15.47:0/2411539636 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c004a20 con 0x7fb1440cf9c0 2024-03-23T09:41:45.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb149789640 1 -- 172.21.15.47:0/2411539636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1441f5690 con 0x7fb1440cf9c0 2024-03-23T09:41:45.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.707+0000 7fb149789640 1 -- 172.21.15.47:0/2411539636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1441f5c50 con 0x7fb1440cf9c0 2024-03-23T09:41:45.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.708+0000 7fb149789640 1 -- 172.21.15.47:0/2411539636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb144072360 con 0x7fb1440cf9c0 2024-03-23T09:41:45.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.708+0000 7fb13b7fe640 1 -- 172.21.15.47:0/2411539636 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb12c004b80 con 0x7fb1440cf9c0 2024-03-23T09:41:45.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.709+0000 7fb13b7fe640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb12406ac00 0x7fb12406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:45.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.709+0000 7fb13b7fe640 1 -- 172.21.15.47:0/2411539636 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb12c0884f0 con 0x7fb1440cf9c0 2024-03-23T09:41:45.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.709+0000 7fb142ffd640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb12406ac00 0x7fb12406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:45.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.710+0000 7fb142ffd640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb12406ac00 0x7fb12406d0c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb13c036000 tx=0x7fb13c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:45.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.713+0000 7fb13b7fe640 1 -- 172.21.15.47:0/2411539636 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb12c0536e0 con 0x7fb1440cf9c0 2024-03-23T09:41:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.890+0000 7fb149789640 1 -- 172.21.15.47:0/2411539636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb144066be0 con 0x7fb1440cf9c0 2024-03-23T09:41:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.891+0000 7fb13b7fe640 1 -- 172.21.15.47:0/2411539636 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb12c056d00 con 0x7fb1440cf9c0 2024-03-23T09:41:45.891 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.893+0000 7fb149789640 1 -- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb12406ac00 msgr2=0x7fb12406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.893+0000 7fb149789640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb12406ac00 0x7fb12406d0c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb13c036000 tx=0x7fb13c03c040 comp rx=0 tx=0).stop 2024-03-23T09:41:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.893+0000 7fb149789640 1 -- 172.21.15.47:0/2411539636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440cf9c0 msgr2=0x7fb1440b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.893+0000 7fb149789640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440cf9c0 0x7fb1440b2200 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00c4d0 tx=0x7fb12c00c990 comp rx=0 tx=0).stop 2024-03-23T09:41:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.893+0000 7fb149789640 1 -- 172.21.15.47:0/2411539636 shutdown_connections 2024-03-23T09:41:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.893+0000 7fb149789640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb12406ac00 0x7fb12406d0c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.893+0000 7fb149789640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb14415cbf0 0x7fb1440b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.893+0000 7fb149789640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440cf9c0 0x7fb1440b2200 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.893+0000 7fb149789640 1 --2- 172.21.15.47:0/2411539636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1440cd140 0x7fb144153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.893+0000 7fb149789640 1 -- 172.21.15.47:0/2411539636 >> 172.21.15.47:0/2411539636 conn(0x7fb14406b160 msgr2=0x7fb14415be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.893+0000 7fb149789640 1 -- 172.21.15.47:0/2411539636 shutdown_connections 2024-03-23T09:41:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:45.893+0000 7fb149789640 1 -- 172.21.15.47:0/2411539636 wait complete. 2024-03-23T09:41:45.903 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.007+0000 7f7d0238b640 1 Processor -- start 2024-03-23T09:41:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.007+0000 7f7d0238b640 1 -- start start 2024-03-23T09:41:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.008+0000 7f7d0238b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7cfc15c990 0x7f7cfc160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.008+0000 7f7d0238b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc0b8350 0x7f7cfc0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.008+0000 7f7d0238b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc161630 con 0x7f7cfc0cdc40 2024-03-23T09:41:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.008+0000 7f7d0238b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc0ce0b0 con 0x7f7cfc15c990 2024-03-23T09:41:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.008+0000 7f7d0238b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0d1030 con 0x7f7cfc0b8350 2024-03-23T09:41:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.008+0000 7f7cfb7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc0b8350 0x7f7cfc0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.008+0000 7f7cfbfff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7cfc15c990 0x7f7cfc160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.008+0000 7f7d00928640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f7cfc0cdc40 0x7f7cfc0b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:54456/0 (socket says 172.21.15.47:54456) 2024-03-23T09:41:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.008+0000 7f7cfb7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc0b8350 0x7f7cfc0cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:37590/0 (socket says 172.21.15.47:37590) 2024-03-23T09:41:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.008+0000 7f7d00928640 1 -- 172.21.15.47:0/4017523277 learned_addr learned my addr 172.21.15.47:0/4017523277 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.009+0000 7f7cfb7fe640 1 -- 172.21.15.47:0/4017523277 >> v2:172.21.15.175:3300/0 conn(0x7f7cfc15c990 msgr2=0x7f7cfc160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.009+0000 7f7cfb7fe640 1 --2- 172.21.15.47:0/4017523277 >> v2:172.21.15.175:3300/0 conn(0x7f7cfc15c990 0x7f7cfc160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.009+0000 7f7cfb7fe640 1 -- 172.21.15.47:0/4017523277 >> v1:172.21.15.175:6789/0 conn(0x7f7cfc0cdc40 legacy=0x7f7cfc0b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.009+0000 7f7cfb7fe640 1 -- 172.21.15.47:0/4017523277 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0cfe10 con 0x7f7cfc0b8350 2024-03-23T09:41:46.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.009+0000 7f7cfbfff640 1 --2- 172.21.15.47:0/4017523277 >> v2:172.21.15.175:3300/0 conn(0x7f7cfc15c990 0x7f7cfc160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:46.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.010+0000 7f7cfb7fe640 1 --2- 172.21.15.47:0/4017523277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc0b8350 0x7f7cfc0cd700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40097a0 tx=0x7f7ce402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f636d2d339befa54 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:46.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.010+0000 7f7cfaffd640 1 -- 172.21.15.47:0/4017523277 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce402e070 con 0x7f7cfc0b8350 2024-03-23T09:41:46.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.010+0000 7f7cfaffd640 1 -- 172.21.15.47:0/4017523277 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce40315a0 con 0x7f7cfc0b8350 2024-03-23T09:41:46.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.010+0000 7f7cfaffd640 1 -- 172.21.15.47:0/4017523277 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4031930 con 0x7f7cfc0b8350 2024-03-23T09:41:46.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.010+0000 7f7d0238b640 1 -- 172.21.15.47:0/4017523277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc0b8350 msgr2=0x7f7cfc0cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.010+0000 7f7d0238b640 1 --2- 172.21.15.47:0/4017523277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc0b8350 0x7f7cfc0cd700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40097a0 tx=0x7f7ce402b840 comp rx=0 tx=0).stop 2024-03-23T09:41:46.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.010+0000 7f7d0238b640 1 -- 172.21.15.47:0/4017523277 shutdown_connections 2024-03-23T09:41:46.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.010+0000 7f7d0238b640 1 --2- 172.21.15.47:0/4017523277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc0b8350 0x7f7cfc0cd700 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.010+0000 7f7d0238b640 1 --2- 172.21.15.47:0/4017523277 >> v2:172.21.15.175:3300/0 conn(0x7f7cfc15c990 0x7f7cfc160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.010+0000 7f7d0238b640 1 -- 172.21.15.47:0/4017523277 >> 172.21.15.47:0/4017523277 conn(0x7f7cfc0659d0 msgr2=0x7f7cfc065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.010+0000 7f7d0238b640 1 -- 172.21.15.47:0/4017523277 shutdown_connections 2024-03-23T09:41:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.010+0000 7f7d0238b640 1 -- 172.21.15.47:0/4017523277 wait complete. 2024-03-23T09:41:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.011+0000 7f7d0238b640 1 Processor -- start 2024-03-23T09:41:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.012+0000 7f7d0238b640 1 -- start start 2024-03-23T09:41:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.012+0000 7f7d0238b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cfc0b8350 0x7f7cfc0c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.012+0000 7f7d0238b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc0cdc40 0x7f7cfc15d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.012+0000 7f7d0238b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cfc15c990 0x7f7cfc15f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.012+0000 7f7d0238b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc15dfd0 con 0x7f7cfc0b8350 2024-03-23T09:41:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.012+0000 7f7d0238b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc15e2b0 con 0x7f7cfc15c990 2024-03-23T09:41:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.012+0000 7f7d0238b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc15e600 con 0x7f7cfc0cdc40 2024-03-23T09:41:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7cfbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cfc0b8350 0x7f7cfc0c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7cfbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cfc0b8350 0x7f7cfc0c6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48628/0 (socket says 172.21.15.47:48628) 2024-03-23T09:41:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7cfbfff640 1 -- 172.21.15.47:0/4030429016 learned_addr learned my addr 172.21.15.47:0/4030429016 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7d00928640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cfc15c990 0x7f7cfc15f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7cfb7fe640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc0cdc40 0x7f7cfc15d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7cfbfff640 1 -- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc0cdc40 msgr2=0x7f7cfc15d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7cfbfff640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc0cdc40 0x7f7cfc15d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7cfbfff640 1 -- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cfc15c990 msgr2=0x7f7cfc15f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7cfbfff640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cfc15c990 0x7f7cfc15f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7cfbfff640 1 -- 172.21.15.47:0/4030429016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec009400 con 0x7f7cfc0b8350 2024-03-23T09:41:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7cfbfff640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cfc0b8350 0x7f7cfc0c6390 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00ebe0 tx=0x7f7cec00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7cf8ff9640 1 -- 172.21.15.47:0/4030429016 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec009760 con 0x7f7cfc0b8350 2024-03-23T09:41:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7cf8ff9640 1 -- 172.21.15.47:0/4030429016 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec00cc30 con 0x7f7cfc0b8350 2024-03-23T09:41:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7d0238b640 1 -- 172.21.15.47:0/4030429016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce4009400 con 0x7f7cfc0b8350 2024-03-23T09:41:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7cf8ff9640 1 -- 172.21.15.47:0/4030429016 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec00eee0 con 0x7f7cfc0b8350 2024-03-23T09:41:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.013+0000 7f7d0238b640 1 -- 172.21.15.47:0/4030429016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc15ebc0 con 0x7f7cfc0b8350 2024-03-23T09:41:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.014+0000 7f7d0238b640 1 -- 172.21.15.47:0/4030429016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc072360 con 0x7f7cfc0b8350 2024-03-23T09:41:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.015+0000 7f7cf8ff9640 1 -- 172.21.15.47:0/4030429016 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7cec013470 con 0x7f7cfc0b8350 2024-03-23T09:41:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.015+0000 7f7cf8ff9640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7cdc06ac00 0x7f7cdc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.015+0000 7f7cf8ff9640 1 -- 172.21.15.47:0/4030429016 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cec089570 con 0x7f7cfc0b8350 2024-03-23T09:41:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.015+0000 7f7cfb7fe640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7cdc06ac00 0x7f7cdc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.016+0000 7f7cfb7fe640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7cdc06ac00 0x7f7cdc06d0c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4036000 tx=0x7f7ce403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:46.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.019+0000 7f7cf8ff9640 1 -- 172.21.15.47:0/4030429016 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7cec054890 con 0x7f7cfc0b8350 2024-03-23T09:41:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.200+0000 7f7d0238b640 1 -- 172.21.15.47:0/4030429016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7cfc066be0 con 0x7f7cfc0b8350 2024-03-23T09:41:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.201+0000 7f7cf8ff9640 1 -- 172.21.15.47:0/4030429016 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7cec057eb0 con 0x7f7cfc0b8350 2024-03-23T09:41:46.201 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.203+0000 7f7d0238b640 1 -- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7cdc06ac00 msgr2=0x7f7cdc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.203+0000 7f7d0238b640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7cdc06ac00 0x7f7cdc06d0c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4036000 tx=0x7f7ce403d040 comp rx=0 tx=0).stop 2024-03-23T09:41:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.203+0000 7f7d0238b640 1 -- 172.21.15.47:0/4030429016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cfc0b8350 msgr2=0x7f7cfc0c6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.203+0000 7f7d0238b640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cfc0b8350 0x7f7cfc0c6390 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00ebe0 tx=0x7f7cec00c520 comp rx=0 tx=0).stop 2024-03-23T09:41:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.203+0000 7f7d0238b640 1 -- 172.21.15.47:0/4030429016 shutdown_connections 2024-03-23T09:41:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.203+0000 7f7d0238b640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7cdc06ac00 0x7f7cdc06d0c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.203+0000 7f7d0238b640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cfc15c990 0x7f7cfc15f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.203+0000 7f7d0238b640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc0cdc40 0x7f7cfc15d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.203+0000 7f7d0238b640 1 --2- 172.21.15.47:0/4030429016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cfc0b8350 0x7f7cfc0c6390 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.203+0000 7f7d0238b640 1 -- 172.21.15.47:0/4030429016 >> 172.21.15.47:0/4030429016 conn(0x7f7cfc0659d0 msgr2=0x7f7cfc0b4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.203+0000 7f7d0238b640 1 -- 172.21.15.47:0/4030429016 shutdown_connections 2024-03-23T09:41:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.203+0000 7f7d0238b640 1 -- 172.21.15.47:0/4030429016 wait complete. 2024-03-23T09:41:46.214 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:41:46.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.316+0000 7fe84f155640 1 Processor -- start 2024-03-23T09:41:46.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.316+0000 7fe84f155640 1 -- start start 2024-03-23T09:41:46.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.317+0000 7fe84f155640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8480cf8a0 0x7fe848167b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.317+0000 7fe84f155640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8480d0ee0 con 0x7fe84815cb40 2024-03-23T09:41:46.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.317+0000 7fe84f155640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8481616f0 con 0x7fe8480b8300 2024-03-23T09:41:46.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.317+0000 7fe84f155640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8480b5a10 con 0x7fe8480cf8a0 2024-03-23T09:41:46.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.317+0000 7fe84cef1640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe84815cb40 0x7fe848160770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47956/0 (socket says 172.21.15.47:47956) 2024-03-23T09:41:46.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.317+0000 7fe84cef1640 1 -- 172.21.15.47:0/4195435077 learned_addr learned my addr 172.21.15.47:0/4195435077 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:46.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.317+0000 7fe84d6f2640 1 --2- 172.21.15.47:0/4195435077 >> v2:172.21.15.175:3300/0 conn(0x7fe8480cf8a0 0x7fe848167b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.317+0000 7fe8477fe640 1 -- 172.21.15.47:0/4195435077 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941901200 0 0) 0x7fe8480d0ee0 con 0x7fe84815cb40 2024-03-23T09:41:46.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.317+0000 7fe8477fe640 1 -- 172.21.15.47:0/4195435077 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe838003520 con 0x7fe84815cb40 2024-03-23T09:41:46.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.317+0000 7fe84d6f2640 1 -- 172.21.15.47:0/4195435077 >> v1:172.21.15.175:6789/0 conn(0x7fe8480b8300 legacy=0x7fe8480cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.317+0000 7fe84d6f2640 1 -- 172.21.15.47:0/4195435077 >> v1:172.21.15.47:6789/0 conn(0x7fe84815cb40 legacy=0x7fe848160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.317+0000 7fe84d6f2640 1 -- 172.21.15.47:0/4195435077 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe848160e80 con 0x7fe8480cf8a0 2024-03-23T09:41:46.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.318+0000 7fe84d6f2640 1 --2- 172.21.15.47:0/4195435077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8480cf8a0 0x7fe848167b00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe840009dc0 tx=0x7fe84002bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76571bb0c85b55cc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:46.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.318+0000 7fe8477fe640 1 -- 172.21.15.47:0/4195435077 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84002ccd0 con 0x7fe8480cf8a0 2024-03-23T09:41:46.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.318+0000 7fe8477fe640 1 -- 172.21.15.47:0/4195435077 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84002ce30 con 0x7fe8480cf8a0 2024-03-23T09:41:46.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.318+0000 7fe8477fe640 1 -- 172.21.15.47:0/4195435077 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe840034660 con 0x7fe8480cf8a0 2024-03-23T09:41:46.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.319+0000 7fe84f155640 1 -- 172.21.15.47:0/4195435077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8480cf8a0 msgr2=0x7fe848167b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.319+0000 7fe84f155640 1 --2- 172.21.15.47:0/4195435077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8480cf8a0 0x7fe848167b00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe840009dc0 tx=0x7fe84002bf20 comp rx=0 tx=0).stop 2024-03-23T09:41:46.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.319+0000 7fe84f155640 1 -- 172.21.15.47:0/4195435077 shutdown_connections 2024-03-23T09:41:46.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.319+0000 7fe84f155640 1 --2- 172.21.15.47:0/4195435077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8480cf8a0 0x7fe848167b00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.319+0000 7fe84f155640 1 -- 172.21.15.47:0/4195435077 >> 172.21.15.47:0/4195435077 conn(0x7fe84806b160 msgr2=0x7fe84806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:46.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.319+0000 7fe84f155640 1 -- 172.21.15.47:0/4195435077 shutdown_connections 2024-03-23T09:41:46.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.319+0000 7fe84f155640 1 -- 172.21.15.47:0/4195435077 wait complete. 2024-03-23T09:41:46.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.320+0000 7fe84f155640 1 Processor -- start 2024-03-23T09:41:46.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.320+0000 7fe84f155640 1 -- start start 2024-03-23T09:41:46.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe84f155640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8480b8300 0x7fe84815f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe84f155640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8480cf8a0 0x7fe8480c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe84f155640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe84815cb40 0x7fe8480c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe84f155640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8480c5aa0 con 0x7fe8480cf8a0 2024-03-23T09:41:46.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe84f155640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84815ec80 con 0x7fe84815cb40 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe84f155640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84814b260 con 0x7fe8480b8300 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe847fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8480cf8a0 0x7fe8480c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe847fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8480cf8a0 0x7fe8480c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48636/0 (socket says 172.21.15.47:48636) 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe847fff640 1 -- 172.21.15.47:0/1703535421 learned_addr learned my addr 172.21.15.47:0/1703535421 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe84d6f2640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe84815cb40 0x7fe8480c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe84cef1640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8480b8300 0x7fe84815f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe847fff640 1 -- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8480b8300 msgr2=0x7fe84815f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe847fff640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8480b8300 0x7fe84815f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe847fff640 1 -- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe84815cb40 msgr2=0x7fe8480c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe847fff640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe84815cb40 0x7fe8480c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe847fff640 1 -- 172.21.15.47:0/1703535421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe840009a70 con 0x7fe8480cf8a0 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe84d6f2640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe84815cb40 0x7fe8480c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.321+0000 7fe847fff640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8480cf8a0 0x7fe8480c5020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe83400c400 tx=0x7fe83400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:46.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.322+0000 7fe8457fa640 1 -- 172.21.15.47:0/1703535421 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8340052d0 con 0x7fe8480cf8a0 2024-03-23T09:41:46.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.322+0000 7fe8457fa640 1 -- 172.21.15.47:0/1703535421 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe834005430 con 0x7fe8480cf8a0 2024-03-23T09:41:46.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.322+0000 7fe84f155640 1 -- 172.21.15.47:0/1703535421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe848158790 con 0x7fe8480cf8a0 2024-03-23T09:41:46.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.322+0000 7fe8457fa640 1 -- 172.21.15.47:0/1703535421 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834003830 con 0x7fe8480cf8a0 2024-03-23T09:41:46.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.322+0000 7fe84f155640 1 -- 172.21.15.47:0/1703535421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84815eed0 con 0x7fe8480cf8a0 2024-03-23T09:41:46.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.323+0000 7fe84f155640 1 -- 172.21.15.47:0/1703535421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe848072360 con 0x7fe8480cf8a0 2024-03-23T09:41:46.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.323+0000 7fe8457fa640 1 -- 172.21.15.47:0/1703535421 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe834003990 con 0x7fe8480cf8a0 2024-03-23T09:41:46.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.323+0000 7fe8457fa640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe82c06ac00 0x7fe82c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.323+0000 7fe8457fa640 1 -- 172.21.15.47:0/1703535421 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe834088840 con 0x7fe8480cf8a0 2024-03-23T09:41:46.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.324+0000 7fe84cef1640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe82c06ac00 0x7fe82c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.324+0000 7fe84cef1640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe82c06ac00 0x7fe82c06d0c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0011e0 tx=0x7fe83c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:46.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.328+0000 7fe8457fa640 1 -- 172.21.15.47:0/1703535421 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe834053ab0 con 0x7fe8480cf8a0 2024-03-23T09:41:46.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.526+0000 7fe84f155640 1 -- 172.21.15.47:0/1703535421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe84815f0f0 con 0x7fe8480cf8a0 2024-03-23T09:41:46.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.526+0000 7fe8457fa640 1 -- 172.21.15.47:0/1703535421 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe834017080 con 0x7fe8480cf8a0 2024-03-23T09:41:46.526 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:41:46.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.528+0000 7fe84f155640 1 -- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe82c06ac00 msgr2=0x7fe82c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.528+0000 7fe84f155640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe82c06ac00 0x7fe82c06d0c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0011e0 tx=0x7fe83c009040 comp rx=0 tx=0).stop 2024-03-23T09:41:46.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.528+0000 7fe84f155640 1 -- 172.21.15.47:0/1703535421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8480cf8a0 msgr2=0x7fe8480c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.528+0000 7fe84f155640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8480cf8a0 0x7fe8480c5020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe83400c400 tx=0x7fe83400c8c0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.528+0000 7fe84f155640 1 -- 172.21.15.47:0/1703535421 shutdown_connections 2024-03-23T09:41:46.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.528+0000 7fe84f155640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe82c06ac00 0x7fe82c06d0c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.528+0000 7fe84f155640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe84815cb40 0x7fe8480c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.528+0000 7fe84f155640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8480cf8a0 0x7fe8480c5020 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.528+0000 7fe84f155640 1 --2- 172.21.15.47:0/1703535421 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8480b8300 0x7fe84815f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.528+0000 7fe84f155640 1 -- 172.21.15.47:0/1703535421 >> 172.21.15.47:0/1703535421 conn(0x7fe84806b160 msgr2=0x7fe8480cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:46.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.528+0000 7fe84f155640 1 -- 172.21.15.47:0/1703535421 shutdown_connections 2024-03-23T09:41:46.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.528+0000 7fe84f155640 1 -- 172.21.15.47:0/1703535421 wait complete. 2024-03-23T09:41:46.538 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-03-23T09:41:46.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:46.539 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-03-23T09:41:46.571 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:41:46.571 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:41:46.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:41:46.572 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-23T09:41:46.628 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-23T09:41:46.629 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:41:46.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.730+0000 7fdce440b640 1 Processor -- start 2024-03-23T09:41:46.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.730+0000 7fdce440b640 1 -- start start 2024-03-23T09:41:46.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.731+0000 7fdce440b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdcdc06a3f0 0x7fdcdc0cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.731+0000 7fdce440b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdcdc15bc70 0x7fdcdc0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.731+0000 7fdce440b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcdc0cd180 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.731+0000 7fdce440b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc15b7e0 con 0x7fdcdc15bc70 2024-03-23T09:41:46.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.731+0000 7fdce440b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc0ccea0 con 0x7fdcdc06a3f0 2024-03-23T09:41:46.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.731+0000 7fdce29a8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdcdc15bc70 0x7fdcdc0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.731+0000 7fdce29a8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdcdc15bc70 0x7fdcdc0cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48652/0 (socket says 172.21.15.47:48652) 2024-03-23T09:41:46.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.731+0000 7fdce29a8640 1 -- 172.21.15.47:0/3011114135 learned_addr learned my addr 172.21.15.47:0/3011114135 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:46.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.732+0000 7fdce19a6640 1 --2- 172.21.15.47:0/3011114135 >> v2:172.21.15.175:3300/0 conn(0x7fdcdc06a3f0 0x7fdcdc0cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.732+0000 7fdce29a8640 1 -- 172.21.15.47:0/3011114135 >> v2:172.21.15.175:3300/0 conn(0x7fdcdc06a3f0 msgr2=0x7fdcdc0cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:46.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.732+0000 7fdce29a8640 1 --2- 172.21.15.47:0/3011114135 >> v2:172.21.15.175:3300/0 conn(0x7fdcdc06a3f0 0x7fdcdc0cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.732+0000 7fdce29a8640 1 -- 172.21.15.47:0/3011114135 >> v1:172.21.15.175:6789/0 conn(0x7fdcdc15a9b0 legacy=0x7fdcdc069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.732+0000 7fdce29a8640 1 -- 172.21.15.47:0/3011114135 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc15b190 con 0x7fdcdc15bc70 2024-03-23T09:41:46.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.732+0000 7fdce19a6640 1 --2- 172.21.15.47:0/3011114135 >> v2:172.21.15.175:3300/0 conn(0x7fdcdc06a3f0 0x7fdcdc0cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:46.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.733+0000 7fdce29a8640 1 --2- 172.21.15.47:0/3011114135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcdc15bc70 0x7fdcdc0cf640 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0711c0 tx=0x7fdcd402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d873b909c9485fc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:46.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.733+0000 7fdce11a5640 1 -- 172.21.15.47:0/3011114135 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd402cb20 con 0x7fdcdc15bc70 2024-03-23T09:41:46.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.733+0000 7fdce11a5640 1 -- 172.21.15.47:0/3011114135 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd402cc80 con 0x7fdcdc15bc70 2024-03-23T09:41:46.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.733+0000 7fdce11a5640 1 -- 172.21.15.47:0/3011114135 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40343f0 con 0x7fdcdc15bc70 2024-03-23T09:41:46.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.733+0000 7fdce440b640 1 -- 172.21.15.47:0/3011114135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcdc15bc70 msgr2=0x7fdcdc0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.733+0000 7fdce440b640 1 --2- 172.21.15.47:0/3011114135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcdc15bc70 0x7fdcdc0cf640 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0711c0 tx=0x7fdcd402bf80 comp rx=0 tx=0).stop 2024-03-23T09:41:46.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.733+0000 7fdce440b640 1 -- 172.21.15.47:0/3011114135 shutdown_connections 2024-03-23T09:41:46.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.733+0000 7fdce440b640 1 --2- 172.21.15.47:0/3011114135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcdc15bc70 0x7fdcdc0cf640 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.733+0000 7fdce440b640 1 --2- 172.21.15.47:0/3011114135 >> v2:172.21.15.175:3300/0 conn(0x7fdcdc06a3f0 0x7fdcdc0cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.733+0000 7fdce440b640 1 -- 172.21.15.47:0/3011114135 >> 172.21.15.47:0/3011114135 conn(0x7fdcdc071400 msgr2=0x7fdcdc071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.733+0000 7fdce440b640 1 -- 172.21.15.47:0/3011114135 shutdown_connections 2024-03-23T09:41:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.733+0000 7fdce440b640 1 -- 172.21.15.47:0/3011114135 wait complete. 2024-03-23T09:41:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.734+0000 7fdce440b640 1 Processor -- start 2024-03-23T09:41:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.735+0000 7fdce440b640 1 -- start start 2024-03-23T09:41:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.735+0000 7fdce440b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcdc06a3f0 0x7fdcdc13ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.735+0000 7fdce440b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcdc15a9b0 0x7fdcdc14ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.735+0000 7fdce440b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcdc15bc70 0x7fdcdc14f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.735+0000 7fdce440b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc150d20 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.735+0000 7fdce440b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc140c80 con 0x7fdcdc15bc70 2024-03-23T09:41:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.735+0000 7fdce440b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc140f60 con 0x7fdcdc06a3f0 2024-03-23T09:41:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.735+0000 7fdce19a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcdc15a9b0 0x7fdcdc14ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.735+0000 7fdce19a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcdc15a9b0 0x7fdcdc14ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48664/0 (socket says 172.21.15.47:48664) 2024-03-23T09:41:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.735+0000 7fdce19a6640 1 -- 172.21.15.47:0/3359135310 learned_addr learned my addr 172.21.15.47:0/3359135310 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.736+0000 7fdce19a6640 1 -- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcdc06a3f0 msgr2=0x7fdcdc13ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.736+0000 7fdce19a6640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcdc06a3f0 0x7fdcdc13ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.736+0000 7fdce19a6640 1 -- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcdc15bc70 msgr2=0x7fdcdc14f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.736+0000 7fdce29a8640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcdc15bc70 0x7fdcdc14f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.736+0000 7fdce19a6640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcdc15bc70 0x7fdcdc14f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.736+0000 7fdce19a6640 1 -- 172.21.15.47:0/3359135310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd4009b30 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.736+0000 7fdce29a8640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcdc15bc70 0x7fdcdc14f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.736+0000 7fdce19a6640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcdc15a9b0 0x7fdcdc14ecd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00e7e0 tx=0x7fdccc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.736+0000 7fdcd2ffd640 1 -- 172.21.15.47:0/3359135310 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc004030 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.736+0000 7fdcd2ffd640 1 -- 172.21.15.47:0/3359135310 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdccc004190 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.736+0000 7fdce440b640 1 -- 172.21.15.47:0/3359135310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc141240 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.736+0000 7fdcd2ffd640 1 -- 172.21.15.47:0/3359135310 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc004500 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.736+0000 7fdce440b640 1 -- 172.21.15.47:0/3359135310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc13fd90 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.737+0000 7fdce440b640 1 -- 172.21.15.47:0/3359135310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc150f90 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.737+0000 7fdcd2ffd640 1 -- 172.21.15.47:0/3359135310 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdccc00c6d0 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.738+0000 7fdcd2ffd640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdcc006ac00 0x7fdcc006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.738+0000 7fdcd2ffd640 1 -- 172.21.15.47:0/3359135310 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdccc088fb0 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.738+0000 7fdce21a7640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdcc006ac00 0x7fdcc006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.739+0000 7fdce21a7640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdcc006ac00 0x7fdcc006d0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8000f60 tx=0x7fdcd80079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.742+0000 7fdcd2ffd640 1 -- 172.21.15.47:0/3359135310 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdccc054220 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.960+0000 7fdce440b640 1 -- 172.21.15.47:0/3359135310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcdc0672f0 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.961+0000 7fdcd2ffd640 1 -- 172.21.15.47:0/3359135310 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4734 (secure 0 0 0) 0x7fdccc057840 con 0x7fdcdc15a9b0 2024-03-23T09:41:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:46.961 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1970049669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1970049669},{"type":"v1","addr":"172.21.15.47:6833","nonce":1970049669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4162785911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4162785911},{"type":"v1","addr":"172.21.15.47:6835","nonce":4162785911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1036174787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1036174787},{"type":"v1","addr":"172.21.15.175:6836","nonce":1036174787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:03.137957+0000","modified":"2024-03-23T09:41:05.147528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1633245139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1633245139},{"type":"v1","addr":"172.21.15.175:6837","nonce":1633245139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T09:41:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2024-03-23T09:41:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.963+0000 7fdce440b640 1 -- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdcc006ac00 msgr2=0x7fdcc006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.963+0000 7fdce440b640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdcc006ac00 0x7fdcc006d0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8000f60 tx=0x7fdcd80079b0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.963+0000 7fdce440b640 1 -- 172.21.15.47:0/3359135310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcdc15a9b0 msgr2=0x7fdcdc14ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.963+0000 7fdce440b640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcdc15a9b0 0x7fdcdc14ecd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00e7e0 tx=0x7fdccc00eca0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.964+0000 7fdce440b640 1 -- 172.21.15.47:0/3359135310 shutdown_connections 2024-03-23T09:41:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.964+0000 7fdce440b640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdcc006ac00 0x7fdcc006d0c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.964+0000 7fdce440b640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcdc15bc70 0x7fdcdc14f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.964+0000 7fdce440b640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcdc15a9b0 0x7fdcdc14ecd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.964+0000 7fdce440b640 1 --2- 172.21.15.47:0/3359135310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcdc06a3f0 0x7fdcdc13ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.964+0000 7fdce440b640 1 -- 172.21.15.47:0/3359135310 >> 172.21.15.47:0/3359135310 conn(0x7fdcdc071400 msgr2=0x7fdcdc15d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.964+0000 7fdce440b640 1 -- 172.21.15.47:0/3359135310 shutdown_connections 2024-03-23T09:41:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:46.964+0000 7fdce440b640 1 -- 172.21.15.47:0/3359135310 wait complete. 2024-03-23T09:41:46.974 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:47.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.075+0000 7f0ea3838640 1 Processor -- start 2024-03-23T09:41:47.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.076+0000 7f0ea3838640 1 -- start start 2024-03-23T09:41:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.076+0000 7f0ea3838640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0e9c15cee0 0x7f0e9c0d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.076+0000 7f0ea3838640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e9c0d23a0 0x7f0e9c0d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.076+0000 7f0ea3838640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9c0d1300 con 0x7f0e9c0b8300 2024-03-23T09:41:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.076+0000 7f0ea3838640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9c0d15e0 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.076+0000 7f0ea3838640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9c0d18c0 con 0x7f0e9c15cee0 2024-03-23T09:41:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.077+0000 7f0ea0dd3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e9c0d23a0 0x7f0e9c0d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.077+0000 7f0ea0dd3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e9c0d23a0 0x7f0e9c0d2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48676/0 (socket says 172.21.15.47:48676) 2024-03-23T09:41:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.077+0000 7f0ea0dd3640 1 -- 172.21.15.47:0/3439210447 learned_addr learned my addr 172.21.15.47:0/3439210447 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.077+0000 7f0ea15d4640 1 --2- 172.21.15.47:0/3439210447 >> v2:172.21.15.175:3300/0 conn(0x7f0e9c15cee0 0x7f0e9c0d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.077+0000 7f0ea0dd3640 1 -- 172.21.15.47:0/3439210447 >> v2:172.21.15.175:3300/0 conn(0x7f0e9c15cee0 msgr2=0x7f0e9c0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.077+0000 7f0ea0dd3640 1 --2- 172.21.15.47:0/3439210447 >> v2:172.21.15.175:3300/0 conn(0x7f0e9c15cee0 0x7f0e9c0d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.077+0000 7f0ea0dd3640 1 -- 172.21.15.47:0/3439210447 >> v1:172.21.15.175:6789/0 conn(0x7f0e9c0b8300 legacy=0x7f0e9c0b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:47.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.077+0000 7f0ea0dd3640 1 -- 172.21.15.47:0/3439210447 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c0d1ba0 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.078+0000 7f0ea0dd3640 1 --2- 172.21.15.47:0/3439210447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c0d23a0 0x7f0e9c0d2870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0d3430 tx=0x7f0e8c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c08ed0b2be4152 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.078+0000 7f0e93fff640 1 -- 172.21.15.47:0/3439210447 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c031870 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.078+0000 7f0e93fff640 1 -- 172.21.15.47:0/3439210447 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c0319d0 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.078+0000 7f0e93fff640 1 -- 172.21.15.47:0/3439210447 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c031d60 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.078+0000 7f0ea3838640 1 -- 172.21.15.47:0/3439210447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c0d23a0 msgr2=0x7f0e9c0d2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.078+0000 7f0ea3838640 1 --2- 172.21.15.47:0/3439210447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c0d23a0 0x7f0e9c0d2870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0d3430 tx=0x7f0e8c02b910 comp rx=0 tx=0).stop 2024-03-23T09:41:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.078+0000 7f0ea3838640 1 -- 172.21.15.47:0/3439210447 shutdown_connections 2024-03-23T09:41:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.078+0000 7f0ea3838640 1 --2- 172.21.15.47:0/3439210447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c0d23a0 0x7f0e9c0d2870 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.078+0000 7f0ea3838640 1 --2- 172.21.15.47:0/3439210447 >> v2:172.21.15.175:3300/0 conn(0x7f0e9c15cee0 0x7f0e9c0d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.079+0000 7f0ea3838640 1 -- 172.21.15.47:0/3439210447 >> 172.21.15.47:0/3439210447 conn(0x7f0e9c065df0 msgr2=0x7f0e9c066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.079+0000 7f0ea3838640 1 -- 172.21.15.47:0/3439210447 shutdown_connections 2024-03-23T09:41:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.079+0000 7f0ea3838640 1 -- 172.21.15.47:0/3439210447 wait complete. 2024-03-23T09:41:47.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.080+0000 7f0ea3838640 1 Processor -- start 2024-03-23T09:41:47.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.080+0000 7f0ea3838640 1 -- start start 2024-03-23T09:41:47.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.080+0000 7f0ea3838640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9c0b8300 0x7f0e9c1f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.080+0000 7f0ea3838640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c0d23a0 0x7f0e9c15fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.080+0000 7f0ea3838640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e9c15cee0 0x7f0e9c156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.080+0000 7f0ea3838640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c14a7b0 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.080+0000 7f0ea3838640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c154eb0 con 0x7f0e9c0b8300 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.080+0000 7f0ea3838640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c155470 con 0x7f0e9c15cee0 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0ea0dd3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c0d23a0 0x7f0e9c15fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0ea0dd3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c0d23a0 0x7f0e9c15fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48690/0 (socket says 172.21.15.47:48690) 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0ea0dd3640 1 -- 172.21.15.47:0/3511443461 learned_addr learned my addr 172.21.15.47:0/3511443461 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0ea15d4640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9c0b8300 0x7f0e9c1f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0ea0dd3640 1 -- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e9c15cee0 msgr2=0x7f0e9c156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0ea0dd3640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e9c15cee0 0x7f0e9c156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0ea0dd3640 1 -- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9c0b8300 msgr2=0x7f0e9c1f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0ea0dd3640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9c0b8300 0x7f0e9c1f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0ea0dd3640 1 -- 172.21.15.47:0/3511443461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c0094d0 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0ea15d4640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9c0b8300 0x7f0e9c1f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0ea0dd3640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c0d23a0 0x7f0e9c15fa90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c000c00 tx=0x7f0e8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0e91ffb640 1 -- 172.21.15.47:0/3511443461 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c03ad30 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0e91ffb640 1 -- 172.21.15.47:0/3511443461 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c0443f0 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0ea3838640 1 -- 172.21.15.47:0/3511443461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9c0cb9e0 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0e91ffb640 1 -- 172.21.15.47:0/3511443461 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c0447a0 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.081+0000 7f0ea3838640 1 -- 172.21.15.47:0/3511443461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9c155100 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.082+0000 7f0ea3838640 1 -- 172.21.15.47:0/3511443461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9c072360 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.083+0000 7f0e91ffb640 1 -- 172.21.15.47:0/3511443461 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0e8c044900 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.083+0000 7f0e91ffb640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0e8006ac00 0x7f0e8006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.083+0000 7f0e91ffb640 1 -- 172.21.15.47:0/3511443461 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e8c02d040 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.084+0000 7f0ea15d4640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0e8006ac00 0x7f0e8006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.084+0000 7f0ea15d4640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0e8006ac00 0x7f0e8006d0c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e98006fd0 tx=0x7f0e98008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.088+0000 7f0e91ffb640 1 -- 172.21.15.47:0/3511443461 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0e8c076d40 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.271+0000 7f0ea3838640 1 -- 172.21.15.47:0/3511443461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e9c151ea0 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.272+0000 7f0e91ffb640 1 -- 172.21.15.47:0/3511443461 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f0e8c035070 con 0x7f0e9c0d23a0 2024-03-23T09:41:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:47.272 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:47.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.274+0000 7f0ea3838640 1 -- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0e8006ac00 msgr2=0x7f0e8006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.274+0000 7f0ea3838640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0e8006ac00 0x7f0e8006d0c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e98006fd0 tx=0x7f0e98008040 comp rx=0 tx=0).stop 2024-03-23T09:41:47.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.274+0000 7f0ea3838640 1 -- 172.21.15.47:0/3511443461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c0d23a0 msgr2=0x7f0e9c15fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.274+0000 7f0ea3838640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c0d23a0 0x7f0e9c15fa90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c000c00 tx=0x7f0e8c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.274+0000 7f0ea3838640 1 -- 172.21.15.47:0/3511443461 shutdown_connections 2024-03-23T09:41:47.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.274+0000 7f0ea3838640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0e8006ac00 0x7f0e8006d0c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.274+0000 7f0ea3838640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e9c15cee0 0x7f0e9c156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.274+0000 7f0ea3838640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c0d23a0 0x7f0e9c15fa90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.274+0000 7f0ea3838640 1 --2- 172.21.15.47:0/3511443461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9c0b8300 0x7f0e9c1f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.275+0000 7f0ea3838640 1 -- 172.21.15.47:0/3511443461 >> 172.21.15.47:0/3511443461 conn(0x7f0e9c065df0 msgr2=0x7f0e9c0661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:47.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.275+0000 7f0ea3838640 1 -- 172.21.15.47:0/3511443461 shutdown_connections 2024-03-23T09:41:47.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.275+0000 7f0ea3838640 1 -- 172.21.15.47:0/3511443461 wait complete. 2024-03-23T09:41:47.285 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:00.911579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:02.107759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:47.285 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.389+0000 7f16278af640 1 Processor -- start 2024-03-23T09:41:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.389+0000 7f16278af640 1 -- start start 2024-03-23T09:41:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.390+0000 7f16278af640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162015c990 0x7f1620160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.390+0000 7f16278af640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f16200cde10 0x7f16200d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.390+0000 7f16278af640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1620161630 con 0x7f16200b8350 2024-03-23T09:41:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.390+0000 7f16278af640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16200d1030 con 0x7f16200cde10 2024-03-23T09:41:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.390+0000 7f16278af640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16200cfe10 con 0x7f162015c990 2024-03-23T09:41:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.390+0000 7f1625e4c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f16200cde10 0x7f16200d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.390+0000 7f1624e4a640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f16200b8350 0x7f16200cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47970/0 (socket says 172.21.15.47:47970) 2024-03-23T09:41:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.390+0000 7f1624e4a640 1 -- 172.21.15.47:0/1025484333 learned_addr learned my addr 172.21.15.47:0/1025484333 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.390+0000 7f162564b640 1 --2- 172.21.15.47:0/1025484333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162015c990 0x7f1620160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.391+0000 7f1625e4c640 1 -- 172.21.15.47:0/1025484333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162015c990 msgr2=0x7f1620160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.391+0000 7f1625e4c640 1 --2- 172.21.15.47:0/1025484333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162015c990 0x7f1620160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.391+0000 7f1625e4c640 1 -- 172.21.15.47:0/1025484333 >> v1:172.21.15.47:6789/0 conn(0x7f16200b8350 legacy=0x7f16200cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.391+0000 7f1625e4c640 1 -- 172.21.15.47:0/1025484333 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16200b5b60 con 0x7f16200cde10 2024-03-23T09:41:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.391+0000 7f161ffff640 1 -- 172.21.15.47:0/1025484333 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209757013 0 0) 0x7f1620161630 con 0x7f16200b8350 2024-03-23T09:41:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.391+0000 7f1625e4c640 1 --2- 172.21.15.47:0/1025484333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16200cde10 0x7f16200d14c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f16100097a0 tx=0x7f161002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efeae2b28bfb10e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.391+0000 7f161ffff640 1 -- 172.21.15.47:0/1025484333 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161002e070 con 0x7f16200cde10 2024-03-23T09:41:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.392+0000 7f161ffff640 1 -- 172.21.15.47:0/1025484333 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16100315a0 con 0x7f16200cde10 2024-03-23T09:41:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.392+0000 7f161ffff640 1 -- 172.21.15.47:0/1025484333 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610031930 con 0x7f16200cde10 2024-03-23T09:41:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.392+0000 7f16278af640 1 -- 172.21.15.47:0/1025484333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16200cde10 msgr2=0x7f16200d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.392+0000 7f16278af640 1 --2- 172.21.15.47:0/1025484333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16200cde10 0x7f16200d14c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f16100097a0 tx=0x7f161002b840 comp rx=0 tx=0).stop 2024-03-23T09:41:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.392+0000 7f16278af640 1 -- 172.21.15.47:0/1025484333 shutdown_connections 2024-03-23T09:41:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.392+0000 7f16278af640 1 --2- 172.21.15.47:0/1025484333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16200cde10 0x7f16200d14c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.392+0000 7f16278af640 1 --2- 172.21.15.47:0/1025484333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162015c990 0x7f1620160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.392+0000 7f16278af640 1 -- 172.21.15.47:0/1025484333 >> 172.21.15.47:0/1025484333 conn(0x7f16200659d0 msgr2=0x7f1620065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.392+0000 7f16278af640 1 -- 172.21.15.47:0/1025484333 shutdown_connections 2024-03-23T09:41:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.392+0000 7f16278af640 1 -- 172.21.15.47:0/1025484333 wait complete. 2024-03-23T09:41:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.393+0000 7f16278af640 1 Processor -- start 2024-03-23T09:41:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.393+0000 7f16278af640 1 -- start start 2024-03-23T09:41:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.394+0000 7f16278af640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16200b8350 0x7f1620160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.394+0000 7f16278af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16200cde10 0x7f16200c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.394+0000 7f16278af640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f162015c990 0x7f1620153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.394+0000 7f16278af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1620158540 con 0x7f16200cde10 2024-03-23T09:41:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.394+0000 7f16278af640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16200c72e0 con 0x7f162015c990 2024-03-23T09:41:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.394+0000 7f16278af640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16200c7590 con 0x7f16200b8350 2024-03-23T09:41:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.394+0000 7f1624e4a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16200cde10 0x7f16200c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.394+0000 7f1624e4a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16200cde10 0x7f16200c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48716/0 (socket says 172.21.15.47:48716) 2024-03-23T09:41:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.394+0000 7f1624e4a640 1 -- 172.21.15.47:0/3810062542 learned_addr learned my addr 172.21.15.47:0/3810062542 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.394+0000 7f162564b640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16200b8350 0x7f1620160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.394+0000 7f1625e4c640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f162015c990 0x7f1620153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.394+0000 7f1624e4a640 1 -- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16200b8350 msgr2=0x7f1620160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.394+0000 7f1624e4a640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16200b8350 0x7f1620160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.395+0000 7f1624e4a640 1 -- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f162015c990 msgr2=0x7f1620153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.395+0000 7f1624e4a640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f162015c990 0x7f1620153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.395+0000 7f1624e4a640 1 -- 172.21.15.47:0/3810062542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1610009400 con 0x7f16200cde10 2024-03-23T09:41:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.395+0000 7f1625e4c640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f162015c990 0x7f1620153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.395+0000 7f1624e4a640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16200cde10 0x7f16200c6da0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f161400c430 tx=0x7f161400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.395+0000 7f161dffb640 1 -- 172.21.15.47:0/3810062542 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16140074e0 con 0x7f16200cde10 2024-03-23T09:41:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.395+0000 7f161dffb640 1 -- 172.21.15.47:0/3810062542 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1614007660 con 0x7f16200cde10 2024-03-23T09:41:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.395+0000 7f16278af640 1 -- 172.21.15.47:0/3810062542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16200c7870 con 0x7f16200cde10 2024-03-23T09:41:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.395+0000 7f161dffb640 1 -- 172.21.15.47:0/3810062542 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614004a20 con 0x7f16200cde10 2024-03-23T09:41:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.395+0000 7f16278af640 1 -- 172.21.15.47:0/3810062542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16200c7e30 con 0x7f16200cde10 2024-03-23T09:41:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.396+0000 7f16278af640 1 -- 172.21.15.47:0/3810062542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1620072360 con 0x7f16200cde10 2024-03-23T09:41:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.396+0000 7f161dffb640 1 -- 172.21.15.47:0/3810062542 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1614004b80 con 0x7f16200cde10 2024-03-23T09:41:47.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.397+0000 7f161dffb640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f160c06ac00 0x7f160c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.397+0000 7f161dffb640 1 -- 172.21.15.47:0/3810062542 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1614089100 con 0x7f16200cde10 2024-03-23T09:41:47.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.397+0000 7f162564b640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f160c06ac00 0x7f160c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.397+0000 7f162564b640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f160c06ac00 0x7f160c06d0c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1618009a60 tx=0x7f1618003990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:47.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.401+0000 7f161dffb640 1 -- 172.21.15.47:0/3810062542 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1614054370 con 0x7f16200cde10 2024-03-23T09:41:47.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.581+0000 7f16278af640 1 -- 172.21.15.47:0/3810062542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1620066be0 con 0x7f16200cde10 2024-03-23T09:41:47.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.582+0000 7f161dffb640 1 -- 172.21.15.47:0/3810062542 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1614057990 con 0x7f16200cde10 2024-03-23T09:41:47.581 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.584+0000 7f16278af640 1 -- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f160c06ac00 msgr2=0x7f160c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.584+0000 7f16278af640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f160c06ac00 0x7f160c06d0c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1618009a60 tx=0x7f1618003990 comp rx=0 tx=0).stop 2024-03-23T09:41:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.584+0000 7f16278af640 1 -- 172.21.15.47:0/3810062542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16200cde10 msgr2=0x7f16200c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.584+0000 7f16278af640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16200cde10 0x7f16200c6da0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f161400c430 tx=0x7f161400c8f0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.584+0000 7f16278af640 1 -- 172.21.15.47:0/3810062542 shutdown_connections 2024-03-23T09:41:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.584+0000 7f16278af640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f160c06ac00 0x7f160c06d0c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.584+0000 7f16278af640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f162015c990 0x7f1620153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.584+0000 7f16278af640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16200cde10 0x7f16200c6da0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.584+0000 7f16278af640 1 --2- 172.21.15.47:0/3810062542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16200b8350 0x7f1620160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.584+0000 7f16278af640 1 -- 172.21.15.47:0/3810062542 >> 172.21.15.47:0/3810062542 conn(0x7f16200659d0 msgr2=0x7f16200b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.584+0000 7f16278af640 1 -- 172.21.15.47:0/3810062542 shutdown_connections 2024-03-23T09:41:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.584+0000 7f16278af640 1 -- 172.21.15.47:0/3810062542 wait complete. 2024-03-23T09:41:47.594 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:41:47.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.698+0000 7f0b5b28f640 1 Processor -- start 2024-03-23T09:41:47.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.698+0000 7f0b5b28f640 1 -- start start 2024-03-23T09:41:47.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5b28f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0b540d23a0 0x7f0b540d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5b28f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b540b8300 0x7f0b540d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5b28f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b540d10a0 con 0x7f0b5415cee0 2024-03-23T09:41:47.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5b28f640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b540d1300 con 0x7f0b540d23a0 2024-03-23T09:41:47.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5b28f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540d15e0 con 0x7f0b540b8300 2024-03-23T09:41:47.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5882a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0b540d23a0 0x7f0b540d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5982c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b540b8300 0x7f0b540d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5902b640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f0b5415cee0 0x7f0b540d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:54490/0 (socket says 172.21.15.47:54490) 2024-03-23T09:41:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5902b640 1 -- 172.21.15.47:0/2665025782 learned_addr learned my addr 172.21.15.47:0/2665025782 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5882a640 1 -- 172.21.15.47:0/2665025782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b540b8300 msgr2=0x7f0b540d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5882a640 1 --2- 172.21.15.47:0/2665025782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b540b8300 0x7f0b540d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5882a640 1 -- 172.21.15.47:0/2665025782 >> v1:172.21.15.175:6789/0 conn(0x7f0b5415cee0 legacy=0x7f0b540d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b4bfff640 1 -- 172.21.15.47:0/2665025782 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472505735 0 0) 0x7f0b540d10a0 con 0x7f0b5415cee0 2024-03-23T09:41:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5882a640 1 -- 172.21.15.47:0/2665025782 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b540d18c0 con 0x7f0b540d23a0 2024-03-23T09:41:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.699+0000 7f0b5982c640 1 --2- 172.21.15.47:0/2665025782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b540b8300 0x7f0b540d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.700+0000 7f0b5882a640 1 --2- 172.21.15.47:0/2665025782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540d23a0 0x7f0b540d2870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0b440097a0 tx=0x7f0b4402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=395013468b0363f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.700+0000 7f0b4bfff640 1 -- 172.21.15.47:0/2665025782 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44031830 con 0x7f0b540d23a0 2024-03-23T09:41:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.700+0000 7f0b4bfff640 1 -- 172.21.15.47:0/2665025782 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b44031990 con 0x7f0b540d23a0 2024-03-23T09:41:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.700+0000 7f0b4bfff640 1 -- 172.21.15.47:0/2665025782 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44031d20 con 0x7f0b540d23a0 2024-03-23T09:41:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.701+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2665025782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540d23a0 msgr2=0x7f0b540d2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.701+0000 7f0b5b28f640 1 --2- 172.21.15.47:0/2665025782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540d23a0 0x7f0b540d2870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0b440097a0 tx=0x7f0b4402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.701+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2665025782 shutdown_connections 2024-03-23T09:41:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.701+0000 7f0b5b28f640 1 --2- 172.21.15.47:0/2665025782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b540b8300 0x7f0b540d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.701+0000 7f0b5b28f640 1 --2- 172.21.15.47:0/2665025782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540d23a0 0x7f0b540d2870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.701+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2665025782 >> 172.21.15.47:0/2665025782 conn(0x7f0b54065df0 msgr2=0x7f0b54066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.701+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2665025782 shutdown_connections 2024-03-23T09:41:47.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.701+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2665025782 wait complete. 2024-03-23T09:41:47.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.702+0000 7f0b5b28f640 1 Processor -- start 2024-03-23T09:41:47.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.702+0000 7f0b5b28f640 1 -- start start 2024-03-23T09:41:47.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5b28f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540b8300 0x7f0b541480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5b28f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b540d23a0 0x7f0b540ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5b28f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b5415cee0 0x7f0b540ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5b28f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540cae40 con 0x7f0b540d23a0 2024-03-23T09:41:47.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5b28f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540cc210 con 0x7f0b540b8300 2024-03-23T09:41:47.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5b28f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540c9230 con 0x7f0b5415cee0 2024-03-23T09:41:47.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5882a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b540d23a0 0x7f0b540ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5882a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b540d23a0 0x7f0b540ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48732/0 (socket says 172.21.15.47:48732) 2024-03-23T09:41:47.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5882a640 1 -- 172.21.15.47:0/2908813513 learned_addr learned my addr 172.21.15.47:0/2908813513 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:47.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5902b640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540b8300 0x7f0b541480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5982c640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b5415cee0 0x7f0b540ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5882a640 1 -- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b5415cee0 msgr2=0x7f0b540ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5882a640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b5415cee0 0x7f0b540ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5882a640 1 -- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540b8300 msgr2=0x7f0b541480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5882a640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540b8300 0x7f0b541480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.703+0000 7f0b5882a640 1 -- 172.21.15.47:0/2908813513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c009a70 con 0x7f0b540d23a0 2024-03-23T09:41:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.704+0000 7f0b5882a640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b540d23a0 0x7f0b540ca3c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0b440098c0 tx=0x7f0b440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.704+0000 7f0b49ffb640 1 -- 172.21.15.47:0/2908813513 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4403a630 con 0x7f0b540d23a0 2024-03-23T09:41:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.704+0000 7f0b49ffb640 1 -- 172.21.15.47:0/2908813513 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4403a7b0 con 0x7f0b540d23a0 2024-03-23T09:41:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.704+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2908813513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b44009400 con 0x7f0b540d23a0 2024-03-23T09:41:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.704+0000 7f0b49ffb640 1 -- 172.21.15.47:0/2908813513 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4403ab60 con 0x7f0b540d23a0 2024-03-23T09:41:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.704+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2908813513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b540c8f50 con 0x7f0b540d23a0 2024-03-23T09:41:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.705+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2908813513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b54072360 con 0x7f0b540d23a0 2024-03-23T09:41:47.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.706+0000 7f0b49ffb640 1 -- 172.21.15.47:0/2908813513 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0b4403acc0 con 0x7f0b540d23a0 2024-03-23T09:41:47.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.706+0000 7f0b49ffb640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b3806ac00 0x7f0b3806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:47.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.706+0000 7f0b49ffb640 1 -- 172.21.15.47:0/2908813513 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b440aae60 con 0x7f0b540d23a0 2024-03-23T09:41:47.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.706+0000 7f0b5902b640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b3806ac00 0x7f0b3806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:47.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.707+0000 7f0b5902b640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b3806ac00 0x7f0b3806d0c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0b50007fd0 tx=0x7f0b50007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:47.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.710+0000 7f0b49ffb640 1 -- 172.21.15.47:0/2908813513 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0b44076210 con 0x7f0b540d23a0 2024-03-23T09:41:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.886+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2908813513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0b54066be0 con 0x7f0b540d23a0 2024-03-23T09:41:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.887+0000 7f0b49ffb640 1 -- 172.21.15.47:0/2908813513 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0b44079830 con 0x7f0b540d23a0 2024-03-23T09:41:47.886 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:41:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.888+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b3806ac00 msgr2=0x7f0b3806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.888+0000 7f0b5b28f640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b3806ac00 0x7f0b3806d0c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0b50007fd0 tx=0x7f0b50007de0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.888+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2908813513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b540d23a0 msgr2=0x7f0b540ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.888+0000 7f0b5b28f640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b540d23a0 0x7f0b540ca3c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0b440098c0 tx=0x7f0b440046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.889+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2908813513 shutdown_connections 2024-03-23T09:41:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.889+0000 7f0b5b28f640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b3806ac00 0x7f0b3806d0c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.889+0000 7f0b5b28f640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b5415cee0 0x7f0b540ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.889+0000 7f0b5b28f640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b540d23a0 0x7f0b540ca3c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.889+0000 7f0b5b28f640 1 --2- 172.21.15.47:0/2908813513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540b8300 0x7f0b541480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.889+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2908813513 >> 172.21.15.47:0/2908813513 conn(0x7f0b54065df0 msgr2=0x7f0b540b6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.889+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2908813513 shutdown_connections 2024-03-23T09:41:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:47.889+0000 7f0b5b28f640 1 -- 172.21.15.47:0/2908813513 wait complete. 2024-03-23T09:41:47.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:41:48.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.003+0000 7fb2ff0d5640 1 Processor -- start 2024-03-23T09:41:48.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.003+0000 7fb2ff0d5640 1 -- start start 2024-03-23T09:41:48.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.004+0000 7fb2ff0d5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f815c990 0x7fb2f8160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.004+0000 7fb2ff0d5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb2f80cde10 0x7fb2f80d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.004+0000 7fb2ff0d5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f8161630 con 0x7fb2f80b8350 2024-03-23T09:41:48.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.004+0000 7fb2ff0d5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f80d1030 con 0x7fb2f80cde10 2024-03-23T09:41:48.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.004+0000 7fb2ff0d5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80cfe10 con 0x7fb2f815c990 2024-03-23T09:41:48.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.004+0000 7fb2fd672640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb2f80cde10 0x7fb2f80d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.004+0000 7fb2fd672640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb2f80cde10 0x7fb2f80d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48734/0 (socket says 172.21.15.47:48734) 2024-03-23T09:41:48.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.004+0000 7fb2fd672640 1 -- 172.21.15.47:0/4059497265 learned_addr learned my addr 172.21.15.47:0/4059497265 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:48.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.004+0000 7fb2fce71640 1 --2- 172.21.15.47:0/4059497265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f815c990 0x7fb2f8160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.004+0000 7fb2fd672640 1 -- 172.21.15.47:0/4059497265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f815c990 msgr2=0x7fb2f8160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:48.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.004+0000 7fb2fd672640 1 --2- 172.21.15.47:0/4059497265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f815c990 0x7fb2f8160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.004+0000 7fb2fd672640 1 -- 172.21.15.47:0/4059497265 >> v1:172.21.15.175:6789/0 conn(0x7fb2f80b8350 legacy=0x7fb2f80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:48.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.004+0000 7fb2fd672640 1 -- 172.21.15.47:0/4059497265 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f80b5b60 con 0x7fb2f80cde10 2024-03-23T09:41:48.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.005+0000 7fb2fd672640 1 --2- 172.21.15.47:0/4059497265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f80cde10 0x7fb2f80d14c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009870 tx=0x7fb2e802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78dc3ab921b7aa57 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:48.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.005+0000 7fb2f77fe640 1 -- 172.21.15.47:0/4059497265 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80317c0 con 0x7fb2f80cde10 2024-03-23T09:41:48.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.005+0000 7fb2f77fe640 1 -- 172.21.15.47:0/4059497265 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e8031920 con 0x7fb2f80cde10 2024-03-23T09:41:48.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.005+0000 7fb2f77fe640 1 -- 172.21.15.47:0/4059497265 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8031cb0 con 0x7fb2f80cde10 2024-03-23T09:41:48.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.006+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/4059497265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f80cde10 msgr2=0x7fb2f80d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.006+0000 7fb2ff0d5640 1 --2- 172.21.15.47:0/4059497265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f80cde10 0x7fb2f80d14c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009870 tx=0x7fb2e802b860 comp rx=0 tx=0).stop 2024-03-23T09:41:48.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.006+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/4059497265 shutdown_connections 2024-03-23T09:41:48.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.006+0000 7fb2ff0d5640 1 --2- 172.21.15.47:0/4059497265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f80cde10 0x7fb2f80d14c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.006+0000 7fb2ff0d5640 1 --2- 172.21.15.47:0/4059497265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f815c990 0x7fb2f8160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.006+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/4059497265 >> 172.21.15.47:0/4059497265 conn(0x7fb2f80659d0 msgr2=0x7fb2f8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:48.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.006+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/4059497265 shutdown_connections 2024-03-23T09:41:48.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.006+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/4059497265 wait complete. 2024-03-23T09:41:48.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.007+0000 7fb2ff0d5640 1 Processor -- start 2024-03-23T09:41:48.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.007+0000 7fb2ff0d5640 1 -- start start 2024-03-23T09:41:48.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2ff0d5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f80b8350 0x7fb2f814d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2ff0d5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f80cde10 0x7fb2f8151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2ff0d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f815c990 0x7fb2f8151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2ff0d5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f81546b0 con 0x7fb2f80b8350 2024-03-23T09:41:48.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2ff0d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f815d0c0 con 0x7fb2f815c990 2024-03-23T09:41:48.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2ff0d5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f815df20 con 0x7fb2f80cde10 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2fce71640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f80b8350 0x7fb2f814d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2fce71640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f80b8350 0x7fb2f814d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48746/0 (socket says 172.21.15.47:48746) 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2fce71640 1 -- 172.21.15.47:0/1534478508 learned_addr learned my addr 172.21.15.47:0/1534478508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2fd672640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f815c990 0x7fb2f8151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2f7fff640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f80cde10 0x7fb2f8151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2fce71640 1 -- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f80cde10 msgr2=0x7fb2f8151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2fce71640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f80cde10 0x7fb2f8151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2fce71640 1 -- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f815c990 msgr2=0x7fb2f8151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2fce71640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f815c990 0x7fb2f8151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2fce71640 1 -- 172.21.15.47:0/1534478508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e80094d0 con 0x7fb2f80b8350 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2fd672640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f815c990 0x7fb2f8151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.008+0000 7fb2fce71640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f80b8350 0x7fb2f814d790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000ce40 tx=0x7fb2f000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.009+0000 7fb2f57fa640 1 -- 172.21.15.47:0/1534478508 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0003b90 con 0x7fb2f80b8350 2024-03-23T09:41:48.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.009+0000 7fb2f57fa640 1 -- 172.21.15.47:0/1534478508 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f0003cf0 con 0x7fb2f80b8350 2024-03-23T09:41:48.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.009+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/1534478508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f815d310 con 0x7fb2f80b8350 2024-03-23T09:41:48.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.009+0000 7fb2f57fa640 1 -- 172.21.15.47:0/1534478508 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0004240 con 0x7fb2f80b8350 2024-03-23T09:41:48.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.009+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/1534478508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f815d8d0 con 0x7fb2f80b8350 2024-03-23T09:41:48.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.010+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/1534478508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f8072360 con 0x7fb2f80b8350 2024-03-23T09:41:48.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.010+0000 7fb2f57fa640 1 -- 172.21.15.47:0/1534478508 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb2f00043a0 con 0x7fb2f80b8350 2024-03-23T09:41:48.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.010+0000 7fb2f57fa640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb2e406ac00 0x7fb2e406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.011+0000 7fb2f57fa640 1 -- 172.21.15.47:0/1534478508 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2f0012070 con 0x7fb2f80b8350 2024-03-23T09:41:48.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.011+0000 7fb2f7fff640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb2e406ac00 0x7fb2e406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.011+0000 7fb2f7fff640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb2e406ac00 0x7fb2e406d0c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80b3070 tx=0x7fb2ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:48.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.015+0000 7fb2f57fa640 1 -- 172.21.15.47:0/1534478508 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb2f0054fd0 con 0x7fb2f80b8350 2024-03-23T09:41:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.192+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/1534478508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2f8066be0 con 0x7fb2f80b8350 2024-03-23T09:41:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.193+0000 7fb2f57fa640 1 -- 172.21.15.47:0/1534478508 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2f0003e60 con 0x7fb2f80b8350 2024-03-23T09:41:48.192 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:41:48.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.194+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb2e406ac00 msgr2=0x7fb2e406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.194+0000 7fb2ff0d5640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb2e406ac00 0x7fb2e406d0c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80b3070 tx=0x7fb2ec008040 comp rx=0 tx=0).stop 2024-03-23T09:41:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.194+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/1534478508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f80b8350 msgr2=0x7fb2f814d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.194+0000 7fb2ff0d5640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f80b8350 0x7fb2f814d790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000ce40 tx=0x7fb2f000a660 comp rx=0 tx=0).stop 2024-03-23T09:41:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.195+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/1534478508 shutdown_connections 2024-03-23T09:41:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.195+0000 7fb2ff0d5640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb2e406ac00 0x7fb2e406d0c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.195+0000 7fb2ff0d5640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f815c990 0x7fb2f8151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.195+0000 7fb2ff0d5640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f80cde10 0x7fb2f8151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.195+0000 7fb2ff0d5640 1 --2- 172.21.15.47:0/1534478508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f80b8350 0x7fb2f814d790 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.195+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/1534478508 >> 172.21.15.47:0/1534478508 conn(0x7fb2f80659d0 msgr2=0x7fb2f80d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.195+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/1534478508 shutdown_connections 2024-03-23T09:41:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.195+0000 7fb2ff0d5640 1 -- 172.21.15.47:0/1534478508 wait complete. 2024-03-23T09:41:48.205 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:41:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.306+0000 7fe7417c3640 1 Processor -- start 2024-03-23T09:41:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.306+0000 7fe7417c3640 1 -- start start 2024-03-23T09:41:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.307+0000 7fe7417c3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe73c15deb0 0x7fe73c0cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.307+0000 7fe7417c3640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe73c0cdf50 0x7fe73c0ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.307+0000 7fe7417c3640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0ce3c0 con 0x7fe73c06df60 2024-03-23T09:41:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.307+0000 7fe7417c3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73c06b470 con 0x7fe73c15deb0 2024-03-23T09:41:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.307+0000 7fe7417c3640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73c06b190 con 0x7fe73c0cdf50 2024-03-23T09:41:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.307+0000 7fe73a7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe73c15deb0 0x7fe73c0cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.307+0000 7fe73affd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe73c06df60 0x7fe73c15a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47984/0 (socket says 172.21.15.47:47984) 2024-03-23T09:41:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.307+0000 7fe73affd640 1 -- 172.21.15.47:0/1140040034 learned_addr learned my addr 172.21.15.47:0/1140040034 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.308+0000 7fe73b7fe640 1 --2- 172.21.15.47:0/1140040034 >> v2:172.21.15.175:3300/0 conn(0x7fe73c0cdf50 0x7fe73c0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.308+0000 7fe73a7fc640 1 -- 172.21.15.47:0/1140040034 >> v2:172.21.15.175:3300/0 conn(0x7fe73c0cdf50 msgr2=0x7fe73c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.308+0000 7fe73a7fc640 1 --2- 172.21.15.47:0/1140040034 >> v2:172.21.15.175:3300/0 conn(0x7fe73c0cdf50 0x7fe73c0ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.308+0000 7fe73a7fc640 1 -- 172.21.15.47:0/1140040034 >> v1:172.21.15.47:6789/0 conn(0x7fe73c06df60 legacy=0x7fe73c15a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.308+0000 7fe73a7fc640 1 -- 172.21.15.47:0/1140040034 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c15d3d0 con 0x7fe73c15deb0 2024-03-23T09:41:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.308+0000 7fe739ffb640 1 -- 172.21.15.47:0/1140040034 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374156254 0 0) 0x7fe734001110 con 0x7fe73c06df60 2024-03-23T09:41:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.308+0000 7fe73b7fe640 1 --2- 172.21.15.47:0/1140040034 >> v2:172.21.15.175:3300/0 conn(0x7fe73c0cdf50 0x7fe73c0ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.308+0000 7fe73a7fc640 1 --2- 172.21.15.47:0/1140040034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe73c15deb0 0x7fe73c0cda10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe73c159d40 tx=0x7fe72402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6a4e7bf0e5edc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.309+0000 7fe739ffb640 1 -- 172.21.15.47:0/1140040034 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72402e070 con 0x7fe73c15deb0 2024-03-23T09:41:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.309+0000 7fe739ffb640 1 -- 172.21.15.47:0/1140040034 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7240315a0 con 0x7fe73c15deb0 2024-03-23T09:41:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.309+0000 7fe739ffb640 1 -- 172.21.15.47:0/1140040034 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724031930 con 0x7fe73c15deb0 2024-03-23T09:41:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.309+0000 7fe7417c3640 1 -- 172.21.15.47:0/1140040034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe73c15deb0 msgr2=0x7fe73c0cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.309+0000 7fe7417c3640 1 --2- 172.21.15.47:0/1140040034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe73c15deb0 0x7fe73c0cda10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe73c159d40 tx=0x7fe72402b840 comp rx=0 tx=0).stop 2024-03-23T09:41:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.309+0000 7fe7417c3640 1 -- 172.21.15.47:0/1140040034 shutdown_connections 2024-03-23T09:41:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.309+0000 7fe7417c3640 1 --2- 172.21.15.47:0/1140040034 >> v2:172.21.15.175:3300/0 conn(0x7fe73c0cdf50 0x7fe73c0ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.309+0000 7fe7417c3640 1 --2- 172.21.15.47:0/1140040034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe73c15deb0 0x7fe73c0cda10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.309+0000 7fe7417c3640 1 -- 172.21.15.47:0/1140040034 >> 172.21.15.47:0/1140040034 conn(0x7fe73c0cb7c0 msgr2=0x7fe73c0cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.309+0000 7fe7417c3640 1 -- 172.21.15.47:0/1140040034 shutdown_connections 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.309+0000 7fe7417c3640 1 -- 172.21.15.47:0/1140040034 wait complete. 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.310+0000 7fe7417c3640 1 Processor -- start 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.310+0000 7fe7417c3640 1 -- start start 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.311+0000 7fe7417c3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73c06df60 0x7fe73c0c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.311+0000 7fe7417c3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe73c0cdf50 0x7fe73c0b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.311+0000 7fe7417c3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe73c15deb0 0x7fe73c157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.311+0000 7fe7417c3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c14d510 con 0x7fe73c0cdf50 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.311+0000 7fe7417c3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0c5e10 con 0x7fe73c06df60 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.311+0000 7fe7417c3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0c5890 con 0x7fe73c15deb0 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.311+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe73c0cdf50 0x7fe73c0b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.311+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe73c0cdf50 0x7fe73c0b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48762/0 (socket says 172.21.15.47:48762) 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.311+0000 7fe73a7fc640 1 -- 172.21.15.47:0/992023440 learned_addr learned my addr 172.21.15.47:0/992023440 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.312+0000 7fe73affd640 1 --2- 172.21.15.47:0/992023440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73c06df60 0x7fe73c0c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.312+0000 7fe73a7fc640 1 -- 172.21.15.47:0/992023440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe73c15deb0 msgr2=0x7fe73c157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.312+0000 7fe73a7fc640 1 --2- 172.21.15.47:0/992023440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe73c15deb0 0x7fe73c157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.312+0000 7fe73a7fc640 1 -- 172.21.15.47:0/992023440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73c06df60 msgr2=0x7fe73c0c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.312+0000 7fe73a7fc640 1 --2- 172.21.15.47:0/992023440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73c06df60 0x7fe73c0c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.312+0000 7fe73a7fc640 1 -- 172.21.15.47:0/992023440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe724009400 con 0x7fe73c0cdf50 2024-03-23T09:41:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.312+0000 7fe73a7fc640 1 --2- 172.21.15.47:0/992023440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe73c0cdf50 0x7fe73c0b0020 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe724034040 tx=0x7fe7240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.312+0000 7fe7337fe640 1 -- 172.21.15.47:0/992023440 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724004230 con 0x7fe73c0cdf50 2024-03-23T09:41:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.312+0000 7fe7337fe640 1 -- 172.21.15.47:0/992023440 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe724004390 con 0x7fe73c0cdf50 2024-03-23T09:41:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.312+0000 7fe7417c3640 1 -- 172.21.15.47:0/992023440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73c0c8a90 con 0x7fe73c0cdf50 2024-03-23T09:41:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.312+0000 7fe7337fe640 1 -- 172.21.15.47:0/992023440 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724032880 con 0x7fe73c0cdf50 2024-03-23T09:41:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.312+0000 7fe7417c3640 1 -- 172.21.15.47:0/992023440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73c0c6300 con 0x7fe73c0cdf50 2024-03-23T09:41:48.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.313+0000 7fe7417c3640 1 -- 172.21.15.47:0/992023440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73c06b690 con 0x7fe73c0cdf50 2024-03-23T09:41:48.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.314+0000 7fe7337fe640 1 -- 172.21.15.47:0/992023440 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe7240329e0 con 0x7fe73c0cdf50 2024-03-23T09:41:48.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.314+0000 7fe7337fe640 1 --2- 172.21.15.47:0/992023440 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe71c06ac00 0x7fe71c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.314+0000 7fe7337fe640 1 -- 172.21.15.47:0/992023440 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7240ab420 con 0x7fe73c0cdf50 2024-03-23T09:41:48.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.314+0000 7fe73affd640 1 --2- 172.21.15.47:0/992023440 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe71c06ac00 0x7fe71c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.315+0000 7fe73affd640 1 --2- 172.21.15.47:0/992023440 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe71c06ac00 0x7fe71c06d0c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe734000f60 tx=0x7fe734009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:48.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.318+0000 7fe7337fe640 1 -- 172.21.15.47:0/992023440 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe724076690 con 0x7fe73c0cdf50 2024-03-23T09:41:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.536+0000 7fe7417c3640 1 -- 172.21.15.47:0/992023440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe73c0672f0 con 0x7fe73c0cdf50 2024-03-23T09:41:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.536+0000 7fe7337fe640 1 -- 172.21.15.47:0/992023440 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4734 (secure 0 0 0) 0x7fe724079cb0 con 0x7fe73c0cdf50 2024-03-23T09:41:48.536 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:48.536 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1970049669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1970049669},{"type":"v1","addr":"172.21.15.47:6833","nonce":1970049669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4162785911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4162785911},{"type":"v1","addr":"172.21.15.47:6835","nonce":4162785911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1036174787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1036174787},{"type":"v1","addr":"172.21.15.175:6836","nonce":1036174787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:03.137957+0000","modified":"2024-03-23T09:41:05.147528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1633245139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1633245139},{"type":"v1","addr":"172.21.15.175:6837","nonce":1633245139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T09:41:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2024-03-23T09:41:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.538+0000 7fe7417c3640 1 -- 172.21.15.47:0/992023440 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe71c06ac00 msgr2=0x7fe71c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.538+0000 7fe7417c3640 1 --2- 172.21.15.47:0/992023440 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe71c06ac00 0x7fe71c06d0c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe734000f60 tx=0x7fe734009040 comp rx=0 tx=0).stop 2024-03-23T09:41:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.538+0000 7fe7417c3640 1 -- 172.21.15.47:0/992023440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe73c0cdf50 msgr2=0x7fe73c0b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.538+0000 7fe7417c3640 1 --2- 172.21.15.47:0/992023440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe73c0cdf50 0x7fe73c0b0020 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe724034040 tx=0x7fe7240046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.538+0000 7fe7417c3640 1 -- 172.21.15.47:0/992023440 shutdown_connections 2024-03-23T09:41:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.538+0000 7fe7417c3640 1 --2- 172.21.15.47:0/992023440 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe71c06ac00 0x7fe71c06d0c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.538+0000 7fe7417c3640 1 --2- 172.21.15.47:0/992023440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe73c15deb0 0x7fe73c157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.538+0000 7fe7417c3640 1 --2- 172.21.15.47:0/992023440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe73c0cdf50 0x7fe73c0b0020 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.538+0000 7fe7417c3640 1 --2- 172.21.15.47:0/992023440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73c06df60 0x7fe73c0c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.538+0000 7fe7417c3640 1 -- 172.21.15.47:0/992023440 >> 172.21.15.47:0/992023440 conn(0x7fe73c0cb7c0 msgr2=0x7fe73c0abdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.538+0000 7fe7417c3640 1 -- 172.21.15.47:0/992023440 shutdown_connections 2024-03-23T09:41:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.538+0000 7fe7417c3640 1 -- 172.21.15.47:0/992023440 wait complete. 2024-03-23T09:41:48.549 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.649+0000 7f3b4ed8b640 1 Processor -- start 2024-03-23T09:41:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.650+0000 7f3b4ed8b640 1 -- start start 2024-03-23T09:41:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.650+0000 7f3b4ed8b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b48099410 0x7f3b480c4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.650+0000 7f3b4ed8b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b48099a40 0x7f3b480cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.650+0000 7f3b4ed8b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b480cb1d0 con 0x7f3b480ceca0 2024-03-23T09:41:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.650+0000 7f3b4ed8b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4815e230 con 0x7f3b48099a40 2024-03-23T09:41:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.650+0000 7f3b4ed8b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480ce2b0 con 0x7f3b48099410 2024-03-23T09:41:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.650+0000 7f3b4d328640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b48099a40 0x7f3b480cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.650+0000 7f3b4d328640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b48099a40 0x7f3b480cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48778/0 (socket says 172.21.15.47:48778) 2024-03-23T09:41:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.650+0000 7f3b4d328640 1 -- 172.21.15.47:0/4020516059 learned_addr learned my addr 172.21.15.47:0/4020516059 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.651+0000 7f3b47fff640 1 --2- 172.21.15.47:0/4020516059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b48099410 0x7f3b480c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.651+0000 7f3b4d328640 1 -- 172.21.15.47:0/4020516059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b48099410 msgr2=0x7f3b480c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.651+0000 7f3b4d328640 1 --2- 172.21.15.47:0/4020516059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b48099410 0x7f3b480c4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.651+0000 7f3b4d328640 1 -- 172.21.15.47:0/4020516059 >> v1:172.21.15.175:6789/0 conn(0x7f3b480ceca0 legacy=0x7f3b480cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.651+0000 7f3b4d328640 1 -- 172.21.15.47:0/4020516059 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b480ce590 con 0x7f3b48099a40 2024-03-23T09:41:48.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.651+0000 7f3b47fff640 1 --2- 172.21.15.47:0/4020516059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b48099410 0x7f3b480c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.652+0000 7f3b4d328640 1 --2- 172.21.15.47:0/4020516059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b48099a40 0x7f3b480cb970 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009e80 tx=0x7f3b4002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba318b892cf52ac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.652+0000 7f3b477fe640 1 -- 172.21.15.47:0/4020516059 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4002ca50 con 0x7f3b48099a40 2024-03-23T09:41:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.652+0000 7f3b477fe640 1 -- 172.21.15.47:0/4020516059 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4002cbb0 con 0x7f3b48099a40 2024-03-23T09:41:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.652+0000 7f3b477fe640 1 -- 172.21.15.47:0/4020516059 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b400343f0 con 0x7f3b48099a40 2024-03-23T09:41:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.652+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/4020516059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b48099a40 msgr2=0x7f3b480cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.652+0000 7f3b4ed8b640 1 --2- 172.21.15.47:0/4020516059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b48099a40 0x7f3b480cb970 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009e80 tx=0x7f3b4002beb0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.652+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/4020516059 shutdown_connections 2024-03-23T09:41:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.652+0000 7f3b4ed8b640 1 --2- 172.21.15.47:0/4020516059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b48099a40 0x7f3b480cb970 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.652+0000 7f3b4ed8b640 1 --2- 172.21.15.47:0/4020516059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b48099410 0x7f3b480c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.652+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/4020516059 >> 172.21.15.47:0/4020516059 conn(0x7f3b480b21d0 msgr2=0x7f3b480b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.652+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/4020516059 shutdown_connections 2024-03-23T09:41:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.652+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/4020516059 wait complete. 2024-03-23T09:41:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.653+0000 7f3b4ed8b640 1 Processor -- start 2024-03-23T09:41:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.653+0000 7f3b4ed8b640 1 -- start start 2024-03-23T09:41:48.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.654+0000 7f3b4ed8b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b48099410 0x7f3b4814c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.654+0000 7f3b4ed8b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b48099a40 0x7f3b481ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.654+0000 7f3b4ed8b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b480ceca0 0x7f3b480b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.654+0000 7f3b4ed8b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480af440 con 0x7f3b480ceca0 2024-03-23T09:41:48.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.654+0000 7f3b4ed8b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480af720 con 0x7f3b48099410 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.654+0000 7f3b4ed8b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480afa00 con 0x7f3b48099a40 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.654+0000 7f3b4d328640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b480ceca0 0x7f3b480b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.654+0000 7f3b4d328640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b480ceca0 0x7f3b480b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48792/0 (socket says 172.21.15.47:48792) 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.654+0000 7f3b4d328640 1 -- 172.21.15.47:0/1754323175 learned_addr learned my addr 172.21.15.47:0/1754323175 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b4cb27640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b48099410 0x7f3b4814c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b47fff640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b48099a40 0x7f3b481ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b4d328640 1 -- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b48099a40 msgr2=0x7f3b481ef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b4d328640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b48099a40 0x7f3b481ef8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b4d328640 1 -- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b48099410 msgr2=0x7f3b4814c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b4d328640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b48099410 0x7f3b4814c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b4d328640 1 -- 172.21.15.47:0/1754323175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b40009b30 con 0x7f3b480ceca0 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b47fff640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b48099a40 0x7f3b481ef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b4d328640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b480ceca0 0x7f3b480b5390 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3b4002bef0 tx=0x7f3b40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b457fa640 1 -- 172.21.15.47:0/1754323175 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40009510 con 0x7f3b480ceca0 2024-03-23T09:41:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b457fa640 1 -- 172.21.15.47:0/1754323175 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4003e450 con 0x7f3b480ceca0 2024-03-23T09:41:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b457fa640 1 -- 172.21.15.47:0/1754323175 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4003e7e0 con 0x7f3b480ceca0 2024-03-23T09:41:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/1754323175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b480afce0 con 0x7f3b480ceca0 2024-03-23T09:41:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.655+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/1754323175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b481efe00 con 0x7f3b480ceca0 2024-03-23T09:41:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.656+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/1754323175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b481eff90 con 0x7f3b480ceca0 2024-03-23T09:41:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.656+0000 7f3b457fa640 1 -- 172.21.15.47:0/1754323175 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3b4003e940 con 0x7f3b480ceca0 2024-03-23T09:41:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.657+0000 7f3b457fa640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b2c06ac00 0x7f3b2c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.657+0000 7f3b457fa640 1 -- 172.21.15.47:0/1754323175 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b400ac0c0 con 0x7f3b480ceca0 2024-03-23T09:41:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.657+0000 7f3b4cb27640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b2c06ac00 0x7f3b2c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.658+0000 7f3b4cb27640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b2c06ac00 0x7f3b2c06d0c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c000f60 tx=0x7f3b3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.661+0000 7f3b457fa640 1 -- 172.21.15.47:0/1754323175 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3b400772a0 con 0x7f3b480ceca0 2024-03-23T09:41:48.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.845+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/1754323175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b480b01a0 con 0x7f3b480ceca0 2024-03-23T09:41:48.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.846+0000 7f3b457fa640 1 -- 172.21.15.47:0/1754323175 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f3b4007a8c0 con 0x7f3b480ceca0 2024-03-23T09:41:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:48.846 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:48.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.847+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b2c06ac00 msgr2=0x7f3b2c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.847+0000 7f3b4ed8b640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b2c06ac00 0x7f3b2c06d0c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c000f60 tx=0x7f3b3c009040 comp rx=0 tx=0).stop 2024-03-23T09:41:48.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.847+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/1754323175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b480ceca0 msgr2=0x7f3b480b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.847+0000 7f3b4ed8b640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b480ceca0 0x7f3b480b5390 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3b4002bef0 tx=0x7f3b40004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:48.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.848+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/1754323175 shutdown_connections 2024-03-23T09:41:48.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.848+0000 7f3b4ed8b640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b2c06ac00 0x7f3b2c06d0c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.848+0000 7f3b4ed8b640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b480ceca0 0x7f3b480b5390 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.848+0000 7f3b4ed8b640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b48099a40 0x7f3b481ef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.848+0000 7f3b4ed8b640 1 --2- 172.21.15.47:0/1754323175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b48099410 0x7f3b4814c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.848+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/1754323175 >> 172.21.15.47:0/1754323175 conn(0x7f3b480b21d0 msgr2=0x7f3b480c97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:48.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.848+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/1754323175 shutdown_connections 2024-03-23T09:41:48.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.848+0000 7f3b4ed8b640 1 -- 172.21.15.47:0/1754323175 wait complete. 2024-03-23T09:41:48.859 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T09:41:48.859 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:41:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.960+0000 7fecac282640 1 Processor -- start 2024-03-23T09:41:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.960+0000 7fecac282640 1 -- start start 2024-03-23T09:41:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.961+0000 7fecac282640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feca415cd00 0x7feca40d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.961+0000 7fecac282640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feca40d1020 0x7feca40cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.961+0000 7fecac282640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca40cfba0 con 0x7feca40cf750 2024-03-23T09:41:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.961+0000 7fecac282640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca415d0e0 con 0x7feca40d1020 2024-03-23T09:41:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.961+0000 7fecac282640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca40d1540 con 0x7feca415cd00 2024-03-23T09:41:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.961+0000 7feca981d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feca40d1020 0x7feca40cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.961+0000 7feca981d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feca40d1020 0x7feca40cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48794/0 (socket says 172.21.15.47:48794) 2024-03-23T09:41:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.961+0000 7fecaa81f640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feca40cf750 0x7feca40b43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47990/0 (socket says 172.21.15.47:47990) 2024-03-23T09:41:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.961+0000 7feca981d640 1 -- 172.21.15.47:0/296412804 learned_addr learned my addr 172.21.15.47:0/296412804 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.961+0000 7fecaa01e640 1 --2- 172.21.15.47:0/296412804 >> v2:172.21.15.175:3300/0 conn(0x7feca415cd00 0x7feca40d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.961+0000 7feca981d640 1 -- 172.21.15.47:0/296412804 >> v2:172.21.15.175:3300/0 conn(0x7feca415cd00 msgr2=0x7feca40d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.962+0000 7feca981d640 1 --2- 172.21.15.47:0/296412804 >> v2:172.21.15.175:3300/0 conn(0x7feca415cd00 0x7feca40d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.962+0000 7feca981d640 1 -- 172.21.15.47:0/296412804 >> v1:172.21.15.47:6789/0 conn(0x7feca40cf750 legacy=0x7feca40b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.962+0000 7feca901c640 1 -- 172.21.15.47:0/296412804 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574702459 0 0) 0x7feca40cfba0 con 0x7feca40cf750 2024-03-23T09:41:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.962+0000 7feca981d640 1 -- 172.21.15.47:0/296412804 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca40d1820 con 0x7feca40d1020 2024-03-23T09:41:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.962+0000 7feca981d640 1 --2- 172.21.15.47:0/296412804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca40d1020 0x7feca40cf210 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feca406c1a0 tx=0x7fec9802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781ef78a11a0ef59 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.962+0000 7feca901c640 1 -- 172.21.15.47:0/296412804 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec98031870 con 0x7feca40d1020 2024-03-23T09:41:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.962+0000 7feca901c640 1 -- 172.21.15.47:0/296412804 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec980319d0 con 0x7feca40d1020 2024-03-23T09:41:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.962+0000 7feca901c640 1 -- 172.21.15.47:0/296412804 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec98031d60 con 0x7feca40d1020 2024-03-23T09:41:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.963+0000 7fecac282640 1 -- 172.21.15.47:0/296412804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca40d1020 msgr2=0x7feca40cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.963+0000 7fecac282640 1 --2- 172.21.15.47:0/296412804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca40d1020 0x7feca40cf210 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feca406c1a0 tx=0x7fec9802b910 comp rx=0 tx=0).stop 2024-03-23T09:41:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.963+0000 7fecac282640 1 -- 172.21.15.47:0/296412804 shutdown_connections 2024-03-23T09:41:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.963+0000 7fecac282640 1 --2- 172.21.15.47:0/296412804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca40d1020 0x7feca40cf210 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.963+0000 7fecac282640 1 --2- 172.21.15.47:0/296412804 >> v2:172.21.15.175:3300/0 conn(0x7feca415cd00 0x7feca40d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.963+0000 7fecac282640 1 -- 172.21.15.47:0/296412804 >> 172.21.15.47:0/296412804 conn(0x7feca406b160 msgr2=0x7feca406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.963+0000 7fecac282640 1 -- 172.21.15.47:0/296412804 shutdown_connections 2024-03-23T09:41:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.963+0000 7fecac282640 1 -- 172.21.15.47:0/296412804 wait complete. 2024-03-23T09:41:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.964+0000 7fecac282640 1 Processor -- start 2024-03-23T09:41:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.964+0000 7fecac282640 1 -- start start 2024-03-23T09:41:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7fecac282640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca40cf750 0x7feca415b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7fecac282640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca40d1020 0x7feca41f1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7fecac282640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feca415cd00 0x7feca41f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7fecac282640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca4155000 con 0x7feca40d1020 2024-03-23T09:41:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7fecac282640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca41552e0 con 0x7feca40cf750 2024-03-23T09:41:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7fecac282640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca40cab20 con 0x7feca415cd00 2024-03-23T09:41:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7feca981d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca40d1020 0x7feca41f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7feca981d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca40d1020 0x7feca41f1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48810/0 (socket says 172.21.15.47:48810) 2024-03-23T09:41:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7feca981d640 1 -- 172.21.15.47:0/3202170045 learned_addr learned my addr 172.21.15.47:0/3202170045 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7fecaa01e640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca40cf750 0x7feca415b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7fecaa81f640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feca415cd00 0x7feca41f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7feca981d640 1 -- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feca415cd00 msgr2=0x7feca41f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7feca981d640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feca415cd00 0x7feca41f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7feca981d640 1 -- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca40cf750 msgr2=0x7feca415b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7feca981d640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca40cf750 0x7feca415b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.965+0000 7feca981d640 1 -- 172.21.15.47:0/3202170045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec980094d0 con 0x7feca40d1020 2024-03-23T09:41:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.966+0000 7feca981d640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca40d1020 0x7feca41f1620 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fec9802b950 tx=0x7fec980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.966+0000 7feca2ffd640 1 -- 172.21.15.47:0/3202170045 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9803abe0 con 0x7feca40d1020 2024-03-23T09:41:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.966+0000 7feca2ffd640 1 -- 172.21.15.47:0/3202170045 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9803ad60 con 0x7feca40d1020 2024-03-23T09:41:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.966+0000 7fecac282640 1 -- 172.21.15.47:0/3202170045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca40cae00 con 0x7feca40d1020 2024-03-23T09:41:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.966+0000 7feca2ffd640 1 -- 172.21.15.47:0/3202170045 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec98044630 con 0x7feca40d1020 2024-03-23T09:41:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.966+0000 7fecac282640 1 -- 172.21.15.47:0/3202170045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca41555c0 con 0x7feca40d1020 2024-03-23T09:41:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.967+0000 7fecac282640 1 -- 172.21.15.47:0/3202170045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca4072360 con 0x7feca40d1020 2024-03-23T09:41:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.967+0000 7feca2ffd640 1 -- 172.21.15.47:0/3202170045 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fec9804d450 con 0x7feca40d1020 2024-03-23T09:41:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.968+0000 7feca2ffd640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fec9006ac00 0x7fec9006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.968+0000 7feca2ffd640 1 -- 172.21.15.47:0/3202170045 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec9802d040 con 0x7feca40d1020 2024-03-23T09:41:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.968+0000 7fecaa01e640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fec9006ac00 0x7fec9006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.968+0000 7fecaa01e640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fec9006ac00 0x7fec9006d0c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009a30 tx=0x7fec9c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:48.972+0000 7feca2ffd640 1 -- 172.21.15.47:0/3202170045 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fec98076ad0 con 0x7feca40d1020 2024-03-23T09:41:49.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.187+0000 7fecac282640 1 -- 172.21.15.47:0/3202170045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feca40cb010 con 0x7feca40d1020 2024-03-23T09:41:49.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.188+0000 7feca2ffd640 1 -- 172.21.15.47:0/3202170045 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4734 (secure 0 0 0) 0x7fec98035020 con 0x7feca40d1020 2024-03-23T09:41:49.188 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:49.188 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1970049669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1970049669},{"type":"v1","addr":"172.21.15.47:6833","nonce":1970049669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4162785911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4162785911},{"type":"v1","addr":"172.21.15.47:6835","nonce":4162785911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1036174787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1036174787},{"type":"v1","addr":"172.21.15.175:6836","nonce":1036174787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:03.137957+0000","modified":"2024-03-23T09:41:05.147528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1633245139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1633245139},{"type":"v1","addr":"172.21.15.175:6837","nonce":1633245139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T09:41:49.188 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2024-03-23T09:41:49.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.190+0000 7fecac282640 1 -- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fec9006ac00 msgr2=0x7fec9006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.190+0000 7fecac282640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fec9006ac00 0x7fec9006d0c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009a30 tx=0x7fec9c008040 comp rx=0 tx=0).stop 2024-03-23T09:41:49.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.190+0000 7fecac282640 1 -- 172.21.15.47:0/3202170045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca40d1020 msgr2=0x7feca41f1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.190+0000 7fecac282640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca40d1020 0x7feca41f1620 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fec9802b950 tx=0x7fec980046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.190+0000 7fecac282640 1 -- 172.21.15.47:0/3202170045 shutdown_connections 2024-03-23T09:41:49.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.190+0000 7fecac282640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fec9006ac00 0x7fec9006d0c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.190+0000 7fecac282640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feca415cd00 0x7feca41f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.190+0000 7fecac282640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca40d1020 0x7feca41f1620 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.190+0000 7fecac282640 1 --2- 172.21.15.47:0/3202170045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca40cf750 0x7feca415b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.190+0000 7fecac282640 1 -- 172.21.15.47:0/3202170045 >> 172.21.15.47:0/3202170045 conn(0x7feca406b160 msgr2=0x7feca40cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:49.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.190+0000 7fecac282640 1 -- 172.21.15.47:0/3202170045 shutdown_connections 2024-03-23T09:41:49.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.190+0000 7fecac282640 1 -- 172.21.15.47:0/3202170045 wait complete. 2024-03-23T09:41:49.200 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:41:49.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.301+0000 7f3215700640 1 Processor -- start 2024-03-23T09:41:49.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.301+0000 7f3215700640 1 -- start start 2024-03-23T09:41:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3215700640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f321015ca90 0x7f3210160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3215700640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f32100b8350 0x7f32100cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3215700640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32100cdc40 0x7f32100b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3215700640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32100ce0d0 con 0x7f32100b8350 2024-03-23T09:41:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3215700640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32100d11e0 con 0x7f321015ca90 2024-03-23T09:41:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3215700640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3210160f60 con 0x7f32100cdc40 2024-03-23T09:41:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3207fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f32100b8350 0x7f32100cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3207fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f32100b8350 0x7f32100cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48818/0 (socket says 172.21.15.47:48818) 2024-03-23T09:41:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3207fff640 1 -- 172.21.15.47:0/92420398 learned_addr learned my addr 172.21.15.47:0/92420398 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f320effd640 1 --2- 172.21.15.47:0/92420398 >> v2:172.21.15.175:3300/0 conn(0x7f321015ca90 0x7f3210160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f320f7fe640 1 --2- 172.21.15.47:0/92420398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32100cdc40 0x7f32100b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3207fff640 1 -- 172.21.15.47:0/92420398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32100cdc40 msgr2=0x7f32100b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3207fff640 1 --2- 172.21.15.47:0/92420398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32100cdc40 0x7f32100b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3207fff640 1 -- 172.21.15.47:0/92420398 >> v2:172.21.15.175:3300/0 conn(0x7f321015ca90 msgr2=0x7f3210160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3207fff640 1 --2- 172.21.15.47:0/92420398 >> v2:172.21.15.175:3300/0 conn(0x7f321015ca90 0x7f3210160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f3207fff640 1 -- 172.21.15.47:0/92420398 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3210161240 con 0x7f32100b8350 2024-03-23T09:41:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.302+0000 7f320f7fe640 1 --2- 172.21.15.47:0/92420398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32100cdc40 0x7f32100b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.303+0000 7f3207fff640 1 --2- 172.21.15.47:0/92420398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32100b8350 0x7f32100cd700 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3210066680 tx=0x7f31f802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300a80c742f004bf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.303+0000 7f320e7fc640 1 -- 172.21.15.47:0/92420398 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f80317e0 con 0x7f32100b8350 2024-03-23T09:41:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.303+0000 7f320e7fc640 1 -- 172.21.15.47:0/92420398 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f8031940 con 0x7f32100b8350 2024-03-23T09:41:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.303+0000 7f320e7fc640 1 -- 172.21.15.47:0/92420398 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8031cd0 con 0x7f32100b8350 2024-03-23T09:41:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.304+0000 7f3215700640 1 -- 172.21.15.47:0/92420398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32100b8350 msgr2=0x7f32100cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.304+0000 7f3215700640 1 --2- 172.21.15.47:0/92420398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32100b8350 0x7f32100cd700 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3210066680 tx=0x7f31f802b880 comp rx=0 tx=0).stop 2024-03-23T09:41:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.304+0000 7f3215700640 1 -- 172.21.15.47:0/92420398 shutdown_connections 2024-03-23T09:41:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.304+0000 7f3215700640 1 --2- 172.21.15.47:0/92420398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32100cdc40 0x7f32100b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.304+0000 7f3215700640 1 --2- 172.21.15.47:0/92420398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32100b8350 0x7f32100cd700 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.304+0000 7f3215700640 1 --2- 172.21.15.47:0/92420398 >> v2:172.21.15.175:3300/0 conn(0x7f321015ca90 0x7f3210160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.304+0000 7f3215700640 1 -- 172.21.15.47:0/92420398 >> 172.21.15.47:0/92420398 conn(0x7f32100659d0 msgr2=0x7f3210065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.304+0000 7f3215700640 1 -- 172.21.15.47:0/92420398 shutdown_connections 2024-03-23T09:41:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.304+0000 7f3215700640 1 -- 172.21.15.47:0/92420398 wait complete. 2024-03-23T09:41:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.305+0000 7f3215700640 1 Processor -- start 2024-03-23T09:41:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.305+0000 7f3215700640 1 -- start start 2024-03-23T09:41:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f3215700640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32100b8350 0x7f32100cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f3215700640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32100cdc40 0x7f32101f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f3215700640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f321015ca90 0x7f32101507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f3215700640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100b2530 con 0x7f321015ca90 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f3215700640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100c9a90 con 0x7f32100b8350 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f3215700640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32101f2740 con 0x7f32100cdc40 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f320f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f321015ca90 0x7f32101507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f320f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f321015ca90 0x7f32101507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48828/0 (socket says 172.21.15.47:48828) 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f320f7fe640 1 -- 172.21.15.47:0/2307581285 learned_addr learned my addr 172.21.15.47:0/2307581285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f320effd640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32100b8350 0x7f32100cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f3207fff640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32100cdc40 0x7f32101f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f320f7fe640 1 -- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32100cdc40 msgr2=0x7f32101f1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f320f7fe640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32100cdc40 0x7f32101f1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f320f7fe640 1 -- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32100b8350 msgr2=0x7f32100cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f320f7fe640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32100b8350 0x7f32100cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f320f7fe640 1 -- 172.21.15.47:0/2307581285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f80094d0 con 0x7f321015ca90 2024-03-23T09:41:49.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f320f7fe640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f321015ca90 0x7f32101507c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3208007db0 tx=0x7f320800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f32077fe640 1 -- 172.21.15.47:0/2307581285 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208003990 con 0x7f321015ca90 2024-03-23T09:41:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f32077fe640 1 -- 172.21.15.47:0/2307581285 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320800f040 con 0x7f321015ca90 2024-03-23T09:41:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f3215700640 1 -- 172.21.15.47:0/2307581285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32101f2a20 con 0x7f321015ca90 2024-03-23T09:41:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f32077fe640 1 -- 172.21.15.47:0/2307581285 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208003d50 con 0x7f321015ca90 2024-03-23T09:41:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.306+0000 7f3215700640 1 -- 172.21.15.47:0/2307581285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32100c9d50 con 0x7f321015ca90 2024-03-23T09:41:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.307+0000 7f3215700640 1 -- 172.21.15.47:0/2307581285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3210072360 con 0x7f321015ca90 2024-03-23T09:41:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.308+0000 7f32077fe640 1 -- 172.21.15.47:0/2307581285 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3208004030 con 0x7f321015ca90 2024-03-23T09:41:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.308+0000 7f32077fe640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f31f006ac00 0x7f31f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.308+0000 7f32077fe640 1 -- 172.21.15.47:0/2307581285 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32080895f0 con 0x7f321015ca90 2024-03-23T09:41:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.308+0000 7f320effd640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f31f006ac00 0x7f31f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.309+0000 7f320effd640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f31f006ac00 0x7f31f006d0c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f32100b7da0 tx=0x7f3200009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:49.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.312+0000 7f32077fe640 1 -- 172.21.15.47:0/2307581285 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f32080547f0 con 0x7f321015ca90 2024-03-23T09:41:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.529+0000 7f3215700640 1 -- 172.21.15.47:0/2307581285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32100c9f70 con 0x7f321015ca90 2024-03-23T09:41:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.529+0000 7f32077fe640 1 -- 172.21.15.47:0/2307581285 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4734 (secure 0 0 0) 0x7f3208057e10 con 0x7f321015ca90 2024-03-23T09:41:49.529 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:49.529 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1970049669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1970049669},{"type":"v1","addr":"172.21.15.47:6833","nonce":1970049669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4162785911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4162785911},{"type":"v1","addr":"172.21.15.47:6835","nonce":4162785911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1036174787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1036174787},{"type":"v1","addr":"172.21.15.175:6836","nonce":1036174787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:03.137957+0000","modified":"2024-03-23T09:41:05.147528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1633245139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1633245139},{"type":"v1","addr":"172.21.15.175:6837","nonce":1633245139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T09:41:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2024-03-23T09:41:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.531+0000 7f3215700640 1 -- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f31f006ac00 msgr2=0x7f31f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.531+0000 7f3215700640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f31f006ac00 0x7f31f006d0c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f32100b7da0 tx=0x7f3200009190 comp rx=0 tx=0).stop 2024-03-23T09:41:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.531+0000 7f3215700640 1 -- 172.21.15.47:0/2307581285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f321015ca90 msgr2=0x7f32101507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.531+0000 7f3215700640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f321015ca90 0x7f32101507c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3208007db0 tx=0x7f320800d4a0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.531+0000 7f3215700640 1 -- 172.21.15.47:0/2307581285 shutdown_connections 2024-03-23T09:41:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.531+0000 7f3215700640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f31f006ac00 0x7f31f006d0c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.531+0000 7f3215700640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f321015ca90 0x7f32101507c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.531+0000 7f3215700640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32100cdc40 0x7f32101f1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.531+0000 7f3215700640 1 --2- 172.21.15.47:0/2307581285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32100b8350 0x7f32100cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.531+0000 7f3215700640 1 -- 172.21.15.47:0/2307581285 >> 172.21.15.47:0/2307581285 conn(0x7f32100659d0 msgr2=0x7f321015e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.531+0000 7f3215700640 1 -- 172.21.15.47:0/2307581285 shutdown_connections 2024-03-23T09:41:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.531+0000 7f3215700640 1 -- 172.21.15.47:0/2307581285 wait complete. 2024-03-23T09:41:49.542 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:49.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.643+0000 7fee53f3e640 1 Processor -- start 2024-03-23T09:41:49.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.643+0000 7fee53f3e640 1 -- start start 2024-03-23T09:41:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.643+0000 7fee53f3e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fee4c15cd00 0x7fee4c0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.643+0000 7fee53f3e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee4c0d1020 0x7fee4c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.643+0000 7fee53f3e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee4c0cf750 0x7fee4c0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.644+0000 7fee53f3e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4c0cfbe0 con 0x7fee4c0cf750 2024-03-23T09:41:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.644+0000 7fee53f3e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4c15d0e0 con 0x7fee4c15cd00 2024-03-23T09:41:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.644+0000 7fee53f3e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0d1540 con 0x7fee4c0d1020 2024-03-23T09:41:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.644+0000 7fee524db640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee4c0cf750 0x7fee4c0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.644+0000 7fee524db640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee4c0cf750 0x7fee4c0b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48842/0 (socket says 172.21.15.47:48842) 2024-03-23T09:41:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.644+0000 7fee524db640 1 -- 172.21.15.47:0/1018197619 learned_addr learned my addr 172.21.15.47:0/1018197619 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.644+0000 7fee51cda640 1 --2- 172.21.15.47:0/1018197619 >> v2:172.21.15.175:3300/0 conn(0x7fee4c15cd00 0x7fee4c0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.644+0000 7fee514d9640 1 --2- 172.21.15.47:0/1018197619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee4c0d1020 0x7fee4c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.644+0000 7fee524db640 1 -- 172.21.15.47:0/1018197619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee4c0d1020 msgr2=0x7fee4c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.644+0000 7fee524db640 1 --2- 172.21.15.47:0/1018197619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee4c0d1020 0x7fee4c0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.644+0000 7fee524db640 1 -- 172.21.15.47:0/1018197619 >> v2:172.21.15.175:3300/0 conn(0x7fee4c15cd00 msgr2=0x7fee4c0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.644+0000 7fee524db640 1 --2- 172.21.15.47:0/1018197619 >> v2:172.21.15.175:3300/0 conn(0x7fee4c15cd00 0x7fee4c0d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.644+0000 7fee524db640 1 -- 172.21.15.47:0/1018197619 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0d1820 con 0x7fee4c0cf750 2024-03-23T09:41:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.645+0000 7fee524db640 1 --2- 172.21.15.47:0/1018197619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c0cf750 0x7fee4c0b43e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fee4c06c1a0 tx=0x7fee4402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3162a4c3c107691 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.645+0000 7fee50cd8640 1 -- 172.21.15.47:0/1018197619 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4402c9e0 con 0x7fee4c0cf750 2024-03-23T09:41:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.645+0000 7fee50cd8640 1 -- 172.21.15.47:0/1018197619 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4402cb40 con 0x7fee4c0cf750 2024-03-23T09:41:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.645+0000 7fee50cd8640 1 -- 172.21.15.47:0/1018197619 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee440343f0 con 0x7fee4c0cf750 2024-03-23T09:41:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.645+0000 7fee53f3e640 1 -- 172.21.15.47:0/1018197619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c0cf750 msgr2=0x7fee4c0b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.645+0000 7fee53f3e640 1 --2- 172.21.15.47:0/1018197619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c0cf750 0x7fee4c0b43e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fee4c06c1a0 tx=0x7fee4402be40 comp rx=0 tx=0).stop 2024-03-23T09:41:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.645+0000 7fee53f3e640 1 -- 172.21.15.47:0/1018197619 shutdown_connections 2024-03-23T09:41:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.645+0000 7fee53f3e640 1 --2- 172.21.15.47:0/1018197619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c0cf750 0x7fee4c0b43e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.645+0000 7fee53f3e640 1 --2- 172.21.15.47:0/1018197619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee4c0d1020 0x7fee4c0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.645+0000 7fee53f3e640 1 --2- 172.21.15.47:0/1018197619 >> v2:172.21.15.175:3300/0 conn(0x7fee4c15cd00 0x7fee4c0d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.645+0000 7fee53f3e640 1 -- 172.21.15.47:0/1018197619 >> 172.21.15.47:0/1018197619 conn(0x7fee4c06b160 msgr2=0x7fee4c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.646+0000 7fee53f3e640 1 -- 172.21.15.47:0/1018197619 shutdown_connections 2024-03-23T09:41:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.646+0000 7fee53f3e640 1 -- 172.21.15.47:0/1018197619 wait complete. 2024-03-23T09:41:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.647+0000 7fee53f3e640 1 Processor -- start 2024-03-23T09:41:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.647+0000 7fee53f3e640 1 -- start start 2024-03-23T09:41:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.647+0000 7fee53f3e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c0cf750 0x7fee4c15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.647+0000 7fee53f3e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee4c0d1020 0x7fee4c0aa540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.647+0000 7fee53f3e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15cd00 0x7fee4c0aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.647+0000 7fee53f3e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0caa50 con 0x7fee4c15cd00 2024-03-23T09:41:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.647+0000 7fee53f3e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c15a6d0 con 0x7fee4c0cf750 2024-03-23T09:41:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.647+0000 7fee53f3e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0c9dd0 con 0x7fee4c0d1020 2024-03-23T09:41:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.647+0000 7fee524db640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15cd00 0x7fee4c0aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee524db640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15cd00 0x7fee4c0aaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48854/0 (socket says 172.21.15.47:48854) 2024-03-23T09:41:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee524db640 1 -- 172.21.15.47:0/3068923255 learned_addr learned my addr 172.21.15.47:0/3068923255 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee51cda640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c0cf750 0x7fee4c15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee514d9640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee4c0d1020 0x7fee4c0aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee524db640 1 -- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee4c0d1020 msgr2=0x7fee4c0aa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee524db640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee4c0d1020 0x7fee4c0aa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee524db640 1 -- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c0cf750 msgr2=0x7fee4c15a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee524db640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c0cf750 0x7fee4c15a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee524db640 1 -- 172.21.15.47:0/3068923255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee44009b30 con 0x7fee4c15cd00 2024-03-23T09:41:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee524db640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15cd00 0x7fee4c0aaa80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fee44036040 tx=0x7fee44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee427fc640 1 -- 172.21.15.47:0/3068923255 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44009510 con 0x7fee4c15cd00 2024-03-23T09:41:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee427fc640 1 -- 172.21.15.47:0/3068923255 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4403e450 con 0x7fee4c15cd00 2024-03-23T09:41:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee53f3e640 1 -- 172.21.15.47:0/3068923255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c0ca0b0 con 0x7fee4c15cd00 2024-03-23T09:41:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee427fc640 1 -- 172.21.15.47:0/3068923255 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4403e800 con 0x7fee4c15cd00 2024-03-23T09:41:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.648+0000 7fee53f3e640 1 -- 172.21.15.47:0/3068923255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c0ca670 con 0x7fee4c15cd00 2024-03-23T09:41:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.649+0000 7fee53f3e640 1 -- 172.21.15.47:0/3068923255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c072360 con 0x7fee4c15cd00 2024-03-23T09:41:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.650+0000 7fee427fc640 1 -- 172.21.15.47:0/3068923255 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fee4403e960 con 0x7fee4c15cd00 2024-03-23T09:41:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.650+0000 7fee427fc640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3006ac00 0x7fee3006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.650+0000 7fee427fc640 1 -- 172.21.15.47:0/3068923255 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee440abf50 con 0x7fee4c15cd00 2024-03-23T09:41:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.650+0000 7fee51cda640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3006ac00 0x7fee3006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.651+0000 7fee51cda640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3006ac00 0x7fee3006d0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fee48009b40 tx=0x7fee48009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.654+0000 7fee427fc640 1 -- 172.21.15.47:0/3068923255 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fee440771e0 con 0x7fee4c15cd00 2024-03-23T09:41:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.835+0000 7fee53f3e640 1 -- 172.21.15.47:0/3068923255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee4c0cab90 con 0x7fee4c15cd00 2024-03-23T09:41:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.836+0000 7fee427fc640 1 -- 172.21.15.47:0/3068923255 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fee4407a800 con 0x7fee4c15cd00 2024-03-23T09:41:49.836 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:49.836 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:04.139393+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:00.911579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:02.107759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.838+0000 7fee53f3e640 1 -- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3006ac00 msgr2=0x7fee3006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.838+0000 7fee53f3e640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3006ac00 0x7fee3006d0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fee48009b40 tx=0x7fee48009190 comp rx=0 tx=0).stop 2024-03-23T09:41:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.838+0000 7fee53f3e640 1 -- 172.21.15.47:0/3068923255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15cd00 msgr2=0x7fee4c0aaa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.838+0000 7fee53f3e640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15cd00 0x7fee4c0aaa80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fee44036040 tx=0x7fee44004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.838+0000 7fee53f3e640 1 -- 172.21.15.47:0/3068923255 shutdown_connections 2024-03-23T09:41:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.838+0000 7fee53f3e640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3006ac00 0x7fee3006d0c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.838+0000 7fee53f3e640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15cd00 0x7fee4c0aaa80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.838+0000 7fee53f3e640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee4c0d1020 0x7fee4c0aa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.838+0000 7fee53f3e640 1 --2- 172.21.15.47:0/3068923255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c0cf750 0x7fee4c15a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.838+0000 7fee53f3e640 1 -- 172.21.15.47:0/3068923255 >> 172.21.15.47:0/3068923255 conn(0x7fee4c06b160 msgr2=0x7fee4c15af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.838+0000 7fee53f3e640 1 -- 172.21.15.47:0/3068923255 shutdown_connections 2024-03-23T09:41:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.838+0000 7fee53f3e640 1 -- 172.21.15.47:0/3068923255 wait complete. 2024-03-23T09:41:49.848 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:41:49.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.951+0000 7f7ead7d5640 1 Processor -- start 2024-03-23T09:41:49.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.951+0000 7f7ead7d5640 1 -- start start 2024-03-23T09:41:49.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.952+0000 7f7ead7d5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ea815cd00 0x7f7ea80d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.952+0000 7f7ead7d5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ea80cf920 0x7f7ea80b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.952+0000 7f7ead7d5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea815d0e0 con 0x7f7ea80d1020 2024-03-23T09:41:49.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.952+0000 7f7ead7d5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea80d1540 con 0x7f7ea815cd00 2024-03-23T09:41:49.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.952+0000 7f7ead7d5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea80d1820 con 0x7f7ea80cf920 2024-03-23T09:41:49.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.952+0000 7f7ea6ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ea815cd00 0x7f7ea80d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.952+0000 7f7ea6ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ea815cd00 0x7f7ea80d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48856/0 (socket says 172.21.15.47:48856) 2024-03-23T09:41:49.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.952+0000 7f7ea6ffd640 1 -- 172.21.15.47:0/1069913922 learned_addr learned my addr 172.21.15.47:0/1069913922 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:49.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.952+0000 7f7ea77fe640 1 --2- 172.21.15.47:0/1069913922 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ea80cf920 0x7f7ea80b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.952+0000 7f7ea6ffd640 1 -- 172.21.15.47:0/1069913922 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ea80cf920 msgr2=0x7f7ea80b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:49.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.952+0000 7f7ea6ffd640 1 --2- 172.21.15.47:0/1069913922 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ea80cf920 0x7f7ea80b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.952+0000 7f7ea6ffd640 1 -- 172.21.15.47:0/1069913922 >> v1:172.21.15.175:6789/0 conn(0x7f7ea80d1020 legacy=0x7f7ea80cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.952+0000 7f7ea6ffd640 1 -- 172.21.15.47:0/1069913922 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea80d1b00 con 0x7f7ea815cd00 2024-03-23T09:41:49.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.953+0000 7f7ea6ffd640 1 --2- 172.21.15.47:0/1069913922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea815cd00 0x7f7ea80d0ae0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009870 tx=0x7f7ea002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff8d98775429d7c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:49.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.953+0000 7f7ea5ffb640 1 -- 172.21.15.47:0/1069913922 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0031640 con 0x7f7ea815cd00 2024-03-23T09:41:49.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.953+0000 7f7ea5ffb640 1 -- 172.21.15.47:0/1069913922 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea00317a0 con 0x7f7ea815cd00 2024-03-23T09:41:49.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.953+0000 7f7ea5ffb640 1 -- 172.21.15.47:0/1069913922 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0031b30 con 0x7f7ea815cd00 2024-03-23T09:41:49.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.954+0000 7f7ead7d5640 1 -- 172.21.15.47:0/1069913922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea815cd00 msgr2=0x7f7ea80d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.954+0000 7f7ead7d5640 1 --2- 172.21.15.47:0/1069913922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea815cd00 0x7f7ea80d0ae0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009870 tx=0x7f7ea002b8f0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.954+0000 7f7ead7d5640 1 -- 172.21.15.47:0/1069913922 shutdown_connections 2024-03-23T09:41:49.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.954+0000 7f7ead7d5640 1 --2- 172.21.15.47:0/1069913922 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ea80cf920 0x7f7ea80b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.954+0000 7f7ead7d5640 1 --2- 172.21.15.47:0/1069913922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea815cd00 0x7f7ea80d0ae0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.954+0000 7f7ead7d5640 1 -- 172.21.15.47:0/1069913922 >> 172.21.15.47:0/1069913922 conn(0x7f7ea806b160 msgr2=0x7f7ea806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:49.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.954+0000 7f7ead7d5640 1 -- 172.21.15.47:0/1069913922 shutdown_connections 2024-03-23T09:41:49.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.954+0000 7f7ead7d5640 1 -- 172.21.15.47:0/1069913922 wait complete. 2024-03-23T09:41:49.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.955+0000 7f7ead7d5640 1 Processor -- start 2024-03-23T09:41:49.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.955+0000 7f7ead7d5640 1 -- start start 2024-03-23T09:41:49.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ead7d5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ea80cf920 0x7f7ea81f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ead7d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ea80d1020 0x7f7ea80cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ead7d5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea815cd00 0x7f7ea80cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ead7d5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea8151310 con 0x7f7ea815cd00 2024-03-23T09:41:49.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ead7d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea8150700 con 0x7f7ea80d1020 2024-03-23T09:41:49.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ead7d5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea81509e0 con 0x7f7ea80cf920 2024-03-23T09:41:49.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ea77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea815cd00 0x7f7ea80cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ea77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea815cd00 0x7f7ea80cbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48858/0 (socket says 172.21.15.47:48858) 2024-03-23T09:41:49.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ea77fe640 1 -- 172.21.15.47:0/949409190 learned_addr learned my addr 172.21.15.47:0/949409190 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:49.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ea67fc640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ea80d1020 0x7f7ea80cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ea6ffd640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ea80cf920 0x7f7ea81f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ea77fe640 1 -- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ea80cf920 msgr2=0x7f7ea81f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ea77fe640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ea80cf920 0x7f7ea81f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ea77fe640 1 -- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ea80d1020 msgr2=0x7f7ea80cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:49.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ea77fe640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ea80d1020 0x7f7ea80cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:49.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ea77fe640 1 -- 172.21.15.47:0/949409190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea00094d0 con 0x7f7ea815cd00 2024-03-23T09:41:49.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ea67fc640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ea80d1020 0x7f7ea80cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:49.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.956+0000 7f7ea77fe640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea815cd00 0x7f7ea80cbf80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e98007e30 tx=0x7f7e9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:49.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.957+0000 7f7e9f7fe640 1 -- 172.21.15.47:0/949409190 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98014420 con 0x7f7ea815cd00 2024-03-23T09:41:49.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.957+0000 7f7e9f7fe640 1 -- 172.21.15.47:0/949409190 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e980145a0 con 0x7f7ea815cd00 2024-03-23T09:41:49.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.957+0000 7f7ead7d5640 1 -- 172.21.15.47:0/949409190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea8150cc0 con 0x7f7ea815cd00 2024-03-23T09:41:49.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.957+0000 7f7e9f7fe640 1 -- 172.21.15.47:0/949409190 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98014950 con 0x7f7ea815cd00 2024-03-23T09:41:49.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.957+0000 7f7ead7d5640 1 -- 172.21.15.47:0/949409190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea81517a0 con 0x7f7ea815cd00 2024-03-23T09:41:49.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.958+0000 7f7ead7d5640 1 -- 172.21.15.47:0/949409190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea8072360 con 0x7f7ea815cd00 2024-03-23T09:41:49.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.958+0000 7f7e9f7fe640 1 -- 172.21.15.47:0/949409190 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7e98014ab0 con 0x7f7ea815cd00 2024-03-23T09:41:49.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.959+0000 7f7e9f7fe640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e8806ac30 0x7f7e8806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:49.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.959+0000 7f7e9f7fe640 1 -- 172.21.15.47:0/949409190 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e98089bb0 con 0x7f7ea815cd00 2024-03-23T09:41:49.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.959+0000 7f7ea6ffd640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e8806ac30 0x7f7e8806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:49.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.959+0000 7f7ea6ffd640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e8806ac30 0x7f7e8806d0f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0036000 tx=0x7f7ea003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:49.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:49.963+0000 7f7e9f7fe640 1 -- 172.21.15.47:0/949409190 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7e98054e20 con 0x7f7ea815cd00 2024-03-23T09:41:50.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.179+0000 7f7ead7d5640 1 -- 172.21.15.47:0/949409190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7ea8066be0 con 0x7f7ea815cd00 2024-03-23T09:41:50.179 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:41:50.179+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:41:50.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.187+0000 7f7e9f7fe640 1 -- 172.21.15.47:0/949409190 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7e98058440 con 0x7f7ea815cd00 2024-03-23T09:41:50.186 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:41:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.189+0000 7f7ead7d5640 1 -- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e8806ac30 msgr2=0x7f7e8806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.189+0000 7f7ead7d5640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e8806ac30 0x7f7e8806d0f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0036000 tx=0x7f7ea003c040 comp rx=0 tx=0).stop 2024-03-23T09:41:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.189+0000 7f7ead7d5640 1 -- 172.21.15.47:0/949409190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea815cd00 msgr2=0x7f7ea80cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.189+0000 7f7ead7d5640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea815cd00 0x7f7ea80cbf80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e98007e30 tx=0x7f7e9800e4b0 comp rx=0 tx=0).stop 2024-03-23T09:41:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.189+0000 7f7ead7d5640 1 -- 172.21.15.47:0/949409190 shutdown_connections 2024-03-23T09:41:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.189+0000 7f7ead7d5640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e8806ac30 0x7f7e8806d0f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.189+0000 7f7ead7d5640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea815cd00 0x7f7ea80cbf80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.189+0000 7f7ead7d5640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ea80d1020 0x7f7ea80cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.189+0000 7f7ead7d5640 1 --2- 172.21.15.47:0/949409190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ea80cf920 0x7f7ea81f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.189+0000 7f7ead7d5640 1 -- 172.21.15.47:0/949409190 >> 172.21.15.47:0/949409190 conn(0x7f7ea806b160 msgr2=0x7f7ea80b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.189+0000 7f7ead7d5640 1 -- 172.21.15.47:0/949409190 shutdown_connections 2024-03-23T09:41:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.189+0000 7f7ead7d5640 1 -- 172.21.15.47:0/949409190 wait complete. 2024-03-23T09:41:50.190 INFO:tasks.ceph.mds.d.smithi175.stderr: -2705> 2024-03-23T09:41:07.725+0000 7fb6071c6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:41:50.199 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T09:41:50.221 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-03-23T09:41:50.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.303+0000 7f40bd4c4640 1 Processor -- start 2024-03-23T09:41:50.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.303+0000 7f40bd4c4640 1 -- start start 2024-03-23T09:41:50.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40bd4c4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f40b815ce10 0x7f40b80cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:50.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40bd4c4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b8161600 con 0x7f40b815ca30 2024-03-23T09:41:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40bd4c4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b8161040 con 0x7f40b80b85d0 2024-03-23T09:41:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40bd4c4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b8160e50 con 0x7f40b815ce10 2024-03-23T09:41:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b6ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f40b815ca30 0x7f40b8160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48004/0 (socket says 172.21.15.47:48004) 2024-03-23T09:41:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b6ffd640 1 -- 172.21.15.47:0/1123529722 learned_addr learned my addr 172.21.15.47:0/1123529722 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b67fc640 1 --2- 172.21.15.47:0/1123529722 >> v2:172.21.15.175:3300/0 conn(0x7f40b815ce10 0x7f40b80cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976145983 0 0) 0x7f40b8161600 con 0x7f40b815ca30 2024-03-23T09:41:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40a4003520 con 0x7f40b815ca30 2024-03-23T09:41:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3177421462 0 0) 0x7f40a4003520 con 0x7f40b815ca30 2024-03-23T09:41:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 >> v2:172.21.15.175:3300/0 conn(0x7f40b815ce10 msgr2=0x7f40b80cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 --2- 172.21.15.47:0/1123529722 >> v2:172.21.15.175:3300/0 conn(0x7f40b815ce10 0x7f40b80cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 >> v1:172.21.15.175:6789/0 conn(0x7f40b80b85d0 legacy=0x7f40b80d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b8161320 con 0x7f40b815ca30 2024-03-23T09:41:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f40a80019f0 con 0x7f40b815ca30 2024-03-23T09:41:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 >> v1:172.21.15.47:6789/0 conn(0x7f40b815ca30 legacy=0x7f40b8160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 --2- 172.21.15.47:0/1123529722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a4003970 0x7f40a4003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 --2- 172.21.15.47:0/1123529722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a4004370 0x7f40a4004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 --2- 172.21.15.47:0/1123529722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4005040 0x7f40a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a400bf70 con 0x7f40a4004370 2024-03-23T09:41:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.304+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a400c0e0 con 0x7f40a4003970 2024-03-23T09:41:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.305+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a400c2a0 con 0x7f40a4005040 2024-03-23T09:41:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.305+0000 7f40b67fc640 1 --2- 172.21.15.47:0/1123529722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a4004370 0x7f40a4004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.305+0000 7f40b6ffd640 1 --2- 172.21.15.47:0/1123529722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a4003970 0x7f40a4003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.305+0000 7f40b77fe640 1 --2- 172.21.15.47:0/1123529722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4005040 0x7f40a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.305+0000 7f40b67fc640 1 -- 172.21.15.47:0/1123529722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4005040 msgr2=0x7f40a400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.305+0000 7f40b67fc640 1 --2- 172.21.15.47:0/1123529722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4005040 0x7f40a400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.305+0000 7f40b67fc640 1 -- 172.21.15.47:0/1123529722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a4003970 msgr2=0x7f40a4003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.305+0000 7f40b67fc640 1 --2- 172.21.15.47:0/1123529722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a4003970 0x7f40a4003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.305+0000 7f40b67fc640 1 -- 172.21.15.47:0/1123529722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4003520 con 0x7f40a4004370 2024-03-23T09:41:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.305+0000 7f40b77fe640 1 -- 172.21.15.47:0/1123529722 reap_dead start 2024-03-23T09:41:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.305+0000 7f40b6ffd640 1 --2- 172.21.15.47:0/1123529722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a4003970 0x7f40a4003d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.306+0000 7f40b67fc640 1 --2- 172.21.15.47:0/1123529722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a4004370 0x7f40a4004840 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009ac0 tx=0x7f40a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9eb8385427e078 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.306+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a002c6d0 con 0x7f40a4004370 2024-03-23T09:41:50.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.306+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a002c830 con 0x7f40a4004370 2024-03-23T09:41:50.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.306+0000 7f40b5ffb640 1 -- 172.21.15.47:0/1123529722 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a002cbc0 con 0x7f40a4004370 2024-03-23T09:41:50.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.306+0000 7f40bd4c4640 1 -- 172.21.15.47:0/1123529722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a4004370 msgr2=0x7f40a4004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:50.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.306+0000 7f40bd4c4640 1 --2- 172.21.15.47:0/1123529722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a4004370 0x7f40a4004840 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009ac0 tx=0x7f40a40057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:50.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.306+0000 7f40bd4c4640 1 -- 172.21.15.47:0/1123529722 shutdown_connections 2024-03-23T09:41:50.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.306+0000 7f40bd4c4640 1 --2- 172.21.15.47:0/1123529722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a4004370 0x7f40a4004840 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:50.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.306+0000 7f40bd4c4640 1 -- 172.21.15.47:0/1123529722 >> 172.21.15.47:0/1123529722 conn(0x7f40b806b160 msgr2=0x7f40b806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:50.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.306+0000 7f40bd4c4640 1 -- 172.21.15.47:0/1123529722 shutdown_connections 2024-03-23T09:41:50.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.306+0000 7f40bd4c4640 1 -- 172.21.15.47:0/1123529722 wait complete. 2024-03-23T09:41:50.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.308+0000 7f40bd4c4640 1 Processor -- start 2024-03-23T09:41:50.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.308+0000 7f40bd4c4640 1 -- start start 2024-03-23T09:41:50.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.308+0000 7f40bd4c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a4004370 0x7f40b81f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.308+0000 7f40bd4c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b815ef10 0x7f40b80caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.308+0000 7f40bd4c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b80cb000 0x7f40b81f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.308+0000 7f40bd4c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b80cb4d0 con 0x7f40b80cb000 2024-03-23T09:41:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.308+0000 7f40bd4c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b80cb7b0 con 0x7f40a4004370 2024-03-23T09:41:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.308+0000 7f40bd4c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b80cba90 con 0x7f40b815ef10 2024-03-23T09:41:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.308+0000 7f40b77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b80cb000 0x7f40b81f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.308+0000 7f40b77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b80cb000 0x7f40b81f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48876/0 (socket says 172.21.15.47:48876) 2024-03-23T09:41:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.308+0000 7f40b77fe640 1 -- 172.21.15.47:0/163843274 learned_addr learned my addr 172.21.15.47:0/163843274 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40b6ffd640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a4004370 0x7f40b81f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40b67fc640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b815ef10 0x7f40b80caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40b77fe640 1 -- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b815ef10 msgr2=0x7f40b80caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40b77fe640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b815ef10 0x7f40b80caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40b77fe640 1 -- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a4004370 msgr2=0x7f40b81f4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40b77fe640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a4004370 0x7f40b81f4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40b77fe640 1 -- 172.21.15.47:0/163843274 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b80cbd70 con 0x7f40b80cb000 2024-03-23T09:41:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40b67fc640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b815ef10 0x7f40b80caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40b6ffd640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a4004370 0x7f40b81f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40b77fe640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b80cb000 0x7f40b81f23a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f40b80d39a0 tx=0x7f40b000e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40af7fe640 1 -- 172.21.15.47:0/163843274 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b00052e0 con 0x7f40b80cb000 2024-03-23T09:41:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40af7fe640 1 -- 172.21.15.47:0/163843274 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b0005440 con 0x7f40b80cb000 2024-03-23T09:41:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40bd4c4640 1 -- 172.21.15.47:0/163843274 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b80cc050 con 0x7f40b80cb000 2024-03-23T09:41:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40af7fe640 1 -- 172.21.15.47:0/163843274 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0003810 con 0x7f40b80cb000 2024-03-23T09:41:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.309+0000 7f40bd4c4640 1 -- 172.21.15.47:0/163843274 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b81f28e0 con 0x7f40b80cb000 2024-03-23T09:41:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.310+0000 7f40bd4c4640 1 -- 172.21.15.47:0/163843274 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b8072360 con 0x7f40b80cb000 2024-03-23T09:41:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.310+0000 7f40af7fe640 1 -- 172.21.15.47:0/163843274 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f40b0003970 con 0x7f40b80cb000 2024-03-23T09:41:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.311+0000 7f40af7fe640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806ac00 0x7f409806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.311+0000 7f40af7fe640 1 -- 172.21.15.47:0/163843274 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f40b0088ba0 con 0x7f40b80cb000 2024-03-23T09:41:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.311+0000 7f40b6ffd640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806ac00 0x7f409806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.312+0000 7f40b6ffd640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806ac00 0x7f409806d0c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f40a8002030 tx=0x7f40a8001250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.315+0000 7f40af7fe640 1 -- 172.21.15.47:0/163843274 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f40b0053d20 con 0x7f40b80cb000 2024-03-23T09:41:50.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:50.535+0000 7f40bd4c4640 1 -- 172.21.15.47:0/163843274 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f40b80d0380 con 0x7f40b80cb000 2024-03-23T09:41:51.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.192+0000 7f40af7fe640 1 -- 172.21.15.47:0/163843274 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f40b00190a0 con 0x7f40b80cb000 2024-03-23T09:41:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.195+0000 7f40bd4c4640 1 -- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806ac00 msgr2=0x7f409806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.195+0000 7f40bd4c4640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806ac00 0x7f409806d0c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f40a8002030 tx=0x7f40a8001250 comp rx=0 tx=0).stop 2024-03-23T09:41:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.195+0000 7f40bd4c4640 1 -- 172.21.15.47:0/163843274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b80cb000 msgr2=0x7f40b81f23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.195+0000 7f40bd4c4640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b80cb000 0x7f40b81f23a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f40b80d39a0 tx=0x7f40b000e820 comp rx=0 tx=0).stop 2024-03-23T09:41:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.195+0000 7f40bd4c4640 1 -- 172.21.15.47:0/163843274 shutdown_connections 2024-03-23T09:41:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.195+0000 7f40bd4c4640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806ac00 0x7f409806d0c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.195+0000 7f40bd4c4640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b80cb000 0x7f40b81f23a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.195+0000 7f40bd4c4640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b815ef10 0x7f40b80caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.195+0000 7f40bd4c4640 1 --2- 172.21.15.47:0/163843274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a4004370 0x7f40b81f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.195+0000 7f40bd4c4640 1 -- 172.21.15.47:0/163843274 >> 172.21.15.47:0/163843274 conn(0x7f40b806b160 msgr2=0x7f40b8156ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.195+0000 7f40bd4c4640 1 -- 172.21.15.47:0/163843274 shutdown_connections 2024-03-23T09:41:51.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.195+0000 7f40bd4c4640 1 -- 172.21.15.47:0/163843274 wait complete. 2024-03-23T09:41:51.208 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T09:41:51.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.317+0000 7fed07120640 1 Processor -- start 2024-03-23T09:41:51.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.317+0000 7fed07120640 1 -- start start 2024-03-23T09:41:51.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fed07120640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fed0015cb50 0x7fed000b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:51.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fed07120640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fed00160780 0x7fed000cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:51.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fed07120640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed000ce0c0 con 0x7fed000cdc70 2024-03-23T09:41:51.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fed07120640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed00161790 con 0x7fed0015cb50 2024-03-23T09:41:51.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fed07120640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed000b5a90 con 0x7fed00160780 2024-03-23T09:41:51.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fed04ebc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fed0015cb50 0x7fed000b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:51.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fed04ebc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fed0015cb50 0x7fed000b84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48888/0 (socket says 172.21.15.47:48888) 2024-03-23T09:41:51.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fed04ebc640 1 -- 172.21.15.47:0/3671142269 learned_addr learned my addr 172.21.15.47:0/3671142269 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:51.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fecfffff640 1 --2- 172.21.15.47:0/3671142269 >> v2:172.21.15.175:3300/0 conn(0x7fed00160780 0x7fed000cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:51.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fed04ebc640 1 -- 172.21.15.47:0/3671142269 >> v2:172.21.15.175:3300/0 conn(0x7fed00160780 msgr2=0x7fed000cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:51.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fed04ebc640 1 --2- 172.21.15.47:0/3671142269 >> v2:172.21.15.175:3300/0 conn(0x7fed00160780 0x7fed000cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:51.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fed04ebc640 1 -- 172.21.15.47:0/3671142269 >> v1:172.21.15.175:6789/0 conn(0x7fed000cdc70 legacy=0x7fed000b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:51.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fed04ebc640 1 -- 172.21.15.47:0/3671142269 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed00160ef0 con 0x7fed0015cb50 2024-03-23T09:41:51.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.318+0000 7fecfffff640 1 --2- 172.21.15.47:0/3671142269 >> v2:172.21.15.175:3300/0 conn(0x7fed00160780 0x7fed000cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:51.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.319+0000 7fed04ebc640 1 --2- 172.21.15.47:0/3671142269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0015cb50 0x7fed000b84a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fed000d39a0 tx=0x7fecf802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9348b49ec815c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:51.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.319+0000 7fecff7fe640 1 -- 172.21.15.47:0/3671142269 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf802ca80 con 0x7fed0015cb50 2024-03-23T09:41:51.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.319+0000 7fecff7fe640 1 -- 172.21.15.47:0/3671142269 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf802cbe0 con 0x7fed0015cb50 2024-03-23T09:41:51.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.319+0000 7fecff7fe640 1 -- 172.21.15.47:0/3671142269 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf80343f0 con 0x7fed0015cb50 2024-03-23T09:41:51.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.320+0000 7fed07120640 1 -- 172.21.15.47:0/3671142269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0015cb50 msgr2=0x7fed000b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:51.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.320+0000 7fed07120640 1 --2- 172.21.15.47:0/3671142269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0015cb50 0x7fed000b84a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fed000d39a0 tx=0x7fecf802bee0 comp rx=0 tx=0).stop 2024-03-23T09:41:51.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.320+0000 7fed07120640 1 -- 172.21.15.47:0/3671142269 shutdown_connections 2024-03-23T09:41:51.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.320+0000 7fed07120640 1 --2- 172.21.15.47:0/3671142269 >> v2:172.21.15.175:3300/0 conn(0x7fed00160780 0x7fed000cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:51.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.320+0000 7fed07120640 1 --2- 172.21.15.47:0/3671142269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0015cb50 0x7fed000b84a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:51.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.320+0000 7fed07120640 1 -- 172.21.15.47:0/3671142269 >> 172.21.15.47:0/3671142269 conn(0x7fed0006b160 msgr2=0x7fed0006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:51.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.320+0000 7fed07120640 1 -- 172.21.15.47:0/3671142269 shutdown_connections 2024-03-23T09:41:51.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.320+0000 7fed07120640 1 -- 172.21.15.47:0/3671142269 wait complete. 2024-03-23T09:41:51.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.321+0000 7fed07120640 1 Processor -- start 2024-03-23T09:41:51.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.321+0000 7fed07120640 1 -- start start 2024-03-23T09:41:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed07120640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed000cdc70 0x7fed0014be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed07120640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed0015cb50 0x7fed001f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed07120640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed00160780 0x7fed00151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed07120640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0014efe0 con 0x7fed00160780 2024-03-23T09:41:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed07120640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0014e3d0 con 0x7fed000cdc70 2024-03-23T09:41:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed07120640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0014e6b0 con 0x7fed0015cb50 2024-03-23T09:41:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed056bd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed00160780 0x7fed00151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed056bd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed00160780 0x7fed00151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48890/0 (socket says 172.21.15.47:48890) 2024-03-23T09:41:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed056bd640 1 -- 172.21.15.47:0/3607540169 learned_addr learned my addr 172.21.15.47:0/3607540169 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed04ebc640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed000cdc70 0x7fed0014be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fecfffff640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed0015cb50 0x7fed001f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed056bd640 1 -- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed0015cb50 msgr2=0x7fed001f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed056bd640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed0015cb50 0x7fed001f3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:51.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed056bd640 1 -- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed000cdc70 msgr2=0x7fed0014be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:51.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed056bd640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed000cdc70 0x7fed0014be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:51.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed056bd640 1 -- 172.21.15.47:0/3607540169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf8009b30 con 0x7fed00160780 2024-03-23T09:41:51.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.322+0000 7fed056bd640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed00160780 0x7fed00151010 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fecf000b410 tx=0x7fecf000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:51.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.323+0000 7fecfd7fa640 1 -- 172.21.15.47:0/3607540169 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0004020 con 0x7fed00160780 2024-03-23T09:41:51.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.323+0000 7fecfd7fa640 1 -- 172.21.15.47:0/3607540169 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf0004180 con 0x7fed00160780 2024-03-23T09:41:51.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.323+0000 7fed07120640 1 -- 172.21.15.47:0/3607540169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0014e990 con 0x7fed00160780 2024-03-23T09:41:51.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.323+0000 7fecfd7fa640 1 -- 172.21.15.47:0/3607540169 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0011630 con 0x7fed00160780 2024-03-23T09:41:51.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.323+0000 7fed07120640 1 -- 172.21.15.47:0/3607540169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0014c340 con 0x7fed00160780 2024-03-23T09:41:51.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.324+0000 7fed07120640 1 -- 172.21.15.47:0/3607540169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed00072360 con 0x7fed00160780 2024-03-23T09:41:51.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.324+0000 7fecfd7fa640 1 -- 172.21.15.47:0/3607540169 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fecf0011790 con 0x7fed00160780 2024-03-23T09:41:51.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.325+0000 7fecfd7fa640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fecec06ac00 0x7fecec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:51.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.325+0000 7fecfd7fa640 1 -- 172.21.15.47:0/3607540169 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5890+0+0 (secure 0 0 0) 0x7fecf0088fe0 con 0x7fed00160780 2024-03-23T09:41:51.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.325+0000 7fed04ebc640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fecec06ac00 0x7fecec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:51.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.325+0000 7fed04ebc640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fecec06ac00 0x7fecec06d0c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fecf8035000 tx=0x7fecf8030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:51.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.329+0000 7fecfd7fa640 1 -- 172.21.15.47:0/3607540169 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fecf00541b0 con 0x7fed00160780 2024-03-23T09:41:51.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:51.506+0000 7fed07120640 1 -- 172.21.15.47:0/3607540169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fed00066be0 con 0x7fed00160780 2024-03-23T09:41:52.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.193+0000 7fecfd7fa640 1 -- 172.21.15.47:0/3607540169 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7fecf00577d0 con 0x7fed00160780 2024-03-23T09:41:52.193 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-03-23T09:41:52.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.197+0000 7fed07120640 1 -- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fecec06ac00 msgr2=0x7fecec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:52.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.197+0000 7fed07120640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fecec06ac00 0x7fecec06d0c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fecf8035000 tx=0x7fecf8030000 comp rx=0 tx=0).stop 2024-03-23T09:41:52.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.197+0000 7fed07120640 1 -- 172.21.15.47:0/3607540169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed00160780 msgr2=0x7fed00151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:52.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.197+0000 7fed07120640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed00160780 0x7fed00151010 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fecf000b410 tx=0x7fecf000b8d0 comp rx=0 tx=0).stop 2024-03-23T09:41:52.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.197+0000 7fed07120640 1 -- 172.21.15.47:0/3607540169 shutdown_connections 2024-03-23T09:41:52.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.197+0000 7fed07120640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fecec06ac00 0x7fecec06d0c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.197+0000 7fed07120640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed00160780 0x7fed00151010 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.197+0000 7fed07120640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed0015cb50 0x7fed001f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.197+0000 7fed07120640 1 --2- 172.21.15.47:0/3607540169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed000cdc70 0x7fed0014be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.197+0000 7fed07120640 1 -- 172.21.15.47:0/3607540169 >> 172.21.15.47:0/3607540169 conn(0x7fed0006b160 msgr2=0x7fed000b66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.197+0000 7fed07120640 1 -- 172.21.15.47:0/3607540169 shutdown_connections 2024-03-23T09:41:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.197+0000 7fed07120640 1 -- 172.21.15.47:0/3607540169 wait complete. 2024-03-23T09:41:52.211 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T09:41:52.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.328+0000 7f6e82d69640 1 Processor -- start 2024-03-23T09:41:52.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.328+0000 7f6e82d69640 1 -- start start 2024-03-23T09:41:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.329+0000 7f6e82d69640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c15a860 0x7f6e7c161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.329+0000 7f6e82d69640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6e7c0cf8a0 0x7f6e7c15d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.329+0000 7f6e82d69640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c160bd0 con 0x7f6e7c15d0f0 2024-03-23T09:41:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.329+0000 7f6e82d69640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c0d2790 con 0x7f6e7c0cf8a0 2024-03-23T09:41:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.329+0000 7f6e82d69640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c15dbb0 con 0x7f6e7c15a860 2024-03-23T09:41:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.329+0000 7f6e81306640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6e7c0cf8a0 0x7f6e7c15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.329+0000 7f6e81306640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6e7c0cf8a0 0x7f6e7c15d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48898/0 (socket says 172.21.15.47:48898) 2024-03-23T09:41:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.329+0000 7f6e81306640 1 -- 172.21.15.47:0/3193611500 learned_addr learned my addr 172.21.15.47:0/3193611500 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.330+0000 7f6e81306640 1 -- 172.21.15.47:0/3193611500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c15a860 msgr2=0x7f6e7c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.330+0000 7f6e81306640 1 --2- 172.21.15.47:0/3193611500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c15a860 0x7f6e7c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.330+0000 7f6e81306640 1 -- 172.21.15.47:0/3193611500 >> v1:172.21.15.175:6789/0 conn(0x7f6e7c15d0f0 legacy=0x7f6e7c0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.330+0000 7f6e81306640 1 -- 172.21.15.47:0/3193611500 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c15de90 con 0x7f6e7c0cf8a0 2024-03-23T09:41:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.330+0000 7f6e81306640 1 --2- 172.21.15.47:0/3193611500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7c0cf8a0 0x7f6e7c15d550 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009870 tx=0x7f6e6c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e749a0f388b3a58 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.331+0000 7f6e7b7fe640 1 -- 172.21.15.47:0/3193611500 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c031640 con 0x7f6e7c0cf8a0 2024-03-23T09:41:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.331+0000 7f6e7b7fe640 1 -- 172.21.15.47:0/3193611500 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6c0317a0 con 0x7f6e7c0cf8a0 2024-03-23T09:41:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.331+0000 7f6e7b7fe640 1 -- 172.21.15.47:0/3193611500 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c031b30 con 0x7f6e7c0cf8a0 2024-03-23T09:41:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.331+0000 7f6e82d69640 1 -- 172.21.15.47:0/3193611500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7c0cf8a0 msgr2=0x7f6e7c15d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.331+0000 7f6e82d69640 1 --2- 172.21.15.47:0/3193611500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7c0cf8a0 0x7f6e7c15d550 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009870 tx=0x7f6e6c02b8f0 comp rx=0 tx=0).stop 2024-03-23T09:41:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.331+0000 7f6e82d69640 1 -- 172.21.15.47:0/3193611500 shutdown_connections 2024-03-23T09:41:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.331+0000 7f6e82d69640 1 --2- 172.21.15.47:0/3193611500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7c0cf8a0 0x7f6e7c15d550 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.331+0000 7f6e82d69640 1 --2- 172.21.15.47:0/3193611500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c15a860 0x7f6e7c161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.331+0000 7f6e82d69640 1 -- 172.21.15.47:0/3193611500 >> 172.21.15.47:0/3193611500 conn(0x7f6e7c0659d0 msgr2=0x7f6e7c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.331+0000 7f6e82d69640 1 -- 172.21.15.47:0/3193611500 shutdown_connections 2024-03-23T09:41:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.331+0000 7f6e82d69640 1 -- 172.21.15.47:0/3193611500 wait complete. 2024-03-23T09:41:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.332+0000 7f6e82d69640 1 Processor -- start 2024-03-23T09:41:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.332+0000 7f6e82d69640 1 -- start start 2024-03-23T09:41:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.333+0000 7f6e82d69640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c0cf8a0 0x7f6e7c0c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.333+0000 7f6e82d69640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7c15a860 0x7f6e7c0c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.333+0000 7f6e82d69640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e7c15d0f0 0x7f6e7c0c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.333+0000 7f6e82d69640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c1f2250 con 0x7f6e7c15a860 2024-03-23T09:41:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.333+0000 7f6e82d69640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0b75f0 con 0x7f6e7c15d0f0 2024-03-23T09:41:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.333+0000 7f6e82d69640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c152de0 con 0x7f6e7c0cf8a0 2024-03-23T09:41:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.333+0000 7f6e7bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7c15a860 0x7f6e7c0c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.333+0000 7f6e7bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7c15a860 0x7f6e7c0c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48914/0 (socket says 172.21.15.47:48914) 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.333+0000 7f6e7bfff640 1 -- 172.21.15.47:0/4009803247 learned_addr learned my addr 172.21.15.47:0/4009803247 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e81306640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e7c15d0f0 0x7f6e7c0c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e80b05640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c0cf8a0 0x7f6e7c0c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e7bfff640 1 -- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c0cf8a0 msgr2=0x7f6e7c0c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e7bfff640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c0cf8a0 0x7f6e7c0c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e7bfff640 1 -- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e7c15d0f0 msgr2=0x7f6e7c0c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e7bfff640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e7c15d0f0 0x7f6e7c0c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e7bfff640 1 -- 172.21.15.47:0/4009803247 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c0094d0 con 0x7f6e7c15a860 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e80b05640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c0cf8a0 0x7f6e7c0c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e81306640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e7c15d0f0 0x7f6e7c0c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e7bfff640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7c15a860 0x7f6e7c0c6d70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000b420 tx=0x7f6e7000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e797fa640 1 -- 172.21.15.47:0/4009803247 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70011830 con 0x7f6e7c15a860 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e797fa640 1 -- 172.21.15.47:0/4009803247 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e70011990 con 0x7f6e7c15a860 2024-03-23T09:41:52.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e82d69640 1 -- 172.21.15.47:0/4009803247 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c1f2460 con 0x7f6e7c15a860 2024-03-23T09:41:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e797fa640 1 -- 172.21.15.47:0/4009803247 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70011d40 con 0x7f6e7c15a860 2024-03-23T09:41:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.334+0000 7f6e82d69640 1 -- 172.21.15.47:0/4009803247 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c1f2a20 con 0x7f6e7c15a860 2024-03-23T09:41:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.335+0000 7f6e82d69640 1 -- 172.21.15.47:0/4009803247 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c072360 con 0x7f6e7c15a860 2024-03-23T09:41:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.336+0000 7f6e797fa640 1 -- 172.21.15.47:0/4009803247 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6e7001b460 con 0x7f6e7c15a860 2024-03-23T09:41:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.336+0000 7f6e797fa640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e6806ac00 0x7f6e6806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.336+0000 7f6e797fa640 1 -- 172.21.15.47:0/4009803247 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5452+0+0 (secure 0 0 0) 0x7f6e70088e10 con 0x7f6e7c15a860 2024-03-23T09:41:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.336+0000 7f6e80b05640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e6806ac00 0x7f6e6806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.337+0000 7f6e80b05640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e6806ac00 0x7f6e6806d0c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009fa0 tx=0x7f6e74009570 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.340+0000 7f6e797fa640 1 -- 172.21.15.47:0/4009803247 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6e700540f0 con 0x7f6e7c15a860 2024-03-23T09:41:52.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:52.521+0000 7f6e82d69640 1 -- 172.21.15.47:0/4009803247 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e7c1f2c40 con 0x7f6e7c15a860 2024-03-23T09:41:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-03-23T09:41:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.200+0000 7f6e797fa640 1 -- 172.21.15.47:0/4009803247 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6e70057710 con 0x7f6e7c15a860 2024-03-23T09:41:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.204+0000 7f6e82d69640 1 -- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e6806ac00 msgr2=0x7f6e6806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.204+0000 7f6e82d69640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e6806ac00 0x7f6e6806d0c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009fa0 tx=0x7f6e74009570 comp rx=0 tx=0).stop 2024-03-23T09:41:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.204+0000 7f6e82d69640 1 -- 172.21.15.47:0/4009803247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7c15a860 msgr2=0x7f6e7c0c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.204+0000 7f6e82d69640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7c15a860 0x7f6e7c0c6d70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000b420 tx=0x7f6e7000b8e0 comp rx=0 tx=0).stop 2024-03-23T09:41:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.204+0000 7f6e82d69640 1 -- 172.21.15.47:0/4009803247 shutdown_connections 2024-03-23T09:41:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.204+0000 7f6e82d69640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e6806ac00 0x7f6e6806d0c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.204+0000 7f6e82d69640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e7c15d0f0 0x7f6e7c0c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.204+0000 7f6e82d69640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7c15a860 0x7f6e7c0c6d70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.204+0000 7f6e82d69640 1 --2- 172.21.15.47:0/4009803247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c0cf8a0 0x7f6e7c0c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.204+0000 7f6e82d69640 1 -- 172.21.15.47:0/4009803247 >> 172.21.15.47:0/4009803247 conn(0x7f6e7c0659d0 msgr2=0x7f6e7c0d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.204+0000 7f6e82d69640 1 -- 172.21.15.47:0/4009803247 shutdown_connections 2024-03-23T09:41:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.204+0000 7f6e82d69640 1 -- 172.21.15.47:0/4009803247 wait complete. 2024-03-23T09:41:53.219 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-23T09:41:53.219 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-23T09:41:53.219 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-23T09:41:53.219 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:41:53.219 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-23T09:41:53.219 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-23T09:41:53.220 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-23T09:41:53.220 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:41:53.220 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-23T09:41:53.220 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-23T09:41:53.220 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-23T09:41:53.220 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:41:53.220 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-23T09:41:53.220 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-23T09:41:53.221 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-23T09:41:53.221 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:41:53.232 INFO:tasks.ceph.mds.b:Stopped 2024-03-23T09:41:53.232 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-23T09:41:53.233 INFO:tasks.ceph.mds.d:Stopped 2024-03-23T09:41:53.233 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-23T09:41:53.234 INFO:tasks.ceph.mds.a:Stopped 2024-03-23T09:41:53.234 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-23T09:41:53.235 INFO:tasks.ceph.mds.c:Stopped 2024-03-23T09:41:53.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-23T09:41:53.275 INFO:tasks.ceph.mds.b:Started 2024-03-23T09:41:53.275 DEBUG:teuthology.parallel:result is None 2024-03-23T09:41:53.276 INFO:tasks.ceph.mds.d:Started 2024-03-23T09:41:53.276 DEBUG:teuthology.parallel:result is None 2024-03-23T09:41:53.277 INFO:tasks.ceph.mds.a:Started 2024-03-23T09:41:53.277 DEBUG:teuthology.parallel:result is None 2024-03-23T09:41:53.278 INFO:tasks.ceph.mds.c:Started 2024-03-23T09:41:53.278 DEBUG:teuthology.parallel:result is None 2024-03-23T09:41:53.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-23T09:41:53.355 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-03-23T09:41:53.355 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-03-23T09:41:53.362 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-03-23T09:41:53.362 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:41:53.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.397+0000 7f5fe26c0640 1 Processor -- start 2024-03-23T09:41:53.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.397+0000 7f5fe26c0640 1 -- start start 2024-03-23T09:41:53.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fe26c0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fdc0cbdf0 0x7f5fdc098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fe26c0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5fdc099740 0x7f5fdc098f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fe26c0640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fdc0ce8f0 con 0x7f5fdc0b10f0 2024-03-23T09:41:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fe26c0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fdc0cc300 con 0x7f5fdc099740 2024-03-23T09:41:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fe26c0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc0cc5c0 con 0x7f5fdc0cbdf0 2024-03-23T09:41:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fdb7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fdc0cbdf0 0x7f5fdc098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fe0c5d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5fdc099740 0x7f5fdc098f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fdbfff640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f5fdc0b10f0 0x7f5fdc0affe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:54574/0 (socket says 172.21.15.47:54574) 2024-03-23T09:41:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fdb7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fdc0cbdf0 0x7f5fdc098a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:37854/0 (socket says 172.21.15.47:37854) 2024-03-23T09:41:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fdbfff640 1 -- 172.21.15.47:0/243319087 learned_addr learned my addr 172.21.15.47:0/243319087 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fe0c5d640 1 -- 172.21.15.47:0/243319087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fdc0cbdf0 msgr2=0x7f5fdc098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fe0c5d640 1 --2- 172.21.15.47:0/243319087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fdc0cbdf0 0x7f5fdc098a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fe0c5d640 1 -- 172.21.15.47:0/243319087 >> v1:172.21.15.175:6789/0 conn(0x7f5fdc0b10f0 legacy=0x7f5fdc0affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fe0c5d640 1 -- 172.21.15.47:0/243319087 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0ce2a0 con 0x7f5fdc099740 2024-03-23T09:41:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fdaffd640 1 -- 172.21.15.47:0/243319087 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674457698 0 0) 0x7f5fdc0ce8f0 con 0x7f5fdc0b10f0 2024-03-23T09:41:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.398+0000 7f5fdb7fe640 1 --2- 172.21.15.47:0/243319087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fdc0cbdf0 0x7f5fdc098a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.399+0000 7f5fe0c5d640 1 --2- 172.21.15.47:0/243319087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc099740 0x7f5fdc098f40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009d90 tx=0x7f5fd402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=170ba2119d4c26d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.399+0000 7f5fdaffd640 1 -- 172.21.15.47:0/243319087 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd402cbc0 con 0x7f5fdc099740 2024-03-23T09:41:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.399+0000 7f5fdaffd640 1 -- 172.21.15.47:0/243319087 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd402cd20 con 0x7f5fdc099740 2024-03-23T09:41:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.399+0000 7f5fdaffd640 1 -- 172.21.15.47:0/243319087 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4034660 con 0x7f5fdc099740 2024-03-23T09:41:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.400+0000 7f5fe26c0640 1 -- 172.21.15.47:0/243319087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc099740 msgr2=0x7f5fdc098f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:53.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.400+0000 7f5fe26c0640 1 --2- 172.21.15.47:0/243319087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc099740 0x7f5fdc098f40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009d90 tx=0x7f5fd402be10 comp rx=0 tx=0).stop 2024-03-23T09:41:53.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.400+0000 7f5fe26c0640 1 -- 172.21.15.47:0/243319087 shutdown_connections 2024-03-23T09:41:53.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.400+0000 7f5fe26c0640 1 --2- 172.21.15.47:0/243319087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc099740 0x7f5fdc098f40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:53.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.400+0000 7f5fe26c0640 1 --2- 172.21.15.47:0/243319087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fdc0cbdf0 0x7f5fdc098a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:53.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.400+0000 7f5fe26c0640 1 -- 172.21.15.47:0/243319087 >> 172.21.15.47:0/243319087 conn(0x7f5fdc0b21c0 msgr2=0x7f5fdc0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:53.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.400+0000 7f5fe26c0640 1 -- 172.21.15.47:0/243319087 shutdown_connections 2024-03-23T09:41:53.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.400+0000 7f5fe26c0640 1 -- 172.21.15.47:0/243319087 wait complete. 2024-03-23T09:41:53.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.401+0000 7f5fe26c0640 1 Processor -- start 2024-03-23T09:41:53.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.401+0000 7f5fe26c0640 1 -- start start 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.402+0000 7f5fe26c0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fdc099740 0x7f5fdc09a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.402+0000 7f5fe26c0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc0b10f0 0x7f5fdc0c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.402+0000 7f5fe26c0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fdc0cbdf0 0x7f5fdc1dfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.402+0000 7f5fe26c0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc1e00f0 con 0x7f5fdc099740 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.402+0000 7f5fe26c0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc0b46d0 con 0x7f5fdc0b10f0 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.402+0000 7f5fe26c0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc0b4920 con 0x7f5fdc0cbdf0 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.402+0000 7f5fdbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fdc099740 0x7f5fdc09a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.402+0000 7f5fdbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fdc099740 0x7f5fdc09a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48950/0 (socket says 172.21.15.47:48950) 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.402+0000 7f5fdbfff640 1 -- 172.21.15.47:0/753358125 learned_addr learned my addr 172.21.15.47:0/753358125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.402+0000 7f5fdb7fe640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc0b10f0 0x7f5fdc0c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.402+0000 7f5fe0c5d640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fdc0cbdf0 0x7f5fdc1dfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.403+0000 7f5fdbfff640 1 -- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fdc0cbdf0 msgr2=0x7f5fdc1dfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.403+0000 7f5fdbfff640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fdc0cbdf0 0x7f5fdc1dfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.403+0000 7f5fdbfff640 1 -- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc0b10f0 msgr2=0x7f5fdc0c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:53.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.403+0000 7f5fdbfff640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc0b10f0 0x7f5fdc0c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:53.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.403+0000 7f5fdbfff640 1 -- 172.21.15.47:0/753358125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc009400 con 0x7f5fdc099740 2024-03-23T09:41:53.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.403+0000 7f5fdbfff640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fdc099740 0x7f5fdc09a3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000c4d0 tx=0x7f5fd000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:53.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.403+0000 7f5fd8ff9640 1 -- 172.21.15.47:0/753358125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00074e0 con 0x7f5fdc099740 2024-03-23T09:41:53.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.403+0000 7f5fd8ff9640 1 -- 172.21.15.47:0/753358125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd0007660 con 0x7f5fdc099740 2024-03-23T09:41:53.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.403+0000 7f5fd8ff9640 1 -- 172.21.15.47:0/753358125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0004a20 con 0x7f5fdc099740 2024-03-23T09:41:53.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.403+0000 7f5fe26c0640 1 -- 172.21.15.47:0/753358125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd4009a70 con 0x7f5fdc099740 2024-03-23T09:41:53.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.403+0000 7f5fe26c0640 1 -- 172.21.15.47:0/753358125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fdc0b4ee0 con 0x7f5fdc099740 2024-03-23T09:41:53.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.404+0000 7f5fe26c0640 1 -- 172.21.15.47:0/753358125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fdc0b0670 con 0x7f5fdc099740 2024-03-23T09:41:53.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.404+0000 7f5fd8ff9640 1 -- 172.21.15.47:0/753358125 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5fd0004b80 con 0x7f5fdc099740 2024-03-23T09:41:53.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.405+0000 7f5fd8ff9640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5fc806ac30 0x7f5fc806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:53.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.405+0000 7f5fd8ff9640 1 -- 172.21.15.47:0/753358125 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5fd0088190 con 0x7f5fdc099740 2024-03-23T09:41:53.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.405+0000 7f5fdb7fe640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5fc806ac30 0x7f5fc806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:53.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.406+0000 7f5fdb7fe640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5fc806ac30 0x7f5fc806d0f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0703f0 tx=0x7f5fcc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:53.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.409+0000 7f5fd8ff9640 1 -- 172.21.15.47:0/753358125 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5fd0053720 con 0x7f5fdc099740 2024-03-23T09:41:53.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:53.603+0000 7f5fe26c0640 1 -- 172.21.15.47:0/753358125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5fdc066be0 con 0x7f5fdc099740 2024-03-23T09:41:54.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.392+0000 7f5fd8ff9640 1 -- 172.21.15.47:0/753358125 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5fd0056d40 con 0x7f5fdc099740 2024-03-23T09:41:54.392 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blocklist entries 2024-03-23T09:41:54.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.395+0000 7f5fe26c0640 1 -- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5fc806ac30 msgr2=0x7f5fc806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:54.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.395+0000 7f5fe26c0640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5fc806ac30 0x7f5fc806d0f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0703f0 tx=0x7f5fcc009190 comp rx=0 tx=0).stop 2024-03-23T09:41:54.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.395+0000 7f5fe26c0640 1 -- 172.21.15.47:0/753358125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fdc099740 msgr2=0x7f5fdc09a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:54.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.395+0000 7f5fe26c0640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fdc099740 0x7f5fdc09a3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000c4d0 tx=0x7f5fd000c990 comp rx=0 tx=0).stop 2024-03-23T09:41:54.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.395+0000 7f5fe26c0640 1 -- 172.21.15.47:0/753358125 shutdown_connections 2024-03-23T09:41:54.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.395+0000 7f5fe26c0640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5fc806ac30 0x7f5fc806d0f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.395+0000 7f5fe26c0640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fdc0cbdf0 0x7f5fdc1dfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.395+0000 7f5fe26c0640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc0b10f0 0x7f5fdc0c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.395+0000 7f5fe26c0640 1 --2- 172.21.15.47:0/753358125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fdc099740 0x7f5fdc09a3e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.395+0000 7f5fe26c0640 1 -- 172.21.15.47:0/753358125 >> 172.21.15.47:0/753358125 conn(0x7f5fdc0b21c0 msgr2=0x7f5fdc0ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:54.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.395+0000 7f5fe26c0640 1 -- 172.21.15.47:0/753358125 shutdown_connections 2024-03-23T09:41:54.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.395+0000 7f5fe26c0640 1 -- 172.21.15.47:0/753358125 wait complete. 2024-03-23T09:41:54.408 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-23T09:41:54.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.516+0000 7f718ace9640 1 Processor -- start 2024-03-23T09:41:54.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.516+0000 7f718ace9640 1 -- start start 2024-03-23T09:41:54.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.517+0000 7f718ace9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f71840cf690 0x7f71840b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.517+0000 7f718ace9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718415d0e0 con 0x7f718415cd00 2024-03-23T09:41:54.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.517+0000 7f718ace9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71840d1540 con 0x7f71840cf210 2024-03-23T09:41:54.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.517+0000 7f718ace9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71840d1820 con 0x7f71840cf690 2024-03-23T09:41:54.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.517+0000 7f7188a85640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f718415cd00 0x7f71840d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48018/0 (socket says 172.21.15.47:48018) 2024-03-23T09:41:54.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.517+0000 7f7188a85640 1 -- 172.21.15.47:0/4288934991 learned_addr learned my addr 172.21.15.47:0/4288934991 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:54.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.517+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906539465 0 0) 0x7f718415d0e0 con 0x7f718415cd00 2024-03-23T09:41:54.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.517+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7170003520 con 0x7f718415cd00 2024-03-23T09:41:54.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.517+0000 7f7189286640 1 --2- 172.21.15.47:0/4288934991 >> v2:172.21.15.175:3300/0 conn(0x7f71840cf690 0x7f71840b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 64347426 0 0) 0x7f7170003520 con 0x7f718415cd00 2024-03-23T09:41:54.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 >> v2:172.21.15.175:3300/0 conn(0x7f71840cf690 msgr2=0x7f71840b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:54.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f71837fe640 1 --2- 172.21.15.47:0/4288934991 >> v2:172.21.15.175:3300/0 conn(0x7f71840cf690 0x7f71840b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 >> v1:172.21.15.175:6789/0 conn(0x7f71840cf210 legacy=0x7f71840b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:54.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71840d1b00 con 0x7f718415cd00 2024-03-23T09:41:54.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f717c0019f0 con 0x7f718415cd00 2024-03-23T09:41:54.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 >> v1:172.21.15.47:6789/0 conn(0x7f718415cd00 legacy=0x7f71840d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f71837fe640 1 --2- 172.21.15.47:0/4288934991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7170003970 0x7f7170003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f71837fe640 1 --2- 172.21.15.47:0/4288934991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71700047c0 0x7f7170004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f71837fe640 1 --2- 172.21.15.47:0/4288934991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7170005180 0x7f717000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71840d1820 con 0x7f7170005180 2024-03-23T09:41:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717000c120 con 0x7f71700047c0 2024-03-23T09:41:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717000c2e0 con 0x7f7170003970 2024-03-23T09:41:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f7189286640 1 --2- 172.21.15.47:0/4288934991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7170005180 0x7f717000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f7188a85640 1 --2- 172.21.15.47:0/4288934991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7170003970 0x7f7170003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f7183fff640 1 --2- 172.21.15.47:0/4288934991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71700047c0 0x7f7170004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f7189286640 1 -- 172.21.15.47:0/4288934991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7170003970 msgr2=0x7f7170003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f7189286640 1 --2- 172.21.15.47:0/4288934991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7170003970 0x7f7170003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f7189286640 1 -- 172.21.15.47:0/4288934991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71700047c0 msgr2=0x7f7170004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f7189286640 1 --2- 172.21.15.47:0/4288934991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71700047c0 0x7f7170004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f7189286640 1 -- 172.21.15.47:0/4288934991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7170003520 con 0x7f7170005180 2024-03-23T09:41:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.518+0000 7f7183fff640 1 --2- 172.21.15.47:0/4288934991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71700047c0 0x7f7170004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.519+0000 7f7189286640 1 -- 172.21.15.47:0/4288934991 reap_dead start 2024-03-23T09:41:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.519+0000 7f7189286640 1 --2- 172.21.15.47:0/4288934991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7170005180 0x7f717000b980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f716c009ac0 tx=0x7f71700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437513b168ec8ff4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.519+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c004020 con 0x7f7170005180 2024-03-23T09:41:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.519+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c004180 con 0x7f7170005180 2024-03-23T09:41:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.519+0000 7f71837fe640 1 -- 172.21.15.47:0/4288934991 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c02c710 con 0x7f7170005180 2024-03-23T09:41:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.520+0000 7f718ace9640 1 -- 172.21.15.47:0/4288934991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7170005180 msgr2=0x7f717000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.520+0000 7f718ace9640 1 --2- 172.21.15.47:0/4288934991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7170005180 0x7f717000b980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f716c009ac0 tx=0x7f71700057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.520+0000 7f718ace9640 1 -- 172.21.15.47:0/4288934991 shutdown_connections 2024-03-23T09:41:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.520+0000 7f718ace9640 1 --2- 172.21.15.47:0/4288934991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7170005180 0x7f717000b980 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.520+0000 7f718ace9640 1 -- 172.21.15.47:0/4288934991 >> 172.21.15.47:0/4288934991 conn(0x7f718406b160 msgr2=0x7f718406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:54.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.520+0000 7f718ace9640 1 -- 172.21.15.47:0/4288934991 shutdown_connections 2024-03-23T09:41:54.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.520+0000 7f718ace9640 1 -- 172.21.15.47:0/4288934991 wait complete. 2024-03-23T09:41:54.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.521+0000 7f718ace9640 1 Processor -- start 2024-03-23T09:41:54.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.521+0000 7f718ace9640 1 -- start start 2024-03-23T09:41:54.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f718ace9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7170005180 0x7f71840b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f718ace9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f718414fc30 0x7f71840b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f718ace9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f718415a380 0x7f718415a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f718ace9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718415ad70 con 0x7f7170005180 2024-03-23T09:41:54.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f718ace9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71841500b0 con 0x7f718414fc30 2024-03-23T09:41:54.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f718ace9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71841f3010 con 0x7f718415a380 2024-03-23T09:41:54.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f7188a85640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7170005180 0x7f71840b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f7188a85640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7170005180 0x7f71840b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48962/0 (socket says 172.21.15.47:48962) 2024-03-23T09:41:54.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f7188a85640 1 -- 172.21.15.47:0/1558398068 learned_addr learned my addr 172.21.15.47:0/1558398068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f7189286640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f718415a380 0x7f718415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f7183fff640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f718414fc30 0x7f71840b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f7188a85640 1 -- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f718415a380 msgr2=0x7f718415a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f7188a85640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f718415a380 0x7f718415a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f7188a85640 1 -- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f718414fc30 msgr2=0x7f71840b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f7188a85640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f718414fc30 0x7f71840b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f7188a85640 1 -- 172.21.15.47:0/1558398068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71841f32f0 con 0x7f7170005180 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f7183fff640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f718414fc30 0x7f71840b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.522+0000 7f7188a85640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7170005180 0x7f71840b4aa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f717c0079d0 tx=0x7f717c00a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.523+0000 7f71817fa640 1 -- 172.21.15.47:0/1558398068 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c0133f0 con 0x7f7170005180 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.523+0000 7f71817fa640 1 -- 172.21.15.47:0/1558398068 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c013570 con 0x7f7170005180 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.523+0000 7f718ace9640 1 -- 172.21.15.47:0/1558398068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71841f35d0 con 0x7f7170005180 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.523+0000 7f71817fa640 1 -- 172.21.15.47:0/1558398068 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c013920 con 0x7f7170005180 2024-03-23T09:41:54.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.523+0000 7f718ace9640 1 -- 172.21.15.47:0/1558398068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71840b3210 con 0x7f7170005180 2024-03-23T09:41:54.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.524+0000 7f718ace9640 1 -- 172.21.15.47:0/1558398068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7184072360 con 0x7f7170005180 2024-03-23T09:41:54.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.524+0000 7f71817fa640 1 -- 172.21.15.47:0/1558398068 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f717c013a80 con 0x7f7170005180 2024-03-23T09:41:54.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.524+0000 7f71817fa640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f716406ac30 0x7f716406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.525+0000 7f71817fa640 1 -- 172.21.15.47:0/1558398068 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f717c089a30 con 0x7f7170005180 2024-03-23T09:41:54.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.525+0000 7f7183fff640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f716406ac30 0x7f716406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.525+0000 7f7183fff640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f716406ac30 0x7f716406d0f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7174000c00 tx=0x7f7174009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:54.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.529+0000 7f71817fa640 1 -- 172.21.15.47:0/1558398068 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f717c054f60 con 0x7f7170005180 2024-03-23T09:41:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.777+0000 7f718ace9640 1 -- 172.21.15.47:0/1558398068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f71841f3af0 con 0x7f7170005180 2024-03-23T09:41:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.778+0000 7f71817fa640 1 -- 172.21.15.47:0/1558398068 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f71841f3af0 con 0x7f7170005180 2024-03-23T09:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2024-03-23T09:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2024-03-23T09:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lo+xoDhAAyxqr1H6eiINwt2X7LUlrSg==", 2024-03-23T09:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZ5OlGhAAuUElzRghNRFGKREdCpxKvA==", 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2024-03-23T09:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lFwyzFRAAKldMOA7oSblA8YMrvntY8w==", 2024-03-23T09:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.d", 2024-03-23T09:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lpiD3IRAApXAdUQ4AkxWHzsFV0qcBng==", 2024-03-23T09:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB7o/5lLvoADBAAgdixkm+jHwQilYw0IAn6EA==", 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2024-03-23T09:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB+o/5lOBfGCRAApglzalO1ztnJzOKg/UAyWA==", 2024-03-23T09:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2024-03-23T09:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCBo/5lRBjlBxAAd45w8/kTMPqxmNyfCS3XYA==", 2024-03-23T09:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCEo/5l+JxrBhAAbR/XqDyPDnN8eLpmvBZ7Mg==", 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCKo/5ltsqwDRAAkqcZ/sEf2oozWFN4AXg/+w==", 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCNo/5lpLHiCxAAB2axUv9gjcYVlij9f+AEHg==", 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCQo/5l2io3ChAAu+lfYcWWBQf43aYzWjCBNg==", 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCTo/5lA7jjCBAAjJgBkp89H8lx6fidl/Z7IA==", 2024-03-23T09:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZFc3JhAAZWmDZOnkW9eo9hqEDYynQw==", 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2024-03-23T09:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lIrJSLxAAPJer3ZP3h8VkImkiJmnKVw==", 2024-03-23T09:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l1+DyKxAAZLKZNbUpUJ0fWPIYNvWK+g==", 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lWqkiNRAAL1q0lEcM/NKW3Pf9LcP21g==", 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB4o/5lyeuKLRAARrfTpPc5BTVYZe/LUREDkg==", 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-mds", 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCeo/5l2uaQARAAALI2s0avq3GOZJ22XpNM3A==", 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-mds" 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-mgr", 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCeo/5l5PGQARAAQnixRXoRQypfUuCyNOOOHw==", 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-osd", 2024-03-23T09:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCeo/5l+fiQARAAvazax7eZ+6QoI8hglhvwHw==", 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-osd" 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rbd", 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCeo/5lqACRARAA8bIVOdmB4Cxxc+iZ86CW3Q==", 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-23T09:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCeo/5lXwiRARAAz2mUTAQAs+sB7BWTNYyGNg==", 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rgw", 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCeo/5loQ+RARAAqXSOy2Eem3+8PAA3ziWfmA==", 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l/0EOCxAAIrgIXRiFSQ1G7Jat5hNm4A==", 2024-03-23T09:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lywMFBxAAIVwjCe6B9EtBMAG2n7cZJw==", 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-23T09:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:41:54.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.780+0000 7f718ace9640 1 -- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f716406ac30 msgr2=0x7f716406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:54.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.780+0000 7f718ace9640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f716406ac30 0x7f716406d0f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7174000c00 tx=0x7f7174009040 comp rx=0 tx=0).stop 2024-03-23T09:41:54.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.780+0000 7f718ace9640 1 -- 172.21.15.47:0/1558398068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7170005180 msgr2=0x7f71840b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:54.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.780+0000 7f718ace9640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7170005180 0x7f71840b4aa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f717c0079d0 tx=0x7f717c00a6e0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.780+0000 7f718ace9640 1 -- 172.21.15.47:0/1558398068 shutdown_connections 2024-03-23T09:41:54.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.780+0000 7f718ace9640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f716406ac30 0x7f716406d0f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.780+0000 7f718ace9640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f718415a380 0x7f718415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.780+0000 7f718ace9640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f718414fc30 0x7f71840b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.780+0000 7f718ace9640 1 --2- 172.21.15.47:0/1558398068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7170005180 0x7f71840b4aa0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.780+0000 7f718ace9640 1 -- 172.21.15.47:0/1558398068 >> 172.21.15.47:0/1558398068 conn(0x7f718406b160 msgr2=0x7f71840b6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:54.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.780+0000 7f718ace9640 1 -- 172.21.15.47:0/1558398068 shutdown_connections 2024-03-23T09:41:54.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.780+0000 7f718ace9640 1 -- 172.21.15.47:0/1558398068 wait complete. 2024-03-23T09:41:54.796 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-23T09:41:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.898+0000 7f31a5751640 1 Processor -- start 2024-03-23T09:41:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.898+0000 7f31a5751640 1 -- start start 2024-03-23T09:41:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f31a5751640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0161400 0x7f31a006b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f31a5751640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f31a006bd40 0x7f31a00ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f31a5751640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a00cdad0 con 0x7f31a0161020 2024-03-23T09:41:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f31a5751640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a006c1d0 con 0x7f31a006bd40 2024-03-23T09:41:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f31a5751640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a00b03c0 con 0x7f31a0161400 2024-03-23T09:41:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f319f7fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f31a006bd40 0x7f31a00ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f319e7fc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0161400 0x7f31a006b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f319e7fc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0161400 0x7f31a006b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:39418/0 (socket says 172.21.15.47:39418) 2024-03-23T09:41:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f319e7fc640 1 -- 172.21.15.47:0/117454508 learned_addr learned my addr 172.21.15.47:0/117454508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f319effd640 1 --1- 172.21.15.47:0/117454508 >> v1:172.21.15.175:6789/0 conn(0x7f31a0161020 0x7f31a00b1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:54482/0 (socket says 172.21.15.47:54482) 2024-03-23T09:41:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f319e7fc640 1 -- 172.21.15.47:0/117454508 >> v2:172.21.15.175:3300/0 conn(0x7f31a006bd40 msgr2=0x7f31a00ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f319e7fc640 1 --2- 172.21.15.47:0/117454508 >> v2:172.21.15.175:3300/0 conn(0x7f31a006bd40 0x7f31a00ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f319e7fc640 1 -- 172.21.15.47:0/117454508 >> v1:172.21.15.175:6789/0 conn(0x7f31a0161020 legacy=0x7f31a00b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.899+0000 7f319e7fc640 1 -- 172.21.15.47:0/117454508 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a00b06a0 con 0x7f31a0161400 2024-03-23T09:41:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.900+0000 7f319e7fc640 1 --2- 172.21.15.47:0/117454508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0161400 0x7f31a006b800 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3194009870 tx=0x7f319402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8682970535098bf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.900+0000 7f319dffb640 1 -- 172.21.15.47:0/117454508 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194030840 con 0x7f31a0161400 2024-03-23T09:41:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.900+0000 7f319dffb640 1 -- 172.21.15.47:0/117454508 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31940309a0 con 0x7f31a0161400 2024-03-23T09:41:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.900+0000 7f319dffb640 1 -- 172.21.15.47:0/117454508 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194030d30 con 0x7f31a0161400 2024-03-23T09:41:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.901+0000 7f31a5751640 1 -- 172.21.15.47:0/117454508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0161400 msgr2=0x7f31a006b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.901+0000 7f31a5751640 1 --2- 172.21.15.47:0/117454508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0161400 0x7f31a006b800 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3194009870 tx=0x7f319402b980 comp rx=0 tx=0).stop 2024-03-23T09:41:54.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.901+0000 7f31a5751640 1 -- 172.21.15.47:0/117454508 shutdown_connections 2024-03-23T09:41:54.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.901+0000 7f31a5751640 1 --2- 172.21.15.47:0/117454508 >> v2:172.21.15.175:3300/0 conn(0x7f31a006bd40 0x7f31a00ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.901+0000 7f31a5751640 1 --2- 172.21.15.47:0/117454508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0161400 0x7f31a006b800 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.901+0000 7f31a5751640 1 -- 172.21.15.47:0/117454508 >> 172.21.15.47:0/117454508 conn(0x7f31a00d0670 msgr2=0x7f31a00d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:54.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.901+0000 7f31a5751640 1 -- 172.21.15.47:0/117454508 shutdown_connections 2024-03-23T09:41:54.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.901+0000 7f31a5751640 1 -- 172.21.15.47:0/117454508 wait complete. 2024-03-23T09:41:54.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.902+0000 7f31a5751640 1 Processor -- start 2024-03-23T09:41:54.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.902+0000 7f31a5751640 1 -- start start 2024-03-23T09:41:54.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f31a5751640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31a006bd40 0x7f31a009dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f31a5751640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0161020 0x7f31a015e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f31a5751640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a0161400 0x7f31a015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f31a5751640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a00b31c0 con 0x7f31a0161400 2024-03-23T09:41:54.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f31a5751640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a009e1d0 con 0x7f31a006bd40 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f31a5751640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a01506e0 con 0x7f31a0161020 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f319f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a0161400 0x7f31a015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f319f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a0161400 0x7f31a015ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48880/0 (socket says 172.21.15.47:48880) 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f319f7fe640 1 -- 172.21.15.47:0/3350399208 learned_addr learned my addr 172.21.15.47:0/3350399208 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f319effd640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31a006bd40 0x7f31a009dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f319e7fc640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0161020 0x7f31a015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f319f7fe640 1 -- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0161020 msgr2=0x7f31a015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f319f7fe640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0161020 0x7f31a015e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f319f7fe640 1 -- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31a006bd40 msgr2=0x7f31a009dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f319f7fe640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31a006bd40 0x7f31a009dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f319f7fe640 1 -- 172.21.15.47:0/3350399208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31940094d0 con 0x7f31a0161400 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f319effd640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31a006bd40 0x7f31a009dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.903+0000 7f319e7fc640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0161020 0x7f31a015e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.904+0000 7f319f7fe640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a0161400 0x7f31a015ecd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f319000efc0 tx=0x7f319000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:54.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.904+0000 7f31877fe640 1 -- 172.21.15.47:0/3350399208 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190014480 con 0x7f31a0161400 2024-03-23T09:41:54.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.904+0000 7f31877fe640 1 -- 172.21.15.47:0/3350399208 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319000f040 con 0x7f31a0161400 2024-03-23T09:41:54.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.904+0000 7f31a5751640 1 -- 172.21.15.47:0/3350399208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a01509c0 con 0x7f31a0161400 2024-03-23T09:41:54.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.904+0000 7f31877fe640 1 -- 172.21.15.47:0/3350399208 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190014820 con 0x7f31a0161400 2024-03-23T09:41:54.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.904+0000 7f31a5751640 1 -- 172.21.15.47:0/3350399208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a0150f80 con 0x7f31a0161400 2024-03-23T09:41:54.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.905+0000 7f31a5751640 1 -- 172.21.15.47:0/3350399208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a00672f0 con 0x7f31a0161400 2024-03-23T09:41:54.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.905+0000 7f31877fe640 1 -- 172.21.15.47:0/3350399208 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3190014980 con 0x7f31a0161400 2024-03-23T09:41:54.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.906+0000 7f31877fe640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f318c06ac00 0x7f318c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:54.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.906+0000 7f31877fe640 1 -- 172.21.15.47:0/3350399208 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3190089bd0 con 0x7f31a0161400 2024-03-23T09:41:54.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.906+0000 7f319effd640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f318c06ac00 0x7f318c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:54.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.906+0000 7f319effd640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f318c06ac00 0x7f318c06d0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3198007fd0 tx=0x7f3198007d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:54.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:54.910+0000 7f31877fe640 1 -- 172.21.15.47:0/3350399208 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3190055100 con 0x7f31a0161400 2024-03-23T09:41:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.158+0000 7f31a5751640 1 -- 172.21.15.47:0/3350399208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f31a00b01a0 con 0x7f31a0161400 2024-03-23T09:41:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.161+0000 7f31877fe640 1 -- 172.21.15.47:0/3350399208 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f31900180a0 con 0x7f31a0161400 2024-03-23T09:41:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.163+0000 7f31a5751640 1 -- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f318c06ac00 msgr2=0x7f318c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.163+0000 7f31a5751640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f318c06ac00 0x7f318c06d0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3198007fd0 tx=0x7f3198007d40 comp rx=0 tx=0).stop 2024-03-23T09:41:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.163+0000 7f31a5751640 1 -- 172.21.15.47:0/3350399208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a0161400 msgr2=0x7f31a015ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.163+0000 7f31a5751640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a0161400 0x7f31a015ecd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f319000efc0 tx=0x7f319000c490 comp rx=0 tx=0).stop 2024-03-23T09:41:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.163+0000 7f31a5751640 1 -- 172.21.15.47:0/3350399208 shutdown_connections 2024-03-23T09:41:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.163+0000 7f31a5751640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f318c06ac00 0x7f318c06d0c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.163+0000 7f31a5751640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a0161400 0x7f31a015ecd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.163+0000 7f31a5751640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0161020 0x7f31a015e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.163+0000 7f31a5751640 1 --2- 172.21.15.47:0/3350399208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31a006bd40 0x7f31a009dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.163+0000 7f31a5751640 1 -- 172.21.15.47:0/3350399208 >> 172.21.15.47:0/3350399208 conn(0x7f31a00d0670 msgr2=0x7f31a00b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.163+0000 7f31a5751640 1 -- 172.21.15.47:0/3350399208 shutdown_connections 2024-03-23T09:41:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.163+0000 7f31a5751640 1 -- 172.21.15.47:0/3350399208 wait complete. 2024-03-23T09:41:55.174 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-23T09:41:55.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.278+0000 7f331db73640 1 Processor -- start 2024-03-23T09:41:55.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.278+0000 7f331db73640 1 -- start start 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.279+0000 7f331db73640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f33180b8350 0x7f33180cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.279+0000 7f331db73640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180cdc40 0x7f33180d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.279+0000 7f331db73640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33180d11e0 con 0x7f331815ca90 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.279+0000 7f331db73640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33180ce0d0 con 0x7f33180b8350 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.279+0000 7f331db73640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3318160f60 con 0x7f33180cdc40 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.279+0000 7f33177fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f331815ca90 0x7f3318160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58394/0 (socket says 172.21.15.47:58394) 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.279+0000 7f33177fe640 1 -- 172.21.15.47:0/2790427702 learned_addr learned my addr 172.21.15.47:0/2790427702 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.279+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780607926 0 0) 0x7f33180d11e0 con 0x7f331815ca90 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.279+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3300003520 con 0x7f331815ca90 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.279+0000 7f3317fff640 1 --2- 172.21.15.47:0/2790427702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180cdc40 0x7f33180d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.279+0000 7f3316ffd640 1 --2- 172.21.15.47:0/2790427702 >> v2:172.21.15.175:3300/0 conn(0x7f33180b8350 0x7f33180cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 799712934 0 0) 0x7f3300003520 con 0x7f331815ca90 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180cdc40 msgr2=0x7f33180d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 --2- 172.21.15.47:0/2790427702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180cdc40 0x7f33180d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 >> v2:172.21.15.175:3300/0 conn(0x7f33180b8350 msgr2=0x7f33180cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 --2- 172.21.15.47:0/2790427702 >> v2:172.21.15.175:3300/0 conn(0x7f33180b8350 0x7f33180cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3318161240 con 0x7f331815ca90 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f3308001670 con 0x7f331815ca90 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 >> v1:172.21.15.47:6789/0 conn(0x7f331815ca90 legacy=0x7f3318160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 --2- 172.21.15.47:0/2790427702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3300003970 0x7f3300003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 --2- 172.21.15.47:0/2790427702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180cdc40 0x7f33000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 --2- 172.21.15.47:0/2790427702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3300004d00 0x7f3300005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33180ce0d0 con 0x7f3300003970 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3318160f60 con 0x7f3300004d00 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330000ba30 con 0x7f33180cdc40 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33177fe640 1 --2- 172.21.15.47:0/2790427702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3300003970 0x7f3300003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f3316ffd640 1 --2- 172.21.15.47:0/2790427702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180cdc40 0x7f33000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f3317fff640 1 --2- 172.21.15.47:0/2790427702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3300004d00 0x7f3300005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33177fe640 1 -- 172.21.15.47:0/2790427702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180cdc40 msgr2=0x7f33000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33177fe640 1 --2- 172.21.15.47:0/2790427702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180cdc40 0x7f33000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33177fe640 1 -- 172.21.15.47:0/2790427702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3300004d00 msgr2=0x7f3300005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33177fe640 1 --2- 172.21.15.47:0/2790427702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3300004d00 0x7f3300005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f33177fe640 1 -- 172.21.15.47:0/2790427702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3300003520 con 0x7f3300003970 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.280+0000 7f3317fff640 1 --2- 172.21.15.47:0/2790427702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3300004d00 0x7f3300005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.281+0000 7f33177fe640 1 --2- 172.21.15.47:0/2790427702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3300003970 0x7f3300003d90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f330800c290 tx=0x7f33000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2947d2842f7d13cc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.281+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308005ad0 con 0x7f3300003970 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.281+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3308005c50 con 0x7f3300003970 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.281+0000 7f33167fc640 1 -- 172.21.15.47:0/2790427702 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330802c8d0 con 0x7f3300003970 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.281+0000 7f331db73640 1 -- 172.21.15.47:0/2790427702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3300003970 msgr2=0x7f3300003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.281+0000 7f331db73640 1 --2- 172.21.15.47:0/2790427702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3300003970 0x7f3300003d90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f330800c290 tx=0x7f33000057f0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.281+0000 7f3317fff640 1 -- 172.21.15.47:0/2790427702 reap_dead start 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.281+0000 7f331db73640 1 -- 172.21.15.47:0/2790427702 shutdown_connections 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.281+0000 7f331db73640 1 -- 172.21.15.47:0/2790427702 >> 172.21.15.47:0/2790427702 conn(0x7f33180659d0 msgr2=0x7f3318065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.281+0000 7f331db73640 1 -- 172.21.15.47:0/2790427702 shutdown_connections 2024-03-23T09:41:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.281+0000 7f331db73640 1 -- 172.21.15.47:0/2790427702 wait complete. 2024-03-23T09:41:55.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.283+0000 7f331db73640 1 Processor -- start 2024-03-23T09:41:55.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.283+0000 7f331db73640 1 -- start start 2024-03-23T09:41:55.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.283+0000 7f331db73640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3300003970 0x7f33181505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.283+0000 7f331db73640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180b3310 0x7f33180b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.283+0000 7f331db73640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3318158f90 0x7f3318159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.283+0000 7f331db73640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33180b2ef0 con 0x7f3318158f90 2024-03-23T09:41:55.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.283+0000 7f331db73640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3318159950 con 0x7f3300003970 2024-03-23T09:41:55.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.283+0000 7f331db73640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33180cc7e0 con 0x7f33180b3310 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f3317fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3318158f90 0x7f3318159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f3317fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3318158f90 0x7f3318159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48900/0 (socket says 172.21.15.47:48900) 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f3317fff640 1 -- 172.21.15.47:0/3216020688 learned_addr learned my addr 172.21.15.47:0/3216020688 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f33177fe640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3300003970 0x7f33181505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f3316ffd640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180b3310 0x7f33180b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f3317fff640 1 -- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180b3310 msgr2=0x7f33180b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f3317fff640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180b3310 0x7f33180b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f3317fff640 1 -- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3300003970 msgr2=0x7f33181505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f3317fff640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3300003970 0x7f33181505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f3317fff640 1 -- 172.21.15.47:0/3216020688 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33180b2550 con 0x7f3318158f90 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f33177fe640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3300003970 0x7f33181505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f3316ffd640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180b3310 0x7f33180b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f3317fff640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3318158f90 0x7f3318159410 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f33180b54c0 tx=0x7f3310010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f32fffff640 1 -- 172.21.15.47:0/3216020688 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331000e050 con 0x7f3318158f90 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f32fffff640 1 -- 172.21.15.47:0/3216020688 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331000f040 con 0x7f3318158f90 2024-03-23T09:41:55.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f331db73640 1 -- 172.21.15.47:0/3216020688 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33180b2830 con 0x7f3318158f90 2024-03-23T09:41:55.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f32fffff640 1 -- 172.21.15.47:0/3216020688 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310003990 con 0x7f3318158f90 2024-03-23T09:41:55.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.284+0000 7f331db73640 1 -- 172.21.15.47:0/3216020688 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3318150ae0 con 0x7f3318158f90 2024-03-23T09:41:55.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.285+0000 7f331db73640 1 -- 172.21.15.47:0/3216020688 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3318072360 con 0x7f3318158f90 2024-03-23T09:41:55.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.286+0000 7f32fffff640 1 -- 172.21.15.47:0/3216020688 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3310015440 con 0x7f3318158f90 2024-03-23T09:41:55.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.286+0000 7f32fffff640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f330406ac00 0x7f330406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.286+0000 7f32fffff640 1 -- 172.21.15.47:0/3216020688 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f33100899b0 con 0x7f3318158f90 2024-03-23T09:41:55.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.286+0000 7f33177fe640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f330406ac00 0x7f330406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.287+0000 7f33177fe640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f330406ac00 0x7f330406d0c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3308034000 tx=0x7f3308007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:55.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.291+0000 7f32fffff640 1 -- 172.21.15.47:0/3216020688 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3310054f70 con 0x7f3318158f90 2024-03-23T09:41:55.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.537+0000 7f331db73640 1 -- 172.21.15.47:0/3216020688 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f33180b3030 con 0x7f3318158f90 2024-03-23T09:41:55.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.540+0000 7f32fffff640 1 -- 172.21.15.47:0/3216020688 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f33180b3030 con 0x7f3318158f90 2024-03-23T09:41:55.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.542+0000 7f331db73640 1 -- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f330406ac00 msgr2=0x7f330406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.542+0000 7f331db73640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f330406ac00 0x7f330406d0c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3308034000 tx=0x7f3308007000 comp rx=0 tx=0).stop 2024-03-23T09:41:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.542+0000 7f331db73640 1 -- 172.21.15.47:0/3216020688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3318158f90 msgr2=0x7f3318159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.542+0000 7f331db73640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3318158f90 0x7f3318159410 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f33180b54c0 tx=0x7f3310010590 comp rx=0 tx=0).stop 2024-03-23T09:41:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.542+0000 7f331db73640 1 -- 172.21.15.47:0/3216020688 shutdown_connections 2024-03-23T09:41:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.542+0000 7f331db73640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f330406ac00 0x7f330406d0c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.542+0000 7f331db73640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3318158f90 0x7f3318159410 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.542+0000 7f331db73640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33180b3310 0x7f33180b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.542+0000 7f331db73640 1 --2- 172.21.15.47:0/3216020688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3300003970 0x7f33181505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.542+0000 7f331db73640 1 -- 172.21.15.47:0/3216020688 >> 172.21.15.47:0/3216020688 conn(0x7f33180659d0 msgr2=0x7f33180b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.543+0000 7f331db73640 1 -- 172.21.15.47:0/3216020688 shutdown_connections 2024-03-23T09:41:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.543+0000 7f331db73640 1 -- 172.21.15.47:0/3216020688 wait complete. 2024-03-23T09:41:55.553 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-23T09:41:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.655+0000 7f169316d640 1 Processor -- start 2024-03-23T09:41:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.655+0000 7f169316d640 1 -- start start 2024-03-23T09:41:55.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.655+0000 7f169316d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f168c15ca90 0x7f168c160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.656+0000 7f169316d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f168c0cde10 0x7f168c0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.656+0000 7f169316d640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168c0d11e0 con 0x7f168c0b8350 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.656+0000 7f169316d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168c160f60 con 0x7f168c0cde10 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.656+0000 7f169316d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c161240 con 0x7f168c15ca90 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.656+0000 7f169170a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f168c0cde10 0x7f168c0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.656+0000 7f169170a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f168c0cde10 0x7f168c0d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48904/0 (socket says 172.21.15.47:48904) 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.656+0000 7f169170a640 1 -- 172.21.15.47:0/2184629250 learned_addr learned my addr 172.21.15.47:0/2184629250 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.656+0000 7f1690f09640 1 --2- 172.21.15.47:0/2184629250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f168c15ca90 0x7f168c160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.656+0000 7f169170a640 1 -- 172.21.15.47:0/2184629250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f168c15ca90 msgr2=0x7f168c160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.656+0000 7f169170a640 1 --2- 172.21.15.47:0/2184629250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f168c15ca90 0x7f168c160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.656+0000 7f169170a640 1 -- 172.21.15.47:0/2184629250 >> v1:172.21.15.175:6789/0 conn(0x7f168c0b8350 legacy=0x7f168c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.656+0000 7f169170a640 1 -- 172.21.15.47:0/2184629250 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c161520 con 0x7f168c0cde10 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.656+0000 7f1690f09640 1 --2- 172.21.15.47:0/2184629250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f168c15ca90 0x7f168c160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.657+0000 7f169170a640 1 --2- 172.21.15.47:0/2184629250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c0cde10 0x7f168c0d1670 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1684009e50 tx=0x7f168402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c851d975cc5ea8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.657+0000 7f168b7fe640 1 -- 172.21.15.47:0/2184629250 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168402c9e0 con 0x7f168c0cde10 2024-03-23T09:41:55.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.657+0000 7f168b7fe640 1 -- 172.21.15.47:0/2184629250 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168402cb40 con 0x7f168c0cde10 2024-03-23T09:41:55.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.657+0000 7f168b7fe640 1 -- 172.21.15.47:0/2184629250 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16840343f0 con 0x7f168c0cde10 2024-03-23T09:41:55.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.657+0000 7f169316d640 1 -- 172.21.15.47:0/2184629250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c0cde10 msgr2=0x7f168c0d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:55.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.657+0000 7f169316d640 1 --2- 172.21.15.47:0/2184629250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c0cde10 0x7f168c0d1670 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1684009e50 tx=0x7f168402be40 comp rx=0 tx=0).stop 2024-03-23T09:41:55.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.657+0000 7f169316d640 1 -- 172.21.15.47:0/2184629250 shutdown_connections 2024-03-23T09:41:55.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.657+0000 7f169316d640 1 --2- 172.21.15.47:0/2184629250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c0cde10 0x7f168c0d1670 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.657+0000 7f169316d640 1 --2- 172.21.15.47:0/2184629250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f168c15ca90 0x7f168c160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.657+0000 7f169316d640 1 -- 172.21.15.47:0/2184629250 >> 172.21.15.47:0/2184629250 conn(0x7f168c0659d0 msgr2=0x7f168c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:55.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.657+0000 7f169316d640 1 -- 172.21.15.47:0/2184629250 shutdown_connections 2024-03-23T09:41:55.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.657+0000 7f169316d640 1 -- 172.21.15.47:0/2184629250 wait complete. 2024-03-23T09:41:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.659+0000 7f169316d640 1 Processor -- start 2024-03-23T09:41:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.659+0000 7f169316d640 1 -- start start 2024-03-23T09:41:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.659+0000 7f169316d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f168c0b8350 0x7f168c1509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.659+0000 7f169316d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f168c0cde10 0x7f168c0b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.659+0000 7f169316d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c15ca90 0x7f168c153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.659+0000 7f169316d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c158e60 con 0x7f168c15ca90 2024-03-23T09:41:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.659+0000 7f169316d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c153640 con 0x7f168c0cde10 2024-03-23T09:41:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.659+0000 7f169316d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c0b5370 con 0x7f168c0b8350 2024-03-23T09:41:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f169170a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c15ca90 0x7f168c153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f169170a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c15ca90 0x7f168c153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48916/0 (socket says 172.21.15.47:48916) 2024-03-23T09:41:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f169170a640 1 -- 172.21.15.47:0/4271052834 learned_addr learned my addr 172.21.15.47:0/4271052834 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f168bfff640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f168c0cde10 0x7f168c0b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f1690f09640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f168c0b8350 0x7f168c1509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f169170a640 1 -- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f168c0b8350 msgr2=0x7f168c1509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f169170a640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f168c0b8350 0x7f168c1509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f169170a640 1 -- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f168c0cde10 msgr2=0x7f168c0b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f169170a640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f168c0cde10 0x7f168c0b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f169170a640 1 -- 172.21.15.47:0/4271052834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1684009b30 con 0x7f168c15ca90 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f1690f09640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f168c0b8350 0x7f168c1509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f168bfff640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f168c0cde10 0x7f168c0b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f169170a640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c15ca90 0x7f168c153a60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1684036040 tx=0x7f1684004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f16897fa640 1 -- 172.21.15.47:0/4271052834 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168403e450 con 0x7f168c15ca90 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f16897fa640 1 -- 172.21.15.47:0/4271052834 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168403e5d0 con 0x7f168c15ca90 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f169316d640 1 -- 172.21.15.47:0/4271052834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168c152cc0 con 0x7f168c15ca90 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f16897fa640 1 -- 172.21.15.47:0/4271052834 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168403e9c0 con 0x7f168c15ca90 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.660+0000 7f169316d640 1 -- 172.21.15.47:0/4271052834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f168c152f80 con 0x7f168c15ca90 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.661+0000 7f169316d640 1 -- 172.21.15.47:0/4271052834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168c072360 con 0x7f168c15ca90 2024-03-23T09:41:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.662+0000 7f16897fa640 1 -- 172.21.15.47:0/4271052834 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f168403eb20 con 0x7f168c15ca90 2024-03-23T09:41:55.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.662+0000 7f16897fa640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f167006ac00 0x7f167006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:55.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.662+0000 7f16897fa640 1 -- 172.21.15.47:0/4271052834 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f16840ac140 con 0x7f168c15ca90 2024-03-23T09:41:55.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.662+0000 7f1690f09640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f167006ac00 0x7f167006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:55.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.663+0000 7f1690f09640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f167006ac00 0x7f167006d0c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1680006fd0 tx=0x7f1680008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:55.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.666+0000 7f16897fa640 1 -- 172.21.15.47:0/4271052834 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1684077700 con 0x7f168c15ca90 2024-03-23T09:41:55.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.912+0000 7f169316d640 1 -- 172.21.15.47:0/4271052834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f168c153780 con 0x7f168c15ca90 2024-03-23T09:41:55.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.915+0000 7f16897fa640 1 -- 172.21.15.47:0/4271052834 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f168c153780 con 0x7f168c15ca90 2024-03-23T09:41:55.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.917+0000 7f169316d640 1 -- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f167006ac00 msgr2=0x7f167006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:55.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.917+0000 7f169316d640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f167006ac00 0x7f167006d0c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1680006fd0 tx=0x7f1680008040 comp rx=0 tx=0).stop 2024-03-23T09:41:55.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.917+0000 7f169316d640 1 -- 172.21.15.47:0/4271052834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c15ca90 msgr2=0x7f168c153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:55.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.917+0000 7f169316d640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c15ca90 0x7f168c153a60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1684036040 tx=0x7f1684004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:55.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.917+0000 7f169316d640 1 -- 172.21.15.47:0/4271052834 shutdown_connections 2024-03-23T09:41:55.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.917+0000 7f169316d640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f167006ac00 0x7f167006d0c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.917+0000 7f169316d640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c15ca90 0x7f168c153a60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.917+0000 7f169316d640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f168c0cde10 0x7f168c0b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.917+0000 7f169316d640 1 --2- 172.21.15.47:0/4271052834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f168c0b8350 0x7f168c1509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:55.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.917+0000 7f169316d640 1 -- 172.21.15.47:0/4271052834 >> 172.21.15.47:0/4271052834 conn(0x7f168c0659d0 msgr2=0x7f168c0b7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:55.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.917+0000 7f169316d640 1 -- 172.21.15.47:0/4271052834 shutdown_connections 2024-03-23T09:41:55.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:55.917+0000 7f169316d640 1 -- 172.21.15.47:0/4271052834 wait complete. 2024-03-23T09:41:55.927 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-23T09:41:56.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.029+0000 7f9ef1207640 1 Processor -- start 2024-03-23T09:41:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.029+0000 7f9ef1207640 1 -- start start 2024-03-23T09:41:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.030+0000 7f9ef1207640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9eec0b8350 0x7f9eec0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.030+0000 7f9ef1207640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eec0ce090 con 0x7f9eec0cdc40 2024-03-23T09:41:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.030+0000 7f9ef1207640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eec0d11e0 con 0x7f9eec15ca90 2024-03-23T09:41:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.030+0000 7f9ef1207640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eec160f60 con 0x7f9eec0b8350 2024-03-23T09:41:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.030+0000 7f9ee3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9eec0b8350 0x7f9eec0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.030+0000 7f9eeb59e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9eec0cdc40 0x7f9eec0d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58402/0 (socket says 172.21.15.47:58402) 2024-03-23T09:41:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.030+0000 7f9eeb59e640 1 -- 172.21.15.47:0/4287954461 learned_addr learned my addr 172.21.15.47:0/4287954461 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.030+0000 7f9ee3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9eec0b8350 0x7f9eec0cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48930/0 (socket says 172.21.15.47:48930) 2024-03-23T09:41:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.030+0000 7f9ee3fff640 1 -- 172.21.15.47:0/4287954461 >> v1:172.21.15.175:6789/0 conn(0x7f9eec15ca90 legacy=0x7f9eec160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.030+0000 7f9ee3fff640 1 -- 172.21.15.47:0/4287954461 >> v1:172.21.15.47:6789/0 conn(0x7f9eec0cdc40 legacy=0x7f9eec0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.030+0000 7f9ee3fff640 1 -- 172.21.15.47:0/4287954461 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec161240 con 0x7f9eec0b8350 2024-03-23T09:41:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.030+0000 7f9eea59c640 1 -- 172.21.15.47:0/4287954461 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114261430 0 0) 0x7f9ee4001490 con 0x7f9eec0cdc40 2024-03-23T09:41:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.031+0000 7f9ee3fff640 1 --2- 172.21.15.47:0/4287954461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eec0b8350 0x7f9eec0cd700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40097a0 tx=0x7f9ed402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c0183809ecf862 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.031+0000 7f9eea59c640 1 -- 172.21.15.47:0/4287954461 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed402e070 con 0x7f9eec0b8350 2024-03-23T09:41:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.031+0000 7f9eea59c640 1 -- 172.21.15.47:0/4287954461 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed40315a0 con 0x7f9eec0b8350 2024-03-23T09:41:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.031+0000 7f9eea59c640 1 -- 172.21.15.47:0/4287954461 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4031930 con 0x7f9eec0b8350 2024-03-23T09:41:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.031+0000 7f9ef1207640 1 -- 172.21.15.47:0/4287954461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eec0b8350 msgr2=0x7f9eec0cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.031+0000 7f9ef1207640 1 --2- 172.21.15.47:0/4287954461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eec0b8350 0x7f9eec0cd700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40097a0 tx=0x7f9ed402b840 comp rx=0 tx=0).stop 2024-03-23T09:41:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.031+0000 7f9ef1207640 1 -- 172.21.15.47:0/4287954461 shutdown_connections 2024-03-23T09:41:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.031+0000 7f9ef1207640 1 --2- 172.21.15.47:0/4287954461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eec0b8350 0x7f9eec0cd700 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.031+0000 7f9ef1207640 1 -- 172.21.15.47:0/4287954461 >> 172.21.15.47:0/4287954461 conn(0x7f9eec0659d0 msgr2=0x7f9eec065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.032+0000 7f9ef1207640 1 -- 172.21.15.47:0/4287954461 shutdown_connections 2024-03-23T09:41:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.032+0000 7f9ef1207640 1 -- 172.21.15.47:0/4287954461 wait complete. 2024-03-23T09:41:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.033+0000 7f9ef1207640 1 Processor -- start 2024-03-23T09:41:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.033+0000 7f9ef1207640 1 -- start start 2024-03-23T09:41:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.033+0000 7f9ef1207640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eec0b8350 0x7f9eec0c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9ef1207640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eec0cdc40 0x7f9eec0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9ef1207640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eec15ca90 0x7f9eec0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9ef1207640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0b2b80 con 0x7f9eec0b8350 2024-03-23T09:41:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9ef1207640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec15b520 con 0x7f9eec0cdc40 2024-03-23T09:41:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9ef1207640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0b24c0 con 0x7f9eec15ca90 2024-03-23T09:41:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9eead9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eec0b8350 0x7f9eec0c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9eead9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eec0b8350 0x7f9eec0c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48946/0 (socket says 172.21.15.47:48946) 2024-03-23T09:41:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9eead9d640 1 -- 172.21.15.47:0/1706646212 learned_addr learned my addr 172.21.15.47:0/1706646212 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9eeb59e640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eec15ca90 0x7f9eec0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9ee3fff640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eec0cdc40 0x7f9eec0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9eead9d640 1 -- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eec15ca90 msgr2=0x7f9eec0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9eead9d640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eec15ca90 0x7f9eec0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9eead9d640 1 -- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eec0cdc40 msgr2=0x7f9eec0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9eead9d640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eec0cdc40 0x7f9eec0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9eead9d640 1 -- 172.21.15.47:0/1706646212 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed4009400 con 0x7f9eec0b8350 2024-03-23T09:41:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9eeb59e640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eec15ca90 0x7f9eec0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9ee3fff640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eec0cdc40 0x7f9eec0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9eead9d640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eec0b8350 0x7f9eec0c87c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00c430 tx=0x7f9edc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9ee37fe640 1 -- 172.21.15.47:0/1706646212 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc011790 con 0x7f9eec0b8350 2024-03-23T09:41:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9ee37fe640 1 -- 172.21.15.47:0/1706646212 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc0118f0 con 0x7f9eec0b8350 2024-03-23T09:41:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9ef1207640 1 -- 172.21.15.47:0/1706646212 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eec0b27a0 con 0x7f9eec0b8350 2024-03-23T09:41:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9ee37fe640 1 -- 172.21.15.47:0/1706646212 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc011ce0 con 0x7f9eec0b8350 2024-03-23T09:41:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.034+0000 7f9ef1207640 1 -- 172.21.15.47:0/1706646212 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eec15a730 con 0x7f9eec0b8350 2024-03-23T09:41:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.035+0000 7f9ef1207640 1 -- 172.21.15.47:0/1706646212 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eec072360 con 0x7f9eec0b8350 2024-03-23T09:41:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.036+0000 7f9ee37fe640 1 -- 172.21.15.47:0/1706646212 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9edc01c460 con 0x7f9eec0b8350 2024-03-23T09:41:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.036+0000 7f9ee37fe640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9ecc06ac00 0x7f9ecc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.036+0000 7f9ee37fe640 1 -- 172.21.15.47:0/1706646212 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9edc088fc0 con 0x7f9eec0b8350 2024-03-23T09:41:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.036+0000 7f9ee3fff640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9ecc06ac00 0x7f9ecc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.037+0000 7f9ee3fff640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9ecc06ac00 0x7f9ecc06d0c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4036000 tx=0x7f9ed4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.040+0000 7f9ee37fe640 1 -- 172.21.15.47:0/1706646212 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9edc0bc580 con 0x7f9eec0b8350 2024-03-23T09:41:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.288+0000 7f9ef1207640 1 -- 172.21.15.47:0/1706646212 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f9eec066be0 con 0x7f9eec0b8350 2024-03-23T09:41:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.292+0000 7f9ee37fe640 1 -- 172.21.15.47:0/1706646212 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9edc0544f0 con 0x7f9eec0b8350 2024-03-23T09:41:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.294+0000 7f9ef1207640 1 -- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9ecc06ac00 msgr2=0x7f9ecc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.294+0000 7f9ef1207640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9ecc06ac00 0x7f9ecc06d0c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4036000 tx=0x7f9ed4008030 comp rx=0 tx=0).stop 2024-03-23T09:41:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.294+0000 7f9ef1207640 1 -- 172.21.15.47:0/1706646212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eec0b8350 msgr2=0x7f9eec0c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.294+0000 7f9ef1207640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eec0b8350 0x7f9eec0c87c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00c430 tx=0x7f9edc00c8f0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.294+0000 7f9ef1207640 1 -- 172.21.15.47:0/1706646212 shutdown_connections 2024-03-23T09:41:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.294+0000 7f9ef1207640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9ecc06ac00 0x7f9ecc06d0c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.294+0000 7f9ef1207640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eec15ca90 0x7f9eec0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.294+0000 7f9ef1207640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eec0cdc40 0x7f9eec0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.294+0000 7f9ef1207640 1 --2- 172.21.15.47:0/1706646212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eec0b8350 0x7f9eec0c87c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.294+0000 7f9ef1207640 1 -- 172.21.15.47:0/1706646212 >> 172.21.15.47:0/1706646212 conn(0x7f9eec0659d0 msgr2=0x7f9eec15e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.294+0000 7f9ef1207640 1 -- 172.21.15.47:0/1706646212 shutdown_connections 2024-03-23T09:41:56.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.294+0000 7f9ef1207640 1 -- 172.21.15.47:0/1706646212 wait complete. 2024-03-23T09:41:56.304 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-23T09:41:56.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.407+0000 7fae44db1640 1 Processor -- start 2024-03-23T09:41:56.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.407+0000 7fae44db1640 1 -- start start 2024-03-23T09:41:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.408+0000 7fae44db1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae4015a610 0x7fae4015cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.408+0000 7fae44db1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4006ae60 con 0x7fae40069ce0 2024-03-23T09:41:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.408+0000 7fae44db1640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4006a250 con 0x7fae400cd020 2024-03-23T09:41:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.408+0000 7fae44db1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4015a9f0 con 0x7fae4015a610 2024-03-23T09:41:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.408+0000 7fae3e59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae4015a610 0x7fae4015cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.408+0000 7fae3e59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae4015a610 0x7fae4015cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48952/0 (socket says 172.21.15.47:48952) 2024-03-23T09:41:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.408+0000 7fae3dd9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fae40069ce0 0x7fae400cc910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58418/0 (socket says 172.21.15.47:58418) 2024-03-23T09:41:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.408+0000 7fae3e59c640 1 -- 172.21.15.47:0/1553626417 learned_addr learned my addr 172.21.15.47:0/1553626417 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.409+0000 7fae3e59c640 1 -- 172.21.15.47:0/1553626417 >> v1:172.21.15.175:6789/0 conn(0x7fae400cd020 legacy=0x7fae4006c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.409+0000 7fae3e59c640 1 -- 172.21.15.47:0/1553626417 >> v1:172.21.15.47:6789/0 conn(0x7fae40069ce0 legacy=0x7fae400cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.409+0000 7fae3e59c640 1 -- 172.21.15.47:0/1553626417 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4006a530 con 0x7fae4015a610 2024-03-23T09:41:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.409+0000 7fae3d59a640 1 -- 172.21.15.47:0/1553626417 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074247934 0 0) 0x7fae4006ae60 con 0x7fae40069ce0 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.409+0000 7fae3e59c640 1 --2- 172.21.15.47:0/1553626417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae4015a610 0x7fae4015cfa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fae30009e50 tx=0x7fae3002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484b8a888dbb35b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.410+0000 7fae3d59a640 1 -- 172.21.15.47:0/1553626417 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3002c9f0 con 0x7fae4015a610 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.410+0000 7fae3d59a640 1 -- 172.21.15.47:0/1553626417 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3002cb50 con 0x7fae4015a610 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.410+0000 7fae3d59a640 1 -- 172.21.15.47:0/1553626417 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae300343f0 con 0x7fae4015a610 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.410+0000 7fae44db1640 1 -- 172.21.15.47:0/1553626417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae4015a610 msgr2=0x7fae4015cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.410+0000 7fae44db1640 1 --2- 172.21.15.47:0/1553626417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae4015a610 0x7fae4015cfa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fae30009e50 tx=0x7fae3002be50 comp rx=0 tx=0).stop 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.410+0000 7fae44db1640 1 -- 172.21.15.47:0/1553626417 shutdown_connections 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.410+0000 7fae44db1640 1 --2- 172.21.15.47:0/1553626417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae4015a610 0x7fae4015cfa0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.410+0000 7fae44db1640 1 -- 172.21.15.47:0/1553626417 >> 172.21.15.47:0/1553626417 conn(0x7fae400cb7c0 msgr2=0x7fae400cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.410+0000 7fae44db1640 1 -- 172.21.15.47:0/1553626417 shutdown_connections 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.410+0000 7fae44db1640 1 -- 172.21.15.47:0/1553626417 wait complete. 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.411+0000 7fae44db1640 1 Processor -- start 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.412+0000 7fae44db1640 1 -- start start 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.412+0000 7fae44db1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae40069ce0 0x7fae400c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.412+0000 7fae44db1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae400cd020 0x7fae401f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.412+0000 7fae44db1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae4015a610 0x7fae400c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.412+0000 7fae44db1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae400c3bf0 con 0x7fae40069ce0 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.412+0000 7fae44db1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae400c3760 con 0x7fae4015a610 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.412+0000 7fae44db1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae400c2870 con 0x7fae400cd020 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.412+0000 7fae3e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae40069ce0 0x7fae400c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.412+0000 7fae3e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae40069ce0 0x7fae400c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48964/0 (socket says 172.21.15.47:48964) 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.412+0000 7fae3e59c640 1 -- 172.21.15.47:0/1882890913 learned_addr learned my addr 172.21.15.47:0/1882890913 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.412+0000 7fae3dd9b640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae400cd020 0x7fae401f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.412+0000 7fae3ed9d640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae4015a610 0x7fae400c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.413+0000 7fae3e59c640 1 -- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae400cd020 msgr2=0x7fae401f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.413+0000 7fae3e59c640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae400cd020 0x7fae401f58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.413+0000 7fae3e59c640 1 -- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae4015a610 msgr2=0x7fae400c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.413+0000 7fae3e59c640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae4015a610 0x7fae400c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.413+0000 7fae3e59c640 1 -- 172.21.15.47:0/1882890913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae30009b30 con 0x7fae40069ce0 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.413+0000 7fae3dd9b640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae400cd020 0x7fae401f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.413+0000 7fae3ed9d640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae4015a610 0x7fae400c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.413+0000 7fae3e59c640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae40069ce0 0x7fae400c5090 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fae30009e50 tx=0x7fae30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.413+0000 7fae36ffd640 1 -- 172.21.15.47:0/1882890913 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3003e450 con 0x7fae40069ce0 2024-03-23T09:41:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.413+0000 7fae36ffd640 1 -- 172.21.15.47:0/1882890913 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3003e5b0 con 0x7fae40069ce0 2024-03-23T09:41:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.413+0000 7fae36ffd640 1 -- 172.21.15.47:0/1882890913 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3003e960 con 0x7fae40069ce0 2024-03-23T09:41:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.413+0000 7fae44db1640 1 -- 172.21.15.47:0/1882890913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae400c2b50 con 0x7fae40069ce0 2024-03-23T09:41:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.413+0000 7fae44db1640 1 -- 172.21.15.47:0/1882890913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae400c3110 con 0x7fae40069ce0 2024-03-23T09:41:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.414+0000 7fae44db1640 1 -- 172.21.15.47:0/1882890913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4006e970 con 0x7fae40069ce0 2024-03-23T09:41:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.414+0000 7fae36ffd640 1 -- 172.21.15.47:0/1882890913 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fae3003eac0 con 0x7fae40069ce0 2024-03-23T09:41:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.415+0000 7fae36ffd640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae2006ac00 0x7fae2006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.415+0000 7fae36ffd640 1 -- 172.21.15.47:0/1882890913 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fae300abf80 con 0x7fae40069ce0 2024-03-23T09:41:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.415+0000 7fae3dd9b640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae2006ac00 0x7fae2006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.416+0000 7fae3dd9b640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae2006ac00 0x7fae2006d0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fae4009cfa0 tx=0x7fae28007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.419+0000 7fae36ffd640 1 -- 172.21.15.47:0/1882890913 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fae300df580 con 0x7fae40069ce0 2024-03-23T09:41:56.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.665+0000 7fae44db1640 1 -- 172.21.15.47:0/1882890913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fae400672f0 con 0x7fae40069ce0 2024-03-23T09:41:56.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.669+0000 7fae36ffd640 1 -- 172.21.15.47:0/1882890913 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7fae300775e0 con 0x7fae40069ce0 2024-03-23T09:41:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.671+0000 7fae44db1640 1 -- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae2006ac00 msgr2=0x7fae2006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.671+0000 7fae44db1640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae2006ac00 0x7fae2006d0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fae4009cfa0 tx=0x7fae28007ca0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.671+0000 7fae44db1640 1 -- 172.21.15.47:0/1882890913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae40069ce0 msgr2=0x7fae400c5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.671+0000 7fae44db1640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae40069ce0 0x7fae400c5090 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fae30009e50 tx=0x7fae30004a40 comp rx=0 tx=0).stop 2024-03-23T09:41:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.671+0000 7fae44db1640 1 -- 172.21.15.47:0/1882890913 shutdown_connections 2024-03-23T09:41:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.671+0000 7fae44db1640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae2006ac00 0x7fae2006d0c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.671+0000 7fae44db1640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae4015a610 0x7fae400c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.671+0000 7fae44db1640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae400cd020 0x7fae401f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.671+0000 7fae44db1640 1 --2- 172.21.15.47:0/1882890913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae40069ce0 0x7fae400c5090 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.671+0000 7fae44db1640 1 -- 172.21.15.47:0/1882890913 >> 172.21.15.47:0/1882890913 conn(0x7fae400cb7c0 msgr2=0x7fae400afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:56.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.671+0000 7fae44db1640 1 -- 172.21.15.47:0/1882890913 shutdown_connections 2024-03-23T09:41:56.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.671+0000 7fae44db1640 1 -- 172.21.15.47:0/1882890913 wait complete. 2024-03-23T09:41:56.681 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-23T09:41:56.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.783+0000 7f8379dfd640 1 Processor -- start 2024-03-23T09:41:56.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.783+0000 7f8379dfd640 1 -- start start 2024-03-23T09:41:56.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.784+0000 7f8379dfd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83740b8350 0x7f83740cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.784+0000 7f8379dfd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83740ce090 con 0x7f837415ca90 2024-03-23T09:41:56.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.784+0000 7f8379dfd640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83740d11e0 con 0x7f83740cdc40 2024-03-23T09:41:56.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.784+0000 7f8379dfd640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8374160f60 con 0x7f83740b8350 2024-03-23T09:41:56.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.784+0000 7f8372ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83740b8350 0x7f83740cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.784+0000 7f83737fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f837415ca90 0x7f8374160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58430/0 (socket says 172.21.15.47:58430) 2024-03-23T09:41:56.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.784+0000 7f8372ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83740b8350 0x7f83740cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48966/0 (socket says 172.21.15.47:48966) 2024-03-23T09:41:56.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.784+0000 7f83737fe640 1 -- 172.21.15.47:0/2494239742 learned_addr learned my addr 172.21.15.47:0/2494239742 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:56.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.784+0000 7f8372ffd640 1 -- 172.21.15.47:0/2494239742 >> v1:172.21.15.175:6789/0 conn(0x7f83740cdc40 legacy=0x7f83740d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:56.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.784+0000 7f8372ffd640 1 -- 172.21.15.47:0/2494239742 >> v1:172.21.15.47:6789/0 conn(0x7f837415ca90 legacy=0x7f8374160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.784+0000 7f8372ffd640 1 -- 172.21.15.47:0/2494239742 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8374161240 con 0x7f83740b8350 2024-03-23T09:41:56.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.785+0000 7f8372ffd640 1 --2- 172.21.15.47:0/2494239742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83740b8350 0x7f83740cd700 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f83680097a0 tx=0x7f836802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ad47bca5ce9609 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:56.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.785+0000 7f83727fc640 1 -- 172.21.15.47:0/2494239742 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836802e070 con 0x7f83740b8350 2024-03-23T09:41:56.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.785+0000 7f83727fc640 1 -- 172.21.15.47:0/2494239742 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83680315a0 con 0x7f83740b8350 2024-03-23T09:41:56.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.785+0000 7f83727fc640 1 -- 172.21.15.47:0/2494239742 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368031930 con 0x7f83740b8350 2024-03-23T09:41:56.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.786+0000 7f8379dfd640 1 -- 172.21.15.47:0/2494239742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83740b8350 msgr2=0x7f83740cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.786+0000 7f8379dfd640 1 --2- 172.21.15.47:0/2494239742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83740b8350 0x7f83740cd700 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f83680097a0 tx=0x7f836802b840 comp rx=0 tx=0).stop 2024-03-23T09:41:56.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.786+0000 7f8379dfd640 1 -- 172.21.15.47:0/2494239742 shutdown_connections 2024-03-23T09:41:56.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.786+0000 7f8379dfd640 1 --2- 172.21.15.47:0/2494239742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83740b8350 0x7f83740cd700 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.786+0000 7f8379dfd640 1 -- 172.21.15.47:0/2494239742 >> 172.21.15.47:0/2494239742 conn(0x7f83740659d0 msgr2=0x7f8374065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:56.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.786+0000 7f8379dfd640 1 -- 172.21.15.47:0/2494239742 shutdown_connections 2024-03-23T09:41:56.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.786+0000 7f8379dfd640 1 -- 172.21.15.47:0/2494239742 wait complete. 2024-03-23T09:41:56.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.787+0000 7f8379dfd640 1 Processor -- start 2024-03-23T09:41:56.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.787+0000 7f8379dfd640 1 -- start start 2024-03-23T09:41:56.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f8379dfd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83740b8350 0x7f83741f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f8379dfd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83740cdc40 0x7f83740cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f8379dfd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f837415ca90 0x7f83740cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f8379dfd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83740b2420 con 0x7f83740b8350 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f8379dfd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83740b2670 con 0x7f837415ca90 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f8379dfd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83740b2950 con 0x7f83740cdc40 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f83737fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83740b8350 0x7f83741f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f83737fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83740b8350 0x7f83741f2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48968/0 (socket says 172.21.15.47:48968) 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f83737fe640 1 -- 172.21.15.47:0/755773841 learned_addr learned my addr 172.21.15.47:0/755773841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f8372ffd640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83740cdc40 0x7f83740cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f8373fff640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f837415ca90 0x7f83740cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f83737fe640 1 -- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83740cdc40 msgr2=0x7f83740cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f83737fe640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83740cdc40 0x7f83740cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f83737fe640 1 -- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f837415ca90 msgr2=0x7f83740cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f83737fe640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f837415ca90 0x7f83740cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:56.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f83737fe640 1 -- 172.21.15.47:0/755773841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8368009400 con 0x7f83740b8350 2024-03-23T09:41:56.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f8372ffd640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83740cdc40 0x7f83740cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:56.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f8373fff640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f837415ca90 0x7f83740cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:56.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.788+0000 7f83737fe640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83740b8350 0x7f83741f2d90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f836400c4a0 tx=0x7f836400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:56.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.789+0000 7f835bfff640 1 -- 172.21.15.47:0/755773841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364005ce0 con 0x7f83740b8350 2024-03-23T09:41:56.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.789+0000 7f835bfff640 1 -- 172.21.15.47:0/755773841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8364005e40 con 0x7f83740b8350 2024-03-23T09:41:56.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.789+0000 7f8379dfd640 1 -- 172.21.15.47:0/755773841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83740b2c30 con 0x7f83740b8350 2024-03-23T09:41:56.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.789+0000 7f835bfff640 1 -- 172.21.15.47:0/755773841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364007270 con 0x7f83740b8350 2024-03-23T09:41:56.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.789+0000 7f8379dfd640 1 -- 172.21.15.47:0/755773841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83740b3430 con 0x7f83740b8350 2024-03-23T09:41:56.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.790+0000 7f8379dfd640 1 -- 172.21.15.47:0/755773841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8374072360 con 0x7f83740b8350 2024-03-23T09:41:56.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.790+0000 7f835bfff640 1 -- 172.21.15.47:0/755773841 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8364007490 con 0x7f83740b8350 2024-03-23T09:41:56.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.791+0000 7f835bfff640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f836006ac00 0x7f836006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:56.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.791+0000 7f835bfff640 1 -- 172.21.15.47:0/755773841 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8364088250 con 0x7f83740b8350 2024-03-23T09:41:56.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.791+0000 7f8372ffd640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f836006ac00 0x7f836006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:56.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.791+0000 7f8372ffd640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f836006ac00 0x7f836006d0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8368036000 tx=0x7f836803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:56.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:56.795+0000 7f835bfff640 1 -- 172.21.15.47:0/755773841 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f83640538c0 con 0x7f83740b8350 2024-03-23T09:41:57.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.047+0000 7f8379dfd640 1 -- 172.21.15.47:0/755773841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f83740b3150 con 0x7f83740b8350 2024-03-23T09:41:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.051+0000 7f835bfff640 1 -- 172.21.15.47:0/755773841 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f83740b3150 con 0x7f83740b8350 2024-03-23T09:41:57.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.053+0000 7f8379dfd640 1 -- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f836006ac00 msgr2=0x7f836006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.053+0000 7f8379dfd640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f836006ac00 0x7f836006d0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8368036000 tx=0x7f836803d040 comp rx=0 tx=0).stop 2024-03-23T09:41:57.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.053+0000 7f8379dfd640 1 -- 172.21.15.47:0/755773841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83740b8350 msgr2=0x7f83741f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.053+0000 7f8379dfd640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83740b8350 0x7f83741f2d90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f836400c4a0 tx=0x7f836400c960 comp rx=0 tx=0).stop 2024-03-23T09:41:57.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.053+0000 7f8379dfd640 1 -- 172.21.15.47:0/755773841 shutdown_connections 2024-03-23T09:41:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.053+0000 7f8379dfd640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f836006ac00 0x7f836006d0c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.053+0000 7f8379dfd640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f837415ca90 0x7f83740cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.053+0000 7f8379dfd640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83740cdc40 0x7f83740cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.053+0000 7f8379dfd640 1 --2- 172.21.15.47:0/755773841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83740b8350 0x7f83741f2d90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.053+0000 7f8379dfd640 1 -- 172.21.15.47:0/755773841 >> 172.21.15.47:0/755773841 conn(0x7f83740659d0 msgr2=0x7f837415d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.053+0000 7f8379dfd640 1 -- 172.21.15.47:0/755773841 shutdown_connections 2024-03-23T09:41:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.053+0000 7f8379dfd640 1 -- 172.21.15.47:0/755773841 wait complete. 2024-03-23T09:41:57.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:57.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.164+0000 7f62ed3ea640 1 Processor -- start 2024-03-23T09:41:57.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.164+0000 7f62ed3ea640 1 -- start start 2024-03-23T09:41:57.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.165+0000 7f62ed3ea640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62e80d23a0 0x7f62e80d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.165+0000 7f62ed3ea640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f62e80b8300 0x7f62e80d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.165+0000 7f62ed3ea640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e80d10a0 con 0x7f62e815cee0 2024-03-23T09:41:57.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.165+0000 7f62ed3ea640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e80d1300 con 0x7f62e80d23a0 2024-03-23T09:41:57.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.165+0000 7f62ed3ea640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e80d15e0 con 0x7f62e80b8300 2024-03-23T09:41:57.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.165+0000 7f62e67fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62e80d23a0 0x7f62e80d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.165+0000 7f62e67fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62e80d23a0 0x7f62e80d2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48980/0 (socket says 172.21.15.47:48980) 2024-03-23T09:41:57.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.165+0000 7f62e67fc640 1 -- 172.21.15.47:0/1284728310 learned_addr learned my addr 172.21.15.47:0/1284728310 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:57.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.165+0000 7f62e77fe640 1 --2- 172.21.15.47:0/1284728310 >> v2:172.21.15.175:3300/0 conn(0x7f62e80b8300 0x7f62e80d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.165+0000 7f62e67fc640 1 -- 172.21.15.47:0/1284728310 >> v2:172.21.15.175:3300/0 conn(0x7f62e80b8300 msgr2=0x7f62e80d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:57.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.165+0000 7f62e67fc640 1 --2- 172.21.15.47:0/1284728310 >> v2:172.21.15.175:3300/0 conn(0x7f62e80b8300 0x7f62e80d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.165+0000 7f62e67fc640 1 -- 172.21.15.47:0/1284728310 >> v1:172.21.15.175:6789/0 conn(0x7f62e815cee0 legacy=0x7f62e80d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.165+0000 7f62e67fc640 1 -- 172.21.15.47:0/1284728310 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e80d18c0 con 0x7f62e80d23a0 2024-03-23T09:41:57.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.166+0000 7f62e67fc640 1 --2- 172.21.15.47:0/1284728310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80d23a0 0x7f62e80d2870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f62e80c9940 tx=0x7f62d002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d6e23a45e2b30f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:57.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.166+0000 7f62e5ffb640 1 -- 172.21.15.47:0/1284728310 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0030860 con 0x7f62e80d23a0 2024-03-23T09:41:57.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.166+0000 7f62e5ffb640 1 -- 172.21.15.47:0/1284728310 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d00309c0 con 0x7f62e80d23a0 2024-03-23T09:41:57.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.166+0000 7f62e5ffb640 1 -- 172.21.15.47:0/1284728310 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0030d50 con 0x7f62e80d23a0 2024-03-23T09:41:57.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.167+0000 7f62ed3ea640 1 -- 172.21.15.47:0/1284728310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80d23a0 msgr2=0x7f62e80d2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.167+0000 7f62ed3ea640 1 --2- 172.21.15.47:0/1284728310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80d23a0 0x7f62e80d2870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f62e80c9940 tx=0x7f62d002b9a0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.167+0000 7f62ed3ea640 1 -- 172.21.15.47:0/1284728310 shutdown_connections 2024-03-23T09:41:57.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.167+0000 7f62ed3ea640 1 --2- 172.21.15.47:0/1284728310 >> v2:172.21.15.175:3300/0 conn(0x7f62e80b8300 0x7f62e80d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.167+0000 7f62ed3ea640 1 --2- 172.21.15.47:0/1284728310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80d23a0 0x7f62e80d2870 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.167+0000 7f62ed3ea640 1 -- 172.21.15.47:0/1284728310 >> 172.21.15.47:0/1284728310 conn(0x7f62e8065df0 msgr2=0x7f62e8066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.167+0000 7f62ed3ea640 1 -- 172.21.15.47:0/1284728310 shutdown_connections 2024-03-23T09:41:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.167+0000 7f62ed3ea640 1 -- 172.21.15.47:0/1284728310 wait complete. 2024-03-23T09:41:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.168+0000 7f62ed3ea640 1 Processor -- start 2024-03-23T09:41:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.168+0000 7f62ed3ea640 1 -- start start 2024-03-23T09:41:57.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62ed3ea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80b8300 0x7f62e80c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62ed3ea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62e80d23a0 0x7f62e80c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62ed3ea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62e815cee0 0x7f62e8157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62ed3ea640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e80c8040 con 0x7f62e80b8300 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62ed3ea640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e80c82e0 con 0x7f62e80d23a0 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62ed3ea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e80c85c0 con 0x7f62e815cee0 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62e6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80b8300 0x7f62e80c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62e6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80b8300 0x7f62e80c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48996/0 (socket says 172.21.15.47:48996) 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62e6ffd640 1 -- 172.21.15.47:0/117203874 learned_addr learned my addr 172.21.15.47:0/117203874 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62e67fc640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62e80d23a0 0x7f62e80c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62e77fe640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62e815cee0 0x7f62e8157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62e6ffd640 1 -- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62e815cee0 msgr2=0x7f62e8157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62e6ffd640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62e815cee0 0x7f62e8157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62e6ffd640 1 -- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62e80d23a0 msgr2=0x7f62e80c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62e6ffd640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62e80d23a0 0x7f62e80c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62e6ffd640 1 -- 172.21.15.47:0/117203874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00094d0 con 0x7f62e80b8300 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.169+0000 7f62e6ffd640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80b8300 0x7f62e80c6240 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f62e000c400 tx=0x7f62e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.170+0000 7f62df7fe640 1 -- 172.21.15.47:0/117203874 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e00052d0 con 0x7f62e80b8300 2024-03-23T09:41:57.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.170+0000 7f62df7fe640 1 -- 172.21.15.47:0/117203874 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e0005430 con 0x7f62e80b8300 2024-03-23T09:41:57.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.170+0000 7f62ed3ea640 1 -- 172.21.15.47:0/117203874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e80c88a0 con 0x7f62e80b8300 2024-03-23T09:41:57.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.170+0000 7f62df7fe640 1 -- 172.21.15.47:0/117203874 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0003830 con 0x7f62e80b8300 2024-03-23T09:41:57.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.170+0000 7f62ed3ea640 1 -- 172.21.15.47:0/117203874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e80c8e60 con 0x7f62e80b8300 2024-03-23T09:41:57.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.171+0000 7f62ed3ea640 1 -- 172.21.15.47:0/117203874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e8072360 con 0x7f62e80b8300 2024-03-23T09:41:57.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.171+0000 7f62df7fe640 1 -- 172.21.15.47:0/117203874 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f62e0003990 con 0x7f62e80b8300 2024-03-23T09:41:57.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.171+0000 7f62df7fe640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f62c806ac30 0x7f62c806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.172+0000 7f62df7fe640 1 -- 172.21.15.47:0/117203874 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f62e0088570 con 0x7f62e80b8300 2024-03-23T09:41:57.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.172+0000 7f62e67fc640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f62c806ac30 0x7f62c806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.172+0000 7f62e67fc640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f62c806ac30 0x7f62c806d0f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f62d0036000 tx=0x7f62d0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:57.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.176+0000 7f62df7fe640 1 -- 172.21.15.47:0/117203874 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f62e0053aa0 con 0x7f62e80b8300 2024-03-23T09:41:57.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.356+0000 7f62ed3ea640 1 -- 172.21.15.47:0/117203874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62e8066be0 con 0x7f62e80b8300 2024-03-23T09:41:57.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.356+0000 7f62df7fe640 1 -- 172.21.15.47:0/117203874 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12389 (secure 0 0 0) 0x7f62e00570c0 con 0x7f62e80b8300 2024-03-23T09:41:57.356 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:57.356 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:54.389930+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:41:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.358+0000 7f62ed3ea640 1 -- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f62c806ac30 msgr2=0x7f62c806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.358+0000 7f62ed3ea640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f62c806ac30 0x7f62c806d0f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f62d0036000 tx=0x7f62d0008030 comp rx=0 tx=0).stop 2024-03-23T09:41:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.358+0000 7f62ed3ea640 1 -- 172.21.15.47:0/117203874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80b8300 msgr2=0x7f62e80c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.358+0000 7f62ed3ea640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80b8300 0x7f62e80c6240 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f62e000c400 tx=0x7f62e000c8c0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.358+0000 7f62ed3ea640 1 -- 172.21.15.47:0/117203874 shutdown_connections 2024-03-23T09:41:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.358+0000 7f62ed3ea640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f62c806ac30 0x7f62c806d0f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.358+0000 7f62ed3ea640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62e815cee0 0x7f62e8157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.358+0000 7f62ed3ea640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62e80d23a0 0x7f62e80c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.358+0000 7f62ed3ea640 1 --2- 172.21.15.47:0/117203874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80b8300 0x7f62e80c6240 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.358+0000 7f62ed3ea640 1 -- 172.21.15.47:0/117203874 >> 172.21.15.47:0/117203874 conn(0x7f62e8065df0 msgr2=0x7f62e80cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.359+0000 7f62ed3ea640 1 -- 172.21.15.47:0/117203874 shutdown_connections 2024-03-23T09:41:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.359+0000 7f62ed3ea640 1 -- 172.21.15.47:0/117203874 wait complete. 2024-03-23T09:41:57.369 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:41:57.370 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:41:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.473+0000 7faee1188640 1 Processor -- start 2024-03-23T09:41:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.474+0000 7faee1188640 1 -- start start 2024-03-23T09:41:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.474+0000 7faee1188640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faedc0b1c60 0x7faedc0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.474+0000 7faee1188640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faedc161450 0x7faedc0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.474+0000 7faee1188640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faedc0cffc0 con 0x7faedc06b690 2024-03-23T09:41:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.474+0000 7faee1188640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faedc0cf3b0 con 0x7faedc161450 2024-03-23T09:41:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.474+0000 7faee1188640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faedc0cf690 con 0x7faedc0b1c60 2024-03-23T09:41:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.474+0000 7faedb59e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faedc161450 0x7faedc0cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.474+0000 7faedad9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7faedc06b690 0x7faedc06ba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58432/0 (socket says 172.21.15.47:58432) 2024-03-23T09:41:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.474+0000 7faedad9d640 1 -- 172.21.15.47:0/20043180 learned_addr learned my addr 172.21.15.47:0/20043180 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.475+0000 7faed3fff640 1 --2- 172.21.15.47:0/20043180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faedc0b1c60 0x7faedc0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.475+0000 7faedb59e640 1 -- 172.21.15.47:0/20043180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faedc0b1c60 msgr2=0x7faedc0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.475+0000 7faedb59e640 1 --2- 172.21.15.47:0/20043180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faedc0b1c60 0x7faedc0ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.475+0000 7faedb59e640 1 -- 172.21.15.47:0/20043180 >> v1:172.21.15.47:6789/0 conn(0x7faedc06b690 legacy=0x7faedc06ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.475+0000 7faedb59e640 1 -- 172.21.15.47:0/20043180 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc0cf970 con 0x7faedc161450 2024-03-23T09:41:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.475+0000 7faed3fff640 1 --2- 172.21.15.47:0/20043180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faedc0b1c60 0x7faedc0ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.475+0000 7faedb59e640 1 --2- 172.21.15.47:0/20043180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faedc161450 0x7faedc0cad30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faed4009d90 tx=0x7faed402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7822fcb2a16e6838 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.476+0000 7faeda59c640 1 -- 172.21.15.47:0/20043180 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed402cce0 con 0x7faedc161450 2024-03-23T09:41:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.476+0000 7faeda59c640 1 -- 172.21.15.47:0/20043180 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed402ce40 con 0x7faedc161450 2024-03-23T09:41:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.476+0000 7faeda59c640 1 -- 172.21.15.47:0/20043180 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4034660 con 0x7faedc161450 2024-03-23T09:41:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.476+0000 7faee1188640 1 -- 172.21.15.47:0/20043180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faedc161450 msgr2=0x7faedc0cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.476+0000 7faee1188640 1 --2- 172.21.15.47:0/20043180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faedc161450 0x7faedc0cad30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faed4009d90 tx=0x7faed402bf30 comp rx=0 tx=0).stop 2024-03-23T09:41:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.476+0000 7faee1188640 1 -- 172.21.15.47:0/20043180 shutdown_connections 2024-03-23T09:41:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.476+0000 7faee1188640 1 --2- 172.21.15.47:0/20043180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faedc161450 0x7faedc0cad30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.476+0000 7faee1188640 1 --2- 172.21.15.47:0/20043180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faedc0b1c60 0x7faedc0ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.476+0000 7faee1188640 1 -- 172.21.15.47:0/20043180 >> 172.21.15.47:0/20043180 conn(0x7faedc0b21d0 msgr2=0x7faedc0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.476+0000 7faee1188640 1 -- 172.21.15.47:0/20043180 shutdown_connections 2024-03-23T09:41:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.476+0000 7faee1188640 1 -- 172.21.15.47:0/20043180 wait complete. 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.477+0000 7faee1188640 1 Processor -- start 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.477+0000 7faee1188640 1 -- start start 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.478+0000 7faee1188640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faedc06b690 0x7faedc0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.478+0000 7faee1188640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faedc161450 0x7faedc0c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.478+0000 7faee1188640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faedc0c4720 0x7faedc146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.478+0000 7faee1188640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc0b3f70 con 0x7faedc161450 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.478+0000 7faee1188640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc0b4d60 con 0x7faedc0c4720 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.478+0000 7faee1188640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faedc0b4150 con 0x7faedc06b690 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.478+0000 7faed3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faedc161450 0x7faedc0c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.478+0000 7faed3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faedc161450 0x7faedc0c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49014/0 (socket says 172.21.15.47:49014) 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.478+0000 7faed3fff640 1 -- 172.21.15.47:0/1463005768 learned_addr learned my addr 172.21.15.47:0/1463005768 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faedb59e640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faedc0c4720 0x7faedc146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faedad9d640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faedc06b690 0x7faedc0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faed3fff640 1 -- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faedc06b690 msgr2=0x7faedc0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faed3fff640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faedc06b690 0x7faedc0ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faed3fff640 1 -- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faedc0c4720 msgr2=0x7faedc146590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faed3fff640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faedc0c4720 0x7faedc146590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faed3fff640 1 -- 172.21.15.47:0/1463005768 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed4009a70 con 0x7faedc161450 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faedad9d640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faedc06b690 0x7faedc0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faed3fff640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faedc161450 0x7faedc0c41e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faec400c7f0 tx=0x7faec400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faed37fe640 1 -- 172.21.15.47:0/1463005768 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4007a30 con 0x7faedc161450 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faed37fe640 1 -- 172.21.15.47:0/1463005768 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec4007b90 con 0x7faedc161450 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faee1188640 1 -- 172.21.15.47:0/1463005768 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faedc0b4430 con 0x7faedc161450 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faed37fe640 1 -- 172.21.15.47:0/1463005768 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec400ce40 con 0x7faedc161450 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.479+0000 7faee1188640 1 -- 172.21.15.47:0/1463005768 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faedc0b49f0 con 0x7faedc161450 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.480+0000 7faee1188640 1 -- 172.21.15.47:0/1463005768 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faedc0b4f10 con 0x7faedc161450 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.480+0000 7faed37fe640 1 -- 172.21.15.47:0/1463005768 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7faec40113f0 con 0x7faedc161450 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.481+0000 7faed37fe640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faebc06ac00 0x7faebc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.481+0000 7faed37fe640 1 -- 172.21.15.47:0/1463005768 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faec4089170 con 0x7faedc161450 2024-03-23T09:41:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.481+0000 7faedad9d640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faebc06ac00 0x7faebc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.482+0000 7faedad9d640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faebc06ac00 0x7faebc06d0c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faecc005880 tx=0x7faecc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:57.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.485+0000 7faed37fe640 1 -- 172.21.15.47:0/1463005768 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7faedc0b4f10 con 0x7faedc161450 2024-03-23T09:41:57.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.663+0000 7faee1188640 1 -- 172.21.15.47:0/1463005768 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faedc0b4ed0 con 0x7faedc161450 2024-03-23T09:41:57.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.664+0000 7faed37fe640 1 -- 172.21.15.47:0/1463005768 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7faec401f020 con 0x7faedc161450 2024-03-23T09:41:57.664 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:41:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.666+0000 7faee1188640 1 -- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faebc06ac00 msgr2=0x7faebc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.666+0000 7faee1188640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faebc06ac00 0x7faebc06d0c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faecc005880 tx=0x7faecc009040 comp rx=0 tx=0).stop 2024-03-23T09:41:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.666+0000 7faee1188640 1 -- 172.21.15.47:0/1463005768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faedc161450 msgr2=0x7faedc0c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.666+0000 7faee1188640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faedc161450 0x7faedc0c41e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faec400c7f0 tx=0x7faec400cbf0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.666+0000 7faee1188640 1 -- 172.21.15.47:0/1463005768 shutdown_connections 2024-03-23T09:41:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.666+0000 7faee1188640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faebc06ac00 0x7faebc06d0c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.666+0000 7faee1188640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faedc0c4720 0x7faedc146590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.666+0000 7faee1188640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faedc161450 0x7faedc0c41e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.666+0000 7faee1188640 1 --2- 172.21.15.47:0/1463005768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faedc06b690 0x7faedc0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.666+0000 7faee1188640 1 -- 172.21.15.47:0/1463005768 >> 172.21.15.47:0/1463005768 conn(0x7faedc0b21d0 msgr2=0x7faedc15fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.666+0000 7faee1188640 1 -- 172.21.15.47:0/1463005768 shutdown_connections 2024-03-23T09:41:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.666+0000 7faee1188640 1 -- 172.21.15.47:0/1463005768 wait complete. 2024-03-23T09:41:57.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T09:41:57.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.777+0000 7fdf2311b640 1 Processor -- start 2024-03-23T09:41:57.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.777+0000 7fdf2311b640 1 -- start start 2024-03-23T09:41:57.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.778+0000 7fdf2311b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdf1c0cf210 0x7fdf1c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.778+0000 7fdf2311b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf1c0b5a90 0x7fdf1c0b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.778+0000 7fdf2311b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf1c0cfbd0 con 0x7fdf1c15cd00 2024-03-23T09:41:57.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.778+0000 7fdf2311b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1c15d0e0 con 0x7fdf1c0b5a90 2024-03-23T09:41:57.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.778+0000 7fdf2311b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1c0d1540 con 0x7fdf1c0cf210 2024-03-23T09:41:57.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.778+0000 7fdf216b8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf1c0b5a90 0x7fdf1c0b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.778+0000 7fdf216b8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf1c0b5a90 0x7fdf1c0b5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49016/0 (socket says 172.21.15.47:49016) 2024-03-23T09:41:57.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.778+0000 7fdf216b8640 1 -- 172.21.15.47:0/1519780912 learned_addr learned my addr 172.21.15.47:0/1519780912 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:57.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.778+0000 7fdf1bfff640 1 --2- 172.21.15.47:0/1519780912 >> v2:172.21.15.175:3300/0 conn(0x7fdf1c0cf210 0x7fdf1c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.778+0000 7fdf216b8640 1 -- 172.21.15.47:0/1519780912 >> v2:172.21.15.175:3300/0 conn(0x7fdf1c0cf210 msgr2=0x7fdf1c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:57.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.778+0000 7fdf216b8640 1 --2- 172.21.15.47:0/1519780912 >> v2:172.21.15.175:3300/0 conn(0x7fdf1c0cf210 0x7fdf1c0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.779+0000 7fdf216b8640 1 -- 172.21.15.47:0/1519780912 >> v1:172.21.15.175:6789/0 conn(0x7fdf1c15cd00 legacy=0x7fdf1c0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:57.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.779+0000 7fdf216b8640 1 -- 172.21.15.47:0/1519780912 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c0d1820 con 0x7fdf1c0b5a90 2024-03-23T09:41:57.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.779+0000 7fdf216b8640 1 --2- 172.21.15.47:0/1519780912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1c0b5a90 0x7fdf1c0b5ee0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0b5520 tx=0x7fdf0c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f12c38a757dbd3e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:57.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.779+0000 7fdf1b7fe640 1 -- 172.21.15.47:0/1519780912 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0317e0 con 0x7fdf1c0b5a90 2024-03-23T09:41:57.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.779+0000 7fdf1b7fe640 1 -- 172.21.15.47:0/1519780912 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c031940 con 0x7fdf1c0b5a90 2024-03-23T09:41:57.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.779+0000 7fdf1b7fe640 1 -- 172.21.15.47:0/1519780912 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c031cd0 con 0x7fdf1c0b5a90 2024-03-23T09:41:57.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.780+0000 7fdf2311b640 1 -- 172.21.15.47:0/1519780912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1c0b5a90 msgr2=0x7fdf1c0b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.780+0000 7fdf2311b640 1 --2- 172.21.15.47:0/1519780912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1c0b5a90 0x7fdf1c0b5ee0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0b5520 tx=0x7fdf0c02b880 comp rx=0 tx=0).stop 2024-03-23T09:41:57.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.780+0000 7fdf2311b640 1 -- 172.21.15.47:0/1519780912 shutdown_connections 2024-03-23T09:41:57.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.780+0000 7fdf2311b640 1 --2- 172.21.15.47:0/1519780912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1c0b5a90 0x7fdf1c0b5ee0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.780+0000 7fdf2311b640 1 --2- 172.21.15.47:0/1519780912 >> v2:172.21.15.175:3300/0 conn(0x7fdf1c0cf210 0x7fdf1c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.780+0000 7fdf2311b640 1 -- 172.21.15.47:0/1519780912 >> 172.21.15.47:0/1519780912 conn(0x7fdf1c06b160 msgr2=0x7fdf1c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:57.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.780+0000 7fdf2311b640 1 -- 172.21.15.47:0/1519780912 shutdown_connections 2024-03-23T09:41:57.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.780+0000 7fdf2311b640 1 -- 172.21.15.47:0/1519780912 wait complete. 2024-03-23T09:41:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.781+0000 7fdf2311b640 1 Processor -- start 2024-03-23T09:41:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.781+0000 7fdf2311b640 1 -- start start 2024-03-23T09:41:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf2311b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf1c0b5a90 0x7fdf1c158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf2311b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf1c0cf210 0x7fdf1c0b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf2311b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1c15cd00 0x7fdf1c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf2311b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0b2850 con 0x7fdf1c15cd00 2024-03-23T09:41:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf2311b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0b3890 con 0x7fdf1c0cf210 2024-03-23T09:41:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf2311b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0b2ab0 con 0x7fdf1c0b5a90 2024-03-23T09:41:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf216b8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1c15cd00 0x7fdf1c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf216b8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1c15cd00 0x7fdf1c0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49018/0 (socket says 172.21.15.47:49018) 2024-03-23T09:41:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf216b8640 1 -- 172.21.15.47:0/309215112 learned_addr learned my addr 172.21.15.47:0/309215112 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf216b8640 1 -- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf1c0b5a90 msgr2=0x7fdf1c158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf20eb7640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf1c0b5a90 0x7fdf1c158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf1bfff640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf1c0cf210 0x7fdf1c0b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf216b8640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf1c0b5a90 0x7fdf1c158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf216b8640 1 -- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf1c0cf210 msgr2=0x7fdf1c0b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf216b8640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf1c0cf210 0x7fdf1c0b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf216b8640 1 -- 172.21.15.47:0/309215112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0094d0 con 0x7fdf1c15cd00 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf1bfff640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf1c0cf210 0x7fdf1c0b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.782+0000 7fdf20eb7640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf1c0b5a90 0x7fdf1c158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.783+0000 7fdf216b8640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1c15cd00 0x7fdf1c0cbc00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0090b0 tx=0x7fdf0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.783+0000 7fdf197fa640 1 -- 172.21.15.47:0/309215112 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c032880 con 0x7fdf1c15cd00 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.783+0000 7fdf197fa640 1 -- 172.21.15.47:0/309215112 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c032a00 con 0x7fdf1c15cd00 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.783+0000 7fdf2311b640 1 -- 172.21.15.47:0/309215112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c0b2d90 con 0x7fdf1c15cd00 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.783+0000 7fdf197fa640 1 -- 172.21.15.47:0/309215112 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c032db0 con 0x7fdf1c15cd00 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.783+0000 7fdf2311b640 1 -- 172.21.15.47:0/309215112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1c0b3350 con 0x7fdf1c15cd00 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.784+0000 7fdf2311b640 1 -- 172.21.15.47:0/309215112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1c072360 con 0x7fdf1c15cd00 2024-03-23T09:41:57.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.784+0000 7fdf197fa640 1 -- 172.21.15.47:0/309215112 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdf0c04d3f0 con 0x7fdf1c15cd00 2024-03-23T09:41:57.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.784+0000 7fdf197fa640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdf0806ac30 0x7fdf0806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:57.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.785+0000 7fdf197fa640 1 -- 172.21.15.47:0/309215112 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdf0c0aace0 con 0x7fdf1c15cd00 2024-03-23T09:41:57.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.785+0000 7fdf20eb7640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdf0806ac30 0x7fdf0806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:57.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.785+0000 7fdf20eb7640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdf0806ac30 0x7fdf0806d0f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf14007e90 tx=0x7fdf140073a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:57.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.789+0000 7fdf197fa640 1 -- 172.21.15.47:0/309215112 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdf0c076210 con 0x7fdf1c15cd00 2024-03-23T09:41:57.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.969+0000 7fdf2311b640 1 -- 172.21.15.47:0/309215112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdf1c072360 con 0x7fdf1c15cd00 2024-03-23T09:41:57.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.969+0000 7fdf197fa640 1 -- 172.21.15.47:0/309215112 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdf0c079830 con 0x7fdf1c15cd00 2024-03-23T09:41:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:41:57.969 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-23T09:41:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:41:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-03-23T09:41:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-03-23T09:41:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:41:57.970 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-23T09:41:57.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.971+0000 7fdf2311b640 1 -- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdf0806ac30 msgr2=0x7fdf0806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.971+0000 7fdf2311b640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdf0806ac30 0x7fdf0806d0f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf14007e90 tx=0x7fdf140073a0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.971+0000 7fdf2311b640 1 -- 172.21.15.47:0/309215112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1c15cd00 msgr2=0x7fdf1c0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:57.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.971+0000 7fdf2311b640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1c15cd00 0x7fdf1c0cbc00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0090b0 tx=0x7fdf0c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.971+0000 7fdf2311b640 1 -- 172.21.15.47:0/309215112 shutdown_connections 2024-03-23T09:41:57.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.971+0000 7fdf2311b640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdf0806ac30 0x7fdf0806d0f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.971+0000 7fdf2311b640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1c15cd00 0x7fdf1c0cbc00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.971+0000 7fdf2311b640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf1c0cf210 0x7fdf1c0b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.971+0000 7fdf2311b640 1 --2- 172.21.15.47:0/309215112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf1c0b5a90 0x7fdf1c158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:57.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.971+0000 7fdf2311b640 1 -- 172.21.15.47:0/309215112 >> 172.21.15.47:0/309215112 conn(0x7fdf1c06b160 msgr2=0x7fdf1c15bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:57.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.971+0000 7fdf2311b640 1 -- 172.21.15.47:0/309215112 shutdown_connections 2024-03-23T09:41:57.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:57.971+0000 7fdf2311b640 1 -- 172.21.15.47:0/309215112 wait complete. 2024-03-23T09:41:57.981 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T09:41:57.982 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T09:41:58.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.087+0000 7fe25dc26640 1 Processor -- start 2024-03-23T09:41:58.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.087+0000 7fe25dc26640 1 -- start start 2024-03-23T09:41:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe25dc26640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe25815a700 0x7fe2580cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe25dc26640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe25815d0c0 0x7fe25815d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe25dc26640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580d27f0 0x7fe2580d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe25dc26640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe25815aae0 con 0x7fe25815a700 2024-03-23T09:41:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe25dc26640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2580d3160 con 0x7fe25815d0c0 2024-03-23T09:41:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe25dc26640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25815dbd0 con 0x7fe2580d27f0 2024-03-23T09:41:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe2577fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe25815a700 0x7fe2580cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe2577fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe25815a700 0x7fe2580cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49020/0 (socket says 172.21.15.47:49020) 2024-03-23T09:41:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe2577fe640 1 -- 172.21.15.47:0/152616603 learned_addr learned my addr 172.21.15.47:0/152616603 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe256ffd640 1 --2- 172.21.15.47:0/152616603 >> v2:172.21.15.175:3300/0 conn(0x7fe25815d0c0 0x7fe25815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe257fff640 1 --2- 172.21.15.47:0/152616603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580d27f0 0x7fe2580d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe2577fe640 1 -- 172.21.15.47:0/152616603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580d27f0 msgr2=0x7fe2580d2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe2577fe640 1 --2- 172.21.15.47:0/152616603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580d27f0 0x7fe2580d2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe2577fe640 1 -- 172.21.15.47:0/152616603 >> v2:172.21.15.175:3300/0 conn(0x7fe25815d0c0 msgr2=0x7fe25815d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe2577fe640 1 --2- 172.21.15.47:0/152616603 >> v2:172.21.15.175:3300/0 conn(0x7fe25815d0c0 0x7fe25815d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe2577fe640 1 -- 172.21.15.47:0/152616603 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25815deb0 con 0x7fe25815a700 2024-03-23T09:41:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe256ffd640 1 --2- 172.21.15.47:0/152616603 >> v2:172.21.15.175:3300/0 conn(0x7fe25815d0c0 0x7fe25815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.088+0000 7fe257fff640 1 --2- 172.21.15.47:0/152616603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580d27f0 0x7fe2580d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:41:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.089+0000 7fe2577fe640 1 --2- 172.21.15.47:0/152616603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe25815a700 0x7fe2580cfb40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe2580cd060 tx=0x7fe24802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8272bcbe6da998 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.089+0000 7fe2567fc640 1 -- 172.21.15.47:0/152616603 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248031860 con 0x7fe25815a700 2024-03-23T09:41:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.089+0000 7fe2567fc640 1 -- 172.21.15.47:0/152616603 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2480319c0 con 0x7fe25815a700 2024-03-23T09:41:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.089+0000 7fe2567fc640 1 -- 172.21.15.47:0/152616603 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248031d50 con 0x7fe25815a700 2024-03-23T09:41:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.090+0000 7fe25dc26640 1 -- 172.21.15.47:0/152616603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe25815a700 msgr2=0x7fe2580cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.090+0000 7fe25dc26640 1 --2- 172.21.15.47:0/152616603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe25815a700 0x7fe2580cfb40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe2580cd060 tx=0x7fe24802b900 comp rx=0 tx=0).stop 2024-03-23T09:41:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.090+0000 7fe25dc26640 1 -- 172.21.15.47:0/152616603 shutdown_connections 2024-03-23T09:41:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.090+0000 7fe25dc26640 1 --2- 172.21.15.47:0/152616603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580d27f0 0x7fe2580d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.090+0000 7fe25dc26640 1 --2- 172.21.15.47:0/152616603 >> v2:172.21.15.175:3300/0 conn(0x7fe25815d0c0 0x7fe25815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.090+0000 7fe25dc26640 1 --2- 172.21.15.47:0/152616603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe25815a700 0x7fe2580cfb40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.090+0000 7fe25dc26640 1 -- 172.21.15.47:0/152616603 >> 172.21.15.47:0/152616603 conn(0x7fe25806afd0 msgr2=0x7fe25806b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.090+0000 7fe25dc26640 1 -- 172.21.15.47:0/152616603 shutdown_connections 2024-03-23T09:41:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.090+0000 7fe25dc26640 1 -- 172.21.15.47:0/152616603 wait complete. 2024-03-23T09:41:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.091+0000 7fe25dc26640 1 Processor -- start 2024-03-23T09:41:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.091+0000 7fe25dc26640 1 -- start start 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe25dc26640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580d27f0 0x7fe2580c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe25dc26640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe25815a700 0x7fe2580c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe25dc26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe25815d0c0 0x7fe2580b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe25dc26640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2580cc060 con 0x7fe25815d0c0 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe25dc26640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2580c70b0 con 0x7fe25815a700 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe25dc26640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2580c7390 con 0x7fe2580d27f0 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe257fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe25815d0c0 0x7fe2580b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe257fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe25815d0c0 0x7fe2580b2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49034/0 (socket says 172.21.15.47:49034) 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe257fff640 1 -- 172.21.15.47:0/2036876531 learned_addr learned my addr 172.21.15.47:0/2036876531 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe2577fe640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580d27f0 0x7fe2580c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe256ffd640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe25815a700 0x7fe2580c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe257fff640 1 -- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580d27f0 msgr2=0x7fe2580c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe257fff640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580d27f0 0x7fe2580c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe257fff640 1 -- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe25815a700 msgr2=0x7fe2580c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe257fff640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe25815a700 0x7fe2580c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.092+0000 7fe257fff640 1 -- 172.21.15.47:0/2036876531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2480094d0 con 0x7fe25815d0c0 2024-03-23T09:41:58.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.093+0000 7fe257fff640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe25815d0c0 0x7fe2580b2180 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe25000cef0 tx=0x7fe25000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:58.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.093+0000 7fe23ffff640 1 -- 172.21.15.47:0/2036876531 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25000d840 con 0x7fe25815d0c0 2024-03-23T09:41:58.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.093+0000 7fe23ffff640 1 -- 172.21.15.47:0/2036876531 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25000d9a0 con 0x7fe25815d0c0 2024-03-23T09:41:58.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.093+0000 7fe23ffff640 1 -- 172.21.15.47:0/2036876531 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25000dd50 con 0x7fe25815d0c0 2024-03-23T09:41:58.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.093+0000 7fe25dc26640 1 -- 172.21.15.47:0/2036876531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2580c7670 con 0x7fe25815d0c0 2024-03-23T09:41:58.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.093+0000 7fe25dc26640 1 -- 172.21.15.47:0/2036876531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2580c7c30 con 0x7fe25815d0c0 2024-03-23T09:41:58.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.094+0000 7fe25dc26640 1 -- 172.21.15.47:0/2036876531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe258072360 con 0x7fe25815d0c0 2024-03-23T09:41:58.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.094+0000 7fe23ffff640 1 -- 172.21.15.47:0/2036876531 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe25001e3b0 con 0x7fe25815d0c0 2024-03-23T09:41:58.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.095+0000 7fe23ffff640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe24406ac00 0x7fe24406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:58.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.095+0000 7fe23ffff640 1 -- 172.21.15.47:0/2036876531 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe250089a10 con 0x7fe25815d0c0 2024-03-23T09:41:58.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.095+0000 7fe2577fe640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe24406ac00 0x7fe24406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:58.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.095+0000 7fe2577fe640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe24406ac00 0x7fe24406d0c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe248036000 tx=0x7fe24803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:58.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.099+0000 7fe23ffff640 1 -- 172.21.15.47:0/2036876531 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe250054f60 con 0x7fe25815d0c0 2024-03-23T09:41:58.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.278+0000 7fe25dc26640 1 -- 172.21.15.47:0/2036876531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe2580d0380 con 0x7fe25815d0c0 2024-03-23T09:41:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.350+0000 7fe23ffff640 1 -- 172.21.15.47:0/2036876531 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe25001a070 con 0x7fe25815d0c0 2024-03-23T09:41:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-03-23T09:41:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.354+0000 7fe25dc26640 1 -- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe24406ac00 msgr2=0x7fe24406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.354+0000 7fe25dc26640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe24406ac00 0x7fe24406d0c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe248036000 tx=0x7fe24803c040 comp rx=0 tx=0).stop 2024-03-23T09:41:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.354+0000 7fe25dc26640 1 -- 172.21.15.47:0/2036876531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe25815d0c0 msgr2=0x7fe2580b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.354+0000 7fe25dc26640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe25815d0c0 0x7fe2580b2180 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe25000cef0 tx=0x7fe25000a710 comp rx=0 tx=0).stop 2024-03-23T09:41:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.354+0000 7fe25dc26640 1 -- 172.21.15.47:0/2036876531 shutdown_connections 2024-03-23T09:41:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.354+0000 7fe25dc26640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe24406ac00 0x7fe24406d0c0 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe248036000 tx=0x7fe24803c040 comp rx=0 tx=0).stop 2024-03-23T09:41:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.354+0000 7fe25dc26640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe25815d0c0 0x7fe2580b2180 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.354+0000 7fe25dc26640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe25815a700 0x7fe2580c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.354+0000 7fe25dc26640 1 --2- 172.21.15.47:0/2036876531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2580d27f0 0x7fe2580c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.354+0000 7fe25dc26640 1 -- 172.21.15.47:0/2036876531 >> 172.21.15.47:0/2036876531 conn(0x7fe25806afd0 msgr2=0x7fe25815b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.355+0000 7fe25dc26640 1 -- 172.21.15.47:0/2036876531 shutdown_connections 2024-03-23T09:41:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.355+0000 7fe25dc26640 1 -- 172.21.15.47:0/2036876531 wait complete. 2024-03-23T09:41:58.366 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T09:41:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.474+0000 7fea8efa2640 1 Processor -- start 2024-03-23T09:41:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.474+0000 7fea8efa2640 1 -- start start 2024-03-23T09:41:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8efa2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea8815c960 0x7fea880d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8efa2640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fea880d3210 0x7fea880cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8efa2640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea880d1480 con 0x7fea880cf060 2024-03-23T09:41:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8efa2640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea880cd500 con 0x7fea880d3210 2024-03-23T09:41:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8efa2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea880cd7e0 con 0x7fea8815c960 2024-03-23T09:41:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8dfa0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea8815c960 0x7fea880d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8d79f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fea880d3210 0x7fea880cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8dfa0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea8815c960 0x7fea880d0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:39552/0 (socket says 172.21.15.47:39552) 2024-03-23T09:41:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8dfa0640 1 -- 172.21.15.47:0/1448173009 learned_addr learned my addr 172.21.15.47:0/1448173009 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8e7a1640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fea880cf060 0x7fea8806aea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:54542/0 (socket says 172.21.15.47:54542) 2024-03-23T09:41:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8dfa0640 1 -- 172.21.15.47:0/1448173009 >> v2:172.21.15.175:3300/0 conn(0x7fea880d3210 msgr2=0x7fea880cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8dfa0640 1 --2- 172.21.15.47:0/1448173009 >> v2:172.21.15.175:3300/0 conn(0x7fea880d3210 0x7fea880cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8dfa0640 1 -- 172.21.15.47:0/1448173009 >> v1:172.21.15.175:6789/0 conn(0x7fea880cf060 legacy=0x7fea8806aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8cf9e640 1 -- 172.21.15.47:0/1448173009 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161011031 0 0) 0x7fea880d1480 con 0x7fea880cf060 2024-03-23T09:41:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8dfa0640 1 -- 172.21.15.47:0/1448173009 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea880cdac0 con 0x7fea8815c960 2024-03-23T09:41:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.475+0000 7fea8d79f640 1 --2- 172.21.15.47:0/1448173009 >> v2:172.21.15.175:3300/0 conn(0x7fea880d3210 0x7fea880cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.476+0000 7fea8dfa0640 1 --2- 172.21.15.47:0/1448173009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea8815c960 0x7fea880d0c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fea84009870 tx=0x7fea8402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e97161cbffd56a8f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.476+0000 7fea8cf9e640 1 -- 172.21.15.47:0/1448173009 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84030840 con 0x7fea8815c960 2024-03-23T09:41:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.476+0000 7fea8cf9e640 1 -- 172.21.15.47:0/1448173009 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea840309a0 con 0x7fea8815c960 2024-03-23T09:41:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.476+0000 7fea8cf9e640 1 -- 172.21.15.47:0/1448173009 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84030d30 con 0x7fea8815c960 2024-03-23T09:41:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.477+0000 7fea8efa2640 1 -- 172.21.15.47:0/1448173009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea8815c960 msgr2=0x7fea880d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.477+0000 7fea8efa2640 1 --2- 172.21.15.47:0/1448173009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea8815c960 0x7fea880d0c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fea84009870 tx=0x7fea8402b980 comp rx=0 tx=0).stop 2024-03-23T09:41:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.477+0000 7fea8efa2640 1 -- 172.21.15.47:0/1448173009 shutdown_connections 2024-03-23T09:41:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.477+0000 7fea8efa2640 1 --2- 172.21.15.47:0/1448173009 >> v2:172.21.15.175:3300/0 conn(0x7fea880d3210 0x7fea880cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.477+0000 7fea8efa2640 1 --2- 172.21.15.47:0/1448173009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea8815c960 0x7fea880d0c50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.477+0000 7fea8efa2640 1 -- 172.21.15.47:0/1448173009 >> 172.21.15.47:0/1448173009 conn(0x7fea88089f90 msgr2=0x7fea880d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.477+0000 7fea8efa2640 1 -- 172.21.15.47:0/1448173009 shutdown_connections 2024-03-23T09:41:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.477+0000 7fea8efa2640 1 -- 172.21.15.47:0/1448173009 wait complete. 2024-03-23T09:41:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.478+0000 7fea8efa2640 1 Processor -- start 2024-03-23T09:41:58.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.478+0000 7fea8efa2640 1 -- start start 2024-03-23T09:41:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8efa2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea880cf060 0x7fea8814db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8efa2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea880d3210 0x7fea880c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8efa2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea8815c960 0x7fea8815f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8efa2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8814e090 con 0x7fea880d3210 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8efa2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8815e3e0 con 0x7fea880cf060 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8efa2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea880c9c80 con 0x7fea8815c960 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8d79f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea880d3210 0x7fea880c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8d79f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea880d3210 0x7fea880c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49044/0 (socket says 172.21.15.47:49044) 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8d79f640 1 -- 172.21.15.47:0/3090999632 learned_addr learned my addr 172.21.15.47:0/3090999632 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8dfa0640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea880cf060 0x7fea8814db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8e7a1640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea8815c960 0x7fea8815f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8d79f640 1 -- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea8815c960 msgr2=0x7fea8815f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8d79f640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea8815c960 0x7fea8815f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8d79f640 1 -- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea880cf060 msgr2=0x7fea8814db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8d79f640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea880cf060 0x7fea8814db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8d79f640 1 -- 172.21.15.47:0/3090999632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea780093a0 con 0x7fea880d3210 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.479+0000 7fea8e7a1640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea8815c960 0x7fea8815f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:41:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.480+0000 7fea8d79f640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea880d3210 0x7fea880c9740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fea7800ebe0 tx=0x7fea7800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.480+0000 7fea7e7fc640 1 -- 172.21.15.47:0/3090999632 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78009750 con 0x7fea880d3210 2024-03-23T09:41:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.480+0000 7fea7e7fc640 1 -- 172.21.15.47:0/3090999632 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7800cc30 con 0x7fea880d3210 2024-03-23T09:41:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.480+0000 7fea8efa2640 1 -- 172.21.15.47:0/3090999632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea840094d0 con 0x7fea880d3210 2024-03-23T09:41:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.480+0000 7fea7e7fc640 1 -- 172.21.15.47:0/3090999632 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7800eee0 con 0x7fea880d3210 2024-03-23T09:41:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.480+0000 7fea8efa2640 1 -- 172.21.15.47:0/3090999632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea880ce6a0 con 0x7fea880d3210 2024-03-23T09:41:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.481+0000 7fea8efa2640 1 -- 172.21.15.47:0/3090999632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea88066be0 con 0x7fea880d3210 2024-03-23T09:41:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.481+0000 7fea7e7fc640 1 -- 172.21.15.47:0/3090999632 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fea78005990 con 0x7fea880d3210 2024-03-23T09:41:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.482+0000 7fea7e7fc640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fea6c06ac00 0x7fea6c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.482+0000 7fea7e7fc640 1 -- 172.21.15.47:0/3090999632 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fea78089660 con 0x7fea880d3210 2024-03-23T09:41:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.482+0000 7fea8dfa0640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fea6c06ac00 0x7fea6c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.483+0000 7fea8dfa0640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fea6c06ac00 0x7fea6c06d0c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fea84036000 tx=0x7fea84008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.486+0000 7fea7e7fc640 1 -- 172.21.15.47:0/3090999632 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fea78054ad0 con 0x7fea880d3210 2024-03-23T09:41:58.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:58.666+0000 7fea8efa2640 1 -- 172.21.15.47:0/3090999632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fea88159270 con 0x7fea880d3210 2024-03-23T09:41:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.352+0000 7fea7e7fc640 1 -- 172.21.15.47:0/3090999632 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7fea780580f0 con 0x7fea880d3210 2024-03-23T09:41:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-03-23T09:41:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.357+0000 7fea8efa2640 1 -- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fea6c06ac00 msgr2=0x7fea6c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.357+0000 7fea8efa2640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fea6c06ac00 0x7fea6c06d0c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fea84036000 tx=0x7fea84008030 comp rx=0 tx=0).stop 2024-03-23T09:41:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.357+0000 7fea8efa2640 1 -- 172.21.15.47:0/3090999632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea880d3210 msgr2=0x7fea880c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.357+0000 7fea8efa2640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea880d3210 0x7fea880c9740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fea7800ebe0 tx=0x7fea7800c520 comp rx=0 tx=0).stop 2024-03-23T09:41:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.366+0000 7fea8efa2640 1 -- 172.21.15.47:0/3090999632 shutdown_connections 2024-03-23T09:41:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.366+0000 7fea8efa2640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fea6c06ac00 0x7fea6c06d0c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.366+0000 7fea8efa2640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea8815c960 0x7fea8815f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.366+0000 7fea8efa2640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea880d3210 0x7fea880c9740 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.366+0000 7fea8efa2640 1 --2- 172.21.15.47:0/3090999632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea880cf060 0x7fea8814db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.366+0000 7fea8efa2640 1 -- 172.21.15.47:0/3090999632 >> 172.21.15.47:0/3090999632 conn(0x7fea88089f90 msgr2=0x7fea881594b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.368+0000 7fea8efa2640 1 -- 172.21.15.47:0/3090999632 shutdown_connections 2024-03-23T09:41:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.368+0000 7fea8efa2640 1 -- 172.21.15.47:0/3090999632 wait complete. 2024-03-23T09:41:59.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T09:41:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.490+0000 7f29cea5b640 1 Processor -- start 2024-03-23T09:41:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.490+0000 7f29cea5b640 1 -- start start 2024-03-23T09:41:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.491+0000 7f29cea5b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c815ff60 0x7f29c80cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.491+0000 7f29cea5b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f29c80b81f0 0x7f29c80cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.491+0000 7f29cea5b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c8160d60 con 0x7f29c80b1b20 2024-03-23T09:41:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.491+0000 7f29cea5b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c8160340 con 0x7f29c80b81f0 2024-03-23T09:41:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.491+0000 7f29cea5b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80b8680 con 0x7f29c815ff60 2024-03-23T09:41:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.491+0000 7f29ccff8640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f29c80b1b20 0x7f29c80d2860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58438/0 (socket says 172.21.15.47:58438) 2024-03-23T09:41:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.491+0000 7f29ccff8640 1 -- 172.21.15.47:0/2315984117 learned_addr learned my addr 172.21.15.47:0/2315984117 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:41:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.491+0000 7f29c7fff640 1 --2- 172.21.15.47:0/2315984117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c815ff60 0x7f29c80cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.491+0000 7f29c77fe640 1 --2- 172.21.15.47:0/2315984117 >> v2:172.21.15.175:3300/0 conn(0x7f29c80b81f0 0x7f29c80cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.492+0000 7f29c6ffd640 1 -- 172.21.15.47:0/2315984117 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3370696687 0 0) 0x7f29c8160d60 con 0x7f29c80b1b20 2024-03-23T09:41:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.492+0000 7f29c6ffd640 1 -- 172.21.15.47:0/2315984117 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b4003520 con 0x7f29c80b1b20 2024-03-23T09:41:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.492+0000 7f29c6ffd640 1 -- 172.21.15.47:0/2315984117 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3413689485 0 0) 0x7f29b4003520 con 0x7f29c80b1b20 2024-03-23T09:41:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.492+0000 7f29c77fe640 1 -- 172.21.15.47:0/2315984117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c815ff60 msgr2=0x7f29c80cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.492+0000 7f29c77fe640 1 --2- 172.21.15.47:0/2315984117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c815ff60 0x7f29c80cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.492+0000 7f29c77fe640 1 -- 172.21.15.47:0/2315984117 >> v1:172.21.15.47:6789/0 conn(0x7f29c80b1b20 legacy=0x7f29c80d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.492+0000 7f29c77fe640 1 -- 172.21.15.47:0/2315984117 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c8160710 con 0x7f29c80b81f0 2024-03-23T09:41:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.492+0000 7f29c7fff640 1 --2- 172.21.15.47:0/2315984117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c815ff60 0x7f29c80cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:41:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.493+0000 7f29c77fe640 1 --2- 172.21.15.47:0/2315984117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80b81f0 0x7f29c80cfd60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f29b00097a0 tx=0x7f29b002b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=970156c955d0232 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.493+0000 7f29c6ffd640 1 -- 172.21.15.47:0/2315984117 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c00019f0 con 0x7f29c80b81f0 2024-03-23T09:41:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.493+0000 7f29c6ffd640 1 -- 172.21.15.47:0/2315984117 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b002e050 con 0x7f29c80b81f0 2024-03-23T09:41:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.493+0000 7f29c6ffd640 1 -- 172.21.15.47:0/2315984117 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0030710 con 0x7f29c80b81f0 2024-03-23T09:41:59.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.493+0000 7f29cea5b640 1 -- 172.21.15.47:0/2315984117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80b81f0 msgr2=0x7f29c80cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:59.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.493+0000 7f29cea5b640 1 --2- 172.21.15.47:0/2315984117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80b81f0 0x7f29c80cfd60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f29b00097a0 tx=0x7f29b002b710 comp rx=0 tx=0).stop 2024-03-23T09:41:59.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.493+0000 7f29cea5b640 1 -- 172.21.15.47:0/2315984117 shutdown_connections 2024-03-23T09:41:59.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.493+0000 7f29cea5b640 1 --2- 172.21.15.47:0/2315984117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80b81f0 0x7f29c80cfd60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.493+0000 7f29cea5b640 1 --2- 172.21.15.47:0/2315984117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c815ff60 0x7f29c80cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.493+0000 7f29cea5b640 1 -- 172.21.15.47:0/2315984117 >> 172.21.15.47:0/2315984117 conn(0x7f29c8065df0 msgr2=0x7f29c8066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:59.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.493+0000 7f29cea5b640 1 -- 172.21.15.47:0/2315984117 shutdown_connections 2024-03-23T09:41:59.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.494+0000 7f29cea5b640 1 -- 172.21.15.47:0/2315984117 wait complete. 2024-03-23T09:41:59.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.495+0000 7f29cea5b640 1 Processor -- start 2024-03-23T09:41:59.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.495+0000 7f29cea5b640 1 -- start start 2024-03-23T09:41:59.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.495+0000 7f29cea5b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c80b1b20 0x7f29c81480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:59.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29cea5b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c815ff60 0x7f29c81f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:59.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29cea5b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80c6670 0x7f29c80b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:59.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29cea5b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c81447d0 con 0x7f29c815ff60 2024-03-23T09:41:59.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29cea5b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c8148630 con 0x7f29c80c6670 2024-03-23T09:41:59.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29cea5b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80c99a0 con 0x7f29c80b1b20 2024-03-23T09:41:59.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29c77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c815ff60 0x7f29c81f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29c77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c815ff60 0x7f29c81f5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49058/0 (socket says 172.21.15.47:49058) 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29c77fe640 1 -- 172.21.15.47:0/3511409087 learned_addr learned my addr 172.21.15.47:0/3511409087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29c7fff640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c80b1b20 0x7f29c81480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29ccff8640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80c6670 0x7f29c80b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29c77fe640 1 -- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c80b1b20 msgr2=0x7f29c81480f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29c77fe640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c80b1b20 0x7f29c81480f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29c77fe640 1 -- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80c6670 msgr2=0x7f29c80b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29c77fe640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80c6670 0x7f29c80b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29c77fe640 1 -- 172.21.15.47:0/3511409087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b8009400 con 0x7f29c815ff60 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29c77fe640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c815ff60 0x7f29c81f5e70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f29b0000c00 tx=0x7f29b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29c4ff9640 1 -- 172.21.15.47:0/3511409087 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b003c620 con 0x7f29c815ff60 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29c4ff9640 1 -- 172.21.15.47:0/3511409087 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b003c7a0 con 0x7f29c815ff60 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29cea5b640 1 -- 172.21.15.47:0/3511409087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b0009400 con 0x7f29c815ff60 2024-03-23T09:41:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29c4ff9640 1 -- 172.21.15.47:0/3511409087 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b003cb50 con 0x7f29c815ff60 2024-03-23T09:41:59.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.496+0000 7f29cea5b640 1 -- 172.21.15.47:0/3511409087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c81f63b0 con 0x7f29c815ff60 2024-03-23T09:41:59.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.498+0000 7f29cea5b640 1 -- 172.21.15.47:0/3511409087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c8144980 con 0x7f29c815ff60 2024-03-23T09:41:59.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.498+0000 7f29c4ff9640 1 -- 172.21.15.47:0/3511409087 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f29b0039070 con 0x7f29c815ff60 2024-03-23T09:41:59.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.498+0000 7f29c4ff9640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f29a806ac00 0x7f29a806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:59.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.498+0000 7f29c4ff9640 1 -- 172.21.15.47:0/3511409087 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f29b00aa8f0 con 0x7f29c815ff60 2024-03-23T09:41:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.499+0000 7f29c7fff640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f29a806ac00 0x7f29a806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.499+0000 7f29c7fff640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f29a806ac00 0x7f29a806d0c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009cd0 tx=0x7f29b8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:59.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.503+0000 7f29c4ff9640 1 -- 172.21.15.47:0/3511409087 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f29c8144980 con 0x7f29c815ff60 2024-03-23T09:41:59.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.718+0000 7f29cea5b640 1 -- 172.21.15.47:0/3511409087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f29c8144980 con 0x7f29c815ff60 2024-03-23T09:41:59.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.723+0000 7f29c4ff9640 1 -- 172.21.15.47:0/3511409087 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v10) v1 ==== 172+0+0 (secure 0 0 0) 0x7f29c8144980 con 0x7f29c815ff60 2024-03-23T09:41:59.724 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 4 and data pool 5 2024-03-23T09:41:59.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.735+0000 7f29cea5b640 1 -- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f29a806ac00 msgr2=0x7f29a806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:59.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.735+0000 7f29cea5b640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f29a806ac00 0x7f29a806d0c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009cd0 tx=0x7f29b8009190 comp rx=0 tx=0).stop 2024-03-23T09:41:59.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.735+0000 7f29cea5b640 1 -- 172.21.15.47:0/3511409087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c815ff60 msgr2=0x7f29c81f5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:59.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.735+0000 7f29cea5b640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c815ff60 0x7f29c81f5e70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f29b0000c00 tx=0x7f29b00046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.736+0000 7f29cea5b640 1 -- 172.21.15.47:0/3511409087 shutdown_connections 2024-03-23T09:41:59.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.736+0000 7f29cea5b640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f29a806ac00 0x7f29a806d0c0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009cd0 tx=0x7f29b8009190 comp rx=0 tx=0).stop 2024-03-23T09:41:59.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.736+0000 7f29cea5b640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80c6670 0x7f29c80b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.736+0000 7f29cea5b640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c815ff60 0x7f29c81f5e70 secure :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f29b0000c00 tx=0x7f29b00046c0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.736+0000 7f29cea5b640 1 --2- 172.21.15.47:0/3511409087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c80b1b20 0x7f29c81480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.736+0000 7f29cea5b640 1 -- 172.21.15.47:0/3511409087 >> 172.21.15.47:0/3511409087 conn(0x7f29c8065df0 msgr2=0x7f29c815ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.738+0000 7f29cea5b640 1 -- 172.21.15.47:0/3511409087 shutdown_connections 2024-03-23T09:41:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.739+0000 7f29cea5b640 1 -- 172.21.15.47:0/3511409087 wait complete. 2024-03-23T09:41:59.755 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:41:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.856+0000 7f2c86c19640 1 Processor -- start 2024-03-23T09:41:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.856+0000 7f2c86c19640 1 -- start start 2024-03-23T09:41:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.857+0000 7f2c86c19640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800cdc40 0x7f2c800b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.857+0000 7f2c86c19640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c800d1470 0x7f2c800d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.857+0000 7f2c86c19640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c800b5990 con 0x7f2c8015ca50 2024-03-23T09:41:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.857+0000 7f2c86c19640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c800b5230 con 0x7f2c800d1470 2024-03-23T09:41:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.857+0000 7f2c86c19640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8015ce30 con 0x7f2c800cdc40 2024-03-23T09:41:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.857+0000 7f2c851b6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c800d1470 0x7f2c800d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.857+0000 7f2c851b6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c800d1470 0x7f2c800d18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49068/0 (socket says 172.21.15.47:49068) 2024-03-23T09:41:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.857+0000 7f2c851b6640 1 -- 172.21.15.47:0/1096002779 learned_addr learned my addr 172.21.15.47:0/1096002779 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.857+0000 7f2c851b6640 1 -- 172.21.15.47:0/1096002779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800cdc40 msgr2=0x7f2c800b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.857+0000 7f2c851b6640 1 --2- 172.21.15.47:0/1096002779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800cdc40 0x7f2c800b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.857+0000 7f2c851b6640 1 -- 172.21.15.47:0/1096002779 >> v1:172.21.15.175:6789/0 conn(0x7f2c8015ca50 legacy=0x7f2c800cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:41:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.857+0000 7f2c851b6640 1 -- 172.21.15.47:0/1096002779 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c800b8800 con 0x7f2c800d1470 2024-03-23T09:41:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.858+0000 7f2c851b6640 1 --2- 172.21.15.47:0/1096002779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c800d1470 0x7f2c800d18e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009e50 tx=0x7f2c7802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9e569578165454 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.858+0000 7f2c7f7fe640 1 -- 172.21.15.47:0/1096002779 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7802c950 con 0x7f2c800d1470 2024-03-23T09:41:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.858+0000 7f2c7f7fe640 1 -- 172.21.15.47:0/1096002779 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7802cab0 con 0x7f2c800d1470 2024-03-23T09:41:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.858+0000 7f2c7f7fe640 1 -- 172.21.15.47:0/1096002779 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7802ce40 con 0x7f2c800d1470 2024-03-23T09:41:59.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.859+0000 7f2c86c19640 1 -- 172.21.15.47:0/1096002779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c800d1470 msgr2=0x7f2c800d18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:59.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.859+0000 7f2c86c19640 1 --2- 172.21.15.47:0/1096002779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c800d1470 0x7f2c800d18e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009e50 tx=0x7f2c7802bdb0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.859+0000 7f2c86c19640 1 -- 172.21.15.47:0/1096002779 shutdown_connections 2024-03-23T09:41:59.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.859+0000 7f2c86c19640 1 --2- 172.21.15.47:0/1096002779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c800d1470 0x7f2c800d18e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.859+0000 7f2c86c19640 1 --2- 172.21.15.47:0/1096002779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800cdc40 0x7f2c800b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.859+0000 7f2c86c19640 1 -- 172.21.15.47:0/1096002779 >> 172.21.15.47:0/1096002779 conn(0x7f2c8006aee0 msgr2=0x7f2c8006b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:41:59.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.859+0000 7f2c86c19640 1 -- 172.21.15.47:0/1096002779 shutdown_connections 2024-03-23T09:41:59.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.859+0000 7f2c86c19640 1 -- 172.21.15.47:0/1096002779 wait complete. 2024-03-23T09:41:59.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.860+0000 7f2c86c19640 1 Processor -- start 2024-03-23T09:41:59.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.860+0000 7f2c86c19640 1 -- start start 2024-03-23T09:41:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c86c19640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c800cdc40 0x7f2c8015f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c86c19640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800d1470 0x7f2c80157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c86c19640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c8015ca50 0x7f2c80146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c86c19640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8015fe60 con 0x7f2c800cdc40 2024-03-23T09:41:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c86c19640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c80158220 con 0x7f2c8015ca50 2024-03-23T09:41:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c86c19640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c80158450 con 0x7f2c800d1470 2024-03-23T09:41:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c849b5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c800cdc40 0x7f2c8015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c849b5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c800cdc40 0x7f2c8015f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49070/0 (socket says 172.21.15.47:49070) 2024-03-23T09:41:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c849b5640 1 -- 172.21.15.47:0/1473454749 learned_addr learned my addr 172.21.15.47:0/1473454749 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c7ffff640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800d1470 0x7f2c80157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c851b6640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c8015ca50 0x7f2c80146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c849b5640 1 -- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800d1470 msgr2=0x7f2c80157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c849b5640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800d1470 0x7f2c80157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c849b5640 1 -- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c8015ca50 msgr2=0x7f2c80146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c849b5640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c8015ca50 0x7f2c80146910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c849b5640 1 -- 172.21.15.47:0/1473454749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c78009b30 con 0x7f2c800cdc40 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c849b5640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c800cdc40 0x7f2c8015f920 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400c410 tx=0x7f2c7400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c7d7fa640 1 -- 172.21.15.47:0/1473454749 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c740035f0 con 0x7f2c800cdc40 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c7d7fa640 1 -- 172.21.15.47:0/1473454749 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c74003750 con 0x7f2c800cdc40 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c86c19640 1 -- 172.21.15.47:0/1473454749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c80158730 con 0x7f2c800cdc40 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c7d7fa640 1 -- 172.21.15.47:0/1473454749 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c74003b00 con 0x7f2c800cdc40 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.861+0000 7f2c86c19640 1 -- 172.21.15.47:0/1473454749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c80158fd0 con 0x7f2c800cdc40 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.862+0000 7f2c86c19640 1 -- 172.21.15.47:0/1473454749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c80072360 con 0x7f2c800cdc40 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.863+0000 7f2c7d7fa640 1 -- 172.21.15.47:0/1473454749 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2c74003c60 con 0x7f2c800cdc40 2024-03-23T09:41:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.863+0000 7f2c7d7fa640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c6406ac30 0x7f2c6406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:41:59.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.863+0000 7f2c7d7fa640 1 -- 172.21.15.47:0/1473454749 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c74088a50 con 0x7f2c800cdc40 2024-03-23T09:41:59.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.863+0000 7f2c7ffff640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c6406ac30 0x7f2c6406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:41:59.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.864+0000 7f2c7ffff640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c6406ac30 0x7f2c6406d0f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c006fd0 tx=0x7f2c6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:41:59.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:41:59.868+0000 7f2c7d7fa640 1 -- 172.21.15.47:0/1473454749 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2c74053ba0 con 0x7f2c800cdc40 2024-03-23T09:42:00.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.047+0000 7f2c86c19640 1 -- 172.21.15.47:0/1473454749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c80158920 con 0x7f2c800cdc40 2024-03-23T09:42:00.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.048+0000 7f2c7d7fa640 1 -- 172.21.15.47:0/1473454749 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16114 (secure 0 0 0) 0x7f2c74017070 con 0x7f2c800cdc40 2024-03-23T09:42:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:00.048 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":33,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:59.719486+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:58.297353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:58.685375+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:42:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.050+0000 7f2c86c19640 1 -- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c6406ac30 msgr2=0x7f2c6406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.050+0000 7f2c86c19640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c6406ac30 0x7f2c6406d0f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c006fd0 tx=0x7f2c6c008040 comp rx=0 tx=0).stop 2024-03-23T09:42:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.050+0000 7f2c86c19640 1 -- 172.21.15.47:0/1473454749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c800cdc40 msgr2=0x7f2c8015f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.050+0000 7f2c86c19640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c800cdc40 0x7f2c8015f920 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400c410 tx=0x7f2c7400c8d0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.050+0000 7f2c86c19640 1 -- 172.21.15.47:0/1473454749 shutdown_connections 2024-03-23T09:42:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.050+0000 7f2c86c19640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c6406ac30 0x7f2c6406d0f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.050+0000 7f2c86c19640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c8015ca50 0x7f2c80146910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.050+0000 7f2c86c19640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800d1470 0x7f2c80157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.050+0000 7f2c86c19640 1 --2- 172.21.15.47:0/1473454749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c800cdc40 0x7f2c8015f920 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.050+0000 7f2c86c19640 1 -- 172.21.15.47:0/1473454749 >> 172.21.15.47:0/1473454749 conn(0x7f2c8006aee0 msgr2=0x7f2c800cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.050+0000 7f2c86c19640 1 -- 172.21.15.47:0/1473454749 shutdown_connections 2024-03-23T09:42:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.050+0000 7f2c86c19640 1 -- 172.21.15.47:0/1473454749 wait complete. 2024-03-23T09:42:00.060 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:42:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.161+0000 7f6dfe88f640 1 Processor -- start 2024-03-23T09:42:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.161+0000 7f6dfe88f640 1 -- start start 2024-03-23T09:42:00.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.162+0000 7f6dfe88f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6df815cbe0 0x7f6df80ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.162+0000 7f6dfe88f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6df80cad40 0x7f6df80cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.162+0000 7f6dfe88f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df80b06d0 0x7f6df80b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.162+0000 7f6dfe88f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df80b1400 con 0x7f6df815cbe0 2024-03-23T09:42:00.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.162+0000 7f6dfe88f640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df80b16e0 con 0x7f6df80cad40 2024-03-23T09:42:00.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.162+0000 7f6dfe88f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80cedb0 con 0x7f6df80b06d0 2024-03-23T09:42:00.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.162+0000 7f6df7fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6df815cbe0 0x7f6df80ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.162+0000 7f6df7fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6df815cbe0 0x7f6df80ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49076/0 (socket says 172.21.15.47:49076) 2024-03-23T09:42:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.162+0000 7f6df7fff640 1 -- 172.21.15.47:0/1328186293 learned_addr learned my addr 172.21.15.47:0/1328186293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.163+0000 7f6df77fe640 1 --2- 172.21.15.47:0/1328186293 >> v2:172.21.15.175:3300/0 conn(0x7f6df80cad40 0x7f6df80cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.163+0000 7f6dfce2c640 1 --2- 172.21.15.47:0/1328186293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df80b06d0 0x7f6df80b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.163+0000 7f6df7fff640 1 -- 172.21.15.47:0/1328186293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df80b06d0 msgr2=0x7f6df80b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.163+0000 7f6df7fff640 1 --2- 172.21.15.47:0/1328186293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df80b06d0 0x7f6df80b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.163+0000 7f6df7fff640 1 -- 172.21.15.47:0/1328186293 >> v2:172.21.15.175:3300/0 conn(0x7f6df80cad40 msgr2=0x7f6df80cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.163+0000 7f6df7fff640 1 --2- 172.21.15.47:0/1328186293 >> v2:172.21.15.175:3300/0 conn(0x7f6df80cad40 0x7f6df80cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.163+0000 7f6df7fff640 1 -- 172.21.15.47:0/1328186293 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df80cf090 con 0x7f6df815cbe0 2024-03-23T09:42:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.163+0000 7f6df77fe640 1 --2- 172.21.15.47:0/1328186293 >> v2:172.21.15.175:3300/0 conn(0x7f6df80cad40 0x7f6df80cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.163+0000 7f6dfce2c640 1 --2- 172.21.15.47:0/1328186293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df80b06d0 0x7f6df80b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.163+0000 7f6df7fff640 1 --2- 172.21.15.47:0/1328186293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df815cbe0 0x7f6df80ccf70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009870 tx=0x7f6dec02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14acdb31ee224950 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.164+0000 7f6df6ffd640 1 -- 172.21.15.47:0/1328186293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0317c0 con 0x7f6df815cbe0 2024-03-23T09:42:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.164+0000 7f6df6ffd640 1 -- 172.21.15.47:0/1328186293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec031920 con 0x7f6df815cbe0 2024-03-23T09:42:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.164+0000 7f6df6ffd640 1 -- 172.21.15.47:0/1328186293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec031cb0 con 0x7f6df815cbe0 2024-03-23T09:42:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.164+0000 7f6dfe88f640 1 -- 172.21.15.47:0/1328186293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df815cbe0 msgr2=0x7f6df80ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.164+0000 7f6dfe88f640 1 --2- 172.21.15.47:0/1328186293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df815cbe0 0x7f6df80ccf70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009870 tx=0x7f6dec02b860 comp rx=0 tx=0).stop 2024-03-23T09:42:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.164+0000 7f6dfe88f640 1 -- 172.21.15.47:0/1328186293 shutdown_connections 2024-03-23T09:42:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.164+0000 7f6dfe88f640 1 --2- 172.21.15.47:0/1328186293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df80b06d0 0x7f6df80b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.164+0000 7f6dfe88f640 1 --2- 172.21.15.47:0/1328186293 >> v2:172.21.15.175:3300/0 conn(0x7f6df80cad40 0x7f6df80cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.164+0000 7f6dfe88f640 1 --2- 172.21.15.47:0/1328186293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df815cbe0 0x7f6df80ccf70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.164+0000 7f6dfe88f640 1 -- 172.21.15.47:0/1328186293 >> 172.21.15.47:0/1328186293 conn(0x7f6df80d06f0 msgr2=0x7f6df80d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.164+0000 7f6dfe88f640 1 -- 172.21.15.47:0/1328186293 shutdown_connections 2024-03-23T09:42:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.164+0000 7f6dfe88f640 1 -- 172.21.15.47:0/1328186293 wait complete. 2024-03-23T09:42:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.165+0000 7f6dfe88f640 1 Processor -- start 2024-03-23T09:42:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.166+0000 7f6dfe88f640 1 -- start start 2024-03-23T09:42:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.166+0000 7f6dfe88f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df80b06d0 0x7f6df80af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.166+0000 7f6dfe88f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df80cad40 0x7f6df80ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.166+0000 7f6dfe88f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df815cbe0 0x7f6df81f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.166+0000 7f6dfe88f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df814b3e0 con 0x7f6df80b06d0 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.166+0000 7f6dfe88f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80af570 con 0x7f6df80cad40 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.166+0000 7f6dfe88f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80af7f0 con 0x7f6df815cbe0 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.166+0000 7f6df7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df80b06d0 0x7f6df80af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.166+0000 7f6df7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df80b06d0 0x7f6df80af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49078/0 (socket says 172.21.15.47:49078) 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.166+0000 7f6df7fff640 1 -- 172.21.15.47:0/122071363 learned_addr learned my addr 172.21.15.47:0/122071363 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6df77fe640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df80cad40 0x7f6df80ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6dfce2c640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df815cbe0 0x7f6df81f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6df7fff640 1 -- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df815cbe0 msgr2=0x7f6df81f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6df7fff640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df815cbe0 0x7f6df81f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6df7fff640 1 -- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df80cad40 msgr2=0x7f6df80ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6df7fff640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df80cad40 0x7f6df80ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6df7fff640 1 -- 172.21.15.47:0/122071363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec0094d0 con 0x7f6df80b06d0 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6df77fe640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df80cad40 0x7f6df80ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6dfce2c640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df815cbe0 0x7f6df81f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6df7fff640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df80b06d0 0x7f6df80af030 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6dec02b8a0 tx=0x7f6dec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6df4ff9640 1 -- 172.21.15.47:0/122071363 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0329a0 con 0x7f6df80b06d0 2024-03-23T09:42:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6df4ff9640 1 -- 172.21.15.47:0/122071363 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec032b40 con 0x7f6df80b06d0 2024-03-23T09:42:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6df4ff9640 1 -- 172.21.15.47:0/122071363 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0453f0 con 0x7f6df80b06d0 2024-03-23T09:42:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6dfe88f640 1 -- 172.21.15.47:0/122071363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df80afad0 con 0x7f6df80b06d0 2024-03-23T09:42:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.167+0000 7f6dfe88f640 1 -- 172.21.15.47:0/122071363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df80b0090 con 0x7f6df80b06d0 2024-03-23T09:42:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.168+0000 7f6dfe88f640 1 -- 172.21.15.47:0/122071363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df80c51c0 con 0x7f6df80b06d0 2024-03-23T09:42:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.169+0000 7f6df4ff9640 1 -- 172.21.15.47:0/122071363 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6dec04e450 con 0x7f6df80b06d0 2024-03-23T09:42:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.169+0000 7f6df4ff9640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6de406ac00 0x7f6de406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.169+0000 7f6df4ff9640 1 -- 172.21.15.47:0/122071363 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6dec0ab700 con 0x7f6df80b06d0 2024-03-23T09:42:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.169+0000 7f6df77fe640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6de406ac00 0x7f6de406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.170+0000 7f6df77fe640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6de406ac00 0x7f6de406d0c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6df80ae880 tx=0x7f6de8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:00.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.173+0000 7f6df4ff9640 1 -- 172.21.15.47:0/122071363 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6dec0768e0 con 0x7f6df80b06d0 2024-03-23T09:42:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.357+0000 7f6dfe88f640 1 -- 172.21.15.47:0/122071363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6df8066be0 con 0x7f6df80b06d0 2024-03-23T09:42:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.358+0000 7f6df4ff9640 1 -- 172.21.15.47:0/122071363 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16114 (secure 0 0 0) 0x7f6dec079f00 con 0x7f6df80b06d0 2024-03-23T09:42:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:00.358 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":33,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:41:59.719486+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:58.297353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:58.685375+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:42:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.360+0000 7f6dfe88f640 1 -- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6de406ac00 msgr2=0x7f6de406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.360+0000 7f6dfe88f640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6de406ac00 0x7f6de406d0c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6df80ae880 tx=0x7f6de8008040 comp rx=0 tx=0).stop 2024-03-23T09:42:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.360+0000 7f6dfe88f640 1 -- 172.21.15.47:0/122071363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df80b06d0 msgr2=0x7f6df80af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.360+0000 7f6dfe88f640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df80b06d0 0x7f6df80af030 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6dec02b8a0 tx=0x7f6dec0046c0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.360+0000 7f6dfe88f640 1 -- 172.21.15.47:0/122071363 shutdown_connections 2024-03-23T09:42:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.360+0000 7f6dfe88f640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6de406ac00 0x7f6de406d0c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.361+0000 7f6dfe88f640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df815cbe0 0x7f6df81f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.361+0000 7f6dfe88f640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df80cad40 0x7f6df80ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.361+0000 7f6dfe88f640 1 --2- 172.21.15.47:0/122071363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df80b06d0 0x7f6df80af030 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.361+0000 7f6dfe88f640 1 -- 172.21.15.47:0/122071363 >> 172.21.15.47:0/122071363 conn(0x7f6df80d06f0 msgr2=0x7f6df806b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.361+0000 7f6dfe88f640 1 -- 172.21.15.47:0/122071363 shutdown_connections 2024-03-23T09:42:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.361+0000 7f6dfe88f640 1 -- 172.21.15.47:0/122071363 wait complete. 2024-03-23T09:42:00.371 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T09:42:00.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.475+0000 7f9e1a31b640 1 Processor -- start 2024-03-23T09:42:00.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.475+0000 7f9e1a31b640 1 -- start start 2024-03-23T09:42:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.476+0000 7f9e1a31b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1415c990 0x7f9e14160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.476+0000 7f9e1a31b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9e140b8350 0x7f9e140cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.476+0000 7f9e1a31b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e14161630 con 0x7f9e140cdc40 2024-03-23T09:42:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.476+0000 7f9e1a31b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e140ce0b0 con 0x7f9e140b8350 2024-03-23T09:42:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.476+0000 7f9e1a31b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e140d1030 con 0x7f9e1415c990 2024-03-23T09:42:00.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.476+0000 7f9e137fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9e140b8350 0x7f9e140cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.476+0000 7f9e13fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1415c990 0x7f9e14160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.477+0000 7f9e137fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9e140b8350 0x7f9e140cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:34432/0 (socket says 172.21.15.47:34432) 2024-03-23T09:42:00.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.477+0000 7f9e137fe640 1 -- 172.21.15.47:0/2396522588 learned_addr learned my addr 172.21.15.47:0/2396522588 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:00.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.477+0000 7f9e188b8640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f9e140cdc40 0x7f9e140b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:54558/0 (socket says 172.21.15.47:54558) 2024-03-23T09:42:00.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.477+0000 7f9e13fff640 1 -- 172.21.15.47:0/2396522588 >> v2:172.21.15.175:3300/0 conn(0x7f9e140b8350 msgr2=0x7f9e140cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:00.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.477+0000 7f9e13fff640 1 --2- 172.21.15.47:0/2396522588 >> v2:172.21.15.175:3300/0 conn(0x7f9e140b8350 0x7f9e140cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.477+0000 7f9e13fff640 1 -- 172.21.15.47:0/2396522588 >> v1:172.21.15.175:6789/0 conn(0x7f9e140cdc40 legacy=0x7f9e140b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.477+0000 7f9e13fff640 1 -- 172.21.15.47:0/2396522588 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e140cfe10 con 0x7f9e1415c990 2024-03-23T09:42:00.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.477+0000 7f9e137fe640 1 --2- 172.21.15.47:0/2396522588 >> v2:172.21.15.175:3300/0 conn(0x7f9e140b8350 0x7f9e140cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:42:00.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.478+0000 7f9e13fff640 1 --2- 172.21.15.47:0/2396522588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1415c990 0x7f9e14160690 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e080097a0 tx=0x7f9e0802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=215287cd5d59cdbc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:00.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.478+0000 7f9e12ffd640 1 -- 172.21.15.47:0/2396522588 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0802e070 con 0x7f9e1415c990 2024-03-23T09:42:00.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.478+0000 7f9e12ffd640 1 -- 172.21.15.47:0/2396522588 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e080315a0 con 0x7f9e1415c990 2024-03-23T09:42:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.478+0000 7f9e12ffd640 1 -- 172.21.15.47:0/2396522588 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08031930 con 0x7f9e1415c990 2024-03-23T09:42:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.478+0000 7f9e1a31b640 1 -- 172.21.15.47:0/2396522588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1415c990 msgr2=0x7f9e14160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.478+0000 7f9e1a31b640 1 --2- 172.21.15.47:0/2396522588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1415c990 0x7f9e14160690 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e080097a0 tx=0x7f9e0802b840 comp rx=0 tx=0).stop 2024-03-23T09:42:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.478+0000 7f9e1a31b640 1 -- 172.21.15.47:0/2396522588 shutdown_connections 2024-03-23T09:42:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.479+0000 7f9e1a31b640 1 --2- 172.21.15.47:0/2396522588 >> v2:172.21.15.175:3300/0 conn(0x7f9e140b8350 0x7f9e140cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.479+0000 7f9e1a31b640 1 --2- 172.21.15.47:0/2396522588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1415c990 0x7f9e14160690 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.479+0000 7f9e1a31b640 1 -- 172.21.15.47:0/2396522588 >> 172.21.15.47:0/2396522588 conn(0x7f9e140659d0 msgr2=0x7f9e14065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.479+0000 7f9e1a31b640 1 -- 172.21.15.47:0/2396522588 shutdown_connections 2024-03-23T09:42:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.479+0000 7f9e1a31b640 1 -- 172.21.15.47:0/2396522588 wait complete. 2024-03-23T09:42:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.480+0000 7f9e1a31b640 1 Processor -- start 2024-03-23T09:42:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.480+0000 7f9e1a31b640 1 -- start start 2024-03-23T09:42:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e1a31b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e140b8350 0x7f9e141f1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e1a31b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e140cdc40 0x7f9e140c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e1a31b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1415c990 0x7f9e140c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e1a31b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e140c7840 con 0x7f9e140cdc40 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e1a31b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e140c7b20 con 0x7f9e140b8350 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e1a31b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e140c7e00 con 0x7f9e1415c990 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e137fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e140cdc40 0x7f9e140c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e137fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e140cdc40 0x7f9e140c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49090/0 (socket says 172.21.15.47:49090) 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e13fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e140b8350 0x7f9e141f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e137fe640 1 -- 172.21.15.47:0/3596907046 learned_addr learned my addr 172.21.15.47:0/3596907046 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e188b8640 1 --2- 172.21.15.47:0/3596907046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1415c990 0x7f9e140c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e137fe640 1 -- 172.21.15.47:0/3596907046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1415c990 msgr2=0x7f9e140c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e137fe640 1 --2- 172.21.15.47:0/3596907046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1415c990 0x7f9e140c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e137fe640 1 -- 172.21.15.47:0/3596907046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e140b8350 msgr2=0x7f9e141f1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e137fe640 1 --2- 172.21.15.47:0/3596907046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e140b8350 0x7f9e141f1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e137fe640 1 -- 172.21.15.47:0/3596907046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e040094d0 con 0x7f9e140cdc40 2024-03-23T09:42:00.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e137fe640 1 --2- 172.21.15.47:0/3596907046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e140cdc40 0x7f9e140c6da0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400e9b0 tx=0x7f9e0400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:00.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e10ff9640 1 -- 172.21.15.47:0/3596907046 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0400c9a0 con 0x7f9e140cdc40 2024-03-23T09:42:00.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e10ff9640 1 -- 172.21.15.47:0/3596907046 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0400cb00 con 0x7f9e140cdc40 2024-03-23T09:42:00.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e1a31b640 1 -- 172.21.15.47:0/3596907046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e08009400 con 0x7f9e140cdc40 2024-03-23T09:42:00.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.481+0000 7f9e10ff9640 1 -- 172.21.15.47:0/3596907046 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0400ce70 con 0x7f9e140cdc40 2024-03-23T09:42:00.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.482+0000 7f9e1a31b640 1 -- 172.21.15.47:0/3596907046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1414dba0 con 0x7f9e140cdc40 2024-03-23T09:42:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.482+0000 7f9e1a31b640 1 -- 172.21.15.47:0/3596907046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e14072360 con 0x7f9e140cdc40 2024-03-23T09:42:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.483+0000 7f9e10ff9640 1 -- 172.21.15.47:0/3596907046 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9e04013470 con 0x7f9e140cdc40 2024-03-23T09:42:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.483+0000 7f9e10ff9640 1 --2- 172.21.15.47:0/3596907046 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e0006ac00 0x7f9e0006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.483+0000 7f9e10ff9640 1 -- 172.21.15.47:0/3596907046 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e04089850 con 0x7f9e140cdc40 2024-03-23T09:42:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.484+0000 7f9e13fff640 1 --2- 172.21.15.47:0/3596907046 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e0006ac00 0x7f9e0006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.484+0000 7f9e13fff640 1 --2- 172.21.15.47:0/3596907046 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e0006ac00 0x7f9e0006d0c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9e08036000 tx=0x7f9e0803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.487+0000 7f9e10ff9640 1 -- 172.21.15.47:0/3596907046 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9e04054ae0 con 0x7f9e140cdc40 2024-03-23T09:42:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.705+0000 7f9e1a31b640 1 -- 172.21.15.47:0/3596907046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9e14066be0 con 0x7f9e140cdc40 2024-03-23T09:42:00.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.723+0000 7f9e10ff9640 1 -- 172.21.15.47:0/3596907046 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v11) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9e040170b0 con 0x7f9e140cdc40 2024-03-23T09:42:00.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.725+0000 7f9e1a31b640 1 -- 172.21.15.47:0/3596907046 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e0006ac00 msgr2=0x7f9e0006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.725+0000 7f9e1a31b640 1 --2- 172.21.15.47:0/3596907046 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e0006ac00 0x7f9e0006d0c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9e08036000 tx=0x7f9e0803d040 comp rx=0 tx=0).stop 2024-03-23T09:42:00.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.725+0000 7f9e1a31b640 1 -- 172.21.15.47:0/3596907046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e140cdc40 msgr2=0x7f9e140c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.725+0000 7f9e1a31b640 1 --2- 172.21.15.47:0/3596907046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e140cdc40 0x7f9e140c6da0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400e9b0 tx=0x7f9e0400ee70 comp rx=0 tx=0).stop 2024-03-23T09:42:00.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.725+0000 7f9e1a31b640 1 -- 172.21.15.47:0/3596907046 shutdown_connections 2024-03-23T09:42:00.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.725+0000 7f9e1a31b640 1 --2- 172.21.15.47:0/3596907046 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e0006ac00 0x7f9e0006d0c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.725+0000 7f9e1a31b640 1 --2- 172.21.15.47:0/3596907046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1415c990 0x7f9e140c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.725+0000 7f9e1a31b640 1 --2- 172.21.15.47:0/3596907046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e140cdc40 0x7f9e140c6da0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.725+0000 7f9e1a31b640 1 --2- 172.21.15.47:0/3596907046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e140b8350 0x7f9e141f1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.725+0000 7f9e1a31b640 1 -- 172.21.15.47:0/3596907046 >> 172.21.15.47:0/3596907046 conn(0x7f9e140659d0 msgr2=0x7f9e140b74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.725+0000 7f9e1a31b640 1 -- 172.21.15.47:0/3596907046 shutdown_connections 2024-03-23T09:42:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.725+0000 7f9e1a31b640 1 -- 172.21.15.47:0/3596907046 wait complete. 2024-03-23T09:42:00.736 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-23T09:42:00.736 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-23T09:42:00.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.840+0000 7f5347a38640 1 Processor -- start 2024-03-23T09:42:00.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.841+0000 7f5347a38640 1 -- start start 2024-03-23T09:42:00.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.841+0000 7f5347a38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f534015ce10 0x7f53400cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.841+0000 7f5347a38640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f53400b85d0 0x7f53400cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.841+0000 7f5347a38640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5340161600 con 0x7f534015ca30 2024-03-23T09:42:00.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.841+0000 7f5347a38640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5340160e50 con 0x7f53400b85d0 2024-03-23T09:42:00.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.841+0000 7f5347a38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53400d25e0 con 0x7f534015ce10 2024-03-23T09:42:00.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.841+0000 7f53457d4640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f534015ca30 0x7f5340160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58440/0 (socket says 172.21.15.47:58440) 2024-03-23T09:42:00.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.841+0000 7f53457d4640 1 -- 172.21.15.47:0/3953094954 learned_addr learned my addr 172.21.15.47:0/3953094954 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:00.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748934981 0 0) 0x7f5340161600 con 0x7f534015ca30 2024-03-23T09:42:00.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f532c003520 con 0x7f534015ca30 2024-03-23T09:42:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5345fd5640 1 --2- 172.21.15.47:0/3953094954 >> v2:172.21.15.175:3300/0 conn(0x7f53400b85d0 0x7f53400cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5344fd3640 1 --2- 172.21.15.47:0/3953094954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f534015ce10 0x7f53400cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 223440107 0 0) 0x7f532c003520 con 0x7f534015ca30 2024-03-23T09:42:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f534015ce10 msgr2=0x7f53400cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 --2- 172.21.15.47:0/3953094954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f534015ce10 0x7f53400cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 >> v2:172.21.15.175:3300/0 conn(0x7f53400b85d0 msgr2=0x7f53400cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 --2- 172.21.15.47:0/3953094954 >> v2:172.21.15.175:3300/0 conn(0x7f53400b85d0 0x7f53400cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5340161040 con 0x7f534015ca30 2024-03-23T09:42:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f533c001670 con 0x7f534015ca30 2024-03-23T09:42:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5344fd3640 1 --2- 172.21.15.47:0/3953094954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f534015ce10 0x7f53400cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5345fd5640 1 --2- 172.21.15.47:0/3953094954 >> v2:172.21.15.175:3300/0 conn(0x7f53400b85d0 0x7f53400cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 >> v1:172.21.15.47:6789/0 conn(0x7f534015ca30 legacy=0x7f5340160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 --2- 172.21.15.47:0/3953094954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f534015ce10 0x7f532c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 --2- 172.21.15.47:0/3953094954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f532c003f60 0x7f532c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 --2- 172.21.15.47:0/3953094954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f532c004d00 0x7f532c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53400d25e0 con 0x7f532c004d00 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c00b9e0 con 0x7f532c003f60 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c00bba0 con 0x7f534015ce10 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5345fd5640 1 --2- 172.21.15.47:0/3953094954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f532c004d00 0x7f532c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f53457d4640 1 --2- 172.21.15.47:0/3953094954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f534015ce10 0x7f532c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.842+0000 7f5344fd3640 1 --2- 172.21.15.47:0/3953094954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f532c003f60 0x7f532c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.843+0000 7f5345fd5640 1 -- 172.21.15.47:0/3953094954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f534015ce10 msgr2=0x7f532c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.843+0000 7f5345fd5640 1 --2- 172.21.15.47:0/3953094954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f534015ce10 0x7f532c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.843+0000 7f5345fd5640 1 -- 172.21.15.47:0/3953094954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f532c003f60 msgr2=0x7f532c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.843+0000 7f5345fd5640 1 --2- 172.21.15.47:0/3953094954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f532c003f60 0x7f532c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.843+0000 7f5345fd5640 1 -- 172.21.15.47:0/3953094954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c003520 con 0x7f532c004d00 2024-03-23T09:42:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.843+0000 7f5345fd5640 1 --2- 172.21.15.47:0/3953094954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f532c004d00 0x7f532c005180 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f53380073e0 tx=0x7f532c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9dc7e2aee8c3d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:00.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.843+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533802cdc0 con 0x7f532c004d00 2024-03-23T09:42:00.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.843+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5338034440 con 0x7f532c004d00 2024-03-23T09:42:00.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.843+0000 7f5337fff640 1 -- 172.21.15.47:0/3953094954 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53380347b0 con 0x7f532c004d00 2024-03-23T09:42:00.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.844+0000 7f5347a38640 1 -- 172.21.15.47:0/3953094954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f532c004d00 msgr2=0x7f532c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.844+0000 7f5347a38640 1 --2- 172.21.15.47:0/3953094954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f532c004d00 0x7f532c005180 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f53380073e0 tx=0x7f532c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.844+0000 7f5345fd5640 1 -- 172.21.15.47:0/3953094954 reap_dead start 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.844+0000 7f5347a38640 1 -- 172.21.15.47:0/3953094954 shutdown_connections 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.844+0000 7f5347a38640 1 -- 172.21.15.47:0/3953094954 >> 172.21.15.47:0/3953094954 conn(0x7f534006b160 msgr2=0x7f534006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.844+0000 7f5347a38640 1 -- 172.21.15.47:0/3953094954 shutdown_connections 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.844+0000 7f5347a38640 1 -- 172.21.15.47:0/3953094954 wait complete. 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.845+0000 7f5347a38640 1 Processor -- start 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.845+0000 7f5347a38640 1 -- start start 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f5347a38640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f532c004d00 0x7f534015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f5347a38640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534015d240 0x7f534015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f5347a38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f534014ceb0 0x7f534014d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f5347a38640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534015dc00 con 0x7f532c004d00 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f5347a38640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5340160460 con 0x7f534015d240 2024-03-23T09:42:00.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f5347a38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534014dba0 con 0x7f534014ceb0 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f53457d4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f532c004d00 0x7f534015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f53457d4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f532c004d00 0x7f534015ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49106/0 (socket says 172.21.15.47:49106) 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f53457d4640 1 -- 172.21.15.47:0/2877386451 learned_addr learned my addr 172.21.15.47:0/2877386451 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f53457d4640 1 -- 172.21.15.47:0/2877386451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f534014ceb0 msgr2=0x7f534014d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f53457d4640 1 --2- 172.21.15.47:0/2877386451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f534014ceb0 0x7f534014d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f53457d4640 1 -- 172.21.15.47:0/2877386451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534015d240 msgr2=0x7f534015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f53457d4640 1 --2- 172.21.15.47:0/2877386451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534015d240 0x7f534015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.846+0000 7f53457d4640 1 -- 172.21.15.47:0/2877386451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534014de80 con 0x7f532c004d00 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.847+0000 7f53457d4640 1 --2- 172.21.15.47:0/2877386451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f532c004d00 0x7f534015ff20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f533c007290 tx=0x7f533c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.847+0000 7f5335ffb640 1 -- 172.21.15.47:0/2877386451 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c00fa20 con 0x7f532c004d00 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.847+0000 7f5335ffb640 1 -- 172.21.15.47:0/2877386451 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c00fba0 con 0x7f532c004d00 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.847+0000 7f5335ffb640 1 -- 172.21.15.47:0/2877386451 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c0133f0 con 0x7f532c004d00 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.847+0000 7f5347a38640 1 -- 172.21.15.47:0/2877386451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534014e160 con 0x7f532c004d00 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.847+0000 7f5347a38640 1 -- 172.21.15.47:0/2877386451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53400b61a0 con 0x7f532c004d00 2024-03-23T09:42:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.848+0000 7f5347a38640 1 -- 172.21.15.47:0/2877386451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5340072360 con 0x7f532c004d00 2024-03-23T09:42:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.848+0000 7f5335ffb640 1 -- 172.21.15.47:0/2877386451 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f533c013550 con 0x7f532c004d00 2024-03-23T09:42:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.849+0000 7f5335ffb640 1 --2- 172.21.15.47:0/2877386451 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f532406ac00 0x7f532406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.849+0000 7f5335ffb640 1 -- 172.21.15.47:0/2877386451 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f533c0896c0 con 0x7f532c004d00 2024-03-23T09:42:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.849+0000 7f5344fd3640 1 --2- 172.21.15.47:0/2877386451 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f532406ac00 0x7f532406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.850+0000 7f5344fd3640 1 --2- 172.21.15.47:0/2877386451 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f532406ac00 0x7f532406d0c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5340151ce0 tx=0x7f5330018040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:00.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:00.853+0000 7f5335ffb640 1 -- 172.21.15.47:0/2877386451 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f533c0548a0 con 0x7f532c004d00 2024-03-23T09:42:01.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.069+0000 7f5347a38640 1 -- 172.21.15.47:0/2877386451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5340066be0 con 0x7f532c004d00 2024-03-23T09:42:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.776+0000 7f5335ffb640 1 -- 172.21.15.47:0/2877386451 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f533c057ec0 con 0x7f532c004d00 2024-03-23T09:42:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.779+0000 7f5347a38640 1 -- 172.21.15.47:0/2877386451 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f532406ac00 msgr2=0x7f532406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.779+0000 7f5347a38640 1 --2- 172.21.15.47:0/2877386451 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f532406ac00 0x7f532406d0c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5340151ce0 tx=0x7f5330018040 comp rx=0 tx=0).stop 2024-03-23T09:42:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.779+0000 7f5347a38640 1 -- 172.21.15.47:0/2877386451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f532c004d00 msgr2=0x7f534015ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.779+0000 7f5347a38640 1 --2- 172.21.15.47:0/2877386451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f532c004d00 0x7f534015ff20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f533c007290 tx=0x7f533c007750 comp rx=0 tx=0).stop 2024-03-23T09:42:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.779+0000 7f5347a38640 1 -- 172.21.15.47:0/2877386451 shutdown_connections 2024-03-23T09:42:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.779+0000 7f5347a38640 1 --2- 172.21.15.47:0/2877386451 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f532406ac00 0x7f532406d0c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.779+0000 7f5347a38640 1 --2- 172.21.15.47:0/2877386451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f534014ceb0 0x7f534014d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.779+0000 7f5347a38640 1 --2- 172.21.15.47:0/2877386451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534015d240 0x7f534015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.779+0000 7f5347a38640 1 --2- 172.21.15.47:0/2877386451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f532c004d00 0x7f534015ff20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.779+0000 7f5347a38640 1 -- 172.21.15.47:0/2877386451 >> 172.21.15.47:0/2877386451 conn(0x7f534006b160 msgr2=0x7f5340152e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.779+0000 7f5347a38640 1 -- 172.21.15.47:0/2877386451 shutdown_connections 2024-03-23T09:42:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.779+0000 7f5347a38640 1 -- 172.21.15.47:0/2877386451 wait complete. 2024-03-23T09:42:01.792 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-23T09:42:01.792 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:42:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.899+0000 7fbfbca5b640 1 Processor -- start 2024-03-23T09:42:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.899+0000 7fbfbca5b640 1 -- start start 2024-03-23T09:42:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.900+0000 7fbfbca5b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbfb815cb40 0x7fbfb8160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.900+0000 7fbfbca5b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8160cb0 0x7fbfb80d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.900+0000 7fbfbca5b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb80d0ee0 con 0x7fbfb80b8300 2024-03-23T09:42:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.900+0000 7fbfbca5b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb80b5a10 con 0x7fbfb815cb40 2024-03-23T09:42:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.900+0000 7fbfbca5b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb81616f0 con 0x7fbfb8160cb0 2024-03-23T09:42:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.900+0000 7fbfb659c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbfb815cb40 0x7fbfb8160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.900+0000 7fbfb659c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbfb815cb40 0x7fbfb8160770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49112/0 (socket says 172.21.15.47:49112) 2024-03-23T09:42:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.900+0000 7fbfb659c640 1 -- 172.21.15.47:0/725303216 learned_addr learned my addr 172.21.15.47:0/725303216 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.900+0000 7fbfb6d9d640 1 --2- 172.21.15.47:0/725303216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8160cb0 0x7fbfb80d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.900+0000 7fbfb659c640 1 -- 172.21.15.47:0/725303216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8160cb0 msgr2=0x7fbfb80d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.900+0000 7fbfb659c640 1 --2- 172.21.15.47:0/725303216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8160cb0 0x7fbfb80d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.900+0000 7fbfb659c640 1 -- 172.21.15.47:0/725303216 >> v1:172.21.15.175:6789/0 conn(0x7fbfb80b8300 legacy=0x7fbfb80cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.900+0000 7fbfb659c640 1 -- 172.21.15.47:0/725303216 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb8161130 con 0x7fbfb815cb40 2024-03-23T09:42:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.901+0000 7fbfb659c640 1 --2- 172.21.15.47:0/725303216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb815cb40 0x7fbfb8160770 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00098a0 tx=0x7fbfb002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dee3d415036fc42 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.901+0000 7fbfb559a640 1 -- 172.21.15.47:0/725303216 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0030840 con 0x7fbfb815cb40 2024-03-23T09:42:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.901+0000 7fbfb559a640 1 -- 172.21.15.47:0/725303216 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb00309a0 con 0x7fbfb815cb40 2024-03-23T09:42:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.901+0000 7fbfb559a640 1 -- 172.21.15.47:0/725303216 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0030d30 con 0x7fbfb815cb40 2024-03-23T09:42:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.902+0000 7fbfbca5b640 1 -- 172.21.15.47:0/725303216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb815cb40 msgr2=0x7fbfb8160770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.902+0000 7fbfbca5b640 1 --2- 172.21.15.47:0/725303216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb815cb40 0x7fbfb8160770 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00098a0 tx=0x7fbfb002b980 comp rx=0 tx=0).stop 2024-03-23T09:42:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.902+0000 7fbfbca5b640 1 -- 172.21.15.47:0/725303216 shutdown_connections 2024-03-23T09:42:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.902+0000 7fbfbca5b640 1 --2- 172.21.15.47:0/725303216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8160cb0 0x7fbfb80d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.902+0000 7fbfbca5b640 1 --2- 172.21.15.47:0/725303216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb815cb40 0x7fbfb8160770 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.902+0000 7fbfbca5b640 1 -- 172.21.15.47:0/725303216 >> 172.21.15.47:0/725303216 conn(0x7fbfb806b160 msgr2=0x7fbfb806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.902+0000 7fbfbca5b640 1 -- 172.21.15.47:0/725303216 shutdown_connections 2024-03-23T09:42:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.902+0000 7fbfbca5b640 1 -- 172.21.15.47:0/725303216 wait complete. 2024-03-23T09:42:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.903+0000 7fbfbca5b640 1 Processor -- start 2024-03-23T09:42:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.903+0000 7fbfbca5b640 1 -- start start 2024-03-23T09:42:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfbca5b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb80b8300 0x7fbfb814dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfbca5b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb815cb40 0x7fbfb80caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfbca5b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb8160cb0 0x7fbfb80cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfbca5b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80cac60 con 0x7fbfb80b8300 2024-03-23T09:42:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfbca5b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80ca6a0 con 0x7fbfb8160cb0 2024-03-23T09:42:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfbca5b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80ca980 con 0x7fbfb815cb40 2024-03-23T09:42:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfb659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb80b8300 0x7fbfb814dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfb659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb80b8300 0x7fbfb814dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49118/0 (socket says 172.21.15.47:49118) 2024-03-23T09:42:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfb659c640 1 -- 172.21.15.47:0/3513925927 learned_addr learned my addr 172.21.15.47:0/3513925927 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfb6d9d640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb8160cb0 0x7fbfb80cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfb5d9b640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb815cb40 0x7fbfb80caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfb659c640 1 -- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb815cb40 msgr2=0x7fbfb80caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfb659c640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb815cb40 0x7fbfb80caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfb659c640 1 -- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb8160cb0 msgr2=0x7fbfb80cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfb659c640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb8160cb0 0x7fbfb80cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfb659c640 1 -- 172.21.15.47:0/3513925927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb00094d0 con 0x7fbfb80b8300 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfb6d9d640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb8160cb0 0x7fbfb80cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.904+0000 7fbfb5d9b640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb815cb40 0x7fbfb80caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.905+0000 7fbfb659c640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb80b8300 0x7fbfb814dca0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00098a0 tx=0x7fbfb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.905+0000 7fbfaeffd640 1 -- 172.21.15.47:0/3513925927 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0039c90 con 0x7fbfb80b8300 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.905+0000 7fbfaeffd640 1 -- 172.21.15.47:0/3513925927 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb0039df0 con 0x7fbfb80b8300 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.905+0000 7fbfbca5b640 1 -- 172.21.15.47:0/3513925927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb815d350 con 0x7fbfb80b8300 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.905+0000 7fbfaeffd640 1 -- 172.21.15.47:0/3513925927 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0043620 con 0x7fbfb80b8300 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.905+0000 7fbfbca5b640 1 -- 172.21.15.47:0/3513925927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb814e1e0 con 0x7fbfb80b8300 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.906+0000 7fbfbca5b640 1 -- 172.21.15.47:0/3513925927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb8072360 con 0x7fbfb80b8300 2024-03-23T09:42:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.906+0000 7fbfaeffd640 1 -- 172.21.15.47:0/3513925927 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbfb0043780 con 0x7fbfb80b8300 2024-03-23T09:42:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.907+0000 7fbfaeffd640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbf9806ac00 0x7fbf9806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.907+0000 7fbfaeffd640 1 -- 172.21.15.47:0/3513925927 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfb003e070 con 0x7fbfb80b8300 2024-03-23T09:42:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.907+0000 7fbfb5d9b640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbf9806ac00 0x7fbf9806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.907+0000 7fbfb5d9b640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbf9806ac00 0x7fbf9806d0c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80c81a0 tx=0x7fbfa0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:01.911+0000 7fbfaeffd640 1 -- 172.21.15.47:0/3513925927 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbfb0077740 con 0x7fbfb80b8300 2024-03-23T09:42:02.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.127+0000 7fbfbca5b640 1 -- 172.21.15.47:0/3513925927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfb8066be0 con 0x7fbfb80b8300 2024-03-23T09:42:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.127+0000 7fbfaeffd640 1 -- 172.21.15.47:0/3513925927 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fbfb007ad60 con 0x7fbfb80b8300 2024-03-23T09:42:02.127 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:02.127 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3373309448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3373309448},{"type":"v1","addr":"172.21.15.175:6836","nonce":3373309448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4118066020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4118066020},{"type":"v1","addr":"172.21.15.175:6837","nonce":4118066020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2916649516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2916649516},{"type":"v1","addr":"172.21.15.47:6835","nonce":2916649516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:59.719783+0000","modified":"2024-03-23T09:42:01.774130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6834/400299689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":400299689},{"type":"v1","addr":"172.21.15.47:6834","nonce":400299689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T09:42:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2024-03-23T09:42:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.130+0000 7fbfbca5b640 1 -- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbf9806ac00 msgr2=0x7fbf9806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.130+0000 7fbfbca5b640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbf9806ac00 0x7fbf9806d0c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80c81a0 tx=0x7fbfa0009040 comp rx=0 tx=0).stop 2024-03-23T09:42:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.130+0000 7fbfbca5b640 1 -- 172.21.15.47:0/3513925927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb80b8300 msgr2=0x7fbfb814dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.130+0000 7fbfbca5b640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb80b8300 0x7fbfb814dca0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00098a0 tx=0x7fbfb0004730 comp rx=0 tx=0).stop 2024-03-23T09:42:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.130+0000 7fbfbca5b640 1 -- 172.21.15.47:0/3513925927 shutdown_connections 2024-03-23T09:42:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.130+0000 7fbfbca5b640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbf9806ac00 0x7fbf9806d0c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.130+0000 7fbfbca5b640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb8160cb0 0x7fbfb80cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.130+0000 7fbfbca5b640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb815cb40 0x7fbfb80caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.130+0000 7fbfbca5b640 1 --2- 172.21.15.47:0/3513925927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb80b8300 0x7fbfb814dca0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.130+0000 7fbfbca5b640 1 -- 172.21.15.47:0/3513925927 >> 172.21.15.47:0/3513925927 conn(0x7fbfb806b160 msgr2=0x7fbfb80c85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:02.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.130+0000 7fbfbca5b640 1 -- 172.21.15.47:0/3513925927 shutdown_connections 2024-03-23T09:42:02.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.130+0000 7fbfbca5b640 1 -- 172.21.15.47:0/3513925927 wait complete. 2024-03-23T09:42:02.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:42:02.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.241+0000 7f312f851640 1 Processor -- start 2024-03-23T09:42:02.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.242+0000 7f312f851640 1 -- start start 2024-03-23T09:42:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.242+0000 7f312f851640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31280cd0f0 0x7f31280cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.242+0000 7f312f851640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31280ca4a0 con 0x7f312806de20 2024-03-23T09:42:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.242+0000 7f312f851640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31280cda10 con 0x7f31280cb200 2024-03-23T09:42:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.242+0000 7f312f851640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31280ca6f0 con 0x7f31280cd0f0 2024-03-23T09:42:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.242+0000 7f312d5ed640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31280cd0f0 0x7f31280cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.242+0000 7f312ddee640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f312806de20 0x7f312806a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58454/0 (socket says 172.21.15.47:58454) 2024-03-23T09:42:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.242+0000 7f312d5ed640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31280cd0f0 0x7f31280cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49132/0 (socket says 172.21.15.47:49132) 2024-03-23T09:42:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.242+0000 7f312ddee640 1 -- 172.21.15.47:0/2807832080 learned_addr learned my addr 172.21.15.47:0/2807832080 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.243+0000 7f312d5ed640 1 -- 172.21.15.47:0/2807832080 >> v1:172.21.15.175:6789/0 conn(0x7f31280cb200 legacy=0x7f312806a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.243+0000 7f312d5ed640 1 -- 172.21.15.47:0/2807832080 >> v1:172.21.15.47:6789/0 conn(0x7f312806de20 legacy=0x7f312806a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.243+0000 7f311ffff640 1 -- 172.21.15.47:0/2807832080 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632320658 0 0) 0x7f31280ca4a0 con 0x7f312806de20 2024-03-23T09:42:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.243+0000 7f312d5ed640 1 -- 172.21.15.47:0/2807832080 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31280ca9d0 con 0x7f31280cd0f0 2024-03-23T09:42:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.244+0000 7f312d5ed640 1 --2- 172.21.15.47:0/2807832080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31280cd0f0 0x7f31280cd4d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f31240097a0 tx=0x7f312402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f88788d3324cae server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.244+0000 7f311ffff640 1 -- 172.21.15.47:0/2807832080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124031830 con 0x7f31280cd0f0 2024-03-23T09:42:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.244+0000 7f311ffff640 1 -- 172.21.15.47:0/2807832080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124031990 con 0x7f31280cd0f0 2024-03-23T09:42:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.244+0000 7f311ffff640 1 -- 172.21.15.47:0/2807832080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124031d20 con 0x7f31280cd0f0 2024-03-23T09:42:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.244+0000 7f312f851640 1 -- 172.21.15.47:0/2807832080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31280cd0f0 msgr2=0x7f31280cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.244+0000 7f312f851640 1 --2- 172.21.15.47:0/2807832080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31280cd0f0 0x7f31280cd4d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f31240097a0 tx=0x7f312402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.244+0000 7f312f851640 1 -- 172.21.15.47:0/2807832080 shutdown_connections 2024-03-23T09:42:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.244+0000 7f312f851640 1 --2- 172.21.15.47:0/2807832080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31280cd0f0 0x7f31280cd4d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.244+0000 7f312f851640 1 -- 172.21.15.47:0/2807832080 >> 172.21.15.47:0/2807832080 conn(0x7f31280cb7c0 msgr2=0x7f31280cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.244+0000 7f312f851640 1 -- 172.21.15.47:0/2807832080 shutdown_connections 2024-03-23T09:42:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.244+0000 7f312f851640 1 -- 172.21.15.47:0/2807832080 wait complete. 2024-03-23T09:42:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.245+0000 7f312f851640 1 Processor -- start 2024-03-23T09:42:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.246+0000 7f312f851640 1 -- start start 2024-03-23T09:42:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.246+0000 7f312f851640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f312806de20 0x7f31281f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.246+0000 7f312f851640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31280cb200 0x7f31280b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.246+0000 7f312f851640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31280cd0f0 0x7f31280c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.246+0000 7f312f851640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31280c8820 con 0x7f31280cb200 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.246+0000 7f312f851640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31280c8a50 con 0x7f312806de20 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.246+0000 7f312f851640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31280c8d30 con 0x7f31280cd0f0 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.246+0000 7f312cdec640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31280cb200 0x7f31280b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.246+0000 7f312cdec640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31280cb200 0x7f31280b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49142/0 (socket says 172.21.15.47:49142) 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.246+0000 7f312cdec640 1 -- 172.21.15.47:0/3029426075 learned_addr learned my addr 172.21.15.47:0/3029426075 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.247+0000 7f312ddee640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31280cd0f0 0x7f31280c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.247+0000 7f312d5ed640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f312806de20 0x7f31281f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.247+0000 7f312cdec640 1 -- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31280cd0f0 msgr2=0x7f31280c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.247+0000 7f312cdec640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31280cd0f0 0x7f31280c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.247+0000 7f312cdec640 1 -- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f312806de20 msgr2=0x7f31281f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.247+0000 7f312cdec640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f312806de20 0x7f31281f1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.247+0000 7f312cdec640 1 -- 172.21.15.47:0/3029426075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3124009400 con 0x7f31280cb200 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.247+0000 7f312cdec640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31280cb200 0x7f31280b3ca0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f311800c450 tx=0x7f311800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.247+0000 7f311dffb640 1 -- 172.21.15.47:0/3029426075 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31180052d0 con 0x7f31280cb200 2024-03-23T09:42:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.247+0000 7f311dffb640 1 -- 172.21.15.47:0/3029426075 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3118005430 con 0x7f31280cb200 2024-03-23T09:42:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.247+0000 7f311dffb640 1 -- 172.21.15.47:0/3029426075 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118012870 con 0x7f31280cb200 2024-03-23T09:42:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.247+0000 7f312f851640 1 -- 172.21.15.47:0/3029426075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31280c9010 con 0x7f31280cb200 2024-03-23T09:42:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.247+0000 7f312f851640 1 -- 172.21.15.47:0/3029426075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31280b3780 con 0x7f31280cb200 2024-03-23T09:42:02.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.248+0000 7f312f851640 1 -- 172.21.15.47:0/3029426075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31280672f0 con 0x7f31280cb200 2024-03-23T09:42:02.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.248+0000 7f311dffb640 1 -- 172.21.15.47:0/3029426075 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f31180129d0 con 0x7f31280cb200 2024-03-23T09:42:02.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.249+0000 7f311dffb640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f310c06ac00 0x7f310c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.249+0000 7f311dffb640 1 -- 172.21.15.47:0/3029426075 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3118088f50 con 0x7f31280cb200 2024-03-23T09:42:02.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.249+0000 7f312d5ed640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f310c06ac00 0x7f310c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.249+0000 7f312d5ed640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f310c06ac00 0x7f310c06d0c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3124036000 tx=0x7f312403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:02.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.253+0000 7f311dffb640 1 -- 172.21.15.47:0/3029426075 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f31180540a0 con 0x7f31280cb200 2024-03-23T09:42:02.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.433+0000 7f312f851640 1 -- 172.21.15.47:0/3029426075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3128066040 con 0x7f31280cb200 2024-03-23T09:42:02.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.434+0000 7f311dffb640 1 -- 172.21.15.47:0/3029426075 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f31180576c0 con 0x7f31280cb200 2024-03-23T09:42:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:02.434 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:00.721483+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:58.297353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:58.685375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:42:02.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.436+0000 7f312f851640 1 -- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f310c06ac00 msgr2=0x7f310c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.436+0000 7f312f851640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f310c06ac00 0x7f310c06d0c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3124036000 tx=0x7f312403c040 comp rx=0 tx=0).stop 2024-03-23T09:42:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.436+0000 7f312f851640 1 -- 172.21.15.47:0/3029426075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31280cb200 msgr2=0x7f31280b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.436+0000 7f312f851640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31280cb200 0x7f31280b3ca0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f311800c450 tx=0x7f311800c910 comp rx=0 tx=0).stop 2024-03-23T09:42:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.436+0000 7f312f851640 1 -- 172.21.15.47:0/3029426075 shutdown_connections 2024-03-23T09:42:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.436+0000 7f312f851640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f310c06ac00 0x7f310c06d0c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.436+0000 7f312f851640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31280cd0f0 0x7f31280c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.436+0000 7f312f851640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31280cb200 0x7f31280b3ca0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.436+0000 7f312f851640 1 --2- 172.21.15.47:0/3029426075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f312806de20 0x7f31281f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.436+0000 7f312f851640 1 -- 172.21.15.47:0/3029426075 >> 172.21.15.47:0/3029426075 conn(0x7f31280cb7c0 msgr2=0x7f31280b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.437+0000 7f312f851640 1 -- 172.21.15.47:0/3029426075 shutdown_connections 2024-03-23T09:42:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.437+0000 7f312f851640 1 -- 172.21.15.47:0/3029426075 wait complete. 2024-03-23T09:42:02.447 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T09:42:02.447 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:42:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.548+0000 7fe5e2f3e640 1 Processor -- start 2024-03-23T09:42:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.548+0000 7fe5e2f3e640 1 -- start start 2024-03-23T09:42:02.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.549+0000 7fe5e2f3e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe5dc15ca90 0x7fe5dc160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.549+0000 7fe5e2f3e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5dc0cde10 0x7fe5dc0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.549+0000 7fe5e2f3e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5dc0d11e0 con 0x7fe5dc0b8350 2024-03-23T09:42:02.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.549+0000 7fe5e2f3e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5dc160f60 con 0x7fe5dc0cde10 2024-03-23T09:42:02.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.549+0000 7fe5e2f3e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5dc161240 con 0x7fe5dc15ca90 2024-03-23T09:42:02.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.549+0000 7fe5e14db640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5dc0cde10 0x7fe5dc0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.549+0000 7fe5e14db640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5dc0cde10 0x7fe5dc0d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49148/0 (socket says 172.21.15.47:49148) 2024-03-23T09:42:02.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.549+0000 7fe5e14db640 1 -- 172.21.15.47:0/2728898131 learned_addr learned my addr 172.21.15.47:0/2728898131 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:02.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.549+0000 7fe5e0cda640 1 --2- 172.21.15.47:0/2728898131 >> v2:172.21.15.175:3300/0 conn(0x7fe5dc15ca90 0x7fe5dc160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.549+0000 7fe5e14db640 1 -- 172.21.15.47:0/2728898131 >> v2:172.21.15.175:3300/0 conn(0x7fe5dc15ca90 msgr2=0x7fe5dc160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:02.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.549+0000 7fe5e14db640 1 --2- 172.21.15.47:0/2728898131 >> v2:172.21.15.175:3300/0 conn(0x7fe5dc15ca90 0x7fe5dc160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.549+0000 7fe5e14db640 1 -- 172.21.15.47:0/2728898131 >> v1:172.21.15.175:6789/0 conn(0x7fe5dc0b8350 legacy=0x7fe5dc0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:02.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.549+0000 7fe5e14db640 1 -- 172.21.15.47:0/2728898131 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc161520 con 0x7fe5dc0cde10 2024-03-23T09:42:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.550+0000 7fe5e14db640 1 --2- 172.21.15.47:0/2728898131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5dc0cde10 0x7fe5dc0d1670 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc066680 tx=0x7fe5d402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529253ae3f1fd73d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.550+0000 7fe5db7fe640 1 -- 172.21.15.47:0/2728898131 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d402c9f0 con 0x7fe5dc0cde10 2024-03-23T09:42:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.550+0000 7fe5db7fe640 1 -- 172.21.15.47:0/2728898131 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d402cb50 con 0x7fe5dc0cde10 2024-03-23T09:42:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.550+0000 7fe5db7fe640 1 -- 172.21.15.47:0/2728898131 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40343f0 con 0x7fe5dc0cde10 2024-03-23T09:42:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.551+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/2728898131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5dc0cde10 msgr2=0x7fe5dc0d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.551+0000 7fe5e2f3e640 1 --2- 172.21.15.47:0/2728898131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5dc0cde10 0x7fe5dc0d1670 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc066680 tx=0x7fe5d402be50 comp rx=0 tx=0).stop 2024-03-23T09:42:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.551+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/2728898131 shutdown_connections 2024-03-23T09:42:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.551+0000 7fe5e2f3e640 1 --2- 172.21.15.47:0/2728898131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5dc0cde10 0x7fe5dc0d1670 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.551+0000 7fe5e2f3e640 1 --2- 172.21.15.47:0/2728898131 >> v2:172.21.15.175:3300/0 conn(0x7fe5dc15ca90 0x7fe5dc160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.551+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/2728898131 >> 172.21.15.47:0/2728898131 conn(0x7fe5dc0659d0 msgr2=0x7fe5dc065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.551+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/2728898131 shutdown_connections 2024-03-23T09:42:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.551+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/2728898131 wait complete. 2024-03-23T09:42:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.552+0000 7fe5e2f3e640 1 Processor -- start 2024-03-23T09:42:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.552+0000 7fe5e2f3e640 1 -- start start 2024-03-23T09:42:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e2f3e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5dc0b8350 0x7fe5dc150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e2f3e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5dc0cde10 0x7fe5dc153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e2f3e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5dc15ca90 0x7fe5dc15f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e2f3e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc1513b0 con 0x7fe5dc0b8350 2024-03-23T09:42:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e2f3e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc151680 con 0x7fe5dc0cde10 2024-03-23T09:42:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e2f3e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc151960 con 0x7fe5dc15ca90 2024-03-23T09:42:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e0cda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5dc0b8350 0x7fe5dc150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e0cda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5dc0b8350 0x7fe5dc150e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49154/0 (socket says 172.21.15.47:49154) 2024-03-23T09:42:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e0cda640 1 -- 172.21.15.47:0/906224876 learned_addr learned my addr 172.21.15.47:0/906224876 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e0cda640 1 -- 172.21.15.47:0/906224876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5dc15ca90 msgr2=0x7fe5dc15f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e0cda640 1 --2- 172.21.15.47:0/906224876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5dc15ca90 0x7fe5dc15f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e0cda640 1 -- 172.21.15.47:0/906224876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5dc0cde10 msgr2=0x7fe5dc153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e0cda640 1 --2- 172.21.15.47:0/906224876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5dc0cde10 0x7fe5dc153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e0cda640 1 -- 172.21.15.47:0/906224876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d4009b30 con 0x7fe5dc0b8350 2024-03-23T09:42:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e0cda640 1 --2- 172.21.15.47:0/906224876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5dc0b8350 0x7fe5dc150e70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000d800 tx=0x7fe5d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5d97fa640 1 -- 172.21.15.47:0/906224876 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000b6c0 con 0x7fe5dc0b8350 2024-03-23T09:42:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5d97fa640 1 -- 172.21.15.47:0/906224876 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d000b820 con 0x7fe5dc0b8350 2024-03-23T09:42:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5d97fa640 1 -- 172.21.15.47:0/906224876 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000bbb0 con 0x7fe5dc0b8350 2024-03-23T09:42:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/906224876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5dc151c40 con 0x7fe5dc0b8350 2024-03-23T09:42:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.553+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/906224876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5dc152200 con 0x7fe5dc0b8350 2024-03-23T09:42:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.554+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/906224876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5dc072360 con 0x7fe5dc0b8350 2024-03-23T09:42:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.555+0000 7fe5d97fa640 1 -- 172.21.15.47:0/906224876 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe5d000bd10 con 0x7fe5dc0b8350 2024-03-23T09:42:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.555+0000 7fe5d97fa640 1 --2- 172.21.15.47:0/906224876 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5c006ac30 0x7fe5c006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.555+0000 7fe5d97fa640 1 -- 172.21.15.47:0/906224876 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5d0089810 con 0x7fe5dc0b8350 2024-03-23T09:42:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.555+0000 7fe5dbfff640 1 --2- 172.21.15.47:0/906224876 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5c006ac30 0x7fe5c006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.556+0000 7fe5dbfff640 1 --2- 172.21.15.47:0/906224876 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5c006ac30 0x7fe5c006d0f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8006fd0 tx=0x7fe5c8006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:02.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.560+0000 7fe5d97fa640 1 -- 172.21.15.47:0/906224876 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe5d0054a80 con 0x7fe5dc0b8350 2024-03-23T09:42:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.735+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/906224876 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5dc160c50 con 0x7fe5c006ac30 2024-03-23T09:42:02.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.742+0000 7fe5d97fa640 1 -- 172.21.15.47:0/906224876 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256110 (secure 0 0 0) 0x7fe5c8013430 con 0x7fe5c006ac30 2024-03-23T09:42:02.745 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:02.746 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:42:02.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.745+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/906224876 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5c006ac30 msgr2=0x7fe5c006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.745+0000 7fe5e2f3e640 1 --2- 172.21.15.47:0/906224876 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5c006ac30 0x7fe5c006d0f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8006fd0 tx=0x7fe5c8006bc0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.745+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/906224876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5dc0b8350 msgr2=0x7fe5dc150e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.745+0000 7fe5e2f3e640 1 --2- 172.21.15.47:0/906224876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5dc0b8350 0x7fe5dc150e70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000d800 tx=0x7fe5d000dcc0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.745+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/906224876 shutdown_connections 2024-03-23T09:42:02.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.745+0000 7fe5e2f3e640 1 --2- 172.21.15.47:0/906224876 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5c006ac30 0x7fe5c006d0f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.745+0000 7fe5e2f3e640 1 --2- 172.21.15.47:0/906224876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5dc15ca90 0x7fe5dc15f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.745+0000 7fe5e2f3e640 1 --2- 172.21.15.47:0/906224876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5dc0cde10 0x7fe5dc153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.745+0000 7fe5e2f3e640 1 --2- 172.21.15.47:0/906224876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5dc0b8350 0x7fe5dc150e70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.745+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/906224876 >> 172.21.15.47:0/906224876 conn(0x7fe5dc0659d0 msgr2=0x7fe5dc15d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:02.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.745+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/906224876 shutdown_connections 2024-03-23T09:42:02.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.745+0000 7fe5e2f3e640 1 -- 172.21.15.47:0/906224876 wait complete. 2024-03-23T09:42:02.756 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-03-23T09:42:02.351943+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236416,"kb_used_data":3776,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738304,"statfs":{"total":773094113280,"available":772852023296,"internally_reserved":0,"allocated":3866624,"data_stored":2181546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.002137"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-23T09:41:53.388976+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:41:53.388976+0000","last_peered":"2024-03-23T09:41:53.388976+0000","last_clean":"2024-03-23T09:41:53.388976+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:41:53.388976+0000","last_undegraded":"2024-03-23T09:41:53.388976+0000","last_fullsized":"2024-03-23T09:41:53.388976+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":929792,"data_stored":715624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27352,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":331776,"data_stored":124637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27356,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31456,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.38},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.26000000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.29799999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.42199999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.49199999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.63900000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.71899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31456,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31452,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.27100000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.318},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.34899999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":0.33600000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.502}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73199999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.77100000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.58299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":925696,"data_stored":715005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:42:02.761 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:42:02.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.897+0000 7f5358377640 1 Processor -- start 2024-03-23T09:42:02.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.897+0000 7f5358377640 1 -- start start 2024-03-23T09:42:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.898+0000 7f5358377640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5350159730 0x7f535015a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.898+0000 7f5358377640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53500c5fb0 0x7f53500cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.898+0000 7f5358377640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53500cf1b0 con 0x7f53500ce5f0 2024-03-23T09:42:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.898+0000 7f5358377640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53500cefb0 con 0x7f5350159730 2024-03-23T09:42:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.898+0000 7f5358377640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535015aca0 con 0x7f53500c5fb0 2024-03-23T09:42:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.898+0000 7f5356113640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5350159730 0x7f535015a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.898+0000 7f5356113640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5350159730 0x7f535015a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49164/0 (socket says 172.21.15.47:49164) 2024-03-23T09:42:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.898+0000 7f5356113640 1 -- 172.21.15.47:0/1156397407 learned_addr learned my addr 172.21.15.47:0/1156397407 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.899+0000 7f5356914640 1 --2- 172.21.15.47:0/1156397407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53500c5fb0 0x7f53500cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.899+0000 7f5356113640 1 -- 172.21.15.47:0/1156397407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53500c5fb0 msgr2=0x7f53500cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.899+0000 7f5356113640 1 --2- 172.21.15.47:0/1156397407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53500c5fb0 0x7f53500cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.899+0000 7f5356113640 1 -- 172.21.15.47:0/1156397407 >> v1:172.21.15.175:6789/0 conn(0x7f53500ce5f0 legacy=0x7f53500c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.899+0000 7f5356113640 1 -- 172.21.15.47:0/1156397407 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53500cf490 con 0x7f5350159730 2024-03-23T09:42:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.900+0000 7f5356113640 1 --2- 172.21.15.47:0/1156397407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350159730 0x7f535015a760 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f534000d180 tx=0x7f534002e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ef2a3b51c064d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.900+0000 7f5355111640 1 -- 172.21.15.47:0/1156397407 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534000b0d0 con 0x7f5350159730 2024-03-23T09:42:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.900+0000 7f5355111640 1 -- 172.21.15.47:0/1156397407 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534000b230 con 0x7f5350159730 2024-03-23T09:42:02.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.900+0000 7f5355111640 1 -- 172.21.15.47:0/1156397407 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340009600 con 0x7f5350159730 2024-03-23T09:42:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.900+0000 7f5358377640 1 -- 172.21.15.47:0/1156397407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350159730 msgr2=0x7f535015a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.900+0000 7f5358377640 1 --2- 172.21.15.47:0/1156397407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350159730 0x7f535015a760 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f534000d180 tx=0x7f534002e160 comp rx=0 tx=0).stop 2024-03-23T09:42:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.900+0000 7f5358377640 1 -- 172.21.15.47:0/1156397407 shutdown_connections 2024-03-23T09:42:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.900+0000 7f5358377640 1 --2- 172.21.15.47:0/1156397407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53500c5fb0 0x7f53500cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.900+0000 7f5358377640 1 --2- 172.21.15.47:0/1156397407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350159730 0x7f535015a760 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.900+0000 7f5358377640 1 -- 172.21.15.47:0/1156397407 >> 172.21.15.47:0/1156397407 conn(0x7f53500b1410 msgr2=0x7f53500b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.900+0000 7f5358377640 1 -- 172.21.15.47:0/1156397407 shutdown_connections 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.900+0000 7f5358377640 1 -- 172.21.15.47:0/1156397407 wait complete. 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.902+0000 7f5358377640 1 Processor -- start 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.902+0000 7f5358377640 1 -- start start 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.902+0000 7f5358377640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53500c5fb0 0x7f53500b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.902+0000 7f5358377640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53500ce5f0 0x7f53500b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.902+0000 7f5358377640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5350159730 0x7f5350147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.902+0000 7f5358377640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53500b2990 con 0x7f53500c5fb0 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.902+0000 7f5358377640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53500b2c70 con 0x7f53500ce5f0 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.902+0000 7f5358377640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53500b2f50 con 0x7f5350159730 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.902+0000 7f5356113640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53500c5fb0 0x7f53500b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.902+0000 7f5356113640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53500c5fb0 0x7f53500b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49176/0 (socket says 172.21.15.47:49176) 2024-03-23T09:42:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.902+0000 7f5356113640 1 -- 172.21.15.47:0/741351397 learned_addr learned my addr 172.21.15.47:0/741351397 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f5355912640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53500ce5f0 0x7f53500b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f5356113640 1 -- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5350159730 msgr2=0x7f5350147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f5356914640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5350159730 0x7f5350147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f5356113640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5350159730 0x7f5350147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f5356113640 1 -- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53500ce5f0 msgr2=0x7f53500b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f5356113640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53500ce5f0 0x7f53500b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f5356113640 1 -- 172.21.15.47:0/741351397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534000bd40 con 0x7f53500c5fb0 2024-03-23T09:42:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f5355912640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53500ce5f0 0x7f53500b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f5356914640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5350159730 0x7f5350147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f5356113640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53500c5fb0 0x7f53500b0830 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5340038040 tx=0x7f5340004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f534effd640 1 -- 172.21.15.47:0/741351397 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534000b720 con 0x7f53500c5fb0 2024-03-23T09:42:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f534effd640 1 -- 172.21.15.47:0/741351397 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5340040660 con 0x7f53500c5fb0 2024-03-23T09:42:02.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f5358377640 1 -- 172.21.15.47:0/741351397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53500b3230 con 0x7f53500c5fb0 2024-03-23T09:42:02.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f534effd640 1 -- 172.21.15.47:0/741351397 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340040a10 con 0x7f53500c5fb0 2024-03-23T09:42:02.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.903+0000 7f5358377640 1 -- 172.21.15.47:0/741351397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53500b0d70 con 0x7f53500c5fb0 2024-03-23T09:42:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.904+0000 7f5358377640 1 -- 172.21.15.47:0/741351397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53500b0fc0 con 0x7f53500c5fb0 2024-03-23T09:42:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.905+0000 7f534effd640 1 -- 172.21.15.47:0/741351397 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5340040b70 con 0x7f53500c5fb0 2024-03-23T09:42:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.905+0000 7f534effd640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f533c06ac00 0x7f533c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.905+0000 7f534effd640 1 -- 172.21.15.47:0/741351397 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53400ae350 con 0x7f53500c5fb0 2024-03-23T09:42:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.905+0000 7f5355912640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f533c06ac00 0x7f533c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.906+0000 7f5355912640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f533c06ac00 0x7f533c06d0c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f53500c90c0 tx=0x7f5344009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:02.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:02.909+0000 7f534effd640 1 -- 172.21.15.47:0/741351397 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f53400795c0 con 0x7f53500c5fb0 2024-03-23T09:42:03.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.083+0000 7f5358377640 1 -- 172.21.15.47:0/741351397 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53500715c0 con 0x7f533c06ac00 2024-03-23T09:42:03.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.090+0000 7f534effd640 1 -- 172.21.15.47:0/741351397 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256110 (secure 0 0 0) 0x7f53440013e0 con 0x7f533c06ac00 2024-03-23T09:42:03.091 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:03.093 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:42:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.092+0000 7f5358377640 1 -- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f533c06ac00 msgr2=0x7f533c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.092+0000 7f5358377640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f533c06ac00 0x7f533c06d0c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f53500c90c0 tx=0x7f5344009040 comp rx=0 tx=0).stop 2024-03-23T09:42:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.092+0000 7f5358377640 1 -- 172.21.15.47:0/741351397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53500c5fb0 msgr2=0x7f53500b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.092+0000 7f5358377640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53500c5fb0 0x7f53500b0830 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5340038040 tx=0x7f5340004a40 comp rx=0 tx=0).stop 2024-03-23T09:42:03.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.092+0000 7f5358377640 1 -- 172.21.15.47:0/741351397 shutdown_connections 2024-03-23T09:42:03.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.092+0000 7f5358377640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f533c06ac00 0x7f533c06d0c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.092+0000 7f5358377640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5350159730 0x7f5350147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.092+0000 7f5358377640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53500ce5f0 0x7f53500b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.092+0000 7f5358377640 1 --2- 172.21.15.47:0/741351397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53500c5fb0 0x7f53500b0830 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.092+0000 7f5358377640 1 -- 172.21.15.47:0/741351397 >> 172.21.15.47:0/741351397 conn(0x7f53500b1410 msgr2=0x7f53500ca710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:03.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.092+0000 7f5358377640 1 -- 172.21.15.47:0/741351397 shutdown_connections 2024-03-23T09:42:03.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.092+0000 7f5358377640 1 -- 172.21.15.47:0/741351397 wait complete. 2024-03-23T09:42:03.103 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-03-23T09:42:02.351943+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236416,"kb_used_data":3776,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738304,"statfs":{"total":773094113280,"available":772852023296,"internally_reserved":0,"allocated":3866624,"data_stored":2181546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.002137"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-23T09:41:53.388976+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:41:53.388976+0000","last_peered":"2024-03-23T09:41:53.388976+0000","last_clean":"2024-03-23T09:41:53.388976+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:41:53.388976+0000","last_undegraded":"2024-03-23T09:41:53.388976+0000","last_fullsized":"2024-03-23T09:41:53.388976+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":929792,"data_stored":715624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27352,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":331776,"data_stored":124637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27356,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31456,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.38},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.26000000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.29799999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.42199999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.49199999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.63900000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.71899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31456,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31452,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.27100000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.318},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.34899999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":0.33600000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.502}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73199999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.77100000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.58299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":925696,"data_stored":715005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:42:03.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:42:03.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.244+0000 7fc1eac51640 1 Processor -- start 2024-03-23T09:42:03.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.245+0000 7fc1eac51640 1 -- start start 2024-03-23T09:42:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.245+0000 7fc1eac51640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc1e415ca90 0x7fc1e4160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.245+0000 7fc1eac51640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e40b8350 0x7fc1e40cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.245+0000 7fc1eac51640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e40d11e0 con 0x7fc1e40cdc40 2024-03-23T09:42:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.245+0000 7fc1eac51640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e40ce0d0 con 0x7fc1e415ca90 2024-03-23T09:42:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.245+0000 7fc1eac51640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e40d2680 con 0x7fc1e40b8350 2024-03-23T09:42:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.245+0000 7fc1e91ee640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc1e40cdc40 0x7fc1e40b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58470/0 (socket says 172.21.15.47:58470) 2024-03-23T09:42:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.245+0000 7fc1e91ee640 1 -- 172.21.15.47:0/969376487 learned_addr learned my addr 172.21.15.47:0/969376487 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.246+0000 7fc1e3fff640 1 --2- 172.21.15.47:0/969376487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e40b8350 0x7fc1e40cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.246+0000 7fc1e89ed640 1 --2- 172.21.15.47:0/969376487 >> v2:172.21.15.175:3300/0 conn(0x7fc1e415ca90 0x7fc1e4160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.246+0000 7fc1e37fe640 1 -- 172.21.15.47:0/969376487 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925517614 0 0) 0x7fc1e40d11e0 con 0x7fc1e40cdc40 2024-03-23T09:42:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.246+0000 7fc1e37fe640 1 -- 172.21.15.47:0/969376487 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1cc003520 con 0x7fc1e40cdc40 2024-03-23T09:42:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.246+0000 7fc1e89ed640 1 -- 172.21.15.47:0/969376487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e40b8350 msgr2=0x7fc1e40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.246+0000 7fc1e89ed640 1 --2- 172.21.15.47:0/969376487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e40b8350 0x7fc1e40cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.246+0000 7fc1e89ed640 1 -- 172.21.15.47:0/969376487 >> v1:172.21.15.47:6789/0 conn(0x7fc1e40cdc40 legacy=0x7fc1e40b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.246+0000 7fc1e89ed640 1 -- 172.21.15.47:0/969376487 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e4160f60 con 0x7fc1e415ca90 2024-03-23T09:42:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.246+0000 7fc1e3fff640 1 --2- 172.21.15.47:0/969376487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e40b8350 0x7fc1e40cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:42:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.247+0000 7fc1e89ed640 1 --2- 172.21.15.47:0/969376487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e415ca90 0x7fc1e4160710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40097a0 tx=0x7fc1d402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf46c27cfffd0086 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.247+0000 7fc1e37fe640 1 -- 172.21.15.47:0/969376487 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d402e070 con 0x7fc1e415ca90 2024-03-23T09:42:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.247+0000 7fc1e37fe640 1 -- 172.21.15.47:0/969376487 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d40315a0 con 0x7fc1e415ca90 2024-03-23T09:42:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.247+0000 7fc1e37fe640 1 -- 172.21.15.47:0/969376487 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4031930 con 0x7fc1e415ca90 2024-03-23T09:42:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.247+0000 7fc1eac51640 1 -- 172.21.15.47:0/969376487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e415ca90 msgr2=0x7fc1e4160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.247+0000 7fc1eac51640 1 --2- 172.21.15.47:0/969376487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e415ca90 0x7fc1e4160710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40097a0 tx=0x7fc1d402b840 comp rx=0 tx=0).stop 2024-03-23T09:42:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.248+0000 7fc1eac51640 1 -- 172.21.15.47:0/969376487 shutdown_connections 2024-03-23T09:42:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.248+0000 7fc1eac51640 1 --2- 172.21.15.47:0/969376487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e40b8350 0x7fc1e40cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.248+0000 7fc1eac51640 1 --2- 172.21.15.47:0/969376487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e415ca90 0x7fc1e4160710 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.248+0000 7fc1eac51640 1 -- 172.21.15.47:0/969376487 >> 172.21.15.47:0/969376487 conn(0x7fc1e40659d0 msgr2=0x7fc1e4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.248+0000 7fc1eac51640 1 -- 172.21.15.47:0/969376487 shutdown_connections 2024-03-23T09:42:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.248+0000 7fc1eac51640 1 -- 172.21.15.47:0/969376487 wait complete. 2024-03-23T09:42:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.249+0000 7fc1eac51640 1 Processor -- start 2024-03-23T09:42:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.249+0000 7fc1eac51640 1 -- start start 2024-03-23T09:42:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.249+0000 7fc1eac51640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e40b8350 0x7fc1e415eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.249+0000 7fc1eac51640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e40cdc40 0x7fc1e4150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1eac51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e415ca90 0x7fc1e4151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1eac51640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e4151b80 con 0x7fc1e415ca90 2024-03-23T09:42:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1eac51640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e415f0d0 con 0x7fc1e40b8350 2024-03-23T09:42:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1eac51640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e41518a0 con 0x7fc1e40cdc40 2024-03-23T09:42:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e91ee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e415ca90 0x7fc1e4151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e91ee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e415ca90 0x7fc1e4151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49190/0 (socket says 172.21.15.47:49190) 2024-03-23T09:42:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e91ee640 1 -- 172.21.15.47:0/449790297 learned_addr learned my addr 172.21.15.47:0/449790297 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e3fff640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e40cdc40 0x7fc1e4150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e89ed640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e40b8350 0x7fc1e415eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e91ee640 1 -- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e40cdc40 msgr2=0x7fc1e4150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e91ee640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e40cdc40 0x7fc1e4150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e91ee640 1 -- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e40b8350 msgr2=0x7fc1e415eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e91ee640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e40b8350 0x7fc1e415eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e91ee640 1 -- 172.21.15.47:0/449790297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d8009400 con 0x7fc1e415ca90 2024-03-23T09:42:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e91ee640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e415ca90 0x7fc1e4151fa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00caa0 tx=0x7fc1dc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e17fa640 1 -- 172.21.15.47:0/449790297 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc00ac00 con 0x7fc1e415ca90 2024-03-23T09:42:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e17fa640 1 -- 172.21.15.47:0/449790297 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1dc00ad60 con 0x7fc1e415ca90 2024-03-23T09:42:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1e17fa640 1 -- 172.21.15.47:0/449790297 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc012620 con 0x7fc1e415ca90 2024-03-23T09:42:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1eac51640 1 -- 172.21.15.47:0/449790297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d4009400 con 0x7fc1e415ca90 2024-03-23T09:42:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.250+0000 7fc1eac51640 1 -- 172.21.15.47:0/449790297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e415f2c0 con 0x7fc1e415ca90 2024-03-23T09:42:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.251+0000 7fc1eac51640 1 -- 172.21.15.47:0/449790297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e4072360 con 0x7fc1e415ca90 2024-03-23T09:42:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.252+0000 7fc1e17fa640 1 -- 172.21.15.47:0/449790297 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc1dc007070 con 0x7fc1e415ca90 2024-03-23T09:42:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.252+0000 7fc1e17fa640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc1d006ac00 0x7fc1d006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.252+0000 7fc1e17fa640 1 -- 172.21.15.47:0/449790297 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1dc089280 con 0x7fc1e415ca90 2024-03-23T09:42:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.252+0000 7fc1e89ed640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc1d006ac00 0x7fc1d006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.253+0000 7fc1e89ed640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc1d006ac00 0x7fc1d006d0c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4036000 tx=0x7fc1d403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:03.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.257+0000 7fc1e17fa640 1 -- 172.21.15.47:0/449790297 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc1dc054510 con 0x7fc1e415ca90 2024-03-23T09:42:03.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.438+0000 7fc1eac51640 1 -- 172.21.15.47:0/449790297 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc1e40d0e70 con 0x7fc1d006ac00 2024-03-23T09:42:03.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.445+0000 7fc1e17fa640 1 -- 172.21.15.47:0/449790297 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256110 (secure 0 0 0) 0x7fc1d40319d0 con 0x7fc1d006ac00 2024-03-23T09:42:03.447 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:03.448 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:42:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.447+0000 7fc1eac51640 1 -- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc1d006ac00 msgr2=0x7fc1d006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.447+0000 7fc1eac51640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc1d006ac00 0x7fc1d006d0c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4036000 tx=0x7fc1d403d040 comp rx=0 tx=0).stop 2024-03-23T09:42:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.447+0000 7fc1eac51640 1 -- 172.21.15.47:0/449790297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e415ca90 msgr2=0x7fc1e4151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.447+0000 7fc1eac51640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e415ca90 0x7fc1e4151fa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00caa0 tx=0x7fc1dc00cf60 comp rx=0 tx=0).stop 2024-03-23T09:42:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.448+0000 7fc1eac51640 1 -- 172.21.15.47:0/449790297 shutdown_connections 2024-03-23T09:42:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.448+0000 7fc1eac51640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc1d006ac00 0x7fc1d006d0c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.448+0000 7fc1eac51640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e415ca90 0x7fc1e4151fa0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.448+0000 7fc1eac51640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e40cdc40 0x7fc1e4150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.448+0000 7fc1eac51640 1 --2- 172.21.15.47:0/449790297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e40b8350 0x7fc1e415eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.448+0000 7fc1eac51640 1 -- 172.21.15.47:0/449790297 >> 172.21.15.47:0/449790297 conn(0x7fc1e40659d0 msgr2=0x7fc1e4156fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.448+0000 7fc1eac51640 1 -- 172.21.15.47:0/449790297 shutdown_connections 2024-03-23T09:42:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.448+0000 7fc1eac51640 1 -- 172.21.15.47:0/449790297 wait complete. 2024-03-23T09:42:03.458 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-03-23T09:42:02.351943+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236416,"kb_used_data":3776,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738304,"statfs":{"total":773094113280,"available":772852023296,"internally_reserved":0,"allocated":3866624,"data_stored":2181546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.002137"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-23T09:41:53.388976+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:41:53.388976+0000","last_peered":"2024-03-23T09:41:53.388976+0000","last_clean":"2024-03-23T09:41:53.388976+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:41:53.388976+0000","last_undegraded":"2024-03-23T09:41:53.388976+0000","last_fullsized":"2024-03-23T09:41:53.388976+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":929792,"data_stored":715624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27352,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":331776,"data_stored":124637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27356,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31456,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.38},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.26000000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.29799999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.42199999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.49199999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.63900000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.71899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31456,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31452,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.27100000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.318},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.34899999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":0.33600000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.502}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73199999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.77100000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.58299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":925696,"data_stored":715005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:42:03.462 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-03-23T09:42:03.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.463 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-03-23T09:42:03.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.463 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-03-23T09:42:03.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.463 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-03-23T09:42:03.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.463 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-03-23T09:42:03.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-03-23T09:42:03.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-03-23T09:42:03.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-03-23T09:42:03.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-03-23T09:42:03.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-03-23T09:42:03.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-03-23T09:42:03.465 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-03-23T09:42:03.465 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-03-23T09:42:03.465 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-03-23T09:42:03.465 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-03-23T09:42:03.465 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-03-23T09:42:03.465 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-03-23T09:42:03.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-03-23T09:42:03.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-03-23T09:42:03.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-03-23T09:42:03.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-03-23T09:42:03.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-03-23T09:42:03.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-03-23T09:42:03.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-03-23T09:42:03.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-03-23T09:42:03.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-03-23T09:42:03.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-03-23T09:42:03.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-03-23T09:42:03.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-03-23T09:42:03.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-03-23T09:42:03.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-03-23T09:42:03.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-03-23T09:42:03.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-03-23T09:42:03.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2024-03-23T09:42:03.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-03-23T09:42:03.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2024-03-23T09:42:03.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-03-23T09:42:03.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2024-03-23T09:42:03.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-03-23T09:42:03.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2024-03-23T09:42:03.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-03-23T09:42:03.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2024-03-23T09:42:03.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-03-23T09:42:03.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2024-03-23T09:42:03.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-03-23T09:42:03.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2024-03-23T09:42:03.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-03-23T09:42:03.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2024-03-23T09:42:03.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-03-23T09:42:03.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2024-03-23T09:42:03.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-03-23T09:42:03.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2024-03-23T09:42:03.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-03-23T09:42:03.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2024-03-23T09:42:03.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-03-23T09:42:03.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2024-03-23T09:42:03.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-03-23T09:42:03.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2024-03-23T09:42:03.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-03-23T09:42:03.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2024-03-23T09:42:03.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-03-23T09:42:03.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2024-03-23T09:42:03.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-03-23T09:42:03.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2024-03-23T09:42:03.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-03-23T09:42:03.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2024-03-23T09:42:03.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-03-23T09:42:03.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2024-03-23T09:42:03.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-03-23T09:42:03.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2024-03-23T09:42:03.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-03-23T09:42:03.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2024-03-23T09:42:03.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-03-23T09:42:03.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2024-03-23T09:42:03.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-03-23T09:42:03.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2024-03-23T09:42:03.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-03-23T09:42:03.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2024-03-23T09:42:03.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-03-23T09:42:03.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2024-03-23T09:42:03.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-03-23T09:42:03.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2024-03-23T09:42:03.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-03-23T09:42:03.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2024-03-23T09:42:03.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-03-23T09:42:03.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2024-03-23T09:42:03.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-03-23T09:42:03.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2024-03-23T09:42:03.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-03-23T09:42:03.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:59.349007+0000', 'last_change': '2024-03-23T09:41:59.349007+0000', 'last_active': '2024-03-23T09:41:59.349007+0000', 'last_peered': '2024-03-23T09:41:59.349007+0000', 'last_clean': '2024-03-23T09:41:59.349007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:59.349007+0000', 'last_undegraded': '2024-03-23T09:41:59.349007+0000', 'last_fullsized': '2024-03-23T09:41:59.349007+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:59.349007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2024-03-23T09:42:03.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:41:58.347492+0000', 'last_change': '2024-03-23T09:41:58.347492+0000', 'last_active': '2024-03-23T09:41:58.347492+0000', 'last_peered': '2024-03-23T09:41:58.347492+0000', 'last_clean': '2024-03-23T09:41:58.347492+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:41:58.347492+0000', 'last_undegraded': '2024-03-23T09:41:58.347492+0000', 'last_fullsized': '2024-03-23T09:41:58.347492+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'last_clean_scrub_stamp': '2024-03-23T09:41:58.347492+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:03.479 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-23T09:42:03.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.598+0000 7f67a6d00640 1 Processor -- start 2024-03-23T09:42:03.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.599+0000 7f67a6d00640 1 -- start start 2024-03-23T09:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.599+0000 7f67a6d00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a015a9f0 0x7f67a0098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.599+0000 7f67a6d00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67a0099860 0x7f67a0098f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.599+0000 7f67a6d00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a00ce760 con 0x7f67a00cb240 2024-03-23T09:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.599+0000 7f67a6d00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a00cea40 con 0x7f67a0099860 2024-03-23T09:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.599+0000 7f67a6d00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a00c4a00 con 0x7f67a015a9f0 2024-03-23T09:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.599+0000 7f679ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67a0099860 0x7f67a0098f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.599+0000 7f67a529d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f67a00cb240 0x7f67a00cb6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58484/0 (socket says 172.21.15.47:58484) 2024-03-23T09:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.599+0000 7f679ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67a0099860 0x7f67a0098f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49200/0 (socket says 172.21.15.47:49200) 2024-03-23T09:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.599+0000 7f67a529d640 1 -- 172.21.15.47:0/2690513307 learned_addr learned my addr 172.21.15.47:0/2690513307 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.600+0000 7f679ffff640 1 -- 172.21.15.47:0/2690513307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a015a9f0 msgr2=0x7f67a0098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.600+0000 7f679ffff640 1 --2- 172.21.15.47:0/2690513307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a015a9f0 0x7f67a0098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.600+0000 7f679ffff640 1 -- 172.21.15.47:0/2690513307 >> v1:172.21.15.47:6789/0 conn(0x7f67a00cb240 legacy=0x7f67a00cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.600+0000 7f679f7fe640 1 -- 172.21.15.47:0/2690513307 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334731546 0 0) 0x7f67a00ce760 con 0x7f67a00cb240 2024-03-23T09:42:03.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.600+0000 7f679ffff640 1 -- 172.21.15.47:0/2690513307 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a00c4340 con 0x7f67a0099860 2024-03-23T09:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.600+0000 7f679ffff640 1 --2- 172.21.15.47:0/2690513307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67a0099860 0x7f67a0098f40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f67940097a0 tx=0x7f679402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469795bff973d882 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.601+0000 7f679f7fe640 1 -- 172.21.15.47:0/2690513307 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679402e070 con 0x7f67a0099860 2024-03-23T09:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.601+0000 7f679f7fe640 1 -- 172.21.15.47:0/2690513307 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67940315a0 con 0x7f67a0099860 2024-03-23T09:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.601+0000 7f679f7fe640 1 -- 172.21.15.47:0/2690513307 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794031930 con 0x7f67a0099860 2024-03-23T09:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.601+0000 7f67a6d00640 1 -- 172.21.15.47:0/2690513307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67a0099860 msgr2=0x7f67a0098f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.601+0000 7f67a6d00640 1 --2- 172.21.15.47:0/2690513307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67a0099860 0x7f67a0098f40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f67940097a0 tx=0x7f679402b840 comp rx=0 tx=0).stop 2024-03-23T09:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.601+0000 7f67a6d00640 1 -- 172.21.15.47:0/2690513307 shutdown_connections 2024-03-23T09:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.601+0000 7f67a6d00640 1 --2- 172.21.15.47:0/2690513307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67a0099860 0x7f67a0098f40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.601+0000 7f67a6d00640 1 --2- 172.21.15.47:0/2690513307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a015a9f0 0x7f67a0098a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.601+0000 7f67a6d00640 1 -- 172.21.15.47:0/2690513307 >> 172.21.15.47:0/2690513307 conn(0x7f67a00b2140 msgr2=0x7f67a00b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.601+0000 7f67a6d00640 1 -- 172.21.15.47:0/2690513307 shutdown_connections 2024-03-23T09:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.601+0000 7f67a6d00640 1 -- 172.21.15.47:0/2690513307 wait complete. 2024-03-23T09:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.602+0000 7f67a6d00640 1 Processor -- start 2024-03-23T09:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.602+0000 7f67a6d00640 1 -- start start 2024-03-23T09:42:03.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.603+0000 7f67a6d00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a0099860 0x7f67a01f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.603+0000 7f67a6d00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67a00cb240 0x7f67a015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.603+0000 7f67a6d00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a015a9f0 0x7f67a015bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.603+0000 7f67a6d00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a015c510 con 0x7f67a00cb240 2024-03-23T09:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.603+0000 7f67a6d00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a006bda0 con 0x7f67a015a9f0 2024-03-23T09:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.603+0000 7f67a6d00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a006c080 con 0x7f67a0099860 2024-03-23T09:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.603+0000 7f679ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67a00cb240 0x7f67a015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.603+0000 7f679ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67a00cb240 0x7f67a015b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49210/0 (socket says 172.21.15.47:49210) 2024-03-23T09:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.603+0000 7f679ffff640 1 -- 172.21.15.47:0/132497323 learned_addr learned my addr 172.21.15.47:0/132497323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.604+0000 7f679ffff640 1 -- 172.21.15.47:0/132497323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a0099860 msgr2=0x7f67a01f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.604+0000 7f679ffff640 1 --2- 172.21.15.47:0/132497323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a0099860 0x7f67a01f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.604+0000 7f679ffff640 1 -- 172.21.15.47:0/132497323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a015a9f0 msgr2=0x7f67a015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.604+0000 7f679ffff640 1 --2- 172.21.15.47:0/132497323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a015a9f0 0x7f67a015bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.604+0000 7f679ffff640 1 -- 172.21.15.47:0/132497323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6794009400 con 0x7f67a00cb240 2024-03-23T09:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.604+0000 7f679ffff640 1 --2- 172.21.15.47:0/132497323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67a00cb240 0x7f67a015b6b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f67940097a0 tx=0x7f67940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.604+0000 7f679d7fa640 1 -- 172.21.15.47:0/132497323 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794004230 con 0x7f67a00cb240 2024-03-23T09:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.604+0000 7f679d7fa640 1 -- 172.21.15.47:0/132497323 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67940043b0 con 0x7f67a00cb240 2024-03-23T09:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.604+0000 7f67a6d00640 1 -- 172.21.15.47:0/132497323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a006c360 con 0x7f67a00cb240 2024-03-23T09:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.604+0000 7f679d7fa640 1 -- 172.21.15.47:0/132497323 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794004520 con 0x7f67a00cb240 2024-03-23T09:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.604+0000 7f67a6d00640 1 -- 172.21.15.47:0/132497323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a006c920 con 0x7f67a00cb240 2024-03-23T09:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.605+0000 7f67a6d00640 1 -- 172.21.15.47:0/132497323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a006cb10 con 0x7f67a00cb240 2024-03-23T09:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.605+0000 7f679d7fa640 1 -- 172.21.15.47:0/132497323 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6794032880 con 0x7f67a00cb240 2024-03-23T09:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.606+0000 7f679d7fa640 1 --2- 172.21.15.47:0/132497323 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f678c06ac00 0x7f678c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.606+0000 7f679d7fa640 1 -- 172.21.15.47:0/132497323 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67940ab4a0 con 0x7f67a00cb240 2024-03-23T09:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.606+0000 7f67a4a9c640 1 --2- 172.21.15.47:0/132497323 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f678c06ac00 0x7f678c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.606+0000 7f67a4a9c640 1 --2- 172.21.15.47:0/132497323 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f678c06ac00 0x7f678c06d0c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f679800ab90 tx=0x7f6798009250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:03.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.610+0000 7f679d7fa640 1 -- 172.21.15.47:0/132497323 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6794076710 con 0x7f67a00cb240 2024-03-23T09:42:03.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.857+0000 7f67a6d00640 1 -- 172.21.15.47:0/132497323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f67a00ce400 con 0x7f67a00cb240 2024-03-23T09:42:03.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.857+0000 7f679d7fa640 1 -- 172.21.15.47:0/132497323 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1569 (secure 0 0 0) 0x7f679403ae60 con 0x7f67a00cb240 2024-03-23T09:42:03.857 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:03.858 INFO:teuthology.orchestra.run.smithi047.stdout:{"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":77,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1711186856,"num_in_osds":8,"osd_in_since":1711186848,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":242089984,"bytes_avail":772852023296,"bytes_total":773094113280,"unknown_pgs_ratio":0.98969072103500366},"fsmap":{"epoch":12,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5196}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-03-23T09:41:58.350978+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-23T09:42:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.859+0000 7f67a6d00640 1 -- 172.21.15.47:0/132497323 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f678c06ac00 msgr2=0x7f678c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.859+0000 7f67a6d00640 1 --2- 172.21.15.47:0/132497323 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f678c06ac00 0x7f678c06d0c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f679800ab90 tx=0x7f6798009250 comp rx=0 tx=0).stop 2024-03-23T09:42:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.859+0000 7f67a6d00640 1 -- 172.21.15.47:0/132497323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67a00cb240 msgr2=0x7f67a015b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.859+0000 7f67a6d00640 1 --2- 172.21.15.47:0/132497323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67a00cb240 0x7f67a015b6b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f67940097a0 tx=0x7f67940046c0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.859+0000 7f67a6d00640 1 -- 172.21.15.47:0/132497323 shutdown_connections 2024-03-23T09:42:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.859+0000 7f67a6d00640 1 --2- 172.21.15.47:0/132497323 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f678c06ac00 0x7f678c06d0c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.859+0000 7f67a6d00640 1 --2- 172.21.15.47:0/132497323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a015a9f0 0x7f67a015bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.859+0000 7f67a6d00640 1 --2- 172.21.15.47:0/132497323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67a00cb240 0x7f67a015b6b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.859+0000 7f67a6d00640 1 --2- 172.21.15.47:0/132497323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a0099860 0x7f67a01f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.859+0000 7f67a6d00640 1 -- 172.21.15.47:0/132497323 >> 172.21.15.47:0/132497323 conn(0x7f67a00b2140 msgr2=0x7f67a00cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.859+0000 7f67a6d00640 1 -- 172.21.15.47:0/132497323 shutdown_connections 2024-03-23T09:42:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.859+0000 7f67a6d00640 1 -- 172.21.15.47:0/132497323 wait complete. 2024-03-23T09:42:03.869 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-23T09:42:03.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:42:03.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.971+0000 7fe1e1b00640 1 Processor -- start 2024-03-23T09:42:03.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.971+0000 7fe1e1b00640 1 -- start start 2024-03-23T09:42:03.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.971+0000 7fe1e1b00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1dc06a3f0 0x7fe1dc0ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.971+0000 7fe1e1b00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1dc0b2390 0x7fe1dc0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.971+0000 7fe1e1b00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1dc0cf320 con 0x7fe1dc0afe10 2024-03-23T09:42:03.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.971+0000 7fe1e1b00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc0ced60 con 0x7fe1dc06a3f0 2024-03-23T09:42:03.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.971+0000 7fe1e1b00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0cf040 con 0x7fe1dc0b2390 2024-03-23T09:42:03.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.971+0000 7fe1daffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1dc06a3f0 0x7fe1dc0ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.972+0000 7fe1db7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe1dc0afe10 0x7fe1dc069ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58496/0 (socket says 172.21.15.47:58496) 2024-03-23T09:42:03.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.972+0000 7fe1db7fe640 1 -- 172.21.15.47:0/3669710424 learned_addr learned my addr 172.21.15.47:0/3669710424 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:03.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.972+0000 7fe1dbfff640 1 --2- 172.21.15.47:0/3669710424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1dc0b2390 0x7fe1dc0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.972+0000 7fe1daffd640 1 -- 172.21.15.47:0/3669710424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1dc0b2390 msgr2=0x7fe1dc0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:03.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.972+0000 7fe1daffd640 1 --2- 172.21.15.47:0/3669710424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1dc0b2390 0x7fe1dc0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.972+0000 7fe1daffd640 1 -- 172.21.15.47:0/3669710424 >> v1:172.21.15.47:6789/0 conn(0x7fe1dc0afe10 legacy=0x7fe1dc069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.972+0000 7fe1da7fc640 1 -- 172.21.15.47:0/3669710424 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123799768 0 0) 0x7fe1dc0cf320 con 0x7fe1dc0afe10 2024-03-23T09:42:03.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.972+0000 7fe1daffd640 1 -- 172.21.15.47:0/3669710424 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0c9c30 con 0x7fe1dc06a3f0 2024-03-23T09:42:03.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.973+0000 7fe1daffd640 1 --2- 172.21.15.47:0/3669710424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc06a3f0 0x7fe1dc0ca430 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00097a0 tx=0x7fe1d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f95bd95e164998 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:03.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.973+0000 7fe1da7fc640 1 -- 172.21.15.47:0/3669710424 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0031830 con 0x7fe1dc06a3f0 2024-03-23T09:42:03.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.973+0000 7fe1da7fc640 1 -- 172.21.15.47:0/3669710424 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d0031990 con 0x7fe1dc06a3f0 2024-03-23T09:42:03.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.973+0000 7fe1da7fc640 1 -- 172.21.15.47:0/3669710424 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0031d20 con 0x7fe1dc06a3f0 2024-03-23T09:42:03.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.973+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3669710424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc06a3f0 msgr2=0x7fe1dc0ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.973+0000 7fe1e1b00640 1 --2- 172.21.15.47:0/3669710424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc06a3f0 0x7fe1dc0ca430 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00097a0 tx=0x7fe1d002b8d0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.973+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3669710424 shutdown_connections 2024-03-23T09:42:03.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.973+0000 7fe1e1b00640 1 --2- 172.21.15.47:0/3669710424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1dc0b2390 0x7fe1dc0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.973+0000 7fe1e1b00640 1 --2- 172.21.15.47:0/3669710424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc06a3f0 0x7fe1dc0ca430 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.973+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3669710424 >> 172.21.15.47:0/3669710424 conn(0x7fe1dc0cf540 msgr2=0x7fe1dc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:03.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.973+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3669710424 shutdown_connections 2024-03-23T09:42:03.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.973+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3669710424 wait complete. 2024-03-23T09:42:03.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.974+0000 7fe1e1b00640 1 Processor -- start 2024-03-23T09:42:03.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.975+0000 7fe1e1b00640 1 -- start start 2024-03-23T09:42:03.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.975+0000 7fe1e1b00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1dc06a3f0 0x7fe1dc0ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.975+0000 7fe1e1b00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1dc0afe10 0x7fe1dc155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.975+0000 7fe1e1b00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc0b2390 0x7fe1dc06e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.975+0000 7fe1e1b00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0ac810 con 0x7fe1dc0b2390 2024-03-23T09:42:03.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.975+0000 7fe1e1b00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc155c50 con 0x7fe1dc06a3f0 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.975+0000 7fe1e1b00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0aca40 con 0x7fe1dc0afe10 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.975+0000 7fe1dbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc0b2390 0x7fe1dc06e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.975+0000 7fe1dbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc0b2390 0x7fe1dc06e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49226/0 (socket says 172.21.15.47:49226) 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.975+0000 7fe1dbfff640 1 -- 172.21.15.47:0/3662773055 learned_addr learned my addr 172.21.15.47:0/3662773055 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.976+0000 7fe1daffd640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1dc0afe10 0x7fe1dc155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.976+0000 7fe1db7fe640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1dc06a3f0 0x7fe1dc0ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.976+0000 7fe1dbfff640 1 -- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1dc0afe10 msgr2=0x7fe1dc155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.976+0000 7fe1dbfff640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1dc0afe10 0x7fe1dc155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.976+0000 7fe1dbfff640 1 -- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1dc06a3f0 msgr2=0x7fe1dc0ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.976+0000 7fe1dbfff640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1dc06a3f0 0x7fe1dc0ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.976+0000 7fe1dbfff640 1 -- 172.21.15.47:0/3662773055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d0009400 con 0x7fe1dc0b2390 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.976+0000 7fe1dbfff640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc0b2390 0x7fe1dc06e3d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009fd0 tx=0x7fe1d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.976+0000 7fe1c3fff640 1 -- 172.21.15.47:0/3662773055 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4014420 con 0x7fe1dc0b2390 2024-03-23T09:42:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.976+0000 7fe1c3fff640 1 -- 172.21.15.47:0/3662773055 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d4014580 con 0x7fe1dc0b2390 2024-03-23T09:42:03.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.976+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3662773055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc0acd20 con 0x7fe1dc0b2390 2024-03-23T09:42:03.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.976+0000 7fe1c3fff640 1 -- 172.21.15.47:0/3662773055 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4014930 con 0x7fe1dc0b2390 2024-03-23T09:42:03.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.976+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3662773055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc0ad2e0 con 0x7fe1dc0b2390 2024-03-23T09:42:03.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.977+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3662773055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc0aeee0 con 0x7fe1dc0b2390 2024-03-23T09:42:03.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.977+0000 7fe1c3fff640 1 -- 172.21.15.47:0/3662773055 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe1d4014a90 con 0x7fe1dc0b2390 2024-03-23T09:42:03.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.978+0000 7fe1c3fff640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe1c806ac00 0x7fe1c806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:03.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.978+0000 7fe1c3fff640 1 -- 172.21.15.47:0/3662773055 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1d4089c80 con 0x7fe1dc0b2390 2024-03-23T09:42:03.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.978+0000 7fe1db7fe640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe1c806ac00 0x7fe1c806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:03.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.979+0000 7fe1db7fe640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe1c806ac00 0x7fe1c806d0c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc007fd0 tx=0x7fe1cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:03.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:03.982+0000 7fe1c3fff640 1 -- 172.21.15.47:0/3662773055 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe1d4054e60 con 0x7fe1dc0b2390 2024-03-23T09:42:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.157+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3662773055 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1dc066be0 con 0x7fe1c806ac00 2024-03-23T09:42:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.164+0000 7fe1c3fff640 1 -- 172.21.15.47:0/3662773055 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256110 (secure 0 0 0) 0x7fe1dc066be0 con 0x7fe1c806ac00 2024-03-23T09:42:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:04.167 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:42:04.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.167+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe1c806ac00 msgr2=0x7fe1c806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:04.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.167+0000 7fe1e1b00640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe1c806ac00 0x7fe1c806d0c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc007fd0 tx=0x7fe1cc009040 comp rx=0 tx=0).stop 2024-03-23T09:42:04.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.167+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3662773055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc0b2390 msgr2=0x7fe1dc06e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:04.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.167+0000 7fe1e1b00640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc0b2390 0x7fe1dc06e3d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009fd0 tx=0x7fe1d400e4b0 comp rx=0 tx=0).stop 2024-03-23T09:42:04.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.167+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3662773055 shutdown_connections 2024-03-23T09:42:04.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.167+0000 7fe1e1b00640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe1c806ac00 0x7fe1c806d0c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:04.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.167+0000 7fe1e1b00640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc0b2390 0x7fe1dc06e3d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:04.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.167+0000 7fe1e1b00640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1dc0afe10 0x7fe1dc155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:04.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.167+0000 7fe1e1b00640 1 --2- 172.21.15.47:0/3662773055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1dc06a3f0 0x7fe1dc0ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:04.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.167+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3662773055 >> 172.21.15.47:0/3662773055 conn(0x7fe1dc0cf540 msgr2=0x7fe1dc0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:04.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.167+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3662773055 shutdown_connections 2024-03-23T09:42:04.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:04.167+0000 7fe1e1b00640 1 -- 172.21.15.47:0/3662773055 wait complete. 2024-03-23T09:42:04.177 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-03-23T09:42:02.351943+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236416,"kb_used_data":3776,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738304,"statfs":{"total":773094113280,"available":772852023296,"internally_reserved":0,"allocated":3866624,"data_stored":2181546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.002137"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:59.349007+0000","last_change":"2024-03-23T09:41:59.349007+0000","last_active":"2024-03-23T09:41:59.349007+0000","last_peered":"2024-03-23T09:41:59.349007+0000","last_clean":"2024-03-23T09:41:59.349007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:59.349007+0000","last_undegraded":"2024-03-23T09:41:59.349007+0000","last_fullsized":"2024-03-23T09:41:59.349007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:41:58.347492+0000","last_change":"2024-03-23T09:41:58.347492+0000","last_active":"2024-03-23T09:41:58.347492+0000","last_peered":"2024-03-23T09:41:58.347492+0000","last_clean":"2024-03-23T09:41:58.347492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:41:58.347492+0000","last_undegraded":"2024-03-23T09:41:58.347492+0000","last_fullsized":"2024-03-23T09:41:58.347492+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-23T09:41:53.388976+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:41:53.388976+0000","last_peered":"2024-03-23T09:41:53.388976+0000","last_clean":"2024-03-23T09:41:53.388976+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:41:53.388976+0000","last_undegraded":"2024-03-23T09:41:53.388976+0000","last_fullsized":"2024-03-23T09:41:53.388976+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":929792,"data_stored":715624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27352,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":331776,"data_stored":124637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27356,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31456,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.38},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.26000000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.29799999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.42199999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.49199999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.63900000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.71899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31456,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31452,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":335872,"data_stored":125256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.27100000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.318},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.34899999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":0.33600000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.502}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73199999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.77100000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.58299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":925696,"data_stored":715005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:42:04.204 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:42:04.204+0000 7f829e497640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:42:07.015 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:42:07.018 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:42:07.181 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:42:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.293+0000 7feccb47d640 1 Processor -- start 2024-03-23T09:42:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.293+0000 7feccb47d640 1 -- start start 2024-03-23T09:42:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.294+0000 7feccb47d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fecc415ca90 0x7fecc4160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.294+0000 7feccb47d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc40b8350 0x7fecc40cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.294+0000 7feccb47d640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc40d11e0 con 0x7fecc40cdc40 2024-03-23T09:42:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.294+0000 7feccb47d640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc40ce0d0 con 0x7fecc415ca90 2024-03-23T09:42:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.294+0000 7feccb47d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc40d2680 con 0x7fecc40b8350 2024-03-23T09:42:07.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.294+0000 7fecc8a18640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc40b8350 0x7fecc40cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:07.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.294+0000 7fecc9219640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fecc415ca90 0x7fecc4160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:07.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.294+0000 7fecc9a1a640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fecc40cdc40 0x7fecc40b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:41398/0 (socket says 172.21.15.47:41398) 2024-03-23T09:42:07.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.294+0000 7fecc9a1a640 1 -- 172.21.15.47:0/2440146477 learned_addr learned my addr 172.21.15.47:0/2440146477 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:07.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.295+0000 7fecbbfff640 1 -- 172.21.15.47:0/2440146477 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510126971 0 0) 0x7fecc40d11e0 con 0x7fecc40cdc40 2024-03-23T09:42:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.295+0000 7fecbbfff640 1 -- 172.21.15.47:0/2440146477 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb0003520 con 0x7fecc40cdc40 2024-03-23T09:42:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.295+0000 7fecc8a18640 1 -- 172.21.15.47:0/2440146477 >> v2:172.21.15.175:3300/0 conn(0x7fecc415ca90 msgr2=0x7fecc4160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.295+0000 7fecc8a18640 1 --2- 172.21.15.47:0/2440146477 >> v2:172.21.15.175:3300/0 conn(0x7fecc415ca90 0x7fecc4160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.295+0000 7fecc8a18640 1 -- 172.21.15.47:0/2440146477 >> v1:172.21.15.175:6789/0 conn(0x7fecc40cdc40 legacy=0x7fecc40b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.295+0000 7fecc8a18640 1 -- 172.21.15.47:0/2440146477 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc4160f60 con 0x7fecc40b8350 2024-03-23T09:42:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.295+0000 7fecc8a18640 1 --2- 172.21.15.47:0/2440146477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc40b8350 0x7fecc40cd700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fecb40097a0 tx=0x7fecb402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f851e435af776a76 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.296+0000 7fecbbfff640 1 -- 172.21.15.47:0/2440146477 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb402e070 con 0x7fecc40b8350 2024-03-23T09:42:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.296+0000 7fecbbfff640 1 -- 172.21.15.47:0/2440146477 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb40315a0 con 0x7fecc40b8350 2024-03-23T09:42:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.296+0000 7fecbbfff640 1 -- 172.21.15.47:0/2440146477 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4031930 con 0x7fecc40b8350 2024-03-23T09:42:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.296+0000 7feccb47d640 1 -- 172.21.15.47:0/2440146477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc40b8350 msgr2=0x7fecc40cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.296+0000 7feccb47d640 1 --2- 172.21.15.47:0/2440146477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc40b8350 0x7fecc40cd700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fecb40097a0 tx=0x7fecb402b840 comp rx=0 tx=0).stop 2024-03-23T09:42:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.296+0000 7feccb47d640 1 -- 172.21.15.47:0/2440146477 shutdown_connections 2024-03-23T09:42:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.296+0000 7feccb47d640 1 --2- 172.21.15.47:0/2440146477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc40b8350 0x7fecc40cd700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.296+0000 7feccb47d640 1 --2- 172.21.15.47:0/2440146477 >> v2:172.21.15.175:3300/0 conn(0x7fecc415ca90 0x7fecc4160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.296+0000 7feccb47d640 1 -- 172.21.15.47:0/2440146477 >> 172.21.15.47:0/2440146477 conn(0x7fecc40659d0 msgr2=0x7fecc4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.296+0000 7feccb47d640 1 -- 172.21.15.47:0/2440146477 shutdown_connections 2024-03-23T09:42:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.296+0000 7feccb47d640 1 -- 172.21.15.47:0/2440146477 wait complete. 2024-03-23T09:42:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.297+0000 7feccb47d640 1 Processor -- start 2024-03-23T09:42:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.297+0000 7feccb47d640 1 -- start start 2024-03-23T09:42:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.298+0000 7feccb47d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecc40b8350 0x7fecc414d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.298+0000 7feccb47d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc40cdc40 0x7fecc40ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.298+0000 7feccb47d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc415ca90 0x7fecc415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.298+0000 7feccb47d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc40caef0 con 0x7fecc40b8350 2024-03-23T09:42:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.298+0000 7feccb47d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc40cb1d0 con 0x7fecc40cdc40 2024-03-23T09:42:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.298+0000 7feccb47d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc40cb4b0 con 0x7fecc415ca90 2024-03-23T09:42:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.298+0000 7fecc9219640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecc40b8350 0x7fecc414d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.298+0000 7fecc9219640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecc40b8350 0x7fecc414d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47570/0 (socket says 172.21.15.47:47570) 2024-03-23T09:42:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.298+0000 7fecc9219640 1 -- 172.21.15.47:0/3628912950 learned_addr learned my addr 172.21.15.47:0/3628912950 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.298+0000 7fecc9a1a640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc415ca90 0x7fecc415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.299+0000 7fecc8a18640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc40cdc40 0x7fecc40ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.299+0000 7fecc9219640 1 -- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc415ca90 msgr2=0x7fecc415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.299+0000 7fecc9219640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc415ca90 0x7fecc415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.299+0000 7fecc9219640 1 -- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc40cdc40 msgr2=0x7fecc40ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.299+0000 7fecc9219640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc40cdc40 0x7fecc40ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.299+0000 7fecc9219640 1 -- 172.21.15.47:0/3628912950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb4009400 con 0x7fecc40b8350 2024-03-23T09:42:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.299+0000 7fecc9219640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecc40b8350 0x7fecc414d670 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fecc000e9b0 tx=0x7fecc000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.299+0000 7fecb9ffb640 1 -- 172.21.15.47:0/3628912950 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0009670 con 0x7fecc40b8350 2024-03-23T09:42:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.299+0000 7fecb9ffb640 1 -- 172.21.15.47:0/3628912950 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc000c9a0 con 0x7fecc40b8350 2024-03-23T09:42:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.299+0000 7fecb9ffb640 1 -- 172.21.15.47:0/3628912950 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc000cd50 con 0x7fecc40b8350 2024-03-23T09:42:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.299+0000 7feccb47d640 1 -- 172.21.15.47:0/3628912950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc40cb790 con 0x7fecc40b8350 2024-03-23T09:42:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.299+0000 7feccb47d640 1 -- 172.21.15.47:0/3628912950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc414dbb0 con 0x7fecc40b8350 2024-03-23T09:42:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.300+0000 7feccb47d640 1 -- 172.21.15.47:0/3628912950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc4072360 con 0x7fecc40b8350 2024-03-23T09:42:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.301+0000 7fecb9ffb640 1 -- 172.21.15.47:0/3628912950 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fecc0013470 con 0x7fecc40b8350 2024-03-23T09:42:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.301+0000 7fecb9ffb640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feca806ac30 0x7feca806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.301+0000 7fecb9ffb640 1 -- 172.21.15.47:0/3628912950 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fecc0089630 con 0x7fecc40b8350 2024-03-23T09:42:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.301+0000 7fecc8a18640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feca806ac30 0x7feca806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.302+0000 7fecc8a18640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feca806ac30 0x7feca806d0f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fecb4036000 tx=0x7fecb403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.305+0000 7fecb9ffb640 1 -- 172.21.15.47:0/3628912950 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fecc0054780 con 0x7fecc40b8350 2024-03-23T09:42:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.484+0000 7feccb47d640 1 -- 172.21.15.47:0/3628912950 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fecc40cb340 con 0x7feca806ac30 2024-03-23T09:42:07.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.493+0000 7fecb9ffb640 1 -- 172.21.15.47:0/3628912950 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289310 (secure 0 0 0) 0x7fecc40cb340 con 0x7feca806ac30 2024-03-23T09:42:07.496 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:07.497 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:42:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.495+0000 7feccb47d640 1 -- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feca806ac30 msgr2=0x7feca806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.495+0000 7feccb47d640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feca806ac30 0x7feca806d0f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fecb4036000 tx=0x7fecb403d040 comp rx=0 tx=0).stop 2024-03-23T09:42:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.495+0000 7feccb47d640 1 -- 172.21.15.47:0/3628912950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecc40b8350 msgr2=0x7fecc414d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.495+0000 7feccb47d640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecc40b8350 0x7fecc414d670 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fecc000e9b0 tx=0x7fecc000ee70 comp rx=0 tx=0).stop 2024-03-23T09:42:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.495+0000 7feccb47d640 1 -- 172.21.15.47:0/3628912950 shutdown_connections 2024-03-23T09:42:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.495+0000 7feccb47d640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feca806ac30 0x7feca806d0f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.495+0000 7feccb47d640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc415ca90 0x7fecc415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.495+0000 7feccb47d640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc40cdc40 0x7fecc40ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.495+0000 7feccb47d640 1 --2- 172.21.15.47:0/3628912950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecc40b8350 0x7fecc414d670 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.495+0000 7feccb47d640 1 -- 172.21.15.47:0/3628912950 >> 172.21.15.47:0/3628912950 conn(0x7fecc40659d0 msgr2=0x7fecc40cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.495+0000 7feccb47d640 1 -- 172.21.15.47:0/3628912950 shutdown_connections 2024-03-23T09:42:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.495+0000 7feccb47d640 1 -- 172.21.15.47:0/3628912950 wait complete. 2024-03-23T09:42:07.507 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-03-23T09:42:06.353118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236752,"kb_used_data":4164,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737968,"statfs":{"total":773094113280,"available":772851679232,"internally_reserved":0,"allocated":4263936,"data_stored":2356042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.995702"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.746577+0000","last_change":"2024-03-23T09:41:59.743860+0000","last_active":"2024-03-23T09:41:59.746577+0000","last_peered":"2024-03-23T09:41:59.746577+0000","last_clean":"2024-03-23T09:41:59.746577+0000","last_became_active":"2024-03-23T09:41:59.743646+0000","last_became_peered":"2024-03-23T09:41:59.743646+0000","last_unstale":"2024-03-23T09:41:59.746577+0000","last_undegraded":"2024-03-23T09:41:59.746577+0000","last_fullsized":"2024-03-23T09:41:59.746577+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:19:37.563727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727818+0000","last_change":"2024-03-23T09:41:59.733843+0000","last_active":"2024-03-23T09:42:00.727818+0000","last_peered":"2024-03-23T09:42:00.727818+0000","last_clean":"2024-03-23T09:42:00.727818+0000","last_became_active":"2024-03-23T09:41:59.733486+0000","last_became_peered":"2024-03-23T09:41:59.733486+0000","last_unstale":"2024-03-23T09:42:00.727818+0000","last_undegraded":"2024-03-23T09:42:00.727818+0000","last_fullsized":"2024-03-23T09:42:00.727818+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:01:08.640291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.729087+0000","last_change":"2024-03-23T09:41:59.747780+0000","last_active":"2024-03-23T09:42:00.729087+0000","last_peered":"2024-03-23T09:42:00.729087+0000","last_clean":"2024-03-23T09:42:00.729087+0000","last_became_active":"2024-03-23T09:41:59.747556+0000","last_became_peered":"2024-03-23T09:41:59.747556+0000","last_unstale":"2024-03-23T09:42:00.729087+0000","last_undegraded":"2024-03-23T09:42:00.729087+0000","last_fullsized":"2024-03-23T09:42:00.729087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:12:49.139347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.741495+0000","last_change":"2024-03-23T09:41:59.740219+0000","last_active":"2024-03-23T09:41:59.741495+0000","last_peered":"2024-03-23T09:41:59.741495+0000","last_clean":"2024-03-23T09:41:59.741495+0000","last_became_active":"2024-03-23T09:41:59.739989+0000","last_became_peered":"2024-03-23T09:41:59.739989+0000","last_unstale":"2024-03-23T09:41:59.741495+0000","last_undegraded":"2024-03-23T09:41:59.741495+0000","last_fullsized":"2024-03-23T09:41:59.741495+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:49:53.274131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.735046+0000","last_change":"2024-03-23T09:41:59.733377+0000","last_active":"2024-03-23T09:41:59.735046+0000","last_peered":"2024-03-23T09:41:59.735046+0000","last_clean":"2024-03-23T09:41:59.735046+0000","last_became_active":"2024-03-23T09:41:59.733142+0000","last_became_peered":"2024-03-23T09:41:59.733142+0000","last_unstale":"2024-03-23T09:41:59.735046+0000","last_undegraded":"2024-03-23T09:41:59.735046+0000","last_fullsized":"2024-03-23T09:41:59.735046+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:46:00.051457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727513+0000","last_change":"2024-03-23T09:41:59.742470+0000","last_active":"2024-03-23T09:42:00.727513+0000","last_peered":"2024-03-23T09:42:00.727513+0000","last_clean":"2024-03-23T09:42:00.727513+0000","last_became_active":"2024-03-23T09:41:59.742215+0000","last_became_peered":"2024-03-23T09:41:59.742215+0000","last_unstale":"2024-03-23T09:42:00.727513+0000","last_undegraded":"2024-03-23T09:42:00.727513+0000","last_fullsized":"2024-03-23T09:42:00.727513+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:09:27.903889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728516+0000","last_change":"2024-03-23T09:41:59.743750+0000","last_active":"2024-03-23T09:42:00.728516+0000","last_peered":"2024-03-23T09:42:00.728516+0000","last_clean":"2024-03-23T09:42:00.728516+0000","last_became_active":"2024-03-23T09:41:59.743041+0000","last_became_peered":"2024-03-23T09:41:59.743041+0000","last_unstale":"2024-03-23T09:42:00.728516+0000","last_undegraded":"2024-03-23T09:42:00.728516+0000","last_fullsized":"2024-03-23T09:42:00.728516+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:35.195208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728329+0000","last_change":"2024-03-23T09:41:59.746635+0000","last_active":"2024-03-23T09:42:00.728329+0000","last_peered":"2024-03-23T09:42:00.728329+0000","last_clean":"2024-03-23T09:42:00.728329+0000","last_became_active":"2024-03-23T09:41:59.745716+0000","last_became_peered":"2024-03-23T09:41:59.745716+0000","last_unstale":"2024-03-23T09:42:00.728329+0000","last_undegraded":"2024-03-23T09:42:00.728329+0000","last_fullsized":"2024-03-23T09:42:00.728329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:35:35.710024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.742417+0000","last_change":"2024-03-23T09:41:59.733435+0000","last_active":"2024-03-23T09:41:59.742417+0000","last_peered":"2024-03-23T09:41:59.742417+0000","last_clean":"2024-03-23T09:41:59.742417+0000","last_became_active":"2024-03-23T09:41:59.733172+0000","last_became_peered":"2024-03-23T09:41:59.733172+0000","last_unstale":"2024-03-23T09:41:59.742417+0000","last_undegraded":"2024-03-23T09:41:59.742417+0000","last_fullsized":"2024-03-23T09:41:59.742417+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:55:43.638268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.738574+0000","last_change":"2024-03-23T09:41:59.735454+0000","last_active":"2024-03-23T09:41:59.738574+0000","last_peered":"2024-03-23T09:41:59.738574+0000","last_clean":"2024-03-23T09:41:59.738574+0000","last_became_active":"2024-03-23T09:41:59.735192+0000","last_became_peered":"2024-03-23T09:41:59.735192+0000","last_unstale":"2024-03-23T09:41:59.738574+0000","last_undegraded":"2024-03-23T09:41:59.738574+0000","last_fullsized":"2024-03-23T09:41:59.738574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:41:33.130472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728715+0000","last_change":"2024-03-23T09:41:59.744362+0000","last_active":"2024-03-23T09:42:00.728715+0000","last_peered":"2024-03-23T09:42:00.728715+0000","last_clean":"2024-03-23T09:42:00.728715+0000","last_became_active":"2024-03-23T09:41:59.744082+0000","last_became_peered":"2024-03-23T09:41:59.744082+0000","last_unstale":"2024-03-23T09:42:00.728715+0000","last_undegraded":"2024-03-23T09:42:00.728715+0000","last_fullsized":"2024-03-23T09:42:00.728715+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:22:01.261490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728977+0000","last_change":"2024-03-23T09:41:59.740710+0000","last_active":"2024-03-23T09:42:00.728977+0000","last_peered":"2024-03-23T09:42:00.728977+0000","last_clean":"2024-03-23T09:42:00.728977+0000","last_became_active":"2024-03-23T09:41:59.740431+0000","last_became_peered":"2024-03-23T09:41:59.740431+0000","last_unstale":"2024-03-23T09:42:00.728977+0000","last_undegraded":"2024-03-23T09:42:00.728977+0000","last_fullsized":"2024-03-23T09:42:00.728977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:42:26.098153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.731403+0000","last_change":"2024-03-23T09:41:59.730577+0000","last_active":"2024-03-23T09:41:59.731403+0000","last_peered":"2024-03-23T09:41:59.731403+0000","last_clean":"2024-03-23T09:41:59.731403+0000","last_became_active":"2024-03-23T09:41:59.730300+0000","last_became_peered":"2024-03-23T09:41:59.730300+0000","last_unstale":"2024-03-23T09:41:59.731403+0000","last_undegraded":"2024-03-23T09:41:59.731403+0000","last_fullsized":"2024-03-23T09:41:59.731403+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:15:10.586620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727567+0000","last_change":"2024-03-23T09:41:59.735816+0000","last_active":"2024-03-23T09:42:00.727567+0000","last_peered":"2024-03-23T09:42:00.727567+0000","last_clean":"2024-03-23T09:42:00.727567+0000","last_became_active":"2024-03-23T09:41:59.735450+0000","last_became_peered":"2024-03-23T09:41:59.735450+0000","last_unstale":"2024-03-23T09:42:00.727567+0000","last_undegraded":"2024-03-23T09:42:00.727567+0000","last_fullsized":"2024-03-23T09:42:00.727567+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:22:01.261490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728856+0000","last_change":"2024-03-23T09:41:59.747238+0000","last_active":"2024-03-23T09:42:00.728856+0000","last_peered":"2024-03-23T09:42:00.728856+0000","last_clean":"2024-03-23T09:42:00.728856+0000","last_became_active":"2024-03-23T09:41:59.746986+0000","last_became_peered":"2024-03-23T09:41:59.746986+0000","last_unstale":"2024-03-23T09:42:00.728856+0000","last_undegraded":"2024-03-23T09:42:00.728856+0000","last_fullsized":"2024-03-23T09:42:00.728856+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:57:51.491099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728962+0000","last_change":"2024-03-23T09:41:59.746803+0000","last_active":"2024-03-23T09:42:00.728962+0000","last_peered":"2024-03-23T09:42:00.728962+0000","last_clean":"2024-03-23T09:42:00.728962+0000","last_became_active":"2024-03-23T09:41:59.746340+0000","last_became_peered":"2024-03-23T09:41:59.746340+0000","last_unstale":"2024-03-23T09:42:00.728962+0000","last_undegraded":"2024-03-23T09:42:00.728962+0000","last_fullsized":"2024-03-23T09:42:00.728962+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:46:37.371761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728574+0000","last_change":"2024-03-23T09:41:59.746267+0000","last_active":"2024-03-23T09:42:00.728574+0000","last_peered":"2024-03-23T09:42:00.728574+0000","last_clean":"2024-03-23T09:42:00.728574+0000","last_became_active":"2024-03-23T09:41:59.744270+0000","last_became_peered":"2024-03-23T09:41:59.744270+0000","last_unstale":"2024-03-23T09:42:00.728574+0000","last_undegraded":"2024-03-23T09:42:00.728574+0000","last_fullsized":"2024-03-23T09:42:00.728574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:16:48.612701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727819+0000","last_change":"2024-03-23T09:41:59.741699+0000","last_active":"2024-03-23T09:42:00.727819+0000","last_peered":"2024-03-23T09:42:00.727819+0000","last_clean":"2024-03-23T09:42:00.727819+0000","last_became_active":"2024-03-23T09:41:59.741442+0000","last_became_peered":"2024-03-23T09:41:59.741442+0000","last_unstale":"2024-03-23T09:42:00.727819+0000","last_undegraded":"2024-03-23T09:42:00.727819+0000","last_fullsized":"2024-03-23T09:42:00.727819+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:43:25.910302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727666+0000","last_change":"2024-03-23T09:41:59.739893+0000","last_active":"2024-03-23T09:42:00.727666+0000","last_peered":"2024-03-23T09:42:00.727666+0000","last_clean":"2024-03-23T09:42:00.727666+0000","last_became_active":"2024-03-23T09:41:59.739649+0000","last_became_peered":"2024-03-23T09:41:59.739649+0000","last_unstale":"2024-03-23T09:42:00.727666+0000","last_undegraded":"2024-03-23T09:42:00.727666+0000","last_fullsized":"2024-03-23T09:42:00.727666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:40:17.281616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.742663+0000","last_change":"2024-03-23T09:41:59.740534+0000","last_active":"2024-03-23T09:41:59.742663+0000","last_peered":"2024-03-23T09:41:59.742663+0000","last_clean":"2024-03-23T09:41:59.742663+0000","last_became_active":"2024-03-23T09:41:59.740269+0000","last_became_peered":"2024-03-23T09:41:59.740269+0000","last_unstale":"2024-03-23T09:41:59.742663+0000","last_undegraded":"2024-03-23T09:41:59.742663+0000","last_fullsized":"2024-03-23T09:41:59.742663+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:55:43.638268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.742806+0000","last_change":"2024-03-23T09:41:59.739101+0000","last_active":"2024-03-23T09:41:59.742806+0000","last_peered":"2024-03-23T09:41:59.742806+0000","last_clean":"2024-03-23T09:41:59.742806+0000","last_became_active":"2024-03-23T09:41:59.737104+0000","last_became_peered":"2024-03-23T09:41:59.737104+0000","last_unstale":"2024-03-23T09:41:59.742806+0000","last_undegraded":"2024-03-23T09:41:59.742806+0000","last_fullsized":"2024-03-23T09:41:59.742806+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:19:37.563727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.742659+0000","last_change":"2024-03-23T09:41:59.736947+0000","last_active":"2024-03-23T09:41:59.742659+0000","last_peered":"2024-03-23T09:41:59.742659+0000","last_clean":"2024-03-23T09:41:59.742659+0000","last_became_active":"2024-03-23T09:41:59.736512+0000","last_became_peered":"2024-03-23T09:41:59.736512+0000","last_unstale":"2024-03-23T09:41:59.742659+0000","last_undegraded":"2024-03-23T09:41:59.742659+0000","last_fullsized":"2024-03-23T09:41:59.742659+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:51:08.525680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.745110+0000","last_change":"2024-03-23T09:41:59.743705+0000","last_active":"2024-03-23T09:41:59.745110+0000","last_peered":"2024-03-23T09:41:59.745110+0000","last_clean":"2024-03-23T09:41:59.745110+0000","last_became_active":"2024-03-23T09:41:59.743472+0000","last_became_peered":"2024-03-23T09:41:59.743472+0000","last_unstale":"2024-03-23T09:41:59.745110+0000","last_undegraded":"2024-03-23T09:41:59.745110+0000","last_fullsized":"2024-03-23T09:41:59.745110+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:46:00.051457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728666+0000","last_change":"2024-03-23T09:41:59.738625+0000","last_active":"2024-03-23T09:42:00.728666+0000","last_peered":"2024-03-23T09:42:00.728666+0000","last_clean":"2024-03-23T09:42:00.728666+0000","last_became_active":"2024-03-23T09:41:59.738379+0000","last_became_peered":"2024-03-23T09:41:59.738379+0000","last_unstale":"2024-03-23T09:42:00.728666+0000","last_undegraded":"2024-03-23T09:42:00.728666+0000","last_fullsized":"2024-03-23T09:42:00.728666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:35:35.710024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.746447+0000","last_change":"2024-03-23T09:41:59.745025+0000","last_active":"2024-03-23T09:41:59.746447+0000","last_peered":"2024-03-23T09:41:59.746447+0000","last_clean":"2024-03-23T09:41:59.746447+0000","last_became_active":"2024-03-23T09:41:59.744765+0000","last_became_peered":"2024-03-23T09:41:59.744765+0000","last_unstale":"2024-03-23T09:41:59.746447+0000","last_undegraded":"2024-03-23T09:41:59.746447+0000","last_fullsized":"2024-03-23T09:41:59.746447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:35:35.710024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727946+0000","last_change":"2024-03-23T09:41:59.747101+0000","last_active":"2024-03-23T09:42:00.727946+0000","last_peered":"2024-03-23T09:42:00.727946+0000","last_clean":"2024-03-23T09:42:00.727946+0000","last_became_active":"2024-03-23T09:41:59.746827+0000","last_became_peered":"2024-03-23T09:41:59.746827+0000","last_unstale":"2024-03-23T09:42:00.727946+0000","last_undegraded":"2024-03-23T09:42:00.727946+0000","last_fullsized":"2024-03-23T09:42:00.727946+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:35.195208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.741178+0000","last_change":"2024-03-23T09:41:59.739666+0000","last_active":"2024-03-23T09:41:59.741178+0000","last_peered":"2024-03-23T09:41:59.741178+0000","last_clean":"2024-03-23T09:41:59.741178+0000","last_became_active":"2024-03-23T09:41:59.739419+0000","last_became_peered":"2024-03-23T09:41:59.739419+0000","last_unstale":"2024-03-23T09:41:59.741178+0000","last_undegraded":"2024-03-23T09:41:59.741178+0000","last_fullsized":"2024-03-23T09:41:59.741178+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:15:10.586620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728894+0000","last_change":"2024-03-23T09:41:59.742537+0000","last_active":"2024-03-23T09:42:00.728894+0000","last_peered":"2024-03-23T09:42:00.728894+0000","last_clean":"2024-03-23T09:42:00.728894+0000","last_became_active":"2024-03-23T09:41:59.742278+0000","last_became_peered":"2024-03-23T09:41:59.742278+0000","last_unstale":"2024-03-23T09:42:00.728894+0000","last_undegraded":"2024-03-23T09:42:00.728894+0000","last_fullsized":"2024-03-23T09:42:00.728894+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:49:53.274131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.745140+0000","last_change":"2024-03-23T09:41:59.741363+0000","last_active":"2024-03-23T09:41:59.745140+0000","last_peered":"2024-03-23T09:41:59.745140+0000","last_clean":"2024-03-23T09:41:59.745140+0000","last_became_active":"2024-03-23T09:41:59.737410+0000","last_became_peered":"2024-03-23T09:41:59.737410+0000","last_unstale":"2024-03-23T09:41:59.745140+0000","last_undegraded":"2024-03-23T09:41:59.745140+0000","last_fullsized":"2024-03-23T09:41:59.745140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:49:53.274131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728441+0000","last_change":"2024-03-23T09:41:59.742296+0000","last_active":"2024-03-23T09:42:00.728441+0000","last_peered":"2024-03-23T09:42:00.728441+0000","last_clean":"2024-03-23T09:42:00.728441+0000","last_became_active":"2024-03-23T09:41:59.741782+0000","last_became_peered":"2024-03-23T09:41:59.741782+0000","last_unstale":"2024-03-23T09:42:00.728441+0000","last_undegraded":"2024-03-23T09:42:00.728441+0000","last_fullsized":"2024-03-23T09:42:00.728441+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:57:51.491099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.738229+0000","last_change":"2024-03-23T09:41:59.736642+0000","last_active":"2024-03-23T09:41:59.738229+0000","last_peered":"2024-03-23T09:41:59.738229+0000","last_clean":"2024-03-23T09:41:59.738229+0000","last_became_active":"2024-03-23T09:41:59.736406+0000","last_became_peered":"2024-03-23T09:41:59.736406+0000","last_unstale":"2024-03-23T09:41:59.738229+0000","last_undegraded":"2024-03-23T09:41:59.738229+0000","last_fullsized":"2024-03-23T09:41:59.738229+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:56:14.122697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728827+0000","last_change":"2024-03-23T09:41:59.746809+0000","last_active":"2024-03-23T09:42:00.728827+0000","last_peered":"2024-03-23T09:42:00.728827+0000","last_clean":"2024-03-23T09:42:00.728827+0000","last_became_active":"2024-03-23T09:41:59.746310+0000","last_became_peered":"2024-03-23T09:41:59.746310+0000","last_unstale":"2024-03-23T09:42:00.728827+0000","last_undegraded":"2024-03-23T09:42:00.728827+0000","last_fullsized":"2024-03-23T09:42:00.728827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:41:03.707267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.745146+0000","last_change":"2024-03-23T09:41:59.743141+0000","last_active":"2024-03-23T09:41:59.745146+0000","last_peered":"2024-03-23T09:41:59.745146+0000","last_clean":"2024-03-23T09:41:59.745146+0000","last_became_active":"2024-03-23T09:41:59.742893+0000","last_became_peered":"2024-03-23T09:41:59.742893+0000","last_unstale":"2024-03-23T09:41:59.745146+0000","last_undegraded":"2024-03-23T09:41:59.745146+0000","last_fullsized":"2024-03-23T09:41:59.745146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:19:37.563727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.749385+0000","last_change":"2024-03-23T09:41:59.375945+0000","last_active":"2024-03-23T09:41:59.749385+0000","last_peered":"2024-03-23T09:41:59.749385+0000","last_clean":"2024-03-23T09:41:59.749385+0000","last_became_active":"2024-03-23T09:41:59.375641+0000","last_became_peered":"2024-03-23T09:41:59.375641+0000","last_unstale":"2024-03-23T09:41:59.749385+0000","last_undegraded":"2024-03-23T09:41:59.749385+0000","last_fullsized":"2024-03-23T09:41:59.749385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:37.112055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.746598+0000","last_change":"2024-03-23T09:41:59.745059+0000","last_active":"2024-03-23T09:41:59.746598+0000","last_peered":"2024-03-23T09:41:59.746598+0000","last_clean":"2024-03-23T09:41:59.746598+0000","last_became_active":"2024-03-23T09:41:59.744811+0000","last_became_peered":"2024-03-23T09:41:59.744811+0000","last_unstale":"2024-03-23T09:41:59.746598+0000","last_undegraded":"2024-03-23T09:41:59.746598+0000","last_fullsized":"2024-03-23T09:41:59.746598+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:46:00.051457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.748739+0000","last_change":"2024-03-23T09:41:59.368083+0000","last_active":"2024-03-23T09:41:59.748739+0000","last_peered":"2024-03-23T09:41:59.748739+0000","last_clean":"2024-03-23T09:41:59.748739+0000","last_became_active":"2024-03-23T09:41:59.367776+0000","last_became_peered":"2024-03-23T09:41:59.367776+0000","last_unstale":"2024-03-23T09:41:59.748739+0000","last_undegraded":"2024-03-23T09:41:59.748739+0000","last_fullsized":"2024-03-23T09:41:59.748739+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:51:40.506737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.743223+0000","last_change":"2024-03-23T09:41:59.736113+0000","last_active":"2024-03-23T09:41:59.743223+0000","last_peered":"2024-03-23T09:41:59.743223+0000","last_clean":"2024-03-23T09:41:59.743223+0000","last_became_active":"2024-03-23T09:41:59.735074+0000","last_became_peered":"2024-03-23T09:41:59.735074+0000","last_unstale":"2024-03-23T09:41:59.743223+0000","last_undegraded":"2024-03-23T09:41:59.743223+0000","last_fullsized":"2024-03-23T09:41:59.743223+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:30:43.061838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727758+0000","last_change":"2024-03-23T09:41:59.370175+0000","last_active":"2024-03-23T09:42:00.727758+0000","last_peered":"2024-03-23T09:42:00.727758+0000","last_clean":"2024-03-23T09:42:00.727758+0000","last_became_active":"2024-03-23T09:41:59.369893+0000","last_became_peered":"2024-03-23T09:41:59.369893+0000","last_unstale":"2024-03-23T09:42:00.727758+0000","last_undegraded":"2024-03-23T09:42:00.727758+0000","last_fullsized":"2024-03-23T09:42:00.727758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:56:13.121182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.737882+0000","last_change":"2024-03-23T09:41:59.733472+0000","last_active":"2024-03-23T09:41:59.737882+0000","last_peered":"2024-03-23T09:41:59.737882+0000","last_clean":"2024-03-23T09:41:59.737882+0000","last_became_active":"2024-03-23T09:41:59.733233+0000","last_became_peered":"2024-03-23T09:41:59.733233+0000","last_unstale":"2024-03-23T09:41:59.737882+0000","last_undegraded":"2024-03-23T09:41:59.737882+0000","last_fullsized":"2024-03-23T09:41:59.737882+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:51:08.525680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.738969+0000","last_change":"2024-03-23T09:41:59.366477+0000","last_active":"2024-03-23T09:41:59.738969+0000","last_peered":"2024-03-23T09:41:59.738969+0000","last_clean":"2024-03-23T09:41:59.738969+0000","last_became_active":"2024-03-23T09:41:59.366200+0000","last_became_peered":"2024-03-23T09:41:59.366200+0000","last_unstale":"2024-03-23T09:41:59.738969+0000","last_undegraded":"2024-03-23T09:41:59.738969+0000","last_fullsized":"2024-03-23T09:41:59.738969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:29:46.827710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728527+0000","last_change":"2024-03-23T09:41:59.742020+0000","last_active":"2024-03-23T09:42:00.728527+0000","last_peered":"2024-03-23T09:42:00.728527+0000","last_clean":"2024-03-23T09:42:00.728527+0000","last_became_active":"2024-03-23T09:41:59.741747+0000","last_became_peered":"2024-03-23T09:41:59.741747+0000","last_unstale":"2024-03-23T09:42:00.728527+0000","last_undegraded":"2024-03-23T09:42:00.728527+0000","last_fullsized":"2024-03-23T09:42:00.728527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:42:26.098153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.747875+0000","last_change":"2024-03-23T09:41:59.373936+0000","last_active":"2024-03-23T09:41:59.747875+0000","last_peered":"2024-03-23T09:41:59.747875+0000","last_clean":"2024-03-23T09:41:59.747875+0000","last_became_active":"2024-03-23T09:41:59.372813+0000","last_became_peered":"2024-03-23T09:41:59.372813+0000","last_unstale":"2024-03-23T09:41:59.747875+0000","last_undegraded":"2024-03-23T09:41:59.747875+0000","last_fullsized":"2024-03-23T09:41:59.747875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:13:43.779572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728979+0000","last_change":"2024-03-23T09:41:59.739577+0000","last_active":"2024-03-23T09:42:00.728979+0000","last_peered":"2024-03-23T09:42:00.728979+0000","last_clean":"2024-03-23T09:42:00.728979+0000","last_became_active":"2024-03-23T09:41:59.739314+0000","last_became_peered":"2024-03-23T09:41:59.739314+0000","last_unstale":"2024-03-23T09:42:00.728979+0000","last_undegraded":"2024-03-23T09:42:00.728979+0000","last_fullsized":"2024-03-23T09:42:00.728979+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:55:43.638268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.747833+0000","last_change":"2024-03-23T09:41:59.368755+0000","last_active":"2024-03-23T09:41:59.747833+0000","last_peered":"2024-03-23T09:41:59.747833+0000","last_clean":"2024-03-23T09:41:59.747833+0000","last_became_active":"2024-03-23T09:41:59.368393+0000","last_became_peered":"2024-03-23T09:41:59.368393+0000","last_unstale":"2024-03-23T09:41:59.747833+0000","last_undegraded":"2024-03-23T09:41:59.747833+0000","last_fullsized":"2024-03-23T09:41:59.747833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:16:20.862020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.742290+0000","last_change":"2024-03-23T09:41:59.735586+0000","last_active":"2024-03-23T09:41:59.742290+0000","last_peered":"2024-03-23T09:41:59.742290+0000","last_clean":"2024-03-23T09:41:59.742290+0000","last_became_active":"2024-03-23T09:41:59.734975+0000","last_became_peered":"2024-03-23T09:41:59.734975+0000","last_unstale":"2024-03-23T09:41:59.742290+0000","last_undegraded":"2024-03-23T09:41:59.742290+0000","last_fullsized":"2024-03-23T09:41:59.742290+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:30:25.030569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727372+0000","last_change":"2024-03-23T09:41:59.365754+0000","last_active":"2024-03-23T09:42:00.727372+0000","last_peered":"2024-03-23T09:42:00.727372+0000","last_clean":"2024-03-23T09:42:00.727372+0000","last_became_active":"2024-03-23T09:41:59.365509+0000","last_became_peered":"2024-03-23T09:41:59.365509+0000","last_unstale":"2024-03-23T09:42:00.727372+0000","last_undegraded":"2024-03-23T09:42:00.727372+0000","last_fullsized":"2024-03-23T09:42:00.727372+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:55:42.636754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.747548+0000","last_change":"2024-03-23T09:41:59.744526+0000","last_active":"2024-03-23T09:41:59.747548+0000","last_peered":"2024-03-23T09:41:59.747548+0000","last_clean":"2024-03-23T09:41:59.747548+0000","last_became_active":"2024-03-23T09:41:59.744232+0000","last_became_peered":"2024-03-23T09:41:59.744232+0000","last_unstale":"2024-03-23T09:41:59.747548+0000","last_undegraded":"2024-03-23T09:41:59.747548+0000","last_fullsized":"2024-03-23T09:41:59.747548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:55:43.638268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.729115+0000","last_change":"2024-03-23T09:41:59.371563+0000","last_active":"2024-03-23T09:41:59.729115+0000","last_peered":"2024-03-23T09:41:59.729115+0000","last_clean":"2024-03-23T09:41:59.729115+0000","last_became_active":"2024-03-23T09:41:59.371290+0000","last_became_peered":"2024-03-23T09:41:59.371290+0000","last_unstale":"2024-03-23T09:41:59.729115+0000","last_undegraded":"2024-03-23T09:41:59.729115+0000","last_fullsized":"2024-03-23T09:41:59.729115+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:29:46.827710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728749+0000","last_change":"2024-03-23T09:41:59.741001+0000","last_active":"2024-03-23T09:42:00.728749+0000","last_peered":"2024-03-23T09:42:00.728749+0000","last_clean":"2024-03-23T09:42:00.728749+0000","last_became_active":"2024-03-23T09:41:59.740501+0000","last_became_peered":"2024-03-23T09:41:59.740501+0000","last_unstale":"2024-03-23T09:42:00.728749+0000","last_undegraded":"2024-03-23T09:42:00.728749+0000","last_fullsized":"2024-03-23T09:42:00.728749+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:16:51.596258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727969+0000","last_change":"2024-03-23T09:41:59.363548+0000","last_active":"2024-03-23T09:42:00.727969+0000","last_peered":"2024-03-23T09:42:00.727969+0000","last_clean":"2024-03-23T09:42:00.727969+0000","last_became_active":"2024-03-23T09:41:59.363202+0000","last_became_peered":"2024-03-23T09:41:59.363202+0000","last_unstale":"2024-03-23T09:42:00.727969+0000","last_undegraded":"2024-03-23T09:42:00.727969+0000","last_fullsized":"2024-03-23T09:42:00.727969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:30:24.029054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727887+0000","last_change":"2024-03-23T09:41:59.741662+0000","last_active":"2024-03-23T09:42:00.727887+0000","last_peered":"2024-03-23T09:42:00.727887+0000","last_clean":"2024-03-23T09:42:00.727887+0000","last_became_active":"2024-03-23T09:41:59.741416+0000","last_became_peered":"2024-03-23T09:41:59.741416+0000","last_unstale":"2024-03-23T09:42:00.727887+0000","last_undegraded":"2024-03-23T09:42:00.727887+0000","last_fullsized":"2024-03-23T09:42:00.727887+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:30:03.139750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.726588+0000","last_change":"2024-03-23T09:41:59.366712+0000","last_active":"2024-03-23T09:41:59.726588+0000","last_peered":"2024-03-23T09:41:59.726588+0000","last_clean":"2024-03-23T09:41:59.726588+0000","last_became_active":"2024-03-23T09:41:59.366452+0000","last_became_peered":"2024-03-23T09:41:59.366452+0000","last_unstale":"2024-03-23T09:41:59.726588+0000","last_undegraded":"2024-03-23T09:41:59.726588+0000","last_fullsized":"2024-03-23T09:41:59.726588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:21:06.988838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727818+0000","last_change":"2024-03-23T09:41:59.745360+0000","last_active":"2024-03-23T09:42:00.727818+0000","last_peered":"2024-03-23T09:42:00.727818+0000","last_clean":"2024-03-23T09:42:00.727818+0000","last_became_active":"2024-03-23T09:41:59.745078+0000","last_became_peered":"2024-03-23T09:41:59.745078+0000","last_unstale":"2024-03-23T09:42:00.727818+0000","last_undegraded":"2024-03-23T09:42:00.727818+0000","last_fullsized":"2024-03-23T09:42:00.727818+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:12:49.139347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.729095+0000","last_change":"2024-03-23T09:41:59.373384+0000","last_active":"2024-03-23T09:42:00.729095+0000","last_peered":"2024-03-23T09:42:00.729095+0000","last_clean":"2024-03-23T09:42:00.729095+0000","last_became_active":"2024-03-23T09:41:59.373094+0000","last_became_peered":"2024-03-23T09:41:59.373094+0000","last_unstale":"2024-03-23T09:42:00.729095+0000","last_undegraded":"2024-03-23T09:42:00.729095+0000","last_fullsized":"2024-03-23T09:42:00.729095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:13:03.585656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.729000+0000","last_change":"2024-03-23T09:41:59.748687+0000","last_active":"2024-03-23T09:42:00.729000+0000","last_peered":"2024-03-23T09:42:00.729000+0000","last_clean":"2024-03-23T09:42:00.729000+0000","last_became_active":"2024-03-23T09:41:59.748454+0000","last_became_peered":"2024-03-23T09:41:59.748454+0000","last_unstale":"2024-03-23T09:42:00.729000+0000","last_undegraded":"2024-03-23T09:42:00.729000+0000","last_fullsized":"2024-03-23T09:42:00.729000+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:22:01.261490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728048+0000","last_change":"2024-03-23T09:41:59.362085+0000","last_active":"2024-03-23T09:42:00.728048+0000","last_peered":"2024-03-23T09:42:00.728048+0000","last_clean":"2024-03-23T09:42:00.728048+0000","last_became_active":"2024-03-23T09:41:59.361307+0000","last_became_peered":"2024-03-23T09:41:59.361307+0000","last_unstale":"2024-03-23T09:42:00.728048+0000","last_undegraded":"2024-03-23T09:42:00.728048+0000","last_fullsized":"2024-03-23T09:42:00.728048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:47:59.195529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.746437+0000","last_change":"2024-03-23T09:41:59.744963+0000","last_active":"2024-03-23T09:41:59.746437+0000","last_peered":"2024-03-23T09:41:59.746437+0000","last_clean":"2024-03-23T09:41:59.746437+0000","last_became_active":"2024-03-23T09:41:59.744667+0000","last_became_peered":"2024-03-23T09:41:59.744667+0000","last_unstale":"2024-03-23T09:41:59.746437+0000","last_undegraded":"2024-03-23T09:41:59.746437+0000","last_fullsized":"2024-03-23T09:41:59.746437+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:30:25.030569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.746223+0000","last_change":"2024-03-23T09:41:59.374278+0000","last_active":"2024-03-23T09:41:59.746223+0000","last_peered":"2024-03-23T09:41:59.746223+0000","last_clean":"2024-03-23T09:41:59.746223+0000","last_became_active":"2024-03-23T09:41:59.372011+0000","last_became_peered":"2024-03-23T09:41:59.372011+0000","last_unstale":"2024-03-23T09:41:59.746223+0000","last_undegraded":"2024-03-23T09:41:59.746223+0000","last_fullsized":"2024-03-23T09:41:59.746223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:21:05.726294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727644+0000","last_change":"2024-03-23T09:41:59.733867+0000","last_active":"2024-03-23T09:42:00.727644+0000","last_peered":"2024-03-23T09:42:00.727644+0000","last_clean":"2024-03-23T09:42:00.727644+0000","last_became_active":"2024-03-23T09:41:59.733463+0000","last_became_peered":"2024-03-23T09:41:59.733463+0000","last_unstale":"2024-03-23T09:42:00.727644+0000","last_undegraded":"2024-03-23T09:42:00.727644+0000","last_fullsized":"2024-03-23T09:42:00.727644+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:24:23.675426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.723790+0000","last_change":"2024-03-23T09:41:59.374952+0000","last_active":"2024-03-23T09:41:59.723790+0000","last_peered":"2024-03-23T09:41:59.723790+0000","last_clean":"2024-03-23T09:41:59.723790+0000","last_became_active":"2024-03-23T09:41:59.374666+0000","last_became_peered":"2024-03-23T09:41:59.374666+0000","last_unstale":"2024-03-23T09:41:59.723790+0000","last_undegraded":"2024-03-23T09:41:59.723790+0000","last_fullsized":"2024-03-23T09:41:59.723790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:47:37.811817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728854+0000","last_change":"2024-03-23T09:41:59.746610+0000","last_active":"2024-03-23T09:42:00.728854+0000","last_peered":"2024-03-23T09:42:00.728854+0000","last_clean":"2024-03-23T09:42:00.728854+0000","last_became_active":"2024-03-23T09:41:59.745415+0000","last_became_peered":"2024-03-23T09:41:59.745415+0000","last_unstale":"2024-03-23T09:42:00.728854+0000","last_undegraded":"2024-03-23T09:42:00.728854+0000","last_fullsized":"2024-03-23T09:42:00.728854+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:16:51.596258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727462+0000","last_change":"2024-03-23T09:41:59.363562+0000","last_active":"2024-03-23T09:42:00.727462+0000","last_peered":"2024-03-23T09:42:00.727462+0000","last_clean":"2024-03-23T09:42:00.727462+0000","last_became_active":"2024-03-23T09:41:59.363310+0000","last_became_peered":"2024-03-23T09:41:59.363310+0000","last_unstale":"2024-03-23T09:42:00.727462+0000","last_undegraded":"2024-03-23T09:42:00.727462+0000","last_fullsized":"2024-03-23T09:42:00.727462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:19:36.562213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.746563+0000","last_change":"2024-03-23T09:41:59.744740+0000","last_active":"2024-03-23T09:41:59.746563+0000","last_peered":"2024-03-23T09:41:59.746563+0000","last_clean":"2024-03-23T09:41:59.746563+0000","last_became_active":"2024-03-23T09:41:59.744494+0000","last_became_peered":"2024-03-23T09:41:59.744494+0000","last_unstale":"2024-03-23T09:41:59.746563+0000","last_undegraded":"2024-03-23T09:41:59.746563+0000","last_fullsized":"2024-03-23T09:41:59.746563+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:48:00.197043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727418+0000","last_change":"2024-03-23T09:41:59.361537+0000","last_active":"2024-03-23T09:42:00.727418+0000","last_peered":"2024-03-23T09:42:00.727418+0000","last_clean":"2024-03-23T09:42:00.727418+0000","last_became_active":"2024-03-23T09:41:59.360636+0000","last_became_peered":"2024-03-23T09:41:59.360636+0000","last_unstale":"2024-03-23T09:42:00.727418+0000","last_undegraded":"2024-03-23T09:42:00.727418+0000","last_fullsized":"2024-03-23T09:42:00.727418+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:09:43.485501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.745618+0000","last_change":"2024-03-23T09:41:59.744274+0000","last_active":"2024-03-23T09:41:59.745618+0000","last_peered":"2024-03-23T09:41:59.745618+0000","last_clean":"2024-03-23T09:41:59.745618+0000","last_became_active":"2024-03-23T09:41:59.743990+0000","last_became_peered":"2024-03-23T09:41:59.743990+0000","last_unstale":"2024-03-23T09:41:59.745618+0000","last_undegraded":"2024-03-23T09:41:59.745618+0000","last_fullsized":"2024-03-23T09:41:59.745618+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:46:00.051457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.744940+0000","last_change":"2024-03-23T09:41:59.369383+0000","last_active":"2024-03-23T09:41:59.744940+0000","last_peered":"2024-03-23T09:41:59.744940+0000","last_clean":"2024-03-23T09:41:59.744940+0000","last_became_active":"2024-03-23T09:41:59.369051+0000","last_became_peered":"2024-03-23T09:41:59.369051+0000","last_unstale":"2024-03-23T09:41:59.744940+0000","last_undegraded":"2024-03-23T09:41:59.744940+0000","last_fullsized":"2024-03-23T09:41:59.744940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:21:05.726294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.729140+0000","last_change":"2024-03-23T09:41:59.744221+0000","last_active":"2024-03-23T09:42:00.729140+0000","last_peered":"2024-03-23T09:42:00.729140+0000","last_clean":"2024-03-23T09:42:00.729140+0000","last_became_active":"2024-03-23T09:41:59.743925+0000","last_became_peered":"2024-03-23T09:41:59.743925+0000","last_unstale":"2024-03-23T09:42:00.729140+0000","last_undegraded":"2024-03-23T09:42:00.729140+0000","last_fullsized":"2024-03-23T09:42:00.729140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:16:48.612701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.742723+0000","last_change":"2024-03-23T09:41:59.370840+0000","last_active":"2024-03-23T09:41:59.742723+0000","last_peered":"2024-03-23T09:41:59.742723+0000","last_clean":"2024-03-23T09:41:59.742723+0000","last_became_active":"2024-03-23T09:41:59.370557+0000","last_became_peered":"2024-03-23T09:41:59.370557+0000","last_unstale":"2024-03-23T09:41:59.742723+0000","last_undegraded":"2024-03-23T09:41:59.742723+0000","last_fullsized":"2024-03-23T09:41:59.742723+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:37.112055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.738781+0000","last_change":"2024-03-23T09:41:59.734307+0000","last_active":"2024-03-23T09:41:59.738781+0000","last_peered":"2024-03-23T09:41:59.738781+0000","last_clean":"2024-03-23T09:41:59.738781+0000","last_became_active":"2024-03-23T09:41:59.734002+0000","last_became_peered":"2024-03-23T09:41:59.734002+0000","last_unstale":"2024-03-23T09:41:59.738781+0000","last_undegraded":"2024-03-23T09:41:59.738781+0000","last_fullsized":"2024-03-23T09:41:59.738781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:30:43.061838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.745553+0000","last_change":"2024-03-23T09:41:59.368171+0000","last_active":"2024-03-23T09:41:59.745553+0000","last_peered":"2024-03-23T09:41:59.745553+0000","last_clean":"2024-03-23T09:41:59.745553+0000","last_became_active":"2024-03-23T09:41:59.367927+0000","last_became_peered":"2024-03-23T09:41:59.367927+0000","last_unstale":"2024-03-23T09:41:59.745553+0000","last_undegraded":"2024-03-23T09:41:59.745553+0000","last_fullsized":"2024-03-23T09:41:59.745553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:20:42.051622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.749909+0000","last_change":"2024-03-23T09:41:59.749197+0000","last_active":"2024-03-23T09:41:59.749909+0000","last_peered":"2024-03-23T09:41:59.749909+0000","last_clean":"2024-03-23T09:41:59.749909+0000","last_became_active":"2024-03-23T09:41:59.748911+0000","last_became_peered":"2024-03-23T09:41:59.748911+0000","last_unstale":"2024-03-23T09:41:59.749909+0000","last_undegraded":"2024-03-23T09:41:59.749909+0000","last_fullsized":"2024-03-23T09:41:59.749909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:19:37.563727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.731210+0000","last_change":"2024-03-23T09:41:59.368034+0000","last_active":"2024-03-23T09:41:59.731210+0000","last_peered":"2024-03-23T09:41:59.731210+0000","last_clean":"2024-03-23T09:41:59.731210+0000","last_became_active":"2024-03-23T09:41:59.367724+0000","last_became_peered":"2024-03-23T09:41:59.367724+0000","last_unstale":"2024-03-23T09:41:59.731210+0000","last_undegraded":"2024-03-23T09:41:59.731210+0000","last_fullsized":"2024-03-23T09:41:59.731210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:47:10.996301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.737454+0000","last_change":"2024-03-23T09:41:59.736920+0000","last_active":"2024-03-23T09:41:59.737454+0000","last_peered":"2024-03-23T09:41:59.737454+0000","last_clean":"2024-03-23T09:41:59.737454+0000","last_became_active":"2024-03-23T09:41:59.736668+0000","last_became_peered":"2024-03-23T09:41:59.736668+0000","last_unstale":"2024-03-23T09:41:59.737454+0000","last_undegraded":"2024-03-23T09:41:59.737454+0000","last_fullsized":"2024-03-23T09:41:59.737454+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:49:53.274131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.741715+0000","last_change":"2024-03-23T09:41:59.371499+0000","last_active":"2024-03-23T09:41:59.741715+0000","last_peered":"2024-03-23T09:41:59.741715+0000","last_clean":"2024-03-23T09:41:59.741715+0000","last_became_active":"2024-03-23T09:41:59.371115+0000","last_became_peered":"2024-03-23T09:41:59.371115+0000","last_unstale":"2024-03-23T09:41:59.741715+0000","last_undegraded":"2024-03-23T09:41:59.741715+0000","last_fullsized":"2024-03-23T09:41:59.741715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:06:14.194641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.733453+0000","last_change":"2024-03-23T09:41:59.731196+0000","last_active":"2024-03-23T09:41:59.733453+0000","last_peered":"2024-03-23T09:41:59.733453+0000","last_clean":"2024-03-23T09:41:59.733453+0000","last_became_active":"2024-03-23T09:41:59.730946+0000","last_became_peered":"2024-03-23T09:41:59.730946+0000","last_unstale":"2024-03-23T09:41:59.733453+0000","last_undegraded":"2024-03-23T09:41:59.733453+0000","last_fullsized":"2024-03-23T09:41:59.733453+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:30:43.061838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.735470+0000","last_change":"2024-03-23T09:41:59.373898+0000","last_active":"2024-03-23T09:41:59.735470+0000","last_peered":"2024-03-23T09:41:59.735470+0000","last_clean":"2024-03-23T09:41:59.735470+0000","last_became_active":"2024-03-23T09:41:59.373635+0000","last_became_peered":"2024-03-23T09:41:59.373635+0000","last_unstale":"2024-03-23T09:41:59.735470+0000","last_undegraded":"2024-03-23T09:41:59.735470+0000","last_fullsized":"2024-03-23T09:41:59.735470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:47:10.996301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.731321+0000","last_change":"2024-03-23T09:41:59.730068+0000","last_active":"2024-03-23T09:41:59.731321+0000","last_peered":"2024-03-23T09:41:59.731321+0000","last_clean":"2024-03-23T09:41:59.731321+0000","last_became_active":"2024-03-23T09:41:59.729830+0000","last_became_peered":"2024-03-23T09:41:59.729830+0000","last_unstale":"2024-03-23T09:41:59.731321+0000","last_undegraded":"2024-03-23T09:41:59.731321+0000","last_fullsized":"2024-03-23T09:41:59.731321+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:30:25.030569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.728758+0000","last_change":"2024-03-23T09:41:59.372632+0000","last_active":"2024-03-23T09:41:59.728758+0000","last_peered":"2024-03-23T09:41:59.728758+0000","last_clean":"2024-03-23T09:41:59.728758+0000","last_became_active":"2024-03-23T09:41:59.372351+0000","last_became_peered":"2024-03-23T09:41:59.372351+0000","last_unstale":"2024-03-23T09:41:59.728758+0000","last_undegraded":"2024-03-23T09:41:59.728758+0000","last_fullsized":"2024-03-23T09:41:59.728758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:36:24.683018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728754+0000","last_change":"2024-03-23T09:41:59.743955+0000","last_active":"2024-03-23T09:42:00.728754+0000","last_peered":"2024-03-23T09:42:00.728754+0000","last_clean":"2024-03-23T09:42:00.728754+0000","last_became_active":"2024-03-23T09:41:59.743493+0000","last_became_peered":"2024-03-23T09:41:59.743493+0000","last_unstale":"2024-03-23T09:42:00.728754+0000","last_undegraded":"2024-03-23T09:42:00.728754+0000","last_fullsized":"2024-03-23T09:42:00.728754+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:12:49.139347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728589+0000","last_change":"2024-03-23T09:41:59.369085+0000","last_active":"2024-03-23T09:42:00.728589+0000","last_peered":"2024-03-23T09:42:00.728589+0000","last_clean":"2024-03-23T09:42:00.728589+0000","last_became_active":"2024-03-23T09:41:59.368758+0000","last_became_peered":"2024-03-23T09:41:59.368758+0000","last_unstale":"2024-03-23T09:42:00.728589+0000","last_undegraded":"2024-03-23T09:42:00.728589+0000","last_fullsized":"2024-03-23T09:42:00.728589+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:56:13.121182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.743950+0000","last_change":"2024-03-23T09:41:59.741176+0000","last_active":"2024-03-23T09:41:59.743950+0000","last_peered":"2024-03-23T09:41:59.743950+0000","last_clean":"2024-03-23T09:41:59.743950+0000","last_became_active":"2024-03-23T09:41:59.740915+0000","last_became_peered":"2024-03-23T09:41:59.740915+0000","last_unstale":"2024-03-23T09:41:59.743950+0000","last_undegraded":"2024-03-23T09:41:59.743950+0000","last_fullsized":"2024-03-23T09:41:59.743950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:51:08.525680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.750599+0000","last_change":"2024-03-23T09:41:59.373947+0000","last_active":"2024-03-23T09:41:59.750599+0000","last_peered":"2024-03-23T09:41:59.750599+0000","last_clean":"2024-03-23T09:41:59.750599+0000","last_became_active":"2024-03-23T09:41:59.373629+0000","last_became_peered":"2024-03-23T09:41:59.373629+0000","last_unstale":"2024-03-23T09:41:59.750599+0000","last_undegraded":"2024-03-23T09:41:59.750599+0000","last_fullsized":"2024-03-23T09:41:59.750599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:29:46.827710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.729180+0000","last_change":"2024-03-23T09:41:59.748322+0000","last_active":"2024-03-23T09:42:00.729180+0000","last_peered":"2024-03-23T09:42:00.729180+0000","last_clean":"2024-03-23T09:42:00.729180+0000","last_became_active":"2024-03-23T09:41:59.748078+0000","last_became_peered":"2024-03-23T09:41:59.748078+0000","last_unstale":"2024-03-23T09:42:00.729180+0000","last_undegraded":"2024-03-23T09:42:00.729180+0000","last_fullsized":"2024-03-23T09:42:00.729180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:36:35.195208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.726680+0000","last_change":"2024-03-23T09:41:59.370815+0000","last_active":"2024-03-23T09:41:59.726680+0000","last_peered":"2024-03-23T09:41:59.726680+0000","last_clean":"2024-03-23T09:41:59.726680+0000","last_became_active":"2024-03-23T09:41:59.370531+0000","last_became_peered":"2024-03-23T09:41:59.370531+0000","last_unstale":"2024-03-23T09:41:59.726680+0000","last_undegraded":"2024-03-23T09:41:59.726680+0000","last_fullsized":"2024-03-23T09:41:59.726680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:21:05.726294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.736851+0000","last_change":"2024-03-23T09:41:59.736020+0000","last_active":"2024-03-23T09:41:59.736851+0000","last_peered":"2024-03-23T09:41:59.736851+0000","last_clean":"2024-03-23T09:41:59.736851+0000","last_became_active":"2024-03-23T09:41:59.735737+0000","last_became_peered":"2024-03-23T09:41:59.735737+0000","last_unstale":"2024-03-23T09:41:59.736851+0000","last_undegraded":"2024-03-23T09:41:59.736851+0000","last_fullsized":"2024-03-23T09:41:59.736851+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:55:43.638268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.723923+0000","last_change":"2024-03-23T09:41:59.369210+0000","last_active":"2024-03-23T09:41:59.723923+0000","last_peered":"2024-03-23T09:41:59.723923+0000","last_clean":"2024-03-23T09:41:59.723923+0000","last_became_active":"2024-03-23T09:41:59.368774+0000","last_became_peered":"2024-03-23T09:41:59.368774+0000","last_unstale":"2024-03-23T09:41:59.723923+0000","last_undegraded":"2024-03-23T09:41:59.723923+0000","last_fullsized":"2024-03-23T09:41:59.723923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:06:14.194641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.738728+0000","last_change":"2024-03-23T09:41:59.735961+0000","last_active":"2024-03-23T09:41:59.738728+0000","last_peered":"2024-03-23T09:41:59.738728+0000","last_clean":"2024-03-23T09:41:59.738728+0000","last_became_active":"2024-03-23T09:41:59.735673+0000","last_became_peered":"2024-03-23T09:41:59.735673+0000","last_unstale":"2024-03-23T09:41:59.738728+0000","last_undegraded":"2024-03-23T09:41:59.738728+0000","last_fullsized":"2024-03-23T09:41:59.738728+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:09:44.487015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.726971+0000","last_change":"2024-03-23T09:41:59.365812+0000","last_active":"2024-03-23T09:41:59.726971+0000","last_peered":"2024-03-23T09:41:59.726971+0000","last_clean":"2024-03-23T09:41:59.726971+0000","last_became_active":"2024-03-23T09:41:59.365554+0000","last_became_peered":"2024-03-23T09:41:59.365554+0000","last_unstale":"2024-03-23T09:41:59.726971+0000","last_undegraded":"2024-03-23T09:41:59.726971+0000","last_fullsized":"2024-03-23T09:41:59.726971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:47:37.811817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.738504+0000","last_change":"2024-03-23T09:41:59.738021+0000","last_active":"2024-03-23T09:41:59.738504+0000","last_peered":"2024-03-23T09:41:59.738504+0000","last_clean":"2024-03-23T09:41:59.738504+0000","last_became_active":"2024-03-23T09:41:59.737766+0000","last_became_peered":"2024-03-23T09:41:59.737766+0000","last_unstale":"2024-03-23T09:41:59.738504+0000","last_undegraded":"2024-03-23T09:41:59.738504+0000","last_fullsized":"2024-03-23T09:41:59.738504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:30:25.030569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.723891+0000","last_change":"2024-03-23T09:41:59.374221+0000","last_active":"2024-03-23T09:41:59.723891+0000","last_peered":"2024-03-23T09:41:59.723891+0000","last_clean":"2024-03-23T09:41:59.723891+0000","last_became_active":"2024-03-23T09:41:59.373711+0000","last_became_peered":"2024-03-23T09:41:59.373711+0000","last_unstale":"2024-03-23T09:41:59.723891+0000","last_undegraded":"2024-03-23T09:41:59.723891+0000","last_fullsized":"2024-03-23T09:41:59.723891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:37.112055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727325+0000","last_change":"2024-03-23T09:41:59.739321+0000","last_active":"2024-03-23T09:42:00.727325+0000","last_peered":"2024-03-23T09:42:00.727325+0000","last_clean":"2024-03-23T09:42:00.727325+0000","last_became_active":"2024-03-23T09:41:59.739053+0000","last_became_peered":"2024-03-23T09:41:59.739053+0000","last_unstale":"2024-03-23T09:42:00.727325+0000","last_undegraded":"2024-03-23T09:42:00.727325+0000","last_fullsized":"2024-03-23T09:42:00.727325+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:33:43.751058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.728877+0000","last_change":"2024-03-23T09:41:59.375856+0000","last_active":"2024-03-23T09:41:59.728877+0000","last_peered":"2024-03-23T09:41:59.728877+0000","last_clean":"2024-03-23T09:41:59.728877+0000","last_became_active":"2024-03-23T09:41:59.375459+0000","last_became_peered":"2024-03-23T09:41:59.375459+0000","last_unstale":"2024-03-23T09:41:59.728877+0000","last_undegraded":"2024-03-23T09:41:59.728877+0000","last_fullsized":"2024-03-23T09:41:59.728877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:22:58.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.742278+0000","last_change":"2024-03-23T09:41:59.735276+0000","last_active":"2024-03-23T09:41:59.742278+0000","last_peered":"2024-03-23T09:41:59.742278+0000","last_clean":"2024-03-23T09:41:59.742278+0000","last_became_active":"2024-03-23T09:41:59.735012+0000","last_became_peered":"2024-03-23T09:41:59.735012+0000","last_unstale":"2024-03-23T09:41:59.742278+0000","last_undegraded":"2024-03-23T09:41:59.742278+0000","last_fullsized":"2024-03-23T09:41:59.742278+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:15:10.586620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.727433+0000","last_change":"2024-03-23T09:41:59.368538+0000","last_active":"2024-03-23T09:42:00.727433+0000","last_peered":"2024-03-23T09:42:00.727433+0000","last_clean":"2024-03-23T09:42:00.727433+0000","last_became_active":"2024-03-23T09:41:59.368289+0000","last_became_peered":"2024-03-23T09:41:59.368289+0000","last_unstale":"2024-03-23T09:42:00.727433+0000","last_undegraded":"2024-03-23T09:42:00.727433+0000","last_fullsized":"2024-03-23T09:42:00.727433+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:45:59.049942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-23T09:42:00.728878+0000","last_change":"2024-03-23T09:41:59.741981+0000","last_active":"2024-03-23T09:42:00.728878+0000","last_peered":"2024-03-23T09:42:00.728878+0000","last_clean":"2024-03-23T09:42:00.728878+0000","last_became_active":"2024-03-23T09:41:59.741694+0000","last_became_peered":"2024-03-23T09:41:59.741694+0000","last_unstale":"2024-03-23T09:42:00.728878+0000","last_undegraded":"2024-03-23T09:42:00.728878+0000","last_fullsized":"2024-03-23T09:42:00.728878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:59.349007+0000","last_clean_scrub_stamp":"2024-03-23T09:41:59.349007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:19:37.563727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.730710+0000","last_change":"2024-03-23T09:41:59.375945+0000","last_active":"2024-03-23T09:41:59.730710+0000","last_peered":"2024-03-23T09:41:59.730710+0000","last_clean":"2024-03-23T09:41:59.730710+0000","last_became_active":"2024-03-23T09:41:59.375606+0000","last_became_peered":"2024-03-23T09:41:59.375606+0000","last_unstale":"2024-03-23T09:41:59.730710+0000","last_undegraded":"2024-03-23T09:41:59.730710+0000","last_fullsized":"2024-03-23T09:41:59.730710+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:41:58.347492+0000","last_clean_scrub_stamp":"2024-03-23T09:41:58.347492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:29:46.827710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-23T09:41:59.732402+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:41:59.732402+0000","last_peered":"2024-03-23T09:41:59.732402+0000","last_clean":"2024-03-23T09:41:59.732402+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:41:59.732402+0000","last_undegraded":"2024-03-23T09:41:59.732402+0000","last_fullsized":"2024-03-23T09:41:59.732402+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":740873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.438},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54100000000000004}]},{"osd":1,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.499}]},{"osd":3,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.48799999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.377}]},{"osd":5,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.39600000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":144112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.60599999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.627},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.66700000000000004}]},{"osd":2,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.68700000000000006}]},{"osd":3,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58999999999999997}]},{"osd":4,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45000000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.72399999999999998}]},{"osd":7,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.501},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.47999999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.505}]},{"osd":3,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.40999999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.53000000000000003}]},{"osd":6,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.56000000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.499},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.58099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":144219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.56100000000000005}]},{"osd":2,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.61499999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51500000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.53900000000000003}]},{"osd":6,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.49399999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.57599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":401408,"data_stored":152001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.32800000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.309},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.373}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.40300000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.58199999999999996}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.57199999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.51700000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.48899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":376832,"data_stored":144185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.251},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.38100000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.42799999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.629}]},{"osd":6,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.71899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31500,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340340,"statfs":{"total":96636764160,"available":96604508160,"internally_reserved":0,"allocated":389120,"data_stored":145701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.246},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.32000000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.28199999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":0.26700000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.69699999999999995}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.60399999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.73399999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.67400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":962560,"data_stored":733864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.28299999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.36799999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.313},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.39100000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.52800000000000002}]},{"osd":5,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.48199999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57099999999999995}]},{"osd":7,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:42:07.512 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T09:42:07.512 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-23T09:42:07.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.654+0000 7f575c4af640 1 Processor -- start 2024-03-23T09:42:07.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.654+0000 7f575c4af640 1 -- start start 2024-03-23T09:42:07.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.655+0000 7f575c4af640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f575415ce80 0x7f57540d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:07.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.655+0000 7f575c4af640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57540b83d0 0x7f57540cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.655+0000 7f575c4af640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57540b6b10 con 0x7f57540d2460 2024-03-23T09:42:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.655+0000 7f575c4af640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57540d13c0 con 0x7f575415ce80 2024-03-23T09:42:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.655+0000 7f575c4af640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57540d16a0 con 0x7f57540b83d0 2024-03-23T09:42:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.655+0000 7f575a24b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f575415ce80 0x7f57540d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.655+0000 7f575a24b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f575415ce80 0x7f57540d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47584/0 (socket says 172.21.15.47:47584) 2024-03-23T09:42:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.655+0000 7f575a24b640 1 -- 172.21.15.47:0/2955637855 learned_addr learned my addr 172.21.15.47:0/2955637855 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.655+0000 7f5759a4a640 1 --2- 172.21.15.47:0/2955637855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57540b83d0 0x7f57540cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.655+0000 7f575a24b640 1 -- 172.21.15.47:0/2955637855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57540b83d0 msgr2=0x7f57540cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.655+0000 7f575a24b640 1 --2- 172.21.15.47:0/2955637855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57540b83d0 0x7f57540cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.655+0000 7f575a24b640 1 -- 172.21.15.47:0/2955637855 >> v1:172.21.15.175:6789/0 conn(0x7f57540d2460 legacy=0x7f57540b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.655+0000 7f575a24b640 1 -- 172.21.15.47:0/2955637855 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57540d1980 con 0x7f575415ce80 2024-03-23T09:42:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.656+0000 7f575a24b640 1 --2- 172.21.15.47:0/2955637855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f575415ce80 0x7f57540d0a50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5750009870 tx=0x7f575002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b1f70500db4c27 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.656+0000 7f5759249640 1 -- 172.21.15.47:0/2955637855 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575002f840 con 0x7f575415ce80 2024-03-23T09:42:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.656+0000 7f5759249640 1 -- 172.21.15.47:0/2955637855 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575002f9a0 con 0x7f575415ce80 2024-03-23T09:42:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.656+0000 7f5759249640 1 -- 172.21.15.47:0/2955637855 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575002fd30 con 0x7f575415ce80 2024-03-23T09:42:07.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.656+0000 7f575c4af640 1 -- 172.21.15.47:0/2955637855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f575415ce80 msgr2=0x7f57540d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:07.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.656+0000 7f575c4af640 1 --2- 172.21.15.47:0/2955637855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f575415ce80 0x7f57540d0a50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5750009870 tx=0x7f575002ba20 comp rx=0 tx=0).stop 2024-03-23T09:42:07.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.656+0000 7f575c4af640 1 -- 172.21.15.47:0/2955637855 shutdown_connections 2024-03-23T09:42:07.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.656+0000 7f575c4af640 1 --2- 172.21.15.47:0/2955637855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57540b83d0 0x7f57540cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.656+0000 7f575c4af640 1 --2- 172.21.15.47:0/2955637855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f575415ce80 0x7f57540d0a50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.656+0000 7f575c4af640 1 -- 172.21.15.47:0/2955637855 >> 172.21.15.47:0/2955637855 conn(0x7f5754065df0 msgr2=0x7f5754066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:07.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.657+0000 7f575c4af640 1 -- 172.21.15.47:0/2955637855 shutdown_connections 2024-03-23T09:42:07.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.657+0000 7f575c4af640 1 -- 172.21.15.47:0/2955637855 wait complete. 2024-03-23T09:42:07.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.658+0000 7f575c4af640 1 Processor -- start 2024-03-23T09:42:07.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.658+0000 7f575c4af640 1 -- start start 2024-03-23T09:42:07.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.658+0000 7f575c4af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57540b83d0 0x7f57540c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:07.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.658+0000 7f575c4af640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57540d2460 0x7f57540b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:07.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.658+0000 7f575c4af640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f575415ce80 0x7f57540b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:07.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.658+0000 7f575c4af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57541537c0 con 0x7f57540b83d0 2024-03-23T09:42:07.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.658+0000 7f575c4af640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5754153a60 con 0x7f575415ce80 2024-03-23T09:42:07.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.658+0000 7f575c4af640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5754159650 con 0x7f57540d2460 2024-03-23T09:42:07.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.658+0000 7f575a24b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57540b83d0 0x7f57540c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:07.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.658+0000 7f575a24b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57540b83d0 0x7f57540c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47592/0 (socket says 172.21.15.47:47592) 2024-03-23T09:42:07.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.658+0000 7f575a24b640 1 -- 172.21.15.47:0/2974333728 learned_addr learned my addr 172.21.15.47:0/2974333728 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:07.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f5759a4a640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57540d2460 0x7f57540b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:07.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f575aa4c640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f575415ce80 0x7f57540b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:07.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f575a24b640 1 -- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57540d2460 msgr2=0x7f57540b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:07.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f575a24b640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57540d2460 0x7f57540b2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f575a24b640 1 -- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f575415ce80 msgr2=0x7f57540b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:07.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f575a24b640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f575415ce80 0x7f57540b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f575a24b640 1 -- 172.21.15.47:0/2974333728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57500094d0 con 0x7f57540b83d0 2024-03-23T09:42:07.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f575aa4c640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f575415ce80 0x7f57540b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:07.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f575a24b640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57540b83d0 0x7f57540c84c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5750000c00 tx=0x7f5750004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:07.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f574affd640 1 -- 172.21.15.47:0/2974333728 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750037c40 con 0x7f57540b83d0 2024-03-23T09:42:07.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f574affd640 1 -- 172.21.15.47:0/2974333728 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5750037da0 con 0x7f57540b83d0 2024-03-23T09:42:07.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f575c4af640 1 -- 172.21.15.47:0/2974333728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5754159930 con 0x7f57540b83d0 2024-03-23T09:42:07.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f574affd640 1 -- 172.21.15.47:0/2974333728 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750041640 con 0x7f57540b83d0 2024-03-23T09:42:07.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.659+0000 7f575c4af640 1 -- 172.21.15.47:0/2974333728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57540c8a00 con 0x7f57540b83d0 2024-03-23T09:42:07.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.660+0000 7f575c4af640 1 -- 172.21.15.47:0/2974333728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5754072360 con 0x7f57540b83d0 2024-03-23T09:42:07.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.661+0000 7f574affd640 1 -- 172.21.15.47:0/2974333728 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f57500417a0 con 0x7f57540b83d0 2024-03-23T09:42:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.661+0000 7f574affd640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f573806ac00 0x7f573806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.661+0000 7f574affd640 1 -- 172.21.15.47:0/2974333728 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f575003c070 con 0x7f57540b83d0 2024-03-23T09:42:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.661+0000 7f5759a4a640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f573806ac00 0x7f573806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.662+0000 7f5759a4a640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f573806ac00 0x7f573806d0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f57540b7f60 tx=0x7f5744008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:07.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.665+0000 7f574affd640 1 -- 172.21.15.47:0/2974333728 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5750077290 con 0x7f57540b83d0 2024-03-23T09:42:07.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.921+0000 7f575c4af640 1 -- 172.21.15.47:0/2974333728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5754066be0 con 0x7f57540b83d0 2024-03-23T09:42:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.924+0000 7f574affd640 1 -- 172.21.15.47:0/2974333728 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f575007a8b0 con 0x7f57540b83d0 2024-03-23T09:42:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-03-23T09:42:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.927+0000 7f575c4af640 1 -- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f573806ac00 msgr2=0x7f573806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.927+0000 7f575c4af640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f573806ac00 0x7f573806d0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f57540b7f60 tx=0x7f5744008040 comp rx=0 tx=0).stop 2024-03-23T09:42:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.927+0000 7f575c4af640 1 -- 172.21.15.47:0/2974333728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57540b83d0 msgr2=0x7f57540c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.927+0000 7f575c4af640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57540b83d0 0x7f57540c84c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5750000c00 tx=0x7f5750004730 comp rx=0 tx=0).stop 2024-03-23T09:42:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.927+0000 7f575c4af640 1 -- 172.21.15.47:0/2974333728 shutdown_connections 2024-03-23T09:42:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.927+0000 7f575c4af640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f573806ac00 0x7f573806d0c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.927+0000 7f575c4af640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f575415ce80 0x7f57540b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.927+0000 7f575c4af640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57540d2460 0x7f57540b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.927+0000 7f575c4af640 1 --2- 172.21.15.47:0/2974333728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57540b83d0 0x7f57540c84c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.927+0000 7f575c4af640 1 -- 172.21.15.47:0/2974333728 >> 172.21.15.47:0/2974333728 conn(0x7f5754065df0 msgr2=0x7f575415f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.927+0000 7f575c4af640 1 -- 172.21.15.47:0/2974333728 shutdown_connections 2024-03-23T09:42:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:07.927+0000 7f575c4af640 1 -- 172.21.15.47:0/2974333728 wait complete. 2024-03-23T09:42:07.938 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:42:08.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.038+0000 7f7a27ff0640 1 Processor -- start 2024-03-23T09:42:08.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.038+0000 7f7a27ff0640 1 -- start start 2024-03-23T09:42:08.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a27ff0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a200d23a0 0x7f7a200d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a27ff0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7a200b8300 0x7f7a200d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a27ff0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a200d10a0 con 0x7f7a2015cee0 2024-03-23T09:42:08.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a27ff0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a200d1300 con 0x7f7a200b8300 2024-03-23T09:42:08.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a27ff0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200d15e0 con 0x7f7a200d23a0 2024-03-23T09:42:08.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a25d8c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7a2015cee0 0x7f7a200d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58096/0 (socket says 172.21.15.47:58096) 2024-03-23T09:42:08.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a25d8c640 1 -- 172.21.15.47:0/413448978 learned_addr learned my addr 172.21.15.47:0/413448978 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:08.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a2558b640 1 --2- 172.21.15.47:0/413448978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a200d23a0 0x7f7a200d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a2658d640 1 --2- 172.21.15.47:0/413448978 >> v2:172.21.15.175:3300/0 conn(0x7f7a200b8300 0x7f7a200d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823679875 0 0) 0x7f7a200d10a0 con 0x7f7a2015cee0 2024-03-23T09:42:08.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a0c003520 con 0x7f7a2015cee0 2024-03-23T09:42:08.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 37674934 0 0) 0x7f7a0c003520 con 0x7f7a2015cee0 2024-03-23T09:42:08.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a200d23a0 msgr2=0x7f7a200d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a24d8a640 1 --2- 172.21.15.47:0/413448978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a200d23a0 0x7f7a200d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 >> v2:172.21.15.175:3300/0 conn(0x7f7a200b8300 msgr2=0x7f7a200d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a24d8a640 1 --2- 172.21.15.47:0/413448978 >> v2:172.21.15.175:3300/0 conn(0x7f7a200b8300 0x7f7a200d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a200d18c0 con 0x7f7a2015cee0 2024-03-23T09:42:08.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f7a1c001690 con 0x7f7a2015cee0 2024-03-23T09:42:08.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a2658d640 1 --2- 172.21.15.47:0/413448978 >> v2:172.21.15.175:3300/0 conn(0x7f7a200b8300 0x7f7a200d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:42:08.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 >> v1:172.21.15.47:6789/0 conn(0x7f7a2015cee0 legacy=0x7f7a200d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a24d8a640 1 --2- 172.21.15.47:0/413448978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a0c003970 0x7f7a0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a24d8a640 1 --2- 172.21.15.47:0/413448978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a0c0047c0 0x7f7a0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.039+0000 7f7a24d8a640 1 --2- 172.21.15.47:0/413448978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a200d23a0 0x7f7a0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.040+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200d1300 con 0x7f7a0c003970 2024-03-23T09:42:08.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.040+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200d15e0 con 0x7f7a0c0047c0 2024-03-23T09:42:08.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.040+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c00b980 con 0x7f7a200d23a0 2024-03-23T09:42:08.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.040+0000 7f7a25d8c640 1 --2- 172.21.15.47:0/413448978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a0c003970 0x7f7a0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.040+0000 7f7a2558b640 1 --2- 172.21.15.47:0/413448978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a0c0047c0 0x7f7a0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.040+0000 7f7a2658d640 1 --2- 172.21.15.47:0/413448978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a200d23a0 0x7f7a0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.040+0000 7f7a25d8c640 1 -- 172.21.15.47:0/413448978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a200d23a0 msgr2=0x7f7a0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.040+0000 7f7a25d8c640 1 --2- 172.21.15.47:0/413448978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a200d23a0 0x7f7a0c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.040+0000 7f7a25d8c640 1 -- 172.21.15.47:0/413448978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a0c0047c0 msgr2=0x7f7a0c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.040+0000 7f7a25d8c640 1 --2- 172.21.15.47:0/413448978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a0c0047c0 0x7f7a0c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.040+0000 7f7a25d8c640 1 -- 172.21.15.47:0/413448978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c003520 con 0x7f7a0c003970 2024-03-23T09:42:08.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.041+0000 7f7a25d8c640 1 --2- 172.21.15.47:0/413448978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a0c003970 0x7f7a0c003d90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00c210 tx=0x7f7a0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80df40a6fea07502 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:08.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.041+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c005ad0 con 0x7f7a0c003970 2024-03-23T09:42:08.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.041+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1c005c50 con 0x7f7a0c003970 2024-03-23T09:42:08.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.041+0000 7f7a24d8a640 1 -- 172.21.15.47:0/413448978 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c02c8b0 con 0x7f7a0c003970 2024-03-23T09:42:08.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.041+0000 7f7a27ff0640 1 -- 172.21.15.47:0/413448978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a0c003970 msgr2=0x7f7a0c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.041+0000 7f7a27ff0640 1 --2- 172.21.15.47:0/413448978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a0c003970 0x7f7a0c003d90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00c210 tx=0x7f7a0c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.041+0000 7f7a2658d640 1 -- 172.21.15.47:0/413448978 reap_dead start 2024-03-23T09:42:08.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.041+0000 7f7a27ff0640 1 -- 172.21.15.47:0/413448978 shutdown_connections 2024-03-23T09:42:08.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.041+0000 7f7a27ff0640 1 -- 172.21.15.47:0/413448978 >> 172.21.15.47:0/413448978 conn(0x7f7a20065df0 msgr2=0x7f7a20066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:08.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.041+0000 7f7a27ff0640 1 -- 172.21.15.47:0/413448978 shutdown_connections 2024-03-23T09:42:08.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.041+0000 7f7a27ff0640 1 -- 172.21.15.47:0/413448978 wait complete. 2024-03-23T09:42:08.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.043+0000 7f7a27ff0640 1 Processor -- start 2024-03-23T09:42:08.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.043+0000 7f7a27ff0640 1 -- start start 2024-03-23T09:42:08.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.043+0000 7f7a27ff0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a0c003970 0x7f7a201f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.043+0000 7f7a27ff0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a200c8a40 0x7f7a201f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.043+0000 7f7a27ff0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2015f2a0 0x7f7a2015f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.043+0000 7f7a27ff0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2015fc60 con 0x7f7a200c8a40 2024-03-23T09:42:08.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.043+0000 7f7a27ff0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a201f63b0 con 0x7f7a0c003970 2024-03-23T09:42:08.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.043+0000 7f7a27ff0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a201f6690 con 0x7f7a2015f2a0 2024-03-23T09:42:08.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.043+0000 7f7a2558b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a200c8a40 0x7f7a201f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.043+0000 7f7a2558b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a200c8a40 0x7f7a201f5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47616/0 (socket says 172.21.15.47:47616) 2024-03-23T09:42:08.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.043+0000 7f7a2558b640 1 -- 172.21.15.47:0/335240547 learned_addr learned my addr 172.21.15.47:0/335240547 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:08.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a25d8c640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a0c003970 0x7f7a201f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a2658d640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2015f2a0 0x7f7a2015f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a2558b640 1 -- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2015f2a0 msgr2=0x7f7a2015f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a2558b640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2015f2a0 0x7f7a2015f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a2558b640 1 -- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a0c003970 msgr2=0x7f7a201f1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a2558b640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a0c003970 0x7f7a201f1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a2558b640 1 -- 172.21.15.47:0/335240547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a201f6970 con 0x7f7a200c8a40 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a2658d640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2015f2a0 0x7f7a2015f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a2558b640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a200c8a40 0x7f7a201f5a60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7a10011960 tx=0x7f7a10011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a167fc640 1 -- 172.21.15.47:0/335240547 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10004240 con 0x7f7a200c8a40 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a167fc640 1 -- 172.21.15.47:0/335240547 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a100043a0 con 0x7f7a200c8a40 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a27ff0640 1 -- 172.21.15.47:0/335240547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a200c9a50 con 0x7f7a200c8a40 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a167fc640 1 -- 172.21.15.47:0/335240547 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000fa20 con 0x7f7a200c8a40 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.044+0000 7f7a27ff0640 1 -- 172.21.15.47:0/335240547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a201f5fa0 con 0x7f7a200c8a40 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.045+0000 7f7a27ff0640 1 -- 172.21.15.47:0/335240547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a20072360 con 0x7f7a200c8a40 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.045+0000 7f7a167fc640 1 -- 172.21.15.47:0/335240547 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7a1000fb80 con 0x7f7a200c8a40 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.046+0000 7f7a167fc640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a0406ac30 0x7f7a0406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.046+0000 7f7a167fc640 1 -- 172.21.15.47:0/335240547 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a10089910 con 0x7f7a200c8a40 2024-03-23T09:42:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.046+0000 7f7a25d8c640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a0406ac30 0x7f7a0406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.047+0000 7f7a25d8c640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a0406ac30 0x7f7a0406d0f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c034000 tx=0x7f7a1c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:08.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.050+0000 7f7a167fc640 1 -- 172.21.15.47:0/335240547 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7a10054b80 con 0x7f7a200c8a40 2024-03-23T09:42:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.266+0000 7f7a27ff0640 1 -- 172.21.15.47:0/335240547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a20066be0 con 0x7f7a200c8a40 2024-03-23T09:42:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.266+0000 7f7a167fc640 1 -- 172.21.15.47:0/335240547 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f7a10023090 con 0x7f7a200c8a40 2024-03-23T09:42:08.267 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:08.267 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3373309448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3373309448},{"type":"v1","addr":"172.21.15.175:6836","nonce":3373309448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4118066020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4118066020},{"type":"v1","addr":"172.21.15.175:6837","nonce":4118066020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2916649516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2916649516},{"type":"v1","addr":"172.21.15.47:6835","nonce":2916649516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:59.719783+0000","modified":"2024-03-23T09:42:01.774130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6834/400299689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":400299689},{"type":"v1","addr":"172.21.15.47:6834","nonce":400299689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T09:42:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2024-03-23T09:42:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.268+0000 7f7a27ff0640 1 -- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a0406ac30 msgr2=0x7f7a0406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.268+0000 7f7a27ff0640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a0406ac30 0x7f7a0406d0f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c034000 tx=0x7f7a1c007000 comp rx=0 tx=0).stop 2024-03-23T09:42:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.268+0000 7f7a27ff0640 1 -- 172.21.15.47:0/335240547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a200c8a40 msgr2=0x7f7a201f5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.268+0000 7f7a27ff0640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a200c8a40 0x7f7a201f5a60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7a10011960 tx=0x7f7a10011e20 comp rx=0 tx=0).stop 2024-03-23T09:42:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.268+0000 7f7a27ff0640 1 -- 172.21.15.47:0/335240547 shutdown_connections 2024-03-23T09:42:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.268+0000 7f7a27ff0640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a0406ac30 0x7f7a0406d0f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.268+0000 7f7a27ff0640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2015f2a0 0x7f7a2015f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.268+0000 7f7a27ff0640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a200c8a40 0x7f7a201f5a60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.268+0000 7f7a27ff0640 1 --2- 172.21.15.47:0/335240547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a0c003970 0x7f7a201f1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.268+0000 7f7a27ff0640 1 -- 172.21.15.47:0/335240547 >> 172.21.15.47:0/335240547 conn(0x7f7a20065df0 msgr2=0x7f7a200cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.268+0000 7f7a27ff0640 1 -- 172.21.15.47:0/335240547 shutdown_connections 2024-03-23T09:42:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.268+0000 7f7a27ff0640 1 -- 172.21.15.47:0/335240547 wait complete. 2024-03-23T09:42:08.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:41:59.719783+0000', 'modified': '2024-03-23T09:42:01.774130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5196}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5196': {'gid': 5196, 'name': 'c', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6834/400299689', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 400299689}, {'type': 'v1', 'addr': '172.21.15.47:6834', 'nonce': 400299689}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:42:08.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:42:08.682 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T09:42:08.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.785+0000 7ff67324a640 1 Processor -- start 2024-03-23T09:42:08.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.785+0000 7ff67324a640 1 -- start start 2024-03-23T09:42:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff67324a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff66c15cd00 0x7ff66c0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff67324a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff66c0d1020 0x7ff66c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff67324a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff66c0cfba0 con 0x7ff66c0cf750 2024-03-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff67324a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66c15d0e0 con 0x7ff66c0d1020 2024-03-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff67324a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66c0d1540 con 0x7ff66c15cd00 2024-03-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff66bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff66c0d1020 0x7ff66c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff66bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff66c0d1020 0x7ff66c0cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47622/0 (socket says 172.21.15.47:47622) 2024-03-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff66bfff640 1 -- 172.21.15.47:0/1879409284 learned_addr learned my addr 172.21.15.47:0/1879409284 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff670fe6640 1 --2- 172.21.15.47:0/1879409284 >> v2:172.21.15.175:3300/0 conn(0x7ff66c15cd00 0x7ff66c0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff66bfff640 1 -- 172.21.15.47:0/1879409284 >> v2:172.21.15.175:3300/0 conn(0x7ff66c15cd00 msgr2=0x7ff66c0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff66bfff640 1 --2- 172.21.15.47:0/1879409284 >> v2:172.21.15.175:3300/0 conn(0x7ff66c15cd00 0x7ff66c0d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff66bfff640 1 -- 172.21.15.47:0/1879409284 >> v1:172.21.15.47:6789/0 conn(0x7ff66c0cf750 legacy=0x7ff66c0b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff66bfff640 1 -- 172.21.15.47:0/1879409284 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c0d1820 con 0x7ff66c0d1020 2024-03-23T09:42:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.786+0000 7ff66b7fe640 1 -- 172.21.15.47:0/1879409284 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056761341 0 0) 0x7ff66c0cfba0 con 0x7ff66c0cf750 2024-03-23T09:42:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.787+0000 7ff66bfff640 1 --2- 172.21.15.47:0/1879409284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff66c0d1020 0x7ff66c0cf210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff66c06c1a0 tx=0x7ff66002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce4da5155e02588 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.787+0000 7ff66b7fe640 1 -- 172.21.15.47:0/1879409284 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660031870 con 0x7ff66c0d1020 2024-03-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.787+0000 7ff66b7fe640 1 -- 172.21.15.47:0/1879409284 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6600319d0 con 0x7ff66c0d1020 2024-03-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.787+0000 7ff66b7fe640 1 -- 172.21.15.47:0/1879409284 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660031d60 con 0x7ff66c0d1020 2024-03-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.788+0000 7ff67324a640 1 -- 172.21.15.47:0/1879409284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff66c0d1020 msgr2=0x7ff66c0cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.788+0000 7ff67324a640 1 --2- 172.21.15.47:0/1879409284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff66c0d1020 0x7ff66c0cf210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff66c06c1a0 tx=0x7ff66002b910 comp rx=0 tx=0).stop 2024-03-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.788+0000 7ff67324a640 1 -- 172.21.15.47:0/1879409284 shutdown_connections 2024-03-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.788+0000 7ff67324a640 1 --2- 172.21.15.47:0/1879409284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff66c0d1020 0x7ff66c0cf210 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.788+0000 7ff67324a640 1 --2- 172.21.15.47:0/1879409284 >> v2:172.21.15.175:3300/0 conn(0x7ff66c15cd00 0x7ff66c0d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.788+0000 7ff67324a640 1 -- 172.21.15.47:0/1879409284 >> 172.21.15.47:0/1879409284 conn(0x7ff66c06b160 msgr2=0x7ff66c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.788+0000 7ff67324a640 1 -- 172.21.15.47:0/1879409284 shutdown_connections 2024-03-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.788+0000 7ff67324a640 1 -- 172.21.15.47:0/1879409284 wait complete. 2024-03-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.789+0000 7ff67324a640 1 Processor -- start 2024-03-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.789+0000 7ff67324a640 1 -- start start 2024-03-23T09:42:08.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff67324a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff66c0cf750 0x7ff66c1f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff67324a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff66c0d1020 0x7ff66c1f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff67324a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff66c15cd00 0x7ff66c15db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff67324a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c1f63f0 con 0x7ff66c0d1020 2024-03-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff67324a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c1f1d40 con 0x7ff66c15cd00 2024-03-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff67324a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c1f5870 con 0x7ff66c0cf750 2024-03-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff66bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff66c0d1020 0x7ff66c1f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff66bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff66c0d1020 0x7ff66c1f2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47636/0 (socket says 172.21.15.47:47636) 2024-03-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff66bfff640 1 -- 172.21.15.47:0/2805345382 learned_addr learned my addr 172.21.15.47:0/2805345382 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff670fe6640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff66c0cf750 0x7ff66c1f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff6717e7640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff66c15cd00 0x7ff66c15db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff66bfff640 1 -- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff66c0cf750 msgr2=0x7ff66c1f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff66bfff640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff66c0cf750 0x7ff66c1f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff66bfff640 1 -- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff66c15cd00 msgr2=0x7ff66c15db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff66bfff640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff66c15cd00 0x7ff66c15db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.790+0000 7ff66bfff640 1 -- 172.21.15.47:0/2805345382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6600094d0 con 0x7ff66c0d1020 2024-03-23T09:42:08.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.791+0000 7ff66bfff640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff66c0d1020 0x7ff66c1f2900 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff660000c00 tx=0x7ff6600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:08.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.791+0000 7ff6697fa640 1 -- 172.21.15.47:0/2805345382 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660004230 con 0x7ff66c0d1020 2024-03-23T09:42:08.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.791+0000 7ff6697fa640 1 -- 172.21.15.47:0/2805345382 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6600043b0 con 0x7ff66c0d1020 2024-03-23T09:42:08.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.791+0000 7ff67324a640 1 -- 172.21.15.47:0/2805345382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66c1f5b50 con 0x7ff66c0d1020 2024-03-23T09:42:08.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.791+0000 7ff6697fa640 1 -- 172.21.15.47:0/2805345382 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660004520 con 0x7ff66c0d1020 2024-03-23T09:42:08.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.791+0000 7ff67324a640 1 -- 172.21.15.47:0/2805345382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66c1f6110 con 0x7ff66c0d1020 2024-03-23T09:42:08.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.792+0000 7ff6697fa640 1 -- 172.21.15.47:0/2805345382 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff660043070 con 0x7ff66c0d1020 2024-03-23T09:42:08.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.793+0000 7ff6697fa640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff65806ac00 0x7ff65806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.793+0000 7ff6697fa640 1 -- 172.21.15.47:0/2805345382 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff66002d040 con 0x7ff66c0d1020 2024-03-23T09:42:08.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.793+0000 7ff670fe6640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff65806ac00 0x7ff65806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.793+0000 7ff670fe6640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff65806ac00 0x7ff65806d0c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff664011340 tx=0x7ff66400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.794+0000 7ff675dc0740 1 Processor -- start 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.794+0000 7ff675dc0740 1 -- start start 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.794+0000 7ff675dc0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebfb750 0x55f33ebb2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.794+0000 7ff675dc0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebb17d0 0x55f33ebb32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.794+0000 7ff675dc0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebadc70 0x55f33ebdda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.794+0000 7ff675dc0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33eb1e9b0 con 0x55f33ebb17d0 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.794+0000 7ff675dc0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33eb1c140 con 0x55f33ebadc70 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.794+0000 7ff675dc0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f33eb17a30 con 0x55f33ebfb750 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff66bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebb17d0 0x55f33ebb32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff66bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebb17d0 0x55f33ebb32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47650/0 (socket says 172.21.15.47:47650) 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff66bfff640 1 -- 172.21.15.47:0/259474359 learned_addr learned my addr 172.21.15.47:0/259474359 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff670fe6640 1 --2- 172.21.15.47:0/259474359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebfb750 0x55f33ebb2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff6717e7640 1 --2- 172.21.15.47:0/259474359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebadc70 0x55f33ebdda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff66bfff640 1 -- 172.21.15.47:0/259474359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebfb750 msgr2=0x55f33ebb2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff66bfff640 1 --2- 172.21.15.47:0/259474359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebfb750 0x55f33ebb2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff66bfff640 1 -- 172.21.15.47:0/259474359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebadc70 msgr2=0x55f33ebdda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff66bfff640 1 --2- 172.21.15.47:0/259474359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebadc70 0x55f33ebdda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff66bfff640 1 -- 172.21.15.47:0/259474359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f33eaa42f0 con 0x55f33ebb17d0 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff670fe6640 1 --2- 172.21.15.47:0/259474359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebfb750 0x55f33ebb2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff6717e7640 1 --2- 172.21.15.47:0/259474359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebadc70 0x55f33ebdda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff66bfff640 1 --2- 172.21.15.47:0/259474359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebb17d0 0x55f33ebb32c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff6600a98d0 tx=0x7ff66007a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a278d1e0ca55403d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff6527fc640 1 -- 172.21.15.47:0/259474359 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66007e050 con 0x55f33ebb17d0 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff6527fc640 1 -- 172.21.15.47:0/259474359 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff660082610 con 0x55f33ebb17d0 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.795+0000 7ff6527fc640 1 -- 172.21.15.47:0/259474359 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6600ab0b0 con 0x55f33ebb17d0 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.796+0000 7ff675dc0740 1 -- 172.21.15.47:0/259474359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebb17d0 msgr2=0x55f33ebb32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.796+0000 7ff675dc0740 1 --2- 172.21.15.47:0/259474359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebb17d0 0x55f33ebb32c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff6600a98d0 tx=0x7ff66007a120 comp rx=0 tx=0).stop 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.796+0000 7ff675dc0740 1 -- 172.21.15.47:0/259474359 shutdown_connections 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.796+0000 7ff675dc0740 1 --2- 172.21.15.47:0/259474359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebadc70 0x55f33ebdda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.796+0000 7ff675dc0740 1 --2- 172.21.15.47:0/259474359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebb17d0 0x55f33ebb32c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.796+0000 7ff675dc0740 1 --2- 172.21.15.47:0/259474359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebfb750 0x55f33ebb2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.796+0000 7ff675dc0740 1 -- 172.21.15.47:0/259474359 >> 172.21.15.47:0/259474359 conn(0x55f33ebafee0 msgr2=0x55f33ebf8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.796+0000 7ff675dc0740 1 -- 172.21.15.47:0/259474359 shutdown_connections 2024-03-23T09:42:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.796+0000 7ff675dc0740 1 -- 172.21.15.47:0/259474359 wait complete. 2024-03-23T09:42:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.796+0000 7ff675dc0740 1 -- start start 2024-03-23T09:42:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.797+0000 7ff675dc0740 1 Processor -- start 2024-03-23T09:42:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.797+0000 7ff675dc0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebb17d0 0x55f33ebd5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.797+0000 7ff675dc0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebfb750 0x55f33ebdb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.797+0000 7ff675dc0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebffa80 0x55f33ebf8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.797+0000 7ff675dc0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33ead6090 con 0x55f33ebffa80 2024-03-23T09:42:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.797+0000 7ff675dc0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33ead5d30 con 0x55f33ebb17d0 2024-03-23T09:42:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.797+0000 7ff675dc0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f33ea9b2a0 con 0x55f33ebfb750 2024-03-23T09:42:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.797+0000 7ff675dc0740 10 client.0 _ll_register_callbacks cb 0x55f33ead1050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:42:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.797+0000 7ff6717e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebffa80 0x55f33ebf8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.797+0000 7ff6717e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebffa80 0x55f33ebf8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47666/0 (socket says 172.21.15.47:47666) 2024-03-23T09:42:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.797+0000 7ff6717e7640 1 -- 172.21.15.47:0/3779329076 learned_addr learned my addr 172.21.15.47:0/3779329076 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.797+0000 7ff66bfff640 1 --2- 172.21.15.47:0/3779329076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebfb750 0x55f33ebdb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff670fe6640 1 --2- 172.21.15.47:0/3779329076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebb17d0 0x55f33ebd5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff6717e7640 1 -- 172.21.15.47:0/3779329076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebfb750 msgr2=0x55f33ebdb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff6717e7640 1 --2- 172.21.15.47:0/3779329076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebfb750 0x55f33ebdb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff6717e7640 1 -- 172.21.15.47:0/3779329076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebb17d0 msgr2=0x55f33ebd5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff6717e7640 1 --2- 172.21.15.47:0/3779329076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebb17d0 0x55f33ebd5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff6717e7640 1 -- 172.21.15.47:0/3779329076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66c1f6110 con 0x55f33ebffa80 2024-03-23T09:42:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff6717e7640 1 --2- 172.21.15.47:0/3779329076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebffa80 0x55f33ebf8e00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff65c003a60 tx=0x7ff65c003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e655346031f5db server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff64b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff64b7fe640 1 -- 172.21.15.47:0/3779329076 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c001c20 con 0x55f33ebffa80 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff64b7fe640 1 -- 172.21.15.47:0/3779329076 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff65c016b10 con 0x55f33ebffa80 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff675dc0740 1 -- 172.21.15.47:0/3779329076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f33e80b3c0 con 0x55f33ebffa80 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff64b7fe640 1 -- 172.21.15.47:0/3779329076 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff65c001d80 con 0x55f33ebffa80 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff675dc0740 10 client.5385 fetch_fsmap learned FSMap version 12 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff675dc0740 1 -- 172.21.15.47:0/3779329076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x55f33eb160b0 con 0x55f33ebffa80 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.798+0000 7ff64b7fe640 1 -- 172.21.15.47:0/3779329076 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f33e80b3c0 con 0x55f33ebffa80 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.799+0000 7ff64b7fe640 1 -- 172.21.15.47:0/3779329076 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 12) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff65c016cd0 con 0x55f33ebffa80 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.799+0000 7ff675dc0740 10 client.5385 fetch_fsmap finished waiting for FSMap version 12 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.799+0000 7ff675dc0740 10 client.5385 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.799+0000 7ff675dc0740 20 client.5385 populate_metadata read hostname 'smithi047' 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.799+0000 7ff675dc0740 1 --2- 172.21.15.47:0/3779329076 >> [v2:172.21.15.47:6832/400299689,v1:172.21.15.47:6834/400299689] conn(0x55f33ebfd110 0x55f33ebfd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.799+0000 7ff675dc0740 4 client.5385 mds_command: new command op to 5196 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.799+0000 7ff675dc0740 1 -- 172.21.15.47:0/3779329076 --> [v2:172.21.15.47:6832/400299689,v1:172.21.15.47:6834/400299689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f33e81c6c0 con 0x55f33ebfd110 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.799+0000 7ff670fe6640 1 --2- 172.21.15.47:0/3779329076 >> [v2:172.21.15.47:6832/400299689,v1:172.21.15.47:6834/400299689] conn(0x55f33ebfd110 0x55f33ebfd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.799+0000 7ff64b7fe640 10 client.5385 ms_handle_connect on v2:172.21.15.47:6832/400299689 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.799+0000 7ff670fe6640 1 --2- 172.21.15.47:0/3779329076 >> [v2:172.21.15.47:6832/400299689,v1:172.21.15.47:6834/400299689] conn(0x55f33ebfd110 0x55f33ebfd4f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1b29d1fc414ca1 server_cookie=ef5b20d6d2294ace in_seq=0 out_seq=0 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff64b7fe640 1 -- 172.21.15.47:0/3779329076 <== mds.0 v2:172.21.15.47:6832/400299689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f33e81c6c0 con 0x55f33ebfd110 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff64b7fe640 10 client.5385 handle_command_reply: tid=0 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff675dc0740 1 client.5385 shutdown 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff675dc0740 1 -- 172.21.15.47:0/3779329076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebffa80 msgr2=0x55f33ebf8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff675dc0740 1 --2- 172.21.15.47:0/3779329076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebffa80 0x55f33ebf8e00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff65c003a60 tx=0x7ff65c003f20 comp rx=0 tx=0).stop 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff675dc0740 1 -- 172.21.15.47:0/3779329076 shutdown_connections 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff675dc0740 1 --2- 172.21.15.47:0/3779329076 >> [v2:172.21.15.47:6832/400299689,v1:172.21.15.47:6834/400299689] conn(0x55f33ebfd110 0x55f33ebfd4f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff675dc0740 1 --2- 172.21.15.47:0/3779329076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebffa80 0x55f33ebf8e00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff675dc0740 1 --2- 172.21.15.47:0/3779329076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebfb750 0x55f33ebdb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff64b7fe640 0 client.5385 ms_handle_reset on v2:172.21.15.47:6832/400299689 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff675dc0740 1 --2- 172.21.15.47:0/3779329076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebb17d0 0x55f33ebd5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff675dc0740 1 -- 172.21.15.47:0/3779329076 >> 172.21.15.47:0/3779329076 conn(0x55f33ebadc70 msgr2=0x55f33ebb1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff675dc0740 1 -- 172.21.15.47:0/3779329076 shutdown_connections 2024-03-23T09:42:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff675dc0740 1 -- 172.21.15.47:0/3779329076 wait complete. 2024-03-23T09:42:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.800+0000 7ff675dc0740 20 client.5385 trim_cache size 0 max 16384 2024-03-23T09:42:08.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.815+0000 7ff675dc0740 1 Processor -- start 2024-03-23T09:42:08.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.816+0000 7ff675dc0740 1 -- start start 2024-03-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.816+0000 7ff675dc0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebb17d0 0x55f33ebe19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.816+0000 7ff675dc0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 0x55f33ebd6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.816+0000 7ff675dc0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebffa80 0x55f33ebb1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.816+0000 7ff675dc0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33eae8510 con 0x55f33ebfb750 2024-03-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.816+0000 7ff675dc0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33ec09090 con 0x55f33ebffa80 2024-03-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.816+0000 7ff675dc0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f33ebb4260 con 0x55f33ebb17d0 2024-03-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.816+0000 7ff66bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 0x55f33ebd6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.816+0000 7ff66bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 0x55f33ebd6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47668/0 (socket says 172.21.15.47:47668) 2024-03-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.816+0000 7ff66bfff640 1 -- 172.21.15.47:0/1852848821 learned_addr learned my addr 172.21.15.47:0/1852848821 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff66bfff640 1 -- 172.21.15.47:0/1852848821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebb17d0 msgr2=0x55f33ebe19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff66bfff640 1 --2- 172.21.15.47:0/1852848821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebb17d0 0x55f33ebe19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff66bfff640 1 -- 172.21.15.47:0/1852848821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebffa80 msgr2=0x55f33ebb1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff66bfff640 1 --2- 172.21.15.47:0/1852848821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebffa80 0x55f33ebb1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff66bfff640 1 -- 172.21.15.47:0/1852848821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f33eab8430 con 0x55f33ebfb750 2024-03-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff66bfff640 1 --2- 172.21.15.47:0/1852848821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 0x55f33ebd6470 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff66002bf20 tx=0x7ff660046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16b3caa890d7b8d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff652ffd640 1 -- 172.21.15.47:0/1852848821 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66007e050 con 0x55f33ebfb750 2024-03-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff652ffd640 1 -- 172.21.15.47:0/1852848821 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6600785c0 con 0x55f33ebfb750 2024-03-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff652ffd640 1 -- 172.21.15.47:0/1852848821 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6600ab0b0 con 0x55f33ebfb750 2024-03-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff675dc0740 1 -- 172.21.15.47:0/1852848821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 msgr2=0x55f33ebd6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff675dc0740 1 --2- 172.21.15.47:0/1852848821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 0x55f33ebd6470 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff66002bf20 tx=0x7ff660046000 comp rx=0 tx=0).stop 2024-03-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff675dc0740 1 -- 172.21.15.47:0/1852848821 shutdown_connections 2024-03-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff675dc0740 1 --2- 172.21.15.47:0/1852848821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebffa80 0x55f33ebb1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff675dc0740 1 --2- 172.21.15.47:0/1852848821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 0x55f33ebd6470 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff675dc0740 1 --2- 172.21.15.47:0/1852848821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebb17d0 0x55f33ebe19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff675dc0740 1 -- 172.21.15.47:0/1852848821 >> 172.21.15.47:0/1852848821 conn(0x55f33ebadc70 msgr2=0x55f33eba8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.817+0000 7ff675dc0740 1 -- 172.21.15.47:0/1852848821 shutdown_connections 2024-03-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.818+0000 7ff675dc0740 1 -- 172.21.15.47:0/1852848821 wait complete. 2024-03-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.818+0000 7ff675dc0740 1 -- start start 2024-03-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.818+0000 7ff675dc0740 1 Processor -- start 2024-03-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff675dc0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebb17d0 0x55f33ebeec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff675dc0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 0x55f33ebb3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff675dc0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebffa80 0x55f33ebda2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff675dc0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33ebb2e20 con 0x55f33ebfb750 2024-03-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff675dc0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33ebef420 con 0x55f33ebb17d0 2024-03-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff675dc0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f33ebfd860 con 0x55f33ebffa80 2024-03-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff675dc0740 10 client.0 _ll_register_callbacks cb 0x7ff660076560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff66bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 0x55f33ebb3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff66bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 0x55f33ebb3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47684/0 (socket says 172.21.15.47:47684) 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff66bfff640 1 -- 172.21.15.47:0/3896263832 learned_addr learned my addr 172.21.15.47:0/3896263832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff66bfff640 1 -- 172.21.15.47:0/3896263832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebffa80 msgr2=0x55f33ebda2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff66bfff640 1 --2- 172.21.15.47:0/3896263832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebffa80 0x55f33ebda2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff66bfff640 1 -- 172.21.15.47:0/3896263832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebb17d0 msgr2=0x55f33ebeec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff66bfff640 1 --2- 172.21.15.47:0/3896263832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebb17d0 0x55f33ebeec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff66bfff640 1 -- 172.21.15.47:0/3896263832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f33eaa42f0 con 0x55f33ebfb750 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff66bfff640 1 --2- 172.21.15.47:0/3896263832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 0x55f33ebb3080 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff660076720 tx=0x7ff6600a9780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5e83b784ba5087 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.819+0000 7ff64b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff64b7fe640 1 -- 172.21.15.47:0/3896263832 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6600802b0 con 0x55f33ebfb750 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff64b7fe640 1 -- 172.21.15.47:0/3896263832 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff660096070 con 0x55f33ebfb750 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff675dc0740 1 -- 172.21.15.47:0/3896263832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff660082610 con 0x55f33ebfb750 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff64b7fe640 1 -- 172.21.15.47:0/3896263832 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff66007e050 con 0x55f33ebfb750 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff675dc0740 10 client.5391 fetch_fsmap learned FSMap version 12 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff675dc0740 1 -- 172.21.15.47:0/3896263832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x55f33eaddb40 con 0x55f33ebfb750 2024-03-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff64b7fe640 1 -- 172.21.15.47:0/3896263832 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff660082610 con 0x55f33ebfb750 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff64b7fe640 1 -- 172.21.15.47:0/3896263832 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 12) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff660082180 con 0x55f33ebfb750 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff675dc0740 10 client.5391 fetch_fsmap finished waiting for FSMap version 12 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff675dc0740 10 client.5391 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff675dc0740 20 client.5391 populate_metadata read hostname 'smithi047' 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff675dc0740 1 --2- 172.21.15.47:0/3896263832 >> [v2:172.21.15.47:6832/400299689,v1:172.21.15.47:6834/400299689] conn(0x55f33ec5eae0 0x55f33ebda810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff675dc0740 4 client.5391 mds_command: new command op to 5196 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff675dc0740 1 -- 172.21.15.47:0/3896263832 --> [v2:172.21.15.47:6832/400299689,v1:172.21.15.47:6834/400299689] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f33e86fd10 con 0x55f33ec5eae0 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff670fe6640 1 --2- 172.21.15.47:0/3896263832 >> [v2:172.21.15.47:6832/400299689,v1:172.21.15.47:6834/400299689] conn(0x55f33ec5eae0 0x55f33ebda810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff64b7fe640 10 client.5391 ms_handle_connect on v2:172.21.15.47:6832/400299689 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.820+0000 7ff670fe6640 1 --2- 172.21.15.47:0/3896263832 >> [v2:172.21.15.47:6832/400299689,v1:172.21.15.47:6834/400299689] conn(0x55f33ec5eae0 0x55f33ebda810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6052afbc2812f56 server_cookie=80ce510c4f1ee122 in_seq=0 out_seq=0 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff64b7fe640 1 -- 172.21.15.47:0/3896263832 <== mds.0 v2:172.21.15.47:6832/400299689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55f33e86fd10 con 0x55f33ec5eae0 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff64b7fe640 10 client.5391 handle_command_reply: tid=0 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff675dc0740 1 client.5391 shutdown 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff675dc0740 1 -- 172.21.15.47:0/3896263832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 msgr2=0x55f33ebb3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff675dc0740 1 --2- 172.21.15.47:0/3896263832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 0x55f33ebb3080 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff660076720 tx=0x7ff6600a9780 comp rx=0 tx=0).stop 2024-03-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff675dc0740 1 -- 172.21.15.47:0/3896263832 shutdown_connections 2024-03-23T09:42:08.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff675dc0740 1 --2- 172.21.15.47:0/3896263832 >> [v2:172.21.15.47:6832/400299689,v1:172.21.15.47:6834/400299689] conn(0x55f33ec5eae0 0x55f33ebda810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff675dc0740 1 --2- 172.21.15.47:0/3896263832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f33ebffa80 0x55f33ebda2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff675dc0740 1 --2- 172.21.15.47:0/3896263832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33ebfb750 0x55f33ebb3080 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff64b7fe640 0 client.5391 ms_handle_reset on v2:172.21.15.47:6832/400299689 2024-03-23T09:42:08.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff675dc0740 1 --2- 172.21.15.47:0/3896263832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f33ebb17d0 0x55f33ebeec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff675dc0740 1 -- 172.21.15.47:0/3896263832 >> 172.21.15.47:0/3896263832 conn(0x55f33ebadc70 msgr2=0x55f33ebd6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:08.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff675dc0740 1 -- 172.21.15.47:0/3896263832 shutdown_connections 2024-03-23T09:42:08.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff675dc0740 1 -- 172.21.15.47:0/3896263832 wait complete. 2024-03-23T09:42:08.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff675dc0740 20 client.5391 trim_cache size 0 max 16384 2024-03-23T09:42:08.828 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:42:08.828 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:42:08.828 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:42:08.829 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 5196, 2024-03-23T09:42:08.829 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:42:08.829 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:42:08.829 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:42:08.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 9.1014449640000006, 2024-03-23T09:42:08.829 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 12, 2024-03-23T09:42:08.829 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 34, 2024-03-23T09:42:08.829 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 34, 2024-03-23T09:42:08.829 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 15.457570950999999 2024-03-23T09:42:08.829 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:42:08.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff67324a640 1 -- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff65806ac00 msgr2=0x7ff65806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff67324a640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff65806ac00 0x7ff65806d0c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff664011340 tx=0x7ff66400b040 comp rx=0 tx=0).stop 2024-03-23T09:42:08.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff67324a640 1 -- 172.21.15.47:0/2805345382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff66c0d1020 msgr2=0x7ff66c1f2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.821+0000 7ff67324a640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff66c0d1020 0x7ff66c1f2900 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff660000c00 tx=0x7ff6600046c0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.822+0000 7ff67324a640 1 -- 172.21.15.47:0/2805345382 shutdown_connections 2024-03-23T09:42:08.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.822+0000 7ff67324a640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff65806ac00 0x7ff65806d0c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.822+0000 7ff67324a640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff66c15cd00 0x7ff66c15db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.822+0000 7ff67324a640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff66c0d1020 0x7ff66c1f2900 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.822+0000 7ff67324a640 1 --2- 172.21.15.47:0/2805345382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff66c0cf750 0x7ff66c1f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.822+0000 7ff67324a640 1 -- 172.21.15.47:0/2805345382 >> 172.21.15.47:0/2805345382 conn(0x7ff66c06b160 msgr2=0x7ff66c15bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:08.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.822+0000 7ff67324a640 1 -- 172.21.15.47:0/2805345382 shutdown_connections 2024-03-23T09:42:08.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.822+0000 7ff67324a640 1 -- 172.21.15.47:0/2805345382 wait complete. 2024-03-23T09:42:08.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:42:08.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.933+0000 7f956762c640 1 Processor -- start 2024-03-23T09:42:08.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.933+0000 7f956762c640 1 -- start start 2024-03-23T09:42:08.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f956762c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95600cf210 0x7f95600cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f956762c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95600b5a90 0x7f95600b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f956762c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95600cfbd0 con 0x7f956015cd00 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f956762c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956015d0e0 con 0x7f95600cf210 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f956762c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95600d1540 con 0x7f95600b5a90 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f9564bc7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95600cf210 0x7f95600cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f9564bc7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95600cf210 0x7f95600cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47688/0 (socket says 172.21.15.47:47688) 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f9564bc7640 1 -- 172.21.15.47:0/1086523516 learned_addr learned my addr 172.21.15.47:0/1086523516 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f9565bc9640 1 --2- 172.21.15.47:0/1086523516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95600b5a90 0x7f95600b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f9564bc7640 1 -- 172.21.15.47:0/1086523516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95600b5a90 msgr2=0x7f95600b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f9564bc7640 1 --2- 172.21.15.47:0/1086523516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95600b5a90 0x7f95600b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f9564bc7640 1 -- 172.21.15.47:0/1086523516 >> v1:172.21.15.175:6789/0 conn(0x7f956015cd00 legacy=0x7f95600d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f9564bc7640 1 -- 172.21.15.47:0/1086523516 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600d1820 con 0x7f95600cf210 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.934+0000 7f9565bc9640 1 --2- 172.21.15.47:0/1086523516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95600b5a90 0x7f95600b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.935+0000 7f9564bc7640 1 --2- 172.21.15.47:0/1086523516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95600cf210 0x7f95600cf690 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9550009870 tx=0x7f955002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e399052397e0625b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.935+0000 7f9557fff640 1 -- 172.21.15.47:0/1086523516 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95500317c0 con 0x7f95600cf210 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.935+0000 7f9557fff640 1 -- 172.21.15.47:0/1086523516 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9550031920 con 0x7f95600cf210 2024-03-23T09:42:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.935+0000 7f9557fff640 1 -- 172.21.15.47:0/1086523516 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550031cb0 con 0x7f95600cf210 2024-03-23T09:42:08.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.935+0000 7f956762c640 1 -- 172.21.15.47:0/1086523516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95600cf210 msgr2=0x7f95600cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.935+0000 7f956762c640 1 --2- 172.21.15.47:0/1086523516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95600cf210 0x7f95600cf690 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9550009870 tx=0x7f955002b860 comp rx=0 tx=0).stop 2024-03-23T09:42:08.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.935+0000 7f956762c640 1 -- 172.21.15.47:0/1086523516 shutdown_connections 2024-03-23T09:42:08.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.935+0000 7f956762c640 1 --2- 172.21.15.47:0/1086523516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95600b5a90 0x7f95600b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.935+0000 7f956762c640 1 --2- 172.21.15.47:0/1086523516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95600cf210 0x7f95600cf690 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.935+0000 7f956762c640 1 -- 172.21.15.47:0/1086523516 >> 172.21.15.47:0/1086523516 conn(0x7f956006b160 msgr2=0x7f956006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:08.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.936+0000 7f956762c640 1 -- 172.21.15.47:0/1086523516 shutdown_connections 2024-03-23T09:42:08.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.936+0000 7f956762c640 1 -- 172.21.15.47:0/1086523516 wait complete. 2024-03-23T09:42:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.937+0000 7f956762c640 1 Processor -- start 2024-03-23T09:42:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.937+0000 7f956762c640 1 -- start start 2024-03-23T09:42:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.937+0000 7f956762c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95600b5a90 0x7f95600cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.937+0000 7f956762c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95600cf210 0x7f9560157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.937+0000 7f956762c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f956015cd00 0x7f9560157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.937+0000 7f956762c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600cc180 con 0x7f956015cd00 2024-03-23T09:42:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.937+0000 7f956762c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95601f23e0 con 0x7f95600b5a90 2024-03-23T09:42:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.937+0000 7f956762c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9560158b60 con 0x7f95600cf210 2024-03-23T09:42:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.937+0000 7f9565bc9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f956015cd00 0x7f9560157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f9565bc9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f956015cd00 0x7f9560157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47696/0 (socket says 172.21.15.47:47696) 2024-03-23T09:42:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f9565bc9640 1 -- 172.21.15.47:0/3343183555 learned_addr learned my addr 172.21.15.47:0/3343183555 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f95653c8640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95600b5a90 0x7f95600cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f9564bc7640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95600cf210 0x7f9560157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f9565bc9640 1 -- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95600cf210 msgr2=0x7f9560157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f9565bc9640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95600cf210 0x7f9560157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f9565bc9640 1 -- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95600b5a90 msgr2=0x7f95600cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f9565bc9640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95600b5a90 0x7f95600cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f9565bc9640 1 -- 172.21.15.47:0/3343183555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95500094d0 con 0x7f956015cd00 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f9565bc9640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f956015cd00 0x7f9560157f70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f955800cef0 tx=0x7f955800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f9555ffb640 1 -- 172.21.15.47:0/3343183555 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955800d840 con 0x7f956015cd00 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f9555ffb640 1 -- 172.21.15.47:0/3343183555 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955800d9c0 con 0x7f956015cd00 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f956762c640 1 -- 172.21.15.47:0/3343183555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95601585a0 con 0x7f956015cd00 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f9555ffb640 1 -- 172.21.15.47:0/3343183555 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955800dd70 con 0x7f956015cd00 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.938+0000 7f956762c640 1 -- 172.21.15.47:0/3343183555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95600c9ad0 con 0x7f956015cd00 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.939+0000 7f956762c640 1 -- 172.21.15.47:0/3343183555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9560072360 con 0x7f956015cd00 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.940+0000 7f9555ffb640 1 -- 172.21.15.47:0/3343183555 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f955801e3b0 con 0x7f956015cd00 2024-03-23T09:42:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.940+0000 7f9555ffb640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f954406ac00 0x7f954406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:08.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.940+0000 7f9555ffb640 1 -- 172.21.15.47:0/3343183555 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9558089e30 con 0x7f956015cd00 2024-03-23T09:42:08.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.940+0000 7f95653c8640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f954406ac00 0x7f954406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:08.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.941+0000 7f95653c8640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f954406ac00 0x7f954406d0c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f955c006fd0 tx=0x7f955c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:08.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:08.944+0000 7f9555ffb640 1 -- 172.21.15.47:0/3343183555 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f95580550d0 con 0x7f956015cd00 2024-03-23T09:42:09.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.161+0000 7f956762c640 1 -- 172.21.15.47:0/3343183555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9560066be0 con 0x7f956015cd00 2024-03-23T09:42:09.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.161+0000 7f9555ffb640 1 -- 172.21.15.47:0/3343183555 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) v1 ==== 53+0+73 (secure 0 0 0) 0x7f955801a020 con 0x7f956015cd00 2024-03-23T09:42:09.162 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:42:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.163+0000 7f956762c640 1 -- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f954406ac00 msgr2=0x7f954406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.163+0000 7f956762c640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f954406ac00 0x7f954406d0c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f955c006fd0 tx=0x7f955c008040 comp rx=0 tx=0).stop 2024-03-23T09:42:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.163+0000 7f956762c640 1 -- 172.21.15.47:0/3343183555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f956015cd00 msgr2=0x7f9560157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.163+0000 7f956762c640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f956015cd00 0x7f9560157f70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f955800cef0 tx=0x7f955800a710 comp rx=0 tx=0).stop 2024-03-23T09:42:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.163+0000 7f956762c640 1 -- 172.21.15.47:0/3343183555 shutdown_connections 2024-03-23T09:42:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.163+0000 7f956762c640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f954406ac00 0x7f954406d0c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:09.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.163+0000 7f956762c640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f956015cd00 0x7f9560157f70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:09.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.163+0000 7f956762c640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95600cf210 0x7f9560157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:09.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.163+0000 7f956762c640 1 --2- 172.21.15.47:0/3343183555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95600b5a90 0x7f95600cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:09.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.163+0000 7f956762c640 1 -- 172.21.15.47:0/3343183555 >> 172.21.15.47:0/3343183555 conn(0x7f956006b160 msgr2=0x7f95600b7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:09.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.163+0000 7f956762c640 1 -- 172.21.15.47:0/3343183555 shutdown_connections 2024-03-23T09:42:09.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:09.163+0000 7f956762c640 1 -- 172.21.15.47:0/3343183555 wait complete. 2024-03-23T09:42:09.174 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:42:09.174 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:42:09.174 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:42:09.175 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:42:09.175 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:42:09.175 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:42:09.175 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:42:09.175 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:42:09.175 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:42:09.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:09.175 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:42:09.190 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:42:09.190 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:42:09.190 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:42:09.190 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:09.191 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:42:09.191 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:09.191 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:42:09.191 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:42:09.191 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:42:09.191 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:42:09.191 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:42:09.191 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:42:09.191 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:42:09.191 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:42:09.191 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:42:09.191 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:42:09.191 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42609sec preferred_lft 42609sec 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:42:09.192 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:42:09.193 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:09.193 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:42:09.193 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:42:09.193 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:42:09.193 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:09.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:09.193 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:42:09.255 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:42:09.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:09.257 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:42:09.313 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:42:09.313 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:42:09.313 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:42:09.313 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:42:09.313 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:42:09.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:09.313 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:42:09.314 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:42:09.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:42:09.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:42:09.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:42:09.314 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:42:09.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:09.409 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:42:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:42:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:42:09.423 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:42:09.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:09.423 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:42:09.490 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:42:09.490 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:42:09.490 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:42:09.545 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:42:09.545 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:42:09.545 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:42:09.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:42:09.666 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:42:09.666 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:42:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:42:09.733 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:42:09.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:09.733 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:42:09.788 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:42:09.788 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:42:09.829 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:42:09.855 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:42:09.856 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:42:09.914 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:42:09.913+0000 7f7e9c0da480 -1 init, newargv = 0x56535ac8fe80 newargc=15 2024-03-23T09:42:09.914 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[51227]: starting ceph client 2024-03-23T09:42:09.918 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[51227]: starting fuse 2024-03-23T09:42:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:42:09.925 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:42:09.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:09.926 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:42:09.981 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:42:09.981 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:42:09.981 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:42:09.981 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:42:09.981 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:42:09.981 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:42:09.982 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:42:09.983 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:42:09.983 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:42:09.983 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:42:09.983 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:42:09.983 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:42:09.983 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:09.983 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:42:09.983 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:42:10.097 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.51227.asok 2024-03-23T09:42:10.283 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51227.asok 2024-03-23T09:42:10.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:10.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51227.asok status 2024-03-23T09:42:10.420 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "51227", 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 5406, 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:42:10.421 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5406 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 265140687 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 265140687 2024-03-23T09:42:10.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:10.423 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.5406 192.168.0.1:0/265140687", 2024-03-23T09:42:10.423 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/265140687", 2024-03-23T09:42:10.423 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:42:10.423 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 12, 2024-03-23T09:42:10.423 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 34, 2024-03-23T09:42:10.423 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:42:10.423 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:42:10.423 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:42:10.423 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:42:10.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:10.429 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:42:10.429 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:42:10.444 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:42:10.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:10.444 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:42:10.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:10.511 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-03-23T09:42:10.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:10.571 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-03-23T09:42:10.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:10.628 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-03-23T09:42:10.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:10.684 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-03-23T09:42:10.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:10.742 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-03-23T09:42:10.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:10.798 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-23T09:42:10.861 INFO:teuthology.orchestra.run.smithi047.stdout:./ 2024-03-23T09:42:10.861 INFO:teuthology.orchestra.run.smithi047.stdout:./rfile 2024-03-23T09:42:10.861 INFO:teuthology.orchestra.run.smithi047.stdout:./parentdir 2024-03-23T09:42:10.861 INFO:teuthology.orchestra.run.smithi047.stdout:./parentdir/pfile 2024-03-23T09:42:10.861 INFO:teuthology.orchestra.run.smithi047.stdout:./parentdir/childdir 2024-03-23T09:42:10.861 INFO:teuthology.orchestra.run.smithi047.stdout:./parentdir/childdir/cfile 2024-03-23T09:42:10.862 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:42:10.862 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:42:10.862 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:42:10.862 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:10.862 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-23T09:42:10.862 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:42:10.962 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:42:11.152 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:42:11.153 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:42:11.153 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:42:11.153 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:11.153 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-03-23T09:42:11.153 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:42:11.254 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627778 2024-03-23T09:42:11.443 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:42:11.443 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:42:11.443 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:42:11.443 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:11.443 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-03-23T09:42:11.444 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:42:11.545 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627776 2024-03-23T09:42:11.734 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:42:11.734 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:42:11.735 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:42:11.735 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:11.735 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-03-23T09:42:11.735 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:42:11.836 INFO:teuthology.orchestra.run.smithi047.stdout:1099511628279 2024-03-23T09:42:12.025 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:42:12.025 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:42:12.026 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:42:12.026 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:12.026 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-03-23T09:42:12.026 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:42:12.127 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627777 2024-03-23T09:42:12.316 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:42:12.316 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:42:12.316 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:42:12.316 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:12.316 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-03-23T09:42:12.316 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:42:12.417 INFO:teuthology.orchestra.run.smithi047.stdout:1099511628280 2024-03-23T09:42:12.606 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:42:12.606 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:42:12.664 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:42:12.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.805+0000 7f02f173e640 1 Processor -- start 2024-03-23T09:42:12.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.805+0000 7f02f173e640 1 -- start start 2024-03-23T09:42:12.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02f173e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec15cb40 0x7f02ec160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:12.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02f173e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f02ec0b8300 0x7f02ec0cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:12.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02f173e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ec0d0ee0 con 0x7f02ec160cb0 2024-03-23T09:42:12.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02f173e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ec0b5a10 con 0x7f02ec0b8300 2024-03-23T09:42:12.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02f173e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec1616f0 con 0x7f02ec15cb40 2024-03-23T09:42:12.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02eb7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f02ec160cb0 0x7f02ec0b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58116/0 (socket says 172.21.15.47:58116) 2024-03-23T09:42:12.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02eb7fe640 1 -- 172.21.15.47:0/4155298807 learned_addr learned my addr 172.21.15.47:0/4155298807 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:12.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02eaffd640 1 --2- 172.21.15.47:0/4155298807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec15cb40 0x7f02ec160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:12.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02ea7fc640 1 --2- 172.21.15.47:0/4155298807 >> v2:172.21.15.175:3300/0 conn(0x7f02ec0b8300 0x7f02ec0cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:12.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604844124 0 0) 0x7f02ec0d0ee0 con 0x7f02ec160cb0 2024-03-23T09:42:12.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02d8003520 con 0x7f02ec160cb0 2024-03-23T09:42:12.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 317820539 0 0) 0x7f02d8003520 con 0x7f02ec160cb0 2024-03-23T09:42:12.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec15cb40 msgr2=0x7f02ec160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:12.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02e9ffb640 1 --2- 172.21.15.47:0/4155298807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec15cb40 0x7f02ec160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:12.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 >> v2:172.21.15.175:3300/0 conn(0x7f02ec0b8300 msgr2=0x7f02ec0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:12.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02e9ffb640 1 --2- 172.21.15.47:0/4155298807 >> v2:172.21.15.175:3300/0 conn(0x7f02ec0b8300 0x7f02ec0cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec161130 con 0x7f02ec160cb0 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.806+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f02dc0019f0 con 0x7f02ec160cb0 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 >> v1:172.21.15.47:6789/0 conn(0x7f02ec160cb0 legacy=0x7f02ec0b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02e9ffb640 1 --2- 172.21.15.47:0/4155298807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec15cb40 0x7f02d8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02e9ffb640 1 --2- 172.21.15.47:0/4155298807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02d8003ef0 0x7f02d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02e9ffb640 1 --2- 172.21.15.47:0/4155298807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d8004d00 0x7f02d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec0b5a10 con 0x7f02d8003ef0 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec1616f0 con 0x7f02d8004d00 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d800b980 con 0x7f02ec15cb40 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02ea7fc640 1 --2- 172.21.15.47:0/4155298807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02d8003ef0 0x7f02d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02eaffd640 1 --2- 172.21.15.47:0/4155298807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec15cb40 0x7f02d8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02eb7fe640 1 --2- 172.21.15.47:0/4155298807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d8004d00 0x7f02d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02ea7fc640 1 -- 172.21.15.47:0/4155298807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec15cb40 msgr2=0x7f02d8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:12.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02ea7fc640 1 --2- 172.21.15.47:0/4155298807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec15cb40 0x7f02d8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:12.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02ea7fc640 1 -- 172.21.15.47:0/4155298807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d8004d00 msgr2=0x7f02d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:12.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02ea7fc640 1 --2- 172.21.15.47:0/4155298807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d8004d00 0x7f02d8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:12.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02ea7fc640 1 -- 172.21.15.47:0/4155298807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d8003520 con 0x7f02d8003ef0 2024-03-23T09:42:12.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.807+0000 7f02eb7fe640 1 --2- 172.21.15.47:0/4155298807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d8004d00 0x7f02d8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:12.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.808+0000 7f02ea7fc640 1 --2- 172.21.15.47:0/4155298807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02d8003ef0 0x7f02d80047c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009a90 tx=0x7f02d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56ab2556a1cc907 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:12.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.808+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d402c6b0 con 0x7f02d8003ef0 2024-03-23T09:42:12.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.808+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d402c810 con 0x7f02d8003ef0 2024-03-23T09:42:12.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.808+0000 7f02e9ffb640 1 -- 172.21.15.47:0/4155298807 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d402cbc0 con 0x7f02d8003ef0 2024-03-23T09:42:12.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.808+0000 7f02f173e640 1 -- 172.21.15.47:0/4155298807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02d8003ef0 msgr2=0x7f02d80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:12.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.808+0000 7f02f173e640 1 --2- 172.21.15.47:0/4155298807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02d8003ef0 0x7f02d80047c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009a90 tx=0x7f02d80057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:12.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.808+0000 7f02eb7fe640 1 -- 172.21.15.47:0/4155298807 reap_dead start 2024-03-23T09:42:12.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.808+0000 7f02f173e640 1 -- 172.21.15.47:0/4155298807 shutdown_connections 2024-03-23T09:42:12.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.808+0000 7f02f173e640 1 -- 172.21.15.47:0/4155298807 >> 172.21.15.47:0/4155298807 conn(0x7f02ec06b160 msgr2=0x7f02ec06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:12.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.808+0000 7f02f173e640 1 -- 172.21.15.47:0/4155298807 shutdown_connections 2024-03-23T09:42:12.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.808+0000 7f02f173e640 1 -- 172.21.15.47:0/4155298807 wait complete. 2024-03-23T09:42:12.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.810+0000 7f02f173e640 1 Processor -- start 2024-03-23T09:42:12.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.810+0000 7f02f173e640 1 -- start start 2024-03-23T09:42:12.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.810+0000 7f02f173e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02d8003ef0 0x7f02ec0b7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:12.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.810+0000 7f02f173e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec15d6f0 0x7f02ec0c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:12.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02f173e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec15db50 0x7f02ec0c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02f173e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec145df0 con 0x7f02d8003ef0 2024-03-23T09:42:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02f173e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec15a580 con 0x7f02ec15d6f0 2024-03-23T09:42:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02f173e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec0b7b90 con 0x7f02ec15db50 2024-03-23T09:42:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02eaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02d8003ef0 0x7f02ec0b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02eaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02d8003ef0 0x7f02ec0b7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47724/0 (socket says 172.21.15.47:47724) 2024-03-23T09:42:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02eaffd640 1 -- 172.21.15.47:0/1643543618 learned_addr learned my addr 172.21.15.47:0/1643543618 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02eb7fe640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec15db50 0x7f02ec0c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02ea7fc640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec15d6f0 0x7f02ec0c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02eaffd640 1 -- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec15db50 msgr2=0x7f02ec0c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02eaffd640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec15db50 0x7f02ec0c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02eaffd640 1 -- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec15d6f0 msgr2=0x7f02ec0c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02eaffd640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec15d6f0 0x7f02ec0c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02eaffd640 1 -- 172.21.15.47:0/1643543618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec15a0f0 con 0x7f02d8003ef0 2024-03-23T09:42:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02eaffd640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02d8003ef0 0x7f02ec0b7650 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f02e40119d0 tx=0x7f02e4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02e37fe640 1 -- 172.21.15.47:0/1643543618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400fb50 con 0x7f02d8003ef0 2024-03-23T09:42:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02e37fe640 1 -- 172.21.15.47:0/1643543618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e400fcb0 con 0x7f02d8003ef0 2024-03-23T09:42:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02f173e640 1 -- 172.21.15.47:0/1643543618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ec0cca10 con 0x7f02d8003ef0 2024-03-23T09:42:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02e37fe640 1 -- 172.21.15.47:0/1643543618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4014440 con 0x7f02d8003ef0 2024-03-23T09:42:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.811+0000 7f02f173e640 1 -- 172.21.15.47:0/1643543618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ec159590 con 0x7f02d8003ef0 2024-03-23T09:42:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.812+0000 7f02f173e640 1 -- 172.21.15.47:0/1643543618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ec072360 con 0x7f02d8003ef0 2024-03-23T09:42:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.813+0000 7f02e37fe640 1 -- 172.21.15.47:0/1643543618 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f02e401d400 con 0x7f02d8003ef0 2024-03-23T09:42:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.813+0000 7f02e37fe640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f02cc06ac30 0x7f02cc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.813+0000 7f02e37fe640 1 -- 172.21.15.47:0/1643543618 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02e4089ab0 con 0x7f02d8003ef0 2024-03-23T09:42:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.813+0000 7f02ea7fc640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f02cc06ac30 0x7f02cc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.814+0000 7f02ea7fc640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f02cc06ac30 0x7f02cc06d0f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f02d4036000 tx=0x7f02d4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.817+0000 7f02e37fe640 1 -- 172.21.15.47:0/1643543618 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f02e4054d20 con 0x7f02d8003ef0 2024-03-23T09:42:12.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:12.999+0000 7f02f173e640 1 -- 172.21.15.47:0/1643543618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02ec066be0 con 0x7f02d8003ef0 2024-03-23T09:42:13.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.000+0000 7f02e37fe640 1 -- 172.21.15.47:0/1643543618 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f02e4058340 con 0x7f02d8003ef0 2024-03-23T09:42:13.000 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:13.001 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:00.721483+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:58.297353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:58.685375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:42:13.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.001+0000 7f02f173e640 1 -- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f02cc06ac30 msgr2=0x7f02cc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.001+0000 7f02f173e640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f02cc06ac30 0x7f02cc06d0f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f02d4036000 tx=0x7f02d4030000 comp rx=0 tx=0).stop 2024-03-23T09:42:13.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.001+0000 7f02f173e640 1 -- 172.21.15.47:0/1643543618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02d8003ef0 msgr2=0x7f02ec0b7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.001+0000 7f02f173e640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02d8003ef0 0x7f02ec0b7650 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f02e40119d0 tx=0x7f02e4011e90 comp rx=0 tx=0).stop 2024-03-23T09:42:13.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.001+0000 7f02f173e640 1 -- 172.21.15.47:0/1643543618 shutdown_connections 2024-03-23T09:42:13.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.001+0000 7f02f173e640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f02cc06ac30 0x7f02cc06d0f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.001+0000 7f02f173e640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec15db50 0x7f02ec0c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.001+0000 7f02f173e640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec15d6f0 0x7f02ec0c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.001+0000 7f02f173e640 1 --2- 172.21.15.47:0/1643543618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02d8003ef0 0x7f02ec0b7650 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.001+0000 7f02f173e640 1 -- 172.21.15.47:0/1643543618 >> 172.21.15.47:0/1643543618 conn(0x7f02ec06b160 msgr2=0x7f02ec0b5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:13.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.002+0000 7f02f173e640 1 -- 172.21.15.47:0/1643543618 shutdown_connections 2024-03-23T09:42:13.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.002+0000 7f02f173e640 1 -- 172.21.15.47:0/1643543618 wait complete. 2024-03-23T09:42:13.013 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:58.297353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:58.685375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:42:13.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:42:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.117+0000 7f3a48abd640 1 Processor -- start 2024-03-23T09:42:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.117+0000 7f3a48abd640 1 -- start start 2024-03-23T09:42:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.118+0000 7f3a48abd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3a440cdcb0 0x7f3a440b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.118+0000 7f3a48abd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a440d2020 0x7f3a440b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.118+0000 7f3a48abd640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a440b72f0 con 0x7f3a4415cb50 2024-03-23T09:42:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.118+0000 7f3a48abd640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a440b7e70 con 0x7f3a440cdcb0 2024-03-23T09:42:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.118+0000 7f3a48abd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4415cf30 con 0x7f3a440d2020 2024-03-23T09:42:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.118+0000 7f3a42d9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a440d2020 0x7f3a440b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.118+0000 7f3a41d9b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3a440cdcb0 0x7f3a440b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.118+0000 7f3a4259c640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f3a4415cb50 0x7f3a440cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:41406/0 (socket says 172.21.15.47:41406) 2024-03-23T09:42:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.118+0000 7f3a4259c640 1 -- 172.21.15.47:0/1200927549 learned_addr learned my addr 172.21.15.47:0/1200927549 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.119+0000 7f3a4159a640 1 -- 172.21.15.47:0/1200927549 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4259856150 0 0) 0x7f3a440b72f0 con 0x7f3a4415cb50 2024-03-23T09:42:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.119+0000 7f3a4159a640 1 -- 172.21.15.47:0/1200927549 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a2c003520 con 0x7f3a4415cb50 2024-03-23T09:42:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.119+0000 7f3a41d9b640 1 -- 172.21.15.47:0/1200927549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a440d2020 msgr2=0x7f3a440b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:13.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.119+0000 7f3a41d9b640 1 --2- 172.21.15.47:0/1200927549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a440d2020 0x7f3a440b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.119+0000 7f3a41d9b640 1 -- 172.21.15.47:0/1200927549 >> v1:172.21.15.175:6789/0 conn(0x7f3a4415cb50 legacy=0x7f3a440cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.119+0000 7f3a41d9b640 1 -- 172.21.15.47:0/1200927549 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a440b75d0 con 0x7f3a440cdcb0 2024-03-23T09:42:13.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.120+0000 7f3a41d9b640 1 --2- 172.21.15.47:0/1200927549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a440cdcb0 0x7f3a440b84a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3a380097a0 tx=0x7f3a3802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64e425ac693b9773 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:13.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.120+0000 7f3a4159a640 1 -- 172.21.15.47:0/1200927549 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38030610 con 0x7f3a440cdcb0 2024-03-23T09:42:13.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.120+0000 7f3a4159a640 1 -- 172.21.15.47:0/1200927549 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a38030770 con 0x7f3a440cdcb0 2024-03-23T09:42:13.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.120+0000 7f3a4159a640 1 -- 172.21.15.47:0/1200927549 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38030b00 con 0x7f3a440cdcb0 2024-03-23T09:42:13.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.120+0000 7f3a48abd640 1 -- 172.21.15.47:0/1200927549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a440cdcb0 msgr2=0x7f3a440b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.120+0000 7f3a48abd640 1 --2- 172.21.15.47:0/1200927549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a440cdcb0 0x7f3a440b84a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3a380097a0 tx=0x7f3a3802b960 comp rx=0 tx=0).stop 2024-03-23T09:42:13.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.120+0000 7f3a48abd640 1 -- 172.21.15.47:0/1200927549 shutdown_connections 2024-03-23T09:42:13.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.120+0000 7f3a48abd640 1 --2- 172.21.15.47:0/1200927549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a440d2020 0x7f3a440b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.120+0000 7f3a48abd640 1 --2- 172.21.15.47:0/1200927549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a440cdcb0 0x7f3a440b84a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.120+0000 7f3a48abd640 1 -- 172.21.15.47:0/1200927549 >> 172.21.15.47:0/1200927549 conn(0x7f3a44065df0 msgr2=0x7f3a44066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.120+0000 7f3a48abd640 1 -- 172.21.15.47:0/1200927549 shutdown_connections 2024-03-23T09:42:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.120+0000 7f3a48abd640 1 -- 172.21.15.47:0/1200927549 wait complete. 2024-03-23T09:42:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.122+0000 7f3a48abd640 1 Processor -- start 2024-03-23T09:42:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.122+0000 7f3a48abd640 1 -- start start 2024-03-23T09:42:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.122+0000 7f3a48abd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a440cdcb0 0x7f3a440b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.122+0000 7f3a48abd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a440d2020 0x7f3a440ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.122+0000 7f3a48abd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a4415cb50 0x7f3a440ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.122+0000 7f3a48abd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a440b1160 con 0x7f3a4415cb50 2024-03-23T09:42:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.122+0000 7f3a48abd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4415ef80 con 0x7f3a440cdcb0 2024-03-23T09:42:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.122+0000 7f3a48abd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a440caf20 con 0x7f3a440d2020 2024-03-23T09:42:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a42d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a4415cb50 0x7f3a440ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a42d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a4415cb50 0x7f3a440ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47738/0 (socket says 172.21.15.47:47738) 2024-03-23T09:42:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a42d9d640 1 -- 172.21.15.47:0/2707883940 learned_addr learned my addr 172.21.15.47:0/2707883940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a4259c640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a440cdcb0 0x7f3a440b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a41d9b640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a440d2020 0x7f3a440ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a42d9d640 1 -- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a440d2020 msgr2=0x7f3a440ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a42d9d640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a440d2020 0x7f3a440ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a42d9d640 1 -- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a440cdcb0 msgr2=0x7f3a440b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a42d9d640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a440cdcb0 0x7f3a440b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a42d9d640 1 -- 172.21.15.47:0/2707883940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a38009400 con 0x7f3a4415cb50 2024-03-23T09:42:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a42d9d640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a4415cb50 0x7f3a440ca8b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400efc0 tx=0x7f3a3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a32ffd640 1 -- 172.21.15.47:0/2707883940 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34009ca0 con 0x7f3a4415cb50 2024-03-23T09:42:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a32ffd640 1 -- 172.21.15.47:0/2707883940 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3400f040 con 0x7f3a4415cb50 2024-03-23T09:42:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a48abd640 1 -- 172.21.15.47:0/2707883940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a440cb200 con 0x7f3a4415cb50 2024-03-23T09:42:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a32ffd640 1 -- 172.21.15.47:0/2707883940 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a340146c0 con 0x7f3a4415cb50 2024-03-23T09:42:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.123+0000 7f3a48abd640 1 -- 172.21.15.47:0/2707883940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a440cb7c0 con 0x7f3a4415cb50 2024-03-23T09:42:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.124+0000 7f3a48abd640 1 -- 172.21.15.47:0/2707883940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a44072360 con 0x7f3a4415cb50 2024-03-23T09:42:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.125+0000 7f3a32ffd640 1 -- 172.21.15.47:0/2707883940 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3a34014820 con 0x7f3a4415cb50 2024-03-23T09:42:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.125+0000 7f3a32ffd640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a2406ac00 0x7f3a2406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.125+0000 7f3a32ffd640 1 -- 172.21.15.47:0/2707883940 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a3408a1d0 con 0x7f3a4415cb50 2024-03-23T09:42:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.126+0000 7f3a4259c640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a2406ac00 0x7f3a2406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.126+0000 7f3a4259c640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a2406ac00 0x7f3a2406d0c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0011e0 tx=0x7f3a3c007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.130+0000 7f3a32ffd640 1 -- 172.21.15.47:0/2707883940 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3a340553c0 con 0x7f3a4415cb50 2024-03-23T09:42:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.307+0000 7f3a48abd640 1 -- 172.21.15.47:0/2707883940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a440b7b50 con 0x7f3a4415cb50 2024-03-23T09:42:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.308+0000 7f3a32ffd640 1 -- 172.21.15.47:0/2707883940 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a340589e0 con 0x7f3a4415cb50 2024-03-23T09:42:13.307 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:42:13.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.309+0000 7f3a48abd640 1 -- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a2406ac00 msgr2=0x7f3a2406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.309+0000 7f3a48abd640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a2406ac00 0x7f3a2406d0c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0011e0 tx=0x7f3a3c007d30 comp rx=0 tx=0).stop 2024-03-23T09:42:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.309+0000 7f3a48abd640 1 -- 172.21.15.47:0/2707883940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a4415cb50 msgr2=0x7f3a440ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.309+0000 7f3a48abd640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a4415cb50 0x7f3a440ca8b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400efc0 tx=0x7f3a3400c490 comp rx=0 tx=0).stop 2024-03-23T09:42:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.309+0000 7f3a48abd640 1 -- 172.21.15.47:0/2707883940 shutdown_connections 2024-03-23T09:42:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.309+0000 7f3a48abd640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a2406ac00 0x7f3a2406d0c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.309+0000 7f3a48abd640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a4415cb50 0x7f3a440ca8b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.309+0000 7f3a48abd640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a440d2020 0x7f3a440ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.309+0000 7f3a48abd640 1 --2- 172.21.15.47:0/2707883940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a440cdcb0 0x7f3a440b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.309+0000 7f3a48abd640 1 -- 172.21.15.47:0/2707883940 >> 172.21.15.47:0/2707883940 conn(0x7f3a44065df0 msgr2=0x7f3a4415bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.309+0000 7f3a48abd640 1 -- 172.21.15.47:0/2707883940 shutdown_connections 2024-03-23T09:42:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.309+0000 7f3a48abd640 1 -- 172.21.15.47:0/2707883940 wait complete. 2024-03-23T09:42:13.319 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:42:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.424+0000 7f88e90f3640 1 Processor -- start 2024-03-23T09:42:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.424+0000 7f88e90f3640 1 -- start start 2024-03-23T09:42:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.425+0000 7f88e90f3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88e40d2f80 0x7f88e41678a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.425+0000 7f88e90f3640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e40d10a0 con 0x7f88e40d23a0 2024-03-23T09:42:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.425+0000 7f88e90f3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e40d1300 con 0x7f88e415cee0 2024-03-23T09:42:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.425+0000 7f88e90f3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e40d15e0 con 0x7f88e40d2f80 2024-03-23T09:42:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.425+0000 7f88e359e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88e40d2f80 0x7f88e41678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.425+0000 7f88e259c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f88e40d23a0 0x7f88e40d2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58122/0 (socket says 172.21.15.47:58122) 2024-03-23T09:42:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.425+0000 7f88e259c640 1 -- 172.21.15.47:0/2416540763 learned_addr learned my addr 172.21.15.47:0/2416540763 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.425+0000 7f88e359e640 1 -- 172.21.15.47:0/2416540763 >> v1:172.21.15.175:6789/0 conn(0x7f88e415cee0 legacy=0x7f88e40d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.425+0000 7f88e359e640 1 -- 172.21.15.47:0/2416540763 >> v1:172.21.15.47:6789/0 conn(0x7f88e40d23a0 legacy=0x7f88e40d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.425+0000 7f88e359e640 1 -- 172.21.15.47:0/2416540763 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e40d18c0 con 0x7f88e40d2f80 2024-03-23T09:42:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.426+0000 7f88e359e640 1 --2- 172.21.15.47:0/2416540763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40d2f80 0x7f88e41678a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009d90 tx=0x7f88d402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3316b043e0c2775c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.426+0000 7f88e1d9b640 1 -- 172.21.15.47:0/2416540763 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d402cc60 con 0x7f88e40d2f80 2024-03-23T09:42:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.426+0000 7f88e1d9b640 1 -- 172.21.15.47:0/2416540763 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d402cdc0 con 0x7f88e40d2f80 2024-03-23T09:42:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.426+0000 7f88e1d9b640 1 -- 172.21.15.47:0/2416540763 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4034660 con 0x7f88e40d2f80 2024-03-23T09:42:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.427+0000 7f88e90f3640 1 -- 172.21.15.47:0/2416540763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40d2f80 msgr2=0x7f88e41678a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.427+0000 7f88e90f3640 1 --2- 172.21.15.47:0/2416540763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40d2f80 0x7f88e41678a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009d90 tx=0x7f88d402beb0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.427+0000 7f88e90f3640 1 -- 172.21.15.47:0/2416540763 shutdown_connections 2024-03-23T09:42:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.427+0000 7f88e90f3640 1 --2- 172.21.15.47:0/2416540763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40d2f80 0x7f88e41678a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.427+0000 7f88e90f3640 1 -- 172.21.15.47:0/2416540763 >> 172.21.15.47:0/2416540763 conn(0x7f88e4065df0 msgr2=0x7f88e4066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:13.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.427+0000 7f88e90f3640 1 -- 172.21.15.47:0/2416540763 shutdown_connections 2024-03-23T09:42:13.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.427+0000 7f88e90f3640 1 -- 172.21.15.47:0/2416540763 wait complete. 2024-03-23T09:42:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.428+0000 7f88e90f3640 1 Processor -- start 2024-03-23T09:42:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.428+0000 7f88e90f3640 1 -- start start 2024-03-23T09:42:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e90f3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88e40d23a0 0x7f88e40b6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e90f3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40d2f80 0x7f88e4149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e90f3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88e415cee0 0x7f88e40c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e90f3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e40cc5e0 con 0x7f88e40d2f80 2024-03-23T09:42:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e90f3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4149b80 con 0x7f88e415cee0 2024-03-23T09:42:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e90f3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e40cbcb0 con 0x7f88e40d23a0 2024-03-23T09:42:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40d2f80 0x7f88e4149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40d2f80 0x7f88e4149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47752/0 (socket says 172.21.15.47:47752) 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e259c640 1 -- 172.21.15.47:0/3247828255 learned_addr learned my addr 172.21.15.47:0/3247828255 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e2d9d640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88e40d23a0 0x7f88e40b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e359e640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88e415cee0 0x7f88e40c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e259c640 1 -- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88e40d23a0 msgr2=0x7f88e40b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e259c640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88e40d23a0 0x7f88e40b6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e259c640 1 -- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88e415cee0 msgr2=0x7f88e40c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e259c640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88e415cee0 0x7f88e40c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e259c640 1 -- 172.21.15.47:0/3247828255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d4009a70 con 0x7f88e40d2f80 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e2d9d640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88e40d23a0 0x7f88e40b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.429+0000 7f88e359e640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88e415cee0 0x7f88e40c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.430+0000 7f88e259c640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40d2f80 0x7f88e4149640 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f88d800c3c0 tx=0x7f88d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.430+0000 7f88cb7fe640 1 -- 172.21.15.47:0/3247828255 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8005ce0 con 0x7f88e40d2f80 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.430+0000 7f88cb7fe640 1 -- 172.21.15.47:0/3247828255 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d8005e40 con 0x7f88e40d2f80 2024-03-23T09:42:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.430+0000 7f88e90f3640 1 -- 172.21.15.47:0/3247828255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e40cbf90 con 0x7f88e40d2f80 2024-03-23T09:42:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.430+0000 7f88cb7fe640 1 -- 172.21.15.47:0/3247828255 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d80072d0 con 0x7f88e40d2f80 2024-03-23T09:42:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.430+0000 7f88e90f3640 1 -- 172.21.15.47:0/3247828255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e414a540 con 0x7f88e40d2f80 2024-03-23T09:42:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.431+0000 7f88e90f3640 1 -- 172.21.15.47:0/3247828255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e4072360 con 0x7f88e40d2f80 2024-03-23T09:42:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.431+0000 7f88cb7fe640 1 -- 172.21.15.47:0/3247828255 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f88d80074f0 con 0x7f88e40d2f80 2024-03-23T09:42:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.432+0000 7f88cb7fe640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f88d006ac00 0x7f88d006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.432+0000 7f88cb7fe640 1 -- 172.21.15.47:0/3247828255 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88d80887a0 con 0x7f88e40d2f80 2024-03-23T09:42:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.432+0000 7f88e2d9d640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f88d006ac00 0x7f88d006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.432+0000 7f88e2d9d640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f88d006ac00 0x7f88d006d0c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f88dc003eb0 tx=0x7f88dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.436+0000 7f88cb7fe640 1 -- 172.21.15.47:0/3247828255 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f88d8053a30 con 0x7f88e40d2f80 2024-03-23T09:42:13.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.616+0000 7f88e90f3640 1 -- 172.21.15.47:0/3247828255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88e4066be0 con 0x7f88e40d2f80 2024-03-23T09:42:13.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.616+0000 7f88cb7fe640 1 -- 172.21.15.47:0/3247828255 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f88d8017020 con 0x7f88e40d2f80 2024-03-23T09:42:13.616 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:42:13.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.618+0000 7f88e90f3640 1 -- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f88d006ac00 msgr2=0x7f88d006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.618+0000 7f88e90f3640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f88d006ac00 0x7f88d006d0c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f88dc003eb0 tx=0x7f88dc009040 comp rx=0 tx=0).stop 2024-03-23T09:42:13.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.618+0000 7f88e90f3640 1 -- 172.21.15.47:0/3247828255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40d2f80 msgr2=0x7f88e4149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.618+0000 7f88e90f3640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40d2f80 0x7f88e4149640 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f88d800c3c0 tx=0x7f88d800c880 comp rx=0 tx=0).stop 2024-03-23T09:42:13.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.618+0000 7f88e90f3640 1 -- 172.21.15.47:0/3247828255 shutdown_connections 2024-03-23T09:42:13.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.618+0000 7f88e90f3640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f88d006ac00 0x7f88d006d0c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.618+0000 7f88e90f3640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88e415cee0 0x7f88e40c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.618+0000 7f88e90f3640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40d2f80 0x7f88e4149640 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.618+0000 7f88e90f3640 1 --2- 172.21.15.47:0/3247828255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88e40d23a0 0x7f88e40b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.618+0000 7f88e90f3640 1 -- 172.21.15.47:0/3247828255 >> 172.21.15.47:0/3247828255 conn(0x7f88e4065df0 msgr2=0x7f88e40b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:13.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.618+0000 7f88e90f3640 1 -- 172.21.15.47:0/3247828255 shutdown_connections 2024-03-23T09:42:13.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.618+0000 7f88e90f3640 1 -- 172.21.15.47:0/3247828255 wait complete. 2024-03-23T09:42:13.629 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:42:13.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.732+0000 7f67757c3640 1 Processor -- start 2024-03-23T09:42:13.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.732+0000 7f67757c3640 1 -- start start 2024-03-23T09:42:13.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.733+0000 7f67757c3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f677015ca30 0x7f6770160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.733+0000 7f67757c3640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f67700cde40 0x7f67700d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.733+0000 7f67757c3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6770161600 con 0x7f677015ce10 2024-03-23T09:42:13.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.733+0000 7f67757c3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6770161040 con 0x7f677015ca30 2024-03-23T09:42:13.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.733+0000 7f67757c3640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6770161320 con 0x7f67700cde40 2024-03-23T09:42:13.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.733+0000 7f676effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f677015ca30 0x7f6770160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.733+0000 7f676effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f677015ca30 0x7f6770160740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47766/0 (socket says 172.21.15.47:47766) 2024-03-23T09:42:13.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.733+0000 7f676effd640 1 -- 172.21.15.47:0/1883630585 learned_addr learned my addr 172.21.15.47:0/1883630585 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:13.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.733+0000 7f676f7fe640 1 --2- 172.21.15.47:0/1883630585 >> v2:172.21.15.175:3300/0 conn(0x7f67700cde40 0x7f67700d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.733+0000 7f676effd640 1 -- 172.21.15.47:0/1883630585 >> v2:172.21.15.175:3300/0 conn(0x7f67700cde40 msgr2=0x7f67700d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:13.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.733+0000 7f676effd640 1 --2- 172.21.15.47:0/1883630585 >> v2:172.21.15.175:3300/0 conn(0x7f67700cde40 0x7f67700d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.733+0000 7f676effd640 1 -- 172.21.15.47:0/1883630585 >> v1:172.21.15.175:6789/0 conn(0x7f677015ce10 legacy=0x7f67700cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.733+0000 7f676effd640 1 -- 172.21.15.47:0/1883630585 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67700d0dd0 con 0x7f677015ca30 2024-03-23T09:42:13.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.734+0000 7f676effd640 1 --2- 172.21.15.47:0/1883630585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f677015ca30 0x7f6770160740 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6768009870 tx=0x7f676802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14be33a3d3683056 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:13.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.734+0000 7f676dffb640 1 -- 172.21.15.47:0/1883630585 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768031870 con 0x7f677015ca30 2024-03-23T09:42:13.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.734+0000 7f676dffb640 1 -- 172.21.15.47:0/1883630585 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67680319d0 con 0x7f677015ca30 2024-03-23T09:42:13.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.734+0000 7f676dffb640 1 -- 172.21.15.47:0/1883630585 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768031d60 con 0x7f677015ca30 2024-03-23T09:42:13.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.735+0000 7f67757c3640 1 -- 172.21.15.47:0/1883630585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f677015ca30 msgr2=0x7f6770160740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.735+0000 7f67757c3640 1 --2- 172.21.15.47:0/1883630585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f677015ca30 0x7f6770160740 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6768009870 tx=0x7f676802b910 comp rx=0 tx=0).stop 2024-03-23T09:42:13.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.735+0000 7f67757c3640 1 -- 172.21.15.47:0/1883630585 shutdown_connections 2024-03-23T09:42:13.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.735+0000 7f67757c3640 1 --2- 172.21.15.47:0/1883630585 >> v2:172.21.15.175:3300/0 conn(0x7f67700cde40 0x7f67700d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.735+0000 7f67757c3640 1 --2- 172.21.15.47:0/1883630585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f677015ca30 0x7f6770160740 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.735+0000 7f67757c3640 1 -- 172.21.15.47:0/1883630585 >> 172.21.15.47:0/1883630585 conn(0x7f677006b160 msgr2=0x7f677006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:13.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.735+0000 7f67757c3640 1 -- 172.21.15.47:0/1883630585 shutdown_connections 2024-03-23T09:42:13.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.735+0000 7f67757c3640 1 -- 172.21.15.47:0/1883630585 wait complete. 2024-03-23T09:42:13.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.736+0000 7f67757c3640 1 Processor -- start 2024-03-23T09:42:13.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.736+0000 7f67757c3640 1 -- start start 2024-03-23T09:42:13.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f67757c3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700cde40 0x7f677015dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f67757c3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015ca30 0x7f6770148ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f67757c3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f677015ce10 0x7f67700c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f67757c3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677015e2c0 con 0x7f677015ce10 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f67757c3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677015d2e0 con 0x7f677015ca30 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f67757c3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67700c8e90 con 0x7f67700cde40 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f676f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f677015ce10 0x7f67700c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f676f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f677015ce10 0x7f67700c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47772/0 (socket says 172.21.15.47:47772) 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f676f7fe640 1 -- 172.21.15.47:0/3358950641 learned_addr learned my addr 172.21.15.47:0/3358950641 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f676effd640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700cde40 0x7f677015dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f676e7fc640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015ca30 0x7f6770148ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f676f7fe640 1 -- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700cde40 msgr2=0x7f677015dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f676f7fe640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700cde40 0x7f677015dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f676f7fe640 1 -- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015ca30 msgr2=0x7f6770148ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f676f7fe640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015ca30 0x7f6770148ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f676f7fe640 1 -- 172.21.15.47:0/3358950641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67680094d0 con 0x7f677015ce10 2024-03-23T09:42:13.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f676effd640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700cde40 0x7f677015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:13.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.737+0000 7f676e7fc640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015ca30 0x7f6770148ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:13.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.738+0000 7f676f7fe640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f677015ce10 0x7f67700c8440 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f676000efc0 tx=0x7f676000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:13.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.738+0000 7f67677fe640 1 -- 172.21.15.47:0/3358950641 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760014480 con 0x7f677015ce10 2024-03-23T09:42:13.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.738+0000 7f67677fe640 1 -- 172.21.15.47:0/3358950641 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676000f040 con 0x7f677015ce10 2024-03-23T09:42:13.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.738+0000 7f67677fe640 1 -- 172.21.15.47:0/3358950641 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760014820 con 0x7f677015ce10 2024-03-23T09:42:13.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.738+0000 7f67757c3640 1 -- 172.21.15.47:0/3358950641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677015d580 con 0x7f677015ce10 2024-03-23T09:42:13.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.738+0000 7f67757c3640 1 -- 172.21.15.47:0/3358950641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67700c9170 con 0x7f677015ce10 2024-03-23T09:42:13.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.739+0000 7f67757c3640 1 -- 172.21.15.47:0/3358950641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6770072360 con 0x7f677015ce10 2024-03-23T09:42:13.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.739+0000 7f67677fe640 1 -- 172.21.15.47:0/3358950641 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6760014980 con 0x7f677015ce10 2024-03-23T09:42:13.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.740+0000 7f67677fe640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f675006ac00 0x7f675006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:13.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.740+0000 7f67677fe640 1 -- 172.21.15.47:0/3358950641 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f676008a200 con 0x7f677015ce10 2024-03-23T09:42:13.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.740+0000 7f676effd640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f675006ac00 0x7f675006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:13.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.740+0000 7f676effd640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f675006ac00 0x7f675006d0c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6768036000 tx=0x7f676803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:13.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.744+0000 7f67677fe640 1 -- 172.21.15.47:0/3358950641 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f67600553e0 con 0x7f677015ce10 2024-03-23T09:42:13.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.922+0000 7f67757c3640 1 -- 172.21.15.47:0/3358950641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f677015daa0 con 0x7f677015ce10 2024-03-23T09:42:13.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.922+0000 7f67677fe640 1 -- 172.21.15.47:0/3358950641 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f677015daa0 con 0x7f677015ce10 2024-03-23T09:42:13.922 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:42:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.924+0000 7f67757c3640 1 -- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f675006ac00 msgr2=0x7f675006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.924+0000 7f67757c3640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f675006ac00 0x7f675006d0c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6768036000 tx=0x7f676803c040 comp rx=0 tx=0).stop 2024-03-23T09:42:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.924+0000 7f67757c3640 1 -- 172.21.15.47:0/3358950641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f677015ce10 msgr2=0x7f67700c8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.924+0000 7f67757c3640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f677015ce10 0x7f67700c8440 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f676000efc0 tx=0x7f676000c490 comp rx=0 tx=0).stop 2024-03-23T09:42:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.924+0000 7f67757c3640 1 -- 172.21.15.47:0/3358950641 shutdown_connections 2024-03-23T09:42:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.924+0000 7f67757c3640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f675006ac00 0x7f675006d0c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.924+0000 7f67757c3640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f677015ce10 0x7f67700c8440 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.924+0000 7f67757c3640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015ca30 0x7f6770148ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.924+0000 7f67757c3640 1 --2- 172.21.15.47:0/3358950641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700cde40 0x7f677015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.924+0000 7f67757c3640 1 -- 172.21.15.47:0/3358950641 >> 172.21.15.47:0/3358950641 conn(0x7f677006b160 msgr2=0x7f67700c5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.924+0000 7f67757c3640 1 -- 172.21.15.47:0/3358950641 shutdown_connections 2024-03-23T09:42:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:13.924+0000 7f67757c3640 1 -- 172.21.15.47:0/3358950641 wait complete. 2024-03-23T09:42:13.935 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:42:14.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.036+0000 7fa53902c640 1 Processor -- start 2024-03-23T09:42:14.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.036+0000 7fa53902c640 1 -- start start 2024-03-23T09:42:14.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.037+0000 7fa53902c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa534098a00 0x7fa534098de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.037+0000 7fa53902c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa534160650 0x7fa53406adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.037+0000 7fa53902c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa534099320 con 0x7fa5340cee60 2024-03-23T09:42:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.037+0000 7fa53902c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53406b6f0 con 0x7fa534098a00 2024-03-23T09:42:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.037+0000 7fa53902c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53406b9d0 con 0x7fa534160650 2024-03-23T09:42:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.037+0000 7fa532d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa534098a00 0x7fa534098de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.037+0000 7fa532d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa534098a00 0x7fa534098de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47774/0 (socket says 172.21.15.47:47774) 2024-03-23T09:42:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.037+0000 7fa532d9d640 1 -- 172.21.15.47:0/3223255701 learned_addr learned my addr 172.21.15.47:0/3223255701 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.037+0000 7fa53259c640 1 --2- 172.21.15.47:0/3223255701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa534160650 0x7fa53406adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.037+0000 7fa532d9d640 1 -- 172.21.15.47:0/3223255701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa534160650 msgr2=0x7fa53406adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.037+0000 7fa532d9d640 1 --2- 172.21.15.47:0/3223255701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa534160650 0x7fa53406adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.037+0000 7fa532d9d640 1 -- 172.21.15.47:0/3223255701 >> v1:172.21.15.175:6789/0 conn(0x7fa5340cee60 legacy=0x7fa5340cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.037+0000 7fa532d9d640 1 -- 172.21.15.47:0/3223255701 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53406bcb0 con 0x7fa534098a00 2024-03-23T09:42:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.038+0000 7fa532d9d640 1 --2- 172.21.15.47:0/3223255701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa534098a00 0x7fa534098de0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009870 tx=0x7fa52c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99e57e24ec16ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.038+0000 7fa531d9b640 1 -- 172.21.15.47:0/3223255701 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c031640 con 0x7fa534098a00 2024-03-23T09:42:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.038+0000 7fa531d9b640 1 -- 172.21.15.47:0/3223255701 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c0317a0 con 0x7fa534098a00 2024-03-23T09:42:14.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.038+0000 7fa531d9b640 1 -- 172.21.15.47:0/3223255701 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c031b30 con 0x7fa534098a00 2024-03-23T09:42:14.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.039+0000 7fa53902c640 1 -- 172.21.15.47:0/3223255701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa534098a00 msgr2=0x7fa534098de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.039+0000 7fa53902c640 1 --2- 172.21.15.47:0/3223255701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa534098a00 0x7fa534098de0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009870 tx=0x7fa52c02b8f0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.039+0000 7fa53902c640 1 -- 172.21.15.47:0/3223255701 shutdown_connections 2024-03-23T09:42:14.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.039+0000 7fa53902c640 1 --2- 172.21.15.47:0/3223255701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa534160650 0x7fa53406adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.039+0000 7fa53902c640 1 --2- 172.21.15.47:0/3223255701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa534098a00 0x7fa534098de0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.039+0000 7fa53902c640 1 -- 172.21.15.47:0/3223255701 >> 172.21.15.47:0/3223255701 conn(0x7fa5340b2200 msgr2=0x7fa5340b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:14.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.039+0000 7fa53902c640 1 -- 172.21.15.47:0/3223255701 shutdown_connections 2024-03-23T09:42:14.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.039+0000 7fa53902c640 1 -- 172.21.15.47:0/3223255701 wait complete. 2024-03-23T09:42:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.040+0000 7fa53902c640 1 Processor -- start 2024-03-23T09:42:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.040+0000 7fa53902c640 1 -- start start 2024-03-23T09:42:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53902c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa534098a00 0x7fa5340b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53902c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5340cee60 0x7fa5340b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53902c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa534160650 0x7fa534159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53902c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340b5240 con 0x7fa5340cee60 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53902c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340b5450 con 0x7fa534160650 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53902c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340b5730 con 0x7fa534098a00 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5340cee60 0x7fa5340b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5340cee60 0x7fa5340b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47790/0 (socket says 172.21.15.47:47790) 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53259c640 1 -- 172.21.15.47:0/2586143091 learned_addr learned my addr 172.21.15.47:0/2586143091 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa532d9d640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa534098a00 0x7fa5340b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53359e640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa534160650 0x7fa534159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53259c640 1 -- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa534098a00 msgr2=0x7fa5340b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53259c640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa534098a00 0x7fa5340b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53259c640 1 -- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa534160650 msgr2=0x7fa534159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53259c640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa534160650 0x7fa534159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53259c640 1 -- 172.21.15.47:0/2586143091 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c0094d0 con 0x7fa5340cee60 2024-03-23T09:42:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa532d9d640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa534098a00 0x7fa5340b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.041+0000 7fa53259c640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5340cee60 0x7fa5340b4d00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa52800d5a0 tx=0x7fa52800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.042+0000 7fa51b7fe640 1 -- 172.21.15.47:0/2586143091 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5280125f0 con 0x7fa5340cee60 2024-03-23T09:42:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.042+0000 7fa51b7fe640 1 -- 172.21.15.47:0/2586143091 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa528012770 con 0x7fa5340cee60 2024-03-23T09:42:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.042+0000 7fa53902c640 1 -- 172.21.15.47:0/2586143091 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5340b5a10 con 0x7fa5340cee60 2024-03-23T09:42:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.042+0000 7fa51b7fe640 1 -- 172.21.15.47:0/2586143091 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528012b20 con 0x7fa5340cee60 2024-03-23T09:42:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.042+0000 7fa53902c640 1 -- 172.21.15.47:0/2586143091 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5340c7880 con 0x7fa5340cee60 2024-03-23T09:42:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.043+0000 7fa53902c640 1 -- 172.21.15.47:0/2586143091 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa534066be0 con 0x7fa5340cee60 2024-03-23T09:42:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.043+0000 7fa51b7fe640 1 -- 172.21.15.47:0/2586143091 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa528012c80 con 0x7fa5340cee60 2024-03-23T09:42:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.043+0000 7fa51b7fe640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa52006ac00 0x7fa52006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.044+0000 7fa51b7fe640 1 -- 172.21.15.47:0/2586143091 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5280898b0 con 0x7fa5340cee60 2024-03-23T09:42:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.044+0000 7fa532d9d640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa52006ac00 0x7fa52006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.044+0000 7fa532d9d640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa52006ac00 0x7fa52006d0c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa52c036000 tx=0x7fa52c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.048+0000 7fa51b7fe640 1 -- 172.21.15.47:0/2586143091 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa5280549f0 con 0x7fa5340cee60 2024-03-23T09:42:14.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.239+0000 7fa53902c640 1 -- 172.21.15.47:0/2586143091 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa53406c550 con 0x7fa5340cee60 2024-03-23T09:42:14.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.240+0000 7fa51b7fe640 1 -- 172.21.15.47:0/2586143091 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa528058010 con 0x7fa5340cee60 2024-03-23T09:42:14.240 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:42:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.242+0000 7fa53902c640 1 -- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa52006ac00 msgr2=0x7fa52006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.242+0000 7fa53902c640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa52006ac00 0x7fa52006d0c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa52c036000 tx=0x7fa52c03c040 comp rx=0 tx=0).stop 2024-03-23T09:42:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.242+0000 7fa53902c640 1 -- 172.21.15.47:0/2586143091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5340cee60 msgr2=0x7fa5340b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.242+0000 7fa53902c640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5340cee60 0x7fa5340b4d00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa52800d5a0 tx=0x7fa52800da60 comp rx=0 tx=0).stop 2024-03-23T09:42:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.242+0000 7fa53902c640 1 -- 172.21.15.47:0/2586143091 shutdown_connections 2024-03-23T09:42:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.242+0000 7fa53902c640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa52006ac00 0x7fa52006d0c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.242+0000 7fa53902c640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa534160650 0x7fa534159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.242+0000 7fa53902c640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5340cee60 0x7fa5340b4d00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.242+0000 7fa53902c640 1 --2- 172.21.15.47:0/2586143091 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa534098a00 0x7fa5340b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.242+0000 7fa53902c640 1 -- 172.21.15.47:0/2586143091 >> 172.21.15.47:0/2586143091 conn(0x7fa5340b2200 msgr2=0x7fa5340cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.242+0000 7fa53902c640 1 -- 172.21.15.47:0/2586143091 shutdown_connections 2024-03-23T09:42:14.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.242+0000 7fa53902c640 1 -- 172.21.15.47:0/2586143091 wait complete. 2024-03-23T09:42:14.252 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:42:14.252 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:42:14.267 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:42:14.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.408+0000 7f95d828b640 1 Processor -- start 2024-03-23T09:42:14.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.408+0000 7f95d828b640 1 -- start start 2024-03-23T09:42:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.409+0000 7f95d828b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f95d006a3f0 0x7f95d00cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.409+0000 7f95d828b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d0071920 con 0x7f95d015cfd0 2024-03-23T09:42:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.409+0000 7f95d828b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d015d4d0 con 0x7f95d0157150 2024-03-23T09:42:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.409+0000 7f95d828b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d0157530 con 0x7f95d006a3f0 2024-03-23T09:42:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.409+0000 7f95d5826640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f95d015cfd0 0x7f95d0069ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58130/0 (socket says 172.21.15.47:58130) 2024-03-23T09:42:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.409+0000 7f95d5826640 1 -- 172.21.15.47:0/903391053 learned_addr learned my addr 172.21.15.47:0/903391053 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d6828640 1 --2- 172.21.15.47:0/903391053 >> v2:172.21.15.175:3300/0 conn(0x7f95d006a3f0 0x7f95d00cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 505179984 0 0) 0x7f95d0071920 con 0x7f95d015cfd0 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b8003520 con 0x7f95d015cfd0 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4228834812 0 0) 0x7f95b8003520 con 0x7f95d015cfd0 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 >> v2:172.21.15.175:3300/0 conn(0x7f95d006a3f0 msgr2=0x7f95d00cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 --2- 172.21.15.47:0/903391053 >> v2:172.21.15.175:3300/0 conn(0x7f95d006a3f0 0x7f95d00cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 >> v1:172.21.15.175:6789/0 conn(0x7f95d0157150 legacy=0x7f95d0071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d015d7b0 con 0x7f95d015cfd0 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f95c4001670 con 0x7f95d015cfd0 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 >> v1:172.21.15.47:6789/0 conn(0x7f95d015cfd0 legacy=0x7f95d0069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 --2- 172.21.15.47:0/903391053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95b8003970 0x7f95b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 --2- 172.21.15.47:0/903391053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b80047c0 0x7f95b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 --2- 172.21.15.47:0/903391053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95b8005180 0x7f95b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0157530 con 0x7f95b80047c0 2024-03-23T09:42:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b800c020 con 0x7f95b8003970 2024-03-23T09:42:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b800c1e0 con 0x7f95b8005180 2024-03-23T09:42:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5826640 1 --2- 172.21.15.47:0/903391053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b80047c0 0x7f95b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d6027640 1 --2- 172.21.15.47:0/903391053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95b8003970 0x7f95b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d6828640 1 --2- 172.21.15.47:0/903391053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95b8005180 0x7f95b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5826640 1 -- 172.21.15.47:0/903391053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95b8005180 msgr2=0x7f95b800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5826640 1 --2- 172.21.15.47:0/903391053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95b8005180 0x7f95b800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5826640 1 -- 172.21.15.47:0/903391053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95b8003970 msgr2=0x7f95b8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5826640 1 --2- 172.21.15.47:0/903391053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95b8003970 0x7f95b8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d5826640 1 -- 172.21.15.47:0/903391053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b8003520 con 0x7f95b80047c0 2024-03-23T09:42:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d6828640 1 --2- 172.21.15.47:0/903391053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95b8005180 0x7f95b800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.410+0000 7f95d6828640 1 -- 172.21.15.47:0/903391053 reap_dead start 2024-03-23T09:42:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.411+0000 7f95d5826640 1 --2- 172.21.15.47:0/903391053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b80047c0 0x7f95b8004c40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c210 tx=0x7f95b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebf89d506e89702 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.411+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4005ad0 con 0x7f95b80047c0 2024-03-23T09:42:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.411+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c4005c30 con 0x7f95b80047c0 2024-03-23T09:42:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.411+0000 7f95d5025640 1 -- 172.21.15.47:0/903391053 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c402ca50 con 0x7f95b80047c0 2024-03-23T09:42:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.412+0000 7f95d828b640 1 -- 172.21.15.47:0/903391053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b80047c0 msgr2=0x7f95b8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.412+0000 7f95d828b640 1 --2- 172.21.15.47:0/903391053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b80047c0 0x7f95b8004c40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c210 tx=0x7f95b80057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.412+0000 7f95d828b640 1 -- 172.21.15.47:0/903391053 shutdown_connections 2024-03-23T09:42:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.412+0000 7f95d828b640 1 --2- 172.21.15.47:0/903391053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b80047c0 0x7f95b8004c40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.412+0000 7f95d828b640 1 -- 172.21.15.47:0/903391053 >> 172.21.15.47:0/903391053 conn(0x7f95d00b1410 msgr2=0x7f95d00b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.412+0000 7f95d828b640 1 -- 172.21.15.47:0/903391053 shutdown_connections 2024-03-23T09:42:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.412+0000 7f95d828b640 1 -- 172.21.15.47:0/903391053 wait complete. 2024-03-23T09:42:14.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.413+0000 7f95d828b640 1 Processor -- start 2024-03-23T09:42:14.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.413+0000 7f95d828b640 1 -- start start 2024-03-23T09:42:14.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d828b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95b80047c0 0x7f95d00c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d828b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95d00ad6a0 0x7f95d00c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d828b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d0153900 0x7f95d00ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d828b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00b3420 con 0x7f95d0153900 2024-03-23T09:42:14.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d828b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00ccc70 con 0x7f95d00ad6a0 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d828b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00b3840 con 0x7f95b80047c0 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d6828640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d0153900 0x7f95d00ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d6828640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d0153900 0x7f95d00ab870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47808/0 (socket says 172.21.15.47:47808) 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d6828640 1 -- 172.21.15.47:0/3316028446 learned_addr learned my addr 172.21.15.47:0/3316028446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d6027640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95b80047c0 0x7f95d00c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d5826640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95d00ad6a0 0x7f95d00c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d6828640 1 -- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95b80047c0 msgr2=0x7f95d00c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d6828640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95b80047c0 0x7f95d00c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d6828640 1 -- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95d00ad6a0 msgr2=0x7f95d00c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d6828640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95d00ad6a0 0x7f95d00c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d6828640 1 -- 172.21.15.47:0/3316028446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00b2500 con 0x7f95d0153900 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d5826640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95d00ad6a0 0x7f95d00c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95d6828640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d0153900 0x7f95d00ab870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f95d0161680 tx=0x7f95c00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:14.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.414+0000 7f95ceffd640 1 -- 172.21.15.47:0/3316028446 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c000e030 con 0x7f95d0153900 2024-03-23T09:42:14.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.415+0000 7f95ceffd640 1 -- 172.21.15.47:0/3316028446 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c0011db0 con 0x7f95d0153900 2024-03-23T09:42:14.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.415+0000 7f95d828b640 1 -- 172.21.15.47:0/3316028446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d00b27a0 con 0x7f95d0153900 2024-03-23T09:42:14.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.415+0000 7f95ceffd640 1 -- 172.21.15.47:0/3316028446 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c00166b0 con 0x7f95d0153900 2024-03-23T09:42:14.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.415+0000 7f95d828b640 1 -- 172.21.15.47:0/3316028446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d00b2d60 con 0x7f95d0153900 2024-03-23T09:42:14.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.415+0000 7f95d828b640 1 -- 172.21.15.47:0/3316028446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d00b3560 con 0x7f95d0153900 2024-03-23T09:42:14.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.416+0000 7f95ceffd640 1 -- 172.21.15.47:0/3316028446 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f95c0016810 con 0x7f95d0153900 2024-03-23T09:42:14.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.416+0000 7f95ceffd640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f95bc06ac30 0x7f95bc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.416+0000 7f95ceffd640 1 -- 172.21.15.47:0/3316028446 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95c000f070 con 0x7f95d0153900 2024-03-23T09:42:14.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.416+0000 7f95d6027640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f95bc06ac30 0x7f95bc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.417+0000 7f95d6027640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f95bc06ac30 0x7f95bc06d0f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f95c8000c00 tx=0x7f95c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:14.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.421+0000 7f95ceffd640 1 -- 172.21.15.47:0/3316028446 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f95d00b3560 con 0x7f95d0153900 2024-03-23T09:42:14.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.599+0000 7f95d828b640 1 -- 172.21.15.47:0/3316028446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95d00b3560 con 0x7f95d0153900 2024-03-23T09:42:14.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.600+0000 7f95ceffd640 1 -- 172.21.15.47:0/3316028446 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f95d00b3560 con 0x7f95d0153900 2024-03-23T09:42:14.600 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:14.600 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:00.721483+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:58.297353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:58.685375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:42:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.602+0000 7f95d828b640 1 -- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f95bc06ac30 msgr2=0x7f95bc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.602+0000 7f95d828b640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f95bc06ac30 0x7f95bc06d0f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f95c8000c00 tx=0x7f95c8009040 comp rx=0 tx=0).stop 2024-03-23T09:42:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.602+0000 7f95d828b640 1 -- 172.21.15.47:0/3316028446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d0153900 msgr2=0x7f95d00ab870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.602+0000 7f95d828b640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d0153900 0x7f95d00ab870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f95d0161680 tx=0x7f95c00115a0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.602+0000 7f95d828b640 1 -- 172.21.15.47:0/3316028446 shutdown_connections 2024-03-23T09:42:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.602+0000 7f95d828b640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f95bc06ac30 0x7f95bc06d0f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.602+0000 7f95d828b640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d0153900 0x7f95d00ab870 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.602+0000 7f95d828b640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95d00ad6a0 0x7f95d00c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.602+0000 7f95d828b640 1 --2- 172.21.15.47:0/3316028446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95b80047c0 0x7f95d00c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.602+0000 7f95d828b640 1 -- 172.21.15.47:0/3316028446 >> 172.21.15.47:0/3316028446 conn(0x7f95d00b1410 msgr2=0x7f95d00af720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.602+0000 7f95d828b640 1 -- 172.21.15.47:0/3316028446 shutdown_connections 2024-03-23T09:42:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.602+0000 7f95d828b640 1 -- 172.21.15.47:0/3316028446 wait complete. 2024-03-23T09:42:14.613 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:58.297353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:58.685375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:42:14.613 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:42:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.716+0000 7fae3788f640 1 Processor -- start 2024-03-23T09:42:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.717+0000 7fae3788f640 1 -- start start 2024-03-23T09:42:14.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.717+0000 7fae3788f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3015ce10 0x7fae300cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.717+0000 7fae3788f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae300b85d0 0x7fae300cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.717+0000 7fae3788f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae30161600 con 0x7fae3015ca30 2024-03-23T09:42:14.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.717+0000 7fae3788f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae30160e50 con 0x7fae300b85d0 2024-03-23T09:42:14.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.717+0000 7fae3788f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae300d25e0 con 0x7fae3015ce10 2024-03-23T09:42:14.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.718+0000 7fae35e2c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae300b85d0 0x7fae300cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.718+0000 7fae3562b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fae3015ca30 0x7fae30160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38180/0 (socket says 172.21.15.47:38180) 2024-03-23T09:42:14.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.718+0000 7fae3562b640 1 -- 172.21.15.47:0/1762658752 learned_addr learned my addr 172.21.15.47:0/1762658752 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:14.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.718+0000 7fae34e2a640 1 --2- 172.21.15.47:0/1762658752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3015ce10 0x7fae300cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.718+0000 7fae35e2c640 1 -- 172.21.15.47:0/1762658752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3015ce10 msgr2=0x7fae300cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:14.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.718+0000 7fae35e2c640 1 --2- 172.21.15.47:0/1762658752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3015ce10 0x7fae300cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.718+0000 7fae35e2c640 1 -- 172.21.15.47:0/1762658752 >> v1:172.21.15.47:6789/0 conn(0x7fae3015ca30 legacy=0x7fae30160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.718+0000 7fae35e2c640 1 -- 172.21.15.47:0/1762658752 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae30161040 con 0x7fae300b85d0 2024-03-23T09:42:14.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.718+0000 7fae2ffff640 1 -- 172.21.15.47:0/1762658752 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172845382 0 0) 0x7fae30161600 con 0x7fae3015ca30 2024-03-23T09:42:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.719+0000 7fae35e2c640 1 --2- 172.21.15.47:0/1762658752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae300b85d0 0x7fae300cdc70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fae20009d90 tx=0x7fae2002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d1e821a07a5238 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.719+0000 7fae2ffff640 1 -- 172.21.15.47:0/1762658752 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2002cc50 con 0x7fae300b85d0 2024-03-23T09:42:14.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.719+0000 7fae2ffff640 1 -- 172.21.15.47:0/1762658752 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2002cdb0 con 0x7fae300b85d0 2024-03-23T09:42:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.719+0000 7fae2ffff640 1 -- 172.21.15.47:0/1762658752 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20034660 con 0x7fae300b85d0 2024-03-23T09:42:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.719+0000 7fae3788f640 1 -- 172.21.15.47:0/1762658752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae300b85d0 msgr2=0x7fae300cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.719+0000 7fae3788f640 1 --2- 172.21.15.47:0/1762658752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae300b85d0 0x7fae300cdc70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fae20009d90 tx=0x7fae2002bea0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.719+0000 7fae3788f640 1 -- 172.21.15.47:0/1762658752 shutdown_connections 2024-03-23T09:42:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.719+0000 7fae3788f640 1 --2- 172.21.15.47:0/1762658752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae300b85d0 0x7fae300cdc70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.719+0000 7fae3788f640 1 --2- 172.21.15.47:0/1762658752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3015ce10 0x7fae300cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.719+0000 7fae3788f640 1 -- 172.21.15.47:0/1762658752 >> 172.21.15.47:0/1762658752 conn(0x7fae3006b160 msgr2=0x7fae3006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.719+0000 7fae3788f640 1 -- 172.21.15.47:0/1762658752 shutdown_connections 2024-03-23T09:42:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.720+0000 7fae3788f640 1 -- 172.21.15.47:0/1762658752 wait complete. 2024-03-23T09:42:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.721+0000 7fae3788f640 1 Processor -- start 2024-03-23T09:42:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.721+0000 7fae3788f640 1 -- start start 2024-03-23T09:42:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.721+0000 7fae3788f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae300b85d0 0x7fae300cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.721+0000 7fae3788f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3015ca30 0x7fae3014ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.721+0000 7fae3788f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae3015ce10 0x7fae3015ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.721+0000 7fae3788f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3015b4c0 con 0x7fae300b85d0 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.721+0000 7fae3788f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae300cb140 con 0x7fae3015ce10 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.721+0000 7fae3788f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae300cb420 con 0x7fae3015ca30 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.721+0000 7fae3562b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae300b85d0 0x7fae300cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.721+0000 7fae3562b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae300b85d0 0x7fae300cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39390/0 (socket says 172.21.15.47:39390) 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.721+0000 7fae3562b640 1 -- 172.21.15.47:0/4060409125 learned_addr learned my addr 172.21.15.47:0/4060409125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae34e2a640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3015ca30 0x7fae3014ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae3562b640 1 -- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3015ca30 msgr2=0x7fae3014ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae3562b640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3015ca30 0x7fae3014ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae35e2c640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae3015ce10 0x7fae3015ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae3562b640 1 -- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae3015ce10 msgr2=0x7fae3015ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae3562b640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae3015ce10 0x7fae3015ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae3562b640 1 -- 172.21.15.47:0/4060409125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae20009a70 con 0x7fae300b85d0 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae34e2a640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3015ca30 0x7fae3014ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae35e2c640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae3015ce10 0x7fae3015ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae3562b640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae300b85d0 0x7fae300cbf00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fae2800c4d0 tx=0x7fae2800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae2dffb640 1 -- 172.21.15.47:0/4060409125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae280074e0 con 0x7fae300b85d0 2024-03-23T09:42:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae2dffb640 1 -- 172.21.15.47:0/4060409125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae28007660 con 0x7fae300b85d0 2024-03-23T09:42:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae3788f640 1 -- 172.21.15.47:0/4060409125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae300cb700 con 0x7fae300b85d0 2024-03-23T09:42:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae2dffb640 1 -- 172.21.15.47:0/4060409125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28004a20 con 0x7fae300b85d0 2024-03-23T09:42:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.722+0000 7fae3788f640 1 -- 172.21.15.47:0/4060409125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3014b1c0 con 0x7fae300b85d0 2024-03-23T09:42:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.723+0000 7fae3788f640 1 -- 172.21.15.47:0/4060409125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae30072360 con 0x7fae300b85d0 2024-03-23T09:42:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.723+0000 7fae2dffb640 1 -- 172.21.15.47:0/4060409125 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fae28004b80 con 0x7fae300b85d0 2024-03-23T09:42:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.724+0000 7fae2dffb640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae1c06ac00 0x7fae1c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.724+0000 7fae2dffb640 1 -- 172.21.15.47:0/4060409125 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae28088860 con 0x7fae300b85d0 2024-03-23T09:42:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.724+0000 7fae34e2a640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae1c06ac00 0x7fae1c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.724+0000 7fae34e2a640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae1c06ac00 0x7fae1c06d0c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fae3014e770 tx=0x7fae24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:14.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.728+0000 7fae2dffb640 1 -- 172.21.15.47:0/4060409125 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fae28053a40 con 0x7fae300b85d0 2024-03-23T09:42:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.907+0000 7fae3788f640 1 -- 172.21.15.47:0/4060409125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae300d0d90 con 0x7fae300b85d0 2024-03-23T09:42:14.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.907+0000 7fae2dffb640 1 -- 172.21.15.47:0/4060409125 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae28018020 con 0x7fae300b85d0 2024-03-23T09:42:14.907 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:42:14.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.909+0000 7fae3788f640 1 -- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae1c06ac00 msgr2=0x7fae1c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.909+0000 7fae3788f640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae1c06ac00 0x7fae1c06d0c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fae3014e770 tx=0x7fae24008040 comp rx=0 tx=0).stop 2024-03-23T09:42:14.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.909+0000 7fae3788f640 1 -- 172.21.15.47:0/4060409125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae300b85d0 msgr2=0x7fae300cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:14.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.909+0000 7fae3788f640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae300b85d0 0x7fae300cbf00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fae2800c4d0 tx=0x7fae2800c990 comp rx=0 tx=0).stop 2024-03-23T09:42:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.909+0000 7fae3788f640 1 -- 172.21.15.47:0/4060409125 shutdown_connections 2024-03-23T09:42:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.909+0000 7fae3788f640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fae1c06ac00 0x7fae1c06d0c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.909+0000 7fae3788f640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae3015ce10 0x7fae3015ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.909+0000 7fae3788f640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3015ca30 0x7fae3014ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.909+0000 7fae3788f640 1 --2- 172.21.15.47:0/4060409125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae300b85d0 0x7fae300cbf00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.909+0000 7fae3788f640 1 -- 172.21.15.47:0/4060409125 >> 172.21.15.47:0/4060409125 conn(0x7fae3006b160 msgr2=0x7fae3006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.909+0000 7fae3788f640 1 -- 172.21.15.47:0/4060409125 shutdown_connections 2024-03-23T09:42:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:14.909+0000 7fae3788f640 1 -- 172.21.15.47:0/4060409125 wait complete. 2024-03-23T09:42:14.919 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:42:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.024+0000 7f9c8d610640 1 Processor -- start 2024-03-23T09:42:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.024+0000 7f9c8d610640 1 -- start start 2024-03-23T09:42:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.024+0000 7f9c8d610640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c8815cb50 0x7f9c880b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.024+0000 7f9c8d610640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c880b72f0 con 0x7f9c880cd5a0 2024-03-23T09:42:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.024+0000 7f9c8d610640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c880b75d0 con 0x7f9c880cd9e0 2024-03-23T09:42:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.024+0000 7f9c8d610640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880b7e70 con 0x7f9c8815cb50 2024-03-23T09:42:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c867fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9c880cd5a0 0x7f9c880d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38184/0 (socket says 172.21.15.47:38184) 2024-03-23T09:42:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c867fc640 1 -- 172.21.15.47:0/3046340103 learned_addr learned my addr 172.21.15.47:0/3046340103 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265062769 0 0) 0x7f9c880b72f0 con 0x7f9c880cd5a0 2024-03-23T09:42:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c86ffd640 1 --2- 172.21.15.47:0/3046340103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c8815cb50 0x7f9c880b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c70003520 con 0x7f9c880cd5a0 2024-03-23T09:42:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2466498843 0 0) 0x7f9c70003520 con 0x7f9c880cd5a0 2024-03-23T09:42:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c8815cb50 msgr2=0x7f9c880b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 --2- 172.21.15.47:0/3046340103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c8815cb50 0x7f9c880b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 >> v1:172.21.15.175:6789/0 conn(0x7f9c880cd9e0 legacy=0x7f9c880b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c880b78b0 con 0x7f9c880cd5a0 2024-03-23T09:42:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f9c7c001670 con 0x7f9c880cd5a0 2024-03-23T09:42:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 >> v1:172.21.15.47:6789/0 conn(0x7f9c880cd5a0 legacy=0x7f9c880d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 --2- 172.21.15.47:0/3046340103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c700039a0 0x7f9c70003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 --2- 172.21.15.47:0/3046340103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c8815cb50 0x7f9c700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 --2- 172.21.15.47:0/3046340103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c70004d00 0x7f9c70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880b7e70 con 0x7f9c70004d00 2024-03-23T09:42:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7000bac0 con 0x7f9c700039a0 2024-03-23T09:42:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7000bc80 con 0x7f9c8815cb50 2024-03-23T09:42:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c877fe640 1 --2- 172.21.15.47:0/3046340103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c70004d00 0x7f9c70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.025+0000 7f9c86ffd640 1 --2- 172.21.15.47:0/3046340103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c700039a0 0x7f9c70003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.026+0000 7f9c867fc640 1 --2- 172.21.15.47:0/3046340103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c8815cb50 0x7f9c700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.026+0000 7f9c877fe640 1 -- 172.21.15.47:0/3046340103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c8815cb50 msgr2=0x7f9c700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.026+0000 7f9c877fe640 1 --2- 172.21.15.47:0/3046340103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c8815cb50 0x7f9c700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.026+0000 7f9c877fe640 1 -- 172.21.15.47:0/3046340103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c700039a0 msgr2=0x7f9c70003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.026+0000 7f9c877fe640 1 --2- 172.21.15.47:0/3046340103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c700039a0 0x7f9c70003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.026+0000 7f9c877fe640 1 -- 172.21.15.47:0/3046340103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c70003520 con 0x7f9c70004d00 2024-03-23T09:42:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.026+0000 7f9c86ffd640 1 --2- 172.21.15.47:0/3046340103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c700039a0 0x7f9c70003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.026+0000 7f9c877fe640 1 --2- 172.21.15.47:0/3046340103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c70004d00 0x7f9c70005180 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9c80001e20 tx=0x7f9c700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceea03108f39b900 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.027+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80003b70 con 0x7f9c70004d00 2024-03-23T09:42:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.027+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c80003cd0 con 0x7f9c70004d00 2024-03-23T09:42:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.027+0000 7f9c85ffb640 1 -- 172.21.15.47:0/3046340103 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80005310 con 0x7f9c70004d00 2024-03-23T09:42:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.027+0000 7f9c8d610640 1 -- 172.21.15.47:0/3046340103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c70004d00 msgr2=0x7f9c70005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.027+0000 7f9c8d610640 1 --2- 172.21.15.47:0/3046340103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c70004d00 0x7f9c70005180 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9c80001e20 tx=0x7f9c700057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.027+0000 7f9c877fe640 1 -- 172.21.15.47:0/3046340103 reap_dead start 2024-03-23T09:42:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.027+0000 7f9c8d610640 1 -- 172.21.15.47:0/3046340103 shutdown_connections 2024-03-23T09:42:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.027+0000 7f9c8d610640 1 -- 172.21.15.47:0/3046340103 >> 172.21.15.47:0/3046340103 conn(0x7f9c88065df0 msgr2=0x7f9c88066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.027+0000 7f9c8d610640 1 -- 172.21.15.47:0/3046340103 shutdown_connections 2024-03-23T09:42:15.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.027+0000 7f9c8d610640 1 -- 172.21.15.47:0/3046340103 wait complete. 2024-03-23T09:42:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.028+0000 7f9c8d610640 1 Processor -- start 2024-03-23T09:42:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.029+0000 7f9c8d610640 1 -- start start 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.029+0000 7f9c8d610640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c8815cf30 0x7f9c88159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.029+0000 7f9c8d610640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c88156950 0x7f9c8815d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.029+0000 7f9c8d610640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c8815d850 0x7f9c880c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.029+0000 7f9c8d610640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8815ee30 con 0x7f9c8815cf30 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.029+0000 7f9c8d610640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c88156480 con 0x7f9c8815d850 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.029+0000 7f9c8d610640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880c6630 con 0x7f9c88156950 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.029+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c8815cf30 0x7f9c88159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.029+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c8815cf30 0x7f9c88159560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39410/0 (socket says 172.21.15.47:39410) 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.029+0000 7f9c86ffd640 1 -- 172.21.15.47:0/4194638052 learned_addr learned my addr 172.21.15.47:0/4194638052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.030+0000 7f9c877fe640 1 --2- 172.21.15.47:0/4194638052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c8815d850 0x7f9c880c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.030+0000 7f9c86ffd640 1 -- 172.21.15.47:0/4194638052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c88156950 msgr2=0x7f9c8815d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.030+0000 7f9c86ffd640 1 --2- 172.21.15.47:0/4194638052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c88156950 0x7f9c8815d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.030+0000 7f9c86ffd640 1 -- 172.21.15.47:0/4194638052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c8815d850 msgr2=0x7f9c880c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.030+0000 7f9c86ffd640 1 --2- 172.21.15.47:0/4194638052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c8815d850 0x7f9c880c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.030+0000 7f9c86ffd640 1 -- 172.21.15.47:0/4194638052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c880c6910 con 0x7f9c8815cf30 2024-03-23T09:42:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.030+0000 7f9c86ffd640 1 --2- 172.21.15.47:0/4194638052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c8815cf30 0x7f9c88159560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9c78011600 tx=0x7f9c78011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.030+0000 7f9c6f7fe640 1 -- 172.21.15.47:0/4194638052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800f920 con 0x7f9c8815cf30 2024-03-23T09:42:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.030+0000 7f9c6f7fe640 1 -- 172.21.15.47:0/4194638052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7800fa80 con 0x7f9c8815cf30 2024-03-23T09:42:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.030+0000 7f9c8d610640 1 -- 172.21.15.47:0/4194638052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c880c6bf0 con 0x7f9c8815cf30 2024-03-23T09:42:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.030+0000 7f9c6f7fe640 1 -- 172.21.15.47:0/4194638052 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800fe10 con 0x7f9c8815cf30 2024-03-23T09:42:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.030+0000 7f9c8d610640 1 -- 172.21.15.47:0/4194638052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c880c71b0 con 0x7f9c8815cf30 2024-03-23T09:42:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.031+0000 7f9c8d610640 1 -- 172.21.15.47:0/4194638052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c880c7370 con 0x7f9c8815cf30 2024-03-23T09:42:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.031+0000 7f9c6f7fe640 1 -- 172.21.15.47:0/4194638052 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9c78015460 con 0x7f9c8815cf30 2024-03-23T09:42:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.032+0000 7f9c6f7fe640 1 --2- 172.21.15.47:0/4194638052 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9c7406ac00 0x7f9c7406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.032+0000 7f9c6f7fe640 1 -- 172.21.15.47:0/4194638052 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c78089c80 con 0x7f9c8815cf30 2024-03-23T09:42:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.032+0000 7f9c867fc640 1 --2- 172.21.15.47:0/4194638052 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9c7406ac00 0x7f9c7406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.033+0000 7f9c867fc640 1 --2- 172.21.15.47:0/4194638052 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9c7406ac00 0x7f9c7406d0c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9c88159510 tx=0x7f9c7c000ff0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:15.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.036+0000 7f9c6f7fe640 1 -- 172.21.15.47:0/4194638052 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9c78054dd0 con 0x7f9c8815cf30 2024-03-23T09:42:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.215+0000 7f9c8d610640 1 -- 172.21.15.47:0/4194638052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c88072360 con 0x7f9c8815cf30 2024-03-23T09:42:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.216+0000 7f9c6f7fe640 1 -- 172.21.15.47:0/4194638052 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9c780583f0 con 0x7f9c8815cf30 2024-03-23T09:42:15.215 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:42:15.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.217+0000 7f9c8d610640 1 -- 172.21.15.47:0/4194638052 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9c7406ac00 msgr2=0x7f9c7406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.217+0000 7f9c8d610640 1 --2- 172.21.15.47:0/4194638052 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9c7406ac00 0x7f9c7406d0c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9c88159510 tx=0x7f9c7c000ff0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.217+0000 7f9c8d610640 1 -- 172.21.15.47:0/4194638052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c8815cf30 msgr2=0x7f9c88159560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.217+0000 7f9c8d610640 1 --2- 172.21.15.47:0/4194638052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c8815cf30 0x7f9c88159560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9c78011600 tx=0x7f9c78011ac0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.218+0000 7f9c8d610640 1 -- 172.21.15.47:0/4194638052 shutdown_connections 2024-03-23T09:42:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.218+0000 7f9c8d610640 1 --2- 172.21.15.47:0/4194638052 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9c7406ac00 0x7f9c7406d0c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.218+0000 7f9c8d610640 1 --2- 172.21.15.47:0/4194638052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c8815d850 0x7f9c880c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.218+0000 7f9c8d610640 1 --2- 172.21.15.47:0/4194638052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c88156950 0x7f9c8815d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.218+0000 7f9c8d610640 1 --2- 172.21.15.47:0/4194638052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c8815cf30 0x7f9c88159560 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.218+0000 7f9c8d610640 1 -- 172.21.15.47:0/4194638052 >> 172.21.15.47:0/4194638052 conn(0x7f9c88065df0 msgr2=0x7f9c880d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.218+0000 7f9c8d610640 1 -- 172.21.15.47:0/4194638052 shutdown_connections 2024-03-23T09:42:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.218+0000 7f9c8d610640 1 -- 172.21.15.47:0/4194638052 wait complete. 2024-03-23T09:42:15.228 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:42:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.332+0000 7f49e7189640 1 Processor -- start 2024-03-23T09:42:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.332+0000 7f49e7189640 1 -- start start 2024-03-23T09:42:15.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e7189640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f49e015c990 0x7f49e0160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e7189640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e00b8350 0x7f49e00cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e7189640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49e00cdc40 0x7f49e00b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e7189640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e00ce0b0 con 0x7f49e00cdc40 2024-03-23T09:42:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e7189640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e0161630 con 0x7f49e015c990 2024-03-23T09:42:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e7189640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e00d1030 con 0x7f49e00b8350 2024-03-23T09:42:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e5726640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49e00cdc40 0x7f49e00b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e5726640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49e00cdc40 0x7f49e00b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39418/0 (socket says 172.21.15.47:39418) 2024-03-23T09:42:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e5726640 1 -- 172.21.15.47:0/401151868 learned_addr learned my addr 172.21.15.47:0/401151868 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49dffff640 1 --2- 172.21.15.47:0/401151868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e00b8350 0x7f49e00cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e4f25640 1 --2- 172.21.15.47:0/401151868 >> v2:172.21.15.175:3300/0 conn(0x7f49e015c990 0x7f49e0160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e5726640 1 -- 172.21.15.47:0/401151868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e00b8350 msgr2=0x7f49e00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e5726640 1 --2- 172.21.15.47:0/401151868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e00b8350 0x7f49e00cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e5726640 1 -- 172.21.15.47:0/401151868 >> v2:172.21.15.175:3300/0 conn(0x7f49e015c990 msgr2=0x7f49e0160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:15.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e5726640 1 --2- 172.21.15.47:0/401151868 >> v2:172.21.15.175:3300/0 conn(0x7f49e015c990 0x7f49e0160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.333+0000 7f49e5726640 1 -- 172.21.15.47:0/401151868 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e00cfe10 con 0x7f49e00cdc40 2024-03-23T09:42:15.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.334+0000 7f49e5726640 1 --2- 172.21.15.47:0/401151868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49e00cdc40 0x7f49e00b4440 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f49e00b8b30 tx=0x7f49d802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdf64a30713241c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:15.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.334+0000 7f49df7fe640 1 -- 172.21.15.47:0/401151868 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d802c9e0 con 0x7f49e00cdc40 2024-03-23T09:42:15.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.334+0000 7f49df7fe640 1 -- 172.21.15.47:0/401151868 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d802cb40 con 0x7f49e00cdc40 2024-03-23T09:42:15.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.334+0000 7f49df7fe640 1 -- 172.21.15.47:0/401151868 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80343f0 con 0x7f49e00cdc40 2024-03-23T09:42:15.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.335+0000 7f49e7189640 1 -- 172.21.15.47:0/401151868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49e00cdc40 msgr2=0x7f49e00b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.335+0000 7f49e7189640 1 --2- 172.21.15.47:0/401151868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49e00cdc40 0x7f49e00b4440 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f49e00b8b30 tx=0x7f49d802be40 comp rx=0 tx=0).stop 2024-03-23T09:42:15.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.335+0000 7f49e7189640 1 -- 172.21.15.47:0/401151868 shutdown_connections 2024-03-23T09:42:15.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.335+0000 7f49e7189640 1 --2- 172.21.15.47:0/401151868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49e00cdc40 0x7f49e00b4440 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.335+0000 7f49e7189640 1 --2- 172.21.15.47:0/401151868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e00b8350 0x7f49e00cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.335+0000 7f49e7189640 1 --2- 172.21.15.47:0/401151868 >> v2:172.21.15.175:3300/0 conn(0x7f49e015c990 0x7f49e0160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.335+0000 7f49e7189640 1 -- 172.21.15.47:0/401151868 >> 172.21.15.47:0/401151868 conn(0x7f49e00659d0 msgr2=0x7f49e0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:15.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.335+0000 7f49e7189640 1 -- 172.21.15.47:0/401151868 shutdown_connections 2024-03-23T09:42:15.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.335+0000 7f49e7189640 1 -- 172.21.15.47:0/401151868 wait complete. 2024-03-23T09:42:15.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.336+0000 7f49e7189640 1 Processor -- start 2024-03-23T09:42:15.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.336+0000 7f49e7189640 1 -- start start 2024-03-23T09:42:15.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49e7189640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e00b8350 0x7f49e0159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49e7189640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49e00cdc40 0x7f49e0158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49e7189640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e015c990 0x7f49e0158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49e7189640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e0150ed0 con 0x7f49e00cdc40 2024-03-23T09:42:15.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49e7189640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e01511b0 con 0x7f49e015c990 2024-03-23T09:42:15.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49e7189640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e015a480 con 0x7f49e00b8350 2024-03-23T09:42:15.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49dffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49e00cdc40 0x7f49e0158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49dffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49e00cdc40 0x7f49e0158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39432/0 (socket says 172.21.15.47:39432) 2024-03-23T09:42:15.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49dffff640 1 -- 172.21.15.47:0/2057437080 learned_addr learned my addr 172.21.15.47:0/2057437080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:15.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49dffff640 1 -- 172.21.15.47:0/2057437080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e00b8350 msgr2=0x7f49e0159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49dffff640 1 --2- 172.21.15.47:0/2057437080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e00b8350 0x7f49e0159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49dffff640 1 -- 172.21.15.47:0/2057437080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e015c990 msgr2=0x7f49e0158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49dffff640 1 --2- 172.21.15.47:0/2057437080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e015c990 0x7f49e0158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49dffff640 1 -- 172.21.15.47:0/2057437080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d8009b30 con 0x7f49e00cdc40 2024-03-23T09:42:15.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49dffff640 1 --2- 172.21.15.47:0/2057437080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49e00cdc40 0x7f49e0158640 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00d800 tx=0x7f49cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:15.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49dd7fa640 1 -- 172.21.15.47:0/2057437080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0096a0 con 0x7f49e00cdc40 2024-03-23T09:42:15.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49dd7fa640 1 -- 172.21.15.47:0/2057437080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc009800 con 0x7f49e00cdc40 2024-03-23T09:42:15.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49e7189640 1 -- 172.21.15.47:0/2057437080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e015a760 con 0x7f49e00cdc40 2024-03-23T09:42:15.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49dd7fa640 1 -- 172.21.15.47:0/2057437080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc009990 con 0x7f49e00cdc40 2024-03-23T09:42:15.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.337+0000 7f49e7189640 1 -- 172.21.15.47:0/2057437080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e015a0d0 con 0x7f49e00cdc40 2024-03-23T09:42:15.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.338+0000 7f49e7189640 1 -- 172.21.15.47:0/2057437080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e0072360 con 0x7f49e00cdc40 2024-03-23T09:42:15.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.339+0000 7f49dd7fa640 1 -- 172.21.15.47:0/2057437080 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f49cc009af0 con 0x7f49e00cdc40 2024-03-23T09:42:15.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.339+0000 7f49dd7fa640 1 --2- 172.21.15.47:0/2057437080 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f49c406ac00 0x7f49c406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.339+0000 7f49dd7fa640 1 -- 172.21.15.47:0/2057437080 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49cc089830 con 0x7f49e00cdc40 2024-03-23T09:42:15.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.339+0000 7f49e4f25640 1 --2- 172.21.15.47:0/2057437080 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f49c406ac00 0x7f49c406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.340+0000 7f49e4f25640 1 --2- 172.21.15.47:0/2057437080 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f49c406ac00 0x7f49c406d0c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f49d40098f0 tx=0x7f49d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:15.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.343+0000 7f49dd7fa640 1 -- 172.21.15.47:0/2057437080 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f49cc054a10 con 0x7f49e00cdc40 2024-03-23T09:42:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.526+0000 7f49e7189640 1 -- 172.21.15.47:0/2057437080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f49e015a8f0 con 0x7f49e00cdc40 2024-03-23T09:42:15.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.526+0000 7f49dd7fa640 1 -- 172.21.15.47:0/2057437080 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f49cc00b3a0 con 0x7f49e00cdc40 2024-03-23T09:42:15.526 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:42:15.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.528+0000 7f49e7189640 1 -- 172.21.15.47:0/2057437080 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f49c406ac00 msgr2=0x7f49c406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.528+0000 7f49e7189640 1 --2- 172.21.15.47:0/2057437080 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f49c406ac00 0x7f49c406d0c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f49d40098f0 tx=0x7f49d4008040 comp rx=0 tx=0).stop 2024-03-23T09:42:15.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.528+0000 7f49e7189640 1 -- 172.21.15.47:0/2057437080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49e00cdc40 msgr2=0x7f49e0158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.528+0000 7f49e7189640 1 --2- 172.21.15.47:0/2057437080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49e00cdc40 0x7f49e0158640 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00d800 tx=0x7f49cc00dcc0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.528+0000 7f49e7189640 1 -- 172.21.15.47:0/2057437080 shutdown_connections 2024-03-23T09:42:15.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.528+0000 7f49e7189640 1 --2- 172.21.15.47:0/2057437080 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f49c406ac00 0x7f49c406d0c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.528+0000 7f49e7189640 1 --2- 172.21.15.47:0/2057437080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e015c990 0x7f49e0158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.528+0000 7f49e7189640 1 --2- 172.21.15.47:0/2057437080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49e00cdc40 0x7f49e0158640 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.528+0000 7f49e7189640 1 --2- 172.21.15.47:0/2057437080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e00b8350 0x7f49e0159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.528+0000 7f49e7189640 1 -- 172.21.15.47:0/2057437080 >> 172.21.15.47:0/2057437080 conn(0x7f49e00659d0 msgr2=0x7f49e00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:15.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.528+0000 7f49e7189640 1 -- 172.21.15.47:0/2057437080 shutdown_connections 2024-03-23T09:42:15.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.528+0000 7f49e7189640 1 -- 172.21.15.47:0/2057437080 wait complete. 2024-03-23T09:42:15.538 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:42:15.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.639+0000 7f53904e9640 1 Processor -- start 2024-03-23T09:42:15.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.639+0000 7f53904e9640 1 -- start start 2024-03-23T09:42:15.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.640+0000 7f53904e9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f538815cd00 0x7f53880d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.640+0000 7f53904e9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f53880d1020 0x7f53880cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.640+0000 7f53904e9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53880cfba0 con 0x7f53880cf750 2024-03-23T09:42:15.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.640+0000 7f53904e9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538815d0e0 con 0x7f538815cd00 2024-03-23T09:42:15.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.640+0000 7f53904e9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53880d1540 con 0x7f53880d1020 2024-03-23T09:42:15.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.640+0000 7f538e285640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f538815cd00 0x7f53880d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.640+0000 7f538ea86640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f53880cf750 0x7f53880b43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38190/0 (socket says 172.21.15.47:38190) 2024-03-23T09:42:15.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.640+0000 7f538ea86640 1 -- 172.21.15.47:0/4036047322 learned_addr learned my addr 172.21.15.47:0/4036047322 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:15.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.641+0000 7f538da84640 1 --2- 172.21.15.47:0/4036047322 >> v2:172.21.15.175:3300/0 conn(0x7f53880d1020 0x7f53880cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.641+0000 7f538e285640 1 -- 172.21.15.47:0/4036047322 >> v2:172.21.15.175:3300/0 conn(0x7f53880d1020 msgr2=0x7f53880cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:15.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.641+0000 7f538e285640 1 --2- 172.21.15.47:0/4036047322 >> v2:172.21.15.175:3300/0 conn(0x7f53880d1020 0x7f53880cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.641+0000 7f538e285640 1 -- 172.21.15.47:0/4036047322 >> v1:172.21.15.47:6789/0 conn(0x7f53880cf750 legacy=0x7f53880b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.641+0000 7f538e285640 1 -- 172.21.15.47:0/4036047322 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53880d1820 con 0x7f538815cd00 2024-03-23T09:42:15.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.641+0000 7f538d283640 1 -- 172.21.15.47:0/4036047322 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3545126878 0 0) 0x7f53880cfba0 con 0x7f53880cf750 2024-03-23T09:42:15.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.642+0000 7f538e285640 1 --2- 172.21.15.47:0/4036047322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f538815cd00 0x7f53880d0ae0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f53880b5520 tx=0x7f538402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2080e26f5947e05 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:15.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.642+0000 7f538d283640 1 -- 172.21.15.47:0/4036047322 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538402e070 con 0x7f538815cd00 2024-03-23T09:42:15.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.642+0000 7f538d283640 1 -- 172.21.15.47:0/4036047322 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53840315a0 con 0x7f538815cd00 2024-03-23T09:42:15.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.642+0000 7f538d283640 1 -- 172.21.15.47:0/4036047322 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384031930 con 0x7f538815cd00 2024-03-23T09:42:15.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.642+0000 7f53904e9640 1 -- 172.21.15.47:0/4036047322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f538815cd00 msgr2=0x7f53880d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.642+0000 7f53904e9640 1 --2- 172.21.15.47:0/4036047322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f538815cd00 0x7f53880d0ae0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f53880b5520 tx=0x7f538402b840 comp rx=0 tx=0).stop 2024-03-23T09:42:15.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.642+0000 7f53904e9640 1 -- 172.21.15.47:0/4036047322 shutdown_connections 2024-03-23T09:42:15.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.642+0000 7f53904e9640 1 --2- 172.21.15.47:0/4036047322 >> v2:172.21.15.175:3300/0 conn(0x7f53880d1020 0x7f53880cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.642+0000 7f53904e9640 1 --2- 172.21.15.47:0/4036047322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f538815cd00 0x7f53880d0ae0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.642+0000 7f53904e9640 1 -- 172.21.15.47:0/4036047322 >> 172.21.15.47:0/4036047322 conn(0x7f538806b160 msgr2=0x7f538806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:15.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.642+0000 7f53904e9640 1 -- 172.21.15.47:0/4036047322 shutdown_connections 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.642+0000 7f53904e9640 1 -- 172.21.15.47:0/4036047322 wait complete. 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.643+0000 7f53904e9640 1 Processor -- start 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.643+0000 7f53904e9640 1 -- start start 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.644+0000 7f53904e9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53880cf750 0x7f53881f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.644+0000 7f53904e9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53880d1020 0x7f53881539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.644+0000 7f53904e9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f538815cd00 0x7f538815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.644+0000 7f53904e9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53881f6350 con 0x7f538815cd00 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.644+0000 7f53904e9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53881f2dc0 con 0x7f53880cf750 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.644+0000 7f53904e9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53880b3de0 con 0x7f53880d1020 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.644+0000 7f538ea86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f538815cd00 0x7f538815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f538ea86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f538815cd00 0x7f538815ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39442/0 (socket says 172.21.15.47:39442) 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f538ea86640 1 -- 172.21.15.47:0/1862716691 learned_addr learned my addr 172.21.15.47:0/1862716691 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f538e285640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53880cf750 0x7f53881f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f538da84640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53880d1020 0x7f53881539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f538ea86640 1 -- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53880d1020 msgr2=0x7f53881539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f538ea86640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53880d1020 0x7f53881539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f538ea86640 1 -- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53880cf750 msgr2=0x7f53881f2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f538ea86640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53880cf750 0x7f53881f2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f538ea86640 1 -- 172.21.15.47:0/1862716691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5384009400 con 0x7f538815cd00 2024-03-23T09:42:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f538e285640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53880cf750 0x7f53881f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f538ea86640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f538815cd00 0x7f538815ba60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f538000caf0 tx=0x7f538000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f537effd640 1 -- 172.21.15.47:0/1862716691 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538000aca0 con 0x7f538815cd00 2024-03-23T09:42:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f537effd640 1 -- 172.21.15.47:0/1862716691 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538000ae00 con 0x7f538815cd00 2024-03-23T09:42:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f53904e9640 1 -- 172.21.15.47:0/1862716691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5388153f00 con 0x7f538815cd00 2024-03-23T09:42:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f537effd640 1 -- 172.21.15.47:0/1862716691 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380012640 con 0x7f538815cd00 2024-03-23T09:42:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.645+0000 7f53904e9640 1 -- 172.21.15.47:0/1862716691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53881f5a20 con 0x7f538815cd00 2024-03-23T09:42:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.646+0000 7f53904e9640 1 -- 172.21.15.47:0/1862716691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5388072360 con 0x7f538815cd00 2024-03-23T09:42:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.647+0000 7f537effd640 1 -- 172.21.15.47:0/1862716691 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f53800127a0 con 0x7f538815cd00 2024-03-23T09:42:15.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.647+0000 7f537effd640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f536c06ac30 0x7f536c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:15.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.647+0000 7f537effd640 1 -- 172.21.15.47:0/1862716691 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5380089830 con 0x7f538815cd00 2024-03-23T09:42:15.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.647+0000 7f538e285640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f536c06ac30 0x7f536c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:15.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.648+0000 7f538e285640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f536c06ac30 0x7f536c06d0f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5384036000 tx=0x7f5384008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:15.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.651+0000 7f537effd640 1 -- 172.21.15.47:0/1862716691 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5380054aa0 con 0x7f538815cd00 2024-03-23T09:42:15.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.845+0000 7f53904e9640 1 -- 172.21.15.47:0/1862716691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5388066be0 con 0x7f538815cd00 2024-03-23T09:42:15.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.845+0000 7f537effd640 1 -- 172.21.15.47:0/1862716691 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7f53800580c0 con 0x7f538815cd00 2024-03-23T09:42:15.845 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:42:15.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.847+0000 7f53904e9640 1 -- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f536c06ac30 msgr2=0x7f536c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.847+0000 7f53904e9640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f536c06ac30 0x7f536c06d0f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5384036000 tx=0x7f5384008030 comp rx=0 tx=0).stop 2024-03-23T09:42:15.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.847+0000 7f53904e9640 1 -- 172.21.15.47:0/1862716691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f538815cd00 msgr2=0x7f538815ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:15.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.847+0000 7f53904e9640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f538815cd00 0x7f538815ba60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f538000caf0 tx=0x7f538000a060 comp rx=0 tx=0).stop 2024-03-23T09:42:15.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.847+0000 7f53904e9640 1 -- 172.21.15.47:0/1862716691 shutdown_connections 2024-03-23T09:42:15.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.847+0000 7f53904e9640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f536c06ac30 0x7f536c06d0f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.847+0000 7f53904e9640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f538815cd00 0x7f538815ba60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.847+0000 7f53904e9640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53880d1020 0x7f53881539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.847+0000 7f53904e9640 1 --2- 172.21.15.47:0/1862716691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53880cf750 0x7f53881f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:15.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.847+0000 7f53904e9640 1 -- 172.21.15.47:0/1862716691 >> 172.21.15.47:0/1862716691 conn(0x7f538806b160 msgr2=0x7f538815d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:15.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.847+0000 7f53904e9640 1 -- 172.21.15.47:0/1862716691 shutdown_connections 2024-03-23T09:42:15.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:15.847+0000 7f53904e9640 1 -- 172.21.15.47:0/1862716691 wait complete. 2024-03-23T09:42:15.857 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:42:15.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:15.857 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:42:15.882 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[51227]: fuse finished with error 0 and tester_r 0 2024-03-23T09:42:15.884 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:42:21.985 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:42:21.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:21.986 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:42:22.004 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:42:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.157+0000 7fc084f38640 1 Processor -- start 2024-03-23T09:42:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.157+0000 7fc084f38640 1 -- start start 2024-03-23T09:42:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.158+0000 7fc084f38640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0800cad40 0x7fc0800b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.158+0000 7fc084f38640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0800b1400 con 0x7fc0800cd680 2024-03-23T09:42:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.158+0000 7fc084f38640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0800b16e0 con 0x7fc08015cbe0 2024-03-23T09:42:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.158+0000 7fc084f38640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0800cedb0 con 0x7fc0800cad40 2024-03-23T09:42:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.158+0000 7fc07ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0800cad40 0x7fc0800b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.158+0000 7fc07dd9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc0800cd680 0x7fc0800b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38198/0 (socket says 172.21.15.47:38198) 2024-03-23T09:42:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.158+0000 7fc07dd9b640 1 -- 172.21.15.47:0/1927022671 learned_addr learned my addr 172.21.15.47:0/1927022671 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.159+0000 7fc07ed9d640 1 -- 172.21.15.47:0/1927022671 >> v1:172.21.15.175:6789/0 conn(0x7fc08015cbe0 legacy=0x7fc0800ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.159+0000 7fc07ed9d640 1 -- 172.21.15.47:0/1927022671 >> v1:172.21.15.47:6789/0 conn(0x7fc0800cd680 legacy=0x7fc0800b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.159+0000 7fc07ed9d640 1 -- 172.21.15.47:0/1927022671 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0800cf090 con 0x7fc0800cad40 2024-03-23T09:42:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.159+0000 7fc07d59a640 1 -- 172.21.15.47:0/1927022671 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032920091 0 0) 0x7fc0800b1400 con 0x7fc0800cd680 2024-03-23T09:42:22.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.160+0000 7fc07ed9d640 1 --2- 172.21.15.47:0/1927022671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0800cad40 0x7fc0800b0de0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc070009d90 tx=0x7fc07002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9032a45718be659a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:22.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.160+0000 7fc07d59a640 1 -- 172.21.15.47:0/1927022671 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07002cc60 con 0x7fc0800cad40 2024-03-23T09:42:22.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.160+0000 7fc07d59a640 1 -- 172.21.15.47:0/1927022671 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07002cdc0 con 0x7fc0800cad40 2024-03-23T09:42:22.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.160+0000 7fc07d59a640 1 -- 172.21.15.47:0/1927022671 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070034660 con 0x7fc0800cad40 2024-03-23T09:42:22.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.160+0000 7fc084f38640 1 -- 172.21.15.47:0/1927022671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0800cad40 msgr2=0x7fc0800b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.160+0000 7fc084f38640 1 --2- 172.21.15.47:0/1927022671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0800cad40 0x7fc0800b0de0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc070009d90 tx=0x7fc07002beb0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.160+0000 7fc084f38640 1 -- 172.21.15.47:0/1927022671 shutdown_connections 2024-03-23T09:42:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.160+0000 7fc084f38640 1 --2- 172.21.15.47:0/1927022671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0800cad40 0x7fc0800b0de0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.160+0000 7fc084f38640 1 -- 172.21.15.47:0/1927022671 >> 172.21.15.47:0/1927022671 conn(0x7fc0800d06f0 msgr2=0x7fc0800d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.160+0000 7fc084f38640 1 -- 172.21.15.47:0/1927022671 shutdown_connections 2024-03-23T09:42:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.160+0000 7fc084f38640 1 -- 172.21.15.47:0/1927022671 wait complete. 2024-03-23T09:42:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.162+0000 7fc084f38640 1 Processor -- start 2024-03-23T09:42:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.162+0000 7fc084f38640 1 -- start start 2024-03-23T09:42:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.162+0000 7fc084f38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0800cad40 0x7fc0800c4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.162+0000 7fc084f38640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0800cd680 0x7fc080158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.162+0000 7fc084f38640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc08015cbe0 0x7fc08015d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.162+0000 7fc084f38640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08015dc20 con 0x7fc0800cd680 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.162+0000 7fc084f38640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0800c8110 con 0x7fc08015cbe0 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.162+0000 7fc084f38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0800c83f0 con 0x7fc0800cad40 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc07dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0800cd680 0x7fc080158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc07dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0800cd680 0x7fc080158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39460/0 (socket says 172.21.15.47:39460) 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc07dd9b640 1 -- 172.21.15.47:0/2707950576 learned_addr learned my addr 172.21.15.47:0/2707950576 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc07ed9d640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc08015cbe0 0x7fc08015d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc07e59c640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0800cad40 0x7fc0800c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc07dd9b640 1 -- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0800cad40 msgr2=0x7fc0800c4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc07dd9b640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0800cad40 0x7fc0800c4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc07dd9b640 1 -- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc08015cbe0 msgr2=0x7fc08015d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc07dd9b640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc08015cbe0 0x7fc08015d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc07dd9b640 1 -- 172.21.15.47:0/2707950576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc070009a70 con 0x7fc0800cd680 2024-03-23T09:42:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc07e59c640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0800cad40 0x7fc0800c4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc07dd9b640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0800cd680 0x7fc080158da0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc06800c4a0 tx=0x7fc06800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc076ffd640 1 -- 172.21.15.47:0/2707950576 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0680074e0 con 0x7fc0800cd680 2024-03-23T09:42:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc076ffd640 1 -- 172.21.15.47:0/2707950576 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc068007640 con 0x7fc0800cd680 2024-03-23T09:42:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc076ffd640 1 -- 172.21.15.47:0/2707950576 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068004a00 con 0x7fc0800cd680 2024-03-23T09:42:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc084f38640 1 -- 172.21.15.47:0/2707950576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0800c86d0 con 0x7fc0800cd680 2024-03-23T09:42:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.163+0000 7fc084f38640 1 -- 172.21.15.47:0/2707950576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0800ca1e0 con 0x7fc0800cd680 2024-03-23T09:42:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.164+0000 7fc084f38640 1 -- 172.21.15.47:0/2707950576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc080066be0 con 0x7fc0800cd680 2024-03-23T09:42:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.165+0000 7fc076ffd640 1 -- 172.21.15.47:0/2707950576 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc068004b60 con 0x7fc0800cd680 2024-03-23T09:42:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.165+0000 7fc076ffd640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc06006ac00 0x7fc06006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.166+0000 7fc076ffd640 1 -- 172.21.15.47:0/2707950576 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc068088540 con 0x7fc0800cd680 2024-03-23T09:42:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.166+0000 7fc07e59c640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc06006ac00 0x7fc06006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.166+0000 7fc07e59c640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc06006ac00 0x7fc06006d0c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc0801f5500 tx=0x7fc078009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:22.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.170+0000 7fc076ffd640 1 -- 172.21.15.47:0/2707950576 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc068053730 con 0x7fc0800cd680 2024-03-23T09:42:22.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.387+0000 7fc084f38640 1 -- 172.21.15.47:0/2707950576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc080098f30 con 0x7fc0800cd680 2024-03-23T09:42:22.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.388+0000 7fc076ffd640 1 -- 172.21.15.47:0/2707950576 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fc068056d50 con 0x7fc0800cd680 2024-03-23T09:42:22.388 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:22.388 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3373309448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3373309448},{"type":"v1","addr":"172.21.15.175:6836","nonce":3373309448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4118066020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4118066020},{"type":"v1","addr":"172.21.15.175:6837","nonce":4118066020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2916649516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2916649516},{"type":"v1","addr":"172.21.15.47:6835","nonce":2916649516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:59.719783+0000","modified":"2024-03-23T09:42:01.774130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6834/400299689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":400299689},{"type":"v1","addr":"172.21.15.47:6834","nonce":400299689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T09:42:22.388 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2024-03-23T09:42:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.390+0000 7fc084f38640 1 -- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc06006ac00 msgr2=0x7fc06006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.390+0000 7fc084f38640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc06006ac00 0x7fc06006d0c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc0801f5500 tx=0x7fc078009040 comp rx=0 tx=0).stop 2024-03-23T09:42:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.390+0000 7fc084f38640 1 -- 172.21.15.47:0/2707950576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0800cd680 msgr2=0x7fc080158da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.390+0000 7fc084f38640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0800cd680 0x7fc080158da0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc06800c4a0 tx=0x7fc06800c960 comp rx=0 tx=0).stop 2024-03-23T09:42:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.390+0000 7fc084f38640 1 -- 172.21.15.47:0/2707950576 shutdown_connections 2024-03-23T09:42:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.390+0000 7fc084f38640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc06006ac00 0x7fc06006d0c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.390+0000 7fc084f38640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc08015cbe0 0x7fc08015d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.390+0000 7fc084f38640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0800cd680 0x7fc080158da0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.390+0000 7fc084f38640 1 --2- 172.21.15.47:0/2707950576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0800cad40 0x7fc0800c4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.390+0000 7fc084f38640 1 -- 172.21.15.47:0/2707950576 >> 172.21.15.47:0/2707950576 conn(0x7fc0800d06f0 msgr2=0x7fc080099450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.390+0000 7fc084f38640 1 -- 172.21.15.47:0/2707950576 shutdown_connections 2024-03-23T09:42:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.390+0000 7fc084f38640 1 -- 172.21.15.47:0/2707950576 wait complete. 2024-03-23T09:42:22.400 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-23T09:42:22.522 INFO:teuthology.orchestra.run.smithi047.stdout:{"message":"","return_code":0} 2024-03-23T09:42:22.523 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-23T09:42:22.523 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:42:22.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.624+0000 7f024d106640 1 Processor -- start 2024-03-23T09:42:22.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.624+0000 7f024d106640 1 -- start start 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024d106640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815cb40 0x7f0248160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024d106640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02480d0ee0 con 0x7f0248160cb0 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024d106640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02481616f0 con 0x7f02480b8300 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024d106640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02480b5a10 con 0x7f024815cb40 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024759e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0248160cb0 0x7f02480d1370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38202/0 (socket says 172.21.15.47:38202) 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024759e640 1 -- 172.21.15.47:0/570692951 learned_addr learned my addr 172.21.15.47:0/570692951 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908637850 0 0) 0x7f02480d0ee0 con 0x7f0248160cb0 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0234003520 con 0x7f0248160cb0 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f0246d9d640 1 --2- 172.21.15.47:0/570692951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815cb40 0x7f0248160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 14190346 0 0) 0x7f0234003520 con 0x7f0248160cb0 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815cb40 msgr2=0x7f0248160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024659c640 1 --2- 172.21.15.47:0/570692951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815cb40 0x7f0248160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 >> v1:172.21.15.175:6789/0 conn(0x7f02480b8300 legacy=0x7f02480cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0248161130 con 0x7f0248160cb0 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f02400019f0 con 0x7f0248160cb0 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.625+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 >> v1:172.21.15.47:6789/0 conn(0x7f0248160cb0 legacy=0x7f02480d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f024659c640 1 --2- 172.21.15.47:0/570692951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02340039a0 0x7f0234003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f024659c640 1 --2- 172.21.15.47:0/570692951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02340047c0 0x7f0234004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f024659c640 1 --2- 172.21.15.47:0/570692951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815cb40 0x7f02340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02480b5a10 con 0x7f02340039a0 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023400bab0 con 0x7f02340047c0 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023400bc70 con 0x7f024815cb40 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f0246d9d640 1 --2- 172.21.15.47:0/570692951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02340039a0 0x7f0234003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f023ffff640 1 --2- 172.21.15.47:0/570692951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02340047c0 0x7f0234004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f024759e640 1 --2- 172.21.15.47:0/570692951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815cb40 0x7f02340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f0246d9d640 1 -- 172.21.15.47:0/570692951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815cb40 msgr2=0x7f02340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f0246d9d640 1 --2- 172.21.15.47:0/570692951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815cb40 0x7f02340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f0246d9d640 1 -- 172.21.15.47:0/570692951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02340047c0 msgr2=0x7f0234004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f0246d9d640 1 --2- 172.21.15.47:0/570692951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02340047c0 0x7f0234004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f0246d9d640 1 -- 172.21.15.47:0/570692951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0234003520 con 0x7f02340039a0 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.626+0000 7f024759e640 1 --2- 172.21.15.47:0/570692951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815cb40 0x7f02340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.627+0000 7f0246d9d640 1 --2- 172.21.15.47:0/570692951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02340039a0 0x7f0234003dc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f023800ba30 tx=0x7f02340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d145e0e542bb42 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.627+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238032640 con 0x7f02340039a0 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.627+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02380327a0 con 0x7f02340039a0 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.627+0000 7f024659c640 1 -- 172.21.15.47:0/570692951 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238032b30 con 0x7f02340039a0 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.627+0000 7f024d106640 1 -- 172.21.15.47:0/570692951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02340039a0 msgr2=0x7f0234003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.627+0000 7f024d106640 1 --2- 172.21.15.47:0/570692951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02340039a0 0x7f0234003dc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f023800ba30 tx=0x7f02340057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.627+0000 7f024759e640 1 -- 172.21.15.47:0/570692951 reap_dead start 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.627+0000 7f024d106640 1 -- 172.21.15.47:0/570692951 shutdown_connections 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.627+0000 7f024d106640 1 -- 172.21.15.47:0/570692951 >> 172.21.15.47:0/570692951 conn(0x7f024806b160 msgr2=0x7f024806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.627+0000 7f024d106640 1 -- 172.21.15.47:0/570692951 shutdown_connections 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.627+0000 7f024d106640 1 -- 172.21.15.47:0/570692951 wait complete. 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.629+0000 7f024d106640 1 Processor -- start 2024-03-23T09:42:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.629+0000 7f024d106640 1 -- start start 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.629+0000 7f024d106640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02340039a0 0x7f02481f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.629+0000 7f024d106640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815a170 0x7f02480cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.629+0000 7f024d106640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0248158ce0 0x7f0248159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.629+0000 7f024d106640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02481f69f0 con 0x7f02340039a0 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.629+0000 7f024d106640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024815e690 con 0x7f0248158ce0 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.629+0000 7f024d106640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024815e970 con 0x7f024815a170 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f0246d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02340039a0 0x7f02481f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f0246d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02340039a0 0x7f02481f64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39480/0 (socket says 172.21.15.47:39480) 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f0246d9d640 1 -- 172.21.15.47:0/398353485 learned_addr learned my addr 172.21.15.47:0/398353485 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f023ffff640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815a170 0x7f02480cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f024759e640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0248158ce0 0x7f0248159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f0246d9d640 1 -- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815a170 msgr2=0x7f02480cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f0246d9d640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815a170 0x7f02480cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f0246d9d640 1 -- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0248158ce0 msgr2=0x7f0248159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f0246d9d640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0248158ce0 0x7f0248159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f0246d9d640 1 -- 172.21.15.47:0/398353485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024815ec50 con 0x7f02340039a0 2024-03-23T09:42:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f023ffff640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815a170 0x7f02480cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f024759e640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0248158ce0 0x7f0248159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f0246d9d640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02340039a0 0x7f02481f64b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f023802bec0 tx=0x7f0238004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f023f7fe640 1 -- 172.21.15.47:0/398353485 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023803ba70 con 0x7f02340039a0 2024-03-23T09:42:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f023f7fe640 1 -- 172.21.15.47:0/398353485 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023803bbd0 con 0x7f02340039a0 2024-03-23T09:42:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f024d106640 1 -- 172.21.15.47:0/398353485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024815ef30 con 0x7f02340039a0 2024-03-23T09:42:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f023f7fe640 1 -- 172.21.15.47:0/398353485 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238030410 con 0x7f02340039a0 2024-03-23T09:42:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.630+0000 7f024d106640 1 -- 172.21.15.47:0/398353485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02481596a0 con 0x7f02340039a0 2024-03-23T09:42:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.631+0000 7f024d106640 1 -- 172.21.15.47:0/398353485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0248072360 con 0x7f02340039a0 2024-03-23T09:42:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.632+0000 7f023f7fe640 1 -- 172.21.15.47:0/398353485 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0238030570 con 0x7f02340039a0 2024-03-23T09:42:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.632+0000 7f023f7fe640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f022806ac00 0x7f022806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.632+0000 7f023f7fe640 1 -- 172.21.15.47:0/398353485 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02380abaf0 con 0x7f02340039a0 2024-03-23T09:42:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.632+0000 7f023ffff640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f022806ac00 0x7f022806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.633+0000 7f023ffff640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f022806ac00 0x7f022806d0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0230000c00 tx=0x7f0230009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.636+0000 7f023f7fe640 1 -- 172.21.15.47:0/398353485 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0238076de0 con 0x7f02340039a0 2024-03-23T09:42:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.850+0000 7f024d106640 1 -- 172.21.15.47:0/398353485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0248066be0 con 0x7f02340039a0 2024-03-23T09:42:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.851+0000 7f023f7fe640 1 -- 172.21.15.47:0/398353485 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f023807a400 con 0x7f02340039a0 2024-03-23T09:42:22.851 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:22.851 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3373309448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3373309448},{"type":"v1","addr":"172.21.15.175:6836","nonce":3373309448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4118066020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4118066020},{"type":"v1","addr":"172.21.15.175:6837","nonce":4118066020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2916649516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2916649516},{"type":"v1","addr":"172.21.15.47:6835","nonce":2916649516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:59.719783+0000","modified":"2024-03-23T09:42:01.774130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6834/400299689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":400299689},{"type":"v1","addr":"172.21.15.47:6834","nonce":400299689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T09:42:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2024-03-23T09:42:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.853+0000 7f024d106640 1 -- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f022806ac00 msgr2=0x7f022806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.853+0000 7f024d106640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f022806ac00 0x7f022806d0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0230000c00 tx=0x7f0230009040 comp rx=0 tx=0).stop 2024-03-23T09:42:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.853+0000 7f024d106640 1 -- 172.21.15.47:0/398353485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02340039a0 msgr2=0x7f02481f64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.853+0000 7f024d106640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02340039a0 0x7f02481f64b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f023802bec0 tx=0x7f0238004730 comp rx=0 tx=0).stop 2024-03-23T09:42:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.853+0000 7f024d106640 1 -- 172.21.15.47:0/398353485 shutdown_connections 2024-03-23T09:42:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.853+0000 7f024d106640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f022806ac00 0x7f022806d0c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.853+0000 7f024d106640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0248158ce0 0x7f0248159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.853+0000 7f024d106640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f024815a170 0x7f02480cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.853+0000 7f024d106640 1 --2- 172.21.15.47:0/398353485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02340039a0 0x7f02481f64b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.853+0000 7f024d106640 1 -- 172.21.15.47:0/398353485 >> 172.21.15.47:0/398353485 conn(0x7f024806b160 msgr2=0x7f02480c6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.853+0000 7f024d106640 1 -- 172.21.15.47:0/398353485 shutdown_connections 2024-03-23T09:42:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:22.853+0000 7f024d106640 1 -- 172.21.15.47:0/398353485 wait complete. 2024-03-23T09:42:22.863 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2024-03-23T09:42:22.982 INFO:teuthology.orchestra.run.smithi047.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-03-23T09:42:22.982 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-03-23T09:42:32.983 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-03-23T09:42:33.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.031+0000 7f977663cf40 1 Processor -- start 2024-03-23T09:42:33.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.031+0000 7f977663cf40 1 -- start start 2024-03-23T09:42:33.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.032+0000 7f977663cf40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560f4da8140 0x5560f4d93920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.032+0000 7f977663cf40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5560f4da8c30 0x5560f4da0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.032+0000 7f977663cf40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5560f4daad20 0x5560f4da1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.032+0000 7f977663cf40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f4c804d0 con 0x5560f4da8140 2024-03-23T09:42:33.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.032+0000 7f977663cf40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f4c1bb60 con 0x5560f4daad20 2024-03-23T09:42:33.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.032+0000 7f977663cf40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560f4c1b800 con 0x5560f4da8c30 2024-03-23T09:42:33.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.032+0000 7f977388a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560f4da8140 0x5560f4d93920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.032+0000 7f977388a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560f4da8140 0x5560f4d93920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45614/0 (socket says 172.21.15.47:45614) 2024-03-23T09:42:33.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.032+0000 7f977388a640 1 -- 172.21.15.47:0/2811920185 learned_addr learned my addr 172.21.15.47:0/2811920185 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:33.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f9773089640 1 --2- 172.21.15.47:0/2811920185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5560f4da8c30 0x5560f4da0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f977408b640 1 --2- 172.21.15.47:0/2811920185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5560f4daad20 0x5560f4da1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f977388a640 1 -- 172.21.15.47:0/2811920185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5560f4da8c30 msgr2=0x5560f4da0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f977388a640 1 --2- 172.21.15.47:0/2811920185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5560f4da8c30 0x5560f4da0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f977388a640 1 -- 172.21.15.47:0/2811920185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5560f4daad20 msgr2=0x5560f4da1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f977388a640 1 --2- 172.21.15.47:0/2811920185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5560f4daad20 0x5560f4da1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f977388a640 1 -- 172.21.15.47:0/2811920185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c00a100 con 0x5560f4da8140 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f977408b640 1 --2- 172.21.15.47:0/2811920185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5560f4daad20 0x5560f4da1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f977388a640 1 --2- 172.21.15.47:0/2811920185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560f4da8140 0x5560f4d93920 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9764002e10 tx=0x7f976400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffae30d10b592976 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f9772087640 1 -- 172.21.15.47:0/2811920185 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976400f400 con 0x5560f4da8140 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f9772087640 1 -- 172.21.15.47:0/2811920185 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9764008030 con 0x5560f4da8140 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f9772087640 1 -- 172.21.15.47:0/2811920185 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976400f7a0 con 0x5560f4da8140 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f977663cf40 1 -- 172.21.15.47:0/2811920185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560f4da8140 msgr2=0x5560f4d93920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.033+0000 7f977663cf40 1 --2- 172.21.15.47:0/2811920185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560f4da8140 0x5560f4d93920 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9764002e10 tx=0x7f976400b5b0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.034+0000 7f977663cf40 1 -- 172.21.15.47:0/2811920185 shutdown_connections 2024-03-23T09:42:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.034+0000 7f977663cf40 1 --2- 172.21.15.47:0/2811920185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5560f4daad20 0x5560f4da1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.034+0000 7f977663cf40 1 --2- 172.21.15.47:0/2811920185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5560f4da8c30 0x5560f4da0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.034+0000 7f977663cf40 1 --2- 172.21.15.47:0/2811920185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560f4da8140 0x5560f4d93920 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.034+0000 7f977663cf40 1 -- 172.21.15.47:0/2811920185 >> 172.21.15.47:0/2811920185 conn(0x5560f4cfee40 msgr2=0x5560f4d99b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.034+0000 7f977663cf40 1 -- 172.21.15.47:0/2811920185 shutdown_connections 2024-03-23T09:42:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.034+0000 7f977663cf40 1 -- 172.21.15.47:0/2811920185 wait complete. 2024-03-23T09:42:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.034+0000 7f977663cf40 1 Processor -- start 2024-03-23T09:42:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.034+0000 7f977663cf40 1 -- start start 2024-03-23T09:42:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f977663cf40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5560f4da8140 0x5560f4d8a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f977663cf40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560f4da8c30 0x5560f4da0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f977663cf40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5560f4daad20 0x5560f4d93920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f977663cf40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f4bfe520 con 0x5560f4da8c30 2024-03-23T09:42:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f977663cf40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f4d9fcd0 con 0x5560f4da8140 2024-03-23T09:42:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f977663cf40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560f4d9ffb0 con 0x5560f4daad20 2024-03-23T09:42:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f9773089640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560f4da8c30 0x5560f4da0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f9773089640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560f4da8c30 0x5560f4da0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45618/0 (socket says 172.21.15.47:45618) 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f9773089640 1 -- 172.21.15.47:0/3999541832 learned_addr learned my addr 172.21.15.47:0/3999541832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f977408b640 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5560f4daad20 0x5560f4d93920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f977388a640 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5560f4da8140 0x5560f4d8a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f9773089640 1 -- 172.21.15.47:0/3999541832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5560f4daad20 msgr2=0x5560f4d93920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f9773089640 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5560f4daad20 0x5560f4d93920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f9773089640 1 -- 172.21.15.47:0/3999541832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5560f4da8140 msgr2=0x5560f4d8a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f9773089640 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5560f4da8140 0x5560f4d8a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.035+0000 7f9773089640 1 -- 172.21.15.47:0/3999541832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976400a040 con 0x5560f4da8c30 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.036+0000 7f977408b640 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5560f4daad20 0x5560f4d93920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.036+0000 7f977388a640 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5560f4da8140 0x5560f4d8a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.036+0000 7f9773089640 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560f4da8c30 0x5560f4da0d70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f976800ce20 tx=0x7f976800a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.036+0000 7f9771085640 1 -- 172.21.15.47:0/3999541832 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976800ada0 con 0x5560f4da8c30 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.036+0000 7f9771085640 1 -- 172.21.15.47:0/3999541832 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976800f400 con 0x5560f4da8c30 2024-03-23T09:42:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.036+0000 7f977663cf40 1 -- 172.21.15.47:0/3999541832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5560f4c1e060 con 0x5560f4da8c30 2024-03-23T09:42:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.036+0000 7f9771085640 1 -- 172.21.15.47:0/3999541832 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976800f7b0 con 0x5560f4da8c30 2024-03-23T09:42:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.036+0000 7f977663cf40 1 -- 172.21.15.47:0/3999541832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560f4c1dac0 con 0x5560f4da8c30 2024-03-23T09:42:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.038+0000 7f9771085640 1 -- 172.21.15.47:0/3999541832 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f976800d040 con 0x5560f4da8c30 2024-03-23T09:42:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.038+0000 7f9771085640 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f97600695e0 0x7f976006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.038+0000 7f9771085640 1 -- 172.21.15.47:0/3999541832 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9768087db0 con 0x5560f4da8c30 2024-03-23T09:42:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.038+0000 7f977663cf40 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x5560f4d9cfc0 0x5560f4d916b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.038+0000 7f977663cf40 1 -- 172.21.15.47:0/3999541832 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x5560f4da12b0 con 0x5560f4d9cfc0 2024-03-23T09:42:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.038+0000 7f977388a640 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f97600695e0 0x7f976006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.038+0000 7f977408b640 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x5560f4d9cfc0 0x5560f4d916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977408b640 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x5560f4d9cfc0 0x5560f4d916b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977388a640 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f97600695e0 0x7f976006baa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9764002d30 tx=0x7f9764009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977408b640 1 -- 172.21.15.47:0/3999541832 <== osd.3 v2:172.21.15.47:6808/37683 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f976c0529a0 con 0x5560f4d9cfc0 2024-03-23T09:42:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977663cf40 1 -- 172.21.15.47:0/3999541832 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x5560f4d9cfc0 msgr2=0x5560f4d916b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977663cf40 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x5560f4d9cfc0 0x5560f4d916b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977663cf40 1 -- 172.21.15.47:0/3999541832 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f97600695e0 msgr2=0x7f976006baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977663cf40 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f97600695e0 0x7f976006baa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9764002d30 tx=0x7f9764009000 comp rx=0 tx=0).stop 2024-03-23T09:42:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977663cf40 1 -- 172.21.15.47:0/3999541832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560f4da8c30 msgr2=0x5560f4da0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977663cf40 1 --2- 172.21.15.47:0/3999541832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560f4da8c30 0x5560f4da0d70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f976800ce20 tx=0x7f976800a5d0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977408b640 1 -- 172.21.15.47:0/3999541832 reap_dead start 2024-03-23T09:42:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977663cf40 1 -- 172.21.15.47:0/3999541832 shutdown_connections 2024-03-23T09:42:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977663cf40 1 -- 172.21.15.47:0/3999541832 >> 172.21.15.47:0/3999541832 conn(0x5560f4cfee40 msgr2=0x5560f4d99b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977663cf40 1 -- 172.21.15.47:0/3999541832 shutdown_connections 2024-03-23T09:42:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.039+0000 7f977663cf40 1 -- 172.21.15.47:0/3999541832 wait complete. 2024-03-23T09:42:33.044 INFO:teuthology.orchestra.run.smithi047.stdout:mytag 2024-03-23T09:42:33.044 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-03-23T09:42:33.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.088+0000 7f902f4d9f40 1 Processor -- start 2024-03-23T09:42:33.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.088+0000 7f902f4d9f40 1 -- start start 2024-03-23T09:42:33.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.089+0000 7f902f4d9f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab5c5812d0 0x55ab5c60fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.089+0000 7f902f4d9f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab5c5f19e0 0x55ab5c5f1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.089+0000 7f902f4d9f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ab5c60cba0 0x55ab5c5ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.089+0000 7f902f4d9f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5c47fb60 con 0x55ab5c5f19e0 2024-03-23T09:42:33.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.089+0000 7f902f4d9f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5c47f800 con 0x55ab5c5812d0 2024-03-23T09:42:33.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.089+0000 7f902f4d9f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab5c462520 con 0x55ab5c60cba0 2024-03-23T09:42:33.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.089+0000 7f9027fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab5c5f19e0 0x55ab5c5f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.089+0000 7f9027fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab5c5f19e0 0x55ab5c5f1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45644/0 (socket says 172.21.15.47:45644) 2024-03-23T09:42:33.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.089+0000 7f9027fff640 1 -- 172.21.15.47:0/1118782466 learned_addr learned my addr 172.21.15.47:0/1118782466 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:33.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.089+0000 7f902d7d7640 1 --2- 172.21.15.47:0/1118782466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ab5c60cba0 0x55ab5c5ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.089+0000 7f902cfd6640 1 --2- 172.21.15.47:0/1118782466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab5c5812d0 0x55ab5c60fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.090+0000 7f9027fff640 1 -- 172.21.15.47:0/1118782466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ab5c60cba0 msgr2=0x55ab5c5ebfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:33.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.090+0000 7f9027fff640 1 --2- 172.21.15.47:0/1118782466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ab5c60cba0 0x55ab5c5ebfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.090+0000 7f9027fff640 1 -- 172.21.15.47:0/1118782466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab5c5812d0 msgr2=0x55ab5c60fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:33.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.090+0000 7f9027fff640 1 --2- 172.21.15.47:0/1118782466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab5c5812d0 0x55ab5c60fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.090+0000 7f9027fff640 1 -- 172.21.15.47:0/1118782466 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902800a100 con 0x55ab5c5f19e0 2024-03-23T09:42:33.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.090+0000 7f9027fff640 1 --2- 172.21.15.47:0/1118782466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab5c5f19e0 0x55ab5c5f1170 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f901800b2f0 tx=0x7f901800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2752676e89875e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.090+0000 7f9026ffd640 1 -- 172.21.15.47:0/1118782466 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018014070 con 0x55ab5c5f19e0 2024-03-23T09:42:33.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.090+0000 7f9026ffd640 1 -- 172.21.15.47:0/1118782466 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901800d500 con 0x55ab5c5f19e0 2024-03-23T09:42:33.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.090+0000 7f9026ffd640 1 -- 172.21.15.47:0/1118782466 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901800d890 con 0x55ab5c5f19e0 2024-03-23T09:42:33.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.090+0000 7f902f4d9f40 1 -- 172.21.15.47:0/1118782466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab5c5f19e0 msgr2=0x55ab5c5f1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.090+0000 7f902f4d9f40 1 --2- 172.21.15.47:0/1118782466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab5c5f19e0 0x55ab5c5f1170 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f901800b2f0 tx=0x7f901800b7b0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.090+0000 7f902f4d9f40 1 -- 172.21.15.47:0/1118782466 shutdown_connections 2024-03-23T09:42:33.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.091+0000 7f902f4d9f40 1 --2- 172.21.15.47:0/1118782466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ab5c60cba0 0x55ab5c5ebfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.091+0000 7f902f4d9f40 1 --2- 172.21.15.47:0/1118782466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab5c5f19e0 0x55ab5c5f1170 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.091+0000 7f902f4d9f40 1 --2- 172.21.15.47:0/1118782466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab5c5812d0 0x55ab5c60fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.091+0000 7f902f4d9f40 1 -- 172.21.15.47:0/1118782466 >> 172.21.15.47:0/1118782466 conn(0x55ab5c562e40 msgr2=0x55ab5c60b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:33.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.091+0000 7f902f4d9f40 1 -- 172.21.15.47:0/1118782466 shutdown_connections 2024-03-23T09:42:33.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.091+0000 7f902f4d9f40 1 -- 172.21.15.47:0/1118782466 wait complete. 2024-03-23T09:42:33.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.091+0000 7f902f4d9f40 1 Processor -- start 2024-03-23T09:42:33.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.091+0000 7f902f4d9f40 1 -- start start 2024-03-23T09:42:33.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f902f4d9f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ab5c5812d0 0x55ab5c5e9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f902f4d9f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab5c5f19e0 0x55ab5c580090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f902f4d9f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab5c60cba0 0x55ab5c608640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f902f4d9f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5c6174f0 con 0x55ab5c5f19e0 2024-03-23T09:42:33.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f902f4d9f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5c6169d0 con 0x55ab5c60cba0 2024-03-23T09:42:33.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f902f4d9f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab5c616c50 con 0x55ab5c5812d0 2024-03-23T09:42:33.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f9027fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab5c5f19e0 0x55ab5c580090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f9027fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab5c5f19e0 0x55ab5c580090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45654/0 (socket says 172.21.15.47:45654) 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f9027fff640 1 -- 172.21.15.47:0/331682879 learned_addr learned my addr 172.21.15.47:0/331682879 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f902cfd6640 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ab5c5812d0 0x55ab5c5e9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f902d7d7640 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab5c60cba0 0x55ab5c608640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f9027fff640 1 -- 172.21.15.47:0/331682879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ab5c5812d0 msgr2=0x55ab5c5e9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f9027fff640 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ab5c5812d0 0x55ab5c5e9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f9027fff640 1 -- 172.21.15.47:0/331682879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab5c60cba0 msgr2=0x55ab5c608640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f9027fff640 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab5c60cba0 0x55ab5c608640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.092+0000 7f9027fff640 1 -- 172.21.15.47:0/331682879 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9018001b40 con 0x55ab5c5f19e0 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.093+0000 7f9027fff640 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab5c5f19e0 0x55ab5c580090 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9018001ed0 tx=0x7f9018000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.093+0000 7f9025ffb640 1 -- 172.21.15.47:0/331682879 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018014070 con 0x55ab5c5f19e0 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.093+0000 7f9025ffb640 1 -- 172.21.15.47:0/331682879 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901801c650 con 0x55ab5c5f19e0 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.093+0000 7f9025ffb640 1 -- 172.21.15.47:0/331682879 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901801c7c0 con 0x55ab5c5f19e0 2024-03-23T09:42:33.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.093+0000 7f902f4d9f40 1 -- 172.21.15.47:0/331682879 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ab5c48ec20 con 0x55ab5c5f19e0 2024-03-23T09:42:33.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.093+0000 7f902f4d9f40 1 -- 172.21.15.47:0/331682879 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab5c482060 con 0x55ab5c5f19e0 2024-03-23T09:42:33.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.094+0000 7f9025ffb640 1 -- 172.21.15.47:0/331682879 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f901800d040 con 0x55ab5c5f19e0 2024-03-23T09:42:33.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.095+0000 7f9025ffb640 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90100695e0 0x7f901006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.095+0000 7f9025ffb640 1 -- 172.21.15.47:0/331682879 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f901800a040 con 0x55ab5c5f19e0 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.095+0000 7f902cfd6640 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90100695e0 0x7f901006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.095+0000 7f902f4d9f40 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55ab5c5805d0 0x55ab5c566d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.095+0000 7f902f4d9f40 1 -- 172.21.15.47:0/331682879 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x55ab5c5f4900 con 0x55ab5c5805d0 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.095+0000 7f902d7d7640 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55ab5c5805d0 0x55ab5c566d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.095+0000 7f902cfd6640 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90100695e0 0x7f901006baa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f90200098e0 tx=0x7f902000c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.095+0000 7f902d7d7640 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55ab5c5805d0 0x55ab5c566d90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.096+0000 7f902d7d7640 1 -- 172.21.15.47:0/331682879 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f90280527a0 con 0x55ab5c5805d0 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.096+0000 7f902f4d9f40 1 -- 172.21.15.47:0/331682879 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55ab5c5805d0 msgr2=0x55ab5c566d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.096+0000 7f902f4d9f40 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55ab5c5805d0 0x55ab5c566d90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.096+0000 7f902f4d9f40 1 -- 172.21.15.47:0/331682879 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90100695e0 msgr2=0x7f901006baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.096+0000 7f902f4d9f40 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90100695e0 0x7f901006baa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f90200098e0 tx=0x7f902000c040 comp rx=0 tx=0).stop 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.096+0000 7f902f4d9f40 1 -- 172.21.15.47:0/331682879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab5c5f19e0 msgr2=0x55ab5c580090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.096+0000 7f902f4d9f40 1 --2- 172.21.15.47:0/331682879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab5c5f19e0 0x55ab5c580090 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9018001ed0 tx=0x7f9018000ed0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.096+0000 7f902d7d7640 1 -- 172.21.15.47:0/331682879 reap_dead start 2024-03-23T09:42:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.096+0000 7f902f4d9f40 1 -- 172.21.15.47:0/331682879 shutdown_connections 2024-03-23T09:42:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.096+0000 7f902f4d9f40 1 -- 172.21.15.47:0/331682879 >> 172.21.15.47:0/331682879 conn(0x55ab5c562e40 msgr2=0x55ab5c5ebfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.096+0000 7f902f4d9f40 1 -- 172.21.15.47:0/331682879 shutdown_connections 2024-03-23T09:42:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.096+0000 7f902f4d9f40 1 -- 172.21.15.47:0/331682879 wait complete. 2024-03-23T09:42:33.099 INFO:teuthology.orchestra.run.smithi047.stdout:mytag 2024-03-23T09:42:33.100 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-03-23T09:42:33.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.183+0000 7f161f036f40 1 Processor -- start 2024-03-23T09:42:33.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.183+0000 7f161f036f40 1 -- start start 2024-03-23T09:42:33.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.183+0000 7f161f036f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556a00f12ab0 0x556a00f02ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.183+0000 7f161f036f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a00f1ab00 0x556a00f01f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.183+0000 7f161f036f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a00fa9b20 0x556a00efff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.183+0000 7f161f036f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a00e1ab60 con 0x556a00fa9b20 2024-03-23T09:42:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.183+0000 7f161f036f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a00e1a800 con 0x556a00f1ab00 2024-03-23T09:42:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.183+0000 7f161f036f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a00dfd520 con 0x556a00f12ab0 2024-03-23T09:42:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161cbdc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a00fa9b20 0x556a00efff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161cbdc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a00fa9b20 0x556a00efff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45666/0 (socket says 172.21.15.47:45666) 2024-03-23T09:42:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161cbdc640 1 -- 172.21.15.47:0/3756464938 learned_addr learned my addr 172.21.15.47:0/3756464938 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161c3db640 1 --2- 172.21.15.47:0/3756464938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556a00f12ab0 0x556a00f02ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161bbda640 1 --2- 172.21.15.47:0/3756464938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a00f1ab00 0x556a00f01f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161cbdc640 1 -- 172.21.15.47:0/3756464938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556a00f12ab0 msgr2=0x556a00f02ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161cbdc640 1 --2- 172.21.15.47:0/3756464938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556a00f12ab0 0x556a00f02ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161cbdc640 1 -- 172.21.15.47:0/3756464938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a00f1ab00 msgr2=0x556a00f01f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161cbdc640 1 --2- 172.21.15.47:0/3756464938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a00f1ab00 0x556a00f01f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161cbdc640 1 -- 172.21.15.47:0/3756464938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1610005d20 con 0x556a00fa9b20 2024-03-23T09:42:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161cbdc640 1 --2- 172.21.15.47:0/3756464938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a00fa9b20 0x556a00efff20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1614001710 tx=0x7f1614001bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851f9f50a9ac92dd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161abd8640 1 -- 172.21.15.47:0/3756464938 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614003650 con 0x556a00fa9b20 2024-03-23T09:42:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161abd8640 1 -- 172.21.15.47:0/3756464938 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16140037b0 con 0x556a00fa9b20 2024-03-23T09:42:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.184+0000 7f161abd8640 1 -- 172.21.15.47:0/3756464938 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614003b20 con 0x556a00fa9b20 2024-03-23T09:42:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.185+0000 7f161f036f40 1 -- 172.21.15.47:0/3756464938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a00fa9b20 msgr2=0x556a00efff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.185+0000 7f161f036f40 1 --2- 172.21.15.47:0/3756464938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a00fa9b20 0x556a00efff20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1614001710 tx=0x7f1614001bd0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.185+0000 7f161f036f40 1 -- 172.21.15.47:0/3756464938 shutdown_connections 2024-03-23T09:42:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.185+0000 7f161f036f40 1 --2- 172.21.15.47:0/3756464938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a00fa9b20 0x556a00efff20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.185+0000 7f161f036f40 1 --2- 172.21.15.47:0/3756464938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a00f1ab00 0x556a00f01f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.185+0000 7f161f036f40 1 --2- 172.21.15.47:0/3756464938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556a00f12ab0 0x556a00f02ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.185+0000 7f161f036f40 1 -- 172.21.15.47:0/3756464938 >> 172.21.15.47:0/3756464938 conn(0x556a00efde40 msgr2=0x556a00efe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.185+0000 7f161f036f40 1 -- 172.21.15.47:0/3756464938 shutdown_connections 2024-03-23T09:42:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.185+0000 7f161f036f40 1 -- 172.21.15.47:0/3756464938 wait complete. 2024-03-23T09:42:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.185+0000 7f161f036f40 1 Processor -- start 2024-03-23T09:42:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.185+0000 7f161f036f40 1 -- start start 2024-03-23T09:42:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.186+0000 7f161f036f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a00f12ab0 0x556a00fa6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.186+0000 7f161f036f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a00f1ab00 0x556a00ec0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.186+0000 7f161f036f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556a00fa9b20 0x556a00fa7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.186+0000 7f161f036f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a00f9e900 con 0x556a00f1ab00 2024-03-23T09:42:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.186+0000 7f161f036f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a00fa0ce0 con 0x556a00f12ab0 2024-03-23T09:42:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.186+0000 7f161f036f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a00f9eba0 con 0x556a00fa9b20 2024-03-23T09:42:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.186+0000 7f161bbda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a00f1ab00 0x556a00ec0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.186+0000 7f161bbda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a00f1ab00 0x556a00ec0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45676/0 (socket says 172.21.15.47:45676) 2024-03-23T09:42:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.186+0000 7f161bbda640 1 -- 172.21.15.47:0/3799788121 learned_addr learned my addr 172.21.15.47:0/3799788121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.186+0000 7f161cbdc640 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556a00fa9b20 0x556a00fa7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.186+0000 7f161c3db640 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a00f12ab0 0x556a00fa6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.187+0000 7f161bbda640 1 -- 172.21.15.47:0/3799788121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556a00fa9b20 msgr2=0x556a00fa7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.187+0000 7f161bbda640 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556a00fa9b20 0x556a00fa7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.187+0000 7f161bbda640 1 -- 172.21.15.47:0/3799788121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a00f12ab0 msgr2=0x556a00fa6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.187+0000 7f161bbda640 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a00f12ab0 0x556a00fa6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.187+0000 7f161bbda640 1 -- 172.21.15.47:0/3799788121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614006cd0 con 0x556a00f1ab00 2024-03-23T09:42:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.187+0000 7f161bbda640 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a00f1ab00 0x556a00ec0f60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1610000c00 tx=0x7f1610002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.187+0000 7f1619bd6640 1 -- 172.21.15.47:0/3799788121 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161005d480 con 0x556a00f1ab00 2024-03-23T09:42:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.187+0000 7f1619bd6640 1 -- 172.21.15.47:0/3799788121 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161005d600 con 0x556a00f1ab00 2024-03-23T09:42:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.187+0000 7f161f036f40 1 -- 172.21.15.47:0/3799788121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556a00e29c20 con 0x556a00f1ab00 2024-03-23T09:42:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.187+0000 7f1619bd6640 1 -- 172.21.15.47:0/3799788121 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161005d9b0 con 0x556a00f1ab00 2024-03-23T09:42:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.187+0000 7f161f036f40 1 -- 172.21.15.47:0/3799788121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a00e1d060 con 0x556a00f1ab00 2024-03-23T09:42:33.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.188+0000 7f1619bd6640 1 -- 172.21.15.47:0/3799788121 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1610059040 con 0x556a00f1ab00 2024-03-23T09:42:33.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.189+0000 7f1619bd6640 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f16080695e0 0x7f160806baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.189+0000 7f1619bd6640 1 -- 172.21.15.47:0/3799788121 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16100ca060 con 0x556a00f1ab00 2024-03-23T09:42:33.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.189+0000 7f161c3db640 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f16080695e0 0x7f160806baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.189+0000 7f161f036f40 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x556a00f99240 0x556a00f9aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.189+0000 7f161f036f40 1 -- 172.21.15.47:0/3799788121 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x556a00f9afa0 con 0x556a00f99240 2024-03-23T09:42:33.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.189+0000 7f161cbdc640 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x556a00f99240 0x556a00f9aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.189+0000 7f161cbdc640 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x556a00f99240 0x556a00f9aa60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.189+0000 7f161c3db640 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f16080695e0 0x7f160806baa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x556a00f9ddd0 tx=0x7f160c005a50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.190+0000 7f161cbdc640 1 -- 172.21.15.47:0/3799788121 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f1614006400 con 0x556a00f99240 2024-03-23T09:42:33.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.190+0000 7f161f036f40 1 -- 172.21.15.47:0/3799788121 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x556a00f99240 msgr2=0x556a00f9aa60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.190+0000 7f161f036f40 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x556a00f99240 0x556a00f9aa60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.190+0000 7f161f036f40 1 -- 172.21.15.47:0/3799788121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f16080695e0 msgr2=0x7f160806baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.190+0000 7f161f036f40 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f16080695e0 0x7f160806baa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x556a00f9ddd0 tx=0x7f160c005a50 comp rx=0 tx=0).stop 2024-03-23T09:42:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.190+0000 7f161f036f40 1 -- 172.21.15.47:0/3799788121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a00f1ab00 msgr2=0x556a00ec0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.190+0000 7f161f036f40 1 --2- 172.21.15.47:0/3799788121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a00f1ab00 0x556a00ec0f60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1610000c00 tx=0x7f1610002fb0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.190+0000 7f161cbdc640 1 -- 172.21.15.47:0/3799788121 reap_dead start 2024-03-23T09:42:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.190+0000 7f161f036f40 1 -- 172.21.15.47:0/3799788121 shutdown_connections 2024-03-23T09:42:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.190+0000 7f161f036f40 1 -- 172.21.15.47:0/3799788121 >> 172.21.15.47:0/3799788121 conn(0x556a00efde40 msgr2=0x556a00efe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.190+0000 7f161f036f40 1 -- 172.21.15.47:0/3799788121 shutdown_connections 2024-03-23T09:42:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.190+0000 7f161f036f40 1 -- 172.21.15.47:0/3799788121 wait complete. 2024-03-23T09:42:33.197 INFO:teuthology.orchestra.run.smithi047.stdout:mytag 2024-03-23T09:42:33.197 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-03-23T09:42:33.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.281+0000 7fa5c184af40 1 Processor -- start 2024-03-23T09:42:33.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.281+0000 7fa5c184af40 1 -- start start 2024-03-23T09:42:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.281+0000 7fa5c184af40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610ddc74f60 0x5610ddc75340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.281+0000 7fa5c184af40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5610ddc5a1d0 0x5610ddc6f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.281+0000 7fa5c184af40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610ddc56140 0x5610ddc6fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.281+0000 7fa5c184af40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ddb4b4d0 con 0x5610ddc74f60 2024-03-23T09:42:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.281+0000 7fa5c184af40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ddae6b60 con 0x5610ddc56140 2024-03-23T09:42:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.281+0000 7fa5c184af40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5610ddae6800 con 0x5610ddc5a1d0 2024-03-23T09:42:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.281+0000 7fa5beac9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610ddc74f60 0x5610ddc75340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.281+0000 7fa5beac9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610ddc74f60 0x5610ddc75340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45684/0 (socket says 172.21.15.47:45684) 2024-03-23T09:42:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.281+0000 7fa5beac9640 1 -- 172.21.15.47:0/1497469427 learned_addr learned my addr 172.21.15.47:0/1497469427 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5beac9640 1 -- 172.21.15.47:0/1497469427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5610ddc5a1d0 msgr2=0x5610ddc6f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5beac9640 1 --2- 172.21.15.47:0/1497469427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5610ddc5a1d0 0x5610ddc6f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5beac9640 1 -- 172.21.15.47:0/1497469427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610ddc56140 msgr2=0x5610ddc6fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5beac9640 1 --2- 172.21.15.47:0/1497469427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610ddc56140 0x5610ddc6fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5beac9640 1 -- 172.21.15.47:0/1497469427 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610ddae9060 con 0x5610ddc74f60 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5beac9640 1 --2- 172.21.15.47:0/1497469427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610ddc74f60 0x5610ddc75340 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009840 tx=0x7fa5b8005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b406b8547a474bc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5bd2c6640 1 -- 172.21.15.47:0/1497469427 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b800f070 con 0x5610ddc74f60 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5bd2c6640 1 -- 172.21.15.47:0/1497469427 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b8010040 con 0x5610ddc74f60 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5bd2c6640 1 -- 172.21.15.47:0/1497469427 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8007a20 con 0x5610ddc74f60 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5c184af40 1 -- 172.21.15.47:0/1497469427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610ddc74f60 msgr2=0x5610ddc75340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5c184af40 1 --2- 172.21.15.47:0/1497469427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610ddc74f60 0x5610ddc75340 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009840 tx=0x7fa5b8005a00 comp rx=0 tx=0).stop 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5c184af40 1 -- 172.21.15.47:0/1497469427 shutdown_connections 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5c184af40 1 --2- 172.21.15.47:0/1497469427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610ddc56140 0x5610ddc6fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5c184af40 1 --2- 172.21.15.47:0/1497469427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5610ddc5a1d0 0x5610ddc6f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5c184af40 1 --2- 172.21.15.47:0/1497469427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610ddc74f60 0x5610ddc75340 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5c184af40 1 -- 172.21.15.47:0/1497469427 >> 172.21.15.47:0/1497469427 conn(0x5610ddbc9e40 msgr2=0x5610ddc74190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5c184af40 1 -- 172.21.15.47:0/1497469427 shutdown_connections 2024-03-23T09:42:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.282+0000 7fa5c184af40 1 -- 172.21.15.47:0/1497469427 wait complete. 2024-03-23T09:42:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.283+0000 7fa5c184af40 1 Processor -- start 2024-03-23T09:42:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.283+0000 7fa5c184af40 1 -- start start 2024-03-23T09:42:33.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5c184af40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5610ddc56140 0x5610ddd0f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5c184af40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610ddc5a1d0 0x5610ddc5bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5c184af40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610ddc74f60 0x5610ddbce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5c184af40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ddac9520 con 0x5610ddc5a1d0 2024-03-23T09:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5c184af40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ddc76080 con 0x5610ddc74f60 2024-03-23T09:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5c184af40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5610ddc762d0 con 0x5610ddc56140 2024-03-23T09:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5be2c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610ddc5a1d0 0x5610ddc5bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5be2c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610ddc5a1d0 0x5610ddc5bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45692/0 (socket says 172.21.15.47:45692) 2024-03-23T09:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5be2c8640 1 -- 172.21.15.47:0/712781727 learned_addr learned my addr 172.21.15.47:0/712781727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5beac9640 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5610ddc56140 0x5610ddd0f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5bf2ca640 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610ddc74f60 0x5610ddbce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5be2c8640 1 -- 172.21.15.47:0/712781727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5610ddc56140 msgr2=0x5610ddd0f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5be2c8640 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5610ddc56140 0x5610ddd0f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5be2c8640 1 -- 172.21.15.47:0/712781727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610ddc74f60 msgr2=0x5610ddbce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5be2c8640 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610ddc74f60 0x5610ddbce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5be2c8640 1 -- 172.21.15.47:0/712781727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b8009410 con 0x5610ddc5a1d0 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.284+0000 7fa5be2c8640 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610ddc5a1d0 0x5610ddc5bed0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8001670 tx=0x7fa5a8001760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.285+0000 7fa5b7fff640 1 -- 172.21.15.47:0/712781727 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800f860 con 0x5610ddc5a1d0 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.285+0000 7fa5b7fff640 1 -- 172.21.15.47:0/712781727 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a800f9c0 con 0x5610ddc5a1d0 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.285+0000 7fa5c184af40 1 -- 172.21.15.47:0/712781727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5610ddae8ac0 con 0x5610ddc5a1d0 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.285+0000 7fa5b7fff640 1 -- 172.21.15.47:0/712781727 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800fd70 con 0x5610ddc5a1d0 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.285+0000 7fa5c184af40 1 -- 172.21.15.47:0/712781727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610ddc5d000 con 0x5610ddc5a1d0 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.286+0000 7fa5b7fff640 1 -- 172.21.15.47:0/712781727 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa5a800e3e0 con 0x5610ddc5a1d0 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.286+0000 7fa5b7fff640 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa5a0069610 0x7fa5a006bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.286+0000 7fa5b7fff640 1 -- 172.21.15.47:0/712781727 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5a8004070 con 0x5610ddc5a1d0 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5c184af40 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x5610ddbe74c0 0x5610ddd18d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.286+0000 7fa5beac9640 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa5a0069610 0x7fa5a006bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5c184af40 1 -- 172.21.15.47:0/712781727 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x5610ddbced70 con 0x5610ddbe74c0 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5bf2ca640 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x5610ddbe74c0 0x5610ddd18d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5bf2ca640 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x5610ddbe74c0 0x5610ddd18d80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5beac9640 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa5a0069610 0x7fa5a006bad0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80097e0 tx=0x7fa5b8013000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5bf2ca640 1 -- 172.21.15.47:0/712781727 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fa5b00524f0 con 0x5610ddbe74c0 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5c184af40 1 -- 172.21.15.47:0/712781727 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x5610ddbe74c0 msgr2=0x5610ddd18d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5c184af40 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x5610ddbe74c0 0x5610ddd18d80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5c184af40 1 -- 172.21.15.47:0/712781727 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa5a0069610 msgr2=0x7fa5a006bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5c184af40 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa5a0069610 0x7fa5a006bad0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80097e0 tx=0x7fa5b8013000 comp rx=0 tx=0).stop 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5c184af40 1 -- 172.21.15.47:0/712781727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610ddc5a1d0 msgr2=0x5610ddc5bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5c184af40 1 --2- 172.21.15.47:0/712781727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610ddc5a1d0 0x5610ddc5bed0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8001670 tx=0x7fa5a8001760 comp rx=0 tx=0).stop 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5bf2ca640 1 -- 172.21.15.47:0/712781727 reap_dead start 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5c184af40 1 -- 172.21.15.47:0/712781727 shutdown_connections 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5c184af40 1 -- 172.21.15.47:0/712781727 >> 172.21.15.47:0/712781727 conn(0x5610ddbc9e40 msgr2=0x5610ddc75340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.287+0000 7fa5c184af40 1 -- 172.21.15.47:0/712781727 shutdown_connections 2024-03-23T09:42:33.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.288+0000 7fa5c184af40 1 -- 172.21.15.47:0/712781727 wait complete. 2024-03-23T09:42:33.292 INFO:teuthology.orchestra.run.smithi047.stdout:mytag 2024-03-23T09:42:33.293 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-03-23T09:42:33.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.374+0000 7f2bba20bf40 1 Processor -- start 2024-03-23T09:42:33.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.374+0000 7f2bba20bf40 1 -- start start 2024-03-23T09:42:33.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bba20bf40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba40027e0 0x55bf37017540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bba20bf40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3700c290 0x55bf370131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bba20bf40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf36f93c60 0x55bf37013700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bba20bf40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf36e91b60 con 0x7f2ba40027e0 2024-03-23T09:42:33.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bba20bf40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf36e91800 con 0x55bf36f93c60 2024-03-23T09:42:33.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bba20bf40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf36e74520 con 0x55bf3700c290 2024-03-23T09:42:33.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bb70de640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba40027e0 0x55bf37017540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bb70de640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba40027e0 0x55bf37017540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45716/0 (socket says 172.21.15.47:45716) 2024-03-23T09:42:33.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bb70de640 1 -- 172.21.15.47:0/3203515320 learned_addr learned my addr 172.21.15.47:0/3203515320 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:33.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bb68dd640 1 --2- 172.21.15.47:0/3203515320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3700c290 0x55bf370131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bb78df640 1 --2- 172.21.15.47:0/3203515320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf36f93c60 0x55bf37013700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bb70de640 1 -- 172.21.15.47:0/3203515320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3700c290 msgr2=0x55bf370131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:33.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bb70de640 1 --2- 172.21.15.47:0/3203515320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3700c290 0x55bf370131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bb70de640 1 -- 172.21.15.47:0/3203515320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf36f93c60 msgr2=0x55bf37013700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bb70de640 1 --2- 172.21.15.47:0/3203515320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf36f93c60 0x55bf37013700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.375+0000 7f2bb70de640 1 -- 172.21.15.47:0/3203515320 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf36ea0c20 con 0x7f2ba40027e0 2024-03-23T09:42:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.376+0000 7f2bb70de640 1 --2- 172.21.15.47:0/3203515320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba40027e0 0x55bf37017540 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0008910 tx=0x7f2bb0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61709081447d747b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.376+0000 7f2bb58db640 1 -- 172.21.15.47:0/3203515320 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000f070 con 0x7f2ba40027e0 2024-03-23T09:42:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.376+0000 7f2bb58db640 1 -- 172.21.15.47:0/3203515320 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb0009880 con 0x7f2ba40027e0 2024-03-23T09:42:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.376+0000 7f2bb58db640 1 -- 172.21.15.47:0/3203515320 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0009a10 con 0x7f2ba40027e0 2024-03-23T09:42:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.376+0000 7f2bba20bf40 1 -- 172.21.15.47:0/3203515320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba40027e0 msgr2=0x55bf37017540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.376+0000 7f2bba20bf40 1 --2- 172.21.15.47:0/3203515320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba40027e0 0x55bf37017540 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0008910 tx=0x7f2bb0005a00 comp rx=0 tx=0).stop 2024-03-23T09:42:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.376+0000 7f2bba20bf40 1 -- 172.21.15.47:0/3203515320 shutdown_connections 2024-03-23T09:42:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.376+0000 7f2bba20bf40 1 --2- 172.21.15.47:0/3203515320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf36f93c60 0x55bf37013700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.376+0000 7f2bba20bf40 1 --2- 172.21.15.47:0/3203515320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3700c290 0x55bf370131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.376+0000 7f2bba20bf40 1 --2- 172.21.15.47:0/3203515320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba40027e0 0x55bf37017540 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.376+0000 7f2bba20bf40 1 -- 172.21.15.47:0/3203515320 >> 172.21.15.47:0/3203515320 conn(0x55bf36f74e40 msgr2=0x55bf36f79cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:33.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.376+0000 7f2bba20bf40 1 -- 172.21.15.47:0/3203515320 shutdown_connections 2024-03-23T09:42:33.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.376+0000 7f2bba20bf40 1 -- 172.21.15.47:0/3203515320 wait complete. 2024-03-23T09:42:33.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.377+0000 7f2bba20bf40 1 Processor -- start 2024-03-23T09:42:33.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.377+0000 7f2bba20bf40 1 -- start start 2024-03-23T09:42:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bba20bf40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf36f93c60 0x55bf3700e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bba20bf40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3700c290 0x55bf3700a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bba20bf40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba40027e0 0x55bf370246a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bba20bf40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf3700ed80 con 0x7f2ba40027e0 2024-03-23T09:42:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bba20bf40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf36f794d0 con 0x55bf3700c290 2024-03-23T09:42:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bba20bf40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf37014110 con 0x55bf36f93c60 2024-03-23T09:42:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb78df640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba40027e0 0x55bf370246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb78df640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba40027e0 0x55bf370246a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45718/0 (socket says 172.21.15.47:45718) 2024-03-23T09:42:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb78df640 1 -- 172.21.15.47:0/2260124954 learned_addr learned my addr 172.21.15.47:0/2260124954 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb70de640 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf36f93c60 0x55bf3700e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb68dd640 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3700c290 0x55bf3700a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb78df640 1 -- 172.21.15.47:0/2260124954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf36f93c60 msgr2=0x55bf3700e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb78df640 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf36f93c60 0x55bf3700e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb78df640 1 -- 172.21.15.47:0/2260124954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3700c290 msgr2=0x55bf3700a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb78df640 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3700c290 0x55bf3700a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb78df640 1 -- 172.21.15.47:0/2260124954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb0008590 con 0x7f2ba40027e0 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb78df640 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba40027e0 0x55bf370246a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8005fd0 tx=0x7f2ba8012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb48d9640 1 -- 172.21.15.47:0/2260124954 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba80146f0 con 0x7f2ba40027e0 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb48d9640 1 -- 172.21.15.47:0/2260124954 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba8014850 con 0x7f2ba40027e0 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bba20bf40 1 -- 172.21.15.47:0/2260124954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bf36e94060 con 0x7f2ba40027e0 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bb48d9640 1 -- 172.21.15.47:0/2260124954 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8014be0 con 0x7f2ba40027e0 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.378+0000 7f2bba20bf40 1 -- 172.21.15.47:0/2260124954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf36e93ac0 con 0x7f2ba40027e0 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.380+0000 7f2bb48d9640 1 -- 172.21.15.47:0/2260124954 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2ba8003040 con 0x7f2ba40027e0 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.380+0000 7f2bb48d9640 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b980695e0 0x7f2b9806baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.380+0000 7f2bb48d9640 1 -- 172.21.15.47:0/2260124954 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ba8088210 con 0x7f2ba40027e0 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.380+0000 7f2bba20bf40 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55bf36f79f20 0x55bf37021aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.380+0000 7f2bb70de640 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b980695e0 0x7f2b9806baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.380+0000 7f2bba20bf40 1 -- 172.21.15.47:0/2260124954 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x55bf3700aa80 con 0x55bf36f79f20 2024-03-23T09:42:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.380+0000 7f2bb68dd640 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55bf36f79f20 0x55bf37021aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bb68dd640 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55bf36f79f20 0x55bf37021aa0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bb70de640 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b980695e0 0x7f2b9806baa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0018040 tx=0x7f2bb000f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bb68dd640 1 -- 172.21.15.47:0/2260124954 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2ba0050070 con 0x55bf36f79f20 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bba20bf40 1 -- 172.21.15.47:0/2260124954 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55bf36f79f20 msgr2=0x55bf37021aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bba20bf40 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55bf36f79f20 0x55bf37021aa0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bba20bf40 1 -- 172.21.15.47:0/2260124954 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b980695e0 msgr2=0x7f2b9806baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bba20bf40 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b980695e0 0x7f2b9806baa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0018040 tx=0x7f2bb000f000 comp rx=0 tx=0).stop 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bba20bf40 1 -- 172.21.15.47:0/2260124954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba40027e0 msgr2=0x55bf370246a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bba20bf40 1 --2- 172.21.15.47:0/2260124954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba40027e0 0x55bf370246a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8005fd0 tx=0x7f2ba8012f90 comp rx=0 tx=0).stop 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bb78df640 1 -- 172.21.15.47:0/2260124954 reap_dead start 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bba20bf40 1 -- 172.21.15.47:0/2260124954 shutdown_connections 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bba20bf40 1 -- 172.21.15.47:0/2260124954 >> 172.21.15.47:0/2260124954 conn(0x55bf36f74e40 msgr2=0x55bf3700bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bba20bf40 1 -- 172.21.15.47:0/2260124954 shutdown_connections 2024-03-23T09:42:33.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.381+0000 7f2bba20bf40 1 -- 172.21.15.47:0/2260124954 wait complete. 2024-03-23T09:42:33.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:42:33.387 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:42:33.387 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:42:33.466 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:42:33.511 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:42:33.511 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:42:33.511 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:42:33.526 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:42:33.526 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:42:33.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:33.526 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:42:33.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:42:33.581 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-23T09:42:33.581 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:42:33.581 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:42:33.599 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:42:33.599 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:42:33.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:33.599 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-23T09:42:33.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:42:33.656 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-23T09:42:33.656 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:42:33.656 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:42:33.671 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:42:33.671 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:42:33.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:33.671 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-23T09:42:33.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:42:33.726 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-23T09:42:33.726 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:42:33.726 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:42:33.743 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:42:33.743 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:42:33.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:33.743 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-23T09:42:33.799 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-23T09:42:33.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:42:33.800 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:42:33.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.900+0000 7fc36d18b640 1 Processor -- start 2024-03-23T09:42:33.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.901+0000 7fc36d18b640 1 -- start start 2024-03-23T09:42:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.901+0000 7fc36d18b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc3680cde10 0x7fc368167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.901+0000 7fc36d18b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3680d11e0 con 0x7fc3680b8350 2024-03-23T09:42:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.901+0000 7fc36d18b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3680d2680 con 0x7fc36815ca90 2024-03-23T09:42:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.901+0000 7fc36d18b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc368160f60 con 0x7fc3680cde10 2024-03-23T09:42:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.901+0000 7fc35ffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc3680b8350 0x7fc3680cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59404/0 (socket says 172.21.15.47:59404) 2024-03-23T09:42:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.901+0000 7fc35ffff640 1 -- 172.21.15.47:0/1386246614 learned_addr learned my addr 172.21.15.47:0/1386246614 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381393016 0 0) 0x7fc3680d11e0 con 0x7fc3680b8350 2024-03-23T09:42:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc354003520 con 0x7fc3680b8350 2024-03-23T09:42:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36759e640 1 --2- 172.21.15.47:0/1386246614 >> v2:172.21.15.175:3300/0 conn(0x7fc3680cde10 0x7fc368167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2538963722 0 0) 0x7fc354003520 con 0x7fc3680b8350 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 >> v2:172.21.15.175:3300/0 conn(0x7fc3680cde10 msgr2=0x7fc368167aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 --2- 172.21.15.47:0/1386246614 >> v2:172.21.15.175:3300/0 conn(0x7fc3680cde10 0x7fc368167aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 >> v1:172.21.15.175:6789/0 conn(0x7fc36815ca90 legacy=0x7fc368160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc368161240 con 0x7fc3680b8350 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fc350001670 con 0x7fc3680b8350 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 >> v1:172.21.15.47:6789/0 conn(0x7fc3680b8350 legacy=0x7fc3680cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 --2- 172.21.15.47:0/1386246614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc354003970 0x7fc354003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 --2- 172.21.15.47:0/1386246614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3540047c0 0x7fc354004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 --2- 172.21.15.47:0/1386246614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3540052a0 0x7fc35400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc368160f60 con 0x7fc3540047c0 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35400c170 con 0x7fc354003970 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35400c330 con 0x7fc3540052a0 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc35ffff640 1 --2- 172.21.15.47:0/1386246614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3540047c0 0x7fc354004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc366d9d640 1 --2- 172.21.15.47:0/1386246614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc354003970 0x7fc354003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36759e640 1 --2- 172.21.15.47:0/1386246614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3540052a0 0x7fc35400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc35ffff640 1 -- 172.21.15.47:0/1386246614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3540052a0 msgr2=0x7fc35400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc35ffff640 1 --2- 172.21.15.47:0/1386246614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3540052a0 0x7fc35400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc35ffff640 1 -- 172.21.15.47:0/1386246614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc354003970 msgr2=0x7fc354003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc35ffff640 1 --2- 172.21.15.47:0/1386246614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc354003970 0x7fc354003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc35ffff640 1 -- 172.21.15.47:0/1386246614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc354003520 con 0x7fc3540047c0 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc36759e640 1 -- 172.21.15.47:0/1386246614 reap_dead start 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.902+0000 7fc366d9d640 1 --2- 172.21.15.47:0/1386246614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc354003970 0x7fc354003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.903+0000 7fc35ffff640 1 --2- 172.21.15.47:0/1386246614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3540047c0 0x7fc354004c40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc35000c210 tx=0x7fc3540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13fdb7069c3c99e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.903+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc350005ad0 con 0x7fc3540047c0 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.903+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc350005c30 con 0x7fc3540047c0 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.903+0000 7fc36659c640 1 -- 172.21.15.47:0/1386246614 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35002ca20 con 0x7fc3540047c0 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.904+0000 7fc36d18b640 1 -- 172.21.15.47:0/1386246614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3540047c0 msgr2=0x7fc354004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.904+0000 7fc36d18b640 1 --2- 172.21.15.47:0/1386246614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3540047c0 0x7fc354004c40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc35000c210 tx=0x7fc3540057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.904+0000 7fc36d18b640 1 -- 172.21.15.47:0/1386246614 shutdown_connections 2024-03-23T09:42:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.904+0000 7fc36d18b640 1 --2- 172.21.15.47:0/1386246614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3540047c0 0x7fc354004c40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.904+0000 7fc36d18b640 1 -- 172.21.15.47:0/1386246614 >> 172.21.15.47:0/1386246614 conn(0x7fc3680659d0 msgr2=0x7fc368065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.904+0000 7fc36d18b640 1 -- 172.21.15.47:0/1386246614 shutdown_connections 2024-03-23T09:42:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.904+0000 7fc36d18b640 1 -- 172.21.15.47:0/1386246614 wait complete. 2024-03-23T09:42:33.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.905+0000 7fc36d18b640 1 Processor -- start 2024-03-23T09:42:33.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.905+0000 7fc36d18b640 1 -- start start 2024-03-23T09:42:33.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36d18b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3540047c0 0x7fc3681f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36d18b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc368159ff0 0x7fc3681f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36d18b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3681591b0 0x7fc368159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36d18b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36815a7d0 con 0x7fc3681591b0 2024-03-23T09:42:33.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36d18b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36815aab0 con 0x7fc3540047c0 2024-03-23T09:42:33.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36d18b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681586d0 con 0x7fc368159ff0 2024-03-23T09:42:33.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3681591b0 0x7fc368159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3681591b0 0x7fc368159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45740/0 (socket says 172.21.15.47:45740) 2024-03-23T09:42:33.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36759e640 1 -- 172.21.15.47:0/1256452599 learned_addr learned my addr 172.21.15.47:0/1256452599 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc366d9d640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3540047c0 0x7fc3681f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc35ffff640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc368159ff0 0x7fc3681f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36759e640 1 -- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc368159ff0 msgr2=0x7fc3681f1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36759e640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc368159ff0 0x7fc3681f1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36759e640 1 -- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3540047c0 msgr2=0x7fc3681f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36759e640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3540047c0 0x7fc3681f6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36759e640 1 -- 172.21.15.47:0/1256452599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3681589b0 con 0x7fc3681591b0 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc366d9d640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3540047c0 0x7fc3681f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.906+0000 7fc36759e640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3681591b0 0x7fc368159630 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc368161830 tx=0x7fc360007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.907+0000 7fc35f7fe640 1 -- 172.21.15.47:0/1256452599 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360014410 con 0x7fc3681591b0 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.907+0000 7fc35f7fe640 1 -- 172.21.15.47:0/1256452599 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36000f050 con 0x7fc3681591b0 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.907+0000 7fc36d18b640 1 -- 172.21.15.47:0/1256452599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc368158c90 con 0x7fc3681591b0 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.907+0000 7fc35f7fe640 1 -- 172.21.15.47:0/1256452599 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3600147d0 con 0x7fc3681591b0 2024-03-23T09:42:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.907+0000 7fc36d18b640 1 -- 172.21.15.47:0/1256452599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3681f22b0 con 0x7fc3681591b0 2024-03-23T09:42:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.908+0000 7fc36d18b640 1 -- 172.21.15.47:0/1256452599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc368072360 con 0x7fc3681591b0 2024-03-23T09:42:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.908+0000 7fc35f7fe640 1 -- 172.21.15.47:0/1256452599 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc360014930 con 0x7fc3681591b0 2024-03-23T09:42:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.908+0000 7fc35f7fe640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc34806ac00 0x7fc34806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.909+0000 7fc35f7fe640 1 -- 172.21.15.47:0/1256452599 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc360089990 con 0x7fc3681591b0 2024-03-23T09:42:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.909+0000 7fc366d9d640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc34806ac00 0x7fc34806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.909+0000 7fc366d9d640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc34806ac00 0x7fc34806d0c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc358000c00 tx=0x7fc358009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:33.913+0000 7fc35f7fe640 1 -- 172.21.15.47:0/1256452599 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc360054b70 con 0x7fc3681591b0 2024-03-23T09:42:34.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.129+0000 7fc36d18b640 1 -- 172.21.15.47:0/1256452599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc368066be0 con 0x7fc3681591b0 2024-03-23T09:42:34.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.130+0000 7fc35f7fe640 1 -- 172.21.15.47:0/1256452599 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fc360058190 con 0x7fc3681591b0 2024-03-23T09:42:34.130 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:34.130 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3373309448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3373309448},{"type":"v1","addr":"172.21.15.175:6836","nonce":3373309448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4118066020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4118066020},{"type":"v1","addr":"172.21.15.175:6837","nonce":4118066020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2916649516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2916649516},{"type":"v1","addr":"172.21.15.47:6835","nonce":2916649516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:59.719783+0000","modified":"2024-03-23T09:42:01.774130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6834/400299689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":400299689},{"type":"v1","addr":"172.21.15.47:6834","nonce":400299689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T09:42:34.130 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2024-03-23T09:42:34.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.132+0000 7fc36d18b640 1 -- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc34806ac00 msgr2=0x7fc34806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.132+0000 7fc36d18b640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc34806ac00 0x7fc34806d0c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc358000c00 tx=0x7fc358009040 comp rx=0 tx=0).stop 2024-03-23T09:42:34.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.132+0000 7fc36d18b640 1 -- 172.21.15.47:0/1256452599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3681591b0 msgr2=0x7fc368159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.132+0000 7fc36d18b640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3681591b0 0x7fc368159630 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc368161830 tx=0x7fc360007700 comp rx=0 tx=0).stop 2024-03-23T09:42:34.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.132+0000 7fc36d18b640 1 -- 172.21.15.47:0/1256452599 shutdown_connections 2024-03-23T09:42:34.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.132+0000 7fc36d18b640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc34806ac00 0x7fc34806d0c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.132+0000 7fc36d18b640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3681591b0 0x7fc368159630 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.132+0000 7fc36d18b640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc368159ff0 0x7fc3681f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.132+0000 7fc36d18b640 1 --2- 172.21.15.47:0/1256452599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3540047c0 0x7fc3681f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.132+0000 7fc36d18b640 1 -- 172.21.15.47:0/1256452599 >> 172.21.15.47:0/1256452599 conn(0x7fc3680659d0 msgr2=0x7fc36815bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:34.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.132+0000 7fc36d18b640 1 -- 172.21.15.47:0/1256452599 shutdown_connections 2024-03-23T09:42:34.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.132+0000 7fc36d18b640 1 -- 172.21.15.47:0/1256452599 wait complete. 2024-03-23T09:42:34.142 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:42:34.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.243+0000 7f95779d2640 1 Processor -- start 2024-03-23T09:42:34.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.243+0000 7f95779d2640 1 -- start start 2024-03-23T09:42:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.244+0000 7f95779d2640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f95701597d0 0x7f957006e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.244+0000 7f95779d2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f957006af10 0x7f95700cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.244+0000 7f95779d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95700cce50 0x7f95700ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.244+0000 7f95779d2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9570065ec0 con 0x7f957006af10 2024-03-23T09:42:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.244+0000 7f95779d2640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95700659d0 con 0x7f95701597d0 2024-03-23T09:42:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.244+0000 7f95779d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957006dbf0 con 0x7f95700cce50 2024-03-23T09:42:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.244+0000 7f9574f6d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f957006af10 0x7f95700cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.244+0000 7f9574f6d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f957006af10 0x7f95700cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45744/0 (socket says 172.21.15.47:45744) 2024-03-23T09:42:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.244+0000 7f9574f6d640 1 -- 172.21.15.47:0/830002761 learned_addr learned my addr 172.21.15.47:0/830002761 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.245+0000 7f9575f6f640 1 --2- 172.21.15.47:0/830002761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95700cce50 0x7f95700ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.245+0000 7f957576e640 1 --2- 172.21.15.47:0/830002761 >> v2:172.21.15.175:3300/0 conn(0x7f95701597d0 0x7f957006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.245+0000 7f9574f6d640 1 -- 172.21.15.47:0/830002761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95700cce50 msgr2=0x7f95700ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:34.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.245+0000 7f9574f6d640 1 --2- 172.21.15.47:0/830002761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95700cce50 0x7f95700ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.245+0000 7f9574f6d640 1 -- 172.21.15.47:0/830002761 >> v2:172.21.15.175:3300/0 conn(0x7f95701597d0 msgr2=0x7f957006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:34.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.245+0000 7f9574f6d640 1 --2- 172.21.15.47:0/830002761 >> v2:172.21.15.175:3300/0 conn(0x7f95701597d0 0x7f957006e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.245+0000 7f9574f6d640 1 -- 172.21.15.47:0/830002761 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9570065be0 con 0x7f957006af10 2024-03-23T09:42:34.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.245+0000 7f9575f6f640 1 --2- 172.21.15.47:0/830002761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95700cce50 0x7f95700ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:34.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.245+0000 7f957576e640 1 --2- 172.21.15.47:0/830002761 >> v2:172.21.15.175:3300/0 conn(0x7f95701597d0 0x7f957006e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:34.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.245+0000 7f9574f6d640 1 --2- 172.21.15.47:0/830002761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f957006af10 0x7f95700cc910 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f957006e650 tx=0x7f956402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba87bffbe27e3f5e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:34.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.246+0000 7f956ffff640 1 -- 172.21.15.47:0/830002761 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640317e0 con 0x7f957006af10 2024-03-23T09:42:34.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.246+0000 7f956ffff640 1 -- 172.21.15.47:0/830002761 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9564031940 con 0x7f957006af10 2024-03-23T09:42:34.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.246+0000 7f956ffff640 1 -- 172.21.15.47:0/830002761 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9564031cd0 con 0x7f957006af10 2024-03-23T09:42:34.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.246+0000 7f95779d2640 1 -- 172.21.15.47:0/830002761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f957006af10 msgr2=0x7f95700cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.246+0000 7f95779d2640 1 --2- 172.21.15.47:0/830002761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f957006af10 0x7f95700cc910 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f957006e650 tx=0x7f956402b880 comp rx=0 tx=0).stop 2024-03-23T09:42:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.246+0000 7f95779d2640 1 -- 172.21.15.47:0/830002761 shutdown_connections 2024-03-23T09:42:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.246+0000 7f95779d2640 1 --2- 172.21.15.47:0/830002761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95700cce50 0x7f95700ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.246+0000 7f95779d2640 1 --2- 172.21.15.47:0/830002761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f957006af10 0x7f95700cc910 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.246+0000 7f95779d2640 1 --2- 172.21.15.47:0/830002761 >> v2:172.21.15.175:3300/0 conn(0x7f95701597d0 0x7f957006e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.246+0000 7f95779d2640 1 -- 172.21.15.47:0/830002761 >> 172.21.15.47:0/830002761 conn(0x7f95700cb7c0 msgr2=0x7f95700cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.246+0000 7f95779d2640 1 -- 172.21.15.47:0/830002761 shutdown_connections 2024-03-23T09:42:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.246+0000 7f95779d2640 1 -- 172.21.15.47:0/830002761 wait complete. 2024-03-23T09:42:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.247+0000 7f95779d2640 1 Processor -- start 2024-03-23T09:42:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.247+0000 7f95779d2640 1 -- start start 2024-03-23T09:42:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.248+0000 7f95779d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f957006af10 0x7f95700b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.248+0000 7f95779d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95700cce50 0x7f9570147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.248+0000 7f95779d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95701597d0 0x7f95700c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.248+0000 7f95779d2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95700b4c50 con 0x7f95700cce50 2024-03-23T09:42:34.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.248+0000 7f95779d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9570147570 con 0x7f957006af10 2024-03-23T09:42:34.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.248+0000 7f95779d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95700c4360 con 0x7f95701597d0 2024-03-23T09:42:34.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.248+0000 7f9574f6d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95700cce50 0x7f9570147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.248+0000 7f9574f6d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95700cce50 0x7f9570147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45760/0 (socket says 172.21.15.47:45760) 2024-03-23T09:42:34.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.248+0000 7f9574f6d640 1 -- 172.21.15.47:0/245973272 learned_addr learned my addr 172.21.15.47:0/245973272 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:34.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.248+0000 7f957576e640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f957006af10 0x7f95700b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.248+0000 7f9575f6f640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95701597d0 0x7f95700c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.249+0000 7f9574f6d640 1 -- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95701597d0 msgr2=0x7f95700c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.249+0000 7f9574f6d640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95701597d0 0x7f95700c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.249+0000 7f9574f6d640 1 -- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f957006af10 msgr2=0x7f95700b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.249+0000 7f9574f6d640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f957006af10 0x7f95700b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.249+0000 7f9574f6d640 1 -- 172.21.15.47:0/245973272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95640094d0 con 0x7f95700cce50 2024-03-23T09:42:34.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.249+0000 7f9575f6f640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95701597d0 0x7f95700c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:34.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.249+0000 7f9574f6d640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95700cce50 0x7f9570147030 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f95640097c0 tx=0x7f95640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:34.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.249+0000 7f956dffb640 1 -- 172.21.15.47:0/245973272 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9564032980 con 0x7f95700cce50 2024-03-23T09:42:34.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.249+0000 7f956dffb640 1 -- 172.21.15.47:0/245973272 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9564032ae0 con 0x7f95700cce50 2024-03-23T09:42:34.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.249+0000 7f95779d2640 1 -- 172.21.15.47:0/245973272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9570159d30 con 0x7f95700cce50 2024-03-23T09:42:34.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.249+0000 7f956dffb640 1 -- 172.21.15.47:0/245973272 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9564032e90 con 0x7f95700cce50 2024-03-23T09:42:34.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.249+0000 7f95779d2640 1 -- 172.21.15.47:0/245973272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95700c4920 con 0x7f95700cce50 2024-03-23T09:42:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.250+0000 7f95779d2640 1 -- 172.21.15.47:0/245973272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95700672f0 con 0x7f95700cce50 2024-03-23T09:42:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.251+0000 7f956dffb640 1 -- 172.21.15.47:0/245973272 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f956404d3f0 con 0x7f95700cce50 2024-03-23T09:42:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.251+0000 7f956dffb640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f955c06ac00 0x7f955c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.251+0000 7f956dffb640 1 -- 172.21.15.47:0/245973272 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95640ab2b0 con 0x7f95700cce50 2024-03-23T09:42:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.251+0000 7f957576e640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f955c06ac00 0x7f955c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.252+0000 7f957576e640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f955c06ac00 0x7f955c06d0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f956000d5d0 tx=0x7f9560007810 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:34.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.255+0000 7f956dffb640 1 -- 172.21.15.47:0/245973272 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9564076400 con 0x7f95700cce50 2024-03-23T09:42:34.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.434+0000 7f95779d2640 1 -- 172.21.15.47:0/245973272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95700cc500 con 0x7f95700cce50 2024-03-23T09:42:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.435+0000 7f956dffb640 1 -- 172.21.15.47:0/245973272 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f9564079a20 con 0x7f95700cce50 2024-03-23T09:42:34.435 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:00.721483+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:58.297353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:58.685375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:42:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.437+0000 7f95779d2640 1 -- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f955c06ac00 msgr2=0x7f955c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.437+0000 7f95779d2640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f955c06ac00 0x7f955c06d0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f956000d5d0 tx=0x7f9560007810 comp rx=0 tx=0).stop 2024-03-23T09:42:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.437+0000 7f95779d2640 1 -- 172.21.15.47:0/245973272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95700cce50 msgr2=0x7f9570147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.437+0000 7f95779d2640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95700cce50 0x7f9570147030 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f95640097c0 tx=0x7f95640046c0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.437+0000 7f95779d2640 1 -- 172.21.15.47:0/245973272 shutdown_connections 2024-03-23T09:42:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.437+0000 7f95779d2640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f955c06ac00 0x7f955c06d0c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.437+0000 7f95779d2640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95701597d0 0x7f95700c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.437+0000 7f95779d2640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95700cce50 0x7f9570147030 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.437+0000 7f95779d2640 1 --2- 172.21.15.47:0/245973272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f957006af10 0x7f95700b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.437+0000 7f95779d2640 1 -- 172.21.15.47:0/245973272 >> 172.21.15.47:0/245973272 conn(0x7f95700cb7c0 msgr2=0x7f95700cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:34.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.437+0000 7f95779d2640 1 -- 172.21.15.47:0/245973272 shutdown_connections 2024-03-23T09:42:34.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.437+0000 7f95779d2640 1 -- 172.21.15.47:0/245973272 wait complete. 2024-03-23T09:42:34.447 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:41:58.297353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:41:58.685375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:42:34.448 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:42:34.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.551+0000 7f0c2d6ea640 1 Processor -- start 2024-03-23T09:42:34.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.551+0000 7f0c2d6ea640 1 -- start start 2024-03-23T09:42:34.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.552+0000 7f0c2d6ea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b8350 0x7f0c28160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.552+0000 7f0c2d6ea640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c28161630 con 0x7f0c2815c990 2024-03-23T09:42:34.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.552+0000 7f0c2d6ea640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c280d1030 con 0x7f0c280cde10 2024-03-23T09:42:34.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.552+0000 7f0c2d6ea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280cfe10 con 0x7f0c280b8350 2024-03-23T09:42:34.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.552+0000 7f0c26ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0c2815c990 0x7f0c280cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59420/0 (socket says 172.21.15.47:59420) 2024-03-23T09:42:34.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.552+0000 7f0c26ffd640 1 -- 172.21.15.47:0/82763023 learned_addr learned my addr 172.21.15.47:0/82763023 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:34.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.552+0000 7f0c277fe640 1 --2- 172.21.15.47:0/82763023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b8350 0x7f0c28160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.552+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041917274 0 0) 0x7f0c28161630 con 0x7f0c2815c990 2024-03-23T09:42:34.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.552+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c14003520 con 0x7f0c2815c990 2024-03-23T09:42:34.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2118371986 0 0) 0x7f0c14003520 con 0x7f0c2815c990 2024-03-23T09:42:34.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b8350 msgr2=0x7f0c28160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:34.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c25ffb640 1 --2- 172.21.15.47:0/82763023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b8350 0x7f0c28160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 >> v1:172.21.15.175:6789/0 conn(0x7f0c280cde10 legacy=0x7f0c280d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c280b5b60 con 0x7f0c2815c990 2024-03-23T09:42:34.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0c20001670 con 0x7f0c2815c990 2024-03-23T09:42:34.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 >> v1:172.21.15.47:6789/0 conn(0x7f0c2815c990 legacy=0x7f0c280cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c25ffb640 1 --2- 172.21.15.47:0/82763023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c140039a0 0x7f0c14003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c25ffb640 1 --2- 172.21.15.47:0/82763023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c14004300 0x7f0c140047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c25ffb640 1 --2- 172.21.15.47:0/82763023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b8350 0x7f0c140050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280cfe10 con 0x7f0c140039a0 2024-03-23T09:42:34.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1400b9d0 con 0x7f0c14004300 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1400bb90 con 0x7f0c280b8350 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c26ffd640 1 --2- 172.21.15.47:0/82763023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c140039a0 0x7f0c14003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c267fc640 1 --2- 172.21.15.47:0/82763023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c14004300 0x7f0c140047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c277fe640 1 --2- 172.21.15.47:0/82763023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b8350 0x7f0c140050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c26ffd640 1 -- 172.21.15.47:0/82763023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b8350 msgr2=0x7f0c140050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c26ffd640 1 --2- 172.21.15.47:0/82763023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b8350 0x7f0c140050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c26ffd640 1 -- 172.21.15.47:0/82763023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c14004300 msgr2=0x7f0c140047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c26ffd640 1 --2- 172.21.15.47:0/82763023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c14004300 0x7f0c140047e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.553+0000 7f0c26ffd640 1 -- 172.21.15.47:0/82763023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c280b5b60 con 0x7f0c140039a0 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.554+0000 7f0c26ffd640 1 --2- 172.21.15.47:0/82763023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c140039a0 0x7f0c14003d80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c240 tx=0x7f0c140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f4db1006e240c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.554+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2002cb60 con 0x7f0c140039a0 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.554+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2002ccc0 con 0x7f0c140039a0 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.554+0000 7f0c25ffb640 1 -- 172.21.15.47:0/82763023 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20005b10 con 0x7f0c140039a0 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.554+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/82763023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c140039a0 msgr2=0x7f0c14003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.554+0000 7f0c2d6ea640 1 --2- 172.21.15.47:0/82763023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c140039a0 0x7f0c14003d80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c240 tx=0x7f0c140057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.554+0000 7f0c277fe640 1 -- 172.21.15.47:0/82763023 reap_dead start 2024-03-23T09:42:34.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.555+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/82763023 shutdown_connections 2024-03-23T09:42:34.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.555+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/82763023 >> 172.21.15.47:0/82763023 conn(0x7f0c280659d0 msgr2=0x7f0c28065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:34.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.555+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/82763023 shutdown_connections 2024-03-23T09:42:34.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.555+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/82763023 wait complete. 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.556+0000 7f0c2d6ea640 1 Processor -- start 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.556+0000 7f0c2d6ea640 1 -- start start 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.556+0000 7f0c2d6ea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c140039a0 0x7f0c2815dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c2d6ea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c2815fa30 0x7f0c281f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c2d6ea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c281f1e10 0x7f0c280cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c2d6ea640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2815feb0 con 0x7f0c140039a0 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c2d6ea640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2815e1b0 con 0x7f0c281f1e10 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c2d6ea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2815e490 con 0x7f0c2815fa30 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c26ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c140039a0 0x7f0c2815dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c26ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c140039a0 0x7f0c2815dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45776/0 (socket says 172.21.15.47:45776) 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c26ffd640 1 -- 172.21.15.47:0/2571860213 learned_addr learned my addr 172.21.15.47:0/2571860213 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c267fc640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c2815fa30 0x7f0c281f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c277fe640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c281f1e10 0x7f0c280cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c26ffd640 1 -- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c2815fa30 msgr2=0x7f0c281f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c26ffd640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c2815fa30 0x7f0c281f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c26ffd640 1 -- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c281f1e10 msgr2=0x7f0c280cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c26ffd640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c281f1e10 0x7f0c280cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c26ffd640 1 -- 172.21.15.47:0/2571860213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2815e770 con 0x7f0c140039a0 2024-03-23T09:42:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c267fc640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c2815fa30 0x7f0c281f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c277fe640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c281f1e10 0x7f0c280cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c26ffd640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c140039a0 0x7f0c2815dab0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c360 tx=0x7f0c2002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c1f7fe640 1 -- 172.21.15.47:0/2571860213 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20004cb0 con 0x7f0c140039a0 2024-03-23T09:42:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c1f7fe640 1 -- 172.21.15.47:0/2571860213 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c20004e10 con 0x7f0c140039a0 2024-03-23T09:42:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/2571860213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2815ea50 con 0x7f0c140039a0 2024-03-23T09:42:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c1f7fe640 1 -- 172.21.15.47:0/2571860213 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20007270 con 0x7f0c140039a0 2024-03-23T09:42:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.557+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/2571860213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c280cbfd0 con 0x7f0c140039a0 2024-03-23T09:42:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.558+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/2571860213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c28072360 con 0x7f0c140039a0 2024-03-23T09:42:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.559+0000 7f0c1f7fe640 1 -- 172.21.15.47:0/2571860213 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0c20032070 con 0x7f0c140039a0 2024-03-23T09:42:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.559+0000 7f0c1f7fe640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c0806ac30 0x7f0c0806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.559+0000 7f0c1f7fe640 1 -- 172.21.15.47:0/2571860213 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c200ab000 con 0x7f0c140039a0 2024-03-23T09:42:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.559+0000 7f0c267fc640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c0806ac30 0x7f0c0806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.560+0000 7f0c267fc640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c0806ac30 0x7f0c0806d0f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0c281f54e0 tx=0x7f0c10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.563+0000 7f0c1f7fe640 1 -- 172.21.15.47:0/2571860213 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0c20076270 con 0x7f0c140039a0 2024-03-23T09:42:34.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.742+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/2571860213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0c28066be0 con 0x7f0c140039a0 2024-03-23T09:42:34.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.742+0000 7f0c1f7fe640 1 -- 172.21.15.47:0/2571860213 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0c20079890 con 0x7f0c140039a0 2024-03-23T09:42:34.742 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:42:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.744+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c0806ac30 msgr2=0x7f0c0806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.744+0000 7f0c2d6ea640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c0806ac30 0x7f0c0806d0f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0c281f54e0 tx=0x7f0c10009040 comp rx=0 tx=0).stop 2024-03-23T09:42:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.744+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/2571860213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c140039a0 msgr2=0x7f0c2815dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.744+0000 7f0c2d6ea640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c140039a0 0x7f0c2815dab0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c360 tx=0x7f0c2002e070 comp rx=0 tx=0).stop 2024-03-23T09:42:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.744+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/2571860213 shutdown_connections 2024-03-23T09:42:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.744+0000 7f0c2d6ea640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c0806ac30 0x7f0c0806d0f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.744+0000 7f0c2d6ea640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c281f1e10 0x7f0c280cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.744+0000 7f0c2d6ea640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c2815fa30 0x7f0c281f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.745+0000 7f0c2d6ea640 1 --2- 172.21.15.47:0/2571860213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c140039a0 0x7f0c2815dab0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.745+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/2571860213 >> 172.21.15.47:0/2571860213 conn(0x7f0c280659d0 msgr2=0x7f0c280b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.745+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/2571860213 shutdown_connections 2024-03-23T09:42:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.745+0000 7f0c2d6ea640 1 -- 172.21.15.47:0/2571860213 wait complete. 2024-03-23T09:42:34.755 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:42:34.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.859+0000 7fa3b14c4640 1 Processor -- start 2024-03-23T09:42:34.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.859+0000 7fa3b14c4640 1 -- start start 2024-03-23T09:42:34.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3b14c4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa3ac0cf9c0 0x7fa3ac0b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3b14c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3ac0cd140 0x7fa3ac0d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3b14c4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ac161570 con 0x7fa3ac15cbf0 2024-03-23T09:42:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3b14c4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ac160960 con 0x7fa3ac0cf9c0 2024-03-23T09:42:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3b14c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac160c40 con 0x7fa3ac0cd140 2024-03-23T09:42:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3aaffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa3ac15cbf0 0x7fa3ac0cf2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44950/0 (socket says 172.21.15.47:44950) 2024-03-23T09:42:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3aaffd640 1 -- 172.21.15.47:0/2240607433 learned_addr learned my addr 172.21.15.47:0/2240607433 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880149194 0 0) 0x7fa3ac161570 con 0x7fa3ac15cbf0 2024-03-23T09:42:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3ab7fe640 1 --2- 172.21.15.47:0/2240607433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3ac0cd140 0x7fa3ac0d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3aa7fc640 1 --2- 172.21.15.47:0/2240607433 >> v2:172.21.15.175:3300/0 conn(0x7fa3ac0cf9c0 0x7fa3ac0b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa394003520 con 0x7fa3ac15cbf0 2024-03-23T09:42:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2137266255 0 0) 0x7fa394003520 con 0x7fa3ac15cbf0 2024-03-23T09:42:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3ac0cd140 msgr2=0x7fa3ac0d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3a9ffb640 1 --2- 172.21.15.47:0/2240607433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3ac0cd140 0x7fa3ac0d0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 >> v2:172.21.15.175:3300/0 conn(0x7fa3ac0cf9c0 msgr2=0x7fa3ac0b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3a9ffb640 1 --2- 172.21.15.47:0/2240607433 >> v2:172.21.15.175:3300/0 conn(0x7fa3ac0cf9c0 0x7fa3ac0b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac160f20 con 0x7fa3ac15cbf0 2024-03-23T09:42:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fa3a4001670 con 0x7fa3ac15cbf0 2024-03-23T09:42:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.860+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 >> v1:172.21.15.47:6789/0 conn(0x7fa3ac15cbf0 legacy=0x7fa3ac0cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3a9ffb640 1 --2- 172.21.15.47:0/2240607433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa394003970 0x7fa394003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3a9ffb640 1 --2- 172.21.15.47:0/2240607433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3ac0cd140 0x7fa3940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3a9ffb640 1 --2- 172.21.15.47:0/2240607433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa394004d00 0x7fa394005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac160960 con 0x7fa394003970 2024-03-23T09:42:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac160c40 con 0x7fa394004d00 2024-03-23T09:42:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39400b980 con 0x7fa3ac0cd140 2024-03-23T09:42:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3aaffd640 1 --2- 172.21.15.47:0/2240607433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa394003970 0x7fa394003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3aa7fc640 1 --2- 172.21.15.47:0/2240607433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3ac0cd140 0x7fa3940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3ab7fe640 1 --2- 172.21.15.47:0/2240607433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa394004d00 0x7fa394005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3aaffd640 1 -- 172.21.15.47:0/2240607433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3ac0cd140 msgr2=0x7fa3940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3aaffd640 1 --2- 172.21.15.47:0/2240607433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3ac0cd140 0x7fa3940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3aaffd640 1 -- 172.21.15.47:0/2240607433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa394004d00 msgr2=0x7fa394005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3aaffd640 1 --2- 172.21.15.47:0/2240607433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa394004d00 0x7fa394005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.861+0000 7fa3aaffd640 1 -- 172.21.15.47:0/2240607433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa394003520 con 0x7fa394003970 2024-03-23T09:42:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.862+0000 7fa3aaffd640 1 --2- 172.21.15.47:0/2240607433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa394003970 0x7fa394003d90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400c290 tx=0x7fa3940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee899f92e074204f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.862+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4005ad0 con 0x7fa394003970 2024-03-23T09:42:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.862+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a4005c30 con 0x7fa394003970 2024-03-23T09:42:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.862+0000 7fa3a9ffb640 1 -- 172.21.15.47:0/2240607433 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a402c8d0 con 0x7fa394003970 2024-03-23T09:42:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.862+0000 7fa3b14c4640 1 -- 172.21.15.47:0/2240607433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa394003970 msgr2=0x7fa394003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.862+0000 7fa3b14c4640 1 --2- 172.21.15.47:0/2240607433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa394003970 0x7fa394003d90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400c290 tx=0x7fa3940057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.862+0000 7fa3ab7fe640 1 -- 172.21.15.47:0/2240607433 reap_dead start 2024-03-23T09:42:34.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.862+0000 7fa3b14c4640 1 -- 172.21.15.47:0/2240607433 shutdown_connections 2024-03-23T09:42:34.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.862+0000 7fa3b14c4640 1 -- 172.21.15.47:0/2240607433 >> 172.21.15.47:0/2240607433 conn(0x7fa3ac06b160 msgr2=0x7fa3ac06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:34.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.862+0000 7fa3b14c4640 1 -- 172.21.15.47:0/2240607433 shutdown_connections 2024-03-23T09:42:34.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.862+0000 7fa3b14c4640 1 -- 172.21.15.47:0/2240607433 wait complete. 2024-03-23T09:42:34.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.864+0000 7fa3b14c4640 1 Processor -- start 2024-03-23T09:42:34.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.864+0000 7fa3b14c4640 1 -- start start 2024-03-23T09:42:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.864+0000 7fa3b14c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa394003970 0x7fa3ac158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.864+0000 7fa3b14c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3ac0b1db0 0x7fa3ac0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.864+0000 7fa3b14c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ac15f0c0 0x7fa3ac15f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.864+0000 7fa3b14c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac15fa80 con 0x7fa3ac15f0c0 2024-03-23T09:42:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.864+0000 7fa3b14c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0b2920 con 0x7fa3ac0b1db0 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.864+0000 7fa3b14c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0b2c00 con 0x7fa394003970 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.864+0000 7fa3ab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ac15f0c0 0x7fa3ac15f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.864+0000 7fa3ab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ac15f0c0 0x7fa3ac15f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44574/0 (socket says 172.21.15.47:44574) 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.864+0000 7fa3ab7fe640 1 -- 172.21.15.47:0/123985751 learned_addr learned my addr 172.21.15.47:0/123985751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3aa7fc640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3ac0b1db0 0x7fa3ac0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3aaffd640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa394003970 0x7fa3ac158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3ab7fe640 1 -- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa394003970 msgr2=0x7fa3ac158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3ab7fe640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa394003970 0x7fa3ac158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3ab7fe640 1 -- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3ac0b1db0 msgr2=0x7fa3ac0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3ab7fe640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3ac0b1db0 0x7fa3ac0b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3ab7fe640 1 -- 172.21.15.47:0/123985751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac0b2ee0 con 0x7fa3ac15f0c0 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3aa7fc640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3ac0b1db0 0x7fa3ac0b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3aaffd640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa394003970 0x7fa3ac158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3ab7fe640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ac15f0c0 0x7fa3ac15f540 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac160720 tx=0x7fa39c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3937fe640 1 -- 172.21.15.47:0/123985751 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00e070 con 0x7fa3ac15f0c0 2024-03-23T09:42:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3937fe640 1 -- 172.21.15.47:0/123985751 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c011e10 con 0x7fa3ac15f0c0 2024-03-23T09:42:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3b14c4640 1 -- 172.21.15.47:0/123985751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ac0b31c0 con 0x7fa3ac15f0c0 2024-03-23T09:42:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3937fe640 1 -- 172.21.15.47:0/123985751 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c003990 con 0x7fa3ac15f0c0 2024-03-23T09:42:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.865+0000 7fa3b14c4640 1 -- 172.21.15.47:0/123985751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ac159290 con 0x7fa3ac15f0c0 2024-03-23T09:42:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.866+0000 7fa3b14c4640 1 -- 172.21.15.47:0/123985751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ac072360 con 0x7fa3ac15f0c0 2024-03-23T09:42:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.866+0000 7fa3937fe640 1 -- 172.21.15.47:0/123985751 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa39c003af0 con 0x7fa3ac15f0c0 2024-03-23T09:42:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.867+0000 7fa3937fe640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa39806ac00 0x7fa39806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.867+0000 7fa3937fe640 1 -- 172.21.15.47:0/123985751 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa39c089af0 con 0x7fa3ac15f0c0 2024-03-23T09:42:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.867+0000 7fa3aaffd640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa39806ac00 0x7fa39806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.868+0000 7fa3aaffd640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa39806ac00 0x7fa39806d0c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4034000 tx=0x7fa3a4007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:34.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:34.871+0000 7fa3937fe640 1 -- 172.21.15.47:0/123985751 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa39c054c20 con 0x7fa3ac15f0c0 2024-03-23T09:42:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.050+0000 7fa3b14c4640 1 -- 172.21.15.47:0/123985751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3ac0b3350 con 0x7fa3ac15f0c0 2024-03-23T09:42:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.051+0000 7fa3937fe640 1 -- 172.21.15.47:0/123985751 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa39c058240 con 0x7fa3ac15f0c0 2024-03-23T09:42:35.051 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:42:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.053+0000 7fa3b14c4640 1 -- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa39806ac00 msgr2=0x7fa39806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.053+0000 7fa3b14c4640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa39806ac00 0x7fa39806d0c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4034000 tx=0x7fa3a4007000 comp rx=0 tx=0).stop 2024-03-23T09:42:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.053+0000 7fa3b14c4640 1 -- 172.21.15.47:0/123985751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ac15f0c0 msgr2=0x7fa3ac15f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.053+0000 7fa3b14c4640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ac15f0c0 0x7fa3ac15f540 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac160720 tx=0x7fa39c0115a0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.053+0000 7fa3b14c4640 1 -- 172.21.15.47:0/123985751 shutdown_connections 2024-03-23T09:42:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.053+0000 7fa3b14c4640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa39806ac00 0x7fa39806d0c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.053+0000 7fa3b14c4640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ac15f0c0 0x7fa3ac15f540 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.053+0000 7fa3b14c4640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3ac0b1db0 0x7fa3ac0b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.053+0000 7fa3b14c4640 1 --2- 172.21.15.47:0/123985751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa394003970 0x7fa3ac158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.053+0000 7fa3b14c4640 1 -- 172.21.15.47:0/123985751 >> 172.21.15.47:0/123985751 conn(0x7fa3ac06b160 msgr2=0x7fa3ac159eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:35.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.053+0000 7fa3b14c4640 1 -- 172.21.15.47:0/123985751 shutdown_connections 2024-03-23T09:42:35.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.053+0000 7fa3b14c4640 1 -- 172.21.15.47:0/123985751 wait complete. 2024-03-23T09:42:35.063 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:42:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.168+0000 7fce36ba2640 1 Processor -- start 2024-03-23T09:42:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.169+0000 7fce36ba2640 1 -- start start 2024-03-23T09:42:35.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.169+0000 7fce36ba2640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fce300cb1e0 0x7fce300ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.169+0000 7fce36ba2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce3006a3f0 0x7fce300aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.169+0000 7fce36ba2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce30161560 con 0x7fce300b18a0 2024-03-23T09:42:35.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.169+0000 7fce36ba2640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce300cf190 con 0x7fce300cb1e0 2024-03-23T09:42:35.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.169+0000 7fce36ba2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce300cb5c0 con 0x7fce3006a3f0 2024-03-23T09:42:35.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.169+0000 7fce3539f640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fce300b18a0 0x7fce30069ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44954/0 (socket says 172.21.15.47:44954) 2024-03-23T09:42:35.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.169+0000 7fce3539f640 1 -- 172.21.15.47:0/624643204 learned_addr learned my addr 172.21.15.47:0/624643204 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:35.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce35ba0640 1 --2- 172.21.15.47:0/624643204 >> v2:172.21.15.175:3300/0 conn(0x7fce300cb1e0 0x7fce300ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce363a1640 1 --2- 172.21.15.47:0/624643204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce3006a3f0 0x7fce300aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952124041 0 0) 0x7fce30161560 con 0x7fce300b18a0 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce1c003520 con 0x7fce300b18a0 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2731577044 0 0) 0x7fce1c003520 con 0x7fce300b18a0 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce3006a3f0 msgr2=0x7fce300aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 --2- 172.21.15.47:0/624643204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce3006a3f0 0x7fce300aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 >> v2:172.21.15.175:3300/0 conn(0x7fce300cb1e0 msgr2=0x7fce300ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 --2- 172.21.15.47:0/624643204 >> v2:172.21.15.175:3300/0 conn(0x7fce300cb1e0 0x7fce300ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce301607a0 con 0x7fce300b18a0 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fce20001670 con 0x7fce300b18a0 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 >> v1:172.21.15.47:6789/0 conn(0x7fce300b18a0 legacy=0x7fce30069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 --2- 172.21.15.47:0/624643204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce1c003970 0x7fce1c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 --2- 172.21.15.47:0/624643204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce1c0047c0 0x7fce1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 --2- 172.21.15.47:0/624643204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce3006a3f0 0x7fce1c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300cf190 con 0x7fce1c0047c0 2024-03-23T09:42:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300cb5c0 con 0x7fce1c003970 2024-03-23T09:42:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c00b980 con 0x7fce3006a3f0 2024-03-23T09:42:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce3539f640 1 --2- 172.21.15.47:0/624643204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce1c0047c0 0x7fce1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce35ba0640 1 --2- 172.21.15.47:0/624643204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce1c003970 0x7fce1c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.170+0000 7fce363a1640 1 --2- 172.21.15.47:0/624643204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce3006a3f0 0x7fce1c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.171+0000 7fce3539f640 1 -- 172.21.15.47:0/624643204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce3006a3f0 msgr2=0x7fce1c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.171+0000 7fce3539f640 1 --2- 172.21.15.47:0/624643204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce3006a3f0 0x7fce1c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.171+0000 7fce3539f640 1 -- 172.21.15.47:0/624643204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce1c003970 msgr2=0x7fce1c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.171+0000 7fce3539f640 1 --2- 172.21.15.47:0/624643204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce1c003970 0x7fce1c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.171+0000 7fce3539f640 1 -- 172.21.15.47:0/624643204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c003520 con 0x7fce1c0047c0 2024-03-23T09:42:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.171+0000 7fce3539f640 1 --2- 172.21.15.47:0/624643204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce1c0047c0 0x7fce1c004c40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c240 tx=0x7fce1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40f4c11205d209 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.171+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2002cb00 con 0x7fce1c0047c0 2024-03-23T09:42:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.171+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2002cc60 con 0x7fce1c0047c0 2024-03-23T09:42:35.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.171+0000 7fce34b9e640 1 -- 172.21.15.47:0/624643204 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce200058d0 con 0x7fce1c0047c0 2024-03-23T09:42:35.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.172+0000 7fce36ba2640 1 -- 172.21.15.47:0/624643204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce1c0047c0 msgr2=0x7fce1c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.172+0000 7fce36ba2640 1 --2- 172.21.15.47:0/624643204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce1c0047c0 0x7fce1c004c40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c240 tx=0x7fce1c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.172+0000 7fce363a1640 1 -- 172.21.15.47:0/624643204 reap_dead start 2024-03-23T09:42:35.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.172+0000 7fce36ba2640 1 -- 172.21.15.47:0/624643204 shutdown_connections 2024-03-23T09:42:35.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.172+0000 7fce36ba2640 1 -- 172.21.15.47:0/624643204 >> 172.21.15.47:0/624643204 conn(0x7fce30098a00 msgr2=0x7fce300b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:35.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.172+0000 7fce36ba2640 1 -- 172.21.15.47:0/624643204 shutdown_connections 2024-03-23T09:42:35.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.172+0000 7fce36ba2640 1 -- 172.21.15.47:0/624643204 wait complete. 2024-03-23T09:42:35.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.173+0000 7fce36ba2640 1 Processor -- start 2024-03-23T09:42:35.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.173+0000 7fce36ba2640 1 -- start start 2024-03-23T09:42:35.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce36ba2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce30143e90 0x7fce3015f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce36ba2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce30144270 0x7fce300c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce36ba2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30147300 0x7fce3015b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce36ba2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce30154d20 con 0x7fce30143e90 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce36ba2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300c3120 con 0x7fce30147300 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce36ba2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce300c4700 con 0x7fce30144270 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce35ba0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce30143e90 0x7fce3015f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce35ba0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce30143e90 0x7fce3015f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44584/0 (socket says 172.21.15.47:44584) 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce35ba0640 1 -- 172.21.15.47:0/2466474966 learned_addr learned my addr 172.21.15.47:0/2466474966 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce363a1640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30147300 0x7fce3015b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce3539f640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce30144270 0x7fce300c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce35ba0640 1 -- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce30144270 msgr2=0x7fce300c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce35ba0640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce30144270 0x7fce300c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce35ba0640 1 -- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30147300 msgr2=0x7fce3015b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce35ba0640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30147300 0x7fce3015b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce35ba0640 1 -- 172.21.15.47:0/2466474966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce300c49e0 con 0x7fce30143e90 2024-03-23T09:42:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce363a1640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30147300 0x7fce3015b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.174+0000 7fce35ba0640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce30143e90 0x7fce3015f7f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fce2c011960 tx=0x7fce2c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.175+0000 7fce267fc640 1 -- 172.21.15.47:0/2466474966 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c00faf0 con 0x7fce30143e90 2024-03-23T09:42:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.175+0000 7fce267fc640 1 -- 172.21.15.47:0/2466474966 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c00fc50 con 0x7fce30143e90 2024-03-23T09:42:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.175+0000 7fce36ba2640 1 -- 172.21.15.47:0/2466474966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce300c4cc0 con 0x7fce30143e90 2024-03-23T09:42:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.175+0000 7fce267fc640 1 -- 172.21.15.47:0/2466474966 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c014440 con 0x7fce30143e90 2024-03-23T09:42:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.175+0000 7fce36ba2640 1 -- 172.21.15.47:0/2466474966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce300c5280 con 0x7fce30143e90 2024-03-23T09:42:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.176+0000 7fce36ba2640 1 -- 172.21.15.47:0/2466474966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce30073ef0 con 0x7fce30143e90 2024-03-23T09:42:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.176+0000 7fce267fc640 1 -- 172.21.15.47:0/2466474966 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fce2c0145a0 con 0x7fce30143e90 2024-03-23T09:42:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.177+0000 7fce267fc640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce1406ac00 0x7fce1406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.177+0000 7fce267fc640 1 -- 172.21.15.47:0/2466474966 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce2c089c30 con 0x7fce30143e90 2024-03-23T09:42:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.177+0000 7fce3539f640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce1406ac00 0x7fce1406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.177+0000 7fce3539f640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce1406ac00 0x7fce1406d0c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fce20034000 tx=0x7fce20007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:35.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.181+0000 7fce267fc640 1 -- 172.21.15.47:0/2466474966 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fce2c054e10 con 0x7fce30143e90 2024-03-23T09:42:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.359+0000 7fce36ba2640 1 -- 172.21.15.47:0/2466474966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce300b5310 con 0x7fce30143e90 2024-03-23T09:42:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.360+0000 7fce267fc640 1 -- 172.21.15.47:0/2466474966 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7fce2c058430 con 0x7fce30143e90 2024-03-23T09:42:35.360 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:42:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.362+0000 7fce36ba2640 1 -- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce1406ac00 msgr2=0x7fce1406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.362+0000 7fce36ba2640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce1406ac00 0x7fce1406d0c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fce20034000 tx=0x7fce20007000 comp rx=0 tx=0).stop 2024-03-23T09:42:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.362+0000 7fce36ba2640 1 -- 172.21.15.47:0/2466474966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce30143e90 msgr2=0x7fce3015f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.362+0000 7fce36ba2640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce30143e90 0x7fce3015f7f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fce2c011960 tx=0x7fce2c011e20 comp rx=0 tx=0).stop 2024-03-23T09:42:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.362+0000 7fce36ba2640 1 -- 172.21.15.47:0/2466474966 shutdown_connections 2024-03-23T09:42:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.362+0000 7fce36ba2640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce1406ac00 0x7fce1406d0c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.362+0000 7fce36ba2640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30147300 0x7fce3015b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.362+0000 7fce36ba2640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce30144270 0x7fce300c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.362+0000 7fce36ba2640 1 --2- 172.21.15.47:0/2466474966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce30143e90 0x7fce3015f7f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.362+0000 7fce36ba2640 1 -- 172.21.15.47:0/2466474966 >> 172.21.15.47:0/2466474966 conn(0x7fce30098a00 msgr2=0x7fce3006d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.362+0000 7fce36ba2640 1 -- 172.21.15.47:0/2466474966 shutdown_connections 2024-03-23T09:42:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.362+0000 7fce36ba2640 1 -- 172.21.15.47:0/2466474966 wait complete. 2024-03-23T09:42:35.372 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:42:35.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.473+0000 7f226fd3c640 1 Processor -- start 2024-03-23T09:42:35.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.473+0000 7f226fd3c640 1 -- start start 2024-03-23T09:42:35.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226fd3c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f226815ca90 0x7f2268160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226fd3c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f22680cde10 0x7f22680d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226fd3c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22680d11e0 con 0x7f22680b8350 2024-03-23T09:42:35.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226fd3c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2268160f60 con 0x7f22680cde10 2024-03-23T09:42:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226fd3c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2268161240 con 0x7f226815ca90 2024-03-23T09:42:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226d2d7640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f22680b8350 0x7f22680cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44968/0 (socket says 172.21.15.47:44968) 2024-03-23T09:42:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226d2d7640 1 -- 172.21.15.47:0/2484855407 learned_addr learned my addr 172.21.15.47:0/2484855407 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226e2d9640 1 --2- 172.21.15.47:0/2484855407 >> v2:172.21.15.175:3300/0 conn(0x7f22680cde10 0x7f22680d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226dad8640 1 --2- 172.21.15.47:0/2484855407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f226815ca90 0x7f2268160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682650052 0 0) 0x7f22680d11e0 con 0x7f22680b8350 2024-03-23T09:42:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2254003520 con 0x7f22680b8350 2024-03-23T09:42:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1295200545 0 0) 0x7f2254003520 con 0x7f22680b8350 2024-03-23T09:42:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f226815ca90 msgr2=0x7f2268160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 --2- 172.21.15.47:0/2484855407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f226815ca90 0x7f2268160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 >> v2:172.21.15.175:3300/0 conn(0x7f22680cde10 msgr2=0x7f22680d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 --2- 172.21.15.47:0/2484855407 >> v2:172.21.15.175:3300/0 conn(0x7f22680cde10 0x7f22680d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2268161520 con 0x7f22680b8350 2024-03-23T09:42:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f2258001670 con 0x7f22680b8350 2024-03-23T09:42:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 >> v1:172.21.15.47:6789/0 conn(0x7f22680b8350 legacy=0x7f22680cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 --2- 172.21.15.47:0/2484855407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2254003970 0x7f2254003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 --2- 172.21.15.47:0/2484855407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22540047c0 0x7f2254004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 --2- 172.21.15.47:0/2484855407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f226815ca90 0x7f2254005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2268160f60 con 0x7f22540047c0 2024-03-23T09:42:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2268161240 con 0x7f2254003970 2024-03-23T09:42:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.474+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225400b980 con 0x7f226815ca90 2024-03-23T09:42:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.475+0000 7f226d2d7640 1 --2- 172.21.15.47:0/2484855407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22540047c0 0x7f2254004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.475+0000 7f226dad8640 1 --2- 172.21.15.47:0/2484855407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2254003970 0x7f2254003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.475+0000 7f226e2d9640 1 --2- 172.21.15.47:0/2484855407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f226815ca90 0x7f2254005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.475+0000 7f226d2d7640 1 -- 172.21.15.47:0/2484855407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f226815ca90 msgr2=0x7f2254005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.475+0000 7f226d2d7640 1 --2- 172.21.15.47:0/2484855407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f226815ca90 0x7f2254005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.475+0000 7f226d2d7640 1 -- 172.21.15.47:0/2484855407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2254003970 msgr2=0x7f2254003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.475+0000 7f226d2d7640 1 --2- 172.21.15.47:0/2484855407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2254003970 0x7f2254003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.475+0000 7f226d2d7640 1 -- 172.21.15.47:0/2484855407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2254003520 con 0x7f22540047c0 2024-03-23T09:42:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.475+0000 7f226e2d9640 1 --2- 172.21.15.47:0/2484855407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f226815ca90 0x7f2254005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.476+0000 7f226d2d7640 1 --2- 172.21.15.47:0/2484855407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22540047c0 0x7f2254004c40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f225800c2c0 tx=0x7f22540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e69d5fff31a3cd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.476+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258005ad0 con 0x7f22540047c0 2024-03-23T09:42:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.476+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2258005c30 con 0x7f22540047c0 2024-03-23T09:42:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.476+0000 7f226cad6640 1 -- 172.21.15.47:0/2484855407 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225802c8b0 con 0x7f22540047c0 2024-03-23T09:42:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.476+0000 7f226fd3c640 1 -- 172.21.15.47:0/2484855407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22540047c0 msgr2=0x7f2254004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.476+0000 7f226fd3c640 1 --2- 172.21.15.47:0/2484855407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22540047c0 0x7f2254004c40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f225800c2c0 tx=0x7f22540057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.476+0000 7f226e2d9640 1 -- 172.21.15.47:0/2484855407 reap_dead start 2024-03-23T09:42:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.476+0000 7f226fd3c640 1 -- 172.21.15.47:0/2484855407 shutdown_connections 2024-03-23T09:42:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.476+0000 7f226fd3c640 1 -- 172.21.15.47:0/2484855407 >> 172.21.15.47:0/2484855407 conn(0x7f22680659d0 msgr2=0x7f2268065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.476+0000 7f226fd3c640 1 -- 172.21.15.47:0/2484855407 shutdown_connections 2024-03-23T09:42:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.477+0000 7f226fd3c640 1 -- 172.21.15.47:0/2484855407 wait complete. 2024-03-23T09:42:35.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.478+0000 7f226fd3c640 1 Processor -- start 2024-03-23T09:42:35.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.478+0000 7f226fd3c640 1 -- start start 2024-03-23T09:42:35.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.478+0000 7f226fd3c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22540047c0 0x7f22680cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.478+0000 7f226fd3c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22680d2e60 0x7f22680d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.478+0000 7f226fd3c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22680b3c90 0x7f22680b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.478+0000 7f226fd3c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226814a710 con 0x7f22540047c0 2024-03-23T09:42:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.478+0000 7f226fd3c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22680b4a30 con 0x7f22680b3c90 2024-03-23T09:42:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.478+0000 7f226fd3c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22680c9a90 con 0x7f22680d2e60 2024-03-23T09:42:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.478+0000 7f226dad8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22540047c0 0x7f22680cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.478+0000 7f226dad8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22540047c0 0x7f22680cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44604/0 (socket says 172.21.15.47:44604) 2024-03-23T09:42:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.478+0000 7f226dad8640 1 -- 172.21.15.47:0/4038715576 learned_addr learned my addr 172.21.15.47:0/4038715576 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f226d2d7640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22680d2e60 0x7f22680d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f226e2d9640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22680b3c90 0x7f22680b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f226dad8640 1 -- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22680d2e60 msgr2=0x7f22680d32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f226dad8640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22680d2e60 0x7f22680d32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f226dad8640 1 -- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22680b3c90 msgr2=0x7f22680b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f226dad8640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22680b3c90 0x7f22680b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f226dad8640 1 -- 172.21.15.47:0/4038715576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22680c9ca0 con 0x7f22540047c0 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f226d2d7640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22680d2e60 0x7f22680d32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f226dad8640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22540047c0 0x7f22680cc550 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f22640118e0 tx=0x7f2264011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f225e7fc640 1 -- 172.21.15.47:0/4038715576 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400faf0 con 0x7f22540047c0 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f225e7fc640 1 -- 172.21.15.47:0/4038715576 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226400fc50 con 0x7f22540047c0 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f226fd3c640 1 -- 172.21.15.47:0/4038715576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22680c9f80 con 0x7f22540047c0 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f225e7fc640 1 -- 172.21.15.47:0/4038715576 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264014440 con 0x7f22540047c0 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.479+0000 7f226fd3c640 1 -- 172.21.15.47:0/4038715576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22680ca540 con 0x7f22540047c0 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.480+0000 7f226fd3c640 1 -- 172.21.15.47:0/4038715576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22680ca700 con 0x7f22540047c0 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.481+0000 7f225e7fc640 1 -- 172.21.15.47:0/4038715576 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f22640145a0 con 0x7f22540047c0 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.481+0000 7f225e7fc640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f224c06ac00 0x7f224c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.481+0000 7f225e7fc640 1 -- 172.21.15.47:0/4038715576 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2264089b30 con 0x7f22540047c0 2024-03-23T09:42:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.481+0000 7f226d2d7640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f224c06ac00 0x7f224c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.482+0000 7f226d2d7640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f224c06ac00 0x7f224c06d0c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2258034000 tx=0x7f2258007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:35.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.485+0000 7f225e7fc640 1 -- 172.21.15.47:0/4038715576 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2264054da0 con 0x7f22540047c0 2024-03-23T09:42:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.699+0000 7f226fd3c640 1 -- 172.21.15.47:0/4038715576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2268072360 con 0x7f22540047c0 2024-03-23T09:42:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.700+0000 7f225e7fc640 1 -- 172.21.15.47:0/4038715576 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f22640583c0 con 0x7f22540047c0 2024-03-23T09:42:35.700 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:35.700 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3373309448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3373309448},{"type":"v1","addr":"172.21.15.175:6836","nonce":3373309448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4118066020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4118066020},{"type":"v1","addr":"172.21.15.175:6837","nonce":4118066020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2916649516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2916649516},{"type":"v1","addr":"172.21.15.47:6835","nonce":2916649516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:59.719783+0000","modified":"2024-03-23T09:42:01.774130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6834/400299689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":400299689},{"type":"v1","addr":"172.21.15.47:6834","nonce":400299689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T09:42:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2024-03-23T09:42:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.702+0000 7f226fd3c640 1 -- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f224c06ac00 msgr2=0x7f224c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.702+0000 7f226fd3c640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f224c06ac00 0x7f224c06d0c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2258034000 tx=0x7f2258007000 comp rx=0 tx=0).stop 2024-03-23T09:42:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.702+0000 7f226fd3c640 1 -- 172.21.15.47:0/4038715576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22540047c0 msgr2=0x7f22680cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.702+0000 7f226fd3c640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22540047c0 0x7f22680cc550 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f22640118e0 tx=0x7f2264011da0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.702+0000 7f226fd3c640 1 -- 172.21.15.47:0/4038715576 shutdown_connections 2024-03-23T09:42:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.702+0000 7f226fd3c640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f224c06ac00 0x7f224c06d0c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.702+0000 7f226fd3c640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22680b3c90 0x7f22680b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.702+0000 7f226fd3c640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22680d2e60 0x7f22680d32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.702+0000 7f226fd3c640 1 --2- 172.21.15.47:0/4038715576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22540047c0 0x7f22680cc550 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.702+0000 7f226fd3c640 1 -- 172.21.15.47:0/4038715576 >> 172.21.15.47:0/4038715576 conn(0x7f22680659d0 msgr2=0x7f22680b6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:35.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.702+0000 7f226fd3c640 1 -- 172.21.15.47:0/4038715576 shutdown_connections 2024-03-23T09:42:35.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.702+0000 7f226fd3c640 1 -- 172.21.15.47:0/4038715576 wait complete. 2024-03-23T09:42:35.712 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:42:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.814+0000 7f392131b640 1 Processor -- start 2024-03-23T09:42:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.814+0000 7f392131b640 1 -- start start 2024-03-23T09:42:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.814+0000 7f392131b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f391c15cb50 0x7f391c0b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.814+0000 7f392131b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f391c0cd9e0 0x7f391c0b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.814+0000 7f392131b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f391c0b72f0 con 0x7f391c0cd5a0 2024-03-23T09:42:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.814+0000 7f392131b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391c0b7e70 con 0x7f391c0cd9e0 2024-03-23T09:42:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.814+0000 7f392131b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391c15cf30 con 0x7f391c15cb50 2024-03-23T09:42:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.815+0000 7f391b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f391c0cd9e0 0x7f391c0b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.815+0000 7f391b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f391c0cd9e0 0x7f391c0b6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44616/0 (socket says 172.21.15.47:44616) 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.815+0000 7f391b7fe640 1 -- 172.21.15.47:0/1985795306 learned_addr learned my addr 172.21.15.47:0/1985795306 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.815+0000 7f391affd640 1 --2- 172.21.15.47:0/1985795306 >> v2:172.21.15.175:3300/0 conn(0x7f391c15cb50 0x7f391c0b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.815+0000 7f391b7fe640 1 -- 172.21.15.47:0/1985795306 >> v2:172.21.15.175:3300/0 conn(0x7f391c15cb50 msgr2=0x7f391c0b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.815+0000 7f391b7fe640 1 --2- 172.21.15.47:0/1985795306 >> v2:172.21.15.175:3300/0 conn(0x7f391c15cb50 0x7f391c0b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.815+0000 7f391b7fe640 1 -- 172.21.15.47:0/1985795306 >> v1:172.21.15.175:6789/0 conn(0x7f391c0cd5a0 legacy=0x7f391c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.815+0000 7f391b7fe640 1 -- 172.21.15.47:0/1985795306 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391c0b75d0 con 0x7f391c0cd9e0 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.815+0000 7f391affd640 1 --2- 172.21.15.47:0/1985795306 >> v2:172.21.15.175:3300/0 conn(0x7f391c15cb50 0x7f391c0b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.816+0000 7f391b7fe640 1 --2- 172.21.15.47:0/1985795306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f391c0cd9e0 0x7f391c0b6a10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f391c0d39a0 tx=0x7f390c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d23876aba925c7a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.816+0000 7f3919ffb640 1 -- 172.21.15.47:0/1985795306 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c02cbb0 con 0x7f391c0cd9e0 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.816+0000 7f3919ffb640 1 -- 172.21.15.47:0/1985795306 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c02cd10 con 0x7f391c0cd9e0 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.816+0000 7f3919ffb640 1 -- 172.21.15.47:0/1985795306 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c034610 con 0x7f391c0cd9e0 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.816+0000 7f392131b640 1 -- 172.21.15.47:0/1985795306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f391c0cd9e0 msgr2=0x7f391c0b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.816+0000 7f392131b640 1 --2- 172.21.15.47:0/1985795306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f391c0cd9e0 0x7f391c0b6a10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f391c0d39a0 tx=0x7f390c02c010 comp rx=0 tx=0).stop 2024-03-23T09:42:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.816+0000 7f392131b640 1 -- 172.21.15.47:0/1985795306 shutdown_connections 2024-03-23T09:42:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.816+0000 7f392131b640 1 --2- 172.21.15.47:0/1985795306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f391c0cd9e0 0x7f391c0b6a10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.816+0000 7f392131b640 1 --2- 172.21.15.47:0/1985795306 >> v2:172.21.15.175:3300/0 conn(0x7f391c15cb50 0x7f391c0b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.816+0000 7f392131b640 1 -- 172.21.15.47:0/1985795306 >> 172.21.15.47:0/1985795306 conn(0x7f391c065df0 msgr2=0x7f391c066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.816+0000 7f392131b640 1 -- 172.21.15.47:0/1985795306 shutdown_connections 2024-03-23T09:42:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.816+0000 7f392131b640 1 -- 172.21.15.47:0/1985795306 wait complete. 2024-03-23T09:42:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.818+0000 7f392131b640 1 Processor -- start 2024-03-23T09:42:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.818+0000 7f392131b640 1 -- start start 2024-03-23T09:42:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.818+0000 7f392131b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f391c0cd5a0 0x7f391c0c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.818+0000 7f392131b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f391c0cd9e0 0x7f391c155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.818+0000 7f392131b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c15cb50 0x7f391c0b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f392131b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c0caa20 con 0x7f391c0cd9e0 2024-03-23T09:42:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f392131b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c1f5590 con 0x7f391c15cb50 2024-03-23T09:42:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f392131b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391c1f5e30 con 0x7f391c0cd5a0 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f391a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f391c0cd9e0 0x7f391c155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f391a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f391c0cd9e0 0x7f391c155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44618/0 (socket says 172.21.15.47:44618) 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f391a7fc640 1 -- 172.21.15.47:0/3729531920 learned_addr learned my addr 172.21.15.47:0/3729531920 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f391b7fe640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c15cb50 0x7f391c0b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f391affd640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f391c0cd5a0 0x7f391c0c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f391a7fc640 1 -- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f391c0cd5a0 msgr2=0x7f391c0c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f391a7fc640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f391c0cd5a0 0x7f391c0c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f391a7fc640 1 -- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c15cb50 msgr2=0x7f391c0b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f391a7fc640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c15cb50 0x7f391c0b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f391a7fc640 1 -- 172.21.15.47:0/3729531920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c009b30 con 0x7f391c0cd9e0 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f391a7fc640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f391c0cd9e0 0x7f391c155da0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f390400c3f0 tx=0x7f390400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f39137fe640 1 -- 172.21.15.47:0/3729531920 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39040052d0 con 0x7f391c0cd9e0 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f39137fe640 1 -- 172.21.15.47:0/3729531920 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3904005450 con 0x7f391c0cd9e0 2024-03-23T09:42:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f392131b640 1 -- 172.21.15.47:0/3729531920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391c159770 con 0x7f391c0cd9e0 2024-03-23T09:42:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f39137fe640 1 -- 172.21.15.47:0/3729531920 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904007260 con 0x7f391c0cd9e0 2024-03-23T09:42:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.819+0000 7f392131b640 1 -- 172.21.15.47:0/3729531920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391c1562e0 con 0x7f391c0cd9e0 2024-03-23T09:42:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.820+0000 7f392131b640 1 -- 172.21.15.47:0/3729531920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391c072360 con 0x7f391c0cd9e0 2024-03-23T09:42:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.821+0000 7f39137fe640 1 -- 172.21.15.47:0/3729531920 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f39040073c0 con 0x7f391c0cd9e0 2024-03-23T09:42:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.821+0000 7f39137fe640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f38fc06ac00 0x7f38fc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.821+0000 7f39137fe640 1 -- 172.21.15.47:0/3729531920 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3904088710 con 0x7f391c0cd9e0 2024-03-23T09:42:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.821+0000 7f391affd640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f38fc06ac00 0x7f38fc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.822+0000 7f391affd640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f38fc06ac00 0x7f38fc06d0c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3914006fd0 tx=0x7f3914008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:35.825+0000 7f39137fe640 1 -- 172.21.15.47:0/3729531920 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f39040538f0 con 0x7f391c0cd9e0 2024-03-23T09:42:36.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.005+0000 7f392131b640 1 -- 172.21.15.47:0/3729531920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f391c066be0 con 0x7f391c0cd9e0 2024-03-23T09:42:36.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.006+0000 7f39137fe640 1 -- 172.21.15.47:0/3729531920 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f3904056f10 con 0x7f391c0cd9e0 2024-03-23T09:42:36.006 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:36.006 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:00.721483+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:58.297353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:58.685375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:42:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.008+0000 7f392131b640 1 -- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f38fc06ac00 msgr2=0x7f38fc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.008+0000 7f392131b640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f38fc06ac00 0x7f38fc06d0c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3914006fd0 tx=0x7f3914008040 comp rx=0 tx=0).stop 2024-03-23T09:42:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.008+0000 7f392131b640 1 -- 172.21.15.47:0/3729531920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f391c0cd9e0 msgr2=0x7f391c155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.008+0000 7f392131b640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f391c0cd9e0 0x7f391c155da0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f390400c3f0 tx=0x7f390400c8b0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.008+0000 7f392131b640 1 -- 172.21.15.47:0/3729531920 shutdown_connections 2024-03-23T09:42:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.008+0000 7f392131b640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f38fc06ac00 0x7f38fc06d0c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.008+0000 7f392131b640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c15cb50 0x7f391c0b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.008+0000 7f392131b640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f391c0cd9e0 0x7f391c155da0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.008+0000 7f392131b640 1 --2- 172.21.15.47:0/3729531920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f391c0cd5a0 0x7f391c0c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.008+0000 7f392131b640 1 -- 172.21.15.47:0/3729531920 >> 172.21.15.47:0/3729531920 conn(0x7f391c065df0 msgr2=0x7f391c0b6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.008+0000 7f392131b640 1 -- 172.21.15.47:0/3729531920 shutdown_connections 2024-03-23T09:42:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.008+0000 7f392131b640 1 -- 172.21.15.47:0/3729531920 wait complete. 2024-03-23T09:42:36.019 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T09:42:36.019 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:42:36.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.120+0000 7f74f7573640 1 Processor -- start 2024-03-23T09:42:36.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.120+0000 7f74f7573640 1 -- start start 2024-03-23T09:42:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f7573640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f74f015cd00 0x7f74f00d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f7573640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f74f00cf920 0x7f74f00b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f7573640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f015d0e0 con 0x7f74f00d1020 2024-03-23T09:42:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f7573640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f00d1540 con 0x7f74f00cf920 2024-03-23T09:42:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f7573640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f00d1820 con 0x7f74f015cd00 2024-03-23T09:42:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f4b0e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f74f00d1020 0x7f74f00cf210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44976/0 (socket says 172.21.15.47:44976) 2024-03-23T09:42:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f4b0e640 1 -- 172.21.15.47:0/3626048138 learned_addr learned my addr 172.21.15.47:0/3626048138 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f5b10640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f74f00cf920 0x7f74f00b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f5b10640 1 -- 172.21.15.47:0/3626048138 >> v2:172.21.15.175:3300/0 conn(0x7f74f015cd00 msgr2=0x7f74f00d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f530f640 1 --2- 172.21.15.47:0/3626048138 >> v2:172.21.15.175:3300/0 conn(0x7f74f015cd00 0x7f74f00d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74e7fff640 1 -- 172.21.15.47:0/3626048138 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3168907173 0 0) 0x7f74f015d0e0 con 0x7f74f00d1020 2024-03-23T09:42:36.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f5b10640 1 --2- 172.21.15.47:0/3626048138 >> v2:172.21.15.175:3300/0 conn(0x7f74f015cd00 0x7f74f00d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f5b10640 1 -- 172.21.15.47:0/3626048138 >> v1:172.21.15.47:6789/0 conn(0x7f74f00d1020 legacy=0x7f74f00cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f5b10640 1 -- 172.21.15.47:0/3626048138 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f00d1b00 con 0x7f74f00cf920 2024-03-23T09:42:36.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.121+0000 7f74f530f640 1 --2- 172.21.15.47:0/3626048138 >> v2:172.21.15.175:3300/0 conn(0x7f74f015cd00 0x7f74f00d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:36.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.122+0000 7f74f5b10640 1 --2- 172.21.15.47:0/3626048138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f00cf920 0x7f74f00b5a90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f74f006c1a0 tx=0x7f74e802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c593922ea0796818 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:36.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.122+0000 7f74e7fff640 1 -- 172.21.15.47:0/3626048138 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e802cbe0 con 0x7f74f00cf920 2024-03-23T09:42:36.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.122+0000 7f74e7fff640 1 -- 172.21.15.47:0/3626048138 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e802cd40 con 0x7f74f00cf920 2024-03-23T09:42:36.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.122+0000 7f74e7fff640 1 -- 172.21.15.47:0/3626048138 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e8034660 con 0x7f74f00cf920 2024-03-23T09:42:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.123+0000 7f74f7573640 1 -- 172.21.15.47:0/3626048138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f00cf920 msgr2=0x7f74f00b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.123+0000 7f74f7573640 1 --2- 172.21.15.47:0/3626048138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f00cf920 0x7f74f00b5a90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f74f006c1a0 tx=0x7f74e802be30 comp rx=0 tx=0).stop 2024-03-23T09:42:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.123+0000 7f74f7573640 1 -- 172.21.15.47:0/3626048138 shutdown_connections 2024-03-23T09:42:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.123+0000 7f74f7573640 1 --2- 172.21.15.47:0/3626048138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f00cf920 0x7f74f00b5a90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.123+0000 7f74f7573640 1 --2- 172.21.15.47:0/3626048138 >> v2:172.21.15.175:3300/0 conn(0x7f74f015cd00 0x7f74f00d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.123+0000 7f74f7573640 1 -- 172.21.15.47:0/3626048138 >> 172.21.15.47:0/3626048138 conn(0x7f74f006b160 msgr2=0x7f74f006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.123+0000 7f74f7573640 1 -- 172.21.15.47:0/3626048138 shutdown_connections 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.123+0000 7f74f7573640 1 -- 172.21.15.47:0/3626048138 wait complete. 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.124+0000 7f74f7573640 1 Processor -- start 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.124+0000 7f74f7573640 1 -- start start 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f7573640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f00cf920 0x7f74f00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f7573640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74f00d1020 0x7f74f00cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f7573640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f015cd00 0x7f74f00a9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f7573640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00b29f0 con 0x7f74f015cd00 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f7573640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00b2fb0 con 0x7f74f00cf920 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f7573640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00b52f0 con 0x7f74f00d1020 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f5b10640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f015cd00 0x7f74f00a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f5b10640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f015cd00 0x7f74f00a9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44646/0 (socket says 172.21.15.47:44646) 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f5b10640 1 -- 172.21.15.47:0/3357263696 learned_addr learned my addr 172.21.15.47:0/3357263696 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f4b0e640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74f00d1020 0x7f74f00cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f530f640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f00cf920 0x7f74f00b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f5b10640 1 -- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74f00d1020 msgr2=0x7f74f00cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f5b10640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74f00d1020 0x7f74f00cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f5b10640 1 -- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f00cf920 msgr2=0x7f74f00b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f5b10640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f00cf920 0x7f74f00b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f5b10640 1 -- 172.21.15.47:0/3357263696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e8009a70 con 0x7f74f015cd00 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f5b10640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f015cd00 0x7f74f00a9330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009e50 tx=0x7f74e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74e5ffb640 1 -- 172.21.15.47:0/3357263696 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e803e610 con 0x7f74f015cd00 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74e5ffb640 1 -- 172.21.15.47:0/3357263696 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e803e770 con 0x7f74f015cd00 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74f7573640 1 -- 172.21.15.47:0/3357263696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f00cbf10 con 0x7f74f015cd00 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.125+0000 7f74e5ffb640 1 -- 172.21.15.47:0/3357263696 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e803eae0 con 0x7f74f015cd00 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.126+0000 7f74f7573640 1 -- 172.21.15.47:0/3357263696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f00b2c40 con 0x7f74f015cd00 2024-03-23T09:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.127+0000 7f74f7573640 1 -- 172.21.15.47:0/3357263696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f0072360 con 0x7f74f015cd00 2024-03-23T09:42:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.127+0000 7f74e5ffb640 1 -- 172.21.15.47:0/3357263696 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f74e803b050 con 0x7f74f015cd00 2024-03-23T09:42:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.127+0000 7f74e5ffb640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f74d406ac00 0x7f74d406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.128+0000 7f74e5ffb640 1 -- 172.21.15.47:0/3357263696 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74e80ab9f0 con 0x7f74f015cd00 2024-03-23T09:42:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.128+0000 7f74f530f640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f74d406ac00 0x7f74d406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.128+0000 7f74f530f640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f74d406ac00 0x7f74d406d0c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00a950 tx=0x7f74ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.132+0000 7f74e5ffb640 1 -- 172.21.15.47:0/3357263696 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f74e8076d80 con 0x7f74f015cd00 2024-03-23T09:42:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.348+0000 7f74f7573640 1 -- 172.21.15.47:0/3357263696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74f0066be0 con 0x7f74f015cd00 2024-03-23T09:42:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.348+0000 7f74e5ffb640 1 -- 172.21.15.47:0/3357263696 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f74e8039020 con 0x7f74f015cd00 2024-03-23T09:42:36.348 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:36.348 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3373309448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3373309448},{"type":"v1","addr":"172.21.15.175:6836","nonce":3373309448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4118066020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4118066020},{"type":"v1","addr":"172.21.15.175:6837","nonce":4118066020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2916649516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2916649516},{"type":"v1","addr":"172.21.15.47:6835","nonce":2916649516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:59.719783+0000","modified":"2024-03-23T09:42:01.774130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6834/400299689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":400299689},{"type":"v1","addr":"172.21.15.47:6834","nonce":400299689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T09:42:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2024-03-23T09:42:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.350+0000 7f74f7573640 1 -- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f74d406ac00 msgr2=0x7f74d406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.350+0000 7f74f7573640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f74d406ac00 0x7f74d406d0c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00a950 tx=0x7f74ec008040 comp rx=0 tx=0).stop 2024-03-23T09:42:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.350+0000 7f74f7573640 1 -- 172.21.15.47:0/3357263696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f015cd00 msgr2=0x7f74f00a9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.350+0000 7f74f7573640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f015cd00 0x7f74f00a9330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009e50 tx=0x7f74e8004a40 comp rx=0 tx=0).stop 2024-03-23T09:42:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.350+0000 7f74f7573640 1 -- 172.21.15.47:0/3357263696 shutdown_connections 2024-03-23T09:42:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.350+0000 7f74f7573640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f74d406ac00 0x7f74d406d0c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.350+0000 7f74f7573640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f015cd00 0x7f74f00a9330 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.350+0000 7f74f7573640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74f00d1020 0x7f74f00cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.350+0000 7f74f7573640 1 --2- 172.21.15.47:0/3357263696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f00cf920 0x7f74f00b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.350+0000 7f74f7573640 1 -- 172.21.15.47:0/3357263696 >> 172.21.15.47:0/3357263696 conn(0x7f74f006b160 msgr2=0x7f74f015d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:36.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.350+0000 7f74f7573640 1 -- 172.21.15.47:0/3357263696 shutdown_connections 2024-03-23T09:42:36.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.350+0000 7f74f7573640 1 -- 172.21.15.47:0/3357263696 wait complete. 2024-03-23T09:42:36.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:42:36.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.461+0000 7fda7b9ae640 1 Processor -- start 2024-03-23T09:42:36.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.461+0000 7fda7b9ae640 1 -- start start 2024-03-23T09:42:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.462+0000 7fda7b9ae640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fda7415a8d0 0x7fda740cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.462+0000 7fda7b9ae640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fda740cb730 0x7fda740cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.462+0000 7fda7b9ae640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda740cf190 0x7fda74098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.462+0000 7fda7b9ae640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda740c41c0 con 0x7fda7415a8d0 2024-03-23T09:42:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.462+0000 7fda7b9ae640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda740cf600 con 0x7fda740cb730 2024-03-23T09:42:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.462+0000 7fda7b9ae640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda740cc610 con 0x7fda740cf190 2024-03-23T09:42:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.462+0000 7fda7974a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fda7415a8d0 0x7fda740cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.462+0000 7fda7974a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fda7415a8d0 0x7fda740cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44656/0 (socket says 172.21.15.47:44656) 2024-03-23T09:42:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.462+0000 7fda7974a640 1 -- 172.21.15.47:0/3074494297 learned_addr learned my addr 172.21.15.47:0/3074494297 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.463+0000 7fda78f49640 1 --2- 172.21.15.47:0/3074494297 >> v2:172.21.15.175:3300/0 conn(0x7fda740cb730 0x7fda740cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.463+0000 7fda79f4b640 1 --2- 172.21.15.47:0/3074494297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda740cf190 0x7fda74098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.463+0000 7fda7974a640 1 -- 172.21.15.47:0/3074494297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda740cf190 msgr2=0x7fda74098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.463+0000 7fda7974a640 1 --2- 172.21.15.47:0/3074494297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda740cf190 0x7fda74098a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.463+0000 7fda7974a640 1 -- 172.21.15.47:0/3074494297 >> v2:172.21.15.175:3300/0 conn(0x7fda740cb730 msgr2=0x7fda740cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.463+0000 7fda7974a640 1 --2- 172.21.15.47:0/3074494297 >> v2:172.21.15.175:3300/0 conn(0x7fda740cb730 0x7fda740cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.463+0000 7fda7974a640 1 -- 172.21.15.47:0/3074494297 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda740b0700 con 0x7fda7415a8d0 2024-03-23T09:42:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.463+0000 7fda7974a640 1 --2- 172.21.15.47:0/3074494297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda7415a8d0 0x7fda740cb1f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fda740b3170 tx=0x7fda6c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1634787444cec79b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.463+0000 7fda73fff640 1 -- 172.21.15.47:0/3074494297 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c0317e0 con 0x7fda7415a8d0 2024-03-23T09:42:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.464+0000 7fda73fff640 1 -- 172.21.15.47:0/3074494297 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c031940 con 0x7fda7415a8d0 2024-03-23T09:42:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.464+0000 7fda73fff640 1 -- 172.21.15.47:0/3074494297 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c031cd0 con 0x7fda7415a8d0 2024-03-23T09:42:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.464+0000 7fda7b9ae640 1 -- 172.21.15.47:0/3074494297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda7415a8d0 msgr2=0x7fda740cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.464+0000 7fda7b9ae640 1 --2- 172.21.15.47:0/3074494297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda7415a8d0 0x7fda740cb1f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fda740b3170 tx=0x7fda6c02b880 comp rx=0 tx=0).stop 2024-03-23T09:42:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.464+0000 7fda7b9ae640 1 -- 172.21.15.47:0/3074494297 shutdown_connections 2024-03-23T09:42:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.464+0000 7fda7b9ae640 1 --2- 172.21.15.47:0/3074494297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda740cf190 0x7fda74098a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.464+0000 7fda7b9ae640 1 --2- 172.21.15.47:0/3074494297 >> v2:172.21.15.175:3300/0 conn(0x7fda740cb730 0x7fda740cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.464+0000 7fda7b9ae640 1 --2- 172.21.15.47:0/3074494297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda7415a8d0 0x7fda740cb1f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.464+0000 7fda7b9ae640 1 -- 172.21.15.47:0/3074494297 >> 172.21.15.47:0/3074494297 conn(0x7fda740d0670 msgr2=0x7fda740d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.464+0000 7fda7b9ae640 1 -- 172.21.15.47:0/3074494297 shutdown_connections 2024-03-23T09:42:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.464+0000 7fda7b9ae640 1 -- 172.21.15.47:0/3074494297 wait complete. 2024-03-23T09:42:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.465+0000 7fda7b9ae640 1 Processor -- start 2024-03-23T09:42:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.465+0000 7fda7b9ae640 1 -- start start 2024-03-23T09:42:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7b9ae640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda740cb730 0x7fda7414ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7b9ae640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda740cf190 0x7fda740c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7b9ae640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda7415a8d0 0x7fda740c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7b9ae640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda74157040 con 0x7fda740cb730 2024-03-23T09:42:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7b9ae640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda740ca560 con 0x7fda740cf190 2024-03-23T09:42:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7b9ae640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda740ca840 con 0x7fda7415a8d0 2024-03-23T09:42:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7974a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda740cb730 0x7fda7414ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7974a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda740cb730 0x7fda7414ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44662/0 (socket says 172.21.15.47:44662) 2024-03-23T09:42:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7974a640 1 -- 172.21.15.47:0/974651773 learned_addr learned my addr 172.21.15.47:0/974651773 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7974a640 1 -- 172.21.15.47:0/974651773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda7415a8d0 msgr2=0x7fda740c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7974a640 1 --2- 172.21.15.47:0/974651773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda7415a8d0 0x7fda740c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7974a640 1 -- 172.21.15.47:0/974651773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda740cf190 msgr2=0x7fda740c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda78f49640 1 --2- 172.21.15.47:0/974651773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda740cf190 0x7fda740c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7974a640 1 --2- 172.21.15.47:0/974651773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda740cf190 0x7fda740c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.466+0000 7fda7974a640 1 -- 172.21.15.47:0/974651773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c0094d0 con 0x7fda740cb730 2024-03-23T09:42:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.467+0000 7fda7974a640 1 --2- 172.21.15.47:0/974651773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda740cb730 0x7fda7414ca60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fda6c02b8c0 tx=0x7fda6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.467+0000 7fda71ffb640 1 -- 172.21.15.47:0/974651773 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c032880 con 0x7fda740cb730 2024-03-23T09:42:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.467+0000 7fda71ffb640 1 -- 172.21.15.47:0/974651773 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c0329e0 con 0x7fda740cb730 2024-03-23T09:42:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.467+0000 7fda7b9ae640 1 -- 172.21.15.47:0/974651773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda740ad590 con 0x7fda740cb730 2024-03-23T09:42:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.467+0000 7fda71ffb640 1 -- 172.21.15.47:0/974651773 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c032d70 con 0x7fda740cb730 2024-03-23T09:42:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.467+0000 7fda7b9ae640 1 -- 172.21.15.47:0/974651773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7414cfa0 con 0x7fda740cb730 2024-03-23T09:42:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.468+0000 7fda7b9ae640 1 -- 172.21.15.47:0/974651773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda740ad3f0 con 0x7fda740cb730 2024-03-23T09:42:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.468+0000 7fda71ffb640 1 -- 172.21.15.47:0/974651773 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fda6c0453f0 con 0x7fda740cb730 2024-03-23T09:42:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.469+0000 7fda71ffb640 1 --2- 172.21.15.47:0/974651773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda6006ac00 0x7fda6006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.469+0000 7fda71ffb640 1 -- 172.21.15.47:0/974651773 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda6c0ab6d0 con 0x7fda740cb730 2024-03-23T09:42:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.469+0000 7fda78f49640 1 --2- 172.21.15.47:0/974651773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda6006ac00 0x7fda6006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.469+0000 7fda78f49640 1 --2- 172.21.15.47:0/974651773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda6006ac00 0x7fda6006d0c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda740c70f0 tx=0x7fda68008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:36.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.473+0000 7fda71ffb640 1 -- 172.21.15.47:0/974651773 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fda740ad3f0 con 0x7fda740cb730 2024-03-23T09:42:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.689+0000 7fda7b9ae640 1 -- 172.21.15.47:0/974651773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda740adab0 con 0x7fda740cb730 2024-03-23T09:42:36.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.690+0000 7fda71ffb640 1 -- 172.21.15.47:0/974651773 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fda740adab0 con 0x7fda740cb730 2024-03-23T09:42:36.690 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:36.690 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3373309448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3373309448},{"type":"v1","addr":"172.21.15.175:6836","nonce":3373309448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4118066020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4118066020},{"type":"v1","addr":"172.21.15.175:6837","nonce":4118066020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2916649516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2916649516},{"type":"v1","addr":"172.21.15.47:6835","nonce":2916649516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:41:59.719783+0000","modified":"2024-03-23T09:42:01.774130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6834/400299689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":400299689},{"type":"v1","addr":"172.21.15.47:6834","nonce":400299689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T09:42:36.690 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2024-03-23T09:42:36.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.692+0000 7fda7b9ae640 1 -- 172.21.15.47:0/974651773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda6006ac00 msgr2=0x7fda6006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.692+0000 7fda7b9ae640 1 --2- 172.21.15.47:0/974651773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda6006ac00 0x7fda6006d0c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda740c70f0 tx=0x7fda68008040 comp rx=0 tx=0).stop 2024-03-23T09:42:36.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.692+0000 7fda7b9ae640 1 -- 172.21.15.47:0/974651773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda740cb730 msgr2=0x7fda7414ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.692+0000 7fda7b9ae640 1 --2- 172.21.15.47:0/974651773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda740cb730 0x7fda7414ca60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fda6c02b8c0 tx=0x7fda6c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.692+0000 7fda7b9ae640 1 -- 172.21.15.47:0/974651773 shutdown_connections 2024-03-23T09:42:36.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.692+0000 7fda7b9ae640 1 --2- 172.21.15.47:0/974651773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda6006ac00 0x7fda6006d0c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.692+0000 7fda7b9ae640 1 --2- 172.21.15.47:0/974651773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda7415a8d0 0x7fda740c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.692+0000 7fda7b9ae640 1 --2- 172.21.15.47:0/974651773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda740cf190 0x7fda740c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.692+0000 7fda7b9ae640 1 --2- 172.21.15.47:0/974651773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda740cb730 0x7fda7414ca60 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.692+0000 7fda7b9ae640 1 -- 172.21.15.47:0/974651773 >> 172.21.15.47:0/974651773 conn(0x7fda740d0670 msgr2=0x7fda740c6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:36.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.692+0000 7fda7b9ae640 1 -- 172.21.15.47:0/974651773 shutdown_connections 2024-03-23T09:42:36.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.692+0000 7fda7b9ae640 1 -- 172.21.15.47:0/974651773 wait complete. 2024-03-23T09:42:36.703 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:42:36.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.803+0000 7f559c08b640 1 Processor -- start 2024-03-23T09:42:36.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.804+0000 7f559c08b640 1 -- start start 2024-03-23T09:42:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.804+0000 7f559c08b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f55940b8350 0x7f55940cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.804+0000 7f559c08b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55940cdc40 0x7f55940d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.804+0000 7f559c08b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55940ce090 con 0x7f559415ca90 2024-03-23T09:42:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.804+0000 7f559c08b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55940d11e0 con 0x7f55940cdc40 2024-03-23T09:42:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.804+0000 7f559c08b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5594160f60 con 0x7f55940b8350 2024-03-23T09:42:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.805+0000 7f559a628640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55940cdc40 0x7f55940d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.805+0000 7f559a628640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55940cdc40 0x7f55940d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44670/0 (socket says 172.21.15.47:44670) 2024-03-23T09:42:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.805+0000 7f559a628640 1 -- 172.21.15.47:0/8311207 learned_addr learned my addr 172.21.15.47:0/8311207 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.805+0000 7f5599626640 1 --2- 172.21.15.47:0/8311207 >> v2:172.21.15.175:3300/0 conn(0x7f55940b8350 0x7f55940cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.805+0000 7f559a628640 1 -- 172.21.15.47:0/8311207 >> v2:172.21.15.175:3300/0 conn(0x7f55940b8350 msgr2=0x7f55940cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.805+0000 7f559a628640 1 --2- 172.21.15.47:0/8311207 >> v2:172.21.15.175:3300/0 conn(0x7f55940b8350 0x7f55940cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.805+0000 7f559a628640 1 -- 172.21.15.47:0/8311207 >> v1:172.21.15.175:6789/0 conn(0x7f559415ca90 legacy=0x7f5594160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.805+0000 7f559a628640 1 -- 172.21.15.47:0/8311207 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5594161240 con 0x7f55940cdc40 2024-03-23T09:42:36.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.806+0000 7f559a628640 1 --2- 172.21.15.47:0/8311207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55940cdc40 0x7f55940d1670 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5594160670 tx=0x7f558402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbec564442afedb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:36.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.806+0000 7f5598e25640 1 -- 172.21.15.47:0/8311207 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558402ca80 con 0x7f55940cdc40 2024-03-23T09:42:36.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.806+0000 7f5598e25640 1 -- 172.21.15.47:0/8311207 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558402cbe0 con 0x7f55940cdc40 2024-03-23T09:42:36.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.806+0000 7f5598e25640 1 -- 172.21.15.47:0/8311207 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55840343f0 con 0x7f55940cdc40 2024-03-23T09:42:36.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.806+0000 7f559c08b640 1 -- 172.21.15.47:0/8311207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55940cdc40 msgr2=0x7f55940d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.806+0000 7f559c08b640 1 --2- 172.21.15.47:0/8311207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55940cdc40 0x7f55940d1670 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5594160670 tx=0x7f558402bee0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.806+0000 7f559c08b640 1 -- 172.21.15.47:0/8311207 shutdown_connections 2024-03-23T09:42:36.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.806+0000 7f559c08b640 1 --2- 172.21.15.47:0/8311207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55940cdc40 0x7f55940d1670 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.806+0000 7f559c08b640 1 --2- 172.21.15.47:0/8311207 >> v2:172.21.15.175:3300/0 conn(0x7f55940b8350 0x7f55940cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.806+0000 7f559c08b640 1 -- 172.21.15.47:0/8311207 >> 172.21.15.47:0/8311207 conn(0x7f55940659d0 msgr2=0x7f5594065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:36.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.806+0000 7f559c08b640 1 -- 172.21.15.47:0/8311207 shutdown_connections 2024-03-23T09:42:36.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.806+0000 7f559c08b640 1 -- 172.21.15.47:0/8311207 wait complete. 2024-03-23T09:42:36.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.807+0000 7f559c08b640 1 Processor -- start 2024-03-23T09:42:36.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.807+0000 7f559c08b640 1 -- start start 2024-03-23T09:42:36.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.808+0000 7f559c08b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55940b8350 0x7f5594150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.808+0000 7f559c08b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55940cdc40 0x7f55940b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.808+0000 7f559c08b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f559415ca90 0x7f55940b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.808+0000 7f559c08b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5594151480 con 0x7f55940b8350 2024-03-23T09:42:36.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.808+0000 7f559c08b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55941f3710 con 0x7f559415ca90 2024-03-23T09:42:36.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.808+0000 7f559c08b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55941f4250 con 0x7f55940cdc40 2024-03-23T09:42:36.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.808+0000 7f5599e27640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55940b8350 0x7f5594150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.808+0000 7f5599e27640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55940b8350 0x7f5594150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44678/0 (socket says 172.21.15.47:44678) 2024-03-23T09:42:36.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.808+0000 7f5599e27640 1 -- 172.21.15.47:0/3719128090 learned_addr learned my addr 172.21.15.47:0/3719128090 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:36.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.809+0000 7f5599e27640 1 -- 172.21.15.47:0/3719128090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55940cdc40 msgr2=0x7f55940b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.809+0000 7f5599e27640 1 --2- 172.21.15.47:0/3719128090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55940cdc40 0x7f55940b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.809+0000 7f5599e27640 1 -- 172.21.15.47:0/3719128090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f559415ca90 msgr2=0x7f55940b7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.809+0000 7f5599e27640 1 --2- 172.21.15.47:0/3719128090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f559415ca90 0x7f55940b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.809+0000 7f5599e27640 1 -- 172.21.15.47:0/3719128090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5584009b30 con 0x7f55940b8350 2024-03-23T09:42:36.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.809+0000 7f5599e27640 1 --2- 172.21.15.47:0/3719128090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55940b8350 0x7f5594150f40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f558c00c450 tx=0x7f558c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:36.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.809+0000 7f55927fc640 1 -- 172.21.15.47:0/3719128090 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c0052d0 con 0x7f55940b8350 2024-03-23T09:42:36.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.809+0000 7f55927fc640 1 -- 172.21.15.47:0/3719128090 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c005430 con 0x7f55940b8350 2024-03-23T09:42:36.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.809+0000 7f559c08b640 1 -- 172.21.15.47:0/3719128090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55941f3920 con 0x7f55940b8350 2024-03-23T09:42:36.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.809+0000 7f55927fc640 1 -- 172.21.15.47:0/3719128090 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c012870 con 0x7f55940b8350 2024-03-23T09:42:36.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.809+0000 7f559c08b640 1 -- 172.21.15.47:0/3719128090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55941f3ee0 con 0x7f55940b8350 2024-03-23T09:42:36.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.810+0000 7f559c08b640 1 -- 172.21.15.47:0/3719128090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5594072360 con 0x7f55940b8350 2024-03-23T09:42:36.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.810+0000 7f55927fc640 1 -- 172.21.15.47:0/3719128090 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f558c0129d0 con 0x7f55940b8350 2024-03-23T09:42:36.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.811+0000 7f55927fc640 1 --2- 172.21.15.47:0/3719128090 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f558006ac00 0x7f558006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:36.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.811+0000 7f55927fc640 1 -- 172.21.15.47:0/3719128090 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f558c088d20 con 0x7f55940b8350 2024-03-23T09:42:36.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.811+0000 7f5599626640 1 --2- 172.21.15.47:0/3719128090 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f558006ac00 0x7f558006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:36.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.811+0000 7f5599626640 1 --2- 172.21.15.47:0/3719128090 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f558006ac00 0x7f558006d0c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5588006a90 tx=0x7f5588006a20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:36.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.815+0000 7f55927fc640 1 -- 172.21.15.47:0/3719128090 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f558c053ef0 con 0x7f55940b8350 2024-03-23T09:42:36.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.993+0000 7f559c08b640 1 -- 172.21.15.47:0/3719128090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55940ce090 con 0x7f55940b8350 2024-03-23T09:42:36.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.994+0000 7f55927fc640 1 -- 172.21.15.47:0/3719128090 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f55940ce090 con 0x7f55940b8350 2024-03-23T09:42:36.994 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:36.994 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:00.721483+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:41:58.297353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T09:41:58.685375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:42:36.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.996+0000 7f559c08b640 1 -- 172.21.15.47:0/3719128090 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f558006ac00 msgr2=0x7f558006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.996+0000 7f559c08b640 1 --2- 172.21.15.47:0/3719128090 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f558006ac00 0x7f558006d0c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5588006a90 tx=0x7f5588006a20 comp rx=0 tx=0).stop 2024-03-23T09:42:36.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.996+0000 7f559c08b640 1 -- 172.21.15.47:0/3719128090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55940b8350 msgr2=0x7f5594150f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:36.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.996+0000 7f559c08b640 1 --2- 172.21.15.47:0/3719128090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55940b8350 0x7f5594150f40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f558c00c450 tx=0x7f558c00c910 comp rx=0 tx=0).stop 2024-03-23T09:42:36.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.996+0000 7f559c08b640 1 -- 172.21.15.47:0/3719128090 shutdown_connections 2024-03-23T09:42:36.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.996+0000 7f559c08b640 1 --2- 172.21.15.47:0/3719128090 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f558006ac00 0x7f558006d0c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.996+0000 7f559c08b640 1 --2- 172.21.15.47:0/3719128090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f559415ca90 0x7f55940b7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.996+0000 7f559c08b640 1 --2- 172.21.15.47:0/3719128090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55940cdc40 0x7f55940b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.996+0000 7f559c08b640 1 --2- 172.21.15.47:0/3719128090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55940b8350 0x7f5594150f40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:36.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.996+0000 7f559c08b640 1 -- 172.21.15.47:0/3719128090 >> 172.21.15.47:0/3719128090 conn(0x7f55940659d0 msgr2=0x7f55940c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:36.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.996+0000 7f559c08b640 1 -- 172.21.15.47:0/3719128090 shutdown_connections 2024-03-23T09:42:36.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:36.996+0000 7f559c08b640 1 -- 172.21.15.47:0/3719128090 wait complete. 2024-03-23T09:42:37.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:42:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.110+0000 7ff9a999f640 1 Processor -- start 2024-03-23T09:42:37.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.110+0000 7ff9a999f640 1 -- start start 2024-03-23T09:42:37.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.110+0000 7ff9a999f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a4098a00 0x7ff9a4098de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:37.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.110+0000 7ff9a999f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a406bec0 con 0x7ff9a40caca0 2024-03-23T09:42:37.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.110+0000 7ff9a999f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a40cf320 con 0x7ff9a40b15f0 2024-03-23T09:42:37.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.110+0000 7ff9a999f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a415e180 con 0x7ff9a4098a00 2024-03-23T09:42:37.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.111+0000 7ff9a37fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff9a40caca0 0x7ff9a40cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44988/0 (socket says 172.21.15.47:44988) 2024-03-23T09:42:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.111+0000 7ff9a37fe640 1 -- 172.21.15.47:0/2571605074 learned_addr learned my addr 172.21.15.47:0/2571605074 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.111+0000 7ff9a2ffd640 1 --2- 172.21.15.47:0/2571605074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a4098a00 0x7ff9a4098de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.111+0000 7ff9a1ffb640 1 -- 172.21.15.47:0/2571605074 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629043881 0 0) 0x7ff994001490 con 0x7ff9a40caca0 2024-03-23T09:42:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.111+0000 7ff9a1ffb640 1 -- 172.21.15.47:0/2571605074 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff990003520 con 0x7ff9a40caca0 2024-03-23T09:42:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.111+0000 7ff9a1ffb640 1 -- 172.21.15.47:0/2571605074 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948882841 0 0) 0x7ff9a40cf320 con 0x7ff9a40b15f0 2024-03-23T09:42:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.111+0000 7ff9a1ffb640 1 -- 172.21.15.47:0/2571605074 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff994001490 con 0x7ff9a40b15f0 2024-03-23T09:42:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.111+0000 7ff9a1ffb640 1 -- 172.21.15.47:0/2571605074 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1207356202 0 0) 0x7ff990003520 con 0x7ff9a40caca0 2024-03-23T09:42:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.111+0000 7ff9a2ffd640 1 -- 172.21.15.47:0/2571605074 >> v1:172.21.15.175:6789/0 conn(0x7ff9a40b15f0 legacy=0x7ff9a406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.111+0000 7ff9a2ffd640 1 -- 172.21.15.47:0/2571605074 >> v1:172.21.15.47:6789/0 conn(0x7ff9a40caca0 legacy=0x7ff9a40cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.111+0000 7ff9a2ffd640 1 -- 172.21.15.47:0/2571605074 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a4099320 con 0x7ff9a4098a00 2024-03-23T09:42:37.113 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:42:37.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.112+0000 7ff9a2ffd640 1 --2- 172.21.15.47:0/2571605074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a4098a00 0x7ff9a4098de0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0097d0 tx=0x7ff99c02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0914353bf73e287 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:37.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.112+0000 7ff9a1ffb640 1 -- 172.21.15.47:0/2571605074 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994001b80 con 0x7ff9a4098a00 2024-03-23T09:42:37.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.112+0000 7ff9a1ffb640 1 -- 172.21.15.47:0/2571605074 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c02e050 con 0x7ff9a4098a00 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.112+0000 7ff9a1ffb640 1 -- 172.21.15.47:0/2571605074 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c030710 con 0x7ff9a4098a00 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.113+0000 7ff9a999f640 1 -- 172.21.15.47:0/2571605074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a4098a00 msgr2=0x7ff9a4098de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.113+0000 7ff9a999f640 1 --2- 172.21.15.47:0/2571605074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a4098a00 0x7ff9a4098de0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0097d0 tx=0x7ff99c02b800 comp rx=0 tx=0).stop 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.113+0000 7ff9a999f640 1 -- 172.21.15.47:0/2571605074 shutdown_connections 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.113+0000 7ff9a999f640 1 --2- 172.21.15.47:0/2571605074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a4098a00 0x7ff9a4098de0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.113+0000 7ff9a999f640 1 -- 172.21.15.47:0/2571605074 >> 172.21.15.47:0/2571605074 conn(0x7ff9a40b2140 msgr2=0x7ff9a40b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.113+0000 7ff9a999f640 1 -- 172.21.15.47:0/2571605074 shutdown_connections 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.113+0000 7ff9a999f640 1 -- 172.21.15.47:0/2571605074 wait complete. 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.114+0000 7ff9a999f640 1 Processor -- start 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.114+0000 7ff9a999f640 1 -- start start 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a999f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a4098a00 0x7ff9a40c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a999f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a40b15f0 0x7ff9a40b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a999f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a40caca0 0x7ff9a40b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a999f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a41483d0 con 0x7ff9a4098a00 2024-03-23T09:42:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a999f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a4147750 con 0x7ff9a40caca0 2024-03-23T09:42:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a999f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a4147a30 con 0x7ff9a40b15f0 2024-03-23T09:42:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a4098a00 0x7ff9a40c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a4098a00 0x7ff9a40c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44694/0 (socket says 172.21.15.47:44694) 2024-03-23T09:42:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a2ffd640 1 -- 172.21.15.47:0/400492229 learned_addr learned my addr 172.21.15.47:0/400492229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a27fc640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a40b15f0 0x7ff9a40b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a37fe640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a40caca0 0x7ff9a40b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a2ffd640 1 -- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a40b15f0 msgr2=0x7ff9a40b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a2ffd640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a40b15f0 0x7ff9a40b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a2ffd640 1 -- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a40caca0 msgr2=0x7ff9a40b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a2ffd640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a40caca0 0x7ff9a40b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a2ffd640 1 -- 172.21.15.47:0/400492229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c009400 con 0x7ff9a4098a00 2024-03-23T09:42:37.118 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:42:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.115+0000 7ff9a2ffd640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a4098a00 0x7ff9a40c8d00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff99c000c00 tx=0x7ff99c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.116+0000 7ff99bfff640 1 -- 172.21.15.47:0/400492229 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c006e10 con 0x7ff9a4098a00 2024-03-23T09:42:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.116+0000 7ff99bfff640 1 -- 172.21.15.47:0/400492229 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c03c620 con 0x7ff9a4098a00 2024-03-23T09:42:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.116+0000 7ff99bfff640 1 -- 172.21.15.47:0/400492229 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c03c9b0 con 0x7ff9a4098a00 2024-03-23T09:42:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.116+0000 7ff9a999f640 1 -- 172.21.15.47:0/400492229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a4147d10 con 0x7ff9a4098a00 2024-03-23T09:42:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.116+0000 7ff9a999f640 1 -- 172.21.15.47:0/400492229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a40c9240 con 0x7ff9a4098a00 2024-03-23T09:42:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.117+0000 7ff9a999f640 1 -- 172.21.15.47:0/400492229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a4066be0 con 0x7ff9a4098a00 2024-03-23T09:42:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.117+0000 7ff99bfff640 1 -- 172.21.15.47:0/400492229 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff99c039070 con 0x7ff9a4098a00 2024-03-23T09:42:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.117+0000 7ff99bfff640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff98406ac00 0x7ff98406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.118+0000 7ff99bfff640 1 -- 172.21.15.47:0/400492229 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff99c0aaa90 con 0x7ff9a4098a00 2024-03-23T09:42:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.118+0000 7ff9a27fc640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff98406ac00 0x7ff98406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.118+0000 7ff9a27fc640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff98406ac00 0x7ff98406d0c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40676e0 tx=0x7ff98c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.122+0000 7ff99bfff640 1 -- 172.21.15.47:0/400492229 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff99c075c70 con 0x7ff9a4098a00 2024-03-23T09:42:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.343+0000 7ff9a999f640 1 -- 172.21.15.47:0/400492229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff9a40b2fc0 con 0x7ff9a4098a00 2024-03-23T09:42:37.343 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:42:37.343+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:42:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.354+0000 7ff99bfff640 1 -- 172.21.15.47:0/400492229 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v13) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff99c037090 con 0x7ff9a4098a00 2024-03-23T09:42:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:42:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.356+0000 7ff9a999f640 1 -- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff98406ac00 msgr2=0x7ff98406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.356+0000 7ff9a999f640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff98406ac00 0x7ff98406d0c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40676e0 tx=0x7ff98c009040 comp rx=0 tx=0).stop 2024-03-23T09:42:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.356+0000 7ff9a999f640 1 -- 172.21.15.47:0/400492229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a4098a00 msgr2=0x7ff9a40c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.356+0000 7ff9a999f640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a4098a00 0x7ff9a40c8d00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff99c000c00 tx=0x7ff99c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.356+0000 7ff9a999f640 1 -- 172.21.15.47:0/400492229 shutdown_connections 2024-03-23T09:42:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.356+0000 7ff9a999f640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff98406ac00 0x7ff98406d0c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.356+0000 7ff9a999f640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a40caca0 0x7ff9a40b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.356+0000 7ff9a999f640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a40b15f0 0x7ff9a40b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.356+0000 7ff9a999f640 1 --2- 172.21.15.47:0/400492229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a4098a00 0x7ff9a40c8d00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.356+0000 7ff9a999f640 1 -- 172.21.15.47:0/400492229 >> 172.21.15.47:0/400492229 conn(0x7ff9a40b2140 msgr2=0x7ff9a40cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.356+0000 7ff9a999f640 1 -- 172.21.15.47:0/400492229 shutdown_connections 2024-03-23T09:42:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.356+0000 7ff9a999f640 1 -- 172.21.15.47:0/400492229 wait complete. 2024-03-23T09:42:37.357 INFO:tasks.ceph.mds.c.smithi047.stderr: -5781> 2024-03-23T09:42:04.204+0000 7f829e497640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:42:37.367 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T09:42:37.387 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:42:37.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.472+0000 7fa7aaea0640 1 Processor -- start 2024-03-23T09:42:37.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.472+0000 7fa7aaea0640 1 -- start start 2024-03-23T09:42:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.473+0000 7fa7aaea0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7a413d810 0x7fa7a40cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.473+0000 7fa7aaea0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa7a40701f0 0x7fa7a4070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.473+0000 7fa7aaea0640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a413ea70 con 0x7fa7a4140210 2024-03-23T09:42:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.473+0000 7fa7aaea0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a413dbf0 con 0x7fa7a413d810 2024-03-23T09:42:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.473+0000 7fa7aaea0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a40a7670 con 0x7fa7a40701f0 2024-03-23T09:42:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.473+0000 7fa7a8c3c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7a413d810 0x7fa7a40cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.473+0000 7fa7a8c3c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7a413d810 0x7fa7a40cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44724/0 (socket says 172.21.15.47:44724) 2024-03-23T09:42:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.473+0000 7fa7a8c3c640 1 -- 172.21.15.47:0/2111504832 learned_addr learned my addr 172.21.15.47:0/2111504832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.473+0000 7fa7a3fff640 1 --2- 172.21.15.47:0/2111504832 >> v2:172.21.15.175:3300/0 conn(0x7fa7a40701f0 0x7fa7a4070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:37.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.473+0000 7fa7a8c3c640 1 -- 172.21.15.47:0/2111504832 >> v2:172.21.15.175:3300/0 conn(0x7fa7a40701f0 msgr2=0x7fa7a4070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:37.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.473+0000 7fa7a8c3c640 1 --2- 172.21.15.47:0/2111504832 >> v2:172.21.15.175:3300/0 conn(0x7fa7a40701f0 0x7fa7a4070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.473+0000 7fa7a8c3c640 1 -- 172.21.15.47:0/2111504832 >> v1:172.21.15.175:6789/0 conn(0x7fa7a4140210 legacy=0x7fa7a4140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:37.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.473+0000 7fa7a8c3c640 1 -- 172.21.15.47:0/2111504832 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a4070b70 con 0x7fa7a413d810 2024-03-23T09:42:37.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.474+0000 7fa7a8c3c640 1 --2- 172.21.15.47:0/2111504832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a413d810 0x7fa7a40cc4b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa7a40b43a0 tx=0x7fa79402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ade72ecb91df3ba server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:37.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.474+0000 7fa7a37fe640 1 -- 172.21.15.47:0/2111504832 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794030860 con 0x7fa7a413d810 2024-03-23T09:42:37.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.474+0000 7fa7a37fe640 1 -- 172.21.15.47:0/2111504832 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7940309c0 con 0x7fa7a413d810 2024-03-23T09:42:37.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.474+0000 7fa7a37fe640 1 -- 172.21.15.47:0/2111504832 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794030d50 con 0x7fa7a413d810 2024-03-23T09:42:37.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.475+0000 7fa7aaea0640 1 -- 172.21.15.47:0/2111504832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a413d810 msgr2=0x7fa7a40cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:37.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.475+0000 7fa7aaea0640 1 --2- 172.21.15.47:0/2111504832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a413d810 0x7fa7a40cc4b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa7a40b43a0 tx=0x7fa79402b9a0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.475+0000 7fa7aaea0640 1 -- 172.21.15.47:0/2111504832 shutdown_connections 2024-03-23T09:42:37.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.475+0000 7fa7aaea0640 1 --2- 172.21.15.47:0/2111504832 >> v2:172.21.15.175:3300/0 conn(0x7fa7a40701f0 0x7fa7a4070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.475+0000 7fa7aaea0640 1 --2- 172.21.15.47:0/2111504832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a413d810 0x7fa7a40cc4b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.475+0000 7fa7aaea0640 1 -- 172.21.15.47:0/2111504832 >> 172.21.15.47:0/2111504832 conn(0x7fa7a406a310 msgr2=0x7fa7a40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:37.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.475+0000 7fa7aaea0640 1 -- 172.21.15.47:0/2111504832 shutdown_connections 2024-03-23T09:42:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.475+0000 7fa7aaea0640 1 -- 172.21.15.47:0/2111504832 wait complete. 2024-03-23T09:42:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.476+0000 7fa7aaea0640 1 Processor -- start 2024-03-23T09:42:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.476+0000 7fa7aaea0640 1 -- start start 2024-03-23T09:42:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7aaea0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7a40701f0 0x7fa7a4139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7aaea0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a413d810 0x7fa7a413b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7aaea0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7a4140210 0x7fa7a413bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7aaea0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4154b30 con 0x7fa7a413d810 2024-03-23T09:42:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7aaea0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4153a90 con 0x7fa7a4140210 2024-03-23T09:42:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7aaea0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4153d70 con 0x7fa7a40701f0 2024-03-23T09:42:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7a3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a413d810 0x7fa7a413b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7a3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a413d810 0x7fa7a413b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44738/0 (socket says 172.21.15.47:44738) 2024-03-23T09:42:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7a3fff640 1 -- 172.21.15.47:0/738039205 learned_addr learned my addr 172.21.15.47:0/738039205 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7a943d640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7a4140210 0x7fa7a413bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7a8c3c640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7a40701f0 0x7fa7a4139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7a3fff640 1 -- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7a40701f0 msgr2=0x7fa7a4139ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7a3fff640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7a40701f0 0x7fa7a4139ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7a3fff640 1 -- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7a4140210 msgr2=0x7fa7a413bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7a3fff640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7a4140210 0x7fa7a413bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7a3fff640 1 -- 172.21.15.47:0/738039205 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7940094d0 con 0x7fa7a413d810 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7a943d640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7a4140210 0x7fa7a413bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7a3fff640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a413d810 0x7fa7a413b980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa79800d800 tx=0x7fa79800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.477+0000 7fa7a17fa640 1 -- 172.21.15.47:0/738039205 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79800b6c0 con 0x7fa7a413d810 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.478+0000 7fa7a17fa640 1 -- 172.21.15.47:0/738039205 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79800b820 con 0x7fa7a413d810 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.478+0000 7fa7aaea0640 1 -- 172.21.15.47:0/738039205 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a4154050 con 0x7fa7a413d810 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.478+0000 7fa7a17fa640 1 -- 172.21.15.47:0/738039205 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79800bbb0 con 0x7fa7a413d810 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.478+0000 7fa7aaea0640 1 -- 172.21.15.47:0/738039205 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a4154610 con 0x7fa7a413d810 2024-03-23T09:42:37.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.479+0000 7fa7aaea0640 1 -- 172.21.15.47:0/738039205 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a4073480 con 0x7fa7a413d810 2024-03-23T09:42:37.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.479+0000 7fa7a17fa640 1 -- 172.21.15.47:0/738039205 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa79800bd10 con 0x7fa7a413d810 2024-03-23T09:42:37.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.479+0000 7fa7a17fa640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa79006ac00 0x7fa79006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:37.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.480+0000 7fa7a17fa640 1 -- 172.21.15.47:0/738039205 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa798089880 con 0x7fa7a413d810 2024-03-23T09:42:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.480+0000 7fa7a8c3c640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa79006ac00 0x7fa79006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.480+0000 7fa7a8c3c640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa79006ac00 0x7fa79006d0c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa794036000 tx=0x7fa794008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.484+0000 7fa7a17fa640 1 -- 172.21.15.47:0/738039205 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa798054a90 con 0x7fa7a413d810 2024-03-23T09:42:37.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:37.703+0000 7fa7aaea0640 1 -- 172.21.15.47:0/738039205 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa7a41547a0 con 0x7fa7a413d810 2024-03-23T09:42:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.354+0000 7fa7a17fa640 1 -- 172.21.15.47:0/738039205 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa798089020 con 0x7fa7a413d810 2024-03-23T09:42:38.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.357+0000 7fa7aaea0640 1 -- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa79006ac00 msgr2=0x7fa79006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:38.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.357+0000 7fa7aaea0640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa79006ac00 0x7fa79006d0c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa794036000 tx=0x7fa794008030 comp rx=0 tx=0).stop 2024-03-23T09:42:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.357+0000 7fa7aaea0640 1 -- 172.21.15.47:0/738039205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a413d810 msgr2=0x7fa7a413b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.357+0000 7fa7aaea0640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a413d810 0x7fa7a413b980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa79800d800 tx=0x7fa79800dcc0 comp rx=0 tx=0).stop 2024-03-23T09:42:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.357+0000 7fa7aaea0640 1 -- 172.21.15.47:0/738039205 shutdown_connections 2024-03-23T09:42:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.357+0000 7fa7aaea0640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa79006ac00 0x7fa79006d0c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.357+0000 7fa7aaea0640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7a4140210 0x7fa7a413bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.357+0000 7fa7aaea0640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a413d810 0x7fa7a413b980 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.357+0000 7fa7aaea0640 1 --2- 172.21.15.47:0/738039205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7a40701f0 0x7fa7a4139ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.357+0000 7fa7aaea0640 1 -- 172.21.15.47:0/738039205 >> 172.21.15.47:0/738039205 conn(0x7fa7a406a310 msgr2=0x7fa7a40b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.358+0000 7fa7aaea0640 1 -- 172.21.15.47:0/738039205 shutdown_connections 2024-03-23T09:42:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.358+0000 7fa7aaea0640 1 -- 172.21.15.47:0/738039205 wait complete. 2024-03-23T09:42:38.371 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T09:42:38.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.523+0000 7fb1a608b640 1 Processor -- start 2024-03-23T09:42:38.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.523+0000 7fb1a608b640 1 -- start start 2024-03-23T09:42:38.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.524+0000 7fb1a608b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a00af910 0x7fb1a00c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:38.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.524+0000 7fb1a608b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a0070260 con 0x7fb1a015ab90 2024-03-23T09:42:38.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.524+0000 7fb1a608b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a0070b00 con 0x7fb1a0070f90 2024-03-23T09:42:38.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.524+0000 7fb1a608b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a00b0320 con 0x7fb1a00af910 2024-03-23T09:42:38.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.524+0000 7fb19f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb1a015ab90 0x7fb1a00cae90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44998/0 (socket says 172.21.15.47:44998) 2024-03-23T09:42:38.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.524+0000 7fb19f7fe640 1 -- 172.21.15.47:0/1549458999 learned_addr learned my addr 172.21.15.47:0/1549458999 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:38.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.524+0000 7fb19ffff640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a00af910 0x7fb1a00c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:38.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.524+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828716388 0 0) 0x7fb1a0070260 con 0x7fb1a015ab90 2024-03-23T09:42:38.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.524+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb188003520 con 0x7fb1a015ab90 2024-03-23T09:42:38.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808805088 0 0) 0x7fb1a0070b00 con 0x7fb1a0070f90 2024-03-23T09:42:38.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1a0070260 con 0x7fb1a0070f90 2024-03-23T09:42:38.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2110920731 0 0) 0x7fb188003520 con 0x7fb1a015ab90 2024-03-23T09:42:38.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a00af910 msgr2=0x7fb1a00c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:38.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a00af910 0x7fb1a00c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:38.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 >> v1:172.21.15.175:6789/0 conn(0x7fb1a0070f90 legacy=0x7fb1a00c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a015f3c0 con 0x7fb1a015ab90 2024-03-23T09:42:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fb190001670 con 0x7fb1a015ab90 2024-03-23T09:42:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 >> v1:172.21.15.47:6789/0 conn(0x7fb1a015ab90 legacy=0x7fb1a00cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb188003b30 0x7fb188003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1880047c0 0x7fb188004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb188005180 0x7fb18800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18800bf70 con 0x7fb1880047c0 2024-03-23T09:42:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18800c0b0 con 0x7fb188005180 2024-03-23T09:42:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18800c270 con 0x7fb188003b30 2024-03-23T09:42:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19ffff640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a00af910 0x7fb1a00c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:42:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19effd640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1880047c0 0x7fb188004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19f7fe640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb188003b30 0x7fb188003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19ffff640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb188005180 0x7fb18800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19effd640 1 -- 172.21.15.47:0/1549458999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb188003b30 msgr2=0x7fb188003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19effd640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb188003b30 0x7fb188003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19effd640 1 -- 172.21.15.47:0/1549458999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb188005180 msgr2=0x7fb18800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19effd640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb188005180 0x7fb18800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19effd640 1 -- 172.21.15.47:0/1549458999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb188003520 con 0x7fb1880047c0 2024-03-23T09:42:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.525+0000 7fb19ffff640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb188005180 0x7fb18800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.526+0000 7fb19effd640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1880047c0 0x7fb188004c40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb1940018c0 tx=0x7fb1880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb9b6894c17e0cf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.526+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1940052d0 con 0x7fb1880047c0 2024-03-23T09:42:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.526+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb194005450 con 0x7fb1880047c0 2024-03-23T09:42:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.526+0000 7fb19e7fc640 1 -- 172.21.15.47:0/1549458999 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194003830 con 0x7fb1880047c0 2024-03-23T09:42:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.527+0000 7fb1a608b640 1 -- 172.21.15.47:0/1549458999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1880047c0 msgr2=0x7fb188004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.527+0000 7fb1a608b640 1 --2- 172.21.15.47:0/1549458999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1880047c0 0x7fb188004c40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb1940018c0 tx=0x7fb1880057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.527+0000 7fb19ffff640 1 -- 172.21.15.47:0/1549458999 reap_dead start 2024-03-23T09:42:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.527+0000 7fb1a608b640 1 -- 172.21.15.47:0/1549458999 shutdown_connections 2024-03-23T09:42:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.527+0000 7fb1a608b640 1 -- 172.21.15.47:0/1549458999 >> 172.21.15.47:0/1549458999 conn(0x7fb1a0069ce0 msgr2=0x7fb1a006a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.527+0000 7fb1a608b640 1 -- 172.21.15.47:0/1549458999 shutdown_connections 2024-03-23T09:42:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.527+0000 7fb1a608b640 1 -- 172.21.15.47:0/1549458999 wait complete. 2024-03-23T09:42:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.528+0000 7fb1a608b640 1 Processor -- start 2024-03-23T09:42:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.528+0000 7fb1a608b640 1 -- start start 2024-03-23T09:42:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb1a608b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1880047c0 0x7fb1a01efc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb1a608b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a0158ff0 0x7fb1a01f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb1a608b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a01510c0 0x7fb1a0151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb1a608b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a01f4740 con 0x7fb1a0158ff0 2024-03-23T09:42:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb1a608b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0151a80 con 0x7fb1880047c0 2024-03-23T09:42:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb1a608b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a014c7e0 con 0x7fb1a01510c0 2024-03-23T09:42:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb19effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a0158ff0 0x7fb1a01f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb19effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a0158ff0 0x7fb1a01f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44754/0 (socket says 172.21.15.47:44754) 2024-03-23T09:42:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb19effd640 1 -- 172.21.15.47:0/787999299 learned_addr learned my addr 172.21.15.47:0/787999299 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb19ffff640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a01510c0 0x7fb1a0151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb19f7fe640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1880047c0 0x7fb1a01efc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb19effd640 1 -- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a01510c0 msgr2=0x7fb1a0151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb19effd640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a01510c0 0x7fb1a0151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb19effd640 1 -- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1880047c0 msgr2=0x7fb1a01efc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb19effd640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1880047c0 0x7fb1a01efc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb19effd640 1 -- 172.21.15.47:0/787999299 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a01f3ea0 con 0x7fb1a0158ff0 2024-03-23T09:42:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb19ffff640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a01510c0 0x7fb1a0151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb19f7fe640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1880047c0 0x7fb1a01efc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.529+0000 7fb19effd640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a0158ff0 0x7fb1a01f37a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb194005040 tx=0x7fb1940012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.530+0000 7fb187fff640 1 -- 172.21.15.47:0/787999299 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1940069e0 con 0x7fb1a0158ff0 2024-03-23T09:42:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.530+0000 7fb187fff640 1 -- 172.21.15.47:0/787999299 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb194006b40 con 0x7fb1a0158ff0 2024-03-23T09:42:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.530+0000 7fb1a608b640 1 -- 172.21.15.47:0/787999299 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a01f4180 con 0x7fb1a0158ff0 2024-03-23T09:42:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.530+0000 7fb187fff640 1 -- 172.21.15.47:0/787999299 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1940035f0 con 0x7fb1a0158ff0 2024-03-23T09:42:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.530+0000 7fb1a608b640 1 -- 172.21.15.47:0/787999299 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a015b110 con 0x7fb1a0158ff0 2024-03-23T09:42:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.531+0000 7fb1a608b640 1 -- 172.21.15.47:0/787999299 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a015af70 con 0x7fb1a0158ff0 2024-03-23T09:42:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.531+0000 7fb187fff640 1 -- 172.21.15.47:0/787999299 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb194003750 con 0x7fb1a0158ff0 2024-03-23T09:42:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.532+0000 7fb187fff640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb18c06ac30 0x7fb18c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.532+0000 7fb187fff640 1 -- 172.21.15.47:0/787999299 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb1940aac60 con 0x7fb1a0158ff0 2024-03-23T09:42:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.532+0000 7fb19f7fe640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb18c06ac30 0x7fb18c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.532+0000 7fb19f7fe640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb18c06ac30 0x7fb18c06d0f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb190001eb0 tx=0x7fb190009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.536+0000 7fb187fff640 1 -- 172.21.15.47:0/787999299 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb1a015af70 con 0x7fb1a0158ff0 2024-03-23T09:42:38.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:38.714+0000 7fb1a608b640 1 -- 172.21.15.47:0/787999299 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb1a00cb580 con 0x7fb1a0158ff0 2024-03-23T09:42:39.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.359+0000 7fb187fff640 1 -- 172.21.15.47:0/787999299 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb1a00cb580 con 0x7fb1a0158ff0 2024-03-23T09:42:39.359 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-03-23T09:42:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.363+0000 7fb1a608b640 1 -- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb18c06ac30 msgr2=0x7fb18c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.363+0000 7fb1a608b640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb18c06ac30 0x7fb18c06d0f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb190001eb0 tx=0x7fb190009040 comp rx=0 tx=0).stop 2024-03-23T09:42:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.363+0000 7fb1a608b640 1 -- 172.21.15.47:0/787999299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a0158ff0 msgr2=0x7fb1a01f37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.363+0000 7fb1a608b640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a0158ff0 0x7fb1a01f37a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb194005040 tx=0x7fb1940012d0 comp rx=0 tx=0).stop 2024-03-23T09:42:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.363+0000 7fb1a608b640 1 -- 172.21.15.47:0/787999299 shutdown_connections 2024-03-23T09:42:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.363+0000 7fb1a608b640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb18c06ac30 0x7fb18c06d0f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.363+0000 7fb1a608b640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a01510c0 0x7fb1a0151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.363+0000 7fb1a608b640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a0158ff0 0x7fb1a01f37a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.363+0000 7fb1a608b640 1 --2- 172.21.15.47:0/787999299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1880047c0 0x7fb1a01efc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.363+0000 7fb1a608b640 1 -- 172.21.15.47:0/787999299 >> 172.21.15.47:0/787999299 conn(0x7fb1a0069ce0 msgr2=0x7fb1a00cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.363+0000 7fb1a608b640 1 -- 172.21.15.47:0/787999299 shutdown_connections 2024-03-23T09:42:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.363+0000 7fb1a608b640 1 -- 172.21.15.47:0/787999299 wait complete. 2024-03-23T09:42:39.377 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T09:42:39.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.488+0000 7fac2b1dc640 1 Processor -- start 2024-03-23T09:42:39.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.488+0000 7fac2b1dc640 1 -- start start 2024-03-23T09:42:39.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac2b1dc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac2415a700 0x7fac240cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:39.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac2b1dc640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fac240d2f00 0x7fac2415d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:39.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac2b1dc640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2415dbd0 con 0x7fac2415d0c0 2024-03-23T09:42:39.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac2b1dc640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2415aae0 con 0x7fac2415a700 2024-03-23T09:42:39.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac2b1dc640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2415deb0 con 0x7fac240d2f00 2024-03-23T09:42:39.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac28f78640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac2415a700 0x7fac240cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:39.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac28f78640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac2415a700 0x7fac240cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44768/0 (socket says 172.21.15.47:44768) 2024-03-23T09:42:39.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac28f78640 1 -- 172.21.15.47:0/847292430 learned_addr learned my addr 172.21.15.47:0/847292430 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:39.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac29779640 1 --2- 172.21.15.47:0/847292430 >> v2:172.21.15.175:3300/0 conn(0x7fac240d2f00 0x7fac2415d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:39.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac28f78640 1 -- 172.21.15.47:0/847292430 >> v2:172.21.15.175:3300/0 conn(0x7fac240d2f00 msgr2=0x7fac2415d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:39.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac28f78640 1 --2- 172.21.15.47:0/847292430 >> v2:172.21.15.175:3300/0 conn(0x7fac240d2f00 0x7fac2415d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:39.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac28f78640 1 -- 172.21.15.47:0/847292430 >> v1:172.21.15.175:6789/0 conn(0x7fac2415d0c0 legacy=0x7fac240d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:39.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac28f78640 1 -- 172.21.15.47:0/847292430 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac240cf340 con 0x7fac2415a700 2024-03-23T09:42:39.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.489+0000 7fac29779640 1 --2- 172.21.15.47:0/847292430 >> v2:172.21.15.175:3300/0 conn(0x7fac240d2f00 0x7fac2415d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:39.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.490+0000 7fac28f78640 1 --2- 172.21.15.47:0/847292430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac2415a700 0x7fac240cfb40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fac2406bf60 tx=0x7fac1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdf9d1ec40d098e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:39.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.490+0000 7fac237fe640 1 -- 172.21.15.47:0/847292430 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c031870 con 0x7fac2415a700 2024-03-23T09:42:39.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.490+0000 7fac237fe640 1 -- 172.21.15.47:0/847292430 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c0319d0 con 0x7fac2415a700 2024-03-23T09:42:39.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.490+0000 7fac237fe640 1 -- 172.21.15.47:0/847292430 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c031d60 con 0x7fac2415a700 2024-03-23T09:42:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.491+0000 7fac2b1dc640 1 -- 172.21.15.47:0/847292430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac2415a700 msgr2=0x7fac240cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.491+0000 7fac2b1dc640 1 --2- 172.21.15.47:0/847292430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac2415a700 0x7fac240cfb40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fac2406bf60 tx=0x7fac1c02b910 comp rx=0 tx=0).stop 2024-03-23T09:42:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.491+0000 7fac2b1dc640 1 -- 172.21.15.47:0/847292430 shutdown_connections 2024-03-23T09:42:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.491+0000 7fac2b1dc640 1 --2- 172.21.15.47:0/847292430 >> v2:172.21.15.175:3300/0 conn(0x7fac240d2f00 0x7fac2415d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.491+0000 7fac2b1dc640 1 --2- 172.21.15.47:0/847292430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac2415a700 0x7fac240cfb40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.491+0000 7fac2b1dc640 1 -- 172.21.15.47:0/847292430 >> 172.21.15.47:0/847292430 conn(0x7fac2406afd0 msgr2=0x7fac2406b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.491+0000 7fac2b1dc640 1 -- 172.21.15.47:0/847292430 shutdown_connections 2024-03-23T09:42:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.491+0000 7fac2b1dc640 1 -- 172.21.15.47:0/847292430 wait complete. 2024-03-23T09:42:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.492+0000 7fac2b1dc640 1 Processor -- start 2024-03-23T09:42:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.492+0000 7fac2b1dc640 1 -- start start 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac2b1dc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240d2f00 0x7fac24154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac2b1dc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac2415a700 0x7fac240b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac2b1dc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac2415d0c0 0x7fac240b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac2b1dc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac24157810 con 0x7fac240d2f00 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac2b1dc640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2415a290 con 0x7fac2415d0c0 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac2b1dc640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac24159fb0 con 0x7fac2415a700 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac28f78640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240d2f00 0x7fac24154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac28f78640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240d2f00 0x7fac24154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44774/0 (socket says 172.21.15.47:44774) 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac28f78640 1 -- 172.21.15.47:0/1735075131 learned_addr learned my addr 172.21.15.47:0/1735075131 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac28f78640 1 -- 172.21.15.47:0/1735075131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac2415a700 msgr2=0x7fac240b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac28f78640 1 --2- 172.21.15.47:0/1735075131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac2415a700 0x7fac240b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac28f78640 1 -- 172.21.15.47:0/1735075131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac2415d0c0 msgr2=0x7fac240b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac28f78640 1 --2- 172.21.15.47:0/1735075131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac2415d0c0 0x7fac240b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.493+0000 7fac28f78640 1 -- 172.21.15.47:0/1735075131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c0094d0 con 0x7fac240d2f00 2024-03-23T09:42:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.494+0000 7fac28f78640 1 --2- 172.21.15.47:0/1735075131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240d2f00 0x7fac24154e00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000c00 tx=0x7fac1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.494+0000 7fac217fa640 1 -- 172.21.15.47:0/1735075131 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c031840 con 0x7fac240d2f00 2024-03-23T09:42:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.494+0000 7fac217fa640 1 -- 172.21.15.47:0/1735075131 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c0443f0 con 0x7fac240d2f00 2024-03-23T09:42:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.494+0000 7fac2b1dc640 1 -- 172.21.15.47:0/1735075131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac241574a0 con 0x7fac240d2f00 2024-03-23T09:42:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.494+0000 7fac217fa640 1 -- 172.21.15.47:0/1735075131 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0447a0 con 0x7fac240d2f00 2024-03-23T09:42:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.494+0000 7fac2b1dc640 1 -- 172.21.15.47:0/1735075131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac240b5f20 con 0x7fac240d2f00 2024-03-23T09:42:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.495+0000 7fac2b1dc640 1 -- 172.21.15.47:0/1735075131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac24072360 con 0x7fac240d2f00 2024-03-23T09:42:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.495+0000 7fac217fa640 1 -- 172.21.15.47:0/1735075131 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fac1c04d450 con 0x7fac240d2f00 2024-03-23T09:42:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.496+0000 7fac217fa640 1 --2- 172.21.15.47:0/1735075131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fac0406ac30 0x7fac0406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.496+0000 7fac217fa640 1 -- 172.21.15.47:0/1735075131 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fac1c02d040 con 0x7fac240d2f00 2024-03-23T09:42:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.496+0000 7fac23fff640 1 --2- 172.21.15.47:0/1735075131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fac0406ac30 0x7fac0406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.496+0000 7fac23fff640 1 --2- 172.21.15.47:0/1735075131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fac0406ac30 0x7fac0406d0f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fac24159370 tx=0x7fac14009230 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:39.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.500+0000 7fac217fa640 1 -- 172.21.15.47:0/1735075131 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fac1c076d20 con 0x7fac240d2f00 2024-03-23T09:42:39.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:39.678+0000 7fac2b1dc640 1 -- 172.21.15.47:0/1735075131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fac240b6170 con 0x7fac240d2f00 2024-03-23T09:42:40.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.362+0000 7fac217fa640 1 -- 172.21.15.47:0/1735075131 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7fac1c0350d0 con 0x7fac240d2f00 2024-03-23T09:42:40.362 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-03-23T09:42:40.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.365+0000 7fac2b1dc640 1 -- 172.21.15.47:0/1735075131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fac0406ac30 msgr2=0x7fac0406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:40.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.365+0000 7fac2b1dc640 1 --2- 172.21.15.47:0/1735075131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fac0406ac30 0x7fac0406d0f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fac24159370 tx=0x7fac14009230 comp rx=0 tx=0).stop 2024-03-23T09:42:40.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.365+0000 7fac2b1dc640 1 -- 172.21.15.47:0/1735075131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240d2f00 msgr2=0x7fac24154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:40.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.365+0000 7fac2b1dc640 1 --2- 172.21.15.47:0/1735075131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240d2f00 0x7fac24154e00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000c00 tx=0x7fac1c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.366+0000 7fac2b1dc640 1 -- 172.21.15.47:0/1735075131 shutdown_connections 2024-03-23T09:42:40.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.366+0000 7fac2b1dc640 1 --2- 172.21.15.47:0/1735075131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fac0406ac30 0x7fac0406d0f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.366+0000 7fac2b1dc640 1 --2- 172.21.15.47:0/1735075131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac2415d0c0 0x7fac240b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.366+0000 7fac2b1dc640 1 --2- 172.21.15.47:0/1735075131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac2415a700 0x7fac240b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.366+0000 7fac2b1dc640 1 --2- 172.21.15.47:0/1735075131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240d2f00 0x7fac24154e00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.366+0000 7fac2b1dc640 1 -- 172.21.15.47:0/1735075131 >> 172.21.15.47:0/1735075131 conn(0x7fac2406afd0 msgr2=0x7fac2406b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:40.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.366+0000 7fac2b1dc640 1 -- 172.21.15.47:0/1735075131 shutdown_connections 2024-03-23T09:42:40.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.366+0000 7fac2b1dc640 1 -- 172.21.15.47:0/1735075131 wait complete. 2024-03-23T09:42:40.379 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-23T09:42:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.487+0000 7f49268e0640 1 Processor -- start 2024-03-23T09:42:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.488+0000 7f49268e0640 1 -- start start 2024-03-23T09:42:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.488+0000 7f49268e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f492006b690 0x7f492006ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.488+0000 7f49268e0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49200b1c60 0x7f49200ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.488+0000 7f49268e0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4920161450 0x7f4920098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.488+0000 7f49268e0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49200cffc0 con 0x7f49200b1c60 2024-03-23T09:42:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.488+0000 7f49268e0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492006bfb0 con 0x7f4920161450 2024-03-23T09:42:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.488+0000 7f49268e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49200cf3b0 con 0x7f492006b690 2024-03-23T09:42:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.488+0000 7f491f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49200b1c60 0x7f49200ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.489+0000 7f491f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49200b1c60 0x7f49200ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44790/0 (socket says 172.21.15.47:44790) 2024-03-23T09:42:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.489+0000 7f491f7fe640 1 -- 172.21.15.47:0/706256937 learned_addr learned my addr 172.21.15.47:0/706256937 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.489+0000 7f491f7fe640 1 -- 172.21.15.47:0/706256937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f492006b690 msgr2=0x7f492006ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.489+0000 7f491f7fe640 1 --2- 172.21.15.47:0/706256937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f492006b690 0x7f492006ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.489+0000 7f491f7fe640 1 -- 172.21.15.47:0/706256937 >> v2:172.21.15.175:3300/0 conn(0x7f4920161450 msgr2=0x7f4920098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.489+0000 7f4924e7d640 1 --2- 172.21.15.47:0/706256937 >> v2:172.21.15.175:3300/0 conn(0x7f4920161450 0x7f4920098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.489+0000 7f491f7fe640 1 --2- 172.21.15.47:0/706256937 >> v2:172.21.15.175:3300/0 conn(0x7f4920161450 0x7f4920098a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.489+0000 7f491f7fe640 1 -- 172.21.15.47:0/706256937 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49200cf690 con 0x7f49200b1c60 2024-03-23T09:42:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.490+0000 7f491f7fe640 1 --2- 172.21.15.47:0/706256937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200b1c60 0x7f49200ce9b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4920160750 tx=0x7f490802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d15ce62f0ed72e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.490+0000 7f491effd640 1 -- 172.21.15.47:0/706256937 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908030860 con 0x7f49200b1c60 2024-03-23T09:42:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.490+0000 7f491effd640 1 -- 172.21.15.47:0/706256937 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49080309c0 con 0x7f49200b1c60 2024-03-23T09:42:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.490+0000 7f491effd640 1 -- 172.21.15.47:0/706256937 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908030d50 con 0x7f49200b1c60 2024-03-23T09:42:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.490+0000 7f49268e0640 1 -- 172.21.15.47:0/706256937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200b1c60 msgr2=0x7f49200ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.490+0000 7f49268e0640 1 --2- 172.21.15.47:0/706256937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200b1c60 0x7f49200ce9b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4920160750 tx=0x7f490802b9a0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.490+0000 7f49268e0640 1 -- 172.21.15.47:0/706256937 shutdown_connections 2024-03-23T09:42:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.490+0000 7f49268e0640 1 --2- 172.21.15.47:0/706256937 >> v2:172.21.15.175:3300/0 conn(0x7f4920161450 0x7f4920098a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.490+0000 7f49268e0640 1 --2- 172.21.15.47:0/706256937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200b1c60 0x7f49200ce9b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.490+0000 7f49268e0640 1 --2- 172.21.15.47:0/706256937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f492006b690 0x7f492006ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.490+0000 7f49268e0640 1 -- 172.21.15.47:0/706256937 >> 172.21.15.47:0/706256937 conn(0x7f49200b21d0 msgr2=0x7f49200b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.490+0000 7f49268e0640 1 -- 172.21.15.47:0/706256937 shutdown_connections 2024-03-23T09:42:40.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.490+0000 7f49268e0640 1 -- 172.21.15.47:0/706256937 wait complete. 2024-03-23T09:42:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.491+0000 7f49268e0640 1 Processor -- start 2024-03-23T09:42:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.492+0000 7f49268e0640 1 -- start start 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.492+0000 7f49268e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f492006b690 0x7f492014e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.492+0000 7f49268e0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4920161450 0x7f492014dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.492+0000 7f49268e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49200ad2d0 0x7f49200b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.492+0000 7f49268e0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49200ae3e0 con 0x7f4920161450 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.492+0000 7f49268e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49200ad7d0 con 0x7f492006b690 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.492+0000 7f49268e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49200adab0 con 0x7f49200ad2d0 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4920161450 0x7f492014dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4920161450 0x7f492014dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44802/0 (socket says 172.21.15.47:44802) 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491f7fe640 1 -- 172.21.15.47:0/2507286637 learned_addr learned my addr 172.21.15.47:0/2507286637 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f4924e7d640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49200ad2d0 0x7f49200b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491ffff640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f492006b690 0x7f492014e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491f7fe640 1 -- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49200ad2d0 msgr2=0x7f49200b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491f7fe640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49200ad2d0 0x7f49200b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491f7fe640 1 -- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f492006b690 msgr2=0x7f492014e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491f7fe640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f492006b690 0x7f492014e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491f7fe640 1 -- 172.21.15.47:0/2507286637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49080094d0 con 0x7f4920161450 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f4924e7d640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49200ad2d0 0x7f49200b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491ffff640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f492006b690 0x7f492014e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491f7fe640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4920161450 0x7f492014dc50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4908000c00 tx=0x7f4908004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491cff9640 1 -- 172.21.15.47:0/2507286637 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908030830 con 0x7f4920161450 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491cff9640 1 -- 172.21.15.47:0/2507286637 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4908043400 con 0x7f4920161450 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f49268e0640 1 -- 172.21.15.47:0/2507286637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49200add90 con 0x7f4920161450 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f491cff9640 1 -- 172.21.15.47:0/2507286637 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49080437b0 con 0x7f4920161450 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.493+0000 7f49268e0640 1 -- 172.21.15.47:0/2507286637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49200b1120 con 0x7f4920161450 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.494+0000 7f49268e0640 1 -- 172.21.15.47:0/2507286637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49200ae550 con 0x7f4920161450 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.495+0000 7f491cff9640 1 -- 172.21.15.47:0/2507286637 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4908043910 con 0x7f4920161450 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.495+0000 7f491cff9640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f490006ac00 0x7f490006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.495+0000 7f491cff9640 1 -- 172.21.15.47:0/2507286637 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f490803e070 con 0x7f4920161450 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.495+0000 7f491ffff640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f490006ac00 0x7f490006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.496+0000 7f491ffff640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f490006ac00 0x7f490006d0c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f491000a870 tx=0x7f4910008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.499+0000 7f491cff9640 1 -- 172.21.15.47:0/2507286637 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f49080778a0 con 0x7f4920161450 2024-03-23T09:42:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.678+0000 7f49268e0640 1 -- 172.21.15.47:0/2507286637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f49200672f0 con 0x7f4920161450 2024-03-23T09:42:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.682+0000 7f491cff9640 1 -- 172.21.15.47:0/2507286637 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f490807aec0 con 0x7f4920161450 2024-03-23T09:42:40.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.684+0000 7f49268e0640 1 -- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f490006ac00 msgr2=0x7f490006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:40.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.684+0000 7f49268e0640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f490006ac00 0x7f490006d0c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f491000a870 tx=0x7f4910008040 comp rx=0 tx=0).stop 2024-03-23T09:42:40.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.684+0000 7f49268e0640 1 -- 172.21.15.47:0/2507286637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4920161450 msgr2=0x7f492014dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:40.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.684+0000 7f49268e0640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4920161450 0x7f492014dc50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4908000c00 tx=0x7f4908004730 comp rx=0 tx=0).stop 2024-03-23T09:42:40.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.684+0000 7f49268e0640 1 -- 172.21.15.47:0/2507286637 shutdown_connections 2024-03-23T09:42:40.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.684+0000 7f49268e0640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f490006ac00 0x7f490006d0c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.684+0000 7f49268e0640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49200ad2d0 0x7f49200b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.684+0000 7f49268e0640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4920161450 0x7f492014dc50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.684+0000 7f49268e0640 1 --2- 172.21.15.47:0/2507286637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f492006b690 0x7f492014e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.684+0000 7f49268e0640 1 -- 172.21.15.47:0/2507286637 >> 172.21.15.47:0/2507286637 conn(0x7f49200b21d0 msgr2=0x7f49200cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:40.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.684+0000 7f49268e0640 1 -- 172.21.15.47:0/2507286637 shutdown_connections 2024-03-23T09:42:40.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.684+0000 7f49268e0640 1 -- 172.21.15.47:0/2507286637 wait complete. 2024-03-23T09:42:40.695 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-03-23T09:42:40.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.796+0000 7fc5121fd640 1 Processor -- start 2024-03-23T09:42:40.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.796+0000 7fc5121fd640 1 -- start start 2024-03-23T09:42:40.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.797+0000 7fc5121fd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c15ca90 0x7fc50c160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.797+0000 7fc5121fd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc50c0cde10 0x7fc50c0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.797+0000 7fc5121fd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50c0d11e0 con 0x7fc50c0b8350 2024-03-23T09:42:40.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.797+0000 7fc5121fd640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50c160f60 con 0x7fc50c0cde10 2024-03-23T09:42:40.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.797+0000 7fc5121fd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c161240 con 0x7fc50c15ca90 2024-03-23T09:42:40.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.797+0000 7fc50affd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc50c0b8350 0x7fc50c0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45014/0 (socket says 172.21.15.47:45014) 2024-03-23T09:42:40.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.797+0000 7fc50affd640 1 -- 172.21.15.47:0/4124106641 learned_addr learned my addr 172.21.15.47:0/4124106641 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.797+0000 7fc50b7fe640 1 --2- 172.21.15.47:0/4124106641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c15ca90 0x7fc50c160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.797+0000 7fc50bfff640 1 --2- 172.21.15.47:0/4124106641 >> v2:172.21.15.175:3300/0 conn(0x7fc50c0cde10 0x7fc50c0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.797+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807004454 0 0) 0x7fc50c0d11e0 con 0x7fc50c0b8350 2024-03-23T09:42:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.797+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4f4003520 con 0x7fc50c0b8350 2024-03-23T09:42:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.797+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2735571555 0 0) 0x7fc4f4003520 con 0x7fc50c0b8350 2024-03-23T09:42:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c15ca90 msgr2=0x7fc50c160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50a7fc640 1 --2- 172.21.15.47:0/4124106641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c15ca90 0x7fc50c160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 >> v2:172.21.15.175:3300/0 conn(0x7fc50c0cde10 msgr2=0x7fc50c0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50a7fc640 1 --2- 172.21.15.47:0/4124106641 >> v2:172.21.15.175:3300/0 conn(0x7fc50c0cde10 0x7fc50c0d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c161520 con 0x7fc50c0b8350 2024-03-23T09:42:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fc500001670 con 0x7fc50c0b8350 2024-03-23T09:42:40.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 >> v1:172.21.15.47:6789/0 conn(0x7fc50c0b8350 legacy=0x7fc50c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:40.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50a7fc640 1 --2- 172.21.15.47:0/4124106641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f4003970 0x7fc4f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50a7fc640 1 --2- 172.21.15.47:0/4124106641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f40047c0 0x7fc4f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50a7fc640 1 --2- 172.21.15.47:0/4124106641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c15ca90 0x7fc4f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c160f60 con 0x7fc4f4003970 2024-03-23T09:42:40.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c161240 con 0x7fc4f40047c0 2024-03-23T09:42:40.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f400b980 con 0x7fc50c15ca90 2024-03-23T09:42:40.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50b7fe640 1 --2- 172.21.15.47:0/4124106641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f4003970 0x7fc4f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50affd640 1 --2- 172.21.15.47:0/4124106641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f40047c0 0x7fc4f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50bfff640 1 --2- 172.21.15.47:0/4124106641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c15ca90 0x7fc4f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50b7fe640 1 -- 172.21.15.47:0/4124106641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c15ca90 msgr2=0x7fc4f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50b7fe640 1 --2- 172.21.15.47:0/4124106641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c15ca90 0x7fc4f4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50b7fe640 1 -- 172.21.15.47:0/4124106641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f40047c0 msgr2=0x7fc4f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50b7fe640 1 --2- 172.21.15.47:0/4124106641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f40047c0 0x7fc4f4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50b7fe640 1 -- 172.21.15.47:0/4124106641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f4003520 con 0x7fc4f4003970 2024-03-23T09:42:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.798+0000 7fc50affd640 1 --2- 172.21.15.47:0/4124106641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f40047c0 0x7fc4f4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.799+0000 7fc50b7fe640 1 --2- 172.21.15.47:0/4124106641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f4003970 0x7fc4f4003d90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009af0 tx=0x7fc4f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be62f5e72c735be6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.799+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc02c6b0 con 0x7fc4f4003970 2024-03-23T09:42:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.799+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc02c810 con 0x7fc4f4003970 2024-03-23T09:42:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.799+0000 7fc50a7fc640 1 -- 172.21.15.47:0/4124106641 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc02cba0 con 0x7fc4f4003970 2024-03-23T09:42:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.799+0000 7fc5121fd640 1 -- 172.21.15.47:0/4124106641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f4003970 msgr2=0x7fc4f4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.799+0000 7fc5121fd640 1 --2- 172.21.15.47:0/4124106641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f4003970 0x7fc4f4003d90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009af0 tx=0x7fc4f40057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.799+0000 7fc50bfff640 1 -- 172.21.15.47:0/4124106641 reap_dead start 2024-03-23T09:42:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.799+0000 7fc5121fd640 1 -- 172.21.15.47:0/4124106641 shutdown_connections 2024-03-23T09:42:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.799+0000 7fc5121fd640 1 -- 172.21.15.47:0/4124106641 >> 172.21.15.47:0/4124106641 conn(0x7fc50c0659d0 msgr2=0x7fc50c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.799+0000 7fc5121fd640 1 -- 172.21.15.47:0/4124106641 shutdown_connections 2024-03-23T09:42:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.800+0000 7fc5121fd640 1 -- 172.21.15.47:0/4124106641 wait complete. 2024-03-23T09:42:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.801+0000 7fc5121fd640 1 Processor -- start 2024-03-23T09:42:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.801+0000 7fc5121fd640 1 -- start start 2024-03-23T09:42:40.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.801+0000 7fc5121fd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f4003970 0x7fc50c1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.801+0000 7fc5121fd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c0cc510 0x7fc50c15f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.801+0000 7fc5121fd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c1f1be0 0x7fc50c15d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.801+0000 7fc5121fd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c152c40 con 0x7fc4f4003970 2024-03-23T09:42:40.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.801+0000 7fc5121fd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c1520a0 con 0x7fc50c0cc510 2024-03-23T09:42:40.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.801+0000 7fc5121fd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c152310 con 0x7fc50c1f1be0 2024-03-23T09:42:40.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc50b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f4003970 0x7fc50c1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc50b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f4003970 0x7fc50c1530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44816/0 (socket says 172.21.15.47:44816) 2024-03-23T09:42:40.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc50b7fe640 1 -- 172.21.15.47:0/2088782753 learned_addr learned my addr 172.21.15.47:0/2088782753 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:40.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc50affd640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c0cc510 0x7fc50c15f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc50bfff640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c1f1be0 0x7fc50c15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc50b7fe640 1 -- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c1f1be0 msgr2=0x7fc50c15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:40.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc50b7fe640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c1f1be0 0x7fc50c15d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc50b7fe640 1 -- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c0cc510 msgr2=0x7fc50c15f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc50b7fe640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c0cc510 0x7fc50c15f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc50b7fe640 1 -- 172.21.15.47:0/2088782753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c1525f0 con 0x7fc4f4003970 2024-03-23T09:42:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc50affd640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c0cc510 0x7fc50c15f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc50b7fe640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f4003970 0x7fc50c1530d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009af0 tx=0x7fc4fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc4f3fff640 1 -- 172.21.15.47:0/2088782753 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc003810 con 0x7fc4f4003970 2024-03-23T09:42:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc4f3fff640 1 -- 172.21.15.47:0/2088782753 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc003970 con 0x7fc4f4003970 2024-03-23T09:42:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc5121fd640 1 -- 172.21.15.47:0/2088782753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50c1528d0 con 0x7fc4f4003970 2024-03-23T09:42:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc4f3fff640 1 -- 172.21.15.47:0/2088782753 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc003ce0 con 0x7fc4f4003970 2024-03-23T09:42:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.802+0000 7fc5121fd640 1 -- 172.21.15.47:0/2088782753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50c1f2060 con 0x7fc4f4003970 2024-03-23T09:42:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.803+0000 7fc5121fd640 1 -- 172.21.15.47:0/2088782753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c072360 con 0x7fc4f4003970 2024-03-23T09:42:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.804+0000 7fc4f3fff640 1 -- 172.21.15.47:0/2088782753 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc4fc034440 con 0x7fc4f4003970 2024-03-23T09:42:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.804+0000 7fc4f3fff640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4f806ac00 0x7fc4f806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.804+0000 7fc4f3fff640 1 -- 172.21.15.47:0/2088782753 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc4fc0aaf20 con 0x7fc4f4003970 2024-03-23T09:42:40.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.804+0000 7fc50affd640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4f806ac00 0x7fc4f806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:40.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.805+0000 7fc50affd640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4f806ac00 0x7fc4f806d0c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0cbfd0 tx=0x7fc500009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:40.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:40.809+0000 7fc4f3fff640 1 -- 172.21.15.47:0/2088782753 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc4fc076660 con 0x7fc4f4003970 2024-03-23T09:42:41.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.059+0000 7fc5121fd640 1 -- 172.21.15.47:0/2088782753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fc50c0cc8f0 con 0x7fc4f4003970 2024-03-23T09:42:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.364+0000 7fc4f3fff640 1 -- 172.21.15.47:0/2088782753 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v99) v1 ==== 142+0+0 (secure 0 0 0) 0x7fc4fc0044e0 con 0x7fc4f4003970 2024-03-23T09:42:41.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.366+0000 7fc5121fd640 1 -- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4f806ac00 msgr2=0x7fc4f806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:41.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.366+0000 7fc5121fd640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4f806ac00 0x7fc4f806d0c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0cbfd0 tx=0x7fc500009040 comp rx=0 tx=0).stop 2024-03-23T09:42:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.366+0000 7fc5121fd640 1 -- 172.21.15.47:0/2088782753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f4003970 msgr2=0x7fc50c1530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.366+0000 7fc5121fd640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f4003970 0x7fc50c1530d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009af0 tx=0x7fc4fc004730 comp rx=0 tx=0).stop 2024-03-23T09:42:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.366+0000 7fc5121fd640 1 -- 172.21.15.47:0/2088782753 shutdown_connections 2024-03-23T09:42:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.366+0000 7fc5121fd640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4f806ac00 0x7fc4f806d0c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.366+0000 7fc5121fd640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c1f1be0 0x7fc50c15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.366+0000 7fc5121fd640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c0cc510 0x7fc50c15f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.366+0000 7fc5121fd640 1 --2- 172.21.15.47:0/2088782753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f4003970 0x7fc50c1530d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.366+0000 7fc5121fd640 1 -- 172.21.15.47:0/2088782753 >> 172.21.15.47:0/2088782753 conn(0x7fc50c0659d0 msgr2=0x7fc50c156fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.366+0000 7fc5121fd640 1 -- 172.21.15.47:0/2088782753 shutdown_connections 2024-03-23T09:42:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.366+0000 7fc5121fd640 1 -- 172.21.15.47:0/2088782753 wait complete. 2024-03-23T09:42:41.377 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-23T09:42:41.377 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-03-23T09:42:41.377 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-23T09:42:41.378 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-03-23T09:42:41.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.479+0000 7f9a247a4640 1 Processor -- start 2024-03-23T09:42:41.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.479+0000 7f9a247a4640 1 -- start start 2024-03-23T09:42:41.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a247a4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9a1c15d0a0 0x7f9a1c0d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:41.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a247a4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a1c06b210 0x7f9a1c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:41.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a247a4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1c0ce090 con 0x7f9a1c15ccc0 2024-03-23T09:42:41.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a247a4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1c06c0e0 con 0x7f9a1c06b210 2024-03-23T09:42:41.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a247a4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1c06bc00 con 0x7f9a1c15d0a0 2024-03-23T09:42:41.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a22d41640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a1c06b210 0x7f9a1c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a22540640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9a1c15ccc0 0x7f9a1c0cd980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45020/0 (socket says 172.21.15.47:45020) 2024-03-23T09:42:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a22540640 1 -- 172.21.15.47:0/4155302848 learned_addr learned my addr 172.21.15.47:0/4155302848 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a21d3f640 1 --2- 172.21.15.47:0/4155302848 >> v2:172.21.15.175:3300/0 conn(0x7f9a1c15d0a0 0x7f9a1c0d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a22d41640 1 -- 172.21.15.47:0/4155302848 >> v2:172.21.15.175:3300/0 conn(0x7f9a1c15d0a0 msgr2=0x7f9a1c0d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a22d41640 1 --2- 172.21.15.47:0/4155302848 >> v2:172.21.15.175:3300/0 conn(0x7f9a1c15d0a0 0x7f9a1c0d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a22d41640 1 -- 172.21.15.47:0/4155302848 >> v1:172.21.15.47:6789/0 conn(0x7f9a1c15ccc0 legacy=0x7f9a1c0cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a2153e640 1 -- 172.21.15.47:0/4155302848 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898515434 0 0) 0x7f9a14001110 con 0x7f9a1c15ccc0 2024-03-23T09:42:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.480+0000 7f9a22d41640 1 -- 172.21.15.47:0/4155302848 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c0d0f00 con 0x7f9a1c06b210 2024-03-23T09:42:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.481+0000 7f9a22d41640 1 --2- 172.21.15.47:0/4155302848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c06b210 0x7f9a1c06b6c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c08b020 tx=0x7f9a1802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73c1558364e44d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.481+0000 7f9a2153e640 1 -- 172.21.15.47:0/4155302848 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1802cd00 con 0x7f9a1c06b210 2024-03-23T09:42:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.481+0000 7f9a2153e640 1 -- 172.21.15.47:0/4155302848 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1802ce60 con 0x7f9a1c06b210 2024-03-23T09:42:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.481+0000 7f9a2153e640 1 -- 172.21.15.47:0/4155302848 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18034660 con 0x7f9a1c06b210 2024-03-23T09:42:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.482+0000 7f9a247a4640 1 -- 172.21.15.47:0/4155302848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c06b210 msgr2=0x7f9a1c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.482+0000 7f9a247a4640 1 --2- 172.21.15.47:0/4155302848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c06b210 0x7f9a1c06b6c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c08b020 tx=0x7f9a1802bf50 comp rx=0 tx=0).stop 2024-03-23T09:42:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.482+0000 7f9a247a4640 1 -- 172.21.15.47:0/4155302848 shutdown_connections 2024-03-23T09:42:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.482+0000 7f9a247a4640 1 --2- 172.21.15.47:0/4155302848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c06b210 0x7f9a1c06b6c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.482+0000 7f9a247a4640 1 --2- 172.21.15.47:0/4155302848 >> v2:172.21.15.175:3300/0 conn(0x7f9a1c15d0a0 0x7f9a1c0d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.482+0000 7f9a247a4640 1 -- 172.21.15.47:0/4155302848 >> 172.21.15.47:0/4155302848 conn(0x7f9a1c0cf430 msgr2=0x7f9a1c0cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.482+0000 7f9a247a4640 1 -- 172.21.15.47:0/4155302848 shutdown_connections 2024-03-23T09:42:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.482+0000 7f9a247a4640 1 -- 172.21.15.47:0/4155302848 wait complete. 2024-03-23T09:42:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.483+0000 7f9a247a4640 1 Processor -- start 2024-03-23T09:42:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.483+0000 7f9a247a4640 1 -- start start 2024-03-23T09:42:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a247a4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c06b210 0x7f9a1c154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a247a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a1c15ccc0 0x7f9a1c0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a247a4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c15d0a0 0x7f9a1c15e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a247a4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c15ee70 con 0x7f9a1c06b210 2024-03-23T09:42:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a247a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c15f150 con 0x7f9a1c15ccc0 2024-03-23T09:42:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a247a4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c15f430 con 0x7f9a1c15d0a0 2024-03-23T09:42:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a22540640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c06b210 0x7f9a1c154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a22540640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c06b210 0x7f9a1c154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44830/0 (socket says 172.21.15.47:44830) 2024-03-23T09:42:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a22540640 1 -- 172.21.15.47:0/410800801 learned_addr learned my addr 172.21.15.47:0/410800801 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a21d3f640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a1c15ccc0 0x7f9a1c0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a22d41640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c15d0a0 0x7f9a1c15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a22540640 1 -- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c15d0a0 msgr2=0x7f9a1c15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a22540640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c15d0a0 0x7f9a1c15e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a22540640 1 -- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a1c15ccc0 msgr2=0x7f9a1c0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a22540640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a1c15ccc0 0x7f9a1c0c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a22540640 1 -- 172.21.15.47:0/410800801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a18009a70 con 0x7f9a1c06b210 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a21d3f640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a1c15ccc0 0x7f9a1c0c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a22540640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c06b210 0x7f9a1c154cf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400c780 tx=0x7f9a1400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a12ffd640 1 -- 172.21.15.47:0/410800801 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a140072d0 con 0x7f9a1c06b210 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a12ffd640 1 -- 172.21.15.47:0/410800801 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a14007450 con 0x7f9a1c06b210 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a12ffd640 1 -- 172.21.15.47:0/410800801 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a140047e0 con 0x7f9a1c06b210 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.484+0000 7f9a247a4640 1 -- 172.21.15.47:0/410800801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c15f710 con 0x7f9a1c06b210 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.485+0000 7f9a247a4640 1 -- 172.21.15.47:0/410800801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c153bc0 con 0x7f9a1c06b210 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.486+0000 7f9a247a4640 1 -- 172.21.15.47:0/410800801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c072360 con 0x7f9a1c06b210 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.486+0000 7f9a12ffd640 1 -- 172.21.15.47:0/410800801 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9a14004960 con 0x7f9a1c06b210 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.486+0000 7f9a12ffd640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9a0006ac00 0x7f9a0006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.486+0000 7f9a12ffd640 1 -- 172.21.15.47:0/410800801 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9a14088120 con 0x7f9a1c06b210 2024-03-23T09:42:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.486+0000 7f9a21d3f640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9a0006ac00 0x7f9a0006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:41.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.487+0000 7f9a21d3f640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9a0006ac00 0x7f9a0006d0c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c006fd0 tx=0x7f9a0c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:41.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.491+0000 7f9a12ffd640 1 -- 172.21.15.47:0/410800801 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9a14053690 con 0x7f9a1c06b210 2024-03-23T09:42:41.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:41.742+0000 7f9a247a4640 1 -- 172.21.15.47:0/410800801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f9a1c0ce090 con 0x7f9a1c06b210 2024-03-23T09:42:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.369+0000 7f9a12ffd640 1 -- 172.21.15.47:0/410800801 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v100) v1 ==== 152+0+0 (secure 0 0 0) 0x7f9a1c0ce090 con 0x7f9a1c06b210 2024-03-23T09:42:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.371+0000 7f9a247a4640 1 -- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9a0006ac00 msgr2=0x7f9a0006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.371+0000 7f9a247a4640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9a0006ac00 0x7f9a0006d0c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c006fd0 tx=0x7f9a0c008040 comp rx=0 tx=0).stop 2024-03-23T09:42:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.371+0000 7f9a247a4640 1 -- 172.21.15.47:0/410800801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c06b210 msgr2=0x7f9a1c154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.371+0000 7f9a247a4640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c06b210 0x7f9a1c154cf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400c780 tx=0x7f9a1400cc40 comp rx=0 tx=0).stop 2024-03-23T09:42:42.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.371+0000 7f9a247a4640 1 -- 172.21.15.47:0/410800801 shutdown_connections 2024-03-23T09:42:42.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.371+0000 7f9a247a4640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9a0006ac00 0x7f9a0006d0c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.371+0000 7f9a247a4640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c15d0a0 0x7f9a1c15e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.371+0000 7f9a247a4640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a1c15ccc0 0x7f9a1c0c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.371+0000 7f9a247a4640 1 --2- 172.21.15.47:0/410800801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c06b210 0x7f9a1c154cf0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.371+0000 7f9a247a4640 1 -- 172.21.15.47:0/410800801 >> 172.21.15.47:0/410800801 conn(0x7f9a1c0cf430 msgr2=0x7f9a1c0cf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:42.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.371+0000 7f9a247a4640 1 -- 172.21.15.47:0/410800801 shutdown_connections 2024-03-23T09:42:42.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.371+0000 7f9a247a4640 1 -- 172.21.15.47:0/410800801 wait complete. 2024-03-23T09:42:42.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-23T09:42:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.490+0000 7fb616b1b640 1 Processor -- start 2024-03-23T09:42:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.490+0000 7fb616b1b640 1 -- start start 2024-03-23T09:42:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb616b1b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb61015ca30 0x7fb610160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb616b1b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb6100cde40 0x7fb6100d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb616b1b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb610161600 con 0x7fb61015ce10 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb616b1b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb610161040 con 0x7fb6100cde40 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb616b1b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb610161320 con 0x7fb61015ca30 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb6150b8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb6100cde40 0x7fb6100d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb60ffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb61015ce10 0x7fb6100cd730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45022/0 (socket says 172.21.15.47:45022) 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb6150b8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb6100cde40 0x7fb6100d15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44844/0 (socket says 172.21.15.47:44844) 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb60ffff640 1 -- 172.21.15.47:0/4004745356 learned_addr learned my addr 172.21.15.47:0/4004745356 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb6148b7640 1 --2- 172.21.15.47:0/4004745356 >> v2:172.21.15.175:3300/0 conn(0x7fb61015ca30 0x7fb610160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb6150b8640 1 -- 172.21.15.47:0/4004745356 >> v2:172.21.15.175:3300/0 conn(0x7fb61015ca30 msgr2=0x7fb610160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb6150b8640 1 --2- 172.21.15.47:0/4004745356 >> v2:172.21.15.175:3300/0 conn(0x7fb61015ca30 0x7fb610160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb6150b8640 1 -- 172.21.15.47:0/4004745356 >> v1:172.21.15.47:6789/0 conn(0x7fb61015ce10 legacy=0x7fb6100cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb60f7fe640 1 -- 172.21.15.47:0/4004745356 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762933204 0 0) 0x7fb610161600 con 0x7fb61015ce10 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.491+0000 7fb6150b8640 1 -- 172.21.15.47:0/4004745356 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6100d0dd0 con 0x7fb6100cde40 2024-03-23T09:42:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.492+0000 7fb6150b8640 1 --2- 172.21.15.47:0/4004745356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6100cde40 0x7fb6100d15d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb608009d90 tx=0x7fb60802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad16033e4c1cec9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.492+0000 7fb60f7fe640 1 -- 172.21.15.47:0/4004745356 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60802cbd0 con 0x7fb6100cde40 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.492+0000 7fb60f7fe640 1 -- 172.21.15.47:0/4004745356 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60802cd30 con 0x7fb6100cde40 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.492+0000 7fb60f7fe640 1 -- 172.21.15.47:0/4004745356 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60802cea0 con 0x7fb6100cde40 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.493+0000 7fb616b1b640 1 -- 172.21.15.47:0/4004745356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6100cde40 msgr2=0x7fb6100d15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.493+0000 7fb616b1b640 1 --2- 172.21.15.47:0/4004745356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6100cde40 0x7fb6100d15d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb608009d90 tx=0x7fb60802be20 comp rx=0 tx=0).stop 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.493+0000 7fb616b1b640 1 -- 172.21.15.47:0/4004745356 shutdown_connections 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.493+0000 7fb616b1b640 1 --2- 172.21.15.47:0/4004745356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6100cde40 0x7fb6100d15d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.493+0000 7fb616b1b640 1 --2- 172.21.15.47:0/4004745356 >> v2:172.21.15.175:3300/0 conn(0x7fb61015ca30 0x7fb610160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.493+0000 7fb616b1b640 1 -- 172.21.15.47:0/4004745356 >> 172.21.15.47:0/4004745356 conn(0x7fb61006b160 msgr2=0x7fb61006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.493+0000 7fb616b1b640 1 -- 172.21.15.47:0/4004745356 shutdown_connections 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.493+0000 7fb616b1b640 1 -- 172.21.15.47:0/4004745356 wait complete. 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.494+0000 7fb616b1b640 1 Processor -- start 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.494+0000 7fb616b1b640 1 -- start start 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb616b1b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6100cde40 0x7fb61015f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb616b1b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb61015ca30 0x7fb6100cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb616b1b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb61015ce10 0x7fb6101f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb616b1b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61015aaf0 con 0x7fb6100cde40 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb616b1b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61015add0 con 0x7fb61015ca30 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb616b1b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61015b0b0 con 0x7fb61015ce10 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb6148b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6100cde40 0x7fb61015f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb6148b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6100cde40 0x7fb61015f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44850/0 (socket says 172.21.15.47:44850) 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb6148b7640 1 -- 172.21.15.47:0/1589579826 learned_addr learned my addr 172.21.15.47:0/1589579826 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb60ffff640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb61015ca30 0x7fb6100cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb6150b8640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb61015ce10 0x7fb6101f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb6148b7640 1 -- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb61015ce10 msgr2=0x7fb6101f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb6148b7640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb61015ce10 0x7fb6101f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb6148b7640 1 -- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb61015ca30 msgr2=0x7fb6100cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb6148b7640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb61015ca30 0x7fb6100cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb6148b7640 1 -- 172.21.15.47:0/1589579826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb608009a70 con 0x7fb6100cde40 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb6150b8640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb61015ce10 0x7fb6101f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb6148b7640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6100cde40 0x7fb61015f870 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb60400d800 tx=0x7fb60400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb60d7fa640 1 -- 172.21.15.47:0/1589579826 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60400b6c0 con 0x7fb6100cde40 2024-03-23T09:42:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb60d7fa640 1 -- 172.21.15.47:0/1589579826 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60400b820 con 0x7fb6100cde40 2024-03-23T09:42:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb616b1b640 1 -- 172.21.15.47:0/1589579826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61015b390 con 0x7fb6100cde40 2024-03-23T09:42:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.495+0000 7fb60d7fa640 1 -- 172.21.15.47:0/1589579826 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60400bbb0 con 0x7fb6100cde40 2024-03-23T09:42:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.496+0000 7fb616b1b640 1 -- 172.21.15.47:0/1589579826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61015fdb0 con 0x7fb6100cde40 2024-03-23T09:42:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.496+0000 7fb616b1b640 1 -- 172.21.15.47:0/1589579826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb610072360 con 0x7fb6100cde40 2024-03-23T09:42:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.497+0000 7fb60d7fa640 1 -- 172.21.15.47:0/1589579826 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb60400bd10 con 0x7fb6100cde40 2024-03-23T09:42:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.497+0000 7fb60d7fa640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5f406ac00 0x7fb5f406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.497+0000 7fb60d7fa640 1 -- 172.21.15.47:0/1589579826 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb6040896a0 con 0x7fb6100cde40 2024-03-23T09:42:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.497+0000 7fb60ffff640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5f406ac00 0x7fb5f406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.498+0000 7fb60ffff640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5f406ac00 0x7fb5f406d0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb610151f60 tx=0x7fb5fc007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.501+0000 7fb60d7fa640 1 -- 172.21.15.47:0/1589579826 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb604054ae0 con 0x7fb6100cde40 2024-03-23T09:42:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.682+0000 7fb616b1b640 1 -- 172.21.15.47:0/1589579826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb610066be0 con 0x7fb6100cde40 2024-03-23T09:42:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.686+0000 7fb60d7fa640 1 -- 172.21.15.47:0/1589579826 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb604058100 con 0x7fb6100cde40 2024-03-23T09:42:42.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.688+0000 7fb616b1b640 1 -- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5f406ac00 msgr2=0x7fb5f406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:42.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.688+0000 7fb616b1b640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5f406ac00 0x7fb5f406d0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb610151f60 tx=0x7fb5fc007c00 comp rx=0 tx=0).stop 2024-03-23T09:42:42.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.688+0000 7fb616b1b640 1 -- 172.21.15.47:0/1589579826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6100cde40 msgr2=0x7fb61015f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:42.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.688+0000 7fb616b1b640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6100cde40 0x7fb61015f870 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb60400d800 tx=0x7fb60400dcc0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.688+0000 7fb616b1b640 1 -- 172.21.15.47:0/1589579826 shutdown_connections 2024-03-23T09:42:42.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.688+0000 7fb616b1b640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5f406ac00 0x7fb5f406d0c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.688+0000 7fb616b1b640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb61015ce10 0x7fb6101f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.688+0000 7fb616b1b640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb61015ca30 0x7fb6100cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.688+0000 7fb616b1b640 1 --2- 172.21.15.47:0/1589579826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6100cde40 0x7fb61015f870 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.688+0000 7fb616b1b640 1 -- 172.21.15.47:0/1589579826 >> 172.21.15.47:0/1589579826 conn(0x7fb61006b160 msgr2=0x7fb6100d28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:42.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.688+0000 7fb616b1b640 1 -- 172.21.15.47:0/1589579826 shutdown_connections 2024-03-23T09:42:42.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.688+0000 7fb616b1b640 1 -- 172.21.15.47:0/1589579826 wait complete. 2024-03-23T09:42:42.698 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:42:42.699 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:42:42.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:42:42.780 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:42:42.780 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:42:42.780 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:42:42.795 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:42:42.795 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:42:42.795 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:42:42.837 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:42:42.837 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:42:42.837 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:42:42.851 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:42:42.852 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:42:42.852 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:42:42.895 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:42:42.895 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-23T09:42:42.895 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:42:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.995+0000 7fe3b5484640 1 Processor -- start 2024-03-23T09:42:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.995+0000 7fe3b5484640 1 -- start start 2024-03-23T09:42:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.996+0000 7fe3b5484640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe3b0143010 0x7fe3b00b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.996+0000 7fe3b5484640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3b00b40e0 0x7fe3b006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.996+0000 7fe3b5484640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b00ccd00 con 0x7fe3b0142a90 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.996+0000 7fe3b5484640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b00b27c0 con 0x7fe3b0143010 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.996+0000 7fe3b5484640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b0142e70 con 0x7fe3b00b40e0 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.996+0000 7fe3aeffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe3b0142a90 0x7fe3b00cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45038/0 (socket says 172.21.15.47:45038) 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.996+0000 7fe3aeffd640 1 -- 172.21.15.47:0/1050939004 learned_addr learned my addr 172.21.15.47:0/1050939004 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.996+0000 7fe3af7fe640 1 --2- 172.21.15.47:0/1050939004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3b00b40e0 0x7fe3b006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.996+0000 7fe3ae7fc640 1 --2- 172.21.15.47:0/1050939004 >> v2:172.21.15.175:3300/0 conn(0x7fe3b0143010 0x7fe3b00b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.996+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633565591 0 0) 0x7fe3b00ccd00 con 0x7fe3b0142a90 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.996+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39c005730 con 0x7fe3b0142a90 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2554232387 0 0) 0x7fe39c005730 con 0x7fe3b0142a90 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3b00b40e0 msgr2=0x7fe3b006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 --2- 172.21.15.47:0/1050939004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3b00b40e0 0x7fe3b006c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 >> v2:172.21.15.175:3300/0 conn(0x7fe3b0143010 msgr2=0x7fe3b00b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 --2- 172.21.15.47:0/1050939004 >> v2:172.21.15.175:3300/0 conn(0x7fe3b0143010 0x7fe3b00b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b00ccfe0 con 0x7fe3b0142a90 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fe3a8001670 con 0x7fe3b0142a90 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 >> v1:172.21.15.47:6789/0 conn(0x7fe3b0142a90 legacy=0x7fe3b00cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 --2- 172.21.15.47:0/1050939004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3b00b40e0 0x7fe39c005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 --2- 172.21.15.47:0/1050939004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c006130 0x7fe39c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 --2- 172.21.15.47:0/1050939004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe39c006f10 0x7fe39c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00b27c0 con 0x7fe39c006f10 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b0142e70 con 0x7fe39c006130 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c00db90 con 0x7fe3b00b40e0 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3af7fe640 1 --2- 172.21.15.47:0/1050939004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe39c006f10 0x7fe39c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3aeffd640 1 --2- 172.21.15.47:0/1050939004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3b00b40e0 0x7fe39c005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3ae7fc640 1 --2- 172.21.15.47:0/1050939004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c006130 0x7fe39c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3af7fe640 1 -- 172.21.15.47:0/1050939004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3b00b40e0 msgr2=0x7fe39c005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3af7fe640 1 --2- 172.21.15.47:0/1050939004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3b00b40e0 0x7fe39c005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3af7fe640 1 -- 172.21.15.47:0/1050939004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c006130 msgr2=0x7fe39c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3af7fe640 1 --2- 172.21.15.47:0/1050939004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c006130 0x7fe39c0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3af7fe640 1 -- 172.21.15.47:0/1050939004 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c005730 con 0x7fe39c006f10 2024-03-23T09:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.997+0000 7fe3ae7fc640 1 --2- 172.21.15.47:0/1050939004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c006130 0x7fe39c0069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.998+0000 7fe3af7fe640 1 --2- 172.21.15.47:0/1050939004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe39c006f10 0x7fe39c007390 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009fc0 tx=0x7fe39c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb9b432b2e491f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.998+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a002cc20 con 0x7fe39c006f10 2024-03-23T09:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.998+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a000f040 con 0x7fe39c006f10 2024-03-23T09:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.998+0000 7fe3adffb640 1 -- 172.21.15.47:0/1050939004 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0034440 con 0x7fe39c006f10 2024-03-23T09:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.999+0000 7fe3b5484640 1 -- 172.21.15.47:0/1050939004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe39c006f10 msgr2=0x7fe39c007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.999+0000 7fe3b5484640 1 --2- 172.21.15.47:0/1050939004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe39c006f10 0x7fe39c007390 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009fc0 tx=0x7fe39c007a00 comp rx=0 tx=0).stop 2024-03-23T09:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.999+0000 7fe3af7fe640 1 -- 172.21.15.47:0/1050939004 reap_dead start 2024-03-23T09:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.999+0000 7fe3b5484640 1 -- 172.21.15.47:0/1050939004 shutdown_connections 2024-03-23T09:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.999+0000 7fe3b5484640 1 -- 172.21.15.47:0/1050939004 >> 172.21.15.47:0/1050939004 conn(0x7fe3b006a310 msgr2=0x7fe3b00b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.999+0000 7fe3b5484640 1 -- 172.21.15.47:0/1050939004 shutdown_connections 2024-03-23T09:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:42.999+0000 7fe3b5484640 1 -- 172.21.15.47:0/1050939004 wait complete. 2024-03-23T09:42:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.000+0000 7fe3b5484640 1 Processor -- start 2024-03-23T09:42:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.000+0000 7fe3b5484640 1 -- start start 2024-03-23T09:42:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3b5484640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c006f10 0x7fe3b00ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3b5484640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3b00af300 0x7fe3b00af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3b5484640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b0138640 0x7fe3b0138ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3b5484640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b0132af0 con 0x7fe3b0138640 2024-03-23T09:42:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3b5484640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b0139000 con 0x7fe39c006f10 2024-03-23T09:42:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3b5484640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00afdc0 con 0x7fe3b00af300 2024-03-23T09:42:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3af7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b0138640 0x7fe3b0138ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3af7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b0138640 0x7fe3b0138ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44866/0 (socket says 172.21.15.47:44866) 2024-03-23T09:42:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3af7fe640 1 -- 172.21.15.47:0/2461602107 learned_addr learned my addr 172.21.15.47:0/2461602107 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3aeffd640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c006f10 0x7fe3b00ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3ae7fc640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3b00af300 0x7fe3b00af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3af7fe640 1 -- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3b00af300 msgr2=0x7fe3b00af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3af7fe640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3b00af300 0x7fe3b00af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3af7fe640 1 -- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c006f10 msgr2=0x7fe3b00ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3af7fe640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c006f10 0x7fe3b00ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3af7fe640 1 -- 172.21.15.47:0/2461602107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b00b00a0 con 0x7fe3b0138640 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.001+0000 7fe3af7fe640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b0138640 0x7fe3b0138ac0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0037040 tx=0x7fe3a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.002+0000 7fe3a77fe640 1 -- 172.21.15.47:0/2461602107 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0043070 con 0x7fe3b0138640 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.002+0000 7fe3a77fe640 1 -- 172.21.15.47:0/2461602107 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a0034ad0 con 0x7fe3b0138640 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.002+0000 7fe3b5484640 1 -- 172.21.15.47:0/2461602107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b0132d00 con 0x7fe3b0138640 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.002+0000 7fe3a77fe640 1 -- 172.21.15.47:0/2461602107 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0034e40 con 0x7fe3b0138640 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.002+0000 7fe3b5484640 1 -- 172.21.15.47:0/2461602107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b00cad70 con 0x7fe3b0138640 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.003+0000 7fe3b5484640 1 -- 172.21.15.47:0/2461602107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b0073480 con 0x7fe3b0138640 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.003+0000 7fe3a77fe640 1 -- 172.21.15.47:0/2461602107 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe3a003e440 con 0x7fe3b0138640 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.003+0000 7fe3a77fe640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe39006ac00 0x7fe39006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.003+0000 7fe3a77fe640 1 -- 172.21.15.47:0/2461602107 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe3a00aa800 con 0x7fe3b0138640 2024-03-23T09:42:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.003+0000 7fe3aeffd640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe39006ac00 0x7fe39006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.004+0000 7fe3aeffd640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe39006ac00 0x7fe39006d0c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8007c60 tx=0x7fe3a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:43.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.008+0000 7fe3a77fe640 1 -- 172.21.15.47:0/2461602107 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe3a0075e90 con 0x7fe3b0138640 2024-03-23T09:42:43.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.222+0000 7fe3b5484640 1 -- 172.21.15.47:0/2461602107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3b0133220 con 0x7fe3b0138640 2024-03-23T09:42:43.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.222+0000 7fe3a77fe640 1 -- 172.21.15.47:0/2461602107 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fe3b0133220 con 0x7fe3b0138640 2024-03-23T09:42:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:43.223 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":14,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3373309448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3373309448},{"type":"v1","addr":"172.21.15.175:6836","nonce":3373309448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4118066020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4118066020},{"type":"v1","addr":"172.21.15.175:6837","nonce":4118066020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2916649516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2916649516},{"type":"v1","addr":"172.21.15.47:6835","nonce":2916649516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/3007562584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3007562584},{"type":"v1","addr":"172.21.15.47:6834","nonce":3007562584}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[]} 2024-03-23T09:42:43.223 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 14 2024-03-23T09:42:43.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.224+0000 7fe3b5484640 1 -- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe39006ac00 msgr2=0x7fe39006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:43.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.224+0000 7fe3b5484640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe39006ac00 0x7fe39006d0c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8007c60 tx=0x7fe3a8009040 comp rx=0 tx=0).stop 2024-03-23T09:42:43.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.224+0000 7fe3b5484640 1 -- 172.21.15.47:0/2461602107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b0138640 msgr2=0x7fe3b0138ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:43.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.224+0000 7fe3b5484640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b0138640 0x7fe3b0138ac0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0037040 tx=0x7fe3a0009750 comp rx=0 tx=0).stop 2024-03-23T09:42:43.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.224+0000 7fe3b5484640 1 -- 172.21.15.47:0/2461602107 shutdown_connections 2024-03-23T09:42:43.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.224+0000 7fe3b5484640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe39006ac00 0x7fe39006d0c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.224+0000 7fe3b5484640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b0138640 0x7fe3b0138ac0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.224+0000 7fe3b5484640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3b00af300 0x7fe3b00af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.224+0000 7fe3b5484640 1 --2- 172.21.15.47:0/2461602107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c006f10 0x7fe3b00ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.224+0000 7fe3b5484640 1 -- 172.21.15.47:0/2461602107 >> 172.21.15.47:0/2461602107 conn(0x7fe3b006a310 msgr2=0x7fe3b00b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:43.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.224+0000 7fe3b5484640 1 -- 172.21.15.47:0/2461602107 shutdown_connections 2024-03-23T09:42:43.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.224+0000 7fe3b5484640 1 -- 172.21.15.47:0/2461602107 wait complete. 2024-03-23T09:42:43.235 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-23T09:42:43.235 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-23T09:42:43.235 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-23T09:42:43.236 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:42:43.236 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-23T09:42:43.236 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-23T09:42:43.236 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-23T09:42:43.236 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:42:43.236 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-23T09:42:43.236 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-23T09:42:43.237 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-23T09:42:43.237 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:42:43.237 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-23T09:42:43.237 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-23T09:42:43.237 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-23T09:42:43.237 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:42:43.245 INFO:tasks.ceph.mds.a:Stopped 2024-03-23T09:42:43.245 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-23T09:42:43.246 INFO:tasks.ceph.mds.c:Stopped 2024-03-23T09:42:43.246 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-23T09:42:43.252 INFO:tasks.ceph.mds.b:Stopped 2024-03-23T09:42:43.252 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-23T09:42:43.253 INFO:tasks.ceph.mds.d:Stopped 2024-03-23T09:42:43.253 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-23T09:42:43.288 INFO:tasks.ceph.mds.a:Started 2024-03-23T09:42:43.288 DEBUG:teuthology.parallel:result is None 2024-03-23T09:42:43.288 INFO:tasks.ceph.mds.c:Started 2024-03-23T09:42:43.289 DEBUG:teuthology.parallel:result is None 2024-03-23T09:42:43.295 INFO:tasks.ceph.mds.b:Started 2024-03-23T09:42:43.296 DEBUG:teuthology.parallel:result is None 2024-03-23T09:42:43.296 INFO:tasks.ceph.mds.d:Started 2024-03-23T09:42:43.296 DEBUG:teuthology.parallel:result is None 2024-03-23T09:42:43.296 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-23T09:42:43.363 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-03-23T09:42:43.363 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:42:43.383 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-03-23T09:42:43.384 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-03-23T09:42:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.403+0000 7f447568f640 1 Processor -- start 2024-03-23T09:42:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.403+0000 7f447568f640 1 -- start start 2024-03-23T09:42:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.404+0000 7f447568f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44701571e0 0x7f447006f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.404+0000 7f447568f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f447006faf0 0x7f447015d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.404+0000 7f447568f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44701575c0 con 0x7f447015d600 2024-03-23T09:42:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.404+0000 7f447568f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447006f190 con 0x7f44701571e0 2024-03-23T09:42:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.404+0000 7f447568f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447015da10 con 0x7f447006faf0 2024-03-23T09:42:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.404+0000 7f446effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44701571e0 0x7f447006f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.404+0000 7f446effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44701571e0 0x7f447006f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44890/0 (socket says 172.21.15.47:44890) 2024-03-23T09:42:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.404+0000 7f446effd640 1 -- 172.21.15.47:0/4264314159 learned_addr learned my addr 172.21.15.47:0/4264314159 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.404+0000 7f446e7fc640 1 --2- 172.21.15.47:0/4264314159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f447006faf0 0x7f447015d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.404+0000 7f446effd640 1 -- 172.21.15.47:0/4264314159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f447006faf0 msgr2=0x7f447015d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.404+0000 7f446effd640 1 --2- 172.21.15.47:0/4264314159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f447006faf0 0x7f447015d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.405+0000 7f446effd640 1 -- 172.21.15.47:0/4264314159 >> v1:172.21.15.175:6789/0 conn(0x7f447015d600 legacy=0x7f44700c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.405+0000 7f446effd640 1 -- 172.21.15.47:0/4264314159 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447015dbf0 con 0x7f44701571e0 2024-03-23T09:42:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.405+0000 7f446effd640 1 --2- 172.21.15.47:0/4264314159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44701571e0 0x7f447006f5b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4460009870 tx=0x7f446002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7548a38c92b1eca9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.406+0000 7f446dffb640 1 -- 172.21.15.47:0/4264314159 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4460031640 con 0x7f44701571e0 2024-03-23T09:42:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.406+0000 7f446dffb640 1 -- 172.21.15.47:0/4264314159 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44600317a0 con 0x7f44701571e0 2024-03-23T09:42:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.406+0000 7f446dffb640 1 -- 172.21.15.47:0/4264314159 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4460031b30 con 0x7f44701571e0 2024-03-23T09:42:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.406+0000 7f447568f640 1 -- 172.21.15.47:0/4264314159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44701571e0 msgr2=0x7f447006f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.406+0000 7f447568f640 1 --2- 172.21.15.47:0/4264314159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44701571e0 0x7f447006f5b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4460009870 tx=0x7f446002b8f0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.406+0000 7f447568f640 1 -- 172.21.15.47:0/4264314159 shutdown_connections 2024-03-23T09:42:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.406+0000 7f447568f640 1 --2- 172.21.15.47:0/4264314159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f447006faf0 0x7f447015d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.406+0000 7f447568f640 1 --2- 172.21.15.47:0/4264314159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44701571e0 0x7f447006f5b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.406+0000 7f447568f640 1 -- 172.21.15.47:0/4264314159 >> 172.21.15.47:0/4264314159 conn(0x7f4470071400 msgr2=0x7f4470071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.406+0000 7f447568f640 1 -- 172.21.15.47:0/4264314159 shutdown_connections 2024-03-23T09:42:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.406+0000 7f447568f640 1 -- 172.21.15.47:0/4264314159 wait complete. 2024-03-23T09:42:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.408+0000 7f447568f640 1 Processor -- start 2024-03-23T09:42:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.408+0000 7f447568f640 1 -- start start 2024-03-23T09:42:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.408+0000 7f447568f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f447006faf0 0x7f447014b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.408+0000 7f447568f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44701571e0 0x7f44700cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f447568f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f447015d600 0x7f44700cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f447568f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44700acef0 con 0x7f44701571e0 2024-03-23T09:42:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f447568f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44700ac300 con 0x7f447006faf0 2024-03-23T09:42:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f447568f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44700ac550 con 0x7f447015d600 2024-03-23T09:42:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f446e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44701571e0 0x7f44700cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f446effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f447006faf0 0x7f447014b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f446e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44701571e0 0x7f44700cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44904/0 (socket says 172.21.15.47:44904) 2024-03-23T09:42:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f446e7fc640 1 -- 172.21.15.47:0/2211083232 learned_addr learned my addr 172.21.15.47:0/2211083232 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f446f7fe640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f447015d600 0x7f44700cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f446e7fc640 1 -- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f447015d600 msgr2=0x7f44700cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f446e7fc640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f447015d600 0x7f44700cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f446e7fc640 1 -- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f447006faf0 msgr2=0x7f447014b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f446e7fc640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f447006faf0 0x7f447014b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f446e7fc640 1 -- 172.21.15.47:0/2211083232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44600094d0 con 0x7f44701571e0 2024-03-23T09:42:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f446effd640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f447006faf0 0x7f447014b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:42:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f446e7fc640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44701571e0 0x7f44700cb250 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f446400e9b0 tx=0x7f446400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f44577fe640 1 -- 172.21.15.47:0/2211083232 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464009670 con 0x7f44701571e0 2024-03-23T09:42:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f447568f640 1 -- 172.21.15.47:0/2211083232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44700ac830 con 0x7f44701571e0 2024-03-23T09:42:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f44577fe640 1 -- 172.21.15.47:0/2211083232 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446400c9a0 con 0x7f44701571e0 2024-03-23T09:42:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.409+0000 7f44577fe640 1 -- 172.21.15.47:0/2211083232 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446400cd50 con 0x7f44701571e0 2024-03-23T09:42:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.410+0000 7f447568f640 1 -- 172.21.15.47:0/2211083232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44700c8ab0 con 0x7f44701571e0 2024-03-23T09:42:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.411+0000 7f447568f640 1 -- 172.21.15.47:0/2211083232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44700ad030 con 0x7f44701571e0 2024-03-23T09:42:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.411+0000 7f44577fe640 1 -- 172.21.15.47:0/2211083232 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4464013470 con 0x7f44701571e0 2024-03-23T09:42:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.411+0000 7f44577fe640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f445c06ac00 0x7f445c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.411+0000 7f44577fe640 1 -- 172.21.15.47:0/2211083232 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4464089290 con 0x7f44701571e0 2024-03-23T09:42:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.412+0000 7f446effd640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f445c06ac00 0x7f445c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.412+0000 7f446effd640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f445c06ac00 0x7f445c06d0c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4460036000 tx=0x7f446003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:43.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.416+0000 7f44577fe640 1 -- 172.21.15.47:0/2211083232 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f44700ad030 con 0x7f44701571e0 2024-03-23T09:42:43.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:43.609+0000 7f447568f640 1 -- 172.21.15.47:0/2211083232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f44700ad030 con 0x7f44701571e0 2024-03-23T09:42:44.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.411+0000 7f44577fe640 1 -- 172.21.15.47:0/2211083232 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) v1 ==== 97+0+0 (secure 0 0 0) 0x7f44700ad030 con 0x7f44701571e0 2024-03-23T09:42:44.411 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blocklist entries 2024-03-23T09:42:44.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.414+0000 7f447568f640 1 -- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f445c06ac00 msgr2=0x7f445c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:44.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.414+0000 7f447568f640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f445c06ac00 0x7f445c06d0c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4460036000 tx=0x7f446003c040 comp rx=0 tx=0).stop 2024-03-23T09:42:44.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.414+0000 7f447568f640 1 -- 172.21.15.47:0/2211083232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44701571e0 msgr2=0x7f44700cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:44.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.414+0000 7f447568f640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44701571e0 0x7f44700cb250 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f446400e9b0 tx=0x7f446400ee70 comp rx=0 tx=0).stop 2024-03-23T09:42:44.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.414+0000 7f447568f640 1 -- 172.21.15.47:0/2211083232 shutdown_connections 2024-03-23T09:42:44.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.414+0000 7f447568f640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f445c06ac00 0x7f445c06d0c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.414+0000 7f447568f640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f447015d600 0x7f44700cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.414+0000 7f447568f640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44701571e0 0x7f44700cb250 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.414+0000 7f447568f640 1 --2- 172.21.15.47:0/2211083232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f447006faf0 0x7f447014b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.414+0000 7f447568f640 1 -- 172.21.15.47:0/2211083232 >> 172.21.15.47:0/2211083232 conn(0x7f4470071400 msgr2=0x7f4470069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:44.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.414+0000 7f447568f640 1 -- 172.21.15.47:0/2211083232 shutdown_connections 2024-03-23T09:42:44.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.414+0000 7f447568f640 1 -- 172.21.15.47:0/2211083232 wait complete. 2024-03-23T09:42:44.427 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-23T09:42:44.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.534+0000 7f3a65e4a640 1 Processor -- start 2024-03-23T09:42:44.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.534+0000 7f3a65e4a640 1 -- start start 2024-03-23T09:42:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.535+0000 7f3a65e4a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3a6015cd00 0x7f3a600d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.535+0000 7f3a65e4a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a6015d0e0 con 0x7f3a600d1020 2024-03-23T09:42:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.535+0000 7f3a5effd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3a600d1020 0x7f3a600cf210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45064/0 (socket says 172.21.15.47:45064) 2024-03-23T09:42:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.535+0000 7f3a5f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3a6015cd00 0x7f3a600d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.535+0000 7f3a5effd640 1 -- 172.21.15.47:0/2757770265 learned_addr learned my addr 172.21.15.47:0/2757770265 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.535+0000 7f3a65e4a640 1 -- 172.21.15.47:0/2757770265 --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a600d1540 con 0x7f3a600cf920 2024-03-23T09:42:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.535+0000 7f3a65e4a640 1 -- 172.21.15.47:0/2757770265 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a600d1820 con 0x7f3a6015cd00 2024-03-23T09:42:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.535+0000 7f3a5e7fc640 1 -- 172.21.15.47:0/2757770265 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392459411 0 0) 0x7f3a6015d0e0 con 0x7f3a600d1020 2024-03-23T09:42:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.535+0000 7f3a5e7fc640 1 -- 172.21.15.47:0/2757770265 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a48003520 con 0x7f3a600d1020 2024-03-23T09:42:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.535+0000 7f3a5f7fe640 1 -- 172.21.15.47:0/2757770265 >> v1:172.21.15.175:6789/0 conn(0x7f3a600cf920 legacy=0x7f3a600b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.535+0000 7f3a5f7fe640 1 -- 172.21.15.47:0/2757770265 >> v1:172.21.15.47:6789/0 conn(0x7f3a600d1020 legacy=0x7f3a600cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.535+0000 7f3a5f7fe640 1 -- 172.21.15.47:0/2757770265 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a600d1b00 con 0x7f3a6015cd00 2024-03-23T09:42:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.536+0000 7f3a5f7fe640 1 --2- 172.21.15.47:0/2757770265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a6015cd00 0x7f3a600d0ae0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3a500097a0 tx=0x7f3a5002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd1e07a308c60c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.536+0000 7f3a5e7fc640 1 -- 172.21.15.47:0/2757770265 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5002e070 con 0x7f3a6015cd00 2024-03-23T09:42:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.536+0000 7f3a5e7fc640 1 -- 172.21.15.47:0/2757770265 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a500315a0 con 0x7f3a6015cd00 2024-03-23T09:42:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.536+0000 7f3a5e7fc640 1 -- 172.21.15.47:0/2757770265 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50031930 con 0x7f3a6015cd00 2024-03-23T09:42:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.537+0000 7f3a65e4a640 1 -- 172.21.15.47:0/2757770265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a6015cd00 msgr2=0x7f3a600d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.537+0000 7f3a65e4a640 1 --2- 172.21.15.47:0/2757770265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a6015cd00 0x7f3a600d0ae0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3a500097a0 tx=0x7f3a5002b840 comp rx=0 tx=0).stop 2024-03-23T09:42:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.537+0000 7f3a65e4a640 1 -- 172.21.15.47:0/2757770265 shutdown_connections 2024-03-23T09:42:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.537+0000 7f3a65e4a640 1 --2- 172.21.15.47:0/2757770265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a6015cd00 0x7f3a600d0ae0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.537+0000 7f3a65e4a640 1 -- 172.21.15.47:0/2757770265 >> 172.21.15.47:0/2757770265 conn(0x7f3a6006b160 msgr2=0x7f3a6006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.537+0000 7f3a65e4a640 1 -- 172.21.15.47:0/2757770265 shutdown_connections 2024-03-23T09:42:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.537+0000 7f3a65e4a640 1 -- 172.21.15.47:0/2757770265 wait complete. 2024-03-23T09:42:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.538+0000 7f3a65e4a640 1 Processor -- start 2024-03-23T09:42:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.538+0000 7f3a65e4a640 1 -- start start 2024-03-23T09:42:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a65e4a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600cf920 0x7f3a600ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a65e4a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a600d1020 0x7f3a60157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a65e4a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a6015cd00 0x7f3a600cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a65e4a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600ccaa0 con 0x7f3a600cf920 2024-03-23T09:42:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a65e4a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600ca980 con 0x7f3a600d1020 2024-03-23T09:42:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a65e4a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600cac20 con 0x7f3a6015cd00 2024-03-23T09:42:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a5f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600cf920 0x7f3a600ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a5f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600cf920 0x7f3a600ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44926/0 (socket says 172.21.15.47:44926) 2024-03-23T09:42:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a5f7fe640 1 -- 172.21.15.47:0/148506963 learned_addr learned my addr 172.21.15.47:0/148506963 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a5effd640 1 --2- 172.21.15.47:0/148506963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a600d1020 0x7f3a60157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a5f7fe640 1 -- 172.21.15.47:0/148506963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a6015cd00 msgr2=0x7f3a600cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a5f7fe640 1 --2- 172.21.15.47:0/148506963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a6015cd00 0x7f3a600cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a5f7fe640 1 -- 172.21.15.47:0/148506963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a600d1020 msgr2=0x7f3a60157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a5f7fe640 1 --2- 172.21.15.47:0/148506963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a600d1020 0x7f3a60157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.539+0000 7f3a5f7fe640 1 -- 172.21.15.47:0/148506963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a50009400 con 0x7f3a600cf920 2024-03-23T09:42:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.540+0000 7f3a5f7fe640 1 --2- 172.21.15.47:0/148506963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600cf920 0x7f3a600ca440 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3a50034040 tx=0x7f3a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.540+0000 7f3a47fff640 1 -- 172.21.15.47:0/148506963 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50032840 con 0x7f3a600cf920 2024-03-23T09:42:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.540+0000 7f3a47fff640 1 -- 172.21.15.47:0/148506963 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a500329c0 con 0x7f3a600cf920 2024-03-23T09:42:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.540+0000 7f3a65e4a640 1 -- 172.21.15.47:0/148506963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a600caf00 con 0x7f3a600cf920 2024-03-23T09:42:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.540+0000 7f3a47fff640 1 -- 172.21.15.47:0/148506963 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50032d70 con 0x7f3a600cf920 2024-03-23T09:42:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.540+0000 7f3a65e4a640 1 -- 172.21.15.47:0/148506963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a600cb4c0 con 0x7f3a600cf920 2024-03-23T09:42:44.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.541+0000 7f3a65e4a640 1 -- 172.21.15.47:0/148506963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a60072360 con 0x7f3a600cf920 2024-03-23T09:42:44.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.541+0000 7f3a47fff640 1 -- 172.21.15.47:0/148506963 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3a5003b3f0 con 0x7f3a600cf920 2024-03-23T09:42:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.542+0000 7f3a47fff640 1 --2- 172.21.15.47:0/148506963 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a4c06ac00 0x7f3a4c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.542+0000 7f3a47fff640 1 -- 172.21.15.47:0/148506963 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3a500aaf50 con 0x7f3a600cf920 2024-03-23T09:42:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.542+0000 7f3a5effd640 1 --2- 172.21.15.47:0/148506963 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a4c06ac00 0x7f3a4c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.543+0000 7f3a5effd640 1 --2- 172.21.15.47:0/148506963 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a4c06ac00 0x7f3a4c06d0c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3a600c8e50 tx=0x7f3a54009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:44.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.546+0000 7f3a47fff640 1 -- 172.21.15.47:0/148506963 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3a50076470 con 0x7f3a600cf920 2024-03-23T09:42:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.794+0000 7f3a65e4a640 1 -- 172.21.15.47:0/148506963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3a60066be0 con 0x7f3a600cf920 2024-03-23T09:42:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.795+0000 7f3a47fff640 1 -- 172.21.15.47:0/148506963 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3a50079a90 con 0x7f3a600cf920 2024-03-23T09:42:44.795 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:44.795 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:42:44.795 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2024-03-23T09:42:44.795 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.795 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2024-03-23T09:42:44.795 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lo+xoDhAAyxqr1H6eiINwt2X7LUlrSg==", 2024-03-23T09:42:44.795 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.795 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:42:44.795 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:42:44.795 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:42:44.795 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZ5OlGhAAuUElzRghNRFGKREdCpxKvA==", 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2024-03-23T09:42:44.796 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lFwyzFRAAKldMOA7oSblA8YMrvntY8w==", 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.d", 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lpiD3IRAApXAdUQ4AkxWHzsFV0qcBng==", 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB7o/5lLvoADBAAgdixkm+jHwQilYw0IAn6EA==", 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB+o/5lOBfGCRAApglzalO1ztnJzOKg/UAyWA==", 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCBo/5lRBjlBxAAd45w8/kTMPqxmNyfCS3XYA==", 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCEo/5l+JxrBhAAbR/XqDyPDnN8eLpmvBZ7Mg==", 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCKo/5ltsqwDRAAkqcZ/sEf2oozWFN4AXg/+w==", 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:42:44.800 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:42:44.801 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.801 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.801 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2024-03-23T09:42:44.801 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCNo/5lpLHiCxAAB2axUv9gjcYVlij9f+AEHg==", 2024-03-23T09:42:44.801 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.801 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:42:44.801 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:42:44.801 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.801 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCQo/5l2io3ChAAu+lfYcWWBQf43aYzWjCBNg==", 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCTo/5lA7jjCBAAjJgBkp89H8lx6fidl/Z7IA==", 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:42:44.802 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZFc3JhAAZWmDZOnkW9eo9hqEDYynQw==", 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.803 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lIrJSLxAAPJer3ZP3h8VkImkiJmnKVw==", 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l1+DyKxAAZLKZNbUpUJ0fWPIYNvWK+g==", 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:42:44.804 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lWqkiNRAAL1q0lEcM/NKW3Pf9LcP21g==", 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB4o/5lyeuKLRAARrfTpPc5BTVYZe/LUREDkg==", 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l/0EOCxAAIrgIXRiFSQ1G7Jat5hNm4A==", 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:42:44.806 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lywMFBxAAIVwjCe6B9EtBMAG2n7cZJw==", 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-23T09:42:44.807 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:42:44.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.796+0000 7f3a65e4a640 1 -- 172.21.15.47:0/148506963 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a4c06ac00 msgr2=0x7f3a4c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:44.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.796+0000 7f3a65e4a640 1 --2- 172.21.15.47:0/148506963 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a4c06ac00 0x7f3a4c06d0c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3a600c8e50 tx=0x7f3a54009040 comp rx=0 tx=0).stop 2024-03-23T09:42:44.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.796+0000 7f3a65e4a640 1 -- 172.21.15.47:0/148506963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600cf920 msgr2=0x7f3a600ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:44.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.796+0000 7f3a65e4a640 1 --2- 172.21.15.47:0/148506963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600cf920 0x7f3a600ca440 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3a50034040 tx=0x7f3a500046c0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.796+0000 7f3a65e4a640 1 -- 172.21.15.47:0/148506963 shutdown_connections 2024-03-23T09:42:44.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.796+0000 7f3a65e4a640 1 --2- 172.21.15.47:0/148506963 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a4c06ac00 0x7f3a4c06d0c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.796+0000 7f3a65e4a640 1 --2- 172.21.15.47:0/148506963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a6015cd00 0x7f3a600cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.796+0000 7f3a65e4a640 1 --2- 172.21.15.47:0/148506963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a600d1020 0x7f3a60157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.796+0000 7f3a65e4a640 1 --2- 172.21.15.47:0/148506963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600cf920 0x7f3a600ca440 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.797+0000 7f3a65e4a640 1 -- 172.21.15.47:0/148506963 >> 172.21.15.47:0/148506963 conn(0x7f3a6006b160 msgr2=0x7f3a6015bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:44.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.797+0000 7f3a65e4a640 1 -- 172.21.15.47:0/148506963 shutdown_connections 2024-03-23T09:42:44.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.797+0000 7f3a65e4a640 1 -- 172.21.15.47:0/148506963 wait complete. 2024-03-23T09:42:44.810 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:42:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.911+0000 7f885213e640 1 Processor -- start 2024-03-23T09:42:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.911+0000 7f885213e640 1 -- start start 2024-03-23T09:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.912+0000 7f885213e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f884c15ca90 0x7f884c160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.912+0000 7f885213e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f884c0cde10 0x7f884c0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.912+0000 7f885213e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c0d11e0 con 0x7f884c0b8350 2024-03-23T09:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.912+0000 7f885213e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884c160f60 con 0x7f884c15ca90 2024-03-23T09:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.912+0000 7f885213e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884c161240 con 0x7f884c0cde10 2024-03-23T09:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.912+0000 7f884b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f884c15ca90 0x7f884c160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.912+0000 7f884b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f884c15ca90 0x7f884c160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39666/0 (socket says 172.21.15.47:39666) 2024-03-23T09:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.912+0000 7f884b7fe640 1 -- 172.21.15.47:0/131736330 learned_addr learned my addr 172.21.15.47:0/131736330 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.912+0000 7f884bfff640 1 --2- 172.21.15.47:0/131736330 >> v2:172.21.15.175:3300/0 conn(0x7f884c0cde10 0x7f884c0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.912+0000 7f884b7fe640 1 -- 172.21.15.47:0/131736330 >> v2:172.21.15.175:3300/0 conn(0x7f884c0cde10 msgr2=0x7f884c0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.912+0000 7f884b7fe640 1 --2- 172.21.15.47:0/131736330 >> v2:172.21.15.175:3300/0 conn(0x7f884c0cde10 0x7f884c0d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.912+0000 7f884b7fe640 1 -- 172.21.15.47:0/131736330 >> v1:172.21.15.175:6789/0 conn(0x7f884c0b8350 legacy=0x7f884c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:44.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.912+0000 7f884b7fe640 1 -- 172.21.15.47:0/131736330 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c161520 con 0x7f884c15ca90 2024-03-23T09:42:44.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.913+0000 7f884b7fe640 1 --2- 172.21.15.47:0/131736330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c15ca90 0x7f884c160710 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f884c0b5930 tx=0x7f883c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba16c237702ce20 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:44.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.913+0000 7f884a7fc640 1 -- 172.21.15.47:0/131736330 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0317e0 con 0x7f884c15ca90 2024-03-23T09:42:44.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.913+0000 7f884a7fc640 1 -- 172.21.15.47:0/131736330 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c031940 con 0x7f884c15ca90 2024-03-23T09:42:44.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.913+0000 7f884a7fc640 1 -- 172.21.15.47:0/131736330 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c031cd0 con 0x7f884c15ca90 2024-03-23T09:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.913+0000 7f885213e640 1 -- 172.21.15.47:0/131736330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c15ca90 msgr2=0x7f884c160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.913+0000 7f885213e640 1 --2- 172.21.15.47:0/131736330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c15ca90 0x7f884c160710 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f884c0b5930 tx=0x7f883c02b880 comp rx=0 tx=0).stop 2024-03-23T09:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.914+0000 7f885213e640 1 -- 172.21.15.47:0/131736330 shutdown_connections 2024-03-23T09:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.914+0000 7f885213e640 1 --2- 172.21.15.47:0/131736330 >> v2:172.21.15.175:3300/0 conn(0x7f884c0cde10 0x7f884c0d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.914+0000 7f885213e640 1 --2- 172.21.15.47:0/131736330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c15ca90 0x7f884c160710 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.914+0000 7f885213e640 1 -- 172.21.15.47:0/131736330 >> 172.21.15.47:0/131736330 conn(0x7f884c0659d0 msgr2=0x7f884c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.914+0000 7f885213e640 1 -- 172.21.15.47:0/131736330 shutdown_connections 2024-03-23T09:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.914+0000 7f885213e640 1 -- 172.21.15.47:0/131736330 wait complete. 2024-03-23T09:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.915+0000 7f885213e640 1 Processor -- start 2024-03-23T09:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.915+0000 7f885213e640 1 -- start start 2024-03-23T09:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f885213e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f884c0b8350 0x7f884c0cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f885213e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c0cde10 0x7f884c0c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f885213e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f884c15ca90 0x7f884c146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:44.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f885213e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0c9fd0 con 0x7f884c0cde10 2024-03-23T09:42:44.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f885213e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0cc590 con 0x7f884c15ca90 2024-03-23T09:42:44.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f885213e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0cc870 con 0x7f884c0b8350 2024-03-23T09:42:44.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f884affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c0cde10 0x7f884c0c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:44.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f884affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c0cde10 0x7f884c0c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39672/0 (socket says 172.21.15.47:39672) 2024-03-23T09:42:44.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f884affd640 1 -- 172.21.15.47:0/3260713410 learned_addr learned my addr 172.21.15.47:0/3260713410 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:44.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f884b7fe640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f884c0b8350 0x7f884c0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:44.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f884bfff640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f884c15ca90 0x7f884c146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:44.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f884affd640 1 -- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f884c0b8350 msgr2=0x7f884c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:44.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f884affd640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f884c0b8350 0x7f884c0cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f884affd640 1 -- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f884c15ca90 msgr2=0x7f884c146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:44.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f884affd640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f884c15ca90 0x7f884c146930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:44.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f884affd640 1 -- 172.21.15.47:0/3260713410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c0094d0 con 0x7f884c0cde10 2024-03-23T09:42:44.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f884bfff640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f884c15ca90 0x7f884c146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:44.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f884affd640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c0cde10 0x7f884c0c9a90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f884000c430 tx=0x7f884000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:44.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f8833fff640 1 -- 172.21.15.47:0/3260713410 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840011790 con 0x7f884c0cde10 2024-03-23T09:42:44.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f8833fff640 1 -- 172.21.15.47:0/3260713410 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88400118f0 con 0x7f884c0cde10 2024-03-23T09:42:44.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f885213e640 1 -- 172.21.15.47:0/3260713410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884c0ccb50 con 0x7f884c0cde10 2024-03-23T09:42:44.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f8833fff640 1 -- 172.21.15.47:0/3260713410 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840011ce0 con 0x7f884c0cde10 2024-03-23T09:42:44.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.916+0000 7f885213e640 1 -- 172.21.15.47:0/3260713410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c0cd110 con 0x7f884c0cde10 2024-03-23T09:42:44.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.917+0000 7f885213e640 1 -- 172.21.15.47:0/3260713410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f884c072360 con 0x7f884c0cde10 2024-03-23T09:42:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.918+0000 7f8833fff640 1 -- 172.21.15.47:0/3260713410 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f884001c460 con 0x7f884c0cde10 2024-03-23T09:42:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.918+0000 7f8833fff640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f883806ac00 0x7f883806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.918+0000 7f8833fff640 1 -- 172.21.15.47:0/3260713410 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8840088fa0 con 0x7f884c0cde10 2024-03-23T09:42:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.919+0000 7f884b7fe640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f883806ac00 0x7f883806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.919+0000 7f884b7fe640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f883806ac00 0x7f883806d0c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f883c036000 tx=0x7f883c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:44.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:44.923+0000 7f8833fff640 1 -- 172.21.15.47:0/3260713410 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f88400bc580 con 0x7f884c0cde10 2024-03-23T09:42:45.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.105+0000 7f885213e640 1 -- 172.21.15.47:0/3260713410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f884c0cd2d0 con 0x7f884c0cde10 2024-03-23T09:42:45.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.106+0000 7f8833fff640 1 -- 172.21.15.47:0/3260713410 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+12389 (secure 0 0 0) 0x7f88400544d0 con 0x7f884c0cde10 2024-03-23T09:42:45.105 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:45.106 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":42,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:44.408681+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:42:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.108+0000 7f885213e640 1 -- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f883806ac00 msgr2=0x7f883806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.108+0000 7f885213e640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f883806ac00 0x7f883806d0c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f883c036000 tx=0x7f883c03d040 comp rx=0 tx=0).stop 2024-03-23T09:42:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.108+0000 7f885213e640 1 -- 172.21.15.47:0/3260713410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c0cde10 msgr2=0x7f884c0c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.108+0000 7f885213e640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c0cde10 0x7f884c0c9a90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f884000c430 tx=0x7f884000c8f0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.108+0000 7f885213e640 1 -- 172.21.15.47:0/3260713410 shutdown_connections 2024-03-23T09:42:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.108+0000 7f885213e640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f883806ac00 0x7f883806d0c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.108+0000 7f885213e640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f884c15ca90 0x7f884c146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.108+0000 7f885213e640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c0cde10 0x7f884c0c9a90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.108+0000 7f885213e640 1 --2- 172.21.15.47:0/3260713410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f884c0b8350 0x7f884c0cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.108+0000 7f885213e640 1 -- 172.21.15.47:0/3260713410 >> 172.21.15.47:0/3260713410 conn(0x7f884c0659d0 msgr2=0x7f884c0b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.108+0000 7f885213e640 1 -- 172.21.15.47:0/3260713410 shutdown_connections 2024-03-23T09:42:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.108+0000 7f885213e640 1 -- 172.21.15.47:0/3260713410 wait complete. 2024-03-23T09:42:45.119 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:42:45.119 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:42:45.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.222+0000 7f8e20069640 1 Processor -- start 2024-03-23T09:42:45.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.223+0000 7f8e20069640 1 -- start start 2024-03-23T09:42:45.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.223+0000 7f8e20069640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8e180cc2d0 0x7f8e18164390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.223+0000 7f8e20069640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1806f260 con 0x7f8e1815cc50 2024-03-23T09:42:45.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.223+0000 7f8e20069640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1806e360 con 0x7f8e180d01b0 2024-03-23T09:42:45.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.223+0000 7f8e20069640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e180d0590 con 0x7f8e180cc2d0 2024-03-23T09:42:45.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.223+0000 7f8e1de05640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8e1815cc50 0x7f8e1806eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41708/0 (socket says 172.21.15.47:41708) 2024-03-23T09:42:45.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.223+0000 7f8e1de05640 1 -- 172.21.15.47:0/615071511 learned_addr learned my addr 172.21.15.47:0/615071511 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:45.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.224+0000 7f8e1e606640 1 --2- 172.21.15.47:0/615071511 >> v2:172.21.15.175:3300/0 conn(0x7f8e180cc2d0 0x7f8e18164390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.224+0000 7f8e1ce03640 1 -- 172.21.15.47:0/615071511 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399115467 0 0) 0x7f8e1806f260 con 0x7f8e1815cc50 2024-03-23T09:42:45.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.224+0000 7f8e1ce03640 1 -- 172.21.15.47:0/615071511 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e00003520 con 0x7f8e1815cc50 2024-03-23T09:42:45.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.224+0000 7f8e1e606640 1 -- 172.21.15.47:0/615071511 >> v1:172.21.15.175:6789/0 conn(0x7f8e180d01b0 legacy=0x7f8e180cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.224+0000 7f8e1e606640 1 -- 172.21.15.47:0/615071511 >> v1:172.21.15.47:6789/0 conn(0x7f8e1815cc50 legacy=0x7f8e1806eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.224+0000 7f8e1e606640 1 -- 172.21.15.47:0/615071511 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e180cdd40 con 0x7f8e180cc2d0 2024-03-23T09:42:45.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.225+0000 7f8e1e606640 1 --2- 172.21.15.47:0/615071511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e180cc2d0 0x7f8e18164390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009d90 tx=0x7f8e0802bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd652e97a61a6177 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:45.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.225+0000 7f8e1ce03640 1 -- 172.21.15.47:0/615071511 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0802cd80 con 0x7f8e180cc2d0 2024-03-23T09:42:45.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.225+0000 7f8e1ce03640 1 -- 172.21.15.47:0/615071511 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e08034440 con 0x7f8e180cc2d0 2024-03-23T09:42:45.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.225+0000 7f8e1ce03640 1 -- 172.21.15.47:0/615071511 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e080347b0 con 0x7f8e180cc2d0 2024-03-23T09:42:45.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.226+0000 7f8e20069640 1 -- 172.21.15.47:0/615071511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e180cc2d0 msgr2=0x7f8e18164390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.226+0000 7f8e20069640 1 --2- 172.21.15.47:0/615071511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e180cc2d0 0x7f8e18164390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009d90 tx=0x7f8e0802bfd0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.226+0000 7f8e20069640 1 -- 172.21.15.47:0/615071511 shutdown_connections 2024-03-23T09:42:45.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.226+0000 7f8e20069640 1 --2- 172.21.15.47:0/615071511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e180cc2d0 0x7f8e18164390 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.226+0000 7f8e20069640 1 -- 172.21.15.47:0/615071511 >> 172.21.15.47:0/615071511 conn(0x7f8e18098a00 msgr2=0x7f8e180cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:45.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.226+0000 7f8e20069640 1 -- 172.21.15.47:0/615071511 shutdown_connections 2024-03-23T09:42:45.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.226+0000 7f8e20069640 1 -- 172.21.15.47:0/615071511 wait complete. 2024-03-23T09:42:45.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.227+0000 7f8e20069640 1 Processor -- start 2024-03-23T09:42:45.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.227+0000 7f8e20069640 1 -- start start 2024-03-23T09:42:45.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e20069640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e180cc2d0 0x7f8e181534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e20069640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e180d01b0 0x7f8e1814c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e20069640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e1815cc50 0x7f8e180aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e20069640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e180b09c0 con 0x7f8e180cc2d0 2024-03-23T09:42:45.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e20069640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e180c7a60 con 0x7f8e180d01b0 2024-03-23T09:42:45.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e20069640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e180c7d40 con 0x7f8e1815cc50 2024-03-23T09:42:45.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e1de05640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e180cc2d0 0x7f8e181534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e1de05640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e180cc2d0 0x7f8e181534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39688/0 (socket says 172.21.15.47:39688) 2024-03-23T09:42:45.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e1de05640 1 -- 172.21.15.47:0/3572696422 learned_addr learned my addr 172.21.15.47:0/3572696422 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:45.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e1e606640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e1815cc50 0x7f8e180aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e1de05640 1 -- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e1815cc50 msgr2=0x7f8e180aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e1d604640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e180d01b0 0x7f8e1814c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e1de05640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e1815cc50 0x7f8e180aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e1de05640 1 -- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e180d01b0 msgr2=0x7f8e1814c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e1de05640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e180d01b0 0x7f8e1814c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e1de05640 1 -- 172.21.15.47:0/3572696422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e08009a70 con 0x7f8e180cc2d0 2024-03-23T09:42:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e1de05640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e180cc2d0 0x7f8e181534b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000c500 tx=0x7f8e1000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e167fc640 1 -- 172.21.15.47:0/3572696422 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10005ce0 con 0x7f8e180cc2d0 2024-03-23T09:42:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.228+0000 7f8e167fc640 1 -- 172.21.15.47:0/3572696422 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e10005e40 con 0x7f8e180cc2d0 2024-03-23T09:42:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.229+0000 7f8e167fc640 1 -- 172.21.15.47:0/3572696422 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10007330 con 0x7f8e180cc2d0 2024-03-23T09:42:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.229+0000 7f8e20069640 1 -- 172.21.15.47:0/3572696422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e180b0650 con 0x7f8e180cc2d0 2024-03-23T09:42:45.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.229+0000 7f8e20069640 1 -- 172.21.15.47:0/3572696422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e180b21b0 con 0x7f8e180cc2d0 2024-03-23T09:42:45.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.230+0000 7f8e20069640 1 -- 172.21.15.47:0/3572696422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e180b04b0 con 0x7f8e180cc2d0 2024-03-23T09:42:45.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.230+0000 7f8e167fc640 1 -- 172.21.15.47:0/3572696422 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8e10007550 con 0x7f8e180cc2d0 2024-03-23T09:42:45.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.230+0000 7f8e167fc640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e0406ac00 0x7f8e0406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.231+0000 7f8e167fc640 1 -- 172.21.15.47:0/3572696422 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8e100885d0 con 0x7f8e180cc2d0 2024-03-23T09:42:45.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.231+0000 7f8e1d604640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e0406ac00 0x7f8e0406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.231+0000 7f8e1d604640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e0406ac00 0x7f8e0406d0c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8e180c5d60 tx=0x7f8e0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:45.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.235+0000 7f8e167fc640 1 -- 172.21.15.47:0/3572696422 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8e180b04b0 con 0x7f8e180cc2d0 2024-03-23T09:42:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.416+0000 7f8e20069640 1 -- 172.21.15.47:0/3572696422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e180b04b0 con 0x7f8e180cc2d0 2024-03-23T09:42:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.417+0000 7f8e167fc640 1 -- 172.21.15.47:0/3572696422 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e180b04b0 con 0x7f8e180cc2d0 2024-03-23T09:42:45.417 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:42:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.419+0000 7f8e20069640 1 -- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e0406ac00 msgr2=0x7f8e0406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.419+0000 7f8e20069640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e0406ac00 0x7f8e0406d0c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8e180c5d60 tx=0x7f8e0c009040 comp rx=0 tx=0).stop 2024-03-23T09:42:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.419+0000 7f8e20069640 1 -- 172.21.15.47:0/3572696422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e180cc2d0 msgr2=0x7f8e181534b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.419+0000 7f8e20069640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e180cc2d0 0x7f8e181534b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000c500 tx=0x7f8e1000c9c0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.419+0000 7f8e20069640 1 -- 172.21.15.47:0/3572696422 shutdown_connections 2024-03-23T09:42:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.419+0000 7f8e20069640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e0406ac00 0x7f8e0406d0c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.419+0000 7f8e20069640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e1815cc50 0x7f8e180aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.419+0000 7f8e20069640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e180d01b0 0x7f8e1814c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.419+0000 7f8e20069640 1 --2- 172.21.15.47:0/3572696422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e180cc2d0 0x7f8e181534b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.419+0000 7f8e20069640 1 -- 172.21.15.47:0/3572696422 >> 172.21.15.47:0/3572696422 conn(0x7f8e18098a00 msgr2=0x7f8e180cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.419+0000 7f8e20069640 1 -- 172.21.15.47:0/3572696422 shutdown_connections 2024-03-23T09:42:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.419+0000 7f8e20069640 1 -- 172.21.15.47:0/3572696422 wait complete. 2024-03-23T09:42:45.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T09:42:45.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.530+0000 7f7cfee96640 1 Processor -- start 2024-03-23T09:42:45.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.530+0000 7f7cfee96640 1 -- start start 2024-03-23T09:42:45.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.530+0000 7f7cfee96640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7cf815cdb0 0x7f7cf8071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.530+0000 7f7cfee96640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf80721f0 con 0x7f7cf815c9d0 2024-03-23T09:42:45.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.530+0000 7f7cfee96640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf806eb80 con 0x7f7cf806a3f0 2024-03-23T09:42:45.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.530+0000 7f7cfee96640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf806edd0 con 0x7f7cf815cdb0 2024-03-23T09:42:45.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.531+0000 7f7cf7fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7cf815cdb0 0x7f7cf8071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.531+0000 7f7cfcc32640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7cf815c9d0 0x7f7cf8069ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41722/0 (socket says 172.21.15.47:41722) 2024-03-23T09:42:45.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.531+0000 7f7cfcc32640 1 -- 172.21.15.47:0/618672694 learned_addr learned my addr 172.21.15.47:0/618672694 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:45.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.531+0000 7f7cf7fff640 1 -- 172.21.15.47:0/618672694 >> v1:172.21.15.175:6789/0 conn(0x7f7cf806a3f0 legacy=0x7f7cf8160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:45.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.531+0000 7f7cf7fff640 1 -- 172.21.15.47:0/618672694 >> v1:172.21.15.47:6789/0 conn(0x7f7cf815c9d0 legacy=0x7f7cf8069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.531+0000 7f7cf7fff640 1 -- 172.21.15.47:0/618672694 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf806f0b0 con 0x7f7cf815cdb0 2024-03-23T09:42:45.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.531+0000 7f7cf77fe640 1 -- 172.21.15.47:0/618672694 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351399867 0 0) 0x7f7cf80721f0 con 0x7f7cf815c9d0 2024-03-23T09:42:45.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.532+0000 7f7cf7fff640 1 --2- 172.21.15.47:0/618672694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cf815cdb0 0x7f7cf8071cb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40097a0 tx=0x7f7ce402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24800b2692b8955e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:45.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.532+0000 7f7cf77fe640 1 -- 172.21.15.47:0/618672694 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4030610 con 0x7f7cf815cdb0 2024-03-23T09:42:45.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.532+0000 7f7cf77fe640 1 -- 172.21.15.47:0/618672694 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce4030770 con 0x7f7cf815cdb0 2024-03-23T09:42:45.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.532+0000 7f7cf77fe640 1 -- 172.21.15.47:0/618672694 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4030b00 con 0x7f7cf815cdb0 2024-03-23T09:42:45.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.532+0000 7f7cfee96640 1 -- 172.21.15.47:0/618672694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cf815cdb0 msgr2=0x7f7cf8071cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.532+0000 7f7cfee96640 1 --2- 172.21.15.47:0/618672694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cf815cdb0 0x7f7cf8071cb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40097a0 tx=0x7f7ce402b960 comp rx=0 tx=0).stop 2024-03-23T09:42:45.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.532+0000 7f7cfee96640 1 -- 172.21.15.47:0/618672694 shutdown_connections 2024-03-23T09:42:45.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.532+0000 7f7cfee96640 1 --2- 172.21.15.47:0/618672694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cf815cdb0 0x7f7cf8071cb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.532+0000 7f7cfee96640 1 -- 172.21.15.47:0/618672694 >> 172.21.15.47:0/618672694 conn(0x7f7cf80b13a0 msgr2=0x7f7cf80b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:45.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.532+0000 7f7cfee96640 1 -- 172.21.15.47:0/618672694 shutdown_connections 2024-03-23T09:42:45.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.532+0000 7f7cfee96640 1 -- 172.21.15.47:0/618672694 wait complete. 2024-03-23T09:42:45.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.533+0000 7f7cfee96640 1 Processor -- start 2024-03-23T09:42:45.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.533+0000 7f7cfee96640 1 -- start start 2024-03-23T09:42:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.534+0000 7f7cfee96640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cf806a3f0 0x7f7cf80af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.534+0000 7f7cfee96640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cf815c9d0 0x7f7cf8152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.534+0000 7f7cfee96640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cf815cdb0 0x7f7cf8153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.534+0000 7f7cfee96640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80ccf90 con 0x7f7cf815cdb0 2024-03-23T09:42:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.534+0000 7f7cfee96640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80cd270 con 0x7f7cf815c9d0 2024-03-23T09:42:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.534+0000 7f7cfee96640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80cd550 con 0x7f7cf806a3f0 2024-03-23T09:42:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.534+0000 7f7cfd433640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cf815cdb0 0x7f7cf8153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.534+0000 7f7cfd433640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cf815cdb0 0x7f7cf8153370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39720/0 (socket says 172.21.15.47:39720) 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.534+0000 7f7cfd433640 1 -- 172.21.15.47:0/3660739559 learned_addr learned my addr 172.21.15.47:0/3660739559 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.535+0000 7f7cfcc32640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cf806a3f0 0x7f7cf80af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.535+0000 7f7cf7fff640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cf815c9d0 0x7f7cf8152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.535+0000 7f7cfd433640 1 -- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cf806a3f0 msgr2=0x7f7cf80af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.535+0000 7f7cfd433640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cf806a3f0 0x7f7cf80af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.535+0000 7f7cfd433640 1 -- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cf815c9d0 msgr2=0x7f7cf8152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.535+0000 7f7cfd433640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cf815c9d0 0x7f7cf8152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.535+0000 7f7cfd433640 1 -- 172.21.15.47:0/3660739559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce4009400 con 0x7f7cf815cdb0 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.535+0000 7f7cfd433640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cf815cdb0 0x7f7cf8153370 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c730 tx=0x7f7cf000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.535+0000 7f7cf57fa640 1 -- 172.21.15.47:0/3660739559 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0003b70 con 0x7f7cf815cdb0 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.535+0000 7f7cf57fa640 1 -- 172.21.15.47:0/3660739559 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf0003cd0 con 0x7f7cf815cdb0 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.535+0000 7f7cf57fa640 1 -- 172.21.15.47:0/3660739559 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf000aa90 con 0x7f7cf815cdb0 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.535+0000 7f7cfee96640 1 -- 172.21.15.47:0/3660739559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf80cd830 con 0x7f7cf815cdb0 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.535+0000 7f7cfee96640 1 -- 172.21.15.47:0/3660739559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf80cf130 con 0x7f7cf815cdb0 2024-03-23T09:42:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.536+0000 7f7cfee96640 1 -- 172.21.15.47:0/3660739559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf81f37e0 con 0x7f7cf815cdb0 2024-03-23T09:42:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.537+0000 7f7cf57fa640 1 -- 172.21.15.47:0/3660739559 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7cf000abf0 con 0x7f7cf815cdb0 2024-03-23T09:42:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.537+0000 7f7cf57fa640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7cdc06ac00 0x7f7cdc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.537+0000 7f7cf57fa640 1 -- 172.21.15.47:0/3660739559 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7cf0088db0 con 0x7f7cf815cdb0 2024-03-23T09:42:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.537+0000 7f7cfcc32640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7cdc06ac00 0x7f7cdc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.538+0000 7f7cfcc32640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7cdc06ac00 0x7f7cdc06d0c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7cec000f60 tx=0x7f7cec007b90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:45.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.541+0000 7f7cf57fa640 1 -- 172.21.15.47:0/3660739559 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7cf81f37e0 con 0x7f7cf815cdb0 2024-03-23T09:42:45.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.724+0000 7f7cfee96640 1 -- 172.21.15.47:0/3660739559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7cf81f37e0 con 0x7f7cf815cdb0 2024-03-23T09:42:45.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.725+0000 7f7cf57fa640 1 -- 172.21.15.47:0/3660739559 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7cf81f37e0 con 0x7f7cf815cdb0 2024-03-23T09:42:45.724 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:45.725 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-23T09:42:45.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:42:45.725 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-03-23T09:42:45.725 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-03-23T09:42:45.725 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:45.725 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-23T09:42:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.727+0000 7f7cfee96640 1 -- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7cdc06ac00 msgr2=0x7f7cdc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.727+0000 7f7cfee96640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7cdc06ac00 0x7f7cdc06d0c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7cec000f60 tx=0x7f7cec007b90 comp rx=0 tx=0).stop 2024-03-23T09:42:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.727+0000 7f7cfee96640 1 -- 172.21.15.47:0/3660739559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cf815cdb0 msgr2=0x7f7cf8153370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.727+0000 7f7cfee96640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cf815cdb0 0x7f7cf8153370 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c730 tx=0x7f7cf000cbf0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.727+0000 7f7cfee96640 1 -- 172.21.15.47:0/3660739559 shutdown_connections 2024-03-23T09:42:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.727+0000 7f7cfee96640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7cdc06ac00 0x7f7cdc06d0c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.727+0000 7f7cfee96640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cf815cdb0 0x7f7cf8153370 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.727+0000 7f7cfee96640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cf815c9d0 0x7f7cf8152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.727+0000 7f7cfee96640 1 --2- 172.21.15.47:0/3660739559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cf806a3f0 0x7f7cf80af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.727+0000 7f7cfee96640 1 -- 172.21.15.47:0/3660739559 >> 172.21.15.47:0/3660739559 conn(0x7f7cf80b13a0 msgr2=0x7f7cf80cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.727+0000 7f7cfee96640 1 -- 172.21.15.47:0/3660739559 shutdown_connections 2024-03-23T09:42:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.727+0000 7f7cfee96640 1 -- 172.21.15.47:0/3660739559 wait complete. 2024-03-23T09:42:45.737 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T09:42:45.737 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T09:42:45.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.842+0000 7ff59815f640 1 Processor -- start 2024-03-23T09:42:45.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.842+0000 7ff59815f640 1 -- start start 2024-03-23T09:42:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.843+0000 7ff59815f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff59015a860 0x7ff590161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.843+0000 7ff59815f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff59015d0f0 0x7ff59015d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.843+0000 7ff59815f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff590160bd0 con 0x7ff5900cf190 2024-03-23T09:42:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.843+0000 7ff59815f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5900d2790 con 0x7ff59015d0f0 2024-03-23T09:42:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.843+0000 7ff59815f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59015dbb0 con 0x7ff59015a860 2024-03-23T09:42:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.843+0000 7ff5956fa640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff59015d0f0 0x7ff59015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.843+0000 7ff5966fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff5900cf190 0x7ff5900cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41738/0 (socket says 172.21.15.47:41738) 2024-03-23T09:42:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.843+0000 7ff5966fc640 1 -- 172.21.15.47:0/3164538853 learned_addr learned my addr 172.21.15.47:0/3164538853 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.843+0000 7ff5956fa640 1 -- 172.21.15.47:0/3164538853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff59015a860 msgr2=0x7ff590161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.843+0000 7ff595efb640 1 --2- 172.21.15.47:0/3164538853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff59015a860 0x7ff590161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.844+0000 7ff5956fa640 1 --2- 172.21.15.47:0/3164538853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff59015a860 0x7ff590161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.844+0000 7ff5956fa640 1 -- 172.21.15.47:0/3164538853 >> v1:172.21.15.47:6789/0 conn(0x7ff5900cf190 legacy=0x7ff5900cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.844+0000 7ff594ef9640 1 -- 172.21.15.47:0/3164538853 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620530699 0 0) 0x7ff590160bd0 con 0x7ff5900cf190 2024-03-23T09:42:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.844+0000 7ff5956fa640 1 -- 172.21.15.47:0/3164538853 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59015de90 con 0x7ff59015d0f0 2024-03-23T09:42:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.844+0000 7ff595efb640 1 --2- 172.21.15.47:0/3164538853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff59015a860 0x7ff590161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.844+0000 7ff5956fa640 1 --2- 172.21.15.47:0/3164538853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59015d0f0 0x7ff59015d550 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff5840097a0 tx=0x7ff58402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ad9427f8ff0970 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.844+0000 7ff594ef9640 1 -- 172.21.15.47:0/3164538853 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584031830 con 0x7ff59015d0f0 2024-03-23T09:42:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.844+0000 7ff594ef9640 1 -- 172.21.15.47:0/3164538853 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff584031990 con 0x7ff59015d0f0 2024-03-23T09:42:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.844+0000 7ff594ef9640 1 -- 172.21.15.47:0/3164538853 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584031d20 con 0x7ff59015d0f0 2024-03-23T09:42:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.845+0000 7ff59815f640 1 -- 172.21.15.47:0/3164538853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59015d0f0 msgr2=0x7ff59015d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.845+0000 7ff59815f640 1 --2- 172.21.15.47:0/3164538853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59015d0f0 0x7ff59015d550 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff5840097a0 tx=0x7ff58402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.845+0000 7ff59815f640 1 -- 172.21.15.47:0/3164538853 shutdown_connections 2024-03-23T09:42:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.845+0000 7ff59815f640 1 --2- 172.21.15.47:0/3164538853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59015d0f0 0x7ff59015d550 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.845+0000 7ff59815f640 1 --2- 172.21.15.47:0/3164538853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff59015a860 0x7ff590161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.845+0000 7ff59815f640 1 -- 172.21.15.47:0/3164538853 >> 172.21.15.47:0/3164538853 conn(0x7ff5900659d0 msgr2=0x7ff590065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.845+0000 7ff59815f640 1 -- 172.21.15.47:0/3164538853 shutdown_connections 2024-03-23T09:42:45.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.845+0000 7ff59815f640 1 -- 172.21.15.47:0/3164538853 wait complete. 2024-03-23T09:42:45.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.846+0000 7ff59815f640 1 Processor -- start 2024-03-23T09:42:45.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.846+0000 7ff59815f640 1 -- start start 2024-03-23T09:42:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff59815f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5900cf190 0x7ff5901f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff59815f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59015a860 0x7ff590155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff59815f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59015d0f0 0x7ff5900cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff59815f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5901f5e00 con 0x7ff59015a860 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff59815f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900c9de0 con 0x7ff59015d0f0 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff59815f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff590156110 con 0x7ff5900cf190 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff5956fa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59015a860 0x7ff590155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff5956fa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59015a860 0x7ff590155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39730/0 (socket says 172.21.15.47:39730) 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff5956fa640 1 -- 172.21.15.47:0/2380604279 learned_addr learned my addr 172.21.15.47:0/2380604279 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff5956fa640 1 -- 172.21.15.47:0/2380604279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5900cf190 msgr2=0x7ff5901f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff5956fa640 1 --2- 172.21.15.47:0/2380604279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5900cf190 0x7ff5901f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff5956fa640 1 -- 172.21.15.47:0/2380604279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59015d0f0 msgr2=0x7ff5900cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff5956fa640 1 --2- 172.21.15.47:0/2380604279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59015d0f0 0x7ff5900cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff5956fa640 1 -- 172.21.15.47:0/2380604279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff584009400 con 0x7ff59015a860 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.847+0000 7ff5956fa640 1 --2- 172.21.15.47:0/2380604279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59015a860 0x7ff590155bd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff584034040 tx=0x7ff5840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.848+0000 7ff58e7fc640 1 -- 172.21.15.47:0/2380604279 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58403aa60 con 0x7ff59015a860 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.848+0000 7ff58e7fc640 1 -- 172.21.15.47:0/2380604279 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58403abe0 con 0x7ff59015a860 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.848+0000 7ff58e7fc640 1 -- 172.21.15.47:0/2380604279 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584044400 con 0x7ff59015a860 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.848+0000 7ff59815f640 1 -- 172.21.15.47:0/2380604279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5900c6260 con 0x7ff59015a860 2024-03-23T09:42:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.848+0000 7ff59815f640 1 -- 172.21.15.47:0/2380604279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5901f5740 con 0x7ff59015a860 2024-03-23T09:42:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.849+0000 7ff59815f640 1 -- 172.21.15.47:0/2380604279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff590072360 con 0x7ff59015a860 2024-03-23T09:42:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.849+0000 7ff58e7fc640 1 -- 172.21.15.47:0/2380604279 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff58404d440 con 0x7ff59015a860 2024-03-23T09:42:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.849+0000 7ff58e7fc640 1 --2- 172.21.15.47:0/2380604279 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff57c06ac00 0x7ff57c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.850+0000 7ff58e7fc640 1 -- 172.21.15.47:0/2380604279 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff5840ab1e0 con 0x7ff59015a860 2024-03-23T09:42:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.850+0000 7ff595efb640 1 --2- 172.21.15.47:0/2380604279 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff57c06ac00 0x7ff57c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.850+0000 7ff595efb640 1 --2- 172.21.15.47:0/2380604279 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff57c06ac00 0x7ff57c06d0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff588006fd0 tx=0x7ff588008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:45.854+0000 7ff58e7fc640 1 -- 172.21.15.47:0/2380604279 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff5840767a0 con 0x7ff59015a860 2024-03-23T09:42:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.034+0000 7ff59815f640 1 -- 172.21.15.47:0/2380604279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff5901f5f40 con 0x7ff59015a860 2024-03-23T09:42:46.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.788+0000 7ff58e7fc640 1 -- 172.21.15.47:0/2380604279 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff5901f5f40 con 0x7ff59015a860 2024-03-23T09:42:46.788 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-03-23T09:42:46.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.792+0000 7ff59815f640 1 -- 172.21.15.47:0/2380604279 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff57c06ac00 msgr2=0x7ff57c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.792+0000 7ff59815f640 1 --2- 172.21.15.47:0/2380604279 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff57c06ac00 0x7ff57c06d0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff588006fd0 tx=0x7ff588008040 comp rx=0 tx=0).stop 2024-03-23T09:42:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.792+0000 7ff59815f640 1 -- 172.21.15.47:0/2380604279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59015a860 msgr2=0x7ff590155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.792+0000 7ff59815f640 1 --2- 172.21.15.47:0/2380604279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59015a860 0x7ff590155bd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff584034040 tx=0x7ff5840046c0 comp rx=0 tx=0).stop 2024-03-23T09:42:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.792+0000 7ff59815f640 1 -- 172.21.15.47:0/2380604279 shutdown_connections 2024-03-23T09:42:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.792+0000 7ff59815f640 1 --2- 172.21.15.47:0/2380604279 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff57c06ac00 0x7ff57c06d0c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.792+0000 7ff59815f640 1 --2- 172.21.15.47:0/2380604279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59015d0f0 0x7ff5900cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.792+0000 7ff59815f640 1 --2- 172.21.15.47:0/2380604279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59015a860 0x7ff590155bd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.792+0000 7ff59815f640 1 --2- 172.21.15.47:0/2380604279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5900cf190 0x7ff5901f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.792+0000 7ff59815f640 1 -- 172.21.15.47:0/2380604279 >> 172.21.15.47:0/2380604279 conn(0x7ff5900659d0 msgr2=0x7ff59015e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.793+0000 7ff59815f640 1 -- 172.21.15.47:0/2380604279 shutdown_connections 2024-03-23T09:42:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.793+0000 7ff59815f640 1 -- 172.21.15.47:0/2380604279 wait complete. 2024-03-23T09:42:46.808 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T09:42:46.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.921+0000 7f2d908b6640 1 Processor -- start 2024-03-23T09:42:46.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.921+0000 7f2d908b6640 1 -- start start 2024-03-23T09:42:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.922+0000 7f2d908b6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d8c0d2f80 0x7f2d8c161210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.922+0000 7f2d908b6640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c0cd940 con 0x7f2d8c0b1aa0 2024-03-23T09:42:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.922+0000 7f2d908b6640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c0b5a80 con 0x7f2d8c15cc70 2024-03-23T09:42:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.922+0000 7f2d908b6640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8c0ce150 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.922+0000 7f2d89d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d8c0d2f80 0x7f2d8c161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.922+0000 7f2d8ad9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2d8c0b1aa0 0x7f2d8c0d13e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41754/0 (socket says 172.21.15.47:41754) 2024-03-23T09:42:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.922+0000 7f2d8ad9d640 1 -- 172.21.15.47:0/2788868096 learned_addr learned my addr 172.21.15.47:0/2788868096 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.922+0000 7f2d89d9b640 1 -- 172.21.15.47:0/2788868096 >> v1:172.21.15.175:6789/0 conn(0x7f2d8c15cc70 legacy=0x7f2d8c0d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.922+0000 7f2d89d9b640 1 -- 172.21.15.47:0/2788868096 >> v1:172.21.15.47:6789/0 conn(0x7f2d8c0b1aa0 legacy=0x7f2d8c0d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.922+0000 7f2d89d9b640 1 -- 172.21.15.47:0/2788868096 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c0c9770 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.922+0000 7f2d8959a640 1 -- 172.21.15.47:0/2788868096 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467600466 0 0) 0x7f2d8c0cd940 con 0x7f2d8c0b1aa0 2024-03-23T09:42:46.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.923+0000 7f2d89d9b640 1 --2- 172.21.15.47:0/2788868096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8c0d2f80 0x7f2d8c161210 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2d800097a0 tx=0x7f2d8002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6070e81a276a10af server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:46.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.923+0000 7f2d8959a640 1 -- 172.21.15.47:0/2788868096 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8002e070 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.923+0000 7f2d8959a640 1 -- 172.21.15.47:0/2788868096 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d800315a0 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.923+0000 7f2d8959a640 1 -- 172.21.15.47:0/2788868096 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80031930 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.924+0000 7f2d908b6640 1 -- 172.21.15.47:0/2788868096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8c0d2f80 msgr2=0x7f2d8c161210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.924+0000 7f2d908b6640 1 --2- 172.21.15.47:0/2788868096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8c0d2f80 0x7f2d8c161210 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2d800097a0 tx=0x7f2d8002b840 comp rx=0 tx=0).stop 2024-03-23T09:42:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.924+0000 7f2d908b6640 1 -- 172.21.15.47:0/2788868096 shutdown_connections 2024-03-23T09:42:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.924+0000 7f2d908b6640 1 --2- 172.21.15.47:0/2788868096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8c0d2f80 0x7f2d8c161210 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.924+0000 7f2d908b6640 1 -- 172.21.15.47:0/2788868096 >> 172.21.15.47:0/2788868096 conn(0x7f2d8c065df0 msgr2=0x7f2d8c066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.924+0000 7f2d908b6640 1 -- 172.21.15.47:0/2788868096 shutdown_connections 2024-03-23T09:42:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.924+0000 7f2d908b6640 1 -- 172.21.15.47:0/2788868096 wait complete. 2024-03-23T09:42:46.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.925+0000 7f2d908b6640 1 Processor -- start 2024-03-23T09:42:46.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.925+0000 7f2d908b6640 1 -- start start 2024-03-23T09:42:46.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d908b6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d8c0b1aa0 0x7f2d8c150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:46.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d908b6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8c0d2f80 0x7f2d8c0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d908b6640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d8c15cc70 0x7f2d8c0c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d908b6640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0b45e0 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d908b6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0c78c0 con 0x7f2d8c0b1aa0 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d908b6640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0c7020 con 0x7f2d8c15cc70 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d89d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8c0d2f80 0x7f2d8c0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d89d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8c0d2f80 0x7f2d8c0c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39746/0 (socket says 172.21.15.47:39746) 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d89d9b640 1 -- 172.21.15.47:0/4232073721 learned_addr learned my addr 172.21.15.47:0/4232073721 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d8ad9d640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d8c15cc70 0x7f2d8c0c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d8a59c640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d8c0b1aa0 0x7f2d8c150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d89d9b640 1 -- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d8c15cc70 msgr2=0x7f2d8c0c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d89d9b640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d8c15cc70 0x7f2d8c0c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d89d9b640 1 -- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d8c0b1aa0 msgr2=0x7f2d8c150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d89d9b640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d8c0b1aa0 0x7f2d8c150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d89d9b640 1 -- 172.21.15.47:0/4232073721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d80009400 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.926+0000 7f2d8ad9d640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d8c15cc70 0x7f2d8c0c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.927+0000 7f2d89d9b640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8c0d2f80 0x7f2d8c0c7c00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2d80034040 tx=0x7f2d800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.927+0000 7f2d7affd640 1 -- 172.21.15.47:0/4232073721 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80032900 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.927+0000 7f2d7affd640 1 -- 172.21.15.47:0/4232073721 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d80032a60 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.927+0000 7f2d7affd640 1 -- 172.21.15.47:0/4232073721 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80032e10 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.927+0000 7f2d908b6640 1 -- 172.21.15.47:0/4232073721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c0c7300 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.927+0000 7f2d908b6640 1 -- 172.21.15.47:0/4232073721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c0b2310 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.928+0000 7f2d908b6640 1 -- 172.21.15.47:0/4232073721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8c072360 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.928+0000 7f2d7affd640 1 -- 172.21.15.47:0/4232073721 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2d8003b3f0 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.929+0000 7f2d7affd640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d6c06ac30 0x7f2d6c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:46.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.929+0000 7f2d7affd640 1 -- 172.21.15.47:0/4232073721 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2d800ab220 con 0x7f2d8c0d2f80 2024-03-23T09:42:46.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.929+0000 7f2d8a59c640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d6c06ac30 0x7f2d6c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:46.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.929+0000 7f2d8a59c640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d6c06ac30 0x7f2d6c06d0f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2d84006fd0 tx=0x7f2d84008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:46.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:46.933+0000 7f2d7affd640 1 -- 172.21.15.47:0/4232073721 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2d800765e0 con 0x7f2d8c0d2f80 2024-03-23T09:42:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.112+0000 7f2d908b6640 1 -- 172.21.15.47:0/4232073721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2d8c0b2500 con 0x7f2d8c0d2f80 2024-03-23T09:42:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.792+0000 7f2d7affd640 1 -- 172.21.15.47:0/4232073721 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2d80079c00 con 0x7f2d8c0d2f80 2024-03-23T09:42:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-03-23T09:42:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.800+0000 7f2d908b6640 1 -- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d6c06ac30 msgr2=0x7f2d6c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.800+0000 7f2d908b6640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d6c06ac30 0x7f2d6c06d0f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2d84006fd0 tx=0x7f2d84008040 comp rx=0 tx=0).stop 2024-03-23T09:42:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.800+0000 7f2d908b6640 1 -- 172.21.15.47:0/4232073721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8c0d2f80 msgr2=0x7f2d8c0c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.800+0000 7f2d908b6640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8c0d2f80 0x7f2d8c0c7c00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2d80034040 tx=0x7f2d800046c0 comp rx=0 tx=0).stop 2024-03-23T09:42:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.800+0000 7f2d908b6640 1 -- 172.21.15.47:0/4232073721 shutdown_connections 2024-03-23T09:42:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.800+0000 7f2d908b6640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d6c06ac30 0x7f2d6c06d0f0 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2d84006fd0 tx=0x7f2d84008040 comp rx=0 tx=0).stop 2024-03-23T09:42:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.800+0000 7f2d908b6640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d8c15cc70 0x7f2d8c0c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.800+0000 7f2d908b6640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8c0d2f80 0x7f2d8c0c7c00 secure :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2d80034040 tx=0x7f2d800046c0 comp rx=0 tx=0).stop 2024-03-23T09:42:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.800+0000 7f2d908b6640 1 --2- 172.21.15.47:0/4232073721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d8c0b1aa0 0x7f2d8c150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.800+0000 7f2d908b6640 1 -- 172.21.15.47:0/4232073721 >> 172.21.15.47:0/4232073721 conn(0x7f2d8c065df0 msgr2=0x7f2d8c0661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:47.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.802+0000 7f2d908b6640 1 -- 172.21.15.47:0/4232073721 shutdown_connections 2024-03-23T09:42:47.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.802+0000 7f2d908b6640 1 -- 172.21.15.47:0/4232073721 wait complete. 2024-03-23T09:42:47.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T09:42:47.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.929+0000 7fd417f38640 1 Processor -- start 2024-03-23T09:42:47.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.929+0000 7fd417f38640 1 -- start start 2024-03-23T09:42:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.930+0000 7fd417f38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ca30 0x7fd410160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.930+0000 7fd417f38640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd4100cde40 0x7fd4100d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.930+0000 7fd417f38640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd410161600 con 0x7fd41015ce10 2024-03-23T09:42:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.930+0000 7fd417f38640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd410161040 con 0x7fd4100cde40 2024-03-23T09:42:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.930+0000 7fd417f38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd410161320 con 0x7fd41015ca30 2024-03-23T09:42:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.930+0000 7fd4154d3640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd41015ce10 0x7fd4100cd730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41756/0 (socket says 172.21.15.47:41756) 2024-03-23T09:42:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.930+0000 7fd4154d3640 1 -- 172.21.15.47:0/1195908078 learned_addr learned my addr 172.21.15.47:0/1195908078 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.930+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372126947 0 0) 0x7fd410161600 con 0x7fd41015ce10 2024-03-23T09:42:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.930+0000 7fd415cd4640 1 --2- 172.21.15.47:0/1195908078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ca30 0x7fd410160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.930+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3f8003520 con 0x7fd41015ce10 2024-03-23T09:42:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.930+0000 7fd4164d5640 1 --2- 172.21.15.47:0/1195908078 >> v2:172.21.15.175:3300/0 conn(0x7fd4100cde40 0x7fd4100d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 212682090 0 0) 0x7fd3f8003520 con 0x7fd41015ce10 2024-03-23T09:42:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ca30 msgr2=0x7fd410160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 --2- 172.21.15.47:0/1195908078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ca30 0x7fd410160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 >> v2:172.21.15.175:3300/0 conn(0x7fd4100cde40 msgr2=0x7fd4100d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 --2- 172.21.15.47:0/1195908078 >> v2:172.21.15.175:3300/0 conn(0x7fd4100cde40 0x7fd4100d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4100d0dd0 con 0x7fd41015ce10 2024-03-23T09:42:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fd404001670 con 0x7fd41015ce10 2024-03-23T09:42:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 >> v1:172.21.15.47:6789/0 conn(0x7fd41015ce10 legacy=0x7fd4100cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 --2- 172.21.15.47:0/1195908078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f8003970 0x7fd3f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 --2- 172.21.15.47:0/1195908078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f80047c0 0x7fd3f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 --2- 172.21.15.47:0/1195908078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ca30 0x7fd3f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd410161040 con 0x7fd3f80047c0 2024-03-23T09:42:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd410161320 con 0x7fd3f8003970 2024-03-23T09:42:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f800b980 con 0x7fd41015ca30 2024-03-23T09:42:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd4154d3640 1 --2- 172.21.15.47:0/1195908078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f80047c0 0x7fd3f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd415cd4640 1 --2- 172.21.15.47:0/1195908078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f8003970 0x7fd3f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd4164d5640 1 --2- 172.21.15.47:0/1195908078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ca30 0x7fd3f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd4154d3640 1 -- 172.21.15.47:0/1195908078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ca30 msgr2=0x7fd3f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd4154d3640 1 --2- 172.21.15.47:0/1195908078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ca30 0x7fd3f8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd4154d3640 1 -- 172.21.15.47:0/1195908078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f8003970 msgr2=0x7fd3f8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd4154d3640 1 --2- 172.21.15.47:0/1195908078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f8003970 0x7fd3f8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:47.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd4154d3640 1 -- 172.21.15.47:0/1195908078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f8003520 con 0x7fd3f80047c0 2024-03-23T09:42:47.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.931+0000 7fd4164d5640 1 --2- 172.21.15.47:0/1195908078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ca30 0x7fd3f8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:47.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.932+0000 7fd4154d3640 1 --2- 172.21.15.47:0/1195908078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f80047c0 0x7fd3f8004c40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd41015d210 tx=0x7fd3f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf6502f8a3c2da6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:47.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.932+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404005ad0 con 0x7fd3f80047c0 2024-03-23T09:42:47.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.932+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd404005c30 con 0x7fd3f80047c0 2024-03-23T09:42:47.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.932+0000 7fd414cd2640 1 -- 172.21.15.47:0/1195908078 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40402c890 con 0x7fd3f80047c0 2024-03-23T09:42:47.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.933+0000 7fd417f38640 1 -- 172.21.15.47:0/1195908078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f80047c0 msgr2=0x7fd3f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:47.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.933+0000 7fd417f38640 1 --2- 172.21.15.47:0/1195908078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f80047c0 0x7fd3f8004c40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd41015d210 tx=0x7fd3f80057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:47.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.933+0000 7fd4164d5640 1 -- 172.21.15.47:0/1195908078 reap_dead start 2024-03-23T09:42:47.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.933+0000 7fd417f38640 1 -- 172.21.15.47:0/1195908078 shutdown_connections 2024-03-23T09:42:47.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.933+0000 7fd417f38640 1 -- 172.21.15.47:0/1195908078 >> 172.21.15.47:0/1195908078 conn(0x7fd41006b160 msgr2=0x7fd41006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:47.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.933+0000 7fd417f38640 1 -- 172.21.15.47:0/1195908078 shutdown_connections 2024-03-23T09:42:47.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.933+0000 7fd417f38640 1 -- 172.21.15.47:0/1195908078 wait complete. 2024-03-23T09:42:47.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.934+0000 7fd417f38640 1 Processor -- start 2024-03-23T09:42:47.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.934+0000 7fd417f38640 1 -- start start 2024-03-23T09:42:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd417f38640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f80047c0 0x7fd4100c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd417f38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ad70 0x7fd4101584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd417f38640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd410160320 0x7fd4101607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd417f38640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd410159ed0 con 0x7fd3f80047c0 2024-03-23T09:42:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd417f38640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4100c8a20 con 0x7fd410160320 2024-03-23T09:42:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd417f38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd410158e30 con 0x7fd41015ad70 2024-03-23T09:42:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd415cd4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f80047c0 0x7fd4100c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd415cd4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f80047c0 0x7fd4100c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39752/0 (socket says 172.21.15.47:39752) 2024-03-23T09:42:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd415cd4640 1 -- 172.21.15.47:0/1542652541 learned_addr learned my addr 172.21.15.47:0/1542652541 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd4154d3640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ad70 0x7fd4101584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd4164d5640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd410160320 0x7fd4101607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd415cd4640 1 -- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ad70 msgr2=0x7fd4101584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd415cd4640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ad70 0x7fd4101584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd415cd4640 1 -- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd410160320 msgr2=0x7fd4101607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd415cd4640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd410160320 0x7fd4101607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd415cd4640 1 -- 172.21.15.47:0/1542652541 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd410159110 con 0x7fd3f80047c0 2024-03-23T09:42:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd4154d3640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ad70 0x7fd4101584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd415cd4640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f80047c0 0x7fd4100c84e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd4080118e0 tx=0x7fd408011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd40e7fc640 1 -- 172.21.15.47:0/1542652541 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40800faf0 con 0x7fd3f80047c0 2024-03-23T09:42:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd40e7fc640 1 -- 172.21.15.47:0/1542652541 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40800fc50 con 0x7fd3f80047c0 2024-03-23T09:42:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd417f38640 1 -- 172.21.15.47:0/1542652541 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4101593f0 con 0x7fd3f80047c0 2024-03-23T09:42:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd40e7fc640 1 -- 172.21.15.47:0/1542652541 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408014440 con 0x7fd3f80047c0 2024-03-23T09:42:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.935+0000 7fd417f38640 1 -- 172.21.15.47:0/1542652541 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4101599b0 con 0x7fd3f80047c0 2024-03-23T09:42:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.937+0000 7fd417f38640 1 -- 172.21.15.47:0/1542652541 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd410072360 con 0x7fd3f80047c0 2024-03-23T09:42:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.937+0000 7fd40e7fc640 1 -- 172.21.15.47:0/1542652541 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd4080145a0 con 0x7fd3f80047c0 2024-03-23T09:42:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.937+0000 7fd40e7fc640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd3fc06ac00 0x7fd3fc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.938+0000 7fd40e7fc640 1 -- 172.21.15.47:0/1542652541 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd408089b00 con 0x7fd3f80047c0 2024-03-23T09:42:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.938+0000 7fd4154d3640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd3fc06ac00 0x7fd3fc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.938+0000 7fd4154d3640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd3fc06ac00 0x7fd3fc06d0c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd404034000 tx=0x7fd404007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:47.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:47.942+0000 7fd40e7fc640 1 -- 172.21.15.47:0/1542652541 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd408054d40 con 0x7fd3f80047c0 2024-03-23T09:42:48.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.164+0000 7fd417f38640 1 -- 172.21.15.47:0/1542652541 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd41015e390 con 0x7fd3f80047c0 2024-03-23T09:42:48.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.168+0000 7fd40e7fc640 1 -- 172.21.15.47:0/1542652541 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v16) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd408058360 con 0x7fd3f80047c0 2024-03-23T09:42:48.168 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 6 and data pool 7 2024-03-23T09:42:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.173+0000 7fd417f38640 1 -- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd3fc06ac00 msgr2=0x7fd3fc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.173+0000 7fd417f38640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd3fc06ac00 0x7fd3fc06d0c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd404034000 tx=0x7fd404007000 comp rx=0 tx=0).stop 2024-03-23T09:42:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.173+0000 7fd417f38640 1 -- 172.21.15.47:0/1542652541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f80047c0 msgr2=0x7fd4100c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.173+0000 7fd417f38640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f80047c0 0x7fd4100c84e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd4080118e0 tx=0x7fd408011da0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.174+0000 7fd417f38640 1 -- 172.21.15.47:0/1542652541 shutdown_connections 2024-03-23T09:42:48.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.174+0000 7fd417f38640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd3fc06ac00 0x7fd3fc06d0c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.174+0000 7fd417f38640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd410160320 0x7fd4101607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.174+0000 7fd417f38640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd41015ad70 0x7fd4101584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.174+0000 7fd417f38640 1 --2- 172.21.15.47:0/1542652541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f80047c0 0x7fd4100c84e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.174+0000 7fd417f38640 1 -- 172.21.15.47:0/1542652541 >> 172.21.15.47:0/1542652541 conn(0x7fd41006b160 msgr2=0x7fd41015bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.178+0000 7fd417f38640 1 -- 172.21.15.47:0/1542652541 shutdown_connections 2024-03-23T09:42:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.179+0000 7fd417f38640 1 -- 172.21.15.47:0/1542652541 wait complete. 2024-03-23T09:42:48.199 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:42:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.299+0000 7f5f81f1b640 1 Processor -- start 2024-03-23T09:42:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.299+0000 7f5f81f1b640 1 -- start start 2024-03-23T09:42:48.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.300+0000 7f5f81f1b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c15a8d0 0x7f5f7c0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.300+0000 7f5f81f1b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7c0cc610 con 0x7f5f7c0cb730 2024-03-23T09:42:48.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.300+0000 7f5f81f1b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7c0b0700 con 0x7f5f7c0cf360 2024-03-23T09:42:48.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.300+0000 7f5f81f1b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c0c41c0 con 0x7f5f7c15a8d0 2024-03-23T09:42:48.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.300+0000 7f5f7affd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5f7c0cb730 0x7f5f7c0cec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41760/0 (socket says 172.21.15.47:41760) 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.300+0000 7f5f7affd640 1 -- 172.21.15.47:0/2255118097 learned_addr learned my addr 172.21.15.47:0/2255118097 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.300+0000 7f5f7b7fe640 1 --2- 172.21.15.47:0/2255118097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c15a8d0 0x7f5f7c0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632536742 0 0) 0x7f5f7c0cc610 con 0x7f5f7c0cb730 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f64003520 con 0x7f5f7c0cb730 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687246161 0 0) 0x7f5f7c0b0700 con 0x7f5f7c0cf360 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f7c0cc610 con 0x7f5f7c0cf360 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1024712 0 0) 0x7f5f64003520 con 0x7f5f7c0cb730 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c15a8d0 msgr2=0x7f5f7c0cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 --2- 172.21.15.47:0/2255118097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c15a8d0 0x7f5f7c0cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 >> v1:172.21.15.175:6789/0 conn(0x7f5f7c0cf360 legacy=0x7f5f7c0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0cbd70 con 0x7f5f7c0cb730 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f5f70001670 con 0x7f5f7c0cb730 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 >> v1:172.21.15.47:6789/0 conn(0x7f5f7c0cb730 legacy=0x7f5f7c0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 --2- 172.21.15.47:0/2255118097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f64003b30 0x7f5f64003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 --2- 172.21.15.47:0/2255118097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c15a8d0 0x7f5f640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 --2- 172.21.15.47:0/2255118097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f64004d00 0x7f5f64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6400ba50 con 0x7f5f64004d00 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6400bbc0 con 0x7f5f64003b30 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6400bd80 con 0x7f5f7c15a8d0 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7bfff640 1 --2- 172.21.15.47:0/2255118097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f64004d00 0x7f5f64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7b7fe640 1 --2- 172.21.15.47:0/2255118097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f64003b30 0x7f5f64003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7affd640 1 --2- 172.21.15.47:0/2255118097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c15a8d0 0x7f5f640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7bfff640 1 -- 172.21.15.47:0/2255118097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c15a8d0 msgr2=0x7f5f640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7bfff640 1 --2- 172.21.15.47:0/2255118097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c15a8d0 0x7f5f640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7bfff640 1 -- 172.21.15.47:0/2255118097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f64003b30 msgr2=0x7f5f64003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7bfff640 1 --2- 172.21.15.47:0/2255118097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f64003b30 0x7f5f64003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.301+0000 7f5f7bfff640 1 -- 172.21.15.47:0/2255118097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f64003520 con 0x7f5f64004d00 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.302+0000 7f5f7bfff640 1 --2- 172.21.15.47:0/2255118097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f64004d00 0x7f5f64005180 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5f74001fd0 tx=0x7f5f640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccac03c4afaa06bb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.302+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74003b70 con 0x7f5f64004d00 2024-03-23T09:42:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.302+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f74003cd0 con 0x7f5f64004d00 2024-03-23T09:42:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.302+0000 7f5f7a7fc640 1 -- 172.21.15.47:0/2255118097 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f740052f0 con 0x7f5f64004d00 2024-03-23T09:42:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.303+0000 7f5f81f1b640 1 -- 172.21.15.47:0/2255118097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f64004d00 msgr2=0x7f5f64005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.303+0000 7f5f81f1b640 1 --2- 172.21.15.47:0/2255118097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f64004d00 0x7f5f64005180 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5f74001fd0 tx=0x7f5f640057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.303+0000 7f5f7bfff640 1 -- 172.21.15.47:0/2255118097 reap_dead start 2024-03-23T09:42:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.303+0000 7f5f81f1b640 1 -- 172.21.15.47:0/2255118097 shutdown_connections 2024-03-23T09:42:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.303+0000 7f5f81f1b640 1 -- 172.21.15.47:0/2255118097 >> 172.21.15.47:0/2255118097 conn(0x7f5f7c0d0670 msgr2=0x7f5f7c0d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.303+0000 7f5f81f1b640 1 -- 172.21.15.47:0/2255118097 shutdown_connections 2024-03-23T09:42:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.303+0000 7f5f81f1b640 1 -- 172.21.15.47:0/2255118097 wait complete. 2024-03-23T09:42:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.304+0000 7f5f81f1b640 1 Processor -- start 2024-03-23T09:42:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.304+0000 7f5f81f1b640 1 -- start start 2024-03-23T09:42:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f81f1b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f64004d00 0x7f5f7c0af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f81f1b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f7c15cf90 0x7f5f7c15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f81f1b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c1534c0 0x7f5f7c1e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f81f1b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c15bf40 con 0x7f5f7c15cf90 2024-03-23T09:42:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f81f1b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c15d700 con 0x7f5f64004d00 2024-03-23T09:42:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f81f1b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c153970 con 0x7f5f7c1534c0 2024-03-23T09:42:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f7affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f7c15cf90 0x7f5f7c15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f7affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f7c15cf90 0x7f5f7c15ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39770/0 (socket says 172.21.15.47:39770) 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f7affd640 1 -- 172.21.15.47:0/639129939 learned_addr learned my addr 172.21.15.47:0/639129939 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f7bfff640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c1534c0 0x7f5f7c1e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f7b7fe640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f64004d00 0x7f5f7c0af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f7affd640 1 -- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c1534c0 msgr2=0x7f5f7c1e68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f7affd640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c1534c0 0x7f5f7c1e68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f7affd640 1 -- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f64004d00 msgr2=0x7f5f7c0af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f7affd640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f64004d00 0x7f5f7c0af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f7affd640 1 -- 172.21.15.47:0/639129939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c15d420 con 0x7f5f7c15cf90 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f7bfff640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c1534c0 0x7f5f7c1e68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.305+0000 7f5f7affd640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f7c15cf90 0x7f5f7c15ba00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5f70007310 tx=0x7f5f700077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.306+0000 7f5f63fff640 1 -- 172.21.15.47:0/639129939 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000f8e0 con 0x7f5f7c15cf90 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.306+0000 7f5f63fff640 1 -- 172.21.15.47:0/639129939 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7000fa40 con 0x7f5f7c15cf90 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.306+0000 7f5f81f1b640 1 -- 172.21.15.47:0/639129939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7c0ae860 con 0x7f5f7c15cf90 2024-03-23T09:42:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.306+0000 7f5f63fff640 1 -- 172.21.15.47:0/639129939 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000fdb0 con 0x7f5f7c15cf90 2024-03-23T09:42:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.306+0000 7f5f81f1b640 1 -- 172.21.15.47:0/639129939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7c0aee20 con 0x7f5f7c15cf90 2024-03-23T09:42:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.307+0000 7f5f81f1b640 1 -- 172.21.15.47:0/639129939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7c0672f0 con 0x7f5f7c15cf90 2024-03-23T09:42:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.307+0000 7f5f63fff640 1 -- 172.21.15.47:0/639129939 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5f7001d460 con 0x7f5f7c15cf90 2024-03-23T09:42:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.307+0000 7f5f63fff640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5f6806ac30 0x7f5f6806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.307+0000 7f5f63fff640 1 -- 172.21.15.47:0/639129939 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f70089450 con 0x7f5f7c15cf90 2024-03-23T09:42:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.308+0000 7f5f7b7fe640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5f6806ac30 0x7f5f6806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.308+0000 7f5f7b7fe640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5f6806ac30 0x7f5f6806d0f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009b00 tx=0x7f5f6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.312+0000 7f5f63fff640 1 -- 172.21.15.47:0/639129939 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5f70054630 con 0x7f5f7c15cf90 2024-03-23T09:42:48.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.491+0000 7f5f81f1b640 1 -- 172.21.15.47:0/639129939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f7c0cdfe0 con 0x7f5f7c15cf90 2024-03-23T09:42:48.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.492+0000 7f5f63fff640 1 -- 172.21.15.47:0/639129939 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16058 (secure 0 0 0) 0x7f5f70057c50 con 0x7f5f7c15cf90 2024-03-23T09:42:48.492 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:48.492 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":45,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:48.165525+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:42:46.053713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-23T09:42:47.131279+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:42:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.494+0000 7f5f81f1b640 1 -- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5f6806ac30 msgr2=0x7f5f6806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.494+0000 7f5f81f1b640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5f6806ac30 0x7f5f6806d0f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009b00 tx=0x7f5f6c008040 comp rx=0 tx=0).stop 2024-03-23T09:42:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.494+0000 7f5f81f1b640 1 -- 172.21.15.47:0/639129939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f7c15cf90 msgr2=0x7f5f7c15ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.494+0000 7f5f81f1b640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f7c15cf90 0x7f5f7c15ba00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5f70007310 tx=0x7f5f700077d0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.494+0000 7f5f81f1b640 1 -- 172.21.15.47:0/639129939 shutdown_connections 2024-03-23T09:42:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.494+0000 7f5f81f1b640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5f6806ac30 0x7f5f6806d0f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.494+0000 7f5f81f1b640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f7c1534c0 0x7f5f7c1e68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.494+0000 7f5f81f1b640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f7c15cf90 0x7f5f7c15ba00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.494+0000 7f5f81f1b640 1 --2- 172.21.15.47:0/639129939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f64004d00 0x7f5f7c0af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.494+0000 7f5f81f1b640 1 -- 172.21.15.47:0/639129939 >> 172.21.15.47:0/639129939 conn(0x7f5f7c0d0670 msgr2=0x7f5f7c0d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.495+0000 7f5f81f1b640 1 -- 172.21.15.47:0/639129939 shutdown_connections 2024-03-23T09:42:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.495+0000 7f5f81f1b640 1 -- 172.21.15.47:0/639129939 wait complete. 2024-03-23T09:42:48.505 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:42:48.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.606+0000 7f11cb85c640 1 Processor -- start 2024-03-23T09:42:48.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.606+0000 7f11cb85c640 1 -- start start 2024-03-23T09:42:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11cb85c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f11c415cee0 0x7f11c40d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11cb85c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f11c40d23a0 0x7f11c40d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11cb85c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c40b8300 0x7f11c40d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11cb85c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c40d1300 con 0x7f11c415cee0 2024-03-23T09:42:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11cb85c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c40d15e0 con 0x7f11c40d23a0 2024-03-23T09:42:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11cb85c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40d18c0 con 0x7f11c40b8300 2024-03-23T09:42:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11c95f8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f11c415cee0 0x7f11c40d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11c95f8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f11c415cee0 0x7f11c40d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39780/0 (socket says 172.21.15.47:39780) 2024-03-23T09:42:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11c95f8640 1 -- 172.21.15.47:0/1193745215 learned_addr learned my addr 172.21.15.47:0/1193745215 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11c8df7640 1 --2- 172.21.15.47:0/1193745215 >> v2:172.21.15.175:3300/0 conn(0x7f11c40d23a0 0x7f11c40d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11c9df9640 1 --2- 172.21.15.47:0/1193745215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c40b8300 0x7f11c40d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11c95f8640 1 -- 172.21.15.47:0/1193745215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c40b8300 msgr2=0x7f11c40d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11c95f8640 1 --2- 172.21.15.47:0/1193745215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c40b8300 0x7f11c40d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11c95f8640 1 -- 172.21.15.47:0/1193745215 >> v2:172.21.15.175:3300/0 conn(0x7f11c40d23a0 msgr2=0x7f11c40d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11c95f8640 1 --2- 172.21.15.47:0/1193745215 >> v2:172.21.15.175:3300/0 conn(0x7f11c40d23a0 0x7f11c40d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11c95f8640 1 -- 172.21.15.47:0/1193745215 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40d1ba0 con 0x7f11c415cee0 2024-03-23T09:42:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11c8df7640 1 --2- 172.21.15.47:0/1193745215 >> v2:172.21.15.175:3300/0 conn(0x7f11c40d23a0 0x7f11c40d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.607+0000 7f11c9df9640 1 --2- 172.21.15.47:0/1193745215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c40b8300 0x7f11c40d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.608+0000 7f11c95f8640 1 --2- 172.21.15.47:0/1193745215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11c415cee0 0x7f11c40d0990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f11c40cf8d0 tx=0x7f11c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ac4bbe3891c012 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.608+0000 7f11bbfff640 1 -- 172.21.15.47:0/1193745215 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0031870 con 0x7f11c415cee0 2024-03-23T09:42:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.608+0000 7f11bbfff640 1 -- 172.21.15.47:0/1193745215 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c00319d0 con 0x7f11c415cee0 2024-03-23T09:42:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.608+0000 7f11bbfff640 1 -- 172.21.15.47:0/1193745215 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0031d60 con 0x7f11c415cee0 2024-03-23T09:42:48.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.609+0000 7f11cb85c640 1 -- 172.21.15.47:0/1193745215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11c415cee0 msgr2=0x7f11c40d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.609+0000 7f11cb85c640 1 --2- 172.21.15.47:0/1193745215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11c415cee0 0x7f11c40d0990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f11c40cf8d0 tx=0x7f11c002b910 comp rx=0 tx=0).stop 2024-03-23T09:42:48.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.609+0000 7f11cb85c640 1 -- 172.21.15.47:0/1193745215 shutdown_connections 2024-03-23T09:42:48.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.609+0000 7f11cb85c640 1 --2- 172.21.15.47:0/1193745215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c40b8300 0x7f11c40d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.609+0000 7f11cb85c640 1 --2- 172.21.15.47:0/1193745215 >> v2:172.21.15.175:3300/0 conn(0x7f11c40d23a0 0x7f11c40d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.609+0000 7f11cb85c640 1 --2- 172.21.15.47:0/1193745215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11c415cee0 0x7f11c40d0990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.609+0000 7f11cb85c640 1 -- 172.21.15.47:0/1193745215 >> 172.21.15.47:0/1193745215 conn(0x7f11c4065df0 msgr2=0x7f11c4066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.609+0000 7f11cb85c640 1 -- 172.21.15.47:0/1193745215 shutdown_connections 2024-03-23T09:42:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.609+0000 7f11cb85c640 1 -- 172.21.15.47:0/1193745215 wait complete. 2024-03-23T09:42:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.610+0000 7f11cb85c640 1 Processor -- start 2024-03-23T09:42:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.610+0000 7f11cb85c640 1 -- start start 2024-03-23T09:42:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11cb85c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c40b8300 0x7f11c4151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11cb85c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11c40d23a0 0x7f11c40cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11cb85c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c415cee0 0x7f11c415a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11cb85c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40cb590 con 0x7f11c40d23a0 2024-03-23T09:42:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11cb85c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40cb870 con 0x7f11c415cee0 2024-03-23T09:42:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11cb85c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40cbb50 con 0x7f11c40b8300 2024-03-23T09:42:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11c8df7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11c40d23a0 0x7f11c40cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11c8df7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11c40d23a0 0x7f11c40cac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39788/0 (socket says 172.21.15.47:39788) 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11c8df7640 1 -- 172.21.15.47:0/2765193519 learned_addr learned my addr 172.21.15.47:0/2765193519 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11c95f8640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c40b8300 0x7f11c4151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11c9df9640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c415cee0 0x7f11c415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11c8df7640 1 -- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c40b8300 msgr2=0x7f11c4151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11c8df7640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c40b8300 0x7f11c4151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11c8df7640 1 -- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c415cee0 msgr2=0x7f11c415a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11c8df7640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c415cee0 0x7f11c415a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11c8df7640 1 -- 172.21.15.47:0/2765193519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c00094d0 con 0x7f11c40d23a0 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11c8df7640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11c40d23a0 0x7f11c40cac40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f11b400e810 tx=0x7f11b400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11b9ffb640 1 -- 172.21.15.47:0/2765193519 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4004030 con 0x7f11c40d23a0 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11b9ffb640 1 -- 172.21.15.47:0/2765193519 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b4004190 con 0x7f11c40d23a0 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11cb85c640 1 -- 172.21.15.47:0/2765193519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c40cbe30 con 0x7f11c40d23a0 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.611+0000 7f11b9ffb640 1 -- 172.21.15.47:0/2765193519 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4004520 con 0x7f11c40d23a0 2024-03-23T09:42:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.612+0000 7f11cb85c640 1 -- 172.21.15.47:0/2765193519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c40cb180 con 0x7f11c40d23a0 2024-03-23T09:42:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.613+0000 7f11cb85c640 1 -- 172.21.15.47:0/2765193519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c4072360 con 0x7f11c40d23a0 2024-03-23T09:42:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.613+0000 7f11b9ffb640 1 -- 172.21.15.47:0/2765193519 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f11b400c6d0 con 0x7f11c40d23a0 2024-03-23T09:42:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.613+0000 7f11b9ffb640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f11a806ac00 0x7f11a806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.613+0000 7f11b9ffb640 1 -- 172.21.15.47:0/2765193519 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11b4088ef0 con 0x7f11c40d23a0 2024-03-23T09:42:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.613+0000 7f11c95f8640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f11a806ac00 0x7f11a806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.614+0000 7f11c95f8640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f11a806ac00 0x7f11a806d0c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f11c0036000 tx=0x7f11c003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.617+0000 7f11b9ffb640 1 -- 172.21.15.47:0/2765193519 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f11b4054180 con 0x7f11c40d23a0 2024-03-23T09:42:48.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.797+0000 7f11cb85c640 1 -- 172.21.15.47:0/2765193519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11c4066be0 con 0x7f11c40d23a0 2024-03-23T09:42:48.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.798+0000 7f11b9ffb640 1 -- 172.21.15.47:0/2765193519 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16058 (secure 0 0 0) 0x7f11b40577a0 con 0x7f11c40d23a0 2024-03-23T09:42:48.798 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:48.798 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":45,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:48.165525+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:42:46.053713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-23T09:42:47.131279+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:42:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.800+0000 7f11cb85c640 1 -- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f11a806ac00 msgr2=0x7f11a806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.800+0000 7f11cb85c640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f11a806ac00 0x7f11a806d0c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f11c0036000 tx=0x7f11c003c040 comp rx=0 tx=0).stop 2024-03-23T09:42:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.800+0000 7f11cb85c640 1 -- 172.21.15.47:0/2765193519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11c40d23a0 msgr2=0x7f11c40cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.800+0000 7f11cb85c640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11c40d23a0 0x7f11c40cac40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f11b400e810 tx=0x7f11b400ecd0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.800+0000 7f11cb85c640 1 -- 172.21.15.47:0/2765193519 shutdown_connections 2024-03-23T09:42:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.800+0000 7f11cb85c640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f11a806ac00 0x7f11a806d0c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.800+0000 7f11cb85c640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c415cee0 0x7f11c415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.800+0000 7f11cb85c640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11c40d23a0 0x7f11c40cac40 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.800+0000 7f11cb85c640 1 --2- 172.21.15.47:0/2765193519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c40b8300 0x7f11c4151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.800+0000 7f11cb85c640 1 -- 172.21.15.47:0/2765193519 >> 172.21.15.47:0/2765193519 conn(0x7f11c4065df0 msgr2=0x7f11c415b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.800+0000 7f11cb85c640 1 -- 172.21.15.47:0/2765193519 shutdown_connections 2024-03-23T09:42:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.800+0000 7f11cb85c640 1 -- 172.21.15.47:0/2765193519 wait complete. 2024-03-23T09:42:48.811 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T09:42:48.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.915+0000 7fee28573640 1 Processor -- start 2024-03-23T09:42:48.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.915+0000 7fee28573640 1 -- start start 2024-03-23T09:42:48.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee28573640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fee2015ca30 0x7fee20160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee28573640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee200cde40 0x7fee200d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee28573640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee20161600 con 0x7fee2015ce10 2024-03-23T09:42:48.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee28573640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee20161040 con 0x7fee2015ca30 2024-03-23T09:42:48.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee28573640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee20161320 con 0x7fee200cde40 2024-03-23T09:42:48.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee25b0e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fee2015ce10 0x7fee200cd730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41770/0 (socket says 172.21.15.47:41770) 2024-03-23T09:42:48.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee25b0e640 1 -- 172.21.15.47:0/1075161325 learned_addr learned my addr 172.21.15.47:0/1075161325 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:48.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee26b10640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee200cde40 0x7fee200d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2630f640 1 --2- 172.21.15.47:0/1075161325 >> v2:172.21.15.175:3300/0 conn(0x7fee2015ca30 0x7fee20160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924574159 0 0) 0x7fee20161600 con 0x7fee2015ce10 2024-03-23T09:42:48.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee0c003520 con 0x7fee2015ce10 2024-03-23T09:42:48.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4232480712 0 0) 0x7fee0c003520 con 0x7fee2015ce10 2024-03-23T09:42:48.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee200cde40 msgr2=0x7fee200d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:48.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee200cde40 0x7fee200d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 >> v2:172.21.15.175:3300/0 conn(0x7fee2015ca30 msgr2=0x7fee20160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 --2- 172.21.15.47:0/1075161325 >> v2:172.21.15.175:3300/0 conn(0x7fee2015ca30 0x7fee20160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee200d0dd0 con 0x7fee2015ce10 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fee10001670 con 0x7fee2015ce10 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 >> v1:172.21.15.47:6789/0 conn(0x7fee2015ce10 legacy=0x7fee200cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee0c003970 0x7fee0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c0047c0 0x7fee0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee200cde40 0x7fee0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee20161040 con 0x7fee0c003970 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee20161320 con 0x7fee0c0047c0 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.916+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c00b980 con 0x7fee200cde40 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.917+0000 7fee2630f640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee0c003970 0x7fee0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.917+0000 7fee25b0e640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c0047c0 0x7fee0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.917+0000 7fee26b10640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee200cde40 0x7fee0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.917+0000 7fee2630f640 1 -- 172.21.15.47:0/1075161325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee200cde40 msgr2=0x7fee0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:48.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.917+0000 7fee2630f640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee200cde40 0x7fee0c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.917+0000 7fee2630f640 1 -- 172.21.15.47:0/1075161325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c0047c0 msgr2=0x7fee0c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:48.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.917+0000 7fee2630f640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c0047c0 0x7fee0c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.917+0000 7fee2630f640 1 -- 172.21.15.47:0/1075161325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c003520 con 0x7fee0c003970 2024-03-23T09:42:48.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.917+0000 7fee25b0e640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c0047c0 0x7fee0c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:48.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.917+0000 7fee26b10640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee200cde40 0x7fee0c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:48.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.918+0000 7fee2630f640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee0c003970 0x7fee0c003d90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009ac0 tx=0x7fee0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6e65b1e1f70ce server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:48.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.918+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c02c710 con 0x7fee0c003970 2024-03-23T09:42:48.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.918+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c02c870 con 0x7fee0c003970 2024-03-23T09:42:48.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.918+0000 7fee2530d640 1 -- 172.21.15.47:0/1075161325 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c02cc00 con 0x7fee0c003970 2024-03-23T09:42:48.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.918+0000 7fee28573640 1 -- 172.21.15.47:0/1075161325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee0c003970 msgr2=0x7fee0c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.918+0000 7fee28573640 1 --2- 172.21.15.47:0/1075161325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee0c003970 0x7fee0c003d90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009ac0 tx=0x7fee0c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.918+0000 7fee26b10640 1 -- 172.21.15.47:0/1075161325 reap_dead start 2024-03-23T09:42:48.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.918+0000 7fee28573640 1 -- 172.21.15.47:0/1075161325 shutdown_connections 2024-03-23T09:42:48.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.918+0000 7fee28573640 1 -- 172.21.15.47:0/1075161325 >> 172.21.15.47:0/1075161325 conn(0x7fee2006b160 msgr2=0x7fee2006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:48.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.918+0000 7fee28573640 1 -- 172.21.15.47:0/1075161325 shutdown_connections 2024-03-23T09:42:48.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.918+0000 7fee28573640 1 -- 172.21.15.47:0/1075161325 wait complete. 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.920+0000 7fee28573640 1 Processor -- start 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.920+0000 7fee28573640 1 -- start start 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.920+0000 7fee28573640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c003970 0x7fee2015d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.920+0000 7fee28573640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee201f22a0 0x7fee20154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.920+0000 7fee28573640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2015ef60 0x7fee200cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.920+0000 7fee28573640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2015a0c0 con 0x7fee201f22a0 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.920+0000 7fee28573640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee201f2780 con 0x7fee0c003970 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.920+0000 7fee28573640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee201f2a60 con 0x7fee2015ef60 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.920+0000 7fee25b0e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee201f22a0 0x7fee20154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.920+0000 7fee25b0e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee201f22a0 0x7fee20154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39802/0 (socket says 172.21.15.47:39802) 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.920+0000 7fee25b0e640 1 -- 172.21.15.47:0/3409808757 learned_addr learned my addr 172.21.15.47:0/3409808757 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.921+0000 7fee2630f640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c003970 0x7fee2015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.921+0000 7fee26b10640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2015ef60 0x7fee200cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.921+0000 7fee25b0e640 1 -- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2015ef60 msgr2=0x7fee200cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.921+0000 7fee25b0e640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2015ef60 0x7fee200cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.921+0000 7fee25b0e640 1 -- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c003970 msgr2=0x7fee2015d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:48.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.921+0000 7fee25b0e640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c003970 0x7fee2015d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:48.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.921+0000 7fee25b0e640 1 -- 172.21.15.47:0/3409808757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee201f2d40 con 0x7fee201f22a0 2024-03-23T09:42:48.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.921+0000 7fee25b0e640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee201f22a0 0x7fee20154010 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fee1000e7d0 tx=0x7fee1000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:48.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.921+0000 7fee16ffd640 1 -- 172.21.15.47:0/3409808757 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1000c870 con 0x7fee201f22a0 2024-03-23T09:42:48.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.921+0000 7fee16ffd640 1 -- 172.21.15.47:0/3409808757 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1000c9d0 con 0x7fee201f22a0 2024-03-23T09:42:48.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.921+0000 7fee28573640 1 -- 172.21.15.47:0/3409808757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee201f3020 con 0x7fee201f22a0 2024-03-23T09:42:48.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.921+0000 7fee16ffd640 1 -- 172.21.15.47:0/3409808757 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1000cd40 con 0x7fee201f22a0 2024-03-23T09:42:48.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.921+0000 7fee28573640 1 -- 172.21.15.47:0/3409808757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee20154550 con 0x7fee201f22a0 2024-03-23T09:42:48.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.922+0000 7fee28573640 1 -- 172.21.15.47:0/3409808757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee20072360 con 0x7fee201f22a0 2024-03-23T09:42:48.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.922+0000 7fee16ffd640 1 -- 172.21.15.47:0/3409808757 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fee1001d460 con 0x7fee201f22a0 2024-03-23T09:42:48.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.923+0000 7fee16ffd640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee0406ac30 0x7fee0406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:48.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.923+0000 7fee16ffd640 1 -- 172.21.15.47:0/3409808757 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee10089160 con 0x7fee201f22a0 2024-03-23T09:42:48.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.923+0000 7fee2630f640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee0406ac30 0x7fee0406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:48.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.923+0000 7fee2630f640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee0406ac30 0x7fee0406d0f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fee1c036000 tx=0x7fee1c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:48.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:48.927+0000 7fee16ffd640 1 -- 172.21.15.47:0/3409808757 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fee10054480 con 0x7fee201f22a0 2024-03-23T09:42:49.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.143+0000 7fee28573640 1 -- 172.21.15.47:0/3409808757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fee2015c290 con 0x7fee201f22a0 2024-03-23T09:42:49.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.172+0000 7fee16ffd640 1 -- 172.21.15.47:0/3409808757 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fee10057aa0 con 0x7fee201f22a0 2024-03-23T09:42:49.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.174+0000 7fee28573640 1 -- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee0406ac30 msgr2=0x7fee0406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:49.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.174+0000 7fee28573640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee0406ac30 0x7fee0406d0f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fee1c036000 tx=0x7fee1c030000 comp rx=0 tx=0).stop 2024-03-23T09:42:49.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.174+0000 7fee28573640 1 -- 172.21.15.47:0/3409808757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee201f22a0 msgr2=0x7fee20154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:49.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.174+0000 7fee28573640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee201f22a0 0x7fee20154010 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fee1000e7d0 tx=0x7fee1000ec90 comp rx=0 tx=0).stop 2024-03-23T09:42:49.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.174+0000 7fee28573640 1 -- 172.21.15.47:0/3409808757 shutdown_connections 2024-03-23T09:42:49.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.174+0000 7fee28573640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee0406ac30 0x7fee0406d0f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:49.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.174+0000 7fee28573640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2015ef60 0x7fee200cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:49.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.174+0000 7fee28573640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee201f22a0 0x7fee20154010 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:49.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.174+0000 7fee28573640 1 --2- 172.21.15.47:0/3409808757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c003970 0x7fee2015d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:49.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.174+0000 7fee28573640 1 -- 172.21.15.47:0/3409808757 >> 172.21.15.47:0/3409808757 conn(0x7fee2006b160 msgr2=0x7fee200d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:49.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.174+0000 7fee28573640 1 -- 172.21.15.47:0/3409808757 shutdown_connections 2024-03-23T09:42:49.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.174+0000 7fee28573640 1 -- 172.21.15.47:0/3409808757 wait complete. 2024-03-23T09:42:49.184 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-23T09:42:49.184 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-23T09:42:49.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.288+0000 7efe8b348640 1 Processor -- start 2024-03-23T09:42:49.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.288+0000 7efe8b348640 1 -- start start 2024-03-23T09:42:49.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.289+0000 7efe8b348640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efe840aedf0 0x7efe84069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:49.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.289+0000 7efe8b348640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe840cf400 con 0x7efe8406a220 2024-03-23T09:42:49.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.289+0000 7efe8b348640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe840cf6e0 con 0x7efe8415a660 2024-03-23T09:42:49.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.289+0000 7efe8b348640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe840d04a0 con 0x7efe840aedf0 2024-03-23T09:42:49.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.289+0000 7efe898e5640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efe8406a220 0x7efe8406eb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41776/0 (socket says 172.21.15.47:41776) 2024-03-23T09:42:49.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.289+0000 7efe898e5640 1 -- 172.21.15.47:0/4184685917 learned_addr learned my addr 172.21.15.47:0/4184685917 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:49.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.289+0000 7efe888e3640 1 --2- 172.21.15.47:0/4184685917 >> v2:172.21.15.175:3300/0 conn(0x7efe840aedf0 0x7efe84069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:49.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.289+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646778774 0 0) 0x7efe840cf400 con 0x7efe8406a220 2024-03-23T09:42:49.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.289+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe6c003520 con 0x7efe8406a220 2024-03-23T09:42:49.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688836965 0 0) 0x7efe840cf6e0 con 0x7efe8415a660 2024-03-23T09:42:49.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe840cf400 con 0x7efe8415a660 2024-03-23T09:42:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3175434343 0 0) 0x7efe6c003520 con 0x7efe8406a220 2024-03-23T09:42:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 >> v2:172.21.15.175:3300/0 conn(0x7efe840aedf0 msgr2=0x7efe84069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 --2- 172.21.15.47:0/4184685917 >> v2:172.21.15.175:3300/0 conn(0x7efe840aedf0 0x7efe84069ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 >> v1:172.21.15.175:6789/0 conn(0x7efe8415a660 legacy=0x7efe840ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe840cf9c0 con 0x7efe8406a220 2024-03-23T09:42:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7efe74001970 con 0x7efe8406a220 2024-03-23T09:42:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe888e3640 1 --2- 172.21.15.47:0/4184685917 >> v2:172.21.15.175:3300/0 conn(0x7efe840aedf0 0x7efe84069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:42:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 >> v1:172.21.15.47:6789/0 conn(0x7efe8406a220 legacy=0x7efe8406eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 --2- 172.21.15.47:0/4184685917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe6c003b50 0x7efe6c003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 --2- 172.21.15.47:0/4184685917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe6c0047c0 0x7efe6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 --2- 172.21.15.47:0/4184685917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe6c005180 0x7efe6c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe840d04a0 con 0x7efe6c0047c0 2024-03-23T09:42:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c00c050 con 0x7efe6c005180 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c00c210 con 0x7efe6c003b50 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe888e3640 1 --2- 172.21.15.47:0/4184685917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe6c0047c0 0x7efe6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe890e4640 1 --2- 172.21.15.47:0/4184685917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe6c003b50 0x7efe6c003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe898e5640 1 --2- 172.21.15.47:0/4184685917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe6c005180 0x7efe6c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe888e3640 1 -- 172.21.15.47:0/4184685917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe6c003b50 msgr2=0x7efe6c003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe888e3640 1 --2- 172.21.15.47:0/4184685917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe6c003b50 0x7efe6c003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe888e3640 1 -- 172.21.15.47:0/4184685917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe6c005180 msgr2=0x7efe6c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe888e3640 1 --2- 172.21.15.47:0/4184685917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe6c005180 0x7efe6c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe888e3640 1 -- 172.21.15.47:0/4184685917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c003520 con 0x7efe6c0047c0 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.290+0000 7efe898e5640 1 -- 172.21.15.47:0/4184685917 reap_dead start 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.291+0000 7efe888e3640 1 --2- 172.21.15.47:0/4184685917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe6c0047c0 0x7efe6c004c40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efe840af2c0 tx=0x7efe6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52907f55cf2bb13 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.291+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78004020 con 0x7efe6c0047c0 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.291+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe78004180 con 0x7efe6c0047c0 2024-03-23T09:42:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.291+0000 7efe83fff640 1 -- 172.21.15.47:0/4184685917 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7802c8a0 con 0x7efe6c0047c0 2024-03-23T09:42:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.292+0000 7efe8b348640 1 -- 172.21.15.47:0/4184685917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe6c0047c0 msgr2=0x7efe6c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.292+0000 7efe8b348640 1 --2- 172.21.15.47:0/4184685917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe6c0047c0 0x7efe6c004c40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efe840af2c0 tx=0x7efe6c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.292+0000 7efe8b348640 1 -- 172.21.15.47:0/4184685917 shutdown_connections 2024-03-23T09:42:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.292+0000 7efe8b348640 1 --2- 172.21.15.47:0/4184685917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe6c0047c0 0x7efe6c004c40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.292+0000 7efe8b348640 1 -- 172.21.15.47:0/4184685917 >> 172.21.15.47:0/4184685917 conn(0x7efe840b13a0 msgr2=0x7efe840b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.292+0000 7efe8b348640 1 -- 172.21.15.47:0/4184685917 shutdown_connections 2024-03-23T09:42:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.292+0000 7efe8b348640 1 -- 172.21.15.47:0/4184685917 wait complete. 2024-03-23T09:42:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.293+0000 7efe8b348640 1 Processor -- start 2024-03-23T09:42:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.293+0000 7efe8b348640 1 -- start start 2024-03-23T09:42:49.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe8b348640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe6c0047c0 0x7efe84152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:49.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe8b348640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe840cb310 0x7efe840cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:49.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe8b348640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe841585f0 0x7efe84158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:49.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe8b348640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe84153130 con 0x7efe840cb310 2024-03-23T09:42:49.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe8b348640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe84158fb0 con 0x7efe6c0047c0 2024-03-23T09:42:49.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe8b348640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe84153dd0 con 0x7efe841585f0 2024-03-23T09:42:49.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe888e3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe840cb310 0x7efe840cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe888e3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe840cb310 0x7efe840cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39816/0 (socket says 172.21.15.47:39816) 2024-03-23T09:42:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe888e3640 1 -- 172.21.15.47:0/4109047322 learned_addr learned my addr 172.21.15.47:0/4109047322 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe890e4640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe6c0047c0 0x7efe84152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe898e5640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe841585f0 0x7efe84158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe888e3640 1 -- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe841585f0 msgr2=0x7efe84158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe888e3640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe841585f0 0x7efe84158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe888e3640 1 -- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe6c0047c0 msgr2=0x7efe84152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe888e3640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe6c0047c0 0x7efe84152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe888e3640 1 -- 172.21.15.47:0/4109047322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe841540b0 con 0x7efe840cb310 2024-03-23T09:42:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe888e3640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe840cb310 0x7efe840cb770 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7efe78036040 tx=0x7efe78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe81ffb640 1 -- 172.21.15.47:0/4109047322 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78005e10 con 0x7efe840cb310 2024-03-23T09:42:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe81ffb640 1 -- 172.21.15.47:0/4109047322 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7802dce0 con 0x7efe840cb310 2024-03-23T09:42:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe8b348640 1 -- 172.21.15.47:0/4109047322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe84154390 con 0x7efe840cb310 2024-03-23T09:42:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.294+0000 7efe81ffb640 1 -- 172.21.15.47:0/4109047322 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78006240 con 0x7efe840cb310 2024-03-23T09:42:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.295+0000 7efe8b348640 1 -- 172.21.15.47:0/4109047322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe840c8a90 con 0x7efe840cb310 2024-03-23T09:42:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.296+0000 7efe8b348640 1 -- 172.21.15.47:0/4109047322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe840cf180 con 0x7efe840cb310 2024-03-23T09:42:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.296+0000 7efe81ffb640 1 -- 172.21.15.47:0/4109047322 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efe780063a0 con 0x7efe840cb310 2024-03-23T09:42:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.296+0000 7efe81ffb640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efe7006ac00 0x7efe7006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.296+0000 7efe81ffb640 1 -- 172.21.15.47:0/4109047322 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe780ab7c0 con 0x7efe840cb310 2024-03-23T09:42:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.297+0000 7efe890e4640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efe7006ac00 0x7efe7006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.297+0000 7efe890e4640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efe7006ac00 0x7efe7006d0c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe7c000c00 tx=0x7efe7c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.301+0000 7efe81ffb640 1 -- 172.21.15.47:0/4109047322 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efe78076910 con 0x7efe840cb310 2024-03-23T09:42:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:49.518+0000 7efe8b348640 1 -- 172.21.15.47:0/4109047322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7efe84066be0 con 0x7efe840cb310 2024-03-23T09:42:50.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.174+0000 7efe81ffb640 1 -- 172.21.15.47:0/4109047322 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7efe7804d020 con 0x7efe840cb310 2024-03-23T09:42:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.176+0000 7efe8b348640 1 -- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efe7006ac00 msgr2=0x7efe7006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.176+0000 7efe8b348640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efe7006ac00 0x7efe7006d0c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe7c000c00 tx=0x7efe7c009040 comp rx=0 tx=0).stop 2024-03-23T09:42:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.176+0000 7efe8b348640 1 -- 172.21.15.47:0/4109047322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe840cb310 msgr2=0x7efe840cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.176+0000 7efe8b348640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe840cb310 0x7efe840cb770 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7efe78036040 tx=0x7efe78004730 comp rx=0 tx=0).stop 2024-03-23T09:42:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.177+0000 7efe8b348640 1 -- 172.21.15.47:0/4109047322 shutdown_connections 2024-03-23T09:42:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.177+0000 7efe8b348640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efe7006ac00 0x7efe7006d0c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.177+0000 7efe8b348640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe841585f0 0x7efe84158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.177+0000 7efe8b348640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe840cb310 0x7efe840cb770 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.177+0000 7efe8b348640 1 --2- 172.21.15.47:0/4109047322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe6c0047c0 0x7efe84152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.177+0000 7efe8b348640 1 -- 172.21.15.47:0/4109047322 >> 172.21.15.47:0/4109047322 conn(0x7efe840b13a0 msgr2=0x7efe840afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:50.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.177+0000 7efe8b348640 1 -- 172.21.15.47:0/4109047322 shutdown_connections 2024-03-23T09:42:50.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.177+0000 7efe8b348640 1 -- 172.21.15.47:0/4109047322 wait complete. 2024-03-23T09:42:50.189 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-23T09:42:50.190 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:42:50.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.297+0000 7fd643db1640 1 Processor -- start 2024-03-23T09:42:50.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.297+0000 7fd643db1640 1 -- start start 2024-03-23T09:42:50.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd643db1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd63c15ca90 0x7fd63c160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd643db1640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd63c0b8350 0x7fd63c0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd643db1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c0ce090 con 0x7fd63c0cdc40 2024-03-23T09:42:50.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd643db1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c0d11e0 con 0x7fd63c15ca90 2024-03-23T09:42:50.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd643db1640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c0d2680 con 0x7fd63c0b8350 2024-03-23T09:42:50.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd641b4d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd63c15ca90 0x7fd63c160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd64234e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd63c0cdc40 0x7fd63c0b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41790/0 (socket says 172.21.15.47:41790) 2024-03-23T09:42:50.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd64234e640 1 -- 172.21.15.47:0/2387623 learned_addr learned my addr 172.21.15.47:0/2387623 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd64134c640 1 --2- 172.21.15.47:0/2387623 >> v2:172.21.15.175:3300/0 conn(0x7fd63c0b8350 0x7fd63c0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd641b4d640 1 -- 172.21.15.47:0/2387623 >> v2:172.21.15.175:3300/0 conn(0x7fd63c0b8350 msgr2=0x7fd63c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd640b4b640 1 -- 172.21.15.47:0/2387623 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161596161 0 0) 0x7fd634001490 con 0x7fd63c0cdc40 2024-03-23T09:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd641b4d640 1 --2- 172.21.15.47:0/2387623 >> v2:172.21.15.175:3300/0 conn(0x7fd63c0b8350 0x7fd63c0cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd641b4d640 1 -- 172.21.15.47:0/2387623 >> v1:172.21.15.47:6789/0 conn(0x7fd63c0cdc40 legacy=0x7fd63c0b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd641b4d640 1 -- 172.21.15.47:0/2387623 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c160f60 con 0x7fd63c15ca90 2024-03-23T09:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.298+0000 7fd64134c640 1 --2- 172.21.15.47:0/2387623 >> v2:172.21.15.175:3300/0 conn(0x7fd63c0b8350 0x7fd63c0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.299+0000 7fd641b4d640 1 --2- 172.21.15.47:0/2387623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd63c15ca90 0x7fd63c160710 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd63c160670 tx=0x7fd63802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6249ee8e3eaf8f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.299+0000 7fd640b4b640 1 -- 172.21.15.47:0/2387623 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63802e070 con 0x7fd63c15ca90 2024-03-23T09:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.299+0000 7fd640b4b640 1 -- 172.21.15.47:0/2387623 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6380315a0 con 0x7fd63c15ca90 2024-03-23T09:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.299+0000 7fd640b4b640 1 -- 172.21.15.47:0/2387623 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638031930 con 0x7fd63c15ca90 2024-03-23T09:42:50.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.300+0000 7fd643db1640 1 -- 172.21.15.47:0/2387623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd63c15ca90 msgr2=0x7fd63c160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.300+0000 7fd643db1640 1 --2- 172.21.15.47:0/2387623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd63c15ca90 0x7fd63c160710 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd63c160670 tx=0x7fd63802b840 comp rx=0 tx=0).stop 2024-03-23T09:42:50.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.300+0000 7fd643db1640 1 -- 172.21.15.47:0/2387623 shutdown_connections 2024-03-23T09:42:50.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.300+0000 7fd643db1640 1 --2- 172.21.15.47:0/2387623 >> v2:172.21.15.175:3300/0 conn(0x7fd63c0b8350 0x7fd63c0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.300+0000 7fd643db1640 1 --2- 172.21.15.47:0/2387623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd63c15ca90 0x7fd63c160710 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.300+0000 7fd643db1640 1 -- 172.21.15.47:0/2387623 >> 172.21.15.47:0/2387623 conn(0x7fd63c0659d0 msgr2=0x7fd63c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.300+0000 7fd643db1640 1 -- 172.21.15.47:0/2387623 shutdown_connections 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.300+0000 7fd643db1640 1 -- 172.21.15.47:0/2387623 wait complete. 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.301+0000 7fd643db1640 1 Processor -- start 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.301+0000 7fd643db1640 1 -- start start 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd643db1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd63c0b8350 0x7fd63c150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd643db1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd63c0cdc40 0x7fd63c0b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd643db1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd63c15ca90 0x7fd63c0b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd643db1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c1f2140 con 0x7fd63c0b8350 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd643db1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c15bf10 con 0x7fd63c0cdc40 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd643db1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c1f6640 con 0x7fd63c15ca90 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd641b4d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd63c0b8350 0x7fd63c150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd641b4d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd63c0b8350 0x7fd63c150ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39838/0 (socket says 172.21.15.47:39838) 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd641b4d640 1 -- 172.21.15.47:0/3963712314 learned_addr learned my addr 172.21.15.47:0/3963712314 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd641b4d640 1 -- 172.21.15.47:0/3963712314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd63c15ca90 msgr2=0x7fd63c0b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd641b4d640 1 --2- 172.21.15.47:0/3963712314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd63c15ca90 0x7fd63c0b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd641b4d640 1 -- 172.21.15.47:0/3963712314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd63c0cdc40 msgr2=0x7fd63c0b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd64134c640 1 --2- 172.21.15.47:0/3963712314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd63c0cdc40 0x7fd63c0b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd641b4d640 1 --2- 172.21.15.47:0/3963712314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd63c0cdc40 0x7fd63c0b2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd641b4d640 1 -- 172.21.15.47:0/3963712314 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd638009400 con 0x7fd63c0b8350 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.302+0000 7fd641b4d640 1 --2- 172.21.15.47:0/3963712314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd63c0b8350 0x7fd63c150ba0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd638009890 tx=0x7fd6380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.303+0000 7fd6327fc640 1 -- 172.21.15.47:0/3963712314 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638032840 con 0x7fd63c0b8350 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.303+0000 7fd6327fc640 1 -- 172.21.15.47:0/3963712314 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6380329a0 con 0x7fd63c0b8350 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.303+0000 7fd643db1640 1 -- 172.21.15.47:0/3963712314 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c1510e0 con 0x7fd63c0b8350 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.303+0000 7fd6327fc640 1 -- 172.21.15.47:0/3963712314 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638032d50 con 0x7fd63c0b8350 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.303+0000 7fd643db1640 1 -- 172.21.15.47:0/3963712314 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c1f2330 con 0x7fd63c0b8350 2024-03-23T09:42:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.304+0000 7fd643db1640 1 -- 172.21.15.47:0/3963712314 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c072360 con 0x7fd63c0b8350 2024-03-23T09:42:50.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.304+0000 7fd6327fc640 1 -- 172.21.15.47:0/3963712314 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd63803b3f0 con 0x7fd63c0b8350 2024-03-23T09:42:50.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.305+0000 7fd6327fc640 1 --2- 172.21.15.47:0/3963712314 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd62006ac30 0x7fd62006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.305+0000 7fd6327fc640 1 -- 172.21.15.47:0/3963712314 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6380ab250 con 0x7fd63c0b8350 2024-03-23T09:42:50.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.305+0000 7fd64134c640 1 --2- 172.21.15.47:0/3963712314 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd62006ac30 0x7fd62006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.305+0000 7fd64134c640 1 --2- 172.21.15.47:0/3963712314 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd62006ac30 0x7fd62006d0f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd63c157f60 tx=0x7fd62c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.309+0000 7fd6327fc640 1 -- 172.21.15.47:0/3963712314 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd6380764c0 con 0x7fd63c0b8350 2024-03-23T09:42:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.528+0000 7fd643db1640 1 -- 172.21.15.47:0/3963712314 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd63c066be0 con 0x7fd63c0b8350 2024-03-23T09:42:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.528+0000 7fd6327fc640 1 -- 172.21.15.47:0/3963712314 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fd638079ae0 con 0x7fd63c0b8350 2024-03-23T09:42:50.528 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:50.528 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1189831856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1189831856},{"type":"v1","addr":"172.21.15.47:6835","nonce":1189831856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/2248064454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2248064454},{"type":"v1","addr":"172.21.15.47:6834","nonce":2248064454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5694,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3806465612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3806465612},{"type":"v1","addr":"172.21.15.175:6835","nonce":3806465612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:42:48.165819+0000","modified":"2024-03-23T09:42:50.171529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1626348200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1626348200},{"type":"v1","addr":"172.21.15.175:6837","nonce":1626348200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-23T09:42:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 18 2024-03-23T09:42:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.530+0000 7fd643db1640 1 -- 172.21.15.47:0/3963712314 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd62006ac30 msgr2=0x7fd62006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.530+0000 7fd643db1640 1 --2- 172.21.15.47:0/3963712314 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd62006ac30 0x7fd62006d0f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd63c157f60 tx=0x7fd62c008040 comp rx=0 tx=0).stop 2024-03-23T09:42:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.530+0000 7fd643db1640 1 -- 172.21.15.47:0/3963712314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd63c0b8350 msgr2=0x7fd63c150ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.530+0000 7fd643db1640 1 --2- 172.21.15.47:0/3963712314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd63c0b8350 0x7fd63c150ba0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd638009890 tx=0x7fd6380046c0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.530+0000 7fd643db1640 1 -- 172.21.15.47:0/3963712314 shutdown_connections 2024-03-23T09:42:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.530+0000 7fd643db1640 1 --2- 172.21.15.47:0/3963712314 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd62006ac30 0x7fd62006d0f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.530+0000 7fd643db1640 1 --2- 172.21.15.47:0/3963712314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd63c15ca90 0x7fd63c0b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.530+0000 7fd643db1640 1 --2- 172.21.15.47:0/3963712314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd63c0cdc40 0x7fd63c0b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.530+0000 7fd643db1640 1 --2- 172.21.15.47:0/3963712314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd63c0b8350 0x7fd63c150ba0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.530+0000 7fd643db1640 1 -- 172.21.15.47:0/3963712314 >> 172.21.15.47:0/3963712314 conn(0x7fd63c0659d0 msgr2=0x7fd63c0c8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.530+0000 7fd643db1640 1 -- 172.21.15.47:0/3963712314 shutdown_connections 2024-03-23T09:42:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.530+0000 7fd643db1640 1 -- 172.21.15.47:0/3963712314 wait complete. 2024-03-23T09:42:50.540 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:42:50.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.641+0000 7faf50a3e640 1 Processor -- start 2024-03-23T09:42:50.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.641+0000 7faf50a3e640 1 -- start start 2024-03-23T09:42:50.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.642+0000 7faf50a3e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faf4c069ce0 0x7faf4c0d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.642+0000 7faf50a3e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faf4c0ae620 0x7faf4c15c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.642+0000 7faf50a3e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c06f150 con 0x7faf4c160720 2024-03-23T09:42:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.642+0000 7faf50a3e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c0cb750 con 0x7faf4c0ae620 2024-03-23T09:42:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.642+0000 7faf50a3e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c0cba30 con 0x7faf4c069ce0 2024-03-23T09:42:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.642+0000 7faf4ad9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faf4c0ae620 0x7faf4c15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.642+0000 7faf49d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7faf4c160720 0x7faf4c06a0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41794/0 (socket says 172.21.15.47:41794) 2024-03-23T09:42:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.642+0000 7faf49d9b640 1 -- 172.21.15.47:0/476641653 learned_addr learned my addr 172.21.15.47:0/476641653 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.643+0000 7faf4a59c640 1 --2- 172.21.15.47:0/476641653 >> v2:172.21.15.175:3300/0 conn(0x7faf4c069ce0 0x7faf4c0d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.643+0000 7faf4ad9d640 1 -- 172.21.15.47:0/476641653 >> v2:172.21.15.175:3300/0 conn(0x7faf4c069ce0 msgr2=0x7faf4c0d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.643+0000 7faf4ad9d640 1 --2- 172.21.15.47:0/476641653 >> v2:172.21.15.175:3300/0 conn(0x7faf4c069ce0 0x7faf4c0d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.643+0000 7faf4ad9d640 1 -- 172.21.15.47:0/476641653 >> v1:172.21.15.47:6789/0 conn(0x7faf4c160720 legacy=0x7faf4c06a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.643+0000 7faf4ad9d640 1 -- 172.21.15.47:0/476641653 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c06e820 con 0x7faf4c0ae620 2024-03-23T09:42:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.643+0000 7faf4959a640 1 -- 172.21.15.47:0/476641653 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673023744 0 0) 0x7faf4c06f150 con 0x7faf4c160720 2024-03-23T09:42:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.643+0000 7faf4ad9d640 1 --2- 172.21.15.47:0/476641653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c0ae620 0x7faf4c15c260 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7faf4c06fa50 tx=0x7faf4402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd9bad2b78abaaf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.644+0000 7faf4959a640 1 -- 172.21.15.47:0/476641653 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4402e070 con 0x7faf4c0ae620 2024-03-23T09:42:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.644+0000 7faf4959a640 1 -- 172.21.15.47:0/476641653 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf440315a0 con 0x7faf4c0ae620 2024-03-23T09:42:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.644+0000 7faf4959a640 1 -- 172.21.15.47:0/476641653 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44031930 con 0x7faf4c0ae620 2024-03-23T09:42:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.644+0000 7faf50a3e640 1 -- 172.21.15.47:0/476641653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c0ae620 msgr2=0x7faf4c15c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.644+0000 7faf50a3e640 1 --2- 172.21.15.47:0/476641653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c0ae620 0x7faf4c15c260 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7faf4c06fa50 tx=0x7faf4402b840 comp rx=0 tx=0).stop 2024-03-23T09:42:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.644+0000 7faf50a3e640 1 -- 172.21.15.47:0/476641653 shutdown_connections 2024-03-23T09:42:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.644+0000 7faf50a3e640 1 --2- 172.21.15.47:0/476641653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c0ae620 0x7faf4c15c260 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.644+0000 7faf50a3e640 1 --2- 172.21.15.47:0/476641653 >> v2:172.21.15.175:3300/0 conn(0x7faf4c069ce0 0x7faf4c0d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.644+0000 7faf50a3e640 1 -- 172.21.15.47:0/476641653 >> 172.21.15.47:0/476641653 conn(0x7faf4c098a00 msgr2=0x7faf4c0cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.644+0000 7faf50a3e640 1 -- 172.21.15.47:0/476641653 shutdown_connections 2024-03-23T09:42:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.644+0000 7faf50a3e640 1 -- 172.21.15.47:0/476641653 wait complete. 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.646+0000 7faf50a3e640 1 Processor -- start 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.646+0000 7faf50a3e640 1 -- start start 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.646+0000 7faf50a3e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4c069ce0 0x7faf4c14fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.646+0000 7faf50a3e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4c0ae620 0x7faf4c14d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.646+0000 7faf50a3e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c160720 0x7faf4c15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.646+0000 7faf50a3e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c15f5d0 con 0x7faf4c160720 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.646+0000 7faf50a3e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c14d390 con 0x7faf4c069ce0 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.646+0000 7faf50a3e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c157f70 con 0x7faf4c0ae620 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.646+0000 7faf4ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c160720 0x7faf4c15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.646+0000 7faf4ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c160720 0x7faf4c15a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39858/0 (socket says 172.21.15.47:39858) 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.646+0000 7faf4ad9d640 1 -- 172.21.15.47:0/1116309816 learned_addr learned my addr 172.21.15.47:0/1116309816 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf4a59c640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4c069ce0 0x7faf4c14fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf49d9b640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4c0ae620 0x7faf4c14d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf4ad9d640 1 -- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4c0ae620 msgr2=0x7faf4c14d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf4ad9d640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4c0ae620 0x7faf4c14d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf4ad9d640 1 -- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4c069ce0 msgr2=0x7faf4c14fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf4ad9d640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4c069ce0 0x7faf4c14fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf4ad9d640 1 -- 172.21.15.47:0/1116309816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf44009400 con 0x7faf4c160720 2024-03-23T09:42:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf4a59c640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4c069ce0 0x7faf4c14fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf4ad9d640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c160720 0x7faf4c15a5a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7faf4402be50 tx=0x7faf440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf42ffd640 1 -- 172.21.15.47:0/1116309816 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44032840 con 0x7faf4c160720 2024-03-23T09:42:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf42ffd640 1 -- 172.21.15.47:0/1116309816 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf440329a0 con 0x7faf4c160720 2024-03-23T09:42:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf50a3e640 1 -- 172.21.15.47:0/1116309816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c157080 con 0x7faf4c160720 2024-03-23T09:42:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf42ffd640 1 -- 172.21.15.47:0/1116309816 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44032d30 con 0x7faf4c160720 2024-03-23T09:42:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.647+0000 7faf50a3e640 1 -- 172.21.15.47:0/1116309816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c157640 con 0x7faf4c160720 2024-03-23T09:42:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.648+0000 7faf50a3e640 1 -- 172.21.15.47:0/1116309816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c158380 con 0x7faf4c160720 2024-03-23T09:42:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.649+0000 7faf42ffd640 1 -- 172.21.15.47:0/1116309816 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7faf4404e430 con 0x7faf4c160720 2024-03-23T09:42:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.649+0000 7faf42ffd640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf2c06ac00 0x7faf2c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.649+0000 7faf42ffd640 1 -- 172.21.15.47:0/1116309816 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf440ab2b0 con 0x7faf4c160720 2024-03-23T09:42:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.649+0000 7faf4a59c640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf2c06ac00 0x7faf2c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.650+0000 7faf4a59c640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf2c06ac00 0x7faf2c06d0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faf3c007440 tx=0x7faf3c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.653+0000 7faf42ffd640 1 -- 172.21.15.47:0/1116309816 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7faf4c158380 con 0x7faf4c160720 2024-03-23T09:42:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.838+0000 7faf50a3e640 1 -- 172.21.15.47:0/1116309816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf4c158380 con 0x7faf4c160720 2024-03-23T09:42:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.839+0000 7faf42ffd640 1 -- 172.21.15.47:0/1116309816 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16045 (secure 0 0 0) 0x7faf4c158380 con 0x7faf4c160720 2024-03-23T09:42:50.839 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:50.839 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":46,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:49.167591+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:42:46.053713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-23T09:42:47.131279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:42:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.841+0000 7faf50a3e640 1 -- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf2c06ac00 msgr2=0x7faf2c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.841+0000 7faf50a3e640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf2c06ac00 0x7faf2c06d0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faf3c007440 tx=0x7faf3c00b040 comp rx=0 tx=0).stop 2024-03-23T09:42:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.841+0000 7faf50a3e640 1 -- 172.21.15.47:0/1116309816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c160720 msgr2=0x7faf4c15a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.841+0000 7faf50a3e640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c160720 0x7faf4c15a5a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7faf4402be50 tx=0x7faf440046c0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.841+0000 7faf50a3e640 1 -- 172.21.15.47:0/1116309816 shutdown_connections 2024-03-23T09:42:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.841+0000 7faf50a3e640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf2c06ac00 0x7faf2c06d0c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.841+0000 7faf50a3e640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c160720 0x7faf4c15a5a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.841+0000 7faf50a3e640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4c0ae620 0x7faf4c14d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.841+0000 7faf50a3e640 1 --2- 172.21.15.47:0/1116309816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4c069ce0 0x7faf4c14fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.841+0000 7faf50a3e640 1 -- 172.21.15.47:0/1116309816 >> 172.21.15.47:0/1116309816 conn(0x7faf4c098a00 msgr2=0x7faf4c0cbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.841+0000 7faf50a3e640 1 -- 172.21.15.47:0/1116309816 shutdown_connections 2024-03-23T09:42:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.841+0000 7faf50a3e640 1 -- 172.21.15.47:0/1116309816 wait complete. 2024-03-23T09:42:50.852 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T09:42:50.852 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:42:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.953+0000 7fa67c63e640 1 Processor -- start 2024-03-23T09:42:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.953+0000 7fa67c63e640 1 -- start start 2024-03-23T09:42:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.954+0000 7fa67c63e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa67415ca90 0x7fa674160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.954+0000 7fa67c63e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6740d11e0 con 0x7fa6740b8350 2024-03-23T09:42:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.954+0000 7fa67c63e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa674160f60 con 0x7fa6740cde10 2024-03-23T09:42:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.954+0000 7fa67c63e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa674161240 con 0x7fa67415ca90 2024-03-23T09:42:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.954+0000 7fa679bd9640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa6740b8350 0x7fa6740cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41798/0 (socket says 172.21.15.47:41798) 2024-03-23T09:42:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.954+0000 7fa679bd9640 1 -- 172.21.15.47:0/3087532707 learned_addr learned my addr 172.21.15.47:0/3087532707 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.954+0000 7fa67a3da640 1 --2- 172.21.15.47:0/3087532707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa67415ca90 0x7fa674160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867934156 0 0) 0x7fa6740d11e0 con 0x7fa6740b8350 2024-03-23T09:42:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa65c003520 con 0x7fa6740b8350 2024-03-23T09:42:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 813435597 0 0) 0x7fa65c003520 con 0x7fa6740b8350 2024-03-23T09:42:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa67415ca90 msgr2=0x7fa674160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 --2- 172.21.15.47:0/3087532707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa67415ca90 0x7fa674160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 >> v1:172.21.15.175:6789/0 conn(0x7fa6740cde10 legacy=0x7fa6740d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa674161520 con 0x7fa6740b8350 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fa668001670 con 0x7fa6740b8350 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 >> v1:172.21.15.47:6789/0 conn(0x7fa6740b8350 legacy=0x7fa6740cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 --2- 172.21.15.47:0/3087532707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c0039a0 0x7fa65c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 --2- 172.21.15.47:0/3087532707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa67415ca90 0x7fa65c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 --2- 172.21.15.47:0/3087532707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa65c004d00 0x7fa65c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa674161240 con 0x7fa65c0039a0 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c00ba30 con 0x7fa65c004d00 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c00bbf0 con 0x7fa67415ca90 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa67a3da640 1 --2- 172.21.15.47:0/3087532707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c0039a0 0x7fa65c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa679bd9640 1 --2- 172.21.15.47:0/3087532707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa67415ca90 0x7fa65c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa67abdb640 1 --2- 172.21.15.47:0/3087532707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa65c004d00 0x7fa65c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa67a3da640 1 -- 172.21.15.47:0/3087532707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa67415ca90 msgr2=0x7fa65c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa67a3da640 1 --2- 172.21.15.47:0/3087532707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa67415ca90 0x7fa65c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa67a3da640 1 -- 172.21.15.47:0/3087532707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa65c004d00 msgr2=0x7fa65c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa67a3da640 1 --2- 172.21.15.47:0/3087532707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa65c004d00 0x7fa65c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa67a3da640 1 -- 172.21.15.47:0/3087532707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c003520 con 0x7fa65c0039a0 2024-03-23T09:42:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.955+0000 7fa67abdb640 1 --2- 172.21.15.47:0/3087532707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa65c004d00 0x7fa65c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.956+0000 7fa67a3da640 1 --2- 172.21.15.47:0/3087532707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c0039a0 0x7fa65c003d80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009fc0 tx=0x7fa65c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7e2d7dabc41629 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.956+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c02cc20 con 0x7fa65c0039a0 2024-03-23T09:42:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.956+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c00f040 con 0x7fa65c0039a0 2024-03-23T09:42:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.956+0000 7fa6793d8640 1 -- 172.21.15.47:0/3087532707 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c034440 con 0x7fa65c0039a0 2024-03-23T09:42:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.957+0000 7fa67c63e640 1 -- 172.21.15.47:0/3087532707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c0039a0 msgr2=0x7fa65c003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.957+0000 7fa67c63e640 1 --2- 172.21.15.47:0/3087532707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c0039a0 0x7fa65c003d80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009fc0 tx=0x7fa65c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.957+0000 7fa67abdb640 1 -- 172.21.15.47:0/3087532707 reap_dead start 2024-03-23T09:42:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.957+0000 7fa67c63e640 1 -- 172.21.15.47:0/3087532707 shutdown_connections 2024-03-23T09:42:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.957+0000 7fa67c63e640 1 -- 172.21.15.47:0/3087532707 >> 172.21.15.47:0/3087532707 conn(0x7fa6740659d0 msgr2=0x7fa674065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.957+0000 7fa67c63e640 1 -- 172.21.15.47:0/3087532707 shutdown_connections 2024-03-23T09:42:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.957+0000 7fa67c63e640 1 -- 172.21.15.47:0/3087532707 wait complete. 2024-03-23T09:42:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.958+0000 7fa67c63e640 1 Processor -- start 2024-03-23T09:42:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.958+0000 7fa67c63e640 1 -- start start 2024-03-23T09:42:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67c63e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c0039a0 0x7fa6740c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67c63e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6740cc6b0 0x7fa6740cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67c63e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6740cb170 0x7fa6740b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67c63e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740cc220 con 0x7fa65c0039a0 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67c63e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740cb610 con 0x7fa6740cc6b0 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67c63e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740cb8f0 con 0x7fa6740cb170 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67a3da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c0039a0 0x7fa6740c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67a3da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c0039a0 0x7fa6740c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39880/0 (socket says 172.21.15.47:39880) 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67a3da640 1 -- 172.21.15.47:0/1655179398 learned_addr learned my addr 172.21.15.47:0/1655179398 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67abdb640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6740cb170 0x7fa6740b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa679bd9640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6740cc6b0 0x7fa6740cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67a3da640 1 -- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6740cb170 msgr2=0x7fa6740b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67a3da640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6740cb170 0x7fa6740b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67a3da640 1 -- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6740cc6b0 msgr2=0x7fa6740cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67a3da640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6740cc6b0 0x7fa6740cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67a3da640 1 -- 172.21.15.47:0/1655179398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6740cbbd0 con 0x7fa65c0039a0 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa679bd9640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6740cc6b0 0x7fa6740cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67a3da640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c0039a0 0x7fa6740c8180 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00e230 tx=0x7fa66c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa672ffd640 1 -- 172.21.15.47:0/1655179398 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c034bd0 con 0x7fa65c0039a0 2024-03-23T09:42:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa672ffd640 1 -- 172.21.15.47:0/1655179398 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c034d50 con 0x7fa65c0039a0 2024-03-23T09:42:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67c63e640 1 -- 172.21.15.47:0/1655179398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6740cbeb0 con 0x7fa65c0039a0 2024-03-23T09:42:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa672ffd640 1 -- 172.21.15.47:0/1655179398 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c003bd0 con 0x7fa65c0039a0 2024-03-23T09:42:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.959+0000 7fa67c63e640 1 -- 172.21.15.47:0/1655179398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6740b2570 con 0x7fa65c0039a0 2024-03-23T09:42:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.960+0000 7fa67c63e640 1 -- 172.21.15.47:0/1655179398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa674072360 con 0x7fa65c0039a0 2024-03-23T09:42:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.961+0000 7fa672ffd640 1 -- 172.21.15.47:0/1655179398 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa66c040450 con 0x7fa65c0039a0 2024-03-23T09:42:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.961+0000 7fa672ffd640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa66006ac00 0x7fa66006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.961+0000 7fa672ffd640 1 -- 172.21.15.47:0/1655179398 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa66c0aba50 con 0x7fa65c0039a0 2024-03-23T09:42:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.961+0000 7fa679bd9640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa66006ac00 0x7fa66006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.962+0000 7fa679bd9640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa66006ac00 0x7fa66006d0c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa6740cca90 tx=0x7fa668009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:50.965+0000 7fa672ffd640 1 -- 172.21.15.47:0/1655179398 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa66c076c30 con 0x7fa65c0039a0 2024-03-23T09:42:51.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.142+0000 7fa67c63e640 1 -- 172.21.15.47:0/1655179398 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6740b5b60 con 0x7fa66006ac00 2024-03-23T09:42:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.150+0000 7fa672ffd640 1 -- 172.21.15.47:0/1655179398 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278383 (secure 0 0 0) 0x7fa668001550 con 0x7fa66006ac00 2024-03-23T09:42:51.152 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:51.154 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:42:51.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.153+0000 7fa67c63e640 1 -- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa66006ac00 msgr2=0x7fa66006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:51.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.153+0000 7fa67c63e640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa66006ac00 0x7fa66006d0c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa6740cca90 tx=0x7fa668009040 comp rx=0 tx=0).stop 2024-03-23T09:42:51.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.153+0000 7fa67c63e640 1 -- 172.21.15.47:0/1655179398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c0039a0 msgr2=0x7fa6740c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:51.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.153+0000 7fa67c63e640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c0039a0 0x7fa6740c8180 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00e230 tx=0x7fa66c009750 comp rx=0 tx=0).stop 2024-03-23T09:42:51.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.153+0000 7fa67c63e640 1 -- 172.21.15.47:0/1655179398 shutdown_connections 2024-03-23T09:42:51.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.153+0000 7fa67c63e640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa66006ac00 0x7fa66006d0c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.153+0000 7fa67c63e640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6740cb170 0x7fa6740b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.153+0000 7fa67c63e640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6740cc6b0 0x7fa6740cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.153+0000 7fa67c63e640 1 --2- 172.21.15.47:0/1655179398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c0039a0 0x7fa6740c8180 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.153+0000 7fa67c63e640 1 -- 172.21.15.47:0/1655179398 >> 172.21.15.47:0/1655179398 conn(0x7fa6740659d0 msgr2=0x7fa67415bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:51.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.153+0000 7fa67c63e640 1 -- 172.21.15.47:0/1655179398 shutdown_connections 2024-03-23T09:42:51.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.153+0000 7fa67c63e640 1 -- 172.21.15.47:0/1655179398 wait complete. 2024-03-23T09:42:51.163 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-03-23T09:42:50.364075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":237592,"kb_used_data":5040,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737128,"statfs":{"total":773094113280,"available":772850819072,"internally_reserved":0,"allocated":5160960,"data_stored":2752870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000781"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792350+0000","last_change":"2024-03-23T09:42:46.792350+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792350+0000","last_undegraded":"2024-03-23T09:42:46.792350+0000","last_fullsized":"2024-03-23T09:42:46.792350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:41.236379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791904+0000","last_change":"2024-03-23T09:42:46.791904+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791904+0000","last_undegraded":"2024-03-23T09:42:46.791904+0000","last_fullsized":"2024-03-23T09:42:46.791904+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793978+0000","last_change":"2024-03-23T09:42:46.793978+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793978+0000","last_undegraded":"2024-03-23T09:42:46.793978+0000","last_fullsized":"2024-03-23T09:42:46.793978+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:21.810507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791103+0000","last_change":"2024-03-23T09:42:46.791103+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791103+0000","last_undegraded":"2024-03-23T09:42:46.791103+0000","last_fullsized":"2024-03-23T09:42:46.791103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:04:24.190076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793327+0000","last_change":"2024-03-23T09:42:46.793327+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793327+0000","last_undegraded":"2024-03-23T09:42:46.793327+0000","last_fullsized":"2024-03-23T09:42:46.793327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:40.886514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791385+0000","last_change":"2024-03-23T09:42:46.791385+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791385+0000","last_undegraded":"2024-03-23T09:42:46.791385+0000","last_fullsized":"2024-03-23T09:42:46.791385+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:58:54.380376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.794200+0000","last_change":"2024-03-23T09:42:46.794200+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.794200+0000","last_undegraded":"2024-03-23T09:42:46.794200+0000","last_fullsized":"2024-03-23T09:42:46.794200+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:04:53.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.796428+0000","last_change":"2024-03-23T09:42:46.796428+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.796428+0000","last_undegraded":"2024-03-23T09:42:46.796428+0000","last_fullsized":"2024-03-23T09:42:46.796428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790630+0000","last_change":"2024-03-23T09:42:46.790630+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790630+0000","last_undegraded":"2024-03-23T09:42:46.790630+0000","last_fullsized":"2024-03-23T09:42:46.790630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:01:51.451920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795158+0000","last_change":"2024-03-23T09:42:46.795158+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795158+0000","last_undegraded":"2024-03-23T09:42:46.795158+0000","last_fullsized":"2024-03-23T09:42:46.795158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:02:27.446246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792484+0000","last_change":"2024-03-23T09:42:46.792484+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792484+0000","last_undegraded":"2024-03-23T09:42:46.792484+0000","last_fullsized":"2024-03-23T09:42:46.792484+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:35:53.515036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790019+0000","last_change":"2024-03-23T09:42:46.790019+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790019+0000","last_undegraded":"2024-03-23T09:42:46.790019+0000","last_fullsized":"2024-03-23T09:42:46.790019+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792903+0000","last_change":"2024-03-23T09:42:46.792903+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792903+0000","last_undegraded":"2024-03-23T09:42:46.792903+0000","last_fullsized":"2024-03-23T09:42:46.792903+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:43:42.174052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791816+0000","last_change":"2024-03-23T09:42:46.791816+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791816+0000","last_undegraded":"2024-03-23T09:42:46.791816+0000","last_fullsized":"2024-03-23T09:42:46.791816+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:32.470235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792213+0000","last_change":"2024-03-23T09:42:46.792213+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792213+0000","last_undegraded":"2024-03-23T09:42:46.792213+0000","last_fullsized":"2024-03-23T09:42:46.792213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:40:05.822261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795999+0000","last_change":"2024-03-23T09:42:46.795999+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795999+0000","last_undegraded":"2024-03-23T09:42:46.795999+0000","last_fullsized":"2024-03-23T09:42:46.795999+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:01:51.451920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790079+0000","last_change":"2024-03-23T09:42:46.790079+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790079+0000","last_undegraded":"2024-03-23T09:42:46.790079+0000","last_fullsized":"2024-03-23T09:42:46.790079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:19.815833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791201+0000","last_change":"2024-03-23T09:42:46.791201+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791201+0000","last_undegraded":"2024-03-23T09:42:46.791201+0000","last_fullsized":"2024-03-23T09:42:46.791201+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:19.815833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793862+0000","last_change":"2024-03-23T09:42:46.793862+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793862+0000","last_undegraded":"2024-03-23T09:42:46.793862+0000","last_fullsized":"2024-03-23T09:42:46.793862+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:47:48.322837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792271+0000","last_change":"2024-03-23T09:42:46.792271+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792271+0000","last_undegraded":"2024-03-23T09:42:46.792271+0000","last_fullsized":"2024-03-23T09:42:46.792271+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:29:32.796473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792275+0000","last_change":"2024-03-23T09:42:46.792275+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792275+0000","last_undegraded":"2024-03-23T09:42:46.792275+0000","last_fullsized":"2024-03-23T09:42:46.792275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:46.555393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793510+0000","last_change":"2024-03-23T09:42:46.793510+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793510+0000","last_undegraded":"2024-03-23T09:42:46.793510+0000","last_fullsized":"2024-03-23T09:42:46.793510+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:14:55.790367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795084+0000","last_change":"2024-03-23T09:42:46.795084+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795084+0000","last_undegraded":"2024-03-23T09:42:46.795084+0000","last_fullsized":"2024-03-23T09:42:46.795084+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:09:03.354548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791854+0000","last_change":"2024-03-23T09:42:46.791854+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791854+0000","last_undegraded":"2024-03-23T09:42:46.791854+0000","last_fullsized":"2024-03-23T09:42:46.791854+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:50:10.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792263+0000","last_change":"2024-03-23T09:42:46.792263+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792263+0000","last_undegraded":"2024-03-23T09:42:46.792263+0000","last_fullsized":"2024-03-23T09:42:46.792263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:27:56.832097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792046+0000","last_change":"2024-03-23T09:42:46.792046+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792046+0000","last_undegraded":"2024-03-23T09:42:46.792046+0000","last_fullsized":"2024-03-23T09:42:46.792046+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:27:56.832097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793721+0000","last_change":"2024-03-23T09:42:46.793721+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793721+0000","last_undegraded":"2024-03-23T09:42:46.793721+0000","last_fullsized":"2024-03-23T09:42:46.793721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:37.970066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791095+0000","last_change":"2024-03-23T09:42:46.791095+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791095+0000","last_undegraded":"2024-03-23T09:42:46.791095+0000","last_fullsized":"2024-03-23T09:42:46.791095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:43.932184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790630+0000","last_change":"2024-03-23T09:42:46.790630+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790630+0000","last_undegraded":"2024-03-23T09:42:46.790630+0000","last_fullsized":"2024-03-23T09:42:46.790630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:02.796426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.794308+0000","last_change":"2024-03-23T09:42:46.794308+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.794308+0000","last_undegraded":"2024-03-23T09:42:46.794308+0000","last_fullsized":"2024-03-23T09:42:46.794308+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:24:36.255934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795352+0000","last_change":"2024-03-23T09:42:46.795352+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795352+0000","last_undegraded":"2024-03-23T09:42:46.795352+0000","last_fullsized":"2024-03-23T09:42:46.795352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:02.796426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793582+0000","last_change":"2024-03-23T09:42:46.793582+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793582+0000","last_undegraded":"2024-03-23T09:42:46.793582+0000","last_fullsized":"2024-03-23T09:42:46.793582+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:04:09.114370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":186,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-23T09:42:44.417082+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:42:44.417082+0000","last_peered":"2024-03-23T09:42:44.417082+0000","last_clean":"2024-03-23T09:42:44.417082+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:42:44.417082+0000","last_undegraded":"2024-03-23T09:42:44.417082+0000","last_fullsized":"2024-03-23T09:42:44.417082+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1097728,"data_stored":789948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54400000000000004}]},{"osd":1,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.50900000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.47699999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27500,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":487424,"data_stored":193144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.52100000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.56299999999999994}]},{"osd":2,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.50700000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.433}]},{"osd":4,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48799999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.45600000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":199580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.439},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.53100000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.51300000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.67600000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.73799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":199580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69899999999999995}]},{"osd":1,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.66300000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.747}]},{"osd":3,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.48599999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.502}]},{"osd":7,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":491520,"data_stored":193763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.19},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.30299999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.223}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.318},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.29199999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.503}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60499999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":503808,"data_stored":199580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.34799999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.316}]},{"osd":3,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.248}]},{"osd":4,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.41999999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.56599999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.436}]},{"osd":7,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.47399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":491520,"data_stored":193763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.441},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":0.42299999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.85199999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71799999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.65600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1081344,"data_stored":783512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.247},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.29499999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.314},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.34100000000000003}]},{"osd":3,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.35399999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.625}]},{"osd":5,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65300000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.66800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:42:51.168 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:42:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.305+0000 7f54143a4640 1 Processor -- start 2024-03-23T09:42:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.305+0000 7f54143a4640 1 -- start start 2024-03-23T09:42:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.306+0000 7f54143a4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f540c15cb40 0x7f540c160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.306+0000 7f54143a4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f540c160cb0 0x7f540c0d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.306+0000 7f54143a4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540c0d0ee0 con 0x7f540c0b8300 2024-03-23T09:42:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.306+0000 7f54143a4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540c0b5a10 con 0x7f540c160cb0 2024-03-23T09:42:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.306+0000 7f54143a4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540c1616f0 con 0x7f540c15cb40 2024-03-23T09:42:51.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.306+0000 7f5412941640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f540c160cb0 0x7f540c0d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:51.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.306+0000 7f5412941640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f540c160cb0 0x7f540c0d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39892/0 (socket says 172.21.15.47:39892) 2024-03-23T09:42:51.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.306+0000 7f5412941640 1 -- 172.21.15.47:0/1925245560 learned_addr learned my addr 172.21.15.47:0/1925245560 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:51.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.306+0000 7f5412941640 1 -- 172.21.15.47:0/1925245560 >> v2:172.21.15.175:3300/0 conn(0x7f540c15cb40 msgr2=0x7f540c160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:51.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.306+0000 7f5412941640 1 --2- 172.21.15.47:0/1925245560 >> v2:172.21.15.175:3300/0 conn(0x7f540c15cb40 0x7f540c160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.306+0000 7f5412941640 1 -- 172.21.15.47:0/1925245560 >> v1:172.21.15.175:6789/0 conn(0x7f540c0b8300 legacy=0x7f540c0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:51.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.306+0000 7f5412941640 1 -- 172.21.15.47:0/1925245560 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c161130 con 0x7f540c160cb0 2024-03-23T09:42:51.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.307+0000 7f5412941640 1 --2- 172.21.15.47:0/1925245560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c160cb0 0x7f540c0d1370 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009e50 tx=0x7f53fc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f8b4400c232a56 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.307+0000 7f541113e640 1 -- 172.21.15.47:0/1925245560 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc02c9f0 con 0x7f540c160cb0 2024-03-23T09:42:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.307+0000 7f541113e640 1 -- 172.21.15.47:0/1925245560 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc02cb50 con 0x7f540c160cb0 2024-03-23T09:42:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.307+0000 7f541113e640 1 -- 172.21.15.47:0/1925245560 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0343f0 con 0x7f540c160cb0 2024-03-23T09:42:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.308+0000 7f54143a4640 1 -- 172.21.15.47:0/1925245560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c160cb0 msgr2=0x7f540c0d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.308+0000 7f54143a4640 1 --2- 172.21.15.47:0/1925245560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c160cb0 0x7f540c0d1370 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009e50 tx=0x7f53fc02be50 comp rx=0 tx=0).stop 2024-03-23T09:42:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.308+0000 7f54143a4640 1 -- 172.21.15.47:0/1925245560 shutdown_connections 2024-03-23T09:42:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.308+0000 7f54143a4640 1 --2- 172.21.15.47:0/1925245560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c160cb0 0x7f540c0d1370 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.308+0000 7f54143a4640 1 --2- 172.21.15.47:0/1925245560 >> v2:172.21.15.175:3300/0 conn(0x7f540c15cb40 0x7f540c160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.308+0000 7f54143a4640 1 -- 172.21.15.47:0/1925245560 >> 172.21.15.47:0/1925245560 conn(0x7f540c06b160 msgr2=0x7f540c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.308+0000 7f54143a4640 1 -- 172.21.15.47:0/1925245560 shutdown_connections 2024-03-23T09:42:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.308+0000 7f54143a4640 1 -- 172.21.15.47:0/1925245560 wait complete. 2024-03-23T09:42:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.309+0000 7f54143a4640 1 Processor -- start 2024-03-23T09:42:51.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.309+0000 7f54143a4640 1 -- start start 2024-03-23T09:42:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f54143a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f540c0b8300 0x7f540c15b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f54143a4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c15cb40 0x7f540c1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f54143a4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f540c160cb0 0x7f540c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f54143a4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c159190 con 0x7f540c15cb40 2024-03-23T09:42:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f54143a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c159470 con 0x7f540c0b8300 2024-03-23T09:42:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f54143a4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c159750 con 0x7f540c160cb0 2024-03-23T09:42:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f541193f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c15cb40 0x7f540c1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f541193f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c15cb40 0x7f540c1587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39896/0 (socket says 172.21.15.47:39896) 2024-03-23T09:42:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f541193f640 1 -- 172.21.15.47:0/2898536254 learned_addr learned my addr 172.21.15.47:0/2898536254 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f5412140640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f540c0b8300 0x7f540c15b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f5412941640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f540c160cb0 0x7f540c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f541193f640 1 -- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f540c160cb0 msgr2=0x7f540c15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f541193f640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f540c160cb0 0x7f540c15f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f541193f640 1 -- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f540c0b8300 msgr2=0x7f540c15b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f541193f640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f540c0b8300 0x7f540c15b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f541193f640 1 -- 172.21.15.47:0/2898536254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53fc009b30 con 0x7f540c15cb40 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f541193f640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c15cb40 0x7f540c1587f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f540000c3f0 tx=0x7f540000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f540affd640 1 -- 172.21.15.47:0/2898536254 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54000035f0 con 0x7f540c15cb40 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f540affd640 1 -- 172.21.15.47:0/2898536254 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5400003750 con 0x7f540c15cb40 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f54143a4640 1 -- 172.21.15.47:0/2898536254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540c159a30 con 0x7f540c15cb40 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f540affd640 1 -- 172.21.15.47:0/2898536254 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400003b00 con 0x7f540c15cb40 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.310+0000 7f54143a4640 1 -- 172.21.15.47:0/2898536254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540c15b840 con 0x7f540c15cb40 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.311+0000 7f54143a4640 1 -- 172.21.15.47:0/2898536254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540c072360 con 0x7f540c15cb40 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.312+0000 7f540affd640 1 -- 172.21.15.47:0/2898536254 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5400003c60 con 0x7f540c15cb40 2024-03-23T09:42:51.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.312+0000 7f540affd640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f53f806ac00 0x7f53f806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.312+0000 7f540affd640 1 -- 172.21.15.47:0/2898536254 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5400088870 con 0x7f540c15cb40 2024-03-23T09:42:51.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.313+0000 7f5412140640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f53f806ac00 0x7f53f806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:51.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.313+0000 7f5412140640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f53f806ac00 0x7f53f806d0c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5404009890 tx=0x7f5404008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:51.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.316+0000 7f540affd640 1 -- 172.21.15.47:0/2898536254 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5400053b00 con 0x7f540c15cb40 2024-03-23T09:42:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.492+0000 7f54143a4640 1 -- 172.21.15.47:0/2898536254 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f540c0d0b70 con 0x7f53f806ac00 2024-03-23T09:42:51.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.499+0000 7f540affd640 1 -- 172.21.15.47:0/2898536254 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278383 (secure 0 0 0) 0x7f5404009480 con 0x7f53f806ac00 2024-03-23T09:42:51.504 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:51.506 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:42:51.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.502+0000 7f54143a4640 1 -- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f53f806ac00 msgr2=0x7f53f806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:51.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.502+0000 7f54143a4640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f53f806ac00 0x7f53f806d0c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5404009890 tx=0x7f5404008040 comp rx=0 tx=0).stop 2024-03-23T09:42:51.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.502+0000 7f54143a4640 1 -- 172.21.15.47:0/2898536254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c15cb40 msgr2=0x7f540c1587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:51.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.502+0000 7f54143a4640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c15cb40 0x7f540c1587f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f540000c3f0 tx=0x7f540000c8b0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.502+0000 7f54143a4640 1 -- 172.21.15.47:0/2898536254 shutdown_connections 2024-03-23T09:42:51.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.502+0000 7f54143a4640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f53f806ac00 0x7f53f806d0c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.502+0000 7f54143a4640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f540c160cb0 0x7f540c15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.502+0000 7f54143a4640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c15cb40 0x7f540c1587f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.502+0000 7f54143a4640 1 --2- 172.21.15.47:0/2898536254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f540c0b8300 0x7f540c15b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.502+0000 7f54143a4640 1 -- 172.21.15.47:0/2898536254 >> 172.21.15.47:0/2898536254 conn(0x7f540c06b160 msgr2=0x7f540c0cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:51.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.502+0000 7f54143a4640 1 -- 172.21.15.47:0/2898536254 shutdown_connections 2024-03-23T09:42:51.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.502+0000 7f54143a4640 1 -- 172.21.15.47:0/2898536254 wait complete. 2024-03-23T09:42:51.512 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-03-23T09:42:50.364075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":237592,"kb_used_data":5040,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737128,"statfs":{"total":773094113280,"available":772850819072,"internally_reserved":0,"allocated":5160960,"data_stored":2752870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000781"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792350+0000","last_change":"2024-03-23T09:42:46.792350+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792350+0000","last_undegraded":"2024-03-23T09:42:46.792350+0000","last_fullsized":"2024-03-23T09:42:46.792350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:41.236379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791904+0000","last_change":"2024-03-23T09:42:46.791904+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791904+0000","last_undegraded":"2024-03-23T09:42:46.791904+0000","last_fullsized":"2024-03-23T09:42:46.791904+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793978+0000","last_change":"2024-03-23T09:42:46.793978+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793978+0000","last_undegraded":"2024-03-23T09:42:46.793978+0000","last_fullsized":"2024-03-23T09:42:46.793978+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:21.810507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791103+0000","last_change":"2024-03-23T09:42:46.791103+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791103+0000","last_undegraded":"2024-03-23T09:42:46.791103+0000","last_fullsized":"2024-03-23T09:42:46.791103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:04:24.190076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793327+0000","last_change":"2024-03-23T09:42:46.793327+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793327+0000","last_undegraded":"2024-03-23T09:42:46.793327+0000","last_fullsized":"2024-03-23T09:42:46.793327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:40.886514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791385+0000","last_change":"2024-03-23T09:42:46.791385+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791385+0000","last_undegraded":"2024-03-23T09:42:46.791385+0000","last_fullsized":"2024-03-23T09:42:46.791385+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:58:54.380376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.794200+0000","last_change":"2024-03-23T09:42:46.794200+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.794200+0000","last_undegraded":"2024-03-23T09:42:46.794200+0000","last_fullsized":"2024-03-23T09:42:46.794200+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:04:53.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.796428+0000","last_change":"2024-03-23T09:42:46.796428+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.796428+0000","last_undegraded":"2024-03-23T09:42:46.796428+0000","last_fullsized":"2024-03-23T09:42:46.796428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790630+0000","last_change":"2024-03-23T09:42:46.790630+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790630+0000","last_undegraded":"2024-03-23T09:42:46.790630+0000","last_fullsized":"2024-03-23T09:42:46.790630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:01:51.451920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795158+0000","last_change":"2024-03-23T09:42:46.795158+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795158+0000","last_undegraded":"2024-03-23T09:42:46.795158+0000","last_fullsized":"2024-03-23T09:42:46.795158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:02:27.446246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792484+0000","last_change":"2024-03-23T09:42:46.792484+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792484+0000","last_undegraded":"2024-03-23T09:42:46.792484+0000","last_fullsized":"2024-03-23T09:42:46.792484+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:35:53.515036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790019+0000","last_change":"2024-03-23T09:42:46.790019+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790019+0000","last_undegraded":"2024-03-23T09:42:46.790019+0000","last_fullsized":"2024-03-23T09:42:46.790019+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792903+0000","last_change":"2024-03-23T09:42:46.792903+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792903+0000","last_undegraded":"2024-03-23T09:42:46.792903+0000","last_fullsized":"2024-03-23T09:42:46.792903+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:43:42.174052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791816+0000","last_change":"2024-03-23T09:42:46.791816+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791816+0000","last_undegraded":"2024-03-23T09:42:46.791816+0000","last_fullsized":"2024-03-23T09:42:46.791816+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:32.470235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792213+0000","last_change":"2024-03-23T09:42:46.792213+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792213+0000","last_undegraded":"2024-03-23T09:42:46.792213+0000","last_fullsized":"2024-03-23T09:42:46.792213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:40:05.822261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795999+0000","last_change":"2024-03-23T09:42:46.795999+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795999+0000","last_undegraded":"2024-03-23T09:42:46.795999+0000","last_fullsized":"2024-03-23T09:42:46.795999+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:01:51.451920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790079+0000","last_change":"2024-03-23T09:42:46.790079+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790079+0000","last_undegraded":"2024-03-23T09:42:46.790079+0000","last_fullsized":"2024-03-23T09:42:46.790079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:19.815833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791201+0000","last_change":"2024-03-23T09:42:46.791201+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791201+0000","last_undegraded":"2024-03-23T09:42:46.791201+0000","last_fullsized":"2024-03-23T09:42:46.791201+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:19.815833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793862+0000","last_change":"2024-03-23T09:42:46.793862+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793862+0000","last_undegraded":"2024-03-23T09:42:46.793862+0000","last_fullsized":"2024-03-23T09:42:46.793862+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:47:48.322837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792271+0000","last_change":"2024-03-23T09:42:46.792271+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792271+0000","last_undegraded":"2024-03-23T09:42:46.792271+0000","last_fullsized":"2024-03-23T09:42:46.792271+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:29:32.796473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792275+0000","last_change":"2024-03-23T09:42:46.792275+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792275+0000","last_undegraded":"2024-03-23T09:42:46.792275+0000","last_fullsized":"2024-03-23T09:42:46.792275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:46.555393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793510+0000","last_change":"2024-03-23T09:42:46.793510+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793510+0000","last_undegraded":"2024-03-23T09:42:46.793510+0000","last_fullsized":"2024-03-23T09:42:46.793510+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:14:55.790367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795084+0000","last_change":"2024-03-23T09:42:46.795084+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795084+0000","last_undegraded":"2024-03-23T09:42:46.795084+0000","last_fullsized":"2024-03-23T09:42:46.795084+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:09:03.354548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791854+0000","last_change":"2024-03-23T09:42:46.791854+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791854+0000","last_undegraded":"2024-03-23T09:42:46.791854+0000","last_fullsized":"2024-03-23T09:42:46.791854+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:50:10.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792263+0000","last_change":"2024-03-23T09:42:46.792263+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792263+0000","last_undegraded":"2024-03-23T09:42:46.792263+0000","last_fullsized":"2024-03-23T09:42:46.792263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:27:56.832097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792046+0000","last_change":"2024-03-23T09:42:46.792046+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792046+0000","last_undegraded":"2024-03-23T09:42:46.792046+0000","last_fullsized":"2024-03-23T09:42:46.792046+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:27:56.832097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793721+0000","last_change":"2024-03-23T09:42:46.793721+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793721+0000","last_undegraded":"2024-03-23T09:42:46.793721+0000","last_fullsized":"2024-03-23T09:42:46.793721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:37.970066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791095+0000","last_change":"2024-03-23T09:42:46.791095+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791095+0000","last_undegraded":"2024-03-23T09:42:46.791095+0000","last_fullsized":"2024-03-23T09:42:46.791095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:43.932184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790630+0000","last_change":"2024-03-23T09:42:46.790630+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790630+0000","last_undegraded":"2024-03-23T09:42:46.790630+0000","last_fullsized":"2024-03-23T09:42:46.790630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:02.796426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.794308+0000","last_change":"2024-03-23T09:42:46.794308+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.794308+0000","last_undegraded":"2024-03-23T09:42:46.794308+0000","last_fullsized":"2024-03-23T09:42:46.794308+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:24:36.255934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795352+0000","last_change":"2024-03-23T09:42:46.795352+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795352+0000","last_undegraded":"2024-03-23T09:42:46.795352+0000","last_fullsized":"2024-03-23T09:42:46.795352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:02.796426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793582+0000","last_change":"2024-03-23T09:42:46.793582+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793582+0000","last_undegraded":"2024-03-23T09:42:46.793582+0000","last_fullsized":"2024-03-23T09:42:46.793582+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:04:09.114370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":186,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-23T09:42:44.417082+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:42:44.417082+0000","last_peered":"2024-03-23T09:42:44.417082+0000","last_clean":"2024-03-23T09:42:44.417082+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:42:44.417082+0000","last_undegraded":"2024-03-23T09:42:44.417082+0000","last_fullsized":"2024-03-23T09:42:44.417082+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1097728,"data_stored":789948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54400000000000004}]},{"osd":1,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.50900000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.47699999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27500,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":487424,"data_stored":193144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.52100000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.56299999999999994}]},{"osd":2,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.50700000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.433}]},{"osd":4,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48799999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.45600000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":199580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.439},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.53100000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.51300000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.67600000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.73799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":199580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69899999999999995}]},{"osd":1,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.66300000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.747}]},{"osd":3,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.48599999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.502}]},{"osd":7,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":491520,"data_stored":193763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.19},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.30299999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.223}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.318},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.29199999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.503}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60499999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":503808,"data_stored":199580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.34799999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.316}]},{"osd":3,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.248}]},{"osd":4,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.41999999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.56599999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.436}]},{"osd":7,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.47399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":491520,"data_stored":193763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.441},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":0.42299999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.85199999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71799999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.65600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1081344,"data_stored":783512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.247},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.29499999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.314},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.34100000000000003}]},{"osd":3,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.35399999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.625}]},{"osd":5,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65300000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.66800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:42:51.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:42:51.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.654+0000 7f9bfea96640 1 Processor -- start 2024-03-23T09:42:51.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.654+0000 7f9bfea96640 1 -- start start 2024-03-23T09:42:51.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.654+0000 7f9bfea96640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9bf80cec50 0x7f9bf80cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.654+0000 7f9bfea96640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bf80b0b90 0x7f9bf80b1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.654+0000 7f9bfea96640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf80cc610 con 0x7f9bf815a8d0 2024-03-23T09:42:51.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.654+0000 7f9bfea96640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf80cf640 con 0x7f9bf80cec50 2024-03-23T09:42:51.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.654+0000 7f9bfea96640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80c41c0 con 0x7f9bf80b0b90 2024-03-23T09:42:51.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.654+0000 7f9bfc832640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9bf815a8d0 0x7f9bf80cb1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41802/0 (socket says 172.21.15.47:41802) 2024-03-23T09:42:51.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.654+0000 7f9bfc832640 1 -- 172.21.15.47:0/4135605794 learned_addr learned my addr 172.21.15.47:0/4135605794 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:51.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bfd033640 1 --2- 172.21.15.47:0/4135605794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bf80b0b90 0x7f9bf80b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:51.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf7fff640 1 --2- 172.21.15.47:0/4135605794 >> v2:172.21.15.175:3300/0 conn(0x7f9bf80cec50 0x7f9bf80cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:51.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2945345312 0 0) 0x7f9bf80cc610 con 0x7f9bf815a8d0 2024-03-23T09:42:51.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9be0003520 con 0x7f9bf815a8d0 2024-03-23T09:42:51.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3098312123 0 0) 0x7f9be0003520 con 0x7f9bf815a8d0 2024-03-23T09:42:51.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bf80b0b90 msgr2=0x7f9bf80b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:51.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 --2- 172.21.15.47:0/4135605794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bf80b0b90 0x7f9bf80b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 >> v2:172.21.15.175:3300/0 conn(0x7f9bf80cec50 msgr2=0x7f9bf80cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:51.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 --2- 172.21.15.47:0/4135605794 >> v2:172.21.15.175:3300/0 conn(0x7f9bf80cec50 0x7f9bf80cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80b0700 con 0x7f9bf815a8d0 2024-03-23T09:42:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f9bf00019f0 con 0x7f9bf815a8d0 2024-03-23T09:42:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf7fff640 1 --2- 172.21.15.47:0/4135605794 >> v2:172.21.15.175:3300/0 conn(0x7f9bf80cec50 0x7f9bf80cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:42:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 >> v1:172.21.15.47:6789/0 conn(0x7f9bf815a8d0 legacy=0x7f9bf80cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 --2- 172.21.15.47:0/4135605794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9be0003970 0x7f9be0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 --2- 172.21.15.47:0/4135605794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9be00047c0 0x7f9be0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 --2- 172.21.15.47:0/4135605794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bf80b0b90 0x7f9be0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80cf640 con 0x7f9be00047c0 2024-03-23T09:42:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be000b980 con 0x7f9be0003970 2024-03-23T09:42:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be000baf0 con 0x7f9bf80b0b90 2024-03-23T09:42:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf7fff640 1 --2- 172.21.15.47:0/4135605794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9be00047c0 0x7f9be0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bfc832640 1 --2- 172.21.15.47:0/4135605794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9be0003970 0x7f9be0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bfd033640 1 --2- 172.21.15.47:0/4135605794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bf80b0b90 0x7f9be0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf7fff640 1 -- 172.21.15.47:0/4135605794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bf80b0b90 msgr2=0x7f9be0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf7fff640 1 --2- 172.21.15.47:0/4135605794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bf80b0b90 0x7f9be0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf7fff640 1 -- 172.21.15.47:0/4135605794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9be0003970 msgr2=0x7f9be0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf7fff640 1 --2- 172.21.15.47:0/4135605794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9be0003970 0x7f9be0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bf7fff640 1 -- 172.21.15.47:0/4135605794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be0003520 con 0x7f9be00047c0 2024-03-23T09:42:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.655+0000 7f9bfd033640 1 --2- 172.21.15.47:0/4135605794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bf80b0b90 0x7f9be0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.656+0000 7f9bf7fff640 1 --2- 172.21.15.47:0/4135605794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9be00047c0 0x7f9be0004c40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009c90 tx=0x7f9be00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0db70fbea7f262 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.656+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec02c980 con 0x7f9be00047c0 2024-03-23T09:42:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.656+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec02cae0 con 0x7f9be00047c0 2024-03-23T09:42:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.656+0000 7f9bf77fe640 1 -- 172.21.15.47:0/4135605794 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec02ce50 con 0x7f9be00047c0 2024-03-23T09:42:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.657+0000 7f9bfea96640 1 -- 172.21.15.47:0/4135605794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9be00047c0 msgr2=0x7f9be0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.657+0000 7f9bfea96640 1 --2- 172.21.15.47:0/4135605794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9be00047c0 0x7f9be0004c40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009c90 tx=0x7f9be00057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.657+0000 7f9bfd033640 1 -- 172.21.15.47:0/4135605794 reap_dead start 2024-03-23T09:42:51.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.657+0000 7f9bfea96640 1 -- 172.21.15.47:0/4135605794 shutdown_connections 2024-03-23T09:42:51.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.657+0000 7f9bfea96640 1 -- 172.21.15.47:0/4135605794 >> 172.21.15.47:0/4135605794 conn(0x7f9bf80d0670 msgr2=0x7f9bf80d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:51.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.657+0000 7f9bfea96640 1 -- 172.21.15.47:0/4135605794 shutdown_connections 2024-03-23T09:42:51.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.657+0000 7f9bfea96640 1 -- 172.21.15.47:0/4135605794 wait complete. 2024-03-23T09:42:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.658+0000 7f9bfea96640 1 Processor -- start 2024-03-23T09:42:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.658+0000 7f9bfea96640 1 -- start start 2024-03-23T09:42:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfea96640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9be00047c0 0x7f9bf80c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfea96640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bf814fc90 0x7f9bf80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfea96640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bf80c4160 0x7f9bf80c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfea96640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80c68d0 con 0x7f9be00047c0 2024-03-23T09:42:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfea96640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80c8bb0 con 0x7f9bf80c4160 2024-03-23T09:42:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfea96640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80c8e90 con 0x7f9bf814fc90 2024-03-23T09:42:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfc832640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9be00047c0 0x7f9bf80c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfc832640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9be00047c0 0x7f9bf80c6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39910/0 (socket says 172.21.15.47:39910) 2024-03-23T09:42:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfc832640 1 -- 172.21.15.47:0/2914846260 learned_addr learned my addr 172.21.15.47:0/2914846260 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfc832640 1 -- 172.21.15.47:0/2914846260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bf814fc90 msgr2=0x7f9bf80b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfc832640 1 --2- 172.21.15.47:0/2914846260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bf814fc90 0x7f9bf80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfc832640 1 -- 172.21.15.47:0/2914846260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bf80c4160 msgr2=0x7f9bf80c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfc832640 1 --2- 172.21.15.47:0/2914846260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bf80c4160 0x7f9bf80c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfc832640 1 -- 172.21.15.47:0/2914846260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80b5b80 con 0x7f9be00047c0 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.659+0000 7f9bfc832640 1 --2- 172.21.15.47:0/2914846260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9be00047c0 0x7f9bf80c6ae0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00079f0 tx=0x7f9bf000a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.660+0000 7f9bf57fa640 1 -- 172.21.15.47:0/2914846260 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0001a30 con 0x7f9be00047c0 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.660+0000 7f9bf57fa640 1 -- 172.21.15.47:0/2914846260 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf00133f0 con 0x7f9be00047c0 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.660+0000 7f9bfea96640 1 -- 172.21.15.47:0/2914846260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf80b5e60 con 0x7f9be00047c0 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.660+0000 7f9bf57fa640 1 -- 172.21.15.47:0/2914846260 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0013780 con 0x7f9be00047c0 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.660+0000 7f9bfea96640 1 -- 172.21.15.47:0/2914846260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf8150110 con 0x7f9be00047c0 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.661+0000 7f9bfea96640 1 -- 172.21.15.47:0/2914846260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf80672f0 con 0x7f9be00047c0 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.661+0000 7f9bf57fa640 1 -- 172.21.15.47:0/2914846260 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9bf00138e0 con 0x7f9be00047c0 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.661+0000 7f9bf57fa640 1 --2- 172.21.15.47:0/2914846260 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9be406ac00 0x7f9be406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.661+0000 7f9bf57fa640 1 -- 172.21.15.47:0/2914846260 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bf0089fe0 con 0x7f9be00047c0 2024-03-23T09:42:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.662+0000 7f9bf7fff640 1 --2- 172.21.15.47:0/2914846260 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9be406ac00 0x7f9be406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:51.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.662+0000 7f9bf7fff640 1 --2- 172.21.15.47:0/2914846260 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9be406ac00 0x7f9be406d0c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9bec036000 tx=0x7f9bec02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:51.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.666+0000 7f9bf57fa640 1 -- 172.21.15.47:0/2914846260 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9bf00551c0 con 0x7f9be00047c0 2024-03-23T09:42:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.846+0000 7f9bfea96640 1 -- 172.21.15.47:0/2914846260 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9bf80cc330 con 0x7f9be406ac00 2024-03-23T09:42:51.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.854+0000 7f9bf57fa640 1 -- 172.21.15.47:0/2914846260 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278383 (secure 0 0 0) 0x7f9bec034440 con 0x7f9be406ac00 2024-03-23T09:42:51.856 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:51.857 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:42:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.856+0000 7f9bfea96640 1 -- 172.21.15.47:0/2914846260 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9be406ac00 msgr2=0x7f9be406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.856+0000 7f9bfea96640 1 --2- 172.21.15.47:0/2914846260 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9be406ac00 0x7f9be406d0c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9bec036000 tx=0x7f9bec02e000 comp rx=0 tx=0).stop 2024-03-23T09:42:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.856+0000 7f9bfea96640 1 -- 172.21.15.47:0/2914846260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9be00047c0 msgr2=0x7f9bf80c6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.856+0000 7f9bfea96640 1 --2- 172.21.15.47:0/2914846260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9be00047c0 0x7f9bf80c6ae0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00079f0 tx=0x7f9bf000a730 comp rx=0 tx=0).stop 2024-03-23T09:42:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.857+0000 7f9bfea96640 1 -- 172.21.15.47:0/2914846260 shutdown_connections 2024-03-23T09:42:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.857+0000 7f9bfea96640 1 --2- 172.21.15.47:0/2914846260 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9be406ac00 0x7f9be406d0c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.857+0000 7f9bfea96640 1 --2- 172.21.15.47:0/2914846260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bf80c4160 0x7f9bf80c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.857+0000 7f9bfea96640 1 --2- 172.21.15.47:0/2914846260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bf814fc90 0x7f9bf80b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.857+0000 7f9bfea96640 1 --2- 172.21.15.47:0/2914846260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9be00047c0 0x7f9bf80c6ae0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.857+0000 7f9bfea96640 1 -- 172.21.15.47:0/2914846260 >> 172.21.15.47:0/2914846260 conn(0x7f9bf80d0670 msgr2=0x7f9bf80cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.857+0000 7f9bfea96640 1 -- 172.21.15.47:0/2914846260 shutdown_connections 2024-03-23T09:42:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:51.857+0000 7f9bfea96640 1 -- 172.21.15.47:0/2914846260 wait complete. 2024-03-23T09:42:51.867 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-03-23T09:42:50.364075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":237592,"kb_used_data":5040,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737128,"statfs":{"total":773094113280,"available":772850819072,"internally_reserved":0,"allocated":5160960,"data_stored":2752870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000781"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792350+0000","last_change":"2024-03-23T09:42:46.792350+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792350+0000","last_undegraded":"2024-03-23T09:42:46.792350+0000","last_fullsized":"2024-03-23T09:42:46.792350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:41.236379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791904+0000","last_change":"2024-03-23T09:42:46.791904+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791904+0000","last_undegraded":"2024-03-23T09:42:46.791904+0000","last_fullsized":"2024-03-23T09:42:46.791904+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793978+0000","last_change":"2024-03-23T09:42:46.793978+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793978+0000","last_undegraded":"2024-03-23T09:42:46.793978+0000","last_fullsized":"2024-03-23T09:42:46.793978+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:21.810507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791103+0000","last_change":"2024-03-23T09:42:46.791103+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791103+0000","last_undegraded":"2024-03-23T09:42:46.791103+0000","last_fullsized":"2024-03-23T09:42:46.791103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:04:24.190076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793327+0000","last_change":"2024-03-23T09:42:46.793327+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793327+0000","last_undegraded":"2024-03-23T09:42:46.793327+0000","last_fullsized":"2024-03-23T09:42:46.793327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:40.886514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791385+0000","last_change":"2024-03-23T09:42:46.791385+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791385+0000","last_undegraded":"2024-03-23T09:42:46.791385+0000","last_fullsized":"2024-03-23T09:42:46.791385+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:58:54.380376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.794200+0000","last_change":"2024-03-23T09:42:46.794200+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.794200+0000","last_undegraded":"2024-03-23T09:42:46.794200+0000","last_fullsized":"2024-03-23T09:42:46.794200+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:04:53.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.796428+0000","last_change":"2024-03-23T09:42:46.796428+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.796428+0000","last_undegraded":"2024-03-23T09:42:46.796428+0000","last_fullsized":"2024-03-23T09:42:46.796428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790630+0000","last_change":"2024-03-23T09:42:46.790630+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790630+0000","last_undegraded":"2024-03-23T09:42:46.790630+0000","last_fullsized":"2024-03-23T09:42:46.790630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:01:51.451920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795158+0000","last_change":"2024-03-23T09:42:46.795158+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795158+0000","last_undegraded":"2024-03-23T09:42:46.795158+0000","last_fullsized":"2024-03-23T09:42:46.795158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:02:27.446246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792484+0000","last_change":"2024-03-23T09:42:46.792484+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792484+0000","last_undegraded":"2024-03-23T09:42:46.792484+0000","last_fullsized":"2024-03-23T09:42:46.792484+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:35:53.515036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790019+0000","last_change":"2024-03-23T09:42:46.790019+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790019+0000","last_undegraded":"2024-03-23T09:42:46.790019+0000","last_fullsized":"2024-03-23T09:42:46.790019+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792903+0000","last_change":"2024-03-23T09:42:46.792903+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792903+0000","last_undegraded":"2024-03-23T09:42:46.792903+0000","last_fullsized":"2024-03-23T09:42:46.792903+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:43:42.174052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791816+0000","last_change":"2024-03-23T09:42:46.791816+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791816+0000","last_undegraded":"2024-03-23T09:42:46.791816+0000","last_fullsized":"2024-03-23T09:42:46.791816+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:32.470235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792213+0000","last_change":"2024-03-23T09:42:46.792213+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792213+0000","last_undegraded":"2024-03-23T09:42:46.792213+0000","last_fullsized":"2024-03-23T09:42:46.792213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:40:05.822261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795999+0000","last_change":"2024-03-23T09:42:46.795999+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795999+0000","last_undegraded":"2024-03-23T09:42:46.795999+0000","last_fullsized":"2024-03-23T09:42:46.795999+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:01:51.451920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790079+0000","last_change":"2024-03-23T09:42:46.790079+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790079+0000","last_undegraded":"2024-03-23T09:42:46.790079+0000","last_fullsized":"2024-03-23T09:42:46.790079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:19.815833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791201+0000","last_change":"2024-03-23T09:42:46.791201+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791201+0000","last_undegraded":"2024-03-23T09:42:46.791201+0000","last_fullsized":"2024-03-23T09:42:46.791201+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:19.815833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793862+0000","last_change":"2024-03-23T09:42:46.793862+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793862+0000","last_undegraded":"2024-03-23T09:42:46.793862+0000","last_fullsized":"2024-03-23T09:42:46.793862+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:47:48.322837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792271+0000","last_change":"2024-03-23T09:42:46.792271+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792271+0000","last_undegraded":"2024-03-23T09:42:46.792271+0000","last_fullsized":"2024-03-23T09:42:46.792271+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:29:32.796473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792275+0000","last_change":"2024-03-23T09:42:46.792275+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792275+0000","last_undegraded":"2024-03-23T09:42:46.792275+0000","last_fullsized":"2024-03-23T09:42:46.792275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:46.555393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793510+0000","last_change":"2024-03-23T09:42:46.793510+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793510+0000","last_undegraded":"2024-03-23T09:42:46.793510+0000","last_fullsized":"2024-03-23T09:42:46.793510+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:14:55.790367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795084+0000","last_change":"2024-03-23T09:42:46.795084+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795084+0000","last_undegraded":"2024-03-23T09:42:46.795084+0000","last_fullsized":"2024-03-23T09:42:46.795084+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:09:03.354548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791854+0000","last_change":"2024-03-23T09:42:46.791854+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791854+0000","last_undegraded":"2024-03-23T09:42:46.791854+0000","last_fullsized":"2024-03-23T09:42:46.791854+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:50:10.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792263+0000","last_change":"2024-03-23T09:42:46.792263+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792263+0000","last_undegraded":"2024-03-23T09:42:46.792263+0000","last_fullsized":"2024-03-23T09:42:46.792263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:27:56.832097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792046+0000","last_change":"2024-03-23T09:42:46.792046+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792046+0000","last_undegraded":"2024-03-23T09:42:46.792046+0000","last_fullsized":"2024-03-23T09:42:46.792046+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:27:56.832097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793721+0000","last_change":"2024-03-23T09:42:46.793721+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793721+0000","last_undegraded":"2024-03-23T09:42:46.793721+0000","last_fullsized":"2024-03-23T09:42:46.793721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:37.970066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791095+0000","last_change":"2024-03-23T09:42:46.791095+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791095+0000","last_undegraded":"2024-03-23T09:42:46.791095+0000","last_fullsized":"2024-03-23T09:42:46.791095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:43.932184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790630+0000","last_change":"2024-03-23T09:42:46.790630+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790630+0000","last_undegraded":"2024-03-23T09:42:46.790630+0000","last_fullsized":"2024-03-23T09:42:46.790630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:02.796426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.794308+0000","last_change":"2024-03-23T09:42:46.794308+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.794308+0000","last_undegraded":"2024-03-23T09:42:46.794308+0000","last_fullsized":"2024-03-23T09:42:46.794308+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:24:36.255934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795352+0000","last_change":"2024-03-23T09:42:46.795352+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795352+0000","last_undegraded":"2024-03-23T09:42:46.795352+0000","last_fullsized":"2024-03-23T09:42:46.795352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:02.796426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793582+0000","last_change":"2024-03-23T09:42:46.793582+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793582+0000","last_undegraded":"2024-03-23T09:42:46.793582+0000","last_fullsized":"2024-03-23T09:42:46.793582+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:04:09.114370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":186,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-23T09:42:44.417082+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:42:44.417082+0000","last_peered":"2024-03-23T09:42:44.417082+0000","last_clean":"2024-03-23T09:42:44.417082+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:42:44.417082+0000","last_undegraded":"2024-03-23T09:42:44.417082+0000","last_fullsized":"2024-03-23T09:42:44.417082+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1097728,"data_stored":789948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54400000000000004}]},{"osd":1,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.50900000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.47699999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27500,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":487424,"data_stored":193144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.52100000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.56299999999999994}]},{"osd":2,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.50700000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.433}]},{"osd":4,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48799999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.45600000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":199580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.439},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.53100000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.51300000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.67600000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.73799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":199580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69899999999999995}]},{"osd":1,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.66300000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.747}]},{"osd":3,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.48599999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.502}]},{"osd":7,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":491520,"data_stored":193763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.19},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.30299999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.223}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.318},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.29199999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.503}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60499999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":503808,"data_stored":199580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.34799999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.316}]},{"osd":3,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.248}]},{"osd":4,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.41999999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.56599999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.436}]},{"osd":7,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.47399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":491520,"data_stored":193763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.441},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":0.42299999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.85199999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71799999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.65600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1081344,"data_stored":783512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.247},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.29499999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.314},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.34100000000000003}]},{"osd":3,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.35399999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.625}]},{"osd":5,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65300000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.66800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:42:51.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2024-03-23T09:42:51.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2024-03-23T09:42:51.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2024-03-23T09:42:51.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2024-03-23T09:42:51.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2024-03-23T09:42:51.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2024-03-23T09:42:51.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2024-03-23T09:42:51.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2024-03-23T09:42:51.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2024-03-23T09:42:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2024-03-23T09:42:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2024-03-23T09:42:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2024-03-23T09:42:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2024-03-23T09:42:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2024-03-23T09:42:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-03-23T09:42:51.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2024-03-23T09:42:51.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2024-03-23T09:42:51.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2024-03-23T09:42:51.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2024-03-23T09:42:51.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2024-03-23T09:42:51.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-03-23T09:42:51.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2024-03-23T09:42:51.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2024-03-23T09:42:51.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2024-03-23T09:42:51.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2024-03-23T09:42:51.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-03-23T09:42:51.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2024-03-23T09:42:51.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2024-03-23T09:42:51.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2024-03-23T09:42:51.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2024-03-23T09:42:51.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2024-03-23T09:42:51.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-03-23T09:42:51.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-03-23T09:42:51.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-03-23T09:42:51.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.792350+0000', 'last_change': '2024-03-23T09:42:46.792350+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.792350+0000', 'last_undegraded': '2024-03-23T09:42:46.792350+0000', 'last_fullsized': '2024-03-23T09:42:46.792350+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T15:07:41.236379+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:42:51.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-03-23T09:42:51.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-03-23T09:42:51.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.791904+0000', 'last_change': '2024-03-23T09:42:46.791904+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.791904+0000', 'last_undegraded': '2024-03-23T09:42:46.791904+0000', 'last_fullsized': '2024-03-23T09:42:46.791904+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:42:51.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-03-23T09:42:51.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-03-23T09:42:51.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.793978+0000', 'last_change': '2024-03-23T09:42:46.793978+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.793978+0000', 'last_undegraded': '2024-03-23T09:42:46.793978+0000', 'last_fullsized': '2024-03-23T09:42:46.793978+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:56:21.810507+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:42:51.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-03-23T09:42:51.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-03-23T09:42:51.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.791103+0000', 'last_change': '2024-03-23T09:42:46.791103+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.791103+0000', 'last_undegraded': '2024-03-23T09:42:46.791103+0000', 'last_fullsized': '2024-03-23T09:42:46.791103+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T13:04:24.190076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:42:51.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-03-23T09:42:51.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-03-23T09:42:51.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.793327+0000', 'last_change': '2024-03-23T09:42:46.793327+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.793327+0000', 'last_undegraded': '2024-03-23T09:42:46.793327+0000', 'last_fullsized': '2024-03-23T09:42:46.793327+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:34:40.886514+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:42:51.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-03-23T09:42:51.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-03-23T09:42:51.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.791385+0000', 'last_change': '2024-03-23T09:42:46.791385+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.791385+0000', 'last_undegraded': '2024-03-23T09:42:46.791385+0000', 'last_fullsized': '2024-03-23T09:42:46.791385+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:58:54.380376+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:42:51.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-03-23T09:42:51.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-03-23T09:42:51.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.794200+0000', 'last_change': '2024-03-23T09:42:46.794200+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.794200+0000', 'last_undegraded': '2024-03-23T09:42:46.794200+0000', 'last_fullsized': '2024-03-23T09:42:46.794200+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:04:53.462828+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:42:51.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-03-23T09:42:51.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-03-23T09:42:51.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.796428+0000', 'last_change': '2024-03-23T09:42:46.796428+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.796428+0000', 'last_undegraded': '2024-03-23T09:42:46.796428+0000', 'last_fullsized': '2024-03-23T09:42:46.796428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:42:51.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-03-23T09:42:51.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-03-23T09:42:51.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.790630+0000', 'last_change': '2024-03-23T09:42:46.790630+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.790630+0000', 'last_undegraded': '2024-03-23T09:42:46.790630+0000', 'last_fullsized': '2024-03-23T09:42:46.790630+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:01:51.451920+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:42:51.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-03-23T09:42:51.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-03-23T09:42:51.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.795158+0000', 'last_change': '2024-03-23T09:42:46.795158+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.795158+0000', 'last_undegraded': '2024-03-23T09:42:46.795158+0000', 'last_fullsized': '2024-03-23T09:42:46.795158+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:02:27.446246+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:42:51.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-03-23T09:42:51.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-03-23T09:42:51.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.792484+0000', 'last_change': '2024-03-23T09:42:46.792484+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.792484+0000', 'last_undegraded': '2024-03-23T09:42:46.792484+0000', 'last_fullsized': '2024-03-23T09:42:46.792484+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:35:53.515036+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:42:51.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-03-23T09:42:51.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-03-23T09:42:51.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.790019+0000', 'last_change': '2024-03-23T09:42:46.790019+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.790019+0000', 'last_undegraded': '2024-03-23T09:42:46.790019+0000', 'last_fullsized': '2024-03-23T09:42:46.790019+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:42:51.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-03-23T09:42:51.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-03-23T09:42:51.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.792903+0000', 'last_change': '2024-03-23T09:42:46.792903+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.792903+0000', 'last_undegraded': '2024-03-23T09:42:46.792903+0000', 'last_fullsized': '2024-03-23T09:42:46.792903+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:43:42.174052+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:42:51.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-03-23T09:42:51.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-03-23T09:42:51.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.791816+0000', 'last_change': '2024-03-23T09:42:46.791816+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.791816+0000', 'last_undegraded': '2024-03-23T09:42:46.791816+0000', 'last_fullsized': '2024-03-23T09:42:46.791816+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T13:10:32.470235+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:42:51.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-03-23T09:42:51.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-03-23T09:42:51.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.792213+0000', 'last_change': '2024-03-23T09:42:46.792213+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.792213+0000', 'last_undegraded': '2024-03-23T09:42:46.792213+0000', 'last_fullsized': '2024-03-23T09:42:46.792213+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T21:40:05.822261+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:42:51.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-03-23T09:42:51.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-03-23T09:42:51.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.795999+0000', 'last_change': '2024-03-23T09:42:46.795999+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.795999+0000', 'last_undegraded': '2024-03-23T09:42:46.795999+0000', 'last_fullsized': '2024-03-23T09:42:46.795999+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:01:51.451920+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:42:51.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-03-23T09:42:51.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-03-23T09:42:51.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.790079+0000', 'last_change': '2024-03-23T09:42:46.790079+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.790079+0000', 'last_undegraded': '2024-03-23T09:42:46.790079+0000', 'last_fullsized': '2024-03-23T09:42:46.790079+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:00:19.815833+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:42:51.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-03-23T09:42:51.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-03-23T09:42:51.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.791201+0000', 'last_change': '2024-03-23T09:42:46.791201+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.791201+0000', 'last_undegraded': '2024-03-23T09:42:46.791201+0000', 'last_fullsized': '2024-03-23T09:42:46.791201+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:00:19.815833+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:42:51.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-03-23T09:42:51.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-03-23T09:42:51.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.793862+0000', 'last_change': '2024-03-23T09:42:46.793862+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.793862+0000', 'last_undegraded': '2024-03-23T09:42:46.793862+0000', 'last_fullsized': '2024-03-23T09:42:46.793862+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:47:48.322837+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:42:51.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-03-23T09:42:51.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-03-23T09:42:51.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.792271+0000', 'last_change': '2024-03-23T09:42:46.792271+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.792271+0000', 'last_undegraded': '2024-03-23T09:42:46.792271+0000', 'last_fullsized': '2024-03-23T09:42:46.792271+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:29:32.796473+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:42:51.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-03-23T09:42:51.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-03-23T09:42:51.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.792275+0000', 'last_change': '2024-03-23T09:42:46.792275+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.792275+0000', 'last_undegraded': '2024-03-23T09:42:46.792275+0000', 'last_fullsized': '2024-03-23T09:42:46.792275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:42:46.555393+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:42:51.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-03-23T09:42:51.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-03-23T09:42:51.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.793510+0000', 'last_change': '2024-03-23T09:42:46.793510+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.793510+0000', 'last_undegraded': '2024-03-23T09:42:46.793510+0000', 'last_fullsized': '2024-03-23T09:42:46.793510+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:14:55.790367+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:42:51.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-03-23T09:42:51.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-03-23T09:42:51.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.795084+0000', 'last_change': '2024-03-23T09:42:46.795084+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.795084+0000', 'last_undegraded': '2024-03-23T09:42:46.795084+0000', 'last_fullsized': '2024-03-23T09:42:46.795084+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:09:03.354548+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:42:51.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-03-23T09:42:51.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-03-23T09:42:51.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.791854+0000', 'last_change': '2024-03-23T09:42:46.791854+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.791854+0000', 'last_undegraded': '2024-03-23T09:42:46.791854+0000', 'last_fullsized': '2024-03-23T09:42:46.791854+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:50:10.568073+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:42:51.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-03-23T09:42:51.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-03-23T09:42:51.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.792263+0000', 'last_change': '2024-03-23T09:42:46.792263+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.792263+0000', 'last_undegraded': '2024-03-23T09:42:46.792263+0000', 'last_fullsized': '2024-03-23T09:42:46.792263+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:27:56.832097+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:42:51.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-03-23T09:42:51.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-03-23T09:42:51.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.792046+0000', 'last_change': '2024-03-23T09:42:46.792046+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.792046+0000', 'last_undegraded': '2024-03-23T09:42:46.792046+0000', 'last_fullsized': '2024-03-23T09:42:46.792046+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:27:56.832097+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:42:51.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-03-23T09:42:51.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-03-23T09:42:51.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.793721+0000', 'last_change': '2024-03-23T09:42:46.793721+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.793721+0000', 'last_undegraded': '2024-03-23T09:42:46.793721+0000', 'last_fullsized': '2024-03-23T09:42:46.793721+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:29:37.970066+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:42:51.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-03-23T09:42:51.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-03-23T09:42:51.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.791095+0000', 'last_change': '2024-03-23T09:42:46.791095+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.791095+0000', 'last_undegraded': '2024-03-23T09:42:46.791095+0000', 'last_fullsized': '2024-03-23T09:42:46.791095+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:04:43.932184+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:42:51.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-03-23T09:42:51.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-03-23T09:42:51.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.790630+0000', 'last_change': '2024-03-23T09:42:46.790630+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.790630+0000', 'last_undegraded': '2024-03-23T09:42:46.790630+0000', 'last_fullsized': '2024-03-23T09:42:46.790630+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:56:02.796426+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:42:51.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-03-23T09:42:51.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-03-23T09:42:51.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.794308+0000', 'last_change': '2024-03-23T09:42:46.794308+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.794308+0000', 'last_undegraded': '2024-03-23T09:42:46.794308+0000', 'last_fullsized': '2024-03-23T09:42:46.794308+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:24:36.255934+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:42:51.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-03-23T09:42:51.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-03-23T09:42:51.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.795352+0000', 'last_change': '2024-03-23T09:42:46.795352+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.795352+0000', 'last_undegraded': '2024-03-23T09:42:46.795352+0000', 'last_fullsized': '2024-03-23T09:42:46.795352+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:56:02.796426+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:42:51.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-03-23T09:42:51.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:42:47.787316+0000', 'last_change': '2024-03-23T09:42:47.787316+0000', 'last_active': '2024-03-23T09:42:47.787316+0000', 'last_peered': '2024-03-23T09:42:47.787316+0000', 'last_clean': '2024-03-23T09:42:47.787316+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:47.787316+0000', 'last_undegraded': '2024-03-23T09:42:47.787316+0000', 'last_fullsized': '2024-03-23T09:42:47.787316+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:47.787316+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:42:51.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-03-23T09:42:51.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:42:46.793582+0000', 'last_change': '2024-03-23T09:42:46.793582+0000', 'last_active': '2024-03-23T09:42:46.782362+0000', 'last_peered': '2024-03-23T09:42:46.782362+0000', 'last_clean': '2024-03-23T09:42:46.782362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:42:46.793582+0000', 'last_undegraded': '2024-03-23T09:42:46.793582+0000', 'last_fullsized': '2024-03-23T09:42:46.793582+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'last_clean_scrub_stamp': '2024-03-23T09:42:46.782362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:04:09.114370+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:42:51.888 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-23T09:42:52.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.007+0000 7f012e773640 1 Processor -- start 2024-03-23T09:42:52.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.007+0000 7f012e773640 1 -- start start 2024-03-23T09:42:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.008+0000 7f012e773640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0128160250 0x7f01280cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.008+0000 7f012e773640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0128099110 0x7f012806bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.008+0000 7f012e773640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01280cea50 con 0x7f012806bb30 2024-03-23T09:42:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.008+0000 7f012e773640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01280cf890 con 0x7f0128160250 2024-03-23T09:42:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.008+0000 7f012e773640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01280d0260 con 0x7f0128099110 2024-03-23T09:42:52.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.008+0000 7f0127fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0128160250 0x7f01280cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:52.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.008+0000 7f0127fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0128160250 0x7f01280cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39924/0 (socket says 172.21.15.47:39924) 2024-03-23T09:42:52.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.008+0000 7f0127fff640 1 -- 172.21.15.47:0/3709456646 learned_addr learned my addr 172.21.15.47:0/3709456646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:52.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.008+0000 7f012cd10640 1 --2- 172.21.15.47:0/3709456646 >> v2:172.21.15.175:3300/0 conn(0x7f0128099110 0x7f012806bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:52.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.008+0000 7f0127fff640 1 -- 172.21.15.47:0/3709456646 >> v2:172.21.15.175:3300/0 conn(0x7f0128099110 msgr2=0x7f012806bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:52.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.008+0000 7f0127fff640 1 --2- 172.21.15.47:0/3709456646 >> v2:172.21.15.175:3300/0 conn(0x7f0128099110 0x7f012806bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.008+0000 7f0127fff640 1 -- 172.21.15.47:0/3709456646 >> v1:172.21.15.175:6789/0 conn(0x7f012806bb30 legacy=0x7f0128098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:52.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.008+0000 7f0127fff640 1 -- 172.21.15.47:0/3709456646 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01280cec80 con 0x7f0128160250 2024-03-23T09:42:52.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.009+0000 7f0127fff640 1 --2- 172.21.15.47:0/3709456646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128160250 0x7f01280cfd20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f012806dc30 tx=0x7f011c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91644d1f5a77ca49 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.009+0000 7f0126ffd640 1 -- 172.21.15.47:0/3709456646 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c031870 con 0x7f0128160250 2024-03-23T09:42:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.009+0000 7f0126ffd640 1 -- 172.21.15.47:0/3709456646 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011c0319d0 con 0x7f0128160250 2024-03-23T09:42:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.009+0000 7f0126ffd640 1 -- 172.21.15.47:0/3709456646 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c031d60 con 0x7f0128160250 2024-03-23T09:42:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.010+0000 7f012e773640 1 -- 172.21.15.47:0/3709456646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128160250 msgr2=0x7f01280cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.010+0000 7f012e773640 1 --2- 172.21.15.47:0/3709456646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128160250 0x7f01280cfd20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f012806dc30 tx=0x7f011c02b910 comp rx=0 tx=0).stop 2024-03-23T09:42:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.010+0000 7f012e773640 1 -- 172.21.15.47:0/3709456646 shutdown_connections 2024-03-23T09:42:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.010+0000 7f012e773640 1 --2- 172.21.15.47:0/3709456646 >> v2:172.21.15.175:3300/0 conn(0x7f0128099110 0x7f012806bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.010+0000 7f012e773640 1 --2- 172.21.15.47:0/3709456646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128160250 0x7f01280cfd20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.010+0000 7f012e773640 1 -- 172.21.15.47:0/3709456646 >> 172.21.15.47:0/3709456646 conn(0x7f01280b2140 msgr2=0x7f01280b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.010+0000 7f012e773640 1 -- 172.21.15.47:0/3709456646 shutdown_connections 2024-03-23T09:42:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.010+0000 7f012e773640 1 -- 172.21.15.47:0/3709456646 wait complete. 2024-03-23T09:42:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.011+0000 7f012e773640 1 Processor -- start 2024-03-23T09:42:52.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.011+0000 7f012e773640 1 -- start start 2024-03-23T09:42:52.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f012e773640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f012806bb30 0x7f0128150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:52.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f012e773640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128099110 0x7f01280b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:52.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f012e773640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0128160250 0x7f01280b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:52.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f012e773640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012815e1c0 con 0x7f0128099110 2024-03-23T09:42:52.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f012e773640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012815d5b0 con 0x7f012806bb30 2024-03-23T09:42:52.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f012e773640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012815d890 con 0x7f0128160250 2024-03-23T09:42:52.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f01277fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128099110 0x7f01280b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:52.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f01277fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128099110 0x7f01280b58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39936/0 (socket says 172.21.15.47:39936) 2024-03-23T09:42:52.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f01277fe640 1 -- 172.21.15.47:0/3941185438 learned_addr learned my addr 172.21.15.47:0/3941185438 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:52.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f012cd10640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0128160250 0x7f01280b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:52.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f0127fff640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f012806bb30 0x7f0128150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f01277fe640 1 -- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0128160250 msgr2=0x7f01280b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f01277fe640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0128160250 0x7f01280b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f01277fe640 1 -- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f012806bb30 msgr2=0x7f0128150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f01277fe640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f012806bb30 0x7f0128150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f01277fe640 1 -- 172.21.15.47:0/3941185438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c0094d0 con 0x7f0128099110 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f0127fff640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f012806bb30 0x7f0128150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.012+0000 7f01277fe640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128099110 0x7f01280b58a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f011800c430 tx=0x7f011800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.013+0000 7f0124ff9640 1 -- 172.21.15.47:0/3941185438 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118011790 con 0x7f0128099110 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.013+0000 7f0124ff9640 1 -- 172.21.15.47:0/3941185438 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01180118f0 con 0x7f0128099110 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.013+0000 7f0124ff9640 1 -- 172.21.15.47:0/3941185438 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118011ce0 con 0x7f0128099110 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.013+0000 7f012e773640 1 -- 172.21.15.47:0/3941185438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012815db70 con 0x7f0128099110 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.013+0000 7f012e773640 1 -- 172.21.15.47:0/3941185438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0128150fc0 con 0x7f0128099110 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.014+0000 7f012e773640 1 -- 172.21.15.47:0/3941185438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0128066be0 con 0x7f0128099110 2024-03-23T09:42:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.014+0000 7f0124ff9640 1 -- 172.21.15.47:0/3941185438 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f011801c460 con 0x7f0128099110 2024-03-23T09:42:52.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.014+0000 7f0124ff9640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f011406ac30 0x7f011406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:52.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.015+0000 7f0124ff9640 1 -- 172.21.15.47:0/3941185438 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01180891f0 con 0x7f0128099110 2024-03-23T09:42:52.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.015+0000 7f0127fff640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f011406ac30 0x7f011406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:52.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.015+0000 7f0127fff640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f011406ac30 0x7f011406d0f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f011c036000 tx=0x7f011c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:52.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.019+0000 7f0124ff9640 1 -- 172.21.15.47:0/3941185438 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f01180543d0 con 0x7f0128099110 2024-03-23T09:42:52.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.267+0000 7f012e773640 1 -- 172.21.15.47:0/3941185438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f01280b2f50 con 0x7f0128099110 2024-03-23T09:42:52.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.267+0000 7f0124ff9640 1 -- 172.21.15.47:0/3941185438 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1656 (secure 0 0 0) 0x7f01180579f0 con 0x7f0128099110 2024-03-23T09:42:52.267 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:52.267 INFO:teuthology.orchestra.run.smithi047.stdout:{"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":126,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":46,"num_osds":8,"num_up_osds":8,"osd_up_since":1711186856,"num_in_osds":8,"osd_in_since":1711186848,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":243294208,"bytes_avail":772850819072,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":18,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"d","status":"up:active","gid":5697}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-03-23T09:41:58.350978+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-23T09:42:52.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.269+0000 7f012e773640 1 -- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f011406ac30 msgr2=0x7f011406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:52.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.269+0000 7f012e773640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f011406ac30 0x7f011406d0f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f011c036000 tx=0x7f011c03c040 comp rx=0 tx=0).stop 2024-03-23T09:42:52.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.269+0000 7f012e773640 1 -- 172.21.15.47:0/3941185438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128099110 msgr2=0x7f01280b58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.269+0000 7f012e773640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128099110 0x7f01280b58a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f011800c430 tx=0x7f011800c8f0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.269+0000 7f012e773640 1 -- 172.21.15.47:0/3941185438 shutdown_connections 2024-03-23T09:42:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.269+0000 7f012e773640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f011406ac30 0x7f011406d0f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.269+0000 7f012e773640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0128160250 0x7f01280b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.269+0000 7f012e773640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128099110 0x7f01280b58a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.269+0000 7f012e773640 1 --2- 172.21.15.47:0/3941185438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f012806bb30 0x7f0128150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.269+0000 7f012e773640 1 -- 172.21.15.47:0/3941185438 >> 172.21.15.47:0/3941185438 conn(0x7f01280b2140 msgr2=0x7f01280b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.269+0000 7f012e773640 1 -- 172.21.15.47:0/3941185438 shutdown_connections 2024-03-23T09:42:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.269+0000 7f012e773640 1 -- 172.21.15.47:0/3941185438 wait complete. 2024-03-23T09:42:52.280 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-23T09:42:52.280 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:42:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.381+0000 7ffb8af39640 1 Processor -- start 2024-03-23T09:42:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.381+0000 7ffb8af39640 1 -- start start 2024-03-23T09:42:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb8af39640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb8415cee0 0x7ffb840d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb8af39640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffb840d23a0 0x7ffb840d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb8af39640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffb840b8300 0x7ffb840d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb8af39640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb840d1300 con 0x7ffb840d23a0 2024-03-23T09:42:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb8af39640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb840d15e0 con 0x7ffb840b8300 2024-03-23T09:42:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb8af39640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb840d18c0 con 0x7ffb8415cee0 2024-03-23T09:42:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb83fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffb840d23a0 0x7ffb840d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb83fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffb840d23a0 0x7ffb840d2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39938/0 (socket says 172.21.15.47:39938) 2024-03-23T09:42:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb83fff640 1 -- 172.21.15.47:0/2500050143 learned_addr learned my addr 172.21.15.47:0/2500050143 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb88cd5640 1 --2- 172.21.15.47:0/2500050143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb8415cee0 0x7ffb840d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb894d6640 1 --2- 172.21.15.47:0/2500050143 >> v2:172.21.15.175:3300/0 conn(0x7ffb840b8300 0x7ffb840d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb83fff640 1 -- 172.21.15.47:0/2500050143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb8415cee0 msgr2=0x7ffb840d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb83fff640 1 --2- 172.21.15.47:0/2500050143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb8415cee0 0x7ffb840d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb83fff640 1 -- 172.21.15.47:0/2500050143 >> v2:172.21.15.175:3300/0 conn(0x7ffb840b8300 msgr2=0x7ffb840d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.382+0000 7ffb83fff640 1 --2- 172.21.15.47:0/2500050143 >> v2:172.21.15.175:3300/0 conn(0x7ffb840b8300 0x7ffb840d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.383+0000 7ffb83fff640 1 -- 172.21.15.47:0/2500050143 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb840d1ba0 con 0x7ffb840d23a0 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.383+0000 7ffb83fff640 1 --2- 172.21.15.47:0/2500050143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb840d23a0 0x7ffb840d2870 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffb840d39e0 tx=0x7ffb7402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0ce8df60e911ed server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.383+0000 7ffb837fe640 1 -- 172.21.15.47:0/2500050143 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7402ca70 con 0x7ffb840d23a0 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.383+0000 7ffb837fe640 1 -- 172.21.15.47:0/2500050143 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7402cbd0 con 0x7ffb840d23a0 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.383+0000 7ffb837fe640 1 -- 172.21.15.47:0/2500050143 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb740343f0 con 0x7ffb840d23a0 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.384+0000 7ffb8af39640 1 -- 172.21.15.47:0/2500050143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb840d23a0 msgr2=0x7ffb840d2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.384+0000 7ffb8af39640 1 --2- 172.21.15.47:0/2500050143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb840d23a0 0x7ffb840d2870 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffb840d39e0 tx=0x7ffb7402bed0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.384+0000 7ffb8af39640 1 -- 172.21.15.47:0/2500050143 shutdown_connections 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.384+0000 7ffb8af39640 1 --2- 172.21.15.47:0/2500050143 >> v2:172.21.15.175:3300/0 conn(0x7ffb840b8300 0x7ffb840d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.384+0000 7ffb8af39640 1 --2- 172.21.15.47:0/2500050143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb840d23a0 0x7ffb840d2870 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.384+0000 7ffb8af39640 1 --2- 172.21.15.47:0/2500050143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb8415cee0 0x7ffb840d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.384+0000 7ffb8af39640 1 -- 172.21.15.47:0/2500050143 >> 172.21.15.47:0/2500050143 conn(0x7ffb84065df0 msgr2=0x7ffb84066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.384+0000 7ffb8af39640 1 -- 172.21.15.47:0/2500050143 shutdown_connections 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.384+0000 7ffb8af39640 1 -- 172.21.15.47:0/2500050143 wait complete. 2024-03-23T09:42:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.385+0000 7ffb8af39640 1 Processor -- start 2024-03-23T09:42:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.385+0000 7ffb8af39640 1 -- start start 2024-03-23T09:42:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb8af39640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb840b8300 0x7ffb841f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb8af39640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb840d23a0 0x7ffb84159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb8af39640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb8415cee0 0x7ffb8414e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb8af39640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb84158970 con 0x7ffb840b8300 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb8af39640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb84158bc0 con 0x7ffb840d23a0 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb8af39640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb84158ea0 con 0x7ffb8415cee0 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb88cd5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb840b8300 0x7ffb841f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb88cd5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb840b8300 0x7ffb841f1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39948/0 (socket says 172.21.15.47:39948) 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb88cd5640 1 -- 172.21.15.47:0/2174271658 learned_addr learned my addr 172.21.15.47:0/2174271658 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb88cd5640 1 -- 172.21.15.47:0/2174271658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb8415cee0 msgr2=0x7ffb8414e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb88cd5640 1 --2- 172.21.15.47:0/2174271658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb8415cee0 0x7ffb8414e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb88cd5640 1 -- 172.21.15.47:0/2174271658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb840d23a0 msgr2=0x7ffb84159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb88cd5640 1 --2- 172.21.15.47:0/2174271658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb840d23a0 0x7ffb84159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.386+0000 7ffb88cd5640 1 -- 172.21.15.47:0/2174271658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb74009b30 con 0x7ffb840b8300 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.387+0000 7ffb88cd5640 1 --2- 172.21.15.47:0/2174271658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb840b8300 0x7ffb841f1d20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00d5f0 tx=0x7ffb7c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.387+0000 7ffb817fa640 1 -- 172.21.15.47:0/2174271658 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0125f0 con 0x7ffb840b8300 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.387+0000 7ffb817fa640 1 -- 172.21.15.47:0/2174271658 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c012750 con 0x7ffb840b8300 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.387+0000 7ffb8af39640 1 -- 172.21.15.47:0/2174271658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb84159180 con 0x7ffb840b8300 2024-03-23T09:42:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.387+0000 7ffb817fa640 1 -- 172.21.15.47:0/2174271658 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c012b00 con 0x7ffb840b8300 2024-03-23T09:42:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.387+0000 7ffb8af39640 1 -- 172.21.15.47:0/2174271658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb84159740 con 0x7ffb840b8300 2024-03-23T09:42:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.388+0000 7ffb8af39640 1 -- 172.21.15.47:0/2174271658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb84072360 con 0x7ffb840b8300 2024-03-23T09:42:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.388+0000 7ffb817fa640 1 -- 172.21.15.47:0/2174271658 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ffb7c012c60 con 0x7ffb840b8300 2024-03-23T09:42:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.389+0000 7ffb817fa640 1 --2- 172.21.15.47:0/2174271658 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffb6406ac00 0x7ffb6406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.389+0000 7ffb817fa640 1 -- 172.21.15.47:0/2174271658 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb7c0898e0 con 0x7ffb840b8300 2024-03-23T09:42:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.389+0000 7ffb83fff640 1 --2- 172.21.15.47:0/2174271658 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffb6406ac00 0x7ffb6406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.390+0000 7ffb83fff640 1 --2- 172.21.15.47:0/2174271658 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffb6406ac00 0x7ffb6406d0c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffb74035000 tx=0x7ffb7402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:52.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.393+0000 7ffb817fa640 1 -- 172.21.15.47:0/2174271658 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ffb7c054a20 con 0x7ffb840b8300 2024-03-23T09:42:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.567+0000 7ffb8af39640 1 -- 172.21.15.47:0/2174271658 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb840b4360 con 0x7ffb6406ac00 2024-03-23T09:42:52.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.575+0000 7ffb817fa640 1 -- 172.21.15.47:0/2174271658 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278383 (secure 0 0 0) 0x7ffb7402c970 con 0x7ffb6406ac00 2024-03-23T09:42:52.577 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:42:52.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.578+0000 7ffb8af39640 1 -- 172.21.15.47:0/2174271658 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffb6406ac00 msgr2=0x7ffb6406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:52.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.578+0000 7ffb8af39640 1 --2- 172.21.15.47:0/2174271658 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffb6406ac00 0x7ffb6406d0c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffb74035000 tx=0x7ffb7402e000 comp rx=0 tx=0).stop 2024-03-23T09:42:52.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.578+0000 7ffb8af39640 1 -- 172.21.15.47:0/2174271658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb840b8300 msgr2=0x7ffb841f1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:52.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.578+0000 7ffb8af39640 1 --2- 172.21.15.47:0/2174271658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb840b8300 0x7ffb841f1d20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00d5f0 tx=0x7ffb7c00dab0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.578+0000 7ffb8af39640 1 -- 172.21.15.47:0/2174271658 shutdown_connections 2024-03-23T09:42:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.578+0000 7ffb8af39640 1 --2- 172.21.15.47:0/2174271658 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffb6406ac00 0x7ffb6406d0c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.578+0000 7ffb8af39640 1 --2- 172.21.15.47:0/2174271658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb8415cee0 0x7ffb8414e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.578+0000 7ffb8af39640 1 --2- 172.21.15.47:0/2174271658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb840d23a0 0x7ffb84159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.578+0000 7ffb8af39640 1 --2- 172.21.15.47:0/2174271658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb840b8300 0x7ffb841f1d20 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.578+0000 7ffb8af39640 1 -- 172.21.15.47:0/2174271658 >> 172.21.15.47:0/2174271658 conn(0x7ffb84065df0 msgr2=0x7ffb840b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.578+0000 7ffb8af39640 1 -- 172.21.15.47:0/2174271658 shutdown_connections 2024-03-23T09:42:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:52.578+0000 7ffb8af39640 1 -- 172.21.15.47:0/2174271658 wait complete. 2024-03-23T09:42:52.588 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-03-23T09:42:52.364498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":237592,"kb_used_data":5040,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737128,"statfs":{"total":773094113280,"available":772850819072,"internally_reserved":0,"allocated":5160960,"data_stored":2752870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.575228"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792350+0000","last_change":"2024-03-23T09:42:46.792350+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792350+0000","last_undegraded":"2024-03-23T09:42:46.792350+0000","last_fullsized":"2024-03-23T09:42:46.792350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:41.236379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791904+0000","last_change":"2024-03-23T09:42:46.791904+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791904+0000","last_undegraded":"2024-03-23T09:42:46.791904+0000","last_fullsized":"2024-03-23T09:42:46.791904+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793978+0000","last_change":"2024-03-23T09:42:46.793978+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793978+0000","last_undegraded":"2024-03-23T09:42:46.793978+0000","last_fullsized":"2024-03-23T09:42:46.793978+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:21.810507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791103+0000","last_change":"2024-03-23T09:42:46.791103+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791103+0000","last_undegraded":"2024-03-23T09:42:46.791103+0000","last_fullsized":"2024-03-23T09:42:46.791103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:04:24.190076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793327+0000","last_change":"2024-03-23T09:42:46.793327+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793327+0000","last_undegraded":"2024-03-23T09:42:46.793327+0000","last_fullsized":"2024-03-23T09:42:46.793327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:40.886514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791385+0000","last_change":"2024-03-23T09:42:46.791385+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791385+0000","last_undegraded":"2024-03-23T09:42:46.791385+0000","last_fullsized":"2024-03-23T09:42:46.791385+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:58:54.380376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.794200+0000","last_change":"2024-03-23T09:42:46.794200+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.794200+0000","last_undegraded":"2024-03-23T09:42:46.794200+0000","last_fullsized":"2024-03-23T09:42:46.794200+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:04:53.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.796428+0000","last_change":"2024-03-23T09:42:46.796428+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.796428+0000","last_undegraded":"2024-03-23T09:42:46.796428+0000","last_fullsized":"2024-03-23T09:42:46.796428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790630+0000","last_change":"2024-03-23T09:42:46.790630+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790630+0000","last_undegraded":"2024-03-23T09:42:46.790630+0000","last_fullsized":"2024-03-23T09:42:46.790630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:01:51.451920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795158+0000","last_change":"2024-03-23T09:42:46.795158+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795158+0000","last_undegraded":"2024-03-23T09:42:46.795158+0000","last_fullsized":"2024-03-23T09:42:46.795158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:02:27.446246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792484+0000","last_change":"2024-03-23T09:42:46.792484+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792484+0000","last_undegraded":"2024-03-23T09:42:46.792484+0000","last_fullsized":"2024-03-23T09:42:46.792484+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:35:53.515036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790019+0000","last_change":"2024-03-23T09:42:46.790019+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790019+0000","last_undegraded":"2024-03-23T09:42:46.790019+0000","last_fullsized":"2024-03-23T09:42:46.790019+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792903+0000","last_change":"2024-03-23T09:42:46.792903+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792903+0000","last_undegraded":"2024-03-23T09:42:46.792903+0000","last_fullsized":"2024-03-23T09:42:46.792903+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:43:42.174052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791816+0000","last_change":"2024-03-23T09:42:46.791816+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791816+0000","last_undegraded":"2024-03-23T09:42:46.791816+0000","last_fullsized":"2024-03-23T09:42:46.791816+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:32.470235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792213+0000","last_change":"2024-03-23T09:42:46.792213+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792213+0000","last_undegraded":"2024-03-23T09:42:46.792213+0000","last_fullsized":"2024-03-23T09:42:46.792213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:40:05.822261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795999+0000","last_change":"2024-03-23T09:42:46.795999+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795999+0000","last_undegraded":"2024-03-23T09:42:46.795999+0000","last_fullsized":"2024-03-23T09:42:46.795999+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:01:51.451920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790079+0000","last_change":"2024-03-23T09:42:46.790079+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790079+0000","last_undegraded":"2024-03-23T09:42:46.790079+0000","last_fullsized":"2024-03-23T09:42:46.790079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:19.815833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791201+0000","last_change":"2024-03-23T09:42:46.791201+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791201+0000","last_undegraded":"2024-03-23T09:42:46.791201+0000","last_fullsized":"2024-03-23T09:42:46.791201+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:19.815833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793862+0000","last_change":"2024-03-23T09:42:46.793862+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793862+0000","last_undegraded":"2024-03-23T09:42:46.793862+0000","last_fullsized":"2024-03-23T09:42:46.793862+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:47:48.322837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792271+0000","last_change":"2024-03-23T09:42:46.792271+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792271+0000","last_undegraded":"2024-03-23T09:42:46.792271+0000","last_fullsized":"2024-03-23T09:42:46.792271+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:29:32.796473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792275+0000","last_change":"2024-03-23T09:42:46.792275+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792275+0000","last_undegraded":"2024-03-23T09:42:46.792275+0000","last_fullsized":"2024-03-23T09:42:46.792275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:46.555393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793510+0000","last_change":"2024-03-23T09:42:46.793510+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793510+0000","last_undegraded":"2024-03-23T09:42:46.793510+0000","last_fullsized":"2024-03-23T09:42:46.793510+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:14:55.790367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795084+0000","last_change":"2024-03-23T09:42:46.795084+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795084+0000","last_undegraded":"2024-03-23T09:42:46.795084+0000","last_fullsized":"2024-03-23T09:42:46.795084+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:09:03.354548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791854+0000","last_change":"2024-03-23T09:42:46.791854+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791854+0000","last_undegraded":"2024-03-23T09:42:46.791854+0000","last_fullsized":"2024-03-23T09:42:46.791854+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:50:10.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792263+0000","last_change":"2024-03-23T09:42:46.792263+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792263+0000","last_undegraded":"2024-03-23T09:42:46.792263+0000","last_fullsized":"2024-03-23T09:42:46.792263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:27:56.832097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.792046+0000","last_change":"2024-03-23T09:42:46.792046+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.792046+0000","last_undegraded":"2024-03-23T09:42:46.792046+0000","last_fullsized":"2024-03-23T09:42:46.792046+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:27:56.832097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793721+0000","last_change":"2024-03-23T09:42:46.793721+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793721+0000","last_undegraded":"2024-03-23T09:42:46.793721+0000","last_fullsized":"2024-03-23T09:42:46.793721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:37.970066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.791095+0000","last_change":"2024-03-23T09:42:46.791095+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.791095+0000","last_undegraded":"2024-03-23T09:42:46.791095+0000","last_fullsized":"2024-03-23T09:42:46.791095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:43.932184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.790630+0000","last_change":"2024-03-23T09:42:46.790630+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.790630+0000","last_undegraded":"2024-03-23T09:42:46.790630+0000","last_fullsized":"2024-03-23T09:42:46.790630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:02.796426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.794308+0000","last_change":"2024-03-23T09:42:46.794308+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.794308+0000","last_undegraded":"2024-03-23T09:42:46.794308+0000","last_fullsized":"2024-03-23T09:42:46.794308+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:24:36.255934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.795352+0000","last_change":"2024-03-23T09:42:46.795352+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.795352+0000","last_undegraded":"2024-03-23T09:42:46.795352+0000","last_fullsized":"2024-03-23T09:42:46.795352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:02.796426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:42:47.787316+0000","last_change":"2024-03-23T09:42:47.787316+0000","last_active":"2024-03-23T09:42:47.787316+0000","last_peered":"2024-03-23T09:42:47.787316+0000","last_clean":"2024-03-23T09:42:47.787316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:47.787316+0000","last_undegraded":"2024-03-23T09:42:47.787316+0000","last_fullsized":"2024-03-23T09:42:47.787316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-23T09:42:46.793582+0000","last_change":"2024-03-23T09:42:46.793582+0000","last_active":"2024-03-23T09:42:46.782362+0000","last_peered":"2024-03-23T09:42:46.782362+0000","last_clean":"2024-03-23T09:42:46.782362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:42:46.793582+0000","last_undegraded":"2024-03-23T09:42:46.793582+0000","last_fullsized":"2024-03-23T09:42:46.793582+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:04:09.114370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":186,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-23T09:42:44.417082+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:42:44.417082+0000","last_peered":"2024-03-23T09:42:44.417082+0000","last_clean":"2024-03-23T09:42:44.417082+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:42:44.417082+0000","last_undegraded":"2024-03-23T09:42:44.417082+0000","last_fullsized":"2024-03-23T09:42:44.417082+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1097728,"data_stored":789948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54400000000000004}]},{"osd":1,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.50900000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.47699999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27500,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":487424,"data_stored":193144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.52100000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.56299999999999994}]},{"osd":2,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.50700000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.433}]},{"osd":4,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48799999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.45600000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":199580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.439},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.53100000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.51300000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.67600000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.73799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":199580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69899999999999995}]},{"osd":1,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.66300000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.747}]},{"osd":3,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.48599999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.502}]},{"osd":7,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":491520,"data_stored":193763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.19},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.30299999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.223}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.318},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.29199999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.503}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60499999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":503808,"data_stored":199580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.34799999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.316}]},{"osd":3,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.248}]},{"osd":4,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.41999999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.56599999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.436}]},{"osd":7,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.47399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":491520,"data_stored":193763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.441},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":0.42299999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.85199999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71799999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.65600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1081344,"data_stored":783512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.247},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.29499999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.314},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.34100000000000003}]},{"osd":3,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.35399999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.625}]},{"osd":5,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65300000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.66800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:42:53.692 INFO:tasks.ceph.mds.d.smithi175.stderr:2024-03-23T09:42:53.692+0000 7f3363a75640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:42:55.593 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:42:55.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.705+0000 7f66accd5640 1 Processor -- start 2024-03-23T09:42:55.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.705+0000 7f66accd5640 1 -- start start 2024-03-23T09:42:55.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.706+0000 7f66accd5640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f66a80b8350 0x7f66a80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:55.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.706+0000 7f66accd5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66a80cdc40 0x7f66a80d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:55.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.706+0000 7f66accd5640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a80d11e0 con 0x7f66a815ca90 2024-03-23T09:42:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.706+0000 7f66accd5640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a80ce0d0 con 0x7f66a80b8350 2024-03-23T09:42:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.706+0000 7f66accd5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a8160f60 con 0x7f66a80cdc40 2024-03-23T09:42:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.706+0000 7f66a659c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f66a815ca90 0x7f66a8160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52004/0 (socket says 172.21.15.47:52004) 2024-03-23T09:42:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.706+0000 7f66a659c640 1 -- 172.21.15.47:0/1796267315 learned_addr learned my addr 172.21.15.47:0/1796267315 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.706+0000 7f66a5d9b640 1 --2- 172.21.15.47:0/1796267315 >> v2:172.21.15.175:3300/0 conn(0x7f66a80b8350 0x7f66a80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.706+0000 7f66a6d9d640 1 --2- 172.21.15.47:0/1796267315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66a80cdc40 0x7f66a80d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.706+0000 7f66a559a640 1 -- 172.21.15.47:0/1796267315 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720925309 0 0) 0x7f66a80d11e0 con 0x7f66a815ca90 2024-03-23T09:42:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.706+0000 7f66a559a640 1 -- 172.21.15.47:0/1796267315 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6690003520 con 0x7f66a815ca90 2024-03-23T09:42:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.707+0000 7f66a559a640 1 -- 172.21.15.47:0/1796267315 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3646921101 0 0) 0x7f6690003520 con 0x7f66a815ca90 2024-03-23T09:42:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.707+0000 7f66a5d9b640 1 -- 172.21.15.47:0/1796267315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66a80cdc40 msgr2=0x7f66a80d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.707+0000 7f66a5d9b640 1 --2- 172.21.15.47:0/1796267315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66a80cdc40 0x7f66a80d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.707+0000 7f66a5d9b640 1 -- 172.21.15.47:0/1796267315 >> v1:172.21.15.47:6789/0 conn(0x7f66a815ca90 legacy=0x7f66a8160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.707+0000 7f66a5d9b640 1 -- 172.21.15.47:0/1796267315 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a8161240 con 0x7f66a80b8350 2024-03-23T09:42:55.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.708+0000 7f66a5d9b640 1 --2- 172.21.15.47:0/1796267315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a80b8350 0x7f66a80cd700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f669c0097a0 tx=0x7f669c02b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=533977fef70588ad server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.708+0000 7f66a559a640 1 -- 172.21.15.47:0/1796267315 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0001670 con 0x7f66a80b8350 2024-03-23T09:42:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.708+0000 7f66a559a640 1 -- 172.21.15.47:0/1796267315 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669c02e050 con 0x7f66a80b8350 2024-03-23T09:42:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.708+0000 7f66a559a640 1 -- 172.21.15.47:0/1796267315 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c030710 con 0x7f66a80b8350 2024-03-23T09:42:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.708+0000 7f66accd5640 1 -- 172.21.15.47:0/1796267315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a80b8350 msgr2=0x7f66a80cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.708+0000 7f66accd5640 1 --2- 172.21.15.47:0/1796267315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a80b8350 0x7f66a80cd700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f669c0097a0 tx=0x7f669c02b710 comp rx=0 tx=0).stop 2024-03-23T09:42:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.708+0000 7f66accd5640 1 -- 172.21.15.47:0/1796267315 shutdown_connections 2024-03-23T09:42:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.708+0000 7f66accd5640 1 --2- 172.21.15.47:0/1796267315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66a80cdc40 0x7f66a80d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.708+0000 7f66accd5640 1 --2- 172.21.15.47:0/1796267315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a80b8350 0x7f66a80cd700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.708+0000 7f66accd5640 1 -- 172.21.15.47:0/1796267315 >> 172.21.15.47:0/1796267315 conn(0x7f66a80659d0 msgr2=0x7f66a8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.708+0000 7f66accd5640 1 -- 172.21.15.47:0/1796267315 shutdown_connections 2024-03-23T09:42:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.708+0000 7f66accd5640 1 -- 172.21.15.47:0/1796267315 wait complete. 2024-03-23T09:42:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.710+0000 7f66accd5640 1 Processor -- start 2024-03-23T09:42:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.710+0000 7f66accd5640 1 -- start start 2024-03-23T09:42:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.710+0000 7f66accd5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66a80b8350 0x7f66a81f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.710+0000 7f66accd5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a80cdc40 0x7f66a80b7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.710+0000 7f66accd5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a815ca90 0x7f66a80b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.710+0000 7f66accd5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a80ca910 con 0x7f66a815ca90 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.710+0000 7f66accd5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a81f2420 con 0x7f66a80cdc40 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.710+0000 7f66accd5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a81f6a30 con 0x7f66a80b8350 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f66a6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a815ca90 0x7f66a80b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f66a6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a815ca90 0x7f66a80b79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59858/0 (socket says 172.21.15.47:59858) 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f66a6d9d640 1 -- 172.21.15.47:0/1595427927 learned_addr learned my addr 172.21.15.47:0/1595427927 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f66a659c640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66a80b8350 0x7f66a81f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f66a5d9b640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a80cdc40 0x7f66a80b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f66a6d9d640 1 -- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66a80b8350 msgr2=0x7f66a81f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f66a6d9d640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66a80b8350 0x7f66a81f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f66a6d9d640 1 -- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a80cdc40 msgr2=0x7f66a80b7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f66a6d9d640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a80cdc40 0x7f66a80b7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f66a6d9d640 1 -- 172.21.15.47:0/1595427927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c009400 con 0x7f66a815ca90 2024-03-23T09:42:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f66a6d9d640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a815ca90 0x7f66a80b79b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f669800efc0 tx=0x7f669800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f6696ffd640 1 -- 172.21.15.47:0/1595427927 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698009ba0 con 0x7f66a815ca90 2024-03-23T09:42:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f6696ffd640 1 -- 172.21.15.47:0/1595427927 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669800f040 con 0x7f66a815ca90 2024-03-23T09:42:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f66accd5640 1 -- 172.21.15.47:0/1595427927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a81f5c70 con 0x7f66a815ca90 2024-03-23T09:42:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f6696ffd640 1 -- 172.21.15.47:0/1595427927 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66980146c0 con 0x7f66a815ca90 2024-03-23T09:42:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.711+0000 7f66accd5640 1 -- 172.21.15.47:0/1595427927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a81f6230 con 0x7f66a815ca90 2024-03-23T09:42:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.712+0000 7f66accd5640 1 -- 172.21.15.47:0/1595427927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a8072360 con 0x7f66a815ca90 2024-03-23T09:42:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.713+0000 7f6696ffd640 1 -- 172.21.15.47:0/1595427927 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6698014820 con 0x7f66a815ca90 2024-03-23T09:42:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.713+0000 7f6696ffd640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f668806ac00 0x7f668806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.714+0000 7f6696ffd640 1 -- 172.21.15.47:0/1595427927 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6698089c50 con 0x7f66a815ca90 2024-03-23T09:42:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.714+0000 7f66a659c640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f668806ac00 0x7f668806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.714+0000 7f66a659c640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f668806ac00 0x7f668806d0c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f66a0006b50 tx=0x7f66a0001dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:55.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.718+0000 7f6696ffd640 1 -- 172.21.15.47:0/1595427927 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f669808f020 con 0x7f66a815ca90 2024-03-23T09:42:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.895+0000 7f66accd5640 1 -- 172.21.15.47:0/1595427927 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f66a80d0e70 con 0x7f668806ac00 2024-03-23T09:42:55.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.903+0000 7f6696ffd640 1 -- 172.21.15.47:0/1595427927 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289393 (secure 0 0 0) 0x7f66a000f950 con 0x7f668806ac00 2024-03-23T09:42:55.906 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:55.907 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:42:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.905+0000 7f66accd5640 1 -- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f668806ac00 msgr2=0x7f668806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.905+0000 7f66accd5640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f668806ac00 0x7f668806d0c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f66a0006b50 tx=0x7f66a0001dc0 comp rx=0 tx=0).stop 2024-03-23T09:42:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.905+0000 7f66accd5640 1 -- 172.21.15.47:0/1595427927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a815ca90 msgr2=0x7f66a80b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.905+0000 7f66accd5640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a815ca90 0x7f66a80b79b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f669800efc0 tx=0x7f669800c490 comp rx=0 tx=0).stop 2024-03-23T09:42:55.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.905+0000 7f66accd5640 1 -- 172.21.15.47:0/1595427927 shutdown_connections 2024-03-23T09:42:55.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.905+0000 7f66accd5640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f668806ac00 0x7f668806d0c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:55.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.905+0000 7f66accd5640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a815ca90 0x7f66a80b79b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:55.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.905+0000 7f66accd5640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a80cdc40 0x7f66a80b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:55.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.905+0000 7f66accd5640 1 --2- 172.21.15.47:0/1595427927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66a80b8350 0x7f66a81f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:55.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.905+0000 7f66accd5640 1 -- 172.21.15.47:0/1595427927 >> 172.21.15.47:0/1595427927 conn(0x7f66a80659d0 msgr2=0x7f66a80cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:55.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.905+0000 7f66accd5640 1 -- 172.21.15.47:0/1595427927 shutdown_connections 2024-03-23T09:42:55.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:55.905+0000 7f66accd5640 1 -- 172.21.15.47:0/1595427927 wait complete. 2024-03-23T09:42:55.917 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-03-23T09:42:54.364982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238032,"kb_used_data":5440,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754736688,"statfs":{"total":773094113280,"available":772850368512,"internally_reserved":0,"allocated":5570560,"data_stored":2935598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.574199"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.175047+0000","last_change":"2024-03-23T09:42:48.189027+0000","last_active":"2024-03-23T09:42:49.175047+0000","last_peered":"2024-03-23T09:42:49.175047+0000","last_clean":"2024-03-23T09:42:49.175047+0000","last_became_active":"2024-03-23T09:42:48.188774+0000","last_became_peered":"2024-03-23T09:42:48.188774+0000","last_unstale":"2024-03-23T09:42:49.175047+0000","last_undegraded":"2024-03-23T09:42:49.175047+0000","last_fullsized":"2024-03-23T09:42:49.175047+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:09:59.751192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997621+0000","last_change":"2024-03-23T09:42:48.194683+0000","last_active":"2024-03-23T09:42:49.997621+0000","last_peered":"2024-03-23T09:42:49.997621+0000","last_clean":"2024-03-23T09:42:49.997621+0000","last_became_active":"2024-03-23T09:42:48.193746+0000","last_became_peered":"2024-03-23T09:42:48.193746+0000","last_unstale":"2024-03-23T09:42:49.997621+0000","last_undegraded":"2024-03-23T09:42:49.997621+0000","last_fullsized":"2024-03-23T09:42:49.997621+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:40:06.827216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.174905+0000","last_change":"2024-03-23T09:42:48.187437+0000","last_active":"2024-03-23T09:42:49.174905+0000","last_peered":"2024-03-23T09:42:49.174905+0000","last_clean":"2024-03-23T09:42:49.174905+0000","last_became_active":"2024-03-23T09:42:48.186965+0000","last_became_peered":"2024-03-23T09:42:48.186965+0000","last_unstale":"2024-03-23T09:42:49.174905+0000","last_undegraded":"2024-03-23T09:42:49.174905+0000","last_fullsized":"2024-03-23T09:42:49.174905+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:47:49.327792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399599+0000","last_change":"2024-03-23T09:42:48.188100+0000","last_active":"2024-03-23T09:42:49.399599+0000","last_peered":"2024-03-23T09:42:49.399599+0000","last_clean":"2024-03-23T09:42:49.399599+0000","last_became_active":"2024-03-23T09:42:48.187819+0000","last_became_peered":"2024-03-23T09:42:48.187819+0000","last_unstale":"2024-03-23T09:42:49.399599+0000","last_undegraded":"2024-03-23T09:42:49.399599+0000","last_fullsized":"2024-03-23T09:42:49.399599+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:40:06.827216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997101+0000","last_change":"2024-03-23T09:42:48.191118+0000","last_active":"2024-03-23T09:42:49.997101+0000","last_peered":"2024-03-23T09:42:49.997101+0000","last_clean":"2024-03-23T09:42:49.997101+0000","last_became_active":"2024-03-23T09:42:48.190842+0000","last_became_peered":"2024-03-23T09:42:48.190842+0000","last_unstale":"2024-03-23T09:42:49.997101+0000","last_undegraded":"2024-03-23T09:42:49.997101+0000","last_fullsized":"2024-03-23T09:42:49.997101+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:09:59.751192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996731+0000","last_change":"2024-03-23T09:42:48.179255+0000","last_active":"2024-03-23T09:42:49.996731+0000","last_peered":"2024-03-23T09:42:49.996731+0000","last_clean":"2024-03-23T09:42:49.996731+0000","last_became_active":"2024-03-23T09:42:48.179016+0000","last_became_peered":"2024-03-23T09:42:48.179016+0000","last_unstale":"2024-03-23T09:42:49.996731+0000","last_undegraded":"2024-03-23T09:42:49.996731+0000","last_fullsized":"2024-03-23T09:42:49.996731+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:31:16.021407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.174698+0000","last_change":"2024-03-23T09:42:48.183798+0000","last_active":"2024-03-23T09:42:49.174698+0000","last_peered":"2024-03-23T09:42:49.174698+0000","last_clean":"2024-03-23T09:42:49.174698+0000","last_became_active":"2024-03-23T09:42:48.181603+0000","last_became_peered":"2024-03-23T09:42:48.181603+0000","last_unstale":"2024-03-23T09:42:49.174698+0000","last_undegraded":"2024-03-23T09:42:49.174698+0000","last_fullsized":"2024-03-23T09:42:49.174698+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:45.688229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399363+0000","last_change":"2024-03-23T09:42:48.188686+0000","last_active":"2024-03-23T09:42:49.399363+0000","last_peered":"2024-03-23T09:42:49.399363+0000","last_clean":"2024-03-23T09:42:49.399363+0000","last_became_active":"2024-03-23T09:42:48.188448+0000","last_became_peered":"2024-03-23T09:42:48.188448+0000","last_unstale":"2024-03-23T09:42:49.399363+0000","last_undegraded":"2024-03-23T09:42:49.399363+0000","last_fullsized":"2024-03-23T09:42:49.399363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:29:33.801428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399765+0000","last_change":"2024-03-23T09:42:48.183930+0000","last_active":"2024-03-23T09:42:49.399765+0000","last_peered":"2024-03-23T09:42:49.399765+0000","last_clean":"2024-03-23T09:42:49.399765+0000","last_became_active":"2024-03-23T09:42:48.183669+0000","last_became_peered":"2024-03-23T09:42:48.183669+0000","last_unstale":"2024-03-23T09:42:49.399765+0000","last_undegraded":"2024-03-23T09:42:49.399765+0000","last_fullsized":"2024-03-23T09:42:49.399765+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:43:43.179006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399806+0000","last_change":"2024-03-23T09:42:48.197176+0000","last_active":"2024-03-23T09:42:49.399806+0000","last_peered":"2024-03-23T09:42:49.399806+0000","last_clean":"2024-03-23T09:42:49.399806+0000","last_became_active":"2024-03-23T09:42:48.196890+0000","last_became_peered":"2024-03-23T09:42:48.196890+0000","last_unstale":"2024-03-23T09:42:49.399806+0000","last_undegraded":"2024-03-23T09:42:49.399806+0000","last_fullsized":"2024-03-23T09:42:49.399806+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:41.891469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399680+0000","last_change":"2024-03-23T09:42:48.187247+0000","last_active":"2024-03-23T09:42:49.399680+0000","last_peered":"2024-03-23T09:42:49.399680+0000","last_clean":"2024-03-23T09:42:49.399680+0000","last_became_active":"2024-03-23T09:42:48.187010+0000","last_became_peered":"2024-03-23T09:42:48.187010+0000","last_unstale":"2024-03-23T09:42:49.399680+0000","last_undegraded":"2024-03-23T09:42:49.399680+0000","last_fullsized":"2024-03-23T09:42:49.399680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:47:49.327792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997287+0000","last_change":"2024-03-23T09:42:48.176925+0000","last_active":"2024-03-23T09:42:49.997287+0000","last_peered":"2024-03-23T09:42:49.997287+0000","last_clean":"2024-03-23T09:42:49.997287+0000","last_became_active":"2024-03-23T09:42:48.176643+0000","last_became_peered":"2024-03-23T09:42:48.176643+0000","last_unstale":"2024-03-23T09:42:49.997287+0000","last_undegraded":"2024-03-23T09:42:49.997287+0000","last_fullsized":"2024-03-23T09:42:49.997287+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:43:43.179006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997228+0000","last_change":"2024-03-23T09:42:48.175294+0000","last_active":"2024-03-23T09:42:49.997228+0000","last_peered":"2024-03-23T09:42:49.997228+0000","last_clean":"2024-03-23T09:42:49.997228+0000","last_became_active":"2024-03-23T09:42:48.175043+0000","last_became_peered":"2024-03-23T09:42:48.175043+0000","last_unstale":"2024-03-23T09:42:49.997228+0000","last_undegraded":"2024-03-23T09:42:49.997228+0000","last_fullsized":"2024-03-23T09:42:49.997228+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:04:10.119324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399523+0000","last_change":"2024-03-23T09:42:48.193062+0000","last_active":"2024-03-23T09:42:49.399523+0000","last_peered":"2024-03-23T09:42:49.399523+0000","last_clean":"2024-03-23T09:42:49.399523+0000","last_became_active":"2024-03-23T09:42:48.192795+0000","last_became_peered":"2024-03-23T09:42:48.192795+0000","last_unstale":"2024-03-23T09:42:49.399523+0000","last_undegraded":"2024-03-23T09:42:49.399523+0000","last_fullsized":"2024-03-23T09:42:49.399523+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:23:15.765256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996738+0000","last_change":"2024-03-23T09:42:48.185467+0000","last_active":"2024-03-23T09:42:49.996738+0000","last_peered":"2024-03-23T09:42:49.996738+0000","last_clean":"2024-03-23T09:42:49.996738+0000","last_became_active":"2024-03-23T09:42:48.185214+0000","last_became_peered":"2024-03-23T09:42:48.185214+0000","last_unstale":"2024-03-23T09:42:49.996738+0000","last_undegraded":"2024-03-23T09:42:49.996738+0000","last_fullsized":"2024-03-23T09:42:49.996738+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:46:25.176131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997169+0000","last_change":"2024-03-23T09:42:48.195596+0000","last_active":"2024-03-23T09:42:49.997169+0000","last_peered":"2024-03-23T09:42:49.997169+0000","last_clean":"2024-03-23T09:42:49.997169+0000","last_became_active":"2024-03-23T09:42:48.194959+0000","last_became_peered":"2024-03-23T09:42:48.194959+0000","last_unstale":"2024-03-23T09:42:49.997169+0000","last_undegraded":"2024-03-23T09:42:49.997169+0000","last_fullsized":"2024-03-23T09:42:49.997169+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:04:25.195030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.175188+0000","last_change":"2024-03-23T09:42:48.189548+0000","last_active":"2024-03-23T09:42:49.175188+0000","last_peered":"2024-03-23T09:42:49.175188+0000","last_clean":"2024-03-23T09:42:49.175188+0000","last_became_active":"2024-03-23T09:42:48.187804+0000","last_became_peered":"2024-03-23T09:42:48.187804+0000","last_unstale":"2024-03-23T09:42:49.175188+0000","last_undegraded":"2024-03-23T09:42:49.175188+0000","last_fullsized":"2024-03-23T09:42:49.175188+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:27:57.837052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.175020+0000","last_change":"2024-03-23T09:42:48.195572+0000","last_active":"2024-03-23T09:42:49.175020+0000","last_peered":"2024-03-23T09:42:49.175020+0000","last_clean":"2024-03-23T09:42:49.175020+0000","last_became_active":"2024-03-23T09:42:48.195309+0000","last_became_peered":"2024-03-23T09:42:48.195309+0000","last_unstale":"2024-03-23T09:42:49.175020+0000","last_undegraded":"2024-03-23T09:42:49.175020+0000","last_fullsized":"2024-03-23T09:42:49.175020+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:35:54.519991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997923+0000","last_change":"2024-03-23T09:42:48.195037+0000","last_active":"2024-03-23T09:42:49.997923+0000","last_peered":"2024-03-23T09:42:49.997923+0000","last_clean":"2024-03-23T09:42:49.997923+0000","last_became_active":"2024-03-23T09:42:48.194731+0000","last_became_peered":"2024-03-23T09:42:48.194731+0000","last_unstale":"2024-03-23T09:42:49.997923+0000","last_undegraded":"2024-03-23T09:42:49.997923+0000","last_fullsized":"2024-03-23T09:42:49.997923+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:14:56.795321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996959+0000","last_change":"2024-03-23T09:42:48.186323+0000","last_active":"2024-03-23T09:42:49.996959+0000","last_peered":"2024-03-23T09:42:49.996959+0000","last_clean":"2024-03-23T09:42:49.996959+0000","last_became_active":"2024-03-23T09:42:48.186054+0000","last_became_peered":"2024-03-23T09:42:48.186054+0000","last_unstale":"2024-03-23T09:42:49.996959+0000","last_undegraded":"2024-03-23T09:42:49.996959+0000","last_fullsized":"2024-03-23T09:42:49.996959+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:04:25.195030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996674+0000","last_change":"2024-03-23T09:42:48.184916+0000","last_active":"2024-03-23T09:42:49.996674+0000","last_peered":"2024-03-23T09:42:49.996674+0000","last_clean":"2024-03-23T09:42:49.996674+0000","last_became_active":"2024-03-23T09:42:48.184670+0000","last_became_peered":"2024-03-23T09:42:48.184670+0000","last_unstale":"2024-03-23T09:42:49.996674+0000","last_undegraded":"2024-03-23T09:42:49.996674+0000","last_fullsized":"2024-03-23T09:42:49.996674+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:08:20.027896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399615+0000","last_change":"2024-03-23T09:42:48.192767+0000","last_active":"2024-03-23T09:42:49.399615+0000","last_peered":"2024-03-23T09:42:49.399615+0000","last_clean":"2024-03-23T09:42:49.399615+0000","last_became_active":"2024-03-23T09:42:48.191811+0000","last_became_peered":"2024-03-23T09:42:48.191811+0000","last_unstale":"2024-03-23T09:42:49.399615+0000","last_undegraded":"2024-03-23T09:42:49.399615+0000","last_fullsized":"2024-03-23T09:42:49.399615+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:46:27.574775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997408+0000","last_change":"2024-03-23T09:42:48.187543+0000","last_active":"2024-03-23T09:42:49.997408+0000","last_peered":"2024-03-23T09:42:49.997408+0000","last_clean":"2024-03-23T09:42:49.997408+0000","last_became_active":"2024-03-23T09:42:48.186995+0000","last_became_peered":"2024-03-23T09:42:48.186995+0000","last_unstale":"2024-03-23T09:42:49.997408+0000","last_undegraded":"2024-03-23T09:42:49.997408+0000","last_fullsized":"2024-03-23T09:42:49.997408+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:58:55.385331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997034+0000","last_change":"2024-03-23T09:42:48.186762+0000","last_active":"2024-03-23T09:42:49.997034+0000","last_peered":"2024-03-23T09:42:49.997034+0000","last_clean":"2024-03-23T09:42:49.997034+0000","last_became_active":"2024-03-23T09:42:48.186492+0000","last_became_peered":"2024-03-23T09:42:48.186492+0000","last_unstale":"2024-03-23T09:42:49.997034+0000","last_undegraded":"2024-03-23T09:42:49.997034+0000","last_fullsized":"2024-03-23T09:42:49.997034+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:26:25.684155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997650+0000","last_change":"2024-03-23T09:42:48.193023+0000","last_active":"2024-03-23T09:42:49.997650+0000","last_peered":"2024-03-23T09:42:49.997650+0000","last_clean":"2024-03-23T09:42:49.997650+0000","last_became_active":"2024-03-23T09:42:48.192740+0000","last_became_peered":"2024-03-23T09:42:48.192740+0000","last_unstale":"2024-03-23T09:42:49.997650+0000","last_undegraded":"2024-03-23T09:42:49.997650+0000","last_fullsized":"2024-03-23T09:42:49.997650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:14.572031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996854+0000","last_change":"2024-03-23T09:42:48.176044+0000","last_active":"2024-03-23T09:42:49.996854+0000","last_peered":"2024-03-23T09:42:49.996854+0000","last_clean":"2024-03-23T09:42:49.996854+0000","last_became_active":"2024-03-23T09:42:48.175747+0000","last_became_peered":"2024-03-23T09:42:48.175747+0000","last_unstale":"2024-03-23T09:42:49.996854+0000","last_undegraded":"2024-03-23T09:42:49.996854+0000","last_fullsized":"2024-03-23T09:42:49.996854+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:14:56.795321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.174967+0000","last_change":"2024-03-23T09:42:48.187206+0000","last_active":"2024-03-23T09:42:49.174967+0000","last_peered":"2024-03-23T09:42:49.174967+0000","last_clean":"2024-03-23T09:42:49.174967+0000","last_became_active":"2024-03-23T09:42:48.186690+0000","last_became_peered":"2024-03-23T09:42:48.186690+0000","last_unstale":"2024-03-23T09:42:49.174967+0000","last_undegraded":"2024-03-23T09:42:49.174967+0000","last_fullsized":"2024-03-23T09:42:49.174967+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:33.475189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997387+0000","last_change":"2024-03-23T09:42:48.176352+0000","last_active":"2024-03-23T09:42:49.997387+0000","last_peered":"2024-03-23T09:42:49.997387+0000","last_clean":"2024-03-23T09:42:49.997387+0000","last_became_active":"2024-03-23T09:42:48.176102+0000","last_became_peered":"2024-03-23T09:42:48.176102+0000","last_unstale":"2024-03-23T09:42:49.997387+0000","last_undegraded":"2024-03-23T09:42:49.997387+0000","last_fullsized":"2024-03-23T09:42:49.997387+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:42.241333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997275+0000","last_change":"2024-03-23T09:42:48.191297+0000","last_active":"2024-03-23T09:42:49.997275+0000","last_peered":"2024-03-23T09:42:49.997275+0000","last_clean":"2024-03-23T09:42:49.997275+0000","last_became_active":"2024-03-23T09:42:48.191049+0000","last_became_peered":"2024-03-23T09:42:48.191049+0000","last_unstale":"2024-03-23T09:42:49.997275+0000","last_undegraded":"2024-03-23T09:42:49.997275+0000","last_fullsized":"2024-03-23T09:42:49.997275+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:14:50.253330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.174623+0000","last_change":"2024-03-23T09:42:48.186294+0000","last_active":"2024-03-23T09:42:49.174623+0000","last_peered":"2024-03-23T09:42:49.174623+0000","last_clean":"2024-03-23T09:42:49.174623+0000","last_became_active":"2024-03-23T09:42:48.186054+0000","last_became_peered":"2024-03-23T09:42:48.186054+0000","last_unstale":"2024-03-23T09:42:49.174623+0000","last_undegraded":"2024-03-23T09:42:49.174623+0000","last_fullsized":"2024-03-23T09:42:49.174623+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:04:25.195030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996640+0000","last_change":"2024-03-23T09:42:48.181787+0000","last_active":"2024-03-23T09:42:49.996640+0000","last_peered":"2024-03-23T09:42:49.996640+0000","last_clean":"2024-03-23T09:42:49.996640+0000","last_became_active":"2024-03-23T09:42:48.181562+0000","last_became_peered":"2024-03-23T09:42:48.181562+0000","last_unstale":"2024-03-23T09:42:49.996640+0000","last_undegraded":"2024-03-23T09:42:49.996640+0000","last_fullsized":"2024-03-23T09:42:49.996640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:52:58.391416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996548+0000","last_change":"2024-03-23T09:42:48.175122+0000","last_active":"2024-03-23T09:42:49.996548+0000","last_peered":"2024-03-23T09:42:49.996548+0000","last_clean":"2024-03-23T09:42:49.996548+0000","last_became_active":"2024-03-23T09:42:48.174823+0000","last_became_peered":"2024-03-23T09:42:48.174823+0000","last_unstale":"2024-03-23T09:42:49.996548+0000","last_undegraded":"2024-03-23T09:42:49.996548+0000","last_fullsized":"2024-03-23T09:42:49.996548+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:14:50.253330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.173515+0000","last_change":"2024-03-23T09:42:48.175572+0000","last_active":"2024-03-23T09:42:49.173515+0000","last_peered":"2024-03-23T09:42:49.173515+0000","last_clean":"2024-03-23T09:42:49.173515+0000","last_became_active":"2024-03-23T09:42:48.175310+0000","last_became_peered":"2024-03-23T09:42:48.175310+0000","last_unstale":"2024-03-23T09:42:49.173515+0000","last_undegraded":"2024-03-23T09:42:49.173515+0000","last_fullsized":"2024-03-23T09:42:49.173515+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:14:56.795321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.196264+0000","last_change":"2024-03-23T09:42:47.809774+0000","last_active":"2024-03-23T09:42:48.196264+0000","last_peered":"2024-03-23T09:42:48.196264+0000","last_clean":"2024-03-23T09:42:48.196264+0000","last_became_active":"2024-03-23T09:42:47.809516+0000","last_became_peered":"2024-03-23T09:42:47.809516+0000","last_unstale":"2024-03-23T09:42:48.196264+0000","last_undegraded":"2024-03-23T09:42:48.196264+0000","last_fullsized":"2024-03-23T09:42:48.196264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:41.236379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997589+0000","last_change":"2024-03-23T09:42:48.188703+0000","last_active":"2024-03-23T09:42:49.997589+0000","last_peered":"2024-03-23T09:42:49.997589+0000","last_clean":"2024-03-23T09:42:49.997589+0000","last_became_active":"2024-03-23T09:42:48.188400+0000","last_became_peered":"2024-03-23T09:42:48.188400+0000","last_unstale":"2024-03-23T09:42:49.997589+0000","last_undegraded":"2024-03-23T09:42:49.997589+0000","last_fullsized":"2024-03-23T09:42:49.997589+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:04:10.119324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.195617+0000","last_change":"2024-03-23T09:42:47.806065+0000","last_active":"2024-03-23T09:42:48.195617+0000","last_peered":"2024-03-23T09:42:48.195617+0000","last_clean":"2024-03-23T09:42:48.195617+0000","last_became_active":"2024-03-23T09:42:47.805815+0000","last_became_peered":"2024-03-23T09:42:47.805815+0000","last_unstale":"2024-03-23T09:42:48.195617+0000","last_undegraded":"2024-03-23T09:42:48.195617+0000","last_fullsized":"2024-03-23T09:42:48.195617+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997484+0000","last_change":"2024-03-23T09:42:48.177470+0000","last_active":"2024-03-23T09:42:49.997484+0000","last_peered":"2024-03-23T09:42:49.997484+0000","last_clean":"2024-03-23T09:42:49.997484+0000","last_became_active":"2024-03-23T09:42:48.177222+0000","last_became_peered":"2024-03-23T09:42:48.177222+0000","last_unstale":"2024-03-23T09:42:49.997484+0000","last_undegraded":"2024-03-23T09:42:49.997484+0000","last_fullsized":"2024-03-23T09:42:49.997484+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:41.891469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.192481+0000","last_change":"2024-03-23T09:42:47.809236+0000","last_active":"2024-03-23T09:42:48.192481+0000","last_peered":"2024-03-23T09:42:48.192481+0000","last_clean":"2024-03-23T09:42:48.192481+0000","last_became_active":"2024-03-23T09:42:47.808974+0000","last_became_peered":"2024-03-23T09:42:47.808974+0000","last_unstale":"2024-03-23T09:42:48.192481+0000","last_undegraded":"2024-03-23T09:42:48.192481+0000","last_fullsized":"2024-03-23T09:42:48.192481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:56:21.810507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.173614+0000","last_change":"2024-03-23T09:42:48.189496+0000","last_active":"2024-03-23T09:42:49.173614+0000","last_peered":"2024-03-23T09:42:49.173614+0000","last_clean":"2024-03-23T09:42:49.173614+0000","last_became_active":"2024-03-23T09:42:48.189253+0000","last_became_peered":"2024-03-23T09:42:48.189253+0000","last_unstale":"2024-03-23T09:42:49.173614+0000","last_undegraded":"2024-03-23T09:42:49.173614+0000","last_fullsized":"2024-03-23T09:42:49.173614+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:04:10.119324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.191225+0000","last_change":"2024-03-23T09:42:47.806679+0000","last_active":"2024-03-23T09:42:48.191225+0000","last_peered":"2024-03-23T09:42:48.191225+0000","last_clean":"2024-03-23T09:42:48.191225+0000","last_became_active":"2024-03-23T09:42:47.806443+0000","last_became_peered":"2024-03-23T09:42:47.806443+0000","last_unstale":"2024-03-23T09:42:48.191225+0000","last_undegraded":"2024-03-23T09:42:48.191225+0000","last_fullsized":"2024-03-23T09:42:48.191225+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:04:24.190076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.174720+0000","last_change":"2024-03-23T09:42:48.185071+0000","last_active":"2024-03-23T09:42:49.174720+0000","last_peered":"2024-03-23T09:42:49.174720+0000","last_clean":"2024-03-23T09:42:49.174720+0000","last_became_active":"2024-03-23T09:42:48.184811+0000","last_became_peered":"2024-03-23T09:42:48.184811+0000","last_unstale":"2024-03-23T09:42:49.174720+0000","last_undegraded":"2024-03-23T09:42:49.174720+0000","last_fullsized":"2024-03-23T09:42:49.174720+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:44.937138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.196375+0000","last_change":"2024-03-23T09:42:47.803236+0000","last_active":"2024-03-23T09:42:48.196375+0000","last_peered":"2024-03-23T09:42:48.196375+0000","last_clean":"2024-03-23T09:42:48.196375+0000","last_became_active":"2024-03-23T09:42:47.802972+0000","last_became_peered":"2024-03-23T09:42:47.802972+0000","last_unstale":"2024-03-23T09:42:48.196375+0000","last_undegraded":"2024-03-23T09:42:48.196375+0000","last_fullsized":"2024-03-23T09:42:48.196375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:40.886514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399717+0000","last_change":"2024-03-23T09:42:48.189429+0000","last_active":"2024-03-23T09:42:49.399717+0000","last_peered":"2024-03-23T09:42:49.399717+0000","last_clean":"2024-03-23T09:42:49.399717+0000","last_became_active":"2024-03-23T09:42:48.189163+0000","last_became_peered":"2024-03-23T09:42:48.189163+0000","last_unstale":"2024-03-23T09:42:49.399717+0000","last_undegraded":"2024-03-23T09:42:49.399717+0000","last_fullsized":"2024-03-23T09:42:49.399717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:45.688229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.196164+0000","last_change":"2024-03-23T09:42:47.806574+0000","last_active":"2024-03-23T09:42:48.196164+0000","last_peered":"2024-03-23T09:42:48.196164+0000","last_clean":"2024-03-23T09:42:48.196164+0000","last_became_active":"2024-03-23T09:42:47.806324+0000","last_became_peered":"2024-03-23T09:42:47.806324+0000","last_unstale":"2024-03-23T09:42:48.196164+0000","last_undegraded":"2024-03-23T09:42:48.196164+0000","last_fullsized":"2024-03-23T09:42:48.196164+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:58:54.380376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399395+0000","last_change":"2024-03-23T09:42:48.188470+0000","last_active":"2024-03-23T09:42:49.399395+0000","last_peered":"2024-03-23T09:42:49.399395+0000","last_clean":"2024-03-23T09:42:49.399395+0000","last_became_active":"2024-03-23T09:42:48.188226+0000","last_became_peered":"2024-03-23T09:42:48.188226+0000","last_unstale":"2024-03-23T09:42:49.399395+0000","last_undegraded":"2024-03-23T09:42:49.399395+0000","last_fullsized":"2024-03-23T09:42:49.399395+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:47.560348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.174079+0000","last_change":"2024-03-23T09:42:47.804313+0000","last_active":"2024-03-23T09:42:48.174079+0000","last_peered":"2024-03-23T09:42:48.174079+0000","last_clean":"2024-03-23T09:42:48.174079+0000","last_became_active":"2024-03-23T09:42:47.804077+0000","last_became_peered":"2024-03-23T09:42:47.804077+0000","last_unstale":"2024-03-23T09:42:48.174079+0000","last_undegraded":"2024-03-23T09:42:48.174079+0000","last_fullsized":"2024-03-23T09:42:48.174079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:04:53.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997232+0000","last_change":"2024-03-23T09:42:48.187606+0000","last_active":"2024-03-23T09:42:49.997232+0000","last_peered":"2024-03-23T09:42:49.997232+0000","last_clean":"2024-03-23T09:42:49.997232+0000","last_became_active":"2024-03-23T09:42:48.187346+0000","last_became_peered":"2024-03-23T09:42:48.187346+0000","last_unstale":"2024-03-23T09:42:49.997232+0000","last_undegraded":"2024-03-23T09:42:49.997232+0000","last_fullsized":"2024-03-23T09:42:49.997232+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:24:37.260888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.177496+0000","last_change":"2024-03-23T09:42:47.812563+0000","last_active":"2024-03-23T09:42:48.177496+0000","last_peered":"2024-03-23T09:42:48.177496+0000","last_clean":"2024-03-23T09:42:48.177496+0000","last_became_active":"2024-03-23T09:42:47.812275+0000","last_became_peered":"2024-03-23T09:42:47.812275+0000","last_unstale":"2024-03-23T09:42:48.177496+0000","last_undegraded":"2024-03-23T09:42:48.177496+0000","last_fullsized":"2024-03-23T09:42:48.177496+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.175053+0000","last_change":"2024-03-23T09:42:48.194885+0000","last_active":"2024-03-23T09:42:49.175053+0000","last_peered":"2024-03-23T09:42:49.175053+0000","last_clean":"2024-03-23T09:42:49.175053+0000","last_became_active":"2024-03-23T09:42:48.194194+0000","last_became_peered":"2024-03-23T09:42:48.194194+0000","last_unstale":"2024-03-23T09:42:49.175053+0000","last_undegraded":"2024-03-23T09:42:49.175053+0000","last_fullsized":"2024-03-23T09:42:49.175053+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:47.560348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.178346+0000","last_change":"2024-03-23T09:42:47.804979+0000","last_active":"2024-03-23T09:42:48.178346+0000","last_peered":"2024-03-23T09:42:48.178346+0000","last_clean":"2024-03-23T09:42:48.178346+0000","last_became_active":"2024-03-23T09:42:47.804696+0000","last_became_peered":"2024-03-23T09:42:47.804696+0000","last_unstale":"2024-03-23T09:42:48.178346+0000","last_undegraded":"2024-03-23T09:42:48.178346+0000","last_fullsized":"2024-03-23T09:42:48.178346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:01:51.451920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.175144+0000","last_change":"2024-03-23T09:42:48.185755+0000","last_active":"2024-03-23T09:42:49.175144+0000","last_peered":"2024-03-23T09:42:49.175144+0000","last_clean":"2024-03-23T09:42:49.175144+0000","last_became_active":"2024-03-23T09:42:48.185515+0000","last_became_peered":"2024-03-23T09:42:48.185515+0000","last_unstale":"2024-03-23T09:42:49.175144+0000","last_undegraded":"2024-03-23T09:42:49.175144+0000","last_fullsized":"2024-03-23T09:42:49.175144+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:04:00.608246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.183675+0000","last_change":"2024-03-23T09:42:47.815670+0000","last_active":"2024-03-23T09:42:48.183675+0000","last_peered":"2024-03-23T09:42:48.183675+0000","last_clean":"2024-03-23T09:42:48.183675+0000","last_became_active":"2024-03-23T09:42:47.815428+0000","last_became_peered":"2024-03-23T09:42:47.815428+0000","last_unstale":"2024-03-23T09:42:48.183675+0000","last_undegraded":"2024-03-23T09:42:48.183675+0000","last_fullsized":"2024-03-23T09:42:48.183675+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:02:27.446246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996988+0000","last_change":"2024-03-23T09:42:48.177946+0000","last_active":"2024-03-23T09:42:49.996988+0000","last_peered":"2024-03-23T09:42:49.996988+0000","last_clean":"2024-03-23T09:42:49.996988+0000","last_became_active":"2024-03-23T09:42:48.177695+0000","last_became_peered":"2024-03-23T09:42:48.177695+0000","last_unstale":"2024-03-23T09:42:49.996988+0000","last_undegraded":"2024-03-23T09:42:49.996988+0000","last_fullsized":"2024-03-23T09:42:49.996988+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:03:18.487401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.192568+0000","last_change":"2024-03-23T09:42:47.807541+0000","last_active":"2024-03-23T09:42:48.192568+0000","last_peered":"2024-03-23T09:42:48.192568+0000","last_clean":"2024-03-23T09:42:48.192568+0000","last_became_active":"2024-03-23T09:42:47.807285+0000","last_became_peered":"2024-03-23T09:42:47.807285+0000","last_unstale":"2024-03-23T09:42:48.192568+0000","last_undegraded":"2024-03-23T09:42:48.192568+0000","last_fullsized":"2024-03-23T09:42:48.192568+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:35:53.515036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.175201+0000","last_change":"2024-03-23T09:42:48.183233+0000","last_active":"2024-03-23T09:42:49.175201+0000","last_peered":"2024-03-23T09:42:49.175201+0000","last_clean":"2024-03-23T09:42:49.175201+0000","last_became_active":"2024-03-23T09:42:48.181922+0000","last_became_peered":"2024-03-23T09:42:48.181922+0000","last_unstale":"2024-03-23T09:42:49.175201+0000","last_undegraded":"2024-03-23T09:42:49.175201+0000","last_fullsized":"2024-03-23T09:42:49.175201+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:23:15.765256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.198494+0000","last_change":"2024-03-23T09:42:47.809007+0000","last_active":"2024-03-23T09:42:48.198494+0000","last_peered":"2024-03-23T09:42:48.198494+0000","last_clean":"2024-03-23T09:42:48.198494+0000","last_became_active":"2024-03-23T09:42:47.808714+0000","last_became_peered":"2024-03-23T09:42:47.808714+0000","last_unstale":"2024-03-23T09:42:48.198494+0000","last_undegraded":"2024-03-23T09:42:48.198494+0000","last_fullsized":"2024-03-23T09:42:48.198494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:49:06.317828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997330+0000","last_change":"2024-03-23T09:42:48.194745+0000","last_active":"2024-03-23T09:42:49.997330+0000","last_peered":"2024-03-23T09:42:49.997330+0000","last_clean":"2024-03-23T09:42:49.997330+0000","last_became_active":"2024-03-23T09:42:48.194222+0000","last_became_peered":"2024-03-23T09:42:48.194222+0000","last_unstale":"2024-03-23T09:42:49.997330+0000","last_undegraded":"2024-03-23T09:42:49.997330+0000","last_fullsized":"2024-03-23T09:42:49.997330+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:04:54.467783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.189873+0000","last_change":"2024-03-23T09:42:47.812568+0000","last_active":"2024-03-23T09:42:48.189873+0000","last_peered":"2024-03-23T09:42:48.189873+0000","last_clean":"2024-03-23T09:42:48.189873+0000","last_became_active":"2024-03-23T09:42:47.812300+0000","last_became_peered":"2024-03-23T09:42:47.812300+0000","last_unstale":"2024-03-23T09:42:48.189873+0000","last_undegraded":"2024-03-23T09:42:48.189873+0000","last_fullsized":"2024-03-23T09:42:48.189873+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:43:42.174052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399939+0000","last_change":"2024-03-23T09:42:48.194525+0000","last_active":"2024-03-23T09:42:49.399939+0000","last_peered":"2024-03-23T09:42:49.399939+0000","last_clean":"2024-03-23T09:42:49.399939+0000","last_became_active":"2024-03-23T09:42:48.193671+0000","last_became_peered":"2024-03-23T09:42:48.193671+0000","last_unstale":"2024-03-23T09:42:49.399939+0000","last_undegraded":"2024-03-23T09:42:49.399939+0000","last_fullsized":"2024-03-23T09:42:49.399939+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:42.241333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.172038+0000","last_change":"2024-03-23T09:42:47.802576+0000","last_active":"2024-03-23T09:42:48.172038+0000","last_peered":"2024-03-23T09:42:48.172038+0000","last_clean":"2024-03-23T09:42:48.172038+0000","last_became_active":"2024-03-23T09:42:47.802245+0000","last_became_peered":"2024-03-23T09:42:47.802245+0000","last_unstale":"2024-03-23T09:42:48.172038+0000","last_undegraded":"2024-03-23T09:42:48.172038+0000","last_fullsized":"2024-03-23T09:42:48.172038+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:32.470235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996547+0000","last_change":"2024-03-23T09:42:48.177748+0000","last_active":"2024-03-23T09:42:49.996547+0000","last_peered":"2024-03-23T09:42:49.996547+0000","last_clean":"2024-03-23T09:42:49.996547+0000","last_became_active":"2024-03-23T09:42:48.177491+0000","last_became_peered":"2024-03-23T09:42:48.177491+0000","last_unstale":"2024-03-23T09:42:49.996547+0000","last_undegraded":"2024-03-23T09:42:49.996547+0000","last_fullsized":"2024-03-23T09:42:49.996547+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:28:45.381260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.172484+0000","last_change":"2024-03-23T09:42:47.810773+0000","last_active":"2024-03-23T09:42:48.172484+0000","last_peered":"2024-03-23T09:42:48.172484+0000","last_clean":"2024-03-23T09:42:48.172484+0000","last_became_active":"2024-03-23T09:42:47.810507+0000","last_became_peered":"2024-03-23T09:42:47.810507+0000","last_unstale":"2024-03-23T09:42:48.172484+0000","last_undegraded":"2024-03-23T09:42:48.172484+0000","last_fullsized":"2024-03-23T09:42:48.172484+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:40:05.822261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.174798+0000","last_change":"2024-03-23T09:42:48.188343+0000","last_active":"2024-03-23T09:42:49.174798+0000","last_peered":"2024-03-23T09:42:49.174798+0000","last_clean":"2024-03-23T09:42:49.174798+0000","last_became_active":"2024-03-23T09:42:48.187816+0000","last_became_peered":"2024-03-23T09:42:48.187816+0000","last_unstale":"2024-03-23T09:42:49.174798+0000","last_undegraded":"2024-03-23T09:42:49.174798+0000","last_fullsized":"2024-03-23T09:42:49.174798+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:46:27.574775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":19,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.195903+0000","last_change":"2024-03-23T09:42:47.811063+0000","last_active":"2024-03-23T09:42:48.195903+0000","last_peered":"2024-03-23T09:42:48.195903+0000","last_clean":"2024-03-23T09:42:48.195903+0000","last_became_active":"2024-03-23T09:42:47.810791+0000","last_became_peered":"2024-03-23T09:42:47.810791+0000","last_unstale":"2024-03-23T09:42:48.195903+0000","last_undegraded":"2024-03-23T09:42:48.195903+0000","last_fullsized":"2024-03-23T09:42:48.195903+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:01:51.451920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996951+0000","last_change":"2024-03-23T09:42:48.175423+0000","last_active":"2024-03-23T09:42:49.996951+0000","last_peered":"2024-03-23T09:42:49.996951+0000","last_clean":"2024-03-23T09:42:49.996951+0000","last_became_active":"2024-03-23T09:42:48.174759+0000","last_became_peered":"2024-03-23T09:42:48.174759+0000","last_unstale":"2024-03-23T09:42:49.996951+0000","last_undegraded":"2024-03-23T09:42:49.996951+0000","last_fullsized":"2024-03-23T09:42:49.996951+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:47:49.327792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.194080+0000","last_change":"2024-03-23T09:42:47.798923+0000","last_active":"2024-03-23T09:42:48.194080+0000","last_peered":"2024-03-23T09:42:48.194080+0000","last_clean":"2024-03-23T09:42:48.194080+0000","last_became_active":"2024-03-23T09:42:47.798648+0000","last_became_peered":"2024-03-23T09:42:47.798648+0000","last_unstale":"2024-03-23T09:42:48.194080+0000","last_undegraded":"2024-03-23T09:42:48.194080+0000","last_fullsized":"2024-03-23T09:42:48.194080+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:19.815833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399781+0000","last_change":"2024-03-23T09:42:48.184086+0000","last_active":"2024-03-23T09:42:49.399781+0000","last_peered":"2024-03-23T09:42:49.399781+0000","last_clean":"2024-03-23T09:42:49.399781+0000","last_became_active":"2024-03-23T09:42:48.183194+0000","last_became_peered":"2024-03-23T09:42:48.183194+0000","last_unstale":"2024-03-23T09:42:49.399781+0000","last_undegraded":"2024-03-23T09:42:49.399781+0000","last_fullsized":"2024-03-23T09:42:49.399781+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:04:54.467783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.187895+0000","last_change":"2024-03-23T09:42:47.801940+0000","last_active":"2024-03-23T09:42:48.187895+0000","last_peered":"2024-03-23T09:42:48.187895+0000","last_clean":"2024-03-23T09:42:48.187895+0000","last_became_active":"2024-03-23T09:42:47.801681+0000","last_became_peered":"2024-03-23T09:42:47.801681+0000","last_unstale":"2024-03-23T09:42:48.187895+0000","last_undegraded":"2024-03-23T09:42:48.187895+0000","last_fullsized":"2024-03-23T09:42:48.187895+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:19.815833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997390+0000","last_change":"2024-03-23T09:42:48.183012+0000","last_active":"2024-03-23T09:42:49.997390+0000","last_peered":"2024-03-23T09:42:49.997390+0000","last_clean":"2024-03-23T09:42:49.997390+0000","last_became_active":"2024-03-23T09:42:48.182739+0000","last_became_peered":"2024-03-23T09:42:48.182739+0000","last_unstale":"2024-03-23T09:42:49.997390+0000","last_undegraded":"2024-03-23T09:42:49.997390+0000","last_fullsized":"2024-03-23T09:42:49.997390+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:03.801381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.190256+0000","last_change":"2024-03-23T09:42:47.805106+0000","last_active":"2024-03-23T09:42:48.190256+0000","last_peered":"2024-03-23T09:42:48.190256+0000","last_clean":"2024-03-23T09:42:48.190256+0000","last_became_active":"2024-03-23T09:42:47.804875+0000","last_became_peered":"2024-03-23T09:42:47.804875+0000","last_unstale":"2024-03-23T09:42:48.190256+0000","last_undegraded":"2024-03-23T09:42:48.190256+0000","last_fullsized":"2024-03-23T09:42:48.190256+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:47:48.322837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997811+0000","last_change":"2024-03-23T09:42:48.191961+0000","last_active":"2024-03-23T09:42:49.997811+0000","last_peered":"2024-03-23T09:42:49.997811+0000","last_clean":"2024-03-23T09:42:49.997811+0000","last_became_active":"2024-03-23T09:42:48.191629+0000","last_became_peered":"2024-03-23T09:42:48.191629+0000","last_unstale":"2024-03-23T09:42:49.997811+0000","last_undegraded":"2024-03-23T09:42:49.997811+0000","last_fullsized":"2024-03-23T09:42:49.997811+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:38.975020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.172458+0000","last_change":"2024-03-23T09:42:47.802036+0000","last_active":"2024-03-23T09:42:48.172458+0000","last_peered":"2024-03-23T09:42:48.172458+0000","last_clean":"2024-03-23T09:42:48.172458+0000","last_became_active":"2024-03-23T09:42:47.801763+0000","last_became_peered":"2024-03-23T09:42:47.801763+0000","last_unstale":"2024-03-23T09:42:48.172458+0000","last_undegraded":"2024-03-23T09:42:48.172458+0000","last_fullsized":"2024-03-23T09:42:48.172458+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:29:32.796473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996833+0000","last_change":"2024-03-23T09:42:48.181756+0000","last_active":"2024-03-23T09:42:49.996833+0000","last_peered":"2024-03-23T09:42:49.996833+0000","last_clean":"2024-03-23T09:42:49.996833+0000","last_became_active":"2024-03-23T09:42:48.181362+0000","last_became_peered":"2024-03-23T09:42:48.181362+0000","last_unstale":"2024-03-23T09:42:49.996833+0000","last_undegraded":"2024-03-23T09:42:49.996833+0000","last_fullsized":"2024-03-23T09:42:49.996833+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:07:47.508118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.195314+0000","last_change":"2024-03-23T09:42:47.802865+0000","last_active":"2024-03-23T09:42:48.195314+0000","last_peered":"2024-03-23T09:42:48.195314+0000","last_clean":"2024-03-23T09:42:48.195314+0000","last_became_active":"2024-03-23T09:42:47.802315+0000","last_became_peered":"2024-03-23T09:42:47.802315+0000","last_unstale":"2024-03-23T09:42:48.195314+0000","last_undegraded":"2024-03-23T09:42:48.195314+0000","last_fullsized":"2024-03-23T09:42:48.195314+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:46.555393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.173437+0000","last_change":"2024-03-23T09:42:48.185314+0000","last_active":"2024-03-23T09:42:49.173437+0000","last_peered":"2024-03-23T09:42:49.173437+0000","last_clean":"2024-03-23T09:42:49.173437+0000","last_became_active":"2024-03-23T09:42:48.185076+0000","last_became_peered":"2024-03-23T09:42:48.185076+0000","last_unstale":"2024-03-23T09:42:49.173437+0000","last_undegraded":"2024-03-23T09:42:49.173437+0000","last_fullsized":"2024-03-23T09:42:49.173437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:47:49.327792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.171622+0000","last_change":"2024-03-23T09:42:47.807582+0000","last_active":"2024-03-23T09:42:48.171622+0000","last_peered":"2024-03-23T09:42:48.171622+0000","last_clean":"2024-03-23T09:42:48.171622+0000","last_became_active":"2024-03-23T09:42:47.807073+0000","last_became_peered":"2024-03-23T09:42:47.807073+0000","last_unstale":"2024-03-23T09:42:48.171622+0000","last_undegraded":"2024-03-23T09:42:48.171622+0000","last_fullsized":"2024-03-23T09:42:48.171622+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:14:55.790367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996625+0000","last_change":"2024-03-23T09:42:48.177540+0000","last_active":"2024-03-23T09:42:49.996625+0000","last_peered":"2024-03-23T09:42:49.996625+0000","last_clean":"2024-03-23T09:42:49.996625+0000","last_became_active":"2024-03-23T09:42:48.177296+0000","last_became_peered":"2024-03-23T09:42:48.177296+0000","last_unstale":"2024-03-23T09:42:49.996625+0000","last_undegraded":"2024-03-23T09:42:49.996625+0000","last_fullsized":"2024-03-23T09:42:49.996625+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:04:54.467783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.182773+0000","last_change":"2024-03-23T09:42:47.812423+0000","last_active":"2024-03-23T09:42:48.182773+0000","last_peered":"2024-03-23T09:42:48.182773+0000","last_clean":"2024-03-23T09:42:48.182773+0000","last_became_active":"2024-03-23T09:42:47.812175+0000","last_became_peered":"2024-03-23T09:42:47.812175+0000","last_unstale":"2024-03-23T09:42:48.182773+0000","last_undegraded":"2024-03-23T09:42:48.182773+0000","last_fullsized":"2024-03-23T09:42:48.182773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:09:03.354548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399408+0000","last_change":"2024-03-23T09:42:48.189076+0000","last_active":"2024-03-23T09:42:49.399408+0000","last_peered":"2024-03-23T09:42:49.399408+0000","last_clean":"2024-03-23T09:42:49.399408+0000","last_became_active":"2024-03-23T09:42:48.188818+0000","last_became_peered":"2024-03-23T09:42:48.188818+0000","last_unstale":"2024-03-23T09:42:49.399408+0000","last_undegraded":"2024-03-23T09:42:49.399408+0000","last_fullsized":"2024-03-23T09:42:49.399408+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:27:22.905237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.174611+0000","last_change":"2024-03-23T09:42:47.803493+0000","last_active":"2024-03-23T09:42:48.174611+0000","last_peered":"2024-03-23T09:42:48.174611+0000","last_clean":"2024-03-23T09:42:48.174611+0000","last_became_active":"2024-03-23T09:42:47.803241+0000","last_became_peered":"2024-03-23T09:42:47.803241+0000","last_unstale":"2024-03-23T09:42:48.174611+0000","last_undegraded":"2024-03-23T09:42:48.174611+0000","last_fullsized":"2024-03-23T09:42:48.174611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:50:10.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.997616+0000","last_change":"2024-03-23T09:42:48.195380+0000","last_active":"2024-03-23T09:42:49.997616+0000","last_peered":"2024-03-23T09:42:49.997616+0000","last_clean":"2024-03-23T09:42:49.997616+0000","last_became_active":"2024-03-23T09:42:48.195054+0000","last_became_peered":"2024-03-23T09:42:48.195054+0000","last_unstale":"2024-03-23T09:42:49.997616+0000","last_undegraded":"2024-03-23T09:42:49.997616+0000","last_fullsized":"2024-03-23T09:42:49.997616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:47:49.327792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.172596+0000","last_change":"2024-03-23T09:42:47.810047+0000","last_active":"2024-03-23T09:42:48.172596+0000","last_peered":"2024-03-23T09:42:48.172596+0000","last_clean":"2024-03-23T09:42:48.172596+0000","last_became_active":"2024-03-23T09:42:47.809800+0000","last_became_peered":"2024-03-23T09:42:47.809800+0000","last_unstale":"2024-03-23T09:42:48.172596+0000","last_undegraded":"2024-03-23T09:42:48.172596+0000","last_fullsized":"2024-03-23T09:42:48.172596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:27:56.832097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399606+0000","last_change":"2024-03-23T09:42:48.189721+0000","last_active":"2024-03-23T09:42:49.399606+0000","last_peered":"2024-03-23T09:42:49.399606+0000","last_clean":"2024-03-23T09:42:49.399606+0000","last_became_active":"2024-03-23T09:42:48.189487+0000","last_became_peered":"2024-03-23T09:42:48.189487+0000","last_unstale":"2024-03-23T09:42:49.399606+0000","last_undegraded":"2024-03-23T09:42:49.399606+0000","last_fullsized":"2024-03-23T09:42:49.399606+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:33.475189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.173085+0000","last_change":"2024-03-23T09:42:47.804445+0000","last_active":"2024-03-23T09:42:48.173085+0000","last_peered":"2024-03-23T09:42:48.173085+0000","last_clean":"2024-03-23T09:42:48.173085+0000","last_became_active":"2024-03-23T09:42:47.804215+0000","last_became_peered":"2024-03-23T09:42:47.804215+0000","last_unstale":"2024-03-23T09:42:48.173085+0000","last_undegraded":"2024-03-23T09:42:48.173085+0000","last_fullsized":"2024-03-23T09:42:48.173085+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:27:56.832097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.175331+0000","last_change":"2024-03-23T09:42:48.182032+0000","last_active":"2024-03-23T09:42:49.175331+0000","last_peered":"2024-03-23T09:42:49.175331+0000","last_clean":"2024-03-23T09:42:49.175331+0000","last_became_active":"2024-03-23T09:42:48.181537+0000","last_became_peered":"2024-03-23T09:42:48.181537+0000","last_unstale":"2024-03-23T09:42:49.175331+0000","last_undegraded":"2024-03-23T09:42:49.175331+0000","last_fullsized":"2024-03-23T09:42:49.175331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:52:05.223675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.171764+0000","last_change":"2024-03-23T09:42:47.809006+0000","last_active":"2024-03-23T09:42:48.171764+0000","last_peered":"2024-03-23T09:42:48.171764+0000","last_clean":"2024-03-23T09:42:48.171764+0000","last_became_active":"2024-03-23T09:42:47.808721+0000","last_became_peered":"2024-03-23T09:42:47.808721+0000","last_unstale":"2024-03-23T09:42:48.171764+0000","last_undegraded":"2024-03-23T09:42:48.171764+0000","last_fullsized":"2024-03-23T09:42:48.171764+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:37.970066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996934+0000","last_change":"2024-03-23T09:42:48.177419+0000","last_active":"2024-03-23T09:42:49.996934+0000","last_peered":"2024-03-23T09:42:49.996934+0000","last_clean":"2024-03-23T09:42:49.996934+0000","last_became_active":"2024-03-23T09:42:48.177170+0000","last_became_peered":"2024-03-23T09:42:48.177170+0000","last_unstale":"2024-03-23T09:42:49.996934+0000","last_undegraded":"2024-03-23T09:42:49.996934+0000","last_fullsized":"2024-03-23T09:42:49.996934+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:50:51.795930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.172208+0000","last_change":"2024-03-23T09:42:47.803568+0000","last_active":"2024-03-23T09:42:48.172208+0000","last_peered":"2024-03-23T09:42:48.172208+0000","last_clean":"2024-03-23T09:42:48.172208+0000","last_became_active":"2024-03-23T09:42:47.803222+0000","last_became_peered":"2024-03-23T09:42:47.803222+0000","last_unstale":"2024-03-23T09:42:48.172208+0000","last_undegraded":"2024-03-23T09:42:48.172208+0000","last_fullsized":"2024-03-23T09:42:48.172208+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:43.932184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996740+0000","last_change":"2024-03-23T09:42:48.176794+0000","last_active":"2024-03-23T09:42:49.996740+0000","last_peered":"2024-03-23T09:42:49.996740+0000","last_clean":"2024-03-23T09:42:49.996740+0000","last_became_active":"2024-03-23T09:42:48.176537+0000","last_became_peered":"2024-03-23T09:42:48.176537+0000","last_unstale":"2024-03-23T09:42:49.996740+0000","last_undegraded":"2024-03-23T09:42:49.996740+0000","last_fullsized":"2024-03-23T09:42:49.996740+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:06:38.700794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.177476+0000","last_change":"2024-03-23T09:42:47.800886+0000","last_active":"2024-03-23T09:42:48.177476+0000","last_peered":"2024-03-23T09:42:48.177476+0000","last_clean":"2024-03-23T09:42:48.177476+0000","last_became_active":"2024-03-23T09:42:47.800582+0000","last_became_peered":"2024-03-23T09:42:47.800582+0000","last_unstale":"2024-03-23T09:42:48.177476+0000","last_undegraded":"2024-03-23T09:42:48.177476+0000","last_fullsized":"2024-03-23T09:42:48.177476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:02.796426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.996931+0000","last_change":"2024-03-23T09:42:48.185051+0000","last_active":"2024-03-23T09:42:49.996931+0000","last_peered":"2024-03-23T09:42:49.996931+0000","last_clean":"2024-03-23T09:42:49.996931+0000","last_became_active":"2024-03-23T09:42:48.184651+0000","last_became_peered":"2024-03-23T09:42:48.184651+0000","last_unstale":"2024-03-23T09:42:49.996931+0000","last_undegraded":"2024-03-23T09:42:49.996931+0000","last_fullsized":"2024-03-23T09:42:49.996931+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:27:22.905237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.171284+0000","last_change":"2024-03-23T09:42:47.811037+0000","last_active":"2024-03-23T09:42:48.171284+0000","last_peered":"2024-03-23T09:42:48.171284+0000","last_clean":"2024-03-23T09:42:48.171284+0000","last_became_active":"2024-03-23T09:42:47.810746+0000","last_became_peered":"2024-03-23T09:42:47.810746+0000","last_unstale":"2024-03-23T09:42:48.171284+0000","last_undegraded":"2024-03-23T09:42:48.171284+0000","last_fullsized":"2024-03-23T09:42:48.171284+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:24:36.255934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.174549+0000","last_change":"2024-03-23T09:42:48.187874+0000","last_active":"2024-03-23T09:42:49.174549+0000","last_peered":"2024-03-23T09:42:49.174549+0000","last_clean":"2024-03-23T09:42:49.174549+0000","last_became_active":"2024-03-23T09:42:48.187607+0000","last_became_peered":"2024-03-23T09:42:48.187607+0000","last_unstale":"2024-03-23T09:42:49.174549+0000","last_undegraded":"2024-03-23T09:42:49.174549+0000","last_fullsized":"2024-03-23T09:42:49.174549+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:50:11.573027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.177674+0000","last_change":"2024-03-23T09:42:47.810719+0000","last_active":"2024-03-23T09:42:48.177674+0000","last_peered":"2024-03-23T09:42:48.177674+0000","last_clean":"2024-03-23T09:42:48.177674+0000","last_became_active":"2024-03-23T09:42:47.810364+0000","last_became_peered":"2024-03-23T09:42:47.810364+0000","last_unstale":"2024-03-23T09:42:48.177674+0000","last_undegraded":"2024-03-23T09:42:48.177674+0000","last_fullsized":"2024-03-23T09:42:48.177674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:02.796426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-23T09:42:49.399834+0000","last_change":"2024-03-23T09:42:48.193686+0000","last_active":"2024-03-23T09:42:49.399834+0000","last_peered":"2024-03-23T09:42:49.399834+0000","last_clean":"2024-03-23T09:42:49.399834+0000","last_became_active":"2024-03-23T09:42:48.193483+0000","last_became_peered":"2024-03-23T09:42:48.193483+0000","last_unstale":"2024-03-23T09:42:49.399834+0000","last_undegraded":"2024-03-23T09:42:49.399834+0000","last_fullsized":"2024-03-23T09:42:49.399834+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:47.787316+0000","last_clean_scrub_stamp":"2024-03-23T09:42:47.787316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:09:59.751192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.193304+0000","last_change":"2024-03-23T09:42:47.806133+0000","last_active":"2024-03-23T09:42:48.193304+0000","last_peered":"2024-03-23T09:42:48.193304+0000","last_clean":"2024-03-23T09:42:48.193304+0000","last_became_active":"2024-03-23T09:42:47.805886+0000","last_became_peered":"2024-03-23T09:42:47.805886+0000","last_unstale":"2024-03-23T09:42:48.193304+0000","last_undegraded":"2024-03-23T09:42:48.193304+0000","last_fullsized":"2024-03-23T09:42:48.193304+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:42:46.782362+0000","last_clean_scrub_stamp":"2024-03-23T09:42:46.782362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:04:09.114370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-23T09:42:48.174707+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:42:48.174707+0000","last_peered":"2024-03-23T09:42:48.174707+0000","last_clean":"2024-03-23T09:42:48.174707+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:42:48.174707+0000","last_undegraded":"2024-03-23T09:42:48.174707+0000","last_fullsized":"2024-03-23T09:42:48.174707+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1138688,"data_stored":809380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46300000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.51800000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.70799999999999996}]},{"osd":4,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.64800000000000002}]},{"osd":5,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.748}]},{"osd":6,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.73799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":218975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.43099999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.47799999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.58599999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.65700000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:41:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.57899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":218992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.439},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.53100000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.51300000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.67600000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.73799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":220486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.44400000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.38400000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.42199999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.47199999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.32600000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.45800000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.497}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31656,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":552960,"data_stored":219560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.308}]},{"osd":1,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.26000000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.36099999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40400000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42899999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.47099999999999997}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.50900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":557056,"data_stored":220508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.40500000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.372}]},{"osd":3,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.42999999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60299999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.65300000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.66500000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:41:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.62},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":544768,"data_stored":218992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.37},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.252}]},{"osd":2,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.34899999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":0.33200000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56100000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.49399999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.45400000000000001}]},{"osd":7,"last update":"Sat Mar 23 09:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.44600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1134592,"data_stored":808705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.443},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.33300000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.28999999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.38}]},{"osd":4,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.73299999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55300000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85999999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.72199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:42:55.921 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T09:42:55.921 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-23T09:42:56.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.064+0000 7fa7facbd640 1 Processor -- start 2024-03-23T09:42:56.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.064+0000 7fa7facbd640 1 -- start start 2024-03-23T09:42:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.064+0000 7fa7facbd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7f40cd710 0x7fa7f40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.064+0000 7fa7facbd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa7f40cdaf0 0x7fa7f41606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.064+0000 7fa7facbd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f40b4450 con 0x7fa7f40b1bc0 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.064+0000 7fa7facbd640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f40d2320 con 0x7fa7f40cd710 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.064+0000 7fa7facbd640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f40d2600 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.064+0000 7fa7f8a59640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7f40cd710 0x7fa7f40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.065+0000 7fa7f8a59640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7f40cd710 0x7fa7f40b4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59862/0 (socket says 172.21.15.47:59862) 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.065+0000 7fa7f8a59640 1 -- 172.21.15.47:0/3930109063 learned_addr learned my addr 172.21.15.47:0/3930109063 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.065+0000 7fa7f3fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa7f40b1bc0 0x7fa7f40d1b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52018/0 (socket says 172.21.15.47:52018) 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.065+0000 7fa7f925a640 1 --2- 172.21.15.47:0/3930109063 >> v2:172.21.15.175:3300/0 conn(0x7fa7f40cdaf0 0x7fa7f41606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.065+0000 7fa7f8a59640 1 -- 172.21.15.47:0/3930109063 >> v2:172.21.15.175:3300/0 conn(0x7fa7f40cdaf0 msgr2=0x7fa7f41606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.065+0000 7fa7f8a59640 1 --2- 172.21.15.47:0/3930109063 >> v2:172.21.15.175:3300/0 conn(0x7fa7f40cdaf0 0x7fa7f41606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.065+0000 7fa7f8a59640 1 -- 172.21.15.47:0/3930109063 >> v1:172.21.15.47:6789/0 conn(0x7fa7f40b1bc0 legacy=0x7fa7f40d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.065+0000 7fa7f8a59640 1 -- 172.21.15.47:0/3930109063 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f40b46a0 con 0x7fa7f40cd710 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.065+0000 7fa7f37fe640 1 -- 172.21.15.47:0/3930109063 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198540688 0 0) 0x7fa7f40b4450 con 0x7fa7f40b1bc0 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.066+0000 7fa7f8a59640 1 --2- 172.21.15.47:0/3930109063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f40cd710 0x7fa7f40b4ea0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa7f406c070 tx=0x7fa7e802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0aeec2df599964 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.066+0000 7fa7f37fe640 1 -- 172.21.15.47:0/3930109063 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8030860 con 0x7fa7f40cd710 2024-03-23T09:42:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.066+0000 7fa7f37fe640 1 -- 172.21.15.47:0/3930109063 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e80309c0 con 0x7fa7f40cd710 2024-03-23T09:42:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.066+0000 7fa7f37fe640 1 -- 172.21.15.47:0/3930109063 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8030d50 con 0x7fa7f40cd710 2024-03-23T09:42:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.066+0000 7fa7facbd640 1 -- 172.21.15.47:0/3930109063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f40cd710 msgr2=0x7fa7f40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.066+0000 7fa7facbd640 1 --2- 172.21.15.47:0/3930109063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f40cd710 0x7fa7f40b4ea0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa7f406c070 tx=0x7fa7e802b9a0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.066+0000 7fa7facbd640 1 -- 172.21.15.47:0/3930109063 shutdown_connections 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.066+0000 7fa7facbd640 1 --2- 172.21.15.47:0/3930109063 >> v2:172.21.15.175:3300/0 conn(0x7fa7f40cdaf0 0x7fa7f41606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.066+0000 7fa7facbd640 1 --2- 172.21.15.47:0/3930109063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f40cd710 0x7fa7f40b4ea0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.066+0000 7fa7facbd640 1 -- 172.21.15.47:0/3930109063 >> 172.21.15.47:0/3930109063 conn(0x7fa7f406afd0 msgr2=0x7fa7f406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.066+0000 7fa7facbd640 1 -- 172.21.15.47:0/3930109063 shutdown_connections 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.066+0000 7fa7facbd640 1 -- 172.21.15.47:0/3930109063 wait complete. 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.067+0000 7fa7facbd640 1 Processor -- start 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.068+0000 7fa7facbd640 1 -- start start 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.068+0000 7fa7facbd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7f40b1bc0 0x7fa7f415f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.068+0000 7fa7facbd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7f40cd710 0x7fa7f414d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.068+0000 7fa7facbd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f40cdaf0 0x7fa7f414dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.068+0000 7fa7facbd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f414e400 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.068+0000 7fa7facbd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f415d1a0 con 0x7fa7f40b1bc0 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.068+0000 7fa7facbd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f415d440 con 0x7fa7f40cd710 2024-03-23T09:42:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.068+0000 7fa7f925a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f40cdaf0 0x7fa7f414dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.068+0000 7fa7f925a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f40cdaf0 0x7fa7f414dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59864/0 (socket says 172.21.15.47:59864) 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.069+0000 7fa7f925a640 1 -- 172.21.15.47:0/3168945650 learned_addr learned my addr 172.21.15.47:0/3168945650 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.069+0000 7fa7f8a59640 1 --2- 172.21.15.47:0/3168945650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7f40b1bc0 0x7fa7f415f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.069+0000 7fa7f925a640 1 -- 172.21.15.47:0/3168945650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7f40cd710 msgr2=0x7fa7f414d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.069+0000 7fa7f925a640 1 --2- 172.21.15.47:0/3168945650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7f40cd710 0x7fa7f414d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.069+0000 7fa7f925a640 1 -- 172.21.15.47:0/3168945650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7f40b1bc0 msgr2=0x7fa7f415f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.069+0000 7fa7f925a640 1 --2- 172.21.15.47:0/3168945650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7f40b1bc0 0x7fa7f415f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.069+0000 7fa7f925a640 1 -- 172.21.15.47:0/3168945650 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e80094d0 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.069+0000 7fa7f925a640 1 --2- 172.21.15.47:0/3168945650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f40cdaf0 0x7fa7f414dec0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec007e60 tx=0x7fa7ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.069+0000 7fa7f17fa640 1 -- 172.21.15.47:0/3168945650 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec014420 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.069+0000 7fa7f17fa640 1 -- 172.21.15.47:0/3168945650 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec0145a0 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.069+0000 7fa7facbd640 1 -- 172.21.15.47:0/3168945650 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f415d720 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.069+0000 7fa7f17fa640 1 -- 172.21.15.47:0/3168945650 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec014950 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.069+0000 7fa7facbd640 1 -- 172.21.15.47:0/3168945650 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f415dce0 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.070+0000 7fa7facbd640 1 -- 172.21.15.47:0/3168945650 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f415df00 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.070+0000 7fa7f17fa640 1 -- 172.21.15.47:0/3168945650 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa7ec014ab0 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.071+0000 7fa7f17fa640 1 --2- 172.21.15.47:0/3168945650 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa7d806ac00 0x7fa7d806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.071+0000 7fa7f17fa640 1 -- 172.21.15.47:0/3168945650 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7ec089ce0 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.071+0000 7fa7f8a59640 1 --2- 172.21.15.47:0/3168945650 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa7d806ac00 0x7fa7d806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.072+0000 7fa7f8a59640 1 --2- 172.21.15.47:0/3168945650 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa7d806ac00 0x7fa7d806d0c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8036000 tx=0x7fa7e803b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:56.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.075+0000 7fa7f17fa640 1 -- 172.21.15.47:0/3168945650 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa7ec054ec0 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.325+0000 7fa7facbd640 1 -- 172.21.15.47:0/3168945650 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa7f40672f0 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.329+0000 7fa7f17fa640 1 -- 172.21.15.47:0/3168945650 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa7ec0584e0 con 0x7fa7f40cdaf0 2024-03-23T09:42:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-03-23T09:42:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.331+0000 7fa7facbd640 1 -- 172.21.15.47:0/3168945650 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa7d806ac00 msgr2=0x7fa7d806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.331+0000 7fa7facbd640 1 --2- 172.21.15.47:0/3168945650 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa7d806ac00 0x7fa7d806d0c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8036000 tx=0x7fa7e803b040 comp rx=0 tx=0).stop 2024-03-23T09:42:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.331+0000 7fa7facbd640 1 -- 172.21.15.47:0/3168945650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f40cdaf0 msgr2=0x7fa7f414dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.331+0000 7fa7facbd640 1 --2- 172.21.15.47:0/3168945650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f40cdaf0 0x7fa7f414dec0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec007e60 tx=0x7fa7ec00e4b0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.331+0000 7fa7facbd640 1 -- 172.21.15.47:0/3168945650 shutdown_connections 2024-03-23T09:42:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.331+0000 7fa7facbd640 1 --2- 172.21.15.47:0/3168945650 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa7d806ac00 0x7fa7d806d0c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.331+0000 7fa7facbd640 1 --2- 172.21.15.47:0/3168945650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f40cdaf0 0x7fa7f414dec0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.331+0000 7fa7facbd640 1 --2- 172.21.15.47:0/3168945650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7f40cd710 0x7fa7f414d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.331+0000 7fa7facbd640 1 --2- 172.21.15.47:0/3168945650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7f40b1bc0 0x7fa7f415f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.331+0000 7fa7facbd640 1 -- 172.21.15.47:0/3168945650 >> 172.21.15.47:0/3168945650 conn(0x7fa7f406afd0 msgr2=0x7fa7f40d1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.331+0000 7fa7facbd640 1 -- 172.21.15.47:0/3168945650 shutdown_connections 2024-03-23T09:42:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.331+0000 7fa7facbd640 1 -- 172.21.15.47:0/3168945650 wait complete. 2024-03-23T09:42:56.342 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:42:56.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.443+0000 7fe2c41bf640 1 Processor -- start 2024-03-23T09:42:56.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.443+0000 7fe2c41bf640 1 -- start start 2024-03-23T09:42:56.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.444+0000 7fe2c41bf640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2bc0b3c30 0x7fe2bc06d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:56.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.444+0000 7fe2c41bf640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe2bc142aa0 0x7fe2bc0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:56.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.444+0000 7fe2c41bf640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc06c8c0 con 0x7fe2bc06c350 2024-03-23T09:42:56.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.444+0000 7fe2c41bf640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2bc06cba0 con 0x7fe2bc142aa0 2024-03-23T09:42:56.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.444+0000 7fe2c41bf640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc06ce80 con 0x7fe2bc0b3c30 2024-03-23T09:42:56.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.444+0000 7fe2c1f5b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2bc0b3c30 0x7fe2bc06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:56.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.444+0000 7fe2c275c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe2bc142aa0 0x7fe2bc0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:56.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.444+0000 7fe2c175a640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fe2bc06c350 0x7fe2bc0b4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:39090/0 (socket says 172.21.15.47:39090) 2024-03-23T09:42:56.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.444+0000 7fe2c175a640 1 -- 172.21.15.47:0/2071897858 learned_addr learned my addr 172.21.15.47:0/2071897858 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:56.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.445+0000 7fe2c1f5b640 1 -- 172.21.15.47:0/2071897858 >> v2:172.21.15.175:3300/0 conn(0x7fe2bc142aa0 msgr2=0x7fe2bc0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:56.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.445+0000 7fe2c1f5b640 1 --2- 172.21.15.47:0/2071897858 >> v2:172.21.15.175:3300/0 conn(0x7fe2bc142aa0 0x7fe2bc0cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.445+0000 7fe2c1f5b640 1 -- 172.21.15.47:0/2071897858 >> v1:172.21.15.175:6789/0 conn(0x7fe2bc06c350 legacy=0x7fe2bc0b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:56.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.445+0000 7fe2c1f5b640 1 -- 172.21.15.47:0/2071897858 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc06d160 con 0x7fe2bc0b3c30 2024-03-23T09:42:56.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.445+0000 7fe2c275c640 1 --2- 172.21.15.47:0/2071897858 >> v2:172.21.15.175:3300/0 conn(0x7fe2bc142aa0 0x7fe2bc0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:42:56.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.445+0000 7fe2c1f5b640 1 --2- 172.21.15.47:0/2071897858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2bc0b3c30 0x7fe2bc06d960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80097a0 tx=0x7fe2b802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e24c312be7333355 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:56.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.446+0000 7fe2c0f59640 1 -- 172.21.15.47:0/2071897858 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8031830 con 0x7fe2bc0b3c30 2024-03-23T09:42:56.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.446+0000 7fe2c0f59640 1 -- 172.21.15.47:0/2071897858 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b8031990 con 0x7fe2bc0b3c30 2024-03-23T09:42:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.446+0000 7fe2c0f59640 1 -- 172.21.15.47:0/2071897858 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8031d20 con 0x7fe2bc0b3c30 2024-03-23T09:42:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.446+0000 7fe2c41bf640 1 -- 172.21.15.47:0/2071897858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2bc0b3c30 msgr2=0x7fe2bc06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.446+0000 7fe2c41bf640 1 --2- 172.21.15.47:0/2071897858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2bc0b3c30 0x7fe2bc06d960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80097a0 tx=0x7fe2b802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.446+0000 7fe2c41bf640 1 -- 172.21.15.47:0/2071897858 shutdown_connections 2024-03-23T09:42:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.446+0000 7fe2c41bf640 1 --2- 172.21.15.47:0/2071897858 >> v2:172.21.15.175:3300/0 conn(0x7fe2bc142aa0 0x7fe2bc0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.446+0000 7fe2c41bf640 1 --2- 172.21.15.47:0/2071897858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2bc0b3c30 0x7fe2bc06d960 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.446+0000 7fe2c41bf640 1 -- 172.21.15.47:0/2071897858 >> 172.21.15.47:0/2071897858 conn(0x7fe2bc06a310 msgr2=0x7fe2bc0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.446+0000 7fe2c41bf640 1 -- 172.21.15.47:0/2071897858 shutdown_connections 2024-03-23T09:42:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.446+0000 7fe2c41bf640 1 -- 172.21.15.47:0/2071897858 wait complete. 2024-03-23T09:42:56.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.447+0000 7fe2c41bf640 1 Processor -- start 2024-03-23T09:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.447+0000 7fe2c41bf640 1 -- start start 2024-03-23T09:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.448+0000 7fe2c41bf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2bc06c350 0x7fe2bc13b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.448+0000 7fe2c41bf640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2bc0b3c30 0x7fe2bc15b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.448+0000 7fe2c41bf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2bc142aa0 0x7fe2bc15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.448+0000 7fe2c41bf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc13bd90 con 0x7fe2bc142aa0 2024-03-23T09:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.448+0000 7fe2c41bf640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc13c070 con 0x7fe2bc06c350 2024-03-23T09:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.448+0000 7fe2c41bf640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc13c350 con 0x7fe2bc0b3c30 2024-03-23T09:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.448+0000 7fe2c275c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2bc142aa0 0x7fe2bc15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.448+0000 7fe2c275c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2bc142aa0 0x7fe2bc15b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59866/0 (socket says 172.21.15.47:59866) 2024-03-23T09:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.448+0000 7fe2c275c640 1 -- 172.21.15.47:0/81773080 learned_addr learned my addr 172.21.15.47:0/81773080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.448+0000 7fe2c1f5b640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2bc06c350 0x7fe2bc13b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.448+0000 7fe2c175a640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2bc0b3c30 0x7fe2bc15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.449+0000 7fe2c275c640 1 -- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2bc0b3c30 msgr2=0x7fe2bc15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.449+0000 7fe2c275c640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2bc0b3c30 0x7fe2bc15b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.449+0000 7fe2c275c640 1 -- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2bc06c350 msgr2=0x7fe2bc13b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.449+0000 7fe2c275c640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2bc06c350 0x7fe2bc13b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.449+0000 7fe2c275c640 1 -- 172.21.15.47:0/81773080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b4009a70 con 0x7fe2bc142aa0 2024-03-23T09:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.449+0000 7fe2c175a640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2bc0b3c30 0x7fe2bc15b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.449+0000 7fe2c275c640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2bc142aa0 0x7fe2bc15b5d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400efc0 tx=0x7fe2b4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.449+0000 7fe2b27fc640 1 -- 172.21.15.47:0/81773080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4009d70 con 0x7fe2bc142aa0 2024-03-23T09:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.449+0000 7fe2b27fc640 1 -- 172.21.15.47:0/81773080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b400f040 con 0x7fe2bc142aa0 2024-03-23T09:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.449+0000 7fe2c41bf640 1 -- 172.21.15.47:0/81773080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b8009400 con 0x7fe2bc142aa0 2024-03-23T09:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.449+0000 7fe2b27fc640 1 -- 172.21.15.47:0/81773080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4003990 con 0x7fe2bc142aa0 2024-03-23T09:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.449+0000 7fe2c41bf640 1 -- 172.21.15.47:0/81773080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc1f2380 con 0x7fe2bc142aa0 2024-03-23T09:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.450+0000 7fe2c41bf640 1 -- 172.21.15.47:0/81773080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc073480 con 0x7fe2bc142aa0 2024-03-23T09:42:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.450+0000 7fe2b27fc640 1 -- 172.21.15.47:0/81773080 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe2b40144a0 con 0x7fe2bc142aa0 2024-03-23T09:42:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.451+0000 7fe2b27fc640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe2a006ac00 0x7fe2a006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.451+0000 7fe2b27fc640 1 -- 172.21.15.47:0/81773080 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2b4089ef0 con 0x7fe2bc142aa0 2024-03-23T09:42:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.451+0000 7fe2c1f5b640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe2a006ac00 0x7fe2a006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.452+0000 7fe2c1f5b640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe2a006ac00 0x7fe2a006d0c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8036000 tx=0x7fe2b803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.455+0000 7fe2b27fc640 1 -- 172.21.15.47:0/81773080 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe2b4055160 con 0x7fe2bc142aa0 2024-03-23T09:42:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.670+0000 7fe2c41bf640 1 -- 172.21.15.47:0/81773080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2bc1f25a0 con 0x7fe2bc142aa0 2024-03-23T09:42:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.671+0000 7fe2b27fc640 1 -- 172.21.15.47:0/81773080 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fe2b4058780 con 0x7fe2bc142aa0 2024-03-23T09:42:56.672 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:56.672 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1189831856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1189831856},{"type":"v1","addr":"172.21.15.47:6835","nonce":1189831856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/2248064454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2248064454},{"type":"v1","addr":"172.21.15.47:6834","nonce":2248064454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5694,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3806465612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3806465612},{"type":"v1","addr":"172.21.15.175:6835","nonce":3806465612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:42:48.165819+0000","modified":"2024-03-23T09:42:50.171529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1626348200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1626348200},{"type":"v1","addr":"172.21.15.175:6837","nonce":1626348200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-23T09:42:56.672 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 18 2024-03-23T09:42:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.673+0000 7fe2c41bf640 1 -- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe2a006ac00 msgr2=0x7fe2a006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.673+0000 7fe2c41bf640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe2a006ac00 0x7fe2a006d0c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8036000 tx=0x7fe2b803c040 comp rx=0 tx=0).stop 2024-03-23T09:42:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.673+0000 7fe2c41bf640 1 -- 172.21.15.47:0/81773080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2bc142aa0 msgr2=0x7fe2bc15b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.673+0000 7fe2c41bf640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2bc142aa0 0x7fe2bc15b5d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400efc0 tx=0x7fe2b4009f90 comp rx=0 tx=0).stop 2024-03-23T09:42:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.673+0000 7fe2c41bf640 1 -- 172.21.15.47:0/81773080 shutdown_connections 2024-03-23T09:42:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.673+0000 7fe2c41bf640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe2a006ac00 0x7fe2a006d0c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.673+0000 7fe2c41bf640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2bc142aa0 0x7fe2bc15b5d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.673+0000 7fe2c41bf640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2bc0b3c30 0x7fe2bc15b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.673+0000 7fe2c41bf640 1 --2- 172.21.15.47:0/81773080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2bc06c350 0x7fe2bc13b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.673+0000 7fe2c41bf640 1 -- 172.21.15.47:0/81773080 >> 172.21.15.47:0/81773080 conn(0x7fe2bc06a310 msgr2=0x7fe2bc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:56.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.673+0000 7fe2c41bf640 1 -- 172.21.15.47:0/81773080 shutdown_connections 2024-03-23T09:42:56.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:56.673+0000 7fe2c41bf640 1 -- 172.21.15.47:0/81773080 wait complete. 2024-03-23T09:42:57.086 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:42:48.165819+0000', 'modified': '2024-03-23T09:42:50.171529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5697}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5697': {'gid': 5697, 'name': 'd', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6837/1626348200', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 1626348200}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 1626348200}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:42:57.087 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:42:57.087 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-23T09:42:57.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.189+0000 7f02839e3640 1 Processor -- start 2024-03-23T09:42:57.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.189+0000 7f02839e3640 1 -- start start 2024-03-23T09:42:57.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f02839e3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f027c0cdb40 0x7f027c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f02839e3640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027c0d09d0 con 0x7f027c0cd6b0 2024-03-23T09:42:57.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f02839e3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027c0b5a80 con 0x7f027c15cc40 2024-03-23T09:42:57.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f02839e3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027c15d020 con 0x7f027c0cdb40 2024-03-23T09:42:57.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0280f7e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f027c0cd6b0 0x7f027c0d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52030/0 (socket says 172.21.15.47:52030) 2024-03-23T09:42:57.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0280f7e640 1 -- 172.21.15.47:0/4126611869 learned_addr learned my addr 172.21.15.47:0/4126611869 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:57.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0281f80640 1 --2- 172.21.15.47:0/4126611869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f027c0cdb40 0x7f027c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2512302000 0 0) 0x7f027c0d09d0 con 0x7f027c0cd6b0 2024-03-23T09:42:57.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0268003520 con 0x7f027c0cd6b0 2024-03-23T09:42:57.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3516215516 0 0) 0x7f0268003520 con 0x7f027c0cd6b0 2024-03-23T09:42:57.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f027c0cdb40 msgr2=0x7f027c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 --2- 172.21.15.47:0/4126611869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f027c0cdb40 0x7f027c0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 >> v1:172.21.15.175:6789/0 conn(0x7f027c15cc40 legacy=0x7f027c160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c1611a0 con 0x7f027c0cd6b0 2024-03-23T09:42:57.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f026c001670 con 0x7f027c0cd6b0 2024-03-23T09:42:57.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 >> v1:172.21.15.47:6789/0 conn(0x7f027c0cd6b0 legacy=0x7f027c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 --2- 172.21.15.47:0/4126611869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f027c0cdb40 0x7f02680039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 --2- 172.21.15.47:0/4126611869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0268003ee0 0x7f02680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 --2- 172.21.15.47:0/4126611869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0268004d00 0x7f0268005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c15d020 con 0x7f0268003ee0 2024-03-23T09:42:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026800b9e0 con 0x7f0268004d00 2024-03-23T09:42:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026800bba0 con 0x7f027c0cdb40 2024-03-23T09:42:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.190+0000 7f0280f7e640 1 --2- 172.21.15.47:0/4126611869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0268003ee0 0x7f02680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.191+0000 7f028177f640 1 --2- 172.21.15.47:0/4126611869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f027c0cdb40 0x7f02680039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.191+0000 7f0281f80640 1 --2- 172.21.15.47:0/4126611869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0268004d00 0x7f0268005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.191+0000 7f0280f7e640 1 -- 172.21.15.47:0/4126611869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f027c0cdb40 msgr2=0x7f02680039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.191+0000 7f0280f7e640 1 --2- 172.21.15.47:0/4126611869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f027c0cdb40 0x7f02680039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.191+0000 7f0280f7e640 1 -- 172.21.15.47:0/4126611869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0268004d00 msgr2=0x7f0268005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.191+0000 7f0280f7e640 1 --2- 172.21.15.47:0/4126611869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0268004d00 0x7f0268005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.191+0000 7f0280f7e640 1 -- 172.21.15.47:0/4126611869 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0268003520 con 0x7f0268003ee0 2024-03-23T09:42:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.191+0000 7f0281f80640 1 --2- 172.21.15.47:0/4126611869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0268004d00 0x7f0268005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.191+0000 7f0280f7e640 1 --2- 172.21.15.47:0/4126611869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0268003ee0 0x7f02680047c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f026c00c210 tx=0x7f02680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffe6bbbf0afce98 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.192+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c005ad0 con 0x7f0268003ee0 2024-03-23T09:42:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.192+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c005c50 con 0x7f0268003ee0 2024-03-23T09:42:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.192+0000 7f0273fff640 1 -- 172.21.15.47:0/4126611869 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c02c810 con 0x7f0268003ee0 2024-03-23T09:42:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.192+0000 7f02839e3640 1 -- 172.21.15.47:0/4126611869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0268003ee0 msgr2=0x7f02680047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.192+0000 7f02839e3640 1 --2- 172.21.15.47:0/4126611869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0268003ee0 0x7f02680047c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f026c00c210 tx=0x7f02680057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.192+0000 7f0281f80640 1 -- 172.21.15.47:0/4126611869 reap_dead start 2024-03-23T09:42:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.192+0000 7f02839e3640 1 -- 172.21.15.47:0/4126611869 shutdown_connections 2024-03-23T09:42:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.192+0000 7f02839e3640 1 -- 172.21.15.47:0/4126611869 >> 172.21.15.47:0/4126611869 conn(0x7f027c06b160 msgr2=0x7f027c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.192+0000 7f02839e3640 1 -- 172.21.15.47:0/4126611869 shutdown_connections 2024-03-23T09:42:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.192+0000 7f02839e3640 1 -- 172.21.15.47:0/4126611869 wait complete. 2024-03-23T09:42:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.193+0000 7f02839e3640 1 Processor -- start 2024-03-23T09:42:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.193+0000 7f02839e3640 1 -- start start 2024-03-23T09:42:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.194+0000 7f02839e3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0268003ee0 0x7f027c15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.194+0000 7f02839e3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f027c15dd80 0x7f027c0c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.194+0000 7f02839e3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f027c15b280 0x7f027c15b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.194+0000 7f02839e3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c15bc40 con 0x7f027c15dd80 2024-03-23T09:42:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.194+0000 7f02839e3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c149280 con 0x7f027c15b280 2024-03-23T09:42:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.194+0000 7f02839e3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027c15e4f0 con 0x7f0268003ee0 2024-03-23T09:42:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.194+0000 7f0280f7e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f027c15dd80 0x7f027c0c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.194+0000 7f0280f7e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f027c15dd80 0x7f027c0c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59892/0 (socket says 172.21.15.47:59892) 2024-03-23T09:42:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.194+0000 7f0280f7e640 1 -- 172.21.15.47:0/2851621618 learned_addr learned my addr 172.21.15.47:0/2851621618 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.194+0000 7f0281f80640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f027c15b280 0x7f027c15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.194+0000 7f028177f640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0268003ee0 0x7f027c15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.195+0000 7f0280f7e640 1 -- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0268003ee0 msgr2=0x7f027c15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.195+0000 7f0280f7e640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0268003ee0 0x7f027c15fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.195+0000 7f0280f7e640 1 -- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f027c15b280 msgr2=0x7f027c15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.195+0000 7f0280f7e640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f027c15b280 0x7f027c15b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.195+0000 7f0280f7e640 1 -- 172.21.15.47:0/2851621618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c148950 con 0x7f027c15dd80 2024-03-23T09:42:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.195+0000 7f0280f7e640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f027c15dd80 0x7f027c0c7a30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f026c02c3b0 tx=0x7f026c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.195+0000 7f0271ffb640 1 -- 172.21.15.47:0/2851621618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c039070 con 0x7f027c15dd80 2024-03-23T09:42:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.195+0000 7f0271ffb640 1 -- 172.21.15.47:0/2851621618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c02cc20 con 0x7f027c15dd80 2024-03-23T09:42:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.195+0000 7f02839e3640 1 -- 172.21.15.47:0/2851621618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027c148c30 con 0x7f027c15dd80 2024-03-23T09:42:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.195+0000 7f0271ffb640 1 -- 172.21.15.47:0/2851621618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c03c040 con 0x7f027c15dd80 2024-03-23T09:42:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.195+0000 7f02839e3640 1 -- 172.21.15.47:0/2851621618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027c15e200 con 0x7f027c15dd80 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.196+0000 7f0271ffb640 1 -- 172.21.15.47:0/2851621618 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f026c032070 con 0x7f027c15dd80 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.197+0000 7f0271ffb640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f026006ac00 0x7f026006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.197+0000 7f0271ffb640 1 -- 172.21.15.47:0/2851621618 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f026c0aa480 con 0x7f027c15dd80 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.197+0000 7f028177f640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f026006ac00 0x7f026006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.198+0000 7f028177f640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f026006ac00 0x7f026006d0c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0278001910 tx=0x7f0278009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.198+0000 7f0286631740 1 Processor -- start 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.198+0000 7f0286631740 1 -- start start 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0286631740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe750 0x5585e6eb5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0286631740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6eb47d0 0x5585e6eb62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0286631740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb0c70 0x5585e6ee0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0286631740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e6e219b0 con 0x5585e6eb0c70 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0286631740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e6e1f140 con 0x5585e6efe750 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0286631740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585e6e1aa30 con 0x5585e6eb47d0 2024-03-23T09:42:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0281f80640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb0c70 0x5585e6ee0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0281f80640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb0c70 0x5585e6ee0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59908/0 (socket says 172.21.15.47:59908) 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0281f80640 1 -- 172.21.15.47:0/1439284438 learned_addr learned my addr 172.21.15.47:0/1439284438 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f028177f640 1 --2- 172.21.15.47:0/1439284438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe750 0x5585e6eb5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0280f7e640 1 --2- 172.21.15.47:0/1439284438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6eb47d0 0x5585e6eb62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0281f80640 1 -- 172.21.15.47:0/1439284438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6eb47d0 msgr2=0x5585e6eb62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0281f80640 1 --2- 172.21.15.47:0/1439284438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6eb47d0 0x5585e6eb62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0281f80640 1 -- 172.21.15.47:0/1439284438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe750 msgr2=0x5585e6eb5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0281f80640 1 --2- 172.21.15.47:0/1439284438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe750 0x5585e6eb5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0281f80640 1 -- 172.21.15.47:0/1439284438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585e6da72f0 con 0x5585e6eb0c70 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0280f7e640 1 --2- 172.21.15.47:0/1439284438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6eb47d0 0x5585e6eb62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0281f80640 1 --2- 172.21.15.47:0/1439284438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb0c70 0x5585e6ee0a60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f027c08ada0 tx=0x7f0274009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1787e8e059748c2f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0266ffd640 1 -- 172.21.15.47:0/1439284438 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027400e070 con 0x5585e6eb0c70 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0266ffd640 1 -- 172.21.15.47:0/1439284438 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027400f040 con 0x5585e6eb0c70 2024-03-23T09:42:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.199+0000 7f0266ffd640 1 -- 172.21.15.47:0/1439284438 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0274016900 con 0x5585e6eb0c70 2024-03-23T09:42:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.200+0000 7f0286631740 1 -- 172.21.15.47:0/1439284438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb0c70 msgr2=0x5585e6ee0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.200+0000 7f0286631740 1 --2- 172.21.15.47:0/1439284438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb0c70 0x5585e6ee0a60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f027c08ada0 tx=0x7f0274009750 comp rx=0 tx=0).stop 2024-03-23T09:42:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.200+0000 7f0286631740 1 -- 172.21.15.47:0/1439284438 shutdown_connections 2024-03-23T09:42:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.200+0000 7f0286631740 1 --2- 172.21.15.47:0/1439284438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb0c70 0x5585e6ee0a60 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.200+0000 7f0286631740 1 --2- 172.21.15.47:0/1439284438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6eb47d0 0x5585e6eb62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.200+0000 7f0286631740 1 --2- 172.21.15.47:0/1439284438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe750 0x5585e6eb5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.200+0000 7f0286631740 1 -- 172.21.15.47:0/1439284438 >> 172.21.15.47:0/1439284438 conn(0x5585e6eb2ee0 msgr2=0x5585e6efba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.200+0000 7f0286631740 1 -- 172.21.15.47:0/1439284438 shutdown_connections 2024-03-23T09:42:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.200+0000 7f0286631740 1 -- 172.21.15.47:0/1439284438 wait complete. 2024-03-23T09:42:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.201+0000 7f0286631740 1 -- start start 2024-03-23T09:42:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.201+0000 7f0286631740 1 Processor -- start 2024-03-23T09:42:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f0286631740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb47d0 0x5585e6f00e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f0286631740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6efe750 0x5585e6ee0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f0286631740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe140 0x5585e6efccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f0286631740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e6dd9090 con 0x5585e6eb47d0 2024-03-23T09:42:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f0286631740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e6dd8d30 con 0x5585e6efe140 2024-03-23T09:42:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f0286631740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585e6d9e2a0 con 0x5585e6efe750 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f0286631740 10 client.0 _ll_register_callbacks cb 0x5585e6dd4050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f028177f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb47d0 0x5585e6f00e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f028177f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb47d0 0x5585e6f00e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59918/0 (socket says 172.21.15.47:59918) 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f028177f640 1 -- 172.21.15.47:0/4234097718 learned_addr learned my addr 172.21.15.47:0/4234097718 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f0281f80640 1 --2- 172.21.15.47:0/4234097718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe140 0x5585e6efccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f0280f7e640 1 --2- 172.21.15.47:0/4234097718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6efe750 0x5585e6ee0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f028177f640 1 -- 172.21.15.47:0/4234097718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6efe750 msgr2=0x5585e6ee0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f028177f640 1 --2- 172.21.15.47:0/4234097718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6efe750 0x5585e6ee0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f028177f640 1 -- 172.21.15.47:0/4234097718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe140 msgr2=0x5585e6efccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f028177f640 1 --2- 172.21.15.47:0/4234097718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe140 0x5585e6efccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f028177f640 1 -- 172.21.15.47:0/4234097718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0274009dd0 con 0x5585e6eb47d0 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f028177f640 1 --2- 172.21.15.47:0/4234097718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb47d0 0x5585e6f00e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0278015a50 tx=0x7f0278007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5ab4dac8c82b3e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f025bfff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:42:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f025bfff640 1 -- 172.21.15.47:0/4234097718 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0278022780 con 0x5585e6eb47d0 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f025bfff640 1 -- 172.21.15.47:0/4234097718 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02780228e0 con 0x5585e6eb47d0 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.202+0000 7f0286631740 1 -- 172.21.15.47:0/4234097718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585e6b0e3c0 con 0x5585e6eb47d0 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.203+0000 7f025bfff640 1 -- 172.21.15.47:0/4234097718 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0278022aa0 con 0x5585e6eb47d0 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.203+0000 7f0286631740 10 client.5859 fetch_fsmap learned FSMap version 18 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.203+0000 7f0286631740 1 -- 172.21.15.47:0/4234097718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5585e6e190b0 con 0x5585e6eb47d0 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.203+0000 7f025bfff640 1 -- 172.21.15.47:0/4234097718 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5585e6b0e3c0 con 0x5585e6eb47d0 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.203+0000 7f025bfff640 1 -- 172.21.15.47:0/4234097718 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 18) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0278022c00 con 0x5585e6eb47d0 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.203+0000 7f0286631740 10 client.5859 fetch_fsmap finished waiting for FSMap version 18 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.203+0000 7f0286631740 10 client.5859 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.203+0000 7f0286631740 20 client.5859 populate_metadata read hostname 'smithi047' 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.203+0000 7f0286631740 1 --2- 172.21.15.47:0/4234097718 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x5585e6f01700 0x5585e6f01ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.203+0000 7f0286631740 4 client.5859 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.203+0000 7f0286631740 1 -- 172.21.15.47:0/4234097718 --> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585e6b1f6c0 con 0x5585e6f01700 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.203+0000 7f0280f7e640 1 --2- 172.21.15.47:0/4234097718 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x5585e6f01700 0x5585e6f01ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.204+0000 7f0280f7e640 1 --2- 172.21.15.47:0/4234097718 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x5585e6f01700 0x5585e6f01ae0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d88782946f012b05 server_cookie=f26b2c5a6ab0ec75 in_seq=0 out_seq=0 2024-03-23T09:42:57.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.204+0000 7f025bfff640 10 client.5859 ms_handle_connect on v2:172.21.15.175:6836/1626348200 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.204+0000 7f025bfff640 1 -- 172.21.15.47:0/4234097718 <== mds.0 v2:172.21.15.175:6836/1626348200 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5585e6b1f6c0 con 0x5585e6f01700 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.204+0000 7f025bfff640 10 client.5859 handle_command_reply: tid=0 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.204+0000 7f0286631740 1 client.5859 shutdown 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.205+0000 7f0286631740 1 -- 172.21.15.47:0/4234097718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb47d0 msgr2=0x5585e6f00e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.205+0000 7f0286631740 1 --2- 172.21.15.47:0/4234097718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb47d0 0x5585e6f00e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0278015a50 tx=0x7f0278007ca0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.205+0000 7f0286631740 1 -- 172.21.15.47:0/4234097718 shutdown_connections 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.205+0000 7f0286631740 1 --2- 172.21.15.47:0/4234097718 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x5585e6f01700 0x5585e6f01ae0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.205+0000 7f0286631740 1 --2- 172.21.15.47:0/4234097718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe140 0x5585e6efccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.205+0000 7f0286631740 1 --2- 172.21.15.47:0/4234097718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6efe750 0x5585e6ee0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.205+0000 7f025bfff640 0 client.5859 ms_handle_reset on v2:172.21.15.175:6836/1626348200 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.205+0000 7f0286631740 1 --2- 172.21.15.47:0/4234097718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6eb47d0 0x5585e6f00e20 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.205+0000 7f0286631740 1 -- 172.21.15.47:0/4234097718 >> 172.21.15.47:0/4234097718 conn(0x5585e6eb0c70 msgr2=0x5585e6eb4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.205+0000 7f0286631740 1 -- 172.21.15.47:0/4234097718 shutdown_connections 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.205+0000 7f0286631740 1 -- 172.21.15.47:0/4234097718 wait complete. 2024-03-23T09:42:57.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.205+0000 7f0286631740 20 client.5859 trim_cache size 0 max 16384 2024-03-23T09:42:57.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.220+0000 7f0286631740 1 Processor -- start 2024-03-23T09:42:57.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.220+0000 7f0286631740 1 -- start start 2024-03-23T09:42:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0286631740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6eb47d0 0x5585e6eabf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0286631740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 0x5585e6efc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0286631740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe750 0x5585e6f02e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0286631740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e6eab640 con 0x5585e6efe140 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0286631740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e6eb6280 con 0x5585e6efe750 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0286631740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585e6ee0770 con 0x5585e6eb47d0 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0280f7e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 0x5585e6efc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0280f7e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 0x5585e6efc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59926/0 (socket says 172.21.15.47:59926) 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0280f7e640 1 -- 172.21.15.47:0/907653863 learned_addr learned my addr 172.21.15.47:0/907653863 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f028177f640 1 --2- 172.21.15.47:0/907653863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6eb47d0 0x5585e6eabf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0281f80640 1 --2- 172.21.15.47:0/907653863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe750 0x5585e6f02e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0280f7e640 1 -- 172.21.15.47:0/907653863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6eb47d0 msgr2=0x5585e6eabf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0280f7e640 1 --2- 172.21.15.47:0/907653863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6eb47d0 0x5585e6eabf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0280f7e640 1 -- 172.21.15.47:0/907653863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe750 msgr2=0x5585e6f02e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0280f7e640 1 --2- 172.21.15.47:0/907653863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe750 0x5585e6f02e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0280f7e640 1 -- 172.21.15.47:0/907653863 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585e6dbb430 con 0x5585e6efe140 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f028177f640 1 --2- 172.21.15.47:0/907653863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6eb47d0 0x5585e6eabf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f0280f7e640 1 --2- 172.21.15.47:0/907653863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 0x5585e6efc330 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f026c078f30 tx=0x7f026c0aa040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1816fbf039913f62 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f02677fe640 1 -- 172.21.15.47:0/907653863 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c0775a0 con 0x5585e6efe140 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f02677fe640 1 -- 172.21.15.47:0/907653863 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c077800 con 0x5585e6efe140 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.221+0000 7f02677fe640 1 -- 172.21.15.47:0/907653863 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c075b70 con 0x5585e6efe140 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.222+0000 7f0286631740 1 -- 172.21.15.47:0/907653863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 msgr2=0x5585e6efc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.222+0000 7f0286631740 1 --2- 172.21.15.47:0/907653863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 0x5585e6efc330 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f026c078f30 tx=0x7f026c0aa040 comp rx=0 tx=0).stop 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.222+0000 7f0286631740 1 -- 172.21.15.47:0/907653863 shutdown_connections 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.222+0000 7f0286631740 1 --2- 172.21.15.47:0/907653863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6efe750 0x5585e6f02e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.222+0000 7f0286631740 1 --2- 172.21.15.47:0/907653863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 0x5585e6efc330 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.222+0000 7f0286631740 1 --2- 172.21.15.47:0/907653863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6eb47d0 0x5585e6eabf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.222+0000 7f0286631740 1 -- 172.21.15.47:0/907653863 >> 172.21.15.47:0/907653863 conn(0x5585e6eb0c70 msgr2=0x5585e6f00e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.222+0000 7f0286631740 1 -- 172.21.15.47:0/907653863 shutdown_connections 2024-03-23T09:42:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.222+0000 7f0286631740 1 -- 172.21.15.47:0/907653863 wait complete. 2024-03-23T09:42:57.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.222+0000 7f0286631740 1 -- start start 2024-03-23T09:42:57.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.223+0000 7f0286631740 1 Processor -- start 2024-03-23T09:42:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.223+0000 7f0286631740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6eb47d0 0x5585e6f01ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.223+0000 7f0286631740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 0x5585e6f5e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.223+0000 7f0286631740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6efe750 0x5585e6f14400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.223+0000 7f0286631740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e6ed70b0 con 0x5585e6efe140 2024-03-23T09:42:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.223+0000 7f0286631740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e6eddbb0 con 0x5585e6eb47d0 2024-03-23T09:42:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.223+0000 7f0286631740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585e6ed74d0 con 0x5585e6efe750 2024-03-23T09:42:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.223+0000 7f0286631740 10 client.0 _ll_register_callbacks cb 0x7f0274009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:42:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.223+0000 7f0280f7e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 0x5585e6f5e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f0280f7e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 0x5585e6f5e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59934/0 (socket says 172.21.15.47:59934) 2024-03-23T09:42:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f0280f7e640 1 -- 172.21.15.47:0/3802790596 learned_addr learned my addr 172.21.15.47:0/3802790596 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f028177f640 1 --2- 172.21.15.47:0/3802790596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6eb47d0 0x5585e6f01ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f0281f80640 1 --2- 172.21.15.47:0/3802790596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6efe750 0x5585e6f14400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f0280f7e640 1 -- 172.21.15.47:0/3802790596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6efe750 msgr2=0x5585e6f14400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f0280f7e640 1 --2- 172.21.15.47:0/3802790596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6efe750 0x5585e6f14400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f0280f7e640 1 -- 172.21.15.47:0/3802790596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6eb47d0 msgr2=0x5585e6f01ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f0280f7e640 1 --2- 172.21.15.47:0/3802790596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6eb47d0 0x5585e6f01ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f0280f7e640 1 -- 172.21.15.47:0/3802790596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027c15e200 con 0x5585e6efe140 2024-03-23T09:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f0280f7e640 1 --2- 172.21.15.47:0/3802790596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 0x5585e6f5e790 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f026c0793f0 tx=0x7f0278015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a98fe5f7a66d03a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f025bfff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f025bfff640 1 -- 172.21.15.47:0/3802790596 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c0aa0c0 con 0x5585e6efe140 2024-03-23T09:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f025bfff640 1 -- 172.21.15.47:0/3802790596 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f026c0aa220 con 0x5585e6efe140 2024-03-23T09:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f0286631740 1 -- 172.21.15.47:0/3802790596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f027400f040 con 0x5585e6efe140 2024-03-23T09:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f025bfff640 1 -- 172.21.15.47:0/3802790596 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f026c0796c0 con 0x5585e6efe140 2024-03-23T09:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f0286631740 10 client.5865 fetch_fsmap learned FSMap version 18 2024-03-23T09:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f0286631740 1 -- 172.21.15.47:0/3802790596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5585e6de0b40 con 0x5585e6efe140 2024-03-23T09:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.224+0000 7f025bfff640 1 -- 172.21.15.47:0/3802790596 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027400f040 con 0x5585e6efe140 2024-03-23T09:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.225+0000 7f025bfff640 1 -- 172.21.15.47:0/3802790596 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 18) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f026c097070 con 0x5585e6efe140 2024-03-23T09:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.225+0000 7f0286631740 10 client.5865 fetch_fsmap finished waiting for FSMap version 18 2024-03-23T09:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.225+0000 7f0286631740 10 client.5865 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.225+0000 7f0286631740 20 client.5865 populate_metadata read hostname 'smithi047' 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.225+0000 7f0286631740 1 --2- 172.21.15.47:0/3802790596 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x5585e6f14940 0x5585e6eb58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.225+0000 7f0286631740 4 client.5865 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.225+0000 7f0286631740 1 -- 172.21.15.47:0/3802790596 --> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5585e6b72d10 con 0x5585e6f14940 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.225+0000 7f028177f640 1 --2- 172.21.15.47:0/3802790596 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x5585e6f14940 0x5585e6eb58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.225+0000 7f025bfff640 10 client.5865 ms_handle_connect on v2:172.21.15.175:6836/1626348200 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.225+0000 7f028177f640 1 --2- 172.21.15.47:0/3802790596 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x5585e6f14940 0x5585e6eb58f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b822c69485ae2e7e server_cookie=b6ac40e0f300ac9b in_seq=0 out_seq=0 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.225+0000 7f025bfff640 1 -- 172.21.15.47:0/3802790596 <== mds.0 v2:172.21.15.175:6836/1626348200 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5585e6b72d10 con 0x5585e6f14940 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.225+0000 7f025bfff640 10 client.5865 handle_command_reply: tid=0 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.225+0000 7f0286631740 1 client.5865 shutdown 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f0286631740 1 -- 172.21.15.47:0/3802790596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 msgr2=0x5585e6f5e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f0286631740 1 --2- 172.21.15.47:0/3802790596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 0x5585e6f5e790 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f026c0793f0 tx=0x7f0278015000 comp rx=0 tx=0).stop 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f0286631740 1 -- 172.21.15.47:0/3802790596 shutdown_connections 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f0286631740 1 --2- 172.21.15.47:0/3802790596 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x5585e6f14940 0x5585e6eb58f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f0286631740 1 --2- 172.21.15.47:0/3802790596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5585e6efe750 0x5585e6f14400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f0286631740 1 --2- 172.21.15.47:0/3802790596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e6efe140 0x5585e6f5e790 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f0286631740 1 --2- 172.21.15.47:0/3802790596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585e6eb47d0 0x5585e6f01ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f025bfff640 0 client.5865 ms_handle_reset on v2:172.21.15.175:6836/1626348200 2024-03-23T09:42:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f0286631740 1 -- 172.21.15.47:0/3802790596 >> 172.21.15.47:0/3802790596 conn(0x5585e6eb0c70 msgr2=0x5585e6f00c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f0286631740 1 -- 172.21.15.47:0/3802790596 shutdown_connections 2024-03-23T09:42:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f0286631740 1 -- 172.21.15.47:0/3802790596 wait complete. 2024-03-23T09:42:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f0286631740 20 client.5865 trim_cache size 0 max 16384 2024-03-23T09:42:57.234 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:42:57.234 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:42:57.235 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:42:57.235 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 5697, 2024-03-23T09:42:57.235 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:42:57.235 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:42:57.235 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:42:57.235 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 9.0599337420000001, 2024-03-23T09:42:57.235 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 18, 2024-03-23T09:42:57.235 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 46, 2024-03-23T09:42:57.235 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 46, 2024-03-23T09:42:57.235 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 13.839607652 2024-03-23T09:42:57.235 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:42:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f02839e3640 1 -- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f026006ac00 msgr2=0x7f026006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f02839e3640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f026006ac00 0x7f026006d0c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0278001910 tx=0x7f0278009040 comp rx=0 tx=0).stop 2024-03-23T09:42:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f02839e3640 1 -- 172.21.15.47:0/2851621618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f027c15dd80 msgr2=0x7f027c0c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f02839e3640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f027c15dd80 0x7f027c0c7a30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f026c02c3b0 tx=0x7f026c001a00 comp rx=0 tx=0).stop 2024-03-23T09:42:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f02839e3640 1 -- 172.21.15.47:0/2851621618 shutdown_connections 2024-03-23T09:42:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f02839e3640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f026006ac00 0x7f026006d0c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f02839e3640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f027c15b280 0x7f027c15b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f02839e3640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f027c15dd80 0x7f027c0c7a30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f02839e3640 1 --2- 172.21.15.47:0/2851621618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0268003ee0 0x7f027c15fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f02839e3640 1 -- 172.21.15.47:0/2851621618 >> 172.21.15.47:0/2851621618 conn(0x7f027c06b160 msgr2=0x7f027c0b6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f02839e3640 1 -- 172.21.15.47:0/2851621618 shutdown_connections 2024-03-23T09:42:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.226+0000 7f02839e3640 1 -- 172.21.15.47:0/2851621618 wait complete. 2024-03-23T09:42:57.238 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:42:57.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.338+0000 7f0bbc436640 1 Processor -- start 2024-03-23T09:42:57.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.338+0000 7f0bbc436640 1 -- start start 2024-03-23T09:42:57.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.339+0000 7f0bbc436640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0bb40b10f0 0x7f0bb40affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.339+0000 7f0bbc436640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40b0520 0x7f0bb4098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.339+0000 7f0bbc436640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb40ce8f0 con 0x7f0bb40cbdf0 2024-03-23T09:42:57.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.339+0000 7f0bbc436640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb40cc300 con 0x7f0bb40b10f0 2024-03-23T09:42:57.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.339+0000 7f0bbc436640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb40cc5c0 con 0x7f0bb40b0520 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.339+0000 7f0bba9d3640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0bb40cbdf0 0x7f0bb40ced80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52042/0 (socket says 172.21.15.47:52042) 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.339+0000 7f0bba9d3640 1 -- 172.21.15.47:0/289999630 learned_addr learned my addr 172.21.15.47:0/289999630 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bba1d2640 1 --2- 172.21.15.47:0/289999630 >> v2:172.21.15.175:3300/0 conn(0x7f0bb40b10f0 0x7f0bb40affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb99d1640 1 --2- 172.21.15.47:0/289999630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40b0520 0x7f0bb4098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333044498 0 0) 0x7f0bb40ce8f0 con 0x7f0bb40cbdf0 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b9c003520 con 0x7f0bb40cbdf0 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 42562187 0 0) 0x7f0b9c003520 con 0x7f0bb40cbdf0 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40b0520 msgr2=0x7f0bb4098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 --2- 172.21.15.47:0/289999630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40b0520 0x7f0bb4098a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 >> v2:172.21.15.175:3300/0 conn(0x7f0bb40b10f0 msgr2=0x7f0bb40affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 --2- 172.21.15.47:0/289999630 >> v2:172.21.15.175:3300/0 conn(0x7f0bb40b10f0 0x7f0bb40affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb40ce2a0 con 0x7f0bb40cbdf0 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0ba4001670 con 0x7f0bb40cbdf0 2024-03-23T09:42:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 >> v1:172.21.15.47:6789/0 conn(0x7f0bb40cbdf0 legacy=0x7f0bb40ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 --2- 172.21.15.47:0/289999630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b9c003970 0x7f0b9c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 --2- 172.21.15.47:0/289999630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b9c0047c0 0x7f0b9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 --2- 172.21.15.47:0/289999630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40b0520 0x7f0b9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb40cc300 con 0x7f0b9c0047c0 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb40cc5c0 con 0x7f0b9c003970 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c00b9b0 con 0x7f0bb40b0520 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bba1d2640 1 --2- 172.21.15.47:0/289999630 >> v2:172.21.15.175:3300/0 conn(0x7f0bb40b10f0 0x7f0bb40affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb99d1640 1 --2- 172.21.15.47:0/289999630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b9c0047c0 0x7f0b9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bba9d3640 1 --2- 172.21.15.47:0/289999630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40b0520 0x7f0b9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bba1d2640 1 --2- 172.21.15.47:0/289999630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b9c003970 0x7f0b9c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb99d1640 1 -- 172.21.15.47:0/289999630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40b0520 msgr2=0x7f0b9c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb99d1640 1 --2- 172.21.15.47:0/289999630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40b0520 0x7f0b9c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb99d1640 1 -- 172.21.15.47:0/289999630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b9c003970 msgr2=0x7f0b9c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb99d1640 1 --2- 172.21.15.47:0/289999630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b9c003970 0x7f0b9c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.340+0000 7f0bb99d1640 1 -- 172.21.15.47:0/289999630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c003520 con 0x7f0b9c0047c0 2024-03-23T09:42:57.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.341+0000 7f0bb99d1640 1 --2- 172.21.15.47:0/289999630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b9c0047c0 0x7f0b9c004c40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009650 tx=0x7f0b9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ca82a763694826 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:57.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.341+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba803b400 con 0x7f0b9c0047c0 2024-03-23T09:42:57.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.341+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba8037050 con 0x7f0b9c0047c0 2024-03-23T09:42:57.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.341+0000 7f0bb91d0640 1 -- 172.21.15.47:0/289999630 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba803b780 con 0x7f0b9c0047c0 2024-03-23T09:42:57.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.342+0000 7f0bbc436640 1 -- 172.21.15.47:0/289999630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b9c0047c0 msgr2=0x7f0b9c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.342+0000 7f0bbc436640 1 --2- 172.21.15.47:0/289999630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b9c0047c0 0x7f0b9c004c40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009650 tx=0x7f0b9c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.342+0000 7f0bba9d3640 1 -- 172.21.15.47:0/289999630 reap_dead start 2024-03-23T09:42:57.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.342+0000 7f0bbc436640 1 -- 172.21.15.47:0/289999630 shutdown_connections 2024-03-23T09:42:57.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.342+0000 7f0bbc436640 1 -- 172.21.15.47:0/289999630 >> 172.21.15.47:0/289999630 conn(0x7f0bb40b21c0 msgr2=0x7f0bb40b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:57.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.342+0000 7f0bbc436640 1 -- 172.21.15.47:0/289999630 shutdown_connections 2024-03-23T09:42:57.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.342+0000 7f0bbc436640 1 -- 172.21.15.47:0/289999630 wait complete. 2024-03-23T09:42:57.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.343+0000 7f0bbc436640 1 Processor -- start 2024-03-23T09:42:57.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.343+0000 7f0bbc436640 1 -- start start 2024-03-23T09:42:57.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bbc436640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b9c0047c0 0x7f0bb409a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bbc436640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40ca990 0x7f0bb41de2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bbc436640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bb41de820 0x7f0bb40b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bbc436640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb409d0a0 con 0x7f0b9c0047c0 2024-03-23T09:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bbc436640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb41deca0 con 0x7f0bb41de820 2024-03-23T09:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bbc436640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb409d2f0 con 0x7f0bb40ca990 2024-03-23T09:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bba1d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b9c0047c0 0x7f0bb409a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bba1d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b9c0047c0 0x7f0bb409a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59954/0 (socket says 172.21.15.47:59954) 2024-03-23T09:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bba1d2640 1 -- 172.21.15.47:0/28830582 learned_addr learned my addr 172.21.15.47:0/28830582 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bba9d3640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bb41de820 0x7f0bb40b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bb99d1640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40ca990 0x7f0bb41de2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bba1d2640 1 -- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40ca990 msgr2=0x7f0bb41de2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bba1d2640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40ca990 0x7f0bb41de2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bba1d2640 1 -- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bb41de820 msgr2=0x7f0bb40b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bba1d2640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bb41de820 0x7f0bb40b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bba1d2640 1 -- 172.21.15.47:0/28830582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb40b1650 con 0x7f0b9c0047c0 2024-03-23T09:42:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bba9d3640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bb41de820 0x7f0bb40b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bb99d1640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40ca990 0x7f0bb41de2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:42:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.344+0000 7f0bba1d2640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b9c0047c0 0x7f0bb409a810 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0bac010410 tx=0x7f0bac0108d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.345+0000 7f0bb2ffd640 1 -- 172.21.15.47:0/28830582 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac0166f0 con 0x7f0b9c0047c0 2024-03-23T09:42:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.345+0000 7f0bb2ffd640 1 -- 172.21.15.47:0/28830582 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bac016850 con 0x7f0b9c0047c0 2024-03-23T09:42:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.345+0000 7f0bb2ffd640 1 -- 172.21.15.47:0/28830582 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac016bc0 con 0x7f0b9c0047c0 2024-03-23T09:42:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.345+0000 7f0bbc436640 1 -- 172.21.15.47:0/28830582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb409d5d0 con 0x7f0b9c0047c0 2024-03-23T09:42:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.345+0000 7f0bbc436640 1 -- 172.21.15.47:0/28830582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb409db90 con 0x7f0b9c0047c0 2024-03-23T09:42:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.346+0000 7f0bbc436640 1 -- 172.21.15.47:0/28830582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb4066be0 con 0x7f0b9c0047c0 2024-03-23T09:42:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.346+0000 7f0bb2ffd640 1 -- 172.21.15.47:0/28830582 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0bac016d20 con 0x7f0b9c0047c0 2024-03-23T09:42:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.347+0000 7f0bb2ffd640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ba006ac00 0x7f0ba006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:57.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.347+0000 7f0bb2ffd640 1 -- 172.21.15.47:0/28830582 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bac089c30 con 0x7f0b9c0047c0 2024-03-23T09:42:57.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.347+0000 7f0bb99d1640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ba006ac00 0x7f0ba006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:57.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.347+0000 7f0bb99d1640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ba006ac00 0x7f0ba006d0c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0ba803d000 tx=0x7f0ba8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:57.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.351+0000 7f0bb2ffd640 1 -- 172.21.15.47:0/28830582 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0bac054ea0 con 0x7f0b9c0047c0 2024-03-23T09:42:57.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.570+0000 7f0bbc436640 1 -- 172.21.15.47:0/28830582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0bb40b6650 con 0x7f0b9c0047c0 2024-03-23T09:42:57.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.570+0000 7f0bb2ffd640 1 -- 172.21.15.47:0/28830582 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0bac01d070 con 0x7f0b9c0047c0 2024-03-23T09:42:57.571 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:42:57.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.572+0000 7f0bbc436640 1 -- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ba006ac00 msgr2=0x7f0ba006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.572+0000 7f0bbc436640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ba006ac00 0x7f0ba006d0c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0ba803d000 tx=0x7f0ba8008030 comp rx=0 tx=0).stop 2024-03-23T09:42:57.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.572+0000 7f0bbc436640 1 -- 172.21.15.47:0/28830582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b9c0047c0 msgr2=0x7f0bb409a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:57.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.572+0000 7f0bbc436640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b9c0047c0 0x7f0bb409a810 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0bac010410 tx=0x7f0bac0108d0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.572+0000 7f0bbc436640 1 -- 172.21.15.47:0/28830582 shutdown_connections 2024-03-23T09:42:57.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.572+0000 7f0bbc436640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ba006ac00 0x7f0ba006d0c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.572+0000 7f0bbc436640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bb41de820 0x7f0bb40b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.572+0000 7f0bbc436640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb40ca990 0x7f0bb41de2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.572+0000 7f0bbc436640 1 --2- 172.21.15.47:0/28830582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b9c0047c0 0x7f0bb409a810 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:57.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.572+0000 7f0bbc436640 1 -- 172.21.15.47:0/28830582 >> 172.21.15.47:0/28830582 conn(0x7f0bb40b21c0 msgr2=0x7f0bb40cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:57.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.572+0000 7f0bbc436640 1 -- 172.21.15.47:0/28830582 shutdown_connections 2024-03-23T09:42:57.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:57.572+0000 7f0bbc436640 1 -- 172.21.15.47:0/28830582 wait complete. 2024-03-23T09:42:57.584 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:42:57.584 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:42:57.584 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:42:57.584 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:42:57.584 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:42:57.584 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:42:57.584 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:42:57.584 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:42:57.584 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:42:57.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:57.585 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:42:57.599 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:42:57.600 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:42:57.601 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:42:57.601 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42561sec preferred_lft 42561sec 2024-03-23T09:42:57.601 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:42:57.601 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:57.601 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:42:57.601 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:42:57.601 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:42:57.601 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:42:57.601 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:42:57.601 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:42:57.601 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:57.601 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:42:57.601 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:57.602 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:42:57.602 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:42:57.602 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:42:57.602 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:57.602 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:42:57.602 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:42:57.602 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:42:57.602 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:42:57.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:57.602 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:42:57.665 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:42:57.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:57.666 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:42:57.721 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:42:57.722 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:42:57.722 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:42:57.722 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:42:57.722 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:42:57.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:57.723 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:42:57.723 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:42:57.723 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:42:57.723 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:42:57.723 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:42:57.723 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:42:57.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:57.816 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:42:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:42:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:42:57.834 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:42:57.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:57.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:42:57.900 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:42:57.900 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:42:57.900 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:42:57.955 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:42:57.955 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:42:57.955 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:42:58.009 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:42:58.076 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:42:58.076 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:42:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:42:58.143 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:42:58.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:58.143 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:42:58.198 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:42:58.198 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:42:58.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:42:58.269 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:42:58.269 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:42:58.328 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:42:58.327+0000 7f0e33c0f480 -1 init, newargv = 0x55d07a45f200 newargc=15 2024-03-23T09:42:58.328 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[54700]: starting ceph client 2024-03-23T09:42:58.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[54700]: starting fuse 2024-03-23T09:42:58.338 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:42:58.339 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:42:58.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:58.339 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:42:58.393 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:42:58.394 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:42:58.394 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:42:58.394 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:42:58.394 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:42:58.394 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:42:58.394 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:42:58.394 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:58.394 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:42:58.394 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:58.395 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:42:58.396 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:42:58.509 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.54700.asok 2024-03-23T09:42:58.695 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54700.asok 2024-03-23T09:42:58.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:58.696 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54700.asok status 2024-03-23T09:42:58.832 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:42:58.832 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "54700", 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 5877, 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:42:58.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5877 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1903486306 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1903486306 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.5877 192.168.0.1:0/1903486306", 2024-03-23T09:42:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/1903486306", 2024-03-23T09:42:58.835 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:42:58.835 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 18, 2024-03-23T09:42:58.835 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 46, 2024-03-23T09:42:58.835 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:42:58.835 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:42:58.835 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:42:58.835 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:42:58.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:58.841 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:42:58.841 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:42:58.857 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:42:58.857 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:42:58.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:58.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:42:58.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:58.925 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-03-23T09:42:58.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:42:58.986 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-03-23T09:42:59.044 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:42:59.044 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:42:59.044 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:42:59.044 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:42:59.044 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-03-23T09:42:59.044 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:42:59.146 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627777 2024-03-23T09:42:59.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:42:59.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.475+0000 7efef9a8b640 1 Processor -- start 2024-03-23T09:42:59.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.475+0000 7efef9a8b640 1 -- start start 2024-03-23T09:42:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef9a8b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef415cb40 0x7efef4160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef9a8b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efef4160cb0 0x7efef40d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef9a8b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef40d0ee0 con 0x7efef40b8300 2024-03-23T09:42:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef9a8b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef40b5a10 con 0x7efef4160cb0 2024-03-23T09:42:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef9a8b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef41616f0 con 0x7efef415cb40 2024-03-23T09:42:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef2ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efef40b8300 0x7efef40cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52052/0 (socket says 172.21.15.47:52052) 2024-03-23T09:42:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef2ffd640 1 -- 172.21.15.47:0/3638113761 learned_addr learned my addr 172.21.15.47:0/3638113761 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:42:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef3fff640 1 --2- 172.21.15.47:0/3638113761 >> v2:172.21.15.175:3300/0 conn(0x7efef4160cb0 0x7efef40d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef37fe640 1 --2- 172.21.15.47:0/3638113761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef415cb40 0x7efef4160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326257852 0 0) 0x7efef40d0ee0 con 0x7efef40b8300 2024-03-23T09:42:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efedc003520 con 0x7efef40b8300 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3208788983 0 0) 0x7efedc003520 con 0x7efef40b8300 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef415cb40 msgr2=0x7efef4160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 --2- 172.21.15.47:0/3638113761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef415cb40 0x7efef4160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 >> v2:172.21.15.175:3300/0 conn(0x7efef4160cb0 msgr2=0x7efef40d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 --2- 172.21.15.47:0/3638113761 >> v2:172.21.15.175:3300/0 conn(0x7efef4160cb0 0x7efef40d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef4161130 con 0x7efef40b8300 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7efee8001670 con 0x7efef40b8300 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 >> v1:172.21.15.47:6789/0 conn(0x7efef40b8300 legacy=0x7efef40cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 --2- 172.21.15.47:0/3638113761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef415cb40 0x7efedc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 --2- 172.21.15.47:0/3638113761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efedc003ef0 0x7efedc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 --2- 172.21.15.47:0/3638113761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc004d00 0x7efedc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef40b5a10 con 0x7efedc003ef0 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef41616f0 con 0x7efedc004d00 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc00b980 con 0x7efef415cb40 2024-03-23T09:42:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.476+0000 7efef2ffd640 1 --2- 172.21.15.47:0/3638113761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efedc003ef0 0x7efedc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.477+0000 7efef3fff640 1 --2- 172.21.15.47:0/3638113761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc004d00 0x7efedc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.477+0000 7efef37fe640 1 --2- 172.21.15.47:0/3638113761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef415cb40 0x7efedc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.477+0000 7efef2ffd640 1 -- 172.21.15.47:0/3638113761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef415cb40 msgr2=0x7efedc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.477+0000 7efef2ffd640 1 --2- 172.21.15.47:0/3638113761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef415cb40 0x7efedc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.477+0000 7efef2ffd640 1 -- 172.21.15.47:0/3638113761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc004d00 msgr2=0x7efedc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.477+0000 7efef2ffd640 1 --2- 172.21.15.47:0/3638113761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc004d00 0x7efedc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.477+0000 7efef2ffd640 1 -- 172.21.15.47:0/3638113761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc003520 con 0x7efedc003ef0 2024-03-23T09:42:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.477+0000 7efef2ffd640 1 --2- 172.21.15.47:0/3638113761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efedc003ef0 0x7efedc0047c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7efef40b57e0 tx=0x7efedc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4474a02a77ad589a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.478+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8005ad0 con 0x7efedc003ef0 2024-03-23T09:42:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.478+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee8005c30 con 0x7efedc003ef0 2024-03-23T09:42:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.478+0000 7efef27fc640 1 -- 172.21.15.47:0/3638113761 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee802c880 con 0x7efedc003ef0 2024-03-23T09:42:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.478+0000 7efef9a8b640 1 -- 172.21.15.47:0/3638113761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efedc003ef0 msgr2=0x7efedc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.478+0000 7efef9a8b640 1 --2- 172.21.15.47:0/3638113761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efedc003ef0 0x7efedc0047c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7efef40b57e0 tx=0x7efedc0057f0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.478+0000 7efef3fff640 1 -- 172.21.15.47:0/3638113761 reap_dead start 2024-03-23T09:42:59.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.478+0000 7efef9a8b640 1 -- 172.21.15.47:0/3638113761 shutdown_connections 2024-03-23T09:42:59.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.478+0000 7efef9a8b640 1 -- 172.21.15.47:0/3638113761 >> 172.21.15.47:0/3638113761 conn(0x7efef406b160 msgr2=0x7efef406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:59.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.478+0000 7efef9a8b640 1 -- 172.21.15.47:0/3638113761 shutdown_connections 2024-03-23T09:42:59.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.478+0000 7efef9a8b640 1 -- 172.21.15.47:0/3638113761 wait complete. 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.479+0000 7efef9a8b640 1 Processor -- start 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.480+0000 7efef9a8b640 1 -- start start 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.480+0000 7efef9a8b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc003ef0 0x7efef415f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.480+0000 7efef9a8b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef40caa00 0x7efef40d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.480+0000 7efef9a8b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef40cae80 0x7efef40d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.480+0000 7efef9a8b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef40cb9a0 con 0x7efef40caa00 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.480+0000 7efef9a8b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef415f440 con 0x7efedc003ef0 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.480+0000 7efef9a8b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef40cb3e0 con 0x7efef40cae80 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.480+0000 7efef2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef40caa00 0x7efef40d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.480+0000 7efef2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef40caa00 0x7efef40d1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59968/0 (socket says 172.21.15.47:59968) 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.480+0000 7efef2ffd640 1 -- 172.21.15.47:0/1063702875 learned_addr learned my addr 172.21.15.47:0/1063702875 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.481+0000 7efef37fe640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc003ef0 0x7efef415f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.481+0000 7efef3fff640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef40cae80 0x7efef40d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.481+0000 7efef2ffd640 1 -- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef40cae80 msgr2=0x7efef40d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.481+0000 7efef2ffd640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef40cae80 0x7efef40d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.481+0000 7efef2ffd640 1 -- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc003ef0 msgr2=0x7efef415f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.481+0000 7efef2ffd640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc003ef0 0x7efef415f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.481+0000 7efef2ffd640 1 -- 172.21.15.47:0/1063702875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef40cb6c0 con 0x7efef40caa00 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.481+0000 7efef2ffd640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef40caa00 0x7efef40d1a70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efee8000f60 tx=0x7efee8000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.481+0000 7efedbfff640 1 -- 172.21.15.47:0/1063702875 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee802dd20 con 0x7efef40caa00 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.481+0000 7efedbfff640 1 -- 172.21.15.47:0/1063702875 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee8007070 con 0x7efef40caa00 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.481+0000 7efef9a8b640 1 -- 172.21.15.47:0/1063702875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef415f0d0 con 0x7efef40caa00 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.481+0000 7efedbfff640 1 -- 172.21.15.47:0/1063702875 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8007420 con 0x7efef40caa00 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.481+0000 7efef9a8b640 1 -- 172.21.15.47:0/1063702875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef41f2be0 con 0x7efef40caa00 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.482+0000 7efef9a8b640 1 -- 172.21.15.47:0/1063702875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef4072360 con 0x7efef40caa00 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.483+0000 7efedbfff640 1 -- 172.21.15.47:0/1063702875 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efee8032070 con 0x7efef40caa00 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.483+0000 7efedbfff640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efee006ac00 0x7efee006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.483+0000 7efedbfff640 1 -- 172.21.15.47:0/1063702875 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efee803c040 con 0x7efef40caa00 2024-03-23T09:42:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.483+0000 7efef37fe640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efee006ac00 0x7efee006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.484+0000 7efef37fe640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efee006ac00 0x7efee006d0c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efef40b8240 tx=0x7efee4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.487+0000 7efedbfff640 1 -- 172.21.15.47:0/1063702875 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efee8076100 con 0x7efef40caa00 2024-03-23T09:42:59.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.705+0000 7efef9a8b640 1 -- 172.21.15.47:0/1063702875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efef415ef30 con 0x7efef40caa00 2024-03-23T09:42:59.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.706+0000 7efedbfff640 1 -- 172.21.15.47:0/1063702875 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4740 (secure 0 0 0) 0x7efef415ef30 con 0x7efef40caa00 2024-03-23T09:42:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:42:59.707 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1189831856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1189831856},{"type":"v1","addr":"172.21.15.47:6835","nonce":1189831856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/2248064454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2248064454},{"type":"v1","addr":"172.21.15.47:6834","nonce":2248064454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5694,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3806465612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3806465612},{"type":"v1","addr":"172.21.15.175:6835","nonce":3806465612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:42:48.165819+0000","modified":"2024-03-23T09:42:50.171529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1626348200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1626348200},{"type":"v1","addr":"172.21.15.175:6837","nonce":1626348200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-23T09:42:59.707 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 18 2024-03-23T09:42:59.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.708+0000 7efef9a8b640 1 -- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efee006ac00 msgr2=0x7efee006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:59.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.708+0000 7efef9a8b640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efee006ac00 0x7efee006d0c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efef40b8240 tx=0x7efee4008040 comp rx=0 tx=0).stop 2024-03-23T09:42:59.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.708+0000 7efef9a8b640 1 -- 172.21.15.47:0/1063702875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef40caa00 msgr2=0x7efef40d1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:59.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.708+0000 7efef9a8b640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef40caa00 0x7efef40d1a70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efee8000f60 tx=0x7efee8000fe0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.708+0000 7efef9a8b640 1 -- 172.21.15.47:0/1063702875 shutdown_connections 2024-03-23T09:42:59.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.708+0000 7efef9a8b640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efee006ac00 0x7efee006d0c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.708+0000 7efef9a8b640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef40cae80 0x7efef40d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.708+0000 7efef9a8b640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef40caa00 0x7efef40d1a70 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.708+0000 7efef9a8b640 1 --2- 172.21.15.47:0/1063702875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc003ef0 0x7efef415f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.708+0000 7efef9a8b640 1 -- 172.21.15.47:0/1063702875 >> 172.21.15.47:0/1063702875 conn(0x7efef406b160 msgr2=0x7efef415bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:59.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.708+0000 7efef9a8b640 1 -- 172.21.15.47:0/1063702875 shutdown_connections 2024-03-23T09:42:59.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.708+0000 7efef9a8b640 1 -- 172.21.15.47:0/1063702875 wait complete. 2024-03-23T09:42:59.719 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-03-23T09:42:59.853 INFO:teuthology.orchestra.run.smithi175.stdout:{"message":"","return_code":0} 2024-03-23T09:42:59.854 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-23T09:42:59.854 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-23T09:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.894+0000 7fef1f20cf40 1 Processor -- start 2024-03-23T09:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.894+0000 7fef1f20cf40 1 -- start start 2024-03-23T09:42:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.895+0000 7fef1f20cf40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e610022900 0x55e61001de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.895+0000 7fef1f20cf40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e61003af70 0x55e610021f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.895+0000 7fef1f20cf40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6100c7ad0 0x55e6100bd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.895+0000 7fef1f20cf40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60ff3ab60 con 0x55e6100c7ad0 2024-03-23T09:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.895+0000 7fef1f20cf40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60ff3a800 con 0x55e610022900 2024-03-23T09:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.895+0000 7fef1f20cf40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e60ff1d520 con 0x55e61003af70 2024-03-23T09:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.895+0000 7fef1c90a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6100c7ad0 0x55e6100bd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.895+0000 7fef1c90a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6100c7ad0 0x55e6100bd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60000/0 (socket says 172.21.15.47:60000) 2024-03-23T09:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.895+0000 7fef1c90a640 1 -- 172.21.15.47:0/481385378 learned_addr learned my addr 172.21.15.47:0/481385378 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.895+0000 7fef1c109640 1 --2- 172.21.15.47:0/481385378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e610022900 0x55e61001de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.895+0000 7fef13fff640 1 --2- 172.21.15.47:0/481385378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e61003af70 0x55e610021f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1c90a640 1 -- 172.21.15.47:0/481385378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e61003af70 msgr2=0x55e610021f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1c90a640 1 --2- 172.21.15.47:0/481385378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e61003af70 0x55e610021f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1c90a640 1 -- 172.21.15.47:0/481385378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e610022900 msgr2=0x55e61001de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1c90a640 1 --2- 172.21.15.47:0/481385378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e610022900 0x55e61001de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1c90a640 1 -- 172.21.15.47:0/481385378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1400a1c0 con 0x55e6100c7ad0 2024-03-23T09:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1c109640 1 --2- 172.21.15.47:0/481385378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e610022900 0x55e61001de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1c90a640 1 --2- 172.21.15.47:0/481385378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6100c7ad0 0x55e6100bd840 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fef140472b0 tx=0x7fef1404f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d99f472d6affd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1b107640 1 -- 172.21.15.47:0/481385378 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1405f410 con 0x55e6100c7ad0 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1b107640 1 -- 172.21.15.47:0/481385378 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1405f570 con 0x55e6100c7ad0 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1b107640 1 -- 172.21.15.47:0/481385378 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1405f8e0 con 0x55e6100c7ad0 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1f20cf40 1 -- 172.21.15.47:0/481385378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6100c7ad0 msgr2=0x55e6100bd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1f20cf40 1 --2- 172.21.15.47:0/481385378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6100c7ad0 0x55e6100bd840 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fef140472b0 tx=0x7fef1404f000 comp rx=0 tx=0).stop 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1f20cf40 1 -- 172.21.15.47:0/481385378 shutdown_connections 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1f20cf40 1 --2- 172.21.15.47:0/481385378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6100c7ad0 0x55e6100bd840 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1f20cf40 1 --2- 172.21.15.47:0/481385378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e61003af70 0x55e610021f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1f20cf40 1 --2- 172.21.15.47:0/481385378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e610022900 0x55e61001de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1f20cf40 1 -- 172.21.15.47:0/481385378 >> 172.21.15.47:0/481385378 conn(0x55e61003e030 msgr2=0x55e6100c6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1f20cf40 1 -- 172.21.15.47:0/481385378 shutdown_connections 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.896+0000 7fef1f20cf40 1 -- 172.21.15.47:0/481385378 wait complete. 2024-03-23T09:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.897+0000 7fef1f20cf40 1 Processor -- start 2024-03-23T09:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.897+0000 7fef1f20cf40 1 -- start start 2024-03-23T09:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1f20cf40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e610022900 0x55e6100c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1f20cf40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e61003af70 0x55e6100c0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1f20cf40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6100c7ad0 0x55e6100c2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1f20cf40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61001e050 con 0x55e6100c7ad0 2024-03-23T09:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1f20cf40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6100c8550 con 0x55e610022900 2024-03-23T09:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1f20cf40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e61001d7e0 con 0x55e61003af70 2024-03-23T09:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1c90a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6100c7ad0 0x55e6100c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1c90a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6100c7ad0 0x55e6100c2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60002/0 (socket says 172.21.15.47:60002) 2024-03-23T09:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1c90a640 1 -- 172.21.15.47:0/3240405721 learned_addr learned my addr 172.21.15.47:0/3240405721 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1c109640 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e610022900 0x55e6100c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef13fff640 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e61003af70 0x55e6100c0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1c90a640 1 -- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e61003af70 msgr2=0x55e6100c0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1c90a640 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e61003af70 0x55e6100c0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1c90a640 1 -- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e610022900 msgr2=0x55e6100c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1c90a640 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e610022900 0x55e6100c08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1c90a640 1 -- 172.21.15.47:0/3240405721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e60ff63dc0 con 0x55e6100c7ad0 2024-03-23T09:42:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef13fff640 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e61003af70 0x55e6100c0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:42:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1c90a640 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6100c7ad0 0x55e6100c2660 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fef1401f840 tx=0x7fef14003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1a105640 1 -- 172.21.15.47:0/3240405721 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1405f410 con 0x55e6100c7ad0 2024-03-23T09:42:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.898+0000 7fef1a105640 1 -- 172.21.15.47:0/3240405721 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1406d070 con 0x55e6100c7ad0 2024-03-23T09:42:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.899+0000 7fef1f20cf40 1 -- 172.21.15.47:0/3240405721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e60ff49c20 con 0x55e6100c7ad0 2024-03-23T09:42:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.899+0000 7fef1a105640 1 -- 172.21.15.47:0/3240405721 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1405fea0 con 0x55e6100c7ad0 2024-03-23T09:42:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.899+0000 7fef1f20cf40 1 -- 172.21.15.47:0/3240405721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e60ff3d060 con 0x55e6100c7ad0 2024-03-23T09:42:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.900+0000 7fef1a105640 1 -- 172.21.15.47:0/3240405721 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fef14073640 con 0x55e6100c7ad0 2024-03-23T09:42:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.900+0000 7fef1a105640 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feefc0695e0 0x7feefc06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.900+0000 7fef1a105640 1 -- 172.21.15.47:0/3240405721 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef1406f070 con 0x55e6100c7ad0 2024-03-23T09:42:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.900+0000 7fef1f20cf40 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55e6100c35a0 0x55e610168ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.900+0000 7fef1f20cf40 1 -- 172.21.15.47:0/3240405721 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46) v8 -- 0x55e610169220 con 0x55e6100c35a0 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.900+0000 7fef1c109640 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feefc0695e0 0x7feefc06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.901+0000 7fef13fff640 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55e6100c35a0 0x55e610168ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.901+0000 7fef1c109640 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feefc0695e0 0x7feefc06baa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55e60fff5380 tx=0x7fef0c00c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.901+0000 7fef13fff640 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55e6100c35a0 0x55e610168ce0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.901+0000 7fef13fff640 1 -- 172.21.15.47:0/3240405721 <== osd.6 v2:172.21.15.175:6808/38160 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fef04001630 con 0x55e6100c35a0 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.901+0000 7fef1f20cf40 1 -- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55e6100c35a0 msgr2=0x55e610168ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.901+0000 7fef1f20cf40 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55e6100c35a0 0x55e610168ce0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.901+0000 7fef1f20cf40 1 -- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feefc0695e0 msgr2=0x7feefc06baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.901+0000 7fef1f20cf40 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feefc0695e0 0x7feefc06baa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55e60fff5380 tx=0x7fef0c00c040 comp rx=0 tx=0).stop 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.901+0000 7fef1f20cf40 1 -- 172.21.15.47:0/3240405721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6100c7ad0 msgr2=0x55e6100c2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.901+0000 7fef1f20cf40 1 --2- 172.21.15.47:0/3240405721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6100c7ad0 0x55e6100c2660 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fef1401f840 tx=0x7fef14003330 comp rx=0 tx=0).stop 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.901+0000 7fef1c90a640 1 -- 172.21.15.47:0/3240405721 reap_dead start 2024-03-23T09:42:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.901+0000 7fef1f20cf40 1 -- 172.21.15.47:0/3240405721 shutdown_connections 2024-03-23T09:42:59.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.901+0000 7fef1f20cf40 1 -- 172.21.15.47:0/3240405721 >> 172.21.15.47:0/3240405721 conn(0x55e61003e030 msgr2=0x55e6100c6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:42:59.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.902+0000 7fef1f20cf40 1 -- 172.21.15.47:0/3240405721 shutdown_connections 2024-03-23T09:42:59.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:42:59.902+0000 7fef1f20cf40 1 -- 172.21.15.47:0/3240405721 wait complete. 2024-03-23T09:42:59.908 INFO:teuthology.orchestra.run.smithi047.stdout:Yalphaparent_a 2024-03-23T09:42:59.909 DEBUG:teuthology.orchestra.run.smithi047:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-23T09:43:00.045 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:43:00.045 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 1099511627777, 2024-03-23T09:43:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "ancestors": [ 2024-03-23T09:43:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "dirino": 1099511627776, 2024-03-23T09:43:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "dname": "alpha", 2024-03-23T09:43:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "version": 2 2024-03-23T09:43:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "dirino": 1, 2024-03-23T09:43:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "dname": "parent_a", 2024-03-23T09:43:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "version": 4 2024-03-23T09:43:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-23T09:43:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 7, 2024-03-23T09:43:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: "old_pools": [] 2024-03-23T09:43:00.047 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:43:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:00.055 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-03-23T09:43:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.095+0000 7fe58a035f40 1 Processor -- start 2024-03-23T09:43:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.095+0000 7fe58a035f40 1 -- start start 2024-03-23T09:43:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.096+0000 7fe58a035f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56299c0054a0 0x56299c072a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.096+0000 7fe58a035f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56299c079370 0x56299c072f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.096+0000 7fe58a035f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56299c090ba0 0x56299c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.096+0000 7fe58a035f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56299bf684d0 con 0x56299c0054a0 2024-03-23T09:43:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.096+0000 7fe58a035f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56299bf03b60 con 0x56299c090ba0 2024-03-23T09:43:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.096+0000 7fe58a035f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x56299bf03800 con 0x56299c079370 2024-03-23T09:43:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.096+0000 7fe587374640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56299c0054a0 0x56299c072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.096+0000 7fe587374640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56299c0054a0 0x56299c072a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60020/0 (socket says 172.21.15.47:60020) 2024-03-23T09:43:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.096+0000 7fe587374640 1 -- 172.21.15.47:0/3552349340 learned_addr learned my addr 172.21.15.47:0/3552349340 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.096+0000 7fe587374640 1 -- 172.21.15.47:0/3552349340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56299c079370 msgr2=0x56299c072f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.096+0000 7fe587374640 1 --2- 172.21.15.47:0/3552349340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56299c079370 0x56299c072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.096+0000 7fe587374640 1 -- 172.21.15.47:0/3552349340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56299c090ba0 msgr2=0x56299c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.096+0000 7fe587b75640 1 --2- 172.21.15.47:0/3552349340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56299c090ba0 0x56299c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe587374640 1 --2- 172.21.15.47:0/3552349340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56299c090ba0 0x56299c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe587374640 1 -- 172.21.15.47:0/3552349340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58000a100 con 0x56299c0054a0 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe587b75640 1 --2- 172.21.15.47:0/3552349340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56299c090ba0 0x56299c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe587374640 1 --2- 172.21.15.47:0/3552349340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56299c0054a0 0x56299c072a50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe578006df0 tx=0x7fe57800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9d6f84219521f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe585b71640 1 -- 172.21.15.47:0/3552349340 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57800e470 con 0x56299c0054a0 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe585b71640 1 -- 172.21.15.47:0/3552349340 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57800e5d0 con 0x56299c0054a0 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe585b71640 1 -- 172.21.15.47:0/3552349340 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57800e980 con 0x56299c0054a0 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe58a035f40 1 -- 172.21.15.47:0/3552349340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56299c0054a0 msgr2=0x56299c072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe58a035f40 1 --2- 172.21.15.47:0/3552349340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56299c0054a0 0x56299c072a50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe578006df0 tx=0x7fe57800b6c0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe58a035f40 1 -- 172.21.15.47:0/3552349340 shutdown_connections 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe58a035f40 1 --2- 172.21.15.47:0/3552349340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56299c090ba0 0x56299c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe58a035f40 1 --2- 172.21.15.47:0/3552349340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56299c079370 0x56299c072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe58a035f40 1 --2- 172.21.15.47:0/3552349340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56299c0054a0 0x56299c072a50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe58a035f40 1 -- 172.21.15.47:0/3552349340 >> 172.21.15.47:0/3552349340 conn(0x56299bfe6e40 msgr2=0x56299c08f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe58a035f40 1 -- 172.21.15.47:0/3552349340 shutdown_connections 2024-03-23T09:43:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.097+0000 7fe58a035f40 1 -- 172.21.15.47:0/3552349340 wait complete. 2024-03-23T09:43:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.098+0000 7fe58a035f40 1 Processor -- start 2024-03-23T09:43:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.098+0000 7fe58a035f40 1 -- start start 2024-03-23T09:43:00.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe58a035f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56299c0054a0 0x56299c08f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe58a035f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56299c079370 0x56299c08d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe58a035f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56299c090ba0 0x56299c079750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe58a035f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56299bee6520 con 0x56299c079370 2024-03-23T09:43:00.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe58a035f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56299c08f6d0 con 0x56299c090ba0 2024-03-23T09:43:00.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe58a035f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x56299c06f6d0 con 0x56299c0054a0 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe586b73640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56299c079370 0x56299c08d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe586b73640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56299c079370 0x56299c08d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60032/0 (socket says 172.21.15.47:60032) 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe586b73640 1 -- 172.21.15.47:0/1240370122 learned_addr learned my addr 172.21.15.47:0/1240370122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe587374640 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56299c0054a0 0x56299c08f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe587b75640 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56299c090ba0 0x56299c079750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe586b73640 1 -- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56299c0054a0 msgr2=0x56299c08f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe586b73640 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56299c0054a0 0x56299c08f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe586b73640 1 -- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56299c090ba0 msgr2=0x56299c079750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe586b73640 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56299c090ba0 0x56299c079750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe586b73640 1 -- 172.21.15.47:0/1240370122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56299bf05cd0 con 0x56299c079370 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe586b73640 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56299c079370 0x56299c08d3d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00bac0 tx=0x7fe57c00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe584b6f640 1 -- 172.21.15.47:0/1240370122 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c009b30 con 0x56299c079370 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe584b6f640 1 -- 172.21.15.47:0/1240370122 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c008030 con 0x56299c079370 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe58a035f40 1 -- 172.21.15.47:0/1240370122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56299bf06060 con 0x56299c079370 2024-03-23T09:43:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe584b6f640 1 -- 172.21.15.47:0/1240370122 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c00f400 con 0x56299c079370 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.099+0000 7fe58a035f40 1 -- 172.21.15.47:0/1240370122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56299bf05ac0 con 0x56299c079370 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.101+0000 7fe584b6f640 1 -- 172.21.15.47:0/1240370122 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe57c00f560 con 0x56299c079370 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.101+0000 7fe584b6f640 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5740695e0 0x7fe57406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.101+0000 7fe584b6f640 1 -- 172.21.15.47:0/1240370122 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe57c087590 con 0x56299c079370 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.101+0000 7fe587374640 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5740695e0 0x7fe57406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.101+0000 7fe58a035f40 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x56299c132ff0 0x56299c1353e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.101+0000 7fe58a035f40 1 -- 172.21.15.47:0/1240370122 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46) v8 -- 0x56299c135920 con 0x56299c132ff0 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.102+0000 7fe587b75640 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x56299c132ff0 0x56299c1353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.102+0000 7fe587374640 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5740695e0 0x7fe57406baa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe578006df0 tx=0x7fe57800f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.102+0000 7fe587b75640 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x56299c132ff0 0x56299c1353e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.103+0000 7fe587b75640 1 -- 172.21.15.47:0/1240370122 <== osd.6 v2:172.21.15.175:6808/38160 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v46'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe580050970 con 0x56299c132ff0 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.103+0000 7fe58a035f40 1 -- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x56299c132ff0 msgr2=0x56299c1353e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.103+0000 7fe58a035f40 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x56299c132ff0 0x56299c1353e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.103+0000 7fe58a035f40 1 -- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5740695e0 msgr2=0x7fe57406baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.103+0000 7fe58a035f40 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5740695e0 0x7fe57406baa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe578006df0 tx=0x7fe57800f000 comp rx=0 tx=0).stop 2024-03-23T09:43:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.103+0000 7fe58a035f40 1 -- 172.21.15.47:0/1240370122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56299c079370 msgr2=0x56299c08d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.103+0000 7fe58a035f40 1 --2- 172.21.15.47:0/1240370122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56299c079370 0x56299c08d3d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00bac0 tx=0x7fe57c00bf80 comp rx=0 tx=0).stop 2024-03-23T09:43:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.103+0000 7fe587b75640 1 -- 172.21.15.47:0/1240370122 reap_dead start 2024-03-23T09:43:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.103+0000 7fe58a035f40 1 -- 172.21.15.47:0/1240370122 shutdown_connections 2024-03-23T09:43:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.103+0000 7fe58a035f40 1 -- 172.21.15.47:0/1240370122 >> 172.21.15.47:0/1240370122 conn(0x56299bfe6e40 msgr2=0x56299c072a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.103+0000 7fe58a035f40 1 -- 172.21.15.47:0/1240370122 shutdown_connections 2024-03-23T09:43:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.103+0000 7fe58a035f40 1 -- 172.21.15.47:0/1240370122 wait complete. 2024-03-23T09:43:00.108 DEBUG:teuthology.orchestra.run.smithi047:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-23T09:43:00.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:00.151 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-03-23T09:43:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.255+0000 7f7173ba9640 1 Processor -- start 2024-03-23T09:43:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.256+0000 7f7173ba9640 1 -- start start 2024-03-23T09:43:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.256+0000 7f7173ba9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f716c0cf8a0 0x7f716c15d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.256+0000 7f7173ba9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716c160bd0 con 0x7f716c15d0f0 2024-03-23T09:43:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.256+0000 7f7173ba9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716c15dbb0 con 0x7f716c15a860 2024-03-23T09:43:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.256+0000 7f7173ba9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716c0d2790 con 0x7f716c0cf8a0 2024-03-23T09:43:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.257+0000 7f7171144640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f716c15d0f0 0x7f716c0cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52078/0 (socket says 172.21.15.47:52078) 2024-03-23T09:43:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.257+0000 7f7171144640 1 -- 172.21.15.47:0/4287224966 learned_addr learned my addr 172.21.15.47:0/4287224966 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.257+0000 7f7172146640 1 --2- 172.21.15.47:0/4287224966 >> v2:172.21.15.175:3300/0 conn(0x7f716c0cf8a0 0x7f716c15d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.257+0000 7f7170943640 1 -- 172.21.15.47:0/4287224966 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695101588 0 0) 0x7f716c160bd0 con 0x7f716c15d0f0 2024-03-23T09:43:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.257+0000 7f7170943640 1 -- 172.21.15.47:0/4287224966 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7154003520 con 0x7f716c15d0f0 2024-03-23T09:43:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.257+0000 7f7172146640 1 -- 172.21.15.47:0/4287224966 >> v1:172.21.15.175:6789/0 conn(0x7f716c15a860 legacy=0x7f716c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.257+0000 7f7172146640 1 -- 172.21.15.47:0/4287224966 >> v1:172.21.15.47:6789/0 conn(0x7f716c15d0f0 legacy=0x7f716c0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.257+0000 7f7172146640 1 -- 172.21.15.47:0/4287224966 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c15de90 con 0x7f716c0cf8a0 2024-03-23T09:43:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.258+0000 7f7172146640 1 --2- 172.21.15.47:0/4287224966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c0cf8a0 0x7f716c15d580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f715c0097a0 tx=0x7f715c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfe123bb4c60ac6c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.258+0000 7f7170943640 1 -- 172.21.15.47:0/4287224966 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c030610 con 0x7f716c0cf8a0 2024-03-23T09:43:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.258+0000 7f7170943640 1 -- 172.21.15.47:0/4287224966 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c030770 con 0x7f716c0cf8a0 2024-03-23T09:43:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.258+0000 7f7170943640 1 -- 172.21.15.47:0/4287224966 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c030ae0 con 0x7f716c0cf8a0 2024-03-23T09:43:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.259+0000 7f7173ba9640 1 -- 172.21.15.47:0/4287224966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c0cf8a0 msgr2=0x7f716c15d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.259+0000 7f7173ba9640 1 --2- 172.21.15.47:0/4287224966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c0cf8a0 0x7f716c15d580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f715c0097a0 tx=0x7f715c02b960 comp rx=0 tx=0).stop 2024-03-23T09:43:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.259+0000 7f7173ba9640 1 -- 172.21.15.47:0/4287224966 shutdown_connections 2024-03-23T09:43:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.259+0000 7f7173ba9640 1 --2- 172.21.15.47:0/4287224966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c0cf8a0 0x7f716c15d580 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.259+0000 7f7173ba9640 1 -- 172.21.15.47:0/4287224966 >> 172.21.15.47:0/4287224966 conn(0x7f716c0659d0 msgr2=0x7f716c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.259+0000 7f7173ba9640 1 -- 172.21.15.47:0/4287224966 shutdown_connections 2024-03-23T09:43:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.259+0000 7f7173ba9640 1 -- 172.21.15.47:0/4287224966 wait complete. 2024-03-23T09:43:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.260+0000 7f7173ba9640 1 Processor -- start 2024-03-23T09:43:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.260+0000 7f7173ba9640 1 -- start start 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7173ba9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f716c0cf8a0 0x7f716c0c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7173ba9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f716c15a860 0x7f716c0b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7173ba9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c15d0f0 0x7f716c1f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7173ba9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0b31e0 con 0x7f716c0cf8a0 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7173ba9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0b3480 con 0x7f716c15d0f0 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7173ba9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0b3760 con 0x7f716c15a860 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7171945640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f716c0cf8a0 0x7f716c0c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7171945640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f716c0cf8a0 0x7f716c0c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60038/0 (socket says 172.21.15.47:60038) 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7171945640 1 -- 172.21.15.47:0/1052705955 learned_addr learned my addr 172.21.15.47:0/1052705955 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7171144640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f716c15a860 0x7f716c0b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7172146640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c15d0f0 0x7f716c1f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7171945640 1 -- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f716c15a860 msgr2=0x7f716c0b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7171945640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f716c15a860 0x7f716c0b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7171945640 1 -- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c15d0f0 msgr2=0x7f716c1f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7171945640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c15d0f0 0x7f716c1f6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7171945640 1 -- 172.21.15.47:0/1052705955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c009400 con 0x7f716c0cf8a0 2024-03-23T09:43:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7171945640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f716c0cf8a0 0x7f716c0c7a10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f716400c790 tx=0x7f716400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f716a7fc640 1 -- 172.21.15.47:0/1052705955 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716400cea0 con 0x7f716c0cf8a0 2024-03-23T09:43:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f716a7fc640 1 -- 172.21.15.47:0/1052705955 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7164006d50 con 0x7f716c0cf8a0 2024-03-23T09:43:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.261+0000 7f7173ba9640 1 -- 172.21.15.47:0/1052705955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c0b3a40 con 0x7f716c0cf8a0 2024-03-23T09:43:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.262+0000 7f716a7fc640 1 -- 172.21.15.47:0/1052705955 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164003bb0 con 0x7f716c0cf8a0 2024-03-23T09:43:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.262+0000 7f7173ba9640 1 -- 172.21.15.47:0/1052705955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c1f6000 con 0x7f716c0cf8a0 2024-03-23T09:43:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.263+0000 7f716a7fc640 1 -- 172.21.15.47:0/1052705955 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7164003d10 con 0x7f716c0cf8a0 2024-03-23T09:43:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.263+0000 7f716a7fc640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f715806ac00 0x7f715806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.263+0000 7f716a7fc640 1 -- 172.21.15.47:0/1052705955 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7164088e70 con 0x7f716c0cf8a0 2024-03-23T09:43:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.264+0000 7f7171144640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f715806ac00 0x7f715806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.264+0000 7f7171144640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f715806ac00 0x7f715806d0c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f716c0b2390 tx=0x7f7160007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.265+0000 7f71767f1740 1 Processor -- start 2024-03-23T09:43:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.265+0000 7f71767f1740 1 -- start start 2024-03-23T09:43:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.265+0000 7f71767f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382336b2f0 0x56382336b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.265+0000 7f71767f1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382336f340 0x563823370de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.265+0000 7f71767f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 0x56382339b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.265+0000 7f71767f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638232dc940 con 0x56382339b080 2024-03-23T09:43:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.265+0000 7f71767f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638232da0d0 con 0x56382336b2f0 2024-03-23T09:43:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.265+0000 7f71767f1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638232d59c0 con 0x56382336f340 2024-03-23T09:43:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f7172146640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 0x56382339b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f7171144640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382336f340 0x563823370de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f7172146640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 0x56382339b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60052/0 (socket says 172.21.15.47:60052) 2024-03-23T09:43:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f7171144640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382336f340 0x563823370de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:60564/0 (socket says 172.21.15.47:60564) 2024-03-23T09:43:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f7172146640 1 -- 172.21.15.47:0/94090520 learned_addr learned my addr 172.21.15.47:0/94090520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f7171945640 1 --2- 172.21.15.47:0/94090520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382336b2f0 0x56382336b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f7172146640 1 -- 172.21.15.47:0/94090520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382336f340 msgr2=0x563823370de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f7172146640 1 --2- 172.21.15.47:0/94090520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382336f340 0x563823370de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f7172146640 1 -- 172.21.15.47:0/94090520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382336b2f0 msgr2=0x56382336b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f7172146640 1 --2- 172.21.15.47:0/94090520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382336b2f0 0x56382336b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f7172146640 1 -- 172.21.15.47:0/94090520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638232a18a0 con 0x56382339b080 2024-03-23T09:43:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f7172146640 1 --2- 172.21.15.47:0/94090520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 0x56382339b500 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f715c0098c0 tx=0x7f715c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ab6879fa060ced server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f71537fe640 1 -- 172.21.15.47:0/94090520 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c004240 con 0x56382339b080 2024-03-23T09:43:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f71537fe640 1 -- 172.21.15.47:0/94090520 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c03f040 con 0x56382339b080 2024-03-23T09:43:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.266+0000 7f71537fe640 1 -- 172.21.15.47:0/94090520 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c030590 con 0x56382339b080 2024-03-23T09:43:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.267+0000 7f71767f1740 1 -- 172.21.15.47:0/94090520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 msgr2=0x56382339b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.267+0000 7f71767f1740 1 --2- 172.21.15.47:0/94090520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 0x56382339b500 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f715c0098c0 tx=0x7f715c030000 comp rx=0 tx=0).stop 2024-03-23T09:43:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.267+0000 7f71767f1740 1 -- 172.21.15.47:0/94090520 shutdown_connections 2024-03-23T09:43:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.267+0000 7f71767f1740 1 --2- 172.21.15.47:0/94090520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 0x56382339b500 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.267+0000 7f71767f1740 1 --2- 172.21.15.47:0/94090520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382336f340 0x563823370de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.267+0000 7f71767f1740 1 --2- 172.21.15.47:0/94090520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382336b2f0 0x56382336b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.267+0000 7f71767f1740 1 -- 172.21.15.47:0/94090520 >> 172.21.15.47:0/94090520 conn(0x56382336d370 msgr2=0x56382336ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.267+0000 7f71767f1740 1 -- 172.21.15.47:0/94090520 shutdown_connections 2024-03-23T09:43:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.267+0000 7f71767f1740 1 -- 172.21.15.47:0/94090520 wait complete. 2024-03-23T09:43:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.268+0000 7f71767f1740 1 -- start start 2024-03-23T09:43:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.268+0000 7f71767f1740 1 Processor -- start 2024-03-23T09:43:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.268+0000 7f71767f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382339b080 0x56382338e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.268+0000 7f71767f1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5638233b67b0 0x563823388100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f71767f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563823388640 0x5638233b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f71767f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563823276a90 con 0x563823388640 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f71767f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563823276730 con 0x56382339b080 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f71767f1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x563823258e00 con 0x5638233b67b0 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f71767f1740 10 client.0 _ll_register_callbacks cb 0x56382328eb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7172146640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563823388640 0x5638233b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7172146640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563823388640 0x5638233b8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60062/0 (socket says 172.21.15.47:60062) 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7172146640 1 -- 172.21.15.47:0/2413868257 learned_addr learned my addr 172.21.15.47:0/2413868257 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7171144640 1 --2- 172.21.15.47:0/2413868257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5638233b67b0 0x563823388100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7171945640 1 --2- 172.21.15.47:0/2413868257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382339b080 0x56382338e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7172146640 1 -- 172.21.15.47:0/2413868257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5638233b67b0 msgr2=0x563823388100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7172146640 1 --2- 172.21.15.47:0/2413868257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5638233b67b0 0x563823388100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7172146640 1 -- 172.21.15.47:0/2413868257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382339b080 msgr2=0x56382338e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7172146640 1 --2- 172.21.15.47:0/2413868257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382339b080 0x56382338e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7172146640 1 -- 172.21.15.47:0/2413868257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c15de90 con 0x563823388640 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7172146640 1 --2- 172.21.15.47:0/2413868257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563823388640 0x5638233b8af0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f715c009890 tx=0x7f715c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb66a487b1e0fb53 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7150ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7150ff9640 1 -- 172.21.15.47:0/2413868257 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c030af0 con 0x563823388640 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f71767f1740 1 -- 172.21.15.47:0/2413868257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563822fc9560 con 0x563823388640 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7150ff9640 1 -- 172.21.15.47:0/2413868257 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f715c030c50 con 0x563823388640 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.269+0000 7f7150ff9640 1 -- 172.21.15.47:0/2413868257 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f715c030e10 con 0x563823388640 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.270+0000 7f71767f1740 10 client.5916 fetch_fsmap learned FSMap version 18 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.270+0000 7f71767f1740 1 -- 172.21.15.47:0/2413868257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5638232d4040 con 0x563823388640 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.270+0000 7f7150ff9640 1 -- 172.21.15.47:0/2413868257 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563822fc9560 con 0x563823388640 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.270+0000 7f7150ff9640 1 -- 172.21.15.47:0/2413868257 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 18) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f715c003610 con 0x563823388640 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.270+0000 7f71767f1740 10 client.5916 fetch_fsmap finished waiting for FSMap version 18 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.270+0000 7f71767f1740 10 client.5916 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.270+0000 7f71767f1740 20 client.5916 populate_metadata read hostname 'smithi047' 2024-03-23T09:43:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.270+0000 7f71767f1740 1 --2- 172.21.15.47:0/2413868257 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x563823399510 0x563823399920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.270+0000 7f71767f1740 4 client.5916 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.270+0000 7f71767f1740 1 -- 172.21.15.47:0/2413868257 --> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563822fda860 con 0x563823399510 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.270+0000 7f7171945640 1 --2- 172.21.15.47:0/2413868257 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x563823399510 0x563823399920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.270+0000 7f7150ff9640 10 client.5916 ms_handle_connect on v2:172.21.15.175:6836/1626348200 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.270+0000 7f7171945640 1 --2- 172.21.15.47:0/2413868257 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x563823399510 0x563823399920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e937c7b27553d275 server_cookie=670afe72e964182f in_seq=0 out_seq=0 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f7150ff9640 1 -- 172.21.15.47:0/2413868257 <== mds.0 v2:172.21.15.175:6836/1626348200 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563822fda860 con 0x563823399510 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f7150ff9640 10 client.5916 handle_command_reply: tid=0 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f71767f1740 1 client.5916 shutdown 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f71767f1740 1 -- 172.21.15.47:0/2413868257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563823388640 msgr2=0x5638233b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f71767f1740 1 --2- 172.21.15.47:0/2413868257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563823388640 0x5638233b8af0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f715c009890 tx=0x7f715c004730 comp rx=0 tx=0).stop 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f71767f1740 1 -- 172.21.15.47:0/2413868257 shutdown_connections 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f71767f1740 1 --2- 172.21.15.47:0/2413868257 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x563823399510 0x563823399920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f71767f1740 1 --2- 172.21.15.47:0/2413868257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563823388640 0x5638233b8af0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f71767f1740 1 --2- 172.21.15.47:0/2413868257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5638233b67b0 0x563823388100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f71767f1740 1 --2- 172.21.15.47:0/2413868257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382339b080 0x56382338e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f71767f1740 1 -- 172.21.15.47:0/2413868257 >> 172.21.15.47:0/2413868257 conn(0x56382336b2f0 msgr2=0x563823371240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f7150ff9640 0 client.5916 ms_handle_reset on v2:172.21.15.175:6836/1626348200 2024-03-23T09:43:00.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f71767f1740 1 -- 172.21.15.47:0/2413868257 shutdown_connections 2024-03-23T09:43:00.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.271+0000 7f71767f1740 1 -- 172.21.15.47:0/2413868257 wait complete. 2024-03-23T09:43:00.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.272+0000 7f71767f1740 20 client.5916 trim_cache size 0 max 16384 2024-03-23T09:43:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.274+0000 7f94e55e7640 1 Processor -- start 2024-03-23T09:43:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.274+0000 7f94e55e7640 1 -- start start 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.275+0000 7f94e55e7640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f94e015a970 0x7f94e00cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.275+0000 7f94e55e7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e00b05b0 0x7f94e00b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.275+0000 7f94e55e7640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e00b18a0 con 0x7f94e00d0710 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.275+0000 7f94e55e7640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e00b12e0 con 0x7f94e015a970 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.275+0000 7f94e55e7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e00b15c0 con 0x7f94e00b05b0 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.275+0000 7f94deffd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f94e015a970 0x7f94e00cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.275+0000 7f94df7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e00b05b0 0x7f94e00b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.275+0000 7f94deffd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f94e015a970 0x7f94e00cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:52946/0 (socket says 172.21.15.47:52946) 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.275+0000 7f94de7fc640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f94e00d0710 0x7f94e00d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:39160/0 (socket says 172.21.15.47:39160) 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.276+0000 7f94deffd640 1 -- 172.21.15.47:0/3941876833 learned_addr learned my addr 172.21.15.47:0/3941876833 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.276+0000 7f94deffd640 1 -- 172.21.15.47:0/3941876833 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e00b05b0 msgr2=0x7f94e00b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.276+0000 7f94deffd640 1 --2- 172.21.15.47:0/3941876833 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e00b05b0 0x7f94e00b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.276+0000 7f94deffd640 1 -- 172.21.15.47:0/3941876833 >> v1:172.21.15.175:6789/0 conn(0x7f94e00d0710 legacy=0x7f94e00d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.276+0000 7f94deffd640 1 -- 172.21.15.47:0/3941876833 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e00ce3f0 con 0x7f94e015a970 2024-03-23T09:43:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.277+0000 7f94deffd640 1 --2- 172.21.15.47:0/3941876833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94e015a970 0x7f94e00cb390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009870 tx=0x7f94d402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e55a9c792a46e35f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.277+0000 7f94ddffb640 1 -- 172.21.15.47:0/3941876833 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4030840 con 0x7f94e015a970 2024-03-23T09:43:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.277+0000 7f94ddffb640 1 -- 172.21.15.47:0/3941876833 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d40309a0 con 0x7f94e015a970 2024-03-23T09:43:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.277+0000 7f94ddffb640 1 -- 172.21.15.47:0/3941876833 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4030d30 con 0x7f94e015a970 2024-03-23T09:43:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.277+0000 7f94e55e7640 1 -- 172.21.15.47:0/3941876833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94e015a970 msgr2=0x7f94e00cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.277+0000 7f94e55e7640 1 --2- 172.21.15.47:0/3941876833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94e015a970 0x7f94e00cb390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009870 tx=0x7f94d402b980 comp rx=0 tx=0).stop 2024-03-23T09:43:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.278+0000 7f94e55e7640 1 -- 172.21.15.47:0/3941876833 shutdown_connections 2024-03-23T09:43:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.278+0000 7f94e55e7640 1 --2- 172.21.15.47:0/3941876833 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e00b05b0 0x7f94e00b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.278+0000 7f94e55e7640 1 --2- 172.21.15.47:0/3941876833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94e015a970 0x7f94e00cb390 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.278+0000 7f94e55e7640 1 -- 172.21.15.47:0/3941876833 >> 172.21.15.47:0/3941876833 conn(0x7f94e00706c0 msgr2=0x7f94e0099af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.278+0000 7f94e55e7640 1 -- 172.21.15.47:0/3941876833 shutdown_connections 2024-03-23T09:43:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.278+0000 7f94e55e7640 1 -- 172.21.15.47:0/3941876833 wait complete. 2024-03-23T09:43:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.280+0000 7f94e55e7640 1 Processor -- start 2024-03-23T09:43:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.280+0000 7f94e55e7640 1 -- start start 2024-03-23T09:43:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94e55e7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94e00b05b0 0x7f94e00c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94e55e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e00d0710 0x7f94e00b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94e55e7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e015a970 0x7f94e01f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94e55e7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e01edb80 con 0x7f94e00d0710 2024-03-23T09:43:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94e55e7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e00c7d50 con 0x7f94e00b05b0 2024-03-23T09:43:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94e55e7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e00b6330 con 0x7f94e015a970 2024-03-23T09:43:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94de7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e00d0710 0x7f94e00b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94de7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e00d0710 0x7f94e00b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60078/0 (socket says 172.21.15.47:60078) 2024-03-23T09:43:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94de7fc640 1 -- 172.21.15.47:0/2497233436 learned_addr learned my addr 172.21.15.47:0/2497233436 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94deffd640 1 --2- 172.21.15.47:0/2497233436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94e00b05b0 0x7f94e00c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94df7fe640 1 --2- 172.21.15.47:0/2497233436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e015a970 0x7f94e01f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94de7fc640 1 -- 172.21.15.47:0/2497233436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e015a970 msgr2=0x7f94e01f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94de7fc640 1 --2- 172.21.15.47:0/2497233436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e015a970 0x7f94e01f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94de7fc640 1 -- 172.21.15.47:0/2497233436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94e00b05b0 msgr2=0x7f94e00c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94de7fc640 1 --2- 172.21.15.47:0/2497233436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94e00b05b0 0x7f94e00c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.281+0000 7f94de7fc640 1 -- 172.21.15.47:0/2497233436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d40094d0 con 0x7f94e00d0710 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.282+0000 7f94de7fc640 1 --2- 172.21.15.47:0/2497233436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e00d0710 0x7f94e00b5df0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f94d000c430 tx=0x7f94d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.282+0000 7f94c77fe640 1 -- 172.21.15.47:0/2497233436 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d00074e0 con 0x7f94e00d0710 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.282+0000 7f94c77fe640 1 -- 172.21.15.47:0/2497233436 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d0007640 con 0x7f94e00d0710 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.282+0000 7f94e55e7640 1 -- 172.21.15.47:0/2497233436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e015d500 con 0x7f94e00d0710 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.282+0000 7f94c77fe640 1 -- 172.21.15.47:0/2497233436 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d0004a00 con 0x7f94e00d0710 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.282+0000 7f94e55e7640 1 -- 172.21.15.47:0/2497233436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e015dac0 con 0x7f94e00d0710 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.282+0000 7f94e55e7640 1 -- 172.21.15.47:0/2497233436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f94e00672f0 con 0x7f94e00d0710 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.284+0000 7f94c77fe640 1 -- 172.21.15.47:0/2497233436 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f94d0004b60 con 0x7f94e00d0710 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.284+0000 7f94c77fe640 1 --2- 172.21.15.47:0/2497233436 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f94cc06ac00 0x7f94cc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.284+0000 7f94c77fe640 1 -- 172.21.15.47:0/2497233436 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94d00896e0 con 0x7f94e00d0710 2024-03-23T09:43:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.284+0000 7f94deffd640 1 --2- 172.21.15.47:0/2497233436 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f94cc06ac00 0x7f94cc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.284+0000 7f94c77fe640 1 -- 172.21.15.47:0/2497233436 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+501 (secure 0 0 0) 0x7f94d0089b50 con 0x7f94e00d0710 2024-03-23T09:43:00.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.284+0000 7f94e55e7640 1 -- 172.21.15.47:0/2497233436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f94e00b02b0 con 0x7f94e00d0710 2024-03-23T09:43:00.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.285+0000 7f94deffd640 1 --2- 172.21.15.47:0/2497233436 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f94cc06ac00 0x7f94cc06d0c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f94d4036000 tx=0x7f94d4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.290+0000 7f71767f1740 1 Processor -- start 2024-03-23T09:43:00.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.290+0000 7f71767f1740 1 -- start start 2024-03-23T09:43:00.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.290+0000 7f71767f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823388640 0x563823366be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.290+0000 7f71767f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 0x56382336fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.290+0000 7f71767f1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5638233b67b0 0x563823370110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.290+0000 7f71767f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563823366990 con 0x56382339b080 2024-03-23T09:43:00.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.290+0000 7f71767f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56382336ed90 con 0x563823388640 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.290+0000 7f71767f1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x563823398e40 con 0x5638233b67b0 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7171144640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 0x56382336fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7171144640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 0x56382336fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60082/0 (socket says 172.21.15.47:60082) 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7171144640 1 -- 172.21.15.47:0/2732564283 learned_addr learned my addr 172.21.15.47:0/2732564283 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7171945640 1 --2- 172.21.15.47:0/2732564283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823388640 0x563823366be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7172146640 1 --2- 172.21.15.47:0/2732564283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5638233b67b0 0x563823370110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7171144640 1 -- 172.21.15.47:0/2732564283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5638233b67b0 msgr2=0x563823370110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7171144640 1 --2- 172.21.15.47:0/2732564283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5638233b67b0 0x563823370110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7171144640 1 -- 172.21.15.47:0/2732564283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823388640 msgr2=0x563823366be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7171144640 1 --2- 172.21.15.47:0/2732564283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823388640 0x563823366be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7171144640 1 -- 172.21.15.47:0/2732564283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563823275e20 con 0x56382339b080 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7171144640 1 --2- 172.21.15.47:0/2732564283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 0x56382336fbd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7160000f60 tx=0x7f7160012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b7d37a6c5b2ae4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7153fff640 1 -- 172.21.15.47:0/2732564283 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160001410 con 0x56382339b080 2024-03-23T09:43:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7153fff640 1 -- 172.21.15.47:0/2732564283 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7160026070 con 0x56382339b080 2024-03-23T09:43:00.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.291+0000 7f7153fff640 1 -- 172.21.15.47:0/2732564283 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160001790 con 0x56382339b080 2024-03-23T09:43:00.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.292+0000 7f71767f1740 1 -- 172.21.15.47:0/2732564283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 msgr2=0x56382336fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.292+0000 7f71767f1740 1 --2- 172.21.15.47:0/2732564283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 0x56382336fbd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7160000f60 tx=0x7f7160012000 comp rx=0 tx=0).stop 2024-03-23T09:43:00.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.292+0000 7f71767f1740 1 -- 172.21.15.47:0/2732564283 shutdown_connections 2024-03-23T09:43:00.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.292+0000 7f71767f1740 1 --2- 172.21.15.47:0/2732564283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5638233b67b0 0x563823370110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.292+0000 7f71767f1740 1 --2- 172.21.15.47:0/2732564283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382339b080 0x56382336fbd0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.292+0000 7f71767f1740 1 --2- 172.21.15.47:0/2732564283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823388640 0x563823366be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.292+0000 7f71767f1740 1 -- 172.21.15.47:0/2732564283 >> 172.21.15.47:0/2732564283 conn(0x56382336b2f0 msgr2=0x56382336b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.292+0000 7f71767f1740 1 -- 172.21.15.47:0/2732564283 shutdown_connections 2024-03-23T09:43:00.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.292+0000 7f71767f1740 1 -- 172.21.15.47:0/2732564283 wait complete. 2024-03-23T09:43:00.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.292+0000 7f71767f1740 1 -- start start 2024-03-23T09:43:00.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.293+0000 7f71767f1740 1 Processor -- start 2024-03-23T09:43:00.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.293+0000 7f71767f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823388640 0x56382339b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.293+0000 7f71767f1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382339b080 0x56382339b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.293+0000 7f71767f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638233b67b0 0x56382338c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.293+0000 7f71767f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56382341d4c0 con 0x5638233b67b0 2024-03-23T09:43:00.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.293+0000 7f71767f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563823372270 con 0x563823388640 2024-03-23T09:43:00.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.293+0000 7f71767f1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x56382341cff0 con 0x56382339b080 2024-03-23T09:43:00.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f71767f1740 10 client.0 _ll_register_callbacks cb 0x7f715c009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:43:00.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7172146640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638233b67b0 0x56382338c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7171144640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382339b080 0x56382339b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7171945640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823388640 0x56382339b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7171144640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382339b080 0x56382339b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:60622/0 (socket says 172.21.15.47:60622) 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7171144640 1 -- 172.21.15.47:0/446760343 learned_addr learned my addr 172.21.15.47:0/446760343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7172146640 1 -- 172.21.15.47:0/446760343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382339b080 msgr2=0x56382339b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7172146640 1 --2- 172.21.15.47:0/446760343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382339b080 0x56382339b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7172146640 1 -- 172.21.15.47:0/446760343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823388640 msgr2=0x56382339b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7172146640 1 --2- 172.21.15.47:0/446760343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823388640 0x56382339b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7172146640 1 -- 172.21.15.47:0/446760343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7154003520 con 0x5638233b67b0 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7171945640 1 --2- 172.21.15.47:0/446760343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823388640 0x56382339b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7172146640 1 --2- 172.21.15.47:0/446760343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638233b67b0 0x56382338c2d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f715c0098c0 tx=0x7f715c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b8645930a7e61f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7150ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7150ff9640 1 -- 172.21.15.47:0/446760343 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c030510 con 0x5638233b67b0 2024-03-23T09:43:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f7150ff9640 1 -- 172.21.15.47:0/446760343 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f715c000e40 con 0x5638233b67b0 2024-03-23T09:43:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.294+0000 7f71767f1740 1 -- 172.21.15.47:0/446760343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638233b9730 con 0x5638233b67b0 2024-03-23T09:43:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f7150ff9640 1 -- 172.21.15.47:0/446760343 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f715c006a20 con 0x5638233b67b0 2024-03-23T09:43:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f71767f1740 10 client.5925 fetch_fsmap learned FSMap version 18 2024-03-23T09:43:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f71767f1740 1 -- 172.21.15.47:0/446760343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x56382329b5f0 con 0x5638233b67b0 2024-03-23T09:43:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f7150ff9640 1 -- 172.21.15.47:0/446760343 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5638233b9730 con 0x5638233b67b0 2024-03-23T09:43:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f7150ff9640 1 -- 172.21.15.47:0/446760343 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 18) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f715c030af0 con 0x5638233b67b0 2024-03-23T09:43:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f71767f1740 10 client.5925 fetch_fsmap finished waiting for FSMap version 18 2024-03-23T09:43:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f71767f1740 10 client.5925 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-03-23T09:43:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f71767f1740 20 client.5925 populate_metadata read hostname 'smithi047' 2024-03-23T09:43:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f71767f1740 1 --2- 172.21.15.47:0/446760343 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x5638233b5da0 0x56382338c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f71767f1740 4 client.5925 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-23T09:43:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f71767f1740 1 -- 172.21.15.47:0/446760343 --> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x7f715c03f040 con 0x5638233b5da0 2024-03-23T09:43:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f7171945640 1 --2- 172.21.15.47:0/446760343 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x5638233b5da0 0x56382338c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f7150ff9640 10 client.5925 ms_handle_connect on v2:172.21.15.175:6836/1626348200 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.295+0000 7f7171945640 1 --2- 172.21.15.47:0/446760343 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x5638233b5da0 0x56382338c810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9589d540c22d54d server_cookie=bf233f49c995faa4 in_seq=0 out_seq=0 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f7150ff9640 1 -- 172.21.15.47:0/446760343 <== mds.0 v2:172.21.15.175:6836/1626348200 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f715c03f040 con 0x5638233b5da0 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f7150ff9640 10 client.5925 handle_command_reply: tid=0 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f71767f1740 1 client.5925 shutdown 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f71767f1740 1 -- 172.21.15.47:0/446760343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638233b67b0 msgr2=0x56382338c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f71767f1740 1 --2- 172.21.15.47:0/446760343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638233b67b0 0x56382338c2d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f715c0098c0 tx=0x7f715c008030 comp rx=0 tx=0).stop 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f71767f1740 1 -- 172.21.15.47:0/446760343 shutdown_connections 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f71767f1740 1 --2- 172.21.15.47:0/446760343 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x5638233b5da0 0x56382338c810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f71767f1740 1 --2- 172.21.15.47:0/446760343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638233b67b0 0x56382338c2d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f71767f1740 1 --2- 172.21.15.47:0/446760343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382339b080 0x56382339b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f71767f1740 1 --2- 172.21.15.47:0/446760343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823388640 0x56382339b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f71767f1740 1 -- 172.21.15.47:0/446760343 >> 172.21.15.47:0/446760343 conn(0x56382336b2f0 msgr2=0x56382336fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f7150ff9640 0 client.5925 ms_handle_reset on v2:172.21.15.175:6836/1626348200 2024-03-23T09:43:00.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f71767f1740 1 -- 172.21.15.47:0/446760343 shutdown_connections 2024-03-23T09:43:00.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.296+0000 7f71767f1740 1 -- 172.21.15.47:0/446760343 wait complete. 2024-03-23T09:43:00.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.297+0000 7f71767f1740 20 client.5925 trim_cache size 0 max 16384 2024-03-23T09:43:00.303 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:43:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-23T09:43:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "fcc4f0f2-1b33-4f47-9cfe-02003121910a", 2024-03-23T09:43:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-23T09:43:00.303 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:43:00.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.297+0000 7f7173ba9640 1 -- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f715806ac00 msgr2=0x7f715806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.297+0000 7f7173ba9640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f715806ac00 0x7f715806d0c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f716c0b2390 tx=0x7f7160007d30 comp rx=0 tx=0).stop 2024-03-23T09:43:00.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.297+0000 7f7173ba9640 1 -- 172.21.15.47:0/1052705955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f716c0cf8a0 msgr2=0x7f716c0c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.297+0000 7f7173ba9640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f716c0cf8a0 0x7f716c0c7a10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f716400c790 tx=0x7f716400cc50 comp rx=0 tx=0).stop 2024-03-23T09:43:00.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.297+0000 7f7173ba9640 1 -- 172.21.15.47:0/1052705955 shutdown_connections 2024-03-23T09:43:00.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.297+0000 7f7173ba9640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f715806ac00 0x7f715806d0c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.297+0000 7f7173ba9640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c15d0f0 0x7f716c1f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.297+0000 7f7173ba9640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f716c15a860 0x7f716c0b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.297+0000 7f7173ba9640 1 --2- 172.21.15.47:0/1052705955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f716c0cf8a0 0x7f716c0c7a10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.297+0000 7f7173ba9640 1 -- 172.21.15.47:0/1052705955 >> 172.21.15.47:0/1052705955 conn(0x7f716c0659d0 msgr2=0x7f716c0b6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.297+0000 7f7173ba9640 1 -- 172.21.15.47:0/1052705955 shutdown_connections 2024-03-23T09:43:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.297+0000 7f7173ba9640 1 -- 172.21.15.47:0/1052705955 wait complete. 2024-03-23T09:43:00.305 INFO:tasks.ceph.mds.d.smithi175.stderr:2024-03-23T09:43:00.297+0000 7f336026e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5877=0-4194304@1} caps={5877=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5877 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x561836c16100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T09:43:00.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:00.308 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-03-23T09:43:00.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.411+0000 7f39c5577640 1 Processor -- start 2024-03-23T09:43:00.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.411+0000 7f39c5577640 1 -- start start 2024-03-23T09:43:00.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.412+0000 7f39c5577640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39c015c990 0x7f39c0160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.412+0000 7f39c5577640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f39c00b8350 0x7f39c00cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.412+0000 7f39c5577640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39c00cdc40 0x7f39c00b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.412+0000 7f39c5577640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c00ce0b0 con 0x7f39c00cdc40 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.412+0000 7f39c5577640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c0161630 con 0x7f39c00b8350 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.412+0000 7f39c5577640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c00d1030 con 0x7f39c015c990 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.412+0000 7f39bf7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39c00cdc40 0x7f39c00b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.412+0000 7f39bf7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39c00cdc40 0x7f39c00b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60096/0 (socket says 172.21.15.47:60096) 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.412+0000 7f39bf7fe640 1 -- 172.21.15.47:0/4185064838 learned_addr learned my addr 172.21.15.47:0/4185064838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.413+0000 7f39be7fc640 1 --2- 172.21.15.47:0/4185064838 >> v2:172.21.15.175:3300/0 conn(0x7f39c00b8350 0x7f39c00cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.413+0000 7f39beffd640 1 --2- 172.21.15.47:0/4185064838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39c015c990 0x7f39c0160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.413+0000 7f39bf7fe640 1 -- 172.21.15.47:0/4185064838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39c015c990 msgr2=0x7f39c0160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.413+0000 7f39bf7fe640 1 --2- 172.21.15.47:0/4185064838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39c015c990 0x7f39c0160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.413+0000 7f39bf7fe640 1 -- 172.21.15.47:0/4185064838 >> v2:172.21.15.175:3300/0 conn(0x7f39c00b8350 msgr2=0x7f39c00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.413+0000 7f39bf7fe640 1 --2- 172.21.15.47:0/4185064838 >> v2:172.21.15.175:3300/0 conn(0x7f39c00b8350 0x7f39c00cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.413+0000 7f39bf7fe640 1 -- 172.21.15.47:0/4185064838 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c00cfe10 con 0x7f39c00cdc40 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.413+0000 7f39beffd640 1 --2- 172.21.15.47:0/4185064838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39c015c990 0x7f39c0160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.413+0000 7f39be7fc640 1 --2- 172.21.15.47:0/4185064838 >> v2:172.21.15.175:3300/0 conn(0x7f39c00b8350 0x7f39c00cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.413+0000 7f39bf7fe640 1 --2- 172.21.15.47:0/4185064838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c00cdc40 0x7f39c00b4440 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f39c00b5540 tx=0x7f39b002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97b8499d292f48f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.414+0000 7f39bdffb640 1 -- 172.21.15.47:0/4185064838 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b00317d0 con 0x7f39c00cdc40 2024-03-23T09:43:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.414+0000 7f39bdffb640 1 -- 172.21.15.47:0/4185064838 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b0031930 con 0x7f39c00cdc40 2024-03-23T09:43:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.414+0000 7f39bdffb640 1 -- 172.21.15.47:0/4185064838 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0031cc0 con 0x7f39c00cdc40 2024-03-23T09:43:00.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.414+0000 7f39c5577640 1 -- 172.21.15.47:0/4185064838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c00cdc40 msgr2=0x7f39c00b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.414+0000 7f39c5577640 1 --2- 172.21.15.47:0/4185064838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c00cdc40 0x7f39c00b4440 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f39c00b5540 tx=0x7f39b002b870 comp rx=0 tx=0).stop 2024-03-23T09:43:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.414+0000 7f39c5577640 1 -- 172.21.15.47:0/4185064838 shutdown_connections 2024-03-23T09:43:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.414+0000 7f39c5577640 1 --2- 172.21.15.47:0/4185064838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c00cdc40 0x7f39c00b4440 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.414+0000 7f39c5577640 1 --2- 172.21.15.47:0/4185064838 >> v2:172.21.15.175:3300/0 conn(0x7f39c00b8350 0x7f39c00cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.414+0000 7f39c5577640 1 --2- 172.21.15.47:0/4185064838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39c015c990 0x7f39c0160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.414+0000 7f39c5577640 1 -- 172.21.15.47:0/4185064838 >> 172.21.15.47:0/4185064838 conn(0x7f39c00659d0 msgr2=0x7f39c0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.414+0000 7f39c5577640 1 -- 172.21.15.47:0/4185064838 shutdown_connections 2024-03-23T09:43:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.414+0000 7f39c5577640 1 -- 172.21.15.47:0/4185064838 wait complete. 2024-03-23T09:43:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.416+0000 7f39c5577640 1 Processor -- start 2024-03-23T09:43:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.416+0000 7f39c5577640 1 -- start start 2024-03-23T09:43:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.416+0000 7f39c5577640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39c00b8350 0x7f39c01509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.416+0000 7f39c5577640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c00cdc40 0x7f39c0151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.416+0000 7f39c5577640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39c015c990 0x7f39c01f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.416+0000 7f39c5577640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c01539a0 con 0x7f39c00cdc40 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.416+0000 7f39c5577640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c0160220 con 0x7f39c015c990 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.416+0000 7f39c5577640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c0150f30 con 0x7f39c00b8350 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.416+0000 7f39be7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c00cdc40 0x7f39c0151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.416+0000 7f39be7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c00cdc40 0x7f39c0151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60110/0 (socket says 172.21.15.47:60110) 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.416+0000 7f39be7fc640 1 -- 172.21.15.47:0/123004799 learned_addr learned my addr 172.21.15.47:0/123004799 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.417+0000 7f39beffd640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39c00b8350 0x7f39c01509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.417+0000 7f39bf7fe640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39c015c990 0x7f39c01f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.417+0000 7f39be7fc640 1 -- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39c00b8350 msgr2=0x7f39c01509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.417+0000 7f39be7fc640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39c00b8350 0x7f39c01509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.417+0000 7f39be7fc640 1 -- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39c015c990 msgr2=0x7f39c01f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.417+0000 7f39be7fc640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39c015c990 0x7f39c01f2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.417+0000 7f39be7fc640 1 -- 172.21.15.47:0/123004799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b00094d0 con 0x7f39c00cdc40 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.417+0000 7f39be7fc640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c00cdc40 0x7f39c0151d10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f39a800c7f0 tx=0x7f39a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.417+0000 7f39b77fe640 1 -- 172.21.15.47:0/123004799 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8007820 con 0x7f39c00cdc40 2024-03-23T09:43:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.417+0000 7f39b77fe640 1 -- 172.21.15.47:0/123004799 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a8007980 con 0x7f39c00cdc40 2024-03-23T09:43:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.417+0000 7f39b77fe640 1 -- 172.21.15.47:0/123004799 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8007d30 con 0x7f39c00cdc40 2024-03-23T09:43:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.417+0000 7f39c5577640 1 -- 172.21.15.47:0/123004799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c0152250 con 0x7f39c00cdc40 2024-03-23T09:43:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.417+0000 7f39c5577640 1 -- 172.21.15.47:0/123004799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c00c7980 con 0x7f39c00cdc40 2024-03-23T09:43:00.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.419+0000 7f39b77fe640 1 -- 172.21.15.47:0/123004799 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f39a80113f0 con 0x7f39c00cdc40 2024-03-23T09:43:00.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.419+0000 7f39b77fe640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f39a006ac00 0x7f39a006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.419+0000 7f39b77fe640 1 -- 172.21.15.47:0/123004799 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39a8089460 con 0x7f39c00cdc40 2024-03-23T09:43:00.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.420+0000 7f39beffd640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f39a006ac00 0x7f39a006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.420+0000 7f39beffd640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f39a006ac00 0x7f39a006d0c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39b80075b0 tx=0x7f39b8007540 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.420+0000 7f39c81c7740 1 Processor -- start 2024-03-23T09:43:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.420+0000 7f39c81c7740 1 -- start start 2024-03-23T09:43:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39c81c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 0x557634438c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39c81c7740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557634408850 0x557634435aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39c81c7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55763440c3b0 0x557634435fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39c81c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557634378dd0 con 0x557634438830 2024-03-23T09:43:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39c81c7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557634376560 con 0x55763440c3b0 2024-03-23T09:43:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39c81c7740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557634371e50 con 0x557634408850 2024-03-23T09:43:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39beffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 0x557634438c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39beffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 0x557634438c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60124/0 (socket says 172.21.15.47:60124) 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39beffd640 1 -- 172.21.15.47:0/3968917720 learned_addr learned my addr 172.21.15.47:0/3968917720 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39bf7fe640 1 --2- 172.21.15.47:0/3968917720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55763440c3b0 0x557634435fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39beffd640 1 -- 172.21.15.47:0/3968917720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557634408850 msgr2=0x557634435aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39be7fc640 1 --2- 172.21.15.47:0/3968917720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557634408850 0x557634435aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39beffd640 1 --2- 172.21.15.47:0/3968917720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557634408850 0x557634435aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39beffd640 1 -- 172.21.15.47:0/3968917720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55763440c3b0 msgr2=0x557634435fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39beffd640 1 --2- 172.21.15.47:0/3968917720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55763440c3b0 0x557634435fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39beffd640 1 -- 172.21.15.47:0/3968917720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557634339470 con 0x557634438830 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.421+0000 7f39bf7fe640 1 --2- 172.21.15.47:0/3968917720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55763440c3b0 0x557634435fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.422+0000 7f39beffd640 1 --2- 172.21.15.47:0/3968917720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 0x557634438c10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c700 tx=0x7f39b8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c484bbd0bd58f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.422+0000 7f39b4ff9640 1 -- 172.21.15.47:0/3968917720 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8018430 con 0x557634438830 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.422+0000 7f39b4ff9640 1 -- 172.21.15.47:0/3968917720 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b8018590 con 0x557634438830 2024-03-23T09:43:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.422+0000 7f39b4ff9640 1 -- 172.21.15.47:0/3968917720 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8018900 con 0x557634438830 2024-03-23T09:43:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.422+0000 7f39c81c7740 1 -- 172.21.15.47:0/3968917720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 msgr2=0x557634438c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.422+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3968917720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 0x557634438c10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c700 tx=0x7f39b8011000 comp rx=0 tx=0).stop 2024-03-23T09:43:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.422+0000 7f39c81c7740 1 -- 172.21.15.47:0/3968917720 shutdown_connections 2024-03-23T09:43:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.422+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3968917720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55763440c3b0 0x557634435fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.422+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3968917720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557634408850 0x557634435aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.422+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3968917720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 0x557634438c10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.422+0000 7f39c81c7740 1 -- 172.21.15.47:0/3968917720 >> 172.21.15.47:0/3968917720 conn(0x55763440aac0 msgr2=0x557634453640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.423+0000 7f39c81c7740 1 -- 172.21.15.47:0/3968917720 shutdown_connections 2024-03-23T09:43:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.423+0000 7f39c81c7740 1 -- 172.21.15.47:0/3968917720 wait complete. 2024-03-23T09:43:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.423+0000 7f39c81c7740 1 -- start start 2024-03-23T09:43:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.423+0000 7f39c81c7740 1 Processor -- start 2024-03-23T09:43:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39c81c7740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55763440c3b0 0x557634425d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39c81c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 0x55763443b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39c81c7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634431270 0x55763443b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39c81c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557634330280 con 0x557634438830 2024-03-23T09:43:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39c81c7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55763432ff20 con 0x557634431270 2024-03-23T09:43:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39c81c7740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576342f53c0 con 0x55763440c3b0 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39c81c7740 10 client.0 _ll_register_callbacks cb 0x557634347380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39be7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 0x55763443b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39be7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 0x55763443b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60132/0 (socket says 172.21.15.47:60132) 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39be7fc640 1 -- 172.21.15.47:0/3358139124 learned_addr learned my addr 172.21.15.47:0/3358139124 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39beffd640 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55763440c3b0 0x557634425d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39bf7fe640 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634431270 0x55763443b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39be7fc640 1 -- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55763440c3b0 msgr2=0x557634425d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39be7fc640 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55763440c3b0 0x557634425d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39be7fc640 1 -- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634431270 msgr2=0x55763443b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39be7fc640 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634431270 0x55763443b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39be7fc640 1 -- 172.21.15.47:0/3358139124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b8014050 con 0x557634438830 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39beffd640 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55763440c3b0 0x557634425d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.424+0000 7f39bf7fe640 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634431270 0x55763443b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f39be7fc640 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 0x55763443b480 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f39a800c7c0 tx=0x7f39a8057cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c9593abee07946 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f399dffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f399dffb640 1 -- 172.21.15.47:0/3358139124 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a80603e0 con 0x557634438830 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f39c81c7740 1 -- 172.21.15.47:0/3358139124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55763439b690 con 0x557634438830 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f399dffb640 1 -- 172.21.15.47:0/3358139124 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39a8060670 con 0x557634438830 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f399dffb640 1 -- 172.21.15.47:0/3358139124 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39a80892e0 con 0x557634438830 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f39c81c7740 10 client.5937 fetch_fsmap learned FSMap version 18 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f39c81c7740 1 -- 172.21.15.47:0/3358139124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5576343704d0 con 0x557634438830 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f399dffb640 1 -- 172.21.15.47:0/3358139124 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55763439b690 con 0x557634438830 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f399dffb640 1 -- 172.21.15.47:0/3358139124 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 18) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f39a805e0c0 con 0x557634438830 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f39c81c7740 10 client.5937 fetch_fsmap finished waiting for FSMap version 18 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f39c81c7740 10 client.5937 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f39c81c7740 20 client.5937 populate_metadata read hostname 'smithi047' 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x557634466b10 0x557634466ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f39c81c7740 4 client.5937 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f39c81c7740 1 -- 172.21.15.47:0/3358139124 --> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557634363eb0 con 0x557634466b10 2024-03-23T09:43:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.425+0000 7f39beffd640 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x557634466b10 0x557634466ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.426+0000 7f399dffb640 10 client.5937 ms_handle_connect on v2:172.21.15.175:6836/1626348200 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.426+0000 7f39beffd640 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x557634466b10 0x557634466ef0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=319c4f5fd3cdc7f0 server_cookie=4497c716fec19e9b in_seq=0 out_seq=0 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.426+0000 7f399dffb640 1 -- 172.21.15.47:0/3358139124 <== mds.0 v2:172.21.15.175:6836/1626348200 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557634363eb0 con 0x557634466b10 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.426+0000 7f399dffb640 10 client.5937 handle_command_reply: tid=0 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.426+0000 7f39c81c7740 1 client.5937 shutdown 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.426+0000 7f39c81c7740 1 -- 172.21.15.47:0/3358139124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 msgr2=0x55763443b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.426+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 0x55763443b480 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f39a800c7c0 tx=0x7f39a8057cd0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.427+0000 7f39c81c7740 1 -- 172.21.15.47:0/3358139124 shutdown_connections 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.427+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x557634466b10 0x557634466ef0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.427+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634431270 0x55763443b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.427+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634438830 0x55763443b480 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.427+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3358139124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55763440c3b0 0x557634425d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.427+0000 7f39c81c7740 1 -- 172.21.15.47:0/3358139124 >> 172.21.15.47:0/3358139124 conn(0x557634408850 msgr2=0x55763440cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.427+0000 7f399dffb640 0 client.5937 ms_handle_reset on v2:172.21.15.175:6836/1626348200 2024-03-23T09:43:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.427+0000 7f39c81c7740 1 -- 172.21.15.47:0/3358139124 shutdown_connections 2024-03-23T09:43:00.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.427+0000 7f39c81c7740 1 -- 172.21.15.47:0/3358139124 wait complete. 2024-03-23T09:43:00.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.427+0000 7f39c81c7740 20 client.5937 trim_cache size 0 max 16384 2024-03-23T09:43:00.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.441+0000 7f39c81c7740 1 Processor -- start 2024-03-23T09:43:00.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.441+0000 7f39c81c7740 1 -- start start 2024-03-23T09:43:00.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39c81c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55763440c3b0 0x557634408180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39c81c7740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557634431270 0x55763440e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39c81c7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634438830 0x55763440c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39c81c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557634456b50 con 0x55763440c3b0 2024-03-23T09:43:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39c81c7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55763443cbc0 con 0x557634438830 2024-03-23T09:43:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39c81c7740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576342fdfb0 con 0x557634431270 2024-03-23T09:43:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39beffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55763440c3b0 0x557634408180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39beffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55763440c3b0 0x557634408180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60136/0 (socket says 172.21.15.47:60136) 2024-03-23T09:43:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39beffd640 1 -- 172.21.15.47:0/3647506938 learned_addr learned my addr 172.21.15.47:0/3647506938 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39be7fc640 1 --2- 172.21.15.47:0/3647506938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557634431270 0x55763440e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39bf7fe640 1 --2- 172.21.15.47:0/3647506938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634438830 0x55763440c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39beffd640 1 -- 172.21.15.47:0/3647506938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557634431270 msgr2=0x55763440e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39beffd640 1 --2- 172.21.15.47:0/3647506938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557634431270 0x55763440e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39beffd640 1 -- 172.21.15.47:0/3647506938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634438830 msgr2=0x55763440c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39beffd640 1 --2- 172.21.15.47:0/3647506938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634438830 0x55763440c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39beffd640 1 -- 172.21.15.47:0/3647506938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576343a49e0 con 0x55763440c3b0 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39be7fc640 1 --2- 172.21.15.47:0/3647506938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557634431270 0x55763440e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39bf7fe640 1 --2- 172.21.15.47:0/3647506938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634438830 0x55763440c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39beffd640 1 --2- 172.21.15.47:0/3647506938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55763440c3b0 0x557634408180 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f39b8025040 tx=0x7f39b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b153f613b933df38 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39b57fa640 1 -- 172.21.15.47:0/3647506938 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8038070 con 0x55763440c3b0 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39b57fa640 1 -- 172.21.15.47:0/3647506938 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b8032070 con 0x55763440c3b0 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.442+0000 7f39b57fa640 1 -- 172.21.15.47:0/3647506938 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8018430 con 0x55763440c3b0 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.443+0000 7f39c81c7740 1 -- 172.21.15.47:0/3647506938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55763440c3b0 msgr2=0x557634408180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.443+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3647506938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55763440c3b0 0x557634408180 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f39b8025040 tx=0x7f39b8016000 comp rx=0 tx=0).stop 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.443+0000 7f39c81c7740 1 -- 172.21.15.47:0/3647506938 shutdown_connections 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.443+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3647506938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634438830 0x55763440c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.443+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3647506938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557634431270 0x55763440e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.443+0000 7f39c81c7740 1 --2- 172.21.15.47:0/3647506938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55763440c3b0 0x557634408180 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.443+0000 7f39c81c7740 1 -- 172.21.15.47:0/3647506938 >> 172.21.15.47:0/3647506938 conn(0x557634408850 msgr2=0x55763443d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.443+0000 7f39c81c7740 1 -- 172.21.15.47:0/3647506938 shutdown_connections 2024-03-23T09:43:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.443+0000 7f39c81c7740 1 -- 172.21.15.47:0/3647506938 wait complete. 2024-03-23T09:43:00.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.443+0000 7f39c81c7740 1 -- start start 2024-03-23T09:43:00.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.443+0000 7f39c81c7740 1 Processor -- start 2024-03-23T09:43:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.444+0000 7f39c81c7740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55763440c3b0 0x5576344040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.444+0000 7f39c81c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634431270 0x55763443bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.444+0000 7f39c81c7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634438830 0x55763443c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.444+0000 7f39c81c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55763440cdb0 con 0x557634431270 2024-03-23T09:43:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.444+0000 7f39c81c7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55763440d090 con 0x557634438830 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.444+0000 7f39c81c7740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576344710f0 con 0x55763440c3b0 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.444+0000 7f39c81c7740 10 client.0 _ll_register_callbacks cb 0x7f39b80077e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.444+0000 7f39be7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634431270 0x55763443bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.444+0000 7f39be7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634431270 0x55763443bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60150/0 (socket says 172.21.15.47:60150) 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.444+0000 7f39be7fc640 1 -- 172.21.15.47:0/1159761142 learned_addr learned my addr 172.21.15.47:0/1159761142 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.444+0000 7f39beffd640 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55763440c3b0 0x5576344040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.444+0000 7f39bf7fe640 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634438830 0x55763443c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39be7fc640 1 -- 172.21.15.47:0/1159761142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55763440c3b0 msgr2=0x5576344040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39be7fc640 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55763440c3b0 0x5576344040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39be7fc640 1 -- 172.21.15.47:0/1159761142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634438830 msgr2=0x55763443c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39be7fc640 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634438830 0x55763443c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39be7fc640 1 -- 172.21.15.47:0/1159761142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557634339470 con 0x557634431270 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39bf7fe640 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634438830 0x55763443c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39be7fc640 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634431270 0x55763443bff0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f39a8060140 tx=0x7f39a8087860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd56a167afa60188 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f399dffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f399dffb640 1 -- 172.21.15.47:0/1159761142 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8007e90 con 0x557634431270 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f399dffb640 1 -- 172.21.15.47:0/1159761142 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39a8060670 con 0x557634431270 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39c81c7740 1 -- 172.21.15.47:0/1159761142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39b8018590 con 0x557634431270 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f399dffb640 1 -- 172.21.15.47:0/1159761142 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39a80892e0 con 0x557634431270 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39c81c7740 10 client.5943 fetch_fsmap learned FSMap version 18 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39c81c7740 1 -- 172.21.15.47:0/1159761142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5576343125c0 con 0x557634431270 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f399dffb640 1 -- 172.21.15.47:0/1159761142 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b8018590 con 0x557634431270 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f399dffb640 1 -- 172.21.15.47:0/1159761142 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 18) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f39a80813f0 con 0x557634431270 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39c81c7740 10 client.5943 fetch_fsmap finished waiting for FSMap version 18 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39c81c7740 10 client.5943 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39c81c7740 20 client.5943 populate_metadata read hostname 'smithi047' 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39c81c7740 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x55763440de70 0x55763440e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39c81c7740 4 client.5943 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39c81c7740 1 -- 172.21.15.47:0/1159761142 --> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5576340654e0 con 0x55763440de70 2024-03-23T09:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.445+0000 7f39beffd640 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x55763440de70 0x55763440e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f399dffb640 10 client.5943 ms_handle_connect on v2:172.21.15.175:6836/1626348200 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f39beffd640 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x55763440de70 0x55763440e280 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=655c36347c6638e4 server_cookie=f8ba3fe0b48a6b4f in_seq=0 out_seq=0 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f399dffb640 1 -- 172.21.15.47:0/1159761142 <== mds.0 v2:172.21.15.175:6836/1626348200 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x5576340654e0 con 0x55763440de70 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f399dffb640 10 client.5943 handle_command_reply: tid=0 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f39c81c7740 1 client.5943 shutdown 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f39c81c7740 1 -- 172.21.15.47:0/1159761142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634431270 msgr2=0x55763443bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f39c81c7740 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634431270 0x55763443bff0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f39a8060140 tx=0x7f39a8087860 comp rx=0 tx=0).stop 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f39c81c7740 1 -- 172.21.15.47:0/1159761142 shutdown_connections 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f39c81c7740 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x55763440de70 0x55763440e280 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f39c81c7740 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557634438830 0x55763443c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f399dffb640 0 client.5943 ms_handle_reset on v2:172.21.15.175:6836/1626348200 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f39c81c7740 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557634431270 0x55763443bff0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f39c81c7740 1 --2- 172.21.15.47:0/1159761142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55763440c3b0 0x5576344040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f39c81c7740 1 -- 172.21.15.47:0/1159761142 >> 172.21.15.47:0/1159761142 conn(0x557634408850 msgr2=0x557634431860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f39c81c7740 1 -- 172.21.15.47:0/1159761142 shutdown_connections 2024-03-23T09:43:00.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f39c81c7740 1 -- 172.21.15.47:0/1159761142 wait complete. 2024-03-23T09:43:00.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.446+0000 7f39c81c7740 20 client.5943 trim_cache size 0 max 16384 2024-03-23T09:43:00.453 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:43:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T09:43:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-23T09:43:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "fcc4f0f2-1b33-4f47-9cfe-02003121910a": { 2024-03-23T09:43:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-23T09:43:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "fcc4f0f2-1b33-4f47-9cfe-02003121910a", 2024-03-23T09:43:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,repair" 2024-03-23T09:43:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:00.453 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:43:00.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.447+0000 7f39c5577640 1 -- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f39a006ac00 msgr2=0x7f39a006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.447+0000 7f39c5577640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f39a006ac00 0x7f39a006d0c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39b80075b0 tx=0x7f39b8007540 comp rx=0 tx=0).stop 2024-03-23T09:43:00.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.447+0000 7f39c5577640 1 -- 172.21.15.47:0/123004799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c00cdc40 msgr2=0x7f39c0151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.447+0000 7f39c5577640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c00cdc40 0x7f39c0151d10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f39a800c7f0 tx=0x7f39a800ccb0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.447+0000 7f39c5577640 1 -- 172.21.15.47:0/123004799 shutdown_connections 2024-03-23T09:43:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.447+0000 7f39c5577640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f39a006ac00 0x7f39a006d0c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.447+0000 7f39c5577640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39c015c990 0x7f39c01f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.447+0000 7f39c5577640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c00cdc40 0x7f39c0151d10 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.447+0000 7f39c5577640 1 --2- 172.21.15.47:0/123004799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39c00b8350 0x7f39c01509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.447+0000 7f39c5577640 1 -- 172.21.15.47:0/123004799 >> 172.21.15.47:0/123004799 conn(0x7f39c00659d0 msgr2=0x7f39c00d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.447+0000 7f39c5577640 1 -- 172.21.15.47:0/123004799 shutdown_connections 2024-03-23T09:43:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.447+0000 7f39c5577640 1 -- 172.21.15.47:0/123004799 wait complete. 2024-03-23T09:43:00.457 INFO:tasks.cephfs.filesystem:scrub status for tag:fcc4f0f2-1b33-4f47-9cfe-02003121910a - {'path': '/', 'tag': 'fcc4f0f2-1b33-4f47-9cfe-02003121910a', 'options': 'recursive,repair'} 2024-03-23T09:43:00.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:00.789+0000 7f94c77fe640 1 -- 172.21.15.47:0/2497233436 <== mon.0 v2:172.21.15.47:3300/0 7 ==== log(6 entries from seq 1 at 2024-03-23T09:43:00.297938+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f94d0058d10 con 0x7f94e00d0710 2024-03-23T09:43:00.789 INFO:teuthology.orchestra.run.smithi047.stdout: cluster: 2024-03-23T09:43:00.789 INFO:teuthology.orchestra.run.smithi047.stdout: id: f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:43:00.789 INFO:teuthology.orchestra.run.smithi047.stdout: health: HEALTH_OK 2024-03-23T09:43:00.789 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:00.789 INFO:teuthology.orchestra.run.smithi047.stdout: services: 2024-03-23T09:43:00.789 INFO:teuthology.orchestra.run.smithi047.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-03-23T09:43:00.789 INFO:teuthology.orchestra.run.smithi047.stdout: mgr: x(active, since 2m), standbys: y 2024-03-23T09:43:00.789 INFO:teuthology.orchestra.run.smithi047.stdout: mds: 1/1 daemons up, 3 standby 2024-03-23T09:43:00.789 INFO:teuthology.orchestra.run.smithi047.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-03-23T09:43:00.789 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout: data: 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout: volumes: 1/1 healthy 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout: pools: 3 pools, 97 pgs 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout: objects: 24 objects, 579 KiB 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout: usage: 232 MiB used, 720 GiB / 720 GiB avail 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout: pgs: 97 active+clean 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout: io: 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout: client: 1.4 KiB/s wr, 0 op/s rd, 4 op/s wr 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:43:00.297938+0000 mds.d [INF] scrub queued for path: / 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:43:00.297941+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-03-23T09:43:00.790 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:43:00.297946+0000 mds.d [INF] scrub summary: active paths [/] 2024-03-23T09:43:00.791 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:43:00.298579+0000 mds.d [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-03-23T09:43:00.791 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:43:00.298589+0000 mds.d [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-03-23T09:43:00.791 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:43:00.299083+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-03-23T09:43:04.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:04.799+0000 7f94c77fe640 1 -- 172.21.15.47:0/2497233436 <== mon.0 v2:172.21.15.47:3300/0 8 ==== log(1 entries from seq 7 at 2024-03-23T09:43:03.391033+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f94d0058610 con 0x7f94e00d0710 2024-03-23T09:43:04.800 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:43:03.391033+0000 mds.d [INF] scrub summary: idle 2024-03-23T09:43:07.241 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:43:07.244 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:43:30.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:30.459 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-03-23T09:43:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.573+0000 7fd2086e9640 1 Processor -- start 2024-03-23T09:43:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.573+0000 7fd2086e9640 1 -- start start 2024-03-23T09:43:30.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.574+0000 7fd2086e9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd2000cad40 0x7fd2000b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.574+0000 7fd2086e9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd20015d010 con 0x7fd20015cc30 2024-03-23T09:43:30.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.574+0000 7fd2086e9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2000b13d0 con 0x7fd2000cd680 2024-03-23T09:43:30.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.574+0000 7fd2086e9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2000b16b0 con 0x7fd2000cad40 2024-03-23T09:43:30.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.574+0000 7fd206c86640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd2000cad40 0x7fd2000b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.575+0000 7fd206485640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd20015cc30 0x7fd2000ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45874/0 (socket says 172.21.15.47:45874) 2024-03-23T09:43:30.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.575+0000 7fd206c86640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd2000cad40 0x7fd2000b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50168/0 (socket says 172.21.15.47:50168) 2024-03-23T09:43:30.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.575+0000 7fd206485640 1 -- 172.21.15.47:0/432490093 learned_addr learned my addr 172.21.15.47:0/432490093 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:30.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.575+0000 7fd206c86640 1 -- 172.21.15.47:0/432490093 >> v1:172.21.15.175:6789/0 conn(0x7fd2000cd680 legacy=0x7fd2000b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:30.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.575+0000 7fd206c86640 1 -- 172.21.15.47:0/432490093 >> v1:172.21.15.47:6789/0 conn(0x7fd20015cc30 legacy=0x7fd2000ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:30.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.575+0000 7fd206c86640 1 -- 172.21.15.47:0/432490093 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2000cedb0 con 0x7fd2000cad40 2024-03-23T09:43:30.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.576+0000 7fd206c86640 1 --2- 172.21.15.47:0/432490093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2000cad40 0x7fd2000b0d20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009d90 tx=0x7fd1f002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1b572e29846bc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:30.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.576+0000 7fd205483640 1 -- 172.21.15.47:0/432490093 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f002cbd0 con 0x7fd2000cad40 2024-03-23T09:43:30.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.576+0000 7fd205483640 1 -- 172.21.15.47:0/432490093 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f002cd30 con 0x7fd2000cad40 2024-03-23T09:43:30.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.576+0000 7fd205483640 1 -- 172.21.15.47:0/432490093 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f002cea0 con 0x7fd2000cad40 2024-03-23T09:43:30.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.576+0000 7fd2086e9640 1 -- 172.21.15.47:0/432490093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2000cad40 msgr2=0x7fd2000b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:30.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.576+0000 7fd2086e9640 1 --2- 172.21.15.47:0/432490093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2000cad40 0x7fd2000b0d20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009d90 tx=0x7fd1f002be20 comp rx=0 tx=0).stop 2024-03-23T09:43:30.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.576+0000 7fd2086e9640 1 -- 172.21.15.47:0/432490093 shutdown_connections 2024-03-23T09:43:30.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.576+0000 7fd2086e9640 1 --2- 172.21.15.47:0/432490093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2000cad40 0x7fd2000b0d20 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.576+0000 7fd2086e9640 1 -- 172.21.15.47:0/432490093 >> 172.21.15.47:0/432490093 conn(0x7fd2000d06f0 msgr2=0x7fd2000d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:30.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.576+0000 7fd2086e9640 1 -- 172.21.15.47:0/432490093 shutdown_connections 2024-03-23T09:43:30.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.576+0000 7fd2086e9640 1 -- 172.21.15.47:0/432490093 wait complete. 2024-03-23T09:43:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.578+0000 7fd2086e9640 1 Processor -- start 2024-03-23T09:43:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.578+0000 7fd2086e9640 1 -- start start 2024-03-23T09:43:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.578+0000 7fd2086e9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2000cad40 0x7fd200153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.578+0000 7fd2086e9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2000cd680 0x7fd2000c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.578+0000 7fd2086e9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd20015cc30 0x7fd2001f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.578+0000 7fd2086e9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000c6bc0 con 0x7fd20015cc30 2024-03-23T09:43:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.578+0000 7fd2086e9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd200157d50 con 0x7fd2000cd680 2024-03-23T09:43:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.578+0000 7fd2086e9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd200158030 con 0x7fd2000cad40 2024-03-23T09:43:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.578+0000 7fd206c86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd20015cc30 0x7fd2001f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.578+0000 7fd206c86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd20015cc30 0x7fd2001f2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50172/0 (socket says 172.21.15.47:50172) 2024-03-23T09:43:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.578+0000 7fd206c86640 1 -- 172.21.15.47:0/4273938443 learned_addr learned my addr 172.21.15.47:0/4273938443 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.579+0000 7fd206485640 1 --2- 172.21.15.47:0/4273938443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2000cad40 0x7fd200153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.579+0000 7fd206c86640 1 -- 172.21.15.47:0/4273938443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2000cad40 msgr2=0x7fd200153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.579+0000 7fd206c86640 1 --2- 172.21.15.47:0/4273938443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2000cad40 0x7fd200153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.579+0000 7fd206c86640 1 -- 172.21.15.47:0/4273938443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2000cd680 msgr2=0x7fd2000c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.579+0000 7fd206c86640 1 --2- 172.21.15.47:0/4273938443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2000cd680 0x7fd2000c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.579+0000 7fd206c86640 1 -- 172.21.15.47:0/4273938443 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f0009a70 con 0x7fd20015cc30 2024-03-23T09:43:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.579+0000 7fd206c86640 1 --2- 172.21.15.47:0/4273938443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd20015cc30 0x7fd2001f2360 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009d90 tx=0x7fd1f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.579+0000 7fd1feffd640 1 -- 172.21.15.47:0/4273938443 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f002cea0 con 0x7fd20015cc30 2024-03-23T09:43:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.579+0000 7fd1feffd640 1 -- 172.21.15.47:0/4273938443 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f003e630 con 0x7fd20015cc30 2024-03-23T09:43:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.579+0000 7fd2086e9640 1 -- 172.21.15.47:0/4273938443 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd200158310 con 0x7fd20015cc30 2024-03-23T09:43:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.579+0000 7fd1feffd640 1 -- 172.21.15.47:0/4273938443 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f003e9e0 con 0x7fd20015cc30 2024-03-23T09:43:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.579+0000 7fd2086e9640 1 -- 172.21.15.47:0/4273938443 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2000c5c80 con 0x7fd20015cc30 2024-03-23T09:43:30.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.581+0000 7fd1feffd640 1 -- 172.21.15.47:0/4273938443 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd1f003b050 con 0x7fd20015cc30 2024-03-23T09:43:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.581+0000 7fd1feffd640 1 --2- 172.21.15.47:0/4273938443 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd1ec06ac00 0x7fd1ec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.581+0000 7fd1feffd640 1 -- 172.21.15.47:0/4273938443 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1f00abc90 con 0x7fd20015cc30 2024-03-23T09:43:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.581+0000 7fd206485640 1 --2- 172.21.15.47:0/4273938443 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd1ec06ac00 0x7fd1ec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.582+0000 7fd206485640 1 --2- 172.21.15.47:0/4273938443 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd1ec06ac00 0x7fd1ec06d0c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8005880 tx=0x7fd1f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.582+0000 7fd20b264740 1 Processor -- start 2024-03-23T09:43:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.582+0000 7fd20b264740 1 -- start start 2024-03-23T09:43:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.583+0000 7fd20b264740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b314ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.583+0000 7fd20b264740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b311e850 0x55e0b314baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.583+0000 7fd20b264740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b314bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.583+0000 7fd20b264740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b308edd0 con 0x55e0b311e850 2024-03-23T09:43:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.583+0000 7fd20b264740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b308c560 con 0x55e0b31223b0 2024-03-23T09:43:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.583+0000 7fd20b264740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3087e50 con 0x55e0b314e830 2024-03-23T09:43:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.583+0000 7fd205c84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b311e850 0x55e0b314baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.583+0000 7fd205c84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b311e850 0x55e0b314baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50184/0 (socket says 172.21.15.47:50184) 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.583+0000 7fd205c84640 1 -- 172.21.15.47:0/2023192173 learned_addr learned my addr 172.21.15.47:0/2023192173 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.583+0000 7fd206485640 1 --2- 172.21.15.47:0/2023192173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b314ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.583+0000 7fd206c86640 1 --2- 172.21.15.47:0/2023192173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b314bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.584+0000 7fd205c84640 1 -- 172.21.15.47:0/2023192173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 msgr2=0x55e0b314ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.584+0000 7fd205c84640 1 --2- 172.21.15.47:0/2023192173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b314ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.584+0000 7fd205c84640 1 -- 172.21.15.47:0/2023192173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 msgr2=0x55e0b314bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.584+0000 7fd205c84640 1 --2- 172.21.15.47:0/2023192173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b314bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.584+0000 7fd205c84640 1 -- 172.21.15.47:0/2023192173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0b304f470 con 0x55e0b311e850 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.584+0000 7fd206c86640 1 --2- 172.21.15.47:0/2023192173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b314bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.584+0000 7fd205c84640 1 --2- 172.21.15.47:0/2023192173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b311e850 0x55e0b314baa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd1f40015a0 tx=0x7fd1f4001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda5d36a6d221021 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.584+0000 7fd1e7fff640 1 -- 172.21.15.47:0/2023192173 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f40052d0 con 0x55e0b311e850 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.584+0000 7fd1e7fff640 1 -- 172.21.15.47:0/2023192173 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f4005430 con 0x55e0b311e850 2024-03-23T09:43:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.584+0000 7fd1e7fff640 1 -- 172.21.15.47:0/2023192173 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400c8e0 con 0x55e0b311e850 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.585+0000 7fd20b264740 1 -- 172.21.15.47:0/2023192173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b311e850 msgr2=0x55e0b314baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.585+0000 7fd20b264740 1 --2- 172.21.15.47:0/2023192173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b311e850 0x55e0b314baa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd1f40015a0 tx=0x7fd1f4001a60 comp rx=0 tx=0).stop 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.585+0000 7fd20b264740 1 -- 172.21.15.47:0/2023192173 shutdown_connections 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.585+0000 7fd20b264740 1 --2- 172.21.15.47:0/2023192173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b314bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.585+0000 7fd20b264740 1 --2- 172.21.15.47:0/2023192173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b311e850 0x55e0b314baa0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.585+0000 7fd20b264740 1 --2- 172.21.15.47:0/2023192173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b314ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.585+0000 7fd20b264740 1 -- 172.21.15.47:0/2023192173 >> 172.21.15.47:0/2023192173 conn(0x55e0b3120ac0 msgr2=0x55e0b3169640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.585+0000 7fd20b264740 1 -- 172.21.15.47:0/2023192173 shutdown_connections 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.585+0000 7fd20b264740 1 -- 172.21.15.47:0/2023192173 wait complete. 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.585+0000 7fd20b264740 1 -- start start 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.585+0000 7fd20b264740 1 Processor -- start 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.586+0000 7fd20b264740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b3149050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.586+0000 7fd20b264740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b3147880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.586+0000 7fd20b264740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b3150a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.586+0000 7fd20b264740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3046280 con 0x55e0b31505e0 2024-03-23T09:43:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.586+0000 7fd20b264740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3045f20 con 0x55e0b31223b0 2024-03-23T09:43:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.586+0000 7fd20b264740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0b300b3c0 con 0x55e0b314e830 2024-03-23T09:43:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.586+0000 7fd20b264740 10 client.0 _ll_register_callbacks cb 0x55e0b305d380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:43:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.586+0000 7fd206c86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b3150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd206c86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b3150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50186/0 (socket says 172.21.15.47:50186) 2024-03-23T09:43:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd206c86640 1 -- 172.21.15.47:0/447755658 learned_addr learned my addr 172.21.15.47:0/447755658 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd205c84640 1 --2- 172.21.15.47:0/447755658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b3147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd206485640 1 --2- 172.21.15.47:0/447755658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b3149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd206c86640 1 -- 172.21.15.47:0/447755658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 msgr2=0x55e0b3147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd206c86640 1 --2- 172.21.15.47:0/447755658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b3147880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd206c86640 1 -- 172.21.15.47:0/447755658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 msgr2=0x55e0b3149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd206c86640 1 --2- 172.21.15.47:0/447755658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b3149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd206c86640 1 -- 172.21.15.47:0/447755658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f40011d0 con 0x55e0b31505e0 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd206c86640 1 --2- 172.21.15.47:0/447755658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b3150a40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd1f007aa10 tx=0x7fd1f007ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b260d04f1a9767ff server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd1e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd1e57fa640 1 -- 172.21.15.47:0/447755658 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f007a400 con 0x55e0b31505e0 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd1e57fa640 1 -- 172.21.15.47:0/447755658 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1f0082c70 con 0x55e0b31505e0 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd20b264740 1 -- 172.21.15.47:0/447755658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0b30b1690 con 0x55e0b31505e0 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd1e57fa640 1 -- 172.21.15.47:0/447755658 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1f0082e30 con 0x55e0b31505e0 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd20b264740 10 client.5958 fetch_fsmap learned FSMap version 18 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd20b264740 1 -- 172.21.15.47:0/447755658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55e0b30864d0 con 0x55e0b31505e0 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.587+0000 7fd1e57fa640 1 -- 172.21.15.47:0/447755658 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e0b30b1690 con 0x55e0b31505e0 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.588+0000 7fd1e57fa640 1 -- 172.21.15.47:0/447755658 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 18) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd1f00a9020 con 0x55e0b31505e0 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.588+0000 7fd20b264740 10 client.5958 fetch_fsmap finished waiting for FSMap version 18 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.588+0000 7fd20b264740 10 client.5958 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-03-23T09:43:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.588+0000 7fd20b264740 20 client.5958 populate_metadata read hostname 'smithi047' 2024-03-23T09:43:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.588+0000 7fd20b264740 1 --2- 172.21.15.47:0/447755658 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x55e0b31838c0 0x55e0b3185cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.588+0000 7fd20b264740 4 client.5958 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:43:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.588+0000 7fd20b264740 1 -- 172.21.15.47:0/447755658 --> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0b3079eb0 con 0x55e0b31838c0 2024-03-23T09:43:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.588+0000 7fd206485640 1 --2- 172.21.15.47:0/447755658 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x55e0b31838c0 0x55e0b3185cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.588+0000 7fd1e57fa640 10 client.5958 ms_handle_connect on v2:172.21.15.175:6836/1626348200 2024-03-23T09:43:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.588+0000 7fd206485640 1 --2- 172.21.15.47:0/447755658 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x55e0b31838c0 0x55e0b3185cb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b614199eeaee8ceb server_cookie=528522757b1f71b5 in_seq=0 out_seq=0 2024-03-23T09:43:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd1e57fa640 1 -- 172.21.15.47:0/447755658 <== mds.0 v2:172.21.15.175:6836/1626348200 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e0b3079eb0 con 0x55e0b31838c0 2024-03-23T09:43:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd1e57fa640 10 client.5958 handle_command_reply: tid=0 2024-03-23T09:43:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd20b264740 1 client.5958 shutdown 2024-03-23T09:43:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd20b264740 1 -- 172.21.15.47:0/447755658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 msgr2=0x55e0b3150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd20b264740 1 --2- 172.21.15.47:0/447755658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b3150a40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd1f007aa10 tx=0x7fd1f007ae00 comp rx=0 tx=0).stop 2024-03-23T09:43:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd20b264740 1 -- 172.21.15.47:0/447755658 shutdown_connections 2024-03-23T09:43:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd20b264740 1 --2- 172.21.15.47:0/447755658 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x55e0b31838c0 0x55e0b3185cb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd20b264740 1 --2- 172.21.15.47:0/447755658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b3150a40 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd20b264740 1 --2- 172.21.15.47:0/447755658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b3147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd1e57fa640 0 client.5958 ms_handle_reset on v2:172.21.15.175:6836/1626348200 2024-03-23T09:43:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd20b264740 1 --2- 172.21.15.47:0/447755658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b3149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd20b264740 1 -- 172.21.15.47:0/447755658 >> 172.21.15.47:0/447755658 conn(0x55e0b311e850 msgr2=0x55e0b3152550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd20b264740 1 -- 172.21.15.47:0/447755658 shutdown_connections 2024-03-23T09:43:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd20b264740 1 -- 172.21.15.47:0/447755658 wait complete. 2024-03-23T09:43:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.589+0000 7fd20b264740 20 client.5958 trim_cache size 0 max 16384 2024-03-23T09:43:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.604+0000 7fd20b264740 1 Processor -- start 2024-03-23T09:43:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.604+0000 7fd20b264740 1 -- start start 2024-03-23T09:43:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.604+0000 7fd20b264740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b3169730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd20b264740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b31434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd20b264740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b3149a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd20b264740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3125f00 con 0x55e0b31505e0 2024-03-23T09:43:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd20b264740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3086ff0 con 0x55e0b31223b0 2024-03-23T09:43:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd20b264740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0b316d3e0 con 0x55e0b314e830 2024-03-23T09:43:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd206c86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b3149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd206c86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b3149a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50194/0 (socket says 172.21.15.47:50194) 2024-03-23T09:43:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd206c86640 1 -- 172.21.15.47:0/1332847914 learned_addr learned my addr 172.21.15.47:0/1332847914 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd206485640 1 --2- 172.21.15.47:0/1332847914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b3169730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd206c86640 1 -- 172.21.15.47:0/1332847914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 msgr2=0x55e0b31434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd205c84640 1 --2- 172.21.15.47:0/1332847914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b31434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd206c86640 1 --2- 172.21.15.47:0/1332847914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b31434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd206c86640 1 -- 172.21.15.47:0/1332847914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 msgr2=0x55e0b3169730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd206c86640 1 --2- 172.21.15.47:0/1332847914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b3169730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd206c86640 1 -- 172.21.15.47:0/1332847914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0b30ba9e0 con 0x55e0b31505e0 2024-03-23T09:43:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd206485640 1 --2- 172.21.15.47:0/1332847914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b3169730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd205c84640 1 --2- 172.21.15.47:0/1332847914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b31434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd206c86640 1 --2- 172.21.15.47:0/1332847914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b3149a30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd1f003eb60 tx=0x7fd1f007c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0a29d45a5989f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd1fcff9640 1 -- 172.21.15.47:0/1332847914 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0076eb0 con 0x55e0b31505e0 2024-03-23T09:43:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd1fcff9640 1 -- 172.21.15.47:0/1332847914 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f007aae0 con 0x55e0b31505e0 2024-03-23T09:43:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.605+0000 7fd1fcff9640 1 -- 172.21.15.47:0/1332847914 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f003ebe0 con 0x55e0b31505e0 2024-03-23T09:43:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.606+0000 7fd20b264740 1 -- 172.21.15.47:0/1332847914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 msgr2=0x55e0b3149a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.606+0000 7fd20b264740 1 --2- 172.21.15.47:0/1332847914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b3149a30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd1f003eb60 tx=0x7fd1f007c000 comp rx=0 tx=0).stop 2024-03-23T09:43:30.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.606+0000 7fd20b264740 1 -- 172.21.15.47:0/1332847914 shutdown_connections 2024-03-23T09:43:30.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.606+0000 7fd20b264740 1 --2- 172.21.15.47:0/1332847914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b3149a30 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.606+0000 7fd20b264740 1 --2- 172.21.15.47:0/1332847914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b31434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.606+0000 7fd20b264740 1 --2- 172.21.15.47:0/1332847914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b3169730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.606+0000 7fd20b264740 1 -- 172.21.15.47:0/1332847914 >> 172.21.15.47:0/1332847914 conn(0x55e0b311e850 msgr2=0x55e0b3169020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:30.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.606+0000 7fd20b264740 1 -- 172.21.15.47:0/1332847914 shutdown_connections 2024-03-23T09:43:30.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.606+0000 7fd20b264740 1 -- 172.21.15.47:0/1332847914 wait complete. 2024-03-23T09:43:30.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.606+0000 7fd20b264740 1 -- start start 2024-03-23T09:43:30.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.606+0000 7fd20b264740 1 Processor -- start 2024-03-23T09:43:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.607+0000 7fd20b264740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b3151310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.607+0000 7fd20b264740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b3151850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.607+0000 7fd20b264740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b316ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.607+0000 7fd20b264740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b316bc30 con 0x55e0b31505e0 2024-03-23T09:43:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.607+0000 7fd20b264740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b31ca070 con 0x55e0b31223b0 2024-03-23T09:43:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.607+0000 7fd20b264740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3146ac0 con 0x55e0b314e830 2024-03-23T09:43:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.607+0000 7fd20b264740 10 client.0 _ll_register_callbacks cb 0x7fd1f4007c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:43:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.607+0000 7fd206c86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b316ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.607+0000 7fd206c86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b316ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50202/0 (socket says 172.21.15.47:50202) 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd206c86640 1 -- 172.21.15.47:0/2589243985 learned_addr learned my addr 172.21.15.47:0/2589243985 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd206c86640 1 -- 172.21.15.47:0/2589243985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 msgr2=0x55e0b3151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd206c86640 1 --2- 172.21.15.47:0/2589243985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b3151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd206c86640 1 -- 172.21.15.47:0/2589243985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 msgr2=0x55e0b3151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd206c86640 1 --2- 172.21.15.47:0/2589243985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b3151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd206c86640 1 -- 172.21.15.47:0/2589243985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f00a6020 con 0x55e0b31505e0 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd206c86640 1 --2- 172.21.15.47:0/2589243985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b316ad80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd1f007ac50 tx=0x7fd1f00aa090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8f45240fcd2315 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd1e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd1e57fa640 1 -- 172.21.15.47:0/2589243985 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0082c70 con 0x55e0b31505e0 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd1e57fa640 1 -- 172.21.15.47:0/2589243985 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1f0082dd0 con 0x55e0b31505e0 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd20b264740 1 -- 172.21.15.47:0/2589243985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0b314f5f0 con 0x55e0b31505e0 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd1e57fa640 1 -- 172.21.15.47:0/2589243985 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1f0079c70 con 0x55e0b31505e0 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd20b264740 10 client.5964 fetch_fsmap learned FSMap version 18 2024-03-23T09:43:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd20b264740 1 -- 172.21.15.47:0/2589243985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55e0b30285c0 con 0x55e0b31505e0 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd1e57fa640 1 -- 172.21.15.47:0/2589243985 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e0b314f5f0 con 0x55e0b31505e0 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.608+0000 7fd1e57fa640 1 -- 172.21.15.47:0/2589243985 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 18) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd1f009a030 con 0x55e0b31505e0 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.609+0000 7fd20b264740 10 client.5964 fetch_fsmap finished waiting for FSMap version 18 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.609+0000 7fd20b264740 10 client.5964 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.609+0000 7fd20b264740 20 client.5964 populate_metadata read hostname 'smithi047' 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.609+0000 7fd20b264740 1 --2- 172.21.15.47:0/2589243985 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x55e0b31c9be0 0x55e0b316b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.609+0000 7fd20b264740 4 client.5964 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.609+0000 7fd20b264740 1 -- 172.21.15.47:0/2589243985 --> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fd1f4005430 con 0x55e0b31c9be0 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.609+0000 7fd206485640 1 --2- 172.21.15.47:0/2589243985 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x55e0b31c9be0 0x55e0b316b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.609+0000 7fd206485640 1 --2- 172.21.15.47:0/2589243985 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x55e0b31c9be0 0x55e0b316b2c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d55c98fc0f109830 server_cookie=d7c944e4de728707 in_seq=0 out_seq=0 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.609+0000 7fd1e57fa640 10 client.5964 ms_handle_connect on v2:172.21.15.175:6836/1626348200 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.609+0000 7fd1e57fa640 1 -- 172.21.15.47:0/2589243985 <== mds.0 v2:172.21.15.175:6836/1626348200 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fd1f4005430 con 0x55e0b31c9be0 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd1e57fa640 10 client.5964 handle_command_reply: tid=0 2024-03-23T09:43:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd20b264740 1 client.5964 shutdown 2024-03-23T09:43:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd20b264740 1 -- 172.21.15.47:0/2589243985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 msgr2=0x55e0b316ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd20b264740 1 --2- 172.21.15.47:0/2589243985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b316ad80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd1f007ac50 tx=0x7fd1f00aa090 comp rx=0 tx=0).stop 2024-03-23T09:43:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd20b264740 1 -- 172.21.15.47:0/2589243985 shutdown_connections 2024-03-23T09:43:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd20b264740 1 --2- 172.21.15.47:0/2589243985 >> [v2:172.21.15.175:6836/1626348200,v1:172.21.15.175:6837/1626348200] conn(0x55e0b31c9be0 0x55e0b316b2c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd20b264740 1 --2- 172.21.15.47:0/2589243985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0b31505e0 0x55e0b316ad80 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd20b264740 1 --2- 172.21.15.47:0/2589243985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e0b314e830 0x55e0b3151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd1e57fa640 0 client.5964 ms_handle_reset on v2:172.21.15.175:6836/1626348200 2024-03-23T09:43:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd20b264740 1 --2- 172.21.15.47:0/2589243985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0b31223b0 0x55e0b3151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd20b264740 1 -- 172.21.15.47:0/2589243985 >> 172.21.15.47:0/2589243985 conn(0x55e0b311e850 msgr2=0x55e0b3169020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd20b264740 1 -- 172.21.15.47:0/2589243985 shutdown_connections 2024-03-23T09:43:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd20b264740 1 -- 172.21.15.47:0/2589243985 wait complete. 2024-03-23T09:43:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd20b264740 20 client.5964 trim_cache size 0 max 16384 2024-03-23T09:43:30.617 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:43:30.617 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-23T09:43:30.617 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-23T09:43:30.617 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:43:30.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd2086e9640 1 -- 172.21.15.47:0/4273938443 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd1ec06ac00 msgr2=0x7fd1ec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:30.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd2086e9640 1 --2- 172.21.15.47:0/4273938443 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd1ec06ac00 0x7fd1ec06d0c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8005880 tx=0x7fd1f8009040 comp rx=0 tx=0).stop 2024-03-23T09:43:30.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd2086e9640 1 -- 172.21.15.47:0/4273938443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd20015cc30 msgr2=0x7fd2001f2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:30.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd2086e9640 1 --2- 172.21.15.47:0/4273938443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd20015cc30 0x7fd2001f2360 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009d90 tx=0x7fd1f0004a40 comp rx=0 tx=0).stop 2024-03-23T09:43:30.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd2086e9640 1 -- 172.21.15.47:0/4273938443 shutdown_connections 2024-03-23T09:43:30.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd2086e9640 1 --2- 172.21.15.47:0/4273938443 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd1ec06ac00 0x7fd1ec06d0c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd2086e9640 1 --2- 172.21.15.47:0/4273938443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd20015cc30 0x7fd2001f2360 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd2086e9640 1 --2- 172.21.15.47:0/4273938443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2000cd680 0x7fd2000c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd2086e9640 1 --2- 172.21.15.47:0/4273938443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2000cad40 0x7fd200153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd2086e9640 1 -- 172.21.15.47:0/4273938443 >> 172.21.15.47:0/4273938443 conn(0x7fd2000d06f0 msgr2=0x7fd20015bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:30.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd2086e9640 1 -- 172.21.15.47:0/4273938443 shutdown_connections 2024-03-23T09:43:30.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.610+0000 7fd2086e9640 1 -- 172.21.15.47:0/4273938443 wait complete. 2024-03-23T09:43:30.621 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T09:43:30.722 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:43:30.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.823+0000 7f065788b640 1 Processor -- start 2024-03-23T09:43:30.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.823+0000 7f065788b640 1 -- start start 2024-03-23T09:43:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f065788b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f065015cd00 0x7f06500d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f065788b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06500d1020 0x7f06500cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f065788b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06500cf750 0x7f06500b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f065788b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06500cfbe0 con 0x7f06500cf750 2024-03-23T09:43:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f065788b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065015d0e0 con 0x7f065015cd00 2024-03-23T09:43:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f065788b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06500d1540 con 0x7f06500d1020 2024-03-23T09:43:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f0655e28640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06500cf750 0x7f06500b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f0655e28640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06500cf750 0x7f06500b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50218/0 (socket says 172.21.15.47:50218) 2024-03-23T09:43:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f0655e28640 1 -- 172.21.15.47:0/3241391768 learned_addr learned my addr 172.21.15.47:0/3241391768 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f0654e26640 1 --2- 172.21.15.47:0/3241391768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06500d1020 0x7f06500cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f0655627640 1 --2- 172.21.15.47:0/3241391768 >> v2:172.21.15.175:3300/0 conn(0x7f065015cd00 0x7f06500d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f0655e28640 1 -- 172.21.15.47:0/3241391768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06500d1020 msgr2=0x7f06500cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f0655e28640 1 --2- 172.21.15.47:0/3241391768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06500d1020 0x7f06500cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f0655e28640 1 -- 172.21.15.47:0/3241391768 >> v2:172.21.15.175:3300/0 conn(0x7f065015cd00 msgr2=0x7f06500d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f0655e28640 1 --2- 172.21.15.47:0/3241391768 >> v2:172.21.15.175:3300/0 conn(0x7f065015cd00 0x7f06500d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f0655e28640 1 -- 172.21.15.47:0/3241391768 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06500d1820 con 0x7f06500cf750 2024-03-23T09:43:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.824+0000 7f0655627640 1 --2- 172.21.15.47:0/3241391768 >> v2:172.21.15.175:3300/0 conn(0x7f065015cd00 0x7f06500d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.825+0000 7f0655e28640 1 --2- 172.21.15.47:0/3241391768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06500cf750 0x7f06500b43e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f065006c1a0 tx=0x7f064802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f302e0b0b7d35192 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.825+0000 7f0647fff640 1 -- 172.21.15.47:0/3241391768 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064802c9e0 con 0x7f06500cf750 2024-03-23T09:43:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.825+0000 7f0647fff640 1 -- 172.21.15.47:0/3241391768 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064802cb40 con 0x7f06500cf750 2024-03-23T09:43:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.825+0000 7f0647fff640 1 -- 172.21.15.47:0/3241391768 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06480343f0 con 0x7f06500cf750 2024-03-23T09:43:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.826+0000 7f065788b640 1 -- 172.21.15.47:0/3241391768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06500cf750 msgr2=0x7f06500b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.826+0000 7f065788b640 1 --2- 172.21.15.47:0/3241391768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06500cf750 0x7f06500b43e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f065006c1a0 tx=0x7f064802be40 comp rx=0 tx=0).stop 2024-03-23T09:43:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.826+0000 7f065788b640 1 -- 172.21.15.47:0/3241391768 shutdown_connections 2024-03-23T09:43:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.826+0000 7f065788b640 1 --2- 172.21.15.47:0/3241391768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06500cf750 0x7f06500b43e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.826+0000 7f065788b640 1 --2- 172.21.15.47:0/3241391768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06500d1020 0x7f06500cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.826+0000 7f065788b640 1 --2- 172.21.15.47:0/3241391768 >> v2:172.21.15.175:3300/0 conn(0x7f065015cd00 0x7f06500d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.826+0000 7f065788b640 1 -- 172.21.15.47:0/3241391768 >> 172.21.15.47:0/3241391768 conn(0x7f065006b160 msgr2=0x7f065006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.826+0000 7f065788b640 1 -- 172.21.15.47:0/3241391768 shutdown_connections 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.826+0000 7f065788b640 1 -- 172.21.15.47:0/3241391768 wait complete. 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.827+0000 7f065788b640 1 Processor -- start 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.827+0000 7f065788b640 1 -- start start 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f065788b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06500cf750 0x7f0650157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f065788b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06500d1020 0x7f06501e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f065788b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f065015cd00 0x7f06501e7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f065788b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06500b3210 con 0x7f06500d1020 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f065788b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06500b34f0 con 0x7f065015cd00 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f065788b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06500b37d0 con 0x7f06500cf750 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0654e26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06500d1020 0x7f06501e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0654e26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06500d1020 0x7f06501e72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50232/0 (socket says 172.21.15.47:50232) 2024-03-23T09:43:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0654e26640 1 -- 172.21.15.47:0/179596502 learned_addr learned my addr 172.21.15.47:0/179596502 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0655627640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06500cf750 0x7f0650157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0655e28640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f065015cd00 0x7f06501e7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0654e26640 1 -- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06500cf750 msgr2=0x7f0650157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0654e26640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06500cf750 0x7f0650157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0654e26640 1 -- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f065015cd00 msgr2=0x7f06501e7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0654e26640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f065015cd00 0x7f06501e7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0654e26640 1 -- 172.21.15.47:0/179596502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0648009b30 con 0x7f06500d1020 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0655627640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06500cf750 0x7f0650157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0655e28640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f065015cd00 0x7f06501e7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0654e26640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06500d1020 0x7f06501e72c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f064000d800 tx=0x7f064000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0645ffb640 1 -- 172.21.15.47:0/179596502 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06400096a0 con 0x7f06500d1020 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0645ffb640 1 -- 172.21.15.47:0/179596502 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0640009800 con 0x7f06500d1020 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f065788b640 1 -- 172.21.15.47:0/179596502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06500b3ab0 con 0x7f06500d1020 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.828+0000 7f0645ffb640 1 -- 172.21.15.47:0/179596502 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640009b90 con 0x7f06500d1020 2024-03-23T09:43:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.829+0000 7f065788b640 1 -- 172.21.15.47:0/179596502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0650158260 con 0x7f06500d1020 2024-03-23T09:43:30.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.829+0000 7f065788b640 1 -- 172.21.15.47:0/179596502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0650072360 con 0x7f06500d1020 2024-03-23T09:43:30.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.830+0000 7f0645ffb640 1 -- 172.21.15.47:0/179596502 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0640009cf0 con 0x7f06500d1020 2024-03-23T09:43:30.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.830+0000 7f0645ffb640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f063406ac00 0x7f063406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:30.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.830+0000 7f0645ffb640 1 -- 172.21.15.47:0/179596502 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06400897c0 con 0x7f06500d1020 2024-03-23T09:43:30.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.831+0000 7f0655627640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f063406ac00 0x7f063406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:30.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.831+0000 7f0655627640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f063406ac00 0x7f063406d0c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f064c006fd0 tx=0x7f064c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:30.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:30.835+0000 7f0645ffb640 1 -- 172.21.15.47:0/179596502 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f06400549a0 con 0x7f06500d1020 2024-03-23T09:43:31.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.050+0000 7f065788b640 1 -- 172.21.15.47:0/179596502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0650066be0 con 0x7f06500d1020 2024-03-23T09:43:31.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.051+0000 7f0645ffb640 1 -- 172.21.15.47:0/179596502 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f064000b440 con 0x7f06500d1020 2024-03-23T09:43:31.051 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:31.051 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1189831856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1189831856},{"type":"v1","addr":"172.21.15.47:6835","nonce":1189831856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/2248064454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2248064454},{"type":"v1","addr":"172.21.15.47:6834","nonce":2248064454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5694,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3806465612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3806465612},{"type":"v1","addr":"172.21.15.175:6835","nonce":3806465612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:42:48.165819+0000","modified":"2024-03-23T09:42:50.171529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1626348200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1626348200},{"type":"v1","addr":"172.21.15.175:6837","nonce":1626348200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-23T09:43:31.051 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 18 2024-03-23T09:43:31.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.053+0000 7f065788b640 1 -- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f063406ac00 msgr2=0x7f063406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.053+0000 7f065788b640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f063406ac00 0x7f063406d0c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f064c006fd0 tx=0x7f064c008040 comp rx=0 tx=0).stop 2024-03-23T09:43:31.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.053+0000 7f065788b640 1 -- 172.21.15.47:0/179596502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06500d1020 msgr2=0x7f06501e72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.053+0000 7f065788b640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06500d1020 0x7f06501e72c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f064000d800 tx=0x7f064000dcc0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.053+0000 7f065788b640 1 -- 172.21.15.47:0/179596502 shutdown_connections 2024-03-23T09:43:31.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.053+0000 7f065788b640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f063406ac00 0x7f063406d0c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.053+0000 7f065788b640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f065015cd00 0x7f06501e7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.053+0000 7f065788b640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06500d1020 0x7f06501e72c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.053+0000 7f065788b640 1 --2- 172.21.15.47:0/179596502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06500cf750 0x7f0650157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.053+0000 7f065788b640 1 -- 172.21.15.47:0/179596502 >> 172.21.15.47:0/179596502 conn(0x7f065006b160 msgr2=0x7f06500b8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:31.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.053+0000 7f065788b640 1 -- 172.21.15.47:0/179596502 shutdown_connections 2024-03-23T09:43:31.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.053+0000 7f065788b640 1 -- 172.21.15.47:0/179596502 wait complete. 2024-03-23T09:43:31.064 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-03-23T09:43:31.194 INFO:teuthology.orchestra.run.smithi175.stdout:{"message":"","return_code":0} 2024-03-23T09:43:31.195 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-23T09:43:31.195 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-23T09:43:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.237+0000 7f13c995ff40 1 Processor -- start 2024-03-23T09:43:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.237+0000 7f13c995ff40 1 -- start start 2024-03-23T09:43:31.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c995ff40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e258672d0 0x559e258d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c995ff40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559e258d79e0 0x559e2584d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c995ff40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e258f2ba0 0x559e2584dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c995ff40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e25765b60 con 0x559e258f2ba0 2024-03-23T09:43:31.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c995ff40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e25765800 con 0x559e258672d0 2024-03-23T09:43:31.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c995ff40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e25748520 con 0x559e258d79e0 2024-03-23T09:43:31.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c7c5f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e258f2ba0 0x559e2584dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c7c5f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e258f2ba0 0x559e2584dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50252/0 (socket says 172.21.15.47:50252) 2024-03-23T09:43:31.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c7c5f640 1 -- 172.21.15.47:0/3835041839 learned_addr learned my addr 172.21.15.47:0/3835041839 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:31.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c745e640 1 --2- 172.21.15.47:0/3835041839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e258672d0 0x559e258d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c6c5d640 1 --2- 172.21.15.47:0/3835041839 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559e258d79e0 0x559e2584d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c7c5f640 1 -- 172.21.15.47:0/3835041839 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559e258d79e0 msgr2=0x559e2584d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:31.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c7c5f640 1 --2- 172.21.15.47:0/3835041839 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559e258d79e0 0x559e2584d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c7c5f640 1 -- 172.21.15.47:0/3835041839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e258672d0 msgr2=0x559e258d4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:31.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c7c5f640 1 --2- 172.21.15.47:0/3835041839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e258672d0 0x559e258d4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c7c5f640 1 -- 172.21.15.47:0/3835041839 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c000a100 con 0x559e258f2ba0 2024-03-23T09:43:31.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c745e640 1 --2- 172.21.15.47:0/3835041839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e258672d0 0x559e258d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:31.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.238+0000 7f13c7c5f640 1 --2- 172.21.15.47:0/3835041839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e258f2ba0 0x559e2584dc30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f13b800a530 tx=0x7f13b800a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953b53a67ef6c63f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:31.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.239+0000 7f13c5c5b640 1 -- 172.21.15.47:0/3835041839 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8008980 con 0x559e258f2ba0 2024-03-23T09:43:31.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.239+0000 7f13c5c5b640 1 -- 172.21.15.47:0/3835041839 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b8008ae0 con 0x559e258f2ba0 2024-03-23T09:43:31.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.239+0000 7f13c5c5b640 1 -- 172.21.15.47:0/3835041839 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8008e70 con 0x559e258f2ba0 2024-03-23T09:43:31.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.239+0000 7f13c995ff40 1 -- 172.21.15.47:0/3835041839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e258f2ba0 msgr2=0x559e2584dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.239+0000 7f13c995ff40 1 --2- 172.21.15.47:0/3835041839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e258f2ba0 0x559e2584dc30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f13b800a530 tx=0x7f13b800a9f0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.239+0000 7f13c995ff40 1 -- 172.21.15.47:0/3835041839 shutdown_connections 2024-03-23T09:43:31.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.239+0000 7f13c995ff40 1 --2- 172.21.15.47:0/3835041839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e258f2ba0 0x559e2584dc30 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.239+0000 7f13c995ff40 1 --2- 172.21.15.47:0/3835041839 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559e258d79e0 0x559e2584d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.239+0000 7f13c995ff40 1 --2- 172.21.15.47:0/3835041839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e258672d0 0x559e258d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.239+0000 7f13c995ff40 1 -- 172.21.15.47:0/3835041839 >> 172.21.15.47:0/3835041839 conn(0x559e25848e40 msgr2=0x559e258d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:31.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.239+0000 7f13c995ff40 1 -- 172.21.15.47:0/3835041839 shutdown_connections 2024-03-23T09:43:31.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.239+0000 7f13c995ff40 1 -- 172.21.15.47:0/3835041839 wait complete. 2024-03-23T09:43:31.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.240+0000 7f13c995ff40 1 Processor -- start 2024-03-23T09:43:31.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.240+0000 7f13c995ff40 1 -- start start 2024-03-23T09:43:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.240+0000 7f13c995ff40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e258672d0 0x559e258ed430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.240+0000 7f13c995ff40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e258d79e0 0x559e25865770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.240+0000 7f13c995ff40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559e258f2ba0 0x559e25865cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.240+0000 7f13c995ff40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2584c5a0 con 0x559e258d79e0 2024-03-23T09:43:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.240+0000 7f13c995ff40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2581b6a0 con 0x559e258672d0 2024-03-23T09:43:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.240+0000 7f13c995ff40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e2584bf70 con 0x559e258f2ba0 2024-03-23T09:43:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.240+0000 7f13c6c5d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e258d79e0 0x559e25865770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c6c5d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e258d79e0 0x559e25865770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50268/0 (socket says 172.21.15.47:50268) 2024-03-23T09:43:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c6c5d640 1 -- 172.21.15.47:0/166149105 learned_addr learned my addr 172.21.15.47:0/166149105 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c745e640 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e258672d0 0x559e258ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c7c5f640 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559e258f2ba0 0x559e25865cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c6c5d640 1 -- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559e258f2ba0 msgr2=0x559e25865cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c6c5d640 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559e258f2ba0 0x559e25865cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c6c5d640 1 -- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e258672d0 msgr2=0x559e258ed430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c6c5d640 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e258672d0 0x559e258ed430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c6c5d640 1 -- 172.21.15.47:0/166149105 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b8005cf0 con 0x559e258d79e0 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c6c5d640 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e258d79e0 0x559e25865770 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00dfd0 tx=0x7f13bc00b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c4c59640 1 -- 172.21.15.47:0/166149105 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc010620 con 0x559e258d79e0 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c4c59640 1 -- 172.21.15.47:0/166149105 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc010780 con 0x559e258d79e0 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c995ff40 1 -- 172.21.15.47:0/166149105 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559e25774c20 con 0x559e258d79e0 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c4c59640 1 -- 172.21.15.47:0/166149105 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc010b30 con 0x559e258d79e0 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.241+0000 7f13c995ff40 1 -- 172.21.15.47:0/166149105 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e25768060 con 0x559e258d79e0 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.243+0000 7f13c4c59640 1 -- 172.21.15.47:0/166149105 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f13bc010c90 con 0x559e258d79e0 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.243+0000 7f13c4c59640 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13b40695e0 0x7f13b406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.243+0000 7f13c4c59640 1 -- 172.21.15.47:0/166149105 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13bc00a040 con 0x559e258d79e0 2024-03-23T09:43:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.243+0000 7f13c745e640 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13b40695e0 0x7f13b406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.243+0000 7f13c995ff40 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x559e258d4900 0x559e2584ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.243+0000 7f13c995ff40 1 -- 172.21.15.47:0/166149105 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46) v8 -- 0x559e2584d410 con 0x559e258d4900 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.243+0000 7f13c7c5f640 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x559e258d4900 0x559e2584ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c745e640 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13b40695e0 0x7f13b406baa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f13c0053000 tx=0x7f13c004c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c7c5f640 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x559e258d4900 0x559e2584ced0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c7c5f640 1 -- 172.21.15.47:0/166149105 <== osd.6 v2:172.21.15.175:6808/38160 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f13b8008cd0 con 0x559e258d4900 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c995ff40 1 -- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x559e258d4900 msgr2=0x559e2584ced0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c995ff40 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x559e258d4900 0x559e2584ced0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c995ff40 1 -- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13b40695e0 msgr2=0x7f13b406baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c995ff40 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13b40695e0 0x7f13b406baa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f13c0053000 tx=0x7f13c004c000 comp rx=0 tx=0).stop 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c995ff40 1 -- 172.21.15.47:0/166149105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e258d79e0 msgr2=0x559e25865770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c995ff40 1 --2- 172.21.15.47:0/166149105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e258d79e0 0x559e25865770 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00dfd0 tx=0x7f13bc00b6a0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c7c5f640 1 -- 172.21.15.47:0/166149105 reap_dead start 2024-03-23T09:43:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c995ff40 1 -- 172.21.15.47:0/166149105 shutdown_connections 2024-03-23T09:43:31.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c995ff40 1 -- 172.21.15.47:0/166149105 >> 172.21.15.47:0/166149105 conn(0x559e25848e40 msgr2=0x559e258d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:31.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c995ff40 1 -- 172.21.15.47:0/166149105 shutdown_connections 2024-03-23T09:43:31.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.244+0000 7f13c995ff40 1 -- 172.21.15.47:0/166149105 wait complete. 2024-03-23T09:43:31.249 INFO:teuthology.orchestra.run.smithi047.stdout:Yalphaparent_a 2024-03-23T09:43:31.250 DEBUG:teuthology.orchestra.run.smithi047:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-23T09:43:31.333 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:43:31.334 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 1099511627777, 2024-03-23T09:43:31.334 INFO:teuthology.orchestra.run.smithi047.stdout: "ancestors": [ 2024-03-23T09:43:31.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:31.334 INFO:teuthology.orchestra.run.smithi047.stdout: "dirino": 1099511627776, 2024-03-23T09:43:31.334 INFO:teuthology.orchestra.run.smithi047.stdout: "dname": "alpha", 2024-03-23T09:43:31.334 INFO:teuthology.orchestra.run.smithi047.stdout: "version": 4 2024-03-23T09:43:31.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:31.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:31.334 INFO:teuthology.orchestra.run.smithi047.stdout: "dirino": 1, 2024-03-23T09:43:31.334 INFO:teuthology.orchestra.run.smithi047.stdout: "dname": "parent_a", 2024-03-23T09:43:31.334 INFO:teuthology.orchestra.run.smithi047.stdout: "version": 7 2024-03-23T09:43:31.334 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:31.335 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-23T09:43:31.335 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 7, 2024-03-23T09:43:31.335 INFO:teuthology.orchestra.run.smithi047.stdout: "old_pools": [] 2024-03-23T09:43:31.335 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:43:31.335 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:31.336 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:43:31.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:43:31.420 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:43:31.463 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:43:31.463 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:43:31.463 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:43:31.479 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:43:31.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.619+0000 7f0554a96640 1 Processor -- start 2024-03-23T09:43:31.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.619+0000 7f0554a96640 1 -- start start 2024-03-23T09:43:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.620+0000 7f0554a96640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055015ca90 0x7f0550160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.620+0000 7f0554a96640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05500d11e0 con 0x7f05500b8350 2024-03-23T09:43:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.620+0000 7f0554a96640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0550160f60 con 0x7f05500cde10 2024-03-23T09:43:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.620+0000 7f0554a96640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0550161240 con 0x7f055015ca90 2024-03-23T09:43:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.620+0000 7f054dd9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f05500b8350 0x7f05500cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45898/0 (socket says 172.21.15.47:45898) 2024-03-23T09:43:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.620+0000 7f054dd9b640 1 -- 172.21.15.47:0/1396387162 learned_addr learned my addr 172.21.15.47:0/1396387162 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.620+0000 7f054e59c640 1 --2- 172.21.15.47:0/1396387162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055015ca90 0x7f0550160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.620+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599346746 0 0) 0x7f05500d11e0 con 0x7f05500b8350 2024-03-23T09:43:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.620+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f053c003520 con 0x7f05500b8350 2024-03-23T09:43:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.620+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802506781 0 0) 0x7f0550160f60 con 0x7f05500cde10 2024-03-23T09:43:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.620+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05500d11e0 con 0x7f05500cde10 2024-03-23T09:43:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.620+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 75655383 0 0) 0x7f053c003520 con 0x7f05500b8350 2024-03-23T09:43:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055015ca90 msgr2=0x7f0550160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054d59a640 1 --2- 172.21.15.47:0/1396387162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055015ca90 0x7f0550160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 >> v1:172.21.15.175:6789/0 conn(0x7f05500cde10 legacy=0x7f05500d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0550161520 con 0x7f05500b8350 2024-03-23T09:43:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0538001670 con 0x7f05500b8350 2024-03-23T09:43:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054e59c640 1 --2- 172.21.15.47:0/1396387162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055015ca90 0x7f0550160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:43:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 >> v1:172.21.15.47:6789/0 conn(0x7f05500b8350 legacy=0x7f05500cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054d59a640 1 --2- 172.21.15.47:0/1396387162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055015ca90 0x7f053c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054d59a640 1 --2- 172.21.15.47:0/1396387162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f053c004070 0x7f053c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054d59a640 1 --2- 172.21.15.47:0/1396387162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c004d00 0x7f053c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0550161240 con 0x7f053c004070 2024-03-23T09:43:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c00bac0 con 0x7f053c004d00 2024-03-23T09:43:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053c00bc80 con 0x7f055015ca90 2024-03-23T09:43:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054dd9b640 1 --2- 172.21.15.47:0/1396387162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f053c004070 0x7f053c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054e59c640 1 --2- 172.21.15.47:0/1396387162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055015ca90 0x7f053c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054ed9d640 1 --2- 172.21.15.47:0/1396387162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c004d00 0x7f053c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054dd9b640 1 -- 172.21.15.47:0/1396387162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055015ca90 msgr2=0x7f053c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054dd9b640 1 --2- 172.21.15.47:0/1396387162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055015ca90 0x7f053c003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054dd9b640 1 -- 172.21.15.47:0/1396387162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c004d00 msgr2=0x7f053c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054dd9b640 1 --2- 172.21.15.47:0/1396387162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c004d00 0x7f053c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.621+0000 7f054dd9b640 1 -- 172.21.15.47:0/1396387162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0540009a70 con 0x7f053c004070 2024-03-23T09:43:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.622+0000 7f054dd9b640 1 --2- 172.21.15.47:0/1396387162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f053c004070 0x7f053c0047c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0538007fd0 tx=0x7f053c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ed64d38539127c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.622+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538005ad0 con 0x7f053c004070 2024-03-23T09:43:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.622+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0538005c50 con 0x7f053c004070 2024-03-23T09:43:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.622+0000 7f054d59a640 1 -- 172.21.15.47:0/1396387162 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053802c830 con 0x7f053c004070 2024-03-23T09:43:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.622+0000 7f0554a96640 1 -- 172.21.15.47:0/1396387162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f053c004070 msgr2=0x7f053c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.622+0000 7f0554a96640 1 --2- 172.21.15.47:0/1396387162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f053c004070 0x7f053c0047c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0538007fd0 tx=0x7f053c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.622+0000 7f054ed9d640 1 -- 172.21.15.47:0/1396387162 reap_dead start 2024-03-23T09:43:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.623+0000 7f0554a96640 1 -- 172.21.15.47:0/1396387162 shutdown_connections 2024-03-23T09:43:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.623+0000 7f0554a96640 1 -- 172.21.15.47:0/1396387162 >> 172.21.15.47:0/1396387162 conn(0x7f05500659d0 msgr2=0x7f0550065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.623+0000 7f0554a96640 1 -- 172.21.15.47:0/1396387162 shutdown_connections 2024-03-23T09:43:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.623+0000 7f0554a96640 1 -- 172.21.15.47:0/1396387162 wait complete. 2024-03-23T09:43:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.624+0000 7f0554a96640 1 Processor -- start 2024-03-23T09:43:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.624+0000 7f0554a96640 1 -- start start 2024-03-23T09:43:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f0554a96640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c004070 0x7f05501f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f0554a96640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05500b4ca0 0x7f0550150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f0554a96640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f055015a640 0x7f055015aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f0554a96640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055015b010 con 0x7f055015a640 2024-03-23T09:43:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f0554a96640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0550159b70 con 0x7f053c004070 2024-03-23T09:43:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f0554a96640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0550158880 con 0x7f05500b4ca0 2024-03-23T09:43:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f054ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f055015a640 0x7f055015aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f054ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f055015a640 0x7f055015aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50298/0 (socket says 172.21.15.47:50298) 2024-03-23T09:43:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f054ed9d640 1 -- 172.21.15.47:0/1875920498 learned_addr learned my addr 172.21.15.47:0/1875920498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f054e59c640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c004070 0x7f05501f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f054dd9b640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05500b4ca0 0x7f0550150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f054ed9d640 1 -- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05500b4ca0 msgr2=0x7f0550150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f054ed9d640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05500b4ca0 0x7f0550150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f054ed9d640 1 -- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c004070 msgr2=0x7f05501f32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f054ed9d640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c004070 0x7f05501f32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f054ed9d640 1 -- 172.21.15.47:0/1875920498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0550158ad0 con 0x7f055015a640 2024-03-23T09:43:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f054ed9d640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f055015a640 0x7f055015aad0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f05500b5950 tx=0x7f054800f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f0546ffd640 1 -- 172.21.15.47:0/1875920498 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548006aa0 con 0x7f055015a640 2024-03-23T09:43:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f0546ffd640 1 -- 172.21.15.47:0/1875920498 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0548006c20 con 0x7f055015a640 2024-03-23T09:43:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f0546ffd640 1 -- 172.21.15.47:0/1875920498 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548005030 con 0x7f055015a640 2024-03-23T09:43:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f0554a96640 1 -- 172.21.15.47:0/1875920498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0550158db0 con 0x7f055015a640 2024-03-23T09:43:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.625+0000 7f0554a96640 1 -- 172.21.15.47:0/1875920498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0550159370 con 0x7f055015a640 2024-03-23T09:43:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.626+0000 7f0554a96640 1 -- 172.21.15.47:0/1875920498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0550072360 con 0x7f055015a640 2024-03-23T09:43:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.627+0000 7f0546ffd640 1 -- 172.21.15.47:0/1875920498 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0548005190 con 0x7f055015a640 2024-03-23T09:43:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.627+0000 7f0546ffd640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f053006ac00 0x7f053006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.627+0000 7f0546ffd640 1 -- 172.21.15.47:0/1875920498 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05480882e0 con 0x7f055015a640 2024-03-23T09:43:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.627+0000 7f054e59c640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f053006ac00 0x7f053006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.628+0000 7f054e59c640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f053006ac00 0x7f053006d0c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0540014590 tx=0x7f054000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.631+0000 7f0546ffd640 1 -- 172.21.15.47:0/1875920498 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0548053600 con 0x7f055015a640 2024-03-23T09:43:31.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.811+0000 7f0554a96640 1 -- 172.21.15.47:0/1875920498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0550066be0 con 0x7f055015a640 2024-03-23T09:43:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.813+0000 7f0546ffd640 1 -- 172.21.15.47:0/1875920498 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16045 (secure 0 0 0) 0x7f0548056c20 con 0x7f055015a640 2024-03-23T09:43:31.813 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:31.813 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":46,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:49.167591+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:42:46.053713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-23T09:42:47.131279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:43:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.814+0000 7f0554a96640 1 -- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f053006ac00 msgr2=0x7f053006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.815+0000 7f0554a96640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f053006ac00 0x7f053006d0c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0540014590 tx=0x7f054000b040 comp rx=0 tx=0).stop 2024-03-23T09:43:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.815+0000 7f0554a96640 1 -- 172.21.15.47:0/1875920498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f055015a640 msgr2=0x7f055015aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.815+0000 7f0554a96640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f055015a640 0x7f055015aad0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f05500b5950 tx=0x7f054800f9f0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.815+0000 7f0554a96640 1 -- 172.21.15.47:0/1875920498 shutdown_connections 2024-03-23T09:43:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.815+0000 7f0554a96640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f053006ac00 0x7f053006d0c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.815+0000 7f0554a96640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f055015a640 0x7f055015aad0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.815+0000 7f0554a96640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05500b4ca0 0x7f0550150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.815+0000 7f0554a96640 1 --2- 172.21.15.47:0/1875920498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c004070 0x7f05501f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.815+0000 7f0554a96640 1 -- 172.21.15.47:0/1875920498 >> 172.21.15.47:0/1875920498 conn(0x7f05500659d0 msgr2=0x7f05500d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.815+0000 7f0554a96640 1 -- 172.21.15.47:0/1875920498 shutdown_connections 2024-03-23T09:43:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.815+0000 7f0554a96640 1 -- 172.21.15.47:0/1875920498 wait complete. 2024-03-23T09:43:31.825 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:42:46.053713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:42:47.131279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:43:31.826 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:43:31.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.930+0000 7f03e2810640 1 Processor -- start 2024-03-23T09:43:31.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.930+0000 7f03e2810640 1 -- start start 2024-03-23T09:43:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.930+0000 7f03e2810640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f03dc15ff60 0x7f03dc0cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.930+0000 7f03e2810640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03dc0b1b20 0x7f03dc0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.930+0000 7f03e2810640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03dc160d60 con 0x7f03dc0b81f0 2024-03-23T09:43:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.930+0000 7f03e2810640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03dc160340 con 0x7f03dc0b1b20 2024-03-23T09:43:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.930+0000 7f03e2810640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03dc0b8680 con 0x7f03dc15ff60 2024-03-23T09:43:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.930+0000 7f03e0dad640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03dc0b1b20 0x7f03dc0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.931+0000 7f03e0dad640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03dc0b1b20 0x7f03dc0cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50314/0 (socket says 172.21.15.47:50314) 2024-03-23T09:43:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.931+0000 7f03e0dad640 1 -- 172.21.15.47:0/2421878308 learned_addr learned my addr 172.21.15.47:0/2421878308 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:31.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.931+0000 7f03dbfff640 1 --2- 172.21.15.47:0/2421878308 >> v2:172.21.15.175:3300/0 conn(0x7f03dc15ff60 0x7f03dc0cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.931+0000 7f03e0dad640 1 -- 172.21.15.47:0/2421878308 >> v2:172.21.15.175:3300/0 conn(0x7f03dc15ff60 msgr2=0x7f03dc0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:31.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.931+0000 7f03e0dad640 1 --2- 172.21.15.47:0/2421878308 >> v2:172.21.15.175:3300/0 conn(0x7f03dc15ff60 0x7f03dc0cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.931+0000 7f03e0dad640 1 -- 172.21.15.47:0/2421878308 >> v1:172.21.15.175:6789/0 conn(0x7f03dc0b81f0 legacy=0x7f03dc0d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.931+0000 7f03e0dad640 1 -- 172.21.15.47:0/2421878308 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc160710 con 0x7f03dc0b1b20 2024-03-23T09:43:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.932+0000 7f03e0dad640 1 --2- 172.21.15.47:0/2421878308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03dc0b1b20 0x7f03dc0cfd60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f03dc0b4e10 tx=0x7f03d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ed5977c16604df server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.932+0000 7f03daffd640 1 -- 172.21.15.47:0/2421878308 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d402ca80 con 0x7f03dc0b1b20 2024-03-23T09:43:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.932+0000 7f03daffd640 1 -- 172.21.15.47:0/2421878308 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d402cbe0 con 0x7f03dc0b1b20 2024-03-23T09:43:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.932+0000 7f03daffd640 1 -- 172.21.15.47:0/2421878308 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d40343f0 con 0x7f03dc0b1b20 2024-03-23T09:43:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.932+0000 7f03e2810640 1 -- 172.21.15.47:0/2421878308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03dc0b1b20 msgr2=0x7f03dc0cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.932+0000 7f03e2810640 1 --2- 172.21.15.47:0/2421878308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03dc0b1b20 0x7f03dc0cfd60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f03dc0b4e10 tx=0x7f03d402bee0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.932+0000 7f03e2810640 1 -- 172.21.15.47:0/2421878308 shutdown_connections 2024-03-23T09:43:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.932+0000 7f03e2810640 1 --2- 172.21.15.47:0/2421878308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03dc0b1b20 0x7f03dc0cfd60 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.932+0000 7f03e2810640 1 --2- 172.21.15.47:0/2421878308 >> v2:172.21.15.175:3300/0 conn(0x7f03dc15ff60 0x7f03dc0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.932+0000 7f03e2810640 1 -- 172.21.15.47:0/2421878308 >> 172.21.15.47:0/2421878308 conn(0x7f03dc065df0 msgr2=0x7f03dc066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.932+0000 7f03e2810640 1 -- 172.21.15.47:0/2421878308 shutdown_connections 2024-03-23T09:43:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.932+0000 7f03e2810640 1 -- 172.21.15.47:0/2421878308 wait complete. 2024-03-23T09:43:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.933+0000 7f03e2810640 1 Processor -- start 2024-03-23T09:43:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.934+0000 7f03e2810640 1 -- start start 2024-03-23T09:43:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.934+0000 7f03e2810640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03dc0b1b20 0x7f03dc153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.934+0000 7f03e2810640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03dc15ff60 0x7f03dc0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.934+0000 7f03e2810640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03dc0cb480 0x7f03dc0d0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.934+0000 7f03e2810640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc0cca10 con 0x7f03dc15ff60 2024-03-23T09:43:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.934+0000 7f03e2810640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc0cb990 con 0x7f03dc0b1b20 2024-03-23T09:43:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.934+0000 7f03e2810640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc0cbc70 con 0x7f03dc0cb480 2024-03-23T09:43:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.934+0000 7f03db7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03dc15ff60 0x7f03dc0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03db7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03dc15ff60 0x7f03dc0caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50326/0 (socket says 172.21.15.47:50326) 2024-03-23T09:43:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03db7fe640 1 -- 172.21.15.47:0/3554144305 learned_addr learned my addr 172.21.15.47:0/3554144305 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03dbfff640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03dc0b1b20 0x7f03dc153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03e0dad640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03dc0cb480 0x7f03dc0d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03db7fe640 1 -- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03dc0cb480 msgr2=0x7f03dc0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03db7fe640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03dc0cb480 0x7f03dc0d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03db7fe640 1 -- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03dc0b1b20 msgr2=0x7f03dc153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03db7fe640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03dc0b1b20 0x7f03dc153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03db7fe640 1 -- 172.21.15.47:0/3554144305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d4009b30 con 0x7f03dc15ff60 2024-03-23T09:43:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03db7fe640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03dc15ff60 0x7f03dc0caf40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f03d000c450 tx=0x7f03d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03d8ff9640 1 -- 172.21.15.47:0/3554144305 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d00052d0 con 0x7f03dc15ff60 2024-03-23T09:43:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03d8ff9640 1 -- 172.21.15.47:0/3554144305 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d0005430 con 0x7f03dc15ff60 2024-03-23T09:43:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03e2810640 1 -- 172.21.15.47:0/3554144305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03dc0cbf50 con 0x7f03dc15ff60 2024-03-23T09:43:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03d8ff9640 1 -- 172.21.15.47:0/3554144305 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0012870 con 0x7f03dc15ff60 2024-03-23T09:43:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.935+0000 7f03e2810640 1 -- 172.21.15.47:0/3554144305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03dc0cc510 con 0x7f03dc15ff60 2024-03-23T09:43:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.936+0000 7f03e2810640 1 -- 172.21.15.47:0/3554144305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03dc0672f0 con 0x7f03dc15ff60 2024-03-23T09:43:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.937+0000 7f03d8ff9640 1 -- 172.21.15.47:0/3554144305 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f03d00129d0 con 0x7f03dc15ff60 2024-03-23T09:43:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.937+0000 7f03d8ff9640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f03c806ac00 0x7f03c806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.937+0000 7f03d8ff9640 1 -- 172.21.15.47:0/3554144305 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03d0088d70 con 0x7f03dc15ff60 2024-03-23T09:43:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.937+0000 7f03dbfff640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f03c806ac00 0x7f03c806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.938+0000 7f03dbfff640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f03c806ac00 0x7f03c806d0c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009aa0 tx=0x7f03cc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:31.941+0000 7f03d8ff9640 1 -- 172.21.15.47:0/3554144305 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f03d0054000 con 0x7f03dc15ff60 2024-03-23T09:43:32.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.120+0000 7f03e2810640 1 -- 172.21.15.47:0/3554144305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03dc0b26b0 con 0x7f03dc15ff60 2024-03-23T09:43:32.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.120+0000 7f03d8ff9640 1 -- 172.21.15.47:0/3554144305 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f03d0057620 con 0x7f03dc15ff60 2024-03-23T09:43:32.120 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:43:32.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.122+0000 7f03e2810640 1 -- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f03c806ac00 msgr2=0x7f03c806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.122+0000 7f03e2810640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f03c806ac00 0x7f03c806d0c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009aa0 tx=0x7f03cc009190 comp rx=0 tx=0).stop 2024-03-23T09:43:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.122+0000 7f03e2810640 1 -- 172.21.15.47:0/3554144305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03dc15ff60 msgr2=0x7f03dc0caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.122+0000 7f03e2810640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03dc15ff60 0x7f03dc0caf40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f03d000c450 tx=0x7f03d000c910 comp rx=0 tx=0).stop 2024-03-23T09:43:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.122+0000 7f03e2810640 1 -- 172.21.15.47:0/3554144305 shutdown_connections 2024-03-23T09:43:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.122+0000 7f03e2810640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f03c806ac00 0x7f03c806d0c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.122+0000 7f03e2810640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03dc0cb480 0x7f03dc0d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.122+0000 7f03e2810640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03dc15ff60 0x7f03dc0caf40 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.122+0000 7f03e2810640 1 --2- 172.21.15.47:0/3554144305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03dc0b1b20 0x7f03dc153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.122+0000 7f03e2810640 1 -- 172.21.15.47:0/3554144305 >> 172.21.15.47:0/3554144305 conn(0x7f03dc065df0 msgr2=0x7f03dc0661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.122+0000 7f03e2810640 1 -- 172.21.15.47:0/3554144305 shutdown_connections 2024-03-23T09:43:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.122+0000 7f03e2810640 1 -- 172.21.15.47:0/3554144305 wait complete. 2024-03-23T09:43:32.132 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:43:32.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.236+0000 7f5a41eea640 1 Processor -- start 2024-03-23T09:43:32.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.236+0000 7f5a41eea640 1 -- start start 2024-03-23T09:43:32.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.237+0000 7f5a41eea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a3c0cd7f0 0x7f5a3c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.237+0000 7f5a41eea640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5a3c0ce950 0x7f5a3c0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.237+0000 7f5a41eea640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a3c0cb270 con 0x7f5a3c15ab80 2024-03-23T09:43:32.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.237+0000 7f5a41eea640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a3c0cb510 con 0x7f5a3c0ce950 2024-03-23T09:43:32.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.237+0000 7f5a41eea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0c4240 con 0x7f5a3c0cd7f0 2024-03-23T09:43:32.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.237+0000 7f5a3b7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5a3c15ab80 0x7f5a3c0cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45900/0 (socket says 172.21.15.47:45900) 2024-03-23T09:43:32.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.237+0000 7f5a3b7fe640 1 -- 172.21.15.47:0/550401674 learned_addr learned my addr 172.21.15.47:0/550401674 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:32.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.237+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128857594 0 0) 0x7f5a3c0cb270 con 0x7f5a3c15ab80 2024-03-23T09:43:32.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a24003520 con 0x7f5a3c15ab80 2024-03-23T09:43:32.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3bfff640 1 --2- 172.21.15.47:0/550401674 >> v2:172.21.15.175:3300/0 conn(0x7f5a3c0ce950 0x7f5a3c0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3affd640 1 --2- 172.21.15.47:0/550401674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a3c0cd7f0 0x7f5a3c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3437650444 0 0) 0x7f5a24003520 con 0x7f5a3c15ab80 2024-03-23T09:43:32.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a3c0cd7f0 msgr2=0x7f5a3c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:32.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 --2- 172.21.15.47:0/550401674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a3c0cd7f0 0x7f5a3c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 >> v2:172.21.15.175:3300/0 conn(0x7f5a3c0ce950 msgr2=0x7f5a3c0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:32.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 --2- 172.21.15.47:0/550401674 >> v2:172.21.15.175:3300/0 conn(0x7f5a3c0ce950 0x7f5a3c0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c0cf740 con 0x7f5a3c15ab80 2024-03-23T09:43:32.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f5a2c001670 con 0x7f5a3c15ab80 2024-03-23T09:43:32.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 >> v1:172.21.15.47:6789/0 conn(0x7f5a3c15ab80 legacy=0x7f5a3c0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 --2- 172.21.15.47:0/550401674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a3c0cd7f0 0x7f5a24003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 --2- 172.21.15.47:0/550401674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a24003ef0 0x7f5a240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 --2- 172.21.15.47:0/550401674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a24004d00 0x7f5a24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0cb510 con 0x7f5a24004d00 2024-03-23T09:43:32.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0c4240 con 0x7f5a24003ef0 2024-03-23T09:43:32.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2400ba80 con 0x7f5a3c0cd7f0 2024-03-23T09:43:32.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3bfff640 1 --2- 172.21.15.47:0/550401674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a24004d00 0x7f5a24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3affd640 1 --2- 172.21.15.47:0/550401674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a24003ef0 0x7f5a240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3b7fe640 1 --2- 172.21.15.47:0/550401674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a3c0cd7f0 0x7f5a24003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3bfff640 1 -- 172.21.15.47:0/550401674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a3c0cd7f0 msgr2=0x7f5a24003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3bfff640 1 --2- 172.21.15.47:0/550401674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a3c0cd7f0 0x7f5a24003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3bfff640 1 -- 172.21.15.47:0/550401674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a24003ef0 msgr2=0x7f5a240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3bfff640 1 --2- 172.21.15.47:0/550401674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a24003ef0 0x7f5a240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.238+0000 7f5a3bfff640 1 -- 172.21.15.47:0/550401674 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a24003520 con 0x7f5a24004d00 2024-03-23T09:43:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.239+0000 7f5a3bfff640 1 --2- 172.21.15.47:0/550401674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a24004d00 0x7f5a24005180 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400e170 tx=0x7f5a240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2979829b2b30378 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.239+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3402cc20 con 0x7f5a24004d00 2024-03-23T09:43:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.239+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3400f040 con 0x7f5a24004d00 2024-03-23T09:43:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.239+0000 7f5a3a7fc640 1 -- 172.21.15.47:0/550401674 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a34034440 con 0x7f5a24004d00 2024-03-23T09:43:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.240+0000 7f5a41eea640 1 -- 172.21.15.47:0/550401674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a24004d00 msgr2=0x7f5a24005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.240+0000 7f5a41eea640 1 --2- 172.21.15.47:0/550401674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a24004d00 0x7f5a24005180 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400e170 tx=0x7f5a240057f0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.240+0000 7f5a3bfff640 1 -- 172.21.15.47:0/550401674 reap_dead start 2024-03-23T09:43:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.240+0000 7f5a41eea640 1 -- 172.21.15.47:0/550401674 shutdown_connections 2024-03-23T09:43:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.240+0000 7f5a41eea640 1 -- 172.21.15.47:0/550401674 >> 172.21.15.47:0/550401674 conn(0x7f5a3c0d06f0 msgr2=0x7f5a3c0d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:32.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.240+0000 7f5a41eea640 1 -- 172.21.15.47:0/550401674 shutdown_connections 2024-03-23T09:43:32.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.240+0000 7f5a41eea640 1 -- 172.21.15.47:0/550401674 wait complete. 2024-03-23T09:43:32.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.241+0000 7f5a41eea640 1 Processor -- start 2024-03-23T09:43:32.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.241+0000 7f5a41eea640 1 -- start start 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a41eea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a3c150510 0x7f5a3c0c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a41eea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a3c1508f0 0x7f5a3c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a41eea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a3c0c9480 0x7f5a3c0af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a41eea640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0c77c0 con 0x7f5a3c150510 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a41eea640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c1efd60 con 0x7f5a3c1508f0 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a41eea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0c9920 con 0x7f5a3c0c9480 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a3b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a3c150510 0x7f5a3c0c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a3b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a3c150510 0x7f5a3c0c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50344/0 (socket says 172.21.15.47:50344) 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a3b7fe640 1 -- 172.21.15.47:0/3565203357 learned_addr learned my addr 172.21.15.47:0/3565203357 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a3affd640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a3c1508f0 0x7f5a3c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a3bfff640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a3c0c9480 0x7f5a3c0af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a3b7fe640 1 -- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a3c0c9480 msgr2=0x7f5a3c0af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a3b7fe640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a3c0c9480 0x7f5a3c0af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a3b7fe640 1 -- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a3c1508f0 msgr2=0x7f5a3c0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a3b7fe640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a3c1508f0 0x7f5a3c0c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a3b7fe640 1 -- 172.21.15.47:0/3565203357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c0c7a60 con 0x7f5a3c150510 2024-03-23T09:43:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.242+0000 7f5a3b7fe640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a3c150510 0x7f5a3c0c8540 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0072c0 tx=0x7f5a2c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.243+0000 7f5a23fff640 1 -- 172.21.15.47:0/3565203357 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c00fa20 con 0x7f5a3c150510 2024-03-23T09:43:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.243+0000 7f5a23fff640 1 -- 172.21.15.47:0/3565203357 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2c00fb80 con 0x7f5a3c150510 2024-03-23T09:43:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.243+0000 7f5a41eea640 1 -- 172.21.15.47:0/3565203357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c0c7d40 con 0x7f5a3c150510 2024-03-23T09:43:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.243+0000 7f5a23fff640 1 -- 172.21.15.47:0/3565203357 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0133f0 con 0x7f5a3c150510 2024-03-23T09:43:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.243+0000 7f5a41eea640 1 -- 172.21.15.47:0/3565203357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c150e90 con 0x7f5a3c150510 2024-03-23T09:43:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.244+0000 7f5a41eea640 1 -- 172.21.15.47:0/3565203357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3c0c8260 con 0x7f5a3c150510 2024-03-23T09:43:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.244+0000 7f5a23fff640 1 -- 172.21.15.47:0/3565203357 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5a2c013550 con 0x7f5a3c150510 2024-03-23T09:43:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.244+0000 7f5a23fff640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5a2806ac00 0x7f5a2806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.245+0000 7f5a23fff640 1 -- 172.21.15.47:0/3565203357 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a2c089550 con 0x7f5a3c150510 2024-03-23T09:43:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.245+0000 7f5a3affd640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5a2806ac00 0x7f5a2806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.245+0000 7f5a3affd640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5a2806ac00 0x7f5a2806d0c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c150cf0 tx=0x7f5a30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:32.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.249+0000 7f5a23fff640 1 -- 172.21.15.47:0/3565203357 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5a3c0c8260 con 0x7f5a3c150510 2024-03-23T09:43:32.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.428+0000 7f5a41eea640 1 -- 172.21.15.47:0/3565203357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a3c0c8260 con 0x7f5a3c150510 2024-03-23T09:43:32.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.428+0000 7f5a23fff640 1 -- 172.21.15.47:0/3565203357 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a2c0547e0 con 0x7f5a3c150510 2024-03-23T09:43:32.428 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:43:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.430+0000 7f5a41eea640 1 -- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5a2806ac00 msgr2=0x7f5a2806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.430+0000 7f5a41eea640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5a2806ac00 0x7f5a2806d0c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c150cf0 tx=0x7f5a30008040 comp rx=0 tx=0).stop 2024-03-23T09:43:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.430+0000 7f5a41eea640 1 -- 172.21.15.47:0/3565203357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a3c150510 msgr2=0x7f5a3c0c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.430+0000 7f5a41eea640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a3c150510 0x7f5a3c0c8540 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0072c0 tx=0x7f5a2c007780 comp rx=0 tx=0).stop 2024-03-23T09:43:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.430+0000 7f5a41eea640 1 -- 172.21.15.47:0/3565203357 shutdown_connections 2024-03-23T09:43:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.430+0000 7f5a41eea640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5a2806ac00 0x7f5a2806d0c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.430+0000 7f5a41eea640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a3c0c9480 0x7f5a3c0af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.430+0000 7f5a41eea640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a3c1508f0 0x7f5a3c0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.430+0000 7f5a41eea640 1 --2- 172.21.15.47:0/3565203357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a3c150510 0x7f5a3c0c8540 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.430+0000 7f5a41eea640 1 -- 172.21.15.47:0/3565203357 >> 172.21.15.47:0/3565203357 conn(0x7f5a3c0d06f0 msgr2=0x7f5a3c06fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.431+0000 7f5a41eea640 1 -- 172.21.15.47:0/3565203357 shutdown_connections 2024-03-23T09:43:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.431+0000 7f5a41eea640 1 -- 172.21.15.47:0/3565203357 wait complete. 2024-03-23T09:43:32.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:43:32.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.545+0000 7f40814b3640 1 Processor -- start 2024-03-23T09:43:32.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.545+0000 7f40814b3640 1 -- start start 2024-03-23T09:43:32.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f40814b3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f407c0cde10 0x7f407c160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f40814b3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f407c0b8350 0x7f407c0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f40814b3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407c161630 con 0x7f407c15c990 2024-03-23T09:43:32.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f40814b3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f407c0d1030 con 0x7f407c0cde10 2024-03-23T09:43:32.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f40814b3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c0cfe10 con 0x7f407c0b8350 2024-03-23T09:43:32.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f4073fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f407c0cde10 0x7f407c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f4073fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f407c0cde10 0x7f407c160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50348/0 (socket says 172.21.15.47:50348) 2024-03-23T09:43:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f4073fff640 1 -- 172.21.15.47:0/1908091460 learned_addr learned my addr 172.21.15.47:0/1908091460 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f407b7fe640 1 --2- 172.21.15.47:0/1908091460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f407c0b8350 0x7f407c0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f4073fff640 1 -- 172.21.15.47:0/1908091460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f407c0b8350 msgr2=0x7f407c0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f4073fff640 1 --2- 172.21.15.47:0/1908091460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f407c0b8350 0x7f407c0d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f4073fff640 1 -- 172.21.15.47:0/1908091460 >> v1:172.21.15.175:6789/0 conn(0x7f407c15c990 legacy=0x7f407c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f4073fff640 1 -- 172.21.15.47:0/1908091460 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c0b5b60 con 0x7f407c0cde10 2024-03-23T09:43:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.546+0000 7f407b7fe640 1 --2- 172.21.15.47:0/1908091460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f407c0b8350 0x7f407c0d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.547+0000 7f4073fff640 1 --2- 172.21.15.47:0/1908091460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c0cde10 0x7f407c160690 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4064009870 tx=0x7f406402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8279bd60031b2f21 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.547+0000 7f407a7fc640 1 -- 172.21.15.47:0/1908091460 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064031640 con 0x7f407c0cde10 2024-03-23T09:43:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.547+0000 7f407a7fc640 1 -- 172.21.15.47:0/1908091460 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40640317a0 con 0x7f407c0cde10 2024-03-23T09:43:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.547+0000 7f407a7fc640 1 -- 172.21.15.47:0/1908091460 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064031b30 con 0x7f407c0cde10 2024-03-23T09:43:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.548+0000 7f40814b3640 1 -- 172.21.15.47:0/1908091460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c0cde10 msgr2=0x7f407c160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.548+0000 7f40814b3640 1 --2- 172.21.15.47:0/1908091460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c0cde10 0x7f407c160690 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4064009870 tx=0x7f406402b8f0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.548+0000 7f40814b3640 1 -- 172.21.15.47:0/1908091460 shutdown_connections 2024-03-23T09:43:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.548+0000 7f40814b3640 1 --2- 172.21.15.47:0/1908091460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f407c0b8350 0x7f407c0d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.548+0000 7f40814b3640 1 --2- 172.21.15.47:0/1908091460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c0cde10 0x7f407c160690 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.548+0000 7f40814b3640 1 -- 172.21.15.47:0/1908091460 >> 172.21.15.47:0/1908091460 conn(0x7f407c0659d0 msgr2=0x7f407c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.548+0000 7f40814b3640 1 -- 172.21.15.47:0/1908091460 shutdown_connections 2024-03-23T09:43:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.548+0000 7f40814b3640 1 -- 172.21.15.47:0/1908091460 wait complete. 2024-03-23T09:43:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.549+0000 7f40814b3640 1 Processor -- start 2024-03-23T09:43:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.549+0000 7f40814b3640 1 -- start start 2024-03-23T09:43:32.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f40814b3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c0b8350 0x7f407c15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f40814b3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f407c0cde10 0x7f407c153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f40814b3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f407c15c990 0x7f407c0b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f40814b3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c159710 con 0x7f407c0b8350 2024-03-23T09:43:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f40814b3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c1599f0 con 0x7f407c0cde10 2024-03-23T09:43:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f40814b3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c159cd0 con 0x7f407c15c990 2024-03-23T09:43:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f407affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c0b8350 0x7f407c15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f407affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c0b8350 0x7f407c15ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50356/0 (socket says 172.21.15.47:50356) 2024-03-23T09:43:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f407affd640 1 -- 172.21.15.47:0/2456883199 learned_addr learned my addr 172.21.15.47:0/2456883199 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f407b7fe640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f407c15c990 0x7f407c0b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f4073fff640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f407c0cde10 0x7f407c153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f407affd640 1 -- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f407c15c990 msgr2=0x7f407c0b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f407affd640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f407c15c990 0x7f407c0b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f407affd640 1 -- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f407c0cde10 msgr2=0x7f407c153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f407affd640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f407c0cde10 0x7f407c153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f407affd640 1 -- 172.21.15.47:0/2456883199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40640094d0 con 0x7f407c0b8350 2024-03-23T09:43:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f407affd640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c0b8350 0x7f407c15ab00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f406c00c400 tx=0x7f406c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f40737fe640 1 -- 172.21.15.47:0/2456883199 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c0052d0 con 0x7f407c0b8350 2024-03-23T09:43:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f40737fe640 1 -- 172.21.15.47:0/2456883199 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406c005430 con 0x7f407c0b8350 2024-03-23T09:43:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f40814b3640 1 -- 172.21.15.47:0/2456883199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407c159fb0 con 0x7f407c0b8350 2024-03-23T09:43:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f40737fe640 1 -- 172.21.15.47:0/2456883199 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c003830 con 0x7f407c0b8350 2024-03-23T09:43:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.550+0000 7f40814b3640 1 -- 172.21.15.47:0/2456883199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407c15b040 con 0x7f407c0b8350 2024-03-23T09:43:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.551+0000 7f40814b3640 1 -- 172.21.15.47:0/2456883199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407c072360 con 0x7f407c0b8350 2024-03-23T09:43:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.552+0000 7f40737fe640 1 -- 172.21.15.47:0/2456883199 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f406c003990 con 0x7f407c0b8350 2024-03-23T09:43:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.552+0000 7f40737fe640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f405c06ac30 0x7f405c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.552+0000 7f40737fe640 1 -- 172.21.15.47:0/2456883199 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f406c0888a0 con 0x7f407c0b8350 2024-03-23T09:43:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.552+0000 7f4073fff640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f405c06ac30 0x7f405c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.553+0000 7f4073fff640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f405c06ac30 0x7f405c06d0f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4064036000 tx=0x7f406403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:32.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.556+0000 7f40737fe640 1 -- 172.21.15.47:0/2456883199 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f406c053a80 con 0x7f407c0b8350 2024-03-23T09:43:32.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.733+0000 7f40814b3640 1 -- 172.21.15.47:0/2456883199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f407c0ce680 con 0x7f407c0b8350 2024-03-23T09:43:32.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.734+0000 7f40737fe640 1 -- 172.21.15.47:0/2456883199 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7f406c017020 con 0x7f407c0b8350 2024-03-23T09:43:32.734 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:43:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.736+0000 7f40814b3640 1 -- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f405c06ac30 msgr2=0x7f405c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.736+0000 7f40814b3640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f405c06ac30 0x7f405c06d0f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4064036000 tx=0x7f406403c040 comp rx=0 tx=0).stop 2024-03-23T09:43:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.736+0000 7f40814b3640 1 -- 172.21.15.47:0/2456883199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c0b8350 msgr2=0x7f407c15ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.736+0000 7f40814b3640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c0b8350 0x7f407c15ab00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f406c00c400 tx=0x7f406c00c8c0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.736+0000 7f40814b3640 1 -- 172.21.15.47:0/2456883199 shutdown_connections 2024-03-23T09:43:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.736+0000 7f40814b3640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f405c06ac30 0x7f405c06d0f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.736+0000 7f40814b3640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f407c15c990 0x7f407c0b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.736+0000 7f40814b3640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f407c0cde10 0x7f407c153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.736+0000 7f40814b3640 1 --2- 172.21.15.47:0/2456883199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c0b8350 0x7f407c15ab00 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.736+0000 7f40814b3640 1 -- 172.21.15.47:0/2456883199 >> 172.21.15.47:0/2456883199 conn(0x7f407c0659d0 msgr2=0x7f407c0b4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.736+0000 7f40814b3640 1 -- 172.21.15.47:0/2456883199 shutdown_connections 2024-03-23T09:43:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.736+0000 7f40814b3640 1 -- 172.21.15.47:0/2456883199 wait complete. 2024-03-23T09:43:32.746 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:43:32.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.848+0000 7f9ff6e96640 1 Processor -- start 2024-03-23T09:43:32.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.848+0000 7f9ff6e96640 1 -- start start 2024-03-23T09:43:32.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.849+0000 7f9ff6e96640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9ff00b8500 0x7f9ff00d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.849+0000 7f9ff6e96640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff00b6090 con 0x7f9ff015cb60 2024-03-23T09:43:32.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.849+0000 7f9ff6e96640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff00b6310 con 0x7f9ff00d0e10 2024-03-23T09:43:32.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.849+0000 7f9ff6e96640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff00b6f90 con 0x7f9ff00b8500 2024-03-23T09:43:32.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.849+0000 7f9ff4c32640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9ff015cb60 0x7f9ff00b5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45902/0 (socket says 172.21.15.47:45902) 2024-03-23T09:43:32.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.849+0000 7f9ff4c32640 1 -- 172.21.15.47:0/1373394733 learned_addr learned my addr 172.21.15.47:0/1373394733 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:32.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.849+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523880880 0 0) 0x7f9ff00b6090 con 0x7f9ff015cb60 2024-03-23T09:43:32.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9feffff640 1 --2- 172.21.15.47:0/1373394733 >> v2:172.21.15.175:3300/0 conn(0x7f9ff00b8500 0x7f9ff00d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fd8003520 con 0x7f9ff015cb60 2024-03-23T09:43:32.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3315240697 0 0) 0x7f9fd8003520 con 0x7f9ff015cb60 2024-03-23T09:43:32.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 >> v2:172.21.15.175:3300/0 conn(0x7f9ff00b8500 msgr2=0x7f9ff00d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:32.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 --2- 172.21.15.47:0/1373394733 >> v2:172.21.15.175:3300/0 conn(0x7f9ff00b8500 0x7f9ff00d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 >> v1:172.21.15.175:6789/0 conn(0x7f9ff00d0e10 legacy=0x7f9ff00cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff00b65f0 con 0x7f9ff015cb60 2024-03-23T09:43:32.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f9fe80019f0 con 0x7f9ff015cb60 2024-03-23T09:43:32.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 >> v1:172.21.15.47:6789/0 conn(0x7f9ff015cb60 legacy=0x7f9ff00b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 --2- 172.21.15.47:0/1373394733 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd8003970 0x7f9fd8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 --2- 172.21.15.47:0/1373394733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd80047c0 0x7f9fd8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 --2- 172.21.15.47:0/1373394733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fd8005180 0x7f9fd800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff00b6f90 con 0x7f9fd80047c0 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd800c120 con 0x7f9fd8005180 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd800c2e0 con 0x7f9fd8003970 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9feffff640 1 --2- 172.21.15.47:0/1373394733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd80047c0 0x7f9fd8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9ff5433640 1 --2- 172.21.15.47:0/1373394733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fd8005180 0x7f9fd800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9ff4c32640 1 --2- 172.21.15.47:0/1373394733 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd8003970 0x7f9fd8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9feffff640 1 -- 172.21.15.47:0/1373394733 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd8003970 msgr2=0x7f9fd8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9feffff640 1 --2- 172.21.15.47:0/1373394733 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd8003970 0x7f9fd8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9feffff640 1 -- 172.21.15.47:0/1373394733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fd8005180 msgr2=0x7f9fd800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9feffff640 1 --2- 172.21.15.47:0/1373394733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fd8005180 0x7f9fd800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9feffff640 1 -- 172.21.15.47:0/1373394733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd8003520 con 0x7f9fd80047c0 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.850+0000 7f9ff5433640 1 -- 172.21.15.47:0/1373394733 reap_dead start 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.851+0000 7f9feffff640 1 --2- 172.21.15.47:0/1373394733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd80047c0 0x7f9fd8004c40 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009ac0 tx=0x7f9fd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ce1b4d598ffac1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.851+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe402c6b0 con 0x7f9fd80047c0 2024-03-23T09:43:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.851+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe402c810 con 0x7f9fd80047c0 2024-03-23T09:43:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.851+0000 7f9fef7fe640 1 -- 172.21.15.47:0/1373394733 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe402cba0 con 0x7f9fd80047c0 2024-03-23T09:43:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.852+0000 7f9ff6e96640 1 -- 172.21.15.47:0/1373394733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd80047c0 msgr2=0x7f9fd8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.852+0000 7f9ff6e96640 1 --2- 172.21.15.47:0/1373394733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd80047c0 0x7f9fd8004c40 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009ac0 tx=0x7f9fd80057f0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.852+0000 7f9ff6e96640 1 -- 172.21.15.47:0/1373394733 shutdown_connections 2024-03-23T09:43:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.852+0000 7f9ff6e96640 1 --2- 172.21.15.47:0/1373394733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd80047c0 0x7f9fd8004c40 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.852+0000 7f9ff6e96640 1 -- 172.21.15.47:0/1373394733 >> 172.21.15.47:0/1373394733 conn(0x7f9ff0065df0 msgr2=0x7f9ff0066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.852+0000 7f9ff6e96640 1 -- 172.21.15.47:0/1373394733 shutdown_connections 2024-03-23T09:43:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.852+0000 7f9ff6e96640 1 -- 172.21.15.47:0/1373394733 wait complete. 2024-03-23T09:43:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.853+0000 7f9ff6e96640 1 Processor -- start 2024-03-23T09:43:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.853+0000 7f9ff6e96640 1 -- start start 2024-03-23T09:43:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9ff6e96640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd80047c0 0x7f9ff014cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9ff6e96640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00cb290 0x7f9ff00cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9ff6e96640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff015f420 0x7f9ff015f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9ff6e96640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff015fe10 con 0x7f9ff00cb290 2024-03-23T09:43:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9ff6e96640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff015dda0 con 0x7f9ff015f420 2024-03-23T09:43:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9ff6e96640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff00cbd30 con 0x7f9fd80047c0 2024-03-23T09:43:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9feffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00cb290 0x7f9ff00cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9feffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00cb290 0x7f9ff00cb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50370/0 (socket says 172.21.15.47:50370) 2024-03-23T09:43:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9feffff640 1 -- 172.21.15.47:0/3674987232 learned_addr learned my addr 172.21.15.47:0/3674987232 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9ff4c32640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd80047c0 0x7f9ff014cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9ff5433640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff015f420 0x7f9ff015f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9feffff640 1 -- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd80047c0 msgr2=0x7f9ff014cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9feffff640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd80047c0 0x7f9ff014cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9feffff640 1 -- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff015f420 msgr2=0x7f9ff015f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9feffff640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff015f420 0x7f9ff015f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9feffff640 1 -- 172.21.15.47:0/3674987232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff015cfe0 con 0x7f9ff00cb290 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9ff5433640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff015f420 0x7f9ff015f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.854+0000 7f9feffff640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00cb290 0x7f9ff00cb710 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009ac0 tx=0x7f9fe4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.855+0000 7f9fed7fa640 1 -- 172.21.15.47:0/3674987232 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4003810 con 0x7f9ff00cb290 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.855+0000 7f9fed7fa640 1 -- 172.21.15.47:0/3674987232 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe4003970 con 0x7f9ff00cb290 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.855+0000 7f9ff6e96640 1 -- 172.21.15.47:0/3674987232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff015d2c0 con 0x7f9ff00cb290 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.855+0000 7f9fed7fa640 1 -- 172.21.15.47:0/3674987232 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4003d00 con 0x7f9ff00cb290 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.855+0000 7f9ff6e96640 1 -- 172.21.15.47:0/3674987232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff015d880 con 0x7f9ff00cb290 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.856+0000 7f9ff6e96640 1 -- 172.21.15.47:0/3674987232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff015da40 con 0x7f9ff00cb290 2024-03-23T09:43:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.856+0000 7f9fed7fa640 1 -- 172.21.15.47:0/3674987232 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9fe4040450 con 0x7f9ff00cb290 2024-03-23T09:43:32.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.856+0000 7f9fed7fa640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9fdc06ac30 0x7f9fdc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:32.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.856+0000 7f9fed7fa640 1 -- 172.21.15.47:0/3674987232 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fe40ab180 con 0x7f9ff00cb290 2024-03-23T09:43:32.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.857+0000 7f9ff4c32640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9fdc06ac30 0x7f9fdc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:32.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.857+0000 7f9ff4c32640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9fdc06ac30 0x7f9fdc06d0f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9ff015dfb0 tx=0x7f9fe8006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:32.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:32.861+0000 7f9fed7fa640 1 -- 172.21.15.47:0/3674987232 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9fe4076410 con 0x7f9ff00cb290 2024-03-23T09:43:33.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.053+0000 7f9ff6e96640 1 -- 172.21.15.47:0/3674987232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ff0072360 con 0x7f9ff00cb290 2024-03-23T09:43:33.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.053+0000 7f9fed7fa640 1 -- 172.21.15.47:0/3674987232 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9fe4003e60 con 0x7f9ff00cb290 2024-03-23T09:43:33.053 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:43:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.055+0000 7f9ff6e96640 1 -- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9fdc06ac30 msgr2=0x7f9fdc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.055+0000 7f9ff6e96640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9fdc06ac30 0x7f9fdc06d0f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9ff015dfb0 tx=0x7f9fe8006040 comp rx=0 tx=0).stop 2024-03-23T09:43:33.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.055+0000 7f9ff6e96640 1 -- 172.21.15.47:0/3674987232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00cb290 msgr2=0x7f9ff00cb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:33.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.055+0000 7f9ff6e96640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00cb290 0x7f9ff00cb710 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009ac0 tx=0x7f9fe4004730 comp rx=0 tx=0).stop 2024-03-23T09:43:33.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.055+0000 7f9ff6e96640 1 -- 172.21.15.47:0/3674987232 shutdown_connections 2024-03-23T09:43:33.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.055+0000 7f9ff6e96640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9fdc06ac30 0x7f9fdc06d0f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.055+0000 7f9ff6e96640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff015f420 0x7f9ff015f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.055+0000 7f9ff6e96640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00cb290 0x7f9ff00cb710 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.055+0000 7f9ff6e96640 1 --2- 172.21.15.47:0/3674987232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd80047c0 0x7f9ff014cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.055+0000 7f9ff6e96640 1 -- 172.21.15.47:0/3674987232 >> 172.21.15.47:0/3674987232 conn(0x7f9ff0065df0 msgr2=0x7f9ff00d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:33.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.055+0000 7f9ff6e96640 1 -- 172.21.15.47:0/3674987232 shutdown_connections 2024-03-23T09:43:33.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.055+0000 7f9ff6e96640 1 -- 172.21.15.47:0/3674987232 wait complete. 2024-03-23T09:43:33.066 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:43:33.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:33.066 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:43:33.091 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[54700]: fuse finished with error 0 and tester_r 0 2024-03-23T09:43:33.092 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:43:33.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:33.093 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:43:33.165 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:43:33.165 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:43:33.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:43:33.183 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:43:33.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:33.183 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-23T09:43:33.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:43:33.239 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-23T09:43:33.239 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:43:33.239 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:43:33.254 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:43:33.254 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:43:33.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:33.254 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-23T09:43:33.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:43:33.308 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-23T09:43:33.308 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:43:33.308 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:43:33.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:43:33.325 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:43:33.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:33.325 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-23T09:43:33.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:43:33.380 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-23T09:43:33.380 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:43:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.482+0000 7f7e32947640 1 Processor -- start 2024-03-23T09:43:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.482+0000 7f7e32947640 1 -- start start 2024-03-23T09:43:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.482+0000 7f7e32947640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e2c0cd7f0 0x7f7e2c0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.482+0000 7f7e32947640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c15ae50 con 0x7f7e2c15aa70 2024-03-23T09:43:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.482+0000 7f7e32947640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c0cf010 con 0x7f7e2c0ce800 2024-03-23T09:43:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.482+0000 7f7e32947640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0cf2f0 con 0x7f7e2c0cd7f0 2024-03-23T09:43:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.483+0000 7f7e2bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7e2c15aa70 0x7f7e2c0cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45916/0 (socket says 172.21.15.47:45916) 2024-03-23T09:43:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.483+0000 7f7e2bfff640 1 -- 172.21.15.47:0/1278116402 learned_addr learned my addr 172.21.15.47:0/1278116402 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.483+0000 7f7e2b7fe640 1 --2- 172.21.15.47:0/1278116402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e2c0cd7f0 0x7f7e2c0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.483+0000 7f7e2affd640 1 -- 172.21.15.47:0/1278116402 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771736086 0 0) 0x7f7e2c15ae50 con 0x7f7e2c15aa70 2024-03-23T09:43:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.483+0000 7f7e2affd640 1 -- 172.21.15.47:0/1278116402 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e14003520 con 0x7f7e2c15aa70 2024-03-23T09:43:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.483+0000 7f7e2affd640 1 -- 172.21.15.47:0/1278116402 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420585132 0 0) 0x7f7e2c0cf010 con 0x7f7e2c0ce800 2024-03-23T09:43:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.483+0000 7f7e2affd640 1 -- 172.21.15.47:0/1278116402 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e2c15ae50 con 0x7f7e2c0ce800 2024-03-23T09:43:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.483+0000 7f7e2b7fe640 1 -- 172.21.15.47:0/1278116402 >> v1:172.21.15.175:6789/0 conn(0x7f7e2c0ce800 legacy=0x7f7e2c0cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.483+0000 7f7e2affd640 1 -- 172.21.15.47:0/1278116402 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2362544155 0 0) 0x7f7e14003520 con 0x7f7e2c15aa70 2024-03-23T09:43:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.483+0000 7f7e2b7fe640 1 -- 172.21.15.47:0/1278116402 >> v1:172.21.15.47:6789/0 conn(0x7f7e2c15aa70 legacy=0x7f7e2c0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.483+0000 7f7e2b7fe640 1 -- 172.21.15.47:0/1278116402 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0cf5d0 con 0x7f7e2c0cd7f0 2024-03-23T09:43:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.484+0000 7f7e2b7fe640 1 --2- 172.21.15.47:0/1278116402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e2c0cd7f0 0x7f7e2c0ce2c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7e200097d0 tx=0x7f7e2002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a5f33e5b73bbe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.484+0000 7f7e2affd640 1 -- 172.21.15.47:0/1278116402 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c001670 con 0x7f7e2c0cd7f0 2024-03-23T09:43:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.484+0000 7f7e2affd640 1 -- 172.21.15.47:0/1278116402 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e20031580 con 0x7f7e2c0cd7f0 2024-03-23T09:43:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.484+0000 7f7e2affd640 1 -- 172.21.15.47:0/1278116402 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e200318f0 con 0x7f7e2c0cd7f0 2024-03-23T09:43:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.485+0000 7f7e32947640 1 -- 172.21.15.47:0/1278116402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e2c0cd7f0 msgr2=0x7f7e2c0ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.485+0000 7f7e32947640 1 --2- 172.21.15.47:0/1278116402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e2c0cd7f0 0x7f7e2c0ce2c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7e200097d0 tx=0x7f7e2002b8a0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.485+0000 7f7e32947640 1 -- 172.21.15.47:0/1278116402 shutdown_connections 2024-03-23T09:43:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.485+0000 7f7e32947640 1 --2- 172.21.15.47:0/1278116402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e2c0cd7f0 0x7f7e2c0ce2c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.485+0000 7f7e32947640 1 -- 172.21.15.47:0/1278116402 >> 172.21.15.47:0/1278116402 conn(0x7f7e2c0d06f0 msgr2=0x7f7e2c0d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.485+0000 7f7e32947640 1 -- 172.21.15.47:0/1278116402 shutdown_connections 2024-03-23T09:43:33.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.485+0000 7f7e32947640 1 -- 172.21.15.47:0/1278116402 wait complete. 2024-03-23T09:43:33.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.486+0000 7f7e32947640 1 Processor -- start 2024-03-23T09:43:33.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.486+0000 7f7e32947640 1 -- start start 2024-03-23T09:43:33.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e32947640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e2c0cd7f0 0x7f7e2c1ef980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:33.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e32947640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e2c0ce800 0x7f7e2c150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:33.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e32947640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2c15aa70 0x7f7e2c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e32947640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c1547c0 con 0x7f7e2c15aa70 2024-03-23T09:43:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e32947640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c1efec0 con 0x7f7e2c0cd7f0 2024-03-23T09:43:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e32947640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c153720 con 0x7f7e2c0ce800 2024-03-23T09:43:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e30ee4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2c15aa70 0x7f7e2c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e30ee4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2c15aa70 0x7f7e2c150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50372/0 (socket says 172.21.15.47:50372) 2024-03-23T09:43:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e30ee4640 1 -- 172.21.15.47:0/3106827646 learned_addr learned my addr 172.21.15.47:0/3106827646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e2bfff640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e2c0cd7f0 0x7f7e2c1ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e2b7fe640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e2c0ce800 0x7f7e2c150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e30ee4640 1 -- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e2c0ce800 msgr2=0x7f7e2c150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e30ee4640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e2c0ce800 0x7f7e2c150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e30ee4640 1 -- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e2c0cd7f0 msgr2=0x7f7e2c1ef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e30ee4640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e2c0cd7f0 0x7f7e2c1ef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e30ee4640 1 -- 172.21.15.47:0/3106827646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20009400 con 0x7f7e2c15aa70 2024-03-23T09:43:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e30ee4640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2c15aa70 0x7f7e2c150ab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c790 tx=0x7f7e2400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e28ff9640 1 -- 172.21.15.47:0/3106827646 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2400cea0 con 0x7f7e2c15aa70 2024-03-23T09:43:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e28ff9640 1 -- 172.21.15.47:0/3106827646 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e24003b70 con 0x7f7e2c15aa70 2024-03-23T09:43:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e32947640 1 -- 172.21.15.47:0/3106827646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c153a00 con 0x7f7e2c15aa70 2024-03-23T09:43:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.487+0000 7f7e28ff9640 1 -- 172.21.15.47:0/3106827646 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2400a850 con 0x7f7e2c15aa70 2024-03-23T09:43:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.488+0000 7f7e32947640 1 -- 172.21.15.47:0/3106827646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c153fc0 con 0x7f7e2c15aa70 2024-03-23T09:43:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.488+0000 7f7e32947640 1 -- 172.21.15.47:0/3106827646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c0c7030 con 0x7f7e2c15aa70 2024-03-23T09:43:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.489+0000 7f7e28ff9640 1 -- 172.21.15.47:0/3106827646 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7e2400a9b0 con 0x7f7e2c15aa70 2024-03-23T09:43:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.489+0000 7f7e28ff9640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e1806ac00 0x7f7e1806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.489+0000 7f7e28ff9640 1 -- 172.21.15.47:0/3106827646 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e24089160 con 0x7f7e2c15aa70 2024-03-23T09:43:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.489+0000 7f7e2bfff640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e1806ac00 0x7f7e1806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.490+0000 7f7e2bfff640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e1806ac00 0x7f7e1806d0c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0011e0 tx=0x7f7e1c001670 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.493+0000 7f7e28ff9640 1 -- 172.21.15.47:0/3106827646 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7e24054370 con 0x7f7e2c15aa70 2024-03-23T09:43:33.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.708+0000 7f7e32947640 1 -- 172.21.15.47:0/3106827646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e2c066be0 con 0x7f7e2c15aa70 2024-03-23T09:43:33.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.709+0000 7f7e28ff9640 1 -- 172.21.15.47:0/3106827646 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f7e24057990 con 0x7f7e2c15aa70 2024-03-23T09:43:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:33.709 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1189831856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1189831856},{"type":"v1","addr":"172.21.15.47:6835","nonce":1189831856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/2248064454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2248064454},{"type":"v1","addr":"172.21.15.47:6834","nonce":2248064454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5694,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3806465612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3806465612},{"type":"v1","addr":"172.21.15.175:6835","nonce":3806465612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:42:48.165819+0000","modified":"2024-03-23T09:42:50.171529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1626348200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1626348200},{"type":"v1","addr":"172.21.15.175:6837","nonce":1626348200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-23T09:43:33.709 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 18 2024-03-23T09:43:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.710+0000 7f7e32947640 1 -- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e1806ac00 msgr2=0x7f7e1806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.710+0000 7f7e32947640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e1806ac00 0x7f7e1806d0c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0011e0 tx=0x7f7e1c001670 comp rx=0 tx=0).stop 2024-03-23T09:43:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.710+0000 7f7e32947640 1 -- 172.21.15.47:0/3106827646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2c15aa70 msgr2=0x7f7e2c150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.710+0000 7f7e32947640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2c15aa70 0x7f7e2c150ab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c790 tx=0x7f7e2400cc50 comp rx=0 tx=0).stop 2024-03-23T09:43:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.710+0000 7f7e32947640 1 -- 172.21.15.47:0/3106827646 shutdown_connections 2024-03-23T09:43:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.710+0000 7f7e32947640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e1806ac00 0x7f7e1806d0c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.710+0000 7f7e32947640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2c15aa70 0x7f7e2c150ab0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.710+0000 7f7e32947640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e2c0ce800 0x7f7e2c150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.710+0000 7f7e32947640 1 --2- 172.21.15.47:0/3106827646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e2c0cd7f0 0x7f7e2c1ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.710+0000 7f7e32947640 1 -- 172.21.15.47:0/3106827646 >> 172.21.15.47:0/3106827646 conn(0x7f7e2c0d06f0 msgr2=0x7f7e2c06e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.711+0000 7f7e32947640 1 -- 172.21.15.47:0/3106827646 shutdown_connections 2024-03-23T09:43:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.711+0000 7f7e32947640 1 -- 172.21.15.47:0/3106827646 wait complete. 2024-03-23T09:43:33.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:43:33.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.822+0000 7fd1749d2640 1 Processor -- start 2024-03-23T09:43:33.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.822+0000 7fd1749d2640 1 -- start start 2024-03-23T09:43:33.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.822+0000 7fd1749d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1700cf210 0x7fd1700cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:33.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.822+0000 7fd1749d2640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd1700b5a90 0x7fd1700b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:33.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.822+0000 7fd1749d2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1700cfbd0 con 0x7fd17015cd00 2024-03-23T09:43:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.822+0000 7fd1749d2640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17015d0e0 con 0x7fd1700b5a90 2024-03-23T09:43:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.822+0000 7fd1749d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700d1540 con 0x7fd1700cf210 2024-03-23T09:43:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.823+0000 7fd16e59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd17015cd00 0x7fd1700d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45924/0 (socket says 172.21.15.47:45924) 2024-03-23T09:43:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.823+0000 7fd16e59c640 1 -- 172.21.15.47:0/1332437215 learned_addr learned my addr 172.21.15.47:0/1332437215 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.823+0000 7fd167fff640 1 --2- 172.21.15.47:0/1332437215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1700cf210 0x7fd1700cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.823+0000 7fd16ed9d640 1 --2- 172.21.15.47:0/1332437215 >> v2:172.21.15.175:3300/0 conn(0x7fd1700b5a90 0x7fd1700b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.823+0000 7fd16dd9b640 1 -- 172.21.15.47:0/1332437215 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709306291 0 0) 0x7fd1700cfbd0 con 0x7fd17015cd00 2024-03-23T09:43:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.823+0000 7fd16dd9b640 1 -- 172.21.15.47:0/1332437215 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd15c003520 con 0x7fd17015cd00 2024-03-23T09:43:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.823+0000 7fd16ed9d640 1 -- 172.21.15.47:0/1332437215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1700cf210 msgr2=0x7fd1700cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.823+0000 7fd16ed9d640 1 --2- 172.21.15.47:0/1332437215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1700cf210 0x7fd1700cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.823+0000 7fd16ed9d640 1 -- 172.21.15.47:0/1332437215 >> v1:172.21.15.47:6789/0 conn(0x7fd17015cd00 legacy=0x7fd1700d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.823+0000 7fd16dd9b640 1 -- 172.21.15.47:0/1332437215 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2992940546 0 0) 0x7fd15c003520 con 0x7fd17015cd00 2024-03-23T09:43:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.823+0000 7fd16ed9d640 1 -- 172.21.15.47:0/1332437215 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1700d1820 con 0x7fd1700b5a90 2024-03-23T09:43:33.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.824+0000 7fd16ed9d640 1 --2- 172.21.15.47:0/1332437215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1700b5a90 0x7fd1700b5f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd168009d90 tx=0x7fd16802bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cefb86dc0abaf465 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.824+0000 7fd16dd9b640 1 -- 172.21.15.47:0/1332437215 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160001670 con 0x7fd1700b5a90 2024-03-23T09:43:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.824+0000 7fd16dd9b640 1 -- 172.21.15.47:0/1332437215 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16802c810 con 0x7fd1700b5a90 2024-03-23T09:43:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.824+0000 7fd16dd9b640 1 -- 172.21.15.47:0/1332437215 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16802cb80 con 0x7fd1700b5a90 2024-03-23T09:43:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.825+0000 7fd1749d2640 1 -- 172.21.15.47:0/1332437215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1700b5a90 msgr2=0x7fd1700b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.825+0000 7fd1749d2640 1 --2- 172.21.15.47:0/1332437215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1700b5a90 0x7fd1700b5f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd168009d90 tx=0x7fd16802bce0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.825+0000 7fd1749d2640 1 -- 172.21.15.47:0/1332437215 shutdown_connections 2024-03-23T09:43:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.825+0000 7fd1749d2640 1 --2- 172.21.15.47:0/1332437215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1700b5a90 0x7fd1700b5f20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.825+0000 7fd1749d2640 1 --2- 172.21.15.47:0/1332437215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1700cf210 0x7fd1700cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.825+0000 7fd1749d2640 1 -- 172.21.15.47:0/1332437215 >> 172.21.15.47:0/1332437215 conn(0x7fd17006b160 msgr2=0x7fd17006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.825+0000 7fd1749d2640 1 -- 172.21.15.47:0/1332437215 shutdown_connections 2024-03-23T09:43:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.825+0000 7fd1749d2640 1 -- 172.21.15.47:0/1332437215 wait complete. 2024-03-23T09:43:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.826+0000 7fd1749d2640 1 Processor -- start 2024-03-23T09:43:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.826+0000 7fd1749d2640 1 -- start start 2024-03-23T09:43:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd1749d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1700b5a90 0x7fd1700b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd1749d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1700cf210 0x7fd170150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd1749d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd17015cd00 0x7fd1700cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd1749d2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700b3270 con 0x7fd1700cf210 2024-03-23T09:43:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd1749d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700b3550 con 0x7fd1700b5a90 2024-03-23T09:43:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd1749d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700b3830 con 0x7fd17015cd00 2024-03-23T09:43:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd167fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1700cf210 0x7fd170150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd167fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1700cf210 0x7fd170150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50388/0 (socket says 172.21.15.47:50388) 2024-03-23T09:43:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd167fff640 1 -- 172.21.15.47:0/3363503243 learned_addr learned my addr 172.21.15.47:0/3363503243 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd16ed9d640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd17015cd00 0x7fd1700cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:33.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd16e59c640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1700b5a90 0x7fd1700b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:33.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd167fff640 1 -- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd17015cd00 msgr2=0x7fd1700cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:33.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd167fff640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd17015cd00 0x7fd1700cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd167fff640 1 -- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1700b5a90 msgr2=0x7fd1700b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:33.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd167fff640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1700b5a90 0x7fd1700b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:33.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd167fff640 1 -- 172.21.15.47:0/3363503243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009a70 con 0x7fd1700cf210 2024-03-23T09:43:33.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd16ed9d640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd17015cd00 0x7fd1700cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:33.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd167fff640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1700cf210 0x7fd170150bb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd15800e9b0 tx=0x7fd15800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:33.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.827+0000 7fd166ffd640 1 -- 172.21.15.47:0/3363503243 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158009670 con 0x7fd1700cf210 2024-03-23T09:43:33.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.828+0000 7fd166ffd640 1 -- 172.21.15.47:0/3363503243 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15800c9a0 con 0x7fd1700cf210 2024-03-23T09:43:33.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.828+0000 7fd1749d2640 1 -- 172.21.15.47:0/3363503243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1700b3b10 con 0x7fd1700cf210 2024-03-23T09:43:33.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.828+0000 7fd166ffd640 1 -- 172.21.15.47:0/3363503243 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15800cd50 con 0x7fd1700cf210 2024-03-23T09:43:33.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.828+0000 7fd1749d2640 1 -- 172.21.15.47:0/3363503243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1700b4fb0 con 0x7fd1700cf210 2024-03-23T09:43:33.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.829+0000 7fd1749d2640 1 -- 172.21.15.47:0/3363503243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd170072360 con 0x7fd1700cf210 2024-03-23T09:43:33.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.829+0000 7fd166ffd640 1 -- 172.21.15.47:0/3363503243 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd158013470 con 0x7fd1700cf210 2024-03-23T09:43:33.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.829+0000 7fd166ffd640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd15006ac30 0x7fd15006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:33.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.830+0000 7fd166ffd640 1 -- 172.21.15.47:0/3363503243 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd158089460 con 0x7fd1700cf210 2024-03-23T09:43:33.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.830+0000 7fd16e59c640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd15006ac30 0x7fd15006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:33.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.830+0000 7fd16e59c640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd15006ac30 0x7fd15006d0f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd160006ad0 tx=0x7fd160001d20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:33.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:33.834+0000 7fd166ffd640 1 -- 172.21.15.47:0/3363503243 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd158054780 con 0x7fd1700cf210 2024-03-23T09:43:34.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.013+0000 7fd1749d2640 1 -- 172.21.15.47:0/3363503243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1700b51d0 con 0x7fd1700cf210 2024-03-23T09:43:34.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.014+0000 7fd166ffd640 1 -- 172.21.15.47:0/3363503243 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16045 (secure 0 0 0) 0x7fd158057da0 con 0x7fd1700cf210 2024-03-23T09:43:34.014 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:34.014 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":46,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:49.167591+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:42:46.053713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-23T09:42:47.131279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:43:34.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.016+0000 7fd1749d2640 1 -- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd15006ac30 msgr2=0x7fd15006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.016+0000 7fd1749d2640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd15006ac30 0x7fd15006d0f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd160006ad0 tx=0x7fd160001d20 comp rx=0 tx=0).stop 2024-03-23T09:43:34.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.016+0000 7fd1749d2640 1 -- 172.21.15.47:0/3363503243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1700cf210 msgr2=0x7fd170150bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.016+0000 7fd1749d2640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1700cf210 0x7fd170150bb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd15800e9b0 tx=0x7fd15800ee70 comp rx=0 tx=0).stop 2024-03-23T09:43:34.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.016+0000 7fd1749d2640 1 -- 172.21.15.47:0/3363503243 shutdown_connections 2024-03-23T09:43:34.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.016+0000 7fd1749d2640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd15006ac30 0x7fd15006d0f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.016+0000 7fd1749d2640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd17015cd00 0x7fd1700cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.016+0000 7fd1749d2640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1700cf210 0x7fd170150bb0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.016+0000 7fd1749d2640 1 --2- 172.21.15.47:0/3363503243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1700b5a90 0x7fd1700b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.016+0000 7fd1749d2640 1 -- 172.21.15.47:0/3363503243 >> 172.21.15.47:0/3363503243 conn(0x7fd17006b160 msgr2=0x7fd17015bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:34.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.016+0000 7fd1749d2640 1 -- 172.21.15.47:0/3363503243 shutdown_connections 2024-03-23T09:43:34.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.016+0000 7fd1749d2640 1 -- 172.21.15.47:0/3363503243 wait complete. 2024-03-23T09:43:34.027 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:42:46.053713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:42:47.131279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:43:34.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:43:34.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.131+0000 7fbcae4c4640 1 Processor -- start 2024-03-23T09:43:34.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.131+0000 7fbcae4c4640 1 -- start start 2024-03-23T09:43:34.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.131+0000 7fbcae4c4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbca815c990 0x7fbca8160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbcae4c4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca8161630 con 0x7fbca80cde10 2024-03-23T09:43:34.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbcae4c4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca80d1030 con 0x7fbca80b8350 2024-03-23T09:43:34.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbcae4c4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca80cfe10 con 0x7fbca815c990 2024-03-23T09:43:34.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbcaca61640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbca80cde10 0x7fbca80d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45926/0 (socket says 172.21.15.47:45926) 2024-03-23T09:43:34.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbcaca61640 1 -- 172.21.15.47:0/3610079966 learned_addr learned my addr 172.21.15.47:0/3610079966 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:34.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504655379 0 0) 0x7fbca8161630 con 0x7fbca80cde10 2024-03-23T09:43:34.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc90003520 con 0x7fbca80cde10 2024-03-23T09:43:34.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca7fff640 1 --2- 172.21.15.47:0/3610079966 >> v2:172.21.15.175:3300/0 conn(0x7fbca815c990 0x7fbca8160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2227511225 0 0) 0x7fbc90003520 con 0x7fbca80cde10 2024-03-23T09:43:34.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 >> v2:172.21.15.175:3300/0 conn(0x7fbca815c990 msgr2=0x7fbca8160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:34.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 --2- 172.21.15.47:0/3610079966 >> v2:172.21.15.175:3300/0 conn(0x7fbca815c990 0x7fbca8160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 >> v1:172.21.15.175:6789/0 conn(0x7fbca80b8350 legacy=0x7fbca80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca80b5b60 con 0x7fbca80cde10 2024-03-23T09:43:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fbca00019f0 con 0x7fbca80cde10 2024-03-23T09:43:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 >> v1:172.21.15.47:6789/0 conn(0x7fbca80cde10 legacy=0x7fbca80d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 --2- 172.21.15.47:0/3610079966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90003970 0x7fbc90003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 --2- 172.21.15.47:0/3610079966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc900047c0 0x7fbc90004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 --2- 172.21.15.47:0/3610079966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc900052a0 0x7fbc9000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80cfe10 con 0x7fbc90003970 2024-03-23T09:43:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9000c170 con 0x7fbc900052a0 2024-03-23T09:43:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9000c330 con 0x7fbc900047c0 2024-03-23T09:43:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.132+0000 7fbca7fff640 1 --2- 172.21.15.47:0/3610079966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90003970 0x7fbc90003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.133+0000 7fbca77fe640 1 --2- 172.21.15.47:0/3610079966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc900047c0 0x7fbc90004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.133+0000 7fbcaca61640 1 --2- 172.21.15.47:0/3610079966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc900052a0 0x7fbc9000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.133+0000 7fbca7fff640 1 -- 172.21.15.47:0/3610079966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc900047c0 msgr2=0x7fbc90004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.133+0000 7fbca7fff640 1 --2- 172.21.15.47:0/3610079966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc900047c0 0x7fbc90004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.133+0000 7fbca7fff640 1 -- 172.21.15.47:0/3610079966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc900052a0 msgr2=0x7fbc9000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.133+0000 7fbca7fff640 1 --2- 172.21.15.47:0/3610079966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc900052a0 0x7fbc9000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.133+0000 7fbca7fff640 1 -- 172.21.15.47:0/3610079966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc90003520 con 0x7fbc90003970 2024-03-23T09:43:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.133+0000 7fbcaca61640 1 -- 172.21.15.47:0/3610079966 reap_dead start 2024-03-23T09:43:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.133+0000 7fbca77fe640 1 --2- 172.21.15.47:0/3610079966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc900047c0 0x7fbc90004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.133+0000 7fbca7fff640 1 --2- 172.21.15.47:0/3610079966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90003970 0x7fbc90003d90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800ba00 tx=0x7fbc900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53fabd7c15729695 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.134+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98033440 con 0x7fbc90003970 2024-03-23T09:43:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.134+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc980335a0 con 0x7fbc90003970 2024-03-23T09:43:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.134+0000 7fbca6ffd640 1 -- 172.21.15.47:0/3610079966 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98033930 con 0x7fbc90003970 2024-03-23T09:43:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.134+0000 7fbcae4c4640 1 -- 172.21.15.47:0/3610079966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90003970 msgr2=0x7fbc90003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.134+0000 7fbcae4c4640 1 --2- 172.21.15.47:0/3610079966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90003970 0x7fbc90003d90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800ba00 tx=0x7fbc900057f0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.134+0000 7fbcae4c4640 1 -- 172.21.15.47:0/3610079966 shutdown_connections 2024-03-23T09:43:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.134+0000 7fbcae4c4640 1 --2- 172.21.15.47:0/3610079966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90003970 0x7fbc90003d90 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.134+0000 7fbcae4c4640 1 -- 172.21.15.47:0/3610079966 >> 172.21.15.47:0/3610079966 conn(0x7fbca80659d0 msgr2=0x7fbca8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.134+0000 7fbcae4c4640 1 -- 172.21.15.47:0/3610079966 shutdown_connections 2024-03-23T09:43:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.134+0000 7fbcae4c4640 1 -- 172.21.15.47:0/3610079966 wait complete. 2024-03-23T09:43:34.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.135+0000 7fbcae4c4640 1 Processor -- start 2024-03-23T09:43:34.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.136+0000 7fbcae4c4640 1 -- start start 2024-03-23T09:43:34.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.136+0000 7fbcae4c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90003970 0x7fbca8148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.136+0000 7fbcae4c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbca815fa90 0x7fbca80b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.136+0000 7fbcae4c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbca815fef0 0x7fbca80b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.136+0000 7fbcae4c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca815def0 con 0x7fbc90003970 2024-03-23T09:43:34.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.136+0000 7fbcae4c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80c75b0 con 0x7fbca815fef0 2024-03-23T09:43:34.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.136+0000 7fbcae4c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80c7890 con 0x7fbca815fa90 2024-03-23T09:43:34.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.136+0000 7fbca7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90003970 0x7fbca8148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.136+0000 7fbca7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90003970 0x7fbca8148ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50412/0 (socket says 172.21.15.47:50412) 2024-03-23T09:43:34.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.136+0000 7fbca7fff640 1 -- 172.21.15.47:0/1652659275 learned_addr learned my addr 172.21.15.47:0/1652659275 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:34.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.136+0000 7fbca77fe640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbca815fa90 0x7fbca80b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.136+0000 7fbcaca61640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbca815fef0 0x7fbca80b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.137+0000 7fbca7fff640 1 -- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbca815fa90 msgr2=0x7fbca80b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.137+0000 7fbca7fff640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbca815fa90 0x7fbca80b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.137+0000 7fbca7fff640 1 -- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbca815fef0 msgr2=0x7fbca80b6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.137+0000 7fbca7fff640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbca815fef0 0x7fbca80b6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.137+0000 7fbca7fff640 1 -- 172.21.15.47:0/1652659275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca80c7b70 con 0x7fbc90003970 2024-03-23T09:43:34.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.137+0000 7fbcaca61640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbca815fef0 0x7fbca80b6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:34.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.137+0000 7fbca7fff640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90003970 0x7fbca8148ac0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbc98035040 tx=0x7fbc98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:34.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.137+0000 7fbca4ff9640 1 -- 172.21.15.47:0/1652659275 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98004220 con 0x7fbc90003970 2024-03-23T09:43:34.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.137+0000 7fbca4ff9640 1 -- 172.21.15.47:0/1652659275 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc98040070 con 0x7fbc90003970 2024-03-23T09:43:34.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.137+0000 7fbcae4c4640 1 -- 172.21.15.47:0/1652659275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca80c7e50 con 0x7fbc90003970 2024-03-23T09:43:34.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.137+0000 7fbca4ff9640 1 -- 172.21.15.47:0/1652659275 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98033e70 con 0x7fbc90003970 2024-03-23T09:43:34.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.137+0000 7fbcae4c4640 1 -- 172.21.15.47:0/1652659275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca815d360 con 0x7fbc90003970 2024-03-23T09:43:34.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.138+0000 7fbcae4c4640 1 -- 172.21.15.47:0/1652659275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca8072360 con 0x7fbc90003970 2024-03-23T09:43:34.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.138+0000 7fbca4ff9640 1 -- 172.21.15.47:0/1652659275 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbc98031050 con 0x7fbc90003970 2024-03-23T09:43:34.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.139+0000 7fbca4ff9640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbc9406ac30 0x7fbc9406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.139+0000 7fbca4ff9640 1 -- 172.21.15.47:0/1652659275 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc980aaef0 con 0x7fbc90003970 2024-03-23T09:43:34.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.139+0000 7fbca77fe640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbc9406ac30 0x7fbc9406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.139+0000 7fbca77fe640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbc9406ac30 0x7fbc9406d0f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c000c00 tx=0x7fbc9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:34.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.143+0000 7fbca4ff9640 1 -- 172.21.15.47:0/1652659275 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbc98076040 con 0x7fbc90003970 2024-03-23T09:43:34.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.320+0000 7fbcae4c4640 1 -- 172.21.15.47:0/1652659275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbca8072360 con 0x7fbc90003970 2024-03-23T09:43:34.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.321+0000 7fbca4ff9640 1 -- 172.21.15.47:0/1652659275 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc980420a0 con 0x7fbc90003970 2024-03-23T09:43:34.321 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:43:34.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.323+0000 7fbcae4c4640 1 -- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbc9406ac30 msgr2=0x7fbc9406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.323+0000 7fbcae4c4640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbc9406ac30 0x7fbc9406d0f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c000c00 tx=0x7fbc9c009040 comp rx=0 tx=0).stop 2024-03-23T09:43:34.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.323+0000 7fbcae4c4640 1 -- 172.21.15.47:0/1652659275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90003970 msgr2=0x7fbca8148ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.323+0000 7fbcae4c4640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90003970 0x7fbca8148ac0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbc98035040 tx=0x7fbc98004730 comp rx=0 tx=0).stop 2024-03-23T09:43:34.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.323+0000 7fbcae4c4640 1 -- 172.21.15.47:0/1652659275 shutdown_connections 2024-03-23T09:43:34.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.323+0000 7fbcae4c4640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbc9406ac30 0x7fbc9406d0f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.323+0000 7fbcae4c4640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbca815fef0 0x7fbca80b6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.323+0000 7fbcae4c4640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbca815fa90 0x7fbca80b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.323+0000 7fbcae4c4640 1 --2- 172.21.15.47:0/1652659275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90003970 0x7fbca8148ac0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.323+0000 7fbcae4c4640 1 -- 172.21.15.47:0/1652659275 >> 172.21.15.47:0/1652659275 conn(0x7fbca80659d0 msgr2=0x7fbca80b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:34.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.323+0000 7fbcae4c4640 1 -- 172.21.15.47:0/1652659275 shutdown_connections 2024-03-23T09:43:34.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.323+0000 7fbcae4c4640 1 -- 172.21.15.47:0/1652659275 wait complete. 2024-03-23T09:43:34.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:43:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.437+0000 7f73df18b640 1 Processor -- start 2024-03-23T09:43:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.437+0000 7f73df18b640 1 -- start start 2024-03-23T09:43:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73df18b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f73d815cbf0 0x7f73d80b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73df18b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f73d80cd140 0x7f73d80cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73df18b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d80cf7f0 0x7f73d80b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73df18b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d80cfc40 con 0x7f73d815cbf0 2024-03-23T09:43:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73df18b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d8161570 con 0x7f73d80cd140 2024-03-23T09:43:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73df18b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d8160960 con 0x7f73d80cf7f0 2024-03-23T09:43:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73dcf27640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f73d815cbf0 0x7f73d80b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73dcf27640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f73d815cbf0 0x7f73d80b1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50414/0 (socket says 172.21.15.47:50414) 2024-03-23T09:43:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73dcf27640 1 -- 172.21.15.47:0/2770653305 learned_addr learned my addr 172.21.15.47:0/2770653305 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73d7fff640 1 --2- 172.21.15.47:0/2770653305 >> v2:172.21.15.175:3300/0 conn(0x7f73d80cd140 0x7f73d80cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73dd728640 1 --2- 172.21.15.47:0/2770653305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d80cf7f0 0x7f73d80b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73dcf27640 1 -- 172.21.15.47:0/2770653305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d80cf7f0 msgr2=0x7f73d80b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73dcf27640 1 --2- 172.21.15.47:0/2770653305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d80cf7f0 0x7f73d80b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73dcf27640 1 -- 172.21.15.47:0/2770653305 >> v2:172.21.15.175:3300/0 conn(0x7f73d80cd140 msgr2=0x7f73d80cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73dcf27640 1 --2- 172.21.15.47:0/2770653305 >> v2:172.21.15.175:3300/0 conn(0x7f73d80cd140 0x7f73d80cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73dcf27640 1 -- 172.21.15.47:0/2770653305 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d8160c40 con 0x7f73d815cbf0 2024-03-23T09:43:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.438+0000 7f73d7fff640 1 --2- 172.21.15.47:0/2770653305 >> v2:172.21.15.175:3300/0 conn(0x7f73d80cd140 0x7f73d80cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.439+0000 7f73dcf27640 1 --2- 172.21.15.47:0/2770653305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73d815cbf0 0x7f73d80b1df0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f73d808ada0 tx=0x7f73cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9de094f98fdc29 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.439+0000 7f73d77fe640 1 -- 172.21.15.47:0/2770653305 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc031870 con 0x7f73d815cbf0 2024-03-23T09:43:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.439+0000 7f73d77fe640 1 -- 172.21.15.47:0/2770653305 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc0319d0 con 0x7f73d815cbf0 2024-03-23T09:43:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.439+0000 7f73d77fe640 1 -- 172.21.15.47:0/2770653305 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc031d60 con 0x7f73d815cbf0 2024-03-23T09:43:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.440+0000 7f73df18b640 1 -- 172.21.15.47:0/2770653305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73d815cbf0 msgr2=0x7f73d80b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.440+0000 7f73df18b640 1 --2- 172.21.15.47:0/2770653305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73d815cbf0 0x7f73d80b1df0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f73d808ada0 tx=0x7f73cc02b910 comp rx=0 tx=0).stop 2024-03-23T09:43:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.440+0000 7f73df18b640 1 -- 172.21.15.47:0/2770653305 shutdown_connections 2024-03-23T09:43:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.440+0000 7f73df18b640 1 --2- 172.21.15.47:0/2770653305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d80cf7f0 0x7f73d80b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.440+0000 7f73df18b640 1 --2- 172.21.15.47:0/2770653305 >> v2:172.21.15.175:3300/0 conn(0x7f73d80cd140 0x7f73d80cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.440+0000 7f73df18b640 1 --2- 172.21.15.47:0/2770653305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73d815cbf0 0x7f73d80b1df0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.440+0000 7f73df18b640 1 -- 172.21.15.47:0/2770653305 >> 172.21.15.47:0/2770653305 conn(0x7f73d806b160 msgr2=0x7f73d806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.440+0000 7f73df18b640 1 -- 172.21.15.47:0/2770653305 shutdown_connections 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.440+0000 7f73df18b640 1 -- 172.21.15.47:0/2770653305 wait complete. 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.441+0000 7f73df18b640 1 Processor -- start 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.441+0000 7f73df18b640 1 -- start start 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73df18b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d80cd140 0x7f73d80cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73df18b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73d80cf7f0 0x7f73d80c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73df18b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d815cbf0 0x7f73d815ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73df18b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80c7df0 con 0x7f73d80cf7f0 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73df18b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d815b160 con 0x7f73d80cd140 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73df18b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80c8090 con 0x7f73d815cbf0 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73d7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73d80cf7f0 0x7f73d80c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73d7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73d80cf7f0 0x7f73d80c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50418/0 (socket says 172.21.15.47:50418) 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73d7fff640 1 -- 172.21.15.47:0/4184248836 learned_addr learned my addr 172.21.15.47:0/4184248836 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73dcf27640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d80cd140 0x7f73d80cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73dd728640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d815cbf0 0x7f73d815ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73d7fff640 1 -- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d815cbf0 msgr2=0x7f73d815ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73d7fff640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d815cbf0 0x7f73d815ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73d7fff640 1 -- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d80cd140 msgr2=0x7f73d80cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73d7fff640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d80cd140 0x7f73d80cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73d7fff640 1 -- 172.21.15.47:0/4184248836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc0094d0 con 0x7f73d80cf7f0 2024-03-23T09:43:34.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73d7fff640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73d80cf7f0 0x7f73d80c78b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f73c400e810 tx=0x7f73c400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:34.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73d57fa640 1 -- 172.21.15.47:0/4184248836 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4004030 con 0x7f73d80cf7f0 2024-03-23T09:43:34.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73d57fa640 1 -- 172.21.15.47:0/4184248836 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c4004190 con 0x7f73d80cf7f0 2024-03-23T09:43:34.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.442+0000 7f73df18b640 1 -- 172.21.15.47:0/4184248836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d80c8370 con 0x7f73d80cf7f0 2024-03-23T09:43:34.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.443+0000 7f73d57fa640 1 -- 172.21.15.47:0/4184248836 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4004520 con 0x7f73d80cf7f0 2024-03-23T09:43:34.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.443+0000 7f73df18b640 1 -- 172.21.15.47:0/4184248836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d80c8930 con 0x7f73d80cf7f0 2024-03-23T09:43:34.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.444+0000 7f73df18b640 1 -- 172.21.15.47:0/4184248836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d8072360 con 0x7f73d80cf7f0 2024-03-23T09:43:34.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.444+0000 7f73d57fa640 1 -- 172.21.15.47:0/4184248836 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f73c400c6d0 con 0x7f73d80cf7f0 2024-03-23T09:43:34.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.444+0000 7f73d57fa640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f73bc06ac30 0x7f73bc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.444+0000 7f73d57fa640 1 -- 172.21.15.47:0/4184248836 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73c4088e10 con 0x7f73d80cf7f0 2024-03-23T09:43:34.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.444+0000 7f73dcf27640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f73bc06ac30 0x7f73bc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.445+0000 7f73dcf27640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f73bc06ac30 0x7f73bc06d0f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f73cc036000 tx=0x7f73cc03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:34.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.448+0000 7f73d57fa640 1 -- 172.21.15.47:0/4184248836 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f73c4054080 con 0x7f73d80cf7f0 2024-03-23T09:43:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.626+0000 7f73df18b640 1 -- 172.21.15.47:0/4184248836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73d8066be0 con 0x7f73d80cf7f0 2024-03-23T09:43:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.626+0000 7f73d57fa640 1 -- 172.21.15.47:0/4184248836 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73c4020020 con 0x7f73d80cf7f0 2024-03-23T09:43:34.626 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:43:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.628+0000 7f73df18b640 1 -- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f73bc06ac30 msgr2=0x7f73bc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.628+0000 7f73df18b640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f73bc06ac30 0x7f73bc06d0f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f73cc036000 tx=0x7f73cc03c040 comp rx=0 tx=0).stop 2024-03-23T09:43:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.628+0000 7f73df18b640 1 -- 172.21.15.47:0/4184248836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73d80cf7f0 msgr2=0x7f73d80c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.628+0000 7f73df18b640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73d80cf7f0 0x7f73d80c78b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f73c400e810 tx=0x7f73c400ecd0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.628+0000 7f73df18b640 1 -- 172.21.15.47:0/4184248836 shutdown_connections 2024-03-23T09:43:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.628+0000 7f73df18b640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f73bc06ac30 0x7f73bc06d0f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.628+0000 7f73df18b640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d815cbf0 0x7f73d815ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.628+0000 7f73df18b640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73d80cf7f0 0x7f73d80c78b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.628+0000 7f73df18b640 1 --2- 172.21.15.47:0/4184248836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d80cd140 0x7f73d80cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.628+0000 7f73df18b640 1 -- 172.21.15.47:0/4184248836 >> 172.21.15.47:0/4184248836 conn(0x7f73d806b160 msgr2=0x7f73d815d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.628+0000 7f73df18b640 1 -- 172.21.15.47:0/4184248836 shutdown_connections 2024-03-23T09:43:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.628+0000 7f73df18b640 1 -- 172.21.15.47:0/4184248836 wait complete. 2024-03-23T09:43:34.639 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:43:34.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.743+0000 7fe1acbae640 1 Processor -- start 2024-03-23T09:43:34.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.743+0000 7fe1acbae640 1 -- start start 2024-03-23T09:43:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1acbae640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1a815c990 0x7fe1a8160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1acbae640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe1a80b8350 0x7fe1a80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1acbae640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a8161630 con 0x7fe1a80cdc40 2024-03-23T09:43:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1acbae640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a80d1030 con 0x7fe1a815c990 2024-03-23T09:43:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1acbae640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a80ce070 con 0x7fe1a80b8350 2024-03-23T09:43:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1a659c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1a815c990 0x7fe1a8160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1a659c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1a815c990 0x7fe1a8160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45734/0 (socket says 172.21.15.47:45734) 2024-03-23T09:43:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1a659c640 1 -- 172.21.15.47:0/2915062793 learned_addr learned my addr 172.21.15.47:0/2915062793 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1a5d9b640 1 --2- 172.21.15.47:0/2915062793 >> v2:172.21.15.175:3300/0 conn(0x7fe1a80b8350 0x7fe1a80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1a659c640 1 -- 172.21.15.47:0/2915062793 >> v2:172.21.15.175:3300/0 conn(0x7fe1a80b8350 msgr2=0x7fe1a80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1a659c640 1 --2- 172.21.15.47:0/2915062793 >> v2:172.21.15.175:3300/0 conn(0x7fe1a80b8350 0x7fe1a80cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1a659c640 1 -- 172.21.15.47:0/2915062793 >> v1:172.21.15.175:6789/0 conn(0x7fe1a80cdc40 legacy=0x7fe1a80b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1a659c640 1 -- 172.21.15.47:0/2915062793 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a80cfe10 con 0x7fe1a815c990 2024-03-23T09:43:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.744+0000 7fe1a5d9b640 1 --2- 172.21.15.47:0/2915062793 >> v2:172.21.15.175:3300/0 conn(0x7fe1a80b8350 0x7fe1a80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.745+0000 7fe1a659c640 1 --2- 172.21.15.47:0/2915062793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a815c990 0x7fe1a8160690 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe1a808ac70 tx=0x7fe19802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4497c2777c98429 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.745+0000 7fe1a559a640 1 -- 172.21.15.47:0/2915062793 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198031870 con 0x7fe1a815c990 2024-03-23T09:43:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.745+0000 7fe1a559a640 1 -- 172.21.15.47:0/2915062793 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1980319d0 con 0x7fe1a815c990 2024-03-23T09:43:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.745+0000 7fe1a559a640 1 -- 172.21.15.47:0/2915062793 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198031d60 con 0x7fe1a815c990 2024-03-23T09:43:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.745+0000 7fe1acbae640 1 -- 172.21.15.47:0/2915062793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a815c990 msgr2=0x7fe1a8160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.745+0000 7fe1acbae640 1 --2- 172.21.15.47:0/2915062793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a815c990 0x7fe1a8160690 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe1a808ac70 tx=0x7fe19802b910 comp rx=0 tx=0).stop 2024-03-23T09:43:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.746+0000 7fe1acbae640 1 -- 172.21.15.47:0/2915062793 shutdown_connections 2024-03-23T09:43:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.746+0000 7fe1acbae640 1 --2- 172.21.15.47:0/2915062793 >> v2:172.21.15.175:3300/0 conn(0x7fe1a80b8350 0x7fe1a80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.746+0000 7fe1acbae640 1 --2- 172.21.15.47:0/2915062793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a815c990 0x7fe1a8160690 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.746+0000 7fe1acbae640 1 -- 172.21.15.47:0/2915062793 >> 172.21.15.47:0/2915062793 conn(0x7fe1a80659d0 msgr2=0x7fe1a8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.746+0000 7fe1acbae640 1 -- 172.21.15.47:0/2915062793 shutdown_connections 2024-03-23T09:43:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.746+0000 7fe1acbae640 1 -- 172.21.15.47:0/2915062793 wait complete. 2024-03-23T09:43:34.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.747+0000 7fe1acbae640 1 Processor -- start 2024-03-23T09:43:34.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.747+0000 7fe1acbae640 1 -- start start 2024-03-23T09:43:34.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1acbae640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1a80b8350 0x7fe1a80c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1acbae640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80cdc40 0x7fe1a8152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1acbae640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1a815c990 0x7fe1a80caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1acbae640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a815dd20 con 0x7fe1a80cdc40 2024-03-23T09:43:34.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1acbae640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a815d0a0 con 0x7fe1a815c990 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1acbae640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a815d380 con 0x7fe1a80b8350 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1a5d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80cdc40 0x7fe1a8152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1a5d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80cdc40 0x7fe1a8152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45742/0 (socket says 172.21.15.47:45742) 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1a5d9b640 1 -- 172.21.15.47:0/1935091433 learned_addr learned my addr 172.21.15.47:0/1935091433 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1a659c640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1a80b8350 0x7fe1a80c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1a6d9d640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1a815c990 0x7fe1a80caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1a5d9b640 1 -- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1a80b8350 msgr2=0x7fe1a80c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1a5d9b640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1a80b8350 0x7fe1a80c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1a5d9b640 1 -- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1a815c990 msgr2=0x7fe1a80caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1a5d9b640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1a815c990 0x7fe1a80caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1a5d9b640 1 -- 172.21.15.47:0/1935091433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1980094d0 con 0x7fe1a80cdc40 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1a6d9d640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1a815c990 0x7fe1a80caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1a5d9b640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80cdc40 0x7fe1a8152570 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00e7e0 tx=0x7fe19c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe196ffd640 1 -- 172.21.15.47:0/1935091433 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c004030 con 0x7fe1a80cdc40 2024-03-23T09:43:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe196ffd640 1 -- 172.21.15.47:0/1935091433 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19c004190 con 0x7fe1a80cdc40 2024-03-23T09:43:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.748+0000 7fe1acbae640 1 -- 172.21.15.47:0/1935091433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a815d660 con 0x7fe1a80cdc40 2024-03-23T09:43:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.749+0000 7fe196ffd640 1 -- 172.21.15.47:0/1935091433 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c004520 con 0x7fe1a80cdc40 2024-03-23T09:43:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.749+0000 7fe1acbae640 1 -- 172.21.15.47:0/1935091433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a80c7290 con 0x7fe1a80cdc40 2024-03-23T09:43:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.749+0000 7fe1acbae640 1 -- 172.21.15.47:0/1935091433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a8072360 con 0x7fe1a80cdc40 2024-03-23T09:43:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.750+0000 7fe196ffd640 1 -- 172.21.15.47:0/1935091433 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe19c00c6d0 con 0x7fe1a80cdc40 2024-03-23T09:43:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.750+0000 7fe196ffd640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe18806ac00 0x7fe18806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.750+0000 7fe196ffd640 1 -- 172.21.15.47:0/1935091433 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe19c088f00 con 0x7fe1a80cdc40 2024-03-23T09:43:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.750+0000 7fe1a659c640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe18806ac00 0x7fe18806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.751+0000 7fe1a659c640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe18806ac00 0x7fe18806d0c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe198036000 tx=0x7fe19803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:34.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.755+0000 7fe196ffd640 1 -- 172.21.15.47:0/1935091433 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe19c054220 con 0x7fe1a80cdc40 2024-03-23T09:43:34.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.934+0000 7fe1acbae640 1 -- 172.21.15.47:0/1935091433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe1a8066be0 con 0x7fe1a80cdc40 2024-03-23T09:43:34.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.934+0000 7fe196ffd640 1 -- 172.21.15.47:0/1935091433 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe19c057840 con 0x7fe1a80cdc40 2024-03-23T09:43:34.934 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:43:34.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.936+0000 7fe1acbae640 1 -- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe18806ac00 msgr2=0x7fe18806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.936+0000 7fe1acbae640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe18806ac00 0x7fe18806d0c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe198036000 tx=0x7fe19803c040 comp rx=0 tx=0).stop 2024-03-23T09:43:34.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.936+0000 7fe1acbae640 1 -- 172.21.15.47:0/1935091433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80cdc40 msgr2=0x7fe1a8152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:34.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.936+0000 7fe1acbae640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80cdc40 0x7fe1a8152570 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00e7e0 tx=0x7fe19c00eca0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.936+0000 7fe1acbae640 1 -- 172.21.15.47:0/1935091433 shutdown_connections 2024-03-23T09:43:34.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.936+0000 7fe1acbae640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe18806ac00 0x7fe18806d0c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.936+0000 7fe1acbae640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1a815c990 0x7fe1a80caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.936+0000 7fe1acbae640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80cdc40 0x7fe1a8152570 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.936+0000 7fe1acbae640 1 --2- 172.21.15.47:0/1935091433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1a80b8350 0x7fe1a80c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:34.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.936+0000 7fe1acbae640 1 -- 172.21.15.47:0/1935091433 >> 172.21.15.47:0/1935091433 conn(0x7fe1a80659d0 msgr2=0x7fe1a8065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:34.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.936+0000 7fe1acbae640 1 -- 172.21.15.47:0/1935091433 shutdown_connections 2024-03-23T09:43:34.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:34.936+0000 7fe1acbae640 1 -- 172.21.15.47:0/1935091433 wait complete. 2024-03-23T09:43:34.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:43:35.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.048+0000 7f4e6181e640 1 Processor -- start 2024-03-23T09:43:35.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.048+0000 7f4e6181e640 1 -- start start 2024-03-23T09:43:35.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e6181e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5c15c9a0 0x7f4e5c06b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e6181e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4e5c06bd40 0x7f4e5c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e6181e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5c06c1a0 con 0x7f4e5c15d770 2024-03-23T09:43:35.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e6181e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5c0b1530 con 0x7f4e5c06bd40 2024-03-23T09:43:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e6181e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c15df60 con 0x7f4e5c15c9a0 2024-03-23T09:43:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e5b7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4e5c15d770 0x7f4e5c0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47066/0 (socket says 172.21.15.47:47066) 2024-03-23T09:43:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e5b7fe640 1 -- 172.21.15.47:0/4105516968 learned_addr learned my addr 172.21.15.47:0/4105516968 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e53fff640 1 --2- 172.21.15.47:0/4105516968 >> v2:172.21.15.175:3300/0 conn(0x7f4e5c06bd40 0x7f4e5c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e5affd640 1 --2- 172.21.15.47:0/4105516968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5c15c9a0 0x7f4e5c06b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e5a7fc640 1 -- 172.21.15.47:0/4105516968 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668991919 0 0) 0x7f4e5c06c1a0 con 0x7f4e5c15d770 2024-03-23T09:43:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e5a7fc640 1 -- 172.21.15.47:0/4105516968 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e48003520 con 0x7f4e5c15d770 2024-03-23T09:43:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e5a7fc640 1 -- 172.21.15.47:0/4105516968 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3244314688 0 0) 0x7f4e48003520 con 0x7f4e5c15d770 2024-03-23T09:43:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e53fff640 1 -- 172.21.15.47:0/4105516968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5c15c9a0 msgr2=0x7f4e5c06b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e53fff640 1 --2- 172.21.15.47:0/4105516968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5c15c9a0 0x7f4e5c06b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e53fff640 1 -- 172.21.15.47:0/4105516968 >> v1:172.21.15.47:6789/0 conn(0x7f4e5c15d770 legacy=0x7f4e5c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e53fff640 1 -- 172.21.15.47:0/4105516968 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c1606e0 con 0x7f4e5c06bd40 2024-03-23T09:43:35.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.049+0000 7f4e5affd640 1 --2- 172.21.15.47:0/4105516968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5c15c9a0 0x7f4e5c06b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:43:35.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.050+0000 7f4e53fff640 1 --2- 172.21.15.47:0/4105516968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e5c06bd40 0x7f4e5c15d230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4e440097a0 tx=0x7f4e4402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5fbe2b71fd68978 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:35.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.050+0000 7f4e5a7fc640 1 -- 172.21.15.47:0/4105516968 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e540019f0 con 0x7f4e5c06bd40 2024-03-23T09:43:35.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.050+0000 7f4e5a7fc640 1 -- 172.21.15.47:0/4105516968 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4402e050 con 0x7f4e5c06bd40 2024-03-23T09:43:35.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.050+0000 7f4e5a7fc640 1 -- 172.21.15.47:0/4105516968 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44030710 con 0x7f4e5c06bd40 2024-03-23T09:43:35.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.051+0000 7f4e6181e640 1 -- 172.21.15.47:0/4105516968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e5c06bd40 msgr2=0x7f4e5c15d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.051+0000 7f4e6181e640 1 --2- 172.21.15.47:0/4105516968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e5c06bd40 0x7f4e5c15d230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4e440097a0 tx=0x7f4e4402b790 comp rx=0 tx=0).stop 2024-03-23T09:43:35.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.051+0000 7f4e6181e640 1 -- 172.21.15.47:0/4105516968 shutdown_connections 2024-03-23T09:43:35.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.051+0000 7f4e6181e640 1 --2- 172.21.15.47:0/4105516968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e5c06bd40 0x7f4e5c15d230 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.051+0000 7f4e6181e640 1 --2- 172.21.15.47:0/4105516968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5c15c9a0 0x7f4e5c06b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.051+0000 7f4e6181e640 1 -- 172.21.15.47:0/4105516968 >> 172.21.15.47:0/4105516968 conn(0x7f4e5c0d0670 msgr2=0x7f4e5c0d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.051+0000 7f4e6181e640 1 -- 172.21.15.47:0/4105516968 shutdown_connections 2024-03-23T09:43:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.051+0000 7f4e6181e640 1 -- 172.21.15.47:0/4105516968 wait complete. 2024-03-23T09:43:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.052+0000 7f4e6181e640 1 Processor -- start 2024-03-23T09:43:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.052+0000 7f4e6181e640 1 -- start start 2024-03-23T09:43:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e6181e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5c06bd40 0x7f4e5c0ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e6181e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e5c15c9a0 0x7f4e5c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e6181e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5c15d770 0x7f4e5c1f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e6181e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c0aeb00 con 0x7f4e5c15d770 2024-03-23T09:43:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e6181e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c0aede0 con 0x7f4e5c15c9a0 2024-03-23T09:43:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e6181e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c0af0c0 con 0x7f4e5c06bd40 2024-03-23T09:43:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e5b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5c15d770 0x7f4e5c1f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e5b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5c15d770 0x7f4e5c1f17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45754/0 (socket says 172.21.15.47:45754) 2024-03-23T09:43:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e5b7fe640 1 -- 172.21.15.47:0/677642880 learned_addr learned my addr 172.21.15.47:0/677642880 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:35.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e5b7fe640 1 -- 172.21.15.47:0/677642880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5c06bd40 msgr2=0x7f4e5c0ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e5b7fe640 1 --2- 172.21.15.47:0/677642880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5c06bd40 0x7f4e5c0ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e5b7fe640 1 -- 172.21.15.47:0/677642880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e5c15c9a0 msgr2=0x7f4e5c06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e5b7fe640 1 --2- 172.21.15.47:0/677642880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e5c15c9a0 0x7f4e5c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e5b7fe640 1 -- 172.21.15.47:0/677642880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c009400 con 0x7f4e5c15d770 2024-03-23T09:43:35.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.053+0000 7f4e5b7fe640 1 --2- 172.21.15.47:0/677642880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5c15d770 0x7f4e5c1f17e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400cc00 tx=0x7f4e5400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:35.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.054+0000 7f4e58ff9640 1 -- 172.21.15.47:0/677642880 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400ce10 con 0x7f4e5c15d770 2024-03-23T09:43:35.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.054+0000 7f4e58ff9640 1 -- 172.21.15.47:0/677642880 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e54012400 con 0x7f4e5c15d770 2024-03-23T09:43:35.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.054+0000 7f4e6181e640 1 -- 172.21.15.47:0/677642880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e44009400 con 0x7f4e5c15d770 2024-03-23T09:43:35.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.054+0000 7f4e58ff9640 1 -- 172.21.15.47:0/677642880 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e540127b0 con 0x7f4e5c15d770 2024-03-23T09:43:35.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.054+0000 7f4e6181e640 1 -- 172.21.15.47:0/677642880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c0ae7f0 con 0x7f4e5c15d770 2024-03-23T09:43:35.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.055+0000 7f4e6181e640 1 -- 172.21.15.47:0/677642880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5c0c91a0 con 0x7f4e5c15d770 2024-03-23T09:43:35.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.055+0000 7f4e58ff9640 1 -- 172.21.15.47:0/677642880 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4e54012910 con 0x7f4e5c15d770 2024-03-23T09:43:35.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.055+0000 7f4e58ff9640 1 --2- 172.21.15.47:0/677642880 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4e3c06ac30 0x7f4e3c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.056+0000 7f4e58ff9640 1 -- 172.21.15.47:0/677642880 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e54089a90 con 0x7f4e5c15d770 2024-03-23T09:43:35.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.056+0000 7f4e5affd640 1 --2- 172.21.15.47:0/677642880 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4e3c06ac30 0x7f4e3c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.056+0000 7f4e5affd640 1 --2- 172.21.15.47:0/677642880 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4e3c06ac30 0x7f4e3c06d0f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c006e80 tx=0x7f4e4c006dd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:35.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.060+0000 7f4e58ff9640 1 -- 172.21.15.47:0/677642880 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4e5c0c91a0 con 0x7f4e5c15d770 2024-03-23T09:43:35.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.274+0000 7f4e6181e640 1 -- 172.21.15.47:0/677642880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e5c0c91a0 con 0x7f4e5c15d770 2024-03-23T09:43:35.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.275+0000 7f4e58ff9640 1 -- 172.21.15.47:0/677642880 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f4e5c0c91a0 con 0x7f4e5c15d770 2024-03-23T09:43:35.275 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:35.275 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1189831856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1189831856},{"type":"v1","addr":"172.21.15.47:6835","nonce":1189831856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/2248064454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2248064454},{"type":"v1","addr":"172.21.15.47:6834","nonce":2248064454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5694,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3806465612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3806465612},{"type":"v1","addr":"172.21.15.175:6835","nonce":3806465612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:42:48.165819+0000","modified":"2024-03-23T09:42:50.171529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1626348200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1626348200},{"type":"v1","addr":"172.21.15.175:6837","nonce":1626348200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-23T09:43:35.275 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 18 2024-03-23T09:43:35.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.276+0000 7f4e6181e640 1 -- 172.21.15.47:0/677642880 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4e3c06ac30 msgr2=0x7f4e3c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.276+0000 7f4e6181e640 1 --2- 172.21.15.47:0/677642880 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4e3c06ac30 0x7f4e3c06d0f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c006e80 tx=0x7f4e4c006dd0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.276+0000 7f4e6181e640 1 -- 172.21.15.47:0/677642880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5c15d770 msgr2=0x7f4e5c1f17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.276+0000 7f4e6181e640 1 --2- 172.21.15.47:0/677642880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5c15d770 0x7f4e5c1f17e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400cc00 tx=0x7f4e5400a430 comp rx=0 tx=0).stop 2024-03-23T09:43:35.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.276+0000 7f4e6181e640 1 -- 172.21.15.47:0/677642880 shutdown_connections 2024-03-23T09:43:35.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.276+0000 7f4e6181e640 1 --2- 172.21.15.47:0/677642880 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4e3c06ac30 0x7f4e3c06d0f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.276+0000 7f4e6181e640 1 --2- 172.21.15.47:0/677642880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5c15d770 0x7f4e5c1f17e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.276+0000 7f4e6181e640 1 --2- 172.21.15.47:0/677642880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e5c15c9a0 0x7f4e5c06eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.276+0000 7f4e6181e640 1 --2- 172.21.15.47:0/677642880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5c06bd40 0x7f4e5c0ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.276+0000 7f4e6181e640 1 -- 172.21.15.47:0/677642880 >> 172.21.15.47:0/677642880 conn(0x7f4e5c0d0670 msgr2=0x7f4e5c06fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:35.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.276+0000 7f4e6181e640 1 -- 172.21.15.47:0/677642880 shutdown_connections 2024-03-23T09:43:35.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.276+0000 7f4e6181e640 1 -- 172.21.15.47:0/677642880 wait complete. 2024-03-23T09:43:35.287 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:43:35.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.388+0000 7f4231f1b640 1 Processor -- start 2024-03-23T09:43:35.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.388+0000 7f4231f1b640 1 -- start start 2024-03-23T09:43:35.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.389+0000 7f4231f1b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c098a00 0x7f422c098de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.389+0000 7f4231f1b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c0b1060 con 0x7f422c06bc50 2024-03-23T09:43:35.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.389+0000 7f4231f1b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c099320 con 0x7f422c0b1680 2024-03-23T09:43:35.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.389+0000 7f4231f1b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0b12c0 con 0x7f422c098a00 2024-03-23T09:43:35.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.389+0000 7f422bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f422c06bc50 0x7f422c0ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47070/0 (socket says 172.21.15.47:47070) 2024-03-23T09:43:35.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.389+0000 7f422bfff640 1 -- 172.21.15.47:0/1659879097 learned_addr learned my addr 172.21.15.47:0/1659879097 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:35.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.389+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187704404 0 0) 0x7f422c0b1060 con 0x7f422c06bc50 2024-03-23T09:43:35.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.389+0000 7f422b7fe640 1 --2- 172.21.15.47:0/1659879097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c098a00 0x7f422c098de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.389+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4214003520 con 0x7f422c06bc50 2024-03-23T09:43:35.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1706706863 0 0) 0x7f4214003520 con 0x7f422c06bc50 2024-03-23T09:43:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c098a00 msgr2=0x7f422c098de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422a7fc640 1 --2- 172.21.15.47:0/1659879097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c098a00 0x7f422c098de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 >> v1:172.21.15.175:6789/0 conn(0x7f422c0b1680 legacy=0x7f422c0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c06ae90 con 0x7f422c06bc50 2024-03-23T09:43:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f42240019f0 con 0x7f422c06bc50 2024-03-23T09:43:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 >> v1:172.21.15.47:6789/0 conn(0x7f422c06bc50 legacy=0x7f422c0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422a7fc640 1 --2- 172.21.15.47:0/1659879097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c098a00 0x7f42140039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422a7fc640 1 --2- 172.21.15.47:0/1659879097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4214003fd0 0x7f42140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422a7fc640 1 --2- 172.21.15.47:0/1659879097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4214004d00 0x7f4214005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0b12c0 con 0x7f4214003fd0 2024-03-23T09:43:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421400bbc0 con 0x7f4214004d00 2024-03-23T09:43:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421400bd80 con 0x7f422c098a00 2024-03-23T09:43:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422affd640 1 --2- 172.21.15.47:0/1659879097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4214003fd0 0x7f42140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422bfff640 1 --2- 172.21.15.47:0/1659879097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4214004d00 0x7f4214005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422b7fe640 1 --2- 172.21.15.47:0/1659879097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c098a00 0x7f42140039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422affd640 1 -- 172.21.15.47:0/1659879097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c098a00 msgr2=0x7f42140039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422affd640 1 --2- 172.21.15.47:0/1659879097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c098a00 0x7f42140039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422affd640 1 -- 172.21.15.47:0/1659879097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4214004d00 msgr2=0x7f4214005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422affd640 1 --2- 172.21.15.47:0/1659879097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4214004d00 0x7f4214005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422affd640 1 -- 172.21.15.47:0/1659879097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4214003520 con 0x7f4214003fd0 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.390+0000 7f422b7fe640 1 --2- 172.21.15.47:0/1659879097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c098a00 0x7f42140039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.391+0000 7f422affd640 1 --2- 172.21.15.47:0/1659879097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4214003fd0 0x7f42140047c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4220001720 tx=0x7f42140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b550066260459a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.391+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200052d0 con 0x7f4214003fd0 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.391+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422002e050 con 0x7f4214003fd0 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.391+0000 7f422a7fc640 1 -- 172.21.15.47:0/1659879097 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220003830 con 0x7f4214003fd0 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.392+0000 7f4231f1b640 1 -- 172.21.15.47:0/1659879097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4214003fd0 msgr2=0x7f42140047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.392+0000 7f4231f1b640 1 --2- 172.21.15.47:0/1659879097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4214003fd0 0x7f42140047c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4220001720 tx=0x7f42140057f0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.392+0000 7f422bfff640 1 -- 172.21.15.47:0/1659879097 reap_dead start 2024-03-23T09:43:35.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.392+0000 7f4231f1b640 1 -- 172.21.15.47:0/1659879097 shutdown_connections 2024-03-23T09:43:35.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.392+0000 7f4231f1b640 1 -- 172.21.15.47:0/1659879097 >> 172.21.15.47:0/1659879097 conn(0x7f422c0b2140 msgr2=0x7f422c0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:35.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.392+0000 7f4231f1b640 1 -- 172.21.15.47:0/1659879097 shutdown_connections 2024-03-23T09:43:35.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.392+0000 7f4231f1b640 1 -- 172.21.15.47:0/1659879097 wait complete. 2024-03-23T09:43:35.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.393+0000 7f4231f1b640 1 Processor -- start 2024-03-23T09:43:35.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.393+0000 7f4231f1b640 1 -- start start 2024-03-23T09:43:35.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f4231f1b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4214003fd0 0x7f422c155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f4231f1b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f422c155e90 0x7f422c1f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f4231f1b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c1558b0 0x7f422c15e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f4231f1b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c158330 con 0x7f422c155e90 2024-03-23T09:43:35.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f4231f1b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c158610 con 0x7f4214003fd0 2024-03-23T09:43:35.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f4231f1b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c1588f0 con 0x7f422c1558b0 2024-03-23T09:43:35.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f422affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f422c155e90 0x7f422c1f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f422affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f422c155e90 0x7f422c1f2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45764/0 (socket says 172.21.15.47:45764) 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f422affd640 1 -- 172.21.15.47:0/2333147464 learned_addr learned my addr 172.21.15.47:0/2333147464 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f422b7fe640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4214003fd0 0x7f422c155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f422bfff640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c1558b0 0x7f422c15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f422affd640 1 -- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c1558b0 msgr2=0x7f422c15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f422affd640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c1558b0 0x7f422c15e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f422affd640 1 -- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4214003fd0 msgr2=0x7f422c155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f422affd640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4214003fd0 0x7f422c155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f422affd640 1 -- 172.21.15.47:0/2333147464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c158bd0 con 0x7f422c155e90 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f422affd640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f422c155e90 0x7f422c1f2770 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4220008fd0 tx=0x7f4220001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f4213fff640 1 -- 172.21.15.47:0/2333147464 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200076c0 con 0x7f422c155e90 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f4213fff640 1 -- 172.21.15.47:0/2333147464 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4220007a30 con 0x7f422c155e90 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f4231f1b640 1 -- 172.21.15.47:0/2333147464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c155650 con 0x7f422c155e90 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.394+0000 7f4213fff640 1 -- 172.21.15.47:0/2333147464 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220006be0 con 0x7f422c155e90 2024-03-23T09:43:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.395+0000 7f4231f1b640 1 -- 172.21.15.47:0/2333147464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c1f2e90 con 0x7f422c155e90 2024-03-23T09:43:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.396+0000 7f4231f1b640 1 -- 172.21.15.47:0/2333147464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c158d60 con 0x7f422c155e90 2024-03-23T09:43:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.396+0000 7f4213fff640 1 -- 172.21.15.47:0/2333147464 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4220006d40 con 0x7f422c155e90 2024-03-23T09:43:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.396+0000 7f4213fff640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f421806ac30 0x7f421806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.396+0000 7f4213fff640 1 -- 172.21.15.47:0/2333147464 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42200ab3a0 con 0x7f422c155e90 2024-03-23T09:43:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.397+0000 7f422b7fe640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f421806ac30 0x7f421806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.397+0000 7f422b7fe640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f421806ac30 0x7f421806d0f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f421c006fd0 tx=0x7f421c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.400+0000 7f4213fff640 1 -- 172.21.15.47:0/2333147464 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4220076750 con 0x7f422c155e90 2024-03-23T09:43:35.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.580+0000 7f4231f1b640 1 -- 172.21.15.47:0/2333147464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f422c066be0 con 0x7f422c155e90 2024-03-23T09:43:35.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.581+0000 7f4213fff640 1 -- 172.21.15.47:0/2333147464 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16045 (secure 0 0 0) 0x7f4220079d70 con 0x7f422c155e90 2024-03-23T09:43:35.581 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:35.581 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":46,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:49.167591+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:42:46.053713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-23T09:42:47.131279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:43:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.583+0000 7f4231f1b640 1 -- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f421806ac30 msgr2=0x7f421806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.583+0000 7f4231f1b640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f421806ac30 0x7f421806d0f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f421c006fd0 tx=0x7f421c008040 comp rx=0 tx=0).stop 2024-03-23T09:43:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.583+0000 7f4231f1b640 1 -- 172.21.15.47:0/2333147464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f422c155e90 msgr2=0x7f422c1f2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.583+0000 7f4231f1b640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f422c155e90 0x7f422c1f2770 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4220008fd0 tx=0x7f4220001120 comp rx=0 tx=0).stop 2024-03-23T09:43:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.583+0000 7f4231f1b640 1 -- 172.21.15.47:0/2333147464 shutdown_connections 2024-03-23T09:43:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.583+0000 7f4231f1b640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f421806ac30 0x7f421806d0f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.583+0000 7f4231f1b640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f422c1558b0 0x7f422c15e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.583+0000 7f4231f1b640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f422c155e90 0x7f422c1f2770 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.583+0000 7f4231f1b640 1 --2- 172.21.15.47:0/2333147464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4214003fd0 0x7f422c155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.583+0000 7f4231f1b640 1 -- 172.21.15.47:0/2333147464 >> 172.21.15.47:0/2333147464 conn(0x7f422c0b2140 msgr2=0x7f422c06c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.583+0000 7f4231f1b640 1 -- 172.21.15.47:0/2333147464 shutdown_connections 2024-03-23T09:43:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.583+0000 7f4231f1b640 1 -- 172.21.15.47:0/2333147464 wait complete. 2024-03-23T09:43:35.594 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T09:43:35.594 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:43:35.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.694+0000 7fbd994ce640 1 Processor -- start 2024-03-23T09:43:35.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.695+0000 7fbd994ce640 1 -- start start 2024-03-23T09:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.695+0000 7fbd994ce640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbd9415ca90 0x7fbd94160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.695+0000 7fbd994ce640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd940d11e0 con 0x7fbd940cde10 2024-03-23T09:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.695+0000 7fbd994ce640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd94160f60 con 0x7fbd940b8350 2024-03-23T09:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.695+0000 7fbd994ce640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd94161240 con 0x7fbd9415ca90 2024-03-23T09:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.695+0000 7fbd937fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbd940cde10 0x7fbd940d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47074/0 (socket says 172.21.15.47:47074) 2024-03-23T09:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.695+0000 7fbd937fe640 1 -- 172.21.15.47:0/2715609360 learned_addr learned my addr 172.21.15.47:0/2715609360 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534668899 0 0) 0x7fbd940d11e0 con 0x7fbd940cde10 2024-03-23T09:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd80003520 con 0x7fbd940cde10 2024-03-23T09:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd92ffd640 1 --2- 172.21.15.47:0/2715609360 >> v2:172.21.15.175:3300/0 conn(0x7fbd9415ca90 0x7fbd94160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2171707971 0 0) 0x7fbd80003520 con 0x7fbd940cde10 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 >> v2:172.21.15.175:3300/0 conn(0x7fbd9415ca90 msgr2=0x7fbd94160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 --2- 172.21.15.47:0/2715609360 >> v2:172.21.15.175:3300/0 conn(0x7fbd9415ca90 0x7fbd94160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 >> v1:172.21.15.175:6789/0 conn(0x7fbd940b8350 legacy=0x7fbd940cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd94161520 con 0x7fbd940cde10 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fbd840019f0 con 0x7fbd940cde10 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 >> v1:172.21.15.47:6789/0 conn(0x7fbd940cde10 legacy=0x7fbd940d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 --2- 172.21.15.47:0/2715609360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd800039c0 0x7fbd80003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 --2- 172.21.15.47:0/2715609360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd800047c0 0x7fbd80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 --2- 172.21.15.47:0/2715609360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd80005250 0x7fbd8000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd94161240 con 0x7fbd80005250 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8000c1a0 con 0x7fbd800047c0 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8000c360 con 0x7fbd800039c0 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd937fe640 1 --2- 172.21.15.47:0/2715609360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd80005250 0x7fbd8000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd927fc640 1 --2- 172.21.15.47:0/2715609360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd800047c0 0x7fbd80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd92ffd640 1 --2- 172.21.15.47:0/2715609360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd800039c0 0x7fbd80003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd937fe640 1 -- 172.21.15.47:0/2715609360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd800039c0 msgr2=0x7fbd80003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd937fe640 1 --2- 172.21.15.47:0/2715609360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd800039c0 0x7fbd80003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd937fe640 1 -- 172.21.15.47:0/2715609360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd800047c0 msgr2=0x7fbd80004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd937fe640 1 --2- 172.21.15.47:0/2715609360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd800047c0 0x7fbd80004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.696+0000 7fbd937fe640 1 -- 172.21.15.47:0/2715609360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd80003520 con 0x7fbd80005250 2024-03-23T09:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.697+0000 7fbd937fe640 1 -- 172.21.15.47:0/2715609360 reap_dead start 2024-03-23T09:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.697+0000 7fbd937fe640 1 --2- 172.21.15.47:0/2715609360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd80005250 0x7fbd8000b980 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400a7e0 tx=0x7fbd800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335be2e7ddad070b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.697+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84034630 con 0x7fbd80005250 2024-03-23T09:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.697+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8402f070 con 0x7fbd80005250 2024-03-23T09:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.697+0000 7fbd91ffb640 1 -- 172.21.15.47:0/2715609360 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd840349d0 con 0x7fbd80005250 2024-03-23T09:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.698+0000 7fbd994ce640 1 -- 172.21.15.47:0/2715609360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd80005250 msgr2=0x7fbd8000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.698+0000 7fbd994ce640 1 --2- 172.21.15.47:0/2715609360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd80005250 0x7fbd8000b980 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400a7e0 tx=0x7fbd800057f0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.698+0000 7fbd994ce640 1 -- 172.21.15.47:0/2715609360 shutdown_connections 2024-03-23T09:43:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.698+0000 7fbd994ce640 1 --2- 172.21.15.47:0/2715609360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd80005250 0x7fbd8000b980 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.698+0000 7fbd994ce640 1 -- 172.21.15.47:0/2715609360 >> 172.21.15.47:0/2715609360 conn(0x7fbd940659d0 msgr2=0x7fbd94065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.698+0000 7fbd994ce640 1 -- 172.21.15.47:0/2715609360 shutdown_connections 2024-03-23T09:43:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.698+0000 7fbd994ce640 1 -- 172.21.15.47:0/2715609360 wait complete. 2024-03-23T09:43:35.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.699+0000 7fbd994ce640 1 Processor -- start 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.699+0000 7fbd994ce640 1 -- start start 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd994ce640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd80005250 0x7fbd940ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd994ce640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd94152ba0 0x7fbd940cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd994ce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd9415f260 0x7fbd9415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd994ce640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9415fc20 con 0x7fbd80005250 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd994ce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd94158490 con 0x7fbd9415f260 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd994ce640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd940b2420 con 0x7fbd94152ba0 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd92ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd80005250 0x7fbd940ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd92ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd80005250 0x7fbd940ca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45780/0 (socket says 172.21.15.47:45780) 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd92ffd640 1 -- 172.21.15.47:0/3999891794 learned_addr learned my addr 172.21.15.47:0/3999891794 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd92ffd640 1 -- 172.21.15.47:0/3999891794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd94152ba0 msgr2=0x7fbd940cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd92ffd640 1 --2- 172.21.15.47:0/3999891794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd94152ba0 0x7fbd940cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd92ffd640 1 -- 172.21.15.47:0/3999891794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd9415f260 msgr2=0x7fbd9415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd937fe640 1 --2- 172.21.15.47:0/3999891794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd9415f260 0x7fbd9415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd92ffd640 1 --2- 172.21.15.47:0/3999891794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd9415f260 0x7fbd9415f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd92ffd640 1 -- 172.21.15.47:0/3999891794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd940cbb30 con 0x7fbd80005250 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.700+0000 7fbd92ffd640 1 --2- 172.21.15.47:0/3999891794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd80005250 0x7fbd940ca4a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00b7c0 tx=0x7fbd8c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.701+0000 7fbd8b7fe640 1 -- 172.21.15.47:0/3999891794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c0107d0 con 0x7fbd80005250 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.701+0000 7fbd8b7fe640 1 -- 172.21.15.47:0/3999891794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c010930 con 0x7fbd80005250 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.701+0000 7fbd8b7fe640 1 -- 172.21.15.47:0/3999891794 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c010cc0 con 0x7fbd80005250 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.701+0000 7fbd994ce640 1 -- 172.21.15.47:0/3999891794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd940cab10 con 0x7fbd80005250 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.701+0000 7fbd994ce640 1 -- 172.21.15.47:0/3999891794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd940b2950 con 0x7fbd80005250 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.702+0000 7fbd994ce640 1 -- 172.21.15.47:0/3999891794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd94072360 con 0x7fbd80005250 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.702+0000 7fbd8b7fe640 1 -- 172.21.15.47:0/3999891794 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbd8c014430 con 0x7fbd80005250 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.702+0000 7fbd8b7fe640 1 --2- 172.21.15.47:0/3999891794 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd7406ac00 0x7fbd7406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.703+0000 7fbd8b7fe640 1 -- 172.21.15.47:0/3999891794 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd8c089aa0 con 0x7fbd80005250 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.703+0000 7fbd927fc640 1 --2- 172.21.15.47:0/3999891794 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd7406ac00 0x7fbd7406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.703+0000 7fbd927fc640 1 --2- 172.21.15.47:0/3999891794 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd7406ac00 0x7fbd7406d0c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbd940c8d30 tx=0x7fbd7c0049b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.707+0000 7fbd8b7fe640 1 -- 172.21.15.47:0/3999891794 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbd8c054be0 con 0x7fbd80005250 2024-03-23T09:43:35.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.927+0000 7fbd994ce640 1 -- 172.21.15.47:0/3999891794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd94066be0 con 0x7fbd80005250 2024-03-23T09:43:35.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.927+0000 7fbd8b7fe640 1 -- 172.21.15.47:0/3999891794 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fbd8c022020 con 0x7fbd80005250 2024-03-23T09:43:35.927 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:35.927 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1189831856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1189831856},{"type":"v1","addr":"172.21.15.47:6835","nonce":1189831856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/2248064454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2248064454},{"type":"v1","addr":"172.21.15.47:6834","nonce":2248064454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5694,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3806465612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3806465612},{"type":"v1","addr":"172.21.15.175:6835","nonce":3806465612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:42:48.165819+0000","modified":"2024-03-23T09:42:50.171529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1626348200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1626348200},{"type":"v1","addr":"172.21.15.175:6837","nonce":1626348200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-23T09:43:35.928 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 18 2024-03-23T09:43:35.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.929+0000 7fbd994ce640 1 -- 172.21.15.47:0/3999891794 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd7406ac00 msgr2=0x7fbd7406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.929+0000 7fbd994ce640 1 --2- 172.21.15.47:0/3999891794 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd7406ac00 0x7fbd7406d0c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbd940c8d30 tx=0x7fbd7c0049b0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.929+0000 7fbd994ce640 1 -- 172.21.15.47:0/3999891794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd80005250 msgr2=0x7fbd940ca4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:35.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.929+0000 7fbd994ce640 1 --2- 172.21.15.47:0/3999891794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd80005250 0x7fbd940ca4a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00b7c0 tx=0x7fbd8c00bc80 comp rx=0 tx=0).stop 2024-03-23T09:43:35.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.929+0000 7fbd994ce640 1 -- 172.21.15.47:0/3999891794 shutdown_connections 2024-03-23T09:43:35.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.929+0000 7fbd994ce640 1 --2- 172.21.15.47:0/3999891794 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd7406ac00 0x7fbd7406d0c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.929+0000 7fbd994ce640 1 --2- 172.21.15.47:0/3999891794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd9415f260 0x7fbd9415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.929+0000 7fbd994ce640 1 --2- 172.21.15.47:0/3999891794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd94152ba0 0x7fbd940cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.929+0000 7fbd994ce640 1 --2- 172.21.15.47:0/3999891794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd80005250 0x7fbd940ca4a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:35.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.929+0000 7fbd994ce640 1 -- 172.21.15.47:0/3999891794 >> 172.21.15.47:0/3999891794 conn(0x7fbd940659d0 msgr2=0x7fbd940b7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:35.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.929+0000 7fbd994ce640 1 -- 172.21.15.47:0/3999891794 shutdown_connections 2024-03-23T09:43:35.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:35.929+0000 7fbd994ce640 1 -- 172.21.15.47:0/3999891794 wait complete. 2024-03-23T09:43:35.940 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:43:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.041+0000 7f9eaae8b640 1 Processor -- start 2024-03-23T09:43:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.041+0000 7f9eaae8b640 1 -- start start 2024-03-23T09:43:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.042+0000 7f9eaae8b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ea415ca90 0x7f9ea4160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.042+0000 7f9eaae8b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9ea40b8350 0x7f9ea40cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.042+0000 7f9eaae8b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea40d11e0 con 0x7f9ea40cdc40 2024-03-23T09:43:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.042+0000 7f9eaae8b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea40ce0d0 con 0x7f9ea40b8350 2024-03-23T09:43:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.042+0000 7f9eaae8b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40d2680 con 0x7f9ea415ca90 2024-03-23T09:43:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.042+0000 7f9ea8c27640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ea415ca90 0x7f9ea4160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.042+0000 7f9ea3fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9ea40b8350 0x7f9ea40cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.042+0000 7f9ea3fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9ea40b8350 0x7f9ea40cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:41882/0 (socket says 172.21.15.47:41882) 2024-03-23T09:43:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.042+0000 7f9ea8c27640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ea415ca90 0x7f9ea4160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:53356/0 (socket says 172.21.15.47:53356) 2024-03-23T09:43:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.042+0000 7f9ea3fff640 1 -- 172.21.15.47:0/1322271850 learned_addr learned my addr 172.21.15.47:0/1322271850 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.042+0000 7f9ea9428640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f9ea40cdc40 0x7f9ea40b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:52918/0 (socket says 172.21.15.47:52918) 2024-03-23T09:43:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.043+0000 7f9ea3fff640 1 -- 172.21.15.47:0/1322271850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ea415ca90 msgr2=0x7f9ea4160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.043+0000 7f9ea3fff640 1 --2- 172.21.15.47:0/1322271850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ea415ca90 0x7f9ea4160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.043+0000 7f9ea3fff640 1 -- 172.21.15.47:0/1322271850 >> v1:172.21.15.175:6789/0 conn(0x7f9ea40cdc40 legacy=0x7f9ea40b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.043+0000 7f9ea3fff640 1 -- 172.21.15.47:0/1322271850 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4160f60 con 0x7f9ea40b8350 2024-03-23T09:43:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.043+0000 7f9ea3fff640 1 --2- 172.21.15.47:0/1322271850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea40b8350 0x7f9ea40cd700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009870 tx=0x7f9e9802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f26b1556defbd79 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.044+0000 7f9ea37fe640 1 -- 172.21.15.47:0/1322271850 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98031640 con 0x7f9ea40b8350 2024-03-23T09:43:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.044+0000 7f9ea37fe640 1 -- 172.21.15.47:0/1322271850 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e980317a0 con 0x7f9ea40b8350 2024-03-23T09:43:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.044+0000 7f9ea37fe640 1 -- 172.21.15.47:0/1322271850 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98031b30 con 0x7f9ea40b8350 2024-03-23T09:43:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.044+0000 7f9eaae8b640 1 -- 172.21.15.47:0/1322271850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea40b8350 msgr2=0x7f9ea40cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.044+0000 7f9eaae8b640 1 --2- 172.21.15.47:0/1322271850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea40b8350 0x7f9ea40cd700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009870 tx=0x7f9e9802b8f0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.044+0000 7f9eaae8b640 1 -- 172.21.15.47:0/1322271850 shutdown_connections 2024-03-23T09:43:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.044+0000 7f9eaae8b640 1 --2- 172.21.15.47:0/1322271850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea40b8350 0x7f9ea40cd700 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.044+0000 7f9eaae8b640 1 --2- 172.21.15.47:0/1322271850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ea415ca90 0x7f9ea4160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.044+0000 7f9eaae8b640 1 -- 172.21.15.47:0/1322271850 >> 172.21.15.47:0/1322271850 conn(0x7f9ea40659d0 msgr2=0x7f9ea4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.044+0000 7f9eaae8b640 1 -- 172.21.15.47:0/1322271850 shutdown_connections 2024-03-23T09:43:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.044+0000 7f9eaae8b640 1 -- 172.21.15.47:0/1322271850 wait complete. 2024-03-23T09:43:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.045+0000 7f9eaae8b640 1 Processor -- start 2024-03-23T09:43:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.045+0000 7f9eaae8b640 1 -- start start 2024-03-23T09:43:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.046+0000 7f9eaae8b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea40b8350 0x7f9ea415ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.046+0000 7f9eaae8b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ea40cdc40 0x7f9ea415f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.046+0000 7f9eaae8b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ea415ca90 0x7f9ea41f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.046+0000 7f9eaae8b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea415e1b0 con 0x7f9ea415ca90 2024-03-23T09:43:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.046+0000 7f9eaae8b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea415d3b0 con 0x7f9ea40b8350 2024-03-23T09:43:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.046+0000 7f9eaae8b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea415d630 con 0x7f9ea40cdc40 2024-03-23T09:43:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.046+0000 7f9ea9428640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ea415ca90 0x7f9ea41f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.046+0000 7f9ea9428640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ea415ca90 0x7f9ea41f22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45792/0 (socket says 172.21.15.47:45792) 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.046+0000 7f9ea9428640 1 -- 172.21.15.47:0/3586723190 learned_addr learned my addr 172.21.15.47:0/3586723190 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.046+0000 7f9ea8c27640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea40b8350 0x7f9ea415ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.046+0000 7f9ea3fff640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ea40cdc40 0x7f9ea415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.047+0000 7f9ea9428640 1 -- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ea40cdc40 msgr2=0x7f9ea415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.047+0000 7f9ea9428640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ea40cdc40 0x7f9ea415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.047+0000 7f9ea9428640 1 -- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea40b8350 msgr2=0x7f9ea415ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.047+0000 7f9ea9428640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea40b8350 0x7f9ea415ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.047+0000 7f9ea9428640 1 -- 172.21.15.47:0/3586723190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e980094d0 con 0x7f9ea415ca90 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.047+0000 7f9ea9428640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ea415ca90 0x7f9ea41f22b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c760 tx=0x7f9e9c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.047+0000 7f9ea17fa640 1 -- 172.21.15.47:0/3586723190 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00ce70 con 0x7f9ea415ca90 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.047+0000 7f9ea17fa640 1 -- 172.21.15.47:0/3586723190 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c00a640 con 0x7f9ea415ca90 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.047+0000 7f9eaae8b640 1 -- 172.21.15.47:0/3586723190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea415d910 con 0x7f9ea415ca90 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.047+0000 7f9ea17fa640 1 -- 172.21.15.47:0/3586723190 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00a9b0 con 0x7f9ea415ca90 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.047+0000 7f9eaae8b640 1 -- 172.21.15.47:0/3586723190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea415ded0 con 0x7f9ea415ca90 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.048+0000 7f9eaae8b640 1 -- 172.21.15.47:0/3586723190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea4072360 con 0x7f9ea415ca90 2024-03-23T09:43:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.048+0000 7f9ea17fa640 1 -- 172.21.15.47:0/3586723190 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9e9c00ab10 con 0x7f9ea415ca90 2024-03-23T09:43:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.049+0000 7f9ea17fa640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e9006ac00 0x7f9e9006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.049+0000 7f9ea17fa640 1 -- 172.21.15.47:0/3586723190 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e9c088a10 con 0x7f9ea415ca90 2024-03-23T09:43:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.049+0000 7f9ea8c27640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e9006ac00 0x7f9e9006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.049+0000 7f9ea8c27640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e9006ac00 0x7f9e9006d0c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009b40 tx=0x7f9e94009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.053+0000 7f9ea17fa640 1 -- 172.21.15.47:0/3586723190 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9e9c053cb0 con 0x7f9ea415ca90 2024-03-23T09:43:36.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.271+0000 7f9eaae8b640 1 -- 172.21.15.47:0/3586723190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ea40c9b20 con 0x7f9ea415ca90 2024-03-23T09:43:36.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.271+0000 7f9ea17fa640 1 -- 172.21.15.47:0/3586723190 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f9e9c0572d0 con 0x7f9ea415ca90 2024-03-23T09:43:36.271 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:36.271 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1189831856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1189831856},{"type":"v1","addr":"172.21.15.47:6835","nonce":1189831856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/2248064454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2248064454},{"type":"v1","addr":"172.21.15.47:6834","nonce":2248064454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5694,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3806465612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3806465612},{"type":"v1","addr":"172.21.15.175:6835","nonce":3806465612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:42:48.165819+0000","modified":"2024-03-23T09:42:50.171529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/1626348200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1626348200},{"type":"v1","addr":"172.21.15.175:6837","nonce":1626348200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-23T09:43:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 18 2024-03-23T09:43:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.273+0000 7f9eaae8b640 1 -- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e9006ac00 msgr2=0x7f9e9006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.273+0000 7f9eaae8b640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e9006ac00 0x7f9e9006d0c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009b40 tx=0x7f9e94009190 comp rx=0 tx=0).stop 2024-03-23T09:43:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.273+0000 7f9eaae8b640 1 -- 172.21.15.47:0/3586723190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ea415ca90 msgr2=0x7f9ea41f22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.273+0000 7f9eaae8b640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ea415ca90 0x7f9ea41f22b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c760 tx=0x7f9e9c00cc20 comp rx=0 tx=0).stop 2024-03-23T09:43:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.273+0000 7f9eaae8b640 1 -- 172.21.15.47:0/3586723190 shutdown_connections 2024-03-23T09:43:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.273+0000 7f9eaae8b640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e9006ac00 0x7f9e9006d0c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.273+0000 7f9eaae8b640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ea415ca90 0x7f9ea41f22b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.273+0000 7f9eaae8b640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ea40cdc40 0x7f9ea415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.273+0000 7f9eaae8b640 1 --2- 172.21.15.47:0/3586723190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea40b8350 0x7f9ea415ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.273+0000 7f9eaae8b640 1 -- 172.21.15.47:0/3586723190 >> 172.21.15.47:0/3586723190 conn(0x7f9ea40659d0 msgr2=0x7f9ea40cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:36.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.273+0000 7f9eaae8b640 1 -- 172.21.15.47:0/3586723190 shutdown_connections 2024-03-23T09:43:36.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.273+0000 7f9eaae8b640 1 -- 172.21.15.47:0/3586723190 wait complete. 2024-03-23T09:43:36.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:43:36.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.385+0000 7f108bba4640 1 Processor -- start 2024-03-23T09:43:36.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.385+0000 7f108bba4640 1 -- start start 2024-03-23T09:43:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.386+0000 7f108bba4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10840b3c30 0x7f108406d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.386+0000 7f108bba4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f108406c350 0x7f10840b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.386+0000 7f108bba4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10840b4550 con 0x7f1084142aa0 2024-03-23T09:43:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.386+0000 7f108bba4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108406c8c0 con 0x7f108406c350 2024-03-23T09:43:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.386+0000 7f108bba4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108406cba0 con 0x7f10840b3c30 2024-03-23T09:43:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.386+0000 7f108913f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f108406c350 0x7f10840b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.386+0000 7f108913f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f108406c350 0x7f10840b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45796/0 (socket says 172.21.15.47:45796) 2024-03-23T09:43:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.386+0000 7f108913f640 1 -- 172.21.15.47:0/3505494851 learned_addr learned my addr 172.21.15.47:0/3505494851 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.386+0000 7f108913f640 1 -- 172.21.15.47:0/3505494851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10840b3c30 msgr2=0x7f108406d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.386+0000 7f1089940640 1 --2- 172.21.15.47:0/3505494851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10840b3c30 0x7f108406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.386+0000 7f108913f640 1 --2- 172.21.15.47:0/3505494851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10840b3c30 0x7f108406d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.386+0000 7f108913f640 1 -- 172.21.15.47:0/3505494851 >> v1:172.21.15.175:6789/0 conn(0x7f1084142aa0 legacy=0x7f10840cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.386+0000 7f108913f640 1 -- 172.21.15.47:0/3505494851 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108406ce80 con 0x7f108406c350 2024-03-23T09:43:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.387+0000 7f108913f640 1 --2- 172.21.15.47:0/3505494851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108406c350 0x7f10840b4010 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1074009870 tx=0x7f107402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba4234f5b8744db server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.387+0000 7f108893e640 1 -- 172.21.15.47:0/3505494851 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074031640 con 0x7f108406c350 2024-03-23T09:43:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.387+0000 7f108893e640 1 -- 172.21.15.47:0/3505494851 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10740317a0 con 0x7f108406c350 2024-03-23T09:43:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.387+0000 7f108893e640 1 -- 172.21.15.47:0/3505494851 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074031b30 con 0x7f108406c350 2024-03-23T09:43:36.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.388+0000 7f108bba4640 1 -- 172.21.15.47:0/3505494851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108406c350 msgr2=0x7f10840b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.388+0000 7f108bba4640 1 --2- 172.21.15.47:0/3505494851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108406c350 0x7f10840b4010 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1074009870 tx=0x7f107402b8f0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.388+0000 7f108bba4640 1 -- 172.21.15.47:0/3505494851 shutdown_connections 2024-03-23T09:43:36.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.388+0000 7f108bba4640 1 --2- 172.21.15.47:0/3505494851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108406c350 0x7f10840b4010 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.388+0000 7f108bba4640 1 --2- 172.21.15.47:0/3505494851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10840b3c30 0x7f108406d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.388+0000 7f108bba4640 1 -- 172.21.15.47:0/3505494851 >> 172.21.15.47:0/3505494851 conn(0x7f108406a310 msgr2=0x7f10840b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:36.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.388+0000 7f108bba4640 1 -- 172.21.15.47:0/3505494851 shutdown_connections 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.388+0000 7f108bba4640 1 -- 172.21.15.47:0/3505494851 wait complete. 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.389+0000 7f108bba4640 1 Processor -- start 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.389+0000 7f108bba4640 1 -- start start 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f108bba4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108406c350 0x7f1084141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f108bba4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10840b3c30 0x7f108413ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f108bba4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1084142aa0 0x7f1084138f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f108bba4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1084141f70 con 0x7f108406c350 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f108bba4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1084142480 con 0x7f1084142aa0 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f108bba4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1084158480 con 0x7f10840b3c30 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f1089940640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108406c350 0x7f1084141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f1089940640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108406c350 0x7f1084141a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45808/0 (socket says 172.21.15.47:45808) 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f1089940640 1 -- 172.21.15.47:0/2787524873 learned_addr learned my addr 172.21.15.47:0/2787524873 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f108913f640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10840b3c30 0x7f108413ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f1089940640 1 -- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10840b3c30 msgr2=0x7f108413ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f108a141640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1084142aa0 0x7f1084138f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f1089940640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10840b3c30 0x7f108413ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f1089940640 1 -- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1084142aa0 msgr2=0x7f1084138f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f1089940640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1084142aa0 0x7f1084138f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f1089940640 1 -- 172.21.15.47:0/2787524873 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10740094d0 con 0x7f108406c350 2024-03-23T09:43:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f1089940640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108406c350 0x7f1084141a30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f108000c7f0 tx=0x7f108000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f107a7fc640 1 -- 172.21.15.47:0/2787524873 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080004220 con 0x7f108406c350 2024-03-23T09:43:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f107a7fc640 1 -- 172.21.15.47:0/2787524873 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10800043a0 con 0x7f108406c350 2024-03-23T09:43:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f108bba4640 1 -- 172.21.15.47:0/2787524873 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10841421a0 con 0x7f108406c350 2024-03-23T09:43:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f107a7fc640 1 -- 172.21.15.47:0/2787524873 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080007860 con 0x7f108406c350 2024-03-23T09:43:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.390+0000 7f108bba4640 1 -- 172.21.15.47:0/2787524873 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1084158110 con 0x7f108406c350 2024-03-23T09:43:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.391+0000 7f108bba4640 1 -- 172.21.15.47:0/2787524873 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1084073480 con 0x7f108406c350 2024-03-23T09:43:36.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.392+0000 7f107a7fc640 1 -- 172.21.15.47:0/2787524873 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1080007bd0 con 0x7f108406c350 2024-03-23T09:43:36.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.392+0000 7f107a7fc640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f106806ac00 0x7f106806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.392+0000 7f107a7fc640 1 -- 172.21.15.47:0/2787524873 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1080089340 con 0x7f108406c350 2024-03-23T09:43:36.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.393+0000 7f108913f640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f106806ac00 0x7f106806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.393+0000 7f108913f640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f106806ac00 0x7f106806d0c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1074036000 tx=0x7f107403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.397+0000 7f107a7fc640 1 -- 172.21.15.47:0/2787524873 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1080054520 con 0x7f108406c350 2024-03-23T09:43:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.577+0000 7f108bba4640 1 -- 172.21.15.47:0/2787524873 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1084066be0 con 0x7f108406c350 2024-03-23T09:43:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.578+0000 7f107a7fc640 1 -- 172.21.15.47:0/2787524873 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16045 (secure 0 0 0) 0x7f1080057b40 con 0x7f108406c350 2024-03-23T09:43:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:36.578 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":46,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:42:49.167591+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:42:46.053713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-23T09:42:47.131279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:43:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.580+0000 7f108bba4640 1 -- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f106806ac00 msgr2=0x7f106806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.580+0000 7f108bba4640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f106806ac00 0x7f106806d0c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1074036000 tx=0x7f107403c040 comp rx=0 tx=0).stop 2024-03-23T09:43:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.580+0000 7f108bba4640 1 -- 172.21.15.47:0/2787524873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108406c350 msgr2=0x7f1084141a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.580+0000 7f108bba4640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108406c350 0x7f1084141a30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f108000c7f0 tx=0x7f108000ccb0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.580+0000 7f108bba4640 1 -- 172.21.15.47:0/2787524873 shutdown_connections 2024-03-23T09:43:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.580+0000 7f108bba4640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f106806ac00 0x7f106806d0c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.580+0000 7f108bba4640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1084142aa0 0x7f1084138f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.580+0000 7f108bba4640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10840b3c30 0x7f108413ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.580+0000 7f108bba4640 1 --2- 172.21.15.47:0/2787524873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108406c350 0x7f1084141a30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.580+0000 7f108bba4640 1 -- 172.21.15.47:0/2787524873 >> 172.21.15.47:0/2787524873 conn(0x7f108406a310 msgr2=0x7f10840701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:36.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.580+0000 7f108bba4640 1 -- 172.21.15.47:0/2787524873 shutdown_connections 2024-03-23T09:43:36.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.580+0000 7f108bba4640 1 -- 172.21.15.47:0/2787524873 wait complete. 2024-03-23T09:43:36.591 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:43:36.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.693+0000 7fe064f73640 1 Processor -- start 2024-03-23T09:43:36.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.693+0000 7fe064f73640 1 -- start start 2024-03-23T09:43:36.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.694+0000 7fe064f73640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe0600cf210 0x7fe0600cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.694+0000 7fe064f73640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0600b5a90 0x7fe0600b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.694+0000 7fe064f73640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0600cfbd0 con 0x7fe06015cd00 2024-03-23T09:43:36.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.694+0000 7fe064f73640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe06015d0e0 con 0x7fe0600b5a90 2024-03-23T09:43:36.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.694+0000 7fe064f73640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0600d1540 con 0x7fe0600cf210 2024-03-23T09:43:36.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.694+0000 7fe05ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0600b5a90 0x7fe0600b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.694+0000 7fe05ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0600b5a90 0x7fe0600b5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45818/0 (socket says 172.21.15.47:45818) 2024-03-23T09:43:36.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.694+0000 7fe05ed9d640 1 -- 172.21.15.47:0/1754335484 learned_addr learned my addr 172.21.15.47:0/1754335484 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:36.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.694+0000 7fe05dd9b640 1 --2- 172.21.15.47:0/1754335484 >> v2:172.21.15.175:3300/0 conn(0x7fe0600cf210 0x7fe0600cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.695+0000 7fe05ed9d640 1 -- 172.21.15.47:0/1754335484 >> v2:172.21.15.175:3300/0 conn(0x7fe0600cf210 msgr2=0x7fe0600cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:36.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.695+0000 7fe05ed9d640 1 --2- 172.21.15.47:0/1754335484 >> v2:172.21.15.175:3300/0 conn(0x7fe0600cf210 0x7fe0600cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.695+0000 7fe05ed9d640 1 -- 172.21.15.47:0/1754335484 >> v1:172.21.15.175:6789/0 conn(0x7fe06015cd00 legacy=0x7fe0600d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:36.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.695+0000 7fe05ed9d640 1 -- 172.21.15.47:0/1754335484 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0600d1820 con 0x7fe0600b5a90 2024-03-23T09:43:36.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.695+0000 7fe05ed9d640 1 --2- 172.21.15.47:0/1754335484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0600b5a90 0x7fe0600b5ee0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe0600b5520 tx=0x7fe05002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db59fb06737ca3c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:36.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.695+0000 7fe05d59a640 1 -- 172.21.15.47:0/1754335484 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05002ca80 con 0x7fe0600b5a90 2024-03-23T09:43:36.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.696+0000 7fe05d59a640 1 -- 172.21.15.47:0/1754335484 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05002cbe0 con 0x7fe0600b5a90 2024-03-23T09:43:36.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.696+0000 7fe05d59a640 1 -- 172.21.15.47:0/1754335484 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0500343f0 con 0x7fe0600b5a90 2024-03-23T09:43:36.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.696+0000 7fe064f73640 1 -- 172.21.15.47:0/1754335484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0600b5a90 msgr2=0x7fe0600b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.696+0000 7fe064f73640 1 --2- 172.21.15.47:0/1754335484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0600b5a90 0x7fe0600b5ee0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe0600b5520 tx=0x7fe05002bee0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.696+0000 7fe064f73640 1 -- 172.21.15.47:0/1754335484 shutdown_connections 2024-03-23T09:43:36.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.696+0000 7fe064f73640 1 --2- 172.21.15.47:0/1754335484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0600b5a90 0x7fe0600b5ee0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.696+0000 7fe064f73640 1 --2- 172.21.15.47:0/1754335484 >> v2:172.21.15.175:3300/0 conn(0x7fe0600cf210 0x7fe0600cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.696+0000 7fe064f73640 1 -- 172.21.15.47:0/1754335484 >> 172.21.15.47:0/1754335484 conn(0x7fe06006b160 msgr2=0x7fe06006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:36.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.696+0000 7fe064f73640 1 -- 172.21.15.47:0/1754335484 shutdown_connections 2024-03-23T09:43:36.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.696+0000 7fe064f73640 1 -- 172.21.15.47:0/1754335484 wait complete. 2024-03-23T09:43:36.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.697+0000 7fe064f73640 1 Processor -- start 2024-03-23T09:43:36.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.697+0000 7fe064f73640 1 -- start start 2024-03-23T09:43:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.698+0000 7fe064f73640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0600b5a90 0x7fe060158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.698+0000 7fe064f73640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0600cf210 0x7fe0600a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.698+0000 7fe064f73640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe06015cd00 0x7fe0600b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.698+0000 7fe064f73640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0601f50e0 con 0x7fe06015cd00 2024-03-23T09:43:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.698+0000 7fe064f73640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0601f44d0 con 0x7fe0600b5a90 2024-03-23T09:43:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.698+0000 7fe064f73640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0601f47b0 con 0x7fe0600cf210 2024-03-23T09:43:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.698+0000 7fe05ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe06015cd00 0x7fe0600b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.698+0000 7fe05ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe06015cd00 0x7fe0600b5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45834/0 (socket says 172.21.15.47:45834) 2024-03-23T09:43:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.698+0000 7fe05ed9d640 1 -- 172.21.15.47:0/102059278 learned_addr learned my addr 172.21.15.47:0/102059278 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe05e59c640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0600b5a90 0x7fe060158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe05dd9b640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0600cf210 0x7fe0600a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe05ed9d640 1 -- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0600cf210 msgr2=0x7fe0600a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe05ed9d640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0600cf210 0x7fe0600a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe05ed9d640 1 -- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0600b5a90 msgr2=0x7fe060158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe05ed9d640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0600b5a90 0x7fe060158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe05ed9d640 1 -- 172.21.15.47:0/102059278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe050009b30 con 0x7fe06015cd00 2024-03-23T09:43:36.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe05e59c640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0600b5a90 0x7fe060158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:36.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe05ed9d640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe06015cd00 0x7fe0600b5e70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe050009f40 tx=0x7fe050004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:36.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe056ffd640 1 -- 172.21.15.47:0/102059278 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe050009510 con 0x7fe06015cd00 2024-03-23T09:43:36.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe056ffd640 1 -- 172.21.15.47:0/102059278 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05003e450 con 0x7fe06015cd00 2024-03-23T09:43:36.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe064f73640 1 -- 172.21.15.47:0/102059278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0601f4a90 con 0x7fe06015cd00 2024-03-23T09:43:36.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe056ffd640 1 -- 172.21.15.47:0/102059278 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05003e7e0 con 0x7fe06015cd00 2024-03-23T09:43:36.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.699+0000 7fe064f73640 1 -- 172.21.15.47:0/102059278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe060159410 con 0x7fe06015cd00 2024-03-23T09:43:36.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.700+0000 7fe064f73640 1 -- 172.21.15.47:0/102059278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe060072360 con 0x7fe06015cd00 2024-03-23T09:43:36.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.700+0000 7fe056ffd640 1 -- 172.21.15.47:0/102059278 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe05002cd50 con 0x7fe06015cd00 2024-03-23T09:43:36.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.701+0000 7fe056ffd640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe04006ac30 0x7fe04006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:36.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.701+0000 7fe056ffd640 1 -- 172.21.15.47:0/102059278 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0500aba40 con 0x7fe06015cd00 2024-03-23T09:43:36.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.701+0000 7fe05e59c640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe04006ac30 0x7fe04006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:36.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.702+0000 7fe05e59c640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe04006ac30 0x7fe04006d0f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe0580015e0 tx=0x7fe058009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:36.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.705+0000 7fe056ffd640 1 -- 172.21.15.47:0/102059278 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe0500ade60 con 0x7fe06015cd00 2024-03-23T09:43:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.924+0000 7fe064f73640 1 -- 172.21.15.47:0/102059278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe0600674f0 con 0x7fe06015cd00 2024-03-23T09:43:36.924 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:43:36.925+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:43:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.931+0000 7fe056ffd640 1 -- 172.21.15.47:0/102059278 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe050076cb0 con 0x7fe06015cd00 2024-03-23T09:43:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:43:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.933+0000 7fe064f73640 1 -- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe04006ac30 msgr2=0x7fe04006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.933+0000 7fe064f73640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe04006ac30 0x7fe04006d0f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe0580015e0 tx=0x7fe058009040 comp rx=0 tx=0).stop 2024-03-23T09:43:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.933+0000 7fe064f73640 1 -- 172.21.15.47:0/102059278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe06015cd00 msgr2=0x7fe0600b5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.933+0000 7fe064f73640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe06015cd00 0x7fe0600b5e70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe050009f40 tx=0x7fe050004a40 comp rx=0 tx=0).stop 2024-03-23T09:43:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.933+0000 7fe064f73640 1 -- 172.21.15.47:0/102059278 shutdown_connections 2024-03-23T09:43:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.933+0000 7fe064f73640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe04006ac30 0x7fe04006d0f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.933+0000 7fe064f73640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe06015cd00 0x7fe0600b5e70 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.933+0000 7fe064f73640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0600cf210 0x7fe0600a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.933+0000 7fe064f73640 1 --2- 172.21.15.47:0/102059278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0600b5a90 0x7fe060158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.933+0000 7fe064f73640 1 -- 172.21.15.47:0/102059278 >> 172.21.15.47:0/102059278 conn(0x7fe06006b160 msgr2=0x7fe0600b7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.933+0000 7fe064f73640 1 -- 172.21.15.47:0/102059278 shutdown_connections 2024-03-23T09:43:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:36.933+0000 7fe064f73640 1 -- 172.21.15.47:0/102059278 wait complete. 2024-03-23T09:43:36.935 INFO:tasks.ceph.mds.d.smithi175.stderr: -4615> 2024-03-23T09:42:53.692+0000 7f3363a75640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:43:36.937 INFO:tasks.ceph.mds.d.smithi175.stderr: -3005> 2024-03-23T09:43:00.297+0000 7f336026e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5877=0-4194304@1} caps={5877=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5877 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x561836c16100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T09:43:36.944 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T09:43:36.968 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-03-23T09:43:37.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.047+0000 7f41dbb73640 1 Processor -- start 2024-03-23T09:43:37.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.047+0000 7f41dbb73640 1 -- start start 2024-03-23T09:43:37.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.048+0000 7f41dbb73640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f41d40d23a0 0x7f41d40d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:37.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.048+0000 7f41dbb73640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41d40b8300 0x7f41d40d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:37.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.048+0000 7f41dbb73640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d40d10a0 con 0x7f41d415cee0 2024-03-23T09:43:37.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.048+0000 7f41dbb73640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d40d1300 con 0x7f41d40b8300 2024-03-23T09:43:37.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.048+0000 7f41dbb73640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d40d15e0 con 0x7f41d40d23a0 2024-03-23T09:43:37.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.048+0000 7f41da110640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41d40b8300 0x7f41d40d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:37.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.048+0000 7f41da110640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41d40b8300 0x7f41d40d2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45836/0 (socket says 172.21.15.47:45836) 2024-03-23T09:43:37.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.048+0000 7f41da110640 1 -- 172.21.15.47:0/1453037846 learned_addr learned my addr 172.21.15.47:0/1453037846 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:37.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.048+0000 7f41d910e640 1 --2- 172.21.15.47:0/1453037846 >> v2:172.21.15.175:3300/0 conn(0x7f41d40d23a0 0x7f41d40d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:37.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.048+0000 7f41da110640 1 -- 172.21.15.47:0/1453037846 >> v2:172.21.15.175:3300/0 conn(0x7f41d40d23a0 msgr2=0x7f41d40d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:37.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.048+0000 7f41da110640 1 --2- 172.21.15.47:0/1453037846 >> v2:172.21.15.175:3300/0 conn(0x7f41d40d23a0 0x7f41d40d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:37.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.048+0000 7f41da110640 1 -- 172.21.15.47:0/1453037846 >> v1:172.21.15.175:6789/0 conn(0x7f41d415cee0 legacy=0x7f41d40d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:37.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.048+0000 7f41da110640 1 -- 172.21.15.47:0/1453037846 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d40d18c0 con 0x7f41d40b8300 2024-03-23T09:43:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.049+0000 7f41da110640 1 --2- 172.21.15.47:0/1453037846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41d40b8300 0x7f41d40d2db0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f41d40cf8d0 tx=0x7f41cc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b28bcf0d2a0f04 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.049+0000 7f41d890d640 1 -- 172.21.15.47:0/1453037846 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc02cb20 con 0x7f41d40b8300 2024-03-23T09:43:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.049+0000 7f41d890d640 1 -- 172.21.15.47:0/1453037846 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc02cc80 con 0x7f41d40b8300 2024-03-23T09:43:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.049+0000 7f41d890d640 1 -- 172.21.15.47:0/1453037846 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc0343f0 con 0x7f41d40b8300 2024-03-23T09:43:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.049+0000 7f41dbb73640 1 -- 172.21.15.47:0/1453037846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41d40b8300 msgr2=0x7f41d40d2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.049+0000 7f41dbb73640 1 --2- 172.21.15.47:0/1453037846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41d40b8300 0x7f41d40d2db0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f41d40cf8d0 tx=0x7f41cc02bf80 comp rx=0 tx=0).stop 2024-03-23T09:43:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.050+0000 7f41dbb73640 1 -- 172.21.15.47:0/1453037846 shutdown_connections 2024-03-23T09:43:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.050+0000 7f41dbb73640 1 --2- 172.21.15.47:0/1453037846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41d40b8300 0x7f41d40d2db0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.050+0000 7f41dbb73640 1 --2- 172.21.15.47:0/1453037846 >> v2:172.21.15.175:3300/0 conn(0x7f41d40d23a0 0x7f41d40d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.050+0000 7f41dbb73640 1 -- 172.21.15.47:0/1453037846 >> 172.21.15.47:0/1453037846 conn(0x7f41d4065df0 msgr2=0x7f41d4066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.050+0000 7f41dbb73640 1 -- 172.21.15.47:0/1453037846 shutdown_connections 2024-03-23T09:43:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.050+0000 7f41dbb73640 1 -- 172.21.15.47:0/1453037846 wait complete. 2024-03-23T09:43:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.051+0000 7f41dbb73640 1 Processor -- start 2024-03-23T09:43:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.051+0000 7f41dbb73640 1 -- start start 2024-03-23T09:43:37.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.051+0000 7f41dbb73640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41d40b8300 0x7f41d414c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.051+0000 7f41dbb73640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d40d23a0 0x7f41d40cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.051+0000 7f41dbb73640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41d415cee0 0x7f41d4152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.051+0000 7f41dbb73640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d40c87d0 con 0x7f41d40b8300 2024-03-23T09:43:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.051+0000 7f41dbb73640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d40c8ab0 con 0x7f41d40d23a0 2024-03-23T09:43:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.051+0000 7f41dbb73640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d40c8d90 con 0x7f41d415cee0 2024-03-23T09:43:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41d990f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41d40b8300 0x7f41d414c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41d990f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41d40b8300 0x7f41d414c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45848/0 (socket says 172.21.15.47:45848) 2024-03-23T09:43:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41d990f640 1 -- 172.21.15.47:0/450826745 learned_addr learned my addr 172.21.15.47:0/450826745 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41d910e640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d40d23a0 0x7f41d40cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41da110640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41d415cee0 0x7f41d4152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41d990f640 1 -- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41d415cee0 msgr2=0x7f41d4152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41d990f640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41d415cee0 0x7f41d4152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41d990f640 1 -- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d40d23a0 msgr2=0x7f41d40cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41d990f640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d40d23a0 0x7f41d40cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41d990f640 1 -- 172.21.15.47:0/450826745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc009b30 con 0x7f41d40b8300 2024-03-23T09:43:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41d990f640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41d40b8300 0x7f41d414c1b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f41d000c400 tx=0x7f41d000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41ca7fc640 1 -- 172.21.15.47:0/450826745 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d00052d0 con 0x7f41d40b8300 2024-03-23T09:43:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41ca7fc640 1 -- 172.21.15.47:0/450826745 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d0005430 con 0x7f41d40b8300 2024-03-23T09:43:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41dbb73640 1 -- 172.21.15.47:0/450826745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d40c9070 con 0x7f41d40b8300 2024-03-23T09:43:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41ca7fc640 1 -- 172.21.15.47:0/450826745 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0003830 con 0x7f41d40b8300 2024-03-23T09:43:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.052+0000 7f41dbb73640 1 -- 172.21.15.47:0/450826745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d415d2c0 con 0x7f41d40b8300 2024-03-23T09:43:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.053+0000 7f41dbb73640 1 -- 172.21.15.47:0/450826745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d4072360 con 0x7f41d40b8300 2024-03-23T09:43:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.054+0000 7f41ca7fc640 1 -- 172.21.15.47:0/450826745 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f41d0003990 con 0x7f41d40b8300 2024-03-23T09:43:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.054+0000 7f41ca7fc640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f41b806ac00 0x7f41b806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.054+0000 7f41ca7fc640 1 -- 172.21.15.47:0/450826745 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f41d00888a0 con 0x7f41d40b8300 2024-03-23T09:43:37.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.054+0000 7f41d910e640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f41b806ac00 0x7f41b806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:37.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.055+0000 7f41d910e640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f41b806ac00 0x7f41b806d0c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f41d41510f0 tx=0x7f41c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:37.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.058+0000 7f41ca7fc640 1 -- 172.21.15.47:0/450826745 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f41d0053ab0 con 0x7f41d40b8300 2024-03-23T09:43:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.279+0000 7f41dbb73640 1 -- 172.21.15.47:0/450826745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f41d40c9260 con 0x7f41d40b8300 2024-03-23T09:43:37.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:43:37.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:43:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.933+0000 7f41ca7fc640 1 -- 172.21.15.47:0/450826745 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f41d00570d0 con 0x7f41d40b8300 2024-03-23T09:43:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.935+0000 7f41dbb73640 1 -- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f41b806ac00 msgr2=0x7f41b806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.935+0000 7f41dbb73640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f41b806ac00 0x7f41b806d0c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f41d41510f0 tx=0x7f41c4008040 comp rx=0 tx=0).stop 2024-03-23T09:43:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.935+0000 7f41dbb73640 1 -- 172.21.15.47:0/450826745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41d40b8300 msgr2=0x7f41d414c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.935+0000 7f41dbb73640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41d40b8300 0x7f41d414c1b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f41d000c400 tx=0x7f41d000c8c0 comp rx=0 tx=0).stop 2024-03-23T09:43:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.935+0000 7f41dbb73640 1 -- 172.21.15.47:0/450826745 shutdown_connections 2024-03-23T09:43:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.935+0000 7f41dbb73640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f41b806ac00 0x7f41b806d0c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.935+0000 7f41dbb73640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41d415cee0 0x7f41d4152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.935+0000 7f41dbb73640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d40d23a0 0x7f41d40cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.935+0000 7f41dbb73640 1 --2- 172.21.15.47:0/450826745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41d40b8300 0x7f41d414c1b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.935+0000 7f41dbb73640 1 -- 172.21.15.47:0/450826745 >> 172.21.15.47:0/450826745 conn(0x7f41d4065df0 msgr2=0x7f41d40d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.936+0000 7f41dbb73640 1 -- 172.21.15.47:0/450826745 shutdown_connections 2024-03-23T09:43:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:37.936+0000 7f41dbb73640 1 -- 172.21.15.47:0/450826745 wait complete. 2024-03-23T09:43:37.948 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T09:43:38.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.058+0000 7f9e3ae8f640 1 Processor -- start 2024-03-23T09:43:38.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.058+0000 7f9e3ae8f640 1 -- start start 2024-03-23T09:43:38.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.059+0000 7f9e3ae8f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9e340b0a80 0x7f9e340cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:38.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.059+0000 7f9e3ae8f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9e340ca340 0x7f9e340ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:38.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.059+0000 7f9e3ae8f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e340cdeb0 con 0x7f9e3415ad10 2024-03-23T09:43:38.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.059+0000 7f9e3ae8f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e340b0f80 con 0x7f9e340ca340 2024-03-23T09:43:38.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.059+0000 7f9e3ae8f640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e340aebd0 con 0x7f9e340b0a80 2024-03-23T09:43:38.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.059+0000 7f9e3942c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9e340ca340 0x7f9e340ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:38.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.059+0000 7f9e38c2b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9e3415ad10 0x7f9e340cbd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47088/0 (socket says 172.21.15.47:47088) 2024-03-23T09:43:38.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.059+0000 7f9e38c2b640 1 -- 172.21.15.47:0/597770415 learned_addr learned my addr 172.21.15.47:0/597770415 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:38.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.060+0000 7f9e33fff640 1 --2- 172.21.15.47:0/597770415 >> v2:172.21.15.175:3300/0 conn(0x7f9e340b0a80 0x7f9e340cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:38.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.060+0000 7f9e3942c640 1 -- 172.21.15.47:0/597770415 >> v2:172.21.15.175:3300/0 conn(0x7f9e340b0a80 msgr2=0x7f9e340cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:38.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.060+0000 7f9e3942c640 1 --2- 172.21.15.47:0/597770415 >> v2:172.21.15.175:3300/0 conn(0x7f9e340b0a80 0x7f9e340cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:38.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.060+0000 7f9e3942c640 1 -- 172.21.15.47:0/597770415 >> v1:172.21.15.47:6789/0 conn(0x7f9e3415ad10 legacy=0x7f9e340cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:38.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.060+0000 7f9e337fe640 1 -- 172.21.15.47:0/597770415 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069380501 0 0) 0x7f9e340cdeb0 con 0x7f9e3415ad10 2024-03-23T09:43:38.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.060+0000 7f9e3942c640 1 -- 172.21.15.47:0/597770415 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e340b1bb0 con 0x7f9e340ca340 2024-03-23T09:43:38.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.060+0000 7f9e3942c640 1 --2- 172.21.15.47:0/597770415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e340ca340 0x7f9e340ca770 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9e34070e80 tx=0x7f9e2402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da14f015c10640b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:38.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.061+0000 7f9e337fe640 1 -- 172.21.15.47:0/597770415 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24030610 con 0x7f9e340ca340 2024-03-23T09:43:38.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.061+0000 7f9e337fe640 1 -- 172.21.15.47:0/597770415 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e24030770 con 0x7f9e340ca340 2024-03-23T09:43:38.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.061+0000 7f9e337fe640 1 -- 172.21.15.47:0/597770415 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24030b00 con 0x7f9e340ca340 2024-03-23T09:43:38.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.061+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/597770415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e340ca340 msgr2=0x7f9e340ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:38.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.061+0000 7f9e3ae8f640 1 --2- 172.21.15.47:0/597770415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e340ca340 0x7f9e340ca770 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9e34070e80 tx=0x7f9e2402b960 comp rx=0 tx=0).stop 2024-03-23T09:43:38.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.061+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/597770415 shutdown_connections 2024-03-23T09:43:38.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.061+0000 7f9e3ae8f640 1 --2- 172.21.15.47:0/597770415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e340ca340 0x7f9e340ca770 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:38.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.061+0000 7f9e3ae8f640 1 --2- 172.21.15.47:0/597770415 >> v2:172.21.15.175:3300/0 conn(0x7f9e340b0a80 0x7f9e340cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:38.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.061+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/597770415 >> 172.21.15.47:0/597770415 conn(0x7f9e34098a00 msgr2=0x7f9e340b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:38.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.061+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/597770415 shutdown_connections 2024-03-23T09:43:38.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.061+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/597770415 wait complete. 2024-03-23T09:43:38.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.062+0000 7f9e3ae8f640 1 Processor -- start 2024-03-23T09:43:38.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.062+0000 7f9e3ae8f640 1 -- start start 2024-03-23T09:43:38.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.063+0000 7f9e3ae8f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e340b0a80 0x7f9e34144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:38.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.063+0000 7f9e3ae8f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e340ca340 0x7f9e340994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:38.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.063+0000 7f9e3ae8f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e3415ad10 0x7f9e341efda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:38.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.063+0000 7f9e3ae8f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e34142b90 con 0x7f9e340ca340 2024-03-23T09:43:38.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.063+0000 7f9e3ae8f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e34155e70 con 0x7f9e3415ad10 2024-03-23T09:43:38.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.063+0000 7f9e3ae8f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e34145140 con 0x7f9e340b0a80 2024-03-23T09:43:38.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.063+0000 7f9e33fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e340ca340 0x7f9e340994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:38.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.063+0000 7f9e33fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e340ca340 0x7f9e340994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45874/0 (socket says 172.21.15.47:45874) 2024-03-23T09:43:38.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.063+0000 7f9e33fff640 1 -- 172.21.15.47:0/2580984442 learned_addr learned my addr 172.21.15.47:0/2580984442 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:38.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.063+0000 7f9e3942c640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e3415ad10 0x7f9e341efda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:38.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.063+0000 7f9e38c2b640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e340b0a80 0x7f9e34144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:38.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.064+0000 7f9e33fff640 1 -- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e340b0a80 msgr2=0x7f9e34144b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:38.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.064+0000 7f9e33fff640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e340b0a80 0x7f9e34144b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:38.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.064+0000 7f9e33fff640 1 -- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e3415ad10 msgr2=0x7f9e341efda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:38.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.064+0000 7f9e33fff640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e3415ad10 0x7f9e341efda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:38.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.064+0000 7f9e33fff640 1 -- 172.21.15.47:0/2580984442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e24009400 con 0x7f9e340ca340 2024-03-23T09:43:38.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.064+0000 7f9e3942c640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e3415ad10 0x7f9e341efda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:38.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.064+0000 7f9e33fff640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e340ca340 0x7f9e340994e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800d7d0 tx=0x7f9e2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:38.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.064+0000 7f9e317fa640 1 -- 172.21.15.47:0/2580984442 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2800b6c0 con 0x7f9e340ca340 2024-03-23T09:43:38.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.064+0000 7f9e317fa640 1 -- 172.21.15.47:0/2580984442 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2800b820 con 0x7f9e340ca340 2024-03-23T09:43:38.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.064+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/2580984442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e34145420 con 0x7f9e340ca340 2024-03-23T09:43:38.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.064+0000 7f9e317fa640 1 -- 172.21.15.47:0/2580984442 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2800bbb0 con 0x7f9e340ca340 2024-03-23T09:43:38.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.064+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/2580984442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e341f02e0 con 0x7f9e340ca340 2024-03-23T09:43:38.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.065+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/2580984442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e34142d00 con 0x7f9e340ca340 2024-03-23T09:43:38.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.066+0000 7f9e317fa640 1 -- 172.21.15.47:0/2580984442 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9e2800bd10 con 0x7f9e340ca340 2024-03-23T09:43:38.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.066+0000 7f9e317fa640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e2006ac00 0x7f9e2006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:38.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.066+0000 7f9e317fa640 1 -- 172.21.15.47:0/2580984442 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9e28089870 con 0x7f9e340ca340 2024-03-23T09:43:38.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.066+0000 7f9e38c2b640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e2006ac00 0x7f9e2006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:38.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.067+0000 7f9e38c2b640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e2006ac00 0x7f9e2006d0c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0012e0 tx=0x7f9e2c005eb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:38.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.070+0000 7f9e317fa640 1 -- 172.21.15.47:0/2580984442 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9e28054a80 con 0x7f9e340ca340 2024-03-23T09:43:38.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.250+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/2580984442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9e340672f0 con 0x7f9e340ca340 2024-03-23T09:43:38.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.938+0000 7f9e317fa640 1 -- 172.21.15.47:0/2580984442 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9e280580a0 con 0x7f9e340ca340 2024-03-23T09:43:38.938 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-03-23T09:43:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.941+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e2006ac00 msgr2=0x7f9e2006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.941+0000 7f9e3ae8f640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e2006ac00 0x7f9e2006d0c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0012e0 tx=0x7f9e2c005eb0 comp rx=0 tx=0).stop 2024-03-23T09:43:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.942+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/2580984442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e340ca340 msgr2=0x7f9e340994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.942+0000 7f9e3ae8f640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e340ca340 0x7f9e340994e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800d7d0 tx=0x7f9e2800dc90 comp rx=0 tx=0).stop 2024-03-23T09:43:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.942+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/2580984442 shutdown_connections 2024-03-23T09:43:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.942+0000 7f9e3ae8f640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e2006ac00 0x7f9e2006d0c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.942+0000 7f9e3ae8f640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e3415ad10 0x7f9e341efda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.942+0000 7f9e3ae8f640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e340ca340 0x7f9e340994e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.942+0000 7f9e3ae8f640 1 --2- 172.21.15.47:0/2580984442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e340b0a80 0x7f9e34144b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:38.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.942+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/2580984442 >> 172.21.15.47:0/2580984442 conn(0x7f9e34098a00 msgr2=0x7f9e340b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:38.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.942+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/2580984442 shutdown_connections 2024-03-23T09:43:38.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:38.942+0000 7f9e3ae8f640 1 -- 172.21.15.47:0/2580984442 wait complete. 2024-03-23T09:43:38.955 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T09:43:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.066+0000 7fbd511fd640 1 Processor -- start 2024-03-23T09:43:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.066+0000 7fbd511fd640 1 -- start start 2024-03-23T09:43:39.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.067+0000 7fbd511fd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd4c15d0f0 0x7fbd4c15d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:39.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.067+0000 7fbd511fd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4c160bd0 con 0x7fbd4c15a860 2024-03-23T09:43:39.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.067+0000 7fbd511fd640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4c15dbb0 con 0x7fbd4c0cf190 2024-03-23T09:43:39.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.067+0000 7fbd511fd640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd4c0d2790 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.067+0000 7fbd4ad9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbd4c15a860 0x7fbd4c161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47092/0 (socket says 172.21.15.47:47092) 2024-03-23T09:43:39.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.067+0000 7fbd43fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd4c15d0f0 0x7fbd4c15d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:39.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.067+0000 7fbd4ad9d640 1 -- 172.21.15.47:0/2680762867 learned_addr learned my addr 172.21.15.47:0/2680762867 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:39.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.067+0000 7fbd43fff640 1 -- 172.21.15.47:0/2680762867 >> v1:172.21.15.175:6789/0 conn(0x7fbd4c0cf190 legacy=0x7fbd4c0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:39.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.067+0000 7fbd43fff640 1 -- 172.21.15.47:0/2680762867 >> v1:172.21.15.47:6789/0 conn(0x7fbd4c15a860 legacy=0x7fbd4c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:39.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.067+0000 7fbd43fff640 1 -- 172.21.15.47:0/2680762867 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c15de90 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.067+0000 7fbd4a59c640 1 -- 172.21.15.47:0/2680762867 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220504656 0 0) 0x7fbd4c160bd0 con 0x7fbd4c15a860 2024-03-23T09:43:39.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.068+0000 7fbd43fff640 1 --2- 172.21.15.47:0/2680762867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd4c15d0f0 0x7fbd4c15d580 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fbd340097a0 tx=0x7fbd3402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65e1a973ed1ac7d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:39.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.068+0000 7fbd4a59c640 1 -- 172.21.15.47:0/2680762867 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd34030610 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.068+0000 7fbd4a59c640 1 -- 172.21.15.47:0/2680762867 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd34030770 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.068+0000 7fbd4a59c640 1 -- 172.21.15.47:0/2680762867 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd34030b00 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.069+0000 7fbd511fd640 1 -- 172.21.15.47:0/2680762867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd4c15d0f0 msgr2=0x7fbd4c15d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:39.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.069+0000 7fbd511fd640 1 --2- 172.21.15.47:0/2680762867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd4c15d0f0 0x7fbd4c15d580 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fbd340097a0 tx=0x7fbd3402b960 comp rx=0 tx=0).stop 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.069+0000 7fbd511fd640 1 -- 172.21.15.47:0/2680762867 shutdown_connections 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.069+0000 7fbd511fd640 1 --2- 172.21.15.47:0/2680762867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd4c15d0f0 0x7fbd4c15d580 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.069+0000 7fbd511fd640 1 -- 172.21.15.47:0/2680762867 >> 172.21.15.47:0/2680762867 conn(0x7fbd4c0659d0 msgr2=0x7fbd4c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.069+0000 7fbd511fd640 1 -- 172.21.15.47:0/2680762867 shutdown_connections 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.069+0000 7fbd511fd640 1 -- 172.21.15.47:0/2680762867 wait complete. 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.070+0000 7fbd511fd640 1 Processor -- start 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.070+0000 7fbd511fd640 1 -- start start 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd511fd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd4c0cf190 0x7fbd4c14eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd511fd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd4c15a860 0x7fbd4c0c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd511fd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd4c15d0f0 0x7fbd4c0b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd511fd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c0c8850 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd511fd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c14f030 con 0x7fbd4c0cf190 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd511fd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c0c7c40 con 0x7fbd4c15a860 2024-03-23T09:43:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd4b59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd4c15d0f0 0x7fbd4c0b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd4b59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd4c15d0f0 0x7fbd4c0b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45892/0 (socket says 172.21.15.47:45892) 2024-03-23T09:43:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd4b59e640 1 -- 172.21.15.47:0/2822618320 learned_addr learned my addr 172.21.15.47:0/2822618320 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd43fff640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd4c15a860 0x7fbd4c0c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd4ad9d640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd4c0cf190 0x7fbd4c14eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd4b59e640 1 -- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd4c15a860 msgr2=0x7fbd4c0c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd4b59e640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd4c15a860 0x7fbd4c0c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd4b59e640 1 -- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd4c0cf190 msgr2=0x7fbd4c14eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd4b59e640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd4c0cf190 0x7fbd4c14eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd4b59e640 1 -- 172.21.15.47:0/2822618320 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd34009400 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd4ad9d640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd4c0cf190 0x7fbd4c14eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd4b59e640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd4c15d0f0 0x7fbd4c0b36e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400c7a0 tx=0x7fbd4400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd437fe640 1 -- 172.21.15.47:0/2822618320 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44003b70 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd437fe640 1 -- 172.21.15.47:0/2822618320 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd44003cf0 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd511fd640 1 -- 172.21.15.47:0/2822618320 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4c0c7f20 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.071+0000 7fbd437fe640 1 -- 172.21.15.47:0/2822618320 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44003ea0 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.072+0000 7fbd511fd640 1 -- 172.21.15.47:0/2822618320 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c0c84e0 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.072+0000 7fbd511fd640 1 -- 172.21.15.47:0/2822618320 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4c072360 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.073+0000 7fbd437fe640 1 -- 172.21.15.47:0/2822618320 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbd4400aa90 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.073+0000 7fbd437fe640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd2c06ac00 0x7fbd2c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:39.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.073+0000 7fbd437fe640 1 -- 172.21.15.47:0/2822618320 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fbd44089060 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.074+0000 7fbd4ad9d640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd2c06ac00 0x7fbd2c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:39.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.074+0000 7fbd4ad9d640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd2c06ac00 0x7fbd2c06d0c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c000f60 tx=0x7fbd3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:39.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.078+0000 7fbd437fe640 1 -- 172.21.15.47:0/2822618320 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbd440543a0 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.257+0000 7fbd511fd640 1 -- 172.21.15.47:0/2822618320 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbd4c0cfb10 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.941+0000 7fbd437fe640 1 -- 172.21.15.47:0/2822618320 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbd440579c0 con 0x7fbd4c15d0f0 2024-03-23T09:43:39.942 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-03-23T09:43:39.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.947+0000 7fbd511fd640 1 -- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd2c06ac00 msgr2=0x7fbd2c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:39.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.947+0000 7fbd511fd640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd2c06ac00 0x7fbd2c06d0c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c000f60 tx=0x7fbd3c009040 comp rx=0 tx=0).stop 2024-03-23T09:43:39.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.947+0000 7fbd511fd640 1 -- 172.21.15.47:0/2822618320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd4c15d0f0 msgr2=0x7fbd4c0b36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:39.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.947+0000 7fbd511fd640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd4c15d0f0 0x7fbd4c0b36e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400c7a0 tx=0x7fbd4400cc60 comp rx=0 tx=0).stop 2024-03-23T09:43:39.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.947+0000 7fbd511fd640 1 -- 172.21.15.47:0/2822618320 shutdown_connections 2024-03-23T09:43:39.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.947+0000 7fbd511fd640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd2c06ac00 0x7fbd2c06d0c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:39.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.947+0000 7fbd511fd640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd4c15d0f0 0x7fbd4c0b36e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:39.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.947+0000 7fbd511fd640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd4c15a860 0x7fbd4c0c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:39.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.947+0000 7fbd511fd640 1 --2- 172.21.15.47:0/2822618320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd4c0cf190 0x7fbd4c14eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:39.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.947+0000 7fbd511fd640 1 -- 172.21.15.47:0/2822618320 >> 172.21.15.47:0/2822618320 conn(0x7fbd4c0659d0 msgr2=0x7fbd4c14f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:39.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.947+0000 7fbd511fd640 1 -- 172.21.15.47:0/2822618320 shutdown_connections 2024-03-23T09:43:39.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:39.948+0000 7fbd511fd640 1 -- 172.21.15.47:0/2822618320 wait complete. 2024-03-23T09:43:39.965 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-23T09:43:40.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.073+0000 7fd7f7a4a640 1 Processor -- start 2024-03-23T09:43:40.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.073+0000 7fd7f7a4a640 1 -- start start 2024-03-23T09:43:40.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f7a4a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd7f015c990 0x7fd7f0160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:40.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f7a4a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd7f00b8350 0x7fd7f00cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:40.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f7a4a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f0161630 con 0x7fd7f00cdc40 2024-03-23T09:43:40.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f7a4a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f00d1030 con 0x7fd7f015c990 2024-03-23T09:43:40.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f7a4a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f00ce070 con 0x7fd7f00b8350 2024-03-23T09:43:40.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f57e6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd7f015c990 0x7fd7f0160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:40.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f57e6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd7f015c990 0x7fd7f0160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45908/0 (socket says 172.21.15.47:45908) 2024-03-23T09:43:40.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f57e6640 1 -- 172.21.15.47:0/4095702915 learned_addr learned my addr 172.21.15.47:0/4095702915 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:40.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f4fe5640 1 --2- 172.21.15.47:0/4095702915 >> v2:172.21.15.175:3300/0 conn(0x7fd7f00b8350 0x7fd7f00cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:40.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f57e6640 1 -- 172.21.15.47:0/4095702915 >> v2:172.21.15.175:3300/0 conn(0x7fd7f00b8350 msgr2=0x7fd7f00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:40.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f57e6640 1 --2- 172.21.15.47:0/4095702915 >> v2:172.21.15.175:3300/0 conn(0x7fd7f00b8350 0x7fd7f00cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f57e6640 1 -- 172.21.15.47:0/4095702915 >> v1:172.21.15.47:6789/0 conn(0x7fd7f00cdc40 legacy=0x7fd7f00b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:40.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7effff640 1 -- 172.21.15.47:0/4095702915 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389676909 0 0) 0x7fd7f0161630 con 0x7fd7f00cdc40 2024-03-23T09:43:40.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f57e6640 1 -- 172.21.15.47:0/4095702915 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00cfe10 con 0x7fd7f015c990 2024-03-23T09:43:40.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.074+0000 7fd7f4fe5640 1 --2- 172.21.15.47:0/4095702915 >> v2:172.21.15.175:3300/0 conn(0x7fd7f00b8350 0x7fd7f00cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:40.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.075+0000 7fd7f57e6640 1 --2- 172.21.15.47:0/4095702915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7f015c990 0x7fd7f0160690 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd7f008ac70 tx=0x7fd7e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cff2689527bdc6b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:40.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.075+0000 7fd7effff640 1 -- 172.21.15.47:0/4095702915 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8031870 con 0x7fd7f015c990 2024-03-23T09:43:40.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.075+0000 7fd7effff640 1 -- 172.21.15.47:0/4095702915 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e80319d0 con 0x7fd7f015c990 2024-03-23T09:43:40.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.075+0000 7fd7effff640 1 -- 172.21.15.47:0/4095702915 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8031d60 con 0x7fd7f015c990 2024-03-23T09:43:40.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.076+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/4095702915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7f015c990 msgr2=0x7fd7f0160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:40.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.076+0000 7fd7f7a4a640 1 --2- 172.21.15.47:0/4095702915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7f015c990 0x7fd7f0160690 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd7f008ac70 tx=0x7fd7e802b910 comp rx=0 tx=0).stop 2024-03-23T09:43:40.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.076+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/4095702915 shutdown_connections 2024-03-23T09:43:40.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.076+0000 7fd7f7a4a640 1 --2- 172.21.15.47:0/4095702915 >> v2:172.21.15.175:3300/0 conn(0x7fd7f00b8350 0x7fd7f00cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.076+0000 7fd7f7a4a640 1 --2- 172.21.15.47:0/4095702915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7f015c990 0x7fd7f0160690 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.076+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/4095702915 >> 172.21.15.47:0/4095702915 conn(0x7fd7f00659d0 msgr2=0x7fd7f0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:40.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.076+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/4095702915 shutdown_connections 2024-03-23T09:43:40.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.076+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/4095702915 wait complete. 2024-03-23T09:43:40.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.077+0000 7fd7f7a4a640 1 Processor -- start 2024-03-23T09:43:40.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.077+0000 7fd7f7a4a640 1 -- start start 2024-03-23T09:43:40.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.078+0000 7fd7f7a4a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7f00b8350 0x7fd7f00b7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.078+0000 7fd7f7a4a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7f00cdc40 0x7fd7f0146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.078+0000 7fd7f7a4a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7f015c990 0x7fd7f0148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.078+0000 7fd7f7a4a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f01480d0 con 0x7fd7f00b8350 2024-03-23T09:43:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.078+0000 7fd7f7a4a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f01474c0 con 0x7fd7f015c990 2024-03-23T09:43:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.078+0000 7fd7f7a4a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f01477a0 con 0x7fd7f00cdc40 2024-03-23T09:43:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.078+0000 7fd7f57e6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7f00b8350 0x7fd7f00b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.078+0000 7fd7f57e6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7f00b8350 0x7fd7f00b7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45922/0 (socket says 172.21.15.47:45922) 2024-03-23T09:43:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.078+0000 7fd7f57e6640 1 -- 172.21.15.47:0/2197693190 learned_addr learned my addr 172.21.15.47:0/2197693190 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.078+0000 7fd7f57e6640 1 -- 172.21.15.47:0/2197693190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7f00cdc40 msgr2=0x7fd7f0146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.078+0000 7fd7f57e6640 1 --2- 172.21.15.47:0/2197693190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7f00cdc40 0x7fd7f0146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.078+0000 7fd7f57e6640 1 -- 172.21.15.47:0/2197693190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7f015c990 msgr2=0x7fd7f0148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.078+0000 7fd7f57e6640 1 --2- 172.21.15.47:0/2197693190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7f015c990 0x7fd7f0148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.079+0000 7fd7f57e6640 1 -- 172.21.15.47:0/2197693190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e80094d0 con 0x7fd7f00b8350 2024-03-23T09:43:40.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.079+0000 7fd7f57e6640 1 --2- 172.21.15.47:0/2197693190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7f00b8350 0x7fd7f00b7130 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd7e802b950 tx=0x7fd7e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:40.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.079+0000 7fd7edffb640 1 -- 172.21.15.47:0/2197693190 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8031860 con 0x7fd7f00b8350 2024-03-23T09:43:40.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.079+0000 7fd7edffb640 1 -- 172.21.15.47:0/2197693190 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e80443f0 con 0x7fd7f00b8350 2024-03-23T09:43:40.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.079+0000 7fd7edffb640 1 -- 172.21.15.47:0/2197693190 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80447a0 con 0x7fd7f00b8350 2024-03-23T09:43:40.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.079+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/2197693190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f0147a80 con 0x7fd7f00b8350 2024-03-23T09:43:40.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.079+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/2197693190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f0149780 con 0x7fd7f00b8350 2024-03-23T09:43:40.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.080+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/2197693190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f0072360 con 0x7fd7f00b8350 2024-03-23T09:43:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.080+0000 7fd7edffb640 1 -- 172.21.15.47:0/2197693190 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd7e804d450 con 0x7fd7f00b8350 2024-03-23T09:43:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.081+0000 7fd7edffb640 1 --2- 172.21.15.47:0/2197693190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd7dc06ac00 0x7fd7dc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.081+0000 7fd7edffb640 1 -- 172.21.15.47:0/2197693190 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd7e802d040 con 0x7fd7f00b8350 2024-03-23T09:43:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.081+0000 7fd7f4fe5640 1 --2- 172.21.15.47:0/2197693190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd7dc06ac00 0x7fd7dc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.082+0000 7fd7f4fe5640 1 --2- 172.21.15.47:0/2197693190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd7dc06ac00 0x7fd7dc06d0c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4006fd0 tx=0x7fd7e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.085+0000 7fd7edffb640 1 -- 172.21.15.47:0/2197693190 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd7e8076cf0 con 0x7fd7f00b8350 2024-03-23T09:43:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.269+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/2197693190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd7f0066be0 con 0x7fd7f00b8350 2024-03-23T09:43:40.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.272+0000 7fd7edffb640 1 -- 172.21.15.47:0/2197693190 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd7e807a310 con 0x7fd7f00b8350 2024-03-23T09:43:40.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.274+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/2197693190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd7dc06ac00 msgr2=0x7fd7dc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.274+0000 7fd7f7a4a640 1 --2- 172.21.15.47:0/2197693190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd7dc06ac00 0x7fd7dc06d0c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4006fd0 tx=0x7fd7e4008040 comp rx=0 tx=0).stop 2024-03-23T09:43:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.274+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/2197693190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7f00b8350 msgr2=0x7fd7f00b7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.274+0000 7fd7f7a4a640 1 --2- 172.21.15.47:0/2197693190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7f00b8350 0x7fd7f00b7130 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd7e802b950 tx=0x7fd7e80046c0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.274+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/2197693190 shutdown_connections 2024-03-23T09:43:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.274+0000 7fd7f7a4a640 1 --2- 172.21.15.47:0/2197693190 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd7dc06ac00 0x7fd7dc06d0c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.275+0000 7fd7f7a4a640 1 --2- 172.21.15.47:0/2197693190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7f015c990 0x7fd7f0148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.275+0000 7fd7f7a4a640 1 --2- 172.21.15.47:0/2197693190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7f00cdc40 0x7fd7f0146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.275+0000 7fd7f7a4a640 1 --2- 172.21.15.47:0/2197693190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7f00b8350 0x7fd7f00b7130 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.275+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/2197693190 >> 172.21.15.47:0/2197693190 conn(0x7fd7f00659d0 msgr2=0x7fd7f00b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.275+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/2197693190 shutdown_connections 2024-03-23T09:43:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.275+0000 7fd7f7a4a640 1 -- 172.21.15.47:0/2197693190 wait complete. 2024-03-23T09:43:40.285 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-03-23T09:43:40.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.386+0000 7f3677296640 1 Processor -- start 2024-03-23T09:43:40.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.386+0000 7f3677296640 1 -- start start 2024-03-23T09:43:40.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3677296640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f367015cb50 0x7f36700b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:40.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3677296640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36700cd5a0 0x7f36700cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:40.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3677296640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f36700d1010 0x7f36700d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:40.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3677296640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36700b7e70 con 0x7f36700d1010 2024-03-23T09:43:40.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3677296640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f367015cf30 con 0x7f367015cb50 2024-03-23T09:43:40.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3677296640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36700cdf20 con 0x7f36700cd5a0 2024-03-23T09:43:40.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3675833640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f36700d1010 0x7f36700d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3675833640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f36700d1010 0x7f36700d14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45934/0 (socket says 172.21.15.47:45934) 2024-03-23T09:43:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3675833640 1 -- 172.21.15.47:0/4252826428 learned_addr learned my addr 172.21.15.47:0/4252826428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3675833640 1 -- 172.21.15.47:0/4252826428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36700cd5a0 msgr2=0x7f36700cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3675833640 1 --2- 172.21.15.47:0/4252826428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36700cd5a0 0x7f36700cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3675833640 1 -- 172.21.15.47:0/4252826428 >> v2:172.21.15.175:3300/0 conn(0x7f367015cb50 msgr2=0x7f36700b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3675032640 1 --2- 172.21.15.47:0/4252826428 >> v2:172.21.15.175:3300/0 conn(0x7f367015cb50 0x7f36700b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3675833640 1 --2- 172.21.15.47:0/4252826428 >> v2:172.21.15.175:3300/0 conn(0x7f367015cb50 0x7f36700b84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.387+0000 7f3675833640 1 -- 172.21.15.47:0/4252826428 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36700b72f0 con 0x7f36700d1010 2024-03-23T09:43:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.388+0000 7f3675833640 1 --2- 172.21.15.47:0/4252826428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36700d1010 0x7f36700d14a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f36700d39a0 tx=0x7f366002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6530e87d4988ac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.388+0000 7f366ffff640 1 -- 172.21.15.47:0/4252826428 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366002cb00 con 0x7f36700d1010 2024-03-23T09:43:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.388+0000 7f366ffff640 1 -- 172.21.15.47:0/4252826428 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366002cc60 con 0x7f36700d1010 2024-03-23T09:43:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.388+0000 7f366ffff640 1 -- 172.21.15.47:0/4252826428 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36600343f0 con 0x7f36700d1010 2024-03-23T09:43:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.389+0000 7f3677296640 1 -- 172.21.15.47:0/4252826428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36700d1010 msgr2=0x7f36700d14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.389+0000 7f3677296640 1 --2- 172.21.15.47:0/4252826428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36700d1010 0x7f36700d14a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f36700d39a0 tx=0x7f366002bf60 comp rx=0 tx=0).stop 2024-03-23T09:43:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.389+0000 7f3677296640 1 -- 172.21.15.47:0/4252826428 shutdown_connections 2024-03-23T09:43:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.389+0000 7f3677296640 1 --2- 172.21.15.47:0/4252826428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36700d1010 0x7f36700d14a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.389+0000 7f3677296640 1 --2- 172.21.15.47:0/4252826428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36700cd5a0 0x7f36700cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.389+0000 7f3677296640 1 --2- 172.21.15.47:0/4252826428 >> v2:172.21.15.175:3300/0 conn(0x7f367015cb50 0x7f36700b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.389+0000 7f3677296640 1 -- 172.21.15.47:0/4252826428 >> 172.21.15.47:0/4252826428 conn(0x7f3670065df0 msgr2=0x7f3670066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:40.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.389+0000 7f3677296640 1 -- 172.21.15.47:0/4252826428 shutdown_connections 2024-03-23T09:43:40.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.389+0000 7f3677296640 1 -- 172.21.15.47:0/4252826428 wait complete. 2024-03-23T09:43:40.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.390+0000 7f3677296640 1 Processor -- start 2024-03-23T09:43:40.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.390+0000 7f3677296640 1 -- start start 2024-03-23T09:43:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3677296640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36700cd5a0 0x7f36701f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3677296640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36700d1010 0x7f36700aa290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3677296640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f367015cb50 0x7f36700cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3677296640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36700a92c0 con 0x7f367015cb50 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3677296640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36700a9e00 con 0x7f36700d1010 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3677296640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36700732c0 con 0x7f36700cd5a0 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3675833640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f367015cb50 0x7f36700cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3675833640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f367015cb50 0x7f36700cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45946/0 (socket says 172.21.15.47:45946) 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3675833640 1 -- 172.21.15.47:0/2794366749 learned_addr learned my addr 172.21.15.47:0/2794366749 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3675032640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36700cd5a0 0x7f36701f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3674831640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36700d1010 0x7f36700aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3675833640 1 -- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36700cd5a0 msgr2=0x7f36701f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3675833640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36700cd5a0 0x7f36701f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3675833640 1 -- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36700d1010 msgr2=0x7f36700aa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3675833640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36700d1010 0x7f36700aa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3675833640 1 -- 172.21.15.47:0/2794366749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3660009b30 con 0x7f367015cb50 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3674831640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36700d1010 0x7f36700aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f3675833640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f367015cb50 0x7f36700cbd40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3660036040 tx=0x7f3660004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.391+0000 7f366dffb640 1 -- 172.21.15.47:0/2794366749 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660009510 con 0x7f367015cb50 2024-03-23T09:43:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.392+0000 7f366dffb640 1 -- 172.21.15.47:0/2794366749 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366003e450 con 0x7f367015cb50 2024-03-23T09:43:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.392+0000 7f3677296640 1 -- 172.21.15.47:0/2794366749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36700a94d0 con 0x7f367015cb50 2024-03-23T09:43:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.392+0000 7f366dffb640 1 -- 172.21.15.47:0/2794366749 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366003e800 con 0x7f367015cb50 2024-03-23T09:43:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.392+0000 7f3677296640 1 -- 172.21.15.47:0/2794366749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36700a9a90 con 0x7f367015cb50 2024-03-23T09:43:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.393+0000 7f3677296640 1 -- 172.21.15.47:0/2794366749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3670072360 con 0x7f367015cb50 2024-03-23T09:43:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.393+0000 7f366dffb640 1 -- 172.21.15.47:0/2794366749 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f366003e960 con 0x7f367015cb50 2024-03-23T09:43:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.394+0000 7f366dffb640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f365c06ac00 0x7f365c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.394+0000 7f366dffb640 1 -- 172.21.15.47:0/2794366749 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f36600abe30 con 0x7f367015cb50 2024-03-23T09:43:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.394+0000 7f3675032640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f365c06ac00 0x7f365c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.395+0000 7f3675032640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f365c06ac00 0x7f365c06d0c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3668006fd0 tx=0x7f3668008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.398+0000 7f366dffb640 1 -- 172.21.15.47:0/2794366749 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3660077450 con 0x7f367015cb50 2024-03-23T09:43:40.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.643+0000 7f3677296640 1 -- 172.21.15.47:0/2794366749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f36700a9fb0 con 0x7f367015cb50 2024-03-23T09:43:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.951+0000 7f366dffb640 1 -- 172.21.15.47:0/2794366749 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v141) v1 ==== 149+0+0 (secure 0 0 0) 0x7f36700a9fb0 con 0x7f367015cb50 2024-03-23T09:43:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.954+0000 7f3677296640 1 -- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f365c06ac00 msgr2=0x7f365c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.954+0000 7f3677296640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f365c06ac00 0x7f365c06d0c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3668006fd0 tx=0x7f3668008040 comp rx=0 tx=0).stop 2024-03-23T09:43:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.954+0000 7f3677296640 1 -- 172.21.15.47:0/2794366749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f367015cb50 msgr2=0x7f36700cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.954+0000 7f3677296640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f367015cb50 0x7f36700cbd40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3660036040 tx=0x7f3660004a40 comp rx=0 tx=0).stop 2024-03-23T09:43:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.954+0000 7f3677296640 1 -- 172.21.15.47:0/2794366749 shutdown_connections 2024-03-23T09:43:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.954+0000 7f3677296640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f365c06ac00 0x7f365c06d0c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.954+0000 7f3677296640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f367015cb50 0x7f36700cbd40 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.954+0000 7f3677296640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36700d1010 0x7f36700aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.954+0000 7f3677296640 1 --2- 172.21.15.47:0/2794366749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36700cd5a0 0x7f36701f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.954+0000 7f3677296640 1 -- 172.21.15.47:0/2794366749 >> 172.21.15.47:0/2794366749 conn(0x7f3670065df0 msgr2=0x7f36700d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.954+0000 7f3677296640 1 -- 172.21.15.47:0/2794366749 shutdown_connections 2024-03-23T09:43:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:40.954+0000 7f3677296640 1 -- 172.21.15.47:0/2794366749 wait complete. 2024-03-23T09:43:40.965 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-03-23T09:43:40.965 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-23T09:43:40.965 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-23T09:43:40.965 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-03-23T09:43:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.068+0000 7f3463d47640 1 Processor -- start 2024-03-23T09:43:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.068+0000 7f3463d47640 1 -- start start 2024-03-23T09:43:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.069+0000 7f3463d47640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f345c15ca90 0x7f345c160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.069+0000 7f3463d47640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f345c0b8350 0x7f345c0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.069+0000 7f3463d47640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345c0ce090 con 0x7f345c0cdc40 2024-03-23T09:43:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.069+0000 7f3463d47640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345c0d11e0 con 0x7f345c15ca90 2024-03-23T09:43:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.069+0000 7f3463d47640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345c0d2680 con 0x7f345c0b8350 2024-03-23T09:43:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.069+0000 7f3461ae3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f345c15ca90 0x7f345c160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.069+0000 7f3461ae3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f345c15ca90 0x7f345c160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45960/0 (socket says 172.21.15.47:45960) 2024-03-23T09:43:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.069+0000 7f3461ae3640 1 -- 172.21.15.47:0/2570728590 learned_addr learned my addr 172.21.15.47:0/2570728590 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.070+0000 7f34612e2640 1 --2- 172.21.15.47:0/2570728590 >> v2:172.21.15.175:3300/0 conn(0x7f345c0b8350 0x7f345c0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.070+0000 7f3461ae3640 1 -- 172.21.15.47:0/2570728590 >> v2:172.21.15.175:3300/0 conn(0x7f345c0b8350 msgr2=0x7f345c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.070+0000 7f3461ae3640 1 --2- 172.21.15.47:0/2570728590 >> v2:172.21.15.175:3300/0 conn(0x7f345c0b8350 0x7f345c0cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.070+0000 7f3461ae3640 1 -- 172.21.15.47:0/2570728590 >> v1:172.21.15.175:6789/0 conn(0x7f345c0cdc40 legacy=0x7f345c0b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:41.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.070+0000 7f3461ae3640 1 -- 172.21.15.47:0/2570728590 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c160f60 con 0x7f345c15ca90 2024-03-23T09:43:41.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.070+0000 7f34612e2640 1 --2- 172.21.15.47:0/2570728590 >> v2:172.21.15.175:3300/0 conn(0x7f345c0b8350 0x7f345c0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:41.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.070+0000 7f3461ae3640 1 --2- 172.21.15.47:0/2570728590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f345c15ca90 0x7f345c160710 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f345c160670 tx=0x7f344c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e23af6b1915288e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:41.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.070+0000 7f3460ae1640 1 -- 172.21.15.47:0/2570728590 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c02c9f0 con 0x7f345c15ca90 2024-03-23T09:43:41.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.071+0000 7f3460ae1640 1 -- 172.21.15.47:0/2570728590 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c02cb50 con 0x7f345c15ca90 2024-03-23T09:43:41.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.071+0000 7f3460ae1640 1 -- 172.21.15.47:0/2570728590 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c0343f0 con 0x7f345c15ca90 2024-03-23T09:43:41.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.071+0000 7f3463d47640 1 -- 172.21.15.47:0/2570728590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f345c15ca90 msgr2=0x7f345c160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:41.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.071+0000 7f3463d47640 1 --2- 172.21.15.47:0/2570728590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f345c15ca90 0x7f345c160710 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f345c160670 tx=0x7f344c02be50 comp rx=0 tx=0).stop 2024-03-23T09:43:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.071+0000 7f3463d47640 1 -- 172.21.15.47:0/2570728590 shutdown_connections 2024-03-23T09:43:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.071+0000 7f3463d47640 1 --2- 172.21.15.47:0/2570728590 >> v2:172.21.15.175:3300/0 conn(0x7f345c0b8350 0x7f345c0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.071+0000 7f3463d47640 1 --2- 172.21.15.47:0/2570728590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f345c15ca90 0x7f345c160710 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.071+0000 7f3463d47640 1 -- 172.21.15.47:0/2570728590 >> 172.21.15.47:0/2570728590 conn(0x7f345c0659d0 msgr2=0x7f345c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.071+0000 7f3463d47640 1 -- 172.21.15.47:0/2570728590 shutdown_connections 2024-03-23T09:43:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.071+0000 7f3463d47640 1 -- 172.21.15.47:0/2570728590 wait complete. 2024-03-23T09:43:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.072+0000 7f3463d47640 1 Processor -- start 2024-03-23T09:43:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.072+0000 7f3463d47640 1 -- start start 2024-03-23T09:43:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3463d47640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f345c0b8350 0x7f345c156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3463d47640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f345c0cdc40 0x7f345c1f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3463d47640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345c15ca90 0x7f345c0d2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3463d47640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c1f63a0 con 0x7f345c0b8350 2024-03-23T09:43:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3463d47640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c1f6680 con 0x7f345c0cdc40 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3463d47640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345c1f6960 con 0x7f345c15ca90 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3461ae3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f345c0b8350 0x7f345c156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3461ae3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f345c0b8350 0x7f345c156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45974/0 (socket says 172.21.15.47:45974) 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3461ae3640 1 -- 172.21.15.47:0/365805418 learned_addr learned my addr 172.21.15.47:0/365805418 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f34612e2640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f345c0cdc40 0x7f345c1f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f34622e4640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345c15ca90 0x7f345c0d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3461ae3640 1 -- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345c15ca90 msgr2=0x7f345c0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3461ae3640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345c15ca90 0x7f345c0d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3461ae3640 1 -- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f345c0cdc40 msgr2=0x7f345c1f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3461ae3640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f345c0cdc40 0x7f345c1f5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f3461ae3640 1 -- 172.21.15.47:0/365805418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c009b30 con 0x7f345c0b8350 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.073+0000 7f34612e2640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f345c0cdc40 0x7f345c1f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.074+0000 7f3461ae3640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f345c0b8350 0x7f345c156090 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f344c000f80 tx=0x7f344c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.074+0000 7f345a7fc640 1 -- 172.21.15.47:0/365805418 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c03e450 con 0x7f345c0b8350 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.074+0000 7f345a7fc640 1 -- 172.21.15.47:0/365805418 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c03e5f0 con 0x7f345c0b8350 2024-03-23T09:43:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.074+0000 7f345a7fc640 1 -- 172.21.15.47:0/365805418 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c03e9a0 con 0x7f345c0b8350 2024-03-23T09:43:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.074+0000 7f3463d47640 1 -- 172.21.15.47:0/365805418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345c15dfa0 con 0x7f345c0b8350 2024-03-23T09:43:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.074+0000 7f3463d47640 1 -- 172.21.15.47:0/365805418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345c1f60b0 con 0x7f345c0b8350 2024-03-23T09:43:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.075+0000 7f3463d47640 1 -- 172.21.15.47:0/365805418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345c072360 con 0x7f345c0b8350 2024-03-23T09:43:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.075+0000 7f345a7fc640 1 -- 172.21.15.47:0/365805418 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f344c03eb00 con 0x7f345c0b8350 2024-03-23T09:43:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.076+0000 7f345a7fc640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f344806ac00 0x7f344806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.076+0000 7f345a7fc640 1 -- 172.21.15.47:0/365805418 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f344c0abdf0 con 0x7f345c0b8350 2024-03-23T09:43:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.076+0000 7f34612e2640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f344806ac00 0x7f344806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.077+0000 7f34612e2640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f344806ac00 0x7f344806d0c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f345c0b4d10 tx=0x7f3450008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.080+0000 7f345a7fc640 1 -- 172.21.15.47:0/365805418 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f344c0773e0 con 0x7f345c0b8350 2024-03-23T09:43:41.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.326+0000 7f3463d47640 1 -- 172.21.15.47:0/365805418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f345c15e4c0 con 0x7f345c0b8350 2024-03-23T09:43:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.953+0000 7f345a7fc640 1 -- 172.21.15.47:0/365805418 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v142) v1 ==== 172+0+0 (secure 0 0 0) 0x7f345c15e4c0 con 0x7f345c0b8350 2024-03-23T09:43:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.955+0000 7f3463d47640 1 -- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f344806ac00 msgr2=0x7f344806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.955+0000 7f3463d47640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f344806ac00 0x7f344806d0c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f345c0b4d10 tx=0x7f3450008040 comp rx=0 tx=0).stop 2024-03-23T09:43:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.955+0000 7f3463d47640 1 -- 172.21.15.47:0/365805418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f345c0b8350 msgr2=0x7f345c156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.955+0000 7f3463d47640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f345c0b8350 0x7f345c156090 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f344c000f80 tx=0x7f344c004a40 comp rx=0 tx=0).stop 2024-03-23T09:43:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.955+0000 7f3463d47640 1 -- 172.21.15.47:0/365805418 shutdown_connections 2024-03-23T09:43:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.955+0000 7f3463d47640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f344806ac00 0x7f344806d0c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.955+0000 7f3463d47640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345c15ca90 0x7f345c0d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.955+0000 7f3463d47640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f345c0cdc40 0x7f345c1f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.955+0000 7f3463d47640 1 --2- 172.21.15.47:0/365805418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f345c0b8350 0x7f345c156090 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.955+0000 7f3463d47640 1 -- 172.21.15.47:0/365805418 >> 172.21.15.47:0/365805418 conn(0x7f345c0659d0 msgr2=0x7f345c0b7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.955+0000 7f3463d47640 1 -- 172.21.15.47:0/365805418 shutdown_connections 2024-03-23T09:43:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:41.955+0000 7f3463d47640 1 -- 172.21.15.47:0/365805418 wait complete. 2024-03-23T09:43:41.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-23T09:43:42.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.082+0000 7ff3abf00640 1 Processor -- start 2024-03-23T09:43:42.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.082+0000 7ff3abf00640 1 -- start start 2024-03-23T09:43:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3abf00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff3a40b8570 0x7ff3a40b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3abf00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3a40d1e40 0x7ff3a40b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3abf00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a40d09a0 con 0x7ff3a415c790 2024-03-23T09:43:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3abf00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a40b7350 con 0x7ff3a40b8570 2024-03-23T09:43:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3abf00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40b7630 con 0x7ff3a40d1e40 2024-03-23T09:43:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3a949b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff3a40b8570 0x7ff3a40b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3a9c9c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff3a415c790 0x7ff3a40cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47100/0 (socket says 172.21.15.47:47100) 2024-03-23T09:43:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3a949b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff3a40b8570 0x7ff3a40b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45980/0 (socket says 172.21.15.47:45980) 2024-03-23T09:43:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3a9c9c640 1 -- 172.21.15.47:0/2219010842 learned_addr learned my addr 172.21.15.47:0/2219010842 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3aa49d640 1 --2- 172.21.15.47:0/2219010842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3a40d1e40 0x7ff3a40b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3a949b640 1 -- 172.21.15.47:0/2219010842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3a40d1e40 msgr2=0x7ff3a40b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3a949b640 1 --2- 172.21.15.47:0/2219010842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3a40d1e40 0x7ff3a40b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3a949b640 1 -- 172.21.15.47:0/2219010842 >> v1:172.21.15.47:6789/0 conn(0x7ff3a415c790 legacy=0x7ff3a40cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3a8c9a640 1 -- 172.21.15.47:0/2219010842 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132111198 0 0) 0x7ff3a40d09a0 con 0x7ff3a415c790 2024-03-23T09:43:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.083+0000 7ff3a949b640 1 -- 172.21.15.47:0/2219010842 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a40b7910 con 0x7ff3a40b8570 2024-03-23T09:43:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.084+0000 7ff3a949b640 1 --2- 172.21.15.47:0/2219010842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a40b8570 0x7ff3a40b5940 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff3940097a0 tx=0x7ff39402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ec4d6c1fed4215 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.084+0000 7ff3a8c9a640 1 -- 172.21.15.47:0/2219010842 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff394030610 con 0x7ff3a40b8570 2024-03-23T09:43:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.084+0000 7ff3a8c9a640 1 -- 172.21.15.47:0/2219010842 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff394030770 con 0x7ff3a40b8570 2024-03-23T09:43:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.084+0000 7ff3a8c9a640 1 -- 172.21.15.47:0/2219010842 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff394030b00 con 0x7ff3a40b8570 2024-03-23T09:43:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.084+0000 7ff3abf00640 1 -- 172.21.15.47:0/2219010842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a40b8570 msgr2=0x7ff3a40b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.084+0000 7ff3abf00640 1 --2- 172.21.15.47:0/2219010842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a40b8570 0x7ff3a40b5940 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff3940097a0 tx=0x7ff39402b960 comp rx=0 tx=0).stop 2024-03-23T09:43:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.085+0000 7ff3abf00640 1 -- 172.21.15.47:0/2219010842 shutdown_connections 2024-03-23T09:43:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.085+0000 7ff3abf00640 1 --2- 172.21.15.47:0/2219010842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3a40d1e40 0x7ff3a40b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.085+0000 7ff3abf00640 1 --2- 172.21.15.47:0/2219010842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a40b8570 0x7ff3a40b5940 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.085+0000 7ff3abf00640 1 -- 172.21.15.47:0/2219010842 >> 172.21.15.47:0/2219010842 conn(0x7ff3a4065df0 msgr2=0x7ff3a4066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.085+0000 7ff3abf00640 1 -- 172.21.15.47:0/2219010842 shutdown_connections 2024-03-23T09:43:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.085+0000 7ff3abf00640 1 -- 172.21.15.47:0/2219010842 wait complete. 2024-03-23T09:43:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.086+0000 7ff3abf00640 1 Processor -- start 2024-03-23T09:43:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.086+0000 7ff3abf00640 1 -- start start 2024-03-23T09:43:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.086+0000 7ff3abf00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a40b8570 0x7ff3a4148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.086+0000 7ff3abf00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3a40d1e40 0x7ff3a415e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.086+0000 7ff3abf00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3a415c790 0x7ff3a415ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.086+0000 7ff3abf00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40c7c30 con 0x7ff3a40b8570 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.086+0000 7ff3abf00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a41491f0 con 0x7ff3a415c790 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.086+0000 7ff3abf00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40c6fb0 con 0x7ff3a40d1e40 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff3a9c9c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a40b8570 0x7ff3a4148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff3a9c9c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a40b8570 0x7ff3a4148cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45984/0 (socket says 172.21.15.47:45984) 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff3a9c9c640 1 -- 172.21.15.47:0/3677504182 learned_addr learned my addr 172.21.15.47:0/3677504182 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff3a949b640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3a40d1e40 0x7ff3a415e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff3aa49d640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3a415c790 0x7ff3a415ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff3a9c9c640 1 -- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3a40d1e40 msgr2=0x7ff3a415e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff3a9c9c640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3a40d1e40 0x7ff3a415e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff3a9c9c640 1 -- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3a415c790 msgr2=0x7ff3a415ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff3a9c9c640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3a415c790 0x7ff3a415ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff3a9c9c640 1 -- 172.21.15.47:0/3677504182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff394009400 con 0x7ff3a40b8570 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff3a9c9c640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a40b8570 0x7ff3a4148cb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c4d0 tx=0x7ff3a000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff39a7fc640 1 -- 172.21.15.47:0/3677504182 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00074e0 con 0x7ff3a40b8570 2024-03-23T09:43:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff39a7fc640 1 -- 172.21.15.47:0/3677504182 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a0007660 con 0x7ff3a40b8570 2024-03-23T09:43:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff3abf00640 1 -- 172.21.15.47:0/3677504182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a40c7ed0 con 0x7ff3a40b8570 2024-03-23T09:43:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff39a7fc640 1 -- 172.21.15.47:0/3677504182 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0004a20 con 0x7ff3a40b8570 2024-03-23T09:43:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.087+0000 7ff3abf00640 1 -- 172.21.15.47:0/3677504182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a4155a70 con 0x7ff3a40b8570 2024-03-23T09:43:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.088+0000 7ff3abf00640 1 -- 172.21.15.47:0/3677504182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a4072360 con 0x7ff3a40b8570 2024-03-23T09:43:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.089+0000 7ff39a7fc640 1 -- 172.21.15.47:0/3677504182 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff3a0004b80 con 0x7ff3a40b8570 2024-03-23T09:43:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.089+0000 7ff39a7fc640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff38806ac00 0x7ff38806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.089+0000 7ff39a7fc640 1 -- 172.21.15.47:0/3677504182 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff3a0088230 con 0x7ff3a40b8570 2024-03-23T09:43:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.089+0000 7ff3a949b640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff38806ac00 0x7ff38806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.090+0000 7ff3a949b640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff38806ac00 0x7ff38806d0c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff394036000 tx=0x7ff39403b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.093+0000 7ff39a7fc640 1 -- 172.21.15.47:0/3677504182 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff3a0053710 con 0x7ff3a40b8570 2024-03-23T09:43:42.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.270+0000 7ff3abf00640 1 -- 172.21.15.47:0/3677504182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff3a40c80c0 con 0x7ff3a40b8570 2024-03-23T09:43:42.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.274+0000 7ff39a7fc640 1 -- 172.21.15.47:0/3677504182 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff3a0056d30 con 0x7ff3a40b8570 2024-03-23T09:43:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.276+0000 7ff3abf00640 1 -- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff38806ac00 msgr2=0x7ff38806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.276+0000 7ff3abf00640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff38806ac00 0x7ff38806d0c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff394036000 tx=0x7ff39403b040 comp rx=0 tx=0).stop 2024-03-23T09:43:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.276+0000 7ff3abf00640 1 -- 172.21.15.47:0/3677504182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a40b8570 msgr2=0x7ff3a4148cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.276+0000 7ff3abf00640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a40b8570 0x7ff3a4148cb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c4d0 tx=0x7ff3a000c990 comp rx=0 tx=0).stop 2024-03-23T09:43:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.276+0000 7ff3abf00640 1 -- 172.21.15.47:0/3677504182 shutdown_connections 2024-03-23T09:43:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.276+0000 7ff3abf00640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff38806ac00 0x7ff38806d0c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.276+0000 7ff3abf00640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3a415c790 0x7ff3a415ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.276+0000 7ff3abf00640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3a40d1e40 0x7ff3a415e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.276+0000 7ff3abf00640 1 --2- 172.21.15.47:0/3677504182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a40b8570 0x7ff3a4148cb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.276+0000 7ff3abf00640 1 -- 172.21.15.47:0/3677504182 >> 172.21.15.47:0/3677504182 conn(0x7ff3a4065df0 msgr2=0x7ff3a40b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.276+0000 7ff3abf00640 1 -- 172.21.15.47:0/3677504182 shutdown_connections 2024-03-23T09:43:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.276+0000 7ff3abf00640 1 -- 172.21.15.47:0/3677504182 wait complete. 2024-03-23T09:43:42.287 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:43:42.287 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:43:42.324 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:43:42.367 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:43:42.367 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:43:42.367 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:43:42.382 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:43:42.382 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:43:42.382 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:43:42.425 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:43:42.426 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:43:42.426 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:43:42.440 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:43:42.441 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:43:42.441 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:43:42.483 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:43:42.483 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-23T09:43:42.483 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:43:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.583+0000 7f244efa4640 1 Processor -- start 2024-03-23T09:43:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.583+0000 7f244efa4640 1 -- start start 2024-03-23T09:43:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.584+0000 7f244efa4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f244815ca90 0x7f2448160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.584+0000 7f244efa4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24480b8350 0x7f24480cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.584+0000 7f244efa4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24480d11e0 con 0x7f24480cdc40 2024-03-23T09:43:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.584+0000 7f244efa4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24480ce0d0 con 0x7f244815ca90 2024-03-23T09:43:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.584+0000 7f244efa4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24480d2680 con 0x7f24480b8350 2024-03-23T09:43:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.584+0000 7f244cd40640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f244815ca90 0x7f2448160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.584+0000 7f244d541640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f24480cdc40 0x7f24480b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47112/0 (socket says 172.21.15.47:47112) 2024-03-23T09:43:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.584+0000 7f244cd40640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f244815ca90 0x7f2448160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45990/0 (socket says 172.21.15.47:45990) 2024-03-23T09:43:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.584+0000 7f244d541640 1 -- 172.21.15.47:0/3329407188 learned_addr learned my addr 172.21.15.47:0/3329407188 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.584+0000 7f244cd40640 1 -- 172.21.15.47:0/3329407188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24480b8350 msgr2=0x7f24480cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.584+0000 7f244cd40640 1 --2- 172.21.15.47:0/3329407188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24480b8350 0x7f24480cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.584+0000 7f244cd40640 1 -- 172.21.15.47:0/3329407188 >> v1:172.21.15.47:6789/0 conn(0x7f24480cdc40 legacy=0x7f24480b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.584+0000 7f244cd40640 1 -- 172.21.15.47:0/3329407188 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2448160f60 con 0x7f244815ca90 2024-03-23T09:43:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.585+0000 7f244cd40640 1 --2- 172.21.15.47:0/3329407188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244815ca90 0x7f2448160710 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f243c0097a0 tx=0x7f243c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0d19dfc409365b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.585+0000 7f24477fe640 1 -- 172.21.15.47:0/3329407188 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c02e070 con 0x7f244815ca90 2024-03-23T09:43:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.585+0000 7f24477fe640 1 -- 172.21.15.47:0/3329407188 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c0315a0 con 0x7f244815ca90 2024-03-23T09:43:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.585+0000 7f24477fe640 1 -- 172.21.15.47:0/3329407188 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c031930 con 0x7f244815ca90 2024-03-23T09:43:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.585+0000 7f244efa4640 1 -- 172.21.15.47:0/3329407188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244815ca90 msgr2=0x7f2448160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.585+0000 7f244efa4640 1 --2- 172.21.15.47:0/3329407188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244815ca90 0x7f2448160710 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f243c0097a0 tx=0x7f243c02b840 comp rx=0 tx=0).stop 2024-03-23T09:43:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.585+0000 7f244efa4640 1 -- 172.21.15.47:0/3329407188 shutdown_connections 2024-03-23T09:43:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.585+0000 7f244efa4640 1 --2- 172.21.15.47:0/3329407188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24480b8350 0x7f24480cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.585+0000 7f244efa4640 1 --2- 172.21.15.47:0/3329407188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244815ca90 0x7f2448160710 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.585+0000 7f244efa4640 1 -- 172.21.15.47:0/3329407188 >> 172.21.15.47:0/3329407188 conn(0x7f24480659d0 msgr2=0x7f2448065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.585+0000 7f244efa4640 1 -- 172.21.15.47:0/3329407188 shutdown_connections 2024-03-23T09:43:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.586+0000 7f244efa4640 1 -- 172.21.15.47:0/3329407188 wait complete. 2024-03-23T09:43:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.587+0000 7f244efa4640 1 Processor -- start 2024-03-23T09:43:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.587+0000 7f244efa4640 1 -- start start 2024-03-23T09:43:42.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.587+0000 7f244efa4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24480b8350 0x7f2448150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.587+0000 7f244efa4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24480cdc40 0x7f24481524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.587+0000 7f244efa4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f244815ca90 0x7f244815eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.587+0000 7f244efa4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24480ca560 con 0x7f24480cdc40 2024-03-23T09:43:42.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f244efa4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24480cc6a0 con 0x7f24480b8350 2024-03-23T09:43:42.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f244efa4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244815e490 con 0x7f244815ca90 2024-03-23T09:43:42.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f2447fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24480cdc40 0x7f24481524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f2447fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24480cdc40 0x7f24481524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46006/0 (socket says 172.21.15.47:46006) 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f2447fff640 1 -- 172.21.15.47:0/1445513618 learned_addr learned my addr 172.21.15.47:0/1445513618 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f244d541640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f244815ca90 0x7f244815eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f244cd40640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24480b8350 0x7f2448150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f2447fff640 1 -- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f244815ca90 msgr2=0x7f244815eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f2447fff640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f244815ca90 0x7f244815eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f2447fff640 1 -- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24480b8350 msgr2=0x7f2448150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f2447fff640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24480b8350 0x7f2448150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f2447fff640 1 -- 172.21.15.47:0/1445513618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c009400 con 0x7f24480cdc40 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f2447fff640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24480cdc40 0x7f24481524a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f243400b410 tx=0x7f243400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f24457fa640 1 -- 172.21.15.47:0/1445513618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434011830 con 0x7f24480cdc40 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f24457fa640 1 -- 172.21.15.47:0/1445513618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24340119b0 con 0x7f24480cdc40 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f244efa4640 1 -- 172.21.15.47:0/1445513618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24480c9bc0 con 0x7f24480cdc40 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f24457fa640 1 -- 172.21.15.47:0/1445513618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434011d60 con 0x7f24480cdc40 2024-03-23T09:43:42.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.588+0000 7f244efa4640 1 -- 172.21.15.47:0/1445513618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24480ca180 con 0x7f24480cdc40 2024-03-23T09:43:42.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.589+0000 7f244efa4640 1 -- 172.21.15.47:0/1445513618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2448072360 con 0x7f24480cdc40 2024-03-23T09:43:42.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.590+0000 7f24457fa640 1 -- 172.21.15.47:0/1445513618 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f243401a400 con 0x7f24480cdc40 2024-03-23T09:43:42.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.590+0000 7f24457fa640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f242c06ac00 0x7f242c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.590+0000 7f24457fa640 1 -- 172.21.15.47:0/1445513618 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2434089130 con 0x7f24480cdc40 2024-03-23T09:43:42.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.590+0000 7f244cd40640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f242c06ac00 0x7f242c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.591+0000 7f244cd40640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f242c06ac00 0x7f242c06d0c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f243c036000 tx=0x7f243c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.594+0000 7f24457fa640 1 -- 172.21.15.47:0/1445513618 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f24340546a0 con 0x7f24480cdc40 2024-03-23T09:43:42.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.811+0000 7f244efa4640 1 -- 172.21.15.47:0/1445513618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2448066be0 con 0x7f24480cdc40 2024-03-23T09:43:42.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.811+0000 7f24457fa640 1 -- 172.21.15.47:0/1445513618 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f2434015de0 con 0x7f24480cdc40 2024-03-23T09:43:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:42.813 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1189831856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1189831856},{"type":"v1","addr":"172.21.15.47:6835","nonce":1189831856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2480484980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2480484980},{"type":"v1","addr":"172.21.15.175:6837","nonce":2480484980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/2248064454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2248064454},{"type":"v1","addr":"172.21.15.47:6834","nonce":2248064454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5694,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3806465612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3806465612},{"type":"v1","addr":"172.21.15.175:6835","nonce":3806465612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[]} 2024-03-23T09:43:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-03-23T09:43:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.813+0000 7f244efa4640 1 -- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f242c06ac00 msgr2=0x7f242c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.813+0000 7f244efa4640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f242c06ac00 0x7f242c06d0c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f243c036000 tx=0x7f243c03d040 comp rx=0 tx=0).stop 2024-03-23T09:43:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.813+0000 7f244efa4640 1 -- 172.21.15.47:0/1445513618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24480cdc40 msgr2=0x7f24481524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.813+0000 7f244efa4640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24480cdc40 0x7f24481524a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f243400b410 tx=0x7f243400b8d0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.813+0000 7f244efa4640 1 -- 172.21.15.47:0/1445513618 shutdown_connections 2024-03-23T09:43:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.813+0000 7f244efa4640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f242c06ac00 0x7f242c06d0c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.813+0000 7f244efa4640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f244815ca90 0x7f244815eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.813+0000 7f244efa4640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24480cdc40 0x7f24481524a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.813+0000 7f244efa4640 1 --2- 172.21.15.47:0/1445513618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24480b8350 0x7f2448150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.813+0000 7f244efa4640 1 -- 172.21.15.47:0/1445513618 >> 172.21.15.47:0/1445513618 conn(0x7f24480659d0 msgr2=0x7f24480c8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.813+0000 7f244efa4640 1 -- 172.21.15.47:0/1445513618 shutdown_connections 2024-03-23T09:43:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.813+0000 7f244efa4640 1 -- 172.21.15.47:0/1445513618 wait complete. 2024-03-23T09:43:42.826 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-23T09:43:42.826 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-23T09:43:42.826 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-23T09:43:42.827 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:43:42.827 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-23T09:43:42.827 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-23T09:43:42.827 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-23T09:43:42.827 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:43:42.827 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-23T09:43:42.827 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-23T09:43:42.828 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-23T09:43:42.828 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:43:42.828 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-23T09:43:42.828 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-23T09:43:42.828 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-23T09:43:42.828 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:43:42.836 INFO:tasks.ceph.mds.a:Stopped 2024-03-23T09:43:42.836 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-23T09:43:42.837 INFO:tasks.ceph.mds.c:Stopped 2024-03-23T09:43:42.837 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-23T09:43:42.840 INFO:tasks.ceph.mds.b:Stopped 2024-03-23T09:43:42.840 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-23T09:43:42.841 INFO:tasks.ceph.mds.d:Stopped 2024-03-23T09:43:42.841 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-23T09:43:42.879 INFO:tasks.ceph.mds.a:Started 2024-03-23T09:43:42.879 DEBUG:teuthology.parallel:result is None 2024-03-23T09:43:42.879 INFO:tasks.ceph.mds.c:Started 2024-03-23T09:43:42.880 DEBUG:teuthology.parallel:result is None 2024-03-23T09:43:42.883 INFO:tasks.ceph.mds.b:Started 2024-03-23T09:43:42.883 DEBUG:teuthology.parallel:result is None 2024-03-23T09:43:42.884 INFO:tasks.ceph.mds.d:Started 2024-03-23T09:43:42.884 DEBUG:teuthology.parallel:result is None 2024-03-23T09:43:42.884 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-23T09:43:42.955 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-03-23T09:43:42.956 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:43:42.969 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-03-23T09:43:42.969 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-03-23T09:43:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.994+0000 7fbd1d030640 1 Processor -- start 2024-03-23T09:43:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.994+0000 7fbd1d030640 1 -- start start 2024-03-23T09:43:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd1d030640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbd1815cd00 0x7fbd180d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd1d030640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd180d1020 0x7fbd180cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd1d030640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd180cf750 0x7fbd180b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd1d030640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd180cfbe0 con 0x7fbd180d1020 2024-03-23T09:43:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd1d030640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd1815d0e0 con 0x7fbd1815cd00 2024-03-23T09:43:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd1d030640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180d1540 con 0x7fbd180cf750 2024-03-23T09:43:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd0ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd180d1020 0x7fbd180cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd0ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd180d1020 0x7fbd180cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46054/0 (socket says 172.21.15.47:46054) 2024-03-23T09:43:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd0ffff640 1 -- 172.21.15.47:0/1768534249 learned_addr learned my addr 172.21.15.47:0/1768534249 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd1759e640 1 --2- 172.21.15.47:0/1768534249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd180cf750 0x7fbd180b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd16d9d640 1 --2- 172.21.15.47:0/1768534249 >> v2:172.21.15.175:3300/0 conn(0x7fbd1815cd00 0x7fbd180d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd0ffff640 1 -- 172.21.15.47:0/1768534249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd180cf750 msgr2=0x7fbd180b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd0ffff640 1 --2- 172.21.15.47:0/1768534249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd180cf750 0x7fbd180b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd0ffff640 1 -- 172.21.15.47:0/1768534249 >> v2:172.21.15.175:3300/0 conn(0x7fbd1815cd00 msgr2=0x7fbd180d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd0ffff640 1 --2- 172.21.15.47:0/1768534249 >> v2:172.21.15.175:3300/0 conn(0x7fbd1815cd00 0x7fbd180d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.995+0000 7fbd0ffff640 1 -- 172.21.15.47:0/1768534249 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd180d1820 con 0x7fbd180d1020 2024-03-23T09:43:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.997+0000 7fbd0ffff640 1 --2- 172.21.15.47:0/1768534249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180d1020 0x7fbd180cf210 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbd1806c1a0 tx=0x7fbd0002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63acbb4e33c8bd08 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.997+0000 7fbd1659c640 1 -- 172.21.15.47:0/1768534249 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd000317e0 con 0x7fbd180d1020 2024-03-23T09:43:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.997+0000 7fbd1659c640 1 -- 172.21.15.47:0/1768534249 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd00031940 con 0x7fbd180d1020 2024-03-23T09:43:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.997+0000 7fbd1659c640 1 -- 172.21.15.47:0/1768534249 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd00031cd0 con 0x7fbd180d1020 2024-03-23T09:43:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.997+0000 7fbd1d030640 1 -- 172.21.15.47:0/1768534249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180d1020 msgr2=0x7fbd180cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.997+0000 7fbd1d030640 1 --2- 172.21.15.47:0/1768534249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180d1020 0x7fbd180cf210 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbd1806c1a0 tx=0x7fbd0002b880 comp rx=0 tx=0).stop 2024-03-23T09:43:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.997+0000 7fbd1d030640 1 -- 172.21.15.47:0/1768534249 shutdown_connections 2024-03-23T09:43:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.997+0000 7fbd1d030640 1 --2- 172.21.15.47:0/1768534249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd180cf750 0x7fbd180b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.997+0000 7fbd1d030640 1 --2- 172.21.15.47:0/1768534249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180d1020 0x7fbd180cf210 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.997+0000 7fbd1d030640 1 --2- 172.21.15.47:0/1768534249 >> v2:172.21.15.175:3300/0 conn(0x7fbd1815cd00 0x7fbd180d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.997+0000 7fbd1d030640 1 -- 172.21.15.47:0/1768534249 >> 172.21.15.47:0/1768534249 conn(0x7fbd1806b160 msgr2=0x7fbd1806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.997+0000 7fbd1d030640 1 -- 172.21.15.47:0/1768534249 shutdown_connections 2024-03-23T09:43:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.997+0000 7fbd1d030640 1 -- 172.21.15.47:0/1768534249 wait complete. 2024-03-23T09:43:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.999+0000 7fbd1d030640 1 Processor -- start 2024-03-23T09:43:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.999+0000 7fbd1d030640 1 -- start start 2024-03-23T09:43:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.999+0000 7fbd1d030640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd180cf750 0x7fbd181f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.999+0000 7fbd1d030640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180d1020 0x7fbd180b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.999+0000 7fbd1d030640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd1815cd00 0x7fbd181f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:42.999+0000 7fbd1d030640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180c9ad0 con 0x7fbd180d1020 2024-03-23T09:43:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd1d030640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180c9db0 con 0x7fbd180cf750 2024-03-23T09:43:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd1d030640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180ca090 con 0x7fbd1815cd00 2024-03-23T09:43:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd0ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180d1020 0x7fbd180b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd0ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180d1020 0x7fbd180b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46058/0 (socket says 172.21.15.47:46058) 2024-03-23T09:43:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd0ffff640 1 -- 172.21.15.47:0/1491462379 learned_addr learned my addr 172.21.15.47:0/1491462379 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd1759e640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd1815cd00 0x7fbd181f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd16d9d640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd180cf750 0x7fbd181f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd0ffff640 1 -- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd1815cd00 msgr2=0x7fbd181f2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd0ffff640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd1815cd00 0x7fbd181f2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd0ffff640 1 -- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd180cf750 msgr2=0x7fbd181f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd0ffff640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd180cf750 0x7fbd181f3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd0ffff640 1 -- 172.21.15.47:0/1491462379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd000094d0 con 0x7fbd180d1020 2024-03-23T09:43:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd1759e640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd1815cd00 0x7fbd181f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:43:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd16d9d640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd180cf750 0x7fbd181f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:43:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.000+0000 7fbd0ffff640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180d1020 0x7fbd180b4640 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbd00000c00 tx=0x7fbd000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.002+0000 7fbd0f7fe640 1 -- 172.21.15.47:0/1491462379 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd00032880 con 0x7fbd180d1020 2024-03-23T09:43:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.002+0000 7fbd1d030640 1 -- 172.21.15.47:0/1491462379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd180ca370 con 0x7fbd180d1020 2024-03-23T09:43:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.002+0000 7fbd0f7fe640 1 -- 172.21.15.47:0/1491462379 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd00031ee0 con 0x7fbd180d1020 2024-03-23T09:43:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.002+0000 7fbd0f7fe640 1 -- 172.21.15.47:0/1491462379 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd00032be0 con 0x7fbd180d1020 2024-03-23T09:43:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.002+0000 7fbd1d030640 1 -- 172.21.15.47:0/1491462379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd180b3b50 con 0x7fbd180d1020 2024-03-23T09:43:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.003+0000 7fbd0f7fe640 1 -- 172.21.15.47:0/1491462379 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbd00032d40 con 0x7fbd180d1020 2024-03-23T09:43:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.003+0000 7fbd1d030640 1 -- 172.21.15.47:0/1491462379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd18072360 con 0x7fbd180d1020 2024-03-23T09:43:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.004+0000 7fbd0f7fe640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbcf806ac30 0x7fbcf806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.004+0000 7fbd0f7fe640 1 -- 172.21.15.47:0/1491462379 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbd000aaf10 con 0x7fbd180d1020 2024-03-23T09:43:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.004+0000 7fbd16d9d640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbcf806ac30 0x7fbcf806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.004+0000 7fbd16d9d640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbcf806ac30 0x7fbcf806d0f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009f30 tx=0x7fbd08009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.008+0000 7fbd0f7fe640 1 -- 172.21.15.47:0/1491462379 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbd000764c0 con 0x7fbd180d1020 2024-03-23T09:43:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:43.199+0000 7fbd1d030640 1 -- 172.21.15.47:0/1491462379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbd180c9c10 con 0x7fbd180d1020 2024-03-23T09:43:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.004+0000 7fbd0f7fe640 1 -- 172.21.15.47:0/1491462379 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbd180c9c10 con 0x7fbd180d1020 2024-03-23T09:43:44.005 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blocklist entries 2024-03-23T09:43:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.007+0000 7fbd1d030640 1 -- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbcf806ac30 msgr2=0x7fbcf806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.007+0000 7fbd1d030640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbcf806ac30 0x7fbcf806d0f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009f30 tx=0x7fbd08009190 comp rx=0 tx=0).stop 2024-03-23T09:43:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.007+0000 7fbd1d030640 1 -- 172.21.15.47:0/1491462379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180d1020 msgr2=0x7fbd180b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.007+0000 7fbd1d030640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180d1020 0x7fbd180b4640 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbd00000c00 tx=0x7fbd000046c0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.007+0000 7fbd1d030640 1 -- 172.21.15.47:0/1491462379 shutdown_connections 2024-03-23T09:43:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.007+0000 7fbd1d030640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbcf806ac30 0x7fbcf806d0f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.007+0000 7fbd1d030640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd1815cd00 0x7fbd181f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.007+0000 7fbd1d030640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180d1020 0x7fbd180b4640 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.007+0000 7fbd1d030640 1 --2- 172.21.15.47:0/1491462379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd180cf750 0x7fbd181f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.007+0000 7fbd1d030640 1 -- 172.21.15.47:0/1491462379 >> 172.21.15.47:0/1491462379 conn(0x7fbd1806b160 msgr2=0x7fbd180c8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.007+0000 7fbd1d030640 1 -- 172.21.15.47:0/1491462379 shutdown_connections 2024-03-23T09:43:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.007+0000 7fbd1d030640 1 -- 172.21.15.47:0/1491462379 wait complete. 2024-03-23T09:43:44.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-23T09:43:44.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.127+0000 7f428af4e640 1 Processor -- start 2024-03-23T09:43:44.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.127+0000 7f428af4e640 1 -- start start 2024-03-23T09:43:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.128+0000 7f428af4e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4284142a90 0x7f42840cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.128+0000 7f428af4e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4284143010 0x7f42840b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.128+0000 7f428af4e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42840ccd00 con 0x7f42840b40e0 2024-03-23T09:43:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.128+0000 7f428af4e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42840b27c0 con 0x7f4284142a90 2024-03-23T09:43:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.128+0000 7f428af4e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4284142e70 con 0x7f4284143010 2024-03-23T09:43:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.128+0000 7f4288cea640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4284142a90 0x7f42840cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.128+0000 7f4288cea640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4284142a90 0x7f42840cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46060/0 (socket says 172.21.15.47:46060) 2024-03-23T09:43:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.128+0000 7f4288cea640 1 -- 172.21.15.47:0/1027316300 learned_addr learned my addr 172.21.15.47:0/1027316300 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.128+0000 7f4283fff640 1 --2- 172.21.15.47:0/1027316300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4284143010 0x7f42840b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.128+0000 7f4288cea640 1 -- 172.21.15.47:0/1027316300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4284143010 msgr2=0x7f42840b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.129+0000 7f4288cea640 1 --2- 172.21.15.47:0/1027316300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4284143010 0x7f42840b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.129+0000 7f4288cea640 1 -- 172.21.15.47:0/1027316300 >> v1:172.21.15.175:6789/0 conn(0x7f42840b40e0 legacy=0x7f428415d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.129+0000 7f4288cea640 1 -- 172.21.15.47:0/1027316300 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42840ccfe0 con 0x7f4284142a90 2024-03-23T09:43:44.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.129+0000 7f4288cea640 1 --2- 172.21.15.47:0/1027316300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4284142a90 0x7f42840cc4b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f427c0098a0 tx=0x7f427c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3342bd8ef84bb037 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:44.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.129+0000 7f42837fe640 1 -- 172.21.15.47:0/1027316300 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c030840 con 0x7f4284142a90 2024-03-23T09:43:44.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.129+0000 7f42837fe640 1 -- 172.21.15.47:0/1027316300 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427c0309a0 con 0x7f4284142a90 2024-03-23T09:43:44.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.129+0000 7f42837fe640 1 -- 172.21.15.47:0/1027316300 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c030d30 con 0x7f4284142a90 2024-03-23T09:43:44.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.130+0000 7f428af4e640 1 -- 172.21.15.47:0/1027316300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4284142a90 msgr2=0x7f42840cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.130+0000 7f428af4e640 1 --2- 172.21.15.47:0/1027316300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4284142a90 0x7f42840cc4b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f427c0098a0 tx=0x7f427c02b980 comp rx=0 tx=0).stop 2024-03-23T09:43:44.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.130+0000 7f428af4e640 1 -- 172.21.15.47:0/1027316300 shutdown_connections 2024-03-23T09:43:44.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.130+0000 7f428af4e640 1 --2- 172.21.15.47:0/1027316300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4284143010 0x7f42840b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.130+0000 7f428af4e640 1 --2- 172.21.15.47:0/1027316300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4284142a90 0x7f42840cc4b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.130+0000 7f428af4e640 1 -- 172.21.15.47:0/1027316300 >> 172.21.15.47:0/1027316300 conn(0x7f428406a310 msgr2=0x7f42840b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:44.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.130+0000 7f428af4e640 1 -- 172.21.15.47:0/1027316300 shutdown_connections 2024-03-23T09:43:44.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.130+0000 7f428af4e640 1 -- 172.21.15.47:0/1027316300 wait complete. 2024-03-23T09:43:44.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.131+0000 7f428af4e640 1 Processor -- start 2024-03-23T09:43:44.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.131+0000 7f428af4e640 1 -- start start 2024-03-23T09:43:44.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f428af4e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42840b40e0 0x7f428415bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f428af4e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4284142a90 0x7f4284132f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f428af4e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4284143010 0x7f42841592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f428af4e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4284133460 con 0x7f42840b40e0 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f428af4e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42841597f0 con 0x7f4284143010 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f428af4e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42841336c0 con 0x7f4284142a90 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f4288cea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42840b40e0 0x7f428415bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f4288cea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42840b40e0 0x7f428415bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46074/0 (socket says 172.21.15.47:46074) 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f4288cea640 1 -- 172.21.15.47:0/219266881 learned_addr learned my addr 172.21.15.47:0/219266881 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f42894eb640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4284143010 0x7f42841592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f4283fff640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4284142a90 0x7f4284132f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f4288cea640 1 -- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4284142a90 msgr2=0x7f4284132f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f4288cea640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4284142a90 0x7f4284132f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f4288cea640 1 -- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4284143010 msgr2=0x7f42841592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f4288cea640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4284143010 0x7f42841592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.132+0000 7f4288cea640 1 -- 172.21.15.47:0/219266881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c0094d0 con 0x7f42840b40e0 2024-03-23T09:43:44.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.133+0000 7f4288cea640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42840b40e0 0x7f428415bd00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f427c0098a0 tx=0x7f427c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:44.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.133+0000 7f42817fa640 1 -- 172.21.15.47:0/219266881 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c047070 con 0x7f42840b40e0 2024-03-23T09:43:44.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.133+0000 7f42817fa640 1 -- 172.21.15.47:0/219266881 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427c039a90 con 0x7f42840b40e0 2024-03-23T09:43:44.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.133+0000 7f428af4e640 1 -- 172.21.15.47:0/219266881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42841339a0 con 0x7f42840b40e0 2024-03-23T09:43:44.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.133+0000 7f42817fa640 1 -- 172.21.15.47:0/219266881 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c039e40 con 0x7f42840b40e0 2024-03-23T09:43:44.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.133+0000 7f428af4e640 1 -- 172.21.15.47:0/219266881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4284133f60 con 0x7f42840b40e0 2024-03-23T09:43:44.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.134+0000 7f428af4e640 1 -- 172.21.15.47:0/219266881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4284073480 con 0x7f42840b40e0 2024-03-23T09:43:44.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.134+0000 7f42817fa640 1 -- 172.21.15.47:0/219266881 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f427c04c410 con 0x7f42840b40e0 2024-03-23T09:43:44.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.135+0000 7f42817fa640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f426406ac50 0x7f426406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.135+0000 7f42817fa640 1 -- 172.21.15.47:0/219266881 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f427c03e070 con 0x7f42840b40e0 2024-03-23T09:43:44.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.135+0000 7f4283fff640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f426406ac50 0x7f426406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.135+0000 7f4283fff640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f426406ac50 0x7f426406d110 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f428415b970 tx=0x7f4274003990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:44.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.139+0000 7f42817fa640 1 -- 172.21.15.47:0/219266881 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f427c076ce0 con 0x7f42840b40e0 2024-03-23T09:43:44.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.388+0000 7f428af4e640 1 -- 172.21.15.47:0/219266881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4284066be0 con 0x7f42840b40e0 2024-03-23T09:43:44.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.389+0000 7f42817fa640 1 -- 172.21.15.47:0/219266881 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f427c07a300 con 0x7f42840b40e0 2024-03-23T09:43:44.390 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:44.390 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:43:44.390 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2024-03-23T09:43:44.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.390 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2024-03-23T09:43:44.390 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lo+xoDhAAyxqr1H6eiINwt2X7LUlrSg==", 2024-03-23T09:43:44.390 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.390 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:43:44.390 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:43:44.390 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:43:44.390 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.390 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.391 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2024-03-23T09:43:44.391 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZ5OlGhAAuUElzRghNRFGKREdCpxKvA==", 2024-03-23T09:43:44.391 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.391 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:43:44.391 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:43:44.391 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:43:44.391 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.391 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.391 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lFwyzFRAAKldMOA7oSblA8YMrvntY8w==", 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.d", 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lpiD3IRAApXAdUQ4AkxWHzsFV0qcBng==", 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:43:44.392 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB7o/5lLvoADBAAgdixkm+jHwQilYw0IAn6EA==", 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2024-03-23T09:43:44.393 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB+o/5lOBfGCRAApglzalO1ztnJzOKg/UAyWA==", 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCBo/5lRBjlBxAAd45w8/kTMPqxmNyfCS3XYA==", 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCEo/5l+JxrBhAAbR/XqDyPDnN8eLpmvBZ7Mg==", 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCKo/5ltsqwDRAAkqcZ/sEf2oozWFN4AXg/+w==", 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:43:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCNo/5lpLHiCxAAB2axUv9gjcYVlij9f+AEHg==", 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCQo/5l2io3ChAAu+lfYcWWBQf43aYzWjCBNg==", 2024-03-23T09:43:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCTo/5lA7jjCBAAjJgBkp89H8lx6fidl/Z7IA==", 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZFc3JhAAZWmDZOnkW9eo9hqEDYynQw==", 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lIrJSLxAAPJer3ZP3h8VkImkiJmnKVw==", 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:43:44.398 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l1+DyKxAAZLKZNbUpUJ0fWPIYNvWK+g==", 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2024-03-23T09:43:44.399 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lWqkiNRAAL1q0lEcM/NKW3Pf9LcP21g==", 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB4o/5lyeuKLRAARrfTpPc5BTVYZe/LUREDkg==", 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:43:44.400 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l/0EOCxAAIrgIXRiFSQ1G7Jat5hNm4A==", 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lywMFBxAAIVwjCe6B9EtBMAG2n7cZJw==", 2024-03-23T09:43:44.401 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:43:44.402 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:43:44.402 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:43:44.402 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:43:44.402 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.402 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:44.402 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-23T09:43:44.402 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:43:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.390+0000 7f428af4e640 1 -- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f426406ac50 msgr2=0x7f426406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.390+0000 7f428af4e640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f426406ac50 0x7f426406d110 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f428415b970 tx=0x7f4274003990 comp rx=0 tx=0).stop 2024-03-23T09:43:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.390+0000 7f428af4e640 1 -- 172.21.15.47:0/219266881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42840b40e0 msgr2=0x7f428415bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.390+0000 7f428af4e640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42840b40e0 0x7f428415bd00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f427c0098a0 tx=0x7f427c004730 comp rx=0 tx=0).stop 2024-03-23T09:43:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.391+0000 7f428af4e640 1 -- 172.21.15.47:0/219266881 shutdown_connections 2024-03-23T09:43:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.391+0000 7f428af4e640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f426406ac50 0x7f426406d110 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.391+0000 7f428af4e640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4284143010 0x7f42841592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.391+0000 7f428af4e640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4284142a90 0x7f4284132f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.391+0000 7f428af4e640 1 --2- 172.21.15.47:0/219266881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42840b40e0 0x7f428415bd00 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.391+0000 7f428af4e640 1 -- 172.21.15.47:0/219266881 >> 172.21.15.47:0/219266881 conn(0x7f428406a310 msgr2=0x7f428406c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.391+0000 7f428af4e640 1 -- 172.21.15.47:0/219266881 shutdown_connections 2024-03-23T09:43:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.391+0000 7f428af4e640 1 -- 172.21.15.47:0/219266881 wait complete. 2024-03-23T09:43:44.405 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:43:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.505+0000 7f2b88da4640 1 Processor -- start 2024-03-23T09:43:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.505+0000 7f2b88da4640 1 -- start start 2024-03-23T09:43:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b88da4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2b8415ca90 0x7f2b84160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b88da4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2b840cde10 0x7f2b840d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b88da4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b840d11e0 con 0x7f2b840b8350 2024-03-23T09:43:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b88da4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b84160f60 con 0x7f2b840cde10 2024-03-23T09:43:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b88da4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b84161240 con 0x7f2b8415ca90 2024-03-23T09:43:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b82d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2b840cde10 0x7f2b840d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b81d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2b840b8350 0x7f2b840cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47144/0 (socket says 172.21.15.47:47144) 2024-03-23T09:43:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b82d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2b840cde10 0x7f2b840d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46076/0 (socket says 172.21.15.47:46076) 2024-03-23T09:43:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b81d9b640 1 -- 172.21.15.47:0/1006486565 learned_addr learned my addr 172.21.15.47:0/1006486565 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b8259c640 1 --2- 172.21.15.47:0/1006486565 >> v2:172.21.15.175:3300/0 conn(0x7f2b8415ca90 0x7f2b84160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b82d9d640 1 -- 172.21.15.47:0/1006486565 >> v2:172.21.15.175:3300/0 conn(0x7f2b8415ca90 msgr2=0x7f2b84160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b82d9d640 1 --2- 172.21.15.47:0/1006486565 >> v2:172.21.15.175:3300/0 conn(0x7f2b8415ca90 0x7f2b84160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b82d9d640 1 -- 172.21.15.47:0/1006486565 >> v1:172.21.15.47:6789/0 conn(0x7f2b840b8350 legacy=0x7f2b840cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.506+0000 7f2b82d9d640 1 -- 172.21.15.47:0/1006486565 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b84161520 con 0x7f2b840cde10 2024-03-23T09:43:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.507+0000 7f2b82d9d640 1 --2- 172.21.15.47:0/1006486565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b840cde10 0x7f2b840d1670 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2b84066680 tx=0x7f2b7c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ef8414c7d5005a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.507+0000 7f2b8159a640 1 -- 172.21.15.47:0/1006486565 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c02cbf0 con 0x7f2b840cde10 2024-03-23T09:43:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.507+0000 7f2b8159a640 1 -- 172.21.15.47:0/1006486565 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7c02cd50 con 0x7f2b840cde10 2024-03-23T09:43:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.507+0000 7f2b8159a640 1 -- 172.21.15.47:0/1006486565 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c034660 con 0x7f2b840cde10 2024-03-23T09:43:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.507+0000 7f2b88da4640 1 -- 172.21.15.47:0/1006486565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b840cde10 msgr2=0x7f2b840d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.507+0000 7f2b88da4640 1 --2- 172.21.15.47:0/1006486565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b840cde10 0x7f2b840d1670 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2b84066680 tx=0x7f2b7c02be40 comp rx=0 tx=0).stop 2024-03-23T09:43:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.508+0000 7f2b88da4640 1 -- 172.21.15.47:0/1006486565 shutdown_connections 2024-03-23T09:43:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.508+0000 7f2b88da4640 1 --2- 172.21.15.47:0/1006486565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b840cde10 0x7f2b840d1670 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.508+0000 7f2b88da4640 1 --2- 172.21.15.47:0/1006486565 >> v2:172.21.15.175:3300/0 conn(0x7f2b8415ca90 0x7f2b84160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.508+0000 7f2b88da4640 1 -- 172.21.15.47:0/1006486565 >> 172.21.15.47:0/1006486565 conn(0x7f2b840659d0 msgr2=0x7f2b84065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.508+0000 7f2b88da4640 1 -- 172.21.15.47:0/1006486565 shutdown_connections 2024-03-23T09:43:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.508+0000 7f2b88da4640 1 -- 172.21.15.47:0/1006486565 wait complete. 2024-03-23T09:43:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.509+0000 7f2b88da4640 1 Processor -- start 2024-03-23T09:43:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.509+0000 7f2b88da4640 1 -- start start 2024-03-23T09:43:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b88da4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b840b8350 0x7f2b8414c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b88da4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b840cde10 0x7f2b840b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b88da4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b8415ca90 0x7f2b8415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b88da4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b841f3880 con 0x7f2b840cde10 2024-03-23T09:43:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b88da4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8414c980 con 0x7f2b8415ca90 2024-03-23T09:43:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b88da4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b841f29d0 con 0x7f2b840b8350 2024-03-23T09:43:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b81d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b840cde10 0x7f2b840b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b81d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b840cde10 0x7f2b840b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46088/0 (socket says 172.21.15.47:46088) 2024-03-23T09:43:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b81d9b640 1 -- 172.21.15.47:0/4150245275 learned_addr learned my addr 172.21.15.47:0/4150245275 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b8259c640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b840b8350 0x7f2b8414c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b82d9d640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b8415ca90 0x7f2b8415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b81d9b640 1 -- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b840b8350 msgr2=0x7f2b8414c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b81d9b640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b840b8350 0x7f2b8414c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b81d9b640 1 -- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b8415ca90 msgr2=0x7f2b8415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b81d9b640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b8415ca90 0x7f2b8415f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b81d9b640 1 -- 172.21.15.47:0/4150245275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c009a70 con 0x7f2b840cde10 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b8259c640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b840b8350 0x7f2b8414c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b81d9b640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b840cde10 0x7f2b840b4cc0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c4d0 tx=0x7f2b7800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b72ffd640 1 -- 172.21.15.47:0/4150245275 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78005ce0 con 0x7f2b840cde10 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b72ffd640 1 -- 172.21.15.47:0/4150245275 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b78005e40 con 0x7f2b840cde10 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b88da4640 1 -- 172.21.15.47:0/4150245275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b841f2c70 con 0x7f2b840cde10 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.510+0000 7f2b72ffd640 1 -- 172.21.15.47:0/4150245275 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78007270 con 0x7f2b840cde10 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.511+0000 7f2b88da4640 1 -- 172.21.15.47:0/4150245275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b841f3230 con 0x7f2b840cde10 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.512+0000 7f2b88da4640 1 -- 172.21.15.47:0/4150245275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b84072360 con 0x7f2b840cde10 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.512+0000 7f2b72ffd640 1 -- 172.21.15.47:0/4150245275 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2b780073d0 con 0x7f2b840cde10 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.512+0000 7f2b72ffd640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b6406ac00 0x7f2b6406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.513+0000 7f2b72ffd640 1 -- 172.21.15.47:0/4150245275 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2b780884c0 con 0x7f2b840cde10 2024-03-23T09:43:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.513+0000 7f2b8259c640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b6406ac00 0x7f2b6406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.513+0000 7f2b8259c640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b6406ac00 0x7f2b6406d0c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2b74006fd0 tx=0x7f2b74008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.517+0000 7f2b72ffd640 1 -- 172.21.15.47:0/4150245275 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2b78053aa0 con 0x7f2b840cde10 2024-03-23T09:43:44.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.698+0000 7f2b88da4640 1 -- 172.21.15.47:0/4150245275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b84066be0 con 0x7f2b840cde10 2024-03-23T09:43:44.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.698+0000 7f2b72ffd640 1 -- 172.21.15.47:0/4150245275 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+12389 (secure 0 0 0) 0x7f2b78017090 con 0x7f2b840cde10 2024-03-23T09:43:44.699 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:44.700 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":54,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:43:44.002461+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:43:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.700+0000 7f2b88da4640 1 -- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b6406ac00 msgr2=0x7f2b6406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.700+0000 7f2b88da4640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b6406ac00 0x7f2b6406d0c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2b74006fd0 tx=0x7f2b74008040 comp rx=0 tx=0).stop 2024-03-23T09:43:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.700+0000 7f2b88da4640 1 -- 172.21.15.47:0/4150245275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b840cde10 msgr2=0x7f2b840b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.700+0000 7f2b88da4640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b840cde10 0x7f2b840b4cc0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c4d0 tx=0x7f2b7800c990 comp rx=0 tx=0).stop 2024-03-23T09:43:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.700+0000 7f2b88da4640 1 -- 172.21.15.47:0/4150245275 shutdown_connections 2024-03-23T09:43:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.700+0000 7f2b88da4640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b6406ac00 0x7f2b6406d0c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.700+0000 7f2b88da4640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b8415ca90 0x7f2b8415f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.700+0000 7f2b88da4640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b840cde10 0x7f2b840b4cc0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.700+0000 7f2b88da4640 1 --2- 172.21.15.47:0/4150245275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b840b8350 0x7f2b8414c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.700+0000 7f2b88da4640 1 -- 172.21.15.47:0/4150245275 >> 172.21.15.47:0/4150245275 conn(0x7f2b840659d0 msgr2=0x7f2b840d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:44.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.700+0000 7f2b88da4640 1 -- 172.21.15.47:0/4150245275 shutdown_connections 2024-03-23T09:43:44.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.700+0000 7f2b88da4640 1 -- 172.21.15.47:0/4150245275 wait complete. 2024-03-23T09:43:44.712 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:43:44.712 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:43:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.815+0000 7fb45924a640 1 Processor -- start 2024-03-23T09:43:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.815+0000 7fb45924a640 1 -- start start 2024-03-23T09:43:44.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.816+0000 7fb45924a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb45415a9b0 0x7fb4540cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.816+0000 7fb45924a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45406a3f0 0x7fb4540cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.816+0000 7fb45924a640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4540cd180 con 0x7fb45415bc70 2024-03-23T09:43:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.816+0000 7fb45924a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb45415b7e0 con 0x7fb45415a9b0 2024-03-23T09:43:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.816+0000 7fb45924a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4540ccea0 con 0x7fb45406a3f0 2024-03-23T09:43:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.816+0000 7fb452d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb45415a9b0 0x7fb4540cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.816+0000 7fb452d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb45415a9b0 0x7fb4540cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38892/0 (socket says 172.21.15.47:38892) 2024-03-23T09:43:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.816+0000 7fb452d9d640 1 -- 172.21.15.47:0/3092095845 learned_addr learned my addr 172.21.15.47:0/3092095845 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.816+0000 7fb45359e640 1 --2- 172.21.15.47:0/3092095845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45406a3f0 0x7fb4540cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.816+0000 7fb452d9d640 1 -- 172.21.15.47:0/3092095845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45406a3f0 msgr2=0x7fb4540cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.816+0000 7fb452d9d640 1 --2- 172.21.15.47:0/3092095845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45406a3f0 0x7fb4540cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.816+0000 7fb452d9d640 1 -- 172.21.15.47:0/3092095845 >> v1:172.21.15.175:6789/0 conn(0x7fb45415bc70 legacy=0x7fb454069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.816+0000 7fb452d9d640 1 -- 172.21.15.47:0/3092095845 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45415b190 con 0x7fb45415a9b0 2024-03-23T09:43:44.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.817+0000 7fb452d9d640 1 --2- 172.21.15.47:0/3092095845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb45415a9b0 0x7fb4540cb970 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0098a0 tx=0x7fb44c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712a97fa02169897 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:44.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.817+0000 7fb451d9b640 1 -- 172.21.15.47:0/3092095845 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c031640 con 0x7fb45415a9b0 2024-03-23T09:43:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.817+0000 7fb451d9b640 1 -- 172.21.15.47:0/3092095845 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44c0317a0 con 0x7fb45415a9b0 2024-03-23T09:43:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.817+0000 7fb451d9b640 1 -- 172.21.15.47:0/3092095845 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c031b30 con 0x7fb45415a9b0 2024-03-23T09:43:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.818+0000 7fb45924a640 1 -- 172.21.15.47:0/3092095845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb45415a9b0 msgr2=0x7fb4540cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.818+0000 7fb45924a640 1 --2- 172.21.15.47:0/3092095845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb45415a9b0 0x7fb4540cb970 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0098a0 tx=0x7fb44c02b8f0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.818+0000 7fb45924a640 1 -- 172.21.15.47:0/3092095845 shutdown_connections 2024-03-23T09:43:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.818+0000 7fb45924a640 1 --2- 172.21.15.47:0/3092095845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45406a3f0 0x7fb4540cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.818+0000 7fb45924a640 1 --2- 172.21.15.47:0/3092095845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb45415a9b0 0x7fb4540cb970 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.818+0000 7fb45924a640 1 -- 172.21.15.47:0/3092095845 >> 172.21.15.47:0/3092095845 conn(0x7fb454071400 msgr2=0x7fb454071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.818+0000 7fb45924a640 1 -- 172.21.15.47:0/3092095845 shutdown_connections 2024-03-23T09:43:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.818+0000 7fb45924a640 1 -- 172.21.15.47:0/3092095845 wait complete. 2024-03-23T09:43:44.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.819+0000 7fb45924a640 1 Processor -- start 2024-03-23T09:43:44.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.819+0000 7fb45924a640 1 -- start start 2024-03-23T09:43:44.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.820+0000 7fb45924a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb45406a3f0 0x7fb45414d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.820+0000 7fb45924a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45415a9b0 0x7fb4541efb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.820+0000 7fb45924a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb45415bc70 0x7fb4541f0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.820+0000 7fb45924a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4540c6b50 con 0x7fb45406a3f0 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.820+0000 7fb45924a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4540c6e30 con 0x7fb45415bc70 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.820+0000 7fb45924a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4541f05b0 con 0x7fb45415a9b0 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.820+0000 7fb452d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb45406a3f0 0x7fb45414d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.820+0000 7fb452d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb45406a3f0 0x7fb45414d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38902/0 (socket says 172.21.15.47:38902) 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.820+0000 7fb452d9d640 1 -- 172.21.15.47:0/1900670550 learned_addr learned my addr 172.21.15.47:0/1900670550 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.820+0000 7fb45359e640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb45415bc70 0x7fb4541f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.820+0000 7fb452d9d640 1 -- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45415a9b0 msgr2=0x7fb4541efb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.820+0000 7fb45259c640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45415a9b0 0x7fb4541efb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.821+0000 7fb452d9d640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45415a9b0 0x7fb4541efb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.821+0000 7fb452d9d640 1 -- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb45415bc70 msgr2=0x7fb4541f0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.821+0000 7fb452d9d640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb45415bc70 0x7fb4541f0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.821+0000 7fb452d9d640 1 -- 172.21.15.47:0/1900670550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c0094d0 con 0x7fb45406a3f0 2024-03-23T09:43:44.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.821+0000 7fb45359e640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb45415bc70 0x7fb4541f0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:44.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.821+0000 7fb45259c640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45415a9b0 0x7fb4541efb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:44.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.821+0000 7fb452d9d640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb45406a3f0 0x7fb45414d5b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb44c000c00 tx=0x7fb44c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:44.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.821+0000 7fb44b7fe640 1 -- 172.21.15.47:0/1900670550 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c03ac60 con 0x7fb45406a3f0 2024-03-23T09:43:44.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.821+0000 7fb44b7fe640 1 -- 172.21.15.47:0/1900670550 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44c03ade0 con 0x7fb45406a3f0 2024-03-23T09:43:44.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.821+0000 7fb45924a640 1 -- 172.21.15.47:0/1900670550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4541501d0 con 0x7fb45406a3f0 2024-03-23T09:43:44.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.821+0000 7fb44b7fe640 1 -- 172.21.15.47:0/1900670550 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c044630 con 0x7fb45406a3f0 2024-03-23T09:43:44.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.821+0000 7fb45924a640 1 -- 172.21.15.47:0/1900670550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb454150790 con 0x7fb45406a3f0 2024-03-23T09:43:44.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.822+0000 7fb45924a640 1 -- 172.21.15.47:0/1900670550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4540672f0 con 0x7fb45406a3f0 2024-03-23T09:43:44.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.822+0000 7fb44b7fe640 1 -- 172.21.15.47:0/1900670550 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb44c02f050 con 0x7fb45406a3f0 2024-03-23T09:43:44.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.823+0000 7fb44b7fe640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb43406ac00 0x7fb43406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:44.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.823+0000 7fb44b7fe640 1 -- 172.21.15.47:0/1900670550 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb44c0ab020 con 0x7fb45406a3f0 2024-03-23T09:43:44.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.823+0000 7fb45259c640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb43406ac00 0x7fb43406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:44.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.823+0000 7fb45259c640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb43406ac00 0x7fb43406d0c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb4540b0870 tx=0x7fb43c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:44.827+0000 7fb44b7fe640 1 -- 172.21.15.47:0/1900670550 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb44c076550 con 0x7fb45406a3f0 2024-03-23T09:43:45.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.009+0000 7fb45924a640 1 -- 172.21.15.47:0/1900670550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4540cd940 con 0x7fb45406a3f0 2024-03-23T09:43:45.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.009+0000 7fb44b7fe640 1 -- 172.21.15.47:0/1900670550 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4540cd940 con 0x7fb45406a3f0 2024-03-23T09:43:45.010 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:43:45.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.012+0000 7fb45924a640 1 -- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb43406ac00 msgr2=0x7fb43406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:45.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.012+0000 7fb45924a640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb43406ac00 0x7fb43406d0c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb4540b0870 tx=0x7fb43c008040 comp rx=0 tx=0).stop 2024-03-23T09:43:45.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.012+0000 7fb45924a640 1 -- 172.21.15.47:0/1900670550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb45406a3f0 msgr2=0x7fb45414d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:45.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.012+0000 7fb45924a640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb45406a3f0 0x7fb45414d5b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb44c000c00 tx=0x7fb44c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.012+0000 7fb45924a640 1 -- 172.21.15.47:0/1900670550 shutdown_connections 2024-03-23T09:43:45.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.012+0000 7fb45924a640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb43406ac00 0x7fb43406d0c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.012+0000 7fb45924a640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb45415bc70 0x7fb4541f0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.012+0000 7fb45924a640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45415a9b0 0x7fb4541efb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.012+0000 7fb45924a640 1 --2- 172.21.15.47:0/1900670550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb45406a3f0 0x7fb45414d5b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.012+0000 7fb45924a640 1 -- 172.21.15.47:0/1900670550 >> 172.21.15.47:0/1900670550 conn(0x7fb454071400 msgr2=0x7fb45415d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:45.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.012+0000 7fb45924a640 1 -- 172.21.15.47:0/1900670550 shutdown_connections 2024-03-23T09:43:45.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.012+0000 7fb45924a640 1 -- 172.21.15.47:0/1900670550 wait complete. 2024-03-23T09:43:45.023 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T09:43:45.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.123+0000 7f09be763640 1 Processor -- start 2024-03-23T09:43:45.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.123+0000 7f09be763640 1 -- start start 2024-03-23T09:43:45.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.124+0000 7f09be763640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f09b815cd00 0x7f09b80d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.124+0000 7f09be763640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09b80d1020 0x7f09b80cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.124+0000 7f09be763640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b815d0e0 con 0x7f09b80cf750 2024-03-23T09:43:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.124+0000 7f09be763640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b80cfbe0 con 0x7f09b815cd00 2024-03-23T09:43:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.124+0000 7f09be763640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80d1540 con 0x7f09b80d1020 2024-03-23T09:43:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.124+0000 7f09b7fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f09b815cd00 0x7f09b80d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.124+0000 7f09b77fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09b80d1020 0x7f09b80cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.124+0000 7f09b7fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f09b815cd00 0x7f09b80d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:39414/0 (socket says 172.21.15.47:39414) 2024-03-23T09:43:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.124+0000 7f09b7fff640 1 -- 172.21.15.47:0/707660332 learned_addr learned my addr 172.21.15.47:0/707660332 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.124+0000 7f09bcd00640 1 --1- 172.21.15.47:0/707660332 >> v1:172.21.15.175:6789/0 conn(0x7f09b80cf750 0x7f09b80b43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:44596/0 (socket says 172.21.15.47:44596) 2024-03-23T09:43:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.125+0000 7f09b77fe640 1 -- 172.21.15.47:0/707660332 >> v2:172.21.15.175:3300/0 conn(0x7f09b815cd00 msgr2=0x7f09b80d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.125+0000 7f09b77fe640 1 --2- 172.21.15.47:0/707660332 >> v2:172.21.15.175:3300/0 conn(0x7f09b815cd00 0x7f09b80d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.125+0000 7f09b77fe640 1 -- 172.21.15.47:0/707660332 >> v1:172.21.15.175:6789/0 conn(0x7f09b80cf750 legacy=0x7f09b80b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:45.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.125+0000 7f09b77fe640 1 -- 172.21.15.47:0/707660332 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b80d1820 con 0x7f09b80d1020 2024-03-23T09:43:45.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.125+0000 7f09b7fff640 1 --2- 172.21.15.47:0/707660332 >> v2:172.21.15.175:3300/0 conn(0x7f09b815cd00 0x7f09b80d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:43:45.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.126+0000 7f09b77fe640 1 --2- 172.21.15.47:0/707660332 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09b80d1020 0x7f09b80cf210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f09a80097a0 tx=0x7f09a802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f895bf887c57364 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:45.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.126+0000 7f09b6ffd640 1 -- 172.21.15.47:0/707660332 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a802e070 con 0x7f09b80d1020 2024-03-23T09:43:45.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.126+0000 7f09b6ffd640 1 -- 172.21.15.47:0/707660332 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a80315a0 con 0x7f09b80d1020 2024-03-23T09:43:45.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.126+0000 7f09b6ffd640 1 -- 172.21.15.47:0/707660332 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a8031930 con 0x7f09b80d1020 2024-03-23T09:43:45.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.126+0000 7f09be763640 1 -- 172.21.15.47:0/707660332 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09b80d1020 msgr2=0x7f09b80cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:45.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.126+0000 7f09be763640 1 --2- 172.21.15.47:0/707660332 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09b80d1020 0x7f09b80cf210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f09a80097a0 tx=0x7f09a802b840 comp rx=0 tx=0).stop 2024-03-23T09:43:45.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.126+0000 7f09be763640 1 -- 172.21.15.47:0/707660332 shutdown_connections 2024-03-23T09:43:45.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.126+0000 7f09be763640 1 --2- 172.21.15.47:0/707660332 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09b80d1020 0x7f09b80cf210 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.126+0000 7f09be763640 1 --2- 172.21.15.47:0/707660332 >> v2:172.21.15.175:3300/0 conn(0x7f09b815cd00 0x7f09b80d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.126+0000 7f09be763640 1 -- 172.21.15.47:0/707660332 >> 172.21.15.47:0/707660332 conn(0x7f09b806b160 msgr2=0x7f09b806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:45.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.126+0000 7f09be763640 1 -- 172.21.15.47:0/707660332 shutdown_connections 2024-03-23T09:43:45.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.126+0000 7f09be763640 1 -- 172.21.15.47:0/707660332 wait complete. 2024-03-23T09:43:45.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.128+0000 7f09be763640 1 Processor -- start 2024-03-23T09:43:45.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.128+0000 7f09be763640 1 -- start start 2024-03-23T09:43:45.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.128+0000 7f09be763640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09b80cf750 0x7f09b8156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.128+0000 7f09be763640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b80d1020 0x7f09b80b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.128+0000 7f09be763640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09b815cd00 0x7f09b80c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.128+0000 7f09be763640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80c7360 con 0x7f09b815cd00 2024-03-23T09:43:45.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.128+0000 7f09be763640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80b4e20 con 0x7f09b80d1020 2024-03-23T09:43:45.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.128+0000 7f09be763640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80b4b80 con 0x7f09b80cf750 2024-03-23T09:43:45.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09bcd00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09b815cd00 0x7f09b80c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09bcd00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09b815cd00 0x7f09b80c6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38904/0 (socket says 172.21.15.47:38904) 2024-03-23T09:43:45.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09bcd00640 1 -- 172.21.15.47:0/1089883991 learned_addr learned my addr 172.21.15.47:0/1089883991 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09b7fff640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09b80cf750 0x7f09b8156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09b77fe640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b80d1020 0x7f09b80b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09bcd00640 1 -- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09b80cf750 msgr2=0x7f09b8156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09bcd00640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09b80cf750 0x7f09b8156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09bcd00640 1 -- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b80d1020 msgr2=0x7f09b80b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09bcd00640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b80d1020 0x7f09b80b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09bcd00640 1 -- 172.21.15.47:0/1089883991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0094d0 con 0x7f09b815cd00 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09b7fff640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09b80cf750 0x7f09b8156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09bcd00640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09b815cd00 0x7f09b80c6490 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f09b000c760 tx=0x7f09b000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09b4ff9640 1 -- 172.21.15.47:0/1089883991 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b000ce70 con 0x7f09b815cd00 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09b4ff9640 1 -- 172.21.15.47:0/1089883991 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b000a640 con 0x7f09b815cd00 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09be763640 1 -- 172.21.15.47:0/1089883991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a8009400 con 0x7f09b815cd00 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09b4ff9640 1 -- 172.21.15.47:0/1089883991 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b000a9b0 con 0x7f09b815cd00 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.129+0000 7f09be763640 1 -- 172.21.15.47:0/1089883991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b80b3250 con 0x7f09b815cd00 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.130+0000 7f09be763640 1 -- 172.21.15.47:0/1089883991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b8072360 con 0x7f09b815cd00 2024-03-23T09:43:45.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.131+0000 7f09b4ff9640 1 -- 172.21.15.47:0/1089883991 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f09b000ab10 con 0x7f09b815cd00 2024-03-23T09:43:45.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.131+0000 7f09b4ff9640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09a406ac00 0x7f09a406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.131+0000 7f09b4ff9640 1 -- 172.21.15.47:0/1089883991 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f09b0088770 con 0x7f09b815cd00 2024-03-23T09:43:45.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.131+0000 7f09b7fff640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09a406ac00 0x7f09a406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.132+0000 7f09b7fff640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09a406ac00 0x7f09a406d0c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f09ac006ca0 tx=0x7f09ac009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:45.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.135+0000 7f09b4ff9640 1 -- 172.21.15.47:0/1089883991 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f09b0053cb0 con 0x7f09b815cd00 2024-03-23T09:43:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.316+0000 7f09be763640 1 -- 172.21.15.47:0/1089883991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f09b8066be0 con 0x7f09b815cd00 2024-03-23T09:43:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.316+0000 7f09b4ff9640 1 -- 172.21.15.47:0/1089883991 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) v1 ==== 84+0+65 (secure 0 0 0) 0x7f09b0017090 con 0x7f09b815cd00 2024-03-23T09:43:45.317 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:45.317 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-23T09:43:45.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:45.317 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-03-23T09:43:45.318 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-03-23T09:43:45.318 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:45.318 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-23T09:43:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.318+0000 7f09be763640 1 -- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09a406ac00 msgr2=0x7f09a406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.318+0000 7f09be763640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09a406ac00 0x7f09a406d0c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f09ac006ca0 tx=0x7f09ac009240 comp rx=0 tx=0).stop 2024-03-23T09:43:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.318+0000 7f09be763640 1 -- 172.21.15.47:0/1089883991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09b815cd00 msgr2=0x7f09b80c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.318+0000 7f09be763640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09b815cd00 0x7f09b80c6490 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f09b000c760 tx=0x7f09b000cc20 comp rx=0 tx=0).stop 2024-03-23T09:43:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.318+0000 7f09be763640 1 -- 172.21.15.47:0/1089883991 shutdown_connections 2024-03-23T09:43:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.318+0000 7f09be763640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09a406ac00 0x7f09a406d0c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.318+0000 7f09be763640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09b815cd00 0x7f09b80c6490 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.318+0000 7f09be763640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b80d1020 0x7f09b80b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.318+0000 7f09be763640 1 --2- 172.21.15.47:0/1089883991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09b80cf750 0x7f09b8156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.318+0000 7f09be763640 1 -- 172.21.15.47:0/1089883991 >> 172.21.15.47:0/1089883991 conn(0x7f09b806b160 msgr2=0x7f09b815d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.318+0000 7f09be763640 1 -- 172.21.15.47:0/1089883991 shutdown_connections 2024-03-23T09:43:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.318+0000 7f09be763640 1 -- 172.21.15.47:0/1089883991 wait complete. 2024-03-23T09:43:45.329 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T09:43:45.330 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T09:43:45.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.434+0000 7fafb9799640 1 Processor -- start 2024-03-23T09:43:45.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.434+0000 7fafb9799640 1 -- start start 2024-03-23T09:43:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.435+0000 7fafb9799640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415ccf0 0x7fafb41606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.435+0000 7fafb9799640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fafb40ae7f0 0x7fafb40aec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.435+0000 7fafb9799640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb40cf500 con 0x7fafb4069ce0 2024-03-23T09:43:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.435+0000 7fafb9799640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb40ce8f0 con 0x7fafb40ae7f0 2024-03-23T09:43:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.435+0000 7fafb9799640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb40cebd0 con 0x7fafb415ccf0 2024-03-23T09:43:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.435+0000 7fafb37fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fafb4069ce0 0x7fafb415d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35994/0 (socket says 172.21.15.47:35994) 2024-03-23T09:43:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.435+0000 7fafb37fe640 1 -- 172.21.15.47:0/2689637500 learned_addr learned my addr 172.21.15.47:0/2689637500 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.435+0000 7fafb27fc640 1 --2- 172.21.15.47:0/2689637500 >> v2:172.21.15.175:3300/0 conn(0x7fafb40ae7f0 0x7fafb40aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.435+0000 7fafb2ffd640 1 --2- 172.21.15.47:0/2689637500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415ccf0 0x7fafb41606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.435+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635433651 0 0) 0x7fafb40cf500 con 0x7fafb4069ce0 2024-03-23T09:43:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa0003520 con 0x7fafb4069ce0 2024-03-23T09:43:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3841180455 0 0) 0x7fafa0003520 con 0x7fafb4069ce0 2024-03-23T09:43:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415ccf0 msgr2=0x7fafb41606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 --2- 172.21.15.47:0/2689637500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415ccf0 0x7fafb41606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 >> v2:172.21.15.175:3300/0 conn(0x7fafb40ae7f0 msgr2=0x7fafb40aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 --2- 172.21.15.47:0/2689637500 >> v2:172.21.15.175:3300/0 conn(0x7fafb40ae7f0 0x7fafb40aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb40ceeb0 con 0x7fafb4069ce0 2024-03-23T09:43:45.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fafa40019f0 con 0x7fafb4069ce0 2024-03-23T09:43:45.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 >> v1:172.21.15.47:6789/0 conn(0x7fafb4069ce0 legacy=0x7fafb415d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:45.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 --2- 172.21.15.47:0/2689637500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafa0003970 0x7fafa0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 --2- 172.21.15.47:0/2689637500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafa00047c0 0x7fafa0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 --2- 172.21.15.47:0/2689637500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415ccf0 0x7fafa0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb40ce8f0 con 0x7fafa00047c0 2024-03-23T09:43:45.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb40cebd0 con 0x7fafa0003970 2024-03-23T09:43:45.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa000b980 con 0x7fafb415ccf0 2024-03-23T09:43:45.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb27fc640 1 --2- 172.21.15.47:0/2689637500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafa00047c0 0x7fafa0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb2ffd640 1 --2- 172.21.15.47:0/2689637500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafa0003970 0x7fafa0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb37fe640 1 --2- 172.21.15.47:0/2689637500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415ccf0 0x7fafa0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb27fc640 1 -- 172.21.15.47:0/2689637500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415ccf0 msgr2=0x7fafa0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:45.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb27fc640 1 --2- 172.21.15.47:0/2689637500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415ccf0 0x7fafa0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb27fc640 1 -- 172.21.15.47:0/2689637500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafa0003970 msgr2=0x7fafa0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:45.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb27fc640 1 --2- 172.21.15.47:0/2689637500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafa0003970 0x7fafa0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.436+0000 7fafb27fc640 1 -- 172.21.15.47:0/2689637500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa0003520 con 0x7fafa00047c0 2024-03-23T09:43:45.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.437+0000 7fafb27fc640 1 --2- 172.21.15.47:0/2689637500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafa00047c0 0x7fafa0004c40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009ac0 tx=0x7fafa00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3265fd2d420a2ea6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:45.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.437+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c02c710 con 0x7fafa00047c0 2024-03-23T09:43:45.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.437+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c02c870 con 0x7fafa00047c0 2024-03-23T09:43:45.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.437+0000 7fafb1ffb640 1 -- 172.21.15.47:0/2689637500 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c02cc00 con 0x7fafa00047c0 2024-03-23T09:43:45.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.438+0000 7fafb9799640 1 -- 172.21.15.47:0/2689637500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafa00047c0 msgr2=0x7fafa0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:45.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.438+0000 7fafb9799640 1 --2- 172.21.15.47:0/2689637500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafa00047c0 0x7fafa0004c40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009ac0 tx=0x7fafa00057f0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.438+0000 7fafb37fe640 1 -- 172.21.15.47:0/2689637500 reap_dead start 2024-03-23T09:43:45.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.438+0000 7fafb9799640 1 -- 172.21.15.47:0/2689637500 shutdown_connections 2024-03-23T09:43:45.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.438+0000 7fafb9799640 1 -- 172.21.15.47:0/2689637500 >> 172.21.15.47:0/2689637500 conn(0x7fafb4098a00 msgr2=0x7fafb40cf780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:45.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.438+0000 7fafb9799640 1 -- 172.21.15.47:0/2689637500 shutdown_connections 2024-03-23T09:43:45.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.438+0000 7fafb9799640 1 -- 172.21.15.47:0/2689637500 wait complete. 2024-03-23T09:43:45.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.439+0000 7fafb9799640 1 Processor -- start 2024-03-23T09:43:45.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.439+0000 7fafb9799640 1 -- start start 2024-03-23T09:43:45.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb9799640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafa00047c0 0x7fafb41f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb9799640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415b9f0 0x7fafb415be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb9799640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafb40b43c0 0x7fafb40b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb9799640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb415c3b0 con 0x7fafa00047c0 2024-03-23T09:43:45.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb9799640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb40b2670 con 0x7fafb40b43c0 2024-03-23T09:43:45.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb9799640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb40b2950 con 0x7fafb415b9f0 2024-03-23T09:43:45.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafa00047c0 0x7fafb41f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafa00047c0 0x7fafb41f1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38914/0 (socket says 172.21.15.47:38914) 2024-03-23T09:43:45.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb2ffd640 1 -- 172.21.15.47:0/3239538607 learned_addr learned my addr 172.21.15.47:0/3239538607 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:45.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb37fe640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafb40b43c0 0x7fafb40b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb27fc640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415b9f0 0x7fafb415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb2ffd640 1 -- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415b9f0 msgr2=0x7fafb415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:45.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb2ffd640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415b9f0 0x7fafb415be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb2ffd640 1 -- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafb40b43c0 msgr2=0x7fafb40b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb2ffd640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafb40b43c0 0x7fafb40b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb2ffd640 1 -- 172.21.15.47:0/3239538607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb40b2c30 con 0x7fafa00047c0 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb37fe640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafb40b43c0 0x7fafb40b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb27fc640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415b9f0 0x7fafb415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafb2ffd640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafa00047c0 0x7fafb41f1af0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fafac0119a0 tx=0x7fafac011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafab7fe640 1 -- 172.21.15.47:0/3239538607 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac00faf0 con 0x7fafa00047c0 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafab7fe640 1 -- 172.21.15.47:0/3239538607 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafac00fc50 con 0x7fafa00047c0 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.440+0000 7fafab7fe640 1 -- 172.21.15.47:0/3239538607 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac014440 con 0x7fafa00047c0 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.441+0000 7fafb9799640 1 -- 172.21.15.47:0/3239538607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb40b2f10 con 0x7fafa00047c0 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.441+0000 7fafb9799640 1 -- 172.21.15.47:0/3239538607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb41f2030 con 0x7fafa00047c0 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.441+0000 7fafb9799640 1 -- 172.21.15.47:0/3239538607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb40672f0 con 0x7fafa00047c0 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.442+0000 7fafab7fe640 1 -- 172.21.15.47:0/3239538607 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fafac0145a0 con 0x7fafa00047c0 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.442+0000 7fafab7fe640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf9406ac00 0x7faf9406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:45.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.443+0000 7fafab7fe640 1 -- 172.21.15.47:0/3239538607 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fafac089730 con 0x7fafa00047c0 2024-03-23T09:43:45.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.443+0000 7fafb27fc640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf9406ac00 0x7faf9406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:45.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.443+0000 7fafb27fc640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf9406ac00 0x7faf9406d0c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faf9c036000 tx=0x7faf9c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:45.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.447+0000 7fafab7fe640 1 -- 172.21.15.47:0/3239538607 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fafac054cf0 con 0x7fafa00047c0 2024-03-23T09:43:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:45.625+0000 7fafb9799640 1 -- 172.21.15.47:0/3239538607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fafb40ac530 con 0x7fafa00047c0 2024-03-23T09:43:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.009+0000 7fafab7fe640 1 -- 172.21.15.47:0/3239538607 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) v1 ==== 138+0+0 (secure 0 0 0) 0x7fafac058310 con 0x7fafa00047c0 2024-03-23T09:43:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-03-23T09:43:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.012+0000 7fafb9799640 1 -- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf9406ac00 msgr2=0x7faf9406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.012+0000 7fafb9799640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf9406ac00 0x7faf9406d0c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faf9c036000 tx=0x7faf9c030000 comp rx=0 tx=0).stop 2024-03-23T09:43:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.012+0000 7fafb9799640 1 -- 172.21.15.47:0/3239538607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafa00047c0 msgr2=0x7fafb41f1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.012+0000 7fafb9799640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafa00047c0 0x7fafb41f1af0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fafac0119a0 tx=0x7fafac011e60 comp rx=0 tx=0).stop 2024-03-23T09:43:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.013+0000 7fafb9799640 1 -- 172.21.15.47:0/3239538607 shutdown_connections 2024-03-23T09:43:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.013+0000 7fafb9799640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faf9406ac00 0x7faf9406d0c0 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faf9c036000 tx=0x7faf9c030000 comp rx=0 tx=0).stop 2024-03-23T09:43:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.013+0000 7fafb9799640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafb40b43c0 0x7fafb40b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.013+0000 7fafb9799640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafb415b9f0 0x7fafb415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.013+0000 7fafb9799640 1 --2- 172.21.15.47:0/3239538607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafa00047c0 0x7fafb41f1af0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.013+0000 7fafb9799640 1 -- 172.21.15.47:0/3239538607 >> 172.21.15.47:0/3239538607 conn(0x7fafb4098a00 msgr2=0x7fafb40b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.013+0000 7fafb9799640 1 -- 172.21.15.47:0/3239538607 shutdown_connections 2024-03-23T09:43:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.014+0000 7fafb9799640 1 -- 172.21.15.47:0/3239538607 wait complete. 2024-03-23T09:43:46.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T09:43:46.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.136+0000 7ff4de81e640 1 Processor -- start 2024-03-23T09:43:46.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.136+0000 7ff4de81e640 1 -- start start 2024-03-23T09:43:46.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.137+0000 7ff4de81e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4d80b5a10 0x7ff4d80d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:46.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.137+0000 7ff4de81e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d80cd5b0 0x7ff4d80cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:46.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.137+0000 7ff4de81e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d80b6010 con 0x7ff4d81600a0 2024-03-23T09:43:46.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.137+0000 7ff4de81e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d80cdf80 con 0x7ff4d80b5a10 2024-03-23T09:43:46.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.137+0000 7ff4de81e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d80b62f0 con 0x7ff4d80cd5b0 2024-03-23T09:43:46.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.137+0000 7ff4d77fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4d80b5a10 0x7ff4d80d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:46.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.137+0000 7ff4d77fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4d80b5a10 0x7ff4d80d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38920/0 (socket says 172.21.15.47:38920) 2024-03-23T09:43:46.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.137+0000 7ff4d77fe640 1 -- 172.21.15.47:0/2588431302 learned_addr learned my addr 172.21.15.47:0/2588431302 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:46.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.137+0000 7ff4dcdbb640 1 --2- 172.21.15.47:0/2588431302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d80cd5b0 0x7ff4d80cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:46.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.137+0000 7ff4d77fe640 1 -- 172.21.15.47:0/2588431302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d80cd5b0 msgr2=0x7ff4d80cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:46.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.137+0000 7ff4d77fe640 1 --2- 172.21.15.47:0/2588431302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d80cd5b0 0x7ff4d80cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:46.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.137+0000 7ff4d77fe640 1 -- 172.21.15.47:0/2588431302 >> v1:172.21.15.175:6789/0 conn(0x7ff4d81600a0 legacy=0x7ff4d80b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:46.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.137+0000 7ff4d77fe640 1 -- 172.21.15.47:0/2588431302 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d80b65d0 con 0x7ff4d80b5a10 2024-03-23T09:43:46.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.138+0000 7ff4d77fe640 1 --2- 172.21.15.47:0/2588431302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d80b5a10 0x7ff4d80d0c40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009870 tx=0x7ff4c802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5753834b7a54f67f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:46.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.138+0000 7ff4d6ffd640 1 -- 172.21.15.47:0/2588431302 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8031640 con 0x7ff4d80b5a10 2024-03-23T09:43:46.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.138+0000 7ff4d6ffd640 1 -- 172.21.15.47:0/2588431302 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c80317a0 con 0x7ff4d80b5a10 2024-03-23T09:43:46.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.138+0000 7ff4d6ffd640 1 -- 172.21.15.47:0/2588431302 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8031b30 con 0x7ff4d80b5a10 2024-03-23T09:43:46.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.139+0000 7ff4de81e640 1 -- 172.21.15.47:0/2588431302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d80b5a10 msgr2=0x7ff4d80d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:46.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.139+0000 7ff4de81e640 1 --2- 172.21.15.47:0/2588431302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d80b5a10 0x7ff4d80d0c40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009870 tx=0x7ff4c802b8f0 comp rx=0 tx=0).stop 2024-03-23T09:43:46.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.139+0000 7ff4de81e640 1 -- 172.21.15.47:0/2588431302 shutdown_connections 2024-03-23T09:43:46.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.139+0000 7ff4de81e640 1 --2- 172.21.15.47:0/2588431302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d80cd5b0 0x7ff4d80cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:46.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.139+0000 7ff4de81e640 1 --2- 172.21.15.47:0/2588431302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d80b5a10 0x7ff4d80d0c40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:46.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.139+0000 7ff4de81e640 1 -- 172.21.15.47:0/2588431302 >> 172.21.15.47:0/2588431302 conn(0x7ff4d806afd0 msgr2=0x7ff4d806b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:46.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.139+0000 7ff4de81e640 1 -- 172.21.15.47:0/2588431302 shutdown_connections 2024-03-23T09:43:46.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.139+0000 7ff4de81e640 1 -- 172.21.15.47:0/2588431302 wait complete. 2024-03-23T09:43:46.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.140+0000 7ff4de81e640 1 Processor -- start 2024-03-23T09:43:46.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.140+0000 7ff4de81e640 1 -- start start 2024-03-23T09:43:46.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4de81e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4d80b5a10 0x7ff4d81562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:46.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4de81e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d80cd5b0 0x7ff4d8150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:46.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4de81e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d81600a0 0x7ff4d80c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:46.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4de81e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d814fcd0 con 0x7ff4d80cd5b0 2024-03-23T09:43:46.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4de81e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d80d25d0 con 0x7ff4d80b5a10 2024-03-23T09:43:46.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4de81e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d8156940 con 0x7ff4d81600a0 2024-03-23T09:43:46.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4d77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d80cd5b0 0x7ff4d8150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:46.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4d77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d80cd5b0 0x7ff4d8150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38936/0 (socket says 172.21.15.47:38936) 2024-03-23T09:43:46.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4d77fe640 1 -- 172.21.15.47:0/2944986350 learned_addr learned my addr 172.21.15.47:0/2944986350 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:46.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4d7fff640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4d80b5a10 0x7ff4d81562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:46.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4dcdbb640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d81600a0 0x7ff4d80c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:46.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4d77fe640 1 -- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d81600a0 msgr2=0x7ff4d80c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:46.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4d77fe640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d81600a0 0x7ff4d80c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:46.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4d77fe640 1 -- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4d80b5a10 msgr2=0x7ff4d81562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:46.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4d77fe640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4d80b5a10 0x7ff4d81562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:46.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.141+0000 7ff4d77fe640 1 -- 172.21.15.47:0/2944986350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c80094d0 con 0x7ff4d80cd5b0 2024-03-23T09:43:46.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.142+0000 7ff4d77fe640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d80cd5b0 0x7ff4d8150120 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009870 tx=0x7ff4c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:46.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.142+0000 7ff4d4ff9640 1 -- 172.21.15.47:0/2944986350 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c803ac60 con 0x7ff4d80cd5b0 2024-03-23T09:43:46.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.142+0000 7ff4d4ff9640 1 -- 172.21.15.47:0/2944986350 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c803adc0 con 0x7ff4d80cd5b0 2024-03-23T09:43:46.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.142+0000 7ff4de81e640 1 -- 172.21.15.47:0/2944986350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d80cc160 con 0x7ff4d80cd5b0 2024-03-23T09:43:46.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.142+0000 7ff4d4ff9640 1 -- 172.21.15.47:0/2944986350 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8044630 con 0x7ff4d80cd5b0 2024-03-23T09:43:46.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.142+0000 7ff4de81e640 1 -- 172.21.15.47:0/2944986350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d80cc720 con 0x7ff4d80cd5b0 2024-03-23T09:43:46.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.143+0000 7ff4de81e640 1 -- 172.21.15.47:0/2944986350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d8150660 con 0x7ff4d80cd5b0 2024-03-23T09:43:46.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.143+0000 7ff4d4ff9640 1 -- 172.21.15.47:0/2944986350 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff4c802f050 con 0x7ff4d80cd5b0 2024-03-23T09:43:46.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.144+0000 7ff4d4ff9640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff4b806ac00 0x7ff4b806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:46.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.144+0000 7ff4d4ff9640 1 -- 172.21.15.47:0/2944986350 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff4c80aaf20 con 0x7ff4d80cd5b0 2024-03-23T09:43:46.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.144+0000 7ff4d7fff640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff4b806ac00 0x7ff4b806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:46.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.145+0000 7ff4d7fff640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff4b806ac00 0x7ff4b806d0c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0006fd0 tx=0x7ff4c0009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:46.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.148+0000 7ff4d4ff9640 1 -- 172.21.15.47:0/2944986350 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff4c80764b0 con 0x7ff4d80cd5b0 2024-03-23T09:43:46.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:46.331+0000 7ff4de81e640 1 -- 172.21.15.47:0/2944986350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff4d8072360 con 0x7ff4d80cd5b0 2024-03-23T09:43:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.016+0000 7ff4d4ff9640 1 -- 172.21.15.47:0/2944986350 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff4c8031e60 con 0x7ff4d80cd5b0 2024-03-23T09:43:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-03-23T09:43:47.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.021+0000 7ff4de81e640 1 -- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff4b806ac00 msgr2=0x7ff4b806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.021+0000 7ff4de81e640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff4b806ac00 0x7ff4b806d0c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0006fd0 tx=0x7ff4c0009060 comp rx=0 tx=0).stop 2024-03-23T09:43:47.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.021+0000 7ff4de81e640 1 -- 172.21.15.47:0/2944986350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d80cd5b0 msgr2=0x7ff4d8150120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.021+0000 7ff4de81e640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d80cd5b0 0x7ff4d8150120 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009870 tx=0x7ff4c80046c0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.024+0000 7ff4de81e640 1 -- 172.21.15.47:0/2944986350 shutdown_connections 2024-03-23T09:43:47.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.024+0000 7ff4de81e640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff4b806ac00 0x7ff4b806d0c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.024+0000 7ff4de81e640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d81600a0 0x7ff4d80c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.024+0000 7ff4de81e640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d80cd5b0 0x7ff4d8150120 secure :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009870 tx=0x7ff4c80046c0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.024+0000 7ff4de81e640 1 --2- 172.21.15.47:0/2944986350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4d80b5a10 0x7ff4d81562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.024+0000 7ff4de81e640 1 -- 172.21.15.47:0/2944986350 >> 172.21.15.47:0/2944986350 conn(0x7ff4d806afd0 msgr2=0x7ff4d80b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:47.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.029+0000 7ff4de81e640 1 -- 172.21.15.47:0/2944986350 shutdown_connections 2024-03-23T09:43:47.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.029+0000 7ff4de81e640 1 -- 172.21.15.47:0/2944986350 wait complete. 2024-03-23T09:43:47.049 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T09:43:47.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.156+0000 7fe72a7d9640 1 Processor -- start 2024-03-23T09:43:47.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.156+0000 7fe72a7d9640 1 -- start start 2024-03-23T09:43:47.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.157+0000 7fe72a7d9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe72415c990 0x7fe724160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.157+0000 7fe72a7d9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7240cde10 0x7fe7240d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.157+0000 7fe72a7d9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe724161630 con 0x7fe7240b8350 2024-03-23T09:43:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.157+0000 7fe72a7d9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7240d1030 con 0x7fe7240cde10 2024-03-23T09:43:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.157+0000 7fe72a7d9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7240cfe10 con 0x7fe72415c990 2024-03-23T09:43:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.157+0000 7fe728d76640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7240cde10 0x7fe7240d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.157+0000 7fe728d76640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7240cde10 0x7fe7240d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38944/0 (socket says 172.21.15.47:38944) 2024-03-23T09:43:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.157+0000 7fe7237fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe7240b8350 0x7fe7240cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36002/0 (socket says 172.21.15.47:36002) 2024-03-23T09:43:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.157+0000 7fe728d76640 1 -- 172.21.15.47:0/3461936462 learned_addr learned my addr 172.21.15.47:0/3461936462 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.158+0000 7fe723fff640 1 --2- 172.21.15.47:0/3461936462 >> v2:172.21.15.175:3300/0 conn(0x7fe72415c990 0x7fe724160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.158+0000 7fe728d76640 1 -- 172.21.15.47:0/3461936462 >> v2:172.21.15.175:3300/0 conn(0x7fe72415c990 msgr2=0x7fe724160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.158+0000 7fe728d76640 1 --2- 172.21.15.47:0/3461936462 >> v2:172.21.15.175:3300/0 conn(0x7fe72415c990 0x7fe724160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.158+0000 7fe728d76640 1 -- 172.21.15.47:0/3461936462 >> v1:172.21.15.47:6789/0 conn(0x7fe7240b8350 legacy=0x7fe7240cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.158+0000 7fe728d76640 1 -- 172.21.15.47:0/3461936462 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7240b5b60 con 0x7fe7240cde10 2024-03-23T09:43:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.158+0000 7fe723fff640 1 --2- 172.21.15.47:0/3461936462 >> v2:172.21.15.175:3300/0 conn(0x7fe72415c990 0x7fe724160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.158+0000 7fe728d76640 1 --2- 172.21.15.47:0/3461936462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7240cde10 0x7fe7240d14c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe7240b8b30 tx=0x7fe71c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3a38beb3e1f912 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.159+0000 7fe722ffd640 1 -- 172.21.15.47:0/3461936462 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c02ca80 con 0x7fe7240cde10 2024-03-23T09:43:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.159+0000 7fe722ffd640 1 -- 172.21.15.47:0/3461936462 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c02cbe0 con 0x7fe7240cde10 2024-03-23T09:43:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.159+0000 7fe722ffd640 1 -- 172.21.15.47:0/3461936462 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c0343f0 con 0x7fe7240cde10 2024-03-23T09:43:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.159+0000 7fe72a7d9640 1 -- 172.21.15.47:0/3461936462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7240cde10 msgr2=0x7fe7240d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.159+0000 7fe72a7d9640 1 --2- 172.21.15.47:0/3461936462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7240cde10 0x7fe7240d14c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe7240b8b30 tx=0x7fe71c02bee0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.159+0000 7fe72a7d9640 1 -- 172.21.15.47:0/3461936462 shutdown_connections 2024-03-23T09:43:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.159+0000 7fe72a7d9640 1 --2- 172.21.15.47:0/3461936462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7240cde10 0x7fe7240d14c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.159+0000 7fe72a7d9640 1 --2- 172.21.15.47:0/3461936462 >> v2:172.21.15.175:3300/0 conn(0x7fe72415c990 0x7fe724160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.159+0000 7fe72a7d9640 1 -- 172.21.15.47:0/3461936462 >> 172.21.15.47:0/3461936462 conn(0x7fe7240659d0 msgr2=0x7fe724065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.159+0000 7fe72a7d9640 1 -- 172.21.15.47:0/3461936462 shutdown_connections 2024-03-23T09:43:47.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.159+0000 7fe72a7d9640 1 -- 172.21.15.47:0/3461936462 wait complete. 2024-03-23T09:43:47.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.160+0000 7fe72a7d9640 1 Processor -- start 2024-03-23T09:43:47.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.160+0000 7fe72a7d9640 1 -- start start 2024-03-23T09:43:47.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.161+0000 7fe72a7d9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7240b8350 0x7fe724155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.161+0000 7fe72a7d9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7240cde10 0x7fe7240c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.161+0000 7fe72a7d9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72415c990 0x7fe724157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.161+0000 7fe72a7d9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe724156440 con 0x7fe72415c990 2024-03-23T09:43:47.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.161+0000 7fe72a7d9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7241f5eb0 con 0x7fe7240cde10 2024-03-23T09:43:47.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.161+0000 7fe72a7d9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7241f31a0 con 0x7fe7240b8350 2024-03-23T09:43:47.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.161+0000 7fe728d76640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72415c990 0x7fe724157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.161+0000 7fe728d76640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72415c990 0x7fe724157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38960/0 (socket says 172.21.15.47:38960) 2024-03-23T09:43:47.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.161+0000 7fe728d76640 1 -- 172.21.15.47:0/1596441122 learned_addr learned my addr 172.21.15.47:0/1596441122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:47.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.161+0000 7fe7237fe640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7240cde10 0x7fe7240c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.161+0000 7fe723fff640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7240b8350 0x7fe724155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.162+0000 7fe728d76640 1 -- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7240b8350 msgr2=0x7fe724155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.162+0000 7fe728d76640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7240b8350 0x7fe724155f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.162+0000 7fe728d76640 1 -- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7240cde10 msgr2=0x7fe7240c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.162+0000 7fe728d76640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7240cde10 0x7fe7240c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.162+0000 7fe728d76640 1 -- 172.21.15.47:0/1596441122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c009b30 con 0x7fe72415c990 2024-03-23T09:43:47.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.162+0000 7fe7237fe640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7240cde10 0x7fe7240c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:47.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.162+0000 7fe728d76640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72415c990 0x7fe724157050 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009f40 tx=0x7fe71c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:47.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.162+0000 7fe720ff9640 1 -- 172.21.15.47:0/1596441122 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c009510 con 0x7fe72415c990 2024-03-23T09:43:47.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.162+0000 7fe720ff9640 1 -- 172.21.15.47:0/1596441122 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c03e450 con 0x7fe72415c990 2024-03-23T09:43:47.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.162+0000 7fe72a7d9640 1 -- 172.21.15.47:0/1596441122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7241f3480 con 0x7fe72415c990 2024-03-23T09:43:47.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.162+0000 7fe720ff9640 1 -- 172.21.15.47:0/1596441122 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c03e800 con 0x7fe72415c990 2024-03-23T09:43:47.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.162+0000 7fe72a7d9640 1 -- 172.21.15.47:0/1596441122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7240c6710 con 0x7fe72415c990 2024-03-23T09:43:47.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.163+0000 7fe72a7d9640 1 -- 172.21.15.47:0/1596441122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe724072360 con 0x7fe72415c990 2024-03-23T09:43:47.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.164+0000 7fe720ff9640 1 -- 172.21.15.47:0/1596441122 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe71c02cd50 con 0x7fe72415c990 2024-03-23T09:43:47.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.164+0000 7fe720ff9640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe71006ac00 0x7fe71006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.164+0000 7fe720ff9640 1 -- 172.21.15.47:0/1596441122 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe71c0abf20 con 0x7fe72415c990 2024-03-23T09:43:47.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.164+0000 7fe723fff640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe71006ac00 0x7fe71006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.165+0000 7fe723fff640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe71006ac00 0x7fe71006d0c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe71400b840 tx=0x7fe714008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:47.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.169+0000 7fe720ff9640 1 -- 172.21.15.47:0/1596441122 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe71c077160 con 0x7fe72415c990 2024-03-23T09:43:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.386+0000 7fe72a7d9640 1 -- 172.21.15.47:0/1596441122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe724066be0 con 0x7fe72415c990 2024-03-23T09:43:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.392+0000 7fe720ff9640 1 -- 172.21.15.47:0/1596441122 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe71c07a780 con 0x7fe72415c990 2024-03-23T09:43:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 8 and data pool 9 2024-03-23T09:43:47.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.401+0000 7fe72a7d9640 1 -- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe71006ac00 msgr2=0x7fe71006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.401+0000 7fe72a7d9640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe71006ac00 0x7fe71006d0c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe71400b840 tx=0x7fe714008040 comp rx=0 tx=0).stop 2024-03-23T09:43:47.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.401+0000 7fe72a7d9640 1 -- 172.21.15.47:0/1596441122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72415c990 msgr2=0x7fe724157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.401+0000 7fe72a7d9640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72415c990 0x7fe724157050 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009f40 tx=0x7fe71c004a40 comp rx=0 tx=0).stop 2024-03-23T09:43:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.402+0000 7fe72a7d9640 1 -- 172.21.15.47:0/1596441122 shutdown_connections 2024-03-23T09:43:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.402+0000 7fe72a7d9640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe71006ac00 0x7fe71006d0c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.402+0000 7fe72a7d9640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72415c990 0x7fe724157050 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.402+0000 7fe72a7d9640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7240cde10 0x7fe7240c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.402+0000 7fe72a7d9640 1 --2- 172.21.15.47:0/1596441122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7240b8350 0x7fe724155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.402+0000 7fe72a7d9640 1 -- 172.21.15.47:0/1596441122 >> 172.21.15.47:0/1596441122 conn(0x7fe7240659d0 msgr2=0x7fe7240d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.406+0000 7fe72a7d9640 1 -- 172.21.15.47:0/1596441122 shutdown_connections 2024-03-23T09:43:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.406+0000 7fe72a7d9640 1 -- 172.21.15.47:0/1596441122 wait complete. 2024-03-23T09:43:47.425 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:43:47.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57ee4ab640 1 Processor -- start 2024-03-23T09:43:47.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57ee4ab640 1 -- start start 2024-03-23T09:43:47.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57ee4ab640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f57e815ca90 0x7f57e8160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57ee4ab640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f57e80cde10 0x7f57e80d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57ee4ab640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e80d11e0 con 0x7f57e80b8350 2024-03-23T09:43:47.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57ee4ab640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e8160f60 con 0x7f57e815ca90 2024-03-23T09:43:47.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57ee4ab640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e8161240 con 0x7f57e80cde10 2024-03-23T09:43:47.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57e7fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f57e815ca90 0x7f57e8160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57e7fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f57e815ca90 0x7f57e8160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38972/0 (socket says 172.21.15.47:38972) 2024-03-23T09:43:47.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57e7fff640 1 -- 172.21.15.47:0/2803280725 learned_addr learned my addr 172.21.15.47:0/2803280725 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:47.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57eca48640 1 --2- 172.21.15.47:0/2803280725 >> v2:172.21.15.175:3300/0 conn(0x7f57e80cde10 0x7f57e80d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57e7fff640 1 -- 172.21.15.47:0/2803280725 >> v2:172.21.15.175:3300/0 conn(0x7f57e80cde10 msgr2=0x7f57e80d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:47.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57e7fff640 1 --2- 172.21.15.47:0/2803280725 >> v2:172.21.15.175:3300/0 conn(0x7f57e80cde10 0x7f57e80d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57e7fff640 1 -- 172.21.15.47:0/2803280725 >> v1:172.21.15.47:6789/0 conn(0x7f57e80b8350 legacy=0x7f57e80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57e7fff640 1 -- 172.21.15.47:0/2803280725 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e8161520 con 0x7f57e815ca90 2024-03-23T09:43:47.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57e6ffd640 1 -- 172.21.15.47:0/2803280725 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782999803 0 0) 0x7f57e80d11e0 con 0x7f57e80b8350 2024-03-23T09:43:47.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.526+0000 7f57eca48640 1 --2- 172.21.15.47:0/2803280725 >> v2:172.21.15.175:3300/0 conn(0x7f57e80cde10 0x7f57e80d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:47.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.527+0000 7f57e7fff640 1 --2- 172.21.15.47:0/2803280725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e815ca90 0x7f57e8160710 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f57e80b5930 tx=0x7f57dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81e29fb88368ed6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:47.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.527+0000 7f57e6ffd640 1 -- 172.21.15.47:0/2803280725 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc031870 con 0x7f57e815ca90 2024-03-23T09:43:47.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.527+0000 7f57e6ffd640 1 -- 172.21.15.47:0/2803280725 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc0319d0 con 0x7f57e815ca90 2024-03-23T09:43:47.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.527+0000 7f57e6ffd640 1 -- 172.21.15.47:0/2803280725 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc031d60 con 0x7f57e815ca90 2024-03-23T09:43:47.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.528+0000 7f57ee4ab640 1 -- 172.21.15.47:0/2803280725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e815ca90 msgr2=0x7f57e8160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.528+0000 7f57ee4ab640 1 --2- 172.21.15.47:0/2803280725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e815ca90 0x7f57e8160710 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f57e80b5930 tx=0x7f57dc02b910 comp rx=0 tx=0).stop 2024-03-23T09:43:47.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.528+0000 7f57ee4ab640 1 -- 172.21.15.47:0/2803280725 shutdown_connections 2024-03-23T09:43:47.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.528+0000 7f57ee4ab640 1 --2- 172.21.15.47:0/2803280725 >> v2:172.21.15.175:3300/0 conn(0x7f57e80cde10 0x7f57e80d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.528+0000 7f57ee4ab640 1 --2- 172.21.15.47:0/2803280725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e815ca90 0x7f57e8160710 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.528+0000 7f57ee4ab640 1 -- 172.21.15.47:0/2803280725 >> 172.21.15.47:0/2803280725 conn(0x7f57e80659d0 msgr2=0x7f57e8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:47.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.528+0000 7f57ee4ab640 1 -- 172.21.15.47:0/2803280725 shutdown_connections 2024-03-23T09:43:47.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.528+0000 7f57ee4ab640 1 -- 172.21.15.47:0/2803280725 wait complete. 2024-03-23T09:43:47.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.529+0000 7f57ee4ab640 1 Processor -- start 2024-03-23T09:43:47.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.529+0000 7f57ee4ab640 1 -- start start 2024-03-23T09:43:47.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57ee4ab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80b8350 0x7f57e8158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57ee4ab640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57e80cde10 0x7f57e8153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57ee4ab640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e815ca90 0x7f57e815d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57ee4ab640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80cc8c0 con 0x7f57e815ca90 2024-03-23T09:43:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57ee4ab640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80cc5e0 con 0x7f57e80cde10 2024-03-23T09:43:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57ee4ab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80b2f60 con 0x7f57e80b8350 2024-03-23T09:43:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57eca48640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e815ca90 0x7f57e815d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57eca48640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e815ca90 0x7f57e815d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38974/0 (socket says 172.21.15.47:38974) 2024-03-23T09:43:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57eca48640 1 -- 172.21.15.47:0/321983765 learned_addr learned my addr 172.21.15.47:0/321983765 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57e77fe640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57e80cde10 0x7f57e8153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57e7fff640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80b8350 0x7f57e8158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57eca48640 1 -- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80b8350 msgr2=0x7f57e8158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57eca48640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80b8350 0x7f57e8158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57eca48640 1 -- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57e80cde10 msgr2=0x7f57e8153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57eca48640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57e80cde10 0x7f57e8153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57eca48640 1 -- 172.21.15.47:0/321983765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc0094d0 con 0x7f57e815ca90 2024-03-23T09:43:47.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57eca48640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e815ca90 0x7f57e815d270 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f57e000ce80 tx=0x7f57e000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:47.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.530+0000 7f57e4ff9640 1 -- 172.21.15.47:0/321983765 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000d840 con 0x7f57e815ca90 2024-03-23T09:43:47.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.531+0000 7f57e4ff9640 1 -- 172.21.15.47:0/321983765 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e000d9e0 con 0x7f57e815ca90 2024-03-23T09:43:47.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.531+0000 7f57ee4ab640 1 -- 172.21.15.47:0/321983765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e80b3240 con 0x7f57e815ca90 2024-03-23T09:43:47.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.531+0000 7f57e4ff9640 1 -- 172.21.15.47:0/321983765 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000dd90 con 0x7f57e815ca90 2024-03-23T09:43:47.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.531+0000 7f57ee4ab640 1 -- 172.21.15.47:0/321983765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e8159180 con 0x7f57e815ca90 2024-03-23T09:43:47.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.531+0000 7f57ee4ab640 1 -- 172.21.15.47:0/321983765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e8072360 con 0x7f57e815ca90 2024-03-23T09:43:47.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.532+0000 7f57e4ff9640 1 -- 172.21.15.47:0/321983765 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f57e0015440 con 0x7f57e815ca90 2024-03-23T09:43:47.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.532+0000 7f57e4ff9640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f57d406ac00 0x7f57d406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.532+0000 7f57e4ff9640 1 -- 172.21.15.47:0/321983765 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57e0089c80 con 0x7f57e815ca90 2024-03-23T09:43:47.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.533+0000 7f57e7fff640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f57d406ac00 0x7f57d406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.533+0000 7f57e7fff640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f57d406ac00 0x7f57d406d0c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f57dc036000 tx=0x7f57dc03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:47.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.537+0000 7f57e4ff9640 1 -- 172.21.15.47:0/321983765 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f57e0054f00 con 0x7f57e815ca90 2024-03-23T09:43:47.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.717+0000 7f57ee4ab640 1 -- 172.21.15.47:0/321983765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57e80b3760 con 0x7f57e815ca90 2024-03-23T09:43:47.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.718+0000 7f57e4ff9640 1 -- 172.21.15.47:0/321983765 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16110 (secure 0 0 0) 0x7f57e80b3760 con 0x7f57e815ca90 2024-03-23T09:43:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:47.718 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":57,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:43:47.388988+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:43:45.646160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-23T09:43:46.351634+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:43:47.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.720+0000 7f57ee4ab640 1 -- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f57d406ac00 msgr2=0x7f57d406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.720+0000 7f57ee4ab640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f57d406ac00 0x7f57d406d0c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f57dc036000 tx=0x7f57dc03c040 comp rx=0 tx=0).stop 2024-03-23T09:43:47.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.720+0000 7f57ee4ab640 1 -- 172.21.15.47:0/321983765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e815ca90 msgr2=0x7f57e815d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.720+0000 7f57ee4ab640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e815ca90 0x7f57e815d270 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f57e000ce80 tx=0x7f57e000a710 comp rx=0 tx=0).stop 2024-03-23T09:43:47.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.720+0000 7f57ee4ab640 1 -- 172.21.15.47:0/321983765 shutdown_connections 2024-03-23T09:43:47.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.720+0000 7f57ee4ab640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f57d406ac00 0x7f57d406d0c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.720+0000 7f57ee4ab640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e815ca90 0x7f57e815d270 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.720+0000 7f57ee4ab640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57e80cde10 0x7f57e8153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.720+0000 7f57ee4ab640 1 --2- 172.21.15.47:0/321983765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80b8350 0x7f57e8158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.720+0000 7f57ee4ab640 1 -- 172.21.15.47:0/321983765 >> 172.21.15.47:0/321983765 conn(0x7f57e80659d0 msgr2=0x7f57e80b6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:47.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.720+0000 7f57ee4ab640 1 -- 172.21.15.47:0/321983765 shutdown_connections 2024-03-23T09:43:47.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.720+0000 7f57ee4ab640 1 -- 172.21.15.47:0/321983765 wait complete. 2024-03-23T09:43:47.731 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:43:47.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.833+0000 7fcb09b5c640 1 Processor -- start 2024-03-23T09:43:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.833+0000 7fcb09b5c640 1 -- start start 2024-03-23T09:43:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.833+0000 7fcb09b5c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcb0415aa70 0x7fcb040cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.833+0000 7fcb09b5c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb040ce9d0 0x7fcb040cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.833+0000 7fcb09b5c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0415ae50 con 0x7fcb040cd620 2024-03-23T09:43:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.833+0000 7fcb09b5c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb040cda90 con 0x7fcb040ce9d0 2024-03-23T09:43:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.833+0000 7fcb09b5c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb040cf010 con 0x7fcb0415aa70 2024-03-23T09:43:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.833+0000 7fcb03fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb040ce9d0 0x7fcb040cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.834+0000 7fcb03fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb040ce9d0 0x7fcb040cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38984/0 (socket says 172.21.15.47:38984) 2024-03-23T09:43:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.834+0000 7fcb03fff640 1 -- 172.21.15.47:0/1059382393 learned_addr learned my addr 172.21.15.47:0/1059382393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.834+0000 7fcb037fe640 1 --2- 172.21.15.47:0/1059382393 >> v2:172.21.15.175:3300/0 conn(0x7fcb0415aa70 0x7fcb040cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.834+0000 7fcb03fff640 1 -- 172.21.15.47:0/1059382393 >> v2:172.21.15.175:3300/0 conn(0x7fcb0415aa70 msgr2=0x7fcb040cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.834+0000 7fcb03fff640 1 --2- 172.21.15.47:0/1059382393 >> v2:172.21.15.175:3300/0 conn(0x7fcb0415aa70 0x7fcb040cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.834+0000 7fcb03fff640 1 -- 172.21.15.47:0/1059382393 >> v1:172.21.15.175:6789/0 conn(0x7fcb040cd620 legacy=0x7fcb040ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.834+0000 7fcb03fff640 1 -- 172.21.15.47:0/1059382393 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040cf2f0 con 0x7fcb040ce9d0 2024-03-23T09:43:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.835+0000 7fcb03fff640 1 --2- 172.21.15.47:0/1059382393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb040ce9d0 0x7fcb040cbb70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcb040cdcc0 tx=0x7fcafc02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35af91715c2dd9ad server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.835+0000 7fcb027fc640 1 -- 172.21.15.47:0/1059382393 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc02cbb0 con 0x7fcb040ce9d0 2024-03-23T09:43:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.835+0000 7fcb027fc640 1 -- 172.21.15.47:0/1059382393 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc02cd10 con 0x7fcb040ce9d0 2024-03-23T09:43:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.835+0000 7fcb027fc640 1 -- 172.21.15.47:0/1059382393 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc034610 con 0x7fcb040ce9d0 2024-03-23T09:43:47.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.835+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1059382393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb040ce9d0 msgr2=0x7fcb040cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.835+0000 7fcb09b5c640 1 --2- 172.21.15.47:0/1059382393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb040ce9d0 0x7fcb040cbb70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcb040cdcc0 tx=0x7fcafc02c010 comp rx=0 tx=0).stop 2024-03-23T09:43:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.835+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1059382393 shutdown_connections 2024-03-23T09:43:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.835+0000 7fcb09b5c640 1 --2- 172.21.15.47:0/1059382393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb040ce9d0 0x7fcb040cbb70 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.835+0000 7fcb09b5c640 1 --2- 172.21.15.47:0/1059382393 >> v2:172.21.15.175:3300/0 conn(0x7fcb0415aa70 0x7fcb040cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.835+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1059382393 >> 172.21.15.47:0/1059382393 conn(0x7fcb040d06f0 msgr2=0x7fcb040d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.835+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1059382393 shutdown_connections 2024-03-23T09:43:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.835+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1059382393 wait complete. 2024-03-23T09:43:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.836+0000 7fcb09b5c640 1 Processor -- start 2024-03-23T09:43:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.837+0000 7fcb09b5c640 1 -- start start 2024-03-23T09:43:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.837+0000 7fcb09b5c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb040cd620 0x7fcb041f4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.837+0000 7fcb09b5c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb040ce9d0 0x7fcb04149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.837+0000 7fcb09b5c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb0415aa70 0x7fcb040b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.837+0000 7fcb09b5c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb041497a0 con 0x7fcb0415aa70 2024-03-23T09:43:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.837+0000 7fcb09b5c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040b5440 con 0x7fcb040cd620 2024-03-23T09:43:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.837+0000 7fcb09b5c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb041f0ab0 con 0x7fcb040ce9d0 2024-03-23T09:43:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.837+0000 7fcb03fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb0415aa70 0x7fcb040b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.837+0000 7fcb03fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb0415aa70 0x7fcb040b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39000/0 (socket says 172.21.15.47:39000) 2024-03-23T09:43:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.837+0000 7fcb03fff640 1 -- 172.21.15.47:0/1888281669 learned_addr learned my addr 172.21.15.47:0/1888281669 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcb037fe640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb040cd620 0x7fcb041f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcb02ffd640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb040ce9d0 0x7fcb04149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcb03fff640 1 -- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb040ce9d0 msgr2=0x7fcb04149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcb03fff640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb040ce9d0 0x7fcb04149260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcb03fff640 1 -- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb040cd620 msgr2=0x7fcb041f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcb03fff640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb040cd620 0x7fcb041f4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcb03fff640 1 -- 172.21.15.47:0/1888281669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc009b30 con 0x7fcb0415aa70 2024-03-23T09:43:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcb02ffd640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb040ce9d0 0x7fcb04149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:47.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcb03fff640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb0415aa70 0x7fcb040b32e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcafc036040 tx=0x7fcafc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:47.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcaebfff640 1 -- 172.21.15.47:0/1888281669 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc009510 con 0x7fcb0415aa70 2024-03-23T09:43:47.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcaebfff640 1 -- 172.21.15.47:0/1888281669 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc03e450 con 0x7fcb0415aa70 2024-03-23T09:43:47.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcaebfff640 1 -- 172.21.15.47:0/1888281669 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc03e800 con 0x7fcb0415aa70 2024-03-23T09:43:47.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1888281669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb041f0d90 con 0x7fcb0415aa70 2024-03-23T09:43:47.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.838+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1888281669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb040b3820 con 0x7fcb0415aa70 2024-03-23T09:43:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.839+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1888281669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb040b55b0 con 0x7fcb0415aa70 2024-03-23T09:43:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.840+0000 7fcaebfff640 1 -- 172.21.15.47:0/1888281669 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcafc03e960 con 0x7fcb0415aa70 2024-03-23T09:43:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.840+0000 7fcaebfff640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcaf006ac00 0x7fcaf006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.840+0000 7fcaebfff640 1 -- 172.21.15.47:0/1888281669 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcafc0ac030 con 0x7fcb0415aa70 2024-03-23T09:43:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.840+0000 7fcb037fe640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcaf006ac00 0x7fcaf006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.841+0000 7fcb037fe640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcaf006ac00 0x7fcaf006d0c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009a30 tx=0x7fcaf4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:47.844+0000 7fcaebfff640 1 -- 172.21.15.47:0/1888281669 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcafc0772a0 con 0x7fcb0415aa70 2024-03-23T09:43:48.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.025+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1888281669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb04066be0 con 0x7fcb0415aa70 2024-03-23T09:43:48.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.026+0000 7fcaebfff640 1 -- 172.21.15.47:0/1888281669 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16110 (secure 0 0 0) 0x7fcafc07a8c0 con 0x7fcb0415aa70 2024-03-23T09:43:48.026 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:48.026 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":57,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:43:47.388988+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:43:45.646160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-23T09:43:46.351634+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:43:48.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.028+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcaf006ac00 msgr2=0x7fcaf006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:48.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.028+0000 7fcb09b5c640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcaf006ac00 0x7fcaf006d0c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009a30 tx=0x7fcaf4008040 comp rx=0 tx=0).stop 2024-03-23T09:43:48.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.028+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1888281669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb0415aa70 msgr2=0x7fcb040b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:48.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.028+0000 7fcb09b5c640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb0415aa70 0x7fcb040b32e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcafc036040 tx=0x7fcafc004a40 comp rx=0 tx=0).stop 2024-03-23T09:43:48.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.028+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1888281669 shutdown_connections 2024-03-23T09:43:48.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.028+0000 7fcb09b5c640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcaf006ac00 0x7fcaf006d0c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.028+0000 7fcb09b5c640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb0415aa70 0x7fcb040b32e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.028+0000 7fcb09b5c640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb040ce9d0 0x7fcb04149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.028+0000 7fcb09b5c640 1 --2- 172.21.15.47:0/1888281669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb040cd620 0x7fcb041f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.028+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1888281669 >> 172.21.15.47:0/1888281669 conn(0x7fcb040d06f0 msgr2=0x7fcb040cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:48.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.028+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1888281669 shutdown_connections 2024-03-23T09:43:48.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.028+0000 7fcb09b5c640 1 -- 172.21.15.47:0/1888281669 wait complete. 2024-03-23T09:43:48.039 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T09:43:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.143+0000 7fb8681fb640 1 Processor -- start 2024-03-23T09:43:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.143+0000 7fb8681fb640 1 -- start start 2024-03-23T09:43:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb8681fb640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb86015cd40 0x7fb8600d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb8681fb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb860099110 0x7fb8600ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb8681fb640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8600cedd0 con 0x7fb86006b090 2024-03-23T09:43:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb8681fb640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb86015d120 con 0x7fb860099110 2024-03-23T09:43:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb8681fb640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb860099760 con 0x7fb86015cd40 2024-03-23T09:43:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb866798640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb860099110 0x7fb8600ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb865796640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb86006b090 0x7fb860098a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36008/0 (socket says 172.21.15.47:36008) 2024-03-23T09:43:48.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb866798640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb860099110 0x7fb8600ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39008/0 (socket says 172.21.15.47:39008) 2024-03-23T09:43:48.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb865796640 1 -- 172.21.15.47:0/438588211 learned_addr learned my addr 172.21.15.47:0/438588211 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:48.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb866798640 1 -- 172.21.15.47:0/438588211 >> v2:172.21.15.175:3300/0 conn(0x7fb86015cd40 msgr2=0x7fb8600d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:48.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb865f97640 1 --2- 172.21.15.47:0/438588211 >> v2:172.21.15.175:3300/0 conn(0x7fb86015cd40 0x7fb8600d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:48.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb866798640 1 --2- 172.21.15.47:0/438588211 >> v2:172.21.15.175:3300/0 conn(0x7fb86015cd40 0x7fb8600d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb864f95640 1 -- 172.21.15.47:0/438588211 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928268245 0 0) 0x7fb8600cedd0 con 0x7fb86006b090 2024-03-23T09:43:48.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb866798640 1 -- 172.21.15.47:0/438588211 >> v1:172.21.15.47:6789/0 conn(0x7fb86006b090 legacy=0x7fb860098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:48.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.144+0000 7fb866798640 1 -- 172.21.15.47:0/438588211 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8600cf020 con 0x7fb860099110 2024-03-23T09:43:48.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.145+0000 7fb866798640 1 --2- 172.21.15.47:0/438588211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860099110 0x7fb8600ce890 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb86006e120 tx=0x7fb85802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872137748efcfb54 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:48.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.145+0000 7fb864f95640 1 -- 172.21.15.47:0/438588211 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858031830 con 0x7fb860099110 2024-03-23T09:43:48.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.145+0000 7fb864f95640 1 -- 172.21.15.47:0/438588211 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb858031990 con 0x7fb860099110 2024-03-23T09:43:48.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.145+0000 7fb864f95640 1 -- 172.21.15.47:0/438588211 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858031d20 con 0x7fb860099110 2024-03-23T09:43:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.146+0000 7fb8681fb640 1 -- 172.21.15.47:0/438588211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860099110 msgr2=0x7fb8600ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.146+0000 7fb8681fb640 1 --2- 172.21.15.47:0/438588211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860099110 0x7fb8600ce890 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb86006e120 tx=0x7fb85802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.146+0000 7fb8681fb640 1 -- 172.21.15.47:0/438588211 shutdown_connections 2024-03-23T09:43:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.146+0000 7fb8681fb640 1 --2- 172.21.15.47:0/438588211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860099110 0x7fb8600ce890 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.146+0000 7fb8681fb640 1 --2- 172.21.15.47:0/438588211 >> v2:172.21.15.175:3300/0 conn(0x7fb86015cd40 0x7fb8600d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.146+0000 7fb8681fb640 1 -- 172.21.15.47:0/438588211 >> 172.21.15.47:0/438588211 conn(0x7fb8600b2140 msgr2=0x7fb8600b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.146+0000 7fb8681fb640 1 -- 172.21.15.47:0/438588211 shutdown_connections 2024-03-23T09:43:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.146+0000 7fb8681fb640 1 -- 172.21.15.47:0/438588211 wait complete. 2024-03-23T09:43:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.147+0000 7fb8681fb640 1 Processor -- start 2024-03-23T09:43:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.147+0000 7fb8681fb640 1 -- start start 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb8681fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb86006b090 0x7fb86009d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb8681fb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860099110 0x7fb86009d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb8681fb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb86015cd40 0x7fb8600b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb8681fb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600b51c0 con 0x7fb860099110 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb8681fb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600b54a0 con 0x7fb86015cd40 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb8681fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600b5780 con 0x7fb86006b090 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb865796640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860099110 0x7fb86009d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb865796640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860099110 0x7fb86009d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39012/0 (socket says 172.21.15.47:39012) 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb865796640 1 -- 172.21.15.47:0/4091222011 learned_addr learned my addr 172.21.15.47:0/4091222011 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb865796640 1 -- 172.21.15.47:0/4091222011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb86006b090 msgr2=0x7fb86009d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb865796640 1 --2- 172.21.15.47:0/4091222011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb86006b090 0x7fb86009d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb865796640 1 -- 172.21.15.47:0/4091222011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb86015cd40 msgr2=0x7fb8600b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb865796640 1 --2- 172.21.15.47:0/4091222011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb86015cd40 0x7fb8600b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb865796640 1 -- 172.21.15.47:0/4091222011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb858009400 con 0x7fb860099110 2024-03-23T09:43:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb865796640 1 --2- 172.21.15.47:0/4091222011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860099110 0x7fb86009d710 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb85400c430 tx=0x7fb85400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb85e7fc640 1 -- 172.21.15.47:0/4091222011 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8540074e0 con 0x7fb860099110 2024-03-23T09:43:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb85e7fc640 1 -- 172.21.15.47:0/4091222011 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb854007660 con 0x7fb860099110 2024-03-23T09:43:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb8681fb640 1 -- 172.21.15.47:0/4091222011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8600b5a60 con 0x7fb860099110 2024-03-23T09:43:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb85e7fc640 1 -- 172.21.15.47:0/4091222011 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854004a20 con 0x7fb860099110 2024-03-23T09:43:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.148+0000 7fb8681fb640 1 -- 172.21.15.47:0/4091222011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8600b18f0 con 0x7fb860099110 2024-03-23T09:43:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.149+0000 7fb8681fb640 1 -- 172.21.15.47:0/4091222011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb860066be0 con 0x7fb860099110 2024-03-23T09:43:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.150+0000 7fb85e7fc640 1 -- 172.21.15.47:0/4091222011 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb854004b80 con 0x7fb860099110 2024-03-23T09:43:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.150+0000 7fb85e7fc640 1 --2- 172.21.15.47:0/4091222011 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb84c06ac00 0x7fb84c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.150+0000 7fb85e7fc640 1 -- 172.21.15.47:0/4091222011 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb854089170 con 0x7fb860099110 2024-03-23T09:43:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.151+0000 7fb865f97640 1 --2- 172.21.15.47:0/4091222011 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb84c06ac00 0x7fb84c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.151+0000 7fb865f97640 1 --2- 172.21.15.47:0/4091222011 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb84c06ac00 0x7fb84c06d0c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb86006e8b0 tx=0x7fb850007390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.155+0000 7fb85e7fc640 1 -- 172.21.15.47:0/4091222011 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb854054350 con 0x7fb860099110 2024-03-23T09:43:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.373+0000 7fb8681fb640 1 -- 172.21.15.47:0/4091222011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb8600cca20 con 0x7fb860099110 2024-03-23T09:43:48.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.401+0000 7fb85e7fc640 1 -- 172.21.15.47:0/4091222011 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb854057970 con 0x7fb860099110 2024-03-23T09:43:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.403+0000 7fb8681fb640 1 -- 172.21.15.47:0/4091222011 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb84c06ac00 msgr2=0x7fb84c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.403+0000 7fb8681fb640 1 --2- 172.21.15.47:0/4091222011 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb84c06ac00 0x7fb84c06d0c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb86006e8b0 tx=0x7fb850007390 comp rx=0 tx=0).stop 2024-03-23T09:43:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.403+0000 7fb8681fb640 1 -- 172.21.15.47:0/4091222011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860099110 msgr2=0x7fb86009d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.403+0000 7fb8681fb640 1 --2- 172.21.15.47:0/4091222011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860099110 0x7fb86009d710 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb85400c430 tx=0x7fb85400c8f0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.403+0000 7fb8681fb640 1 -- 172.21.15.47:0/4091222011 shutdown_connections 2024-03-23T09:43:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.403+0000 7fb8681fb640 1 --2- 172.21.15.47:0/4091222011 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb84c06ac00 0x7fb84c06d0c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.403+0000 7fb8681fb640 1 --2- 172.21.15.47:0/4091222011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb86015cd40 0x7fb8600b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.403+0000 7fb8681fb640 1 --2- 172.21.15.47:0/4091222011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860099110 0x7fb86009d710 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.403+0000 7fb8681fb640 1 --2- 172.21.15.47:0/4091222011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb86006b090 0x7fb86009d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.403+0000 7fb8681fb640 1 -- 172.21.15.47:0/4091222011 >> 172.21.15.47:0/4091222011 conn(0x7fb8600b2140 msgr2=0x7fb86006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.403+0000 7fb8681fb640 1 -- 172.21.15.47:0/4091222011 shutdown_connections 2024-03-23T09:43:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.403+0000 7fb8681fb640 1 -- 172.21.15.47:0/4091222011 wait complete. 2024-03-23T09:43:48.414 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-23T09:43:48.414 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-23T09:43:48.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.518+0000 7fb603a1e640 1 Processor -- start 2024-03-23T09:43:48.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.518+0000 7fb603a1e640 1 -- start start 2024-03-23T09:43:48.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.519+0000 7fb603a1e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5fc15c990 0x7fb5fc160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:48.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.519+0000 7fb603a1e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc161630 con 0x7fb5fc0cde10 2024-03-23T09:43:48.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.519+0000 7fb603a1e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc0d1030 con 0x7fb5fc0b8350 2024-03-23T09:43:48.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.519+0000 7fb603a1e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5fc0cfe10 con 0x7fb5fc15c990 2024-03-23T09:43:48.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.519+0000 7fb6017ba640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5fc15c990 0x7fb5fc160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:48.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.519+0000 7fb601fbb640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb5fc0cde10 0x7fb5fc0d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36024/0 (socket says 172.21.15.47:36024) 2024-03-23T09:43:48.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.519+0000 7fb6017ba640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5fc15c990 0x7fb5fc160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39016/0 (socket says 172.21.15.47:39016) 2024-03-23T09:43:48.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.519+0000 7fb601fbb640 1 -- 172.21.15.47:0/2740046076 learned_addr learned my addr 172.21.15.47:0/2740046076 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:48.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.519+0000 7fb6017ba640 1 -- 172.21.15.47:0/2740046076 >> v1:172.21.15.175:6789/0 conn(0x7fb5fc0b8350 legacy=0x7fb5fc0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:48.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.519+0000 7fb6017ba640 1 -- 172.21.15.47:0/2740046076 >> v1:172.21.15.47:6789/0 conn(0x7fb5fc0cde10 legacy=0x7fb5fc0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:48.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.519+0000 7fb5f3fff640 1 -- 172.21.15.47:0/2740046076 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404520038 0 0) 0x7fb5fc161630 con 0x7fb5fc0cde10 2024-03-23T09:43:48.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.519+0000 7fb6017ba640 1 -- 172.21.15.47:0/2740046076 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc0b5b60 con 0x7fb5fc15c990 2024-03-23T09:43:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.520+0000 7fb6017ba640 1 --2- 172.21.15.47:0/2740046076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc15c990 0x7fb5fc160690 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80097a0 tx=0x7fb5f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa0dc94156032ef server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.520+0000 7fb5f3fff640 1 -- 172.21.15.47:0/2740046076 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f802e070 con 0x7fb5fc15c990 2024-03-23T09:43:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.520+0000 7fb5f3fff640 1 -- 172.21.15.47:0/2740046076 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f80315a0 con 0x7fb5fc15c990 2024-03-23T09:43:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.520+0000 7fb5f3fff640 1 -- 172.21.15.47:0/2740046076 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8031930 con 0x7fb5fc15c990 2024-03-23T09:43:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.520+0000 7fb603a1e640 1 -- 172.21.15.47:0/2740046076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc15c990 msgr2=0x7fb5fc160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.520+0000 7fb603a1e640 1 --2- 172.21.15.47:0/2740046076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc15c990 0x7fb5fc160690 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80097a0 tx=0x7fb5f802b840 comp rx=0 tx=0).stop 2024-03-23T09:43:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.520+0000 7fb603a1e640 1 -- 172.21.15.47:0/2740046076 shutdown_connections 2024-03-23T09:43:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.521+0000 7fb603a1e640 1 --2- 172.21.15.47:0/2740046076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc15c990 0x7fb5fc160690 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.521+0000 7fb603a1e640 1 -- 172.21.15.47:0/2740046076 >> 172.21.15.47:0/2740046076 conn(0x7fb5fc0659d0 msgr2=0x7fb5fc065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.521+0000 7fb603a1e640 1 -- 172.21.15.47:0/2740046076 shutdown_connections 2024-03-23T09:43:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.521+0000 7fb603a1e640 1 -- 172.21.15.47:0/2740046076 wait complete. 2024-03-23T09:43:48.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.522+0000 7fb603a1e640 1 Processor -- start 2024-03-23T09:43:48.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.522+0000 7fb603a1e640 1 -- start start 2024-03-23T09:43:48.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.522+0000 7fb603a1e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5fc0b8350 0x7fb5fc153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:48.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.522+0000 7fb603a1e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5fc0cde10 0x7fb5fc0c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:48.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.522+0000 7fb603a1e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc15c990 0x7fb5fc15fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:48.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.522+0000 7fb603a1e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0c7280 con 0x7fb5fc15c990 2024-03-23T09:43:48.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.522+0000 7fb603a1e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc154080 con 0x7fb5fc0b8350 2024-03-23T09:43:48.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.522+0000 7fb603a1e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0c63b0 con 0x7fb5fc0cde10 2024-03-23T09:43:48.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb601fbb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc15c990 0x7fb5fc15fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:48.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb601fbb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc15c990 0x7fb5fc15fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39018/0 (socket says 172.21.15.47:39018) 2024-03-23T09:43:48.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb601fbb640 1 -- 172.21.15.47:0/38552991 learned_addr learned my addr 172.21.15.47:0/38552991 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:48.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb6017ba640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5fc0b8350 0x7fb5fc153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:48.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb600fb9640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5fc0cde10 0x7fb5fc0c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:48.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb601fbb640 1 -- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5fc0cde10 msgr2=0x7fb5fc0c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:48.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb601fbb640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5fc0cde10 0x7fb5fc0c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb601fbb640 1 -- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5fc0b8350 msgr2=0x7fb5fc153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb601fbb640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5fc0b8350 0x7fb5fc153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb601fbb640 1 -- 172.21.15.47:0/38552991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f8009400 con 0x7fb5fc15c990 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb600fb9640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5fc0cde10 0x7fb5fc0c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb6017ba640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5fc0b8350 0x7fb5fc153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb601fbb640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc15c990 0x7fb5fc15fce0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400c810 tx=0x7fb5f400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb5f1ffb640 1 -- 172.21.15.47:0/38552991 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400a5e0 con 0x7fb5fc15c990 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb5f1ffb640 1 -- 172.21.15.47:0/38552991 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f400a760 con 0x7fb5fc15c990 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb603a1e640 1 -- 172.21.15.47:0/38552991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc0c91d0 con 0x7fb5fc15c990 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb5f1ffb640 1 -- 172.21.15.47:0/38552991 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400ab10 con 0x7fb5fc15c990 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.523+0000 7fb603a1e640 1 -- 172.21.15.47:0/38552991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc0c68e0 con 0x7fb5fc15c990 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.524+0000 7fb603a1e640 1 -- 172.21.15.47:0/38552991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5fc072360 con 0x7fb5fc15c990 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.525+0000 7fb5f1ffb640 1 -- 172.21.15.47:0/38552991 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb5f400ac70 con 0x7fb5fc15c990 2024-03-23T09:43:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.525+0000 7fb5f1ffb640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5e006ac00 0x7fb5e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.525+0000 7fb5f1ffb640 1 -- 172.21.15.47:0/38552991 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5f4088b00 con 0x7fb5fc15c990 2024-03-23T09:43:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.525+0000 7fb6017ba640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5e006ac00 0x7fb5e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.526+0000 7fb6017ba640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5e006ac00 0x7fb5e006d0c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8036000 tx=0x7fb5f803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.529+0000 7fb5f1ffb640 1 -- 172.21.15.47:0/38552991 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb5f4053d70 con 0x7fb5fc15c990 2024-03-23T09:43:48.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:48.745+0000 7fb603a1e640 1 -- 172.21.15.47:0/38552991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb5fc0c73c0 con 0x7fb5fc15c990 2024-03-23T09:43:48.967 INFO:tasks.ceph.mds.d.smithi175.stderr:2024-03-23T09:43:48.967+0000 7fa917ab8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:43:49.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.403+0000 7fb5f1ffb640 1 -- 172.21.15.47:0/38552991 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb5fc0c73c0 con 0x7fb5fc15c990 2024-03-23T09:43:49.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.406+0000 7fb603a1e640 1 -- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5e006ac00 msgr2=0x7fb5e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:49.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.406+0000 7fb603a1e640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5e006ac00 0x7fb5e006d0c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8036000 tx=0x7fb5f803d040 comp rx=0 tx=0).stop 2024-03-23T09:43:49.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.406+0000 7fb603a1e640 1 -- 172.21.15.47:0/38552991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc15c990 msgr2=0x7fb5fc15fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:49.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.406+0000 7fb603a1e640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc15c990 0x7fb5fc15fce0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400c810 tx=0x7fb5f400ccd0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.406+0000 7fb603a1e640 1 -- 172.21.15.47:0/38552991 shutdown_connections 2024-03-23T09:43:49.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.406+0000 7fb603a1e640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5e006ac00 0x7fb5e006d0c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.406+0000 7fb603a1e640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc15c990 0x7fb5fc15fce0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.406+0000 7fb603a1e640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5fc0cde10 0x7fb5fc0c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.406+0000 7fb603a1e640 1 --2- 172.21.15.47:0/38552991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5fc0b8350 0x7fb5fc153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.406+0000 7fb603a1e640 1 -- 172.21.15.47:0/38552991 >> 172.21.15.47:0/38552991 conn(0x7fb5fc0659d0 msgr2=0x7fb5fc15bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:49.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.406+0000 7fb603a1e640 1 -- 172.21.15.47:0/38552991 shutdown_connections 2024-03-23T09:43:49.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.406+0000 7fb603a1e640 1 -- 172.21.15.47:0/38552991 wait complete. 2024-03-23T09:43:49.419 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-23T09:43:49.419 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:43:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.527+0000 7fb09b399640 1 Processor -- start 2024-03-23T09:43:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.527+0000 7fb09b399640 1 -- start start 2024-03-23T09:43:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.528+0000 7fb09b399640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb09415ca90 0x7fb094160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.528+0000 7fb09b399640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb0940b8350 0x7fb0940cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.528+0000 7fb09b399640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0940ce090 con 0x7fb0940cdc40 2024-03-23T09:43:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.528+0000 7fb09b399640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0940d11e0 con 0x7fb0940b8350 2024-03-23T09:43:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.528+0000 7fb09b399640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0940d2680 con 0x7fb09415ca90 2024-03-23T09:43:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.528+0000 7fb098934640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb0940b8350 0x7fb0940cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.528+0000 7fb098934640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb0940b8350 0x7fb0940cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39034/0 (socket says 172.21.15.47:39034) 2024-03-23T09:43:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.528+0000 7fb098934640 1 -- 172.21.15.47:0/3307621572 learned_addr learned my addr 172.21.15.47:0/3307621572 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.528+0000 7fb099936640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb0940cdc40 0x7fb0940b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36030/0 (socket says 172.21.15.47:36030) 2024-03-23T09:43:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.528+0000 7fb098934640 1 -- 172.21.15.47:0/3307621572 >> v2:172.21.15.175:3300/0 conn(0x7fb09415ca90 msgr2=0x7fb094160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.528+0000 7fb098934640 1 --2- 172.21.15.47:0/3307621572 >> v2:172.21.15.175:3300/0 conn(0x7fb09415ca90 0x7fb094160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.528+0000 7fb098934640 1 -- 172.21.15.47:0/3307621572 >> v1:172.21.15.47:6789/0 conn(0x7fb0940cdc40 legacy=0x7fb0940b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.528+0000 7fb098934640 1 -- 172.21.15.47:0/3307621572 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb094160f60 con 0x7fb0940b8350 2024-03-23T09:43:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.529+0000 7fb098934640 1 --2- 172.21.15.47:0/3307621572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0940b8350 0x7fb0940cd700 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb094066680 tx=0x7fb08402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bcc07cb3115e33 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.529+0000 7fb08bfff640 1 -- 172.21.15.47:0/3307621572 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084031870 con 0x7fb0940b8350 2024-03-23T09:43:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.529+0000 7fb08bfff640 1 -- 172.21.15.47:0/3307621572 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0840319d0 con 0x7fb0940b8350 2024-03-23T09:43:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.529+0000 7fb08bfff640 1 -- 172.21.15.47:0/3307621572 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084031d60 con 0x7fb0940b8350 2024-03-23T09:43:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.529+0000 7fb09b399640 1 -- 172.21.15.47:0/3307621572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0940b8350 msgr2=0x7fb0940cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.529+0000 7fb09b399640 1 --2- 172.21.15.47:0/3307621572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0940b8350 0x7fb0940cd700 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb094066680 tx=0x7fb08402b910 comp rx=0 tx=0).stop 2024-03-23T09:43:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.530+0000 7fb09b399640 1 -- 172.21.15.47:0/3307621572 shutdown_connections 2024-03-23T09:43:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.530+0000 7fb09b399640 1 --2- 172.21.15.47:0/3307621572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0940b8350 0x7fb0940cd700 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.530+0000 7fb09b399640 1 --2- 172.21.15.47:0/3307621572 >> v2:172.21.15.175:3300/0 conn(0x7fb09415ca90 0x7fb094160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.530+0000 7fb09b399640 1 -- 172.21.15.47:0/3307621572 >> 172.21.15.47:0/3307621572 conn(0x7fb0940659d0 msgr2=0x7fb094065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.530+0000 7fb09b399640 1 -- 172.21.15.47:0/3307621572 shutdown_connections 2024-03-23T09:43:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.530+0000 7fb09b399640 1 -- 172.21.15.47:0/3307621572 wait complete. 2024-03-23T09:43:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.531+0000 7fb09b399640 1 Processor -- start 2024-03-23T09:43:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.531+0000 7fb09b399640 1 -- start start 2024-03-23T09:43:49.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.531+0000 7fb09b399640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0940b8350 0x7fb0940b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:49.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb09b399640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0940cdc40 0x7fb094150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:49.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb09b399640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb09415ca90 0x7fb094153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:49.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb09b399640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940caef0 con 0x7fb09415ca90 2024-03-23T09:43:49.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb09b399640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09415a8b0 con 0x7fb0940b8350 2024-03-23T09:43:49.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb09b399640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09415ab90 con 0x7fb0940cdc40 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb099936640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb09415ca90 0x7fb094153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb099936640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb09415ca90 0x7fb094153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39044/0 (socket says 172.21.15.47:39044) 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb099936640 1 -- 172.21.15.47:0/641332575 learned_addr learned my addr 172.21.15.47:0/641332575 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb099936640 1 -- 172.21.15.47:0/641332575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0940cdc40 msgr2=0x7fb094150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb099936640 1 --2- 172.21.15.47:0/641332575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0940cdc40 0x7fb094150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb099936640 1 -- 172.21.15.47:0/641332575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0940b8350 msgr2=0x7fb0940b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb099135640 1 --2- 172.21.15.47:0/641332575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0940b8350 0x7fb0940b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb099936640 1 --2- 172.21.15.47:0/641332575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0940b8350 0x7fb0940b6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb099936640 1 -- 172.21.15.47:0/641332575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0840094d0 con 0x7fb09415ca90 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb099936640 1 --2- 172.21.15.47:0/641332575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb09415ca90 0x7fb094153760 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00c780 tx=0x7fb08c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb089ffb640 1 -- 172.21.15.47:0/641332575 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c00ce90 con 0x7fb09415ca90 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb089ffb640 1 -- 172.21.15.47:0/641332575 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c00a640 con 0x7fb09415ca90 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb09b399640 1 -- 172.21.15.47:0/641332575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0940cab80 con 0x7fb09415ca90 2024-03-23T09:43:49.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.532+0000 7fb089ffb640 1 -- 172.21.15.47:0/641332575 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c00a9f0 con 0x7fb09415ca90 2024-03-23T09:43:49.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.533+0000 7fb09b399640 1 -- 172.21.15.47:0/641332575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0940b73a0 con 0x7fb09415ca90 2024-03-23T09:43:49.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.534+0000 7fb09b399640 1 -- 172.21.15.47:0/641332575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb094072360 con 0x7fb09415ca90 2024-03-23T09:43:49.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.534+0000 7fb089ffb640 1 -- 172.21.15.47:0/641332575 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb08c00ab50 con 0x7fb09415ca90 2024-03-23T09:43:49.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.535+0000 7fb089ffb640 1 --2- 172.21.15.47:0/641332575 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb07806ac00 0x7fb07806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:49.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.535+0000 7fb089ffb640 1 -- 172.21.15.47:0/641332575 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb08c0889c0 con 0x7fb09415ca90 2024-03-23T09:43:49.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.535+0000 7fb099135640 1 --2- 172.21.15.47:0/641332575 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb07806ac00 0x7fb07806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:49.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.536+0000 7fb099135640 1 --2- 172.21.15.47:0/641332575 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb07806ac00 0x7fb07806d0c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb09000a810 tx=0x7fb090008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:49.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.539+0000 7fb089ffb640 1 -- 172.21.15.47:0/641332575 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb08c053bb0 con 0x7fb09415ca90 2024-03-23T09:43:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.754+0000 7fb09b399640 1 -- 172.21.15.47:0/641332575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0940ce090 con 0x7fb09415ca90 2024-03-23T09:43:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.754+0000 7fb089ffb640 1 -- 172.21.15.47:0/641332575 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fb0940ce090 con 0x7fb09415ca90 2024-03-23T09:43:49.755 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:49.755 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/375513546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":375513546},{"type":"v1","addr":"172.21.15.175:6837","nonce":375513546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/3650756747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3650756747},{"type":"v1","addr":"172.21.15.47:6833","nonce":3650756747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2893805543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2893805543},{"type":"v1","addr":"172.21.15.47:6835","nonce":2893805543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:43:47.389279+0000","modified":"2024-03-23T09:43:49.400979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/2498456351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2498456351},{"type":"v1","addr":"172.21.15.175:6836","nonce":2498456351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-23T09:43:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2024-03-23T09:43:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.756+0000 7fb09b399640 1 -- 172.21.15.47:0/641332575 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb07806ac00 msgr2=0x7fb07806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.756+0000 7fb09b399640 1 --2- 172.21.15.47:0/641332575 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb07806ac00 0x7fb07806d0c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb09000a810 tx=0x7fb090008040 comp rx=0 tx=0).stop 2024-03-23T09:43:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.756+0000 7fb09b399640 1 -- 172.21.15.47:0/641332575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb09415ca90 msgr2=0x7fb094153760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.756+0000 7fb09b399640 1 --2- 172.21.15.47:0/641332575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb09415ca90 0x7fb094153760 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00c780 tx=0x7fb08c00cc40 comp rx=0 tx=0).stop 2024-03-23T09:43:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.756+0000 7fb09b399640 1 -- 172.21.15.47:0/641332575 shutdown_connections 2024-03-23T09:43:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.756+0000 7fb09b399640 1 --2- 172.21.15.47:0/641332575 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb07806ac00 0x7fb07806d0c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.756+0000 7fb09b399640 1 --2- 172.21.15.47:0/641332575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb09415ca90 0x7fb094153760 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.756+0000 7fb09b399640 1 --2- 172.21.15.47:0/641332575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0940cdc40 0x7fb094150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.756+0000 7fb09b399640 1 --2- 172.21.15.47:0/641332575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0940b8350 0x7fb0940b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.756+0000 7fb09b399640 1 -- 172.21.15.47:0/641332575 >> 172.21.15.47:0/641332575 conn(0x7fb0940659d0 msgr2=0x7fb0940cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.756+0000 7fb09b399640 1 -- 172.21.15.47:0/641332575 shutdown_connections 2024-03-23T09:43:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.756+0000 7fb09b399640 1 -- 172.21.15.47:0/641332575 wait complete. 2024-03-23T09:43:49.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:43:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.868+0000 7f4314ee0640 1 Processor -- start 2024-03-23T09:43:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.868+0000 7f4314ee0640 1 -- start start 2024-03-23T09:43:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.869+0000 7f4314ee0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f43100cde10 0x7f4310167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.869+0000 7f4314ee0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43100d11e0 con 0x7f431015ca90 2024-03-23T09:43:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.869+0000 7f4314ee0640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43100d2680 con 0x7f43100b8350 2024-03-23T09:43:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.869+0000 7f4314ee0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4310160f60 con 0x7f43100cde10 2024-03-23T09:43:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.869+0000 7f430ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f43100cde10 0x7f4310167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.869+0000 7f430ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f43100cde10 0x7f4310167aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39056/0 (socket says 172.21.15.47:39056) 2024-03-23T09:43:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.869+0000 7f430e59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f431015ca90 0x7f4310160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36032/0 (socket says 172.21.15.47:36032) 2024-03-23T09:43:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.869+0000 7f430ed9d640 1 -- 172.21.15.47:0/1907108899 learned_addr learned my addr 172.21.15.47:0/1907108899 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.869+0000 7f430ed9d640 1 -- 172.21.15.47:0/1907108899 >> v1:172.21.15.175:6789/0 conn(0x7f43100b8350 legacy=0x7f43100cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.869+0000 7f430ed9d640 1 -- 172.21.15.47:0/1907108899 >> v1:172.21.15.47:6789/0 conn(0x7f431015ca90 legacy=0x7f4310160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.869+0000 7f430ed9d640 1 -- 172.21.15.47:0/1907108899 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310161240 con 0x7f43100cde10 2024-03-23T09:43:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.870+0000 7f430ed9d640 1 --2- 172.21.15.47:0/1907108899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43100cde10 0x7f4310167aa0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4308009e50 tx=0x7f430802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef2c19dcb418bb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.870+0000 7f430d59a640 1 -- 172.21.15.47:0/1907108899 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430802cb20 con 0x7f43100cde10 2024-03-23T09:43:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.870+0000 7f430d59a640 1 -- 172.21.15.47:0/1907108899 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430802cc80 con 0x7f43100cde10 2024-03-23T09:43:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.870+0000 7f430d59a640 1 -- 172.21.15.47:0/1907108899 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43080343f0 con 0x7f43100cde10 2024-03-23T09:43:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.870+0000 7f4314ee0640 1 -- 172.21.15.47:0/1907108899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43100cde10 msgr2=0x7f4310167aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.870+0000 7f4314ee0640 1 --2- 172.21.15.47:0/1907108899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43100cde10 0x7f4310167aa0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4308009e50 tx=0x7f430802bf80 comp rx=0 tx=0).stop 2024-03-23T09:43:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.871+0000 7f4314ee0640 1 -- 172.21.15.47:0/1907108899 shutdown_connections 2024-03-23T09:43:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.871+0000 7f4314ee0640 1 --2- 172.21.15.47:0/1907108899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43100cde10 0x7f4310167aa0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.871+0000 7f4314ee0640 1 -- 172.21.15.47:0/1907108899 >> 172.21.15.47:0/1907108899 conn(0x7f43100659d0 msgr2=0x7f4310065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.871+0000 7f4314ee0640 1 -- 172.21.15.47:0/1907108899 shutdown_connections 2024-03-23T09:43:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.871+0000 7f4314ee0640 1 -- 172.21.15.47:0/1907108899 wait complete. 2024-03-23T09:43:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.872+0000 7f4314ee0640 1 Processor -- start 2024-03-23T09:43:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.872+0000 7f4314ee0640 1 -- start start 2024-03-23T09:43:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f4314ee0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43100b8350 0x7f43100b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f4314ee0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43100cde10 0x7f431014cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f4314ee0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f431015ca90 0x7f431015e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f4314ee0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43100b4eb0 con 0x7f43100b8350 2024-03-23T09:43:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f4314ee0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43100b45e0 con 0x7f43100cde10 2024-03-23T09:43:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f4314ee0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43100b28b0 con 0x7f431015ca90 2024-03-23T09:43:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f430e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43100b8350 0x7f43100b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f430e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43100b8350 0x7f43100b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39068/0 (socket says 172.21.15.47:39068) 2024-03-23T09:43:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f430e59c640 1 -- 172.21.15.47:0/3293932537 learned_addr learned my addr 172.21.15.47:0/3293932537 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f430dd9b640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43100cde10 0x7f431014cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f430ed9d640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f431015ca90 0x7f431015e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f430e59c640 1 -- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f431015ca90 msgr2=0x7f431015e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f430e59c640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f431015ca90 0x7f431015e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f430e59c640 1 -- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43100cde10 msgr2=0x7f431014cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f430e59c640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43100cde10 0x7f431014cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f430e59c640 1 -- 172.21.15.47:0/3293932537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4308009b30 con 0x7f43100b8350 2024-03-23T09:43:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f430e59c640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43100b8350 0x7f43100b4970 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f430000c4d0 tx=0x7f430000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.873+0000 7f42feffd640 1 -- 172.21.15.47:0/3293932537 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300007480 con 0x7f43100b8350 2024-03-23T09:43:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.874+0000 7f42feffd640 1 -- 172.21.15.47:0/3293932537 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43000075e0 con 0x7f43100b8350 2024-03-23T09:43:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.874+0000 7f4314ee0640 1 -- 172.21.15.47:0/3293932537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43100b2b90 con 0x7f43100b8350 2024-03-23T09:43:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.874+0000 7f42feffd640 1 -- 172.21.15.47:0/3293932537 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43000049e0 con 0x7f43100b8350 2024-03-23T09:43:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.874+0000 7f4314ee0640 1 -- 172.21.15.47:0/3293932537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43100b3150 con 0x7f43100b8350 2024-03-23T09:43:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.874+0000 7f4314ee0640 1 -- 172.21.15.47:0/3293932537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4310072360 con 0x7f43100b8350 2024-03-23T09:43:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.875+0000 7f42feffd640 1 -- 172.21.15.47:0/3293932537 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4300004b40 con 0x7f43100b8350 2024-03-23T09:43:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.875+0000 7f42feffd640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f42f006ac00 0x7f42f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.875+0000 7f42feffd640 1 -- 172.21.15.47:0/3293932537 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4300088820 con 0x7f43100b8350 2024-03-23T09:43:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.876+0000 7f430dd9b640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f42f006ac00 0x7f42f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.876+0000 7f430dd9b640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f42f006ac00 0x7f42f006d0c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f43101f5b70 tx=0x7f4304009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:49.880+0000 7f42feffd640 1 -- 172.21.15.47:0/3293932537 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4300053a90 con 0x7f43100b8350 2024-03-23T09:43:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.059+0000 7f4314ee0640 1 -- 172.21.15.47:0/3293932537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4310066be0 con 0x7f43100b8350 2024-03-23T09:43:50.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.060+0000 7f42feffd640 1 -- 172.21.15.47:0/3293932537 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16097 (secure 0 0 0) 0x7f4300018090 con 0x7f43100b8350 2024-03-23T09:43:50.060 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:50.060 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":58,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:43:48.390657+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:43:45.646160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-23T09:43:46.351634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:43:50.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.062+0000 7f4314ee0640 1 -- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f42f006ac00 msgr2=0x7f42f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.062+0000 7f4314ee0640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f42f006ac00 0x7f42f006d0c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f43101f5b70 tx=0x7f4304009040 comp rx=0 tx=0).stop 2024-03-23T09:43:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.062+0000 7f4314ee0640 1 -- 172.21.15.47:0/3293932537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43100b8350 msgr2=0x7f43100b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.062+0000 7f4314ee0640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43100b8350 0x7f43100b4970 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f430000c4d0 tx=0x7f430000c990 comp rx=0 tx=0).stop 2024-03-23T09:43:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.062+0000 7f4314ee0640 1 -- 172.21.15.47:0/3293932537 shutdown_connections 2024-03-23T09:43:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.062+0000 7f4314ee0640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f42f006ac00 0x7f42f006d0c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.062+0000 7f4314ee0640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f431015ca90 0x7f431015e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.062+0000 7f4314ee0640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43100cde10 0x7f431014cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.062+0000 7f4314ee0640 1 --2- 172.21.15.47:0/3293932537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43100b8350 0x7f43100b4970 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.062+0000 7f4314ee0640 1 -- 172.21.15.47:0/3293932537 >> 172.21.15.47:0/3293932537 conn(0x7f43100659d0 msgr2=0x7f431015d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.062+0000 7f4314ee0640 1 -- 172.21.15.47:0/3293932537 shutdown_connections 2024-03-23T09:43:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.062+0000 7f4314ee0640 1 -- 172.21.15.47:0/3293932537 wait complete. 2024-03-23T09:43:50.073 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T09:43:50.073 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:43:50.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.174+0000 7fb6e5769640 1 Processor -- start 2024-03-23T09:43:50.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.174+0000 7fb6e5769640 1 -- start start 2024-03-23T09:43:50.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6e5769640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e015cd00 0x7fb6e00d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6e5769640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb6e00d1020 0x7fb6e00cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6e5769640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb6e00cf750 0x7fb6e00b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6e5769640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e00cfbe0 con 0x7fb6e00d1020 2024-03-23T09:43:50.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6e5769640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e015d0e0 con 0x7fb6e00cf750 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6e5769640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e00d1540 con 0x7fb6e015cd00 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6d7fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb6e00d1020 0x7fb6e00cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6d7fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb6e00d1020 0x7fb6e00cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39076/0 (socket says 172.21.15.47:39076) 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6d7fff640 1 -- 172.21.15.47:0/3664990566 learned_addr learned my addr 172.21.15.47:0/3664990566 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6df7fe640 1 --2- 172.21.15.47:0/3664990566 >> v2:172.21.15.175:3300/0 conn(0x7fb6e00cf750 0x7fb6e00b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6deffd640 1 --2- 172.21.15.47:0/3664990566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e015cd00 0x7fb6e00d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6d7fff640 1 -- 172.21.15.47:0/3664990566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e015cd00 msgr2=0x7fb6e00d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6d7fff640 1 --2- 172.21.15.47:0/3664990566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e015cd00 0x7fb6e00d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6d7fff640 1 -- 172.21.15.47:0/3664990566 >> v2:172.21.15.175:3300/0 conn(0x7fb6e00cf750 msgr2=0x7fb6e00b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6d7fff640 1 --2- 172.21.15.47:0/3664990566 >> v2:172.21.15.175:3300/0 conn(0x7fb6e00cf750 0x7fb6e00b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.175+0000 7fb6d7fff640 1 -- 172.21.15.47:0/3664990566 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e00d1820 con 0x7fb6e00d1020 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.176+0000 7fb6d7fff640 1 --2- 172.21.15.47:0/3664990566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e00d1020 0x7fb6e00cf210 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00b8040 tx=0x7fb6c802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a48b7ef2f25cae1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.176+0000 7fb6de7fc640 1 -- 172.21.15.47:0/3664990566 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80317e0 con 0x7fb6e00d1020 2024-03-23T09:43:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.176+0000 7fb6de7fc640 1 -- 172.21.15.47:0/3664990566 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c8031940 con 0x7fb6e00d1020 2024-03-23T09:43:50.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.176+0000 7fb6de7fc640 1 -- 172.21.15.47:0/3664990566 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8031cd0 con 0x7fb6e00d1020 2024-03-23T09:43:50.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.177+0000 7fb6e5769640 1 -- 172.21.15.47:0/3664990566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e00d1020 msgr2=0x7fb6e00cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.177+0000 7fb6e5769640 1 --2- 172.21.15.47:0/3664990566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e00d1020 0x7fb6e00cf210 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00b8040 tx=0x7fb6c802b880 comp rx=0 tx=0).stop 2024-03-23T09:43:50.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.177+0000 7fb6e5769640 1 -- 172.21.15.47:0/3664990566 shutdown_connections 2024-03-23T09:43:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.177+0000 7fb6e5769640 1 --2- 172.21.15.47:0/3664990566 >> v2:172.21.15.175:3300/0 conn(0x7fb6e00cf750 0x7fb6e00b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.177+0000 7fb6e5769640 1 --2- 172.21.15.47:0/3664990566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e00d1020 0x7fb6e00cf210 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.177+0000 7fb6e5769640 1 --2- 172.21.15.47:0/3664990566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e015cd00 0x7fb6e00d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.177+0000 7fb6e5769640 1 -- 172.21.15.47:0/3664990566 >> 172.21.15.47:0/3664990566 conn(0x7fb6e006b160 msgr2=0x7fb6e006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.177+0000 7fb6e5769640 1 -- 172.21.15.47:0/3664990566 shutdown_connections 2024-03-23T09:43:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.177+0000 7fb6e5769640 1 -- 172.21.15.47:0/3664990566 wait complete. 2024-03-23T09:43:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.178+0000 7fb6e5769640 1 Processor -- start 2024-03-23T09:43:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.178+0000 7fb6e5769640 1 -- start start 2024-03-23T09:43:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6e5769640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e00cf750 0x7fb6e0158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6e5769640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e00d1020 0x7fb6e0154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6e5769640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6e015cd00 0x7fb6e0152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6e5769640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e00c72d0 con 0x7fb6e00cf750 2024-03-23T09:43:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6e5769640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e0158b90 con 0x7fb6e015cd00 2024-03-23T09:43:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6e5769640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e0155800 con 0x7fb6e00d1020 2024-03-23T09:43:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6deffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e00cf750 0x7fb6e0158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6deffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e00cf750 0x7fb6e0158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39092/0 (socket says 172.21.15.47:39092) 2024-03-23T09:43:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6deffd640 1 -- 172.21.15.47:0/852386347 learned_addr learned my addr 172.21.15.47:0/852386347 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6deffd640 1 -- 172.21.15.47:0/852386347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e00d1020 msgr2=0x7fb6e0154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6deffd640 1 --2- 172.21.15.47:0/852386347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e00d1020 0x7fb6e0154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6deffd640 1 -- 172.21.15.47:0/852386347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6e015cd00 msgr2=0x7fb6e0152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6deffd640 1 --2- 172.21.15.47:0/852386347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6e015cd00 0x7fb6e0152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6deffd640 1 -- 172.21.15.47:0/852386347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c80094d0 con 0x7fb6e00cf750 2024-03-23T09:43:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.179+0000 7fb6deffd640 1 --2- 172.21.15.47:0/852386347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e00cf750 0x7fb6e0158650 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000d5f0 tx=0x7fb6d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.180+0000 7fb6d77fe640 1 -- 172.21.15.47:0/852386347 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d00125f0 con 0x7fb6e00cf750 2024-03-23T09:43:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.180+0000 7fb6d77fe640 1 -- 172.21.15.47:0/852386347 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d0012750 con 0x7fb6e00cf750 2024-03-23T09:43:50.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.180+0000 7fb6e5769640 1 -- 172.21.15.47:0/852386347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e0155ae0 con 0x7fb6e00cf750 2024-03-23T09:43:50.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.180+0000 7fb6d77fe640 1 -- 172.21.15.47:0/852386347 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0012b00 con 0x7fb6e00cf750 2024-03-23T09:43:50.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.180+0000 7fb6e5769640 1 -- 172.21.15.47:0/852386347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e00c6ef0 con 0x7fb6e00cf750 2024-03-23T09:43:50.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.181+0000 7fb6e5769640 1 -- 172.21.15.47:0/852386347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e0072360 con 0x7fb6e00cf750 2024-03-23T09:43:50.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.181+0000 7fb6d77fe640 1 -- 172.21.15.47:0/852386347 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb6d0012c60 con 0x7fb6e00cf750 2024-03-23T09:43:50.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.181+0000 7fb6d77fe640 1 --2- 172.21.15.47:0/852386347 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb6c006ac00 0x7fb6c006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.182+0000 7fb6d77fe640 1 -- 172.21.15.47:0/852386347 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6d0089890 con 0x7fb6e00cf750 2024-03-23T09:43:50.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.182+0000 7fb6d7fff640 1 --2- 172.21.15.47:0/852386347 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb6c006ac00 0x7fb6c006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.182+0000 7fb6d7fff640 1 --2- 172.21.15.47:0/852386347 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb6c006ac00 0x7fb6c006d0c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8036000 tx=0x7fb6c8030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:50.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.186+0000 7fb6d77fe640 1 -- 172.21.15.47:0/852386347 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb6d0054a70 con 0x7fb6e00cf750 2024-03-23T09:43:50.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.362+0000 7fb6e5769640 1 -- 172.21.15.47:0/852386347 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6e00d20c0 con 0x7fb6c006ac00 2024-03-23T09:43:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.370+0000 7fb6d77fe640 1 -- 172.21.15.47:0/852386347 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286507 (secure 0 0 0) 0x7fb6c8031ab0 con 0x7fb6c006ac00 2024-03-23T09:43:50.374 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:43:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.373+0000 7fb6e5769640 1 -- 172.21.15.47:0/852386347 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb6c006ac00 msgr2=0x7fb6c006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.373+0000 7fb6e5769640 1 --2- 172.21.15.47:0/852386347 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb6c006ac00 0x7fb6c006d0c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8036000 tx=0x7fb6c8030000 comp rx=0 tx=0).stop 2024-03-23T09:43:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.373+0000 7fb6e5769640 1 -- 172.21.15.47:0/852386347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e00cf750 msgr2=0x7fb6e0158650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.373+0000 7fb6e5769640 1 --2- 172.21.15.47:0/852386347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e00cf750 0x7fb6e0158650 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000d5f0 tx=0x7fb6d000dab0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.374+0000 7fb6e5769640 1 -- 172.21.15.47:0/852386347 shutdown_connections 2024-03-23T09:43:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.374+0000 7fb6e5769640 1 --2- 172.21.15.47:0/852386347 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb6c006ac00 0x7fb6c006d0c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.374+0000 7fb6e5769640 1 --2- 172.21.15.47:0/852386347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6e015cd00 0x7fb6e0152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.374+0000 7fb6e5769640 1 --2- 172.21.15.47:0/852386347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e00d1020 0x7fb6e0154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.374+0000 7fb6e5769640 1 --2- 172.21.15.47:0/852386347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e00cf750 0x7fb6e0158650 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.374+0000 7fb6e5769640 1 -- 172.21.15.47:0/852386347 >> 172.21.15.47:0/852386347 conn(0x7fb6e006b160 msgr2=0x7fb6e00cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.374+0000 7fb6e5769640 1 -- 172.21.15.47:0/852386347 shutdown_connections 2024-03-23T09:43:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.374+0000 7fb6e5769640 1 -- 172.21.15.47:0/852386347 wait complete. 2024-03-23T09:43:50.384 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-03-23T09:43:48.395313+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238916,"kb_used_data":6352,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754735804,"statfs":{"total":773094113280,"available":772849463296,"internally_reserved":0,"allocated":6504448,"data_stored":3349262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.018158"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.028360+0000","last_change":"2024-03-23T09:43:47.028360+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.028360+0000","last_undegraded":"2024-03-23T09:43:47.028360+0000","last_fullsized":"2024-03-23T09:43:47.028360+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:54:41.177596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021545+0000","last_change":"2024-03-23T09:43:47.021545+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021545+0000","last_undegraded":"2024-03-23T09:43:47.021545+0000","last_fullsized":"2024-03-23T09:43:47.021545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:36.262351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.027083+0000","last_change":"2024-03-23T09:43:47.027083+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.027083+0000","last_undegraded":"2024-03-23T09:43:47.027083+0000","last_fullsized":"2024-03-23T09:43:47.027083+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:22:26.903375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032839+0000","last_change":"2024-03-23T09:43:47.032839+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032839+0000","last_undegraded":"2024-03-23T09:43:47.032839+0000","last_fullsized":"2024-03-23T09:43:47.032839+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026471+0000","last_change":"2024-03-23T09:43:47.026471+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026471+0000","last_undegraded":"2024-03-23T09:43:47.026471+0000","last_fullsized":"2024-03-23T09:43:47.026471+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:17.338788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022332+0000","last_change":"2024-03-23T09:43:47.022332+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022332+0000","last_undegraded":"2024-03-23T09:43:47.022332+0000","last_fullsized":"2024-03-23T09:43:47.022332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023985+0000","last_change":"2024-03-23T09:43:47.023985+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023985+0000","last_undegraded":"2024-03-23T09:43:47.023985+0000","last_fullsized":"2024-03-23T09:43:47.023985+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:44:18.230315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021778+0000","last_change":"2024-03-23T09:43:47.021778+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021778+0000","last_undegraded":"2024-03-23T09:43:47.021778+0000","last_fullsized":"2024-03-23T09:43:47.021778+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:55:17.926828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023723+0000","last_change":"2024-03-23T09:43:47.023723+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023723+0000","last_undegraded":"2024-03-23T09:43:47.023723+0000","last_fullsized":"2024-03-23T09:43:47.023723+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:19.820479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.020703+0000","last_change":"2024-03-23T09:43:47.020703+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.020703+0000","last_undegraded":"2024-03-23T09:43:47.020703+0000","last_fullsized":"2024-03-23T09:43:47.020703+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:35:30.364827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.020633+0000","last_change":"2024-03-23T09:43:47.020633+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.020633+0000","last_undegraded":"2024-03-23T09:43:47.020633+0000","last_fullsized":"2024-03-23T09:43:47.020633+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:48:50.443256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032222+0000","last_change":"2024-03-23T09:43:47.032222+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032222+0000","last_undegraded":"2024-03-23T09:43:47.032222+0000","last_fullsized":"2024-03-23T09:43:47.032222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026161+0000","last_change":"2024-03-23T09:43:47.026161+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026161+0000","last_undegraded":"2024-03-23T09:43:47.026161+0000","last_fullsized":"2024-03-23T09:43:47.026161+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023765+0000","last_change":"2024-03-23T09:43:47.023765+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023765+0000","last_undegraded":"2024-03-23T09:43:47.023765+0000","last_fullsized":"2024-03-23T09:43:47.023765+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:05:58.075182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023448+0000","last_change":"2024-03-23T09:43:47.023448+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023448+0000","last_undegraded":"2024-03-23T09:43:47.023448+0000","last_fullsized":"2024-03-23T09:43:47.023448+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:56.576713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021932+0000","last_change":"2024-03-23T09:43:47.021932+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021932+0000","last_undegraded":"2024-03-23T09:43:47.021932+0000","last_fullsized":"2024-03-23T09:43:47.021932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:21.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023522+0000","last_change":"2024-03-23T09:43:47.023522+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023522+0000","last_undegraded":"2024-03-23T09:43:47.023522+0000","last_fullsized":"2024-03-23T09:43:47.023522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:50:34.378460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025211+0000","last_change":"2024-03-23T09:43:47.025211+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025211+0000","last_undegraded":"2024-03-23T09:43:47.025211+0000","last_fullsized":"2024-03-23T09:43:47.025211+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:49.207297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026927+0000","last_change":"2024-03-23T09:43:47.026927+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026927+0000","last_undegraded":"2024-03-23T09:43:47.026927+0000","last_fullsized":"2024-03-23T09:43:47.026927+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:59.729962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024967+0000","last_change":"2024-03-23T09:43:47.024967+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024967+0000","last_undegraded":"2024-03-23T09:43:47.024967+0000","last_fullsized":"2024-03-23T09:43:47.024967+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:07:44.532537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025472+0000","last_change":"2024-03-23T09:43:47.025472+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025472+0000","last_undegraded":"2024-03-23T09:43:47.025472+0000","last_fullsized":"2024-03-23T09:43:47.025472+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:19:12.228744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026145+0000","last_change":"2024-03-23T09:43:47.026145+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026145+0000","last_undegraded":"2024-03-23T09:43:47.026145+0000","last_fullsized":"2024-03-23T09:43:47.026145+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:47:24.835835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024831+0000","last_change":"2024-03-23T09:43:47.024831+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024831+0000","last_undegraded":"2024-03-23T09:43:47.024831+0000","last_fullsized":"2024-03-23T09:43:47.024831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:10:55.583385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024086+0000","last_change":"2024-03-23T09:43:47.024086+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024086+0000","last_undegraded":"2024-03-23T09:43:47.024086+0000","last_fullsized":"2024-03-23T09:43:47.024086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:54:41.177596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.027386+0000","last_change":"2024-03-23T09:43:47.027386+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.027386+0000","last_undegraded":"2024-03-23T09:43:47.027386+0000","last_fullsized":"2024-03-23T09:43:47.027386+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:08.737490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032083+0000","last_change":"2024-03-23T09:43:47.032083+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032083+0000","last_undegraded":"2024-03-23T09:43:47.032083+0000","last_fullsized":"2024-03-23T09:43:47.032083+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:45:37.390159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024006+0000","last_change":"2024-03-23T09:43:47.024006+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024006+0000","last_undegraded":"2024-03-23T09:43:47.024006+0000","last_fullsized":"2024-03-23T09:43:47.024006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:11.106544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026420+0000","last_change":"2024-03-23T09:43:47.026420+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026420+0000","last_undegraded":"2024-03-23T09:43:47.026420+0000","last_fullsized":"2024-03-23T09:43:47.026420+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024600+0000","last_change":"2024-03-23T09:43:47.024600+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024600+0000","last_undegraded":"2024-03-23T09:43:47.024600+0000","last_fullsized":"2024-03-23T09:43:47.024600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:04.150928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023232+0000","last_change":"2024-03-23T09:43:47.023232+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023232+0000","last_undegraded":"2024-03-23T09:43:47.023232+0000","last_fullsized":"2024-03-23T09:43:47.023232+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026157+0000","last_change":"2024-03-23T09:43:47.026157+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026157+0000","last_undegraded":"2024-03-23T09:43:47.026157+0000","last_fullsized":"2024-03-23T09:43:47.026157+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:46:16.837675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021430+0000","last_change":"2024-03-23T09:43:47.021430+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021430+0000","last_undegraded":"2024-03-23T09:43:47.021430+0000","last_fullsized":"2024-03-23T09:43:47.021430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:02:58.178256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023483+0000","last_change":"2024-03-23T09:43:47.023483+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023483+0000","last_undegraded":"2024-03-23T09:43:47.023483+0000","last_fullsized":"2024-03-23T09:43:47.023483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.038474+0000","last_change":"2024-03-23T09:43:47.036626+0000","last_active":"2024-03-23T09:43:47.038474+0000","last_peered":"2024-03-23T09:43:47.038474+0000","last_clean":"2024-03-23T09:43:47.038474+0000","last_became_active":"2024-03-23T09:43:47.036383+0000","last_became_peered":"2024-03-23T09:43:47.036383+0000","last_unstale":"2024-03-23T09:43:47.038474+0000","last_undegraded":"2024-03-23T09:43:47.038474+0000","last_fullsized":"2024-03-23T09:43:47.038474+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:18.815610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.020741+0000","last_change":"2024-03-23T09:43:47.020741+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.020741+0000","last_undegraded":"2024-03-23T09:43:47.020741+0000","last_fullsized":"2024-03-23T09:43:47.020741+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:21.051130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.038950+0000","last_change":"2024-03-23T09:43:47.037068+0000","last_active":"2024-03-23T09:43:47.038950+0000","last_peered":"2024-03-23T09:43:47.038950+0000","last_clean":"2024-03-23T09:43:47.038950+0000","last_became_active":"2024-03-23T09:43:47.036452+0000","last_became_peered":"2024-03-23T09:43:47.036452+0000","last_unstale":"2024-03-23T09:43:47.038950+0000","last_undegraded":"2024-03-23T09:43:47.038950+0000","last_fullsized":"2024-03-23T09:43:47.038950+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:26.308593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021823+0000","last_change":"2024-03-23T09:43:47.021823+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021823+0000","last_undegraded":"2024-03-23T09:43:47.021823+0000","last_fullsized":"2024-03-23T09:43:47.021823+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.033586+0000","last_change":"2024-03-23T09:43:47.029691+0000","last_active":"2024-03-23T09:43:47.033586+0000","last_peered":"2024-03-23T09:43:47.033586+0000","last_clean":"2024-03-23T09:43:47.033586+0000","last_became_active":"2024-03-23T09:43:47.029427+0000","last_became_peered":"2024-03-23T09:43:47.029427+0000","last_unstale":"2024-03-23T09:43:47.033586+0000","last_undegraded":"2024-03-23T09:43:47.033586+0000","last_fullsized":"2024-03-23T09:43:47.033586+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023508+0000","last_change":"2024-03-23T09:43:47.023508+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023508+0000","last_undegraded":"2024-03-23T09:43:47.023508+0000","last_fullsized":"2024-03-23T09:43:47.023508+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:45.147489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040183+0000","last_change":"2024-03-23T09:43:47.039140+0000","last_active":"2024-03-23T09:43:47.040183+0000","last_peered":"2024-03-23T09:43:47.040183+0000","last_clean":"2024-03-23T09:43:47.040183+0000","last_became_active":"2024-03-23T09:43:47.038851+0000","last_became_peered":"2024-03-23T09:43:47.038851+0000","last_unstale":"2024-03-23T09:43:47.040183+0000","last_undegraded":"2024-03-23T09:43:47.040183+0000","last_fullsized":"2024-03-23T09:43:47.040183+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:18.815610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024459+0000","last_change":"2024-03-23T09:43:47.024459+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024459+0000","last_undegraded":"2024-03-23T09:43:47.024459+0000","last_fullsized":"2024-03-23T09:43:47.024459+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:28:32.181328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037202+0000","last_change":"2024-03-23T09:43:47.031421+0000","last_active":"2024-03-23T09:43:47.037202+0000","last_peered":"2024-03-23T09:43:47.037202+0000","last_clean":"2024-03-23T09:43:47.037202+0000","last_became_active":"2024-03-23T09:43:47.031157+0000","last_became_peered":"2024-03-23T09:43:47.031157+0000","last_unstale":"2024-03-23T09:43:47.037202+0000","last_undegraded":"2024-03-23T09:43:47.037202+0000","last_fullsized":"2024-03-23T09:43:47.037202+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:49.393118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025198+0000","last_change":"2024-03-23T09:43:47.025198+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025198+0000","last_undegraded":"2024-03-23T09:43:47.025198+0000","last_fullsized":"2024-03-23T09:43:47.025198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:42:29.904309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.035316+0000","last_change":"2024-03-23T09:43:47.034482+0000","last_active":"2024-03-23T09:43:47.035316+0000","last_peered":"2024-03-23T09:43:47.035316+0000","last_clean":"2024-03-23T09:43:47.035316+0000","last_became_active":"2024-03-23T09:43:47.034190+0000","last_became_peered":"2024-03-23T09:43:47.034190+0000","last_unstale":"2024-03-23T09:43:47.035316+0000","last_undegraded":"2024-03-23T09:43:47.035316+0000","last_fullsized":"2024-03-23T09:43:47.035316+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:27:00.248727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023485+0000","last_change":"2024-03-23T09:43:47.023485+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023485+0000","last_undegraded":"2024-03-23T09:43:47.023485+0000","last_fullsized":"2024-03-23T09:43:47.023485+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:59.729962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036543+0000","last_change":"2024-03-23T09:43:47.034996+0000","last_active":"2024-03-23T09:43:47.036543+0000","last_peered":"2024-03-23T09:43:47.036543+0000","last_clean":"2024-03-23T09:43:47.036543+0000","last_became_active":"2024-03-23T09:43:47.034715+0000","last_became_peered":"2024-03-23T09:43:47.034715+0000","last_unstale":"2024-03-23T09:43:47.036543+0000","last_undegraded":"2024-03-23T09:43:47.036543+0000","last_fullsized":"2024-03-23T09:43:47.036543+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022665+0000","last_change":"2024-03-23T09:43:47.022665+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022665+0000","last_undegraded":"2024-03-23T09:43:47.022665+0000","last_fullsized":"2024-03-23T09:43:47.022665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:25:06.163966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039588+0000","last_change":"2024-03-23T09:43:47.036762+0000","last_active":"2024-03-23T09:43:47.039588+0000","last_peered":"2024-03-23T09:43:47.039588+0000","last_clean":"2024-03-23T09:43:47.039588+0000","last_became_active":"2024-03-23T09:43:47.036510+0000","last_became_peered":"2024-03-23T09:43:47.036510+0000","last_unstale":"2024-03-23T09:43:47.039588+0000","last_undegraded":"2024-03-23T09:43:47.039588+0000","last_fullsized":"2024-03-23T09:43:47.039588+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:58.725092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023652+0000","last_change":"2024-03-23T09:43:47.023652+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023652+0000","last_undegraded":"2024-03-23T09:43:47.023652+0000","last_fullsized":"2024-03-23T09:43:47.023652+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:36:27.003657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037419+0000","last_change":"2024-03-23T09:43:47.034507+0000","last_active":"2024-03-23T09:43:47.037419+0000","last_peered":"2024-03-23T09:43:47.037419+0000","last_clean":"2024-03-23T09:43:47.037419+0000","last_became_active":"2024-03-23T09:43:47.034260+0000","last_became_peered":"2024-03-23T09:43:47.034260+0000","last_unstale":"2024-03-23T09:43:47.037419+0000","last_undegraded":"2024-03-23T09:43:47.037419+0000","last_fullsized":"2024-03-23T09:43:47.037419+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025795+0000","last_change":"2024-03-23T09:43:47.025795+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025795+0000","last_undegraded":"2024-03-23T09:43:47.025795+0000","last_fullsized":"2024-03-23T09:43:47.025795+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037161+0000","last_change":"2024-03-23T09:43:47.036141+0000","last_active":"2024-03-23T09:43:47.037161+0000","last_peered":"2024-03-23T09:43:47.037161+0000","last_clean":"2024-03-23T09:43:47.037161+0000","last_became_active":"2024-03-23T09:43:47.035805+0000","last_became_peered":"2024-03-23T09:43:47.035805+0000","last_unstale":"2024-03-23T09:43:47.037161+0000","last_undegraded":"2024-03-23T09:43:47.037161+0000","last_fullsized":"2024-03-23T09:43:47.037161+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:22.465435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023205+0000","last_change":"2024-03-23T09:43:47.023205+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023205+0000","last_undegraded":"2024-03-23T09:43:47.023205+0000","last_fullsized":"2024-03-23T09:43:47.023205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:28:37.758076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037387+0000","last_change":"2024-03-23T09:43:47.032774+0000","last_active":"2024-03-23T09:43:47.037387+0000","last_peered":"2024-03-23T09:43:47.037387+0000","last_clean":"2024-03-23T09:43:47.037387+0000","last_became_active":"2024-03-23T09:43:47.032528+0000","last_became_peered":"2024-03-23T09:43:47.032528+0000","last_unstale":"2024-03-23T09:43:47.037387+0000","last_undegraded":"2024-03-23T09:43:47.037387+0000","last_fullsized":"2024-03-23T09:43:47.037387+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026681+0000","last_change":"2024-03-23T09:43:47.026681+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026681+0000","last_undegraded":"2024-03-23T09:43:47.026681+0000","last_fullsized":"2024-03-23T09:43:47.026681+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:11.106544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037521+0000","last_change":"2024-03-23T09:43:47.033945+0000","last_active":"2024-03-23T09:43:47.037521+0000","last_peered":"2024-03-23T09:43:47.037521+0000","last_clean":"2024-03-23T09:43:47.037521+0000","last_became_active":"2024-03-23T09:43:47.033037+0000","last_became_peered":"2024-03-23T09:43:47.033037+0000","last_unstale":"2024-03-23T09:43:47.037521+0000","last_undegraded":"2024-03-23T09:43:47.037521+0000","last_fullsized":"2024-03-23T09:43:47.037521+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:28:36.753206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.029812+0000","last_change":"2024-03-23T09:43:47.029812+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.029812+0000","last_undegraded":"2024-03-23T09:43:47.029812+0000","last_fullsized":"2024-03-23T09:43:47.029812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:51:10.516410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036612+0000","last_change":"2024-03-23T09:43:47.034748+0000","last_active":"2024-03-23T09:43:47.036612+0000","last_peered":"2024-03-23T09:43:47.036612+0000","last_clean":"2024-03-23T09:43:47.036612+0000","last_became_active":"2024-03-23T09:43:47.034340+0000","last_became_peered":"2024-03-23T09:43:47.034340+0000","last_unstale":"2024-03-23T09:43:47.036612+0000","last_undegraded":"2024-03-23T09:43:47.036612+0000","last_fullsized":"2024-03-23T09:43:47.036612+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:06.713892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024286+0000","last_change":"2024-03-23T09:43:47.024286+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024286+0000","last_undegraded":"2024-03-23T09:43:47.024286+0000","last_fullsized":"2024-03-23T09:43:47.024286+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039159+0000","last_change":"2024-03-23T09:43:47.034963+0000","last_active":"2024-03-23T09:43:47.039159+0000","last_peered":"2024-03-23T09:43:47.039159+0000","last_clean":"2024-03-23T09:43:47.039159+0000","last_became_active":"2024-03-23T09:43:47.034680+0000","last_became_peered":"2024-03-23T09:43:47.034680+0000","last_unstale":"2024-03-23T09:43:47.039159+0000","last_undegraded":"2024-03-23T09:43:47.039159+0000","last_fullsized":"2024-03-23T09:43:47.039159+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:15:54.128943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023273+0000","last_change":"2024-03-23T09:43:47.023273+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023273+0000","last_undegraded":"2024-03-23T09:43:47.023273+0000","last_fullsized":"2024-03-23T09:43:47.023273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:27:01.253597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040165+0000","last_change":"2024-03-23T09:43:47.038681+0000","last_active":"2024-03-23T09:43:47.040165+0000","last_peered":"2024-03-23T09:43:47.040165+0000","last_clean":"2024-03-23T09:43:47.040165+0000","last_became_active":"2024-03-23T09:43:47.038410+0000","last_became_peered":"2024-03-23T09:43:47.038410+0000","last_unstale":"2024-03-23T09:43:47.040165+0000","last_undegraded":"2024-03-23T09:43:47.040165+0000","last_fullsized":"2024-03-23T09:43:47.040165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:19.400345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024358+0000","last_change":"2024-03-23T09:43:47.024358+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024358+0000","last_undegraded":"2024-03-23T09:43:47.024358+0000","last_fullsized":"2024-03-23T09:43:47.024358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:02:58.178256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.042320+0000","last_change":"2024-03-23T09:43:47.036966+0000","last_active":"2024-03-23T09:43:47.042320+0000","last_peered":"2024-03-23T09:43:47.042320+0000","last_clean":"2024-03-23T09:43:47.042320+0000","last_became_active":"2024-03-23T09:43:47.036691+0000","last_became_peered":"2024-03-23T09:43:47.036691+0000","last_unstale":"2024-03-23T09:43:47.042320+0000","last_undegraded":"2024-03-23T09:43:47.042320+0000","last_fullsized":"2024-03-23T09:43:47.042320+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024769+0000","last_change":"2024-03-23T09:43:47.024769+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024769+0000","last_undegraded":"2024-03-23T09:43:47.024769+0000","last_fullsized":"2024-03-23T09:43:47.024769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037227+0000","last_change":"2024-03-23T09:43:47.036026+0000","last_active":"2024-03-23T09:43:47.037227+0000","last_peered":"2024-03-23T09:43:47.037227+0000","last_clean":"2024-03-23T09:43:47.037227+0000","last_became_active":"2024-03-23T09:43:47.035690+0000","last_became_peered":"2024-03-23T09:43:47.035690+0000","last_unstale":"2024-03-23T09:43:47.037227+0000","last_undegraded":"2024-03-23T09:43:47.037227+0000","last_fullsized":"2024-03-23T09:43:47.037227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:20.475983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.033106+0000","last_change":"2024-03-23T09:43:47.033106+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.033106+0000","last_undegraded":"2024-03-23T09:43:47.033106+0000","last_fullsized":"2024-03-23T09:43:47.033106+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034465+0000","last_change":"2024-03-23T09:43:47.032740+0000","last_active":"2024-03-23T09:43:47.034465+0000","last_peered":"2024-03-23T09:43:47.034465+0000","last_clean":"2024-03-23T09:43:47.034465+0000","last_became_active":"2024-03-23T09:43:47.032456+0000","last_became_peered":"2024-03-23T09:43:47.032456+0000","last_unstale":"2024-03-23T09:43:47.034465+0000","last_undegraded":"2024-03-23T09:43:47.034465+0000","last_fullsized":"2024-03-23T09:43:47.034465+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021027+0000","last_change":"2024-03-23T09:43:47.021027+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021027+0000","last_undegraded":"2024-03-23T09:43:47.021027+0000","last_fullsized":"2024-03-23T09:43:47.021027+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:07:10.261476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040144+0000","last_change":"2024-03-23T09:43:47.038642+0000","last_active":"2024-03-23T09:43:47.040144+0000","last_peered":"2024-03-23T09:43:47.040144+0000","last_clean":"2024-03-23T09:43:47.040144+0000","last_became_active":"2024-03-23T09:43:47.038385+0000","last_became_peered":"2024-03-23T09:43:47.038385+0000","last_unstale":"2024-03-23T09:43:47.040144+0000","last_undegraded":"2024-03-23T09:43:47.040144+0000","last_fullsized":"2024-03-23T09:43:47.040144+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:33.981220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026250+0000","last_change":"2024-03-23T09:43:47.026250+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026250+0000","last_undegraded":"2024-03-23T09:43:47.026250+0000","last_fullsized":"2024-03-23T09:43:47.026250+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:21.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034573+0000","last_change":"2024-03-23T09:43:47.032632+0000","last_active":"2024-03-23T09:43:47.034573+0000","last_peered":"2024-03-23T09:43:47.034573+0000","last_clean":"2024-03-23T09:43:47.034573+0000","last_became_active":"2024-03-23T09:43:47.032375+0000","last_became_peered":"2024-03-23T09:43:47.032375+0000","last_unstale":"2024-03-23T09:43:47.034573+0000","last_undegraded":"2024-03-23T09:43:47.034573+0000","last_fullsized":"2024-03-23T09:43:47.034573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.029300+0000","last_change":"2024-03-23T09:43:47.029300+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.029300+0000","last_undegraded":"2024-03-23T09:43:47.029300+0000","last_fullsized":"2024-03-23T09:43:47.029300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:14:35.775608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036589+0000","last_change":"2024-03-23T09:43:47.030434+0000","last_active":"2024-03-23T09:43:47.036589+0000","last_peered":"2024-03-23T09:43:47.036589+0000","last_clean":"2024-03-23T09:43:47.036589+0000","last_became_active":"2024-03-23T09:43:47.030164+0000","last_became_peered":"2024-03-23T09:43:47.030164+0000","last_unstale":"2024-03-23T09:43:47.036589+0000","last_undegraded":"2024-03-23T09:43:47.036589+0000","last_fullsized":"2024-03-23T09:43:47.036589+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:09.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022414+0000","last_change":"2024-03-23T09:43:47.022414+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022414+0000","last_undegraded":"2024-03-23T09:43:47.022414+0000","last_fullsized":"2024-03-23T09:43:47.022414+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:22:26.903375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034938+0000","last_change":"2024-03-23T09:43:47.031235+0000","last_active":"2024-03-23T09:43:47.034938+0000","last_peered":"2024-03-23T09:43:47.034938+0000","last_clean":"2024-03-23T09:43:47.034938+0000","last_became_active":"2024-03-23T09:43:47.030949+0000","last_became_peered":"2024-03-23T09:43:47.030949+0000","last_unstale":"2024-03-23T09:43:47.034938+0000","last_undegraded":"2024-03-23T09:43:47.034938+0000","last_fullsized":"2024-03-23T09:43:47.034938+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:58:56.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022818+0000","last_change":"2024-03-23T09:43:47.022818+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022818+0000","last_undegraded":"2024-03-23T09:43:47.022818+0000","last_fullsized":"2024-03-23T09:43:47.022818+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040173+0000","last_change":"2024-03-23T09:43:47.038838+0000","last_active":"2024-03-23T09:43:47.040173+0000","last_peered":"2024-03-23T09:43:47.040173+0000","last_clean":"2024-03-23T09:43:47.040173+0000","last_became_active":"2024-03-23T09:43:47.038574+0000","last_became_peered":"2024-03-23T09:43:47.038574+0000","last_unstale":"2024-03-23T09:43:47.040173+0000","last_undegraded":"2024-03-23T09:43:47.040173+0000","last_fullsized":"2024-03-23T09:43:47.040173+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.030328+0000","last_change":"2024-03-23T09:43:47.030328+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.030328+0000","last_undegraded":"2024-03-23T09:43:47.030328+0000","last_fullsized":"2024-03-23T09:43:47.030328+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:49.207297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.031785+0000","last_change":"2024-03-23T09:43:47.029629+0000","last_active":"2024-03-23T09:43:47.031785+0000","last_peered":"2024-03-23T09:43:47.031785+0000","last_clean":"2024-03-23T09:43:47.031785+0000","last_became_active":"2024-03-23T09:43:47.029338+0000","last_became_peered":"2024-03-23T09:43:47.029338+0000","last_unstale":"2024-03-23T09:43:47.031785+0000","last_undegraded":"2024-03-23T09:43:47.031785+0000","last_fullsized":"2024-03-23T09:43:47.031785+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:15:54.128943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032692+0000","last_change":"2024-03-23T09:43:47.032692+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032692+0000","last_undegraded":"2024-03-23T09:43:47.032692+0000","last_fullsized":"2024-03-23T09:43:47.032692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036684+0000","last_change":"2024-03-23T09:43:47.035203+0000","last_active":"2024-03-23T09:43:47.036684+0000","last_peered":"2024-03-23T09:43:47.036684+0000","last_clean":"2024-03-23T09:43:47.036684+0000","last_became_active":"2024-03-23T09:43:47.034912+0000","last_became_peered":"2024-03-23T09:43:47.034912+0000","last_unstale":"2024-03-23T09:43:47.036684+0000","last_undegraded":"2024-03-23T09:43:47.036684+0000","last_fullsized":"2024-03-23T09:43:47.036684+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:36:25.998787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.019914+0000","last_change":"2024-03-23T09:43:47.019914+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.019914+0000","last_undegraded":"2024-03-23T09:43:47.019914+0000","last_fullsized":"2024-03-23T09:43:47.019914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034026+0000","last_change":"2024-03-23T09:43:47.029065+0000","last_active":"2024-03-23T09:43:47.034026+0000","last_peered":"2024-03-23T09:43:47.034026+0000","last_clean":"2024-03-23T09:43:47.034026+0000","last_became_active":"2024-03-23T09:43:47.028797+0000","last_became_peered":"2024-03-23T09:43:47.028797+0000","last_unstale":"2024-03-23T09:43:47.034026+0000","last_undegraded":"2024-03-23T09:43:47.034026+0000","last_fullsized":"2024-03-23T09:43:47.034026+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:32:33.947908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022836+0000","last_change":"2024-03-23T09:43:47.022836+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022836+0000","last_undegraded":"2024-03-23T09:43:47.022836+0000","last_fullsized":"2024-03-23T09:43:47.022836+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:19:12.228744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040108+0000","last_change":"2024-03-23T09:43:47.038126+0000","last_active":"2024-03-23T09:43:47.040108+0000","last_peered":"2024-03-23T09:43:47.040108+0000","last_clean":"2024-03-23T09:43:47.040108+0000","last_became_active":"2024-03-23T09:43:47.037850+0000","last_became_peered":"2024-03-23T09:43:47.037850+0000","last_unstale":"2024-03-23T09:43:47.040108+0000","last_undegraded":"2024-03-23T09:43:47.040108+0000","last_fullsized":"2024-03-23T09:43:47.040108+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023991+0000","last_change":"2024-03-23T09:43:47.023991+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023991+0000","last_undegraded":"2024-03-23T09:43:47.023991+0000","last_fullsized":"2024-03-23T09:43:47.023991+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:59.768596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037679+0000","last_change":"2024-03-23T09:43:47.036954+0000","last_active":"2024-03-23T09:43:47.037679+0000","last_peered":"2024-03-23T09:43:47.037679+0000","last_clean":"2024-03-23T09:43:47.037679+0000","last_became_active":"2024-03-23T09:43:47.036655+0000","last_became_peered":"2024-03-23T09:43:47.036655+0000","last_unstale":"2024-03-23T09:43:47.037679+0000","last_undegraded":"2024-03-23T09:43:47.037679+0000","last_fullsized":"2024-03-23T09:43:47.037679+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:05:51.129365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023834+0000","last_change":"2024-03-23T09:43:47.023834+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023834+0000","last_undegraded":"2024-03-23T09:43:47.023834+0000","last_fullsized":"2024-03-23T09:43:47.023834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:45.147489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037249+0000","last_change":"2024-03-23T09:43:47.032771+0000","last_active":"2024-03-23T09:43:47.037249+0000","last_peered":"2024-03-23T09:43:47.037249+0000","last_clean":"2024-03-23T09:43:47.037249+0000","last_became_active":"2024-03-23T09:43:47.032489+0000","last_became_peered":"2024-03-23T09:43:47.032489+0000","last_unstale":"2024-03-23T09:43:47.037249+0000","last_undegraded":"2024-03-23T09:43:47.037249+0000","last_fullsized":"2024-03-23T09:43:47.037249+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:48:49.438386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.019914+0000","last_change":"2024-03-23T09:43:47.019914+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.019914+0000","last_undegraded":"2024-03-23T09:43:47.019914+0000","last_fullsized":"2024-03-23T09:43:47.019914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:24:18.285067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.033031+0000","last_change":"2024-03-23T09:43:47.030522+0000","last_active":"2024-03-23T09:43:47.033031+0000","last_peered":"2024-03-23T09:43:47.033031+0000","last_clean":"2024-03-23T09:43:47.033031+0000","last_became_active":"2024-03-23T09:43:47.030216+0000","last_became_peered":"2024-03-23T09:43:47.030216+0000","last_unstale":"2024-03-23T09:43:47.033031+0000","last_undegraded":"2024-03-23T09:43:47.033031+0000","last_fullsized":"2024-03-23T09:43:47.033031+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:32:33.947908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022218+0000","last_change":"2024-03-23T09:43:47.022218+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022218+0000","last_undegraded":"2024-03-23T09:43:47.022218+0000","last_fullsized":"2024-03-23T09:43:47.022218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:30:34.056749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039369+0000","last_change":"2024-03-23T09:43:47.038282+0000","last_active":"2024-03-23T09:43:47.039369+0000","last_peered":"2024-03-23T09:43:47.039369+0000","last_clean":"2024-03-23T09:43:47.039369+0000","last_became_active":"2024-03-23T09:43:47.037787+0000","last_became_peered":"2024-03-23T09:43:47.037787+0000","last_unstale":"2024-03-23T09:43:47.039369+0000","last_undegraded":"2024-03-23T09:43:47.039369+0000","last_fullsized":"2024-03-23T09:43:47.039369+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:10.101674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021303+0000","last_change":"2024-03-23T09:43:47.021303+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021303+0000","last_undegraded":"2024-03-23T09:43:47.021303+0000","last_fullsized":"2024-03-23T09:43:47.021303+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:52:24.758413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039546+0000","last_change":"2024-03-23T09:43:47.037606+0000","last_active":"2024-03-23T09:43:47.039546+0000","last_peered":"2024-03-23T09:43:47.039546+0000","last_clean":"2024-03-23T09:43:47.039546+0000","last_became_active":"2024-03-23T09:43:47.037315+0000","last_became_peered":"2024-03-23T09:43:47.037315+0000","last_unstale":"2024-03-23T09:43:47.039546+0000","last_undegraded":"2024-03-23T09:43:47.039546+0000","last_fullsized":"2024-03-23T09:43:47.039546+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:22.192970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":200,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-23T09:43:46.013864+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:43:46.013864+0000","last_peered":"2024-03-23T09:43:46.013864+0000","last_clean":"2024-03-23T09:43:46.013864+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:43:46.013864+0000","last_undegraded":"2024-03-23T09:43:46.013864+0000","last_fullsized":"2024-03-23T09:43:46.013864+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1265664,"data_stored":864722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69299999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.62}]},{"osd":2,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.65900000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72699999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56999999999999995}]},{"osd":3,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.433,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58399999999999996}]},{"osd":4,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.377,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72099999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45300000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66500000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71499999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73799999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53700000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":267468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.499,"15min":0.498},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69299999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.54100000000000004}]},{"osd":1,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.40200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.42699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.627}]},{"osd":2,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.41299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.46100000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.496}]},{"osd":3,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52900000000000003,"15min":0.52300000000000002},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.73199999999999998,"5min":0.752,"15min":0.752},"last":0.60999999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.38700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.67500000000000004}]},{"osd":5,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.72399999999999998,"5min":0.747,"15min":0.747},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":268087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48099999999999998,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.42199999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52100000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52900000000000003,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.434,"5min":0.376,"15min":0.376},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.50700000000000001,"15min":0.49299999999999999},"min":{"1min":0.46700000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.46800000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.54100000000000004}]},{"osd":4,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47999999999999998,"15min":0.47699999999999998},"min":{"1min":0.33300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.439},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.374,"15min":0.374},"max":{"1min":0.69999999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63400000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.34799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.749,"15min":0.749},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.51800000000000002},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70099999999999996,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.60899999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.44600000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71499999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.621}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":268087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.69899999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.49199999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.46400000000000002},"min":{"1min":0.35699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.498,"15min":0.498},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.40600000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.42999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.752,"5min":0.755,"15min":0.755},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.747,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.50800000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70399999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.30199999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68799999999999994,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.308,"5min":0.253,"15min":0.253},"max":{"1min":0.68400000000000005,"5min":1.008,"15min":1.008},"last":0.36499999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.34000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70099999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.32400000000000001}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65800000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56200000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":671744,"data_stored":274354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.307,"15min":0.30499999999999999},"min":{"1min":0.25900000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30399999999999999,"15min":0.30199999999999999},"min":{"1min":0.25600000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.42899999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.437},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.55100000000000005}]},{"osd":2,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.313,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76500000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53100000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.73499999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.47399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81499999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.42899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.70999999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.81000000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.41799999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71499999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.67800000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.626},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.59999999999999998},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.71599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":671744,"data_stored":274354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.34000000000000002,"15min":0.34399999999999997},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.48799999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.25,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48799999999999999,"5min":0.872,"15min":0.872},"last":0.23000000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.629,"15min":0.629},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58599999999999997,"5min":0.746,"15min":0.746},"last":0.34100000000000003}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.47399999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.40200000000000002,"15min":0.40699999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55800000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.32300000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72499999999999998,"5min":1.216,"15min":1.216},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.753,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.624,"15min":0.63200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.71599999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75800000000000001,"5min":1.101,"15min":1.101},"last":0.66800000000000004}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.70699999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.63800000000000001,"15min":0.64800000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.72699999999999998,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.76500000000000001,"5min":1.155,"15min":1.155},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72099999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":659456,"data_stored":268087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33600000000000002,"15min":0.33300000000000002},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.53300000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61899999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.30299999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48399999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.374,"15min":0.37},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.55200000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41099999999999998,"15min":0.40699999999999997},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70499999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60899999999999999,"5min":2.75,"15min":2.75},"last":0.41499999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.45500000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91100000000000003,"5min":2.778,"15min":2.778},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68999999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.46300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88100000000000001,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.441},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.628}]},{"osd":7,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65400000000000003,"15min":0.64700000000000002},"min":{"1min":0.51800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":0.92400000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.76000000000000001,"15min":0.77800000000000002},"min":{"1min":0.44600000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.85099999999999998,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.52800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1261568,"data_stored":864103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32200000000000001,"15min":0.31900000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.499,"5min":0.504,"15min":0.504},"last":0.316},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.34499999999999997},"min":{"1min":0.25900000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.51300000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.30099999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36099999999999999,"15min":0.35899999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36099999999999999},"min":{"1min":0.27000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.33700000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.379,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.42299999999999999,"15min":0.41999999999999998},"min":{"1min":0.32100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.40999999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53600000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.81499999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54600000000000004}]},{"osd":6,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83299999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.88400000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55500000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:43:50.389 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:43:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.525+0000 7f481c5a4640 1 Processor -- start 2024-03-23T09:43:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.525+0000 7f481c5a4640 1 -- start start 2024-03-23T09:43:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.526+0000 7f481c5a4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48140cd090 0x7f48140cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.526+0000 7f481c5a4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f48140cb8c0 0x7f48140af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.526+0000 7f481c5a4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48140cd9b0 con 0x7f481415ab50 2024-03-23T09:43:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.526+0000 7f481c5a4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48140cbd50 con 0x7f48140cb8c0 2024-03-23T09:43:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.526+0000 7f481c5a4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48140cbf50 con 0x7f48140cd090 2024-03-23T09:43:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.526+0000 7f481ab41640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f48140cb8c0 0x7f48140af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.526+0000 7f481ab41640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f48140cb8c0 0x7f48140af9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39102/0 (socket says 172.21.15.47:39102) 2024-03-23T09:43:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.526+0000 7f4819b3f640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f481415ab50 0x7f48140cb1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36042/0 (socket says 172.21.15.47:36042) 2024-03-23T09:43:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.526+0000 7f481ab41640 1 -- 172.21.15.47:0/4159302648 learned_addr learned my addr 172.21.15.47:0/4159302648 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.527+0000 7f481a340640 1 --2- 172.21.15.47:0/4159302648 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48140cd090 0x7f48140cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.527+0000 7f481ab41640 1 -- 172.21.15.47:0/4159302648 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48140cd090 msgr2=0x7f48140cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.527+0000 7f481ab41640 1 --2- 172.21.15.47:0/4159302648 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48140cd090 0x7f48140cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.527+0000 7f481ab41640 1 -- 172.21.15.47:0/4159302648 >> v1:172.21.15.47:6789/0 conn(0x7f481415ab50 legacy=0x7f48140cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.527+0000 7f481ab41640 1 -- 172.21.15.47:0/4159302648 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48140cc230 con 0x7f48140cb8c0 2024-03-23T09:43:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.527+0000 7f481933e640 1 -- 172.21.15.47:0/4159302648 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294225176 0 0) 0x7f48140cd9b0 con 0x7f481415ab50 2024-03-23T09:43:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.528+0000 7f481ab41640 1 --2- 172.21.15.47:0/4159302648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48140cb8c0 0x7f48140af9f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4804009e50 tx=0x7f480402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a1f94b656624b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.528+0000 7f481933e640 1 -- 172.21.15.47:0/4159302648 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480402cb90 con 0x7f48140cb8c0 2024-03-23T09:43:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.528+0000 7f481933e640 1 -- 172.21.15.47:0/4159302648 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480402ccf0 con 0x7f48140cb8c0 2024-03-23T09:43:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.528+0000 7f481933e640 1 -- 172.21.15.47:0/4159302648 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804034610 con 0x7f48140cb8c0 2024-03-23T09:43:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.528+0000 7f481c5a4640 1 -- 172.21.15.47:0/4159302648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48140cb8c0 msgr2=0x7f48140af9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.528+0000 7f481c5a4640 1 --2- 172.21.15.47:0/4159302648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48140cb8c0 0x7f48140af9f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4804009e50 tx=0x7f480402bff0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.528+0000 7f481c5a4640 1 -- 172.21.15.47:0/4159302648 shutdown_connections 2024-03-23T09:43:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.528+0000 7f481c5a4640 1 --2- 172.21.15.47:0/4159302648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48140cb8c0 0x7f48140af9f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.528+0000 7f481c5a4640 1 --2- 172.21.15.47:0/4159302648 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48140cd090 0x7f48140cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.528+0000 7f481c5a4640 1 -- 172.21.15.47:0/4159302648 >> 172.21.15.47:0/4159302648 conn(0x7f48140d06a0 msgr2=0x7f48140d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.528+0000 7f481c5a4640 1 -- 172.21.15.47:0/4159302648 shutdown_connections 2024-03-23T09:43:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.528+0000 7f481c5a4640 1 -- 172.21.15.47:0/4159302648 wait complete. 2024-03-23T09:43:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.529+0000 7f481c5a4640 1 Processor -- start 2024-03-23T09:43:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.530+0000 7f481c5a4640 1 -- start start 2024-03-23T09:43:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.530+0000 7f481c5a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48140cb8c0 0x7f4814153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.530+0000 7f481c5a4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48140cd090 0x7f48140c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.530+0000 7f481c5a4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f481415ab50 0x7f48140b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.530+0000 7f481c5a4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481406c110 con 0x7f48140cd090 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.530+0000 7f481c5a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481406ce00 con 0x7f48140cb8c0 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.530+0000 7f481c5a4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481406c840 con 0x7f481415ab50 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.530+0000 7f4819b3f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48140cd090 0x7f48140c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.530+0000 7f4819b3f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48140cd090 0x7f48140c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39112/0 (socket says 172.21.15.47:39112) 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.530+0000 7f4819b3f640 1 -- 172.21.15.47:0/2393617577 learned_addr learned my addr 172.21.15.47:0/2393617577 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f481ab41640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f481415ab50 0x7f48140b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f4819b3f640 1 -- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f481415ab50 msgr2=0x7f48140b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f481a340640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48140cb8c0 0x7f4814153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f4819b3f640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f481415ab50 0x7f48140b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f4819b3f640 1 -- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48140cb8c0 msgr2=0x7f4814153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f4819b3f640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48140cb8c0 0x7f4814153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f4819b3f640 1 -- 172.21.15.47:0/2393617577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4804009b30 con 0x7f48140cd090 2024-03-23T09:43:50.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f481ab41640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f481415ab50 0x7f48140b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:50.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f481a340640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48140cb8c0 0x7f4814153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:50.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f4819b3f640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48140cd090 0x7f48140c9f60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f480800c410 tx=0x7f480800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:50.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f4812ffd640 1 -- 172.21.15.47:0/2393617577 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48080074e0 con 0x7f48140cd090 2024-03-23T09:43:50.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f4812ffd640 1 -- 172.21.15.47:0/2393617577 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4808007660 con 0x7f48140cd090 2024-03-23T09:43:50.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f481c5a4640 1 -- 172.21.15.47:0/2393617577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481406cb20 con 0x7f48140cd090 2024-03-23T09:43:50.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f4812ffd640 1 -- 172.21.15.47:0/2393617577 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808004a20 con 0x7f48140cd090 2024-03-23T09:43:50.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.531+0000 7f481c5a4640 1 -- 172.21.15.47:0/2393617577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481406bbf0 con 0x7f48140cd090 2024-03-23T09:43:50.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.532+0000 7f481c5a4640 1 -- 172.21.15.47:0/2393617577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481406be10 con 0x7f48140cd090 2024-03-23T09:43:50.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.532+0000 7f4812ffd640 1 -- 172.21.15.47:0/2393617577 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4808004b80 con 0x7f48140cd090 2024-03-23T09:43:50.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.533+0000 7f4812ffd640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f480006ac00 0x7f480006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.533+0000 7f4812ffd640 1 -- 172.21.15.47:0/2393617577 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4808089140 con 0x7f48140cd090 2024-03-23T09:43:50.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.533+0000 7f481a340640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f480006ac00 0x7f480006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.534+0000 7f481a340640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f480006ac00 0x7f480006d0c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f480c006fd0 tx=0x7f480c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:50.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.537+0000 7f4812ffd640 1 -- 172.21.15.47:0/2393617577 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4808057db0 con 0x7f48140cd090 2024-03-23T09:43:50.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.711+0000 7f481c5a4640 1 -- 172.21.15.47:0/2393617577 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48140ccad0 con 0x7f480006ac00 2024-03-23T09:43:50.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.719+0000 7f4812ffd640 1 -- 172.21.15.47:0/2393617577 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286507 (secure 0 0 0) 0x7f480c009e10 con 0x7f480006ac00 2024-03-23T09:43:50.721 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:50.722 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:43:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.722+0000 7f481c5a4640 1 -- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f480006ac00 msgr2=0x7f480006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.722+0000 7f481c5a4640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f480006ac00 0x7f480006d0c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f480c006fd0 tx=0x7f480c008040 comp rx=0 tx=0).stop 2024-03-23T09:43:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.722+0000 7f481c5a4640 1 -- 172.21.15.47:0/2393617577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48140cd090 msgr2=0x7f48140c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.722+0000 7f481c5a4640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48140cd090 0x7f48140c9f60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f480800c410 tx=0x7f480800c8d0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.722+0000 7f481c5a4640 1 -- 172.21.15.47:0/2393617577 shutdown_connections 2024-03-23T09:43:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.722+0000 7f481c5a4640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f480006ac00 0x7f480006d0c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.722+0000 7f481c5a4640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f481415ab50 0x7f48140b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.722+0000 7f481c5a4640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48140cd090 0x7f48140c9f60 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.722+0000 7f481c5a4640 1 --2- 172.21.15.47:0/2393617577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48140cb8c0 0x7f4814153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.722+0000 7f481c5a4640 1 -- 172.21.15.47:0/2393617577 >> 172.21.15.47:0/2393617577 conn(0x7f48140d06a0 msgr2=0x7f48140c6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.722+0000 7f481c5a4640 1 -- 172.21.15.47:0/2393617577 shutdown_connections 2024-03-23T09:43:50.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.722+0000 7f481c5a4640 1 -- 172.21.15.47:0/2393617577 wait complete. 2024-03-23T09:43:50.732 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-03-23T09:43:50.378328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238916,"kb_used_data":6352,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754735804,"statfs":{"total":773094113280,"available":772849463296,"internally_reserved":0,"allocated":6504448,"data_stored":3349262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.367743"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.028360+0000","last_change":"2024-03-23T09:43:47.028360+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.028360+0000","last_undegraded":"2024-03-23T09:43:47.028360+0000","last_fullsized":"2024-03-23T09:43:47.028360+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:54:41.177596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021545+0000","last_change":"2024-03-23T09:43:47.021545+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021545+0000","last_undegraded":"2024-03-23T09:43:47.021545+0000","last_fullsized":"2024-03-23T09:43:47.021545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:36.262351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.027083+0000","last_change":"2024-03-23T09:43:47.027083+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.027083+0000","last_undegraded":"2024-03-23T09:43:47.027083+0000","last_fullsized":"2024-03-23T09:43:47.027083+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:22:26.903375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032839+0000","last_change":"2024-03-23T09:43:47.032839+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032839+0000","last_undegraded":"2024-03-23T09:43:47.032839+0000","last_fullsized":"2024-03-23T09:43:47.032839+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026471+0000","last_change":"2024-03-23T09:43:47.026471+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026471+0000","last_undegraded":"2024-03-23T09:43:47.026471+0000","last_fullsized":"2024-03-23T09:43:47.026471+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:17.338788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022332+0000","last_change":"2024-03-23T09:43:47.022332+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022332+0000","last_undegraded":"2024-03-23T09:43:47.022332+0000","last_fullsized":"2024-03-23T09:43:47.022332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023985+0000","last_change":"2024-03-23T09:43:47.023985+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023985+0000","last_undegraded":"2024-03-23T09:43:47.023985+0000","last_fullsized":"2024-03-23T09:43:47.023985+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:44:18.230315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021778+0000","last_change":"2024-03-23T09:43:47.021778+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021778+0000","last_undegraded":"2024-03-23T09:43:47.021778+0000","last_fullsized":"2024-03-23T09:43:47.021778+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:55:17.926828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023723+0000","last_change":"2024-03-23T09:43:47.023723+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023723+0000","last_undegraded":"2024-03-23T09:43:47.023723+0000","last_fullsized":"2024-03-23T09:43:47.023723+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:19.820479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.020703+0000","last_change":"2024-03-23T09:43:47.020703+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.020703+0000","last_undegraded":"2024-03-23T09:43:47.020703+0000","last_fullsized":"2024-03-23T09:43:47.020703+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:35:30.364827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.020633+0000","last_change":"2024-03-23T09:43:47.020633+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.020633+0000","last_undegraded":"2024-03-23T09:43:47.020633+0000","last_fullsized":"2024-03-23T09:43:47.020633+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:48:50.443256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032222+0000","last_change":"2024-03-23T09:43:47.032222+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032222+0000","last_undegraded":"2024-03-23T09:43:47.032222+0000","last_fullsized":"2024-03-23T09:43:47.032222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026161+0000","last_change":"2024-03-23T09:43:47.026161+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026161+0000","last_undegraded":"2024-03-23T09:43:47.026161+0000","last_fullsized":"2024-03-23T09:43:47.026161+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023765+0000","last_change":"2024-03-23T09:43:47.023765+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023765+0000","last_undegraded":"2024-03-23T09:43:47.023765+0000","last_fullsized":"2024-03-23T09:43:47.023765+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:05:58.075182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023448+0000","last_change":"2024-03-23T09:43:47.023448+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023448+0000","last_undegraded":"2024-03-23T09:43:47.023448+0000","last_fullsized":"2024-03-23T09:43:47.023448+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:56.576713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021932+0000","last_change":"2024-03-23T09:43:47.021932+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021932+0000","last_undegraded":"2024-03-23T09:43:47.021932+0000","last_fullsized":"2024-03-23T09:43:47.021932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:21.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023522+0000","last_change":"2024-03-23T09:43:47.023522+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023522+0000","last_undegraded":"2024-03-23T09:43:47.023522+0000","last_fullsized":"2024-03-23T09:43:47.023522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:50:34.378460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025211+0000","last_change":"2024-03-23T09:43:47.025211+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025211+0000","last_undegraded":"2024-03-23T09:43:47.025211+0000","last_fullsized":"2024-03-23T09:43:47.025211+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:49.207297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026927+0000","last_change":"2024-03-23T09:43:47.026927+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026927+0000","last_undegraded":"2024-03-23T09:43:47.026927+0000","last_fullsized":"2024-03-23T09:43:47.026927+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:59.729962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024967+0000","last_change":"2024-03-23T09:43:47.024967+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024967+0000","last_undegraded":"2024-03-23T09:43:47.024967+0000","last_fullsized":"2024-03-23T09:43:47.024967+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:07:44.532537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025472+0000","last_change":"2024-03-23T09:43:47.025472+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025472+0000","last_undegraded":"2024-03-23T09:43:47.025472+0000","last_fullsized":"2024-03-23T09:43:47.025472+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:19:12.228744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026145+0000","last_change":"2024-03-23T09:43:47.026145+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026145+0000","last_undegraded":"2024-03-23T09:43:47.026145+0000","last_fullsized":"2024-03-23T09:43:47.026145+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:47:24.835835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024831+0000","last_change":"2024-03-23T09:43:47.024831+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024831+0000","last_undegraded":"2024-03-23T09:43:47.024831+0000","last_fullsized":"2024-03-23T09:43:47.024831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:10:55.583385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024086+0000","last_change":"2024-03-23T09:43:47.024086+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024086+0000","last_undegraded":"2024-03-23T09:43:47.024086+0000","last_fullsized":"2024-03-23T09:43:47.024086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:54:41.177596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.027386+0000","last_change":"2024-03-23T09:43:47.027386+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.027386+0000","last_undegraded":"2024-03-23T09:43:47.027386+0000","last_fullsized":"2024-03-23T09:43:47.027386+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:08.737490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032083+0000","last_change":"2024-03-23T09:43:47.032083+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032083+0000","last_undegraded":"2024-03-23T09:43:47.032083+0000","last_fullsized":"2024-03-23T09:43:47.032083+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:45:37.390159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024006+0000","last_change":"2024-03-23T09:43:47.024006+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024006+0000","last_undegraded":"2024-03-23T09:43:47.024006+0000","last_fullsized":"2024-03-23T09:43:47.024006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:11.106544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026420+0000","last_change":"2024-03-23T09:43:47.026420+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026420+0000","last_undegraded":"2024-03-23T09:43:47.026420+0000","last_fullsized":"2024-03-23T09:43:47.026420+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024600+0000","last_change":"2024-03-23T09:43:47.024600+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024600+0000","last_undegraded":"2024-03-23T09:43:47.024600+0000","last_fullsized":"2024-03-23T09:43:47.024600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:04.150928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023232+0000","last_change":"2024-03-23T09:43:47.023232+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023232+0000","last_undegraded":"2024-03-23T09:43:47.023232+0000","last_fullsized":"2024-03-23T09:43:47.023232+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026157+0000","last_change":"2024-03-23T09:43:47.026157+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026157+0000","last_undegraded":"2024-03-23T09:43:47.026157+0000","last_fullsized":"2024-03-23T09:43:47.026157+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:46:16.837675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021430+0000","last_change":"2024-03-23T09:43:47.021430+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021430+0000","last_undegraded":"2024-03-23T09:43:47.021430+0000","last_fullsized":"2024-03-23T09:43:47.021430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:02:58.178256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023483+0000","last_change":"2024-03-23T09:43:47.023483+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023483+0000","last_undegraded":"2024-03-23T09:43:47.023483+0000","last_fullsized":"2024-03-23T09:43:47.023483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.038474+0000","last_change":"2024-03-23T09:43:47.036626+0000","last_active":"2024-03-23T09:43:47.038474+0000","last_peered":"2024-03-23T09:43:47.038474+0000","last_clean":"2024-03-23T09:43:47.038474+0000","last_became_active":"2024-03-23T09:43:47.036383+0000","last_became_peered":"2024-03-23T09:43:47.036383+0000","last_unstale":"2024-03-23T09:43:47.038474+0000","last_undegraded":"2024-03-23T09:43:47.038474+0000","last_fullsized":"2024-03-23T09:43:47.038474+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:18.815610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.020741+0000","last_change":"2024-03-23T09:43:47.020741+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.020741+0000","last_undegraded":"2024-03-23T09:43:47.020741+0000","last_fullsized":"2024-03-23T09:43:47.020741+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:21.051130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.038950+0000","last_change":"2024-03-23T09:43:47.037068+0000","last_active":"2024-03-23T09:43:47.038950+0000","last_peered":"2024-03-23T09:43:47.038950+0000","last_clean":"2024-03-23T09:43:47.038950+0000","last_became_active":"2024-03-23T09:43:47.036452+0000","last_became_peered":"2024-03-23T09:43:47.036452+0000","last_unstale":"2024-03-23T09:43:47.038950+0000","last_undegraded":"2024-03-23T09:43:47.038950+0000","last_fullsized":"2024-03-23T09:43:47.038950+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:26.308593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021823+0000","last_change":"2024-03-23T09:43:47.021823+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021823+0000","last_undegraded":"2024-03-23T09:43:47.021823+0000","last_fullsized":"2024-03-23T09:43:47.021823+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.033586+0000","last_change":"2024-03-23T09:43:47.029691+0000","last_active":"2024-03-23T09:43:47.033586+0000","last_peered":"2024-03-23T09:43:47.033586+0000","last_clean":"2024-03-23T09:43:47.033586+0000","last_became_active":"2024-03-23T09:43:47.029427+0000","last_became_peered":"2024-03-23T09:43:47.029427+0000","last_unstale":"2024-03-23T09:43:47.033586+0000","last_undegraded":"2024-03-23T09:43:47.033586+0000","last_fullsized":"2024-03-23T09:43:47.033586+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023508+0000","last_change":"2024-03-23T09:43:47.023508+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023508+0000","last_undegraded":"2024-03-23T09:43:47.023508+0000","last_fullsized":"2024-03-23T09:43:47.023508+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:45.147489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040183+0000","last_change":"2024-03-23T09:43:47.039140+0000","last_active":"2024-03-23T09:43:47.040183+0000","last_peered":"2024-03-23T09:43:47.040183+0000","last_clean":"2024-03-23T09:43:47.040183+0000","last_became_active":"2024-03-23T09:43:47.038851+0000","last_became_peered":"2024-03-23T09:43:47.038851+0000","last_unstale":"2024-03-23T09:43:47.040183+0000","last_undegraded":"2024-03-23T09:43:47.040183+0000","last_fullsized":"2024-03-23T09:43:47.040183+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:18.815610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024459+0000","last_change":"2024-03-23T09:43:47.024459+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024459+0000","last_undegraded":"2024-03-23T09:43:47.024459+0000","last_fullsized":"2024-03-23T09:43:47.024459+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:28:32.181328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037202+0000","last_change":"2024-03-23T09:43:47.031421+0000","last_active":"2024-03-23T09:43:47.037202+0000","last_peered":"2024-03-23T09:43:47.037202+0000","last_clean":"2024-03-23T09:43:47.037202+0000","last_became_active":"2024-03-23T09:43:47.031157+0000","last_became_peered":"2024-03-23T09:43:47.031157+0000","last_unstale":"2024-03-23T09:43:47.037202+0000","last_undegraded":"2024-03-23T09:43:47.037202+0000","last_fullsized":"2024-03-23T09:43:47.037202+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:49.393118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025198+0000","last_change":"2024-03-23T09:43:47.025198+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025198+0000","last_undegraded":"2024-03-23T09:43:47.025198+0000","last_fullsized":"2024-03-23T09:43:47.025198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:42:29.904309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.035316+0000","last_change":"2024-03-23T09:43:47.034482+0000","last_active":"2024-03-23T09:43:47.035316+0000","last_peered":"2024-03-23T09:43:47.035316+0000","last_clean":"2024-03-23T09:43:47.035316+0000","last_became_active":"2024-03-23T09:43:47.034190+0000","last_became_peered":"2024-03-23T09:43:47.034190+0000","last_unstale":"2024-03-23T09:43:47.035316+0000","last_undegraded":"2024-03-23T09:43:47.035316+0000","last_fullsized":"2024-03-23T09:43:47.035316+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:27:00.248727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023485+0000","last_change":"2024-03-23T09:43:47.023485+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023485+0000","last_undegraded":"2024-03-23T09:43:47.023485+0000","last_fullsized":"2024-03-23T09:43:47.023485+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:59.729962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036543+0000","last_change":"2024-03-23T09:43:47.034996+0000","last_active":"2024-03-23T09:43:47.036543+0000","last_peered":"2024-03-23T09:43:47.036543+0000","last_clean":"2024-03-23T09:43:47.036543+0000","last_became_active":"2024-03-23T09:43:47.034715+0000","last_became_peered":"2024-03-23T09:43:47.034715+0000","last_unstale":"2024-03-23T09:43:47.036543+0000","last_undegraded":"2024-03-23T09:43:47.036543+0000","last_fullsized":"2024-03-23T09:43:47.036543+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022665+0000","last_change":"2024-03-23T09:43:47.022665+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022665+0000","last_undegraded":"2024-03-23T09:43:47.022665+0000","last_fullsized":"2024-03-23T09:43:47.022665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:25:06.163966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039588+0000","last_change":"2024-03-23T09:43:47.036762+0000","last_active":"2024-03-23T09:43:47.039588+0000","last_peered":"2024-03-23T09:43:47.039588+0000","last_clean":"2024-03-23T09:43:47.039588+0000","last_became_active":"2024-03-23T09:43:47.036510+0000","last_became_peered":"2024-03-23T09:43:47.036510+0000","last_unstale":"2024-03-23T09:43:47.039588+0000","last_undegraded":"2024-03-23T09:43:47.039588+0000","last_fullsized":"2024-03-23T09:43:47.039588+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:58.725092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023652+0000","last_change":"2024-03-23T09:43:47.023652+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023652+0000","last_undegraded":"2024-03-23T09:43:47.023652+0000","last_fullsized":"2024-03-23T09:43:47.023652+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:36:27.003657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037419+0000","last_change":"2024-03-23T09:43:47.034507+0000","last_active":"2024-03-23T09:43:47.037419+0000","last_peered":"2024-03-23T09:43:47.037419+0000","last_clean":"2024-03-23T09:43:47.037419+0000","last_became_active":"2024-03-23T09:43:47.034260+0000","last_became_peered":"2024-03-23T09:43:47.034260+0000","last_unstale":"2024-03-23T09:43:47.037419+0000","last_undegraded":"2024-03-23T09:43:47.037419+0000","last_fullsized":"2024-03-23T09:43:47.037419+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025795+0000","last_change":"2024-03-23T09:43:47.025795+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025795+0000","last_undegraded":"2024-03-23T09:43:47.025795+0000","last_fullsized":"2024-03-23T09:43:47.025795+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037161+0000","last_change":"2024-03-23T09:43:47.036141+0000","last_active":"2024-03-23T09:43:47.037161+0000","last_peered":"2024-03-23T09:43:47.037161+0000","last_clean":"2024-03-23T09:43:47.037161+0000","last_became_active":"2024-03-23T09:43:47.035805+0000","last_became_peered":"2024-03-23T09:43:47.035805+0000","last_unstale":"2024-03-23T09:43:47.037161+0000","last_undegraded":"2024-03-23T09:43:47.037161+0000","last_fullsized":"2024-03-23T09:43:47.037161+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:22.465435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023205+0000","last_change":"2024-03-23T09:43:47.023205+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023205+0000","last_undegraded":"2024-03-23T09:43:47.023205+0000","last_fullsized":"2024-03-23T09:43:47.023205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:28:37.758076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037387+0000","last_change":"2024-03-23T09:43:47.032774+0000","last_active":"2024-03-23T09:43:47.037387+0000","last_peered":"2024-03-23T09:43:47.037387+0000","last_clean":"2024-03-23T09:43:47.037387+0000","last_became_active":"2024-03-23T09:43:47.032528+0000","last_became_peered":"2024-03-23T09:43:47.032528+0000","last_unstale":"2024-03-23T09:43:47.037387+0000","last_undegraded":"2024-03-23T09:43:47.037387+0000","last_fullsized":"2024-03-23T09:43:47.037387+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026681+0000","last_change":"2024-03-23T09:43:47.026681+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026681+0000","last_undegraded":"2024-03-23T09:43:47.026681+0000","last_fullsized":"2024-03-23T09:43:47.026681+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:11.106544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037521+0000","last_change":"2024-03-23T09:43:47.033945+0000","last_active":"2024-03-23T09:43:47.037521+0000","last_peered":"2024-03-23T09:43:47.037521+0000","last_clean":"2024-03-23T09:43:47.037521+0000","last_became_active":"2024-03-23T09:43:47.033037+0000","last_became_peered":"2024-03-23T09:43:47.033037+0000","last_unstale":"2024-03-23T09:43:47.037521+0000","last_undegraded":"2024-03-23T09:43:47.037521+0000","last_fullsized":"2024-03-23T09:43:47.037521+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:28:36.753206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.029812+0000","last_change":"2024-03-23T09:43:47.029812+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.029812+0000","last_undegraded":"2024-03-23T09:43:47.029812+0000","last_fullsized":"2024-03-23T09:43:47.029812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:51:10.516410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036612+0000","last_change":"2024-03-23T09:43:47.034748+0000","last_active":"2024-03-23T09:43:47.036612+0000","last_peered":"2024-03-23T09:43:47.036612+0000","last_clean":"2024-03-23T09:43:47.036612+0000","last_became_active":"2024-03-23T09:43:47.034340+0000","last_became_peered":"2024-03-23T09:43:47.034340+0000","last_unstale":"2024-03-23T09:43:47.036612+0000","last_undegraded":"2024-03-23T09:43:47.036612+0000","last_fullsized":"2024-03-23T09:43:47.036612+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:06.713892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024286+0000","last_change":"2024-03-23T09:43:47.024286+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024286+0000","last_undegraded":"2024-03-23T09:43:47.024286+0000","last_fullsized":"2024-03-23T09:43:47.024286+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039159+0000","last_change":"2024-03-23T09:43:47.034963+0000","last_active":"2024-03-23T09:43:47.039159+0000","last_peered":"2024-03-23T09:43:47.039159+0000","last_clean":"2024-03-23T09:43:47.039159+0000","last_became_active":"2024-03-23T09:43:47.034680+0000","last_became_peered":"2024-03-23T09:43:47.034680+0000","last_unstale":"2024-03-23T09:43:47.039159+0000","last_undegraded":"2024-03-23T09:43:47.039159+0000","last_fullsized":"2024-03-23T09:43:47.039159+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:15:54.128943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023273+0000","last_change":"2024-03-23T09:43:47.023273+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023273+0000","last_undegraded":"2024-03-23T09:43:47.023273+0000","last_fullsized":"2024-03-23T09:43:47.023273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:27:01.253597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040165+0000","last_change":"2024-03-23T09:43:47.038681+0000","last_active":"2024-03-23T09:43:47.040165+0000","last_peered":"2024-03-23T09:43:47.040165+0000","last_clean":"2024-03-23T09:43:47.040165+0000","last_became_active":"2024-03-23T09:43:47.038410+0000","last_became_peered":"2024-03-23T09:43:47.038410+0000","last_unstale":"2024-03-23T09:43:47.040165+0000","last_undegraded":"2024-03-23T09:43:47.040165+0000","last_fullsized":"2024-03-23T09:43:47.040165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:19.400345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024358+0000","last_change":"2024-03-23T09:43:47.024358+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024358+0000","last_undegraded":"2024-03-23T09:43:47.024358+0000","last_fullsized":"2024-03-23T09:43:47.024358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:02:58.178256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.042320+0000","last_change":"2024-03-23T09:43:47.036966+0000","last_active":"2024-03-23T09:43:47.042320+0000","last_peered":"2024-03-23T09:43:47.042320+0000","last_clean":"2024-03-23T09:43:47.042320+0000","last_became_active":"2024-03-23T09:43:47.036691+0000","last_became_peered":"2024-03-23T09:43:47.036691+0000","last_unstale":"2024-03-23T09:43:47.042320+0000","last_undegraded":"2024-03-23T09:43:47.042320+0000","last_fullsized":"2024-03-23T09:43:47.042320+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024769+0000","last_change":"2024-03-23T09:43:47.024769+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024769+0000","last_undegraded":"2024-03-23T09:43:47.024769+0000","last_fullsized":"2024-03-23T09:43:47.024769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037227+0000","last_change":"2024-03-23T09:43:47.036026+0000","last_active":"2024-03-23T09:43:47.037227+0000","last_peered":"2024-03-23T09:43:47.037227+0000","last_clean":"2024-03-23T09:43:47.037227+0000","last_became_active":"2024-03-23T09:43:47.035690+0000","last_became_peered":"2024-03-23T09:43:47.035690+0000","last_unstale":"2024-03-23T09:43:47.037227+0000","last_undegraded":"2024-03-23T09:43:47.037227+0000","last_fullsized":"2024-03-23T09:43:47.037227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:20.475983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.033106+0000","last_change":"2024-03-23T09:43:47.033106+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.033106+0000","last_undegraded":"2024-03-23T09:43:47.033106+0000","last_fullsized":"2024-03-23T09:43:47.033106+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034465+0000","last_change":"2024-03-23T09:43:47.032740+0000","last_active":"2024-03-23T09:43:47.034465+0000","last_peered":"2024-03-23T09:43:47.034465+0000","last_clean":"2024-03-23T09:43:47.034465+0000","last_became_active":"2024-03-23T09:43:47.032456+0000","last_became_peered":"2024-03-23T09:43:47.032456+0000","last_unstale":"2024-03-23T09:43:47.034465+0000","last_undegraded":"2024-03-23T09:43:47.034465+0000","last_fullsized":"2024-03-23T09:43:47.034465+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021027+0000","last_change":"2024-03-23T09:43:47.021027+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021027+0000","last_undegraded":"2024-03-23T09:43:47.021027+0000","last_fullsized":"2024-03-23T09:43:47.021027+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:07:10.261476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040144+0000","last_change":"2024-03-23T09:43:47.038642+0000","last_active":"2024-03-23T09:43:47.040144+0000","last_peered":"2024-03-23T09:43:47.040144+0000","last_clean":"2024-03-23T09:43:47.040144+0000","last_became_active":"2024-03-23T09:43:47.038385+0000","last_became_peered":"2024-03-23T09:43:47.038385+0000","last_unstale":"2024-03-23T09:43:47.040144+0000","last_undegraded":"2024-03-23T09:43:47.040144+0000","last_fullsized":"2024-03-23T09:43:47.040144+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:33.981220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026250+0000","last_change":"2024-03-23T09:43:47.026250+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026250+0000","last_undegraded":"2024-03-23T09:43:47.026250+0000","last_fullsized":"2024-03-23T09:43:47.026250+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:21.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034573+0000","last_change":"2024-03-23T09:43:47.032632+0000","last_active":"2024-03-23T09:43:47.034573+0000","last_peered":"2024-03-23T09:43:47.034573+0000","last_clean":"2024-03-23T09:43:47.034573+0000","last_became_active":"2024-03-23T09:43:47.032375+0000","last_became_peered":"2024-03-23T09:43:47.032375+0000","last_unstale":"2024-03-23T09:43:47.034573+0000","last_undegraded":"2024-03-23T09:43:47.034573+0000","last_fullsized":"2024-03-23T09:43:47.034573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.029300+0000","last_change":"2024-03-23T09:43:47.029300+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.029300+0000","last_undegraded":"2024-03-23T09:43:47.029300+0000","last_fullsized":"2024-03-23T09:43:47.029300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:14:35.775608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036589+0000","last_change":"2024-03-23T09:43:47.030434+0000","last_active":"2024-03-23T09:43:47.036589+0000","last_peered":"2024-03-23T09:43:47.036589+0000","last_clean":"2024-03-23T09:43:47.036589+0000","last_became_active":"2024-03-23T09:43:47.030164+0000","last_became_peered":"2024-03-23T09:43:47.030164+0000","last_unstale":"2024-03-23T09:43:47.036589+0000","last_undegraded":"2024-03-23T09:43:47.036589+0000","last_fullsized":"2024-03-23T09:43:47.036589+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:09.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022414+0000","last_change":"2024-03-23T09:43:47.022414+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022414+0000","last_undegraded":"2024-03-23T09:43:47.022414+0000","last_fullsized":"2024-03-23T09:43:47.022414+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:22:26.903375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034938+0000","last_change":"2024-03-23T09:43:47.031235+0000","last_active":"2024-03-23T09:43:47.034938+0000","last_peered":"2024-03-23T09:43:47.034938+0000","last_clean":"2024-03-23T09:43:47.034938+0000","last_became_active":"2024-03-23T09:43:47.030949+0000","last_became_peered":"2024-03-23T09:43:47.030949+0000","last_unstale":"2024-03-23T09:43:47.034938+0000","last_undegraded":"2024-03-23T09:43:47.034938+0000","last_fullsized":"2024-03-23T09:43:47.034938+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:58:56.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022818+0000","last_change":"2024-03-23T09:43:47.022818+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022818+0000","last_undegraded":"2024-03-23T09:43:47.022818+0000","last_fullsized":"2024-03-23T09:43:47.022818+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040173+0000","last_change":"2024-03-23T09:43:47.038838+0000","last_active":"2024-03-23T09:43:47.040173+0000","last_peered":"2024-03-23T09:43:47.040173+0000","last_clean":"2024-03-23T09:43:47.040173+0000","last_became_active":"2024-03-23T09:43:47.038574+0000","last_became_peered":"2024-03-23T09:43:47.038574+0000","last_unstale":"2024-03-23T09:43:47.040173+0000","last_undegraded":"2024-03-23T09:43:47.040173+0000","last_fullsized":"2024-03-23T09:43:47.040173+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.030328+0000","last_change":"2024-03-23T09:43:47.030328+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.030328+0000","last_undegraded":"2024-03-23T09:43:47.030328+0000","last_fullsized":"2024-03-23T09:43:47.030328+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:49.207297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.031785+0000","last_change":"2024-03-23T09:43:47.029629+0000","last_active":"2024-03-23T09:43:47.031785+0000","last_peered":"2024-03-23T09:43:47.031785+0000","last_clean":"2024-03-23T09:43:47.031785+0000","last_became_active":"2024-03-23T09:43:47.029338+0000","last_became_peered":"2024-03-23T09:43:47.029338+0000","last_unstale":"2024-03-23T09:43:47.031785+0000","last_undegraded":"2024-03-23T09:43:47.031785+0000","last_fullsized":"2024-03-23T09:43:47.031785+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:15:54.128943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032692+0000","last_change":"2024-03-23T09:43:47.032692+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032692+0000","last_undegraded":"2024-03-23T09:43:47.032692+0000","last_fullsized":"2024-03-23T09:43:47.032692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036684+0000","last_change":"2024-03-23T09:43:47.035203+0000","last_active":"2024-03-23T09:43:47.036684+0000","last_peered":"2024-03-23T09:43:47.036684+0000","last_clean":"2024-03-23T09:43:47.036684+0000","last_became_active":"2024-03-23T09:43:47.034912+0000","last_became_peered":"2024-03-23T09:43:47.034912+0000","last_unstale":"2024-03-23T09:43:47.036684+0000","last_undegraded":"2024-03-23T09:43:47.036684+0000","last_fullsized":"2024-03-23T09:43:47.036684+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:36:25.998787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.019914+0000","last_change":"2024-03-23T09:43:47.019914+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.019914+0000","last_undegraded":"2024-03-23T09:43:47.019914+0000","last_fullsized":"2024-03-23T09:43:47.019914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034026+0000","last_change":"2024-03-23T09:43:47.029065+0000","last_active":"2024-03-23T09:43:47.034026+0000","last_peered":"2024-03-23T09:43:47.034026+0000","last_clean":"2024-03-23T09:43:47.034026+0000","last_became_active":"2024-03-23T09:43:47.028797+0000","last_became_peered":"2024-03-23T09:43:47.028797+0000","last_unstale":"2024-03-23T09:43:47.034026+0000","last_undegraded":"2024-03-23T09:43:47.034026+0000","last_fullsized":"2024-03-23T09:43:47.034026+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:32:33.947908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022836+0000","last_change":"2024-03-23T09:43:47.022836+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022836+0000","last_undegraded":"2024-03-23T09:43:47.022836+0000","last_fullsized":"2024-03-23T09:43:47.022836+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:19:12.228744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040108+0000","last_change":"2024-03-23T09:43:47.038126+0000","last_active":"2024-03-23T09:43:47.040108+0000","last_peered":"2024-03-23T09:43:47.040108+0000","last_clean":"2024-03-23T09:43:47.040108+0000","last_became_active":"2024-03-23T09:43:47.037850+0000","last_became_peered":"2024-03-23T09:43:47.037850+0000","last_unstale":"2024-03-23T09:43:47.040108+0000","last_undegraded":"2024-03-23T09:43:47.040108+0000","last_fullsized":"2024-03-23T09:43:47.040108+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023991+0000","last_change":"2024-03-23T09:43:47.023991+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023991+0000","last_undegraded":"2024-03-23T09:43:47.023991+0000","last_fullsized":"2024-03-23T09:43:47.023991+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:59.768596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037679+0000","last_change":"2024-03-23T09:43:47.036954+0000","last_active":"2024-03-23T09:43:47.037679+0000","last_peered":"2024-03-23T09:43:47.037679+0000","last_clean":"2024-03-23T09:43:47.037679+0000","last_became_active":"2024-03-23T09:43:47.036655+0000","last_became_peered":"2024-03-23T09:43:47.036655+0000","last_unstale":"2024-03-23T09:43:47.037679+0000","last_undegraded":"2024-03-23T09:43:47.037679+0000","last_fullsized":"2024-03-23T09:43:47.037679+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:05:51.129365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023834+0000","last_change":"2024-03-23T09:43:47.023834+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023834+0000","last_undegraded":"2024-03-23T09:43:47.023834+0000","last_fullsized":"2024-03-23T09:43:47.023834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:45.147489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037249+0000","last_change":"2024-03-23T09:43:47.032771+0000","last_active":"2024-03-23T09:43:47.037249+0000","last_peered":"2024-03-23T09:43:47.037249+0000","last_clean":"2024-03-23T09:43:47.037249+0000","last_became_active":"2024-03-23T09:43:47.032489+0000","last_became_peered":"2024-03-23T09:43:47.032489+0000","last_unstale":"2024-03-23T09:43:47.037249+0000","last_undegraded":"2024-03-23T09:43:47.037249+0000","last_fullsized":"2024-03-23T09:43:47.037249+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:48:49.438386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.019914+0000","last_change":"2024-03-23T09:43:47.019914+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.019914+0000","last_undegraded":"2024-03-23T09:43:47.019914+0000","last_fullsized":"2024-03-23T09:43:47.019914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:24:18.285067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.033031+0000","last_change":"2024-03-23T09:43:47.030522+0000","last_active":"2024-03-23T09:43:47.033031+0000","last_peered":"2024-03-23T09:43:47.033031+0000","last_clean":"2024-03-23T09:43:47.033031+0000","last_became_active":"2024-03-23T09:43:47.030216+0000","last_became_peered":"2024-03-23T09:43:47.030216+0000","last_unstale":"2024-03-23T09:43:47.033031+0000","last_undegraded":"2024-03-23T09:43:47.033031+0000","last_fullsized":"2024-03-23T09:43:47.033031+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:32:33.947908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022218+0000","last_change":"2024-03-23T09:43:47.022218+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022218+0000","last_undegraded":"2024-03-23T09:43:47.022218+0000","last_fullsized":"2024-03-23T09:43:47.022218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:30:34.056749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039369+0000","last_change":"2024-03-23T09:43:47.038282+0000","last_active":"2024-03-23T09:43:47.039369+0000","last_peered":"2024-03-23T09:43:47.039369+0000","last_clean":"2024-03-23T09:43:47.039369+0000","last_became_active":"2024-03-23T09:43:47.037787+0000","last_became_peered":"2024-03-23T09:43:47.037787+0000","last_unstale":"2024-03-23T09:43:47.039369+0000","last_undegraded":"2024-03-23T09:43:47.039369+0000","last_fullsized":"2024-03-23T09:43:47.039369+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:10.101674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021303+0000","last_change":"2024-03-23T09:43:47.021303+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021303+0000","last_undegraded":"2024-03-23T09:43:47.021303+0000","last_fullsized":"2024-03-23T09:43:47.021303+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:52:24.758413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039546+0000","last_change":"2024-03-23T09:43:47.037606+0000","last_active":"2024-03-23T09:43:47.039546+0000","last_peered":"2024-03-23T09:43:47.039546+0000","last_clean":"2024-03-23T09:43:47.039546+0000","last_became_active":"2024-03-23T09:43:47.037315+0000","last_became_peered":"2024-03-23T09:43:47.037315+0000","last_unstale":"2024-03-23T09:43:47.039546+0000","last_undegraded":"2024-03-23T09:43:47.039546+0000","last_fullsized":"2024-03-23T09:43:47.039546+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:22.192970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":200,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-23T09:43:46.013864+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:43:46.013864+0000","last_peered":"2024-03-23T09:43:46.013864+0000","last_clean":"2024-03-23T09:43:46.013864+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:43:46.013864+0000","last_undegraded":"2024-03-23T09:43:46.013864+0000","last_fullsized":"2024-03-23T09:43:46.013864+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1265664,"data_stored":864722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69299999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.62}]},{"osd":2,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.65900000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72699999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56999999999999995}]},{"osd":3,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.433,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58399999999999996}]},{"osd":4,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.377,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72099999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45300000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66500000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71499999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73799999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53700000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":267468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.499,"15min":0.498},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69299999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.54100000000000004}]},{"osd":1,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.40200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.42699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.627}]},{"osd":2,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.41299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.46100000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.496}]},{"osd":3,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52900000000000003,"15min":0.52300000000000002},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.73199999999999998,"5min":0.752,"15min":0.752},"last":0.60999999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.38700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.67500000000000004}]},{"osd":5,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.72399999999999998,"5min":0.747,"15min":0.747},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":268087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48099999999999998,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.42199999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52100000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52900000000000003,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.434,"5min":0.376,"15min":0.376},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.50700000000000001,"15min":0.49299999999999999},"min":{"1min":0.46700000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.46800000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.54100000000000004}]},{"osd":4,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47999999999999998,"15min":0.47699999999999998},"min":{"1min":0.33300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.439},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.374,"15min":0.374},"max":{"1min":0.69999999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63400000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.34799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.749,"15min":0.749},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.51800000000000002},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70099999999999996,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.60899999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.44600000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71499999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.621}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":268087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.69899999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.49199999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.46400000000000002},"min":{"1min":0.35699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.498,"15min":0.498},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.40600000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.42999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.752,"5min":0.755,"15min":0.755},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.747,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.50800000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70399999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.30199999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68799999999999994,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.308,"5min":0.253,"15min":0.253},"max":{"1min":0.68400000000000005,"5min":1.008,"15min":1.008},"last":0.36499999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.34000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70099999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.32400000000000001}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65800000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56200000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":671744,"data_stored":274354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.307,"15min":0.30499999999999999},"min":{"1min":0.25900000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30399999999999999,"15min":0.30199999999999999},"min":{"1min":0.25600000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.42899999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.437},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.55100000000000005}]},{"osd":2,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.313,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76500000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53100000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.73499999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.47399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81499999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.42899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.70999999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.81000000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.41799999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71499999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.67800000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.626},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.59999999999999998},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.71599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":671744,"data_stored":274354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.34000000000000002,"15min":0.34399999999999997},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.48799999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.25,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48799999999999999,"5min":0.872,"15min":0.872},"last":0.23000000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.629,"15min":0.629},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58599999999999997,"5min":0.746,"15min":0.746},"last":0.34100000000000003}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.47399999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.40200000000000002,"15min":0.40699999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55800000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.32300000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72499999999999998,"5min":1.216,"15min":1.216},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.753,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.624,"15min":0.63200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.71599999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75800000000000001,"5min":1.101,"15min":1.101},"last":0.66800000000000004}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.70699999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.63800000000000001,"15min":0.64800000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.72699999999999998,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.76500000000000001,"5min":1.155,"15min":1.155},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72099999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":659456,"data_stored":268087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33600000000000002,"15min":0.33300000000000002},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.53300000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61899999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.30299999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48399999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.374,"15min":0.37},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.55200000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41099999999999998,"15min":0.40699999999999997},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70499999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60899999999999999,"5min":2.75,"15min":2.75},"last":0.41499999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.45500000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91100000000000003,"5min":2.778,"15min":2.778},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68999999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.46300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88100000000000001,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.441},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.628}]},{"osd":7,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65400000000000003,"15min":0.64700000000000002},"min":{"1min":0.51800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":0.92400000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.76000000000000001,"15min":0.77800000000000002},"min":{"1min":0.44600000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.85099999999999998,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.52800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1261568,"data_stored":864103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32200000000000001,"15min":0.31900000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.499,"5min":0.504,"15min":0.504},"last":0.316},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.34499999999999997},"min":{"1min":0.25900000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.51300000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.30099999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36099999999999999,"15min":0.35899999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36099999999999999},"min":{"1min":0.27000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.33700000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.379,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.42299999999999999,"15min":0.41999999999999998},"min":{"1min":0.32100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.40999999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53600000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.81499999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54600000000000004}]},{"osd":6,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83299999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.88400000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55500000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:43:50.736 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:43:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.873+0000 7f8990d77640 1 Processor -- start 2024-03-23T09:43:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.873+0000 7f8990d77640 1 -- start start 2024-03-23T09:43:50.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.874+0000 7f8990d77640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898c15cd00 0x7f898c0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.874+0000 7f8990d77640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f898c0cf920 0x7f898c0b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.874+0000 7f8990d77640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898c15d0e0 con 0x7f898c0d1020 2024-03-23T09:43:50.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.874+0000 7f8990d77640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898c0d1540 con 0x7f898c0cf920 2024-03-23T09:43:50.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.874+0000 7f8990d77640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c0d1820 con 0x7f898c15cd00 2024-03-23T09:43:50.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.874+0000 7f898ad9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f898c0cf920 0x7f898c0b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.874+0000 7f898ad9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f898c0cf920 0x7f898c0b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39126/0 (socket says 172.21.15.47:39126) 2024-03-23T09:43:50.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.874+0000 7f8989d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f898c0d1020 0x7f898c0cf210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36052/0 (socket says 172.21.15.47:36052) 2024-03-23T09:43:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.874+0000 7f898ad9d640 1 -- 172.21.15.47:0/2916721026 learned_addr learned my addr 172.21.15.47:0/2916721026 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.874+0000 7f898a59c640 1 --2- 172.21.15.47:0/2916721026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898c15cd00 0x7f898c0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.875+0000 7f898ad9d640 1 -- 172.21.15.47:0/2916721026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898c15cd00 msgr2=0x7f898c0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.875+0000 7f898ad9d640 1 --2- 172.21.15.47:0/2916721026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898c15cd00 0x7f898c0d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.875+0000 7f898ad9d640 1 -- 172.21.15.47:0/2916721026 >> v1:172.21.15.47:6789/0 conn(0x7f898c0d1020 legacy=0x7f898c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.875+0000 7f898ad9d640 1 -- 172.21.15.47:0/2916721026 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c0d1b00 con 0x7f898c0cf920 2024-03-23T09:43:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.875+0000 7f898959a640 1 -- 172.21.15.47:0/2916721026 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001799628 0 0) 0x7f898c15d0e0 con 0x7f898c0d1020 2024-03-23T09:43:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.875+0000 7f898a59c640 1 --2- 172.21.15.47:0/2916721026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898c15cd00 0x7f898c0d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.875+0000 7f898ad9d640 1 --2- 172.21.15.47:0/2916721026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c0cf920 0x7f898c0b5a90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f897c009e50 tx=0x7f897c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81eafd0ecd98f77 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.875+0000 7f898959a640 1 -- 172.21.15.47:0/2916721026 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c02ca70 con 0x7f898c0cf920 2024-03-23T09:43:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.875+0000 7f898959a640 1 -- 172.21.15.47:0/2916721026 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c02cbd0 con 0x7f898c0cf920 2024-03-23T09:43:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.875+0000 7f898959a640 1 -- 172.21.15.47:0/2916721026 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c0343f0 con 0x7f898c0cf920 2024-03-23T09:43:50.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.876+0000 7f8990d77640 1 -- 172.21.15.47:0/2916721026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c0cf920 msgr2=0x7f898c0b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.876+0000 7f8990d77640 1 --2- 172.21.15.47:0/2916721026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c0cf920 0x7f898c0b5a90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f897c009e50 tx=0x7f897c02bed0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.876+0000 7f8990d77640 1 -- 172.21.15.47:0/2916721026 shutdown_connections 2024-03-23T09:43:50.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.876+0000 7f8990d77640 1 --2- 172.21.15.47:0/2916721026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c0cf920 0x7f898c0b5a90 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.876+0000 7f8990d77640 1 --2- 172.21.15.47:0/2916721026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898c15cd00 0x7f898c0d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.876+0000 7f8990d77640 1 -- 172.21.15.47:0/2916721026 >> 172.21.15.47:0/2916721026 conn(0x7f898c06b160 msgr2=0x7f898c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:50.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.876+0000 7f8990d77640 1 -- 172.21.15.47:0/2916721026 shutdown_connections 2024-03-23T09:43:50.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.876+0000 7f8990d77640 1 -- 172.21.15.47:0/2916721026 wait complete. 2024-03-23T09:43:50.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.877+0000 7f8990d77640 1 Processor -- start 2024-03-23T09:43:50.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.877+0000 7f8990d77640 1 -- start start 2024-03-23T09:43:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.878+0000 7f8990d77640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f898c0cf920 0x7f898c0c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.878+0000 7f8990d77640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898c0d1020 0x7f898c159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.878+0000 7f8990d77640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c15cd00 0x7f898c0cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.878+0000 7f8990d77640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c0c7f70 con 0x7f898c15cd00 2024-03-23T09:43:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.878+0000 7f8990d77640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c0c7360 con 0x7f898c0cf920 2024-03-23T09:43:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.878+0000 7f8990d77640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c0c7640 con 0x7f898c0d1020 2024-03-23T09:43:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.878+0000 7f898ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c15cd00 0x7f898c0cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.878+0000 7f898ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c15cd00 0x7f898c0cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39132/0 (socket says 172.21.15.47:39132) 2024-03-23T09:43:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.878+0000 7f898ad9d640 1 -- 172.21.15.47:0/2726612860 learned_addr learned my addr 172.21.15.47:0/2726612860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.879+0000 7f8989d9b640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898c0d1020 0x7f898c159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.879+0000 7f898ad9d640 1 -- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898c0d1020 msgr2=0x7f898c159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.879+0000 7f898a59c640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f898c0cf920 0x7f898c0c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.879+0000 7f898ad9d640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898c0d1020 0x7f898c159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.879+0000 7f898ad9d640 1 -- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f898c0cf920 msgr2=0x7f898c0c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.879+0000 7f898ad9d640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f898c0cf920 0x7f898c0c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.879+0000 7f898ad9d640 1 -- 172.21.15.47:0/2726612860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c009b30 con 0x7f898c15cd00 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.879+0000 7f898ad9d640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c15cd00 0x7f898c0cbde0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f897c009e50 tx=0x7f897c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.879+0000 7f8982ffd640 1 -- 172.21.15.47:0/2726612860 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c009510 con 0x7f898c15cd00 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.879+0000 7f8982ffd640 1 -- 172.21.15.47:0/2726612860 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c03e450 con 0x7f898c15cd00 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.879+0000 7f8982ffd640 1 -- 172.21.15.47:0/2726612860 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c03e800 con 0x7f898c15cd00 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.879+0000 7f8990d77640 1 -- 172.21.15.47:0/2726612860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c0c7920 con 0x7f898c15cd00 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.879+0000 7f8990d77640 1 -- 172.21.15.47:0/2726612860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c15a4b0 con 0x7f898c15cd00 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.880+0000 7f8990d77640 1 -- 172.21.15.47:0/2726612860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c072360 con 0x7f898c15cd00 2024-03-23T09:43:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.880+0000 7f8982ffd640 1 -- 172.21.15.47:0/2726612860 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f897c03e960 con 0x7f898c15cd00 2024-03-23T09:43:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.881+0000 7f8982ffd640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f896c06ac00 0x7f896c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.881+0000 7f8982ffd640 1 -- 172.21.15.47:0/2726612860 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f897c0ac0e0 con 0x7f898c15cd00 2024-03-23T09:43:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.881+0000 7f898a59c640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f896c06ac00 0x7f896c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.882+0000 7f898a59c640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f896c06ac00 0x7f896c06d0c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f89840096d0 tx=0x7f8984008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:50.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:50.885+0000 7f8982ffd640 1 -- 172.21.15.47:0/2726612860 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f897c0772c0 con 0x7f898c15cd00 2024-03-23T09:43:51.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.064+0000 7f8990d77640 1 -- 172.21.15.47:0/2726612860 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f898c0d25e0 con 0x7f896c06ac00 2024-03-23T09:43:51.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.072+0000 7f8982ffd640 1 -- 172.21.15.47:0/2726612860 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286507 (secure 0 0 0) 0x7f898400e4c0 con 0x7f896c06ac00 2024-03-23T09:43:51.074 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:51.076 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:43:51.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.075+0000 7f8990d77640 1 -- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f896c06ac00 msgr2=0x7f896c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:51.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.075+0000 7f8990d77640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f896c06ac00 0x7f896c06d0c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f89840096d0 tx=0x7f8984008040 comp rx=0 tx=0).stop 2024-03-23T09:43:51.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.075+0000 7f8990d77640 1 -- 172.21.15.47:0/2726612860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c15cd00 msgr2=0x7f898c0cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:51.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.075+0000 7f8990d77640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c15cd00 0x7f898c0cbde0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f897c009e50 tx=0x7f897c004a40 comp rx=0 tx=0).stop 2024-03-23T09:43:51.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.075+0000 7f8990d77640 1 -- 172.21.15.47:0/2726612860 shutdown_connections 2024-03-23T09:43:51.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.075+0000 7f8990d77640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f896c06ac00 0x7f896c06d0c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.075+0000 7f8990d77640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c15cd00 0x7f898c0cbde0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.075+0000 7f8990d77640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898c0d1020 0x7f898c159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.075+0000 7f8990d77640 1 --2- 172.21.15.47:0/2726612860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f898c0cf920 0x7f898c0c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.075+0000 7f8990d77640 1 -- 172.21.15.47:0/2726612860 >> 172.21.15.47:0/2726612860 conn(0x7f898c06b160 msgr2=0x7f898c15ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:51.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.075+0000 7f8990d77640 1 -- 172.21.15.47:0/2726612860 shutdown_connections 2024-03-23T09:43:51.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.075+0000 7f8990d77640 1 -- 172.21.15.47:0/2726612860 wait complete. 2024-03-23T09:43:51.086 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-03-23T09:43:50.378328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238916,"kb_used_data":6352,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754735804,"statfs":{"total":773094113280,"available":772849463296,"internally_reserved":0,"allocated":6504448,"data_stored":3349262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.367743"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.028360+0000","last_change":"2024-03-23T09:43:47.028360+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.028360+0000","last_undegraded":"2024-03-23T09:43:47.028360+0000","last_fullsized":"2024-03-23T09:43:47.028360+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:54:41.177596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021545+0000","last_change":"2024-03-23T09:43:47.021545+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021545+0000","last_undegraded":"2024-03-23T09:43:47.021545+0000","last_fullsized":"2024-03-23T09:43:47.021545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:36.262351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.027083+0000","last_change":"2024-03-23T09:43:47.027083+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.027083+0000","last_undegraded":"2024-03-23T09:43:47.027083+0000","last_fullsized":"2024-03-23T09:43:47.027083+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:22:26.903375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032839+0000","last_change":"2024-03-23T09:43:47.032839+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032839+0000","last_undegraded":"2024-03-23T09:43:47.032839+0000","last_fullsized":"2024-03-23T09:43:47.032839+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026471+0000","last_change":"2024-03-23T09:43:47.026471+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026471+0000","last_undegraded":"2024-03-23T09:43:47.026471+0000","last_fullsized":"2024-03-23T09:43:47.026471+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:17.338788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022332+0000","last_change":"2024-03-23T09:43:47.022332+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022332+0000","last_undegraded":"2024-03-23T09:43:47.022332+0000","last_fullsized":"2024-03-23T09:43:47.022332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023985+0000","last_change":"2024-03-23T09:43:47.023985+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023985+0000","last_undegraded":"2024-03-23T09:43:47.023985+0000","last_fullsized":"2024-03-23T09:43:47.023985+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:44:18.230315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021778+0000","last_change":"2024-03-23T09:43:47.021778+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021778+0000","last_undegraded":"2024-03-23T09:43:47.021778+0000","last_fullsized":"2024-03-23T09:43:47.021778+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:55:17.926828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023723+0000","last_change":"2024-03-23T09:43:47.023723+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023723+0000","last_undegraded":"2024-03-23T09:43:47.023723+0000","last_fullsized":"2024-03-23T09:43:47.023723+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:19.820479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.020703+0000","last_change":"2024-03-23T09:43:47.020703+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.020703+0000","last_undegraded":"2024-03-23T09:43:47.020703+0000","last_fullsized":"2024-03-23T09:43:47.020703+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:35:30.364827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.020633+0000","last_change":"2024-03-23T09:43:47.020633+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.020633+0000","last_undegraded":"2024-03-23T09:43:47.020633+0000","last_fullsized":"2024-03-23T09:43:47.020633+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:48:50.443256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032222+0000","last_change":"2024-03-23T09:43:47.032222+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032222+0000","last_undegraded":"2024-03-23T09:43:47.032222+0000","last_fullsized":"2024-03-23T09:43:47.032222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026161+0000","last_change":"2024-03-23T09:43:47.026161+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026161+0000","last_undegraded":"2024-03-23T09:43:47.026161+0000","last_fullsized":"2024-03-23T09:43:47.026161+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023765+0000","last_change":"2024-03-23T09:43:47.023765+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023765+0000","last_undegraded":"2024-03-23T09:43:47.023765+0000","last_fullsized":"2024-03-23T09:43:47.023765+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:05:58.075182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023448+0000","last_change":"2024-03-23T09:43:47.023448+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023448+0000","last_undegraded":"2024-03-23T09:43:47.023448+0000","last_fullsized":"2024-03-23T09:43:47.023448+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:56.576713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021932+0000","last_change":"2024-03-23T09:43:47.021932+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021932+0000","last_undegraded":"2024-03-23T09:43:47.021932+0000","last_fullsized":"2024-03-23T09:43:47.021932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:21.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023522+0000","last_change":"2024-03-23T09:43:47.023522+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023522+0000","last_undegraded":"2024-03-23T09:43:47.023522+0000","last_fullsized":"2024-03-23T09:43:47.023522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:50:34.378460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025211+0000","last_change":"2024-03-23T09:43:47.025211+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025211+0000","last_undegraded":"2024-03-23T09:43:47.025211+0000","last_fullsized":"2024-03-23T09:43:47.025211+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:49.207297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026927+0000","last_change":"2024-03-23T09:43:47.026927+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026927+0000","last_undegraded":"2024-03-23T09:43:47.026927+0000","last_fullsized":"2024-03-23T09:43:47.026927+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:59.729962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024967+0000","last_change":"2024-03-23T09:43:47.024967+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024967+0000","last_undegraded":"2024-03-23T09:43:47.024967+0000","last_fullsized":"2024-03-23T09:43:47.024967+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:07:44.532537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025472+0000","last_change":"2024-03-23T09:43:47.025472+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025472+0000","last_undegraded":"2024-03-23T09:43:47.025472+0000","last_fullsized":"2024-03-23T09:43:47.025472+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:19:12.228744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026145+0000","last_change":"2024-03-23T09:43:47.026145+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026145+0000","last_undegraded":"2024-03-23T09:43:47.026145+0000","last_fullsized":"2024-03-23T09:43:47.026145+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:47:24.835835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024831+0000","last_change":"2024-03-23T09:43:47.024831+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024831+0000","last_undegraded":"2024-03-23T09:43:47.024831+0000","last_fullsized":"2024-03-23T09:43:47.024831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:10:55.583385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024086+0000","last_change":"2024-03-23T09:43:47.024086+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024086+0000","last_undegraded":"2024-03-23T09:43:47.024086+0000","last_fullsized":"2024-03-23T09:43:47.024086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:54:41.177596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.027386+0000","last_change":"2024-03-23T09:43:47.027386+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.027386+0000","last_undegraded":"2024-03-23T09:43:47.027386+0000","last_fullsized":"2024-03-23T09:43:47.027386+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:08.737490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032083+0000","last_change":"2024-03-23T09:43:47.032083+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032083+0000","last_undegraded":"2024-03-23T09:43:47.032083+0000","last_fullsized":"2024-03-23T09:43:47.032083+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:45:37.390159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024006+0000","last_change":"2024-03-23T09:43:47.024006+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024006+0000","last_undegraded":"2024-03-23T09:43:47.024006+0000","last_fullsized":"2024-03-23T09:43:47.024006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:11.106544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026420+0000","last_change":"2024-03-23T09:43:47.026420+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026420+0000","last_undegraded":"2024-03-23T09:43:47.026420+0000","last_fullsized":"2024-03-23T09:43:47.026420+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024600+0000","last_change":"2024-03-23T09:43:47.024600+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024600+0000","last_undegraded":"2024-03-23T09:43:47.024600+0000","last_fullsized":"2024-03-23T09:43:47.024600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:04.150928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023232+0000","last_change":"2024-03-23T09:43:47.023232+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023232+0000","last_undegraded":"2024-03-23T09:43:47.023232+0000","last_fullsized":"2024-03-23T09:43:47.023232+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026157+0000","last_change":"2024-03-23T09:43:47.026157+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026157+0000","last_undegraded":"2024-03-23T09:43:47.026157+0000","last_fullsized":"2024-03-23T09:43:47.026157+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:46:16.837675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021430+0000","last_change":"2024-03-23T09:43:47.021430+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021430+0000","last_undegraded":"2024-03-23T09:43:47.021430+0000","last_fullsized":"2024-03-23T09:43:47.021430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:02:58.178256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023483+0000","last_change":"2024-03-23T09:43:47.023483+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023483+0000","last_undegraded":"2024-03-23T09:43:47.023483+0000","last_fullsized":"2024-03-23T09:43:47.023483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.038474+0000","last_change":"2024-03-23T09:43:47.036626+0000","last_active":"2024-03-23T09:43:47.038474+0000","last_peered":"2024-03-23T09:43:47.038474+0000","last_clean":"2024-03-23T09:43:47.038474+0000","last_became_active":"2024-03-23T09:43:47.036383+0000","last_became_peered":"2024-03-23T09:43:47.036383+0000","last_unstale":"2024-03-23T09:43:47.038474+0000","last_undegraded":"2024-03-23T09:43:47.038474+0000","last_fullsized":"2024-03-23T09:43:47.038474+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:18.815610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.020741+0000","last_change":"2024-03-23T09:43:47.020741+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.020741+0000","last_undegraded":"2024-03-23T09:43:47.020741+0000","last_fullsized":"2024-03-23T09:43:47.020741+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:21.051130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.038950+0000","last_change":"2024-03-23T09:43:47.037068+0000","last_active":"2024-03-23T09:43:47.038950+0000","last_peered":"2024-03-23T09:43:47.038950+0000","last_clean":"2024-03-23T09:43:47.038950+0000","last_became_active":"2024-03-23T09:43:47.036452+0000","last_became_peered":"2024-03-23T09:43:47.036452+0000","last_unstale":"2024-03-23T09:43:47.038950+0000","last_undegraded":"2024-03-23T09:43:47.038950+0000","last_fullsized":"2024-03-23T09:43:47.038950+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:26.308593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021823+0000","last_change":"2024-03-23T09:43:47.021823+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021823+0000","last_undegraded":"2024-03-23T09:43:47.021823+0000","last_fullsized":"2024-03-23T09:43:47.021823+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.033586+0000","last_change":"2024-03-23T09:43:47.029691+0000","last_active":"2024-03-23T09:43:47.033586+0000","last_peered":"2024-03-23T09:43:47.033586+0000","last_clean":"2024-03-23T09:43:47.033586+0000","last_became_active":"2024-03-23T09:43:47.029427+0000","last_became_peered":"2024-03-23T09:43:47.029427+0000","last_unstale":"2024-03-23T09:43:47.033586+0000","last_undegraded":"2024-03-23T09:43:47.033586+0000","last_fullsized":"2024-03-23T09:43:47.033586+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023508+0000","last_change":"2024-03-23T09:43:47.023508+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023508+0000","last_undegraded":"2024-03-23T09:43:47.023508+0000","last_fullsized":"2024-03-23T09:43:47.023508+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:45.147489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040183+0000","last_change":"2024-03-23T09:43:47.039140+0000","last_active":"2024-03-23T09:43:47.040183+0000","last_peered":"2024-03-23T09:43:47.040183+0000","last_clean":"2024-03-23T09:43:47.040183+0000","last_became_active":"2024-03-23T09:43:47.038851+0000","last_became_peered":"2024-03-23T09:43:47.038851+0000","last_unstale":"2024-03-23T09:43:47.040183+0000","last_undegraded":"2024-03-23T09:43:47.040183+0000","last_fullsized":"2024-03-23T09:43:47.040183+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:18.815610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024459+0000","last_change":"2024-03-23T09:43:47.024459+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024459+0000","last_undegraded":"2024-03-23T09:43:47.024459+0000","last_fullsized":"2024-03-23T09:43:47.024459+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:28:32.181328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037202+0000","last_change":"2024-03-23T09:43:47.031421+0000","last_active":"2024-03-23T09:43:47.037202+0000","last_peered":"2024-03-23T09:43:47.037202+0000","last_clean":"2024-03-23T09:43:47.037202+0000","last_became_active":"2024-03-23T09:43:47.031157+0000","last_became_peered":"2024-03-23T09:43:47.031157+0000","last_unstale":"2024-03-23T09:43:47.037202+0000","last_undegraded":"2024-03-23T09:43:47.037202+0000","last_fullsized":"2024-03-23T09:43:47.037202+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:49.393118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025198+0000","last_change":"2024-03-23T09:43:47.025198+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025198+0000","last_undegraded":"2024-03-23T09:43:47.025198+0000","last_fullsized":"2024-03-23T09:43:47.025198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:42:29.904309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.035316+0000","last_change":"2024-03-23T09:43:47.034482+0000","last_active":"2024-03-23T09:43:47.035316+0000","last_peered":"2024-03-23T09:43:47.035316+0000","last_clean":"2024-03-23T09:43:47.035316+0000","last_became_active":"2024-03-23T09:43:47.034190+0000","last_became_peered":"2024-03-23T09:43:47.034190+0000","last_unstale":"2024-03-23T09:43:47.035316+0000","last_undegraded":"2024-03-23T09:43:47.035316+0000","last_fullsized":"2024-03-23T09:43:47.035316+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:27:00.248727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023485+0000","last_change":"2024-03-23T09:43:47.023485+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023485+0000","last_undegraded":"2024-03-23T09:43:47.023485+0000","last_fullsized":"2024-03-23T09:43:47.023485+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:59.729962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036543+0000","last_change":"2024-03-23T09:43:47.034996+0000","last_active":"2024-03-23T09:43:47.036543+0000","last_peered":"2024-03-23T09:43:47.036543+0000","last_clean":"2024-03-23T09:43:47.036543+0000","last_became_active":"2024-03-23T09:43:47.034715+0000","last_became_peered":"2024-03-23T09:43:47.034715+0000","last_unstale":"2024-03-23T09:43:47.036543+0000","last_undegraded":"2024-03-23T09:43:47.036543+0000","last_fullsized":"2024-03-23T09:43:47.036543+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022665+0000","last_change":"2024-03-23T09:43:47.022665+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022665+0000","last_undegraded":"2024-03-23T09:43:47.022665+0000","last_fullsized":"2024-03-23T09:43:47.022665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:25:06.163966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039588+0000","last_change":"2024-03-23T09:43:47.036762+0000","last_active":"2024-03-23T09:43:47.039588+0000","last_peered":"2024-03-23T09:43:47.039588+0000","last_clean":"2024-03-23T09:43:47.039588+0000","last_became_active":"2024-03-23T09:43:47.036510+0000","last_became_peered":"2024-03-23T09:43:47.036510+0000","last_unstale":"2024-03-23T09:43:47.039588+0000","last_undegraded":"2024-03-23T09:43:47.039588+0000","last_fullsized":"2024-03-23T09:43:47.039588+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:58.725092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023652+0000","last_change":"2024-03-23T09:43:47.023652+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023652+0000","last_undegraded":"2024-03-23T09:43:47.023652+0000","last_fullsized":"2024-03-23T09:43:47.023652+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:36:27.003657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037419+0000","last_change":"2024-03-23T09:43:47.034507+0000","last_active":"2024-03-23T09:43:47.037419+0000","last_peered":"2024-03-23T09:43:47.037419+0000","last_clean":"2024-03-23T09:43:47.037419+0000","last_became_active":"2024-03-23T09:43:47.034260+0000","last_became_peered":"2024-03-23T09:43:47.034260+0000","last_unstale":"2024-03-23T09:43:47.037419+0000","last_undegraded":"2024-03-23T09:43:47.037419+0000","last_fullsized":"2024-03-23T09:43:47.037419+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025795+0000","last_change":"2024-03-23T09:43:47.025795+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025795+0000","last_undegraded":"2024-03-23T09:43:47.025795+0000","last_fullsized":"2024-03-23T09:43:47.025795+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037161+0000","last_change":"2024-03-23T09:43:47.036141+0000","last_active":"2024-03-23T09:43:47.037161+0000","last_peered":"2024-03-23T09:43:47.037161+0000","last_clean":"2024-03-23T09:43:47.037161+0000","last_became_active":"2024-03-23T09:43:47.035805+0000","last_became_peered":"2024-03-23T09:43:47.035805+0000","last_unstale":"2024-03-23T09:43:47.037161+0000","last_undegraded":"2024-03-23T09:43:47.037161+0000","last_fullsized":"2024-03-23T09:43:47.037161+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:22.465435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023205+0000","last_change":"2024-03-23T09:43:47.023205+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023205+0000","last_undegraded":"2024-03-23T09:43:47.023205+0000","last_fullsized":"2024-03-23T09:43:47.023205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:28:37.758076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037387+0000","last_change":"2024-03-23T09:43:47.032774+0000","last_active":"2024-03-23T09:43:47.037387+0000","last_peered":"2024-03-23T09:43:47.037387+0000","last_clean":"2024-03-23T09:43:47.037387+0000","last_became_active":"2024-03-23T09:43:47.032528+0000","last_became_peered":"2024-03-23T09:43:47.032528+0000","last_unstale":"2024-03-23T09:43:47.037387+0000","last_undegraded":"2024-03-23T09:43:47.037387+0000","last_fullsized":"2024-03-23T09:43:47.037387+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026681+0000","last_change":"2024-03-23T09:43:47.026681+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026681+0000","last_undegraded":"2024-03-23T09:43:47.026681+0000","last_fullsized":"2024-03-23T09:43:47.026681+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:11.106544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037521+0000","last_change":"2024-03-23T09:43:47.033945+0000","last_active":"2024-03-23T09:43:47.037521+0000","last_peered":"2024-03-23T09:43:47.037521+0000","last_clean":"2024-03-23T09:43:47.037521+0000","last_became_active":"2024-03-23T09:43:47.033037+0000","last_became_peered":"2024-03-23T09:43:47.033037+0000","last_unstale":"2024-03-23T09:43:47.037521+0000","last_undegraded":"2024-03-23T09:43:47.037521+0000","last_fullsized":"2024-03-23T09:43:47.037521+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:28:36.753206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.029812+0000","last_change":"2024-03-23T09:43:47.029812+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.029812+0000","last_undegraded":"2024-03-23T09:43:47.029812+0000","last_fullsized":"2024-03-23T09:43:47.029812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:51:10.516410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036612+0000","last_change":"2024-03-23T09:43:47.034748+0000","last_active":"2024-03-23T09:43:47.036612+0000","last_peered":"2024-03-23T09:43:47.036612+0000","last_clean":"2024-03-23T09:43:47.036612+0000","last_became_active":"2024-03-23T09:43:47.034340+0000","last_became_peered":"2024-03-23T09:43:47.034340+0000","last_unstale":"2024-03-23T09:43:47.036612+0000","last_undegraded":"2024-03-23T09:43:47.036612+0000","last_fullsized":"2024-03-23T09:43:47.036612+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:06.713892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024286+0000","last_change":"2024-03-23T09:43:47.024286+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024286+0000","last_undegraded":"2024-03-23T09:43:47.024286+0000","last_fullsized":"2024-03-23T09:43:47.024286+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039159+0000","last_change":"2024-03-23T09:43:47.034963+0000","last_active":"2024-03-23T09:43:47.039159+0000","last_peered":"2024-03-23T09:43:47.039159+0000","last_clean":"2024-03-23T09:43:47.039159+0000","last_became_active":"2024-03-23T09:43:47.034680+0000","last_became_peered":"2024-03-23T09:43:47.034680+0000","last_unstale":"2024-03-23T09:43:47.039159+0000","last_undegraded":"2024-03-23T09:43:47.039159+0000","last_fullsized":"2024-03-23T09:43:47.039159+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:15:54.128943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023273+0000","last_change":"2024-03-23T09:43:47.023273+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023273+0000","last_undegraded":"2024-03-23T09:43:47.023273+0000","last_fullsized":"2024-03-23T09:43:47.023273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:27:01.253597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040165+0000","last_change":"2024-03-23T09:43:47.038681+0000","last_active":"2024-03-23T09:43:47.040165+0000","last_peered":"2024-03-23T09:43:47.040165+0000","last_clean":"2024-03-23T09:43:47.040165+0000","last_became_active":"2024-03-23T09:43:47.038410+0000","last_became_peered":"2024-03-23T09:43:47.038410+0000","last_unstale":"2024-03-23T09:43:47.040165+0000","last_undegraded":"2024-03-23T09:43:47.040165+0000","last_fullsized":"2024-03-23T09:43:47.040165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:19.400345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024358+0000","last_change":"2024-03-23T09:43:47.024358+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024358+0000","last_undegraded":"2024-03-23T09:43:47.024358+0000","last_fullsized":"2024-03-23T09:43:47.024358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:02:58.178256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.042320+0000","last_change":"2024-03-23T09:43:47.036966+0000","last_active":"2024-03-23T09:43:47.042320+0000","last_peered":"2024-03-23T09:43:47.042320+0000","last_clean":"2024-03-23T09:43:47.042320+0000","last_became_active":"2024-03-23T09:43:47.036691+0000","last_became_peered":"2024-03-23T09:43:47.036691+0000","last_unstale":"2024-03-23T09:43:47.042320+0000","last_undegraded":"2024-03-23T09:43:47.042320+0000","last_fullsized":"2024-03-23T09:43:47.042320+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024769+0000","last_change":"2024-03-23T09:43:47.024769+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024769+0000","last_undegraded":"2024-03-23T09:43:47.024769+0000","last_fullsized":"2024-03-23T09:43:47.024769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037227+0000","last_change":"2024-03-23T09:43:47.036026+0000","last_active":"2024-03-23T09:43:47.037227+0000","last_peered":"2024-03-23T09:43:47.037227+0000","last_clean":"2024-03-23T09:43:47.037227+0000","last_became_active":"2024-03-23T09:43:47.035690+0000","last_became_peered":"2024-03-23T09:43:47.035690+0000","last_unstale":"2024-03-23T09:43:47.037227+0000","last_undegraded":"2024-03-23T09:43:47.037227+0000","last_fullsized":"2024-03-23T09:43:47.037227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:20.475983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.033106+0000","last_change":"2024-03-23T09:43:47.033106+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.033106+0000","last_undegraded":"2024-03-23T09:43:47.033106+0000","last_fullsized":"2024-03-23T09:43:47.033106+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034465+0000","last_change":"2024-03-23T09:43:47.032740+0000","last_active":"2024-03-23T09:43:47.034465+0000","last_peered":"2024-03-23T09:43:47.034465+0000","last_clean":"2024-03-23T09:43:47.034465+0000","last_became_active":"2024-03-23T09:43:47.032456+0000","last_became_peered":"2024-03-23T09:43:47.032456+0000","last_unstale":"2024-03-23T09:43:47.034465+0000","last_undegraded":"2024-03-23T09:43:47.034465+0000","last_fullsized":"2024-03-23T09:43:47.034465+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021027+0000","last_change":"2024-03-23T09:43:47.021027+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021027+0000","last_undegraded":"2024-03-23T09:43:47.021027+0000","last_fullsized":"2024-03-23T09:43:47.021027+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:07:10.261476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040144+0000","last_change":"2024-03-23T09:43:47.038642+0000","last_active":"2024-03-23T09:43:47.040144+0000","last_peered":"2024-03-23T09:43:47.040144+0000","last_clean":"2024-03-23T09:43:47.040144+0000","last_became_active":"2024-03-23T09:43:47.038385+0000","last_became_peered":"2024-03-23T09:43:47.038385+0000","last_unstale":"2024-03-23T09:43:47.040144+0000","last_undegraded":"2024-03-23T09:43:47.040144+0000","last_fullsized":"2024-03-23T09:43:47.040144+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:33.981220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026250+0000","last_change":"2024-03-23T09:43:47.026250+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026250+0000","last_undegraded":"2024-03-23T09:43:47.026250+0000","last_fullsized":"2024-03-23T09:43:47.026250+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:21.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034573+0000","last_change":"2024-03-23T09:43:47.032632+0000","last_active":"2024-03-23T09:43:47.034573+0000","last_peered":"2024-03-23T09:43:47.034573+0000","last_clean":"2024-03-23T09:43:47.034573+0000","last_became_active":"2024-03-23T09:43:47.032375+0000","last_became_peered":"2024-03-23T09:43:47.032375+0000","last_unstale":"2024-03-23T09:43:47.034573+0000","last_undegraded":"2024-03-23T09:43:47.034573+0000","last_fullsized":"2024-03-23T09:43:47.034573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.029300+0000","last_change":"2024-03-23T09:43:47.029300+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.029300+0000","last_undegraded":"2024-03-23T09:43:47.029300+0000","last_fullsized":"2024-03-23T09:43:47.029300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:14:35.775608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036589+0000","last_change":"2024-03-23T09:43:47.030434+0000","last_active":"2024-03-23T09:43:47.036589+0000","last_peered":"2024-03-23T09:43:47.036589+0000","last_clean":"2024-03-23T09:43:47.036589+0000","last_became_active":"2024-03-23T09:43:47.030164+0000","last_became_peered":"2024-03-23T09:43:47.030164+0000","last_unstale":"2024-03-23T09:43:47.036589+0000","last_undegraded":"2024-03-23T09:43:47.036589+0000","last_fullsized":"2024-03-23T09:43:47.036589+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:09.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022414+0000","last_change":"2024-03-23T09:43:47.022414+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022414+0000","last_undegraded":"2024-03-23T09:43:47.022414+0000","last_fullsized":"2024-03-23T09:43:47.022414+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:22:26.903375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034938+0000","last_change":"2024-03-23T09:43:47.031235+0000","last_active":"2024-03-23T09:43:47.034938+0000","last_peered":"2024-03-23T09:43:47.034938+0000","last_clean":"2024-03-23T09:43:47.034938+0000","last_became_active":"2024-03-23T09:43:47.030949+0000","last_became_peered":"2024-03-23T09:43:47.030949+0000","last_unstale":"2024-03-23T09:43:47.034938+0000","last_undegraded":"2024-03-23T09:43:47.034938+0000","last_fullsized":"2024-03-23T09:43:47.034938+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:58:56.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022818+0000","last_change":"2024-03-23T09:43:47.022818+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022818+0000","last_undegraded":"2024-03-23T09:43:47.022818+0000","last_fullsized":"2024-03-23T09:43:47.022818+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040173+0000","last_change":"2024-03-23T09:43:47.038838+0000","last_active":"2024-03-23T09:43:47.040173+0000","last_peered":"2024-03-23T09:43:47.040173+0000","last_clean":"2024-03-23T09:43:47.040173+0000","last_became_active":"2024-03-23T09:43:47.038574+0000","last_became_peered":"2024-03-23T09:43:47.038574+0000","last_unstale":"2024-03-23T09:43:47.040173+0000","last_undegraded":"2024-03-23T09:43:47.040173+0000","last_fullsized":"2024-03-23T09:43:47.040173+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.030328+0000","last_change":"2024-03-23T09:43:47.030328+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.030328+0000","last_undegraded":"2024-03-23T09:43:47.030328+0000","last_fullsized":"2024-03-23T09:43:47.030328+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:49.207297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.031785+0000","last_change":"2024-03-23T09:43:47.029629+0000","last_active":"2024-03-23T09:43:47.031785+0000","last_peered":"2024-03-23T09:43:47.031785+0000","last_clean":"2024-03-23T09:43:47.031785+0000","last_became_active":"2024-03-23T09:43:47.029338+0000","last_became_peered":"2024-03-23T09:43:47.029338+0000","last_unstale":"2024-03-23T09:43:47.031785+0000","last_undegraded":"2024-03-23T09:43:47.031785+0000","last_fullsized":"2024-03-23T09:43:47.031785+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:15:54.128943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032692+0000","last_change":"2024-03-23T09:43:47.032692+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032692+0000","last_undegraded":"2024-03-23T09:43:47.032692+0000","last_fullsized":"2024-03-23T09:43:47.032692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036684+0000","last_change":"2024-03-23T09:43:47.035203+0000","last_active":"2024-03-23T09:43:47.036684+0000","last_peered":"2024-03-23T09:43:47.036684+0000","last_clean":"2024-03-23T09:43:47.036684+0000","last_became_active":"2024-03-23T09:43:47.034912+0000","last_became_peered":"2024-03-23T09:43:47.034912+0000","last_unstale":"2024-03-23T09:43:47.036684+0000","last_undegraded":"2024-03-23T09:43:47.036684+0000","last_fullsized":"2024-03-23T09:43:47.036684+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:36:25.998787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.019914+0000","last_change":"2024-03-23T09:43:47.019914+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.019914+0000","last_undegraded":"2024-03-23T09:43:47.019914+0000","last_fullsized":"2024-03-23T09:43:47.019914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034026+0000","last_change":"2024-03-23T09:43:47.029065+0000","last_active":"2024-03-23T09:43:47.034026+0000","last_peered":"2024-03-23T09:43:47.034026+0000","last_clean":"2024-03-23T09:43:47.034026+0000","last_became_active":"2024-03-23T09:43:47.028797+0000","last_became_peered":"2024-03-23T09:43:47.028797+0000","last_unstale":"2024-03-23T09:43:47.034026+0000","last_undegraded":"2024-03-23T09:43:47.034026+0000","last_fullsized":"2024-03-23T09:43:47.034026+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:32:33.947908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022836+0000","last_change":"2024-03-23T09:43:47.022836+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022836+0000","last_undegraded":"2024-03-23T09:43:47.022836+0000","last_fullsized":"2024-03-23T09:43:47.022836+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:19:12.228744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040108+0000","last_change":"2024-03-23T09:43:47.038126+0000","last_active":"2024-03-23T09:43:47.040108+0000","last_peered":"2024-03-23T09:43:47.040108+0000","last_clean":"2024-03-23T09:43:47.040108+0000","last_became_active":"2024-03-23T09:43:47.037850+0000","last_became_peered":"2024-03-23T09:43:47.037850+0000","last_unstale":"2024-03-23T09:43:47.040108+0000","last_undegraded":"2024-03-23T09:43:47.040108+0000","last_fullsized":"2024-03-23T09:43:47.040108+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023991+0000","last_change":"2024-03-23T09:43:47.023991+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023991+0000","last_undegraded":"2024-03-23T09:43:47.023991+0000","last_fullsized":"2024-03-23T09:43:47.023991+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:59.768596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037679+0000","last_change":"2024-03-23T09:43:47.036954+0000","last_active":"2024-03-23T09:43:47.037679+0000","last_peered":"2024-03-23T09:43:47.037679+0000","last_clean":"2024-03-23T09:43:47.037679+0000","last_became_active":"2024-03-23T09:43:47.036655+0000","last_became_peered":"2024-03-23T09:43:47.036655+0000","last_unstale":"2024-03-23T09:43:47.037679+0000","last_undegraded":"2024-03-23T09:43:47.037679+0000","last_fullsized":"2024-03-23T09:43:47.037679+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:05:51.129365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023834+0000","last_change":"2024-03-23T09:43:47.023834+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023834+0000","last_undegraded":"2024-03-23T09:43:47.023834+0000","last_fullsized":"2024-03-23T09:43:47.023834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:45.147489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037249+0000","last_change":"2024-03-23T09:43:47.032771+0000","last_active":"2024-03-23T09:43:47.037249+0000","last_peered":"2024-03-23T09:43:47.037249+0000","last_clean":"2024-03-23T09:43:47.037249+0000","last_became_active":"2024-03-23T09:43:47.032489+0000","last_became_peered":"2024-03-23T09:43:47.032489+0000","last_unstale":"2024-03-23T09:43:47.037249+0000","last_undegraded":"2024-03-23T09:43:47.037249+0000","last_fullsized":"2024-03-23T09:43:47.037249+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:48:49.438386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.019914+0000","last_change":"2024-03-23T09:43:47.019914+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.019914+0000","last_undegraded":"2024-03-23T09:43:47.019914+0000","last_fullsized":"2024-03-23T09:43:47.019914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:24:18.285067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.033031+0000","last_change":"2024-03-23T09:43:47.030522+0000","last_active":"2024-03-23T09:43:47.033031+0000","last_peered":"2024-03-23T09:43:47.033031+0000","last_clean":"2024-03-23T09:43:47.033031+0000","last_became_active":"2024-03-23T09:43:47.030216+0000","last_became_peered":"2024-03-23T09:43:47.030216+0000","last_unstale":"2024-03-23T09:43:47.033031+0000","last_undegraded":"2024-03-23T09:43:47.033031+0000","last_fullsized":"2024-03-23T09:43:47.033031+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:32:33.947908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022218+0000","last_change":"2024-03-23T09:43:47.022218+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022218+0000","last_undegraded":"2024-03-23T09:43:47.022218+0000","last_fullsized":"2024-03-23T09:43:47.022218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:30:34.056749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039369+0000","last_change":"2024-03-23T09:43:47.038282+0000","last_active":"2024-03-23T09:43:47.039369+0000","last_peered":"2024-03-23T09:43:47.039369+0000","last_clean":"2024-03-23T09:43:47.039369+0000","last_became_active":"2024-03-23T09:43:47.037787+0000","last_became_peered":"2024-03-23T09:43:47.037787+0000","last_unstale":"2024-03-23T09:43:47.039369+0000","last_undegraded":"2024-03-23T09:43:47.039369+0000","last_fullsized":"2024-03-23T09:43:47.039369+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:10.101674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021303+0000","last_change":"2024-03-23T09:43:47.021303+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021303+0000","last_undegraded":"2024-03-23T09:43:47.021303+0000","last_fullsized":"2024-03-23T09:43:47.021303+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:52:24.758413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039546+0000","last_change":"2024-03-23T09:43:47.037606+0000","last_active":"2024-03-23T09:43:47.039546+0000","last_peered":"2024-03-23T09:43:47.039546+0000","last_clean":"2024-03-23T09:43:47.039546+0000","last_became_active":"2024-03-23T09:43:47.037315+0000","last_became_peered":"2024-03-23T09:43:47.037315+0000","last_unstale":"2024-03-23T09:43:47.039546+0000","last_undegraded":"2024-03-23T09:43:47.039546+0000","last_fullsized":"2024-03-23T09:43:47.039546+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:22.192970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":200,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-23T09:43:46.013864+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:43:46.013864+0000","last_peered":"2024-03-23T09:43:46.013864+0000","last_clean":"2024-03-23T09:43:46.013864+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:43:46.013864+0000","last_undegraded":"2024-03-23T09:43:46.013864+0000","last_fullsized":"2024-03-23T09:43:46.013864+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1265664,"data_stored":864722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69299999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.62}]},{"osd":2,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.65900000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72699999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56999999999999995}]},{"osd":3,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.433,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58399999999999996}]},{"osd":4,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.377,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72099999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45300000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66500000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71499999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73799999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53700000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":267468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.499,"15min":0.498},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69299999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.54100000000000004}]},{"osd":1,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.40200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.42699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.627}]},{"osd":2,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.41299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.46100000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.496}]},{"osd":3,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52900000000000003,"15min":0.52300000000000002},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.73199999999999998,"5min":0.752,"15min":0.752},"last":0.60999999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.38700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.67500000000000004}]},{"osd":5,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.72399999999999998,"5min":0.747,"15min":0.747},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":268087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48099999999999998,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.42199999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52100000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52900000000000003,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.434,"5min":0.376,"15min":0.376},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.50700000000000001,"15min":0.49299999999999999},"min":{"1min":0.46700000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.46800000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.54100000000000004}]},{"osd":4,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47999999999999998,"15min":0.47699999999999998},"min":{"1min":0.33300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.439},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.374,"15min":0.374},"max":{"1min":0.69999999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63400000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.34799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.749,"15min":0.749},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.51800000000000002},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70099999999999996,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.60899999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.44600000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71499999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.621}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":268087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.69899999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.49199999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.46400000000000002},"min":{"1min":0.35699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.498,"15min":0.498},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.40600000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.42999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.752,"5min":0.755,"15min":0.755},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.747,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.50800000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70399999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.30199999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68799999999999994,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.308,"5min":0.253,"15min":0.253},"max":{"1min":0.68400000000000005,"5min":1.008,"15min":1.008},"last":0.36499999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.34000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70099999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.32400000000000001}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65800000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56200000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":671744,"data_stored":274354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.307,"15min":0.30499999999999999},"min":{"1min":0.25900000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30399999999999999,"15min":0.30199999999999999},"min":{"1min":0.25600000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.42899999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.437},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.55100000000000005}]},{"osd":2,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.313,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76500000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53100000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.73499999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.47399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81499999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.42899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.70999999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.81000000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.41799999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71499999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.67800000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.626},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.59999999999999998},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.71599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":671744,"data_stored":274354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.34000000000000002,"15min":0.34399999999999997},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.48799999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.25,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48799999999999999,"5min":0.872,"15min":0.872},"last":0.23000000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.629,"15min":0.629},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58599999999999997,"5min":0.746,"15min":0.746},"last":0.34100000000000003}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.47399999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.40200000000000002,"15min":0.40699999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55800000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.32300000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72499999999999998,"5min":1.216,"15min":1.216},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.753,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.624,"15min":0.63200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.71599999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75800000000000001,"5min":1.101,"15min":1.101},"last":0.66800000000000004}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.70699999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.63800000000000001,"15min":0.64800000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.72699999999999998,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.76500000000000001,"5min":1.155,"15min":1.155},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72099999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":659456,"data_stored":268087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33600000000000002,"15min":0.33300000000000002},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.53300000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61899999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.30299999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48399999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.374,"15min":0.37},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.55200000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41099999999999998,"15min":0.40699999999999997},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70499999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60899999999999999,"5min":2.75,"15min":2.75},"last":0.41499999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.45500000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91100000000000003,"5min":2.778,"15min":2.778},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68999999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.46300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88100000000000001,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.441},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.628}]},{"osd":7,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65400000000000003,"15min":0.64700000000000002},"min":{"1min":0.51800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":0.92400000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.76000000000000001,"15min":0.77800000000000002},"min":{"1min":0.44600000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.85099999999999998,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.52800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1261568,"data_stored":864103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32200000000000001,"15min":0.31900000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.499,"5min":0.504,"15min":0.504},"last":0.316},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.34499999999999997},"min":{"1min":0.25900000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.51300000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.30099999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36099999999999999,"15min":0.35899999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36099999999999999},"min":{"1min":0.27000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.33700000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.379,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.42299999999999999,"15min":0.41999999999999998},"min":{"1min":0.32100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.40999999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53600000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.81499999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54600000000000004}]},{"osd":6,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83299999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.88400000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55500000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:43:51.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2024-03-23T09:43:51.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.028360+0000', 'last_change': '2024-03-23T09:43:47.028360+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.028360+0000', 'last_undegraded': '2024-03-23T09:43:47.028360+0000', 'last_fullsized': '2024-03-23T09:43:47.028360+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:54:41.177596+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:43:51.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2024-03-23T09:43:51.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.021545+0000', 'last_change': '2024-03-23T09:43:47.021545+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.021545+0000', 'last_undegraded': '2024-03-23T09:43:47.021545+0000', 'last_fullsized': '2024-03-23T09:43:47.021545+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T13:00:36.262351+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:43:51.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2024-03-23T09:43:51.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.027083+0000', 'last_change': '2024-03-23T09:43:47.027083+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.027083+0000', 'last_undegraded': '2024-03-23T09:43:47.027083+0000', 'last_fullsized': '2024-03-23T09:43:47.027083+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:22:26.903375+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:43:51.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2024-03-23T09:43:51.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.032839+0000', 'last_change': '2024-03-23T09:43:47.032839+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.032839+0000', 'last_undegraded': '2024-03-23T09:43:47.032839+0000', 'last_fullsized': '2024-03-23T09:43:47.032839+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.026471+0000', 'last_change': '2024-03-23T09:43:47.026471+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.026471+0000', 'last_undegraded': '2024-03-23T09:43:47.026471+0000', 'last_fullsized': '2024-03-23T09:43:47.026471+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:06:17.338788+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.022332+0000', 'last_change': '2024-03-23T09:43:47.022332+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.022332+0000', 'last_undegraded': '2024-03-23T09:43:47.022332+0000', 'last_fullsized': '2024-03-23T09:43:47.022332+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023985+0000', 'last_change': '2024-03-23T09:43:47.023985+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023985+0000', 'last_undegraded': '2024-03-23T09:43:47.023985+0000', 'last_fullsized': '2024-03-23T09:43:47.023985+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:44:18.230315+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.021778+0000', 'last_change': '2024-03-23T09:43:47.021778+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.021778+0000', 'last_undegraded': '2024-03-23T09:43:47.021778+0000', 'last_fullsized': '2024-03-23T09:43:47.021778+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:55:17.926828+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023723+0000', 'last_change': '2024-03-23T09:43:47.023723+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023723+0000', 'last_undegraded': '2024-03-23T09:43:47.023723+0000', 'last_fullsized': '2024-03-23T09:43:47.023723+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:30:19.820479+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.020703+0000', 'last_change': '2024-03-23T09:43:47.020703+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.020703+0000', 'last_undegraded': '2024-03-23T09:43:47.020703+0000', 'last_fullsized': '2024-03-23T09:43:47.020703+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:35:30.364827+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2024-03-23T09:43:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.020633+0000', 'last_change': '2024-03-23T09:43:47.020633+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.020633+0000', 'last_undegraded': '2024-03-23T09:43:47.020633+0000', 'last_fullsized': '2024-03-23T09:43:47.020633+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:48:50.443256+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:43:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2024-03-23T09:43:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.032222+0000', 'last_change': '2024-03-23T09:43:47.032222+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.032222+0000', 'last_undegraded': '2024-03-23T09:43:47.032222+0000', 'last_fullsized': '2024-03-23T09:43:47.032222+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:43:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2024-03-23T09:43:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.026161+0000', 'last_change': '2024-03-23T09:43:47.026161+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.026161+0000', 'last_undegraded': '2024-03-23T09:43:47.026161+0000', 'last_fullsized': '2024-03-23T09:43:47.026161+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:43:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2024-03-23T09:43:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023765+0000', 'last_change': '2024-03-23T09:43:47.023765+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023765+0000', 'last_undegraded': '2024-03-23T09:43:47.023765+0000', 'last_fullsized': '2024-03-23T09:43:47.023765+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:05:58.075182+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:43:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2024-03-23T09:43:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023448+0000', 'last_change': '2024-03-23T09:43:47.023448+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023448+0000', 'last_undegraded': '2024-03-23T09:43:47.023448+0000', 'last_fullsized': '2024-03-23T09:43:47.023448+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:00:56.576713+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:43:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2024-03-23T09:43:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.021932+0000', 'last_change': '2024-03-23T09:43:47.021932+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.021932+0000', 'last_undegraded': '2024-03-23T09:43:47.021932+0000', 'last_fullsized': '2024-03-23T09:43:47.021932+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:34:21.480853+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:43:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2024-03-23T09:43:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023522+0000', 'last_change': '2024-03-23T09:43:47.023522+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023522+0000', 'last_undegraded': '2024-03-23T09:43:47.023522+0000', 'last_fullsized': '2024-03-23T09:43:47.023522+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:50:34.378460+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.025211+0000', 'last_change': '2024-03-23T09:43:47.025211+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.025211+0000', 'last_undegraded': '2024-03-23T09:43:47.025211+0000', 'last_fullsized': '2024-03-23T09:43:47.025211+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:05:49.207297+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.026927+0000', 'last_change': '2024-03-23T09:43:47.026927+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.026927+0000', 'last_undegraded': '2024-03-23T09:43:47.026927+0000', 'last_fullsized': '2024-03-23T09:43:47.026927+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:59:59.729962+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.024967+0000', 'last_change': '2024-03-23T09:43:47.024967+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.024967+0000', 'last_undegraded': '2024-03-23T09:43:47.024967+0000', 'last_fullsized': '2024-03-23T09:43:47.024967+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:07:44.532537+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.025472+0000', 'last_change': '2024-03-23T09:43:47.025472+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.025472+0000', 'last_undegraded': '2024-03-23T09:43:47.025472+0000', 'last_fullsized': '2024-03-23T09:43:47.025472+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:19:12.228744+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.026145+0000', 'last_change': '2024-03-23T09:43:47.026145+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.026145+0000', 'last_undegraded': '2024-03-23T09:43:47.026145+0000', 'last_fullsized': '2024-03-23T09:43:47.026145+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:47:24.835835+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.024831+0000', 'last_change': '2024-03-23T09:43:47.024831+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.024831+0000', 'last_undegraded': '2024-03-23T09:43:47.024831+0000', 'last_fullsized': '2024-03-23T09:43:47.024831+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:10:55.583385+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2024-03-23T09:43:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.024086+0000', 'last_change': '2024-03-23T09:43:47.024086+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.024086+0000', 'last_undegraded': '2024-03-23T09:43:47.024086+0000', 'last_fullsized': '2024-03-23T09:43:47.024086+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:54:41.177596+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:43:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2024-03-23T09:43:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.027386+0000', 'last_change': '2024-03-23T09:43:47.027386+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.027386+0000', 'last_undegraded': '2024-03-23T09:43:47.027386+0000', 'last_fullsized': '2024-03-23T09:43:47.027386+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:36:08.737490+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:43:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2024-03-23T09:43:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.032083+0000', 'last_change': '2024-03-23T09:43:47.032083+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.032083+0000', 'last_undegraded': '2024-03-23T09:43:47.032083+0000', 'last_fullsized': '2024-03-23T09:43:47.032083+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:45:37.390159+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:43:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2024-03-23T09:43:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.024006+0000', 'last_change': '2024-03-23T09:43:47.024006+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.024006+0000', 'last_undegraded': '2024-03-23T09:43:47.024006+0000', 'last_fullsized': '2024-03-23T09:43:47.024006+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:37:11.106544+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:43:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2024-03-23T09:43:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.026420+0000', 'last_change': '2024-03-23T09:43:47.026420+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.026420+0000', 'last_undegraded': '2024-03-23T09:43:47.026420+0000', 'last_fullsized': '2024-03-23T09:43:47.026420+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:43:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2024-03-23T09:43:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.024600+0000', 'last_change': '2024-03-23T09:43:47.024600+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.024600+0000', 'last_undegraded': '2024-03-23T09:43:47.024600+0000', 'last_fullsized': '2024-03-23T09:43:47.024600+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:00:04.150928+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:43:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2024-03-23T09:43:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023232+0000', 'last_change': '2024-03-23T09:43:47.023232+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023232+0000', 'last_undegraded': '2024-03-23T09:43:47.023232+0000', 'last_fullsized': '2024-03-23T09:43:47.023232+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:43:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2024-03-23T09:43:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.026157+0000', 'last_change': '2024-03-23T09:43:47.026157+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.026157+0000', 'last_undegraded': '2024-03-23T09:43:47.026157+0000', 'last_fullsized': '2024-03-23T09:43:47.026157+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:46:16.837675+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:43:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2024-03-23T09:43:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.021430+0000', 'last_change': '2024-03-23T09:43:47.021430+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.021430+0000', 'last_undegraded': '2024-03-23T09:43:47.021430+0000', 'last_fullsized': '2024-03-23T09:43:47.021430+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:02:58.178256+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:43:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2024-03-23T09:43:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023483+0000', 'last_change': '2024-03-23T09:43:47.023483+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023483+0000', 'last_undegraded': '2024-03-23T09:43:47.023483+0000', 'last_fullsized': '2024-03-23T09:43:47.023483+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:43:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2024-03-23T09:43:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.020741+0000', 'last_change': '2024-03-23T09:43:47.020741+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.020741+0000', 'last_undegraded': '2024-03-23T09:43:47.020741+0000', 'last_fullsized': '2024-03-23T09:43:47.020741+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T13:05:21.051130+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:43:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2024-03-23T09:43:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.021823+0000', 'last_change': '2024-03-23T09:43:47.021823+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.021823+0000', 'last_undegraded': '2024-03-23T09:43:47.021823+0000', 'last_fullsized': '2024-03-23T09:43:47.021823+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:43:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2024-03-23T09:43:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023508+0000', 'last_change': '2024-03-23T09:43:47.023508+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023508+0000', 'last_undegraded': '2024-03-23T09:43:47.023508+0000', 'last_fullsized': '2024-03-23T09:43:47.023508+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T15:58:45.147489+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:43:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2024-03-23T09:43:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.024459+0000', 'last_change': '2024-03-23T09:43:47.024459+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.024459+0000', 'last_undegraded': '2024-03-23T09:43:47.024459+0000', 'last_fullsized': '2024-03-23T09:43:47.024459+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:28:32.181328+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:43:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2024-03-23T09:43:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.025198+0000', 'last_change': '2024-03-23T09:43:47.025198+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.025198+0000', 'last_undegraded': '2024-03-23T09:43:47.025198+0000', 'last_fullsized': '2024-03-23T09:43:47.025198+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:42:29.904309+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:43:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2024-03-23T09:43:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023485+0000', 'last_change': '2024-03-23T09:43:47.023485+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023485+0000', 'last_undegraded': '2024-03-23T09:43:47.023485+0000', 'last_fullsized': '2024-03-23T09:43:47.023485+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:59:59.729962+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:43:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2024-03-23T09:43:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.022665+0000', 'last_change': '2024-03-23T09:43:47.022665+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.022665+0000', 'last_undegraded': '2024-03-23T09:43:47.022665+0000', 'last_fullsized': '2024-03-23T09:43:47.022665+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:25:06.163966+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:43:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2024-03-23T09:43:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023652+0000', 'last_change': '2024-03-23T09:43:47.023652+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023652+0000', 'last_undegraded': '2024-03-23T09:43:47.023652+0000', 'last_fullsized': '2024-03-23T09:43:47.023652+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:36:27.003657+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:43:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2024-03-23T09:43:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.025795+0000', 'last_change': '2024-03-23T09:43:47.025795+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.025795+0000', 'last_undegraded': '2024-03-23T09:43:47.025795+0000', 'last_fullsized': '2024-03-23T09:43:47.025795+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:43:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2024-03-23T09:43:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023205+0000', 'last_change': '2024-03-23T09:43:47.023205+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023205+0000', 'last_undegraded': '2024-03-23T09:43:47.023205+0000', 'last_fullsized': '2024-03-23T09:43:47.023205+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T15:28:37.758076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:43:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2024-03-23T09:43:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.026681+0000', 'last_change': '2024-03-23T09:43:47.026681+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.026681+0000', 'last_undegraded': '2024-03-23T09:43:47.026681+0000', 'last_fullsized': '2024-03-23T09:43:47.026681+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:37:11.106544+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:43:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2024-03-23T09:43:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.029812+0000', 'last_change': '2024-03-23T09:43:47.029812+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.029812+0000', 'last_undegraded': '2024-03-23T09:43:47.029812+0000', 'last_fullsized': '2024-03-23T09:43:47.029812+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T09:51:10.516410+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:43:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2024-03-23T09:43:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.024286+0000', 'last_change': '2024-03-23T09:43:47.024286+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.024286+0000', 'last_undegraded': '2024-03-23T09:43:47.024286+0000', 'last_fullsized': '2024-03-23T09:43:47.024286+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:43:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2024-03-23T09:43:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023273+0000', 'last_change': '2024-03-23T09:43:47.023273+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023273+0000', 'last_undegraded': '2024-03-23T09:43:47.023273+0000', 'last_fullsized': '2024-03-23T09:43:47.023273+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:27:01.253597+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:43:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2024-03-23T09:43:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.024358+0000', 'last_change': '2024-03-23T09:43:47.024358+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.024358+0000', 'last_undegraded': '2024-03-23T09:43:47.024358+0000', 'last_fullsized': '2024-03-23T09:43:47.024358+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:02:58.178256+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:43:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2024-03-23T09:43:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.024769+0000', 'last_change': '2024-03-23T09:43:47.024769+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.024769+0000', 'last_undegraded': '2024-03-23T09:43:47.024769+0000', 'last_fullsized': '2024-03-23T09:43:47.024769+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:43:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2024-03-23T09:43:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.033106+0000', 'last_change': '2024-03-23T09:43:47.033106+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.033106+0000', 'last_undegraded': '2024-03-23T09:43:47.033106+0000', 'last_fullsized': '2024-03-23T09:43:47.033106+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:43:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2024-03-23T09:43:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.021027+0000', 'last_change': '2024-03-23T09:43:47.021027+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.021027+0000', 'last_undegraded': '2024-03-23T09:43:47.021027+0000', 'last_fullsized': '2024-03-23T09:43:47.021027+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:07:10.261476+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:43:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2024-03-23T09:43:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.026250+0000', 'last_change': '2024-03-23T09:43:47.026250+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.026250+0000', 'last_undegraded': '2024-03-23T09:43:47.026250+0000', 'last_fullsized': '2024-03-23T09:43:47.026250+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:34:21.480853+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:43:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2024-03-23T09:43:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.029300+0000', 'last_change': '2024-03-23T09:43:47.029300+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.029300+0000', 'last_undegraded': '2024-03-23T09:43:47.029300+0000', 'last_fullsized': '2024-03-23T09:43:47.029300+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T21:14:35.775608+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:43:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2024-03-23T09:43:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.022414+0000', 'last_change': '2024-03-23T09:43:47.022414+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.022414+0000', 'last_undegraded': '2024-03-23T09:43:47.022414+0000', 'last_fullsized': '2024-03-23T09:43:47.022414+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:22:26.903375+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:43:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2024-03-23T09:43:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.022818+0000', 'last_change': '2024-03-23T09:43:47.022818+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.022818+0000', 'last_undegraded': '2024-03-23T09:43:47.022818+0000', 'last_fullsized': '2024-03-23T09:43:47.022818+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:43:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2024-03-23T09:43:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.030328+0000', 'last_change': '2024-03-23T09:43:47.030328+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.030328+0000', 'last_undegraded': '2024-03-23T09:43:47.030328+0000', 'last_fullsized': '2024-03-23T09:43:47.030328+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:05:49.207297+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:43:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2024-03-23T09:43:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.032692+0000', 'last_change': '2024-03-23T09:43:47.032692+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.032692+0000', 'last_undegraded': '2024-03-23T09:43:47.032692+0000', 'last_fullsized': '2024-03-23T09:43:47.032692+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:43:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2024-03-23T09:43:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.019914+0000', 'last_change': '2024-03-23T09:43:47.019914+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.019914+0000', 'last_undegraded': '2024-03-23T09:43:47.019914+0000', 'last_fullsized': '2024-03-23T09:43:47.019914+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:43:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2024-03-23T09:43:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.022836+0000', 'last_change': '2024-03-23T09:43:47.022836+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.022836+0000', 'last_undegraded': '2024-03-23T09:43:47.022836+0000', 'last_fullsized': '2024-03-23T09:43:47.022836+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:19:12.228744+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:43:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2024-03-23T09:43:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023991+0000', 'last_change': '2024-03-23T09:43:47.023991+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023991+0000', 'last_undegraded': '2024-03-23T09:43:47.023991+0000', 'last_fullsized': '2024-03-23T09:43:47.023991+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T13:07:59.768596+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:43:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2024-03-23T09:43:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.023834+0000', 'last_change': '2024-03-23T09:43:47.023834+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.023834+0000', 'last_undegraded': '2024-03-23T09:43:47.023834+0000', 'last_fullsized': '2024-03-23T09:43:47.023834+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T15:58:45.147489+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:43:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2024-03-23T09:43:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.019914+0000', 'last_change': '2024-03-23T09:43:47.019914+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.019914+0000', 'last_undegraded': '2024-03-23T09:43:47.019914+0000', 'last_fullsized': '2024-03-23T09:43:47.019914+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:24:18.285067+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:43:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2024-03-23T09:43:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.022218+0000', 'last_change': '2024-03-23T09:43:47.022218+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.022218+0000', 'last_undegraded': '2024-03-23T09:43:47.022218+0000', 'last_fullsized': '2024-03-23T09:43:47.022218+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:30:34.056749+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:43:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2024-03-23T09:43:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:43:47.021303+0000', 'last_change': '2024-03-23T09:43:47.021303+0000', 'last_active': '2024-03-23T09:43:47.010185+0000', 'last_peered': '2024-03-23T09:43:47.010185+0000', 'last_clean': '2024-03-23T09:43:47.010185+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:43:47.021303+0000', 'last_undegraded': '2024-03-23T09:43:47.021303+0000', 'last_fullsized': '2024-03-23T09:43:47.021303+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'last_clean_scrub_stamp': '2024-03-23T09:43:47.010185+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T09:52:24.758413+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:43:51.101 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-23T09:43:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.226+0000 7f9694070640 1 Processor -- start 2024-03-23T09:43:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.226+0000 7f9694070640 1 -- start start 2024-03-23T09:43:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.227+0000 7f9694070640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f968c15aba0 0x7f968c0b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.227+0000 7f9694070640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f968c0ccb00 0x7f968c0cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.227+0000 7f9694070640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968c06acc0 con 0x7f968c0b1770 2024-03-23T09:43:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.227+0000 7f9694070640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968c15af80 con 0x7f968c0ccb00 2024-03-23T09:43:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.227+0000 7f9694070640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968c0cbdd0 con 0x7f968c15aba0 2024-03-23T09:43:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.227+0000 7f969260d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f968c0ccb00 0x7f968c0cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.227+0000 7f969260d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f968c0ccb00 0x7f968c0cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39138/0 (socket says 172.21.15.47:39138) 2024-03-23T09:43:51.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.227+0000 7f969260d640 1 -- 172.21.15.47:0/1010036618 learned_addr learned my addr 172.21.15.47:0/1010036618 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:51.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.227+0000 7f9691e0c640 1 --2- 172.21.15.47:0/1010036618 >> v2:172.21.15.175:3300/0 conn(0x7f968c15aba0 0x7f968c0b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:51.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.227+0000 7f969260d640 1 -- 172.21.15.47:0/1010036618 >> v2:172.21.15.175:3300/0 conn(0x7f968c15aba0 msgr2=0x7f968c0b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:51.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.227+0000 7f969260d640 1 --2- 172.21.15.47:0/1010036618 >> v2:172.21.15.175:3300/0 conn(0x7f968c15aba0 0x7f968c0b41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.227+0000 7f969260d640 1 -- 172.21.15.47:0/1010036618 >> v1:172.21.15.175:6789/0 conn(0x7f968c0b1770 legacy=0x7f968c0b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:51.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.227+0000 7f969260d640 1 -- 172.21.15.47:0/1010036618 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c0cc670 con 0x7f968c0ccb00 2024-03-23T09:43:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.228+0000 7f969260d640 1 --2- 172.21.15.47:0/1010036618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f968c0ccb00 0x7f968c0cd950 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f968c070860 tx=0x7f968402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81abaa56d3cbd10e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.228+0000 7f9690e0a640 1 -- 172.21.15.47:0/1010036618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684031870 con 0x7f968c0ccb00 2024-03-23T09:43:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.228+0000 7f9690e0a640 1 -- 172.21.15.47:0/1010036618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96840319d0 con 0x7f968c0ccb00 2024-03-23T09:43:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.228+0000 7f9690e0a640 1 -- 172.21.15.47:0/1010036618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684031d60 con 0x7f968c0ccb00 2024-03-23T09:43:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.229+0000 7f9694070640 1 -- 172.21.15.47:0/1010036618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f968c0ccb00 msgr2=0x7f968c0cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.229+0000 7f9694070640 1 --2- 172.21.15.47:0/1010036618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f968c0ccb00 0x7f968c0cd950 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f968c070860 tx=0x7f968402b910 comp rx=0 tx=0).stop 2024-03-23T09:43:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.229+0000 7f9694070640 1 -- 172.21.15.47:0/1010036618 shutdown_connections 2024-03-23T09:43:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.229+0000 7f9694070640 1 --2- 172.21.15.47:0/1010036618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f968c0ccb00 0x7f968c0cd950 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.229+0000 7f9694070640 1 --2- 172.21.15.47:0/1010036618 >> v2:172.21.15.175:3300/0 conn(0x7f968c15aba0 0x7f968c0b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.229+0000 7f9694070640 1 -- 172.21.15.47:0/1010036618 >> 172.21.15.47:0/1010036618 conn(0x7f968c098a00 msgr2=0x7f968c0b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.229+0000 7f9694070640 1 -- 172.21.15.47:0/1010036618 shutdown_connections 2024-03-23T09:43:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.229+0000 7f9694070640 1 -- 172.21.15.47:0/1010036618 wait complete. 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.230+0000 7f9694070640 1 Processor -- start 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.230+0000 7f9694070640 1 -- start start 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9694070640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f968c0b1770 0x7f968c0998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9694070640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f968c0ccb00 0x7f968c0ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9694070640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f968c15aba0 0x7f968c0adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9694070640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c09ce00 con 0x7f968c0b1770 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9694070640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c09c180 con 0x7f968c15aba0 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9694070640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c09c460 con 0x7f968c0ccb00 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9691e0c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f968c0b1770 0x7f968c0998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9691e0c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f968c0b1770 0x7f968c0998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39144/0 (socket says 172.21.15.47:39144) 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9691e0c640 1 -- 172.21.15.47:0/3471661742 learned_addr learned my addr 172.21.15.47:0/3471661742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f969260d640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f968c15aba0 0x7f968c0adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f969160b640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f968c0ccb00 0x7f968c0ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:51.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9691e0c640 1 -- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f968c0ccb00 msgr2=0x7f968c0ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:51.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9691e0c640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f968c0ccb00 0x7f968c0ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9691e0c640 1 -- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f968c15aba0 msgr2=0x7f968c0adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:51.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9691e0c640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f968c15aba0 0x7f968c0adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9691e0c640 1 -- 172.21.15.47:0/3471661742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96840094d0 con 0x7f968c0b1770 2024-03-23T09:43:51.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f969160b640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f968c0ccb00 0x7f968c0ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:51.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9691e0c640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f968c0b1770 0x7f968c0998f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f967c007e40 tx=0x7f967c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:51.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f968a7fc640 1 -- 172.21.15.47:0/3471661742 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c014620 con 0x7f968c0b1770 2024-03-23T09:43:51.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f968a7fc640 1 -- 172.21.15.47:0/3471661742 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c00ee90 con 0x7f968c0b1770 2024-03-23T09:43:51.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f9694070640 1 -- 172.21.15.47:0/3471661742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c09c740 con 0x7f968c0b1770 2024-03-23T09:43:51.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.231+0000 7f968a7fc640 1 -- 172.21.15.47:0/3471661742 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c0149c0 con 0x7f968c0b1770 2024-03-23T09:43:51.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.232+0000 7f9694070640 1 -- 172.21.15.47:0/3471661742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c099e30 con 0x7f968c0b1770 2024-03-23T09:43:51.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.232+0000 7f9694070640 1 -- 172.21.15.47:0/3471661742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c099ff0 con 0x7f968c0b1770 2024-03-23T09:43:51.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.233+0000 7f968a7fc640 1 -- 172.21.15.47:0/3471661742 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f967c014b20 con 0x7f968c0b1770 2024-03-23T09:43:51.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.233+0000 7f968a7fc640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f967806ac00 0x7f967806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:51.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.233+0000 7f968a7fc640 1 -- 172.21.15.47:0/3471661742 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f967c089d60 con 0x7f968c0b1770 2024-03-23T09:43:51.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.233+0000 7f969160b640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f967806ac00 0x7f967806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:51.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.234+0000 7f969160b640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f967806ac00 0x7f967806d0c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f968c0c2fb0 tx=0x7f9680009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:51.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.238+0000 7f968a7fc640 1 -- 172.21.15.47:0/3471661742 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f967c054ea0 con 0x7f968c0b1770 2024-03-23T09:43:51.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.487+0000 7f9694070640 1 -- 172.21.15.47:0/3471661742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f968c09d070 con 0x7f968c0b1770 2024-03-23T09:43:51.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.487+0000 7f968a7fc640 1 -- 172.21.15.47:0/3471661742 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1581 (secure 0 0 0) 0x7f967c0584c0 con 0x7f968c0b1770 2024-03-23T09:43:51.487 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:51.487 INFO:teuthology.orchestra.run.smithi047.stdout:{"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":185,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":58,"num_osds":8,"num_up_osds":8,"osd_up_since":1711186856,"num_in_osds":8,"osd_in_since":1711186848,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":244649984,"bytes_avail":772849463296,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":24,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":6171}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-03-23T09:43:10.369143+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-23T09:43:51.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.489+0000 7f9694070640 1 -- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f967806ac00 msgr2=0x7f967806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:51.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.489+0000 7f9694070640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f967806ac00 0x7f967806d0c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f968c0c2fb0 tx=0x7f9680009040 comp rx=0 tx=0).stop 2024-03-23T09:43:51.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.489+0000 7f9694070640 1 -- 172.21.15.47:0/3471661742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f968c0b1770 msgr2=0x7f968c0998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:51.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.489+0000 7f9694070640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f968c0b1770 0x7f968c0998f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f967c007e40 tx=0x7f967c00e4b0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.489+0000 7f9694070640 1 -- 172.21.15.47:0/3471661742 shutdown_connections 2024-03-23T09:43:51.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.489+0000 7f9694070640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f967806ac00 0x7f967806d0c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.489+0000 7f9694070640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f968c15aba0 0x7f968c0adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.489+0000 7f9694070640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f968c0ccb00 0x7f968c0ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.489+0000 7f9694070640 1 --2- 172.21.15.47:0/3471661742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f968c0b1770 0x7f968c0998f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.489+0000 7f9694070640 1 -- 172.21.15.47:0/3471661742 >> 172.21.15.47:0/3471661742 conn(0x7f968c098a00 msgr2=0x7f968c0ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.489+0000 7f9694070640 1 -- 172.21.15.47:0/3471661742 shutdown_connections 2024-03-23T09:43:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.489+0000 7f9694070640 1 -- 172.21.15.47:0/3471661742 wait complete. 2024-03-23T09:43:51.500 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-23T09:43:51.500 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:43:51.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.601+0000 7fba7f038640 1 Processor -- start 2024-03-23T09:43:51.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.601+0000 7fba7f038640 1 -- start start 2024-03-23T09:43:51.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7f038640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fba7815cb40 0x7fba78160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:51.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7f038640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba780b8300 0x7fba780cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7f038640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba78160cb0 0x7fba780cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7f038640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba780b5a10 con 0x7fba78160cb0 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7f038640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba780d0ee0 con 0x7fba7815cb40 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7f038640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba781616f0 con 0x7fba780b8300 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7d5d5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba78160cb0 0x7fba780cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7d5d5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba78160cb0 0x7fba780cf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39148/0 (socket says 172.21.15.47:39148) 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7d5d5640 1 -- 172.21.15.47:0/213330579 learned_addr learned my addr 172.21.15.47:0/213330579 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba77fff640 1 --2- 172.21.15.47:0/213330579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba780b8300 0x7fba780cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7cdd4640 1 --2- 172.21.15.47:0/213330579 >> v2:172.21.15.175:3300/0 conn(0x7fba7815cb40 0x7fba78160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7d5d5640 1 -- 172.21.15.47:0/213330579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba780b8300 msgr2=0x7fba780cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7d5d5640 1 --2- 172.21.15.47:0/213330579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba780b8300 0x7fba780cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7d5d5640 1 -- 172.21.15.47:0/213330579 >> v2:172.21.15.175:3300/0 conn(0x7fba7815cb40 msgr2=0x7fba78160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7d5d5640 1 --2- 172.21.15.47:0/213330579 >> v2:172.21.15.175:3300/0 conn(0x7fba7815cb40 0x7fba78160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.602+0000 7fba7d5d5640 1 -- 172.21.15.47:0/213330579 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba78161130 con 0x7fba78160cb0 2024-03-23T09:43:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.603+0000 7fba7d5d5640 1 --2- 172.21.15.47:0/213330579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba78160cb0 0x7fba780cf6d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fba780cd130 tx=0x7fba7002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c9b680c2b4da7f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:51.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.603+0000 7fba777fe640 1 -- 172.21.15.47:0/213330579 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70031870 con 0x7fba78160cb0 2024-03-23T09:43:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.603+0000 7fba777fe640 1 -- 172.21.15.47:0/213330579 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba700319d0 con 0x7fba78160cb0 2024-03-23T09:43:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.603+0000 7fba777fe640 1 -- 172.21.15.47:0/213330579 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70031d60 con 0x7fba78160cb0 2024-03-23T09:43:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.604+0000 7fba7f038640 1 -- 172.21.15.47:0/213330579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba78160cb0 msgr2=0x7fba780cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.604+0000 7fba7f038640 1 --2- 172.21.15.47:0/213330579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba78160cb0 0x7fba780cf6d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fba780cd130 tx=0x7fba7002b910 comp rx=0 tx=0).stop 2024-03-23T09:43:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.604+0000 7fba7f038640 1 -- 172.21.15.47:0/213330579 shutdown_connections 2024-03-23T09:43:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.604+0000 7fba7f038640 1 --2- 172.21.15.47:0/213330579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba78160cb0 0x7fba780cf6d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.604+0000 7fba7f038640 1 --2- 172.21.15.47:0/213330579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba780b8300 0x7fba780cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.604+0000 7fba7f038640 1 --2- 172.21.15.47:0/213330579 >> v2:172.21.15.175:3300/0 conn(0x7fba7815cb40 0x7fba78160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.604+0000 7fba7f038640 1 -- 172.21.15.47:0/213330579 >> 172.21.15.47:0/213330579 conn(0x7fba7806b160 msgr2=0x7fba7806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.604+0000 7fba7f038640 1 -- 172.21.15.47:0/213330579 shutdown_connections 2024-03-23T09:43:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.604+0000 7fba7f038640 1 -- 172.21.15.47:0/213330579 wait complete. 2024-03-23T09:43:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.605+0000 7fba7f038640 1 Processor -- start 2024-03-23T09:43:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.605+0000 7fba7f038640 1 -- start start 2024-03-23T09:43:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7f038640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba780b8300 0x7fba7814cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7f038640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba7815cb40 0x7fba780b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7f038640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba78160cb0 0x7fba780b7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7f038640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7815d9f0 con 0x7fba780b8300 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7f038640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba781541c0 con 0x7fba7815cb40 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7f038640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba78153ee0 con 0x7fba78160cb0 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7cdd4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba780b8300 0x7fba7814cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7cdd4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba780b8300 0x7fba7814cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39150/0 (socket says 172.21.15.47:39150) 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7cdd4640 1 -- 172.21.15.47:0/121287306 learned_addr learned my addr 172.21.15.47:0/121287306 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7cdd4640 1 -- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba78160cb0 msgr2=0x7fba780b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7d5d5640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba78160cb0 0x7fba780b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba77fff640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba7815cb40 0x7fba780b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7cdd4640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba78160cb0 0x7fba780b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7cdd4640 1 -- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba7815cb40 msgr2=0x7fba780b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7cdd4640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba7815cb40 0x7fba780b6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7cdd4640 1 -- 172.21.15.47:0/121287306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba700094d0 con 0x7fba780b8300 2024-03-23T09:43:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7d5d5640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba78160cb0 0x7fba780b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba77fff640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba7815cb40 0x7fba780b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba7cdd4640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba780b8300 0x7fba7814cf40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fba68007ef0 tx=0x7fba6800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.606+0000 7fba757fa640 1 -- 172.21.15.47:0/121287306 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68014620 con 0x7fba780b8300 2024-03-23T09:43:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.607+0000 7fba757fa640 1 -- 172.21.15.47:0/121287306 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba68014780 con 0x7fba780b8300 2024-03-23T09:43:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.607+0000 7fba7f038640 1 -- 172.21.15.47:0/121287306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7815d330 con 0x7fba780b8300 2024-03-23T09:43:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.607+0000 7fba757fa640 1 -- 172.21.15.47:0/121287306 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68014b30 con 0x7fba780b8300 2024-03-23T09:43:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.607+0000 7fba7f038640 1 -- 172.21.15.47:0/121287306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7814d480 con 0x7fba780b8300 2024-03-23T09:43:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.608+0000 7fba7f038640 1 -- 172.21.15.47:0/121287306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7815dc60 con 0x7fba780b8300 2024-03-23T09:43:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.608+0000 7fba757fa640 1 -- 172.21.15.47:0/121287306 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fba68014c90 con 0x7fba780b8300 2024-03-23T09:43:51.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.608+0000 7fba757fa640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba6406ac30 0x7fba6406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:51.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.608+0000 7fba757fa640 1 -- 172.21.15.47:0/121287306 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba68089c50 con 0x7fba780b8300 2024-03-23T09:43:51.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.609+0000 7fba77fff640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba6406ac30 0x7fba6406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:51.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.609+0000 7fba77fff640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba6406ac30 0x7fba6406d0f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fba78152cf0 tx=0x7fba6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.613+0000 7fba757fa640 1 -- 172.21.15.47:0/121287306 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fba6808f020 con 0x7fba780b8300 2024-03-23T09:43:51.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.791+0000 7fba7f038640 1 -- 172.21.15.47:0/121287306 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fba780d0b70 con 0x7fba6406ac30 2024-03-23T09:43:51.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.799+0000 7fba757fa640 1 -- 172.21.15.47:0/121287306 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286507 (secure 0 0 0) 0x7fba6c009db0 con 0x7fba6406ac30 2024-03-23T09:43:51.801 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:43:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.802+0000 7fba7f038640 1 -- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba6406ac30 msgr2=0x7fba6406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.802+0000 7fba7f038640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba6406ac30 0x7fba6406d0f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fba78152cf0 tx=0x7fba6c008040 comp rx=0 tx=0).stop 2024-03-23T09:43:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.802+0000 7fba7f038640 1 -- 172.21.15.47:0/121287306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba780b8300 msgr2=0x7fba7814cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.802+0000 7fba7f038640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba780b8300 0x7fba7814cf40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fba68007ef0 tx=0x7fba6800e4b0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.802+0000 7fba7f038640 1 -- 172.21.15.47:0/121287306 shutdown_connections 2024-03-23T09:43:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.802+0000 7fba7f038640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba6406ac30 0x7fba6406d0f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.802+0000 7fba7f038640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba78160cb0 0x7fba780b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.802+0000 7fba7f038640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba7815cb40 0x7fba780b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.802+0000 7fba7f038640 1 --2- 172.21.15.47:0/121287306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba780b8300 0x7fba7814cf40 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.802+0000 7fba7f038640 1 -- 172.21.15.47:0/121287306 >> 172.21.15.47:0/121287306 conn(0x7fba7806b160 msgr2=0x7fba781604f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.802+0000 7fba7f038640 1 -- 172.21.15.47:0/121287306 shutdown_connections 2024-03-23T09:43:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:51.802+0000 7fba7f038640 1 -- 172.21.15.47:0/121287306 wait complete. 2024-03-23T09:43:51.812 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-03-23T09:43:50.378328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238916,"kb_used_data":6352,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754735804,"statfs":{"total":773094113280,"available":772849463296,"internally_reserved":0,"allocated":6504448,"data_stored":3349262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.367743"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.028360+0000","last_change":"2024-03-23T09:43:47.028360+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.028360+0000","last_undegraded":"2024-03-23T09:43:47.028360+0000","last_fullsized":"2024-03-23T09:43:47.028360+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:54:41.177596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021545+0000","last_change":"2024-03-23T09:43:47.021545+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021545+0000","last_undegraded":"2024-03-23T09:43:47.021545+0000","last_fullsized":"2024-03-23T09:43:47.021545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:36.262351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.027083+0000","last_change":"2024-03-23T09:43:47.027083+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.027083+0000","last_undegraded":"2024-03-23T09:43:47.027083+0000","last_fullsized":"2024-03-23T09:43:47.027083+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:22:26.903375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032839+0000","last_change":"2024-03-23T09:43:47.032839+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032839+0000","last_undegraded":"2024-03-23T09:43:47.032839+0000","last_fullsized":"2024-03-23T09:43:47.032839+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026471+0000","last_change":"2024-03-23T09:43:47.026471+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026471+0000","last_undegraded":"2024-03-23T09:43:47.026471+0000","last_fullsized":"2024-03-23T09:43:47.026471+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:17.338788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022332+0000","last_change":"2024-03-23T09:43:47.022332+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022332+0000","last_undegraded":"2024-03-23T09:43:47.022332+0000","last_fullsized":"2024-03-23T09:43:47.022332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023985+0000","last_change":"2024-03-23T09:43:47.023985+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023985+0000","last_undegraded":"2024-03-23T09:43:47.023985+0000","last_fullsized":"2024-03-23T09:43:47.023985+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:44:18.230315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021778+0000","last_change":"2024-03-23T09:43:47.021778+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021778+0000","last_undegraded":"2024-03-23T09:43:47.021778+0000","last_fullsized":"2024-03-23T09:43:47.021778+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:55:17.926828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023723+0000","last_change":"2024-03-23T09:43:47.023723+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023723+0000","last_undegraded":"2024-03-23T09:43:47.023723+0000","last_fullsized":"2024-03-23T09:43:47.023723+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:19.820479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.020703+0000","last_change":"2024-03-23T09:43:47.020703+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.020703+0000","last_undegraded":"2024-03-23T09:43:47.020703+0000","last_fullsized":"2024-03-23T09:43:47.020703+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:35:30.364827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.020633+0000","last_change":"2024-03-23T09:43:47.020633+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.020633+0000","last_undegraded":"2024-03-23T09:43:47.020633+0000","last_fullsized":"2024-03-23T09:43:47.020633+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:48:50.443256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032222+0000","last_change":"2024-03-23T09:43:47.032222+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032222+0000","last_undegraded":"2024-03-23T09:43:47.032222+0000","last_fullsized":"2024-03-23T09:43:47.032222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026161+0000","last_change":"2024-03-23T09:43:47.026161+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026161+0000","last_undegraded":"2024-03-23T09:43:47.026161+0000","last_fullsized":"2024-03-23T09:43:47.026161+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023765+0000","last_change":"2024-03-23T09:43:47.023765+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023765+0000","last_undegraded":"2024-03-23T09:43:47.023765+0000","last_fullsized":"2024-03-23T09:43:47.023765+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:05:58.075182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023448+0000","last_change":"2024-03-23T09:43:47.023448+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023448+0000","last_undegraded":"2024-03-23T09:43:47.023448+0000","last_fullsized":"2024-03-23T09:43:47.023448+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:56.576713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021932+0000","last_change":"2024-03-23T09:43:47.021932+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021932+0000","last_undegraded":"2024-03-23T09:43:47.021932+0000","last_fullsized":"2024-03-23T09:43:47.021932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:21.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023522+0000","last_change":"2024-03-23T09:43:47.023522+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023522+0000","last_undegraded":"2024-03-23T09:43:47.023522+0000","last_fullsized":"2024-03-23T09:43:47.023522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:50:34.378460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025211+0000","last_change":"2024-03-23T09:43:47.025211+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025211+0000","last_undegraded":"2024-03-23T09:43:47.025211+0000","last_fullsized":"2024-03-23T09:43:47.025211+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:49.207297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026927+0000","last_change":"2024-03-23T09:43:47.026927+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026927+0000","last_undegraded":"2024-03-23T09:43:47.026927+0000","last_fullsized":"2024-03-23T09:43:47.026927+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:59.729962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024967+0000","last_change":"2024-03-23T09:43:47.024967+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024967+0000","last_undegraded":"2024-03-23T09:43:47.024967+0000","last_fullsized":"2024-03-23T09:43:47.024967+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:07:44.532537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025472+0000","last_change":"2024-03-23T09:43:47.025472+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025472+0000","last_undegraded":"2024-03-23T09:43:47.025472+0000","last_fullsized":"2024-03-23T09:43:47.025472+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:19:12.228744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026145+0000","last_change":"2024-03-23T09:43:47.026145+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026145+0000","last_undegraded":"2024-03-23T09:43:47.026145+0000","last_fullsized":"2024-03-23T09:43:47.026145+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:47:24.835835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024831+0000","last_change":"2024-03-23T09:43:47.024831+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024831+0000","last_undegraded":"2024-03-23T09:43:47.024831+0000","last_fullsized":"2024-03-23T09:43:47.024831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:10:55.583385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024086+0000","last_change":"2024-03-23T09:43:47.024086+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024086+0000","last_undegraded":"2024-03-23T09:43:47.024086+0000","last_fullsized":"2024-03-23T09:43:47.024086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:54:41.177596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.027386+0000","last_change":"2024-03-23T09:43:47.027386+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.027386+0000","last_undegraded":"2024-03-23T09:43:47.027386+0000","last_fullsized":"2024-03-23T09:43:47.027386+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:08.737490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032083+0000","last_change":"2024-03-23T09:43:47.032083+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032083+0000","last_undegraded":"2024-03-23T09:43:47.032083+0000","last_fullsized":"2024-03-23T09:43:47.032083+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:45:37.390159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024006+0000","last_change":"2024-03-23T09:43:47.024006+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024006+0000","last_undegraded":"2024-03-23T09:43:47.024006+0000","last_fullsized":"2024-03-23T09:43:47.024006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:11.106544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026420+0000","last_change":"2024-03-23T09:43:47.026420+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026420+0000","last_undegraded":"2024-03-23T09:43:47.026420+0000","last_fullsized":"2024-03-23T09:43:47.026420+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024600+0000","last_change":"2024-03-23T09:43:47.024600+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024600+0000","last_undegraded":"2024-03-23T09:43:47.024600+0000","last_fullsized":"2024-03-23T09:43:47.024600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:04.150928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023232+0000","last_change":"2024-03-23T09:43:47.023232+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023232+0000","last_undegraded":"2024-03-23T09:43:47.023232+0000","last_fullsized":"2024-03-23T09:43:47.023232+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026157+0000","last_change":"2024-03-23T09:43:47.026157+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026157+0000","last_undegraded":"2024-03-23T09:43:47.026157+0000","last_fullsized":"2024-03-23T09:43:47.026157+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:46:16.837675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021430+0000","last_change":"2024-03-23T09:43:47.021430+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021430+0000","last_undegraded":"2024-03-23T09:43:47.021430+0000","last_fullsized":"2024-03-23T09:43:47.021430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:02:58.178256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023483+0000","last_change":"2024-03-23T09:43:47.023483+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023483+0000","last_undegraded":"2024-03-23T09:43:47.023483+0000","last_fullsized":"2024-03-23T09:43:47.023483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.038474+0000","last_change":"2024-03-23T09:43:47.036626+0000","last_active":"2024-03-23T09:43:47.038474+0000","last_peered":"2024-03-23T09:43:47.038474+0000","last_clean":"2024-03-23T09:43:47.038474+0000","last_became_active":"2024-03-23T09:43:47.036383+0000","last_became_peered":"2024-03-23T09:43:47.036383+0000","last_unstale":"2024-03-23T09:43:47.038474+0000","last_undegraded":"2024-03-23T09:43:47.038474+0000","last_fullsized":"2024-03-23T09:43:47.038474+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:18.815610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.020741+0000","last_change":"2024-03-23T09:43:47.020741+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.020741+0000","last_undegraded":"2024-03-23T09:43:47.020741+0000","last_fullsized":"2024-03-23T09:43:47.020741+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:21.051130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.038950+0000","last_change":"2024-03-23T09:43:47.037068+0000","last_active":"2024-03-23T09:43:47.038950+0000","last_peered":"2024-03-23T09:43:47.038950+0000","last_clean":"2024-03-23T09:43:47.038950+0000","last_became_active":"2024-03-23T09:43:47.036452+0000","last_became_peered":"2024-03-23T09:43:47.036452+0000","last_unstale":"2024-03-23T09:43:47.038950+0000","last_undegraded":"2024-03-23T09:43:47.038950+0000","last_fullsized":"2024-03-23T09:43:47.038950+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:26.308593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021823+0000","last_change":"2024-03-23T09:43:47.021823+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021823+0000","last_undegraded":"2024-03-23T09:43:47.021823+0000","last_fullsized":"2024-03-23T09:43:47.021823+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.033586+0000","last_change":"2024-03-23T09:43:47.029691+0000","last_active":"2024-03-23T09:43:47.033586+0000","last_peered":"2024-03-23T09:43:47.033586+0000","last_clean":"2024-03-23T09:43:47.033586+0000","last_became_active":"2024-03-23T09:43:47.029427+0000","last_became_peered":"2024-03-23T09:43:47.029427+0000","last_unstale":"2024-03-23T09:43:47.033586+0000","last_undegraded":"2024-03-23T09:43:47.033586+0000","last_fullsized":"2024-03-23T09:43:47.033586+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023508+0000","last_change":"2024-03-23T09:43:47.023508+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023508+0000","last_undegraded":"2024-03-23T09:43:47.023508+0000","last_fullsized":"2024-03-23T09:43:47.023508+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:45.147489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040183+0000","last_change":"2024-03-23T09:43:47.039140+0000","last_active":"2024-03-23T09:43:47.040183+0000","last_peered":"2024-03-23T09:43:47.040183+0000","last_clean":"2024-03-23T09:43:47.040183+0000","last_became_active":"2024-03-23T09:43:47.038851+0000","last_became_peered":"2024-03-23T09:43:47.038851+0000","last_unstale":"2024-03-23T09:43:47.040183+0000","last_undegraded":"2024-03-23T09:43:47.040183+0000","last_fullsized":"2024-03-23T09:43:47.040183+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:18.815610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024459+0000","last_change":"2024-03-23T09:43:47.024459+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024459+0000","last_undegraded":"2024-03-23T09:43:47.024459+0000","last_fullsized":"2024-03-23T09:43:47.024459+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:28:32.181328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037202+0000","last_change":"2024-03-23T09:43:47.031421+0000","last_active":"2024-03-23T09:43:47.037202+0000","last_peered":"2024-03-23T09:43:47.037202+0000","last_clean":"2024-03-23T09:43:47.037202+0000","last_became_active":"2024-03-23T09:43:47.031157+0000","last_became_peered":"2024-03-23T09:43:47.031157+0000","last_unstale":"2024-03-23T09:43:47.037202+0000","last_undegraded":"2024-03-23T09:43:47.037202+0000","last_fullsized":"2024-03-23T09:43:47.037202+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:49.393118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025198+0000","last_change":"2024-03-23T09:43:47.025198+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025198+0000","last_undegraded":"2024-03-23T09:43:47.025198+0000","last_fullsized":"2024-03-23T09:43:47.025198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:42:29.904309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.035316+0000","last_change":"2024-03-23T09:43:47.034482+0000","last_active":"2024-03-23T09:43:47.035316+0000","last_peered":"2024-03-23T09:43:47.035316+0000","last_clean":"2024-03-23T09:43:47.035316+0000","last_became_active":"2024-03-23T09:43:47.034190+0000","last_became_peered":"2024-03-23T09:43:47.034190+0000","last_unstale":"2024-03-23T09:43:47.035316+0000","last_undegraded":"2024-03-23T09:43:47.035316+0000","last_fullsized":"2024-03-23T09:43:47.035316+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:27:00.248727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023485+0000","last_change":"2024-03-23T09:43:47.023485+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023485+0000","last_undegraded":"2024-03-23T09:43:47.023485+0000","last_fullsized":"2024-03-23T09:43:47.023485+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:59.729962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036543+0000","last_change":"2024-03-23T09:43:47.034996+0000","last_active":"2024-03-23T09:43:47.036543+0000","last_peered":"2024-03-23T09:43:47.036543+0000","last_clean":"2024-03-23T09:43:47.036543+0000","last_became_active":"2024-03-23T09:43:47.034715+0000","last_became_peered":"2024-03-23T09:43:47.034715+0000","last_unstale":"2024-03-23T09:43:47.036543+0000","last_undegraded":"2024-03-23T09:43:47.036543+0000","last_fullsized":"2024-03-23T09:43:47.036543+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022665+0000","last_change":"2024-03-23T09:43:47.022665+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022665+0000","last_undegraded":"2024-03-23T09:43:47.022665+0000","last_fullsized":"2024-03-23T09:43:47.022665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:25:06.163966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039588+0000","last_change":"2024-03-23T09:43:47.036762+0000","last_active":"2024-03-23T09:43:47.039588+0000","last_peered":"2024-03-23T09:43:47.039588+0000","last_clean":"2024-03-23T09:43:47.039588+0000","last_became_active":"2024-03-23T09:43:47.036510+0000","last_became_peered":"2024-03-23T09:43:47.036510+0000","last_unstale":"2024-03-23T09:43:47.039588+0000","last_undegraded":"2024-03-23T09:43:47.039588+0000","last_fullsized":"2024-03-23T09:43:47.039588+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:58.725092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023652+0000","last_change":"2024-03-23T09:43:47.023652+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023652+0000","last_undegraded":"2024-03-23T09:43:47.023652+0000","last_fullsized":"2024-03-23T09:43:47.023652+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:36:27.003657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037419+0000","last_change":"2024-03-23T09:43:47.034507+0000","last_active":"2024-03-23T09:43:47.037419+0000","last_peered":"2024-03-23T09:43:47.037419+0000","last_clean":"2024-03-23T09:43:47.037419+0000","last_became_active":"2024-03-23T09:43:47.034260+0000","last_became_peered":"2024-03-23T09:43:47.034260+0000","last_unstale":"2024-03-23T09:43:47.037419+0000","last_undegraded":"2024-03-23T09:43:47.037419+0000","last_fullsized":"2024-03-23T09:43:47.037419+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.025795+0000","last_change":"2024-03-23T09:43:47.025795+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.025795+0000","last_undegraded":"2024-03-23T09:43:47.025795+0000","last_fullsized":"2024-03-23T09:43:47.025795+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037161+0000","last_change":"2024-03-23T09:43:47.036141+0000","last_active":"2024-03-23T09:43:47.037161+0000","last_peered":"2024-03-23T09:43:47.037161+0000","last_clean":"2024-03-23T09:43:47.037161+0000","last_became_active":"2024-03-23T09:43:47.035805+0000","last_became_peered":"2024-03-23T09:43:47.035805+0000","last_unstale":"2024-03-23T09:43:47.037161+0000","last_undegraded":"2024-03-23T09:43:47.037161+0000","last_fullsized":"2024-03-23T09:43:47.037161+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:22.465435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023205+0000","last_change":"2024-03-23T09:43:47.023205+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023205+0000","last_undegraded":"2024-03-23T09:43:47.023205+0000","last_fullsized":"2024-03-23T09:43:47.023205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:28:37.758076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037387+0000","last_change":"2024-03-23T09:43:47.032774+0000","last_active":"2024-03-23T09:43:47.037387+0000","last_peered":"2024-03-23T09:43:47.037387+0000","last_clean":"2024-03-23T09:43:47.037387+0000","last_became_active":"2024-03-23T09:43:47.032528+0000","last_became_peered":"2024-03-23T09:43:47.032528+0000","last_unstale":"2024-03-23T09:43:47.037387+0000","last_undegraded":"2024-03-23T09:43:47.037387+0000","last_fullsized":"2024-03-23T09:43:47.037387+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026681+0000","last_change":"2024-03-23T09:43:47.026681+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026681+0000","last_undegraded":"2024-03-23T09:43:47.026681+0000","last_fullsized":"2024-03-23T09:43:47.026681+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:11.106544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037521+0000","last_change":"2024-03-23T09:43:47.033945+0000","last_active":"2024-03-23T09:43:47.037521+0000","last_peered":"2024-03-23T09:43:47.037521+0000","last_clean":"2024-03-23T09:43:47.037521+0000","last_became_active":"2024-03-23T09:43:47.033037+0000","last_became_peered":"2024-03-23T09:43:47.033037+0000","last_unstale":"2024-03-23T09:43:47.037521+0000","last_undegraded":"2024-03-23T09:43:47.037521+0000","last_fullsized":"2024-03-23T09:43:47.037521+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:28:36.753206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.029812+0000","last_change":"2024-03-23T09:43:47.029812+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.029812+0000","last_undegraded":"2024-03-23T09:43:47.029812+0000","last_fullsized":"2024-03-23T09:43:47.029812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:51:10.516410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036612+0000","last_change":"2024-03-23T09:43:47.034748+0000","last_active":"2024-03-23T09:43:47.036612+0000","last_peered":"2024-03-23T09:43:47.036612+0000","last_clean":"2024-03-23T09:43:47.036612+0000","last_became_active":"2024-03-23T09:43:47.034340+0000","last_became_peered":"2024-03-23T09:43:47.034340+0000","last_unstale":"2024-03-23T09:43:47.036612+0000","last_undegraded":"2024-03-23T09:43:47.036612+0000","last_fullsized":"2024-03-23T09:43:47.036612+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:06.713892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024286+0000","last_change":"2024-03-23T09:43:47.024286+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024286+0000","last_undegraded":"2024-03-23T09:43:47.024286+0000","last_fullsized":"2024-03-23T09:43:47.024286+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039159+0000","last_change":"2024-03-23T09:43:47.034963+0000","last_active":"2024-03-23T09:43:47.039159+0000","last_peered":"2024-03-23T09:43:47.039159+0000","last_clean":"2024-03-23T09:43:47.039159+0000","last_became_active":"2024-03-23T09:43:47.034680+0000","last_became_peered":"2024-03-23T09:43:47.034680+0000","last_unstale":"2024-03-23T09:43:47.039159+0000","last_undegraded":"2024-03-23T09:43:47.039159+0000","last_fullsized":"2024-03-23T09:43:47.039159+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:15:54.128943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023273+0000","last_change":"2024-03-23T09:43:47.023273+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023273+0000","last_undegraded":"2024-03-23T09:43:47.023273+0000","last_fullsized":"2024-03-23T09:43:47.023273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:27:01.253597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040165+0000","last_change":"2024-03-23T09:43:47.038681+0000","last_active":"2024-03-23T09:43:47.040165+0000","last_peered":"2024-03-23T09:43:47.040165+0000","last_clean":"2024-03-23T09:43:47.040165+0000","last_became_active":"2024-03-23T09:43:47.038410+0000","last_became_peered":"2024-03-23T09:43:47.038410+0000","last_unstale":"2024-03-23T09:43:47.040165+0000","last_undegraded":"2024-03-23T09:43:47.040165+0000","last_fullsized":"2024-03-23T09:43:47.040165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:19.400345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024358+0000","last_change":"2024-03-23T09:43:47.024358+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024358+0000","last_undegraded":"2024-03-23T09:43:47.024358+0000","last_fullsized":"2024-03-23T09:43:47.024358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:02:58.178256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.042320+0000","last_change":"2024-03-23T09:43:47.036966+0000","last_active":"2024-03-23T09:43:47.042320+0000","last_peered":"2024-03-23T09:43:47.042320+0000","last_clean":"2024-03-23T09:43:47.042320+0000","last_became_active":"2024-03-23T09:43:47.036691+0000","last_became_peered":"2024-03-23T09:43:47.036691+0000","last_unstale":"2024-03-23T09:43:47.042320+0000","last_undegraded":"2024-03-23T09:43:47.042320+0000","last_fullsized":"2024-03-23T09:43:47.042320+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.024769+0000","last_change":"2024-03-23T09:43:47.024769+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.024769+0000","last_undegraded":"2024-03-23T09:43:47.024769+0000","last_fullsized":"2024-03-23T09:43:47.024769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037227+0000","last_change":"2024-03-23T09:43:47.036026+0000","last_active":"2024-03-23T09:43:47.037227+0000","last_peered":"2024-03-23T09:43:47.037227+0000","last_clean":"2024-03-23T09:43:47.037227+0000","last_became_active":"2024-03-23T09:43:47.035690+0000","last_became_peered":"2024-03-23T09:43:47.035690+0000","last_unstale":"2024-03-23T09:43:47.037227+0000","last_undegraded":"2024-03-23T09:43:47.037227+0000","last_fullsized":"2024-03-23T09:43:47.037227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:20.475983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.033106+0000","last_change":"2024-03-23T09:43:47.033106+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.033106+0000","last_undegraded":"2024-03-23T09:43:47.033106+0000","last_fullsized":"2024-03-23T09:43:47.033106+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034465+0000","last_change":"2024-03-23T09:43:47.032740+0000","last_active":"2024-03-23T09:43:47.034465+0000","last_peered":"2024-03-23T09:43:47.034465+0000","last_clean":"2024-03-23T09:43:47.034465+0000","last_became_active":"2024-03-23T09:43:47.032456+0000","last_became_peered":"2024-03-23T09:43:47.032456+0000","last_unstale":"2024-03-23T09:43:47.034465+0000","last_undegraded":"2024-03-23T09:43:47.034465+0000","last_fullsized":"2024-03-23T09:43:47.034465+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021027+0000","last_change":"2024-03-23T09:43:47.021027+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021027+0000","last_undegraded":"2024-03-23T09:43:47.021027+0000","last_fullsized":"2024-03-23T09:43:47.021027+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:07:10.261476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040144+0000","last_change":"2024-03-23T09:43:47.038642+0000","last_active":"2024-03-23T09:43:47.040144+0000","last_peered":"2024-03-23T09:43:47.040144+0000","last_clean":"2024-03-23T09:43:47.040144+0000","last_became_active":"2024-03-23T09:43:47.038385+0000","last_became_peered":"2024-03-23T09:43:47.038385+0000","last_unstale":"2024-03-23T09:43:47.040144+0000","last_undegraded":"2024-03-23T09:43:47.040144+0000","last_fullsized":"2024-03-23T09:43:47.040144+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:33.981220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.026250+0000","last_change":"2024-03-23T09:43:47.026250+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.026250+0000","last_undegraded":"2024-03-23T09:43:47.026250+0000","last_fullsized":"2024-03-23T09:43:47.026250+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:21.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034573+0000","last_change":"2024-03-23T09:43:47.032632+0000","last_active":"2024-03-23T09:43:47.034573+0000","last_peered":"2024-03-23T09:43:47.034573+0000","last_clean":"2024-03-23T09:43:47.034573+0000","last_became_active":"2024-03-23T09:43:47.032375+0000","last_became_peered":"2024-03-23T09:43:47.032375+0000","last_unstale":"2024-03-23T09:43:47.034573+0000","last_undegraded":"2024-03-23T09:43:47.034573+0000","last_fullsized":"2024-03-23T09:43:47.034573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.029300+0000","last_change":"2024-03-23T09:43:47.029300+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.029300+0000","last_undegraded":"2024-03-23T09:43:47.029300+0000","last_fullsized":"2024-03-23T09:43:47.029300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:14:35.775608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036589+0000","last_change":"2024-03-23T09:43:47.030434+0000","last_active":"2024-03-23T09:43:47.036589+0000","last_peered":"2024-03-23T09:43:47.036589+0000","last_clean":"2024-03-23T09:43:47.036589+0000","last_became_active":"2024-03-23T09:43:47.030164+0000","last_became_peered":"2024-03-23T09:43:47.030164+0000","last_unstale":"2024-03-23T09:43:47.036589+0000","last_undegraded":"2024-03-23T09:43:47.036589+0000","last_fullsized":"2024-03-23T09:43:47.036589+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:09.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022414+0000","last_change":"2024-03-23T09:43:47.022414+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022414+0000","last_undegraded":"2024-03-23T09:43:47.022414+0000","last_fullsized":"2024-03-23T09:43:47.022414+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:22:26.903375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034938+0000","last_change":"2024-03-23T09:43:47.031235+0000","last_active":"2024-03-23T09:43:47.034938+0000","last_peered":"2024-03-23T09:43:47.034938+0000","last_clean":"2024-03-23T09:43:47.034938+0000","last_became_active":"2024-03-23T09:43:47.030949+0000","last_became_peered":"2024-03-23T09:43:47.030949+0000","last_unstale":"2024-03-23T09:43:47.034938+0000","last_undegraded":"2024-03-23T09:43:47.034938+0000","last_fullsized":"2024-03-23T09:43:47.034938+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:58:56.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022818+0000","last_change":"2024-03-23T09:43:47.022818+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022818+0000","last_undegraded":"2024-03-23T09:43:47.022818+0000","last_fullsized":"2024-03-23T09:43:47.022818+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040173+0000","last_change":"2024-03-23T09:43:47.038838+0000","last_active":"2024-03-23T09:43:47.040173+0000","last_peered":"2024-03-23T09:43:47.040173+0000","last_clean":"2024-03-23T09:43:47.040173+0000","last_became_active":"2024-03-23T09:43:47.038574+0000","last_became_peered":"2024-03-23T09:43:47.038574+0000","last_unstale":"2024-03-23T09:43:47.040173+0000","last_undegraded":"2024-03-23T09:43:47.040173+0000","last_fullsized":"2024-03-23T09:43:47.040173+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.030328+0000","last_change":"2024-03-23T09:43:47.030328+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.030328+0000","last_undegraded":"2024-03-23T09:43:47.030328+0000","last_fullsized":"2024-03-23T09:43:47.030328+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:49.207297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.031785+0000","last_change":"2024-03-23T09:43:47.029629+0000","last_active":"2024-03-23T09:43:47.031785+0000","last_peered":"2024-03-23T09:43:47.031785+0000","last_clean":"2024-03-23T09:43:47.031785+0000","last_became_active":"2024-03-23T09:43:47.029338+0000","last_became_peered":"2024-03-23T09:43:47.029338+0000","last_unstale":"2024-03-23T09:43:47.031785+0000","last_undegraded":"2024-03-23T09:43:47.031785+0000","last_fullsized":"2024-03-23T09:43:47.031785+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:15:54.128943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.032692+0000","last_change":"2024-03-23T09:43:47.032692+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.032692+0000","last_undegraded":"2024-03-23T09:43:47.032692+0000","last_fullsized":"2024-03-23T09:43:47.032692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.036684+0000","last_change":"2024-03-23T09:43:47.035203+0000","last_active":"2024-03-23T09:43:47.036684+0000","last_peered":"2024-03-23T09:43:47.036684+0000","last_clean":"2024-03-23T09:43:47.036684+0000","last_became_active":"2024-03-23T09:43:47.034912+0000","last_became_peered":"2024-03-23T09:43:47.034912+0000","last_unstale":"2024-03-23T09:43:47.036684+0000","last_undegraded":"2024-03-23T09:43:47.036684+0000","last_fullsized":"2024-03-23T09:43:47.036684+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:36:25.998787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.019914+0000","last_change":"2024-03-23T09:43:47.019914+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.019914+0000","last_undegraded":"2024-03-23T09:43:47.019914+0000","last_fullsized":"2024-03-23T09:43:47.019914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.034026+0000","last_change":"2024-03-23T09:43:47.029065+0000","last_active":"2024-03-23T09:43:47.034026+0000","last_peered":"2024-03-23T09:43:47.034026+0000","last_clean":"2024-03-23T09:43:47.034026+0000","last_became_active":"2024-03-23T09:43:47.028797+0000","last_became_peered":"2024-03-23T09:43:47.028797+0000","last_unstale":"2024-03-23T09:43:47.034026+0000","last_undegraded":"2024-03-23T09:43:47.034026+0000","last_fullsized":"2024-03-23T09:43:47.034026+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:32:33.947908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022836+0000","last_change":"2024-03-23T09:43:47.022836+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022836+0000","last_undegraded":"2024-03-23T09:43:47.022836+0000","last_fullsized":"2024-03-23T09:43:47.022836+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:19:12.228744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.040108+0000","last_change":"2024-03-23T09:43:47.038126+0000","last_active":"2024-03-23T09:43:47.040108+0000","last_peered":"2024-03-23T09:43:47.040108+0000","last_clean":"2024-03-23T09:43:47.040108+0000","last_became_active":"2024-03-23T09:43:47.037850+0000","last_became_peered":"2024-03-23T09:43:47.037850+0000","last_unstale":"2024-03-23T09:43:47.040108+0000","last_undegraded":"2024-03-23T09:43:47.040108+0000","last_fullsized":"2024-03-23T09:43:47.040108+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023991+0000","last_change":"2024-03-23T09:43:47.023991+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023991+0000","last_undegraded":"2024-03-23T09:43:47.023991+0000","last_fullsized":"2024-03-23T09:43:47.023991+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:59.768596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037679+0000","last_change":"2024-03-23T09:43:47.036954+0000","last_active":"2024-03-23T09:43:47.037679+0000","last_peered":"2024-03-23T09:43:47.037679+0000","last_clean":"2024-03-23T09:43:47.037679+0000","last_became_active":"2024-03-23T09:43:47.036655+0000","last_became_peered":"2024-03-23T09:43:47.036655+0000","last_unstale":"2024-03-23T09:43:47.037679+0000","last_undegraded":"2024-03-23T09:43:47.037679+0000","last_fullsized":"2024-03-23T09:43:47.037679+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:05:51.129365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.023834+0000","last_change":"2024-03-23T09:43:47.023834+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.023834+0000","last_undegraded":"2024-03-23T09:43:47.023834+0000","last_fullsized":"2024-03-23T09:43:47.023834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:45.147489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.037249+0000","last_change":"2024-03-23T09:43:47.032771+0000","last_active":"2024-03-23T09:43:47.037249+0000","last_peered":"2024-03-23T09:43:47.037249+0000","last_clean":"2024-03-23T09:43:47.037249+0000","last_became_active":"2024-03-23T09:43:47.032489+0000","last_became_peered":"2024-03-23T09:43:47.032489+0000","last_unstale":"2024-03-23T09:43:47.037249+0000","last_undegraded":"2024-03-23T09:43:47.037249+0000","last_fullsized":"2024-03-23T09:43:47.037249+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:48:49.438386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.019914+0000","last_change":"2024-03-23T09:43:47.019914+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.019914+0000","last_undegraded":"2024-03-23T09:43:47.019914+0000","last_fullsized":"2024-03-23T09:43:47.019914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:24:18.285067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.033031+0000","last_change":"2024-03-23T09:43:47.030522+0000","last_active":"2024-03-23T09:43:47.033031+0000","last_peered":"2024-03-23T09:43:47.033031+0000","last_clean":"2024-03-23T09:43:47.033031+0000","last_became_active":"2024-03-23T09:43:47.030216+0000","last_became_peered":"2024-03-23T09:43:47.030216+0000","last_unstale":"2024-03-23T09:43:47.033031+0000","last_undegraded":"2024-03-23T09:43:47.033031+0000","last_fullsized":"2024-03-23T09:43:47.033031+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:32:33.947908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.022218+0000","last_change":"2024-03-23T09:43:47.022218+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.022218+0000","last_undegraded":"2024-03-23T09:43:47.022218+0000","last_fullsized":"2024-03-23T09:43:47.022218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:30:34.056749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039369+0000","last_change":"2024-03-23T09:43:47.038282+0000","last_active":"2024-03-23T09:43:47.039369+0000","last_peered":"2024-03-23T09:43:47.039369+0000","last_clean":"2024-03-23T09:43:47.039369+0000","last_became_active":"2024-03-23T09:43:47.037787+0000","last_became_peered":"2024-03-23T09:43:47.037787+0000","last_unstale":"2024-03-23T09:43:47.039369+0000","last_undegraded":"2024-03-23T09:43:47.039369+0000","last_fullsized":"2024-03-23T09:43:47.039369+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:10.101674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-03-23T09:43:47.021303+0000","last_change":"2024-03-23T09:43:47.021303+0000","last_active":"2024-03-23T09:43:47.010185+0000","last_peered":"2024-03-23T09:43:47.010185+0000","last_clean":"2024-03-23T09:43:47.010185+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:43:47.021303+0000","last_undegraded":"2024-03-23T09:43:47.021303+0000","last_fullsized":"2024-03-23T09:43:47.021303+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:52:24.758413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.039546+0000","last_change":"2024-03-23T09:43:47.037606+0000","last_active":"2024-03-23T09:43:47.039546+0000","last_peered":"2024-03-23T09:43:47.039546+0000","last_clean":"2024-03-23T09:43:47.039546+0000","last_became_active":"2024-03-23T09:43:47.037315+0000","last_became_peered":"2024-03-23T09:43:47.037315+0000","last_unstale":"2024-03-23T09:43:47.039546+0000","last_undegraded":"2024-03-23T09:43:47.039546+0000","last_fullsized":"2024-03-23T09:43:47.039546+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:22.192970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":200,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-23T09:43:46.013864+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:43:46.013864+0000","last_peered":"2024-03-23T09:43:46.013864+0000","last_clean":"2024-03-23T09:43:46.013864+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:43:46.013864+0000","last_undegraded":"2024-03-23T09:43:46.013864+0000","last_fullsized":"2024-03-23T09:43:46.013864+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1265664,"data_stored":864722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69299999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.62}]},{"osd":2,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.65900000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72699999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56999999999999995}]},{"osd":3,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.433,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58399999999999996}]},{"osd":4,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.377,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72099999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45300000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66500000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71499999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73799999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53700000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":267468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.499,"15min":0.498},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69299999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.54100000000000004}]},{"osd":1,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.40200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.42699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.627}]},{"osd":2,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.41299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.46100000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.496}]},{"osd":3,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52900000000000003,"15min":0.52300000000000002},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.73199999999999998,"5min":0.752,"15min":0.752},"last":0.60999999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.38700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.67500000000000004}]},{"osd":5,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.72399999999999998,"5min":0.747,"15min":0.747},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":268087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48099999999999998,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.42199999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52100000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52900000000000003,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.434,"5min":0.376,"15min":0.376},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.50700000000000001,"15min":0.49299999999999999},"min":{"1min":0.46700000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.46800000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.54100000000000004}]},{"osd":4,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47999999999999998,"15min":0.47699999999999998},"min":{"1min":0.33300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.439},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.374,"15min":0.374},"max":{"1min":0.69999999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63400000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.34799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.749,"15min":0.749},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.51800000000000002},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70099999999999996,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.60899999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.44600000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71499999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.621}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":268087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.69899999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.49199999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.46400000000000002},"min":{"1min":0.35699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.498,"15min":0.498},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.40600000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.42999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.752,"5min":0.755,"15min":0.755},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.747,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.50800000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70399999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.30199999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68799999999999994,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.308,"5min":0.253,"15min":0.253},"max":{"1min":0.68400000000000005,"5min":1.008,"15min":1.008},"last":0.36499999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.34000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70099999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.32400000000000001}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65800000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56200000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":671744,"data_stored":274354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.307,"15min":0.30499999999999999},"min":{"1min":0.25900000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30399999999999999,"15min":0.30199999999999999},"min":{"1min":0.25600000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.42899999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.437},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.55100000000000005}]},{"osd":2,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.313,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76500000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53100000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.73499999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.47399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81499999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.42899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.70999999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.81000000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.41799999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71499999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.67800000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.626},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.59999999999999998},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.71599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":671744,"data_stored":274354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.34000000000000002,"15min":0.34399999999999997},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.48799999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.25,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48799999999999999,"5min":0.872,"15min":0.872},"last":0.23000000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.629,"15min":0.629},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58599999999999997,"5min":0.746,"15min":0.746},"last":0.34100000000000003}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.47399999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.40200000000000002,"15min":0.40699999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55800000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.32300000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72499999999999998,"5min":1.216,"15min":1.216},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.753,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.624,"15min":0.63200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.71599999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75800000000000001,"5min":1.101,"15min":1.101},"last":0.66800000000000004}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.70699999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.63800000000000001,"15min":0.64800000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.72699999999999998,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.76500000000000001,"5min":1.155,"15min":1.155},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72099999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":659456,"data_stored":268087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33600000000000002,"15min":0.33300000000000002},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.53300000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61899999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.30299999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48399999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.374,"15min":0.37},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.55200000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41099999999999998,"15min":0.40699999999999997},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70499999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60899999999999999,"5min":2.75,"15min":2.75},"last":0.41499999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.45500000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91100000000000003,"5min":2.778,"15min":2.778},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68999999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.46300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88100000000000001,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.441},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.628}]},{"osd":7,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65400000000000003,"15min":0.64700000000000002},"min":{"1min":0.51800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":0.92400000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.76000000000000001,"15min":0.77800000000000002},"min":{"1min":0.44600000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.85099999999999998,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.52800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1261568,"data_stored":864103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32200000000000001,"15min":0.31900000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.499,"5min":0.504,"15min":0.504},"last":0.316},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.34499999999999997},"min":{"1min":0.25900000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.51300000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.30099999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36099999999999999,"15min":0.35899999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36099999999999999},"min":{"1min":0.27000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.33700000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.379,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.42299999999999999,"15min":0.41999999999999998},"min":{"1min":0.32100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.40999999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53600000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.81499999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54600000000000004}]},{"osd":6,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83299999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.88400000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55500000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:43:54.818 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:43:54.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.931+0000 7feeb8327640 1 Processor -- start 2024-03-23T09:43:54.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.931+0000 7feeb8327640 1 -- start start 2024-03-23T09:43:54.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.932+0000 7feeb8327640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feeb00d2f80 0x7feeb01678a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:54.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.932+0000 7feeb8327640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb00d10a0 con 0x7feeb015cee0 2024-03-23T09:43:54.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.932+0000 7feeb8327640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb00d1300 con 0x7feeb00d23a0 2024-03-23T09:43:54.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.932+0000 7feeb8327640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb00d15e0 con 0x7feeb00d2f80 2024-03-23T09:43:54.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.933+0000 7feeb68c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feeb00d2f80 0x7feeb01678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:54.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.933+0000 7feeb60c3640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feeb015cee0 0x7feeb00d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44728/0 (socket says 172.21.15.47:44728) 2024-03-23T09:43:54.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.933+0000 7feeb60c3640 1 -- 172.21.15.47:0/1435944866 learned_addr learned my addr 172.21.15.47:0/1435944866 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:54.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.933+0000 7feeb68c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feeb00d2f80 0x7feeb01678a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38006/0 (socket says 172.21.15.47:38006) 2024-03-23T09:43:54.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.933+0000 7feeb68c4640 1 -- 172.21.15.47:0/1435944866 >> v1:172.21.15.175:6789/0 conn(0x7feeb00d23a0 legacy=0x7feeb00d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:54.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.933+0000 7feeb68c4640 1 -- 172.21.15.47:0/1435944866 >> v1:172.21.15.47:6789/0 conn(0x7feeb015cee0 legacy=0x7feeb00d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:54.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.933+0000 7feeb68c4640 1 -- 172.21.15.47:0/1435944866 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb00d18c0 con 0x7feeb00d2f80 2024-03-23T09:43:54.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.933+0000 7feeb50c1640 1 -- 172.21.15.47:0/1435944866 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229780210 0 0) 0x7feeb00d10a0 con 0x7feeb015cee0 2024-03-23T09:43:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.934+0000 7feeb68c4640 1 --2- 172.21.15.47:0/1435944866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb00d2f80 0x7feeb01678a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7feea8009d90 tx=0x7feea802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de648637b9c3795 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.934+0000 7feeb50c1640 1 -- 172.21.15.47:0/1435944866 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea802cc60 con 0x7feeb00d2f80 2024-03-23T09:43:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.934+0000 7feeb50c1640 1 -- 172.21.15.47:0/1435944866 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea802cdc0 con 0x7feeb00d2f80 2024-03-23T09:43:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.934+0000 7feeb50c1640 1 -- 172.21.15.47:0/1435944866 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8034660 con 0x7feeb00d2f80 2024-03-23T09:43:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.934+0000 7feeb8327640 1 -- 172.21.15.47:0/1435944866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb00d2f80 msgr2=0x7feeb01678a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.934+0000 7feeb8327640 1 --2- 172.21.15.47:0/1435944866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb00d2f80 0x7feeb01678a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7feea8009d90 tx=0x7feea802beb0 comp rx=0 tx=0).stop 2024-03-23T09:43:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.935+0000 7feeb8327640 1 -- 172.21.15.47:0/1435944866 shutdown_connections 2024-03-23T09:43:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.935+0000 7feeb8327640 1 --2- 172.21.15.47:0/1435944866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb00d2f80 0x7feeb01678a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.935+0000 7feeb8327640 1 -- 172.21.15.47:0/1435944866 >> 172.21.15.47:0/1435944866 conn(0x7feeb0065df0 msgr2=0x7feeb0066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.935+0000 7feeb8327640 1 -- 172.21.15.47:0/1435944866 shutdown_connections 2024-03-23T09:43:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.935+0000 7feeb8327640 1 -- 172.21.15.47:0/1435944866 wait complete. 2024-03-23T09:43:54.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.936+0000 7feeb8327640 1 Processor -- start 2024-03-23T09:43:54.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.936+0000 7feeb8327640 1 -- start start 2024-03-23T09:43:54.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb8327640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb00d23a0 0x7feeb00b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:54.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb8327640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feeb00d2f80 0x7feeb0148870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:54.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb8327640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feeb015cee0 0x7feeb00c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:54.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb8327640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb00c7350 con 0x7feeb00d23a0 2024-03-23T09:43:54.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb8327640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb015fba0 con 0x7feeb00d2f80 2024-03-23T09:43:54.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb8327640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb0148db0 con 0x7feeb015cee0 2024-03-23T09:43:54.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb60c3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb00d23a0 0x7feeb00b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:54.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb60c3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb00d23a0 0x7feeb00b6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38018/0 (socket says 172.21.15.47:38018) 2024-03-23T09:43:54.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb60c3640 1 -- 172.21.15.47:0/3852916340 learned_addr learned my addr 172.21.15.47:0/3852916340 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:54.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb58c2640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feeb00d2f80 0x7feeb0148870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb68c4640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feeb015cee0 0x7feeb00c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb60c3640 1 -- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feeb015cee0 msgr2=0x7feeb00c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb60c3640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feeb015cee0 0x7feeb00c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb60c3640 1 -- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feeb00d2f80 msgr2=0x7feeb0148870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb60c3640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feeb00d2f80 0x7feeb0148870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb60c3640 1 -- 172.21.15.47:0/3852916340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea8009a70 con 0x7feeb00d23a0 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb58c2640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feeb00d2f80 0x7feeb0148870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb68c4640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feeb015cee0 0x7feeb00c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feeb60c3640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb00d23a0 0x7feeb00b6dc0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c4d0 tx=0x7feeac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.937+0000 7feea6ffd640 1 -- 172.21.15.47:0/3852916340 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac0074e0 con 0x7feeb00d23a0 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.938+0000 7feea6ffd640 1 -- 172.21.15.47:0/3852916340 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac007660 con 0x7feeb00d23a0 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.938+0000 7feeb8327640 1 -- 172.21.15.47:0/3852916340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb00c63f0 con 0x7feeb00d23a0 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.938+0000 7feea6ffd640 1 -- 172.21.15.47:0/3852916340 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac004a20 con 0x7feeb00d23a0 2024-03-23T09:43:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.938+0000 7feeb8327640 1 -- 172.21.15.47:0/3852916340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb00c69b0 con 0x7feeb00d23a0 2024-03-23T09:43:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.939+0000 7feeb8327640 1 -- 172.21.15.47:0/3852916340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb0072360 con 0x7feeb00d23a0 2024-03-23T09:43:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.939+0000 7feea6ffd640 1 -- 172.21.15.47:0/3852916340 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7feeac004b80 con 0x7feeb00d23a0 2024-03-23T09:43:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.939+0000 7feea6ffd640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee9406ac00 0x7fee9406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.939+0000 7feea6ffd640 1 -- 172.21.15.47:0/3852916340 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feeac088560 con 0x7feeb00d23a0 2024-03-23T09:43:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.940+0000 7feeb58c2640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee9406ac00 0x7fee9406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.940+0000 7feeb58c2640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee9406ac00 0x7fee9406d0c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feea0006fd0 tx=0x7feea0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:54.944+0000 7feea6ffd640 1 -- 172.21.15.47:0/3852916340 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7feeac053740 con 0x7feeb00d23a0 2024-03-23T09:43:55.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.123+0000 7feeb8327640 1 -- 172.21.15.47:0/3852916340 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feeb00d1e80 con 0x7fee9406ac00 2024-03-23T09:43:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.131+0000 7feea6ffd640 1 -- 172.21.15.47:0/3852916340 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289443 (secure 0 0 0) 0x7feea0009e20 con 0x7fee9406ac00 2024-03-23T09:43:55.133 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:55.135 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:43:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.134+0000 7feeb8327640 1 -- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee9406ac00 msgr2=0x7fee9406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.134+0000 7feeb8327640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee9406ac00 0x7fee9406d0c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feea0006fd0 tx=0x7feea0008040 comp rx=0 tx=0).stop 2024-03-23T09:43:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.134+0000 7feeb8327640 1 -- 172.21.15.47:0/3852916340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb00d23a0 msgr2=0x7feeb00b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.134+0000 7feeb8327640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb00d23a0 0x7feeb00b6dc0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c4d0 tx=0x7feeac00c990 comp rx=0 tx=0).stop 2024-03-23T09:43:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.134+0000 7feeb8327640 1 -- 172.21.15.47:0/3852916340 shutdown_connections 2024-03-23T09:43:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.134+0000 7feeb8327640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee9406ac00 0x7fee9406d0c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.134+0000 7feeb8327640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feeb015cee0 0x7feeb00c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.134+0000 7feeb8327640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feeb00d2f80 0x7feeb0148870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.134+0000 7feeb8327640 1 --2- 172.21.15.47:0/3852916340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb00d23a0 0x7feeb00b6dc0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.134+0000 7feeb8327640 1 -- 172.21.15.47:0/3852916340 >> 172.21.15.47:0/3852916340 conn(0x7feeb0065df0 msgr2=0x7feeb00cd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:55.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.134+0000 7feeb8327640 1 -- 172.21.15.47:0/3852916340 shutdown_connections 2024-03-23T09:43:55.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.134+0000 7feeb8327640 1 -- 172.21.15.47:0/3852916340 wait complete. 2024-03-23T09:43:55.144 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-03-23T09:43:54.379261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239240,"kb_used_data":6656,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754735480,"statfs":{"total":773094113280,"available":772849131520,"internally_reserved":0,"allocated":6815744,"data_stored":3483654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.360408"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.408143+0000","last_change":"2024-03-23T09:43:47.406739+0000","last_active":"2024-03-23T09:43:47.408143+0000","last_peered":"2024-03-23T09:43:47.408143+0000","last_clean":"2024-03-23T09:43:47.408143+0000","last_became_active":"2024-03-23T09:43:47.406444+0000","last_became_peered":"2024-03-23T09:43:47.406444+0000","last_unstale":"2024-03-23T09:43:47.408143+0000","last_undegraded":"2024-03-23T09:43:47.408143+0000","last_fullsized":"2024-03-23T09:43:47.408143+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:54:41.177596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.406914+0000","last_change":"2024-03-23T09:43:47.404134+0000","last_active":"2024-03-23T09:43:47.406914+0000","last_peered":"2024-03-23T09:43:47.406914+0000","last_clean":"2024-03-23T09:43:47.406914+0000","last_became_active":"2024-03-23T09:43:47.403885+0000","last_became_peered":"2024-03-23T09:43:47.403885+0000","last_unstale":"2024-03-23T09:43:47.406914+0000","last_undegraded":"2024-03-23T09:43:47.406914+0000","last_fullsized":"2024-03-23T09:43:47.406914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:36.262351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.408767+0000","last_change":"2024-03-23T09:43:47.402873+0000","last_active":"2024-03-23T09:43:47.408767+0000","last_peered":"2024-03-23T09:43:47.408767+0000","last_clean":"2024-03-23T09:43:47.408767+0000","last_became_active":"2024-03-23T09:43:47.402560+0000","last_became_peered":"2024-03-23T09:43:47.402560+0000","last_unstale":"2024-03-23T09:43:47.408767+0000","last_undegraded":"2024-03-23T09:43:47.408767+0000","last_fullsized":"2024-03-23T09:43:47.408767+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:22:26.903375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.413952+0000","last_change":"2024-03-23T09:43:47.410570+0000","last_active":"2024-03-23T09:43:47.413952+0000","last_peered":"2024-03-23T09:43:47.413952+0000","last_clean":"2024-03-23T09:43:47.413952+0000","last_became_active":"2024-03-23T09:43:47.408915+0000","last_became_peered":"2024-03-23T09:43:47.408915+0000","last_unstale":"2024-03-23T09:43:47.413952+0000","last_undegraded":"2024-03-23T09:43:47.413952+0000","last_fullsized":"2024-03-23T09:43:47.413952+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.411574+0000","last_change":"2024-03-23T09:43:47.410894+0000","last_active":"2024-03-23T09:43:47.411574+0000","last_peered":"2024-03-23T09:43:47.411574+0000","last_clean":"2024-03-23T09:43:47.411574+0000","last_became_active":"2024-03-23T09:43:47.410645+0000","last_became_peered":"2024-03-23T09:43:47.410645+0000","last_unstale":"2024-03-23T09:43:47.411574+0000","last_undegraded":"2024-03-23T09:43:47.411574+0000","last_fullsized":"2024-03-23T09:43:47.411574+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:17.338788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.404520+0000","last_change":"2024-03-23T09:43:47.403388+0000","last_active":"2024-03-23T09:43:47.404520+0000","last_peered":"2024-03-23T09:43:47.404520+0000","last_clean":"2024-03-23T09:43:47.404520+0000","last_became_active":"2024-03-23T09:43:47.403081+0000","last_became_peered":"2024-03-23T09:43:47.403081+0000","last_unstale":"2024-03-23T09:43:47.404520+0000","last_undegraded":"2024-03-23T09:43:47.404520+0000","last_fullsized":"2024-03-23T09:43:47.404520+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.415322+0000","last_change":"2024-03-23T09:43:47.412476+0000","last_active":"2024-03-23T09:43:47.415322+0000","last_peered":"2024-03-23T09:43:47.415322+0000","last_clean":"2024-03-23T09:43:47.415322+0000","last_became_active":"2024-03-23T09:43:47.412197+0000","last_became_peered":"2024-03-23T09:43:47.412197+0000","last_unstale":"2024-03-23T09:43:47.415322+0000","last_undegraded":"2024-03-23T09:43:47.415322+0000","last_fullsized":"2024-03-23T09:43:47.415322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:44:18.230315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.413176+0000","last_change":"2024-03-23T09:43:47.406766+0000","last_active":"2024-03-23T09:43:47.413176+0000","last_peered":"2024-03-23T09:43:47.413176+0000","last_clean":"2024-03-23T09:43:47.413176+0000","last_became_active":"2024-03-23T09:43:47.406526+0000","last_became_peered":"2024-03-23T09:43:47.406526+0000","last_unstale":"2024-03-23T09:43:47.413176+0000","last_undegraded":"2024-03-23T09:43:47.413176+0000","last_fullsized":"2024-03-23T09:43:47.413176+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:55:17.926828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.412761+0000","last_change":"2024-03-23T09:43:47.408028+0000","last_active":"2024-03-23T09:43:47.412761+0000","last_peered":"2024-03-23T09:43:47.412761+0000","last_clean":"2024-03-23T09:43:47.412761+0000","last_became_active":"2024-03-23T09:43:47.407736+0000","last_became_peered":"2024-03-23T09:43:47.407736+0000","last_unstale":"2024-03-23T09:43:47.412761+0000","last_undegraded":"2024-03-23T09:43:47.412761+0000","last_fullsized":"2024-03-23T09:43:47.412761+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:19.820479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.412710+0000","last_change":"2024-03-23T09:43:47.409839+0000","last_active":"2024-03-23T09:43:47.412710+0000","last_peered":"2024-03-23T09:43:47.412710+0000","last_clean":"2024-03-23T09:43:47.412710+0000","last_became_active":"2024-03-23T09:43:47.409594+0000","last_became_peered":"2024-03-23T09:43:47.409594+0000","last_unstale":"2024-03-23T09:43:47.412710+0000","last_undegraded":"2024-03-23T09:43:47.412710+0000","last_fullsized":"2024-03-23T09:43:47.412710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:35:30.364827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.409374+0000","last_change":"2024-03-23T09:43:47.403761+0000","last_active":"2024-03-23T09:43:47.409374+0000","last_peered":"2024-03-23T09:43:47.409374+0000","last_clean":"2024-03-23T09:43:47.409374+0000","last_became_active":"2024-03-23T09:43:47.403474+0000","last_became_peered":"2024-03-23T09:43:47.403474+0000","last_unstale":"2024-03-23T09:43:47.409374+0000","last_undegraded":"2024-03-23T09:43:47.409374+0000","last_fullsized":"2024-03-23T09:43:47.409374+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:48:50.443256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.413976+0000","last_change":"2024-03-23T09:43:47.411224+0000","last_active":"2024-03-23T09:43:47.413976+0000","last_peered":"2024-03-23T09:43:47.413976+0000","last_clean":"2024-03-23T09:43:47.413976+0000","last_became_active":"2024-03-23T09:43:47.408615+0000","last_became_peered":"2024-03-23T09:43:47.408615+0000","last_unstale":"2024-03-23T09:43:47.413976+0000","last_undegraded":"2024-03-23T09:43:47.413976+0000","last_fullsized":"2024-03-23T09:43:47.413976+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.417802+0000","last_change":"2024-03-23T09:43:47.412423+0000","last_active":"2024-03-23T09:43:47.417802+0000","last_peered":"2024-03-23T09:43:47.417802+0000","last_clean":"2024-03-23T09:43:47.417802+0000","last_became_active":"2024-03-23T09:43:47.412171+0000","last_became_peered":"2024-03-23T09:43:47.412171+0000","last_unstale":"2024-03-23T09:43:47.417802+0000","last_undegraded":"2024-03-23T09:43:47.417802+0000","last_fullsized":"2024-03-23T09:43:47.417802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.406614+0000","last_change":"2024-03-23T09:43:47.404966+0000","last_active":"2024-03-23T09:43:47.406614+0000","last_peered":"2024-03-23T09:43:47.406614+0000","last_clean":"2024-03-23T09:43:47.406614+0000","last_became_active":"2024-03-23T09:43:47.404687+0000","last_became_peered":"2024-03-23T09:43:47.404687+0000","last_unstale":"2024-03-23T09:43:47.406614+0000","last_undegraded":"2024-03-23T09:43:47.406614+0000","last_fullsized":"2024-03-23T09:43:47.406614+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:05:58.075182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.415135+0000","last_change":"2024-03-23T09:43:47.410709+0000","last_active":"2024-03-23T09:43:47.415135+0000","last_peered":"2024-03-23T09:43:47.415135+0000","last_clean":"2024-03-23T09:43:47.415135+0000","last_became_active":"2024-03-23T09:43:47.410418+0000","last_became_peered":"2024-03-23T09:43:47.410418+0000","last_unstale":"2024-03-23T09:43:47.415135+0000","last_undegraded":"2024-03-23T09:43:47.415135+0000","last_fullsized":"2024-03-23T09:43:47.415135+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:56.576713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.408925+0000","last_change":"2024-03-23T09:43:47.407642+0000","last_active":"2024-03-23T09:43:47.408925+0000","last_peered":"2024-03-23T09:43:47.408925+0000","last_clean":"2024-03-23T09:43:47.408925+0000","last_became_active":"2024-03-23T09:43:47.407411+0000","last_became_peered":"2024-03-23T09:43:47.407411+0000","last_unstale":"2024-03-23T09:43:47.408925+0000","last_undegraded":"2024-03-23T09:43:47.408925+0000","last_fullsized":"2024-03-23T09:43:47.408925+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:21.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.406983+0000","last_change":"2024-03-23T09:43:47.406179+0000","last_active":"2024-03-23T09:43:47.406983+0000","last_peered":"2024-03-23T09:43:47.406983+0000","last_clean":"2024-03-23T09:43:47.406983+0000","last_became_active":"2024-03-23T09:43:47.405943+0000","last_became_peered":"2024-03-23T09:43:47.405943+0000","last_unstale":"2024-03-23T09:43:47.406983+0000","last_undegraded":"2024-03-23T09:43:47.406983+0000","last_fullsized":"2024-03-23T09:43:47.406983+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:50:34.378460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.408549+0000","last_change":"2024-03-23T09:43:47.406047+0000","last_active":"2024-03-23T09:43:47.408549+0000","last_peered":"2024-03-23T09:43:47.408549+0000","last_clean":"2024-03-23T09:43:47.408549+0000","last_became_active":"2024-03-23T09:43:47.405778+0000","last_became_peered":"2024-03-23T09:43:47.405778+0000","last_unstale":"2024-03-23T09:43:47.408549+0000","last_undegraded":"2024-03-23T09:43:47.408549+0000","last_fullsized":"2024-03-23T09:43:47.408549+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:49.207297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.408064+0000","last_change":"2024-03-23T09:43:47.406090+0000","last_active":"2024-03-23T09:43:47.408064+0000","last_peered":"2024-03-23T09:43:47.408064+0000","last_clean":"2024-03-23T09:43:47.408064+0000","last_became_active":"2024-03-23T09:43:47.405848+0000","last_became_peered":"2024-03-23T09:43:47.405848+0000","last_unstale":"2024-03-23T09:43:47.408064+0000","last_undegraded":"2024-03-23T09:43:47.408064+0000","last_fullsized":"2024-03-23T09:43:47.408064+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:59.729962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.413560+0000","last_change":"2024-03-23T09:43:47.409790+0000","last_active":"2024-03-23T09:43:47.413560+0000","last_peered":"2024-03-23T09:43:47.413560+0000","last_clean":"2024-03-23T09:43:47.413560+0000","last_became_active":"2024-03-23T09:43:47.409551+0000","last_became_peered":"2024-03-23T09:43:47.409551+0000","last_unstale":"2024-03-23T09:43:47.413560+0000","last_undegraded":"2024-03-23T09:43:47.413560+0000","last_fullsized":"2024-03-23T09:43:47.413560+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:07:44.532537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.417761+0000","last_change":"2024-03-23T09:43:47.413841+0000","last_active":"2024-03-23T09:43:47.417761+0000","last_peered":"2024-03-23T09:43:47.417761+0000","last_clean":"2024-03-23T09:43:47.417761+0000","last_became_active":"2024-03-23T09:43:47.413553+0000","last_became_peered":"2024-03-23T09:43:47.413553+0000","last_unstale":"2024-03-23T09:43:47.417761+0000","last_undegraded":"2024-03-23T09:43:47.417761+0000","last_fullsized":"2024-03-23T09:43:47.417761+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:19:12.228744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.406948+0000","last_change":"2024-03-23T09:43:47.405365+0000","last_active":"2024-03-23T09:43:47.406948+0000","last_peered":"2024-03-23T09:43:47.406948+0000","last_clean":"2024-03-23T09:43:47.406948+0000","last_became_active":"2024-03-23T09:43:47.404814+0000","last_became_peered":"2024-03-23T09:43:47.404814+0000","last_unstale":"2024-03-23T09:43:47.406948+0000","last_undegraded":"2024-03-23T09:43:47.406948+0000","last_fullsized":"2024-03-23T09:43:47.406948+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:47:24.835835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.409983+0000","last_change":"2024-03-23T09:43:47.408652+0000","last_active":"2024-03-23T09:43:47.409983+0000","last_peered":"2024-03-23T09:43:47.409983+0000","last_clean":"2024-03-23T09:43:47.409983+0000","last_became_active":"2024-03-23T09:43:47.408350+0000","last_became_peered":"2024-03-23T09:43:47.408350+0000","last_unstale":"2024-03-23T09:43:47.409983+0000","last_undegraded":"2024-03-23T09:43:47.409983+0000","last_fullsized":"2024-03-23T09:43:47.409983+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:10:55.583385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.417752+0000","last_change":"2024-03-23T09:43:47.413202+0000","last_active":"2024-03-23T09:43:47.417752+0000","last_peered":"2024-03-23T09:43:47.417752+0000","last_clean":"2024-03-23T09:43:47.417752+0000","last_became_active":"2024-03-23T09:43:47.412947+0000","last_became_peered":"2024-03-23T09:43:47.412947+0000","last_unstale":"2024-03-23T09:43:47.417752+0000","last_undegraded":"2024-03-23T09:43:47.417752+0000","last_fullsized":"2024-03-23T09:43:47.417752+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:54:41.177596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.408126+0000","last_change":"2024-03-23T09:43:47.406433+0000","last_active":"2024-03-23T09:43:47.408126+0000","last_peered":"2024-03-23T09:43:47.408126+0000","last_clean":"2024-03-23T09:43:47.408126+0000","last_became_active":"2024-03-23T09:43:47.406105+0000","last_became_peered":"2024-03-23T09:43:47.406105+0000","last_unstale":"2024-03-23T09:43:47.408126+0000","last_undegraded":"2024-03-23T09:43:47.408126+0000","last_fullsized":"2024-03-23T09:43:47.408126+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:08.737490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.417963+0000","last_change":"2024-03-23T09:43:47.413783+0000","last_active":"2024-03-23T09:43:47.417963+0000","last_peered":"2024-03-23T09:43:47.417963+0000","last_clean":"2024-03-23T09:43:47.417963+0000","last_became_active":"2024-03-23T09:43:47.413532+0000","last_became_peered":"2024-03-23T09:43:47.413532+0000","last_unstale":"2024-03-23T09:43:47.417963+0000","last_undegraded":"2024-03-23T09:43:47.417963+0000","last_fullsized":"2024-03-23T09:43:47.417963+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:45:37.390159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.408330+0000","last_change":"2024-03-23T09:43:47.405118+0000","last_active":"2024-03-23T09:43:47.408330+0000","last_peered":"2024-03-23T09:43:47.408330+0000","last_clean":"2024-03-23T09:43:47.408330+0000","last_became_active":"2024-03-23T09:43:47.404858+0000","last_became_peered":"2024-03-23T09:43:47.404858+0000","last_unstale":"2024-03-23T09:43:47.408330+0000","last_undegraded":"2024-03-23T09:43:47.408330+0000","last_fullsized":"2024-03-23T09:43:47.408330+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:11.106544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.413044+0000","last_change":"2024-03-23T09:43:47.411729+0000","last_active":"2024-03-23T09:43:47.413044+0000","last_peered":"2024-03-23T09:43:47.413044+0000","last_clean":"2024-03-23T09:43:47.413044+0000","last_became_active":"2024-03-23T09:43:47.411465+0000","last_became_peered":"2024-03-23T09:43:47.411465+0000","last_unstale":"2024-03-23T09:43:47.413044+0000","last_undegraded":"2024-03-23T09:43:47.413044+0000","last_fullsized":"2024-03-23T09:43:47.413044+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.416248+0000","last_change":"2024-03-23T09:43:47.413605+0000","last_active":"2024-03-23T09:43:47.416248+0000","last_peered":"2024-03-23T09:43:47.416248+0000","last_clean":"2024-03-23T09:43:47.416248+0000","last_became_active":"2024-03-23T09:43:47.413211+0000","last_became_peered":"2024-03-23T09:43:47.413211+0000","last_unstale":"2024-03-23T09:43:47.416248+0000","last_undegraded":"2024-03-23T09:43:47.416248+0000","last_fullsized":"2024-03-23T09:43:47.416248+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:00:04.150928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.414927+0000","last_change":"2024-03-23T09:43:47.409593+0000","last_active":"2024-03-23T09:43:47.414927+0000","last_peered":"2024-03-23T09:43:47.414927+0000","last_clean":"2024-03-23T09:43:47.414927+0000","last_became_active":"2024-03-23T09:43:47.409325+0000","last_became_peered":"2024-03-23T09:43:47.409325+0000","last_unstale":"2024-03-23T09:43:47.414927+0000","last_undegraded":"2024-03-23T09:43:47.414927+0000","last_fullsized":"2024-03-23T09:43:47.414927+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.400098+0000","last_change":"2024-03-23T09:43:47.396553+0000","last_active":"2024-03-23T09:43:47.400098+0000","last_peered":"2024-03-23T09:43:47.400098+0000","last_clean":"2024-03-23T09:43:47.400098+0000","last_became_active":"2024-03-23T09:43:47.396310+0000","last_became_peered":"2024-03-23T09:43:47.396310+0000","last_unstale":"2024-03-23T09:43:47.400098+0000","last_undegraded":"2024-03-23T09:43:47.400098+0000","last_fullsized":"2024-03-23T09:43:47.400098+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:46:16.837675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.414130+0000","last_change":"2024-03-23T09:43:47.413608+0000","last_active":"2024-03-23T09:43:47.414130+0000","last_peered":"2024-03-23T09:43:47.414130+0000","last_clean":"2024-03-23T09:43:47.414130+0000","last_became_active":"2024-03-23T09:43:47.413389+0000","last_became_peered":"2024-03-23T09:43:47.413389+0000","last_unstale":"2024-03-23T09:43:47.414130+0000","last_undegraded":"2024-03-23T09:43:47.414130+0000","last_fullsized":"2024-03-23T09:43:47.414130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:02:58.178256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.411219+0000","last_change":"2024-03-23T09:43:47.410718+0000","last_active":"2024-03-23T09:43:47.411219+0000","last_peered":"2024-03-23T09:43:47.411219+0000","last_clean":"2024-03-23T09:43:47.411219+0000","last_became_active":"2024-03-23T09:43:47.410489+0000","last_became_peered":"2024-03-23T09:43:47.410489+0000","last_unstale":"2024-03-23T09:43:47.411219+0000","last_undegraded":"2024-03-23T09:43:47.411219+0000","last_fullsized":"2024-03-23T09:43:47.411219+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.420583+0000","last_change":"2024-03-23T09:43:47.036626+0000","last_active":"2024-03-23T09:43:47.420583+0000","last_peered":"2024-03-23T09:43:47.420583+0000","last_clean":"2024-03-23T09:43:47.420583+0000","last_became_active":"2024-03-23T09:43:47.036383+0000","last_became_peered":"2024-03-23T09:43:47.036383+0000","last_unstale":"2024-03-23T09:43:47.420583+0000","last_undegraded":"2024-03-23T09:43:47.420583+0000","last_fullsized":"2024-03-23T09:43:47.420583+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:18.815610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.404555+0000","last_change":"2024-03-23T09:43:47.402524+0000","last_active":"2024-03-23T09:43:47.404555+0000","last_peered":"2024-03-23T09:43:47.404555+0000","last_clean":"2024-03-23T09:43:47.404555+0000","last_became_active":"2024-03-23T09:43:47.402283+0000","last_became_peered":"2024-03-23T09:43:47.402283+0000","last_unstale":"2024-03-23T09:43:47.404555+0000","last_undegraded":"2024-03-23T09:43:47.404555+0000","last_fullsized":"2024-03-23T09:43:47.404555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:05:21.051130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.420149+0000","last_change":"2024-03-23T09:43:47.037068+0000","last_active":"2024-03-23T09:43:47.420149+0000","last_peered":"2024-03-23T09:43:47.420149+0000","last_clean":"2024-03-23T09:43:47.420149+0000","last_became_active":"2024-03-23T09:43:47.036452+0000","last_became_peered":"2024-03-23T09:43:47.036452+0000","last_unstale":"2024-03-23T09:43:47.420149+0000","last_undegraded":"2024-03-23T09:43:47.420149+0000","last_fullsized":"2024-03-23T09:43:47.420149+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:26.308593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.408257+0000","last_change":"2024-03-23T09:43:47.406360+0000","last_active":"2024-03-23T09:43:47.408257+0000","last_peered":"2024-03-23T09:43:47.408257+0000","last_clean":"2024-03-23T09:43:47.408257+0000","last_became_active":"2024-03-23T09:43:47.406107+0000","last_became_peered":"2024-03-23T09:43:47.406107+0000","last_unstale":"2024-03-23T09:43:47.408257+0000","last_undegraded":"2024-03-23T09:43:47.408257+0000","last_fullsized":"2024-03-23T09:43:47.408257+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.416986+0000","last_change":"2024-03-23T09:43:47.029691+0000","last_active":"2024-03-23T09:43:47.416986+0000","last_peered":"2024-03-23T09:43:47.416986+0000","last_clean":"2024-03-23T09:43:47.416986+0000","last_became_active":"2024-03-23T09:43:47.029427+0000","last_became_peered":"2024-03-23T09:43:47.029427+0000","last_unstale":"2024-03-23T09:43:47.416986+0000","last_undegraded":"2024-03-23T09:43:47.416986+0000","last_fullsized":"2024-03-23T09:43:47.416986+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.418645+0000","last_change":"2024-03-23T09:43:47.417632+0000","last_active":"2024-03-23T09:43:47.418645+0000","last_peered":"2024-03-23T09:43:47.418645+0000","last_clean":"2024-03-23T09:43:47.418645+0000","last_became_active":"2024-03-23T09:43:47.417387+0000","last_became_peered":"2024-03-23T09:43:47.417387+0000","last_unstale":"2024-03-23T09:43:47.418645+0000","last_undegraded":"2024-03-23T09:43:47.418645+0000","last_fullsized":"2024-03-23T09:43:47.418645+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:45.147489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.418396+0000","last_change":"2024-03-23T09:43:47.039140+0000","last_active":"2024-03-23T09:43:47.418396+0000","last_peered":"2024-03-23T09:43:47.418396+0000","last_clean":"2024-03-23T09:43:47.418396+0000","last_became_active":"2024-03-23T09:43:47.038851+0000","last_became_peered":"2024-03-23T09:43:47.038851+0000","last_unstale":"2024-03-23T09:43:47.418396+0000","last_undegraded":"2024-03-23T09:43:47.418396+0000","last_fullsized":"2024-03-23T09:43:47.418396+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:30:18.815610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.406964+0000","last_change":"2024-03-23T09:43:47.405521+0000","last_active":"2024-03-23T09:43:47.406964+0000","last_peered":"2024-03-23T09:43:47.406964+0000","last_clean":"2024-03-23T09:43:47.406964+0000","last_became_active":"2024-03-23T09:43:47.405081+0000","last_became_peered":"2024-03-23T09:43:47.405081+0000","last_unstale":"2024-03-23T09:43:47.406964+0000","last_undegraded":"2024-03-23T09:43:47.406964+0000","last_fullsized":"2024-03-23T09:43:47.406964+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:28:32.181328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.406729+0000","last_change":"2024-03-23T09:43:47.031421+0000","last_active":"2024-03-23T09:43:47.406729+0000","last_peered":"2024-03-23T09:43:47.406729+0000","last_clean":"2024-03-23T09:43:47.406729+0000","last_became_active":"2024-03-23T09:43:47.031157+0000","last_became_peered":"2024-03-23T09:43:47.031157+0000","last_unstale":"2024-03-23T09:43:47.406729+0000","last_undegraded":"2024-03-23T09:43:47.406729+0000","last_fullsized":"2024-03-23T09:43:47.406729+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:49.393118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.413712+0000","last_change":"2024-03-23T09:43:47.410051+0000","last_active":"2024-03-23T09:43:47.413712+0000","last_peered":"2024-03-23T09:43:47.413712+0000","last_clean":"2024-03-23T09:43:47.413712+0000","last_became_active":"2024-03-23T09:43:47.409816+0000","last_became_peered":"2024-03-23T09:43:47.409816+0000","last_unstale":"2024-03-23T09:43:47.413712+0000","last_undegraded":"2024-03-23T09:43:47.413712+0000","last_fullsized":"2024-03-23T09:43:47.413712+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:42:29.904309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.420251+0000","last_change":"2024-03-23T09:43:47.034482+0000","last_active":"2024-03-23T09:43:47.420251+0000","last_peered":"2024-03-23T09:43:47.420251+0000","last_clean":"2024-03-23T09:43:47.420251+0000","last_became_active":"2024-03-23T09:43:47.034190+0000","last_became_peered":"2024-03-23T09:43:47.034190+0000","last_unstale":"2024-03-23T09:43:47.420251+0000","last_undegraded":"2024-03-23T09:43:47.420251+0000","last_fullsized":"2024-03-23T09:43:47.420251+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:27:00.248727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.417704+0000","last_change":"2024-03-23T09:43:47.409802+0000","last_active":"2024-03-23T09:43:47.417704+0000","last_peered":"2024-03-23T09:43:47.417704+0000","last_clean":"2024-03-23T09:43:47.417704+0000","last_became_active":"2024-03-23T09:43:47.409483+0000","last_became_peered":"2024-03-23T09:43:47.409483+0000","last_unstale":"2024-03-23T09:43:47.417704+0000","last_undegraded":"2024-03-23T09:43:47.417704+0000","last_fullsized":"2024-03-23T09:43:47.417704+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:59.729962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.404447+0000","last_change":"2024-03-23T09:43:47.034996+0000","last_active":"2024-03-23T09:43:47.404447+0000","last_peered":"2024-03-23T09:43:47.404447+0000","last_clean":"2024-03-23T09:43:47.404447+0000","last_became_active":"2024-03-23T09:43:47.034715+0000","last_became_peered":"2024-03-23T09:43:47.034715+0000","last_unstale":"2024-03-23T09:43:47.404447+0000","last_undegraded":"2024-03-23T09:43:47.404447+0000","last_fullsized":"2024-03-23T09:43:47.404447+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.413249+0000","last_change":"2024-03-23T09:43:47.407493+0000","last_active":"2024-03-23T09:43:47.413249+0000","last_peered":"2024-03-23T09:43:47.413249+0000","last_clean":"2024-03-23T09:43:47.413249+0000","last_became_active":"2024-03-23T09:43:47.407110+0000","last_became_peered":"2024-03-23T09:43:47.407110+0000","last_unstale":"2024-03-23T09:43:47.413249+0000","last_undegraded":"2024-03-23T09:43:47.413249+0000","last_fullsized":"2024-03-23T09:43:47.413249+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:25:06.163966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.394451+0000","last_change":"2024-03-23T09:43:47.036762+0000","last_active":"2024-03-23T09:43:47.394451+0000","last_peered":"2024-03-23T09:43:47.394451+0000","last_clean":"2024-03-23T09:43:47.394451+0000","last_became_active":"2024-03-23T09:43:47.036510+0000","last_became_peered":"2024-03-23T09:43:47.036510+0000","last_unstale":"2024-03-23T09:43:47.394451+0000","last_undegraded":"2024-03-23T09:43:47.394451+0000","last_fullsized":"2024-03-23T09:43:47.394451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:58.725092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.415617+0000","last_change":"2024-03-23T09:43:47.409791+0000","last_active":"2024-03-23T09:43:47.415617+0000","last_peered":"2024-03-23T09:43:47.415617+0000","last_clean":"2024-03-23T09:43:47.415617+0000","last_became_active":"2024-03-23T09:43:47.409337+0000","last_became_peered":"2024-03-23T09:43:47.409337+0000","last_unstale":"2024-03-23T09:43:47.415617+0000","last_undegraded":"2024-03-23T09:43:47.415617+0000","last_fullsized":"2024-03-23T09:43:47.415617+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:36:27.003657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.395700+0000","last_change":"2024-03-23T09:43:47.034507+0000","last_active":"2024-03-23T09:43:47.395700+0000","last_peered":"2024-03-23T09:43:47.395700+0000","last_clean":"2024-03-23T09:43:47.395700+0000","last_became_active":"2024-03-23T09:43:47.034260+0000","last_became_peered":"2024-03-23T09:43:47.034260+0000","last_unstale":"2024-03-23T09:43:47.395700+0000","last_undegraded":"2024-03-23T09:43:47.395700+0000","last_fullsized":"2024-03-23T09:43:47.395700+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.409768+0000","last_change":"2024-03-23T09:43:47.404378+0000","last_active":"2024-03-23T09:43:47.409768+0000","last_peered":"2024-03-23T09:43:47.409768+0000","last_clean":"2024-03-23T09:43:47.409768+0000","last_became_active":"2024-03-23T09:43:47.404106+0000","last_became_peered":"2024-03-23T09:43:47.404106+0000","last_unstale":"2024-03-23T09:43:47.409768+0000","last_undegraded":"2024-03-23T09:43:47.409768+0000","last_fullsized":"2024-03-23T09:43:47.409768+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:23:27.313463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.397178+0000","last_change":"2024-03-23T09:43:47.036141+0000","last_active":"2024-03-23T09:43:47.397178+0000","last_peered":"2024-03-23T09:43:47.397178+0000","last_clean":"2024-03-23T09:43:47.397178+0000","last_became_active":"2024-03-23T09:43:47.035805+0000","last_became_peered":"2024-03-23T09:43:47.035805+0000","last_unstale":"2024-03-23T09:43:47.397178+0000","last_undegraded":"2024-03-23T09:43:47.397178+0000","last_fullsized":"2024-03-23T09:43:47.397178+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:22.465435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.408596+0000","last_change":"2024-03-23T09:43:47.407609+0000","last_active":"2024-03-23T09:43:47.408596+0000","last_peered":"2024-03-23T09:43:47.408596+0000","last_clean":"2024-03-23T09:43:47.408596+0000","last_became_active":"2024-03-23T09:43:47.407354+0000","last_became_peered":"2024-03-23T09:43:47.407354+0000","last_unstale":"2024-03-23T09:43:47.408596+0000","last_undegraded":"2024-03-23T09:43:47.408596+0000","last_fullsized":"2024-03-23T09:43:47.408596+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:28:37.758076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.414443+0000","last_change":"2024-03-23T09:43:47.032774+0000","last_active":"2024-03-23T09:43:47.414443+0000","last_peered":"2024-03-23T09:43:47.414443+0000","last_clean":"2024-03-23T09:43:47.414443+0000","last_became_active":"2024-03-23T09:43:47.032528+0000","last_became_peered":"2024-03-23T09:43:47.032528+0000","last_unstale":"2024-03-23T09:43:47.414443+0000","last_undegraded":"2024-03-23T09:43:47.414443+0000","last_fullsized":"2024-03-23T09:43:47.414443+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.408801+0000","last_change":"2024-03-23T09:43:47.405669+0000","last_active":"2024-03-23T09:43:47.408801+0000","last_peered":"2024-03-23T09:43:47.408801+0000","last_clean":"2024-03-23T09:43:47.408801+0000","last_became_active":"2024-03-23T09:43:47.405332+0000","last_became_peered":"2024-03-23T09:43:47.405332+0000","last_unstale":"2024-03-23T09:43:47.408801+0000","last_undegraded":"2024-03-23T09:43:47.408801+0000","last_fullsized":"2024-03-23T09:43:47.408801+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:11.106544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.420343+0000","last_change":"2024-03-23T09:43:47.033945+0000","last_active":"2024-03-23T09:43:47.420343+0000","last_peered":"2024-03-23T09:43:47.420343+0000","last_clean":"2024-03-23T09:43:47.420343+0000","last_became_active":"2024-03-23T09:43:47.033037+0000","last_became_peered":"2024-03-23T09:43:47.033037+0000","last_unstale":"2024-03-23T09:43:47.420343+0000","last_undegraded":"2024-03-23T09:43:47.420343+0000","last_fullsized":"2024-03-23T09:43:47.420343+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:28:36.753206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.415035+0000","last_change":"2024-03-23T09:43:47.412001+0000","last_active":"2024-03-23T09:43:47.415035+0000","last_peered":"2024-03-23T09:43:47.415035+0000","last_clean":"2024-03-23T09:43:47.415035+0000","last_became_active":"2024-03-23T09:43:47.411541+0000","last_became_peered":"2024-03-23T09:43:47.411541+0000","last_unstale":"2024-03-23T09:43:47.415035+0000","last_undegraded":"2024-03-23T09:43:47.415035+0000","last_fullsized":"2024-03-23T09:43:47.415035+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:51:10.516410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.417728+0000","last_change":"2024-03-23T09:43:47.034748+0000","last_active":"2024-03-23T09:43:47.417728+0000","last_peered":"2024-03-23T09:43:47.417728+0000","last_clean":"2024-03-23T09:43:47.417728+0000","last_became_active":"2024-03-23T09:43:47.034340+0000","last_became_peered":"2024-03-23T09:43:47.034340+0000","last_unstale":"2024-03-23T09:43:47.417728+0000","last_undegraded":"2024-03-23T09:43:47.417728+0000","last_fullsized":"2024-03-23T09:43:47.417728+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:06.713892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.408690+0000","last_change":"2024-03-23T09:43:47.405831+0000","last_active":"2024-03-23T09:43:47.408690+0000","last_peered":"2024-03-23T09:43:47.408690+0000","last_clean":"2024-03-23T09:43:47.408690+0000","last_became_active":"2024-03-23T09:43:47.405514+0000","last_became_peered":"2024-03-23T09:43:47.405514+0000","last_unstale":"2024-03-23T09:43:47.408690+0000","last_undegraded":"2024-03-23T09:43:47.408690+0000","last_fullsized":"2024-03-23T09:43:47.408690+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.400283+0000","last_change":"2024-03-23T09:43:47.034963+0000","last_active":"2024-03-23T09:43:47.400283+0000","last_peered":"2024-03-23T09:43:47.400283+0000","last_clean":"2024-03-23T09:43:47.400283+0000","last_became_active":"2024-03-23T09:43:47.034680+0000","last_became_peered":"2024-03-23T09:43:47.034680+0000","last_unstale":"2024-03-23T09:43:47.400283+0000","last_undegraded":"2024-03-23T09:43:47.400283+0000","last_fullsized":"2024-03-23T09:43:47.400283+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:15:54.128943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.405525+0000","last_change":"2024-03-23T09:43:47.402949+0000","last_active":"2024-03-23T09:43:47.405525+0000","last_peered":"2024-03-23T09:43:47.405525+0000","last_clean":"2024-03-23T09:43:47.405525+0000","last_became_active":"2024-03-23T09:43:47.402679+0000","last_became_peered":"2024-03-23T09:43:47.402679+0000","last_unstale":"2024-03-23T09:43:47.405525+0000","last_undegraded":"2024-03-23T09:43:47.405525+0000","last_fullsized":"2024-03-23T09:43:47.405525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:27:01.253597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.404667+0000","last_change":"2024-03-23T09:43:47.038681+0000","last_active":"2024-03-23T09:43:47.404667+0000","last_peered":"2024-03-23T09:43:47.404667+0000","last_clean":"2024-03-23T09:43:47.404667+0000","last_became_active":"2024-03-23T09:43:47.038410+0000","last_became_peered":"2024-03-23T09:43:47.038410+0000","last_unstale":"2024-03-23T09:43:47.404667+0000","last_undegraded":"2024-03-23T09:43:47.404667+0000","last_fullsized":"2024-03-23T09:43:47.404667+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:19.400345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.410001+0000","last_change":"2024-03-23T09:43:47.408704+0000","last_active":"2024-03-23T09:43:47.410001+0000","last_peered":"2024-03-23T09:43:47.410001+0000","last_clean":"2024-03-23T09:43:47.410001+0000","last_became_active":"2024-03-23T09:43:47.408471+0000","last_became_peered":"2024-03-23T09:43:47.408471+0000","last_unstale":"2024-03-23T09:43:47.410001+0000","last_undegraded":"2024-03-23T09:43:47.410001+0000","last_fullsized":"2024-03-23T09:43:47.410001+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:02:58.178256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":19,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.418213+0000","last_change":"2024-03-23T09:43:47.036966+0000","last_active":"2024-03-23T09:43:47.418213+0000","last_peered":"2024-03-23T09:43:47.418213+0000","last_clean":"2024-03-23T09:43:47.418213+0000","last_became_active":"2024-03-23T09:43:47.036691+0000","last_became_peered":"2024-03-23T09:43:47.036691+0000","last_unstale":"2024-03-23T09:43:47.418213+0000","last_undegraded":"2024-03-23T09:43:47.418213+0000","last_fullsized":"2024-03-23T09:43:47.418213+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.410030+0000","last_change":"2024-03-23T09:43:47.408606+0000","last_active":"2024-03-23T09:43:47.410030+0000","last_peered":"2024-03-23T09:43:47.410030+0000","last_clean":"2024-03-23T09:43:47.410030+0000","last_became_active":"2024-03-23T09:43:47.408302+0000","last_became_peered":"2024-03-23T09:43:47.408302+0000","last_unstale":"2024-03-23T09:43:47.410030+0000","last_undegraded":"2024-03-23T09:43:47.410030+0000","last_fullsized":"2024-03-23T09:43:47.410030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.417555+0000","last_change":"2024-03-23T09:43:47.036026+0000","last_active":"2024-03-23T09:43:47.417555+0000","last_peered":"2024-03-23T09:43:47.417555+0000","last_clean":"2024-03-23T09:43:47.417555+0000","last_became_active":"2024-03-23T09:43:47.035690+0000","last_became_peered":"2024-03-23T09:43:47.035690+0000","last_unstale":"2024-03-23T09:43:47.417555+0000","last_undegraded":"2024-03-23T09:43:47.417555+0000","last_fullsized":"2024-03-23T09:43:47.417555+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:20.475983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.420279+0000","last_change":"2024-03-23T09:43:47.416989+0000","last_active":"2024-03-23T09:43:47.420279+0000","last_peered":"2024-03-23T09:43:47.420279+0000","last_clean":"2024-03-23T09:43:47.420279+0000","last_became_active":"2024-03-23T09:43:47.416742+0000","last_became_peered":"2024-03-23T09:43:47.416742+0000","last_unstale":"2024-03-23T09:43:47.420279+0000","last_undegraded":"2024-03-23T09:43:47.420279+0000","last_fullsized":"2024-03-23T09:43:47.420279+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:17:23.470304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.417623+0000","last_change":"2024-03-23T09:43:47.032740+0000","last_active":"2024-03-23T09:43:47.417623+0000","last_peered":"2024-03-23T09:43:47.417623+0000","last_clean":"2024-03-23T09:43:47.417623+0000","last_became_active":"2024-03-23T09:43:47.032456+0000","last_became_peered":"2024-03-23T09:43:47.032456+0000","last_unstale":"2024-03-23T09:43:47.417623+0000","last_undegraded":"2024-03-23T09:43:47.417623+0000","last_fullsized":"2024-03-23T09:43:47.417623+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:37.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.413543+0000","last_change":"2024-03-23T09:43:47.406924+0000","last_active":"2024-03-23T09:43:47.413543+0000","last_peered":"2024-03-23T09:43:47.413543+0000","last_clean":"2024-03-23T09:43:47.413543+0000","last_became_active":"2024-03-23T09:43:47.406645+0000","last_became_peered":"2024-03-23T09:43:47.406645+0000","last_unstale":"2024-03-23T09:43:47.413543+0000","last_undegraded":"2024-03-23T09:43:47.413543+0000","last_fullsized":"2024-03-23T09:43:47.413543+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:07:10.261476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.418269+0000","last_change":"2024-03-23T09:43:47.038642+0000","last_active":"2024-03-23T09:43:47.418269+0000","last_peered":"2024-03-23T09:43:47.418269+0000","last_clean":"2024-03-23T09:43:47.418269+0000","last_became_active":"2024-03-23T09:43:47.038385+0000","last_became_peered":"2024-03-23T09:43:47.038385+0000","last_unstale":"2024-03-23T09:43:47.418269+0000","last_undegraded":"2024-03-23T09:43:47.418269+0000","last_fullsized":"2024-03-23T09:43:47.418269+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:33.981220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.414847+0000","last_change":"2024-03-23T09:43:47.408928+0000","last_active":"2024-03-23T09:43:47.414847+0000","last_peered":"2024-03-23T09:43:47.414847+0000","last_clean":"2024-03-23T09:43:47.414847+0000","last_became_active":"2024-03-23T09:43:47.407890+0000","last_became_peered":"2024-03-23T09:43:47.407890+0000","last_unstale":"2024-03-23T09:43:47.414847+0000","last_undegraded":"2024-03-23T09:43:47.414847+0000","last_fullsized":"2024-03-23T09:43:47.414847+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:34:21.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.398578+0000","last_change":"2024-03-23T09:43:47.032632+0000","last_active":"2024-03-23T09:43:47.398578+0000","last_peered":"2024-03-23T09:43:47.398578+0000","last_clean":"2024-03-23T09:43:47.398578+0000","last_became_active":"2024-03-23T09:43:47.032375+0000","last_became_peered":"2024-03-23T09:43:47.032375+0000","last_unstale":"2024-03-23T09:43:47.398578+0000","last_undegraded":"2024-03-23T09:43:47.398578+0000","last_fullsized":"2024-03-23T09:43:47.398578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.415015+0000","last_change":"2024-03-23T09:43:47.411806+0000","last_active":"2024-03-23T09:43:47.415015+0000","last_peered":"2024-03-23T09:43:47.415015+0000","last_clean":"2024-03-23T09:43:47.415015+0000","last_became_active":"2024-03-23T09:43:47.410966+0000","last_became_peered":"2024-03-23T09:43:47.410966+0000","last_unstale":"2024-03-23T09:43:47.415015+0000","last_undegraded":"2024-03-23T09:43:47.415015+0000","last_fullsized":"2024-03-23T09:43:47.415015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:14:35.775608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.419990+0000","last_change":"2024-03-23T09:43:47.030434+0000","last_active":"2024-03-23T09:43:47.419990+0000","last_peered":"2024-03-23T09:43:47.419990+0000","last_clean":"2024-03-23T09:43:47.419990+0000","last_became_active":"2024-03-23T09:43:47.030164+0000","last_became_peered":"2024-03-23T09:43:47.030164+0000","last_unstale":"2024-03-23T09:43:47.419990+0000","last_undegraded":"2024-03-23T09:43:47.419990+0000","last_fullsized":"2024-03-23T09:43:47.419990+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:09.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.409536+0000","last_change":"2024-03-23T09:43:47.408837+0000","last_active":"2024-03-23T09:43:47.409536+0000","last_peered":"2024-03-23T09:43:47.409536+0000","last_clean":"2024-03-23T09:43:47.409536+0000","last_became_active":"2024-03-23T09:43:47.408611+0000","last_became_peered":"2024-03-23T09:43:47.408611+0000","last_unstale":"2024-03-23T09:43:47.409536+0000","last_undegraded":"2024-03-23T09:43:47.409536+0000","last_fullsized":"2024-03-23T09:43:47.409536+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:22:26.903375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.392302+0000","last_change":"2024-03-23T09:43:47.031235+0000","last_active":"2024-03-23T09:43:47.392302+0000","last_peered":"2024-03-23T09:43:47.392302+0000","last_clean":"2024-03-23T09:43:47.392302+0000","last_became_active":"2024-03-23T09:43:47.030949+0000","last_became_peered":"2024-03-23T09:43:47.030949+0000","last_unstale":"2024-03-23T09:43:47.392302+0000","last_undegraded":"2024-03-23T09:43:47.392302+0000","last_fullsized":"2024-03-23T09:43:47.392302+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:58:56.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.406493+0000","last_change":"2024-03-23T09:43:47.403291+0000","last_active":"2024-03-23T09:43:47.406493+0000","last_peered":"2024-03-23T09:43:47.406493+0000","last_clean":"2024-03-23T09:43:47.406493+0000","last_became_active":"2024-03-23T09:43:47.403053+0000","last_became_peered":"2024-03-23T09:43:47.403053+0000","last_unstale":"2024-03-23T09:43:47.406493+0000","last_undegraded":"2024-03-23T09:43:47.406493+0000","last_fullsized":"2024-03-23T09:43:47.406493+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:01:10.750528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.404786+0000","last_change":"2024-03-23T09:43:47.038838+0000","last_active":"2024-03-23T09:43:47.404786+0000","last_peered":"2024-03-23T09:43:47.404786+0000","last_clean":"2024-03-23T09:43:47.404786+0000","last_became_active":"2024-03-23T09:43:47.038574+0000","last_became_peered":"2024-03-23T09:43:47.038574+0000","last_unstale":"2024-03-23T09:43:47.404786+0000","last_undegraded":"2024-03-23T09:43:47.404786+0000","last_fullsized":"2024-03-23T09:43:47.404786+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:55.571843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.417602+0000","last_change":"2024-03-23T09:43:47.412620+0000","last_active":"2024-03-23T09:43:47.417602+0000","last_peered":"2024-03-23T09:43:47.417602+0000","last_clean":"2024-03-23T09:43:47.417602+0000","last_became_active":"2024-03-23T09:43:47.408350+0000","last_became_peered":"2024-03-23T09:43:47.408350+0000","last_unstale":"2024-03-23T09:43:47.417602+0000","last_undegraded":"2024-03-23T09:43:47.417602+0000","last_fullsized":"2024-03-23T09:43:47.417602+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:49.207297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.417840+0000","last_change":"2024-03-23T09:43:47.029629+0000","last_active":"2024-03-23T09:43:47.417840+0000","last_peered":"2024-03-23T09:43:47.417840+0000","last_clean":"2024-03-23T09:43:47.417840+0000","last_became_active":"2024-03-23T09:43:47.029338+0000","last_became_peered":"2024-03-23T09:43:47.029338+0000","last_unstale":"2024-03-23T09:43:47.417840+0000","last_undegraded":"2024-03-23T09:43:47.417840+0000","last_fullsized":"2024-03-23T09:43:47.417840+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:15:54.128943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.417677+0000","last_change":"2024-03-23T09:43:47.413196+0000","last_active":"2024-03-23T09:43:47.417677+0000","last_peered":"2024-03-23T09:43:47.417677+0000","last_clean":"2024-03-23T09:43:47.417677+0000","last_became_active":"2024-03-23T09:43:47.412934+0000","last_became_peered":"2024-03-23T09:43:47.412934+0000","last_unstale":"2024-03-23T09:43:47.417677+0000","last_undegraded":"2024-03-23T09:43:47.417677+0000","last_fullsized":"2024-03-23T09:43:47.417677+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:08:07.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.402192+0000","last_change":"2024-03-23T09:43:47.035203+0000","last_active":"2024-03-23T09:43:47.402192+0000","last_peered":"2024-03-23T09:43:47.402192+0000","last_clean":"2024-03-23T09:43:47.402192+0000","last_became_active":"2024-03-23T09:43:47.034912+0000","last_became_peered":"2024-03-23T09:43:47.034912+0000","last_unstale":"2024-03-23T09:43:47.402192+0000","last_undegraded":"2024-03-23T09:43:47.402192+0000","last_fullsized":"2024-03-23T09:43:47.402192+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:36:25.998787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.408578+0000","last_change":"2024-03-23T09:43:47.407546+0000","last_active":"2024-03-23T09:43:47.408578+0000","last_peered":"2024-03-23T09:43:47.408578+0000","last_clean":"2024-03-23T09:43:47.408578+0000","last_became_active":"2024-03-23T09:43:47.407300+0000","last_became_peered":"2024-03-23T09:43:47.407300+0000","last_unstale":"2024-03-23T09:43:47.408578+0000","last_undegraded":"2024-03-23T09:43:47.408578+0000","last_fullsized":"2024-03-23T09:43:47.408578+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:50.397987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.400396+0000","last_change":"2024-03-23T09:43:47.029065+0000","last_active":"2024-03-23T09:43:47.400396+0000","last_peered":"2024-03-23T09:43:47.400396+0000","last_clean":"2024-03-23T09:43:47.400396+0000","last_became_active":"2024-03-23T09:43:47.028797+0000","last_became_peered":"2024-03-23T09:43:47.028797+0000","last_unstale":"2024-03-23T09:43:47.400396+0000","last_undegraded":"2024-03-23T09:43:47.400396+0000","last_fullsized":"2024-03-23T09:43:47.400396+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:32:33.947908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.418939+0000","last_change":"2024-03-23T09:43:47.418570+0000","last_active":"2024-03-23T09:43:47.418939+0000","last_peered":"2024-03-23T09:43:47.418939+0000","last_clean":"2024-03-23T09:43:47.418939+0000","last_became_active":"2024-03-23T09:43:47.418280+0000","last_became_peered":"2024-03-23T09:43:47.418280+0000","last_unstale":"2024-03-23T09:43:47.418939+0000","last_undegraded":"2024-03-23T09:43:47.418939+0000","last_fullsized":"2024-03-23T09:43:47.418939+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:19:12.228744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.408462+0000","last_change":"2024-03-23T09:43:47.038126+0000","last_active":"2024-03-23T09:43:47.408462+0000","last_peered":"2024-03-23T09:43:47.408462+0000","last_clean":"2024-03-23T09:43:47.408462+0000","last_became_active":"2024-03-23T09:43:47.037850+0000","last_became_peered":"2024-03-23T09:43:47.037850+0000","last_unstale":"2024-03-23T09:43:47.408462+0000","last_undegraded":"2024-03-23T09:43:47.408462+0000","last_fullsized":"2024-03-23T09:43:47.408462+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:00:28.042714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.413416+0000","last_change":"2024-03-23T09:43:47.409169+0000","last_active":"2024-03-23T09:43:47.413416+0000","last_peered":"2024-03-23T09:43:47.413416+0000","last_clean":"2024-03-23T09:43:47.413416+0000","last_became_active":"2024-03-23T09:43:47.408914+0000","last_became_peered":"2024-03-23T09:43:47.408914+0000","last_unstale":"2024-03-23T09:43:47.413416+0000","last_undegraded":"2024-03-23T09:43:47.413416+0000","last_fullsized":"2024-03-23T09:43:47.413416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:07:59.768596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.392309+0000","last_change":"2024-03-23T09:43:47.036954+0000","last_active":"2024-03-23T09:43:47.392309+0000","last_peered":"2024-03-23T09:43:47.392309+0000","last_clean":"2024-03-23T09:43:47.392309+0000","last_became_active":"2024-03-23T09:43:47.036655+0000","last_became_peered":"2024-03-23T09:43:47.036655+0000","last_unstale":"2024-03-23T09:43:47.392309+0000","last_undegraded":"2024-03-23T09:43:47.392309+0000","last_fullsized":"2024-03-23T09:43:47.392309+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:05:51.129365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.417626+0000","last_change":"2024-03-23T09:43:47.409973+0000","last_active":"2024-03-23T09:43:47.417626+0000","last_peered":"2024-03-23T09:43:47.417626+0000","last_clean":"2024-03-23T09:43:47.417626+0000","last_became_active":"2024-03-23T09:43:47.409683+0000","last_became_peered":"2024-03-23T09:43:47.409683+0000","last_unstale":"2024-03-23T09:43:47.417626+0000","last_undegraded":"2024-03-23T09:43:47.417626+0000","last_fullsized":"2024-03-23T09:43:47.417626+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:45.147489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.397313+0000","last_change":"2024-03-23T09:43:47.032771+0000","last_active":"2024-03-23T09:43:47.397313+0000","last_peered":"2024-03-23T09:43:47.397313+0000","last_clean":"2024-03-23T09:43:47.397313+0000","last_became_active":"2024-03-23T09:43:47.032489+0000","last_became_peered":"2024-03-23T09:43:47.032489+0000","last_unstale":"2024-03-23T09:43:47.397313+0000","last_undegraded":"2024-03-23T09:43:47.397313+0000","last_fullsized":"2024-03-23T09:43:47.397313+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:48:49.438386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.407526+0000","last_change":"2024-03-23T09:43:47.405952+0000","last_active":"2024-03-23T09:43:47.407526+0000","last_peered":"2024-03-23T09:43:47.407526+0000","last_clean":"2024-03-23T09:43:47.407526+0000","last_became_active":"2024-03-23T09:43:47.405669+0000","last_became_peered":"2024-03-23T09:43:47.405669+0000","last_unstale":"2024-03-23T09:43:47.407526+0000","last_undegraded":"2024-03-23T09:43:47.407526+0000","last_fullsized":"2024-03-23T09:43:47.407526+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:24:18.285067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.392421+0000","last_change":"2024-03-23T09:43:47.030522+0000","last_active":"2024-03-23T09:43:47.392421+0000","last_peered":"2024-03-23T09:43:47.392421+0000","last_clean":"2024-03-23T09:43:47.392421+0000","last_became_active":"2024-03-23T09:43:47.030216+0000","last_became_peered":"2024-03-23T09:43:47.030216+0000","last_unstale":"2024-03-23T09:43:47.392421+0000","last_undegraded":"2024-03-23T09:43:47.392421+0000","last_fullsized":"2024-03-23T09:43:47.392421+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:32:33.947908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.413231+0000","last_change":"2024-03-23T09:43:47.407343+0000","last_active":"2024-03-23T09:43:47.413231+0000","last_peered":"2024-03-23T09:43:47.413231+0000","last_clean":"2024-03-23T09:43:47.413231+0000","last_became_active":"2024-03-23T09:43:47.406764+0000","last_became_peered":"2024-03-23T09:43:47.406764+0000","last_unstale":"2024-03-23T09:43:47.413231+0000","last_undegraded":"2024-03-23T09:43:47.413231+0000","last_fullsized":"2024-03-23T09:43:47.413231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:30:34.056749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.397216+0000","last_change":"2024-03-23T09:43:47.038282+0000","last_active":"2024-03-23T09:43:47.397216+0000","last_peered":"2024-03-23T09:43:47.397216+0000","last_clean":"2024-03-23T09:43:47.397216+0000","last_became_active":"2024-03-23T09:43:47.037787+0000","last_became_peered":"2024-03-23T09:43:47.037787+0000","last_unstale":"2024-03-23T09:43:47.397216+0000","last_undegraded":"2024-03-23T09:43:47.397216+0000","last_fullsized":"2024-03-23T09:43:47.397216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:10.101674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.413520+0000","last_change":"2024-03-23T09:43:47.411435+0000","last_active":"2024-03-23T09:43:47.413520+0000","last_peered":"2024-03-23T09:43:47.413520+0000","last_clean":"2024-03-23T09:43:47.413520+0000","last_became_active":"2024-03-23T09:43:47.410344+0000","last_became_peered":"2024-03-23T09:43:47.410344+0000","last_unstale":"2024-03-23T09:43:47.413520+0000","last_undegraded":"2024-03-23T09:43:47.413520+0000","last_fullsized":"2024-03-23T09:43:47.413520+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:47.010185+0000","last_clean_scrub_stamp":"2024-03-23T09:43:47.010185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:52:24.758413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.401162+0000","last_change":"2024-03-23T09:43:47.037606+0000","last_active":"2024-03-23T09:43:47.401162+0000","last_peered":"2024-03-23T09:43:47.401162+0000","last_clean":"2024-03-23T09:43:47.401162+0000","last_became_active":"2024-03-23T09:43:47.037315+0000","last_became_peered":"2024-03-23T09:43:47.037315+0000","last_unstale":"2024-03-23T09:43:47.401162+0000","last_undegraded":"2024-03-23T09:43:47.401162+0000","last_fullsized":"2024-03-23T09:43:47.401162+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:43:46.005316+0000","last_clean_scrub_stamp":"2024-03-23T09:43:46.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:22.192970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-23T09:43:47.395138+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:43:47.395138+0000","last_peered":"2024-03-23T09:43:47.395138+0000","last_clean":"2024-03-23T09:43:47.395138+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:43:47.395138+0000","last_undegraded":"2024-03-23T09:43:47.395138+0000","last_fullsized":"2024-03-23T09:43:47.395138+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1298432,"data_stored":877957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.48999999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69299999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40699999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.65900000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72699999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.64700000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.433,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.625},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.53600000000000003}]},{"osd":4,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.377,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72099999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.44400000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66500000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51700000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71499999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73799999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":692224,"data_stored":286810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.499,"15min":0.498},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69299999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.496},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.63900000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.40200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.42699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.47899999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.41299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.46100000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.625}]},{"osd":3,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52900000000000003,"15min":0.52300000000000002},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.73199999999999998,"5min":0.752,"15min":0.752},"last":0.51000000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.38700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.72399999999999998,"5min":0.747,"15min":0.747},"last":0.53200000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":704512,"data_stored":288081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48099999999999998,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.45600000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52900000000000003,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.434,"5min":0.376,"15min":0.376},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.38700000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.50700000000000001,"15min":0.49299999999999999},"min":{"1min":0.46700000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.46800000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.437}]},{"osd":4,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47999999999999998,"15min":0.47699999999999998},"min":{"1min":0.33300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.498},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.374,"15min":0.374},"max":{"1min":0.69999999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63800000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.34799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.749,"15min":0.749},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.51800000000000002},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70099999999999996,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.59699999999999998}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.44600000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71499999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.54100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":288993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.69899999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.624}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.46400000000000002},"min":{"1min":0.35699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.498,"15min":0.498},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51700000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.42999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.752,"5min":0.755,"15min":0.755},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.747,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47499999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70399999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.53000000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.30199999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68799999999999994,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.308,"5min":0.253,"15min":0.253},"max":{"1min":0.68400000000000005,"5min":1.008,"15min":1.008},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.34000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70099999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65800000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.70199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":704512,"data_stored":287485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.307,"15min":0.30499999999999999},"min":{"1min":0.25900000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30399999999999999,"15min":0.30199999999999999},"min":{"1min":0.25600000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.42899999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.36799999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.313,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.44400000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76500000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53100000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.621}]},{"osd":5,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.47399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81499999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.42899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.70999999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.64000000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.41799999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71499999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.65200000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.59999999999999998},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.73999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":704512,"data_stored":287485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.34000000000000002,"15min":0.34399999999999997},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.48799999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.25,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48799999999999999,"5min":0.872,"15min":0.872},"last":0.29499999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.629,"15min":0.629},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58599999999999997,"5min":0.746,"15min":0.746},"last":0.36499999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.47399999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.40200000000000002,"15min":0.40699999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55800000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.28100000000000003}]},{"osd":4,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72499999999999998,"5min":1.216,"15min":1.216},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.753,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.624}]},{"osd":5,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.624,"15min":0.63200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.71599999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75800000000000001,"5min":1.101,"15min":1.101},"last":0.54000000000000004}]},{"osd":6,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.70699999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.63800000000000001,"15min":0.64800000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.72699999999999998,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.70499999999999996}]},{"osd":7,"last update":"Sat Mar 23 09:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.76500000000000001,"5min":1.155,"15min":1.155},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72099999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":700416,"data_stored":287519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33600000000000002,"15min":0.33300000000000002},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.53300000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61899999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.23499999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48399999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.251},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.374,"15min":0.37},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.35199999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41099999999999998,"15min":0.40699999999999997},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70499999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.316},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60899999999999999,"5min":2.75,"15min":2.75},"last":0.32800000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.45500000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91100000000000003,"5min":2.778,"15min":2.778},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52600000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.46300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88100000000000001,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57599999999999996}]},{"osd":7,"last update":"Sat Mar 23 09:42:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65400000000000003,"15min":0.64700000000000002},"min":{"1min":0.51800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":0.92400000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.76000000000000001,"15min":0.77800000000000002},"min":{"1min":0.44600000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.85099999999999998,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.628}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1302528,"data_stored":879324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32200000000000001,"15min":0.31900000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.499,"5min":0.504,"15min":0.504},"last":0.44},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.34499999999999997},"min":{"1min":0.25900000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.51300000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.38300000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36099999999999999,"15min":0.35899999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36099999999999999},"min":{"1min":0.27000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.42499999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.379,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.441,"5min":0.42299999999999999,"15min":0.41999999999999998},"min":{"1min":0.32100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.72499999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.81499999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.69899999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83299999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.88400000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.76700000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.73699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:43:55.148 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T09:43:55.149 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-23T09:43:55.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.292+0000 7f3d8e20b640 1 Processor -- start 2024-03-23T09:43:55.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.292+0000 7f3d8e20b640 1 -- start start 2024-03-23T09:43:55.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.293+0000 7f3d8e20b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d8815c990 0x7f3d880d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:55.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.293+0000 7f3d8e20b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d880d2020 con 0x7f3d880cdca0 2024-03-23T09:43:55.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.293+0000 7f3d8e20b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d880d1410 con 0x7f3d880d24b0 2024-03-23T09:43:55.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.293+0000 7f3d8e20b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880d16f0 con 0x7f3d8815c990 2024-03-23T09:43:55.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.293+0000 7f3d87fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3d880cdca0 0x7f3d880b5a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44744/0 (socket says 172.21.15.47:44744) 2024-03-23T09:43:55.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.293+0000 7f3d87fff640 1 -- 172.21.15.47:0/826852574 learned_addr learned my addr 172.21.15.47:0/826852574 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:55.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.294+0000 7f3d877fe640 1 --2- 172.21.15.47:0/826852574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d8815c990 0x7f3d880d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:55.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.294+0000 7f3d867fc640 1 -- 172.21.15.47:0/826852574 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836621923 0 0) 0x7f3d880d2020 con 0x7f3d880cdca0 2024-03-23T09:43:55.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.294+0000 7f3d867fc640 1 -- 172.21.15.47:0/826852574 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d70003520 con 0x7f3d880cdca0 2024-03-23T09:43:55.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.294+0000 7f3d867fc640 1 -- 172.21.15.47:0/826852574 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494170629 0 0) 0x7f3d880d1410 con 0x7f3d880d24b0 2024-03-23T09:43:55.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.294+0000 7f3d867fc640 1 -- 172.21.15.47:0/826852574 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d880d2020 con 0x7f3d880d24b0 2024-03-23T09:43:55.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.294+0000 7f3d877fe640 1 -- 172.21.15.47:0/826852574 >> v1:172.21.15.175:6789/0 conn(0x7f3d880d24b0 legacy=0x7f3d880cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.294+0000 7f3d877fe640 1 -- 172.21.15.47:0/826852574 >> v1:172.21.15.47:6789/0 conn(0x7f3d880cdca0 legacy=0x7f3d880b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.294+0000 7f3d877fe640 1 -- 172.21.15.47:0/826852574 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d880d19d0 con 0x7f3d8815c990 2024-03-23T09:43:55.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.295+0000 7f3d877fe640 1 --2- 172.21.15.47:0/826852574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d8815c990 0x7f3d880d0970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3d780097d0 tx=0x7f3d7802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94090120e63099ee server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:55.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.295+0000 7f3d867fc640 1 -- 172.21.15.47:0/826852574 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78031890 con 0x7f3d8815c990 2024-03-23T09:43:55.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.295+0000 7f3d867fc640 1 -- 172.21.15.47:0/826852574 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d780319f0 con 0x7f3d8815c990 2024-03-23T09:43:55.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.295+0000 7f3d867fc640 1 -- 172.21.15.47:0/826852574 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78031d80 con 0x7f3d8815c990 2024-03-23T09:43:55.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.295+0000 7f3d8e20b640 1 -- 172.21.15.47:0/826852574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d8815c990 msgr2=0x7f3d880d0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.295+0000 7f3d8e20b640 1 --2- 172.21.15.47:0/826852574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d8815c990 0x7f3d880d0970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3d780097d0 tx=0x7f3d7802b930 comp rx=0 tx=0).stop 2024-03-23T09:43:55.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.296+0000 7f3d8e20b640 1 -- 172.21.15.47:0/826852574 shutdown_connections 2024-03-23T09:43:55.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.296+0000 7f3d8e20b640 1 --2- 172.21.15.47:0/826852574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d8815c990 0x7f3d880d0970 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.296+0000 7f3d8e20b640 1 -- 172.21.15.47:0/826852574 >> 172.21.15.47:0/826852574 conn(0x7f3d8806af20 msgr2=0x7f3d8806b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:55.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.296+0000 7f3d8e20b640 1 -- 172.21.15.47:0/826852574 shutdown_connections 2024-03-23T09:43:55.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.296+0000 7f3d8e20b640 1 -- 172.21.15.47:0/826852574 wait complete. 2024-03-23T09:43:55.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.297+0000 7f3d8e20b640 1 Processor -- start 2024-03-23T09:43:55.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.297+0000 7f3d8e20b640 1 -- start start 2024-03-23T09:43:55.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.297+0000 7f3d8e20b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d880cdca0 0x7f3d881f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:55.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.297+0000 7f3d8e20b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d880d24b0 0x7f3d88146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:55.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.297+0000 7f3d8e20b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8815c990 0x7f3d881ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:55.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d8e20b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d88146860 con 0x7f3d880cdca0 2024-03-23T09:43:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d8e20b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880b3c60 con 0x7f3d8815c990 2024-03-23T09:43:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d8e20b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d88143880 con 0x7f3d880d24b0 2024-03-23T09:43:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d877fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d880cdca0 0x7f3d881f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d877fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d880cdca0 0x7f3d881f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38026/0 (socket says 172.21.15.47:38026) 2024-03-23T09:43:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d877fe640 1 -- 172.21.15.47:0/3832430790 learned_addr learned my addr 172.21.15.47:0/3832430790 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d87fff640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8815c990 0x7f3d881ef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d86ffd640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d880d24b0 0x7f3d88146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d877fe640 1 -- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d880d24b0 msgr2=0x7f3d88146320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d877fe640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d880d24b0 0x7f3d88146320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d877fe640 1 -- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8815c990 msgr2=0x7f3d881ef4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d877fe640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8815c990 0x7f3d881ef4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d877fe640 1 -- 172.21.15.47:0/3832430790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d78009400 con 0x7f3d880cdca0 2024-03-23T09:43:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d877fe640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d880cdca0 0x7f3d881f5800 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f3d78034040 tx=0x7f3d780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d6ffff640 1 -- 172.21.15.47:0/3832430790 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7803bbf0 con 0x7f3d880cdca0 2024-03-23T09:43:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d6ffff640 1 -- 172.21.15.47:0/3832430790 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7803bd70 con 0x7f3d880cdca0 2024-03-23T09:43:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d8e20b640 1 -- 172.21.15.47:0/3832430790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d881f5de0 con 0x7f3d880cdca0 2024-03-23T09:43:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d6ffff640 1 -- 172.21.15.47:0/3832430790 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78045630 con 0x7f3d880cdca0 2024-03-23T09:43:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.298+0000 7f3d8e20b640 1 -- 172.21.15.47:0/3832430790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d881f63a0 con 0x7f3d880cdca0 2024-03-23T09:43:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.299+0000 7f3d8e20b640 1 -- 172.21.15.47:0/3832430790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d880b3dd0 con 0x7f3d880cdca0 2024-03-23T09:43:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.300+0000 7f3d6ffff640 1 -- 172.21.15.47:0/3832430790 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3d7804e450 con 0x7f3d880cdca0 2024-03-23T09:43:55.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.300+0000 7f3d6ffff640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3d7406ac00 0x7f3d7406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:55.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.300+0000 7f3d6ffff640 1 -- 172.21.15.47:0/3832430790 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d780ab7c0 con 0x7f3d880cdca0 2024-03-23T09:43:55.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.300+0000 7f3d86ffd640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3d7406ac00 0x7f3d7406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:55.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.301+0000 7f3d86ffd640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3d7406ac00 0x7f3d7406d0c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3d880cdbd0 tx=0x7f3d7c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:55.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.304+0000 7f3d6ffff640 1 -- 172.21.15.47:0/3832430790 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3d78076b70 con 0x7f3d880cdca0 2024-03-23T09:43:55.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.554+0000 7f3d8e20b640 1 -- 172.21.15.47:0/3832430790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3d880672f0 con 0x7f3d880cdca0 2024-03-23T09:43:55.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.557+0000 7f3d6ffff640 1 -- 172.21.15.47:0/3832430790 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3d780350e0 con 0x7f3d880cdca0 2024-03-23T09:43:55.557 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-03-23T09:43:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.559+0000 7f3d8e20b640 1 -- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3d7406ac00 msgr2=0x7f3d7406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.559+0000 7f3d8e20b640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3d7406ac00 0x7f3d7406d0c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3d880cdbd0 tx=0x7f3d7c009040 comp rx=0 tx=0).stop 2024-03-23T09:43:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.559+0000 7f3d8e20b640 1 -- 172.21.15.47:0/3832430790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d880cdca0 msgr2=0x7f3d881f5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.559+0000 7f3d8e20b640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d880cdca0 0x7f3d881f5800 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f3d78034040 tx=0x7f3d780046c0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.559+0000 7f3d8e20b640 1 -- 172.21.15.47:0/3832430790 shutdown_connections 2024-03-23T09:43:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.559+0000 7f3d8e20b640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3d7406ac00 0x7f3d7406d0c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.559+0000 7f3d8e20b640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8815c990 0x7f3d881ef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.559+0000 7f3d8e20b640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d880d24b0 0x7f3d88146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.559+0000 7f3d8e20b640 1 --2- 172.21.15.47:0/3832430790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d880cdca0 0x7f3d881f5800 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.559+0000 7f3d8e20b640 1 -- 172.21.15.47:0/3832430790 >> 172.21.15.47:0/3832430790 conn(0x7f3d8806af20 msgr2=0x7f3d880b6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.560+0000 7f3d8e20b640 1 -- 172.21.15.47:0/3832430790 shutdown_connections 2024-03-23T09:43:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.560+0000 7f3d8e20b640 1 -- 172.21.15.47:0/3832430790 wait complete. 2024-03-23T09:43:55.570 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:43:55.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.671+0000 7f9872096640 1 Processor -- start 2024-03-23T09:43:55.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.671+0000 7f9872096640 1 -- start start 2024-03-23T09:43:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.672+0000 7f9872096640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f986c15ca90 0x7f986c160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.672+0000 7f9872096640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986c0d11e0 con 0x7f986c0cde10 2024-03-23T09:43:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.672+0000 7f9872096640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986c160f60 con 0x7f986c0b8350 2024-03-23T09:43:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.672+0000 7f9872096640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f986c161240 con 0x7f986c15ca90 2024-03-23T09:43:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.672+0000 7f986b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f986c15ca90 0x7f986c160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.672+0000 7f986bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f986c0cde10 0x7f986c0d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44748/0 (socket says 172.21.15.47:44748) 2024-03-23T09:43:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.672+0000 7f986b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f986c15ca90 0x7f986c160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38042/0 (socket says 172.21.15.47:38042) 2024-03-23T09:43:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.672+0000 7f986bfff640 1 -- 172.21.15.47:0/2036042021 learned_addr learned my addr 172.21.15.47:0/2036042021 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.672+0000 7f986b7fe640 1 -- 172.21.15.47:0/2036042021 >> v1:172.21.15.175:6789/0 conn(0x7f986c0b8350 legacy=0x7f986c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.672+0000 7f986b7fe640 1 -- 172.21.15.47:0/2036042021 >> v1:172.21.15.47:6789/0 conn(0x7f986c0cde10 legacy=0x7f986c0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.672+0000 7f986b7fe640 1 -- 172.21.15.47:0/2036042021 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c161520 con 0x7f986c15ca90 2024-03-23T09:43:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.673+0000 7f986b7fe640 1 --2- 172.21.15.47:0/2036042021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c15ca90 0x7f986c160710 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f985c0097a0 tx=0x7f985c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b432bf640eb3cf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:55.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.673+0000 7f986a7fc640 1 -- 172.21.15.47:0/2036042021 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c031830 con 0x7f986c15ca90 2024-03-23T09:43:55.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.673+0000 7f986a7fc640 1 -- 172.21.15.47:0/2036042021 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985c031990 con 0x7f986c15ca90 2024-03-23T09:43:55.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.673+0000 7f986a7fc640 1 -- 172.21.15.47:0/2036042021 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c031d20 con 0x7f986c15ca90 2024-03-23T09:43:55.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.673+0000 7f9872096640 1 -- 172.21.15.47:0/2036042021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c15ca90 msgr2=0x7f986c160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.673+0000 7f9872096640 1 --2- 172.21.15.47:0/2036042021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c15ca90 0x7f986c160710 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f985c0097a0 tx=0x7f985c02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.673+0000 7f9872096640 1 -- 172.21.15.47:0/2036042021 shutdown_connections 2024-03-23T09:43:55.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.673+0000 7f9872096640 1 --2- 172.21.15.47:0/2036042021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c15ca90 0x7f986c160710 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.673+0000 7f9872096640 1 -- 172.21.15.47:0/2036042021 >> 172.21.15.47:0/2036042021 conn(0x7f986c0659d0 msgr2=0x7f986c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:55.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.674+0000 7f9872096640 1 -- 172.21.15.47:0/2036042021 shutdown_connections 2024-03-23T09:43:55.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.674+0000 7f9872096640 1 -- 172.21.15.47:0/2036042021 wait complete. 2024-03-23T09:43:55.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.675+0000 7f9872096640 1 Processor -- start 2024-03-23T09:43:55.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.675+0000 7f9872096640 1 -- start start 2024-03-23T09:43:55.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f9872096640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c0b8350 0x7f986c0cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:55.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f9872096640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986c0cde10 0x7f986c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:55.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f9872096640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f986c15ca90 0x7f986c0cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:55.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f9872096640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c0b28a0 con 0x7f986c0b8350 2024-03-23T09:43:55.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f9872096640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c152c70 con 0x7f986c15ca90 2024-03-23T09:43:55.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f9872096640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986c152f50 con 0x7f986c0cde10 2024-03-23T09:43:55.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f986b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c0b8350 0x7f986c0cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:55.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f986b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c0b8350 0x7f986c0cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38044/0 (socket says 172.21.15.47:38044) 2024-03-23T09:43:55.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f986b7fe640 1 -- 172.21.15.47:0/3751009003 learned_addr learned my addr 172.21.15.47:0/3751009003 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:55.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f986bfff640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f986c15ca90 0x7f986c0cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f986affd640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986c0cde10 0x7f986c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f986b7fe640 1 -- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986c0cde10 msgr2=0x7f986c0b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f986b7fe640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986c0cde10 0x7f986c0b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f986b7fe640 1 -- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f986c15ca90 msgr2=0x7f986c0cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f986b7fe640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f986c15ca90 0x7f986c0cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f986b7fe640 1 -- 172.21.15.47:0/3751009003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c009400 con 0x7f986c0b8350 2024-03-23T09:43:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f986b7fe640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c0b8350 0x7f986c0cc5e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f985c034040 tx=0x7f985c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f9863fff640 1 -- 172.21.15.47:0/3751009003 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c03aa60 con 0x7f986c0b8350 2024-03-23T09:43:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f9863fff640 1 -- 172.21.15.47:0/3751009003 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985c03abe0 con 0x7f986c0b8350 2024-03-23T09:43:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f9872096640 1 -- 172.21.15.47:0/3751009003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c0b2b40 con 0x7f986c0b8350 2024-03-23T09:43:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f9863fff640 1 -- 172.21.15.47:0/3751009003 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c044400 con 0x7f986c0b8350 2024-03-23T09:43:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.676+0000 7f9872096640 1 -- 172.21.15.47:0/3751009003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c0cb510 con 0x7f986c0b8350 2024-03-23T09:43:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.677+0000 7f9872096640 1 -- 172.21.15.47:0/3751009003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986c0b3190 con 0x7f986c0b8350 2024-03-23T09:43:55.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.678+0000 7f9863fff640 1 -- 172.21.15.47:0/3751009003 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f985c04d440 con 0x7f986c0b8350 2024-03-23T09:43:55.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.678+0000 7f9863fff640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f984c06ac00 0x7f984c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:55.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.678+0000 7f9863fff640 1 -- 172.21.15.47:0/3751009003 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f985c0ab620 con 0x7f986c0b8350 2024-03-23T09:43:55.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.678+0000 7f986affd640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f984c06ac00 0x7f984c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:55.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.679+0000 7f986affd640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f984c06ac00 0x7f984c06d0c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f986c1f6430 tx=0x7f9854009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:55.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.682+0000 7f9863fff640 1 -- 172.21.15.47:0/3751009003 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f985c076800 con 0x7f986c0b8350 2024-03-23T09:43:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.899+0000 7f9872096640 1 -- 172.21.15.47:0/3751009003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f986c072360 con 0x7f986c0b8350 2024-03-23T09:43:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.899+0000 7f9863fff640 1 -- 172.21.15.47:0/3751009003 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f985c079e20 con 0x7f986c0b8350 2024-03-23T09:43:55.899 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:55.899 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/375513546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":375513546},{"type":"v1","addr":"172.21.15.175:6837","nonce":375513546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/3650756747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3650756747},{"type":"v1","addr":"172.21.15.47:6833","nonce":3650756747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2893805543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2893805543},{"type":"v1","addr":"172.21.15.47:6835","nonce":2893805543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:43:47.389279+0000","modified":"2024-03-23T09:43:49.400979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/2498456351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2498456351},{"type":"v1","addr":"172.21.15.175:6836","nonce":2498456351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-23T09:43:55.900 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2024-03-23T09:43:55.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.901+0000 7f9872096640 1 -- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f984c06ac00 msgr2=0x7f984c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.901+0000 7f9872096640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f984c06ac00 0x7f984c06d0c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f986c1f6430 tx=0x7f9854009040 comp rx=0 tx=0).stop 2024-03-23T09:43:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.901+0000 7f9872096640 1 -- 172.21.15.47:0/3751009003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c0b8350 msgr2=0x7f986c0cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.901+0000 7f9872096640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c0b8350 0x7f986c0cc5e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f985c034040 tx=0x7f985c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.901+0000 7f9872096640 1 -- 172.21.15.47:0/3751009003 shutdown_connections 2024-03-23T09:43:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.901+0000 7f9872096640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f984c06ac00 0x7f984c06d0c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.901+0000 7f9872096640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f986c15ca90 0x7f986c0cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.901+0000 7f9872096640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986c0cde10 0x7f986c0b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.901+0000 7f9872096640 1 --2- 172.21.15.47:0/3751009003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c0b8350 0x7f986c0cc5e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.901+0000 7f9872096640 1 -- 172.21.15.47:0/3751009003 >> 172.21.15.47:0/3751009003 conn(0x7f986c0659d0 msgr2=0x7f986c15e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.901+0000 7f9872096640 1 -- 172.21.15.47:0/3751009003 shutdown_connections 2024-03-23T09:43:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:55.902+0000 7f9872096640 1 -- 172.21.15.47:0/3751009003 wait complete. 2024-03-23T09:43:56.314 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:43:47.389279+0000', 'modified': '2024-03-23T09:43:49.400979+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6171}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6171': {'gid': 6171, 'name': 'd', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6836/2498456351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 2498456351}, {'type': 'v1', 'addr': '172.21.15.175:6836', 'nonce': 2498456351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:43:56.314 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:43:56.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-23T09:43:56.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.419+0000 7f517422c640 1 Processor -- start 2024-03-23T09:43:56.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.419+0000 7f517422c640 1 -- start start 2024-03-23T09:43:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f517422c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c098a00 0x7f516c098de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f517422c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f516c160650 0x7f516c06adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f517422c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516c099320 con 0x7f516c0cee60 2024-03-23T09:43:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f517422c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516c06b6f0 con 0x7f516c160650 2024-03-23T09:43:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f517422c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c06b9d0 con 0x7f516c098a00 2024-03-23T09:43:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f51717c7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f516c160650 0x7f516c06adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f51727c9640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f516c0cee60 0x7f516c0cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44750/0 (socket says 172.21.15.47:44750) 2024-03-23T09:43:56.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f51727c9640 1 -- 172.21.15.47:0/1362479310 learned_addr learned my addr 172.21.15.47:0/1362479310 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:56.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f5171fc8640 1 --2- 172.21.15.47:0/1362479310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c098a00 0x7f516c098de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f51717c7640 1 -- 172.21.15.47:0/1362479310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c098a00 msgr2=0x7f516c098de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:56.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f51717c7640 1 --2- 172.21.15.47:0/1362479310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c098a00 0x7f516c098de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f51717c7640 1 -- 172.21.15.47:0/1362479310 >> v1:172.21.15.47:6789/0 conn(0x7f516c0cee60 legacy=0x7f516c0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f51717c7640 1 -- 172.21.15.47:0/1362479310 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c06bcb0 con 0x7f516c160650 2024-03-23T09:43:56.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.420+0000 7f5170fc6640 1 -- 172.21.15.47:0/1362479310 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680469264 0 0) 0x7f516c099320 con 0x7f516c0cee60 2024-03-23T09:43:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.421+0000 7f51717c7640 1 --2- 172.21.15.47:0/1362479310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c160650 0x7f516c06adf0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f515c0097a0 tx=0x7f515c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2718a0c26762ab20 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.421+0000 7f5170fc6640 1 -- 172.21.15.47:0/1362479310 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c02e070 con 0x7f516c160650 2024-03-23T09:43:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.421+0000 7f5170fc6640 1 -- 172.21.15.47:0/1362479310 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c0315a0 con 0x7f516c160650 2024-03-23T09:43:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.421+0000 7f5170fc6640 1 -- 172.21.15.47:0/1362479310 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c031930 con 0x7f516c160650 2024-03-23T09:43:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.422+0000 7f517422c640 1 -- 172.21.15.47:0/1362479310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c160650 msgr2=0x7f516c06adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.422+0000 7f517422c640 1 --2- 172.21.15.47:0/1362479310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c160650 0x7f516c06adf0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f515c0097a0 tx=0x7f515c02b840 comp rx=0 tx=0).stop 2024-03-23T09:43:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.422+0000 7f517422c640 1 -- 172.21.15.47:0/1362479310 shutdown_connections 2024-03-23T09:43:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.422+0000 7f517422c640 1 --2- 172.21.15.47:0/1362479310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c160650 0x7f516c06adf0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.422+0000 7f517422c640 1 --2- 172.21.15.47:0/1362479310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c098a00 0x7f516c098de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.422+0000 7f517422c640 1 -- 172.21.15.47:0/1362479310 >> 172.21.15.47:0/1362479310 conn(0x7f516c0b2200 msgr2=0x7f516c0b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.422+0000 7f517422c640 1 -- 172.21.15.47:0/1362479310 shutdown_connections 2024-03-23T09:43:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.422+0000 7f517422c640 1 -- 172.21.15.47:0/1362479310 wait complete. 2024-03-23T09:43:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.423+0000 7f517422c640 1 Processor -- start 2024-03-23T09:43:56.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.423+0000 7f517422c640 1 -- start start 2024-03-23T09:43:56.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f517422c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c098a00 0x7f516c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f517422c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c0cee60 0x7f516c15cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f517422c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c160650 0x7f516c15d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f517422c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c1e8770 con 0x7f516c160650 2024-03-23T09:43:56.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f517422c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c1e7b60 con 0x7f516c0cee60 2024-03-23T09:43:56.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f517422c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c1e7e40 con 0x7f516c098a00 2024-03-23T09:43:56.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51727c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c160650 0x7f516c15d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51727c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c160650 0x7f516c15d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38072/0 (socket says 172.21.15.47:38072) 2024-03-23T09:43:56.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51727c9640 1 -- 172.21.15.47:0/989052237 learned_addr learned my addr 172.21.15.47:0/989052237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:56.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f5171fc8640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c098a00 0x7f516c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51717c7640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c0cee60 0x7f516c15cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51727c9640 1 -- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c098a00 msgr2=0x7f516c15b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51727c9640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c098a00 0x7f516c15b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51727c9640 1 -- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c0cee60 msgr2=0x7f516c15cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51727c9640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c0cee60 0x7f516c15cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51727c9640 1 -- 172.21.15.47:0/989052237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c009400 con 0x7f516c160650 2024-03-23T09:43:56.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51717c7640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c0cee60 0x7f516c15cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:56.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51727c9640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c160650 0x7f516c15d290 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f516400c700 tx=0x7f516400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:56.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51627fc640 1 -- 172.21.15.47:0/989052237 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516400ce10 con 0x7f516c160650 2024-03-23T09:43:56.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51627fc640 1 -- 172.21.15.47:0/989052237 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516400a640 con 0x7f516c160650 2024-03-23T09:43:56.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f517422c640 1 -- 172.21.15.47:0/989052237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516c1e8120 con 0x7f516c160650 2024-03-23T09:43:56.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f51627fc640 1 -- 172.21.15.47:0/989052237 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516400aa70 con 0x7f516c160650 2024-03-23T09:43:56.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.424+0000 7f517422c640 1 -- 172.21.15.47:0/989052237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c15bd50 con 0x7f516c160650 2024-03-23T09:43:56.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.426+0000 7f51627fc640 1 -- 172.21.15.47:0/989052237 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f516400abd0 con 0x7f516c160650 2024-03-23T09:43:56.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.426+0000 7f51627fc640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515006ac00 0x7f515006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.426+0000 7f51627fc640 1 -- 172.21.15.47:0/989052237 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5164088730 con 0x7f516c160650 2024-03-23T09:43:56.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.427+0000 7f5171fc8640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515006ac00 0x7f515006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.427+0000 7f5171fc8640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515006ac00 0x7f515006d0c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f516c0cbd60 tx=0x7f5168008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:56.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.428+0000 7f5176da6740 1 Processor -- start 2024-03-23T09:43:56.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.428+0000 7f5176da6740 1 -- start start 2024-03-23T09:43:56.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.428+0000 7f5176da6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c1ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.428+0000 7f5176da6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c1f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.428+0000 7f5176da6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c19c70 0x556864c49a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.428+0000 7f5176da6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556864b8a9b0 con 0x556864c1d7d0 2024-03-23T09:43:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.428+0000 7f5176da6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556864b88140 con 0x556864c67750 2024-03-23T09:43:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.428+0000 7f5176da6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556864b83a30 con 0x556864c19c70 2024-03-23T09:43:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.428+0000 7f51717c7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c1f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.428+0000 7f51717c7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c1f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38078/0 (socket says 172.21.15.47:38078) 2024-03-23T09:43:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.428+0000 7f51717c7640 1 -- 172.21.15.47:0/3944537810 learned_addr learned my addr 172.21.15.47:0/3944537810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.429+0000 7f5171fc8640 1 --2- 172.21.15.47:0/3944537810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c1ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.429+0000 7f51727c9640 1 --2- 172.21.15.47:0/3944537810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c19c70 0x556864c49a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.429+0000 7f51717c7640 1 -- 172.21.15.47:0/3944537810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c19c70 msgr2=0x556864c49a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.429+0000 7f51717c7640 1 --2- 172.21.15.47:0/3944537810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c19c70 0x556864c49a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.429+0000 7f51717c7640 1 -- 172.21.15.47:0/3944537810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 msgr2=0x556864c1ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.429+0000 7f51717c7640 1 --2- 172.21.15.47:0/3944537810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c1ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.429+0000 7f51717c7640 1 -- 172.21.15.47:0/3944537810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556864b102f0 con 0x556864c1d7d0 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.429+0000 7f51717c7640 1 --2- 172.21.15.47:0/3944537810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c1f2c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f515c034040 tx=0x7f515c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be826293ca250b63 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.429+0000 7f514f7fe640 1 -- 172.21.15.47:0/3944537810 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c041070 con 0x556864c1d7d0 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.429+0000 7f514f7fe640 1 -- 172.21.15.47:0/3944537810 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c004330 con 0x556864c1d7d0 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.429+0000 7f514f7fe640 1 -- 172.21.15.47:0/3944537810 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c031930 con 0x556864c1d7d0 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.430+0000 7f5176da6740 1 -- 172.21.15.47:0/3944537810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 msgr2=0x556864c1f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.430+0000 7f5176da6740 1 --2- 172.21.15.47:0/3944537810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c1f2c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f515c034040 tx=0x7f515c030000 comp rx=0 tx=0).stop 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.430+0000 7f5176da6740 1 -- 172.21.15.47:0/3944537810 shutdown_connections 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.430+0000 7f5176da6740 1 --2- 172.21.15.47:0/3944537810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c19c70 0x556864c49a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.430+0000 7f5176da6740 1 --2- 172.21.15.47:0/3944537810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c1f2c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.430+0000 7f5176da6740 1 --2- 172.21.15.47:0/3944537810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c1ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.430+0000 7f5176da6740 1 -- 172.21.15.47:0/3944537810 >> 172.21.15.47:0/3944537810 conn(0x556864c1bee0 msgr2=0x556864c64a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.430+0000 7f5176da6740 1 -- 172.21.15.47:0/3944537810 shutdown_connections 2024-03-23T09:43:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.430+0000 7f5176da6740 1 -- 172.21.15.47:0/3944537810 wait complete. 2024-03-23T09:43:56.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.430+0000 7f5176da6740 1 -- start start 2024-03-23T09:43:56.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.430+0000 7f5176da6740 1 Processor -- start 2024-03-23T09:43:56.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.431+0000 7f5176da6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c41900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.431+0000 7f5176da6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c40380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.431+0000 7f5176da6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c6cd10 0x556864c69130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.431+0000 7f5176da6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556864b42090 con 0x556864c1d7d0 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.431+0000 7f5176da6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556864b41d30 con 0x556864c67750 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.431+0000 7f5176da6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556864b072a0 con 0x556864c6cd10 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.431+0000 7f5176da6740 10 client.0 _ll_register_callbacks cb 0x556864b3d050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.431+0000 7f5171fc8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c41900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.431+0000 7f5171fc8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c41900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38086/0 (socket says 172.21.15.47:38086) 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.431+0000 7f5171fc8640 1 -- 172.21.15.47:0/3845495073 learned_addr learned my addr 172.21.15.47:0/3845495073 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.431+0000 7f51717c7640 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c40380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f51727c9640 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c6cd10 0x556864c69130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f5171fc8640 1 -- 172.21.15.47:0/3845495073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c6cd10 msgr2=0x556864c69130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f5171fc8640 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c6cd10 0x556864c69130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f5171fc8640 1 -- 172.21.15.47:0/3845495073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 msgr2=0x556864c40380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f5171fc8640 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c40380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f5171fc8640 1 -- 172.21.15.47:0/3845495073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c06bcb0 con 0x556864c1d7d0 2024-03-23T09:43:56.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f51727c9640 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c6cd10 0x556864c69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f5171fc8640 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c41900 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f51680167d0 tx=0x7f51680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ac23a62200ee07 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f514cff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f514cff9640 1 -- 172.21.15.47:0/3845495073 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51680149b0 con 0x556864c1d7d0 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f5176da6740 1 -- 172.21.15.47:0/3845495073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568648773c0 con 0x556864c1d7d0 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f514cff9640 1 -- 172.21.15.47:0/3845495073 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5168014b10 con 0x556864c1d7d0 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f514cff9640 1 -- 172.21.15.47:0/3845495073 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5168014cd0 con 0x556864c1d7d0 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f5176da6740 10 client.6351 fetch_fsmap learned FSMap version 24 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f5176da6740 1 -- 172.21.15.47:0/3845495073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x556864b820b0 con 0x556864c1d7d0 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f514cff9640 1 -- 172.21.15.47:0/3845495073 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5568648773c0 con 0x556864c1d7d0 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f514cff9640 1 -- 172.21.15.47:0/3845495073 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f516802b440 con 0x556864c1d7d0 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.432+0000 7f5176da6740 10 client.6351 fetch_fsmap finished waiting for FSMap version 24 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.433+0000 7f5176da6740 10 client.6351 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.433+0000 7f5176da6740 20 client.6351 populate_metadata read hostname 'smithi047' 2024-03-23T09:43:56.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.433+0000 7f5176da6740 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556864c45420 0x556864c6a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.433+0000 7f5176da6740 4 client.6351 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.433+0000 7f5176da6740 1 -- 172.21.15.47:0/3845495073 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568648886c0 con 0x556864c45420 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.433+0000 7f51717c7640 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556864c45420 0x556864c6a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.433+0000 7f514cff9640 10 client.6351 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.433+0000 7f51717c7640 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556864c45420 0x556864c6a6e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7719300a50df04f2 server_cookie=d6a9174413cbf255 in_seq=0 out_seq=0 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f514cff9640 1 -- 172.21.15.47:0/3845495073 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5568648886c0 con 0x556864c45420 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f514cff9640 10 client.6351 handle_command_reply: tid=0 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f5176da6740 1 client.6351 shutdown 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f5176da6740 1 -- 172.21.15.47:0/3845495073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 msgr2=0x556864c41900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f5176da6740 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c41900 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f51680167d0 tx=0x7f51680046c0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f5176da6740 1 -- 172.21.15.47:0/3845495073 shutdown_connections 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f5176da6740 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556864c45420 0x556864c6a6e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f5176da6740 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c6cd10 0x556864c69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f514cff9640 0 client.6351 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:43:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f5176da6740 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c40380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f5176da6740 1 --2- 172.21.15.47:0/3845495073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c41900 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f5176da6740 1 -- 172.21.15.47:0/3845495073 >> 172.21.15.47:0/3845495073 conn(0x556864c19c70 msgr2=0x556864c192b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:56.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f5176da6740 1 -- 172.21.15.47:0/3845495073 shutdown_connections 2024-03-23T09:43:56.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f5176da6740 1 -- 172.21.15.47:0/3845495073 wait complete. 2024-03-23T09:43:56.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.434+0000 7f5176da6740 20 client.6351 trim_cache size 0 max 16384 2024-03-23T09:43:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.449+0000 7f5176da6740 1 Processor -- start 2024-03-23T09:43:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.449+0000 7f5176da6740 1 -- start start 2024-03-23T09:43:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5176da6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c65170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5176da6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c86c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5176da6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c6cd10 0x556864c19100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5176da6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556864c14600 con 0x556864c1d7d0 2024-03-23T09:43:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5176da6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556864c1f900 con 0x556864c67750 2024-03-23T09:43:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5176da6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556864bb8220 con 0x556864c6cd10 2024-03-23T09:43:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5171fc8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c65170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5171fc8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c65170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38098/0 (socket says 172.21.15.47:38098) 2024-03-23T09:43:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5171fc8640 1 -- 172.21.15.47:0/1216149958 learned_addr learned my addr 172.21.15.47:0/1216149958 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5171fc8640 1 -- 172.21.15.47:0/1216149958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c6cd10 msgr2=0x556864c19100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5171fc8640 1 --2- 172.21.15.47:0/1216149958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c6cd10 0x556864c19100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5171fc8640 1 -- 172.21.15.47:0/1216149958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 msgr2=0x556864c86c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5171fc8640 1 --2- 172.21.15.47:0/1216149958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c86c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5171fc8640 1 -- 172.21.15.47:0/1216149958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556864b24430 con 0x556864c1d7d0 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.450+0000 7f5171fc8640 1 --2- 172.21.15.47:0/1216149958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c65170 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5168010d70 tx=0x7f5168023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b977de9df3c436 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.451+0000 7f514ffff640 1 -- 172.21.15.47:0/1216149958 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168006a20 con 0x556864c1d7d0 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.451+0000 7f514ffff640 1 -- 172.21.15.47:0/1216149958 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51680076d0 con 0x556864c1d7d0 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.451+0000 7f514ffff640 1 -- 172.21.15.47:0/1216149958 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516802b9a0 con 0x556864c1d7d0 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.451+0000 7f5176da6740 1 -- 172.21.15.47:0/1216149958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 msgr2=0x556864c65170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.451+0000 7f5176da6740 1 --2- 172.21.15.47:0/1216149958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c65170 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5168010d70 tx=0x7f5168023000 comp rx=0 tx=0).stop 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.451+0000 7f5176da6740 1 -- 172.21.15.47:0/1216149958 shutdown_connections 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.451+0000 7f5176da6740 1 --2- 172.21.15.47:0/1216149958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c6cd10 0x556864c19100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.451+0000 7f5176da6740 1 --2- 172.21.15.47:0/1216149958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c86c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.451+0000 7f5176da6740 1 --2- 172.21.15.47:0/1216149958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c1d7d0 0x556864c65170 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.451+0000 7f5176da6740 1 -- 172.21.15.47:0/1216149958 >> 172.21.15.47:0/1216149958 conn(0x556864c19c70 msgr2=0x556864c6c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.451+0000 7f5176da6740 1 -- 172.21.15.47:0/1216149958 shutdown_connections 2024-03-23T09:43:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.451+0000 7f5176da6740 1 -- 172.21.15.47:0/1216149958 wait complete. 2024-03-23T09:43:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.452+0000 7f5176da6740 1 -- start start 2024-03-23T09:43:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.452+0000 7f5176da6740 1 Processor -- start 2024-03-23T09:43:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f5176da6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c1d7d0 0x556864cc4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f5176da6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c62430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f5176da6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c6cd10 0x556864ccb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f5176da6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556864c196a0 con 0x556864c6cd10 2024-03-23T09:43:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f5176da6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556864c62ee0 con 0x556864c67750 2024-03-23T09:43:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f5176da6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556864c19810 con 0x556864c1d7d0 2024-03-23T09:43:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f5176da6740 10 client.0 _ll_register_callbacks cb 0x7f515c009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:43:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f51727c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c6cd10 0x556864ccb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f51727c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c6cd10 0x556864ccb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38114/0 (socket says 172.21.15.47:38114) 2024-03-23T09:43:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f51727c9640 1 -- 172.21.15.47:0/3874670203 learned_addr learned my addr 172.21.15.47:0/3874670203 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f51717c7640 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c62430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f5171fc8640 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c1d7d0 0x556864cc4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f51727c9640 1 -- 172.21.15.47:0/3874670203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c1d7d0 msgr2=0x556864cc4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f51727c9640 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c1d7d0 0x556864cc4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f51727c9640 1 -- 172.21.15.47:0/3874670203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 msgr2=0x556864c62430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f51727c9640 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c62430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f51727c9640 1 -- 172.21.15.47:0/3874670203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556864b820b0 con 0x556864c6cd10 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f51717c7640 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c62430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f51727c9640 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c6cd10 0x556864ccb1c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5164086d10 tx=0x7f5164057130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52626ba1ad881c49 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f514cff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f514cff9640 1 -- 172.21.15.47:0/3874670203 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516406f070 con 0x556864c6cd10 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f514cff9640 1 -- 172.21.15.47:0/3874670203 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f516405fa50 con 0x556864c6cd10 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f5176da6740 1 -- 172.21.15.47:0/3874670203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f515c004330 con 0x556864c6cd10 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f514cff9640 1 -- 172.21.15.47:0/3874670203 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f516405b2c0 con 0x556864c6cd10 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f5176da6740 10 client.6357 fetch_fsmap learned FSMap version 24 2024-03-23T09:43:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.453+0000 7f5176da6740 1 -- 172.21.15.47:0/3874670203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x556864b49b40 con 0x556864c6cd10 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.454+0000 7f514cff9640 1 -- 172.21.15.47:0/3874670203 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c004330 con 0x556864c6cd10 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.454+0000 7f514cff9640 1 -- 172.21.15.47:0/3874670203 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f516405f5c0 con 0x556864c6cd10 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.454+0000 7f5176da6740 10 client.6357 fetch_fsmap finished waiting for FSMap version 24 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.454+0000 7f5176da6740 10 client.6357 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.454+0000 7f5176da6740 20 client.6357 populate_metadata read hostname 'smithi047' 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.454+0000 7f5176da6740 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556864c7d590 0x556864c7d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.454+0000 7f5176da6740 4 client.6357 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.454+0000 7f5176da6740 1 -- 172.21.15.47:0/3874670203 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5568648dbd10 con 0x556864c7d590 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.454+0000 7f5171fc8640 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556864c7d590 0x556864c7d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.454+0000 7f514cff9640 10 client.6357 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.454+0000 7f5171fc8640 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556864c7d590 0x556864c7d970 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45c2859cb4c7e36 server_cookie=385ba1f5ae5e7038 in_seq=0 out_seq=0 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f514cff9640 1 -- 172.21.15.47:0/3874670203 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5568648dbd10 con 0x556864c7d590 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f514cff9640 10 client.6357 handle_command_reply: tid=0 2024-03-23T09:43:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f5176da6740 1 client.6357 shutdown 2024-03-23T09:43:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f5176da6740 1 -- 172.21.15.47:0/3874670203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c6cd10 msgr2=0x556864ccb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f5176da6740 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c6cd10 0x556864ccb1c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5164086d10 tx=0x7f5164057130 comp rx=0 tx=0).stop 2024-03-23T09:43:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f5176da6740 1 -- 172.21.15.47:0/3874670203 shutdown_connections 2024-03-23T09:43:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f5176da6740 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556864c7d590 0x556864c7d970 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f514cff9640 0 client.6357 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:43:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f5176da6740 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556864c6cd10 0x556864ccb1c0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f5176da6740 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556864c67750 0x556864c62430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f5176da6740 1 --2- 172.21.15.47:0/3874670203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556864c1d7d0 0x556864cc4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f5176da6740 1 -- 172.21.15.47:0/3874670203 >> 172.21.15.47:0/3874670203 conn(0x556864c19c70 msgr2=0x556864c6c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f5176da6740 1 -- 172.21.15.47:0/3874670203 shutdown_connections 2024-03-23T09:43:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f5176da6740 1 -- 172.21.15.47:0/3874670203 wait complete. 2024-03-23T09:43:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f5176da6740 20 client.6357 trim_cache size 0 max 16384 2024-03-23T09:43:56.461 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:43:56.461 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:43:56.461 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:43:56.461 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 6171, 2024-03-23T09:43:56.461 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:43:56.461 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:43:56.461 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:43:56.461 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 9.0645474440000005, 2024-03-23T09:43:56.461 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 24, 2024-03-23T09:43:56.461 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 58, 2024-03-23T09:43:56.461 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 58, 2024-03-23T09:43:56.461 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 13.482974884000001 2024-03-23T09:43:56.461 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:43:56.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f517422c640 1 -- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515006ac00 msgr2=0x7f515006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f517422c640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515006ac00 0x7f515006d0c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f516c0cbd60 tx=0x7f5168008040 comp rx=0 tx=0).stop 2024-03-23T09:43:56.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f517422c640 1 -- 172.21.15.47:0/989052237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c160650 msgr2=0x7f516c15d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f517422c640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c160650 0x7f516c15d290 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f516400c700 tx=0x7f516400cbc0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f517422c640 1 -- 172.21.15.47:0/989052237 shutdown_connections 2024-03-23T09:43:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f517422c640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515006ac00 0x7f515006d0c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f517422c640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c160650 0x7f516c15d290 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f517422c640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c0cee60 0x7f516c15cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f517422c640 1 --2- 172.21.15.47:0/989052237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c098a00 0x7f516c15b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f517422c640 1 -- 172.21.15.47:0/989052237 >> 172.21.15.47:0/989052237 conn(0x7f516c0b2200 msgr2=0x7f516c15e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f517422c640 1 -- 172.21.15.47:0/989052237 shutdown_connections 2024-03-23T09:43:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.455+0000 7f517422c640 1 -- 172.21.15.47:0/989052237 wait complete. 2024-03-23T09:43:56.466 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:43:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.566+0000 7f01988b7640 1 Processor -- start 2024-03-23T09:43:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.566+0000 7f01988b7640 1 -- start start 2024-03-23T09:43:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.567+0000 7f01988b7640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f019406f210 0x7f019406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.567+0000 7f01988b7640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01940cf2f0 con 0x7f01940b28e0 2024-03-23T09:43:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.567+0000 7f01988b7640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019406fb30 con 0x7f01940ce780 2024-03-23T09:43:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.567+0000 7f01988b7640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01940d01c0 con 0x7f019406f210 2024-03-23T09:43:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.567+0000 7f0192d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f01940b28e0 0x7f01940b2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44752/0 (socket says 172.21.15.47:44752) 2024-03-23T09:43:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.567+0000 7f0192d9d640 1 -- 172.21.15.47:0/3402198234 learned_addr learned my addr 172.21.15.47:0/3402198234 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.567+0000 7f019259c640 1 --2- 172.21.15.47:0/3402198234 >> v2:172.21.15.175:3300/0 conn(0x7f019406f210 0x7f019406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.568+0000 7f019159a640 1 -- 172.21.15.47:0/3402198234 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499548083 0 0) 0x7f01940cf2f0 con 0x7f01940b28e0 2024-03-23T09:43:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.568+0000 7f019159a640 1 -- 172.21.15.47:0/3402198234 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0180003520 con 0x7f01940b28e0 2024-03-23T09:43:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.568+0000 7f019259c640 1 -- 172.21.15.47:0/3402198234 >> v1:172.21.15.175:6789/0 conn(0x7f01940ce780 legacy=0x7f01940cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.568+0000 7f019259c640 1 -- 172.21.15.47:0/3402198234 >> v1:172.21.15.47:6789/0 conn(0x7f01940b28e0 legacy=0x7f01940b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.568+0000 7f019259c640 1 -- 172.21.15.47:0/3402198234 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01940cf540 con 0x7f019406f210 2024-03-23T09:43:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.569+0000 7f019259c640 1 --2- 172.21.15.47:0/3402198234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f019406f210 0x7f019406f5f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0184009d90 tx=0x7f018402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cb33ef5e5096a62 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.569+0000 7f019159a640 1 -- 172.21.15.47:0/3402198234 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018402cc60 con 0x7f019406f210 2024-03-23T09:43:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.569+0000 7f019159a640 1 -- 172.21.15.47:0/3402198234 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018402cdc0 con 0x7f019406f210 2024-03-23T09:43:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.569+0000 7f019159a640 1 -- 172.21.15.47:0/3402198234 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184034660 con 0x7f019406f210 2024-03-23T09:43:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.569+0000 7f01988b7640 1 -- 172.21.15.47:0/3402198234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f019406f210 msgr2=0x7f019406f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.569+0000 7f01988b7640 1 --2- 172.21.15.47:0/3402198234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f019406f210 0x7f019406f5f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0184009d90 tx=0x7f018402beb0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.569+0000 7f01988b7640 1 -- 172.21.15.47:0/3402198234 shutdown_connections 2024-03-23T09:43:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.569+0000 7f01988b7640 1 --2- 172.21.15.47:0/3402198234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f019406f210 0x7f019406f5f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.569+0000 7f01988b7640 1 -- 172.21.15.47:0/3402198234 >> 172.21.15.47:0/3402198234 conn(0x7f0194071400 msgr2=0x7f0194071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.570+0000 7f01988b7640 1 -- 172.21.15.47:0/3402198234 shutdown_connections 2024-03-23T09:43:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.570+0000 7f01988b7640 1 -- 172.21.15.47:0/3402198234 wait complete. 2024-03-23T09:43:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.571+0000 7f01988b7640 1 Processor -- start 2024-03-23T09:43:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.571+0000 7f01988b7640 1 -- start start 2024-03-23T09:43:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.571+0000 7f01988b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f019406f210 0x7f019414a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f01988b7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01940b28e0 0x7f01940cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f01988b7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01940ce780 0x7f01940b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f01988b7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0194155b40 con 0x7f019406f210 2024-03-23T09:43:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f01988b7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01940ca7f0 con 0x7f01940b28e0 2024-03-23T09:43:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f01988b7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01940caad0 con 0x7f01940ce780 2024-03-23T09:43:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f019259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f019406f210 0x7f019414a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f019259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f019406f210 0x7f019414a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38120/0 (socket says 172.21.15.47:38120) 2024-03-23T09:43:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f019259c640 1 -- 172.21.15.47:0/942451593 learned_addr learned my addr 172.21.15.47:0/942451593 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f0192d9d640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01940ce780 0x7f01940b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f0191d9b640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01940b28e0 0x7f01940cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f019259c640 1 -- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01940ce780 msgr2=0x7f01940b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f019259c640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01940ce780 0x7f01940b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f019259c640 1 -- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01940b28e0 msgr2=0x7f01940cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f019259c640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01940b28e0 0x7f01940cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f019259c640 1 -- 172.21.15.47:0/942451593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0184009a70 con 0x7f019406f210 2024-03-23T09:43:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f0191d9b640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01940b28e0 0x7f01940cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f019259c640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f019406f210 0x7f019414a0a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0184009d90 tx=0x7f0184004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f018affd640 1 -- 172.21.15.47:0/942451593 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184040650 con 0x7f019406f210 2024-03-23T09:43:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f018affd640 1 -- 172.21.15.47:0/942451593 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01840407b0 con 0x7f019406f210 2024-03-23T09:43:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f01988b7640 1 -- 172.21.15.47:0/942451593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01940cadb0 con 0x7f019406f210 2024-03-23T09:43:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f018affd640 1 -- 172.21.15.47:0/942451593 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184040b60 con 0x7f019406f210 2024-03-23T09:43:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.572+0000 7f01988b7640 1 -- 172.21.15.47:0/942451593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01940cb370 con 0x7f019406f210 2024-03-23T09:43:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.573+0000 7f01988b7640 1 -- 172.21.15.47:0/942451593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01940672f0 con 0x7f019406f210 2024-03-23T09:43:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.574+0000 7f018affd640 1 -- 172.21.15.47:0/942451593 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0184040cc0 con 0x7f019406f210 2024-03-23T09:43:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.574+0000 7f018affd640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f017406ac00 0x7f017406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.574+0000 7f018affd640 1 -- 172.21.15.47:0/942451593 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01840ac340 con 0x7f019406f210 2024-03-23T09:43:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.574+0000 7f0191d9b640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f017406ac00 0x7f017406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.575+0000 7f0191d9b640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f017406ac00 0x7f017406d0c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f017c007fd0 tx=0x7f017c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.578+0000 7f018affd640 1 -- 172.21.15.47:0/942451593 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f01840776d0 con 0x7f019406f210 2024-03-23T09:43:56.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.794+0000 7f01988b7640 1 -- 172.21.15.47:0/942451593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f01940cb560 con 0x7f019406f210 2024-03-23T09:43:56.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.795+0000 7f018affd640 1 -- 172.21.15.47:0/942451593 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f018407acf0 con 0x7f019406f210 2024-03-23T09:43:56.795 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:43:56.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.797+0000 7f01988b7640 1 -- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f017406ac00 msgr2=0x7f017406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.797+0000 7f01988b7640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f017406ac00 0x7f017406d0c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f017c007fd0 tx=0x7f017c009040 comp rx=0 tx=0).stop 2024-03-23T09:43:56.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.797+0000 7f01988b7640 1 -- 172.21.15.47:0/942451593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f019406f210 msgr2=0x7f019414a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:56.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.797+0000 7f01988b7640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f019406f210 0x7f019414a0a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0184009d90 tx=0x7f0184004a40 comp rx=0 tx=0).stop 2024-03-23T09:43:56.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.797+0000 7f01988b7640 1 -- 172.21.15.47:0/942451593 shutdown_connections 2024-03-23T09:43:56.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.797+0000 7f01988b7640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f017406ac00 0x7f017406d0c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.797+0000 7f01988b7640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01940ce780 0x7f01940b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.797+0000 7f01988b7640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01940b28e0 0x7f01940cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.797+0000 7f01988b7640 1 --2- 172.21.15.47:0/942451593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f019406f210 0x7f019414a0a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:56.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.797+0000 7f01988b7640 1 -- 172.21.15.47:0/942451593 >> 172.21.15.47:0/942451593 conn(0x7f0194071400 msgr2=0x7f01940b4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:56.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.797+0000 7f01988b7640 1 -- 172.21.15.47:0/942451593 shutdown_connections 2024-03-23T09:43:56.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:56.797+0000 7f01988b7640 1 -- 172.21.15.47:0/942451593 wait complete. 2024-03-23T09:43:56.807 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:43:56.807 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:43:56.807 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:43:56.807 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:43:56.807 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:43:56.807 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:43:56.807 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:43:56.807 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:43:56.807 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:43:56.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:56.808 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:43:56.823 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:43:56.823 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:43:56.823 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:43:56.823 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:43:56.823 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:43:56.823 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:43:56.823 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:43:56.823 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:43:56.823 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:43:56.823 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:43:56.823 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42502sec preferred_lft 42502sec 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:43:56.824 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:43:56.825 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:43:56.825 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:43:56.825 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:43:56.825 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:43:56.825 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:43:56.825 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:43:56.825 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:43:56.825 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:43:56.825 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:43:56.825 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:43:56.825 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:43:56.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:56.826 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:43:56.889 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:43:56.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:56.891 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:43:56.946 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:43:56.946 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:43:56.946 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:43:56.946 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:43:56.946 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:43:56.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:56.946 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:43:56.946 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:43:56.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:43:56.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:43:56.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:43:56.947 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:43:57.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:57.041 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:43:57.055 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:43:57.055 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:43:57.055 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:43:57.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:57.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:43:57.121 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:43:57.122 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:43:57.122 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:43:57.176 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:43:57.176 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:43:57.176 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:43:57.230 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:43:57.297 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:43:57.297 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:43:57.363 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:43:57.365 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:43:57.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:57.365 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:43:57.420 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:43:57.420 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:43:57.463 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:43:57.489 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:43:57.489 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:43:57.548 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:43:57.548+0000 7f10b65f9480 -1 init, newargv = 0x564e4ad2e180 newargc=15 2024-03-23T09:43:57.548 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[57909]: starting ceph client 2024-03-23T09:43:57.553 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[57909]: starting fuse 2024-03-23T09:43:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:43:57.560 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:43:57.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:57.561 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:43:57.615 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:43:57.615 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:43:57.615 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:43:57.615 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:43:57.615 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:43:57.616 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:43:57.617 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:43:57.617 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:43:57.617 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:43:57.617 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:43:57.617 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:43:57.617 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:43:57.617 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:43:57.617 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:43:57.617 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:43:57.731 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.57909.asok 2024-03-23T09:43:57.918 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.57909.asok 2024-03-23T09:43:57.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:57.918 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.57909.asok status 2024-03-23T09:43:58.054 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:43:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:43:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:43:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:43:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:43:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:43:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:43:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "57909", 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 6369, 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 6369 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 652690074 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 652690074 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.6369 192.168.0.1:0/652690074", 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/652690074", 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 24, 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 58, 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:43:58.056 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:43:58.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:58.063 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:43:58.063 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:43:58.077 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:43:58.078 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:43:58.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:58.078 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:43:58.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:58.145 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-03-23T09:43:58.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:58.206 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-03-23T09:43:58.265 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:43:58.265 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:43:58.265 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:43:58.265 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:43:58.265 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-03-23T09:43:58.265 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:43:58.366 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627777 2024-03-23T09:43:58.555 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:43:58.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.697+0000 7fa190e0b640 1 Processor -- start 2024-03-23T09:43:58.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.697+0000 7fa190e0b640 1 -- start start 2024-03-23T09:43:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.698+0000 7fa190e0b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa18c15aa70 0x7fa18c0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.698+0000 7fa190e0b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa18c0ce9d0 0x7fa18c0cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.698+0000 7fa190e0b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18c15ae50 con 0x7fa18c0cd620 2024-03-23T09:43:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.698+0000 7fa190e0b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c0cda90 con 0x7fa18c0ce9d0 2024-03-23T09:43:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.698+0000 7fa190e0b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c0cf010 con 0x7fa18c15aa70 2024-03-23T09:43:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.698+0000 7fa18ad9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa18c0ce9d0 0x7fa18c0cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.698+0000 7fa189d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa18c0cd620 0x7fa18c0ce2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44758/0 (socket says 172.21.15.47:44758) 2024-03-23T09:43:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.698+0000 7fa189d9b640 1 -- 172.21.15.47:0/1224844211 learned_addr learned my addr 172.21.15.47:0/1224844211 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.699+0000 7fa18ad9d640 1 -- 172.21.15.47:0/1224844211 >> v2:172.21.15.175:3300/0 conn(0x7fa18c15aa70 msgr2=0x7fa18c0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.699+0000 7fa18ad9d640 1 --2- 172.21.15.47:0/1224844211 >> v2:172.21.15.175:3300/0 conn(0x7fa18c15aa70 0x7fa18c0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.699+0000 7fa18ad9d640 1 -- 172.21.15.47:0/1224844211 >> v1:172.21.15.47:6789/0 conn(0x7fa18c0cd620 legacy=0x7fa18c0ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:58.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.699+0000 7fa18959a640 1 -- 172.21.15.47:0/1224844211 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589756571 0 0) 0x7fa18c15ae50 con 0x7fa18c0cd620 2024-03-23T09:43:58.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.699+0000 7fa18ad9d640 1 -- 172.21.15.47:0/1224844211 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c0cf2f0 con 0x7fa18c0ce9d0 2024-03-23T09:43:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.699+0000 7fa18ad9d640 1 --2- 172.21.15.47:0/1224844211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18c0ce9d0 0x7fa18c0cbb70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0cdcc0 tx=0x7fa17c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb655e6c8aa2a4de server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.700+0000 7fa18959a640 1 -- 172.21.15.47:0/1224844211 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c02cd00 con 0x7fa18c0ce9d0 2024-03-23T09:43:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.700+0000 7fa18959a640 1 -- 172.21.15.47:0/1224844211 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c02ce60 con 0x7fa18c0ce9d0 2024-03-23T09:43:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.700+0000 7fa18959a640 1 -- 172.21.15.47:0/1224844211 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c034660 con 0x7fa18c0ce9d0 2024-03-23T09:43:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.700+0000 7fa190e0b640 1 -- 172.21.15.47:0/1224844211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18c0ce9d0 msgr2=0x7fa18c0cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.700+0000 7fa190e0b640 1 --2- 172.21.15.47:0/1224844211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18c0ce9d0 0x7fa18c0cbb70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0cdcc0 tx=0x7fa17c02bf50 comp rx=0 tx=0).stop 2024-03-23T09:43:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.700+0000 7fa190e0b640 1 -- 172.21.15.47:0/1224844211 shutdown_connections 2024-03-23T09:43:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.700+0000 7fa190e0b640 1 --2- 172.21.15.47:0/1224844211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18c0ce9d0 0x7fa18c0cbb70 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.700+0000 7fa190e0b640 1 --2- 172.21.15.47:0/1224844211 >> v2:172.21.15.175:3300/0 conn(0x7fa18c15aa70 0x7fa18c0cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:58.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.700+0000 7fa190e0b640 1 -- 172.21.15.47:0/1224844211 >> 172.21.15.47:0/1224844211 conn(0x7fa18c0d06f0 msgr2=0x7fa18c0d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:58.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.700+0000 7fa190e0b640 1 -- 172.21.15.47:0/1224844211 shutdown_connections 2024-03-23T09:43:58.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.700+0000 7fa190e0b640 1 -- 172.21.15.47:0/1224844211 wait complete. 2024-03-23T09:43:58.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.701+0000 7fa190e0b640 1 Processor -- start 2024-03-23T09:43:58.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.702+0000 7fa190e0b640 1 -- start start 2024-03-23T09:43:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.702+0000 7fa190e0b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa18c0cd620 0x7fa18c152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.702+0000 7fa190e0b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa18c0ce9d0 0x7fa18c0c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.702+0000 7fa190e0b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18c15aa70 0x7fa18c06f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.702+0000 7fa190e0b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0c5e40 con 0x7fa18c15aa70 2024-03-23T09:43:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.702+0000 7fa190e0b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0c6980 con 0x7fa18c0ce9d0 2024-03-23T09:43:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.702+0000 7fa190e0b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0c6050 con 0x7fa18c0cd620 2024-03-23T09:43:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.702+0000 7fa18ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18c15aa70 0x7fa18c06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.702+0000 7fa18ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18c15aa70 0x7fa18c06f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38138/0 (socket says 172.21.15.47:38138) 2024-03-23T09:43:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.702+0000 7fa18ad9d640 1 -- 172.21.15.47:0/2353678099 learned_addr learned my addr 172.21.15.47:0/2353678099 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.703+0000 7fa189d9b640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa18c0ce9d0 0x7fa18c0c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.703+0000 7fa18a59c640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa18c0cd620 0x7fa18c152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.703+0000 7fa18ad9d640 1 -- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa18c0cd620 msgr2=0x7fa18c152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.703+0000 7fa18ad9d640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa18c0cd620 0x7fa18c152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.703+0000 7fa18ad9d640 1 -- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa18c0ce9d0 msgr2=0x7fa18c0c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.703+0000 7fa18ad9d640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa18c0ce9d0 0x7fa18c0c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.703+0000 7fa18ad9d640 1 -- 172.21.15.47:0/2353678099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c009a70 con 0x7fa18c15aa70 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.703+0000 7fa18ad9d640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18c15aa70 0x7fa18c06f410 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa17c02c560 tx=0x7fa17c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.703+0000 7fa182ffd640 1 -- 172.21.15.47:0/2353678099 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c03f680 con 0x7fa18c15aa70 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.703+0000 7fa182ffd640 1 -- 172.21.15.47:0/2353678099 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c03f7e0 con 0x7fa18c15aa70 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.703+0000 7fa190e0b640 1 -- 172.21.15.47:0/2353678099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18c0c6330 con 0x7fa18c15aa70 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.703+0000 7fa182ffd640 1 -- 172.21.15.47:0/2353678099 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c03fb90 con 0x7fa18c15aa70 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.703+0000 7fa190e0b640 1 -- 172.21.15.47:0/2353678099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18c1f2f60 con 0x7fa18c15aa70 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.704+0000 7fa190e0b640 1 -- 172.21.15.47:0/2353678099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18c0c6af0 con 0x7fa18c15aa70 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.704+0000 7fa182ffd640 1 -- 172.21.15.47:0/2353678099 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa17c03b040 con 0x7fa18c15aa70 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.705+0000 7fa182ffd640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa16c06ac30 0x7fa16c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.705+0000 7fa182ffd640 1 -- 172.21.15.47:0/2353678099 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa17c0abee0 con 0x7fa18c15aa70 2024-03-23T09:43:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.705+0000 7fa18a59c640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa16c06ac30 0x7fa16c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:58.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.705+0000 7fa18a59c640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa16c06ac30 0x7fa16c06d0f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa18400b810 tx=0x7fa184008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.709+0000 7fa182ffd640 1 -- 172.21.15.47:0/2353678099 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa17c077150 con 0x7fa18c15aa70 2024-03-23T09:43:58.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.924+0000 7fa190e0b640 1 -- 172.21.15.47:0/2353678099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa18c066be0 con 0x7fa18c15aa70 2024-03-23T09:43:58.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.924+0000 7fa182ffd640 1 -- 172.21.15.47:0/2353678099 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fa17c039090 con 0x7fa18c15aa70 2024-03-23T09:43:58.924 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:58.925 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/375513546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":375513546},{"type":"v1","addr":"172.21.15.175:6837","nonce":375513546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/3650756747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3650756747},{"type":"v1","addr":"172.21.15.47:6833","nonce":3650756747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2893805543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2893805543},{"type":"v1","addr":"172.21.15.47:6835","nonce":2893805543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:43:47.389279+0000","modified":"2024-03-23T09:43:49.400979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/2498456351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2498456351},{"type":"v1","addr":"172.21.15.175:6836","nonce":2498456351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-23T09:43:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2024-03-23T09:43:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.926+0000 7fa190e0b640 1 -- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa16c06ac30 msgr2=0x7fa16c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.926+0000 7fa190e0b640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa16c06ac30 0x7fa16c06d0f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa18400b810 tx=0x7fa184008040 comp rx=0 tx=0).stop 2024-03-23T09:43:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.926+0000 7fa190e0b640 1 -- 172.21.15.47:0/2353678099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18c15aa70 msgr2=0x7fa18c06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.926+0000 7fa190e0b640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18c15aa70 0x7fa18c06f410 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa17c02c560 tx=0x7fa17c004a40 comp rx=0 tx=0).stop 2024-03-23T09:43:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.926+0000 7fa190e0b640 1 -- 172.21.15.47:0/2353678099 shutdown_connections 2024-03-23T09:43:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.926+0000 7fa190e0b640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa16c06ac30 0x7fa16c06d0f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.926+0000 7fa190e0b640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18c15aa70 0x7fa18c06f410 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.926+0000 7fa190e0b640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa18c0ce9d0 0x7fa18c0c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.926+0000 7fa190e0b640 1 --2- 172.21.15.47:0/2353678099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa18c0cd620 0x7fa18c152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.926+0000 7fa190e0b640 1 -- 172.21.15.47:0/2353678099 >> 172.21.15.47:0/2353678099 conn(0x7fa18c0d06f0 msgr2=0x7fa18c0cbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.926+0000 7fa190e0b640 1 -- 172.21.15.47:0/2353678099 shutdown_connections 2024-03-23T09:43:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:58.926+0000 7fa190e0b640 1 -- 172.21.15.47:0/2353678099 wait complete. 2024-03-23T09:43:58.937 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-03-23T09:43:59.073 INFO:teuthology.orchestra.run.smithi175.stdout:{"message":"","return_code":0} 2024-03-23T09:43:59.073 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-23T09:43:59.073 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-23T09:43:59.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.114+0000 7f7434a38f40 1 Processor -- start 2024-03-23T09:43:59.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.115+0000 7f7434a38f40 1 -- start start 2024-03-23T09:43:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.115+0000 7f7434a38f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d117bcab0 0x561d11844880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.115+0000 7f7434a38f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d117c4b00 0x561d1184ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.115+0000 7f7434a38f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d11853b20 0x561d11855b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.115+0000 7f7434a38f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d116c4b60 con 0x561d117bcab0 2024-03-23T09:43:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.115+0000 7f7434a38f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d116c4800 con 0x561d11853b20 2024-03-23T09:43:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.115+0000 7f7434a38f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d116a7520 con 0x561d117c4b00 2024-03-23T09:43:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.115+0000 7f7431ddd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d117bcab0 0x561d11844880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.115+0000 7f7431ddd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d117bcab0 0x561d11844880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38160/0 (socket says 172.21.15.47:38160) 2024-03-23T09:43:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.115+0000 7f7431ddd640 1 -- 172.21.15.47:0/335018302 learned_addr learned my addr 172.21.15.47:0/335018302 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f7431ddd640 1 -- 172.21.15.47:0/335018302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d117c4b00 msgr2=0x561d1184ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f74315dc640 1 --2- 172.21.15.47:0/335018302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d117c4b00 0x561d1184ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f7431ddd640 1 --2- 172.21.15.47:0/335018302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d117c4b00 0x561d1184ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f7431ddd640 1 -- 172.21.15.47:0/335018302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d11853b20 msgr2=0x561d11855b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f74325de640 1 --2- 172.21.15.47:0/335018302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d11853b20 0x561d11855b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f7431ddd640 1 --2- 172.21.15.47:0/335018302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d11853b20 0x561d11855b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f7431ddd640 1 -- 172.21.15.47:0/335018302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c005de0 con 0x561d117bcab0 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f74325de640 1 --2- 172.21.15.47:0/335018302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d11853b20 0x561d11855b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f7431ddd640 1 --2- 172.21.15.47:0/335018302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d117bcab0 0x561d11844880 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f742c00a5e0 tx=0x7f742c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93036aa63e9269f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f742bfff640 1 -- 172.21.15.47:0/335018302 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c008a40 con 0x561d117bcab0 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f742bfff640 1 -- 172.21.15.47:0/335018302 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c008bc0 con 0x561d117bcab0 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f742bfff640 1 -- 172.21.15.47:0/335018302 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c013440 con 0x561d117bcab0 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f7434a38f40 1 -- 172.21.15.47:0/335018302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d117bcab0 msgr2=0x561d11844880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f7434a38f40 1 --2- 172.21.15.47:0/335018302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d117bcab0 0x561d11844880 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f742c00a5e0 tx=0x7f742c005f90 comp rx=0 tx=0).stop 2024-03-23T09:43:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f7434a38f40 1 -- 172.21.15.47:0/335018302 shutdown_connections 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f7434a38f40 1 --2- 172.21.15.47:0/335018302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d11853b20 0x561d11855b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f7434a38f40 1 --2- 172.21.15.47:0/335018302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d117c4b00 0x561d1184ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f7434a38f40 1 --2- 172.21.15.47:0/335018302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d117bcab0 0x561d11844880 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.116+0000 7f7434a38f40 1 -- 172.21.15.47:0/335018302 >> 172.21.15.47:0/335018302 conn(0x561d117a7e40 msgr2=0x561d11856880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.117+0000 7f7434a38f40 1 -- 172.21.15.47:0/335018302 shutdown_connections 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.117+0000 7f7434a38f40 1 -- 172.21.15.47:0/335018302 wait complete. 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.117+0000 7f7434a38f40 1 Processor -- start 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.117+0000 7f7434a38f40 1 -- start start 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f7434a38f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d117bcab0 0x561d1182ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f7434a38f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d117c4b00 0x561d1183e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f7434a38f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d11853b20 0x561d1184d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f7434a38f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d117aa9d0 con 0x561d117c4b00 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f7434a38f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d11841980 con 0x561d11853b20 2024-03-23T09:43:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f7434a38f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d11841c60 con 0x561d117bcab0 2024-03-23T09:43:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f74315dc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d117c4b00 0x561d1183e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f74315dc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d117c4b00 0x561d1183e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38162/0 (socket says 172.21.15.47:38162) 2024-03-23T09:43:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f74315dc640 1 -- 172.21.15.47:0/431802613 learned_addr learned my addr 172.21.15.47:0/431802613 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f7431ddd640 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d117bcab0 0x561d1182ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f74325de640 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d11853b20 0x561d1184d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f74315dc640 1 -- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d117bcab0 msgr2=0x561d1182ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f74315dc640 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d117bcab0 0x561d1182ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f74315dc640 1 -- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d11853b20 msgr2=0x561d1184d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f74315dc640 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d11853b20 0x561d1184d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f74315dc640 1 -- 172.21.15.47:0/431802613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c005d70 con 0x561d117c4b00 2024-03-23T09:43:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f74325de640 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d11853b20 0x561d1184d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f7431ddd640 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d117bcab0 0x561d1182ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.118+0000 7f74315dc640 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d117c4b00 0x561d1183e930 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f741c01d390 tx=0x7f741c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.119+0000 7f742affd640 1 -- 172.21.15.47:0/431802613 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c053d40 con 0x561d117c4b00 2024-03-23T09:43:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.119+0000 7f742affd640 1 -- 172.21.15.47:0/431802613 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c053620 con 0x561d117c4b00 2024-03-23T09:43:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.119+0000 7f7434a38f40 1 -- 172.21.15.47:0/431802613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561d116d3c20 con 0x561d117c4b00 2024-03-23T09:43:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.119+0000 7f742affd640 1 -- 172.21.15.47:0/431802613 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c053ea0 con 0x561d117c4b00 2024-03-23T09:43:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.119+0000 7f7434a38f40 1 -- 172.21.15.47:0/431802613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d116c7060 con 0x561d117c4b00 2024-03-23T09:43:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.120+0000 7f742affd640 1 -- 172.21.15.47:0/431802613 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f741c05a040 con 0x561d117c4b00 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.120+0000 7f742affd640 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f74140695e0 0x7f741406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.120+0000 7f742affd640 1 -- 172.21.15.47:0/431802613 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f741c0c9b80 con 0x561d117c4b00 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.120+0000 7f7434a38f40 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x561d1184d9e0 0x561d117ab190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.120+0000 7f7434a38f40 1 -- 172.21.15.47:0/431802613 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58) v8 -- 0x561d1183e150 con 0x561d1184d9e0 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.120+0000 7f7431ddd640 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f74140695e0 0x7f741406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.121+0000 7f74325de640 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x561d1184d9e0 0x561d117ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.121+0000 7f7431ddd640 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f74140695e0 0x7f741406baa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f742c000c00 tx=0x7f742c015040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.121+0000 7f74325de640 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x561d1184d9e0 0x561d117ab190 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.121+0000 7f74325de640 1 -- 172.21.15.47:0/431802613 <== osd.5 v2:172.21.15.175:6816/38161 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f7424003040 con 0x561d1184d9e0 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.121+0000 7f7434a38f40 1 -- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x561d1184d9e0 msgr2=0x561d117ab190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.121+0000 7f7434a38f40 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x561d1184d9e0 0x561d117ab190 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.122+0000 7f7434a38f40 1 -- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f74140695e0 msgr2=0x7f741406baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.122+0000 7f7434a38f40 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f74140695e0 0x7f741406baa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f742c000c00 tx=0x7f742c015040 comp rx=0 tx=0).stop 2024-03-23T09:43:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.122+0000 7f7434a38f40 1 -- 172.21.15.47:0/431802613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d117c4b00 msgr2=0x561d1183e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.122+0000 7f7434a38f40 1 --2- 172.21.15.47:0/431802613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d117c4b00 0x561d1183e930 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f741c01d390 tx=0x7f741c002fb0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.122+0000 7f74325de640 1 -- 172.21.15.47:0/431802613 reap_dead start 2024-03-23T09:43:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.122+0000 7f7434a38f40 1 -- 172.21.15.47:0/431802613 shutdown_connections 2024-03-23T09:43:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.122+0000 7f7434a38f40 1 -- 172.21.15.47:0/431802613 >> 172.21.15.47:0/431802613 conn(0x561d117a7e40 msgr2=0x561d118513e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.122+0000 7f7434a38f40 1 -- 172.21.15.47:0/431802613 shutdown_connections 2024-03-23T09:43:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.122+0000 7f7434a38f40 1 -- 172.21.15.47:0/431802613 wait complete. 2024-03-23T09:43:59.127 INFO:teuthology.orchestra.run.smithi047.stdout:Xfiledir_test 2024-03-23T09:43:59.127 DEBUG:teuthology.orchestra.run.smithi047:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-23T09:43:59.210 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:43:59.210 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 1099511627777, 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: "ancestors": [ 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: "dirino": 1099511627776, 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: "dname": "file", 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: "version": 2 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: "dirino": 1, 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: "dname": "dir_test", 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: "version": 4 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 9, 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout: "old_pools": [] 2024-03-23T09:43:59.211 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:43:59.212 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:43:59.213 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-03-23T09:43:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.295+0000 7f5fbbc36f40 1 Processor -- start 2024-03-23T09:43:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.295+0000 7f5fbbc36f40 1 -- start start 2024-03-23T09:43:59.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fbbc36f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa00030b0 0x560aedccaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fbbc36f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560aedccb0d0 0x560aedc40250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fbbc36f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aedc3e520 0x560aedcb3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fbbc36f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aedba44d0 con 0x560aedc3e520 2024-03-23T09:43:59.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fbbc36f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aedb3fb60 con 0x560aedccb0d0 2024-03-23T09:43:59.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fbbc36f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x560aedb3f800 con 0x7f5fa00030b0 2024-03-23T09:43:59.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fb969a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aedc3e520 0x560aedcb3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fb969a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aedc3e520 0x560aedcb3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38174/0 (socket says 172.21.15.47:38174) 2024-03-23T09:43:59.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fb969a640 1 -- 172.21.15.47:0/3902089186 learned_addr learned my addr 172.21.15.47:0/3902089186 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fb8698640 1 --2- 172.21.15.47:0/3902089186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560aedccb0d0 0x560aedc40250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fb8e99640 1 --2- 172.21.15.47:0/3902089186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa00030b0 0x560aedccaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fb969a640 1 -- 172.21.15.47:0/3902089186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa00030b0 msgr2=0x560aedccaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fb969a640 1 --2- 172.21.15.47:0/3902089186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa00030b0 0x560aedccaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fb969a640 1 -- 172.21.15.47:0/3902089186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560aedccb0d0 msgr2=0x560aedc40250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fb969a640 1 --2- 172.21.15.47:0/3902089186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560aedccb0d0 0x560aedc40250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fb969a640 1 -- 172.21.15.47:0/3902089186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560aedb42060 con 0x560aedc3e520 2024-03-23T09:43:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fb8698640 1 --2- 172.21.15.47:0/3902089186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560aedccb0d0 0x560aedc40250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.296+0000 7f5fb8e99640 1 --2- 172.21.15.47:0/3902089186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa00030b0 0x560aedccaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.297+0000 7f5fb969a640 1 --2- 172.21.15.47:0/3902089186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aedc3e520 0x560aedcb3a30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8001930 tx=0x7f5fa8006920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce835c36ea45d61 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.297+0000 7f5fb7696640 1 -- 172.21.15.47:0/3902089186 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8012360 con 0x560aedc3e520 2024-03-23T09:43:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.297+0000 7f5fb7696640 1 -- 172.21.15.47:0/3902089186 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa800c070 con 0x560aedc3e520 2024-03-23T09:43:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.297+0000 7f5fb7696640 1 -- 172.21.15.47:0/3902089186 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8012700 con 0x560aedc3e520 2024-03-23T09:43:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.297+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/3902089186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aedc3e520 msgr2=0x560aedcb3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.297+0000 7f5fbbc36f40 1 --2- 172.21.15.47:0/3902089186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aedc3e520 0x560aedcb3a30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8001930 tx=0x7f5fa8006920 comp rx=0 tx=0).stop 2024-03-23T09:43:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.297+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/3902089186 shutdown_connections 2024-03-23T09:43:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.297+0000 7f5fbbc36f40 1 --2- 172.21.15.47:0/3902089186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aedc3e520 0x560aedcb3a30 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.297+0000 7f5fbbc36f40 1 --2- 172.21.15.47:0/3902089186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560aedccb0d0 0x560aedc40250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.297+0000 7f5fbbc36f40 1 --2- 172.21.15.47:0/3902089186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa00030b0 0x560aedccaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.297+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/3902089186 >> 172.21.15.47:0/3902089186 conn(0x560aedc22e40 msgr2=0x560aedcb1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.297+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/3902089186 shutdown_connections 2024-03-23T09:43:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.297+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/3902089186 wait complete. 2024-03-23T09:43:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.298+0000 7f5fbbc36f40 1 Processor -- start 2024-03-23T09:43:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.298+0000 7f5fbbc36f40 1 -- start start 2024-03-23T09:43:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.298+0000 7f5fbbc36f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aedc3e520 0x560aedcaa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.298+0000 7f5fbbc36f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560aedccb0d0 0x560aedcb3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.298+0000 7f5fbbc36f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa00030b0 0x560aedcb3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.298+0000 7f5fbbc36f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aedb22520 con 0x560aedc3e520 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fbbc36f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aedcdb5a0 con 0x560aedccb0d0 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fbbc36f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x560aedcccd10 con 0x7f5fa00030b0 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb8e99640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aedc3e520 0x560aedcaa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb8e99640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aedc3e520 0x560aedcaa870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38182/0 (socket says 172.21.15.47:38182) 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb8e99640 1 -- 172.21.15.47:0/2860550621 learned_addr learned my addr 172.21.15.47:0/2860550621 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb8698640 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560aedccb0d0 0x560aedcb3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb969a640 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa00030b0 0x560aedcb3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb8e99640 1 -- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa00030b0 msgr2=0x560aedcb3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb8e99640 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa00030b0 0x560aedcb3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb8e99640 1 -- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560aedccb0d0 msgr2=0x560aedcb3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb8e99640 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560aedccb0d0 0x560aedcb3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb8e99640 1 -- 172.21.15.47:0/2860550621 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa800e040 con 0x560aedc3e520 2024-03-23T09:43:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb8e99640 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aedc3e520 0x560aedcaa870 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0002da0 tx=0x7f5fb000d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb6694640 1 -- 172.21.15.47:0/2860550621 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000ded0 con 0x560aedc3e520 2024-03-23T09:43:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb6694640 1 -- 172.21.15.47:0/2860550621 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb0008040 con 0x560aedc3e520 2024-03-23T09:43:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/2860550621 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560aedb41ac0 con 0x560aedc3e520 2024-03-23T09:43:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fb6694640 1 -- 172.21.15.47:0/2860550621 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0012680 con 0x560aedc3e520 2024-03-23T09:43:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.299+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/2860550621 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560aedc25650 con 0x560aedc3e520 2024-03-23T09:43:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.301+0000 7f5fb6694640 1 -- 172.21.15.47:0/2860550621 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5fb00127e0 con 0x560aedc3e520 2024-03-23T09:43:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.301+0000 7f5fb6694640 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5fa4069610 0x7f5fa406bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.301+0000 7f5fb6694640 1 -- 172.21.15.47:0/2860550621 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fb0088090 con 0x560aedc3e520 2024-03-23T09:43:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.301+0000 7f5fb8698640 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5fa4069610 0x7f5fa406bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.301+0000 7f5fbbc36f40 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x560aedd6f160 0x560aedd71550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.301+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/2860550621 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58) v8 -- 0x560aedd71a90 con 0x560aedd6f160 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.301+0000 7f5fb969a640 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x560aedd6f160 0x560aedd71550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.302+0000 7f5fb8698640 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5fa4069610 0x7f5fa406bad0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009810 tx=0x7f5fac04c070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.302+0000 7f5fb969a640 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x560aedd6f160 0x560aedd71550 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.303+0000 7f5fb969a640 1 -- 172.21.15.47:0/2860550621 <== osd.5 v2:172.21.15.175:6816/38161 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v58'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5fa801c040 con 0x560aedd6f160 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.303+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x560aedd6f160 msgr2=0x560aedd71550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.303+0000 7f5fbbc36f40 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x560aedd6f160 0x560aedd71550 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.303+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5fa4069610 msgr2=0x7f5fa406bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.303+0000 7f5fbbc36f40 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5fa4069610 0x7f5fa406bad0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009810 tx=0x7f5fac04c070 comp rx=0 tx=0).stop 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.303+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/2860550621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aedc3e520 msgr2=0x560aedcaa870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.303+0000 7f5fbbc36f40 1 --2- 172.21.15.47:0/2860550621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aedc3e520 0x560aedcaa870 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0002da0 tx=0x7f5fb000d630 comp rx=0 tx=0).stop 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.303+0000 7f5fb969a640 1 -- 172.21.15.47:0/2860550621 reap_dead start 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.303+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/2860550621 shutdown_connections 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.303+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/2860550621 >> 172.21.15.47:0/2860550621 conn(0x560aedc22e40 msgr2=0x560aedc40210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.303+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/2860550621 shutdown_connections 2024-03-23T09:43:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.303+0000 7f5fbbc36f40 1 -- 172.21.15.47:0/2860550621 wait complete. 2024-03-23T09:43:59.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:59.308 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-03-23T09:43:59.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.451+0000 7fb67883e640 1 Processor -- start 2024-03-23T09:43:59.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.451+0000 7fb67883e640 1 -- start start 2024-03-23T09:43:59.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.452+0000 7fb67883e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb67415d0f0 0x7fb67415d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.452+0000 7fb67883e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb674160bd0 con 0x7fb6740cf190 2024-03-23T09:43:59.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.452+0000 7fb67883e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67415dbb0 con 0x7fb67415a860 2024-03-23T09:43:59.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.452+0000 7fb67883e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6740d2790 con 0x7fb67415d0f0 2024-03-23T09:43:59.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.452+0000 7fb671d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb67415d0f0 0x7fb67415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.452+0000 7fb672d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb6740cf190 0x7fb6740cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44788/0 (socket says 172.21.15.47:44788) 2024-03-23T09:43:59.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.452+0000 7fb671d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb67415d0f0 0x7fb67415d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38198/0 (socket says 172.21.15.47:38198) 2024-03-23T09:43:59.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.452+0000 7fb672d9d640 1 -- 172.21.15.47:0/3931395886 learned_addr learned my addr 172.21.15.47:0/3931395886 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:59.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.453+0000 7fb671d9b640 1 -- 172.21.15.47:0/3931395886 >> v1:172.21.15.175:6789/0 conn(0x7fb67415a860 legacy=0x7fb674161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.453+0000 7fb671d9b640 1 -- 172.21.15.47:0/3931395886 >> v1:172.21.15.47:6789/0 conn(0x7fb6740cf190 legacy=0x7fb6740cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.453+0000 7fb671d9b640 1 -- 172.21.15.47:0/3931395886 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67415de90 con 0x7fb67415d0f0 2024-03-23T09:43:59.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.453+0000 7fb671d9b640 1 --2- 172.21.15.47:0/3931395886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb67415d0f0 0x7fb67415d580 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0097a0 tx=0x7fb65c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab13740be8046c24 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.453+0000 7fb67159a640 1 -- 172.21.15.47:0/3931395886 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c031830 con 0x7fb67415d0f0 2024-03-23T09:43:59.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.453+0000 7fb67159a640 1 -- 172.21.15.47:0/3931395886 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65c031990 con 0x7fb67415d0f0 2024-03-23T09:43:59.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.453+0000 7fb67159a640 1 -- 172.21.15.47:0/3931395886 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c031d20 con 0x7fb67415d0f0 2024-03-23T09:43:59.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.454+0000 7fb67883e640 1 -- 172.21.15.47:0/3931395886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb67415d0f0 msgr2=0x7fb67415d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.454+0000 7fb67883e640 1 --2- 172.21.15.47:0/3931395886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb67415d0f0 0x7fb67415d580 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0097a0 tx=0x7fb65c02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.454+0000 7fb67883e640 1 -- 172.21.15.47:0/3931395886 shutdown_connections 2024-03-23T09:43:59.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.454+0000 7fb67883e640 1 --2- 172.21.15.47:0/3931395886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb67415d0f0 0x7fb67415d580 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.454+0000 7fb67883e640 1 -- 172.21.15.47:0/3931395886 >> 172.21.15.47:0/3931395886 conn(0x7fb6740659d0 msgr2=0x7fb674065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.454+0000 7fb67883e640 1 -- 172.21.15.47:0/3931395886 shutdown_connections 2024-03-23T09:43:59.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.454+0000 7fb67883e640 1 -- 172.21.15.47:0/3931395886 wait complete. 2024-03-23T09:43:59.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.455+0000 7fb67883e640 1 Processor -- start 2024-03-23T09:43:59.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.455+0000 7fb67883e640 1 -- start start 2024-03-23T09:43:59.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.456+0000 7fb67883e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6740cf190 0x7fb6740b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.456+0000 7fb67883e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb67415a860 0x7fb674152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.456+0000 7fb67883e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb67415d0f0 0x7fb6740b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.456+0000 7fb67883e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740b4500 con 0x7fb67415d0f0 2024-03-23T09:43:59.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.456+0000 7fb67883e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740b2f20 con 0x7fb6740cf190 2024-03-23T09:43:59.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.456+0000 7fb67883e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740b3200 con 0x7fb67415a860 2024-03-23T09:43:59.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.456+0000 7fb672d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb67415d0f0 0x7fb6740b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.456+0000 7fb672d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb67415d0f0 0x7fb6740b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38204/0 (socket says 172.21.15.47:38204) 2024-03-23T09:43:59.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.456+0000 7fb672d9d640 1 -- 172.21.15.47:0/3778466660 learned_addr learned my addr 172.21.15.47:0/3778466660 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.456+0000 7fb671d9b640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb67415a860 0x7fb674152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.456+0000 7fb67259c640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6740cf190 0x7fb6740b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.457+0000 7fb672d9d640 1 -- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb67415a860 msgr2=0x7fb674152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.457+0000 7fb672d9d640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb67415a860 0x7fb674152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.457+0000 7fb672d9d640 1 -- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6740cf190 msgr2=0x7fb6740b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.457+0000 7fb672d9d640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6740cf190 0x7fb6740b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.457+0000 7fb672d9d640 1 -- 172.21.15.47:0/3778466660 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c009400 con 0x7fb67415d0f0 2024-03-23T09:43:59.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.457+0000 7fb672d9d640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb67415d0f0 0x7fb6740b2420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb66400cbc0 tx=0x7fb66400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.457+0000 7fb66affd640 1 -- 172.21.15.47:0/3778466660 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66400ad40 con 0x7fb67415d0f0 2024-03-23T09:43:59.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.457+0000 7fb66affd640 1 -- 172.21.15.47:0/3778466660 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb664012400 con 0x7fb67415d0f0 2024-03-23T09:43:59.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.457+0000 7fb67883e640 1 -- 172.21.15.47:0/3778466660 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6740b34e0 con 0x7fb67415d0f0 2024-03-23T09:43:59.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.457+0000 7fb66affd640 1 -- 172.21.15.47:0/3778466660 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66400aea0 con 0x7fb67415d0f0 2024-03-23T09:43:59.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.457+0000 7fb67883e640 1 -- 172.21.15.47:0/3778466660 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6740b3aa0 con 0x7fb67415d0f0 2024-03-23T09:43:59.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.458+0000 7fb66affd640 1 -- 172.21.15.47:0/3778466660 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb664007070 con 0x7fb67415d0f0 2024-03-23T09:43:59.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.459+0000 7fb66affd640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb65406ac00 0x7fb65406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.459+0000 7fb66affd640 1 -- 172.21.15.47:0/3778466660 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6640891c0 con 0x7fb67415d0f0 2024-03-23T09:43:59.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.459+0000 7fb67259c640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb65406ac00 0x7fb65406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.460+0000 7fb67259c640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb65406ac00 0x7fb65406d0c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb66c004f60 tx=0x7fb66c007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.460+0000 7fb67b3b2740 1 Processor -- start 2024-03-23T09:43:59.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.460+0000 7fb67b3b2740 1 -- start start 2024-03-23T09:43:59.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67b3b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 0x556ccc9a4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67b3b2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d19a0 0x556ccc9d4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67b3b2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9a4aa0 0x556ccc9d4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67b3b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccc915c20 con 0x556ccc9d15c0 2024-03-23T09:43:59.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67b3b2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccc9133b0 con 0x556ccc9a4aa0 2024-03-23T09:43:59.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67b3b2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ccc90eca0 con 0x556ccc9d19a0 2024-03-23T09:43:59.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 0x556ccc9a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 0x556ccc9a4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38216/0 (socket says 172.21.15.47:38216) 2024-03-23T09:43:59.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67259c640 1 -- 172.21.15.47:0/3872677769 learned_addr learned my addr 172.21.15.47:0/3872677769 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb671d9b640 1 --2- 172.21.15.47:0/3872677769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d19a0 0x556ccc9d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb672d9d640 1 --2- 172.21.15.47:0/3872677769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9a4aa0 0x556ccc9d4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67259c640 1 -- 172.21.15.47:0/3872677769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d19a0 msgr2=0x556ccc9d4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67259c640 1 --2- 172.21.15.47:0/3872677769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d19a0 0x556ccc9d4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67259c640 1 -- 172.21.15.47:0/3872677769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9a4aa0 msgr2=0x556ccc9d4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67259c640 1 --2- 172.21.15.47:0/3872677769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9a4aa0 0x556ccc9d4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67259c640 1 -- 172.21.15.47:0/3872677769 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ccc8b5710 con 0x556ccc9d15c0 2024-03-23T09:43:59.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb672d9d640 1 --2- 172.21.15.47:0/3872677769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9a4aa0 0x556ccc9d4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:59.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.461+0000 7fb67259c640 1 --2- 172.21.15.47:0/3872677769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 0x556ccc9a4560 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb66c003cd0 tx=0x7fb66c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6515398d159518 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.462+0000 7fb653fff640 1 -- 172.21.15.47:0/3872677769 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c00fce0 con 0x556ccc9d15c0 2024-03-23T09:43:59.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.462+0000 7fb653fff640 1 -- 172.21.15.47:0/3872677769 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c00fe40 con 0x556ccc9d15c0 2024-03-23T09:43:59.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.462+0000 7fb653fff640 1 -- 172.21.15.47:0/3872677769 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c0047f0 con 0x556ccc9d15c0 2024-03-23T09:43:59.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.462+0000 7fb67b3b2740 1 -- 172.21.15.47:0/3872677769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 msgr2=0x556ccc9a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.462+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/3872677769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 0x556ccc9a4560 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb66c003cd0 tx=0x7fb66c012000 comp rx=0 tx=0).stop 2024-03-23T09:43:59.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.462+0000 7fb67b3b2740 1 -- 172.21.15.47:0/3872677769 shutdown_connections 2024-03-23T09:43:59.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.462+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/3872677769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9a4aa0 0x556ccc9d4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.462+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/3872677769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d19a0 0x556ccc9d4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.462+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/3872677769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 0x556ccc9a4560 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.462+0000 7fb67b3b2740 1 -- 172.21.15.47:0/3872677769 >> 172.21.15.47:0/3872677769 conn(0x556ccc9a67d0 msgr2=0x556ccc9ef350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.462+0000 7fb67b3b2740 1 -- 172.21.15.47:0/3872677769 shutdown_connections 2024-03-23T09:43:59.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.462+0000 7fb67b3b2740 1 -- 172.21.15.47:0/3872677769 wait complete. 2024-03-23T09:43:59.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.463+0000 7fb67b3b2740 1 -- start start 2024-03-23T09:43:59.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.463+0000 7fb67b3b2740 1 Processor -- start 2024-03-23T09:43:59.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb67b3b2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d15c0 0x556ccc9c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb67b3b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 0x556ccc9e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb67b3b2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9fed80 0x556ccc9d6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb67b3b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccc8cce70 con 0x556ccc9d19a0 2024-03-23T09:43:59.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb67b3b2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccc8ccb10 con 0x556ccc9fed80 2024-03-23T09:43:59.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb67b3b2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ccc892060 con 0x556ccc9d15c0 2024-03-23T09:43:59.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb67b3b2740 10 client.0 _ll_register_callbacks cb 0x556ccc908310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:43:59.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb671d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 0x556ccc9e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb671d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 0x556ccc9e5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38224/0 (socket says 172.21.15.47:38224) 2024-03-23T09:43:59.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb671d9b640 1 -- 172.21.15.47:0/2343181517 learned_addr learned my addr 172.21.15.47:0/2343181517 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb672d9d640 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9fed80 0x556ccc9d6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb67259c640 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d15c0 0x556ccc9c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb671d9b640 1 -- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d15c0 msgr2=0x556ccc9c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb671d9b640 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d15c0 0x556ccc9c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb671d9b640 1 -- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9fed80 msgr2=0x556ccc9d6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb671d9b640 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9fed80 0x556ccc9d6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb671d9b640 1 -- 172.21.15.47:0/2343181517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66c003980 con 0x556ccc9d19a0 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb672d9d640 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9fed80 0x556ccc9d6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb67259c640 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d15c0 0x556ccc9c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb671d9b640 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 0x556ccc9e5e70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb65c000c00 tx=0x7fb65c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cc6841fa7149bc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.464+0000 7fb6517fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb6517fa640 1 -- 172.21.15.47:0/2343181517 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c03a8f0 con 0x556ccc9d19a0 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb6517fa640 1 -- 172.21.15.47:0/2343181517 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb65c03f040 con 0x556ccc9d19a0 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb67b3b2740 1 -- 172.21.15.47:0/2343181517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ccc8fffc0 con 0x556ccc9d19a0 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb6517fa640 1 -- 172.21.15.47:0/2343181517 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb65c03aa50 con 0x556ccc9d19a0 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb67b3b2740 10 client.6414 fetch_fsmap learned FSMap version 24 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb67b3b2740 1 -- 172.21.15.47:0/2343181517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x556ccc90d320 con 0x556ccc9d19a0 2024-03-23T09:43:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb6517fa640 1 -- 172.21.15.47:0/2343181517 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556ccc8fffc0 con 0x556ccc9d19a0 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb6517fa640 1 -- 172.21.15.47:0/2343181517 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb65c043070 con 0x556ccc9d19a0 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb67b3b2740 10 client.6414 fetch_fsmap finished waiting for FSMap version 24 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb67b3b2740 10 client.6414 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb67b3b2740 20 client.6414 populate_metadata read hostname 'smithi047' 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ccc9d3980 0x556ccca0c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb67b3b2740 4 client.6414 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb67b3b2740 1 -- 172.21.15.47:0/2343181517 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ccc602490 con 0x556ccc9d3980 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.465+0000 7fb67259c640 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ccc9d3980 0x556ccca0c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.466+0000 7fb6517fa640 10 client.6414 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.466+0000 7fb67259c640 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ccc9d3980 0x556ccca0c520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24066b0e32a3c44c server_cookie=5b1bf0c4a249cc00 in_seq=0 out_seq=0 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.466+0000 7fb6517fa640 1 -- 172.21.15.47:0/2343181517 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556ccc602490 con 0x556ccc9d3980 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.466+0000 7fb6517fa640 10 client.6414 handle_command_reply: tid=0 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.466+0000 7fb67b3b2740 1 client.6414 shutdown 2024-03-23T09:43:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.466+0000 7fb67b3b2740 1 -- 172.21.15.47:0/2343181517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 msgr2=0x556ccc9e5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.466+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 0x556ccc9e5e70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb65c000c00 tx=0x7fb65c004730 comp rx=0 tx=0).stop 2024-03-23T09:43:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.467+0000 7fb67b3b2740 1 -- 172.21.15.47:0/2343181517 shutdown_connections 2024-03-23T09:43:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.467+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ccc9d3980 0x556ccca0c520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.467+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9fed80 0x556ccc9d6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.467+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 0x556ccc9e5e70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.467+0000 7fb6517fa640 0 client.6414 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:43:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.467+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/2343181517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d15c0 0x556ccc9c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.467+0000 7fb67b3b2740 1 -- 172.21.15.47:0/2343181517 >> 172.21.15.47:0/2343181517 conn(0x556ccc9a4aa0 msgr2=0x556ccc9a8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.467+0000 7fb67b3b2740 1 -- 172.21.15.47:0/2343181517 shutdown_connections 2024-03-23T09:43:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.467+0000 7fb67b3b2740 1 -- 172.21.15.47:0/2343181517 wait complete. 2024-03-23T09:43:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.467+0000 7fb67b3b2740 20 client.6414 trim_cache size 0 max 16384 2024-03-23T09:43:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.481+0000 7fb67b3b2740 1 Processor -- start 2024-03-23T09:43:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.482+0000 7fb67b3b2740 1 -- start start 2024-03-23T09:43:59.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.482+0000 7fb67b3b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 0x556ccc9a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.482+0000 7fb67b3b2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d19a0 0x556ccc9a8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.482+0000 7fb67b3b2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9fed80 0x556ccc9d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.482+0000 7fb67b3b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccc9a89c0 con 0x556ccc9d15c0 2024-03-23T09:43:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.482+0000 7fb67b3b2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccc9d57d0 con 0x556ccc9fed80 2024-03-23T09:43:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.482+0000 7fb67b3b2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ccc9cc550 con 0x556ccc9d19a0 2024-03-23T09:43:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.482+0000 7fb67259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 0x556ccc9a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.482+0000 7fb67259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 0x556ccc9a7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38230/0 (socket says 172.21.15.47:38230) 2024-03-23T09:43:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.482+0000 7fb67259c640 1 -- 172.21.15.47:0/1518069555 learned_addr learned my addr 172.21.15.47:0/1518069555 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.482+0000 7fb67259c640 1 -- 172.21.15.47:0/1518069555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d19a0 msgr2=0x556ccc9a8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb67259c640 1 --2- 172.21.15.47:0/1518069555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d19a0 0x556ccc9a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb67259c640 1 -- 172.21.15.47:0/1518069555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9fed80 msgr2=0x556ccc9d6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb67259c640 1 --2- 172.21.15.47:0/1518069555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9fed80 0x556ccc9d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb67259c640 1 -- 172.21.15.47:0/1518069555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ccc8af100 con 0x556ccc9d15c0 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb67259c640 1 --2- 172.21.15.47:0/1518069555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 0x556ccc9a7f40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb66c023040 tx=0x7fb66c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6bbefd25e40c41 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb668ff9640 1 -- 172.21.15.47:0/1518069555 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c00fce0 con 0x556ccc9d15c0 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb668ff9640 1 -- 172.21.15.47:0/1518069555 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c004bb0 con 0x556ccc9d15c0 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb668ff9640 1 -- 172.21.15.47:0/1518069555 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c032400 con 0x556ccc9d15c0 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb67b3b2740 1 -- 172.21.15.47:0/1518069555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 msgr2=0x556ccc9a7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/1518069555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 0x556ccc9a7f40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb66c023040 tx=0x7fb66c014000 comp rx=0 tx=0).stop 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb67b3b2740 1 -- 172.21.15.47:0/1518069555 shutdown_connections 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/1518069555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9fed80 0x556ccc9d6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/1518069555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9d19a0 0x556ccc9a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/1518069555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d15c0 0x556ccc9a7f40 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb67b3b2740 1 -- 172.21.15.47:0/1518069555 >> 172.21.15.47:0/1518069555 conn(0x556ccc9a4aa0 msgr2=0x556ccc9d8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.483+0000 7fb67b3b2740 1 -- 172.21.15.47:0/1518069555 shutdown_connections 2024-03-23T09:43:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.484+0000 7fb67b3b2740 1 -- 172.21.15.47:0/1518069555 wait complete. 2024-03-23T09:43:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.484+0000 7fb67b3b2740 1 -- start start 2024-03-23T09:43:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.484+0000 7fb67b3b2740 1 Processor -- start 2024-03-23T09:43:59.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb67b3b2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9d15c0 0x556ccc9a8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb67b3b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 0x556ccca55150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb67b3b2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9fed80 0x556ccca55690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb67b3b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccca59f50 con 0x556ccc9d19a0 2024-03-23T09:43:59.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb67b3b2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccca55bd0 con 0x556ccc9d15c0 2024-03-23T09:43:59.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb67b3b2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ccca07b90 con 0x556ccc9fed80 2024-03-23T09:43:59.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb67b3b2740 10 client.0 _ll_register_callbacks cb 0x7fb65c009130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:43:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb671d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 0x556ccca55150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb671d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 0x556ccca55150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38232/0 (socket says 172.21.15.47:38232) 2024-03-23T09:43:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb671d9b640 1 -- 172.21.15.47:0/3525676816 learned_addr learned my addr 172.21.15.47:0/3525676816 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb67259c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9d15c0 0x556ccc9a8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb672d9d640 1 --2- 172.21.15.47:0/3525676816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9fed80 0x556ccca55690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb671d9b640 1 -- 172.21.15.47:0/3525676816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9fed80 msgr2=0x556ccca55690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb671d9b640 1 --2- 172.21.15.47:0/3525676816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9fed80 0x556ccca55690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb671d9b640 1 -- 172.21.15.47:0/3525676816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9d15c0 msgr2=0x556ccc9a8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb671d9b640 1 --2- 172.21.15.47:0/3525676816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9d15c0 0x556ccc9a8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb671d9b640 1 -- 172.21.15.47:0/3525676816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ccc8b5710 con 0x556ccc9d19a0 2024-03-23T09:43:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb671d9b640 1 --2- 172.21.15.47:0/3525676816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 0x556ccca55150 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0097a0 tx=0x7fb65c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4cc78e42530e2e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.485+0000 7fb6517fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:43:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb6517fa640 1 -- 172.21.15.47:0/3525676816 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c044e10 con 0x556ccc9d19a0 2024-03-23T09:43:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb6517fa640 1 -- 172.21.15.47:0/3525676816 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb65c000e40 con 0x556ccc9d19a0 2024-03-23T09:43:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb67b3b2740 1 -- 172.21.15.47:0/3525676816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb66c00fe40 con 0x556ccc9d19a0 2024-03-23T09:43:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb6517fa640 1 -- 172.21.15.47:0/3525676816 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb65c031d10 con 0x556ccc9d19a0 2024-03-23T09:43:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb67b3b2740 10 client.6420 fetch_fsmap learned FSMap version 24 2024-03-23T09:43:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb67b3b2740 1 -- 172.21.15.47:0/3525676816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x556ccc8d48d0 con 0x556ccc9d19a0 2024-03-23T09:43:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb6517fa640 1 -- 172.21.15.47:0/3525676816 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c00fe40 con 0x556ccc9d19a0 2024-03-23T09:43:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb6517fa640 1 -- 172.21.15.47:0/3525676816 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb65c004040 con 0x556ccc9d19a0 2024-03-23T09:43:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb67b3b2740 10 client.6420 fetch_fsmap finished waiting for FSMap version 24 2024-03-23T09:43:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb67b3b2740 10 client.6420 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-23T09:43:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb67b3b2740 20 client.6420 populate_metadata read hostname 'smithi047' 2024-03-23T09:43:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/3525676816 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ccc9d6b70 0x556ccc9fda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb67b3b2740 4 client.6420 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-03-23T09:43:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb67b3b2740 1 -- 172.21.15.47:0/3525676816 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x556ccc613790 con 0x556ccc9d6b70 2024-03-23T09:43:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.486+0000 7fb67259c640 1 --2- 172.21.15.47:0/3525676816 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ccc9d6b70 0x556ccc9fda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.487+0000 7fb6517fa640 10 client.6420 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:43:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.487+0000 7fb67259c640 1 --2- 172.21.15.47:0/3525676816 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ccc9d6b70 0x556ccc9fda20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4333565f320adffd server_cookie=838d7e3aee80bee4 in_seq=0 out_seq=0 2024-03-23T09:43:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.487+0000 7fb6517fa640 1 -- 172.21.15.47:0/3525676816 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556ccc613790 con 0x556ccc9d6b70 2024-03-23T09:43:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.487+0000 7fb6517fa640 10 client.6420 handle_command_reply: tid=0 2024-03-23T09:43:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.487+0000 7fb67b3b2740 1 client.6420 shutdown 2024-03-23T09:43:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.487+0000 7fb67b3b2740 1 -- 172.21.15.47:0/3525676816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 msgr2=0x556ccca55150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.487+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/3525676816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 0x556ccca55150 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0097a0 tx=0x7fb65c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67b3b2740 1 -- 172.21.15.47:0/3525676816 shutdown_connections 2024-03-23T09:43:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/3525676816 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ccc9d6b70 0x556ccc9fda20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/3525676816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ccc9fed80 0x556ccca55690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/3525676816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ccc9d19a0 0x556ccca55150 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67b3b2740 1 --2- 172.21.15.47:0/3525676816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ccc9d15c0 0x556ccc9a8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb6517fa640 0 client.6420 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:43:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67b3b2740 1 -- 172.21.15.47:0/3525676816 >> 172.21.15.47:0/3525676816 conn(0x556ccc9a4aa0 msgr2=0x556ccc9a3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67b3b2740 1 -- 172.21.15.47:0/3525676816 shutdown_connections 2024-03-23T09:43:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67b3b2740 1 -- 172.21.15.47:0/3525676816 wait complete. 2024-03-23T09:43:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67b3b2740 20 client.6420 trim_cache size 0 max 16384 2024-03-23T09:43:59.493 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:43:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-23T09:43:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "b4ec6f77-8e49-44de-a1d0-d30e1ea8f610", 2024-03-23T09:43:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-23T09:43:59.493 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:43:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67883e640 1 -- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb65406ac00 msgr2=0x7fb65406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67883e640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb65406ac00 0x7fb65406d0c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb66c004f60 tx=0x7fb66c007e30 comp rx=0 tx=0).stop 2024-03-23T09:43:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67883e640 1 -- 172.21.15.47:0/3778466660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb67415d0f0 msgr2=0x7fb6740b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67883e640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb67415d0f0 0x7fb6740b2420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb66400cbc0 tx=0x7fb66400a430 comp rx=0 tx=0).stop 2024-03-23T09:43:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67883e640 1 -- 172.21.15.47:0/3778466660 shutdown_connections 2024-03-23T09:43:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67883e640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb65406ac00 0x7fb65406d0c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67883e640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb67415d0f0 0x7fb6740b2420 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67883e640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb67415a860 0x7fb674152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67883e640 1 --2- 172.21.15.47:0/3778466660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6740cf190 0x7fb6740b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67883e640 1 -- 172.21.15.47:0/3778466660 >> 172.21.15.47:0/3778466660 conn(0x7fb6740659d0 msgr2=0x7fb6740d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67883e640 1 -- 172.21.15.47:0/3778466660 shutdown_connections 2024-03-23T09:43:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.488+0000 7fb67883e640 1 -- 172.21.15.47:0/3778466660 wait complete. 2024-03-23T09:43:59.495 INFO:tasks.ceph.mds.d.smithi175.stderr:2024-03-23T09:43:59.488+0000 7fa9142b1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6369=0-4194304@1} caps={6369=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6369 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x559dc139fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T09:43:59.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:43:59.499 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-23T09:43:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.601+0000 7f4c4ed20640 1 Processor -- start 2024-03-23T09:43:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.601+0000 7f4c4ed20640 1 -- start start 2024-03-23T09:43:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c4ed20640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4c4815ce10 0x7f4c480cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c4ed20640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c480b85d0 0x7f4c480cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c4ed20640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c48161600 con 0x7f4c4815ca30 2024-03-23T09:43:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c4ed20640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c48160e50 con 0x7f4c4815ce10 2024-03-23T09:43:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c4ed20640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c480d25e0 con 0x7f4c480b85d0 2024-03-23T09:43:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c4cabc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4c4815ca30 0x7f4c48160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44790/0 (socket says 172.21.15.47:44790) 2024-03-23T09:43:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c4cabc640 1 -- 172.21.15.47:0/629113110 learned_addr learned my addr 172.21.15.47:0/629113110 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 192286884 0 0) 0x7f4c48161600 con 0x7f4c4815ca30 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c4d2bd640 1 --2- 172.21.15.47:0/629113110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c480b85d0 0x7f4c480cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/629113110 >> v2:172.21.15.175:3300/0 conn(0x7f4c4815ce10 0x7f4c480cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c30003520 con 0x7f4c4815ca30 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2326722009 0 0) 0x7f4c30003520 con 0x7f4c4815ca30 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c480b85d0 msgr2=0x7f4c480cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c3f7fe640 1 --2- 172.21.15.47:0/629113110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c480b85d0 0x7f4c480cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 >> v2:172.21.15.175:3300/0 conn(0x7f4c4815ce10 msgr2=0x7f4c480cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c3f7fe640 1 --2- 172.21.15.47:0/629113110 >> v2:172.21.15.175:3300/0 conn(0x7f4c4815ce10 0x7f4c480cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c48161040 con 0x7f4c4815ca30 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f4c44001670 con 0x7f4c4815ca30 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 >> v1:172.21.15.47:6789/0 conn(0x7f4c4815ca30 legacy=0x7f4c48160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c3f7fe640 1 --2- 172.21.15.47:0/629113110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c480b85d0 0x7f4c30003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.602+0000 7f4c3f7fe640 1 --2- 172.21.15.47:0/629113110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c30003ef0 0x7f4c300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.603+0000 7f4c3f7fe640 1 --2- 172.21.15.47:0/629113110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c30004d00 0x7f4c30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.603+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c480d25e0 con 0x7f4c30003ef0 2024-03-23T09:43:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.603+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3000b980 con 0x7f4c30004d00 2024-03-23T09:43:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.603+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3000bac0 con 0x7f4c480b85d0 2024-03-23T09:43:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.603+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/629113110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c30003ef0 0x7f4c300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.603+0000 7f4c4cabc640 1 --2- 172.21.15.47:0/629113110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c480b85d0 0x7f4c30003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.603+0000 7f4c4d2bd640 1 --2- 172.21.15.47:0/629113110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c30004d00 0x7f4c30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.603+0000 7f4c3ffff640 1 -- 172.21.15.47:0/629113110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c480b85d0 msgr2=0x7f4c30003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.603+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/629113110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c480b85d0 0x7f4c30003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.603+0000 7f4c3ffff640 1 -- 172.21.15.47:0/629113110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c30004d00 msgr2=0x7f4c30005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.603+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/629113110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c30004d00 0x7f4c30005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.603+0000 7f4c3ffff640 1 -- 172.21.15.47:0/629113110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c30003520 con 0x7f4c30003ef0 2024-03-23T09:43:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.604+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/629113110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c30003ef0 0x7f4c300047c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009ac0 tx=0x7f4c300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cb64a1494d8d8c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.604+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3802c6d0 con 0x7f4c30003ef0 2024-03-23T09:43:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.604+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3802c830 con 0x7f4c30003ef0 2024-03-23T09:43:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.604+0000 7f4c3f7fe640 1 -- 172.21.15.47:0/629113110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3802cbc0 con 0x7f4c30003ef0 2024-03-23T09:43:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.604+0000 7f4c4ed20640 1 -- 172.21.15.47:0/629113110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c30003ef0 msgr2=0x7f4c300047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.604+0000 7f4c4ed20640 1 --2- 172.21.15.47:0/629113110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c30003ef0 0x7f4c300047c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009ac0 tx=0x7f4c300057f0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.604+0000 7f4c4d2bd640 1 -- 172.21.15.47:0/629113110 reap_dead start 2024-03-23T09:43:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.604+0000 7f4c4ed20640 1 -- 172.21.15.47:0/629113110 shutdown_connections 2024-03-23T09:43:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.604+0000 7f4c4ed20640 1 -- 172.21.15.47:0/629113110 >> 172.21.15.47:0/629113110 conn(0x7f4c4806b160 msgr2=0x7f4c4806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.604+0000 7f4c4ed20640 1 -- 172.21.15.47:0/629113110 shutdown_connections 2024-03-23T09:43:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.604+0000 7f4c4ed20640 1 -- 172.21.15.47:0/629113110 wait complete. 2024-03-23T09:43:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.605+0000 7f4c4ed20640 1 Processor -- start 2024-03-23T09:43:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.605+0000 7f4c4ed20640 1 -- start start 2024-03-23T09:43:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.606+0000 7f4c4ed20640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c30003ef0 0x7f4c480c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.606+0000 7f4c4ed20640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c4815ab80 0x7f4c4815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.606+0000 7f4c4ed20640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c480b75f0 0x7f4c480b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.606+0000 7f4c4ed20640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4815b540 con 0x7f4c30003ef0 2024-03-23T09:43:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.606+0000 7f4c4ed20640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c480b7fb0 con 0x7f4c480b75f0 2024-03-23T09:43:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.606+0000 7f4c4ed20640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4815b720 con 0x7f4c4815ab80 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.606+0000 7f4c4cabc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c30003ef0 0x7f4c480c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.606+0000 7f4c4cabc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c30003ef0 0x7f4c480c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38252/0 (socket says 172.21.15.47:38252) 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.606+0000 7f4c4cabc640 1 -- 172.21.15.47:0/3580840374 learned_addr learned my addr 172.21.15.47:0/3580840374 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c4d2bd640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c480b75f0 0x7f4c480b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c4815ab80 0x7f4c4815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c4cabc640 1 -- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c4815ab80 msgr2=0x7f4c4815b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c4cabc640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c4815ab80 0x7f4c4815b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c4cabc640 1 -- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c480b75f0 msgr2=0x7f4c480b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c4cabc640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c480b75f0 0x7f4c480b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c4cabc640 1 -- 172.21.15.47:0/3580840374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4815ba00 con 0x7f4c30003ef0 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c4815ab80 0x7f4c4815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c4d2bd640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c480b75f0 0x7f4c480b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c4cabc640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c30003ef0 0x7f4c480c8300 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4c44007310 tx=0x7f4c440077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c3d7fa640 1 -- 172.21.15.47:0/3580840374 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400f940 con 0x7f4c30003ef0 2024-03-23T09:43:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c3d7fa640 1 -- 172.21.15.47:0/3580840374 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4400fac0 con 0x7f4c30003ef0 2024-03-23T09:43:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c3d7fa640 1 -- 172.21.15.47:0/3580840374 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400fe70 con 0x7f4c30003ef0 2024-03-23T09:43:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c4ed20640 1 -- 172.21.15.47:0/3580840374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4815bce0 con 0x7f4c30003ef0 2024-03-23T09:43:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.607+0000 7f4c4ed20640 1 -- 172.21.15.47:0/3580840374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c48158e00 con 0x7f4c30003ef0 2024-03-23T09:43:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.609+0000 7f4c3d7fa640 1 -- 172.21.15.47:0/3580840374 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4c4401d460 con 0x7f4c30003ef0 2024-03-23T09:43:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.609+0000 7f4c3d7fa640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4c3406ac30 0x7f4c3406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.609+0000 7f4c3d7fa640 1 -- 172.21.15.47:0/3580840374 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c440894d0 con 0x7f4c30003ef0 2024-03-23T09:43:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.609+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4c3406ac30 0x7f4c3406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.610+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4c3406ac30 0x7f4c3406d0f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4c38036000 tx=0x7f4c38030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.610+0000 7f4c5196b740 1 Processor -- start 2024-03-23T09:43:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.610+0000 7f4c5196b740 1 -- start start 2024-03-23T09:43:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c5196b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 0x556ae611dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c5196b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60ed850 0x556ae611aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c5196b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae60f13b0 0x556ae611afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c5196b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae605ddd0 con 0x556ae60ed850 2024-03-23T09:43:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c5196b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae605b560 con 0x556ae611d830 2024-03-23T09:43:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c5196b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ae6056e50 con 0x556ae60f13b0 2024-03-23T09:43:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c3ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60ed850 0x556ae611aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c3ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60ed850 0x556ae611aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38256/0 (socket says 172.21.15.47:38256) 2024-03-23T09:43:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c3ffff640 1 -- 172.21.15.47:0/1614145122 learned_addr learned my addr 172.21.15.47:0/1614145122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c4cabc640 1 --2- 172.21.15.47:0/1614145122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 0x556ae611dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c4d2bd640 1 --2- 172.21.15.47:0/1614145122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae60f13b0 0x556ae611afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c3ffff640 1 -- 172.21.15.47:0/1614145122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae60f13b0 msgr2=0x556ae611afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/1614145122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae60f13b0 0x556ae611afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c3ffff640 1 -- 172.21.15.47:0/1614145122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 msgr2=0x556ae611dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/1614145122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 0x556ae611dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c3ffff640 1 -- 172.21.15.47:0/1614145122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ae601e470 con 0x556ae60ed850 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c4cabc640 1 --2- 172.21.15.47:0/1614145122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 0x556ae611dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.611+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/1614145122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60ed850 0x556ae611aaa0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4c3802c2d0 tx=0x7f4c3802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b138b6f4c00704cc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.612+0000 7f4c2effd640 1 -- 172.21.15.47:0/1614145122 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38003610 con 0x556ae60ed850 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.612+0000 7f4c2effd640 1 -- 172.21.15.47:0/1614145122 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38003770 con 0x556ae60ed850 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.612+0000 7f4c2effd640 1 -- 172.21.15.47:0/1614145122 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38003b20 con 0x556ae60ed850 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.612+0000 7f4c5196b740 1 -- 172.21.15.47:0/1614145122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60ed850 msgr2=0x556ae611aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.612+0000 7f4c5196b740 1 --2- 172.21.15.47:0/1614145122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60ed850 0x556ae611aaa0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4c3802c2d0 tx=0x7f4c3802e000 comp rx=0 tx=0).stop 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.612+0000 7f4c5196b740 1 -- 172.21.15.47:0/1614145122 shutdown_connections 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.612+0000 7f4c5196b740 1 --2- 172.21.15.47:0/1614145122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae60f13b0 0x556ae611afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.612+0000 7f4c5196b740 1 --2- 172.21.15.47:0/1614145122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60ed850 0x556ae611aaa0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.612+0000 7f4c5196b740 1 --2- 172.21.15.47:0/1614145122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 0x556ae611dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.612+0000 7f4c5196b740 1 -- 172.21.15.47:0/1614145122 >> 172.21.15.47:0/1614145122 conn(0x556ae60efac0 msgr2=0x556ae6138640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.612+0000 7f4c5196b740 1 -- 172.21.15.47:0/1614145122 shutdown_connections 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.612+0000 7f4c5196b740 1 -- 172.21.15.47:0/1614145122 wait complete. 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.613+0000 7f4c5196b740 1 -- start start 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.613+0000 7f4c5196b740 1 Processor -- start 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c5196b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60f13b0 0x556ae610ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c5196b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 0x556ae61204f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c5196b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae6115d20 0x556ae6120a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c5196b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae6015280 con 0x556ae60f13b0 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c5196b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae6014f20 con 0x556ae611d830 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c5196b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ae5fda3c0 con 0x556ae6115d20 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c5196b740 10 client.0 _ll_register_callbacks cb 0x556ae602c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c4cabc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60f13b0 0x556ae610ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c4cabc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60f13b0 0x556ae610ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38262/0 (socket says 172.21.15.47:38262) 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c4cabc640 1 -- 172.21.15.47:0/333895121 learned_addr learned my addr 172.21.15.47:0/333895121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/333895121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 0x556ae61204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c4d2bd640 1 --2- 172.21.15.47:0/333895121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae6115d20 0x556ae6120a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c4cabc640 1 -- 172.21.15.47:0/333895121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae6115d20 msgr2=0x556ae6120a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c4cabc640 1 --2- 172.21.15.47:0/333895121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae6115d20 0x556ae6120a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c4cabc640 1 -- 172.21.15.47:0/333895121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 msgr2=0x556ae61204f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c4cabc640 1 --2- 172.21.15.47:0/333895121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 0x556ae61204f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c4cabc640 1 -- 172.21.15.47:0/333895121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c30003520 con 0x556ae60f13b0 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c4cabc640 1 --2- 172.21.15.47:0/333895121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60f13b0 0x556ae610ad60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400f5a0 tx=0x7f4c440113f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ccc1e1686832d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c277fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c277fe640 1 -- 172.21.15.47:0/333895121 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c440892e0 con 0x556ae60f13b0 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c277fe640 1 -- 172.21.15.47:0/333895121 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c440607f0 con 0x556ae60f13b0 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c5196b740 1 -- 172.21.15.47:0/333895121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ae6080690 con 0x556ae60f13b0 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c277fe640 1 -- 172.21.15.47:0/333895121 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c44060360 con 0x556ae60f13b0 2024-03-23T09:43:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.614+0000 7f4c5196b740 10 client.6432 fetch_fsmap learned FSMap version 24 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.615+0000 7f4c5196b740 1 -- 172.21.15.47:0/333895121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x556ae60554d0 con 0x556ae60f13b0 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.615+0000 7f4c277fe640 1 -- 172.21.15.47:0/333895121 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c44074080 con 0x556ae60f13b0 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.615+0000 7f4c277fe640 1 -- 172.21.15.47:0/333895121 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4c44012070 con 0x556ae60f13b0 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.615+0000 7f4c5196b740 10 client.6432 fetch_fsmap finished waiting for FSMap version 24 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.615+0000 7f4c5196b740 10 client.6432 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.615+0000 7f4c5196b740 20 client.6432 populate_metadata read hostname 'smithi047' 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.615+0000 7f4c5196b740 1 --2- 172.21.15.47:0/333895121 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ae614bb10 0x556ae614bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.615+0000 7f4c5196b740 4 client.6432 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.615+0000 7f4c5196b740 1 -- 172.21.15.47:0/333895121 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ae6048eb0 con 0x556ae614bb10 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.615+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/333895121 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ae614bb10 0x556ae614bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.615+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/333895121 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ae614bb10 0x556ae614bef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cc2e21daa0c2cd5 server_cookie=b57a8f272cb15f43 in_seq=0 out_seq=0 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.615+0000 7f4c277fe640 10 client.6432 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c277fe640 1 -- 172.21.15.47:0/333895121 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556ae6048eb0 con 0x556ae614bb10 2024-03-23T09:43:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c277fe640 10 client.6432 handle_command_reply: tid=0 2024-03-23T09:43:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c5196b740 1 client.6432 shutdown 2024-03-23T09:43:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c5196b740 1 -- 172.21.15.47:0/333895121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60f13b0 msgr2=0x556ae610ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c5196b740 1 --2- 172.21.15.47:0/333895121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60f13b0 0x556ae610ad60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400f5a0 tx=0x7f4c440113f0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c5196b740 1 -- 172.21.15.47:0/333895121 shutdown_connections 2024-03-23T09:43:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c5196b740 1 --2- 172.21.15.47:0/333895121 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ae614bb10 0x556ae614bef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c5196b740 1 --2- 172.21.15.47:0/333895121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae6115d20 0x556ae6120a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c5196b740 1 --2- 172.21.15.47:0/333895121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 0x556ae61204f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c277fe640 0 client.6432 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:43:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c5196b740 1 --2- 172.21.15.47:0/333895121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae60f13b0 0x556ae610ad60 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c5196b740 1 -- 172.21.15.47:0/333895121 >> 172.21.15.47:0/333895121 conn(0x556ae60ed850 msgr2=0x556ae60f1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c5196b740 1 -- 172.21.15.47:0/333895121 shutdown_connections 2024-03-23T09:43:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.616+0000 7f4c5196b740 1 -- 172.21.15.47:0/333895121 wait complete. 2024-03-23T09:43:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.617+0000 7f4c5196b740 20 client.6432 trim_cache size 0 max 16384 2024-03-23T09:43:59.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.631+0000 7f4c5196b740 1 Processor -- start 2024-03-23T09:43:59.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.631+0000 7f4c5196b740 1 -- start start 2024-03-23T09:43:59.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.631+0000 7f4c5196b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae60f13b0 0x556ae6114d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.631+0000 7f4c5196b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 0x556ae60ec910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.631+0000 7f4c5196b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 0x556ae6138a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.631+0000 7f4c5196b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae615aea0 con 0x556ae6115d20 2024-03-23T09:43:59.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.631+0000 7f4c5196b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae615ac70 con 0x556ae611d830 2024-03-23T09:43:59.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.631+0000 7f4c5196b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ae5fe2fb0 con 0x556ae60f13b0 2024-03-23T09:43:59.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.632+0000 7f4c3ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 0x556ae60ec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.632+0000 7f4c3ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 0x556ae60ec910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38274/0 (socket says 172.21.15.47:38274) 2024-03-23T09:43:59.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.632+0000 7f4c3ffff640 1 -- 172.21.15.47:0/3612296649 learned_addr learned my addr 172.21.15.47:0/3612296649 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.632+0000 7f4c3ffff640 1 -- 172.21.15.47:0/3612296649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae60f13b0 msgr2=0x556ae6114d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.632+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/3612296649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae60f13b0 0x556ae6114d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.632+0000 7f4c3ffff640 1 -- 172.21.15.47:0/3612296649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 msgr2=0x556ae6138a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.632+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/3612296649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 0x556ae6138a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.632+0000 7f4c3ffff640 1 -- 172.21.15.47:0/3612296649 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ae60899e0 con 0x556ae6115d20 2024-03-23T09:43:59.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.632+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/3612296649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 0x556ae60ec910 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009be0 tx=0x7f4c3803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11605d7ed5b35730 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.632+0000 7f4c2f7fe640 1 -- 172.21.15.47:0/3612296649 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38003610 con 0x556ae6115d20 2024-03-23T09:43:59.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.632+0000 7f4c2f7fe640 1 -- 172.21.15.47:0/3612296649 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38005990 con 0x556ae6115d20 2024-03-23T09:43:59.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.632+0000 7f4c2f7fe640 1 -- 172.21.15.47:0/3612296649 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38005d20 con 0x556ae6115d20 2024-03-23T09:43:59.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.633+0000 7f4c5196b740 1 -- 172.21.15.47:0/3612296649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 msgr2=0x556ae60ec910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.633+0000 7f4c5196b740 1 --2- 172.21.15.47:0/3612296649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 0x556ae60ec910 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009be0 tx=0x7f4c3803f000 comp rx=0 tx=0).stop 2024-03-23T09:43:59.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.633+0000 7f4c5196b740 1 -- 172.21.15.47:0/3612296649 shutdown_connections 2024-03-23T09:43:59.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.633+0000 7f4c5196b740 1 --2- 172.21.15.47:0/3612296649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae611d830 0x556ae6138a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.633+0000 7f4c5196b740 1 --2- 172.21.15.47:0/3612296649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 0x556ae60ec910 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.633+0000 7f4c5196b740 1 --2- 172.21.15.47:0/3612296649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae60f13b0 0x556ae6114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.633+0000 7f4c5196b740 1 -- 172.21.15.47:0/3612296649 >> 172.21.15.47:0/3612296649 conn(0x556ae60ed850 msgr2=0x556ae60e9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.633+0000 7f4c5196b740 1 -- 172.21.15.47:0/3612296649 shutdown_connections 2024-03-23T09:43:59.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.633+0000 7f4c5196b740 1 -- 172.21.15.47:0/3612296649 wait complete. 2024-03-23T09:43:59.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.633+0000 7f4c5196b740 1 -- start start 2024-03-23T09:43:59.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.633+0000 7f4c5196b740 1 Processor -- start 2024-03-23T09:43:59.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c5196b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae60f13b0 0x556ae60f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c5196b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 0x556ae619e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c5196b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae611d830 0x556ae619e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c5196b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae619ec70 con 0x556ae6115d20 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c5196b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae6114f60 con 0x556ae60f13b0 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c5196b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ae60f2920 con 0x556ae611d830 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c5196b740 10 client.0 _ll_register_callbacks cb 0x7f4c38037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c3ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 0x556ae619e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c3ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 0x556ae619e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38276/0 (socket says 172.21.15.47:38276) 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c3ffff640 1 -- 172.21.15.47:0/91153376 learned_addr learned my addr 172.21.15.47:0/91153376 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c4d2bd640 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae611d830 0x556ae619e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c4cabc640 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae60f13b0 0x556ae60f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c3ffff640 1 -- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae611d830 msgr2=0x556ae619e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae611d830 0x556ae619e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c3ffff640 1 -- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae60f13b0 msgr2=0x556ae60f2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:43:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae60f13b0 0x556ae60f2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c3ffff640 1 -- 172.21.15.47:0/91153376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ae601e470 con 0x556ae6115d20 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c4d2bd640 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae611d830 0x556ae619e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.634+0000 7f4c4cabc640 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae60f13b0 0x556ae60f2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c3ffff640 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 0x556ae619e180 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009be0 tx=0x7f4c380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2a147362a3f1f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c277fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c277fe640 1 -- 172.21.15.47:0/91153376 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38005e00 con 0x556ae6115d20 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c277fe640 1 -- 172.21.15.47:0/91153376 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c38062aa0 con 0x556ae6115d20 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c5196b740 1 -- 172.21.15.47:0/91153376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c38003770 con 0x556ae6115d20 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c277fe640 1 -- 172.21.15.47:0/91153376 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c38062c60 con 0x556ae6115d20 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c5196b740 10 client.6438 fetch_fsmap learned FSMap version 24 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c5196b740 1 -- 172.21.15.47:0/91153376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x7f4c48160240 con 0x556ae6115d20 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c277fe640 1 -- 172.21.15.47:0/91153376 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38003770 con 0x556ae6115d20 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c277fe640 1 -- 172.21.15.47:0/91153376 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4c380038e0 con 0x556ae6115d20 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c5196b740 10 client.6438 fetch_fsmap finished waiting for FSMap version 24 2024-03-23T09:43:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c5196b740 10 client.6438 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c5196b740 20 client.6438 populate_metadata read hostname 'smithi047' 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c5196b740 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ae6116600 0x556ae61169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c5196b740 4 client.6438 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c5196b740 1 -- 172.21.15.47:0/91153376 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556ae5d4a4e0 con 0x556ae6116600 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.635+0000 7f4c4cabc640 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ae6116600 0x556ae61169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c4cabc640 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ae6116600 0x556ae61169e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=535704309e4d639a server_cookie=9e4d6b856068d303 in_seq=0 out_seq=0 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c277fe640 10 client.6438 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c277fe640 1 -- 172.21.15.47:0/91153376 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x556ae5d4a4e0 con 0x556ae6116600 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c277fe640 10 client.6438 handle_command_reply: tid=0 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c5196b740 1 client.6438 shutdown 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c5196b740 1 -- 172.21.15.47:0/91153376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 msgr2=0x556ae619e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c5196b740 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 0x556ae619e180 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009be0 tx=0x7f4c380046c0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c5196b740 1 -- 172.21.15.47:0/91153376 shutdown_connections 2024-03-23T09:43:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c5196b740 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x556ae6116600 0x556ae61169e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c5196b740 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556ae611d830 0x556ae619e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c5196b740 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ae6115d20 0x556ae619e180 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c277fe640 0 client.6438 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:43:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c5196b740 1 --2- 172.21.15.47:0/91153376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ae60f13b0 0x556ae60f2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c5196b740 1 -- 172.21.15.47:0/91153376 >> 172.21.15.47:0/91153376 conn(0x556ae60ed850 msgr2=0x556ae61214e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c5196b740 1 -- 172.21.15.47:0/91153376 shutdown_connections 2024-03-23T09:43:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c5196b740 1 -- 172.21.15.47:0/91153376 wait complete. 2024-03-23T09:43:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.636+0000 7f4c5196b740 20 client.6438 trim_cache size 0 max 16384 2024-03-23T09:43:59.642 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:43:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T09:43:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-23T09:43:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "b4ec6f77-8e49-44de-a1d0-d30e1ea8f610": { 2024-03-23T09:43:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-23T09:43:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "b4ec6f77-8e49-44de-a1d0-d30e1ea8f610", 2024-03-23T09:43:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive" 2024-03-23T09:43:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:43:59.643 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:43:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.637+0000 7f4c4ed20640 1 -- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4c3406ac30 msgr2=0x7f4c3406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.637+0000 7f4c4ed20640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4c3406ac30 0x7f4c3406d0f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4c38036000 tx=0x7f4c38030000 comp rx=0 tx=0).stop 2024-03-23T09:43:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.637+0000 7f4c4ed20640 1 -- 172.21.15.47:0/3580840374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c30003ef0 msgr2=0x7f4c480c8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:43:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.637+0000 7f4c4ed20640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c30003ef0 0x7f4c480c8300 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4c44007310 tx=0x7f4c440077d0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.637+0000 7f4c4ed20640 1 -- 172.21.15.47:0/3580840374 shutdown_connections 2024-03-23T09:43:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.637+0000 7f4c4ed20640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4c3406ac30 0x7f4c3406d0f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.637+0000 7f4c4ed20640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c480b75f0 0x7f4c480b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.637+0000 7f4c4ed20640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c4815ab80 0x7f4c4815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.637+0000 7f4c4ed20640 1 --2- 172.21.15.47:0/3580840374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c30003ef0 0x7f4c480c8300 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:43:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.637+0000 7f4c4ed20640 1 -- 172.21.15.47:0/3580840374 >> 172.21.15.47:0/3580840374 conn(0x7f4c4806b160 msgr2=0x7f4c480cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:43:59.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.637+0000 7f4c4ed20640 1 -- 172.21.15.47:0/3580840374 shutdown_connections 2024-03-23T09:43:59.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:43:59.637+0000 7f4c4ed20640 1 -- 172.21.15.47:0/3580840374 wait complete. 2024-03-23T09:43:59.647 INFO:tasks.cephfs.filesystem:scrub status for tag:b4ec6f77-8e49-44de-a1d0-d30e1ea8f610 - {'path': '/', 'tag': 'b4ec6f77-8e49-44de-a1d0-d30e1ea8f610', 'options': 'recursive'} 2024-03-23T09:44:04.065 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:44:04.065+0000 7fefe5743640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T09:44:07.375 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:44:07.378 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:44:29.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:44:29.648 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-23T09:44:29.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.762+0000 7fa626355640 1 Processor -- start 2024-03-23T09:44:29.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.762+0000 7fa626355640 1 -- start start 2024-03-23T09:44:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.762+0000 7fa626355640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa6200cde10 0x7fa620160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.762+0000 7fa626355640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa620161630 con 0x7fa6200b8350 2024-03-23T09:44:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.762+0000 7fa626355640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6200d1030 con 0x7fa62015c990 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.762+0000 7fa626355640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6200cfe10 con 0x7fa6200cde10 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa6248f2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa6200b8350 0x7fa6200d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32862/0 (socket says 172.21.15.47:32862) 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa6248f2640 1 -- 172.21.15.47:0/4259530787 learned_addr learned my addr 172.21.15.47:0/4259530787 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898598237 0 0) 0x7fa620161630 con 0x7fa6200b8350 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa608003520 con 0x7fa6200b8350 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61f7fe640 1 --2- 172.21.15.47:0/4259530787 >> v2:172.21.15.175:3300/0 conn(0x7fa6200cde10 0x7fa620160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2746586975 0 0) 0x7fa608003520 con 0x7fa6200b8350 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 >> v2:172.21.15.175:3300/0 conn(0x7fa6200cde10 msgr2=0x7fa620160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 --2- 172.21.15.47:0/4259530787 >> v2:172.21.15.175:3300/0 conn(0x7fa6200cde10 0x7fa620160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 >> v1:172.21.15.175:6789/0 conn(0x7fa62015c990 legacy=0x7fa6200cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6200b5b60 con 0x7fa6200b8350 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fa6180019f0 con 0x7fa6200b8350 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 >> v1:172.21.15.47:6789/0 conn(0x7fa6200b8350 legacy=0x7fa6200d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 --2- 172.21.15.47:0/4259530787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608003970 0x7fa608003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 --2- 172.21.15.47:0/4259530787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6080047c0 0x7fa608004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 --2- 172.21.15.47:0/4259530787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa608005180 0x7fa60800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6200cfe10 con 0x7fa608003970 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60800c120 con 0x7fa6080047c0 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60800c2e0 con 0x7fa608005180 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.763+0000 7fa61ffff640 1 --2- 172.21.15.47:0/4259530787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608003970 0x7fa608003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.764+0000 7fa61f7fe640 1 --2- 172.21.15.47:0/4259530787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6080047c0 0x7fa608004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.764+0000 7fa6248f2640 1 --2- 172.21.15.47:0/4259530787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa608005180 0x7fa60800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.764+0000 7fa61ffff640 1 -- 172.21.15.47:0/4259530787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa608005180 msgr2=0x7fa60800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.764+0000 7fa61ffff640 1 --2- 172.21.15.47:0/4259530787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa608005180 0x7fa60800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.764+0000 7fa61ffff640 1 -- 172.21.15.47:0/4259530787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6080047c0 msgr2=0x7fa608004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.764+0000 7fa61ffff640 1 --2- 172.21.15.47:0/4259530787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6080047c0 0x7fa608004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.764+0000 7fa61ffff640 1 -- 172.21.15.47:0/4259530787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa608003520 con 0x7fa608003970 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.764+0000 7fa6248f2640 1 -- 172.21.15.47:0/4259530787 reap_dead start 2024-03-23T09:44:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.764+0000 7fa61f7fe640 1 --2- 172.21.15.47:0/4259530787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6080047c0 0x7fa608004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:44:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.764+0000 7fa61ffff640 1 --2- 172.21.15.47:0/4259530787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608003970 0x7fa608003d90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa610001750 tx=0x7fa6080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1373e88a6f806d24 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.765+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6100052d0 con 0x7fa608003970 2024-03-23T09:44:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.765+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61002e050 con 0x7fa608003970 2024-03-23T09:44:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.765+0000 7fa61effd640 1 -- 172.21.15.47:0/4259530787 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610003830 con 0x7fa608003970 2024-03-23T09:44:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.765+0000 7fa626355640 1 -- 172.21.15.47:0/4259530787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608003970 msgr2=0x7fa608003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.765+0000 7fa626355640 1 --2- 172.21.15.47:0/4259530787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608003970 0x7fa608003d90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa610001750 tx=0x7fa6080057f0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.765+0000 7fa626355640 1 -- 172.21.15.47:0/4259530787 shutdown_connections 2024-03-23T09:44:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.765+0000 7fa626355640 1 --2- 172.21.15.47:0/4259530787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608003970 0x7fa608003d90 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.765+0000 7fa626355640 1 -- 172.21.15.47:0/4259530787 >> 172.21.15.47:0/4259530787 conn(0x7fa6200659d0 msgr2=0x7fa620065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.765+0000 7fa626355640 1 -- 172.21.15.47:0/4259530787 shutdown_connections 2024-03-23T09:44:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.765+0000 7fa626355640 1 -- 172.21.15.47:0/4259530787 wait complete. 2024-03-23T09:44:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.766+0000 7fa626355640 1 Processor -- start 2024-03-23T09:44:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.767+0000 7fa626355640 1 -- start start 2024-03-23T09:44:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.767+0000 7fa626355640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa608003970 0x7fa62015f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.767+0000 7fa626355640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62015ea80 0x7fa6201596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.767+0000 7fa626355640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa620150310 0x7fa62015a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.767+0000 7fa626355640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62015e5f0 con 0x7fa62015ea80 2024-03-23T09:44:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.767+0000 7fa626355640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62015dcc0 con 0x7fa608003970 2024-03-23T09:44:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.767+0000 7fa626355640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62015dfa0 con 0x7fa620150310 2024-03-23T09:44:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.767+0000 7fa61f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62015ea80 0x7fa6201596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.767+0000 7fa61f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62015ea80 0x7fa6201596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60462/0 (socket says 172.21.15.47:60462) 2024-03-23T09:44:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.767+0000 7fa61f7fe640 1 -- 172.21.15.47:0/3004630074 learned_addr learned my addr 172.21.15.47:0/3004630074 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa61ffff640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa608003970 0x7fa62015f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa6248f2640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa620150310 0x7fa62015a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa61f7fe640 1 -- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa620150310 msgr2=0x7fa62015a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa61f7fe640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa620150310 0x7fa62015a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa61f7fe640 1 -- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa608003970 msgr2=0x7fa62015f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa61f7fe640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa608003970 0x7fa62015f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa61f7fe640 1 -- 172.21.15.47:0/3004630074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62015e280 con 0x7fa62015ea80 2024-03-23T09:44:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa6248f2640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa620150310 0x7fa62015a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:44:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa61ffff640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa608003970 0x7fa62015f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:44:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa61f7fe640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62015ea80 0x7fa6201596c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa6140114e0 tx=0x7fa6140119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa61cff9640 1 -- 172.21.15.47:0/3004630074 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614015870 con 0x7fa62015ea80 2024-03-23T09:44:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa61cff9640 1 -- 172.21.15.47:0/3004630074 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6140159d0 con 0x7fa62015ea80 2024-03-23T09:44:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa626355640 1 -- 172.21.15.47:0/3004630074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6201507c0 con 0x7fa62015ea80 2024-03-23T09:44:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa61cff9640 1 -- 172.21.15.47:0/3004630074 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614015d80 con 0x7fa62015ea80 2024-03-23T09:44:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.768+0000 7fa626355640 1 -- 172.21.15.47:0/3004630074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62015af00 con 0x7fa62015ea80 2024-03-23T09:44:29.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.770+0000 7fa61cff9640 1 -- 172.21.15.47:0/3004630074 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa61401d400 con 0x7fa62015ea80 2024-03-23T09:44:29.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.770+0000 7fa61cff9640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa60c06ac30 0x7fa60c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.771+0000 7fa61cff9640 1 -- 172.21.15.47:0/3004630074 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa614089a30 con 0x7fa62015ea80 2024-03-23T09:44:29.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.771+0000 7fa61ffff640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa60c06ac30 0x7fa60c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.771+0000 7fa61ffff640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa60c06ac30 0x7fa60c06d0f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa610033000 tx=0x7fa610009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.772+0000 7fa628fa6740 1 Processor -- start 2024-03-23T09:44:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.772+0000 7fa628fa6740 1 -- start start 2024-03-23T09:44:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.772+0000 7fa628fa6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4ccb8830 0x55de4ccb8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.772+0000 7fa628fa6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4cc88850 0x55de4ccb5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.772+0000 7fa628fa6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cc8c3b0 0x55de4ccb5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.772+0000 7fa628fa6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4cbf8dd0 con 0x55de4ccb8830 2024-03-23T09:44:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.772+0000 7fa628fa6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4cbf6560 con 0x55de4cc8c3b0 2024-03-23T09:44:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.772+0000 7fa628fa6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de4cbf1e50 con 0x55de4cc88850 2024-03-23T09:44:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.772+0000 7fa61ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4ccb8830 0x55de4ccb8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.772+0000 7fa61ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4ccb8830 0x55de4ccb8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60464/0 (socket says 172.21.15.47:60464) 2024-03-23T09:44:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.772+0000 7fa61ffff640 1 -- 172.21.15.47:0/1233244699 learned_addr learned my addr 172.21.15.47:0/1233244699 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.773+0000 7fa6248f2640 1 --2- 172.21.15.47:0/1233244699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cc8c3b0 0x55de4ccb5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.773+0000 7fa61f7fe640 1 --2- 172.21.15.47:0/1233244699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4cc88850 0x55de4ccb5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.773+0000 7fa61ffff640 1 -- 172.21.15.47:0/1233244699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4cc88850 msgr2=0x55de4ccb5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.773+0000 7fa61ffff640 1 --2- 172.21.15.47:0/1233244699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4cc88850 0x55de4ccb5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.773+0000 7fa61ffff640 1 -- 172.21.15.47:0/1233244699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cc8c3b0 msgr2=0x55de4ccb5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.773+0000 7fa61ffff640 1 --2- 172.21.15.47:0/1233244699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cc8c3b0 0x55de4ccb5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.773+0000 7fa61ffff640 1 -- 172.21.15.47:0/1233244699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de4cbb9470 con 0x55de4ccb8830 2024-03-23T09:44:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.773+0000 7fa6248f2640 1 --2- 172.21.15.47:0/1233244699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cc8c3b0 0x55de4ccb5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:44:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.773+0000 7fa61ffff640 1 --2- 172.21.15.47:0/1233244699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4ccb8830 0x55de4ccb8c10 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa610001840 tx=0x7fa61002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b94c0925793d8b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.773+0000 7fa605ffb640 1 -- 172.21.15.47:0/1233244699 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610007650 con 0x55de4ccb8830 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.773+0000 7fa605ffb640 1 -- 172.21.15.47:0/1233244699 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6100067d0 con 0x55de4ccb8830 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.773+0000 7fa605ffb640 1 -- 172.21.15.47:0/1233244699 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610006b40 con 0x55de4ccb8830 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.774+0000 7fa628fa6740 1 -- 172.21.15.47:0/1233244699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4ccb8830 msgr2=0x55de4ccb8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.774+0000 7fa628fa6740 1 --2- 172.21.15.47:0/1233244699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4ccb8830 0x55de4ccb8c10 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa610001840 tx=0x7fa61002f000 comp rx=0 tx=0).stop 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.774+0000 7fa628fa6740 1 -- 172.21.15.47:0/1233244699 shutdown_connections 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.774+0000 7fa628fa6740 1 --2- 172.21.15.47:0/1233244699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cc8c3b0 0x55de4ccb5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.774+0000 7fa628fa6740 1 --2- 172.21.15.47:0/1233244699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4cc88850 0x55de4ccb5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.774+0000 7fa628fa6740 1 --2- 172.21.15.47:0/1233244699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4ccb8830 0x55de4ccb8c10 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.774+0000 7fa628fa6740 1 -- 172.21.15.47:0/1233244699 >> 172.21.15.47:0/1233244699 conn(0x55de4cc8aac0 msgr2=0x55de4ccd3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.774+0000 7fa628fa6740 1 -- 172.21.15.47:0/1233244699 shutdown_connections 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.774+0000 7fa628fa6740 1 -- 172.21.15.47:0/1233244699 wait complete. 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.774+0000 7fa628fa6740 1 -- start start 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.775+0000 7fa628fa6740 1 Processor -- start 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.775+0000 7fa628fa6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cc8c3b0 0x55de4ccbb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.775+0000 7fa628fa6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4ccb8830 0x55de4ccb1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.775+0000 7fa628fa6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 0x55de4ccb0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.775+0000 7fa628fa6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4cbb0280 con 0x55de4cce2df0 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.775+0000 7fa628fa6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4cbaff20 con 0x55de4cc8c3b0 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.775+0000 7fa628fa6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de4cb753c0 con 0x55de4ccb8830 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.775+0000 7fa628fa6740 10 client.0 _ll_register_callbacks cb 0x55de4cbc7380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.775+0000 7fa6248f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 0x55de4ccb0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.775+0000 7fa6248f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 0x55de4ccb0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60472/0 (socket says 172.21.15.47:60472) 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.775+0000 7fa6248f2640 1 -- 172.21.15.47:0/306722408 learned_addr learned my addr 172.21.15.47:0/306722408 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.775+0000 7fa61f7fe640 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4ccb8830 0x55de4ccb1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa61ffff640 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cc8c3b0 0x55de4ccbb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa6248f2640 1 -- 172.21.15.47:0/306722408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4ccb8830 msgr2=0x55de4ccb1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa6248f2640 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4ccb8830 0x55de4ccb1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa6248f2640 1 -- 172.21.15.47:0/306722408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cc8c3b0 msgr2=0x55de4ccbb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa6248f2640 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cc8c3b0 0x55de4ccbb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa6248f2640 1 -- 172.21.15.47:0/306722408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa608003520 con 0x55de4cce2df0 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa61ffff640 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cc8c3b0 0x55de4ccbb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa6248f2640 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 0x55de4ccb0b10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa6200d0d90 tx=0x7fa618007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d47440fee783ca server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa5feffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa5feffd640 1 -- 172.21.15.47:0/306722408 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa618010870 con 0x55de4cce2df0 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa5feffd640 1 -- 172.21.15.47:0/306722408 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6180109d0 con 0x55de4cce2df0 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa628fa6740 1 -- 172.21.15.47:0/306722408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de4cc1b690 con 0x55de4cce2df0 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa5feffd640 1 -- 172.21.15.47:0/306722408 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa618010b90 con 0x55de4cce2df0 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa628fa6740 10 client.6450 fetch_fsmap learned FSMap version 25 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa628fa6740 1 -- 172.21.15.47:0/306722408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55de4cbf04d0 con 0x55de4cce2df0 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa5feffd640 1 -- 172.21.15.47:0/306722408 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55de4cc1b690 con 0x55de4cce2df0 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.776+0000 7fa5feffd640 1 -- 172.21.15.47:0/306722408 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa618010cf0 con 0x55de4cce2df0 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.777+0000 7fa628fa6740 10 client.6450 fetch_fsmap finished waiting for FSMap version 25 2024-03-23T09:44:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.777+0000 7fa628fa6740 10 client.6450 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-23T09:44:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.777+0000 7fa628fa6740 20 client.6450 populate_metadata read hostname 'smithi047' 2024-03-23T09:44:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.777+0000 7fa628fa6740 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55de4ccb90e0 0x55de4ccb94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.777+0000 7fa628fa6740 4 client.6450 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:44:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.777+0000 7fa628fa6740 1 -- 172.21.15.47:0/306722408 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de4cbe3eb0 con 0x55de4ccb90e0 2024-03-23T09:44:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.777+0000 7fa61ffff640 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55de4ccb90e0 0x55de4ccb94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.777+0000 7fa5feffd640 10 client.6450 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.777+0000 7fa61ffff640 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55de4ccb90e0 0x55de4ccb94f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d4c68a651a53c66 server_cookie=aa6c6c37beba6a8b in_seq=0 out_seq=0 2024-03-23T09:44:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa5feffd640 1 -- 172.21.15.47:0/306722408 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55de4cbe3eb0 con 0x55de4ccb90e0 2024-03-23T09:44:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa5feffd640 10 client.6450 handle_command_reply: tid=0 2024-03-23T09:44:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa628fa6740 1 client.6450 shutdown 2024-03-23T09:44:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa628fa6740 1 -- 172.21.15.47:0/306722408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 msgr2=0x55de4ccb0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa628fa6740 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 0x55de4ccb0b10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa6200d0d90 tx=0x7fa618007720 comp rx=0 tx=0).stop 2024-03-23T09:44:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa628fa6740 1 -- 172.21.15.47:0/306722408 shutdown_connections 2024-03-23T09:44:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa628fa6740 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55de4ccb90e0 0x55de4ccb94f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa628fa6740 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 0x55de4ccb0b10 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa628fa6740 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4ccb8830 0x55de4ccb1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa5feffd640 0 client.6450 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa628fa6740 1 --2- 172.21.15.47:0/306722408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cc8c3b0 0x55de4ccbb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa628fa6740 1 -- 172.21.15.47:0/306722408 >> 172.21.15.47:0/306722408 conn(0x55de4cc88850 msgr2=0x55de4cc8e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa628fa6740 1 -- 172.21.15.47:0/306722408 shutdown_connections 2024-03-23T09:44:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa628fa6740 1 -- 172.21.15.47:0/306722408 wait complete. 2024-03-23T09:44:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.778+0000 7fa628fa6740 20 client.6450 trim_cache size 0 max 16384 2024-03-23T09:44:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.793+0000 7fa628fa6740 1 Processor -- start 2024-03-23T09:44:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.793+0000 7fa628fa6740 1 -- start start 2024-03-23T09:44:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.793+0000 7fa628fa6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4cc8c3b0 0x55de4cce25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.793+0000 7fa628fa6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4ccb8830 0x55de4ccb5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.793+0000 7fa628fa6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 0x55de4cc84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.793+0000 7fa628fa6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4cb7af10 con 0x55de4cce2df0 2024-03-23T09:44:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.793+0000 7fa628fa6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4ccbd210 con 0x55de4ccb8830 2024-03-23T09:44:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.793+0000 7fa628fa6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de4cc8f590 con 0x55de4cc8c3b0 2024-03-23T09:44:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.793+0000 7fa6248f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 0x55de4cc84090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.793+0000 7fa6248f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 0x55de4cc84090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60474/0 (socket says 172.21.15.47:60474) 2024-03-23T09:44:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.793+0000 7fa6248f2640 1 -- 172.21.15.47:0/2883645188 learned_addr learned my addr 172.21.15.47:0/2883645188 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.794+0000 7fa6248f2640 1 -- 172.21.15.47:0/2883645188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4cc8c3b0 msgr2=0x55de4cce25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.794+0000 7fa61ffff640 1 --2- 172.21.15.47:0/2883645188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4cc8c3b0 0x55de4cce25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.794+0000 7fa61f7fe640 1 --2- 172.21.15.47:0/2883645188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4ccb8830 0x55de4ccb5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.794+0000 7fa6248f2640 1 --2- 172.21.15.47:0/2883645188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4cc8c3b0 0x55de4cce25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.794+0000 7fa6248f2640 1 -- 172.21.15.47:0/2883645188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4ccb8830 msgr2=0x55de4ccb5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.794+0000 7fa6248f2640 1 --2- 172.21.15.47:0/2883645188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4ccb8830 0x55de4ccb5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.794+0000 7fa6248f2640 1 -- 172.21.15.47:0/2883645188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de4cc249e0 con 0x55de4cce2df0 2024-03-23T09:44:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.794+0000 7fa6248f2640 1 --2- 172.21.15.47:0/2883645188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 0x55de4cc84090 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa6180012d0 tx=0x7fa618013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ef50c02582ff54 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.794+0000 7fa6067fc640 1 -- 172.21.15.47:0/2883645188 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61800b070 con 0x55de4cce2df0 2024-03-23T09:44:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.794+0000 7fa6067fc640 1 -- 172.21.15.47:0/2883645188 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6180144a0 con 0x55de4cce2df0 2024-03-23T09:44:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.794+0000 7fa6067fc640 1 -- 172.21.15.47:0/2883645188 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6180148b0 con 0x55de4cce2df0 2024-03-23T09:44:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.795+0000 7fa628fa6740 1 -- 172.21.15.47:0/2883645188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 msgr2=0x55de4cc84090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.795+0000 7fa628fa6740 1 --2- 172.21.15.47:0/2883645188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 0x55de4cc84090 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa6180012d0 tx=0x7fa618013000 comp rx=0 tx=0).stop 2024-03-23T09:44:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.795+0000 7fa628fa6740 1 -- 172.21.15.47:0/2883645188 shutdown_connections 2024-03-23T09:44:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.795+0000 7fa628fa6740 1 --2- 172.21.15.47:0/2883645188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cce2df0 0x55de4cc84090 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.795+0000 7fa628fa6740 1 --2- 172.21.15.47:0/2883645188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4ccb8830 0x55de4ccb5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.795+0000 7fa628fa6740 1 --2- 172.21.15.47:0/2883645188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4cc8c3b0 0x55de4cce25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.795+0000 7fa628fa6740 1 -- 172.21.15.47:0/2883645188 >> 172.21.15.47:0/2883645188 conn(0x55de4cc88850 msgr2=0x55de4cc88ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:29.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.795+0000 7fa628fa6740 1 -- 172.21.15.47:0/2883645188 shutdown_connections 2024-03-23T09:44:29.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.795+0000 7fa628fa6740 1 -- 172.21.15.47:0/2883645188 wait complete. 2024-03-23T09:44:29.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.795+0000 7fa628fa6740 1 -- start start 2024-03-23T09:44:29.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.795+0000 7fa628fa6740 1 Processor -- start 2024-03-23T09:44:29.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.796+0000 7fa628fa6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cc8c3b0 0x55de4cd3a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.796+0000 7fa628fa6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4ccb8830 0x55de4ccd48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.796+0000 7fa628fa6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cce2df0 0x55de4cc84030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.796+0000 7fa628fa6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4ccdd2d0 con 0x55de4cc8c3b0 2024-03-23T09:44:29.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.796+0000 7fa628fa6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4ccd5470 con 0x55de4cce2df0 2024-03-23T09:44:29.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.796+0000 7fa628fa6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de4ccdd440 con 0x55de4ccb8830 2024-03-23T09:44:29.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.796+0000 7fa628fa6740 10 client.0 _ll_register_callbacks cb 0x7fa6080036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:44:29.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.796+0000 7fa61ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cc8c3b0 0x55de4cd3a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.796+0000 7fa61ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cc8c3b0 0x55de4cd3a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60482/0 (socket says 172.21.15.47:60482) 2024-03-23T09:44:29.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.796+0000 7fa61ffff640 1 -- 172.21.15.47:0/343560829 learned_addr learned my addr 172.21.15.47:0/343560829 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:29.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa61f7fe640 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4ccb8830 0x55de4ccd48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa6248f2640 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cce2df0 0x55de4cc84030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa61ffff640 1 -- 172.21.15.47:0/343560829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4ccb8830 msgr2=0x55de4ccd48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa61ffff640 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4ccb8830 0x55de4ccd48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa61ffff640 1 -- 172.21.15.47:0/343560829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cce2df0 msgr2=0x55de4cc84030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa61ffff640 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cce2df0 0x55de4cc84030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa61ffff640 1 -- 172.21.15.47:0/343560829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de4cbf04d0 con 0x55de4cc8c3b0 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa61f7fe640 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4ccb8830 0x55de4ccd48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa61ffff640 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cc8c3b0 0x55de4cd3a640 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa610001720 tx=0x7fa610001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0371aa5c293b9fe server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa5feffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa5feffd640 1 -- 172.21.15.47:0/343560829 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610007590 con 0x55de4cc8c3b0 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa5feffd640 1 -- 172.21.15.47:0/343560829 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa610043050 con 0x55de4cc8c3b0 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa628fa6740 1 -- 172.21.15.47:0/343560829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6100067d0 con 0x55de4cc8c3b0 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa5feffd640 1 -- 172.21.15.47:0/343560829 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa610059250 con 0x55de4cc8c3b0 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa628fa6740 10 client.6456 fetch_fsmap learned FSMap version 25 2024-03-23T09:44:29.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa628fa6740 1 -- 172.21.15.47:0/343560829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55de4cb925c0 con 0x55de4cc8c3b0 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa5feffd640 1 -- 172.21.15.47:0/343560829 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6100067d0 con 0x55de4cc8c3b0 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa5feffd640 1 -- 172.21.15.47:0/343560829 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa610006ac0 con 0x55de4cc8c3b0 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa628fa6740 10 client.6456 fetch_fsmap finished waiting for FSMap version 25 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa628fa6740 10 client.6456 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa628fa6740 20 client.6456 populate_metadata read hostname 'smithi047' 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa628fa6740 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55de4ccde880 0x55de4cc84570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa628fa6740 4 client.6456 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.797+0000 7fa628fa6740 1 -- 172.21.15.47:0/343560829 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55de4c8e54e0 con 0x55de4ccde880 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa61f7fe640 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55de4ccde880 0x55de4cc84570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa61f7fe640 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55de4ccde880 0x55de4cc84570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=749d124d15c6711e server_cookie=3dee16c0bc78199d in_seq=0 out_seq=0 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa5feffd640 10 client.6456 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa5feffd640 1 -- 172.21.15.47:0/343560829 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55de4c8e54e0 con 0x55de4ccde880 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa5feffd640 10 client.6456 handle_command_reply: tid=0 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa628fa6740 1 client.6456 shutdown 2024-03-23T09:44:29.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa628fa6740 1 -- 172.21.15.47:0/343560829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cc8c3b0 msgr2=0x55de4cd3a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa628fa6740 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cc8c3b0 0x55de4cd3a640 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa610001720 tx=0x7fa610001190 comp rx=0 tx=0).stop 2024-03-23T09:44:29.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa628fa6740 1 -- 172.21.15.47:0/343560829 shutdown_connections 2024-03-23T09:44:29.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa628fa6740 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55de4ccde880 0x55de4cc84570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa628fa6740 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de4cce2df0 0x55de4cc84030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa5feffd640 0 client.6456 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:29.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa628fa6740 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de4ccb8830 0x55de4ccd48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa628fa6740 1 --2- 172.21.15.47:0/343560829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de4cc8c3b0 0x55de4cd3a640 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa628fa6740 1 -- 172.21.15.47:0/343560829 >> 172.21.15.47:0/343560829 conn(0x55de4cc88850 msgr2=0x55de4ccba330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:29.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.798+0000 7fa628fa6740 1 -- 172.21.15.47:0/343560829 shutdown_connections 2024-03-23T09:44:29.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa628fa6740 1 -- 172.21.15.47:0/343560829 wait complete. 2024-03-23T09:44:29.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa628fa6740 20 client.6456 trim_cache size 0 max 16384 2024-03-23T09:44:29.805 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:44:29.806 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-23T09:44:29.806 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-23T09:44:29.806 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:44:29.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa626355640 1 -- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa60c06ac30 msgr2=0x7fa60c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa626355640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa60c06ac30 0x7fa60c06d0f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa610033000 tx=0x7fa610009030 comp rx=0 tx=0).stop 2024-03-23T09:44:29.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa626355640 1 -- 172.21.15.47:0/3004630074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62015ea80 msgr2=0x7fa6201596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa626355640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62015ea80 0x7fa6201596c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa6140114e0 tx=0x7fa6140119a0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa626355640 1 -- 172.21.15.47:0/3004630074 shutdown_connections 2024-03-23T09:44:29.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa626355640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa60c06ac30 0x7fa60c06d0f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa626355640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa620150310 0x7fa62015a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa626355640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62015ea80 0x7fa6201596c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa626355640 1 --2- 172.21.15.47:0/3004630074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa608003970 0x7fa62015f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa626355640 1 -- 172.21.15.47:0/3004630074 >> 172.21.15.47:0/3004630074 conn(0x7fa6200659d0 msgr2=0x7fa6200b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:29.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa626355640 1 -- 172.21.15.47:0/3004630074 shutdown_connections 2024-03-23T09:44:29.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.799+0000 7fa626355640 1 -- 172.21.15.47:0/3004630074 wait complete. 2024-03-23T09:44:29.811 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T09:44:29.811 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:44:29.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.910+0000 7f7f8192e640 1 Processor -- start 2024-03-23T09:44:29.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.911+0000 7f7f8192e640 1 -- start start 2024-03-23T09:44:29.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.911+0000 7f7f8192e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f7c06c350 0x7f7f7c06d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.911+0000 7f7f8192e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7f7c142aa0 0x7f7f7c0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.911+0000 7f7f8192e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7c06c8c0 con 0x7f7f7c0b3c30 2024-03-23T09:44:29.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.911+0000 7f7f8192e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7c06cba0 con 0x7f7f7c142aa0 2024-03-23T09:44:29.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.911+0000 7f7f8192e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c06ce80 con 0x7f7f7c06c350 2024-03-23T09:44:29.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.912+0000 7f7f7a7fc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f7c06c350 0x7f7f7c06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.912+0000 7f7f7b7fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7f7c142aa0 0x7f7f7c0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.912+0000 7f7f7a7fc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f7c06c350 0x7f7f7c06d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:52272/0 (socket says 172.21.15.47:52272) 2024-03-23T09:44:29.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.912+0000 7f7f7a7fc640 1 -- 172.21.15.47:0/250612639 learned_addr learned my addr 172.21.15.47:0/250612639 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:29.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.912+0000 7f7f7b7fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7f7c142aa0 0x7f7f7c0cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:57642/0 (socket says 172.21.15.47:57642) 2024-03-23T09:44:29.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.912+0000 7f7f7affd640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f7f7c0b3c30 0x7f7f7c0b4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47766/0 (socket says 172.21.15.47:47766) 2024-03-23T09:44:29.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.912+0000 7f7f7a7fc640 1 -- 172.21.15.47:0/250612639 >> v2:172.21.15.175:3300/0 conn(0x7f7f7c142aa0 msgr2=0x7f7f7c0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:29.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.912+0000 7f7f7a7fc640 1 --2- 172.21.15.47:0/250612639 >> v2:172.21.15.175:3300/0 conn(0x7f7f7c142aa0 0x7f7f7c0cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.912+0000 7f7f7a7fc640 1 -- 172.21.15.47:0/250612639 >> v1:172.21.15.175:6789/0 conn(0x7f7f7c0b3c30 legacy=0x7f7f7c0b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.912+0000 7f7f7a7fc640 1 -- 172.21.15.47:0/250612639 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c06d160 con 0x7f7f7c06c350 2024-03-23T09:44:29.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.912+0000 7f7f7b7fe640 1 --2- 172.21.15.47:0/250612639 >> v2:172.21.15.175:3300/0 conn(0x7f7f7c142aa0 0x7f7f7c0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:44:29.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.913+0000 7f7f7a7fc640 1 --2- 172.21.15.47:0/250612639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f7c06c350 0x7f7f7c06d960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009870 tx=0x7f7f7002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a36e32b121cdeefe server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.913+0000 7f7f79ffb640 1 -- 172.21.15.47:0/250612639 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f70030840 con 0x7f7f7c06c350 2024-03-23T09:44:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.913+0000 7f7f79ffb640 1 -- 172.21.15.47:0/250612639 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f700309a0 con 0x7f7f7c06c350 2024-03-23T09:44:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.913+0000 7f7f79ffb640 1 -- 172.21.15.47:0/250612639 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f70030d30 con 0x7f7f7c06c350 2024-03-23T09:44:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.914+0000 7f7f8192e640 1 -- 172.21.15.47:0/250612639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f7c06c350 msgr2=0x7f7f7c06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.914+0000 7f7f8192e640 1 --2- 172.21.15.47:0/250612639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f7c06c350 0x7f7f7c06d960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009870 tx=0x7f7f7002b980 comp rx=0 tx=0).stop 2024-03-23T09:44:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.914+0000 7f7f8192e640 1 -- 172.21.15.47:0/250612639 shutdown_connections 2024-03-23T09:44:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.914+0000 7f7f8192e640 1 --2- 172.21.15.47:0/250612639 >> v2:172.21.15.175:3300/0 conn(0x7f7f7c142aa0 0x7f7f7c0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.914+0000 7f7f8192e640 1 --2- 172.21.15.47:0/250612639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f7c06c350 0x7f7f7c06d960 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.914+0000 7f7f8192e640 1 -- 172.21.15.47:0/250612639 >> 172.21.15.47:0/250612639 conn(0x7f7f7c06a310 msgr2=0x7f7f7c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.914+0000 7f7f8192e640 1 -- 172.21.15.47:0/250612639 shutdown_connections 2024-03-23T09:44:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.914+0000 7f7f8192e640 1 -- 172.21.15.47:0/250612639 wait complete. 2024-03-23T09:44:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.915+0000 7f7f8192e640 1 Processor -- start 2024-03-23T09:44:29.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.915+0000 7f7f8192e640 1 -- start start 2024-03-23T09:44:29.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f8192e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f7c06c350 0x7f7f7c15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f8192e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f7c0b3c30 0x7f7f7c131fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f8192e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f7c142aa0 0x7f7f7c1324f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f8192e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0c3d20 con 0x7f7f7c142aa0 2024-03-23T09:44:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f8192e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0c3a40 con 0x7f7f7c0b3c30 2024-03-23T09:44:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f8192e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c15b970 con 0x7f7f7c06c350 2024-03-23T09:44:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f7b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f7c142aa0 0x7f7f7c1324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f7b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f7c142aa0 0x7f7f7c1324f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60496/0 (socket says 172.21.15.47:60496) 2024-03-23T09:44:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f7b7fe640 1 -- 172.21.15.47:0/3277240457 learned_addr learned my addr 172.21.15.47:0/3277240457 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f7a7fc640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f7c0b3c30 0x7f7f7c131fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f7affd640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f7c06c350 0x7f7f7c15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f7b7fe640 1 -- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f7c06c350 msgr2=0x7f7f7c15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f7b7fe640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f7c06c350 0x7f7f7c15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f7b7fe640 1 -- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f7c0b3c30 msgr2=0x7f7f7c131fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:29.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f7b7fe640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f7c0b3c30 0x7f7f7c131fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:29.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f7b7fe640 1 -- 172.21.15.47:0/3277240457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f74009a70 con 0x7f7f7c142aa0 2024-03-23T09:44:29.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f7b7fe640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f7c142aa0 0x7f7f7c1324f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400efc0 tx=0x7f7f74009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:29.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f63fff640 1 -- 172.21.15.47:0/3277240457 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74009d70 con 0x7f7f7c142aa0 2024-03-23T09:44:29.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.916+0000 7f7f63fff640 1 -- 172.21.15.47:0/3277240457 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7400f040 con 0x7f7f7c142aa0 2024-03-23T09:44:29.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.917+0000 7f7f8192e640 1 -- 172.21.15.47:0/3277240457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f700094d0 con 0x7f7f7c142aa0 2024-03-23T09:44:29.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.917+0000 7f7f63fff640 1 -- 172.21.15.47:0/3277240457 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74003990 con 0x7f7f7c142aa0 2024-03-23T09:44:29.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.917+0000 7f7f8192e640 1 -- 172.21.15.47:0/3277240457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c15bf30 con 0x7f7f7c142aa0 2024-03-23T09:44:29.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.918+0000 7f7f8192e640 1 -- 172.21.15.47:0/3277240457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7c073480 con 0x7f7f7c142aa0 2024-03-23T09:44:29.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.918+0000 7f7f63fff640 1 -- 172.21.15.47:0/3277240457 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7f740144a0 con 0x7f7f7c142aa0 2024-03-23T09:44:29.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.918+0000 7f7f63fff640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7f6806ac00 0x7f7f6806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:29.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.919+0000 7f7f63fff640 1 -- 172.21.15.47:0/3277240457 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f74089e90 con 0x7f7f7c142aa0 2024-03-23T09:44:29.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.919+0000 7f7f7affd640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7f6806ac00 0x7f7f6806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:29.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.919+0000 7f7f7affd640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7f6806ac00 0x7f7f6806d0c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c000f60 tx=0x7f7f6c000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:29.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:29.923+0000 7f7f63fff640 1 -- 172.21.15.47:0/3277240457 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7f740551b0 con 0x7f7f7c142aa0 2024-03-23T09:44:30.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.139+0000 7f7f8192e640 1 -- 172.21.15.47:0/3277240457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f7c066be0 con 0x7f7f7c142aa0 2024-03-23T09:44:30.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.140+0000 7f7f63fff640 1 -- 172.21.15.47:0/3277240457 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f7f740587d0 con 0x7f7f7c142aa0 2024-03-23T09:44:30.141 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:44:30.142 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/375513546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":375513546},{"type":"v1","addr":"172.21.15.175:6837","nonce":375513546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/3650756747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3650756747},{"type":"v1","addr":"172.21.15.47:6833","nonce":3650756747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2893805543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2893805543},{"type":"v1","addr":"172.21.15.47:6835","nonce":2893805543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:43:47.389279+0000","modified":"2024-03-23T09:44:04.065813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6836/2498456351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2498456351},{"type":"v1","addr":"172.21.15.175:6836","nonce":2498456351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-23T09:44:30.142 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 25 2024-03-23T09:44:30.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.142+0000 7f7f8192e640 1 -- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7f6806ac00 msgr2=0x7f7f6806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.142+0000 7f7f8192e640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7f6806ac00 0x7f7f6806d0c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c000f60 tx=0x7f7f6c000ed0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.142+0000 7f7f8192e640 1 -- 172.21.15.47:0/3277240457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f7c142aa0 msgr2=0x7f7f7c1324f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.142+0000 7f7f8192e640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f7c142aa0 0x7f7f7c1324f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400efc0 tx=0x7f7f74009f90 comp rx=0 tx=0).stop 2024-03-23T09:44:30.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.142+0000 7f7f8192e640 1 -- 172.21.15.47:0/3277240457 shutdown_connections 2024-03-23T09:44:30.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.142+0000 7f7f8192e640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7f6806ac00 0x7f7f6806d0c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.142+0000 7f7f8192e640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f7c142aa0 0x7f7f7c1324f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.142+0000 7f7f8192e640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f7c0b3c30 0x7f7f7c131fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.142+0000 7f7f8192e640 1 --2- 172.21.15.47:0/3277240457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f7c06c350 0x7f7f7c15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.142+0000 7f7f8192e640 1 -- 172.21.15.47:0/3277240457 >> 172.21.15.47:0/3277240457 conn(0x7f7f7c06a310 msgr2=0x7f7f7c0b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.142+0000 7f7f8192e640 1 -- 172.21.15.47:0/3277240457 shutdown_connections 2024-03-23T09:44:30.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.142+0000 7f7f8192e640 1 -- 172.21.15.47:0/3277240457 wait complete. 2024-03-23T09:44:30.153 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-03-23T09:44:30.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.255+0000 7f4415028640 1 Processor -- start 2024-03-23T09:44:30.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.255+0000 7f4415028640 1 -- start start 2024-03-23T09:44:30.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.256+0000 7f4415028640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44100cde10 0x7f4410160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.256+0000 7f4415028640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f44100b8350 0x7f44100d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.256+0000 7f4415028640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4410161630 con 0x7f441015c990 2024-03-23T09:44:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.256+0000 7f4415028640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44100d1030 con 0x7f44100cde10 2024-03-23T09:44:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.256+0000 7f4415028640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44100cfe10 con 0x7f44100b8350 2024-03-23T09:44:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.256+0000 7f440e59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44100cde10 0x7f4410160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.256+0000 7f440e59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44100cde10 0x7f4410160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60504/0 (socket says 172.21.15.47:60504) 2024-03-23T09:44:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.256+0000 7f440e59c640 1 -- 172.21.15.47:0/1850201083 learned_addr learned my addr 172.21.15.47:0/1850201083 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.256+0000 7f440f59e640 1 --2- 172.21.15.47:0/1850201083 >> v2:172.21.15.175:3300/0 conn(0x7f44100b8350 0x7f44100d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.256+0000 7f440e59c640 1 -- 172.21.15.47:0/1850201083 >> v2:172.21.15.175:3300/0 conn(0x7f44100b8350 msgr2=0x7f44100d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.256+0000 7f440e59c640 1 --2- 172.21.15.47:0/1850201083 >> v2:172.21.15.175:3300/0 conn(0x7f44100b8350 0x7f44100d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.256+0000 7f440e59c640 1 -- 172.21.15.47:0/1850201083 >> v1:172.21.15.175:6789/0 conn(0x7f441015c990 legacy=0x7f44100cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.256+0000 7f440e59c640 1 -- 172.21.15.47:0/1850201083 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44100b5b60 con 0x7f44100cde10 2024-03-23T09:44:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.257+0000 7f440e59c640 1 --2- 172.21.15.47:0/1850201083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44100cde10 0x7f4410160690 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f44100d0fa0 tx=0x7f43f802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b992e07dce10cd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.257+0000 7f440dd9b640 1 -- 172.21.15.47:0/1850201083 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f80317e0 con 0x7f44100cde10 2024-03-23T09:44:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.257+0000 7f440dd9b640 1 -- 172.21.15.47:0/1850201083 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f8031940 con 0x7f44100cde10 2024-03-23T09:44:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.257+0000 7f440dd9b640 1 -- 172.21.15.47:0/1850201083 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f8031cd0 con 0x7f44100cde10 2024-03-23T09:44:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.258+0000 7f4415028640 1 -- 172.21.15.47:0/1850201083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44100cde10 msgr2=0x7f4410160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.258+0000 7f4415028640 1 --2- 172.21.15.47:0/1850201083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44100cde10 0x7f4410160690 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f44100d0fa0 tx=0x7f43f802b880 comp rx=0 tx=0).stop 2024-03-23T09:44:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.258+0000 7f4415028640 1 -- 172.21.15.47:0/1850201083 shutdown_connections 2024-03-23T09:44:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.258+0000 7f4415028640 1 --2- 172.21.15.47:0/1850201083 >> v2:172.21.15.175:3300/0 conn(0x7f44100b8350 0x7f44100d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.258+0000 7f4415028640 1 --2- 172.21.15.47:0/1850201083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44100cde10 0x7f4410160690 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.258+0000 7f4415028640 1 -- 172.21.15.47:0/1850201083 >> 172.21.15.47:0/1850201083 conn(0x7f44100659d0 msgr2=0x7f4410065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.258+0000 7f4415028640 1 -- 172.21.15.47:0/1850201083 shutdown_connections 2024-03-23T09:44:30.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.258+0000 7f4415028640 1 -- 172.21.15.47:0/1850201083 wait complete. 2024-03-23T09:44:30.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.259+0000 7f4415028640 1 Processor -- start 2024-03-23T09:44:30.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.259+0000 7f4415028640 1 -- start start 2024-03-23T09:44:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f4415028640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44100b8350 0x7f441015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f4415028640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44100cde10 0x7f4410150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f4415028640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015c990 0x7f44100b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f4415028640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441015a170 con 0x7f44100b8350 2024-03-23T09:44:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f4415028640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441015a450 con 0x7f44100cde10 2024-03-23T09:44:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f4415028640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441015a730 con 0x7f441015c990 2024-03-23T09:44:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f440ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44100b8350 0x7f441015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f440ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44100b8350 0x7f441015eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60506/0 (socket says 172.21.15.47:60506) 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f440ed9d640 1 -- 172.21.15.47:0/4058231259 learned_addr learned my addr 172.21.15.47:0/4058231259 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f440e59c640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44100cde10 0x7f4410150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f440f59e640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015c990 0x7f44100b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f440ed9d640 1 -- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015c990 msgr2=0x7f44100b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f440ed9d640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015c990 0x7f44100b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f440ed9d640 1 -- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44100cde10 msgr2=0x7f4410150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f440ed9d640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44100cde10 0x7f4410150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.260+0000 7f440ed9d640 1 -- 172.21.15.47:0/4058231259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f80094d0 con 0x7f44100b8350 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.261+0000 7f440ed9d640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44100b8350 0x7f441015eb70 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f440800c430 tx=0x7f440800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.261+0000 7f44077fe640 1 -- 172.21.15.47:0/4058231259 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408011790 con 0x7f44100b8350 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.261+0000 7f44077fe640 1 -- 172.21.15.47:0/4058231259 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44080118f0 con 0x7f44100b8350 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.261+0000 7f44077fe640 1 -- 172.21.15.47:0/4058231259 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408011ce0 con 0x7f44100b8350 2024-03-23T09:44:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.261+0000 7f4415028640 1 -- 172.21.15.47:0/4058231259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441015aa10 con 0x7f44100b8350 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.261+0000 7f4415028640 1 -- 172.21.15.47:0/4058231259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4410151380 con 0x7f44100b8350 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.263+0000 7f44077fe640 1 -- 172.21.15.47:0/4058231259 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f440801c460 con 0x7f44100b8350 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.263+0000 7f44077fe640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f43f006ac00 0x7f43f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.263+0000 7f44077fe640 1 -- 172.21.15.47:0/4058231259 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4408089380 con 0x7f44100b8350 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.263+0000 7f440e59c640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f43f006ac00 0x7f43f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.264+0000 7f4417b95740 1 Processor -- start 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.264+0000 7f440e59c640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f43f006ac00 0x7f43f006d0c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f43f8036000 tx=0x7f43f803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.264+0000 7f4417b95740 1 -- start start 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f4417b95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 0x55f6febb6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f4417b95740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb866d0 0x55f6febb3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f4417b95740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6feb8a230 0x55f6febb3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f4417b95740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6feaf76a0 con 0x55f6feb8a230 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f4417b95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6feaf4c70 con 0x55f6febb66b0 2024-03-23T09:44:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f4417b95740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6feaf0560 con 0x55f6feb866d0 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f440f59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6feb8a230 0x55f6febb3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f440f59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6feb8a230 0x55f6febb3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60512/0 (socket says 172.21.15.47:60512) 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f440f59e640 1 -- 172.21.15.47:0/2591156617 learned_addr learned my addr 172.21.15.47:0/2591156617 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f440ed9d640 1 --2- 172.21.15.47:0/2591156617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 0x55f6febb6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f440e59c640 1 --2- 172.21.15.47:0/2591156617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb866d0 0x55f6febb3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f440f59e640 1 -- 172.21.15.47:0/2591156617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb866d0 msgr2=0x55f6febb3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f440f59e640 1 --2- 172.21.15.47:0/2591156617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb866d0 0x55f6febb3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f440f59e640 1 -- 172.21.15.47:0/2591156617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 msgr2=0x55f6febb6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f440f59e640 1 --2- 172.21.15.47:0/2591156617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 0x55f6febb6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f440f59e640 1 -- 172.21.15.47:0/2591156617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6feab7d50 con 0x55f6feb8a230 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.265+0000 7f440f59e640 1 --2- 172.21.15.47:0/2591156617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6feb8a230 0x55f6febb3e60 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4400007e00 tx=0x7f440000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf28281e2e5b0a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.266+0000 7f4404ff9640 1 -- 172.21.15.47:0/2591156617 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440000ee90 con 0x55f6feb8a230 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.266+0000 7f4404ff9640 1 -- 172.21.15.47:0/2591156617 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4400011680 con 0x55f6feb8a230 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.266+0000 7f4404ff9640 1 -- 172.21.15.47:0/2591156617 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44000117f0 con 0x55f6feb8a230 2024-03-23T09:44:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.266+0000 7f4417b95740 1 -- 172.21.15.47:0/2591156617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6feb8a230 msgr2=0x55f6febb3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.266+0000 7f4417b95740 1 --2- 172.21.15.47:0/2591156617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6feb8a230 0x55f6febb3e60 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4400007e00 tx=0x7f440000e6c0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.266+0000 7f4417b95740 1 -- 172.21.15.47:0/2591156617 shutdown_connections 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.266+0000 7f4417b95740 1 --2- 172.21.15.47:0/2591156617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6feb8a230 0x55f6febb3e60 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.266+0000 7f4417b95740 1 --2- 172.21.15.47:0/2591156617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb866d0 0x55f6febb3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.266+0000 7f4417b95740 1 --2- 172.21.15.47:0/2591156617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 0x55f6febb6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.266+0000 7f4417b95740 1 -- 172.21.15.47:0/2591156617 >> 172.21.15.47:0/2591156617 conn(0x55f6feb88940 msgr2=0x55f6febd14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.266+0000 7f4417b95740 1 -- 172.21.15.47:0/2591156617 shutdown_connections 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.266+0000 7f4417b95740 1 -- 172.21.15.47:0/2591156617 wait complete. 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.267+0000 7f4417b95740 1 -- start start 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.267+0000 7f4417b95740 1 Processor -- start 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f4417b95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6feb8a230 0x55f6febac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f4417b95740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6febb66b0 0x55f6febaf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f4417b95740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6febb94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f4417b95740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6fe7c93f0 con 0x55f6febb6a90 2024-03-23T09:44:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f4417b95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6fe8793d0 con 0x55f6feb8a230 2024-03-23T09:44:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f4417b95740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6fea73a10 con 0x55f6febb66b0 2024-03-23T09:44:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f4417b95740 10 client.0 _ll_register_callbacks cb 0x55f6feae9d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:44:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f440f59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6febb94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f440f59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6febb94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60524/0 (socket says 172.21.15.47:60524) 2024-03-23T09:44:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f440f59e640 1 -- 172.21.15.47:0/3169294510 learned_addr learned my addr 172.21.15.47:0/3169294510 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f440ed9d640 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6feb8a230 0x55f6febac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f440e59c640 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6febb66b0 0x55f6febaf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f440f59e640 1 -- 172.21.15.47:0/3169294510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6febb66b0 msgr2=0x55f6febaf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f440f59e640 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6febb66b0 0x55f6febaf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f440f59e640 1 -- 172.21.15.47:0/3169294510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6feb8a230 msgr2=0x55f6febac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f440f59e640 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6feb8a230 0x55f6febac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f440f59e640 1 -- 172.21.15.47:0/3169294510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4400008040 con 0x55f6febb6a90 2024-03-23T09:44:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f440e59c640 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6febb66b0 0x55f6febaf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f440f59e640 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6febb94c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4400015000 tx=0x7f4400004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3499f7997e35d820 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f43edffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f43edffb640 1 -- 172.21.15.47:0/3169294510 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440001f070 con 0x55f6febb6a90 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f43edffb640 1 -- 172.21.15.47:0/3169294510 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4400011a00 con 0x55f6febb6a90 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.268+0000 7f4417b95740 1 -- 172.21.15.47:0/3169294510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6feb19150 con 0x55f6febb6a90 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f43edffb640 1 -- 172.21.15.47:0/3169294510 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4400007a40 con 0x55f6febb6a90 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f4417b95740 10 client.6471 fetch_fsmap learned FSMap version 25 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f4417b95740 1 -- 172.21.15.47:0/3169294510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55f6feab2760 con 0x55f6febb6a90 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f43edffb640 1 -- 172.21.15.47:0/3169294510 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f6feb19150 con 0x55f6febb6a90 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f43edffb640 1 -- 172.21.15.47:0/3169294510 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4400011bc0 con 0x55f6febb6a90 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f4417b95740 10 client.6471 fetch_fsmap finished waiting for FSMap version 25 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f4417b95740 10 client.6471 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f4417b95740 20 client.6471 populate_metadata read hostname 'smithi047' 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f4417b95740 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55f6febaf890 0x55f6febee870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f4417b95740 4 client.6471 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:44:30.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f4417b95740 1 -- 172.21.15.47:0/3169294510 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6feafb650 con 0x55f6febaf890 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f440ed9d640 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55f6febaf890 0x55f6febee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f440ed9d640 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55f6febaf890 0x55f6febee870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b09a7241151f0c21 server_cookie=e99511824d10574f in_seq=0 out_seq=0 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.269+0000 7f43edffb640 10 client.6471 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f43edffb640 1 -- 172.21.15.47:0/3169294510 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f6feafb650 con 0x55f6febaf890 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f43edffb640 10 client.6471 handle_command_reply: tid=0 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f4417b95740 1 client.6471 shutdown 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f4417b95740 1 -- 172.21.15.47:0/3169294510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 msgr2=0x55f6febb94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f4417b95740 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6febb94c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4400015000 tx=0x7f4400004ab0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f4417b95740 1 -- 172.21.15.47:0/3169294510 shutdown_connections 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f4417b95740 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55f6febaf890 0x55f6febee870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f4417b95740 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6febb94c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f4417b95740 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6febb66b0 0x55f6febaf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f43edffb640 0 client.6471 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f4417b95740 1 --2- 172.21.15.47:0/3169294510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6feb8a230 0x55f6febac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f4417b95740 1 -- 172.21.15.47:0/3169294510 >> 172.21.15.47:0/3169294510 conn(0x55f6feb866d0 msgr2=0x55f6feb8aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f4417b95740 1 -- 172.21.15.47:0/3169294510 shutdown_connections 2024-03-23T09:44:30.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f4417b95740 1 -- 172.21.15.47:0/3169294510 wait complete. 2024-03-23T09:44:30.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.270+0000 7f4417b95740 20 client.6471 trim_cache size 0 max 16384 2024-03-23T09:44:30.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.282+0000 7f4417b95740 1 Processor -- start 2024-03-23T09:44:30.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.282+0000 7f4417b95740 1 -- start start 2024-03-23T09:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.282+0000 7f4417b95740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb8a230 0x55f6febb77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.282+0000 7f4417b95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 0x55f6fec31200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.282+0000 7f4417b95740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6fec31740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.282+0000 7f4417b95740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6febd2540 con 0x55f6febb6a90 2024-03-23T09:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.282+0000 7f4417b95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6febba410 con 0x55f6febb66b0 2024-03-23T09:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.282+0000 7f4417b95740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6febb2620 con 0x55f6feb8a230 2024-03-23T09:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.282+0000 7f440f59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6fec31740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.282+0000 7f440f59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6fec31740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60534/0 (socket says 172.21.15.47:60534) 2024-03-23T09:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.282+0000 7f440f59e640 1 -- 172.21.15.47:0/3872872381 learned_addr learned my addr 172.21.15.47:0/3872872381 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f440f59e640 1 -- 172.21.15.47:0/3872872381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb8a230 msgr2=0x55f6febb77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f440f59e640 1 --2- 172.21.15.47:0/3872872381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb8a230 0x55f6febb77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f440f59e640 1 -- 172.21.15.47:0/3872872381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 msgr2=0x55f6fec31200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f440f59e640 1 --2- 172.21.15.47:0/3872872381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 0x55f6fec31200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f440f59e640 1 -- 172.21.15.47:0/3872872381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6fea90ad0 con 0x55f6febb6a90 2024-03-23T09:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f440f59e640 1 --2- 172.21.15.47:0/3872872381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6fec31740 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4400000f80 tx=0x7f440001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744680fa3063376d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f44057fa640 1 -- 172.21.15.47:0/3872872381 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440000ee90 con 0x55f6febb6a90 2024-03-23T09:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f44057fa640 1 -- 172.21.15.47:0/3872872381 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440000ebc0 con 0x55f6febb6a90 2024-03-23T09:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f44057fa640 1 -- 172.21.15.47:0/3872872381 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400003ef0 con 0x55f6febb6a90 2024-03-23T09:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f4417b95740 1 -- 172.21.15.47:0/3872872381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 msgr2=0x55f6fec31740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f4417b95740 1 --2- 172.21.15.47:0/3872872381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6fec31740 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4400000f80 tx=0x7f440001f000 comp rx=0 tx=0).stop 2024-03-23T09:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f4417b95740 1 -- 172.21.15.47:0/3872872381 shutdown_connections 2024-03-23T09:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f4417b95740 1 --2- 172.21.15.47:0/3872872381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6fec31740 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f4417b95740 1 --2- 172.21.15.47:0/3872872381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 0x55f6fec31200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f4417b95740 1 --2- 172.21.15.47:0/3872872381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb8a230 0x55f6febb77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f4417b95740 1 -- 172.21.15.47:0/3872872381 >> 172.21.15.47:0/3872872381 conn(0x55f6feb866d0 msgr2=0x55f6febf2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f4417b95740 1 -- 172.21.15.47:0/3872872381 shutdown_connections 2024-03-23T09:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.283+0000 7f4417b95740 1 -- 172.21.15.47:0/3872872381 wait complete. 2024-03-23T09:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.284+0000 7f4417b95740 1 -- start start 2024-03-23T09:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.284+0000 7f4417b95740 1 Processor -- start 2024-03-23T09:44:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f4417b95740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb8a230 0x55f6febaf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f4417b95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 0x55f6fec30300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f4417b95740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6fec30840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f4417b95740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6febe6dc0 con 0x55f6febb6a90 2024-03-23T09:44:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f4417b95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6febd3ae0 con 0x55f6febb66b0 2024-03-23T09:44:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f4417b95740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6febaf550 con 0x55f6feb8a230 2024-03-23T09:44:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f4417b95740 10 client.0 _ll_register_callbacks cb 0x7f4408057f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:44:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f440f59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6fec30840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f440f59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6fec30840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60550/0 (socket says 172.21.15.47:60550) 2024-03-23T09:44:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f440f59e640 1 -- 172.21.15.47:0/2048896838 learned_addr learned my addr 172.21.15.47:0/2048896838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f440e59c640 1 --2- 172.21.15.47:0/2048896838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 0x55f6fec30300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f440ed9d640 1 --2- 172.21.15.47:0/2048896838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb8a230 0x55f6febaf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f440f59e640 1 -- 172.21.15.47:0/2048896838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb8a230 msgr2=0x55f6febaf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f440f59e640 1 --2- 172.21.15.47:0/2048896838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb8a230 0x55f6febaf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f440f59e640 1 -- 172.21.15.47:0/2048896838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 msgr2=0x55f6fec30300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f440f59e640 1 --2- 172.21.15.47:0/2048896838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 0x55f6fec30300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f440f59e640 1 -- 172.21.15.47:0/2048896838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6feab2760 con 0x55f6febb6a90 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f440f59e640 1 --2- 172.21.15.47:0/2048896838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6fec30840 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4400034040 tx=0x7f4400007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4fdde82da59468 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f43edffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f43edffb640 1 -- 172.21.15.47:0/2048896838 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440001f070 con 0x55f6febb6a90 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f43edffb640 1 -- 172.21.15.47:0/2048896838 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4400004710 con 0x55f6febb6a90 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.285+0000 7f4417b95740 1 -- 172.21.15.47:0/2048896838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4400011680 con 0x55f6febb6a90 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.286+0000 7f43edffb640 1 -- 172.21.15.47:0/2048896838 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f440000ee90 con 0x55f6febb6a90 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.286+0000 7f4417b95740 10 client.6477 fetch_fsmap learned FSMap version 25 2024-03-23T09:44:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.286+0000 7f4417b95740 1 -- 172.21.15.47:0/2048896838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55f6feab65b0 con 0x55f6febb6a90 2024-03-23T09:44:30.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.286+0000 7f43edffb640 1 -- 172.21.15.47:0/2048896838 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4400011680 con 0x55f6febb6a90 2024-03-23T09:44:30.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.286+0000 7f43edffb640 1 -- 172.21.15.47:0/2048896838 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4400041020 con 0x55f6febb6a90 2024-03-23T09:44:30.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.286+0000 7f4417b95740 10 client.6477 fetch_fsmap finished waiting for FSMap version 25 2024-03-23T09:44:30.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.286+0000 7f4417b95740 10 client.6477 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T09:44:30.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.286+0000 7f4417b95740 20 client.6477 populate_metadata read hostname 'smithi047' 2024-03-23T09:44:30.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.286+0000 7f4417b95740 1 --2- 172.21.15.47:0/2048896838 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55f6febd0ea0 0x55f6febd1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.286+0000 7f4417b95740 4 client.6477 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-23T09:44:30.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.286+0000 7f4417b95740 1 -- 172.21.15.47:0/2048896838 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55f6fe7a18d0 con 0x55f6febd0ea0 2024-03-23T09:44:30.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.286+0000 7f440ed9d640 1 --2- 172.21.15.47:0/2048896838 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55f6febd0ea0 0x55f6febd1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f43edffb640 10 client.6477 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:30.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f440ed9d640 1 --2- 172.21.15.47:0/2048896838 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55f6febd0ea0 0x55f6febd1280 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc757975b71f9ead server_cookie=ef7e61866673cc2d in_seq=0 out_seq=0 2024-03-23T09:44:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f43edffb640 1 -- 172.21.15.47:0/2048896838 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x55f6fe7a18d0 con 0x55f6febd0ea0 2024-03-23T09:44:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f43edffb640 10 client.6477 handle_command_reply: tid=0 2024-03-23T09:44:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4417b95740 1 client.6477 shutdown 2024-03-23T09:44:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4417b95740 1 -- 172.21.15.47:0/2048896838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 msgr2=0x55f6fec30840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4417b95740 1 --2- 172.21.15.47:0/2048896838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6fec30840 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4400034040 tx=0x7f4400007860 comp rx=0 tx=0).stop 2024-03-23T09:44:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4417b95740 1 -- 172.21.15.47:0/2048896838 shutdown_connections 2024-03-23T09:44:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4417b95740 1 --2- 172.21.15.47:0/2048896838 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55f6febd0ea0 0x55f6febd1280 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4417b95740 1 --2- 172.21.15.47:0/2048896838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6febb6a90 0x55f6fec30840 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4417b95740 1 --2- 172.21.15.47:0/2048896838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6febb66b0 0x55f6fec30300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f43edffb640 0 client.6477 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4417b95740 1 --2- 172.21.15.47:0/2048896838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f6feb8a230 0x55f6febaf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4417b95740 1 -- 172.21.15.47:0/2048896838 >> 172.21.15.47:0/2048896838 conn(0x55f6feb866d0 msgr2=0x55f6febb37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4417b95740 1 -- 172.21.15.47:0/2048896838 shutdown_connections 2024-03-23T09:44:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4417b95740 1 -- 172.21.15.47:0/2048896838 wait complete. 2024-03-23T09:44:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4417b95740 20 client.6477 trim_cache size 0 max 16384 2024-03-23T09:44:30.295 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:44:30.295 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-23T09:44:30.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:44:30.295 INFO:teuthology.orchestra.run.smithi047.stdout: "damage_type": "backtrace", 2024-03-23T09:44:30.295 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 2189748726, 2024-03-23T09:44:30.295 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 1099511627777, 2024-03-23T09:44:30.295 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/dir_test/file" 2024-03-23T09:44:30.295 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:44:30.296 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-23T09:44:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4415028640 1 -- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f43f006ac00 msgr2=0x7f43f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4415028640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f43f006ac00 0x7f43f006d0c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f43f8036000 tx=0x7f43f803d040 comp rx=0 tx=0).stop 2024-03-23T09:44:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4415028640 1 -- 172.21.15.47:0/4058231259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44100b8350 msgr2=0x7f441015eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.287+0000 7f4415028640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44100b8350 0x7f441015eb70 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f440800c430 tx=0x7f440800c8f0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.288+0000 7f4415028640 1 -- 172.21.15.47:0/4058231259 shutdown_connections 2024-03-23T09:44:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.288+0000 7f4415028640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f43f006ac00 0x7f43f006d0c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.288+0000 7f4415028640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015c990 0x7f44100b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.288+0000 7f4415028640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44100cde10 0x7f4410150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.288+0000 7f4415028640 1 --2- 172.21.15.47:0/4058231259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44100b8350 0x7f441015eb70 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.288+0000 7f4415028640 1 -- 172.21.15.47:0/4058231259 >> 172.21.15.47:0/4058231259 conn(0x7f44100659d0 msgr2=0x7f441015bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.288+0000 7f4415028640 1 -- 172.21.15.47:0/4058231259 shutdown_connections 2024-03-23T09:44:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.288+0000 7f4415028640 1 -- 172.21.15.47:0/4058231259 wait complete. 2024-03-23T09:44:30.299 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-23T09:44:30.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.398+0000 7f82899ea640 1 Processor -- start 2024-03-23T09:44:30.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.398+0000 7f82899ea640 1 -- start start 2024-03-23T09:44:30.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.399+0000 7f82899ea640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f82840b8300 0x7f82840cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.399+0000 7f82899ea640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f82840cf6d0 0x7f82840d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.399+0000 7f82899ea640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82840d0ee0 con 0x7f828415cb40 2024-03-23T09:44:30.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.399+0000 7f82899ea640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82840b5a10 con 0x7f82840b8300 2024-03-23T09:44:30.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.399+0000 7f82899ea640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82840cfb60 con 0x7f82840cf6d0 2024-03-23T09:44:30.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.399+0000 7f8282ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f828415cb40 0x7f8284160770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32866/0 (socket says 172.21.15.47:32866) 2024-03-23T09:44:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.399+0000 7f8282ffd640 1 -- 172.21.15.47:0/1712867859 learned_addr learned my addr 172.21.15.47:0/1712867859 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:44:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.399+0000 7f82827fc640 1 --2- 172.21.15.47:0/1712867859 >> v2:172.21.15.47:3300/0 conn(0x7f82840b8300 0x7f82840cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.399+0000 7f82827fc640 1 -- 172.21.15.47:0/1712867859 >> v2:172.21.15.175:3300/0 conn(0x7f82840cf6d0 msgr2=0x7f82840d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.399+0000 7f82827fc640 1 --2- 172.21.15.47:0/1712867859 >> v2:172.21.15.175:3300/0 conn(0x7f82840cf6d0 0x7f82840d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.399+0000 7f82827fc640 1 -- 172.21.15.47:0/1712867859 >> v1:172.21.15.47:6789/0 conn(0x7f828415cb40 legacy=0x7f8284160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.399+0000 7f8281ffb640 1 -- 172.21.15.47:0/1712867859 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135102120 0 0) 0x7f82840d0ee0 con 0x7f828415cb40 2024-03-23T09:44:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.399+0000 7f82827fc640 1 -- 172.21.15.47:0/1712867859 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82841616f0 con 0x7f82840b8300 2024-03-23T09:44:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.400+0000 7f82827fc640 1 --2- 172.21.15.47:0/1712867859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82840b8300 0x7f82840cf190 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f826c0097a0 tx=0x7f826c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed503358a3b1515 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.400+0000 7f8281ffb640 1 -- 172.21.15.47:0/1712867859 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c031830 con 0x7f82840b8300 2024-03-23T09:44:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.400+0000 7f8281ffb640 1 -- 172.21.15.47:0/1712867859 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826c031990 con 0x7f82840b8300 2024-03-23T09:44:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.400+0000 7f8281ffb640 1 -- 172.21.15.47:0/1712867859 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c031d20 con 0x7f82840b8300 2024-03-23T09:44:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.401+0000 7f82899ea640 1 -- 172.21.15.47:0/1712867859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82840b8300 msgr2=0x7f82840cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.401+0000 7f82899ea640 1 --2- 172.21.15.47:0/1712867859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82840b8300 0x7f82840cf190 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f826c0097a0 tx=0x7f826c02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.401+0000 7f82899ea640 1 -- 172.21.15.47:0/1712867859 shutdown_connections 2024-03-23T09:44:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.401+0000 7f82899ea640 1 --2- 172.21.15.47:0/1712867859 >> v2:172.21.15.175:3300/0 conn(0x7f82840cf6d0 0x7f82840d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.401+0000 7f82899ea640 1 --2- 172.21.15.47:0/1712867859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82840b8300 0x7f82840cf190 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.401+0000 7f82899ea640 1 -- 172.21.15.47:0/1712867859 >> 172.21.15.47:0/1712867859 conn(0x7f828406b160 msgr2=0x7f828406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.401+0000 7f82899ea640 1 -- 172.21.15.47:0/1712867859 shutdown_connections 2024-03-23T09:44:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.401+0000 7f82899ea640 1 -- 172.21.15.47:0/1712867859 wait complete. 2024-03-23T09:44:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.402+0000 7f82899ea640 1 Processor -- start 2024-03-23T09:44:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.402+0000 7f82899ea640 1 -- start start 2024-03-23T09:44:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82899ea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82840b8300 0x7f82840c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82899ea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82840cf6d0 0x7f82840a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82899ea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828415cb40 0x7f8284159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82899ea640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828414a2b0 con 0x7f82840cf6d0 2024-03-23T09:44:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82899ea640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82841496a0 con 0x7f828415cb40 2024-03-23T09:44:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82899ea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8284149980 con 0x7f82840b8300 2024-03-23T09:44:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82827fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82840cf6d0 0x7f82840a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82827fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82840cf6d0 0x7f82840a97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60572/0 (socket says 172.21.15.47:60572) 2024-03-23T09:44:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82827fc640 1 -- 172.21.15.47:0/3697121644 learned_addr learned my addr 172.21.15.47:0/3697121644 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f8282ffd640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82840b8300 0x7f82840c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82837fe640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828415cb40 0x7f8284159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82827fc640 1 -- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82840b8300 msgr2=0x7f82840c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82827fc640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82840b8300 0x7f82840c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82827fc640 1 -- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828415cb40 msgr2=0x7f8284159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82827fc640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828415cb40 0x7f8284159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.403+0000 7f82827fc640 1 -- 172.21.15.47:0/3697121644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c009400 con 0x7f82840cf6d0 2024-03-23T09:44:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.404+0000 7f82827fc640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82840cf6d0 0x7f82840a97c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f826c000c00 tx=0x7f826c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.404+0000 7f827bfff640 1 -- 172.21.15.47:0/3697121644 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c004230 con 0x7f82840cf6d0 2024-03-23T09:44:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.404+0000 7f827bfff640 1 -- 172.21.15.47:0/3697121644 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826c004390 con 0x7f82840cf6d0 2024-03-23T09:44:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.404+0000 7f82899ea640 1 -- 172.21.15.47:0/3697121644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8284149c60 con 0x7f82840cf6d0 2024-03-23T09:44:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.404+0000 7f827bfff640 1 -- 172.21.15.47:0/3697121644 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c03a940 con 0x7f82840cf6d0 2024-03-23T09:44:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.404+0000 7f82899ea640 1 -- 172.21.15.47:0/3697121644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828415cf20 con 0x7f82840cf6d0 2024-03-23T09:44:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.405+0000 7f82899ea640 1 -- 172.21.15.47:0/3697121644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8284072360 con 0x7f82840cf6d0 2024-03-23T09:44:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.405+0000 7f827bfff640 1 -- 172.21.15.47:0/3697121644 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f826c03aaa0 con 0x7f82840cf6d0 2024-03-23T09:44:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.406+0000 7f827bfff640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f826406ac00 0x7f826406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.406+0000 7f827bfff640 1 -- 172.21.15.47:0/3697121644 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f826c0ab980 con 0x7f82840cf6d0 2024-03-23T09:44:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.406+0000 7f8282ffd640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f826406ac00 0x7f826406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.406+0000 7f8282ffd640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f826406ac00 0x7f826406d0c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f82740012e0 tx=0x7f8274001630 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.410+0000 7f827bfff640 1 -- 172.21.15.47:0/3697121644 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f826c076c00 con 0x7f82840cf6d0 2024-03-23T09:44:30.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.624+0000 7f82899ea640 1 -- 172.21.15.47:0/3697121644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8284066be0 con 0x7f82840cf6d0 2024-03-23T09:44:30.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.625+0000 7f827bfff640 1 -- 172.21.15.47:0/3697121644 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f826c035080 con 0x7f82840cf6d0 2024-03-23T09:44:30.626 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:44:30.626 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-23T09:44:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.626+0000 7f82899ea640 1 -- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f826406ac00 msgr2=0x7f826406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.626+0000 7f82899ea640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f826406ac00 0x7f826406d0c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f82740012e0 tx=0x7f8274001630 comp rx=0 tx=0).stop 2024-03-23T09:44:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.626+0000 7f82899ea640 1 -- 172.21.15.47:0/3697121644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82840cf6d0 msgr2=0x7f82840a97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.626+0000 7f82899ea640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82840cf6d0 0x7f82840a97c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f826c000c00 tx=0x7f826c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.626+0000 7f82899ea640 1 -- 172.21.15.47:0/3697121644 shutdown_connections 2024-03-23T09:44:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.626+0000 7f82899ea640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f826406ac00 0x7f826406d0c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.626+0000 7f82899ea640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828415cb40 0x7f8284159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.626+0000 7f82899ea640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82840cf6d0 0x7f82840a97c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.626+0000 7f82899ea640 1 --2- 172.21.15.47:0/3697121644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82840b8300 0x7f82840c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.626+0000 7f82899ea640 1 -- 172.21.15.47:0/3697121644 >> 172.21.15.47:0/3697121644 conn(0x7f828406b160 msgr2=0x7f82840d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.627+0000 7f82899ea640 1 -- 172.21.15.47:0/3697121644 shutdown_connections 2024-03-23T09:44:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.627+0000 7f82899ea640 1 -- 172.21.15.47:0/3697121644 wait complete. 2024-03-23T09:44:30.638 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-23T09:44:30.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:44:30.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-03-23T09:44:30.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.742+0000 7f28cc6b6640 1 Processor -- start 2024-03-23T09:44:30.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.742+0000 7f28cc6b6640 1 -- start start 2024-03-23T09:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.743+0000 7f28cc6b6640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c415a700 0x7f28c40cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.743+0000 7f28cc6b6640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f28c415d0c0 0x7f28c415d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.743+0000 7f28cc6b6640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c415dbd0 con 0x7f28c40d27f0 2024-03-23T09:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.743+0000 7f28cc6b6640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c415aae0 con 0x7f28c415d0c0 2024-03-23T09:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.743+0000 7f28cc6b6640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c415deb0 con 0x7f28c415a700 2024-03-23T09:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.743+0000 7f28cac53640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f28c40d27f0 0x7f28c40d2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32882/0 (socket says 172.21.15.47:32882) 2024-03-23T09:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.743+0000 7f28cac53640 1 -- 172.21.15.47:0/1245045303 learned_addr learned my addr 172.21.15.47:0/1245045303 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.743+0000 7f28ca452640 1 --2- 172.21.15.47:0/1245045303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c415a700 0x7f28c40cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.743+0000 7f28c9c51640 1 --2- 172.21.15.47:0/1245045303 >> v2:172.21.15.175:3300/0 conn(0x7f28c415d0c0 0x7f28c415d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.744+0000 7f28c9450640 1 -- 172.21.15.47:0/1245045303 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310352816 0 0) 0x7f28c415dbd0 con 0x7f28c40d27f0 2024-03-23T09:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.744+0000 7f28c9450640 1 -- 172.21.15.47:0/1245045303 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28ac003520 con 0x7f28c40d27f0 2024-03-23T09:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.744+0000 7f28c9c51640 1 -- 172.21.15.47:0/1245045303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c415a700 msgr2=0x7f28c40cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.744+0000 7f28c9c51640 1 --2- 172.21.15.47:0/1245045303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c415a700 0x7f28c40cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.744+0000 7f28c9c51640 1 -- 172.21.15.47:0/1245045303 >> v1:172.21.15.47:6789/0 conn(0x7f28c40d27f0 legacy=0x7f28c40d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.744+0000 7f28c9c51640 1 -- 172.21.15.47:0/1245045303 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c40cf340 con 0x7f28c415d0c0 2024-03-23T09:44:30.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.744+0000 7f28ca452640 1 --2- 172.21.15.47:0/1245045303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c415a700 0x7f28c40cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:44:30.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.745+0000 7f28c9c51640 1 --2- 172.21.15.47:0/1245045303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c415d0c0 0x7f28c415d500 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f28b80097a0 tx=0x7f28b802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3070d50a3da267e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.745+0000 7f28c9450640 1 -- 172.21.15.47:0/1245045303 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8031830 con 0x7f28c415d0c0 2024-03-23T09:44:30.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.745+0000 7f28c9450640 1 -- 172.21.15.47:0/1245045303 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b8031990 con 0x7f28c415d0c0 2024-03-23T09:44:30.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.745+0000 7f28c9450640 1 -- 172.21.15.47:0/1245045303 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8031d00 con 0x7f28c415d0c0 2024-03-23T09:44:30.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.745+0000 7f28cc6b6640 1 -- 172.21.15.47:0/1245045303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c415d0c0 msgr2=0x7f28c415d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.745+0000 7f28cc6b6640 1 --2- 172.21.15.47:0/1245045303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c415d0c0 0x7f28c415d500 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f28b80097a0 tx=0x7f28b802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.745+0000 7f28cc6b6640 1 -- 172.21.15.47:0/1245045303 shutdown_connections 2024-03-23T09:44:30.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.745+0000 7f28cc6b6640 1 --2- 172.21.15.47:0/1245045303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c415d0c0 0x7f28c415d500 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.745+0000 7f28cc6b6640 1 --2- 172.21.15.47:0/1245045303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c415a700 0x7f28c40cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.745+0000 7f28cc6b6640 1 -- 172.21.15.47:0/1245045303 >> 172.21.15.47:0/1245045303 conn(0x7f28c406afd0 msgr2=0x7f28c406b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.745+0000 7f28cc6b6640 1 -- 172.21.15.47:0/1245045303 shutdown_connections 2024-03-23T09:44:30.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.745+0000 7f28cc6b6640 1 -- 172.21.15.47:0/1245045303 wait complete. 2024-03-23T09:44:30.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.747+0000 7f28cc6b6640 1 Processor -- start 2024-03-23T09:44:30.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.747+0000 7f28cc6b6640 1 -- start start 2024-03-23T09:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.747+0000 7f28cc6b6640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c40d27f0 0x7f28c41590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.747+0000 7f28cc6b6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c415a700 0x7f28c4156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.747+0000 7f28cc6b6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c415d0c0 0x7f28c4147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.747+0000 7f28cc6b6640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c41596c0 con 0x7f28c415a700 2024-03-23T09:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.747+0000 7f28cc6b6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c4156b90 con 0x7f28c415d0c0 2024-03-23T09:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.747+0000 7f28cc6b6640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c40c6070 con 0x7f28c40d27f0 2024-03-23T09:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.747+0000 7f28c9c51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c415a700 0x7f28c4156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.747+0000 7f28c9c51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c415a700 0x7f28c4156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60578/0 (socket says 172.21.15.47:60578) 2024-03-23T09:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.747+0000 7f28c9c51640 1 -- 172.21.15.47:0/67625366 learned_addr learned my addr 172.21.15.47:0/67625366 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.748+0000 7f28cac53640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c415d0c0 0x7f28c4147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.748+0000 7f28ca452640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c40d27f0 0x7f28c41590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.748+0000 7f28c9c51640 1 -- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c40d27f0 msgr2=0x7f28c41590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.748+0000 7f28c9c51640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c40d27f0 0x7f28c41590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.748+0000 7f28c9c51640 1 -- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c415d0c0 msgr2=0x7f28c4147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.748+0000 7f28c9c51640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c415d0c0 0x7f28c4147cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.748+0000 7f28c9c51640 1 -- 172.21.15.47:0/67625366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc009400 con 0x7f28c415a700 2024-03-23T09:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.748+0000 7f28c9c51640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c415a700 0x7f28c4156650 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f28b8034040 tx=0x7f28b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.748+0000 7f28c2ffd640 1 -- 172.21.15.47:0/67625366 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8004240 con 0x7f28c415a700 2024-03-23T09:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.748+0000 7f28c2ffd640 1 -- 172.21.15.47:0/67625366 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b803f070 con 0x7f28c415a700 2024-03-23T09:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.748+0000 7f28cc6b6640 1 -- 172.21.15.47:0/67625366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b8009400 con 0x7f28c415a700 2024-03-23T09:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.748+0000 7f28c2ffd640 1 -- 172.21.15.47:0/67625366 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8003610 con 0x7f28c415a700 2024-03-23T09:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.748+0000 7f28cc6b6640 1 -- 172.21.15.47:0/67625366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c40b6e20 con 0x7f28c415a700 2024-03-23T09:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.750+0000 7f28c2ffd640 1 -- 172.21.15.47:0/67625366 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f28b8041030 con 0x7f28c415a700 2024-03-23T09:44:30.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.750+0000 7f28c2ffd640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f28b006ac00 0x7f28b006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.750+0000 7f28c2ffd640 1 -- 172.21.15.47:0/67625366 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28b80ab170 con 0x7f28c415a700 2024-03-23T09:44:30.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.750+0000 7f28ca452640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f28b006ac00 0x7f28b006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.751+0000 7f28ca452640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f28b006ac00 0x7f28b006d0c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009cd0 tx=0x7f28bc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.751+0000 7f28cf22c740 1 Processor -- start 2024-03-23T09:44:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.751+0000 7f28cf22c740 1 -- start start 2024-03-23T09:44:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28cf22c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821cfa680 0x55b821ccd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28cf22c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 0x55b821cfd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28cf22c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821ccdb60 0x55b821cfd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28cf22c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b821c3ece0 con 0x55b821cfaa60 2024-03-23T09:44:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28cf22c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b821c3c470 con 0x55b821cfa680 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28cf22c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b821c37d60 con 0x55b821ccdb60 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28c9c51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 0x55b821cfd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28c9c51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 0x55b821cfd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60582/0 (socket says 172.21.15.47:60582) 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28c9c51640 1 -- 172.21.15.47:0/605561172 learned_addr learned my addr 172.21.15.47:0/605561172 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28ca452640 1 --2- 172.21.15.47:0/605561172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821cfa680 0x55b821ccd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28cac53640 1 --2- 172.21.15.47:0/605561172 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821ccdb60 0x55b821cfd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28c9c51640 1 -- 172.21.15.47:0/605561172 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821ccdb60 msgr2=0x55b821cfd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28c9c51640 1 --2- 172.21.15.47:0/605561172 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821ccdb60 0x55b821cfd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28c9c51640 1 -- 172.21.15.47:0/605561172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821cfa680 msgr2=0x55b821ccd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28c9c51640 1 --2- 172.21.15.47:0/605561172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821cfa680 0x55b821ccd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28c9c51640 1 -- 172.21.15.47:0/605561172 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b821bde7d0 con 0x55b821cfaa60 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28ca452640 1 --2- 172.21.15.47:0/605561172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821cfa680 0x55b821ccd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.752+0000 7f28c9c51640 1 --2- 172.21.15.47:0/605561172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 0x55b821cfd410 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f28b80a96c0 tx=0x7f28b8079b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be02c534d7b9f06a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.753+0000 7f28abfff640 1 -- 172.21.15.47:0/605561172 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8082400 con 0x55b821cfaa60 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.753+0000 7f28abfff640 1 -- 172.21.15.47:0/605561172 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b8078460 con 0x55b821cfaa60 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.753+0000 7f28abfff640 1 -- 172.21.15.47:0/605561172 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b80795b0 con 0x55b821cfaa60 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.753+0000 7f28cf22c740 1 -- 172.21.15.47:0/605561172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 msgr2=0x55b821cfd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.753+0000 7f28cf22c740 1 --2- 172.21.15.47:0/605561172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 0x55b821cfd410 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f28b80a96c0 tx=0x7f28b8079b50 comp rx=0 tx=0).stop 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.753+0000 7f28cf22c740 1 -- 172.21.15.47:0/605561172 shutdown_connections 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.753+0000 7f28cf22c740 1 --2- 172.21.15.47:0/605561172 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821ccdb60 0x55b821cfd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.753+0000 7f28cf22c740 1 --2- 172.21.15.47:0/605561172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 0x55b821cfd410 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.753+0000 7f28cf22c740 1 --2- 172.21.15.47:0/605561172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821cfa680 0x55b821ccd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.753+0000 7f28cf22c740 1 -- 172.21.15.47:0/605561172 >> 172.21.15.47:0/605561172 conn(0x55b821ccf890 msgr2=0x55b821d18410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.753+0000 7f28cf22c740 1 -- 172.21.15.47:0/605561172 shutdown_connections 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.753+0000 7f28cf22c740 1 -- 172.21.15.47:0/605561172 wait complete. 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.754+0000 7f28cf22c740 1 -- start start 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.754+0000 7f28cf22c740 1 Processor -- start 2024-03-23T09:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28cf22c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 0x55b821d28540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28cf22c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfaa60 0x55b821cfd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28cf22c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 0x55b821d005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28cf22c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b821bf5f30 con 0x55b821cfa680 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28cf22c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b821bf5bd0 con 0x55b821d00170 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28cf22c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b821bbb190 con 0x55b821cfaa60 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28cf22c740 10 client.0 _ll_register_callbacks cb 0x55b821c313d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28ca452640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 0x55b821d28540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28ca452640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 0x55b821d28540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60590/0 (socket says 172.21.15.47:60590) 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28ca452640 1 -- 172.21.15.47:0/4078903446 learned_addr learned my addr 172.21.15.47:0/4078903446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28ca452640 1 -- 172.21.15.47:0/4078903446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfaa60 msgr2=0x55b821cfd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28ca452640 1 --2- 172.21.15.47:0/4078903446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfaa60 0x55b821cfd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28ca452640 1 -- 172.21.15.47:0/4078903446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 msgr2=0x55b821d005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28ca452640 1 --2- 172.21.15.47:0/4078903446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 0x55b821d005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28ca452640 1 -- 172.21.15.47:0/4078903446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c40b6e20 con 0x55b821cfa680 2024-03-23T09:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28ca452640 1 --2- 172.21.15.47:0/4078903446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 0x55b821d28540 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0093d0 tx=0x7f28bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0737608b713eb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.755+0000 7f28a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28a97fa640 1 -- 172.21.15.47:0/4078903446 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0138f0 con 0x55b821cfa680 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28a97fa640 1 -- 172.21.15.47:0/4078903446 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28bc01c040 con 0x55b821cfa680 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28cf22c740 1 -- 172.21.15.47:0/4078903446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b821c29080 con 0x55b821cfa680 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28a97fa640 1 -- 172.21.15.47:0/4078903446 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28bc013a50 con 0x55b821cfa680 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28cf22c740 10 client.6498 fetch_fsmap learned FSMap version 25 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28cf22c740 1 -- 172.21.15.47:0/4078903446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55b821c363e0 con 0x55b821cfa680 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28a97fa640 1 -- 172.21.15.47:0/4078903446 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b821c29080 con 0x55b821cfa680 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28a97fa640 1 -- 172.21.15.47:0/4078903446 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f28bc013bb0 con 0x55b821cfa680 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28cf22c740 10 client.6498 fetch_fsmap finished waiting for FSMap version 25 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28cf22c740 10 client.6498 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28cf22c740 20 client.6498 populate_metadata read hostname 'smithi047' 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28cf22c740 1 --2- 172.21.15.47:0/4078903446 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55b821cfb350 0x55b821cfb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28cf22c740 4 client.6498 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28cf22c740 1 -- 172.21.15.47:0/4078903446 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b82192b620 con 0x55b821cfb350 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.756+0000 7f28c9c51640 1 --2- 172.21.15.47:0/4078903446 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55b821cfb350 0x55b821cfb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.757+0000 7f28a97fa640 10 client.6498 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.757+0000 7f28c9c51640 1 --2- 172.21.15.47:0/4078903446 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55b821cfb350 0x55b821cfb730 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=182d35c79f973e44 server_cookie=1bb532ecbb705d95 in_seq=0 out_seq=0 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.757+0000 7f28a97fa640 1 -- 172.21.15.47:0/4078903446 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b82192b620 con 0x55b821cfb350 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.757+0000 7f28a97fa640 10 client.6498 handle_command_reply: tid=0 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.757+0000 7f28cf22c740 1 client.6498 shutdown 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.757+0000 7f28cf22c740 1 -- 172.21.15.47:0/4078903446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 msgr2=0x55b821d28540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.757+0000 7f28cf22c740 1 --2- 172.21.15.47:0/4078903446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 0x55b821d28540 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0093d0 tx=0x7f28bc0046c0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.758+0000 7f28cf22c740 1 -- 172.21.15.47:0/4078903446 shutdown_connections 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.758+0000 7f28cf22c740 1 --2- 172.21.15.47:0/4078903446 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55b821cfb350 0x55b821cfb730 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.758+0000 7f28cf22c740 1 --2- 172.21.15.47:0/4078903446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 0x55b821d005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.758+0000 7f28cf22c740 1 --2- 172.21.15.47:0/4078903446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfaa60 0x55b821cfd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.758+0000 7f28a97fa640 0 client.6498 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.758+0000 7f28cf22c740 1 --2- 172.21.15.47:0/4078903446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 0x55b821d28540 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.758+0000 7f28cf22c740 1 -- 172.21.15.47:0/4078903446 >> 172.21.15.47:0/4078903446 conn(0x55b821ccdb60 msgr2=0x55b821cd15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.758+0000 7f28cf22c740 1 -- 172.21.15.47:0/4078903446 shutdown_connections 2024-03-23T09:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.758+0000 7f28cf22c740 1 -- 172.21.15.47:0/4078903446 wait complete. 2024-03-23T09:44:30.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.758+0000 7f28cf22c740 20 client.6498 trim_cache size 0 max 16384 2024-03-23T09:44:30.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.772+0000 7f28cf22c740 1 Processor -- start 2024-03-23T09:44:30.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.772+0000 7f28cf22c740 1 -- start start 2024-03-23T09:44:30.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28cf22c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfa680 0x55b821d0f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28cf22c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 0x55b821d11df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28cf22c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 0x55b821d12330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28cf22c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b821bd8020 con 0x55b821cfaa60 2024-03-23T09:44:30.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28cf22c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b821cfc0f0 con 0x55b821d00170 2024-03-23T09:44:30.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28cf22c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b821bc4040 con 0x55b821cfa680 2024-03-23T09:44:30.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28c9c51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 0x55b821d11df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28c9c51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 0x55b821d11df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60604/0 (socket says 172.21.15.47:60604) 2024-03-23T09:44:30.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28c9c51640 1 -- 172.21.15.47:0/3065696464 learned_addr learned my addr 172.21.15.47:0/3065696464 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28cac53640 1 --2- 172.21.15.47:0/3065696464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 0x55b821d12330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28ca452640 1 --2- 172.21.15.47:0/3065696464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfa680 0x55b821d0f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28c9c51640 1 -- 172.21.15.47:0/3065696464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfa680 msgr2=0x55b821d0f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28c9c51640 1 --2- 172.21.15.47:0/3065696464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfa680 0x55b821d0f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28c9c51640 1 -- 172.21.15.47:0/3065696464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 msgr2=0x55b821d12330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28c9c51640 1 --2- 172.21.15.47:0/3065696464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 0x55b821d12330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28c9c51640 1 -- 172.21.15.47:0/3065696464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b821bd81c0 con 0x55b821cfaa60 2024-03-23T09:44:30.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28cac53640 1 --2- 172.21.15.47:0/3065696464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 0x55b821d12330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:44:30.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28ca452640 1 --2- 172.21.15.47:0/3065696464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfa680 0x55b821d0f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:44:30.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28c9c51640 1 --2- 172.21.15.47:0/3065696464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 0x55b821d11df0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f28b80097a0 tx=0x7f28b807a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91cf0d2919cc38a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28c0ff9640 1 -- 172.21.15.47:0/3065696464 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b80a7070 con 0x55b821cfaa60 2024-03-23T09:44:30.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28c0ff9640 1 -- 172.21.15.47:0/3065696464 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b8078280 con 0x55b821cfaa60 2024-03-23T09:44:30.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.773+0000 7f28c0ff9640 1 -- 172.21.15.47:0/3065696464 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b80a9040 con 0x55b821cfaa60 2024-03-23T09:44:30.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.774+0000 7f28cf22c740 1 -- 172.21.15.47:0/3065696464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 msgr2=0x55b821d11df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.774+0000 7f28cf22c740 1 --2- 172.21.15.47:0/3065696464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 0x55b821d11df0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f28b80097a0 tx=0x7f28b807a240 comp rx=0 tx=0).stop 2024-03-23T09:44:30.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.774+0000 7f28cf22c740 1 -- 172.21.15.47:0/3065696464 shutdown_connections 2024-03-23T09:44:30.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.774+0000 7f28cf22c740 1 --2- 172.21.15.47:0/3065696464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 0x55b821d12330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.774+0000 7f28cf22c740 1 --2- 172.21.15.47:0/3065696464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfaa60 0x55b821d11df0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.774+0000 7f28cf22c740 1 --2- 172.21.15.47:0/3065696464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfa680 0x55b821d0f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.774+0000 7f28cf22c740 1 -- 172.21.15.47:0/3065696464 >> 172.21.15.47:0/3065696464 conn(0x55b821ccdb60 msgr2=0x55b821d1ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.774+0000 7f28cf22c740 1 -- 172.21.15.47:0/3065696464 shutdown_connections 2024-03-23T09:44:30.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.774+0000 7f28cf22c740 1 -- 172.21.15.47:0/3065696464 wait complete. 2024-03-23T09:44:30.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.774+0000 7f28cf22c740 1 -- start start 2024-03-23T09:44:30.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.775+0000 7f28cf22c740 1 Processor -- start 2024-03-23T09:44:30.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.775+0000 7f28cf22c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 0x55b821d7e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.775+0000 7f28cf22c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfaa60 0x55b821d26cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.775+0000 7f28cf22c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 0x55b821d27210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.775+0000 7f28cf22c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b821d0f720 con 0x55b821cfa680 2024-03-23T09:44:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.775+0000 7f28cf22c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b821cf5f50 con 0x55b821d00170 2024-03-23T09:44:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.775+0000 7f28cf22c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b821d0f4d0 con 0x55b821cfaa60 2024-03-23T09:44:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.775+0000 7f28cf22c740 10 client.0 _ll_register_callbacks cb 0x7f28bc009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:44:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28ca452640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 0x55b821d7e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28ca452640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 0x55b821d7e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60612/0 (socket says 172.21.15.47:60612) 2024-03-23T09:44:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28ca452640 1 -- 172.21.15.47:0/2455984350 learned_addr learned my addr 172.21.15.47:0/2455984350 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28cac53640 1 --2- 172.21.15.47:0/2455984350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 0x55b821d27210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28c9c51640 1 --2- 172.21.15.47:0/2455984350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfaa60 0x55b821d26cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28ca452640 1 -- 172.21.15.47:0/2455984350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfaa60 msgr2=0x55b821d26cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28ca452640 1 --2- 172.21.15.47:0/2455984350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfaa60 0x55b821d26cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28ca452640 1 -- 172.21.15.47:0/2455984350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 msgr2=0x55b821d27210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28ca452640 1 --2- 172.21.15.47:0/2455984350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 0x55b821d27210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28ca452640 1 -- 172.21.15.47:0/2455984350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b821bde7d0 con 0x55b821cfa680 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28ca452640 1 --2- 172.21.15.47:0/2455984350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 0x55b821d7e530 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f28bc014b80 tx=0x7f28bc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee70ec0ab5a7bec server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28a97fa640 1 -- 172.21.15.47:0/2455984350 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc004300 con 0x55b821cfa680 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28a97fa640 1 -- 172.21.15.47:0/2455984350 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28bc004460 con 0x55b821cfa680 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28cf22c740 1 -- 172.21.15.47:0/2455984350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28b8078460 con 0x55b821cfa680 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28a97fa640 1 -- 172.21.15.47:0/2455984350 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28bc004620 con 0x55b821cfa680 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28cf22c740 10 client.6504 fetch_fsmap learned FSMap version 25 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28cf22c740 1 -- 172.21.15.47:0/2455984350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55b821bfd990 con 0x55b821cfa680 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.776+0000 7f28a97fa640 1 -- 172.21.15.47:0/2455984350 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b8078460 con 0x55b821cfa680 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.777+0000 7f28a97fa640 1 -- 172.21.15.47:0/2455984350 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f28bc003610 con 0x55b821cfa680 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.777+0000 7f28cf22c740 10 client.6504 fetch_fsmap finished waiting for FSMap version 25 2024-03-23T09:44:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.777+0000 7f28cf22c740 10 client.6504 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.777+0000 7f28cf22c740 20 client.6504 populate_metadata read hostname 'smithi047' 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.777+0000 7f28cf22c740 1 --2- 172.21.15.47:0/2455984350 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55b821cfc410 0x55b821d82140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.777+0000 7f28cf22c740 4 client.6504 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.777+0000 7f28cf22c740 1 -- 172.21.15.47:0/2455984350 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) v1 -- 0x55b82193c920 con 0x55b821cfc410 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.777+0000 7f28c9c51640 1 --2- 172.21.15.47:0/2455984350 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55b821cfc410 0x55b821d82140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.777+0000 7f28c9c51640 1 --2- 172.21.15.47:0/2455984350 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55b821cfc410 0x55b821d82140 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1857f9dc0e4a2837 server_cookie=4785e5fab164f9df in_seq=0 out_seq=0 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.777+0000 7f28a97fa640 10 client.6504 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28a97fa640 1 -- 172.21.15.47:0/2455984350 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b82193c920 con 0x55b821cfc410 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28a97fa640 10 client.6504 handle_command_reply: tid=0 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28cf22c740 1 client.6504 shutdown 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28cf22c740 1 -- 172.21.15.47:0/2455984350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 msgr2=0x55b821d7e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28cf22c740 1 --2- 172.21.15.47:0/2455984350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 0x55b821d7e530 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f28bc014b80 tx=0x7f28bc019000 comp rx=0 tx=0).stop 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28cf22c740 1 -- 172.21.15.47:0/2455984350 shutdown_connections 2024-03-23T09:44:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28cf22c740 1 --2- 172.21.15.47:0/2455984350 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55b821cfc410 0x55b821d82140 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28cf22c740 1 --2- 172.21.15.47:0/2455984350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b821d00170 0x55b821d27210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28cf22c740 1 --2- 172.21.15.47:0/2455984350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b821cfaa60 0x55b821d26cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28a97fa640 0 client.6504 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:30.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28cf22c740 1 --2- 172.21.15.47:0/2455984350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b821cfa680 0x55b821d7e530 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28cf22c740 1 -- 172.21.15.47:0/2455984350 >> 172.21.15.47:0/2455984350 conn(0x55b821ccdb60 msgr2=0x55b821d28350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28cf22c740 1 -- 172.21.15.47:0/2455984350 shutdown_connections 2024-03-23T09:44:30.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28cf22c740 1 -- 172.21.15.47:0/2455984350 wait complete. 2024-03-23T09:44:30.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.778+0000 7f28cf22c740 20 client.6504 trim_cache size 0 max 16384 2024-03-23T09:44:30.786 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:44:30.786 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-23T09:44:30.787 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "de47f180-be3e-4a6b-9877-9fb63f82df18", 2024-03-23T09:44:30.787 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-23T09:44:30.787 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:44:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.779+0000 7f28cc6b6640 1 -- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f28b006ac00 msgr2=0x7f28b006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.779+0000 7f28cc6b6640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f28b006ac00 0x7f28b006d0c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009cd0 tx=0x7f28bc009190 comp rx=0 tx=0).stop 2024-03-23T09:44:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.779+0000 7f28cc6b6640 1 -- 172.21.15.47:0/67625366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c415a700 msgr2=0x7f28c4156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.779+0000 7f28cc6b6640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c415a700 0x7f28c4156650 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f28b8034040 tx=0x7f28b80046c0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.779+0000 7f28cc6b6640 1 -- 172.21.15.47:0/67625366 shutdown_connections 2024-03-23T09:44:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.779+0000 7f28cc6b6640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f28b006ac00 0x7f28b006d0c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.779+0000 7f28cc6b6640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c415d0c0 0x7f28c4147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.779+0000 7f28cc6b6640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c415a700 0x7f28c4156650 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.779+0000 7f28cc6b6640 1 --2- 172.21.15.47:0/67625366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c40d27f0 0x7f28c41590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.779+0000 7f28cc6b6640 1 -- 172.21.15.47:0/67625366 >> 172.21.15.47:0/67625366 conn(0x7f28c406afd0 msgr2=0x7f28c406b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.779+0000 7f28cc6b6640 1 -- 172.21.15.47:0/67625366 shutdown_connections 2024-03-23T09:44:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.779+0000 7f28cc6b6640 1 -- 172.21.15.47:0/67625366 wait complete. 2024-03-23T09:44:30.789 INFO:tasks.ceph.mds.d.smithi175.stderr:2024-03-23T09:44:30.779+0000 7fa9142b1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-03-23T09:43:58.266000+0000 1=1+0) (iversion lock) caps={6369=pAsLsXsFs/-@2},l=6369(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x559dc139fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T09:44:30.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:44:30.790 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-23T09:44:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.892+0000 7f294024a640 1 Processor -- start 2024-03-23T09:44:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.893+0000 7f294024a640 1 -- start start 2024-03-23T09:44:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.893+0000 7f294024a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29380cde10 0x7f2938160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.893+0000 7f294024a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2938161630 con 0x7f29380b8350 2024-03-23T09:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f294024a640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29380d1030 con 0x7f293815c990 2024-03-23T09:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f294024a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29380cfe10 con 0x7f29380cde10 2024-03-23T09:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293e7e7640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f29380b8350 0x7f29380d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32892/0 (socket says 172.21.15.47:32892) 2024-03-23T09:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293e7e7640 1 -- 172.21.15.47:0/1527635345 learned_addr learned my addr 172.21.15.47:0/1527635345 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293d7e5640 1 --2- 172.21.15.47:0/1527635345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29380cde10 0x7f2938160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40248033 0 0) 0x7f2938161630 con 0x7f29380b8350 2024-03-23T09:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2924003520 con 0x7f29380b8350 2024-03-23T09:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4255786831 0 0) 0x7f2924003520 con 0x7f29380b8350 2024-03-23T09:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29380cde10 msgr2=0x7f2938160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 --2- 172.21.15.47:0/1527635345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29380cde10 0x7f2938160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 >> v1:172.21.15.175:6789/0 conn(0x7f293815c990 legacy=0x7f29380cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29380b5b60 con 0x7f29380b8350 2024-03-23T09:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f29340019f0 con 0x7f29380b8350 2024-03-23T09:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 >> v1:172.21.15.47:6789/0 conn(0x7f29380b8350 legacy=0x7f29380d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 --2- 172.21.15.47:0/1527635345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29240039a0 0x7f2924003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 --2- 172.21.15.47:0/1527635345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29380cde10 0x7f29240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 --2- 172.21.15.47:0/1527635345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2924004d00 0x7f2924005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29380cfe10 con 0x7f2924004d00 2024-03-23T09:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292400bb40 con 0x7f29240039a0 2024-03-23T09:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292400bd00 con 0x7f29380cde10 2024-03-23T09:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293e7e7640 1 --2- 172.21.15.47:0/1527635345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2924004d00 0x7f2924005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293d7e5640 1 --2- 172.21.15.47:0/1527635345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29380cde10 0x7f29240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.894+0000 7f293dfe6640 1 --2- 172.21.15.47:0/1527635345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29240039a0 0x7f2924003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.895+0000 7f293e7e7640 1 -- 172.21.15.47:0/1527635345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29380cde10 msgr2=0x7f29240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.895+0000 7f293e7e7640 1 --2- 172.21.15.47:0/1527635345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29380cde10 0x7f29240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.895+0000 7f293e7e7640 1 -- 172.21.15.47:0/1527635345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29240039a0 msgr2=0x7f2924003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.895+0000 7f293e7e7640 1 --2- 172.21.15.47:0/1527635345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29240039a0 0x7f2924003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.895+0000 7f293e7e7640 1 -- 172.21.15.47:0/1527635345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2924003520 con 0x7f2924004d00 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.895+0000 7f293e7e7640 1 --2- 172.21.15.47:0/1527635345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2924004d00 0x7f2924005180 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f293400a830 tx=0x7f29240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dce93ec336ae338 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.896+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29340072d0 con 0x7f2924004d00 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.896+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293402f070 con 0x7f2924004d00 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.896+0000 7f293cfe4640 1 -- 172.21.15.47:0/1527635345 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29340076b0 con 0x7f2924004d00 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.896+0000 7f294024a640 1 -- 172.21.15.47:0/1527635345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2924004d00 msgr2=0x7f2924005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.896+0000 7f294024a640 1 --2- 172.21.15.47:0/1527635345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2924004d00 0x7f2924005180 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f293400a830 tx=0x7f29240057f0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.896+0000 7f293e7e7640 1 -- 172.21.15.47:0/1527635345 reap_dead start 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.896+0000 7f294024a640 1 -- 172.21.15.47:0/1527635345 shutdown_connections 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.896+0000 7f294024a640 1 -- 172.21.15.47:0/1527635345 >> 172.21.15.47:0/1527635345 conn(0x7f29380659d0 msgr2=0x7f2938065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.896+0000 7f294024a640 1 -- 172.21.15.47:0/1527635345 shutdown_connections 2024-03-23T09:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.896+0000 7f294024a640 1 -- 172.21.15.47:0/1527635345 wait complete. 2024-03-23T09:44:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.897+0000 7f294024a640 1 Processor -- start 2024-03-23T09:44:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.897+0000 7f294024a640 1 -- start start 2024-03-23T09:44:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.898+0000 7f294024a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f293815f2a0 0x7f29380c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.898+0000 7f294024a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2938158db0 0x7f29380c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.898+0000 7f294024a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f293815f680 0x7f2938152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.898+0000 7f294024a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2938153880 con 0x7f293815f680 2024-03-23T09:44:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.898+0000 7f294024a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293815d0f0 con 0x7f293815f2a0 2024-03-23T09:44:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.898+0000 7f294024a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293815d3d0 con 0x7f2938158db0 2024-03-23T09:44:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.898+0000 7f293e7e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f293815f680 0x7f2938152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.898+0000 7f293e7e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f293815f680 0x7f2938152b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60632/0 (socket says 172.21.15.47:60632) 2024-03-23T09:44:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.898+0000 7f293e7e7640 1 -- 172.21.15.47:0/1435370398 learned_addr learned my addr 172.21.15.47:0/1435370398 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.898+0000 7f293dfe6640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f293815f2a0 0x7f29380c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.898+0000 7f293d7e5640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2938158db0 0x7f29380c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.899+0000 7f293e7e7640 1 -- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2938158db0 msgr2=0x7f29380c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.899+0000 7f293e7e7640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2938158db0 0x7f29380c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.899+0000 7f293e7e7640 1 -- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f293815f2a0 msgr2=0x7f29380c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.899+0000 7f293e7e7640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f293815f2a0 0x7f29380c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.899+0000 7f293e7e7640 1 -- 172.21.15.47:0/1435370398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293815d6b0 con 0x7f293815f680 2024-03-23T09:44:30.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.899+0000 7f293e7e7640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f293815f680 0x7f2938152b10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f293400a830 tx=0x7f2934001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.899+0000 7f292e7fc640 1 -- 172.21.15.47:0/1435370398 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934044070 con 0x7f293815f680 2024-03-23T09:44:30.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.899+0000 7f292e7fc640 1 -- 172.21.15.47:0/1435370398 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2934001ae0 con 0x7f293815f680 2024-03-23T09:44:30.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.899+0000 7f294024a640 1 -- 172.21.15.47:0/1435370398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293815d990 con 0x7f293815f680 2024-03-23T09:44:30.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.899+0000 7f292e7fc640 1 -- 172.21.15.47:0/1435370398 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934007500 con 0x7f293815f680 2024-03-23T09:44:30.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.899+0000 7f294024a640 1 -- 172.21.15.47:0/1435370398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2938153230 con 0x7f293815f680 2024-03-23T09:44:30.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.900+0000 7f292e7fc640 1 -- 172.21.15.47:0/1435370398 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2934031030 con 0x7f293815f680 2024-03-23T09:44:30.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.901+0000 7f292e7fc640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f291c06ac30 0x7f291c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.901+0000 7f292e7fc640 1 -- 172.21.15.47:0/1435370398 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29340aa810 con 0x7f293815f680 2024-03-23T09:44:30.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.901+0000 7f293dfe6640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f291c06ac30 0x7f291c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.902+0000 7f293dfe6640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f291c06ac30 0x7f291c06d0f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f29300035f0 tx=0x7f2930009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.902+0000 7f2942dc0740 1 Processor -- start 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.902+0000 7f2942dc0740 1 -- start start 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f2942dc0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af53830 0x55eb9af53c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f2942dc0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af23850 0x55eb9af50aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f2942dc0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af273b0 0x55eb9af50fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f2942dc0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb9ae93dd0 con 0x55eb9af23850 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f2942dc0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb9ae91560 con 0x55eb9af53830 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f2942dc0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb9ae8ce50 con 0x55eb9af273b0 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f293d7e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af23850 0x55eb9af50aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f293d7e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af23850 0x55eb9af50aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60636/0 (socket says 172.21.15.47:60636) 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f293d7e5640 1 -- 172.21.15.47:0/3214460827 learned_addr learned my addr 172.21.15.47:0/3214460827 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f293dfe6640 1 --2- 172.21.15.47:0/3214460827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af53830 0x55eb9af53c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f293e7e7640 1 --2- 172.21.15.47:0/3214460827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af273b0 0x55eb9af50fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f293d7e5640 1 -- 172.21.15.47:0/3214460827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af273b0 msgr2=0x55eb9af50fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f293d7e5640 1 --2- 172.21.15.47:0/3214460827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af273b0 0x55eb9af50fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f293d7e5640 1 -- 172.21.15.47:0/3214460827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af53830 msgr2=0x55eb9af53c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f293d7e5640 1 --2- 172.21.15.47:0/3214460827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af53830 0x55eb9af53c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f293d7e5640 1 -- 172.21.15.47:0/3214460827 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb9ae54470 con 0x55eb9af23850 2024-03-23T09:44:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f293d7e5640 1 --2- 172.21.15.47:0/3214460827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af23850 0x55eb9af50aa0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2928009ba0 tx=0x7f2928009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481daa5686bed113 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f291b7fe640 1 -- 172.21.15.47:0/3214460827 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928014ee0 con 0x55eb9af23850 2024-03-23T09:44:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f291b7fe640 1 -- 172.21.15.47:0/3214460827 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2928004030 con 0x55eb9af23850 2024-03-23T09:44:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.903+0000 7f291b7fe640 1 -- 172.21.15.47:0/3214460827 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928009ea0 con 0x55eb9af23850 2024-03-23T09:44:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.904+0000 7f2942dc0740 1 -- 172.21.15.47:0/3214460827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af23850 msgr2=0x55eb9af50aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.904+0000 7f2942dc0740 1 --2- 172.21.15.47:0/3214460827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af23850 0x55eb9af50aa0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2928009ba0 tx=0x7f2928009190 comp rx=0 tx=0).stop 2024-03-23T09:44:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.904+0000 7f2942dc0740 1 -- 172.21.15.47:0/3214460827 shutdown_connections 2024-03-23T09:44:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.904+0000 7f2942dc0740 1 --2- 172.21.15.47:0/3214460827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af273b0 0x55eb9af50fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.904+0000 7f2942dc0740 1 --2- 172.21.15.47:0/3214460827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af23850 0x55eb9af50aa0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.904+0000 7f2942dc0740 1 --2- 172.21.15.47:0/3214460827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af53830 0x55eb9af53c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.904+0000 7f2942dc0740 1 -- 172.21.15.47:0/3214460827 >> 172.21.15.47:0/3214460827 conn(0x55eb9af25ac0 msgr2=0x55eb9af6e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.904+0000 7f2942dc0740 1 -- 172.21.15.47:0/3214460827 shutdown_connections 2024-03-23T09:44:30.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.904+0000 7f2942dc0740 1 -- 172.21.15.47:0/3214460827 wait complete. 2024-03-23T09:44:30.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.905+0000 7f2942dc0740 1 -- start start 2024-03-23T09:44:30.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.905+0000 7f2942dc0740 1 Processor -- start 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f2942dc0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 0x55eb9af661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f2942dc0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 0x55eb9af3fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f2942dc0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af56b40 0x55eb9af66a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f2942dc0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb9ae4b280 con 0x55eb9af53830 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f2942dc0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb9ae4af20 con 0x55eb9af273b0 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f2942dc0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb9ae103c0 con 0x55eb9af56b40 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f2942dc0740 10 client.0 _ll_register_callbacks cb 0x55eb9ae62380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f293d7e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 0x55eb9af3fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f293d7e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 0x55eb9af3fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60638/0 (socket says 172.21.15.47:60638) 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f293d7e5640 1 -- 172.21.15.47:0/719441452 learned_addr learned my addr 172.21.15.47:0/719441452 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f293e7e7640 1 --2- 172.21.15.47:0/719441452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af56b40 0x55eb9af66a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f293dfe6640 1 --2- 172.21.15.47:0/719441452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 0x55eb9af661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f293d7e5640 1 -- 172.21.15.47:0/719441452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af56b40 msgr2=0x55eb9af66a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f293d7e5640 1 --2- 172.21.15.47:0/719441452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af56b40 0x55eb9af66a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f293d7e5640 1 -- 172.21.15.47:0/719441452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 msgr2=0x55eb9af661e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f293d7e5640 1 --2- 172.21.15.47:0/719441452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 0x55eb9af661e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f293d7e5640 1 -- 172.21.15.47:0/719441452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29280097d0 con 0x55eb9af53830 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f293d7e5640 1 --2- 172.21.15.47:0/719441452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 0x55eb9af3fce0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2928008010 tx=0x7f29280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5619cc0c16e07e62 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f2918ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f2918ff9640 1 -- 172.21.15.47:0/719441452 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928025070 con 0x55eb9af53830 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f2918ff9640 1 -- 172.21.15.47:0/719441452 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f292801a030 con 0x55eb9af53830 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.906+0000 7f2942dc0740 1 -- 172.21.15.47:0/719441452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb9aeb6690 con 0x55eb9af53830 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.907+0000 7f2918ff9640 1 -- 172.21.15.47:0/719441452 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2928009ea0 con 0x55eb9af53830 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.907+0000 7f2942dc0740 10 client.6516 fetch_fsmap learned FSMap version 25 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.907+0000 7f2942dc0740 1 -- 172.21.15.47:0/719441452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55eb9ae8b4d0 con 0x55eb9af53830 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.907+0000 7f2918ff9640 1 -- 172.21.15.47:0/719441452 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb9aeb6690 con 0x55eb9af53830 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.907+0000 7f2918ff9640 1 -- 172.21.15.47:0/719441452 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2928011070 con 0x55eb9af53830 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.907+0000 7f2942dc0740 10 client.6516 fetch_fsmap finished waiting for FSMap version 25 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.907+0000 7f2942dc0740 10 client.6516 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-23T09:44:30.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.907+0000 7f2942dc0740 20 client.6516 populate_metadata read hostname 'smithi047' 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.907+0000 7f2942dc0740 1 --2- 172.21.15.47:0/719441452 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55eb9af40be0 0x55eb9af650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.907+0000 7f2942dc0740 4 client.6516 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.907+0000 7f2942dc0740 1 -- 172.21.15.47:0/719441452 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb9ae7eeb0 con 0x55eb9af40be0 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.907+0000 7f293dfe6640 1 --2- 172.21.15.47:0/719441452 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55eb9af40be0 0x55eb9af650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f2918ff9640 10 client.6516 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f293dfe6640 1 --2- 172.21.15.47:0/719441452 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55eb9af40be0 0x55eb9af650d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f1a03cf4d7ed1a server_cookie=df87e5afa7d4eaa7 in_seq=0 out_seq=0 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f2918ff9640 1 -- 172.21.15.47:0/719441452 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55eb9ae7eeb0 con 0x55eb9af40be0 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f2918ff9640 10 client.6516 handle_command_reply: tid=0 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f2942dc0740 1 client.6516 shutdown 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f2942dc0740 1 -- 172.21.15.47:0/719441452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 msgr2=0x55eb9af3fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f2942dc0740 1 --2- 172.21.15.47:0/719441452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 0x55eb9af3fce0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2928008010 tx=0x7f29280046c0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f2942dc0740 1 -- 172.21.15.47:0/719441452 shutdown_connections 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f2942dc0740 1 --2- 172.21.15.47:0/719441452 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55eb9af40be0 0x55eb9af650d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f2942dc0740 1 --2- 172.21.15.47:0/719441452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af56b40 0x55eb9af66a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f2942dc0740 1 --2- 172.21.15.47:0/719441452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 0x55eb9af3fce0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f2918ff9640 0 client.6516 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:30.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f2942dc0740 1 --2- 172.21.15.47:0/719441452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 0x55eb9af661e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.908+0000 7f2942dc0740 1 -- 172.21.15.47:0/719441452 >> 172.21.15.47:0/719441452 conn(0x55eb9af23850 msgr2=0x55eb9af27c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.909+0000 7f2942dc0740 1 -- 172.21.15.47:0/719441452 shutdown_connections 2024-03-23T09:44:30.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.909+0000 7f2942dc0740 1 -- 172.21.15.47:0/719441452 wait complete. 2024-03-23T09:44:30.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.909+0000 7f2942dc0740 20 client.6516 trim_cache size 0 max 16384 2024-03-23T09:44:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.923+0000 7f2942dc0740 1 Processor -- start 2024-03-23T09:44:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.923+0000 7f2942dc0740 1 -- start start 2024-03-23T09:44:30.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f2942dc0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 0x55eb9af29b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f2942dc0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 0x55eb9af53c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f2942dc0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af56b40 0x55eb9af54150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f2942dc0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb9ae19050 con 0x55eb9af53830 2024-03-23T09:44:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f2942dc0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb9af562d0 con 0x55eb9af273b0 2024-03-23T09:44:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f2942dc0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb9af90fc0 con 0x55eb9af56b40 2024-03-23T09:44:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f293d7e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 0x55eb9af53c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f293d7e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 0x55eb9af53c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60642/0 (socket says 172.21.15.47:60642) 2024-03-23T09:44:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f293d7e5640 1 -- 172.21.15.47:0/2431958843 learned_addr learned my addr 172.21.15.47:0/2431958843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f293d7e5640 1 -- 172.21.15.47:0/2431958843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af56b40 msgr2=0x55eb9af54150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f293d7e5640 1 --2- 172.21.15.47:0/2431958843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af56b40 0x55eb9af54150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f293d7e5640 1 -- 172.21.15.47:0/2431958843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 msgr2=0x55eb9af29b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f293d7e5640 1 --2- 172.21.15.47:0/2431958843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 0x55eb9af29b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f293d7e5640 1 -- 172.21.15.47:0/2431958843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb9aebf9e0 con 0x55eb9af53830 2024-03-23T09:44:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f293d7e5640 1 --2- 172.21.15.47:0/2431958843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 0x55eb9af53c10 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2928003ec0 tx=0x7f2928025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1997ee1627652f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f291bfff640 1 -- 172.21.15.47:0/2431958843 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928009ea0 con 0x55eb9af53830 2024-03-23T09:44:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f291bfff640 1 -- 172.21.15.47:0/2431958843 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29280043b0 con 0x55eb9af53830 2024-03-23T09:44:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.924+0000 7f291bfff640 1 -- 172.21.15.47:0/2431958843 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928003b10 con 0x55eb9af53830 2024-03-23T09:44:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.925+0000 7f2942dc0740 1 -- 172.21.15.47:0/2431958843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 msgr2=0x55eb9af53c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.925+0000 7f2942dc0740 1 --2- 172.21.15.47:0/2431958843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 0x55eb9af53c10 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2928003ec0 tx=0x7f2928025000 comp rx=0 tx=0).stop 2024-03-23T09:44:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.925+0000 7f2942dc0740 1 -- 172.21.15.47:0/2431958843 shutdown_connections 2024-03-23T09:44:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.925+0000 7f2942dc0740 1 --2- 172.21.15.47:0/2431958843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af56b40 0x55eb9af54150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.925+0000 7f2942dc0740 1 --2- 172.21.15.47:0/2431958843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af53830 0x55eb9af53c10 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.925+0000 7f2942dc0740 1 --2- 172.21.15.47:0/2431958843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 0x55eb9af29b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.925+0000 7f2942dc0740 1 -- 172.21.15.47:0/2431958843 >> 172.21.15.47:0/2431958843 conn(0x55eb9af23850 msgr2=0x55eb9af3faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.925+0000 7f2942dc0740 1 -- 172.21.15.47:0/2431958843 shutdown_connections 2024-03-23T09:44:30.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.925+0000 7f2942dc0740 1 -- 172.21.15.47:0/2431958843 wait complete. 2024-03-23T09:44:30.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.925+0000 7f2942dc0740 1 -- start start 2024-03-23T09:44:30.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.926+0000 7f2942dc0740 1 Processor -- start 2024-03-23T09:44:30.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.926+0000 7f2942dc0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 0x55eb9af53c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.926+0000 7f2942dc0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af53830 0x55eb9af54150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.926+0000 7f2942dc0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af56b40 0x55eb9af4f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.926+0000 7f2942dc0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb9af50be0 con 0x55eb9af56b40 2024-03-23T09:44:30.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.926+0000 7f2942dc0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb9af4ec20 con 0x55eb9af273b0 2024-03-23T09:44:30.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.926+0000 7f2942dc0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb9af4ef00 con 0x55eb9af53830 2024-03-23T09:44:30.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.926+0000 7f2942dc0740 10 client.0 _ll_register_callbacks cb 0x7f29300052b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:44:30.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f293e7e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af56b40 0x55eb9af4f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f293e7e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af56b40 0x55eb9af4f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60648/0 (socket says 172.21.15.47:60648) 2024-03-23T09:44:30.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f293e7e7640 1 -- 172.21.15.47:0/1475114377 learned_addr learned my addr 172.21.15.47:0/1475114377 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:44:30.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f293d7e5640 1 --2- 172.21.15.47:0/1475114377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af53830 0x55eb9af54150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f293dfe6640 1 --2- 172.21.15.47:0/1475114377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 0x55eb9af53c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f293e7e7640 1 -- 172.21.15.47:0/1475114377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af53830 msgr2=0x55eb9af54150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f293e7e7640 1 --2- 172.21.15.47:0/1475114377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af53830 0x55eb9af54150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f293e7e7640 1 -- 172.21.15.47:0/1475114377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 msgr2=0x55eb9af53c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f293e7e7640 1 --2- 172.21.15.47:0/1475114377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 0x55eb9af53c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f293e7e7640 1 -- 172.21.15.47:0/1475114377 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb9ae8b4d0 con 0x55eb9af56b40 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f293e7e7640 1 --2- 172.21.15.47:0/1475114377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af56b40 0x55eb9af4f420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f293400a0d0 tx=0x7f2934079ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cad1203da40a6d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f2918ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f2918ff9640 1 -- 172.21.15.47:0/1475114377 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934081b20 con 0x55eb9af56b40 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f2942dc0740 1 -- 172.21.15.47:0/1475114377 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2928004030 con 0x55eb9af56b40 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f2918ff9640 1 -- 172.21.15.47:0/1475114377 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29340011c0 con 0x55eb9af56b40 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f2918ff9640 1 -- 172.21.15.47:0/1475114377 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2934075e40 con 0x55eb9af56b40 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f2942dc0740 10 client.6522 fetch_fsmap learned FSMap version 25 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f2942dc0740 1 -- 172.21.15.47:0/1475114377 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55eb9ae2d5c0 con 0x55eb9af56b40 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.927+0000 7f2918ff9640 1 -- 172.21.15.47:0/1475114377 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2934085020 con 0x55eb9af56b40 2024-03-23T09:44:30.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f2918ff9640 1 -- 172.21.15.47:0/1475114377 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2934081690 con 0x55eb9af56b40 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f2942dc0740 10 client.6522 fetch_fsmap finished waiting for FSMap version 25 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f2942dc0740 10 client.6522 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f2942dc0740 20 client.6522 populate_metadata read hostname 'smithi047' 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f2942dc0740 1 --2- 172.21.15.47:0/1475114377 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55eb9af4f960 0x55eb9afd3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f2942dc0740 4 client.6522 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f2942dc0740 1 -- 172.21.15.47:0/1475114377 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eb9ab804e0 con 0x55eb9af4f960 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f293dfe6640 1 --2- 172.21.15.47:0/1475114377 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55eb9af4f960 0x55eb9afd3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f2918ff9640 10 client.6522 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f293dfe6640 1 --2- 172.21.15.47:0/1475114377 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55eb9af4f960 0x55eb9afd3c30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a193b7154b64dac server_cookie=c5d100d3a1d9d309 in_seq=0 out_seq=0 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f2918ff9640 1 -- 172.21.15.47:0/1475114377 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55eb9ab804e0 con 0x55eb9af4f960 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f2918ff9640 10 client.6522 handle_command_reply: tid=0 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f2942dc0740 1 client.6522 shutdown 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f2942dc0740 1 -- 172.21.15.47:0/1475114377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af56b40 msgr2=0x55eb9af4f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.928+0000 7f2942dc0740 1 --2- 172.21.15.47:0/1475114377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af56b40 0x55eb9af4f420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f293400a0d0 tx=0x7f2934079ef0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f2942dc0740 1 -- 172.21.15.47:0/1475114377 shutdown_connections 2024-03-23T09:44:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f2942dc0740 1 --2- 172.21.15.47:0/1475114377 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55eb9af4f960 0x55eb9afd3c30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f2942dc0740 1 --2- 172.21.15.47:0/1475114377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb9af56b40 0x55eb9af4f420 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f2918ff9640 0 client.6522 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:44:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f2942dc0740 1 --2- 172.21.15.47:0/1475114377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb9af53830 0x55eb9af54150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f2942dc0740 1 --2- 172.21.15.47:0/1475114377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb9af273b0 0x55eb9af53c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f2942dc0740 1 -- 172.21.15.47:0/1475114377 >> 172.21.15.47:0/1475114377 conn(0x55eb9af23850 msgr2=0x55eb9af3faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f2942dc0740 1 -- 172.21.15.47:0/1475114377 shutdown_connections 2024-03-23T09:44:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f2942dc0740 1 -- 172.21.15.47:0/1475114377 wait complete. 2024-03-23T09:44:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f2942dc0740 20 client.6522 trim_cache size 0 max 16384 2024-03-23T09:44:30.936 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:44:30.937 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T09:44:30.937 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-23T09:44:30.937 INFO:teuthology.orchestra.run.smithi047.stdout: "de47f180-be3e-4a6b-9877-9fb63f82df18": { 2024-03-23T09:44:30.937 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-23T09:44:30.937 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "de47f180-be3e-4a6b-9877-9fb63f82df18", 2024-03-23T09:44:30.937 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,repair,force" 2024-03-23T09:44:30.937 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:44:30.937 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:44:30.937 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:44:30.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f294024a640 1 -- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f291c06ac30 msgr2=0x7f291c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f294024a640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f291c06ac30 0x7f291c06d0f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f29300035f0 tx=0x7f2930009040 comp rx=0 tx=0).stop 2024-03-23T09:44:30.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f294024a640 1 -- 172.21.15.47:0/1435370398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f293815f680 msgr2=0x7f2938152b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:44:30.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f294024a640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f293815f680 0x7f2938152b10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f293400a830 tx=0x7f2934001d80 comp rx=0 tx=0).stop 2024-03-23T09:44:30.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f294024a640 1 -- 172.21.15.47:0/1435370398 shutdown_connections 2024-03-23T09:44:30.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f294024a640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f291c06ac30 0x7f291c06d0f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f294024a640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f293815f680 0x7f2938152b10 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f294024a640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2938158db0 0x7f29380c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f294024a640 1 --2- 172.21.15.47:0/1435370398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f293815f2a0 0x7f29380c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:44:30.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f294024a640 1 -- 172.21.15.47:0/1435370398 >> 172.21.15.47:0/1435370398 conn(0x7f29380659d0 msgr2=0x7f2938157050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:44:30.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f294024a640 1 -- 172.21.15.47:0/1435370398 shutdown_connections 2024-03-23T09:44:30.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:44:30.929+0000 7f294024a640 1 -- 172.21.15.47:0/1435370398 wait complete. 2024-03-23T09:44:30.940 INFO:tasks.cephfs.filesystem:scrub status for tag:de47f180-be3e-4a6b-9877-9fb63f82df18 - {'path': '/', 'tag': 'de47f180-be3e-4a6b-9877-9fb63f82df18', 'options': 'recursive,repair,force'} 2024-03-23T09:44:37.440 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:44:37.443 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:45:00.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:00.942 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-23T09:45:01.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.058+0000 7f2d1e65c640 1 Processor -- start 2024-03-23T09:45:01.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.058+0000 7f2d1e65c640 1 -- start start 2024-03-23T09:45:01.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d1e65c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2d1815cbf0 0x7f2d180b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d1e65c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d180cd140 0x7f2d180cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d1e65c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d180cf7f0 0x7f2d180b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d1e65c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d180cfc40 con 0x7f2d180cd140 2024-03-23T09:45:01.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d1e65c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d18161570 con 0x7f2d1815cbf0 2024-03-23T09:45:01.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d1e65c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d18160960 con 0x7f2d180cf7f0 2024-03-23T09:45:01.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d177fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d180cd140 0x7f2d180cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d177fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d180cd140 0x7f2d180cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58236/0 (socket says 172.21.15.47:58236) 2024-03-23T09:45:01.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d177fe640 1 -- 172.21.15.47:0/3911167252 learned_addr learned my addr 172.21.15.47:0/3911167252 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d17fff640 1 --2- 172.21.15.47:0/3911167252 >> v2:172.21.15.175:3300/0 conn(0x7f2d1815cbf0 0x7f2d180b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d1cbf9640 1 --2- 172.21.15.47:0/3911167252 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d180cf7f0 0x7f2d180b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d177fe640 1 -- 172.21.15.47:0/3911167252 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d180cf7f0 msgr2=0x7f2d180b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d177fe640 1 --2- 172.21.15.47:0/3911167252 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d180cf7f0 0x7f2d180b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d177fe640 1 -- 172.21.15.47:0/3911167252 >> v2:172.21.15.175:3300/0 conn(0x7f2d1815cbf0 msgr2=0x7f2d180b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d177fe640 1 --2- 172.21.15.47:0/3911167252 >> v2:172.21.15.175:3300/0 conn(0x7f2d1815cbf0 0x7f2d180b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d177fe640 1 -- 172.21.15.47:0/3911167252 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d18160c40 con 0x7f2d180cd140 2024-03-23T09:45:01.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.059+0000 7f2d17fff640 1 --2- 172.21.15.47:0/3911167252 >> v2:172.21.15.175:3300/0 conn(0x7f2d1815cbf0 0x7f2d180b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:01.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.060+0000 7f2d177fe640 1 --2- 172.21.15.47:0/3911167252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d180cd140 0x7f2d180cf2b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2d180b5400 tx=0x7f2d0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03af733dd8e771d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.060+0000 7f2d16ffd640 1 -- 172.21.15.47:0/3911167252 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08031870 con 0x7f2d180cd140 2024-03-23T09:45:01.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.060+0000 7f2d16ffd640 1 -- 172.21.15.47:0/3911167252 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d080319d0 con 0x7f2d180cd140 2024-03-23T09:45:01.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.060+0000 7f2d16ffd640 1 -- 172.21.15.47:0/3911167252 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08031d60 con 0x7f2d180cd140 2024-03-23T09:45:01.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.061+0000 7f2d1e65c640 1 -- 172.21.15.47:0/3911167252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d180cd140 msgr2=0x7f2d180cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.061+0000 7f2d1e65c640 1 --2- 172.21.15.47:0/3911167252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d180cd140 0x7f2d180cf2b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2d180b5400 tx=0x7f2d0802b910 comp rx=0 tx=0).stop 2024-03-23T09:45:01.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.061+0000 7f2d1e65c640 1 -- 172.21.15.47:0/3911167252 shutdown_connections 2024-03-23T09:45:01.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.061+0000 7f2d1e65c640 1 --2- 172.21.15.47:0/3911167252 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d180cf7f0 0x7f2d180b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.061+0000 7f2d1e65c640 1 --2- 172.21.15.47:0/3911167252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d180cd140 0x7f2d180cf2b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.061+0000 7f2d1e65c640 1 --2- 172.21.15.47:0/3911167252 >> v2:172.21.15.175:3300/0 conn(0x7f2d1815cbf0 0x7f2d180b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.061+0000 7f2d1e65c640 1 -- 172.21.15.47:0/3911167252 >> 172.21.15.47:0/3911167252 conn(0x7f2d1806b160 msgr2=0x7f2d1806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.061+0000 7f2d1e65c640 1 -- 172.21.15.47:0/3911167252 shutdown_connections 2024-03-23T09:45:01.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.061+0000 7f2d1e65c640 1 -- 172.21.15.47:0/3911167252 wait complete. 2024-03-23T09:45:01.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.062+0000 7f2d1e65c640 1 Processor -- start 2024-03-23T09:45:01.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.062+0000 7f2d1e65c640 1 -- start start 2024-03-23T09:45:01.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.063+0000 7f2d1e65c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d180cd140 0x7f2d180cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.063+0000 7f2d1e65c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d180cf7f0 0x7f2d180c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.063+0000 7f2d1e65c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d1815cbf0 0x7f2d180ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.063+0000 7f2d1e65c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d180ca5f0 con 0x7f2d180cf7f0 2024-03-23T09:45:01.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.063+0000 7f2d1e65c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d180ca8c0 con 0x7f2d1815cbf0 2024-03-23T09:45:01.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.063+0000 7f2d1e65c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d180caba0 con 0x7f2d180cd140 2024-03-23T09:45:01.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.063+0000 7f2d177fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d180cf7f0 0x7f2d180c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.063+0000 7f2d177fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d180cf7f0 0x7f2d180c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58250/0 (socket says 172.21.15.47:58250) 2024-03-23T09:45:01.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.063+0000 7f2d177fe640 1 -- 172.21.15.47:0/3304674228 learned_addr learned my addr 172.21.15.47:0/3304674228 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d17fff640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d180cd140 0x7f2d180cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d1cbf9640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d1815cbf0 0x7f2d180ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d177fe640 1 -- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d180cd140 msgr2=0x7f2d180cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d177fe640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d180cd140 0x7f2d180cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d177fe640 1 -- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d1815cbf0 msgr2=0x7f2d180ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d177fe640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d1815cbf0 0x7f2d180ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d177fe640 1 -- 172.21.15.47:0/3304674228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080094d0 con 0x7f2d180cf7f0 2024-03-23T09:45:01.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d17fff640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d180cd140 0x7f2d180cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:01.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d1cbf9640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d1815cbf0 0x7f2d180ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:01.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d177fe640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d180cf7f0 0x7f2d180c9b70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2d0802b950 tx=0x7f2d080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d14ff9640 1 -- 172.21.15.47:0/3304674228 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0803ad30 con 0x7f2d180cf7f0 2024-03-23T09:45:01.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d14ff9640 1 -- 172.21.15.47:0/3304674228 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d080443f0 con 0x7f2d180cf7f0 2024-03-23T09:45:01.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d1e65c640 1 -- 172.21.15.47:0/3304674228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d180cae80 con 0x7f2d180cf7f0 2024-03-23T09:45:01.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d14ff9640 1 -- 172.21.15.47:0/3304674228 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d080447a0 con 0x7f2d180cf7f0 2024-03-23T09:45:01.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.064+0000 7f2d1e65c640 1 -- 172.21.15.47:0/3304674228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d180cb440 con 0x7f2d180cf7f0 2024-03-23T09:45:01.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.065+0000 7f2d14ff9640 1 -- 172.21.15.47:0/3304674228 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2d0804d450 con 0x7f2d180cf7f0 2024-03-23T09:45:01.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.066+0000 7f2d14ff9640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d0406ac00 0x7f2d0406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.066+0000 7f2d14ff9640 1 -- 172.21.15.47:0/3304674228 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d0802d040 con 0x7f2d180cf7f0 2024-03-23T09:45:01.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.066+0000 7f2d17fff640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d0406ac00 0x7f2d0406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.067+0000 7f2d17fff640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d0406ac00 0x7f2d0406d0c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c006fd0 tx=0x7f2d0c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.067+0000 7f2d211d5740 1 Processor -- start 2024-03-23T09:45:01.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.067+0000 7f2d211d5740 1 -- start start 2024-03-23T09:45:01.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d211d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e64830 0x55c578e64c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d211d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e34850 0x55c578e61aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d211d5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 0x55c578e61fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d211d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c578da4dd0 con 0x55c578e64830 2024-03-23T09:45:01.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d211d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c578da2560 con 0x55c578e34850 2024-03-23T09:45:01.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d211d5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c578d9de50 con 0x55c578e383b0 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d17fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e64830 0x55c578e64c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d17fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e64830 0x55c578e64c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58254/0 (socket says 172.21.15.47:58254) 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d17fff640 1 -- 172.21.15.47:0/3205635195 learned_addr learned my addr 172.21.15.47:0/3205635195 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d177fe640 1 --2- 172.21.15.47:0/3205635195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e34850 0x55c578e61aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d1cbf9640 1 --2- 172.21.15.47:0/3205635195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 0x55c578e61fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d17fff640 1 -- 172.21.15.47:0/3205635195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 msgr2=0x55c578e61fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d17fff640 1 --2- 172.21.15.47:0/3205635195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 0x55c578e61fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d17fff640 1 -- 172.21.15.47:0/3205635195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e34850 msgr2=0x55c578e61aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d17fff640 1 --2- 172.21.15.47:0/3205635195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e34850 0x55c578e61aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d17fff640 1 -- 172.21.15.47:0/3205635195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c578d65470 con 0x55c578e64830 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d177fe640 1 --2- 172.21.15.47:0/3205635195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e34850 0x55c578e61aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2d17fff640 1 --2- 172.21.15.47:0/3205635195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e64830 0x55c578e64c10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00fa30 tx=0x7f2d0c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf85339885eb06df server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2cfdffb640 1 -- 172.21.15.47:0/3205635195 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c014be0 con 0x55c578e64830 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2cfdffb640 1 -- 172.21.15.47:0/3205635195 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0c019070 con 0x55c578e64830 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.068+0000 7f2cfdffb640 1 -- 172.21.15.47:0/3205635195 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c020440 con 0x55c578e64830 2024-03-23T09:45:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.069+0000 7f2d211d5740 1 -- 172.21.15.47:0/3205635195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e64830 msgr2=0x55c578e64c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.069+0000 7f2d211d5740 1 --2- 172.21.15.47:0/3205635195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e64830 0x55c578e64c10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00fa30 tx=0x7f2d0c012000 comp rx=0 tx=0).stop 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.069+0000 7f2d211d5740 1 -- 172.21.15.47:0/3205635195 shutdown_connections 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.069+0000 7f2d211d5740 1 --2- 172.21.15.47:0/3205635195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 0x55c578e61fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.069+0000 7f2d211d5740 1 --2- 172.21.15.47:0/3205635195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e34850 0x55c578e61aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.069+0000 7f2d211d5740 1 --2- 172.21.15.47:0/3205635195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e64830 0x55c578e64c10 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.069+0000 7f2d211d5740 1 -- 172.21.15.47:0/3205635195 >> 172.21.15.47:0/3205635195 conn(0x55c578e36ac0 msgr2=0x55c578e7f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.069+0000 7f2d211d5740 1 -- 172.21.15.47:0/3205635195 shutdown_connections 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.069+0000 7f2d211d5740 1 -- 172.21.15.47:0/3205635195 wait complete. 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.069+0000 7f2d211d5740 1 -- start start 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.070+0000 7f2d211d5740 1 Processor -- start 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.070+0000 7f2d211d5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 0x55c578e664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.070+0000 7f2d211d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 0x55c578e590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.070+0000 7f2d211d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 0x55c578e5ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.070+0000 7f2d211d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c578d5c280 con 0x55c578e59d50 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.070+0000 7f2d211d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c578d5bf20 con 0x55c578e64830 2024-03-23T09:45:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.070+0000 7f2d211d5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c578d213c0 con 0x55c578e383b0 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d211d5740 10 client.0 _ll_register_callbacks cb 0x55c578d73380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d1cbf9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 0x55c578e5ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d1cbf9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 0x55c578e5ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58268/0 (socket says 172.21.15.47:58268) 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d17fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 0x55c578e664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d1cbf9640 1 -- 172.21.15.47:0/2771218778 learned_addr learned my addr 172.21.15.47:0/2771218778 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d177fe640 1 --2- 172.21.15.47:0/2771218778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 0x55c578e590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d1cbf9640 1 -- 172.21.15.47:0/2771218778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 msgr2=0x55c578e664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d1cbf9640 1 --2- 172.21.15.47:0/2771218778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 0x55c578e664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d1cbf9640 1 -- 172.21.15.47:0/2771218778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 msgr2=0x55c578e590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d1cbf9640 1 --2- 172.21.15.47:0/2771218778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 0x55c578e590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d1cbf9640 1 -- 172.21.15.47:0/2771218778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c00e040 con 0x55c578e59d50 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d1cbf9640 1 --2- 172.21.15.47:0/2771218778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 0x55c578e5ba60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2d10007a00 tx=0x7f2d10007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129a6a476505a3e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2cf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2cf6ffd640 1 -- 172.21.15.47:0/2771218778 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10011a20 con 0x55c578e59d50 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2cf6ffd640 1 -- 172.21.15.47:0/2771218778 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d10011b80 con 0x55c578e59d50 2024-03-23T09:45:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d211d5740 1 -- 172.21.15.47:0/2771218778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c578dc7690 con 0x55c578e59d50 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2cf6ffd640 1 -- 172.21.15.47:0/2771218778 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d10011d40 con 0x55c578e59d50 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d211d5740 10 client.6534 fetch_fsmap learned FSMap version 26 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.071+0000 7f2d211d5740 1 -- 172.21.15.47:0/2771218778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55c578d9c4d0 con 0x55c578e59d50 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.072+0000 7f2cf6ffd640 1 -- 172.21.15.47:0/2771218778 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c578dc7690 con 0x55c578e59d50 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.072+0000 7f2cf6ffd640 1 -- 172.21.15.47:0/2771218778 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2d10014070 con 0x55c578e59d50 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.072+0000 7f2d211d5740 10 client.6534 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.072+0000 7f2d211d5740 10 client.6534 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.072+0000 7f2d211d5740 20 client.6534 populate_metadata read hostname 'smithi047' 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.072+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2771218778 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55c578e81ba0 0x55c578e81f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.072+0000 7f2d211d5740 4 client.6534 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.072+0000 7f2d211d5740 1 -- 172.21.15.47:0/2771218778 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c578d8feb0 con 0x55c578e81ba0 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.072+0000 7f2d17fff640 1 --2- 172.21.15.47:0/2771218778 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55c578e81ba0 0x55c578e81f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.072+0000 7f2cf6ffd640 10 client.6534 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.072+0000 7f2d17fff640 1 --2- 172.21.15.47:0/2771218778 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55c578e81ba0 0x55c578e81f80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e76394c6d806f2a server_cookie=faf74a0f96656c31 in_seq=0 out_seq=0 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2cf6ffd640 1 -- 172.21.15.47:0/2771218778 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c578d8feb0 con 0x55c578e81ba0 2024-03-23T09:45:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2cf6ffd640 10 client.6534 handle_command_reply: tid=0 2024-03-23T09:45:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2d211d5740 1 client.6534 shutdown 2024-03-23T09:45:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2d211d5740 1 -- 172.21.15.47:0/2771218778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 msgr2=0x55c578e5ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2771218778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 0x55c578e5ba60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2d10007a00 tx=0x7f2d10007ec0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2d211d5740 1 -- 172.21.15.47:0/2771218778 shutdown_connections 2024-03-23T09:45:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2771218778 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55c578e81ba0 0x55c578e81f80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2771218778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 0x55c578e5ba60 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2771218778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 0x55c578e590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2771218778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 0x55c578e664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2d211d5740 1 -- 172.21.15.47:0/2771218778 >> 172.21.15.47:0/2771218778 conn(0x55c578e34850 msgr2=0x55c578e38c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2cf6ffd640 0 client.6534 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:45:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2d211d5740 1 -- 172.21.15.47:0/2771218778 shutdown_connections 2024-03-23T09:45:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.073+0000 7f2d211d5740 1 -- 172.21.15.47:0/2771218778 wait complete. 2024-03-23T09:45:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.074+0000 7f2d211d5740 20 client.6534 trim_cache size 0 max 16384 2024-03-23T09:45:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.088+0000 7f2d211d5740 1 Processor -- start 2024-03-23T09:45:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.088+0000 7f2d211d5740 1 -- start start 2024-03-23T09:45:01.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d211d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e383b0 0x55c578e664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d211d5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e59d50 0x55c578e64c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d211d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 0x55c578e5ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d211d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c578e3ac80 con 0x55c578e383b0 2024-03-23T09:45:01.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d211d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c578ea1920 con 0x55c578e64830 2024-03-23T09:45:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d211d5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c578e5bc00 con 0x55c578e59d50 2024-03-23T09:45:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d17fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e383b0 0x55c578e664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d17fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e383b0 0x55c578e664d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58282/0 (socket says 172.21.15.47:58282) 2024-03-23T09:45:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d17fff640 1 -- 172.21.15.47:0/2865557097 learned_addr learned my addr 172.21.15.47:0/2865557097 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d177fe640 1 --2- 172.21.15.47:0/2865557097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e59d50 0x55c578e64c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d1cbf9640 1 --2- 172.21.15.47:0/2865557097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 0x55c578e5ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d17fff640 1 -- 172.21.15.47:0/2865557097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e59d50 msgr2=0x55c578e64c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d17fff640 1 --2- 172.21.15.47:0/2865557097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e59d50 0x55c578e64c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d17fff640 1 -- 172.21.15.47:0/2865557097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 msgr2=0x55c578e5ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d17fff640 1 --2- 172.21.15.47:0/2865557097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 0x55c578e5ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d17fff640 1 -- 172.21.15.47:0/2865557097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c578dd09e0 con 0x55c578e383b0 2024-03-23T09:45:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2d17fff640 1 --2- 172.21.15.47:0/2865557097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e383b0 0x55c578e664d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c010010 tx=0x7f2d0c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33d29955bfbc11 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2cfe7fc640 1 -- 172.21.15.47:0/2865557097 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c030040 con 0x55c578e383b0 2024-03-23T09:45:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2cfe7fc640 1 -- 172.21.15.47:0/2865557097 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0c036420 con 0x55c578e383b0 2024-03-23T09:45:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.089+0000 7f2cfe7fc640 1 -- 172.21.15.47:0/2865557097 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c014ea0 con 0x55c578e383b0 2024-03-23T09:45:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.090+0000 7f2d211d5740 1 -- 172.21.15.47:0/2865557097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e383b0 msgr2=0x55c578e664d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.090+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2865557097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e383b0 0x55c578e664d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c010010 tx=0x7f2d0c014000 comp rx=0 tx=0).stop 2024-03-23T09:45:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.090+0000 7f2d211d5740 1 -- 172.21.15.47:0/2865557097 shutdown_connections 2024-03-23T09:45:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.090+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2865557097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 0x55c578e5ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.090+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2865557097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e59d50 0x55c578e64c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.090+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2865557097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e383b0 0x55c578e664d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.090+0000 7f2d211d5740 1 -- 172.21.15.47:0/2865557097 >> 172.21.15.47:0/2865557097 conn(0x55c578e34850 msgr2=0x55c578e61680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.090+0000 7f2d211d5740 1 -- 172.21.15.47:0/2865557097 shutdown_connections 2024-03-23T09:45:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.090+0000 7f2d211d5740 1 -- 172.21.15.47:0/2865557097 wait complete. 2024-03-23T09:45:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.091+0000 7f2d211d5740 1 -- start start 2024-03-23T09:45:01.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.091+0000 7f2d211d5740 1 Processor -- start 2024-03-23T09:45:01.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d211d5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 0x55c578ecd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d211d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 0x55c578ee8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d211d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 0x55c578e98700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d211d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c578ecdb80 con 0x55c578e59d50 2024-03-23T09:45:01.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d211d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c578ee8520 con 0x55c578e64830 2024-03-23T09:45:01.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d211d5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c578ee9470 con 0x55c578e383b0 2024-03-23T09:45:01.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d211d5740 10 client.0 _ll_register_callbacks cb 0x7f2d0c009100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d177fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 0x55c578ee8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d177fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 0x55c578ee8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58288/0 (socket says 172.21.15.47:58288) 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d177fe640 1 -- 172.21.15.47:0/2652766244 learned_addr learned my addr 172.21.15.47:0/2652766244 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d17fff640 1 --2- 172.21.15.47:0/2652766244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 0x55c578ecd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d1cbf9640 1 --2- 172.21.15.47:0/2652766244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 0x55c578e98700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d177fe640 1 -- 172.21.15.47:0/2652766244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 msgr2=0x55c578ecd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d177fe640 1 --2- 172.21.15.47:0/2652766244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 0x55c578ecd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d177fe640 1 -- 172.21.15.47:0/2652766244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 msgr2=0x55c578e98700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d177fe640 1 --2- 172.21.15.47:0/2652766244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 0x55c578e98700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d177fe640 1 -- 172.21.15.47:0/2652766244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c578d65470 con 0x55c578e59d50 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d177fe640 1 --2- 172.21.15.47:0/2652766244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 0x55c578ee8f30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2d08006da0 tx=0x7f2d0807afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccaf27b6cc2190b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2cf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2cf6ffd640 1 -- 172.21.15.47:0/2652766244 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08082be0 con 0x55c578e59d50 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2cf6ffd640 1 -- 172.21.15.47:0/2652766244 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d08079ce0 con 0x55c578e59d50 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.092+0000 7f2d211d5740 1 -- 172.21.15.47:0/2652766244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d0c019070 con 0x55c578e59d50 2024-03-23T09:45:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2cf6ffd640 1 -- 172.21.15.47:0/2652766244 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d080991f0 con 0x55c578e59d50 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2d211d5740 10 client.6540 fetch_fsmap learned FSMap version 26 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2d211d5740 1 -- 172.21.15.47:0/2652766244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55c578d3e5c0 con 0x55c578e59d50 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2cf6ffd640 1 -- 172.21.15.47:0/2652766244 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0c019070 con 0x55c578e59d50 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2cf6ffd640 1 -- 172.21.15.47:0/2652766244 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2d08082750 con 0x55c578e59d50 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2d211d5740 10 client.6540 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2d211d5740 10 client.6540 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2d211d5740 20 client.6540 populate_metadata read hostname 'smithi047' 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2652766244 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55c578ece240 0x55c578e98c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2d211d5740 4 client.6540 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2d211d5740 1 -- 172.21.15.47:0/2652766244 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c578a914e0 con 0x55c578ece240 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2d17fff640 1 --2- 172.21.15.47:0/2652766244 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55c578ece240 0x55c578e98c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2d17fff640 1 --2- 172.21.15.47:0/2652766244 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55c578ece240 0x55c578e98c40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=948ad00578955600 server_cookie=e4645635dd22085f in_seq=0 out_seq=0 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.093+0000 7f2cf6ffd640 10 client.6540 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2cf6ffd640 1 -- 172.21.15.47:0/2652766244 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c578a914e0 con 0x55c578ece240 2024-03-23T09:45:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2cf6ffd640 10 client.6540 handle_command_reply: tid=0 2024-03-23T09:45:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d211d5740 1 client.6540 shutdown 2024-03-23T09:45:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d211d5740 1 -- 172.21.15.47:0/2652766244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 msgr2=0x55c578ee8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2652766244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 0x55c578ee8f30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2d08006da0 tx=0x7f2d0807afb0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d211d5740 1 -- 172.21.15.47:0/2652766244 shutdown_connections 2024-03-23T09:45:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2652766244 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x55c578ece240 0x55c578e98c40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2652766244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c578e64830 0x55c578e98700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2652766244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c578e59d50 0x55c578ee8f30 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2cf6ffd640 0 client.6540 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:45:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d211d5740 1 --2- 172.21.15.47:0/2652766244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c578e383b0 0x55c578ecd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d211d5740 1 -- 172.21.15.47:0/2652766244 >> 172.21.15.47:0/2652766244 conn(0x55c578e34850 msgr2=0x55c578e61470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d211d5740 1 -- 172.21.15.47:0/2652766244 shutdown_connections 2024-03-23T09:45:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d211d5740 1 -- 172.21.15.47:0/2652766244 wait complete. 2024-03-23T09:45:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d211d5740 20 client.6540 trim_cache size 0 max 16384 2024-03-23T09:45:01.100 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:45:01.100 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-23T09:45:01.100 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-23T09:45:01.100 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:45:01.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d1e65c640 1 -- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d0406ac00 msgr2=0x7f2d0406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d1e65c640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d0406ac00 0x7f2d0406d0c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c006fd0 tx=0x7f2d0c008040 comp rx=0 tx=0).stop 2024-03-23T09:45:01.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d1e65c640 1 -- 172.21.15.47:0/3304674228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d180cf7f0 msgr2=0x7f2d180c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d1e65c640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d180cf7f0 0x7f2d180c9b70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2d0802b950 tx=0x7f2d080046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d1e65c640 1 -- 172.21.15.47:0/3304674228 shutdown_connections 2024-03-23T09:45:01.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.094+0000 7f2d1e65c640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d0406ac00 0x7f2d0406d0c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.095+0000 7f2d1e65c640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d1815cbf0 0x7f2d180ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.095+0000 7f2d1e65c640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d180cf7f0 0x7f2d180c9b70 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.095+0000 7f2d1e65c640 1 --2- 172.21.15.47:0/3304674228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d180cd140 0x7f2d180cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.095+0000 7f2d1e65c640 1 -- 172.21.15.47:0/3304674228 >> 172.21.15.47:0/3304674228 conn(0x7f2d1806b160 msgr2=0x7f2d180d1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.095+0000 7f2d1e65c640 1 -- 172.21.15.47:0/3304674228 shutdown_connections 2024-03-23T09:45:01.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.095+0000 7f2d1e65c640 1 -- 172.21.15.47:0/3304674228 wait complete. 2024-03-23T09:45:01.105 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T09:45:01.105 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:45:01.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.206+0000 7fe050f7d640 1 Processor -- start 2024-03-23T09:45:01.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.206+0000 7fe050f7d640 1 -- start start 2024-03-23T09:45:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe050f7d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe04c0cf210 0x7fe04c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe050f7d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe04c0b5a90 0x7fe04c0b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe050f7d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04c0cfbd0 con 0x7fe04c15cd00 2024-03-23T09:45:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe050f7d640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04c15d0e0 con 0x7fe04c0b5a90 2024-03-23T09:45:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe050f7d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0d1540 con 0x7fe04c0cf210 2024-03-23T09:45:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe04a59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe04c15cd00 0x7fe04c0d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40348/0 (socket says 172.21.15.47:40348) 2024-03-23T09:45:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe04a59c640 1 -- 172.21.15.47:0/1995181374 learned_addr learned my addr 172.21.15.47:0/1995181374 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe04ad9d640 1 --2- 172.21.15.47:0/1995181374 >> v2:172.21.15.175:3300/0 conn(0x7fe04c0b5a90 0x7fe04c0b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe043fff640 1 --2- 172.21.15.47:0/1995181374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe04c0cf210 0x7fe04c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464348954 0 0) 0x7fe04c0cfbd0 con 0x7fe04c15cd00 2024-03-23T09:45:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe038003520 con 0x7fe04c15cd00 2024-03-23T09:45:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2254239239 0 0) 0x7fe038003520 con 0x7fe04c15cd00 2024-03-23T09:45:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe04c0cf210 msgr2=0x7fe04c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 --2- 172.21.15.47:0/1995181374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe04c0cf210 0x7fe04c0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 >> v2:172.21.15.175:3300/0 conn(0x7fe04c0b5a90 msgr2=0x7fe04c0b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 --2- 172.21.15.47:0/1995181374 >> v2:172.21.15.175:3300/0 conn(0x7fe04c0b5a90 0x7fe04c0b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c0d1820 con 0x7fe04c15cd00 2024-03-23T09:45:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fe03c001670 con 0x7fe04c15cd00 2024-03-23T09:45:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 >> v1:172.21.15.47:6789/0 conn(0x7fe04c15cd00 legacy=0x7fe04c0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 --2- 172.21.15.47:0/1995181374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe04c0cf210 0x7fe038003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 --2- 172.21.15.47:0/1995181374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe038003ef0 0x7fe0380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 --2- 172.21.15.47:0/1995181374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe038004d00 0x7fe038005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c15d0e0 con 0x7fe038003ef0 2024-03-23T09:45:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0d1540 con 0x7fe038004d00 2024-03-23T09:45:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.207+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03800b9d0 con 0x7fe04c0cf210 2024-03-23T09:45:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.208+0000 7fe043fff640 1 --2- 172.21.15.47:0/1995181374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe038003ef0 0x7fe0380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.208+0000 7fe04ad9d640 1 --2- 172.21.15.47:0/1995181374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe038004d00 0x7fe038005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.208+0000 7fe04a59c640 1 --2- 172.21.15.47:0/1995181374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe04c0cf210 0x7fe038003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.208+0000 7fe043fff640 1 -- 172.21.15.47:0/1995181374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe04c0cf210 msgr2=0x7fe038003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.208+0000 7fe043fff640 1 --2- 172.21.15.47:0/1995181374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe04c0cf210 0x7fe038003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.208+0000 7fe043fff640 1 -- 172.21.15.47:0/1995181374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe038004d00 msgr2=0x7fe038005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.208+0000 7fe043fff640 1 --2- 172.21.15.47:0/1995181374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe038004d00 0x7fe038005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.208+0000 7fe043fff640 1 -- 172.21.15.47:0/1995181374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe038003520 con 0x7fe038003ef0 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.209+0000 7fe043fff640 1 --2- 172.21.15.47:0/1995181374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe038003ef0 0x7fe0380047c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe034009af0 tx=0x7fe0380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e898609ab57ebce1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.209+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03402c6b0 con 0x7fe038003ef0 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.209+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03402c810 con 0x7fe038003ef0 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.209+0000 7fe049d9b640 1 -- 172.21.15.47:0/1995181374 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03402cba0 con 0x7fe038003ef0 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.209+0000 7fe050f7d640 1 -- 172.21.15.47:0/1995181374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe038003ef0 msgr2=0x7fe0380047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.209+0000 7fe050f7d640 1 --2- 172.21.15.47:0/1995181374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe038003ef0 0x7fe0380047c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe034009af0 tx=0x7fe0380057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.209+0000 7fe04ad9d640 1 -- 172.21.15.47:0/1995181374 reap_dead start 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.209+0000 7fe050f7d640 1 -- 172.21.15.47:0/1995181374 shutdown_connections 2024-03-23T09:45:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.209+0000 7fe050f7d640 1 -- 172.21.15.47:0/1995181374 >> 172.21.15.47:0/1995181374 conn(0x7fe04c06b160 msgr2=0x7fe04c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.209+0000 7fe050f7d640 1 -- 172.21.15.47:0/1995181374 shutdown_connections 2024-03-23T09:45:01.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.209+0000 7fe050f7d640 1 -- 172.21.15.47:0/1995181374 wait complete. 2024-03-23T09:45:01.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.211+0000 7fe050f7d640 1 Processor -- start 2024-03-23T09:45:01.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.211+0000 7fe050f7d640 1 -- start start 2024-03-23T09:45:01.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.211+0000 7fe050f7d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe038003ef0 0x7fe04c145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.211+0000 7fe050f7d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe04c0b5040 0x7fe04c0c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.211+0000 7fe050f7d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe04c0b2380 0x7fe04c1f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.211+0000 7fe050f7d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0b3fa0 con 0x7fe04c0b5040 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.211+0000 7fe050f7d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0b4280 con 0x7fe04c0b2380 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.211+0000 7fe050f7d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0b4560 con 0x7fe038003ef0 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.211+0000 7fe043fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe04c0b5040 0x7fe04c0c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.211+0000 7fe043fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe04c0b5040 0x7fe04c0c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58310/0 (socket says 172.21.15.47:58310) 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.211+0000 7fe043fff640 1 -- 172.21.15.47:0/773791143 learned_addr learned my addr 172.21.15.47:0/773791143 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.212+0000 7fe04a59c640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe038003ef0 0x7fe04c145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.212+0000 7fe04ad9d640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe04c0b2380 0x7fe04c1f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.212+0000 7fe043fff640 1 -- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe038003ef0 msgr2=0x7fe04c145450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.212+0000 7fe043fff640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe038003ef0 0x7fe04c145450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.212+0000 7fe043fff640 1 -- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe04c0b2380 msgr2=0x7fe04c1f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.212+0000 7fe043fff640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe04c0b2380 0x7fe04c1f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.212+0000 7fe043fff640 1 -- 172.21.15.47:0/773791143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c0b4840 con 0x7fe04c0b5040 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.212+0000 7fe043fff640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe04c0b5040 0x7fe04c0c88d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe034037040 tx=0x7fe034004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.212+0000 7fe042ffd640 1 -- 172.21.15.47:0/773791143 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034003810 con 0x7fe04c0b5040 2024-03-23T09:45:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.212+0000 7fe042ffd640 1 -- 172.21.15.47:0/773791143 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe034003970 con 0x7fe04c0b5040 2024-03-23T09:45:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.212+0000 7fe050f7d640 1 -- 172.21.15.47:0/773791143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04c0b4b20 con 0x7fe04c0b5040 2024-03-23T09:45:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.212+0000 7fe042ffd640 1 -- 172.21.15.47:0/773791143 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034003ce0 con 0x7fe04c0b5040 2024-03-23T09:45:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.212+0000 7fe050f7d640 1 -- 172.21.15.47:0/773791143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04c0b2830 con 0x7fe04c0b5040 2024-03-23T09:45:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.213+0000 7fe050f7d640 1 -- 172.21.15.47:0/773791143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04c072360 con 0x7fe04c0b5040 2024-03-23T09:45:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.213+0000 7fe042ffd640 1 -- 172.21.15.47:0/773791143 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe034047450 con 0x7fe04c0b5040 2024-03-23T09:45:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.214+0000 7fe042ffd640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe02c06ac30 0x7fe02c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.214+0000 7fe042ffd640 1 -- 172.21.15.47:0/773791143 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0340ab530 con 0x7fe04c0b5040 2024-03-23T09:45:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.214+0000 7fe04a59c640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe02c06ac30 0x7fe02c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.215+0000 7fe04a59c640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe02c06ac30 0x7fe02c06d0f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0015d0 tx=0x7fe03c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.218+0000 7fe042ffd640 1 -- 172.21.15.47:0/773791143 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe0340767c0 con 0x7fe04c0b5040 2024-03-23T09:45:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.438+0000 7fe050f7d640 1 -- 172.21.15.47:0/773791143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe04c0b4cb0 con 0x7fe04c0b5040 2024-03-23T09:45:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.439+0000 7fe042ffd640 1 -- 172.21.15.47:0/773791143 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fe0340044e0 con 0x7fe04c0b5040 2024-03-23T09:45:01.439 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:01.439 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/375513546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":375513546},{"type":"v1","addr":"172.21.15.175:6837","nonce":375513546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/3650756747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3650756747},{"type":"v1","addr":"172.21.15.47:6833","nonce":3650756747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2893805543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2893805543},{"type":"v1","addr":"172.21.15.47:6835","nonce":2893805543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:43:47.389279+0000","modified":"2024-03-23T09:44:35.825401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.175:6836/2498456351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2498456351},{"type":"v1","addr":"172.21.15.175:6836","nonce":2498456351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-23T09:45:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2024-03-23T09:45:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.441+0000 7fe050f7d640 1 -- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe02c06ac30 msgr2=0x7fe02c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.441+0000 7fe050f7d640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe02c06ac30 0x7fe02c06d0f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0015d0 tx=0x7fe03c009040 comp rx=0 tx=0).stop 2024-03-23T09:45:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.441+0000 7fe050f7d640 1 -- 172.21.15.47:0/773791143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe04c0b5040 msgr2=0x7fe04c0c88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.441+0000 7fe050f7d640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe04c0b5040 0x7fe04c0c88d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe034037040 tx=0x7fe034004730 comp rx=0 tx=0).stop 2024-03-23T09:45:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.441+0000 7fe050f7d640 1 -- 172.21.15.47:0/773791143 shutdown_connections 2024-03-23T09:45:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.441+0000 7fe050f7d640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe02c06ac30 0x7fe02c06d0f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.441+0000 7fe050f7d640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe04c0b2380 0x7fe04c1f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.441+0000 7fe050f7d640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe04c0b5040 0x7fe04c0c88d0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.441+0000 7fe050f7d640 1 --2- 172.21.15.47:0/773791143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe038003ef0 0x7fe04c145450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.441+0000 7fe050f7d640 1 -- 172.21.15.47:0/773791143 >> 172.21.15.47:0/773791143 conn(0x7fe04c06b160 msgr2=0x7fe04c0d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.441+0000 7fe050f7d640 1 -- 172.21.15.47:0/773791143 shutdown_connections 2024-03-23T09:45:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.441+0000 7fe050f7d640 1 -- 172.21.15.47:0/773791143 wait complete. 2024-03-23T09:45:01.452 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-03-23T09:45:01.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.555+0000 7f49959a4640 1 Processor -- start 2024-03-23T09:45:01.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.555+0000 7f49959a4640 1 -- start start 2024-03-23T09:45:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.555+0000 7f49959a4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f499015cd40 0x7f49900d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.555+0000 7f49959a4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f499006b090 0x7f49900ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.555+0000 7f49959a4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49900cedd0 con 0x7f4990098a00 2024-03-23T09:45:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.556+0000 7f49959a4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499015d120 con 0x7f499015cd40 2024-03-23T09:45:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.556+0000 7f49959a4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4990099760 con 0x7f499006b090 2024-03-23T09:45:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.556+0000 7f498effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f499015cd40 0x7f49900d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.556+0000 7f498effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f499015cd40 0x7f49900d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58314/0 (socket says 172.21.15.47:58314) 2024-03-23T09:45:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.556+0000 7f498f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4990098a00 0x7f4990098e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40354/0 (socket says 172.21.15.47:40354) 2024-03-23T09:45:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.556+0000 7f498effd640 1 -- 172.21.15.47:0/213430838 learned_addr learned my addr 172.21.15.47:0/213430838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.556+0000 7f4987fff640 1 --2- 172.21.15.47:0/213430838 >> v2:172.21.15.175:3300/0 conn(0x7f499006b090 0x7f49900ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.556+0000 7f498effd640 1 -- 172.21.15.47:0/213430838 >> v2:172.21.15.175:3300/0 conn(0x7f499006b090 msgr2=0x7f49900ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.556+0000 7f498effd640 1 --2- 172.21.15.47:0/213430838 >> v2:172.21.15.175:3300/0 conn(0x7f499006b090 0x7f49900ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.556+0000 7f498effd640 1 -- 172.21.15.47:0/213430838 >> v1:172.21.15.47:6789/0 conn(0x7f4990098a00 legacy=0x7f4990098e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.556+0000 7f498effd640 1 -- 172.21.15.47:0/213430838 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900cf020 con 0x7f499015cd40 2024-03-23T09:45:01.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.556+0000 7f498e7fc640 1 -- 172.21.15.47:0/213430838 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001648467 0 0) 0x7f49900cedd0 con 0x7f4990098a00 2024-03-23T09:45:01.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.557+0000 7f498effd640 1 --2- 172.21.15.47:0/213430838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f499015cd40 0x7f49900d00c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f49900722b0 tx=0x7f498002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51aefb93a7528d85 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.557+0000 7f498e7fc640 1 -- 172.21.15.47:0/213430838 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980030860 con 0x7f499015cd40 2024-03-23T09:45:01.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.557+0000 7f498e7fc640 1 -- 172.21.15.47:0/213430838 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49800309c0 con 0x7f499015cd40 2024-03-23T09:45:01.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.557+0000 7f498e7fc640 1 -- 172.21.15.47:0/213430838 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980030d50 con 0x7f499015cd40 2024-03-23T09:45:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.557+0000 7f49959a4640 1 -- 172.21.15.47:0/213430838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f499015cd40 msgr2=0x7f49900d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.557+0000 7f49959a4640 1 --2- 172.21.15.47:0/213430838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f499015cd40 0x7f49900d00c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f49900722b0 tx=0x7f498002b9a0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.557+0000 7f49959a4640 1 -- 172.21.15.47:0/213430838 shutdown_connections 2024-03-23T09:45:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.557+0000 7f49959a4640 1 --2- 172.21.15.47:0/213430838 >> v2:172.21.15.175:3300/0 conn(0x7f499006b090 0x7f49900ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.557+0000 7f49959a4640 1 --2- 172.21.15.47:0/213430838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f499015cd40 0x7f49900d00c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.557+0000 7f49959a4640 1 -- 172.21.15.47:0/213430838 >> 172.21.15.47:0/213430838 conn(0x7f49900b2140 msgr2=0x7f49900b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.558+0000 7f49959a4640 1 -- 172.21.15.47:0/213430838 shutdown_connections 2024-03-23T09:45:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.558+0000 7f49959a4640 1 -- 172.21.15.47:0/213430838 wait complete. 2024-03-23T09:45:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.559+0000 7f49959a4640 1 Processor -- start 2024-03-23T09:45:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.559+0000 7f49959a4640 1 -- start start 2024-03-23T09:45:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.559+0000 7f49959a4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f499006b090 0x7f499014cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.559+0000 7f49959a4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4990098a00 0x7f499006e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.559+0000 7f49959a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f499015cd40 0x7f499006eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.559+0000 7f49959a4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49901f24e0 con 0x7f4990098a00 2024-03-23T09:45:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.559+0000 7f49959a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49900b0e10 con 0x7f499015cd40 2024-03-23T09:45:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.559+0000 7f49959a4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49901f26f0 con 0x7f499006b090 2024-03-23T09:45:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.559+0000 7f4987fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4990098a00 0x7f499006e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.559+0000 7f4987fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4990098a00 0x7f499006e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58316/0 (socket says 172.21.15.47:58316) 2024-03-23T09:45:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.559+0000 7f4987fff640 1 -- 172.21.15.47:0/3296111308 learned_addr learned my addr 172.21.15.47:0/3296111308 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.560+0000 7f4987fff640 1 -- 172.21.15.47:0/3296111308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f499006b090 msgr2=0x7f499014cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.560+0000 7f4987fff640 1 --2- 172.21.15.47:0/3296111308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f499006b090 0x7f499014cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.560+0000 7f4987fff640 1 -- 172.21.15.47:0/3296111308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f499015cd40 msgr2=0x7f499006eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.560+0000 7f4987fff640 1 --2- 172.21.15.47:0/3296111308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f499015cd40 0x7f499006eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.560+0000 7f4987fff640 1 -- 172.21.15.47:0/3296111308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49800094d0 con 0x7f4990098a00 2024-03-23T09:45:01.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.560+0000 7f4987fff640 1 --2- 172.21.15.47:0/3296111308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4990098a00 0x7f499006e860 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f497800d5f0 tx=0x7f497800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.560+0000 7f498cff9640 1 -- 172.21.15.47:0/3296111308 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49780125f0 con 0x7f4990098a00 2024-03-23T09:45:01.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.560+0000 7f498cff9640 1 -- 172.21.15.47:0/3296111308 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4978012750 con 0x7f4990098a00 2024-03-23T09:45:01.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.560+0000 7f498cff9640 1 -- 172.21.15.47:0/3296111308 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978012b00 con 0x7f4990098a00 2024-03-23T09:45:01.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.560+0000 7f49959a4640 1 -- 172.21.15.47:0/3296111308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49901f29d0 con 0x7f4990098a00 2024-03-23T09:45:01.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.560+0000 7f49959a4640 1 -- 172.21.15.47:0/3296111308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49900b0100 con 0x7f4990098a00 2024-03-23T09:45:01.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.562+0000 7f498cff9640 1 -- 172.21.15.47:0/3296111308 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4978012c60 con 0x7f4990098a00 2024-03-23T09:45:01.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.562+0000 7f498cff9640 1 --2- 172.21.15.47:0/3296111308 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f497006ac00 0x7f497006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.562+0000 7f498cff9640 1 -- 172.21.15.47:0/3296111308 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4978089820 con 0x7f4990098a00 2024-03-23T09:45:01.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.562+0000 7f498effd640 1 --2- 172.21.15.47:0/3296111308 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f497006ac00 0x7f497006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.563+0000 7f498effd640 1 --2- 172.21.15.47:0/3296111308 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f497006ac00 0x7f497006d0c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4980036000 tx=0x7f498003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.563+0000 7f49985f9740 1 Processor -- start 2024-03-23T09:45:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.563+0000 7f49985f9740 1 -- start start 2024-03-23T09:45:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f49985f9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111c56b0 0x5620111c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f49985f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111956d0 0x5620111c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f49985f9740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562011199230 0x5620111c2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f49985f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620111066a0 con 0x5620111956d0 2024-03-23T09:45:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f49985f9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562011103c70 con 0x5620111c56b0 2024-03-23T09:45:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f49985f9740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620110ff560 con 0x562011199230 2024-03-23T09:45:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f4987fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111956d0 0x5620111c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f4987fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111956d0 0x5620111c2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58318/0 (socket says 172.21.15.47:58318) 2024-03-23T09:45:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f4987fff640 1 -- 172.21.15.47:0/3656398346 learned_addr learned my addr 172.21.15.47:0/3656398346 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f498f7fe640 1 --2- 172.21.15.47:0/3656398346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562011199230 0x5620111c2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f498effd640 1 --2- 172.21.15.47:0/3656398346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111c56b0 0x5620111c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f4987fff640 1 -- 172.21.15.47:0/3656398346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562011199230 msgr2=0x5620111c2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f4987fff640 1 --2- 172.21.15.47:0/3656398346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562011199230 0x5620111c2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f4987fff640 1 -- 172.21.15.47:0/3656398346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111c56b0 msgr2=0x5620111c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f4987fff640 1 --2- 172.21.15.47:0/3656398346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111c56b0 0x5620111c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f4987fff640 1 -- 172.21.15.47:0/3656398346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620110c6d50 con 0x5620111956d0 2024-03-23T09:45:01.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f4987fff640 1 --2- 172.21.15.47:0/3656398346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111956d0 0x5620111c2920 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f49780583b0 tx=0x7f4978058db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1eace2cd56407d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f49857fa640 1 -- 172.21.15.47:0/3656398346 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978060aa0 con 0x5620111956d0 2024-03-23T09:45:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f49857fa640 1 -- 172.21.15.47:0/3656398346 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4978056a50 con 0x5620111956d0 2024-03-23T09:45:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.564+0000 7f49857fa640 1 -- 172.21.15.47:0/3656398346 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978054dc0 con 0x5620111956d0 2024-03-23T09:45:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.565+0000 7f49985f9740 1 -- 172.21.15.47:0/3656398346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111956d0 msgr2=0x5620111c2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.565+0000 7f49985f9740 1 --2- 172.21.15.47:0/3656398346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111956d0 0x5620111c2920 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f49780583b0 tx=0x7f4978058db0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.565+0000 7f49985f9740 1 -- 172.21.15.47:0/3656398346 shutdown_connections 2024-03-23T09:45:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.565+0000 7f49985f9740 1 --2- 172.21.15.47:0/3656398346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562011199230 0x5620111c2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.565+0000 7f49985f9740 1 --2- 172.21.15.47:0/3656398346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111956d0 0x5620111c2920 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.565+0000 7f49985f9740 1 --2- 172.21.15.47:0/3656398346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111c56b0 0x5620111c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.565+0000 7f49985f9740 1 -- 172.21.15.47:0/3656398346 >> 172.21.15.47:0/3656398346 conn(0x562011197940 msgr2=0x5620111e04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.565+0000 7f49985f9740 1 -- 172.21.15.47:0/3656398346 shutdown_connections 2024-03-23T09:45:01.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.565+0000 7f49985f9740 1 -- 172.21.15.47:0/3656398346 wait complete. 2024-03-23T09:45:01.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.566+0000 7f49985f9740 1 -- start start 2024-03-23T09:45:01.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.566+0000 7f49985f9740 1 Processor -- start 2024-03-23T09:45:01.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f49985f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 0x5620111ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f49985f9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111c56b0 0x5620111b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f49985f9740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111ae260 0x5620111ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f49985f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562010dd83f0 con 0x562011199230 2024-03-23T09:45:01.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f49985f9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562010e883d0 con 0x5620111c56b0 2024-03-23T09:45:01.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f49985f9740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x562011082a10 con 0x5620111ae260 2024-03-23T09:45:01.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f49985f9740 10 client.0 _ll_register_callbacks cb 0x5620110f8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:45:01.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f498effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 0x5620111ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f498effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 0x5620111ac5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58332/0 (socket says 172.21.15.47:58332) 2024-03-23T09:45:01.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f498effd640 1 -- 172.21.15.47:0/2176430293 learned_addr learned my addr 172.21.15.47:0/2176430293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f4987fff640 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111c56b0 0x5620111b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f498effd640 1 -- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111ae260 msgr2=0x5620111ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f498f7fe640 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111ae260 0x5620111ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f498effd640 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111ae260 0x5620111ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f498effd640 1 -- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111c56b0 msgr2=0x5620111b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f498effd640 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111c56b0 0x5620111b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f498effd640 1 -- 172.21.15.47:0/2176430293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49900b0100 con 0x562011199230 2024-03-23T09:45:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f4987fff640 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111c56b0 0x5620111b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f498f7fe640 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111ae260 0x5620111ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f498effd640 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 0x5620111ac5a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f498002bfb0 tx=0x7f4980004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f5acd7127add9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f496e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:45:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f496e7fc640 1 -- 172.21.15.47:0/2176430293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980043990 con 0x562011199230 2024-03-23T09:45:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f496e7fc640 1 -- 172.21.15.47:0/2176430293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f498003e070 con 0x562011199230 2024-03-23T09:45:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.567+0000 7f49985f9740 1 -- 172.21.15.47:0/2176430293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562011128150 con 0x562011199230 2024-03-23T09:45:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f496e7fc640 1 -- 172.21.15.47:0/2176430293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4980043af0 con 0x562011199230 2024-03-23T09:45:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f49985f9740 10 client.6561 fetch_fsmap learned FSMap version 26 2024-03-23T09:45:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f49985f9740 1 -- 172.21.15.47:0/2176430293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5620110c1760 con 0x562011199230 2024-03-23T09:45:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f496e7fc640 1 -- 172.21.15.47:0/2176430293 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562011128150 con 0x562011199230 2024-03-23T09:45:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f496e7fc640 1 -- 172.21.15.47:0/2176430293 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4980043c50 con 0x562011199230 2024-03-23T09:45:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f49985f9740 10 client.6561 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T09:45:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f49985f9740 10 client.6561 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T09:45:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f49985f9740 20 client.6561 populate_metadata read hostname 'smithi047' 2024-03-23T09:45:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f49985f9740 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x5620111af1b0 0x5620111af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f49985f9740 4 client.6561 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:45:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f49985f9740 1 -- 172.21.15.47:0/2176430293 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56201110a650 con 0x5620111af1b0 2024-03-23T09:45:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f4987fff640 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x5620111af1b0 0x5620111af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f496e7fc640 10 client.6561 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:45:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.568+0000 7f4987fff640 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x5620111af1b0 0x5620111af590 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6edb42b83ec267f server_cookie=b3bcb88886fdf683 in_seq=0 out_seq=0 2024-03-23T09:45:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f496e7fc640 1 -- 172.21.15.47:0/2176430293 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56201110a650 con 0x5620111af1b0 2024-03-23T09:45:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f496e7fc640 10 client.6561 handle_command_reply: tid=0 2024-03-23T09:45:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f49985f9740 1 client.6561 shutdown 2024-03-23T09:45:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f49985f9740 1 -- 172.21.15.47:0/2176430293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 msgr2=0x5620111ac5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f49985f9740 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 0x5620111ac5a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f498002bfb0 tx=0x7f4980004730 comp rx=0 tx=0).stop 2024-03-23T09:45:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f49985f9740 1 -- 172.21.15.47:0/2176430293 shutdown_connections 2024-03-23T09:45:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f49985f9740 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x5620111af1b0 0x5620111af590 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f49985f9740 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111ae260 0x5620111ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f49985f9740 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111c56b0 0x5620111b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f496e7fc640 0 client.6561 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:45:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f49985f9740 1 --2- 172.21.15.47:0/2176430293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 0x5620111ac5a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f49985f9740 1 -- 172.21.15.47:0/2176430293 >> 172.21.15.47:0/2176430293 conn(0x5620111956d0 msgr2=0x56201119b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f49985f9740 1 -- 172.21.15.47:0/2176430293 shutdown_connections 2024-03-23T09:45:01.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f49985f9740 1 -- 172.21.15.47:0/2176430293 wait complete. 2024-03-23T09:45:01.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.569+0000 7f49985f9740 20 client.6561 trim_cache size 0 max 16384 2024-03-23T09:45:01.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.581+0000 7f49985f9740 1 Processor -- start 2024-03-23T09:45:01.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.581+0000 7f49985f9740 1 -- start start 2024-03-23T09:45:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.581+0000 7f49985f9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562011199230 0x5620111e2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.581+0000 7f49985f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111ae260 0x562011199730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.581+0000 7f49985f9740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111c56b0 0x562011199c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.581+0000 7f49985f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562011195f40 con 0x5620111ae260 2024-03-23T09:45:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.581+0000 7f49985f9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620111e3f50 con 0x562011199230 2024-03-23T09:45:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.581+0000 7f49985f9740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620111f3370 con 0x5620111c56b0 2024-03-23T09:45:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.581+0000 7f4987fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111ae260 0x562011199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.582+0000 7f4987fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111ae260 0x562011199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58342/0 (socket says 172.21.15.47:58342) 2024-03-23T09:45:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.582+0000 7f4987fff640 1 -- 172.21.15.47:0/426964979 learned_addr learned my addr 172.21.15.47:0/426964979 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.582+0000 7f498effd640 1 --2- 172.21.15.47:0/426964979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562011199230 0x5620111e2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.582+0000 7f498f7fe640 1 --2- 172.21.15.47:0/426964979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111c56b0 0x562011199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.582+0000 7f4987fff640 1 -- 172.21.15.47:0/426964979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111c56b0 msgr2=0x562011199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.582+0000 7f4987fff640 1 --2- 172.21.15.47:0/426964979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111c56b0 0x562011199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.582+0000 7f4987fff640 1 -- 172.21.15.47:0/426964979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562011199230 msgr2=0x5620111e2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.582+0000 7f4987fff640 1 --2- 172.21.15.47:0/426964979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562011199230 0x5620111e2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.582+0000 7f4987fff640 1 -- 172.21.15.47:0/426964979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56201109fad0 con 0x5620111ae260 2024-03-23T09:45:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.582+0000 7f4987fff640 1 --2- 172.21.15.47:0/426964979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111ae260 0x562011199730 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f49780583b0 tx=0x7f497806d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605db8e350b545b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.582+0000 7f4985ffb640 1 -- 172.21.15.47:0/426964979 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978060b20 con 0x5620111ae260 2024-03-23T09:45:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.582+0000 7f4985ffb640 1 -- 172.21.15.47:0/426964979 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49780567f0 con 0x5620111ae260 2024-03-23T09:45:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.582+0000 7f4985ffb640 1 -- 172.21.15.47:0/426964979 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978054dc0 con 0x5620111ae260 2024-03-23T09:45:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.583+0000 7f49985f9740 1 -- 172.21.15.47:0/426964979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111ae260 msgr2=0x562011199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.583+0000 7f49985f9740 1 --2- 172.21.15.47:0/426964979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111ae260 0x562011199730 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f49780583b0 tx=0x7f497806d000 comp rx=0 tx=0).stop 2024-03-23T09:45:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.583+0000 7f49985f9740 1 -- 172.21.15.47:0/426964979 shutdown_connections 2024-03-23T09:45:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.583+0000 7f49985f9740 1 --2- 172.21.15.47:0/426964979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111c56b0 0x562011199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.583+0000 7f49985f9740 1 --2- 172.21.15.47:0/426964979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620111ae260 0x562011199730 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.583+0000 7f49985f9740 1 --2- 172.21.15.47:0/426964979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562011199230 0x5620111e2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.583+0000 7f49985f9740 1 -- 172.21.15.47:0/426964979 >> 172.21.15.47:0/426964979 conn(0x5620111956d0 msgr2=0x562011190c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.583+0000 7f49985f9740 1 -- 172.21.15.47:0/426964979 shutdown_connections 2024-03-23T09:45:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.583+0000 7f49985f9740 1 -- 172.21.15.47:0/426964979 wait complete. 2024-03-23T09:45:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.583+0000 7f49985f9740 1 -- start start 2024-03-23T09:45:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.583+0000 7f49985f9740 1 Processor -- start 2024-03-23T09:45:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f49985f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 0x5620111ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f49985f9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111ae260 0x5620111e1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f49985f9740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111c56b0 0x562011199a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f49985f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620111e1c90 con 0x562011199230 2024-03-23T09:45:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f49985f9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620111b15a0 con 0x5620111ae260 2024-03-23T09:45:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f49985f9740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620111f9f70 con 0x5620111c56b0 2024-03-23T09:45:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f49985f9740 10 client.0 _ll_register_callbacks cb 0x7f49800096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:45:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f498effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 0x5620111ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f498effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 0x5620111ac590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58350/0 (socket says 172.21.15.47:58350) 2024-03-23T09:45:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f498effd640 1 -- 172.21.15.47:0/2125275805 learned_addr learned my addr 172.21.15.47:0/2125275805 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f4987fff640 1 --2- 172.21.15.47:0/2125275805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111ae260 0x5620111e1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f498f7fe640 1 --2- 172.21.15.47:0/2125275805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111c56b0 0x562011199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f498effd640 1 -- 172.21.15.47:0/2125275805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111c56b0 msgr2=0x562011199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f498effd640 1 --2- 172.21.15.47:0/2125275805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111c56b0 0x562011199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f498effd640 1 -- 172.21.15.47:0/2125275805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111ae260 msgr2=0x5620111e1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f498effd640 1 --2- 172.21.15.47:0/2125275805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111ae260 0x5620111e1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.584+0000 7f498effd640 1 -- 172.21.15.47:0/2125275805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620110c6d50 con 0x562011199230 2024-03-23T09:45:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f498effd640 1 --2- 172.21.15.47:0/2125275805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 0x5620111ac590 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f498002bf30 tx=0x7f4980008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24aa46c5bba2300 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f496e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:45:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f496e7fc640 1 -- 172.21.15.47:0/2125275805 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498004d6e0 con 0x562011199230 2024-03-23T09:45:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f496e7fc640 1 -- 172.21.15.47:0/2125275805 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4980062030 con 0x562011199230 2024-03-23T09:45:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f49985f9740 1 -- 172.21.15.47:0/2125275805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4978056a50 con 0x562011199230 2024-03-23T09:45:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f496e7fc640 1 -- 172.21.15.47:0/2125275805 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4980043790 con 0x562011199230 2024-03-23T09:45:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f49985f9740 10 client.6567 fetch_fsmap learned FSMap version 26 2024-03-23T09:45:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f49985f9740 1 -- 172.21.15.47:0/2125275805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5620110c55b0 con 0x562011199230 2024-03-23T09:45:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f496e7fc640 1 -- 172.21.15.47:0/2125275805 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4978056a50 con 0x562011199230 2024-03-23T09:45:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f496e7fc640 1 -- 172.21.15.47:0/2125275805 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f49800438f0 con 0x562011199230 2024-03-23T09:45:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f49985f9740 10 client.6567 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T09:45:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f49985f9740 10 client.6567 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T09:45:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f49985f9740 20 client.6567 populate_metadata read hostname 'smithi047' 2024-03-23T09:45:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f49985f9740 1 --2- 172.21.15.47:0/2125275805 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x5620111c7aa0 0x5620111c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f49985f9740 4 client.6567 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-23T09:45:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.585+0000 7f49985f9740 1 -- 172.21.15.47:0/2125275805 --> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x562010db08d0 con 0x5620111c7aa0 2024-03-23T09:45:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f4987fff640 1 --2- 172.21.15.47:0/2125275805 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x5620111c7aa0 0x5620111c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f496e7fc640 10 client.6567 ms_handle_connect on v2:172.21.15.175:6834/2498456351 2024-03-23T09:45:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f4987fff640 1 --2- 172.21.15.47:0/2125275805 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x5620111c7aa0 0x5620111c7e80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec8aff4e80d6bcf6 server_cookie=8b35ca04c3ef0d23 in_seq=0 out_seq=0 2024-03-23T09:45:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f496e7fc640 1 -- 172.21.15.47:0/2125275805 <== mds.0 v2:172.21.15.175:6834/2498456351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562010db08d0 con 0x5620111c7aa0 2024-03-23T09:45:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f496e7fc640 10 client.6567 handle_command_reply: tid=0 2024-03-23T09:45:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f49985f9740 1 client.6567 shutdown 2024-03-23T09:45:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f49985f9740 1 -- 172.21.15.47:0/2125275805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 msgr2=0x5620111ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f49985f9740 1 --2- 172.21.15.47:0/2125275805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 0x5620111ac590 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f498002bf30 tx=0x7f4980008030 comp rx=0 tx=0).stop 2024-03-23T09:45:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f49985f9740 1 -- 172.21.15.47:0/2125275805 shutdown_connections 2024-03-23T09:45:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f49985f9740 1 --2- 172.21.15.47:0/2125275805 >> [v2:172.21.15.175:6834/2498456351,v1:172.21.15.175:6836/2498456351] conn(0x5620111c7aa0 0x5620111c7e80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f49985f9740 1 --2- 172.21.15.47:0/2125275805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5620111c56b0 0x562011199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f49985f9740 1 --2- 172.21.15.47:0/2125275805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620111ae260 0x5620111e1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f496e7fc640 0 client.6567 ms_handle_reset on v2:172.21.15.175:6834/2498456351 2024-03-23T09:45:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f49985f9740 1 --2- 172.21.15.47:0/2125275805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562011199230 0x5620111ac590 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f49985f9740 1 -- 172.21.15.47:0/2125275805 >> 172.21.15.47:0/2125275805 conn(0x5620111956d0 msgr2=0x562011190c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f49985f9740 1 -- 172.21.15.47:0/2125275805 shutdown_connections 2024-03-23T09:45:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.586+0000 7f49985f9740 1 -- 172.21.15.47:0/2125275805 wait complete. 2024-03-23T09:45:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.587+0000 7f49985f9740 20 client.6567 trim_cache size 0 max 16384 2024-03-23T09:45:01.598 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:01.599 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-23T09:45:01.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.587+0000 7f49959a4640 1 -- 172.21.15.47:0/3296111308 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f497006ac00 msgr2=0x7f497006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.587+0000 7f49959a4640 1 --2- 172.21.15.47:0/3296111308 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f497006ac00 0x7f497006d0c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4980036000 tx=0x7f498003b040 comp rx=0 tx=0).stop 2024-03-23T09:45:01.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.587+0000 7f49959a4640 1 -- 172.21.15.47:0/3296111308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4990098a00 msgr2=0x7f499006e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.587+0000 7f49959a4640 1 --2- 172.21.15.47:0/3296111308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4990098a00 0x7f499006e860 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f497800d5f0 tx=0x7f497800dab0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.587+0000 7f49959a4640 1 -- 172.21.15.47:0/3296111308 shutdown_connections 2024-03-23T09:45:01.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.587+0000 7f49959a4640 1 --2- 172.21.15.47:0/3296111308 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f497006ac00 0x7f497006d0c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.587+0000 7f49959a4640 1 --2- 172.21.15.47:0/3296111308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f499015cd40 0x7f499006eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.587+0000 7f49959a4640 1 --2- 172.21.15.47:0/3296111308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4990098a00 0x7f499006e860 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.587+0000 7f49959a4640 1 --2- 172.21.15.47:0/3296111308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f499006b090 0x7f499014cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.587+0000 7f49959a4640 1 -- 172.21.15.47:0/3296111308 >> 172.21.15.47:0/3296111308 conn(0x7f49900b2140 msgr2=0x7f499006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.587+0000 7f49959a4640 1 -- 172.21.15.47:0/3296111308 shutdown_connections 2024-03-23T09:45:01.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.587+0000 7f49959a4640 1 -- 172.21.15.47:0/3296111308 wait complete. 2024-03-23T09:45:01.601 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-23T09:45:01.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.701+0000 7fc5b3d4e640 1 Processor -- start 2024-03-23T09:45:01.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.701+0000 7fc5b3d4e640 1 -- start start 2024-03-23T09:45:01.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b3d4e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5ac15cd00 0x7fc5ac0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b3d4e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5ac0d1020 0x7fc5ac0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b3d4e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc5ac0cf750 0x7fc5ac0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b3d4e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ac0cfbe0 con 0x7fc5ac15cd00 2024-03-23T09:45:01.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b3d4e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ac15d0e0 con 0x7fc5ac0cf750 2024-03-23T09:45:01.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b3d4e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0d1540 con 0x7fc5ac0d1020 2024-03-23T09:45:01.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b1aea640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5ac15cd00 0x7fc5ac0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b1aea640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5ac15cd00 0x7fc5ac0d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58360/0 (socket says 172.21.15.47:58360) 2024-03-23T09:45:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b1aea640 1 -- 172.21.15.47:0/1081215713 learned_addr learned my addr 172.21.15.47:0/1081215713 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b22eb640 1 --2- 172.21.15.47:0/1081215713 >> v2:172.21.15.175:3300/0 conn(0x7fc5ac0cf750 0x7fc5ac0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b12e9640 1 --2- 172.21.15.47:0/1081215713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5ac0d1020 0x7fc5ac0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b1aea640 1 -- 172.21.15.47:0/1081215713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5ac0d1020 msgr2=0x7fc5ac0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b1aea640 1 --2- 172.21.15.47:0/1081215713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5ac0d1020 0x7fc5ac0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b1aea640 1 -- 172.21.15.47:0/1081215713 >> v2:172.21.15.175:3300/0 conn(0x7fc5ac0cf750 msgr2=0x7fc5ac0b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b1aea640 1 --2- 172.21.15.47:0/1081215713 >> v2:172.21.15.175:3300/0 conn(0x7fc5ac0cf750 0x7fc5ac0b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.702+0000 7fc5b1aea640 1 -- 172.21.15.47:0/1081215713 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac0d1820 con 0x7fc5ac15cd00 2024-03-23T09:45:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.703+0000 7fc5b1aea640 1 --2- 172.21.15.47:0/1081215713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5ac15cd00 0x7fc5ac0d0ae0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0b8040 tx=0x7fc5a402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b870e7df17b1f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.703+0000 7fc5b0ae8640 1 -- 172.21.15.47:0/1081215713 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a40317e0 con 0x7fc5ac15cd00 2024-03-23T09:45:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.703+0000 7fc5b0ae8640 1 -- 172.21.15.47:0/1081215713 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a4031940 con 0x7fc5ac15cd00 2024-03-23T09:45:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.703+0000 7fc5b0ae8640 1 -- 172.21.15.47:0/1081215713 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4031cd0 con 0x7fc5ac15cd00 2024-03-23T09:45:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.704+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/1081215713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5ac15cd00 msgr2=0x7fc5ac0d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.704+0000 7fc5b3d4e640 1 --2- 172.21.15.47:0/1081215713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5ac15cd00 0x7fc5ac0d0ae0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0b8040 tx=0x7fc5a402b880 comp rx=0 tx=0).stop 2024-03-23T09:45:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.704+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/1081215713 shutdown_connections 2024-03-23T09:45:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.704+0000 7fc5b3d4e640 1 --2- 172.21.15.47:0/1081215713 >> v2:172.21.15.175:3300/0 conn(0x7fc5ac0cf750 0x7fc5ac0b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.704+0000 7fc5b3d4e640 1 --2- 172.21.15.47:0/1081215713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5ac0d1020 0x7fc5ac0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.704+0000 7fc5b3d4e640 1 --2- 172.21.15.47:0/1081215713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5ac15cd00 0x7fc5ac0d0ae0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.704+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/1081215713 >> 172.21.15.47:0/1081215713 conn(0x7fc5ac06b160 msgr2=0x7fc5ac06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.704+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/1081215713 shutdown_connections 2024-03-23T09:45:01.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.704+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/1081215713 wait complete. 2024-03-23T09:45:01.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.705+0000 7fc5b3d4e640 1 Processor -- start 2024-03-23T09:45:01.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.705+0000 7fc5b3d4e640 1 -- start start 2024-03-23T09:45:01.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b3d4e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5ac0cf750 0x7fc5ac0b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b3d4e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5ac0d1020 0x7fc5ac1f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b3d4e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5ac15cd00 0x7fc5ac0b6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b3d4e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac14c640 con 0x7fc5ac0d1020 2024-03-23T09:45:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b3d4e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac14cf40 con 0x7fc5ac0cf750 2024-03-23T09:45:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b3d4e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac1f65e0 con 0x7fc5ac15cd00 2024-03-23T09:45:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b12e9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5ac0d1020 0x7fc5ac1f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b12e9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5ac0d1020 0x7fc5ac1f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58366/0 (socket says 172.21.15.47:58366) 2024-03-23T09:45:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b12e9640 1 -- 172.21.15.47:0/871171774 learned_addr learned my addr 172.21.15.47:0/871171774 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b22eb640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5ac15cd00 0x7fc5ac0b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b1aea640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5ac0cf750 0x7fc5ac0b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b12e9640 1 -- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5ac15cd00 msgr2=0x7fc5ac0b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b12e9640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5ac15cd00 0x7fc5ac0b6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b12e9640 1 -- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5ac0cf750 msgr2=0x7fc5ac0b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b12e9640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5ac0cf750 0x7fc5ac0b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b12e9640 1 -- 172.21.15.47:0/871171774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40094d0 con 0x7fc5ac0d1020 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.706+0000 7fc5b12e9640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5ac0d1020 0x7fc5ac1f56d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000d800 tx=0x7fc5a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.707+0000 7fc5aa7fc640 1 -- 172.21.15.47:0/871171774 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000b6c0 con 0x7fc5ac0d1020 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.707+0000 7fc5aa7fc640 1 -- 172.21.15.47:0/871171774 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a000b820 con 0x7fc5ac0d1020 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.707+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/871171774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ac1f6020 con 0x7fc5ac0d1020 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.707+0000 7fc5aa7fc640 1 -- 172.21.15.47:0/871171774 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000bbb0 con 0x7fc5ac0d1020 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.707+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/871171774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac14cbd0 con 0x7fc5ac0d1020 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.708+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/871171774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac072360 con 0x7fc5ac0d1020 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.708+0000 7fc5aa7fc640 1 -- 172.21.15.47:0/871171774 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc5a000bd10 con 0x7fc5ac0d1020 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.709+0000 7fc5aa7fc640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc59806ac00 0x7fc59806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.709+0000 7fc5aa7fc640 1 -- 172.21.15.47:0/871171774 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5a00899a0 con 0x7fc5ac0d1020 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.709+0000 7fc5b1aea640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc59806ac00 0x7fc59806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.709+0000 7fc5b1aea640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc59806ac00 0x7fc59806d0c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4036000 tx=0x7fc5a403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:01.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.713+0000 7fc5aa7fc640 1 -- 172.21.15.47:0/871171774 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc5a0054ae0 con 0x7fc5ac0d1020 2024-03-23T09:45:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.932+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/871171774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc5ac066be0 con 0x7fc5ac0d1020 2024-03-23T09:45:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.932+0000 7fc5aa7fc640 1 -- 172.21.15.47:0/871171774 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc5a0058100 con 0x7fc5ac0d1020 2024-03-23T09:45:01.932 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:01.932 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-23T09:45:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.934+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc59806ac00 msgr2=0x7fc59806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.934+0000 7fc5b3d4e640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc59806ac00 0x7fc59806d0c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4036000 tx=0x7fc5a403d040 comp rx=0 tx=0).stop 2024-03-23T09:45:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.934+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/871171774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5ac0d1020 msgr2=0x7fc5ac1f56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.934+0000 7fc5b3d4e640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5ac0d1020 0x7fc5ac1f56d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000d800 tx=0x7fc5a000dcc0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.934+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/871171774 shutdown_connections 2024-03-23T09:45:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.934+0000 7fc5b3d4e640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc59806ac00 0x7fc59806d0c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.934+0000 7fc5b3d4e640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5ac15cd00 0x7fc5ac0b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.934+0000 7fc5b3d4e640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5ac0d1020 0x7fc5ac1f56d0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.934+0000 7fc5b3d4e640 1 --2- 172.21.15.47:0/871171774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5ac0cf750 0x7fc5ac0b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.934+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/871171774 >> 172.21.15.47:0/871171774 conn(0x7fc5ac06b160 msgr2=0x7fc5ac0cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.934+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/871171774 shutdown_connections 2024-03-23T09:45:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:01.934+0000 7fc5b3d4e640 1 -- 172.21.15.47:0/871171774 wait complete. 2024-03-23T09:45:01.944 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-23T09:45:01.944 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:45:01.945 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:45:01.993 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:45:02.041 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:45:02.041 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:45:02.041 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:45:02.057 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:45:02.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.197+0000 7f40acd63640 1 Processor -- start 2024-03-23T09:45:02.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.197+0000 7f40acd63640 1 -- start start 2024-03-23T09:45:02.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.197+0000 7f40acd63640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40a815ca90 0x7f40a8160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.197+0000 7f40acd63640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a80b8350 0x7f40a80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.197+0000 7f40acd63640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a80d11e0 con 0x7f40a80cdc40 2024-03-23T09:45:02.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.197+0000 7f40acd63640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a80ce0d0 con 0x7f40a815ca90 2024-03-23T09:45:02.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.197+0000 7f40acd63640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a80d2680 con 0x7f40a80b8350 2024-03-23T09:45:02.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.198+0000 7f40a659c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40a815ca90 0x7f40a8160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.198+0000 7f40a659c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40a815ca90 0x7f40a8160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58372/0 (socket says 172.21.15.47:58372) 2024-03-23T09:45:02.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.198+0000 7f40a6d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f40a80cdc40 0x7f40a80b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40370/0 (socket says 172.21.15.47:40370) 2024-03-23T09:45:02.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.198+0000 7f40a659c640 1 -- 172.21.15.47:0/3426707871 learned_addr learned my addr 172.21.15.47:0/3426707871 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:02.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.198+0000 7f40a5d9b640 1 --2- 172.21.15.47:0/3426707871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a80b8350 0x7f40a80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.198+0000 7f40a659c640 1 -- 172.21.15.47:0/3426707871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a80b8350 msgr2=0x7f40a80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:02.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.198+0000 7f40a659c640 1 --2- 172.21.15.47:0/3426707871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a80b8350 0x7f40a80cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.198+0000 7f40a659c640 1 -- 172.21.15.47:0/3426707871 >> v1:172.21.15.47:6789/0 conn(0x7f40a80cdc40 legacy=0x7f40a80b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.198+0000 7f40a559a640 1 -- 172.21.15.47:0/3426707871 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523969666 0 0) 0x7f40a80d11e0 con 0x7f40a80cdc40 2024-03-23T09:45:02.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.198+0000 7f40a659c640 1 -- 172.21.15.47:0/3426707871 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a8160f60 con 0x7f40a815ca90 2024-03-23T09:45:02.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.199+0000 7f40a659c640 1 --2- 172.21.15.47:0/3426707871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a815ca90 0x7f40a8160710 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009870 tx=0x7f40a002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b4c54aaf39761 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:02.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.199+0000 7f40a559a640 1 -- 172.21.15.47:0/3426707871 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0031640 con 0x7f40a815ca90 2024-03-23T09:45:02.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.199+0000 7f40a559a640 1 -- 172.21.15.47:0/3426707871 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a00317a0 con 0x7f40a815ca90 2024-03-23T09:45:02.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.199+0000 7f40a559a640 1 -- 172.21.15.47:0/3426707871 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0031b30 con 0x7f40a815ca90 2024-03-23T09:45:02.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.199+0000 7f40acd63640 1 -- 172.21.15.47:0/3426707871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a815ca90 msgr2=0x7f40a8160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.199+0000 7f40acd63640 1 --2- 172.21.15.47:0/3426707871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a815ca90 0x7f40a8160710 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009870 tx=0x7f40a002b8f0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.199+0000 7f40acd63640 1 -- 172.21.15.47:0/3426707871 shutdown_connections 2024-03-23T09:45:02.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.199+0000 7f40acd63640 1 --2- 172.21.15.47:0/3426707871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a80b8350 0x7f40a80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.199+0000 7f40acd63640 1 --2- 172.21.15.47:0/3426707871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a815ca90 0x7f40a8160710 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.199+0000 7f40acd63640 1 -- 172.21.15.47:0/3426707871 >> 172.21.15.47:0/3426707871 conn(0x7f40a80659d0 msgr2=0x7f40a8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:02.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.199+0000 7f40acd63640 1 -- 172.21.15.47:0/3426707871 shutdown_connections 2024-03-23T09:45:02.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.200+0000 7f40acd63640 1 -- 172.21.15.47:0/3426707871 wait complete. 2024-03-23T09:45:02.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.201+0000 7f40acd63640 1 Processor -- start 2024-03-23T09:45:02.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.201+0000 7f40acd63640 1 -- start start 2024-03-23T09:45:02.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.201+0000 7f40acd63640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a80b8350 0x7f40a815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.201+0000 7f40acd63640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a80cdc40 0x7f40a80cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.201+0000 7f40acd63640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a815ca90 0x7f40a81501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.201+0000 7f40acd63640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a815d450 con 0x7f40a80b8350 2024-03-23T09:45:02.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.201+0000 7f40acd63640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a815d730 con 0x7f40a815ca90 2024-03-23T09:45:02.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.201+0000 7f40acd63640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a815da10 con 0x7f40a80cdc40 2024-03-23T09:45:02.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40a659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a80b8350 0x7f40a815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40a659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a80b8350 0x7f40a815ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58378/0 (socket says 172.21.15.47:58378) 2024-03-23T09:45:02.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40a659c640 1 -- 172.21.15.47:0/3438403776 learned_addr learned my addr 172.21.15.47:0/3438403776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:02.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40a5d9b640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a80cdc40 0x7f40a80cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40a6d9d640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a815ca90 0x7f40a81501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40a659c640 1 -- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a80cdc40 msgr2=0x7f40a80cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40a659c640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a80cdc40 0x7f40a80cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40a659c640 1 -- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a815ca90 msgr2=0x7f40a81501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40a659c640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a815ca90 0x7f40a81501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40a659c640 1 -- 172.21.15.47:0/3438403776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a00094d0 con 0x7f40a80b8350 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40a5d9b640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a80cdc40 0x7f40a80cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40a6d9d640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a815ca90 0x7f40a81501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40a659c640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a80b8350 0x7f40a815ce70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f40a0000c00 tx=0x7f40a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f409effd640 1 -- 172.21.15.47:0/3438403776 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0004230 con 0x7f40a80b8350 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f409effd640 1 -- 172.21.15.47:0/3438403776 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a0004390 con 0x7f40a80b8350 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40acd63640 1 -- 172.21.15.47:0/3438403776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a80cbc80 con 0x7f40a80b8350 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f409effd640 1 -- 172.21.15.47:0/3438403776 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a003baa0 con 0x7f40a80b8350 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.202+0000 7f40acd63640 1 -- 172.21.15.47:0/3438403776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a8150700 con 0x7f40a80b8350 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.203+0000 7f40acd63640 1 -- 172.21.15.47:0/3438403776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a8072360 con 0x7f40a80b8350 2024-03-23T09:45:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.204+0000 7f409effd640 1 -- 172.21.15.47:0/3438403776 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f40a0043070 con 0x7f40a80b8350 2024-03-23T09:45:02.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.204+0000 7f409effd640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f408806ac00 0x7f408806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.204+0000 7f409effd640 1 -- 172.21.15.47:0/3438403776 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40a00ab130 con 0x7f40a80b8350 2024-03-23T09:45:02.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.204+0000 7f40a5d9b640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f408806ac00 0x7f408806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.205+0000 7f40a5d9b640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f408806ac00 0x7f408806d0c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4090006fd0 tx=0x7f4090008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:02.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.208+0000 7f409effd640 1 -- 172.21.15.47:0/3438403776 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f40a00763a0 con 0x7f40a80b8350 2024-03-23T09:45:02.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.390+0000 7f40acd63640 1 -- 172.21.15.47:0/3438403776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40a8066be0 con 0x7f40a80b8350 2024-03-23T09:45:02.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.392+0000 7f409effd640 1 -- 172.21.15.47:0/3438403776 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16097 (secure 0 0 0) 0x7f40a00799c0 con 0x7f40a80b8350 2024-03-23T09:45:02.392 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:02.392 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":58,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:43:48.390657+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:43:45.646160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-23T09:43:46.351634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:45:02.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.394+0000 7f40acd63640 1 -- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f408806ac00 msgr2=0x7f408806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.394+0000 7f40acd63640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f408806ac00 0x7f408806d0c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4090006fd0 tx=0x7f4090008040 comp rx=0 tx=0).stop 2024-03-23T09:45:02.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.394+0000 7f40acd63640 1 -- 172.21.15.47:0/3438403776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a80b8350 msgr2=0x7f40a815ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.394+0000 7f40acd63640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a80b8350 0x7f40a815ce70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f40a0000c00 tx=0x7f40a00046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.394+0000 7f40acd63640 1 -- 172.21.15.47:0/3438403776 shutdown_connections 2024-03-23T09:45:02.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.394+0000 7f40acd63640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f408806ac00 0x7f408806d0c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.394+0000 7f40acd63640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a815ca90 0x7f40a81501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.394+0000 7f40acd63640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a80cdc40 0x7f40a80cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.394+0000 7f40acd63640 1 --2- 172.21.15.47:0/3438403776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a80b8350 0x7f40a815ce70 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.394+0000 7f40acd63640 1 -- 172.21.15.47:0/3438403776 >> 172.21.15.47:0/3438403776 conn(0x7f40a80659d0 msgr2=0x7f40a8065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:02.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.394+0000 7f40acd63640 1 -- 172.21.15.47:0/3438403776 shutdown_connections 2024-03-23T09:45:02.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.394+0000 7f40acd63640 1 -- 172.21.15.47:0/3438403776 wait complete. 2024-03-23T09:45:02.404 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:43:45.646160+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:43:46.351634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:45:02.405 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:45:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.508+0000 7f6b6b1d9640 1 Processor -- start 2024-03-23T09:45:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.508+0000 7f6b6b1d9640 1 -- start start 2024-03-23T09:45:02.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.509+0000 7f6b6b1d9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640cde10 0x7f6b64160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.509+0000 7f6b6b1d9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b640b8350 0x7f6b640d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.509+0000 7f6b6b1d9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b64161630 con 0x7f6b6415c990 2024-03-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.509+0000 7f6b6b1d9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b640d1030 con 0x7f6b640b8350 2024-03-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.509+0000 7f6b6b1d9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640cfe10 con 0x7f6b640cde10 2024-03-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.509+0000 7f6b69776640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b640b8350 0x7f6b640d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.509+0000 7f6b69776640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b640b8350 0x7f6b640d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58392/0 (socket says 172.21.15.47:58392) 2024-03-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.509+0000 7f6b69776640 1 -- 172.21.15.47:0/6714578 learned_addr learned my addr 172.21.15.47:0/6714578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.509+0000 7f6b63fff640 1 --2- 172.21.15.47:0/6714578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640cde10 0x7f6b64160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.509+0000 7f6b69776640 1 -- 172.21.15.47:0/6714578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640cde10 msgr2=0x7f6b64160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.509+0000 7f6b69776640 1 --2- 172.21.15.47:0/6714578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640cde10 0x7f6b64160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.509+0000 7f6b69776640 1 -- 172.21.15.47:0/6714578 >> v1:172.21.15.175:6789/0 conn(0x7f6b6415c990 legacy=0x7f6b640cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.509+0000 7f6b69776640 1 -- 172.21.15.47:0/6714578 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b640b5b60 con 0x7f6b640b8350 2024-03-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.510+0000 7f6b69776640 1 --2- 172.21.15.47:0/6714578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b640b8350 0x7f6b640d14c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009e50 tx=0x7f6b5c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ea5b34186caf62 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.510+0000 7f6b637fe640 1 -- 172.21.15.47:0/6714578 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c02c9e0 con 0x7f6b640b8350 2024-03-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.510+0000 7f6b637fe640 1 -- 172.21.15.47:0/6714578 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c02cb40 con 0x7f6b640b8350 2024-03-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.510+0000 7f6b637fe640 1 -- 172.21.15.47:0/6714578 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0343f0 con 0x7f6b640b8350 2024-03-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.511+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/6714578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b640b8350 msgr2=0x7f6b640d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.511+0000 7f6b6b1d9640 1 --2- 172.21.15.47:0/6714578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b640b8350 0x7f6b640d14c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009e50 tx=0x7f6b5c02be40 comp rx=0 tx=0).stop 2024-03-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.511+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/6714578 shutdown_connections 2024-03-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.511+0000 7f6b6b1d9640 1 --2- 172.21.15.47:0/6714578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b640b8350 0x7f6b640d14c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.511+0000 7f6b6b1d9640 1 --2- 172.21.15.47:0/6714578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640cde10 0x7f6b64160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.511+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/6714578 >> 172.21.15.47:0/6714578 conn(0x7f6b640659d0 msgr2=0x7f6b64065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.511+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/6714578 shutdown_connections 2024-03-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.511+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/6714578 wait complete. 2024-03-23T09:45:02.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.512+0000 7f6b6b1d9640 1 Processor -- start 2024-03-23T09:45:02.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.512+0000 7f6b6b1d9640 1 -- start start 2024-03-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.513+0000 7f6b6b1d9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b640b8350 0x7f6b6415d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.513+0000 7f6b6b1d9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640cde10 0x7f6b640c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.513+0000 7f6b6b1d9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b6415c990 0x7f6b6414b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.513+0000 7f6b6b1d9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640c9a90 con 0x7f6b6415c990 2024-03-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.513+0000 7f6b6b1d9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b64155f70 con 0x7f6b640b8350 2024-03-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.513+0000 7f6b6b1d9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6415fda0 con 0x7f6b640cde10 2024-03-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.513+0000 7f6b69776640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b6415c990 0x7f6b6414b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.513+0000 7f6b69776640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b6415c990 0x7f6b6414b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58394/0 (socket says 172.21.15.47:58394) 2024-03-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.513+0000 7f6b69776640 1 -- 172.21.15.47:0/107856287 learned_addr learned my addr 172.21.15.47:0/107856287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.513+0000 7f6b69776640 1 -- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640cde10 msgr2=0x7f6b640c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.514+0000 7f6b69776640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640cde10 0x7f6b640c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.514+0000 7f6b69776640 1 -- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b640b8350 msgr2=0x7f6b6415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.514+0000 7f6b68f75640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b640b8350 0x7f6b6415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.514+0000 7f6b69776640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b640b8350 0x7f6b6415d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.514+0000 7f6b69776640 1 -- 172.21.15.47:0/107856287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c009b30 con 0x7f6b6415c990 2024-03-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.514+0000 7f6b68f75640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b640b8350 0x7f6b6415d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.514+0000 7f6b69776640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b6415c990 0x7f6b6414b8a0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c036040 tx=0x7f6b5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.514+0000 7f6b617fa640 1 -- 172.21.15.47:0/107856287 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c009510 con 0x7f6b6415c990 2024-03-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.514+0000 7f6b617fa640 1 -- 172.21.15.47:0/107856287 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c03e450 con 0x7f6b6415c990 2024-03-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.514+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/107856287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b64160080 con 0x7f6b6415c990 2024-03-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.514+0000 7f6b617fa640 1 -- 172.21.15.47:0/107856287 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c03e7e0 con 0x7f6b6415c990 2024-03-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.514+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/107856287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b640c9fc0 con 0x7f6b6415c990 2024-03-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.515+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/107856287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b64072360 con 0x7f6b6415c990 2024-03-23T09:45:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.515+0000 7f6b617fa640 1 -- 172.21.15.47:0/107856287 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6b5c03e940 con 0x7f6b6415c990 2024-03-23T09:45:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.516+0000 7f6b617fa640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b5006ac00 0x7f6b5006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.516+0000 7f6b617fa640 1 -- 172.21.15.47:0/107856287 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b5c0ac020 con 0x7f6b6415c990 2024-03-23T09:45:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.516+0000 7f6b68f75640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b5006ac00 0x7f6b5006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.517+0000 7f6b68f75640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b5006ac00 0x7f6b5006d0c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b54000f60 tx=0x7f6b54009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:02.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.520+0000 7f6b617fa640 1 -- 172.21.15.47:0/107856287 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6b5c0773b0 con 0x7f6b6415c990 2024-03-23T09:45:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.698+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/107856287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b64066be0 con 0x7f6b6415c990 2024-03-23T09:45:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.698+0000 7f6b617fa640 1 -- 172.21.15.47:0/107856287 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b5c07a9d0 con 0x7f6b6415c990 2024-03-23T09:45:02.698 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:45:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.700+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b5006ac00 msgr2=0x7f6b5006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.700+0000 7f6b6b1d9640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b5006ac00 0x7f6b5006d0c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b54000f60 tx=0x7f6b54009040 comp rx=0 tx=0).stop 2024-03-23T09:45:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.700+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/107856287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b6415c990 msgr2=0x7f6b6414b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.700+0000 7f6b6b1d9640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b6415c990 0x7f6b6414b8a0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c036040 tx=0x7f6b5c004a40 comp rx=0 tx=0).stop 2024-03-23T09:45:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.700+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/107856287 shutdown_connections 2024-03-23T09:45:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.700+0000 7f6b6b1d9640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b5006ac00 0x7f6b5006d0c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.700+0000 7f6b6b1d9640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b6415c990 0x7f6b6414b8a0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.700+0000 7f6b6b1d9640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640cde10 0x7f6b640c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.700+0000 7f6b6b1d9640 1 --2- 172.21.15.47:0/107856287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b640b8350 0x7f6b6415d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.700+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/107856287 >> 172.21.15.47:0/107856287 conn(0x7f6b640659d0 msgr2=0x7f6b640b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.700+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/107856287 shutdown_connections 2024-03-23T09:45:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.700+0000 7f6b6b1d9640 1 -- 172.21.15.47:0/107856287 wait complete. 2024-03-23T09:45:02.710 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:45:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.814+0000 7ff9a53d2640 1 Processor -- start 2024-03-23T09:45:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.814+0000 7ff9a53d2640 1 -- start start 2024-03-23T09:45:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff9a53d2640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff9a015cbf0 0x7ff9a00b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff9a53d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cf9c0 0x7ff9a00d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff9a53d2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a0161570 con 0x7ff9a00cd140 2024-03-23T09:45:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff9a53d2640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a0160960 con 0x7ff9a015cbf0 2024-03-23T09:45:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff9a53d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a0160c40 con 0x7ff9a00cf9c0 2024-03-23T09:45:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff997fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff9a00cd140 0x7ff9a00cf2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40372/0 (socket says 172.21.15.47:40372) 2024-03-23T09:45:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff997fff640 1 -- 172.21.15.47:0/3533275094 learned_addr learned my addr 172.21.15.47:0/3533275094 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff99effd640 1 --2- 172.21.15.47:0/3533275094 >> v2:172.21.15.175:3300/0 conn(0x7ff9a015cbf0 0x7ff9a00b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff99f7fe640 1 --2- 172.21.15.47:0/3533275094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cf9c0 0x7ff9a00d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184036425 0 0) 0x7ff9a0161570 con 0x7ff9a00cd140 2024-03-23T09:45:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff98c003520 con 0x7ff9a00cd140 2024-03-23T09:45:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 855658342 0 0) 0x7ff98c003520 con 0x7ff9a00cd140 2024-03-23T09:45:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cf9c0 msgr2=0x7ff9a00d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff99e7fc640 1 --2- 172.21.15.47:0/3533275094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cf9c0 0x7ff9a00d0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 >> v2:172.21.15.175:3300/0 conn(0x7ff9a015cbf0 msgr2=0x7ff9a00b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff99e7fc640 1 --2- 172.21.15.47:0/3533275094 >> v2:172.21.15.175:3300/0 conn(0x7ff9a015cbf0 0x7ff9a00b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a0160f20 con 0x7ff9a00cd140 2024-03-23T09:45:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7ff988001670 con 0x7ff9a00cd140 2024-03-23T09:45:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.815+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 >> v1:172.21.15.47:6789/0 conn(0x7ff9a00cd140 legacy=0x7ff9a00cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff99e7fc640 1 --2- 172.21.15.47:0/3533275094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff98c003970 0x7ff98c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff99e7fc640 1 --2- 172.21.15.47:0/3533275094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff98c0047c0 0x7ff98c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff99e7fc640 1 --2- 172.21.15.47:0/3533275094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cf9c0 0x7ff98c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a0160960 con 0x7ff98c0047c0 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a0160c40 con 0x7ff98c003970 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c00b980 con 0x7ff9a00cf9c0 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff997fff640 1 --2- 172.21.15.47:0/3533275094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff98c0047c0 0x7ff98c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff99effd640 1 --2- 172.21.15.47:0/3533275094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff98c003970 0x7ff98c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff99f7fe640 1 --2- 172.21.15.47:0/3533275094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cf9c0 0x7ff98c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff997fff640 1 -- 172.21.15.47:0/3533275094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cf9c0 msgr2=0x7ff98c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff997fff640 1 --2- 172.21.15.47:0/3533275094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cf9c0 0x7ff98c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff997fff640 1 -- 172.21.15.47:0/3533275094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff98c003970 msgr2=0x7ff98c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff997fff640 1 --2- 172.21.15.47:0/3533275094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff98c003970 0x7ff98c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.816+0000 7ff997fff640 1 -- 172.21.15.47:0/3533275094 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c003520 con 0x7ff98c0047c0 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.817+0000 7ff997fff640 1 --2- 172.21.15.47:0/3533275094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff98c0047c0 0x7ff98c004c40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c240 tx=0x7ff98c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df173da3616ba37 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.817+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98802cb00 con 0x7ff98c0047c0 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.817+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98802cc60 con 0x7ff98c0047c0 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.817+0000 7ff99e7fc640 1 -- 172.21.15.47:0/3533275094 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880058d0 con 0x7ff98c0047c0 2024-03-23T09:45:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.817+0000 7ff9a53d2640 1 -- 172.21.15.47:0/3533275094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff98c0047c0 msgr2=0x7ff98c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.817+0000 7ff9a53d2640 1 --2- 172.21.15.47:0/3533275094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff98c0047c0 0x7ff98c004c40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c240 tx=0x7ff98c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.817+0000 7ff99f7fe640 1 -- 172.21.15.47:0/3533275094 reap_dead start 2024-03-23T09:45:02.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.817+0000 7ff9a53d2640 1 -- 172.21.15.47:0/3533275094 shutdown_connections 2024-03-23T09:45:02.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.817+0000 7ff9a53d2640 1 -- 172.21.15.47:0/3533275094 >> 172.21.15.47:0/3533275094 conn(0x7ff9a006b160 msgr2=0x7ff9a006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:02.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.817+0000 7ff9a53d2640 1 -- 172.21.15.47:0/3533275094 shutdown_connections 2024-03-23T09:45:02.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.817+0000 7ff9a53d2640 1 -- 172.21.15.47:0/3533275094 wait complete. 2024-03-23T09:45:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.819+0000 7ff9a53d2640 1 Processor -- start 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.819+0000 7ff9a53d2640 1 -- start start 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.819+0000 7ff9a53d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff98c0047c0 0x7ff9a015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.819+0000 7ff9a53d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a00b38d0 0x7ff9a00b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.819+0000 7ff9a53d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cbb80 0x7ff9a00cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.819+0000 7ff9a53d2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00cc540 con 0x7ff9a00b38d0 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.819+0000 7ff9a53d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00a9450 con 0x7ff98c0047c0 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.819+0000 7ff9a53d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a015d1f0 con 0x7ff9a00cbb80 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.819+0000 7ff997fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a00b38d0 0x7ff9a00b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.819+0000 7ff997fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a00b38d0 0x7ff9a00b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58422/0 (socket says 172.21.15.47:58422) 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.819+0000 7ff997fff640 1 -- 172.21.15.47:0/2683287456 learned_addr learned my addr 172.21.15.47:0/2683287456 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff99effd640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff98c0047c0 0x7ff9a015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff99f7fe640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cbb80 0x7ff9a00cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff997fff640 1 -- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cbb80 msgr2=0x7ff9a00cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff997fff640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cbb80 0x7ff9a00cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff997fff640 1 -- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff98c0047c0 msgr2=0x7ff9a015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff997fff640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff98c0047c0 0x7ff9a015ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff997fff640 1 -- 172.21.15.47:0/2683287456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a015d490 con 0x7ff9a00b38d0 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff99f7fe640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cbb80 0x7ff9a00cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff99effd640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff98c0047c0 0x7ff9a015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff997fff640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a00b38d0 0x7ff9a00b3d50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c360 tx=0x7ff988001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff9977fe640 1 -- 172.21.15.47:0/2683287456 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988005d40 con 0x7ff9a00b38d0 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff9977fe640 1 -- 172.21.15.47:0/2683287456 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff988007070 con 0x7ff9a00b38d0 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff9a53d2640 1 -- 172.21.15.47:0/2683287456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a015d770 con 0x7ff9a00b38d0 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff9977fe640 1 -- 172.21.15.47:0/2683287456 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988005ea0 con 0x7ff9a00b38d0 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.820+0000 7ff9a53d2640 1 -- 172.21.15.47:0/2683287456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a015dd30 con 0x7ff9a00b38d0 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.821+0000 7ff9a53d2640 1 -- 172.21.15.47:0/2683287456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a0072360 con 0x7ff9a00b38d0 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.822+0000 7ff9977fe640 1 -- 172.21.15.47:0/2683287456 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff988032070 con 0x7ff9a00b38d0 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.822+0000 7ff9977fe640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff98006ac30 0x7ff98006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.822+0000 7ff9977fe640 1 -- 172.21.15.47:0/2683287456 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9880aa7c0 con 0x7ff9a00b38d0 2024-03-23T09:45:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.822+0000 7ff99effd640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff98006ac30 0x7ff98006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:02.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.823+0000 7ff99effd640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff98006ac30 0x7ff98006d0f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff990009bc0 tx=0x7ff990008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:02.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:02.826+0000 7ff9977fe640 1 -- 172.21.15.47:0/2683287456 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff9880759a0 con 0x7ff9a00b38d0 2024-03-23T09:45:03.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.008+0000 7ff9a53d2640 1 -- 172.21.15.47:0/2683287456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9a0066be0 con 0x7ff9a00b38d0 2024-03-23T09:45:03.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.008+0000 7ff9977fe640 1 -- 172.21.15.47:0/2683287456 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9880450a0 con 0x7ff9a00b38d0 2024-03-23T09:45:03.008 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:45:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.010+0000 7ff9a53d2640 1 -- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff98006ac30 msgr2=0x7ff98006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.010+0000 7ff9a53d2640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff98006ac30 0x7ff98006d0f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff990009bc0 tx=0x7ff990008040 comp rx=0 tx=0).stop 2024-03-23T09:45:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.010+0000 7ff9a53d2640 1 -- 172.21.15.47:0/2683287456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a00b38d0 msgr2=0x7ff9a00b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.010+0000 7ff9a53d2640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a00b38d0 0x7ff9a00b3d50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c360 tx=0x7ff988001a00 comp rx=0 tx=0).stop 2024-03-23T09:45:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.010+0000 7ff9a53d2640 1 -- 172.21.15.47:0/2683287456 shutdown_connections 2024-03-23T09:45:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.010+0000 7ff9a53d2640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff98006ac30 0x7ff98006d0f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.010+0000 7ff9a53d2640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9a00cbb80 0x7ff9a00cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.010+0000 7ff9a53d2640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a00b38d0 0x7ff9a00b3d50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.010+0000 7ff9a53d2640 1 --2- 172.21.15.47:0/2683287456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff98c0047c0 0x7ff9a015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.010+0000 7ff9a53d2640 1 -- 172.21.15.47:0/2683287456 >> 172.21.15.47:0/2683287456 conn(0x7ff9a006b160 msgr2=0x7ff9a00b5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.010+0000 7ff9a53d2640 1 -- 172.21.15.47:0/2683287456 shutdown_connections 2024-03-23T09:45:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.010+0000 7ff9a53d2640 1 -- 172.21.15.47:0/2683287456 wait complete. 2024-03-23T09:45:03.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:45:03.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.125+0000 7fbed2947640 1 Processor -- start 2024-03-23T09:45:03.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.125+0000 7fbed2947640 1 -- start start 2024-03-23T09:45:03.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.126+0000 7fbed2947640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbecc15cee0 0x7fbecc0d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.126+0000 7fbed2947640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbecc0d23a0 0x7fbecc0d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.126+0000 7fbed2947640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc0d1300 con 0x7fbecc0b8300 2024-03-23T09:45:03.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.126+0000 7fbed2947640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbecc0d15e0 con 0x7fbecc15cee0 2024-03-23T09:45:03.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.126+0000 7fbed2947640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0d18c0 con 0x7fbecc0d23a0 2024-03-23T09:45:03.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.126+0000 7fbed0ee4640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbecc0b8300 0x7fbecc0b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40384/0 (socket says 172.21.15.47:40384) 2024-03-23T09:45:03.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.126+0000 7fbed0ee4640 1 -- 172.21.15.47:0/2007211223 learned_addr learned my addr 172.21.15.47:0/2007211223 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:03.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.126+0000 7fbecb7fe640 1 --2- 172.21.15.47:0/2007211223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbecc0d23a0 0x7fbecc0d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.126+0000 7fbecbfff640 1 --2- 172.21.15.47:0/2007211223 >> v2:172.21.15.175:3300/0 conn(0x7fbecc15cee0 0x7fbecc0d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.126+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581246903 0 0) 0x7fbecc0d1300 con 0x7fbecc0b8300 2024-03-23T09:45:03.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.126+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeb4003520 con 0x7fbecc0b8300 2024-03-23T09:45:03.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1410293545 0 0) 0x7fbeb4003520 con 0x7fbecc0b8300 2024-03-23T09:45:03.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbecc0d23a0 msgr2=0x7fbecc0d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:03.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 --2- 172.21.15.47:0/2007211223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbecc0d23a0 0x7fbecc0d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 >> v2:172.21.15.175:3300/0 conn(0x7fbecc15cee0 msgr2=0x7fbecc0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:03.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 --2- 172.21.15.47:0/2007211223 >> v2:172.21.15.175:3300/0 conn(0x7fbecc15cee0 0x7fbecc0d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc0d1ba0 con 0x7fbecc0b8300 2024-03-23T09:45:03.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fbec40019f0 con 0x7fbecc0b8300 2024-03-23T09:45:03.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 >> v1:172.21.15.47:6789/0 conn(0x7fbecc0b8300 legacy=0x7fbecc0b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:03.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 --2- 172.21.15.47:0/2007211223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbeb4003970 0x7fbeb4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 --2- 172.21.15.47:0/2007211223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbecc0d23a0 0x7fbeb40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 --2- 172.21.15.47:0/2007211223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeb4004d00 0x7fbeb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0d15e0 con 0x7fbeb4004d00 2024-03-23T09:45:03.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0d18c0 con 0x7fbeb4003970 2024-03-23T09:45:03.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb400b980 con 0x7fbecc0d23a0 2024-03-23T09:45:03.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbed0ee4640 1 --2- 172.21.15.47:0/2007211223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeb4004d00 0x7fbeb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecbfff640 1 --2- 172.21.15.47:0/2007211223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbeb4003970 0x7fbeb4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbecb7fe640 1 --2- 172.21.15.47:0/2007211223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbecc0d23a0 0x7fbeb40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbed0ee4640 1 -- 172.21.15.47:0/2007211223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbecc0d23a0 msgr2=0x7fbeb40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:03.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbed0ee4640 1 --2- 172.21.15.47:0/2007211223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbecc0d23a0 0x7fbeb40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbed0ee4640 1 -- 172.21.15.47:0/2007211223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbeb4003970 msgr2=0x7fbeb4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbed0ee4640 1 --2- 172.21.15.47:0/2007211223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbeb4003970 0x7fbeb4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.127+0000 7fbed0ee4640 1 -- 172.21.15.47:0/2007211223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb4003520 con 0x7fbeb4004d00 2024-03-23T09:45:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.128+0000 7fbed0ee4640 1 --2- 172.21.15.47:0/2007211223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeb4004d00 0x7fbeb4005180 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0b5870 tx=0x7fbeb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2c56a249288fda server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.128+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4034630 con 0x7fbeb4004d00 2024-03-23T09:45:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.128+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec402f070 con 0x7fbeb4004d00 2024-03-23T09:45:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.128+0000 7fbecaffd640 1 -- 172.21.15.47:0/2007211223 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec40349f0 con 0x7fbeb4004d00 2024-03-23T09:45:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.129+0000 7fbed2947640 1 -- 172.21.15.47:0/2007211223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeb4004d00 msgr2=0x7fbeb4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.129+0000 7fbed2947640 1 --2- 172.21.15.47:0/2007211223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeb4004d00 0x7fbeb4005180 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0b5870 tx=0x7fbeb40057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.129+0000 7fbed0ee4640 1 -- 172.21.15.47:0/2007211223 reap_dead start 2024-03-23T09:45:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.129+0000 7fbed2947640 1 -- 172.21.15.47:0/2007211223 shutdown_connections 2024-03-23T09:45:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.129+0000 7fbed2947640 1 -- 172.21.15.47:0/2007211223 >> 172.21.15.47:0/2007211223 conn(0x7fbecc065df0 msgr2=0x7fbecc066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.129+0000 7fbed2947640 1 -- 172.21.15.47:0/2007211223 shutdown_connections 2024-03-23T09:45:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.129+0000 7fbed2947640 1 -- 172.21.15.47:0/2007211223 wait complete. 2024-03-23T09:45:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.130+0000 7fbed2947640 1 Processor -- start 2024-03-23T09:45:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.130+0000 7fbed2947640 1 -- start start 2024-03-23T09:45:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbed2947640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbecc1f18f0 0x7fbecc156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbed2947640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbecc1f1cd0 0x7fbecc0c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbed2947640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbecc0ca3c0 0x7fbecc0ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbed2947640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0cad80 con 0x7fbecc1f18f0 2024-03-23T09:45:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbed2947640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0c7980 con 0x7fbecc0ca3c0 2024-03-23T09:45:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbed2947640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc1f2130 con 0x7fbecc1f1cd0 2024-03-23T09:45:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbecbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbecc1f18f0 0x7fbecc156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbecbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbecc1f18f0 0x7fbecc156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58434/0 (socket says 172.21.15.47:58434) 2024-03-23T09:45:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbecbfff640 1 -- 172.21.15.47:0/2553149700 learned_addr learned my addr 172.21.15.47:0/2553149700 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbecb7fe640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbecc1f1cd0 0x7fbecc0c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbed0ee4640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbecc0ca3c0 0x7fbecc0ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbecbfff640 1 -- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbecc1f1cd0 msgr2=0x7fbecc0c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbecbfff640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbecc1f1cd0 0x7fbecc0c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbecbfff640 1 -- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbecc0ca3c0 msgr2=0x7fbecc0ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:03.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbecbfff640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbecc0ca3c0 0x7fbecc0ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbecbfff640 1 -- 172.21.15.47:0/2553149700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc0c7bb0 con 0x7fbecc1f18f0 2024-03-23T09:45:03.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbecbfff640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbecc1f18f0 0x7fbecc156ac0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fbec0011960 tx=0x7fbec0011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:03.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbec8ff9640 1 -- 172.21.15.47:0/2553149700 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec000fa30 con 0x7fbecc1f18f0 2024-03-23T09:45:03.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbec8ff9640 1 -- 172.21.15.47:0/2553149700 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec000fb90 con 0x7fbecc1f18f0 2024-03-23T09:45:03.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbec8ff9640 1 -- 172.21.15.47:0/2553149700 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0014440 con 0x7fbecc1f18f0 2024-03-23T09:45:03.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.131+0000 7fbed2947640 1 -- 172.21.15.47:0/2553149700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc0c7e90 con 0x7fbecc1f18f0 2024-03-23T09:45:03.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.132+0000 7fbed2947640 1 -- 172.21.15.47:0/2553149700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc0c8450 con 0x7fbecc1f18f0 2024-03-23T09:45:03.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.133+0000 7fbed2947640 1 -- 172.21.15.47:0/2553149700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbecc072360 con 0x7fbecc1f18f0 2024-03-23T09:45:03.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.133+0000 7fbec8ff9640 1 -- 172.21.15.47:0/2553149700 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbec00145a0 con 0x7fbecc1f18f0 2024-03-23T09:45:03.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.134+0000 7fbec8ff9640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbeb806ac00 0x7fbeb806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.134+0000 7fbec8ff9640 1 -- 172.21.15.47:0/2553149700 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbec0089a80 con 0x7fbecc1f18f0 2024-03-23T09:45:03.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.134+0000 7fbecb7fe640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbeb806ac00 0x7fbeb806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.134+0000 7fbecb7fe640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbeb806ac00 0x7fbeb806d0c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbecc15dd80 tx=0x7fbebc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.138+0000 7fbec8ff9640 1 -- 172.21.15.47:0/2553149700 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbec0054c60 con 0x7fbecc1f18f0 2024-03-23T09:45:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.315+0000 7fbed2947640 1 -- 172.21.15.47:0/2553149700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbecc066be0 con 0x7fbecc1f18f0 2024-03-23T09:45:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.316+0000 7fbec8ff9640 1 -- 172.21.15.47:0/2553149700 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbec0022020 con 0x7fbecc1f18f0 2024-03-23T09:45:03.316 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:45:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.318+0000 7fbed2947640 1 -- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbeb806ac00 msgr2=0x7fbeb806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.318+0000 7fbed2947640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbeb806ac00 0x7fbeb806d0c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbecc15dd80 tx=0x7fbebc009190 comp rx=0 tx=0).stop 2024-03-23T09:45:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.318+0000 7fbed2947640 1 -- 172.21.15.47:0/2553149700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbecc1f18f0 msgr2=0x7fbecc156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.318+0000 7fbed2947640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbecc1f18f0 0x7fbecc156ac0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fbec0011960 tx=0x7fbec0011d60 comp rx=0 tx=0).stop 2024-03-23T09:45:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.318+0000 7fbed2947640 1 -- 172.21.15.47:0/2553149700 shutdown_connections 2024-03-23T09:45:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.318+0000 7fbed2947640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbeb806ac00 0x7fbeb806d0c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.318+0000 7fbed2947640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbecc0ca3c0 0x7fbecc0ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.318+0000 7fbed2947640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbecc1f1cd0 0x7fbecc0c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.318+0000 7fbed2947640 1 --2- 172.21.15.47:0/2553149700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbecc1f18f0 0x7fbecc156ac0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.318+0000 7fbed2947640 1 -- 172.21.15.47:0/2553149700 >> 172.21.15.47:0/2553149700 conn(0x7fbecc065df0 msgr2=0x7fbecc0d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.318+0000 7fbed2947640 1 -- 172.21.15.47:0/2553149700 shutdown_connections 2024-03-23T09:45:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.318+0000 7fbed2947640 1 -- 172.21.15.47:0/2553149700 wait complete. 2024-03-23T09:45:03.328 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:45:03.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.430+0000 7fd20087d640 1 Processor -- start 2024-03-23T09:45:03.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.430+0000 7fd20087d640 1 -- start start 2024-03-23T09:45:03.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.431+0000 7fd20087d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd1fc0cf210 0x7fd1fc0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.431+0000 7fd20087d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1fc0b5a90 0x7fd1fc0b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.431+0000 7fd20087d640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1fc0cfbd0 con 0x7fd1fc15cd00 2024-03-23T09:45:03.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.431+0000 7fd20087d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1fc15d0e0 con 0x7fd1fc0cf210 2024-03-23T09:45:03.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.431+0000 7fd20087d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc0d1540 con 0x7fd1fc0b5a90 2024-03-23T09:45:03.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.431+0000 7fd1f9d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd1fc0cf210 0x7fd1fc0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.431+0000 7fd1f9d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd1fc0cf210 0x7fd1fc0cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58438/0 (socket says 172.21.15.47:58438) 2024-03-23T09:45:03.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.431+0000 7fd1f9d9b640 1 -- 172.21.15.47:0/4267577336 learned_addr learned my addr 172.21.15.47:0/4267577336 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:03.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.431+0000 7fd1fad9d640 1 --2- 172.21.15.47:0/4267577336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1fc0b5a90 0x7fd1fc0b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.431+0000 7fd1f9d9b640 1 -- 172.21.15.47:0/4267577336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1fc0b5a90 msgr2=0x7fd1fc0b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:03.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.431+0000 7fd1f9d9b640 1 --2- 172.21.15.47:0/4267577336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1fc0b5a90 0x7fd1fc0b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.431+0000 7fd1f9d9b640 1 -- 172.21.15.47:0/4267577336 >> v1:172.21.15.175:6789/0 conn(0x7fd1fc15cd00 legacy=0x7fd1fc0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:03.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.431+0000 7fd1f9d9b640 1 -- 172.21.15.47:0/4267577336 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc0d1820 con 0x7fd1fc0cf210 2024-03-23T09:45:03.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.432+0000 7fd1f9d9b640 1 --2- 172.21.15.47:0/4267577336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1fc0cf210 0x7fd1fc0cf690 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009870 tx=0x7fd1e402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483cdecf879de72d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:03.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.432+0000 7fd1f959a640 1 -- 172.21.15.47:0/4267577336 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40317c0 con 0x7fd1fc0cf210 2024-03-23T09:45:03.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.432+0000 7fd1f959a640 1 -- 172.21.15.47:0/4267577336 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e4031920 con 0x7fd1fc0cf210 2024-03-23T09:45:03.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.432+0000 7fd1f959a640 1 -- 172.21.15.47:0/4267577336 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4031cb0 con 0x7fd1fc0cf210 2024-03-23T09:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.433+0000 7fd20087d640 1 -- 172.21.15.47:0/4267577336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1fc0cf210 msgr2=0x7fd1fc0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.433+0000 7fd20087d640 1 --2- 172.21.15.47:0/4267577336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1fc0cf210 0x7fd1fc0cf690 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009870 tx=0x7fd1e402b860 comp rx=0 tx=0).stop 2024-03-23T09:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.433+0000 7fd20087d640 1 -- 172.21.15.47:0/4267577336 shutdown_connections 2024-03-23T09:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.433+0000 7fd20087d640 1 --2- 172.21.15.47:0/4267577336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1fc0b5a90 0x7fd1fc0b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.433+0000 7fd20087d640 1 --2- 172.21.15.47:0/4267577336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1fc0cf210 0x7fd1fc0cf690 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.433+0000 7fd20087d640 1 -- 172.21.15.47:0/4267577336 >> 172.21.15.47:0/4267577336 conn(0x7fd1fc06b160 msgr2=0x7fd1fc06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.433+0000 7fd20087d640 1 -- 172.21.15.47:0/4267577336 shutdown_connections 2024-03-23T09:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.433+0000 7fd20087d640 1 -- 172.21.15.47:0/4267577336 wait complete. 2024-03-23T09:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.434+0000 7fd20087d640 1 Processor -- start 2024-03-23T09:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.434+0000 7fd20087d640 1 -- start start 2024-03-23T09:45:03.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd20087d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1fc0b5a90 0x7fd1fc154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd20087d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1fc0cf210 0x7fd1fc154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd20087d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1fc15cd00 0x7fd1fc0b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd20087d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc1550b0 con 0x7fd1fc0cf210 2024-03-23T09:45:03.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd20087d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc155350 con 0x7fd1fc15cd00 2024-03-23T09:45:03.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd20087d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc155630 con 0x7fd1fc0b5a90 2024-03-23T09:45:03.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd1f9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1fc0cf210 0x7fd1fc154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd1f9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1fc0cf210 0x7fd1fc154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58452/0 (socket says 172.21.15.47:58452) 2024-03-23T09:45:03.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd1f9d9b640 1 -- 172.21.15.47:0/1393379803 learned_addr learned my addr 172.21.15.47:0/1393379803 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:03.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd1fad9d640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1fc15cd00 0x7fd1fc0b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd1fa59c640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1fc0b5a90 0x7fd1fc154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd1f9d9b640 1 -- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1fc0b5a90 msgr2=0x7fd1fc154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:03.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd1f9d9b640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1fc0b5a90 0x7fd1fc154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd1f9d9b640 1 -- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1fc15cd00 msgr2=0x7fd1fc0b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd1f9d9b640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1fc15cd00 0x7fd1fc0b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd1f9d9b640 1 -- 172.21.15.47:0/1393379803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e40094d0 con 0x7fd1fc0cf210 2024-03-23T09:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.435+0000 7fd1fad9d640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1fc15cd00 0x7fd1fc0b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.436+0000 7fd1f9d9b640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1fc0cf210 0x7fd1fc154890 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4000c00 tx=0x7fd1e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.436+0000 7fd1f2ffd640 1 -- 172.21.15.47:0/1393379803 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4032860 con 0x7fd1fc0cf210 2024-03-23T09:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.436+0000 7fd1f2ffd640 1 -- 172.21.15.47:0/1393379803 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e40329c0 con 0x7fd1fc0cf210 2024-03-23T09:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.436+0000 7fd20087d640 1 -- 172.21.15.47:0/1393379803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1fc155910 con 0x7fd1fc0cf210 2024-03-23T09:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.436+0000 7fd1f2ffd640 1 -- 172.21.15.47:0/1393379803 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4032d70 con 0x7fd1fc0cf210 2024-03-23T09:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.436+0000 7fd20087d640 1 -- 172.21.15.47:0/1393379803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1fc1522f0 con 0x7fd1fc0cf210 2024-03-23T09:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.437+0000 7fd20087d640 1 -- 172.21.15.47:0/1393379803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1fc072360 con 0x7fd1fc0cf210 2024-03-23T09:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.437+0000 7fd1f2ffd640 1 -- 172.21.15.47:0/1393379803 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd1e404e450 con 0x7fd1fc0cf210 2024-03-23T09:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.438+0000 7fd1f2ffd640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd1dc06ac30 0x7fd1dc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:03.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.438+0000 7fd1f2ffd640 1 -- 172.21.15.47:0/1393379803 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1e40ab600 con 0x7fd1fc0cf210 2024-03-23T09:45:03.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.438+0000 7fd1fa59c640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd1dc06ac30 0x7fd1dc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:03.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.438+0000 7fd1fa59c640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd1dc06ac30 0x7fd1dc06d0f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4006fd0 tx=0x7fd1f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:03.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.442+0000 7fd1f2ffd640 1 -- 172.21.15.47:0/1393379803 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd1e4076750 con 0x7fd1fc0cf210 2024-03-23T09:45:03.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.635+0000 7fd20087d640 1 -- 172.21.15.47:0/1393379803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd1fc066be0 con 0x7fd1fc0cf210 2024-03-23T09:45:03.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.635+0000 7fd1f2ffd640 1 -- 172.21.15.47:0/1393379803 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd1e4079d70 con 0x7fd1fc0cf210 2024-03-23T09:45:03.635 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:45:03.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.637+0000 7fd20087d640 1 -- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd1dc06ac30 msgr2=0x7fd1dc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:03.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.637+0000 7fd20087d640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd1dc06ac30 0x7fd1dc06d0f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4006fd0 tx=0x7fd1f4008040 comp rx=0 tx=0).stop 2024-03-23T09:45:03.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.637+0000 7fd20087d640 1 -- 172.21.15.47:0/1393379803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1fc0cf210 msgr2=0x7fd1fc154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:03.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.637+0000 7fd20087d640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1fc0cf210 0x7fd1fc154890 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4000c00 tx=0x7fd1e40046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.637+0000 7fd20087d640 1 -- 172.21.15.47:0/1393379803 shutdown_connections 2024-03-23T09:45:03.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.637+0000 7fd20087d640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd1dc06ac30 0x7fd1dc06d0f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.637+0000 7fd20087d640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1fc15cd00 0x7fd1fc0b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.637+0000 7fd20087d640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1fc0cf210 0x7fd1fc154890 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.637+0000 7fd20087d640 1 --2- 172.21.15.47:0/1393379803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1fc0b5a90 0x7fd1fc154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:03.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.637+0000 7fd20087d640 1 -- 172.21.15.47:0/1393379803 >> 172.21.15.47:0/1393379803 conn(0x7fd1fc06b160 msgr2=0x7fd1fc0b7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:03.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.637+0000 7fd20087d640 1 -- 172.21.15.47:0/1393379803 shutdown_connections 2024-03-23T09:45:03.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:03.637+0000 7fd20087d640 1 -- 172.21.15.47:0/1393379803 wait complete. 2024-03-23T09:45:03.648 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:45:03.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:03.648 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:03.673 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[57909]: fuse finished with error 0 and tester_r 0 2024-03-23T09:45:03.674 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:45:03.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:03.675 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:45:03.830 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:45:03.830 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:45:03.849 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:45:03.849 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:45:03.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:03.850 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-23T09:45:03.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:45:03.907 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-23T09:45:03.907 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:45:03.907 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:45:03.924 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:45:03.924 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:45:03.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:03.924 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-23T09:45:03.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:45:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-23T09:45:03.979 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:45:03.979 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:45:03.996 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:45:03.996 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:45:03.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:03.996 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-23T09:45:04.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:45:04.053 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-23T09:45:04.053 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:45:04.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.155+0000 7f809bed5640 1 Processor -- start 2024-03-23T09:45:04.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.155+0000 7f809bed5640 1 -- start start 2024-03-23T09:45:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f809bed5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f809415cb40 0x7f8094160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f809bed5640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f80940b8300 0x7f80940cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f809bed5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094160cb0 0x7f80940cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f809bed5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80940b5a10 con 0x7f809415cb40 2024-03-23T09:45:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f809bed5640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80940d0ee0 con 0x7f80940b8300 2024-03-23T09:45:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f809bed5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80941616f0 con 0x7f8094160cb0 2024-03-23T09:45:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f8099c71640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f809415cb40 0x7f8094160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f8099c71640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f809415cb40 0x7f8094160770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58466/0 (socket says 172.21.15.47:58466) 2024-03-23T09:45:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f8099c71640 1 -- 172.21.15.47:0/670874375 learned_addr learned my addr 172.21.15.47:0/670874375 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f8099470640 1 --2- 172.21.15.47:0/670874375 >> v2:172.21.15.175:3300/0 conn(0x7f80940b8300 0x7f80940cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f809a472640 1 --2- 172.21.15.47:0/670874375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094160cb0 0x7f80940cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f8099c71640 1 -- 172.21.15.47:0/670874375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094160cb0 msgr2=0x7f80940cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f8099c71640 1 --2- 172.21.15.47:0/670874375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094160cb0 0x7f80940cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f8099c71640 1 -- 172.21.15.47:0/670874375 >> v2:172.21.15.175:3300/0 conn(0x7f80940b8300 msgr2=0x7f80940cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f8099c71640 1 --2- 172.21.15.47:0/670874375 >> v2:172.21.15.175:3300/0 conn(0x7f80940b8300 0x7f80940cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.156+0000 7f8099c71640 1 -- 172.21.15.47:0/670874375 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094161130 con 0x7f809415cb40 2024-03-23T09:45:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.157+0000 7f8099c71640 1 --2- 172.21.15.47:0/670874375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f809415cb40 0x7f8094160770 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8090009870 tx=0x7f809002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29763243780245 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.157+0000 7f8098c6f640 1 -- 172.21.15.47:0/670874375 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8090031640 con 0x7f809415cb40 2024-03-23T09:45:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.157+0000 7f8098c6f640 1 -- 172.21.15.47:0/670874375 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80900317a0 con 0x7f809415cb40 2024-03-23T09:45:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.157+0000 7f8098c6f640 1 -- 172.21.15.47:0/670874375 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8090031b30 con 0x7f809415cb40 2024-03-23T09:45:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.158+0000 7f809bed5640 1 -- 172.21.15.47:0/670874375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f809415cb40 msgr2=0x7f8094160770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.158+0000 7f809bed5640 1 --2- 172.21.15.47:0/670874375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f809415cb40 0x7f8094160770 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8090009870 tx=0x7f809002b8f0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.158+0000 7f809bed5640 1 -- 172.21.15.47:0/670874375 shutdown_connections 2024-03-23T09:45:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.158+0000 7f809bed5640 1 --2- 172.21.15.47:0/670874375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094160cb0 0x7f80940cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.158+0000 7f809bed5640 1 --2- 172.21.15.47:0/670874375 >> v2:172.21.15.175:3300/0 conn(0x7f80940b8300 0x7f80940cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.158+0000 7f809bed5640 1 --2- 172.21.15.47:0/670874375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f809415cb40 0x7f8094160770 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.158+0000 7f809bed5640 1 -- 172.21.15.47:0/670874375 >> 172.21.15.47:0/670874375 conn(0x7f809406b160 msgr2=0x7f809406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.158+0000 7f809bed5640 1 -- 172.21.15.47:0/670874375 shutdown_connections 2024-03-23T09:45:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.158+0000 7f809bed5640 1 -- 172.21.15.47:0/670874375 wait complete. 2024-03-23T09:45:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.159+0000 7f809bed5640 1 Processor -- start 2024-03-23T09:45:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.159+0000 7f809bed5640 1 -- start start 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f809bed5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80940b8300 0x7f809415d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f809bed5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f809415cb40 0x7f809415fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f809bed5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8094160cb0 0x7f809415ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f809bed5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809415d760 con 0x7f809415cb40 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f809bed5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809415e580 con 0x7f8094160cb0 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f809bed5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809415fff0 con 0x7f80940b8300 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f8099470640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f809415cb40 0x7f809415fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f8099470640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f809415cb40 0x7f809415fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58468/0 (socket says 172.21.15.47:58468) 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f8099470640 1 -- 172.21.15.47:0/3165001446 learned_addr learned my addr 172.21.15.47:0/3165001446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f8099c71640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80940b8300 0x7f809415d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f809a472640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8094160cb0 0x7f809415ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f8099470640 1 -- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80940b8300 msgr2=0x7f809415d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f8099470640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80940b8300 0x7f809415d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f8099470640 1 -- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8094160cb0 msgr2=0x7f809415ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f8099470640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8094160cb0 0x7f809415ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f8099470640 1 -- 172.21.15.47:0/3165001446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80900094d0 con 0x7f809415cb40 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f8099470640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f809415cb40 0x7f809415fab0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f808400d800 tx=0x7f808400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f808a7fc640 1 -- 172.21.15.47:0/3165001446 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80840096a0 con 0x7f809415cb40 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f808a7fc640 1 -- 172.21.15.47:0/3165001446 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8084009800 con 0x7f809415cb40 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f809bed5640 1 -- 172.21.15.47:0/3165001446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809415d970 con 0x7f809415cb40 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f808a7fc640 1 -- 172.21.15.47:0/3165001446 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084009b90 con 0x7f809415cb40 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.160+0000 7f809bed5640 1 -- 172.21.15.47:0/3165001446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809415df30 con 0x7f809415cb40 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.161+0000 7f809bed5640 1 -- 172.21.15.47:0/3165001446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8094072360 con 0x7f809415cb40 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.162+0000 7f808a7fc640 1 -- 172.21.15.47:0/3165001446 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8084009cf0 con 0x7f809415cb40 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.162+0000 7f808a7fc640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f807806ac00 0x7f807806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.162+0000 7f808a7fc640 1 -- 172.21.15.47:0/3165001446 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80840896d0 con 0x7f809415cb40 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.162+0000 7f8099c71640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f807806ac00 0x7f807806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.163+0000 7f8099c71640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f807806ac00 0x7f807806d0c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8090036000 tx=0x7f809003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:04.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.166+0000 7f808a7fc640 1 -- 172.21.15.47:0/3165001446 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8084054960 con 0x7f809415cb40 2024-03-23T09:45:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.380+0000 7f809bed5640 1 -- 172.21.15.47:0/3165001446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f809415e6f0 con 0x7f809415cb40 2024-03-23T09:45:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.380+0000 7f808a7fc640 1 -- 172.21.15.47:0/3165001446 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f8084057f80 con 0x7f809415cb40 2024-03-23T09:45:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:04.381 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/375513546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":375513546},{"type":"v1","addr":"172.21.15.175:6837","nonce":375513546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/3650756747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3650756747},{"type":"v1","addr":"172.21.15.47:6833","nonce":3650756747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2893805543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2893805543},{"type":"v1","addr":"172.21.15.47:6835","nonce":2893805543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:43:47.389279+0000","modified":"2024-03-23T09:44:35.825401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.175:6836/2498456351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2498456351},{"type":"v1","addr":"172.21.15.175:6836","nonce":2498456351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-23T09:45:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2024-03-23T09:45:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.382+0000 7f809bed5640 1 -- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f807806ac00 msgr2=0x7f807806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.382+0000 7f809bed5640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f807806ac00 0x7f807806d0c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8090036000 tx=0x7f809003c040 comp rx=0 tx=0).stop 2024-03-23T09:45:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.382+0000 7f809bed5640 1 -- 172.21.15.47:0/3165001446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f809415cb40 msgr2=0x7f809415fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.382+0000 7f809bed5640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f809415cb40 0x7f809415fab0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f808400d800 tx=0x7f808400dcc0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.382+0000 7f809bed5640 1 -- 172.21.15.47:0/3165001446 shutdown_connections 2024-03-23T09:45:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.382+0000 7f809bed5640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f807806ac00 0x7f807806d0c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.382+0000 7f809bed5640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8094160cb0 0x7f809415ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.382+0000 7f809bed5640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f809415cb40 0x7f809415fab0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.382+0000 7f809bed5640 1 --2- 172.21.15.47:0/3165001446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80940b8300 0x7f809415d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.382+0000 7f809bed5640 1 -- 172.21.15.47:0/3165001446 >> 172.21.15.47:0/3165001446 conn(0x7f809406b160 msgr2=0x7f80940c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:04.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.382+0000 7f809bed5640 1 -- 172.21.15.47:0/3165001446 shutdown_connections 2024-03-23T09:45:04.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.382+0000 7f809bed5640 1 -- 172.21.15.47:0/3165001446 wait complete. 2024-03-23T09:45:04.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:45:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.494+0000 7f3286ad5640 1 Processor -- start 2024-03-23T09:45:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.494+0000 7f3286ad5640 1 -- start start 2024-03-23T09:45:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f3286ad5640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f328015cd00 0x7f32800d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f3286ad5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cf920 0x7f32800b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f3286ad5640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328015d0e0 con 0x7f32800d1020 2024-03-23T09:45:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f3286ad5640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32800d1540 con 0x7f328015cd00 2024-03-23T09:45:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f3286ad5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32800d1820 con 0x7f32800cf920 2024-03-23T09:45:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f327ffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f32800d1020 0x7f32800cf210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40386/0 (socket says 172.21.15.47:40386) 2024-03-23T09:45:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f327ffff640 1 -- 172.21.15.47:0/1898730204 learned_addr learned my addr 172.21.15.47:0/1898730204 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f3285072640 1 --2- 172.21.15.47:0/1898730204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cf920 0x7f32800b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f3284871640 1 --2- 172.21.15.47:0/1898730204 >> v2:172.21.15.175:3300/0 conn(0x7f328015cd00 0x7f32800d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480793390 0 0) 0x7f328015d0e0 con 0x7f32800d1020 2024-03-23T09:45:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3270003520 con 0x7f32800d1020 2024-03-23T09:45:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 712698564 0 0) 0x7f3270003520 con 0x7f32800d1020 2024-03-23T09:45:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cf920 msgr2=0x7f32800b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f327f7fe640 1 --2- 172.21.15.47:0/1898730204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cf920 0x7f32800b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 >> v2:172.21.15.175:3300/0 conn(0x7f328015cd00 msgr2=0x7f32800d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f327f7fe640 1 --2- 172.21.15.47:0/1898730204 >> v2:172.21.15.175:3300/0 conn(0x7f328015cd00 0x7f32800d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32800d1b00 con 0x7f32800d1020 2024-03-23T09:45:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.495+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f326c001670 con 0x7f32800d1020 2024-03-23T09:45:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 >> v1:172.21.15.47:6789/0 conn(0x7f32800d1020 legacy=0x7f32800cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f327f7fe640 1 --2- 172.21.15.47:0/1898730204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3270003970 0x7f3270003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f327f7fe640 1 --2- 172.21.15.47:0/1898730204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cf920 0x7f32700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f327f7fe640 1 --2- 172.21.15.47:0/1898730204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3270004d00 0x7f3270005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32800d1540 con 0x7f3270004d00 2024-03-23T09:45:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32800d1820 con 0x7f3270003970 2024-03-23T09:45:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327000b980 con 0x7f32800cf920 2024-03-23T09:45:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f3285072640 1 --2- 172.21.15.47:0/1898730204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3270004d00 0x7f3270005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f3284871640 1 --2- 172.21.15.47:0/1898730204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3270003970 0x7f3270003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f327ffff640 1 --2- 172.21.15.47:0/1898730204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cf920 0x7f32700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f3285072640 1 -- 172.21.15.47:0/1898730204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cf920 msgr2=0x7f32700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f3285072640 1 --2- 172.21.15.47:0/1898730204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cf920 0x7f32700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f3285072640 1 -- 172.21.15.47:0/1898730204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3270003970 msgr2=0x7f3270003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f3285072640 1 --2- 172.21.15.47:0/1898730204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3270003970 0x7f3270003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.496+0000 7f3285072640 1 -- 172.21.15.47:0/1898730204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3270003520 con 0x7f3270004d00 2024-03-23T09:45:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.497+0000 7f3285072640 1 --2- 172.21.15.47:0/1898730204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3270004d00 0x7f3270005180 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f32800cfd30 tx=0x7f32700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2f3dcdfe4c1a4f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.497+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327802cb90 con 0x7f3270004d00 2024-03-23T09:45:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.497+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327800f040 con 0x7f3270004d00 2024-03-23T09:45:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.497+0000 7f327f7fe640 1 -- 172.21.15.47:0/1898730204 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278034440 con 0x7f3270004d00 2024-03-23T09:45:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.497+0000 7f3286ad5640 1 -- 172.21.15.47:0/1898730204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3270004d00 msgr2=0x7f3270005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.497+0000 7f3286ad5640 1 --2- 172.21.15.47:0/1898730204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3270004d00 0x7f3270005180 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f32800cfd30 tx=0x7f32700057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.497+0000 7f3285072640 1 -- 172.21.15.47:0/1898730204 reap_dead start 2024-03-23T09:45:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.497+0000 7f3286ad5640 1 -- 172.21.15.47:0/1898730204 shutdown_connections 2024-03-23T09:45:04.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.497+0000 7f3286ad5640 1 -- 172.21.15.47:0/1898730204 >> 172.21.15.47:0/1898730204 conn(0x7f328006b160 msgr2=0x7f328006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:04.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.498+0000 7f3286ad5640 1 -- 172.21.15.47:0/1898730204 shutdown_connections 2024-03-23T09:45:04.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.498+0000 7f3286ad5640 1 -- 172.21.15.47:0/1898730204 wait complete. 2024-03-23T09:45:04.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.499+0000 7f3286ad5640 1 Processor -- start 2024-03-23T09:45:04.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.499+0000 7f3286ad5640 1 -- start start 2024-03-23T09:45:04.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.499+0000 7f3286ad5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3270004d00 0x7f3280150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.499+0000 7f3286ad5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32800cc1f0 0x7f328015d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.499+0000 7f3286ad5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cc650 0x7f328015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f3286ad5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32800cb8a0 con 0x7f32800cc1f0 2024-03-23T09:45:04.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f3286ad5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32800cac20 con 0x7f3270004d00 2024-03-23T09:45:04.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f3286ad5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32800caf00 con 0x7f32800cc650 2024-03-23T09:45:04.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f327ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32800cc1f0 0x7f328015d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f327ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32800cc1f0 0x7f328015d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58494/0 (socket says 172.21.15.47:58494) 2024-03-23T09:45:04.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f327ffff640 1 -- 172.21.15.47:0/2144923731 learned_addr learned my addr 172.21.15.47:0/2144923731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:04.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f3285072640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cc650 0x7f328015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f327ffff640 1 -- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cc650 msgr2=0x7f328015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f3284871640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3270004d00 0x7f3280150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f327ffff640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cc650 0x7f328015d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f327ffff640 1 -- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3270004d00 msgr2=0x7f3280150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f327ffff640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3270004d00 0x7f3280150f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f327ffff640 1 -- 172.21.15.47:0/2144923731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32800cb1e0 con 0x7f32800cc1f0 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f3285072640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cc650 0x7f328015d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f327ffff640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32800cc1f0 0x7f328015d290 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f326c007310 tx=0x7f326c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f327d7fa640 1 -- 172.21.15.47:0/2144923731 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c007c10 con 0x7f32800cc1f0 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f327d7fa640 1 -- 172.21.15.47:0/2144923731 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c00f940 con 0x7f32800cc1f0 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f3286ad5640 1 -- 172.21.15.47:0/2144923731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32800cb4c0 con 0x7f32800cc1f0 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f327d7fa640 1 -- 172.21.15.47:0/2144923731 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c00fcf0 con 0x7f32800cc1f0 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.500+0000 7f3286ad5640 1 -- 172.21.15.47:0/2144923731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328015b320 con 0x7f32800cc1f0 2024-03-23T09:45:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.501+0000 7f3286ad5640 1 -- 172.21.15.47:0/2144923731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3280072360 con 0x7f32800cc1f0 2024-03-23T09:45:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.502+0000 7f327d7fa640 1 -- 172.21.15.47:0/2144923731 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f326c01d460 con 0x7f32800cc1f0 2024-03-23T09:45:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.502+0000 7f327d7fa640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f326406ac00 0x7f326406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.502+0000 7f327d7fa640 1 -- 172.21.15.47:0/2144923731 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f326c0895f0 con 0x7f32800cc1f0 2024-03-23T09:45:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.502+0000 7f3284871640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f326406ac00 0x7f326406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.503+0000 7f3284871640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f326406ac00 0x7f326406d0c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3274009a60 tx=0x7f3274008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.506+0000 7f327d7fa640 1 -- 172.21.15.47:0/2144923731 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f326c054730 con 0x7f32800cc1f0 2024-03-23T09:45:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.688+0000 7f3286ad5640 1 -- 172.21.15.47:0/2144923731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3280066be0 con 0x7f32800cc1f0 2024-03-23T09:45:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.689+0000 7f327d7fa640 1 -- 172.21.15.47:0/2144923731 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16097 (secure 0 0 0) 0x7f326c057d50 con 0x7f32800cc1f0 2024-03-23T09:45:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:04.689 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":58,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:43:48.390657+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:43:45.646160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-23T09:43:46.351634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:45:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.691+0000 7f3286ad5640 1 -- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f326406ac00 msgr2=0x7f326406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.691+0000 7f3286ad5640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f326406ac00 0x7f326406d0c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3274009a60 tx=0x7f3274008040 comp rx=0 tx=0).stop 2024-03-23T09:45:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.691+0000 7f3286ad5640 1 -- 172.21.15.47:0/2144923731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32800cc1f0 msgr2=0x7f328015d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.691+0000 7f3286ad5640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32800cc1f0 0x7f328015d290 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f326c007310 tx=0x7f326c0077d0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.691+0000 7f3286ad5640 1 -- 172.21.15.47:0/2144923731 shutdown_connections 2024-03-23T09:45:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.691+0000 7f3286ad5640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f326406ac00 0x7f326406d0c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.691+0000 7f3286ad5640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32800cc650 0x7f328015d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.691+0000 7f3286ad5640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32800cc1f0 0x7f328015d290 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.691+0000 7f3286ad5640 1 --2- 172.21.15.47:0/2144923731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3270004d00 0x7f3280150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.691+0000 7f3286ad5640 1 -- 172.21.15.47:0/2144923731 >> 172.21.15.47:0/2144923731 conn(0x7f328006b160 msgr2=0x7f32800b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.691+0000 7f3286ad5640 1 -- 172.21.15.47:0/2144923731 shutdown_connections 2024-03-23T09:45:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.691+0000 7f3286ad5640 1 -- 172.21.15.47:0/2144923731 wait complete. 2024-03-23T09:45:04.702 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:43:45.646160+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:43:46.351634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:45:04.702 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:45:04.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.805+0000 7f1b1bd47640 1 Processor -- start 2024-03-23T09:45:04.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.805+0000 7f1b1bd47640 1 -- start start 2024-03-23T09:45:04.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.806+0000 7f1b1bd47640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b140cd7f0 0x7f1b140ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.806+0000 7f1b1bd47640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b140cb270 con 0x7f1b140ce950 2024-03-23T09:45:04.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.806+0000 7f1b1bd47640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b140cb510 con 0x7f1b1415ab80 2024-03-23T09:45:04.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.806+0000 7f1b1bd47640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b140c4240 con 0x7f1b140cd7f0 2024-03-23T09:45:04.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.806+0000 7f1b1a2e4640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1b140ce950 0x7f1b140cbcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38846/0 (socket says 172.21.15.47:38846) 2024-03-23T09:45:04.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.806+0000 7f1b1a2e4640 1 -- 172.21.15.47:0/3899797466 learned_addr learned my addr 172.21.15.47:0/3899797466 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:04.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.807+0000 7f1b192e2640 1 --2- 172.21.15.47:0/3899797466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b140cd7f0 0x7f1b140ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.807+0000 7f1b18ae1640 1 -- 172.21.15.47:0/3899797466 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048274374 0 0) 0x7f1b140cb270 con 0x7f1b140ce950 2024-03-23T09:45:04.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.807+0000 7f1b18ae1640 1 -- 172.21.15.47:0/3899797466 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1afc003520 con 0x7f1b140ce950 2024-03-23T09:45:04.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.807+0000 7f1b18ae1640 1 -- 172.21.15.47:0/3899797466 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964848654 0 0) 0x7f1b140cb510 con 0x7f1b1415ab80 2024-03-23T09:45:04.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.807+0000 7f1b18ae1640 1 -- 172.21.15.47:0/3899797466 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b140cb270 con 0x7f1b1415ab80 2024-03-23T09:45:04.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.807+0000 7f1b18ae1640 1 -- 172.21.15.47:0/3899797466 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2719265904 0 0) 0x7f1afc003520 con 0x7f1b140ce950 2024-03-23T09:45:04.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.807+0000 7f1b192e2640 1 -- 172.21.15.47:0/3899797466 >> v1:172.21.15.175:6789/0 conn(0x7f1b1415ab80 legacy=0x7f1b140cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.807+0000 7f1b192e2640 1 -- 172.21.15.47:0/3899797466 >> v1:172.21.15.47:6789/0 conn(0x7f1b140ce950 legacy=0x7f1b140cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.807+0000 7f1b192e2640 1 -- 172.21.15.47:0/3899797466 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b140cf740 con 0x7f1b140cd7f0 2024-03-23T09:45:04.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.808+0000 7f1b192e2640 1 --2- 172.21.15.47:0/3899797466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b140cd7f0 0x7f1b140ce410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1b080097d0 tx=0x7f1b0802b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1a873edce07b458 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:04.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.808+0000 7f1b18ae1640 1 -- 172.21.15.47:0/3899797466 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c001670 con 0x7f1b140cd7f0 2024-03-23T09:45:04.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.808+0000 7f1b18ae1640 1 -- 172.21.15.47:0/3899797466 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0802e050 con 0x7f1b140cd7f0 2024-03-23T09:45:04.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.808+0000 7f1b18ae1640 1 -- 172.21.15.47:0/3899797466 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08030710 con 0x7f1b140cd7f0 2024-03-23T09:45:04.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.808+0000 7f1b1bd47640 1 -- 172.21.15.47:0/3899797466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b140cd7f0 msgr2=0x7f1b140ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.808+0000 7f1b1bd47640 1 --2- 172.21.15.47:0/3899797466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b140cd7f0 0x7f1b140ce410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1b080097d0 tx=0x7f1b0802b770 comp rx=0 tx=0).stop 2024-03-23T09:45:04.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.808+0000 7f1b1bd47640 1 -- 172.21.15.47:0/3899797466 shutdown_connections 2024-03-23T09:45:04.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.808+0000 7f1b1bd47640 1 --2- 172.21.15.47:0/3899797466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b140cd7f0 0x7f1b140ce410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.808+0000 7f1b1bd47640 1 -- 172.21.15.47:0/3899797466 >> 172.21.15.47:0/3899797466 conn(0x7f1b140d06f0 msgr2=0x7f1b140d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:04.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.809+0000 7f1b1bd47640 1 -- 172.21.15.47:0/3899797466 shutdown_connections 2024-03-23T09:45:04.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.809+0000 7f1b1bd47640 1 -- 172.21.15.47:0/3899797466 wait complete. 2024-03-23T09:45:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.810+0000 7f1b1bd47640 1 Processor -- start 2024-03-23T09:45:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.810+0000 7f1b1bd47640 1 -- start start 2024-03-23T09:45:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.810+0000 7f1b1bd47640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140cd7f0 0x7f1b140b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.810+0000 7f1b1bd47640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b140ce950 0x7f1b14159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.810+0000 7f1b1bd47640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b1415ab80 0x7f1b1414eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.810+0000 7f1b1bd47640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b14155da0 con 0x7f1b140cd7f0 2024-03-23T09:45:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.810+0000 7f1b1bd47640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b14156080 con 0x7f1b140ce950 2024-03-23T09:45:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.810+0000 7f1b1bd47640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b14156360 con 0x7f1b1415ab80 2024-03-23T09:45:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.810+0000 7f1b19ae3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140cd7f0 0x7f1b140b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.810+0000 7f1b19ae3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140cd7f0 0x7f1b140b5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45248/0 (socket says 172.21.15.47:45248) 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.810+0000 7f1b19ae3640 1 -- 172.21.15.47:0/1128477263 learned_addr learned my addr 172.21.15.47:0/1128477263 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.811+0000 7f1b192e2640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b140ce950 0x7f1b14159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.811+0000 7f1b1a2e4640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b1415ab80 0x7f1b1414eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.811+0000 7f1b19ae3640 1 -- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b1415ab80 msgr2=0x7f1b1414eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.811+0000 7f1b19ae3640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b1415ab80 0x7f1b1414eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.811+0000 7f1b19ae3640 1 -- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b140ce950 msgr2=0x7f1b14159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.811+0000 7f1b19ae3640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b140ce950 0x7f1b14159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.811+0000 7f1b19ae3640 1 -- 172.21.15.47:0/1128477263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b08009400 con 0x7f1b140cd7f0 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.811+0000 7f1b19ae3640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140cd7f0 0x7f1b140b5dd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400c7a0 tx=0x7f1b0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.811+0000 7f1b127fc640 1 -- 172.21.15.47:0/1128477263 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04003b70 con 0x7f1b140cd7f0 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.811+0000 7f1b127fc640 1 -- 172.21.15.47:0/1128477263 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b04003cd0 con 0x7f1b140cd7f0 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.811+0000 7f1b1bd47640 1 -- 172.21.15.47:0/1128477263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b14156640 con 0x7f1b140cd7f0 2024-03-23T09:45:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.811+0000 7f1b127fc640 1 -- 172.21.15.47:0/1128477263 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0400aa90 con 0x7f1b140cd7f0 2024-03-23T09:45:04.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.811+0000 7f1b1bd47640 1 -- 172.21.15.47:0/1128477263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1414f0b0 con 0x7f1b140cd7f0 2024-03-23T09:45:04.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.812+0000 7f1b1bd47640 1 -- 172.21.15.47:0/1128477263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b14156b60 con 0x7f1b140cd7f0 2024-03-23T09:45:04.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.813+0000 7f1b127fc640 1 -- 172.21.15.47:0/1128477263 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1b0400abf0 con 0x7f1b140cd7f0 2024-03-23T09:45:04.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.813+0000 7f1b127fc640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b0006ac00 0x7f1b0006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:04.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.813+0000 7f1b127fc640 1 -- 172.21.15.47:0/1128477263 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b040891d0 con 0x7f1b140cd7f0 2024-03-23T09:45:04.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.813+0000 7f1b192e2640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b0006ac00 0x7f1b0006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:04.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.814+0000 7f1b192e2640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b0006ac00 0x7f1b0006d0c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1b08034000 tx=0x7f1b08008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:04.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.817+0000 7f1b127fc640 1 -- 172.21.15.47:0/1128477263 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1b14156b60 con 0x7f1b140cd7f0 2024-03-23T09:45:04.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.994+0000 7f1b1bd47640 1 -- 172.21.15.47:0/1128477263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b14156b60 con 0x7f1b140cd7f0 2024-03-23T09:45:04.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.995+0000 7f1b127fc640 1 -- 172.21.15.47:0/1128477263 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b14156b60 con 0x7f1b140cd7f0 2024-03-23T09:45:04.995 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:45:04.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.997+0000 7f1b1bd47640 1 -- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b0006ac00 msgr2=0x7f1b0006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.997+0000 7f1b1bd47640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b0006ac00 0x7f1b0006d0c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1b08034000 tx=0x7f1b08008030 comp rx=0 tx=0).stop 2024-03-23T09:45:04.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.997+0000 7f1b1bd47640 1 -- 172.21.15.47:0/1128477263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140cd7f0 msgr2=0x7f1b140b5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:04.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.997+0000 7f1b1bd47640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140cd7f0 0x7f1b140b5dd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400c7a0 tx=0x7f1b0400cc60 comp rx=0 tx=0).stop 2024-03-23T09:45:04.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.997+0000 7f1b1bd47640 1 -- 172.21.15.47:0/1128477263 shutdown_connections 2024-03-23T09:45:04.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.997+0000 7f1b1bd47640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1b0006ac00 0x7f1b0006d0c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.997+0000 7f1b1bd47640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b1415ab80 0x7f1b1414eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.997+0000 7f1b1bd47640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b140ce950 0x7f1b14159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.997+0000 7f1b1bd47640 1 --2- 172.21.15.47:0/1128477263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140cd7f0 0x7f1b140b5dd0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:04.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.997+0000 7f1b1bd47640 1 -- 172.21.15.47:0/1128477263 >> 172.21.15.47:0/1128477263 conn(0x7f1b140d06f0 msgr2=0x7f1b140ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:04.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.997+0000 7f1b1bd47640 1 -- 172.21.15.47:0/1128477263 shutdown_connections 2024-03-23T09:45:04.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:04.997+0000 7f1b1bd47640 1 -- 172.21.15.47:0/1128477263 wait complete. 2024-03-23T09:45:05.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:45:05.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.112+0000 7ff4cc573640 1 Processor -- start 2024-03-23T09:45:05.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.112+0000 7ff4cc573640 1 -- start start 2024-03-23T09:45:05.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.112+0000 7ff4cc573640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c40b8350 0x7ff4c4160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.112+0000 7ff4cc573640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c4161630 con 0x7ff4c40cde10 2024-03-23T09:45:05.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.112+0000 7ff4cc573640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c40d1030 con 0x7ff4c415c990 2024-03-23T09:45:05.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.112+0000 7ff4cc573640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40cfe10 con 0x7ff4c40b8350 2024-03-23T09:45:05.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c9b0e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff4c40cde10 0x7ff4c40d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38850/0 (socket says 172.21.15.47:38850) 2024-03-23T09:45:05.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c9b0e640 1 -- 172.21.15.47:0/742723795 learned_addr learned my addr 172.21.15.47:0/742723795 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:05.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4cab10640 1 --2- 172.21.15.47:0/742723795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c40b8350 0x7ff4c4160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610855055 0 0) 0x7ff4c4161630 con 0x7ff4c40cde10 2024-03-23T09:45:05.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4b0003520 con 0x7ff4c40cde10 2024-03-23T09:45:05.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598648121 0 0) 0x7ff4c40d1030 con 0x7ff4c415c990 2024-03-23T09:45:05.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4c4161630 con 0x7ff4c415c990 2024-03-23T09:45:05.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 831236598 0 0) 0x7ff4b0003520 con 0x7ff4c40cde10 2024-03-23T09:45:05.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c40b8350 msgr2=0x7ff4c4160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:05.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 --2- 172.21.15.47:0/742723795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c40b8350 0x7ff4c4160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 >> v1:172.21.15.175:6789/0 conn(0x7ff4c415c990 legacy=0x7ff4c40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c40b5b60 con 0x7ff4c40cde10 2024-03-23T09:45:05.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7ff4b4001670 con 0x7ff4c40cde10 2024-03-23T09:45:05.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 >> v1:172.21.15.47:6789/0 conn(0x7ff4c40cde10 legacy=0x7ff4c40d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 --2- 172.21.15.47:0/742723795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c40b8350 0x7ff4b0003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 --2- 172.21.15.47:0/742723795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b00040e0 0x7ff4b00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 --2- 172.21.15.47:0/742723795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4b0004d00 0x7ff4b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40cfe10 con 0x7ff4b0004d00 2024-03-23T09:45:05.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b000bb40 con 0x7ff4b00040e0 2024-03-23T09:45:05.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b000bd00 con 0x7ff4c40b8350 2024-03-23T09:45:05.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4cab10640 1 --2- 172.21.15.47:0/742723795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4b0004d00 0x7ff4b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4ca30f640 1 --2- 172.21.15.47:0/742723795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c40b8350 0x7ff4b0003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.113+0000 7ff4c9b0e640 1 --2- 172.21.15.47:0/742723795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b00040e0 0x7ff4b00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.114+0000 7ff4cab10640 1 -- 172.21.15.47:0/742723795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c40b8350 msgr2=0x7ff4b0003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:05.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.114+0000 7ff4cab10640 1 --2- 172.21.15.47:0/742723795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c40b8350 0x7ff4b0003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.114+0000 7ff4cab10640 1 -- 172.21.15.47:0/742723795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b00040e0 msgr2=0x7ff4b00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:05.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.114+0000 7ff4cab10640 1 --2- 172.21.15.47:0/742723795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b00040e0 0x7ff4b00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.114+0000 7ff4cab10640 1 -- 172.21.15.47:0/742723795 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b0003520 con 0x7ff4b0004d00 2024-03-23T09:45:05.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.114+0000 7ff4c9b0e640 1 --2- 172.21.15.47:0/742723795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b00040e0 0x7ff4b00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:05.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.114+0000 7ff4cab10640 1 --2- 172.21.15.47:0/742723795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4b0004d00 0x7ff4b0005180 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009fc0 tx=0x7ff4b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd32ad2c1f4bc4a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:05.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.115+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc02cc20 con 0x7ff4b0004d00 2024-03-23T09:45:05.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.115+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc00f040 con 0x7ff4b0004d00 2024-03-23T09:45:05.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.115+0000 7ff4c930d640 1 -- 172.21.15.47:0/742723795 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc034440 con 0x7ff4b0004d00 2024-03-23T09:45:05.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.115+0000 7ff4cc573640 1 -- 172.21.15.47:0/742723795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4b0004d00 msgr2=0x7ff4b0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.115+0000 7ff4cc573640 1 --2- 172.21.15.47:0/742723795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4b0004d00 0x7ff4b0005180 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009fc0 tx=0x7ff4b00057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.115+0000 7ff4cab10640 1 -- 172.21.15.47:0/742723795 reap_dead start 2024-03-23T09:45:05.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.115+0000 7ff4cc573640 1 -- 172.21.15.47:0/742723795 shutdown_connections 2024-03-23T09:45:05.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.115+0000 7ff4cc573640 1 -- 172.21.15.47:0/742723795 >> 172.21.15.47:0/742723795 conn(0x7ff4c40659d0 msgr2=0x7ff4c4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:05.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.115+0000 7ff4cc573640 1 -- 172.21.15.47:0/742723795 shutdown_connections 2024-03-23T09:45:05.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.115+0000 7ff4cc573640 1 -- 172.21.15.47:0/742723795 wait complete. 2024-03-23T09:45:05.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.116+0000 7ff4cc573640 1 Processor -- start 2024-03-23T09:45:05.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.116+0000 7ff4cc573640 1 -- start start 2024-03-23T09:45:05.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4cc573640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b0004d00 0x7ff4c40c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4cc573640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c40b4920 0x7ff4c40b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4cc573640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c4147d60 0x7ff4c41481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4cc573640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40c82f0 con 0x7ff4c40b4920 2024-03-23T09:45:05.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4cc573640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40b52e0 con 0x7ff4b0004d00 2024-03-23T09:45:05.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4cc573640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c41f5ca0 con 0x7ff4c4147d60 2024-03-23T09:45:05.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4c9b0e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c40b4920 0x7ff4c40b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4c9b0e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c40b4920 0x7ff4c40b4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45266/0 (socket says 172.21.15.47:45266) 2024-03-23T09:45:05.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4c9b0e640 1 -- 172.21.15.47:0/1909673195 learned_addr learned my addr 172.21.15.47:0/1909673195 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:05.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4cab10640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c4147d60 0x7ff4c41481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4ca30f640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b0004d00 0x7ff4c40c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4c9b0e640 1 -- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c4147d60 msgr2=0x7ff4c41481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4c9b0e640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c4147d60 0x7ff4c41481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4c9b0e640 1 -- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b0004d00 msgr2=0x7ff4c40c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4c9b0e640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b0004d00 0x7ff4c40c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4c9b0e640 1 -- 172.21.15.47:0/1909673195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c40c84e0 con 0x7ff4c40b4920 2024-03-23T09:45:05.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4cab10640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c4147d60 0x7ff4c41481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:05.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.117+0000 7ff4ca30f640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b0004d00 0x7ff4c40c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:05.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.118+0000 7ff4c9b0e640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c40b4920 0x7ff4c40b4da0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400e7b0 tx=0x7ff4b400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:05.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.118+0000 7ff4baffd640 1 -- 172.21.15.47:0/1909673195 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400c660 con 0x7ff4c40b4920 2024-03-23T09:45:05.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.118+0000 7ff4baffd640 1 -- 172.21.15.47:0/1909673195 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b400c7c0 con 0x7ff4c40b4920 2024-03-23T09:45:05.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.118+0000 7ff4cc573640 1 -- 172.21.15.47:0/1909673195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c40c87c0 con 0x7ff4c40b4920 2024-03-23T09:45:05.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.118+0000 7ff4baffd640 1 -- 172.21.15.47:0/1909673195 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400cb30 con 0x7ff4c40b4920 2024-03-23T09:45:05.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.118+0000 7ff4cc573640 1 -- 172.21.15.47:0/1909673195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c40c8d80 con 0x7ff4c40b4920 2024-03-23T09:45:05.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.119+0000 7ff4cc573640 1 -- 172.21.15.47:0/1909673195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c4072360 con 0x7ff4c40b4920 2024-03-23T09:45:05.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.119+0000 7ff4baffd640 1 -- 172.21.15.47:0/1909673195 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff4b400cc90 con 0x7ff4c40b4920 2024-03-23T09:45:05.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.120+0000 7ff4baffd640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff4a806ac00 0x7ff4a806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.120+0000 7ff4baffd640 1 -- 172.21.15.47:0/1909673195 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4b4089270 con 0x7ff4c40b4920 2024-03-23T09:45:05.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.120+0000 7ff4ca30f640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff4a806ac00 0x7ff4a806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.120+0000 7ff4ca30f640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff4a806ac00 0x7ff4a806d0c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00036b0 tx=0x7ff4c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:05.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.124+0000 7ff4baffd640 1 -- 172.21.15.47:0/1909673195 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff4b40543b0 con 0x7ff4c40b4920 2024-03-23T09:45:05.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.303+0000 7ff4cc573640 1 -- 172.21.15.47:0/1909673195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4c4161330 con 0x7ff4c40b4920 2024-03-23T09:45:05.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.304+0000 7ff4baffd640 1 -- 172.21.15.47:0/1909673195 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff4c4161330 con 0x7ff4c40b4920 2024-03-23T09:45:05.304 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:45:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.306+0000 7ff4cc573640 1 -- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff4a806ac00 msgr2=0x7ff4a806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.306+0000 7ff4cc573640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff4a806ac00 0x7ff4a806d0c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00036b0 tx=0x7ff4c0009040 comp rx=0 tx=0).stop 2024-03-23T09:45:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.306+0000 7ff4cc573640 1 -- 172.21.15.47:0/1909673195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c40b4920 msgr2=0x7ff4c40b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.306+0000 7ff4cc573640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c40b4920 0x7ff4c40b4da0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400e7b0 tx=0x7ff4b400ec70 comp rx=0 tx=0).stop 2024-03-23T09:45:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.306+0000 7ff4cc573640 1 -- 172.21.15.47:0/1909673195 shutdown_connections 2024-03-23T09:45:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.306+0000 7ff4cc573640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff4a806ac00 0x7ff4a806d0c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.306+0000 7ff4cc573640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c4147d60 0x7ff4c41481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.306+0000 7ff4cc573640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c40b4920 0x7ff4c40b4da0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.306+0000 7ff4cc573640 1 --2- 172.21.15.47:0/1909673195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b0004d00 0x7ff4c40c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.306+0000 7ff4cc573640 1 -- 172.21.15.47:0/1909673195 >> 172.21.15.47:0/1909673195 conn(0x7ff4c40659d0 msgr2=0x7ff4c40d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.306+0000 7ff4cc573640 1 -- 172.21.15.47:0/1909673195 shutdown_connections 2024-03-23T09:45:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.306+0000 7ff4cc573640 1 -- 172.21.15.47:0/1909673195 wait complete. 2024-03-23T09:45:05.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:45:05.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.420+0000 7f0fb30e0640 1 Processor -- start 2024-03-23T09:45:05.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.420+0000 7f0fb30e0640 1 -- start start 2024-03-23T09:45:05.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.421+0000 7f0fb30e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fac0cde10 0x7f0fac160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.421+0000 7f0fb30e0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fac161630 con 0x7f0fac15c990 2024-03-23T09:45:05.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.421+0000 7f0fb30e0640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fac0d1030 con 0x7f0fac0b8350 2024-03-23T09:45:05.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.421+0000 7f0fb30e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0cfe10 con 0x7f0fac0cde10 2024-03-23T09:45:05.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.421+0000 7f0fb0e7c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0fac15c990 0x7f0fac0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38862/0 (socket says 172.21.15.47:38862) 2024-03-23T09:45:05.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.421+0000 7f0fb0e7c640 1 -- 172.21.15.47:0/1398619568 learned_addr learned my addr 172.21.15.47:0/1398619568 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:05.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658976767 0 0) 0x7f0fac161630 con 0x7f0fac15c990 2024-03-23T09:45:05.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f94003520 con 0x7f0fac15c990 2024-03-23T09:45:05.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fabfff640 1 --2- 172.21.15.47:0/1398619568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fac0cde10 0x7f0fac160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3940106003 0 0) 0x7f0f94003520 con 0x7f0fac15c990 2024-03-23T09:45:05.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fac0cde10 msgr2=0x7f0fac160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:05.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 --2- 172.21.15.47:0/1398619568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fac0cde10 0x7f0fac160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 >> v1:172.21.15.175:6789/0 conn(0x7f0fac0b8350 legacy=0x7f0fac0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac0b5b60 con 0x7f0fac15c990 2024-03-23T09:45:05.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0fa40019f0 con 0x7f0fac15c990 2024-03-23T09:45:05.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 >> v1:172.21.15.47:6789/0 conn(0x7f0fac15c990 legacy=0x7f0fac0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 --2- 172.21.15.47:0/1398619568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f940039a0 0x7f0f94003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 --2- 172.21.15.47:0/1398619568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f940047c0 0x7f0f94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 --2- 172.21.15.47:0/1398619568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fac0cde10 0x7f0f94005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0cfe10 con 0x7f0f940047c0 2024-03-23T09:45:05.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9400bb40 con 0x7f0f940039a0 2024-03-23T09:45:05.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9400bd00 con 0x7f0fac0cde10 2024-03-23T09:45:05.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fabfff640 1 --2- 172.21.15.47:0/1398619568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f940047c0 0x7f0f94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fb0e7c640 1 --2- 172.21.15.47:0/1398619568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f940039a0 0x7f0f94003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fb167d640 1 --2- 172.21.15.47:0/1398619568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fac0cde10 0x7f0f94005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fabfff640 1 -- 172.21.15.47:0/1398619568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fac0cde10 msgr2=0x7f0f94005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:05.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fabfff640 1 --2- 172.21.15.47:0/1398619568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fac0cde10 0x7f0f94005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fabfff640 1 -- 172.21.15.47:0/1398619568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f940039a0 msgr2=0x7f0f94003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:05.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fabfff640 1 --2- 172.21.15.47:0/1398619568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f940039a0 0x7f0f94003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.422+0000 7f0fabfff640 1 -- 172.21.15.47:0/1398619568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f94003520 con 0x7f0f940047c0 2024-03-23T09:45:05.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.423+0000 7f0fabfff640 1 --2- 172.21.15.47:0/1398619568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f940047c0 0x7f0f94004c40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009ac0 tx=0x7f0f940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df8bdb2f2fad5bb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:05.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.423+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa002c6b0 con 0x7f0f940047c0 2024-03-23T09:45:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.423+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa002c810 con 0x7f0f940047c0 2024-03-23T09:45:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.423+0000 7f0fab7fe640 1 -- 172.21.15.47:0/1398619568 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa002cba0 con 0x7f0f940047c0 2024-03-23T09:45:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.424+0000 7f0fb30e0640 1 -- 172.21.15.47:0/1398619568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f940047c0 msgr2=0x7f0f94004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.424+0000 7f0fb30e0640 1 --2- 172.21.15.47:0/1398619568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f940047c0 0x7f0f94004c40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009ac0 tx=0x7f0f940057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.424+0000 7f0fb167d640 1 -- 172.21.15.47:0/1398619568 reap_dead start 2024-03-23T09:45:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.424+0000 7f0fb30e0640 1 -- 172.21.15.47:0/1398619568 shutdown_connections 2024-03-23T09:45:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.424+0000 7f0fb30e0640 1 -- 172.21.15.47:0/1398619568 >> 172.21.15.47:0/1398619568 conn(0x7f0fac0659d0 msgr2=0x7f0fac065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.424+0000 7f0fb30e0640 1 -- 172.21.15.47:0/1398619568 shutdown_connections 2024-03-23T09:45:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.424+0000 7f0fb30e0640 1 -- 172.21.15.47:0/1398619568 wait complete. 2024-03-23T09:45:05.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.425+0000 7f0fb30e0640 1 Processor -- start 2024-03-23T09:45:05.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.425+0000 7f0fb30e0640 1 -- start start 2024-03-23T09:45:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb30e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f940047c0 0x7f0fac15e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb30e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fac15a8f0 0x7f0fac14fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb30e0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fac15ad70 0x7f0fac150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb30e0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac1505d0 con 0x7f0fac15ad70 2024-03-23T09:45:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb30e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac15ff60 con 0x7f0f940047c0 2024-03-23T09:45:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb30e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac15fc80 con 0x7f0fac15a8f0 2024-03-23T09:45:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb167d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fac15ad70 0x7f0fac150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb167d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fac15ad70 0x7f0fac150090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45288/0 (socket says 172.21.15.47:45288) 2024-03-23T09:45:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb167d640 1 -- 172.21.15.47:0/37969514 learned_addr learned my addr 172.21.15.47:0/37969514 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb0e7c640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f940047c0 0x7f0fac15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fabfff640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fac15a8f0 0x7f0fac14fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb167d640 1 -- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fac15a8f0 msgr2=0x7f0fac14fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb167d640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fac15a8f0 0x7f0fac14fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb167d640 1 -- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f940047c0 msgr2=0x7f0fac15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb167d640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f940047c0 0x7f0fac15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb167d640 1 -- 172.21.15.47:0/37969514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac15dba0 con 0x7f0fac15ad70 2024-03-23T09:45:05.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.426+0000 7f0fb167d640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fac15ad70 0x7f0fac150090 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c000c00 tx=0x7f0f9c00e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:05.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.427+0000 7f0fa97fa640 1 -- 172.21.15.47:0/37969514 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0052e0 con 0x7f0fac15ad70 2024-03-23T09:45:05.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.427+0000 7f0fa97fa640 1 -- 172.21.15.47:0/37969514 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9c005440 con 0x7f0fac15ad70 2024-03-23T09:45:05.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.427+0000 7f0fa97fa640 1 -- 172.21.15.47:0/37969514 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c003830 con 0x7f0fac15ad70 2024-03-23T09:45:05.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.427+0000 7f0fb30e0640 1 -- 172.21.15.47:0/37969514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fac15de80 con 0x7f0fac15ad70 2024-03-23T09:45:05.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.427+0000 7f0fb30e0640 1 -- 172.21.15.47:0/37969514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fac150f50 con 0x7f0fac15ad70 2024-03-23T09:45:05.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.428+0000 7f0fb30e0640 1 -- 172.21.15.47:0/37969514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fac072360 con 0x7f0fac15ad70 2024-03-23T09:45:05.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.428+0000 7f0fa97fa640 1 -- 172.21.15.47:0/37969514 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0f9c003990 con 0x7f0fac15ad70 2024-03-23T09:45:05.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.428+0000 7f0fa97fa640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0f9806ac30 0x7f0f9806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.429+0000 7f0fa97fa640 1 -- 172.21.15.47:0/37969514 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f9c088970 con 0x7f0fac15ad70 2024-03-23T09:45:05.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.429+0000 7f0fb0e7c640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0f9806ac30 0x7f0f9806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.429+0000 7f0fb0e7c640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0f9806ac30 0x7f0f9806d0f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4000f80 tx=0x7f0fa4006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:05.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.433+0000 7f0fa97fa640 1 -- 172.21.15.47:0/37969514 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0f9c053b50 con 0x7f0fac15ad70 2024-03-23T09:45:05.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.613+0000 7f0fb30e0640 1 -- 172.21.15.47:0/37969514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0fac15e3a0 con 0x7f0fac15ad70 2024-03-23T09:45:05.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.613+0000 7f0fa97fa640 1 -- 172.21.15.47:0/37969514 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0fac15e3a0 con 0x7f0fac15ad70 2024-03-23T09:45:05.613 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:45:05.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.615+0000 7f0fb30e0640 1 -- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0f9806ac30 msgr2=0x7f0f9806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.615+0000 7f0fb30e0640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0f9806ac30 0x7f0f9806d0f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4000f80 tx=0x7f0fa4006040 comp rx=0 tx=0).stop 2024-03-23T09:45:05.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.615+0000 7f0fb30e0640 1 -- 172.21.15.47:0/37969514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fac15ad70 msgr2=0x7f0fac150090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.615+0000 7f0fb30e0640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fac15ad70 0x7f0fac150090 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c000c00 tx=0x7f0f9c00e8a0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.615+0000 7f0fb30e0640 1 -- 172.21.15.47:0/37969514 shutdown_connections 2024-03-23T09:45:05.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.615+0000 7f0fb30e0640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0f9806ac30 0x7f0f9806d0f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.615+0000 7f0fb30e0640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fac15ad70 0x7f0fac150090 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.615+0000 7f0fb30e0640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fac15a8f0 0x7f0fac14fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.615+0000 7f0fb30e0640 1 --2- 172.21.15.47:0/37969514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f940047c0 0x7f0fac15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.615+0000 7f0fb30e0640 1 -- 172.21.15.47:0/37969514 >> 172.21.15.47:0/37969514 conn(0x7f0fac0659d0 msgr2=0x7f0fac0b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:05.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.615+0000 7f0fb30e0640 1 -- 172.21.15.47:0/37969514 shutdown_connections 2024-03-23T09:45:05.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.615+0000 7f0fb30e0640 1 -- 172.21.15.47:0/37969514 wait complete. 2024-03-23T09:45:05.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:45:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.726+0000 7f9b8fc99640 1 Processor -- start 2024-03-23T09:45:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.726+0000 7f9b8fc99640 1 -- start start 2024-03-23T09:45:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.727+0000 7f9b8fc99640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9b8815cee0 0x7f9b880d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.727+0000 7f9b8fc99640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b880d2f80 0x7f9b880b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.727+0000 7f9b8fc99640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b880d1300 con 0x7f9b880d23a0 2024-03-23T09:45:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.727+0000 7f9b8fc99640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b880d15e0 con 0x7f9b8815cee0 2024-03-23T09:45:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.727+0000 7f9b8fc99640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b880d18c0 con 0x7f9b880d2f80 2024-03-23T09:45:05.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.727+0000 7f9b8da35640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9b8815cee0 0x7f9b880d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.728+0000 7f9b8e236640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b880d2f80 0x7f9b880b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.728+0000 7f9b8d234640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f9b880d23a0 0x7f9b880d2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:41164/0 (socket says 172.21.15.47:41164) 2024-03-23T09:45:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.728+0000 7f9b8d234640 1 -- 172.21.15.47:0/3195217317 learned_addr learned my addr 172.21.15.47:0/3195217317 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.728+0000 7f9b8da35640 1 -- 172.21.15.47:0/3195217317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b880d2f80 msgr2=0x7f9b880b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.728+0000 7f9b8da35640 1 --2- 172.21.15.47:0/3195217317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b880d2f80 0x7f9b880b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.728+0000 7f9b8da35640 1 -- 172.21.15.47:0/3195217317 >> v1:172.21.15.175:6789/0 conn(0x7f9b880d23a0 legacy=0x7f9b880d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.728+0000 7f9b8ca33640 1 -- 172.21.15.47:0/3195217317 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882959198 0 0) 0x7f9b880d1300 con 0x7f9b880d23a0 2024-03-23T09:45:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.728+0000 7f9b8da35640 1 -- 172.21.15.47:0/3195217317 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b880d1ba0 con 0x7f9b8815cee0 2024-03-23T09:45:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.729+0000 7f9b8da35640 1 --2- 172.21.15.47:0/3195217317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8815cee0 0x7f9b880d0990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9b800097a0 tx=0x7f9b8002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45e48981a66e5239 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.729+0000 7f9b8ca33640 1 -- 172.21.15.47:0/3195217317 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80031830 con 0x7f9b8815cee0 2024-03-23T09:45:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.729+0000 7f9b8ca33640 1 -- 172.21.15.47:0/3195217317 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b80031990 con 0x7f9b8815cee0 2024-03-23T09:45:05.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.729+0000 7f9b8ca33640 1 -- 172.21.15.47:0/3195217317 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80031d20 con 0x7f9b8815cee0 2024-03-23T09:45:05.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.729+0000 7f9b8fc99640 1 -- 172.21.15.47:0/3195217317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8815cee0 msgr2=0x7f9b880d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.729+0000 7f9b8fc99640 1 --2- 172.21.15.47:0/3195217317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8815cee0 0x7f9b880d0990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9b800097a0 tx=0x7f9b8002b8d0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.729+0000 7f9b8fc99640 1 -- 172.21.15.47:0/3195217317 shutdown_connections 2024-03-23T09:45:05.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.729+0000 7f9b8fc99640 1 --2- 172.21.15.47:0/3195217317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b880d2f80 0x7f9b880b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.729+0000 7f9b8fc99640 1 --2- 172.21.15.47:0/3195217317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8815cee0 0x7f9b880d0990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.729+0000 7f9b8fc99640 1 -- 172.21.15.47:0/3195217317 >> 172.21.15.47:0/3195217317 conn(0x7f9b88065df0 msgr2=0x7f9b88066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:05.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.730+0000 7f9b8fc99640 1 -- 172.21.15.47:0/3195217317 shutdown_connections 2024-03-23T09:45:05.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.730+0000 7f9b8fc99640 1 -- 172.21.15.47:0/3195217317 wait complete. 2024-03-23T09:45:05.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.731+0000 7f9b8fc99640 1 Processor -- start 2024-03-23T09:45:05.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.731+0000 7f9b8fc99640 1 -- start start 2024-03-23T09:45:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.731+0000 7f9b8fc99640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b880d23a0 0x7f9b880cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.731+0000 7f9b8fc99640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880d2f80 0x7f9b88149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.731+0000 7f9b8fc99640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8815cee0 0x7f9b880c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.731+0000 7f9b8fc99640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b881f3360 con 0x7f9b880d2f80 2024-03-23T09:45:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.731+0000 7f9b8fc99640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b881f2a30 con 0x7f9b8815cee0 2024-03-23T09:45:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.731+0000 7f9b8fc99640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b881f2d10 con 0x7f9b880d23a0 2024-03-23T09:45:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.731+0000 7f9b8d234640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880d2f80 0x7f9b88149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b8d234640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880d2f80 0x7f9b88149d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45296/0 (socket says 172.21.15.47:45296) 2024-03-23T09:45:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b8d234640 1 -- 172.21.15.47:0/585217934 learned_addr learned my addr 172.21.15.47:0/585217934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b8e236640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8815cee0 0x7f9b880c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b8da35640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b880d23a0 0x7f9b880cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b8d234640 1 -- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b880d23a0 msgr2=0x7f9b880cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b8d234640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b880d23a0 0x7f9b880cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b8d234640 1 -- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8815cee0 msgr2=0x7f9b880c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b8d234640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8815cee0 0x7f9b880c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b8d234640 1 -- 172.21.15.47:0/585217934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b80009400 con 0x7f9b880d2f80 2024-03-23T09:45:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b8da35640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b880d23a0 0x7f9b880cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b8d234640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880d2f80 0x7f9b88149d40 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c4b0 tx=0x7f9b7c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b867fc640 1 -- 172.21.15.47:0/585217934 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0074e0 con 0x7f9b880d2f80 2024-03-23T09:45:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b867fc640 1 -- 172.21.15.47:0/585217934 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c007660 con 0x7f9b880d2f80 2024-03-23T09:45:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b8fc99640 1 -- 172.21.15.47:0/585217934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b881f2ff0 con 0x7f9b880d2f80 2024-03-23T09:45:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b867fc640 1 -- 172.21.15.47:0/585217934 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c004a20 con 0x7f9b880d2f80 2024-03-23T09:45:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.732+0000 7f9b8fc99640 1 -- 172.21.15.47:0/585217934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b880cdba0 con 0x7f9b880d2f80 2024-03-23T09:45:05.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.733+0000 7f9b8fc99640 1 -- 172.21.15.47:0/585217934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b88072360 con 0x7f9b880d2f80 2024-03-23T09:45:05.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.734+0000 7f9b867fc640 1 -- 172.21.15.47:0/585217934 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9b7c004b80 con 0x7f9b880d2f80 2024-03-23T09:45:05.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.734+0000 7f9b867fc640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b7406ac00 0x7f9b7406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:05.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.734+0000 7f9b867fc640 1 -- 172.21.15.47:0/585217934 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b7c088490 con 0x7f9b880d2f80 2024-03-23T09:45:05.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.734+0000 7f9b8da35640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b7406ac00 0x7f9b7406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:05.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.735+0000 7f9b8da35640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b7406ac00 0x7f9b7406d0c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9b80036000 tx=0x7f9b8003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:05.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.738+0000 7f9b867fc640 1 -- 172.21.15.47:0/585217934 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9b7c053760 con 0x7f9b880d2f80 2024-03-23T09:45:05.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.955+0000 7f9b8fc99640 1 -- 172.21.15.47:0/585217934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b8815ac80 con 0x7f9b880d2f80 2024-03-23T09:45:05.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.955+0000 7f9b867fc640 1 -- 172.21.15.47:0/585217934 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f9b7c056d80 con 0x7f9b880d2f80 2024-03-23T09:45:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:05.956 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/375513546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":375513546},{"type":"v1","addr":"172.21.15.175:6837","nonce":375513546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/3650756747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3650756747},{"type":"v1","addr":"172.21.15.47:6833","nonce":3650756747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2893805543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2893805543},{"type":"v1","addr":"172.21.15.47:6835","nonce":2893805543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:43:47.389279+0000","modified":"2024-03-23T09:44:35.825401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.175:6836/2498456351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2498456351},{"type":"v1","addr":"172.21.15.175:6836","nonce":2498456351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-23T09:45:05.956 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2024-03-23T09:45:05.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.957+0000 7f9b8fc99640 1 -- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b7406ac00 msgr2=0x7f9b7406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.957+0000 7f9b8fc99640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b7406ac00 0x7f9b7406d0c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9b80036000 tx=0x7f9b8003c040 comp rx=0 tx=0).stop 2024-03-23T09:45:05.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.957+0000 7f9b8fc99640 1 -- 172.21.15.47:0/585217934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880d2f80 msgr2=0x7f9b88149d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:05.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.957+0000 7f9b8fc99640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880d2f80 0x7f9b88149d40 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c4b0 tx=0x7f9b7c00c970 comp rx=0 tx=0).stop 2024-03-23T09:45:05.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.957+0000 7f9b8fc99640 1 -- 172.21.15.47:0/585217934 shutdown_connections 2024-03-23T09:45:05.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.957+0000 7f9b8fc99640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b7406ac00 0x7f9b7406d0c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.957+0000 7f9b8fc99640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8815cee0 0x7f9b880c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.957+0000 7f9b8fc99640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880d2f80 0x7f9b88149d40 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.957+0000 7f9b8fc99640 1 --2- 172.21.15.47:0/585217934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b880d23a0 0x7f9b880cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:05.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.957+0000 7f9b8fc99640 1 -- 172.21.15.47:0/585217934 >> 172.21.15.47:0/585217934 conn(0x7f9b88065df0 msgr2=0x7f9b880cd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:05.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.957+0000 7f9b8fc99640 1 -- 172.21.15.47:0/585217934 shutdown_connections 2024-03-23T09:45:05.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:05.957+0000 7f9b8fc99640 1 -- 172.21.15.47:0/585217934 wait complete. 2024-03-23T09:45:05.968 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:45:06.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.069+0000 7f0c4f026640 1 Processor -- start 2024-03-23T09:45:06.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.069+0000 7f0c4f026640 1 -- start start 2024-03-23T09:45:06.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.069+0000 7f0c4f026640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c480b8350 0x7f0c480cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.069+0000 7f0c4f026640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0c480cdc40 0x7f0c480d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.069+0000 7f0c4f026640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c480d11e0 con 0x7f0c4815ca90 2024-03-23T09:45:06.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.069+0000 7f0c4f026640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c480ce0d0 con 0x7f0c480cdc40 2024-03-23T09:45:06.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.069+0000 7f0c4f026640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48160f60 con 0x7f0c480b8350 2024-03-23T09:45:06.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.069+0000 7f0c4cdc2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0c4815ca90 0x7f0c48160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38866/0 (socket says 172.21.15.47:38866) 2024-03-23T09:45:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.069+0000 7f0c4cdc2640 1 -- 172.21.15.47:0/3228323382 learned_addr learned my addr 172.21.15.47:0/3228323382 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c4d5c3640 1 --2- 172.21.15.47:0/3228323382 >> v2:172.21.15.175:3300/0 conn(0x7f0c480cdc40 0x7f0c480d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c47fff640 1 --2- 172.21.15.47:0/3228323382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c480b8350 0x7f0c480cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007020491 0 0) 0x7f0c480d11e0 con 0x7f0c4815ca90 2024-03-23T09:45:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c38003520 con 0x7f0c4815ca90 2024-03-23T09:45:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1765626957 0 0) 0x7f0c38003520 con 0x7f0c4815ca90 2024-03-23T09:45:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c480b8350 msgr2=0x7f0c480cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 --2- 172.21.15.47:0/3228323382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c480b8350 0x7f0c480cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 >> v2:172.21.15.175:3300/0 conn(0x7f0c480cdc40 msgr2=0x7f0c480d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 --2- 172.21.15.47:0/3228323382 >> v2:172.21.15.175:3300/0 conn(0x7f0c480cdc40 0x7f0c480d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c48161240 con 0x7f0c4815ca90 2024-03-23T09:45:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0c3c001670 con 0x7f0c4815ca90 2024-03-23T09:45:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 >> v1:172.21.15.47:6789/0 conn(0x7f0c4815ca90 legacy=0x7f0c48160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 --2- 172.21.15.47:0/3228323382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38003970 0x7f0c38003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 --2- 172.21.15.47:0/3228323382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c380047c0 0x7f0c38004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 --2- 172.21.15.47:0/3228323382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c480b8350 0x7f0c38005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c480ce0d0 con 0x7f0c38003970 2024-03-23T09:45:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48160f60 con 0x7f0c380047c0 2024-03-23T09:45:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3800b980 con 0x7f0c480b8350 2024-03-23T09:45:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c4cdc2640 1 --2- 172.21.15.47:0/3228323382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38003970 0x7f0c38003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c4d5c3640 1 --2- 172.21.15.47:0/3228323382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c480b8350 0x7f0c38005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c47fff640 1 --2- 172.21.15.47:0/3228323382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c380047c0 0x7f0c38004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c4cdc2640 1 -- 172.21.15.47:0/3228323382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c480b8350 msgr2=0x7f0c38005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c4cdc2640 1 --2- 172.21.15.47:0/3228323382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c480b8350 0x7f0c38005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c4cdc2640 1 -- 172.21.15.47:0/3228323382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c380047c0 msgr2=0x7f0c38004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c4cdc2640 1 --2- 172.21.15.47:0/3228323382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c380047c0 0x7f0c38004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.070+0000 7f0c4cdc2640 1 -- 172.21.15.47:0/3228323382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c38003520 con 0x7f0c38003970 2024-03-23T09:45:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.071+0000 7f0c4cdc2640 1 --2- 172.21.15.47:0/3228323382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38003970 0x7f0c38003d90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00c2c0 tx=0x7f0c380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d0ba47f3aa6078 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.071+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c005ad0 con 0x7f0c38003970 2024-03-23T09:45:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.071+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c005c30 con 0x7f0c38003970 2024-03-23T09:45:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.071+0000 7f0c477fe640 1 -- 172.21.15.47:0/3228323382 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c02c8b0 con 0x7f0c38003970 2024-03-23T09:45:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.072+0000 7f0c4f026640 1 -- 172.21.15.47:0/3228323382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38003970 msgr2=0x7f0c38003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.072+0000 7f0c4f026640 1 --2- 172.21.15.47:0/3228323382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38003970 0x7f0c38003d90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00c2c0 tx=0x7f0c380057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.072+0000 7f0c4d5c3640 1 -- 172.21.15.47:0/3228323382 reap_dead start 2024-03-23T09:45:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.072+0000 7f0c4f026640 1 -- 172.21.15.47:0/3228323382 shutdown_connections 2024-03-23T09:45:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.072+0000 7f0c4f026640 1 -- 172.21.15.47:0/3228323382 >> 172.21.15.47:0/3228323382 conn(0x7f0c480659d0 msgr2=0x7f0c48065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.072+0000 7f0c4f026640 1 -- 172.21.15.47:0/3228323382 shutdown_connections 2024-03-23T09:45:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.072+0000 7f0c4f026640 1 -- 172.21.15.47:0/3228323382 wait complete. 2024-03-23T09:45:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.073+0000 7f0c4f026640 1 Processor -- start 2024-03-23T09:45:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.073+0000 7f0c4f026640 1 -- start start 2024-03-23T09:45:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c4f026640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c38003970 0x7f0c481f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c4f026640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48155d40 0x7f0c480d2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c4f026640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c48156180 0x7f0c480d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c4f026640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c481f33d0 con 0x7f0c48155d40 2024-03-23T09:45:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c4f026640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c481587d0 con 0x7f0c38003970 2024-03-23T09:45:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c4f026640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48158aa0 con 0x7f0c48156180 2024-03-23T09:45:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c47fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48155d40 0x7f0c480d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c47fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48155d40 0x7f0c480d2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45304/0 (socket says 172.21.15.47:45304) 2024-03-23T09:45:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c47fff640 1 -- 172.21.15.47:0/1920303058 learned_addr learned my addr 172.21.15.47:0/1920303058 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c4cdc2640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c38003970 0x7f0c481f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c4d5c3640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c48156180 0x7f0c480d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c47fff640 1 -- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c48156180 msgr2=0x7f0c480d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c47fff640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c48156180 0x7f0c480d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c47fff640 1 -- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c38003970 msgr2=0x7f0c481f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c47fff640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c38003970 0x7f0c481f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c47fff640 1 -- 172.21.15.47:0/1920303058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c48158d80 con 0x7f0c48155d40 2024-03-23T09:45:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.074+0000 7f0c4cdc2640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c38003970 0x7f0c481f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.075+0000 7f0c47fff640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48155d40 0x7f0c480d2500 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0c34011540 tx=0x7f0c34011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.075+0000 7f0c457fa640 1 -- 172.21.15.47:0/1920303058 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34015930 con 0x7f0c48155d40 2024-03-23T09:45:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.075+0000 7f0c457fa640 1 -- 172.21.15.47:0/1920303058 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c34015a90 con 0x7f0c48155d40 2024-03-23T09:45:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.075+0000 7f0c4f026640 1 -- 172.21.15.47:0/1920303058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c48159060 con 0x7f0c48155d40 2024-03-23T09:45:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.075+0000 7f0c457fa640 1 -- 172.21.15.47:0/1920303058 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34015e60 con 0x7f0c48155d40 2024-03-23T09:45:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.075+0000 7f0c4f026640 1 -- 172.21.15.47:0/1920303058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c48159620 con 0x7f0c48155d40 2024-03-23T09:45:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.076+0000 7f0c4f026640 1 -- 172.21.15.47:0/1920303058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c48072360 con 0x7f0c48155d40 2024-03-23T09:45:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.076+0000 7f0c457fa640 1 -- 172.21.15.47:0/1920303058 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0c3400f460 con 0x7f0c48155d40 2024-03-23T09:45:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.076+0000 7f0c457fa640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c2c06ac00 0x7f0c2c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.077+0000 7f0c457fa640 1 -- 172.21.15.47:0/1920303058 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c34089b80 con 0x7f0c48155d40 2024-03-23T09:45:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.077+0000 7f0c4cdc2640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c2c06ac00 0x7f0c2c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.077+0000 7f0c4cdc2640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c2c06ac00 0x7f0c2c06d0c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c034000 tx=0x7f0c3c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:06.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.081+0000 7f0c457fa640 1 -- 172.21.15.47:0/1920303058 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0c34054d60 con 0x7f0c48155d40 2024-03-23T09:45:06.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.259+0000 7f0c4f026640 1 -- 172.21.15.47:0/1920303058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c48066be0 con 0x7f0c48155d40 2024-03-23T09:45:06.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.260+0000 7f0c457fa640 1 -- 172.21.15.47:0/1920303058 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16097 (secure 0 0 0) 0x7f0c34058380 con 0x7f0c48155d40 2024-03-23T09:45:06.260 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":58,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:43:48.390657+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:43:45.646160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-23T09:43:46.351634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:45:06.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.262+0000 7f0c4f026640 1 -- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c2c06ac00 msgr2=0x7f0c2c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.262+0000 7f0c4f026640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c2c06ac00 0x7f0c2c06d0c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c034000 tx=0x7f0c3c007000 comp rx=0 tx=0).stop 2024-03-23T09:45:06.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.262+0000 7f0c4f026640 1 -- 172.21.15.47:0/1920303058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48155d40 msgr2=0x7f0c480d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.262+0000 7f0c4f026640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48155d40 0x7f0c480d2500 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0c34011540 tx=0x7f0c34011a00 comp rx=0 tx=0).stop 2024-03-23T09:45:06.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.262+0000 7f0c4f026640 1 -- 172.21.15.47:0/1920303058 shutdown_connections 2024-03-23T09:45:06.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.262+0000 7f0c4f026640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c2c06ac00 0x7f0c2c06d0c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.262+0000 7f0c4f026640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c48156180 0x7f0c480d2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.262+0000 7f0c4f026640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48155d40 0x7f0c480d2500 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.262+0000 7f0c4f026640 1 --2- 172.21.15.47:0/1920303058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c38003970 0x7f0c481f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.262+0000 7f0c4f026640 1 -- 172.21.15.47:0/1920303058 >> 172.21.15.47:0/1920303058 conn(0x7f0c480659d0 msgr2=0x7f0c4815d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:06.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.262+0000 7f0c4f026640 1 -- 172.21.15.47:0/1920303058 shutdown_connections 2024-03-23T09:45:06.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.262+0000 7f0c4f026640 1 -- 172.21.15.47:0/1920303058 wait complete. 2024-03-23T09:45:06.273 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T09:45:06.273 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:45:06.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.375+0000 7f9f7d7e7640 1 Processor -- start 2024-03-23T09:45:06.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.375+0000 7f9f7d7e7640 1 -- start start 2024-03-23T09:45:06.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.375+0000 7f9f7d7e7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f780b8350 0x7f9f780cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.375+0000 7f9f7d7e7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f780cdc40 0x7f9f780d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.375+0000 7f9f7d7e7640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f780d11e0 con 0x7f9f7815ca90 2024-03-23T09:45:06.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.375+0000 7f9f7d7e7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f780ce0d0 con 0x7f9f780b8350 2024-03-23T09:45:06.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.375+0000 7f9f7d7e7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f78160f60 con 0x7f9f780cdc40 2024-03-23T09:45:06.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.376+0000 7f9f767fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f780b8350 0x7f9f780cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.376+0000 7f9f76ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9f7815ca90 0x7f9f78160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38868/0 (socket says 172.21.15.47:38868) 2024-03-23T09:45:06.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.376+0000 7f9f76ffd640 1 -- 172.21.15.47:0/1861876128 learned_addr learned my addr 172.21.15.47:0/1861876128 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:06.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.376+0000 7f9f777fe640 1 --2- 172.21.15.47:0/1861876128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f780cdc40 0x7f9f780d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.376+0000 7f9f767fc640 1 -- 172.21.15.47:0/1861876128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f780cdc40 msgr2=0x7f9f780d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:06.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.376+0000 7f9f767fc640 1 --2- 172.21.15.47:0/1861876128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f780cdc40 0x7f9f780d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.376+0000 7f9f767fc640 1 -- 172.21.15.47:0/1861876128 >> v1:172.21.15.47:6789/0 conn(0x7f9f7815ca90 legacy=0x7f9f78160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.376+0000 7f9f767fc640 1 -- 172.21.15.47:0/1861876128 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f78161240 con 0x7f9f780b8350 2024-03-23T09:45:06.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.377+0000 7f9f767fc640 1 --2- 172.21.15.47:0/1861876128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f780b8350 0x7f9f780cd700 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0097a0 tx=0x7f9f6c02b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1614fb2b2297ea server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:06.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.377+0000 7f9f75ffb640 1 -- 172.21.15.47:0/1861876128 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c030710 con 0x7f9f780b8350 2024-03-23T09:45:06.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.377+0000 7f9f75ffb640 1 -- 172.21.15.47:0/1861876128 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c02e050 con 0x7f9f780b8350 2024-03-23T09:45:06.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.377+0000 7f9f75ffb640 1 -- 172.21.15.47:0/1861876128 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c030a90 con 0x7f9f780b8350 2024-03-23T09:45:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.377+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1861876128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f780b8350 msgr2=0x7f9f780cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.377+0000 7f9f7d7e7640 1 --2- 172.21.15.47:0/1861876128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f780b8350 0x7f9f780cd700 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0097a0 tx=0x7f9f6c02b660 comp rx=0 tx=0).stop 2024-03-23T09:45:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.377+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1861876128 shutdown_connections 2024-03-23T09:45:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.377+0000 7f9f7d7e7640 1 --2- 172.21.15.47:0/1861876128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f780cdc40 0x7f9f780d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.377+0000 7f9f7d7e7640 1 --2- 172.21.15.47:0/1861876128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f780b8350 0x7f9f780cd700 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.377+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1861876128 >> 172.21.15.47:0/1861876128 conn(0x7f9f780659d0 msgr2=0x7f9f78065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.377+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1861876128 shutdown_connections 2024-03-23T09:45:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.377+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1861876128 wait complete. 2024-03-23T09:45:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.379+0000 7f9f7d7e7640 1 Processor -- start 2024-03-23T09:45:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.379+0000 7f9f7d7e7640 1 -- start start 2024-03-23T09:45:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.379+0000 7f9f7d7e7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f780b8350 0x7f9f780ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.379+0000 7f9f7d7e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f780cdc40 0x7f9f78150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.379+0000 7f9f7d7e7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f7815ca90 0x7f9f780b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.379+0000 7f9f7d7e7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f78152530 con 0x7f9f780cdc40 2024-03-23T09:45:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.379+0000 7f9f7d7e7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f78151480 con 0x7f9f780b8350 2024-03-23T09:45:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.379+0000 7f9f7d7e7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f780caba0 con 0x7f9f7815ca90 2024-03-23T09:45:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.379+0000 7f9f767fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f780cdc40 0x7f9f78150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.379+0000 7f9f767fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f780cdc40 0x7f9f78150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45332/0 (socket says 172.21.15.47:45332) 2024-03-23T09:45:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.379+0000 7f9f767fc640 1 -- 172.21.15.47:0/1684127609 learned_addr learned my addr 172.21.15.47:0/1684127609 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f777fe640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f7815ca90 0x7f9f780b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f76ffd640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f780b8350 0x7f9f780ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f767fc640 1 -- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f7815ca90 msgr2=0x7f9f780b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f767fc640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f7815ca90 0x7f9f780b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f767fc640 1 -- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f780b8350 msgr2=0x7f9f780ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f767fc640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f780b8350 0x7f9f780ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f767fc640 1 -- 172.21.15.47:0/1684127609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6c009400 con 0x7f9f780cdc40 2024-03-23T09:45:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f777fe640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f7815ca90 0x7f9f780b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f767fc640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f780cdc40 0x7f9f78150f40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c035040 tx=0x7f9f6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f5f7fe640 1 -- 172.21.15.47:0/1684127609 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c03b8d0 con 0x7f9f780cdc40 2024-03-23T09:45:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f5f7fe640 1 -- 172.21.15.47:0/1684127609 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c03ba50 con 0x7f9f780cdc40 2024-03-23T09:45:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f5f7fe640 1 -- 172.21.15.47:0/1684127609 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c03be00 con 0x7f9f780cdc40 2024-03-23T09:45:06.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1684127609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f780cae80 con 0x7f9f780cdc40 2024-03-23T09:45:06.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.380+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1684127609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f780cb440 con 0x7f9f780cdc40 2024-03-23T09:45:06.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.381+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1684127609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f78072360 con 0x7f9f780cdc40 2024-03-23T09:45:06.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.381+0000 7f9f5f7fe640 1 -- 172.21.15.47:0/1684127609 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9f6c039040 con 0x7f9f780cdc40 2024-03-23T09:45:06.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.382+0000 7f9f5f7fe640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f6406ac00 0x7f9f6406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.382+0000 7f9f5f7fe640 1 -- 172.21.15.47:0/1684127609 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f6c0aa690 con 0x7f9f780cdc40 2024-03-23T09:45:06.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.382+0000 7f9f76ffd640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f6406ac00 0x7f9f6406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.382+0000 7f9f76ffd640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f6406ac00 0x7f9f6406d0c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9f68007fd0 tx=0x7f9f68007c20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:06.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.386+0000 7f9f5f7fe640 1 -- 172.21.15.47:0/1684127609 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9f6c075900 con 0x7f9f780cdc40 2024-03-23T09:45:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.602+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1684127609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f78066be0 con 0x7f9f780cdc40 2024-03-23T09:45:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.603+0000 7f9f5f7fe640 1 -- 172.21.15.47:0/1684127609 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f9f6c078f20 con 0x7f9f780cdc40 2024-03-23T09:45:06.603 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:06.603 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/375513546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":375513546},{"type":"v1","addr":"172.21.15.175:6837","nonce":375513546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/3650756747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3650756747},{"type":"v1","addr":"172.21.15.47:6833","nonce":3650756747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2893805543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2893805543},{"type":"v1","addr":"172.21.15.47:6835","nonce":2893805543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:43:47.389279+0000","modified":"2024-03-23T09:44:35.825401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.175:6836/2498456351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2498456351},{"type":"v1","addr":"172.21.15.175:6836","nonce":2498456351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-23T09:45:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2024-03-23T09:45:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.604+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f6406ac00 msgr2=0x7f9f6406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.604+0000 7f9f7d7e7640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f6406ac00 0x7f9f6406d0c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9f68007fd0 tx=0x7f9f68007c20 comp rx=0 tx=0).stop 2024-03-23T09:45:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.604+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1684127609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f780cdc40 msgr2=0x7f9f78150f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.604+0000 7f9f7d7e7640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f780cdc40 0x7f9f78150f40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c035040 tx=0x7f9f6c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.604+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1684127609 shutdown_connections 2024-03-23T09:45:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.604+0000 7f9f7d7e7640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f6406ac00 0x7f9f6406d0c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.604+0000 7f9f7d7e7640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f7815ca90 0x7f9f780b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.604+0000 7f9f7d7e7640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f780cdc40 0x7f9f78150f40 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.604+0000 7f9f7d7e7640 1 --2- 172.21.15.47:0/1684127609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f780b8350 0x7f9f780ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.604+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1684127609 >> 172.21.15.47:0/1684127609 conn(0x7f9f780659d0 msgr2=0x7f9f780b78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:06.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.605+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1684127609 shutdown_connections 2024-03-23T09:45:06.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.605+0000 7f9f7d7e7640 1 -- 172.21.15.47:0/1684127609 wait complete. 2024-03-23T09:45:06.616 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:45:06.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.716+0000 7f179681e640 1 Processor -- start 2024-03-23T09:45:06.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.716+0000 7f179681e640 1 -- start start 2024-03-23T09:45:06.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.717+0000 7f179681e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f17900cde10 0x7f1790167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.717+0000 7f179681e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17900d11e0 con 0x7f17900b8350 2024-03-23T09:45:06.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.717+0000 7f179681e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17900d2680 con 0x7f179015ca90 2024-03-23T09:45:06.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.717+0000 7f179681e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1790160f60 con 0x7f17900cde10 2024-03-23T09:45:06.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.717+0000 7f1794dbb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f17900cde10 0x7f1790167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.717+0000 7f178f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f17900b8350 0x7f17900cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38884/0 (socket says 172.21.15.47:38884) 2024-03-23T09:45:06.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.717+0000 7f178f7fe640 1 -- 172.21.15.47:0/1339031990 learned_addr learned my addr 172.21.15.47:0/1339031990 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:06.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.718+0000 7f1794dbb640 1 -- 172.21.15.47:0/1339031990 >> v1:172.21.15.175:6789/0 conn(0x7f179015ca90 legacy=0x7f1790160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:06.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.718+0000 7f1794dbb640 1 -- 172.21.15.47:0/1339031990 >> v1:172.21.15.47:6789/0 conn(0x7f17900b8350 legacy=0x7f17900cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.718+0000 7f1794dbb640 1 -- 172.21.15.47:0/1339031990 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1790161240 con 0x7f17900cde10 2024-03-23T09:45:06.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.718+0000 7f178effd640 1 -- 172.21.15.47:0/1339031990 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128376906 0 0) 0x7f17900d11e0 con 0x7f17900b8350 2024-03-23T09:45:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.718+0000 7f1794dbb640 1 --2- 172.21.15.47:0/1339031990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17900cde10 0x7f1790167aa0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d90 tx=0x7f178802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff17540ead031c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.718+0000 7f178effd640 1 -- 172.21.15.47:0/1339031990 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178802cbd0 con 0x7f17900cde10 2024-03-23T09:45:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.719+0000 7f178effd640 1 -- 172.21.15.47:0/1339031990 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178802cd30 con 0x7f17900cde10 2024-03-23T09:45:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.719+0000 7f178effd640 1 -- 172.21.15.47:0/1339031990 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178802cea0 con 0x7f17900cde10 2024-03-23T09:45:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.719+0000 7f179681e640 1 -- 172.21.15.47:0/1339031990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17900cde10 msgr2=0x7f1790167aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.719+0000 7f179681e640 1 --2- 172.21.15.47:0/1339031990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17900cde10 0x7f1790167aa0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d90 tx=0x7f178802be20 comp rx=0 tx=0).stop 2024-03-23T09:45:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.719+0000 7f179681e640 1 -- 172.21.15.47:0/1339031990 shutdown_connections 2024-03-23T09:45:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.719+0000 7f179681e640 1 --2- 172.21.15.47:0/1339031990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17900cde10 0x7f1790167aa0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.719+0000 7f179681e640 1 -- 172.21.15.47:0/1339031990 >> 172.21.15.47:0/1339031990 conn(0x7f17900659d0 msgr2=0x7f1790065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.719+0000 7f179681e640 1 -- 172.21.15.47:0/1339031990 shutdown_connections 2024-03-23T09:45:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.719+0000 7f179681e640 1 -- 172.21.15.47:0/1339031990 wait complete. 2024-03-23T09:45:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.720+0000 7f179681e640 1 Processor -- start 2024-03-23T09:45:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.720+0000 7f179681e640 1 -- start start 2024-03-23T09:45:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f179681e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17900b8350 0x7f1790150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f179681e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17900cde10 0x7f17901f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f179681e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179015ca90 0x7f17900b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f179681e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900b4880 con 0x7f179015ca90 2024-03-23T09:45:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f179681e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900b4e70 con 0x7f17900cde10 2024-03-23T09:45:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f179681e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17900b5150 con 0x7f17900b8350 2024-03-23T09:45:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f1794dbb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179015ca90 0x7f17900b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f1794dbb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179015ca90 0x7f17900b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45354/0 (socket says 172.21.15.47:45354) 2024-03-23T09:45:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f1794dbb640 1 -- 172.21.15.47:0/1502754464 learned_addr learned my addr 172.21.15.47:0/1502754464 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f178f7fe640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17900cde10 0x7f17901f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f178ffff640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17900b8350 0x7f1790150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f1794dbb640 1 -- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17900b8350 msgr2=0x7f1790150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f1794dbb640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17900b8350 0x7f1790150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f1794dbb640 1 -- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17900cde10 msgr2=0x7f17901f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f1794dbb640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17900cde10 0x7f17901f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f1794dbb640 1 -- 172.21.15.47:0/1502754464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1788009a70 con 0x7f179015ca90 2024-03-23T09:45:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.721+0000 7f178f7fe640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17900cde10 0x7f17901f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.722+0000 7f1794dbb640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179015ca90 0x7f17900b4340 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d90 tx=0x7f1788004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.722+0000 7f178cff9640 1 -- 172.21.15.47:0/1502754464 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178802cea0 con 0x7f179015ca90 2024-03-23T09:45:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.722+0000 7f178cff9640 1 -- 172.21.15.47:0/1502754464 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178803e630 con 0x7f179015ca90 2024-03-23T09:45:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.722+0000 7f179681e640 1 -- 172.21.15.47:0/1502754464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17901f5960 con 0x7f179015ca90 2024-03-23T09:45:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.722+0000 7f178cff9640 1 -- 172.21.15.47:0/1502754464 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178803e9e0 con 0x7f179015ca90 2024-03-23T09:45:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.722+0000 7f179681e640 1 -- 172.21.15.47:0/1502754464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17901f5f20 con 0x7f179015ca90 2024-03-23T09:45:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.723+0000 7f179681e640 1 -- 172.21.15.47:0/1502754464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1790072360 con 0x7f179015ca90 2024-03-23T09:45:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.723+0000 7f178cff9640 1 -- 172.21.15.47:0/1502754464 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f178803b050 con 0x7f179015ca90 2024-03-23T09:45:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.724+0000 7f178cff9640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f177c06ac00 0x7f177c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.724+0000 7f178cff9640 1 -- 172.21.15.47:0/1502754464 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17880abc60 con 0x7f179015ca90 2024-03-23T09:45:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.724+0000 7f178ffff640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f177c06ac00 0x7f177c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.724+0000 7f178ffff640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f177c06ac00 0x7f177c06d0c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1780004ea0 tx=0x7f1780009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.728+0000 7f178cff9640 1 -- 172.21.15.47:0/1502754464 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1788076e40 con 0x7f179015ca90 2024-03-23T09:45:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.944+0000 7f179681e640 1 -- 172.21.15.47:0/1502754464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1790071f40 con 0x7f179015ca90 2024-03-23T09:45:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.944+0000 7f178cff9640 1 -- 172.21.15.47:0/1502754464 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f178804e090 con 0x7f179015ca90 2024-03-23T09:45:06.944 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:06.944 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/375513546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":375513546},{"type":"v1","addr":"172.21.15.175:6837","nonce":375513546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/3650756747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3650756747},{"type":"v1","addr":"172.21.15.47:6833","nonce":3650756747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2893805543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2893805543},{"type":"v1","addr":"172.21.15.47:6835","nonce":2893805543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:43:47.389279+0000","modified":"2024-03-23T09:44:35.825401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.175:6836/2498456351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2498456351},{"type":"v1","addr":"172.21.15.175:6836","nonce":2498456351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-23T09:45:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2024-03-23T09:45:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.946+0000 7f179681e640 1 -- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f177c06ac00 msgr2=0x7f177c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.946+0000 7f179681e640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f177c06ac00 0x7f177c06d0c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1780004ea0 tx=0x7f1780009040 comp rx=0 tx=0).stop 2024-03-23T09:45:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.946+0000 7f179681e640 1 -- 172.21.15.47:0/1502754464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179015ca90 msgr2=0x7f17900b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.946+0000 7f179681e640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179015ca90 0x7f17900b4340 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d90 tx=0x7f1788004a40 comp rx=0 tx=0).stop 2024-03-23T09:45:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.946+0000 7f179681e640 1 -- 172.21.15.47:0/1502754464 shutdown_connections 2024-03-23T09:45:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.946+0000 7f179681e640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f177c06ac00 0x7f177c06d0c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.946+0000 7f179681e640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179015ca90 0x7f17900b4340 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.946+0000 7f179681e640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17900cde10 0x7f17901f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.946+0000 7f179681e640 1 --2- 172.21.15.47:0/1502754464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17900b8350 0x7f1790150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.946+0000 7f179681e640 1 -- 172.21.15.47:0/1502754464 >> 172.21.15.47:0/1502754464 conn(0x7f17900659d0 msgr2=0x7f17901571e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.946+0000 7f179681e640 1 -- 172.21.15.47:0/1502754464 shutdown_connections 2024-03-23T09:45:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:06.946+0000 7f179681e640 1 -- 172.21.15.47:0/1502754464 wait complete. 2024-03-23T09:45:06.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:45:07.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.058+0000 7f21dc916640 1 Processor -- start 2024-03-23T09:45:07.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.058+0000 7f21dc916640 1 -- start start 2024-03-23T09:45:07.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.059+0000 7f21dc916640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f21d815aa70 0x7f21d80cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.059+0000 7f21dc916640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f21d80ce9d0 0x7f21d80cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.059+0000 7f21dc916640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d815ae50 con 0x7f21d80cd620 2024-03-23T09:45:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.059+0000 7f21dc916640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d80cda90 con 0x7f21d80ce9d0 2024-03-23T09:45:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.059+0000 7f21dc916640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d80cf010 con 0x7f21d815aa70 2024-03-23T09:45:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.059+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f21d80ce9d0 0x7f21d80cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.059+0000 7f21d5d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f21d80cd620 0x7f21d80ce2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38892/0 (socket says 172.21.15.47:38892) 2024-03-23T09:45:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.059+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f21d80ce9d0 0x7f21d80cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45364/0 (socket says 172.21.15.47:45364) 2024-03-23T09:45:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.059+0000 7f21d5d9b640 1 -- 172.21.15.47:0/3994283872 learned_addr learned my addr 172.21.15.47:0/3994283872 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.060+0000 7f21d659c640 1 --2- 172.21.15.47:0/3994283872 >> v2:172.21.15.175:3300/0 conn(0x7f21d815aa70 0x7f21d80cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.060+0000 7f21d6d9d640 1 -- 172.21.15.47:0/3994283872 >> v2:172.21.15.175:3300/0 conn(0x7f21d815aa70 msgr2=0x7f21d80cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.060+0000 7f21d6d9d640 1 --2- 172.21.15.47:0/3994283872 >> v2:172.21.15.175:3300/0 conn(0x7f21d815aa70 0x7f21d80cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.060+0000 7f21d6d9d640 1 -- 172.21.15.47:0/3994283872 >> v1:172.21.15.47:6789/0 conn(0x7f21d80cd620 legacy=0x7f21d80ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.060+0000 7f21d6d9d640 1 -- 172.21.15.47:0/3994283872 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80cf2f0 con 0x7f21d80ce9d0 2024-03-23T09:45:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.060+0000 7f21d559a640 1 -- 172.21.15.47:0/3994283872 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029467843 0 0) 0x7f21d815ae50 con 0x7f21d80cd620 2024-03-23T09:45:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.060+0000 7f21d6d9d640 1 --2- 172.21.15.47:0/3994283872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d80ce9d0 0x7f21d80cbb70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f21d80cdcc0 tx=0x7f21c802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da08b2c47a01aa70 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.061+0000 7f21d559a640 1 -- 172.21.15.47:0/3994283872 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c802cd00 con 0x7f21d80ce9d0 2024-03-23T09:45:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.061+0000 7f21d559a640 1 -- 172.21.15.47:0/3994283872 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c802ce60 con 0x7f21d80ce9d0 2024-03-23T09:45:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.061+0000 7f21d559a640 1 -- 172.21.15.47:0/3994283872 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8034660 con 0x7f21d80ce9d0 2024-03-23T09:45:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.061+0000 7f21dc916640 1 -- 172.21.15.47:0/3994283872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d80ce9d0 msgr2=0x7f21d80cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.061+0000 7f21dc916640 1 --2- 172.21.15.47:0/3994283872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d80ce9d0 0x7f21d80cbb70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f21d80cdcc0 tx=0x7f21c802bf50 comp rx=0 tx=0).stop 2024-03-23T09:45:07.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.061+0000 7f21dc916640 1 -- 172.21.15.47:0/3994283872 shutdown_connections 2024-03-23T09:45:07.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.061+0000 7f21dc916640 1 --2- 172.21.15.47:0/3994283872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d80ce9d0 0x7f21d80cbb70 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.061+0000 7f21dc916640 1 --2- 172.21.15.47:0/3994283872 >> v2:172.21.15.175:3300/0 conn(0x7f21d815aa70 0x7f21d80cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.061+0000 7f21dc916640 1 -- 172.21.15.47:0/3994283872 >> 172.21.15.47:0/3994283872 conn(0x7f21d80d06f0 msgr2=0x7f21d80d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:07.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.061+0000 7f21dc916640 1 -- 172.21.15.47:0/3994283872 shutdown_connections 2024-03-23T09:45:07.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.061+0000 7f21dc916640 1 -- 172.21.15.47:0/3994283872 wait complete. 2024-03-23T09:45:07.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.062+0000 7f21dc916640 1 Processor -- start 2024-03-23T09:45:07.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.063+0000 7f21dc916640 1 -- start start 2024-03-23T09:45:07.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.063+0000 7f21dc916640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d80cd620 0x7f21d8155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.063+0000 7f21dc916640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d80ce9d0 0x7f21d815d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.063+0000 7f21dc916640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21d815aa70 0x7f21d80c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.063+0000 7f21dc916640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8156030 con 0x7f21d80ce9d0 2024-03-23T09:45:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.063+0000 7f21dc916640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d815c900 con 0x7f21d80cd620 2024-03-23T09:45:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.063+0000 7f21dc916640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d815cbe0 con 0x7f21d815aa70 2024-03-23T09:45:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.063+0000 7f21d5d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d80ce9d0 0x7f21d815d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.063+0000 7f21d5d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d80ce9d0 0x7f21d815d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45378/0 (socket says 172.21.15.47:45378) 2024-03-23T09:45:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.063+0000 7f21d5d9b640 1 -- 172.21.15.47:0/3809998696 learned_addr learned my addr 172.21.15.47:0/3809998696 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:07.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.063+0000 7f21d659c640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d80cd620 0x7f21d8155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.064+0000 7f21d6d9d640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21d815aa70 0x7f21d80c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.064+0000 7f21d5d9b640 1 -- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21d815aa70 msgr2=0x7f21d80c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.064+0000 7f21d5d9b640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21d815aa70 0x7f21d80c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.064+0000 7f21d5d9b640 1 -- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d80cd620 msgr2=0x7f21d8155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.064+0000 7f21d5d9b640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d80cd620 0x7f21d8155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.064+0000 7f21d5d9b640 1 -- 172.21.15.47:0/3809998696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c8009a70 con 0x7f21d80ce9d0 2024-03-23T09:45:07.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.064+0000 7f21d5d9b640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d80ce9d0 0x7f21d815d9a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f21c000c4a0 tx=0x7f21c000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:07.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.064+0000 7f21ceffd640 1 -- 172.21.15.47:0/3809998696 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0005ce0 con 0x7f21d80ce9d0 2024-03-23T09:45:07.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.064+0000 7f21ceffd640 1 -- 172.21.15.47:0/3809998696 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c0005e40 con 0x7f21d80ce9d0 2024-03-23T09:45:07.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.064+0000 7f21dc916640 1 -- 172.21.15.47:0/3809998696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d815cec0 con 0x7f21d80ce9d0 2024-03-23T09:45:07.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.064+0000 7f21ceffd640 1 -- 172.21.15.47:0/3809998696 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c00072b0 con 0x7f21d80ce9d0 2024-03-23T09:45:07.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.064+0000 7f21dc916640 1 -- 172.21.15.47:0/3809998696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d815d480 con 0x7f21d80ce9d0 2024-03-23T09:45:07.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.065+0000 7f21dc916640 1 -- 172.21.15.47:0/3809998696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d815d6a0 con 0x7f21d80ce9d0 2024-03-23T09:45:07.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.065+0000 7f21ceffd640 1 -- 172.21.15.47:0/3809998696 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f21c0007410 con 0x7f21d80ce9d0 2024-03-23T09:45:07.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.066+0000 7f21ceffd640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f21b806ac00 0x7f21b806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.066+0000 7f21ceffd640 1 -- 172.21.15.47:0/3809998696 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21c0088900 con 0x7f21d80ce9d0 2024-03-23T09:45:07.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.066+0000 7f21d659c640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f21b806ac00 0x7f21b806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.067+0000 7f21d659c640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f21b806ac00 0x7f21b806d0c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f21d0006fd0 tx=0x7f21d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:07.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.070+0000 7f21ceffd640 1 -- 172.21.15.47:0/3809998696 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f21c0053ae0 con 0x7f21d80ce9d0 2024-03-23T09:45:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.252+0000 7f21dc916640 1 -- 172.21.15.47:0/3809998696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21d8066be0 con 0x7f21d80ce9d0 2024-03-23T09:45:07.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.253+0000 7f21ceffd640 1 -- 172.21.15.47:0/3809998696 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16097 (secure 0 0 0) 0x7f21c0017090 con 0x7f21d80ce9d0 2024-03-23T09:45:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:07.254 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":58,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:43:48.390657+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:43:45.646160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-23T09:43:46.351634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:45:07.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.256+0000 7f21dc916640 1 -- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f21b806ac00 msgr2=0x7f21b806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.256+0000 7f21dc916640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f21b806ac00 0x7f21b806d0c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f21d0006fd0 tx=0x7f21d0008040 comp rx=0 tx=0).stop 2024-03-23T09:45:07.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.256+0000 7f21dc916640 1 -- 172.21.15.47:0/3809998696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d80ce9d0 msgr2=0x7f21d815d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.256+0000 7f21dc916640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d80ce9d0 0x7f21d815d9a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f21c000c4a0 tx=0x7f21c000c960 comp rx=0 tx=0).stop 2024-03-23T09:45:07.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.256+0000 7f21dc916640 1 -- 172.21.15.47:0/3809998696 shutdown_connections 2024-03-23T09:45:07.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.256+0000 7f21dc916640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f21b806ac00 0x7f21b806d0c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.256+0000 7f21dc916640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21d815aa70 0x7f21d80c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.256+0000 7f21dc916640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d80ce9d0 0x7f21d815d9a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.256+0000 7f21dc916640 1 --2- 172.21.15.47:0/3809998696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d80cd620 0x7f21d8155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.256+0000 7f21dc916640 1 -- 172.21.15.47:0/3809998696 >> 172.21.15.47:0/3809998696 conn(0x7f21d80d06f0 msgr2=0x7f21d806e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:07.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.256+0000 7f21dc916640 1 -- 172.21.15.47:0/3809998696 shutdown_connections 2024-03-23T09:45:07.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.256+0000 7f21dc916640 1 -- 172.21.15.47:0/3809998696 wait complete. 2024-03-23T09:45:07.267 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:45:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.369+0000 7fee52d31640 1 Processor -- start 2024-03-23T09:45:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.369+0000 7fee52d31640 1 -- start start 2024-03-23T09:45:07.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.369+0000 7fee52d31640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fee4c0cf690 0x7fee4c0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.369+0000 7fee52d31640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c15d0e0 con 0x7fee4c0cf210 2024-03-23T09:45:07.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.369+0000 7fee52d31640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c0d1540 con 0x7fee4c15cd00 2024-03-23T09:45:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.369+0000 7fee52d31640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4c0d1820 con 0x7fee4c0cf690 2024-03-23T09:45:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fee4c0cf210 0x7fee4c0b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38894/0 (socket says 172.21.15.47:38894) 2024-03-23T09:45:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4bfff640 1 -- 172.21.15.47:0/207539515 learned_addr learned my addr 172.21.15.47:0/207539515 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee512ce640 1 --2- 172.21.15.47:0/207539515 >> v2:172.21.15.175:3300/0 conn(0x7fee4c0cf690 0x7fee4c0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946275751 0 0) 0x7fee4c15d0e0 con 0x7fee4c0cf210 2024-03-23T09:45:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee3c003520 con 0x7fee4c0cf210 2024-03-23T09:45:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 139893161 0 0) 0x7fee3c003520 con 0x7fee4c0cf210 2024-03-23T09:45:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 >> v2:172.21.15.175:3300/0 conn(0x7fee4c0cf690 msgr2=0x7fee4c0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 --2- 172.21.15.47:0/207539515 >> v2:172.21.15.175:3300/0 conn(0x7fee4c0cf690 0x7fee4c0b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 >> v1:172.21.15.175:6789/0 conn(0x7fee4c15cd00 legacy=0x7fee4c0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0d1b00 con 0x7fee4c0cf210 2024-03-23T09:45:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fee38001670 con 0x7fee4c0cf210 2024-03-23T09:45:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 >> v1:172.21.15.47:6789/0 conn(0x7fee4c0cf210 legacy=0x7fee4c0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 --2- 172.21.15.47:0/207539515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee3c003970 0x7fee3c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 --2- 172.21.15.47:0/207539515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c0047c0 0x7fee3c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 --2- 172.21.15.47:0/207539515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c005180 0x7fee3c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0d1820 con 0x7fee3c005180 2024-03-23T09:45:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c00c120 con 0x7fee3c003970 2024-03-23T09:45:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c00c2e0 con 0x7fee3c0047c0 2024-03-23T09:45:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee512ce640 1 --2- 172.21.15.47:0/207539515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c005180 0x7fee3c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee50acd640 1 --2- 172.21.15.47:0/207539515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee3c003970 0x7fee3c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.370+0000 7fee4bfff640 1 --2- 172.21.15.47:0/207539515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c0047c0 0x7fee3c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.371+0000 7fee512ce640 1 -- 172.21.15.47:0/207539515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c0047c0 msgr2=0x7fee3c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.371+0000 7fee512ce640 1 --2- 172.21.15.47:0/207539515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c0047c0 0x7fee3c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.371+0000 7fee512ce640 1 -- 172.21.15.47:0/207539515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee3c003970 msgr2=0x7fee3c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.371+0000 7fee512ce640 1 --2- 172.21.15.47:0/207539515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee3c003970 0x7fee3c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.371+0000 7fee512ce640 1 -- 172.21.15.47:0/207539515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c003520 con 0x7fee3c005180 2024-03-23T09:45:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.371+0000 7fee512ce640 1 -- 172.21.15.47:0/207539515 reap_dead start 2024-03-23T09:45:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.371+0000 7fee512ce640 1 --2- 172.21.15.47:0/207539515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c005180 0x7fee3c00b980 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fee44009fc0 tx=0x7fee3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229fa15431b34518 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.371+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4402cc60 con 0x7fee3c005180 2024-03-23T09:45:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.372+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4400f040 con 0x7fee3c005180 2024-03-23T09:45:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.372+0000 7fee4b7fe640 1 -- 172.21.15.47:0/207539515 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44003990 con 0x7fee3c005180 2024-03-23T09:45:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.372+0000 7fee52d31640 1 -- 172.21.15.47:0/207539515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c005180 msgr2=0x7fee3c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.372+0000 7fee52d31640 1 --2- 172.21.15.47:0/207539515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c005180 0x7fee3c00b980 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fee44009fc0 tx=0x7fee3c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.372+0000 7fee52d31640 1 -- 172.21.15.47:0/207539515 shutdown_connections 2024-03-23T09:45:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.372+0000 7fee52d31640 1 --2- 172.21.15.47:0/207539515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c005180 0x7fee3c00b980 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.372+0000 7fee52d31640 1 -- 172.21.15.47:0/207539515 >> 172.21.15.47:0/207539515 conn(0x7fee4c06b160 msgr2=0x7fee4c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.372+0000 7fee52d31640 1 -- 172.21.15.47:0/207539515 shutdown_connections 2024-03-23T09:45:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.372+0000 7fee52d31640 1 -- 172.21.15.47:0/207539515 wait complete. 2024-03-23T09:45:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.373+0000 7fee52d31640 1 Processor -- start 2024-03-23T09:45:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.373+0000 7fee52d31640 1 -- start start 2024-03-23T09:45:07.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.374+0000 7fee52d31640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c005180 0x7fee4c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.374+0000 7fee52d31640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15a070 0x7fee4c0b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.374+0000 7fee52d31640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c0b3da0 0x7fee4c0c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.374+0000 7fee52d31640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c159db0 con 0x7fee4c15a070 2024-03-23T09:45:07.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.374+0000 7fee52d31640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0b4220 con 0x7fee4c0b3da0 2024-03-23T09:45:07.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.374+0000 7fee52d31640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0b4500 con 0x7fee3c005180 2024-03-23T09:45:07.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.374+0000 7fee4bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15a070 0x7fee4c0b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.374+0000 7fee4bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15a070 0x7fee4c0b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45406/0 (socket says 172.21.15.47:45406) 2024-03-23T09:45:07.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.374+0000 7fee4bfff640 1 -- 172.21.15.47:0/1855013630 learned_addr learned my addr 172.21.15.47:0/1855013630 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:07.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.374+0000 7fee512ce640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c0b3da0 0x7fee4c0c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.374+0000 7fee50acd640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c005180 0x7fee4c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.375+0000 7fee4bfff640 1 -- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c005180 msgr2=0x7fee4c159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.375+0000 7fee4bfff640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c005180 0x7fee4c159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.375+0000 7fee4bfff640 1 -- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c0b3da0 msgr2=0x7fee4c0c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.375+0000 7fee4bfff640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c0b3da0 0x7fee4c0c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.375+0000 7fee4bfff640 1 -- 172.21.15.47:0/1855013630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0b47e0 con 0x7fee4c15a070 2024-03-23T09:45:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.375+0000 7fee50acd640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c005180 0x7fee4c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.375+0000 7fee4bfff640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15a070 0x7fee4c0b3860 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fee38000fb0 tx=0x7fee38007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.375+0000 7fee497fa640 1 -- 172.21.15.47:0/1855013630 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3800f990 con 0x7fee4c15a070 2024-03-23T09:45:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.375+0000 7fee497fa640 1 -- 172.21.15.47:0/1855013630 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3800fb10 con 0x7fee4c15a070 2024-03-23T09:45:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.375+0000 7fee52d31640 1 -- 172.21.15.47:0/1855013630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c0c6530 con 0x7fee4c15a070 2024-03-23T09:45:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.375+0000 7fee497fa640 1 -- 172.21.15.47:0/1855013630 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3800fec0 con 0x7fee4c15a070 2024-03-23T09:45:07.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.375+0000 7fee52d31640 1 -- 172.21.15.47:0/1855013630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c0c8e50 con 0x7fee4c15a070 2024-03-23T09:45:07.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.376+0000 7fee52d31640 1 -- 172.21.15.47:0/1855013630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c072360 con 0x7fee4c15a070 2024-03-23T09:45:07.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.376+0000 7fee497fa640 1 -- 172.21.15.47:0/1855013630 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fee3801d460 con 0x7fee4c15a070 2024-03-23T09:45:07.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.377+0000 7fee497fa640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3006ac00 0x7fee3006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.377+0000 7fee497fa640 1 -- 172.21.15.47:0/1855013630 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee38089430 con 0x7fee4c15a070 2024-03-23T09:45:07.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.377+0000 7fee50acd640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3006ac00 0x7fee3006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.377+0000 7fee50acd640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3006ac00 0x7fee3006d0c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fee40000c00 tx=0x7fee40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:07.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.381+0000 7fee497fa640 1 -- 172.21.15.47:0/1855013630 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fee38054600 con 0x7fee4c15a070 2024-03-23T09:45:07.583 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:45:07.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:45:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.600+0000 7fee52d31640 1 -- 172.21.15.47:0/1855013630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fee4c0c6720 con 0x7fee4c15a070 2024-03-23T09:45:07.600 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:45:07.601+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:45:07.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.612+0000 7fee497fa640 1 -- 172.21.15.47:0/1855013630 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7fee38057c20 con 0x7fee4c15a070 2024-03-23T09:45:07.612 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:45:07.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.614+0000 7fee52d31640 1 -- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3006ac00 msgr2=0x7fee3006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.614+0000 7fee52d31640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3006ac00 0x7fee3006d0c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fee40000c00 tx=0x7fee40009040 comp rx=0 tx=0).stop 2024-03-23T09:45:07.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.614+0000 7fee52d31640 1 -- 172.21.15.47:0/1855013630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15a070 msgr2=0x7fee4c0b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.614+0000 7fee52d31640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15a070 0x7fee4c0b3860 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fee38000fb0 tx=0x7fee38007720 comp rx=0 tx=0).stop 2024-03-23T09:45:07.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.614+0000 7fee52d31640 1 -- 172.21.15.47:0/1855013630 shutdown_connections 2024-03-23T09:45:07.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.614+0000 7fee52d31640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3006ac00 0x7fee3006d0c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.614+0000 7fee52d31640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c0b3da0 0x7fee4c0c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.614+0000 7fee52d31640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee4c15a070 0x7fee4c0b3860 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.614+0000 7fee52d31640 1 --2- 172.21.15.47:0/1855013630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c005180 0x7fee4c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.614+0000 7fee52d31640 1 -- 172.21.15.47:0/1855013630 >> 172.21.15.47:0/1855013630 conn(0x7fee4c06b160 msgr2=0x7fee4c15af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:07.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.614+0000 7fee52d31640 1 -- 172.21.15.47:0/1855013630 shutdown_connections 2024-03-23T09:45:07.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.614+0000 7fee52d31640 1 -- 172.21.15.47:0/1855013630 wait complete. 2024-03-23T09:45:07.616 INFO:tasks.ceph.mds.d.smithi175.stderr: -6230> 2024-03-23T09:43:48.967+0000 7fa917ab8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:45:07.619 INFO:tasks.ceph.mds.d.smithi175.stderr: -4488> 2024-03-23T09:43:59.488+0000 7fa9142b1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6369=0-4194304@1} caps={6369=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6369 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x559dc139fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T09:45:07.621 INFO:tasks.ceph.mds.d.smithi175.stderr: -2704> 2024-03-23T09:44:30.779+0000 7fa9142b1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-03-23T09:43:58.266000+0000 1=1+0) (iversion lock) caps={6369=pAsLsXsFs/-@2},l=6369(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x559dc139fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T09:45:07.624 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T09:45:07.649 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-03-23T09:45:07.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.771+0000 7f0d826e0640 1 Processor -- start 2024-03-23T09:45:07.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.771+0000 7f0d826e0640 1 -- start start 2024-03-23T09:45:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d826e0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0d7c0d23a0 0x7f0d7c0d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d826e0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7c0d10a0 con 0x7f0d7c15cee0 2024-03-23T09:45:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d826e0640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7c0d1300 con 0x7f0d7c0b8300 2024-03-23T09:45:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d826e0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c0d15e0 con 0x7f0d7c0d23a0 2024-03-23T09:45:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0d7c15cee0 0x7f0d7c0d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38904/0 (socket says 172.21.15.47:38904) 2024-03-23T09:45:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2946211480 learned_addr learned my addr 172.21.15.47:0/2946211480 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7b7fe640 1 --2- 172.21.15.47:0/2946211480 >> v2:172.21.15.175:3300/0 conn(0x7f0d7c0d23a0 0x7f0d7c0d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441643890 0 0) 0x7f0d7c0d10a0 con 0x7f0d7c15cee0 2024-03-23T09:45:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d64003520 con 0x7f0d7c15cee0 2024-03-23T09:45:07.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 853463991 0 0) 0x7f0d64003520 con 0x7f0d7c15cee0 2024-03-23T09:45:07.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 >> v2:172.21.15.175:3300/0 conn(0x7f0d7c0d23a0 msgr2=0x7f0d7c0d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:07.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 --2- 172.21.15.47:0/2946211480 >> v2:172.21.15.175:3300/0 conn(0x7f0d7c0d23a0 0x7f0d7c0d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 >> v1:172.21.15.175:6789/0 conn(0x7f0d7c0b8300 legacy=0x7f0d7c0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0d18c0 con 0x7f0d7c15cee0 2024-03-23T09:45:07.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0d70001670 con 0x7f0d7c15cee0 2024-03-23T09:45:07.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 >> v1:172.21.15.47:6789/0 conn(0x7f0d7c15cee0 legacy=0x7f0d7c0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 --2- 172.21.15.47:0/2946211480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d64003970 0x7f0d64003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 --2- 172.21.15.47:0/2946211480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d640047c0 0x7f0d64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 --2- 172.21.15.47:0/2946211480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d64005180 0x7f0d6400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0d15e0 con 0x7f0d64003970 2024-03-23T09:45:07.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6400c020 con 0x7f0d64005180 2024-03-23T09:45:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6400c1e0 con 0x7f0d640047c0 2024-03-23T09:45:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.772+0000 7f0d7bfff640 1 --2- 172.21.15.47:0/2946211480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d64003970 0x7f0d64003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.773+0000 7f0d7b7fe640 1 --2- 172.21.15.47:0/2946211480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d640047c0 0x7f0d64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.773+0000 7f0d80c7d640 1 --2- 172.21.15.47:0/2946211480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d64005180 0x7f0d6400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.773+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2946211480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d640047c0 msgr2=0x7f0d64004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.773+0000 7f0d7bfff640 1 --2- 172.21.15.47:0/2946211480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d640047c0 0x7f0d64004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.773+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2946211480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d64005180 msgr2=0x7f0d6400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.773+0000 7f0d7bfff640 1 --2- 172.21.15.47:0/2946211480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d64005180 0x7f0d6400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.773+0000 7f0d80c7d640 1 -- 172.21.15.47:0/2946211480 reap_dead start 2024-03-23T09:45:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.773+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2946211480 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d64003520 con 0x7f0d64003970 2024-03-23T09:45:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.773+0000 7f0d7b7fe640 1 --2- 172.21.15.47:0/2946211480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d640047c0 0x7f0d64004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.773+0000 7f0d7bfff640 1 --2- 172.21.15.47:0/2946211480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d64003970 0x7f0d64003d90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000c240 tx=0x7f0d640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd417580340f87f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.774+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70005ad0 con 0x7f0d64003970 2024-03-23T09:45:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.774+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d70005c30 con 0x7f0d64003970 2024-03-23T09:45:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.774+0000 7f0d7affd640 1 -- 172.21.15.47:0/2946211480 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7002caf0 con 0x7f0d64003970 2024-03-23T09:45:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.774+0000 7f0d826e0640 1 -- 172.21.15.47:0/2946211480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d64003970 msgr2=0x7f0d64003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.774+0000 7f0d826e0640 1 --2- 172.21.15.47:0/2946211480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d64003970 0x7f0d64003d90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000c240 tx=0x7f0d640057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.774+0000 7f0d826e0640 1 -- 172.21.15.47:0/2946211480 shutdown_connections 2024-03-23T09:45:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.774+0000 7f0d826e0640 1 --2- 172.21.15.47:0/2946211480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d64003970 0x7f0d64003d90 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.774+0000 7f0d826e0640 1 -- 172.21.15.47:0/2946211480 >> 172.21.15.47:0/2946211480 conn(0x7f0d7c065df0 msgr2=0x7f0d7c066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.774+0000 7f0d826e0640 1 -- 172.21.15.47:0/2946211480 shutdown_connections 2024-03-23T09:45:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.774+0000 7f0d826e0640 1 -- 172.21.15.47:0/2946211480 wait complete. 2024-03-23T09:45:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.775+0000 7f0d826e0640 1 Processor -- start 2024-03-23T09:45:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.776+0000 7f0d826e0640 1 -- start start 2024-03-23T09:45:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.776+0000 7f0d826e0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d64003970 0x7f0d7c0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.776+0000 7f0d826e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0c99b0 0x7f0d7c151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.776+0000 7f0d826e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d7c0c53a0 0x7f0d7c0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.776+0000 7f0d826e0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0c9f00 con 0x7f0d64003970 2024-03-23T09:45:07.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.776+0000 7f0d826e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0ca1e0 con 0x7f0d7c0c99b0 2024-03-23T09:45:07.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.776+0000 7f0d826e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0ca4c0 con 0x7f0d7c0c53a0 2024-03-23T09:45:07.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.776+0000 7f0d7bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d64003970 0x7f0d7c0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.776+0000 7f0d7bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d64003970 0x7f0d7c0cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45426/0 (socket says 172.21.15.47:45426) 2024-03-23T09:45:07.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.776+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2863217899 learned_addr learned my addr 172.21.15.47:0/2863217899 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:07.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.776+0000 7f0d80c7d640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d7c0c53a0 0x7f0d7c0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.777+0000 7f0d7b7fe640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0c99b0 0x7f0d7c151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.777+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d7c0c53a0 msgr2=0x7f0d7c0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.777+0000 7f0d7bfff640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d7c0c53a0 0x7f0d7c0cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.777+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0c99b0 msgr2=0x7f0d7c151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:07.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.777+0000 7f0d7bfff640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0c99b0 0x7f0d7c151d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:07.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.777+0000 7f0d7bfff640 1 -- 172.21.15.47:0/2863217899 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0ca7a0 con 0x7f0d64003970 2024-03-23T09:45:07.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.777+0000 7f0d7bfff640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d64003970 0x7f0d7c0cafa0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0d70001930 tx=0x7f0d7002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:07.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.777+0000 7f0d78ff9640 1 -- 172.21.15.47:0/2863217899 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7003c040 con 0x7f0d64003970 2024-03-23T09:45:07.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.777+0000 7f0d78ff9640 1 -- 172.21.15.47:0/2863217899 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7002ce00 con 0x7f0d64003970 2024-03-23T09:45:07.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.777+0000 7f0d826e0640 1 -- 172.21.15.47:0/2863217899 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c0caa80 con 0x7f0d64003970 2024-03-23T09:45:07.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.777+0000 7f0d78ff9640 1 -- 172.21.15.47:0/2863217899 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d700072b0 con 0x7f0d64003970 2024-03-23T09:45:07.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.777+0000 7f0d826e0640 1 -- 172.21.15.47:0/2863217899 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c0c5820 con 0x7f0d64003970 2024-03-23T09:45:07.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.778+0000 7f0d826e0640 1 -- 172.21.15.47:0/2863217899 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c072360 con 0x7f0d64003970 2024-03-23T09:45:07.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.778+0000 7f0d78ff9640 1 -- 172.21.15.47:0/2863217899 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0d70032070 con 0x7f0d64003970 2024-03-23T09:45:07.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.779+0000 7f0d78ff9640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d6806ac30 0x7f0d6806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:07.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.779+0000 7f0d78ff9640 1 -- 172.21.15.47:0/2863217899 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0d700aa970 con 0x7f0d64003970 2024-03-23T09:45:07.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.779+0000 7f0d7b7fe640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d6806ac30 0x7f0d6806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:07.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.779+0000 7f0d7b7fe640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d6806ac30 0x7f0d6806d0f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0c8620 tx=0x7f0d6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:07.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:07.783+0000 7f0d78ff9640 1 -- 172.21.15.47:0/2863217899 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0d70075af0 con 0x7f0d64003970 2024-03-23T09:45:08.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.001+0000 7f0d826e0640 1 -- 172.21.15.47:0/2863217899 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0d7c066be0 con 0x7f0d64003970 2024-03-23T09:45:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.614+0000 7f0d78ff9640 1 -- 172.21.15.47:0/2863217899 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0d70079110 con 0x7f0d64003970 2024-03-23T09:45:08.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.616+0000 7f0d826e0640 1 -- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d6806ac30 msgr2=0x7f0d6806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:08.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.616+0000 7f0d826e0640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d6806ac30 0x7f0d6806d0f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0c8620 tx=0x7f0d6c008040 comp rx=0 tx=0).stop 2024-03-23T09:45:08.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.616+0000 7f0d826e0640 1 -- 172.21.15.47:0/2863217899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d64003970 msgr2=0x7f0d7c0cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:08.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.616+0000 7f0d826e0640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d64003970 0x7f0d7c0cafa0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0d70001930 tx=0x7f0d7002e070 comp rx=0 tx=0).stop 2024-03-23T09:45:08.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.616+0000 7f0d826e0640 1 -- 172.21.15.47:0/2863217899 shutdown_connections 2024-03-23T09:45:08.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.616+0000 7f0d826e0640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d6806ac30 0x7f0d6806d0f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:08.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.616+0000 7f0d826e0640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d7c0c53a0 0x7f0d7c0cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:08.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.616+0000 7f0d826e0640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d7c0c99b0 0x7f0d7c151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:08.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.616+0000 7f0d826e0640 1 --2- 172.21.15.47:0/2863217899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d64003970 0x7f0d7c0cafa0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:08.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.616+0000 7f0d826e0640 1 -- 172.21.15.47:0/2863217899 >> 172.21.15.47:0/2863217899 conn(0x7f0d7c065df0 msgr2=0x7f0d7c0cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:08.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.616+0000 7f0d826e0640 1 -- 172.21.15.47:0/2863217899 shutdown_connections 2024-03-23T09:45:08.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.616+0000 7f0d826e0640 1 -- 172.21.15.47:0/2863217899 wait complete. 2024-03-23T09:45:08.628 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T09:45:08.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.737+0000 7f3bb008f640 1 Processor -- start 2024-03-23T09:45:08.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.738+0000 7f3bb008f640 1 -- start start 2024-03-23T09:45:08.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.738+0000 7f3bb008f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3ba815cbe0 0x7f3ba80cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:08.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.738+0000 7f3bb008f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ba80b0db0 0x7f3ba80cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:08.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.738+0000 7f3bb008f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba80b1400 con 0x7f3ba80cad40 2024-03-23T09:45:08.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.738+0000 7f3bb008f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba80b16e0 con 0x7f3ba80b0db0 2024-03-23T09:45:08.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.738+0000 7f3bb008f640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba80cedb0 con 0x7f3ba815cbe0 2024-03-23T09:45:08.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.738+0000 7f3bae62c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ba80b0db0 0x7f3ba80cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:08.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.739+0000 7f3bae62c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ba80b0db0 0x7f3ba80cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45430/0 (socket says 172.21.15.47:45430) 2024-03-23T09:45:08.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.739+0000 7f3bae62c640 1 -- 172.21.15.47:0/1448218027 learned_addr learned my addr 172.21.15.47:0/1448218027 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:08.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.739+0000 7f3bade2b640 1 --2- 172.21.15.47:0/1448218027 >> v2:172.21.15.175:3300/0 conn(0x7f3ba815cbe0 0x7f3ba80cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:08.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.739+0000 7f3bae62c640 1 -- 172.21.15.47:0/1448218027 >> v2:172.21.15.175:3300/0 conn(0x7f3ba815cbe0 msgr2=0x7f3ba80cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:08.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.739+0000 7f3bae62c640 1 --2- 172.21.15.47:0/1448218027 >> v2:172.21.15.175:3300/0 conn(0x7f3ba815cbe0 0x7f3ba80cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:08.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.739+0000 7f3bae62c640 1 -- 172.21.15.47:0/1448218027 >> v1:172.21.15.175:6789/0 conn(0x7f3ba80cad40 legacy=0x7f3ba80b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:08.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.739+0000 7f3bae62c640 1 -- 172.21.15.47:0/1448218027 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80cf090 con 0x7f3ba80b0db0 2024-03-23T09:45:08.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.740+0000 7f3bae62c640 1 --2- 172.21.15.47:0/1448218027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba80b0db0 0x7f3ba80cd610 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80cdc70 tx=0x7f3b9802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0b9b5229d8ee81 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:08.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.740+0000 7f3bace29640 1 -- 172.21.15.47:0/1448218027 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9802ca80 con 0x7f3ba80b0db0 2024-03-23T09:45:08.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.740+0000 7f3bace29640 1 -- 172.21.15.47:0/1448218027 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9802cbe0 con 0x7f3ba80b0db0 2024-03-23T09:45:08.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.740+0000 7f3bace29640 1 -- 172.21.15.47:0/1448218027 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b980343f0 con 0x7f3ba80b0db0 2024-03-23T09:45:08.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.740+0000 7f3bb008f640 1 -- 172.21.15.47:0/1448218027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba80b0db0 msgr2=0x7f3ba80cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:08.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.740+0000 7f3bb008f640 1 --2- 172.21.15.47:0/1448218027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba80b0db0 0x7f3ba80cd610 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80cdc70 tx=0x7f3b9802bee0 comp rx=0 tx=0).stop 2024-03-23T09:45:08.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.740+0000 7f3bb008f640 1 -- 172.21.15.47:0/1448218027 shutdown_connections 2024-03-23T09:45:08.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.740+0000 7f3bb008f640 1 --2- 172.21.15.47:0/1448218027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba80b0db0 0x7f3ba80cd610 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:08.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.740+0000 7f3bb008f640 1 --2- 172.21.15.47:0/1448218027 >> v2:172.21.15.175:3300/0 conn(0x7f3ba815cbe0 0x7f3ba80cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:08.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.740+0000 7f3bb008f640 1 -- 172.21.15.47:0/1448218027 >> 172.21.15.47:0/1448218027 conn(0x7f3ba80d06f0 msgr2=0x7f3ba80d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:08.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.740+0000 7f3bb008f640 1 -- 172.21.15.47:0/1448218027 shutdown_connections 2024-03-23T09:45:08.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.740+0000 7f3bb008f640 1 -- 172.21.15.47:0/1448218027 wait complete. 2024-03-23T09:45:08.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.742+0000 7f3bb008f640 1 Processor -- start 2024-03-23T09:45:08.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.742+0000 7f3bb008f640 1 -- start start 2024-03-23T09:45:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.742+0000 7f3bb008f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ba80b0db0 0x7f3ba815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.742+0000 7f3bb008f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ba80cad40 0x7f3ba814f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.742+0000 7f3bb008f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba815cbe0 0x7f3ba814fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.742+0000 7f3bb008f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80c69c0 con 0x7f3ba815cbe0 2024-03-23T09:45:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.742+0000 7f3bb008f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80c5d40 con 0x7f3ba80cad40 2024-03-23T09:45:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.742+0000 7f3bb008f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80c6020 con 0x7f3ba80b0db0 2024-03-23T09:45:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.742+0000 7f3bae62c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba815cbe0 0x7f3ba814fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.742+0000 7f3bae62c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba815cbe0 0x7f3ba814fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45446/0 (socket says 172.21.15.47:45446) 2024-03-23T09:45:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.742+0000 7f3bae62c640 1 -- 172.21.15.47:0/2982381674 learned_addr learned my addr 172.21.15.47:0/2982381674 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3bad62a640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ba80cad40 0x7f3ba814f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3bade2b640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ba80b0db0 0x7f3ba815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3bae62c640 1 -- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ba80b0db0 msgr2=0x7f3ba815e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3bae62c640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ba80b0db0 0x7f3ba815e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3bae62c640 1 -- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ba80cad40 msgr2=0x7f3ba814f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3bae62c640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ba80cad40 0x7f3ba814f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3bae62c640 1 -- 172.21.15.47:0/2982381674 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b98009b30 con 0x7f3ba815cbe0 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3bad62a640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ba80cad40 0x7f3ba814f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3bae62c640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba815cbe0 0x7f3ba814fec0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3b98036040 tx=0x7f3b98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3ba67fc640 1 -- 172.21.15.47:0/2982381674 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98009510 con 0x7f3ba815cbe0 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3ba67fc640 1 -- 172.21.15.47:0/2982381674 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9803e450 con 0x7f3ba815cbe0 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3bb008f640 1 -- 172.21.15.47:0/2982381674 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba80c6300 con 0x7f3ba815cbe0 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3ba67fc640 1 -- 172.21.15.47:0/2982381674 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9803e800 con 0x7f3ba815cbe0 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.743+0000 7f3bb008f640 1 -- 172.21.15.47:0/2982381674 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba80c59f0 con 0x7f3ba815cbe0 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.744+0000 7f3bb008f640 1 -- 172.21.15.47:0/2982381674 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba80c6b00 con 0x7f3ba815cbe0 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.745+0000 7f3ba67fc640 1 -- 172.21.15.47:0/2982381674 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3b9802cd50 con 0x7f3ba815cbe0 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.745+0000 7f3ba67fc640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b9406ac00 0x7f3b9406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.745+0000 7f3ba67fc640 1 -- 172.21.15.47:0/2982381674 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3b980abec0 con 0x7f3ba815cbe0 2024-03-23T09:45:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.745+0000 7f3bade2b640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b9406ac00 0x7f3b9406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:08.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.746+0000 7f3bade2b640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b9406ac00 0x7f3b9406d0c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0006fd0 tx=0x7f3ba0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:08.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.749+0000 7f3ba67fc640 1 -- 172.21.15.47:0/2982381674 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3ba80c6b00 con 0x7f3ba815cbe0 2024-03-23T09:45:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:08.929+0000 7f3bb008f640 1 -- 172.21.15.47:0/2982381674 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3ba80c6b00 con 0x7f3ba815cbe0 2024-03-23T09:45:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:09.899+0000 7f3ba67fc640 1 -- 172.21.15.47:0/2982381674 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3ba80c6b00 con 0x7f3ba815cbe0 2024-03-23T09:45:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-03-23T09:45:09.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:09.904+0000 7f3bb008f640 1 -- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b9406ac00 msgr2=0x7f3b9406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:09.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:09.904+0000 7f3bb008f640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b9406ac00 0x7f3b9406d0c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0006fd0 tx=0x7f3ba0008040 comp rx=0 tx=0).stop 2024-03-23T09:45:09.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:09.904+0000 7f3bb008f640 1 -- 172.21.15.47:0/2982381674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba815cbe0 msgr2=0x7f3ba814fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:09.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:09.904+0000 7f3bb008f640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba815cbe0 0x7f3ba814fec0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3b98036040 tx=0x7f3b98004a40 comp rx=0 tx=0).stop 2024-03-23T09:45:09.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:09.904+0000 7f3bb008f640 1 -- 172.21.15.47:0/2982381674 shutdown_connections 2024-03-23T09:45:09.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:09.904+0000 7f3bb008f640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b9406ac00 0x7f3b9406d0c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:09.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:09.904+0000 7f3bb008f640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba815cbe0 0x7f3ba814fec0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:09.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:09.904+0000 7f3bb008f640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ba80cad40 0x7f3ba814f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:09.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:09.904+0000 7f3bb008f640 1 --2- 172.21.15.47:0/2982381674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ba80b0db0 0x7f3ba815e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:09.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:09.904+0000 7f3bb008f640 1 -- 172.21.15.47:0/2982381674 >> 172.21.15.47:0/2982381674 conn(0x7f3ba80d06f0 msgr2=0x7f3ba80cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:09.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:09.904+0000 7f3bb008f640 1 -- 172.21.15.47:0/2982381674 shutdown_connections 2024-03-23T09:45:09.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:09.905+0000 7f3bb008f640 1 -- 172.21.15.47:0/2982381674 wait complete. 2024-03-23T09:45:09.918 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T09:45:10.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.032+0000 7fd688819640 1 Processor -- start 2024-03-23T09:45:10.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.032+0000 7fd688819640 1 -- start start 2024-03-23T09:45:10.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.032+0000 7fd688819640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd68415a860 0x7fd684161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:10.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.033+0000 7fd688819640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd68415d0f0 0x7fd68415d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:10.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.033+0000 7fd688819640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd684160bd0 con 0x7fd6840cf190 2024-03-23T09:45:10.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.033+0000 7fd688819640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6840d2790 con 0x7fd68415d0f0 2024-03-23T09:45:10.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.033+0000 7fd688819640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68415dbb0 con 0x7fd68415a860 2024-03-23T09:45:10.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.033+0000 7fd681d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd68415d0f0 0x7fd68415d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:10.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.033+0000 7fd681d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd68415d0f0 0x7fd68415d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45462/0 (socket says 172.21.15.47:45462) 2024-03-23T09:45:10.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.033+0000 7fd681d9b640 1 -- 172.21.15.47:0/559352487 learned_addr learned my addr 172.21.15.47:0/559352487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:10.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.033+0000 7fd681d9b640 1 -- 172.21.15.47:0/559352487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd68415a860 msgr2=0x7fd684161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:10.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.033+0000 7fd68259c640 1 --2- 172.21.15.47:0/559352487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd68415a860 0x7fd684161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:10.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.033+0000 7fd681d9b640 1 --2- 172.21.15.47:0/559352487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd68415a860 0x7fd684161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:10.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.033+0000 7fd681d9b640 1 -- 172.21.15.47:0/559352487 >> v1:172.21.15.175:6789/0 conn(0x7fd6840cf190 legacy=0x7fd6840cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:10.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.033+0000 7fd681d9b640 1 -- 172.21.15.47:0/559352487 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68415de90 con 0x7fd68415d0f0 2024-03-23T09:45:10.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.034+0000 7fd681d9b640 1 --2- 172.21.15.47:0/559352487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd68415d0f0 0x7fd68415d550 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd678009870 tx=0x7fd67802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12ec29ae51cf8dd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:10.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.034+0000 7fd68159a640 1 -- 172.21.15.47:0/559352487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678031640 con 0x7fd68415d0f0 2024-03-23T09:45:10.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.034+0000 7fd68159a640 1 -- 172.21.15.47:0/559352487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6780317a0 con 0x7fd68415d0f0 2024-03-23T09:45:10.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.034+0000 7fd68159a640 1 -- 172.21.15.47:0/559352487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678031b30 con 0x7fd68415d0f0 2024-03-23T09:45:10.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.034+0000 7fd688819640 1 -- 172.21.15.47:0/559352487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd68415d0f0 msgr2=0x7fd68415d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:10.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.034+0000 7fd688819640 1 --2- 172.21.15.47:0/559352487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd68415d0f0 0x7fd68415d550 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd678009870 tx=0x7fd67802b8f0 comp rx=0 tx=0).stop 2024-03-23T09:45:10.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.035+0000 7fd688819640 1 -- 172.21.15.47:0/559352487 shutdown_connections 2024-03-23T09:45:10.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.035+0000 7fd688819640 1 --2- 172.21.15.47:0/559352487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd68415d0f0 0x7fd68415d550 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:10.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.035+0000 7fd688819640 1 --2- 172.21.15.47:0/559352487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd68415a860 0x7fd684161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:10.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.035+0000 7fd688819640 1 -- 172.21.15.47:0/559352487 >> 172.21.15.47:0/559352487 conn(0x7fd6840659d0 msgr2=0x7fd684065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:10.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.035+0000 7fd688819640 1 -- 172.21.15.47:0/559352487 shutdown_connections 2024-03-23T09:45:10.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.035+0000 7fd688819640 1 -- 172.21.15.47:0/559352487 wait complete. 2024-03-23T09:45:10.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.036+0000 7fd688819640 1 Processor -- start 2024-03-23T09:45:10.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.036+0000 7fd688819640 1 -- start start 2024-03-23T09:45:10.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd688819640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6840cf190 0x7fd6840cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:10.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd688819640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd68415a860 0x7fd6840c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:10.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd688819640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd68415d0f0 0x7fd68414f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:10.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd688819640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840cb2f0 con 0x7fd68415a860 2024-03-23T09:45:10.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd688819640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840cb5d0 con 0x7fd6840cf190 2024-03-23T09:45:10.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd688819640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840cb8b0 con 0x7fd68415d0f0 2024-03-23T09:45:10.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd681d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd68415a860 0x7fd6840c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:10.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd681d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd68415a860 0x7fd6840c8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45468/0 (socket says 172.21.15.47:45468) 2024-03-23T09:45:10.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd681d9b640 1 -- 172.21.15.47:0/3493088746 learned_addr learned my addr 172.21.15.47:0/3493088746 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:10.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd68259c640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6840cf190 0x7fd6840cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:10.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd682d9d640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd68415d0f0 0x7fd68414f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:10.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd681d9b640 1 -- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd68415d0f0 msgr2=0x7fd68414f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:10.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd681d9b640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd68415d0f0 0x7fd68414f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:10.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd681d9b640 1 -- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6840cf190 msgr2=0x7fd6840cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:10.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd681d9b640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6840cf190 0x7fd6840cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:10.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd681d9b640 1 -- 172.21.15.47:0/3493088746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6780094d0 con 0x7fd68415a860 2024-03-23T09:45:10.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd681d9b640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd68415a860 0x7fd6840c8d70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd678000c00 tx=0x7fd6780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:10.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd672ffd640 1 -- 172.21.15.47:0/3493088746 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67803ac60 con 0x7fd68415a860 2024-03-23T09:45:10.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd672ffd640 1 -- 172.21.15.47:0/3493088746 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67803ade0 con 0x7fd68415a860 2024-03-23T09:45:10.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd672ffd640 1 -- 172.21.15.47:0/3493088746 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678044630 con 0x7fd68415a860 2024-03-23T09:45:10.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.037+0000 7fd688819640 1 -- 172.21.15.47:0/3493088746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6840cbb90 con 0x7fd68415a860 2024-03-23T09:45:10.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.038+0000 7fd688819640 1 -- 172.21.15.47:0/3493088746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6840b6be0 con 0x7fd68415a860 2024-03-23T09:45:10.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.038+0000 7fd688819640 1 -- 172.21.15.47:0/3493088746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd684072360 con 0x7fd68415a860 2024-03-23T09:45:10.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.039+0000 7fd672ffd640 1 -- 172.21.15.47:0/3493088746 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd67802f050 con 0x7fd68415a860 2024-03-23T09:45:10.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.039+0000 7fd672ffd640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd66406ac00 0x7fd66406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:10.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.039+0000 7fd672ffd640 1 -- 172.21.15.47:0/3493088746 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fd6780ab380 con 0x7fd68415a860 2024-03-23T09:45:10.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.040+0000 7fd68259c640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd66406ac00 0x7fd66406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:10.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.040+0000 7fd68259c640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd66406ac00 0x7fd66406d0c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd6740070b0 tx=0x7fd674007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:10.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.044+0000 7fd672ffd640 1 -- 172.21.15.47:0/3493088746 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd678076620 con 0x7fd68415a860 2024-03-23T09:45:10.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.223+0000 7fd688819640 1 -- 172.21.15.47:0/3493088746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd684066be0 con 0x7fd68415a860 2024-03-23T09:45:10.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.905+0000 7fd672ffd640 1 -- 172.21.15.47:0/3493088746 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd67804e020 con 0x7fd68415a860 2024-03-23T09:45:10.905 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-03-23T09:45:10.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.908+0000 7fd688819640 1 -- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd66406ac00 msgr2=0x7fd66406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:10.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.908+0000 7fd688819640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd66406ac00 0x7fd66406d0c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd6740070b0 tx=0x7fd674007040 comp rx=0 tx=0).stop 2024-03-23T09:45:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.908+0000 7fd688819640 1 -- 172.21.15.47:0/3493088746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd68415a860 msgr2=0x7fd6840c8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.908+0000 7fd688819640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd68415a860 0x7fd6840c8d70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd678000c00 tx=0x7fd6780046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.908+0000 7fd688819640 1 -- 172.21.15.47:0/3493088746 shutdown_connections 2024-03-23T09:45:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.908+0000 7fd688819640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd66406ac00 0x7fd66406d0c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.908+0000 7fd688819640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd68415d0f0 0x7fd68414f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.908+0000 7fd688819640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd68415a860 0x7fd6840c8d70 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.908+0000 7fd688819640 1 --2- 172.21.15.47:0/3493088746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6840cf190 0x7fd6840cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.908+0000 7fd688819640 1 -- 172.21.15.47:0/3493088746 >> 172.21.15.47:0/3493088746 conn(0x7fd6840659d0 msgr2=0x7fd68415e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.909+0000 7fd688819640 1 -- 172.21.15.47:0/3493088746 shutdown_connections 2024-03-23T09:45:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:10.909+0000 7fd688819640 1 -- 172.21.15.47:0/3493088746 wait complete. 2024-03-23T09:45:10.922 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-23T09:45:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.031+0000 7f3f89c10640 1 Processor -- start 2024-03-23T09:45:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.031+0000 7f3f89c10640 1 -- start start 2024-03-23T09:45:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.032+0000 7f3f89c10640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f840cec50 0x7f3f840cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.032+0000 7f3f89c10640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3f840b05f0 0x7f3f840b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.032+0000 7f3f89c10640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f840cb210 con 0x7f3f8415ac40 2024-03-23T09:45:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.032+0000 7f3f89c10640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f840cbac0 con 0x7f3f840b05f0 2024-03-23T09:45:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.032+0000 7f3f89c10640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f840cf610 con 0x7f3f840cec50 2024-03-23T09:45:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.033+0000 7f3f82ffd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f840cec50 0x7f3f840cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.033+0000 7f3f83fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3f840b05f0 0x7f3f840b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.033+0000 7f3f837fe640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f3f8415ac40 0x7f3f840cbee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:41204/0 (socket says 172.21.15.47:41204) 2024-03-23T09:45:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.033+0000 7f3f837fe640 1 -- 172.21.15.47:0/4157740037 learned_addr learned my addr 172.21.15.47:0/4157740037 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.033+0000 7f3f827fc640 1 -- 172.21.15.47:0/4157740037 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890264676 0 0) 0x7f3f840cb210 con 0x7f3f8415ac40 2024-03-23T09:45:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.033+0000 7f3f827fc640 1 -- 172.21.15.47:0/4157740037 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f6c003520 con 0x7f3f8415ac40 2024-03-23T09:45:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.033+0000 7f3f82ffd640 1 -- 172.21.15.47:0/4157740037 >> v2:172.21.15.175:3300/0 conn(0x7f3f840b05f0 msgr2=0x7f3f840b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.033+0000 7f3f82ffd640 1 --2- 172.21.15.47:0/4157740037 >> v2:172.21.15.175:3300/0 conn(0x7f3f840b05f0 0x7f3f840b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.033+0000 7f3f82ffd640 1 -- 172.21.15.47:0/4157740037 >> v1:172.21.15.175:6789/0 conn(0x7f3f8415ac40 legacy=0x7f3f840cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.033+0000 7f3f82ffd640 1 -- 172.21.15.47:0/4157740037 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f840cc5f0 con 0x7f3f840cec50 2024-03-23T09:45:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.033+0000 7f3f83fff640 1 --2- 172.21.15.47:0/4157740037 >> v2:172.21.15.175:3300/0 conn(0x7f3f840b05f0 0x7f3f840b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:45:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.034+0000 7f3f82ffd640 1 --2- 172.21.15.47:0/4157740037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f840cec50 0x7f3f840cf0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3f780097a0 tx=0x7f3f7802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffd728430dd00694 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.034+0000 7f3f827fc640 1 -- 172.21.15.47:0/4157740037 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78031830 con 0x7f3f840cec50 2024-03-23T09:45:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.034+0000 7f3f827fc640 1 -- 172.21.15.47:0/4157740037 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f78031990 con 0x7f3f840cec50 2024-03-23T09:45:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.034+0000 7f3f827fc640 1 -- 172.21.15.47:0/4157740037 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78031d00 con 0x7f3f840cec50 2024-03-23T09:45:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.034+0000 7f3f89c10640 1 -- 172.21.15.47:0/4157740037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f840cec50 msgr2=0x7f3f840cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.034+0000 7f3f89c10640 1 --2- 172.21.15.47:0/4157740037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f840cec50 0x7f3f840cf0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3f780097a0 tx=0x7f3f7802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.035+0000 7f3f89c10640 1 -- 172.21.15.47:0/4157740037 shutdown_connections 2024-03-23T09:45:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.035+0000 7f3f89c10640 1 --2- 172.21.15.47:0/4157740037 >> v2:172.21.15.175:3300/0 conn(0x7f3f840b05f0 0x7f3f840b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.035+0000 7f3f89c10640 1 --2- 172.21.15.47:0/4157740037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f840cec50 0x7f3f840cf0d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.035+0000 7f3f89c10640 1 -- 172.21.15.47:0/4157740037 >> 172.21.15.47:0/4157740037 conn(0x7f3f840d0670 msgr2=0x7f3f840d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.035+0000 7f3f89c10640 1 -- 172.21.15.47:0/4157740037 shutdown_connections 2024-03-23T09:45:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.035+0000 7f3f89c10640 1 -- 172.21.15.47:0/4157740037 wait complete. 2024-03-23T09:45:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.036+0000 7f3f89c10640 1 Processor -- start 2024-03-23T09:45:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.036+0000 7f3f89c10640 1 -- start start 2024-03-23T09:45:11.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f89c10640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840b05f0 0x7f3f840b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:11.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f89c10640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f840cec50 0x7f3f840c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:11.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f89c10640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f8415ac40 0x7f3f84155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:11.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f89c10640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f840b3840 con 0x7f3f840b05f0 2024-03-23T09:45:11.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f89c10640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f840b3b20 con 0x7f3f8415ac40 2024-03-23T09:45:11.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f89c10640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f840b3e00 con 0x7f3f840cec50 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f837fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840b05f0 0x7f3f840b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f837fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840b05f0 0x7f3f840b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45478/0 (socket says 172.21.15.47:45478) 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f837fe640 1 -- 172.21.15.47:0/463280512 learned_addr learned my addr 172.21.15.47:0/463280512 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f82ffd640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f840cec50 0x7f3f840c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f83fff640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f8415ac40 0x7f3f84155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f837fe640 1 -- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f840cec50 msgr2=0x7f3f840c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f837fe640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f840cec50 0x7f3f840c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f837fe640 1 -- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f8415ac40 msgr2=0x7f3f84155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f837fe640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f8415ac40 0x7f3f84155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f837fe640 1 -- 172.21.15.47:0/463280512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7c009a70 con 0x7f3f840b05f0 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f837fe640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840b05f0 0x7f3f840b3260 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400c390 tx=0x7f3f7400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.037+0000 7f3f6bfff640 1 -- 172.21.15.47:0/463280512 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74005ce0 con 0x7f3f840b05f0 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.038+0000 7f3f6bfff640 1 -- 172.21.15.47:0/463280512 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f74005e40 con 0x7f3f840b05f0 2024-03-23T09:45:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.038+0000 7f3f6bfff640 1 -- 172.21.15.47:0/463280512 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f740072f0 con 0x7f3f840b05f0 2024-03-23T09:45:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.038+0000 7f3f89c10640 1 -- 172.21.15.47:0/463280512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f78009400 con 0x7f3f840b05f0 2024-03-23T09:45:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.038+0000 7f3f89c10640 1 -- 172.21.15.47:0/463280512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f840b43c0 con 0x7f3f840b05f0 2024-03-23T09:45:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.039+0000 7f3f89c10640 1 -- 172.21.15.47:0/463280512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f840c7720 con 0x7f3f840b05f0 2024-03-23T09:45:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.039+0000 7f3f6bfff640 1 -- 172.21.15.47:0/463280512 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3f74007450 con 0x7f3f840b05f0 2024-03-23T09:45:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.039+0000 7f3f6bfff640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3f7006ac00 0x7f3f7006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.039+0000 7f3f6bfff640 1 -- 172.21.15.47:0/463280512 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3f740884e0 con 0x7f3f840b05f0 2024-03-23T09:45:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.040+0000 7f3f82ffd640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3f7006ac00 0x7f3f7006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.040+0000 7f3f82ffd640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3f7006ac00 0x7f3f7006d0c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3f78036000 tx=0x7f3f78008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:11.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.044+0000 7f3f6bfff640 1 -- 172.21.15.47:0/463280512 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3f74053a70 con 0x7f3f840b05f0 2024-03-23T09:45:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.221+0000 7f3f89c10640 1 -- 172.21.15.47:0/463280512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3f840672f0 con 0x7f3f840b05f0 2024-03-23T09:45:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.224+0000 7f3f6bfff640 1 -- 172.21.15.47:0/463280512 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3f74057090 con 0x7f3f840b05f0 2024-03-23T09:45:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.226+0000 7f3f89c10640 1 -- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3f7006ac00 msgr2=0x7f3f7006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.226+0000 7f3f89c10640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3f7006ac00 0x7f3f7006d0c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3f78036000 tx=0x7f3f78008030 comp rx=0 tx=0).stop 2024-03-23T09:45:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.226+0000 7f3f89c10640 1 -- 172.21.15.47:0/463280512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840b05f0 msgr2=0x7f3f840b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.226+0000 7f3f89c10640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840b05f0 0x7f3f840b3260 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400c390 tx=0x7f3f7400c790 comp rx=0 tx=0).stop 2024-03-23T09:45:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.226+0000 7f3f89c10640 1 -- 172.21.15.47:0/463280512 shutdown_connections 2024-03-23T09:45:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.226+0000 7f3f89c10640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3f7006ac00 0x7f3f7006d0c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.226+0000 7f3f89c10640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f8415ac40 0x7f3f84155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.226+0000 7f3f89c10640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f840cec50 0x7f3f840c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.226+0000 7f3f89c10640 1 --2- 172.21.15.47:0/463280512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840b05f0 0x7f3f840b3260 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.226+0000 7f3f89c10640 1 -- 172.21.15.47:0/463280512 >> 172.21.15.47:0/463280512 conn(0x7f3f840d0670 msgr2=0x7f3f840d0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.226+0000 7f3f89c10640 1 -- 172.21.15.47:0/463280512 shutdown_connections 2024-03-23T09:45:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.227+0000 7f3f89c10640 1 -- 172.21.15.47:0/463280512 wait complete. 2024-03-23T09:45:11.237 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-03-23T09:45:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.338+0000 7f8a7e377640 1 Processor -- start 2024-03-23T09:45:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.338+0000 7f8a7e377640 1 -- start start 2024-03-23T09:45:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.339+0000 7f8a7e377640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7815ca90 0x7f8a78160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.339+0000 7f8a7e377640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a780d11e0 con 0x7f8a780b8350 2024-03-23T09:45:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.339+0000 7f8a7e377640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a78160f60 con 0x7f8a780cde10 2024-03-23T09:45:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.339+0000 7f8a7e377640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a78161240 con 0x7f8a7815ca90 2024-03-23T09:45:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.339+0000 7f8a777fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8a780b8350 0x7f8a780cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38916/0 (socket says 172.21.15.47:38916) 2024-03-23T09:45:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.339+0000 7f8a777fe640 1 -- 172.21.15.47:0/3661149578 learned_addr learned my addr 172.21.15.47:0/3661149578 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.339+0000 7f8a77fff640 1 --2- 172.21.15.47:0/3661149578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7815ca90 0x7f8a78160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.339+0000 7f8a76ffd640 1 -- 172.21.15.47:0/3661149578 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312392063 0 0) 0x7f8a780d11e0 con 0x7f8a780b8350 2024-03-23T09:45:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.340+0000 7f8a76ffd640 1 -- 172.21.15.47:0/3661149578 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a60003520 con 0x7f8a780b8350 2024-03-23T09:45:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.340+0000 7f8a76ffd640 1 -- 172.21.15.47:0/3661149578 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937053203 0 0) 0x7f8a78160f60 con 0x7f8a780cde10 2024-03-23T09:45:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.340+0000 7f8a77fff640 1 -- 172.21.15.47:0/3661149578 >> v1:172.21.15.175:6789/0 conn(0x7f8a780cde10 legacy=0x7f8a780d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.340+0000 7f8a77fff640 1 -- 172.21.15.47:0/3661149578 >> v1:172.21.15.47:6789/0 conn(0x7f8a780b8350 legacy=0x7f8a780cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.340+0000 7f8a77fff640 1 -- 172.21.15.47:0/3661149578 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a78161520 con 0x7f8a7815ca90 2024-03-23T09:45:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.341+0000 7f8a77fff640 1 --2- 172.21.15.47:0/3661149578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7815ca90 0x7f8a78160710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0097d0 tx=0x7f8a6c02b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5e146439a1645e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.341+0000 7f8a76ffd640 1 -- 172.21.15.47:0/3661149578 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68001670 con 0x7f8a7815ca90 2024-03-23T09:45:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.341+0000 7f8a76ffd640 1 -- 172.21.15.47:0/3661149578 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c02e050 con 0x7f8a7815ca90 2024-03-23T09:45:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.341+0000 7f8a76ffd640 1 -- 172.21.15.47:0/3661149578 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c030710 con 0x7f8a7815ca90 2024-03-23T09:45:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.341+0000 7f8a7e377640 1 -- 172.21.15.47:0/3661149578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7815ca90 msgr2=0x7f8a78160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.341+0000 7f8a7e377640 1 --2- 172.21.15.47:0/3661149578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7815ca90 0x7f8a78160710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0097d0 tx=0x7f8a6c02b690 comp rx=0 tx=0).stop 2024-03-23T09:45:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.341+0000 7f8a7e377640 1 -- 172.21.15.47:0/3661149578 shutdown_connections 2024-03-23T09:45:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.341+0000 7f8a7e377640 1 --2- 172.21.15.47:0/3661149578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7815ca90 0x7f8a78160710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.341+0000 7f8a7e377640 1 -- 172.21.15.47:0/3661149578 >> 172.21.15.47:0/3661149578 conn(0x7f8a780659d0 msgr2=0x7f8a78065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.341+0000 7f8a7e377640 1 -- 172.21.15.47:0/3661149578 shutdown_connections 2024-03-23T09:45:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.341+0000 7f8a7e377640 1 -- 172.21.15.47:0/3661149578 wait complete. 2024-03-23T09:45:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.342+0000 7f8a7e377640 1 Processor -- start 2024-03-23T09:45:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.343+0000 7f8a7e377640 1 -- start start 2024-03-23T09:45:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.343+0000 7f8a7e377640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a780b8350 0x7f8a781f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.343+0000 7f8a7e377640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a780cde10 0x7f8a7815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.343+0000 7f8a7e377640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7815ca90 0x7f8a78153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.343+0000 7f8a7e377640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a781f6ba0 con 0x7f8a780b8350 2024-03-23T09:45:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.343+0000 7f8a7e377640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a780cae20 con 0x7f8a780cde10 2024-03-23T09:45:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.343+0000 7f8a7e377640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a780cb100 con 0x7f8a7815ca90 2024-03-23T09:45:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.343+0000 7f8a77fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a780b8350 0x7f8a781f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.343+0000 7f8a77fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a780b8350 0x7f8a781f6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45480/0 (socket says 172.21.15.47:45480) 2024-03-23T09:45:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.343+0000 7f8a77fff640 1 -- 172.21.15.47:0/2528106639 learned_addr learned my addr 172.21.15.47:0/2528106639 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a7c914640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7815ca90 0x7f8a78153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a777fe640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a780cde10 0x7f8a7815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a77fff640 1 -- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7815ca90 msgr2=0x7f8a78153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a77fff640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7815ca90 0x7f8a78153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a77fff640 1 -- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a780cde10 msgr2=0x7f8a7815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a77fff640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a780cde10 0x7f8a7815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a77fff640 1 -- 172.21.15.47:0/2528106639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c009400 con 0x7f8a780b8350 2024-03-23T09:45:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a7c914640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7815ca90 0x7f8a78153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a77fff640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a780b8350 0x7f8a781f6660 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c02b6d0 tx=0x7f8a6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a74ff9640 1 -- 172.21.15.47:0/2528106639 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c032810 con 0x7f8a780b8350 2024-03-23T09:45:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a74ff9640 1 -- 172.21.15.47:0/2528106639 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c02be90 con 0x7f8a780b8350 2024-03-23T09:45:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a7e377640 1 -- 172.21.15.47:0/2528106639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a780cb3e0 con 0x7f8a780b8350 2024-03-23T09:45:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a74ff9640 1 -- 172.21.15.47:0/2528106639 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c032b70 con 0x7f8a780b8350 2024-03-23T09:45:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.344+0000 7f8a7e377640 1 -- 172.21.15.47:0/2528106639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a780cb9a0 con 0x7f8a780b8350 2024-03-23T09:45:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.345+0000 7f8a7e377640 1 -- 172.21.15.47:0/2528106639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a78072360 con 0x7f8a780b8350 2024-03-23T09:45:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.346+0000 7f8a74ff9640 1 -- 172.21.15.47:0/2528106639 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8a6c039070 con 0x7f8a780b8350 2024-03-23T09:45:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.346+0000 7f8a74ff9640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8a6406ac00 0x7f8a6406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.346+0000 7f8a74ff9640 1 -- 172.21.15.47:0/2528106639 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8a6c0ab1b0 con 0x7f8a780b8350 2024-03-23T09:45:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.346+0000 7f8a777fe640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8a6406ac00 0x7f8a6406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.347+0000 7f8a777fe640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8a6406ac00 0x7f8a6406d0c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8a7815b630 tx=0x7f8a680018f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.351+0000 7f8a74ff9640 1 -- 172.21.15.47:0/2528106639 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8a6c076690 con 0x7f8a780b8350 2024-03-23T09:45:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.598+0000 7f8a7e377640 1 -- 172.21.15.47:0/2528106639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f8a78066be0 con 0x7f8a780b8350 2024-03-23T09:45:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.905+0000 7f8a74ff9640 1 -- 172.21.15.47:0/2528106639 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v204) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8a6c079cb0 con 0x7f8a780b8350 2024-03-23T09:45:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.907+0000 7f8a7e377640 1 -- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8a6406ac00 msgr2=0x7f8a6406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.907+0000 7f8a7e377640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8a6406ac00 0x7f8a6406d0c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8a7815b630 tx=0x7f8a680018f0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.907+0000 7f8a7e377640 1 -- 172.21.15.47:0/2528106639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a780b8350 msgr2=0x7f8a781f6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.907+0000 7f8a7e377640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a780b8350 0x7f8a781f6660 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c02b6d0 tx=0x7f8a6c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.907+0000 7f8a7e377640 1 -- 172.21.15.47:0/2528106639 shutdown_connections 2024-03-23T09:45:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.907+0000 7f8a7e377640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8a6406ac00 0x7f8a6406d0c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.907+0000 7f8a7e377640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7815ca90 0x7f8a78153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.907+0000 7f8a7e377640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a780cde10 0x7f8a7815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.907+0000 7f8a7e377640 1 --2- 172.21.15.47:0/2528106639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a780b8350 0x7f8a781f6660 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:11.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.907+0000 7f8a7e377640 1 -- 172.21.15.47:0/2528106639 >> 172.21.15.47:0/2528106639 conn(0x7f8a780659d0 msgr2=0x7f8a7815bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:11.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.907+0000 7f8a7e377640 1 -- 172.21.15.47:0/2528106639 shutdown_connections 2024-03-23T09:45:11.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:11.907+0000 7f8a7e377640 1 -- 172.21.15.47:0/2528106639 wait complete. 2024-03-23T09:45:11.918 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-23T09:45:11.919 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-23T09:45:11.919 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-23T09:45:11.919 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-03-23T09:45:12.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.020+0000 7f7bd34b6640 1 Processor -- start 2024-03-23T09:45:12.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.021+0000 7f7bd34b6640 1 -- start start 2024-03-23T09:45:12.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.021+0000 7f7bd34b6640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7bcc0cd7f0 0x7f7bcc0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:12.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.021+0000 7f7bd34b6640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc15ae50 con 0x7f7bcc0ce800 2024-03-23T09:45:12.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.021+0000 7f7bd34b6640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc0cf010 con 0x7f7bcc15aa70 2024-03-23T09:45:12.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.021+0000 7f7bd34b6640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc0cf2f0 con 0x7f7bcc0cd7f0 2024-03-23T09:45:12.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.021+0000 7f7bd1a53640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7bcc0ce800 0x7f7bcc0cbb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38928/0 (socket says 172.21.15.47:38928) 2024-03-23T09:45:12.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.021+0000 7f7bd1a53640 1 -- 172.21.15.47:0/943835631 learned_addr learned my addr 172.21.15.47:0/943835631 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:12.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991503254 0 0) 0x7f7bcc15ae50 con 0x7f7bcc0ce800 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bd0a51640 1 --2- 172.21.15.47:0/943835631 >> v2:172.21.15.175:3300/0 conn(0x7f7bcc0cd7f0 0x7f7bcc0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb4003520 con 0x7f7bcc0ce800 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2299739719 0 0) 0x7f7bb4003520 con 0x7f7bcc0ce800 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 >> v2:172.21.15.175:3300/0 conn(0x7f7bcc0cd7f0 msgr2=0x7f7bcc0ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 --2- 172.21.15.47:0/943835631 >> v2:172.21.15.175:3300/0 conn(0x7f7bcc0cd7f0 0x7f7bcc0ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 >> v1:172.21.15.175:6789/0 conn(0x7f7bcc15aa70 legacy=0x7f7bcc0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0cf5d0 con 0x7f7bcc0ce800 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f7bbc0019f0 con 0x7f7bcc0ce800 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 >> v1:172.21.15.47:6789/0 conn(0x7f7bcc0ce800 legacy=0x7f7bcc0cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 --2- 172.21.15.47:0/943835631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bb4003970 0x7f7bb4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 --2- 172.21.15.47:0/943835631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb40047c0 0x7f7bb4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 --2- 172.21.15.47:0/943835631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb4005180 0x7f7bb400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0cf2f0 con 0x7f7bb4003970 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb400c030 con 0x7f7bb40047c0 2024-03-23T09:45:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb400c1f0 con 0x7f7bb4005180 2024-03-23T09:45:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bd1252640 1 --2- 172.21.15.47:0/943835631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bb4003970 0x7f7bb4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bd0a51640 1 --2- 172.21.15.47:0/943835631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb40047c0 0x7f7bb4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bd1a53640 1 --2- 172.21.15.47:0/943835631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb4005180 0x7f7bb400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bd1252640 1 -- 172.21.15.47:0/943835631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb4005180 msgr2=0x7f7bb400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bd1252640 1 --2- 172.21.15.47:0/943835631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb4005180 0x7f7bb400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bd1252640 1 -- 172.21.15.47:0/943835631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb40047c0 msgr2=0x7f7bb4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.022+0000 7f7bd1252640 1 --2- 172.21.15.47:0/943835631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb40047c0 0x7f7bb4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.023+0000 7f7bd1252640 1 -- 172.21.15.47:0/943835631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb4003520 con 0x7f7bb4003970 2024-03-23T09:45:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.023+0000 7f7bd0a51640 1 --2- 172.21.15.47:0/943835631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb40047c0 0x7f7bb4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.023+0000 7f7bd1a53640 1 -- 172.21.15.47:0/943835631 reap_dead start 2024-03-23T09:45:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.023+0000 7f7bd1252640 1 --2- 172.21.15.47:0/943835631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bb4003970 0x7f7bb4003d90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4001750 tx=0x7f7bb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f5e47676fd06f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.023+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40052d0 con 0x7f7bb4003970 2024-03-23T09:45:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.023+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc402e050 con 0x7f7bb4003970 2024-03-23T09:45:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.023+0000 7f7bcbfff640 1 -- 172.21.15.47:0/943835631 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4003830 con 0x7f7bb4003970 2024-03-23T09:45:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.024+0000 7f7bd34b6640 1 -- 172.21.15.47:0/943835631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bb4003970 msgr2=0x7f7bb4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.024+0000 7f7bd34b6640 1 --2- 172.21.15.47:0/943835631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bb4003970 0x7f7bb4003d90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4001750 tx=0x7f7bb40057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.024+0000 7f7bd34b6640 1 -- 172.21.15.47:0/943835631 shutdown_connections 2024-03-23T09:45:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.024+0000 7f7bd34b6640 1 --2- 172.21.15.47:0/943835631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bb4003970 0x7f7bb4003d90 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.024+0000 7f7bd34b6640 1 -- 172.21.15.47:0/943835631 >> 172.21.15.47:0/943835631 conn(0x7f7bcc0d06f0 msgr2=0x7f7bcc0d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.024+0000 7f7bd34b6640 1 -- 172.21.15.47:0/943835631 shutdown_connections 2024-03-23T09:45:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.024+0000 7f7bd34b6640 1 -- 172.21.15.47:0/943835631 wait complete. 2024-03-23T09:45:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.025+0000 7f7bd34b6640 1 Processor -- start 2024-03-23T09:45:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.025+0000 7f7bd34b6640 1 -- start start 2024-03-23T09:45:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd34b6640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb4003970 0x7f7bcc150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd34b6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bcc0c6c40 0x7f7bcc0c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd34b6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc15cdd0 0x7f7bcc15d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd34b6640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc15c940 con 0x7f7bcc15cdd0 2024-03-23T09:45:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd34b6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0c70c0 con 0x7f7bcc0c6c40 2024-03-23T09:45:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd34b6640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0c5310 con 0x7f7bb4003970 2024-03-23T09:45:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd1a53640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc15cdd0 0x7f7bcc15d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd1a53640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc15cdd0 0x7f7bcc15d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45488/0 (socket says 172.21.15.47:45488) 2024-03-23T09:45:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd1a53640 1 -- 172.21.15.47:0/3279799291 learned_addr learned my addr 172.21.15.47:0/3279799291 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd0a51640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bcc0c6c40 0x7f7bcc0c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd1252640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb4003970 0x7f7bcc150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd1a53640 1 -- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb4003970 msgr2=0x7f7bcc150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd1a53640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb4003970 0x7f7bcc150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd1a53640 1 -- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bcc0c6c40 msgr2=0x7f7bcc0c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd1a53640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bcc0c6c40 0x7f7bcc0c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd1a53640 1 -- 172.21.15.47:0/3279799291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0c42c0 con 0x7f7bcc15cdd0 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.026+0000 7f7bd0a51640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bcc0c6c40 0x7f7bcc0c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.027+0000 7f7bd1a53640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc15cdd0 0x7f7bcc15d250 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0d1570 tx=0x7f7bbc007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.027+0000 7f7bc9ffb640 1 -- 172.21.15.47:0/3279799291 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc017070 con 0x7f7bcc15cdd0 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.027+0000 7f7bd34b6640 1 -- 172.21.15.47:0/3279799291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc0c45a0 con 0x7f7bcc15cdd0 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.027+0000 7f7bc9ffb640 1 -- 172.21.15.47:0/3279799291 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc00eaa0 con 0x7f7bcc15cdd0 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.027+0000 7f7bc9ffb640 1 -- 172.21.15.47:0/3279799291 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc00ec30 con 0x7f7bcc15cdd0 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.027+0000 7f7bd34b6640 1 -- 172.21.15.47:0/3279799291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc0c8810 con 0x7f7bcc15cdd0 2024-03-23T09:45:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.028+0000 7f7bd34b6640 1 -- 172.21.15.47:0/3279799291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc15caf0 con 0x7f7bcc15cdd0 2024-03-23T09:45:12.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.028+0000 7f7bc9ffb640 1 -- 172.21.15.47:0/3279799291 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7bbc00ed90 con 0x7f7bcc15cdd0 2024-03-23T09:45:12.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.028+0000 7f7bc9ffb640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7bb806ac00 0x7f7bb806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:12.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.029+0000 7f7bc9ffb640 1 -- 172.21.15.47:0/3279799291 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7bbc088c00 con 0x7f7bcc15cdd0 2024-03-23T09:45:12.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.029+0000 7f7bd1252640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7bb806ac00 0x7f7bb806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:12.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.029+0000 7f7bd1252640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7bb806ac00 0x7f7bb806d0c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4033000 tx=0x7f7bc4009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:12.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.033+0000 7f7bc9ffb640 1 -- 172.21.15.47:0/3279799291 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7bcc15caf0 con 0x7f7bcc15cdd0 2024-03-23T09:45:12.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.280+0000 7f7bd34b6640 1 -- 172.21.15.47:0/3279799291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7f7bcc15caf0 con 0x7f7bcc15cdd0 2024-03-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.910+0000 7f7bc9ffb640 1 -- 172.21.15.47:0/3279799291 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v205) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7bcc15caf0 con 0x7f7bcc15cdd0 2024-03-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.912+0000 7f7bd34b6640 1 -- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7bb806ac00 msgr2=0x7f7bb806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.912+0000 7f7bd34b6640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7bb806ac00 0x7f7bb806d0c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4033000 tx=0x7f7bc4009030 comp rx=0 tx=0).stop 2024-03-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.912+0000 7f7bd34b6640 1 -- 172.21.15.47:0/3279799291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc15cdd0 msgr2=0x7f7bcc15d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.912+0000 7f7bd34b6640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc15cdd0 0x7f7bcc15d250 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0d1570 tx=0x7f7bbc007bc0 comp rx=0 tx=0).stop 2024-03-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.913+0000 7f7bd34b6640 1 -- 172.21.15.47:0/3279799291 shutdown_connections 2024-03-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.913+0000 7f7bd34b6640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7bb806ac00 0x7f7bb806d0c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.913+0000 7f7bd34b6640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc15cdd0 0x7f7bcc15d250 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.913+0000 7f7bd34b6640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bcc0c6c40 0x7f7bcc0c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.913+0000 7f7bd34b6640 1 --2- 172.21.15.47:0/3279799291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb4003970 0x7f7bcc150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.913+0000 7f7bd34b6640 1 -- 172.21.15.47:0/3279799291 >> 172.21.15.47:0/3279799291 conn(0x7f7bcc0d06f0 msgr2=0x7f7bcc157720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.913+0000 7f7bd34b6640 1 -- 172.21.15.47:0/3279799291 shutdown_connections 2024-03-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:12.913+0000 7f7bd34b6640 1 -- 172.21.15.47:0/3279799291 wait complete. 2024-03-23T09:45:12.926 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.036+0000 7f081c612640 1 Processor -- start 2024-03-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.036+0000 7f081c612640 1 -- start start 2024-03-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f081c612640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f08140b8350 0x7f0814160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f081c612640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0814161630 con 0x7f08140cde10 2024-03-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f081c612640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08140d1030 con 0x7f081415c990 2024-03-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f081c612640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08140cfe10 con 0x7f08140b8350 2024-03-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f0819bad640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f08140cde10 0x7f08140d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38930/0 (socket says 172.21.15.47:38930) 2024-03-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f0819bad640 1 -- 172.21.15.47:0/3998309611 learned_addr learned my addr 172.21.15.47:0/3998309611 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1546200911 0 0) 0x7f0814161630 con 0x7f08140cde10 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f081abaf640 1 --2- 172.21.15.47:0/3998309611 >> v2:172.21.15.175:3300/0 conn(0x7f08140b8350 0x7f0814160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0800003520 con 0x7f08140cde10 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2642531761 0 0) 0x7f0800003520 con 0x7f08140cde10 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 >> v2:172.21.15.175:3300/0 conn(0x7f08140b8350 msgr2=0x7f0814160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 --2- 172.21.15.47:0/3998309611 >> v2:172.21.15.175:3300/0 conn(0x7f08140b8350 0x7f0814160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 >> v1:172.21.15.175:6789/0 conn(0x7f081415c990 legacy=0x7f08140cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08140b5b60 con 0x7f08140cde10 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0808001670 con 0x7f08140cde10 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 >> v1:172.21.15.47:6789/0 conn(0x7f08140cde10 legacy=0x7f08140d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 --2- 172.21.15.47:0/3998309611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800003970 0x7f0800003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 --2- 172.21.15.47:0/3998309611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08000047c0 0x7f0800004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 --2- 172.21.15.47:0/3998309611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0800005180 0x7f080000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08140cfe10 con 0x7f0800005180 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080000c120 con 0x7f0800003970 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080000c2e0 con 0x7f08000047c0 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.037+0000 7f081abaf640 1 --2- 172.21.15.47:0/3998309611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0800005180 0x7f080000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.038+0000 7f081a3ae640 1 --2- 172.21.15.47:0/3998309611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800003970 0x7f0800003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.038+0000 7f0819bad640 1 --2- 172.21.15.47:0/3998309611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08000047c0 0x7f0800004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.038+0000 7f081abaf640 1 -- 172.21.15.47:0/3998309611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08000047c0 msgr2=0x7f0800004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.038+0000 7f081abaf640 1 --2- 172.21.15.47:0/3998309611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08000047c0 0x7f0800004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.038+0000 7f081abaf640 1 -- 172.21.15.47:0/3998309611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800003970 msgr2=0x7f0800003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.038+0000 7f081abaf640 1 --2- 172.21.15.47:0/3998309611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800003970 0x7f0800003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.038+0000 7f081abaf640 1 -- 172.21.15.47:0/3998309611 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800003520 con 0x7f0800005180 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.038+0000 7f0819bad640 1 --2- 172.21.15.47:0/3998309611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08000047c0 0x7f0800004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.038+0000 7f081abaf640 1 -- 172.21.15.47:0/3998309611 reap_dead start 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.038+0000 7f081abaf640 1 --2- 172.21.15.47:0/3998309611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0800005180 0x7f080000b980 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f080c009fc0 tx=0x7f08000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2600b051566d14b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.039+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c02cc60 con 0x7f0800005180 2024-03-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.039+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c00f040 con 0x7f0800005180 2024-03-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.039+0000 7f08193ac640 1 -- 172.21.15.47:0/3998309611 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c003990 con 0x7f0800005180 2024-03-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.039+0000 7f081c612640 1 -- 172.21.15.47:0/3998309611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0800005180 msgr2=0x7f080000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.039+0000 7f081c612640 1 --2- 172.21.15.47:0/3998309611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0800005180 0x7f080000b980 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f080c009fc0 tx=0x7f08000057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.039+0000 7f081c612640 1 -- 172.21.15.47:0/3998309611 shutdown_connections 2024-03-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.039+0000 7f081c612640 1 --2- 172.21.15.47:0/3998309611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0800005180 0x7f080000b980 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.039+0000 7f081c612640 1 -- 172.21.15.47:0/3998309611 >> 172.21.15.47:0/3998309611 conn(0x7f08140659d0 msgr2=0x7f0814065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.039+0000 7f081c612640 1 -- 172.21.15.47:0/3998309611 shutdown_connections 2024-03-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.039+0000 7f081c612640 1 -- 172.21.15.47:0/3998309611 wait complete. 2024-03-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.040+0000 7f081c612640 1 Processor -- start 2024-03-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.040+0000 7f081c612640 1 -- start start 2024-03-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.041+0000 7f081c612640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800005180 0x7f08140ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.041+0000 7f081c612640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0814150550 0x7f081415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.041+0000 7f081c612640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081415afe0 0x7f081415d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.041+0000 7f081c612640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814158dc0 con 0x7f0814150550 2024-03-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.041+0000 7f081c612640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081415b490 con 0x7f0800005180 2024-03-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.041+0000 7f081c612640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08140ca920 con 0x7f081415afe0 2024-03-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.041+0000 7f0819bad640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0814150550 0x7f081415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.041+0000 7f0819bad640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0814150550 0x7f081415aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45516/0 (socket says 172.21.15.47:45516) 2024-03-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.041+0000 7f0819bad640 1 -- 172.21.15.47:0/3266479699 learned_addr learned my addr 172.21.15.47:0/3266479699 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.041+0000 7f081abaf640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081415afe0 0x7f081415d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.041+0000 7f081a3ae640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800005180 0x7f08140ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.042+0000 7f0819bad640 1 -- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081415afe0 msgr2=0x7f081415d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.042+0000 7f0819bad640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081415afe0 0x7f081415d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.042+0000 7f0819bad640 1 -- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800005180 msgr2=0x7f08140ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.042+0000 7f0819bad640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800005180 0x7f08140ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.042+0000 7f0819bad640 1 -- 172.21.15.47:0/3266479699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081415e100 con 0x7f0814150550 2024-03-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.042+0000 7f0819bad640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0814150550 0x7f081415aaa0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f080800e790 tx=0x7f080800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.042+0000 7f0806ffd640 1 -- 172.21.15.47:0/3266479699 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080800ee40 con 0x7f0814150550 2024-03-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.042+0000 7f0806ffd640 1 -- 172.21.15.47:0/3266479699 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080800c870 con 0x7f0814150550 2024-03-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.042+0000 7f081c612640 1 -- 172.21.15.47:0/3266479699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0814158420 con 0x7f0814150550 2024-03-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.042+0000 7f0806ffd640 1 -- 172.21.15.47:0/3266479699 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080800cc20 con 0x7f0814150550 2024-03-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.042+0000 7f081c612640 1 -- 172.21.15.47:0/3266479699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08141589e0 con 0x7f0814150550 2024-03-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.043+0000 7f081c612640 1 -- 172.21.15.47:0/3266479699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0814072360 con 0x7f0814150550 2024-03-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.043+0000 7f0806ffd640 1 -- 172.21.15.47:0/3266479699 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f080801d460 con 0x7f0814150550 2024-03-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.044+0000 7f0806ffd640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f07f806ac30 0x7f07f806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.044+0000 7f0806ffd640 1 -- 172.21.15.47:0/3266479699 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0808088f40 con 0x7f0814150550 2024-03-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.044+0000 7f081a3ae640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f07f806ac30 0x7f07f806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.044+0000 7f081a3ae640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f07f806ac30 0x7f07f806d0f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0810000c00 tx=0x7f0810009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.048+0000 7f0806ffd640 1 -- 172.21.15.47:0/3266479699 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0808054420 con 0x7f0814150550 2024-03-23T09:45:13.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.226+0000 7f081c612640 1 -- 172.21.15.47:0/3266479699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0814158f00 con 0x7f0814150550 2024-03-23T09:45:13.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.230+0000 7f0806ffd640 1 -- 172.21.15.47:0/3266479699 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0814158f00 con 0x7f0814150550 2024-03-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.232+0000 7f081c612640 1 -- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f07f806ac30 msgr2=0x7f07f806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.232+0000 7f081c612640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f07f806ac30 0x7f07f806d0f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0810000c00 tx=0x7f0810009040 comp rx=0 tx=0).stop 2024-03-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.232+0000 7f081c612640 1 -- 172.21.15.47:0/3266479699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0814150550 msgr2=0x7f081415aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.232+0000 7f081c612640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0814150550 0x7f081415aaa0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f080800e790 tx=0x7f080800ec50 comp rx=0 tx=0).stop 2024-03-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.232+0000 7f081c612640 1 -- 172.21.15.47:0/3266479699 shutdown_connections 2024-03-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.232+0000 7f081c612640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f07f806ac30 0x7f07f806d0f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.232+0000 7f081c612640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081415afe0 0x7f081415d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.232+0000 7f081c612640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0814150550 0x7f081415aaa0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.232+0000 7f081c612640 1 --2- 172.21.15.47:0/3266479699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800005180 0x7f08140ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.232+0000 7f081c612640 1 -- 172.21.15.47:0/3266479699 >> 172.21.15.47:0/3266479699 conn(0x7f08140659d0 msgr2=0x7f08140b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.232+0000 7f081c612640 1 -- 172.21.15.47:0/3266479699 shutdown_connections 2024-03-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.232+0000 7f081c612640 1 -- 172.21.15.47:0/3266479699 wait complete. 2024-03-23T09:45:13.243 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:45:13.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:45:13.289 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:45:13.331 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:45:13.331 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:45:13.331 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:45:13.346 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:45:13.346 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:45:13.346 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:45:13.391 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:45:13.391 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:45:13.391 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:45:13.406 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:45:13.406 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:45:13.406 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:45:13.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:45:13.449 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-23T09:45:13.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.549+0000 7fe2e4684640 1 Processor -- start 2024-03-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.549+0000 7fe2e4684640 1 -- start start 2024-03-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.550+0000 7fe2e4684640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2dc15a8d0 0x7fe2dc0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.550+0000 7fe2e4684640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2dc0cc610 con 0x7fe2dc0cb730 2024-03-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.550+0000 7fe2e4684640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2dc0b0700 con 0x7fe2dc0cf360 2024-03-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.550+0000 7fe2e4684640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2dc0c41c0 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.550+0000 7fe2e2420640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2dc15a8d0 0x7fe2dc0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.550+0000 7fe2e1c1f640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe2dc0cb730 0x7fe2dc0cec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38938/0 (socket says 172.21.15.47:38938) 2024-03-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.550+0000 7fe2e1c1f640 1 -- 172.21.15.47:0/907976425 learned_addr learned my addr 172.21.15.47:0/907976425 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.551+0000 7fe2e2420640 1 -- 172.21.15.47:0/907976425 >> v1:172.21.15.175:6789/0 conn(0x7fe2dc0cf360 legacy=0x7fe2dc0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.551+0000 7fe2e2420640 1 -- 172.21.15.47:0/907976425 >> v1:172.21.15.47:6789/0 conn(0x7fe2dc0cb730 legacy=0x7fe2dc0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.551+0000 7fe2e2420640 1 -- 172.21.15.47:0/907976425 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc0cbd70 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.551+0000 7fe2e2420640 1 --2- 172.21.15.47:0/907976425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2dc15a8d0 0x7fe2dc0cb1f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009d90 tx=0x7fe2cc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482f091aef42c372 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.551+0000 7fe2e141e640 1 -- 172.21.15.47:0/907976425 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc02cbd0 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.551+0000 7fe2e141e640 1 -- 172.21.15.47:0/907976425 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc02cd30 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.551+0000 7fe2e141e640 1 -- 172.21.15.47:0/907976425 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc02cea0 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.552+0000 7fe2e4684640 1 -- 172.21.15.47:0/907976425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2dc15a8d0 msgr2=0x7fe2dc0cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.552+0000 7fe2e4684640 1 --2- 172.21.15.47:0/907976425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2dc15a8d0 0x7fe2dc0cb1f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009d90 tx=0x7fe2cc02be20 comp rx=0 tx=0).stop 2024-03-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.552+0000 7fe2e4684640 1 -- 172.21.15.47:0/907976425 shutdown_connections 2024-03-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.552+0000 7fe2e4684640 1 --2- 172.21.15.47:0/907976425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2dc15a8d0 0x7fe2dc0cb1f0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.552+0000 7fe2e4684640 1 -- 172.21.15.47:0/907976425 >> 172.21.15.47:0/907976425 conn(0x7fe2dc0d0670 msgr2=0x7fe2dc0d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.552+0000 7fe2e4684640 1 -- 172.21.15.47:0/907976425 shutdown_connections 2024-03-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.552+0000 7fe2e4684640 1 -- 172.21.15.47:0/907976425 wait complete. 2024-03-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.553+0000 7fe2e4684640 1 Processor -- start 2024-03-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.553+0000 7fe2e4684640 1 -- start start 2024-03-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e4684640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2dc0cb730 0x7fe2dc146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e4684640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2dc0cf360 0x7fe2dc15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e4684640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2dc15a8d0 0x7fe2dc15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e4684640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc147800 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e4684640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0c9730 con 0x7fe2dc0cf360 2024-03-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e4684640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0c9a10 con 0x7fe2dc0cb730 2024-03-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e2c21640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2dc15a8d0 0x7fe2dc15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e2c21640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2dc15a8d0 0x7fe2dc15b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45532/0 (socket says 172.21.15.47:45532) 2024-03-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e2c21640 1 -- 172.21.15.47:0/767020014 learned_addr learned my addr 172.21.15.47:0/767020014 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e2420640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2dc0cb730 0x7fe2dc146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e1c1f640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2dc0cf360 0x7fe2dc15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e2c21640 1 -- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2dc0cb730 msgr2=0x7fe2dc146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e2c21640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2dc0cb730 0x7fe2dc146e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e2c21640 1 -- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2dc0cf360 msgr2=0x7fe2dc15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e2c21640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2dc0cf360 0x7fe2dc15afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e2c21640 1 -- 172.21.15.47:0/767020014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc009a70 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e2420640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2dc0cb730 0x7fe2dc146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.554+0000 7fe2e1c1f640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2dc0cf360 0x7fe2dc15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.555+0000 7fe2e2c21640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2dc15a8d0 0x7fe2dc15b4f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400c390 tx=0x7fe2d400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.555+0000 7fe2daffd640 1 -- 172.21.15.47:0/767020014 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d40035f0 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.555+0000 7fe2daffd640 1 -- 172.21.15.47:0/767020014 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d4003750 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.555+0000 7fe2e4684640 1 -- 172.21.15.47:0/767020014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2dc0c9cf0 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.555+0000 7fe2daffd640 1 -- 172.21.15.47:0/767020014 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4003b00 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.555+0000 7fe2e4684640 1 -- 172.21.15.47:0/767020014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2dc1473d0 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.556+0000 7fe2e4684640 1 -- 172.21.15.47:0/767020014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2dc1479d0 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.556+0000 7fe2daffd640 1 -- 172.21.15.47:0/767020014 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe2d4003c60 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.556+0000 7fe2daffd640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe2c806ac30 0x7fe2c806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.557+0000 7fe2daffd640 1 -- 172.21.15.47:0/767020014 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe2d4088660 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.557+0000 7fe2e2420640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe2c806ac30 0x7fe2c806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.557+0000 7fe2e2420640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe2c806ac30 0x7fe2c806d0f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc036000 tx=0x7fe2cc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.561+0000 7fe2daffd640 1 -- 172.21.15.47:0/767020014 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe2d4053b40 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.775+0000 7fe2e4684640 1 -- 172.21.15.47:0/767020014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2dc0672f0 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.775+0000 7fe2daffd640 1 -- 172.21.15.47:0/767020014 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3526 (secure 0 0 0) 0x7fe2d4017090 con 0x7fe2dc15a8d0 2024-03-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/83763458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":83763458},{"type":"v1","addr":"172.21.15.175:6836","nonce":83763458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/375513546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":375513546},{"type":"v1","addr":"172.21.15.175:6837","nonce":375513546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6159,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/3650756747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3650756747},{"type":"v1","addr":"172.21.15.47:6833","nonce":3650756747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2893805543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2893805543},{"type":"v1","addr":"172.21.15.47:6835","nonce":2893805543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[]} 2024-03-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.777+0000 7fe2e4684640 1 -- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe2c806ac30 msgr2=0x7fe2c806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.777+0000 7fe2e4684640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe2c806ac30 0x7fe2c806d0f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc036000 tx=0x7fe2cc030000 comp rx=0 tx=0).stop 2024-03-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.777+0000 7fe2e4684640 1 -- 172.21.15.47:0/767020014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2dc15a8d0 msgr2=0x7fe2dc15b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.777+0000 7fe2e4684640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2dc15a8d0 0x7fe2dc15b4f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400c390 tx=0x7fe2d400c850 comp rx=0 tx=0).stop 2024-03-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.777+0000 7fe2e4684640 1 -- 172.21.15.47:0/767020014 shutdown_connections 2024-03-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.777+0000 7fe2e4684640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe2c806ac30 0x7fe2c806d0f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.777+0000 7fe2e4684640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2dc15a8d0 0x7fe2dc15b4f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.777+0000 7fe2e4684640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2dc0cf360 0x7fe2dc15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.777+0000 7fe2e4684640 1 --2- 172.21.15.47:0/767020014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2dc0cb730 0x7fe2dc146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.777+0000 7fe2e4684640 1 -- 172.21.15.47:0/767020014 >> 172.21.15.47:0/767020014 conn(0x7fe2dc0d0670 msgr2=0x7fe2dc06fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.777+0000 7fe2e4684640 1 -- 172.21.15.47:0/767020014 shutdown_connections 2024-03-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.777+0000 7fe2e4684640 1 -- 172.21.15.47:0/767020014 wait complete. 2024-03-23T09:45:13.788 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-23T09:45:13.788 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-23T09:45:13.788 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-23T09:45:13.788 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:45:13.789 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-23T09:45:13.789 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-23T09:45:13.789 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-23T09:45:13.789 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:45:13.789 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-23T09:45:13.789 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-23T09:45:13.789 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-23T09:45:13.790 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:45:13.790 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-23T09:45:13.790 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-23T09:45:13.790 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-23T09:45:13.790 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:45:13.798 INFO:tasks.ceph.mds.a:Stopped 2024-03-23T09:45:13.798 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-23T09:45:13.799 INFO:tasks.ceph.mds.c:Stopped 2024-03-23T09:45:13.799 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-23T09:45:13.802 INFO:tasks.ceph.mds.b:Stopped 2024-03-23T09:45:13.802 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-23T09:45:13.803 INFO:tasks.ceph.mds.d:Stopped 2024-03-23T09:45:13.803 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-23T09:45:13.840 INFO:tasks.ceph.mds.a:Started 2024-03-23T09:45:13.840 DEBUG:teuthology.parallel:result is None 2024-03-23T09:45:13.841 INFO:tasks.ceph.mds.c:Started 2024-03-23T09:45:13.841 DEBUG:teuthology.parallel:result is None 2024-03-23T09:45:13.845 INFO:tasks.ceph.mds.b:Started 2024-03-23T09:45:13.845 DEBUG:teuthology.parallel:result is None 2024-03-23T09:45:13.846 INFO:tasks.ceph.mds.d:Started 2024-03-23T09:45:13.846 DEBUG:teuthology.parallel:result is None 2024-03-23T09:45:13.846 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-23T09:45:13.917 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-03-23T09:45:13.917 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:45:13.917 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:45:13.918+0000 7f839fc28640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-03-23T09:45:13.929 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-03-23T09:45:13.930 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-03-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.955+0000 7fba9fc15640 1 Processor -- start 2024-03-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.956+0000 7fba9fc15640 1 -- start start 2024-03-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.956+0000 7fba9fc15640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9815cd00 0x7fba980d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.956+0000 7fba9fc15640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba980d1020 0x7fba980cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.956+0000 7fba9fc15640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fba980cf750 0x7fba980b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.956+0000 7fba9fc15640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba980cfbe0 con 0x7fba980d1020 2024-03-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.956+0000 7fba9fc15640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9815d0e0 con 0x7fba980cf750 2024-03-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.956+0000 7fba9fc15640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba980d1540 con 0x7fba9815cd00 2024-03-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.957+0000 7fba9d1b0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba980d1020 0x7fba980cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.957+0000 7fba9d1b0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba980d1020 0x7fba980cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45574/0 (socket says 172.21.15.47:45574) 2024-03-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.957+0000 7fba9d1b0640 1 -- 172.21.15.47:0/1993887294 learned_addr learned my addr 172.21.15.47:0/1993887294 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.957+0000 7fba9d9b1640 1 --2- 172.21.15.47:0/1993887294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9815cd00 0x7fba980d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.957+0000 7fba9e1b2640 1 --2- 172.21.15.47:0/1993887294 >> v2:172.21.15.175:3300/0 conn(0x7fba980cf750 0x7fba980b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.957+0000 7fba9d1b0640 1 -- 172.21.15.47:0/1993887294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9815cd00 msgr2=0x7fba980d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.957+0000 7fba9d1b0640 1 --2- 172.21.15.47:0/1993887294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9815cd00 0x7fba980d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.957+0000 7fba9d1b0640 1 -- 172.21.15.47:0/1993887294 >> v2:172.21.15.175:3300/0 conn(0x7fba980cf750 msgr2=0x7fba980b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.957+0000 7fba9d1b0640 1 --2- 172.21.15.47:0/1993887294 >> v2:172.21.15.175:3300/0 conn(0x7fba980cf750 0x7fba980b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.957+0000 7fba9d1b0640 1 -- 172.21.15.47:0/1993887294 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba980d1820 con 0x7fba980d1020 2024-03-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.957+0000 7fba9d9b1640 1 --2- 172.21.15.47:0/1993887294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9815cd00 0x7fba980d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.958+0000 7fba9d1b0640 1 --2- 172.21.15.47:0/1993887294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba980d1020 0x7fba980cf210 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fba980b8040 tx=0x7fba8802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ee3efacb1c74fb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.959+0000 7fba9c9af640 1 -- 172.21.15.47:0/1993887294 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba880317e0 con 0x7fba980d1020 2024-03-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.959+0000 7fba9c9af640 1 -- 172.21.15.47:0/1993887294 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba88031940 con 0x7fba980d1020 2024-03-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.959+0000 7fba9c9af640 1 -- 172.21.15.47:0/1993887294 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88031cd0 con 0x7fba980d1020 2024-03-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.960+0000 7fba9fc15640 1 -- 172.21.15.47:0/1993887294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba980d1020 msgr2=0x7fba980cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.960+0000 7fba9fc15640 1 --2- 172.21.15.47:0/1993887294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba980d1020 0x7fba980cf210 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fba980b8040 tx=0x7fba8802b880 comp rx=0 tx=0).stop 2024-03-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.960+0000 7fba9fc15640 1 -- 172.21.15.47:0/1993887294 shutdown_connections 2024-03-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.960+0000 7fba9fc15640 1 --2- 172.21.15.47:0/1993887294 >> v2:172.21.15.175:3300/0 conn(0x7fba980cf750 0x7fba980b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.960+0000 7fba9fc15640 1 --2- 172.21.15.47:0/1993887294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba980d1020 0x7fba980cf210 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.960+0000 7fba9fc15640 1 --2- 172.21.15.47:0/1993887294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9815cd00 0x7fba980d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.960+0000 7fba9fc15640 1 -- 172.21.15.47:0/1993887294 >> 172.21.15.47:0/1993887294 conn(0x7fba9806b160 msgr2=0x7fba9806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.960+0000 7fba9fc15640 1 -- 172.21.15.47:0/1993887294 shutdown_connections 2024-03-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.960+0000 7fba9fc15640 1 -- 172.21.15.47:0/1993887294 wait complete. 2024-03-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.961+0000 7fba9fc15640 1 Processor -- start 2024-03-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.961+0000 7fba9fc15640 1 -- start start 2024-03-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.962+0000 7fba9fc15640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba980cf750 0x7fba98157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.962+0000 7fba9fc15640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba980d1020 0x7fba980c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.962+0000 7fba9fc15640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9815cd00 0x7fba980c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.962+0000 7fba9fc15640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba980cc7c0 con 0x7fba9815cd00 2024-03-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.962+0000 7fba9fc15640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba980cbc40 con 0x7fba980cf750 2024-03-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.962+0000 7fba9fc15640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba980cbf20 con 0x7fba980d1020 2024-03-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.962+0000 7fba9e1b2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9815cd00 0x7fba980c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.962+0000 7fba9e1b2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9815cd00 0x7fba980c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45582/0 (socket says 172.21.15.47:45582) 2024-03-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.962+0000 7fba9e1b2640 1 -- 172.21.15.47:0/1486579028 learned_addr learned my addr 172.21.15.47:0/1486579028 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.962+0000 7fba9d9b1640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba980cf750 0x7fba98157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.962+0000 7fba9d1b0640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba980d1020 0x7fba980c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.962+0000 7fba9e1b2640 1 -- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba980d1020 msgr2=0x7fba980c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.962+0000 7fba9e1b2640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba980d1020 0x7fba980c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.963+0000 7fba9e1b2640 1 -- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba980cf750 msgr2=0x7fba98157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.963+0000 7fba9e1b2640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba980cf750 0x7fba98157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.963+0000 7fba9e1b2640 1 -- 172.21.15.47:0/1486579028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba880094d0 con 0x7fba9815cd00 2024-03-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.963+0000 7fba9d9b1640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba980cf750 0x7fba98157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.963+0000 7fba9e1b2640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9815cd00 0x7fba980c7460 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fba90007e60 tx=0x7fba9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.963+0000 7fba8e7fc640 1 -- 172.21.15.47:0/1486579028 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90014620 con 0x7fba9815cd00 2024-03-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.963+0000 7fba8e7fc640 1 -- 172.21.15.47:0/1486579028 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9000ee90 con 0x7fba9815cd00 2024-03-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.963+0000 7fba8e7fc640 1 -- 172.21.15.47:0/1486579028 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba900149e0 con 0x7fba9815cd00 2024-03-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.963+0000 7fba9fc15640 1 -- 172.21.15.47:0/1486579028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba980cc200 con 0x7fba9815cd00 2024-03-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.963+0000 7fba9fc15640 1 -- 172.21.15.47:0/1486579028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba980cb930 con 0x7fba9815cd00 2024-03-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.964+0000 7fba8e7fc640 1 -- 172.21.15.47:0/1486579028 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fba90014b40 con 0x7fba9815cd00 2024-03-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.965+0000 7fba9fc15640 1 -- 172.21.15.47:0/1486579028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba98072360 con 0x7fba9815cd00 2024-03-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.965+0000 7fba8e7fc640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba7c06ac00 0x7fba7c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.965+0000 7fba8e7fc640 1 -- 172.21.15.47:0/1486579028 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fba900898f0 con 0x7fba9815cd00 2024-03-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.965+0000 7fba9d9b1640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba7c06ac00 0x7fba7c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.965+0000 7fba9d9b1640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba7c06ac00 0x7fba7c06d0c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fba980b8400 tx=0x7fba94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:13.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:13.969+0000 7fba8e7fc640 1 -- 172.21.15.47:0/1486579028 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fba90054de0 con 0x7fba9815cd00 2024-03-23T09:45:14.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-03-23T09:45:14.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-03-23T09:45:14.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-03-23T09:45:14.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.163+0000 7fba9fc15640 1 -- 172.21.15.47:0/1486579028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fba98066be0 con 0x7fba9815cd00 2024-03-23T09:45:14.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.971+0000 7fba8e7fc640 1 -- 172.21.15.47:0/1486579028 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v66) v1 ==== 97+0+0 (secure 0 0 0) 0x7fba90058400 con 0x7fba9815cd00 2024-03-23T09:45:14.972 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blocklist entries 2024-03-23T09:45:14.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.974+0000 7fba9fc15640 1 -- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba7c06ac00 msgr2=0x7fba7c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:14.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.974+0000 7fba9fc15640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba7c06ac00 0x7fba7c06d0c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fba980b8400 tx=0x7fba94008040 comp rx=0 tx=0).stop 2024-03-23T09:45:14.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.974+0000 7fba9fc15640 1 -- 172.21.15.47:0/1486579028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9815cd00 msgr2=0x7fba980c7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:14.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.974+0000 7fba9fc15640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9815cd00 0x7fba980c7460 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fba90007e60 tx=0x7fba9000e4b0 comp rx=0 tx=0).stop 2024-03-23T09:45:14.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.974+0000 7fba9fc15640 1 -- 172.21.15.47:0/1486579028 shutdown_connections 2024-03-23T09:45:14.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.974+0000 7fba9fc15640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba7c06ac00 0x7fba7c06d0c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:14.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.974+0000 7fba9fc15640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9815cd00 0x7fba980c7460 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:14.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.974+0000 7fba9fc15640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba980d1020 0x7fba980c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:14.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.974+0000 7fba9fc15640 1 --2- 172.21.15.47:0/1486579028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba980cf750 0x7fba98157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:14.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.974+0000 7fba9fc15640 1 -- 172.21.15.47:0/1486579028 >> 172.21.15.47:0/1486579028 conn(0x7fba9806b160 msgr2=0x7fba9815d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:14.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.974+0000 7fba9fc15640 1 -- 172.21.15.47:0/1486579028 shutdown_connections 2024-03-23T09:45:14.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:14.974+0000 7fba9fc15640 1 -- 172.21.15.47:0/1486579028 wait complete. 2024-03-23T09:45:14.987 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-23T09:45:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.095+0000 7f956f657640 1 Processor -- start 2024-03-23T09:45:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.095+0000 7f956f657640 1 -- start start 2024-03-23T09:45:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.096+0000 7f956f657640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815cd00 0x7f95680d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.096+0000 7f956f657640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95680cf920 0x7f95680b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.096+0000 7f956f657640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f956815d0e0 con 0x7f95680d1020 2024-03-23T09:45:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.096+0000 7f956f657640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95680d1540 con 0x7f95680cf920 2024-03-23T09:45:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.096+0000 7f956f657640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95680d1820 con 0x7f956815cd00 2024-03-23T09:45:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.096+0000 7f956dbf4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95680cf920 0x7f95680b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.096+0000 7f956cbf2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f95680d1020 0x7f95680cf210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47522/0 (socket says 172.21.15.47:47522) 2024-03-23T09:45:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.096+0000 7f956cbf2640 1 -- 172.21.15.47:0/3461981734 learned_addr learned my addr 172.21.15.47:0/3461981734 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.096+0000 7f956d3f3640 1 --2- 172.21.15.47:0/3461981734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815cd00 0x7f95680d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.096+0000 7f956dbf4640 1 -- 172.21.15.47:0/3461981734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815cd00 msgr2=0x7f95680d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.096+0000 7f956dbf4640 1 --2- 172.21.15.47:0/3461981734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815cd00 0x7f95680d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.096+0000 7f956dbf4640 1 -- 172.21.15.47:0/3461981734 >> v1:172.21.15.47:6789/0 conn(0x7f95680d1020 legacy=0x7f95680cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.096+0000 7f956dbf4640 1 -- 172.21.15.47:0/3461981734 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95680d1b00 con 0x7f95680cf920 2024-03-23T09:45:15.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.097+0000 7f956dbf4640 1 --2- 172.21.15.47:0/3461981734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95680cf920 0x7f95680b5a90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9558009d90 tx=0x7f955802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2a616f921d9088 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:15.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.097+0000 7f9567fff640 1 -- 172.21.15.47:0/3461981734 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955802cbc0 con 0x7f95680cf920 2024-03-23T09:45:15.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.097+0000 7f9567fff640 1 -- 172.21.15.47:0/3461981734 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955802cd20 con 0x7f95680cf920 2024-03-23T09:45:15.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.097+0000 7f9567fff640 1 -- 172.21.15.47:0/3461981734 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9558034660 con 0x7f95680cf920 2024-03-23T09:45:15.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.098+0000 7f956f657640 1 -- 172.21.15.47:0/3461981734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95680cf920 msgr2=0x7f95680b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.098+0000 7f956f657640 1 --2- 172.21.15.47:0/3461981734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95680cf920 0x7f95680b5a90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9558009d90 tx=0x7f955802be10 comp rx=0 tx=0).stop 2024-03-23T09:45:15.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.098+0000 7f956f657640 1 -- 172.21.15.47:0/3461981734 shutdown_connections 2024-03-23T09:45:15.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.098+0000 7f956f657640 1 --2- 172.21.15.47:0/3461981734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95680cf920 0x7f95680b5a90 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.098+0000 7f956f657640 1 --2- 172.21.15.47:0/3461981734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815cd00 0x7f95680d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.098+0000 7f956f657640 1 -- 172.21.15.47:0/3461981734 >> 172.21.15.47:0/3461981734 conn(0x7f956806b160 msgr2=0x7f956806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:15.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.098+0000 7f956f657640 1 -- 172.21.15.47:0/3461981734 shutdown_connections 2024-03-23T09:45:15.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.098+0000 7f956f657640 1 -- 172.21.15.47:0/3461981734 wait complete. 2024-03-23T09:45:15.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.099+0000 7f956f657640 1 Processor -- start 2024-03-23T09:45:15.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.099+0000 7f956f657640 1 -- start start 2024-03-23T09:45:15.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956f657640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95680cf920 0x7f95680b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956f657640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95680d1020 0x7f9568159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956f657640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815cd00 0x7f95680cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956f657640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95680b3350 con 0x7f95680d1020 2024-03-23T09:45:15.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956f657640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95680b3cb0 con 0x7f95680cf920 2024-03-23T09:45:15.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956f657640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95680b26d0 con 0x7f956815cd00 2024-03-23T09:45:15.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956cbf2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95680d1020 0x7f9568159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956cbf2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95680d1020 0x7f9568159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47756/0 (socket says 172.21.15.47:47756) 2024-03-23T09:45:15.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956cbf2640 1 -- 172.21.15.47:0/2712599484 learned_addr learned my addr 172.21.15.47:0/2712599484 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:15.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956cbf2640 1 -- 172.21.15.47:0/2712599484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815cd00 msgr2=0x7f95680cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956cbf2640 1 --2- 172.21.15.47:0/2712599484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815cd00 0x7f95680cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956cbf2640 1 -- 172.21.15.47:0/2712599484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95680cf920 msgr2=0x7f95680b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956cbf2640 1 --2- 172.21.15.47:0/2712599484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95680cf920 0x7f95680b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.100+0000 7f956cbf2640 1 -- 172.21.15.47:0/2712599484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9558009a70 con 0x7f95680d1020 2024-03-23T09:45:15.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.101+0000 7f956cbf2640 1 --2- 172.21.15.47:0/2712599484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95680d1020 0x7f9568159a20 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f955c00c4a0 tx=0x7f955c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:15.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.101+0000 7f9565ffb640 1 -- 172.21.15.47:0/2712599484 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c005ce0 con 0x7f95680d1020 2024-03-23T09:45:15.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.101+0000 7f9565ffb640 1 -- 172.21.15.47:0/2712599484 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c005e40 con 0x7f95680d1020 2024-03-23T09:45:15.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.101+0000 7f9565ffb640 1 -- 172.21.15.47:0/2712599484 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c007270 con 0x7f95680d1020 2024-03-23T09:45:15.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.101+0000 7f956f657640 1 -- 172.21.15.47:0/2712599484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95680b29b0 con 0x7f95680d1020 2024-03-23T09:45:15.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.101+0000 7f956f657640 1 -- 172.21.15.47:0/2712599484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95680b2f70 con 0x7f95680d1020 2024-03-23T09:45:15.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.102+0000 7f956f657640 1 -- 172.21.15.47:0/2712599484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9568072360 con 0x7f95680d1020 2024-03-23T09:45:15.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.102+0000 7f9565ffb640 1 -- 172.21.15.47:0/2712599484 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f955c007490 con 0x7f95680d1020 2024-03-23T09:45:15.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.103+0000 7f9565ffb640 1 --2- 172.21.15.47:0/2712599484 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f955406ac00 0x7f955406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.103+0000 7f9565ffb640 1 -- 172.21.15.47:0/2712599484 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f955c088420 con 0x7f95680d1020 2024-03-23T09:45:15.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.103+0000 7f956d3f3640 1 --2- 172.21.15.47:0/2712599484 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f955406ac00 0x7f955406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.104+0000 7f956d3f3640 1 --2- 172.21.15.47:0/2712599484 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f955406ac00 0x7f955406d0c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9560006fd0 tx=0x7f9560008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:15.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.107+0000 7f9565ffb640 1 -- 172.21.15.47:0/2712599484 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f955c053a90 con 0x7f95680d1020 2024-03-23T09:45:15.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.352+0000 7f956f657640 1 -- 172.21.15.47:0/2712599484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f95680b3490 con 0x7f95680d1020 2024-03-23T09:45:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.353+0000 7f9565ffb640 1 -- 172.21.15.47:0/2712599484 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f95680b3490 con 0x7f95680d1020 2024-03-23T09:45:15.353 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:15.353 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:45:15.353 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2024-03-23T09:45:15.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.353 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2024-03-23T09:45:15.353 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lo+xoDhAAyxqr1H6eiINwt2X7LUlrSg==", 2024-03-23T09:45:15.353 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:45:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:45:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:45:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2024-03-23T09:45:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZ5OlGhAAuUElzRghNRFGKREdCpxKvA==", 2024-03-23T09:45:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:45:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:45:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lFwyzFRAAKldMOA7oSblA8YMrvntY8w==", 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.d", 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lpiD3IRAApXAdUQ4AkxWHzsFV0qcBng==", 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB7o/5lLvoADBAAgdixkm+jHwQilYw0IAn6EA==", 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:45:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB+o/5lOBfGCRAApglzalO1ztnJzOKg/UAyWA==", 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCBo/5lRBjlBxAAd45w8/kTMPqxmNyfCS3XYA==", 2024-03-23T09:45:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCEo/5l+JxrBhAAbR/XqDyPDnN8eLpmvBZ7Mg==", 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCKo/5ltsqwDRAAkqcZ/sEf2oozWFN4AXg/+w==", 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCNo/5lpLHiCxAAB2axUv9gjcYVlij9f+AEHg==", 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:45:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCQo/5l2io3ChAAu+lfYcWWBQf43aYzWjCBNg==", 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCTo/5lA7jjCBAAjJgBkp89H8lx6fidl/Z7IA==", 2024-03-23T09:45:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZFc3JhAAZWmDZOnkW9eo9hqEDYynQw==", 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lIrJSLxAAPJer3ZP3h8VkImkiJmnKVw==", 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l1+DyKxAAZLKZNbUpUJ0fWPIYNvWK+g==", 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lWqkiNRAAL1q0lEcM/NKW3Pf9LcP21g==", 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB4o/5lyeuKLRAARrfTpPc5BTVYZe/LUREDkg==", 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l/0EOCxAAIrgIXRiFSQ1G7Jat5hNm4A==", 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:45:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2024-03-23T09:45:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lywMFBxAAIVwjCe6B9EtBMAG2n7cZJw==", 2024-03-23T09:45:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:45:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:45:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:45:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:45:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-23T09:45:15.365 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:45:15.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.355+0000 7f956f657640 1 -- 172.21.15.47:0/2712599484 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f955406ac00 msgr2=0x7f955406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.355+0000 7f956f657640 1 --2- 172.21.15.47:0/2712599484 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f955406ac00 0x7f955406d0c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9560006fd0 tx=0x7f9560008040 comp rx=0 tx=0).stop 2024-03-23T09:45:15.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.355+0000 7f956f657640 1 -- 172.21.15.47:0/2712599484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95680d1020 msgr2=0x7f9568159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.355+0000 7f956f657640 1 --2- 172.21.15.47:0/2712599484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95680d1020 0x7f9568159a20 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f955c00c4a0 tx=0x7f955c00c960 comp rx=0 tx=0).stop 2024-03-23T09:45:15.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.355+0000 7f956f657640 1 -- 172.21.15.47:0/2712599484 shutdown_connections 2024-03-23T09:45:15.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.355+0000 7f956f657640 1 --2- 172.21.15.47:0/2712599484 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f955406ac00 0x7f955406d0c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.355+0000 7f956f657640 1 --2- 172.21.15.47:0/2712599484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815cd00 0x7f95680cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.355+0000 7f956f657640 1 --2- 172.21.15.47:0/2712599484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95680d1020 0x7f9568159a20 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.355+0000 7f956f657640 1 --2- 172.21.15.47:0/2712599484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95680cf920 0x7f95680b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.355+0000 7f956f657640 1 -- 172.21.15.47:0/2712599484 >> 172.21.15.47:0/2712599484 conn(0x7f956806b160 msgr2=0x7f956815bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:15.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.355+0000 7f956f657640 1 -- 172.21.15.47:0/2712599484 shutdown_connections 2024-03-23T09:45:15.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.355+0000 7f956f657640 1 -- 172.21.15.47:0/2712599484 wait complete. 2024-03-23T09:45:15.368 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:45:15.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.469+0000 7f2dd95c9640 1 Processor -- start 2024-03-23T09:45:15.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.469+0000 7f2dd95c9640 1 -- start start 2024-03-23T09:45:15.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.470+0000 7f2dd95c9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2dd40ced00 0x7f2dd415a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.470+0000 7f2dd95c9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd40c58a0 0x7f2dd40c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.470+0000 7f2dd95c9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd40cf1b0 con 0x7f2dd4159730 2024-03-23T09:45:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.470+0000 7f2dd95c9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd415aca0 con 0x7f2dd40ced00 2024-03-23T09:45:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.470+0000 7f2dd95c9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40c6250 con 0x7f2dd40c58a0 2024-03-23T09:45:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.470+0000 7f2dd27fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2dd40ced00 0x7f2dd415a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.470+0000 7f2dd27fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2dd40ced00 0x7f2dd415a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47760/0 (socket says 172.21.15.47:47760) 2024-03-23T09:45:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.470+0000 7f2dd27fc640 1 -- 172.21.15.47:0/1216816747 learned_addr learned my addr 172.21.15.47:0/1216816747 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.470+0000 7f2dd37fe640 1 --2- 172.21.15.47:0/1216816747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd40c58a0 0x7f2dd40c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.470+0000 7f2dd27fc640 1 -- 172.21.15.47:0/1216816747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd40c58a0 msgr2=0x7f2dd40c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.470+0000 7f2dd27fc640 1 --2- 172.21.15.47:0/1216816747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd40c58a0 0x7f2dd40c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.471+0000 7f2dd27fc640 1 -- 172.21.15.47:0/1216816747 >> v1:172.21.15.175:6789/0 conn(0x7f2dd4159730 legacy=0x7f2dd40ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.471+0000 7f2dd27fc640 1 -- 172.21.15.47:0/1216816747 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd40cf490 con 0x7f2dd40ced00 2024-03-23T09:45:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.471+0000 7f2dd37fe640 1 --2- 172.21.15.47:0/1216816747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd40c58a0 0x7f2dd40c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:15.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.471+0000 7f2dd27fc640 1 --2- 172.21.15.47:0/1216816747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd40ced00 0x7f2dd415a760 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0098a0 tx=0x7f2dbc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf39f7c290a45d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:15.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.471+0000 7f2dd1ffb640 1 -- 172.21.15.47:0/1216816747 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc030840 con 0x7f2dd40ced00 2024-03-23T09:45:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.471+0000 7f2dd1ffb640 1 -- 172.21.15.47:0/1216816747 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc0309a0 con 0x7f2dd40ced00 2024-03-23T09:45:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.471+0000 7f2dd1ffb640 1 -- 172.21.15.47:0/1216816747 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc030d30 con 0x7f2dd40ced00 2024-03-23T09:45:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.472+0000 7f2dd95c9640 1 -- 172.21.15.47:0/1216816747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd40ced00 msgr2=0x7f2dd415a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.472+0000 7f2dd95c9640 1 --2- 172.21.15.47:0/1216816747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd40ced00 0x7f2dd415a760 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0098a0 tx=0x7f2dbc02b980 comp rx=0 tx=0).stop 2024-03-23T09:45:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.472+0000 7f2dd95c9640 1 -- 172.21.15.47:0/1216816747 shutdown_connections 2024-03-23T09:45:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.472+0000 7f2dd95c9640 1 --2- 172.21.15.47:0/1216816747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd40c58a0 0x7f2dd40c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.472+0000 7f2dd95c9640 1 --2- 172.21.15.47:0/1216816747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd40ced00 0x7f2dd415a760 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.472+0000 7f2dd95c9640 1 -- 172.21.15.47:0/1216816747 >> 172.21.15.47:0/1216816747 conn(0x7f2dd40b1410 msgr2=0x7f2dd40b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.472+0000 7f2dd95c9640 1 -- 172.21.15.47:0/1216816747 shutdown_connections 2024-03-23T09:45:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.472+0000 7f2dd95c9640 1 -- 172.21.15.47:0/1216816747 wait complete. 2024-03-23T09:45:15.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.473+0000 7f2dd95c9640 1 Processor -- start 2024-03-23T09:45:15.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.473+0000 7f2dd95c9640 1 -- start start 2024-03-23T09:45:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd95c9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd40c58a0 0x7f2dd41f0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd95c9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd40ced00 0x7f2dd40c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd95c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd4159730 0x7f2dd40c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd95c9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40c7dc0 con 0x7f2dd4159730 2024-03-23T09:45:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd95c9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40c71b0 con 0x7f2dd40c58a0 2024-03-23T09:45:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd95c9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40c7490 con 0x7f2dd40ced00 2024-03-23T09:45:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd4159730 0x7f2dd40c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd4159730 0x7f2dd40c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47762/0 (socket says 172.21.15.47:47762) 2024-03-23T09:45:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd37fe640 1 -- 172.21.15.47:0/2743004761 learned_addr learned my addr 172.21.15.47:0/2743004761 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd27fc640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd40ced00 0x7f2dd40c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd2ffd640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd40c58a0 0x7f2dd41f0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd37fe640 1 -- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd40ced00 msgr2=0x7f2dd40c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd37fe640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd40ced00 0x7f2dd40c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd37fe640 1 -- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd40c58a0 msgr2=0x7f2dd41f0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd37fe640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd40c58a0 0x7f2dd41f0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd37fe640 1 -- 172.21.15.47:0/2743004761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc0094d0 con 0x7f2dd4159730 2024-03-23T09:45:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.474+0000 7f2dd2ffd640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd40c58a0 0x7f2dd41f0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.475+0000 7f2dd37fe640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd4159730 0x7f2dd40c8e10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4007de0 tx=0x7f2dc400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.475+0000 7f2dcb7fe640 1 -- 172.21.15.47:0/2743004761 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4003990 con 0x7f2dd4159730 2024-03-23T09:45:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.475+0000 7f2dcb7fe640 1 -- 172.21.15.47:0/2743004761 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc400f040 con 0x7f2dd4159730 2024-03-23T09:45:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.475+0000 7f2dd95c9640 1 -- 172.21.15.47:0/2743004761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd40c7770 con 0x7f2dd4159730 2024-03-23T09:45:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.475+0000 7f2dcb7fe640 1 -- 172.21.15.47:0/2743004761 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4003d30 con 0x7f2dd4159730 2024-03-23T09:45:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.475+0000 7f2dd95c9640 1 -- 172.21.15.47:0/2743004761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd41f08b0 con 0x7f2dd4159730 2024-03-23T09:45:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.476+0000 7f2dd95c9640 1 -- 172.21.15.47:0/2743004761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd40c7f30 con 0x7f2dd4159730 2024-03-23T09:45:15.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.476+0000 7f2dcb7fe640 1 -- 172.21.15.47:0/2743004761 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2dc4004030 con 0x7f2dd4159730 2024-03-23T09:45:15.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.477+0000 7f2dcb7fe640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2db406ac30 0x7f2db406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.477+0000 7f2dcb7fe640 1 -- 172.21.15.47:0/2743004761 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2dc4089490 con 0x7f2dd4159730 2024-03-23T09:45:15.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.477+0000 7f2dd2ffd640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2db406ac30 0x7f2db406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.477+0000 7f2dd2ffd640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2db406ac30 0x7f2db406d0f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc000f60 tx=0x7f2dcc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:15.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.481+0000 7f2dcb7fe640 1 -- 172.21.15.47:0/2743004761 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2dc40583b0 con 0x7f2dd4159730 2024-03-23T09:45:15.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.660+0000 7f2dd95c9640 1 -- 172.21.15.47:0/2743004761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2dd40672f0 con 0x7f2dd4159730 2024-03-23T09:45:15.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.661+0000 7f2dcb7fe640 1 -- 172.21.15.47:0/2743004761 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+12389 (secure 0 0 0) 0x7f2dc4003e90 con 0x7f2dd4159730 2024-03-23T09:45:15.661 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:15.661 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":66,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:45:14.962145+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:45:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.663+0000 7f2dd95c9640 1 -- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2db406ac30 msgr2=0x7f2db406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.663+0000 7f2dd95c9640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2db406ac30 0x7f2db406d0f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc000f60 tx=0x7f2dcc009040 comp rx=0 tx=0).stop 2024-03-23T09:45:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.663+0000 7f2dd95c9640 1 -- 172.21.15.47:0/2743004761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd4159730 msgr2=0x7f2dd40c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.663+0000 7f2dd95c9640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd4159730 0x7f2dd40c8e10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4007de0 tx=0x7f2dc400d4a0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.663+0000 7f2dd95c9640 1 -- 172.21.15.47:0/2743004761 shutdown_connections 2024-03-23T09:45:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.663+0000 7f2dd95c9640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2db406ac30 0x7f2db406d0f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.663+0000 7f2dd95c9640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd4159730 0x7f2dd40c8e10 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.663+0000 7f2dd95c9640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd40ced00 0x7f2dd40c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.663+0000 7f2dd95c9640 1 --2- 172.21.15.47:0/2743004761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd40c58a0 0x7f2dd41f0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.663+0000 7f2dd95c9640 1 -- 172.21.15.47:0/2743004761 >> 172.21.15.47:0/2743004761 conn(0x7f2dd40b1410 msgr2=0x7f2dd40b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.663+0000 7f2dd95c9640 1 -- 172.21.15.47:0/2743004761 shutdown_connections 2024-03-23T09:45:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.663+0000 7f2dd95c9640 1 -- 172.21.15.47:0/2743004761 wait complete. 2024-03-23T09:45:15.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:45:15.675 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:45:15.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.779+0000 7ff40a320640 1 Processor -- start 2024-03-23T09:45:15.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.779+0000 7ff40a320640 1 -- start start 2024-03-23T09:45:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.779+0000 7ff40a320640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff40415ca30 0x7ff404160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.779+0000 7ff40a320640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff404161600 con 0x7ff4040cde40 2024-03-23T09:45:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.779+0000 7ff40a320640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff404161040 con 0x7ff40415ce10 2024-03-23T09:45:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.779+0000 7ff40a320640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff404161320 con 0x7ff40415ca30 2024-03-23T09:45:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff4088bd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff4040cde40 0x7ff4040d15d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47530/0 (socket says 172.21.15.47:47530) 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff4088bd640 1 -- 172.21.15.47:0/2042905745 learned_addr learned my addr 172.21.15.47:0/2042905745 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff403fff640 1 --2- 172.21.15.47:0/2042905745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff40415ca30 0x7ff404160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214465318 0 0) 0x7ff404161600 con 0x7ff4040cde40 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3f0003520 con 0x7ff4040cde40 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1938809599 0 0) 0x7ff3f0003520 con 0x7ff4040cde40 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff40415ca30 msgr2=0x7ff404160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 --2- 172.21.15.47:0/2042905745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff40415ca30 0x7ff404160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 >> v1:172.21.15.175:6789/0 conn(0x7ff40415ce10 legacy=0x7ff4040cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4040d0dd0 con 0x7ff4040cde40 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7ff3fc0019f0 con 0x7ff4040cde40 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 >> v1:172.21.15.47:6789/0 conn(0x7ff4040cde40 legacy=0x7ff4040d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 --2- 172.21.15.47:0/2042905745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff40415ca30 0x7ff3f00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 --2- 172.21.15.47:0/2042905745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3f0003f20 0x7ff3f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 --2- 172.21.15.47:0/2042905745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f0004d00 0x7ff3f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff404161320 con 0x7ff3f0004d00 2024-03-23T09:45:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f000ba60 con 0x7ff3f0003f20 2024-03-23T09:45:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f000bc20 con 0x7ff40415ca30 2024-03-23T09:45:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff4088bd640 1 --2- 172.21.15.47:0/2042905745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f0004d00 0x7ff3f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff403fff640 1 --2- 172.21.15.47:0/2042905745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff40415ca30 0x7ff3f00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.780+0000 7ff4037fe640 1 --2- 172.21.15.47:0/2042905745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3f0003f20 0x7ff3f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.781+0000 7ff4088bd640 1 -- 172.21.15.47:0/2042905745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff40415ca30 msgr2=0x7ff3f00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.781+0000 7ff4088bd640 1 --2- 172.21.15.47:0/2042905745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff40415ca30 0x7ff3f00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.781+0000 7ff4088bd640 1 -- 172.21.15.47:0/2042905745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3f0003f20 msgr2=0x7ff3f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.781+0000 7ff4088bd640 1 --2- 172.21.15.47:0/2042905745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3f0003f20 0x7ff3f00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.781+0000 7ff4088bd640 1 -- 172.21.15.47:0/2042905745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f0003520 con 0x7ff3f0004d00 2024-03-23T09:45:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.781+0000 7ff4088bd640 1 --2- 172.21.15.47:0/2042905745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f0004d00 0x7ff3f0005180 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00a830 tx=0x7ff3f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa1e4039c7c74db server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.782+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc0072d0 con 0x7ff3f0004d00 2024-03-23T09:45:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.782+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3fc02f070 con 0x7ff3f0004d00 2024-03-23T09:45:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.782+0000 7ff402ffd640 1 -- 172.21.15.47:0/2042905745 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc007690 con 0x7ff3f0004d00 2024-03-23T09:45:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.782+0000 7ff40a320640 1 -- 172.21.15.47:0/2042905745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f0004d00 msgr2=0x7ff3f0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.782+0000 7ff40a320640 1 --2- 172.21.15.47:0/2042905745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f0004d00 0x7ff3f0005180 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00a830 tx=0x7ff3f00057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.782+0000 7ff4088bd640 1 -- 172.21.15.47:0/2042905745 reap_dead start 2024-03-23T09:45:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.782+0000 7ff40a320640 1 -- 172.21.15.47:0/2042905745 shutdown_connections 2024-03-23T09:45:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.782+0000 7ff40a320640 1 -- 172.21.15.47:0/2042905745 >> 172.21.15.47:0/2042905745 conn(0x7ff40406b160 msgr2=0x7ff40406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.782+0000 7ff40a320640 1 -- 172.21.15.47:0/2042905745 shutdown_connections 2024-03-23T09:45:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.782+0000 7ff40a320640 1 -- 172.21.15.47:0/2042905745 wait complete. 2024-03-23T09:45:15.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.783+0000 7ff40a320640 1 Processor -- start 2024-03-23T09:45:15.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.783+0000 7ff40a320640 1 -- start start 2024-03-23T09:45:15.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.784+0000 7ff40a320640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3f0004d00 0x7ff404152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.784+0000 7ff40a320640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4040cb790 0x7ff4040c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.784+0000 7ff40a320640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4040cbc10 0x7ff4040c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.784+0000 7ff40a320640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040c8060 con 0x7ff4040cbc10 2024-03-23T09:45:15.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.784+0000 7ff40a320640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4041f3170 con 0x7ff4040cb790 2024-03-23T09:45:15.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.784+0000 7ff40a320640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff404151da0 con 0x7ff3f0004d00 2024-03-23T09:45:15.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.784+0000 7ff4088bd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4040cbc10 0x7ff4040c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.784+0000 7ff4088bd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4040cbc10 0x7ff4040c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47788/0 (socket says 172.21.15.47:47788) 2024-03-23T09:45:15.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.784+0000 7ff4088bd640 1 -- 172.21.15.47:0/2381035754 learned_addr learned my addr 172.21.15.47:0/2381035754 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:15.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.784+0000 7ff403fff640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3f0004d00 0x7ff404152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.784+0000 7ff4037fe640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4040cb790 0x7ff4040c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.785+0000 7ff4088bd640 1 -- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3f0004d00 msgr2=0x7ff404152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.785+0000 7ff4088bd640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3f0004d00 0x7ff404152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.785+0000 7ff4088bd640 1 -- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4040cb790 msgr2=0x7ff4040c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.785+0000 7ff4088bd640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4040cb790 0x7ff4040c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.785+0000 7ff4088bd640 1 -- 172.21.15.47:0/2381035754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4041f2e90 con 0x7ff4040cbc10 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.785+0000 7ff4037fe640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4040cb790 0x7ff4040c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.785+0000 7ff4088bd640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4040cbc10 0x7ff4040c7b20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc02cc40 tx=0x7ff3fc001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.785+0000 7ff400ff9640 1 -- 172.21.15.47:0/2381035754 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc004d60 con 0x7ff4040cbc10 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.785+0000 7ff400ff9640 1 -- 172.21.15.47:0/2381035754 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3fc001250 con 0x7ff4040cbc10 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.785+0000 7ff400ff9640 1 -- 172.21.15.47:0/2381035754 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc0072d0 con 0x7ff4040cbc10 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.785+0000 7ff40a320640 1 -- 172.21.15.47:0/2381035754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff404151750 con 0x7ff4040cbc10 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.785+0000 7ff40a320640 1 -- 172.21.15.47:0/2381035754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4041f2180 con 0x7ff4040cbc10 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.786+0000 7ff40a320640 1 -- 172.21.15.47:0/2381035754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff404072360 con 0x7ff4040cbc10 2024-03-23T09:45:15.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.787+0000 7ff400ff9640 1 -- 172.21.15.47:0/2381035754 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff3fc031030 con 0x7ff4040cbc10 2024-03-23T09:45:15.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.787+0000 7ff400ff9640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff3e406ac30 0x7ff3e406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:15.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.787+0000 7ff400ff9640 1 -- 172.21.15.47:0/2381035754 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff3fc0aa000 con 0x7ff4040cbc10 2024-03-23T09:45:15.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.787+0000 7ff403fff640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff3e406ac30 0x7ff3e406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:15.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.788+0000 7ff403fff640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff3e406ac30 0x7ff3e406d0f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009bc0 tx=0x7ff3ec009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:15.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.791+0000 7ff400ff9640 1 -- 172.21.15.47:0/2381035754 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff3fc0755c0 con 0x7ff4040cbc10 2024-03-23T09:45:15.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.970+0000 7ff40a320640 1 -- 172.21.15.47:0/2381035754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff404151f30 con 0x7ff4040cbc10 2024-03-23T09:45:15.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.971+0000 7ff400ff9640 1 -- 172.21.15.47:0/2381035754 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3fc078be0 con 0x7ff4040cbc10 2024-03-23T09:45:15.972 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:45:15.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.973+0000 7ff40a320640 1 -- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff3e406ac30 msgr2=0x7ff3e406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.973+0000 7ff40a320640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff3e406ac30 0x7ff3e406d0f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009bc0 tx=0x7ff3ec009190 comp rx=0 tx=0).stop 2024-03-23T09:45:15.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.973+0000 7ff40a320640 1 -- 172.21.15.47:0/2381035754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4040cbc10 msgr2=0x7ff4040c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:15.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.973+0000 7ff40a320640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4040cbc10 0x7ff4040c7b20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc02cc40 tx=0x7ff3fc001d80 comp rx=0 tx=0).stop 2024-03-23T09:45:15.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.973+0000 7ff40a320640 1 -- 172.21.15.47:0/2381035754 shutdown_connections 2024-03-23T09:45:15.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.973+0000 7ff40a320640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff3e406ac30 0x7ff3e406d0f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.973+0000 7ff40a320640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4040cbc10 0x7ff4040c7b20 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.973+0000 7ff40a320640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4040cb790 0x7ff4040c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.973+0000 7ff40a320640 1 --2- 172.21.15.47:0/2381035754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3f0004d00 0x7ff404152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:15.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.973+0000 7ff40a320640 1 -- 172.21.15.47:0/2381035754 >> 172.21.15.47:0/2381035754 conn(0x7ff40406b160 msgr2=0x7ff4041540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:15.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.973+0000 7ff40a320640 1 -- 172.21.15.47:0/2381035754 shutdown_connections 2024-03-23T09:45:15.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:15.973+0000 7ff40a320640 1 -- 172.21.15.47:0/2381035754 wait complete. 2024-03-23T09:45:15.984 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T09:45:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.085+0000 7fa02fe10640 1 Processor -- start 2024-03-23T09:45:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.085+0000 7fa02fe10640 1 -- start start 2024-03-23T09:45:16.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.086+0000 7fa02fe10640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa0280cf210 0x7fa0280cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.086+0000 7fa02fe10640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0280cfbd0 con 0x7fa0280b5a90 2024-03-23T09:45:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.086+0000 7fa02fe10640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02815d0e0 con 0x7fa02815cd00 2024-03-23T09:45:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.086+0000 7fa02fe10640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0280d1540 con 0x7fa0280cf210 2024-03-23T09:45:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.086+0000 7fa02e3ad640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa0280b5a90 0x7fa0280b5ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47534/0 (socket says 172.21.15.47:47534) 2024-03-23T09:45:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.086+0000 7fa02e3ad640 1 -- 172.21.15.47:0/3892560889 learned_addr learned my addr 172.21.15.47:0/3892560889 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.086+0000 7fa02d3ab640 1 --2- 172.21.15.47:0/3892560889 >> v2:172.21.15.175:3300/0 conn(0x7fa0280cf210 0x7fa0280cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.086+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895657243 0 0) 0x7fa0280cfbd0 con 0x7fa0280b5a90 2024-03-23T09:45:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.086+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa010003520 con 0x7fa0280b5a90 2024-03-23T09:45:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.086+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1170116700 0 0) 0x7fa010003520 con 0x7fa0280b5a90 2024-03-23T09:45:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.086+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 >> v2:172.21.15.175:3300/0 conn(0x7fa0280cf210 msgr2=0x7fa0280cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.086+0000 7fa02cbaa640 1 --2- 172.21.15.47:0/3892560889 >> v2:172.21.15.175:3300/0 conn(0x7fa0280cf210 0x7fa0280cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.086+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 >> v1:172.21.15.175:6789/0 conn(0x7fa02815cd00 legacy=0x7fa0280d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0280d1820 con 0x7fa0280b5a90 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fa020001670 con 0x7fa0280b5a90 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 >> v1:172.21.15.47:6789/0 conn(0x7fa0280b5a90 legacy=0x7fa0280b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02cbaa640 1 --2- 172.21.15.47:0/3892560889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa010003970 0x7fa010003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02cbaa640 1 --2- 172.21.15.47:0/3892560889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0100047c0 0x7fa010004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02cbaa640 1 --2- 172.21.15.47:0/3892560889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa010005180 0x7fa01000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280d1540 con 0x7fa010003970 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01000c120 con 0x7fa010005180 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01000c2e0 con 0x7fa0100047c0 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02dbac640 1 --2- 172.21.15.47:0/3892560889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa010003970 0x7fa010003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02d3ab640 1 --2- 172.21.15.47:0/3892560889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0100047c0 0x7fa010004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02e3ad640 1 --2- 172.21.15.47:0/3892560889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa010005180 0x7fa01000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02dbac640 1 -- 172.21.15.47:0/3892560889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0100047c0 msgr2=0x7fa010004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02dbac640 1 --2- 172.21.15.47:0/3892560889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0100047c0 0x7fa010004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02dbac640 1 -- 172.21.15.47:0/3892560889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa010005180 msgr2=0x7fa01000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02dbac640 1 --2- 172.21.15.47:0/3892560889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa010005180 0x7fa01000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02dbac640 1 -- 172.21.15.47:0/3892560889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa010003520 con 0x7fa010003970 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02d3ab640 1 --2- 172.21.15.47:0/3892560889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0100047c0 0x7fa010004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02e3ad640 1 --2- 172.21.15.47:0/3892560889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa010005180 0x7fa01000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.087+0000 7fa02e3ad640 1 -- 172.21.15.47:0/3892560889 reap_dead start 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.088+0000 7fa02dbac640 1 --2- 172.21.15.47:0/3892560889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa010003970 0x7fa010003d90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa018001e50 tx=0x7fa0100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1ba41a04fa9240 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.088+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018003b70 con 0x7fa010003970 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.088+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa018003cd0 con 0x7fa010003970 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.088+0000 7fa02cbaa640 1 -- 172.21.15.47:0/3892560889 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018005310 con 0x7fa010003970 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.088+0000 7fa02fe10640 1 -- 172.21.15.47:0/3892560889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa010003970 msgr2=0x7fa010003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.088+0000 7fa02fe10640 1 --2- 172.21.15.47:0/3892560889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa010003970 0x7fa010003d90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa018001e50 tx=0x7fa0100057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.088+0000 7fa02fe10640 1 -- 172.21.15.47:0/3892560889 shutdown_connections 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.088+0000 7fa02fe10640 1 --2- 172.21.15.47:0/3892560889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa010003970 0x7fa010003d90 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.088+0000 7fa02fe10640 1 -- 172.21.15.47:0/3892560889 >> 172.21.15.47:0/3892560889 conn(0x7fa02806b160 msgr2=0x7fa02806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.089+0000 7fa02fe10640 1 -- 172.21.15.47:0/3892560889 shutdown_connections 2024-03-23T09:45:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.089+0000 7fa02fe10640 1 -- 172.21.15.47:0/3892560889 wait complete. 2024-03-23T09:45:16.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.090+0000 7fa02fe10640 1 Processor -- start 2024-03-23T09:45:16.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.090+0000 7fa02fe10640 1 -- start start 2024-03-23T09:45:16.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.090+0000 7fa02fe10640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa010003970 0x7fa0280cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:16.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.090+0000 7fa02fe10640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0280cc1c0 0x7fa0281f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:16.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.090+0000 7fa02fe10640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0280b2890 0x7fa0280b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.090+0000 7fa02fe10640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280ca440 con 0x7fa0280b2890 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.090+0000 7fa02fe10640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280ca690 con 0x7fa010003970 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.090+0000 7fa02fe10640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280ca970 con 0x7fa0280cc1c0 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02e3ad640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0280b2890 0x7fa0280b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02e3ad640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0280b2890 0x7fa0280b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47802/0 (socket says 172.21.15.47:47802) 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02e3ad640 1 -- 172.21.15.47:0/1646930302 learned_addr learned my addr 172.21.15.47:0/1646930302 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02d3ab640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0280cc1c0 0x7fa0281f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02dbac640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa010003970 0x7fa0280cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02e3ad640 1 -- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0280cc1c0 msgr2=0x7fa0281f1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02e3ad640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0280cc1c0 0x7fa0281f1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02e3ad640 1 -- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa010003970 msgr2=0x7fa0280cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02e3ad640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa010003970 0x7fa0280cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02e3ad640 1 -- 172.21.15.47:0/1646930302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0280cac50 con 0x7fa0280b2890 2024-03-23T09:45:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02dbac640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa010003970 0x7fa0280cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02e3ad640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0280b2890 0x7fa0280b2d10 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa020000fb0 tx=0x7fa02000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa0267fc640 1 -- 172.21.15.47:0/1646930302 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02000eed0 con 0x7fa0280b2890 2024-03-23T09:45:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa0267fc640 1 -- 172.21.15.47:0/1646930302 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02000c590 con 0x7fa0280b2890 2024-03-23T09:45:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02fe10640 1 -- 172.21.15.47:0/1646930302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0280caf30 con 0x7fa0280b2890 2024-03-23T09:45:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa0267fc640 1 -- 172.21.15.47:0/1646930302 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02000c920 con 0x7fa0280b2890 2024-03-23T09:45:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.091+0000 7fa02fe10640 1 -- 172.21.15.47:0/1646930302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0281f1f50 con 0x7fa0280b2890 2024-03-23T09:45:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.092+0000 7fa02fe10640 1 -- 172.21.15.47:0/1646930302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa028072360 con 0x7fa0280b2890 2024-03-23T09:45:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.093+0000 7fa0267fc640 1 -- 172.21.15.47:0/1646930302 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa02000ca80 con 0x7fa0280b2890 2024-03-23T09:45:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.093+0000 7fa0267fc640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa01406ac00 0x7fa01406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.093+0000 7fa0267fc640 1 -- 172.21.15.47:0/1646930302 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa020088bf0 con 0x7fa0280b2890 2024-03-23T09:45:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.093+0000 7fa02dbac640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa01406ac00 0x7fa01406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.094+0000 7fa02dbac640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa01406ac00 0x7fa01406d0c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa018034000 tx=0x7fa018009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:16.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.097+0000 7fa0267fc640 1 -- 172.21.15.47:0/1646930302 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa020057c40 con 0x7fa0280b2890 2024-03-23T09:45:16.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.279+0000 7fa02fe10640 1 -- 172.21.15.47:0/1646930302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa028072360 con 0x7fa0280b2890 2024-03-23T09:45:16.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.280+0000 7fa0267fc640 1 -- 172.21.15.47:0/1646930302 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v66) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa0200577d0 con 0x7fa0280b2890 2024-03-23T09:45:16.281 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:16.281 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-23T09:45:16.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:45:16.281 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-03-23T09:45:16.281 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-03-23T09:45:16.281 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:16.281 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-23T09:45:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.281+0000 7fa02fe10640 1 -- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa01406ac00 msgr2=0x7fa01406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.281+0000 7fa02fe10640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa01406ac00 0x7fa01406d0c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa018034000 tx=0x7fa018009030 comp rx=0 tx=0).stop 2024-03-23T09:45:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.282+0000 7fa02fe10640 1 -- 172.21.15.47:0/1646930302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0280b2890 msgr2=0x7fa0280b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.282+0000 7fa02fe10640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0280b2890 0x7fa0280b2d10 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa020000fb0 tx=0x7fa02000eb90 comp rx=0 tx=0).stop 2024-03-23T09:45:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.282+0000 7fa02fe10640 1 -- 172.21.15.47:0/1646930302 shutdown_connections 2024-03-23T09:45:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.282+0000 7fa02fe10640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa01406ac00 0x7fa01406d0c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.282+0000 7fa02fe10640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0280b2890 0x7fa0280b2d10 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.282+0000 7fa02fe10640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0280cc1c0 0x7fa0281f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.282+0000 7fa02fe10640 1 --2- 172.21.15.47:0/1646930302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa010003970 0x7fa0280cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.282+0000 7fa02fe10640 1 -- 172.21.15.47:0/1646930302 >> 172.21.15.47:0/1646930302 conn(0x7fa02806b160 msgr2=0x7fa02815e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.282+0000 7fa02fe10640 1 -- 172.21.15.47:0/1646930302 shutdown_connections 2024-03-23T09:45:16.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.282+0000 7fa02fe10640 1 -- 172.21.15.47:0/1646930302 wait complete. 2024-03-23T09:45:16.293 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T09:45:16.293 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T09:45:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.397+0000 7f555b4e9640 1 Processor -- start 2024-03-23T09:45:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.397+0000 7f555b4e9640 1 -- start start 2024-03-23T09:45:16.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.397+0000 7f555b4e9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555415d0f0 0x7f555415d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:16.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.397+0000 7f555b4e9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5554160bd0 con 0x7f555415a860 2024-03-23T09:45:16.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.397+0000 7f555b4e9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555415dbb0 con 0x7f55540cf190 2024-03-23T09:45:16.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.397+0000 7f555b4e9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55540d2790 con 0x7f555415d0f0 2024-03-23T09:45:16.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.398+0000 7f5559285640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f555415a860 0x7f5554161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47542/0 (socket says 172.21.15.47:47542) 2024-03-23T09:45:16.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.398+0000 7f5559285640 1 -- 172.21.15.47:0/901286897 learned_addr learned my addr 172.21.15.47:0/901286897 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:16.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.398+0000 7f5558a84640 1 --2- 172.21.15.47:0/901286897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555415d0f0 0x7f555415d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:16.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.398+0000 7f5553fff640 1 -- 172.21.15.47:0/901286897 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78693060 0 0) 0x7f5554160bd0 con 0x7f555415a860 2024-03-23T09:45:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.398+0000 7f5553fff640 1 -- 172.21.15.47:0/901286897 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f553c003520 con 0x7f555415a860 2024-03-23T09:45:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.398+0000 7f5553fff640 1 -- 172.21.15.47:0/901286897 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971488768 0 0) 0x7f555415dbb0 con 0x7f55540cf190 2024-03-23T09:45:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.398+0000 7f5553fff640 1 -- 172.21.15.47:0/901286897 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5554160bd0 con 0x7f55540cf190 2024-03-23T09:45:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.398+0000 7f5558a84640 1 -- 172.21.15.47:0/901286897 >> v1:172.21.15.175:6789/0 conn(0x7f55540cf190 legacy=0x7f55540cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.398+0000 7f5558a84640 1 -- 172.21.15.47:0/901286897 >> v1:172.21.15.47:6789/0 conn(0x7f555415a860 legacy=0x7f5554161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.398+0000 7f5558a84640 1 -- 172.21.15.47:0/901286897 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555415de90 con 0x7f555415d0f0 2024-03-23T09:45:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.399+0000 7f5558a84640 1 --2- 172.21.15.47:0/901286897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555415d0f0 0x7f555415d550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55480097d0 tx=0x7f554802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80e8f0427687562 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.399+0000 7f5553fff640 1 -- 172.21.15.47:0/901286897 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548031890 con 0x7f555415d0f0 2024-03-23T09:45:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.399+0000 7f5553fff640 1 -- 172.21.15.47:0/901286897 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55480319f0 con 0x7f555415d0f0 2024-03-23T09:45:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.399+0000 7f5553fff640 1 -- 172.21.15.47:0/901286897 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548031d60 con 0x7f555415d0f0 2024-03-23T09:45:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.400+0000 7f555b4e9640 1 -- 172.21.15.47:0/901286897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555415d0f0 msgr2=0x7f555415d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.400+0000 7f555b4e9640 1 --2- 172.21.15.47:0/901286897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555415d0f0 0x7f555415d550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55480097d0 tx=0x7f554802b930 comp rx=0 tx=0).stop 2024-03-23T09:45:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.400+0000 7f555b4e9640 1 -- 172.21.15.47:0/901286897 shutdown_connections 2024-03-23T09:45:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.400+0000 7f555b4e9640 1 --2- 172.21.15.47:0/901286897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555415d0f0 0x7f555415d550 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.400+0000 7f555b4e9640 1 -- 172.21.15.47:0/901286897 >> 172.21.15.47:0/901286897 conn(0x7f55540659d0 msgr2=0x7f5554065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.400+0000 7f555b4e9640 1 -- 172.21.15.47:0/901286897 shutdown_connections 2024-03-23T09:45:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.400+0000 7f555b4e9640 1 -- 172.21.15.47:0/901286897 wait complete. 2024-03-23T09:45:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.401+0000 7f555b4e9640 1 Processor -- start 2024-03-23T09:45:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.401+0000 7f555b4e9640 1 -- start start 2024-03-23T09:45:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f555b4e9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55540cf190 0x7f555415aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f555b4e9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555415a860 0x7f555414f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f555b4e9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f555415d0f0 0x7f555414f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f555b4e9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55540c84d0 con 0x7f55540cf190 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f555b4e9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55540c87b0 con 0x7f555415d0f0 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f555b4e9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55540c8a90 con 0x7f555415a860 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5559285640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55540cf190 0x7f555415aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5559285640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55540cf190 0x7f555415aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47806/0 (socket says 172.21.15.47:47806) 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5559285640 1 -- 172.21.15.47:0/3097189844 learned_addr learned my addr 172.21.15.47:0/3097189844 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5558a84640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555415a860 0x7f555414f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5559a86640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f555415d0f0 0x7f555414f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5559285640 1 -- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555415a860 msgr2=0x7f555414f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5559285640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555415a860 0x7f555414f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5559285640 1 -- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f555415d0f0 msgr2=0x7f555414f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5559285640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f555415d0f0 0x7f555414f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5559285640 1 -- 172.21.15.47:0/3097189844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5548009400 con 0x7f55540cf190 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5559a86640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f555415d0f0 0x7f555414f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5559285640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55540cf190 0x7f555415aeb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f554c00c500 tx=0x7f554c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5551ffb640 1 -- 172.21.15.47:0/3097189844 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c005ce0 con 0x7f55540cf190 2024-03-23T09:45:16.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5551ffb640 1 -- 172.21.15.47:0/3097189844 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554c005e40 con 0x7f55540cf190 2024-03-23T09:45:16.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f5551ffb640 1 -- 172.21.15.47:0/3097189844 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c0072f0 con 0x7f55540cf190 2024-03-23T09:45:16.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.402+0000 7f555b4e9640 1 -- 172.21.15.47:0/3097189844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55540c8d70 con 0x7f55540cf190 2024-03-23T09:45:16.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.403+0000 7f555b4e9640 1 -- 172.21.15.47:0/3097189844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555415b3f0 con 0x7f55540cf190 2024-03-23T09:45:16.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.404+0000 7f555b4e9640 1 -- 172.21.15.47:0/3097189844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555415b640 con 0x7f55540cf190 2024-03-23T09:45:16.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.404+0000 7f5551ffb640 1 -- 172.21.15.47:0/3097189844 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f554c007450 con 0x7f55540cf190 2024-03-23T09:45:16.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.404+0000 7f5551ffb640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f554006ac00 0x7f554006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:16.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.404+0000 7f5551ffb640 1 -- 172.21.15.47:0/3097189844 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f554c088530 con 0x7f55540cf190 2024-03-23T09:45:16.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.405+0000 7f5558a84640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f554006ac00 0x7f554006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:16.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.405+0000 7f5558a84640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f554006ac00 0x7f554006d0c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5548036000 tx=0x7f554803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:16.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.409+0000 7f5551ffb640 1 -- 172.21.15.47:0/3097189844 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f554c053ba0 con 0x7f55540cf190 2024-03-23T09:45:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.589+0000 7f555b4e9640 1 -- 172.21.15.47:0/3097189844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5554072360 con 0x7f55540cf190 2024-03-23T09:45:16.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.923+0000 7f5551ffb640 1 -- 172.21.15.47:0/3097189844 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v67) v1 ==== 138+0+0 (secure 0 0 0) 0x7f554c0170b0 con 0x7f55540cf190 2024-03-23T09:45:16.925 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-03-23T09:45:16.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.927+0000 7f555b4e9640 1 -- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f554006ac00 msgr2=0x7f554006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.927+0000 7f555b4e9640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f554006ac00 0x7f554006d0c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5548036000 tx=0x7f554803c040 comp rx=0 tx=0).stop 2024-03-23T09:45:16.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.927+0000 7f555b4e9640 1 -- 172.21.15.47:0/3097189844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55540cf190 msgr2=0x7f555415aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:16.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.927+0000 7f555b4e9640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55540cf190 0x7f555415aeb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f554c00c500 tx=0x7f554c00c9c0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.927+0000 7f555b4e9640 1 -- 172.21.15.47:0/3097189844 shutdown_connections 2024-03-23T09:45:16.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.927+0000 7f555b4e9640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f554006ac00 0x7f554006d0c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.927+0000 7f555b4e9640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f555415d0f0 0x7f555414f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.927+0000 7f555b4e9640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555415a860 0x7f555414f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.927+0000 7f555b4e9640 1 --2- 172.21.15.47:0/3097189844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55540cf190 0x7f555415aeb0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:16.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.927+0000 7f555b4e9640 1 -- 172.21.15.47:0/3097189844 >> 172.21.15.47:0/3097189844 conn(0x7f55540659d0 msgr2=0x7f555415e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:16.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.928+0000 7f555b4e9640 1 -- 172.21.15.47:0/3097189844 shutdown_connections 2024-03-23T09:45:16.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:16.928+0000 7f555b4e9640 1 -- 172.21.15.47:0/3097189844 wait complete. 2024-03-23T09:45:16.941 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T09:45:17.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.049+0000 7f71ee79f640 1 Processor -- start 2024-03-23T09:45:17.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.049+0000 7f71ee79f640 1 -- start start 2024-03-23T09:45:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.050+0000 7f71ee79f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71e80d1df0 0x7f71e80b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.050+0000 7f71ee79f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e80b4d40 con 0x7f71e815cbb0 2024-03-23T09:45:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.050+0000 7f71ee79f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e815cf90 con 0x7f71e80b48f0 2024-03-23T09:45:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.050+0000 7f71ee79f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e80b5210 con 0x7f71e80d1df0 2024-03-23T09:45:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.050+0000 7f71e77fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71e80d1df0 0x7f71e80b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.050+0000 7f71e77fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71e80d1df0 0x7f71e80b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47814/0 (socket says 172.21.15.47:47814) 2024-03-23T09:45:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.050+0000 7f71e7fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f71e815cbb0 0x7f71e80b8250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47554/0 (socket says 172.21.15.47:47554) 2024-03-23T09:45:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.050+0000 7f71e77fe640 1 -- 172.21.15.47:0/3321974286 learned_addr learned my addr 172.21.15.47:0/3321974286 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.050+0000 7f71e77fe640 1 -- 172.21.15.47:0/3321974286 >> v1:172.21.15.175:6789/0 conn(0x7f71e80b48f0 legacy=0x7f71e80b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.050+0000 7f71e77fe640 1 -- 172.21.15.47:0/3321974286 >> v1:172.21.15.47:6789/0 conn(0x7f71e815cbb0 legacy=0x7f71e80b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.050+0000 7f71e77fe640 1 -- 172.21.15.47:0/3321974286 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e80d0d50 con 0x7f71e80d1df0 2024-03-23T09:45:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.050+0000 7f71e6ffd640 1 -- 172.21.15.47:0/3321974286 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799311278 0 0) 0x7f71d0001110 con 0x7f71e815cbb0 2024-03-23T09:45:17.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.051+0000 7f71e77fe640 1 --2- 172.21.15.47:0/3321974286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71e80d1df0 0x7f71e80b43b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009870 tx=0x7f71d802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54056feb12017c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:17.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.051+0000 7f71e6ffd640 1 -- 172.21.15.47:0/3321974286 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8030860 con 0x7f71e80d1df0 2024-03-23T09:45:17.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.051+0000 7f71e6ffd640 1 -- 172.21.15.47:0/3321974286 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d80309c0 con 0x7f71e80d1df0 2024-03-23T09:45:17.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.051+0000 7f71e6ffd640 1 -- 172.21.15.47:0/3321974286 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8030d50 con 0x7f71e80d1df0 2024-03-23T09:45:17.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.052+0000 7f71ee79f640 1 -- 172.21.15.47:0/3321974286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71e80d1df0 msgr2=0x7f71e80b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:17.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.052+0000 7f71ee79f640 1 --2- 172.21.15.47:0/3321974286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71e80d1df0 0x7f71e80b43b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009870 tx=0x7f71d802b9a0 comp rx=0 tx=0).stop 2024-03-23T09:45:17.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.052+0000 7f71ee79f640 1 -- 172.21.15.47:0/3321974286 shutdown_connections 2024-03-23T09:45:17.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.052+0000 7f71ee79f640 1 --2- 172.21.15.47:0/3321974286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71e80d1df0 0x7f71e80b43b0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:17.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.052+0000 7f71ee79f640 1 -- 172.21.15.47:0/3321974286 >> 172.21.15.47:0/3321974286 conn(0x7f71e8065dc0 msgr2=0x7f71e80661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:17.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.052+0000 7f71ee79f640 1 -- 172.21.15.47:0/3321974286 shutdown_connections 2024-03-23T09:45:17.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.052+0000 7f71ee79f640 1 -- 172.21.15.47:0/3321974286 wait complete. 2024-03-23T09:45:17.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.053+0000 7f71ee79f640 1 Processor -- start 2024-03-23T09:45:17.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.053+0000 7f71ee79f640 1 -- start start 2024-03-23T09:45:17.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71ee79f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71e80b48f0 0x7f71e81f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:17.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71ee79f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71e80d1df0 0x7f71e80ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:17.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71ee79f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71e815cbb0 0x7f71e80b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:17.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71ee79f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80cc990 con 0x7f71e80b48f0 2024-03-23T09:45:17.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71ee79f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e814a080 con 0x7f71e80d1df0 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71ee79f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80cc3d0 con 0x7f71e815cbb0 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71e7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71e80b48f0 0x7f71e81f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71e7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71e80b48f0 0x7f71e81f1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47824/0 (socket says 172.21.15.47:47824) 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71e7fff640 1 -- 172.21.15.47:0/1813218025 learned_addr learned my addr 172.21.15.47:0/1813218025 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71e77fe640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71e80d1df0 0x7f71e80ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71ecd3c640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71e815cbb0 0x7f71e80b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71e7fff640 1 -- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71e815cbb0 msgr2=0x7f71e80b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71e7fff640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71e815cbb0 0x7f71e80b6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71e7fff640 1 -- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71e80d1df0 msgr2=0x7f71e80ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71e7fff640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71e80d1df0 0x7f71e80ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71e7fff640 1 -- 172.21.15.47:0/1813218025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d80094d0 con 0x7f71e80b48f0 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71e7fff640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71e80b48f0 0x7f71e81f1ba0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f71d000c430 tx=0x7f71d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71e4ff9640 1 -- 172.21.15.47:0/1813218025 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d00074e0 con 0x7f71e80b48f0 2024-03-23T09:45:17.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71e4ff9640 1 -- 172.21.15.47:0/1813218025 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d0007660 con 0x7f71e80b48f0 2024-03-23T09:45:17.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71ee79f640 1 -- 172.21.15.47:0/1813218025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e80cc6b0 con 0x7f71e80b48f0 2024-03-23T09:45:17.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71e4ff9640 1 -- 172.21.15.47:0/1813218025 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0004a20 con 0x7f71e80b48f0 2024-03-23T09:45:17.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.054+0000 7f71ee79f640 1 -- 172.21.15.47:0/1813218025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e8149b60 con 0x7f71e80b48f0 2024-03-23T09:45:17.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.055+0000 7f71ee79f640 1 -- 172.21.15.47:0/1813218025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e8072360 con 0x7f71e80b48f0 2024-03-23T09:45:17.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.056+0000 7f71e4ff9640 1 -- 172.21.15.47:0/1813218025 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f71d0004b80 con 0x7f71e80b48f0 2024-03-23T09:45:17.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.056+0000 7f71e4ff9640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f71c806ac30 0x7f71c806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:17.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.056+0000 7f71e4ff9640 1 -- 172.21.15.47:0/1813218025 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f71d0088d90 con 0x7f71e80b48f0 2024-03-23T09:45:17.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.056+0000 7f71e77fe640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f71c806ac30 0x7f71c806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:17.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.057+0000 7f71e77fe640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f71c806ac30 0x7f71c806d0f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f71d8036000 tx=0x7f71d8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:17.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.061+0000 7f71e4ff9640 1 -- 172.21.15.47:0/1813218025 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f71d00541e0 con 0x7f71e80b48f0 2024-03-23T09:45:17.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.241+0000 7f71ee79f640 1 -- 172.21.15.47:0/1813218025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f71e80b4d40 con 0x7f71e80b48f0 2024-03-23T09:45:17.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.927+0000 7f71e4ff9640 1 -- 172.21.15.47:0/1813218025 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v68) v1 ==== 170+0+0 (secure 0 0 0) 0x7f71e80b4d40 con 0x7f71e80b48f0 2024-03-23T09:45:17.928 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-03-23T09:45:17.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.932+0000 7f71ee79f640 1 -- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f71c806ac30 msgr2=0x7f71c806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:17.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.932+0000 7f71ee79f640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f71c806ac30 0x7f71c806d0f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f71d8036000 tx=0x7f71d8008030 comp rx=0 tx=0).stop 2024-03-23T09:45:17.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.932+0000 7f71ee79f640 1 -- 172.21.15.47:0/1813218025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71e80b48f0 msgr2=0x7f71e81f1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:17.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.932+0000 7f71ee79f640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71e80b48f0 0x7f71e81f1ba0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f71d000c430 tx=0x7f71d000c8f0 comp rx=0 tx=0).stop 2024-03-23T09:45:17.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.933+0000 7f71ee79f640 1 -- 172.21.15.47:0/1813218025 shutdown_connections 2024-03-23T09:45:17.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.933+0000 7f71ee79f640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f71c806ac30 0x7f71c806d0f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:17.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.933+0000 7f71ee79f640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71e815cbb0 0x7f71e80b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:17.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.933+0000 7f71ee79f640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71e80d1df0 0x7f71e80ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:17.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.933+0000 7f71ee79f640 1 --2- 172.21.15.47:0/1813218025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71e80b48f0 0x7f71e81f1ba0 secure :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f71d000c430 tx=0x7f71d000c8f0 comp rx=0 tx=0).stop 2024-03-23T09:45:17.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.933+0000 7f71ee79f640 1 -- 172.21.15.47:0/1813218025 >> 172.21.15.47:0/1813218025 conn(0x7f71e8065dc0 msgr2=0x7f71e80d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:17.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.933+0000 7f71ee79f640 1 -- 172.21.15.47:0/1813218025 shutdown_connections 2024-03-23T09:45:17.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:17.933+0000 7f71ee79f640 1 -- 172.21.15.47:0/1813218025 wait complete. 2024-03-23T09:45:17.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T09:45:18.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.064+0000 7f7a59399640 1 Processor -- start 2024-03-23T09:45:18.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.064+0000 7f7a59399640 1 -- start start 2024-03-23T09:45:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.065+0000 7f7a59399640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7a5415a850 0x7f7a5406abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.065+0000 7f7a59399640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5415ac30 con 0x7f7a540c9aa0 2024-03-23T09:45:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.065+0000 7f7a59399640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a540ae640 con 0x7f7a54069ce0 2024-03-23T09:45:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.065+0000 7f7a59399640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5406d150 con 0x7f7a5415a850 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.065+0000 7f7a537fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7a540c9aa0 0x7f7a5406d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47558/0 (socket says 172.21.15.47:47558) 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.065+0000 7f7a537fe640 1 -- 172.21.15.47:0/69919374 learned_addr learned my addr 172.21.15.47:0/69919374 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a52ffd640 1 --2- 172.21.15.47:0/69919374 >> v2:172.21.15.175:3300/0 conn(0x7f7a5415a850 0x7f7a5406abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022851065 0 0) 0x7f7a5415ac30 con 0x7f7a540c9aa0 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a40003520 con 0x7f7a540c9aa0 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3304930065 0 0) 0x7f7a40003520 con 0x7f7a540c9aa0 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 >> v2:172.21.15.175:3300/0 conn(0x7f7a5415a850 msgr2=0x7f7a5406abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 --2- 172.21.15.47:0/69919374 >> v2:172.21.15.175:3300/0 conn(0x7f7a5415a850 0x7f7a5406abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 >> v1:172.21.15.175:6789/0 conn(0x7f7a54069ce0 legacy=0x7f7a540c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5406b270 con 0x7f7a540c9aa0 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f7a440019f0 con 0x7f7a540c9aa0 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 >> v1:172.21.15.47:6789/0 conn(0x7f7a540c9aa0 legacy=0x7f7a5406d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 --2- 172.21.15.47:0/69919374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a40003970 0x7f7a40003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 --2- 172.21.15.47:0/69919374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a400047c0 0x7f7a40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 --2- 172.21.15.47:0/69919374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a40005180 0x7f7a4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5406d150 con 0x7f7a40005180 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4000c120 con 0x7f7a40003970 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4000c2e0 con 0x7f7a400047c0 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a537fe640 1 --2- 172.21.15.47:0/69919374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a40005180 0x7f7a4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a52ffd640 1 --2- 172.21.15.47:0/69919374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a40003970 0x7f7a40003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a527fc640 1 --2- 172.21.15.47:0/69919374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a400047c0 0x7f7a40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a537fe640 1 -- 172.21.15.47:0/69919374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a400047c0 msgr2=0x7f7a40004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a537fe640 1 --2- 172.21.15.47:0/69919374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a400047c0 0x7f7a40004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a537fe640 1 -- 172.21.15.47:0/69919374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a40003970 msgr2=0x7f7a40003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a537fe640 1 --2- 172.21.15.47:0/69919374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a40003970 0x7f7a40003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.066+0000 7f7a537fe640 1 -- 172.21.15.47:0/69919374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a40003520 con 0x7f7a40005180 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.067+0000 7f7a537fe640 1 -- 172.21.15.47:0/69919374 reap_dead start 2024-03-23T09:45:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.067+0000 7f7a537fe640 1 --2- 172.21.15.47:0/69919374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a40005180 0x7f7a4000b980 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400a800 tx=0x7f7a400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3060fa71a00dc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.067+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440072d0 con 0x7f7a40005180 2024-03-23T09:45:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.067+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4402f070 con 0x7f7a40005180 2024-03-23T09:45:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.067+0000 7f7a51ffb640 1 -- 172.21.15.47:0/69919374 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44007670 con 0x7f7a40005180 2024-03-23T09:45:18.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.068+0000 7f7a59399640 1 -- 172.21.15.47:0/69919374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a40005180 msgr2=0x7f7a4000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.068+0000 7f7a59399640 1 --2- 172.21.15.47:0/69919374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a40005180 0x7f7a4000b980 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400a800 tx=0x7f7a400057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.068+0000 7f7a59399640 1 -- 172.21.15.47:0/69919374 shutdown_connections 2024-03-23T09:45:18.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.068+0000 7f7a59399640 1 --2- 172.21.15.47:0/69919374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a40005180 0x7f7a4000b980 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.068+0000 7f7a59399640 1 -- 172.21.15.47:0/69919374 >> 172.21.15.47:0/69919374 conn(0x7f7a540cb7c0 msgr2=0x7f7a540cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.068+0000 7f7a59399640 1 -- 172.21.15.47:0/69919374 shutdown_connections 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.068+0000 7f7a59399640 1 -- 172.21.15.47:0/69919374 wait complete. 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.069+0000 7f7a59399640 1 Processor -- start 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.069+0000 7f7a59399640 1 -- start start 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a59399640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a40005180 0x7f7a54157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a59399640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a540cd320 0x7f7a540cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a59399640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a5415cbb0 0x7f7a5415d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a59399640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5415d570 con 0x7f7a40005180 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a59399640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540c76c0 con 0x7f7a5415cbb0 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a59399640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540ad030 con 0x7f7a540cd320 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a52ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a40005180 0x7f7a54157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a52ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a40005180 0x7f7a54157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47840/0 (socket says 172.21.15.47:47840) 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a52ffd640 1 -- 172.21.15.47:0/2927749737 learned_addr learned my addr 172.21.15.47:0/2927749737 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a527fc640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a540cd320 0x7f7a540cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a537fe640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a5415cbb0 0x7f7a5415d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a52ffd640 1 -- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a540cd320 msgr2=0x7f7a540cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a52ffd640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a540cd320 0x7f7a540cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a52ffd640 1 -- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a5415cbb0 msgr2=0x7f7a5415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a52ffd640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a5415cbb0 0x7f7a5415d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.070+0000 7f7a52ffd640 1 -- 172.21.15.47:0/2927749737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a540cde90 con 0x7f7a40005180 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.071+0000 7f7a52ffd640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a40005180 0x7f7a54157480 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c011910 tx=0x7f7a4c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.071+0000 7f7a4b7fe640 1 -- 172.21.15.47:0/2927749737 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00fa90 con 0x7f7a40005180 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.071+0000 7f7a4b7fe640 1 -- 172.21.15.47:0/2927749737 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c00fbf0 con 0x7f7a40005180 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.071+0000 7f7a59399640 1 -- 172.21.15.47:0/2927749737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a540ce170 con 0x7f7a40005180 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.071+0000 7f7a4b7fe640 1 -- 172.21.15.47:0/2927749737 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c014440 con 0x7f7a40005180 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.071+0000 7f7a59399640 1 -- 172.21.15.47:0/2927749737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a541579c0 con 0x7f7a40005180 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.072+0000 7f7a59399640 1 -- 172.21.15.47:0/2927749737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a540672f0 con 0x7f7a40005180 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.072+0000 7f7a4b7fe640 1 -- 172.21.15.47:0/2927749737 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7a4c0145a0 con 0x7f7a40005180 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.073+0000 7f7a4b7fe640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a3406ac00 0x7f7a3406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.073+0000 7f7a4b7fe640 1 -- 172.21.15.47:0/2927749737 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f7a4c089a00 con 0x7f7a40005180 2024-03-23T09:45:18.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.073+0000 7f7a527fc640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a3406ac00 0x7f7a3406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.073+0000 7f7a527fc640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a3406ac00 0x7f7a3406d0c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c000c00 tx=0x7f7a3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:18.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.077+0000 7f7a4b7fe640 1 -- 172.21.15.47:0/2927749737 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7a4c054cd0 con 0x7f7a40005180 2024-03-23T09:45:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.294+0000 7f7a59399640 1 -- 172.21.15.47:0/2927749737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7a54065a60 con 0x7f7a40005180 2024-03-23T09:45:18.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.300+0000 7f7a4b7fe640 1 -- 172.21.15.47:0/2927749737 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v30) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7a4c0582f0 con 0x7f7a40005180 2024-03-23T09:45:18.301 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 10 and data pool 11 2024-03-23T09:45:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.305+0000 7f7a59399640 1 -- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a3406ac00 msgr2=0x7f7a3406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.305+0000 7f7a59399640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a3406ac00 0x7f7a3406d0c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c000c00 tx=0x7f7a3c009040 comp rx=0 tx=0).stop 2024-03-23T09:45:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.305+0000 7f7a59399640 1 -- 172.21.15.47:0/2927749737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a40005180 msgr2=0x7f7a54157480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.305+0000 7f7a59399640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a40005180 0x7f7a54157480 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c011910 tx=0x7f7a4c011dd0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.310+0000 7f7a59399640 1 -- 172.21.15.47:0/2927749737 shutdown_connections 2024-03-23T09:45:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.310+0000 7f7a59399640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a3406ac00 0x7f7a3406d0c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.310+0000 7f7a59399640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a5415cbb0 0x7f7a5415d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.310+0000 7f7a59399640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a540cd320 0x7f7a540cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.310+0000 7f7a59399640 1 --2- 172.21.15.47:0/2927749737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a40005180 0x7f7a54157480 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.311+0000 7f7a59399640 1 -- 172.21.15.47:0/2927749737 >> 172.21.15.47:0/2927749737 conn(0x7f7a540cb7c0 msgr2=0x7f7a540b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.313+0000 7f7a59399640 1 -- 172.21.15.47:0/2927749737 shutdown_connections 2024-03-23T09:45:18.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.313+0000 7f7a59399640 1 -- 172.21.15.47:0/2927749737 wait complete. 2024-03-23T09:45:18.331 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:45:18.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.431+0000 7f997bce9640 1 Processor -- start 2024-03-23T09:45:18.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.431+0000 7f997bce9640 1 -- start start 2024-03-23T09:45:18.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.432+0000 7f997bce9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f99740cf690 0x7f99740b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.432+0000 7f997bce9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997415d0e0 con 0x7f997415cd00 2024-03-23T09:45:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.432+0000 7f997bce9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99740d1540 con 0x7f99740cf210 2024-03-23T09:45:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.432+0000 7f997bce9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99740d1820 con 0x7f99740cf690 2024-03-23T09:45:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.432+0000 7f9979a85640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f997415cd00 0x7f99740d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47572/0 (socket says 172.21.15.47:47572) 2024-03-23T09:45:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.432+0000 7f9979a85640 1 -- 172.21.15.47:0/66634341 learned_addr learned my addr 172.21.15.47:0/66634341 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.432+0000 7f997a286640 1 --2- 172.21.15.47:0/66634341 >> v2:172.21.15.175:3300/0 conn(0x7f99740cf690 0x7f99740b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.433+0000 7f9978a83640 1 -- 172.21.15.47:0/66634341 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230416897 0 0) 0x7f997415d0e0 con 0x7f997415cd00 2024-03-23T09:45:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.433+0000 7f9978a83640 1 -- 172.21.15.47:0/66634341 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9960003520 con 0x7f997415cd00 2024-03-23T09:45:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.433+0000 7f997a286640 1 -- 172.21.15.47:0/66634341 >> v1:172.21.15.175:6789/0 conn(0x7f99740cf210 legacy=0x7f99740b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.433+0000 7f997a286640 1 -- 172.21.15.47:0/66634341 >> v1:172.21.15.47:6789/0 conn(0x7f997415cd00 legacy=0x7f99740d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.433+0000 7f9978a83640 1 -- 172.21.15.47:0/66634341 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 221271839 0 0) 0x7f9960003520 con 0x7f997415cd00 2024-03-23T09:45:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.433+0000 7f997a286640 1 -- 172.21.15.47:0/66634341 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740d1b00 con 0x7f99740cf690 2024-03-23T09:45:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.434+0000 7f997a286640 1 --2- 172.21.15.47:0/66634341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99740cf690 0x7f99740b61a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9970009d90 tx=0x7f997002bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c20c56dde807cc0f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.434+0000 7f9978a83640 1 -- 172.21.15.47:0/66634341 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c001670 con 0x7f99740cf690 2024-03-23T09:45:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.434+0000 7f9978a83640 1 -- 172.21.15.47:0/66634341 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997002c8b0 con 0x7f99740cf690 2024-03-23T09:45:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.434+0000 7f9978a83640 1 -- 172.21.15.47:0/66634341 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997002cc20 con 0x7f99740cf690 2024-03-23T09:45:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.434+0000 7f997bce9640 1 -- 172.21.15.47:0/66634341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99740cf690 msgr2=0x7f99740b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.434+0000 7f997bce9640 1 --2- 172.21.15.47:0/66634341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99740cf690 0x7f99740b61a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9970009d90 tx=0x7f997002bd80 comp rx=0 tx=0).stop 2024-03-23T09:45:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.434+0000 7f997bce9640 1 -- 172.21.15.47:0/66634341 shutdown_connections 2024-03-23T09:45:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.434+0000 7f997bce9640 1 --2- 172.21.15.47:0/66634341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99740cf690 0x7f99740b61a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.434+0000 7f997bce9640 1 -- 172.21.15.47:0/66634341 >> 172.21.15.47:0/66634341 conn(0x7f997406b160 msgr2=0x7f997406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.435+0000 7f997bce9640 1 -- 172.21.15.47:0/66634341 shutdown_connections 2024-03-23T09:45:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.435+0000 7f997bce9640 1 -- 172.21.15.47:0/66634341 wait complete. 2024-03-23T09:45:18.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.436+0000 7f997bce9640 1 Processor -- start 2024-03-23T09:45:18.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.436+0000 7f997bce9640 1 -- start start 2024-03-23T09:45:18.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.436+0000 7f997bce9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99740cf210 0x7f997415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.436+0000 7f997bce9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99740cf690 0x7f99740a9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.436+0000 7f997bce9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f997415cd00 0x7f99740b7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.436+0000 7f997bce9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99740aa7e0 con 0x7f99740cf690 2024-03-23T09:45:18.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.436+0000 7f997bce9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99740aaac0 con 0x7f99740cf210 2024-03-23T09:45:18.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.436+0000 7f997bce9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99740aada0 con 0x7f997415cd00 2024-03-23T09:45:18.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.436+0000 7f9979284640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99740cf690 0x7f99740a9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.436+0000 7f9979284640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99740cf690 0x7f99740a9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47850/0 (socket says 172.21.15.47:47850) 2024-03-23T09:45:18.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f9979284640 1 -- 172.21.15.47:0/472748580 learned_addr learned my addr 172.21.15.47:0/472748580 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:18.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f9979a85640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99740cf210 0x7f997415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f997a286640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f997415cd00 0x7f99740b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f9979284640 1 -- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f997415cd00 msgr2=0x7f99740b7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f9979284640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f997415cd00 0x7f99740b7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f9979284640 1 -- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99740cf210 msgr2=0x7f997415a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f9979284640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99740cf210 0x7f997415a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f9979284640 1 -- 172.21.15.47:0/472748580 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9970009a70 con 0x7f99740cf690 2024-03-23T09:45:18.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f9979a85640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99740cf210 0x7f997415a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:18.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f997a286640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f997415cd00 0x7f99740b7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:18.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f9979284640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99740cf690 0x7f99740a9ce0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f996400c450 tx=0x7f996400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:18.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f996a7fc640 1 -- 172.21.15.47:0/472748580 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99640052d0 con 0x7f99740cf690 2024-03-23T09:45:18.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f996a7fc640 1 -- 172.21.15.47:0/472748580 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9964005430 con 0x7f99740cf690 2024-03-23T09:45:18.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f997bce9640 1 -- 172.21.15.47:0/472748580 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99740ab080 con 0x7f99740cf690 2024-03-23T09:45:18.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f996a7fc640 1 -- 172.21.15.47:0/472748580 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964012870 con 0x7f99740cf690 2024-03-23T09:45:18.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.437+0000 7f997bce9640 1 -- 172.21.15.47:0/472748580 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997415a870 con 0x7f99740cf690 2024-03-23T09:45:18.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.438+0000 7f997bce9640 1 -- 172.21.15.47:0/472748580 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9974072360 con 0x7f99740cf690 2024-03-23T09:45:18.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.439+0000 7f996a7fc640 1 -- 172.21.15.47:0/472748580 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f99640129d0 con 0x7f99740cf690 2024-03-23T09:45:18.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.439+0000 7f996a7fc640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f995806ac00 0x7f995806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.439+0000 7f996a7fc640 1 -- 172.21.15.47:0/472748580 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9964088eb0 con 0x7f99740cf690 2024-03-23T09:45:18.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.439+0000 7f9979a85640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f995806ac00 0x7f995806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.440+0000 7f9979a85640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f995806ac00 0x7f995806d0c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f996c005880 tx=0x7f996c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:18.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.443+0000 7f996a7fc640 1 -- 172.21.15.47:0/472748580 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f99640540a0 con 0x7f99740cf690 2024-03-23T09:45:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.626+0000 7f997bce9640 1 -- 172.21.15.47:0/472748580 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99740aca70 con 0x7f99740cf690 2024-03-23T09:45:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.628+0000 7f996a7fc640 1 -- 172.21.15.47:0/472748580 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16113 (secure 0 0 0) 0x7f99640576c0 con 0x7f99740cf690 2024-03-23T09:45:18.629 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:18.629 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":69,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:45:18.296349+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:45:16.609941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-23T09:45:17.261503+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:45:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.630+0000 7f997bce9640 1 -- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f995806ac00 msgr2=0x7f995806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.630+0000 7f997bce9640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f995806ac00 0x7f995806d0c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f996c005880 tx=0x7f996c009040 comp rx=0 tx=0).stop 2024-03-23T09:45:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.630+0000 7f997bce9640 1 -- 172.21.15.47:0/472748580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99740cf690 msgr2=0x7f99740a9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.630+0000 7f997bce9640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99740cf690 0x7f99740a9ce0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f996400c450 tx=0x7f996400c910 comp rx=0 tx=0).stop 2024-03-23T09:45:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.630+0000 7f997bce9640 1 -- 172.21.15.47:0/472748580 shutdown_connections 2024-03-23T09:45:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.630+0000 7f997bce9640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f995806ac00 0x7f995806d0c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.630+0000 7f997bce9640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f997415cd00 0x7f99740b7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.630+0000 7f997bce9640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99740cf690 0x7f99740a9ce0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.630+0000 7f997bce9640 1 --2- 172.21.15.47:0/472748580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99740cf210 0x7f997415a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.630+0000 7f997bce9640 1 -- 172.21.15.47:0/472748580 >> 172.21.15.47:0/472748580 conn(0x7f997406b160 msgr2=0x7f99740cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.630+0000 7f997bce9640 1 -- 172.21.15.47:0/472748580 shutdown_connections 2024-03-23T09:45:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.630+0000 7f997bce9640 1 -- 172.21.15.47:0/472748580 wait complete. 2024-03-23T09:45:18.642 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:45:18.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.742+0000 7fce8cad5640 1 Processor -- start 2024-03-23T09:45:18.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.742+0000 7fce8cad5640 1 -- start start 2024-03-23T09:45:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.742+0000 7fce8cad5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce880b2670 0x7fce880b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.742+0000 7fce8cad5640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fce8815d010 0x7fce88070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.742+0000 7fce8cad5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce880cb110 con 0x7fce8815d470 2024-03-23T09:45:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.742+0000 7fce8cad5640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8806cb40 con 0x7fce8815d010 2024-03-23T09:45:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.742+0000 7fce8cad5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8815d9e0 con 0x7fce880b2670 2024-03-23T09:45:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.743+0000 7fce8659c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce880b2670 0x7fce880b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.743+0000 7fce7ffff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fce8815d010 0x7fce88070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.743+0000 7fce86d9d640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fce8815d470 0x7fce88160680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:42726/0 (socket says 172.21.15.47:42726) 2024-03-23T09:45:18.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.743+0000 7fce8659c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce880b2670 0x7fce880b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:41966/0 (socket says 172.21.15.47:41966) 2024-03-23T09:45:18.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.743+0000 7fce86d9d640 1 -- 172.21.15.47:0/268786386 learned_addr learned my addr 172.21.15.47:0/268786386 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:18.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.743+0000 7fce7ffff640 1 -- 172.21.15.47:0/268786386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce880b2670 msgr2=0x7fce880b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:18.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.743+0000 7fce7ffff640 1 --2- 172.21.15.47:0/268786386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce880b2670 0x7fce880b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.743+0000 7fce7ffff640 1 -- 172.21.15.47:0/268786386 >> v1:172.21.15.175:6789/0 conn(0x7fce8815d470 legacy=0x7fce88160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.743+0000 7fce85d9b640 1 -- 172.21.15.47:0/268786386 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713075465 0 0) 0x7fce880cb110 con 0x7fce8815d470 2024-03-23T09:45:18.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.743+0000 7fce7ffff640 1 -- 172.21.15.47:0/268786386 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8815dcc0 con 0x7fce8815d010 2024-03-23T09:45:18.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.744+0000 7fce7ffff640 1 --2- 172.21.15.47:0/268786386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815d010 0x7fce88070660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fce700097a0 tx=0x7fce7002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d0fa4f4e7f073e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:18.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.744+0000 7fce85d9b640 1 -- 172.21.15.47:0/268786386 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7002e070 con 0x7fce8815d010 2024-03-23T09:45:18.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.744+0000 7fce85d9b640 1 -- 172.21.15.47:0/268786386 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce700315a0 con 0x7fce8815d010 2024-03-23T09:45:18.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.744+0000 7fce85d9b640 1 -- 172.21.15.47:0/268786386 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70031930 con 0x7fce8815d010 2024-03-23T09:45:18.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.745+0000 7fce8cad5640 1 -- 172.21.15.47:0/268786386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815d010 msgr2=0x7fce88070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.745+0000 7fce8cad5640 1 --2- 172.21.15.47:0/268786386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815d010 0x7fce88070660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fce700097a0 tx=0x7fce7002b840 comp rx=0 tx=0).stop 2024-03-23T09:45:18.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.745+0000 7fce8cad5640 1 -- 172.21.15.47:0/268786386 shutdown_connections 2024-03-23T09:45:18.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.745+0000 7fce8cad5640 1 --2- 172.21.15.47:0/268786386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815d010 0x7fce88070660 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.745+0000 7fce8cad5640 1 --2- 172.21.15.47:0/268786386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce880b2670 0x7fce880b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.745+0000 7fce8cad5640 1 -- 172.21.15.47:0/268786386 >> 172.21.15.47:0/268786386 conn(0x7fce880b06e0 msgr2=0x7fce880b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:18.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.745+0000 7fce8cad5640 1 -- 172.21.15.47:0/268786386 shutdown_connections 2024-03-23T09:45:18.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.745+0000 7fce8cad5640 1 -- 172.21.15.47:0/268786386 wait complete. 2024-03-23T09:45:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.746+0000 7fce8cad5640 1 Processor -- start 2024-03-23T09:45:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.746+0000 7fce8cad5640 1 -- start start 2024-03-23T09:45:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce8cad5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce880b2670 0x7fce88150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce8cad5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce8815d010 0x7fce880adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce8cad5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815d470 0x7fce88154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce8cad5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce880acab0 con 0x7fce8815d010 2024-03-23T09:45:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce8cad5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce880acd90 con 0x7fce8815d470 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce8cad5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce880ad070 con 0x7fce880b2670 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce8815d010 0x7fce880adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce8815d010 0x7fce880adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47864/0 (socket says 172.21.15.47:47864) 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce7ffff640 1 -- 172.21.15.47:0/2709129296 learned_addr learned my addr 172.21.15.47:0/2709129296 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce8659c640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce880b2670 0x7fce88150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce86d9d640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815d470 0x7fce88154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce7ffff640 1 -- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce880b2670 msgr2=0x7fce88150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce7ffff640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce880b2670 0x7fce88150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce7ffff640 1 -- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815d470 msgr2=0x7fce88154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce7ffff640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815d470 0x7fce88154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce7ffff640 1 -- 172.21.15.47:0/2709129296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce70009400 con 0x7fce8815d010 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce7ffff640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce8815d010 0x7fce880adf50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fce700097a0 tx=0x7fce700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.747+0000 7fce7effd640 1 -- 172.21.15.47:0/2709129296 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70004230 con 0x7fce8815d010 2024-03-23T09:45:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.748+0000 7fce7effd640 1 -- 172.21.15.47:0/2709129296 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce70004390 con 0x7fce8815d010 2024-03-23T09:45:18.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.748+0000 7fce7effd640 1 -- 172.21.15.47:0/2709129296 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70003850 con 0x7fce8815d010 2024-03-23T09:45:18.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.748+0000 7fce8cad5640 1 -- 172.21.15.47:0/2709129296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce880ad350 con 0x7fce8815d010 2024-03-23T09:45:18.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.748+0000 7fce8cad5640 1 -- 172.21.15.47:0/2709129296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce88155170 con 0x7fce8815d010 2024-03-23T09:45:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.749+0000 7fce8cad5640 1 -- 172.21.15.47:0/2709129296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce880ae490 con 0x7fce8815d010 2024-03-23T09:45:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.749+0000 7fce7effd640 1 -- 172.21.15.47:0/2709129296 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fce700039b0 con 0x7fce8815d010 2024-03-23T09:45:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.749+0000 7fce7effd640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce6806ac00 0x7fce6806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.749+0000 7fce7effd640 1 -- 172.21.15.47:0/2709129296 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce700ab3b0 con 0x7fce8815d010 2024-03-23T09:45:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.750+0000 7fce8659c640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce6806ac00 0x7fce6806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.750+0000 7fce8659c640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce6806ac00 0x7fce6806d0c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fce88155970 tx=0x7fce78009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.754+0000 7fce7effd640 1 -- 172.21.15.47:0/2709129296 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fce700767a0 con 0x7fce8815d010 2024-03-23T09:45:18.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.934+0000 7fce8cad5640 1 -- 172.21.15.47:0/2709129296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce88155330 con 0x7fce8815d010 2024-03-23T09:45:18.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.936+0000 7fce7effd640 1 -- 172.21.15.47:0/2709129296 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16113 (secure 0 0 0) 0x7fce70079dc0 con 0x7fce8815d010 2024-03-23T09:45:18.936 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:18.937 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":69,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:45:18.296349+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:45:16.609941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-23T09:45:17.261503+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:45:18.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.937+0000 7fce8cad5640 1 -- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce6806ac00 msgr2=0x7fce6806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.937+0000 7fce8cad5640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce6806ac00 0x7fce6806d0c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fce88155970 tx=0x7fce78009190 comp rx=0 tx=0).stop 2024-03-23T09:45:18.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.937+0000 7fce8cad5640 1 -- 172.21.15.47:0/2709129296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce8815d010 msgr2=0x7fce880adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:18.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.937+0000 7fce8cad5640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce8815d010 0x7fce880adf50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fce700097a0 tx=0x7fce700046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.937+0000 7fce8cad5640 1 -- 172.21.15.47:0/2709129296 shutdown_connections 2024-03-23T09:45:18.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.938+0000 7fce8cad5640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce6806ac00 0x7fce6806d0c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.938+0000 7fce8cad5640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815d470 0x7fce88154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.938+0000 7fce8cad5640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce8815d010 0x7fce880adf50 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.938+0000 7fce8cad5640 1 --2- 172.21.15.47:0/2709129296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce880b2670 0x7fce88150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:18.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.938+0000 7fce8cad5640 1 -- 172.21.15.47:0/2709129296 >> 172.21.15.47:0/2709129296 conn(0x7fce880b06e0 msgr2=0x7fce88070660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:18.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.938+0000 7fce8cad5640 1 -- 172.21.15.47:0/2709129296 shutdown_connections 2024-03-23T09:45:18.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:18.938+0000 7fce8cad5640 1 -- 172.21.15.47:0/2709129296 wait complete. 2024-03-23T09:45:18.950 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T09:45:19.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.053+0000 7fb7753c3640 1 Processor -- start 2024-03-23T09:45:19.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.053+0000 7fb7753c3640 1 -- start start 2024-03-23T09:45:19.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb7753c3640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb7700cde10 0x7fb770160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb7753c3640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb770161630 con 0x7fb7700b8350 2024-03-23T09:45:19.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb7753c3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7700d1030 con 0x7fb77015c990 2024-03-23T09:45:19.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb7753c3640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7700cfe10 con 0x7fb7700cde10 2024-03-23T09:45:19.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb76f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb7700b8350 0x7fb7700d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47574/0 (socket says 172.21.15.47:47574) 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb76f7fe640 1 -- 172.21.15.47:0/3484709463 learned_addr learned my addr 172.21.15.47:0/3484709463 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb767fff640 1 --2- 172.21.15.47:0/3484709463 >> v2:172.21.15.175:3300/0 conn(0x7fb7700cde10 0x7fb770160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651559746 0 0) 0x7fb770161630 con 0x7fb7700b8350 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb75c003520 con 0x7fb7700b8350 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2281005137 0 0) 0x7fb75c003520 con 0x7fb7700b8350 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 >> v2:172.21.15.175:3300/0 conn(0x7fb7700cde10 msgr2=0x7fb770160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb76e7fc640 1 --2- 172.21.15.47:0/3484709463 >> v2:172.21.15.175:3300/0 conn(0x7fb7700cde10 0x7fb770160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 >> v1:172.21.15.175:6789/0 conn(0x7fb77015c990 legacy=0x7fb7700cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7700b5b60 con 0x7fb7700b8350 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fb7680019f0 con 0x7fb7700b8350 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.054+0000 7fb767fff640 1 --2- 172.21.15.47:0/3484709463 >> v2:172.21.15.175:3300/0 conn(0x7fb7700cde10 0x7fb770160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 >> v1:172.21.15.47:6789/0 conn(0x7fb7700b8350 legacy=0x7fb7700d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76e7fc640 1 --2- 172.21.15.47:0/3484709463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb75c003970 0x7fb75c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76e7fc640 1 --2- 172.21.15.47:0/3484709463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb75c0047c0 0x7fb75c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76e7fc640 1 --2- 172.21.15.47:0/3484709463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb75c005180 0x7fb75c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7700cfe10 con 0x7fb75c003970 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c00c120 con 0x7fb75c005180 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c00c2e0 con 0x7fb75c0047c0 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76effd640 1 --2- 172.21.15.47:0/3484709463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb75c003970 0x7fb75c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb767fff640 1 --2- 172.21.15.47:0/3484709463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb75c0047c0 0x7fb75c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76f7fe640 1 --2- 172.21.15.47:0/3484709463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb75c005180 0x7fb75c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76effd640 1 -- 172.21.15.47:0/3484709463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb75c0047c0 msgr2=0x7fb75c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76effd640 1 --2- 172.21.15.47:0/3484709463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb75c0047c0 0x7fb75c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76effd640 1 -- 172.21.15.47:0/3484709463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb75c005180 msgr2=0x7fb75c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76effd640 1 --2- 172.21.15.47:0/3484709463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb75c005180 0x7fb75c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76f7fe640 1 -- 172.21.15.47:0/3484709463 reap_dead start 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.055+0000 7fb76effd640 1 -- 172.21.15.47:0/3484709463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c003520 con 0x7fb75c003970 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.056+0000 7fb76effd640 1 --2- 172.21.15.47:0/3484709463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb75c003970 0x7fb75c003d90 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb760001750 tx=0x7fb75c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc751a49a677615 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.056+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7600052d0 con 0x7fb75c003970 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.056+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76002e050 con 0x7fb75c003970 2024-03-23T09:45:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.056+0000 7fb76e7fc640 1 -- 172.21.15.47:0/3484709463 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760003830 con 0x7fb75c003970 2024-03-23T09:45:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.056+0000 7fb7753c3640 1 -- 172.21.15.47:0/3484709463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb75c003970 msgr2=0x7fb75c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.056+0000 7fb7753c3640 1 --2- 172.21.15.47:0/3484709463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb75c003970 0x7fb75c003d90 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb760001750 tx=0x7fb75c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.056+0000 7fb7753c3640 1 -- 172.21.15.47:0/3484709463 shutdown_connections 2024-03-23T09:45:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.056+0000 7fb7753c3640 1 --2- 172.21.15.47:0/3484709463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb75c003970 0x7fb75c003d90 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.056+0000 7fb7753c3640 1 -- 172.21.15.47:0/3484709463 >> 172.21.15.47:0/3484709463 conn(0x7fb7700659d0 msgr2=0x7fb770065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.056+0000 7fb7753c3640 1 -- 172.21.15.47:0/3484709463 shutdown_connections 2024-03-23T09:45:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.056+0000 7fb7753c3640 1 -- 172.21.15.47:0/3484709463 wait complete. 2024-03-23T09:45:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.058+0000 7fb7753c3640 1 Processor -- start 2024-03-23T09:45:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.058+0000 7fb7753c3640 1 -- start start 2024-03-23T09:45:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.058+0000 7fb7753c3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb75c003970 0x7fb77015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.058+0000 7fb7753c3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb77015f650 0x7fb77015fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.058+0000 7fb7753c3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7700c7b40 0x7fb7700c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.058+0000 7fb7753c3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7701508a0 con 0x7fb77015f650 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.058+0000 7fb7753c3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb770160010 con 0x7fb7700c7b40 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.058+0000 7fb7753c3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7700c8500 con 0x7fb75c003970 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.058+0000 7fb767fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb77015f650 0x7fb77015fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.058+0000 7fb767fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb77015f650 0x7fb77015fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47884/0 (socket says 172.21.15.47:47884) 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.058+0000 7fb767fff640 1 -- 172.21.15.47:0/1124798099 learned_addr learned my addr 172.21.15.47:0/1124798099 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.059+0000 7fb76effd640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb75c003970 0x7fb77015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.059+0000 7fb76f7fe640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7700c7b40 0x7fb7700c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.059+0000 7fb767fff640 1 -- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb75c003970 msgr2=0x7fb77015b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.059+0000 7fb767fff640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb75c003970 0x7fb77015b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.059+0000 7fb767fff640 1 -- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7700c7b40 msgr2=0x7fb7700c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.059+0000 7fb767fff640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7700c7b40 0x7fb7700c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.059+0000 7fb767fff640 1 -- 172.21.15.47:0/1124798099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7700c7060 con 0x7fb77015f650 2024-03-23T09:45:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.059+0000 7fb767fff640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb77015f650 0x7fb77015fad0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb758011cd0 tx=0x7fb75800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.059+0000 7fb7677fe640 1 -- 172.21.15.47:0/1124798099 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75800fd20 con 0x7fb77015f650 2024-03-23T09:45:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.059+0000 7fb7677fe640 1 -- 172.21.15.47:0/1124798099 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75800fea0 con 0x7fb77015f650 2024-03-23T09:45:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.059+0000 7fb7753c3640 1 -- 172.21.15.47:0/1124798099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7700c6450 con 0x7fb77015f650 2024-03-23T09:45:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.059+0000 7fb7677fe640 1 -- 172.21.15.47:0/1124798099 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758004260 con 0x7fb77015f650 2024-03-23T09:45:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.059+0000 7fb7753c3640 1 -- 172.21.15.47:0/1124798099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7700c6a10 con 0x7fb77015f650 2024-03-23T09:45:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.060+0000 7fb7753c3640 1 -- 172.21.15.47:0/1124798099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb770072360 con 0x7fb77015f650 2024-03-23T09:45:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.060+0000 7fb7677fe640 1 -- 172.21.15.47:0/1124798099 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb7580043c0 con 0x7fb77015f650 2024-03-23T09:45:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.061+0000 7fb7677fe640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb75006ac00 0x7fb75006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.061+0000 7fb7677fe640 1 -- 172.21.15.47:0/1124798099 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb758089a70 con 0x7fb77015f650 2024-03-23T09:45:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.061+0000 7fb76effd640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb75006ac00 0x7fb75006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.061+0000 7fb76effd640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb75006ac00 0x7fb75006d0c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb760033000 tx=0x7fb760009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:19.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.065+0000 7fb7677fe640 1 -- 172.21.15.47:0/1124798099 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb758054c50 con 0x7fb77015f650 2024-03-23T09:45:19.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.282+0000 7fb7753c3640 1 -- 172.21.15.47:0/1124798099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb7700c71f0 con 0x7fb77015f650 2024-03-23T09:45:19.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.302+0000 7fb7677fe640 1 -- 172.21.15.47:0/1124798099 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb758022070 con 0x7fb77015f650 2024-03-23T09:45:19.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.304+0000 7fb7753c3640 1 -- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb75006ac00 msgr2=0x7fb75006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:19.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.304+0000 7fb7753c3640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb75006ac00 0x7fb75006d0c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb760033000 tx=0x7fb760009030 comp rx=0 tx=0).stop 2024-03-23T09:45:19.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.304+0000 7fb7753c3640 1 -- 172.21.15.47:0/1124798099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb77015f650 msgr2=0x7fb77015fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:19.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.304+0000 7fb7753c3640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb77015f650 0x7fb77015fad0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb758011cd0 tx=0x7fb75800f610 comp rx=0 tx=0).stop 2024-03-23T09:45:19.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.304+0000 7fb7753c3640 1 -- 172.21.15.47:0/1124798099 shutdown_connections 2024-03-23T09:45:19.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.304+0000 7fb7753c3640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb75006ac00 0x7fb75006d0c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.304+0000 7fb7753c3640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7700c7b40 0x7fb7700c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.304+0000 7fb7753c3640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb77015f650 0x7fb77015fad0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.304+0000 7fb7753c3640 1 --2- 172.21.15.47:0/1124798099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb75c003970 0x7fb77015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.304+0000 7fb7753c3640 1 -- 172.21.15.47:0/1124798099 >> 172.21.15.47:0/1124798099 conn(0x7fb7700659d0 msgr2=0x7fb770156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:19.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.304+0000 7fb7753c3640 1 -- 172.21.15.47:0/1124798099 shutdown_connections 2024-03-23T09:45:19.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.304+0000 7fb7753c3640 1 -- 172.21.15.47:0/1124798099 wait complete. 2024-03-23T09:45:19.316 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-23T09:45:19.316 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-23T09:45:19.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.418+0000 7f9395896640 1 Processor -- start 2024-03-23T09:45:19.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.418+0000 7f9395896640 1 -- start start 2024-03-23T09:45:19.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.419+0000 7f9395896640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f93900b1c60 0x7f93900ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.419+0000 7f9395896640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9390161450 0x7f93900cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.419+0000 7f9395896640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93900cffc0 con 0x7f939006b690 2024-03-23T09:45:19.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.419+0000 7f9395896640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93900cf3b0 con 0x7f93900b1c60 2024-03-23T09:45:19.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.419+0000 7f9395896640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93900cf690 con 0x7f9390161450 2024-03-23T09:45:19.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.419+0000 7f938e7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f93900b1c60 0x7f93900ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.419+0000 7f938e7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f93900b1c60 0x7f93900ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47890/0 (socket says 172.21.15.47:47890) 2024-03-23T09:45:19.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.419+0000 7f938e7fc640 1 -- 172.21.15.47:0/3078609838 learned_addr learned my addr 172.21.15.47:0/3078609838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:19.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.420+0000 7f938f7fe640 1 --2- 172.21.15.47:0/3078609838 >> v2:172.21.15.175:3300/0 conn(0x7f9390161450 0x7f93900cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.420+0000 7f938e7fc640 1 -- 172.21.15.47:0/3078609838 >> v2:172.21.15.175:3300/0 conn(0x7f9390161450 msgr2=0x7f93900cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:19.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.420+0000 7f938e7fc640 1 --2- 172.21.15.47:0/3078609838 >> v2:172.21.15.175:3300/0 conn(0x7f9390161450 0x7f93900cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.420+0000 7f938e7fc640 1 -- 172.21.15.47:0/3078609838 >> v1:172.21.15.175:6789/0 conn(0x7f939006b690 legacy=0x7f939006ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:19.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.420+0000 7f938e7fc640 1 -- 172.21.15.47:0/3078609838 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93900cf970 con 0x7f93900b1c60 2024-03-23T09:45:19.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.420+0000 7f938e7fc640 1 --2- 172.21.15.47:0/3078609838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93900b1c60 0x7f93900ce9b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f93900cd110 tx=0x7f937802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59034211e5c1b11b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:19.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.421+0000 7f938dffb640 1 -- 172.21.15.47:0/3078609838 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378030860 con 0x7f93900b1c60 2024-03-23T09:45:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.421+0000 7f938dffb640 1 -- 172.21.15.47:0/3078609838 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93780309c0 con 0x7f93900b1c60 2024-03-23T09:45:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.421+0000 7f938dffb640 1 -- 172.21.15.47:0/3078609838 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378030d50 con 0x7f93900b1c60 2024-03-23T09:45:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.421+0000 7f9395896640 1 -- 172.21.15.47:0/3078609838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93900b1c60 msgr2=0x7f93900ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.421+0000 7f9395896640 1 --2- 172.21.15.47:0/3078609838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93900b1c60 0x7f93900ce9b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f93900cd110 tx=0x7f937802b9a0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.421+0000 7f9395896640 1 -- 172.21.15.47:0/3078609838 shutdown_connections 2024-03-23T09:45:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.421+0000 7f9395896640 1 --2- 172.21.15.47:0/3078609838 >> v2:172.21.15.175:3300/0 conn(0x7f9390161450 0x7f93900cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.421+0000 7f9395896640 1 --2- 172.21.15.47:0/3078609838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93900b1c60 0x7f93900ce9b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.421+0000 7f9395896640 1 -- 172.21.15.47:0/3078609838 >> 172.21.15.47:0/3078609838 conn(0x7f93900b21d0 msgr2=0x7f93900b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.421+0000 7f9395896640 1 -- 172.21.15.47:0/3078609838 shutdown_connections 2024-03-23T09:45:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.421+0000 7f9395896640 1 -- 172.21.15.47:0/3078609838 wait complete. 2024-03-23T09:45:19.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.422+0000 7f9395896640 1 Processor -- start 2024-03-23T09:45:19.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.423+0000 7f9395896640 1 -- start start 2024-03-23T09:45:19.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.423+0000 7f9395896640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f939006b690 0x7f93900c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.423+0000 7f9395896640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9390161450 0x7f9390154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.423+0000 7f9395896640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93900ca200 0x7f93900cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.423+0000 7f9395896640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93900ca6e0 con 0x7f9390161450 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.423+0000 7f9395896640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93900c4380 con 0x7f939006b690 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.423+0000 7f9395896640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9390154a80 con 0x7f93900ca200 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.423+0000 7f938e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9390161450 0x7f9390154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.423+0000 7f938e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9390161450 0x7f9390154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47892/0 (socket says 172.21.15.47:47892) 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.423+0000 7f938e7fc640 1 -- 172.21.15.47:0/45434571 learned_addr learned my addr 172.21.15.47:0/45434571 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f938f7fe640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93900ca200 0x7f93900cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f938effd640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f939006b690 0x7f93900c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f938e7fc640 1 -- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93900ca200 msgr2=0x7f93900cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f938e7fc640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93900ca200 0x7f93900cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f938e7fc640 1 -- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f939006b690 msgr2=0x7f93900c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f938e7fc640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f939006b690 0x7f93900c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f938e7fc640 1 -- 172.21.15.47:0/45434571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93780094d0 con 0x7f9390161450 2024-03-23T09:45:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f938f7fe640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93900ca200 0x7f93900cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f938e7fc640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9390161450 0x7f9390154540 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f937802b9e0 tx=0x7f9378004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f9387fff640 1 -- 172.21.15.47:0/45434571 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378039d40 con 0x7f9390161450 2024-03-23T09:45:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f9387fff640 1 -- 172.21.15.47:0/45434571 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9378043400 con 0x7f9390161450 2024-03-23T09:45:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f9395896640 1 -- 172.21.15.47:0/45434571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93900b62e0 con 0x7f9390161450 2024-03-23T09:45:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f9387fff640 1 -- 172.21.15.47:0/45434571 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378039ea0 con 0x7f9390161450 2024-03-23T09:45:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.424+0000 7f9395896640 1 -- 172.21.15.47:0/45434571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93900b5d20 con 0x7f9390161450 2024-03-23T09:45:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.425+0000 7f9395896640 1 -- 172.21.15.47:0/45434571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93900c4530 con 0x7f9390161450 2024-03-23T09:45:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.425+0000 7f9387fff640 1 -- 172.21.15.47:0/45434571 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f93780437b0 con 0x7f9390161450 2024-03-23T09:45:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.426+0000 7f9387fff640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f937006ac00 0x7f937006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.426+0000 7f9387fff640 1 -- 172.21.15.47:0/45434571 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f937803e070 con 0x7f9390161450 2024-03-23T09:45:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.426+0000 7f938effd640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f937006ac00 0x7f937006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.427+0000 7f938effd640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f937006ac00 0x7f937006d0c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f93880017f0 tx=0x7f9388009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:19.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.430+0000 7f9387fff640 1 -- 172.21.15.47:0/45434571 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f93900c4530 con 0x7f9390161450 2024-03-23T09:45:19.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:19.644+0000 7f9395896640 1 -- 172.21.15.47:0/45434571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f93900c4530 con 0x7f9390161450 2024-03-23T09:45:19.923 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:45:19.922+0000 7f8399c1c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:45:20.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.303+0000 7f9387fff640 1 -- 172.21.15.47:0/45434571 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f93900c4530 con 0x7f9390161450 2024-03-23T09:45:20.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.306+0000 7f9395896640 1 -- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f937006ac00 msgr2=0x7f937006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:20.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.306+0000 7f9395896640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f937006ac00 0x7f937006d0c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f93880017f0 tx=0x7f9388009040 comp rx=0 tx=0).stop 2024-03-23T09:45:20.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.306+0000 7f9395896640 1 -- 172.21.15.47:0/45434571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9390161450 msgr2=0x7f9390154540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:20.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.306+0000 7f9395896640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9390161450 0x7f9390154540 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f937802b9e0 tx=0x7f9378004730 comp rx=0 tx=0).stop 2024-03-23T09:45:20.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.306+0000 7f9395896640 1 -- 172.21.15.47:0/45434571 shutdown_connections 2024-03-23T09:45:20.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.306+0000 7f9395896640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f937006ac00 0x7f937006d0c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.306+0000 7f9395896640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93900ca200 0x7f93900cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.306+0000 7f9395896640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9390161450 0x7f9390154540 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.306+0000 7f9395896640 1 --2- 172.21.15.47:0/45434571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f939006b690 0x7f93900c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.306+0000 7f9395896640 1 -- 172.21.15.47:0/45434571 >> 172.21.15.47:0/45434571 conn(0x7f93900b21d0 msgr2=0x7f939006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:20.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.306+0000 7f9395896640 1 -- 172.21.15.47:0/45434571 shutdown_connections 2024-03-23T09:45:20.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.306+0000 7f9395896640 1 -- 172.21.15.47:0/45434571 wait complete. 2024-03-23T09:45:20.320 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-23T09:45:20.320 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:45:20.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.426+0000 7fe33e7f0640 1 Processor -- start 2024-03-23T09:45:20.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.426+0000 7fe33e7f0640 1 -- start start 2024-03-23T09:45:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.427+0000 7fe33e7f0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe33815cbe0 0x7fe3380ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.427+0000 7fe33e7f0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3380cad40 0x7fe3380cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.427+0000 7fe33e7f0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3380b06d0 0x7fe3380b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.427+0000 7fe33e7f0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3380b1400 con 0x7fe3380b06d0 2024-03-23T09:45:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.427+0000 7fe33e7f0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3380b16e0 con 0x7fe33815cbe0 2024-03-23T09:45:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.427+0000 7fe33e7f0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3380cedb0 con 0x7fe3380cad40 2024-03-23T09:45:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.427+0000 7fe33cd8d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3380b06d0 0x7fe3380b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.427+0000 7fe33cd8d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3380b06d0 0x7fe3380b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47894/0 (socket says 172.21.15.47:47894) 2024-03-23T09:45:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.427+0000 7fe33cd8d640 1 -- 172.21.15.47:0/2959488211 learned_addr learned my addr 172.21.15.47:0/2959488211 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.428+0000 7fe337fff640 1 --2- 172.21.15.47:0/2959488211 >> v2:172.21.15.175:3300/0 conn(0x7fe33815cbe0 0x7fe3380ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.428+0000 7fe3377fe640 1 --2- 172.21.15.47:0/2959488211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3380cad40 0x7fe3380cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.428+0000 7fe33cd8d640 1 -- 172.21.15.47:0/2959488211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3380cad40 msgr2=0x7fe3380cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.428+0000 7fe33cd8d640 1 --2- 172.21.15.47:0/2959488211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3380cad40 0x7fe3380cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.428+0000 7fe33cd8d640 1 -- 172.21.15.47:0/2959488211 >> v2:172.21.15.175:3300/0 conn(0x7fe33815cbe0 msgr2=0x7fe3380ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.428+0000 7fe33cd8d640 1 --2- 172.21.15.47:0/2959488211 >> v2:172.21.15.175:3300/0 conn(0x7fe33815cbe0 0x7fe3380ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.428+0000 7fe33cd8d640 1 -- 172.21.15.47:0/2959488211 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3380cf090 con 0x7fe3380b06d0 2024-03-23T09:45:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.428+0000 7fe33cd8d640 1 --2- 172.21.15.47:0/2959488211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3380b06d0 0x7fe3380b0b60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe3380d1530 tx=0x7fe33002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58920fe5585c6a4a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:20.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.429+0000 7fe336ffd640 1 -- 172.21.15.47:0/2959488211 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33002c9e0 con 0x7fe3380b06d0 2024-03-23T09:45:20.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.429+0000 7fe336ffd640 1 -- 172.21.15.47:0/2959488211 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33002cb40 con 0x7fe3380b06d0 2024-03-23T09:45:20.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.429+0000 7fe336ffd640 1 -- 172.21.15.47:0/2959488211 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3300343f0 con 0x7fe3380b06d0 2024-03-23T09:45:20.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.429+0000 7fe33e7f0640 1 -- 172.21.15.47:0/2959488211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3380b06d0 msgr2=0x7fe3380b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:20.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.429+0000 7fe33e7f0640 1 --2- 172.21.15.47:0/2959488211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3380b06d0 0x7fe3380b0b60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe3380d1530 tx=0x7fe33002be40 comp rx=0 tx=0).stop 2024-03-23T09:45:20.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.429+0000 7fe33e7f0640 1 -- 172.21.15.47:0/2959488211 shutdown_connections 2024-03-23T09:45:20.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.429+0000 7fe33e7f0640 1 --2- 172.21.15.47:0/2959488211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3380b06d0 0x7fe3380b0b60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.429+0000 7fe33e7f0640 1 --2- 172.21.15.47:0/2959488211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3380cad40 0x7fe3380cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.429+0000 7fe33e7f0640 1 --2- 172.21.15.47:0/2959488211 >> v2:172.21.15.175:3300/0 conn(0x7fe33815cbe0 0x7fe3380ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.429+0000 7fe33e7f0640 1 -- 172.21.15.47:0/2959488211 >> 172.21.15.47:0/2959488211 conn(0x7fe3380d06f0 msgr2=0x7fe3380d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:20.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.429+0000 7fe33e7f0640 1 -- 172.21.15.47:0/2959488211 shutdown_connections 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.429+0000 7fe33e7f0640 1 -- 172.21.15.47:0/2959488211 wait complete. 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.430+0000 7fe33e7f0640 1 Processor -- start 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.431+0000 7fe33e7f0640 1 -- start start 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.431+0000 7fe33e7f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3380b06d0 0x7fe33815d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.431+0000 7fe33e7f0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3380cad40 0x7fe33815dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.431+0000 7fe33e7f0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe33815cbe0 0x7fe3380c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.431+0000 7fe33e7f0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33815d890 con 0x7fe3380b06d0 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.431+0000 7fe33e7f0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3380ae4a0 con 0x7fe3380cad40 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.431+0000 7fe33e7f0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3380ae780 con 0x7fe33815cbe0 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.431+0000 7fe337fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3380b06d0 0x7fe33815d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.431+0000 7fe337fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3380b06d0 0x7fe33815d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47896/0 (socket says 172.21.15.47:47896) 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.431+0000 7fe337fff640 1 -- 172.21.15.47:0/4114192604 learned_addr learned my addr 172.21.15.47:0/4114192604 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.432+0000 7fe3377fe640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3380cad40 0x7fe33815dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.432+0000 7fe33cd8d640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe33815cbe0 0x7fe3380c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.432+0000 7fe337fff640 1 -- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe33815cbe0 msgr2=0x7fe3380c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.432+0000 7fe337fff640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe33815cbe0 0x7fe3380c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.432+0000 7fe337fff640 1 -- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3380cad40 msgr2=0x7fe33815dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.432+0000 7fe337fff640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3380cad40 0x7fe33815dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.432+0000 7fe337fff640 1 -- 172.21.15.47:0/4114192604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe330009b30 con 0x7fe3380b06d0 2024-03-23T09:45:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.432+0000 7fe337fff640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3380b06d0 0x7fe33815d1d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe32800d5f0 tx=0x7fe32800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.432+0000 7fe334ff9640 1 -- 172.21.15.47:0/4114192604 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3280125f0 con 0x7fe3380b06d0 2024-03-23T09:45:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.432+0000 7fe334ff9640 1 -- 172.21.15.47:0/4114192604 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe328012750 con 0x7fe3380b06d0 2024-03-23T09:45:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.432+0000 7fe334ff9640 1 -- 172.21.15.47:0/4114192604 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328012b00 con 0x7fe3380b06d0 2024-03-23T09:45:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.432+0000 7fe33e7f0640 1 -- 172.21.15.47:0/4114192604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3380aea60 con 0x7fe3380b06d0 2024-03-23T09:45:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.432+0000 7fe33e7f0640 1 -- 172.21.15.47:0/4114192604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3380b32e0 con 0x7fe3380b06d0 2024-03-23T09:45:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.433+0000 7fe33e7f0640 1 -- 172.21.15.47:0/4114192604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3380b34a0 con 0x7fe3380b06d0 2024-03-23T09:45:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.434+0000 7fe334ff9640 1 -- 172.21.15.47:0/4114192604 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe328012c60 con 0x7fe3380b06d0 2024-03-23T09:45:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.434+0000 7fe334ff9640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe32406ac00 0x7fe32406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.434+0000 7fe334ff9640 1 -- 172.21.15.47:0/4114192604 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe328089730 con 0x7fe3380b06d0 2024-03-23T09:45:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.434+0000 7fe3377fe640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe32406ac00 0x7fe32406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.435+0000 7fe3377fe640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe32406ac00 0x7fe32406d0c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe3380c7040 tx=0x7fe32c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.438+0000 7fe334ff9640 1 -- 172.21.15.47:0/4114192604 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe3280549a0 con 0x7fe3380b06d0 2024-03-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.655+0000 7fe33e7f0640 1 -- 172.21.15.47:0/4114192604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe338066be0 con 0x7fe3380b06d0 2024-03-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.656+0000 7fe334ff9640 1 -- 172.21.15.47:0/4114192604 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4736 (secure 0 0 0) 0x7fe328057fc0 con 0x7fe3380b06d0 2024-03-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3862818105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3862818105},{"type":"v1","addr":"172.21.15.175:6836","nonce":3862818105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3615592850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3615592850},{"type":"v1","addr":"172.21.15.175:6837","nonce":3615592850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/832829684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":832829684},{"type":"v1","addr":"172.21.15.47:6834","nonce":832829684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:45:18.296714+0000","modified":"2024-03-23T09:45:20.302269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6744},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6744":{"gid":6744,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/372771687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":372771687},{"type":"v1","addr":"172.21.15.47:6835","nonce":372771687}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.658+0000 7fe33e7f0640 1 -- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe32406ac00 msgr2=0x7fe32406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.658+0000 7fe33e7f0640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe32406ac00 0x7fe32406d0c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe3380c7040 tx=0x7fe32c008040 comp rx=0 tx=0).stop 2024-03-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.658+0000 7fe33e7f0640 1 -- 172.21.15.47:0/4114192604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3380b06d0 msgr2=0x7fe33815d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.658+0000 7fe33e7f0640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3380b06d0 0x7fe33815d1d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe32800d5f0 tx=0x7fe32800dab0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.658+0000 7fe33e7f0640 1 -- 172.21.15.47:0/4114192604 shutdown_connections 2024-03-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.658+0000 7fe33e7f0640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe32406ac00 0x7fe32406d0c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.658+0000 7fe33e7f0640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe33815cbe0 0x7fe3380c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.658+0000 7fe33e7f0640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3380cad40 0x7fe33815dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.658+0000 7fe33e7f0640 1 --2- 172.21.15.47:0/4114192604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3380b06d0 0x7fe33815d1d0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.658+0000 7fe33e7f0640 1 -- 172.21.15.47:0/4114192604 >> 172.21.15.47:0/4114192604 conn(0x7fe3380d06f0 msgr2=0x7fe33806b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.658+0000 7fe33e7f0640 1 -- 172.21.15.47:0/4114192604 shutdown_connections 2024-03-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.658+0000 7fe33e7f0640 1 -- 172.21.15.47:0/4114192604 wait complete. 2024-03-23T09:45:20.670 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:45:20.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-03-23T09:45:20.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-03-23T09:45:20.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-03-23T09:45:20.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.770+0000 7f63aedd2640 1 Processor -- start 2024-03-23T09:45:20.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.770+0000 7f63aedd2640 1 -- start start 2024-03-23T09:45:20.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63aedd2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63a815cd00 0x7f63a80d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63aedd2640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f63a80cf920 0x7f63a80b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63aedd2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a815d0e0 con 0x7f63a80d1020 2024-03-23T09:45:20.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63aedd2640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a80d1540 con 0x7f63a80cf920 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63aedd2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80d1820 con 0x7f63a815cd00 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a7fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f63a80d1020 0x7f63a80cf210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47576/0 (socket says 172.21.15.47:47576) 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a7fff640 1 -- 172.21.15.47:0/3673048663 learned_addr learned my addr 172.21.15.47:0/3673048663 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63ad36f640 1 --2- 172.21.15.47:0/3673048663 >> v2:172.21.15.175:3300/0 conn(0x7f63a80cf920 0x7f63a80b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63acb6e640 1 --2- 172.21.15.47:0/3673048663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63a815cd00 0x7f63a80d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948709859 0 0) 0x7f63a815d0e0 con 0x7f63a80d1020 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6398003520 con 0x7f63a80d1020 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1651998077 0 0) 0x7f6398003520 con 0x7f63a80d1020 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63a815cd00 msgr2=0x7f63a80d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 --2- 172.21.15.47:0/3673048663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63a815cd00 0x7f63a80d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 >> v2:172.21.15.175:3300/0 conn(0x7f63a80cf920 msgr2=0x7f63a80b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 --2- 172.21.15.47:0/3673048663 >> v2:172.21.15.175:3300/0 conn(0x7f63a80cf920 0x7f63a80b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a80d1b00 con 0x7f63a80d1020 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f6394001670 con 0x7f63a80d1020 2024-03-23T09:45:20.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 >> v1:172.21.15.47:6789/0 conn(0x7f63a80d1020 legacy=0x7f63a80cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 --2- 172.21.15.47:0/3673048663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63a815cd00 0x7f6398003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 --2- 172.21.15.47:0/3673048663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6398003ef0 0x7f63980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 --2- 172.21.15.47:0/3673048663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6398004d00 0x7f6398005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80d1540 con 0x7f6398004d00 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80d1820 con 0x7f6398003ef0 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639800b9d0 con 0x7f63a815cd00 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.771+0000 7f63ad36f640 1 --2- 172.21.15.47:0/3673048663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6398004d00 0x7f6398005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.772+0000 7f63acb6e640 1 --2- 172.21.15.47:0/3673048663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63a815cd00 0x7f6398003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.772+0000 7f63a7fff640 1 --2- 172.21.15.47:0/3673048663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6398003ef0 0x7f63980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.772+0000 7f63ad36f640 1 -- 172.21.15.47:0/3673048663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63a815cd00 msgr2=0x7f6398003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.772+0000 7f63ad36f640 1 --2- 172.21.15.47:0/3673048663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63a815cd00 0x7f6398003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.772+0000 7f63ad36f640 1 -- 172.21.15.47:0/3673048663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6398003ef0 msgr2=0x7f63980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.772+0000 7f63ad36f640 1 --2- 172.21.15.47:0/3673048663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6398003ef0 0x7f63980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.772+0000 7f63ad36f640 1 -- 172.21.15.47:0/3673048663 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6398003520 con 0x7f6398004d00 2024-03-23T09:45:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.772+0000 7f63ad36f640 1 --2- 172.21.15.47:0/3673048663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6398004d00 0x7f6398005180 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009f90 tx=0x7f63980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa9bcc042aae987 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:20.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.773+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a002cbd0 con 0x7f6398004d00 2024-03-23T09:45:20.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.773+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a000f040 con 0x7f6398004d00 2024-03-23T09:45:20.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.773+0000 7f63a77fe640 1 -- 172.21.15.47:0/3673048663 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0034440 con 0x7f6398004d00 2024-03-23T09:45:20.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.773+0000 7f63aedd2640 1 -- 172.21.15.47:0/3673048663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6398004d00 msgr2=0x7f6398005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:20.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.773+0000 7f63aedd2640 1 --2- 172.21.15.47:0/3673048663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6398004d00 0x7f6398005180 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009f90 tx=0x7f63980057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.773+0000 7f63ad36f640 1 -- 172.21.15.47:0/3673048663 reap_dead start 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.773+0000 7f63aedd2640 1 -- 172.21.15.47:0/3673048663 shutdown_connections 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.773+0000 7f63aedd2640 1 -- 172.21.15.47:0/3673048663 >> 172.21.15.47:0/3673048663 conn(0x7f63a806b160 msgr2=0x7f63a806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.773+0000 7f63aedd2640 1 -- 172.21.15.47:0/3673048663 shutdown_connections 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.773+0000 7f63aedd2640 1 -- 172.21.15.47:0/3673048663 wait complete. 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.774+0000 7f63aedd2640 1 Processor -- start 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.774+0000 7f63aedd2640 1 -- start start 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.775+0000 7f63aedd2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6398004d00 0x7f63a814e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.775+0000 7f63aedd2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a80b5450 0x7f63a80b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.775+0000 7f63aedd2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63a80cb830 0x7f63a80cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.775+0000 7f63aedd2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80cc1f0 con 0x7f63a80cb830 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.775+0000 7f63aedd2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80c9e40 con 0x7f63a80b5450 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.775+0000 7f63aedd2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80b36a0 con 0x7f6398004d00 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.775+0000 7f63ad36f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63a80cb830 0x7f63a80cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.775+0000 7f63ad36f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63a80cb830 0x7f63a80cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47906/0 (socket says 172.21.15.47:47906) 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.775+0000 7f63ad36f640 1 -- 172.21.15.47:0/2383859736 learned_addr learned my addr 172.21.15.47:0/2383859736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.775+0000 7f63a7fff640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a80b5450 0x7f63a80b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.775+0000 7f63acb6e640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6398004d00 0x7f63a814e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.776+0000 7f63ad36f640 1 -- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6398004d00 msgr2=0x7f63a814e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.776+0000 7f63ad36f640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6398004d00 0x7f63a814e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.776+0000 7f63ad36f640 1 -- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a80b5450 msgr2=0x7f63a80b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.776+0000 7f63ad36f640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a80b5450 0x7f63a80b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.776+0000 7f63ad36f640 1 -- 172.21.15.47:0/2383859736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a80b3980 con 0x7f63a80cb830 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.776+0000 7f63ad36f640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63a80cb830 0x7f63a80cbcb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009f90 tx=0x7f63a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.776+0000 7f63a57fa640 1 -- 172.21.15.47:0/2383859736 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0034d50 con 0x7f63a80cb830 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.776+0000 7f63a57fa640 1 -- 172.21.15.47:0/2383859736 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a003e440 con 0x7f63a80cb830 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.776+0000 7f63aedd2640 1 -- 172.21.15.47:0/2383859736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a80c9ad0 con 0x7f63a80cb830 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.776+0000 7f63a57fa640 1 -- 172.21.15.47:0/2383859736 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a003e7f0 con 0x7f63a80cb830 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.776+0000 7f63aedd2640 1 -- 172.21.15.47:0/2383859736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a814e600 con 0x7f63a80cb830 2024-03-23T09:45:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.777+0000 7f63aedd2640 1 -- 172.21.15.47:0/2383859736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a8072360 con 0x7f63a80cb830 2024-03-23T09:45:20.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.777+0000 7f63a57fa640 1 -- 172.21.15.47:0/2383859736 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f63a003e950 con 0x7f63a80cb830 2024-03-23T09:45:20.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.778+0000 7f63a57fa640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f638c06ac30 0x7f638c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:20.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.778+0000 7f63a57fa640 1 -- 172.21.15.47:0/2383859736 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63a00aaef0 con 0x7f63a80cb830 2024-03-23T09:45:20.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.778+0000 7f63acb6e640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f638c06ac30 0x7f638c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:20.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.778+0000 7f63acb6e640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f638c06ac30 0x7f638c06d0f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f639c009b00 tx=0x7f639c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.782+0000 7f63a57fa640 1 -- 172.21.15.47:0/2383859736 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f63a0076160 con 0x7f63a80cb830 2024-03-23T09:45:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.962+0000 7f63aedd2640 1 -- 172.21.15.47:0/2383859736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63a8066be0 con 0x7f63a80cb830 2024-03-23T09:45:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.964+0000 7f63a57fa640 1 -- 172.21.15.47:0/2383859736 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16100 (secure 0 0 0) 0x7f63a0039090 con 0x7f63a80cb830 2024-03-23T09:45:20.965 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:20.965 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":70,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:45:19.298605+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:45:16.609941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-23T09:45:17.261503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:45:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.965+0000 7f63aedd2640 1 -- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f638c06ac30 msgr2=0x7f638c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.965+0000 7f63aedd2640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f638c06ac30 0x7f638c06d0f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f639c009b00 tx=0x7f639c008040 comp rx=0 tx=0).stop 2024-03-23T09:45:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.965+0000 7f63aedd2640 1 -- 172.21.15.47:0/2383859736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63a80cb830 msgr2=0x7f63a80cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.965+0000 7f63aedd2640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63a80cb830 0x7f63a80cbcb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009f90 tx=0x7f63a0009750 comp rx=0 tx=0).stop 2024-03-23T09:45:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.966+0000 7f63aedd2640 1 -- 172.21.15.47:0/2383859736 shutdown_connections 2024-03-23T09:45:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.966+0000 7f63aedd2640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f638c06ac30 0x7f638c06d0f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.966+0000 7f63aedd2640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63a80cb830 0x7f63a80cbcb0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.966+0000 7f63aedd2640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a80b5450 0x7f63a80b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.966+0000 7f63aedd2640 1 --2- 172.21.15.47:0/2383859736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6398004d00 0x7f63a814e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.966+0000 7f63aedd2640 1 -- 172.21.15.47:0/2383859736 >> 172.21.15.47:0/2383859736 conn(0x7f63a806b160 msgr2=0x7f63a815e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.966+0000 7f63aedd2640 1 -- 172.21.15.47:0/2383859736 shutdown_connections 2024-03-23T09:45:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:20.966+0000 7f63aedd2640 1 -- 172.21.15.47:0/2383859736 wait complete. 2024-03-23T09:45:20.978 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T09:45:20.978 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:45:21.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.078+0000 7f025ea69640 1 Processor -- start 2024-03-23T09:45:21.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.078+0000 7f025ea69640 1 -- start start 2024-03-23T09:45:21.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.079+0000 7f025ea69640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025815ca90 0x7f0258160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.079+0000 7f025ea69640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f02580cde10 0x7f02580d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.079+0000 7f025ea69640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02580d11e0 con 0x7f02580b8350 2024-03-23T09:45:21.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.079+0000 7f025ea69640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0258160f60 con 0x7f02580cde10 2024-03-23T09:45:21.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.079+0000 7f025ea69640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0258161240 con 0x7f025815ca90 2024-03-23T09:45:21.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.079+0000 7f025d006640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f02580cde10 0x7f02580d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.079+0000 7f0257fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f02580b8350 0x7f02580cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47586/0 (socket says 172.21.15.47:47586) 2024-03-23T09:45:21.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.079+0000 7f0257fff640 1 -- 172.21.15.47:0/3188808649 learned_addr learned my addr 172.21.15.47:0/3188808649 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:21.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.079+0000 7f025c805640 1 --2- 172.21.15.47:0/3188808649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025815ca90 0x7f0258160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.080+0000 7f025d006640 1 -- 172.21.15.47:0/3188808649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025815ca90 msgr2=0x7f0258160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:21.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.080+0000 7f025d006640 1 --2- 172.21.15.47:0/3188808649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025815ca90 0x7f0258160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.080+0000 7f025d006640 1 -- 172.21.15.47:0/3188808649 >> v1:172.21.15.47:6789/0 conn(0x7f02580b8350 legacy=0x7f02580cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.080+0000 7f025d006640 1 -- 172.21.15.47:0/3188808649 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0258161520 con 0x7f02580cde10 2024-03-23T09:45:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.080+0000 7f025d006640 1 --2- 172.21.15.47:0/3188808649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02580cde10 0x7f02580d1670 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f02480097a0 tx=0x7f024802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368fcedc5c6d5c81 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.080+0000 7f02577fe640 1 -- 172.21.15.47:0/3188808649 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024802e070 con 0x7f02580cde10 2024-03-23T09:45:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.080+0000 7f02577fe640 1 -- 172.21.15.47:0/3188808649 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02480315a0 con 0x7f02580cde10 2024-03-23T09:45:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.080+0000 7f02577fe640 1 -- 172.21.15.47:0/3188808649 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248031930 con 0x7f02580cde10 2024-03-23T09:45:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.081+0000 7f025ea69640 1 -- 172.21.15.47:0/3188808649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02580cde10 msgr2=0x7f02580d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.081+0000 7f025ea69640 1 --2- 172.21.15.47:0/3188808649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02580cde10 0x7f02580d1670 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f02480097a0 tx=0x7f024802b840 comp rx=0 tx=0).stop 2024-03-23T09:45:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.081+0000 7f025ea69640 1 -- 172.21.15.47:0/3188808649 shutdown_connections 2024-03-23T09:45:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.081+0000 7f025ea69640 1 --2- 172.21.15.47:0/3188808649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02580cde10 0x7f02580d1670 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.081+0000 7f025ea69640 1 --2- 172.21.15.47:0/3188808649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025815ca90 0x7f0258160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.081+0000 7f025ea69640 1 -- 172.21.15.47:0/3188808649 >> 172.21.15.47:0/3188808649 conn(0x7f02580659d0 msgr2=0x7f0258065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:21.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.081+0000 7f025ea69640 1 -- 172.21.15.47:0/3188808649 shutdown_connections 2024-03-23T09:45:21.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.081+0000 7f025ea69640 1 -- 172.21.15.47:0/3188808649 wait complete. 2024-03-23T09:45:21.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.082+0000 7f025ea69640 1 Processor -- start 2024-03-23T09:45:21.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.082+0000 7f025ea69640 1 -- start start 2024-03-23T09:45:21.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f025ea69640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02580b8350 0x7f0258150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f025ea69640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02580cde10 0x7f02580b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f025ea69640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025815ca90 0x7f0258159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f025ea69640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02580b3af0 con 0x7f02580cde10 2024-03-23T09:45:21.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f025ea69640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0258151050 con 0x7f02580b8350 2024-03-23T09:45:21.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f025ea69640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02580b2e70 con 0x7f025815ca90 2024-03-23T09:45:21.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f0257fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02580cde10 0x7f02580b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f0257fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02580cde10 0x7f02580b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47928/0 (socket says 172.21.15.47:47928) 2024-03-23T09:45:21.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f0257fff640 1 -- 172.21.15.47:0/289358337 learned_addr learned my addr 172.21.15.47:0/289358337 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:21.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f025d006640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025815ca90 0x7f0258159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f025c805640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02580b8350 0x7f0258150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f0257fff640 1 -- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025815ca90 msgr2=0x7f0258159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f0257fff640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025815ca90 0x7f0258159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f0257fff640 1 -- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02580b8350 msgr2=0x7f0258150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f0257fff640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02580b8350 0x7f0258150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.083+0000 7f0257fff640 1 -- 172.21.15.47:0/289358337 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0248009400 con 0x7f02580cde10 2024-03-23T09:45:21.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.084+0000 7f0257fff640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02580cde10 0x7f02580b2420 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c480 tx=0x7f024c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:21.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.084+0000 7f02557fa640 1 -- 172.21.15.47:0/289358337 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c005ce0 con 0x7f02580cde10 2024-03-23T09:45:21.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.084+0000 7f02557fa640 1 -- 172.21.15.47:0/289358337 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c005e40 con 0x7f02580cde10 2024-03-23T09:45:21.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.084+0000 7f025ea69640 1 -- 172.21.15.47:0/289358337 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02580b3150 con 0x7f02580cde10 2024-03-23T09:45:21.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.084+0000 7f02557fa640 1 -- 172.21.15.47:0/289358337 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c0072d0 con 0x7f02580cde10 2024-03-23T09:45:21.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.084+0000 7f025ea69640 1 -- 172.21.15.47:0/289358337 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02580b3710 con 0x7f02580cde10 2024-03-23T09:45:21.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.085+0000 7f025ea69640 1 -- 172.21.15.47:0/289358337 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0258072360 con 0x7f02580cde10 2024-03-23T09:45:21.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.085+0000 7f02557fa640 1 -- 172.21.15.47:0/289358337 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f024c0074f0 con 0x7f02580cde10 2024-03-23T09:45:21.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.086+0000 7f02557fa640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f024406ac00 0x7f024406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.086+0000 7f02557fa640 1 -- 172.21.15.47:0/289358337 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f024c088890 con 0x7f02580cde10 2024-03-23T09:45:21.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.086+0000 7f025c805640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f024406ac00 0x7f024406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.086+0000 7f025c805640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f024406ac00 0x7f024406d0c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0250009a60 tx=0x7f0250003990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:21.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.090+0000 7f02557fa640 1 -- 172.21.15.47:0/289358337 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f024c053a80 con 0x7f02580cde10 2024-03-23T09:45:21.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.269+0000 7f025ea69640 1 -- 172.21.15.47:0/289358337 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0258160c50 con 0x7f024406ac00 2024-03-23T09:45:21.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.277+0000 7f02557fa640 1 -- 172.21.15.47:0/289358337 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278669 (secure 0 0 0) 0x7f0250004780 con 0x7f024406ac00 2024-03-23T09:45:21.281 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:21.283 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:45:21.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.280+0000 7f025ea69640 1 -- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f024406ac00 msgr2=0x7f024406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.280+0000 7f025ea69640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f024406ac00 0x7f024406d0c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0250009a60 tx=0x7f0250003990 comp rx=0 tx=0).stop 2024-03-23T09:45:21.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.280+0000 7f025ea69640 1 -- 172.21.15.47:0/289358337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02580cde10 msgr2=0x7f02580b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.280+0000 7f025ea69640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02580cde10 0x7f02580b2420 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c480 tx=0x7f024c00c940 comp rx=0 tx=0).stop 2024-03-23T09:45:21.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.280+0000 7f025ea69640 1 -- 172.21.15.47:0/289358337 shutdown_connections 2024-03-23T09:45:21.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.280+0000 7f025ea69640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f024406ac00 0x7f024406d0c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.280+0000 7f025ea69640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025815ca90 0x7f0258159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.280+0000 7f025ea69640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02580cde10 0x7f02580b2420 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.280+0000 7f025ea69640 1 --2- 172.21.15.47:0/289358337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02580b8350 0x7f0258150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.280+0000 7f025ea69640 1 -- 172.21.15.47:0/289358337 >> 172.21.15.47:0/289358337 conn(0x7f02580659d0 msgr2=0x7f025815e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:21.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.280+0000 7f025ea69640 1 -- 172.21.15.47:0/289358337 shutdown_connections 2024-03-23T09:45:21.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.280+0000 7f025ea69640 1 -- 172.21.15.47:0/289358337 wait complete. 2024-03-23T09:45:21.292 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-03-23T09:45:20.400686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244136,"kb_used_data":7472,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754730584,"statfs":{"total":773094113280,"available":772844118016,"internally_reserved":0,"allocated":7651328,"data_stored":3848982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":242208312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000971"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926611+0000","last_change":"2024-03-23T09:45:16.926611+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926611+0000","last_undegraded":"2024-03-23T09:45:16.926611+0000","last_fullsized":"2024-03-23T09:45:16.926611+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:07.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930742+0000","last_change":"2024-03-23T09:45:16.930742+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930742+0000","last_undegraded":"2024-03-23T09:45:16.930742+0000","last_fullsized":"2024-03-23T09:45:16.930742+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:10.301294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926660+0000","last_change":"2024-03-23T09:45:16.926660+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926660+0000","last_undegraded":"2024-03-23T09:45:16.926660+0000","last_fullsized":"2024-03-23T09:45:16.926660+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:18.127827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929237+0000","last_change":"2024-03-23T09:45:16.929237+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929237+0000","last_undegraded":"2024-03-23T09:45:16.929237+0000","last_fullsized":"2024-03-23T09:45:16.929237+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:28:14.722915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929116+0000","last_change":"2024-03-23T09:45:16.929116+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929116+0000","last_undegraded":"2024-03-23T09:45:16.929116+0000","last_fullsized":"2024-03-23T09:45:16.929116+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:18:20.415053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930935+0000","last_change":"2024-03-23T09:45:16.930935+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930935+0000","last_undegraded":"2024-03-23T09:45:16.930935+0000","last_fullsized":"2024-03-23T09:45:16.930935+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:53:08.530702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926321+0000","last_change":"2024-03-23T09:45:16.926321+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926321+0000","last_undegraded":"2024-03-23T09:45:16.926321+0000","last_fullsized":"2024-03-23T09:45:16.926321+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:44.872119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.927934+0000","last_change":"2024-03-23T09:45:16.927934+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.927934+0000","last_undegraded":"2024-03-23T09:45:16.927934+0000","last_fullsized":"2024-03-23T09:45:16.927934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:00:47.619314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926649+0000","last_change":"2024-03-23T09:45:16.926649+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926649+0000","last_undegraded":"2024-03-23T09:45:16.926649+0000","last_fullsized":"2024-03-23T09:45:16.926649+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:24.280980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929151+0000","last_change":"2024-03-23T09:45:16.929151+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929151+0000","last_undegraded":"2024-03-23T09:45:16.929151+0000","last_fullsized":"2024-03-23T09:45:16.929151+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:24.280980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.928831+0000","last_change":"2024-03-23T09:45:16.928831+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.928831+0000","last_undegraded":"2024-03-23T09:45:16.928831+0000","last_fullsized":"2024-03-23T09:45:16.928831+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:30:06.113245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926342+0000","last_change":"2024-03-23T09:45:16.926342+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926342+0000","last_undegraded":"2024-03-23T09:45:16.926342+0000","last_fullsized":"2024-03-23T09:45:16.926342+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:14:57.219705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926273+0000","last_change":"2024-03-23T09:45:16.926273+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926273+0000","last_undegraded":"2024-03-23T09:45:16.926273+0000","last_fullsized":"2024-03-23T09:45:16.926273+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:18:57.489494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930470+0000","last_change":"2024-03-23T09:45:16.930470+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930470+0000","last_undegraded":"2024-03-23T09:45:16.930470+0000","last_fullsized":"2024-03-23T09:45:16.930470+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:30:11.821980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.933740+0000","last_change":"2024-03-23T09:45:16.933740+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.933740+0000","last_undegraded":"2024-03-23T09:45:16.933740+0000","last_fullsized":"2024-03-23T09:45:16.933740+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:06:34.170725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926591+0000","last_change":"2024-03-23T09:45:16.926591+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926591+0000","last_undegraded":"2024-03-23T09:45:16.926591+0000","last_fullsized":"2024-03-23T09:45:16.926591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:04.521715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930328+0000","last_change":"2024-03-23T09:45:16.930328+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930328+0000","last_undegraded":"2024-03-23T09:45:16.930328+0000","last_fullsized":"2024-03-23T09:45:16.930328+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:18:57.489494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926592+0000","last_change":"2024-03-23T09:45:16.926592+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926592+0000","last_undegraded":"2024-03-23T09:45:16.926592+0000","last_fullsized":"2024-03-23T09:45:16.926592+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:00:16.438765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930133+0000","last_change":"2024-03-23T09:45:16.930133+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930133+0000","last_undegraded":"2024-03-23T09:45:16.930133+0000","last_fullsized":"2024-03-23T09:45:16.930133+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:14:57.219705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.927565+0000","last_change":"2024-03-23T09:45:16.927565+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.927565+0000","last_undegraded":"2024-03-23T09:45:16.927565+0000","last_fullsized":"2024-03-23T09:45:16.927565+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:44:44.243931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929144+0000","last_change":"2024-03-23T09:45:16.929144+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929144+0000","last_undegraded":"2024-03-23T09:45:16.929144+0000","last_fullsized":"2024-03-23T09:45:16.929144+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:38:12.502514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.933380+0000","last_change":"2024-03-23T09:45:16.933380+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.933380+0000","last_undegraded":"2024-03-23T09:45:16.933380+0000","last_fullsized":"2024-03-23T09:45:16.933380+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:38:17.881935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930448+0000","last_change":"2024-03-23T09:45:16.930448+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930448+0000","last_undegraded":"2024-03-23T09:45:16.930448+0000","last_fullsized":"2024-03-23T09:45:16.930448+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:07.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929343+0000","last_change":"2024-03-23T09:45:16.929343+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929343+0000","last_undegraded":"2024-03-23T09:45:16.929343+0000","last_fullsized":"2024-03-23T09:45:16.929343+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:03:17.222905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.933450+0000","last_change":"2024-03-23T09:45:16.933450+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.933450+0000","last_undegraded":"2024-03-23T09:45:16.933450+0000","last_fullsized":"2024-03-23T09:45:16.933450+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:41:11.998454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.927946+0000","last_change":"2024-03-23T09:45:16.927946+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.927946+0000","last_undegraded":"2024-03-23T09:45:16.927946+0000","last_fullsized":"2024-03-23T09:45:16.927946+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:42:34.167069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930417+0000","last_change":"2024-03-23T09:45:16.930417+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930417+0000","last_undegraded":"2024-03-23T09:45:16.930417+0000","last_fullsized":"2024-03-23T09:45:16.930417+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:40:34.327822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.927933+0000","last_change":"2024-03-23T09:45:16.927933+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.927933+0000","last_undegraded":"2024-03-23T09:45:16.927933+0000","last_fullsized":"2024-03-23T09:45:16.927933+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:53:03.362600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926273+0000","last_change":"2024-03-23T09:45:16.926273+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926273+0000","last_undegraded":"2024-03-23T09:45:16.926273+0000","last_fullsized":"2024-03-23T09:45:16.926273+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:11:42.583654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926761+0000","last_change":"2024-03-23T09:45:16.926761+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926761+0000","last_undegraded":"2024-03-23T09:45:16.926761+0000","last_fullsized":"2024-03-23T09:45:16.926761+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:53:08.530702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929338+0000","last_change":"2024-03-23T09:45:16.929338+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929338+0000","last_undegraded":"2024-03-23T09:45:16.929338+0000","last_fullsized":"2024-03-23T09:45:16.929338+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:11:42.583654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930323+0000","last_change":"2024-03-23T09:45:16.930323+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930323+0000","last_undegraded":"2024-03-23T09:45:16.930323+0000","last_fullsized":"2024-03-23T09:45:16.930323+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:04.521715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":212,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-03-23T09:45:14.973650+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:45:14.973650+0000","last_peered":"2024-03-23T09:45:14.973650+0000","last_clean":"2024-03-23T09:45:14.973650+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:45:14.973650+0000","last_undegraded":"2024-03-23T09:45:14.973650+0000","last_fullsized":"2024-03-23T09:45:14.973650+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":926962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53100000000000003,"15min":0.48999999999999999},"min":{"1min":0.35699999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72199999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54500000000000004,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.76100000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54000000000000004,"15min":0.51200000000000001},"min":{"1min":0.42199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.751,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.439,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78500000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.69699999999999995}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.44400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.496}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.48399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.84699999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82699999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68000000000000005}]},{"osd":4,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.497,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.622,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.33200000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65200000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.61899999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.38300000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56699999999999995,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":0.874,"15min":0.874},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.379,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.42399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":335975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.371,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.315},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.38900000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53000000000000003,"15min":0.503},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.71699999999999997,"5min":0.751,"15min":0.751},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68200000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.61399999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65600000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.41699999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.71499999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.44600000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.45600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.45300000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.70599999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57499999999999996}]},{"osd":4,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.48599999999999999,"15min":0.504},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.626,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.376},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.52700000000000002,"15min":0.54700000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.69799999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.46999999999999997}]},{"osd":5,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.50700000000000001,"15min":0.53000000000000003},"min":{"1min":0.36199999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54400000000000004,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.40200000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.625}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":336594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.51600000000000001,"15min":0.48199999999999998},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70799999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55800000000000005,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70699999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.48999999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52700000000000002,"15min":0.49199999999999999},"min":{"1min":0.41999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.504},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.65000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.44500000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.36599999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53800000000000003,"15min":0.504},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56699999999999995,"15min":0.53400000000000003},"min":{"1min":0.42699999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.69499999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.502}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45900000000000002,"15min":0.46999999999999997},"min":{"1min":0.29599999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67900000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.34799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71999999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.32900000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.498,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.627,"5min":0.749,"15min":0.749},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60299999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.34699999999999998}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.746,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56399999999999995,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":330777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.437,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.747,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.502,"15min":0.47399999999999998},"min":{"1min":0.36899999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.63100000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51700000000000002,"15min":0.504},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.73099999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.41599999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.53800000000000003},"min":{"1min":0.50700000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75800000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.503},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.79900000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66600000000000004}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68400000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.371},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.40500000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.71799999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38500000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46300000000000002,"15min":0.46899999999999997},"min":{"1min":0.32600000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67900000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.309},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.48899999999999999,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.66000000000000003,"5min":1.008,"15min":1.008},"last":0.40100000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.307,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60999999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.432},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.67200000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48599999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52500000000000002,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.371,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.68799999999999994,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":802816,"data_stored":330777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.34100000000000003,"15min":0.316},"min":{"1min":0.28100000000000003,"5min":0.183,"15min":0.183},"max":{"1min":0.627,"5min":1.45,"15min":1.45},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.315},"min":{"1min":0.23100000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.51900000000000002,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.34100000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33400000000000002,"15min":0.312},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54300000000000004,"5min":1.323,"15min":1.323},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.36799999999999999,"15min":0.34899999999999998},"min":{"1min":0.315,"5min":0.218,"15min":0.218},"max":{"1min":0.67800000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.46500000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.38100000000000001},"min":{"1min":0.309,"5min":0.251,"15min":0.251},"max":{"1min":0.61599999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.42999999999999999,"15min":0.41199999999999998},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70899999999999996,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.749,"5min":1.337,"15min":1.337},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.54500000000000004},"min":{"1min":0.437,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":2.758,"15min":2.758},"last":0.74399999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77500000000000002,"5min":1.276,"15min":1.276},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86499999999999999,"5min":1.895,"15min":1.895},"last":0.76300000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.62,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70699999999999996,"5min":5.4539999999999997,"15min":5.4539999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83399999999999996,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.64600000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.60899999999999999},"min":{"1min":0.45500000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89300000000000002,"5min":3.7400000000000002,"15min":3.7400000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65300000000000002,"15min":0.61899999999999999},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":802816,"data_stored":330777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.249,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.314},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36799999999999999,"15min":0.378},"min":{"1min":0.26200000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52200000000000002,"5min":0.872,"15min":0.872},"last":0.27000000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.28499999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.629,"15min":0.629},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.39000000000000001},"min":{"1min":0.24299999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58699999999999997,"5min":0.746,"15min":0.746},"last":0.32900000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.34100000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61399999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.27800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.55000000000000004,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.38400000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.44900000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84699999999999998,"5min":1.216,"15min":1.216},"last":0.436},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85499999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.65900000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.626,"15min":0.63300000000000001},"min":{"1min":0.51200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.754},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83999999999999997,"5min":1.101,"15min":1.101},"last":0.58499999999999996}]},{"osd":6,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.53100000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89100000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63900000000000001,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89500000000000002,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.78900000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.56899999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.872,"5min":1.155,"15min":1.155},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64400000000000002,"15min":0.65400000000000003},"min":{"1min":0.46999999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82099999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":802816,"data_stored":330777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34799999999999998,"15min":0.33700000000000002},"min":{"1min":0.247,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.5,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42599999999999999,"15min":0.47799999999999998},"min":{"1min":0.23400000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.71999999999999997,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.36399999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.34499999999999997},"min":{"1min":0.251,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39000000000000001,"15min":0.375},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57399999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.29099999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42199999999999999,"15min":0.40999999999999998},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63700000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.45100000000000001,"15min":0.50700000000000001},"min":{"1min":0.29199999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.61799999999999999,"5min":2.75,"15min":2.75},"last":0.311}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.48999999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.70399999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94199999999999995,"5min":2.778,"15min":2.778},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89000000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.64100000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66500000000000004,"15min":0.70899999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66900000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89200000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.75900000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.59399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1404928,"data_stored":926343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33400000000000002,"15min":0.32300000000000001},"min":{"1min":0.21299999999999999,"5min":0.184,"15min":0.184},"max":{"1min":0.51300000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.35099999999999998},"min":{"1min":0.23999999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.35699999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.38100000000000001,"15min":0.36699999999999999},"min":{"1min":0.28899999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55400000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.29999999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55200000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.31900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.36799999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.48999999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.622,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.53200000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56100000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.44600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70799999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.504,"5min":0.379,"15min":0.379},"max":{"1min":0.875,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.501}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82399999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.64400000000000002},"min":{"1min":0.54200000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89400000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.64300000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.51000000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85799999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.626},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.501,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80700000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:45:21.296 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:45:21.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.431+0000 7f0ddac1c640 1 Processor -- start 2024-03-23T09:45:21.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.431+0000 7f0ddac1c640 1 -- start start 2024-03-23T09:45:21.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.432+0000 7f0ddac1c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0dd413d810 0x7f0dd40cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.432+0000 7f0ddac1c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dd4140920 0x7f0dd4070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.432+0000 7f0ddac1c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd4140db0 con 0x7f0dd40701f0 2024-03-23T09:45:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.432+0000 7f0ddac1c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd413dbf0 con 0x7f0dd413d810 2024-03-23T09:45:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.432+0000 7f0ddac1c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40a7670 con 0x7f0dd4140920 2024-03-23T09:45:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.432+0000 7f0dd89b8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0dd413d810 0x7f0dd40cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.432+0000 7f0dd89b8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0dd413d810 0x7f0dd40cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47934/0 (socket says 172.21.15.47:47934) 2024-03-23T09:45:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.432+0000 7f0dd89b8640 1 -- 172.21.15.47:0/1958566465 learned_addr learned my addr 172.21.15.47:0/1958566465 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.432+0000 7f0dd91b9640 1 --2- 172.21.15.47:0/1958566465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dd4140920 0x7f0dd4070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.432+0000 7f0dd89b8640 1 -- 172.21.15.47:0/1958566465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dd4140920 msgr2=0x7f0dd4070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.432+0000 7f0dd89b8640 1 --2- 172.21.15.47:0/1958566465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dd4140920 0x7f0dd4070630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.432+0000 7f0dd89b8640 1 -- 172.21.15.47:0/1958566465 >> v1:172.21.15.175:6789/0 conn(0x7f0dd40701f0 legacy=0x7f0dd4140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.433+0000 7f0dd89b8640 1 -- 172.21.15.47:0/1958566465 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd413ea70 con 0x7f0dd413d810 2024-03-23T09:45:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.433+0000 7f0dd89b8640 1 --2- 172.21.15.47:0/1958566465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dd413d810 0x7f0dd40cc4b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0dd406a9e0 tx=0x7f0dcc02bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68fb712b03d217b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.433+0000 7f0dd37fe640 1 -- 172.21.15.47:0/1958566465 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc02cac0 con 0x7f0dd413d810 2024-03-23T09:45:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.433+0000 7f0dd37fe640 1 -- 172.21.15.47:0/1958566465 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc02cc20 con 0x7f0dd413d810 2024-03-23T09:45:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.433+0000 7f0dd37fe640 1 -- 172.21.15.47:0/1958566465 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0343f0 con 0x7f0dd413d810 2024-03-23T09:45:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.434+0000 7f0ddac1c640 1 -- 172.21.15.47:0/1958566465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dd413d810 msgr2=0x7f0dd40cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.434+0000 7f0ddac1c640 1 --2- 172.21.15.47:0/1958566465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dd413d810 0x7f0dd40cc4b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0dd406a9e0 tx=0x7f0dcc02bf20 comp rx=0 tx=0).stop 2024-03-23T09:45:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.434+0000 7f0ddac1c640 1 -- 172.21.15.47:0/1958566465 shutdown_connections 2024-03-23T09:45:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.434+0000 7f0ddac1c640 1 --2- 172.21.15.47:0/1958566465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dd4140920 0x7f0dd4070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.434+0000 7f0ddac1c640 1 --2- 172.21.15.47:0/1958566465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dd413d810 0x7f0dd40cc4b0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.434+0000 7f0ddac1c640 1 -- 172.21.15.47:0/1958566465 >> 172.21.15.47:0/1958566465 conn(0x7f0dd406a310 msgr2=0x7f0dd40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.434+0000 7f0ddac1c640 1 -- 172.21.15.47:0/1958566465 shutdown_connections 2024-03-23T09:45:21.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.434+0000 7f0ddac1c640 1 -- 172.21.15.47:0/1958566465 wait complete. 2024-03-23T09:45:21.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.435+0000 7f0ddac1c640 1 Processor -- start 2024-03-23T09:45:21.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.435+0000 7f0ddac1c640 1 -- start start 2024-03-23T09:45:21.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0ddac1c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dd40701f0 0x7f0dd4136b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0ddac1c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dd413d810 0x7f0dd41363f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0ddac1c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dd4140920 0x7f0dd40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0ddac1c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd406e350 con 0x7f0dd40701f0 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0ddac1c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40ad4f0 con 0x7f0dd4140920 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0ddac1c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40ac730 con 0x7f0dd413d810 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0dd89b8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dd40701f0 0x7f0dd4136b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0dd89b8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dd40701f0 0x7f0dd4136b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47940/0 (socket says 172.21.15.47:47940) 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0dd89b8640 1 -- 172.21.15.47:0/581235885 learned_addr learned my addr 172.21.15.47:0/581235885 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0dd89b8640 1 -- 172.21.15.47:0/581235885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dd413d810 msgr2=0x7f0dd41363f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0dd89b8640 1 --2- 172.21.15.47:0/581235885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dd413d810 0x7f0dd41363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0dd89b8640 1 -- 172.21.15.47:0/581235885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dd4140920 msgr2=0x7f0dd40b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0dd89b8640 1 --2- 172.21.15.47:0/581235885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dd4140920 0x7f0dd40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.436+0000 7f0dd89b8640 1 -- 172.21.15.47:0/581235885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc009b30 con 0x7f0dd40701f0 2024-03-23T09:45:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.437+0000 7f0dd89b8640 1 --2- 172.21.15.47:0/581235885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dd40701f0 0x7f0dd4136b90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc000f80 tx=0x7f0dcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:21.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.437+0000 7f0dd17fa640 1 -- 172.21.15.47:0/581235885 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc03e450 con 0x7f0dd40701f0 2024-03-23T09:45:21.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.437+0000 7f0dd17fa640 1 -- 172.21.15.47:0/581235885 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc03e5f0 con 0x7f0dd40701f0 2024-03-23T09:45:21.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.437+0000 7f0ddac1c640 1 -- 172.21.15.47:0/581235885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd40aca10 con 0x7f0dd40701f0 2024-03-23T09:45:21.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.437+0000 7f0dd17fa640 1 -- 172.21.15.47:0/581235885 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc03e9a0 con 0x7f0dd40701f0 2024-03-23T09:45:21.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.437+0000 7f0ddac1c640 1 -- 172.21.15.47:0/581235885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd40acfd0 con 0x7f0dd40701f0 2024-03-23T09:45:21.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.438+0000 7f0ddac1c640 1 -- 172.21.15.47:0/581235885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd4073480 con 0x7f0dd40701f0 2024-03-23T09:45:21.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.438+0000 7f0dd17fa640 1 -- 172.21.15.47:0/581235885 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0dcc03eb00 con 0x7f0dd40701f0 2024-03-23T09:45:21.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.439+0000 7f0dd17fa640 1 --2- 172.21.15.47:0/581235885 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0dc006ac00 0x7f0dc006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.439+0000 7f0dd17fa640 1 -- 172.21.15.47:0/581235885 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0dcc0ac220 con 0x7f0dd40701f0 2024-03-23T09:45:21.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.439+0000 7f0dd3fff640 1 --2- 172.21.15.47:0/581235885 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0dc006ac00 0x7f0dc006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.439+0000 7f0dd3fff640 1 --2- 172.21.15.47:0/581235885 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0dc006ac00 0x7f0dc006d0c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40ad840 tx=0x7f0dc80079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:21.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.443+0000 7f0dd17fa640 1 -- 172.21.15.47:0/581235885 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0dcc0775b0 con 0x7f0dd40701f0 2024-03-23T09:45:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.619+0000 7f0ddac1c640 1 -- 172.21.15.47:0/581235885 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0dd413e140 con 0x7f0dc006ac00 2024-03-23T09:45:21.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.627+0000 7f0dd17fa640 1 -- 172.21.15.47:0/581235885 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278669 (secure 0 0 0) 0x7f0dc8014400 con 0x7f0dc006ac00 2024-03-23T09:45:21.630 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:45:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.629+0000 7f0ddac1c640 1 -- 172.21.15.47:0/581235885 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0dc006ac00 msgr2=0x7f0dc006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.629+0000 7f0ddac1c640 1 --2- 172.21.15.47:0/581235885 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0dc006ac00 0x7f0dc006d0c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40ad840 tx=0x7f0dc80079b0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.629+0000 7f0ddac1c640 1 -- 172.21.15.47:0/581235885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dd40701f0 msgr2=0x7f0dd4136b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.629+0000 7f0ddac1c640 1 --2- 172.21.15.47:0/581235885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dd40701f0 0x7f0dd4136b90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc000f80 tx=0x7f0dcc004a40 comp rx=0 tx=0).stop 2024-03-23T09:45:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.629+0000 7f0ddac1c640 1 -- 172.21.15.47:0/581235885 shutdown_connections 2024-03-23T09:45:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.629+0000 7f0ddac1c640 1 --2- 172.21.15.47:0/581235885 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0dc006ac00 0x7f0dc006d0c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.629+0000 7f0ddac1c640 1 --2- 172.21.15.47:0/581235885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dd4140920 0x7f0dd40b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.629+0000 7f0ddac1c640 1 --2- 172.21.15.47:0/581235885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dd413d810 0x7f0dd41363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.629+0000 7f0ddac1c640 1 --2- 172.21.15.47:0/581235885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dd40701f0 0x7f0dd4136b90 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.629+0000 7f0ddac1c640 1 -- 172.21.15.47:0/581235885 >> 172.21.15.47:0/581235885 conn(0x7f0dd406a310 msgr2=0x7f0dd40af660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.629+0000 7f0ddac1c640 1 -- 172.21.15.47:0/581235885 shutdown_connections 2024-03-23T09:45:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.629+0000 7f0ddac1c640 1 -- 172.21.15.47:0/581235885 wait complete. 2024-03-23T09:45:21.641 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-03-23T09:45:20.400686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244136,"kb_used_data":7472,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754730584,"statfs":{"total":773094113280,"available":772844118016,"internally_reserved":0,"allocated":7651328,"data_stored":3848982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":242208312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000971"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926611+0000","last_change":"2024-03-23T09:45:16.926611+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926611+0000","last_undegraded":"2024-03-23T09:45:16.926611+0000","last_fullsized":"2024-03-23T09:45:16.926611+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:07.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930742+0000","last_change":"2024-03-23T09:45:16.930742+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930742+0000","last_undegraded":"2024-03-23T09:45:16.930742+0000","last_fullsized":"2024-03-23T09:45:16.930742+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:10.301294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926660+0000","last_change":"2024-03-23T09:45:16.926660+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926660+0000","last_undegraded":"2024-03-23T09:45:16.926660+0000","last_fullsized":"2024-03-23T09:45:16.926660+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:18.127827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929237+0000","last_change":"2024-03-23T09:45:16.929237+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929237+0000","last_undegraded":"2024-03-23T09:45:16.929237+0000","last_fullsized":"2024-03-23T09:45:16.929237+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:28:14.722915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929116+0000","last_change":"2024-03-23T09:45:16.929116+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929116+0000","last_undegraded":"2024-03-23T09:45:16.929116+0000","last_fullsized":"2024-03-23T09:45:16.929116+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:18:20.415053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930935+0000","last_change":"2024-03-23T09:45:16.930935+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930935+0000","last_undegraded":"2024-03-23T09:45:16.930935+0000","last_fullsized":"2024-03-23T09:45:16.930935+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:53:08.530702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926321+0000","last_change":"2024-03-23T09:45:16.926321+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926321+0000","last_undegraded":"2024-03-23T09:45:16.926321+0000","last_fullsized":"2024-03-23T09:45:16.926321+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:44.872119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.927934+0000","last_change":"2024-03-23T09:45:16.927934+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.927934+0000","last_undegraded":"2024-03-23T09:45:16.927934+0000","last_fullsized":"2024-03-23T09:45:16.927934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:00:47.619314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926649+0000","last_change":"2024-03-23T09:45:16.926649+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926649+0000","last_undegraded":"2024-03-23T09:45:16.926649+0000","last_fullsized":"2024-03-23T09:45:16.926649+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:24.280980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929151+0000","last_change":"2024-03-23T09:45:16.929151+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929151+0000","last_undegraded":"2024-03-23T09:45:16.929151+0000","last_fullsized":"2024-03-23T09:45:16.929151+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:24.280980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.928831+0000","last_change":"2024-03-23T09:45:16.928831+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.928831+0000","last_undegraded":"2024-03-23T09:45:16.928831+0000","last_fullsized":"2024-03-23T09:45:16.928831+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:30:06.113245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926342+0000","last_change":"2024-03-23T09:45:16.926342+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926342+0000","last_undegraded":"2024-03-23T09:45:16.926342+0000","last_fullsized":"2024-03-23T09:45:16.926342+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:14:57.219705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926273+0000","last_change":"2024-03-23T09:45:16.926273+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926273+0000","last_undegraded":"2024-03-23T09:45:16.926273+0000","last_fullsized":"2024-03-23T09:45:16.926273+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:18:57.489494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930470+0000","last_change":"2024-03-23T09:45:16.930470+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930470+0000","last_undegraded":"2024-03-23T09:45:16.930470+0000","last_fullsized":"2024-03-23T09:45:16.930470+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:30:11.821980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.933740+0000","last_change":"2024-03-23T09:45:16.933740+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.933740+0000","last_undegraded":"2024-03-23T09:45:16.933740+0000","last_fullsized":"2024-03-23T09:45:16.933740+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:06:34.170725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926591+0000","last_change":"2024-03-23T09:45:16.926591+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926591+0000","last_undegraded":"2024-03-23T09:45:16.926591+0000","last_fullsized":"2024-03-23T09:45:16.926591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:04.521715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930328+0000","last_change":"2024-03-23T09:45:16.930328+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930328+0000","last_undegraded":"2024-03-23T09:45:16.930328+0000","last_fullsized":"2024-03-23T09:45:16.930328+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:18:57.489494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926592+0000","last_change":"2024-03-23T09:45:16.926592+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926592+0000","last_undegraded":"2024-03-23T09:45:16.926592+0000","last_fullsized":"2024-03-23T09:45:16.926592+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:00:16.438765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930133+0000","last_change":"2024-03-23T09:45:16.930133+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930133+0000","last_undegraded":"2024-03-23T09:45:16.930133+0000","last_fullsized":"2024-03-23T09:45:16.930133+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:14:57.219705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.927565+0000","last_change":"2024-03-23T09:45:16.927565+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.927565+0000","last_undegraded":"2024-03-23T09:45:16.927565+0000","last_fullsized":"2024-03-23T09:45:16.927565+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:44:44.243931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929144+0000","last_change":"2024-03-23T09:45:16.929144+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929144+0000","last_undegraded":"2024-03-23T09:45:16.929144+0000","last_fullsized":"2024-03-23T09:45:16.929144+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:38:12.502514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.933380+0000","last_change":"2024-03-23T09:45:16.933380+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.933380+0000","last_undegraded":"2024-03-23T09:45:16.933380+0000","last_fullsized":"2024-03-23T09:45:16.933380+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:38:17.881935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930448+0000","last_change":"2024-03-23T09:45:16.930448+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930448+0000","last_undegraded":"2024-03-23T09:45:16.930448+0000","last_fullsized":"2024-03-23T09:45:16.930448+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:07.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929343+0000","last_change":"2024-03-23T09:45:16.929343+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929343+0000","last_undegraded":"2024-03-23T09:45:16.929343+0000","last_fullsized":"2024-03-23T09:45:16.929343+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:03:17.222905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.933450+0000","last_change":"2024-03-23T09:45:16.933450+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.933450+0000","last_undegraded":"2024-03-23T09:45:16.933450+0000","last_fullsized":"2024-03-23T09:45:16.933450+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:41:11.998454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.927946+0000","last_change":"2024-03-23T09:45:16.927946+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.927946+0000","last_undegraded":"2024-03-23T09:45:16.927946+0000","last_fullsized":"2024-03-23T09:45:16.927946+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:42:34.167069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930417+0000","last_change":"2024-03-23T09:45:16.930417+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930417+0000","last_undegraded":"2024-03-23T09:45:16.930417+0000","last_fullsized":"2024-03-23T09:45:16.930417+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:40:34.327822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.927933+0000","last_change":"2024-03-23T09:45:16.927933+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.927933+0000","last_undegraded":"2024-03-23T09:45:16.927933+0000","last_fullsized":"2024-03-23T09:45:16.927933+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:53:03.362600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926273+0000","last_change":"2024-03-23T09:45:16.926273+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926273+0000","last_undegraded":"2024-03-23T09:45:16.926273+0000","last_fullsized":"2024-03-23T09:45:16.926273+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:11:42.583654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926761+0000","last_change":"2024-03-23T09:45:16.926761+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926761+0000","last_undegraded":"2024-03-23T09:45:16.926761+0000","last_fullsized":"2024-03-23T09:45:16.926761+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:53:08.530702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929338+0000","last_change":"2024-03-23T09:45:16.929338+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929338+0000","last_undegraded":"2024-03-23T09:45:16.929338+0000","last_fullsized":"2024-03-23T09:45:16.929338+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:11:42.583654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930323+0000","last_change":"2024-03-23T09:45:16.930323+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930323+0000","last_undegraded":"2024-03-23T09:45:16.930323+0000","last_fullsized":"2024-03-23T09:45:16.930323+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:04.521715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":212,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-03-23T09:45:14.973650+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:45:14.973650+0000","last_peered":"2024-03-23T09:45:14.973650+0000","last_clean":"2024-03-23T09:45:14.973650+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:45:14.973650+0000","last_undegraded":"2024-03-23T09:45:14.973650+0000","last_fullsized":"2024-03-23T09:45:14.973650+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":926962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53100000000000003,"15min":0.48999999999999999},"min":{"1min":0.35699999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72199999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54500000000000004,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.76100000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54000000000000004,"15min":0.51200000000000001},"min":{"1min":0.42199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.751,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.439,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78500000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.69699999999999995}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.44400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.496}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.48399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.84699999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82699999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68000000000000005}]},{"osd":4,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.497,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.622,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.33200000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65200000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.61899999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.38300000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56699999999999995,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":0.874,"15min":0.874},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.379,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.42399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":335975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.371,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.315},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.38900000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53000000000000003,"15min":0.503},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.71699999999999997,"5min":0.751,"15min":0.751},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68200000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.61399999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65600000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.41699999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.71499999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.44600000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.45600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.45300000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.70599999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57499999999999996}]},{"osd":4,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.48599999999999999,"15min":0.504},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.626,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.376},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.52700000000000002,"15min":0.54700000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.69799999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.46999999999999997}]},{"osd":5,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.50700000000000001,"15min":0.53000000000000003},"min":{"1min":0.36199999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54400000000000004,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.40200000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.625}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":336594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.51600000000000001,"15min":0.48199999999999998},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70799999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55800000000000005,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70699999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.48999999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52700000000000002,"15min":0.49199999999999999},"min":{"1min":0.41999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.504},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.65000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.44500000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.36599999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53800000000000003,"15min":0.504},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56699999999999995,"15min":0.53400000000000003},"min":{"1min":0.42699999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.69499999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.502}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45900000000000002,"15min":0.46999999999999997},"min":{"1min":0.29599999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67900000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.34799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71999999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.32900000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.498,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.627,"5min":0.749,"15min":0.749},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60299999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.34699999999999998}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.746,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56399999999999995,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":330777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.437,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.747,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.502,"15min":0.47399999999999998},"min":{"1min":0.36899999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.63100000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51700000000000002,"15min":0.504},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.73099999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.41599999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.53800000000000003},"min":{"1min":0.50700000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75800000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.503},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.79900000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66600000000000004}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68400000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.371},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.40500000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.71799999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38500000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46300000000000002,"15min":0.46899999999999997},"min":{"1min":0.32600000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67900000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.309},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.48899999999999999,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.66000000000000003,"5min":1.008,"15min":1.008},"last":0.40100000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.307,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60999999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.432},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.67200000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48599999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52500000000000002,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.371,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.68799999999999994,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":802816,"data_stored":330777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.34100000000000003,"15min":0.316},"min":{"1min":0.28100000000000003,"5min":0.183,"15min":0.183},"max":{"1min":0.627,"5min":1.45,"15min":1.45},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.315},"min":{"1min":0.23100000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.51900000000000002,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.34100000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33400000000000002,"15min":0.312},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54300000000000004,"5min":1.323,"15min":1.323},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.36799999999999999,"15min":0.34899999999999998},"min":{"1min":0.315,"5min":0.218,"15min":0.218},"max":{"1min":0.67800000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.46500000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.38100000000000001},"min":{"1min":0.309,"5min":0.251,"15min":0.251},"max":{"1min":0.61599999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.42999999999999999,"15min":0.41199999999999998},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70899999999999996,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.749,"5min":1.337,"15min":1.337},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.54500000000000004},"min":{"1min":0.437,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":2.758,"15min":2.758},"last":0.74399999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77500000000000002,"5min":1.276,"15min":1.276},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86499999999999999,"5min":1.895,"15min":1.895},"last":0.76300000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.62,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70699999999999996,"5min":5.4539999999999997,"15min":5.4539999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83399999999999996,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.64600000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.60899999999999999},"min":{"1min":0.45500000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89300000000000002,"5min":3.7400000000000002,"15min":3.7400000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65300000000000002,"15min":0.61899999999999999},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":802816,"data_stored":330777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.249,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.314},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36799999999999999,"15min":0.378},"min":{"1min":0.26200000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52200000000000002,"5min":0.872,"15min":0.872},"last":0.27000000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.28499999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.629,"15min":0.629},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.39000000000000001},"min":{"1min":0.24299999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58699999999999997,"5min":0.746,"15min":0.746},"last":0.32900000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.34100000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61399999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.27800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.55000000000000004,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.38400000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.44900000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84699999999999998,"5min":1.216,"15min":1.216},"last":0.436},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85499999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.65900000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.626,"15min":0.63300000000000001},"min":{"1min":0.51200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.754},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83999999999999997,"5min":1.101,"15min":1.101},"last":0.58499999999999996}]},{"osd":6,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.53100000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89100000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63900000000000001,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89500000000000002,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.78900000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.56899999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.872,"5min":1.155,"15min":1.155},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64400000000000002,"15min":0.65400000000000003},"min":{"1min":0.46999999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82099999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":802816,"data_stored":330777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34799999999999998,"15min":0.33700000000000002},"min":{"1min":0.247,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.5,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42599999999999999,"15min":0.47799999999999998},"min":{"1min":0.23400000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.71999999999999997,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.36399999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.34499999999999997},"min":{"1min":0.251,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39000000000000001,"15min":0.375},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57399999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.29099999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42199999999999999,"15min":0.40999999999999998},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63700000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.45100000000000001,"15min":0.50700000000000001},"min":{"1min":0.29199999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.61799999999999999,"5min":2.75,"15min":2.75},"last":0.311}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.48999999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.70399999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94199999999999995,"5min":2.778,"15min":2.778},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89000000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.64100000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66500000000000004,"15min":0.70899999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66900000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89200000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.75900000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.59399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1404928,"data_stored":926343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33400000000000002,"15min":0.32300000000000001},"min":{"1min":0.21299999999999999,"5min":0.184,"15min":0.184},"max":{"1min":0.51300000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.35099999999999998},"min":{"1min":0.23999999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.35699999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.38100000000000001,"15min":0.36699999999999999},"min":{"1min":0.28899999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55400000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.29999999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55200000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.31900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.36799999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.48999999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.622,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.53200000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56100000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.44600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70799999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.504,"5min":0.379,"15min":0.379},"max":{"1min":0.875,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.501}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82399999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.64400000000000002},"min":{"1min":0.54200000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89400000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.64300000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.51000000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85799999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.626},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.501,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80700000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:45:21.645 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:45:21.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.782+0000 7f9495626640 1 Processor -- start 2024-03-23T09:45:21.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.782+0000 7f9495626640 1 -- start start 2024-03-23T09:45:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.782+0000 7f9495626640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94900b8350 0x7f94900cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.782+0000 7f9495626640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f94900cdc40 0x7f94900d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.782+0000 7f9495626640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94900ce090 con 0x7f949015ca90 2024-03-23T09:45:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.782+0000 7f9495626640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94900d11e0 con 0x7f94900b8350 2024-03-23T09:45:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.782+0000 7f9495626640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9490160f60 con 0x7f94900cdc40 2024-03-23T09:45:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.782+0000 7f948e7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94900b8350 0x7f94900cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.782+0000 7f948e7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94900b8350 0x7f94900cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47952/0 (socket says 172.21.15.47:47952) 2024-03-23T09:45:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.782+0000 7f948e7fc640 1 -- 172.21.15.47:0/3662419477 learned_addr learned my addr 172.21.15.47:0/3662419477 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.783+0000 7f948f7fe640 1 --2- 172.21.15.47:0/3662419477 >> v2:172.21.15.175:3300/0 conn(0x7f94900cdc40 0x7f94900d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.783+0000 7f948e7fc640 1 -- 172.21.15.47:0/3662419477 >> v2:172.21.15.175:3300/0 conn(0x7f94900cdc40 msgr2=0x7f94900d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:21.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.783+0000 7f948e7fc640 1 --2- 172.21.15.47:0/3662419477 >> v2:172.21.15.175:3300/0 conn(0x7f94900cdc40 0x7f94900d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.783+0000 7f948e7fc640 1 -- 172.21.15.47:0/3662419477 >> v1:172.21.15.175:6789/0 conn(0x7f949015ca90 legacy=0x7f9490160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:21.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.783+0000 7f948e7fc640 1 -- 172.21.15.47:0/3662419477 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9490161240 con 0x7f94900b8350 2024-03-23T09:45:21.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.783+0000 7f948e7fc640 1 --2- 172.21.15.47:0/3662419477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94900b8350 0x7f94900cd700 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f94900b5930 tx=0x7f947802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21bbf0ec6c9d0e1e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:21.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.784+0000 7f948dffb640 1 -- 172.21.15.47:0/3662419477 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94780317e0 con 0x7f94900b8350 2024-03-23T09:45:21.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.784+0000 7f948dffb640 1 -- 172.21.15.47:0/3662419477 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9478031940 con 0x7f94900b8350 2024-03-23T09:45:21.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.784+0000 7f948dffb640 1 -- 172.21.15.47:0/3662419477 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478031cd0 con 0x7f94900b8350 2024-03-23T09:45:21.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.784+0000 7f9495626640 1 -- 172.21.15.47:0/3662419477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94900b8350 msgr2=0x7f94900cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.784+0000 7f9495626640 1 --2- 172.21.15.47:0/3662419477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94900b8350 0x7f94900cd700 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f94900b5930 tx=0x7f947802b880 comp rx=0 tx=0).stop 2024-03-23T09:45:21.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.784+0000 7f9495626640 1 -- 172.21.15.47:0/3662419477 shutdown_connections 2024-03-23T09:45:21.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.784+0000 7f9495626640 1 --2- 172.21.15.47:0/3662419477 >> v2:172.21.15.175:3300/0 conn(0x7f94900cdc40 0x7f94900d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.784+0000 7f9495626640 1 --2- 172.21.15.47:0/3662419477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94900b8350 0x7f94900cd700 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.784+0000 7f9495626640 1 -- 172.21.15.47:0/3662419477 >> 172.21.15.47:0/3662419477 conn(0x7f94900659d0 msgr2=0x7f9490065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:21.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.784+0000 7f9495626640 1 -- 172.21.15.47:0/3662419477 shutdown_connections 2024-03-23T09:45:21.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.784+0000 7f9495626640 1 -- 172.21.15.47:0/3662419477 wait complete. 2024-03-23T09:45:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.785+0000 7f9495626640 1 Processor -- start 2024-03-23T09:45:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.786+0000 7f9495626640 1 -- start start 2024-03-23T09:45:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.786+0000 7f9495626640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94900b8350 0x7f949014bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.786+0000 7f9495626640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94900cdc40 0x7f9490159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.786+0000 7f9495626640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949015ca90 0x7f94900b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.786+0000 7f9495626640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949015b2a0 con 0x7f949015ca90 2024-03-23T09:45:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.786+0000 7f9495626640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949015b6c0 con 0x7f94900b8350 2024-03-23T09:45:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.786+0000 7f9495626640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949014c310 con 0x7f94900cdc40 2024-03-23T09:45:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.786+0000 7f948f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949015ca90 0x7f94900b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.786+0000 7f948f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949015ca90 0x7f94900b5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47966/0 (socket says 172.21.15.47:47966) 2024-03-23T09:45:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.786+0000 7f948f7fe640 1 -- 172.21.15.47:0/1905048873 learned_addr learned my addr 172.21.15.47:0/1905048873 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.786+0000 7f948e7fc640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94900cdc40 0x7f9490159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.786+0000 7f948effd640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94900b8350 0x7f949014bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.787+0000 7f948f7fe640 1 -- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94900cdc40 msgr2=0x7f9490159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.787+0000 7f948f7fe640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94900cdc40 0x7f9490159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.787+0000 7f948f7fe640 1 -- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94900b8350 msgr2=0x7f949014bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.787+0000 7f948f7fe640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94900b8350 0x7f949014bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.787+0000 7f948f7fe640 1 -- 172.21.15.47:0/1905048873 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780094d0 con 0x7f949015ca90 2024-03-23T09:45:21.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.787+0000 7f948f7fe640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949015ca90 0x7f94900b5b20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f9480007e40 tx=0x7f948000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:21.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.787+0000 7f94877fe640 1 -- 172.21.15.47:0/1905048873 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480014620 con 0x7f949015ca90 2024-03-23T09:45:21.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.787+0000 7f94877fe640 1 -- 172.21.15.47:0/1905048873 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948000ee90 con 0x7f949015ca90 2024-03-23T09:45:21.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.787+0000 7f9495626640 1 -- 172.21.15.47:0/1905048873 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949015a620 con 0x7f949015ca90 2024-03-23T09:45:21.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.787+0000 7f94877fe640 1 -- 172.21.15.47:0/1905048873 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94800149e0 con 0x7f949015ca90 2024-03-23T09:45:21.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.787+0000 7f9495626640 1 -- 172.21.15.47:0/1905048873 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949015abe0 con 0x7f949015ca90 2024-03-23T09:45:21.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.788+0000 7f9495626640 1 -- 172.21.15.47:0/1905048873 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9490072360 con 0x7f949015ca90 2024-03-23T09:45:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.788+0000 7f94877fe640 1 -- 172.21.15.47:0/1905048873 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9480014b40 con 0x7f949015ca90 2024-03-23T09:45:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.789+0000 7f94877fe640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f947006ac00 0x7f947006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.789+0000 7f94877fe640 1 -- 172.21.15.47:0/1905048873 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9480089d80 con 0x7f949015ca90 2024-03-23T09:45:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.789+0000 7f948effd640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f947006ac00 0x7f947006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.790+0000 7f948effd640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f947006ac00 0x7f947006d0c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9488004ea0 tx=0x7f9488009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:21.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.794+0000 7f94877fe640 1 -- 172.21.15.47:0/1905048873 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9480054ec0 con 0x7f949015ca90 2024-03-23T09:45:21.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.967+0000 7f9495626640 1 -- 172.21.15.47:0/1905048873 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94900d0e70 con 0x7f947006ac00 2024-03-23T09:45:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.975+0000 7f94877fe640 1 -- 172.21.15.47:0/1905048873 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278669 (secure 0 0 0) 0x7f9488005420 con 0x7f947006ac00 2024-03-23T09:45:21.978 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:45:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.977+0000 7f9495626640 1 -- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f947006ac00 msgr2=0x7f947006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.977+0000 7f9495626640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f947006ac00 0x7f947006d0c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9488004ea0 tx=0x7f9488009040 comp rx=0 tx=0).stop 2024-03-23T09:45:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.977+0000 7f9495626640 1 -- 172.21.15.47:0/1905048873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949015ca90 msgr2=0x7f94900b5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.977+0000 7f9495626640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949015ca90 0x7f94900b5b20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f9480007e40 tx=0x7f948000e4b0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.977+0000 7f9495626640 1 -- 172.21.15.47:0/1905048873 shutdown_connections 2024-03-23T09:45:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.977+0000 7f9495626640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f947006ac00 0x7f947006d0c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.977+0000 7f9495626640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949015ca90 0x7f94900b5b20 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.977+0000 7f9495626640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94900cdc40 0x7f9490159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.977+0000 7f9495626640 1 --2- 172.21.15.47:0/1905048873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94900b8350 0x7f949014bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.977+0000 7f9495626640 1 -- 172.21.15.47:0/1905048873 >> 172.21.15.47:0/1905048873 conn(0x7f94900659d0 msgr2=0x7f949015fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.977+0000 7f9495626640 1 -- 172.21.15.47:0/1905048873 shutdown_connections 2024-03-23T09:45:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:21.977+0000 7f9495626640 1 -- 172.21.15.47:0/1905048873 wait complete. 2024-03-23T09:45:21.989 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-03-23T09:45:20.400686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244136,"kb_used_data":7472,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754730584,"statfs":{"total":773094113280,"available":772844118016,"internally_reserved":0,"allocated":7651328,"data_stored":3848982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":242208312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000971"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926611+0000","last_change":"2024-03-23T09:45:16.926611+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926611+0000","last_undegraded":"2024-03-23T09:45:16.926611+0000","last_fullsized":"2024-03-23T09:45:16.926611+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:07.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930742+0000","last_change":"2024-03-23T09:45:16.930742+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930742+0000","last_undegraded":"2024-03-23T09:45:16.930742+0000","last_fullsized":"2024-03-23T09:45:16.930742+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:10.301294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926660+0000","last_change":"2024-03-23T09:45:16.926660+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926660+0000","last_undegraded":"2024-03-23T09:45:16.926660+0000","last_fullsized":"2024-03-23T09:45:16.926660+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:18.127827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929237+0000","last_change":"2024-03-23T09:45:16.929237+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929237+0000","last_undegraded":"2024-03-23T09:45:16.929237+0000","last_fullsized":"2024-03-23T09:45:16.929237+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:28:14.722915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929116+0000","last_change":"2024-03-23T09:45:16.929116+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929116+0000","last_undegraded":"2024-03-23T09:45:16.929116+0000","last_fullsized":"2024-03-23T09:45:16.929116+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:18:20.415053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930935+0000","last_change":"2024-03-23T09:45:16.930935+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930935+0000","last_undegraded":"2024-03-23T09:45:16.930935+0000","last_fullsized":"2024-03-23T09:45:16.930935+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:53:08.530702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926321+0000","last_change":"2024-03-23T09:45:16.926321+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926321+0000","last_undegraded":"2024-03-23T09:45:16.926321+0000","last_fullsized":"2024-03-23T09:45:16.926321+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:44.872119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.927934+0000","last_change":"2024-03-23T09:45:16.927934+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.927934+0000","last_undegraded":"2024-03-23T09:45:16.927934+0000","last_fullsized":"2024-03-23T09:45:16.927934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:00:47.619314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926649+0000","last_change":"2024-03-23T09:45:16.926649+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926649+0000","last_undegraded":"2024-03-23T09:45:16.926649+0000","last_fullsized":"2024-03-23T09:45:16.926649+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:24.280980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929151+0000","last_change":"2024-03-23T09:45:16.929151+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929151+0000","last_undegraded":"2024-03-23T09:45:16.929151+0000","last_fullsized":"2024-03-23T09:45:16.929151+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:24.280980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.928831+0000","last_change":"2024-03-23T09:45:16.928831+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.928831+0000","last_undegraded":"2024-03-23T09:45:16.928831+0000","last_fullsized":"2024-03-23T09:45:16.928831+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:30:06.113245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926342+0000","last_change":"2024-03-23T09:45:16.926342+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926342+0000","last_undegraded":"2024-03-23T09:45:16.926342+0000","last_fullsized":"2024-03-23T09:45:16.926342+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:14:57.219705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926273+0000","last_change":"2024-03-23T09:45:16.926273+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926273+0000","last_undegraded":"2024-03-23T09:45:16.926273+0000","last_fullsized":"2024-03-23T09:45:16.926273+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:18:57.489494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930470+0000","last_change":"2024-03-23T09:45:16.930470+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930470+0000","last_undegraded":"2024-03-23T09:45:16.930470+0000","last_fullsized":"2024-03-23T09:45:16.930470+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:30:11.821980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.933740+0000","last_change":"2024-03-23T09:45:16.933740+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.933740+0000","last_undegraded":"2024-03-23T09:45:16.933740+0000","last_fullsized":"2024-03-23T09:45:16.933740+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:06:34.170725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926591+0000","last_change":"2024-03-23T09:45:16.926591+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926591+0000","last_undegraded":"2024-03-23T09:45:16.926591+0000","last_fullsized":"2024-03-23T09:45:16.926591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:04.521715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930328+0000","last_change":"2024-03-23T09:45:16.930328+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930328+0000","last_undegraded":"2024-03-23T09:45:16.930328+0000","last_fullsized":"2024-03-23T09:45:16.930328+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:18:57.489494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926592+0000","last_change":"2024-03-23T09:45:16.926592+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926592+0000","last_undegraded":"2024-03-23T09:45:16.926592+0000","last_fullsized":"2024-03-23T09:45:16.926592+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:00:16.438765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930133+0000","last_change":"2024-03-23T09:45:16.930133+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930133+0000","last_undegraded":"2024-03-23T09:45:16.930133+0000","last_fullsized":"2024-03-23T09:45:16.930133+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:14:57.219705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.927565+0000","last_change":"2024-03-23T09:45:16.927565+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.927565+0000","last_undegraded":"2024-03-23T09:45:16.927565+0000","last_fullsized":"2024-03-23T09:45:16.927565+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:44:44.243931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929144+0000","last_change":"2024-03-23T09:45:16.929144+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929144+0000","last_undegraded":"2024-03-23T09:45:16.929144+0000","last_fullsized":"2024-03-23T09:45:16.929144+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:38:12.502514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.933380+0000","last_change":"2024-03-23T09:45:16.933380+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.933380+0000","last_undegraded":"2024-03-23T09:45:16.933380+0000","last_fullsized":"2024-03-23T09:45:16.933380+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:38:17.881935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930448+0000","last_change":"2024-03-23T09:45:16.930448+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930448+0000","last_undegraded":"2024-03-23T09:45:16.930448+0000","last_fullsized":"2024-03-23T09:45:16.930448+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:07.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929343+0000","last_change":"2024-03-23T09:45:16.929343+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929343+0000","last_undegraded":"2024-03-23T09:45:16.929343+0000","last_fullsized":"2024-03-23T09:45:16.929343+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:03:17.222905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.933450+0000","last_change":"2024-03-23T09:45:16.933450+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.933450+0000","last_undegraded":"2024-03-23T09:45:16.933450+0000","last_fullsized":"2024-03-23T09:45:16.933450+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:41:11.998454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.927946+0000","last_change":"2024-03-23T09:45:16.927946+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.927946+0000","last_undegraded":"2024-03-23T09:45:16.927946+0000","last_fullsized":"2024-03-23T09:45:16.927946+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:42:34.167069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930417+0000","last_change":"2024-03-23T09:45:16.930417+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930417+0000","last_undegraded":"2024-03-23T09:45:16.930417+0000","last_fullsized":"2024-03-23T09:45:16.930417+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:40:34.327822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.927933+0000","last_change":"2024-03-23T09:45:16.927933+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.927933+0000","last_undegraded":"2024-03-23T09:45:16.927933+0000","last_fullsized":"2024-03-23T09:45:16.927933+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:53:03.362600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926273+0000","last_change":"2024-03-23T09:45:16.926273+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926273+0000","last_undegraded":"2024-03-23T09:45:16.926273+0000","last_fullsized":"2024-03-23T09:45:16.926273+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:11:42.583654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.926761+0000","last_change":"2024-03-23T09:45:16.926761+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.926761+0000","last_undegraded":"2024-03-23T09:45:16.926761+0000","last_fullsized":"2024-03-23T09:45:16.926761+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:53:08.530702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.929338+0000","last_change":"2024-03-23T09:45:16.929338+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.929338+0000","last_undegraded":"2024-03-23T09:45:16.929338+0000","last_fullsized":"2024-03-23T09:45:16.929338+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:11:42.583654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:45:17.923595+0000","last_change":"2024-03-23T09:45:17.923595+0000","last_active":"2024-03-23T09:45:17.923595+0000","last_peered":"2024-03-23T09:45:17.923595+0000","last_clean":"2024-03-23T09:45:17.923595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:17.923595+0000","last_undegraded":"2024-03-23T09:45:17.923595+0000","last_fullsized":"2024-03-23T09:45:17.923595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-03-23T09:45:16.930323+0000","last_change":"2024-03-23T09:45:16.930323+0000","last_active":"2024-03-23T09:45:16.919445+0000","last_peered":"2024-03-23T09:45:16.919445+0000","last_clean":"2024-03-23T09:45:16.919445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:45:16.930323+0000","last_undegraded":"2024-03-23T09:45:16.930323+0000","last_fullsized":"2024-03-23T09:45:16.930323+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:04.521715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":212,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-03-23T09:45:14.973650+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:45:14.973650+0000","last_peered":"2024-03-23T09:45:14.973650+0000","last_clean":"2024-03-23T09:45:14.973650+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:45:14.973650+0000","last_undegraded":"2024-03-23T09:45:14.973650+0000","last_fullsized":"2024-03-23T09:45:14.973650+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":926962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53100000000000003,"15min":0.48999999999999999},"min":{"1min":0.35699999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72199999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54500000000000004,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.76100000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54000000000000004,"15min":0.51200000000000001},"min":{"1min":0.42199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.751,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.439,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78500000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.69699999999999995}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.44400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.496}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.48399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.84699999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82699999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68000000000000005}]},{"osd":4,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.497,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.622,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.33200000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65200000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.61899999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.38300000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56699999999999995,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":0.874,"15min":0.874},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.379,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.42399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":335975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.371,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.315},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.38900000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53000000000000003,"15min":0.503},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.71699999999999997,"5min":0.751,"15min":0.751},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68200000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.61399999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65600000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.41699999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.71499999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.44600000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.45600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.45300000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.70599999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57499999999999996}]},{"osd":4,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.48599999999999999,"15min":0.504},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.626,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.376},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.52700000000000002,"15min":0.54700000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.69799999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.46999999999999997}]},{"osd":5,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.50700000000000001,"15min":0.53000000000000003},"min":{"1min":0.36199999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54400000000000004,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.40200000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.625}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":336594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.51600000000000001,"15min":0.48199999999999998},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70799999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55800000000000005,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70699999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.48999999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52700000000000002,"15min":0.49199999999999999},"min":{"1min":0.41999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.504},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.65000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.44500000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.36599999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53800000000000003,"15min":0.504},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56699999999999995,"15min":0.53400000000000003},"min":{"1min":0.42699999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.69499999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.502}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45900000000000002,"15min":0.46999999999999997},"min":{"1min":0.29599999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67900000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.34799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71999999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.32900000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.498,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.627,"5min":0.749,"15min":0.749},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60299999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.34699999999999998}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.746,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56399999999999995,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":330777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.437,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.747,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.502,"15min":0.47399999999999998},"min":{"1min":0.36899999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.63100000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51700000000000002,"15min":0.504},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.73099999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.41599999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.53800000000000003},"min":{"1min":0.50700000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75800000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.503},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.79900000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66600000000000004}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68400000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.371},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.40500000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.71799999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38500000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46300000000000002,"15min":0.46899999999999997},"min":{"1min":0.32600000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67900000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.309},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.48899999999999999,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.66000000000000003,"5min":1.008,"15min":1.008},"last":0.40100000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.307,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60999999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.432},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.67200000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48599999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52500000000000002,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.371,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.68799999999999994,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":802816,"data_stored":330777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.34100000000000003,"15min":0.316},"min":{"1min":0.28100000000000003,"5min":0.183,"15min":0.183},"max":{"1min":0.627,"5min":1.45,"15min":1.45},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.315},"min":{"1min":0.23100000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.51900000000000002,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.34100000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33400000000000002,"15min":0.312},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54300000000000004,"5min":1.323,"15min":1.323},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.36799999999999999,"15min":0.34899999999999998},"min":{"1min":0.315,"5min":0.218,"15min":0.218},"max":{"1min":0.67800000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.46500000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.38100000000000001},"min":{"1min":0.309,"5min":0.251,"15min":0.251},"max":{"1min":0.61599999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.42999999999999999,"15min":0.41199999999999998},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70899999999999996,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.749,"5min":1.337,"15min":1.337},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.54500000000000004},"min":{"1min":0.437,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":2.758,"15min":2.758},"last":0.74399999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77500000000000002,"5min":1.276,"15min":1.276},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86499999999999999,"5min":1.895,"15min":1.895},"last":0.76300000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.62,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70699999999999996,"5min":5.4539999999999997,"15min":5.4539999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83399999999999996,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.64600000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.60899999999999999},"min":{"1min":0.45500000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89300000000000002,"5min":3.7400000000000002,"15min":3.7400000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65300000000000002,"15min":0.61899999999999999},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":802816,"data_stored":330777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.249,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.314},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36799999999999999,"15min":0.378},"min":{"1min":0.26200000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52200000000000002,"5min":0.872,"15min":0.872},"last":0.27000000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.28499999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.629,"15min":0.629},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.39000000000000001},"min":{"1min":0.24299999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58699999999999997,"5min":0.746,"15min":0.746},"last":0.32900000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.34100000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61399999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.27800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.55000000000000004,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.38400000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.44900000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84699999999999998,"5min":1.216,"15min":1.216},"last":0.436},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85499999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.65900000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.626,"15min":0.63300000000000001},"min":{"1min":0.51200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.754},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83999999999999997,"5min":1.101,"15min":1.101},"last":0.58499999999999996}]},{"osd":6,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.53100000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89100000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63900000000000001,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89500000000000002,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.78900000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.56899999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.872,"5min":1.155,"15min":1.155},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64400000000000002,"15min":0.65400000000000003},"min":{"1min":0.46999999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82099999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":802816,"data_stored":330777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34799999999999998,"15min":0.33700000000000002},"min":{"1min":0.247,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.5,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42599999999999999,"15min":0.47799999999999998},"min":{"1min":0.23400000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.71999999999999997,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.36399999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.34499999999999997},"min":{"1min":0.251,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39000000000000001,"15min":0.375},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57399999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.29099999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42199999999999999,"15min":0.40999999999999998},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63700000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.45100000000000001,"15min":0.50700000000000001},"min":{"1min":0.29199999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.61799999999999999,"5min":2.75,"15min":2.75},"last":0.311}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.48999999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.70399999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94199999999999995,"5min":2.778,"15min":2.778},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89000000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.64100000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66500000000000004,"15min":0.70899999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66900000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89200000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.75900000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.59399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1404928,"data_stored":926343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33400000000000002,"15min":0.32300000000000001},"min":{"1min":0.21299999999999999,"5min":0.184,"15min":0.184},"max":{"1min":0.51300000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.35099999999999998},"min":{"1min":0.23999999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.35699999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.38100000000000001,"15min":0.36699999999999999},"min":{"1min":0.28899999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55400000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.29999999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55200000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.31900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.36799999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.48999999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.622,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.53200000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56100000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.44600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70799999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.504,"5min":0.379,"15min":0.379},"max":{"1min":0.875,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.501}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82399999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.64400000000000002},"min":{"1min":0.54200000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89400000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.64300000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.51000000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85799999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.626},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.501,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80700000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:45:21.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2024-03-23T09:45:21.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2024-03-23T09:45:21.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2024-03-23T09:45:21.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2024-03-23T09:45:21.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2024-03-23T09:45:21.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2024-03-23T09:45:21.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2024-03-23T09:45:21.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2024-03-23T09:45:21.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2024-03-23T09:45:21.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2024-03-23T09:45:21.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2024-03-23T09:45:21.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2024-03-23T09:45:21.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2024-03-23T09:45:21.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2024-03-23T09:45:21.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2024-03-23T09:45:21.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2024-03-23T09:45:21.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2024-03-23T09:45:21.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2024-03-23T09:45:21.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2024-03-23T09:45:21.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2024-03-23T09:45:21.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2024-03-23T09:45:21.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2024-03-23T09:45:21.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2024-03-23T09:45:21.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2024-03-23T09:45:21.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2024-03-23T09:45:21.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2024-03-23T09:45:21.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2024-03-23T09:45:21.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2024-03-23T09:45:21.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2024-03-23T09:45:21.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2024-03-23T09:45:21.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2024-03-23T09:45:21.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2024-03-23T09:45:21.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-03-23T09:45:21.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:21.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2024-03-23T09:45:21.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.926611+0000', 'last_change': '2024-03-23T09:45:16.926611+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.926611+0000', 'last_undegraded': '2024-03-23T09:45:16.926611+0000', 'last_fullsized': '2024-03-23T09:45:16.926611+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:50:07.051277+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:45:21.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2024-03-23T09:45:21.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2024-03-23T09:45:22.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.930742+0000', 'last_change': '2024-03-23T09:45:16.930742+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.930742+0000', 'last_undegraded': '2024-03-23T09:45:16.930742+0000', 'last_fullsized': '2024-03-23T09:45:16.930742+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:01:10.301294+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:45:22.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2024-03-23T09:45:22.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2024-03-23T09:45:22.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.926660+0000', 'last_change': '2024-03-23T09:45:16.926660+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.926660+0000', 'last_undegraded': '2024-03-23T09:45:16.926660+0000', 'last_fullsized': '2024-03-23T09:45:16.926660+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:03:18.127827+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:45:22.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-03-23T09:45:22.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2024-03-23T09:45:22.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.929237+0000', 'last_change': '2024-03-23T09:45:16.929237+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.929237+0000', 'last_undegraded': '2024-03-23T09:45:16.929237+0000', 'last_fullsized': '2024-03-23T09:45:16.929237+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:28:14.722915+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:45:22.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2024-03-23T09:45:22.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2024-03-23T09:45:22.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.929116+0000', 'last_change': '2024-03-23T09:45:16.929116+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.929116+0000', 'last_undegraded': '2024-03-23T09:45:16.929116+0000', 'last_fullsized': '2024-03-23T09:45:16.929116+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:18:20.415053+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:45:22.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2024-03-23T09:45:22.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2024-03-23T09:45:22.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.930935+0000', 'last_change': '2024-03-23T09:45:16.930935+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.930935+0000', 'last_undegraded': '2024-03-23T09:45:16.930935+0000', 'last_fullsized': '2024-03-23T09:45:16.930935+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:53:08.530702+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:45:22.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2024-03-23T09:45:22.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2024-03-23T09:45:22.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.926321+0000', 'last_change': '2024-03-23T09:45:16.926321+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.926321+0000', 'last_undegraded': '2024-03-23T09:45:16.926321+0000', 'last_fullsized': '2024-03-23T09:45:16.926321+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:33:44.872119+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:45:22.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-03-23T09:45:22.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2024-03-23T09:45:22.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.927934+0000', 'last_change': '2024-03-23T09:45:16.927934+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.927934+0000', 'last_undegraded': '2024-03-23T09:45:16.927934+0000', 'last_fullsized': '2024-03-23T09:45:16.927934+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:00:47.619314+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:45:22.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2024-03-23T09:45:22.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2024-03-23T09:45:22.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.926649+0000', 'last_change': '2024-03-23T09:45:16.926649+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.926649+0000', 'last_undegraded': '2024-03-23T09:45:16.926649+0000', 'last_fullsized': '2024-03-23T09:45:16.926649+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:12:24.280980+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:45:22.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-03-23T09:45:22.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2024-03-23T09:45:22.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.929151+0000', 'last_change': '2024-03-23T09:45:16.929151+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.929151+0000', 'last_undegraded': '2024-03-23T09:45:16.929151+0000', 'last_fullsized': '2024-03-23T09:45:16.929151+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T14:12:24.280980+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:45:22.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2024-03-23T09:45:22.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2024-03-23T09:45:22.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.928831+0000', 'last_change': '2024-03-23T09:45:16.928831+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.928831+0000', 'last_undegraded': '2024-03-23T09:45:16.928831+0000', 'last_fullsized': '2024-03-23T09:45:16.928831+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:30:06.113245+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:45:22.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2024-03-23T09:45:22.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2024-03-23T09:45:22.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.926342+0000', 'last_change': '2024-03-23T09:45:16.926342+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.926342+0000', 'last_undegraded': '2024-03-23T09:45:16.926342+0000', 'last_fullsized': '2024-03-23T09:45:16.926342+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:14:57.219705+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:45:22.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-03-23T09:45:22.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2024-03-23T09:45:22.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.926273+0000', 'last_change': '2024-03-23T09:45:16.926273+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.926273+0000', 'last_undegraded': '2024-03-23T09:45:16.926273+0000', 'last_fullsized': '2024-03-23T09:45:16.926273+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:18:57.489494+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:45:22.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2024-03-23T09:45:22.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2024-03-23T09:45:22.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.930470+0000', 'last_change': '2024-03-23T09:45:16.930470+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.930470+0000', 'last_undegraded': '2024-03-23T09:45:16.930470+0000', 'last_fullsized': '2024-03-23T09:45:16.930470+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T21:30:11.821980+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:45:22.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-03-23T09:45:22.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2024-03-23T09:45:22.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.933740+0000', 'last_change': '2024-03-23T09:45:16.933740+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.933740+0000', 'last_undegraded': '2024-03-23T09:45:16.933740+0000', 'last_fullsized': '2024-03-23T09:45:16.933740+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:06:34.170725+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:45:22.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2024-03-23T09:45:22.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2024-03-23T09:45:22.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.926591+0000', 'last_change': '2024-03-23T09:45:16.926591+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.926591+0000', 'last_undegraded': '2024-03-23T09:45:16.926591+0000', 'last_fullsized': '2024-03-23T09:45:16.926591+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:47:04.521715+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:45:22.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2024-03-23T09:45:22.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2024-03-23T09:45:22.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.930328+0000', 'last_change': '2024-03-23T09:45:16.930328+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.930328+0000', 'last_undegraded': '2024-03-23T09:45:16.930328+0000', 'last_fullsized': '2024-03-23T09:45:16.930328+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:18:57.489494+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:45:22.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-03-23T09:45:22.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2024-03-23T09:45:22.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.926592+0000', 'last_change': '2024-03-23T09:45:16.926592+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.926592+0000', 'last_undegraded': '2024-03-23T09:45:16.926592+0000', 'last_fullsized': '2024-03-23T09:45:16.926592+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:00:16.438765+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:45:22.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2024-03-23T09:45:22.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2024-03-23T09:45:22.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.930133+0000', 'last_change': '2024-03-23T09:45:16.930133+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.930133+0000', 'last_undegraded': '2024-03-23T09:45:16.930133+0000', 'last_fullsized': '2024-03-23T09:45:16.930133+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:14:57.219705+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:45:22.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2024-03-23T09:45:22.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2024-03-23T09:45:22.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.927565+0000', 'last_change': '2024-03-23T09:45:16.927565+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.927565+0000', 'last_undegraded': '2024-03-23T09:45:16.927565+0000', 'last_fullsized': '2024-03-23T09:45:16.927565+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:44:44.243931+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:45:22.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2024-03-23T09:45:22.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2024-03-23T09:45:22.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.929144+0000', 'last_change': '2024-03-23T09:45:16.929144+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.929144+0000', 'last_undegraded': '2024-03-23T09:45:16.929144+0000', 'last_fullsized': '2024-03-23T09:45:16.929144+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:38:12.502514+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:45:22.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2024-03-23T09:45:22.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2024-03-23T09:45:22.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.933380+0000', 'last_change': '2024-03-23T09:45:16.933380+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.933380+0000', 'last_undegraded': '2024-03-23T09:45:16.933380+0000', 'last_fullsized': '2024-03-23T09:45:16.933380+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:38:17.881935+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:45:22.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2024-03-23T09:45:22.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2024-03-23T09:45:22.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.930448+0000', 'last_change': '2024-03-23T09:45:16.930448+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.930448+0000', 'last_undegraded': '2024-03-23T09:45:16.930448+0000', 'last_fullsized': '2024-03-23T09:45:16.930448+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:50:07.051277+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:45:22.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-03-23T09:45:22.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2024-03-23T09:45:22.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.929343+0000', 'last_change': '2024-03-23T09:45:16.929343+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.929343+0000', 'last_undegraded': '2024-03-23T09:45:16.929343+0000', 'last_fullsized': '2024-03-23T09:45:16.929343+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T13:03:17.222905+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:45:22.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2024-03-23T09:45:22.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2024-03-23T09:45:22.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.933450+0000', 'last_change': '2024-03-23T09:45:16.933450+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.933450+0000', 'last_undegraded': '2024-03-23T09:45:16.933450+0000', 'last_fullsized': '2024-03-23T09:45:16.933450+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T15:41:11.998454+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:45:22.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-03-23T09:45:22.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2024-03-23T09:45:22.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.927946+0000', 'last_change': '2024-03-23T09:45:16.927946+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.927946+0000', 'last_undegraded': '2024-03-23T09:45:16.927946+0000', 'last_fullsized': '2024-03-23T09:45:16.927946+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T15:42:34.167069+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:45:22.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-03-23T09:45:22.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2024-03-23T09:45:22.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.930417+0000', 'last_change': '2024-03-23T09:45:16.930417+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.930417+0000', 'last_undegraded': '2024-03-23T09:45:16.930417+0000', 'last_fullsized': '2024-03-23T09:45:16.930417+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T12:40:34.327822+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:45:22.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2024-03-23T09:45:22.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2024-03-23T09:45:22.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.927933+0000', 'last_change': '2024-03-23T09:45:16.927933+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.927933+0000', 'last_undegraded': '2024-03-23T09:45:16.927933+0000', 'last_fullsized': '2024-03-23T09:45:16.927933+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T09:53:03.362600+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:45:22.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2024-03-23T09:45:22.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2024-03-23T09:45:22.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.926273+0000', 'last_change': '2024-03-23T09:45:16.926273+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.926273+0000', 'last_undegraded': '2024-03-23T09:45:16.926273+0000', 'last_fullsized': '2024-03-23T09:45:16.926273+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:11:42.583654+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:45:22.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2024-03-23T09:45:22.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2024-03-23T09:45:22.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.926761+0000', 'last_change': '2024-03-23T09:45:16.926761+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.926761+0000', 'last_undegraded': '2024-03-23T09:45:16.926761+0000', 'last_fullsized': '2024-03-23T09:45:16.926761+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:53:08.530702+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:45:22.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2024-03-23T09:45:22.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2024-03-23T09:45:22.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.929338+0000', 'last_change': '2024-03-23T09:45:16.929338+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.929338+0000', 'last_undegraded': '2024-03-23T09:45:16.929338+0000', 'last_fullsized': '2024-03-23T09:45:16.929338+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:11:42.583654+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:45:22.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2024-03-23T09:45:22.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:45:17.923595+0000', 'last_change': '2024-03-23T09:45:17.923595+0000', 'last_active': '2024-03-23T09:45:17.923595+0000', 'last_peered': '2024-03-23T09:45:17.923595+0000', 'last_clean': '2024-03-23T09:45:17.923595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:17.923595+0000', 'last_undegraded': '2024-03-23T09:45:17.923595+0000', 'last_fullsized': '2024-03-23T09:45:17.923595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:17.923595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:45:22.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2024-03-23T09:45:22.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:45:16.930323+0000', 'last_change': '2024-03-23T09:45:16.930323+0000', 'last_active': '2024-03-23T09:45:16.919445+0000', 'last_peered': '2024-03-23T09:45:16.919445+0000', 'last_clean': '2024-03-23T09:45:16.919445+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:45:16.930323+0000', 'last_undegraded': '2024-03-23T09:45:16.930323+0000', 'last_fullsized': '2024-03-23T09:45:16.930323+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'last_clean_scrub_stamp': '2024-03-23T09:45:16.919445+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:47:04.521715+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:45:22.010 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-23T09:45:22.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.128+0000 7f380db0a640 1 Processor -- start 2024-03-23T09:45:22.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.128+0000 7f380db0a640 1 -- start start 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.129+0000 7f380db0a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380815cb50 0x7f38080b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.129+0000 7f380db0a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f38080cd9e0 0x7f38080b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.129+0000 7f380db0a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38080b72f0 con 0x7f38080cd5a0 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.129+0000 7f380db0a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38080b7e70 con 0x7f38080cd9e0 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.129+0000 7f380db0a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380815cf30 con 0x7f380815cb50 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.129+0000 7f3806ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f38080cd5a0 0x7f38080d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47590/0 (socket says 172.21.15.47:47590) 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.129+0000 7f3806ffd640 1 -- 172.21.15.47:0/1466483295 learned_addr learned my addr 172.21.15.47:0/1466483295 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.129+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619594992 0 0) 0x7f38080b72f0 con 0x7f38080cd5a0 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.129+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f0003520 con 0x7f38080cd5a0 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.129+0000 7f38077fe640 1 --2- 172.21.15.47:0/1466483295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380815cb50 0x7f38080b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.129+0000 7f3807fff640 1 --2- 172.21.15.47:0/1466483295 >> v2:172.21.15.175:3300/0 conn(0x7f38080cd9e0 0x7f38080b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.129+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1536651553 0 0) 0x7f37f0003520 con 0x7f38080cd5a0 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380815cb50 msgr2=0x7f38080b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38067fc640 1 --2- 172.21.15.47:0/1466483295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380815cb50 0x7f38080b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 >> v2:172.21.15.175:3300/0 conn(0x7f38080cd9e0 msgr2=0x7f38080b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38067fc640 1 --2- 172.21.15.47:0/1466483295 >> v2:172.21.15.175:3300/0 conn(0x7f38080cd9e0 0x7f38080b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38080b75d0 con 0x7f38080cd5a0 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f37fc001670 con 0x7f38080cd5a0 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f3807fff640 1 --2- 172.21.15.47:0/1466483295 >> v2:172.21.15.175:3300/0 conn(0x7f38080cd9e0 0x7f38080b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 >> v1:172.21.15.47:6789/0 conn(0x7f38080cd5a0 legacy=0x7f38080d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38067fc640 1 --2- 172.21.15.47:0/1466483295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37f0003970 0x7f37f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38067fc640 1 --2- 172.21.15.47:0/1466483295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380815cb50 0x7f37f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38067fc640 1 --2- 172.21.15.47:0/1466483295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37f0004d00 0x7f37f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080b7e70 con 0x7f37f0004d00 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380815cf30 con 0x7f37f0003970 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f000ba40 con 0x7f380815cb50 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f3807fff640 1 --2- 172.21.15.47:0/1466483295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37f0004d00 0x7f37f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f38077fe640 1 --2- 172.21.15.47:0/1466483295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37f0003970 0x7f37f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f3806ffd640 1 --2- 172.21.15.47:0/1466483295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380815cb50 0x7f37f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f3807fff640 1 -- 172.21.15.47:0/1466483295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380815cb50 msgr2=0x7f37f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f3807fff640 1 --2- 172.21.15.47:0/1466483295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380815cb50 0x7f37f00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f3807fff640 1 -- 172.21.15.47:0/1466483295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37f0003970 msgr2=0x7f37f0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f3807fff640 1 --2- 172.21.15.47:0/1466483295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37f0003970 0x7f37f0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.130+0000 7f3807fff640 1 -- 172.21.15.47:0/1466483295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f0003520 con 0x7f37f0004d00 2024-03-23T09:45:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.131+0000 7f3807fff640 1 --2- 172.21.15.47:0/1466483295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37f0004d00 0x7f37f0005180 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f38000073e0 tx=0x7f37f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959221fe83a57408 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.131+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380002cdc0 con 0x7f37f0004d00 2024-03-23T09:45:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.131+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3800034440 con 0x7f37f0004d00 2024-03-23T09:45:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.131+0000 7f38067fc640 1 -- 172.21.15.47:0/1466483295 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38000347b0 con 0x7f37f0004d00 2024-03-23T09:45:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.131+0000 7f380db0a640 1 -- 172.21.15.47:0/1466483295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37f0004d00 msgr2=0x7f37f0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.131+0000 7f380db0a640 1 --2- 172.21.15.47:0/1466483295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37f0004d00 0x7f37f0005180 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f38000073e0 tx=0x7f37f00057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.131+0000 7f3807fff640 1 -- 172.21.15.47:0/1466483295 reap_dead start 2024-03-23T09:45:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.131+0000 7f380db0a640 1 -- 172.21.15.47:0/1466483295 shutdown_connections 2024-03-23T09:45:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.131+0000 7f380db0a640 1 -- 172.21.15.47:0/1466483295 >> 172.21.15.47:0/1466483295 conn(0x7f3808065df0 msgr2=0x7f3808066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.131+0000 7f380db0a640 1 -- 172.21.15.47:0/1466483295 shutdown_connections 2024-03-23T09:45:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.132+0000 7f380db0a640 1 -- 172.21.15.47:0/1466483295 wait complete. 2024-03-23T09:45:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.133+0000 7f380db0a640 1 Processor -- start 2024-03-23T09:45:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.133+0000 7f380db0a640 1 -- start start 2024-03-23T09:45:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.133+0000 7f380db0a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38080c7ab0 0x7f3808158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.133+0000 7f380db0a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38080ca2b0 0x7f38081591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.133+0000 7f380db0a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38080c7e90 0x7f38081f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.133+0000 7f380db0a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38081f35e0 con 0x7f38080c7e90 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.133+0000 7f380db0a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38081f2970 con 0x7f38080ca2b0 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.133+0000 7f380db0a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38081f2c50 con 0x7f38080c7ab0 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f3807fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38080c7e90 0x7f38081f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f3807fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38080c7e90 0x7f38081f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47982/0 (socket says 172.21.15.47:47982) 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f3807fff640 1 -- 172.21.15.47:0/216677492 learned_addr learned my addr 172.21.15.47:0/216677492 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f3806ffd640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38080ca2b0 0x7f38081591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f38077fe640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38080c7ab0 0x7f3808158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f3807fff640 1 -- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38080c7ab0 msgr2=0x7f3808158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f3807fff640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38080c7ab0 0x7f3808158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f3807fff640 1 -- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38080ca2b0 msgr2=0x7f38081591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f3807fff640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38080ca2b0 0x7f38081591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f3807fff640 1 -- 172.21.15.47:0/216677492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38081f2f30 con 0x7f38080c7e90 2024-03-23T09:45:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f3806ffd640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38080ca2b0 0x7f38081591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f38077fe640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38080c7ab0 0x7f3808158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f3807fff640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38080c7e90 0x7f38081f5a10 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3800002230 tx=0x7f3800004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f37effff640 1 -- 172.21.15.47:0/216677492 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800034e20 con 0x7f38080c7e90 2024-03-23T09:45:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f37effff640 1 -- 172.21.15.47:0/216677492 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380003e440 con 0x7f38080c7e90 2024-03-23T09:45:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f380db0a640 1 -- 172.21.15.47:0/216677492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38081f3210 con 0x7f38080c7e90 2024-03-23T09:45:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f37effff640 1 -- 172.21.15.47:0/216677492 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380003e7f0 con 0x7f38080c7e90 2024-03-23T09:45:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.134+0000 7f380db0a640 1 -- 172.21.15.47:0/216677492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38080ccbb0 con 0x7f38080c7e90 2024-03-23T09:45:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.135+0000 7f380db0a640 1 -- 172.21.15.47:0/216677492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3808072360 con 0x7f38080c7e90 2024-03-23T09:45:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.136+0000 7f37effff640 1 -- 172.21.15.47:0/216677492 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f380003e950 con 0x7f38080c7e90 2024-03-23T09:45:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.136+0000 7f37effff640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f37f406ac00 0x7f37f406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.136+0000 7f37effff640 1 -- 172.21.15.47:0/216677492 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38000ab5a0 con 0x7f38080c7e90 2024-03-23T09:45:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.136+0000 7f38077fe640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f37f406ac00 0x7f37f406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.138+0000 7f38077fe640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f37f406ac00 0x7f37f406d0c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f37f80093d0 tx=0x7f37f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.141+0000 7f37effff640 1 -- 172.21.15.47:0/216677492 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f38000768a0 con 0x7f38080c7e90 2024-03-23T09:45:22.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.392+0000 7f380db0a640 1 -- 172.21.15.47:0/216677492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3808066be0 con 0x7f38080c7e90 2024-03-23T09:45:22.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.393+0000 7f37effff640 1 -- 172.21.15.47:0/216677492 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1108 (secure 0 0 0) 0x7f3800079ec0 con 0x7f38080c7e90 2024-03-23T09:45:22.393 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:{"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":276,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":70,"num_osds":8,"num_up_osds":8,"osd_up_since":1711186856,"num_in_osds":8,"osd_in_since":1711186848,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":249995264,"bytes_avail":772844118016,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":32,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6744}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-03-23T09:45:08.398156+0000","services":{}},"progress_events":{}} 2024-03-23T09:45:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.395+0000 7f380db0a640 1 -- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f37f406ac00 msgr2=0x7f37f406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.395+0000 7f380db0a640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f37f406ac00 0x7f37f406d0c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f37f80093d0 tx=0x7f37f8008040 comp rx=0 tx=0).stop 2024-03-23T09:45:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.395+0000 7f380db0a640 1 -- 172.21.15.47:0/216677492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38080c7e90 msgr2=0x7f38081f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.395+0000 7f380db0a640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38080c7e90 0x7f38081f5a10 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3800002230 tx=0x7f3800004ab0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.395+0000 7f380db0a640 1 -- 172.21.15.47:0/216677492 shutdown_connections 2024-03-23T09:45:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.395+0000 7f380db0a640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f37f406ac00 0x7f37f406d0c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.395+0000 7f380db0a640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38080c7e90 0x7f38081f5a10 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.395+0000 7f380db0a640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38080ca2b0 0x7f38081591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.395+0000 7f380db0a640 1 --2- 172.21.15.47:0/216677492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38080c7ab0 0x7f3808158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.395+0000 7f380db0a640 1 -- 172.21.15.47:0/216677492 >> 172.21.15.47:0/216677492 conn(0x7f3808065df0 msgr2=0x7f38080b5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:22.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.395+0000 7f380db0a640 1 -- 172.21.15.47:0/216677492 shutdown_connections 2024-03-23T09:45:22.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.395+0000 7f380db0a640 1 -- 172.21.15.47:0/216677492 wait complete. 2024-03-23T09:45:22.405 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-23T09:45:22.406 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:45:22.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.507+0000 7f2a7fbc4640 1 Processor -- start 2024-03-23T09:45:22.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.507+0000 7f2a7fbc4640 1 -- start start 2024-03-23T09:45:22.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.508+0000 7f2a7fbc4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a780cfd20 0x7f2a78098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.508+0000 7f2a7fbc4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2a78098f40 0x7f2a780caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.508+0000 7f2a7fbc4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a780cea50 con 0x7f2a78160250 2024-03-23T09:45:22.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.508+0000 7f2a7fbc4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a780993d0 con 0x7f2a78098f40 2024-03-23T09:45:22.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.508+0000 7f2a7fbc4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a780cf890 con 0x7f2a780cfd20 2024-03-23T09:45:22.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.508+0000 7f2a7e161640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2a78098f40 0x7f2a780caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.508+0000 7f2a7d15f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a780cfd20 0x7f2a78098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.508+0000 7f2a7d960640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f2a78160250 0x7f2a7806bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:42756/0 (socket says 172.21.15.47:42756) 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.508+0000 7f2a7e161640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2a78098f40 0x7f2a780caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:59720/0 (socket says 172.21.15.47:59720) 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.508+0000 7f2a7d15f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a780cfd20 0x7f2a78098a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:42132/0 (socket says 172.21.15.47:42132) 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.508+0000 7f2a7d960640 1 -- 172.21.15.47:0/2699195586 learned_addr learned my addr 172.21.15.47:0/2699195586 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.509+0000 7f2a7c95e640 1 -- 172.21.15.47:0/2699195586 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401270275 0 0) 0x7f2a780cea50 con 0x7f2a78160250 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.509+0000 7f2a7c95e640 1 -- 172.21.15.47:0/2699195586 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a60003520 con 0x7f2a78160250 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.509+0000 7f2a7d15f640 1 -- 172.21.15.47:0/2699195586 >> v2:172.21.15.175:3300/0 conn(0x7f2a78098f40 msgr2=0x7f2a780caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.509+0000 7f2a7d15f640 1 --2- 172.21.15.47:0/2699195586 >> v2:172.21.15.175:3300/0 conn(0x7f2a78098f40 0x7f2a780caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.509+0000 7f2a7d15f640 1 -- 172.21.15.47:0/2699195586 >> v1:172.21.15.175:6789/0 conn(0x7f2a78160250 legacy=0x7f2a7806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.509+0000 7f2a7d15f640 1 -- 172.21.15.47:0/2699195586 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a780cec80 con 0x7f2a780cfd20 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.509+0000 7f2a7e161640 1 --2- 172.21.15.47:0/2699195586 >> v2:172.21.15.175:3300/0 conn(0x7f2a78098f40 0x7f2a780caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.509+0000 7f2a7d15f640 1 --2- 172.21.15.47:0/2699195586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a780cfd20 0x7f2a78098a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0097a0 tx=0x7f2a6c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a58ac5ae5d47e54a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.510+0000 7f2a7c95e640 1 -- 172.21.15.47:0/2699195586 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c031830 con 0x7f2a780cfd20 2024-03-23T09:45:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.510+0000 7f2a7c95e640 1 -- 172.21.15.47:0/2699195586 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c031990 con 0x7f2a780cfd20 2024-03-23T09:45:22.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.510+0000 7f2a7c95e640 1 -- 172.21.15.47:0/2699195586 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c031d00 con 0x7f2a780cfd20 2024-03-23T09:45:22.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.510+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/2699195586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a780cfd20 msgr2=0x7f2a78098a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.510+0000 7f2a7fbc4640 1 --2- 172.21.15.47:0/2699195586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a780cfd20 0x7f2a78098a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0097a0 tx=0x7f2a6c02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.510+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/2699195586 shutdown_connections 2024-03-23T09:45:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.510+0000 7f2a7fbc4640 1 --2- 172.21.15.47:0/2699195586 >> v2:172.21.15.175:3300/0 conn(0x7f2a78098f40 0x7f2a780caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.510+0000 7f2a7fbc4640 1 --2- 172.21.15.47:0/2699195586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a780cfd20 0x7f2a78098a00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.510+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/2699195586 >> 172.21.15.47:0/2699195586 conn(0x7f2a780b2140 msgr2=0x7f2a780b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.510+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/2699195586 shutdown_connections 2024-03-23T09:45:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.510+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/2699195586 wait complete. 2024-03-23T09:45:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.511+0000 7f2a7fbc4640 1 Processor -- start 2024-03-23T09:45:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.511+0000 7f2a7fbc4640 1 -- start start 2024-03-23T09:45:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.512+0000 7f2a7fbc4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a78098f40 0x7f2a78158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.512+0000 7f2a7fbc4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a780cfd20 0x7f2a7815ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.512+0000 7f2a7fbc4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78160250 0x7f2a78160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.512+0000 7f2a7fbc4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a780b4ba0 con 0x7f2a78160250 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.512+0000 7f2a7fbc4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781f1ed0 con 0x7f2a780cfd20 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.512+0000 7f2a7fbc4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7815fa90 con 0x7f2a78098f40 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.512+0000 7f2a7e161640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78160250 0x7f2a78160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.512+0000 7f2a7e161640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78160250 0x7f2a78160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47992/0 (socket says 172.21.15.47:47992) 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.512+0000 7f2a7e161640 1 -- 172.21.15.47:0/295717666 learned_addr learned my addr 172.21.15.47:0/295717666 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.512+0000 7f2a7d960640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a78098f40 0x7f2a78158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.513+0000 7f2a7d15f640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a780cfd20 0x7f2a7815ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.513+0000 7f2a7e161640 1 -- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a78098f40 msgr2=0x7f2a78158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.513+0000 7f2a7e161640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a78098f40 0x7f2a78158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.513+0000 7f2a7e161640 1 -- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a780cfd20 msgr2=0x7f2a7815ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.513+0000 7f2a7e161640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a780cfd20 0x7f2a7815ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.513+0000 7f2a7e161640 1 -- 172.21.15.47:0/295717666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a68009a70 con 0x7f2a78160250 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.513+0000 7f2a7d960640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a78098f40 0x7f2a78158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.513+0000 7f2a7e161640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78160250 0x7f2a78160630 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800efc0 tx=0x7f2a68009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:22.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.513+0000 7f2a767fc640 1 -- 172.21.15.47:0/295717666 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68009d70 con 0x7f2a78160250 2024-03-23T09:45:22.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.513+0000 7f2a767fc640 1 -- 172.21.15.47:0/295717666 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6800f040 con 0x7f2a78160250 2024-03-23T09:45:22.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.513+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/295717666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6c009400 con 0x7f2a78160250 2024-03-23T09:45:22.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.513+0000 7f2a767fc640 1 -- 172.21.15.47:0/295717666 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68003990 con 0x7f2a78160250 2024-03-23T09:45:22.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.513+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/295717666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7815f4d0 con 0x7f2a78160250 2024-03-23T09:45:22.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.514+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/295717666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a78066be0 con 0x7f2a78160250 2024-03-23T09:45:22.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.514+0000 7f2a767fc640 1 -- 172.21.15.47:0/295717666 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2a680144a0 con 0x7f2a78160250 2024-03-23T09:45:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.515+0000 7f2a767fc640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a6406ac00 0x7f2a6406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.515+0000 7f2a767fc640 1 -- 172.21.15.47:0/295717666 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a6808a010 con 0x7f2a78160250 2024-03-23T09:45:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.515+0000 7f2a7d960640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a6406ac00 0x7f2a6406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.515+0000 7f2a7d960640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a6406ac00 0x7f2a6406d0c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007fd0 tx=0x7f2a70009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:22.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.519+0000 7f2a767fc640 1 -- 172.21.15.47:0/295717666 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2a680551f0 con 0x7f2a78160250 2024-03-23T09:45:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.694+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/295717666 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a780d0180 con 0x7f2a6406ac00 2024-03-23T09:45:22.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.702+0000 7f2a767fc640 1 -- 172.21.15.47:0/295717666 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289420 (secure 0 0 0) 0x7f2a700013d0 con 0x7f2a6406ac00 2024-03-23T09:45:22.704 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:22.706 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:45:22.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.705+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a6406ac00 msgr2=0x7f2a6406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:22.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.705+0000 7f2a7fbc4640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a6406ac00 0x7f2a6406d0c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007fd0 tx=0x7f2a70009040 comp rx=0 tx=0).stop 2024-03-23T09:45:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.705+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/295717666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78160250 msgr2=0x7f2a78160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.705+0000 7f2a7fbc4640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78160250 0x7f2a78160630 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800efc0 tx=0x7f2a68009f90 comp rx=0 tx=0).stop 2024-03-23T09:45:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.705+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/295717666 shutdown_connections 2024-03-23T09:45:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.705+0000 7f2a7fbc4640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2a6406ac00 0x7f2a6406d0c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.705+0000 7f2a7fbc4640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78160250 0x7f2a78160630 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.705+0000 7f2a7fbc4640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a780cfd20 0x7f2a7815ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.705+0000 7f2a7fbc4640 1 --2- 172.21.15.47:0/295717666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a78098f40 0x7f2a78158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.705+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/295717666 >> 172.21.15.47:0/295717666 conn(0x7f2a780b2140 msgr2=0x7f2a7806c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.705+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/295717666 shutdown_connections 2024-03-23T09:45:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:22.705+0000 7f2a7fbc4640 1 -- 172.21.15.47:0/295717666 wait complete. 2024-03-23T09:45:22.715 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-03-23T09:45:22.401136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244536,"kb_used_data":7872,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754730184,"statfs":{"total":773094113280,"available":772843708416,"internally_reserved":0,"allocated":8060928,"data_stored":4031774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":242208312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.476605"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306532+0000","last_change":"2024-03-23T09:45:18.324776+0000","last_active":"2024-03-23T09:45:19.306532+0000","last_peered":"2024-03-23T09:45:19.306532+0000","last_clean":"2024-03-23T09:45:19.306532+0000","last_became_active":"2024-03-23T09:45:18.324369+0000","last_became_peered":"2024-03-23T09:45:18.324369+0000","last_unstale":"2024-03-23T09:45:19.306532+0000","last_undegraded":"2024-03-23T09:45:19.306532+0000","last_fullsized":"2024-03-23T09:45:19.306532+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:27:48.512037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.390006+0000","last_change":"2024-03-23T09:45:18.326597+0000","last_active":"2024-03-23T09:45:20.390006+0000","last_peered":"2024-03-23T09:45:20.390006+0000","last_clean":"2024-03-23T09:45:20.390006+0000","last_became_active":"2024-03-23T09:45:18.326253+0000","last_became_peered":"2024-03-23T09:45:18.326253+0000","last_unstale":"2024-03-23T09:45:20.390006+0000","last_undegraded":"2024-03-23T09:45:20.390006+0000","last_fullsized":"2024-03-23T09:45:20.390006+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:55:13.005801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602228+0000","last_change":"2024-03-23T09:45:18.311402+0000","last_active":"2024-03-23T09:45:20.602228+0000","last_peered":"2024-03-23T09:45:20.602228+0000","last_clean":"2024-03-23T09:45:20.602228+0000","last_became_active":"2024-03-23T09:45:18.311152+0000","last_became_peered":"2024-03-23T09:45:18.311152+0000","last_unstale":"2024-03-23T09:45:20.602228+0000","last_undegraded":"2024-03-23T09:45:20.602228+0000","last_fullsized":"2024-03-23T09:45:20.602228+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:59:26.151585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.304413+0000","last_change":"2024-03-23T09:45:18.322785+0000","last_active":"2024-03-23T09:45:19.304413+0000","last_peered":"2024-03-23T09:45:19.304413+0000","last_clean":"2024-03-23T09:45:19.304413+0000","last_became_active":"2024-03-23T09:45:18.322511+0000","last_became_peered":"2024-03-23T09:45:18.322511+0000","last_unstale":"2024-03-23T09:45:19.304413+0000","last_undegraded":"2024-03-23T09:45:19.304413+0000","last_fullsized":"2024-03-23T09:45:19.304413+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:27:54.349415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601942+0000","last_change":"2024-03-23T09:45:18.310827+0000","last_active":"2024-03-23T09:45:20.601942+0000","last_peered":"2024-03-23T09:45:20.601942+0000","last_clean":"2024-03-23T09:45:20.601942+0000","last_became_active":"2024-03-23T09:45:18.310576+0000","last_became_peered":"2024-03-23T09:45:18.310576+0000","last_unstale":"2024-03-23T09:45:20.601942+0000","last_undegraded":"2024-03-23T09:45:20.601942+0000","last_fullsized":"2024-03-23T09:45:20.601942+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:37:52.741638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389635+0000","last_change":"2024-03-23T09:45:18.326944+0000","last_active":"2024-03-23T09:45:20.389635+0000","last_peered":"2024-03-23T09:45:20.389635+0000","last_clean":"2024-03-23T09:45:20.389635+0000","last_became_active":"2024-03-23T09:45:18.326663+0000","last_became_peered":"2024-03-23T09:45:18.326663+0000","last_unstale":"2024-03-23T09:45:20.389635+0000","last_undegraded":"2024-03-23T09:45:20.389635+0000","last_fullsized":"2024-03-23T09:45:20.389635+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:08:57.748599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601573+0000","last_change":"2024-03-23T09:45:18.316836+0000","last_active":"2024-03-23T09:45:20.601573+0000","last_peered":"2024-03-23T09:45:20.601573+0000","last_clean":"2024-03-23T09:45:20.601573+0000","last_became_active":"2024-03-23T09:45:18.316481+0000","last_became_peered":"2024-03-23T09:45:18.316481+0000","last_unstale":"2024-03-23T09:45:20.601573+0000","last_undegraded":"2024-03-23T09:45:20.601573+0000","last_fullsized":"2024-03-23T09:45:20.601573+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:37.045371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389520+0000","last_change":"2024-03-23T09:45:18.321349+0000","last_active":"2024-03-23T09:45:20.389520+0000","last_peered":"2024-03-23T09:45:20.389520+0000","last_clean":"2024-03-23T09:45:20.389520+0000","last_became_active":"2024-03-23T09:45:18.320774+0000","last_became_peered":"2024-03-23T09:45:18.320774+0000","last_unstale":"2024-03-23T09:45:20.389520+0000","last_undegraded":"2024-03-23T09:45:20.389520+0000","last_fullsized":"2024-03-23T09:45:20.389520+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:11.305444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602033+0000","last_change":"2024-03-23T09:45:18.317806+0000","last_active":"2024-03-23T09:45:20.602033+0000","last_peered":"2024-03-23T09:45:20.602033+0000","last_clean":"2024-03-23T09:45:20.602033+0000","last_became_active":"2024-03-23T09:45:18.317576+0000","last_became_peered":"2024-03-23T09:45:18.317576+0000","last_unstale":"2024-03-23T09:45:20.602033+0000","last_undegraded":"2024-03-23T09:45:20.602033+0000","last_fullsized":"2024-03-23T09:45:20.602033+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:21.291320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306083+0000","last_change":"2024-03-23T09:45:18.317249+0000","last_active":"2024-03-23T09:45:19.306083+0000","last_peered":"2024-03-23T09:45:19.306083+0000","last_clean":"2024-03-23T09:45:19.306083+0000","last_became_active":"2024-03-23T09:45:18.316969+0000","last_became_peered":"2024-03-23T09:45:18.316969+0000","last_unstale":"2024-03-23T09:45:19.306083+0000","last_undegraded":"2024-03-23T09:45:19.306083+0000","last_fullsized":"2024-03-23T09:45:19.306083+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:05:38.755296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305697+0000","last_change":"2024-03-23T09:45:18.321400+0000","last_active":"2024-03-23T09:45:19.305697+0000","last_peered":"2024-03-23T09:45:19.305697+0000","last_clean":"2024-03-23T09:45:19.305697+0000","last_became_active":"2024-03-23T09:45:18.320735+0000","last_became_peered":"2024-03-23T09:45:18.320735+0000","last_unstale":"2024-03-23T09:45:19.305697+0000","last_undegraded":"2024-03-23T09:45:19.305697+0000","last_fullsized":"2024-03-23T09:45:19.305697+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:34:05.217690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.304573+0000","last_change":"2024-03-23T09:45:18.310359+0000","last_active":"2024-03-23T09:45:19.304573+0000","last_peered":"2024-03-23T09:45:19.304573+0000","last_clean":"2024-03-23T09:45:19.304573+0000","last_became_active":"2024-03-23T09:45:18.310100+0000","last_became_peered":"2024-03-23T09:45:18.310100+0000","last_unstale":"2024-03-23T09:45:19.304573+0000","last_undegraded":"2024-03-23T09:45:19.304573+0000","last_fullsized":"2024-03-23T09:45:19.304573+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:21.291320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601579+0000","last_change":"2024-03-23T09:45:18.326738+0000","last_active":"2024-03-23T09:45:20.601579+0000","last_peered":"2024-03-23T09:45:20.601579+0000","last_clean":"2024-03-23T09:45:20.601579+0000","last_became_active":"2024-03-23T09:45:18.326424+0000","last_became_peered":"2024-03-23T09:45:18.326424+0000","last_unstale":"2024-03-23T09:45:20.601579+0000","last_undegraded":"2024-03-23T09:45:20.601579+0000","last_fullsized":"2024-03-23T09:45:20.601579+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:42:53.609522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601739+0000","last_change":"2024-03-23T09:45:18.311761+0000","last_active":"2024-03-23T09:45:20.601739+0000","last_peered":"2024-03-23T09:45:20.601739+0000","last_clean":"2024-03-23T09:45:20.601739+0000","last_became_active":"2024-03-23T09:45:18.311498+0000","last_became_peered":"2024-03-23T09:45:18.311498+0000","last_unstale":"2024-03-23T09:45:20.601739+0000","last_undegraded":"2024-03-23T09:45:20.601739+0000","last_fullsized":"2024-03-23T09:45:20.601739+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:27:54.349415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601507+0000","last_change":"2024-03-23T09:45:18.316586+0000","last_active":"2024-03-23T09:45:20.601507+0000","last_peered":"2024-03-23T09:45:20.601507+0000","last_clean":"2024-03-23T09:45:20.601507+0000","last_became_active":"2024-03-23T09:45:18.316308+0000","last_became_peered":"2024-03-23T09:45:18.316308+0000","last_unstale":"2024-03-23T09:45:20.601507+0000","last_undegraded":"2024-03-23T09:45:20.601507+0000","last_fullsized":"2024-03-23T09:45:20.601507+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:22:16.856887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306488+0000","last_change":"2024-03-23T09:45:18.323303+0000","last_active":"2024-03-23T09:45:19.306488+0000","last_peered":"2024-03-23T09:45:19.306488+0000","last_clean":"2024-03-23T09:45:19.306488+0000","last_became_active":"2024-03-23T09:45:18.323013+0000","last_became_peered":"2024-03-23T09:45:18.323013+0000","last_unstale":"2024-03-23T09:45:19.306488+0000","last_undegraded":"2024-03-23T09:45:19.306488+0000","last_fullsized":"2024-03-23T09:45:19.306488+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:55:13.005801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602120+0000","last_change":"2024-03-23T09:45:18.317725+0000","last_active":"2024-03-23T09:45:20.602120+0000","last_peered":"2024-03-23T09:45:20.602120+0000","last_clean":"2024-03-23T09:45:20.602120+0000","last_became_active":"2024-03-23T09:45:18.317465+0000","last_became_peered":"2024-03-23T09:45:18.317465+0000","last_unstale":"2024-03-23T09:45:20.602120+0000","last_undegraded":"2024-03-23T09:45:20.602120+0000","last_fullsized":"2024-03-23T09:45:20.602120+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:25.285130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.304601+0000","last_change":"2024-03-23T09:45:18.326765+0000","last_active":"2024-03-23T09:45:19.304601+0000","last_peered":"2024-03-23T09:45:19.304601+0000","last_clean":"2024-03-23T09:45:19.304601+0000","last_became_active":"2024-03-23T09:45:18.326429+0000","last_became_peered":"2024-03-23T09:45:18.326429+0000","last_unstale":"2024-03-23T09:45:19.304601+0000","last_undegraded":"2024-03-23T09:45:19.304601+0000","last_fullsized":"2024-03-23T09:45:19.304601+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:15:29.666312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389951+0000","last_change":"2024-03-23T09:45:18.320301+0000","last_active":"2024-03-23T09:45:20.389951+0000","last_peered":"2024-03-23T09:45:20.389951+0000","last_clean":"2024-03-23T09:45:20.389951+0000","last_became_active":"2024-03-23T09:45:18.319899+0000","last_became_peered":"2024-03-23T09:45:18.319899+0000","last_unstale":"2024-03-23T09:45:20.389951+0000","last_undegraded":"2024-03-23T09:45:20.389951+0000","last_fullsized":"2024-03-23T09:45:20.389951+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:40:35.331972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601750+0000","last_change":"2024-03-23T09:45:18.311110+0000","last_active":"2024-03-23T09:45:20.601750+0000","last_peered":"2024-03-23T09:45:20.601750+0000","last_clean":"2024-03-23T09:45:20.601750+0000","last_became_active":"2024-03-23T09:45:18.310830+0000","last_became_peered":"2024-03-23T09:45:18.310830+0000","last_unstale":"2024-03-23T09:45:20.601750+0000","last_undegraded":"2024-03-23T09:45:20.601750+0000","last_fullsized":"2024-03-23T09:45:20.601750+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:29:21.273818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306257+0000","last_change":"2024-03-23T09:45:18.312000+0000","last_active":"2024-03-23T09:45:19.306257+0000","last_peered":"2024-03-23T09:45:19.306257+0000","last_clean":"2024-03-23T09:45:19.306257+0000","last_became_active":"2024-03-23T09:45:18.311691+0000","last_became_peered":"2024-03-23T09:45:18.311691+0000","last_unstale":"2024-03-23T09:45:19.306257+0000","last_undegraded":"2024-03-23T09:45:19.306257+0000","last_fullsized":"2024-03-23T09:45:19.306257+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:40:35.331972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305779+0000","last_change":"2024-03-23T09:45:18.316673+0000","last_active":"2024-03-23T09:45:19.305779+0000","last_peered":"2024-03-23T09:45:19.305779+0000","last_clean":"2024-03-23T09:45:19.305779+0000","last_became_active":"2024-03-23T09:45:18.316397+0000","last_became_peered":"2024-03-23T09:45:18.316397+0000","last_unstale":"2024-03-23T09:45:19.305779+0000","last_undegraded":"2024-03-23T09:45:19.305779+0000","last_fullsized":"2024-03-23T09:45:19.305779+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:55.517694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305564+0000","last_change":"2024-03-23T09:45:18.320457+0000","last_active":"2024-03-23T09:45:19.305564+0000","last_peered":"2024-03-23T09:45:19.305564+0000","last_clean":"2024-03-23T09:45:19.305564+0000","last_became_active":"2024-03-23T09:45:18.317940+0000","last_became_peered":"2024-03-23T09:45:18.317940+0000","last_unstale":"2024-03-23T09:45:19.305564+0000","last_undegraded":"2024-03-23T09:45:19.305564+0000","last_fullsized":"2024-03-23T09:45:19.305564+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:47:30.691595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305985+0000","last_change":"2024-03-23T09:45:18.323849+0000","last_active":"2024-03-23T09:45:19.305985+0000","last_peered":"2024-03-23T09:45:19.305985+0000","last_clean":"2024-03-23T09:45:19.305985+0000","last_became_active":"2024-03-23T09:45:18.323595+0000","last_became_peered":"2024-03-23T09:45:18.323595+0000","last_unstale":"2024-03-23T09:45:19.305985+0000","last_undegraded":"2024-03-23T09:45:19.305985+0000","last_fullsized":"2024-03-23T09:45:19.305985+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:55.873572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306018+0000","last_change":"2024-03-23T09:45:18.318830+0000","last_active":"2024-03-23T09:45:19.306018+0000","last_peered":"2024-03-23T09:45:19.306018+0000","last_clean":"2024-03-23T09:45:19.306018+0000","last_became_active":"2024-03-23T09:45:18.318575+0000","last_became_peered":"2024-03-23T09:45:18.318575+0000","last_unstale":"2024-03-23T09:45:19.306018+0000","last_undegraded":"2024-03-23T09:45:19.306018+0000","last_fullsized":"2024-03-23T09:45:19.306018+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:37:52.579596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601855+0000","last_change":"2024-03-23T09:45:18.327740+0000","last_active":"2024-03-23T09:45:20.601855+0000","last_peered":"2024-03-23T09:45:20.601855+0000","last_clean":"2024-03-23T09:45:20.601855+0000","last_became_active":"2024-03-23T09:45:18.327469+0000","last_became_peered":"2024-03-23T09:45:18.327469+0000","last_unstale":"2024-03-23T09:45:20.601855+0000","last_undegraded":"2024-03-23T09:45:20.601855+0000","last_fullsized":"2024-03-23T09:45:20.601855+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:22:16.856887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602441+0000","last_change":"2024-03-23T09:45:18.309404+0000","last_active":"2024-03-23T09:45:20.602441+0000","last_peered":"2024-03-23T09:45:20.602441+0000","last_clean":"2024-03-23T09:45:20.602441+0000","last_became_active":"2024-03-23T09:45:18.309119+0000","last_became_peered":"2024-03-23T09:45:18.309119+0000","last_unstale":"2024-03-23T09:45:20.602441+0000","last_undegraded":"2024-03-23T09:45:20.602441+0000","last_fullsized":"2024-03-23T09:45:20.602441+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:51:13.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306624+0000","last_change":"2024-03-23T09:45:18.318114+0000","last_active":"2024-03-23T09:45:19.306624+0000","last_peered":"2024-03-23T09:45:19.306624+0000","last_clean":"2024-03-23T09:45:19.306624+0000","last_became_active":"2024-03-23T09:45:18.317822+0000","last_became_peered":"2024-03-23T09:45:18.317822+0000","last_unstale":"2024-03-23T09:45:19.306624+0000","last_undegraded":"2024-03-23T09:45:19.306624+0000","last_fullsized":"2024-03-23T09:45:19.306624+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:08.055427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602263+0000","last_change":"2024-03-23T09:45:18.314678+0000","last_active":"2024-03-23T09:45:20.602263+0000","last_peered":"2024-03-23T09:45:20.602263+0000","last_clean":"2024-03-23T09:45:20.602263+0000","last_became_active":"2024-03-23T09:45:18.314408+0000","last_became_peered":"2024-03-23T09:45:18.314408+0000","last_unstale":"2024-03-23T09:45:20.602263+0000","last_undegraded":"2024-03-23T09:45:20.602263+0000","last_fullsized":"2024-03-23T09:45:20.602263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:37:16.370619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601633+0000","last_change":"2024-03-23T09:45:18.327731+0000","last_active":"2024-03-23T09:45:20.601633+0000","last_peered":"2024-03-23T09:45:20.601633+0000","last_clean":"2024-03-23T09:45:20.601633+0000","last_became_active":"2024-03-23T09:45:18.327458+0000","last_became_peered":"2024-03-23T09:45:18.327458+0000","last_unstale":"2024-03-23T09:45:20.601633+0000","last_undegraded":"2024-03-23T09:45:20.601633+0000","last_fullsized":"2024-03-23T09:45:20.601633+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:42:53.609522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601702+0000","last_change":"2024-03-23T09:45:18.319723+0000","last_active":"2024-03-23T09:45:20.601702+0000","last_peered":"2024-03-23T09:45:20.601702+0000","last_clean":"2024-03-23T09:45:20.601702+0000","last_became_active":"2024-03-23T09:45:18.319451+0000","last_became_peered":"2024-03-23T09:45:18.319451+0000","last_unstale":"2024-03-23T09:45:20.601702+0000","last_undegraded":"2024-03-23T09:45:20.601702+0000","last_fullsized":"2024-03-23T09:45:20.601702+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:29:21.273818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389575+0000","last_change":"2024-03-23T09:45:18.321163+0000","last_active":"2024-03-23T09:45:20.389575+0000","last_peered":"2024-03-23T09:45:20.389575+0000","last_clean":"2024-03-23T09:45:20.389575+0000","last_became_active":"2024-03-23T09:45:18.320511+0000","last_became_peered":"2024-03-23T09:45:18.320511+0000","last_unstale":"2024-03-23T09:45:20.389575+0000","last_undegraded":"2024-03-23T09:45:20.389575+0000","last_fullsized":"2024-03-23T09:45:20.389575+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:44:45.248082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306385+0000","last_change":"2024-03-23T09:45:18.327116+0000","last_active":"2024-03-23T09:45:19.306385+0000","last_peered":"2024-03-23T09:45:19.306385+0000","last_clean":"2024-03-23T09:45:19.306385+0000","last_became_active":"2024-03-23T09:45:18.326826+0000","last_became_peered":"2024-03-23T09:45:18.326826+0000","last_unstale":"2024-03-23T09:45:19.306385+0000","last_undegraded":"2024-03-23T09:45:19.306385+0000","last_fullsized":"2024-03-23T09:45:19.306385+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:18:21.419203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.323005+0000","last_change":"2024-03-23T09:45:17.939332+0000","last_active":"2024-03-23T09:45:18.323005+0000","last_peered":"2024-03-23T09:45:18.323005+0000","last_clean":"2024-03-23T09:45:18.323005+0000","last_became_active":"2024-03-23T09:45:17.938957+0000","last_became_peered":"2024-03-23T09:45:17.938957+0000","last_unstale":"2024-03-23T09:45:18.323005+0000","last_undegraded":"2024-03-23T09:45:18.323005+0000","last_fullsized":"2024-03-23T09:45:18.323005+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:07.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602200+0000","last_change":"2024-03-23T09:45:18.316911+0000","last_active":"2024-03-23T09:45:20.602200+0000","last_peered":"2024-03-23T09:45:20.602200+0000","last_clean":"2024-03-23T09:45:20.602200+0000","last_became_active":"2024-03-23T09:45:18.316661+0000","last_became_peered":"2024-03-23T09:45:18.316661+0000","last_unstale":"2024-03-23T09:45:20.602200+0000","last_undegraded":"2024-03-23T09:45:20.602200+0000","last_fullsized":"2024-03-23T09:45:20.602200+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:18:58.493644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.315818+0000","last_change":"2024-03-23T09:45:17.945601+0000","last_active":"2024-03-23T09:45:18.315818+0000","last_peered":"2024-03-23T09:45:18.315818+0000","last_clean":"2024-03-23T09:45:18.315818+0000","last_became_active":"2024-03-23T09:45:17.945281+0000","last_became_peered":"2024-03-23T09:45:17.945281+0000","last_unstale":"2024-03-23T09:45:18.315818+0000","last_undegraded":"2024-03-23T09:45:18.315818+0000","last_fullsized":"2024-03-23T09:45:18.315818+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:10.301294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389687+0000","last_change":"2024-03-23T09:45:18.319672+0000","last_active":"2024-03-23T09:45:20.389687+0000","last_peered":"2024-03-23T09:45:20.389687+0000","last_clean":"2024-03-23T09:45:20.389687+0000","last_became_active":"2024-03-23T09:45:18.319343+0000","last_became_peered":"2024-03-23T09:45:18.319343+0000","last_unstale":"2024-03-23T09:45:20.389687+0000","last_undegraded":"2024-03-23T09:45:20.389687+0000","last_fullsized":"2024-03-23T09:45:20.389687+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:37:52.579596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.327582+0000","last_change":"2024-03-23T09:45:17.950331+0000","last_active":"2024-03-23T09:45:18.327582+0000","last_peered":"2024-03-23T09:45:18.327582+0000","last_clean":"2024-03-23T09:45:18.327582+0000","last_became_active":"2024-03-23T09:45:17.950030+0000","last_became_peered":"2024-03-23T09:45:17.950030+0000","last_unstale":"2024-03-23T09:45:18.327582+0000","last_undegraded":"2024-03-23T09:45:18.327582+0000","last_fullsized":"2024-03-23T09:45:18.327582+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:18.127827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305805+0000","last_change":"2024-03-23T09:45:18.314183+0000","last_active":"2024-03-23T09:45:19.305805+0000","last_peered":"2024-03-23T09:45:19.305805+0000","last_clean":"2024-03-23T09:45:19.305805+0000","last_became_active":"2024-03-23T09:45:18.313797+0000","last_became_peered":"2024-03-23T09:45:18.313797+0000","last_unstale":"2024-03-23T09:45:19.305805+0000","last_undegraded":"2024-03-23T09:45:19.305805+0000","last_fullsized":"2024-03-23T09:45:19.305805+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:48:09.676506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.319132+0000","last_change":"2024-03-23T09:45:17.947339+0000","last_active":"2024-03-23T09:45:18.319132+0000","last_peered":"2024-03-23T09:45:18.319132+0000","last_clean":"2024-03-23T09:45:18.319132+0000","last_became_active":"2024-03-23T09:45:17.947071+0000","last_became_peered":"2024-03-23T09:45:17.947071+0000","last_unstale":"2024-03-23T09:45:18.319132+0000","last_undegraded":"2024-03-23T09:45:18.319132+0000","last_fullsized":"2024-03-23T09:45:18.319132+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:28:14.722915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601704+0000","last_change":"2024-03-23T09:45:18.315901+0000","last_active":"2024-03-23T09:45:20.601704+0000","last_peered":"2024-03-23T09:45:20.601704+0000","last_clean":"2024-03-23T09:45:20.601704+0000","last_became_active":"2024-03-23T09:45:18.315635+0000","last_became_peered":"2024-03-23T09:45:18.315635+0000","last_unstale":"2024-03-23T09:45:20.601704+0000","last_undegraded":"2024-03-23T09:45:20.601704+0000","last_fullsized":"2024-03-23T09:45:20.601704+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:55.517694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.320302+0000","last_change":"2024-03-23T09:45:17.946905+0000","last_active":"2024-03-23T09:45:18.320302+0000","last_peered":"2024-03-23T09:45:18.320302+0000","last_clean":"2024-03-23T09:45:18.320302+0000","last_became_active":"2024-03-23T09:45:17.946592+0000","last_became_peered":"2024-03-23T09:45:17.946592+0000","last_unstale":"2024-03-23T09:45:18.320302+0000","last_undegraded":"2024-03-23T09:45:18.320302+0000","last_fullsized":"2024-03-23T09:45:18.320302+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:18:20.415053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306286+0000","last_change":"2024-03-23T09:45:18.323933+0000","last_active":"2024-03-23T09:45:19.306286+0000","last_peered":"2024-03-23T09:45:19.306286+0000","last_clean":"2024-03-23T09:45:19.306286+0000","last_became_active":"2024-03-23T09:45:18.323614+0000","last_became_peered":"2024-03-23T09:45:18.323614+0000","last_unstale":"2024-03-23T09:45:19.306286+0000","last_undegraded":"2024-03-23T09:45:19.306286+0000","last_fullsized":"2024-03-23T09:45:19.306286+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:08:57.748599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.326348+0000","last_change":"2024-03-23T09:45:17.948070+0000","last_active":"2024-03-23T09:45:18.326348+0000","last_peered":"2024-03-23T09:45:18.326348+0000","last_clean":"2024-03-23T09:45:18.326348+0000","last_became_active":"2024-03-23T09:45:17.947757+0000","last_became_peered":"2024-03-23T09:45:17.947757+0000","last_unstale":"2024-03-23T09:45:18.326348+0000","last_undegraded":"2024-03-23T09:45:18.326348+0000","last_fullsized":"2024-03-23T09:45:18.326348+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:53:08.530702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601897+0000","last_change":"2024-03-23T09:45:18.307469+0000","last_active":"2024-03-23T09:45:20.601897+0000","last_peered":"2024-03-23T09:45:20.601897+0000","last_clean":"2024-03-23T09:45:20.601897+0000","last_became_active":"2024-03-23T09:45:18.307196+0000","last_became_peered":"2024-03-23T09:45:18.307196+0000","last_unstale":"2024-03-23T09:45:20.601897+0000","last_undegraded":"2024-03-23T09:45:20.601897+0000","last_fullsized":"2024-03-23T09:45:20.601897+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:26:14.593410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.303235+0000","last_change":"2024-03-23T09:45:17.947891+0000","last_active":"2024-03-23T09:45:18.303235+0000","last_peered":"2024-03-23T09:45:18.303235+0000","last_clean":"2024-03-23T09:45:18.303235+0000","last_became_active":"2024-03-23T09:45:17.947523+0000","last_became_peered":"2024-03-23T09:45:17.947523+0000","last_unstale":"2024-03-23T09:45:18.303235+0000","last_undegraded":"2024-03-23T09:45:18.303235+0000","last_fullsized":"2024-03-23T09:45:18.303235+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:44.872119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601604+0000","last_change":"2024-03-23T09:45:18.324291+0000","last_active":"2024-03-23T09:45:20.601604+0000","last_peered":"2024-03-23T09:45:20.601604+0000","last_clean":"2024-03-23T09:45:20.601604+0000","last_became_active":"2024-03-23T09:45:18.324031+0000","last_became_peered":"2024-03-23T09:45:18.324031+0000","last_unstale":"2024-03-23T09:45:20.601604+0000","last_undegraded":"2024-03-23T09:45:20.601604+0000","last_fullsized":"2024-03-23T09:45:20.601604+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:37.045371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.300181+0000","last_change":"2024-03-23T09:45:17.935722+0000","last_active":"2024-03-23T09:45:18.300181+0000","last_peered":"2024-03-23T09:45:18.300181+0000","last_clean":"2024-03-23T09:45:18.300181+0000","last_became_active":"2024-03-23T09:45:17.935483+0000","last_became_peered":"2024-03-23T09:45:17.935483+0000","last_unstale":"2024-03-23T09:45:18.300181+0000","last_undegraded":"2024-03-23T09:45:18.300181+0000","last_fullsized":"2024-03-23T09:45:18.300181+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:00:47.619314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601918+0000","last_change":"2024-03-23T09:45:18.325513+0000","last_active":"2024-03-23T09:45:20.601918+0000","last_peered":"2024-03-23T09:45:20.601918+0000","last_clean":"2024-03-23T09:45:20.601918+0000","last_became_active":"2024-03-23T09:45:18.325126+0000","last_became_peered":"2024-03-23T09:45:18.325126+0000","last_unstale":"2024-03-23T09:45:20.601918+0000","last_undegraded":"2024-03-23T09:45:20.601918+0000","last_fullsized":"2024-03-23T09:45:20.601918+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:11.381182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.305364+0000","last_change":"2024-03-23T09:45:17.945994+0000","last_active":"2024-03-23T09:45:18.305364+0000","last_peered":"2024-03-23T09:45:18.305364+0000","last_clean":"2024-03-23T09:45:18.305364+0000","last_became_active":"2024-03-23T09:45:17.945734+0000","last_became_peered":"2024-03-23T09:45:17.945734+0000","last_unstale":"2024-03-23T09:45:18.305364+0000","last_undegraded":"2024-03-23T09:45:18.305364+0000","last_fullsized":"2024-03-23T09:45:18.305364+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:24.280980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306788+0000","last_change":"2024-03-23T09:45:18.327993+0000","last_active":"2024-03-23T09:45:19.306788+0000","last_peered":"2024-03-23T09:45:19.306788+0000","last_clean":"2024-03-23T09:45:19.306788+0000","last_became_active":"2024-03-23T09:45:18.327747+0000","last_became_peered":"2024-03-23T09:45:18.327747+0000","last_unstale":"2024-03-23T09:45:19.306788+0000","last_undegraded":"2024-03-23T09:45:19.306788+0000","last_fullsized":"2024-03-23T09:45:19.306788+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:30:07.117395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.318632+0000","last_change":"2024-03-23T09:45:17.950258+0000","last_active":"2024-03-23T09:45:18.318632+0000","last_peered":"2024-03-23T09:45:18.318632+0000","last_clean":"2024-03-23T09:45:18.318632+0000","last_became_active":"2024-03-23T09:45:17.950003+0000","last_became_peered":"2024-03-23T09:45:17.950003+0000","last_unstale":"2024-03-23T09:45:18.318632+0000","last_undegraded":"2024-03-23T09:45:18.318632+0000","last_fullsized":"2024-03-23T09:45:18.318632+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:24.280980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389983+0000","last_change":"2024-03-23T09:45:18.329752+0000","last_active":"2024-03-23T09:45:20.389983+0000","last_peered":"2024-03-23T09:45:20.389983+0000","last_clean":"2024-03-23T09:45:20.389983+0000","last_became_active":"2024-03-23T09:45:18.329490+0000","last_became_peered":"2024-03-23T09:45:18.329490+0000","last_unstale":"2024-03-23T09:45:20.389983+0000","last_undegraded":"2024-03-23T09:45:20.389983+0000","last_fullsized":"2024-03-23T09:45:20.389983+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:55.517694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.314931+0000","last_change":"2024-03-23T09:45:17.947667+0000","last_active":"2024-03-23T09:45:18.314931+0000","last_peered":"2024-03-23T09:45:18.314931+0000","last_clean":"2024-03-23T09:45:18.314931+0000","last_became_active":"2024-03-23T09:45:17.947349+0000","last_became_peered":"2024-03-23T09:45:17.947349+0000","last_unstale":"2024-03-23T09:45:18.314931+0000","last_undegraded":"2024-03-23T09:45:18.314931+0000","last_fullsized":"2024-03-23T09:45:18.314931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:30:06.113245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601535+0000","last_change":"2024-03-23T09:45:18.312041+0000","last_active":"2024-03-23T09:45:20.601535+0000","last_peered":"2024-03-23T09:45:20.601535+0000","last_clean":"2024-03-23T09:45:20.601535+0000","last_became_active":"2024-03-23T09:45:18.311764+0000","last_became_peered":"2024-03-23T09:45:18.311764+0000","last_unstale":"2024-03-23T09:45:20.601535+0000","last_undegraded":"2024-03-23T09:45:20.601535+0000","last_fullsized":"2024-03-23T09:45:20.601535+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:53:09.534852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"69'6","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.322294+0000","last_change":"2024-03-23T09:45:17.938237+0000","last_active":"2024-03-23T09:45:18.322294+0000","last_peered":"2024-03-23T09:45:18.322294+0000","last_clean":"2024-03-23T09:45:18.322294+0000","last_became_active":"2024-03-23T09:45:17.937927+0000","last_became_peered":"2024-03-23T09:45:17.937927+0000","last_unstale":"2024-03-23T09:45:18.322294+0000","last_undegraded":"2024-03-23T09:45:18.322294+0000","last_fullsized":"2024-03-23T09:45:18.322294+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:14:57.219705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306425+0000","last_change":"2024-03-23T09:45:18.324722+0000","last_active":"2024-03-23T09:45:19.306425+0000","last_peered":"2024-03-23T09:45:19.306425+0000","last_clean":"2024-03-23T09:45:19.306425+0000","last_became_active":"2024-03-23T09:45:18.324416+0000","last_became_peered":"2024-03-23T09:45:18.324416+0000","last_unstale":"2024-03-23T09:45:19.306425+0000","last_undegraded":"2024-03-23T09:45:19.306425+0000","last_fullsized":"2024-03-23T09:45:19.306425+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:30:08.468124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"69'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.325261+0000","last_change":"2024-03-23T09:45:17.940256+0000","last_active":"2024-03-23T09:45:18.325261+0000","last_peered":"2024-03-23T09:45:18.325261+0000","last_clean":"2024-03-23T09:45:18.325261+0000","last_became_active":"2024-03-23T09:45:17.939971+0000","last_became_peered":"2024-03-23T09:45:17.939971+0000","last_unstale":"2024-03-23T09:45:18.325261+0000","last_undegraded":"2024-03-23T09:45:18.325261+0000","last_fullsized":"2024-03-23T09:45:18.325261+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:18:57.489494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601698+0000","last_change":"2024-03-23T09:45:18.319626+0000","last_active":"2024-03-23T09:45:20.601698+0000","last_peered":"2024-03-23T09:45:20.601698+0000","last_clean":"2024-03-23T09:45:20.601698+0000","last_became_active":"2024-03-23T09:45:18.319350+0000","last_became_peered":"2024-03-23T09:45:18.319350+0000","last_unstale":"2024-03-23T09:45:20.601698+0000","last_undegraded":"2024-03-23T09:45:20.601698+0000","last_fullsized":"2024-03-23T09:45:20.601698+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:32:17.615400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.312268+0000","last_change":"2024-03-23T09:45:17.948009+0000","last_active":"2024-03-23T09:45:18.312268+0000","last_peered":"2024-03-23T09:45:18.312268+0000","last_clean":"2024-03-23T09:45:18.312268+0000","last_became_active":"2024-03-23T09:45:17.947641+0000","last_became_peered":"2024-03-23T09:45:17.947641+0000","last_unstale":"2024-03-23T09:45:18.312268+0000","last_undegraded":"2024-03-23T09:45:18.312268+0000","last_fullsized":"2024-03-23T09:45:18.312268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:30:11.821980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601329+0000","last_change":"2024-03-23T09:45:18.322694+0000","last_active":"2024-03-23T09:45:20.601329+0000","last_peered":"2024-03-23T09:45:20.601329+0000","last_clean":"2024-03-23T09:45:20.601329+0000","last_became_active":"2024-03-23T09:45:18.322404+0000","last_became_peered":"2024-03-23T09:45:18.322404+0000","last_unstale":"2024-03-23T09:45:20.601329+0000","last_undegraded":"2024-03-23T09:45:20.601329+0000","last_fullsized":"2024-03-23T09:45:20.601329+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:32:17.615400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.311518+0000","last_change":"2024-03-23T09:45:17.950505+0000","last_active":"2024-03-23T09:45:18.311518+0000","last_peered":"2024-03-23T09:45:18.311518+0000","last_clean":"2024-03-23T09:45:18.311518+0000","last_became_active":"2024-03-23T09:45:17.949582+0000","last_became_peered":"2024-03-23T09:45:17.949582+0000","last_unstale":"2024-03-23T09:45:18.311518+0000","last_undegraded":"2024-03-23T09:45:18.311518+0000","last_fullsized":"2024-03-23T09:45:18.311518+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:06:34.170725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602033+0000","last_change":"2024-03-23T09:45:18.315915+0000","last_active":"2024-03-23T09:45:20.602033+0000","last_peered":"2024-03-23T09:45:20.602033+0000","last_clean":"2024-03-23T09:45:20.602033+0000","last_became_active":"2024-03-23T09:45:18.315511+0000","last_became_peered":"2024-03-23T09:45:18.315511+0000","last_unstale":"2024-03-23T09:45:20.602033+0000","last_undegraded":"2024-03-23T09:45:20.602033+0000","last_fullsized":"2024-03-23T09:45:20.602033+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:45.876269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.324770+0000","last_change":"2024-03-23T09:45:17.946673+0000","last_active":"2024-03-23T09:45:18.324770+0000","last_peered":"2024-03-23T09:45:18.324770+0000","last_clean":"2024-03-23T09:45:18.324770+0000","last_became_active":"2024-03-23T09:45:17.945942+0000","last_became_peered":"2024-03-23T09:45:17.945942+0000","last_unstale":"2024-03-23T09:45:18.324770+0000","last_undegraded":"2024-03-23T09:45:18.324770+0000","last_fullsized":"2024-03-23T09:45:18.324770+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:04.521715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306190+0000","last_change":"2024-03-23T09:45:18.312814+0000","last_active":"2024-03-23T09:45:19.306190+0000","last_peered":"2024-03-23T09:45:19.306190+0000","last_clean":"2024-03-23T09:45:19.306190+0000","last_became_active":"2024-03-23T09:45:18.312534+0000","last_became_peered":"2024-03-23T09:45:18.312534+0000","last_unstale":"2024-03-23T09:45:19.306190+0000","last_undegraded":"2024-03-23T09:45:19.306190+0000","last_fullsized":"2024-03-23T09:45:19.306190+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:11.305444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.326930+0000","last_change":"2024-03-23T09:45:17.947497+0000","last_active":"2024-03-23T09:45:18.326930+0000","last_peered":"2024-03-23T09:45:18.326930+0000","last_clean":"2024-03-23T09:45:18.326930+0000","last_became_active":"2024-03-23T09:45:17.947042+0000","last_became_peered":"2024-03-23T09:45:17.947042+0000","last_unstale":"2024-03-23T09:45:18.326930+0000","last_undegraded":"2024-03-23T09:45:18.326930+0000","last_fullsized":"2024-03-23T09:45:18.326930+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:18:57.489494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306634+0000","last_change":"2024-03-23T09:45:18.318012+0000","last_active":"2024-03-23T09:45:19.306634+0000","last_peered":"2024-03-23T09:45:19.306634+0000","last_clean":"2024-03-23T09:45:19.306634+0000","last_became_active":"2024-03-23T09:45:18.317245+0000","last_became_peered":"2024-03-23T09:45:18.317245+0000","last_unstale":"2024-03-23T09:45:19.306634+0000","last_undegraded":"2024-03-23T09:45:19.306634+0000","last_fullsized":"2024-03-23T09:45:19.306634+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:22:16.856887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.328531+0000","last_change":"2024-03-23T09:45:17.950950+0000","last_active":"2024-03-23T09:45:18.328531+0000","last_peered":"2024-03-23T09:45:18.328531+0000","last_clean":"2024-03-23T09:45:18.328531+0000","last_became_active":"2024-03-23T09:45:17.950650+0000","last_became_peered":"2024-03-23T09:45:17.950650+0000","last_unstale":"2024-03-23T09:45:18.328531+0000","last_undegraded":"2024-03-23T09:45:18.328531+0000","last_fullsized":"2024-03-23T09:45:18.328531+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:00:16.438765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.304504+0000","last_change":"2024-03-23T09:45:18.309598+0000","last_active":"2024-03-23T09:45:19.304504+0000","last_peered":"2024-03-23T09:45:19.304504+0000","last_clean":"2024-03-23T09:45:19.304504+0000","last_became_active":"2024-03-23T09:45:18.309286+0000","last_became_peered":"2024-03-23T09:45:18.309286+0000","last_unstale":"2024-03-23T09:45:19.304504+0000","last_undegraded":"2024-03-23T09:45:19.304504+0000","last_fullsized":"2024-03-23T09:45:19.304504+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:37:52.741638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.329213+0000","last_change":"2024-03-23T09:45:17.951573+0000","last_active":"2024-03-23T09:45:18.329213+0000","last_peered":"2024-03-23T09:45:18.329213+0000","last_clean":"2024-03-23T09:45:18.329213+0000","last_became_active":"2024-03-23T09:45:17.951220+0000","last_became_peered":"2024-03-23T09:45:17.951220+0000","last_unstale":"2024-03-23T09:45:18.329213+0000","last_undegraded":"2024-03-23T09:45:18.329213+0000","last_fullsized":"2024-03-23T09:45:18.329213+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:14:57.219705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601582+0000","last_change":"2024-03-23T09:45:18.308088+0000","last_active":"2024-03-23T09:45:20.601582+0000","last_peered":"2024-03-23T09:45:20.601582+0000","last_clean":"2024-03-23T09:45:20.601582+0000","last_became_active":"2024-03-23T09:45:18.307829+0000","last_became_peered":"2024-03-23T09:45:18.307829+0000","last_unstale":"2024-03-23T09:45:20.601582+0000","last_undegraded":"2024-03-23T09:45:20.601582+0000","last_fullsized":"2024-03-23T09:45:20.601582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:15:29.666312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.304280+0000","last_change":"2024-03-23T09:45:17.949064+0000","last_active":"2024-03-23T09:45:18.304280+0000","last_peered":"2024-03-23T09:45:18.304280+0000","last_clean":"2024-03-23T09:45:18.304280+0000","last_became_active":"2024-03-23T09:45:17.948780+0000","last_became_peered":"2024-03-23T09:45:17.948780+0000","last_unstale":"2024-03-23T09:45:18.304280+0000","last_undegraded":"2024-03-23T09:45:18.304280+0000","last_fullsized":"2024-03-23T09:45:18.304280+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:44:44.243931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602464+0000","last_change":"2024-03-23T09:45:18.313613+0000","last_active":"2024-03-23T09:45:20.602464+0000","last_peered":"2024-03-23T09:45:20.602464+0000","last_clean":"2024-03-23T09:45:20.602464+0000","last_became_active":"2024-03-23T09:45:18.313379+0000","last_became_peered":"2024-03-23T09:45:18.313379+0000","last_unstale":"2024-03-23T09:45:20.602464+0000","last_undegraded":"2024-03-23T09:45:20.602464+0000","last_fullsized":"2024-03-23T09:45:20.602464+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:39:14.090168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.310809+0000","last_change":"2024-03-23T09:45:17.947913+0000","last_active":"2024-03-23T09:45:18.310809+0000","last_peered":"2024-03-23T09:45:18.310809+0000","last_clean":"2024-03-23T09:45:18.310809+0000","last_became_active":"2024-03-23T09:45:17.947456+0000","last_became_peered":"2024-03-23T09:45:17.947456+0000","last_unstale":"2024-03-23T09:45:18.310809+0000","last_undegraded":"2024-03-23T09:45:18.310809+0000","last_fullsized":"2024-03-23T09:45:18.310809+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:38:12.502514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602093+0000","last_change":"2024-03-23T09:45:18.322902+0000","last_active":"2024-03-23T09:45:20.602093+0000","last_peered":"2024-03-23T09:45:20.602093+0000","last_clean":"2024-03-23T09:45:20.602093+0000","last_became_active":"2024-03-23T09:45:18.322377+0000","last_became_peered":"2024-03-23T09:45:18.322377+0000","last_unstale":"2024-03-23T09:45:20.602093+0000","last_undegraded":"2024-03-23T09:45:20.602093+0000","last_fullsized":"2024-03-23T09:45:20.602093+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:50:22.913033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.311637+0000","last_change":"2024-03-23T09:45:17.948761+0000","last_active":"2024-03-23T09:45:18.311637+0000","last_peered":"2024-03-23T09:45:18.311637+0000","last_clean":"2024-03-23T09:45:18.311637+0000","last_became_active":"2024-03-23T09:45:17.948489+0000","last_became_peered":"2024-03-23T09:45:17.948489+0000","last_unstale":"2024-03-23T09:45:18.311637+0000","last_undegraded":"2024-03-23T09:45:18.311637+0000","last_fullsized":"2024-03-23T09:45:18.311637+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:38:17.881935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602329+0000","last_change":"2024-03-23T09:45:18.321516+0000","last_active":"2024-03-23T09:45:20.602329+0000","last_peered":"2024-03-23T09:45:20.602329+0000","last_clean":"2024-03-23T09:45:20.602329+0000","last_became_active":"2024-03-23T09:45:18.321263+0000","last_became_peered":"2024-03-23T09:45:18.321263+0000","last_unstale":"2024-03-23T09:45:20.602329+0000","last_undegraded":"2024-03-23T09:45:20.602329+0000","last_fullsized":"2024-03-23T09:45:20.602329+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:09.172394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.308310+0000","last_change":"2024-03-23T09:45:17.946167+0000","last_active":"2024-03-23T09:45:18.308310+0000","last_peered":"2024-03-23T09:45:18.308310+0000","last_clean":"2024-03-23T09:45:18.308310+0000","last_became_active":"2024-03-23T09:45:17.945907+0000","last_became_peered":"2024-03-23T09:45:17.945907+0000","last_unstale":"2024-03-23T09:45:18.308310+0000","last_undegraded":"2024-03-23T09:45:18.308310+0000","last_fullsized":"2024-03-23T09:45:18.308310+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:07.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602432+0000","last_change":"2024-03-23T09:45:18.313201+0000","last_active":"2024-03-23T09:45:20.602432+0000","last_peered":"2024-03-23T09:45:20.602432+0000","last_clean":"2024-03-23T09:45:20.602432+0000","last_became_active":"2024-03-23T09:45:18.312952+0000","last_became_peered":"2024-03-23T09:45:18.312952+0000","last_unstale":"2024-03-23T09:45:20.602432+0000","last_undegraded":"2024-03-23T09:45:20.602432+0000","last_fullsized":"2024-03-23T09:45:20.602432+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:27:48.512037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.326845+0000","last_change":"2024-03-23T09:45:17.947308+0000","last_active":"2024-03-23T09:45:18.326845+0000","last_peered":"2024-03-23T09:45:18.326845+0000","last_clean":"2024-03-23T09:45:18.326845+0000","last_became_active":"2024-03-23T09:45:17.947033+0000","last_became_peered":"2024-03-23T09:45:17.947033+0000","last_unstale":"2024-03-23T09:45:18.326845+0000","last_undegraded":"2024-03-23T09:45:18.326845+0000","last_fullsized":"2024-03-23T09:45:18.326845+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:03:17.222905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601884+0000","last_change":"2024-03-23T09:45:18.317454+0000","last_active":"2024-03-23T09:45:20.601884+0000","last_peered":"2024-03-23T09:45:20.601884+0000","last_clean":"2024-03-23T09:45:20.601884+0000","last_became_active":"2024-03-23T09:45:18.317198+0000","last_became_peered":"2024-03-23T09:45:18.317198+0000","last_unstale":"2024-03-23T09:45:20.601884+0000","last_undegraded":"2024-03-23T09:45:20.601884+0000","last_fullsized":"2024-03-23T09:45:20.601884+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:13:13.309261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.312550+0000","last_change":"2024-03-23T09:45:17.948850+0000","last_active":"2024-03-23T09:45:18.312550+0000","last_peered":"2024-03-23T09:45:18.312550+0000","last_clean":"2024-03-23T09:45:18.312550+0000","last_became_active":"2024-03-23T09:45:17.948594+0000","last_became_peered":"2024-03-23T09:45:17.948594+0000","last_unstale":"2024-03-23T09:45:18.312550+0000","last_undegraded":"2024-03-23T09:45:18.312550+0000","last_fullsized":"2024-03-23T09:45:18.312550+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:41:11.998454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305756+0000","last_change":"2024-03-23T09:45:18.322694+0000","last_active":"2024-03-23T09:45:19.305756+0000","last_peered":"2024-03-23T09:45:19.305756+0000","last_clean":"2024-03-23T09:45:19.305756+0000","last_became_active":"2024-03-23T09:45:18.322464+0000","last_became_peered":"2024-03-23T09:45:18.322464+0000","last_unstale":"2024-03-23T09:45:19.305756+0000","last_undegraded":"2024-03-23T09:45:19.305756+0000","last_fullsized":"2024-03-23T09:45:19.305756+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:49:08.104714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.300731+0000","last_change":"2024-03-23T09:45:17.946875+0000","last_active":"2024-03-23T09:45:18.300731+0000","last_peered":"2024-03-23T09:45:18.300731+0000","last_clean":"2024-03-23T09:45:18.300731+0000","last_became_active":"2024-03-23T09:45:17.946588+0000","last_became_peered":"2024-03-23T09:45:17.946588+0000","last_unstale":"2024-03-23T09:45:18.300731+0000","last_undegraded":"2024-03-23T09:45:18.300731+0000","last_fullsized":"2024-03-23T09:45:18.300731+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:42:34.167069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305914+0000","last_change":"2024-03-23T09:45:18.323720+0000","last_active":"2024-03-23T09:45:19.305914+0000","last_peered":"2024-03-23T09:45:19.305914+0000","last_clean":"2024-03-23T09:45:19.305914+0000","last_became_active":"2024-03-23T09:45:18.323465+0000","last_became_peered":"2024-03-23T09:45:18.323465+0000","last_unstale":"2024-03-23T09:45:19.305914+0000","last_undegraded":"2024-03-23T09:45:19.305914+0000","last_fullsized":"2024-03-23T09:45:19.305914+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:49:04.736990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.308655+0000","last_change":"2024-03-23T09:45:17.949762+0000","last_active":"2024-03-23T09:45:18.308655+0000","last_peered":"2024-03-23T09:45:18.308655+0000","last_clean":"2024-03-23T09:45:18.308655+0000","last_became_active":"2024-03-23T09:45:17.949466+0000","last_became_peered":"2024-03-23T09:45:17.949466+0000","last_unstale":"2024-03-23T09:45:18.308655+0000","last_undegraded":"2024-03-23T09:45:18.308655+0000","last_fullsized":"2024-03-23T09:45:18.308655+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:40:34.327822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601834+0000","last_change":"2024-03-23T09:45:18.316330+0000","last_active":"2024-03-23T09:45:20.601834+0000","last_peered":"2024-03-23T09:45:20.601834+0000","last_clean":"2024-03-23T09:45:20.601834+0000","last_became_active":"2024-03-23T09:45:18.316103+0000","last_became_peered":"2024-03-23T09:45:18.316103+0000","last_unstale":"2024-03-23T09:45:20.601834+0000","last_undegraded":"2024-03-23T09:45:20.601834+0000","last_fullsized":"2024-03-23T09:45:20.601834+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:38:13.506664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.300449+0000","last_change":"2024-03-23T09:45:17.950763+0000","last_active":"2024-03-23T09:45:18.300449+0000","last_peered":"2024-03-23T09:45:18.300449+0000","last_clean":"2024-03-23T09:45:18.300449+0000","last_became_active":"2024-03-23T09:45:17.950457+0000","last_became_peered":"2024-03-23T09:45:17.950457+0000","last_unstale":"2024-03-23T09:45:18.300449+0000","last_undegraded":"2024-03-23T09:45:18.300449+0000","last_fullsized":"2024-03-23T09:45:18.300449+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:53:03.362600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389725+0000","last_change":"2024-03-23T09:45:18.326030+0000","last_active":"2024-03-23T09:45:20.389725+0000","last_peered":"2024-03-23T09:45:20.389725+0000","last_clean":"2024-03-23T09:45:20.389725+0000","last_became_active":"2024-03-23T09:45:18.325710+0000","last_became_peered":"2024-03-23T09:45:18.325710+0000","last_unstale":"2024-03-23T09:45:20.389725+0000","last_undegraded":"2024-03-23T09:45:20.389725+0000","last_fullsized":"2024-03-23T09:45:20.389725+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:05:38.755296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.303347+0000","last_change":"2024-03-23T09:45:17.945578+0000","last_active":"2024-03-23T09:45:18.303347+0000","last_peered":"2024-03-23T09:45:18.303347+0000","last_clean":"2024-03-23T09:45:18.303347+0000","last_became_active":"2024-03-23T09:45:17.945288+0000","last_became_peered":"2024-03-23T09:45:17.945288+0000","last_unstale":"2024-03-23T09:45:18.303347+0000","last_undegraded":"2024-03-23T09:45:18.303347+0000","last_fullsized":"2024-03-23T09:45:18.303347+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:11:42.583654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.304179+0000","last_change":"2024-03-23T09:45:18.309444+0000","last_active":"2024-03-23T09:45:19.304179+0000","last_peered":"2024-03-23T09:45:19.304179+0000","last_clean":"2024-03-23T09:45:19.304179+0000","last_became_active":"2024-03-23T09:45:18.309182+0000","last_became_peered":"2024-03-23T09:45:18.309182+0000","last_unstale":"2024-03-23T09:45:19.304179+0000","last_undegraded":"2024-03-23T09:45:19.304179+0000","last_fullsized":"2024-03-23T09:45:19.304179+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:19.131977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.305287+0000","last_change":"2024-03-23T09:45:17.948306+0000","last_active":"2024-03-23T09:45:18.305287+0000","last_peered":"2024-03-23T09:45:18.305287+0000","last_clean":"2024-03-23T09:45:18.305287+0000","last_became_active":"2024-03-23T09:45:17.948042+0000","last_became_peered":"2024-03-23T09:45:17.948042+0000","last_unstale":"2024-03-23T09:45:18.305287+0000","last_undegraded":"2024-03-23T09:45:18.305287+0000","last_fullsized":"2024-03-23T09:45:18.305287+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:53:08.530702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389527+0000","last_change":"2024-03-23T09:45:18.317449+0000","last_active":"2024-03-23T09:45:20.389527+0000","last_peered":"2024-03-23T09:45:20.389527+0000","last_clean":"2024-03-23T09:45:20.389527+0000","last_became_active":"2024-03-23T09:45:18.316688+0000","last_became_peered":"2024-03-23T09:45:18.316688+0000","last_unstale":"2024-03-23T09:45:20.389527+0000","last_undegraded":"2024-03-23T09:45:20.389527+0000","last_fullsized":"2024-03-23T09:45:20.389527+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:08.055427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.312227+0000","last_change":"2024-03-23T09:45:17.946349+0000","last_active":"2024-03-23T09:45:18.312227+0000","last_peered":"2024-03-23T09:45:18.312227+0000","last_clean":"2024-03-23T09:45:18.312227+0000","last_became_active":"2024-03-23T09:45:17.945310+0000","last_became_peered":"2024-03-23T09:45:17.945310+0000","last_unstale":"2024-03-23T09:45:18.312227+0000","last_undegraded":"2024-03-23T09:45:18.312227+0000","last_fullsized":"2024-03-23T09:45:18.312227+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:11:42.583654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602147+0000","last_change":"2024-03-23T09:45:18.327523+0000","last_active":"2024-03-23T09:45:20.602147+0000","last_peered":"2024-03-23T09:45:20.602147+0000","last_clean":"2024-03-23T09:45:20.602147+0000","last_became_active":"2024-03-23T09:45:18.327369+0000","last_became_peered":"2024-03-23T09:45:18.327369+0000","last_unstale":"2024-03-23T09:45:20.602147+0000","last_undegraded":"2024-03-23T09:45:20.602147+0000","last_fullsized":"2024-03-23T09:45:20.602147+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:03:50.754016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.320459+0000","last_change":"2024-03-23T09:45:17.947893+0000","last_active":"2024-03-23T09:45:18.320459+0000","last_peered":"2024-03-23T09:45:18.320459+0000","last_clean":"2024-03-23T09:45:18.320459+0000","last_became_active":"2024-03-23T09:45:17.947615+0000","last_became_peered":"2024-03-23T09:45:17.947615+0000","last_unstale":"2024-03-23T09:45:18.320459+0000","last_undegraded":"2024-03-23T09:45:18.320459+0000","last_fullsized":"2024-03-23T09:45:18.320459+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:04.521715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":215,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.305504+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:45:18.305504+0000","last_peered":"2024-03-23T09:45:18.305504+0000","last_clean":"2024-03-23T09:45:18.305504+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:45:18.305504+0000","last_undegraded":"2024-03-23T09:45:18.305504+0000","last_fullsized":"2024-03-23T09:45:18.305504+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1454080,"data_stored":946928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53100000000000003,"15min":0.48999999999999999},"min":{"1min":0.35699999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72199999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54500000000000004,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65500000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54000000000000004,"15min":0.51200000000000001},"min":{"1min":0.42199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.751,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.439,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78500000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55100000000000005}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.44400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60899999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.48399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.84699999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82699999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.497,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.622,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.33200000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65200000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35899999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.38300000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.46300000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56699999999999995,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":0.874,"15min":0.874},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.379,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56699999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":355417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.371,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.38900000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.5}]},{"osd":1,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53000000000000003,"15min":0.503},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.71699999999999997,"5min":0.751,"15min":0.751},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68200000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.628}]},{"osd":2,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65600000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.41699999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.71499999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.443}]},{"osd":3,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.45600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.45300000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.70599999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.48599999999999999,"15min":0.504},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.626,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.52700000000000002,"15min":0.54700000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.69799999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.70799999999999996}]},{"osd":5,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.50700000000000001,"15min":0.53000000000000003},"min":{"1min":0.36199999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54400000000000004,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53900000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.65800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":356034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.51600000000000001,"15min":0.48199999999999998},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70799999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.432},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55800000000000005,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70699999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.44600000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52700000000000002,"15min":0.49199999999999999},"min":{"1min":0.41999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.504},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.65000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.504}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57599999999999996}]},{"osd":3,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53800000000000003,"15min":0.504},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56699999999999995,"15min":0.53400000000000003},"min":{"1min":0.42699999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.69499999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.53600000000000003}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45900000000000002,"15min":0.46999999999999997},"min":{"1min":0.29599999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67900000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.34799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71999999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.60099999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.498,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.627,"5min":0.749,"15min":0.749},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60299999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.47599999999999998}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.746,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56399999999999995,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.61499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":356014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.437,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.747,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.34899999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.502,"15min":0.47399999999999998},"min":{"1min":0.36899999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.63100000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51700000000000002,"15min":0.504},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.73099999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.46700000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.53800000000000003},"min":{"1min":0.50700000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75800000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.79900000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.53700000000000003}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68400000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.40500000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.71799999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.44700000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46300000000000002,"15min":0.46899999999999997},"min":{"1min":0.32600000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67900000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.48899999999999999,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.66000000000000003,"5min":1.008,"15min":1.008},"last":0.52600000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.307,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60999999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.67200000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61199999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52500000000000002,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.496},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.371,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.68799999999999994,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56200000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":860160,"data_stored":356056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.34100000000000003,"15min":0.316},"min":{"1min":0.28100000000000003,"5min":0.183,"15min":0.183},"max":{"1min":0.627,"5min":1.45,"15min":1.45},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.315},"min":{"1min":0.23100000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.51900000000000002,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.26800000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33400000000000002,"15min":0.312},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54300000000000004,"5min":1.323,"15min":1.323},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.36799999999999999,"15min":0.34899999999999998},"min":{"1min":0.315,"5min":0.218,"15min":0.218},"max":{"1min":0.67800000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.36199999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.38100000000000001},"min":{"1min":0.309,"5min":0.251,"15min":0.251},"max":{"1min":0.61599999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.42999999999999999,"15min":0.41199999999999998},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70899999999999996,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.749,"5min":1.337,"15min":1.337},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.54500000000000004},"min":{"1min":0.437,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":2.758,"15min":2.758},"last":0.434}]},{"osd":5,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77500000000000002,"5min":1.276,"15min":1.276},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86499999999999999,"5min":1.895,"15min":1.895},"last":0.54700000000000004}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.62,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70699999999999996,"5min":5.4539999999999997,"15min":5.4539999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83399999999999996,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.47499999999999998}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.60899999999999999},"min":{"1min":0.45500000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89300000000000002,"5min":3.7400000000000002,"15min":3.7400000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65300000000000002,"15min":0.61899999999999999},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.44400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":864256,"data_stored":357508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.249,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.314},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36799999999999999,"15min":0.378},"min":{"1min":0.26200000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52200000000000002,"5min":0.872,"15min":0.872},"last":0.30099999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.28499999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.629,"15min":0.629},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.39000000000000001},"min":{"1min":0.24299999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58699999999999997,"5min":0.746,"15min":0.746},"last":0.34899999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.34100000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61399999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.27800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.55000000000000004,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.37}]},{"osd":4,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.44900000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84699999999999998,"5min":1.216,"15min":1.216},"last":0.442},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85499999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.626,"15min":0.63300000000000001},"min":{"1min":0.51200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83999999999999997,"5min":1.101,"15min":1.101},"last":0.61499999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.53100000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89100000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63900000000000001,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89500000000000002,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.54500000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.56899999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.872,"5min":1.155,"15min":1.155},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64400000000000002,"15min":0.65400000000000003},"min":{"1min":0.46999999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82099999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.497}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":872448,"data_stored":358124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34799999999999998,"15min":0.33700000000000002},"min":{"1min":0.247,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.5,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42599999999999999,"15min":0.47799999999999998},"min":{"1min":0.23400000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.71999999999999997,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.38700000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.34499999999999997},"min":{"1min":0.251,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39000000000000001,"15min":0.375},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57399999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.307}]},{"osd":3,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42199999999999999,"15min":0.40999999999999998},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63700000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.45100000000000001,"15min":0.50700000000000001},"min":{"1min":0.29199999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.61799999999999999,"5min":2.75,"15min":2.75},"last":0.35599999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.71499999999999997}]},{"osd":5,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.70399999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94199999999999995,"5min":2.778,"15min":2.778},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89000000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68999999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66500000000000004,"15min":0.70899999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65200000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89200000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.75900000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1441792,"data_stored":945693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33400000000000002,"15min":0.32300000000000001},"min":{"1min":0.21299999999999999,"5min":0.184,"15min":0.184},"max":{"1min":0.51300000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.35099999999999998},"min":{"1min":0.23999999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.25900000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.38100000000000001,"15min":0.36699999999999999},"min":{"1min":0.28899999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55400000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.36099999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55200000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.318},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.31900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.40000000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.48999999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.53200000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53700000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.44600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70799999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.504,"5min":0.379,"15min":0.379},"max":{"1min":0.875,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.45700000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82399999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.433},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.64400000000000002},"min":{"1min":0.54200000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89400000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57699999999999996}]},{"osd":7,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.51000000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85799999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.501,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80700000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:45:25.749 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:45:25.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.863+0000 7f0491bfb640 1 Processor -- start 2024-03-23T09:45:25.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.863+0000 7f0491bfb640 1 -- start start 2024-03-23T09:45:25.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.863+0000 7f0491bfb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f048c160250 0x7f048c0cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:25.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.863+0000 7f0491bfb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c06bb30 0x7f048c06bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:25.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.863+0000 7f0491bfb640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048c0cea50 con 0x7f048c098a00 2024-03-23T09:45:25.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.863+0000 7f0491bfb640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048c0cf890 con 0x7f048c160250 2024-03-23T09:45:25.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.863+0000 7f0491bfb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0d0260 con 0x7f048c06bb30 2024-03-23T09:45:25.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.864+0000 7f048b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f048c160250 0x7f048c0cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:25.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.864+0000 7f048bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f048c098a00 0x7f048c0caca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41280/0 (socket says 172.21.15.47:41280) 2024-03-23T09:45:25.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.864+0000 7f048bfff640 1 -- 172.21.15.47:0/1145563476 learned_addr learned my addr 172.21.15.47:0/1145563476 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:25.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.864+0000 7f0482ffd640 1 --2- 172.21.15.47:0/1145563476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c06bb30 0x7f048c06bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:25.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.864+0000 7f048b7fe640 1 -- 172.21.15.47:0/1145563476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c06bb30 msgr2=0x7f048c06bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:25.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.864+0000 7f048b7fe640 1 --2- 172.21.15.47:0/1145563476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c06bb30 0x7f048c06bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:25.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.864+0000 7f048b7fe640 1 -- 172.21.15.47:0/1145563476 >> v1:172.21.15.47:6789/0 conn(0x7f048c098a00 legacy=0x7f048c0caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:25.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.864+0000 7f048b7fe640 1 -- 172.21.15.47:0/1145563476 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c0cec80 con 0x7f048c160250 2024-03-23T09:45:25.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.865+0000 7f048b7fe640 1 --2- 172.21.15.47:0/1145563476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f048c160250 0x7f048c0cfd20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f047c0097a0 tx=0x7f047c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3328a160947e0f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:25.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.865+0000 7f048affd640 1 -- 172.21.15.47:0/1145563476 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c031830 con 0x7f048c160250 2024-03-23T09:45:25.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.865+0000 7f048affd640 1 -- 172.21.15.47:0/1145563476 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c031990 con 0x7f048c160250 2024-03-23T09:45:25.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.865+0000 7f048affd640 1 -- 172.21.15.47:0/1145563476 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c031d20 con 0x7f048c160250 2024-03-23T09:45:25.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.865+0000 7f0491bfb640 1 -- 172.21.15.47:0/1145563476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f048c160250 msgr2=0x7f048c0cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:25.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.865+0000 7f0491bfb640 1 --2- 172.21.15.47:0/1145563476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f048c160250 0x7f048c0cfd20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f047c0097a0 tx=0x7f047c02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:45:25.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.865+0000 7f0491bfb640 1 -- 172.21.15.47:0/1145563476 shutdown_connections 2024-03-23T09:45:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.865+0000 7f0491bfb640 1 --2- 172.21.15.47:0/1145563476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c06bb30 0x7f048c06bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.865+0000 7f0491bfb640 1 --2- 172.21.15.47:0/1145563476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f048c160250 0x7f048c0cfd20 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.865+0000 7f0491bfb640 1 -- 172.21.15.47:0/1145563476 >> 172.21.15.47:0/1145563476 conn(0x7f048c0b2140 msgr2=0x7f048c0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.865+0000 7f0491bfb640 1 -- 172.21.15.47:0/1145563476 shutdown_connections 2024-03-23T09:45:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.866+0000 7f0491bfb640 1 -- 172.21.15.47:0/1145563476 wait complete. 2024-03-23T09:45:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.867+0000 7f0491bfb640 1 Processor -- start 2024-03-23T09:45:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.867+0000 7f0491bfb640 1 -- start start 2024-03-23T09:45:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0491bfb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f048c06bb30 0x7f048c1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0491bfb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f048c098a00 0x7f048c0b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0491bfb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c160250 0x7f048c0b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0491bfb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c1616d0 con 0x7f048c098a00 2024-03-23T09:45:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0491bfb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c15dbf0 con 0x7f048c06bb30 2024-03-23T09:45:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0491bfb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c15e690 con 0x7f048c160250 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0482ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f048c098a00 0x7f048c0b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0482ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f048c098a00 0x7f048c0b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35910/0 (socket says 172.21.15.47:35910) 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0482ffd640 1 -- 172.21.15.47:0/2241462655 learned_addr learned my addr 172.21.15.47:0/2241462655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f048b7fe640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f048c06bb30 0x7f048c1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f048bfff640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c160250 0x7f048c0b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0482ffd640 1 -- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c160250 msgr2=0x7f048c0b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0482ffd640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c160250 0x7f048c0b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0482ffd640 1 -- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f048c06bb30 msgr2=0x7f048c1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0482ffd640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f048c06bb30 0x7f048c1579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0482ffd640 1 -- 172.21.15.47:0/2241462655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c009400 con 0x7f048c098a00 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0482ffd640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f048c098a00 0x7f048c0b35a0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f047400d800 tx=0x7f047400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0488ff9640 1 -- 172.21.15.47:0/2241462655 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047400b6c0 con 0x7f048c098a00 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0488ff9640 1 -- 172.21.15.47:0/2241462655 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047400b820 con 0x7f048c098a00 2024-03-23T09:45:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.868+0000 7f0491bfb640 1 -- 172.21.15.47:0/2241462655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c1613f0 con 0x7f048c098a00 2024-03-23T09:45:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.869+0000 7f0488ff9640 1 -- 172.21.15.47:0/2241462655 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047400bbb0 con 0x7f048c098a00 2024-03-23T09:45:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.869+0000 7f0491bfb640 1 -- 172.21.15.47:0/2241462655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c15e170 con 0x7f048c098a00 2024-03-23T09:45:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.870+0000 7f0491bfb640 1 -- 172.21.15.47:0/2241462655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c15e330 con 0x7f048c098a00 2024-03-23T09:45:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.870+0000 7f0488ff9640 1 -- 172.21.15.47:0/2241462655 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f047400bd10 con 0x7f048c098a00 2024-03-23T09:45:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.870+0000 7f0488ff9640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f046c06ac00 0x7f046c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.871+0000 7f0488ff9640 1 -- 172.21.15.47:0/2241462655 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04740898b0 con 0x7f048c098a00 2024-03-23T09:45:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.871+0000 7f048b7fe640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f046c06ac00 0x7f046c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.871+0000 7f048b7fe640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f046c06ac00 0x7f046c06d0c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f047c036000 tx=0x7f047c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:25.875+0000 7f0488ff9640 1 -- 172.21.15.47:0/2241462655 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0474054a90 con 0x7f048c098a00 2024-03-23T09:45:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.054+0000 7f0491bfb640 1 -- 172.21.15.47:0/2241462655 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f048c15da70 con 0x7f046c06ac00 2024-03-23T09:45:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.062+0000 7f0488ff9640 1 -- 172.21.15.47:0/2241462655 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289420 (secure 0 0 0) 0x7f047c044400 con 0x7f046c06ac00 2024-03-23T09:45:26.064 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:45:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.065+0000 7f0491bfb640 1 -- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f046c06ac00 msgr2=0x7f046c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.065+0000 7f0491bfb640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f046c06ac00 0x7f046c06d0c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f047c036000 tx=0x7f047c03c040 comp rx=0 tx=0).stop 2024-03-23T09:45:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.065+0000 7f0491bfb640 1 -- 172.21.15.47:0/2241462655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f048c098a00 msgr2=0x7f048c0b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.065+0000 7f0491bfb640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f048c098a00 0x7f048c0b35a0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f047400d800 tx=0x7f047400dcc0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.065+0000 7f0491bfb640 1 -- 172.21.15.47:0/2241462655 shutdown_connections 2024-03-23T09:45:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.065+0000 7f0491bfb640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f046c06ac00 0x7f046c06d0c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.065+0000 7f0491bfb640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c160250 0x7f048c0b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.065+0000 7f0491bfb640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f048c098a00 0x7f048c0b35a0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.065+0000 7f0491bfb640 1 --2- 172.21.15.47:0/2241462655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f048c06bb30 0x7f048c1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.065+0000 7f0491bfb640 1 -- 172.21.15.47:0/2241462655 >> 172.21.15.47:0/2241462655 conn(0x7f048c0b2140 msgr2=0x7f048c06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.065+0000 7f0491bfb640 1 -- 172.21.15.47:0/2241462655 shutdown_connections 2024-03-23T09:45:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.065+0000 7f0491bfb640 1 -- 172.21.15.47:0/2241462655 wait complete. 2024-03-23T09:45:26.075 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-03-23T09:45:24.401740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244536,"kb_used_data":7872,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754730184,"statfs":{"total":773094113280,"available":772843708416,"internally_reserved":0,"allocated":8060928,"data_stored":4031774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":242208312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.472839"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306532+0000","last_change":"2024-03-23T09:45:18.324776+0000","last_active":"2024-03-23T09:45:19.306532+0000","last_peered":"2024-03-23T09:45:19.306532+0000","last_clean":"2024-03-23T09:45:19.306532+0000","last_became_active":"2024-03-23T09:45:18.324369+0000","last_became_peered":"2024-03-23T09:45:18.324369+0000","last_unstale":"2024-03-23T09:45:19.306532+0000","last_undegraded":"2024-03-23T09:45:19.306532+0000","last_fullsized":"2024-03-23T09:45:19.306532+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:27:48.512037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.390006+0000","last_change":"2024-03-23T09:45:18.326597+0000","last_active":"2024-03-23T09:45:20.390006+0000","last_peered":"2024-03-23T09:45:20.390006+0000","last_clean":"2024-03-23T09:45:20.390006+0000","last_became_active":"2024-03-23T09:45:18.326253+0000","last_became_peered":"2024-03-23T09:45:18.326253+0000","last_unstale":"2024-03-23T09:45:20.390006+0000","last_undegraded":"2024-03-23T09:45:20.390006+0000","last_fullsized":"2024-03-23T09:45:20.390006+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:55:13.005801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602228+0000","last_change":"2024-03-23T09:45:18.311402+0000","last_active":"2024-03-23T09:45:20.602228+0000","last_peered":"2024-03-23T09:45:20.602228+0000","last_clean":"2024-03-23T09:45:20.602228+0000","last_became_active":"2024-03-23T09:45:18.311152+0000","last_became_peered":"2024-03-23T09:45:18.311152+0000","last_unstale":"2024-03-23T09:45:20.602228+0000","last_undegraded":"2024-03-23T09:45:20.602228+0000","last_fullsized":"2024-03-23T09:45:20.602228+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:59:26.151585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.304413+0000","last_change":"2024-03-23T09:45:18.322785+0000","last_active":"2024-03-23T09:45:19.304413+0000","last_peered":"2024-03-23T09:45:19.304413+0000","last_clean":"2024-03-23T09:45:19.304413+0000","last_became_active":"2024-03-23T09:45:18.322511+0000","last_became_peered":"2024-03-23T09:45:18.322511+0000","last_unstale":"2024-03-23T09:45:19.304413+0000","last_undegraded":"2024-03-23T09:45:19.304413+0000","last_fullsized":"2024-03-23T09:45:19.304413+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:27:54.349415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601942+0000","last_change":"2024-03-23T09:45:18.310827+0000","last_active":"2024-03-23T09:45:20.601942+0000","last_peered":"2024-03-23T09:45:20.601942+0000","last_clean":"2024-03-23T09:45:20.601942+0000","last_became_active":"2024-03-23T09:45:18.310576+0000","last_became_peered":"2024-03-23T09:45:18.310576+0000","last_unstale":"2024-03-23T09:45:20.601942+0000","last_undegraded":"2024-03-23T09:45:20.601942+0000","last_fullsized":"2024-03-23T09:45:20.601942+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:37:52.741638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389635+0000","last_change":"2024-03-23T09:45:18.326944+0000","last_active":"2024-03-23T09:45:20.389635+0000","last_peered":"2024-03-23T09:45:20.389635+0000","last_clean":"2024-03-23T09:45:20.389635+0000","last_became_active":"2024-03-23T09:45:18.326663+0000","last_became_peered":"2024-03-23T09:45:18.326663+0000","last_unstale":"2024-03-23T09:45:20.389635+0000","last_undegraded":"2024-03-23T09:45:20.389635+0000","last_fullsized":"2024-03-23T09:45:20.389635+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:08:57.748599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601573+0000","last_change":"2024-03-23T09:45:18.316836+0000","last_active":"2024-03-23T09:45:20.601573+0000","last_peered":"2024-03-23T09:45:20.601573+0000","last_clean":"2024-03-23T09:45:20.601573+0000","last_became_active":"2024-03-23T09:45:18.316481+0000","last_became_peered":"2024-03-23T09:45:18.316481+0000","last_unstale":"2024-03-23T09:45:20.601573+0000","last_undegraded":"2024-03-23T09:45:20.601573+0000","last_fullsized":"2024-03-23T09:45:20.601573+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:37.045371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389520+0000","last_change":"2024-03-23T09:45:18.321349+0000","last_active":"2024-03-23T09:45:20.389520+0000","last_peered":"2024-03-23T09:45:20.389520+0000","last_clean":"2024-03-23T09:45:20.389520+0000","last_became_active":"2024-03-23T09:45:18.320774+0000","last_became_peered":"2024-03-23T09:45:18.320774+0000","last_unstale":"2024-03-23T09:45:20.389520+0000","last_undegraded":"2024-03-23T09:45:20.389520+0000","last_fullsized":"2024-03-23T09:45:20.389520+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:11.305444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602033+0000","last_change":"2024-03-23T09:45:18.317806+0000","last_active":"2024-03-23T09:45:20.602033+0000","last_peered":"2024-03-23T09:45:20.602033+0000","last_clean":"2024-03-23T09:45:20.602033+0000","last_became_active":"2024-03-23T09:45:18.317576+0000","last_became_peered":"2024-03-23T09:45:18.317576+0000","last_unstale":"2024-03-23T09:45:20.602033+0000","last_undegraded":"2024-03-23T09:45:20.602033+0000","last_fullsized":"2024-03-23T09:45:20.602033+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:21.291320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306083+0000","last_change":"2024-03-23T09:45:18.317249+0000","last_active":"2024-03-23T09:45:19.306083+0000","last_peered":"2024-03-23T09:45:19.306083+0000","last_clean":"2024-03-23T09:45:19.306083+0000","last_became_active":"2024-03-23T09:45:18.316969+0000","last_became_peered":"2024-03-23T09:45:18.316969+0000","last_unstale":"2024-03-23T09:45:19.306083+0000","last_undegraded":"2024-03-23T09:45:19.306083+0000","last_fullsized":"2024-03-23T09:45:19.306083+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:05:38.755296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305697+0000","last_change":"2024-03-23T09:45:18.321400+0000","last_active":"2024-03-23T09:45:19.305697+0000","last_peered":"2024-03-23T09:45:19.305697+0000","last_clean":"2024-03-23T09:45:19.305697+0000","last_became_active":"2024-03-23T09:45:18.320735+0000","last_became_peered":"2024-03-23T09:45:18.320735+0000","last_unstale":"2024-03-23T09:45:19.305697+0000","last_undegraded":"2024-03-23T09:45:19.305697+0000","last_fullsized":"2024-03-23T09:45:19.305697+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:34:05.217690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.304573+0000","last_change":"2024-03-23T09:45:18.310359+0000","last_active":"2024-03-23T09:45:19.304573+0000","last_peered":"2024-03-23T09:45:19.304573+0000","last_clean":"2024-03-23T09:45:19.304573+0000","last_became_active":"2024-03-23T09:45:18.310100+0000","last_became_peered":"2024-03-23T09:45:18.310100+0000","last_unstale":"2024-03-23T09:45:19.304573+0000","last_undegraded":"2024-03-23T09:45:19.304573+0000","last_fullsized":"2024-03-23T09:45:19.304573+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:21.291320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601579+0000","last_change":"2024-03-23T09:45:18.326738+0000","last_active":"2024-03-23T09:45:20.601579+0000","last_peered":"2024-03-23T09:45:20.601579+0000","last_clean":"2024-03-23T09:45:20.601579+0000","last_became_active":"2024-03-23T09:45:18.326424+0000","last_became_peered":"2024-03-23T09:45:18.326424+0000","last_unstale":"2024-03-23T09:45:20.601579+0000","last_undegraded":"2024-03-23T09:45:20.601579+0000","last_fullsized":"2024-03-23T09:45:20.601579+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:42:53.609522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601739+0000","last_change":"2024-03-23T09:45:18.311761+0000","last_active":"2024-03-23T09:45:20.601739+0000","last_peered":"2024-03-23T09:45:20.601739+0000","last_clean":"2024-03-23T09:45:20.601739+0000","last_became_active":"2024-03-23T09:45:18.311498+0000","last_became_peered":"2024-03-23T09:45:18.311498+0000","last_unstale":"2024-03-23T09:45:20.601739+0000","last_undegraded":"2024-03-23T09:45:20.601739+0000","last_fullsized":"2024-03-23T09:45:20.601739+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:27:54.349415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601507+0000","last_change":"2024-03-23T09:45:18.316586+0000","last_active":"2024-03-23T09:45:20.601507+0000","last_peered":"2024-03-23T09:45:20.601507+0000","last_clean":"2024-03-23T09:45:20.601507+0000","last_became_active":"2024-03-23T09:45:18.316308+0000","last_became_peered":"2024-03-23T09:45:18.316308+0000","last_unstale":"2024-03-23T09:45:20.601507+0000","last_undegraded":"2024-03-23T09:45:20.601507+0000","last_fullsized":"2024-03-23T09:45:20.601507+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:22:16.856887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306488+0000","last_change":"2024-03-23T09:45:18.323303+0000","last_active":"2024-03-23T09:45:19.306488+0000","last_peered":"2024-03-23T09:45:19.306488+0000","last_clean":"2024-03-23T09:45:19.306488+0000","last_became_active":"2024-03-23T09:45:18.323013+0000","last_became_peered":"2024-03-23T09:45:18.323013+0000","last_unstale":"2024-03-23T09:45:19.306488+0000","last_undegraded":"2024-03-23T09:45:19.306488+0000","last_fullsized":"2024-03-23T09:45:19.306488+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:55:13.005801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602120+0000","last_change":"2024-03-23T09:45:18.317725+0000","last_active":"2024-03-23T09:45:20.602120+0000","last_peered":"2024-03-23T09:45:20.602120+0000","last_clean":"2024-03-23T09:45:20.602120+0000","last_became_active":"2024-03-23T09:45:18.317465+0000","last_became_peered":"2024-03-23T09:45:18.317465+0000","last_unstale":"2024-03-23T09:45:20.602120+0000","last_undegraded":"2024-03-23T09:45:20.602120+0000","last_fullsized":"2024-03-23T09:45:20.602120+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:25.285130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.304601+0000","last_change":"2024-03-23T09:45:18.326765+0000","last_active":"2024-03-23T09:45:19.304601+0000","last_peered":"2024-03-23T09:45:19.304601+0000","last_clean":"2024-03-23T09:45:19.304601+0000","last_became_active":"2024-03-23T09:45:18.326429+0000","last_became_peered":"2024-03-23T09:45:18.326429+0000","last_unstale":"2024-03-23T09:45:19.304601+0000","last_undegraded":"2024-03-23T09:45:19.304601+0000","last_fullsized":"2024-03-23T09:45:19.304601+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:15:29.666312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389951+0000","last_change":"2024-03-23T09:45:18.320301+0000","last_active":"2024-03-23T09:45:20.389951+0000","last_peered":"2024-03-23T09:45:20.389951+0000","last_clean":"2024-03-23T09:45:20.389951+0000","last_became_active":"2024-03-23T09:45:18.319899+0000","last_became_peered":"2024-03-23T09:45:18.319899+0000","last_unstale":"2024-03-23T09:45:20.389951+0000","last_undegraded":"2024-03-23T09:45:20.389951+0000","last_fullsized":"2024-03-23T09:45:20.389951+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:40:35.331972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601750+0000","last_change":"2024-03-23T09:45:18.311110+0000","last_active":"2024-03-23T09:45:20.601750+0000","last_peered":"2024-03-23T09:45:20.601750+0000","last_clean":"2024-03-23T09:45:20.601750+0000","last_became_active":"2024-03-23T09:45:18.310830+0000","last_became_peered":"2024-03-23T09:45:18.310830+0000","last_unstale":"2024-03-23T09:45:20.601750+0000","last_undegraded":"2024-03-23T09:45:20.601750+0000","last_fullsized":"2024-03-23T09:45:20.601750+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:29:21.273818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306257+0000","last_change":"2024-03-23T09:45:18.312000+0000","last_active":"2024-03-23T09:45:19.306257+0000","last_peered":"2024-03-23T09:45:19.306257+0000","last_clean":"2024-03-23T09:45:19.306257+0000","last_became_active":"2024-03-23T09:45:18.311691+0000","last_became_peered":"2024-03-23T09:45:18.311691+0000","last_unstale":"2024-03-23T09:45:19.306257+0000","last_undegraded":"2024-03-23T09:45:19.306257+0000","last_fullsized":"2024-03-23T09:45:19.306257+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:40:35.331972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305779+0000","last_change":"2024-03-23T09:45:18.316673+0000","last_active":"2024-03-23T09:45:19.305779+0000","last_peered":"2024-03-23T09:45:19.305779+0000","last_clean":"2024-03-23T09:45:19.305779+0000","last_became_active":"2024-03-23T09:45:18.316397+0000","last_became_peered":"2024-03-23T09:45:18.316397+0000","last_unstale":"2024-03-23T09:45:19.305779+0000","last_undegraded":"2024-03-23T09:45:19.305779+0000","last_fullsized":"2024-03-23T09:45:19.305779+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:55.517694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305564+0000","last_change":"2024-03-23T09:45:18.320457+0000","last_active":"2024-03-23T09:45:19.305564+0000","last_peered":"2024-03-23T09:45:19.305564+0000","last_clean":"2024-03-23T09:45:19.305564+0000","last_became_active":"2024-03-23T09:45:18.317940+0000","last_became_peered":"2024-03-23T09:45:18.317940+0000","last_unstale":"2024-03-23T09:45:19.305564+0000","last_undegraded":"2024-03-23T09:45:19.305564+0000","last_fullsized":"2024-03-23T09:45:19.305564+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:47:30.691595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305985+0000","last_change":"2024-03-23T09:45:18.323849+0000","last_active":"2024-03-23T09:45:19.305985+0000","last_peered":"2024-03-23T09:45:19.305985+0000","last_clean":"2024-03-23T09:45:19.305985+0000","last_became_active":"2024-03-23T09:45:18.323595+0000","last_became_peered":"2024-03-23T09:45:18.323595+0000","last_unstale":"2024-03-23T09:45:19.305985+0000","last_undegraded":"2024-03-23T09:45:19.305985+0000","last_fullsized":"2024-03-23T09:45:19.305985+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:55.873572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306018+0000","last_change":"2024-03-23T09:45:18.318830+0000","last_active":"2024-03-23T09:45:19.306018+0000","last_peered":"2024-03-23T09:45:19.306018+0000","last_clean":"2024-03-23T09:45:19.306018+0000","last_became_active":"2024-03-23T09:45:18.318575+0000","last_became_peered":"2024-03-23T09:45:18.318575+0000","last_unstale":"2024-03-23T09:45:19.306018+0000","last_undegraded":"2024-03-23T09:45:19.306018+0000","last_fullsized":"2024-03-23T09:45:19.306018+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:37:52.579596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601855+0000","last_change":"2024-03-23T09:45:18.327740+0000","last_active":"2024-03-23T09:45:20.601855+0000","last_peered":"2024-03-23T09:45:20.601855+0000","last_clean":"2024-03-23T09:45:20.601855+0000","last_became_active":"2024-03-23T09:45:18.327469+0000","last_became_peered":"2024-03-23T09:45:18.327469+0000","last_unstale":"2024-03-23T09:45:20.601855+0000","last_undegraded":"2024-03-23T09:45:20.601855+0000","last_fullsized":"2024-03-23T09:45:20.601855+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:22:16.856887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602441+0000","last_change":"2024-03-23T09:45:18.309404+0000","last_active":"2024-03-23T09:45:20.602441+0000","last_peered":"2024-03-23T09:45:20.602441+0000","last_clean":"2024-03-23T09:45:20.602441+0000","last_became_active":"2024-03-23T09:45:18.309119+0000","last_became_peered":"2024-03-23T09:45:18.309119+0000","last_unstale":"2024-03-23T09:45:20.602441+0000","last_undegraded":"2024-03-23T09:45:20.602441+0000","last_fullsized":"2024-03-23T09:45:20.602441+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:51:13.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306624+0000","last_change":"2024-03-23T09:45:18.318114+0000","last_active":"2024-03-23T09:45:19.306624+0000","last_peered":"2024-03-23T09:45:19.306624+0000","last_clean":"2024-03-23T09:45:19.306624+0000","last_became_active":"2024-03-23T09:45:18.317822+0000","last_became_peered":"2024-03-23T09:45:18.317822+0000","last_unstale":"2024-03-23T09:45:19.306624+0000","last_undegraded":"2024-03-23T09:45:19.306624+0000","last_fullsized":"2024-03-23T09:45:19.306624+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:08.055427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602263+0000","last_change":"2024-03-23T09:45:18.314678+0000","last_active":"2024-03-23T09:45:20.602263+0000","last_peered":"2024-03-23T09:45:20.602263+0000","last_clean":"2024-03-23T09:45:20.602263+0000","last_became_active":"2024-03-23T09:45:18.314408+0000","last_became_peered":"2024-03-23T09:45:18.314408+0000","last_unstale":"2024-03-23T09:45:20.602263+0000","last_undegraded":"2024-03-23T09:45:20.602263+0000","last_fullsized":"2024-03-23T09:45:20.602263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:37:16.370619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601633+0000","last_change":"2024-03-23T09:45:18.327731+0000","last_active":"2024-03-23T09:45:20.601633+0000","last_peered":"2024-03-23T09:45:20.601633+0000","last_clean":"2024-03-23T09:45:20.601633+0000","last_became_active":"2024-03-23T09:45:18.327458+0000","last_became_peered":"2024-03-23T09:45:18.327458+0000","last_unstale":"2024-03-23T09:45:20.601633+0000","last_undegraded":"2024-03-23T09:45:20.601633+0000","last_fullsized":"2024-03-23T09:45:20.601633+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:42:53.609522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601702+0000","last_change":"2024-03-23T09:45:18.319723+0000","last_active":"2024-03-23T09:45:20.601702+0000","last_peered":"2024-03-23T09:45:20.601702+0000","last_clean":"2024-03-23T09:45:20.601702+0000","last_became_active":"2024-03-23T09:45:18.319451+0000","last_became_peered":"2024-03-23T09:45:18.319451+0000","last_unstale":"2024-03-23T09:45:20.601702+0000","last_undegraded":"2024-03-23T09:45:20.601702+0000","last_fullsized":"2024-03-23T09:45:20.601702+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:29:21.273818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389575+0000","last_change":"2024-03-23T09:45:18.321163+0000","last_active":"2024-03-23T09:45:20.389575+0000","last_peered":"2024-03-23T09:45:20.389575+0000","last_clean":"2024-03-23T09:45:20.389575+0000","last_became_active":"2024-03-23T09:45:18.320511+0000","last_became_peered":"2024-03-23T09:45:18.320511+0000","last_unstale":"2024-03-23T09:45:20.389575+0000","last_undegraded":"2024-03-23T09:45:20.389575+0000","last_fullsized":"2024-03-23T09:45:20.389575+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:44:45.248082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306385+0000","last_change":"2024-03-23T09:45:18.327116+0000","last_active":"2024-03-23T09:45:19.306385+0000","last_peered":"2024-03-23T09:45:19.306385+0000","last_clean":"2024-03-23T09:45:19.306385+0000","last_became_active":"2024-03-23T09:45:18.326826+0000","last_became_peered":"2024-03-23T09:45:18.326826+0000","last_unstale":"2024-03-23T09:45:19.306385+0000","last_undegraded":"2024-03-23T09:45:19.306385+0000","last_fullsized":"2024-03-23T09:45:19.306385+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:18:21.419203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.323005+0000","last_change":"2024-03-23T09:45:17.939332+0000","last_active":"2024-03-23T09:45:18.323005+0000","last_peered":"2024-03-23T09:45:18.323005+0000","last_clean":"2024-03-23T09:45:18.323005+0000","last_became_active":"2024-03-23T09:45:17.938957+0000","last_became_peered":"2024-03-23T09:45:17.938957+0000","last_unstale":"2024-03-23T09:45:18.323005+0000","last_undegraded":"2024-03-23T09:45:18.323005+0000","last_fullsized":"2024-03-23T09:45:18.323005+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:07.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602200+0000","last_change":"2024-03-23T09:45:18.316911+0000","last_active":"2024-03-23T09:45:20.602200+0000","last_peered":"2024-03-23T09:45:20.602200+0000","last_clean":"2024-03-23T09:45:20.602200+0000","last_became_active":"2024-03-23T09:45:18.316661+0000","last_became_peered":"2024-03-23T09:45:18.316661+0000","last_unstale":"2024-03-23T09:45:20.602200+0000","last_undegraded":"2024-03-23T09:45:20.602200+0000","last_fullsized":"2024-03-23T09:45:20.602200+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:18:58.493644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.315818+0000","last_change":"2024-03-23T09:45:17.945601+0000","last_active":"2024-03-23T09:45:18.315818+0000","last_peered":"2024-03-23T09:45:18.315818+0000","last_clean":"2024-03-23T09:45:18.315818+0000","last_became_active":"2024-03-23T09:45:17.945281+0000","last_became_peered":"2024-03-23T09:45:17.945281+0000","last_unstale":"2024-03-23T09:45:18.315818+0000","last_undegraded":"2024-03-23T09:45:18.315818+0000","last_fullsized":"2024-03-23T09:45:18.315818+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:10.301294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389687+0000","last_change":"2024-03-23T09:45:18.319672+0000","last_active":"2024-03-23T09:45:20.389687+0000","last_peered":"2024-03-23T09:45:20.389687+0000","last_clean":"2024-03-23T09:45:20.389687+0000","last_became_active":"2024-03-23T09:45:18.319343+0000","last_became_peered":"2024-03-23T09:45:18.319343+0000","last_unstale":"2024-03-23T09:45:20.389687+0000","last_undegraded":"2024-03-23T09:45:20.389687+0000","last_fullsized":"2024-03-23T09:45:20.389687+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:37:52.579596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.327582+0000","last_change":"2024-03-23T09:45:17.950331+0000","last_active":"2024-03-23T09:45:18.327582+0000","last_peered":"2024-03-23T09:45:18.327582+0000","last_clean":"2024-03-23T09:45:18.327582+0000","last_became_active":"2024-03-23T09:45:17.950030+0000","last_became_peered":"2024-03-23T09:45:17.950030+0000","last_unstale":"2024-03-23T09:45:18.327582+0000","last_undegraded":"2024-03-23T09:45:18.327582+0000","last_fullsized":"2024-03-23T09:45:18.327582+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:18.127827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305805+0000","last_change":"2024-03-23T09:45:18.314183+0000","last_active":"2024-03-23T09:45:19.305805+0000","last_peered":"2024-03-23T09:45:19.305805+0000","last_clean":"2024-03-23T09:45:19.305805+0000","last_became_active":"2024-03-23T09:45:18.313797+0000","last_became_peered":"2024-03-23T09:45:18.313797+0000","last_unstale":"2024-03-23T09:45:19.305805+0000","last_undegraded":"2024-03-23T09:45:19.305805+0000","last_fullsized":"2024-03-23T09:45:19.305805+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:48:09.676506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.319132+0000","last_change":"2024-03-23T09:45:17.947339+0000","last_active":"2024-03-23T09:45:18.319132+0000","last_peered":"2024-03-23T09:45:18.319132+0000","last_clean":"2024-03-23T09:45:18.319132+0000","last_became_active":"2024-03-23T09:45:17.947071+0000","last_became_peered":"2024-03-23T09:45:17.947071+0000","last_unstale":"2024-03-23T09:45:18.319132+0000","last_undegraded":"2024-03-23T09:45:18.319132+0000","last_fullsized":"2024-03-23T09:45:18.319132+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:28:14.722915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601704+0000","last_change":"2024-03-23T09:45:18.315901+0000","last_active":"2024-03-23T09:45:20.601704+0000","last_peered":"2024-03-23T09:45:20.601704+0000","last_clean":"2024-03-23T09:45:20.601704+0000","last_became_active":"2024-03-23T09:45:18.315635+0000","last_became_peered":"2024-03-23T09:45:18.315635+0000","last_unstale":"2024-03-23T09:45:20.601704+0000","last_undegraded":"2024-03-23T09:45:20.601704+0000","last_fullsized":"2024-03-23T09:45:20.601704+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:55.517694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.320302+0000","last_change":"2024-03-23T09:45:17.946905+0000","last_active":"2024-03-23T09:45:18.320302+0000","last_peered":"2024-03-23T09:45:18.320302+0000","last_clean":"2024-03-23T09:45:18.320302+0000","last_became_active":"2024-03-23T09:45:17.946592+0000","last_became_peered":"2024-03-23T09:45:17.946592+0000","last_unstale":"2024-03-23T09:45:18.320302+0000","last_undegraded":"2024-03-23T09:45:18.320302+0000","last_fullsized":"2024-03-23T09:45:18.320302+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:18:20.415053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306286+0000","last_change":"2024-03-23T09:45:18.323933+0000","last_active":"2024-03-23T09:45:19.306286+0000","last_peered":"2024-03-23T09:45:19.306286+0000","last_clean":"2024-03-23T09:45:19.306286+0000","last_became_active":"2024-03-23T09:45:18.323614+0000","last_became_peered":"2024-03-23T09:45:18.323614+0000","last_unstale":"2024-03-23T09:45:19.306286+0000","last_undegraded":"2024-03-23T09:45:19.306286+0000","last_fullsized":"2024-03-23T09:45:19.306286+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:08:57.748599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.326348+0000","last_change":"2024-03-23T09:45:17.948070+0000","last_active":"2024-03-23T09:45:18.326348+0000","last_peered":"2024-03-23T09:45:18.326348+0000","last_clean":"2024-03-23T09:45:18.326348+0000","last_became_active":"2024-03-23T09:45:17.947757+0000","last_became_peered":"2024-03-23T09:45:17.947757+0000","last_unstale":"2024-03-23T09:45:18.326348+0000","last_undegraded":"2024-03-23T09:45:18.326348+0000","last_fullsized":"2024-03-23T09:45:18.326348+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:53:08.530702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601897+0000","last_change":"2024-03-23T09:45:18.307469+0000","last_active":"2024-03-23T09:45:20.601897+0000","last_peered":"2024-03-23T09:45:20.601897+0000","last_clean":"2024-03-23T09:45:20.601897+0000","last_became_active":"2024-03-23T09:45:18.307196+0000","last_became_peered":"2024-03-23T09:45:18.307196+0000","last_unstale":"2024-03-23T09:45:20.601897+0000","last_undegraded":"2024-03-23T09:45:20.601897+0000","last_fullsized":"2024-03-23T09:45:20.601897+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:26:14.593410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.303235+0000","last_change":"2024-03-23T09:45:17.947891+0000","last_active":"2024-03-23T09:45:18.303235+0000","last_peered":"2024-03-23T09:45:18.303235+0000","last_clean":"2024-03-23T09:45:18.303235+0000","last_became_active":"2024-03-23T09:45:17.947523+0000","last_became_peered":"2024-03-23T09:45:17.947523+0000","last_unstale":"2024-03-23T09:45:18.303235+0000","last_undegraded":"2024-03-23T09:45:18.303235+0000","last_fullsized":"2024-03-23T09:45:18.303235+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:44.872119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601604+0000","last_change":"2024-03-23T09:45:18.324291+0000","last_active":"2024-03-23T09:45:20.601604+0000","last_peered":"2024-03-23T09:45:20.601604+0000","last_clean":"2024-03-23T09:45:20.601604+0000","last_became_active":"2024-03-23T09:45:18.324031+0000","last_became_peered":"2024-03-23T09:45:18.324031+0000","last_unstale":"2024-03-23T09:45:20.601604+0000","last_undegraded":"2024-03-23T09:45:20.601604+0000","last_fullsized":"2024-03-23T09:45:20.601604+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:37.045371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.300181+0000","last_change":"2024-03-23T09:45:17.935722+0000","last_active":"2024-03-23T09:45:18.300181+0000","last_peered":"2024-03-23T09:45:18.300181+0000","last_clean":"2024-03-23T09:45:18.300181+0000","last_became_active":"2024-03-23T09:45:17.935483+0000","last_became_peered":"2024-03-23T09:45:17.935483+0000","last_unstale":"2024-03-23T09:45:18.300181+0000","last_undegraded":"2024-03-23T09:45:18.300181+0000","last_fullsized":"2024-03-23T09:45:18.300181+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:00:47.619314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601918+0000","last_change":"2024-03-23T09:45:18.325513+0000","last_active":"2024-03-23T09:45:20.601918+0000","last_peered":"2024-03-23T09:45:20.601918+0000","last_clean":"2024-03-23T09:45:20.601918+0000","last_became_active":"2024-03-23T09:45:18.325126+0000","last_became_peered":"2024-03-23T09:45:18.325126+0000","last_unstale":"2024-03-23T09:45:20.601918+0000","last_undegraded":"2024-03-23T09:45:20.601918+0000","last_fullsized":"2024-03-23T09:45:20.601918+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:11.381182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.305364+0000","last_change":"2024-03-23T09:45:17.945994+0000","last_active":"2024-03-23T09:45:18.305364+0000","last_peered":"2024-03-23T09:45:18.305364+0000","last_clean":"2024-03-23T09:45:18.305364+0000","last_became_active":"2024-03-23T09:45:17.945734+0000","last_became_peered":"2024-03-23T09:45:17.945734+0000","last_unstale":"2024-03-23T09:45:18.305364+0000","last_undegraded":"2024-03-23T09:45:18.305364+0000","last_fullsized":"2024-03-23T09:45:18.305364+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:24.280980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306788+0000","last_change":"2024-03-23T09:45:18.327993+0000","last_active":"2024-03-23T09:45:19.306788+0000","last_peered":"2024-03-23T09:45:19.306788+0000","last_clean":"2024-03-23T09:45:19.306788+0000","last_became_active":"2024-03-23T09:45:18.327747+0000","last_became_peered":"2024-03-23T09:45:18.327747+0000","last_unstale":"2024-03-23T09:45:19.306788+0000","last_undegraded":"2024-03-23T09:45:19.306788+0000","last_fullsized":"2024-03-23T09:45:19.306788+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:30:07.117395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.318632+0000","last_change":"2024-03-23T09:45:17.950258+0000","last_active":"2024-03-23T09:45:18.318632+0000","last_peered":"2024-03-23T09:45:18.318632+0000","last_clean":"2024-03-23T09:45:18.318632+0000","last_became_active":"2024-03-23T09:45:17.950003+0000","last_became_peered":"2024-03-23T09:45:17.950003+0000","last_unstale":"2024-03-23T09:45:18.318632+0000","last_undegraded":"2024-03-23T09:45:18.318632+0000","last_fullsized":"2024-03-23T09:45:18.318632+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:12:24.280980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389983+0000","last_change":"2024-03-23T09:45:18.329752+0000","last_active":"2024-03-23T09:45:20.389983+0000","last_peered":"2024-03-23T09:45:20.389983+0000","last_clean":"2024-03-23T09:45:20.389983+0000","last_became_active":"2024-03-23T09:45:18.329490+0000","last_became_peered":"2024-03-23T09:45:18.329490+0000","last_unstale":"2024-03-23T09:45:20.389983+0000","last_undegraded":"2024-03-23T09:45:20.389983+0000","last_fullsized":"2024-03-23T09:45:20.389983+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:55.517694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.314931+0000","last_change":"2024-03-23T09:45:17.947667+0000","last_active":"2024-03-23T09:45:18.314931+0000","last_peered":"2024-03-23T09:45:18.314931+0000","last_clean":"2024-03-23T09:45:18.314931+0000","last_became_active":"2024-03-23T09:45:17.947349+0000","last_became_peered":"2024-03-23T09:45:17.947349+0000","last_unstale":"2024-03-23T09:45:18.314931+0000","last_undegraded":"2024-03-23T09:45:18.314931+0000","last_fullsized":"2024-03-23T09:45:18.314931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:30:06.113245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601535+0000","last_change":"2024-03-23T09:45:18.312041+0000","last_active":"2024-03-23T09:45:20.601535+0000","last_peered":"2024-03-23T09:45:20.601535+0000","last_clean":"2024-03-23T09:45:20.601535+0000","last_became_active":"2024-03-23T09:45:18.311764+0000","last_became_peered":"2024-03-23T09:45:18.311764+0000","last_unstale":"2024-03-23T09:45:20.601535+0000","last_undegraded":"2024-03-23T09:45:20.601535+0000","last_fullsized":"2024-03-23T09:45:20.601535+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:53:09.534852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"69'6","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.322294+0000","last_change":"2024-03-23T09:45:17.938237+0000","last_active":"2024-03-23T09:45:18.322294+0000","last_peered":"2024-03-23T09:45:18.322294+0000","last_clean":"2024-03-23T09:45:18.322294+0000","last_became_active":"2024-03-23T09:45:17.937927+0000","last_became_peered":"2024-03-23T09:45:17.937927+0000","last_unstale":"2024-03-23T09:45:18.322294+0000","last_undegraded":"2024-03-23T09:45:18.322294+0000","last_fullsized":"2024-03-23T09:45:18.322294+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:14:57.219705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306425+0000","last_change":"2024-03-23T09:45:18.324722+0000","last_active":"2024-03-23T09:45:19.306425+0000","last_peered":"2024-03-23T09:45:19.306425+0000","last_clean":"2024-03-23T09:45:19.306425+0000","last_became_active":"2024-03-23T09:45:18.324416+0000","last_became_peered":"2024-03-23T09:45:18.324416+0000","last_unstale":"2024-03-23T09:45:19.306425+0000","last_undegraded":"2024-03-23T09:45:19.306425+0000","last_fullsized":"2024-03-23T09:45:19.306425+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:30:08.468124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"69'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.325261+0000","last_change":"2024-03-23T09:45:17.940256+0000","last_active":"2024-03-23T09:45:18.325261+0000","last_peered":"2024-03-23T09:45:18.325261+0000","last_clean":"2024-03-23T09:45:18.325261+0000","last_became_active":"2024-03-23T09:45:17.939971+0000","last_became_peered":"2024-03-23T09:45:17.939971+0000","last_unstale":"2024-03-23T09:45:18.325261+0000","last_undegraded":"2024-03-23T09:45:18.325261+0000","last_fullsized":"2024-03-23T09:45:18.325261+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:18:57.489494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601698+0000","last_change":"2024-03-23T09:45:18.319626+0000","last_active":"2024-03-23T09:45:20.601698+0000","last_peered":"2024-03-23T09:45:20.601698+0000","last_clean":"2024-03-23T09:45:20.601698+0000","last_became_active":"2024-03-23T09:45:18.319350+0000","last_became_peered":"2024-03-23T09:45:18.319350+0000","last_unstale":"2024-03-23T09:45:20.601698+0000","last_undegraded":"2024-03-23T09:45:20.601698+0000","last_fullsized":"2024-03-23T09:45:20.601698+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:32:17.615400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.312268+0000","last_change":"2024-03-23T09:45:17.948009+0000","last_active":"2024-03-23T09:45:18.312268+0000","last_peered":"2024-03-23T09:45:18.312268+0000","last_clean":"2024-03-23T09:45:18.312268+0000","last_became_active":"2024-03-23T09:45:17.947641+0000","last_became_peered":"2024-03-23T09:45:17.947641+0000","last_unstale":"2024-03-23T09:45:18.312268+0000","last_undegraded":"2024-03-23T09:45:18.312268+0000","last_fullsized":"2024-03-23T09:45:18.312268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:30:11.821980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601329+0000","last_change":"2024-03-23T09:45:18.322694+0000","last_active":"2024-03-23T09:45:20.601329+0000","last_peered":"2024-03-23T09:45:20.601329+0000","last_clean":"2024-03-23T09:45:20.601329+0000","last_became_active":"2024-03-23T09:45:18.322404+0000","last_became_peered":"2024-03-23T09:45:18.322404+0000","last_unstale":"2024-03-23T09:45:20.601329+0000","last_undegraded":"2024-03-23T09:45:20.601329+0000","last_fullsized":"2024-03-23T09:45:20.601329+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:32:17.615400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.311518+0000","last_change":"2024-03-23T09:45:17.950505+0000","last_active":"2024-03-23T09:45:18.311518+0000","last_peered":"2024-03-23T09:45:18.311518+0000","last_clean":"2024-03-23T09:45:18.311518+0000","last_became_active":"2024-03-23T09:45:17.949582+0000","last_became_peered":"2024-03-23T09:45:17.949582+0000","last_unstale":"2024-03-23T09:45:18.311518+0000","last_undegraded":"2024-03-23T09:45:18.311518+0000","last_fullsized":"2024-03-23T09:45:18.311518+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:06:34.170725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602033+0000","last_change":"2024-03-23T09:45:18.315915+0000","last_active":"2024-03-23T09:45:20.602033+0000","last_peered":"2024-03-23T09:45:20.602033+0000","last_clean":"2024-03-23T09:45:20.602033+0000","last_became_active":"2024-03-23T09:45:18.315511+0000","last_became_peered":"2024-03-23T09:45:18.315511+0000","last_unstale":"2024-03-23T09:45:20.602033+0000","last_undegraded":"2024-03-23T09:45:20.602033+0000","last_fullsized":"2024-03-23T09:45:20.602033+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:45.876269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.324770+0000","last_change":"2024-03-23T09:45:17.946673+0000","last_active":"2024-03-23T09:45:18.324770+0000","last_peered":"2024-03-23T09:45:18.324770+0000","last_clean":"2024-03-23T09:45:18.324770+0000","last_became_active":"2024-03-23T09:45:17.945942+0000","last_became_peered":"2024-03-23T09:45:17.945942+0000","last_unstale":"2024-03-23T09:45:18.324770+0000","last_undegraded":"2024-03-23T09:45:18.324770+0000","last_fullsized":"2024-03-23T09:45:18.324770+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:04.521715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306190+0000","last_change":"2024-03-23T09:45:18.312814+0000","last_active":"2024-03-23T09:45:19.306190+0000","last_peered":"2024-03-23T09:45:19.306190+0000","last_clean":"2024-03-23T09:45:19.306190+0000","last_became_active":"2024-03-23T09:45:18.312534+0000","last_became_peered":"2024-03-23T09:45:18.312534+0000","last_unstale":"2024-03-23T09:45:19.306190+0000","last_undegraded":"2024-03-23T09:45:19.306190+0000","last_fullsized":"2024-03-23T09:45:19.306190+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:11.305444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.326930+0000","last_change":"2024-03-23T09:45:17.947497+0000","last_active":"2024-03-23T09:45:18.326930+0000","last_peered":"2024-03-23T09:45:18.326930+0000","last_clean":"2024-03-23T09:45:18.326930+0000","last_became_active":"2024-03-23T09:45:17.947042+0000","last_became_peered":"2024-03-23T09:45:17.947042+0000","last_unstale":"2024-03-23T09:45:18.326930+0000","last_undegraded":"2024-03-23T09:45:18.326930+0000","last_fullsized":"2024-03-23T09:45:18.326930+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:18:57.489494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.306634+0000","last_change":"2024-03-23T09:45:18.318012+0000","last_active":"2024-03-23T09:45:19.306634+0000","last_peered":"2024-03-23T09:45:19.306634+0000","last_clean":"2024-03-23T09:45:19.306634+0000","last_became_active":"2024-03-23T09:45:18.317245+0000","last_became_peered":"2024-03-23T09:45:18.317245+0000","last_unstale":"2024-03-23T09:45:19.306634+0000","last_undegraded":"2024-03-23T09:45:19.306634+0000","last_fullsized":"2024-03-23T09:45:19.306634+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:22:16.856887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.328531+0000","last_change":"2024-03-23T09:45:17.950950+0000","last_active":"2024-03-23T09:45:18.328531+0000","last_peered":"2024-03-23T09:45:18.328531+0000","last_clean":"2024-03-23T09:45:18.328531+0000","last_became_active":"2024-03-23T09:45:17.950650+0000","last_became_peered":"2024-03-23T09:45:17.950650+0000","last_unstale":"2024-03-23T09:45:18.328531+0000","last_undegraded":"2024-03-23T09:45:18.328531+0000","last_fullsized":"2024-03-23T09:45:18.328531+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:00:16.438765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.304504+0000","last_change":"2024-03-23T09:45:18.309598+0000","last_active":"2024-03-23T09:45:19.304504+0000","last_peered":"2024-03-23T09:45:19.304504+0000","last_clean":"2024-03-23T09:45:19.304504+0000","last_became_active":"2024-03-23T09:45:18.309286+0000","last_became_peered":"2024-03-23T09:45:18.309286+0000","last_unstale":"2024-03-23T09:45:19.304504+0000","last_undegraded":"2024-03-23T09:45:19.304504+0000","last_fullsized":"2024-03-23T09:45:19.304504+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:37:52.741638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.329213+0000","last_change":"2024-03-23T09:45:17.951573+0000","last_active":"2024-03-23T09:45:18.329213+0000","last_peered":"2024-03-23T09:45:18.329213+0000","last_clean":"2024-03-23T09:45:18.329213+0000","last_became_active":"2024-03-23T09:45:17.951220+0000","last_became_peered":"2024-03-23T09:45:17.951220+0000","last_unstale":"2024-03-23T09:45:18.329213+0000","last_undegraded":"2024-03-23T09:45:18.329213+0000","last_fullsized":"2024-03-23T09:45:18.329213+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:14:57.219705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601582+0000","last_change":"2024-03-23T09:45:18.308088+0000","last_active":"2024-03-23T09:45:20.601582+0000","last_peered":"2024-03-23T09:45:20.601582+0000","last_clean":"2024-03-23T09:45:20.601582+0000","last_became_active":"2024-03-23T09:45:18.307829+0000","last_became_peered":"2024-03-23T09:45:18.307829+0000","last_unstale":"2024-03-23T09:45:20.601582+0000","last_undegraded":"2024-03-23T09:45:20.601582+0000","last_fullsized":"2024-03-23T09:45:20.601582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:15:29.666312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.304280+0000","last_change":"2024-03-23T09:45:17.949064+0000","last_active":"2024-03-23T09:45:18.304280+0000","last_peered":"2024-03-23T09:45:18.304280+0000","last_clean":"2024-03-23T09:45:18.304280+0000","last_became_active":"2024-03-23T09:45:17.948780+0000","last_became_peered":"2024-03-23T09:45:17.948780+0000","last_unstale":"2024-03-23T09:45:18.304280+0000","last_undegraded":"2024-03-23T09:45:18.304280+0000","last_fullsized":"2024-03-23T09:45:18.304280+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:44:44.243931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602464+0000","last_change":"2024-03-23T09:45:18.313613+0000","last_active":"2024-03-23T09:45:20.602464+0000","last_peered":"2024-03-23T09:45:20.602464+0000","last_clean":"2024-03-23T09:45:20.602464+0000","last_became_active":"2024-03-23T09:45:18.313379+0000","last_became_peered":"2024-03-23T09:45:18.313379+0000","last_unstale":"2024-03-23T09:45:20.602464+0000","last_undegraded":"2024-03-23T09:45:20.602464+0000","last_fullsized":"2024-03-23T09:45:20.602464+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:39:14.090168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.310809+0000","last_change":"2024-03-23T09:45:17.947913+0000","last_active":"2024-03-23T09:45:18.310809+0000","last_peered":"2024-03-23T09:45:18.310809+0000","last_clean":"2024-03-23T09:45:18.310809+0000","last_became_active":"2024-03-23T09:45:17.947456+0000","last_became_peered":"2024-03-23T09:45:17.947456+0000","last_unstale":"2024-03-23T09:45:18.310809+0000","last_undegraded":"2024-03-23T09:45:18.310809+0000","last_fullsized":"2024-03-23T09:45:18.310809+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:38:12.502514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602093+0000","last_change":"2024-03-23T09:45:18.322902+0000","last_active":"2024-03-23T09:45:20.602093+0000","last_peered":"2024-03-23T09:45:20.602093+0000","last_clean":"2024-03-23T09:45:20.602093+0000","last_became_active":"2024-03-23T09:45:18.322377+0000","last_became_peered":"2024-03-23T09:45:18.322377+0000","last_unstale":"2024-03-23T09:45:20.602093+0000","last_undegraded":"2024-03-23T09:45:20.602093+0000","last_fullsized":"2024-03-23T09:45:20.602093+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:50:22.913033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.311637+0000","last_change":"2024-03-23T09:45:17.948761+0000","last_active":"2024-03-23T09:45:18.311637+0000","last_peered":"2024-03-23T09:45:18.311637+0000","last_clean":"2024-03-23T09:45:18.311637+0000","last_became_active":"2024-03-23T09:45:17.948489+0000","last_became_peered":"2024-03-23T09:45:17.948489+0000","last_unstale":"2024-03-23T09:45:18.311637+0000","last_undegraded":"2024-03-23T09:45:18.311637+0000","last_fullsized":"2024-03-23T09:45:18.311637+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:38:17.881935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602329+0000","last_change":"2024-03-23T09:45:18.321516+0000","last_active":"2024-03-23T09:45:20.602329+0000","last_peered":"2024-03-23T09:45:20.602329+0000","last_clean":"2024-03-23T09:45:20.602329+0000","last_became_active":"2024-03-23T09:45:18.321263+0000","last_became_peered":"2024-03-23T09:45:18.321263+0000","last_unstale":"2024-03-23T09:45:20.602329+0000","last_undegraded":"2024-03-23T09:45:20.602329+0000","last_fullsized":"2024-03-23T09:45:20.602329+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:09.172394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.308310+0000","last_change":"2024-03-23T09:45:17.946167+0000","last_active":"2024-03-23T09:45:18.308310+0000","last_peered":"2024-03-23T09:45:18.308310+0000","last_clean":"2024-03-23T09:45:18.308310+0000","last_became_active":"2024-03-23T09:45:17.945907+0000","last_became_peered":"2024-03-23T09:45:17.945907+0000","last_unstale":"2024-03-23T09:45:18.308310+0000","last_undegraded":"2024-03-23T09:45:18.308310+0000","last_fullsized":"2024-03-23T09:45:18.308310+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:07.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602432+0000","last_change":"2024-03-23T09:45:18.313201+0000","last_active":"2024-03-23T09:45:20.602432+0000","last_peered":"2024-03-23T09:45:20.602432+0000","last_clean":"2024-03-23T09:45:20.602432+0000","last_became_active":"2024-03-23T09:45:18.312952+0000","last_became_peered":"2024-03-23T09:45:18.312952+0000","last_unstale":"2024-03-23T09:45:20.602432+0000","last_undegraded":"2024-03-23T09:45:20.602432+0000","last_fullsized":"2024-03-23T09:45:20.602432+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:27:48.512037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.326845+0000","last_change":"2024-03-23T09:45:17.947308+0000","last_active":"2024-03-23T09:45:18.326845+0000","last_peered":"2024-03-23T09:45:18.326845+0000","last_clean":"2024-03-23T09:45:18.326845+0000","last_became_active":"2024-03-23T09:45:17.947033+0000","last_became_peered":"2024-03-23T09:45:17.947033+0000","last_unstale":"2024-03-23T09:45:18.326845+0000","last_undegraded":"2024-03-23T09:45:18.326845+0000","last_fullsized":"2024-03-23T09:45:18.326845+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:03:17.222905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601884+0000","last_change":"2024-03-23T09:45:18.317454+0000","last_active":"2024-03-23T09:45:20.601884+0000","last_peered":"2024-03-23T09:45:20.601884+0000","last_clean":"2024-03-23T09:45:20.601884+0000","last_became_active":"2024-03-23T09:45:18.317198+0000","last_became_peered":"2024-03-23T09:45:18.317198+0000","last_unstale":"2024-03-23T09:45:20.601884+0000","last_undegraded":"2024-03-23T09:45:20.601884+0000","last_fullsized":"2024-03-23T09:45:20.601884+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:13:13.309261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.312550+0000","last_change":"2024-03-23T09:45:17.948850+0000","last_active":"2024-03-23T09:45:18.312550+0000","last_peered":"2024-03-23T09:45:18.312550+0000","last_clean":"2024-03-23T09:45:18.312550+0000","last_became_active":"2024-03-23T09:45:17.948594+0000","last_became_peered":"2024-03-23T09:45:17.948594+0000","last_unstale":"2024-03-23T09:45:18.312550+0000","last_undegraded":"2024-03-23T09:45:18.312550+0000","last_fullsized":"2024-03-23T09:45:18.312550+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:41:11.998454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305756+0000","last_change":"2024-03-23T09:45:18.322694+0000","last_active":"2024-03-23T09:45:19.305756+0000","last_peered":"2024-03-23T09:45:19.305756+0000","last_clean":"2024-03-23T09:45:19.305756+0000","last_became_active":"2024-03-23T09:45:18.322464+0000","last_became_peered":"2024-03-23T09:45:18.322464+0000","last_unstale":"2024-03-23T09:45:19.305756+0000","last_undegraded":"2024-03-23T09:45:19.305756+0000","last_fullsized":"2024-03-23T09:45:19.305756+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:49:08.104714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.300731+0000","last_change":"2024-03-23T09:45:17.946875+0000","last_active":"2024-03-23T09:45:18.300731+0000","last_peered":"2024-03-23T09:45:18.300731+0000","last_clean":"2024-03-23T09:45:18.300731+0000","last_became_active":"2024-03-23T09:45:17.946588+0000","last_became_peered":"2024-03-23T09:45:17.946588+0000","last_unstale":"2024-03-23T09:45:18.300731+0000","last_undegraded":"2024-03-23T09:45:18.300731+0000","last_fullsized":"2024-03-23T09:45:18.300731+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:42:34.167069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.305914+0000","last_change":"2024-03-23T09:45:18.323720+0000","last_active":"2024-03-23T09:45:19.305914+0000","last_peered":"2024-03-23T09:45:19.305914+0000","last_clean":"2024-03-23T09:45:19.305914+0000","last_became_active":"2024-03-23T09:45:18.323465+0000","last_became_peered":"2024-03-23T09:45:18.323465+0000","last_unstale":"2024-03-23T09:45:19.305914+0000","last_undegraded":"2024-03-23T09:45:19.305914+0000","last_fullsized":"2024-03-23T09:45:19.305914+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:49:04.736990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.308655+0000","last_change":"2024-03-23T09:45:17.949762+0000","last_active":"2024-03-23T09:45:18.308655+0000","last_peered":"2024-03-23T09:45:18.308655+0000","last_clean":"2024-03-23T09:45:18.308655+0000","last_became_active":"2024-03-23T09:45:17.949466+0000","last_became_peered":"2024-03-23T09:45:17.949466+0000","last_unstale":"2024-03-23T09:45:18.308655+0000","last_undegraded":"2024-03-23T09:45:18.308655+0000","last_fullsized":"2024-03-23T09:45:18.308655+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:40:34.327822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.601834+0000","last_change":"2024-03-23T09:45:18.316330+0000","last_active":"2024-03-23T09:45:20.601834+0000","last_peered":"2024-03-23T09:45:20.601834+0000","last_clean":"2024-03-23T09:45:20.601834+0000","last_became_active":"2024-03-23T09:45:18.316103+0000","last_became_peered":"2024-03-23T09:45:18.316103+0000","last_unstale":"2024-03-23T09:45:20.601834+0000","last_undegraded":"2024-03-23T09:45:20.601834+0000","last_fullsized":"2024-03-23T09:45:20.601834+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:38:13.506664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.300449+0000","last_change":"2024-03-23T09:45:17.950763+0000","last_active":"2024-03-23T09:45:18.300449+0000","last_peered":"2024-03-23T09:45:18.300449+0000","last_clean":"2024-03-23T09:45:18.300449+0000","last_became_active":"2024-03-23T09:45:17.950457+0000","last_became_peered":"2024-03-23T09:45:17.950457+0000","last_unstale":"2024-03-23T09:45:18.300449+0000","last_undegraded":"2024-03-23T09:45:18.300449+0000","last_fullsized":"2024-03-23T09:45:18.300449+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:53:03.362600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389725+0000","last_change":"2024-03-23T09:45:18.326030+0000","last_active":"2024-03-23T09:45:20.389725+0000","last_peered":"2024-03-23T09:45:20.389725+0000","last_clean":"2024-03-23T09:45:20.389725+0000","last_became_active":"2024-03-23T09:45:18.325710+0000","last_became_peered":"2024-03-23T09:45:18.325710+0000","last_unstale":"2024-03-23T09:45:20.389725+0000","last_undegraded":"2024-03-23T09:45:20.389725+0000","last_fullsized":"2024-03-23T09:45:20.389725+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:05:38.755296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.303347+0000","last_change":"2024-03-23T09:45:17.945578+0000","last_active":"2024-03-23T09:45:18.303347+0000","last_peered":"2024-03-23T09:45:18.303347+0000","last_clean":"2024-03-23T09:45:18.303347+0000","last_became_active":"2024-03-23T09:45:17.945288+0000","last_became_peered":"2024-03-23T09:45:17.945288+0000","last_unstale":"2024-03-23T09:45:18.303347+0000","last_undegraded":"2024-03-23T09:45:18.303347+0000","last_fullsized":"2024-03-23T09:45:18.303347+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:11:42.583654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:19.304179+0000","last_change":"2024-03-23T09:45:18.309444+0000","last_active":"2024-03-23T09:45:19.304179+0000","last_peered":"2024-03-23T09:45:19.304179+0000","last_clean":"2024-03-23T09:45:19.304179+0000","last_became_active":"2024-03-23T09:45:18.309182+0000","last_became_peered":"2024-03-23T09:45:18.309182+0000","last_unstale":"2024-03-23T09:45:19.304179+0000","last_undegraded":"2024-03-23T09:45:19.304179+0000","last_fullsized":"2024-03-23T09:45:19.304179+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:19.131977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.305287+0000","last_change":"2024-03-23T09:45:17.948306+0000","last_active":"2024-03-23T09:45:18.305287+0000","last_peered":"2024-03-23T09:45:18.305287+0000","last_clean":"2024-03-23T09:45:18.305287+0000","last_became_active":"2024-03-23T09:45:17.948042+0000","last_became_peered":"2024-03-23T09:45:17.948042+0000","last_unstale":"2024-03-23T09:45:18.305287+0000","last_undegraded":"2024-03-23T09:45:18.305287+0000","last_fullsized":"2024-03-23T09:45:18.305287+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:53:08.530702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.389527+0000","last_change":"2024-03-23T09:45:18.317449+0000","last_active":"2024-03-23T09:45:20.389527+0000","last_peered":"2024-03-23T09:45:20.389527+0000","last_clean":"2024-03-23T09:45:20.389527+0000","last_became_active":"2024-03-23T09:45:18.316688+0000","last_became_peered":"2024-03-23T09:45:18.316688+0000","last_unstale":"2024-03-23T09:45:20.389527+0000","last_undegraded":"2024-03-23T09:45:20.389527+0000","last_fullsized":"2024-03-23T09:45:20.389527+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:08.055427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.312227+0000","last_change":"2024-03-23T09:45:17.946349+0000","last_active":"2024-03-23T09:45:18.312227+0000","last_peered":"2024-03-23T09:45:18.312227+0000","last_clean":"2024-03-23T09:45:18.312227+0000","last_became_active":"2024-03-23T09:45:17.945310+0000","last_became_peered":"2024-03-23T09:45:17.945310+0000","last_unstale":"2024-03-23T09:45:18.312227+0000","last_undegraded":"2024-03-23T09:45:18.312227+0000","last_fullsized":"2024-03-23T09:45:18.312227+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:11:42.583654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-23T09:45:20.602147+0000","last_change":"2024-03-23T09:45:18.327523+0000","last_active":"2024-03-23T09:45:20.602147+0000","last_peered":"2024-03-23T09:45:20.602147+0000","last_clean":"2024-03-23T09:45:20.602147+0000","last_became_active":"2024-03-23T09:45:18.327369+0000","last_became_peered":"2024-03-23T09:45:18.327369+0000","last_unstale":"2024-03-23T09:45:20.602147+0000","last_undegraded":"2024-03-23T09:45:20.602147+0000","last_fullsized":"2024-03-23T09:45:20.602147+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:17.923595+0000","last_clean_scrub_stamp":"2024-03-23T09:45:17.923595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:03:50.754016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.320459+0000","last_change":"2024-03-23T09:45:17.947893+0000","last_active":"2024-03-23T09:45:18.320459+0000","last_peered":"2024-03-23T09:45:18.320459+0000","last_clean":"2024-03-23T09:45:18.320459+0000","last_became_active":"2024-03-23T09:45:17.947615+0000","last_became_peered":"2024-03-23T09:45:17.947615+0000","last_unstale":"2024-03-23T09:45:18.320459+0000","last_undegraded":"2024-03-23T09:45:18.320459+0000","last_fullsized":"2024-03-23T09:45:18.320459+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:45:16.919445+0000","last_clean_scrub_stamp":"2024-03-23T09:45:16.919445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:04.521715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":215,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-23T09:45:18.305504+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:45:18.305504+0000","last_peered":"2024-03-23T09:45:18.305504+0000","last_clean":"2024-03-23T09:45:18.305504+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:45:18.305504+0000","last_undegraded":"2024-03-23T09:45:18.305504+0000","last_fullsized":"2024-03-23T09:45:18.305504+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1454080,"data_stored":946928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53100000000000003,"15min":0.48999999999999999},"min":{"1min":0.35699999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72199999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54500000000000004,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65500000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54000000000000004,"15min":0.51200000000000001},"min":{"1min":0.42199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.751,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.439,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78500000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55100000000000005}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.44400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60899999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.48399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.84699999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82699999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.497,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.622,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.33200000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65200000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35899999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.38300000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.46300000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56699999999999995,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":0.874,"15min":0.874},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.379,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56699999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":355417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.371,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.38900000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.5}]},{"osd":1,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53000000000000003,"15min":0.503},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.71699999999999997,"5min":0.751,"15min":0.751},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68200000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.628}]},{"osd":2,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65600000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.41699999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.71499999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.443}]},{"osd":3,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.45600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.45300000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.70599999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.48599999999999999,"15min":0.504},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.626,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.52700000000000002,"15min":0.54700000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.69799999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.70799999999999996}]},{"osd":5,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.50700000000000001,"15min":0.53000000000000003},"min":{"1min":0.36199999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54400000000000004,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53900000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.65800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":356034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.51600000000000001,"15min":0.48199999999999998},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70799999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.432},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55800000000000005,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70699999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.44600000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52700000000000002,"15min":0.49199999999999999},"min":{"1min":0.41999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.504},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.65000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.504}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57599999999999996}]},{"osd":3,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53800000000000003,"15min":0.504},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56699999999999995,"15min":0.53400000000000003},"min":{"1min":0.42699999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.69499999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.53600000000000003}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45900000000000002,"15min":0.46999999999999997},"min":{"1min":0.29599999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67900000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.34799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71999999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.60099999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.498,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.627,"5min":0.749,"15min":0.749},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60299999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.47599999999999998}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.746,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56399999999999995,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.61499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":356014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.437,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.747,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.34899999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.502,"15min":0.47399999999999998},"min":{"1min":0.36899999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.63100000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51700000000000002,"15min":0.504},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.73099999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.46700000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.53800000000000003},"min":{"1min":0.50700000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75800000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.79900000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.53700000000000003}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68400000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.40500000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.71799999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.44700000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46300000000000002,"15min":0.46899999999999997},"min":{"1min":0.32600000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67900000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.48899999999999999,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.66000000000000003,"5min":1.008,"15min":1.008},"last":0.52600000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.307,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60999999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.67200000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61199999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52500000000000002,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.496},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.371,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.68799999999999994,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56200000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":860160,"data_stored":356056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.34100000000000003,"15min":0.316},"min":{"1min":0.28100000000000003,"5min":0.183,"15min":0.183},"max":{"1min":0.627,"5min":1.45,"15min":1.45},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.315},"min":{"1min":0.23100000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.51900000000000002,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.26800000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33400000000000002,"15min":0.312},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54300000000000004,"5min":1.323,"15min":1.323},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.36799999999999999,"15min":0.34899999999999998},"min":{"1min":0.315,"5min":0.218,"15min":0.218},"max":{"1min":0.67800000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.36199999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.38100000000000001},"min":{"1min":0.309,"5min":0.251,"15min":0.251},"max":{"1min":0.61599999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.42999999999999999,"15min":0.41199999999999998},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70899999999999996,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.749,"5min":1.337,"15min":1.337},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.54500000000000004},"min":{"1min":0.437,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":2.758,"15min":2.758},"last":0.434}]},{"osd":5,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77500000000000002,"5min":1.276,"15min":1.276},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86499999999999999,"5min":1.895,"15min":1.895},"last":0.54700000000000004}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.62,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70699999999999996,"5min":5.4539999999999997,"15min":5.4539999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83399999999999996,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.47499999999999998}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.60899999999999999},"min":{"1min":0.45500000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89300000000000002,"5min":3.7400000000000002,"15min":3.7400000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65300000000000002,"15min":0.61899999999999999},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.44400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":864256,"data_stored":357508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.249,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.314},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36799999999999999,"15min":0.378},"min":{"1min":0.26200000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52200000000000002,"5min":0.872,"15min":0.872},"last":0.30099999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.28499999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.629,"15min":0.629},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.39000000000000001},"min":{"1min":0.24299999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58699999999999997,"5min":0.746,"15min":0.746},"last":0.34899999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.34100000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61399999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.27800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.55000000000000004,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.37}]},{"osd":4,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.44900000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84699999999999998,"5min":1.216,"15min":1.216},"last":0.442},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85499999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.626,"15min":0.63300000000000001},"min":{"1min":0.51200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83999999999999997,"5min":1.101,"15min":1.101},"last":0.61499999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.53100000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89100000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63900000000000001,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89500000000000002,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.54500000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.56899999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.872,"5min":1.155,"15min":1.155},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64400000000000002,"15min":0.65400000000000003},"min":{"1min":0.46999999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82099999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.497}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":872448,"data_stored":358124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34799999999999998,"15min":0.33700000000000002},"min":{"1min":0.247,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.5,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42599999999999999,"15min":0.47799999999999998},"min":{"1min":0.23400000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.71999999999999997,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.38700000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.34499999999999997},"min":{"1min":0.251,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39000000000000001,"15min":0.375},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57399999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.307}]},{"osd":3,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42199999999999999,"15min":0.40999999999999998},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63700000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.45100000000000001,"15min":0.50700000000000001},"min":{"1min":0.29199999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.61799999999999999,"5min":2.75,"15min":2.75},"last":0.35599999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.71499999999999997}]},{"osd":5,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.70399999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94199999999999995,"5min":2.778,"15min":2.778},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89000000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68999999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66500000000000004,"15min":0.70899999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65200000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89200000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.75900000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1441792,"data_stored":945693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33400000000000002,"15min":0.32300000000000001},"min":{"1min":0.21299999999999999,"5min":0.184,"15min":0.184},"max":{"1min":0.51300000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.35099999999999998},"min":{"1min":0.23999999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.25900000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.38100000000000001,"15min":0.36699999999999999},"min":{"1min":0.28899999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55400000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.36099999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55200000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.318},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.31900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.40000000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.48999999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.53200000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53700000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.44600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70799999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.504,"5min":0.379,"15min":0.379},"max":{"1min":0.875,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.45700000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82399999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.433},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.64400000000000002},"min":{"1min":0.54200000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89400000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57699999999999996}]},{"osd":7,"last update":"Sat Mar 23 09:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.51000000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85799999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.501,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80700000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:45:26.080 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T09:45:26.080 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-23T09:45:26.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.224+0000 7f65c352a640 1 Processor -- start 2024-03-23T09:45:26.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.224+0000 7f65c352a640 1 -- start start 2024-03-23T09:45:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c352a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f65bc15cbb0 0x7f65bc0b8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c352a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65bc0d1df0 0x7f65bc0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c352a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f65bc0b48f0 0x7f65bc0d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c352a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc0b4da0 con 0x7f65bc0b48f0 2024-03-23T09:45:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c352a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc0b5210 con 0x7f65bc15cbb0 2024-03-23T09:45:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c352a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc15cf90 con 0x7f65bc0d1df0 2024-03-23T09:45:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c1ac7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f65bc0b48f0 0x7f65bc0d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c1ac7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f65bc0b48f0 0x7f65bc0d22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35926/0 (socket says 172.21.15.47:35926) 2024-03-23T09:45:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c1ac7640 1 -- 172.21.15.47:0/4207129100 learned_addr learned my addr 172.21.15.47:0/4207129100 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c12c6640 1 --2- 172.21.15.47:0/4207129100 >> v2:172.21.15.175:3300/0 conn(0x7f65bc15cbb0 0x7f65bc0b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c0ac5640 1 --2- 172.21.15.47:0/4207129100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65bc0d1df0 0x7f65bc0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c1ac7640 1 -- 172.21.15.47:0/4207129100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65bc0d1df0 msgr2=0x7f65bc0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c1ac7640 1 --2- 172.21.15.47:0/4207129100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65bc0d1df0 0x7f65bc0b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c1ac7640 1 -- 172.21.15.47:0/4207129100 >> v2:172.21.15.175:3300/0 conn(0x7f65bc15cbb0 msgr2=0x7f65bc0b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:26.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c1ac7640 1 --2- 172.21.15.47:0/4207129100 >> v2:172.21.15.175:3300/0 conn(0x7f65bc15cbb0 0x7f65bc0b8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c1ac7640 1 -- 172.21.15.47:0/4207129100 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc0d0d50 con 0x7f65bc0b48f0 2024-03-23T09:45:26.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.225+0000 7f65c0ac5640 1 --2- 172.21.15.47:0/4207129100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65bc0d1df0 0x7f65bc0b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:26.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.226+0000 7f65c1ac7640 1 --2- 172.21.15.47:0/4207129100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65bc0b48f0 0x7f65bc0d22a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0ccf30 tx=0x7f65b402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857088cca88a8508 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.226+0000 7f65b3fff640 1 -- 172.21.15.47:0/4207129100 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b402ca70 con 0x7f65bc0b48f0 2024-03-23T09:45:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.226+0000 7f65b3fff640 1 -- 172.21.15.47:0/4207129100 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b402cbd0 con 0x7f65bc0b48f0 2024-03-23T09:45:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.226+0000 7f65b3fff640 1 -- 172.21.15.47:0/4207129100 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b40343f0 con 0x7f65bc0b48f0 2024-03-23T09:45:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.227+0000 7f65c352a640 1 -- 172.21.15.47:0/4207129100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65bc0b48f0 msgr2=0x7f65bc0d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.227+0000 7f65c352a640 1 --2- 172.21.15.47:0/4207129100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65bc0b48f0 0x7f65bc0d22a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0ccf30 tx=0x7f65b402bed0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.227+0000 7f65c352a640 1 -- 172.21.15.47:0/4207129100 shutdown_connections 2024-03-23T09:45:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.227+0000 7f65c352a640 1 --2- 172.21.15.47:0/4207129100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65bc0b48f0 0x7f65bc0d22a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.227+0000 7f65c352a640 1 --2- 172.21.15.47:0/4207129100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65bc0d1df0 0x7f65bc0b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.227+0000 7f65c352a640 1 --2- 172.21.15.47:0/4207129100 >> v2:172.21.15.175:3300/0 conn(0x7f65bc15cbb0 0x7f65bc0b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.227+0000 7f65c352a640 1 -- 172.21.15.47:0/4207129100 >> 172.21.15.47:0/4207129100 conn(0x7f65bc065dc0 msgr2=0x7f65bc0661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.227+0000 7f65c352a640 1 -- 172.21.15.47:0/4207129100 shutdown_connections 2024-03-23T09:45:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.227+0000 7f65c352a640 1 -- 172.21.15.47:0/4207129100 wait complete. 2024-03-23T09:45:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.228+0000 7f65c352a640 1 Processor -- start 2024-03-23T09:45:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.228+0000 7f65c352a640 1 -- start start 2024-03-23T09:45:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c352a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65bc0b48f0 0x7f65bc153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c352a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65bc0d1df0 0x7f65bc0b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c352a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65bc15cbb0 0x7f65bc15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c352a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc14d0c0 con 0x7f65bc0b48f0 2024-03-23T09:45:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c352a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc0b72c0 con 0x7f65bc0d1df0 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c352a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc14ca70 con 0x7f65bc15cbb0 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c12c6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65bc0b48f0 0x7f65bc153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c12c6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65bc0b48f0 0x7f65bc153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35928/0 (socket says 172.21.15.47:35928) 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c12c6640 1 -- 172.21.15.47:0/2312319319 learned_addr learned my addr 172.21.15.47:0/2312319319 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c0ac5640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65bc0d1df0 0x7f65bc0b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c1ac7640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65bc15cbb0 0x7f65bc15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c12c6640 1 -- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65bc15cbb0 msgr2=0x7f65bc15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c12c6640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65bc15cbb0 0x7f65bc15d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c12c6640 1 -- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65bc0d1df0 msgr2=0x7f65bc0b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c12c6640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65bc0d1df0 0x7f65bc0b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c12c6640 1 -- 172.21.15.47:0/2312319319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b4009b30 con 0x7f65bc0b48f0 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c1ac7640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65bc15cbb0 0x7f65bc15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c12c6640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65bc0b48f0 0x7f65bc153190 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f65b800d800 tx=0x7f65b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65b1ffb640 1 -- 172.21.15.47:0/2312319319 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b800b6c0 con 0x7f65bc0b48f0 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65b1ffb640 1 -- 172.21.15.47:0/2312319319 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b800b820 con 0x7f65bc0b48f0 2024-03-23T09:45:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65c352a640 1 -- 172.21.15.47:0/2312319319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc14cd50 con 0x7f65bc0b48f0 2024-03-23T09:45:26.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.229+0000 7f65b1ffb640 1 -- 172.21.15.47:0/2312319319 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b800bbb0 con 0x7f65bc0b48f0 2024-03-23T09:45:26.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.230+0000 7f65c352a640 1 -- 172.21.15.47:0/2312319319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc1527c0 con 0x7f65bc0b48f0 2024-03-23T09:45:26.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.231+0000 7f65c352a640 1 -- 172.21.15.47:0/2312319319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65bc072360 con 0x7f65bc0b48f0 2024-03-23T09:45:26.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.231+0000 7f65b1ffb640 1 -- 172.21.15.47:0/2312319319 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f65b800bd10 con 0x7f65bc0b48f0 2024-03-23T09:45:26.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.231+0000 7f65b1ffb640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f65a006ac00 0x7f65a006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:26.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.231+0000 7f65b1ffb640 1 -- 172.21.15.47:0/2312319319 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65b8089860 con 0x7f65bc0b48f0 2024-03-23T09:45:26.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.232+0000 7f65c0ac5640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f65a006ac00 0x7f65a006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:26.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.232+0000 7f65c0ac5640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f65a006ac00 0x7f65a006d0c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f65bc14f9b0 tx=0x7f65ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:26.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.236+0000 7f65b1ffb640 1 -- 172.21.15.47:0/2312319319 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f65b8054ad0 con 0x7f65bc0b48f0 2024-03-23T09:45:26.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.488+0000 7f65c352a640 1 -- 172.21.15.47:0/2312319319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f65bc14c8d0 con 0x7f65bc0b48f0 2024-03-23T09:45:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.492+0000 7f65b1ffb640 1 -- 172.21.15.47:0/2312319319 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f65bc14c8d0 con 0x7f65bc0b48f0 2024-03-23T09:45:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-03-23T09:45:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.494+0000 7f65c352a640 1 -- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f65a006ac00 msgr2=0x7f65a006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.494+0000 7f65c352a640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f65a006ac00 0x7f65a006d0c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f65bc14f9b0 tx=0x7f65ac008040 comp rx=0 tx=0).stop 2024-03-23T09:45:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.494+0000 7f65c352a640 1 -- 172.21.15.47:0/2312319319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65bc0b48f0 msgr2=0x7f65bc153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.494+0000 7f65c352a640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65bc0b48f0 0x7f65bc153190 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f65b800d800 tx=0x7f65b800dcc0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.494+0000 7f65c352a640 1 -- 172.21.15.47:0/2312319319 shutdown_connections 2024-03-23T09:45:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.494+0000 7f65c352a640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f65a006ac00 0x7f65a006d0c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.494+0000 7f65c352a640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65bc15cbb0 0x7f65bc15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.494+0000 7f65c352a640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65bc0d1df0 0x7f65bc0b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.494+0000 7f65c352a640 1 --2- 172.21.15.47:0/2312319319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65bc0b48f0 0x7f65bc153190 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.494+0000 7f65c352a640 1 -- 172.21.15.47:0/2312319319 >> 172.21.15.47:0/2312319319 conn(0x7f65bc065dc0 msgr2=0x7f65bc0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.494+0000 7f65c352a640 1 -- 172.21.15.47:0/2312319319 shutdown_connections 2024-03-23T09:45:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.494+0000 7f65c352a640 1 -- 172.21.15.47:0/2312319319 wait complete. 2024-03-23T09:45:26.504 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:45:26.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.604+0000 7fe118fc9640 1 Processor -- start 2024-03-23T09:45:26.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.604+0000 7fe118fc9640 1 -- start start 2024-03-23T09:45:26.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.605+0000 7fe118fc9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe11415ca90 0x7fe114160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:26.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.605+0000 7fe118fc9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe1140cde10 0x7fe1140d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:26.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.605+0000 7fe118fc9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1140d11e0 con 0x7fe1140b8350 2024-03-23T09:45:26.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.605+0000 7fe118fc9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe114160f60 con 0x7fe1140cde10 2024-03-23T09:45:26.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.605+0000 7fe118fc9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe114161240 con 0x7fe11415ca90 2024-03-23T09:45:26.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.606+0000 7fe112d9d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe1140cde10 0x7fe1140d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:26.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.606+0000 7fe11259c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe11415ca90 0x7fe114160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:26.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.606+0000 7fe112d9d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe1140cde10 0x7fe1140d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:37702/0 (socket says 172.21.15.47:37702) 2024-03-23T09:45:26.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.606+0000 7fe112d9d640 1 -- 172.21.15.47:0/3878866181 learned_addr learned my addr 172.21.15.47:0/3878866181 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:26.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.606+0000 7fe112d9d640 1 -- 172.21.15.47:0/3878866181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe11415ca90 msgr2=0x7fe114160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:26.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.606+0000 7fe112d9d640 1 --2- 172.21.15.47:0/3878866181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe11415ca90 0x7fe114160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.606+0000 7fe112d9d640 1 -- 172.21.15.47:0/3878866181 >> v1:172.21.15.175:6789/0 conn(0x7fe1140b8350 legacy=0x7fe1140cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.606+0000 7fe112d9d640 1 -- 172.21.15.47:0/3878866181 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe114161520 con 0x7fe1140cde10 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.606+0000 7fe11159a640 1 -- 172.21.15.47:0/3878866181 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233862122 0 0) 0x7fe1140d11e0 con 0x7fe1140b8350 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.606+0000 7fe11259c640 1 --2- 172.21.15.47:0/3878866181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe11415ca90 0x7fe114160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.607+0000 7fe112d9d640 1 --2- 172.21.15.47:0/3878866181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1140cde10 0x7fe1140d1670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe104009e50 tx=0x7fe10402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c189e44981d288c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.607+0000 7fe11159a640 1 -- 172.21.15.47:0/3878866181 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10402ca70 con 0x7fe1140cde10 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.607+0000 7fe11159a640 1 -- 172.21.15.47:0/3878866181 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10402cbd0 con 0x7fe1140cde10 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.607+0000 7fe11159a640 1 -- 172.21.15.47:0/3878866181 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1040343f0 con 0x7fe1140cde10 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.607+0000 7fe118fc9640 1 -- 172.21.15.47:0/3878866181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1140cde10 msgr2=0x7fe1140d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.607+0000 7fe118fc9640 1 --2- 172.21.15.47:0/3878866181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1140cde10 0x7fe1140d1670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe104009e50 tx=0x7fe10402bed0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.608+0000 7fe118fc9640 1 -- 172.21.15.47:0/3878866181 shutdown_connections 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.608+0000 7fe118fc9640 1 --2- 172.21.15.47:0/3878866181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1140cde10 0x7fe1140d1670 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.608+0000 7fe118fc9640 1 --2- 172.21.15.47:0/3878866181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe11415ca90 0x7fe114160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.608+0000 7fe118fc9640 1 -- 172.21.15.47:0/3878866181 >> 172.21.15.47:0/3878866181 conn(0x7fe1140659d0 msgr2=0x7fe114065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.608+0000 7fe118fc9640 1 -- 172.21.15.47:0/3878866181 shutdown_connections 2024-03-23T09:45:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.608+0000 7fe118fc9640 1 -- 172.21.15.47:0/3878866181 wait complete. 2024-03-23T09:45:26.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.609+0000 7fe118fc9640 1 Processor -- start 2024-03-23T09:45:26.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.609+0000 7fe118fc9640 1 -- start start 2024-03-23T09:45:26.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe118fc9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1140b8350 0x7fe11414c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:26.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe118fc9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1140cde10 0x7fe114152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:26.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe118fc9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe11415ca90 0x7fe114152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:26.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe118fc9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11414c980 con 0x7fe1140cde10 2024-03-23T09:45:26.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe118fc9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe114153580 con 0x7fe11415ca90 2024-03-23T09:45:26.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe118fc9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe114153860 con 0x7fe1140b8350 2024-03-23T09:45:26.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe111d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1140cde10 0x7fe114152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:26.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe111d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1140cde10 0x7fe114152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35938/0 (socket says 172.21.15.47:35938) 2024-03-23T09:45:26.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe111d9b640 1 -- 172.21.15.47:0/288330881 learned_addr learned my addr 172.21.15.47:0/288330881 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe11259c640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1140b8350 0x7fe11414c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe112d9d640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe11415ca90 0x7fe114152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe111d9b640 1 -- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1140b8350 msgr2=0x7fe11414c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe111d9b640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1140b8350 0x7fe11414c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe111d9b640 1 -- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe11415ca90 msgr2=0x7fe114152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe111d9b640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe11415ca90 0x7fe114152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe111d9b640 1 -- 172.21.15.47:0/288330881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c009400 con 0x7fe1140cde10 2024-03-23T09:45:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe111d9b640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1140cde10 0x7fe114152a50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c410 tx=0x7fe10800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe102ffd640 1 -- 172.21.15.47:0/288330881 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1080035f0 con 0x7fe1140cde10 2024-03-23T09:45:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe102ffd640 1 -- 172.21.15.47:0/288330881 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe108003770 con 0x7fe1140cde10 2024-03-23T09:45:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe118fc9640 1 -- 172.21.15.47:0/288330881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe104009b30 con 0x7fe1140cde10 2024-03-23T09:45:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe102ffd640 1 -- 172.21.15.47:0/288330881 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108003b20 con 0x7fe1140cde10 2024-03-23T09:45:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.610+0000 7fe118fc9640 1 -- 172.21.15.47:0/288330881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe114153e20 con 0x7fe1140cde10 2024-03-23T09:45:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.611+0000 7fe118fc9640 1 -- 172.21.15.47:0/288330881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe114072360 con 0x7fe1140cde10 2024-03-23T09:45:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.612+0000 7fe102ffd640 1 -- 172.21.15.47:0/288330881 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe108003c80 con 0x7fe1140cde10 2024-03-23T09:45:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.612+0000 7fe102ffd640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe0f406ac00 0x7fe0f406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.612+0000 7fe102ffd640 1 -- 172.21.15.47:0/288330881 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1080889c0 con 0x7fe1140cde10 2024-03-23T09:45:26.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.612+0000 7fe11259c640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe0f406ac00 0x7fe0f406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:26.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.613+0000 7fe11259c640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe0f406ac00 0x7fe0f406d0c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009c60 tx=0x7fe10c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.617+0000 7fe102ffd640 1 -- 172.21.15.47:0/288330881 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe108053ba0 con 0x7fe1140cde10 2024-03-23T09:45:26.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.832+0000 7fe118fc9640 1 -- 172.21.15.47:0/288330881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe114066be0 con 0x7fe1140cde10 2024-03-23T09:45:26.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.832+0000 7fe102ffd640 1 -- 172.21.15.47:0/288330881 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4736 (secure 0 0 0) 0x7fe108017020 con 0x7fe1140cde10 2024-03-23T09:45:26.832 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:26.832 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3862818105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3862818105},{"type":"v1","addr":"172.21.15.175:6836","nonce":3862818105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3615592850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3615592850},{"type":"v1","addr":"172.21.15.175:6837","nonce":3615592850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/832829684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":832829684},{"type":"v1","addr":"172.21.15.47:6834","nonce":832829684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:45:18.296714+0000","modified":"2024-03-23T09:45:20.302269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6744},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6744":{"gid":6744,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/372771687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":372771687},{"type":"v1","addr":"172.21.15.47:6835","nonce":372771687}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-23T09:45:26.833 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-23T09:45:26.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.834+0000 7fe118fc9640 1 -- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe0f406ac00 msgr2=0x7fe0f406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:26.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.834+0000 7fe118fc9640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe0f406ac00 0x7fe0f406d0c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009c60 tx=0x7fe10c008040 comp rx=0 tx=0).stop 2024-03-23T09:45:26.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.834+0000 7fe118fc9640 1 -- 172.21.15.47:0/288330881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1140cde10 msgr2=0x7fe114152a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:26.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.834+0000 7fe118fc9640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1140cde10 0x7fe114152a50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c410 tx=0x7fe10800c8d0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.834+0000 7fe118fc9640 1 -- 172.21.15.47:0/288330881 shutdown_connections 2024-03-23T09:45:26.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.834+0000 7fe118fc9640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe0f406ac00 0x7fe0f406d0c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.834+0000 7fe118fc9640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe11415ca90 0x7fe114152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.834+0000 7fe118fc9640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1140cde10 0x7fe114152a50 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.834+0000 7fe118fc9640 1 --2- 172.21.15.47:0/288330881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1140b8350 0x7fe11414c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:26.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.834+0000 7fe118fc9640 1 -- 172.21.15.47:0/288330881 >> 172.21.15.47:0/288330881 conn(0x7fe1140659d0 msgr2=0x7fe114157510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:26.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.834+0000 7fe118fc9640 1 -- 172.21.15.47:0/288330881 shutdown_connections 2024-03-23T09:45:26.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:26.834+0000 7fe118fc9640 1 -- 172.21.15.47:0/288330881 wait complete. 2024-03-23T09:45:27.248 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:45:18.296714+0000', 'modified': '2024-03-23T09:45:20.302269+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6744}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6744': {'gid': 6744, 'name': 'c', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/372771687', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6833', 'nonce': 372771687}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 372771687}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:45:27.248 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:45:27.248 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T09:45:27.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.354+0000 7f25bb58e640 1 Processor -- start 2024-03-23T09:45:27.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.354+0000 7f25bb58e640 1 -- start start 2024-03-23T09:45:27.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25bb58e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b415cd00 0x7f25b40d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25bb58e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f25b40cf920 0x7f25b40b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25bb58e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b415d0e0 con 0x7f25b40d1020 2024-03-23T09:45:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25bb58e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b40d1540 con 0x7f25b40cf920 2024-03-23T09:45:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25bb58e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40d1820 con 0x7f25b415cd00 2024-03-23T09:45:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25b8b29640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f25b40d1020 0x7f25b40cf210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41288/0 (socket says 172.21.15.47:41288) 2024-03-23T09:45:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25b8b29640 1 -- 172.21.15.47:0/3659203160 learned_addr learned my addr 172.21.15.47:0/3659203160 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25b932a640 1 --2- 172.21.15.47:0/3659203160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b415cd00 0x7f25b40d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25b9b2b640 1 --2- 172.21.15.47:0/3659203160 >> v2:172.21.15.175:3300/0 conn(0x7f25b40cf920 0x7f25b40b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650695820 0 0) 0x7f25b415d0e0 con 0x7f25b40d1020 2024-03-23T09:45:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25a0003520 con 0x7f25b40d1020 2024-03-23T09:45:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2640089785 0 0) 0x7f25a0003520 con 0x7f25b40d1020 2024-03-23T09:45:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b415cd00 msgr2=0x7f25b40d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 --2- 172.21.15.47:0/3659203160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b415cd00 0x7f25b40d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 >> v2:172.21.15.175:3300/0 conn(0x7f25b40cf920 msgr2=0x7f25b40b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 --2- 172.21.15.47:0/3659203160 >> v2:172.21.15.175:3300/0 conn(0x7f25b40cf920 0x7f25b40b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b40d1b00 con 0x7f25b40d1020 2024-03-23T09:45:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f25a4001670 con 0x7f25b40d1020 2024-03-23T09:45:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 >> v1:172.21.15.47:6789/0 conn(0x7f25b40d1020 legacy=0x7f25b40cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 --2- 172.21.15.47:0/3659203160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a0003970 0x7f25a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 --2- 172.21.15.47:0/3659203160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b415cd00 0x7f25a00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 --2- 172.21.15.47:0/3659203160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a0004d00 0x7f25a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40d1540 con 0x7f25a0004d00 2024-03-23T09:45:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40d1820 con 0x7f25a0003970 2024-03-23T09:45:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a000b980 con 0x7f25b415cd00 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.355+0000 7f25b9b2b640 1 --2- 172.21.15.47:0/3659203160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a0004d00 0x7f25a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.356+0000 7f25b932a640 1 --2- 172.21.15.47:0/3659203160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a0003970 0x7f25a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.356+0000 7f25b8b29640 1 --2- 172.21.15.47:0/3659203160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b415cd00 0x7f25a00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.356+0000 7f25b9b2b640 1 -- 172.21.15.47:0/3659203160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b415cd00 msgr2=0x7f25a00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.356+0000 7f25b9b2b640 1 --2- 172.21.15.47:0/3659203160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b415cd00 0x7f25a00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.356+0000 7f25b9b2b640 1 -- 172.21.15.47:0/3659203160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a0003970 msgr2=0x7f25a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.356+0000 7f25b9b2b640 1 --2- 172.21.15.47:0/3659203160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a0003970 0x7f25a0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.356+0000 7f25b9b2b640 1 -- 172.21.15.47:0/3659203160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0003520 con 0x7f25a0004d00 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.357+0000 7f25b9b2b640 1 --2- 172.21.15.47:0/3659203160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a0004d00 0x7f25a0005180 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f25b40d0420 tx=0x7f25a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a2add597fe4a35 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.357+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac02cbd0 con 0x7f25a0004d00 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.357+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac00f040 con 0x7f25a0004d00 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.357+0000 7f25abfff640 1 -- 172.21.15.47:0/3659203160 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac034440 con 0x7f25a0004d00 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.357+0000 7f25bb58e640 1 -- 172.21.15.47:0/3659203160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a0004d00 msgr2=0x7f25a0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.357+0000 7f25bb58e640 1 --2- 172.21.15.47:0/3659203160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a0004d00 0x7f25a0005180 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f25b40d0420 tx=0x7f25a00057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.357+0000 7f25b9b2b640 1 -- 172.21.15.47:0/3659203160 reap_dead start 2024-03-23T09:45:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.357+0000 7f25bb58e640 1 -- 172.21.15.47:0/3659203160 shutdown_connections 2024-03-23T09:45:27.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.357+0000 7f25bb58e640 1 -- 172.21.15.47:0/3659203160 >> 172.21.15.47:0/3659203160 conn(0x7f25b406b160 msgr2=0x7f25b406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:27.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.357+0000 7f25bb58e640 1 -- 172.21.15.47:0/3659203160 shutdown_connections 2024-03-23T09:45:27.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.357+0000 7f25bb58e640 1 -- 172.21.15.47:0/3659203160 wait complete. 2024-03-23T09:45:27.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.358+0000 7f25bb58e640 1 Processor -- start 2024-03-23T09:45:27.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.358+0000 7f25bb58e640 1 -- start start 2024-03-23T09:45:27.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.359+0000 7f25bb58e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b415aba0 0x7f25b40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.359+0000 7f25bb58e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25b415af80 0x7f25b40aa270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.359+0000 7f25bb58e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b40cbcd0 0x7f25b40c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.359+0000 7f25bb58e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40b23b0 con 0x7f25b415af80 2024-03-23T09:45:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.359+0000 7f25bb58e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40b2690 con 0x7f25b415aba0 2024-03-23T09:45:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.359+0000 7f25bb58e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40b2970 con 0x7f25b40cbcd0 2024-03-23T09:45:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.359+0000 7f25b8b29640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25b415af80 0x7f25b40aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.359+0000 7f25b8b29640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25b415af80 0x7f25b40aa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35964/0 (socket says 172.21.15.47:35964) 2024-03-23T09:45:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.359+0000 7f25b8b29640 1 -- 172.21.15.47:0/3578700825 learned_addr learned my addr 172.21.15.47:0/3578700825 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.359+0000 7f25b9b2b640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b40cbcd0 0x7f25b40c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.359+0000 7f25b932a640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b415aba0 0x7f25b40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.360+0000 7f25b8b29640 1 -- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b40cbcd0 msgr2=0x7f25b40c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.360+0000 7f25b8b29640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b40cbcd0 0x7f25b40c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.360+0000 7f25b8b29640 1 -- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b415aba0 msgr2=0x7f25b40b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.360+0000 7f25b8b29640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b415aba0 0x7f25b40b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.360+0000 7f25b8b29640 1 -- 172.21.15.47:0/3578700825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b40b2c50 con 0x7f25b415af80 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.360+0000 7f25b9b2b640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b40cbcd0 0x7f25b40c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.360+0000 7f25b8b29640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25b415af80 0x7f25b40aa270 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f25a4007290 tx=0x7f25a4007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.360+0000 7f25a9ffb640 1 -- 172.21.15.47:0/3578700825 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a400f9a0 con 0x7f25b415af80 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.360+0000 7f25a9ffb640 1 -- 172.21.15.47:0/3578700825 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a400fb00 con 0x7f25b415af80 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.360+0000 7f25bb58e640 1 -- 172.21.15.47:0/3578700825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b40cc160 con 0x7f25b415af80 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.360+0000 7f25a9ffb640 1 -- 172.21.15.47:0/3578700825 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a400feb0 con 0x7f25b415af80 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.360+0000 7f25bb58e640 1 -- 172.21.15.47:0/3578700825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b40c9f70 con 0x7f25b415af80 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.361+0000 7f25a9ffb640 1 -- 172.21.15.47:0/3578700825 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f25a401d460 con 0x7f25b415af80 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.362+0000 7f25a9ffb640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f259806ac00 0x7f259806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.362+0000 7f25a9ffb640 1 -- 172.21.15.47:0/3578700825 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25a4089600 con 0x7f25b415af80 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.362+0000 7f25b932a640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f259806ac00 0x7f259806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.363+0000 7f25b932a640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f259806ac00 0x7f259806d0c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f25b40d0f70 tx=0x7f25b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.363+0000 7f25be1d6740 1 Processor -- start 2024-03-23T09:45:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.363+0000 7f25be1d6740 1 -- start start 2024-03-23T09:45:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25be1d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93fb750 0x5601e93b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25be1d6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e93b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25be1d6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93adc70 0x5601e93dda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25be1d6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e931e9b0 con 0x5601e93b17d0 2024-03-23T09:45:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25be1d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e931c140 con 0x5601e93fb750 2024-03-23T09:45:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25be1d6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601e9317a30 con 0x5601e93adc70 2024-03-23T09:45:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25b8b29640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e93b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25b8b29640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e93b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35976/0 (socket says 172.21.15.47:35976) 2024-03-23T09:45:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25b8b29640 1 -- 172.21.15.47:0/3663137127 learned_addr learned my addr 172.21.15.47:0/3663137127 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25b9b2b640 1 --2- 172.21.15.47:0/3663137127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93adc70 0x5601e93dda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25b932a640 1 --2- 172.21.15.47:0/3663137127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93fb750 0x5601e93b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25b8b29640 1 -- 172.21.15.47:0/3663137127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93adc70 msgr2=0x5601e93dda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25b8b29640 1 --2- 172.21.15.47:0/3663137127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93adc70 0x5601e93dda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25b8b29640 1 -- 172.21.15.47:0/3663137127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93fb750 msgr2=0x5601e93b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25b8b29640 1 --2- 172.21.15.47:0/3663137127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93fb750 0x5601e93b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25b8b29640 1 -- 172.21.15.47:0/3663137127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601e92a42f0 con 0x5601e93b17d0 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25b932a640 1 --2- 172.21.15.47:0/3663137127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93fb750 0x5601e93b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f25b8b29640 1 --2- 172.21.15.47:0/3663137127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e93b32c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f25a4087be0 tx=0x7f25a4057fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a31ed75bef8590 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f259effd640 1 -- 172.21.15.47:0/3663137127 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4060920 con 0x5601e93b17d0 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f259effd640 1 -- 172.21.15.47:0/3663137127 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a40568d0 con 0x5601e93b17d0 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.364+0000 7f259effd640 1 -- 172.21.15.47:0/3663137127 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4060490 con 0x5601e93b17d0 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.365+0000 7f25be1d6740 1 -- 172.21.15.47:0/3663137127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 msgr2=0x5601e93b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.365+0000 7f25be1d6740 1 --2- 172.21.15.47:0/3663137127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e93b32c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f25a4087be0 tx=0x7f25a4057fc0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.365+0000 7f25be1d6740 1 -- 172.21.15.47:0/3663137127 shutdown_connections 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.365+0000 7f25be1d6740 1 --2- 172.21.15.47:0/3663137127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93adc70 0x5601e93dda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.365+0000 7f25be1d6740 1 --2- 172.21.15.47:0/3663137127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e93b32c0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.365+0000 7f25be1d6740 1 --2- 172.21.15.47:0/3663137127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93fb750 0x5601e93b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.365+0000 7f25be1d6740 1 -- 172.21.15.47:0/3663137127 >> 172.21.15.47:0/3663137127 conn(0x5601e93afee0 msgr2=0x5601e93f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.365+0000 7f25be1d6740 1 -- 172.21.15.47:0/3663137127 shutdown_connections 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.365+0000 7f25be1d6740 1 -- 172.21.15.47:0/3663137127 wait complete. 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.365+0000 7f25be1d6740 1 -- start start 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.366+0000 7f25be1d6740 1 Processor -- start 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.366+0000 7f25be1d6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e93d9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.366+0000 7f25be1d6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93fb750 0x5601e93fddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.366+0000 7f25be1d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93d9d60 0x5601e93fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.366+0000 7f25be1d6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e92d6090 con 0x5601e93b17d0 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.366+0000 7f25be1d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e92d5d30 con 0x5601e93d9d60 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.366+0000 7f25be1d6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601e929b2a0 con 0x5601e93fb750 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25be1d6740 10 client.0 _ll_register_callbacks cb 0x5601e92d1050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25b932a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e93d9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25b932a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e93d9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35978/0 (socket says 172.21.15.47:35978) 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25b932a640 1 -- 172.21.15.47:0/2295664450 learned_addr learned my addr 172.21.15.47:0/2295664450 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25b9b2b640 1 --2- 172.21.15.47:0/2295664450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93d9d60 0x5601e93fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25b8b29640 1 --2- 172.21.15.47:0/2295664450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93fb750 0x5601e93fddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25b932a640 1 -- 172.21.15.47:0/2295664450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93fb750 msgr2=0x5601e93fddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25b932a640 1 --2- 172.21.15.47:0/2295664450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93fb750 0x5601e93fddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25b932a640 1 -- 172.21.15.47:0/2295664450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93d9d60 msgr2=0x5601e93fe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25b932a640 1 --2- 172.21.15.47:0/2295664450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93d9d60 0x5601e93fe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25b932a640 1 -- 172.21.15.47:0/2295664450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b40c9f70 con 0x5601e93b17d0 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25b932a640 1 --2- 172.21.15.47:0/2295664450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e93d9160 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f25b0014f20 tx=0x7f25b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c7e0bb7bc9b474 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f2593fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f2593fff640 1 -- 172.21.15.47:0/2295664450 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b001c040 con 0x5601e93b17d0 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f2593fff640 1 -- 172.21.15.47:0/2295664450 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25b0016700 con 0x5601e93b17d0 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25be1d6740 1 -- 172.21.15.47:0/2295664450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601e900b3c0 con 0x5601e93b17d0 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f2593fff640 1 -- 172.21.15.47:0/2295664450 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b00168c0 con 0x5601e93b17d0 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25be1d6740 10 client.6906 fetch_fsmap learned FSMap version 32 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f25be1d6740 1 -- 172.21.15.47:0/2295664450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5601e93160b0 con 0x5601e93b17d0 2024-03-23T09:45:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.367+0000 7f2593fff640 1 -- 172.21.15.47:0/2295664450 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601e900b3c0 con 0x5601e93b17d0 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.368+0000 7f2593fff640 1 -- 172.21.15.47:0/2295664450 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f25b0022050 con 0x5601e93b17d0 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.368+0000 7f25be1d6740 10 client.6906 fetch_fsmap finished waiting for FSMap version 32 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.368+0000 7f25be1d6740 10 client.6906 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.368+0000 7f25be1d6740 20 client.6906 populate_metadata read hostname 'smithi047' 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.368+0000 7f25be1d6740 1 --2- 172.21.15.47:0/2295664450 >> [v2:172.21.15.47:6833/372771687,v1:172.21.15.47:6835/372771687] conn(0x5601e9413ab0 0x5601e9415ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.368+0000 7f25be1d6740 4 client.6906 mds_command: new command op to 6744 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.368+0000 7f25be1d6740 1 -- 172.21.15.47:0/2295664450 --> [v2:172.21.15.47:6833/372771687,v1:172.21.15.47:6835/372771687] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601e901c6c0 con 0x5601e9413ab0 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.368+0000 7f25b8b29640 1 --2- 172.21.15.47:0/2295664450 >> [v2:172.21.15.47:6833/372771687,v1:172.21.15.47:6835/372771687] conn(0x5601e9413ab0 0x5601e9415ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.368+0000 7f25b8b29640 1 --2- 172.21.15.47:0/2295664450 >> [v2:172.21.15.47:6833/372771687,v1:172.21.15.47:6835/372771687] conn(0x5601e9413ab0 0x5601e9415ea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f7b5a9979058149 server_cookie=55041061278fbcd7 in_seq=0 out_seq=0 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.368+0000 7f2593fff640 10 client.6906 ms_handle_connect on v2:172.21.15.47:6833/372771687 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f2593fff640 1 -- 172.21.15.47:0/2295664450 <== mds.0 v2:172.21.15.47:6833/372771687 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x7f25a40755b0 con 0x5601e9413ab0 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f2593fff640 10 client.6906 handle_command_reply: tid=0 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f25be1d6740 1 client.6906 shutdown 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f25be1d6740 1 -- 172.21.15.47:0/2295664450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 msgr2=0x5601e93d9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f25be1d6740 1 --2- 172.21.15.47:0/2295664450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e93d9160 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f25b0014f20 tx=0x7f25b0004730 comp rx=0 tx=0).stop 2024-03-23T09:45:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f25be1d6740 1 -- 172.21.15.47:0/2295664450 shutdown_connections 2024-03-23T09:45:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f25be1d6740 1 --2- 172.21.15.47:0/2295664450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93d9d60 0x5601e93fe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f25be1d6740 1 --2- 172.21.15.47:0/2295664450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93fb750 0x5601e93fddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f25be1d6740 1 --2- 172.21.15.47:0/2295664450 >> [v2:172.21.15.47:6833/372771687,v1:172.21.15.47:6835/372771687] conn(0x5601e9413ab0 0x5601e9415ea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f25be1d6740 1 --2- 172.21.15.47:0/2295664450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e93d9160 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f2593fff640 0 client.6906 ms_handle_reset on v2:172.21.15.47:6833/372771687 2024-03-23T09:45:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f25be1d6740 1 -- 172.21.15.47:0/2295664450 >> 172.21.15.47:0/2295664450 conn(0x5601e93adc70 msgr2=0x5601e93b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f25be1d6740 1 -- 172.21.15.47:0/2295664450 shutdown_connections 2024-03-23T09:45:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f25be1d6740 1 -- 172.21.15.47:0/2295664450 wait complete. 2024-03-23T09:45:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.369+0000 7f25be1d6740 20 client.6906 trim_cache size 0 max 16384 2024-03-23T09:45:27.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.385+0000 7f25be1d6740 1 Processor -- start 2024-03-23T09:45:27.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.385+0000 7f25be1d6740 1 -- start start 2024-03-23T09:45:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25be1d6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93b17d0 0x5601e94591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25be1d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93d9d60 0x5601e93f7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25be1d6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93fb750 0x5601e93f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25be1d6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e93d5a70 con 0x5601e93fb750 2024-03-23T09:45:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25be1d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e93ad9b0 con 0x5601e93d9d60 2024-03-23T09:45:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25be1d6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601e92d13a0 con 0x5601e93b17d0 2024-03-23T09:45:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25b9b2b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93fb750 0x5601e93f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25b9b2b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93fb750 0x5601e93f7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35984/0 (socket says 172.21.15.47:35984) 2024-03-23T09:45:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25b9b2b640 1 -- 172.21.15.47:0/1847484085 learned_addr learned my addr 172.21.15.47:0/1847484085 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25b9b2b640 1 -- 172.21.15.47:0/1847484085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93b17d0 msgr2=0x5601e94591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25b9b2b640 1 --2- 172.21.15.47:0/1847484085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93b17d0 0x5601e94591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25b9b2b640 1 -- 172.21.15.47:0/1847484085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93d9d60 msgr2=0x5601e93f7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25b9b2b640 1 --2- 172.21.15.47:0/1847484085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93d9d60 0x5601e93f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25b9b2b640 1 -- 172.21.15.47:0/1847484085 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601e92b8430 con 0x5601e93fb750 2024-03-23T09:45:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.386+0000 7f25b9b2b640 1 --2- 172.21.15.47:0/1847484085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93fb750 0x5601e93f7aa0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f25ac037040 tx=0x7f25ac02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cc950cf16a0d01 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.387+0000 7f259f7fe640 1 -- 172.21.15.47:0/1847484085 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac02cbd0 con 0x5601e93fb750 2024-03-23T09:45:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.387+0000 7f259f7fe640 1 -- 172.21.15.47:0/1847484085 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac02cd30 con 0x5601e93fb750 2024-03-23T09:45:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.387+0000 7f259f7fe640 1 -- 172.21.15.47:0/1847484085 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac02cea0 con 0x5601e93fb750 2024-03-23T09:45:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.387+0000 7f25be1d6740 1 -- 172.21.15.47:0/1847484085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93fb750 msgr2=0x5601e93f7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.387+0000 7f25be1d6740 1 --2- 172.21.15.47:0/1847484085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93fb750 0x5601e93f7aa0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f25ac037040 tx=0x7f25ac02e070 comp rx=0 tx=0).stop 2024-03-23T09:45:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.387+0000 7f25be1d6740 1 -- 172.21.15.47:0/1847484085 shutdown_connections 2024-03-23T09:45:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.387+0000 7f25be1d6740 1 --2- 172.21.15.47:0/1847484085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93fb750 0x5601e93f7aa0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.387+0000 7f25be1d6740 1 --2- 172.21.15.47:0/1847484085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93d9d60 0x5601e93f7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.387+0000 7f25be1d6740 1 --2- 172.21.15.47:0/1847484085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93b17d0 0x5601e94591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.387+0000 7f25be1d6740 1 -- 172.21.15.47:0/1847484085 >> 172.21.15.47:0/1847484085 conn(0x5601e93adc70 msgr2=0x5601e9419ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:27.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.387+0000 7f25be1d6740 1 -- 172.21.15.47:0/1847484085 shutdown_connections 2024-03-23T09:45:27.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.387+0000 7f25be1d6740 1 -- 172.21.15.47:0/1847484085 wait complete. 2024-03-23T09:45:27.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.388+0000 7f25be1d6740 1 -- start start 2024-03-23T09:45:27.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.388+0000 7f25be1d6740 1 Processor -- start 2024-03-23T09:45:27.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25be1d6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e94116d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25be1d6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93d9d60 0x5601e9411c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25be1d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93fb750 0x5601e93ff030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25be1d6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e93dee50 con 0x5601e93b17d0 2024-03-23T09:45:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25be1d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e93b3b50 con 0x5601e93fb750 2024-03-23T09:45:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25be1d6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601e93b3da0 con 0x5601e93d9d60 2024-03-23T09:45:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25be1d6740 10 client.0 _ll_register_callbacks cb 0x7f25a40548f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:45:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25b932a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e94116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25b932a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e94116d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35986/0 (socket says 172.21.15.47:35986) 2024-03-23T09:45:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25b932a640 1 -- 172.21.15.47:0/1590357535 learned_addr learned my addr 172.21.15.47:0/1590357535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25b9b2b640 1 --2- 172.21.15.47:0/1590357535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93fb750 0x5601e93ff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25b8b29640 1 --2- 172.21.15.47:0/1590357535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93d9d60 0x5601e9411c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25b932a640 1 -- 172.21.15.47:0/1590357535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93d9d60 msgr2=0x5601e9411c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25b932a640 1 --2- 172.21.15.47:0/1590357535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93d9d60 0x5601e9411c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25b932a640 1 -- 172.21.15.47:0/1590357535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93fb750 msgr2=0x5601e93ff030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25b932a640 1 --2- 172.21.15.47:0/1590357535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93fb750 0x5601e93ff030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25b932a640 1 -- 172.21.15.47:0/1590357535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a0003520 con 0x5601e93b17d0 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25b932a640 1 --2- 172.21.15.47:0/1590357535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e94116d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f25b00147c0 tx=0x7f25b0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a3f29a0e89acf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f2593fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f2593fff640 1 -- 172.21.15.47:0/1590357535 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b002fc20 con 0x5601e93b17d0 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f2593fff640 1 -- 172.21.15.47:0/1590357535 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25b002fd80 con 0x5601e93b17d0 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.389+0000 7f25be1d6740 1 -- 172.21.15.47:0/1590357535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601e93d6710 con 0x5601e93b17d0 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f2593fff640 1 -- 172.21.15.47:0/1590357535 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b0003e70 con 0x5601e93b17d0 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f25be1d6740 10 client.6912 fetch_fsmap learned FSMap version 32 2024-03-23T09:45:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f25be1d6740 1 -- 172.21.15.47:0/1590357535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5601e92ddb40 con 0x5601e93b17d0 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f2593fff640 1 -- 172.21.15.47:0/1590357535 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601e93d6710 con 0x5601e93b17d0 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f2593fff640 1 -- 172.21.15.47:0/1590357535 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f25b0022050 con 0x5601e93b17d0 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f25be1d6740 10 client.6912 fetch_fsmap finished waiting for FSMap version 32 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f25be1d6740 10 client.6912 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f25be1d6740 20 client.6912 populate_metadata read hostname 'smithi047' 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f25be1d6740 1 --2- 172.21.15.47:0/1590357535 >> [v2:172.21.15.47:6833/372771687,v1:172.21.15.47:6835/372771687] conn(0x5601e93ff570 0x5601e93ff950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f25be1d6740 4 client.6912 mds_command: new command op to 6744 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f25be1d6740 1 -- 172.21.15.47:0/1590357535 --> [v2:172.21.15.47:6833/372771687,v1:172.21.15.47:6835/372771687] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f25a40568d0 con 0x5601e93ff570 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f25b8b29640 1 --2- 172.21.15.47:0/1590357535 >> [v2:172.21.15.47:6833/372771687,v1:172.21.15.47:6835/372771687] conn(0x5601e93ff570 0x5601e93ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f2593fff640 10 client.6912 ms_handle_connect on v2:172.21.15.47:6833/372771687 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f25b8b29640 1 --2- 172.21.15.47:0/1590357535 >> [v2:172.21.15.47:6833/372771687,v1:172.21.15.47:6835/372771687] conn(0x5601e93ff570 0x5601e93ff950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d8b6faec49b9c7 server_cookie=a27672cd92ff0bbd in_seq=0 out_seq=0 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f2593fff640 1 -- 172.21.15.47:0/1590357535 <== mds.0 v2:172.21.15.47:6833/372771687 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x7f25a40568d0 con 0x5601e93ff570 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f2593fff640 10 client.6912 handle_command_reply: tid=0 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.390+0000 7f25be1d6740 1 client.6912 shutdown 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25be1d6740 1 -- 172.21.15.47:0/1590357535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 msgr2=0x5601e94116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25be1d6740 1 --2- 172.21.15.47:0/1590357535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e94116d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f25b00147c0 tx=0x7f25b0009190 comp rx=0 tx=0).stop 2024-03-23T09:45:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25be1d6740 1 -- 172.21.15.47:0/1590357535 shutdown_connections 2024-03-23T09:45:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25be1d6740 1 --2- 172.21.15.47:0/1590357535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601e93fb750 0x5601e93ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25be1d6740 1 --2- 172.21.15.47:0/1590357535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5601e93d9d60 0x5601e9411c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25be1d6740 1 --2- 172.21.15.47:0/1590357535 >> [v2:172.21.15.47:6833/372771687,v1:172.21.15.47:6835/372771687] conn(0x5601e93ff570 0x5601e93ff950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25be1d6740 1 --2- 172.21.15.47:0/1590357535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601e93b17d0 0x5601e94116d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25be1d6740 1 -- 172.21.15.47:0/1590357535 >> 172.21.15.47:0/1590357535 conn(0x5601e93adc70 msgr2=0x5601e93fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f2593fff640 0 client.6912 ms_handle_reset on v2:172.21.15.47:6833/372771687 2024-03-23T09:45:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25be1d6740 1 -- 172.21.15.47:0/1590357535 shutdown_connections 2024-03-23T09:45:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25be1d6740 1 -- 172.21.15.47:0/1590357535 wait complete. 2024-03-23T09:45:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25be1d6740 20 client.6912 trim_cache size 0 max 16384 2024-03-23T09:45:27.397 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:45:27.397 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:45:27.398 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:45:27.398 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 6744, 2024-03-23T09:45:27.398 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:45:27.398 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:45:27.398 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:45:27.398 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 9.0931192060000008, 2024-03-23T09:45:27.398 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 32, 2024-03-23T09:45:27.398 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 70, 2024-03-23T09:45:27.398 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 70, 2024-03-23T09:45:27.398 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 13.471869922 2024-03-23T09:45:27.398 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:45:27.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25bb58e640 1 -- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f259806ac00 msgr2=0x7f259806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25bb58e640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f259806ac00 0x7f259806d0c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f25b40d0f70 tx=0x7f25b0008040 comp rx=0 tx=0).stop 2024-03-23T09:45:27.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25bb58e640 1 -- 172.21.15.47:0/3578700825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25b415af80 msgr2=0x7f25b40aa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25bb58e640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25b415af80 0x7f25b40aa270 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f25a4007290 tx=0x7f25a4007750 comp rx=0 tx=0).stop 2024-03-23T09:45:27.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25bb58e640 1 -- 172.21.15.47:0/3578700825 shutdown_connections 2024-03-23T09:45:27.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25bb58e640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f259806ac00 0x7f259806d0c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25bb58e640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25b40cbcd0 0x7f25b40c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25bb58e640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25b415af80 0x7f25b40aa270 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25bb58e640 1 --2- 172.21.15.47:0/3578700825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b415aba0 0x7f25b40b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25bb58e640 1 -- 172.21.15.47:0/3578700825 >> 172.21.15.47:0/3578700825 conn(0x7f25b406b160 msgr2=0x7f25b406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:27.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25bb58e640 1 -- 172.21.15.47:0/3578700825 shutdown_connections 2024-03-23T09:45:27.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.391+0000 7f25bb58e640 1 -- 172.21.15.47:0/3578700825 wait complete. 2024-03-23T09:45:27.402 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:45:27.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.502+0000 7fb1c8a41640 1 Processor -- start 2024-03-23T09:45:27.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.502+0000 7fb1c8a41640 1 -- start start 2024-03-23T09:45:27.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.503+0000 7fb1c8a41640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb1c415cc40 0x7fb1c4160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.503+0000 7fb1c8a41640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c40d09d0 con 0x7fb1c4160c90 2024-03-23T09:45:27.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.503+0000 7fb1c8a41640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c40b5a80 con 0x7fb1c40cddc0 2024-03-23T09:45:27.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.503+0000 7fb1c8a41640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c415d020 con 0x7fb1c415cc40 2024-03-23T09:45:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.503+0000 7fb1c1d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb1c4160c90 0x7fb1c40cd6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41302/0 (socket says 172.21.15.47:41302) 2024-03-23T09:45:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.503+0000 7fb1c1d9b640 1 -- 172.21.15.47:0/183776339 learned_addr learned my addr 172.21.15.47:0/183776339 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c259c640 1 --2- 172.21.15.47:0/183776339 >> v2:172.21.15.175:3300/0 conn(0x7fb1c415cc40 0x7fb1c4160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350213121 0 0) 0x7fb1c40d09d0 con 0x7fb1c4160c90 2024-03-23T09:45:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1b0003520 con 0x7fb1c4160c90 2024-03-23T09:45:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 151639450 0 0) 0x7fb1b0003520 con 0x7fb1c4160c90 2024-03-23T09:45:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 >> v2:172.21.15.175:3300/0 conn(0x7fb1c415cc40 msgr2=0x7fb1c4160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 --2- 172.21.15.47:0/183776339 >> v2:172.21.15.175:3300/0 conn(0x7fb1c415cc40 0x7fb1c4160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 >> v1:172.21.15.175:6789/0 conn(0x7fb1c40cddc0 legacy=0x7fb1c40d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c41611a0 con 0x7fb1c4160c90 2024-03-23T09:45:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fb1ac001670 con 0x7fb1c4160c90 2024-03-23T09:45:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 >> v1:172.21.15.47:6789/0 conn(0x7fb1c4160c90 legacy=0x7fb1c40cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 --2- 172.21.15.47:0/183776339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1b0003970 0x7fb1b0003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 --2- 172.21.15.47:0/183776339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1b0004370 0x7fb1b0004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 --2- 172.21.15.47:0/183776339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b0005040 0x7fb1b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c415d020 con 0x7fb1b0003970 2024-03-23T09:45:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b000bfa0 con 0x7fb1b0005040 2024-03-23T09:45:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b000c160 con 0x7fb1b0004370 2024-03-23T09:45:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c259c640 1 --2- 172.21.15.47:0/183776339 >> v2:172.21.15.175:3300/0 conn(0x7fb1c415cc40 0x7fb1c4160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:45:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c259c640 1 --2- 172.21.15.47:0/183776339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1b0003970 0x7fb1b0003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c1d9b640 1 --2- 172.21.15.47:0/183776339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1b0004370 0x7fb1b0004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c2d9d640 1 --2- 172.21.15.47:0/183776339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b0005040 0x7fb1b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c259c640 1 -- 172.21.15.47:0/183776339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1b0004370 msgr2=0x7fb1b0004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c259c640 1 --2- 172.21.15.47:0/183776339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1b0004370 0x7fb1b0004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c259c640 1 -- 172.21.15.47:0/183776339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b0005040 msgr2=0x7fb1b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c259c640 1 --2- 172.21.15.47:0/183776339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b0005040 0x7fb1b000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c259c640 1 -- 172.21.15.47:0/183776339 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b0003520 con 0x7fb1b0003970 2024-03-23T09:45:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.504+0000 7fb1c2d9d640 1 -- 172.21.15.47:0/183776339 reap_dead start 2024-03-23T09:45:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.505+0000 7fb1c259c640 1 --2- 172.21.15.47:0/183776339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1b0003970 0x7fb1b0003d50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009c70 tx=0x7fb1b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74a1e61a8fc5ba7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.505+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc004020 con 0x7fb1b0003970 2024-03-23T09:45:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.505+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc004180 con 0x7fb1b0003970 2024-03-23T09:45:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.505+0000 7fb1c159a640 1 -- 172.21.15.47:0/183776339 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc02c8f0 con 0x7fb1b0003970 2024-03-23T09:45:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.506+0000 7fb1c8a41640 1 -- 172.21.15.47:0/183776339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1b0003970 msgr2=0x7fb1b0003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.506+0000 7fb1c8a41640 1 --2- 172.21.15.47:0/183776339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1b0003970 0x7fb1b0003d50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009c70 tx=0x7fb1b00057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.506+0000 7fb1c8a41640 1 -- 172.21.15.47:0/183776339 shutdown_connections 2024-03-23T09:45:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.506+0000 7fb1c8a41640 1 --2- 172.21.15.47:0/183776339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1b0003970 0x7fb1b0003d50 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.506+0000 7fb1c8a41640 1 -- 172.21.15.47:0/183776339 >> 172.21.15.47:0/183776339 conn(0x7fb1c406b160 msgr2=0x7fb1c406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.506+0000 7fb1c8a41640 1 -- 172.21.15.47:0/183776339 shutdown_connections 2024-03-23T09:45:27.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.506+0000 7fb1c8a41640 1 -- 172.21.15.47:0/183776339 wait complete. 2024-03-23T09:45:27.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.507+0000 7fb1c8a41640 1 Processor -- start 2024-03-23T09:45:27.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.508+0000 7fb1c8a41640 1 -- start start 2024-03-23T09:45:27.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.508+0000 7fb1c8a41640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b0003970 0x7fb1c40d2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.508+0000 7fb1c8a41640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1c40cb940 0x7fb1c415a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.508+0000 7fb1c8a41640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1c41f4f40 0x7fb1c415af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.508+0000 7fb1c8a41640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4158f70 con 0x7fb1c40cb940 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.508+0000 7fb1c8a41640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40b52e0 con 0x7fb1b0003970 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.508+0000 7fb1c8a41640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c415ff90 con 0x7fb1c41f4f40 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.508+0000 7fb1c1d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1c40cb940 0x7fb1c415a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.508+0000 7fb1c1d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1c40cb940 0x7fb1c415a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36004/0 (socket says 172.21.15.47:36004) 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.508+0000 7fb1c1d9b640 1 -- 172.21.15.47:0/2332840115 learned_addr learned my addr 172.21.15.47:0/2332840115 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.509+0000 7fb1c2d9d640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1c41f4f40 0x7fb1c415af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.509+0000 7fb1c259c640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b0003970 0x7fb1c40d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.509+0000 7fb1c1d9b640 1 -- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1c41f4f40 msgr2=0x7fb1c415af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.509+0000 7fb1c1d9b640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1c41f4f40 0x7fb1c415af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.509+0000 7fb1c1d9b640 1 -- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b0003970 msgr2=0x7fb1c40d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.509+0000 7fb1c1d9b640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b0003970 0x7fb1c40d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.509+0000 7fb1c1d9b640 1 -- 172.21.15.47:0/2332840115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c40b4d20 con 0x7fb1c40cb940 2024-03-23T09:45:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.509+0000 7fb1c1d9b640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1c40cb940 0x7fb1c415a9c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0072f0 tx=0x7fb1ac0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.509+0000 7fb1baffd640 1 -- 172.21.15.47:0/2332840115 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00f8e0 con 0x7fb1c40cb940 2024-03-23T09:45:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.509+0000 7fb1baffd640 1 -- 172.21.15.47:0/2332840115 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac00fa80 con 0x7fb1c40cb940 2024-03-23T09:45:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.509+0000 7fb1c8a41640 1 -- 172.21.15.47:0/2332840115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c40b5000 con 0x7fb1c40cb940 2024-03-23T09:45:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.509+0000 7fb1baffd640 1 -- 172.21.15.47:0/2332840115 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00fe30 con 0x7fb1c40cb940 2024-03-23T09:45:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.509+0000 7fb1c8a41640 1 -- 172.21.15.47:0/2332840115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c4159390 con 0x7fb1c40cb940 2024-03-23T09:45:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.510+0000 7fb1c8a41640 1 -- 172.21.15.47:0/2332840115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c4072360 con 0x7fb1c40cb940 2024-03-23T09:45:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.510+0000 7fb1baffd640 1 -- 172.21.15.47:0/2332840115 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb1ac01d460 con 0x7fb1c40cb940 2024-03-23T09:45:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.511+0000 7fb1baffd640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb1a406ac00 0x7fb1a406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.511+0000 7fb1baffd640 1 -- 172.21.15.47:0/2332840115 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1ac089360 con 0x7fb1c40cb940 2024-03-23T09:45:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.511+0000 7fb1c259c640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb1a406ac00 0x7fb1a406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.512+0000 7fb1c259c640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb1a406ac00 0x7fb1a406d0c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc035000 tx=0x7fb1bc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.515+0000 7fb1baffd640 1 -- 172.21.15.47:0/2332840115 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb1ac054680 con 0x7fb1c40cb940 2024-03-23T09:45:27.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.732+0000 7fb1c8a41640 1 -- 172.21.15.47:0/2332840115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb1c4066be0 con 0x7fb1c40cb940 2024-03-23T09:45:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.733+0000 7fb1baffd640 1 -- 172.21.15.47:0/2332840115 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb1ac057ca0 con 0x7fb1c40cb940 2024-03-23T09:45:27.733 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:45:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.735+0000 7fb1c8a41640 1 -- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb1a406ac00 msgr2=0x7fb1a406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.735+0000 7fb1c8a41640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb1a406ac00 0x7fb1a406d0c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc035000 tx=0x7fb1bc030000 comp rx=0 tx=0).stop 2024-03-23T09:45:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.735+0000 7fb1c8a41640 1 -- 172.21.15.47:0/2332840115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1c40cb940 msgr2=0x7fb1c415a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.735+0000 7fb1c8a41640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1c40cb940 0x7fb1c415a9c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0072f0 tx=0x7fb1ac0077b0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.735+0000 7fb1c8a41640 1 -- 172.21.15.47:0/2332840115 shutdown_connections 2024-03-23T09:45:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.735+0000 7fb1c8a41640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb1a406ac00 0x7fb1a406d0c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.735+0000 7fb1c8a41640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1c41f4f40 0x7fb1c415af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.735+0000 7fb1c8a41640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1c40cb940 0x7fb1c415a9c0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.735+0000 7fb1c8a41640 1 --2- 172.21.15.47:0/2332840115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b0003970 0x7fb1c40d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.735+0000 7fb1c8a41640 1 -- 172.21.15.47:0/2332840115 >> 172.21.15.47:0/2332840115 conn(0x7fb1c406b160 msgr2=0x7fb1c4156e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.735+0000 7fb1c8a41640 1 -- 172.21.15.47:0/2332840115 shutdown_connections 2024-03-23T09:45:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:27.735+0000 7fb1c8a41640 1 -- 172.21.15.47:0/2332840115 wait complete. 2024-03-23T09:45:27.745 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:45:27.745 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:45:27.746 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:45:27.746 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:45:27.746 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:27.746 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:45:27.746 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:45:27.746 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:45:27.746 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:45:27.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:27.746 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:45:27.761 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:45:27.762 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:45:27.762 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:45:27.762 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:27.762 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:45:27.762 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:27.762 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:45:27.762 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:45:27.762 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:45:27.762 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:45:27.762 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:45:27.762 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:45:27.762 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42411sec preferred_lft 42411sec 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:45:27.763 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:27.764 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:45:27.764 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:45:27.764 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:45:27.764 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:27.764 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:45:27.764 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:45:27.764 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:45:27.764 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:27.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:27.764 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:45:27.827 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:45:27.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:27.829 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:45:27.884 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:45:27.884 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:45:27.884 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:45:27.884 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:45:27.884 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:45:27.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:27.885 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:45:27.885 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:45:27.885 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:45:27.885 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:45:27.885 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:45:27.885 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:45:27.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:27.980 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:45:27.994 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:45:27.994 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:45:27.994 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:45:27.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:27.994 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:45:28.060 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:45:28.061 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:45:28.061 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:28.115 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:45:28.115 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:45:28.116 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:28.170 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:45:28.237 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:45:28.237 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:45:28.303 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:45:28.304 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:45:28.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:28.305 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:45:28.359 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:45:28.359 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:45:28.402 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:45:28.429 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:45:28.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:45:28.488 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:45:28.488+0000 7f6b71c13480 -1 init, newargv = 0x55c265ef2fd0 newargc=15 2024-03-23T09:45:28.488 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[61509]: starting ceph client 2024-03-23T09:45:28.492 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[61509]: starting fuse 2024-03-23T09:45:28.498 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:45:28.499 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:45:28.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:28.500 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:45:28.555 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:45:28.555 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:45:28.555 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:45:28.555 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:45:28.555 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:45:28.555 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:45:28.556 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:45:28.557 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:45:28.557 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:45:28.557 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:45:28.557 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:45:28.557 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:45:28.671 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.61509.asok 2024-03-23T09:45:28.857 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.61509.asok 2024-03-23T09:45:28.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:28.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.61509.asok status 2024-03-23T09:45:28.995 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:45:28.995 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:45:28.995 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:45:28.995 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:45:28.995 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:45:28.995 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:45:28.995 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:45:28.995 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "61509", 2024-03-23T09:45:28.995 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:45:28.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:28.995 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:45:28.995 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 6924, 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 6924 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 376512733 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:45:28.996 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 376512733 2024-03-23T09:45:28.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:28.997 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.6924 192.168.0.1:0/376512733", 2024-03-23T09:45:28.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/376512733", 2024-03-23T09:45:28.997 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:45:28.997 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 32, 2024-03-23T09:45:28.997 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 70, 2024-03-23T09:45:28.997 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:45:28.997 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:45:28.997 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:45:28.997 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:45:29.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:29.003 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:45:29.003 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:29.018 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:45:29.018 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:29.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:29.018 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:29.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:29.085 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-03-23T09:45:29.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:29.145 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-03-23T09:45:29.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:29.202 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-03-23T09:45:29.260 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:45:29.260 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:45:29.260 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:45:29.260 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:45:29.261 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-03-23T09:45:29.261 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:45:29.361 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627776 2024-03-23T09:45:29.551 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:45:29.551 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:45:29.606 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:45:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.747+0000 7f7796d63640 1 Processor -- start 2024-03-23T09:45:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.747+0000 7f7796d63640 1 -- start start 2024-03-23T09:45:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f7796d63640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f779015a940 0x7f779006ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f7796d63640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f77900cc010 0x7f779006f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f7796d63640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f779006fb00 0x7f779015e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f7796d63640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77900cd0b0 con 0x7f77900cc010 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f7796d63640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77900cc4a0 con 0x7f779015a940 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f7796d63640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77900cc780 con 0x7f779006fb00 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f778ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f77900cc010 0x7f779006f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f778ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f77900cc010 0x7f779006f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36012/0 (socket says 172.21.15.47:36012) 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f778ffff640 1 -- 172.21.15.47:0/1626253418 learned_addr learned my addr 172.21.15.47:0/1626253418 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f7795300640 1 --2- 172.21.15.47:0/1626253418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f779006fb00 0x7f779015e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f7794aff640 1 --2- 172.21.15.47:0/1626253418 >> v2:172.21.15.175:3300/0 conn(0x7f779015a940 0x7f779006ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f778ffff640 1 -- 172.21.15.47:0/1626253418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f779006fb00 msgr2=0x7f779015e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f778ffff640 1 --2- 172.21.15.47:0/1626253418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f779006fb00 0x7f779015e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f778ffff640 1 -- 172.21.15.47:0/1626253418 >> v2:172.21.15.175:3300/0 conn(0x7f779015a940 msgr2=0x7f779006ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f778ffff640 1 --2- 172.21.15.47:0/1626253418 >> v2:172.21.15.175:3300/0 conn(0x7f779015a940 0x7f779006ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f778ffff640 1 -- 172.21.15.47:0/1626253418 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77900cca60 con 0x7f77900cc010 2024-03-23T09:45:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f7795300640 1 --2- 172.21.15.47:0/1626253418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f779006fb00 0x7f779015e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.748+0000 7f7794aff640 1 --2- 172.21.15.47:0/1626253418 >> v2:172.21.15.175:3300/0 conn(0x7f779015a940 0x7f779006ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:29.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.749+0000 7f778ffff640 1 --2- 172.21.15.47:0/1626253418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77900cc010 0x7f779006f2d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f779006ada0 tx=0x7f777c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf7523544919fe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:29.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.749+0000 7f778f7fe640 1 -- 172.21.15.47:0/1626253418 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c031870 con 0x7f77900cc010 2024-03-23T09:45:29.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.749+0000 7f778f7fe640 1 -- 172.21.15.47:0/1626253418 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c0319d0 con 0x7f77900cc010 2024-03-23T09:45:29.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.749+0000 7f778f7fe640 1 -- 172.21.15.47:0/1626253418 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c031d60 con 0x7f77900cc010 2024-03-23T09:45:29.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.750+0000 7f7796d63640 1 -- 172.21.15.47:0/1626253418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77900cc010 msgr2=0x7f779006f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:29.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.750+0000 7f7796d63640 1 --2- 172.21.15.47:0/1626253418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77900cc010 0x7f779006f2d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f779006ada0 tx=0x7f777c02b910 comp rx=0 tx=0).stop 2024-03-23T09:45:29.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.750+0000 7f7796d63640 1 -- 172.21.15.47:0/1626253418 shutdown_connections 2024-03-23T09:45:29.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.750+0000 7f7796d63640 1 --2- 172.21.15.47:0/1626253418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f779006fb00 0x7f779015e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.750+0000 7f7796d63640 1 --2- 172.21.15.47:0/1626253418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77900cc010 0x7f779006f2d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.750+0000 7f7796d63640 1 --2- 172.21.15.47:0/1626253418 >> v2:172.21.15.175:3300/0 conn(0x7f779015a940 0x7f779006ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.750+0000 7f7796d63640 1 -- 172.21.15.47:0/1626253418 >> 172.21.15.47:0/1626253418 conn(0x7f77900b13a0 msgr2=0x7f77900b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.750+0000 7f7796d63640 1 -- 172.21.15.47:0/1626253418 shutdown_connections 2024-03-23T09:45:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.750+0000 7f7796d63640 1 -- 172.21.15.47:0/1626253418 wait complete. 2024-03-23T09:45:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.751+0000 7f7796d63640 1 Processor -- start 2024-03-23T09:45:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.751+0000 7f7796d63640 1 -- start start 2024-03-23T09:45:29.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f7796d63640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f779006fb00 0x7f77900ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:29.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f7796d63640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f779015a940 0x7f7790156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:29.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f7796d63640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7790159e80 0x7f7790144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:29.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f7796d63640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77900acd80 con 0x7f779015a940 2024-03-23T09:45:29.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f7796d63640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77900ad210 con 0x7f779006fb00 2024-03-23T09:45:29.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f7796d63640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77900ac170 con 0x7f7790159e80 2024-03-23T09:45:29.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f778ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f779015a940 0x7f7790156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:29.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f778ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f779015a940 0x7f7790156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36024/0 (socket says 172.21.15.47:36024) 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f778ffff640 1 -- 172.21.15.47:0/644652133 learned_addr learned my addr 172.21.15.47:0/644652133 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f7795300640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7790159e80 0x7f7790144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f7794aff640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f779006fb00 0x7f77900ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f778ffff640 1 -- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7790159e80 msgr2=0x7f7790144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f778ffff640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7790159e80 0x7f7790144820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f778ffff640 1 -- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f779006fb00 msgr2=0x7f77900ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f778ffff640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f779006fb00 0x7f77900ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f778ffff640 1 -- 172.21.15.47:0/644652133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c0094d0 con 0x7f779015a940 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.752+0000 7f778ffff640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f779015a940 0x7f7790156d50 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f777c02b950 tx=0x7f777c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.753+0000 7f778d7fa640 1 -- 172.21.15.47:0/644652133 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c031820 con 0x7f779015a940 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.753+0000 7f778d7fa640 1 -- 172.21.15.47:0/644652133 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c0443f0 con 0x7f779015a940 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.753+0000 7f7796d63640 1 -- 172.21.15.47:0/644652133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779015aea0 con 0x7f779015a940 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.753+0000 7f778d7fa640 1 -- 172.21.15.47:0/644652133 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0447a0 con 0x7f779015a940 2024-03-23T09:45:29.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.753+0000 7f7796d63640 1 -- 172.21.15.47:0/644652133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77900ac730 con 0x7f779015a940 2024-03-23T09:45:29.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.754+0000 7f7796d63640 1 -- 172.21.15.47:0/644652133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7790071c50 con 0x7f779015a940 2024-03-23T09:45:29.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.754+0000 7f778d7fa640 1 -- 172.21.15.47:0/644652133 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f777c04d450 con 0x7f779015a940 2024-03-23T09:45:29.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.755+0000 7f778d7fa640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f777406ac00 0x7f777406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:29.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.755+0000 7f778d7fa640 1 -- 172.21.15.47:0/644652133 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f777c02d040 con 0x7f779015a940 2024-03-23T09:45:29.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.755+0000 7f7794aff640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f777406ac00 0x7f777406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:29.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.755+0000 7f7794aff640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f777406ac00 0x7f777406d0c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f778400aac0 tx=0x7f7784008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:29.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.759+0000 7f778d7fa640 1 -- 172.21.15.47:0/644652133 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f777c076c70 con 0x7f779015a940 2024-03-23T09:45:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.939+0000 7f7796d63640 1 -- 172.21.15.47:0/644652133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7790066be0 con 0x7f779015a940 2024-03-23T09:45:29.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.940+0000 7f778d7fa640 1 -- 172.21.15.47:0/644652133 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16100 (secure 0 0 0) 0x7f777c035090 con 0x7f779015a940 2024-03-23T09:45:29.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:29.940 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":70,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:45:19.298605+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:45:16.609941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-23T09:45:17.261503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:45:29.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.942+0000 7f7796d63640 1 -- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f777406ac00 msgr2=0x7f777406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:29.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.942+0000 7f7796d63640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f777406ac00 0x7f777406d0c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f778400aac0 tx=0x7f7784008040 comp rx=0 tx=0).stop 2024-03-23T09:45:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.942+0000 7f7796d63640 1 -- 172.21.15.47:0/644652133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f779015a940 msgr2=0x7f7790156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.942+0000 7f7796d63640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f779015a940 0x7f7790156d50 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f777c02b950 tx=0x7f777c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.942+0000 7f7796d63640 1 -- 172.21.15.47:0/644652133 shutdown_connections 2024-03-23T09:45:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.942+0000 7f7796d63640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f777406ac00 0x7f777406d0c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.942+0000 7f7796d63640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7790159e80 0x7f7790144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.942+0000 7f7796d63640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f779015a940 0x7f7790156d50 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.942+0000 7f7796d63640 1 --2- 172.21.15.47:0/644652133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f779006fb00 0x7f77900ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.942+0000 7f7796d63640 1 -- 172.21.15.47:0/644652133 >> 172.21.15.47:0/644652133 conn(0x7f77900b13a0 msgr2=0x7f7790155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.942+0000 7f7796d63640 1 -- 172.21.15.47:0/644652133 shutdown_connections 2024-03-23T09:45:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:29.942+0000 7f7796d63640 1 -- 172.21.15.47:0/644652133 wait complete. 2024-03-23T09:45:29.953 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:45:16.609941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:45:17.261503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:45:29.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:45:30.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.057+0000 7f7a3f2db640 1 Processor -- start 2024-03-23T09:45:30.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.058+0000 7f7a3f2db640 1 -- start start 2024-03-23T09:45:30.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.058+0000 7f7a3f2db640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7a3815ca30 0x7f7a38160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.058+0000 7f7a3f2db640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a380cde40 0x7f7a380d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.058+0000 7f7a3f2db640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a38161600 con 0x7f7a3815ce10 2024-03-23T09:45:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.058+0000 7f7a3f2db640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a38161040 con 0x7f7a3815ca30 2024-03-23T09:45:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.058+0000 7f7a3f2db640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38161320 con 0x7f7a380cde40 2024-03-23T09:45:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.058+0000 7f7a3c876640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7a3815ce10 0x7f7a380cd730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41304/0 (socket says 172.21.15.47:41304) 2024-03-23T09:45:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.058+0000 7f7a3c876640 1 -- 172.21.15.47:0/3006645038 learned_addr learned my addr 172.21.15.47:0/3006645038 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a3d878640 1 --2- 172.21.15.47:0/3006645038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a380cde40 0x7f7a380d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a3d077640 1 --2- 172.21.15.47:0/3006645038 >> v2:172.21.15.175:3300/0 conn(0x7f7a3815ca30 0x7f7a38160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930942291 0 0) 0x7f7a38161600 con 0x7f7a3815ce10 2024-03-23T09:45:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a20003520 con 0x7f7a3815ce10 2024-03-23T09:45:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4016792187 0 0) 0x7f7a20003520 con 0x7f7a3815ce10 2024-03-23T09:45:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a380cde40 msgr2=0x7f7a380d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 --2- 172.21.15.47:0/3006645038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a380cde40 0x7f7a380d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 >> v2:172.21.15.175:3300/0 conn(0x7f7a3815ca30 msgr2=0x7f7a38160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 --2- 172.21.15.47:0/3006645038 >> v2:172.21.15.175:3300/0 conn(0x7f7a3815ca30 0x7f7a38160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a380d0dd0 con 0x7f7a3815ce10 2024-03-23T09:45:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f7a2c001670 con 0x7f7a3815ce10 2024-03-23T09:45:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a3d077640 1 --2- 172.21.15.47:0/3006645038 >> v2:172.21.15.175:3300/0 conn(0x7f7a3815ca30 0x7f7a38160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:45:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 >> v1:172.21.15.47:6789/0 conn(0x7f7a3815ce10 legacy=0x7f7a380cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 --2- 172.21.15.47:0/3006645038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a20003970 0x7f7a20003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 --2- 172.21.15.47:0/3006645038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a380cde40 0x7f7a200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 --2- 172.21.15.47:0/3006645038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a20004d00 0x7f7a20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38161040 con 0x7f7a20004d00 2024-03-23T09:45:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38161320 con 0x7f7a20003970 2024-03-23T09:45:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2000b980 con 0x7f7a380cde40 2024-03-23T09:45:30.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a3d878640 1 --2- 172.21.15.47:0/3006645038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a20004d00 0x7f7a20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a3d077640 1 --2- 172.21.15.47:0/3006645038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a20003970 0x7f7a20003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.059+0000 7f7a3c876640 1 --2- 172.21.15.47:0/3006645038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a380cde40 0x7f7a200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.060+0000 7f7a3d878640 1 -- 172.21.15.47:0/3006645038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a380cde40 msgr2=0x7f7a200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:30.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.060+0000 7f7a3d878640 1 --2- 172.21.15.47:0/3006645038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a380cde40 0x7f7a200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.060+0000 7f7a3d878640 1 -- 172.21.15.47:0/3006645038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a20003970 msgr2=0x7f7a20003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:30.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.060+0000 7f7a3d878640 1 --2- 172.21.15.47:0/3006645038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a20003970 0x7f7a20003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.060+0000 7f7a3d878640 1 -- 172.21.15.47:0/3006645038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a20003520 con 0x7f7a20004d00 2024-03-23T09:45:30.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.060+0000 7f7a3d878640 1 --2- 172.21.15.47:0/3006645038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a20004d00 0x7f7a20005180 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009ac0 tx=0x7f7a200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69add9088251ca9a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:30.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.060+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3002c680 con 0x7f7a20004d00 2024-03-23T09:45:30.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.060+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3002c7e0 con 0x7f7a20004d00 2024-03-23T09:45:30.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.060+0000 7f7a37fff640 1 -- 172.21.15.47:0/3006645038 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3002cb90 con 0x7f7a20004d00 2024-03-23T09:45:30.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.061+0000 7f7a3f2db640 1 -- 172.21.15.47:0/3006645038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a20004d00 msgr2=0x7f7a20005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.061+0000 7f7a3f2db640 1 --2- 172.21.15.47:0/3006645038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a20004d00 0x7f7a20005180 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009ac0 tx=0x7f7a200057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.061+0000 7f7a3d878640 1 -- 172.21.15.47:0/3006645038 reap_dead start 2024-03-23T09:45:30.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.061+0000 7f7a3f2db640 1 -- 172.21.15.47:0/3006645038 shutdown_connections 2024-03-23T09:45:30.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.061+0000 7f7a3f2db640 1 -- 172.21.15.47:0/3006645038 >> 172.21.15.47:0/3006645038 conn(0x7f7a3806b160 msgr2=0x7f7a3806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:30.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.061+0000 7f7a3f2db640 1 -- 172.21.15.47:0/3006645038 shutdown_connections 2024-03-23T09:45:30.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.061+0000 7f7a3f2db640 1 -- 172.21.15.47:0/3006645038 wait complete. 2024-03-23T09:45:30.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.062+0000 7f7a3f2db640 1 Processor -- start 2024-03-23T09:45:30.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.062+0000 7f7a3f2db640 1 -- start start 2024-03-23T09:45:30.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3f2db640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a38152cd0 0x7f7a3815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3f2db640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a381583c0 0x7f7a380c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3f2db640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a38155940 0x7f7a38155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3f2db640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38156300 con 0x7f7a38155940 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3f2db640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380c7620 con 0x7f7a381583c0 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3f2db640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380c78c0 con 0x7f7a38152cd0 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3d878640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a38155940 0x7f7a38155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3d878640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a38155940 0x7f7a38155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36034/0 (socket says 172.21.15.47:36034) 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3d878640 1 -- 172.21.15.47:0/804985680 learned_addr learned my addr 172.21.15.47:0/804985680 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3d077640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a38152cd0 0x7f7a3815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3c876640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a381583c0 0x7f7a380c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3d878640 1 -- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a38152cd0 msgr2=0x7f7a3815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3d878640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a38152cd0 0x7f7a3815bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3d878640 1 -- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a381583c0 msgr2=0x7f7a380c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3d878640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a381583c0 0x7f7a380c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.063+0000 7f7a3d878640 1 -- 172.21.15.47:0/804985680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3815e7b0 con 0x7f7a38155940 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.064+0000 7f7a3d878640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a38155940 0x7f7a38155dc0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7a30037040 tx=0x7f7a30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.064+0000 7f7a35ffb640 1 -- 172.21.15.47:0/804985680 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3002c680 con 0x7f7a38155940 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.064+0000 7f7a35ffb640 1 -- 172.21.15.47:0/804985680 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3003e870 con 0x7f7a38155940 2024-03-23T09:45:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.064+0000 7f7a3f2db640 1 -- 172.21.15.47:0/804985680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3815ea90 con 0x7f7a38155940 2024-03-23T09:45:30.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.064+0000 7f7a35ffb640 1 -- 172.21.15.47:0/804985680 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3003ec20 con 0x7f7a38155940 2024-03-23T09:45:30.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.064+0000 7f7a3f2db640 1 -- 172.21.15.47:0/804985680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a380c7ba0 con 0x7f7a38155940 2024-03-23T09:45:30.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.065+0000 7f7a3f2db640 1 -- 172.21.15.47:0/804985680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a38072360 con 0x7f7a38155940 2024-03-23T09:45:30.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.066+0000 7f7a35ffb640 1 -- 172.21.15.47:0/804985680 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7a30034440 con 0x7f7a38155940 2024-03-23T09:45:30.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.066+0000 7f7a35ffb640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a2406ac00 0x7f7a2406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.066+0000 7f7a35ffb640 1 -- 172.21.15.47:0/804985680 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a300aaa40 con 0x7f7a38155940 2024-03-23T09:45:30.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.066+0000 7f7a3d077640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a2406ac00 0x7f7a2406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.067+0000 7f7a3d077640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a2406ac00 0x7f7a2406d0c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009fd0 tx=0x7f7a2800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:30.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.070+0000 7f7a35ffb640 1 -- 172.21.15.47:0/804985680 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7a300ace60 con 0x7f7a38155940 2024-03-23T09:45:30.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.253+0000 7f7a3f2db640 1 -- 172.21.15.47:0/804985680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a38072360 con 0x7f7a38155940 2024-03-23T09:45:30.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.253+0000 7f7a35ffb640 1 -- 172.21.15.47:0/804985680 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a30039070 con 0x7f7a38155940 2024-03-23T09:45:30.253 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:45:30.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.255+0000 7f7a3f2db640 1 -- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a2406ac00 msgr2=0x7f7a2406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.255+0000 7f7a3f2db640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a2406ac00 0x7f7a2406d0c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009fd0 tx=0x7f7a2800b040 comp rx=0 tx=0).stop 2024-03-23T09:45:30.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.255+0000 7f7a3f2db640 1 -- 172.21.15.47:0/804985680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a38155940 msgr2=0x7f7a38155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.255+0000 7f7a3f2db640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a38155940 0x7f7a38155dc0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7a30037040 tx=0x7f7a30004730 comp rx=0 tx=0).stop 2024-03-23T09:45:30.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.255+0000 7f7a3f2db640 1 -- 172.21.15.47:0/804985680 shutdown_connections 2024-03-23T09:45:30.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.255+0000 7f7a3f2db640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a2406ac00 0x7f7a2406d0c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.255+0000 7f7a3f2db640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a38155940 0x7f7a38155dc0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.255+0000 7f7a3f2db640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a381583c0 0x7f7a380c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.255+0000 7f7a3f2db640 1 --2- 172.21.15.47:0/804985680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a38152cd0 0x7f7a3815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.255+0000 7f7a3f2db640 1 -- 172.21.15.47:0/804985680 >> 172.21.15.47:0/804985680 conn(0x7f7a3806b160 msgr2=0x7f7a38154050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:30.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.255+0000 7f7a3f2db640 1 -- 172.21.15.47:0/804985680 shutdown_connections 2024-03-23T09:45:30.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.255+0000 7f7a3f2db640 1 -- 172.21.15.47:0/804985680 wait complete. 2024-03-23T09:45:30.266 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:45:30.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.369+0000 7f307e1fd640 1 Processor -- start 2024-03-23T09:45:30.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.369+0000 7f307e1fd640 1 -- start start 2024-03-23T09:45:30.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.370+0000 7f307e1fd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f307815ca30 0x7f3078160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.370+0000 7f307e1fd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f307815ce10 0x7f30780cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.370+0000 7f307e1fd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30780d25e0 con 0x7f30780b85d0 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.370+0000 7f307e1fd640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3078161600 con 0x7f307815ca30 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.370+0000 7f307e1fd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3078161040 con 0x7f307815ce10 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.370+0000 7f30777fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f307815ca30 0x7f3078160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.370+0000 7f3077fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f30780b85d0 0x7f30780b43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41320/0 (socket says 172.21.15.47:41320) 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.370+0000 7f3077fff640 1 -- 172.21.15.47:0/2981577284 learned_addr learned my addr 172.21.15.47:0/2981577284 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.370+0000 7f30767fc640 1 -- 172.21.15.47:0/2981577284 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 461536924 0 0) 0x7f30780d25e0 con 0x7f30780b85d0 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.370+0000 7f3076ffd640 1 --2- 172.21.15.47:0/2981577284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f307815ce10 0x7f30780cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.370+0000 7f30767fc640 1 -- 172.21.15.47:0/2981577284 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3060003520 con 0x7f30780b85d0 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.371+0000 7f30777fe640 1 -- 172.21.15.47:0/2981577284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f307815ce10 msgr2=0x7f30780cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.371+0000 7f30777fe640 1 --2- 172.21.15.47:0/2981577284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f307815ce10 0x7f30780cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.371+0000 7f30777fe640 1 -- 172.21.15.47:0/2981577284 >> v1:172.21.15.47:6789/0 conn(0x7f30780b85d0 legacy=0x7f30780b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.371+0000 7f30777fe640 1 -- 172.21.15.47:0/2981577284 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3078161320 con 0x7f307815ca30 2024-03-23T09:45:30.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.371+0000 7f30777fe640 1 --2- 172.21.15.47:0/2981577284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307815ca30 0x7f3078160740 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f30680097a0 tx=0x7f306802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cd3c4ddd3327d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:30.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.371+0000 7f30767fc640 1 -- 172.21.15.47:0/2981577284 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068031830 con 0x7f307815ca30 2024-03-23T09:45:30.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.371+0000 7f30767fc640 1 -- 172.21.15.47:0/2981577284 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3068031990 con 0x7f307815ca30 2024-03-23T09:45:30.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.371+0000 7f30767fc640 1 -- 172.21.15.47:0/2981577284 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068031d20 con 0x7f307815ca30 2024-03-23T09:45:30.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.372+0000 7f307e1fd640 1 -- 172.21.15.47:0/2981577284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307815ca30 msgr2=0x7f3078160740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.372+0000 7f307e1fd640 1 --2- 172.21.15.47:0/2981577284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307815ca30 0x7f3078160740 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f30680097a0 tx=0x7f306802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.372+0000 7f307e1fd640 1 -- 172.21.15.47:0/2981577284 shutdown_connections 2024-03-23T09:45:30.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.372+0000 7f307e1fd640 1 --2- 172.21.15.47:0/2981577284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f307815ce10 0x7f30780cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.372+0000 7f307e1fd640 1 --2- 172.21.15.47:0/2981577284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307815ca30 0x7f3078160740 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.372+0000 7f307e1fd640 1 -- 172.21.15.47:0/2981577284 >> 172.21.15.47:0/2981577284 conn(0x7f307806b160 msgr2=0x7f307806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:30.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.372+0000 7f307e1fd640 1 -- 172.21.15.47:0/2981577284 shutdown_connections 2024-03-23T09:45:30.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.372+0000 7f307e1fd640 1 -- 172.21.15.47:0/2981577284 wait complete. 2024-03-23T09:45:30.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.373+0000 7f307e1fd640 1 Processor -- start 2024-03-23T09:45:30.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.373+0000 7f307e1fd640 1 -- start start 2024-03-23T09:45:30.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f307e1fd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30780b85d0 0x7f30781f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f307e1fd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307815ca30 0x7f307815bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f307e1fd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f307815ce10 0x7f3078151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f307e1fd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30781f2330 con 0x7f307815ca30 2024-03-23T09:45:30.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f307e1fd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307815f180 con 0x7f30780b85d0 2024-03-23T09:45:30.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f307e1fd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307815c1c0 con 0x7f307815ce10 2024-03-23T09:45:30.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f3076ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307815ca30 0x7f307815bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f3076ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307815ca30 0x7f307815bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36058/0 (socket says 172.21.15.47:36058) 2024-03-23T09:45:30.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f3076ffd640 1 -- 172.21.15.47:0/1605754226 learned_addr learned my addr 172.21.15.47:0/1605754226 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:30.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f3077fff640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f307815ce10 0x7f3078151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f30777fe640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30780b85d0 0x7f30781f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f3076ffd640 1 -- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f307815ce10 msgr2=0x7f3078151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f3076ffd640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f307815ce10 0x7f3078151850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f3076ffd640 1 -- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30780b85d0 msgr2=0x7f30781f1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f3076ffd640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30780b85d0 0x7f30781f1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.374+0000 7f3076ffd640 1 -- 172.21.15.47:0/1605754226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3068009400 con 0x7f307815ca30 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.375+0000 7f30777fe640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30780b85d0 0x7f30781f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.375+0000 7f3077fff640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f307815ce10 0x7f3078151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.375+0000 7f3076ffd640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307815ca30 0x7f307815bc80 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f306c00d5c0 tx=0x7f306c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.375+0000 7f305ffff640 1 -- 172.21.15.47:0/1605754226 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c0125f0 con 0x7f307815ca30 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.375+0000 7f305ffff640 1 -- 172.21.15.47:0/1605754226 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c012750 con 0x7f307815ca30 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.375+0000 7f307e1fd640 1 -- 172.21.15.47:0/1605754226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307815e290 con 0x7f307815ca30 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.375+0000 7f305ffff640 1 -- 172.21.15.47:0/1605754226 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c012b00 con 0x7f307815ca30 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.375+0000 7f307e1fd640 1 -- 172.21.15.47:0/1605754226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307815e850 con 0x7f307815ca30 2024-03-23T09:45:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.376+0000 7f307e1fd640 1 -- 172.21.15.47:0/1605754226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3078072360 con 0x7f307815ca30 2024-03-23T09:45:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.376+0000 7f305ffff640 1 -- 172.21.15.47:0/1605754226 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f306c012c60 con 0x7f307815ca30 2024-03-23T09:45:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.377+0000 7f305ffff640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f306406ac30 0x7f306406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.377+0000 7f305ffff640 1 -- 172.21.15.47:0/1605754226 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f306c089840 con 0x7f307815ca30 2024-03-23T09:45:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.377+0000 7f30777fe640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f306406ac30 0x7f306406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.377+0000 7f30777fe640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f306406ac30 0x7f306406d0f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3068036000 tx=0x7f306803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:30.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.381+0000 7f305ffff640 1 -- 172.21.15.47:0/1605754226 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f306c054a20 con 0x7f307815ca30 2024-03-23T09:45:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.559+0000 7f307e1fd640 1 -- 172.21.15.47:0/1605754226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3078066be0 con 0x7f307815ca30 2024-03-23T09:45:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.559+0000 7f305ffff640 1 -- 172.21.15.47:0/1605754226 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f306c058040 con 0x7f307815ca30 2024-03-23T09:45:30.559 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:45:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.561+0000 7f307e1fd640 1 -- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f306406ac30 msgr2=0x7f306406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.561+0000 7f307e1fd640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f306406ac30 0x7f306406d0f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3068036000 tx=0x7f306803c040 comp rx=0 tx=0).stop 2024-03-23T09:45:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.561+0000 7f307e1fd640 1 -- 172.21.15.47:0/1605754226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307815ca30 msgr2=0x7f307815bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.561+0000 7f307e1fd640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307815ca30 0x7f307815bc80 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f306c00d5c0 tx=0x7f306c00da80 comp rx=0 tx=0).stop 2024-03-23T09:45:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.561+0000 7f307e1fd640 1 -- 172.21.15.47:0/1605754226 shutdown_connections 2024-03-23T09:45:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.561+0000 7f307e1fd640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f306406ac30 0x7f306406d0f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.561+0000 7f307e1fd640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f307815ce10 0x7f3078151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.561+0000 7f307e1fd640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307815ca30 0x7f307815bc80 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.561+0000 7f307e1fd640 1 --2- 172.21.15.47:0/1605754226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30780b85d0 0x7f30781f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.561+0000 7f307e1fd640 1 -- 172.21.15.47:0/1605754226 >> 172.21.15.47:0/1605754226 conn(0x7f307806b160 msgr2=0x7f30780d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.561+0000 7f307e1fd640 1 -- 172.21.15.47:0/1605754226 shutdown_connections 2024-03-23T09:45:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.561+0000 7f307e1fd640 1 -- 172.21.15.47:0/1605754226 wait complete. 2024-03-23T09:45:30.571 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:45:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.674+0000 7ff5fb399640 1 Processor -- start 2024-03-23T09:45:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.675+0000 7ff5fb399640 1 -- start start 2024-03-23T09:45:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.675+0000 7ff5fb399640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff5f4099110 0x7ff5f40d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.675+0000 7ff5fb399640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff5f406b090 0x7ff5f40ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.675+0000 7ff5fb399640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f40cedd0 con 0x7ff5f415cd40 2024-03-23T09:45:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.675+0000 7ff5fb399640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f415d120 con 0x7ff5f406b090 2024-03-23T09:45:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.675+0000 7ff5fb399640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f4099760 con 0x7ff5f4099110 2024-03-23T09:45:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.675+0000 7ff5f9135640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff5f415cd40 0x7ff5f4098a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41324/0 (socket says 172.21.15.47:41324) 2024-03-23T09:45:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.675+0000 7ff5f9135640 1 -- 172.21.15.47:0/1706783447 learned_addr learned my addr 172.21.15.47:0/1706783447 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.675+0000 7ff5f9936640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff5f406b090 0x7ff5f40ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.676+0000 7ff5f8934640 1 --2- 172.21.15.47:0/1706783447 >> v2:172.21.15.175:3300/0 conn(0x7ff5f4099110 0x7ff5f40d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.676+0000 7ff5f9936640 1 -- 172.21.15.47:0/1706783447 >> v2:172.21.15.175:3300/0 conn(0x7ff5f4099110 msgr2=0x7ff5f40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.676+0000 7ff5f9936640 1 --2- 172.21.15.47:0/1706783447 >> v2:172.21.15.175:3300/0 conn(0x7ff5f4099110 0x7ff5f40d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.676+0000 7ff5f9936640 1 -- 172.21.15.47:0/1706783447 >> v1:172.21.15.47:6789/0 conn(0x7ff5f415cd40 legacy=0x7ff5f4098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.676+0000 7ff5f9936640 1 -- 172.21.15.47:0/1706783447 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f40cf020 con 0x7ff5f406b090 2024-03-23T09:45:30.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.676+0000 7ff5ebfff640 1 -- 172.21.15.47:0/1706783447 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013525959 0 0) 0x7ff5f40cedd0 con 0x7ff5f415cd40 2024-03-23T09:45:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.677+0000 7ff5f9936640 1 --2- 172.21.15.47:0/1706783447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f406b090 0x7ff5f40ce890 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff5f406ce00 tx=0x7ff5ec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7c310e1d036a46 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.677+0000 7ff5ebfff640 1 -- 172.21.15.47:0/1706783447 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec02cc80 con 0x7ff5f406b090 2024-03-23T09:45:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.677+0000 7ff5ebfff640 1 -- 172.21.15.47:0/1706783447 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec02cde0 con 0x7ff5f406b090 2024-03-23T09:45:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.677+0000 7ff5ebfff640 1 -- 172.21.15.47:0/1706783447 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec034660 con 0x7ff5f406b090 2024-03-23T09:45:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.677+0000 7ff5fb399640 1 -- 172.21.15.47:0/1706783447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f406b090 msgr2=0x7ff5f40ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.677+0000 7ff5fb399640 1 --2- 172.21.15.47:0/1706783447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f406b090 0x7ff5f40ce890 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff5f406ce00 tx=0x7ff5ec02bed0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.677+0000 7ff5fb399640 1 -- 172.21.15.47:0/1706783447 shutdown_connections 2024-03-23T09:45:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.677+0000 7ff5fb399640 1 --2- 172.21.15.47:0/1706783447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f406b090 0x7ff5f40ce890 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.677+0000 7ff5fb399640 1 --2- 172.21.15.47:0/1706783447 >> v2:172.21.15.175:3300/0 conn(0x7ff5f4099110 0x7ff5f40d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.677+0000 7ff5fb399640 1 -- 172.21.15.47:0/1706783447 >> 172.21.15.47:0/1706783447 conn(0x7ff5f40b2140 msgr2=0x7ff5f40b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.677+0000 7ff5fb399640 1 -- 172.21.15.47:0/1706783447 shutdown_connections 2024-03-23T09:45:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.677+0000 7ff5fb399640 1 -- 172.21.15.47:0/1706783447 wait complete. 2024-03-23T09:45:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.679+0000 7ff5fb399640 1 Processor -- start 2024-03-23T09:45:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.679+0000 7ff5fb399640 1 -- start start 2024-03-23T09:45:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.679+0000 7ff5fb399640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f406b090 0x7ff5f40adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.679+0000 7ff5fb399640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f4099110 0x7ff5f40b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.679+0000 7ff5fb399640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f415cd40 0x7ff5f414c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.679+0000 7ff5fb399640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f41557a0 con 0x7ff5f406b090 2024-03-23T09:45:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.679+0000 7ff5fb399640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f4155470 con 0x7ff5f4099110 2024-03-23T09:45:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.679+0000 7ff5fb399640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f40ae270 con 0x7ff5f415cd40 2024-03-23T09:45:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.679+0000 7ff5f9135640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f406b090 0x7ff5f40adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.679+0000 7ff5f9135640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f406b090 0x7ff5f40adc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36082/0 (socket says 172.21.15.47:36082) 2024-03-23T09:45:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.679+0000 7ff5f9135640 1 -- 172.21.15.47:0/1881998349 learned_addr learned my addr 172.21.15.47:0/1881998349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.680+0000 7ff5f9135640 1 -- 172.21.15.47:0/1881998349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f415cd40 msgr2=0x7ff5f414c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.680+0000 7ff5f9135640 1 --2- 172.21.15.47:0/1881998349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f415cd40 0x7ff5f414c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.680+0000 7ff5f9135640 1 -- 172.21.15.47:0/1881998349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f4099110 msgr2=0x7ff5f40b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.680+0000 7ff5f9135640 1 --2- 172.21.15.47:0/1881998349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f4099110 0x7ff5f40b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.680+0000 7ff5f9135640 1 -- 172.21.15.47:0/1881998349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec009a70 con 0x7ff5f406b090 2024-03-23T09:45:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.680+0000 7ff5f9135640 1 --2- 172.21.15.47:0/1881998349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f406b090 0x7ff5f40adc90 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000c430 tx=0x7ff5f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.680+0000 7ff5e9ffb640 1 -- 172.21.15.47:0/1881998349 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f00074e0 con 0x7ff5f406b090 2024-03-23T09:45:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.680+0000 7ff5e9ffb640 1 -- 172.21.15.47:0/1881998349 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f0007660 con 0x7ff5f406b090 2024-03-23T09:45:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.680+0000 7ff5fb399640 1 -- 172.21.15.47:0/1881998349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f40ae550 con 0x7ff5f406b090 2024-03-23T09:45:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.680+0000 7ff5e9ffb640 1 -- 172.21.15.47:0/1881998349 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0004a20 con 0x7ff5f406b090 2024-03-23T09:45:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.680+0000 7ff5fb399640 1 -- 172.21.15.47:0/1881998349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f40ae830 con 0x7ff5f406b090 2024-03-23T09:45:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.681+0000 7ff5fb399640 1 -- 172.21.15.47:0/1881998349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f40d00c0 con 0x7ff5f406b090 2024-03-23T09:45:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.682+0000 7ff5e9ffb640 1 -- 172.21.15.47:0/1881998349 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff5f0004b80 con 0x7ff5f406b090 2024-03-23T09:45:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.682+0000 7ff5e9ffb640 1 --2- 172.21.15.47:0/1881998349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5d806ac00 0x7ff5d806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.682+0000 7ff5e9ffb640 1 -- 172.21.15.47:0/1881998349 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5f0089200 con 0x7ff5f406b090 2024-03-23T09:45:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.682+0000 7ff5f8934640 1 --2- 172.21.15.47:0/1881998349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5d806ac00 0x7ff5d806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.683+0000 7ff5f8934640 1 --2- 172.21.15.47:0/1881998349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5d806ac00 0x7ff5d806d0c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff5f406bbf0 tx=0x7ff5e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.686+0000 7ff5e9ffb640 1 -- 172.21.15.47:0/1881998349 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff5f00543e0 con 0x7ff5f406b090 2024-03-23T09:45:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.866+0000 7ff5fb399640 1 -- 172.21.15.47:0/1881998349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5f4066be0 con 0x7ff5f406b090 2024-03-23T09:45:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.867+0000 7ff5e9ffb640 1 -- 172.21.15.47:0/1881998349 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5f0057a00 con 0x7ff5f406b090 2024-03-23T09:45:30.867 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:45:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.869+0000 7ff5fb399640 1 -- 172.21.15.47:0/1881998349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5d806ac00 msgr2=0x7ff5d806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.869+0000 7ff5fb399640 1 --2- 172.21.15.47:0/1881998349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5d806ac00 0x7ff5d806d0c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff5f406bbf0 tx=0x7ff5e4008040 comp rx=0 tx=0).stop 2024-03-23T09:45:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.869+0000 7ff5fb399640 1 -- 172.21.15.47:0/1881998349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f406b090 msgr2=0x7ff5f40adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.869+0000 7ff5fb399640 1 --2- 172.21.15.47:0/1881998349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f406b090 0x7ff5f40adc90 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000c430 tx=0x7ff5f000c8f0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.869+0000 7ff5fb399640 1 -- 172.21.15.47:0/1881998349 shutdown_connections 2024-03-23T09:45:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.869+0000 7ff5fb399640 1 --2- 172.21.15.47:0/1881998349 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5d806ac00 0x7ff5d806d0c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.869+0000 7ff5fb399640 1 --2- 172.21.15.47:0/1881998349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f415cd40 0x7ff5f414c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.869+0000 7ff5fb399640 1 --2- 172.21.15.47:0/1881998349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f4099110 0x7ff5f40b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.869+0000 7ff5fb399640 1 --2- 172.21.15.47:0/1881998349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f406b090 0x7ff5f40adc90 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.869+0000 7ff5fb399640 1 -- 172.21.15.47:0/1881998349 >> 172.21.15.47:0/1881998349 conn(0x7ff5f40b2140 msgr2=0x7ff5f415abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:30.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.869+0000 7ff5fb399640 1 -- 172.21.15.47:0/1881998349 shutdown_connections 2024-03-23T09:45:30.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.869+0000 7ff5fb399640 1 -- 172.21.15.47:0/1881998349 wait complete. 2024-03-23T09:45:30.879 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:45:30.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.981+0000 7f943b1d5640 1 Processor -- start 2024-03-23T09:45:30.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.981+0000 7f943b1d5640 1 -- start start 2024-03-23T09:45:30.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.981+0000 7f943b1d5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f943415cb40 0x7f9434160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.982+0000 7f943b1d5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9434160cb0 0x7f94340d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.982+0000 7f943b1d5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94340d0ee0 con 0x7f94340b8300 2024-03-23T09:45:30.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.982+0000 7f943b1d5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94340b5a10 con 0x7f9434160cb0 2024-03-23T09:45:30.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.982+0000 7f943b1d5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94341616f0 con 0x7f943415cb40 2024-03-23T09:45:30.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.982+0000 7f9439772640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9434160cb0 0x7f94340d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.982+0000 7f9439772640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9434160cb0 0x7f94340d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36094/0 (socket says 172.21.15.47:36094) 2024-03-23T09:45:30.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.982+0000 7f9439772640 1 -- 172.21.15.47:0/533307358 learned_addr learned my addr 172.21.15.47:0/533307358 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:30.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.982+0000 7f9438f71640 1 --2- 172.21.15.47:0/533307358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f943415cb40 0x7f9434160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.982+0000 7f9439772640 1 -- 172.21.15.47:0/533307358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f943415cb40 msgr2=0x7f9434160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:30.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.982+0000 7f9439772640 1 --2- 172.21.15.47:0/533307358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f943415cb40 0x7f9434160770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.982+0000 7f9439772640 1 -- 172.21.15.47:0/533307358 >> v1:172.21.15.175:6789/0 conn(0x7f94340b8300 legacy=0x7f94340cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:30.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.982+0000 7f9439772640 1 -- 172.21.15.47:0/533307358 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9434161130 con 0x7f9434160cb0 2024-03-23T09:45:30.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.983+0000 7f9439772640 1 --2- 172.21.15.47:0/533307358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9434160cb0 0x7f94340d1370 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f942c009e50 tx=0x7f942c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdfa63600b51272 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:30.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.983+0000 7f94337fe640 1 -- 172.21.15.47:0/533307358 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c02ca70 con 0x7f9434160cb0 2024-03-23T09:45:30.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.983+0000 7f94337fe640 1 -- 172.21.15.47:0/533307358 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c02cbd0 con 0x7f9434160cb0 2024-03-23T09:45:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.983+0000 7f94337fe640 1 -- 172.21.15.47:0/533307358 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0343f0 con 0x7f9434160cb0 2024-03-23T09:45:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.983+0000 7f943b1d5640 1 -- 172.21.15.47:0/533307358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9434160cb0 msgr2=0x7f94340d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.983+0000 7f943b1d5640 1 --2- 172.21.15.47:0/533307358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9434160cb0 0x7f94340d1370 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f942c009e50 tx=0x7f942c02bed0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.984+0000 7f943b1d5640 1 -- 172.21.15.47:0/533307358 shutdown_connections 2024-03-23T09:45:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.984+0000 7f943b1d5640 1 --2- 172.21.15.47:0/533307358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9434160cb0 0x7f94340d1370 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.984+0000 7f943b1d5640 1 --2- 172.21.15.47:0/533307358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f943415cb40 0x7f9434160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.984+0000 7f943b1d5640 1 -- 172.21.15.47:0/533307358 >> 172.21.15.47:0/533307358 conn(0x7f943406b160 msgr2=0x7f943406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.984+0000 7f943b1d5640 1 -- 172.21.15.47:0/533307358 shutdown_connections 2024-03-23T09:45:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.984+0000 7f943b1d5640 1 -- 172.21.15.47:0/533307358 wait complete. 2024-03-23T09:45:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.985+0000 7f943b1d5640 1 Processor -- start 2024-03-23T09:45:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.985+0000 7f943b1d5640 1 -- start start 2024-03-23T09:45:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f943b1d5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94340b8300 0x7f94341531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f943b1d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943415cb40 0x7f9434153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f943b1d5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9434160cb0 0x7f9434153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f943b1d5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9434154740 con 0x7f9434160cb0 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f943b1d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9434143890 con 0x7f943415cb40 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f943b1d5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9434154190 con 0x7f94340b8300 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f9439772640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9434160cb0 0x7f9434153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f9439772640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9434160cb0 0x7f9434153c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36098/0 (socket says 172.21.15.47:36098) 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f9439772640 1 -- 172.21.15.47:0/628236306 learned_addr learned my addr 172.21.15.47:0/628236306 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f9438f71640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94340b8300 0x7f94341531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f9433fff640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943415cb40 0x7f9434153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f9439772640 1 -- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94340b8300 msgr2=0x7f94341531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f9439772640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94340b8300 0x7f94341531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f9439772640 1 -- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943415cb40 msgr2=0x7f9434153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f9439772640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943415cb40 0x7f9434153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f9439772640 1 -- 172.21.15.47:0/628236306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c009b30 con 0x7f9434160cb0 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f9433fff640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943415cb40 0x7f9434153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.986+0000 7f9439772640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9434160cb0 0x7f9434153c50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f942c036040 tx=0x7f942c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:30.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.987+0000 7f94317fa640 1 -- 172.21.15.47:0/628236306 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c040670 con 0x7f9434160cb0 2024-03-23T09:45:30.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.987+0000 7f94317fa640 1 -- 172.21.15.47:0/628236306 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c0407f0 con 0x7f9434160cb0 2024-03-23T09:45:30.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.987+0000 7f943b1d5640 1 -- 172.21.15.47:0/628236306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9434154460 con 0x7f9434160cb0 2024-03-23T09:45:30.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.987+0000 7f94317fa640 1 -- 172.21.15.47:0/628236306 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c040ba0 con 0x7f9434160cb0 2024-03-23T09:45:30.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.987+0000 7f943b1d5640 1 -- 172.21.15.47:0/628236306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9434143e10 con 0x7f9434160cb0 2024-03-23T09:45:30.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.987+0000 7f943b1d5640 1 -- 172.21.15.47:0/628236306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9434072360 con 0x7f9434160cb0 2024-03-23T09:45:30.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.988+0000 7f94317fa640 1 -- 172.21.15.47:0/628236306 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f942c040d00 con 0x7f9434160cb0 2024-03-23T09:45:30.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.988+0000 7f94317fa640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f941806ac00 0x7f941806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:30.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.988+0000 7f94317fa640 1 -- 172.21.15.47:0/628236306 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f942c0ac400 con 0x7f9434160cb0 2024-03-23T09:45:30.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.989+0000 7f9438f71640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f941806ac00 0x7f941806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:30.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.989+0000 7f9438f71640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f941806ac00 0x7f941806d0c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f942800aac0 tx=0x7f9428008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:30.993+0000 7f94317fa640 1 -- 172.21.15.47:0/628236306 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f942c077790 con 0x7f9434160cb0 2024-03-23T09:45:31.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.185+0000 7f943b1d5640 1 -- 172.21.15.47:0/628236306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9434144000 con 0x7f9434160cb0 2024-03-23T09:45:31.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.186+0000 7f94317fa640 1 -- 172.21.15.47:0/628236306 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) v1 ==== 80+0+0 (secure 0 0 0) 0x7f942c07adb0 con 0x7f9434160cb0 2024-03-23T09:45:31.186 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:45:31.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.188+0000 7f943b1d5640 1 -- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f941806ac00 msgr2=0x7f941806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.188+0000 7f943b1d5640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f941806ac00 0x7f941806d0c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f942800aac0 tx=0x7f9428008040 comp rx=0 tx=0).stop 2024-03-23T09:45:31.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.188+0000 7f943b1d5640 1 -- 172.21.15.47:0/628236306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9434160cb0 msgr2=0x7f9434153c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.188+0000 7f943b1d5640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9434160cb0 0x7f9434153c50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f942c036040 tx=0x7f942c004a40 comp rx=0 tx=0).stop 2024-03-23T09:45:31.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.188+0000 7f943b1d5640 1 -- 172.21.15.47:0/628236306 shutdown_connections 2024-03-23T09:45:31.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.188+0000 7f943b1d5640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f941806ac00 0x7f941806d0c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.188+0000 7f943b1d5640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9434160cb0 0x7f9434153c50 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.188+0000 7f943b1d5640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943415cb40 0x7f9434153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.188+0000 7f943b1d5640 1 --2- 172.21.15.47:0/628236306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94340b8300 0x7f94341531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.188+0000 7f943b1d5640 1 -- 172.21.15.47:0/628236306 >> 172.21.15.47:0/628236306 conn(0x7f943406b160 msgr2=0x7f94340b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:31.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.188+0000 7f943b1d5640 1 -- 172.21.15.47:0/628236306 shutdown_connections 2024-03-23T09:45:31.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.188+0000 7f943b1d5640 1 -- 172.21.15.47:0/628236306 wait complete. 2024-03-23T09:45:31.199 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:45:31.199 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:45:31.214 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:45:31.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.354+0000 7f5497ffb640 1 Processor -- start 2024-03-23T09:45:31.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.354+0000 7f5497ffb640 1 -- start start 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5497ffb640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f549015ca90 0x7f5490160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5497ffb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f54900cde10 0x7f54900d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5497ffb640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54900d11e0 con 0x7f54900b8350 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5497ffb640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5490160f60 con 0x7f54900cde10 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5497ffb640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5490161240 con 0x7f549015ca90 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5496598640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f54900cde10 0x7f54900d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5496598640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f54900cde10 0x7f54900d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36112/0 (socket says 172.21.15.47:36112) 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5496598640 1 -- 172.21.15.47:0/2771044183 learned_addr learned my addr 172.21.15.47:0/2771044183 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5495d97640 1 --2- 172.21.15.47:0/2771044183 >> v2:172.21.15.175:3300/0 conn(0x7f549015ca90 0x7f5490160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5496598640 1 -- 172.21.15.47:0/2771044183 >> v2:172.21.15.175:3300/0 conn(0x7f549015ca90 msgr2=0x7f5490160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5496598640 1 --2- 172.21.15.47:0/2771044183 >> v2:172.21.15.175:3300/0 conn(0x7f549015ca90 0x7f5490160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5496598640 1 -- 172.21.15.47:0/2771044183 >> v1:172.21.15.175:6789/0 conn(0x7f54900b8350 legacy=0x7f54900cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5496598640 1 -- 172.21.15.47:0/2771044183 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5490161520 con 0x7f54900cde10 2024-03-23T09:45:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.355+0000 7f5495d97640 1 --2- 172.21.15.47:0/2771044183 >> v2:172.21.15.175:3300/0 conn(0x7f549015ca90 0x7f5490160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:31.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.356+0000 7f5496598640 1 --2- 172.21.15.47:0/2771044183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54900cde10 0x7f54900d1670 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5490066680 tx=0x7f548802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f68fb31ece4ecfa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:31.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.356+0000 7f5494d95640 1 -- 172.21.15.47:0/2771044183 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548802ca80 con 0x7f54900cde10 2024-03-23T09:45:31.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.356+0000 7f5494d95640 1 -- 172.21.15.47:0/2771044183 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548802cbe0 con 0x7f54900cde10 2024-03-23T09:45:31.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.356+0000 7f5494d95640 1 -- 172.21.15.47:0/2771044183 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880343f0 con 0x7f54900cde10 2024-03-23T09:45:31.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.356+0000 7f5497ffb640 1 -- 172.21.15.47:0/2771044183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54900cde10 msgr2=0x7f54900d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.356+0000 7f5497ffb640 1 --2- 172.21.15.47:0/2771044183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54900cde10 0x7f54900d1670 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5490066680 tx=0x7f548802bee0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.356+0000 7f5497ffb640 1 -- 172.21.15.47:0/2771044183 shutdown_connections 2024-03-23T09:45:31.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.356+0000 7f5497ffb640 1 --2- 172.21.15.47:0/2771044183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54900cde10 0x7f54900d1670 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.356+0000 7f5497ffb640 1 --2- 172.21.15.47:0/2771044183 >> v2:172.21.15.175:3300/0 conn(0x7f549015ca90 0x7f5490160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.356+0000 7f5497ffb640 1 -- 172.21.15.47:0/2771044183 >> 172.21.15.47:0/2771044183 conn(0x7f54900659d0 msgr2=0x7f5490065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.357+0000 7f5497ffb640 1 -- 172.21.15.47:0/2771044183 shutdown_connections 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.357+0000 7f5497ffb640 1 -- 172.21.15.47:0/2771044183 wait complete. 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.358+0000 7f5497ffb640 1 Processor -- start 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.358+0000 7f5497ffb640 1 -- start start 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.358+0000 7f5497ffb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54900b8350 0x7f54900b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.358+0000 7f5497ffb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54900cde10 0x7f5490151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.358+0000 7f5497ffb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549015ca90 0x7f5490153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.358+0000 7f5497ffb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54900b4050 con 0x7f54900cde10 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.358+0000 7f5497ffb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5490153860 con 0x7f549015ca90 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.358+0000 7f5497ffb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54900b33d0 con 0x7f54900b8350 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.358+0000 7f5495596640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54900cde10 0x7f5490151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5495596640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54900cde10 0x7f5490151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36118/0 (socket says 172.21.15.47:36118) 2024-03-23T09:45:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5495596640 1 -- 172.21.15.47:0/3008042782 learned_addr learned my addr 172.21.15.47:0/3008042782 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5495d97640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54900b8350 0x7f54900b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5496598640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549015ca90 0x7f5490153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5495596640 1 -- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54900b8350 msgr2=0x7f54900b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5495596640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54900b8350 0x7f54900b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5495596640 1 -- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549015ca90 msgr2=0x7f5490153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5495596640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549015ca90 0x7f5490153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5495596640 1 -- 172.21.15.47:0/3008042782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488009b30 con 0x7f54900cde10 2024-03-23T09:45:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5495d97640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54900b8350 0x7f54900b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5496598640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549015ca90 0x7f5490153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5495596640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54900cde10 0x7f5490151060 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f548000c400 tx=0x7f548000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f54867fc640 1 -- 172.21.15.47:0/3008042782 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54800052d0 con 0x7f54900cde10 2024-03-23T09:45:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f54867fc640 1 -- 172.21.15.47:0/3008042782 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480005430 con 0x7f54900cde10 2024-03-23T09:45:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5497ffb640 1 -- 172.21.15.47:0/3008042782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54900b36b0 con 0x7f54900cde10 2024-03-23T09:45:31.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f54867fc640 1 -- 172.21.15.47:0/3008042782 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480003830 con 0x7f54900cde10 2024-03-23T09:45:31.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.359+0000 7f5497ffb640 1 -- 172.21.15.47:0/3008042782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54900b3c70 con 0x7f54900cde10 2024-03-23T09:45:31.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.360+0000 7f5497ffb640 1 -- 172.21.15.47:0/3008042782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5490072360 con 0x7f54900cde10 2024-03-23T09:45:31.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.361+0000 7f54867fc640 1 -- 172.21.15.47:0/3008042782 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5480003990 con 0x7f54900cde10 2024-03-23T09:45:31.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.361+0000 7f54867fc640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f547406ac30 0x7f547406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.361+0000 7f54867fc640 1 -- 172.21.15.47:0/3008042782 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5480088830 con 0x7f54900cde10 2024-03-23T09:45:31.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.361+0000 7f5495d97640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f547406ac30 0x7f547406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.362+0000 7f5495d97640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f547406ac30 0x7f547406d0f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f548c006fd0 tx=0x7f548c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:31.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.365+0000 7f54867fc640 1 -- 172.21.15.47:0/3008042782 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5480053aa0 con 0x7f54900cde10 2024-03-23T09:45:31.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.548+0000 7f5497ffb640 1 -- 172.21.15.47:0/3008042782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54900b4190 con 0x7f54900cde10 2024-03-23T09:45:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.549+0000 7f54867fc640 1 -- 172.21.15.47:0/3008042782 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16100 (secure 0 0 0) 0x7f54900b4190 con 0x7f54900cde10 2024-03-23T09:45:31.549 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:31.549 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":70,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:45:19.298605+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:45:16.609941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-23T09:45:17.261503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:45:31.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.551+0000 7f5497ffb640 1 -- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f547406ac30 msgr2=0x7f547406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.551+0000 7f5497ffb640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f547406ac30 0x7f547406d0f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f548c006fd0 tx=0x7f548c008040 comp rx=0 tx=0).stop 2024-03-23T09:45:31.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.551+0000 7f5497ffb640 1 -- 172.21.15.47:0/3008042782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54900cde10 msgr2=0x7f5490151060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.551+0000 7f5497ffb640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54900cde10 0x7f5490151060 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f548000c400 tx=0x7f548000c8c0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.551+0000 7f5497ffb640 1 -- 172.21.15.47:0/3008042782 shutdown_connections 2024-03-23T09:45:31.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.551+0000 7f5497ffb640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f547406ac30 0x7f547406d0f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.551+0000 7f5497ffb640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549015ca90 0x7f5490153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.551+0000 7f5497ffb640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54900cde10 0x7f5490151060 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.551+0000 7f5497ffb640 1 --2- 172.21.15.47:0/3008042782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54900b8350 0x7f54900b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.551+0000 7f5497ffb640 1 -- 172.21.15.47:0/3008042782 >> 172.21.15.47:0/3008042782 conn(0x7f54900659d0 msgr2=0x7f54900b7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:31.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.551+0000 7f5497ffb640 1 -- 172.21.15.47:0/3008042782 shutdown_connections 2024-03-23T09:45:31.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.551+0000 7f5497ffb640 1 -- 172.21.15.47:0/3008042782 wait complete. 2024-03-23T09:45:31.562 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:45:16.609941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:45:17.261503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:45:31.562 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:45:31.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.665+0000 7f1c978e5640 1 Processor -- start 2024-03-23T09:45:31.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.665+0000 7f1c978e5640 1 -- start start 2024-03-23T09:45:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.666+0000 7f1c978e5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c900cb2f0 0x7f1c9015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.666+0000 7f1c978e5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1c90069ce0 0x7f1c9006a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.666+0000 7f1c978e5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c900cb730 con 0x7f1c900b0e10 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.666+0000 7f1c978e5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c900b11f0 con 0x7f1c90069ce0 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.666+0000 7f1c978e5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9006a690 con 0x7f1c900cb2f0 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.666+0000 7f1c95e82640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1c90069ce0 0x7f1c9006a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.666+0000 7f1c95e82640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1c90069ce0 0x7f1c9006a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36122/0 (socket says 172.21.15.47:36122) 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.666+0000 7f1c95e82640 1 -- 172.21.15.47:0/839730477 learned_addr learned my addr 172.21.15.47:0/839730477 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.667+0000 7f1c94e80640 1 --2- 172.21.15.47:0/839730477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c900cb2f0 0x7f1c9015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.667+0000 7f1c95e82640 1 -- 172.21.15.47:0/839730477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c900cb2f0 msgr2=0x7f1c9015b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.667+0000 7f1c95e82640 1 --2- 172.21.15.47:0/839730477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c900cb2f0 0x7f1c9015b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.667+0000 7f1c95e82640 1 -- 172.21.15.47:0/839730477 >> v1:172.21.15.175:6789/0 conn(0x7f1c900b0e10 legacy=0x7f1c900d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.667+0000 7f1c95e82640 1 -- 172.21.15.47:0/839730477 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c900cb970 con 0x7f1c90069ce0 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.667+0000 7f1c94e80640 1 --2- 172.21.15.47:0/839730477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c900cb2f0 0x7f1c9015b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.667+0000 7f1c95e82640 1 --2- 172.21.15.47:0/839730477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c90069ce0 0x7f1c9006a150 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009870 tx=0x7f1c8802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d4c43807eabab9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.668+0000 7f1c8ffff640 1 -- 172.21.15.47:0/839730477 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88031640 con 0x7f1c90069ce0 2024-03-23T09:45:31.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.668+0000 7f1c8ffff640 1 -- 172.21.15.47:0/839730477 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c880317a0 con 0x7f1c90069ce0 2024-03-23T09:45:31.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.668+0000 7f1c8ffff640 1 -- 172.21.15.47:0/839730477 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88031b30 con 0x7f1c90069ce0 2024-03-23T09:45:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.668+0000 7f1c978e5640 1 -- 172.21.15.47:0/839730477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c90069ce0 msgr2=0x7f1c9006a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.668+0000 7f1c978e5640 1 --2- 172.21.15.47:0/839730477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c90069ce0 0x7f1c9006a150 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009870 tx=0x7f1c8802b8f0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.668+0000 7f1c978e5640 1 -- 172.21.15.47:0/839730477 shutdown_connections 2024-03-23T09:45:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.668+0000 7f1c978e5640 1 --2- 172.21.15.47:0/839730477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c90069ce0 0x7f1c9006a150 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.668+0000 7f1c978e5640 1 --2- 172.21.15.47:0/839730477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c900cb2f0 0x7f1c9015b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.668+0000 7f1c978e5640 1 -- 172.21.15.47:0/839730477 >> 172.21.15.47:0/839730477 conn(0x7f1c90098a00 msgr2=0x7f1c900cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.668+0000 7f1c978e5640 1 -- 172.21.15.47:0/839730477 shutdown_connections 2024-03-23T09:45:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.668+0000 7f1c978e5640 1 -- 172.21.15.47:0/839730477 wait complete. 2024-03-23T09:45:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.669+0000 7f1c978e5640 1 Processor -- start 2024-03-23T09:45:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.669+0000 7f1c978e5640 1 -- start start 2024-03-23T09:45:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.670+0000 7f1c978e5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c90069ce0 0x7f1c90157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.670+0000 7f1c978e5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c900b0e10 0x7f1c900c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.670+0000 7f1c978e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c900cb2f0 0x7f1c900ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.670+0000 7f1c978e5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c900c73b0 con 0x7f1c900cb2f0 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.670+0000 7f1c978e5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c900c63f0 con 0x7f1c900b0e10 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.670+0000 7f1c978e5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c900c6a80 con 0x7f1c90069ce0 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.670+0000 7f1c95e82640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c900cb2f0 0x7f1c900ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.670+0000 7f1c95e82640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c900cb2f0 0x7f1c900ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36126/0 (socket says 172.21.15.47:36126) 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.670+0000 7f1c95e82640 1 -- 172.21.15.47:0/2784199897 learned_addr learned my addr 172.21.15.47:0/2784199897 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.670+0000 7f1c95681640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c90069ce0 0x7f1c90157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.670+0000 7f1c94e80640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c900b0e10 0x7f1c900c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.671+0000 7f1c95e82640 1 -- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c90069ce0 msgr2=0x7f1c90157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.671+0000 7f1c95e82640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c90069ce0 0x7f1c90157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.671+0000 7f1c95e82640 1 -- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c900b0e10 msgr2=0x7f1c900c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.671+0000 7f1c95e82640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c900b0e10 0x7f1c900c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.671+0000 7f1c95e82640 1 -- 172.21.15.47:0/2784199897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c880094d0 con 0x7f1c900cb2f0 2024-03-23T09:45:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.671+0000 7f1c94e80640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c900b0e10 0x7f1c900c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:31.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.671+0000 7f1c95681640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c90069ce0 0x7f1c90157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:31.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.671+0000 7f1c95e82640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c900cb2f0 0x7f1c900ad580 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802b930 tx=0x7f1c880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:31.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.671+0000 7f1c8dffb640 1 -- 172.21.15.47:0/2784199897 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8803ac60 con 0x7f1c900cb2f0 2024-03-23T09:45:31.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.671+0000 7f1c8dffb640 1 -- 172.21.15.47:0/2784199897 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8803ade0 con 0x7f1c900cb2f0 2024-03-23T09:45:31.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.671+0000 7f1c978e5640 1 -- 172.21.15.47:0/2784199897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c900c6d60 con 0x7f1c900cb2f0 2024-03-23T09:45:31.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.671+0000 7f1c8dffb640 1 -- 172.21.15.47:0/2784199897 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88044630 con 0x7f1c900cb2f0 2024-03-23T09:45:31.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.671+0000 7f1c978e5640 1 -- 172.21.15.47:0/2784199897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c90158060 con 0x7f1c900cb2f0 2024-03-23T09:45:31.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.672+0000 7f1c978e5640 1 -- 172.21.15.47:0/2784199897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c900c7560 con 0x7f1c900cb2f0 2024-03-23T09:45:31.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.673+0000 7f1c8dffb640 1 -- 172.21.15.47:0/2784199897 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1c8802f050 con 0x7f1c900cb2f0 2024-03-23T09:45:31.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.673+0000 7f1c8dffb640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1c7c06ac00 0x7f1c7c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.673+0000 7f1c8dffb640 1 -- 172.21.15.47:0/2784199897 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c880ab1e0 con 0x7f1c900cb2f0 2024-03-23T09:45:31.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.673+0000 7f1c95681640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1c7c06ac00 0x7f1c7c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.674+0000 7f1c95681640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1c7c06ac00 0x7f1c7c06d0c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000c420 tx=0x7f1c80009570 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:31.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.677+0000 7f1c8dffb640 1 -- 172.21.15.47:0/2784199897 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1c900c7560 con 0x7f1c900cb2f0 2024-03-23T09:45:31.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.857+0000 7f1c978e5640 1 -- 172.21.15.47:0/2784199897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c900cb730 con 0x7f1c900cb2f0 2024-03-23T09:45:31.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.857+0000 7f1c8dffb640 1 -- 172.21.15.47:0/2784199897 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c900cb730 con 0x7f1c900cb2f0 2024-03-23T09:45:31.858 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:45:31.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.860+0000 7f1c978e5640 1 -- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1c7c06ac00 msgr2=0x7f1c7c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.860+0000 7f1c978e5640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1c7c06ac00 0x7f1c7c06d0c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000c420 tx=0x7f1c80009570 comp rx=0 tx=0).stop 2024-03-23T09:45:31.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.860+0000 7f1c978e5640 1 -- 172.21.15.47:0/2784199897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c900cb2f0 msgr2=0x7f1c900ad580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.860+0000 7f1c978e5640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c900cb2f0 0x7f1c900ad580 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802b930 tx=0x7f1c880046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.860+0000 7f1c978e5640 1 -- 172.21.15.47:0/2784199897 shutdown_connections 2024-03-23T09:45:31.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.860+0000 7f1c978e5640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1c7c06ac00 0x7f1c7c06d0c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.860+0000 7f1c978e5640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c900cb2f0 0x7f1c900ad580 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.860+0000 7f1c978e5640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c900b0e10 0x7f1c900c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.860+0000 7f1c978e5640 1 --2- 172.21.15.47:0/2784199897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c90069ce0 0x7f1c90157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.860+0000 7f1c978e5640 1 -- 172.21.15.47:0/2784199897 >> 172.21.15.47:0/2784199897 conn(0x7f1c90098a00 msgr2=0x7f1c9006dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:31.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.860+0000 7f1c978e5640 1 -- 172.21.15.47:0/2784199897 shutdown_connections 2024-03-23T09:45:31.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.860+0000 7f1c978e5640 1 -- 172.21.15.47:0/2784199897 wait complete. 2024-03-23T09:45:31.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:45:31.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.974+0000 7fe9a2caf640 1 Processor -- start 2024-03-23T09:45:31.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.974+0000 7fe9a2caf640 1 -- start start 2024-03-23T09:45:31.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.974+0000 7fe9a2caf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe99c15c990 0x7fe99c160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.975+0000 7fe9a2caf640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe99c0cde10 0x7fe99c0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.975+0000 7fe9a2caf640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99c161630 con 0x7fe99c0b8350 2024-03-23T09:45:31.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.975+0000 7fe9a2caf640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99c0d1030 con 0x7fe99c15c990 2024-03-23T09:45:31.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.975+0000 7fe9a2caf640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99c0cfe10 con 0x7fe99c0cde10 2024-03-23T09:45:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.975+0000 7fe9a0a4b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe99c15c990 0x7fe99c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.975+0000 7fe9a0a4b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe99c15c990 0x7fe99c160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36136/0 (socket says 172.21.15.47:36136) 2024-03-23T09:45:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.975+0000 7fe9a0a4b640 1 -- 172.21.15.47:0/2477063324 learned_addr learned my addr 172.21.15.47:0/2477063324 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.975+0000 7fe9a0a4b640 1 -- 172.21.15.47:0/2477063324 >> v2:172.21.15.175:3300/0 conn(0x7fe99c0cde10 msgr2=0x7fe99c0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.975+0000 7fe9a124c640 1 --2- 172.21.15.47:0/2477063324 >> v2:172.21.15.175:3300/0 conn(0x7fe99c0cde10 0x7fe99c0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.975+0000 7fe9a0a4b640 1 --2- 172.21.15.47:0/2477063324 >> v2:172.21.15.175:3300/0 conn(0x7fe99c0cde10 0x7fe99c0d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.975+0000 7fe9a0a4b640 1 -- 172.21.15.47:0/2477063324 >> v1:172.21.15.175:6789/0 conn(0x7fe99c0b8350 legacy=0x7fe99c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.975+0000 7fe9a0a4b640 1 -- 172.21.15.47:0/2477063324 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c0b5b60 con 0x7fe99c15c990 2024-03-23T09:45:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.976+0000 7fe9a0a4b640 1 --2- 172.21.15.47:0/2477063324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe99c15c990 0x7fe99c160690 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0b87e0 tx=0x7fe99002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e00473d70a639c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.976+0000 7fe99b7fe640 1 -- 172.21.15.47:0/2477063324 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9900317e0 con 0x7fe99c15c990 2024-03-23T09:45:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.976+0000 7fe99b7fe640 1 -- 172.21.15.47:0/2477063324 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe990031940 con 0x7fe99c15c990 2024-03-23T09:45:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.976+0000 7fe99b7fe640 1 -- 172.21.15.47:0/2477063324 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990031cd0 con 0x7fe99c15c990 2024-03-23T09:45:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.976+0000 7fe9a2caf640 1 -- 172.21.15.47:0/2477063324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe99c15c990 msgr2=0x7fe99c160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.976+0000 7fe9a2caf640 1 --2- 172.21.15.47:0/2477063324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe99c15c990 0x7fe99c160690 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0b87e0 tx=0x7fe99002b880 comp rx=0 tx=0).stop 2024-03-23T09:45:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.977+0000 7fe9a2caf640 1 -- 172.21.15.47:0/2477063324 shutdown_connections 2024-03-23T09:45:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.977+0000 7fe9a2caf640 1 --2- 172.21.15.47:0/2477063324 >> v2:172.21.15.175:3300/0 conn(0x7fe99c0cde10 0x7fe99c0d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.977+0000 7fe9a2caf640 1 --2- 172.21.15.47:0/2477063324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe99c15c990 0x7fe99c160690 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.977+0000 7fe9a2caf640 1 -- 172.21.15.47:0/2477063324 >> 172.21.15.47:0/2477063324 conn(0x7fe99c0659d0 msgr2=0x7fe99c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.977+0000 7fe9a2caf640 1 -- 172.21.15.47:0/2477063324 shutdown_connections 2024-03-23T09:45:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.977+0000 7fe9a2caf640 1 -- 172.21.15.47:0/2477063324 wait complete. 2024-03-23T09:45:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.978+0000 7fe9a2caf640 1 Processor -- start 2024-03-23T09:45:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.978+0000 7fe9a2caf640 1 -- start start 2024-03-23T09:45:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a2caf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe99c0b8350 0x7fe99c153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a2caf640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe99c0cde10 0x7fe99c0c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a2caf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c15c990 0x7fe99c0b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a2caf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c15d440 con 0x7fe99c0b8350 2024-03-23T09:45:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a2caf640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c15d720 con 0x7fe99c15c990 2024-03-23T09:45:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a2caf640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c15da00 con 0x7fe99c0cde10 2024-03-23T09:45:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a0a4b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe99c0b8350 0x7fe99c153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a0a4b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe99c0b8350 0x7fe99c153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36148/0 (socket says 172.21.15.47:36148) 2024-03-23T09:45:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a0a4b640 1 -- 172.21.15.47:0/1587597567 learned_addr learned my addr 172.21.15.47:0/1587597567 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe99bfff640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe99c0cde10 0x7fe99c0c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a124c640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c15c990 0x7fe99c0b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a0a4b640 1 -- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe99c0cde10 msgr2=0x7fe99c0c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a0a4b640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe99c0cde10 0x7fe99c0c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a0a4b640 1 -- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c15c990 msgr2=0x7fe99c0b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a0a4b640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c15c990 0x7fe99c0b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a0a4b640 1 -- 172.21.15.47:0/1587597567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9900094d0 con 0x7fe99c0b8350 2024-03-23T09:45:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe99bfff640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe99c0cde10 0x7fe99c0c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a124c640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c15c990 0x7fe99c0b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a0a4b640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe99c0b8350 0x7fe99c153ce0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe990000c00 tx=0x7fe9900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9997fa640 1 -- 172.21.15.47:0/1587597567 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990032980 con 0x7fe99c0b8350 2024-03-23T09:45:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9997fa640 1 -- 172.21.15.47:0/1587597567 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe990032ae0 con 0x7fe99c0b8350 2024-03-23T09:45:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a2caf640 1 -- 172.21.15.47:0/1587597567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99c15dce0 con 0x7fe99c0b8350 2024-03-23T09:45:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9997fa640 1 -- 172.21.15.47:0/1587597567 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990032e90 con 0x7fe99c0b8350 2024-03-23T09:45:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.979+0000 7fe9a2caf640 1 -- 172.21.15.47:0/1587597567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99c15d0d0 con 0x7fe99c0b8350 2024-03-23T09:45:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.981+0000 7fe9a2caf640 1 -- 172.21.15.47:0/1587597567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99c072360 con 0x7fe99c0b8350 2024-03-23T09:45:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.981+0000 7fe9997fa640 1 -- 172.21.15.47:0/1587597567 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe99004e450 con 0x7fe99c0b8350 2024-03-23T09:45:31.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.981+0000 7fe9997fa640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe98006ac00 0x7fe98006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:31.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.981+0000 7fe9997fa640 1 -- 172.21.15.47:0/1587597567 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9900ab720 con 0x7fe99c0b8350 2024-03-23T09:45:31.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.982+0000 7fe99bfff640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe98006ac00 0x7fe98006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:31.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.982+0000 7fe99bfff640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe98006ac00 0x7fe98006d0c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0cd1c0 tx=0x7fe988009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:31.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:31.986+0000 7fe9997fa640 1 -- 172.21.15.47:0/1587597567 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe9900769b0 con 0x7fe99c0b8350 2024-03-23T09:45:32.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.163+0000 7fe9a2caf640 1 -- 172.21.15.47:0/1587597567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe99c15e200 con 0x7fe99c0b8350 2024-03-23T09:45:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.164+0000 7fe9997fa640 1 -- 172.21.15.47:0/1587597567 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe990079fd0 con 0x7fe99c0b8350 2024-03-23T09:45:32.164 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:45:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.166+0000 7fe9a2caf640 1 -- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe98006ac00 msgr2=0x7fe98006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.166+0000 7fe9a2caf640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe98006ac00 0x7fe98006d0c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0cd1c0 tx=0x7fe988009040 comp rx=0 tx=0).stop 2024-03-23T09:45:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.166+0000 7fe9a2caf640 1 -- 172.21.15.47:0/1587597567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe99c0b8350 msgr2=0x7fe99c153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.166+0000 7fe9a2caf640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe99c0b8350 0x7fe99c153ce0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe990000c00 tx=0x7fe9900046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.166+0000 7fe9a2caf640 1 -- 172.21.15.47:0/1587597567 shutdown_connections 2024-03-23T09:45:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.166+0000 7fe9a2caf640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe98006ac00 0x7fe98006d0c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.166+0000 7fe9a2caf640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c15c990 0x7fe99c0b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.166+0000 7fe9a2caf640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe99c0cde10 0x7fe99c0c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.166+0000 7fe9a2caf640 1 --2- 172.21.15.47:0/1587597567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe99c0b8350 0x7fe99c153ce0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.166+0000 7fe9a2caf640 1 -- 172.21.15.47:0/1587597567 >> 172.21.15.47:0/1587597567 conn(0x7fe99c0659d0 msgr2=0x7fe99c0b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.166+0000 7fe9a2caf640 1 -- 172.21.15.47:0/1587597567 shutdown_connections 2024-03-23T09:45:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.166+0000 7fe9a2caf640 1 -- 172.21.15.47:0/1587597567 wait complete. 2024-03-23T09:45:32.176 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:45:32.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.279+0000 7f30ec0c4640 1 Processor -- start 2024-03-23T09:45:32.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.280+0000 7f30ec0c4640 1 -- start start 2024-03-23T09:45:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.280+0000 7f30ec0c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30e415c990 0x7f30e4160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.280+0000 7f30ec0c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30e40b8350 0x7f30e40cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.280+0000 7f30ec0c4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e4161630 con 0x7f30e40cdc40 2024-03-23T09:45:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.280+0000 7f30ec0c4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e40ce0b0 con 0x7f30e415c990 2024-03-23T09:45:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.280+0000 7f30ec0c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40d1030 con 0x7f30e40b8350 2024-03-23T09:45:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.281+0000 7f30e9e60640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30e415c990 0x7f30e4160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.281+0000 7f30e9e60640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30e415c990 0x7f30e4160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36154/0 (socket says 172.21.15.47:36154) 2024-03-23T09:45:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.281+0000 7f30e9e60640 1 -- 172.21.15.47:0/1021333524 learned_addr learned my addr 172.21.15.47:0/1021333524 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.281+0000 7f30e965f640 1 --2- 172.21.15.47:0/1021333524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30e40b8350 0x7f30e40cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.281+0000 7f30e9e60640 1 -- 172.21.15.47:0/1021333524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30e40b8350 msgr2=0x7f30e40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.281+0000 7f30e9e60640 1 --2- 172.21.15.47:0/1021333524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30e40b8350 0x7f30e40cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.281+0000 7f30e9e60640 1 -- 172.21.15.47:0/1021333524 >> v1:172.21.15.175:6789/0 conn(0x7f30e40cdc40 legacy=0x7f30e40b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.281+0000 7f30e9e60640 1 -- 172.21.15.47:0/1021333524 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e40cfe10 con 0x7f30e415c990 2024-03-23T09:45:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.282+0000 7f30e9e60640 1 --2- 172.21.15.47:0/1021333524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e415c990 0x7f30e4160690 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009870 tx=0x7f30e002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b926546222620e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.282+0000 7f30e8e5e640 1 -- 172.21.15.47:0/1021333524 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e0031640 con 0x7f30e415c990 2024-03-23T09:45:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.282+0000 7f30e8e5e640 1 -- 172.21.15.47:0/1021333524 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e00317a0 con 0x7f30e415c990 2024-03-23T09:45:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.282+0000 7f30e8e5e640 1 -- 172.21.15.47:0/1021333524 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e0031b30 con 0x7f30e415c990 2024-03-23T09:45:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.282+0000 7f30ec0c4640 1 -- 172.21.15.47:0/1021333524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e415c990 msgr2=0x7f30e4160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.282+0000 7f30ec0c4640 1 --2- 172.21.15.47:0/1021333524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e415c990 0x7f30e4160690 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009870 tx=0x7f30e002b8f0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.282+0000 7f30ec0c4640 1 -- 172.21.15.47:0/1021333524 shutdown_connections 2024-03-23T09:45:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.282+0000 7f30ec0c4640 1 --2- 172.21.15.47:0/1021333524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30e40b8350 0x7f30e40cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.282+0000 7f30ec0c4640 1 --2- 172.21.15.47:0/1021333524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e415c990 0x7f30e4160690 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.282+0000 7f30ec0c4640 1 -- 172.21.15.47:0/1021333524 >> 172.21.15.47:0/1021333524 conn(0x7f30e40659d0 msgr2=0x7f30e4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.282+0000 7f30ec0c4640 1 -- 172.21.15.47:0/1021333524 shutdown_connections 2024-03-23T09:45:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.282+0000 7f30ec0c4640 1 -- 172.21.15.47:0/1021333524 wait complete. 2024-03-23T09:45:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.284+0000 7f30ec0c4640 1 Processor -- start 2024-03-23T09:45:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.284+0000 7f30ec0c4640 1 -- start start 2024-03-23T09:45:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.284+0000 7f30ec0c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30e40b8350 0x7f30e4152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.284+0000 7f30ec0c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30e40cdc40 0x7f30e4150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.284+0000 7f30ec0c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e415c990 0x7f30e4154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.284+0000 7f30ec0c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40ca4d0 con 0x7f30e415c990 2024-03-23T09:45:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.284+0000 7f30ec0c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40cb050 con 0x7f30e40b8350 2024-03-23T09:45:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.284+0000 7f30ec0c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4151510 con 0x7f30e40cdc40 2024-03-23T09:45:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.284+0000 7f30ea661640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e415c990 0x7f30e4154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.284+0000 7f30ea661640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e415c990 0x7f30e4154340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36156/0 (socket says 172.21.15.47:36156) 2024-03-23T09:45:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.284+0000 7f30ea661640 1 -- 172.21.15.47:0/2536975872 learned_addr learned my addr 172.21.15.47:0/2536975872 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.285+0000 7f30e965f640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30e40cdc40 0x7f30e4150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.285+0000 7f30e9e60640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30e40b8350 0x7f30e4152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.285+0000 7f30ea661640 1 -- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30e40cdc40 msgr2=0x7f30e4150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.285+0000 7f30ea661640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30e40cdc40 0x7f30e4150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.285+0000 7f30ea661640 1 -- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30e40b8350 msgr2=0x7f30e4152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.285+0000 7f30ea661640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30e40b8350 0x7f30e4152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.285+0000 7f30ea661640 1 -- 172.21.15.47:0/2536975872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e00094d0 con 0x7f30e415c990 2024-03-23T09:45:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.285+0000 7f30ea661640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e415c990 0x7f30e4154340 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c7d0 tx=0x7f30dc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.285+0000 7f30da7fc640 1 -- 172.21.15.47:0/2536975872 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc003b70 con 0x7f30e415c990 2024-03-23T09:45:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.285+0000 7f30da7fc640 1 -- 172.21.15.47:0/2536975872 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc003cf0 con 0x7f30e415c990 2024-03-23T09:45:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.285+0000 7f30ec0c4640 1 -- 172.21.15.47:0/2536975872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e40ca720 con 0x7f30e415c990 2024-03-23T09:45:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.285+0000 7f30da7fc640 1 -- 172.21.15.47:0/2536975872 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00aa90 con 0x7f30e415c990 2024-03-23T09:45:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.285+0000 7f30ec0c4640 1 -- 172.21.15.47:0/2536975872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e40cace0 con 0x7f30e415c990 2024-03-23T09:45:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.286+0000 7f30ec0c4640 1 -- 172.21.15.47:0/2536975872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e4072360 con 0x7f30e415c990 2024-03-23T09:45:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.286+0000 7f30da7fc640 1 -- 172.21.15.47:0/2536975872 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f30dc00abf0 con 0x7f30e415c990 2024-03-23T09:45:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.287+0000 7f30da7fc640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f30c806ac00 0x7f30c806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.287+0000 7f30da7fc640 1 -- 172.21.15.47:0/2536975872 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30dc089000 con 0x7f30e415c990 2024-03-23T09:45:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.287+0000 7f30e9e60640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f30c806ac00 0x7f30c806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.288+0000 7f30e9e60640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f30c806ac00 0x7f30c806d0c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f30e0036000 tx=0x7f30e003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:32.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.291+0000 7f30da7fc640 1 -- 172.21.15.47:0/2536975872 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f30dc054320 con 0x7f30e415c990 2024-03-23T09:45:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.473+0000 7f30ec0c4640 1 -- 172.21.15.47:0/2536975872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f30e4066be0 con 0x7f30e415c990 2024-03-23T09:45:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.473+0000 7f30da7fc640 1 -- 172.21.15.47:0/2536975872 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f30dc003e60 con 0x7f30e415c990 2024-03-23T09:45:32.473 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:45:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.475+0000 7f30ec0c4640 1 -- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f30c806ac00 msgr2=0x7f30c806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.475+0000 7f30ec0c4640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f30c806ac00 0x7f30c806d0c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f30e0036000 tx=0x7f30e003c040 comp rx=0 tx=0).stop 2024-03-23T09:45:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.475+0000 7f30ec0c4640 1 -- 172.21.15.47:0/2536975872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e415c990 msgr2=0x7f30e4154340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.475+0000 7f30ec0c4640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e415c990 0x7f30e4154340 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c7d0 tx=0x7f30dc00cc90 comp rx=0 tx=0).stop 2024-03-23T09:45:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.475+0000 7f30ec0c4640 1 -- 172.21.15.47:0/2536975872 shutdown_connections 2024-03-23T09:45:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.475+0000 7f30ec0c4640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f30c806ac00 0x7f30c806d0c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.475+0000 7f30ec0c4640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e415c990 0x7f30e4154340 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.475+0000 7f30ec0c4640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30e40cdc40 0x7f30e4150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.475+0000 7f30ec0c4640 1 --2- 172.21.15.47:0/2536975872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30e40b8350 0x7f30e4152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.475+0000 7f30ec0c4640 1 -- 172.21.15.47:0/2536975872 >> 172.21.15.47:0/2536975872 conn(0x7f30e40659d0 msgr2=0x7f30e40d1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.475+0000 7f30ec0c4640 1 -- 172.21.15.47:0/2536975872 shutdown_connections 2024-03-23T09:45:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.475+0000 7f30ec0c4640 1 -- 172.21.15.47:0/2536975872 wait complete. 2024-03-23T09:45:32.486 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:45:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.587+0000 7fc6fcfd2640 1 Processor -- start 2024-03-23T09:45:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.587+0000 7fc6fcfd2640 1 -- start start 2024-03-23T09:45:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6fcfd2640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc6f815cc40 0x7fc6f8160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6fcfd2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6f8160c90 0x7fc6f80cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6fcfd2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f80d09d0 con 0x7fc6f80cdbf0 2024-03-23T09:45:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6fcfd2640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f80ce0a0 con 0x7fc6f815cc40 2024-03-23T09:45:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6fcfd2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f80b5a80 con 0x7fc6f8160c90 2024-03-23T09:45:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6f6d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc6f80cdbf0 0x7fc6f80b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41340/0 (socket says 172.21.15.47:41340) 2024-03-23T09:45:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6f6d9d640 1 -- 172.21.15.47:0/2870910702 learned_addr learned my addr 172.21.15.47:0/2870910702 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523668121 0 0) 0x7fc6f80d09d0 con 0x7fc6f80cdbf0 2024-03-23T09:45:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6e4003520 con 0x7fc6f80cdbf0 2024-03-23T09:45:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6f5d9b640 1 --2- 172.21.15.47:0/2870910702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6f8160c90 0x7fc6f80cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6f659c640 1 --2- 172.21.15.47:0/2870910702 >> v2:172.21.15.175:3300/0 conn(0x7fc6f815cc40 0x7fc6f8160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 818548696 0 0) 0x7fc6e4003520 con 0x7fc6f80cdbf0 2024-03-23T09:45:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6f8160c90 msgr2=0x7fc6f80cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6f559a640 1 --2- 172.21.15.47:0/2870910702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6f8160c90 0x7fc6f80cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 >> v2:172.21.15.175:3300/0 conn(0x7fc6f815cc40 msgr2=0x7fc6f8160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6f559a640 1 --2- 172.21.15.47:0/2870910702 >> v2:172.21.15.175:3300/0 conn(0x7fc6f815cc40 0x7fc6f8160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f815d020 con 0x7fc6f80cdbf0 2024-03-23T09:45:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.588+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fc6e80019f0 con 0x7fc6f80cdbf0 2024-03-23T09:45:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 >> v1:172.21.15.47:6789/0 conn(0x7fc6f80cdbf0 legacy=0x7fc6f80b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f559a640 1 --2- 172.21.15.47:0/2870910702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6e4003970 0x7fc6e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f559a640 1 --2- 172.21.15.47:0/2870910702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6f8160c90 0x7fc6e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f559a640 1 --2- 172.21.15.47:0/2870910702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6e4004d00 0x7fc6e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f80ce0a0 con 0x7fc6e4004d00 2024-03-23T09:45:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f80b5a80 con 0x7fc6e4003970 2024-03-23T09:45:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e400b980 con 0x7fc6f8160c90 2024-03-23T09:45:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f6d9d640 1 --2- 172.21.15.47:0/2870910702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6e4004d00 0x7fc6e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f659c640 1 --2- 172.21.15.47:0/2870910702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6e4003970 0x7fc6e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f5d9b640 1 --2- 172.21.15.47:0/2870910702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6f8160c90 0x7fc6e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f6d9d640 1 -- 172.21.15.47:0/2870910702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6f8160c90 msgr2=0x7fc6e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f6d9d640 1 --2- 172.21.15.47:0/2870910702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6f8160c90 0x7fc6e40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f6d9d640 1 -- 172.21.15.47:0/2870910702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6e4003970 msgr2=0x7fc6e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f6d9d640 1 --2- 172.21.15.47:0/2870910702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6e4003970 0x7fc6e4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.589+0000 7fc6f6d9d640 1 -- 172.21.15.47:0/2870910702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e4003520 con 0x7fc6e4004d00 2024-03-23T09:45:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.590+0000 7fc6f6d9d640 1 --2- 172.21.15.47:0/2870910702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6e4004d00 0x7fc6e4005180 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800a900 tx=0x7fc6e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4137a0f85174b92e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.590+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8034630 con 0x7fc6e4004d00 2024-03-23T09:45:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.590+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e802f070 con 0x7fc6e4004d00 2024-03-23T09:45:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.590+0000 7fc6f559a640 1 -- 172.21.15.47:0/2870910702 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80349d0 con 0x7fc6e4004d00 2024-03-23T09:45:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.590+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/2870910702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6e4004d00 msgr2=0x7fc6e4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.590+0000 7fc6fcfd2640 1 --2- 172.21.15.47:0/2870910702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6e4004d00 0x7fc6e4005180 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800a900 tx=0x7fc6e40057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.590+0000 7fc6f6d9d640 1 -- 172.21.15.47:0/2870910702 reap_dead start 2024-03-23T09:45:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.590+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/2870910702 shutdown_connections 2024-03-23T09:45:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.591+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/2870910702 >> 172.21.15.47:0/2870910702 conn(0x7fc6f806b160 msgr2=0x7fc6f806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.591+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/2870910702 shutdown_connections 2024-03-23T09:45:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.591+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/2870910702 wait complete. 2024-03-23T09:45:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.592+0000 7fc6fcfd2640 1 Processor -- start 2024-03-23T09:45:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.592+0000 7fc6fcfd2640 1 -- start start 2024-03-23T09:45:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6fcfd2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6e4004d00 0x7fc6f80c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6fcfd2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f80c8ff0 0x7fc6f815ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6fcfd2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f80c6410 0x7fc6f815f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6fcfd2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f81485a0 con 0x7fc6f80c8ff0 2024-03-23T09:45:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6fcfd2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f80c74d0 con 0x7fc6f80c6410 2024-03-23T09:45:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6fcfd2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f80c7d30 con 0x7fc6e4004d00 2024-03-23T09:45:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6f5d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f80c8ff0 0x7fc6f815ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6f5d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f80c8ff0 0x7fc6f815ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36172/0 (socket says 172.21.15.47:36172) 2024-03-23T09:45:32.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6f5d9b640 1 -- 172.21.15.47:0/1430697591 learned_addr learned my addr 172.21.15.47:0/1430697591 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:32.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6f659c640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6e4004d00 0x7fc6f80c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6f6d9d640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f80c6410 0x7fc6f815f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6f5d9b640 1 -- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6e4004d00 msgr2=0x7fc6f80c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6f5d9b640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6e4004d00 0x7fc6f80c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6f5d9b640 1 -- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f80c6410 msgr2=0x7fc6f815f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6f5d9b640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f80c6410 0x7fc6f815f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6f5d9b640 1 -- 172.21.15.47:0/1430697591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f80c7770 con 0x7fc6f80c8ff0 2024-03-23T09:45:32.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6f6d9d640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f80c6410 0x7fc6f815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:32.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6f5d9b640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f80c8ff0 0x7fc6f815ed80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00119a0 tx=0x7fc6e0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:32.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6eeffd640 1 -- 172.21.15.47:0/1430697591 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000fa30 con 0x7fc6f80c8ff0 2024-03-23T09:45:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6eeffd640 1 -- 172.21.15.47:0/1430697591 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e000fb90 con 0x7fc6f80c8ff0 2024-03-23T09:45:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/1430697591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f80c7a50 con 0x7fc6f80c8ff0 2024-03-23T09:45:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6eeffd640 1 -- 172.21.15.47:0/1430697591 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0014440 con 0x7fc6f80c8ff0 2024-03-23T09:45:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.593+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/1430697591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f80c8c70 con 0x7fc6f80c8ff0 2024-03-23T09:45:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.594+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/1430697591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f80c8e60 con 0x7fc6f80c8ff0 2024-03-23T09:45:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.595+0000 7fc6eeffd640 1 -- 172.21.15.47:0/1430697591 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc6e00145a0 con 0x7fc6f80c8ff0 2024-03-23T09:45:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.595+0000 7fc6eeffd640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc6d806ac30 0x7fc6d806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.595+0000 7fc6eeffd640 1 -- 172.21.15.47:0/1430697591 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6e0089bf0 con 0x7fc6f80c8ff0 2024-03-23T09:45:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.595+0000 7fc6f659c640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc6d806ac30 0x7fc6d806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.596+0000 7fc6f659c640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc6d806ac30 0x7fc6d806d0f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009bc0 tx=0x7fc6f0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:32.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.599+0000 7fc6eeffd640 1 -- 172.21.15.47:0/1430697591 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc6e0054dd0 con 0x7fc6f80c8ff0 2024-03-23T09:45:32.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.792+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/1430697591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc6f8072360 con 0x7fc6f80c8ff0 2024-03-23T09:45:32.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.792+0000 7fc6eeffd640 1 -- 172.21.15.47:0/1430697591 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc6e00583f0 con 0x7fc6f80c8ff0 2024-03-23T09:45:32.793 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:45:32.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.794+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc6d806ac30 msgr2=0x7fc6d806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.794+0000 7fc6fcfd2640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc6d806ac30 0x7fc6d806d0f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009bc0 tx=0x7fc6f0009190 comp rx=0 tx=0).stop 2024-03-23T09:45:32.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.794+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/1430697591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f80c8ff0 msgr2=0x7fc6f815ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:32.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.794+0000 7fc6fcfd2640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f80c8ff0 0x7fc6f815ed80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00119a0 tx=0x7fc6e0011da0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.794+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/1430697591 shutdown_connections 2024-03-23T09:45:32.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.794+0000 7fc6fcfd2640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc6d806ac30 0x7fc6d806d0f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.794+0000 7fc6fcfd2640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f80c6410 0x7fc6f815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.794+0000 7fc6fcfd2640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f80c8ff0 0x7fc6f815ed80 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.794+0000 7fc6fcfd2640 1 --2- 172.21.15.47:0/1430697591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6e4004d00 0x7fc6f80c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:32.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.795+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/1430697591 >> 172.21.15.47:0/1430697591 conn(0x7fc6f806b160 msgr2=0x7fc6f80b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:32.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.795+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/1430697591 shutdown_connections 2024-03-23T09:45:32.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:32.795+0000 7fc6fcfd2640 1 -- 172.21.15.47:0/1430697591 wait complete. 2024-03-23T09:45:32.805 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:45:32.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:32.805 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:32.830 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[61509]: fuse finished with error 0 and tester_r 0 2024-03-23T09:45:32.832 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:45:32.881 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:45:32.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:32.881 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:45:32.895 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:45:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.036+0000 7f14f9eab640 1 Processor -- start 2024-03-23T09:45:33.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.037+0000 7f14f9eab640 1 -- start start 2024-03-23T09:45:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.037+0000 7f14f9eab640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14f415a8d0 0x7f14f40cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.037+0000 7f14f9eab640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f14f40cb730 0x7f14f40cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.037+0000 7f14f9eab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f40cf190 0x7f14f4098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.037+0000 7f14f9eab640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f40c41c0 con 0x7f14f415a8d0 2024-03-23T09:45:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.037+0000 7f14f9eab640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f40cf600 con 0x7f14f40cb730 2024-03-23T09:45:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.037+0000 7f14f9eab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f40cc610 con 0x7f14f40cf190 2024-03-23T09:45:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.037+0000 7f14f37fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14f415a8d0 0x7f14f40cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.037+0000 7f14f37fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14f415a8d0 0x7f14f40cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36188/0 (socket says 172.21.15.47:36188) 2024-03-23T09:45:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.037+0000 7f14f37fe640 1 -- 172.21.15.47:0/2541910381 learned_addr learned my addr 172.21.15.47:0/2541910381 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.038+0000 7f14f37fe640 1 -- 172.21.15.47:0/2541910381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f40cf190 msgr2=0x7f14f4098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.038+0000 7f14f37fe640 1 --2- 172.21.15.47:0/2541910381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f40cf190 0x7f14f4098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.038+0000 7f14f37fe640 1 -- 172.21.15.47:0/2541910381 >> v2:172.21.15.175:3300/0 conn(0x7f14f40cb730 msgr2=0x7f14f40cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.038+0000 7f14f37fe640 1 --2- 172.21.15.47:0/2541910381 >> v2:172.21.15.175:3300/0 conn(0x7f14f40cb730 0x7f14f40cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.038+0000 7f14f37fe640 1 -- 172.21.15.47:0/2541910381 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f40b0700 con 0x7f14f415a8d0 2024-03-23T09:45:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.038+0000 7f14f37fe640 1 --2- 172.21.15.47:0/2541910381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14f415a8d0 0x7f14f40cb1f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f14f40b3170 tx=0x7f14e402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f33fd39ff9d389b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.039+0000 7f14f27fc640 1 -- 172.21.15.47:0/2541910381 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e40317e0 con 0x7f14f415a8d0 2024-03-23T09:45:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.039+0000 7f14f27fc640 1 -- 172.21.15.47:0/2541910381 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e4031940 con 0x7f14f415a8d0 2024-03-23T09:45:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.039+0000 7f14f27fc640 1 -- 172.21.15.47:0/2541910381 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e4031cd0 con 0x7f14f415a8d0 2024-03-23T09:45:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.039+0000 7f14f9eab640 1 -- 172.21.15.47:0/2541910381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14f415a8d0 msgr2=0x7f14f40cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.039+0000 7f14f9eab640 1 --2- 172.21.15.47:0/2541910381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14f415a8d0 0x7f14f40cb1f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f14f40b3170 tx=0x7f14e402b880 comp rx=0 tx=0).stop 2024-03-23T09:45:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.039+0000 7f14f9eab640 1 -- 172.21.15.47:0/2541910381 shutdown_connections 2024-03-23T09:45:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.039+0000 7f14f9eab640 1 --2- 172.21.15.47:0/2541910381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f40cf190 0x7f14f4098a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.039+0000 7f14f9eab640 1 --2- 172.21.15.47:0/2541910381 >> v2:172.21.15.175:3300/0 conn(0x7f14f40cb730 0x7f14f40cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.039+0000 7f14f9eab640 1 --2- 172.21.15.47:0/2541910381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14f415a8d0 0x7f14f40cb1f0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.039+0000 7f14f9eab640 1 -- 172.21.15.47:0/2541910381 >> 172.21.15.47:0/2541910381 conn(0x7f14f40d0670 msgr2=0x7f14f40d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.039+0000 7f14f9eab640 1 -- 172.21.15.47:0/2541910381 shutdown_connections 2024-03-23T09:45:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.039+0000 7f14f9eab640 1 -- 172.21.15.47:0/2541910381 wait complete. 2024-03-23T09:45:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.040+0000 7f14f9eab640 1 Processor -- start 2024-03-23T09:45:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.040+0000 7f14f9eab640 1 -- start start 2024-03-23T09:45:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.041+0000 7f14f9eab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f40cb730 0x7f14f41f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.041+0000 7f14f9eab640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14f40cf190 0x7f14f41f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.041+0000 7f14f9eab640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14f415a8d0 0x7f14f40ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.041+0000 7f14f9eab640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f40c96f0 con 0x7f14f40cf190 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.041+0000 7f14f9eab640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f40c9c90 con 0x7f14f415a8d0 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.041+0000 7f14f9eab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f41f4640 con 0x7f14f40cb730 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.041+0000 7f14f2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14f40cf190 0x7f14f41f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.041+0000 7f14f2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14f40cf190 0x7f14f41f12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36204/0 (socket says 172.21.15.47:36204) 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.041+0000 7f14f2ffd640 1 -- 172.21.15.47:0/1869065509 learned_addr learned my addr 172.21.15.47:0/1869065509 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.041+0000 7f14f37fe640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f40cb730 0x7f14f41f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.041+0000 7f14f3fff640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14f415a8d0 0x7f14f40ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.042+0000 7f14f2ffd640 1 -- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f40cb730 msgr2=0x7f14f41f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.042+0000 7f14f2ffd640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f40cb730 0x7f14f41f3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.042+0000 7f14f2ffd640 1 -- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14f415a8d0 msgr2=0x7f14f40ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.042+0000 7f14f2ffd640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14f415a8d0 0x7f14f40ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.042+0000 7f14f2ffd640 1 -- 172.21.15.47:0/1869065509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e40094d0 con 0x7f14f40cf190 2024-03-23T09:45:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.042+0000 7f14f2ffd640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14f40cf190 0x7f14f41f12e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f14e800b4a0 tx=0x7f14e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.042+0000 7f14dbfff640 1 -- 172.21.15.47:0/1869065509 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8010610 con 0x7f14f40cf190 2024-03-23T09:45:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.042+0000 7f14dbfff640 1 -- 172.21.15.47:0/1869065509 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e8010770 con 0x7f14f40cf190 2024-03-23T09:45:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.042+0000 7f14f9eab640 1 -- 172.21.15.47:0/1869065509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f41f4080 con 0x7f14f40cf190 2024-03-23T09:45:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.042+0000 7f14dbfff640 1 -- 172.21.15.47:0/1869065509 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8010b20 con 0x7f14f40cf190 2024-03-23T09:45:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.042+0000 7f14f9eab640 1 -- 172.21.15.47:0/1869065509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f40c9920 con 0x7f14f40cf190 2024-03-23T09:45:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.043+0000 7f14f9eab640 1 -- 172.21.15.47:0/1869065509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f40c9e00 con 0x7f14f40cf190 2024-03-23T09:45:33.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.043+0000 7f14dbfff640 1 -- 172.21.15.47:0/1869065509 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f14e8010c80 con 0x7f14f40cf190 2024-03-23T09:45:33.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.044+0000 7f14dbfff640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f14e006ac00 0x7f14e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.044+0000 7f14dbfff640 1 -- 172.21.15.47:0/1869065509 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14e8088e20 con 0x7f14f40cf190 2024-03-23T09:45:33.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.044+0000 7f14f37fe640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f14e006ac00 0x7f14e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.045+0000 7f14f37fe640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f14e006ac00 0x7f14e006d0c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f14e4036000 tx=0x7f14e403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:33.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.048+0000 7f14dbfff640 1 -- 172.21.15.47:0/1869065509 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f14e8054140 con 0x7f14f40cf190 2024-03-23T09:45:33.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.270+0000 7f14f9eab640 1 -- 172.21.15.47:0/1869065509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14f40672f0 con 0x7f14f40cf190 2024-03-23T09:45:33.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.271+0000 7f14dbfff640 1 -- 172.21.15.47:0/1869065509 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f14e801e090 con 0x7f14f40cf190 2024-03-23T09:45:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:33.271 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3862818105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3862818105},{"type":"v1","addr":"172.21.15.175:6836","nonce":3862818105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3615592850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3615592850},{"type":"v1","addr":"172.21.15.175:6837","nonce":3615592850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/832829684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":832829684},{"type":"v1","addr":"172.21.15.47:6834","nonce":832829684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:45:18.296714+0000","modified":"2024-03-23T09:45:20.302269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6744},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6744":{"gid":6744,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/372771687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":372771687},{"type":"v1","addr":"172.21.15.47:6835","nonce":372771687}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-23T09:45:33.271 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-23T09:45:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.273+0000 7f14f9eab640 1 -- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f14e006ac00 msgr2=0x7f14e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.273+0000 7f14f9eab640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f14e006ac00 0x7f14e006d0c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f14e4036000 tx=0x7f14e403d040 comp rx=0 tx=0).stop 2024-03-23T09:45:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.273+0000 7f14f9eab640 1 -- 172.21.15.47:0/1869065509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14f40cf190 msgr2=0x7f14f41f12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.273+0000 7f14f9eab640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14f40cf190 0x7f14f41f12e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f14e800b4a0 tx=0x7f14e800b960 comp rx=0 tx=0).stop 2024-03-23T09:45:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.273+0000 7f14f9eab640 1 -- 172.21.15.47:0/1869065509 shutdown_connections 2024-03-23T09:45:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.273+0000 7f14f9eab640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f14e006ac00 0x7f14e006d0c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.273+0000 7f14f9eab640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14f415a8d0 0x7f14f40ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.273+0000 7f14f9eab640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14f40cf190 0x7f14f41f12e0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.273+0000 7f14f9eab640 1 --2- 172.21.15.47:0/1869065509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f40cb730 0x7f14f41f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.273+0000 7f14f9eab640 1 -- 172.21.15.47:0/1869065509 >> 172.21.15.47:0/1869065509 conn(0x7f14f40d0670 msgr2=0x7f14f4157c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:33.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.273+0000 7f14f9eab640 1 -- 172.21.15.47:0/1869065509 shutdown_connections 2024-03-23T09:45:33.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.273+0000 7f14f9eab640 1 -- 172.21.15.47:0/1869065509 wait complete. 2024-03-23T09:45:33.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-23T09:45:33.404 INFO:teuthology.orchestra.run.smithi047.stdout:{"message":"","return_code":0} 2024-03-23T09:45:33.405 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-23T09:45:33.405 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:45:33.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.506+0000 7f44ff9fb640 1 Processor -- start 2024-03-23T09:45:33.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.506+0000 7f44ff9fb640 1 -- start start 2024-03-23T09:45:33.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.507+0000 7f44ff9fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f815cd00 0x7f44f80d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.507+0000 7f44ff9fb640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f44f80d1020 0x7f44f80cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.507+0000 7f44ff9fb640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f815d0e0 con 0x7f44f80cf750 2024-03-23T09:45:33.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.507+0000 7f44ff9fb640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f80cfbe0 con 0x7f44f80d1020 2024-03-23T09:45:33.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.507+0000 7f44ff9fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f80d1540 con 0x7f44f815cd00 2024-03-23T09:45:33.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.508+0000 7f44fd797640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f815cd00 0x7f44f80d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.508+0000 7f44fcf96640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f44f80d1020 0x7f44f80cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.508+0000 7f44fd797640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f815cd00 0x7f44f80d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:58414/0 (socket says 172.21.15.47:58414) 2024-03-23T09:45:33.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.508+0000 7f44fd797640 1 -- 172.21.15.47:0/1787616999 learned_addr learned my addr 172.21.15.47:0/1787616999 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:33.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.508+0000 7f44fd797640 1 -- 172.21.15.47:0/1787616999 >> v2:172.21.15.175:3300/0 conn(0x7f44f80d1020 msgr2=0x7f44f80cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:33.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.508+0000 7f44fd797640 1 --2- 172.21.15.47:0/1787616999 >> v2:172.21.15.175:3300/0 conn(0x7f44f80d1020 0x7f44f80cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.508+0000 7f44fd797640 1 -- 172.21.15.47:0/1787616999 >> v1:172.21.15.175:6789/0 conn(0x7f44f80cf750 legacy=0x7f44f80b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.508+0000 7f44effff640 1 -- 172.21.15.47:0/1787616999 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153459389 0 0) 0x7f44f815d0e0 con 0x7f44f80cf750 2024-03-23T09:45:33.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.508+0000 7f44fd797640 1 -- 172.21.15.47:0/1787616999 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f80d1820 con 0x7f44f815cd00 2024-03-23T09:45:33.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.508+0000 7f44fcf96640 1 --2- 172.21.15.47:0/1787616999 >> v2:172.21.15.175:3300/0 conn(0x7f44f80d1020 0x7f44f80cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.509+0000 7f44fd797640 1 --2- 172.21.15.47:0/1787616999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f815cd00 0x7f44f80d0ae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009e50 tx=0x7f44f402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbc0151a90567d87 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.509+0000 7f44effff640 1 -- 172.21.15.47:0/1787616999 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f402ca70 con 0x7f44f815cd00 2024-03-23T09:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.509+0000 7f44effff640 1 -- 172.21.15.47:0/1787616999 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f402cbd0 con 0x7f44f815cd00 2024-03-23T09:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.509+0000 7f44effff640 1 -- 172.21.15.47:0/1787616999 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f40343f0 con 0x7f44f815cd00 2024-03-23T09:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.509+0000 7f44ff9fb640 1 -- 172.21.15.47:0/1787616999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f815cd00 msgr2=0x7f44f80d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.509+0000 7f44ff9fb640 1 --2- 172.21.15.47:0/1787616999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f815cd00 0x7f44f80d0ae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009e50 tx=0x7f44f402bed0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.510+0000 7f44ff9fb640 1 -- 172.21.15.47:0/1787616999 shutdown_connections 2024-03-23T09:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.510+0000 7f44ff9fb640 1 --2- 172.21.15.47:0/1787616999 >> v2:172.21.15.175:3300/0 conn(0x7f44f80d1020 0x7f44f80cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.510+0000 7f44ff9fb640 1 --2- 172.21.15.47:0/1787616999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f815cd00 0x7f44f80d0ae0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.510+0000 7f44ff9fb640 1 -- 172.21.15.47:0/1787616999 >> 172.21.15.47:0/1787616999 conn(0x7f44f806b160 msgr2=0x7f44f806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:33.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.510+0000 7f44ff9fb640 1 -- 172.21.15.47:0/1787616999 shutdown_connections 2024-03-23T09:45:33.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.510+0000 7f44ff9fb640 1 -- 172.21.15.47:0/1787616999 wait complete. 2024-03-23T09:45:33.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.511+0000 7f44ff9fb640 1 Processor -- start 2024-03-23T09:45:33.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.511+0000 7f44ff9fb640 1 -- start start 2024-03-23T09:45:33.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.511+0000 7f44ff9fb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f80cf750 0x7f44f8151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.511+0000 7f44ff9fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f80d1020 0x7f44f80c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.511+0000 7f44ff9fb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44f815cd00 0x7f44f80c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.511+0000 7f44ff9fb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f80c72e0 con 0x7f44f80cf750 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.511+0000 7f44ff9fb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f81515d0 con 0x7f44f815cd00 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.511+0000 7f44ff9fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f80b46c0 con 0x7f44f80d1020 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44fd797640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f80cf750 0x7f44f8151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44fd797640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f80cf750 0x7f44f8151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36210/0 (socket says 172.21.15.47:36210) 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44fd797640 1 -- 172.21.15.47:0/2007167429 learned_addr learned my addr 172.21.15.47:0/2007167429 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44fcf96640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f80d1020 0x7f44f80c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44fdf98640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44f815cd00 0x7f44f80c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44fd797640 1 -- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f80d1020 msgr2=0x7f44f80c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44fd797640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f80d1020 0x7f44f80c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44fd797640 1 -- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44f815cd00 msgr2=0x7f44f80c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44fd797640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44f815cd00 0x7f44f80c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44fd797640 1 -- 172.21.15.47:0/2007167429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e8009400 con 0x7f44f80cf750 2024-03-23T09:45:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44fd797640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f80cf750 0x7f44f8151090 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009e50 tx=0x7f44f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44edffb640 1 -- 172.21.15.47:0/2007167429 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4009510 con 0x7f44f80cf750 2024-03-23T09:45:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44edffb640 1 -- 172.21.15.47:0/2007167429 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f403e450 con 0x7f44f80cf750 2024-03-23T09:45:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44ff9fb640 1 -- 172.21.15.47:0/2007167429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f4009b30 con 0x7f44f80cf750 2024-03-23T09:45:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.512+0000 7f44edffb640 1 -- 172.21.15.47:0/2007167429 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f403e600 con 0x7f44f80cf750 2024-03-23T09:45:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.513+0000 7f44ff9fb640 1 -- 172.21.15.47:0/2007167429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f80b2060 con 0x7f44f80cf750 2024-03-23T09:45:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.514+0000 7f44ff9fb640 1 -- 172.21.15.47:0/2007167429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f8072360 con 0x7f44f80cf750 2024-03-23T09:45:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.514+0000 7f44edffb640 1 -- 172.21.15.47:0/2007167429 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f44f403e760 con 0x7f44f80cf750 2024-03-23T09:45:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.514+0000 7f44edffb640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f44dc06ac00 0x7f44dc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.515+0000 7f44edffb640 1 -- 172.21.15.47:0/2007167429 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44f40abbe0 con 0x7f44f80cf750 2024-03-23T09:45:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.515+0000 7f44fcf96640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f44dc06ac00 0x7f44dc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.515+0000 7f44fcf96640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f44dc06ac00 0x7f44dc06d0c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f44f806bc00 tx=0x7f44e8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:33.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.519+0000 7f44edffb640 1 -- 172.21.15.47:0/2007167429 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f44f4076e80 con 0x7f44f80cf750 2024-03-23T09:45:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.739+0000 7f44ff9fb640 1 -- 172.21.15.47:0/2007167429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f44f8066be0 con 0x7f44f80cf750 2024-03-23T09:45:33.739 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:45:33.739+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:45:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.747+0000 7f44edffb640 1 -- 172.21.15.47:0/2007167429 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f44f407a4a0 con 0x7f44f80cf750 2024-03-23T09:45:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:45:33.748 INFO:tasks.ceph.mds.c.smithi047.stderr: -4459> 2024-03-23T09:45:13.918+0000 7f839fc28640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-03-23T09:45:33.749 INFO:tasks.ceph.mds.c.smithi047.stderr: -3361> 2024-03-23T09:45:19.922+0000 7f8399c1c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:45:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.750+0000 7f44ff9fb640 1 -- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f44dc06ac00 msgr2=0x7f44dc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.750+0000 7f44ff9fb640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f44dc06ac00 0x7f44dc06d0c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f44f806bc00 tx=0x7f44e8009190 comp rx=0 tx=0).stop 2024-03-23T09:45:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.750+0000 7f44ff9fb640 1 -- 172.21.15.47:0/2007167429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f80cf750 msgr2=0x7f44f8151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.750+0000 7f44ff9fb640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f80cf750 0x7f44f8151090 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009e50 tx=0x7f44f4004a40 comp rx=0 tx=0).stop 2024-03-23T09:45:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.750+0000 7f44ff9fb640 1 -- 172.21.15.47:0/2007167429 shutdown_connections 2024-03-23T09:45:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.750+0000 7f44ff9fb640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f44dc06ac00 0x7f44dc06d0c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.750+0000 7f44ff9fb640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44f815cd00 0x7f44f80c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.750+0000 7f44ff9fb640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f80d1020 0x7f44f80c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.750+0000 7f44ff9fb640 1 --2- 172.21.15.47:0/2007167429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f80cf750 0x7f44f8151090 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.750+0000 7f44ff9fb640 1 -- 172.21.15.47:0/2007167429 >> 172.21.15.47:0/2007167429 conn(0x7f44f806b160 msgr2=0x7f44f815bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.750+0000 7f44ff9fb640 1 -- 172.21.15.47:0/2007167429 shutdown_connections 2024-03-23T09:45:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.750+0000 7f44ff9fb640 1 -- 172.21.15.47:0/2007167429 wait complete. 2024-03-23T09:45:33.760 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-03-23T09:45:33.777 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:45:33.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.803+0000 7f388c435f40 1 Processor -- start 2024-03-23T09:45:33.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.803+0000 7f388c435f40 1 -- start start 2024-03-23T09:45:33.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f388c435f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55654c25dd30 0x55654c2e3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f388c435f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55654c25e110 0x55654c2ef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f388c435f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55654c2f2990 0x55654c2ef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f388c435f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654c165800 con 0x55654c25e110 2024-03-23T09:45:33.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f388c435f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654c148520 con 0x55654c2f2990 2024-03-23T09:45:33.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f388c435f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55654c24b560 con 0x55654c25dd30 2024-03-23T09:45:33.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f3888e41640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55654c25e110 0x55654c2ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f3888e41640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55654c25e110 0x55654c2ef350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36230/0 (socket says 172.21.15.47:36230) 2024-03-23T09:45:33.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f3888e41640 1 -- 172.21.15.47:0/1822978130 learned_addr learned my addr 172.21.15.47:0/1822978130 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:33.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f3889642640 1 --2- 172.21.15.47:0/1822978130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55654c25dd30 0x55654c2e3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f3889e43640 1 --2- 172.21.15.47:0/1822978130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55654c2f2990 0x55654c2ef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f3888e41640 1 -- 172.21.15.47:0/1822978130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55654c25dd30 msgr2=0x55654c2e3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:33.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f3888e41640 1 --2- 172.21.15.47:0/1822978130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55654c25dd30 0x55654c2e3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f3888e41640 1 -- 172.21.15.47:0/1822978130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55654c2f2990 msgr2=0x55654c2ef890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f3888e41640 1 --2- 172.21.15.47:0/1822978130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55654c2f2990 0x55654c2ef890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f3888e41640 1 -- 172.21.15.47:0/1822978130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3884005de0 con 0x55654c25e110 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f3889642640 1 --2- 172.21.15.47:0/1822978130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55654c25dd30 0x55654c2e3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f3888e41640 1 --2- 172.21.15.47:0/1822978130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55654c25e110 0x55654c2ef350 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f387400a770 tx=0x7f387400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c632862f28e675 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f38837fe640 1 -- 172.21.15.47:0/1822978130 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387400f070 con 0x55654c25e110 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f38837fe640 1 -- 172.21.15.47:0/1822978130 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3874007600 con 0x55654c25e110 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.804+0000 7f38837fe640 1 -- 172.21.15.47:0/1822978130 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874007990 con 0x55654c25e110 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.805+0000 7f388c435f40 1 -- 172.21.15.47:0/1822978130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55654c25e110 msgr2=0x55654c2ef350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.805+0000 7f388c435f40 1 --2- 172.21.15.47:0/1822978130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55654c25e110 0x55654c2ef350 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f387400a770 tx=0x7f387400a860 comp rx=0 tx=0).stop 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.805+0000 7f388c435f40 1 -- 172.21.15.47:0/1822978130 shutdown_connections 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.805+0000 7f388c435f40 1 --2- 172.21.15.47:0/1822978130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55654c2f2990 0x55654c2ef890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.805+0000 7f388c435f40 1 --2- 172.21.15.47:0/1822978130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55654c25e110 0x55654c2ef350 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.805+0000 7f388c435f40 1 --2- 172.21.15.47:0/1822978130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55654c25dd30 0x55654c2e3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.805+0000 7f388c435f40 1 -- 172.21.15.47:0/1822978130 >> 172.21.15.47:0/1822978130 conn(0x55654c248e40 msgr2=0x55654c265940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:33.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.805+0000 7f388c435f40 1 -- 172.21.15.47:0/1822978130 shutdown_connections 2024-03-23T09:45:33.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.805+0000 7f388c435f40 1 -- 172.21.15.47:0/1822978130 wait complete. 2024-03-23T09:45:33.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.806+0000 7f388c435f40 1 Processor -- start 2024-03-23T09:45:33.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.806+0000 7f388c435f40 1 -- start start 2024-03-23T09:45:33.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.806+0000 7f388c435f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55654c25dd30 0x55654c2ebe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.806+0000 7f388c435f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55654c25e110 0x55654c265b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.806+0000 7f388c435f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55654c2f2990 0x55654c2e79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.806+0000 7f388c435f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654c24e0b0 con 0x55654c25dd30 2024-03-23T09:45:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.806+0000 7f388c435f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654c2e2810 con 0x55654c25e110 2024-03-23T09:45:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.806+0000 7f388c435f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55654c24ddd0 con 0x55654c2f2990 2024-03-23T09:45:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.806+0000 7f3889642640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55654c25dd30 0x55654c2ebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.806+0000 7f3889642640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55654c25dd30 0x55654c2ebe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36240/0 (socket says 172.21.15.47:36240) 2024-03-23T09:45:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.806+0000 7f3889642640 1 -- 172.21.15.47:0/1588101183 learned_addr learned my addr 172.21.15.47:0/1588101183 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.807+0000 7f3889e43640 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55654c2f2990 0x55654c2e79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.807+0000 7f3888e41640 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55654c25e110 0x55654c265b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.807+0000 7f3889642640 1 -- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55654c2f2990 msgr2=0x55654c2e79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.807+0000 7f3889642640 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55654c2f2990 0x55654c2e79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.807+0000 7f3889642640 1 -- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55654c25e110 msgr2=0x55654c265b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.807+0000 7f3889642640 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55654c25e110 0x55654c265b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.807+0000 7f3889642640 1 -- 172.21.15.47:0/1588101183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387400a420 con 0x55654c25dd30 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.807+0000 7f3889642640 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55654c25dd30 0x55654c2ebe60 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f387c00e260 tx=0x7f387c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.807+0000 7f38827fc640 1 -- 172.21.15.47:0/1588101183 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c013870 con 0x55654c25dd30 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.807+0000 7f38827fc640 1 -- 172.21.15.47:0/1588101183 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c0139d0 con 0x55654c25dd30 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.807+0000 7f388c435f40 1 -- 172.21.15.47:0/1588101183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55654c18edc0 con 0x55654c25dd30 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.807+0000 7f38827fc640 1 -- 172.21.15.47:0/1588101183 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c013d60 con 0x55654c25dd30 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.807+0000 7f388c435f40 1 -- 172.21.15.47:0/1588101183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55654c174c20 con 0x55654c25dd30 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.809+0000 7f38827fc640 1 -- 172.21.15.47:0/1588101183 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f387c014410 con 0x55654c25dd30 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.809+0000 7f38827fc640 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f386c069610 0x7f386c06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.809+0000 7f38827fc640 1 -- 172.21.15.47:0/1588101183 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f387c088870 con 0x55654c25dd30 2024-03-23T09:45:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.809+0000 7f3888e41640 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f386c069610 0x7f386c06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.809+0000 7f388c435f40 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55654c266070 0x55654c390b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.809+0000 7f388c435f40 1 -- 172.21.15.47:0/1588101183 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55654c2e2aa0 con 0x55654c266070 2024-03-23T09:45:33.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.809+0000 7f3889e43640 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55654c266070 0x55654c390b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.810+0000 7f3888e41640 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f386c069610 0x7f386c06bad0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3874000c00 tx=0x7f387400c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:33.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.810+0000 7f3889e43640 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55654c266070 0x55654c390b60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:33.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.811+0000 7f3889e43640 1 -- 172.21.15.47:0/1588101183 <== osd.4 v2:172.21.15.175:6800/38159 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v71'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f38840526f0 con 0x55654c266070 2024-03-23T09:45:33.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.811+0000 7f388c435f40 1 -- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55654c266070 msgr2=0x55654c390b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.811+0000 7f388c435f40 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55654c266070 0x55654c390b60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.811+0000 7f388c435f40 1 -- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f386c069610 msgr2=0x7f386c06bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.811+0000 7f388c435f40 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f386c069610 0x7f386c06bad0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3874000c00 tx=0x7f387400c000 comp rx=0 tx=0).stop 2024-03-23T09:45:33.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.811+0000 7f388c435f40 1 -- 172.21.15.47:0/1588101183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55654c25dd30 msgr2=0x55654c2ebe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.811+0000 7f388c435f40 1 --2- 172.21.15.47:0/1588101183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55654c25dd30 0x55654c2ebe60 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f387c00e260 tx=0x7f387c00e720 comp rx=0 tx=0).stop 2024-03-23T09:45:33.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.811+0000 7f3889e43640 1 -- 172.21.15.47:0/1588101183 reap_dead start 2024-03-23T09:45:33.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.811+0000 7f388c435f40 1 -- 172.21.15.47:0/1588101183 shutdown_connections 2024-03-23T09:45:33.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.811+0000 7f388c435f40 1 -- 172.21.15.47:0/1588101183 >> 172.21.15.47:0/1588101183 conn(0x55654c248e40 msgr2=0x55654c2ef310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:33.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.811+0000 7f388c435f40 1 -- 172.21.15.47:0/1588101183 shutdown_connections 2024-03-23T09:45:33.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.811+0000 7f388c435f40 1 -- 172.21.15.47:0/1588101183 wait complete. 2024-03-23T09:45:33.815 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-23T09:45:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.960+0000 7f985168f640 1 Processor -- start 2024-03-23T09:45:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.960+0000 7f985168f640 1 -- start start 2024-03-23T09:45:33.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.961+0000 7f985168f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984c1571e0 0x7f984c06f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.961+0000 7f985168f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f984c06faf0 0x7f984c15d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.961+0000 7f985168f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f984c1575c0 con 0x7f984c15d600 2024-03-23T09:45:33.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.961+0000 7f985168f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984c06f190 con 0x7f984c06faf0 2024-03-23T09:45:33.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.961+0000 7f985168f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984c15da10 con 0x7f984c1571e0 2024-03-23T09:45:33.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.961+0000 7f984a7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f984c06faf0 0x7f984c15d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.961+0000 7f984b7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f984c15d600 0x7f984c0c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41370/0 (socket says 172.21.15.47:41370) 2024-03-23T09:45:33.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.961+0000 7f984b7fe640 1 -- 172.21.15.47:0/2279015295 learned_addr learned my addr 172.21.15.47:0/2279015295 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:33.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.961+0000 7f984affd640 1 --2- 172.21.15.47:0/2279015295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984c1571e0 0x7f984c06f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.961+0000 7f984a7fc640 1 -- 172.21.15.47:0/2279015295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984c1571e0 msgr2=0x7f984c06f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.961+0000 7f984a7fc640 1 --2- 172.21.15.47:0/2279015295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984c1571e0 0x7f984c06f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.961+0000 7f984a7fc640 1 -- 172.21.15.47:0/2279015295 >> v1:172.21.15.47:6789/0 conn(0x7f984c15d600 legacy=0x7f984c0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.961+0000 7f9849ffb640 1 -- 172.21.15.47:0/2279015295 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201730379 0 0) 0x7f984c1575c0 con 0x7f984c15d600 2024-03-23T09:45:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.962+0000 7f984a7fc640 1 -- 172.21.15.47:0/2279015295 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c15dbf0 con 0x7f984c06faf0 2024-03-23T09:45:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.962+0000 7f984a7fc640 1 --2- 172.21.15.47:0/2279015295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c06faf0 0x7f984c15d0c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f98340097a0 tx=0x7f983402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ca02268d83a7c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.962+0000 7f9849ffb640 1 -- 172.21.15.47:0/2279015295 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983402e070 con 0x7f984c06faf0 2024-03-23T09:45:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.962+0000 7f9849ffb640 1 -- 172.21.15.47:0/2279015295 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98340315a0 con 0x7f984c06faf0 2024-03-23T09:45:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.962+0000 7f9849ffb640 1 -- 172.21.15.47:0/2279015295 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834031930 con 0x7f984c06faf0 2024-03-23T09:45:33.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.963+0000 7f985168f640 1 -- 172.21.15.47:0/2279015295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c06faf0 msgr2=0x7f984c15d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.963+0000 7f985168f640 1 --2- 172.21.15.47:0/2279015295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c06faf0 0x7f984c15d0c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f98340097a0 tx=0x7f983402b840 comp rx=0 tx=0).stop 2024-03-23T09:45:33.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.963+0000 7f985168f640 1 -- 172.21.15.47:0/2279015295 shutdown_connections 2024-03-23T09:45:33.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.963+0000 7f985168f640 1 --2- 172.21.15.47:0/2279015295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c06faf0 0x7f984c15d0c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.963+0000 7f985168f640 1 --2- 172.21.15.47:0/2279015295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984c1571e0 0x7f984c06f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.963+0000 7f985168f640 1 -- 172.21.15.47:0/2279015295 >> 172.21.15.47:0/2279015295 conn(0x7f984c071400 msgr2=0x7f984c071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:33.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.963+0000 7f985168f640 1 -- 172.21.15.47:0/2279015295 shutdown_connections 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.963+0000 7f985168f640 1 -- 172.21.15.47:0/2279015295 wait complete. 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.964+0000 7f985168f640 1 Processor -- start 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.964+0000 7f985168f640 1 -- start start 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f985168f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984c06faf0 0x7f984c14e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f985168f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f984c1571e0 0x7f984c1537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f985168f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c15d600 0x7f984c153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f985168f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c0acd00 con 0x7f984c15d600 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f985168f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c0ac0f0 con 0x7f984c1571e0 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f985168f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984c0ac3d0 con 0x7f984c06faf0 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f984b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c15d600 0x7f984c153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f984b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c15d600 0x7f984c153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36248/0 (socket says 172.21.15.47:36248) 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f984b7fe640 1 -- 172.21.15.47:0/522831552 learned_addr learned my addr 172.21.15.47:0/522831552 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f984affd640 1 --2- 172.21.15.47:0/522831552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984c06faf0 0x7f984c14e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f984b7fe640 1 -- 172.21.15.47:0/522831552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984c06faf0 msgr2=0x7f984c14e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f984b7fe640 1 --2- 172.21.15.47:0/522831552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984c06faf0 0x7f984c14e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f984b7fe640 1 -- 172.21.15.47:0/522831552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f984c1571e0 msgr2=0x7f984c1537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:33.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f984b7fe640 1 --2- 172.21.15.47:0/522831552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f984c1571e0 0x7f984c1537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:33.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.965+0000 7f984b7fe640 1 -- 172.21.15.47:0/522831552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9834009400 con 0x7f984c15d600 2024-03-23T09:45:33.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.966+0000 7f984b7fe640 1 --2- 172.21.15.47:0/522831552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c15d600 0x7f984c153ce0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f983c00c780 tx=0x7f983c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:33.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.966+0000 7f98437fe640 1 -- 172.21.15.47:0/522831552 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c00ce90 con 0x7f984c15d600 2024-03-23T09:45:33.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.966+0000 7f98437fe640 1 -- 172.21.15.47:0/522831552 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c00a640 con 0x7f984c15d600 2024-03-23T09:45:33.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.966+0000 7f985168f640 1 -- 172.21.15.47:0/522831552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984c0ac6b0 con 0x7f984c15d600 2024-03-23T09:45:33.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.966+0000 7f98437fe640 1 -- 172.21.15.47:0/522831552 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c00a9d0 con 0x7f984c15d600 2024-03-23T09:45:33.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.966+0000 7f985168f640 1 -- 172.21.15.47:0/522831552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c159970 con 0x7f984c15d600 2024-03-23T09:45:33.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.967+0000 7f985168f640 1 -- 172.21.15.47:0/522831552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984c0672f0 con 0x7f984c15d600 2024-03-23T09:45:33.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.967+0000 7f98437fe640 1 -- 172.21.15.47:0/522831552 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f983c00ab30 con 0x7f984c15d600 2024-03-23T09:45:33.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.968+0000 7f98437fe640 1 --2- 172.21.15.47:0/522831552 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f982c06ac30 0x7f982c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:33.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.968+0000 7f98437fe640 1 -- 172.21.15.47:0/522831552 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f983c088ac0 con 0x7f984c15d600 2024-03-23T09:45:33.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.968+0000 7f984affd640 1 --2- 172.21.15.47:0/522831552 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f982c06ac30 0x7f982c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:33.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.968+0000 7f984affd640 1 --2- 172.21.15.47:0/522831552 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f982c06ac30 0x7f982c06d0f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9844006fd0 tx=0x7f9844008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:33.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:33.972+0000 7f98437fe640 1 -- 172.21.15.47:0/522831552 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f983c053cd0 con 0x7f984c15d600 2024-03-23T09:45:34.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.189+0000 7f985168f640 1 -- 172.21.15.47:0/522831552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f984c0cc1f0 con 0x7f984c15d600 2024-03-23T09:45:34.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.749+0000 7f98437fe640 1 -- 172.21.15.47:0/522831552 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v34) v1 ==== 162+0+0 (secure 0 0 0) 0x7f983c018020 con 0x7f984c15d600 2024-03-23T09:45:34.750 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-23T09:45:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.752+0000 7f985168f640 1 -- 172.21.15.47:0/522831552 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f982c06ac30 msgr2=0x7f982c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.752+0000 7f985168f640 1 --2- 172.21.15.47:0/522831552 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f982c06ac30 0x7f982c06d0f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9844006fd0 tx=0x7f9844008040 comp rx=0 tx=0).stop 2024-03-23T09:45:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.752+0000 7f985168f640 1 -- 172.21.15.47:0/522831552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c15d600 msgr2=0x7f984c153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.752+0000 7f985168f640 1 --2- 172.21.15.47:0/522831552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c15d600 0x7f984c153ce0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f983c00c780 tx=0x7f983c00cc40 comp rx=0 tx=0).stop 2024-03-23T09:45:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.752+0000 7f985168f640 1 -- 172.21.15.47:0/522831552 shutdown_connections 2024-03-23T09:45:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.752+0000 7f985168f640 1 --2- 172.21.15.47:0/522831552 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f982c06ac30 0x7f982c06d0f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.752+0000 7f985168f640 1 --2- 172.21.15.47:0/522831552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c15d600 0x7f984c153ce0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.752+0000 7f985168f640 1 --2- 172.21.15.47:0/522831552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f984c1571e0 0x7f984c1537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.752+0000 7f985168f640 1 --2- 172.21.15.47:0/522831552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984c06faf0 0x7f984c14e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.752+0000 7f985168f640 1 -- 172.21.15.47:0/522831552 >> 172.21.15.47:0/522831552 conn(0x7f984c071400 msgr2=0x7f984c069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.752+0000 7f985168f640 1 -- 172.21.15.47:0/522831552 shutdown_connections 2024-03-23T09:45:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.752+0000 7f985168f640 1 -- 172.21.15.47:0/522831552 wait complete. 2024-03-23T09:45:34.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:45:34.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.872+0000 7f0a1fce9640 1 Processor -- start 2024-03-23T09:45:34.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.873+0000 7f0a1fce9640 1 -- start start 2024-03-23T09:45:34.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.873+0000 7f0a1fce9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a1815aa70 0x7f0a180cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:34.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.873+0000 7f0a1fce9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0a180ce9d0 0x7f0a180cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:34.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.873+0000 7f0a1fce9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1815ae50 con 0x7f0a180cd620 2024-03-23T09:45:34.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.873+0000 7f0a1fce9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a180cda90 con 0x7f0a1815aa70 2024-03-23T09:45:34.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.873+0000 7f0a1fce9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a180cf010 con 0x7f0a180ce9d0 2024-03-23T09:45:34.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.874+0000 7f0a1da85640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a1815aa70 0x7f0a180cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:34.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.874+0000 7f0a1da85640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a1815aa70 0x7f0a180cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49670/0 (socket says 172.21.15.47:49670) 2024-03-23T09:45:34.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.874+0000 7f0a1da85640 1 -- 172.21.15.47:0/3940951287 learned_addr learned my addr 172.21.15.47:0/3940951287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:34.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.874+0000 7f0a1e286640 1 --2- 172.21.15.47:0/3940951287 >> v2:172.21.15.175:3300/0 conn(0x7f0a180ce9d0 0x7f0a180cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:34.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.874+0000 7f0a1da85640 1 -- 172.21.15.47:0/3940951287 >> v2:172.21.15.175:3300/0 conn(0x7f0a180ce9d0 msgr2=0x7f0a180cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:34.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.874+0000 7f0a1da85640 1 --2- 172.21.15.47:0/3940951287 >> v2:172.21.15.175:3300/0 conn(0x7f0a180ce9d0 0x7f0a180cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:34.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.874+0000 7f0a1da85640 1 -- 172.21.15.47:0/3940951287 >> v1:172.21.15.175:6789/0 conn(0x7f0a180cd620 legacy=0x7f0a180ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:34.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.874+0000 7f0a1da85640 1 -- 172.21.15.47:0/3940951287 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a180cf2f0 con 0x7f0a1815aa70 2024-03-23T09:45:34.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.874+0000 7f0a1e286640 1 --2- 172.21.15.47:0/3940951287 >> v2:172.21.15.175:3300/0 conn(0x7f0a180ce9d0 0x7f0a180cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:34.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.875+0000 7f0a1da85640 1 --2- 172.21.15.47:0/3940951287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1815aa70 0x7f0a180cd0e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f0a18099d50 tx=0x7f0a1402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84838d1b9f6acec server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:34.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.875+0000 7f0a1ca83640 1 -- 172.21.15.47:0/3940951287 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1402cbb0 con 0x7f0a1815aa70 2024-03-23T09:45:34.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.875+0000 7f0a1ca83640 1 -- 172.21.15.47:0/3940951287 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1402cd10 con 0x7f0a1815aa70 2024-03-23T09:45:34.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.875+0000 7f0a1ca83640 1 -- 172.21.15.47:0/3940951287 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14034610 con 0x7f0a1815aa70 2024-03-23T09:45:34.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.875+0000 7f0a1fce9640 1 -- 172.21.15.47:0/3940951287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1815aa70 msgr2=0x7f0a180cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:34.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.875+0000 7f0a1fce9640 1 --2- 172.21.15.47:0/3940951287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1815aa70 0x7f0a180cd0e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f0a18099d50 tx=0x7f0a1402c010 comp rx=0 tx=0).stop 2024-03-23T09:45:34.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.875+0000 7f0a1fce9640 1 -- 172.21.15.47:0/3940951287 shutdown_connections 2024-03-23T09:45:34.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.875+0000 7f0a1fce9640 1 --2- 172.21.15.47:0/3940951287 >> v2:172.21.15.175:3300/0 conn(0x7f0a180ce9d0 0x7f0a180cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:34.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.875+0000 7f0a1fce9640 1 --2- 172.21.15.47:0/3940951287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1815aa70 0x7f0a180cd0e0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:34.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.875+0000 7f0a1fce9640 1 -- 172.21.15.47:0/3940951287 >> 172.21.15.47:0/3940951287 conn(0x7f0a180d06f0 msgr2=0x7f0a180d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:34.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.875+0000 7f0a1fce9640 1 -- 172.21.15.47:0/3940951287 shutdown_connections 2024-03-23T09:45:34.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.875+0000 7f0a1fce9640 1 -- 172.21.15.47:0/3940951287 wait complete. 2024-03-23T09:45:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.877+0000 7f0a1fce9640 1 Processor -- start 2024-03-23T09:45:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.877+0000 7f0a1fce9640 1 -- start start 2024-03-23T09:45:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.877+0000 7f0a1fce9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a180cd620 0x7f0a1814ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.877+0000 7f0a1fce9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a180ce9d0 0x7f0a181f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.877+0000 7f0a1fce9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a1815aa70 0x7f0a180c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.877+0000 7f0a1fce9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1815df90 con 0x7f0a180ce9d0 2024-03-23T09:45:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.877+0000 7f0a1fce9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181f4030 con 0x7f0a1815aa70 2024-03-23T09:45:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.877+0000 7f0a1fce9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181f4310 con 0x7f0a180cd620 2024-03-23T09:45:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.877+0000 7f0a1d284640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a180ce9d0 0x7f0a181f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.877+0000 7f0a1d284640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a180ce9d0 0x7f0a181f3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49682/0 (socket says 172.21.15.47:49682) 2024-03-23T09:45:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.877+0000 7f0a1d284640 1 -- 172.21.15.47:0/1983680653 learned_addr learned my addr 172.21.15.47:0/1983680653 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.878+0000 7f0a1da85640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a180cd620 0x7f0a1814ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.878+0000 7f0a1d284640 1 -- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a180cd620 msgr2=0x7f0a1814ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.878+0000 7f0a1d284640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a180cd620 0x7f0a1814ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.878+0000 7f0a1d284640 1 -- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a1815aa70 msgr2=0x7f0a180c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.878+0000 7f0a1d284640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a1815aa70 0x7f0a180c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.878+0000 7f0a1d284640 1 -- 172.21.15.47:0/1983680653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a14009b30 con 0x7f0a180ce9d0 2024-03-23T09:45:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.878+0000 7f0a1da85640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a180cd620 0x7f0a1814ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.878+0000 7f0a1d284640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a180ce9d0 0x7f0a181f3730 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800c430 tx=0x7f0a0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.878+0000 7f0a0e7fc640 1 -- 172.21.15.47:0/1983680653 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080052d0 con 0x7f0a180ce9d0 2024-03-23T09:45:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.878+0000 7f0a0e7fc640 1 -- 172.21.15.47:0/1983680653 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a08005430 con 0x7f0a180ce9d0 2024-03-23T09:45:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.878+0000 7f0a0e7fc640 1 -- 172.21.15.47:0/1983680653 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08003850 con 0x7f0a180ce9d0 2024-03-23T09:45:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.878+0000 7f0a1fce9640 1 -- 172.21.15.47:0/1983680653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a181f45f0 con 0x7f0a180ce9d0 2024-03-23T09:45:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.878+0000 7f0a1fce9640 1 -- 172.21.15.47:0/1983680653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1815b3c0 con 0x7f0a180ce9d0 2024-03-23T09:45:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.879+0000 7f0a1fce9640 1 -- 172.21.15.47:0/1983680653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a18066be0 con 0x7f0a180ce9d0 2024-03-23T09:45:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.880+0000 7f0a0e7fc640 1 -- 172.21.15.47:0/1983680653 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0a080039b0 con 0x7f0a180ce9d0 2024-03-23T09:45:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.880+0000 7f0a0e7fc640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09fc06ac00 0x7f09fc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.880+0000 7f0a0e7fc640 1 -- 172.21.15.47:0/1983680653 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0a080888e0 con 0x7f0a180ce9d0 2024-03-23T09:45:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.880+0000 7f0a1da85640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09fc06ac00 0x7f09fc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.881+0000 7f0a1da85640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09fc06ac00 0x7f09fc06d0c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0a14035000 tx=0x7f0a14030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:34.884+0000 7f0a0e7fc640 1 -- 172.21.15.47:0/1983680653 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0a08053af0 con 0x7f0a180ce9d0 2024-03-23T09:45:35.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.101+0000 7f0a1fce9640 1 -- 172.21.15.47:0/1983680653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a180ca0d0 con 0x7f0a180ce9d0 2024-03-23T09:45:35.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.102+0000 7f0a0e7fc640 1 -- 172.21.15.47:0/1983680653 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f0a08017090 con 0x7f0a180ce9d0 2024-03-23T09:45:35.102 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:35.102 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3862818105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3862818105},{"type":"v1","addr":"172.21.15.175:6836","nonce":3862818105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3615592850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3615592850},{"type":"v1","addr":"172.21.15.175:6837","nonce":3615592850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/832829684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":832829684},{"type":"v1","addr":"172.21.15.47:6834","nonce":832829684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:45:18.296714+0000","modified":"2024-03-23T09:45:34.750740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7005":{"gid":7005,"name":"c","rank":0,"incarnation":35,"state":"up:replay","state_seq":1,"addr":"172.21.15.47:6835/4141889683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4141889683},{"type":"v1","addr":"172.21.15.47:6835","nonce":4141889683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-23T09:45:35.102 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 35 2024-03-23T09:45:35.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.104+0000 7f0a1fce9640 1 -- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09fc06ac00 msgr2=0x7f09fc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:35.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.104+0000 7f0a1fce9640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09fc06ac00 0x7f09fc06d0c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0a14035000 tx=0x7f0a14030000 comp rx=0 tx=0).stop 2024-03-23T09:45:35.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.104+0000 7f0a1fce9640 1 -- 172.21.15.47:0/1983680653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a180ce9d0 msgr2=0x7f0a181f3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:35.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.104+0000 7f0a1fce9640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a180ce9d0 0x7f0a181f3730 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800c430 tx=0x7f0a0800c8f0 comp rx=0 tx=0).stop 2024-03-23T09:45:35.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.104+0000 7f0a1fce9640 1 -- 172.21.15.47:0/1983680653 shutdown_connections 2024-03-23T09:45:35.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.104+0000 7f0a1fce9640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09fc06ac00 0x7f09fc06d0c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:35.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.104+0000 7f0a1fce9640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a1815aa70 0x7f0a180c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:35.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.104+0000 7f0a1fce9640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a180ce9d0 0x7f0a181f3730 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:35.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.104+0000 7f0a1fce9640 1 --2- 172.21.15.47:0/1983680653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a180cd620 0x7f0a1814ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:35.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.104+0000 7f0a1fce9640 1 -- 172.21.15.47:0/1983680653 >> 172.21.15.47:0/1983680653 conn(0x7f0a180d06f0 msgr2=0x7f0a181584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:35.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.104+0000 7f0a1fce9640 1 -- 172.21.15.47:0/1983680653 shutdown_connections 2024-03-23T09:45:35.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:35.104+0000 7f0a1fce9640 1 -- 172.21.15.47:0/1983680653 wait complete. 2024-03-23T09:45:35.516 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:45:18.296714+0000', 'modified': '2024-03-23T09:45:34.750740+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7005}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7005': {'gid': 7005, 'name': 'c', 'rank': 0, 'incarnation': 35, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.47:6835/4141889683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6833', 'nonce': 4141889683}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 4141889683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:45:35.516 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7005:up:replay 2024-03-23T09:45:36.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:45:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.631+0000 7fc6034b9640 1 Processor -- start 2024-03-23T09:45:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.631+0000 7fc6034b9640 1 -- start start 2024-03-23T09:45:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc6034b9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc5fc0b8350 0x7fc5fc0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc6034b9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5fc0ce090 con 0x7fc5fc15ca90 2024-03-23T09:45:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc6034b9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5fc0d11e0 con 0x7fc5fc0cdc40 2024-03-23T09:45:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc6034b9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5fc160f60 con 0x7fc5fc0b8350 2024-03-23T09:45:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc601255640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc5fc15ca90 0x7fc5fc160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58684/0 (socket says 172.21.15.47:58684) 2024-03-23T09:45:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc601255640 1 -- 172.21.15.47:0/1347105449 learned_addr learned my addr 172.21.15.47:0/1347105449 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc600a54640 1 --2- 172.21.15.47:0/1347105449 >> v2:172.21.15.175:3300/0 conn(0x7fc5fc0b8350 0x7fc5fc0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844650458 0 0) 0x7fc5f8001110 con 0x7fc5fc15ca90 2024-03-23T09:45:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5e8003520 con 0x7fc5fc15ca90 2024-03-23T09:45:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 491480431 0 0) 0x7fc5e8003520 con 0x7fc5fc15ca90 2024-03-23T09:45:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 >> v2:172.21.15.175:3300/0 conn(0x7fc5fc0b8350 msgr2=0x7fc5fc0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc5f3fff640 1 --2- 172.21.15.47:0/1347105449 >> v2:172.21.15.175:3300/0 conn(0x7fc5fc0b8350 0x7fc5fc0cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 >> v1:172.21.15.175:6789/0 conn(0x7fc5fc0cdc40 legacy=0x7fc5fc0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc161240 con 0x7fc5fc15ca90 2024-03-23T09:45:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.632+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fc5f8001800 con 0x7fc5fc15ca90 2024-03-23T09:45:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 >> v1:172.21.15.47:6789/0 conn(0x7fc5fc15ca90 legacy=0x7fc5fc160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc5f3fff640 1 --2- 172.21.15.47:0/1347105449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e8003970 0x7fc5e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc5f3fff640 1 --2- 172.21.15.47:0/1347105449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5e80047c0 0x7fc5e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc5f3fff640 1 --2- 172.21.15.47:0/1347105449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5e8005180 0x7fc5e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc160f60 con 0x7fc5e8003970 2024-03-23T09:45:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e800c120 con 0x7fc5e8005180 2024-03-23T09:45:36.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e800c2e0 con 0x7fc5e80047c0 2024-03-23T09:45:36.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc601255640 1 --2- 172.21.15.47:0/1347105449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e8003970 0x7fc5e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:36.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc600a54640 1 --2- 172.21.15.47:0/1347105449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5e80047c0 0x7fc5e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:36.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc601a56640 1 --2- 172.21.15.47:0/1347105449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5e8005180 0x7fc5e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:36.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc601255640 1 -- 172.21.15.47:0/1347105449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5e80047c0 msgr2=0x7fc5e8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:36.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc601255640 1 --2- 172.21.15.47:0/1347105449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5e80047c0 0x7fc5e8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:36.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc601255640 1 -- 172.21.15.47:0/1347105449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5e8005180 msgr2=0x7fc5e800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:36.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc601255640 1 --2- 172.21.15.47:0/1347105449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5e8005180 0x7fc5e800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:36.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc601255640 1 -- 172.21.15.47:0/1347105449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e8003520 con 0x7fc5e8003970 2024-03-23T09:45:36.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.633+0000 7fc601a56640 1 -- 172.21.15.47:0/1347105449 reap_dead start 2024-03-23T09:45:36.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.634+0000 7fc601255640 1 --2- 172.21.15.47:0/1347105449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e8003970 0x7fc5e8003d90 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800c240 tx=0x7fc5e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6435788856da4d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:36.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.634+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8005ad0 con 0x7fc5e8003970 2024-03-23T09:45:36.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.634+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f8005c50 con 0x7fc5e8003970 2024-03-23T09:45:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.634+0000 7fc5f3fff640 1 -- 172.21.15.47:0/1347105449 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f802c8b0 con 0x7fc5e8003970 2024-03-23T09:45:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.634+0000 7fc6034b9640 1 -- 172.21.15.47:0/1347105449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e8003970 msgr2=0x7fc5e8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.634+0000 7fc6034b9640 1 --2- 172.21.15.47:0/1347105449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e8003970 0x7fc5e8003d90 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800c240 tx=0x7fc5e80057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.634+0000 7fc6034b9640 1 -- 172.21.15.47:0/1347105449 shutdown_connections 2024-03-23T09:45:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.634+0000 7fc6034b9640 1 --2- 172.21.15.47:0/1347105449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e8003970 0x7fc5e8003d90 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.634+0000 7fc6034b9640 1 -- 172.21.15.47:0/1347105449 >> 172.21.15.47:0/1347105449 conn(0x7fc5fc0659d0 msgr2=0x7fc5fc065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.635+0000 7fc6034b9640 1 -- 172.21.15.47:0/1347105449 shutdown_connections 2024-03-23T09:45:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.635+0000 7fc6034b9640 1 -- 172.21.15.47:0/1347105449 wait complete. 2024-03-23T09:45:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.636+0000 7fc6034b9640 1 Processor -- start 2024-03-23T09:45:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.636+0000 7fc6034b9640 1 -- start start 2024-03-23T09:45:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.636+0000 7fc6034b9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5e8003970 0x7fc5fc1516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.636+0000 7fc6034b9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5fc0b4690 0x7fc5fc153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.636+0000 7fc6034b9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5fc0cf200 0x7fc5fc0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.636+0000 7fc6034b9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc0cfbc0 con 0x7fc5fc0b4690 2024-03-23T09:45:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.636+0000 7fc6034b9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc0ca3e0 con 0x7fc5e8003970 2024-03-23T09:45:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.636+0000 7fc6034b9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc152570 con 0x7fc5fc0cf200 2024-03-23T09:45:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc600a54640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5fc0b4690 0x7fc5fc153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc600a54640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5fc0b4690 0x7fc5fc153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49708/0 (socket says 172.21.15.47:49708) 2024-03-23T09:45:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc600a54640 1 -- 172.21.15.47:0/3340440748 learned_addr learned my addr 172.21.15.47:0/3340440748 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc601a56640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5fc0cf200 0x7fc5fc0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc601255640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5e8003970 0x7fc5fc1516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc600a54640 1 -- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5fc0cf200 msgr2=0x7fc5fc0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc600a54640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5fc0cf200 0x7fc5fc0cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc600a54640 1 -- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5e8003970 msgr2=0x7fc5fc1516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc600a54640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5e8003970 0x7fc5fc1516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc600a54640 1 -- 172.21.15.47:0/3340440748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc151cd0 con 0x7fc5fc0b4690 2024-03-23T09:45:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc600a54640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5fc0b4690 0x7fc5fc153760 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec011700 tx=0x7fc5ec011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc5f1ffb640 1 -- 172.21.15.47:0/3340440748 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec00f9c0 con 0x7fc5fc0b4690 2024-03-23T09:45:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc5f1ffb640 1 -- 172.21.15.47:0/3340440748 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec00fb20 con 0x7fc5fc0b4690 2024-03-23T09:45:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc6034b9640 1 -- 172.21.15.47:0/3340440748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5fc151fb0 con 0x7fc5fc0b4690 2024-03-23T09:45:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.637+0000 7fc5f1ffb640 1 -- 172.21.15.47:0/3340440748 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec00fe90 con 0x7fc5fc0b4690 2024-03-23T09:45:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.638+0000 7fc6034b9640 1 -- 172.21.15.47:0/3340440748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5fc0ca610 con 0x7fc5fc0b4690 2024-03-23T09:45:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.639+0000 7fc6034b9640 1 -- 172.21.15.47:0/3340440748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5fc072360 con 0x7fc5fc0b4690 2024-03-23T09:45:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.639+0000 7fc5f1ffb640 1 -- 172.21.15.47:0/3340440748 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc5ec015460 con 0x7fc5fc0b4690 2024-03-23T09:45:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.639+0000 7fc5f1ffb640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc5e006ac30 0x7fc5e006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.639+0000 7fc5f1ffb640 1 -- 172.21.15.47:0/3340440748 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc5ec089b30 con 0x7fc5fc0b4690 2024-03-23T09:45:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.640+0000 7fc601255640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc5e006ac30 0x7fc5e006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.640+0000 7fc601255640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc5e006ac30 0x7fc5e006d0f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8034000 tx=0x7fc5f8009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.644+0000 7fc5f1ffb640 1 -- 172.21.15.47:0/3340440748 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc5ec054cd0 con 0x7fc5fc0b4690 2024-03-23T09:45:36.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.861+0000 7fc6034b9640 1 -- 172.21.15.47:0/3340440748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5fc066be0 con 0x7fc5fc0b4690 2024-03-23T09:45:36.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.861+0000 7fc5f1ffb640 1 -- 172.21.15.47:0/3340440748 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fc5ec0582f0 con 0x7fc5fc0b4690 2024-03-23T09:45:36.861 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:36.862 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3862818105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3862818105},{"type":"v1","addr":"172.21.15.175:6836","nonce":3862818105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3615592850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3615592850},{"type":"v1","addr":"172.21.15.175:6837","nonce":3615592850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/832829684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":832829684},{"type":"v1","addr":"172.21.15.47:6834","nonce":832829684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:45:18.296714+0000","modified":"2024-03-23T09:45:36.753362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7005":{"gid":7005,"name":"c","rank":0,"incarnation":35,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.47:6835/4141889683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4141889683},{"type":"v1","addr":"172.21.15.47:6835","nonce":4141889683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-23T09:45:36.862 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 37 2024-03-23T09:45:36.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.863+0000 7fc6034b9640 1 -- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc5e006ac30 msgr2=0x7fc5e006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:36.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.863+0000 7fc6034b9640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc5e006ac30 0x7fc5e006d0f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8034000 tx=0x7fc5f8009030 comp rx=0 tx=0).stop 2024-03-23T09:45:36.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.863+0000 7fc6034b9640 1 -- 172.21.15.47:0/3340440748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5fc0b4690 msgr2=0x7fc5fc153760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:36.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.863+0000 7fc6034b9640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5fc0b4690 0x7fc5fc153760 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec011700 tx=0x7fc5ec011bc0 comp rx=0 tx=0).stop 2024-03-23T09:45:36.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.863+0000 7fc6034b9640 1 -- 172.21.15.47:0/3340440748 shutdown_connections 2024-03-23T09:45:36.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.863+0000 7fc6034b9640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc5e006ac30 0x7fc5e006d0f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:36.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.863+0000 7fc6034b9640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5fc0cf200 0x7fc5fc0cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:36.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.863+0000 7fc6034b9640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5fc0b4690 0x7fc5fc153760 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:36.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.863+0000 7fc6034b9640 1 --2- 172.21.15.47:0/3340440748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5e8003970 0x7fc5fc1516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:36.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.863+0000 7fc6034b9640 1 -- 172.21.15.47:0/3340440748 >> 172.21.15.47:0/3340440748 conn(0x7fc5fc0659d0 msgr2=0x7fc5fc1578c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:36.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.863+0000 7fc6034b9640 1 -- 172.21.15.47:0/3340440748 shutdown_connections 2024-03-23T09:45:36.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:36.863+0000 7fc6034b9640 1 -- 172.21.15.47:0/3340440748 wait complete. 2024-03-23T09:45:37.276 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:45:18.296714+0000', 'modified': '2024-03-23T09:45:36.753362+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7005}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7005': {'gid': 7005, 'name': 'c', 'rank': 0, 'incarnation': 35, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.47:6835/4141889683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6833', 'nonce': 4141889683}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 4141889683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:45:37.277 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7005:up:rejoin 2024-03-23T09:45:37.646 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:45:37.649 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:45:38.277 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:45:38.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.391+0000 7f9d12789640 1 Processor -- start 2024-03-23T09:45:38.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.391+0000 7f9d12789640 1 -- start start 2024-03-23T09:45:38.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.391+0000 7f9d12789640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9d0c15cb40 0x7f9d0c160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:38.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.391+0000 7f9d12789640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0d0ee0 con 0x7f9d0c160cb0 2024-03-23T09:45:38.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.391+0000 7f9d12789640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c1616f0 con 0x7f9d0c0b8300 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.391+0000 7f9d12789640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0c0b5a10 con 0x7f9d0c15cb40 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d10d26640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9d0c160cb0 0x7f9d0c0d1370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58688/0 (socket says 172.21.15.47:58688) 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d10d26640 1 -- 172.21.15.47:0/260862242 learned_addr learned my addr 172.21.15.47:0/260862242 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0bfff640 1 --2- 172.21.15.47:0/260862242 >> v2:172.21.15.175:3300/0 conn(0x7f9d0c15cb40 0x7f9d0c160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773593894 0 0) 0x7f9d0c0d0ee0 con 0x7f9d0c160cb0 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf4003520 con 0x7f9d0c160cb0 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 158667892 0 0) 0x7f9cf4003520 con 0x7f9d0c160cb0 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 >> v2:172.21.15.175:3300/0 conn(0x7f9d0c15cb40 msgr2=0x7f9d0c160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 --2- 172.21.15.47:0/260862242 >> v2:172.21.15.175:3300/0 conn(0x7f9d0c15cb40 0x7f9d0c160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 >> v1:172.21.15.175:6789/0 conn(0x7f9d0c0b8300 legacy=0x7f9d0c0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c161130 con 0x7f9d0c160cb0 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f9d040019f0 con 0x7f9d0c160cb0 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 >> v1:172.21.15.47:6789/0 conn(0x7f9d0c160cb0 legacy=0x7f9d0c0d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 --2- 172.21.15.47:0/260862242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cf4003970 0x7f9cf4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 --2- 172.21.15.47:0/260862242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cf40047c0 0x7f9cf4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 --2- 172.21.15.47:0/260862242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cf40051d0 0x7f9cf400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0b5a10 con 0x7f9cf4003970 2024-03-23T09:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf400c100 con 0x7f9cf40047c0 2024-03-23T09:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf400c2c0 con 0x7f9cf40051d0 2024-03-23T09:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0bfff640 1 --2- 172.21.15.47:0/260862242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cf4003970 0x7f9cf4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.392+0000 7f9d0b7fe640 1 --2- 172.21.15.47:0/260862242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cf40047c0 0x7f9cf4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.393+0000 7f9d10d26640 1 --2- 172.21.15.47:0/260862242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cf40051d0 0x7f9cf400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.393+0000 7f9d0bfff640 1 -- 172.21.15.47:0/260862242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cf40051d0 msgr2=0x7f9cf400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.393+0000 7f9d0bfff640 1 --2- 172.21.15.47:0/260862242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cf40051d0 0x7f9cf400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.393+0000 7f9d0bfff640 1 -- 172.21.15.47:0/260862242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cf40047c0 msgr2=0x7f9cf4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.393+0000 7f9d0bfff640 1 --2- 172.21.15.47:0/260862242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cf40047c0 0x7f9cf4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.393+0000 7f9d0bfff640 1 -- 172.21.15.47:0/260862242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf4003520 con 0x7f9cf4003970 2024-03-23T09:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.393+0000 7f9d10d26640 1 -- 172.21.15.47:0/260862242 reap_dead start 2024-03-23T09:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.393+0000 7f9d0bfff640 1 --2- 172.21.15.47:0/260862242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cf4003970 0x7f9cf4003d90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009ac0 tx=0x7f9cf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad963ae08d90a5a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.394+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc02c6b0 con 0x7f9cf4003970 2024-03-23T09:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.394+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc02c810 con 0x7f9cf4003970 2024-03-23T09:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.394+0000 7f9d0affd640 1 -- 172.21.15.47:0/260862242 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc02cba0 con 0x7f9cf4003970 2024-03-23T09:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.394+0000 7f9d12789640 1 -- 172.21.15.47:0/260862242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cf4003970 msgr2=0x7f9cf4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.394+0000 7f9d12789640 1 --2- 172.21.15.47:0/260862242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cf4003970 0x7f9cf4003d90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009ac0 tx=0x7f9cf40057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.394+0000 7f9d12789640 1 -- 172.21.15.47:0/260862242 shutdown_connections 2024-03-23T09:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.394+0000 7f9d12789640 1 --2- 172.21.15.47:0/260862242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cf4003970 0x7f9cf4003d90 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.394+0000 7f9d12789640 1 -- 172.21.15.47:0/260862242 >> 172.21.15.47:0/260862242 conn(0x7f9d0c06b160 msgr2=0x7f9d0c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.394+0000 7f9d12789640 1 -- 172.21.15.47:0/260862242 shutdown_connections 2024-03-23T09:45:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.394+0000 7f9d12789640 1 -- 172.21.15.47:0/260862242 wait complete. 2024-03-23T09:45:38.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.395+0000 7f9d12789640 1 Processor -- start 2024-03-23T09:45:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.396+0000 7f9d12789640 1 -- start start 2024-03-23T09:45:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.396+0000 7f9d12789640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cf4003970 0x7f9d0c154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.396+0000 7f9d12789640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d0c1f2e00 0x7f9d0c14e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.396+0000 7f9d12789640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d0c14eab0 0x7f9d0c14ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.396+0000 7f9d12789640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c1534c0 con 0x7f9cf4003970 2024-03-23T09:45:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.396+0000 7f9d12789640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c1537a0 con 0x7f9d0c14eab0 2024-03-23T09:45:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.396+0000 7f9d12789640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c153a80 con 0x7f9d0c1f2e00 2024-03-23T09:45:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.396+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cf4003970 0x7f9d0c154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.396+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cf4003970 0x7f9d0c154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49724/0 (socket says 172.21.15.47:49724) 2024-03-23T09:45:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.396+0000 7f9d0bfff640 1 -- 172.21.15.47:0/3467071187 learned_addr learned my addr 172.21.15.47:0/3467071187 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.397+0000 7f9d0b7fe640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d0c1f2e00 0x7f9d0c14e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.397+0000 7f9d10d26640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d0c14eab0 0x7f9d0c14ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.397+0000 7f9d0bfff640 1 -- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d0c1f2e00 msgr2=0x7f9d0c14e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.397+0000 7f9d0bfff640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d0c1f2e00 0x7f9d0c14e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.397+0000 7f9d0bfff640 1 -- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d0c14eab0 msgr2=0x7f9d0c14ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.397+0000 7f9d0bfff640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d0c14eab0 0x7f9d0c14ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.397+0000 7f9d0bfff640 1 -- 172.21.15.47:0/3467071187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c153d60 con 0x7f9cf4003970 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.397+0000 7f9d0bfff640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cf4003970 0x7f9d0c154560 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc02c250 tx=0x7f9cfc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.397+0000 7f9d08ff9640 1 -- 172.21.15.47:0/3467071187 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc003810 con 0x7f9cf4003970 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.397+0000 7f9d08ff9640 1 -- 172.21.15.47:0/3467071187 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc003970 con 0x7f9cf4003970 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.397+0000 7f9d12789640 1 -- 172.21.15.47:0/3467071187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c154040 con 0x7f9cf4003970 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.397+0000 7f9d08ff9640 1 -- 172.21.15.47:0/3467071187 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc003d00 con 0x7f9cf4003970 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.397+0000 7f9d12789640 1 -- 172.21.15.47:0/3467071187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c1f3280 con 0x7f9cf4003970 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.398+0000 7f9d12789640 1 -- 172.21.15.47:0/3467071187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c072360 con 0x7f9cf4003970 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.399+0000 7f9d08ff9640 1 -- 172.21.15.47:0/3467071187 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9cfc047450 con 0x7f9cf4003970 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.399+0000 7f9d08ff9640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9cf806ac00 0x7f9cf806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.399+0000 7f9d08ff9640 1 -- 172.21.15.47:0/3467071187 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9cfc0ab750 con 0x7f9cf4003970 2024-03-23T09:45:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.399+0000 7f9d0b7fe640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9cf806ac00 0x7f9cf806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.400+0000 7f9d0b7fe640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9cf806ac00 0x7f9cf806d0c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9d00000c00 tx=0x7f9d00009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.403+0000 7f9d08ff9640 1 -- 172.21.15.47:0/3467071187 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9cfc076aa0 con 0x7f9cf4003970 2024-03-23T09:45:38.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.618+0000 7f9d12789640 1 -- 172.21.15.47:0/3467071187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d0c066be0 con 0x7f9cf4003970 2024-03-23T09:45:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.619+0000 7f9d08ff9640 1 -- 172.21.15.47:0/3467071187 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f9cfc003e60 con 0x7f9cf4003970 2024-03-23T09:45:38.619 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:38.619 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3862818105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3862818105},{"type":"v1","addr":"172.21.15.175:6836","nonce":3862818105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3615592850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3615592850},{"type":"v1","addr":"172.21.15.175:6837","nonce":3615592850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/832829684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":832829684},{"type":"v1","addr":"172.21.15.47:6834","nonce":832829684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:45:18.296714+0000","modified":"2024-03-23T09:45:37.755132+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7005":{"gid":7005,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6835/4141889683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4141889683},{"type":"v1","addr":"172.21.15.47:6835","nonce":4141889683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-23T09:45:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 38 2024-03-23T09:45:38.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.621+0000 7f9d12789640 1 -- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9cf806ac00 msgr2=0x7f9cf806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:38.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.621+0000 7f9d12789640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9cf806ac00 0x7f9cf806d0c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9d00000c00 tx=0x7f9d00009040 comp rx=0 tx=0).stop 2024-03-23T09:45:38.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.621+0000 7f9d12789640 1 -- 172.21.15.47:0/3467071187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cf4003970 msgr2=0x7f9d0c154560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:38.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.621+0000 7f9d12789640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cf4003970 0x7f9d0c154560 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc02c250 tx=0x7f9cfc004730 comp rx=0 tx=0).stop 2024-03-23T09:45:38.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.621+0000 7f9d12789640 1 -- 172.21.15.47:0/3467071187 shutdown_connections 2024-03-23T09:45:38.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.621+0000 7f9d12789640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9cf806ac00 0x7f9cf806d0c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:38.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.621+0000 7f9d12789640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d0c14eab0 0x7f9d0c14ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:38.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.621+0000 7f9d12789640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d0c1f2e00 0x7f9d0c14e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:38.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.621+0000 7f9d12789640 1 --2- 172.21.15.47:0/3467071187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cf4003970 0x7f9d0c154560 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:38.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.621+0000 7f9d12789640 1 -- 172.21.15.47:0/3467071187 >> 172.21.15.47:0/3467071187 conn(0x7f9d0c06b160 msgr2=0x7f9d0c0d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:38.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.621+0000 7f9d12789640 1 -- 172.21.15.47:0/3467071187 shutdown_connections 2024-03-23T09:45:38.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:38.621+0000 7f9d12789640 1 -- 172.21.15.47:0/3467071187 wait complete. 2024-03-23T09:45:39.034 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:45:18.296714+0000', 'modified': '2024-03-23T09:45:37.755132+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7005}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7005': {'gid': 7005, 'name': 'c', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.47:6835/4141889683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6833', 'nonce': 4141889683}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 4141889683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:45:39.034 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:45:39.034 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T09:45:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.140+0000 7ff1df52e640 1 Processor -- start 2024-03-23T09:45:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.140+0000 7ff1df52e640 1 -- start start 2024-03-23T09:45:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.141+0000 7ff1df52e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff1d815cc40 0x7ff1d8160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.141+0000 7ff1df52e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d80d09d0 con 0x7ff1d8160c90 2024-03-23T09:45:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.141+0000 7ff1df52e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d80b5a80 con 0x7ff1d80cddc0 2024-03-23T09:45:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.141+0000 7ff1df52e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d815d020 con 0x7ff1d815cc40 2024-03-23T09:45:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.141+0000 7ff1dcac9640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff1d8160c90 0x7ff1d80cd6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58700/0 (socket says 172.21.15.47:58700) 2024-03-23T09:45:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.141+0000 7ff1dcac9640 1 -- 172.21.15.47:0/1164450495 learned_addr learned my addr 172.21.15.47:0/1164450495 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.141+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742134311 0 0) 0x7ff1d80d09d0 con 0x7ff1d8160c90 2024-03-23T09:45:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.141+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1c0003520 con 0x7ff1d8160c90 2024-03-23T09:45:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.141+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/1164450495 >> v2:172.21.15.175:3300/0 conn(0x7ff1d815cc40 0x7ff1d8160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.141+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3694917715 0 0) 0x7ff1c0003520 con 0x7ff1d8160c90 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 >> v2:172.21.15.175:3300/0 conn(0x7ff1d815cc40 msgr2=0x7ff1d8160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1d7fff640 1 --2- 172.21.15.47:0/1164450495 >> v2:172.21.15.175:3300/0 conn(0x7ff1d815cc40 0x7ff1d8160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 >> v1:172.21.15.175:6789/0 conn(0x7ff1d80cddc0 legacy=0x7ff1d80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d81611a0 con 0x7ff1d8160c90 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7ff1cc001670 con 0x7ff1d8160c90 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 >> v1:172.21.15.47:6789/0 conn(0x7ff1d8160c90 legacy=0x7ff1d80cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1d7fff640 1 --2- 172.21.15.47:0/1164450495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1c0003970 0x7ff1c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1d7fff640 1 --2- 172.21.15.47:0/1164450495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1c00047c0 0x7ff1c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1d7fff640 1 --2- 172.21.15.47:0/1164450495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1c0005180 0x7ff1c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d815d020 con 0x7ff1c0005180 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c000c0d0 con 0x7ff1c00047c0 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c000c290 con 0x7ff1c0003970 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1ddacb640 1 --2- 172.21.15.47:0/1164450495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1c0005180 0x7ff1c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1dcac9640 1 --2- 172.21.15.47:0/1164450495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1c00047c0 0x7ff1c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/1164450495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1c0003970 0x7ff1c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1ddacb640 1 -- 172.21.15.47:0/1164450495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1c0003970 msgr2=0x7ff1c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1ddacb640 1 --2- 172.21.15.47:0/1164450495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1c0003970 0x7ff1c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1ddacb640 1 -- 172.21.15.47:0/1164450495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1c00047c0 msgr2=0x7ff1c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1ddacb640 1 --2- 172.21.15.47:0/1164450495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1c00047c0 0x7ff1c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.142+0000 7ff1ddacb640 1 -- 172.21.15.47:0/1164450495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c0003520 con 0x7ff1c0005180 2024-03-23T09:45:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.143+0000 7ff1ddacb640 1 -- 172.21.15.47:0/1164450495 reap_dead start 2024-03-23T09:45:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.143+0000 7ff1ddacb640 1 --2- 172.21.15.47:0/1164450495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1c0005180 0x7ff1c000b980 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8001e50 tx=0x7ff1c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c293c8f1dfe03b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.143+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8003b70 con 0x7ff1c0005180 2024-03-23T09:45:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.143+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c8003d10 con 0x7ff1c0005180 2024-03-23T09:45:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.143+0000 7ff1d7fff640 1 -- 172.21.15.47:0/1164450495 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8005310 con 0x7ff1c0005180 2024-03-23T09:45:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.143+0000 7ff1df52e640 1 -- 172.21.15.47:0/1164450495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1c0005180 msgr2=0x7ff1c000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.143+0000 7ff1df52e640 1 --2- 172.21.15.47:0/1164450495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1c0005180 0x7ff1c000b980 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8001e50 tx=0x7ff1c00057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.144+0000 7ff1df52e640 1 -- 172.21.15.47:0/1164450495 shutdown_connections 2024-03-23T09:45:39.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.144+0000 7ff1df52e640 1 --2- 172.21.15.47:0/1164450495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1c0005180 0x7ff1c000b980 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.144+0000 7ff1df52e640 1 -- 172.21.15.47:0/1164450495 >> 172.21.15.47:0/1164450495 conn(0x7ff1d806b160 msgr2=0x7ff1d806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:39.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.144+0000 7ff1df52e640 1 -- 172.21.15.47:0/1164450495 shutdown_connections 2024-03-23T09:45:39.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.144+0000 7ff1df52e640 1 -- 172.21.15.47:0/1164450495 wait complete. 2024-03-23T09:45:39.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.145+0000 7ff1df52e640 1 Processor -- start 2024-03-23T09:45:39.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.145+0000 7ff1df52e640 1 -- start start 2024-03-23T09:45:39.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.145+0000 7ff1df52e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1c0005180 0x7ff1d8150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.145+0000 7ff1df52e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1d80c7790 0x7ff1d80c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.145+0000 7ff1df52e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d815efe0 0x7ff1d815f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1df52e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80c7300 con 0x7ff1d80c7790 2024-03-23T09:45:39.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1df52e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80c6410 con 0x7ff1d815efe0 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1df52e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80c66f0 con 0x7ff1c0005180 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1dcac9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1d80c7790 0x7ff1d80c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1dcac9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1d80c7790 0x7ff1d80c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49732/0 (socket says 172.21.15.47:49732) 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1dcac9640 1 -- 172.21.15.47:0/2454535981 learned_addr learned my addr 172.21.15.47:0/2454535981 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1c0005180 0x7ff1d8150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1ddacb640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d815efe0 0x7ff1d815f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1dcac9640 1 -- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1c0005180 msgr2=0x7ff1d8150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1dcac9640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1c0005180 0x7ff1d8150470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1dcac9640 1 -- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d815efe0 msgr2=0x7ff1d815f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1dcac9640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d815efe0 0x7ff1d815f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1dcac9640 1 -- 172.21.15.47:0/2454535981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d80c69d0 con 0x7ff1d80c7790 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1ddacb640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d815efe0 0x7ff1d815f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1dcac9640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1d80c7790 0x7ff1d80c7bf0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0072c0 tx=0x7ff1cc007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1d5ffb640 1 -- 172.21.15.47:0/2454535981 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0014c0 con 0x7ff1d80c7790 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1d5ffb640 1 -- 172.21.15.47:0/2454535981 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc00f960 con 0x7ff1d80c7790 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1df52e640 1 -- 172.21.15.47:0/2454535981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d80c6cb0 con 0x7ff1d80c7790 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1d5ffb640 1 -- 172.21.15.47:0/2454535981 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc00fd10 con 0x7ff1d80c7790 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.146+0000 7ff1df52e640 1 -- 172.21.15.47:0/2454535981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d81509b0 con 0x7ff1d80c7790 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.148+0000 7ff1d5ffb640 1 -- 172.21.15.47:0/2454535981 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff1cc01d460 con 0x7ff1d80c7790 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.148+0000 7ff1d5ffb640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff1c406ac30 0x7ff1c406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.149+0000 7ff1d5ffb640 1 -- 172.21.15.47:0/2454535981 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff1cc0896d0 con 0x7ff1d80c7790 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.149+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff1c406ac30 0x7ff1c406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.149+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff1c406ac30 0x7ff1c406d0f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0006fd0 tx=0x7ff1d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.149+0000 7ff1e217e740 1 Processor -- start 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.149+0000 7ff1e217e740 1 -- start start 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.150+0000 7ff1e217e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562003f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.150+0000 7ff1e217e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5562003f27d0 0x5562003f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.150+0000 7ff1e217e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003eec70 0x55620041ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.150+0000 7ff1e217e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55620035f9b0 con 0x55620043c750 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.150+0000 7ff1e217e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55620035d140 con 0x5562003eec70 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.150+0000 7ff1e217e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556200358a30 con 0x5562003f27d0 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.150+0000 7ff1dd2ca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562003f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.150+0000 7ff1dd2ca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562003f3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49740/0 (socket says 172.21.15.47:49740) 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.150+0000 7ff1dd2ca640 1 -- 172.21.15.47:0/3705539290 learned_addr learned my addr 172.21.15.47:0/3705539290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.150+0000 7ff1dcac9640 1 --2- 172.21.15.47:0/3705539290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5562003f27d0 0x5562003f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.150+0000 7ff1ddacb640 1 --2- 172.21.15.47:0/3705539290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003eec70 0x55620041ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1dd2ca640 1 -- 172.21.15.47:0/3705539290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5562003f27d0 msgr2=0x5562003f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/3705539290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5562003f27d0 0x5562003f42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1dd2ca640 1 -- 172.21.15.47:0/3705539290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003eec70 msgr2=0x55620041ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/3705539290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003eec70 0x55620041ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1dd2ca640 1 -- 172.21.15.47:0/3705539290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562002e52f0 con 0x55620043c750 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1dcac9640 1 --2- 172.21.15.47:0/3705539290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5562003f27d0 0x5562003f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/3705539290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562003f3d80 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000b8d0 tx=0x7ff1d0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cd3148d78a5c35 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1beffd640 1 -- 172.21.15.47:0/3705539290 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000be50 con 0x55620043c750 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1beffd640 1 -- 172.21.15.47:0/3705539290 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d00159d0 con 0x55620043c750 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1beffd640 1 -- 172.21.15.47:0/3705539290 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0015d40 con 0x55620043c750 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1e217e740 1 -- 172.21.15.47:0/3705539290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 msgr2=0x5562003f3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3705539290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562003f3d80 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000b8d0 tx=0x7ff1d0004610 comp rx=0 tx=0).stop 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1e217e740 1 -- 172.21.15.47:0/3705539290 shutdown_connections 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3705539290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003eec70 0x55620041ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3705539290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5562003f27d0 0x5562003f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3705539290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562003f3d80 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1e217e740 1 -- 172.21.15.47:0/3705539290 >> 172.21.15.47:0/3705539290 conn(0x5562003f0ee0 msgr2=0x556200439a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.151+0000 7ff1e217e740 1 -- 172.21.15.47:0/3705539290 shutdown_connections 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.152+0000 7ff1e217e740 1 -- 172.21.15.47:0/3705539290 wait complete. 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.152+0000 7ff1e217e740 1 -- start start 2024-03-23T09:45:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.152+0000 7ff1e217e740 1 Processor -- start 2024-03-23T09:45:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1e217e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003f27d0 0x556200422af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1e217e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562004418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1e217e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043ac90 0x55620043b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1e217e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556200317090 con 0x55620043c750 2024-03-23T09:45:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1e217e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556200316d30 con 0x5562003f27d0 2024-03-23T09:45:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1e217e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562002dc2a0 con 0x55620043ac90 2024-03-23T09:45:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1e217e740 10 client.0 _ll_register_callbacks cb 0x556200312050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:45:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1dcac9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562004418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1dcac9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562004418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49746/0 (socket says 172.21.15.47:49746) 2024-03-23T09:45:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1dcac9640 1 -- 172.21.15.47:0/3849847266 learned_addr learned my addr 172.21.15.47:0/3849847266 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/3849847266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003f27d0 0x556200422af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1ddacb640 1 --2- 172.21.15.47:0/3849847266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043ac90 0x55620043b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1dcac9640 1 -- 172.21.15.47:0/3849847266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043ac90 msgr2=0x55620043b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1dcac9640 1 --2- 172.21.15.47:0/3849847266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043ac90 0x55620043b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1dcac9640 1 -- 172.21.15.47:0/3849847266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003f27d0 msgr2=0x556200422af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1dcac9640 1 --2- 172.21.15.47:0/3849847266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003f27d0 0x556200422af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.153+0000 7ff1dcac9640 1 -- 172.21.15.47:0/3849847266 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d000e040 con 0x55620043c750 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1dcac9640 1 --2- 172.21.15.47:0/3849847266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562004418f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc007290 tx=0x7ff1cc00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9998c0aa90c159e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1b7fff640 1 -- 172.21.15.47:0/3849847266 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0605d0 con 0x55620043c750 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1b7fff640 1 -- 172.21.15.47:0/3849847266 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff1cc05c210 con 0x55620043c750 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1e217e740 1 -- 172.21.15.47:0/3849847266 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55620004c3c0 con 0x55620043c750 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1b7fff640 1 -- 172.21.15.47:0/3849847266 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1cc060860 con 0x55620043c750 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1e217e740 10 client.7053 fetch_fsmap learned FSMap version 38 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1e217e740 1 -- 172.21.15.47:0/3849847266 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x5562003570b0 con 0x55620043c750 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1b7fff640 1 -- 172.21.15.47:0/3849847266 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55620004c3c0 con 0x55620043c750 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1b7fff640 1 -- 172.21.15.47:0/3849847266 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 38) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff1cc06d040 con 0x55620043c750 2024-03-23T09:45:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1e217e740 10 client.7053 fetch_fsmap finished waiting for FSMap version 38 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1e217e740 10 client.7053 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1e217e740 20 client.7053 populate_metadata read hostname 'smithi047' 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3849847266 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x556200403960 0x556200403d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1e217e740 4 client.7053 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1e217e740 1 -- 172.21.15.47:0/3849847266 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55620005d6c0 con 0x556200403960 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.154+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/3849847266 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x556200403960 0x556200403d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/3849847266 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x556200403960 0x556200403d40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c8281e60633a3f9 server_cookie=fac422a45a700469 in_seq=0 out_seq=0 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1b7fff640 10 client.7053 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1b7fff640 1 -- 172.21.15.47:0/3849847266 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55620005d6c0 con 0x556200403960 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1b7fff640 10 client.7053 handle_command_reply: tid=0 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1e217e740 1 client.7053 shutdown 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1e217e740 1 -- 172.21.15.47:0/3849847266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 msgr2=0x5562004418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3849847266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562004418f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc007290 tx=0x7ff1cc00ff30 comp rx=0 tx=0).stop 2024-03-23T09:45:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1e217e740 1 -- 172.21.15.47:0/3849847266 shutdown_connections 2024-03-23T09:45:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3849847266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043ac90 0x55620043b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3849847266 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x556200403960 0x556200403d40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3849847266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562004418f0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3849847266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003f27d0 0x556200422af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1b7fff640 0 client.7053 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:45:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.155+0000 7ff1e217e740 1 -- 172.21.15.47:0/3849847266 >> 172.21.15.47:0/3849847266 conn(0x5562003eec70 msgr2=0x5562003f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.156+0000 7ff1e217e740 1 -- 172.21.15.47:0/3849847266 shutdown_connections 2024-03-23T09:45:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.156+0000 7ff1e217e740 1 -- 172.21.15.47:0/3849847266 wait complete. 2024-03-23T09:45:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.156+0000 7ff1e217e740 20 client.7053 trim_cache size 0 max 16384 2024-03-23T09:45:39.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.171+0000 7ff1e217e740 1 Processor -- start 2024-03-23T09:45:39.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.171+0000 7ff1e217e740 1 -- start start 2024-03-23T09:45:39.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.171+0000 7ff1e217e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003f27d0 0x55620041b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.171+0000 7ff1e217e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043ac90 0x556200415b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.171+0000 7ff1e217e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562003f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.171+0000 7ff1e217e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55620041c940 con 0x55620043c750 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.171+0000 7ff1e217e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562004536b0 con 0x5562003f27d0 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.171+0000 7ff1e217e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562003294e0 con 0x55620043ac90 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.171+0000 7ff1ddacb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562003f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.171+0000 7ff1ddacb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562003f2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49754/0 (socket says 172.21.15.47:49754) 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.171+0000 7ff1ddacb640 1 -- 172.21.15.47:0/3351753453 learned_addr learned my addr 172.21.15.47:0/3351753453 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.172+0000 7ff1dcac9640 1 --2- 172.21.15.47:0/3351753453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043ac90 0x556200415b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.172+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/3351753453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003f27d0 0x55620041b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.172+0000 7ff1ddacb640 1 -- 172.21.15.47:0/3351753453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043ac90 msgr2=0x556200415b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.172+0000 7ff1ddacb640 1 --2- 172.21.15.47:0/3351753453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043ac90 0x556200415b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.172+0000 7ff1ddacb640 1 -- 172.21.15.47:0/3351753453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003f27d0 msgr2=0x55620041b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.172+0000 7ff1ddacb640 1 --2- 172.21.15.47:0/3351753453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003f27d0 0x55620041b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.172+0000 7ff1ddacb640 1 -- 172.21.15.47:0/3351753453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562002f9430 con 0x55620043c750 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.172+0000 7ff1ddacb640 1 --2- 172.21.15.47:0/3351753453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562003f2ce0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff1c802c060 tx=0x7ff1c8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27551307e77fd5a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.172+0000 7ff1bf7fe640 1 -- 172.21.15.47:0/3351753453 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c803f070 con 0x55620043c750 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.172+0000 7ff1bf7fe640 1 -- 172.21.15.47:0/3351753453 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c8040040 con 0x55620043c750 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.172+0000 7ff1bf7fe640 1 -- 172.21.15.47:0/3351753453 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c80052d0 con 0x55620043c750 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.173+0000 7ff1e217e740 1 -- 172.21.15.47:0/3351753453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 msgr2=0x5562003f2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.173+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3351753453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562003f2ce0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff1c802c060 tx=0x7ff1c8030000 comp rx=0 tx=0).stop 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.173+0000 7ff1e217e740 1 -- 172.21.15.47:0/3351753453 shutdown_connections 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.173+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3351753453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55620043c750 0x5562003f2ce0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.173+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3351753453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043ac90 0x556200415b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.173+0000 7ff1e217e740 1 --2- 172.21.15.47:0/3351753453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562003f27d0 0x55620041b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.173+0000 7ff1e217e740 1 -- 172.21.15.47:0/3351753453 >> 172.21.15.47:0/3351753453 conn(0x5562003eec70 msgr2=0x5562004418f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.173+0000 7ff1e217e740 1 -- 172.21.15.47:0/3351753453 shutdown_connections 2024-03-23T09:45:39.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.173+0000 7ff1e217e740 1 -- 172.21.15.47:0/3351753453 wait complete. 2024-03-23T09:45:39.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.173+0000 7ff1e217e740 1 -- start start 2024-03-23T09:45:39.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.173+0000 7ff1e217e740 1 Processor -- start 2024-03-23T09:45:39.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.174+0000 7ff1e217e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562003f27d0 0x55620049ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.174+0000 7ff1e217e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55620043ac90 0x55620041a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.174+0000 7ff1e217e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043c750 0x55620041a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.174+0000 7ff1e217e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556200440260 con 0x5562003f27d0 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.174+0000 7ff1e217e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55620049f3a0 con 0x55620043ac90 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.174+0000 7ff1e217e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562004006a0 con 0x55620043c750 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.174+0000 7ff1e217e740 10 client.0 _ll_register_callbacks cb 0x7ff1d0004280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.174+0000 7ff1dd2ca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562003f27d0 0x55620049ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.174+0000 7ff1dd2ca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562003f27d0 0x55620049ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49762/0 (socket says 172.21.15.47:49762) 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.174+0000 7ff1dd2ca640 1 -- 172.21.15.47:0/392490531 learned_addr learned my addr 172.21.15.47:0/392490531 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.174+0000 7ff1dcac9640 1 --2- 172.21.15.47:0/392490531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55620043ac90 0x55620041a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.174+0000 7ff1ddacb640 1 --2- 172.21.15.47:0/392490531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043c750 0x55620041a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1dd2ca640 1 -- 172.21.15.47:0/392490531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043c750 msgr2=0x55620041a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/392490531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043c750 0x55620041a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1dd2ca640 1 -- 172.21.15.47:0/392490531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55620043ac90 msgr2=0x55620041a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/392490531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55620043ac90 0x55620041a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1dd2ca640 1 -- 172.21.15.47:0/392490531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c0003520 con 0x5562003f27d0 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1dd2ca640 1 --2- 172.21.15.47:0/392490531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562003f27d0 0x55620049ee60 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0008010 tx=0x7ff1d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20092c3174447d52 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:39.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1b7fff640 1 -- 172.21.15.47:0/392490531 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00379f0 con 0x5562003f27d0 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1b7fff640 1 -- 172.21.15.47:0/392490531 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff1d0030040 con 0x5562003f27d0 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1e217e740 1 -- 172.21.15.47:0/392490531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1d00159d0 con 0x5562003f27d0 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1b7fff640 1 -- 172.21.15.47:0/392490531 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d0037440 con 0x5562003f27d0 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1e217e740 10 client.7059 fetch_fsmap learned FSMap version 38 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1e217e740 1 -- 172.21.15.47:0/392490531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x55620031eb40 con 0x5562003f27d0 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1b7fff640 1 -- 172.21.15.47:0/392490531 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d00159d0 con 0x5562003f27d0 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1b7fff640 1 -- 172.21.15.47:0/392490531 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 38) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff1d0037b50 con 0x5562003f27d0 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1e217e740 10 client.7059 fetch_fsmap finished waiting for FSMap version 38 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1e217e740 10 client.7059 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1e217e740 20 client.7059 populate_metadata read hostname 'smithi047' 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1e217e740 1 --2- 172.21.15.47:0/392490531 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x556200448c20 0x55620043cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1e217e740 4 client.7059 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.175+0000 7ff1e217e740 1 -- 172.21.15.47:0/392490531 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5562000b0d10 con 0x556200448c20 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1dcac9640 1 --2- 172.21.15.47:0/392490531 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x556200448c20 0x55620043cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1b7fff640 10 client.7059 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:45:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1dcac9640 1 --2- 172.21.15.47:0/392490531 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x556200448c20 0x55620043cb30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47edc9f3f9158ce9 server_cookie=5c84b954a9357712 in_seq=0 out_seq=0 2024-03-23T09:45:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1b7fff640 1 -- 172.21.15.47:0/392490531 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5562000b0d10 con 0x556200448c20 2024-03-23T09:45:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1b7fff640 10 client.7059 handle_command_reply: tid=0 2024-03-23T09:45:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1e217e740 1 client.7059 shutdown 2024-03-23T09:45:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1e217e740 1 -- 172.21.15.47:0/392490531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562003f27d0 msgr2=0x55620049ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1e217e740 1 --2- 172.21.15.47:0/392490531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562003f27d0 0x55620049ee60 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0008010 tx=0x7ff1d00046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1e217e740 1 -- 172.21.15.47:0/392490531 shutdown_connections 2024-03-23T09:45:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1e217e740 1 --2- 172.21.15.47:0/392490531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55620043c750 0x55620041a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1e217e740 1 --2- 172.21.15.47:0/392490531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55620043ac90 0x55620041a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1e217e740 1 --2- 172.21.15.47:0/392490531 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x556200448c20 0x55620043cb30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1e217e740 1 --2- 172.21.15.47:0/392490531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562003f27d0 0x55620049ee60 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1b7fff640 0 client.7059 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:45:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1e217e740 1 -- 172.21.15.47:0/392490531 >> 172.21.15.47:0/392490531 conn(0x5562003eec70 msgr2=0x556200415b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1e217e740 1 -- 172.21.15.47:0/392490531 shutdown_connections 2024-03-23T09:45:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1e217e740 1 -- 172.21.15.47:0/392490531 wait complete. 2024-03-23T09:45:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.176+0000 7ff1e217e740 20 client.7059 trim_cache size 0 max 16384 2024-03-23T09:45:39.182 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:45:39.182 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:45:39.182 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:45:39.182 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 7005, 2024-03-23T09:45:39.182 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:45:39.182 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:45:39.182 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:45:39.182 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 4.4228160069999998, 2024-03-23T09:45:39.183 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 38, 2024-03-23T09:45:39.183 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 71, 2024-03-23T09:45:39.183 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 71, 2024-03-23T09:45:39.183 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 5.3976779170000002 2024-03-23T09:45:39.183 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:45:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.177+0000 7ff1df52e640 1 -- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff1c406ac30 msgr2=0x7ff1c406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.177+0000 7ff1df52e640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff1c406ac30 0x7ff1c406d0f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0006fd0 tx=0x7ff1d0008040 comp rx=0 tx=0).stop 2024-03-23T09:45:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.177+0000 7ff1df52e640 1 -- 172.21.15.47:0/2454535981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1d80c7790 msgr2=0x7ff1d80c7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.177+0000 7ff1df52e640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1d80c7790 0x7ff1d80c7bf0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0072c0 tx=0x7ff1cc007780 comp rx=0 tx=0).stop 2024-03-23T09:45:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.177+0000 7ff1df52e640 1 -- 172.21.15.47:0/2454535981 shutdown_connections 2024-03-23T09:45:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.177+0000 7ff1df52e640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff1c406ac30 0x7ff1c406d0f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.177+0000 7ff1df52e640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d815efe0 0x7ff1d815f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.177+0000 7ff1df52e640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1d80c7790 0x7ff1d80c7bf0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.177+0000 7ff1df52e640 1 --2- 172.21.15.47:0/2454535981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1c0005180 0x7ff1d8150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.177+0000 7ff1df52e640 1 -- 172.21.15.47:0/2454535981 >> 172.21.15.47:0/2454535981 conn(0x7ff1d806b160 msgr2=0x7ff1d80c9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.177+0000 7ff1df52e640 1 -- 172.21.15.47:0/2454535981 shutdown_connections 2024-03-23T09:45:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.177+0000 7ff1df52e640 1 -- 172.21.15.47:0/2454535981 wait complete. 2024-03-23T09:45:39.187 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:45:39.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.288+0000 7f45353b1640 1 Processor -- start 2024-03-23T09:45:39.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.288+0000 7f45353b1640 1 -- start start 2024-03-23T09:45:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.289+0000 7f45353b1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453015d0a0 0x7f45300d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.289+0000 7f45353b1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f453006b210 0x7f453006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.289+0000 7f45353b1640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45300ce090 con 0x7f453015ccc0 2024-03-23T09:45:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.289+0000 7f45353b1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453006c0e0 con 0x7f453006b210 2024-03-23T09:45:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.289+0000 7f45353b1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453006bc00 con 0x7f453015d0a0 2024-03-23T09:45:39.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.289+0000 7f452f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f453006b210 0x7f453006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.289+0000 7f452f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f453006b210 0x7f453006b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49766/0 (socket says 172.21.15.47:49766) 2024-03-23T09:45:39.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.289+0000 7f452f7fe640 1 -- 172.21.15.47:0/3380754005 learned_addr learned my addr 172.21.15.47:0/3380754005 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:39.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.289+0000 7f452e7fc640 1 --2- 172.21.15.47:0/3380754005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453015d0a0 0x7f45300d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.289+0000 7f452f7fe640 1 -- 172.21.15.47:0/3380754005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453015d0a0 msgr2=0x7f45300d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:39.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.289+0000 7f452f7fe640 1 --2- 172.21.15.47:0/3380754005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453015d0a0 0x7f45300d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.289+0000 7f452f7fe640 1 -- 172.21.15.47:0/3380754005 >> v1:172.21.15.175:6789/0 conn(0x7f453015ccc0 legacy=0x7f45300cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:39.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.289+0000 7f452f7fe640 1 -- 172.21.15.47:0/3380754005 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45300d0f00 con 0x7f453006b210 2024-03-23T09:45:39.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.290+0000 7f452f7fe640 1 --2- 172.21.15.47:0/3380754005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453006b210 0x7f453006b6c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4520009e50 tx=0x7f452002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8c165659ef77b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:39.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.290+0000 7f452dffb640 1 -- 172.21.15.47:0/3380754005 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452002cb00 con 0x7f453006b210 2024-03-23T09:45:39.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.290+0000 7f452dffb640 1 -- 172.21.15.47:0/3380754005 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452002cc60 con 0x7f453006b210 2024-03-23T09:45:39.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.290+0000 7f452dffb640 1 -- 172.21.15.47:0/3380754005 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45200343f0 con 0x7f453006b210 2024-03-23T09:45:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.291+0000 7f45353b1640 1 -- 172.21.15.47:0/3380754005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453006b210 msgr2=0x7f453006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.291+0000 7f45353b1640 1 --2- 172.21.15.47:0/3380754005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453006b210 0x7f453006b6c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4520009e50 tx=0x7f452002bf60 comp rx=0 tx=0).stop 2024-03-23T09:45:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.291+0000 7f45353b1640 1 -- 172.21.15.47:0/3380754005 shutdown_connections 2024-03-23T09:45:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.291+0000 7f45353b1640 1 --2- 172.21.15.47:0/3380754005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453006b210 0x7f453006b6c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.291+0000 7f45353b1640 1 --2- 172.21.15.47:0/3380754005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453015d0a0 0x7f45300d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.291+0000 7f45353b1640 1 -- 172.21.15.47:0/3380754005 >> 172.21.15.47:0/3380754005 conn(0x7f45300cf430 msgr2=0x7f45300cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.291+0000 7f45353b1640 1 -- 172.21.15.47:0/3380754005 shutdown_connections 2024-03-23T09:45:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.291+0000 7f45353b1640 1 -- 172.21.15.47:0/3380754005 wait complete. 2024-03-23T09:45:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.292+0000 7f45353b1640 1 Processor -- start 2024-03-23T09:45:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.292+0000 7f45353b1640 1 -- start start 2024-03-23T09:45:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f45353b1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f453006b210 0x7f45300aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f45353b1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453015ccc0 0x7f453015dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f45353b1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453015d0a0 0x7f45300a9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f45353b1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45301f1ff0 con 0x7f453015ccc0 2024-03-23T09:45:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f45353b1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453015f980 con 0x7f453006b210 2024-03-23T09:45:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f45353b1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4530156e00 con 0x7f453015d0a0 2024-03-23T09:45:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f452e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453015ccc0 0x7f453015dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f452e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453015ccc0 0x7f453015dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49774/0 (socket says 172.21.15.47:49774) 2024-03-23T09:45:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f452e7fc640 1 -- 172.21.15.47:0/4077030843 learned_addr learned my addr 172.21.15.47:0/4077030843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f452e7fc640 1 -- 172.21.15.47:0/4077030843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453015d0a0 msgr2=0x7f45300a9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f452e7fc640 1 --2- 172.21.15.47:0/4077030843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453015d0a0 0x7f45300a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f452e7fc640 1 -- 172.21.15.47:0/4077030843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f453006b210 msgr2=0x7f45300aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f452e7fc640 1 --2- 172.21.15.47:0/4077030843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f453006b210 0x7f45300aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f452e7fc640 1 -- 172.21.15.47:0/4077030843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4520009b30 con 0x7f453015ccc0 2024-03-23T09:45:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f452e7fc640 1 --2- 172.21.15.47:0/4077030843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453015ccc0 0x7f453015dea0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f451800d5d0 tx=0x7f451800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f45277fe640 1 -- 172.21.15.47:0/4077030843 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518004240 con 0x7f453015ccc0 2024-03-23T09:45:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f45277fe640 1 -- 172.21.15.47:0/4077030843 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45180043a0 con 0x7f453015ccc0 2024-03-23T09:45:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f45353b1640 1 -- 172.21.15.47:0/4077030843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453015e740 con 0x7f453015ccc0 2024-03-23T09:45:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f45277fe640 1 -- 172.21.15.47:0/4077030843 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518003830 con 0x7f453015ccc0 2024-03-23T09:45:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.293+0000 7f45353b1640 1 -- 172.21.15.47:0/4077030843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453015ed00 con 0x7f453015ccc0 2024-03-23T09:45:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.294+0000 7f45353b1640 1 -- 172.21.15.47:0/4077030843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4530072360 con 0x7f453015ccc0 2024-03-23T09:45:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.295+0000 7f45277fe640 1 -- 172.21.15.47:0/4077030843 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4518003990 con 0x7f453015ccc0 2024-03-23T09:45:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.295+0000 7f45277fe640 1 --2- 172.21.15.47:0/4077030843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f451006ac00 0x7f451006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.295+0000 7f45277fe640 1 -- 172.21.15.47:0/4077030843 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4518089500 con 0x7f453015ccc0 2024-03-23T09:45:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.296+0000 7f452effd640 1 --2- 172.21.15.47:0/4077030843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f451006ac00 0x7f451006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.296+0000 7f452effd640 1 --2- 172.21.15.47:0/4077030843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f451006ac00 0x7f451006d0c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4528003cc0 tx=0x7f45280079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:39.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.300+0000 7f45277fe640 1 -- 172.21.15.47:0/4077030843 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4518054710 con 0x7f453015ccc0 2024-03-23T09:45:39.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.515+0000 7f45353b1640 1 -- 172.21.15.47:0/4077030843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4530066be0 con 0x7f453015ccc0 2024-03-23T09:45:39.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.515+0000 7f45277fe640 1 -- 172.21.15.47:0/4077030843 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4518057d30 con 0x7f453015ccc0 2024-03-23T09:45:39.516 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:45:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.517+0000 7f45353b1640 1 -- 172.21.15.47:0/4077030843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f451006ac00 msgr2=0x7f451006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.517+0000 7f45353b1640 1 --2- 172.21.15.47:0/4077030843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f451006ac00 0x7f451006d0c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4528003cc0 tx=0x7f45280079b0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.517+0000 7f45353b1640 1 -- 172.21.15.47:0/4077030843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453015ccc0 msgr2=0x7f453015dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.517+0000 7f45353b1640 1 --2- 172.21.15.47:0/4077030843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453015ccc0 0x7f453015dea0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f451800d5d0 tx=0x7f451800da90 comp rx=0 tx=0).stop 2024-03-23T09:45:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.518+0000 7f45353b1640 1 -- 172.21.15.47:0/4077030843 shutdown_connections 2024-03-23T09:45:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.518+0000 7f45353b1640 1 --2- 172.21.15.47:0/4077030843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f451006ac00 0x7f451006d0c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.518+0000 7f45353b1640 1 --2- 172.21.15.47:0/4077030843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453015d0a0 0x7f45300a9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.518+0000 7f45353b1640 1 --2- 172.21.15.47:0/4077030843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453015ccc0 0x7f453015dea0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.518+0000 7f45353b1640 1 --2- 172.21.15.47:0/4077030843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f453006b210 0x7f45300aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.518+0000 7f45353b1640 1 -- 172.21.15.47:0/4077030843 >> 172.21.15.47:0/4077030843 conn(0x7f45300cf430 msgr2=0x7f453015a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.518+0000 7f45353b1640 1 -- 172.21.15.47:0/4077030843 shutdown_connections 2024-03-23T09:45:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:39.518+0000 7f45353b1640 1 -- 172.21.15.47:0/4077030843 wait complete. 2024-03-23T09:45:39.528 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:45:39.528 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:45:39.529 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:45:39.529 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:45:39.529 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:39.529 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:45:39.529 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:45:39.529 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:45:39.529 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:45:39.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:39.529 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:45:39.544 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:45:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:45:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:45:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:45:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:39.544 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:45:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:45:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:45:39.544 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42399sec preferred_lft 42399sec 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:45:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:45:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:45:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:45:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:39.546 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:45:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:45:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:45:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:39.546 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:45:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:45:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:45:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:45:39.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:39.547 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:45:39.610 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:45:39.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:39.611 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:45:39.667 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:45:39.667 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:45:39.667 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:45:39.667 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:45:39.667 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:45:39.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:39.667 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:45:39.667 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:45:39.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:45:39.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:45:39.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:45:39.668 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:45:39.754 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:45:39.754+0000 7f4fbb675640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:45:39.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:39.761 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:45:39.775 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:45:39.775 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:45:39.775 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:45:39.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:39.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:45:39.843 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:45:39.843 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:45:39.843 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:39.897 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:45:39.897 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:45:39.897 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:39.951 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:45:40.018 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:45:40.018 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:45:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:45:40.085 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:45:40.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:40.085 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:45:40.140 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:45:40.140 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:45:40.183 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:45:40.209 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:45:40.209 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:45:40.268 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:45:40.268+0000 7f6a8493f480 -1 init, newargv = 0x560acec9b310 newargc=15 2024-03-23T09:45:40.268 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[62907]: starting ceph client 2024-03-23T09:45:40.272 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[62907]: starting fuse 2024-03-23T09:45:40.277 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:45:40.278 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:45:40.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:40.279 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:45:40.333 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:45:40.333 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:45:40.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:45:40.333 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:45:40.333 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:45:40.333 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:45:40.334 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:45:40.335 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:45:40.335 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:45:40.335 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:45:40.335 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:45:40.335 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:45:40.335 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:45:40.335 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:45:40.335 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:45:40.335 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:45:40.449 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.62907.asok 2024-03-23T09:45:40.636 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.62907.asok 2024-03-23T09:45:40.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:40.636 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.62907.asok status 2024-03-23T09:45:40.773 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:45:40.773 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:45:40.773 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:45:40.773 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:45:40.774 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:45:40.774 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:45:40.774 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:45:40.774 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "62907", 2024-03-23T09:45:40.774 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:45:40.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:40.775 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:45:40.775 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:45:40.775 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 7071, 2024-03-23T09:45:40.775 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:45:40.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:45:40.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:45:40.775 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7071 2024-03-23T09:45:40.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:40.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:45:40.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:45:40.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:45:40.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1535354486 2024-03-23T09:45:40.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:40.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:40.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:45:40.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:45:40.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:45:40.777 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1535354486 2024-03-23T09:45:40.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:45:40.777 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.7071 192.168.0.1:0/1535354486", 2024-03-23T09:45:40.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/1535354486", 2024-03-23T09:45:40.777 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:45:40.777 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 38, 2024-03-23T09:45:40.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 71, 2024-03-23T09:45:40.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:45:40.777 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:45:40.777 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:45:40.777 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:45:40.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:40.781 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:45:40.781 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:40.796 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:45:40.796 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:40.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:40.796 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:40.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:40.863 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-03-23T09:45:40.922 INFO:teuthology.orchestra.run.smithi047.stdout:file_undamaged 2024-03-23T09:45:40.922 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:45:40.922 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> import json 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> import sys 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> try: 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> except OSError as e: 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> sys.exit(e.errno) 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> print(json.dumps( 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> dict([(a, getattr(s, a)) for a in attrs]), 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> indent=2)) 2024-03-23T09:45:40.923 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:45:41.205 INFO:teuthology.orchestra.run.smithi047.stderr:daemon-helper: command failed with exit status 2 2024-03-23T09:45:41.210 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-23T09:45:41.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:41.211 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-03-23T09:45:41.268 INFO:teuthology.orchestra.run.smithi047.stdout:2 2024-03-23T09:45:41.269 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:45:41.269 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:45:41.324 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:45:41.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.465+0000 7efee242c640 1 Processor -- start 2024-03-23T09:45:41.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.465+0000 7efee242c640 1 -- start start 2024-03-23T09:45:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.466+0000 7efee242c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efedc0cad40 0x7efedc0b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.466+0000 7efee242c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efedc0b1400 con 0x7efedc15cbe0 2024-03-23T09:45:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.466+0000 7efee242c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efedc0b16e0 con 0x7efedc0cd680 2024-03-23T09:45:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.466+0000 7efee242c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efedc0cedb0 con 0x7efedc0cad40 2024-03-23T09:45:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.466+0000 7efedbfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efedc15cbe0 0x7efedc0ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58710/0 (socket says 172.21.15.47:58710) 2024-03-23T09:45:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.466+0000 7efedbfff640 1 -- 172.21.15.47:0/3624235591 learned_addr learned my addr 172.21.15.47:0/3624235591 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efee09c9640 1 --2- 172.21.15.47:0/3624235591 >> v2:172.21.15.175:3300/0 conn(0x7efedc0cad40 0x7efedc0b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804150864 0 0) 0x7efedc0b1400 con 0x7efedc15cbe0 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efec4003520 con 0x7efedc15cbe0 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 563205648 0 0) 0x7efec4003520 con 0x7efedc15cbe0 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 >> v2:172.21.15.175:3300/0 conn(0x7efedc0cad40 msgr2=0x7efedc0b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 --2- 172.21.15.47:0/3624235591 >> v2:172.21.15.175:3300/0 conn(0x7efedc0cad40 0x7efedc0b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 >> v1:172.21.15.175:6789/0 conn(0x7efedc0cd680 legacy=0x7efedc0b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc0cf090 con 0x7efedc15cbe0 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7efed0001670 con 0x7efedc15cbe0 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 >> v1:172.21.15.47:6789/0 conn(0x7efedc15cbe0 legacy=0x7efedc0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 --2- 172.21.15.47:0/3624235591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4003970 0x7efec4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 --2- 172.21.15.47:0/3624235591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec40047c0 0x7efec4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 --2- 172.21.15.47:0/3624235591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efec4005180 0x7efec400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc0cedb0 con 0x7efec4003970 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec400c120 con 0x7efec40047c0 2024-03-23T09:45:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec400c2e0 con 0x7efec4005180 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedbfff640 1 --2- 172.21.15.47:0/3624235591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4003970 0x7efec4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedb7fe640 1 --2- 172.21.15.47:0/3624235591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec40047c0 0x7efec4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efee09c9640 1 --2- 172.21.15.47:0/3624235591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efec4005180 0x7efec400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedbfff640 1 -- 172.21.15.47:0/3624235591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efec4005180 msgr2=0x7efec400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedbfff640 1 --2- 172.21.15.47:0/3624235591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efec4005180 0x7efec400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedbfff640 1 -- 172.21.15.47:0/3624235591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec40047c0 msgr2=0x7efec4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedbfff640 1 --2- 172.21.15.47:0/3624235591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec40047c0 0x7efec4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efedbfff640 1 -- 172.21.15.47:0/3624235591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec4003520 con 0x7efec4003970 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efee09c9640 1 --2- 172.21.15.47:0/3624235591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efec4005180 0x7efec400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.467+0000 7efee09c9640 1 -- 172.21.15.47:0/3624235591 reap_dead start 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.468+0000 7efedbfff640 1 --2- 172.21.15.47:0/3624235591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4003970 0x7efec4003d90 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7efed000c240 tx=0x7efec40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd814b7d11042c6b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.468+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0005ad0 con 0x7efec4003970 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.468+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed0005c30 con 0x7efec4003970 2024-03-23T09:45:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.468+0000 7efedaffd640 1 -- 172.21.15.47:0/3624235591 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed002c850 con 0x7efec4003970 2024-03-23T09:45:41.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.469+0000 7efee242c640 1 -- 172.21.15.47:0/3624235591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4003970 msgr2=0x7efec4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.469+0000 7efee242c640 1 --2- 172.21.15.47:0/3624235591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4003970 0x7efec4003d90 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7efed000c240 tx=0x7efec40057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.469+0000 7efee242c640 1 -- 172.21.15.47:0/3624235591 shutdown_connections 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.469+0000 7efee242c640 1 --2- 172.21.15.47:0/3624235591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4003970 0x7efec4003d90 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.469+0000 7efee242c640 1 -- 172.21.15.47:0/3624235591 >> 172.21.15.47:0/3624235591 conn(0x7efedc0d06f0 msgr2=0x7efedc0d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.469+0000 7efee242c640 1 -- 172.21.15.47:0/3624235591 shutdown_connections 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.469+0000 7efee242c640 1 -- 172.21.15.47:0/3624235591 wait complete. 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.470+0000 7efee242c640 1 Processor -- start 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.470+0000 7efee242c640 1 -- start start 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee242c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efec4003970 0x7efedc0b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee242c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc159030 0x7efedc0b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee242c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efedc159550 0x7efedc1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee242c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc159f20 con 0x7efedc159550 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee242c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc15d370 con 0x7efedc159030 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee242c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc15d650 con 0x7efec4003970 2024-03-23T09:45:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee09c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efedc159550 0x7efedc1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee09c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efedc159550 0x7efedc1599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49794/0 (socket says 172.21.15.47:49794) 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee09c9640 1 -- 172.21.15.47:0/1916003052 learned_addr learned my addr 172.21.15.47:0/1916003052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efedbfff640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efec4003970 0x7efedc0b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee09c9640 1 -- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efec4003970 msgr2=0x7efedc0b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee09c9640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efec4003970 0x7efedc0b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee09c9640 1 -- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc159030 msgr2=0x7efedc0b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efedb7fe640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc159030 0x7efedc0b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee09c9640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc159030 0x7efedc0b5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee09c9640 1 -- 172.21.15.47:0/1916003052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc15d930 con 0x7efedc159550 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.471+0000 7efee09c9640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efedc159550 0x7efedc1599e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7efedc0b32b0 tx=0x7efed4010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.472+0000 7efed8ff9640 1 -- 172.21.15.47:0/1916003052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed401b070 con 0x7efedc159550 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.472+0000 7efed8ff9640 1 -- 172.21.15.47:0/1916003052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed400f040 con 0x7efedc159550 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.472+0000 7efee242c640 1 -- 172.21.15.47:0/1916003052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efedc15dc10 con 0x7efedc159550 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.472+0000 7efed8ff9640 1 -- 172.21.15.47:0/1916003052 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400e050 con 0x7efedc159550 2024-03-23T09:45:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.472+0000 7efee242c640 1 -- 172.21.15.47:0/1916003052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc0b6450 con 0x7efedc159550 2024-03-23T09:45:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.473+0000 7efee242c640 1 -- 172.21.15.47:0/1916003052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efedc15e130 con 0x7efedc159550 2024-03-23T09:45:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.473+0000 7efed8ff9640 1 -- 172.21.15.47:0/1916003052 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efed4003bd0 con 0x7efedc159550 2024-03-23T09:45:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.473+0000 7efed8ff9640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efec806ac30 0x7efec806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.473+0000 7efed8ff9640 1 -- 172.21.15.47:0/1916003052 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efed4089790 con 0x7efedc159550 2024-03-23T09:45:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.474+0000 7efedbfff640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efec806ac30 0x7efec806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.474+0000 7efedbfff640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efec806ac30 0x7efec806d0f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efed0034000 tx=0x7efed0009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:41.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.478+0000 7efed8ff9640 1 -- 172.21.15.47:0/1916003052 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efedc15e130 con 0x7efedc159550 2024-03-23T09:45:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.659+0000 7efee242c640 1 -- 172.21.15.47:0/1916003052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efedc15e130 con 0x7efedc159550 2024-03-23T09:45:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.660+0000 7efed8ff9640 1 -- 172.21.15.47:0/1916003052 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16227 (secure 0 0 0) 0x7efedc15e130 con 0x7efedc159550 2024-03-23T09:45:41.660 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:41.660 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":71,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:45:33.740401+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:45:16.609941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-23T09:45:17.261503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/372771687":"2024-03-24T09:45:33.740376+0000","172.21.15.47:6833/372771687":"2024-03-24T09:45:33.740376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:45:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.662+0000 7efee242c640 1 -- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efec806ac30 msgr2=0x7efec806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.662+0000 7efee242c640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efec806ac30 0x7efec806d0f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efed0034000 tx=0x7efed0009030 comp rx=0 tx=0).stop 2024-03-23T09:45:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.662+0000 7efee242c640 1 -- 172.21.15.47:0/1916003052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efedc159550 msgr2=0x7efedc1599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.662+0000 7efee242c640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efedc159550 0x7efedc1599e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7efedc0b32b0 tx=0x7efed4010590 comp rx=0 tx=0).stop 2024-03-23T09:45:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.662+0000 7efee242c640 1 -- 172.21.15.47:0/1916003052 shutdown_connections 2024-03-23T09:45:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.662+0000 7efee242c640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efec806ac30 0x7efec806d0f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.662+0000 7efee242c640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efedc159550 0x7efedc1599e0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.662+0000 7efee242c640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efedc159030 0x7efedc0b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.662+0000 7efee242c640 1 --2- 172.21.15.47:0/1916003052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efec4003970 0x7efedc0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.662+0000 7efee242c640 1 -- 172.21.15.47:0/1916003052 >> 172.21.15.47:0/1916003052 conn(0x7efedc0d06f0 msgr2=0x7efedc15b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.662+0000 7efee242c640 1 -- 172.21.15.47:0/1916003052 shutdown_connections 2024-03-23T09:45:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.662+0000 7efee242c640 1 -- 172.21.15.47:0/1916003052 wait complete. 2024-03-23T09:45:41.673 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:45:16.609941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:45:17.261503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:45:41.673 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:45:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.777+0000 7faecafb1640 1 Processor -- start 2024-03-23T09:45:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.777+0000 7faecafb1640 1 -- start start 2024-03-23T09:45:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.777+0000 7faecafb1640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faec40cde10 0x7faec4160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.777+0000 7faecafb1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec4161630 con 0x7faec40b8350 2024-03-23T09:45:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.777+0000 7faecafb1640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec40d1030 con 0x7faec415c990 2024-03-23T09:45:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.777+0000 7faecafb1640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec40cfe10 con 0x7faec40cde10 2024-03-23T09:45:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.778+0000 7faec954e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7faec40b8350 0x7faec40d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58722/0 (socket says 172.21.15.47:58722) 2024-03-23T09:45:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.778+0000 7faec954e640 1 -- 172.21.15.47:0/1461337864 learned_addr learned my addr 172.21.15.47:0/1461337864 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.778+0000 7faec3fff640 1 --2- 172.21.15.47:0/1461337864 >> v2:172.21.15.175:3300/0 conn(0x7faec40cde10 0x7faec4160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.778+0000 7faec37fe640 1 -- 172.21.15.47:0/1461337864 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020109492 0 0) 0x7faec4161630 con 0x7faec40b8350 2024-03-23T09:45:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.778+0000 7faec37fe640 1 -- 172.21.15.47:0/1461337864 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeb4003520 con 0x7faec40b8350 2024-03-23T09:45:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.778+0000 7faec37fe640 1 -- 172.21.15.47:0/1461337864 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1638271301 0 0) 0x7faeb4003520 con 0x7faec40b8350 2024-03-23T09:45:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.778+0000 7faec3fff640 1 -- 172.21.15.47:0/1461337864 >> v1:172.21.15.175:6789/0 conn(0x7faec415c990 legacy=0x7faec40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.778+0000 7faec3fff640 1 -- 172.21.15.47:0/1461337864 >> v1:172.21.15.47:6789/0 conn(0x7faec40b8350 legacy=0x7faec40d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.778+0000 7faec3fff640 1 -- 172.21.15.47:0/1461337864 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec40b5b60 con 0x7faec40cde10 2024-03-23T09:45:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.779+0000 7faec3fff640 1 --2- 172.21.15.47:0/1461337864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec40cde10 0x7faec4160690 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faeb00097a0 tx=0x7faeb002b600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aaa1cec21fb79ff server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.779+0000 7faec37fe640 1 -- 172.21.15.47:0/1461337864 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc0019f0 con 0x7faec40cde10 2024-03-23T09:45:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.779+0000 7faec37fe640 1 -- 172.21.15.47:0/1461337864 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb002e050 con 0x7faec40cde10 2024-03-23T09:45:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.779+0000 7faec37fe640 1 -- 172.21.15.47:0/1461337864 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb0030510 con 0x7faec40cde10 2024-03-23T09:45:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.780+0000 7faecafb1640 1 -- 172.21.15.47:0/1461337864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec40cde10 msgr2=0x7faec4160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.780+0000 7faecafb1640 1 --2- 172.21.15.47:0/1461337864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec40cde10 0x7faec4160690 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faeb00097a0 tx=0x7faeb002b600 comp rx=0 tx=0).stop 2024-03-23T09:45:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.780+0000 7faecafb1640 1 -- 172.21.15.47:0/1461337864 shutdown_connections 2024-03-23T09:45:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.780+0000 7faecafb1640 1 --2- 172.21.15.47:0/1461337864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec40cde10 0x7faec4160690 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.780+0000 7faecafb1640 1 -- 172.21.15.47:0/1461337864 >> 172.21.15.47:0/1461337864 conn(0x7faec40659d0 msgr2=0x7faec4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.780+0000 7faecafb1640 1 -- 172.21.15.47:0/1461337864 shutdown_connections 2024-03-23T09:45:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.780+0000 7faecafb1640 1 -- 172.21.15.47:0/1461337864 wait complete. 2024-03-23T09:45:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.781+0000 7faecafb1640 1 Processor -- start 2024-03-23T09:45:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.781+0000 7faecafb1640 1 -- start start 2024-03-23T09:45:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faecafb1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faec40b8350 0x7faec4156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faecafb1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec40cde10 0x7faec41591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faecafb1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec415c990 0x7faec41574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faecafb1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec4159e10 con 0x7faec40b8350 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faecafb1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec4156780 con 0x7faec40cde10 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faecafb1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec4159b30 con 0x7faec415c990 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faec8d4d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faec40b8350 0x7faec4156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faec8d4d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faec40b8350 0x7faec4156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49808/0 (socket says 172.21.15.47:49808) 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faec8d4d640 1 -- 172.21.15.47:0/16038323 learned_addr learned my addr 172.21.15.47:0/16038323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faec954e640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec415c990 0x7faec41574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faec3fff640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec40cde10 0x7faec41591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faec8d4d640 1 -- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec415c990 msgr2=0x7faec41574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faec8d4d640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec415c990 0x7faec41574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faec8d4d640 1 -- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec40cde10 msgr2=0x7faec41591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faec8d4d640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec40cde10 0x7faec41591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faec8d4d640 1 -- 172.21.15.47:0/16038323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb0009400 con 0x7faec40b8350 2024-03-23T09:45:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faec954e640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec415c990 0x7faec41574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faec3fff640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec40cde10 0x7faec41591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.782+0000 7faec8d4d640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faec40b8350 0x7faec4156240 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7faeb800c7b0 tx=0x7faeb800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.783+0000 7faec17fa640 1 -- 172.21.15.47:0/16038323 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8005270 con 0x7faec40b8350 2024-03-23T09:45:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.783+0000 7faec17fa640 1 -- 172.21.15.47:0/16038323 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb80053f0 con 0x7faec40b8350 2024-03-23T09:45:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.783+0000 7faecafb1640 1 -- 172.21.15.47:0/16038323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec40c9c30 con 0x7faec40b8350 2024-03-23T09:45:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.783+0000 7faec17fa640 1 -- 172.21.15.47:0/16038323 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb800a880 con 0x7faec40b8350 2024-03-23T09:45:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.783+0000 7faecafb1640 1 -- 172.21.15.47:0/16038323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec40ca1f0 con 0x7faec40b8350 2024-03-23T09:45:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.784+0000 7faecafb1640 1 -- 172.21.15.47:0/16038323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec4072360 con 0x7faec40b8350 2024-03-23T09:45:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.784+0000 7faec17fa640 1 -- 172.21.15.47:0/16038323 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7faeb800a9e0 con 0x7faec40b8350 2024-03-23T09:45:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.784+0000 7faec17fa640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faea806ac30 0x7faea806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.785+0000 7faec17fa640 1 -- 172.21.15.47:0/16038323 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7faeb8088b10 con 0x7faec40b8350 2024-03-23T09:45:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.785+0000 7faec3fff640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faea806ac30 0x7faea806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:41.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.785+0000 7faec3fff640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faea806ac30 0x7faea806d0f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7faeb0034000 tx=0x7faeb0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:41.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.789+0000 7faec17fa640 1 -- 172.21.15.47:0/16038323 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7faeb8053c10 con 0x7faec40b8350 2024-03-23T09:45:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.967+0000 7faecafb1640 1 -- 172.21.15.47:0/16038323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faec40c9a70 con 0x7faec40b8350 2024-03-23T09:45:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.967+0000 7faec17fa640 1 -- 172.21.15.47:0/16038323 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7faeb80012d0 con 0x7faec40b8350 2024-03-23T09:45:41.967 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:45:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.969+0000 7faecafb1640 1 -- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faea806ac30 msgr2=0x7faea806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.969+0000 7faecafb1640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faea806ac30 0x7faea806d0f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7faeb0034000 tx=0x7faeb0008030 comp rx=0 tx=0).stop 2024-03-23T09:45:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.969+0000 7faecafb1640 1 -- 172.21.15.47:0/16038323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faec40b8350 msgr2=0x7faec4156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.969+0000 7faecafb1640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faec40b8350 0x7faec4156240 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7faeb800c7b0 tx=0x7faeb800cc70 comp rx=0 tx=0).stop 2024-03-23T09:45:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.969+0000 7faecafb1640 1 -- 172.21.15.47:0/16038323 shutdown_connections 2024-03-23T09:45:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.969+0000 7faecafb1640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faea806ac30 0x7faea806d0f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.969+0000 7faecafb1640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec415c990 0x7faec41574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.969+0000 7faecafb1640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec40cde10 0x7faec41591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.969+0000 7faecafb1640 1 --2- 172.21.15.47:0/16038323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faec40b8350 0x7faec4156240 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.969+0000 7faecafb1640 1 -- 172.21.15.47:0/16038323 >> 172.21.15.47:0/16038323 conn(0x7faec40659d0 msgr2=0x7faec40b5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.969+0000 7faecafb1640 1 -- 172.21.15.47:0/16038323 shutdown_connections 2024-03-23T09:45:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:41.969+0000 7faecafb1640 1 -- 172.21.15.47:0/16038323 wait complete. 2024-03-23T09:45:41.980 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:45:42.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.084+0000 7f6580da2640 1 Processor -- start 2024-03-23T09:45:42.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.084+0000 7f6580da2640 1 -- start start 2024-03-23T09:45:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.085+0000 7f6580da2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f657c15a8e0 0x7f657c0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.085+0000 7f6580da2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c15acc0 con 0x7f657c069ce0 2024-03-23T09:45:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.085+0000 7f6580da2640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c0caf20 con 0x7f657c0b08e0 2024-03-23T09:45:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.085+0000 7f6580da2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657c0afc90 con 0x7f657c15a8e0 2024-03-23T09:45:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.085+0000 7f657b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f657c15a8e0 0x7f657c0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.085+0000 7f657affd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f657c069ce0 0x7f657c0b01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58726/0 (socket says 172.21.15.47:58726) 2024-03-23T09:45:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.085+0000 7f657affd640 1 -- 172.21.15.47:0/1920920405 learned_addr learned my addr 172.21.15.47:0/1920920405 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.086+0000 7f657b7fe640 1 -- 172.21.15.47:0/1920920405 >> v1:172.21.15.175:6789/0 conn(0x7f657c0b08e0 legacy=0x7f657c0cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.086+0000 7f657b7fe640 1 -- 172.21.15.47:0/1920920405 >> v1:172.21.15.47:6789/0 conn(0x7f657c069ce0 legacy=0x7f657c0b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.086+0000 7f657b7fe640 1 -- 172.21.15.47:0/1920920405 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c0af110 con 0x7f657c15a8e0 2024-03-23T09:45:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.086+0000 7f657b7fe640 1 --2- 172.21.15.47:0/1920920405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f657c15a8e0 0x7f657c0aeb30 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f656c0097a0 tx=0x7f656c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477bed981468c5dc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.086+0000 7f657a7fc640 1 -- 172.21.15.47:0/1920920405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c031830 con 0x7f657c15a8e0 2024-03-23T09:45:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.087+0000 7f657a7fc640 1 -- 172.21.15.47:0/1920920405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656c031990 con 0x7f657c15a8e0 2024-03-23T09:45:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.087+0000 7f657a7fc640 1 -- 172.21.15.47:0/1920920405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c031d20 con 0x7f657c15a8e0 2024-03-23T09:45:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.087+0000 7f6580da2640 1 -- 172.21.15.47:0/1920920405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f657c15a8e0 msgr2=0x7f657c0aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.087+0000 7f6580da2640 1 --2- 172.21.15.47:0/1920920405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f657c15a8e0 0x7f657c0aeb30 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f656c0097a0 tx=0x7f656c02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.087+0000 7f6580da2640 1 -- 172.21.15.47:0/1920920405 shutdown_connections 2024-03-23T09:45:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.087+0000 7f6580da2640 1 --2- 172.21.15.47:0/1920920405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f657c15a8e0 0x7f657c0aeb30 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.087+0000 7f6580da2640 1 -- 172.21.15.47:0/1920920405 >> 172.21.15.47:0/1920920405 conn(0x7f657c098a00 msgr2=0x7f657c0b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.087+0000 7f6580da2640 1 -- 172.21.15.47:0/1920920405 shutdown_connections 2024-03-23T09:45:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.087+0000 7f6580da2640 1 -- 172.21.15.47:0/1920920405 wait complete. 2024-03-23T09:45:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.088+0000 7f6580da2640 1 Processor -- start 2024-03-23T09:45:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.088+0000 7f6580da2640 1 -- start start 2024-03-23T09:45:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.089+0000 7f6580da2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f657c069ce0 0x7f657c0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.089+0000 7f6580da2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f657c0b08e0 0x7f657c1f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.089+0000 7f6580da2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f657c15a8e0 0x7f657c0c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.089+0000 7f6580da2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c150c10 con 0x7f657c069ce0 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.089+0000 7f6580da2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c150ef0 con 0x7f657c0b08e0 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.089+0000 7f6580da2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c1511d0 con 0x7f657c15a8e0 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.089+0000 7f657b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f657c069ce0 0x7f657c0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.089+0000 7f657b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f657c069ce0 0x7f657c0ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49838/0 (socket says 172.21.15.47:49838) 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.089+0000 7f657b7fe640 1 -- 172.21.15.47:0/471115334 learned_addr learned my addr 172.21.15.47:0/471115334 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.090+0000 7f657b7fe640 1 -- 172.21.15.47:0/471115334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f657c15a8e0 msgr2=0x7f657c0c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.090+0000 7f657b7fe640 1 --2- 172.21.15.47:0/471115334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f657c15a8e0 0x7f657c0c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.090+0000 7f657affd640 1 --2- 172.21.15.47:0/471115334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f657c0b08e0 0x7f657c1f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.090+0000 7f657b7fe640 1 -- 172.21.15.47:0/471115334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f657c0b08e0 msgr2=0x7f657c1f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.090+0000 7f657b7fe640 1 --2- 172.21.15.47:0/471115334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f657c0b08e0 0x7f657c1f1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.090+0000 7f657b7fe640 1 -- 172.21.15.47:0/471115334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c009400 con 0x7f657c069ce0 2024-03-23T09:45:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.090+0000 7f657b7fe640 1 --2- 172.21.15.47:0/471115334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f657c069ce0 0x7f657c0ce170 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f656c0098c0 tx=0x7f656c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.090+0000 7f6563fff640 1 -- 172.21.15.47:0/471115334 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c03ba70 con 0x7f657c069ce0 2024-03-23T09:45:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.090+0000 7f6563fff640 1 -- 172.21.15.47:0/471115334 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656c03bbf0 con 0x7f657c069ce0 2024-03-23T09:45:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.090+0000 7f6580da2640 1 -- 172.21.15.47:0/471115334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c0cd970 con 0x7f657c069ce0 2024-03-23T09:45:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.090+0000 7f6563fff640 1 -- 172.21.15.47:0/471115334 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c0453f0 con 0x7f657c069ce0 2024-03-23T09:45:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.090+0000 7f6580da2640 1 -- 172.21.15.47:0/471115334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c0ce6b0 con 0x7f657c069ce0 2024-03-23T09:45:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.091+0000 7f6580da2640 1 -- 172.21.15.47:0/471115334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c0ce900 con 0x7f657c069ce0 2024-03-23T09:45:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.091+0000 7f6563fff640 1 -- 172.21.15.47:0/471115334 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f656c04e450 con 0x7f657c069ce0 2024-03-23T09:45:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.092+0000 7f6563fff640 1 --2- 172.21.15.47:0/471115334 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f656806ac00 0x7f656806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.092+0000 7f6563fff640 1 -- 172.21.15.47:0/471115334 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f656c0ab670 con 0x7f657c069ce0 2024-03-23T09:45:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.092+0000 7f657affd640 1 --2- 172.21.15.47:0/471115334 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f656806ac00 0x7f656806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.093+0000 7f657affd640 1 --2- 172.21.15.47:0/471115334 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f656806ac00 0x7f656806d0c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f657c151c70 tx=0x7f6570009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:42.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.096+0000 7f6563fff640 1 -- 172.21.15.47:0/471115334 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f656c0767f0 con 0x7f657c069ce0 2024-03-23T09:45:42.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.274+0000 7f6580da2640 1 -- 172.21.15.47:0/471115334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f657c0b1c00 con 0x7f657c069ce0 2024-03-23T09:45:42.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.275+0000 7f6563fff640 1 -- 172.21.15.47:0/471115334 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f656c079e10 con 0x7f657c069ce0 2024-03-23T09:45:42.275 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:45:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.277+0000 7f6580da2640 1 -- 172.21.15.47:0/471115334 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f656806ac00 msgr2=0x7f656806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.277+0000 7f6580da2640 1 --2- 172.21.15.47:0/471115334 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f656806ac00 0x7f656806d0c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f657c151c70 tx=0x7f6570009040 comp rx=0 tx=0).stop 2024-03-23T09:45:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.277+0000 7f6580da2640 1 -- 172.21.15.47:0/471115334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f657c069ce0 msgr2=0x7f657c0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.277+0000 7f6580da2640 1 --2- 172.21.15.47:0/471115334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f657c069ce0 0x7f657c0ce170 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f656c0098c0 tx=0x7f656c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.277+0000 7f6580da2640 1 -- 172.21.15.47:0/471115334 shutdown_connections 2024-03-23T09:45:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.277+0000 7f6580da2640 1 --2- 172.21.15.47:0/471115334 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f656806ac00 0x7f656806d0c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.277+0000 7f6580da2640 1 --2- 172.21.15.47:0/471115334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f657c15a8e0 0x7f657c0c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.277+0000 7f6580da2640 1 --2- 172.21.15.47:0/471115334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f657c0b08e0 0x7f657c1f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.277+0000 7f6580da2640 1 --2- 172.21.15.47:0/471115334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f657c069ce0 0x7f657c0ce170 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.277+0000 7f6580da2640 1 -- 172.21.15.47:0/471115334 >> 172.21.15.47:0/471115334 conn(0x7f657c098a00 msgr2=0x7f657c06d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.277+0000 7f6580da2640 1 -- 172.21.15.47:0/471115334 shutdown_connections 2024-03-23T09:45:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.277+0000 7f6580da2640 1 -- 172.21.15.47:0/471115334 wait complete. 2024-03-23T09:45:42.287 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:45:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.393+0000 7f6464b9f640 1 Processor -- start 2024-03-23T09:45:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.393+0000 7f6464b9f640 1 -- start start 2024-03-23T09:45:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.394+0000 7f6464b9f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f646015cbf0 0x7f64600b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.394+0000 7f6464b9f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f64600cd140 0x7f64600cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.394+0000 7f6464b9f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6460161570 con 0x7f64600cf7f0 2024-03-23T09:45:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.394+0000 7f6464b9f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64600cfc00 con 0x7f646015cbf0 2024-03-23T09:45:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.394+0000 7f6464b9f640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6460160960 con 0x7f64600cd140 2024-03-23T09:45:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.394+0000 7f645e59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f646015cbf0 0x7f64600b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.394+0000 7f645e59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f646015cbf0 0x7f64600b1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49846/0 (socket says 172.21.15.47:49846) 2024-03-23T09:45:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.394+0000 7f645e59c640 1 -- 172.21.15.47:0/2298616617 learned_addr learned my addr 172.21.15.47:0/2298616617 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.394+0000 7f645dd9b640 1 --2- 172.21.15.47:0/2298616617 >> v2:172.21.15.175:3300/0 conn(0x7f64600cd140 0x7f64600cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.394+0000 7f645e59c640 1 -- 172.21.15.47:0/2298616617 >> v2:172.21.15.175:3300/0 conn(0x7f64600cd140 msgr2=0x7f64600cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.394+0000 7f645e59c640 1 --2- 172.21.15.47:0/2298616617 >> v2:172.21.15.175:3300/0 conn(0x7f64600cd140 0x7f64600cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.394+0000 7f645e59c640 1 -- 172.21.15.47:0/2298616617 >> v1:172.21.15.175:6789/0 conn(0x7f64600cf7f0 legacy=0x7f64600b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.394+0000 7f645e59c640 1 -- 172.21.15.47:0/2298616617 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6460160c40 con 0x7f646015cbf0 2024-03-23T09:45:42.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.395+0000 7f645e59c640 1 --2- 172.21.15.47:0/2298616617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646015cbf0 0x7f64600b1df0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f646008ada0 tx=0x7f645002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659572fb570b5dff server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:42.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.395+0000 7f645d59a640 1 -- 172.21.15.47:0/2298616617 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645002ca80 con 0x7f646015cbf0 2024-03-23T09:45:42.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.395+0000 7f645d59a640 1 -- 172.21.15.47:0/2298616617 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645002cbe0 con 0x7f646015cbf0 2024-03-23T09:45:42.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.395+0000 7f645d59a640 1 -- 172.21.15.47:0/2298616617 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500343f0 con 0x7f646015cbf0 2024-03-23T09:45:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.396+0000 7f6464b9f640 1 -- 172.21.15.47:0/2298616617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646015cbf0 msgr2=0x7f64600b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.396+0000 7f6464b9f640 1 --2- 172.21.15.47:0/2298616617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646015cbf0 0x7f64600b1df0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f646008ada0 tx=0x7f645002bee0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.396+0000 7f6464b9f640 1 -- 172.21.15.47:0/2298616617 shutdown_connections 2024-03-23T09:45:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.396+0000 7f6464b9f640 1 --2- 172.21.15.47:0/2298616617 >> v2:172.21.15.175:3300/0 conn(0x7f64600cd140 0x7f64600cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.396+0000 7f6464b9f640 1 --2- 172.21.15.47:0/2298616617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646015cbf0 0x7f64600b1df0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.396+0000 7f6464b9f640 1 -- 172.21.15.47:0/2298616617 >> 172.21.15.47:0/2298616617 conn(0x7f646006b160 msgr2=0x7f646006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.396+0000 7f6464b9f640 1 -- 172.21.15.47:0/2298616617 shutdown_connections 2024-03-23T09:45:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.396+0000 7f6464b9f640 1 -- 172.21.15.47:0/2298616617 wait complete. 2024-03-23T09:45:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.397+0000 7f6464b9f640 1 Processor -- start 2024-03-23T09:45:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.397+0000 7f6464b9f640 1 -- start start 2024-03-23T09:45:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f6464b9f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64600cd140 0x7f646014ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f6464b9f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64600cf7f0 0x7f646015f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f6464b9f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646015cbf0 0x7f64600cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f6464b9f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64601f3610 con 0x7f646015cbf0 2024-03-23T09:45:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f6464b9f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64600c9d20 con 0x7f64600cd140 2024-03-23T09:45:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f6464b9f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64600ca000 con 0x7f64600cf7f0 2024-03-23T09:45:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f645ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646015cbf0 0x7f64600cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f645ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646015cbf0 0x7f64600cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49860/0 (socket says 172.21.15.47:49860) 2024-03-23T09:45:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f645ed9d640 1 -- 172.21.15.47:0/4251870841 learned_addr learned my addr 172.21.15.47:0/4251870841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f645ed9d640 1 -- 172.21.15.47:0/4251870841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64600cf7f0 msgr2=0x7f646015f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f645ed9d640 1 --2- 172.21.15.47:0/4251870841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64600cf7f0 0x7f646015f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f645ed9d640 1 -- 172.21.15.47:0/4251870841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64600cd140 msgr2=0x7f646014ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f645ed9d640 1 --2- 172.21.15.47:0/4251870841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64600cd140 0x7f646014ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f645ed9d640 1 -- 172.21.15.47:0/4251870841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6450009b30 con 0x7f646015cbf0 2024-03-23T09:45:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f645ed9d640 1 --2- 172.21.15.47:0/4251870841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646015cbf0 0x7f64600cb730 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f645800c410 tx=0x7f645800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.398+0000 7f6456ffd640 1 -- 172.21.15.47:0/4251870841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6458011790 con 0x7f646015cbf0 2024-03-23T09:45:42.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.399+0000 7f6456ffd640 1 -- 172.21.15.47:0/4251870841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64580118f0 con 0x7f646015cbf0 2024-03-23T09:45:42.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.399+0000 7f6464b9f640 1 -- 172.21.15.47:0/4251870841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64600ca2e0 con 0x7f646015cbf0 2024-03-23T09:45:42.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.399+0000 7f6456ffd640 1 -- 172.21.15.47:0/4251870841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6458011ce0 con 0x7f646015cbf0 2024-03-23T09:45:42.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.399+0000 7f6464b9f640 1 -- 172.21.15.47:0/4251870841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64600ca8a0 con 0x7f646015cbf0 2024-03-23T09:45:42.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.399+0000 7f6464b9f640 1 -- 172.21.15.47:0/4251870841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6460072360 con 0x7f646015cbf0 2024-03-23T09:45:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.400+0000 7f6456ffd640 1 -- 172.21.15.47:0/4251870841 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f645801c460 con 0x7f646015cbf0 2024-03-23T09:45:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.400+0000 7f6456ffd640 1 --2- 172.21.15.47:0/4251870841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f644006ac00 0x7f644006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.400+0000 7f6456ffd640 1 -- 172.21.15.47:0/4251870841 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6458089060 con 0x7f646015cbf0 2024-03-23T09:45:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.401+0000 7f645e59c640 1 --2- 172.21.15.47:0/4251870841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f644006ac00 0x7f644006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.401+0000 7f645e59c640 1 --2- 172.21.15.47:0/4251870841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f644006ac00 0x7f644006d0c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6450035000 tx=0x7f645002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:42.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.404+0000 7f6456ffd640 1 -- 172.21.15.47:0/4251870841 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f64580541e0 con 0x7f646015cbf0 2024-03-23T09:45:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.586+0000 7f6464b9f640 1 -- 172.21.15.47:0/4251870841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64600c6f50 con 0x7f646015cbf0 2024-03-23T09:45:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.586+0000 7f6456ffd640 1 -- 172.21.15.47:0/4251870841 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6458057800 con 0x7f646015cbf0 2024-03-23T09:45:42.586 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:45:42.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.588+0000 7f6464b9f640 1 -- 172.21.15.47:0/4251870841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f644006ac00 msgr2=0x7f644006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.588+0000 7f6464b9f640 1 --2- 172.21.15.47:0/4251870841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f644006ac00 0x7f644006d0c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6450035000 tx=0x7f645002e000 comp rx=0 tx=0).stop 2024-03-23T09:45:42.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.588+0000 7f6464b9f640 1 -- 172.21.15.47:0/4251870841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646015cbf0 msgr2=0x7f64600cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.588+0000 7f6464b9f640 1 --2- 172.21.15.47:0/4251870841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646015cbf0 0x7f64600cb730 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f645800c410 tx=0x7f645800c8d0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.588+0000 7f6464b9f640 1 -- 172.21.15.47:0/4251870841 shutdown_connections 2024-03-23T09:45:42.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.588+0000 7f6464b9f640 1 --2- 172.21.15.47:0/4251870841 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f644006ac00 0x7f644006d0c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.588+0000 7f6464b9f640 1 --2- 172.21.15.47:0/4251870841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646015cbf0 0x7f64600cb730 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.588+0000 7f6464b9f640 1 --2- 172.21.15.47:0/4251870841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64600cf7f0 0x7f646015f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.588+0000 7f6464b9f640 1 --2- 172.21.15.47:0/4251870841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64600cd140 0x7f646014ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.588+0000 7f6464b9f640 1 -- 172.21.15.47:0/4251870841 >> 172.21.15.47:0/4251870841 conn(0x7f646006b160 msgr2=0x7f64600d1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:42.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.588+0000 7f6464b9f640 1 -- 172.21.15.47:0/4251870841 shutdown_connections 2024-03-23T09:45:42.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.589+0000 7f6464b9f640 1 -- 172.21.15.47:0/4251870841 wait complete. 2024-03-23T09:45:42.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:45:42.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.701+0000 7f06e8ac0640 1 Processor -- start 2024-03-23T09:45:42.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.701+0000 7f06e8ac0640 1 -- start start 2024-03-23T09:45:42.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.701+0000 7f06e8ac0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06e40ceca0 0x7f06e40c4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.702+0000 7f06e8ac0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06e4099a40 0x7f06e40cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.702+0000 7f06e8ac0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e40cb1d0 con 0x7f06e4099410 2024-03-23T09:45:42.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.702+0000 7f06e8ac0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e415e230 con 0x7f06e40ceca0 2024-03-23T09:45:42.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.702+0000 7f06e8ac0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e40ce2b0 con 0x7f06e4099a40 2024-03-23T09:45:42.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.702+0000 7f06e259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06e40ceca0 0x7f06e40c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:42.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.702+0000 7f06dbfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f06e4099410 0x7f06e40cf080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58736/0 (socket says 172.21.15.47:58736) 2024-03-23T09:45:42.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.702+0000 7f06dbfff640 1 -- 172.21.15.47:0/1538230556 learned_addr learned my addr 172.21.15.47:0/1538230556 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:42.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.702+0000 7f06e2d9d640 1 --2- 172.21.15.47:0/1538230556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06e4099a40 0x7f06e40cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:42.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.702+0000 7f06e259c640 1 -- 172.21.15.47:0/1538230556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06e4099a40 msgr2=0x7f06e40cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:42.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.702+0000 7f06e259c640 1 --2- 172.21.15.47:0/1538230556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06e4099a40 0x7f06e40cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.702+0000 7f06e259c640 1 -- 172.21.15.47:0/1538230556 >> v1:172.21.15.47:6789/0 conn(0x7f06e4099410 legacy=0x7f06e40cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.702+0000 7f06e259c640 1 -- 172.21.15.47:0/1538230556 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e40ce590 con 0x7f06e40ceca0 2024-03-23T09:45:42.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.702+0000 7f06e1d9b640 1 -- 172.21.15.47:0/1538230556 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244877186 0 0) 0x7f06e40cb1d0 con 0x7f06e4099410 2024-03-23T09:45:42.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.703+0000 7f06e259c640 1 --2- 172.21.15.47:0/1538230556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e40ceca0 0x7f06e40c4230 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f06d40097a0 tx=0x7f06d402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7fedcfd3aff2da server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.703+0000 7f06e1d9b640 1 -- 172.21.15.47:0/1538230556 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d402e070 con 0x7f06e40ceca0 2024-03-23T09:45:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.703+0000 7f06e1d9b640 1 -- 172.21.15.47:0/1538230556 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d40315a0 con 0x7f06e40ceca0 2024-03-23T09:45:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.703+0000 7f06e1d9b640 1 -- 172.21.15.47:0/1538230556 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4031930 con 0x7f06e40ceca0 2024-03-23T09:45:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.703+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1538230556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e40ceca0 msgr2=0x7f06e40c4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.703+0000 7f06e8ac0640 1 --2- 172.21.15.47:0/1538230556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e40ceca0 0x7f06e40c4230 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f06d40097a0 tx=0x7f06d402b840 comp rx=0 tx=0).stop 2024-03-23T09:45:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.703+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1538230556 shutdown_connections 2024-03-23T09:45:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.703+0000 7f06e8ac0640 1 --2- 172.21.15.47:0/1538230556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06e4099a40 0x7f06e40cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.703+0000 7f06e8ac0640 1 --2- 172.21.15.47:0/1538230556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e40ceca0 0x7f06e40c4230 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.703+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1538230556 >> 172.21.15.47:0/1538230556 conn(0x7f06e40b21d0 msgr2=0x7f06e40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.703+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1538230556 shutdown_connections 2024-03-23T09:45:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.704+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1538230556 wait complete. 2024-03-23T09:45:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.705+0000 7f06e8ac0640 1 Processor -- start 2024-03-23T09:45:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.705+0000 7f06e8ac0640 1 -- start start 2024-03-23T09:45:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.705+0000 7f06e8ac0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06e4099410 0x7f06e40c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.705+0000 7f06e8ac0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e4099a40 0x7f06e415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.705+0000 7f06e8ac0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e40ceca0 0x7f06e40ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.705+0000 7f06e8ac0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e415b890 con 0x7f06e4099a40 2024-03-23T09:45:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06e8ac0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e415ac10 con 0x7f06e40ceca0 2024-03-23T09:45:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06e8ac0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e415aef0 con 0x7f06e4099410 2024-03-23T09:45:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06dbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e4099a40 0x7f06e415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06dbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e4099a40 0x7f06e415a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49884/0 (socket says 172.21.15.47:49884) 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06dbfff640 1 -- 172.21.15.47:0/1261405808 learned_addr learned my addr 172.21.15.47:0/1261405808 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06e259c640 1 --2- 172.21.15.47:0/1261405808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06e4099410 0x7f06e40c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06dbfff640 1 -- 172.21.15.47:0/1261405808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06e4099410 msgr2=0x7f06e40c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06dbfff640 1 --2- 172.21.15.47:0/1261405808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06e4099410 0x7f06e40c9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06dbfff640 1 -- 172.21.15.47:0/1261405808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e40ceca0 msgr2=0x7f06e40ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06dbfff640 1 --2- 172.21.15.47:0/1261405808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e40ceca0 0x7f06e40ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06dbfff640 1 -- 172.21.15.47:0/1261405808 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d4009400 con 0x7f06e4099a40 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06dbfff640 1 --2- 172.21.15.47:0/1261405808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e4099a40 0x7f06e415a2a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00c430 tx=0x7f06cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06daffd640 1 -- 172.21.15.47:0/1261405808 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0074e0 con 0x7f06e4099a40 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06daffd640 1 -- 172.21.15.47:0/1261405808 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06cc007660 con 0x7f06e4099a40 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1261405808 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e415b1d0 con 0x7f06e4099a40 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06daffd640 1 -- 172.21.15.47:0/1261405808 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc004a20 con 0x7f06e4099a40 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.706+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1261405808 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e40c9f70 con 0x7f06e4099a40 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.707+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1261405808 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e415b9d0 con 0x7f06e4099a40 2024-03-23T09:45:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.708+0000 7f06daffd640 1 -- 172.21.15.47:0/1261405808 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f06cc004b80 con 0x7f06e4099a40 2024-03-23T09:45:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.708+0000 7f06daffd640 1 --2- 172.21.15.47:0/1261405808 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f06c406ac00 0x7f06c406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.708+0000 7f06daffd640 1 -- 172.21.15.47:0/1261405808 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f06cc0891f0 con 0x7f06e4099a40 2024-03-23T09:45:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.708+0000 7f06e259c640 1 --2- 172.21.15.47:0/1261405808 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f06c406ac00 0x7f06c406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.709+0000 7f06e259c640 1 --2- 172.21.15.47:0/1261405808 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f06c406ac00 0x7f06c406d0c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f06d4036000 tx=0x7f06d4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:42.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.712+0000 7f06daffd640 1 -- 172.21.15.47:0/1261405808 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f06e415b9d0 con 0x7f06e4099a40 2024-03-23T09:45:42.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.905+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1261405808 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f06e415b9d0 con 0x7f06e4099a40 2024-03-23T09:45:42.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.905+0000 7f06daffd640 1 -- 172.21.15.47:0/1261405808 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v71) v1 ==== 80+0+120 (secure 0 0 0) 0x7f06e415b9d0 con 0x7f06e4099a40 2024-03-23T09:45:42.906 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6835/372771687 2024-03-24T09:45:33.740376+0000 2024-03-23T09:45:42.906 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6833/372771687 2024-03-24T09:45:33.740376+0000 2024-03-23T09:45:42.906 INFO:teuthology.orchestra.run.smithi047.stderr:listed 2 entries 2024-03-23T09:45:42.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.907+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1261405808 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f06c406ac00 msgr2=0x7f06c406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.907+0000 7f06e8ac0640 1 --2- 172.21.15.47:0/1261405808 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f06c406ac00 0x7f06c406d0c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f06d4036000 tx=0x7f06d4008030 comp rx=0 tx=0).stop 2024-03-23T09:45:42.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.907+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1261405808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e4099a40 msgr2=0x7f06e415a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:42.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.907+0000 7f06e8ac0640 1 --2- 172.21.15.47:0/1261405808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e4099a40 0x7f06e415a2a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00c430 tx=0x7f06cc00c8f0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.907+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1261405808 shutdown_connections 2024-03-23T09:45:42.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.907+0000 7f06e8ac0640 1 --2- 172.21.15.47:0/1261405808 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f06c406ac00 0x7f06c406d0c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.907+0000 7f06e8ac0640 1 --2- 172.21.15.47:0/1261405808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e40ceca0 0x7f06e40ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.907+0000 7f06e8ac0640 1 --2- 172.21.15.47:0/1261405808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e4099a40 0x7f06e415a2a0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.907+0000 7f06e8ac0640 1 --2- 172.21.15.47:0/1261405808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06e4099410 0x7f06e40c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:42.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.907+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1261405808 >> 172.21.15.47:0/1261405808 conn(0x7f06e40b21d0 msgr2=0x7f06e40c5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:42.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.907+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1261405808 shutdown_connections 2024-03-23T09:45:42.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:42.907+0000 7f06e8ac0640 1 -- 172.21.15.47:0/1261405808 wait complete. 2024-03-23T09:45:42.918 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:45:42.918 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:45:42.933 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:45:43.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.073+0000 7fb93cce0640 1 Processor -- start 2024-03-23T09:45:43.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.073+0000 7fb93cce0640 1 -- start start 2024-03-23T09:45:43.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.074+0000 7fb93cce0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb9380b8350 0x7fb9380cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.074+0000 7fb93cce0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9380cdc40 0x7fb9380d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.074+0000 7fb93cce0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9380d11e0 con 0x7fb93815ca90 2024-03-23T09:45:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.074+0000 7fb93cce0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9380ce0d0 con 0x7fb9380b8350 2024-03-23T09:45:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.074+0000 7fb93cce0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb938160f60 con 0x7fb9380cdc40 2024-03-23T09:45:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.074+0000 7fb935d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb9380b8350 0x7fb9380cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.074+0000 7fb93659c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb93815ca90 0x7fb938160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58750/0 (socket says 172.21.15.47:58750) 2024-03-23T09:45:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.074+0000 7fb93659c640 1 -- 172.21.15.47:0/1274643100 learned_addr learned my addr 172.21.15.47:0/1274643100 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.074+0000 7fb936d9d640 1 --2- 172.21.15.47:0/1274643100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9380cdc40 0x7fb9380d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.074+0000 7fb935d9b640 1 -- 172.21.15.47:0/1274643100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9380cdc40 msgr2=0x7fb9380d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.074+0000 7fb935d9b640 1 --2- 172.21.15.47:0/1274643100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9380cdc40 0x7fb9380d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.074+0000 7fb935d9b640 1 -- 172.21.15.47:0/1274643100 >> v1:172.21.15.47:6789/0 conn(0x7fb93815ca90 legacy=0x7fb938160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.074+0000 7fb93559a640 1 -- 172.21.15.47:0/1274643100 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960660512 0 0) 0x7fb9380d11e0 con 0x7fb93815ca90 2024-03-23T09:45:43.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.075+0000 7fb935d9b640 1 -- 172.21.15.47:0/1274643100 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb938161240 con 0x7fb9380b8350 2024-03-23T09:45:43.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.075+0000 7fb935d9b640 1 --2- 172.21.15.47:0/1274643100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9380b8350 0x7fb9380cd700 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb9200097a0 tx=0x7fb92002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe40e5eab47894c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:43.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.075+0000 7fb93559a640 1 -- 172.21.15.47:0/1274643100 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92002e070 con 0x7fb9380b8350 2024-03-23T09:45:43.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.075+0000 7fb93559a640 1 -- 172.21.15.47:0/1274643100 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9200315a0 con 0x7fb9380b8350 2024-03-23T09:45:43.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.075+0000 7fb93559a640 1 -- 172.21.15.47:0/1274643100 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920031930 con 0x7fb9380b8350 2024-03-23T09:45:43.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.076+0000 7fb93cce0640 1 -- 172.21.15.47:0/1274643100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9380b8350 msgr2=0x7fb9380cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.076+0000 7fb93cce0640 1 --2- 172.21.15.47:0/1274643100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9380b8350 0x7fb9380cd700 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb9200097a0 tx=0x7fb92002b840 comp rx=0 tx=0).stop 2024-03-23T09:45:43.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.076+0000 7fb93cce0640 1 -- 172.21.15.47:0/1274643100 shutdown_connections 2024-03-23T09:45:43.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.076+0000 7fb93cce0640 1 --2- 172.21.15.47:0/1274643100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9380cdc40 0x7fb9380d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.076+0000 7fb93cce0640 1 --2- 172.21.15.47:0/1274643100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9380b8350 0x7fb9380cd700 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.076+0000 7fb93cce0640 1 -- 172.21.15.47:0/1274643100 >> 172.21.15.47:0/1274643100 conn(0x7fb9380659d0 msgr2=0x7fb938065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:43.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.076+0000 7fb93cce0640 1 -- 172.21.15.47:0/1274643100 shutdown_connections 2024-03-23T09:45:43.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.076+0000 7fb93cce0640 1 -- 172.21.15.47:0/1274643100 wait complete. 2024-03-23T09:45:43.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.077+0000 7fb93cce0640 1 Processor -- start 2024-03-23T09:45:43.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.077+0000 7fb93cce0640 1 -- start start 2024-03-23T09:45:43.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb93cce0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9380b8350 0x7fb938153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb93cce0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9380cdc40 0x7fb9380b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb93cce0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb93815ca90 0x7fb9380cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb93cce0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9381f6950 con 0x7fb93815ca90 2024-03-23T09:45:43.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb93cce0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9380b5360 con 0x7fb9380cdc40 2024-03-23T09:45:43.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb93cce0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9381f5cd0 con 0x7fb9380b8350 2024-03-23T09:45:43.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb93815ca90 0x7fb9380cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb93815ca90 0x7fb9380cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49902/0 (socket says 172.21.15.47:49902) 2024-03-23T09:45:43.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb936d9d640 1 -- 172.21.15.47:0/936961661 learned_addr learned my addr 172.21.15.47:0/936961661 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:43.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb93659c640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9380b8350 0x7fb938153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb935d9b640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9380cdc40 0x7fb9380b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb936d9d640 1 -- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9380b8350 msgr2=0x7fb938153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb936d9d640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9380b8350 0x7fb938153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb936d9d640 1 -- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9380cdc40 msgr2=0x7fb9380b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb936d9d640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9380cdc40 0x7fb9380b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.078+0000 7fb936d9d640 1 -- 172.21.15.47:0/936961661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb920009400 con 0x7fb93815ca90 2024-03-23T09:45:43.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.079+0000 7fb936d9d640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb93815ca90 0x7fb9380cc1c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb93000d5d0 tx=0x7fb93000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:43.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.079+0000 7fb92effd640 1 -- 172.21.15.47:0/936961661 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9300125f0 con 0x7fb93815ca90 2024-03-23T09:45:43.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.079+0000 7fb92effd640 1 -- 172.21.15.47:0/936961661 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb930012770 con 0x7fb93815ca90 2024-03-23T09:45:43.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.079+0000 7fb93cce0640 1 -- 172.21.15.47:0/936961661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9381f5fb0 con 0x7fb93815ca90 2024-03-23T09:45:43.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.079+0000 7fb92effd640 1 -- 172.21.15.47:0/936961661 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930012b20 con 0x7fb93815ca90 2024-03-23T09:45:43.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.079+0000 7fb93cce0640 1 -- 172.21.15.47:0/936961661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9381f6570 con 0x7fb93815ca90 2024-03-23T09:45:43.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.080+0000 7fb93cce0640 1 -- 172.21.15.47:0/936961661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb938072360 con 0x7fb93815ca90 2024-03-23T09:45:43.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.080+0000 7fb92effd640 1 -- 172.21.15.47:0/936961661 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb930012c80 con 0x7fb93815ca90 2024-03-23T09:45:43.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.081+0000 7fb92effd640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb91806ac00 0x7fb91806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.081+0000 7fb92effd640 1 -- 172.21.15.47:0/936961661 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb930089890 con 0x7fb93815ca90 2024-03-23T09:45:43.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.081+0000 7fb93659c640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb91806ac00 0x7fb91806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.081+0000 7fb93659c640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb91806ac00 0x7fb91806d0c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb9280012e0 tx=0x7fb928007ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:43.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.085+0000 7fb92effd640 1 -- 172.21.15.47:0/936961661 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb930054a20 con 0x7fb93815ca90 2024-03-23T09:45:43.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.266+0000 7fb93cce0640 1 -- 172.21.15.47:0/936961661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9381f6a90 con 0x7fb93815ca90 2024-03-23T09:45:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.267+0000 7fb92effd640 1 -- 172.21.15.47:0/936961661 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16227 (secure 0 0 0) 0x7fb9381f6a90 con 0x7fb93815ca90 2024-03-23T09:45:43.267 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:45:43.268 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":71,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:45:33.740401+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:45:16.609941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-23T09:45:17.261503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/372771687":"2024-03-24T09:45:33.740376+0000","172.21.15.47:6833/372771687":"2024-03-24T09:45:33.740376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:45:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.269+0000 7fb93cce0640 1 -- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb91806ac00 msgr2=0x7fb91806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.269+0000 7fb93cce0640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb91806ac00 0x7fb91806d0c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb9280012e0 tx=0x7fb928007ed0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.269+0000 7fb93cce0640 1 -- 172.21.15.47:0/936961661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb93815ca90 msgr2=0x7fb9380cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.269+0000 7fb93cce0640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb93815ca90 0x7fb9380cc1c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb93000d5d0 tx=0x7fb93000da90 comp rx=0 tx=0).stop 2024-03-23T09:45:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.269+0000 7fb93cce0640 1 -- 172.21.15.47:0/936961661 shutdown_connections 2024-03-23T09:45:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.269+0000 7fb93cce0640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb91806ac00 0x7fb91806d0c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.269+0000 7fb93cce0640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb93815ca90 0x7fb9380cc1c0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.269+0000 7fb93cce0640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9380cdc40 0x7fb9380b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.269+0000 7fb93cce0640 1 --2- 172.21.15.47:0/936961661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9380b8350 0x7fb938153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.269+0000 7fb93cce0640 1 -- 172.21.15.47:0/936961661 >> 172.21.15.47:0/936961661 conn(0x7fb9380659d0 msgr2=0x7fb93815fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.269+0000 7fb93cce0640 1 -- 172.21.15.47:0/936961661 shutdown_connections 2024-03-23T09:45:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.269+0000 7fb93cce0640 1 -- 172.21.15.47:0/936961661 wait complete. 2024-03-23T09:45:43.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:45:16.609941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:45:17.261503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:45:43.281 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:45:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.384+0000 7efc5782c640 1 Processor -- start 2024-03-23T09:45:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.384+0000 7efc5782c640 1 -- start start 2024-03-23T09:45:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc5782c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efc5015c990 0x7efc50160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc5782c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc500b8350 0x7efc500cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc5782c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc50161630 con 0x7efc500cdc40 2024-03-23T09:45:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc5782c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc500ce0b0 con 0x7efc5015c990 2024-03-23T09:45:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc5782c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc500d1030 con 0x7efc500b8350 2024-03-23T09:45:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc555c8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efc5015c990 0x7efc50160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc55dc9640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efc500cdc40 0x7efc500b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58758/0 (socket says 172.21.15.47:58758) 2024-03-23T09:45:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc55dc9640 1 -- 172.21.15.47:0/959735551 learned_addr learned my addr 172.21.15.47:0/959735551 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc54dc7640 1 --2- 172.21.15.47:0/959735551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc500b8350 0x7efc500cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc555c8640 1 -- 172.21.15.47:0/959735551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc500b8350 msgr2=0x7efc500cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc555c8640 1 --2- 172.21.15.47:0/959735551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc500b8350 0x7efc500cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc555c8640 1 -- 172.21.15.47:0/959735551 >> v1:172.21.15.47:6789/0 conn(0x7efc500cdc40 legacy=0x7efc500b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc47fff640 1 -- 172.21.15.47:0/959735551 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440165691 0 0) 0x7efc50161630 con 0x7efc500cdc40 2024-03-23T09:45:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.385+0000 7efc555c8640 1 -- 172.21.15.47:0/959735551 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc500cfe10 con 0x7efc5015c990 2024-03-23T09:45:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.386+0000 7efc555c8640 1 --2- 172.21.15.47:0/959735551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5015c990 0x7efc50160690 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0097a0 tx=0x7efc4c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd8a0cd7aea115b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.386+0000 7efc47fff640 1 -- 172.21.15.47:0/959735551 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c02e070 con 0x7efc5015c990 2024-03-23T09:45:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.386+0000 7efc47fff640 1 -- 172.21.15.47:0/959735551 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c0315a0 con 0x7efc5015c990 2024-03-23T09:45:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.386+0000 7efc47fff640 1 -- 172.21.15.47:0/959735551 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c031930 con 0x7efc5015c990 2024-03-23T09:45:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.386+0000 7efc5782c640 1 -- 172.21.15.47:0/959735551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5015c990 msgr2=0x7efc50160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.386+0000 7efc5782c640 1 --2- 172.21.15.47:0/959735551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5015c990 0x7efc50160690 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0097a0 tx=0x7efc4c02b840 comp rx=0 tx=0).stop 2024-03-23T09:45:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.386+0000 7efc5782c640 1 -- 172.21.15.47:0/959735551 shutdown_connections 2024-03-23T09:45:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.386+0000 7efc5782c640 1 --2- 172.21.15.47:0/959735551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc500b8350 0x7efc500cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.386+0000 7efc5782c640 1 --2- 172.21.15.47:0/959735551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5015c990 0x7efc50160690 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.386+0000 7efc5782c640 1 -- 172.21.15.47:0/959735551 >> 172.21.15.47:0/959735551 conn(0x7efc500659d0 msgr2=0x7efc50065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.387+0000 7efc5782c640 1 -- 172.21.15.47:0/959735551 shutdown_connections 2024-03-23T09:45:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.387+0000 7efc5782c640 1 -- 172.21.15.47:0/959735551 wait complete. 2024-03-23T09:45:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.388+0000 7efc5782c640 1 Processor -- start 2024-03-23T09:45:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.388+0000 7efc5782c640 1 -- start start 2024-03-23T09:45:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.388+0000 7efc5782c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc500b8350 0x7efc50156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.388+0000 7efc5782c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc500cdc40 0x7efc500c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.388+0000 7efc5782c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5015c990 0x7efc500c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.388+0000 7efc5782c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc500cc730 con 0x7efc5015c990 2024-03-23T09:45:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.388+0000 7efc5782c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc501f6bc0 con 0x7efc500b8350 2024-03-23T09:45:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.388+0000 7efc5782c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc501f68e0 con 0x7efc500cdc40 2024-03-23T09:45:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.388+0000 7efc55dc9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5015c990 0x7efc500c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.388+0000 7efc55dc9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5015c990 0x7efc500c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49922/0 (socket says 172.21.15.47:49922) 2024-03-23T09:45:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.388+0000 7efc55dc9640 1 -- 172.21.15.47:0/3257877285 learned_addr learned my addr 172.21.15.47:0/3257877285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.389+0000 7efc54dc7640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc500cdc40 0x7efc500c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.389+0000 7efc555c8640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc500b8350 0x7efc50156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.389+0000 7efc55dc9640 1 -- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc500cdc40 msgr2=0x7efc500c7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.389+0000 7efc55dc9640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc500cdc40 0x7efc500c7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.389+0000 7efc55dc9640 1 -- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc500b8350 msgr2=0x7efc50156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.389+0000 7efc55dc9640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc500b8350 0x7efc50156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.389+0000 7efc55dc9640 1 -- 172.21.15.47:0/3257877285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c009400 con 0x7efc5015c990 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.389+0000 7efc55dc9640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5015c990 0x7efc500c7940 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efc4800c7e0 tx=0x7efc4800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.389+0000 7efc45ffb640 1 -- 172.21.15.47:0/3257877285 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4800a640 con 0x7efc5015c990 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.389+0000 7efc45ffb640 1 -- 172.21.15.47:0/3257877285 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4800a7c0 con 0x7efc5015c990 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.389+0000 7efc5782c640 1 -- 172.21.15.47:0/3257877285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5015df40 con 0x7efc5015c990 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.389+0000 7efc45ffb640 1 -- 172.21.15.47:0/3257877285 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4800ab70 con 0x7efc5015c990 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.389+0000 7efc5782c640 1 -- 172.21.15.47:0/3257877285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5015e500 con 0x7efc5015c990 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.390+0000 7efc5782c640 1 -- 172.21.15.47:0/3257877285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc50072360 con 0x7efc5015c990 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.391+0000 7efc45ffb640 1 -- 172.21.15.47:0/3257877285 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efc4800acd0 con 0x7efc5015c990 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.391+0000 7efc45ffb640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efc3406ac00 0x7efc3406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.391+0000 7efc45ffb640 1 -- 172.21.15.47:0/3257877285 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efc48088900 con 0x7efc5015c990 2024-03-23T09:45:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.392+0000 7efc555c8640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efc3406ac00 0x7efc3406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.392+0000 7efc555c8640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efc3406ac00 0x7efc3406d0c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efc4c036000 tx=0x7efc4c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.395+0000 7efc45ffb640 1 -- 172.21.15.47:0/3257877285 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efc48053b20 con 0x7efc5015c990 2024-03-23T09:45:43.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.572+0000 7efc5782c640 1 -- 172.21.15.47:0/3257877285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc5015e720 con 0x7efc5015c990 2024-03-23T09:45:43.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.572+0000 7efc45ffb640 1 -- 172.21.15.47:0/3257877285 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc48018070 con 0x7efc5015c990 2024-03-23T09:45:43.572 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:45:43.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.574+0000 7efc5782c640 1 -- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efc3406ac00 msgr2=0x7efc3406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.574+0000 7efc5782c640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efc3406ac00 0x7efc3406d0c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efc4c036000 tx=0x7efc4c03d040 comp rx=0 tx=0).stop 2024-03-23T09:45:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.574+0000 7efc5782c640 1 -- 172.21.15.47:0/3257877285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5015c990 msgr2=0x7efc500c7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.574+0000 7efc5782c640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5015c990 0x7efc500c7940 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efc4800c7e0 tx=0x7efc4800cca0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.574+0000 7efc5782c640 1 -- 172.21.15.47:0/3257877285 shutdown_connections 2024-03-23T09:45:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.574+0000 7efc5782c640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efc3406ac00 0x7efc3406d0c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.574+0000 7efc5782c640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5015c990 0x7efc500c7940 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.574+0000 7efc5782c640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc500cdc40 0x7efc500c7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.574+0000 7efc5782c640 1 --2- 172.21.15.47:0/3257877285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc500b8350 0x7efc50156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.574+0000 7efc5782c640 1 -- 172.21.15.47:0/3257877285 >> 172.21.15.47:0/3257877285 conn(0x7efc500659d0 msgr2=0x7efc500d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.574+0000 7efc5782c640 1 -- 172.21.15.47:0/3257877285 shutdown_connections 2024-03-23T09:45:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.574+0000 7efc5782c640 1 -- 172.21.15.47:0/3257877285 wait complete. 2024-03-23T09:45:43.585 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:45:43.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.688+0000 7f3a839ae640 1 Processor -- start 2024-03-23T09:45:43.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.688+0000 7f3a839ae640 1 -- start start 2024-03-23T09:45:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.689+0000 7f3a839ae640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a7c15cc40 0x7f3a7c0b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.689+0000 7f3a839ae640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7c0ce9c0 con 0x7f3a7c0cf9b0 2024-03-23T09:45:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.689+0000 7f3a839ae640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7c06b5c0 con 0x7f3a7c06b000 2024-03-23T09:45:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.689+0000 7f3a839ae640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c0cebf0 con 0x7f3a7c15cc40 2024-03-23T09:45:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.689+0000 7f3a81f4b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3a7c0cf9b0 0x7f3a7c098a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58766/0 (socket says 172.21.15.47:58766) 2024-03-23T09:45:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.689+0000 7f3a81f4b640 1 -- 172.21.15.47:0/1344604787 learned_addr learned my addr 172.21.15.47:0/1344604787 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.689+0000 7f3a8174a640 1 --2- 172.21.15.47:0/1344604787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a7c15cc40 0x7f3a7c0b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.689+0000 7f3a7bfff640 1 -- 172.21.15.47:0/1344604787 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2548312598 0 0) 0x7f3a7c0ce9c0 con 0x7f3a7c0cf9b0 2024-03-23T09:45:43.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.689+0000 7f3a7bfff640 1 -- 172.21.15.47:0/1344604787 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a64003520 con 0x7f3a7c0cf9b0 2024-03-23T09:45:43.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.690+0000 7f3a7bfff640 1 -- 172.21.15.47:0/1344604787 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324964801 0 0) 0x7f3a7c06b5c0 con 0x7f3a7c06b000 2024-03-23T09:45:43.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.690+0000 7f3a7bfff640 1 -- 172.21.15.47:0/1344604787 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a7c0ce9c0 con 0x7f3a7c06b000 2024-03-23T09:45:43.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.690+0000 7f3a7bfff640 1 -- 172.21.15.47:0/1344604787 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3460675825 0 0) 0x7f3a64003520 con 0x7f3a7c0cf9b0 2024-03-23T09:45:43.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.690+0000 7f3a8174a640 1 -- 172.21.15.47:0/1344604787 >> v1:172.21.15.175:6789/0 conn(0x7f3a7c06b000 legacy=0x7f3a7c0caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.690+0000 7f3a8174a640 1 -- 172.21.15.47:0/1344604787 >> v1:172.21.15.47:6789/0 conn(0x7f3a7c0cf9b0 legacy=0x7f3a7c098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.690+0000 7f3a8174a640 1 -- 172.21.15.47:0/1344604787 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c0ceed0 con 0x7f3a7c15cc40 2024-03-23T09:45:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.690+0000 7f3a8174a640 1 --2- 172.21.15.47:0/1344604787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a7c15cc40 0x7f3a7c0b1780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a740097d0 tx=0x7f3a7402b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27ed5f61826d22f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.691+0000 7f3a7bfff640 1 -- 172.21.15.47:0/1344604787 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0019f0 con 0x7f3a7c15cc40 2024-03-23T09:45:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.691+0000 7f3a7bfff640 1 -- 172.21.15.47:0/1344604787 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7402e050 con 0x7f3a7c15cc40 2024-03-23T09:45:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.691+0000 7f3a7bfff640 1 -- 172.21.15.47:0/1344604787 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74030710 con 0x7f3a7c15cc40 2024-03-23T09:45:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.691+0000 7f3a839ae640 1 -- 172.21.15.47:0/1344604787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a7c15cc40 msgr2=0x7f3a7c0b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.691+0000 7f3a839ae640 1 --2- 172.21.15.47:0/1344604787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a7c15cc40 0x7f3a7c0b1780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a740097d0 tx=0x7f3a7402b800 comp rx=0 tx=0).stop 2024-03-23T09:45:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.691+0000 7f3a839ae640 1 -- 172.21.15.47:0/1344604787 shutdown_connections 2024-03-23T09:45:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.691+0000 7f3a839ae640 1 --2- 172.21.15.47:0/1344604787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a7c15cc40 0x7f3a7c0b1780 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.691+0000 7f3a839ae640 1 -- 172.21.15.47:0/1344604787 >> 172.21.15.47:0/1344604787 conn(0x7f3a7c0b2140 msgr2=0x7f3a7c0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.691+0000 7f3a839ae640 1 -- 172.21.15.47:0/1344604787 shutdown_connections 2024-03-23T09:45:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.691+0000 7f3a839ae640 1 -- 172.21.15.47:0/1344604787 wait complete. 2024-03-23T09:45:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.693+0000 7f3a839ae640 1 Processor -- start 2024-03-23T09:45:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.693+0000 7f3a839ae640 1 -- start start 2024-03-23T09:45:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.693+0000 7f3a839ae640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a7c06b000 0x7f3a7c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.693+0000 7f3a839ae640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a7c0cf9b0 0x7f3a7c0ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.693+0000 7f3a839ae640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a7c15cc40 0x7f3a7c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.693+0000 7f3a839ae640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c15a1c0 con 0x7f3a7c06b000 2024-03-23T09:45:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.693+0000 7f3a839ae640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c0c4180 con 0x7f3a7c15cc40 2024-03-23T09:45:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.693+0000 7f3a839ae640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c0c4fe0 con 0x7f3a7c0cf9b0 2024-03-23T09:45:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.693+0000 7f3a8174a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a7c06b000 0x7f3a7c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.693+0000 7f3a8174a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a7c06b000 0x7f3a7c159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49934/0 (socket says 172.21.15.47:49934) 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.693+0000 7f3a8174a640 1 -- 172.21.15.47:0/1562604709 learned_addr learned my addr 172.21.15.47:0/1562604709 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a81f4b640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a7c15cc40 0x7f3a7c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a80f49640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a7c0cf9b0 0x7f3a7c0ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a8174a640 1 -- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a7c0cf9b0 msgr2=0x7f3a7c0ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a8174a640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a7c0cf9b0 0x7f3a7c0ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a8174a640 1 -- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a7c15cc40 msgr2=0x7f3a7c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a8174a640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a7c15cc40 0x7f3a7c06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a8174a640 1 -- 172.21.15.47:0/1562604709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a74009400 con 0x7f3a7c06b000 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a80f49640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a7c0cf9b0 0x7f3a7c0ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a81f4b640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a7c15cc40 0x7f3a7c06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a8174a640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a7c06b000 0x7f3a7c159c80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3a7402b840 tx=0x7f3a740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a79ffb640 1 -- 172.21.15.47:0/1562604709 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7403c620 con 0x7f3a7c06b000 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a79ffb640 1 -- 172.21.15.47:0/1562604709 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7403c7a0 con 0x7f3a7c06b000 2024-03-23T09:45:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a839ae640 1 -- 172.21.15.47:0/1562604709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c0ca610 con 0x7f3a7c06b000 2024-03-23T09:45:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a79ffb640 1 -- 172.21.15.47:0/1562604709 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7403cb50 con 0x7f3a7c06b000 2024-03-23T09:45:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.694+0000 7f3a839ae640 1 -- 172.21.15.47:0/1562604709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c0c46b0 con 0x7f3a7c06b000 2024-03-23T09:45:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.695+0000 7f3a839ae640 1 -- 172.21.15.47:0/1562604709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c066be0 con 0x7f3a7c06b000 2024-03-23T09:45:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.696+0000 7f3a79ffb640 1 -- 172.21.15.47:0/1562604709 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3a74039070 con 0x7f3a7c06b000 2024-03-23T09:45:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.696+0000 7f3a79ffb640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a6806ac00 0x7f3a6806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.696+0000 7f3a79ffb640 1 -- 172.21.15.47:0/1562604709 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3a740aaaf0 con 0x7f3a7c06b000 2024-03-23T09:45:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.696+0000 7f3a80f49640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a6806ac00 0x7f3a6806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.697+0000 7f3a80f49640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a6806ac00 0x7f3a6806d0c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0c9610 tx=0x7f3a70009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.701+0000 7f3a79ffb640 1 -- 172.21.15.47:0/1562604709 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3a74075c70 con 0x7f3a7c06b000 2024-03-23T09:45:43.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.884+0000 7f3a839ae640 1 -- 172.21.15.47:0/1562604709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a7c0b2fe0 con 0x7f3a7c06b000 2024-03-23T09:45:43.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.884+0000 7f3a79ffb640 1 -- 172.21.15.47:0/1562604709 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a7404a020 con 0x7f3a7c06b000 2024-03-23T09:45:43.884 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:45:43.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.886+0000 7f3a839ae640 1 -- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a6806ac00 msgr2=0x7f3a6806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.886+0000 7f3a839ae640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a6806ac00 0x7f3a6806d0c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0c9610 tx=0x7f3a70009040 comp rx=0 tx=0).stop 2024-03-23T09:45:43.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.886+0000 7f3a839ae640 1 -- 172.21.15.47:0/1562604709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a7c06b000 msgr2=0x7f3a7c159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:43.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.886+0000 7f3a839ae640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a7c06b000 0x7f3a7c159c80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3a7402b840 tx=0x7f3a740046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.886+0000 7f3a839ae640 1 -- 172.21.15.47:0/1562604709 shutdown_connections 2024-03-23T09:45:43.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.886+0000 7f3a839ae640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3a6806ac00 0x7f3a6806d0c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.886+0000 7f3a839ae640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a7c15cc40 0x7f3a7c06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.886+0000 7f3a839ae640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a7c0cf9b0 0x7f3a7c0ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.886+0000 7f3a839ae640 1 --2- 172.21.15.47:0/1562604709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a7c06b000 0x7f3a7c159c80 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:43.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.886+0000 7f3a839ae640 1 -- 172.21.15.47:0/1562604709 >> 172.21.15.47:0/1562604709 conn(0x7f3a7c0b2140 msgr2=0x7f3a7c15add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:43.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.887+0000 7f3a839ae640 1 -- 172.21.15.47:0/1562604709 shutdown_connections 2024-03-23T09:45:43.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:43.887+0000 7f3a839ae640 1 -- 172.21.15.47:0/1562604709 wait complete. 2024-03-23T09:45:43.897 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:45:44.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.001+0000 7f85e3a42640 1 Processor -- start 2024-03-23T09:45:44.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.001+0000 7f85e3a42640 1 -- start start 2024-03-23T09:45:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85e3a42640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc0b8350 0x7f85dc160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85e3a42640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85dc161630 con 0x7f85dc15c990 2024-03-23T09:45:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85e3a42640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85dc0d1030 con 0x7f85dc0cde10 2024-03-23T09:45:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85e3a42640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0cfe10 con 0x7f85dc0b8350 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85e17de640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f85dc15c990 0x7f85dc0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58772/0 (socket says 172.21.15.47:58772) 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85e17de640 1 -- 172.21.15.47:0/2678859607 learned_addr learned my addr 172.21.15.47:0/2678859607 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790180838 0 0) 0x7f85dc161630 con 0x7f85dc15c990 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85c8003520 con 0x7f85dc15c990 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85e1fdf640 1 --2- 172.21.15.47:0/2678859607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc0b8350 0x7f85dc160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 185744400 0 0) 0x7f85c8003520 con 0x7f85dc15c990 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc0b8350 msgr2=0x7f85dc160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 --2- 172.21.15.47:0/2678859607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc0b8350 0x7f85dc160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 >> v1:172.21.15.175:6789/0 conn(0x7f85dc0cde10 legacy=0x7f85dc0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc0b5b60 con 0x7f85dc15c990 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f85d8001670 con 0x7f85dc15c990 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 >> v1:172.21.15.47:6789/0 conn(0x7f85dc15c990 legacy=0x7f85dc0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 --2- 172.21.15.47:0/2678859607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c80039a0 0x7f85c8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 --2- 172.21.15.47:0/2678859607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc0b8350 0x7f85c80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 --2- 172.21.15.47:0/2678859607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85c8004d00 0x7f85c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0cfe10 con 0x7f85c8004d00 2024-03-23T09:45:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80142c0 con 0x7f85c80039a0 2024-03-23T09:45:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.002+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8014480 con 0x7f85dc0b8350 2024-03-23T09:45:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.003+0000 7f85e1fdf640 1 --2- 172.21.15.47:0/2678859607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85c8004d00 0x7f85c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.003+0000 7f85e0fdd640 1 --2- 172.21.15.47:0/2678859607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc0b8350 0x7f85c80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.003+0000 7f85e17de640 1 --2- 172.21.15.47:0/2678859607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c80039a0 0x7f85c8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.003+0000 7f85e1fdf640 1 -- 172.21.15.47:0/2678859607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc0b8350 msgr2=0x7f85c80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.003+0000 7f85e1fdf640 1 --2- 172.21.15.47:0/2678859607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc0b8350 0x7f85c80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.003+0000 7f85e1fdf640 1 -- 172.21.15.47:0/2678859607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c80039a0 msgr2=0x7f85c8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.003+0000 7f85e1fdf640 1 --2- 172.21.15.47:0/2678859607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c80039a0 0x7f85c8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.003+0000 7f85e1fdf640 1 -- 172.21.15.47:0/2678859607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c8003520 con 0x7f85c8004d00 2024-03-23T09:45:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.004+0000 7f85e1fdf640 1 --2- 172.21.15.47:0/2678859607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85c8004d00 0x7f85c8005180 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009fc0 tx=0x7f85c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70a6c635e7d5a45 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.004+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d402cc20 con 0x7f85c8004d00 2024-03-23T09:45:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.004+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d400f040 con 0x7f85c8004d00 2024-03-23T09:45:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.004+0000 7f85d3fff640 1 -- 172.21.15.47:0/2678859607 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4034440 con 0x7f85c8004d00 2024-03-23T09:45:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.004+0000 7f85e3a42640 1 -- 172.21.15.47:0/2678859607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85c8004d00 msgr2=0x7f85c8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.004+0000 7f85e3a42640 1 --2- 172.21.15.47:0/2678859607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85c8004d00 0x7f85c8005180 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009fc0 tx=0x7f85c80057f0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.004+0000 7f85e1fdf640 1 -- 172.21.15.47:0/2678859607 reap_dead start 2024-03-23T09:45:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.004+0000 7f85e3a42640 1 -- 172.21.15.47:0/2678859607 shutdown_connections 2024-03-23T09:45:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.004+0000 7f85e3a42640 1 -- 172.21.15.47:0/2678859607 >> 172.21.15.47:0/2678859607 conn(0x7f85dc0659d0 msgr2=0x7f85dc065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.004+0000 7f85e3a42640 1 -- 172.21.15.47:0/2678859607 shutdown_connections 2024-03-23T09:45:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.004+0000 7f85e3a42640 1 -- 172.21.15.47:0/2678859607 wait complete. 2024-03-23T09:45:44.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.006+0000 7f85e3a42640 1 Processor -- start 2024-03-23T09:45:44.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.006+0000 7f85e3a42640 1 -- start start 2024-03-23T09:45:44.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.006+0000 7f85e3a42640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c8004d00 0x7f85dc14fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.006+0000 7f85e3a42640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc0c8720 0x7f85dc0c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.006+0000 7f85e3a42640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85dc151f50 0x7f85dc154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.006+0000 7f85e3a42640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0c8300 con 0x7f85dc151f50 2024-03-23T09:45:44.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.006+0000 7f85e3a42640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0c7680 con 0x7f85c8004d00 2024-03-23T09:45:44.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.006+0000 7f85e3a42640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0c7960 con 0x7f85dc0c8720 2024-03-23T09:45:44.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.006+0000 7f85e1fdf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85dc151f50 0x7f85dc154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.006+0000 7f85e1fdf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85dc151f50 0x7f85dc154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49960/0 (socket says 172.21.15.47:49960) 2024-03-23T09:45:44.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.006+0000 7f85e1fdf640 1 -- 172.21.15.47:0/4190351442 learned_addr learned my addr 172.21.15.47:0/4190351442 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:44.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.007+0000 7f85e0fdd640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc0c8720 0x7f85dc0c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.007+0000 7f85e1fdf640 1 -- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc0c8720 msgr2=0x7f85dc0c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.007+0000 7f85e1fdf640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc0c8720 0x7f85dc0c6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.007+0000 7f85e17de640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c8004d00 0x7f85dc14fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.007+0000 7f85e1fdf640 1 -- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c8004d00 msgr2=0x7f85dc14fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.007+0000 7f85e1fdf640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c8004d00 0x7f85dc14fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.007+0000 7f85e1fdf640 1 -- 172.21.15.47:0/4190351442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc0c7c40 con 0x7f85dc151f50 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.007+0000 7f85e1fdf640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85dc151f50 0x7f85dc154d40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f85d4002230 tx=0x7f85d4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.007+0000 7f85d1ffb640 1 -- 172.21.15.47:0/4190351442 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4043070 con 0x7f85dc151f50 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.007+0000 7f85d1ffb640 1 -- 172.21.15.47:0/4190351442 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d4034ad0 con 0x7f85dc151f50 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.007+0000 7f85e3a42640 1 -- 172.21.15.47:0/4190351442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85dc0c7f20 con 0x7f85dc151f50 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.007+0000 7f85d1ffb640 1 -- 172.21.15.47:0/4190351442 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4034e80 con 0x7f85dc151f50 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.007+0000 7f85e3a42640 1 -- 172.21.15.47:0/4190351442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85dc0c72d0 con 0x7f85dc151f50 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.008+0000 7f85e3a42640 1 -- 172.21.15.47:0/4190351442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85dc072360 con 0x7f85dc151f50 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.008+0000 7f85d1ffb640 1 -- 172.21.15.47:0/4190351442 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f85d403e440 con 0x7f85dc151f50 2024-03-23T09:45:44.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.009+0000 7f85d1ffb640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f85c006ac00 0x7f85c006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.009+0000 7f85d1ffb640 1 -- 172.21.15.47:0/4190351442 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f85d40aafd0 con 0x7f85dc151f50 2024-03-23T09:45:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.009+0000 7f85e17de640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f85c006ac00 0x7f85c006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.010+0000 7f85e17de640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f85c006ac00 0x7f85c006d0c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f85d8000c00 tx=0x7f85d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.013+0000 7f85d1ffb640 1 -- 172.21.15.47:0/4190351442 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f85d40761e0 con 0x7f85dc151f50 2024-03-23T09:45:44.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.192+0000 7f85e3a42640 1 -- 172.21.15.47:0/4190351442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f85dc0c8440 con 0x7f85dc151f50 2024-03-23T09:45:44.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.192+0000 7f85d1ffb640 1 -- 172.21.15.47:0/4190351442 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f85dc0c8440 con 0x7f85dc151f50 2024-03-23T09:45:44.193 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:45:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.194+0000 7f85e3a42640 1 -- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f85c006ac00 msgr2=0x7f85c006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.194+0000 7f85e3a42640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f85c006ac00 0x7f85c006d0c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f85d8000c00 tx=0x7f85d8009040 comp rx=0 tx=0).stop 2024-03-23T09:45:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.194+0000 7f85e3a42640 1 -- 172.21.15.47:0/4190351442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85dc151f50 msgr2=0x7f85dc154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.194+0000 7f85e3a42640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85dc151f50 0x7f85dc154d40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f85d4002230 tx=0x7f85d4009750 comp rx=0 tx=0).stop 2024-03-23T09:45:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.194+0000 7f85e3a42640 1 -- 172.21.15.47:0/4190351442 shutdown_connections 2024-03-23T09:45:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.194+0000 7f85e3a42640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f85c006ac00 0x7f85c006d0c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.194+0000 7f85e3a42640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85dc151f50 0x7f85dc154d40 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.194+0000 7f85e3a42640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc0c8720 0x7f85dc0c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.194+0000 7f85e3a42640 1 --2- 172.21.15.47:0/4190351442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c8004d00 0x7f85dc14fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.195+0000 7f85e3a42640 1 -- 172.21.15.47:0/4190351442 >> 172.21.15.47:0/4190351442 conn(0x7f85dc0659d0 msgr2=0x7f85dc0b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.195+0000 7f85e3a42640 1 -- 172.21.15.47:0/4190351442 shutdown_connections 2024-03-23T09:45:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.195+0000 7f85e3a42640 1 -- 172.21.15.47:0/4190351442 wait complete. 2024-03-23T09:45:44.205 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:45:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.306+0000 7fd34db3c640 1 Processor -- start 2024-03-23T09:45:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.306+0000 7fd34db3c640 1 -- start start 2024-03-23T09:45:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.307+0000 7fd34db3c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd34815cd00 0x7fd3480d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.307+0000 7fd34db3c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd3480d1020 0x7fd3480cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.307+0000 7fd34db3c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3480cfba0 con 0x7fd3480cf750 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.307+0000 7fd34db3c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34815d0e0 con 0x7fd34815cd00 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.307+0000 7fd34db3c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3480d1540 con 0x7fd3480d1020 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.307+0000 7fd3477fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd34815cd00 0x7fd3480d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.307+0000 7fd3477fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd34815cd00 0x7fd3480d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49964/0 (socket says 172.21.15.47:49964) 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.307+0000 7fd3477fe640 1 -- 172.21.15.47:0/2008703237 learned_addr learned my addr 172.21.15.47:0/2008703237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.308+0000 7fd346ffd640 1 --2- 172.21.15.47:0/2008703237 >> v2:172.21.15.175:3300/0 conn(0x7fd3480d1020 0x7fd3480cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.308+0000 7fd3477fe640 1 -- 172.21.15.47:0/2008703237 >> v2:172.21.15.175:3300/0 conn(0x7fd3480d1020 msgr2=0x7fd3480cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.308+0000 7fd3477fe640 1 --2- 172.21.15.47:0/2008703237 >> v2:172.21.15.175:3300/0 conn(0x7fd3480d1020 0x7fd3480cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.308+0000 7fd3477fe640 1 -- 172.21.15.47:0/2008703237 >> v1:172.21.15.175:6789/0 conn(0x7fd3480cf750 legacy=0x7fd3480b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.308+0000 7fd3477fe640 1 -- 172.21.15.47:0/2008703237 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3480d1820 con 0x7fd34815cd00 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.308+0000 7fd3477fe640 1 --2- 172.21.15.47:0/2008703237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34815cd00 0x7fd3480d0ae0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fd34806c1a0 tx=0x7fd33802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d027d0258b08c158 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.309+0000 7fd3467fc640 1 -- 172.21.15.47:0/2008703237 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3380317c0 con 0x7fd34815cd00 2024-03-23T09:45:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.309+0000 7fd3467fc640 1 -- 172.21.15.47:0/2008703237 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd338031920 con 0x7fd34815cd00 2024-03-23T09:45:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.309+0000 7fd3467fc640 1 -- 172.21.15.47:0/2008703237 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338031cb0 con 0x7fd34815cd00 2024-03-23T09:45:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.309+0000 7fd34db3c640 1 -- 172.21.15.47:0/2008703237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34815cd00 msgr2=0x7fd3480d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.309+0000 7fd34db3c640 1 --2- 172.21.15.47:0/2008703237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34815cd00 0x7fd3480d0ae0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fd34806c1a0 tx=0x7fd33802b860 comp rx=0 tx=0).stop 2024-03-23T09:45:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.309+0000 7fd34db3c640 1 -- 172.21.15.47:0/2008703237 shutdown_connections 2024-03-23T09:45:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.309+0000 7fd34db3c640 1 --2- 172.21.15.47:0/2008703237 >> v2:172.21.15.175:3300/0 conn(0x7fd3480d1020 0x7fd3480cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.309+0000 7fd34db3c640 1 --2- 172.21.15.47:0/2008703237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34815cd00 0x7fd3480d0ae0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.309+0000 7fd34db3c640 1 -- 172.21.15.47:0/2008703237 >> 172.21.15.47:0/2008703237 conn(0x7fd34806b160 msgr2=0x7fd34806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.309+0000 7fd34db3c640 1 -- 172.21.15.47:0/2008703237 shutdown_connections 2024-03-23T09:45:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.309+0000 7fd34db3c640 1 -- 172.21.15.47:0/2008703237 wait complete. 2024-03-23T09:45:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.310+0000 7fd34db3c640 1 Processor -- start 2024-03-23T09:45:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.311+0000 7fd34db3c640 1 -- start start 2024-03-23T09:45:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.311+0000 7fd34db3c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3480cf750 0x7fd3480b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.311+0000 7fd34db3c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3480d1020 0x7fd3480a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.311+0000 7fd34db3c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34815cd00 0x7fd34814beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.311+0000 7fd34db3c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34814e630 con 0x7fd34815cd00 2024-03-23T09:45:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.311+0000 7fd34db3c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3480b5360 con 0x7fd3480cf750 2024-03-23T09:45:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.311+0000 7fd34db3c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3480a98e0 con 0x7fd3480d1020 2024-03-23T09:45:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.311+0000 7fd347fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34815cd00 0x7fd34814beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.311+0000 7fd347fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34815cd00 0x7fd34814beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49980/0 (socket says 172.21.15.47:49980) 2024-03-23T09:45:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.311+0000 7fd347fff640 1 -- 172.21.15.47:0/2691895142 learned_addr learned my addr 172.21.15.47:0/2691895142 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd346ffd640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3480d1020 0x7fd3480a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd3477fe640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3480cf750 0x7fd3480b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd347fff640 1 -- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3480d1020 msgr2=0x7fd3480a93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd347fff640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3480d1020 0x7fd3480a93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd347fff640 1 -- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3480cf750 msgr2=0x7fd3480b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd347fff640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3480cf750 0x7fd3480b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd347fff640 1 -- 172.21.15.47:0/2691895142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3380094d0 con 0x7fd34815cd00 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd346ffd640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3480d1020 0x7fd3480a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd347fff640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34815cd00 0x7fd34814beb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd34000c780 tx=0x7fd34000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd32ffff640 1 -- 172.21.15.47:0/2691895142 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd340004b60 con 0x7fd34815cd00 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd32ffff640 1 -- 172.21.15.47:0/2691895142 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34000ce90 con 0x7fd34815cd00 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd34db3c640 1 -- 172.21.15.47:0/2691895142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34814c3f0 con 0x7fd34815cd00 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd32ffff640 1 -- 172.21.15.47:0/2691895142 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34000a7f0 con 0x7fd34815cd00 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.312+0000 7fd34db3c640 1 -- 172.21.15.47:0/2691895142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34814dc90 con 0x7fd34815cd00 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.313+0000 7fd34db3c640 1 -- 172.21.15.47:0/2691895142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd348072360 con 0x7fd34815cd00 2024-03-23T09:45:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.313+0000 7fd32ffff640 1 -- 172.21.15.47:0/2691895142 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd34000a950 con 0x7fd34815cd00 2024-03-23T09:45:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.314+0000 7fd32ffff640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd33406ac00 0x7fd33406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.314+0000 7fd32ffff640 1 -- 172.21.15.47:0/2691895142 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd340089460 con 0x7fd34815cd00 2024-03-23T09:45:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.314+0000 7fd3477fe640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd33406ac00 0x7fd33406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.314+0000 7fd3477fe640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd33406ac00 0x7fd33406d0c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd338036000 tx=0x7fd33803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:44.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.318+0000 7fd32ffff640 1 -- 172.21.15.47:0/2691895142 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd340054670 con 0x7fd34815cd00 2024-03-23T09:45:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.508+0000 7fd34db3c640 1 -- 172.21.15.47:0/2691895142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd348066be0 con 0x7fd34815cd00 2024-03-23T09:45:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.509+0000 7fd32ffff640 1 -- 172.21.15.47:0/2691895142 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v71) v1 ==== 80+0+120 (secure 0 0 0) 0x7fd340057c90 con 0x7fd34815cd00 2024-03-23T09:45:44.509 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6835/372771687 2024-03-24T09:45:33.740376+0000 2024-03-23T09:45:44.509 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6833/372771687 2024-03-24T09:45:33.740376+0000 2024-03-23T09:45:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:listed 2 entries 2024-03-23T09:45:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.511+0000 7fd34db3c640 1 -- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd33406ac00 msgr2=0x7fd33406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.511+0000 7fd34db3c640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd33406ac00 0x7fd33406d0c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd338036000 tx=0x7fd33803d040 comp rx=0 tx=0).stop 2024-03-23T09:45:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.511+0000 7fd34db3c640 1 -- 172.21.15.47:0/2691895142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34815cd00 msgr2=0x7fd34814beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.511+0000 7fd34db3c640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34815cd00 0x7fd34814beb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd34000c780 tx=0x7fd34000cc40 comp rx=0 tx=0).stop 2024-03-23T09:45:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.511+0000 7fd34db3c640 1 -- 172.21.15.47:0/2691895142 shutdown_connections 2024-03-23T09:45:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.511+0000 7fd34db3c640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd33406ac00 0x7fd33406d0c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.511+0000 7fd34db3c640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34815cd00 0x7fd34814beb0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.511+0000 7fd34db3c640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3480d1020 0x7fd3480a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.511+0000 7fd34db3c640 1 --2- 172.21.15.47:0/2691895142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3480cf750 0x7fd3480b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.511+0000 7fd34db3c640 1 -- 172.21.15.47:0/2691895142 >> 172.21.15.47:0/2691895142 conn(0x7fd34806b160 msgr2=0x7fd3480c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.511+0000 7fd34db3c640 1 -- 172.21.15.47:0/2691895142 shutdown_connections 2024-03-23T09:45:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:44.511+0000 7fd34db3c640 1 -- 172.21.15.47:0/2691895142 wait complete. 2024-03-23T09:45:44.521 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:45:44.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:44.521 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:45:44.546 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[62907]: fuse finished with error 0 and tester_r 0 2024-03-23T09:45:44.548 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:45:44.819 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:45:44.819+0000 7fefe5743640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T09:45:50.649 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:45:50.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:50.649 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:45:50.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:50.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-03-23T09:45:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.826+0000 7f790bc84640 1 Processor -- start 2024-03-23T09:45:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.826+0000 7f790bc84640 1 -- start start 2024-03-23T09:45:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f790bc84640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f79040cde40 0x7f79040b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f790bc84640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7904160780 0x7f79040d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f790bc84640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79040b5a90 con 0x7f790415cb50 2024-03-23T09:45:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f790bc84640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7904161790 con 0x7f7904160780 2024-03-23T09:45:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f790bc84640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7904160ef0 con 0x7f79040cde40 2024-03-23T09:45:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f7909a20640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f790415cb50 0x7f79040cd730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34542/0 (socket says 172.21.15.47:34542) 2024-03-23T09:45:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f7909a20640 1 -- 172.21.15.47:0/3945090231 learned_addr learned my addr 172.21.15.47:0/3945090231 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f790a221640 1 --2- 172.21.15.47:0/3945090231 >> v2:172.21.15.47:3300/0 conn(0x7f7904160780 0x7f79040d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f790921f640 1 --2- 172.21.15.47:0/3945090231 >> v2:172.21.15.175:3300/0 conn(0x7f79040cde40 0x7f79040b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f790a221640 1 -- 172.21.15.47:0/3945090231 >> v2:172.21.15.175:3300/0 conn(0x7f79040cde40 msgr2=0x7f79040b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f790a221640 1 --2- 172.21.15.47:0/3945090231 >> v2:172.21.15.175:3300/0 conn(0x7f79040cde40 0x7f79040b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f790a221640 1 -- 172.21.15.47:0/3945090231 >> v1:172.21.15.47:6789/0 conn(0x7f790415cb50 legacy=0x7f79040cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f790a221640 1 -- 172.21.15.47:0/3945090231 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79041611d0 con 0x7f7904160780 2024-03-23T09:45:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.827+0000 7f790921f640 1 --2- 172.21.15.47:0/3945090231 >> v2:172.21.15.175:3300/0 conn(0x7f79040cde40 0x7f79040b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.828+0000 7f790a221640 1 --2- 172.21.15.47:0/3945090231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7904160780 0x7f79040d1490 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f79040d39a0 tx=0x7f78fc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e68e08f75405d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.828+0000 7f7908a1e640 1 -- 172.21.15.47:0/3945090231 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc02cc80 con 0x7f7904160780 2024-03-23T09:45:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.828+0000 7f7908a1e640 1 -- 172.21.15.47:0/3945090231 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc02cde0 con 0x7f7904160780 2024-03-23T09:45:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.828+0000 7f7908a1e640 1 -- 172.21.15.47:0/3945090231 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc034660 con 0x7f7904160780 2024-03-23T09:45:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.829+0000 7f790bc84640 1 -- 172.21.15.47:0/3945090231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7904160780 msgr2=0x7f79040d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.829+0000 7f790bc84640 1 --2- 172.21.15.47:0/3945090231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7904160780 0x7f79040d1490 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f79040d39a0 tx=0x7f78fc02bed0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.829+0000 7f790bc84640 1 -- 172.21.15.47:0/3945090231 shutdown_connections 2024-03-23T09:45:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.829+0000 7f790bc84640 1 --2- 172.21.15.47:0/3945090231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7904160780 0x7f79040d1490 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.829+0000 7f790bc84640 1 --2- 172.21.15.47:0/3945090231 >> v2:172.21.15.175:3300/0 conn(0x7f79040cde40 0x7f79040b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.829+0000 7f790bc84640 1 -- 172.21.15.47:0/3945090231 >> 172.21.15.47:0/3945090231 conn(0x7f790406b160 msgr2=0x7f790406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.829+0000 7f790bc84640 1 -- 172.21.15.47:0/3945090231 shutdown_connections 2024-03-23T09:45:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.829+0000 7f790bc84640 1 -- 172.21.15.47:0/3945090231 wait complete. 2024-03-23T09:45:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.830+0000 7f790bc84640 1 Processor -- start 2024-03-23T09:45:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.830+0000 7f790bc84640 1 -- start start 2024-03-23T09:45:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790bc84640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79040cde40 0x7f79040cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790bc84640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f790415cb50 0x7f7904150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790bc84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7904160780 0x7f79040cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790bc84640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79041508d0 con 0x7f7904160780 2024-03-23T09:45:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790bc84640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79040c9b60 con 0x7f790415cb50 2024-03-23T09:45:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790bc84640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79040c9e40 con 0x7f79040cde40 2024-03-23T09:45:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790a221640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7904160780 0x7f79040cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790a221640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7904160780 0x7f79040cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42656/0 (socket says 172.21.15.47:42656) 2024-03-23T09:45:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790a221640 1 -- 172.21.15.47:0/1656943664 learned_addr learned my addr 172.21.15.47:0/1656943664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f7909a20640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79040cde40 0x7f79040cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790921f640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f790415cb50 0x7f7904150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790a221640 1 -- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79040cde40 msgr2=0x7f79040cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790a221640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79040cde40 0x7f79040cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790a221640 1 -- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f790415cb50 msgr2=0x7f7904150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790a221640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f790415cb50 0x7f7904150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790a221640 1 -- 172.21.15.47:0/1656943664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc009a70 con 0x7f7904160780 2024-03-23T09:45:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f7909a20640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79040cde40 0x7f79040cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.831+0000 7f790a221640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7904160780 0x7f79040cb6a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009e80 tx=0x7f78fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.833+0000 7f78fa7fc640 1 -- 172.21.15.47:0/1656943664 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc040650 con 0x7f7904160780 2024-03-23T09:45:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.833+0000 7f78fa7fc640 1 -- 172.21.15.47:0/1656943664 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc0407b0 con 0x7f7904160780 2024-03-23T09:45:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.833+0000 7f790bc84640 1 -- 172.21.15.47:0/1656943664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79040ca120 con 0x7f7904160780 2024-03-23T09:45:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.833+0000 7f78fa7fc640 1 -- 172.21.15.47:0/1656943664 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc040b60 con 0x7f7904160780 2024-03-23T09:45:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.833+0000 7f790bc84640 1 -- 172.21.15.47:0/1656943664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79040ca6e0 con 0x7f7904160780 2024-03-23T09:45:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.834+0000 7f78fa7fc640 1 -- 172.21.15.47:0/1656943664 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f78fc040cc0 con 0x7f7904160780 2024-03-23T09:45:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.835+0000 7f78fa7fc640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f78e806ac00 0x7f78e806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.835+0000 7f78fa7fc640 1 -- 172.21.15.47:0/1656943664 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f78fc0ac0c0 con 0x7f7904160780 2024-03-23T09:45:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.835+0000 7f7909a20640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f78e806ac00 0x7f78e806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.836+0000 7f7909a20640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f78e806ac00 0x7f78e806d0c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7900005880 tx=0x7f7900009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.836+0000 7f790e7f7740 1 Processor -- start 2024-03-23T09:45:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.836+0000 7f790e7f7740 1 -- start start 2024-03-23T09:45:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790e7f7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb610 0x55b1d31be840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790e7f7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 0x55b1d31ee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790e7f7740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31bed80 0x55b1d31ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790e7f7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d312ee60 con 0x55b1d31eb9f0 2024-03-23T09:45:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790e7f7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d312c5f0 con 0x55b1d31eb610 2024-03-23T09:45:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790e7f7740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1d3127ee0 con 0x55b1d31bed80 2024-03-23T09:45:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790921f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 0x55b1d31ee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790921f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 0x55b1d31ee3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42672/0 (socket says 172.21.15.47:42672) 2024-03-23T09:45:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790921f640 1 -- 172.21.15.47:0/2290054500 learned_addr learned my addr 172.21.15.47:0/2290054500 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f7909a20640 1 --2- 172.21.15.47:0/2290054500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb610 0x55b1d31be840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790a221640 1 --2- 172.21.15.47:0/2290054500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31bed80 0x55b1d31ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790921f640 1 -- 172.21.15.47:0/2290054500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31bed80 msgr2=0x55b1d31ee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790921f640 1 --2- 172.21.15.47:0/2290054500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31bed80 0x55b1d31ee8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790921f640 1 -- 172.21.15.47:0/2290054500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb610 msgr2=0x55b1d31be840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790921f640 1 --2- 172.21.15.47:0/2290054500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb610 0x55b1d31be840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.837+0000 7f790921f640 1 -- 172.21.15.47:0/2290054500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1d3126560 con 0x55b1d31eb9f0 2024-03-23T09:45:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.838+0000 7f790921f640 1 --2- 172.21.15.47:0/2290054500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 0x55b1d31ee3a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009a80 tx=0x7f78f4009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc8fc2dab26dbaa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.838+0000 7f78e77fe640 1 -- 172.21.15.47:0/2290054500 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4010dc0 con 0x55b1d31eb9f0 2024-03-23T09:45:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.838+0000 7f78e77fe640 1 -- 172.21.15.47:0/2290054500 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f400c050 con 0x55b1d31eb9f0 2024-03-23T09:45:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.838+0000 7f78e77fe640 1 -- 172.21.15.47:0/2290054500 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4015670 con 0x55b1d31eb9f0 2024-03-23T09:45:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.838+0000 7f790e7f7740 1 -- 172.21.15.47:0/2290054500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 msgr2=0x55b1d31ee3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.838+0000 7f790e7f7740 1 --2- 172.21.15.47:0/2290054500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 0x55b1d31ee3a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009a80 tx=0x7f78f4009f40 comp rx=0 tx=0).stop 2024-03-23T09:45:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.839+0000 7f790e7f7740 1 -- 172.21.15.47:0/2290054500 shutdown_connections 2024-03-23T09:45:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.839+0000 7f790e7f7740 1 --2- 172.21.15.47:0/2290054500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31bed80 0x55b1d31ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.839+0000 7f790e7f7740 1 --2- 172.21.15.47:0/2290054500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 0x55b1d31ee3a0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.839+0000 7f790e7f7740 1 --2- 172.21.15.47:0/2290054500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb610 0x55b1d31be840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.839+0000 7f790e7f7740 1 -- 172.21.15.47:0/2290054500 >> 172.21.15.47:0/2290054500 conn(0x55b1d31c0ab0 msgr2=0x55b1d32093a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.839+0000 7f790e7f7740 1 -- 172.21.15.47:0/2290054500 shutdown_connections 2024-03-23T09:45:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.839+0000 7f790e7f7740 1 -- 172.21.15.47:0/2290054500 wait complete. 2024-03-23T09:45:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.839+0000 7f790e7f7740 1 -- start start 2024-03-23T09:45:50.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.839+0000 7f790e7f7740 1 Processor -- start 2024-03-23T09:45:50.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.840+0000 7f790e7f7740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31eb610 0x55b1d32019b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.840+0000 7f790e7f7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb9f0 0x55b1d320ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.840+0000 7f790e7f7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31f0df0 0x55b1d3218d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.840+0000 7f790e7f7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d30e61d0 con 0x55b1d31f0df0 2024-03-23T09:45:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.840+0000 7f790e7f7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d30e5e70 con 0x55b1d31eb9f0 2024-03-23T09:45:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.840+0000 7f790e7f7740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1d30ab0a0 con 0x55b1d31eb610 2024-03-23T09:45:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.840+0000 7f790e7f7740 10 client.0 _ll_register_callbacks cb 0x55b1d3121550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:45:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.840+0000 7f790a221640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31f0df0 0x55b1d3218d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.840+0000 7f790a221640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31f0df0 0x55b1d3218d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42688/0 (socket says 172.21.15.47:42688) 2024-03-23T09:45:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.840+0000 7f790a221640 1 -- 172.21.15.47:0/58945138 learned_addr learned my addr 172.21.15.47:0/58945138 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.840+0000 7f790921f640 1 --2- 172.21.15.47:0/58945138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb9f0 0x55b1d320ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.840+0000 7f7909a20640 1 --2- 172.21.15.47:0/58945138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31eb610 0x55b1d32019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f790a221640 1 -- 172.21.15.47:0/58945138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31eb610 msgr2=0x55b1d32019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f790a221640 1 --2- 172.21.15.47:0/58945138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31eb610 0x55b1d32019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f790a221640 1 -- 172.21.15.47:0/58945138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb9f0 msgr2=0x55b1d320ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f790a221640 1 --2- 172.21.15.47:0/58945138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb9f0 0x55b1d320ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f790a221640 1 -- 172.21.15.47:0/58945138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f4009680 con 0x55b1d31f0df0 2024-03-23T09:45:50.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f790a221640 1 --2- 172.21.15.47:0/58945138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31f0df0 0x55b1d3218d30 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f78fc02bf10 tx=0x7f78fc07b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4923e6b8856095 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:50.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f78e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:45:50.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f78e4ff9640 1 -- 172.21.15.47:0/58945138 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc0834a0 con 0x55b1d31f0df0 2024-03-23T09:45:50.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f78e4ff9640 1 -- 172.21.15.47:0/58945138 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f78fc03cde0 con 0x55b1d31f0df0 2024-03-23T09:45:50.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f790e7f7740 1 -- 172.21.15.47:0/58945138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1d3151720 con 0x55b1d31f0df0 2024-03-23T09:45:50.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f78e4ff9640 1 -- 172.21.15.47:0/58945138 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78fc0777c0 con 0x55b1d31f0df0 2024-03-23T09:45:50.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f790e7f7740 10 client.7158 fetch_fsmap learned FSMap version 39 2024-03-23T09:45:50.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f790e7f7740 1 -- 172.21.15.47:0/58945138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55b1d30c81b0 con 0x55b1d31f0df0 2024-03-23T09:45:50.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f78e4ff9640 1 -- 172.21.15.47:0/58945138 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc087080 con 0x55b1d31f0df0 2024-03-23T09:45:50.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f78e4ff9640 1 -- 172.21.15.47:0/58945138 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f78fc050020 con 0x55b1d31f0df0 2024-03-23T09:45:50.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f790e7f7740 10 client.7158 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T09:45:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f790e7f7740 10 client.7158 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-23T09:45:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f790e7f7740 20 client.7158 populate_metadata read hostname 'smithi047' 2024-03-23T09:45:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.841+0000 7f790e7f7740 1 --2- 172.21.15.47:0/58945138 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55b1d31df910 0x55b1d3202580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.842+0000 7f790e7f7740 4 client.7158 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:45:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.842+0000 7f790e7f7740 1 -- 172.21.15.47:0/58945138 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1d3119b80 con 0x55b1d31df910 2024-03-23T09:45:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.842+0000 7f7909a20640 1 --2- 172.21.15.47:0/58945138 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55b1d31df910 0x55b1d3202580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.842+0000 7f7909a20640 1 --2- 172.21.15.47:0/58945138 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55b1d31df910 0x55b1d3202580 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79da029248577b15 server_cookie=bdd0ded8c06c459f in_seq=0 out_seq=0 2024-03-23T09:45:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.842+0000 7f78e4ff9640 10 client.7158 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:45:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.842+0000 7f78e4ff9640 1 -- 172.21.15.47:0/58945138 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b1d3119b80 con 0x55b1d31df910 2024-03-23T09:45:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.842+0000 7f78e4ff9640 10 client.7158 handle_command_reply: tid=0 2024-03-23T09:45:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.842+0000 7f790e7f7740 1 client.7158 shutdown 2024-03-23T09:45:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.843+0000 7f790e7f7740 1 -- 172.21.15.47:0/58945138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31f0df0 msgr2=0x55b1d3218d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.843+0000 7f790e7f7740 1 --2- 172.21.15.47:0/58945138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31f0df0 0x55b1d3218d30 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f78fc02bf10 tx=0x7f78fc07b870 comp rx=0 tx=0).stop 2024-03-23T09:45:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.843+0000 7f790e7f7740 1 -- 172.21.15.47:0/58945138 shutdown_connections 2024-03-23T09:45:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.843+0000 7f790e7f7740 1 --2- 172.21.15.47:0/58945138 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55b1d31df910 0x55b1d3202580 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.843+0000 7f790e7f7740 1 --2- 172.21.15.47:0/58945138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31f0df0 0x55b1d3218d30 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.843+0000 7f790e7f7740 1 --2- 172.21.15.47:0/58945138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb9f0 0x55b1d320ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.843+0000 7f790e7f7740 1 --2- 172.21.15.47:0/58945138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31eb610 0x55b1d32019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.843+0000 7f78e4ff9640 0 client.7158 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:45:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.843+0000 7f790e7f7740 1 -- 172.21.15.47:0/58945138 >> 172.21.15.47:0/58945138 conn(0x55b1d31bed80 msgr2=0x55b1d31c2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.843+0000 7f790e7f7740 1 -- 172.21.15.47:0/58945138 shutdown_connections 2024-03-23T09:45:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.843+0000 7f790e7f7740 1 -- 172.21.15.47:0/58945138 wait complete. 2024-03-23T09:45:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.843+0000 7f790e7f7740 20 client.7158 trim_cache size 0 max 16384 2024-03-23T09:45:50.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.857+0000 7f790e7f7740 1 Processor -- start 2024-03-23T09:45:50.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.858+0000 7f790e7f7740 1 -- start start 2024-03-23T09:45:50.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.858+0000 7f790e7f7740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31eb610 0x55b1d31bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.858+0000 7f790e7f7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 0x55b1d32090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.858+0000 7f790e7f7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31f0df0 0x55b1d3209620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.858+0000 7f790e7f7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d322b6b0 con 0x55b1d31eb9f0 2024-03-23T09:45:50.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.858+0000 7f790e7f7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d30b4140 con 0x55b1d31f0df0 2024-03-23T09:45:50.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.858+0000 7f790e7f7740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1d31f1e20 con 0x55b1d31eb610 2024-03-23T09:45:50.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.858+0000 7f790921f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 0x55b1d32090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.858+0000 7f790921f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 0x55b1d32090e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42698/0 (socket says 172.21.15.47:42698) 2024-03-23T09:45:50.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.858+0000 7f790921f640 1 -- 172.21.15.47:0/3080065953 learned_addr learned my addr 172.21.15.47:0/3080065953 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:50.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f790921f640 1 -- 172.21.15.47:0/3080065953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31eb610 msgr2=0x55b1d31bda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f790921f640 1 --2- 172.21.15.47:0/3080065953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31eb610 0x55b1d31bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f790921f640 1 -- 172.21.15.47:0/3080065953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31f0df0 msgr2=0x55b1d3209620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f790921f640 1 --2- 172.21.15.47:0/3080065953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31f0df0 0x55b1d3209620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f790921f640 1 -- 172.21.15.47:0/3080065953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1d30edab0 con 0x55b1d31eb9f0 2024-03-23T09:45:50.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f790921f640 1 --2- 172.21.15.47:0/3080065953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 0x55b1d32090e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009970 tx=0x7f78f4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6480586cbac2e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:50.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f78e7fff640 1 -- 172.21.15.47:0/3080065953 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4024040 con 0x55b1d31eb9f0 2024-03-23T09:45:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f78e7fff640 1 -- 172.21.15.47:0/3080065953 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f4010dc0 con 0x55b1d31eb9f0 2024-03-23T09:45:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f78e7fff640 1 -- 172.21.15.47:0/3080065953 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4004260 con 0x55b1d31eb9f0 2024-03-23T09:45:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f790e7f7740 1 -- 172.21.15.47:0/3080065953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 msgr2=0x55b1d32090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f790e7f7740 1 --2- 172.21.15.47:0/3080065953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 0x55b1d32090e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009970 tx=0x7f78f4015000 comp rx=0 tx=0).stop 2024-03-23T09:45:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f790e7f7740 1 -- 172.21.15.47:0/3080065953 shutdown_connections 2024-03-23T09:45:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f790e7f7740 1 --2- 172.21.15.47:0/3080065953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31f0df0 0x55b1d3209620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f790e7f7740 1 --2- 172.21.15.47:0/3080065953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb9f0 0x55b1d32090e0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f790e7f7740 1 --2- 172.21.15.47:0/3080065953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31eb610 0x55b1d31bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.859+0000 7f790e7f7740 1 -- 172.21.15.47:0/3080065953 >> 172.21.15.47:0/3080065953 conn(0x55b1d31bed80 msgr2=0x55b1d31ee640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.860+0000 7f790e7f7740 1 -- 172.21.15.47:0/3080065953 shutdown_connections 2024-03-23T09:45:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.860+0000 7f790e7f7740 1 -- 172.21.15.47:0/3080065953 wait complete. 2024-03-23T09:45:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.860+0000 7f790e7f7740 1 -- start start 2024-03-23T09:45:50.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.860+0000 7f790e7f7740 1 Processor -- start 2024-03-23T09:45:50.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f790e7f7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb610 0x55b1d31ff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f790e7f7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb9f0 0x55b1d326cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f790e7f7740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31f0df0 0x55b1d326d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f790e7f7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d326d5b0 con 0x55b1d31eb610 2024-03-23T09:45:50.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f790e7f7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d3273880 con 0x55b1d31eb9f0 2024-03-23T09:45:50.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f790e7f7740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1d31bdab0 con 0x55b1d31f0df0 2024-03-23T09:45:50.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f790e7f7740 10 client.0 _ll_register_callbacks cb 0x7f78fc07abf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:45:50.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f7909a20640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb610 0x55b1d31ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f7909a20640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb610 0x55b1d31ff130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42710/0 (socket says 172.21.15.47:42710) 2024-03-23T09:45:50.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f7909a20640 1 -- 172.21.15.47:0/2716320154 learned_addr learned my addr 172.21.15.47:0/2716320154 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:50.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f790921f640 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb9f0 0x55b1d326cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f790a221640 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31f0df0 0x55b1d326d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f7909a20640 1 -- 172.21.15.47:0/2716320154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31f0df0 msgr2=0x55b1d326d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f7909a20640 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31f0df0 0x55b1d326d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f7909a20640 1 -- 172.21.15.47:0/2716320154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb9f0 msgr2=0x55b1d326cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f7909a20640 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb9f0 0x55b1d326cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f7909a20640 1 -- 172.21.15.47:0/2716320154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1d3126560 con 0x55b1d31eb610 2024-03-23T09:45:50.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.861+0000 7f790921f640 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb9f0 0x55b1d326cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:50.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f7909a20640 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb610 0x55b1d31ff130 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7900001eb0 tx=0x7f7900005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb11d75e14636d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:50.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f78e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:45:50.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f78e4ff9640 1 -- 172.21.15.47:0/2716320154 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79000071d0 con 0x55b1d31eb610 2024-03-23T09:45:50.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f78e4ff9640 1 -- 172.21.15.47:0/2716320154 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7900007330 con 0x55b1d31eb610 2024-03-23T09:45:50.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f790e7f7740 1 -- 172.21.15.47:0/2716320154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78f400c050 con 0x55b1d31eb610 2024-03-23T09:45:50.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f78e4ff9640 1 -- 172.21.15.47:0/2716320154 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79000074f0 con 0x55b1d31eb610 2024-03-23T09:45:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f790e7f7740 10 client.7164 fetch_fsmap learned FSMap version 39 2024-03-23T09:45:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f790e7f7740 1 -- 172.21.15.47:0/2716320154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55b1d30d42e0 con 0x55b1d31eb610 2024-03-23T09:45:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f78e4ff9640 1 -- 172.21.15.47:0/2716320154 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f400c050 con 0x55b1d31eb610 2024-03-23T09:45:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f78e4ff9640 1 -- 172.21.15.47:0/2716320154 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7900020050 con 0x55b1d31eb610 2024-03-23T09:45:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f790e7f7740 10 client.7164 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T09:45:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f790e7f7740 10 client.7164 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-23T09:45:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f790e7f7740 20 client.7164 populate_metadata read hostname 'smithi047' 2024-03-23T09:45:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f790e7f7740 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55b1d31ba130 0x55b1d3270060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f790e7f7740 4 client.7164 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-03-23T09:45:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f790e7f7740 1 -- 172.21.15.47:0/2716320154 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) v1 -- 0x55b1d2e1b700 con 0x55b1d31ba130 2024-03-23T09:45:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.862+0000 7f790921f640 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55b1d31ba130 0x55b1d3270060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f790921f640 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55b1d31ba130 0x55b1d3270060 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6641251164ce1b0b server_cookie=f4ced7af9c2cf47a in_seq=0 out_seq=0 2024-03-23T09:45:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f78e4ff9640 10 client.7164 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:45:50.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f78e4ff9640 1 -- 172.21.15.47:0/2716320154 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b1d2e1b700 con 0x55b1d31ba130 2024-03-23T09:45:50.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f78e4ff9640 10 client.7164 handle_command_reply: tid=0 2024-03-23T09:45:50.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f790e7f7740 1 client.7164 shutdown 2024-03-23T09:45:50.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f790e7f7740 1 -- 172.21.15.47:0/2716320154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb610 msgr2=0x55b1d31ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f790e7f7740 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb610 0x55b1d31ff130 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7900001eb0 tx=0x7f7900005ae0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f790e7f7740 1 -- 172.21.15.47:0/2716320154 shutdown_connections 2024-03-23T09:45:50.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f790e7f7740 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b1d31f0df0 0x55b1d326d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f790e7f7740 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1d31eb9f0 0x55b1d326cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f790e7f7740 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55b1d31ba130 0x55b1d3270060 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f790e7f7740 1 --2- 172.21.15.47:0/2716320154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1d31eb610 0x55b1d31ff130 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f78e4ff9640 0 client.7164 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:45:50.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f790e7f7740 1 -- 172.21.15.47:0/2716320154 >> 172.21.15.47:0/2716320154 conn(0x55b1d31bed80 msgr2=0x55b1d31f06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:50.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f790e7f7740 1 -- 172.21.15.47:0/2716320154 shutdown_connections 2024-03-23T09:45:50.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.863+0000 7f790e7f7740 1 -- 172.21.15.47:0/2716320154 wait complete. 2024-03-23T09:45:50.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.864+0000 7f790e7f7740 20 client.7164 trim_cache size 0 max 16384 2024-03-23T09:45:50.871 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:45:50.871 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-23T09:45:50.871 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "57482fb1-fe34-4a44-862e-d8010c0efaef", 2024-03-23T09:45:50.871 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-23T09:45:50.872 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:45:50.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.864+0000 7f790bc84640 1 -- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f78e806ac00 msgr2=0x7f78e806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.864+0000 7f790bc84640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f78e806ac00 0x7f78e806d0c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7900005880 tx=0x7f7900009040 comp rx=0 tx=0).stop 2024-03-23T09:45:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.864+0000 7f790bc84640 1 -- 172.21.15.47:0/1656943664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7904160780 msgr2=0x7f79040cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.864+0000 7f790bc84640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7904160780 0x7f79040cb6a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009e80 tx=0x7f78fc004a40 comp rx=0 tx=0).stop 2024-03-23T09:45:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.864+0000 7f790bc84640 1 -- 172.21.15.47:0/1656943664 shutdown_connections 2024-03-23T09:45:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.864+0000 7f790bc84640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f78e806ac00 0x7f78e806d0c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.864+0000 7f790bc84640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7904160780 0x7f79040cb6a0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.864+0000 7f790bc84640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f790415cb50 0x7f7904150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.864+0000 7f790bc84640 1 --2- 172.21.15.47:0/1656943664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79040cde40 0x7f79040cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.864+0000 7f790bc84640 1 -- 172.21.15.47:0/1656943664 >> 172.21.15.47:0/1656943664 conn(0x7f790406b160 msgr2=0x7f7904151120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.864+0000 7f790bc84640 1 -- 172.21.15.47:0/1656943664 shutdown_connections 2024-03-23T09:45:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.864+0000 7f790bc84640 1 -- 172.21.15.47:0/1656943664 wait complete. 2024-03-23T09:45:50.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:45:50.875 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-23T09:45:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.978+0000 7f2cb832e640 1 Processor -- start 2024-03-23T09:45:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.978+0000 7f2cb832e640 1 -- start start 2024-03-23T09:45:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb832e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cb015c990 0x7f2cb0160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb832e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2cb00b8350 0x7f2cb00cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb832e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb0161630 con 0x7f2cb00cdc40 2024-03-23T09:45:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb832e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb00ce0b0 con 0x7f2cb00b8350 2024-03-23T09:45:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb832e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb00d1030 con 0x7f2cb015c990 2024-03-23T09:45:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb58c9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2cb00b8350 0x7f2cb00cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb68cb640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2cb00cdc40 0x7f2cb00b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34544/0 (socket says 172.21.15.47:34544) 2024-03-23T09:45:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb58c9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2cb00b8350 0x7f2cb00cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42714/0 (socket says 172.21.15.47:42714) 2024-03-23T09:45:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb68cb640 1 -- 172.21.15.47:0/1453005668 learned_addr learned my addr 172.21.15.47:0/1453005668 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:45:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/1453005668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cb015c990 0x7f2cb0160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb58c9640 1 -- 172.21.15.47:0/1453005668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cb015c990 msgr2=0x7f2cb0160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/1453005668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cb015c990 0x7f2cb0160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb58c9640 1 -- 172.21.15.47:0/1453005668 >> v1:172.21.15.47:6789/0 conn(0x7f2cb00cdc40 legacy=0x7f2cb00b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.979+0000 7f2cb58c9640 1 -- 172.21.15.47:0/1453005668 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb00cfe10 con 0x7f2cb00b8350 2024-03-23T09:45:50.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.980+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/1453005668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb00b8350 0x7f2cb00cd700 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00097a0 tx=0x7f2ca002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed8eed50ceec91c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:50.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.980+0000 7f2cb50c8640 1 -- 172.21.15.47:0/1453005668 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca002e070 con 0x7f2cb00b8350 2024-03-23T09:45:50.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.980+0000 7f2cb50c8640 1 -- 172.21.15.47:0/1453005668 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca00315a0 con 0x7f2cb00b8350 2024-03-23T09:45:50.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.980+0000 7f2cb50c8640 1 -- 172.21.15.47:0/1453005668 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0031930 con 0x7f2cb00b8350 2024-03-23T09:45:50.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.980+0000 7f2cb832e640 1 -- 172.21.15.47:0/1453005668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb00b8350 msgr2=0x7f2cb00cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.980+0000 7f2cb832e640 1 --2- 172.21.15.47:0/1453005668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb00b8350 0x7f2cb00cd700 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00097a0 tx=0x7f2ca002b840 comp rx=0 tx=0).stop 2024-03-23T09:45:50.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.981+0000 7f2cb832e640 1 -- 172.21.15.47:0/1453005668 shutdown_connections 2024-03-23T09:45:50.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.981+0000 7f2cb832e640 1 --2- 172.21.15.47:0/1453005668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb00b8350 0x7f2cb00cd700 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.981+0000 7f2cb832e640 1 --2- 172.21.15.47:0/1453005668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cb015c990 0x7f2cb0160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.981+0000 7f2cb832e640 1 -- 172.21.15.47:0/1453005668 >> 172.21.15.47:0/1453005668 conn(0x7f2cb00659d0 msgr2=0x7f2cb0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:50.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.981+0000 7f2cb832e640 1 -- 172.21.15.47:0/1453005668 shutdown_connections 2024-03-23T09:45:50.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.981+0000 7f2cb832e640 1 -- 172.21.15.47:0/1453005668 wait complete. 2024-03-23T09:45:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.982+0000 7f2cb832e640 1 Processor -- start 2024-03-23T09:45:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.982+0000 7f2cb832e640 1 -- start start 2024-03-23T09:45:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.982+0000 7f2cb832e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cb00b8350 0x7f2cb0150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.982+0000 7f2cb832e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb00cdc40 0x7f2cb01592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.982+0000 7f2cb832e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cb015c990 0x7f2cb01538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.982+0000 7f2cb832e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb00c6e60 con 0x7f2cb00cdc40 2024-03-23T09:45:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.982+0000 7f2cb832e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb0159800 con 0x7f2cb015c990 2024-03-23T09:45:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.982+0000 7f2cb832e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb00c6530 con 0x7f2cb00b8350 2024-03-23T09:45:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb58c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb00cdc40 0x7f2cb01592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb58c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb00cdc40 0x7f2cb01592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42722/0 (socket says 172.21.15.47:42722) 2024-03-23T09:45:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb58c9640 1 -- 172.21.15.47:0/2453119684 learned_addr learned my addr 172.21.15.47:0/2453119684 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cb00b8350 0x7f2cb0150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb68cb640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cb015c990 0x7f2cb01538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb58c9640 1 -- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cb00b8350 msgr2=0x7f2cb0150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cb00b8350 0x7f2cb0150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb58c9640 1 -- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cb015c990 msgr2=0x7f2cb01538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cb015c990 0x7f2cb01538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb58c9640 1 -- 172.21.15.47:0/2453119684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca0009400 con 0x7f2cb00cdc40 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cb00b8350 0x7f2cb0150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb68cb640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cb015c990 0x7f2cb01538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb00cdc40 0x7f2cb01592c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0034040 tx=0x7f2ca00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2ca6ffd640 1 -- 172.21.15.47:0/2453119684 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0032840 con 0x7f2cb00cdc40 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2ca6ffd640 1 -- 172.21.15.47:0/2453119684 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca00329a0 con 0x7f2cb00cdc40 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb832e640 1 -- 172.21.15.47:0/2453119684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb00c6810 con 0x7f2cb00cdc40 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2ca6ffd640 1 -- 172.21.15.47:0/2453119684 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0032d50 con 0x7f2cb00cdc40 2024-03-23T09:45:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.983+0000 7f2cb832e640 1 -- 172.21.15.47:0/2453119684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb00c8d00 con 0x7f2cb00cdc40 2024-03-23T09:45:50.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.985+0000 7f2ca6ffd640 1 -- 172.21.15.47:0/2453119684 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2ca004e430 con 0x7f2cb00cdc40 2024-03-23T09:45:50.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.985+0000 7f2ca6ffd640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c9406ac00 0x7f2c9406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.985+0000 7f2ca6ffd640 1 -- 172.21.15.47:0/2453119684 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2ca00ab390 con 0x7f2cb00cdc40 2024-03-23T09:45:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.985+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c9406ac00 0x7f2c9406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.986+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c9406ac00 0x7f2c9406d0c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2cac006fd0 tx=0x7f2cac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.986+0000 7f2cbaf7b740 1 Processor -- start 2024-03-23T09:45:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.986+0000 7f2cbaf7b740 1 -- start start 2024-03-23T09:45:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cbaf7b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e227bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cbaf7b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224b850 0x5588e2278aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cbaf7b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e224f3b0 0x5588e2278fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cbaf7b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e21bbdd0 con 0x5588e227b830 2024-03-23T09:45:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cbaf7b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e21b9560 con 0x5588e224b850 2024-03-23T09:45:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cbaf7b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588e21b4e50 con 0x5588e224f3b0 2024-03-23T09:45:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cb60ca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e227bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cb60ca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e227bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42726/0 (socket says 172.21.15.47:42726) 2024-03-23T09:45:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cb60ca640 1 -- 172.21.15.47:0/4236668614 learned_addr learned my addr 172.21.15.47:0/4236668614 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cb68cb640 1 --2- 172.21.15.47:0/4236668614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e224f3b0 0x5588e2278fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/4236668614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224b850 0x5588e2278aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cb60ca640 1 -- 172.21.15.47:0/4236668614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e224f3b0 msgr2=0x5588e2278fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/4236668614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e224f3b0 0x5588e2278fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cb60ca640 1 -- 172.21.15.47:0/4236668614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224b850 msgr2=0x5588e2278aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/4236668614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224b850 0x5588e2278aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cb60ca640 1 -- 172.21.15.47:0/4236668614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588e217c470 con 0x5588e227b830 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cb68cb640 1 --2- 172.21.15.47:0/4236668614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e224f3b0 0x5588e2278fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.987+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/4236668614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224b850 0x5588e2278aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.988+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/4236668614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e227bc10 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0042a0 tx=0x7f2cac011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2a274ef64d8ad9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.988+0000 7f2c93fff640 1 -- 172.21.15.47:0/4236668614 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac01d070 con 0x5588e227b830 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.988+0000 7f2c93fff640 1 -- 172.21.15.47:0/4236668614 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac004390 con 0x5588e227b830 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.988+0000 7f2c93fff640 1 -- 172.21.15.47:0/4236668614 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00eea0 con 0x5588e227b830 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.988+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/4236668614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 msgr2=0x5588e227bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.988+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/4236668614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e227bc10 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0042a0 tx=0x7f2cac011000 comp rx=0 tx=0).stop 2024-03-23T09:45:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.988+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/4236668614 shutdown_connections 2024-03-23T09:45:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.988+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/4236668614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e224f3b0 0x5588e2278fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.988+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/4236668614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224b850 0x5588e2278aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.988+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/4236668614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e227bc10 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.988+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/4236668614 >> 172.21.15.47:0/4236668614 conn(0x5588e224dac0 msgr2=0x5588e2296640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.988+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/4236668614 shutdown_connections 2024-03-23T09:45:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.988+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/4236668614 wait complete. 2024-03-23T09:45:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.989+0000 7f2cbaf7b740 1 -- start start 2024-03-23T09:45:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.989+0000 7f2cbaf7b740 1 Processor -- start 2024-03-23T09:45:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cbaf7b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e224f3b0 0x5588e227d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cbaf7b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e2268c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cbaf7b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e227c170 0x5588e227e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cbaf7b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e2173280 con 0x5588e227b830 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cbaf7b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e2172f20 con 0x5588e227c170 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cbaf7b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588e21383c0 con 0x5588e224f3b0 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cbaf7b740 10 client.0 _ll_register_callbacks cb 0x5588e218a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cb58c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e2268c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cb58c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e2268c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42732/0 (socket says 172.21.15.47:42732) 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cb58c9640 1 -- 172.21.15.47:0/102630379 learned_addr learned my addr 172.21.15.47:0/102630379 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cb68cb640 1 --2- 172.21.15.47:0/102630379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e227c170 0x5588e227e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/102630379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e224f3b0 0x5588e227d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cb58c9640 1 -- 172.21.15.47:0/102630379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e224f3b0 msgr2=0x5588e227d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/102630379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e224f3b0 0x5588e227d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cb58c9640 1 -- 172.21.15.47:0/102630379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e227c170 msgr2=0x5588e227e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/102630379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e227c170 0x5588e227e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.990+0000 7f2cb58c9640 1 -- 172.21.15.47:0/102630379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac004030 con 0x5588e227b830 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/102630379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e2268c30 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2ca002be50 tx=0x7f2ca0079c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab614aa4d354bfae server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2c917fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:45:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2c917fa640 1 -- 172.21.15.47:0/102630379 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca007e090 con 0x5588e227b830 2024-03-23T09:45:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2c917fa640 1 -- 172.21.15.47:0/102630379 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2ca0082650 con 0x5588e227b830 2024-03-23T09:45:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/102630379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588e21de690 con 0x5588e227b830 2024-03-23T09:45:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2c917fa640 1 -- 172.21.15.47:0/102630379 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca0076970 con 0x5588e227b830 2024-03-23T09:45:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2cbaf7b740 10 client.7179 fetch_fsmap learned FSMap version 39 2024-03-23T09:45:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/102630379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5588e21b34d0 con 0x5588e227b830 2024-03-23T09:45:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2c917fa640 1 -- 172.21.15.47:0/102630379 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5588e21de690 con 0x5588e227b830 2024-03-23T09:45:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2c917fa640 1 -- 172.21.15.47:0/102630379 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2ca0079750 con 0x5588e227b830 2024-03-23T09:45:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2cbaf7b740 10 client.7179 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T09:45:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2cbaf7b740 10 client.7179 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-23T09:45:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2cbaf7b740 20 client.7179 populate_metadata read hostname 'smithi047' 2024-03-23T09:45:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/102630379 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5588e22a61b0 0x5588e22a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2cbaf7b740 4 client.7179 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/102630379 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588e21a6eb0 con 0x5588e22a61b0 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.991+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/102630379 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5588e22a61b0 0x5588e22a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2c917fa640 10 client.7179 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/102630379 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5588e22a61b0 0x5588e22a6590 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59cb05cc58a8b551 server_cookie=479fe081041a385b in_seq=0 out_seq=0 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2c917fa640 1 -- 172.21.15.47:0/102630379 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5588e21a6eb0 con 0x5588e22a61b0 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2c917fa640 10 client.7179 handle_command_reply: tid=0 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2cbaf7b740 1 client.7179 shutdown 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/102630379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 msgr2=0x5588e2268c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/102630379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e2268c30 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2ca002be50 tx=0x7f2ca0079c80 comp rx=0 tx=0).stop 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/102630379 shutdown_connections 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/102630379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e227c170 0x5588e227e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/102630379 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5588e22a61b0 0x5588e22a6590 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/102630379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e2268c30 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/102630379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e224f3b0 0x5588e227d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2c917fa640 0 client.7179 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:45:51.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/102630379 >> 172.21.15.47:0/102630379 conn(0x5588e224b850 msgr2=0x5588e227f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:51.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/102630379 shutdown_connections 2024-03-23T09:45:51.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.992+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/102630379 wait complete. 2024-03-23T09:45:51.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:50.993+0000 7f2cbaf7b740 20 client.7179 trim_cache size 0 max 16384 2024-03-23T09:45:51.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.007+0000 7f2cbaf7b740 1 Processor -- start 2024-03-23T09:45:51.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.007+0000 7f2cbaf7b740 1 -- start start 2024-03-23T09:45:51.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cbaf7b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224f3b0 0x5588e2299340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:51.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cbaf7b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e2273c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cbaf7b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e227c170 0x5588e224a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cbaf7b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e227f730 con 0x5588e227b830 2024-03-23T09:45:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cbaf7b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e2299880 con 0x5588e224f3b0 2024-03-23T09:45:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cbaf7b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588e227f4e0 con 0x5588e227c170 2024-03-23T09:45:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cb58c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e2273c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cb58c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e2273c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42746/0 (socket says 172.21.15.47:42746) 2024-03-23T09:45:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cb58c9640 1 -- 172.21.15.47:0/3942110075 learned_addr learned my addr 172.21.15.47:0/3942110075 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cb68cb640 1 --2- 172.21.15.47:0/3942110075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e227c170 0x5588e224a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/3942110075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224f3b0 0x5588e2299340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cb58c9640 1 -- 172.21.15.47:0/3942110075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e227c170 msgr2=0x5588e224a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/3942110075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e227c170 0x5588e224a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cb58c9640 1 -- 172.21.15.47:0/3942110075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224f3b0 msgr2=0x5588e2299340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/3942110075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224f3b0 0x5588e2299340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cb58c9640 1 -- 172.21.15.47:0/3942110075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588e21e79e0 con 0x5588e227b830 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/3942110075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e2273c60 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0079f60 tx=0x7f2ca0091000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f12b3d50d70aaee server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2ca4e0c640 1 -- 172.21.15.47:0/3942110075 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0082650 con 0x5588e227b830 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2ca4e0c640 1 -- 172.21.15.47:0/3942110075 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca007e090 con 0x5588e227b830 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.008+0000 7f2ca4e0c640 1 -- 172.21.15.47:0/3942110075 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0076970 con 0x5588e227b830 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.009+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/3942110075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 msgr2=0x5588e2273c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.009+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/3942110075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e2273c60 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0079f60 tx=0x7f2ca0091000 comp rx=0 tx=0).stop 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.009+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/3942110075 shutdown_connections 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.009+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/3942110075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e227c170 0x5588e224a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.009+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/3942110075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227b830 0x5588e2273c60 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.009+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/3942110075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224f3b0 0x5588e2299340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.009+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/3942110075 >> 172.21.15.47:0/3942110075 conn(0x5588e224b850 msgr2=0x5588e2247030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.009+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/3942110075 shutdown_connections 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.009+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/3942110075 wait complete. 2024-03-23T09:45:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.009+0000 7f2cbaf7b740 1 -- start start 2024-03-23T09:45:51.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.009+0000 7f2cbaf7b740 1 Processor -- start 2024-03-23T09:45:51.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.010+0000 7f2cbaf7b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224f3b0 0x5588e227f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:51.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.010+0000 7f2cbaf7b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e227b830 0x5588e22f9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:51.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.010+0000 7f2cbaf7b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227c170 0x5588e22af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:51.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.010+0000 7f2cbaf7b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e2276650 con 0x5588e227c170 2024-03-23T09:45:51.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.010+0000 7f2cbaf7b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e2276ab0 con 0x5588e224f3b0 2024-03-23T09:45:51.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.010+0000 7f2cbaf7b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588e2300630 con 0x5588e227b830 2024-03-23T09:45:51.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.010+0000 7f2cbaf7b740 10 client.0 _ll_register_callbacks cb 0x7f2cac009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.010+0000 7f2cb68cb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227c170 0x5588e22af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.010+0000 7f2cb68cb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227c170 0x5588e22af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42762/0 (socket says 172.21.15.47:42762) 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.010+0000 7f2cb68cb640 1 -- 172.21.15.47:0/127612809 learned_addr learned my addr 172.21.15.47:0/127612809 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/127612809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224f3b0 0x5588e227f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2cb58c9640 1 --2- 172.21.15.47:0/127612809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e227b830 0x5588e22f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2cb68cb640 1 -- 172.21.15.47:0/127612809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e227b830 msgr2=0x5588e22f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2cb68cb640 1 --2- 172.21.15.47:0/127612809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e227b830 0x5588e22f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2cb68cb640 1 -- 172.21.15.47:0/127612809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224f3b0 msgr2=0x5588e227f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2cb68cb640 1 --2- 172.21.15.47:0/127612809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224f3b0 0x5588e227f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2cb68cb640 1 -- 172.21.15.47:0/127612809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca0091e70 con 0x5588e227c170 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2cb68cb640 1 --2- 172.21.15.47:0/127612809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227c170 0x5588e22af1f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8007780 tx=0x7f2ca8007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab83e1d810c70ae server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2c917fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2c917fa640 1 -- 172.21.15.47:0/127612809 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8007ac0 con 0x5588e227c170 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2c917fa640 1 -- 172.21.15.47:0/127612809 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2ca8013ac0 con 0x5588e227c170 2024-03-23T09:45:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/127612809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cac004390 con 0x5588e227c170 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2c917fa640 1 -- 172.21.15.47:0/127612809 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca8013c80 con 0x5588e227c170 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2cbaf7b740 10 client.7185 fetch_fsmap learned FSMap version 39 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/127612809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5588e21555c0 con 0x5588e227c170 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.011+0000 7f2c917fa640 1 -- 172.21.15.47:0/127612809 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac004390 con 0x5588e227c170 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2c917fa640 1 -- 172.21.15.47:0/127612809 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2ca8016450 con 0x5588e227c170 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 10 client.7185 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 10 client.7185 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 20 client.7185 populate_metadata read hostname 'smithi047' 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/127612809 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5588e22af730 0x5588e22fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 4 client.7185 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/127612809 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5588e1ea84e0 con 0x5588e22af730 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/127612809 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5588e22af730 0x5588e22fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cb60ca640 1 --2- 172.21.15.47:0/127612809 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5588e22af730 0x5588e22fc4b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7547e61bb324a5 server_cookie=752102b53fef695b in_seq=0 out_seq=0 2024-03-23T09:45:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2c917fa640 10 client.7185 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2c917fa640 1 -- 172.21.15.47:0/127612809 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5588e1ea84e0 con 0x5588e22af730 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2c917fa640 10 client.7185 handle_command_reply: tid=0 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 1 client.7185 shutdown 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/127612809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227c170 msgr2=0x5588e22af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/127612809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227c170 0x5588e22af1f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8007780 tx=0x7f2ca8007870 comp rx=0 tx=0).stop 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/127612809 shutdown_connections 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/127612809 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5588e22af730 0x5588e22fc4b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/127612809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588e227c170 0x5588e22af1f0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/127612809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5588e227b830 0x5588e22f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 1 --2- 172.21.15.47:0/127612809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588e224f3b0 0x5588e227f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2c917fa640 0 client.7185 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/127612809 >> 172.21.15.47:0/127612809 conn(0x5588e224b850 msgr2=0x5588e224a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/127612809 shutdown_connections 2024-03-23T09:45:51.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.012+0000 7f2cbaf7b740 1 -- 172.21.15.47:0/127612809 wait complete. 2024-03-23T09:45:51.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.013+0000 7f2cbaf7b740 20 client.7185 trim_cache size 0 max 16384 2024-03-23T09:45:51.019 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:45:51.019 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T09:45:51.019 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-23T09:45:51.019 INFO:teuthology.orchestra.run.smithi047.stdout: "57482fb1-fe34-4a44-862e-d8010c0efaef": { 2024-03-23T09:45:51.019 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/subdir", 2024-03-23T09:45:51.019 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "57482fb1-fe34-4a44-862e-d8010c0efaef", 2024-03-23T09:45:51.019 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive" 2024-03-23T09:45:51.019 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:51.019 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:45:51.020 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:45:51.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.013+0000 7f2cb832e640 1 -- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c9406ac00 msgr2=0x7f2c9406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.013+0000 7f2cb832e640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c9406ac00 0x7f2c9406d0c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2cac006fd0 tx=0x7f2cac008040 comp rx=0 tx=0).stop 2024-03-23T09:45:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.013+0000 7f2cb832e640 1 -- 172.21.15.47:0/2453119684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb00cdc40 msgr2=0x7f2cb01592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:45:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.013+0000 7f2cb832e640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb00cdc40 0x7f2cb01592c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0034040 tx=0x7f2ca00046c0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.013+0000 7f2cb832e640 1 -- 172.21.15.47:0/2453119684 shutdown_connections 2024-03-23T09:45:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.013+0000 7f2cb832e640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c9406ac00 0x7f2c9406d0c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.013+0000 7f2cb832e640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cb015c990 0x7f2cb01538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.013+0000 7f2cb832e640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb00cdc40 0x7f2cb01592c0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.013+0000 7f2cb832e640 1 --2- 172.21.15.47:0/2453119684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cb00b8350 0x7f2cb0150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:45:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.013+0000 7f2cb832e640 1 -- 172.21.15.47:0/2453119684 >> 172.21.15.47:0/2453119684 conn(0x7f2cb00659d0 msgr2=0x7f2cb00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:45:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.013+0000 7f2cb832e640 1 -- 172.21.15.47:0/2453119684 shutdown_connections 2024-03-23T09:45:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:45:51.013+0000 7f2cb832e640 1 -- 172.21.15.47:0/2453119684 wait complete. 2024-03-23T09:45:51.023 INFO:tasks.cephfs.filesystem:scrub status for tag:57482fb1-fe34-4a44-862e-d8010c0efaef - {'path': '/subdir', 'tag': '57482fb1-fe34-4a44-862e-d8010c0efaef', 'options': 'recursive'} 2024-03-23T09:46:07.794 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:46:07.798 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:46:21.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:21.025 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-23T09:46:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.140+0000 7f22e524a640 1 Processor -- start 2024-03-23T09:46:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.140+0000 7f22e524a640 1 -- start start 2024-03-23T09:46:21.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.141+0000 7f22e524a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f22e00cde10 0x7f22e0160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.141+0000 7f22e524a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e0161630 con 0x7f22e015c990 2024-03-23T09:46:21.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.141+0000 7f22e524a640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e00d1030 con 0x7f22e00b8350 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.141+0000 7f22e524a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e00cfe10 con 0x7f22e00cde10 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.141+0000 7f22ded9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f22e015c990 0x7f22e00cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51234/0 (socket says 172.21.15.47:51234) 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.141+0000 7f22ded9d640 1 -- 172.21.15.47:0/4056099219 learned_addr learned my addr 172.21.15.47:0/4056099219 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.141+0000 7f22de59c640 1 --2- 172.21.15.47:0/4056099219 >> v2:172.21.15.175:3300/0 conn(0x7f22e00cde10 0x7f22e0160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.141+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731993365 0 0) 0x7f22e0161630 con 0x7f22e015c990 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.141+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22cc003520 con 0x7f22e015c990 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3252725960 0 0) 0x7f22cc003520 con 0x7f22e015c990 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 >> v2:172.21.15.175:3300/0 conn(0x7f22e00cde10 msgr2=0x7f22e0160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ddd9b640 1 --2- 172.21.15.47:0/4056099219 >> v2:172.21.15.175:3300/0 conn(0x7f22e00cde10 0x7f22e0160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 >> v1:172.21.15.175:6789/0 conn(0x7f22e00b8350 legacy=0x7f22e00d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00b5b60 con 0x7f22e015c990 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f22d00019f0 con 0x7f22e015c990 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 >> v1:172.21.15.47:6789/0 conn(0x7f22e015c990 legacy=0x7f22e00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ddd9b640 1 --2- 172.21.15.47:0/4056099219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22cc003970 0x7f22cc003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ddd9b640 1 --2- 172.21.15.47:0/4056099219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22cc004370 0x7f22cc004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ddd9b640 1 --2- 172.21.15.47:0/4056099219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22cc0050e0 0x7f22cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e00cfe10 con 0x7f22cc004370 2024-03-23T09:46:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc00c020 con 0x7f22cc003970 2024-03-23T09:46:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc00c1e0 con 0x7f22cc0050e0 2024-03-23T09:46:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22de59c640 1 --2- 172.21.15.47:0/4056099219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22cc004370 0x7f22cc004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ded9d640 1 --2- 172.21.15.47:0/4056099219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22cc003970 0x7f22cc003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22df59e640 1 --2- 172.21.15.47:0/4056099219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22cc0050e0 0x7f22cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22de59c640 1 -- 172.21.15.47:0/4056099219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22cc0050e0 msgr2=0x7f22cc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22de59c640 1 --2- 172.21.15.47:0/4056099219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22cc0050e0 0x7f22cc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22de59c640 1 -- 172.21.15.47:0/4056099219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22cc003970 msgr2=0x7f22cc003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22de59c640 1 --2- 172.21.15.47:0/4056099219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22cc003970 0x7f22cc003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22de59c640 1 -- 172.21.15.47:0/4056099219 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc003520 con 0x7f22cc004370 2024-03-23T09:46:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22ded9d640 1 --2- 172.21.15.47:0/4056099219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22cc003970 0x7f22cc003d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.142+0000 7f22df59e640 1 -- 172.21.15.47:0/4056099219 reap_dead start 2024-03-23T09:46:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.143+0000 7f22de59c640 1 --2- 172.21.15.47:0/4056099219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22cc004370 0x7f22cc004840 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009ac0 tx=0x7f22cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a728c94430e03cd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.143+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c802c6b0 con 0x7f22cc004370 2024-03-23T09:46:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.143+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c802c810 con 0x7f22cc004370 2024-03-23T09:46:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.143+0000 7f22ddd9b640 1 -- 172.21.15.47:0/4056099219 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c802cba0 con 0x7f22cc004370 2024-03-23T09:46:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.144+0000 7f22e524a640 1 -- 172.21.15.47:0/4056099219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22cc004370 msgr2=0x7f22cc004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.144+0000 7f22e524a640 1 --2- 172.21.15.47:0/4056099219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22cc004370 0x7f22cc004840 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009ac0 tx=0x7f22cc0057f0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.144+0000 7f22e524a640 1 -- 172.21.15.47:0/4056099219 shutdown_connections 2024-03-23T09:46:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.144+0000 7f22e524a640 1 --2- 172.21.15.47:0/4056099219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22cc004370 0x7f22cc004840 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.144+0000 7f22e524a640 1 -- 172.21.15.47:0/4056099219 >> 172.21.15.47:0/4056099219 conn(0x7f22e00659d0 msgr2=0x7f22e0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.144+0000 7f22e524a640 1 -- 172.21.15.47:0/4056099219 shutdown_connections 2024-03-23T09:46:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.144+0000 7f22e524a640 1 -- 172.21.15.47:0/4056099219 wait complete. 2024-03-23T09:46:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.145+0000 7f22e524a640 1 Processor -- start 2024-03-23T09:46:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.145+0000 7f22e524a640 1 -- start start 2024-03-23T09:46:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22e524a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22cc004370 0x7f22e0153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22e524a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22e00b26c0 0x7f22e00b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22e524a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e015a790 0x7f22e015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22e524a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e00b3080 con 0x7f22cc004370 2024-03-23T09:46:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22e524a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e015b150 con 0x7f22e015a790 2024-03-23T09:46:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22e524a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e00ccdd0 con 0x7f22e00b26c0 2024-03-23T09:46:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22ded9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22cc004370 0x7f22e0153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22ded9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22cc004370 0x7f22e0153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41476/0 (socket says 172.21.15.47:41476) 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22ded9d640 1 -- 172.21.15.47:0/4114276086 learned_addr learned my addr 172.21.15.47:0/4114276086 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22df59e640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e015a790 0x7f22e015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22de59c640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22e00b26c0 0x7f22e00b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22ded9d640 1 -- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22e00b26c0 msgr2=0x7f22e00b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22ded9d640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22e00b26c0 0x7f22e00b2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22ded9d640 1 -- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e015a790 msgr2=0x7f22e015ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22ded9d640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e015a790 0x7f22e015ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.146+0000 7f22ded9d640 1 -- 172.21.15.47:0/4114276086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e0154320 con 0x7f22cc004370 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.147+0000 7f22df59e640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e015a790 0x7f22e015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.147+0000 7f22ded9d640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22cc004370 0x7f22e0153de0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f22d0005fd0 tx=0x7f22d000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.147+0000 7f22d77fe640 1 -- 172.21.15.47:0/4114276086 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0013420 con 0x7f22cc004370 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.147+0000 7f22d77fe640 1 -- 172.21.15.47:0/4114276086 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d0013580 con 0x7f22cc004370 2024-03-23T09:46:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.147+0000 7f22e524a640 1 -- 172.21.15.47:0/4114276086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e01f20a0 con 0x7f22cc004370 2024-03-23T09:46:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.147+0000 7f22d77fe640 1 -- 172.21.15.47:0/4114276086 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d00138f0 con 0x7f22cc004370 2024-03-23T09:46:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.147+0000 7f22e524a640 1 -- 172.21.15.47:0/4114276086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e01f2360 con 0x7f22cc004370 2024-03-23T09:46:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.148+0000 7f22d77fe640 1 -- 172.21.15.47:0/4114276086 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f22d0013a50 con 0x7f22cc004370 2024-03-23T09:46:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.149+0000 7f22d77fe640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f22c006ac00 0x7f22c006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.149+0000 7f22d77fe640 1 -- 172.21.15.47:0/4114276086 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f22d0089c30 con 0x7f22cc004370 2024-03-23T09:46:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.149+0000 7f22de59c640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f22c006ac00 0x7f22c006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.150+0000 7f22de59c640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f22c006ac00 0x7f22c006d0c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f22c8036000 tx=0x7f22c8030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.150+0000 7f22e7dc4740 1 Processor -- start 2024-03-23T09:46:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.150+0000 7f22e7dc4740 1 -- start start 2024-03-23T09:46:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22e7dc4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f92776830 0x555f92776c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22e7dc4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92746850 0x555f92773aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22e7dc4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f9274a3b0 0x555f92773fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22e7dc4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f926b6dd0 con 0x555f92746850 2024-03-23T09:46:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22e7dc4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f926b4560 con 0x555f9274a3b0 2024-03-23T09:46:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22e7dc4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f926afe50 con 0x555f92776830 2024-03-23T09:46:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22de59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92746850 0x555f92773aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22de59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92746850 0x555f92773aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41480/0 (socket says 172.21.15.47:41480) 2024-03-23T09:46:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22de59c640 1 -- 172.21.15.47:0/1346092166 learned_addr learned my addr 172.21.15.47:0/1346092166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22ded9d640 1 --2- 172.21.15.47:0/1346092166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f92776830 0x555f92776c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22df59e640 1 --2- 172.21.15.47:0/1346092166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f9274a3b0 0x555f92773fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22de59c640 1 -- 172.21.15.47:0/1346092166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f92776830 msgr2=0x555f92776c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22de59c640 1 --2- 172.21.15.47:0/1346092166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f92776830 0x555f92776c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22de59c640 1 -- 172.21.15.47:0/1346092166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f9274a3b0 msgr2=0x555f92773fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22de59c640 1 --2- 172.21.15.47:0/1346092166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f9274a3b0 0x555f92773fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22de59c640 1 -- 172.21.15.47:0/1346092166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f92677470 con 0x555f92746850 2024-03-23T09:46:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.151+0000 7f22df59e640 1 --2- 172.21.15.47:0/1346092166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f9274a3b0 0x555f92773fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.152+0000 7f22de59c640 1 --2- 172.21.15.47:0/1346092166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92746850 0x555f92773aa0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f22c8037040 tx=0x7f22c802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8ae5479f7c36d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.152+0000 7f22d4ff9640 1 -- 172.21.15.47:0/1346092166 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c802cdc0 con 0x555f92746850 2024-03-23T09:46:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.152+0000 7f22d4ff9640 1 -- 172.21.15.47:0/1346092166 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c8003610 con 0x555f92746850 2024-03-23T09:46:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.152+0000 7f22d4ff9640 1 -- 172.21.15.47:0/1346092166 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c80039a0 con 0x555f92746850 2024-03-23T09:46:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.152+0000 7f22e7dc4740 1 -- 172.21.15.47:0/1346092166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92746850 msgr2=0x555f92773aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.152+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/1346092166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92746850 0x555f92773aa0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f22c8037040 tx=0x7f22c802e000 comp rx=0 tx=0).stop 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.152+0000 7f22e7dc4740 1 -- 172.21.15.47:0/1346092166 shutdown_connections 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.152+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/1346092166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f9274a3b0 0x555f92773fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.152+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/1346092166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92746850 0x555f92773aa0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.152+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/1346092166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f92776830 0x555f92776c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.152+0000 7f22e7dc4740 1 -- 172.21.15.47:0/1346092166 >> 172.21.15.47:0/1346092166 conn(0x555f92748ac0 msgr2=0x555f92791640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.153+0000 7f22e7dc4740 1 -- 172.21.15.47:0/1346092166 shutdown_connections 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.153+0000 7f22e7dc4740 1 -- 172.21.15.47:0/1346092166 wait complete. 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.153+0000 7f22e7dc4740 1 -- start start 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.153+0000 7f22e7dc4740 1 Processor -- start 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22e7dc4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f9274a3b0 0x555f92763bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22e7dc4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 0x555f92779360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22e7dc4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 0x555f927909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22e7dc4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f9266e280 con 0x555f92776830 2024-03-23T09:46:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22e7dc4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f9266df20 con 0x555f927798a0 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22e7dc4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f926333c0 con 0x555f9274a3b0 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22e7dc4740 10 client.0 _ll_register_callbacks cb 0x555f92685380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22de59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 0x555f92779360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22de59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 0x555f92779360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41496/0 (socket says 172.21.15.47:41496) 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22de59c640 1 -- 172.21.15.47:0/1690237067 learned_addr learned my addr 172.21.15.47:0/1690237067 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22ded9d640 1 --2- 172.21.15.47:0/1690237067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f9274a3b0 0x555f92763bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22df59e640 1 --2- 172.21.15.47:0/1690237067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 0x555f927909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22de59c640 1 -- 172.21.15.47:0/1690237067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f9274a3b0 msgr2=0x555f92763bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22de59c640 1 --2- 172.21.15.47:0/1690237067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f9274a3b0 0x555f92763bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22de59c640 1 -- 172.21.15.47:0/1690237067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 msgr2=0x555f927909e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22de59c640 1 --2- 172.21.15.47:0/1690237067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 0x555f927909e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.154+0000 7f22de59c640 1 -- 172.21.15.47:0/1690237067 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22cc003520 con 0x555f92776830 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22de59c640 1 --2- 172.21.15.47:0/1690237067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 0x555f92779360 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009a90 tx=0x7f22c802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d33bf8d4bfc1d31 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22bdffb640 1 -- 172.21.15.47:0/1690237067 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8003c50 con 0x555f92776830 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22bdffb640 1 -- 172.21.15.47:0/1690237067 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f22c8003db0 con 0x555f92776830 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22e7dc4740 1 -- 172.21.15.47:0/1690237067 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f926d9690 con 0x555f92776830 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22bdffb640 1 -- 172.21.15.47:0/1690237067 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22c8034440 con 0x555f92776830 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22e7dc4740 10 client.7197 fetch_fsmap learned FSMap version 39 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22e7dc4740 1 -- 172.21.15.47:0/1690237067 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x555f926ae4d0 con 0x555f92776830 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22bdffb640 1 -- 172.21.15.47:0/1690237067 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555f926d9690 con 0x555f92776830 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22bdffb640 1 -- 172.21.15.47:0/1690237067 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f22c80345a0 con 0x555f92776830 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22e7dc4740 10 client.7197 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22e7dc4740 10 client.7197 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22e7dc4740 20 client.7197 populate_metadata read hostname 'smithi047' 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/1690237067 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x555f927ac770 0x555f927aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22e7dc4740 4 client.7197 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:46:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22e7dc4740 1 -- 172.21.15.47:0/1690237067 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f926a1eb0 con 0x555f927ac770 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.155+0000 7f22ded9d640 1 --2- 172.21.15.47:0/1690237067 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x555f927ac770 0x555f927aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22bdffb640 10 client.7197 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22ded9d640 1 --2- 172.21.15.47:0/1690237067 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x555f927ac770 0x555f927aeb60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9176b0b948c062e0 server_cookie=40d7c35d275858fe in_seq=0 out_seq=0 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22bdffb640 1 -- 172.21.15.47:0/1690237067 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555f926a1eb0 con 0x555f927ac770 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22bdffb640 10 client.7197 handle_command_reply: tid=0 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22e7dc4740 1 client.7197 shutdown 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22e7dc4740 1 -- 172.21.15.47:0/1690237067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 msgr2=0x555f92779360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/1690237067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 0x555f92779360 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009a90 tx=0x7f22c802e070 comp rx=0 tx=0).stop 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22e7dc4740 1 -- 172.21.15.47:0/1690237067 shutdown_connections 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/1690237067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 0x555f927909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/1690237067 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x555f927ac770 0x555f927aeb60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/1690237067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 0x555f92779360 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/1690237067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f9274a3b0 0x555f92763bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22bdffb640 0 client.7197 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22e7dc4740 1 -- 172.21.15.47:0/1690237067 >> 172.21.15.47:0/1690237067 conn(0x555f92746850 msgr2=0x555f9277a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.156+0000 7f22e7dc4740 1 -- 172.21.15.47:0/1690237067 shutdown_connections 2024-03-23T09:46:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.157+0000 7f22e7dc4740 1 -- 172.21.15.47:0/1690237067 wait complete. 2024-03-23T09:46:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.157+0000 7f22e7dc4740 20 client.7197 trim_cache size 0 max 16384 2024-03-23T09:46:21.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.171+0000 7f22e7dc4740 1 Processor -- start 2024-03-23T09:46:21.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.171+0000 7f22e7dc4740 1 -- start start 2024-03-23T09:46:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22e7dc4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f9274a3b0 0x555f92778b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22e7dc4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 0x555f92779080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22e7dc4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 0x555f927aa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22e7dc4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f92795560 con 0x555f92776830 2024-03-23T09:46:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22e7dc4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f927b0ef0 con 0x555f927798a0 2024-03-23T09:46:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22e7dc4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f92746fb0 con 0x555f9274a3b0 2024-03-23T09:46:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22de59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 0x555f92779080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22de59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 0x555f92779080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41502/0 (socket says 172.21.15.47:41502) 2024-03-23T09:46:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22de59c640 1 -- 172.21.15.47:0/2234391770 learned_addr learned my addr 172.21.15.47:0/2234391770 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22ded9d640 1 --2- 172.21.15.47:0/2234391770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f9274a3b0 0x555f92778b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22de59c640 1 -- 172.21.15.47:0/2234391770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f9274a3b0 msgr2=0x555f92778b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22de59c640 1 --2- 172.21.15.47:0/2234391770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f9274a3b0 0x555f92778b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22de59c640 1 -- 172.21.15.47:0/2234391770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 msgr2=0x555f927aa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22de59c640 1 --2- 172.21.15.47:0/2234391770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 0x555f927aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22de59c640 1 -- 172.21.15.47:0/2234391770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f926e29e0 con 0x555f92776830 2024-03-23T09:46:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22de59c640 1 --2- 172.21.15.47:0/2234391770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 0x555f92779080 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f22c8004650 tx=0x7f22c8054000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d24f60656bc4ed server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22d57fa640 1 -- 172.21.15.47:0/2234391770 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8034ea0 con 0x555f92776830 2024-03-23T09:46:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22d57fa640 1 -- 172.21.15.47:0/2234391770 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c800f000 con 0x555f92776830 2024-03-23T09:46:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.172+0000 7f22d57fa640 1 -- 172.21.15.47:0/2234391770 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8034680 con 0x555f92776830 2024-03-23T09:46:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.173+0000 7f22e7dc4740 1 -- 172.21.15.47:0/2234391770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 msgr2=0x555f92779080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.173+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/2234391770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 0x555f92779080 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f22c8004650 tx=0x7f22c8054000 comp rx=0 tx=0).stop 2024-03-23T09:46:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.173+0000 7f22e7dc4740 1 -- 172.21.15.47:0/2234391770 shutdown_connections 2024-03-23T09:46:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.173+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/2234391770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 0x555f927aa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.173+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/2234391770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f92776830 0x555f92779080 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.173+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/2234391770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f9274a3b0 0x555f92778b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.173+0000 7f22e7dc4740 1 -- 172.21.15.47:0/2234391770 >> 172.21.15.47:0/2234391770 conn(0x555f92746850 msgr2=0x555f92745910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.173+0000 7f22e7dc4740 1 -- 172.21.15.47:0/2234391770 shutdown_connections 2024-03-23T09:46:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.173+0000 7f22e7dc4740 1 -- 172.21.15.47:0/2234391770 wait complete. 2024-03-23T09:46:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.174+0000 7f22e7dc4740 1 -- start start 2024-03-23T09:46:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.174+0000 7f22e7dc4740 1 Processor -- start 2024-03-23T09:46:21.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.174+0000 7f22e7dc4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f9274a3b0 0x555f92771340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.174+0000 7f22e7dc4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f92776830 0x555f92778b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.174+0000 7f22e7dc4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 0x555f92779080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.174+0000 7f22e7dc4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f927ab3c0 con 0x555f9274a3b0 2024-03-23T09:46:21.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.174+0000 7f22e7dc4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f92771e60 con 0x555f927798a0 2024-03-23T09:46:21.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.174+0000 7f22e7dc4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f927720b0 con 0x555f92776830 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.174+0000 7f22e7dc4740 10 client.0 _ll_register_callbacks cb 0x7f22c8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.174+0000 7f22ded9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f9274a3b0 0x555f92771340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.174+0000 7f22ded9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f9274a3b0 0x555f92771340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41514/0 (socket says 172.21.15.47:41514) 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.174+0000 7f22ded9d640 1 -- 172.21.15.47:0/3655674457 learned_addr learned my addr 172.21.15.47:0/3655674457 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22de59c640 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f92776830 0x555f92778b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22df59e640 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 0x555f92779080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22ded9d640 1 -- 172.21.15.47:0/3655674457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f92776830 msgr2=0x555f92778b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22ded9d640 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f92776830 0x555f92778b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22ded9d640 1 -- 172.21.15.47:0/3655674457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 msgr2=0x555f92779080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22ded9d640 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 0x555f92779080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22ded9d640 1 -- 172.21.15.47:0/3655674457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f926ae4d0 con 0x555f9274a3b0 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22de59c640 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f92776830 0x555f92778b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22df59e640 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 0x555f92779080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22ded9d640 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f9274a3b0 0x555f92771340 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f22d00012e0 tx=0x7f22d0001730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3780da1dfadd22e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:46:21.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22bdffb640 1 -- 172.21.15.47:0/3655674457 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d005c870 con 0x555f9274a3b0 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22bdffb640 1 -- 172.21.15.47:0/3655674457 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f22d0060e30 con 0x555f9274a3b0 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22e7dc4740 1 -- 172.21.15.47:0/3655674457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22c8003610 con 0x555f9274a3b0 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22bdffb640 1 -- 172.21.15.47:0/3655674457 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d0055150 con 0x555f9274a3b0 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22e7dc4740 10 client.7203 fetch_fsmap learned FSMap version 39 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22e7dc4740 1 -- 172.21.15.47:0/3655674457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x555f926505c0 con 0x555f9274a3b0 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22bdffb640 1 -- 172.21.15.47:0/3655674457 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c8003610 con 0x555f9274a3b0 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.175+0000 7f22bdffb640 1 -- 172.21.15.47:0/3655674457 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f22d00772f0 con 0x555f9274a3b0 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 10 client.7203 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 10 client.7203 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 20 client.7203 populate_metadata read hostname 'smithi047' 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x555f92771880 0x555f927aa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 4 client.7203 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 1 -- 172.21.15.47:0/3655674457 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555f923a34e0 con 0x555f92771880 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22de59c640 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x555f92771880 0x555f927aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22bdffb640 10 client.7203 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22de59c640 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x555f92771880 0x555f927aa320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f35e6c7b2ad21853 server_cookie=2df4ee969ee4ad71 in_seq=0 out_seq=0 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22bdffb640 1 -- 172.21.15.47:0/3655674457 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555f923a34e0 con 0x555f92771880 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22bdffb640 10 client.7203 handle_command_reply: tid=0 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 1 client.7203 shutdown 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 1 -- 172.21.15.47:0/3655674457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f9274a3b0 msgr2=0x555f92771340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f9274a3b0 0x555f92771340 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f22d00012e0 tx=0x7f22d0001730 comp rx=0 tx=0).stop 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 1 -- 172.21.15.47:0/3655674457 shutdown_connections 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f927798a0 0x555f92779080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f92776830 0x555f92778b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x555f92771880 0x555f927aa320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 1 --2- 172.21.15.47:0/3655674457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f9274a3b0 0x555f92771340 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22bdffb640 0 client.7203 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 1 -- 172.21.15.47:0/3655674457 >> 172.21.15.47:0/3655674457 conn(0x555f92746850 msgr2=0x555f92772810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 1 -- 172.21.15.47:0/3655674457 shutdown_connections 2024-03-23T09:46:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.176+0000 7f22e7dc4740 1 -- 172.21.15.47:0/3655674457 wait complete. 2024-03-23T09:46:21.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.177+0000 7f22e7dc4740 20 client.7203 trim_cache size 0 max 16384 2024-03-23T09:46:21.183 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:46:21.183 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-23T09:46:21.183 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-23T09:46:21.183 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:46:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.177+0000 7f22e524a640 1 -- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f22c006ac00 msgr2=0x7f22c006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.177+0000 7f22e524a640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f22c006ac00 0x7f22c006d0c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f22c8036000 tx=0x7f22c8030000 comp rx=0 tx=0).stop 2024-03-23T09:46:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.177+0000 7f22e524a640 1 -- 172.21.15.47:0/4114276086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22cc004370 msgr2=0x7f22e0153de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.177+0000 7f22e524a640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22cc004370 0x7f22e0153de0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f22d0005fd0 tx=0x7f22d000e530 comp rx=0 tx=0).stop 2024-03-23T09:46:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.177+0000 7f22e524a640 1 -- 172.21.15.47:0/4114276086 shutdown_connections 2024-03-23T09:46:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.177+0000 7f22e524a640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f22c006ac00 0x7f22c006d0c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.177+0000 7f22e524a640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e015a790 0x7f22e015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.177+0000 7f22e524a640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22e00b26c0 0x7f22e00b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.177+0000 7f22e524a640 1 --2- 172.21.15.47:0/4114276086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22cc004370 0x7f22e0153de0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.177+0000 7f22e524a640 1 -- 172.21.15.47:0/4114276086 >> 172.21.15.47:0/4114276086 conn(0x7f22e00659d0 msgr2=0x7f22e00b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.177+0000 7f22e524a640 1 -- 172.21.15.47:0/4114276086 shutdown_connections 2024-03-23T09:46:21.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.177+0000 7f22e524a640 1 -- 172.21.15.47:0/4114276086 wait complete. 2024-03-23T09:46:21.188 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T09:46:21.188 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:46:21.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.289+0000 7f7272270640 1 Processor -- start 2024-03-23T09:46:21.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.289+0000 7f7272270640 1 -- start start 2024-03-23T09:46:21.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.290+0000 7f7272270640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f726c15ca90 0x7f726c160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.290+0000 7f7272270640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f726c0b8350 0x7f726c0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.290+0000 7f7272270640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726c0ce090 con 0x7f726c0cdc40 2024-03-23T09:46:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.290+0000 7f7272270640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726c0d11e0 con 0x7f726c15ca90 2024-03-23T09:46:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.290+0000 7f7272270640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726c0d2680 con 0x7f726c0b8350 2024-03-23T09:46:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.290+0000 7f726bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f726c15ca90 0x7f726c160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.290+0000 7f727080d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f726c0cdc40 0x7f726c0b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51248/0 (socket says 172.21.15.47:51248) 2024-03-23T09:46:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.290+0000 7f727080d640 1 -- 172.21.15.47:0/3043020588 learned_addr learned my addr 172.21.15.47:0/3043020588 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.290+0000 7f726bfff640 1 -- 172.21.15.47:0/3043020588 >> v2:172.21.15.175:3300/0 conn(0x7f726c0b8350 msgr2=0x7f726c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.290+0000 7f726bfff640 1 --2- 172.21.15.47:0/3043020588 >> v2:172.21.15.175:3300/0 conn(0x7f726c0b8350 0x7f726c0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.290+0000 7f726bfff640 1 -- 172.21.15.47:0/3043020588 >> v1:172.21.15.47:6789/0 conn(0x7f726c0cdc40 legacy=0x7f726c0b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.290+0000 7f726bfff640 1 -- 172.21.15.47:0/3043020588 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c160f60 con 0x7f726c15ca90 2024-03-23T09:46:21.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.290+0000 7f726affd640 1 -- 172.21.15.47:0/3043020588 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289528514 0 0) 0x7f7264001490 con 0x7f726c0cdc40 2024-03-23T09:46:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.291+0000 7f726bfff640 1 --2- 172.21.15.47:0/3043020588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f726c15ca90 0x7f726c160710 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f726c160670 tx=0x7f725c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6eeee22652168a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.291+0000 7f726affd640 1 -- 172.21.15.47:0/3043020588 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c02e070 con 0x7f726c15ca90 2024-03-23T09:46:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.291+0000 7f726affd640 1 -- 172.21.15.47:0/3043020588 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c0315a0 con 0x7f726c15ca90 2024-03-23T09:46:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.291+0000 7f726affd640 1 -- 172.21.15.47:0/3043020588 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c031930 con 0x7f726c15ca90 2024-03-23T09:46:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.292+0000 7f7272270640 1 -- 172.21.15.47:0/3043020588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f726c15ca90 msgr2=0x7f726c160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.292+0000 7f7272270640 1 --2- 172.21.15.47:0/3043020588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f726c15ca90 0x7f726c160710 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f726c160670 tx=0x7f725c02b840 comp rx=0 tx=0).stop 2024-03-23T09:46:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.292+0000 7f7272270640 1 -- 172.21.15.47:0/3043020588 shutdown_connections 2024-03-23T09:46:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.292+0000 7f7272270640 1 --2- 172.21.15.47:0/3043020588 >> v2:172.21.15.175:3300/0 conn(0x7f726c0b8350 0x7f726c0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.292+0000 7f7272270640 1 --2- 172.21.15.47:0/3043020588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f726c15ca90 0x7f726c160710 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.292+0000 7f7272270640 1 -- 172.21.15.47:0/3043020588 >> 172.21.15.47:0/3043020588 conn(0x7f726c0659d0 msgr2=0x7f726c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.292+0000 7f7272270640 1 -- 172.21.15.47:0/3043020588 shutdown_connections 2024-03-23T09:46:21.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.292+0000 7f7272270640 1 -- 172.21.15.47:0/3043020588 wait complete. 2024-03-23T09:46:21.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.293+0000 7f7272270640 1 Processor -- start 2024-03-23T09:46:21.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.293+0000 7f7272270640 1 -- start start 2024-03-23T09:46:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f7272270640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f726c0b8350 0x7f726c151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f7272270640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f726c0cdc40 0x7f726c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f7272270640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f726c15ca90 0x7f726c0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f7272270640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c154d10 con 0x7f726c0b8350 2024-03-23T09:46:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f7272270640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c154ff0 con 0x7f726c0cdc40 2024-03-23T09:46:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f7272270640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726c1552d0 con 0x7f726c15ca90 2024-03-23T09:46:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f726bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f726c0b8350 0x7f726c151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f726bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f726c0b8350 0x7f726c151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41530/0 (socket says 172.21.15.47:41530) 2024-03-23T09:46:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f726bfff640 1 -- 172.21.15.47:0/661913578 learned_addr learned my addr 172.21.15.47:0/661913578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f726bfff640 1 -- 172.21.15.47:0/661913578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f726c15ca90 msgr2=0x7f726c0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f726bfff640 1 --2- 172.21.15.47:0/661913578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f726c15ca90 0x7f726c0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f726bfff640 1 -- 172.21.15.47:0/661913578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f726c0cdc40 msgr2=0x7f726c155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f726bfff640 1 --2- 172.21.15.47:0/661913578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f726c0cdc40 0x7f726c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f726bfff640 1 -- 172.21.15.47:0/661913578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c009400 con 0x7f726c0b8350 2024-03-23T09:46:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.294+0000 7f726bfff640 1 --2- 172.21.15.47:0/661913578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f726c0b8350 0x7f726c151060 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f725c034040 tx=0x7f725c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.295+0000 7f7268ff9640 1 -- 172.21.15.47:0/661913578 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c032900 con 0x7f726c0b8350 2024-03-23T09:46:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.295+0000 7f7268ff9640 1 -- 172.21.15.47:0/661913578 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c032a60 con 0x7f726c0b8350 2024-03-23T09:46:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.295+0000 7f7272270640 1 -- 172.21.15.47:0/661913578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726c1555b0 con 0x7f726c0b8350 2024-03-23T09:46:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.295+0000 7f7268ff9640 1 -- 172.21.15.47:0/661913578 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c032e10 con 0x7f726c0b8350 2024-03-23T09:46:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.295+0000 7f7272270640 1 -- 172.21.15.47:0/661913578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726c0cc9e0 con 0x7f726c0b8350 2024-03-23T09:46:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.296+0000 7f7272270640 1 -- 172.21.15.47:0/661913578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726c072360 con 0x7f726c0b8350 2024-03-23T09:46:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.296+0000 7f7268ff9640 1 -- 172.21.15.47:0/661913578 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f725c03b3f0 con 0x7f726c0b8350 2024-03-23T09:46:21.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.296+0000 7f7268ff9640 1 --2- 172.21.15.47:0/661913578 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f725806ac00 0x7f725806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.297+0000 7f7268ff9640 1 -- 172.21.15.47:0/661913578 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f725c0ab450 con 0x7f726c0b8350 2024-03-23T09:46:21.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.297+0000 7f726b7fe640 1 --2- 172.21.15.47:0/661913578 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f725806ac00 0x7f725806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.297+0000 7f726b7fe640 1 --2- 172.21.15.47:0/661913578 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f725806ac00 0x7f725806d0c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f726c1515a0 tx=0x7f7260006bf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.301+0000 7f7268ff9640 1 -- 172.21.15.47:0/661913578 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f725c076630 con 0x7f726c0b8350 2024-03-23T09:46:21.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.520+0000 7f7272270640 1 -- 172.21.15.47:0/661913578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f726c154e50 con 0x7f726c0b8350 2024-03-23T09:46:21.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.520+0000 7f7268ff9640 1 -- 172.21.15.47:0/661913578 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f726c154e50 con 0x7f726c0b8350 2024-03-23T09:46:21.520 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:46:21.521 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3862818105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3862818105},{"type":"v1","addr":"172.21.15.175:6836","nonce":3862818105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3615592850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3615592850},{"type":"v1","addr":"172.21.15.175:6837","nonce":3615592850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/832829684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":832829684},{"type":"v1","addr":"172.21.15.47:6834","nonce":832829684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:45:18.296714+0000","modified":"2024-03-23T09:45:44.820490+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7005":{"gid":7005,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":6,"addr":"172.21.15.47:6835/4141889683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4141889683},{"type":"v1","addr":"172.21.15.47:6835","nonce":4141889683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-23T09:46:21.521 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 39 2024-03-23T09:46:21.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.522+0000 7f7272270640 1 -- 172.21.15.47:0/661913578 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f725806ac00 msgr2=0x7f725806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.522+0000 7f7272270640 1 --2- 172.21.15.47:0/661913578 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f725806ac00 0x7f725806d0c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f726c1515a0 tx=0x7f7260006bf0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.522+0000 7f7272270640 1 -- 172.21.15.47:0/661913578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f726c0b8350 msgr2=0x7f726c151060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.522+0000 7f7272270640 1 --2- 172.21.15.47:0/661913578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f726c0b8350 0x7f726c151060 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f725c034040 tx=0x7f725c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.522+0000 7f7272270640 1 -- 172.21.15.47:0/661913578 shutdown_connections 2024-03-23T09:46:21.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.522+0000 7f7272270640 1 --2- 172.21.15.47:0/661913578 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f725806ac00 0x7f725806d0c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.522+0000 7f7272270640 1 --2- 172.21.15.47:0/661913578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f726c15ca90 0x7f726c0cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.522+0000 7f7272270640 1 --2- 172.21.15.47:0/661913578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f726c0cdc40 0x7f726c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.522+0000 7f7272270640 1 --2- 172.21.15.47:0/661913578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f726c0b8350 0x7f726c151060 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.522+0000 7f7272270640 1 -- 172.21.15.47:0/661913578 >> 172.21.15.47:0/661913578 conn(0x7f726c0659d0 msgr2=0x7f726c0b6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.522+0000 7f7272270640 1 -- 172.21.15.47:0/661913578 shutdown_connections 2024-03-23T09:46:21.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.522+0000 7f7272270640 1 -- 172.21.15.47:0/661913578 wait complete. 2024-03-23T09:46:21.533 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-23T09:46:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.636+0000 7f7bc1927640 1 Processor -- start 2024-03-23T09:46:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.636+0000 7f7bc1927640 1 -- start start 2024-03-23T09:46:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bc1927640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7bbc15cee0 0x7f7bbc0d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bc1927640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bbc0d2f80 0x7f7bbc0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bc1927640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bbc0d1300 con 0x7f7bbc0d23a0 2024-03-23T09:46:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bc1927640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bbc0d15e0 con 0x7f7bbc15cee0 2024-03-23T09:46:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bc1927640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc0d18c0 con 0x7f7bbc0d2f80 2024-03-23T09:46:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bbb7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bbc0d2f80 0x7f7bbc0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bbaffd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7bbc15cee0 0x7f7bbc0d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bbaffd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7bbc15cee0 0x7f7bbc0d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:33894/0 (socket says 172.21.15.47:33894) 2024-03-23T09:46:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bb3fff640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f7bbc0d23a0 0x7f7bbc0d2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:55152/0 (socket says 172.21.15.47:55152) 2024-03-23T09:46:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bbb7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bbc0d2f80 0x7f7bbc0b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:47828/0 (socket says 172.21.15.47:47828) 2024-03-23T09:46:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bbaffd640 1 -- 172.21.15.47:0/1645711326 learned_addr learned my addr 172.21.15.47:0/1645711326 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bbaffd640 1 -- 172.21.15.47:0/1645711326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bbc0d2f80 msgr2=0x7f7bbc0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/1645711326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bbc0d2f80 0x7f7bbc0b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bbaffd640 1 -- 172.21.15.47:0/1645711326 >> v1:172.21.15.175:6789/0 conn(0x7f7bbc0d23a0 legacy=0x7f7bbc0d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bbaffd640 1 -- 172.21.15.47:0/1645711326 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc0d1ba0 con 0x7f7bbc15cee0 2024-03-23T09:46:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.637+0000 7f7bba7fc640 1 -- 172.21.15.47:0/1645711326 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767077082 0 0) 0x7f7bbc0d1300 con 0x7f7bbc0d23a0 2024-03-23T09:46:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.638+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/1645711326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bbc15cee0 0x7f7bbc0d0990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009870 tx=0x7f7bac02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4616ab8c5c5788f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.638+0000 7f7bba7fc640 1 -- 172.21.15.47:0/1645711326 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac030840 con 0x7f7bbc15cee0 2024-03-23T09:46:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.638+0000 7f7bba7fc640 1 -- 172.21.15.47:0/1645711326 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac0309a0 con 0x7f7bbc15cee0 2024-03-23T09:46:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.638+0000 7f7bba7fc640 1 -- 172.21.15.47:0/1645711326 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac030d30 con 0x7f7bbc15cee0 2024-03-23T09:46:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.639+0000 7f7bc1927640 1 -- 172.21.15.47:0/1645711326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bbc15cee0 msgr2=0x7f7bbc0d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.639+0000 7f7bc1927640 1 --2- 172.21.15.47:0/1645711326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bbc15cee0 0x7f7bbc0d0990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009870 tx=0x7f7bac02b980 comp rx=0 tx=0).stop 2024-03-23T09:46:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.639+0000 7f7bc1927640 1 -- 172.21.15.47:0/1645711326 shutdown_connections 2024-03-23T09:46:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.639+0000 7f7bc1927640 1 --2- 172.21.15.47:0/1645711326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bbc0d2f80 0x7f7bbc0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.639+0000 7f7bc1927640 1 --2- 172.21.15.47:0/1645711326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bbc15cee0 0x7f7bbc0d0990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.639+0000 7f7bc1927640 1 -- 172.21.15.47:0/1645711326 >> 172.21.15.47:0/1645711326 conn(0x7f7bbc065df0 msgr2=0x7f7bbc066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.639+0000 7f7bc1927640 1 -- 172.21.15.47:0/1645711326 shutdown_connections 2024-03-23T09:46:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.639+0000 7f7bc1927640 1 -- 172.21.15.47:0/1645711326 wait complete. 2024-03-23T09:46:21.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.640+0000 7f7bc1927640 1 Processor -- start 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.640+0000 7f7bc1927640 1 -- start start 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bc1927640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bbc0d23a0 0x7f7bbc14e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bc1927640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bbc0d2f80 0x7f7bbc0c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bc1927640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bbc15cee0 0x7f7bbc0c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bc1927640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc0c99b0 con 0x7f7bbc0d2f80 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bc1927640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc14e5f0 con 0x7f7bbc0d23a0 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bc1927640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc0c9c00 con 0x7f7bbc15cee0 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bb3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bbc0d2f80 0x7f7bbc0c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bb3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bbc0d2f80 0x7f7bbc0c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41532/0 (socket says 172.21.15.47:41532) 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bb3fff640 1 -- 172.21.15.47:0/308745560 learned_addr learned my addr 172.21.15.47:0/308745560 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bbc0d23a0 0x7f7bbc14e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bbb7fe640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bbc15cee0 0x7f7bbc0c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bb3fff640 1 -- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bbc15cee0 msgr2=0x7f7bbc0c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bb3fff640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bbc15cee0 0x7f7bbc0c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bb3fff640 1 -- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bbc0d23a0 msgr2=0x7f7bbc14e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bb3fff640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bbc0d23a0 0x7f7bbc14e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bb3fff640 1 -- 172.21.15.47:0/308745560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac0094d0 con 0x7f7bbc0d2f80 2024-03-23T09:46:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bbb7fe640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bbc15cee0 0x7f7bbc0c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.641+0000 7f7bb3fff640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bbc0d2f80 0x7f7bbc0c7440 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400c4b0 tx=0x7f7ba400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.642+0000 7f7bb8ff9640 1 -- 172.21.15.47:0/308745560 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40074e0 con 0x7f7bbc0d2f80 2024-03-23T09:46:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.642+0000 7f7bb8ff9640 1 -- 172.21.15.47:0/308745560 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba4007660 con 0x7f7bbc0d2f80 2024-03-23T09:46:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.642+0000 7f7bc1927640 1 -- 172.21.15.47:0/308745560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc0c9ee0 con 0x7f7bbc0d2f80 2024-03-23T09:46:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.642+0000 7f7bb8ff9640 1 -- 172.21.15.47:0/308745560 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4004a20 con 0x7f7bbc0d2f80 2024-03-23T09:46:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.642+0000 7f7bc1927640 1 -- 172.21.15.47:0/308745560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc0ca4a0 con 0x7f7bbc0d2f80 2024-03-23T09:46:21.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.643+0000 7f7bb8ff9640 1 -- 172.21.15.47:0/308745560 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7ba4004b80 con 0x7f7bbc0d2f80 2024-03-23T09:46:21.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.644+0000 7f7bb8ff9640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b9c06ac30 0x7f7b9c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.644+0000 7f7bb8ff9640 1 -- 172.21.15.47:0/308745560 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7ba4088820 con 0x7f7bbc0d2f80 2024-03-23T09:46:21.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.644+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b9c06ac30 0x7f7b9c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.645+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b9c06ac30 0x7f7b9c06d0f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7bac036000 tx=0x7f7bac008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.645+0000 7f7bc4571740 1 Processor -- start 2024-03-23T09:46:21.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.645+0000 7f7bc4571740 1 -- start start 2024-03-23T09:46:21.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bc4571740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dab36b0 0x55f76dab3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bc4571740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da836d0 0x55f76dab0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bc4571740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 0x55f76dab0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bc4571740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76d9f46a0 con 0x55f76da87230 2024-03-23T09:46:21.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bc4571740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76d9f1c70 con 0x55f76dab36b0 2024-03-23T09:46:21.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bc4571740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f76d9ed560 con 0x55f76da836d0 2024-03-23T09:46:21.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bbb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 0x55f76dab0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bbb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 0x55f76dab0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41546/0 (socket says 172.21.15.47:41546) 2024-03-23T09:46:21.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bbb7fe640 1 -- 172.21.15.47:0/2581797586 learned_addr learned my addr 172.21.15.47:0/2581797586 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bb3fff640 1 --2- 172.21.15.47:0/2581797586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da836d0 0x55f76dab0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/2581797586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dab36b0 0x55f76dab3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bbb7fe640 1 -- 172.21.15.47:0/2581797586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da836d0 msgr2=0x55f76dab0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bbb7fe640 1 --2- 172.21.15.47:0/2581797586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da836d0 0x55f76dab0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bbb7fe640 1 -- 172.21.15.47:0/2581797586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dab36b0 msgr2=0x55f76dab3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bbb7fe640 1 --2- 172.21.15.47:0/2581797586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dab36b0 0x55f76dab3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bbb7fe640 1 -- 172.21.15.47:0/2581797586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f76d9b4d50 con 0x55f76da87230 2024-03-23T09:46:21.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bb3fff640 1 --2- 172.21.15.47:0/2581797586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da836d0 0x55f76dab0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:21.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/2581797586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dab36b0 0x55f76dab3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bbb7fe640 1 --2- 172.21.15.47:0/2581797586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 0x55f76dab0e60 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4007680 tx=0x7f7bb4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5198c086c0ae1652 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bb17fa640 1 -- 172.21.15.47:0/2581797586 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400e030 con 0x55f76da87230 2024-03-23T09:46:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bb17fa640 1 -- 172.21.15.47:0/2581797586 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb4016680 con 0x55f76da87230 2024-03-23T09:46:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.646+0000 7f7bb17fa640 1 -- 172.21.15.47:0/2581797586 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4009d70 con 0x55f76da87230 2024-03-23T09:46:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.647+0000 7f7bc4571740 1 -- 172.21.15.47:0/2581797586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 msgr2=0x55f76dab0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.647+0000 7f7bc4571740 1 --2- 172.21.15.47:0/2581797586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 0x55f76dab0e60 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4007680 tx=0x7f7bb4009750 comp rx=0 tx=0).stop 2024-03-23T09:46:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.647+0000 7f7bc4571740 1 -- 172.21.15.47:0/2581797586 shutdown_connections 2024-03-23T09:46:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.647+0000 7f7bc4571740 1 --2- 172.21.15.47:0/2581797586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 0x55f76dab0e60 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.647+0000 7f7bc4571740 1 --2- 172.21.15.47:0/2581797586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da836d0 0x55f76dab0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.647+0000 7f7bc4571740 1 --2- 172.21.15.47:0/2581797586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dab36b0 0x55f76dab3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.647+0000 7f7bc4571740 1 -- 172.21.15.47:0/2581797586 >> 172.21.15.47:0/2581797586 conn(0x55f76da85940 msgr2=0x55f76dace4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.647+0000 7f7bc4571740 1 -- 172.21.15.47:0/2581797586 shutdown_connections 2024-03-23T09:46:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.647+0000 7f7bc4571740 1 -- 172.21.15.47:0/2581797586 wait complete. 2024-03-23T09:46:21.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.648+0000 7f7bc4571740 1 -- start start 2024-03-23T09:46:21.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.648+0000 7f7bc4571740 1 Processor -- start 2024-03-23T09:46:21.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.648+0000 7f7bc4571740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 0x55f76dab6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.648+0000 7f7bc4571740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76dab36b0 0x55f76dab6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bc4571740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dae12a0 0x55f76daac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bc4571740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76d6c63f0 con 0x55f76da87230 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bc4571740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76d7763d0 con 0x55f76dae12a0 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bc4571740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f76d970a10 con 0x55f76dab36b0 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bc4571740 10 client.0 _ll_register_callbacks cb 0x55f76d9e6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bbaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 0x55f76dab6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bbaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 0x55f76dab6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41558/0 (socket says 172.21.15.47:41558) 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bbaffd640 1 -- 172.21.15.47:0/3221720402 learned_addr learned my addr 172.21.15.47:0/3221720402 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bb3fff640 1 --2- 172.21.15.47:0/3221720402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76dab36b0 0x55f76dab6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bbb7fe640 1 --2- 172.21.15.47:0/3221720402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dae12a0 0x55f76daac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bbaffd640 1 -- 172.21.15.47:0/3221720402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76dab36b0 msgr2=0x55f76dab6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/3221720402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76dab36b0 0x55f76dab6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bbaffd640 1 -- 172.21.15.47:0/3221720402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dae12a0 msgr2=0x55f76daac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/3221720402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dae12a0 0x55f76daac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bbaffd640 1 -- 172.21.15.47:0/3221720402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb4007330 con 0x55f76da87230 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/3221720402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 0x55f76dab6510 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7bac02bf90 tx=0x7f7bac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a67396b743b057f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7b9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7b9a7fc640 1 -- 172.21.15.47:0/3221720402 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac039ae0 con 0x55f76da87230 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7b9a7fc640 1 -- 172.21.15.47:0/3221720402 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7bac03e040 con 0x55f76da87230 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bc4571740 1 -- 172.21.15.47:0/3221720402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f76da16150 con 0x55f76da87230 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7b9a7fc640 1 -- 172.21.15.47:0/3221720402 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bac039c40 con 0x55f76da87230 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bc4571740 10 client.7221 fetch_fsmap learned FSMap version 39 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.649+0000 7f7bc4571740 1 -- 172.21.15.47:0/3221720402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55f76d9af760 con 0x55f76da87230 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.650+0000 7f7b9a7fc640 1 -- 172.21.15.47:0/3221720402 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f76da16150 con 0x55f76da87230 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.650+0000 7f7b9a7fc640 1 -- 172.21.15.47:0/3221720402 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7bac042050 con 0x55f76da87230 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.650+0000 7f7bc4571740 10 client.7221 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.650+0000 7f7bc4571740 10 client.7221 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.650+0000 7f7bc4571740 20 client.7221 populate_metadata read hostname 'smithi047' 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.650+0000 7f7bc4571740 1 --2- 172.21.15.47:0/3221720402 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55f76dae98b0 0x55f76daebca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.650+0000 7f7bc4571740 4 client.7221 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.650+0000 7f7bc4571740 1 -- 172.21.15.47:0/3221720402 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f76d9f8650 con 0x55f76dae98b0 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.650+0000 7f7bb3fff640 1 --2- 172.21.15.47:0/3221720402 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55f76dae98b0 0x55f76daebca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.650+0000 7f7b9a7fc640 10 client.7221 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.650+0000 7f7bb3fff640 1 --2- 172.21.15.47:0/3221720402 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55f76dae98b0 0x55f76daebca0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a081d17dd3bc789b server_cookie=c5f22d932556c106 in_seq=0 out_seq=0 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7b9a7fc640 1 -- 172.21.15.47:0/3221720402 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f76d9f8650 con 0x55f76dae98b0 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7b9a7fc640 10 client.7221 handle_command_reply: tid=0 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7bc4571740 1 client.7221 shutdown 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7bc4571740 1 -- 172.21.15.47:0/3221720402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 msgr2=0x55f76dab6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7bc4571740 1 --2- 172.21.15.47:0/3221720402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 0x55f76dab6510 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7bac02bf90 tx=0x7f7bac004730 comp rx=0 tx=0).stop 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7bc4571740 1 -- 172.21.15.47:0/3221720402 shutdown_connections 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7bc4571740 1 --2- 172.21.15.47:0/3221720402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dae12a0 0x55f76daac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7bc4571740 1 --2- 172.21.15.47:0/3221720402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76dab36b0 0x55f76dab6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7bc4571740 1 --2- 172.21.15.47:0/3221720402 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55f76dae98b0 0x55f76daebca0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7bc4571740 1 --2- 172.21.15.47:0/3221720402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76da87230 0x55f76dab6510 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7b9a7fc640 0 client.7221 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:21.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7bc4571740 1 -- 172.21.15.47:0/3221720402 >> 172.21.15.47:0/3221720402 conn(0x55f76da836d0 msgr2=0x55f76da87ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7bc4571740 1 -- 172.21.15.47:0/3221720402 shutdown_connections 2024-03-23T09:46:21.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7bc4571740 1 -- 172.21.15.47:0/3221720402 wait complete. 2024-03-23T09:46:21.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.651+0000 7f7bc4571740 20 client.7221 trim_cache size 0 max 16384 2024-03-23T09:46:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.662+0000 7f7bc4571740 1 Processor -- start 2024-03-23T09:46:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.662+0000 7f7bc4571740 1 -- start start 2024-03-23T09:46:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.663+0000 7f7bc4571740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da87230 0x55f76dad20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.663+0000 7f7bc4571740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dab36b0 0x55f76da972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.663+0000 7f7bc4571740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dae12a0 0x55f76daab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.663+0000 7f7bc4571740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76daa81e0 con 0x55f76dab36b0 2024-03-23T09:46:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.663+0000 7f7bc4571740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76da97800 con 0x55f76dae12a0 2024-03-23T09:46:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.663+0000 7f7bc4571740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f76dae0960 con 0x55f76da87230 2024-03-23T09:46:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.663+0000 7f7bb3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dab36b0 0x55f76da972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.663+0000 7f7bb3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dab36b0 0x55f76da972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41560/0 (socket says 172.21.15.47:41560) 2024-03-23T09:46:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.663+0000 7f7bb3fff640 1 -- 172.21.15.47:0/613829898 learned_addr learned my addr 172.21.15.47:0/613829898 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.663+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/613829898 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da87230 0x55f76dad20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.664+0000 7f7bbb7fe640 1 --2- 172.21.15.47:0/613829898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dae12a0 0x55f76daab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.664+0000 7f7bb3fff640 1 -- 172.21.15.47:0/613829898 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da87230 msgr2=0x55f76dad20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.664+0000 7f7bb3fff640 1 --2- 172.21.15.47:0/613829898 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da87230 0x55f76dad20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.664+0000 7f7bb3fff640 1 -- 172.21.15.47:0/613829898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dae12a0 msgr2=0x55f76daab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.664+0000 7f7bb3fff640 1 --2- 172.21.15.47:0/613829898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dae12a0 0x55f76daab7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.664+0000 7f7bb3fff640 1 -- 172.21.15.47:0/613829898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f76d98dad0 con 0x55f76dab36b0 2024-03-23T09:46:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.664+0000 7f7bbb7fe640 1 --2- 172.21.15.47:0/613829898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dae12a0 0x55f76daab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.664+0000 7f7bb3fff640 1 --2- 172.21.15.47:0/613829898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dab36b0 0x55f76da972c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40574a0 tx=0x7f7ba405b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a801f2690f9e86e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.664+0000 7f7bb1ffb640 1 -- 172.21.15.47:0/613829898 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40558e0 con 0x55f76dab36b0 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.664+0000 7f7bb1ffb640 1 -- 172.21.15.47:0/613829898 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba4055b40 con 0x55f76dab36b0 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.664+0000 7f7bb1ffb640 1 -- 172.21.15.47:0/613829898 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4056c90 con 0x55f76dab36b0 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.664+0000 7f7bc4571740 1 -- 172.21.15.47:0/613829898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dab36b0 msgr2=0x55f76da972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.664+0000 7f7bc4571740 1 --2- 172.21.15.47:0/613829898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dab36b0 0x55f76da972c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40574a0 tx=0x7f7ba405b040 comp rx=0 tx=0).stop 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.665+0000 7f7bc4571740 1 -- 172.21.15.47:0/613829898 shutdown_connections 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.665+0000 7f7bc4571740 1 --2- 172.21.15.47:0/613829898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dae12a0 0x55f76daab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.665+0000 7f7bc4571740 1 --2- 172.21.15.47:0/613829898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dab36b0 0x55f76da972c0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.665+0000 7f7bc4571740 1 --2- 172.21.15.47:0/613829898 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da87230 0x55f76dad20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.665+0000 7f7bc4571740 1 -- 172.21.15.47:0/613829898 >> 172.21.15.47:0/613829898 conn(0x55f76da836d0 msgr2=0x55f76daa8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.665+0000 7f7bc4571740 1 -- 172.21.15.47:0/613829898 shutdown_connections 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.665+0000 7f7bc4571740 1 -- 172.21.15.47:0/613829898 wait complete. 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.665+0000 7f7bc4571740 1 -- start start 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.665+0000 7f7bc4571740 1 Processor -- start 2024-03-23T09:46:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bc4571740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da87230 0x55f76daab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bc4571740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dab36b0 0x55f76dab6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bc4571740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dae12a0 0x55f76dab6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bc4571740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76daac0f0 con 0x55f76dae12a0 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bc4571740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76dab0f10 con 0x55f76dab36b0 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bc4571740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f76daabe30 con 0x55f76da87230 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bc4571740 10 client.0 _ll_register_callbacks cb 0x7f7bac0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bbb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dae12a0 0x55f76dab6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bbb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dae12a0 0x55f76dab6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41562/0 (socket says 172.21.15.47:41562) 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bbb7fe640 1 -- 172.21.15.47:0/1440491756 learned_addr learned my addr 172.21.15.47:0/1440491756 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/1440491756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da87230 0x55f76daab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bbb7fe640 1 -- 172.21.15.47:0/1440491756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da87230 msgr2=0x55f76daab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bbb7fe640 1 --2- 172.21.15.47:0/1440491756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da87230 0x55f76daab8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bbb7fe640 1 -- 172.21.15.47:0/1440491756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dab36b0 msgr2=0x55f76dab6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bb3fff640 1 --2- 172.21.15.47:0/1440491756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dab36b0 0x55f76dab6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bbb7fe640 1 --2- 172.21.15.47:0/1440491756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dab36b0 0x55f76dab6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bbb7fe640 1 -- 172.21.15.47:0/1440491756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc0ca4a0 con 0x55f76dae12a0 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7bbb7fe640 1 --2- 172.21.15.47:0/1440491756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dae12a0 0x55f76dab6950 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4007130 tx=0x7f7bb4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9148a93a2d9d41a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.666+0000 7f7b9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7b9a7fc640 1 -- 172.21.15.47:0/1440491756 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4024070 con 0x55f76dae12a0 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7b9a7fc640 1 -- 172.21.15.47:0/1440491756 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7bb400f070 con 0x55f76dae12a0 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7bc4571740 1 -- 172.21.15.47:0/1440491756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bb4016680 con 0x55f76dae12a0 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7b9a7fc640 1 -- 172.21.15.47:0/1440491756 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bb400e040 con 0x55f76dae12a0 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7bc4571740 10 client.7227 fetch_fsmap learned FSMap version 39 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7bc4571740 1 -- 172.21.15.47:0/1440491756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55f76d9b35b0 con 0x55f76dae12a0 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7b9a7fc640 1 -- 172.21.15.47:0/1440491756 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb4016680 con 0x55f76dae12a0 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7b9a7fc640 1 -- 172.21.15.47:0/1440491756 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7bb4014070 con 0x55f76dae12a0 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7bc4571740 10 client.7227 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7bc4571740 10 client.7227 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7bc4571740 20 client.7227 populate_metadata read hostname 'smithi047' 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7bc4571740 1 --2- 172.21.15.47:0/1440491756 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55f76da88a60 0x55f76da88e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7bc4571740 4 client.7227 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7bc4571740 1 -- 172.21.15.47:0/1440491756 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55f76d69e8d0 con 0x55f76da88a60 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/1440491756 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55f76da88a60 0x55f76da88e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7b9a7fc640 10 client.7227 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.667+0000 7f7bbaffd640 1 --2- 172.21.15.47:0/1440491756 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55f76da88a60 0x55f76da88e40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f1a3e3eeb709cc4 server_cookie=ee24738dfdaed775 in_seq=0 out_seq=0 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7b9a7fc640 1 -- 172.21.15.47:0/1440491756 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+237 (crc 0 0 0) 0x55f76d69e8d0 con 0x55f76da88a60 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7b9a7fc640 10 client.7227 handle_command_reply: tid=0 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc4571740 1 client.7227 shutdown 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc4571740 1 -- 172.21.15.47:0/1440491756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dae12a0 msgr2=0x55f76dab6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc4571740 1 --2- 172.21.15.47:0/1440491756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dae12a0 0x55f76dab6950 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4007130 tx=0x7f7bb4004ab0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc4571740 1 -- 172.21.15.47:0/1440491756 shutdown_connections 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc4571740 1 --2- 172.21.15.47:0/1440491756 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55f76da88a60 0x55f76da88e40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc4571740 1 --2- 172.21.15.47:0/1440491756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76dae12a0 0x55f76dab6950 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc4571740 1 --2- 172.21.15.47:0/1440491756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f76dab36b0 0x55f76dab6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7b9a7fc640 0 client.7227 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc4571740 1 --2- 172.21.15.47:0/1440491756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f76da87230 0x55f76daab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc4571740 1 -- 172.21.15.47:0/1440491756 >> 172.21.15.47:0/1440491756 conn(0x55f76da836d0 msgr2=0x55f76da897a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc4571740 1 -- 172.21.15.47:0/1440491756 shutdown_connections 2024-03-23T09:46:21.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc4571740 1 -- 172.21.15.47:0/1440491756 wait complete. 2024-03-23T09:46:21.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc4571740 20 client.7227 trim_cache size 0 max 16384 2024-03-23T09:46:21.677 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:46:21.677 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-23T09:46:21.677 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:46:21.677 INFO:teuthology.orchestra.run.smithi047.stdout: "damage_type": "dentry", 2024-03-23T09:46:21.677 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 1024055635, 2024-03-23T09:46:21.677 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 1099511627776, 2024-03-23T09:46:21.677 INFO:teuthology.orchestra.run.smithi047.stdout: "frag": "*", 2024-03-23T09:46:21.677 INFO:teuthology.orchestra.run.smithi047.stdout: "dname": "file_to_be_damaged", 2024-03-23T09:46:21.678 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_id": "head", 2024-03-23T09:46:21.678 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/subdir/file_to_be_damaged" 2024-03-23T09:46:21.678 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:46:21.678 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-23T09:46:21.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc1927640 1 -- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b9c06ac30 msgr2=0x7f7b9c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc1927640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b9c06ac30 0x7f7b9c06d0f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7bac036000 tx=0x7f7bac008030 comp rx=0 tx=0).stop 2024-03-23T09:46:21.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc1927640 1 -- 172.21.15.47:0/308745560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bbc0d2f80 msgr2=0x7f7bbc0c7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc1927640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bbc0d2f80 0x7f7bbc0c7440 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400c4b0 tx=0x7f7ba400c970 comp rx=0 tx=0).stop 2024-03-23T09:46:21.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc1927640 1 -- 172.21.15.47:0/308745560 shutdown_connections 2024-03-23T09:46:21.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc1927640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b9c06ac30 0x7f7b9c06d0f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc1927640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bbc15cee0 0x7f7bbc0c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc1927640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bbc0d2f80 0x7f7bbc0c7440 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc1927640 1 --2- 172.21.15.47:0/308745560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bbc0d23a0 0x7f7bbc14e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.668+0000 7f7bc1927640 1 -- 172.21.15.47:0/308745560 >> 172.21.15.47:0/308745560 conn(0x7f7bbc065df0 msgr2=0x7f7bbc15c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.669+0000 7f7bc1927640 1 -- 172.21.15.47:0/308745560 shutdown_connections 2024-03-23T09:46:21.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.669+0000 7f7bc1927640 1 -- 172.21.15.47:0/308745560 wait complete. 2024-03-23T09:46:21.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-23T09:46:21.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.780+0000 7f5faf248640 1 Processor -- start 2024-03-23T09:46:21.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.780+0000 7f5faf248640 1 -- start start 2024-03-23T09:46:21.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.781+0000 7f5faf248640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5fa80b8360 0x7f5fa815d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.781+0000 7f5faf248640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5fa80b8740 0x7f5fa80cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.781+0000 7f5faf248640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa80cfdd0 con 0x7f5fa80cf960 2024-03-23T09:46:21.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.781+0000 7f5faf248640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa80b45e0 con 0x7f5fa80b8360 2024-03-23T09:46:21.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.781+0000 7f5faf248640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa80b48c0 con 0x7f5fa80b8740 2024-03-23T09:46:21.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.781+0000 7f5facfe4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5fa80b8360 0x7f5fa815d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.781+0000 7f5facfe4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5fa80b8360 0x7f5fa815d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41570/0 (socket says 172.21.15.47:41570) 2024-03-23T09:46:21.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.781+0000 7f5facfe4640 1 -- 172.21.15.47:0/1229566399 learned_addr learned my addr 172.21.15.47:0/1229566399 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.781+0000 7f5fa7fff640 1 --2- 172.21.15.47:0/1229566399 >> v2:172.21.15.175:3300/0 conn(0x7f5fa80b8740 0x7f5fa80cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.782+0000 7f5facfe4640 1 -- 172.21.15.47:0/1229566399 >> v2:172.21.15.175:3300/0 conn(0x7f5fa80b8740 msgr2=0x7f5fa80cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.782+0000 7f5facfe4640 1 --2- 172.21.15.47:0/1229566399 >> v2:172.21.15.175:3300/0 conn(0x7f5fa80b8740 0x7f5fa80cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.782+0000 7f5facfe4640 1 -- 172.21.15.47:0/1229566399 >> v1:172.21.15.175:6789/0 conn(0x7f5fa80cf960 legacy=0x7f5fa80cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:21.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.782+0000 7f5facfe4640 1 -- 172.21.15.47:0/1229566399 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa80b4ba0 con 0x7f5fa80b8360 2024-03-23T09:46:21.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.782+0000 7f5fa7fff640 1 --2- 172.21.15.47:0/1229566399 >> v2:172.21.15.175:3300/0 conn(0x7f5fa80b8740 0x7f5fa80cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.782+0000 7f5facfe4640 1 --2- 172.21.15.47:0/1229566399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fa80b8360 0x7f5fa815d0c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80d0540 tx=0x7f5fa002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f563709bd1622ef0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.782+0000 7f5fa77fe640 1 -- 172.21.15.47:0/1229566399 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa002c9f0 con 0x7f5fa80b8360 2024-03-23T09:46:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.782+0000 7f5fa77fe640 1 -- 172.21.15.47:0/1229566399 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa002cb50 con 0x7f5fa80b8360 2024-03-23T09:46:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.783+0000 7f5fa77fe640 1 -- 172.21.15.47:0/1229566399 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00343f0 con 0x7f5fa80b8360 2024-03-23T09:46:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.783+0000 7f5faf248640 1 -- 172.21.15.47:0/1229566399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fa80b8360 msgr2=0x7f5fa815d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.783+0000 7f5faf248640 1 --2- 172.21.15.47:0/1229566399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fa80b8360 0x7f5fa815d0c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80d0540 tx=0x7f5fa002be50 comp rx=0 tx=0).stop 2024-03-23T09:46:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.783+0000 7f5faf248640 1 -- 172.21.15.47:0/1229566399 shutdown_connections 2024-03-23T09:46:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.783+0000 7f5faf248640 1 --2- 172.21.15.47:0/1229566399 >> v2:172.21.15.175:3300/0 conn(0x7f5fa80b8740 0x7f5fa80cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.783+0000 7f5faf248640 1 --2- 172.21.15.47:0/1229566399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fa80b8360 0x7f5fa815d0c0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.783+0000 7f5faf248640 1 -- 172.21.15.47:0/1229566399 >> 172.21.15.47:0/1229566399 conn(0x7f5fa80659d0 msgr2=0x7f5fa8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:21.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.783+0000 7f5faf248640 1 -- 172.21.15.47:0/1229566399 shutdown_connections 2024-03-23T09:46:21.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.783+0000 7f5faf248640 1 -- 172.21.15.47:0/1229566399 wait complete. 2024-03-23T09:46:21.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.784+0000 7f5faf248640 1 Processor -- start 2024-03-23T09:46:21.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.784+0000 7f5faf248640 1 -- start start 2024-03-23T09:46:21.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.785+0000 7f5faf248640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa80b8360 0x7f5fa80cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.785+0000 7f5faf248640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa80b8740 0x7f5fa80d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.785+0000 7f5faf248640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fa80cf960 0x7f5fa81f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.785+0000 7f5faf248640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80cb2d0 con 0x7f5fa80cf960 2024-03-23T09:46:21.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.785+0000 7f5faf248640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80cb570 con 0x7f5fa80b8740 2024-03-23T09:46:21.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.785+0000 7f5faf248640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80cb850 con 0x7f5fa80b8360 2024-03-23T09:46:21.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.785+0000 7f5fad7e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fa80cf960 0x7f5fa81f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.785+0000 7f5fad7e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fa80cf960 0x7f5fa81f4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41586/0 (socket says 172.21.15.47:41586) 2024-03-23T09:46:21.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.785+0000 7f5fad7e5640 1 -- 172.21.15.47:0/1642660153 learned_addr learned my addr 172.21.15.47:0/1642660153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.786+0000 7f5fa7fff640 1 --2- 172.21.15.47:0/1642660153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa80b8740 0x7f5fa80d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.786+0000 7f5fad7e5640 1 -- 172.21.15.47:0/1642660153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa80b8360 msgr2=0x7f5fa80cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.786+0000 7f5fad7e5640 1 --2- 172.21.15.47:0/1642660153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa80b8360 0x7f5fa80cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.786+0000 7f5fad7e5640 1 -- 172.21.15.47:0/1642660153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa80b8740 msgr2=0x7f5fa80d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.786+0000 7f5fad7e5640 1 --2- 172.21.15.47:0/1642660153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa80b8740 0x7f5fa80d23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.786+0000 7f5fad7e5640 1 -- 172.21.15.47:0/1642660153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa0009b30 con 0x7f5fa80cf960 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.786+0000 7f5fad7e5640 1 --2- 172.21.15.47:0/1642660153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fa80cf960 0x7f5fa81f4d70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000c410 tx=0x7f5f9000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.786+0000 7f5fa57fa640 1 -- 172.21.15.47:0/1642660153 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90011790 con 0x7f5fa80cf960 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.786+0000 7f5fa57fa640 1 -- 172.21.15.47:0/1642660153 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f900118f0 con 0x7f5fa80cf960 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.786+0000 7f5fa57fa640 1 -- 172.21.15.47:0/1642660153 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90011ce0 con 0x7f5fa80cf960 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.786+0000 7f5faf248640 1 -- 172.21.15.47:0/1642660153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa80cbb30 con 0x7f5fa80cf960 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.786+0000 7f5faf248640 1 -- 172.21.15.47:0/1642660153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa80cc0f0 con 0x7f5fa80cf960 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.787+0000 7f5faf248640 1 -- 172.21.15.47:0/1642660153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa8072360 con 0x7f5fa80cf960 2024-03-23T09:46:21.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.787+0000 7f5fa57fa640 1 -- 172.21.15.47:0/1642660153 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5f9001c460 con 0x7f5fa80cf960 2024-03-23T09:46:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.788+0000 7f5fa57fa640 1 --2- 172.21.15.47:0/1642660153 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5f8806ac00 0x7f5f8806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.788+0000 7f5fa57fa640 1 -- 172.21.15.47:0/1642660153 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5f900890d0 con 0x7f5fa80cf960 2024-03-23T09:46:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.788+0000 7f5facfe4640 1 --2- 172.21.15.47:0/1642660153 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5f8806ac00 0x7f5f8806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.788+0000 7f5facfe4640 1 --2- 172.21.15.47:0/1642660153 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5f8806ac00 0x7f5f8806d0c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0035000 tx=0x7f5fa002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:21.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:21.792+0000 7f5fa57fa640 1 -- 172.21.15.47:0/1642660153 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5f90054220 con 0x7f5fa80cf960 2024-03-23T09:46:22.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.016+0000 7f5faf248640 1 -- 172.21.15.47:0/1642660153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5fa8066be0 con 0x7f5fa80cf960 2024-03-23T09:46:22.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.016+0000 7f5fa57fa640 1 -- 172.21.15.47:0/1642660153 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f5f90057840 con 0x7f5fa80cf960 2024-03-23T09:46:22.016 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:46:22.016 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-23T09:46:22.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.018+0000 7f5faf248640 1 -- 172.21.15.47:0/1642660153 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5f8806ac00 msgr2=0x7f5f8806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.018+0000 7f5faf248640 1 --2- 172.21.15.47:0/1642660153 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5f8806ac00 0x7f5f8806d0c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0035000 tx=0x7f5fa002e000 comp rx=0 tx=0).stop 2024-03-23T09:46:22.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.018+0000 7f5faf248640 1 -- 172.21.15.47:0/1642660153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fa80cf960 msgr2=0x7f5fa81f4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.018+0000 7f5faf248640 1 --2- 172.21.15.47:0/1642660153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fa80cf960 0x7f5fa81f4d70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000c410 tx=0x7f5f9000c8d0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.018+0000 7f5faf248640 1 -- 172.21.15.47:0/1642660153 shutdown_connections 2024-03-23T09:46:22.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.018+0000 7f5faf248640 1 --2- 172.21.15.47:0/1642660153 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5f8806ac00 0x7f5f8806d0c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.018+0000 7f5faf248640 1 --2- 172.21.15.47:0/1642660153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fa80cf960 0x7f5fa81f4d70 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.018+0000 7f5faf248640 1 --2- 172.21.15.47:0/1642660153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa80b8740 0x7f5fa80d23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.018+0000 7f5faf248640 1 --2- 172.21.15.47:0/1642660153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fa80b8360 0x7f5fa80cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.018+0000 7f5faf248640 1 -- 172.21.15.47:0/1642660153 >> 172.21.15.47:0/1642660153 conn(0x7f5fa80659d0 msgr2=0x7f5fa80c8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:22.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.018+0000 7f5faf248640 1 -- 172.21.15.47:0/1642660153 shutdown_connections 2024-03-23T09:46:22.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.018+0000 7f5faf248640 1 -- 172.21.15.47:0/1642660153 wait complete. 2024-03-23T09:46:22.029 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-23T09:46:22.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:22.029 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-03-23T09:46:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.135+0000 7fb7e6c41640 1 Processor -- start 2024-03-23T09:46:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.135+0000 7fb7e6c41640 1 -- start start 2024-03-23T09:46:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.135+0000 7fb7e6c41640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb7e015a850 0x7fb7e00cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.135+0000 7fb7e6c41640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e00cea40 0x7fb7e00cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.135+0000 7fb7e6c41640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e00cb410 con 0x7fb7e015ac30 2024-03-23T09:46:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.135+0000 7fb7e6c41640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e00cee90 con 0x7fb7e015a850 2024-03-23T09:46:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.135+0000 7fb7e6c41640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00cf0a0 con 0x7fb7e00cea40 2024-03-23T09:46:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.135+0000 7fb7e49dd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb7e015a850 0x7fb7e00cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.136+0000 7fb7dffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb7e015ac30 0x7fb7e00ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51252/0 (socket says 172.21.15.47:51252) 2024-03-23T09:46:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.136+0000 7fb7dffff640 1 -- 172.21.15.47:0/3195909679 learned_addr learned my addr 172.21.15.47:0/3195909679 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.136+0000 7fb7e51de640 1 --2- 172.21.15.47:0/3195909679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e00cea40 0x7fb7e00cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.136+0000 7fb7e49dd640 1 -- 172.21.15.47:0/3195909679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e00cea40 msgr2=0x7fb7e00cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.136+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/3195909679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e00cea40 0x7fb7e00cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.136+0000 7fb7e49dd640 1 -- 172.21.15.47:0/3195909679 >> v1:172.21.15.47:6789/0 conn(0x7fb7e015ac30 legacy=0x7fb7e00ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.136+0000 7fb7e49dd640 1 -- 172.21.15.47:0/3195909679 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e00cf380 con 0x7fb7e015a850 2024-03-23T09:46:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.136+0000 7fb7df7fe640 1 -- 172.21.15.47:0/3195909679 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271181410 0 0) 0x7fb7e00cb410 con 0x7fb7e015ac30 2024-03-23T09:46:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.137+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/3195909679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e015a850 0x7fb7e00cd0e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009d90 tx=0x7fb7d802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319f7df6759042f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.137+0000 7fb7df7fe640 1 -- 172.21.15.47:0/3195909679 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d802cce0 con 0x7fb7e015a850 2024-03-23T09:46:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.137+0000 7fb7df7fe640 1 -- 172.21.15.47:0/3195909679 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d802ce40 con 0x7fb7e015a850 2024-03-23T09:46:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.137+0000 7fb7df7fe640 1 -- 172.21.15.47:0/3195909679 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8034660 con 0x7fb7e015a850 2024-03-23T09:46:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.137+0000 7fb7e6c41640 1 -- 172.21.15.47:0/3195909679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e015a850 msgr2=0x7fb7e00cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.137+0000 7fb7e6c41640 1 --2- 172.21.15.47:0/3195909679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e015a850 0x7fb7e00cd0e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009d90 tx=0x7fb7d802bf30 comp rx=0 tx=0).stop 2024-03-23T09:46:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.137+0000 7fb7e6c41640 1 -- 172.21.15.47:0/3195909679 shutdown_connections 2024-03-23T09:46:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.137+0000 7fb7e6c41640 1 --2- 172.21.15.47:0/3195909679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e00cea40 0x7fb7e00cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.137+0000 7fb7e6c41640 1 --2- 172.21.15.47:0/3195909679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e015a850 0x7fb7e00cd0e0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.137+0000 7fb7e6c41640 1 -- 172.21.15.47:0/3195909679 >> 172.21.15.47:0/3195909679 conn(0x7fb7e00d06f0 msgr2=0x7fb7e00d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.137+0000 7fb7e6c41640 1 -- 172.21.15.47:0/3195909679 shutdown_connections 2024-03-23T09:46:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.137+0000 7fb7e6c41640 1 -- 172.21.15.47:0/3195909679 wait complete. 2024-03-23T09:46:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.139+0000 7fb7e6c41640 1 Processor -- start 2024-03-23T09:46:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.139+0000 7fb7e6c41640 1 -- start start 2024-03-23T09:46:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.139+0000 7fb7e6c41640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e00cea40 0x7fb7e006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.139+0000 7fb7e6c41640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e015a850 0x7fb7e006e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.139+0000 7fb7e6c41640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e015ac30 0x7fb7e006b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.139+0000 7fb7e6c41640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e006b250 con 0x7fb7e00cea40 2024-03-23T09:46:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.139+0000 7fb7e6c41640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e006f2d0 con 0x7fb7e015a850 2024-03-23T09:46:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.139+0000 7fb7e6c41640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e006af20 con 0x7fb7e015ac30 2024-03-23T09:46:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7e49dd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e00cea40 0x7fb7e006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7e49dd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e00cea40 0x7fb7e006c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41604/0 (socket says 172.21.15.47:41604) 2024-03-23T09:46:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7e49dd640 1 -- 172.21.15.47:0/2567981508 learned_addr learned my addr 172.21.15.47:0/2567981508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7e51de640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e015ac30 0x7fb7e006b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7dffff640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e015a850 0x7fb7e006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7e49dd640 1 -- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e015ac30 msgr2=0x7fb7e006b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e015ac30 0x7fb7e006b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7e49dd640 1 -- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e015a850 msgr2=0x7fb7e006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e015a850 0x7fb7e006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7e49dd640 1 -- 172.21.15.47:0/2567981508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d8009a70 con 0x7fb7e00cea40 2024-03-23T09:46:22.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7dffff640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e015a850 0x7fb7e006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:22.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e00cea40 0x7fb7e006c740 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009d90 tx=0x7fb7d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7dd7fa640 1 -- 172.21.15.47:0/2567981508 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d803d600 con 0x7fb7e00cea40 2024-03-23T09:46:22.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7dd7fa640 1 -- 172.21.15.47:0/2567981508 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d803d760 con 0x7fb7e00cea40 2024-03-23T09:46:22.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7dd7fa640 1 -- 172.21.15.47:0/2567981508 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d803db10 con 0x7fb7e00cea40 2024-03-23T09:46:22.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7e6c41640 1 -- 172.21.15.47:0/2567981508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e006ed10 con 0x7fb7e00cea40 2024-03-23T09:46:22.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.140+0000 7fb7e6c41640 1 -- 172.21.15.47:0/2567981508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e006cc80 con 0x7fb7e00cea40 2024-03-23T09:46:22.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.142+0000 7fb7dd7fa640 1 -- 172.21.15.47:0/2567981508 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb7d803b040 con 0x7fb7e00cea40 2024-03-23T09:46:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.142+0000 7fb7dd7fa640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb7cc06ac00 0x7fb7cc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.142+0000 7fb7dd7fa640 1 -- 172.21.15.47:0/2567981508 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb7d80ab260 con 0x7fb7e00cea40 2024-03-23T09:46:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.142+0000 7fb7dffff640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb7cc06ac00 0x7fb7cc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.143+0000 7fb7dffff640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb7cc06ac00 0x7fb7cc06d0c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00b65f0 tx=0x7fb7d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.143+0000 7fb7e97b8740 1 Processor -- start 2024-03-23T09:46:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.143+0000 7fb7e97b8740 1 -- start start 2024-03-23T09:46:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e97b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 0x55fe489b1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e97b8740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489dea50 0x55fe489e1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e97b8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489b1b50 0x55fe489e1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e97b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe48922b70 con 0x55fe489de670 2024-03-23T09:46:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e97b8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe48920300 con 0x55fe489b1b50 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e97b8740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe4891bbf0 con 0x55fe489dea50 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e49dd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 0x55fe489b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e49dd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 0x55fe489b1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41620/0 (socket says 172.21.15.47:41620) 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e49dd640 1 -- 172.21.15.47:0/1990583203 learned_addr learned my addr 172.21.15.47:0/1990583203 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e51de640 1 --2- 172.21.15.47:0/1990583203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489b1b50 0x55fe489e1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7dffff640 1 --2- 172.21.15.47:0/1990583203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489dea50 0x55fe489e1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e49dd640 1 -- 172.21.15.47:0/1990583203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489dea50 msgr2=0x55fe489e1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/1990583203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489dea50 0x55fe489e1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e49dd640 1 -- 172.21.15.47:0/1990583203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489b1b50 msgr2=0x55fe489e1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/1990583203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489b1b50 0x55fe489e1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e49dd640 1 -- 172.21.15.47:0/1990583203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe488c2660 con 0x55fe489de670 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7dffff640 1 --2- 172.21.15.47:0/1990583203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489dea50 0x55fe489e1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.144+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/1990583203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 0x55fe489b1610 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8081fc0 tx=0x7fb7d8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d825803bab7dd14 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.145+0000 7fb7c67fc640 1 -- 172.21.15.47:0/1990583203 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80ab0b0 con 0x55fe489de670 2024-03-23T09:46:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.145+0000 7fb7c67fc640 1 -- 172.21.15.47:0/1990583203 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d807a100 con 0x55fe489de670 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.145+0000 7fb7c67fc640 1 -- 172.21.15.47:0/1990583203 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d807df30 con 0x55fe489de670 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.145+0000 7fb7e97b8740 1 -- 172.21.15.47:0/1990583203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 msgr2=0x55fe489b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.145+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/1990583203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 0x55fe489b1610 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8081fc0 tx=0x7fb7d8009750 comp rx=0 tx=0).stop 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.145+0000 7fb7e97b8740 1 -- 172.21.15.47:0/1990583203 shutdown_connections 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.145+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/1990583203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489b1b50 0x55fe489e1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.145+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/1990583203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489dea50 0x55fe489e1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.145+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/1990583203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 0x55fe489b1610 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.145+0000 7fb7e97b8740 1 -- 172.21.15.47:0/1990583203 >> 172.21.15.47:0/1990583203 conn(0x55fe489b3880 msgr2=0x55fe489fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.145+0000 7fb7e97b8740 1 -- 172.21.15.47:0/1990583203 shutdown_connections 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.145+0000 7fb7e97b8740 1 -- 172.21.15.47:0/1990583203 wait complete. 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.146+0000 7fb7e97b8740 1 -- start start 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.146+0000 7fb7e97b8740 1 Processor -- start 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e97b8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489de670 0x55fe48a0c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e97b8740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489dea50 0x55fe489e2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e97b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 0x55fe489e2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e97b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe488d9dc0 con 0x55fe489d9c60 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e97b8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe488d9a60 con 0x55fe489de670 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e97b8740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe4889efb0 con 0x55fe489dea50 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e97b8740 10 client.0 _ll_register_callbacks cb 0x55fe48915260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e51de640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 0x55fe489e2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e51de640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 0x55fe489e2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41622/0 (socket says 172.21.15.47:41622) 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e51de640 1 -- 172.21.15.47:0/4182494111 learned_addr learned my addr 172.21.15.47:0/4182494111 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7dffff640 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489dea50 0x55fe489e2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489de670 0x55fe48a0c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e51de640 1 -- 172.21.15.47:0/4182494111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489dea50 msgr2=0x55fe489e2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e51de640 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489dea50 0x55fe489e2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e51de640 1 -- 172.21.15.47:0/4182494111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489de670 msgr2=0x55fe48a0c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e51de640 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489de670 0x55fe48a0c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e51de640 1 -- 172.21.15.47:0/4182494111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e006cc80 con 0x55fe489d9c60 2024-03-23T09:46:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489de670 0x55fe48a0c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7dffff640 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489dea50 0x55fe489e2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e51de640 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 0x55fe489e2dc0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000bef0 tx=0x7fb7d0011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77a4b019dbf42eb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7bf7fe640 1 -- 172.21.15.47:0/4182494111 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0011e60 con 0x55fe489d9c60 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7bf7fe640 1 -- 172.21.15.47:0/4182494111 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb7d0004030 con 0x55fe489d9c60 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.147+0000 7fb7e97b8740 1 -- 172.21.15.47:0/4182494111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe48943d10 con 0x55fe489d9c60 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7bf7fe640 1 -- 172.21.15.47:0/4182494111 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7d00041f0 con 0x55fe489d9c60 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7e97b8740 10 client.7248 fetch_fsmap learned FSMap version 39 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7e97b8740 1 -- 172.21.15.47:0/4182494111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55fe4891a270 con 0x55fe489d9c60 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7bf7fe640 1 -- 172.21.15.47:0/4182494111 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fe48943d10 con 0x55fe489d9c60 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7bf7fe640 1 -- 172.21.15.47:0/4182494111 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb7d0004350 con 0x55fe489d9c60 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7e97b8740 10 client.7248 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7e97b8740 10 client.7248 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7e97b8740 20 client.7248 populate_metadata read hostname 'smithi047' 2024-03-23T09:46:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55fe489df8b0 0x55fe489dfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7e97b8740 4 client.7248 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7e97b8740 1 -- 172.21.15.47:0/4182494111 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe4860f7d0 con 0x55fe489df8b0 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55fe489df8b0 0x55fe489dfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55fe489df8b0 0x55fe489dfcc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8233c0061e6f76d server_cookie=fceea261ba16ebe9 in_seq=0 out_seq=0 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.148+0000 7fb7bf7fe640 10 client.7248 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7bf7fe640 1 -- 172.21.15.47:0/4182494111 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fe4860f7d0 con 0x55fe489df8b0 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7bf7fe640 10 client.7248 handle_command_reply: tid=0 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7e97b8740 1 client.7248 shutdown 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7e97b8740 1 -- 172.21.15.47:0/4182494111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 msgr2=0x55fe489e2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 0x55fe489e2dc0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000bef0 tx=0x7fb7d0011630 comp rx=0 tx=0).stop 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7e97b8740 1 -- 172.21.15.47:0/4182494111 shutdown_connections 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55fe489df8b0 0x55fe489dfcc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 0x55fe489e2dc0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489dea50 0x55fe489e2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7bf7fe640 0 client.7248 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/4182494111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489de670 0x55fe48a0c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7e97b8740 1 -- 172.21.15.47:0/4182494111 >> 172.21.15.47:0/4182494111 conn(0x55fe489b1b50 msgr2=0x55fe489b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7e97b8740 1 -- 172.21.15.47:0/4182494111 shutdown_connections 2024-03-23T09:46:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7e97b8740 1 -- 172.21.15.47:0/4182494111 wait complete. 2024-03-23T09:46:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.149+0000 7fb7e97b8740 20 client.7248 trim_cache size 0 max 16384 2024-03-23T09:46:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.164+0000 7fb7e97b8740 1 Processor -- start 2024-03-23T09:46:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.164+0000 7fb7e97b8740 1 -- start start 2024-03-23T09:46:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.164+0000 7fb7e97b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 0x55fe48a15860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.164+0000 7fb7e97b8740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489de670 0x55fe489b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.164+0000 7fb7e97b8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489dea50 0x55fe48a0acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.164+0000 7fb7e97b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe48a15da0 con 0x55fe489d9c60 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.164+0000 7fb7e97b8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe48a000f0 con 0x55fe489dea50 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.164+0000 7fb7e97b8740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe48a0eb10 con 0x55fe489de670 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.164+0000 7fb7e49dd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 0x55fe48a15860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.164+0000 7fb7e49dd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 0x55fe48a15860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41630/0 (socket says 172.21.15.47:41630) 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.164+0000 7fb7e49dd640 1 -- 172.21.15.47:0/2468955589 learned_addr learned my addr 172.21.15.47:0/2468955589 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e49dd640 1 -- 172.21.15.47:0/2468955589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489de670 msgr2=0x55fe489b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/2468955589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489de670 0x55fe489b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e49dd640 1 -- 172.21.15.47:0/2468955589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489dea50 msgr2=0x55fe48a0acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/2468955589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489dea50 0x55fe48a0acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e49dd640 1 -- 172.21.15.47:0/2468955589 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe488bc050 con 0x55fe489d9c60 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/2468955589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 0x55fe48a15860 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80765b0 tx=0x7fb7d803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca970022d4bb11c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7c6ffd640 1 -- 172.21.15.47:0/2468955589 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80784a0 con 0x55fe489d9c60 2024-03-23T09:46:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7c6ffd640 1 -- 172.21.15.47:0/2468955589 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d80800b0 con 0x55fe489d9c60 2024-03-23T09:46:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7c6ffd640 1 -- 172.21.15.47:0/2468955589 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8080220 con 0x55fe489d9c60 2024-03-23T09:46:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e97b8740 1 -- 172.21.15.47:0/2468955589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 msgr2=0x55fe48a15860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/2468955589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 0x55fe48a15860 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80765b0 tx=0x7fb7d803f000 comp rx=0 tx=0).stop 2024-03-23T09:46:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e97b8740 1 -- 172.21.15.47:0/2468955589 shutdown_connections 2024-03-23T09:46:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/2468955589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489dea50 0x55fe48a0acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/2468955589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489de670 0x55fe489b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/2468955589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489d9c60 0x55fe48a15860 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e97b8740 1 -- 172.21.15.47:0/2468955589 >> 172.21.15.47:0/2468955589 conn(0x55fe489b1b50 msgr2=0x55fe489b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.165+0000 7fb7e97b8740 1 -- 172.21.15.47:0/2468955589 shutdown_connections 2024-03-23T09:46:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.166+0000 7fb7e97b8740 1 -- 172.21.15.47:0/2468955589 wait complete. 2024-03-23T09:46:22.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.166+0000 7fb7e97b8740 1 -- start start 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.166+0000 7fb7e97b8740 1 Processor -- start 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7e97b8740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489d9c60 0x55fe489d8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7e97b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 0x55fe489d9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7e97b8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489dea50 0x55fe48a0b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7e97b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe48a490c0 con 0x55fe489de670 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7e97b8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe48a494e0 con 0x55fe489dea50 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7e97b8740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe489fc7d0 con 0x55fe489d9c60 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7e97b8740 10 client.0 _ll_register_callbacks cb 0x7fb7d000ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7dffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 0x55fe489d9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7dffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 0x55fe489d9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41634/0 (socket says 172.21.15.47:41634) 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7dffff640 1 -- 172.21.15.47:0/3402527819 learned_addr learned my addr 172.21.15.47:0/3402527819 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7dffff640 1 -- 172.21.15.47:0/3402527819 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489d9c60 msgr2=0x55fe489d8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7dffff640 1 --2- 172.21.15.47:0/3402527819 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489d9c60 0x55fe489d8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7dffff640 1 -- 172.21.15.47:0/3402527819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489dea50 msgr2=0x55fe48a0b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7dffff640 1 --2- 172.21.15.47:0/3402527819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489dea50 0x55fe48a0b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7dffff640 1 -- 172.21.15.47:0/3402527819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe488c2660 con 0x55fe489de670 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7dffff640 1 --2- 172.21.15.47:0/3402527819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 0x55fe489d9210 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00c7240 tx=0x7fb7d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f090c311a058c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.167+0000 7fb7bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7bf7fe640 1 -- 172.21.15.47:0/3402527819 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d40013c0 con 0x55fe489de670 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7bf7fe640 1 -- 172.21.15.47:0/3402527819 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb7d4022b90 con 0x55fe489de670 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7e97b8740 1 -- 172.21.15.47:0/3402527819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7d807a100 con 0x55fe489de670 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7bf7fe640 1 -- 172.21.15.47:0/3402527819 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7d4022d50 con 0x55fe489de670 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7e97b8740 10 client.7254 fetch_fsmap learned FSMap version 39 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7e97b8740 1 -- 172.21.15.47:0/3402527819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55fe488e1820 con 0x55fe489de670 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7bf7fe640 1 -- 172.21.15.47:0/3402527819 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d807a100 con 0x55fe489de670 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7bf7fe640 1 -- 172.21.15.47:0/3402527819 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb7d402a070 con 0x55fe489de670 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7e97b8740 10 client.7254 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7e97b8740 10 client.7254 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-23T09:46:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7e97b8740 20 client.7254 populate_metadata read hostname 'smithi047' 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/3402527819 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55fe489b7890 0x55fe48a0be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7e97b8740 4 client.7254 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7e97b8740 1 -- 172.21.15.47:0/3402527819 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) v1 -- 0x55fe48620ad0 con 0x55fe489b7890 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.168+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/3402527819 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55fe489b7890 0x55fe48a0be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7bf7fe640 10 client.7254 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7e49dd640 1 --2- 172.21.15.47:0/3402527819 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55fe489b7890 0x55fe48a0be30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b36a2109ab982e78 server_cookie=227fa04347e7430d in_seq=0 out_seq=0 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7bf7fe640 1 -- 172.21.15.47:0/3402527819 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fe48620ad0 con 0x55fe489b7890 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7bf7fe640 10 client.7254 handle_command_reply: tid=0 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7e97b8740 1 client.7254 shutdown 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7e97b8740 1 -- 172.21.15.47:0/3402527819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 msgr2=0x55fe489d9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/3402527819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 0x55fe489d9210 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00c7240 tx=0x7fb7d40057f0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7e97b8740 1 -- 172.21.15.47:0/3402527819 shutdown_connections 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/3402527819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe489dea50 0x55fe48a0b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/3402527819 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55fe489b7890 0x55fe48a0be30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/3402527819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe489de670 0x55fe489d9210 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7e97b8740 1 --2- 172.21.15.47:0/3402527819 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fe489d9c60 0x55fe489d8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7bf7fe640 0 client.7254 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7e97b8740 1 -- 172.21.15.47:0/3402527819 >> 172.21.15.47:0/3402527819 conn(0x55fe489b1b50 msgr2=0x55fe489b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7e97b8740 1 -- 172.21.15.47:0/3402527819 shutdown_connections 2024-03-23T09:46:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.169+0000 7fb7e97b8740 1 -- 172.21.15.47:0/3402527819 wait complete. 2024-03-23T09:46:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.170+0000 7fb7e97b8740 20 client.7254 trim_cache size 0 max 16384 2024-03-23T09:46:22.174 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:46:22.174 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-23T09:46:22.174 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "0b206dbd-23b9-4203-9bd8-5bda08e13e37", 2024-03-23T09:46:22.174 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-23T09:46:22.174 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:46:22.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.170+0000 7fb7e6c41640 1 -- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb7cc06ac00 msgr2=0x7fb7cc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.170+0000 7fb7e6c41640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb7cc06ac00 0x7fb7cc06d0c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00b65f0 tx=0x7fb7d4009040 comp rx=0 tx=0).stop 2024-03-23T09:46:22.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.170+0000 7fb7e6c41640 1 -- 172.21.15.47:0/2567981508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e00cea40 msgr2=0x7fb7e006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.170+0000 7fb7e6c41640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e00cea40 0x7fb7e006c740 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009d90 tx=0x7fb7d8004a40 comp rx=0 tx=0).stop 2024-03-23T09:46:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.170+0000 7fb7e6c41640 1 -- 172.21.15.47:0/2567981508 shutdown_connections 2024-03-23T09:46:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.170+0000 7fb7e6c41640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb7cc06ac00 0x7fb7cc06d0c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.170+0000 7fb7e6c41640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e015ac30 0x7fb7e006b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.170+0000 7fb7e6c41640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e015a850 0x7fb7e006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.170+0000 7fb7e6c41640 1 --2- 172.21.15.47:0/2567981508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e00cea40 0x7fb7e006c740 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.170+0000 7fb7e6c41640 1 -- 172.21.15.47:0/2567981508 >> 172.21.15.47:0/2567981508 conn(0x7fb7e00d06f0 msgr2=0x7fb7e00ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.170+0000 7fb7e6c41640 1 -- 172.21.15.47:0/2567981508 shutdown_connections 2024-03-23T09:46:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.170+0000 7fb7e6c41640 1 -- 172.21.15.47:0/2567981508 wait complete. 2024-03-23T09:46:22.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:22.181 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-23T09:46:22.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.283+0000 7f93b740b640 1 Processor -- start 2024-03-23T09:46:22.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.284+0000 7f93b740b640 1 -- start start 2024-03-23T09:46:22.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.284+0000 7f93b740b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93b015cd40 0x7f93b00d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.284+0000 7f93b740b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f93b0099110 0x7f93b00ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.284+0000 7f93b740b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b00cedd0 con 0x7f93b006b090 2024-03-23T09:46:22.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.284+0000 7f93b740b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b015d120 con 0x7f93b0099110 2024-03-23T09:46:22.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.284+0000 7f93b740b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b0099760 con 0x7f93b015cd40 2024-03-23T09:46:22.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.285+0000 7f93b59a8640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f93b0099110 0x7f93b00ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.285+0000 7f93b51a7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93b015cd40 0x7f93b00d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.285+0000 7f93b51a7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93b015cd40 0x7f93b00d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:47928/0 (socket says 172.21.15.47:47928) 2024-03-23T09:46:22.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.285+0000 7f93b49a6640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f93b006b090 0x7f93b0098a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:55172/0 (socket says 172.21.15.47:55172) 2024-03-23T09:46:22.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.285+0000 7f93b51a7640 1 -- 172.21.15.47:0/157531393 learned_addr learned my addr 172.21.15.47:0/157531393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:22.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.285+0000 7f93b59a8640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f93b0099110 0x7f93b00ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:34024/0 (socket says 172.21.15.47:34024) 2024-03-23T09:46:22.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.285+0000 7f93b59a8640 1 -- 172.21.15.47:0/157531393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93b015cd40 msgr2=0x7f93b00d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.285+0000 7f93b59a8640 1 --2- 172.21.15.47:0/157531393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93b015cd40 0x7f93b00d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.285+0000 7f93b59a8640 1 -- 172.21.15.47:0/157531393 >> v1:172.21.15.175:6789/0 conn(0x7f93b006b090 legacy=0x7f93b0098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.285+0000 7f93b59a8640 1 -- 172.21.15.47:0/157531393 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b00cf020 con 0x7f93b0099110 2024-03-23T09:46:22.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.285+0000 7f93a7fff640 1 -- 172.21.15.47:0/157531393 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847886833 0 0) 0x7f93b00cedd0 con 0x7f93b006b090 2024-03-23T09:46:22.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.285+0000 7f93b51a7640 1 --2- 172.21.15.47:0/157531393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93b015cd40 0x7f93b00d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:46:22.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.286+0000 7f93b59a8640 1 --2- 172.21.15.47:0/157531393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b0099110 0x7f93b00ce890 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009d90 tx=0x7f93a802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11b63c33df7c56da server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.286+0000 7f93a7fff640 1 -- 172.21.15.47:0/157531393 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a802cc50 con 0x7f93b0099110 2024-03-23T09:46:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.286+0000 7f93a7fff640 1 -- 172.21.15.47:0/157531393 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a802cdb0 con 0x7f93b0099110 2024-03-23T09:46:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.286+0000 7f93a7fff640 1 -- 172.21.15.47:0/157531393 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8034660 con 0x7f93b0099110 2024-03-23T09:46:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.286+0000 7f93b740b640 1 -- 172.21.15.47:0/157531393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b0099110 msgr2=0x7f93b00ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.286+0000 7f93b740b640 1 --2- 172.21.15.47:0/157531393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b0099110 0x7f93b00ce890 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009d90 tx=0x7f93a802bea0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.287+0000 7f93b740b640 1 -- 172.21.15.47:0/157531393 shutdown_connections 2024-03-23T09:46:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.287+0000 7f93b740b640 1 --2- 172.21.15.47:0/157531393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b0099110 0x7f93b00ce890 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.287+0000 7f93b740b640 1 --2- 172.21.15.47:0/157531393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93b015cd40 0x7f93b00d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.287+0000 7f93b740b640 1 -- 172.21.15.47:0/157531393 >> 172.21.15.47:0/157531393 conn(0x7f93b00b2140 msgr2=0x7f93b00b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.287+0000 7f93b740b640 1 -- 172.21.15.47:0/157531393 shutdown_connections 2024-03-23T09:46:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.287+0000 7f93b740b640 1 -- 172.21.15.47:0/157531393 wait complete. 2024-03-23T09:46:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.288+0000 7f93b740b640 1 Processor -- start 2024-03-23T09:46:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.288+0000 7f93b740b640 1 -- start start 2024-03-23T09:46:22.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.288+0000 7f93b740b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93b006b090 0x7f93b0153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.288+0000 7f93b740b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b0099110 0x7f93b006b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b740b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93b015cd40 0x7f93b006bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b740b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b0154940 con 0x7f93b006b090 2024-03-23T09:46:22.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b740b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b0154c20 con 0x7f93b0099110 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b740b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b0154f00 con 0x7f93b015cd40 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b51a7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93b006b090 0x7f93b0153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b51a7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93b006b090 0x7f93b0153c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41650/0 (socket says 172.21.15.47:41650) 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b51a7640 1 -- 172.21.15.47:0/398510248 learned_addr learned my addr 172.21.15.47:0/398510248 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b49a6640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b0099110 0x7f93b006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b59a8640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93b015cd40 0x7f93b006bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b51a7640 1 -- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93b015cd40 msgr2=0x7f93b006bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b51a7640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93b015cd40 0x7f93b006bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b51a7640 1 -- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b0099110 msgr2=0x7f93b006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b51a7640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b0099110 0x7f93b006b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b51a7640 1 -- 172.21.15.47:0/398510248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac0094d0 con 0x7f93b006b090 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b51a7640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93b006b090 0x7f93b0153c20 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e9d0 tx=0x7f93ac00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93a5ffb640 1 -- 172.21.15.47:0/398510248 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac00ca60 con 0x7f93b006b090 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93a5ffb640 1 -- 172.21.15.47:0/398510248 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ac00cbc0 con 0x7f93b006b090 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93b740b640 1 -- 172.21.15.47:0/398510248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a8009a70 con 0x7f93b006b090 2024-03-23T09:46:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.289+0000 7f93a5ffb640 1 -- 172.21.15.47:0/398510248 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac013470 con 0x7f93b006b090 2024-03-23T09:46:22.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.290+0000 7f93b740b640 1 -- 172.21.15.47:0/398510248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b01554c0 con 0x7f93b006b090 2024-03-23T09:46:22.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.291+0000 7f93a5ffb640 1 -- 172.21.15.47:0/398510248 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f93ac0135d0 con 0x7f93b006b090 2024-03-23T09:46:22.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.291+0000 7f93a5ffb640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f939406ac00 0x7f939406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.291+0000 7f93a5ffb640 1 -- 172.21.15.47:0/398510248 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f93ac0896d0 con 0x7f93b006b090 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.292+0000 7f93b49a6640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f939406ac00 0x7f939406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.292+0000 7f93b49a6640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f939406ac00 0x7f939406d0c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f93b0154160 tx=0x7f93a0007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.292+0000 7f93b9f7f740 1 Processor -- start 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.292+0000 7f93b9f7f740 1 -- start start 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.293+0000 7f93b9f7f740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8eae830 0x5577d8eaec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.293+0000 7f93b9f7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e7e850 0x5577d8eabaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.293+0000 7f93b9f7f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 0x5577d8eabfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.293+0000 7f93b9f7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d8deedd0 con 0x5577d8e7e850 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.293+0000 7f93b9f7f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d8dec560 con 0x5577d8e823b0 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.293+0000 7f93b9f7f740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577d8de7e50 con 0x5577d8eae830 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.293+0000 7f93b49a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e7e850 0x5577d8eabaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.293+0000 7f93b49a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e7e850 0x5577d8eabaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41664/0 (socket says 172.21.15.47:41664) 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.293+0000 7f93b49a6640 1 -- 172.21.15.47:0/2972049917 learned_addr learned my addr 172.21.15.47:0/2972049917 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.293+0000 7f93b59a8640 1 --2- 172.21.15.47:0/2972049917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 0x5577d8eabfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.293+0000 7f93b51a7640 1 --2- 172.21.15.47:0/2972049917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8eae830 0x5577d8eaec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.293+0000 7f93b49a6640 1 -- 172.21.15.47:0/2972049917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8eae830 msgr2=0x5577d8eaec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.293+0000 7f93b49a6640 1 --2- 172.21.15.47:0/2972049917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8eae830 0x5577d8eaec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f93b49a6640 1 -- 172.21.15.47:0/2972049917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 msgr2=0x5577d8eabfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f93b49a6640 1 --2- 172.21.15.47:0/2972049917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 0x5577d8eabfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f93b49a6640 1 -- 172.21.15.47:0/2972049917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577d8daf470 con 0x5577d8e7e850 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f93b49a6640 1 --2- 172.21.15.47:0/2972049917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e7e850 0x5577d8eabaa0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f93a0007fd0 tx=0x7f93a0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcafddb0094afa34 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f939affd640 1 -- 172.21.15.47:0/2972049917 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0001cf0 con 0x5577d8e7e850 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f939affd640 1 -- 172.21.15.47:0/2972049917 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a0013b50 con 0x5577d8e7e850 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f939affd640 1 -- 172.21.15.47:0/2972049917 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0012350 con 0x5577d8e7e850 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f93b9f7f740 1 -- 172.21.15.47:0/2972049917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e7e850 msgr2=0x5577d8eabaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/2972049917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e7e850 0x5577d8eabaa0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f93a0007fd0 tx=0x7f93a0013000 comp rx=0 tx=0).stop 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f93b9f7f740 1 -- 172.21.15.47:0/2972049917 shutdown_connections 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/2972049917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 0x5577d8eabfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/2972049917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e7e850 0x5577d8eabaa0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/2972049917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8eae830 0x5577d8eaec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f93b9f7f740 1 -- 172.21.15.47:0/2972049917 >> 172.21.15.47:0/2972049917 conn(0x5577d8e80ac0 msgr2=0x5577d8ec9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f93b9f7f740 1 -- 172.21.15.47:0/2972049917 shutdown_connections 2024-03-23T09:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.294+0000 7f93b9f7f740 1 -- 172.21.15.47:0/2972049917 wait complete. 2024-03-23T09:46:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.295+0000 7f93b9f7f740 1 -- start start 2024-03-23T09:46:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.295+0000 7f93b9f7f740 1 Processor -- start 2024-03-23T09:46:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b9f7f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 0x5577d8eccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b9f7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 0x5577d8ed8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b9f7f740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8ea94e0 0x5577d8eb0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b9f7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d8da6280 con 0x5577d8eae830 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b9f7f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d8da5f20 con 0x5577d8e823b0 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b9f7f740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577d8d6b3c0 con 0x5577d8ea94e0 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b9f7f740 10 client.0 _ll_register_callbacks cb 0x5577d8dbd380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b49a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 0x5577d8ed8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b49a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 0x5577d8ed8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41674/0 (socket says 172.21.15.47:41674) 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b49a6640 1 -- 172.21.15.47:0/1588382558 learned_addr learned my addr 172.21.15.47:0/1588382558 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b59a8640 1 --2- 172.21.15.47:0/1588382558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8ea94e0 0x5577d8eb0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b51a7640 1 --2- 172.21.15.47:0/1588382558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 0x5577d8eccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b49a6640 1 -- 172.21.15.47:0/1588382558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8ea94e0 msgr2=0x5577d8eb0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b49a6640 1 --2- 172.21.15.47:0/1588382558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8ea94e0 0x5577d8eb0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b49a6640 1 -- 172.21.15.47:0/1588382558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 msgr2=0x5577d8eccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b49a6640 1 --2- 172.21.15.47:0/1588382558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 0x5577d8eccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.296+0000 7f93b49a6640 1 -- 172.21.15.47:0/1588382558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a0004c70 con 0x5577d8eae830 2024-03-23T09:46:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f93b49a6640 1 --2- 172.21.15.47:0/1588382558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 0x5577d8ed8ff0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f93a001f000 tx=0x7f93a0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2f2ae392977eaf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f938ffff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f938ffff640 1 -- 172.21.15.47:0/1588382558 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0001cf0 con 0x5577d8eae830 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1588382558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577d8e11690 con 0x5577d8eae830 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f938ffff640 1 -- 172.21.15.47:0/1588382558 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f93a001b070 con 0x5577d8eae830 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f938ffff640 1 -- 172.21.15.47:0/1588382558 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93a0021a70 con 0x5577d8eae830 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f93b9f7f740 10 client.7263 fetch_fsmap learned FSMap version 39 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1588382558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5577d8de64d0 con 0x5577d8eae830 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f938ffff640 1 -- 172.21.15.47:0/1588382558 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5577d8e11690 con 0x5577d8eae830 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f938ffff640 1 -- 172.21.15.47:0/1588382558 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f93a0012070 con 0x5577d8eae830 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f93b9f7f740 10 client.7263 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f93b9f7f740 10 client.7263 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f93b9f7f740 20 client.7263 populate_metadata read hostname 'smithi047' 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/1588382558 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5577d8e9b6a0 0x5577d8e9ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f93b9f7f740 4 client.7263 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:46:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1588382558 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577d8dd9eb0 con 0x5577d8e9b6a0 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.297+0000 7f93b51a7640 1 --2- 172.21.15.47:0/1588382558 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5577d8e9b6a0 0x5577d8e9ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f938ffff640 10 client.7263 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f93b51a7640 1 --2- 172.21.15.47:0/1588382558 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5577d8e9b6a0 0x5577d8e9ba80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f968e012c32402d server_cookie=a889c2df368d2b87 in_seq=0 out_seq=0 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f938ffff640 1 -- 172.21.15.47:0/1588382558 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5577d8dd9eb0 con 0x5577d8e9b6a0 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f938ffff640 10 client.7263 handle_command_reply: tid=0 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f93b9f7f740 1 client.7263 shutdown 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1588382558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 msgr2=0x5577d8ed8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/1588382558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 0x5577d8ed8ff0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f93a001f000 tx=0x7f93a0000ed0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1588382558 shutdown_connections 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/1588382558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8ea94e0 0x5577d8eb0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/1588382558 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5577d8e9b6a0 0x5577d8e9ba80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/1588382558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 0x5577d8ed8ff0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/1588382558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 0x5577d8eccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f938ffff640 0 client.7263 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.298+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1588382558 >> 172.21.15.47:0/1588382558 conn(0x5577d8e7e850 msgr2=0x5577d8e82b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.299+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1588382558 shutdown_connections 2024-03-23T09:46:22.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.299+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1588382558 wait complete. 2024-03-23T09:46:22.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.299+0000 7f93b9f7f740 20 client.7263 trim_cache size 0 max 16384 2024-03-23T09:46:22.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.314+0000 7f93b9f7f740 1 Processor -- start 2024-03-23T09:46:22.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.314+0000 7f93b9f7f740 1 -- start start 2024-03-23T09:46:22.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b9f7f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 0x5577d8ea8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b9f7f740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8ea94e0 0x5577d8e7d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b9f7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 0x5577d8e7de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b9f7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d8e9ba80 con 0x5577d8eae830 2024-03-23T09:46:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b9f7f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d8eb0dc0 con 0x5577d8e823b0 2024-03-23T09:46:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b9f7f740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577d8ea9150 con 0x5577d8ea94e0 2024-03-23T09:46:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b59a8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 0x5577d8e7de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b59a8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 0x5577d8e7de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41686/0 (socket says 172.21.15.47:41686) 2024-03-23T09:46:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b59a8640 1 -- 172.21.15.47:0/2662332308 learned_addr learned my addr 172.21.15.47:0/2662332308 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b51a7640 1 --2- 172.21.15.47:0/2662332308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 0x5577d8ea8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b49a6640 1 --2- 172.21.15.47:0/2662332308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8ea94e0 0x5577d8e7d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b59a8640 1 -- 172.21.15.47:0/2662332308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8ea94e0 msgr2=0x5577d8e7d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b59a8640 1 --2- 172.21.15.47:0/2662332308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8ea94e0 0x5577d8e7d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b59a8640 1 -- 172.21.15.47:0/2662332308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 msgr2=0x5577d8ea8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b59a8640 1 --2- 172.21.15.47:0/2662332308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 0x5577d8ea8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b59a8640 1 -- 172.21.15.47:0/2662332308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577d8e1a9e0 con 0x5577d8eae830 2024-03-23T09:46:22.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b51a7640 1 --2- 172.21.15.47:0/2662332308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 0x5577d8ea8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:22.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.315+0000 7f93b59a8640 1 --2- 172.21.15.47:0/2662332308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 0x5577d8e7de50 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009eb0 tx=0x7f93a8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74be3f62996f79f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.316+0000 7f939b7fe640 1 -- 172.21.15.47:0/2662332308 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8003b90 con 0x5577d8eae830 2024-03-23T09:46:22.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.316+0000 7f939b7fe640 1 -- 172.21.15.47:0/2662332308 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a8045040 con 0x5577d8eae830 2024-03-23T09:46:22.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.316+0000 7f939b7fe640 1 -- 172.21.15.47:0/2662332308 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8004040 con 0x5577d8eae830 2024-03-23T09:46:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.316+0000 7f93b9f7f740 1 -- 172.21.15.47:0/2662332308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 msgr2=0x5577d8e7de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.316+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/2662332308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 0x5577d8e7de50 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009eb0 tx=0x7f93a8030000 comp rx=0 tx=0).stop 2024-03-23T09:46:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.316+0000 7f93b9f7f740 1 -- 172.21.15.47:0/2662332308 shutdown_connections 2024-03-23T09:46:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.316+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/2662332308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8eae830 0x5577d8e7de50 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.316+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/2662332308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8ea94e0 0x5577d8e7d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.316+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/2662332308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8e823b0 0x5577d8ea8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.316+0000 7f93b9f7f740 1 -- 172.21.15.47:0/2662332308 >> 172.21.15.47:0/2662332308 conn(0x5577d8e7e850 msgr2=0x5577d8eb0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.317+0000 7f93b9f7f740 1 -- 172.21.15.47:0/2662332308 shutdown_connections 2024-03-23T09:46:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.317+0000 7f93b9f7f740 1 -- 172.21.15.47:0/2662332308 wait complete. 2024-03-23T09:46:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.317+0000 7f93b9f7f740 1 -- start start 2024-03-23T09:46:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.317+0000 7f93b9f7f740 1 Processor -- start 2024-03-23T09:46:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.318+0000 7f93b9f7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e823b0 0x5577d8ee20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.318+0000 7f93b9f7f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8ea94e0 0x5577d8ee25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.318+0000 7f93b9f7f740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8eae830 0x5577d8f2f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.318+0000 7f93b9f7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d8f32d40 con 0x5577d8e823b0 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.318+0000 7f93b9f7f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d8ea9b00 con 0x5577d8ea94e0 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.318+0000 7f93b9f7f740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577d8e801f0 con 0x5577d8eae830 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.318+0000 7f93b9f7f740 10 client.0 _ll_register_callbacks cb 0x7f93a0007380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.318+0000 7f93b51a7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e823b0 0x5577d8ee20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.318+0000 7f93b51a7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e823b0 0x5577d8ee20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41694/0 (socket says 172.21.15.47:41694) 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.318+0000 7f93b51a7640 1 -- 172.21.15.47:0/1762081666 learned_addr learned my addr 172.21.15.47:0/1762081666 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.318+0000 7f93b59a8640 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8eae830 0x5577d8f2f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.318+0000 7f93b49a6640 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8ea94e0 0x5577d8ee25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b51a7640 1 -- 172.21.15.47:0/1762081666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8eae830 msgr2=0x5577d8f2f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b51a7640 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8eae830 0x5577d8f2f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b51a7640 1 -- 172.21.15.47:0/1762081666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8ea94e0 msgr2=0x5577d8ee25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b51a7640 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8ea94e0 0x5577d8ee25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b51a7640 1 -- 172.21.15.47:0/1762081666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b00cf020 con 0x5577d8e823b0 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b49a6640 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8ea94e0 0x5577d8ee25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b59a8640 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8eae830 0x5577d8f2f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b51a7640 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e823b0 0x5577d8ee20b0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0584a0 tx=0x7f93ac077050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472ecd2f0687ac33 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f938ffff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f938ffff640 1 -- 172.21.15.47:0/1762081666 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac054c20 con 0x5577d8e823b0 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f938ffff640 1 -- 172.21.15.47:0/1762081666 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f93ac054d80 con 0x5577d8e823b0 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1762081666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93a0013b50 con 0x5577d8e823b0 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f938ffff640 1 -- 172.21.15.47:0/1762081666 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93ac0603c0 con 0x5577d8e823b0 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b9f7f740 10 client.7269 fetch_fsmap learned FSMap version 39 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1762081666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5577d8d885c0 con 0x5577d8e823b0 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f938ffff640 1 -- 172.21.15.47:0/1762081666 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a0013b50 con 0x5577d8e823b0 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f938ffff640 1 -- 172.21.15.47:0/1762081666 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f93ac011070 con 0x5577d8e823b0 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b9f7f740 10 client.7269 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T09:46:22.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b9f7f740 10 client.7269 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b9f7f740 20 client.7269 populate_metadata read hostname 'smithi047' 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5577d8f25df0 0x5577d8f2fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b9f7f740 4 client.7269 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.319+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1762081666 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5577d8adb4e0 con 0x5577d8f25df0 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f93b49a6640 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5577d8f25df0 0x5577d8f2fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f938ffff640 10 client.7269 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f93b49a6640 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5577d8f25df0 0x5577d8f2fa60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6010561820c8d5b8 server_cookie=5b54e11ed5bd543e in_seq=0 out_seq=0 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f938ffff640 1 -- 172.21.15.47:0/1762081666 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x5577d8adb4e0 con 0x5577d8f25df0 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f938ffff640 10 client.7269 handle_command_reply: tid=0 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f93b9f7f740 1 client.7269 shutdown 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1762081666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e823b0 msgr2=0x5577d8ee20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e823b0 0x5577d8ee20b0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0584a0 tx=0x7f93ac077050 comp rx=0 tx=0).stop 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1762081666 shutdown_connections 2024-03-23T09:46:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5577d8eae830 0x5577d8f2f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577d8ea94e0 0x5577d8ee25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5577d8f25df0 0x5577d8f2fa60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f93b9f7f740 1 --2- 172.21.15.47:0/1762081666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577d8e823b0 0x5577d8ee20b0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f938ffff640 0 client.7269 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.320+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1762081666 >> 172.21.15.47:0/1762081666 conn(0x5577d8e7e850 msgr2=0x5577d8eb24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1762081666 shutdown_connections 2024-03-23T09:46:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b9f7f740 1 -- 172.21.15.47:0/1762081666 wait complete. 2024-03-23T09:46:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b9f7f740 20 client.7269 trim_cache size 0 max 16384 2024-03-23T09:46:22.326 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:46:22.326 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T09:46:22.327 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-23T09:46:22.327 INFO:teuthology.orchestra.run.smithi047.stdout: "0b206dbd-23b9-4203-9bd8-5bda08e13e37": { 2024-03-23T09:46:22.327 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/subdir", 2024-03-23T09:46:22.327 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "0b206dbd-23b9-4203-9bd8-5bda08e13e37", 2024-03-23T09:46:22.327 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,repair" 2024-03-23T09:46:22.327 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:46:22.327 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:46:22.327 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:46:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b740b640 1 -- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f939406ac00 msgr2=0x7f939406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b740b640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f939406ac00 0x7f939406d0c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f93b0154160 tx=0x7f93a0007de0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b740b640 1 -- 172.21.15.47:0/398510248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93b006b090 msgr2=0x7f93b0153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b740b640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93b006b090 0x7f93b0153c20 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e9d0 tx=0x7f93ac00ee90 comp rx=0 tx=0).stop 2024-03-23T09:46:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b740b640 1 -- 172.21.15.47:0/398510248 shutdown_connections 2024-03-23T09:46:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b740b640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f939406ac00 0x7f939406d0c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b740b640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93b015cd40 0x7f93b006bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b740b640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b0099110 0x7f93b006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b740b640 1 --2- 172.21.15.47:0/398510248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93b006b090 0x7f93b0153c20 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b740b640 1 -- 172.21.15.47:0/398510248 >> 172.21.15.47:0/398510248 conn(0x7f93b00b2140 msgr2=0x7f93b0159500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b740b640 1 -- 172.21.15.47:0/398510248 shutdown_connections 2024-03-23T09:46:22.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:22.321+0000 7f93b740b640 1 -- 172.21.15.47:0/398510248 wait complete. 2024-03-23T09:46:22.332 INFO:tasks.cephfs.filesystem:scrub status for tag:0b206dbd-23b9-4203-9bd8-5bda08e13e37 - {'path': '/subdir', 'tag': '0b206dbd-23b9-4203-9bd8-5bda08e13e37', 'options': 'recursive,repair'} 2024-03-23T09:46:37.930 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:46:37.934 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:46:52.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:52.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-23T09:46:52.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.448+0000 7f449acab640 1 Processor -- start 2024-03-23T09:46:52.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.448+0000 7f449acab640 1 -- start start 2024-03-23T09:46:52.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.448+0000 7f449acab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449415cc30 0x7f44940ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.448+0000 7f449acab640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44940cd4b0 0x7f44940b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.448+0000 7f449acab640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449415d010 con 0x7f44940cad40 2024-03-23T09:46:52.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.448+0000 7f449acab640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44940cd940 con 0x7f44940cd4b0 2024-03-23T09:46:52.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.448+0000 7f449acab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44940b13d0 con 0x7f449415cc30 2024-03-23T09:46:52.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.449+0000 7f4499248640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44940cd4b0 0x7f44940b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.449+0000 7f4499248640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44940cd4b0 0x7f44940b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36102/0 (socket says 172.21.15.47:36102) 2024-03-23T09:46:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.449+0000 7f4499248640 1 -- 172.21.15.47:0/4294434000 learned_addr learned my addr 172.21.15.47:0/4294434000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.449+0000 7f4498a47640 1 --2- 172.21.15.47:0/4294434000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449415cc30 0x7f44940ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.449+0000 7f4499248640 1 -- 172.21.15.47:0/4294434000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449415cc30 msgr2=0x7f44940ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.449+0000 7f4499248640 1 --2- 172.21.15.47:0/4294434000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449415cc30 0x7f44940ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.449+0000 7f4499248640 1 -- 172.21.15.47:0/4294434000 >> v1:172.21.15.175:6789/0 conn(0x7f44940cad40 legacy=0x7f44940b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.449+0000 7f4499248640 1 -- 172.21.15.47:0/4294434000 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44940b16b0 con 0x7f44940cd4b0 2024-03-23T09:46:52.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.450+0000 7f4499248640 1 --2- 172.21.15.47:0/4294434000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44940cd4b0 0x7f44940b0d20 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f448c009e50 tx=0x7f448c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30066411a70e961e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.450+0000 7f44937fe640 1 -- 172.21.15.47:0/4294434000 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c02ca70 con 0x7f44940cd4b0 2024-03-23T09:46:52.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.450+0000 7f44937fe640 1 -- 172.21.15.47:0/4294434000 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c02cbd0 con 0x7f44940cd4b0 2024-03-23T09:46:52.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.450+0000 7f44937fe640 1 -- 172.21.15.47:0/4294434000 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c0343f0 con 0x7f44940cd4b0 2024-03-23T09:46:52.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.450+0000 7f449acab640 1 -- 172.21.15.47:0/4294434000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44940cd4b0 msgr2=0x7f44940b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.450+0000 7f449acab640 1 --2- 172.21.15.47:0/4294434000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44940cd4b0 0x7f44940b0d20 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f448c009e50 tx=0x7f448c02bed0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.450+0000 7f449acab640 1 -- 172.21.15.47:0/4294434000 shutdown_connections 2024-03-23T09:46:52.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.450+0000 7f449acab640 1 --2- 172.21.15.47:0/4294434000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44940cd4b0 0x7f44940b0d20 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.450+0000 7f449acab640 1 --2- 172.21.15.47:0/4294434000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449415cc30 0x7f44940ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.450+0000 7f449acab640 1 -- 172.21.15.47:0/4294434000 >> 172.21.15.47:0/4294434000 conn(0x7f44940d06f0 msgr2=0x7f44940d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.450+0000 7f449acab640 1 -- 172.21.15.47:0/4294434000 shutdown_connections 2024-03-23T09:46:52.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.450+0000 7f449acab640 1 -- 172.21.15.47:0/4294434000 wait complete. 2024-03-23T09:46:52.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.452+0000 7f449acab640 1 Processor -- start 2024-03-23T09:46:52.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.452+0000 7f449acab640 1 -- start start 2024-03-23T09:46:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.452+0000 7f449acab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44940cad40 0x7f449415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.452+0000 7f449acab640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44940cd4b0 0x7f44940c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.452+0000 7f449acab640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f449415cc30 0x7f44941f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.452+0000 7f449acab640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44940c52c0 con 0x7f449415cc30 2024-03-23T09:46:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.452+0000 7f449acab640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44940c46b0 con 0x7f44940cd4b0 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.452+0000 7f449acab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44940c4990 con 0x7f44940cad40 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f4499248640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f449415cc30 0x7f44941f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f4499248640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f449415cc30 0x7f44941f5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36108/0 (socket says 172.21.15.47:36108) 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f4499248640 1 -- 172.21.15.47:0/1940566685 learned_addr learned my addr 172.21.15.47:0/1940566685 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f4498a47640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44940cad40 0x7f449415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f4493fff640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44940cd4b0 0x7f44940c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f4499248640 1 -- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44940cad40 msgr2=0x7f449415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f4499248640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44940cad40 0x7f449415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f4499248640 1 -- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44940cd4b0 msgr2=0x7f44940c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f4499248640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44940cd4b0 0x7f44940c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f4499248640 1 -- 172.21.15.47:0/1940566685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c009b30 con 0x7f449415cc30 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f4493fff640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44940cd4b0 0x7f44940c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f4499248640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f449415cc30 0x7f44941f5da0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f448c009e50 tx=0x7f448c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f44917fa640 1 -- 172.21.15.47:0/1940566685 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c009510 con 0x7f449415cc30 2024-03-23T09:46:52.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f44917fa640 1 -- 172.21.15.47:0/1940566685 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c03e450 con 0x7f449415cc30 2024-03-23T09:46:52.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f44917fa640 1 -- 172.21.15.47:0/1940566685 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c03e800 con 0x7f449415cc30 2024-03-23T09:46:52.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f449acab640 1 -- 172.21.15.47:0/1940566685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44940c4c70 con 0x7f449415cc30 2024-03-23T09:46:52.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.453+0000 7f449acab640 1 -- 172.21.15.47:0/1940566685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449415bc20 con 0x7f449415cc30 2024-03-23T09:46:52.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.455+0000 7f44917fa640 1 -- 172.21.15.47:0/1940566685 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f448c03e960 con 0x7f449415cc30 2024-03-23T09:46:52.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.455+0000 7f44917fa640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f447806ac00 0x7f447806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.455+0000 7f44917fa640 1 -- 172.21.15.47:0/1940566685 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f448c0ac090 con 0x7f449415cc30 2024-03-23T09:46:52.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.455+0000 7f4498a47640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f447806ac00 0x7f447806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.456+0000 7f4498a47640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f447806ac00 0x7f447806d0c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f44880079a0 tx=0x7f4488008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.456+0000 7f449d81a740 1 Processor -- start 2024-03-23T09:46:52.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.456+0000 7f449d81a740 1 -- start start 2024-03-23T09:46:52.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f449d81a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 0x5568a6c8bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f449d81a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5b850 0x5568a6c88aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f449d81a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c5f3b0 0x5568a6c88fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f449d81a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a6bcbdd0 con 0x5568a6c8b830 2024-03-23T09:46:52.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f449d81a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a6bc9560 con 0x5568a6c5b850 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f449d81a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568a6bc4e50 con 0x5568a6c5f3b0 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f4498a47640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 0x5568a6c8bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f4498a47640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 0x5568a6c8bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36118/0 (socket says 172.21.15.47:36118) 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f4498a47640 1 -- 172.21.15.47:0/2382365947 learned_addr learned my addr 172.21.15.47:0/2382365947 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f4493fff640 1 --2- 172.21.15.47:0/2382365947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5b850 0x5568a6c88aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f4499248640 1 --2- 172.21.15.47:0/2382365947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c5f3b0 0x5568a6c88fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f4498a47640 1 -- 172.21.15.47:0/2382365947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c5f3b0 msgr2=0x5568a6c88fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f4498a47640 1 --2- 172.21.15.47:0/2382365947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c5f3b0 0x5568a6c88fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f4498a47640 1 -- 172.21.15.47:0/2382365947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5b850 msgr2=0x5568a6c88aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f4498a47640 1 --2- 172.21.15.47:0/2382365947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5b850 0x5568a6c88aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.457+0000 7f4498a47640 1 -- 172.21.15.47:0/2382365947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568a6b8c470 con 0x5568a6c8b830 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.458+0000 7f4498a47640 1 --2- 172.21.15.47:0/2382365947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 0x5568a6c8bc10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4488010ac0 tx=0x7f4488013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec265a9ae377462c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.458+0000 7f447e7fc640 1 -- 172.21.15.47:0/2382365947 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488016c10 con 0x5568a6c8b830 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.458+0000 7f447e7fc640 1 -- 172.21.15.47:0/2382365947 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448801a070 con 0x5568a6c8b830 2024-03-23T09:46:52.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.458+0000 7f447e7fc640 1 -- 172.21.15.47:0/2382365947 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488021440 con 0x5568a6c8b830 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.458+0000 7f449d81a740 1 -- 172.21.15.47:0/2382365947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 msgr2=0x5568a6c8bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.458+0000 7f449d81a740 1 --2- 172.21.15.47:0/2382365947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 0x5568a6c8bc10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4488010ac0 tx=0x7f4488013000 comp rx=0 tx=0).stop 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.458+0000 7f449d81a740 1 -- 172.21.15.47:0/2382365947 shutdown_connections 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.458+0000 7f449d81a740 1 --2- 172.21.15.47:0/2382365947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c5f3b0 0x5568a6c88fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.458+0000 7f449d81a740 1 --2- 172.21.15.47:0/2382365947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5b850 0x5568a6c88aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.458+0000 7f449d81a740 1 --2- 172.21.15.47:0/2382365947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 0x5568a6c8bc10 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.458+0000 7f449d81a740 1 -- 172.21.15.47:0/2382365947 >> 172.21.15.47:0/2382365947 conn(0x5568a6c5dac0 msgr2=0x5568a6ca6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.458+0000 7f449d81a740 1 -- 172.21.15.47:0/2382365947 shutdown_connections 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.458+0000 7f449d81a740 1 -- 172.21.15.47:0/2382365947 wait complete. 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.459+0000 7f449d81a740 1 -- start start 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.459+0000 7f449d81a740 1 Processor -- start 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f449d81a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c5f3b0 0x5568a6c75890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f449d81a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 0x5568a6c72c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f449d81a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c745c0 0x5568a6ca9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f449d81a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a6b83280 con 0x5568a6c8b830 2024-03-23T09:46:52.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f449d81a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a6b82f20 con 0x5568a6c745c0 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f449d81a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568a6b483c0 con 0x5568a6c5f3b0 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f449d81a740 10 client.0 _ll_register_callbacks cb 0x5568a6b9a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f4493fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 0x5568a6c72c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f4493fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 0x5568a6c72c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36124/0 (socket says 172.21.15.47:36124) 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f4493fff640 1 -- 172.21.15.47:0/2354732565 learned_addr learned my addr 172.21.15.47:0/2354732565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f4498a47640 1 --2- 172.21.15.47:0/2354732565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c5f3b0 0x5568a6c75890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f4499248640 1 --2- 172.21.15.47:0/2354732565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c745c0 0x5568a6ca9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f4493fff640 1 -- 172.21.15.47:0/2354732565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c5f3b0 msgr2=0x5568a6c75890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f4493fff640 1 --2- 172.21.15.47:0/2354732565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c5f3b0 0x5568a6c75890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f4493fff640 1 -- 172.21.15.47:0/2354732565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c745c0 msgr2=0x5568a6ca9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f4493fff640 1 --2- 172.21.15.47:0/2354732565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c745c0 0x5568a6ca9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.460+0000 7f4493fff640 1 -- 172.21.15.47:0/2354732565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448800e040 con 0x5568a6c8b830 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f4493fff640 1 --2- 172.21.15.47:0/2354732565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 0x5568a6c72c20 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4480005560 tx=0x7f44800105b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251eaa196be8e13f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f44737fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:46:52.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f44737fe640 1 -- 172.21.15.47:0/2354732565 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480015660 con 0x5568a6c8b830 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f44737fe640 1 -- 172.21.15.47:0/2354732565 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f44800157c0 con 0x5568a6c8b830 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f449d81a740 1 -- 172.21.15.47:0/2354732565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568a6bee690 con 0x5568a6c8b830 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f44737fe640 1 -- 172.21.15.47:0/2354732565 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4480015980 con 0x5568a6c8b830 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f449d81a740 10 client.7281 fetch_fsmap learned FSMap version 40 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f449d81a740 1 -- 172.21.15.47:0/2354732565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x5568a6bc34d0 con 0x5568a6c8b830 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f44737fe640 1 -- 172.21.15.47:0/2354732565 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5568a6bee690 con 0x5568a6c8b830 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f44737fe640 1 -- 172.21.15.47:0/2354732565 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4480015ae0 con 0x5568a6c8b830 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f449d81a740 10 client.7281 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f449d81a740 10 client.7281 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f449d81a740 20 client.7281 populate_metadata read hostname 'smithi047' 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f449d81a740 1 --2- 172.21.15.47:0/2354732565 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5568a6ca6a20 0x5568a6c84880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f449d81a740 4 client.7281 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f449d81a740 1 -- 172.21.15.47:0/2354732565 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568a6bb6eb0 con 0x5568a6ca6a20 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.461+0000 7f4498a47640 1 --2- 172.21.15.47:0/2354732565 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5568a6ca6a20 0x5568a6c84880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f4498a47640 1 --2- 172.21.15.47:0/2354732565 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5568a6ca6a20 0x5568a6c84880 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=489b83a3915208d6 server_cookie=5ceb880762b5e41 in_seq=0 out_seq=0 2024-03-23T09:46:52.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f44737fe640 10 client.7281 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f44737fe640 1 -- 172.21.15.47:0/2354732565 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5568a6bb6eb0 con 0x5568a6ca6a20 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f44737fe640 10 client.7281 handle_command_reply: tid=0 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f449d81a740 1 client.7281 shutdown 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f449d81a740 1 -- 172.21.15.47:0/2354732565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 msgr2=0x5568a6c72c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f449d81a740 1 --2- 172.21.15.47:0/2354732565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 0x5568a6c72c20 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4480005560 tx=0x7f44800105b0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f449d81a740 1 -- 172.21.15.47:0/2354732565 shutdown_connections 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f449d81a740 1 --2- 172.21.15.47:0/2354732565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c745c0 0x5568a6ca9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f449d81a740 1 --2- 172.21.15.47:0/2354732565 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5568a6ca6a20 0x5568a6c84880 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f449d81a740 1 --2- 172.21.15.47:0/2354732565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c8b830 0x5568a6c72c20 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f449d81a740 1 --2- 172.21.15.47:0/2354732565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c5f3b0 0x5568a6c75890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f449d81a740 1 -- 172.21.15.47:0/2354732565 >> 172.21.15.47:0/2354732565 conn(0x5568a6c5b850 msgr2=0x5568a6c8f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.462+0000 7f44737fe640 0 client.7281 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.463+0000 7f449d81a740 1 -- 172.21.15.47:0/2354732565 shutdown_connections 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.463+0000 7f449d81a740 1 -- 172.21.15.47:0/2354732565 wait complete. 2024-03-23T09:46:52.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.463+0000 7f449d81a740 20 client.7281 trim_cache size 0 max 16384 2024-03-23T09:46:52.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.477+0000 7f449d81a740 1 Processor -- start 2024-03-23T09:46:52.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.477+0000 7f449d81a740 1 -- start start 2024-03-23T09:46:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f449d81a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5f3b0 0x5568a6d032f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f449d81a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 0x5568a6c5a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f449d81a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c8b830 0x5568a6c5ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f449d81a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a6c5d3c0 con 0x5568a6c745c0 2024-03-23T09:46:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f449d81a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a6c843d0 con 0x5568a6c5f3b0 2024-03-23T09:46:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f449d81a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568a6c576f0 con 0x5568a6c8b830 2024-03-23T09:46:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f4493fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 0x5568a6c5a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f4493fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 0x5568a6c5a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36136/0 (socket says 172.21.15.47:36136) 2024-03-23T09:46:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f4493fff640 1 -- 172.21.15.47:0/369042535 learned_addr learned my addr 172.21.15.47:0/369042535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f4499248640 1 --2- 172.21.15.47:0/369042535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c8b830 0x5568a6c5ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f4498a47640 1 --2- 172.21.15.47:0/369042535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5f3b0 0x5568a6d032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f4493fff640 1 -- 172.21.15.47:0/369042535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c8b830 msgr2=0x5568a6c5ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f4493fff640 1 --2- 172.21.15.47:0/369042535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c8b830 0x5568a6c5ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f4493fff640 1 -- 172.21.15.47:0/369042535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5f3b0 msgr2=0x5568a6d032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f4493fff640 1 --2- 172.21.15.47:0/369042535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5f3b0 0x5568a6d032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f4493fff640 1 -- 172.21.15.47:0/369042535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568a6bf79e0 con 0x5568a6c745c0 2024-03-23T09:46:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f4498a47640 1 --2- 172.21.15.47:0/369042535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5f3b0 0x5568a6d032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f4493fff640 1 --2- 172.21.15.47:0/369042535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 0x5568a6c5a910 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4480001eb0 tx=0x7f448000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78f27c570beecc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f447effd640 1 -- 172.21.15.47:0/369042535 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800067d0 con 0x5568a6c745c0 2024-03-23T09:46:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.478+0000 7f447effd640 1 -- 172.21.15.47:0/369042535 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4480006930 con 0x5568a6c745c0 2024-03-23T09:46:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.479+0000 7f447effd640 1 -- 172.21.15.47:0/369042535 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480006cc0 con 0x5568a6c745c0 2024-03-23T09:46:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.479+0000 7f449d81a740 1 -- 172.21.15.47:0/369042535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 msgr2=0x5568a6c5a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.479+0000 7f449d81a740 1 --2- 172.21.15.47:0/369042535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 0x5568a6c5a910 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4480001eb0 tx=0x7f448000a000 comp rx=0 tx=0).stop 2024-03-23T09:46:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.479+0000 7f449d81a740 1 -- 172.21.15.47:0/369042535 shutdown_connections 2024-03-23T09:46:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.479+0000 7f449d81a740 1 --2- 172.21.15.47:0/369042535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c8b830 0x5568a6c5ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.479+0000 7f449d81a740 1 --2- 172.21.15.47:0/369042535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 0x5568a6c5a910 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.479+0000 7f449d81a740 1 --2- 172.21.15.47:0/369042535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5f3b0 0x5568a6d032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.479+0000 7f449d81a740 1 -- 172.21.15.47:0/369042535 >> 172.21.15.47:0/369042535 conn(0x5568a6c5b850 msgr2=0x5568a6ca9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.479+0000 7f449d81a740 1 -- 172.21.15.47:0/369042535 shutdown_connections 2024-03-23T09:46:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.479+0000 7f449d81a740 1 -- 172.21.15.47:0/369042535 wait complete. 2024-03-23T09:46:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.480+0000 7f449d81a740 1 -- start start 2024-03-23T09:46:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.480+0000 7f449d81a740 1 Processor -- start 2024-03-23T09:46:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f449d81a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5f3b0 0x5568a6c755a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f449d81a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 0x5568a6c8e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f449d81a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c8b830 0x5568a6cf4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f449d81a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a6d0dad0 con 0x5568a6c745c0 2024-03-23T09:46:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f449d81a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a6c8ef60 con 0x5568a6c5f3b0 2024-03-23T09:46:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f449d81a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568a6c86950 con 0x5568a6c8b830 2024-03-23T09:46:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f449d81a740 10 client.0 _ll_register_callbacks cb 0x7f4480004870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:46:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f4493fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 0x5568a6c8e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f4493fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 0x5568a6c8e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36150/0 (socket says 172.21.15.47:36150) 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f4493fff640 1 -- 172.21.15.47:0/538348997 learned_addr learned my addr 172.21.15.47:0/538348997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f4499248640 1 --2- 172.21.15.47:0/538348997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c8b830 0x5568a6cf4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f4498a47640 1 --2- 172.21.15.47:0/538348997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5f3b0 0x5568a6c755a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f4493fff640 1 -- 172.21.15.47:0/538348997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c8b830 msgr2=0x5568a6cf4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f4493fff640 1 --2- 172.21.15.47:0/538348997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c8b830 0x5568a6cf4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f4493fff640 1 -- 172.21.15.47:0/538348997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5f3b0 msgr2=0x5568a6c755a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f4493fff640 1 --2- 172.21.15.47:0/538348997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5f3b0 0x5568a6c755a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f4493fff640 1 -- 172.21.15.47:0/538348997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568a6bc34d0 con 0x5568a6c745c0 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f4493fff640 1 --2- 172.21.15.47:0/538348997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 0x5568a6c8e610 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f4480011010 tx=0x7f4480003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0d5dd5f0e35c04 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f44737fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f44737fe640 1 -- 172.21.15.47:0/538348997 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800067d0 con 0x5568a6c745c0 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f44737fe640 1 -- 172.21.15.47:0/538348997 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4480000e40 con 0x5568a6c745c0 2024-03-23T09:46:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.481+0000 7f449d81a740 1 -- 172.21.15.47:0/538348997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f448801a070 con 0x5568a6c745c0 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f44737fe640 1 -- 172.21.15.47:0/538348997 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4480015c30 con 0x5568a6c745c0 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f449d81a740 10 client.7287 fetch_fsmap learned FSMap version 40 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f449d81a740 1 -- 172.21.15.47:0/538348997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x5568a6b655c0 con 0x5568a6c745c0 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f44737fe640 1 -- 172.21.15.47:0/538348997 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448801a070 con 0x5568a6c745c0 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f44737fe640 1 -- 172.21.15.47:0/538348997 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4480039030 con 0x5568a6c745c0 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f449d81a740 10 client.7287 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f449d81a740 10 client.7287 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f449d81a740 20 client.7287 populate_metadata read hostname 'smithi047' 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f449d81a740 1 --2- 172.21.15.47:0/538348997 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5568a6cf5110 0x5568a6d0c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f449d81a740 4 client.7287 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f449d81a740 1 -- 172.21.15.47:0/538348997 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5568a68b84e0 con 0x5568a6cf5110 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f4498a47640 1 --2- 172.21.15.47:0/538348997 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5568a6cf5110 0x5568a6d0c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f4498a47640 1 --2- 172.21.15.47:0/538348997 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5568a6cf5110 0x5568a6d0c900 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6760b079dc42e1e7 server_cookie=98cb711845b0a2db in_seq=0 out_seq=0 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f44737fe640 10 client.7287 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f44737fe640 1 -- 172.21.15.47:0/538348997 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5568a68b84e0 con 0x5568a6cf5110 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f44737fe640 10 client.7287 handle_command_reply: tid=0 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.482+0000 7f449d81a740 1 client.7287 shutdown 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449d81a740 1 -- 172.21.15.47:0/538348997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 msgr2=0x5568a6c8e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449d81a740 1 --2- 172.21.15.47:0/538348997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 0x5568a6c8e610 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f4480011010 tx=0x7f4480003760 comp rx=0 tx=0).stop 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449d81a740 1 -- 172.21.15.47:0/538348997 shutdown_connections 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449d81a740 1 --2- 172.21.15.47:0/538348997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5568a6c8b830 0x5568a6cf4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449d81a740 1 --2- 172.21.15.47:0/538348997 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x5568a6cf5110 0x5568a6d0c900 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449d81a740 1 --2- 172.21.15.47:0/538348997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5568a6c745c0 0x5568a6c8e610 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449d81a740 1 --2- 172.21.15.47:0/538348997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568a6c5f3b0 0x5568a6c755a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f44737fe640 0 client.7287 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449d81a740 1 -- 172.21.15.47:0/538348997 >> 172.21.15.47:0/538348997 conn(0x5568a6c5b850 msgr2=0x5568a6c5a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449d81a740 1 -- 172.21.15.47:0/538348997 shutdown_connections 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449d81a740 1 -- 172.21.15.47:0/538348997 wait complete. 2024-03-23T09:46:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449d81a740 20 client.7287 trim_cache size 0 max 16384 2024-03-23T09:46:52.491 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:46:52.491 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-23T09:46:52.491 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-23T09:46:52.491 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:46:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449acab640 1 -- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f447806ac00 msgr2=0x7f447806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449acab640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f447806ac00 0x7f447806d0c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f44880079a0 tx=0x7f4488008040 comp rx=0 tx=0).stop 2024-03-23T09:46:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449acab640 1 -- 172.21.15.47:0/1940566685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f449415cc30 msgr2=0x7f44941f5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449acab640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f449415cc30 0x7f44941f5da0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f448c009e50 tx=0x7f448c004a40 comp rx=0 tx=0).stop 2024-03-23T09:46:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449acab640 1 -- 172.21.15.47:0/1940566685 shutdown_connections 2024-03-23T09:46:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449acab640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f447806ac00 0x7f447806d0c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449acab640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f449415cc30 0x7f44941f5da0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449acab640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44940cd4b0 0x7f44940c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449acab640 1 --2- 172.21.15.47:0/1940566685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44940cad40 0x7f449415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.483+0000 7f449acab640 1 -- 172.21.15.47:0/1940566685 >> 172.21.15.47:0/1940566685 conn(0x7f44940d06f0 msgr2=0x7f449415a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.484+0000 7f449acab640 1 -- 172.21.15.47:0/1940566685 shutdown_connections 2024-03-23T09:46:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.484+0000 7f449acab640 1 -- 172.21.15.47:0/1940566685 wait complete. 2024-03-23T09:46:52.495 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T09:46:52.495 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:46:52.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.596+0000 7fa9c62e5640 1 Processor -- start 2024-03-23T09:46:52.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.596+0000 7fa9c62e5640 1 -- start start 2024-03-23T09:46:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.597+0000 7fa9c62e5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9c0160610 0x7fa9c00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.597+0000 7fa9c62e5640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa9c00ae450 0x7fa9c01609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.597+0000 7fa9c62e5640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c00cebf0 con 0x7fa9c015d020 2024-03-23T09:46:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.597+0000 7fa9c62e5640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c01614d0 con 0x7fa9c00ae450 2024-03-23T09:46:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.597+0000 7fa9c62e5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c0069ea0 con 0x7fa9c0160610 2024-03-23T09:46:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.597+0000 7fa9c4882640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa9c015d020 0x7fa9c015d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:49890/0 (socket says 172.21.15.47:49890) 2024-03-23T09:46:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.597+0000 7fa9c4882640 1 -- 172.21.15.47:0/2785433809 learned_addr learned my addr 172.21.15.47:0/2785433809 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.597+0000 7fa9bf7fe640 1 --2- 172.21.15.47:0/2785433809 >> v2:172.21.15.175:3300/0 conn(0x7fa9c00ae450 0x7fa9c01609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.597+0000 7fa9bffff640 1 --2- 172.21.15.47:0/2785433809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9c0160610 0x7fa9c00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.597+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181344883 0 0) 0x7fa9c00cebf0 con 0x7fa9c015d020 2024-03-23T09:46:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.597+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9a8003520 con 0x7fa9c015d020 2024-03-23T09:46:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1732092248 0 0) 0x7fa9a8003520 con 0x7fa9c015d020 2024-03-23T09:46:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9c0160610 msgr2=0x7fa9c00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 --2- 172.21.15.47:0/2785433809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9c0160610 0x7fa9c00ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 >> v2:172.21.15.175:3300/0 conn(0x7fa9c00ae450 msgr2=0x7fa9c01609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 --2- 172.21.15.47:0/2785433809 >> v2:172.21.15.175:3300/0 conn(0x7fa9c00ae450 0x7fa9c01609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c00ae8b0 con 0x7fa9c015d020 2024-03-23T09:46:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fa9b80019f0 con 0x7fa9c015d020 2024-03-23T09:46:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9bf7fe640 1 --2- 172.21.15.47:0/2785433809 >> v2:172.21.15.175:3300/0 conn(0x7fa9c00ae450 0x7fa9c01609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:46:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 >> v1:172.21.15.47:6789/0 conn(0x7fa9c015d020 legacy=0x7fa9c015d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 --2- 172.21.15.47:0/2785433809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9c0160610 0x7fa9a8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 --2- 172.21.15.47:0/2785433809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9a8003ef0 0x7fa9a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 --2- 172.21.15.47:0/2785433809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a8004d00 0x7fa9a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c0069ea0 con 0x7fa9a8003ef0 2024-03-23T09:46:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a800b9e0 con 0x7fa9a8004d00 2024-03-23T09:46:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a800bba0 con 0x7fa9c0160610 2024-03-23T09:46:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9bf7fe640 1 --2- 172.21.15.47:0/2785433809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9a8003ef0 0x7fa9a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9bffff640 1 --2- 172.21.15.47:0/2785433809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9c0160610 0x7fa9a8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9c4882640 1 --2- 172.21.15.47:0/2785433809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a8004d00 0x7fa9a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9bf7fe640 1 -- 172.21.15.47:0/2785433809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9c0160610 msgr2=0x7fa9a8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9bf7fe640 1 --2- 172.21.15.47:0/2785433809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9c0160610 0x7fa9a8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9bf7fe640 1 -- 172.21.15.47:0/2785433809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a8004d00 msgr2=0x7fa9a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9bf7fe640 1 --2- 172.21.15.47:0/2785433809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a8004d00 0x7fa9a8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.598+0000 7fa9bf7fe640 1 -- 172.21.15.47:0/2785433809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8003520 con 0x7fa9a8003ef0 2024-03-23T09:46:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.599+0000 7fa9bf7fe640 1 --2- 172.21.15.47:0/2785433809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9a8003ef0 0x7fa9a80047c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4009c90 tx=0x7fa9a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c557548f3b86f93f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.599+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b402c980 con 0x7fa9a8003ef0 2024-03-23T09:46:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.599+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b402cae0 con 0x7fa9a8003ef0 2024-03-23T09:46:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.599+0000 7fa9beffd640 1 -- 172.21.15.47:0/2785433809 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4034440 con 0x7fa9a8003ef0 2024-03-23T09:46:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.599+0000 7fa9c62e5640 1 -- 172.21.15.47:0/2785433809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9a8003ef0 msgr2=0x7fa9a80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.599+0000 7fa9c62e5640 1 --2- 172.21.15.47:0/2785433809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9a8003ef0 0x7fa9a80047c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4009c90 tx=0x7fa9a80057f0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.599+0000 7fa9c4882640 1 -- 172.21.15.47:0/2785433809 reap_dead start 2024-03-23T09:46:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.599+0000 7fa9c62e5640 1 -- 172.21.15.47:0/2785433809 shutdown_connections 2024-03-23T09:46:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.599+0000 7fa9c62e5640 1 -- 172.21.15.47:0/2785433809 >> 172.21.15.47:0/2785433809 conn(0x7fa9c00ced80 msgr2=0x7fa9c00cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.599+0000 7fa9c62e5640 1 -- 172.21.15.47:0/2785433809 shutdown_connections 2024-03-23T09:46:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.600+0000 7fa9c62e5640 1 -- 172.21.15.47:0/2785433809 wait complete. 2024-03-23T09:46:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.601+0000 7fa9c62e5640 1 Processor -- start 2024-03-23T09:46:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.601+0000 7fa9c62e5640 1 -- start start 2024-03-23T09:46:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.601+0000 7fa9c62e5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a8003ef0 0x7fa9c0145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.601+0000 7fa9c62e5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9c00b3b30 0x7fa9c00afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.601+0000 7fa9c62e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9c00b0080 0x7fa9c0157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.601+0000 7fa9c62e5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c0158a70 con 0x7fa9c00b0080 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9c62e5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c015a3e0 con 0x7fa9c00b3b30 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9c62e5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c00b0500 con 0x7fa9a8003ef0 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9c4882640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9c00b0080 0x7fa9c0157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9c4882640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9c00b0080 0x7fa9c0157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36158/0 (socket says 172.21.15.47:36158) 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9c4882640 1 -- 172.21.15.47:0/266092849 learned_addr learned my addr 172.21.15.47:0/266092849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9bffff640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a8003ef0 0x7fa9c0145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9bf7fe640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9c00b3b30 0x7fa9c00afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9c4882640 1 -- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a8003ef0 msgr2=0x7fa9c0145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9c4882640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a8003ef0 0x7fa9c0145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9c4882640 1 -- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9c00b3b30 msgr2=0x7fa9c00afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9c4882640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9c00b3b30 0x7fa9c00afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9c4882640 1 -- 172.21.15.47:0/266092849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c00b36a0 con 0x7fa9c00b0080 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9bf7fe640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9c00b3b30 0x7fa9c00afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9c4882640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9c00b0080 0x7fa9c0157e60 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa9c006b850 tx=0x7fa9b8007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9bcff9640 1 -- 172.21.15.47:0/266092849 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b800abc0 con 0x7fa9c00b0080 2024-03-23T09:46:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9bcff9640 1 -- 172.21.15.47:0/266092849 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b800ad40 con 0x7fa9c00b0080 2024-03-23T09:46:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9c62e5640 1 -- 172.21.15.47:0/266092849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c0145d40 con 0x7fa9c00b0080 2024-03-23T09:46:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9bcff9640 1 -- 172.21.15.47:0/266092849 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8013630 con 0x7fa9c00b0080 2024-03-23T09:46:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.602+0000 7fa9c62e5640 1 -- 172.21.15.47:0/266092849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c00b3050 con 0x7fa9c00b0080 2024-03-23T09:46:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.603+0000 7fa9c62e5640 1 -- 172.21.15.47:0/266092849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c0066be0 con 0x7fa9c00b0080 2024-03-23T09:46:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.604+0000 7fa9bcff9640 1 -- 172.21.15.47:0/266092849 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa9b8013790 con 0x7fa9c00b0080 2024-03-23T09:46:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.604+0000 7fa9bcff9640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa9ac06ac00 0x7fa9ac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.604+0000 7fa9bcff9640 1 -- 172.21.15.47:0/266092849 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa9b8089ae0 con 0x7fa9c00b0080 2024-03-23T09:46:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.604+0000 7fa9bffff640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa9ac06ac00 0x7fa9ac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.605+0000 7fa9bffff640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa9ac06ac00 0x7fa9ac06d0c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009a00 tx=0x7fa9b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.609+0000 7fa9bcff9640 1 -- 172.21.15.47:0/266092849 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa9b8054cf0 con 0x7fa9c00b0080 2024-03-23T09:46:52.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.824+0000 7fa9c62e5640 1 -- 172.21.15.47:0/266092849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9c009cea0 con 0x7fa9c00b0080 2024-03-23T09:46:52.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.824+0000 7fa9bcff9640 1 -- 172.21.15.47:0/266092849 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4741 (secure 0 0 0) 0x7fa9b8058310 con 0x7fa9c00b0080 2024-03-23T09:46:52.825 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:46:52.825 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3862818105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3862818105},{"type":"v1","addr":"172.21.15.175:6836","nonce":3862818105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3615592850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3615592850},{"type":"v1","addr":"172.21.15.175:6837","nonce":3615592850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/832829684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":832829684},{"type":"v1","addr":"172.21.15.47:6834","nonce":832829684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:45:18.296714+0000","modified":"2024-03-23T09:46:24.891678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7005":{"gid":7005,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":16,"addr":"172.21.15.47:6835/4141889683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4141889683},{"type":"v1","addr":"172.21.15.47:6835","nonce":4141889683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-23T09:46:52.826 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-23T09:46:52.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.826+0000 7fa9c62e5640 1 -- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa9ac06ac00 msgr2=0x7fa9ac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.826+0000 7fa9c62e5640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa9ac06ac00 0x7fa9ac06d0c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009a00 tx=0x7fa9b0008040 comp rx=0 tx=0).stop 2024-03-23T09:46:52.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.826+0000 7fa9c62e5640 1 -- 172.21.15.47:0/266092849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9c00b0080 msgr2=0x7fa9c0157e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.826+0000 7fa9c62e5640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9c00b0080 0x7fa9c0157e60 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa9c006b850 tx=0x7fa9b8007b00 comp rx=0 tx=0).stop 2024-03-23T09:46:52.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.826+0000 7fa9c62e5640 1 -- 172.21.15.47:0/266092849 shutdown_connections 2024-03-23T09:46:52.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.826+0000 7fa9c62e5640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa9ac06ac00 0x7fa9ac06d0c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.826+0000 7fa9c62e5640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9c00b0080 0x7fa9c0157e60 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.826+0000 7fa9c62e5640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9c00b3b30 0x7fa9c00afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.826+0000 7fa9c62e5640 1 --2- 172.21.15.47:0/266092849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a8003ef0 0x7fa9c0145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.826+0000 7fa9c62e5640 1 -- 172.21.15.47:0/266092849 >> 172.21.15.47:0/266092849 conn(0x7fa9c00ced80 msgr2=0x7fa9c00cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.826+0000 7fa9c62e5640 1 -- 172.21.15.47:0/266092849 shutdown_connections 2024-03-23T09:46:52.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.826+0000 7fa9c62e5640 1 -- 172.21.15.47:0/266092849 wait complete. 2024-03-23T09:46:52.838 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-23T09:46:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.941+0000 7f4bba638640 1 Processor -- start 2024-03-23T09:46:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.941+0000 7f4bba638640 1 -- start start 2024-03-23T09:46:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bba638640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4bb415c990 0x7f4bb4160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bba638640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bb40b8350 0x7f4bb40cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bba638640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb4161630 con 0x7f4bb40cdc40 2024-03-23T09:46:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bba638640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb40ce0b0 con 0x7f4bb415c990 2024-03-23T09:46:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bba638640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb40d1030 con 0x7f4bb40b8350 2024-03-23T09:46:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bb8bd5640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4bb40cdc40 0x7f4bb40b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:49894/0 (socket says 172.21.15.47:49894) 2024-03-23T09:46:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bb8bd5640 1 -- 172.21.15.47:0/2521599345 learned_addr learned my addr 172.21.15.47:0/2521599345 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bb3fff640 1 --2- 172.21.15.47:0/2521599345 >> v2:172.21.15.175:3300/0 conn(0x7f4bb415c990 0x7f4bb4160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bb37fe640 1 --2- 172.21.15.47:0/2521599345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bb40b8350 0x7f4bb40cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bb2ffd640 1 -- 172.21.15.47:0/2521599345 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971010137 0 0) 0x7f4bb4161630 con 0x7f4bb40cdc40 2024-03-23T09:46:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bb2ffd640 1 -- 172.21.15.47:0/2521599345 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b9c003520 con 0x7f4bb40cdc40 2024-03-23T09:46:52.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bb3fff640 1 -- 172.21.15.47:0/2521599345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bb40b8350 msgr2=0x7f4bb40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bb3fff640 1 --2- 172.21.15.47:0/2521599345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bb40b8350 0x7f4bb40cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.942+0000 7f4bb3fff640 1 -- 172.21.15.47:0/2521599345 >> v1:172.21.15.47:6789/0 conn(0x7f4bb40cdc40 legacy=0x7f4bb40b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.943+0000 7f4bb3fff640 1 -- 172.21.15.47:0/2521599345 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb40cfe10 con 0x7f4bb415c990 2024-03-23T09:46:52.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.943+0000 7f4bb37fe640 1 --2- 172.21.15.47:0/2521599345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bb40b8350 0x7f4bb40cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:46:52.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.943+0000 7f4bb3fff640 1 --2- 172.21.15.47:0/2521599345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb415c990 0x7f4bb4160690 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40097a0 tx=0x7f4ba402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27303d5283539cd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.943+0000 7f4bb2ffd640 1 -- 172.21.15.47:0/2521599345 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba402e070 con 0x7f4bb415c990 2024-03-23T09:46:52.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.943+0000 7f4bb2ffd640 1 -- 172.21.15.47:0/2521599345 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba40315a0 con 0x7f4bb415c990 2024-03-23T09:46:52.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.944+0000 7f4bb2ffd640 1 -- 172.21.15.47:0/2521599345 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4031910 con 0x7f4bb415c990 2024-03-23T09:46:52.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.944+0000 7f4bba638640 1 -- 172.21.15.47:0/2521599345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb415c990 msgr2=0x7f4bb4160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.944+0000 7f4bba638640 1 --2- 172.21.15.47:0/2521599345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb415c990 0x7f4bb4160690 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40097a0 tx=0x7f4ba402b840 comp rx=0 tx=0).stop 2024-03-23T09:46:52.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.944+0000 7f4bba638640 1 -- 172.21.15.47:0/2521599345 shutdown_connections 2024-03-23T09:46:52.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.944+0000 7f4bba638640 1 --2- 172.21.15.47:0/2521599345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bb40b8350 0x7f4bb40cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.944+0000 7f4bba638640 1 --2- 172.21.15.47:0/2521599345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb415c990 0x7f4bb4160690 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.944+0000 7f4bba638640 1 -- 172.21.15.47:0/2521599345 >> 172.21.15.47:0/2521599345 conn(0x7f4bb40659d0 msgr2=0x7f4bb4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.944+0000 7f4bba638640 1 -- 172.21.15.47:0/2521599345 shutdown_connections 2024-03-23T09:46:52.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.944+0000 7f4bba638640 1 -- 172.21.15.47:0/2521599345 wait complete. 2024-03-23T09:46:52.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.945+0000 7f4bba638640 1 Processor -- start 2024-03-23T09:46:52.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.945+0000 7f4bba638640 1 -- start start 2024-03-23T09:46:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.946+0000 7f4bba638640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb40b8350 0x7f4bb4160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.946+0000 7f4bba638640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bb40cdc40 0x7f4bb415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.946+0000 7f4bba638640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb415c990 0x7f4bb40c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.946+0000 7f4bba638640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb415a2d0 con 0x7f4bb40b8350 2024-03-23T09:46:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.946+0000 7f4bba638640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb40c77e0 con 0x7f4bb415c990 2024-03-23T09:46:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.946+0000 7f4bba638640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb40c7ac0 con 0x7f4bb40cdc40 2024-03-23T09:46:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.946+0000 7f4bb3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb40b8350 0x7f4bb4160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.946+0000 7f4bb3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb40b8350 0x7f4bb4160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36172/0 (socket says 172.21.15.47:36172) 2024-03-23T09:46:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.946+0000 7f4bb3fff640 1 -- 172.21.15.47:0/2558576270 learned_addr learned my addr 172.21.15.47:0/2558576270 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bb37fe640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bb40cdc40 0x7f4bb415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bb8bd5640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb415c990 0x7f4bb40c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bb3fff640 1 -- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bb40cdc40 msgr2=0x7f4bb415aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bb3fff640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bb40cdc40 0x7f4bb415aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bb3fff640 1 -- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb415c990 msgr2=0x7f4bb40c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bb3fff640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb415c990 0x7f4bb40c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bb3fff640 1 -- 172.21.15.47:0/2558576270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba8009400 con 0x7f4bb40b8350 2024-03-23T09:46:52.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bb8bd5640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb415c990 0x7f4bb40c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:52.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bb3fff640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb40b8350 0x7f4bb4160280 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4034040 tx=0x7f4ba40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bb0ff9640 1 -- 172.21.15.47:0/2558576270 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4032630 con 0x7f4bb40b8350 2024-03-23T09:46:52.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bb0ff9640 1 -- 172.21.15.47:0/2558576270 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba4032790 con 0x7f4bb40b8350 2024-03-23T09:46:52.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bba638640 1 -- 172.21.15.47:0/2558576270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba4009400 con 0x7f4bb40b8350 2024-03-23T09:46:52.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bb0ff9640 1 -- 172.21.15.47:0/2558576270 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4032b00 con 0x7f4bb40b8350 2024-03-23T09:46:52.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.947+0000 7f4bba638640 1 -- 172.21.15.47:0/2558576270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb40c8360 con 0x7f4bb40b8350 2024-03-23T09:46:52.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.948+0000 7f4bb0ff9640 1 -- 172.21.15.47:0/2558576270 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4ba4032c60 con 0x7f4bb40b8350 2024-03-23T09:46:52.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.949+0000 7f4bb0ff9640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ba006ac00 0x7f4ba006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.949+0000 7f4bb0ff9640 1 -- 172.21.15.47:0/2558576270 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4ba40ab240 con 0x7f4bb40b8350 2024-03-23T09:46:52.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.949+0000 7f4bb37fe640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ba006ac00 0x7f4ba006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.949+0000 7f4bb37fe640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ba006ac00 0x7f4ba006d0c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40ccd20 tx=0x7f4ba8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.950+0000 7f4bbd1aa740 1 Processor -- start 2024-03-23T09:46:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.950+0000 7f4bbd1aa740 1 -- start start 2024-03-23T09:46:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bbd1aa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e20b6b0 0x55e95e20ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bbd1aa740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e1db6d0 0x55e95e208920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bbd1aa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e1df230 0x55e95e208e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bbd1aa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95e14c6a0 con 0x55e95e1df230 2024-03-23T09:46:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bbd1aa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95e149c70 con 0x55e95e20b6b0 2024-03-23T09:46:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bbd1aa740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e95e145560 con 0x55e95e1db6d0 2024-03-23T09:46:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bb8bd5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e1df230 0x55e95e208e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bb8bd5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e1df230 0x55e95e208e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36180/0 (socket says 172.21.15.47:36180) 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bb8bd5640 1 -- 172.21.15.47:0/3747365274 learned_addr learned my addr 172.21.15.47:0/3747365274 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bb3fff640 1 --2- 172.21.15.47:0/3747365274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e20b6b0 0x55e95e20ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bb37fe640 1 --2- 172.21.15.47:0/3747365274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e1db6d0 0x55e95e208920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bb8bd5640 1 -- 172.21.15.47:0/3747365274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e1db6d0 msgr2=0x55e95e208920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bb8bd5640 1 --2- 172.21.15.47:0/3747365274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e1db6d0 0x55e95e208920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bb8bd5640 1 -- 172.21.15.47:0/3747365274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e20b6b0 msgr2=0x55e95e20ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bb8bd5640 1 --2- 172.21.15.47:0/3747365274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e20b6b0 0x55e95e20ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bb8bd5640 1 -- 172.21.15.47:0/3747365274 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e95e10cd50 con 0x55e95e1df230 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4bb8bd5640 1 --2- 172.21.15.47:0/3747365274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e1df230 0x55e95e208e60 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f4bac002050 tx=0x7f4bac00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b3e2da8918cd54 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4b99ffb640 1 -- 172.21.15.47:0/3747365274 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac00b070 con 0x55e95e1df230 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4b99ffb640 1 -- 172.21.15.47:0/3747365274 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bac013670 con 0x55e95e1df230 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.951+0000 7f4b99ffb640 1 -- 172.21.15.47:0/3747365274 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac0139e0 con 0x55e95e1df230 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.952+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/3747365274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e1df230 msgr2=0x55e95e208e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.952+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/3747365274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e1df230 0x55e95e208e60 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f4bac002050 tx=0x7f4bac00d520 comp rx=0 tx=0).stop 2024-03-23T09:46:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.952+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/3747365274 shutdown_connections 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.952+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/3747365274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e1df230 0x55e95e208e60 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.952+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/3747365274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e1db6d0 0x55e95e208920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.952+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/3747365274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e20b6b0 0x55e95e20ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.952+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/3747365274 >> 172.21.15.47:0/3747365274 conn(0x55e95e1dd940 msgr2=0x55e95e2264c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.952+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/3747365274 shutdown_connections 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.952+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/3747365274 wait complete. 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.953+0000 7f4bbd1aa740 1 -- start start 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.953+0000 7f4bbd1aa740 1 Processor -- start 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.953+0000 7f4bbd1aa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e1df230 0x55e95e20e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bbd1aa740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e20b6b0 0x55e95e21ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bbd1aa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20d410 0x55e95e220480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bbd1aa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95de1e3f0 con 0x55e95e20d410 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bbd1aa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95dece3d0 con 0x55e95e1df230 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bbd1aa740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e95e0c8a10 con 0x55e95e20b6b0 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bbd1aa740 10 client.0 _ll_register_callbacks cb 0x55e95e13ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:46:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bb8bd5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20d410 0x55e95e220480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bb8bd5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20d410 0x55e95e220480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36186/0 (socket says 172.21.15.47:36186) 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bb8bd5640 1 -- 172.21.15.47:0/2048724568 learned_addr learned my addr 172.21.15.47:0/2048724568 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bb3fff640 1 --2- 172.21.15.47:0/2048724568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e1df230 0x55e95e20e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bb37fe640 1 --2- 172.21.15.47:0/2048724568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e20b6b0 0x55e95e21ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bb8bd5640 1 -- 172.21.15.47:0/2048724568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e20b6b0 msgr2=0x55e95e21ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bb8bd5640 1 --2- 172.21.15.47:0/2048724568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e20b6b0 0x55e95e21ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bb8bd5640 1 -- 172.21.15.47:0/2048724568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e1df230 msgr2=0x55e95e20e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bb8bd5640 1 --2- 172.21.15.47:0/2048724568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e1df230 0x55e95e20e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bb8bd5640 1 -- 172.21.15.47:0/2048724568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9c003520 con 0x55e95e20d410 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bb8bd5640 1 --2- 172.21.15.47:0/2048724568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20d410 0x55e95e220480 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f4bac012000 tx=0x7f4bac001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2485c0b3038d08 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4b92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4b92ffd640 1 -- 172.21.15.47:0/2048724568 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac001ce0 con 0x55e95e20d410 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4b92ffd640 1 -- 172.21.15.47:0/2048724568 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4bac013470 con 0x55e95e20d410 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/2048724568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e95e16e150 con 0x55e95e20d410 2024-03-23T09:46:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4b92ffd640 1 -- 172.21.15.47:0/2048724568 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bac00de40 con 0x55e95e20d410 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bbd1aa740 10 client.7305 fetch_fsmap learned FSMap version 40 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.954+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/2048724568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55e95e107760 con 0x55e95e20d410 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.955+0000 7f4b92ffd640 1 -- 172.21.15.47:0/2048724568 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e95e16e150 con 0x55e95e20d410 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.955+0000 7f4b92ffd640 1 -- 172.21.15.47:0/2048724568 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4bac0072e0 con 0x55e95e20d410 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.955+0000 7f4bbd1aa740 10 client.7305 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.955+0000 7f4bbd1aa740 10 client.7305 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.955+0000 7f4bbd1aa740 20 client.7305 populate_metadata read hostname 'smithi047' 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.955+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/2048724568 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55e95e21e310 0x55e95e21e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.955+0000 7f4bbd1aa740 4 client.7305 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.955+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/2048724568 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e95e150650 con 0x55e95e21e310 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.955+0000 7f4bb3fff640 1 --2- 172.21.15.47:0/2048724568 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55e95e21e310 0x55e95e21e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.955+0000 7f4bb3fff640 1 --2- 172.21.15.47:0/2048724568 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55e95e21e310 0x55e95e21e6f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba1682474f41c3a server_cookie=42e986bfd7763d16 in_seq=0 out_seq=0 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.955+0000 7f4b92ffd640 10 client.7305 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4b92ffd640 1 -- 172.21.15.47:0/2048724568 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e95e150650 con 0x55e95e21e310 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4b92ffd640 10 client.7305 handle_command_reply: tid=0 2024-03-23T09:46:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4bbd1aa740 1 client.7305 shutdown 2024-03-23T09:46:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/2048724568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20d410 msgr2=0x55e95e220480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/2048724568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20d410 0x55e95e220480 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f4bac012000 tx=0x7f4bac001250 comp rx=0 tx=0).stop 2024-03-23T09:46:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/2048724568 shutdown_connections 2024-03-23T09:46:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/2048724568 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55e95e21e310 0x55e95e21e6f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/2048724568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20d410 0x55e95e220480 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/2048724568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e20b6b0 0x55e95e21ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/2048724568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e1df230 0x55e95e20e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4b92ffd640 0 client.7305 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/2048724568 >> 172.21.15.47:0/2048724568 conn(0x55e95e1db6d0 msgr2=0x55e95e1dfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/2048724568 shutdown_connections 2024-03-23T09:46:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/2048724568 wait complete. 2024-03-23T09:46:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.956+0000 7f4bbd1aa740 20 client.7305 trim_cache size 0 max 16384 2024-03-23T09:46:52.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.967+0000 7f4bbd1aa740 1 Processor -- start 2024-03-23T09:46:52.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.967+0000 7f4bbd1aa740 1 -- start start 2024-03-23T09:46:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bbd1aa740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e1df230 0x55e95e2213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bbd1aa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 0x55e95e227750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bbd1aa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e20d410 0x55e95e208780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bbd1aa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95e286d90 con 0x55e95e20b6b0 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bbd1aa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95e1e0250 con 0x55e95e20d410 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bbd1aa740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e95e229390 con 0x55e95e1df230 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bb37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 0x55e95e227750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bb37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 0x55e95e227750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36188/0 (socket says 172.21.15.47:36188) 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bb37fe640 1 -- 172.21.15.47:0/3729034572 learned_addr learned my addr 172.21.15.47:0/3729034572 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bb37fe640 1 -- 172.21.15.47:0/3729034572 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e1df230 msgr2=0x55e95e2213e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bb37fe640 1 --2- 172.21.15.47:0/3729034572 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e1df230 0x55e95e2213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bb37fe640 1 -- 172.21.15.47:0/3729034572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e20d410 msgr2=0x55e95e208780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bb37fe640 1 --2- 172.21.15.47:0/3729034572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e20d410 0x55e95e208780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bb37fe640 1 -- 172.21.15.47:0/3729034572 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e95e0e5ad0 con 0x55e95e20b6b0 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.968+0000 7f4bb37fe640 1 --2- 172.21.15.47:0/3729034572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 0x55e95e227750 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80154a0 tx=0x7f4ba8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd470dba92d0c26 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.969+0000 7f4b9a7fc640 1 -- 172.21.15.47:0/3729034572 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba802c470 con 0x55e95e20b6b0 2024-03-23T09:46:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.969+0000 7f4b9a7fc640 1 -- 172.21.15.47:0/3729034572 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba802c5d0 con 0x55e95e20b6b0 2024-03-23T09:46:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.969+0000 7f4b9a7fc640 1 -- 172.21.15.47:0/3729034572 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba802c940 con 0x55e95e20b6b0 2024-03-23T09:46:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.969+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/3729034572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 msgr2=0x55e95e227750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.969+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/3729034572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 0x55e95e227750 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80154a0 tx=0x7f4ba8014000 comp rx=0 tx=0).stop 2024-03-23T09:46:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.969+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/3729034572 shutdown_connections 2024-03-23T09:46:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.969+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/3729034572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e20d410 0x55e95e208780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.969+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/3729034572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 0x55e95e227750 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.969+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/3729034572 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e1df230 0x55e95e2213e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.969+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/3729034572 >> 172.21.15.47:0/3729034572 conn(0x55e95e1db6d0 msgr2=0x55e95e1da4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.969+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/3729034572 shutdown_connections 2024-03-23T09:46:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.969+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/3729034572 wait complete. 2024-03-23T09:46:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.970+0000 7f4bbd1aa740 1 -- start start 2024-03-23T09:46:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.970+0000 7f4bbd1aa740 1 Processor -- start 2024-03-23T09:46:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bbd1aa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e1df230 0x55e95e285f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bbd1aa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 0x55e95e208780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bbd1aa740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e20d410 0x55e95e208cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bbd1aa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95e2388c0 con 0x55e95e20b6b0 2024-03-23T09:46:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bbd1aa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95e286870 con 0x55e95e1df230 2024-03-23T09:46:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bbd1aa740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e95e286480 con 0x55e95e20d410 2024-03-23T09:46:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bbd1aa740 10 client.0 _ll_register_callbacks cb 0x7f4bac016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:46:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bb37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 0x55e95e208780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bb37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 0x55e95e208780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36204/0 (socket says 172.21.15.47:36204) 2024-03-23T09:46:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bb37fe640 1 -- 172.21.15.47:0/494042663 learned_addr learned my addr 172.21.15.47:0/494042663 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bb3fff640 1 --2- 172.21.15.47:0/494042663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e1df230 0x55e95e285f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bb8bd5640 1 --2- 172.21.15.47:0/494042663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e20d410 0x55e95e208cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bb37fe640 1 -- 172.21.15.47:0/494042663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e20d410 msgr2=0x55e95e208cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bb37fe640 1 --2- 172.21.15.47:0/494042663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e20d410 0x55e95e208cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bb37fe640 1 -- 172.21.15.47:0/494042663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e1df230 msgr2=0x55e95e285f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bb37fe640 1 --2- 172.21.15.47:0/494042663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e1df230 0x55e95e285f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bb37fe640 1 -- 172.21.15.47:0/494042663 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba8011d20 con 0x55e95e20b6b0 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bb37fe640 1 --2- 172.21.15.47:0/494042663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 0x55e95e208780 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8015470 tx=0x7f4ba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d65631641043c58 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4b92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4b92ffd640 1 -- 172.21.15.47:0/494042663 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8009810 con 0x55e95e20b6b0 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/494042663 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4bac013670 con 0x55e95e20b6b0 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.971+0000 7f4b92ffd640 1 -- 172.21.15.47:0/494042663 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4ba801b040 con 0x55e95e20b6b0 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4b92ffd640 1 -- 172.21.15.47:0/494042663 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ba8023400 con 0x55e95e20b6b0 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4bbd1aa740 10 client.7311 fetch_fsmap learned FSMap version 40 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/494042663 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55e95e10b5b0 con 0x55e95e20b6b0 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4b92ffd640 1 -- 172.21.15.47:0/494042663 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bac013670 con 0x55e95e20b6b0 2024-03-23T09:46:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4b92ffd640 1 -- 172.21.15.47:0/494042663 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4ba8023560 con 0x55e95e20b6b0 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4bbd1aa740 10 client.7311 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4bbd1aa740 10 client.7311 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4bbd1aa740 20 client.7311 populate_metadata read hostname 'smithi047' 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/494042663 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55e95e21e980 0x55e95e21ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4bbd1aa740 4 client.7311 mds_command: new command op to 7005 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/494042663 --> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55e95ddf68d0 con 0x55e95e21e980 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4bb3fff640 1 --2- 172.21.15.47:0/494042663 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55e95e21e980 0x55e95e21ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4b92ffd640 10 client.7311 ms_handle_connect on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4bb3fff640 1 --2- 172.21.15.47:0/494042663 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55e95e21e980 0x55e95e21ed60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=804082ae15871f65 server_cookie=f55ee5497cc6b192 in_seq=0 out_seq=0 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4b92ffd640 1 -- 172.21.15.47:0/494042663 <== mds.0 v2:172.21.15.47:6833/4141889683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e95ddf68d0 con 0x55e95e21e980 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4b92ffd640 10 client.7311 handle_command_reply: tid=0 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.972+0000 7f4bbd1aa740 1 client.7311 shutdown 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/494042663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 msgr2=0x55e95e208780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/494042663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 0x55e95e208780 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8015470 tx=0x7f4ba80046c0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/494042663 shutdown_connections 2024-03-23T09:46:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/494042663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e95e20d410 0x55e95e208cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/494042663 >> [v2:172.21.15.47:6833/4141889683,v1:172.21.15.47:6835/4141889683] conn(0x55e95e21e980 0x55e95e21ed60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/494042663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e95e20b6b0 0x55e95e208780 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bbd1aa740 1 --2- 172.21.15.47:0/494042663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e95e1df230 0x55e95e285f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/494042663 >> 172.21.15.47:0/494042663 conn(0x55e95e1db6d0 msgr2=0x55e95e1da4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4b92ffd640 0 client.7311 ms_handle_reset on v2:172.21.15.47:6833/4141889683 2024-03-23T09:46:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/494042663 shutdown_connections 2024-03-23T09:46:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bbd1aa740 1 -- 172.21.15.47:0/494042663 wait complete. 2024-03-23T09:46:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bbd1aa740 20 client.7311 trim_cache size 0 max 16384 2024-03-23T09:46:52.979 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:46:52.979 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-23T09:46:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bba638640 1 -- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ba006ac00 msgr2=0x7f4ba006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bba638640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ba006ac00 0x7f4ba006d0c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40ccd20 tx=0x7f4ba8008040 comp rx=0 tx=0).stop 2024-03-23T09:46:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bba638640 1 -- 172.21.15.47:0/2558576270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb40b8350 msgr2=0x7f4bb4160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bba638640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb40b8350 0x7f4bb4160280 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4034040 tx=0x7f4ba40046c0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bba638640 1 -- 172.21.15.47:0/2558576270 shutdown_connections 2024-03-23T09:46:52.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bba638640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ba006ac00 0x7f4ba006d0c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bba638640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb415c990 0x7f4bb40c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bba638640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bb40cdc40 0x7f4bb415aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bba638640 1 --2- 172.21.15.47:0/2558576270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb40b8350 0x7f4bb4160280 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:52.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bba638640 1 -- 172.21.15.47:0/2558576270 >> 172.21.15.47:0/2558576270 conn(0x7f4bb40659d0 msgr2=0x7f4bb40b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:52.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bba638640 1 -- 172.21.15.47:0/2558576270 shutdown_connections 2024-03-23T09:46:52.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:52.973+0000 7f4bba638640 1 -- 172.21.15.47:0/2558576270 wait complete. 2024-03-23T09:46:52.985 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-23T09:46:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.084+0000 7f8674661640 1 Processor -- start 2024-03-23T09:46:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.084+0000 7f8674661640 1 -- start start 2024-03-23T09:46:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.084+0000 7f8674661640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f866c15e110 0x7f866c0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.084+0000 7f8674661640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f866c0cee40 0x7f866c0c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.084+0000 7f8674661640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866c0cf390 con 0x7f866c098a00 2024-03-23T09:46:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.084+0000 7f8674661640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866c0cf670 con 0x7f866c0cee40 2024-03-23T09:46:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.084+0000 7f8674661640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866c0cb4b0 con 0x7f866c15e110 2024-03-23T09:46:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.085+0000 7f8671bfc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f866c0cee40 0x7f866c0c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.085+0000 7f8671bfc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f866c0cee40 0x7f866c0c41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36218/0 (socket says 172.21.15.47:36218) 2024-03-23T09:46:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.085+0000 7f8671bfc640 1 -- 172.21.15.47:0/341517283 learned_addr learned my addr 172.21.15.47:0/341517283 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.085+0000 7f8671bfc640 1 -- 172.21.15.47:0/341517283 >> v2:172.21.15.175:3300/0 conn(0x7f866c15e110 msgr2=0x7f866c0b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.085+0000 7f8671bfc640 1 --2- 172.21.15.47:0/341517283 >> v2:172.21.15.175:3300/0 conn(0x7f866c15e110 0x7f866c0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.085+0000 7f8671bfc640 1 -- 172.21.15.47:0/341517283 >> v1:172.21.15.175:6789/0 conn(0x7f866c098a00 legacy=0x7f866c0cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.085+0000 7f8671bfc640 1 -- 172.21.15.47:0/341517283 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0cb790 con 0x7f866c0cee40 2024-03-23T09:46:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.086+0000 7f8671bfc640 1 --2- 172.21.15.47:0/341517283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f866c0cee40 0x7f866c0c41a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f866c09e5b0 tx=0x7f866002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9567cd8bce327344 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.086+0000 7f86713fb640 1 -- 172.21.15.47:0/341517283 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660031870 con 0x7f866c0cee40 2024-03-23T09:46:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.086+0000 7f86713fb640 1 -- 172.21.15.47:0/341517283 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86600319d0 con 0x7f866c0cee40 2024-03-23T09:46:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.086+0000 7f86713fb640 1 -- 172.21.15.47:0/341517283 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660031d60 con 0x7f866c0cee40 2024-03-23T09:46:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.086+0000 7f8674661640 1 -- 172.21.15.47:0/341517283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f866c0cee40 msgr2=0x7f866c0c41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.086+0000 7f8674661640 1 --2- 172.21.15.47:0/341517283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f866c0cee40 0x7f866c0c41a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f866c09e5b0 tx=0x7f866002b910 comp rx=0 tx=0).stop 2024-03-23T09:46:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.086+0000 7f8674661640 1 -- 172.21.15.47:0/341517283 shutdown_connections 2024-03-23T09:46:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.086+0000 7f8674661640 1 --2- 172.21.15.47:0/341517283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f866c0cee40 0x7f866c0c41a0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.086+0000 7f8674661640 1 --2- 172.21.15.47:0/341517283 >> v2:172.21.15.175:3300/0 conn(0x7f866c15e110 0x7f866c0b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.087+0000 7f8674661640 1 -- 172.21.15.47:0/341517283 >> 172.21.15.47:0/341517283 conn(0x7f866c0b2140 msgr2=0x7f866c0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.087+0000 7f8674661640 1 -- 172.21.15.47:0/341517283 shutdown_connections 2024-03-23T09:46:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.087+0000 7f8674661640 1 -- 172.21.15.47:0/341517283 wait complete. 2024-03-23T09:46:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.088+0000 7f8674661640 1 Processor -- start 2024-03-23T09:46:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.088+0000 7f8674661640 1 -- start start 2024-03-23T09:46:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.088+0000 7f8674661640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f866c098a00 0x7f866c158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.088+0000 7f8674661640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f866c0cee40 0x7f866c09bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.088+0000 7f8674661640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f866c15e110 0x7f866c0c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.088+0000 7f8674661640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0b41c0 con 0x7f866c0cee40 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.088+0000 7f8674661640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0b4d00 con 0x7f866c15e110 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.088+0000 7f8674661640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0b43d0 con 0x7f866c098a00 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f8671bfc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f866c0cee40 0x7f866c09bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f8671bfc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f866c0cee40 0x7f866c09bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36230/0 (socket says 172.21.15.47:36230) 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f8671bfc640 1 -- 172.21.15.47:0/167316549 learned_addr learned my addr 172.21.15.47:0/167316549 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f86723fd640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f866c098a00 0x7f866c158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f8672bfe640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f866c15e110 0x7f866c0c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f8671bfc640 1 -- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f866c098a00 msgr2=0x7f866c158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f8671bfc640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f866c098a00 0x7f866c158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f8671bfc640 1 -- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f866c15e110 msgr2=0x7f866c0c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f8671bfc640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f866c15e110 0x7f866c0c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f8671bfc640 1 -- 172.21.15.47:0/167316549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86600094d0 con 0x7f866c0cee40 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f8671bfc640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f866c0cee40 0x7f866c09bbf0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8660000c00 tx=0x7f86600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f866affd640 1 -- 172.21.15.47:0/167316549 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660031840 con 0x7f866c0cee40 2024-03-23T09:46:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f866affd640 1 -- 172.21.15.47:0/167316549 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86600443f0 con 0x7f866c0cee40 2024-03-23T09:46:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f8674661640 1 -- 172.21.15.47:0/167316549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866c0b46b0 con 0x7f866c0cee40 2024-03-23T09:46:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f866affd640 1 -- 172.21.15.47:0/167316549 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86600447a0 con 0x7f866c0cee40 2024-03-23T09:46:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.089+0000 7f8674661640 1 -- 172.21.15.47:0/167316549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c159240 con 0x7f866c0cee40 2024-03-23T09:46:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.090+0000 7f8674661640 1 -- 172.21.15.47:0/167316549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866c0b4e90 con 0x7f866c0cee40 2024-03-23T09:46:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.091+0000 7f866affd640 1 -- 172.21.15.47:0/167316549 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f866004d450 con 0x7f866c0cee40 2024-03-23T09:46:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.091+0000 7f866affd640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f865806ac30 0x7f865806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.091+0000 7f866affd640 1 -- 172.21.15.47:0/167316549 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f866002d040 con 0x7f866c0cee40 2024-03-23T09:46:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.091+0000 7f86723fd640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f865806ac30 0x7f865806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.092+0000 7f86723fd640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f865806ac30 0x7f865806d0f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f866c09e510 tx=0x7f8664008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.096+0000 7f866affd640 1 -- 172.21.15.47:0/167316549 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8660076d20 con 0x7f866c0cee40 2024-03-23T09:46:53.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.315+0000 7f8674661640 1 -- 172.21.15.47:0/167316549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f866c066be0 con 0x7f866c0cee40 2024-03-23T09:46:53.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.316+0000 7f866affd640 1 -- 172.21.15.47:0/167316549 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f8660035080 con 0x7f866c0cee40 2024-03-23T09:46:53.317 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:46:53.317 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-23T09:46:53.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.318+0000 7f8674661640 1 -- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f865806ac30 msgr2=0x7f865806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:53.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.318+0000 7f8674661640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f865806ac30 0x7f865806d0f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f866c09e510 tx=0x7f8664008040 comp rx=0 tx=0).stop 2024-03-23T09:46:53.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.318+0000 7f8674661640 1 -- 172.21.15.47:0/167316549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f866c0cee40 msgr2=0x7f866c09bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:53.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.318+0000 7f8674661640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f866c0cee40 0x7f866c09bbf0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8660000c00 tx=0x7f86600046c0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.318+0000 7f8674661640 1 -- 172.21.15.47:0/167316549 shutdown_connections 2024-03-23T09:46:53.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.318+0000 7f8674661640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f865806ac30 0x7f865806d0f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.318+0000 7f8674661640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f866c15e110 0x7f866c0c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.318+0000 7f8674661640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f866c0cee40 0x7f866c09bbf0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.318+0000 7f8674661640 1 --2- 172.21.15.47:0/167316549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f866c098a00 0x7f866c158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.318+0000 7f8674661640 1 -- 172.21.15.47:0/167316549 >> 172.21.15.47:0/167316549 conn(0x7f866c0b2140 msgr2=0x7f866c15bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:53.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.318+0000 7f8674661640 1 -- 172.21.15.47:0/167316549 shutdown_connections 2024-03-23T09:46:53.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.318+0000 7f8674661640 1 -- 172.21.15.47:0/167316549 wait complete. 2024-03-23T09:46:53.329 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-23T09:46:53.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:46:53.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.429+0000 7fa29ad63640 1 Processor -- start 2024-03-23T09:46:53.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.429+0000 7fa29ad63640 1 -- start start 2024-03-23T09:46:53.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.430+0000 7fa29ad63640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa29415cc40 0x7fa294160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.430+0000 7fa29ad63640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa294160c90 0x7fa2940cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.430+0000 7fa29ad63640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2940d09d0 con 0x7fa2940cdbf0 2024-03-23T09:46:53.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.430+0000 7fa29ad63640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2940ce0a0 con 0x7fa29415cc40 2024-03-23T09:46:53.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.430+0000 7fa29ad63640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940b5a80 con 0x7fa294160c90 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.430+0000 7fa299300640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa2940cdbf0 0x7fa2940b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:49896/0 (socket says 172.21.15.47:49896) 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.430+0000 7fa299300640 1 -- 172.21.15.47:0/39518755 learned_addr learned my addr 172.21.15.47:0/39518755 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.430+0000 7fa293fff640 1 --2- 172.21.15.47:0/39518755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa294160c90 0x7fa2940cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.430+0000 7fa298aff640 1 --2- 172.21.15.47:0/39518755 >> v2:172.21.15.175:3300/0 conn(0x7fa29415cc40 0x7fa294160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.430+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235012268 0 0) 0x7fa2940d09d0 con 0x7fa2940cdbf0 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.430+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa280003520 con 0x7fa2940cdbf0 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1810854344 0 0) 0x7fa280003520 con 0x7fa2940cdbf0 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa294160c90 msgr2=0x7fa2940cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 --2- 172.21.15.47:0/39518755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa294160c90 0x7fa2940cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 >> v2:172.21.15.175:3300/0 conn(0x7fa29415cc40 msgr2=0x7fa294160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 --2- 172.21.15.47:0/39518755 >> v2:172.21.15.175:3300/0 conn(0x7fa29415cc40 0x7fa294160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29415d020 con 0x7fa2940cdbf0 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fa27c001670 con 0x7fa2940cdbf0 2024-03-23T09:46:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 >> v1:172.21.15.47:6789/0 conn(0x7fa2940cdbf0 legacy=0x7fa2940b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 --2- 172.21.15.47:0/39518755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa280003970 0x7fa280003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 --2- 172.21.15.47:0/39518755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2800047c0 0x7fa280004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 --2- 172.21.15.47:0/39518755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa294160c90 0x7fa280005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940ce0a0 con 0x7fa280003970 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940b5a80 con 0x7fa2800047c0 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28000b980 con 0x7fa294160c90 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa298aff640 1 --2- 172.21.15.47:0/39518755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa280003970 0x7fa280003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa293fff640 1 --2- 172.21.15.47:0/39518755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2800047c0 0x7fa280004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa299300640 1 --2- 172.21.15.47:0/39518755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa294160c90 0x7fa280005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa298aff640 1 -- 172.21.15.47:0/39518755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa294160c90 msgr2=0x7fa280005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa298aff640 1 --2- 172.21.15.47:0/39518755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa294160c90 0x7fa280005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa298aff640 1 -- 172.21.15.47:0/39518755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2800047c0 msgr2=0x7fa280004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa298aff640 1 --2- 172.21.15.47:0/39518755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2800047c0 0x7fa280004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa298aff640 1 -- 172.21.15.47:0/39518755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa280003520 con 0x7fa280003970 2024-03-23T09:46:53.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.431+0000 7fa299300640 1 --2- 172.21.15.47:0/39518755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa294160c90 0x7fa280005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:53.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.432+0000 7fa298aff640 1 --2- 172.21.15.47:0/39518755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa280003970 0x7fa280003d90 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009ac0 tx=0x7fa2800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7fcc488e7b89e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:53.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.432+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c02c710 con 0x7fa280003970 2024-03-23T09:46:53.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.432+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28c02c870 con 0x7fa280003970 2024-03-23T09:46:53.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.432+0000 7fa2937fe640 1 -- 172.21.15.47:0/39518755 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c02cc00 con 0x7fa280003970 2024-03-23T09:46:53.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.433+0000 7fa29ad63640 1 -- 172.21.15.47:0/39518755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa280003970 msgr2=0x7fa280003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:53.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.433+0000 7fa29ad63640 1 --2- 172.21.15.47:0/39518755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa280003970 0x7fa280003d90 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009ac0 tx=0x7fa2800057f0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.433+0000 7fa299300640 1 -- 172.21.15.47:0/39518755 reap_dead start 2024-03-23T09:46:53.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.433+0000 7fa29ad63640 1 -- 172.21.15.47:0/39518755 shutdown_connections 2024-03-23T09:46:53.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.433+0000 7fa29ad63640 1 -- 172.21.15.47:0/39518755 >> 172.21.15.47:0/39518755 conn(0x7fa29406b160 msgr2=0x7fa29406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:53.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.433+0000 7fa29ad63640 1 -- 172.21.15.47:0/39518755 shutdown_connections 2024-03-23T09:46:53.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.433+0000 7fa29ad63640 1 -- 172.21.15.47:0/39518755 wait complete. 2024-03-23T09:46:53.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.434+0000 7fa29ad63640 1 Processor -- start 2024-03-23T09:46:53.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.434+0000 7fa29ad63640 1 -- start start 2024-03-23T09:46:53.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa29ad63640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa280003970 0x7fa294153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa29ad63640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2941f1fc0 0x7fa29415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa29ad63640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa29415acb0 0x7fa29415e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa29ad63640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa294158a20 con 0x7fa280003970 2024-03-23T09:46:53.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa29ad63640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29415df90 con 0x7fa2941f1fc0 2024-03-23T09:46:53.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa29ad63640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29415e270 con 0x7fa29415acb0 2024-03-23T09:46:53.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa298aff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa280003970 0x7fa294153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:53.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa298aff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa280003970 0x7fa294153fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36242/0 (socket says 172.21.15.47:36242) 2024-03-23T09:46:53.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa298aff640 1 -- 172.21.15.47:0/55935661 learned_addr learned my addr 172.21.15.47:0/55935661 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:53.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa298aff640 1 -- 172.21.15.47:0/55935661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa29415acb0 msgr2=0x7fa29415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa298aff640 1 --2- 172.21.15.47:0/55935661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa29415acb0 0x7fa29415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa298aff640 1 -- 172.21.15.47:0/55935661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2941f1fc0 msgr2=0x7fa29415a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa298aff640 1 --2- 172.21.15.47:0/55935661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2941f1fc0 0x7fa29415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa298aff640 1 -- 172.21.15.47:0/55935661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa294158c00 con 0x7fa280003970 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa298aff640 1 --2- 172.21.15.47:0/55935661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa280003970 0x7fa294153fb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa28c000c00 tx=0x7fa28c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa2917fa640 1 -- 172.21.15.47:0/55935661 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c003810 con 0x7fa280003970 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa2917fa640 1 -- 172.21.15.47:0/55935661 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28c02de90 con 0x7fa280003970 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa29ad63640 1 -- 172.21.15.47:0/55935661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa294158ee0 con 0x7fa280003970 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.435+0000 7fa2917fa640 1 -- 172.21.15.47:0/55935661 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c003bb0 con 0x7fa280003970 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.436+0000 7fa29ad63640 1 -- 172.21.15.47:0/55935661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29415dc60 con 0x7fa280003970 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.437+0000 7fa29ad63640 1 -- 172.21.15.47:0/55935661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa294072360 con 0x7fa280003970 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.437+0000 7fa2917fa640 1 -- 172.21.15.47:0/55935661 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa28c003d10 con 0x7fa280003970 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.437+0000 7fa2917fa640 1 --2- 172.21.15.47:0/55935661 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa27406ac00 0x7fa27406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.437+0000 7fa2917fa640 1 -- 172.21.15.47:0/55935661 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa28c0ab220 con 0x7fa280003970 2024-03-23T09:46:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.438+0000 7fa293fff640 1 --2- 172.21.15.47:0/55935661 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa27406ac00 0x7fa27406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:53.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.438+0000 7fa293fff640 1 --2- 172.21.15.47:0/55935661 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa27406ac00 0x7fa27406d0c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa2941f23a0 tx=0x7fa284009750 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:53.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.442+0000 7fa2917fa640 1 -- 172.21.15.47:0/55935661 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa28c076370 con 0x7fa280003970 2024-03-23T09:46:53.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.655+0000 7fa29ad63640 1 -- 172.21.15.47:0/55935661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa294066be0 con 0x7fa280003970 2024-03-23T09:46:53.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.656+0000 7fa2917fa640 1 -- 172.21.15.47:0/55935661 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa28c079990 con 0x7fa280003970 2024-03-23T09:46:53.657 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:46:53.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.657+0000 7fa29ad63640 1 -- 172.21.15.47:0/55935661 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa27406ac00 msgr2=0x7fa27406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:53.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.657+0000 7fa29ad63640 1 --2- 172.21.15.47:0/55935661 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa27406ac00 0x7fa27406d0c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa2941f23a0 tx=0x7fa284009750 comp rx=0 tx=0).stop 2024-03-23T09:46:53.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.658+0000 7fa29ad63640 1 -- 172.21.15.47:0/55935661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa280003970 msgr2=0x7fa294153fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:53.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.658+0000 7fa29ad63640 1 --2- 172.21.15.47:0/55935661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa280003970 0x7fa294153fb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa28c000c00 tx=0x7fa28c004730 comp rx=0 tx=0).stop 2024-03-23T09:46:53.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.658+0000 7fa29ad63640 1 -- 172.21.15.47:0/55935661 shutdown_connections 2024-03-23T09:46:53.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.658+0000 7fa29ad63640 1 --2- 172.21.15.47:0/55935661 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa27406ac00 0x7fa27406d0c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.658+0000 7fa29ad63640 1 --2- 172.21.15.47:0/55935661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa29415acb0 0x7fa29415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.658+0000 7fa29ad63640 1 --2- 172.21.15.47:0/55935661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2941f1fc0 0x7fa29415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.658+0000 7fa29ad63640 1 --2- 172.21.15.47:0/55935661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa280003970 0x7fa294153fb0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:53.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.658+0000 7fa29ad63640 1 -- 172.21.15.47:0/55935661 >> 172.21.15.47:0/55935661 conn(0x7fa29406b160 msgr2=0x7fa2940b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:53.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.658+0000 7fa29ad63640 1 -- 172.21.15.47:0/55935661 shutdown_connections 2024-03-23T09:46:53.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:53.658+0000 7fa29ad63640 1 -- 172.21.15.47:0/55935661 wait complete. 2024-03-23T09:46:53.669 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:46:53.669 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:46:53.669 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:46:53.669 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:46:53.669 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:46:53.669 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:46:53.670 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:46:53.670 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:46:53.670 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:46:53.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:53.670 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:46:53.685 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:46:53.685 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:46:53.685 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:46:53.685 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:46:53.686 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:46:53.686 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:46:53.686 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:46:53.686 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:46:53.686 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:46:53.686 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:46:53.686 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:46:53.686 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:46:53.686 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:46:53.686 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:46:53.686 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:46:53.686 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:46:53.687 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42325sec preferred_lft 42325sec 2024-03-23T09:46:53.687 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:46:53.687 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:46:53.687 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:46:53.687 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:46:53.687 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:46:53.687 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:46:53.687 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:46:53.687 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:46:53.687 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:46:53.687 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:46:53.687 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:46:53.687 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:46:53.688 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:46:53.688 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:46:53.688 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:46:53.688 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:46:53.688 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:46:53.688 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:46:53.688 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:46:53.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:53.688 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:46:53.751 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:46:53.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:53.753 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:46:53.807 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:46:53.808 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:46:53.808 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:46:53.808 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:46:53.808 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:46:53.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:53.808 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:46:53.808 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:46:53.808 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:46:53.808 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:46:53.808 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:46:53.809 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:46:53.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:53.903 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:46:53.917 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:46:53.917 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:46:53.917 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:46:53.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:53.918 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:46:53.985 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:46:53.985 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:46:53.985 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:46:54.039 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:46:54.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:46:54.040 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:46:54.094 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:46:54.162 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:46:54.162 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:46:54.227 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:46:54.228 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:46:54.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:54.229 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:46:54.283 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:46:54.283 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:46:54.326 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:46:54.352 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:46:54.352 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:46:54.410 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:46:54.409+0000 7fd31ea4b480 -1 init, newargv = 0x55d413286ed0 newargc=15 2024-03-23T09:46:54.411 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[64597]: starting ceph client 2024-03-23T09:46:54.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[64597]: starting fuse 2024-03-23T09:46:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:46:54.422 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:46:54.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:54.423 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:46:54.477 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:46:54.477 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:46:54.477 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:46:54.478 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:46:54.479 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:46:54.479 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:46:54.479 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:46:54.479 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:46:54.479 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:46:54.479 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:46:54.479 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:46:54.479 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:46:54.479 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:46:54.479 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:46:54.479 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:46:54.593 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.64597.asok 2024-03-23T09:46:54.779 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.64597.asok 2024-03-23T09:46:54.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:54.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.64597.asok status 2024-03-23T09:46:54.916 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:46:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:46:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:46:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:46:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:46:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:46:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:46:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "64597", 2024-03-23T09:46:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:46:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:46:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:46:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 7332, 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7332 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1373845669 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:46:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:46:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:46:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1373845669 2024-03-23T09:46:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:46:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.7332 192.168.0.1:0/1373845669", 2024-03-23T09:46:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/1373845669", 2024-03-23T09:46:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:46:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 40, 2024-03-23T09:46:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 73, 2024-03-23T09:46:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:46:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:46:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:46:54.919 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:46:54.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:54.925 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:46:54.925 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:46:54.941 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:46:54.941 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:46:54.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:54.941 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:46:55.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:55.008 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-03-23T09:46:55.064 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:46:55.065 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-03-23T09:46:55.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.146+0000 7fd0efc3af40 1 Processor -- start 2024-03-23T09:46:55.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.146+0000 7fd0efc3af40 1 -- start start 2024-03-23T09:46:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.146+0000 7fd0efc3af40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641930e0a00 0x5641931b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.146+0000 7fd0efc3af40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564193198910 0x5641931b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.146+0000 7fd0efc3af40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56419319b6d0 0x5641931b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.146+0000 7fd0efc3af40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564193026b60 con 0x5641930e0a00 2024-03-23T09:46:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.146+0000 7fd0efc3af40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564193026800 con 0x564193198910 2024-03-23T09:46:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.146+0000 7fd0efc3af40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x564193009520 con 0x56419319b6d0 2024-03-23T09:46:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.146+0000 7fd0ecfdd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641930e0a00 0x5641931b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.146+0000 7fd0ecfdd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641930e0a00 0x5641931b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43282/0 (socket says 172.21.15.47:43282) 2024-03-23T09:46:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.146+0000 7fd0ecfdd640 1 -- 172.21.15.47:0/4006812873 learned_addr learned my addr 172.21.15.47:0/4006812873 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.147+0000 7fd0ec7dc640 1 --2- 172.21.15.47:0/4006812873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564193198910 0x5641931b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.147+0000 7fd0ed7de640 1 --2- 172.21.15.47:0/4006812873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56419319b6d0 0x5641931b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.147+0000 7fd0ecfdd640 1 -- 172.21.15.47:0/4006812873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56419319b6d0 msgr2=0x5641931b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.147+0000 7fd0ecfdd640 1 --2- 172.21.15.47:0/4006812873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56419319b6d0 0x5641931b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.147+0000 7fd0ecfdd640 1 -- 172.21.15.47:0/4006812873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564193198910 msgr2=0x5641931b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.147+0000 7fd0ecfdd640 1 --2- 172.21.15.47:0/4006812873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564193198910 0x5641931b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.147+0000 7fd0ecfdd640 1 -- 172.21.15.47:0/4006812873 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc00c3f0 con 0x5641930e0a00 2024-03-23T09:46:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.147+0000 7fd0ecfdd640 1 --2- 172.21.15.47:0/4006812873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641930e0a00 0x5641931b2750 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400a5e0 tx=0x7fd0e4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b8095e7ffeb8c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.147+0000 7fd0eb7da640 1 -- 172.21.15.47:0/4006812873 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4008a40 con 0x5641930e0a00 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.147+0000 7fd0eb7da640 1 -- 172.21.15.47:0/4006812873 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e4008bc0 con 0x5641930e0a00 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.147+0000 7fd0eb7da640 1 -- 172.21.15.47:0/4006812873 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4013440 con 0x5641930e0a00 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.148+0000 7fd0efc3af40 1 -- 172.21.15.47:0/4006812873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641930e0a00 msgr2=0x5641931b2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.148+0000 7fd0efc3af40 1 --2- 172.21.15.47:0/4006812873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641930e0a00 0x5641931b2750 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400a5e0 tx=0x7fd0e4005f90 comp rx=0 tx=0).stop 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.148+0000 7fd0efc3af40 1 -- 172.21.15.47:0/4006812873 shutdown_connections 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.148+0000 7fd0efc3af40 1 --2- 172.21.15.47:0/4006812873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56419319b6d0 0x5641931b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.148+0000 7fd0efc3af40 1 --2- 172.21.15.47:0/4006812873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564193198910 0x5641931b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.148+0000 7fd0efc3af40 1 --2- 172.21.15.47:0/4006812873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641930e0a00 0x5641931b2750 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.148+0000 7fd0efc3af40 1 -- 172.21.15.47:0/4006812873 >> 172.21.15.47:0/4006812873 conn(0x564193109e40 msgr2=0x564193127410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.148+0000 7fd0efc3af40 1 -- 172.21.15.47:0/4006812873 shutdown_connections 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.148+0000 7fd0efc3af40 1 -- 172.21.15.47:0/4006812873 wait complete. 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.148+0000 7fd0efc3af40 1 Processor -- start 2024-03-23T09:46:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.148+0000 7fd0efc3af40 1 -- start start 2024-03-23T09:46:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0efc3af40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5641930e0a00 0x564193127cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0efc3af40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564193198910 0x564193191dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0efc3af40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56419319b6d0 0x564193249d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0efc3af40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564193195c00 con 0x564193198910 2024-03-23T09:46:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0efc3af40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641931b3df0 con 0x56419319b6d0 2024-03-23T09:46:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0efc3af40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x564193128200 con 0x5641930e0a00 2024-03-23T09:46:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0ec7dc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564193198910 0x564193191dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0ec7dc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564193198910 0x564193191dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43298/0 (socket says 172.21.15.47:43298) 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0ec7dc640 1 -- 172.21.15.47:0/1168858008 learned_addr learned my addr 172.21.15.47:0/1168858008 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0ecfdd640 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5641930e0a00 0x564193127cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0ed7de640 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56419319b6d0 0x564193249d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0ec7dc640 1 -- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5641930e0a00 msgr2=0x564193127cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0ec7dc640 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5641930e0a00 0x564193127cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0ec7dc640 1 -- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56419319b6d0 msgr2=0x564193249d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0ec7dc640 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56419319b6d0 0x564193249d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0ec7dc640 1 -- 172.21.15.47:0/1168858008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e4005d70 con 0x564193198910 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0ecfdd640 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5641930e0a00 0x564193127cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0ed7de640 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56419319b6d0 0x564193249d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.149+0000 7fd0ec7dc640 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564193198910 0x564193191dc0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400b6e0 tx=0x7fd0d400bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.150+0000 7fd0ea7d8640 1 -- 172.21.15.47:0/1168858008 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4005c70 con 0x564193198910 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.150+0000 7fd0ea7d8640 1 -- 172.21.15.47:0/1168858008 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d400ab40 con 0x564193198910 2024-03-23T09:46:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.150+0000 7fd0ea7d8640 1 -- 172.21.15.47:0/1168858008 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4016440 con 0x564193198910 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.150+0000 7fd0efc3af40 1 -- 172.21.15.47:0/1168858008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564193035c20 con 0x564193198910 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.150+0000 7fd0efc3af40 1 -- 172.21.15.47:0/1168858008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564193029060 con 0x564193198910 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.151+0000 7fd0ea7d8640 1 -- 172.21.15.47:0/1168858008 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd0d40165a0 con 0x564193198910 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.151+0000 7fd0ea7d8640 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd0cc0695e0 0x7fd0cc06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.151+0000 7fd0ea7d8640 1 -- 172.21.15.47:0/1168858008 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd0d4088b00 con 0x564193198910 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.151+0000 7fd0ecfdd640 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd0cc0695e0 0x7fd0cc06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.151+0000 7fd0efc3af40 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x56419324a2d0 0x5641930dcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.152+0000 7fd0efc3af40 1 -- 172.21.15.47:0/1168858008 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73) v8 -- 0x5641931938a0 con 0x56419324a2d0 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.152+0000 7fd0ed7de640 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x56419324a2d0 0x5641930dcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.152+0000 7fd0ecfdd640 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd0cc0695e0 0x7fd0cc06baa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4000c00 tx=0x7fd0e4015040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.152+0000 7fd0ed7de640 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x56419324a2d0 0x5641930dcbb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.153+0000 7fd0ed7de640 1 -- 172.21.15.47:0/1168858008 <== osd.4 v2:172.21.15.175:6800/38159 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v73'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd0dc052070 con 0x56419324a2d0 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.154+0000 7fd0efc3af40 1 -- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x56419324a2d0 msgr2=0x5641930dcbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.154+0000 7fd0efc3af40 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x56419324a2d0 0x5641930dcbb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.154+0000 7fd0efc3af40 1 -- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd0cc0695e0 msgr2=0x7fd0cc06baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.154+0000 7fd0efc3af40 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd0cc0695e0 0x7fd0cc06baa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4000c00 tx=0x7fd0e4015040 comp rx=0 tx=0).stop 2024-03-23T09:46:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.154+0000 7fd0efc3af40 1 -- 172.21.15.47:0/1168858008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564193198910 msgr2=0x564193191dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.154+0000 7fd0efc3af40 1 --2- 172.21.15.47:0/1168858008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564193198910 0x564193191dc0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400b6e0 tx=0x7fd0d400bba0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.154+0000 7fd0ed7de640 1 -- 172.21.15.47:0/1168858008 reap_dead start 2024-03-23T09:46:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.154+0000 7fd0efc3af40 1 -- 172.21.15.47:0/1168858008 shutdown_connections 2024-03-23T09:46:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.154+0000 7fd0efc3af40 1 -- 172.21.15.47:0/1168858008 >> 172.21.15.47:0/1168858008 conn(0x564193109e40 msgr2=0x5641931b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.154+0000 7fd0efc3af40 1 -- 172.21.15.47:0/1168858008 shutdown_connections 2024-03-23T09:46:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.154+0000 7fd0efc3af40 1 -- 172.21.15.47:0/1168858008 wait complete. 2024-03-23T09:46:55.159 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:46:55.159 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:46:55.239 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:46:55.285 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:46:55.285 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:46:55.285 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:46:55.300 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:46:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.440+0000 7f3f036af640 1 Processor -- start 2024-03-23T09:46:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.440+0000 7f3f036af640 1 -- start start 2024-03-23T09:46:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.441+0000 7f3f036af640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3efc15cb50 0x7f3efc0b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.441+0000 7f3f036af640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc0b72f0 con 0x7f3efc0cd5a0 2024-03-23T09:46:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.441+0000 7f3f036af640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc0b75d0 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.441+0000 7f3f036af640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3efc0b7e70 con 0x7f3efc15cb50 2024-03-23T09:46:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.441+0000 7f3f0144b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3efc15cb50 0x7f3efc0b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.441+0000 7f3f00c4a640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3efc0cd5a0 0x7f3efc0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52806/0 (socket says 172.21.15.47:52806) 2024-03-23T09:46:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.441+0000 7f3f00c4a640 1 -- 172.21.15.47:0/1490640012 learned_addr learned my addr 172.21.15.47:0/1490640012 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.441+0000 7f3f0144b640 1 -- 172.21.15.47:0/1490640012 >> v1:172.21.15.175:6789/0 conn(0x7f3efc0cd9e0 legacy=0x7f3efc0b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.441+0000 7f3f0144b640 1 -- 172.21.15.47:0/1490640012 >> v1:172.21.15.47:6789/0 conn(0x7f3efc0cd5a0 legacy=0x7f3efc0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.441+0000 7f3f0144b640 1 -- 172.21.15.47:0/1490640012 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc0b78b0 con 0x7f3efc15cb50 2024-03-23T09:46:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.441+0000 7f3ef3fff640 1 -- 172.21.15.47:0/1490640012 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 219923135 0 0) 0x7f3efc0b72f0 con 0x7f3efc0cd5a0 2024-03-23T09:46:55.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.442+0000 7f3f0144b640 1 --2- 172.21.15.47:0/1490640012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc15cb50 0x7f3efc0b84a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80097a0 tx=0x7f3ef802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3a234a3edfce37 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:55.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.442+0000 7f3ef3fff640 1 -- 172.21.15.47:0/1490640012 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8030610 con 0x7f3efc15cb50 2024-03-23T09:46:55.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.442+0000 7f3ef3fff640 1 -- 172.21.15.47:0/1490640012 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef8030770 con 0x7f3efc15cb50 2024-03-23T09:46:55.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.442+0000 7f3ef3fff640 1 -- 172.21.15.47:0/1490640012 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8030b00 con 0x7f3efc15cb50 2024-03-23T09:46:55.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.443+0000 7f3f036af640 1 -- 172.21.15.47:0/1490640012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc15cb50 msgr2=0x7f3efc0b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.443+0000 7f3f036af640 1 --2- 172.21.15.47:0/1490640012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc15cb50 0x7f3efc0b84a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80097a0 tx=0x7f3ef802b960 comp rx=0 tx=0).stop 2024-03-23T09:46:55.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.443+0000 7f3f036af640 1 -- 172.21.15.47:0/1490640012 shutdown_connections 2024-03-23T09:46:55.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.443+0000 7f3f036af640 1 --2- 172.21.15.47:0/1490640012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc15cb50 0x7f3efc0b84a0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.443+0000 7f3f036af640 1 -- 172.21.15.47:0/1490640012 >> 172.21.15.47:0/1490640012 conn(0x7f3efc065df0 msgr2=0x7f3efc066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:55.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.443+0000 7f3f036af640 1 -- 172.21.15.47:0/1490640012 shutdown_connections 2024-03-23T09:46:55.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.443+0000 7f3f036af640 1 -- 172.21.15.47:0/1490640012 wait complete. 2024-03-23T09:46:55.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.444+0000 7f3f036af640 1 Processor -- start 2024-03-23T09:46:55.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.444+0000 7f3f036af640 1 -- start start 2024-03-23T09:46:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f036af640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc0cd5a0 0x7f3efc0b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f036af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc0cd9e0 0x7f3efc1f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f036af640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc15cb50 0x7f3efc1f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f036af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc14cb40 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f036af640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc14ce20 con 0x7f3efc15cb50 2024-03-23T09:46:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f036af640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc14d100 con 0x7f3efc0cd5a0 2024-03-23T09:46:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f00c4a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc0cd9e0 0x7f3efc1f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f00c4a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc0cd9e0 0x7f3efc1f5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43326/0 (socket says 172.21.15.47:43326) 2024-03-23T09:46:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f00c4a640 1 -- 172.21.15.47:0/3954828228 learned_addr learned my addr 172.21.15.47:0/3954828228 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f00c4a640 1 -- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc0cd5a0 msgr2=0x7f3efc0b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f00c4a640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc0cd5a0 0x7f3efc0b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f00c4a640 1 -- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc15cb50 msgr2=0x7f3efc1f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f01c4c640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc15cb50 0x7f3efc1f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f00c4a640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc15cb50 0x7f3efc1f5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f00c4a640 1 -- 172.21.15.47:0/3954828228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef8009400 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f01c4c640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc15cb50 0x7f3efc1f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3f00c4a640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc0cd9e0 0x7f3efc1f5570 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c4d0 tx=0x7f3eec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.445+0000 7f3ef1ffb640 1 -- 172.21.15.47:0/3954828228 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0074e0 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.446+0000 7f3ef1ffb640 1 -- 172.21.15.47:0/3954828228 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec007660 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.446+0000 7f3ef1ffb640 1 -- 172.21.15.47:0/3954828228 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec004a20 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.446+0000 7f3f036af640 1 -- 172.21.15.47:0/3954828228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3efc14d3e0 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.446+0000 7f3f036af640 1 -- 172.21.15.47:0/3954828228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3efc1f62e0 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.447+0000 7f3f036af640 1 -- 172.21.15.47:0/3954828228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3efc072360 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.447+0000 7f3ef1ffb640 1 -- 172.21.15.47:0/3954828228 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3eec004b80 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.447+0000 7f3ef1ffb640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ee006ac00 0x7f3ee006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.447+0000 7f3ef1ffb640 1 -- 172.21.15.47:0/3954828228 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f3eec088590 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.448+0000 7f3f0144b640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ee006ac00 0x7f3ee006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.448+0000 7f3f0144b640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ee006ac00 0x7f3ee006d0c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8036000 tx=0x7f3ef8030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:55.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.452+0000 7f3ef1ffb640 1 -- 172.21.15.47:0/3954828228 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3eec0536e0 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.633+0000 7f3f036af640 1 -- 172.21.15.47:0/3954828228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3efc14cc80 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.634+0000 7f3ef1ffb640 1 -- 172.21.15.47:0/3954828228 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16324 (secure 0 0 0) 0x7f3efc14cc80 con 0x7f3efc0cd9e0 2024-03-23T09:46:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:46:55.636 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":73,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:45:52.838915+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:45:16.609941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-23T09:45:17.261503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.3","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/372771687":"2024-03-24T09:45:33.740376+0000","172.21.15.47:6833/372771687":"2024-03-24T09:45:33.740376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:46:55.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.636+0000 7f3f036af640 1 -- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ee006ac00 msgr2=0x7f3ee006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.636+0000 7f3f036af640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ee006ac00 0x7f3ee006d0c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8036000 tx=0x7f3ef8030000 comp rx=0 tx=0).stop 2024-03-23T09:46:55.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.636+0000 7f3f036af640 1 -- 172.21.15.47:0/3954828228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc0cd9e0 msgr2=0x7f3efc1f5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.636+0000 7f3f036af640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc0cd9e0 0x7f3efc1f5570 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c4d0 tx=0x7f3eec00c990 comp rx=0 tx=0).stop 2024-03-23T09:46:55.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.636+0000 7f3f036af640 1 -- 172.21.15.47:0/3954828228 shutdown_connections 2024-03-23T09:46:55.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.636+0000 7f3f036af640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ee006ac00 0x7f3ee006d0c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.636+0000 7f3f036af640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc15cb50 0x7f3efc1f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.636+0000 7f3f036af640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc0cd9e0 0x7f3efc1f5570 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.636+0000 7f3f036af640 1 --2- 172.21.15.47:0/3954828228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc0cd5a0 0x7f3efc0b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.636+0000 7f3f036af640 1 -- 172.21.15.47:0/3954828228 >> 172.21.15.47:0/3954828228 conn(0x7f3efc065df0 msgr2=0x7f3efc15bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:55.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.636+0000 7f3f036af640 1 -- 172.21.15.47:0/3954828228 shutdown_connections 2024-03-23T09:46:55.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.636+0000 7f3f036af640 1 -- 172.21.15.47:0/3954828228 wait complete. 2024-03-23T09:46:55.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:45:16.609941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:45:17.261503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:46:55.648 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:46:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.751+0000 7fe4c73ae640 1 Processor -- start 2024-03-23T09:46:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.752+0000 7fe4c73ae640 1 -- start start 2024-03-23T09:46:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.752+0000 7fe4c73ae640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4c015c990 0x7fe4c0160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.752+0000 7fe4c73ae640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c0161630 con 0x7fe4c00cde10 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.752+0000 7fe4c73ae640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c00d1030 con 0x7fe4c00b8350 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.752+0000 7fe4c73ae640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00cfe10 con 0x7fe4c015c990 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.752+0000 7fe4c594b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe4c00cde10 0x7fe4c00d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52820/0 (socket says 172.21.15.47:52820) 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.752+0000 7fe4c594b640 1 -- 172.21.15.47:0/3713135070 learned_addr learned my addr 172.21.15.47:0/3713135070 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4c514a640 1 --2- 172.21.15.47:0/3713135070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4c015c990 0x7fe4c0160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450788608 0 0) 0x7fe4c0161630 con 0x7fe4c00cde10 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4ac003520 con 0x7fe4c00cde10 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 5249355 0 0) 0x7fe4ac003520 con 0x7fe4c00cde10 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4c015c990 msgr2=0x7fe4c0160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 --2- 172.21.15.47:0/3713135070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4c015c990 0x7fe4c0160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 >> v1:172.21.15.175:6789/0 conn(0x7fe4c00b8350 legacy=0x7fe4c00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c00b5b60 con 0x7fe4c00cde10 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fe4b80019f0 con 0x7fe4c00cde10 2024-03-23T09:46:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 >> v1:172.21.15.47:6789/0 conn(0x7fe4c00cde10 legacy=0x7fe4c00d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 --2- 172.21.15.47:0/3713135070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac0039a0 0x7fe4ac003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 --2- 172.21.15.47:0/3713135070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4c015c990 0x7fe4ac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 --2- 172.21.15.47:0/3713135070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4ac004d00 0x7fe4ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00cfe10 con 0x7fe4ac0039a0 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac00ba30 con 0x7fe4ac004d00 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac00bbf0 con 0x7fe4c015c990 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4c514a640 1 --2- 172.21.15.47:0/3713135070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac0039a0 0x7fe4ac003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4c594b640 1 --2- 172.21.15.47:0/3713135070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4ac004d00 0x7fe4ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.753+0000 7fe4c4949640 1 --2- 172.21.15.47:0/3713135070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4c015c990 0x7fe4ac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.754+0000 7fe4c514a640 1 -- 172.21.15.47:0/3713135070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4c015c990 msgr2=0x7fe4ac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.754+0000 7fe4c514a640 1 --2- 172.21.15.47:0/3713135070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4c015c990 0x7fe4ac0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.754+0000 7fe4c514a640 1 -- 172.21.15.47:0/3713135070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4ac004d00 msgr2=0x7fe4ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.754+0000 7fe4c514a640 1 --2- 172.21.15.47:0/3713135070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4ac004d00 0x7fe4ac005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.754+0000 7fe4c514a640 1 -- 172.21.15.47:0/3713135070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac003520 con 0x7fe4ac0039a0 2024-03-23T09:46:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.754+0000 7fe4c514a640 1 --2- 172.21.15.47:0/3713135070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac0039a0 0x7fe4ac003d80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009ac0 tx=0x7fe4ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b522a04f1431f4f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.755+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc02c6b0 con 0x7fe4ac0039a0 2024-03-23T09:46:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.755+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc02c810 con 0x7fe4ac0039a0 2024-03-23T09:46:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.755+0000 7fe4b7fff640 1 -- 172.21.15.47:0/3713135070 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc02cba0 con 0x7fe4ac0039a0 2024-03-23T09:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.755+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3713135070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac0039a0 msgr2=0x7fe4ac003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.755+0000 7fe4c73ae640 1 --2- 172.21.15.47:0/3713135070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac0039a0 0x7fe4ac003d80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009ac0 tx=0x7fe4ac0057f0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.755+0000 7fe4c594b640 1 -- 172.21.15.47:0/3713135070 reap_dead start 2024-03-23T09:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.755+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3713135070 shutdown_connections 2024-03-23T09:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.755+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3713135070 >> 172.21.15.47:0/3713135070 conn(0x7fe4c00659d0 msgr2=0x7fe4c0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.755+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3713135070 shutdown_connections 2024-03-23T09:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.755+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3713135070 wait complete. 2024-03-23T09:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.756+0000 7fe4c73ae640 1 Processor -- start 2024-03-23T09:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.756+0000 7fe4c73ae640 1 -- start start 2024-03-23T09:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c73ae640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4ac0039a0 0x7fe4c00c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c73ae640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4c01f1d90 0x7fe4c00b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c73ae640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4c01f2210 0x7fe4c00b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c73ae640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00c72e0 con 0x7fe4c01f1d90 2024-03-23T09:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c73ae640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c015f110 con 0x7fe4c01f2210 2024-03-23T09:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c73ae640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c015e500 con 0x7fe4ac0039a0 2024-03-23T09:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c4949640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4c01f1d90 0x7fe4c00b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c4949640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4c01f1d90 0x7fe4c00b6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43332/0 (socket says 172.21.15.47:43332) 2024-03-23T09:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c4949640 1 -- 172.21.15.47:0/3552391696 learned_addr learned my addr 172.21.15.47:0/3552391696 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c514a640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4ac0039a0 0x7fe4c00c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c594b640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4c01f2210 0x7fe4c00b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c4949640 1 -- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4ac0039a0 msgr2=0x7fe4c00c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c4949640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4ac0039a0 0x7fe4c00c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c4949640 1 -- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4c01f2210 msgr2=0x7fe4c00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c4949640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4c01f2210 0x7fe4c00b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.757+0000 7fe4c4949640 1 -- 172.21.15.47:0/3552391696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c015e7e0 con 0x7fe4c01f1d90 2024-03-23T09:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.758+0000 7fe4c4949640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4c01f1d90 0x7fe4c00b6ec0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000f860 tx=0x7fe4b000fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.758+0000 7fe4b5ffb640 1 -- 172.21.15.47:0/3552391696 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00014c0 con 0x7fe4c01f1d90 2024-03-23T09:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.758+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3552391696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c015eac0 con 0x7fe4c01f1d90 2024-03-23T09:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.758+0000 7fe4b5ffb640 1 -- 172.21.15.47:0/3552391696 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b00052e0 con 0x7fe4c01f1d90 2024-03-23T09:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.758+0000 7fe4b5ffb640 1 -- 172.21.15.47:0/3552391696 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b000d8f0 con 0x7fe4c01f1d90 2024-03-23T09:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.758+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3552391696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c015f500 con 0x7fe4c01f1d90 2024-03-23T09:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.759+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3552391696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c0072360 con 0x7fe4c01f1d90 2024-03-23T09:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.759+0000 7fe4b5ffb640 1 -- 172.21.15.47:0/3552391696 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe4b000da50 con 0x7fe4c01f1d90 2024-03-23T09:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.760+0000 7fe4b5ffb640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe4a406ac00 0x7fe4a406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.760+0000 7fe4b5ffb640 1 -- 172.21.15.47:0/3552391696 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe4b00893b0 con 0x7fe4c01f1d90 2024-03-23T09:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.760+0000 7fe4c514a640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe4a406ac00 0x7fe4a406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.760+0000 7fe4c514a640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe4a406ac00 0x7fe4a406d0c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc036000 tx=0x7fe4bc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.764+0000 7fe4b5ffb640 1 -- 172.21.15.47:0/3552391696 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe4b0054530 con 0x7fe4c01f1d90 2024-03-23T09:46:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.946+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3552391696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4c0066be0 con 0x7fe4c01f1d90 2024-03-23T09:46:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.946+0000 7fe4b5ffb640 1 -- 172.21.15.47:0/3552391696 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4b0057b50 con 0x7fe4c01f1d90 2024-03-23T09:46:55.947 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:46:55.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.948+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe4a406ac00 msgr2=0x7fe4a406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.948+0000 7fe4c73ae640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe4a406ac00 0x7fe4a406d0c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc036000 tx=0x7fe4bc030000 comp rx=0 tx=0).stop 2024-03-23T09:46:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.948+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3552391696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4c01f1d90 msgr2=0x7fe4c00b6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.948+0000 7fe4c73ae640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4c01f1d90 0x7fe4c00b6ec0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000f860 tx=0x7fe4b000fd20 comp rx=0 tx=0).stop 2024-03-23T09:46:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.948+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3552391696 shutdown_connections 2024-03-23T09:46:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.948+0000 7fe4c73ae640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe4a406ac00 0x7fe4a406d0c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.948+0000 7fe4c73ae640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4c01f2210 0x7fe4c00b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.948+0000 7fe4c73ae640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4c01f1d90 0x7fe4c00b6ec0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.948+0000 7fe4c73ae640 1 --2- 172.21.15.47:0/3552391696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4ac0039a0 0x7fe4c00c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.948+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3552391696 >> 172.21.15.47:0/3552391696 conn(0x7fe4c00659d0 msgr2=0x7fe4c00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.948+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3552391696 shutdown_connections 2024-03-23T09:46:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:55.948+0000 7fe4c73ae640 1 -- 172.21.15.47:0/3552391696 wait complete. 2024-03-23T09:46:55.960 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:46:56.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.063+0000 7ff2a89c3640 1 Processor -- start 2024-03-23T09:46:56.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.063+0000 7ff2a89c3640 1 -- start start 2024-03-23T09:46:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.064+0000 7ff2a89c3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a415cb50 0x7ff2a40b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.064+0000 7ff2a89c3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff2a40cd5a0 0x7ff2a40cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.064+0000 7ff2a89c3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a40cdf20 con 0x7ff2a40d1010 2024-03-23T09:46:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.064+0000 7ff2a89c3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a40b7e70 con 0x7ff2a40cd5a0 2024-03-23T09:46:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.064+0000 7ff2a89c3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a415cf30 con 0x7ff2a415cb50 2024-03-23T09:46:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.064+0000 7ff29bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff2a40cd5a0 0x7ff2a40cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.064+0000 7ff29bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff2a40cd5a0 0x7ff2a40cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43334/0 (socket says 172.21.15.47:43334) 2024-03-23T09:46:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.064+0000 7ff29bfff640 1 -- 172.21.15.47:0/801177971 learned_addr learned my addr 172.21.15.47:0/801177971 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.064+0000 7ff2a259c640 1 --2- 172.21.15.47:0/801177971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a415cb50 0x7ff2a40b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.064+0000 7ff29bfff640 1 -- 172.21.15.47:0/801177971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a415cb50 msgr2=0x7ff2a40b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.064+0000 7ff29bfff640 1 --2- 172.21.15.47:0/801177971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a415cb50 0x7ff2a40b84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.064+0000 7ff29bfff640 1 -- 172.21.15.47:0/801177971 >> v1:172.21.15.175:6789/0 conn(0x7ff2a40d1010 legacy=0x7ff2a40d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.064+0000 7ff29bfff640 1 -- 172.21.15.47:0/801177971 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40b72f0 con 0x7ff2a40cd5a0 2024-03-23T09:46:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.065+0000 7ff29bfff640 1 --2- 172.21.15.47:0/801177971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a40cd5a0 0x7ff2a40cd9e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009870 tx=0x7ff28c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce61dee9bd86092 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.065+0000 7ff2a1d9b640 1 -- 172.21.15.47:0/801177971 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c030840 con 0x7ff2a40cd5a0 2024-03-23T09:46:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.065+0000 7ff2a1d9b640 1 -- 172.21.15.47:0/801177971 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c0309a0 con 0x7ff2a40cd5a0 2024-03-23T09:46:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.065+0000 7ff2a1d9b640 1 -- 172.21.15.47:0/801177971 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c030d30 con 0x7ff2a40cd5a0 2024-03-23T09:46:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.065+0000 7ff2a89c3640 1 -- 172.21.15.47:0/801177971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a40cd5a0 msgr2=0x7ff2a40cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.065+0000 7ff2a89c3640 1 --2- 172.21.15.47:0/801177971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a40cd5a0 0x7ff2a40cd9e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009870 tx=0x7ff28c02b980 comp rx=0 tx=0).stop 2024-03-23T09:46:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.066+0000 7ff2a89c3640 1 -- 172.21.15.47:0/801177971 shutdown_connections 2024-03-23T09:46:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.066+0000 7ff2a89c3640 1 --2- 172.21.15.47:0/801177971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a40cd5a0 0x7ff2a40cd9e0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.066+0000 7ff2a89c3640 1 --2- 172.21.15.47:0/801177971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a415cb50 0x7ff2a40b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.066+0000 7ff2a89c3640 1 -- 172.21.15.47:0/801177971 >> 172.21.15.47:0/801177971 conn(0x7ff2a4065df0 msgr2=0x7ff2a4066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.066+0000 7ff2a89c3640 1 -- 172.21.15.47:0/801177971 shutdown_connections 2024-03-23T09:46:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.066+0000 7ff2a89c3640 1 -- 172.21.15.47:0/801177971 wait complete. 2024-03-23T09:46:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.067+0000 7ff2a89c3640 1 Processor -- start 2024-03-23T09:46:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.067+0000 7ff2a89c3640 1 -- start start 2024-03-23T09:46:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a89c3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a40cd5a0 0x7ff2a40cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a89c3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a40d1010 0x7ff2a415e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a89c3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a415cb50 0x7ff2a41f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a89c3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40c9eb0 con 0x7ff2a415cb50 2024-03-23T09:46:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a89c3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a4157890 con 0x7ff2a40d1010 2024-03-23T09:46:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a89c3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a415eff0 con 0x7ff2a40cd5a0 2024-03-23T09:46:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a415cb50 0x7ff2a41f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a415cb50 0x7ff2a41f2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43336/0 (socket says 172.21.15.47:43336) 2024-03-23T09:46:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a2d9d640 1 -- 172.21.15.47:0/2914059645 learned_addr learned my addr 172.21.15.47:0/2914059645 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a259c640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a40cd5a0 0x7ff2a40cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff29bfff640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a40d1010 0x7ff2a415e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a2d9d640 1 -- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a40cd5a0 msgr2=0x7ff2a40cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a2d9d640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a40cd5a0 0x7ff2a40cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a2d9d640 1 -- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a40d1010 msgr2=0x7ff2a415e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a2d9d640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a40d1010 0x7ff2a415e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a2d9d640 1 -- 172.21.15.47:0/2914059645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c0094d0 con 0x7ff2a415cb50 2024-03-23T09:46:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a2d9d640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a415cb50 0x7ff2a41f2e10 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00c780 tx=0x7ff29c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff29affd640 1 -- 172.21.15.47:0/2914059645 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c004b60 con 0x7ff2a415cb50 2024-03-23T09:46:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff29affd640 1 -- 172.21.15.47:0/2914059645 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c00ce90 con 0x7ff2a415cb50 2024-03-23T09:46:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a89c3640 1 -- 172.21.15.47:0/2914059645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a415ea30 con 0x7ff2a415cb50 2024-03-23T09:46:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff29affd640 1 -- 172.21.15.47:0/2914059645 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c00a850 con 0x7ff2a415cb50 2024-03-23T09:46:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.068+0000 7ff2a89c3640 1 -- 172.21.15.47:0/2914059645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a40c9ad0 con 0x7ff2a415cb50 2024-03-23T09:46:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.069+0000 7ff2a89c3640 1 -- 172.21.15.47:0/2914059645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a4072360 con 0x7ff2a415cb50 2024-03-23T09:46:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.070+0000 7ff29affd640 1 -- 172.21.15.47:0/2914059645 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff29c00a9b0 con 0x7ff2a415cb50 2024-03-23T09:46:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.070+0000 7ff29affd640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff28406ac00 0x7ff28406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.070+0000 7ff29affd640 1 -- 172.21.15.47:0/2914059645 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff29c0893a0 con 0x7ff2a415cb50 2024-03-23T09:46:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.070+0000 7ff2a259c640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff28406ac00 0x7ff28406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.071+0000 7ff2a259c640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff28406ac00 0x7ff28406d0c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff29400aa20 tx=0x7ff294008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:56.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.074+0000 7ff29affd640 1 -- 172.21.15.47:0/2914059645 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff29c054630 con 0x7ff2a415cb50 2024-03-23T09:46:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.252+0000 7ff2a89c3640 1 -- 172.21.15.47:0/2914059645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2a40c9ff0 con 0x7ff2a415cb50 2024-03-23T09:46:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.252+0000 7ff29affd640 1 -- 172.21.15.47:0/2914059645 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff2a40c9ff0 con 0x7ff2a415cb50 2024-03-23T09:46:56.254 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:46:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.255+0000 7ff2a89c3640 1 -- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff28406ac00 msgr2=0x7ff28406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.255+0000 7ff2a89c3640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff28406ac00 0x7ff28406d0c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff29400aa20 tx=0x7ff294008040 comp rx=0 tx=0).stop 2024-03-23T09:46:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.255+0000 7ff2a89c3640 1 -- 172.21.15.47:0/2914059645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a415cb50 msgr2=0x7ff2a41f2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.255+0000 7ff2a89c3640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a415cb50 0x7ff2a41f2e10 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00c780 tx=0x7ff29c00cc40 comp rx=0 tx=0).stop 2024-03-23T09:46:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.255+0000 7ff2a89c3640 1 -- 172.21.15.47:0/2914059645 shutdown_connections 2024-03-23T09:46:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.255+0000 7ff2a89c3640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff28406ac00 0x7ff28406d0c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.255+0000 7ff2a89c3640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a415cb50 0x7ff2a41f2e10 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.255+0000 7ff2a89c3640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a40d1010 0x7ff2a415e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.255+0000 7ff2a89c3640 1 --2- 172.21.15.47:0/2914059645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a40cd5a0 0x7ff2a40cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.255+0000 7ff2a89c3640 1 -- 172.21.15.47:0/2914059645 >> 172.21.15.47:0/2914059645 conn(0x7ff2a4065df0 msgr2=0x7ff2a40b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.255+0000 7ff2a89c3640 1 -- 172.21.15.47:0/2914059645 shutdown_connections 2024-03-23T09:46:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.255+0000 7ff2a89c3640 1 -- 172.21.15.47:0/2914059645 wait complete. 2024-03-23T09:46:56.266 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:46:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.368+0000 7f8cedad5640 1 Processor -- start 2024-03-23T09:46:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.369+0000 7f8cedad5640 1 -- start start 2024-03-23T09:46:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.369+0000 7f8cedad5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815c990 0x7f8ce8160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.369+0000 7f8cedad5640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce8161630 con 0x7f8ce80cde10 2024-03-23T09:46:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.369+0000 7f8cedad5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce80d1030 con 0x7f8ce80b8350 2024-03-23T09:46:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.369+0000 7f8cedad5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce80cfe10 con 0x7f8ce815c990 2024-03-23T09:46:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.369+0000 7f8ce7fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8ce80cde10 0x7f8ce80d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52826/0 (socket says 172.21.15.47:52826) 2024-03-23T09:46:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.369+0000 7f8ce7fff640 1 -- 172.21.15.47:0/785362122 learned_addr learned my addr 172.21.15.47:0/785362122 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce77fe640 1 --2- 172.21.15.47:0/785362122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815c990 0x7f8ce8160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248907931 0 0) 0x7f8ce8161630 con 0x7f8ce80cde10 2024-03-23T09:46:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cd0003520 con 0x7f8ce80cde10 2024-03-23T09:46:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2191688771 0 0) 0x7f8cd0003520 con 0x7f8ce80cde10 2024-03-23T09:46:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815c990 msgr2=0x7f8ce8160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 --2- 172.21.15.47:0/785362122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815c990 0x7f8ce8160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 >> v1:172.21.15.175:6789/0 conn(0x7f8ce80b8350 legacy=0x7f8ce80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce80b5b60 con 0x7f8ce80cde10 2024-03-23T09:46:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f8ce00019f0 con 0x7f8ce80cde10 2024-03-23T09:46:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 >> v1:172.21.15.47:6789/0 conn(0x7f8ce80cde10 legacy=0x7f8ce80d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 --2- 172.21.15.47:0/785362122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cd00039a0 0x7f8cd0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 --2- 172.21.15.47:0/785362122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cd00047c0 0x7f8cd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 --2- 172.21.15.47:0/785362122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815c990 0x7f8cd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce80cfe10 con 0x7f8cd00039a0 2024-03-23T09:46:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd000bab0 con 0x7f8cd00047c0 2024-03-23T09:46:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd000bc70 con 0x7f8ce815c990 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce77fe640 1 --2- 172.21.15.47:0/785362122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cd00039a0 0x7f8cd0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce6ffd640 1 --2- 172.21.15.47:0/785362122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cd00047c0 0x7f8cd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce7fff640 1 --2- 172.21.15.47:0/785362122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815c990 0x7f8cd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce77fe640 1 -- 172.21.15.47:0/785362122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815c990 msgr2=0x7f8cd00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce77fe640 1 --2- 172.21.15.47:0/785362122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815c990 0x7f8cd00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce77fe640 1 -- 172.21.15.47:0/785362122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cd00047c0 msgr2=0x7f8cd0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce77fe640 1 --2- 172.21.15.47:0/785362122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cd00047c0 0x7f8cd0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce77fe640 1 -- 172.21.15.47:0/785362122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd0003520 con 0x7f8cd00039a0 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.370+0000 7f8ce6ffd640 1 --2- 172.21.15.47:0/785362122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cd00047c0 0x7f8cd0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.371+0000 7f8ce77fe640 1 --2- 172.21.15.47:0/785362122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cd00039a0 0x7f8cd0003dc0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009ac0 tx=0x7f8cd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cbfa48a20bc94b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.371+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd802c6b0 con 0x7f8cd00039a0 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.371+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd802c810 con 0x7f8cd00039a0 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.371+0000 7f8ce67fc640 1 -- 172.21.15.47:0/785362122 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd802cba0 con 0x7f8cd00039a0 2024-03-23T09:46:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.372+0000 7f8cedad5640 1 -- 172.21.15.47:0/785362122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cd00039a0 msgr2=0x7f8cd0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.372+0000 7f8cedad5640 1 --2- 172.21.15.47:0/785362122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cd00039a0 0x7f8cd0003dc0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009ac0 tx=0x7f8cd00057f0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.372+0000 7f8ce7fff640 1 -- 172.21.15.47:0/785362122 reap_dead start 2024-03-23T09:46:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.372+0000 7f8cedad5640 1 -- 172.21.15.47:0/785362122 shutdown_connections 2024-03-23T09:46:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.372+0000 7f8cedad5640 1 -- 172.21.15.47:0/785362122 >> 172.21.15.47:0/785362122 conn(0x7f8ce80659d0 msgr2=0x7f8ce8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.372+0000 7f8cedad5640 1 -- 172.21.15.47:0/785362122 shutdown_connections 2024-03-23T09:46:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.372+0000 7f8cedad5640 1 -- 172.21.15.47:0/785362122 wait complete. 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.373+0000 7f8cedad5640 1 Processor -- start 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.373+0000 7f8cedad5640 1 -- start start 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8cedad5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cd00039a0 0x7f8ce815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8cedad5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815fc00 0x7f8ce80cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8cedad5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ce8160060 0x7f8ce80cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8cedad5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce80c9a90 con 0x7f8cd00039a0 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8cedad5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce80c9ca0 con 0x7f8ce8160060 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8cedad5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce80c9f80 con 0x7f8ce815fc00 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8ce77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cd00039a0 0x7f8ce815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8ce77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cd00039a0 0x7f8ce815ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43344/0 (socket says 172.21.15.47:43344) 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8ce77fe640 1 -- 172.21.15.47:0/3554322709 learned_addr learned my addr 172.21.15.47:0/3554322709 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8ce6ffd640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815fc00 0x7f8ce80cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8ce7fff640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ce8160060 0x7f8ce80cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8ce77fe640 1 -- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815fc00 msgr2=0x7f8ce80cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8ce77fe640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815fc00 0x7f8ce80cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8ce77fe640 1 -- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ce8160060 msgr2=0x7f8ce80cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8ce77fe640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ce8160060 0x7f8ce80cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8ce77fe640 1 -- 172.21.15.47:0/3554322709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce80ca260 con 0x7f8cd00039a0 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8ce6ffd640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815fc00 0x7f8ce80cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8ce7fff640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ce8160060 0x7f8ce80cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.374+0000 7f8ce77fe640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cd00039a0 0x7f8ce815ddb0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8cd802c250 tx=0x7f8cd8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.375+0000 7f8ccffff640 1 -- 172.21.15.47:0/3554322709 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8003810 con 0x7f8cd00039a0 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.375+0000 7f8ccffff640 1 -- 172.21.15.47:0/3554322709 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd8003970 con 0x7f8cd00039a0 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.375+0000 7f8cedad5640 1 -- 172.21.15.47:0/3554322709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce80ca540 con 0x7f8cd00039a0 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.375+0000 7f8ccffff640 1 -- 172.21.15.47:0/3554322709 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8003ce0 con 0x7f8cd00039a0 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.375+0000 7f8cedad5640 1 -- 172.21.15.47:0/3554322709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce815e2f0 con 0x7f8cd00039a0 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.376+0000 7f8cedad5640 1 -- 172.21.15.47:0/3554322709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce8072360 con 0x7f8cd00039a0 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.376+0000 7f8ccffff640 1 -- 172.21.15.47:0/3554322709 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8cd8047450 con 0x7f8cd00039a0 2024-03-23T09:46:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.376+0000 7f8ccffff640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8cd406ac00 0x7f8cd406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.377+0000 7f8ccffff640 1 -- 172.21.15.47:0/3554322709 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f8cd80ab5d0 con 0x7f8cd00039a0 2024-03-23T09:46:56.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.377+0000 7f8ce6ffd640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8cd406ac00 0x7f8cd406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.377+0000 7f8ce6ffd640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8cd406ac00 0x7f8cd406d0c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8ce81f53a0 tx=0x7f8cdc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:56.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.381+0000 7f8ccffff640 1 -- 172.21.15.47:0/3554322709 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8cd80767c0 con 0x7f8cd00039a0 2024-03-23T09:46:56.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.559+0000 7f8cedad5640 1 -- 172.21.15.47:0/3554322709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ce8066be0 con 0x7f8cd00039a0 2024-03-23T09:46:56.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.559+0000 7f8ccffff640 1 -- 172.21.15.47:0/3554322709 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8cd80044e0 con 0x7f8cd00039a0 2024-03-23T09:46:56.561 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:46:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.561+0000 7f8cedad5640 1 -- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8cd406ac00 msgr2=0x7f8cd406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.561+0000 7f8cedad5640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8cd406ac00 0x7f8cd406d0c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8ce81f53a0 tx=0x7f8cdc009040 comp rx=0 tx=0).stop 2024-03-23T09:46:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.561+0000 7f8cedad5640 1 -- 172.21.15.47:0/3554322709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cd00039a0 msgr2=0x7f8ce815ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.561+0000 7f8cedad5640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cd00039a0 0x7f8ce815ddb0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8cd802c250 tx=0x7f8cd8004730 comp rx=0 tx=0).stop 2024-03-23T09:46:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.561+0000 7f8cedad5640 1 -- 172.21.15.47:0/3554322709 shutdown_connections 2024-03-23T09:46:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.561+0000 7f8cedad5640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8cd406ac00 0x7f8cd406d0c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.561+0000 7f8cedad5640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ce8160060 0x7f8ce80cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.561+0000 7f8cedad5640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce815fc00 0x7f8ce80cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.561+0000 7f8cedad5640 1 --2- 172.21.15.47:0/3554322709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cd00039a0 0x7f8ce815ddb0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.561+0000 7f8cedad5640 1 -- 172.21.15.47:0/3554322709 >> 172.21.15.47:0/3554322709 conn(0x7f8ce80659d0 msgr2=0x7f8ce8156fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.562+0000 7f8cedad5640 1 -- 172.21.15.47:0/3554322709 shutdown_connections 2024-03-23T09:46:56.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.562+0000 7f8cedad5640 1 -- 172.21.15.47:0/3554322709 wait complete. 2024-03-23T09:46:56.573 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:46:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.673+0000 7fd61bbd5640 1 Processor -- start 2024-03-23T09:46:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.673+0000 7fd61bbd5640 1 -- start start 2024-03-23T09:46:56.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.674+0000 7fd61bbd5640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd61415cd00 0x7fd6140d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.674+0000 7fd61bbd5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6140d1020 0x7fd6140cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.674+0000 7fd61bbd5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6140cfba0 con 0x7fd6140cf750 2024-03-23T09:46:56.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.674+0000 7fd61bbd5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61415d0e0 con 0x7fd6140d1020 2024-03-23T09:46:56.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.674+0000 7fd61bbd5640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6140d1540 con 0x7fd61415cd00 2024-03-23T09:46:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.674+0000 7fd619170640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6140d1020 0x7fd6140cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.674+0000 7fd619170640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6140d1020 0x7fd6140cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43350/0 (socket says 172.21.15.47:43350) 2024-03-23T09:46:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.674+0000 7fd619170640 1 -- 172.21.15.47:0/3167167654 learned_addr learned my addr 172.21.15.47:0/3167167654 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.674+0000 7fd619971640 1 --2- 172.21.15.47:0/3167167654 >> v2:172.21.15.175:3300/0 conn(0x7fd61415cd00 0x7fd6140d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.674+0000 7fd619170640 1 -- 172.21.15.47:0/3167167654 >> v2:172.21.15.175:3300/0 conn(0x7fd61415cd00 msgr2=0x7fd6140d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.674+0000 7fd619170640 1 --2- 172.21.15.47:0/3167167654 >> v2:172.21.15.175:3300/0 conn(0x7fd61415cd00 0x7fd6140d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.675+0000 7fd619170640 1 -- 172.21.15.47:0/3167167654 >> v1:172.21.15.175:6789/0 conn(0x7fd6140cf750 legacy=0x7fd6140b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.675+0000 7fd619170640 1 -- 172.21.15.47:0/3167167654 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6140d1820 con 0x7fd6140d1020 2024-03-23T09:46:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.675+0000 7fd619971640 1 --2- 172.21.15.47:0/3167167654 >> v2:172.21.15.175:3300/0 conn(0x7fd61415cd00 0x7fd6140d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.675+0000 7fd619170640 1 --2- 172.21.15.47:0/3167167654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6140d1020 0x7fd6140cf210 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fd6140b5520 tx=0x7fd60402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc407a31692e7bb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.675+0000 7fd61896f640 1 -- 172.21.15.47:0/3167167654 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6040317c0 con 0x7fd6140d1020 2024-03-23T09:46:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.675+0000 7fd61896f640 1 -- 172.21.15.47:0/3167167654 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd604031920 con 0x7fd6140d1020 2024-03-23T09:46:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.675+0000 7fd61896f640 1 -- 172.21.15.47:0/3167167654 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd604031cb0 con 0x7fd6140d1020 2024-03-23T09:46:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.676+0000 7fd61bbd5640 1 -- 172.21.15.47:0/3167167654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6140d1020 msgr2=0x7fd6140cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.676+0000 7fd61bbd5640 1 --2- 172.21.15.47:0/3167167654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6140d1020 0x7fd6140cf210 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fd6140b5520 tx=0x7fd60402b860 comp rx=0 tx=0).stop 2024-03-23T09:46:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.676+0000 7fd61bbd5640 1 -- 172.21.15.47:0/3167167654 shutdown_connections 2024-03-23T09:46:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.676+0000 7fd61bbd5640 1 --2- 172.21.15.47:0/3167167654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6140d1020 0x7fd6140cf210 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.676+0000 7fd61bbd5640 1 --2- 172.21.15.47:0/3167167654 >> v2:172.21.15.175:3300/0 conn(0x7fd61415cd00 0x7fd6140d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.676+0000 7fd61bbd5640 1 -- 172.21.15.47:0/3167167654 >> 172.21.15.47:0/3167167654 conn(0x7fd61406b160 msgr2=0x7fd61406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.676+0000 7fd61bbd5640 1 -- 172.21.15.47:0/3167167654 shutdown_connections 2024-03-23T09:46:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.676+0000 7fd61bbd5640 1 -- 172.21.15.47:0/3167167654 wait complete. 2024-03-23T09:46:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd61bbd5640 1 Processor -- start 2024-03-23T09:46:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd61bbd5640 1 -- start start 2024-03-23T09:46:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd61bbd5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6140cf750 0x7fd614151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd61bbd5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6140d1020 0x7fd6140cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd61bbd5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd61415cd00 0x7fd6140cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd61bbd5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6141542f0 con 0x7fd6140cf750 2024-03-23T09:46:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd61bbd5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140cb4b0 con 0x7fd6140d1020 2024-03-23T09:46:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd61bbd5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6141545b0 con 0x7fd61415cd00 2024-03-23T09:46:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd619971640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6140cf750 0x7fd614151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd619971640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6140cf750 0x7fd614151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43356/0 (socket says 172.21.15.47:43356) 2024-03-23T09:46:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd619971640 1 -- 172.21.15.47:0/670611220 learned_addr learned my addr 172.21.15.47:0/670611220 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd619170640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6140d1020 0x7fd6140cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd61a172640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd61415cd00 0x7fd6140cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd619971640 1 -- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd61415cd00 msgr2=0x7fd6140cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd619971640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd61415cd00 0x7fd6140cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd619971640 1 -- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6140d1020 msgr2=0x7fd6140cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd619971640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6140d1020 0x7fd6140cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd619971640 1 -- 172.21.15.47:0/670611220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6040094d0 con 0x7fd6140cf750 2024-03-23T09:46:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.678+0000 7fd619971640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6140cf750 0x7fd614151910 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00ce00 tx=0x7fd60c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.679+0000 7fd60a7fc640 1 -- 172.21.15.47:0/670611220 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00f680 con 0x7fd6140cf750 2024-03-23T09:46:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.679+0000 7fd60a7fc640 1 -- 172.21.15.47:0/670611220 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c00f7e0 con 0x7fd6140cf750 2024-03-23T09:46:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.679+0000 7fd61bbd5640 1 -- 172.21.15.47:0/670611220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6140cac10 con 0x7fd6140cf750 2024-03-23T09:46:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.679+0000 7fd60a7fc640 1 -- 172.21.15.47:0/670611220 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00fbb0 con 0x7fd6140cf750 2024-03-23T09:46:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.679+0000 7fd61bbd5640 1 -- 172.21.15.47:0/670611220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6140cb1d0 con 0x7fd6140cf750 2024-03-23T09:46:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.680+0000 7fd61bbd5640 1 -- 172.21.15.47:0/670611220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd614072360 con 0x7fd6140cf750 2024-03-23T09:46:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.680+0000 7fd60a7fc640 1 -- 172.21.15.47:0/670611220 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd60c00b040 con 0x7fd6140cf750 2024-03-23T09:46:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.681+0000 7fd60a7fc640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5f806ac00 0x7fd5f806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.681+0000 7fd60a7fc640 1 -- 172.21.15.47:0/670611220 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd60c089ce0 con 0x7fd6140cf750 2024-03-23T09:46:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.681+0000 7fd619170640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5f806ac00 0x7fd5f806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.681+0000 7fd619170640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5f806ac00 0x7fd5f806d0c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd604036000 tx=0x7fd60403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.686+0000 7fd60a7fc640 1 -- 172.21.15.47:0/670611220 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd60c054f70 con 0x7fd6140cf750 2024-03-23T09:46:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.875+0000 7fd61bbd5640 1 -- 172.21.15.47:0/670611220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd614066be0 con 0x7fd6140cf750 2024-03-23T09:46:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.875+0000 7fd60a7fc640 1 -- 172.21.15.47:0/670611220 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v73) v1 ==== 80+0+120 (secure 0 0 0) 0x7fd60c058590 con 0x7fd6140cf750 2024-03-23T09:46:56.877 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6835/372771687 2024-03-24T09:45:33.740376+0000 2024-03-23T09:46:56.877 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6833/372771687 2024-03-24T09:45:33.740376+0000 2024-03-23T09:46:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:listed 2 entries 2024-03-23T09:46:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.877+0000 7fd61bbd5640 1 -- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5f806ac00 msgr2=0x7fd5f806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.877+0000 7fd61bbd5640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5f806ac00 0x7fd5f806d0c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd604036000 tx=0x7fd60403d040 comp rx=0 tx=0).stop 2024-03-23T09:46:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.877+0000 7fd61bbd5640 1 -- 172.21.15.47:0/670611220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6140cf750 msgr2=0x7fd614151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.877+0000 7fd61bbd5640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6140cf750 0x7fd614151910 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00ce00 tx=0x7fd60c00a650 comp rx=0 tx=0).stop 2024-03-23T09:46:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.877+0000 7fd61bbd5640 1 -- 172.21.15.47:0/670611220 shutdown_connections 2024-03-23T09:46:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.877+0000 7fd61bbd5640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5f806ac00 0x7fd5f806d0c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.877+0000 7fd61bbd5640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd61415cd00 0x7fd6140cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.877+0000 7fd61bbd5640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6140d1020 0x7fd6140cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.877+0000 7fd61bbd5640 1 --2- 172.21.15.47:0/670611220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6140cf750 0x7fd614151910 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.877+0000 7fd61bbd5640 1 -- 172.21.15.47:0/670611220 >> 172.21.15.47:0/670611220 conn(0x7fd61406b160 msgr2=0x7fd61415ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.877+0000 7fd61bbd5640 1 -- 172.21.15.47:0/670611220 shutdown_connections 2024-03-23T09:46:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:56.877+0000 7fd61bbd5640 1 -- 172.21.15.47:0/670611220 wait complete. 2024-03-23T09:46:56.889 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:46:56.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:56.889 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:46:56.914 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[64597]: fuse finished with error 0 and tester_r 0 2024-03-23T09:46:56.915 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:46:56.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:56.916 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:46:57.002 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:46:57.002 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:46:57.020 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:46:57.020 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:46:57.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:57.021 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-23T09:46:57.077 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-23T09:46:57.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:46:57.078 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:46:57.078 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:46:57.093 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:46:57.093 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:46:57.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:57.093 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-23T09:46:57.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:46:57.148 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-23T09:46:57.148 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:46:57.148 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:46:57.166 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:46:57.166 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:46:57.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:46:57.167 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-23T09:46:57.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:46:57.224 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-23T09:46:57.225 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:46:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.325+0000 7f1375d9f640 1 Processor -- start 2024-03-23T09:46:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.325+0000 7f1375d9f640 1 -- start start 2024-03-23T09:46:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.326+0000 7f1375d9f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137015c9d0 0x7f1370071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.326+0000 7f1375d9f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f137015cdb0 0x7f1370069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.326+0000 7f1375d9f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13700721f0 con 0x7f13700aecc0 2024-03-23T09:46:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.326+0000 7f1375d9f640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137006eb80 con 0x7f137015cdb0 2024-03-23T09:46:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.326+0000 7f1375d9f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137006edd0 con 0x7f137015c9d0 2024-03-23T09:46:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.326+0000 7f136f7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137015c9d0 0x7f1370071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.326+0000 7f136effd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f137015cdb0 0x7f1370069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.326+0000 7f136ffff640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f13700aecc0 0x7f13700cf5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:55148/0 (socket says 172.21.15.47:55148) 2024-03-23T09:46:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.326+0000 7f136f7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137015c9d0 0x7f1370071cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:53464/0 (socket says 172.21.15.47:53464) 2024-03-23T09:46:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.326+0000 7f136ffff640 1 -- 172.21.15.47:0/1299395949 learned_addr learned my addr 172.21.15.47:0/1299395949 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.327+0000 7f136f7fe640 1 -- 172.21.15.47:0/1299395949 >> v2:172.21.15.175:3300/0 conn(0x7f137015cdb0 msgr2=0x7f1370069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.327+0000 7f136f7fe640 1 --2- 172.21.15.47:0/1299395949 >> v2:172.21.15.175:3300/0 conn(0x7f137015cdb0 0x7f1370069ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.327+0000 7f136e7fc640 1 -- 172.21.15.47:0/1299395949 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199611734 0 0) 0x7f13700721f0 con 0x7f13700aecc0 2024-03-23T09:46:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.327+0000 7f136f7fe640 1 -- 172.21.15.47:0/1299395949 >> v1:172.21.15.175:6789/0 conn(0x7f13700aecc0 legacy=0x7f13700cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.327+0000 7f136f7fe640 1 -- 172.21.15.47:0/1299395949 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137006f0b0 con 0x7f137015c9d0 2024-03-23T09:46:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.328+0000 7f136f7fe640 1 --2- 172.21.15.47:0/1299395949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137015c9d0 0x7f1370071cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13600097a0 tx=0x7f136002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea556221ede98655 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.328+0000 7f136e7fc640 1 -- 172.21.15.47:0/1299395949 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360031830 con 0x7f137015c9d0 2024-03-23T09:46:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.328+0000 7f136e7fc640 1 -- 172.21.15.47:0/1299395949 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1360031990 con 0x7f137015c9d0 2024-03-23T09:46:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.328+0000 7f136e7fc640 1 -- 172.21.15.47:0/1299395949 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360031d20 con 0x7f137015c9d0 2024-03-23T09:46:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.328+0000 7f1375d9f640 1 -- 172.21.15.47:0/1299395949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137015c9d0 msgr2=0x7f1370071cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.328+0000 7f1375d9f640 1 --2- 172.21.15.47:0/1299395949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137015c9d0 0x7f1370071cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13600097a0 tx=0x7f136002b8d0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.328+0000 7f1375d9f640 1 -- 172.21.15.47:0/1299395949 shutdown_connections 2024-03-23T09:46:57.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.328+0000 7f1375d9f640 1 --2- 172.21.15.47:0/1299395949 >> v2:172.21.15.175:3300/0 conn(0x7f137015cdb0 0x7f1370069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.328+0000 7f1375d9f640 1 --2- 172.21.15.47:0/1299395949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137015c9d0 0x7f1370071cb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.328+0000 7f1375d9f640 1 -- 172.21.15.47:0/1299395949 >> 172.21.15.47:0/1299395949 conn(0x7f13700b13a0 msgr2=0x7f13700b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:57.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.329+0000 7f1375d9f640 1 -- 172.21.15.47:0/1299395949 shutdown_connections 2024-03-23T09:46:57.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.329+0000 7f1375d9f640 1 -- 172.21.15.47:0/1299395949 wait complete. 2024-03-23T09:46:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.330+0000 7f1375d9f640 1 Processor -- start 2024-03-23T09:46:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.330+0000 7f1375d9f640 1 -- start start 2024-03-23T09:46:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f1375d9f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13700aecc0 0x7f1370154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f1375d9f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137015c9d0 0x7f13700cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f1375d9f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137015cdb0 0x7f13700cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f1375d9f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137013eff0 con 0x7f13700aecc0 2024-03-23T09:46:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f1375d9f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137015f720 con 0x7f137015cdb0 2024-03-23T09:46:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f1375d9f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137015fa00 con 0x7f137015c9d0 2024-03-23T09:46:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f136f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13700aecc0 0x7f1370154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f136f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13700aecc0 0x7f1370154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43366/0 (socket says 172.21.15.47:43366) 2024-03-23T09:46:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f136f7fe640 1 -- 172.21.15.47:0/412006173 learned_addr learned my addr 172.21.15.47:0/412006173 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f136effd640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137015c9d0 0x7f13700cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f136ffff640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137015cdb0 0x7f13700cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f136f7fe640 1 -- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137015c9d0 msgr2=0x7f13700cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f136f7fe640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137015c9d0 0x7f13700cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f136f7fe640 1 -- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137015cdb0 msgr2=0x7f13700cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f136f7fe640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137015cdb0 0x7f13700cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f136f7fe640 1 -- 172.21.15.47:0/412006173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1360009400 con 0x7f13700aecc0 2024-03-23T09:46:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f136ffff640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137015cdb0 0x7f13700cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f136f7fe640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13700aecc0 0x7f1370154e70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f136002bee0 tx=0x7f13600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f1357fff640 1 -- 172.21.15.47:0/412006173 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136003a6f0 con 0x7f13700aecc0 2024-03-23T09:46:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f1357fff640 1 -- 172.21.15.47:0/412006173 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136003a870 con 0x7f13700aecc0 2024-03-23T09:46:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f1375d9f640 1 -- 172.21.15.47:0/412006173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137015fce0 con 0x7f13700aecc0 2024-03-23T09:46:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f1357fff640 1 -- 172.21.15.47:0/412006173 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136003ac20 con 0x7f13700aecc0 2024-03-23T09:46:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.331+0000 7f1375d9f640 1 -- 172.21.15.47:0/412006173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13701553b0 con 0x7f13700aecc0 2024-03-23T09:46:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.332+0000 7f1375d9f640 1 -- 172.21.15.47:0/412006173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1370066be0 con 0x7f13700aecc0 2024-03-23T09:46:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.333+0000 7f1357fff640 1 -- 172.21.15.47:0/412006173 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1360030380 con 0x7f13700aecc0 2024-03-23T09:46:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.333+0000 7f1357fff640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f135c06ac00 0x7f135c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.333+0000 7f1357fff640 1 -- 172.21.15.47:0/412006173 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f13600aae20 con 0x7f13700aecc0 2024-03-23T09:46:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.334+0000 7f136effd640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f135c06ac00 0x7f135c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.334+0000 7f136effd640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f135c06ac00 0x7f135c06d0c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f137015e730 tx=0x7f1364008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:57.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.338+0000 7f1357fff640 1 -- 172.21.15.47:0/412006173 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1360075e60 con 0x7f13700aecc0 2024-03-23T09:46:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.552+0000 7f1375d9f640 1 -- 172.21.15.47:0/412006173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13700717c0 con 0x7f13700aecc0 2024-03-23T09:46:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.552+0000 7f1357fff640 1 -- 172.21.15.47:0/412006173 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f1360048090 con 0x7f13700aecc0 2024-03-23T09:46:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:46:57.553 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3862818105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3862818105},{"type":"v1","addr":"172.21.15.175:6836","nonce":3862818105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3615592850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3615592850},{"type":"v1","addr":"172.21.15.175:6837","nonce":3615592850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/832829684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":832829684},{"type":"v1","addr":"172.21.15.47:6834","nonce":832829684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:45:18.296714+0000","modified":"2024-03-23T09:46:24.891678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7005":{"gid":7005,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":16,"addr":"172.21.15.47:6835/4141889683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4141889683},{"type":"v1","addr":"172.21.15.47:6835","nonce":4141889683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-23T09:46:57.553 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-23T09:46:57.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.554+0000 7f1375d9f640 1 -- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f135c06ac00 msgr2=0x7f135c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.554+0000 7f1375d9f640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f135c06ac00 0x7f135c06d0c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f137015e730 tx=0x7f1364008040 comp rx=0 tx=0).stop 2024-03-23T09:46:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.554+0000 7f1375d9f640 1 -- 172.21.15.47:0/412006173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13700aecc0 msgr2=0x7f1370154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.554+0000 7f1375d9f640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13700aecc0 0x7f1370154e70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f136002bee0 tx=0x7f13600046c0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.554+0000 7f1375d9f640 1 -- 172.21.15.47:0/412006173 shutdown_connections 2024-03-23T09:46:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.554+0000 7f1375d9f640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f135c06ac00 0x7f135c06d0c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.554+0000 7f1375d9f640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137015cdb0 0x7f13700cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.554+0000 7f1375d9f640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137015c9d0 0x7f13700cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.554+0000 7f1375d9f640 1 --2- 172.21.15.47:0/412006173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13700aecc0 0x7f1370154e70 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.554+0000 7f1375d9f640 1 -- 172.21.15.47:0/412006173 >> 172.21.15.47:0/412006173 conn(0x7f13700b13a0 msgr2=0x7f13700b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.554+0000 7f1375d9f640 1 -- 172.21.15.47:0/412006173 shutdown_connections 2024-03-23T09:46:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.554+0000 7f1375d9f640 1 -- 172.21.15.47:0/412006173 wait complete. 2024-03-23T09:46:57.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:46:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.665+0000 7f679abc3640 1 Processor -- start 2024-03-23T09:46:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.665+0000 7f679abc3640 1 -- start start 2024-03-23T09:46:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.666+0000 7f679abc3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f679415a8d0 0x7f67940cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.666+0000 7f679abc3640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67940cc610 con 0x7f67940cf360 2024-03-23T09:46:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.666+0000 7f679abc3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67940b0700 con 0x7f67940cb730 2024-03-23T09:46:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.666+0000 7f679abc3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67940c41c0 con 0x7f679415a8d0 2024-03-23T09:46:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.666+0000 7f679895f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f679415a8d0 0x7f67940cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.666+0000 7f6799160640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f67940cf360 0x7f67940b0b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52836/0 (socket says 172.21.15.47:52836) 2024-03-23T09:46:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.666+0000 7f6799160640 1 -- 172.21.15.47:0/1661939842 learned_addr learned my addr 172.21.15.47:0/1661939842 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.667+0000 7f679895f640 1 -- 172.21.15.47:0/1661939842 >> v1:172.21.15.175:6789/0 conn(0x7f67940cb730 legacy=0x7f67940cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.667+0000 7f679895f640 1 -- 172.21.15.47:0/1661939842 >> v1:172.21.15.47:6789/0 conn(0x7f67940cf360 legacy=0x7f67940b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.667+0000 7f67937fe640 1 -- 172.21.15.47:0/1661939842 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185256548 0 0) 0x7f67940cc610 con 0x7f67940cf360 2024-03-23T09:46:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.667+0000 7f679895f640 1 -- 172.21.15.47:0/1661939842 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67940cbd70 con 0x7f679415a8d0 2024-03-23T09:46:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.667+0000 7f679895f640 1 --2- 172.21.15.47:0/1661939842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679415a8d0 0x7f67940cb1f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f678c0097a0 tx=0x7f678c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852a739e377199cf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.668+0000 7f67937fe640 1 -- 172.21.15.47:0/1661939842 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c02e070 con 0x7f679415a8d0 2024-03-23T09:46:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.668+0000 7f67937fe640 1 -- 172.21.15.47:0/1661939842 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c0315a0 con 0x7f679415a8d0 2024-03-23T09:46:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.668+0000 7f67937fe640 1 -- 172.21.15.47:0/1661939842 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c031930 con 0x7f679415a8d0 2024-03-23T09:46:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.668+0000 7f679abc3640 1 -- 172.21.15.47:0/1661939842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679415a8d0 msgr2=0x7f67940cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.668+0000 7f679abc3640 1 --2- 172.21.15.47:0/1661939842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679415a8d0 0x7f67940cb1f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f678c0097a0 tx=0x7f678c02b840 comp rx=0 tx=0).stop 2024-03-23T09:46:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.668+0000 7f679abc3640 1 -- 172.21.15.47:0/1661939842 shutdown_connections 2024-03-23T09:46:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.668+0000 7f679abc3640 1 --2- 172.21.15.47:0/1661939842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679415a8d0 0x7f67940cb1f0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.668+0000 7f679abc3640 1 -- 172.21.15.47:0/1661939842 >> 172.21.15.47:0/1661939842 conn(0x7f67940d0670 msgr2=0x7f67940d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:57.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.668+0000 7f679abc3640 1 -- 172.21.15.47:0/1661939842 shutdown_connections 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.668+0000 7f679abc3640 1 -- 172.21.15.47:0/1661939842 wait complete. 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.669+0000 7f679abc3640 1 Processor -- start 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.669+0000 7f679abc3640 1 -- start start 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.670+0000 7f679abc3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67940cb730 0x7f67940afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.670+0000 7f679abc3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67940cf360 0x7f67940b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.670+0000 7f679abc3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f679415a8d0 0x7f67940b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.670+0000 7f679abc3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679415c0a0 con 0x7f67940cb730 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.670+0000 7f679abc3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67940b00f0 con 0x7f67940cf360 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.670+0000 7f679abc3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67940b3570 con 0x7f679415a8d0 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.670+0000 7f679895f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67940cb730 0x7f67940afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.670+0000 7f679895f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67940cb730 0x7f67940afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43382/0 (socket says 172.21.15.47:43382) 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.670+0000 7f679895f640 1 -- 172.21.15.47:0/3517978386 learned_addr learned my addr 172.21.15.47:0/3517978386 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.670+0000 7f6799160640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f679415a8d0 0x7f67940b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.671+0000 7f6793fff640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67940cf360 0x7f67940b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.671+0000 7f679895f640 1 -- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f679415a8d0 msgr2=0x7f67940b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.671+0000 7f679895f640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f679415a8d0 0x7f67940b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.671+0000 7f679895f640 1 -- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67940cf360 msgr2=0x7f67940b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.671+0000 7f679895f640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67940cf360 0x7f67940b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.671+0000 7f679895f640 1 -- 172.21.15.47:0/3517978386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c009400 con 0x7f67940cb730 2024-03-23T09:46:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.671+0000 7f679895f640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67940cb730 0x7f67940afbb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f678c034040 tx=0x7f678c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.671+0000 7f67917fa640 1 -- 172.21.15.47:0/3517978386 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c004230 con 0x7f67940cb730 2024-03-23T09:46:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.671+0000 7f67917fa640 1 -- 172.21.15.47:0/3517978386 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c0043b0 con 0x7f67940cb730 2024-03-23T09:46:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.671+0000 7f679abc3640 1 -- 172.21.15.47:0/3517978386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679415b420 con 0x7f67940cb730 2024-03-23T09:46:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.671+0000 7f67917fa640 1 -- 172.21.15.47:0/3517978386 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c004520 con 0x7f67940cb730 2024-03-23T09:46:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.671+0000 7f679abc3640 1 -- 172.21.15.47:0/3517978386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679415b9e0 con 0x7f67940cb730 2024-03-23T09:46:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.672+0000 7f679abc3640 1 -- 172.21.15.47:0/3517978386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679415c1e0 con 0x7f67940cb730 2024-03-23T09:46:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.672+0000 7f67917fa640 1 -- 172.21.15.47:0/3517978386 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f678c032880 con 0x7f67940cb730 2024-03-23T09:46:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.673+0000 7f67917fa640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f677406ac00 0x7f677406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.673+0000 7f67917fa640 1 -- 172.21.15.47:0/3517978386 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f678c0ab580 con 0x7f67940cb730 2024-03-23T09:46:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.673+0000 7f6793fff640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f677406ac00 0x7f677406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.673+0000 7f6793fff640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f677406ac00 0x7f677406d0c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f67941573d0 tx=0x7f67840094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.677+0000 7f67917fa640 1 -- 172.21.15.47:0/3517978386 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f679415c1e0 con 0x7f67940cb730 2024-03-23T09:46:57.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.860+0000 7f679abc3640 1 -- 172.21.15.47:0/3517978386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67940672f0 con 0x7f67940cb730 2024-03-23T09:46:57.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.861+0000 7f67917fa640 1 -- 172.21.15.47:0/3517978386 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16324 (secure 0 0 0) 0x7f678c080030 con 0x7f67940cb730 2024-03-23T09:46:57.861 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:46:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":73,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:45:52.838915+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:45:16.609941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-23T09:45:17.261503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.3","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/372771687":"2024-03-24T09:45:33.740376+0000","172.21.15.47:6833/372771687":"2024-03-24T09:45:33.740376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:46:57.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.863+0000 7f679abc3640 1 -- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f677406ac00 msgr2=0x7f677406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.863+0000 7f679abc3640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f677406ac00 0x7f677406d0c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f67941573d0 tx=0x7f67840094f0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.863+0000 7f679abc3640 1 -- 172.21.15.47:0/3517978386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67940cb730 msgr2=0x7f67940afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.863+0000 7f679abc3640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67940cb730 0x7f67940afbb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f678c034040 tx=0x7f678c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.863+0000 7f679abc3640 1 -- 172.21.15.47:0/3517978386 shutdown_connections 2024-03-23T09:46:57.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.863+0000 7f679abc3640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f677406ac00 0x7f677406d0c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.863+0000 7f679abc3640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f679415a8d0 0x7f67940b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.863+0000 7f679abc3640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67940cf360 0x7f67940b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.863+0000 7f679abc3640 1 --2- 172.21.15.47:0/3517978386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67940cb730 0x7f67940afbb0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.863+0000 7f679abc3640 1 -- 172.21.15.47:0/3517978386 >> 172.21.15.47:0/3517978386 conn(0x7f67940d0670 msgr2=0x7f67940c6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:57.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.863+0000 7f679abc3640 1 -- 172.21.15.47:0/3517978386 shutdown_connections 2024-03-23T09:46:57.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.863+0000 7f679abc3640 1 -- 172.21.15.47:0/3517978386 wait complete. 2024-03-23T09:46:57.874 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:45:16.609941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:45:17.261503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:46:57.874 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:46:57.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.978+0000 7f53c49b1640 1 Processor -- start 2024-03-23T09:46:57.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.978+0000 7f53c49b1640 1 -- start start 2024-03-23T09:46:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.979+0000 7f53c49b1640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f53c015cc30 0x7f53c00ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.979+0000 7f53c49b1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53c00cd4b0 0x7f53c00b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.979+0000 7f53c49b1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c00cd900 con 0x7f53c00cad40 2024-03-23T09:46:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.979+0000 7f53c49b1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c015d010 con 0x7f53c00cd4b0 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.979+0000 7f53c49b1640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c00b13d0 con 0x7f53c015cc30 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.979+0000 7f53bed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53c00cd4b0 0x7f53c00b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.979+0000 7f53bdd9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f53c00cad40 0x7f53c00b0610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52844/0 (socket says 172.21.15.47:52844) 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.979+0000 7f53bdd9b640 1 -- 172.21.15.47:0/160974758 learned_addr learned my addr 172.21.15.47:0/160974758 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.979+0000 7f53bed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53c00cd4b0 0x7f53c00b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43384/0 (socket says 172.21.15.47:43384) 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.980+0000 7f53bed9d640 1 -- 172.21.15.47:0/160974758 >> v2:172.21.15.175:3300/0 conn(0x7f53c015cc30 msgr2=0x7f53c00ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.980+0000 7f53bed9d640 1 --2- 172.21.15.47:0/160974758 >> v2:172.21.15.175:3300/0 conn(0x7f53c015cc30 0x7f53c00ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.980+0000 7f53bed9d640 1 -- 172.21.15.47:0/160974758 >> v1:172.21.15.47:6789/0 conn(0x7f53c00cad40 legacy=0x7f53c00b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.980+0000 7f53bed9d640 1 -- 172.21.15.47:0/160974758 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c00b16b0 con 0x7f53c00cd4b0 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.980+0000 7f53bd59a640 1 -- 172.21.15.47:0/160974758 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916167400 0 0) 0x7f53a8001110 con 0x7f53c00cad40 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.980+0000 7f53bed9d640 1 --2- 172.21.15.47:0/160974758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c00cd4b0 0x7f53c00b0d20 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f53c00cdca0 tx=0x7f53b002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3794e64ac2e3aefe server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.980+0000 7f53bd59a640 1 -- 172.21.15.47:0/160974758 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b002cc80 con 0x7f53c00cd4b0 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.980+0000 7f53bd59a640 1 -- 172.21.15.47:0/160974758 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b002cde0 con 0x7f53c00cd4b0 2024-03-23T09:46:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.980+0000 7f53bd59a640 1 -- 172.21.15.47:0/160974758 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0034660 con 0x7f53c00cd4b0 2024-03-23T09:46:57.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.981+0000 7f53c49b1640 1 -- 172.21.15.47:0/160974758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c00cd4b0 msgr2=0x7f53c00b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.981+0000 7f53c49b1640 1 --2- 172.21.15.47:0/160974758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c00cd4b0 0x7f53c00b0d20 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f53c00cdca0 tx=0x7f53b002bed0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.981+0000 7f53c49b1640 1 -- 172.21.15.47:0/160974758 shutdown_connections 2024-03-23T09:46:57.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.981+0000 7f53c49b1640 1 --2- 172.21.15.47:0/160974758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c00cd4b0 0x7f53c00b0d20 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.981+0000 7f53c49b1640 1 --2- 172.21.15.47:0/160974758 >> v2:172.21.15.175:3300/0 conn(0x7f53c015cc30 0x7f53c00ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.981+0000 7f53c49b1640 1 -- 172.21.15.47:0/160974758 >> 172.21.15.47:0/160974758 conn(0x7f53c00d06f0 msgr2=0x7f53c00d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.981+0000 7f53c49b1640 1 -- 172.21.15.47:0/160974758 shutdown_connections 2024-03-23T09:46:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.981+0000 7f53c49b1640 1 -- 172.21.15.47:0/160974758 wait complete. 2024-03-23T09:46:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.982+0000 7f53c49b1640 1 Processor -- start 2024-03-23T09:46:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.982+0000 7f53c49b1640 1 -- start start 2024-03-23T09:46:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53c49b1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53c00cad40 0x7f53c0148ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53c49b1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c00cd4b0 0x7f53c00c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53c49b1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c015cc30 0x7f53c00c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53c49b1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c01490e0 con 0x7f53c00cd4b0 2024-03-23T09:46:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53c49b1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c015d280 con 0x7f53c00cad40 2024-03-23T09:46:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53c49b1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c00c4720 con 0x7f53c015cc30 2024-03-23T09:46:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53bdd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c00cd4b0 0x7f53c00c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53bdd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c00cd4b0 0x7f53c00c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43394/0 (socket says 172.21.15.47:43394) 2024-03-23T09:46:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53bdd9b640 1 -- 172.21.15.47:0/227270118 learned_addr learned my addr 172.21.15.47:0/227270118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53bed9d640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c015cc30 0x7f53c00c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53be59c640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53c00cad40 0x7f53c0148ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53bdd9b640 1 -- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c015cc30 msgr2=0x7f53c00c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53bdd9b640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c015cc30 0x7f53c00c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53bdd9b640 1 -- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53c00cad40 msgr2=0x7f53c0148ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53bdd9b640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53c00cad40 0x7f53c0148ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53bdd9b640 1 -- 172.21.15.47:0/227270118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b0009a70 con 0x7f53c00cd4b0 2024-03-23T09:46:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.983+0000 7f53bed9d640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c015cc30 0x7f53c00c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.984+0000 7f53bdd9b640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c00cd4b0 0x7f53c00c41e0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f53a800c780 tx=0x7f53a800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.984+0000 7f53b6ffd640 1 -- 172.21.15.47:0/227270118 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a80072d0 con 0x7f53c00cd4b0 2024-03-23T09:46:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.984+0000 7f53b6ffd640 1 -- 172.21.15.47:0/227270118 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a8007450 con 0x7f53c00cd4b0 2024-03-23T09:46:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.984+0000 7f53b6ffd640 1 -- 172.21.15.47:0/227270118 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a80047e0 con 0x7f53c00cd4b0 2024-03-23T09:46:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.984+0000 7f53c49b1640 1 -- 172.21.15.47:0/227270118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c015d520 con 0x7f53c00cd4b0 2024-03-23T09:46:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.984+0000 7f53c49b1640 1 -- 172.21.15.47:0/227270118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c015dae0 con 0x7f53c00cd4b0 2024-03-23T09:46:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.985+0000 7f53c49b1640 1 -- 172.21.15.47:0/227270118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c0066be0 con 0x7f53c00cd4b0 2024-03-23T09:46:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.985+0000 7f53b6ffd640 1 -- 172.21.15.47:0/227270118 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f53a8004960 con 0x7f53c00cd4b0 2024-03-23T09:46:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.986+0000 7f53b6ffd640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f53a006ac00 0x7f53a006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.986+0000 7f53b6ffd640 1 -- 172.21.15.47:0/227270118 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f53a8088470 con 0x7f53c00cd4b0 2024-03-23T09:46:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.986+0000 7f53be59c640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f53a006ac00 0x7f53a006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.986+0000 7f53be59c640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f53a006ac00 0x7f53a006d0c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f53b800a770 tx=0x7f53b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:57.990+0000 7f53b6ffd640 1 -- 172.21.15.47:0/227270118 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f53a80535c0 con 0x7f53c00cd4b0 2024-03-23T09:46:58.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.168+0000 7f53c49b1640 1 -- 172.21.15.47:0/227270118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53c00cd900 con 0x7f53c00cd4b0 2024-03-23T09:46:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.169+0000 7f53b6ffd640 1 -- 172.21.15.47:0/227270118 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53c00cd900 con 0x7f53c00cd4b0 2024-03-23T09:46:58.169 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:46:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.171+0000 7f53c49b1640 1 -- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f53a006ac00 msgr2=0x7f53a006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.171+0000 7f53c49b1640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f53a006ac00 0x7f53a006d0c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f53b800a770 tx=0x7f53b8008040 comp rx=0 tx=0).stop 2024-03-23T09:46:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.171+0000 7f53c49b1640 1 -- 172.21.15.47:0/227270118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c00cd4b0 msgr2=0x7f53c00c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.171+0000 7f53c49b1640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c00cd4b0 0x7f53c00c41e0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f53a800c780 tx=0x7f53a800cc40 comp rx=0 tx=0).stop 2024-03-23T09:46:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.171+0000 7f53c49b1640 1 -- 172.21.15.47:0/227270118 shutdown_connections 2024-03-23T09:46:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.171+0000 7f53c49b1640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f53a006ac00 0x7f53a006d0c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.171+0000 7f53c49b1640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c015cc30 0x7f53c00c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.171+0000 7f53c49b1640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c00cd4b0 0x7f53c00c41e0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.171+0000 7f53c49b1640 1 --2- 172.21.15.47:0/227270118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53c00cad40 0x7f53c0148ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.171+0000 7f53c49b1640 1 -- 172.21.15.47:0/227270118 >> 172.21.15.47:0/227270118 conn(0x7f53c00d06f0 msgr2=0x7f53c006ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.171+0000 7f53c49b1640 1 -- 172.21.15.47:0/227270118 shutdown_connections 2024-03-23T09:46:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.171+0000 7f53c49b1640 1 -- 172.21.15.47:0/227270118 wait complete. 2024-03-23T09:46:58.182 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:46:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.285+0000 7f99143e7640 1 Processor -- start 2024-03-23T09:46:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.285+0000 7f99143e7640 1 -- start start 2024-03-23T09:46:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.286+0000 7f99143e7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f990c0caf80 0x7f990c0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.286+0000 7f99143e7640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f990c0cb360 0x7f990c161000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.286+0000 7f99143e7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f990c0b0810 0x7f990c0b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.286+0000 7f99143e7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990c0b11c0 con 0x7f990c0caf80 2024-03-23T09:46:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.286+0000 7f99143e7640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990c0d0810 con 0x7f990c0cb360 2024-03-23T09:46:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.286+0000 7f99143e7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990c160b70 con 0x7f990c0b0810 2024-03-23T09:46:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.286+0000 7f9912183640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f990c0caf80 0x7f990c0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.286+0000 7f9912183640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f990c0caf80 0x7f990c0cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43402/0 (socket says 172.21.15.47:43402) 2024-03-23T09:46:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.286+0000 7f9912183640 1 -- 172.21.15.47:0/4146113695 learned_addr learned my addr 172.21.15.47:0/4146113695 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.286+0000 7f9912984640 1 --2- 172.21.15.47:0/4146113695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f990c0b0810 0x7f990c0b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.287+0000 7f9911982640 1 --2- 172.21.15.47:0/4146113695 >> v2:172.21.15.175:3300/0 conn(0x7f990c0cb360 0x7f990c161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.287+0000 7f9912183640 1 -- 172.21.15.47:0/4146113695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f990c0b0810 msgr2=0x7f990c0b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:58.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.287+0000 7f9912183640 1 --2- 172.21.15.47:0/4146113695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f990c0b0810 0x7f990c0b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.287+0000 7f9912183640 1 -- 172.21.15.47:0/4146113695 >> v2:172.21.15.175:3300/0 conn(0x7f990c0cb360 msgr2=0x7f990c161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:58.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.287+0000 7f9912183640 1 --2- 172.21.15.47:0/4146113695 >> v2:172.21.15.175:3300/0 conn(0x7f990c0cb360 0x7f990c161000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.287+0000 7f9912183640 1 -- 172.21.15.47:0/4146113695 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c0cef80 con 0x7f990c0caf80 2024-03-23T09:46:58.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.287+0000 7f9912183640 1 --2- 172.21.15.47:0/4146113695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f990c0caf80 0x7f990c0cb9f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f990c0b0670 tx=0x7f98fc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef95b5acdd5c3cb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:58.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.287+0000 7f9911181640 1 -- 172.21.15.47:0/4146113695 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc02ca70 con 0x7f990c0caf80 2024-03-23T09:46:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.288+0000 7f9911181640 1 -- 172.21.15.47:0/4146113695 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98fc02cbd0 con 0x7f990c0caf80 2024-03-23T09:46:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.288+0000 7f9911181640 1 -- 172.21.15.47:0/4146113695 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc0343f0 con 0x7f990c0caf80 2024-03-23T09:46:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.288+0000 7f99143e7640 1 -- 172.21.15.47:0/4146113695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f990c0caf80 msgr2=0x7f990c0cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.288+0000 7f99143e7640 1 --2- 172.21.15.47:0/4146113695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f990c0caf80 0x7f990c0cb9f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f990c0b0670 tx=0x7f98fc02bed0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.288+0000 7f99143e7640 1 -- 172.21.15.47:0/4146113695 shutdown_connections 2024-03-23T09:46:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.288+0000 7f99143e7640 1 --2- 172.21.15.47:0/4146113695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f990c0b0810 0x7f990c0b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.288+0000 7f99143e7640 1 --2- 172.21.15.47:0/4146113695 >> v2:172.21.15.175:3300/0 conn(0x7f990c0cb360 0x7f990c161000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.288+0000 7f99143e7640 1 --2- 172.21.15.47:0/4146113695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f990c0caf80 0x7f990c0cb9f0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.288+0000 7f99143e7640 1 -- 172.21.15.47:0/4146113695 >> 172.21.15.47:0/4146113695 conn(0x7f990c098a00 msgr2=0x7f990c0cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.288+0000 7f99143e7640 1 -- 172.21.15.47:0/4146113695 shutdown_connections 2024-03-23T09:46:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.288+0000 7f99143e7640 1 -- 172.21.15.47:0/4146113695 wait complete. 2024-03-23T09:46:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.289+0000 7f99143e7640 1 Processor -- start 2024-03-23T09:46:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.289+0000 7f99143e7640 1 -- start start 2024-03-23T09:46:58.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f99143e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f990c0b0810 0x7f990c0ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f99143e7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f990c0caf80 0x7f990c153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f99143e7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f990c0cb360 0x7f990c0aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f99143e7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c154330 con 0x7f990c0b0810 2024-03-23T09:46:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f99143e7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c0ad210 con 0x7f990c0caf80 2024-03-23T09:46:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f99143e7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990c0ad4f0 con 0x7f990c0cb360 2024-03-23T09:46:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f9912183640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f990c0b0810 0x7f990c0ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f9912183640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f990c0b0810 0x7f990c0ae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43406/0 (socket says 172.21.15.47:43406) 2024-03-23T09:46:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f9912183640 1 -- 172.21.15.47:0/597063963 learned_addr learned my addr 172.21.15.47:0/597063963 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f9912984640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f990c0cb360 0x7f990c0aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f9911982640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f990c0caf80 0x7f990c153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f9912183640 1 -- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f990c0cb360 msgr2=0x7f990c0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f9912183640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f990c0cb360 0x7f990c0aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f9912183640 1 -- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f990c0caf80 msgr2=0x7f990c153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f9912183640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f990c0caf80 0x7f990c153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.290+0000 7f9912183640 1 -- 172.21.15.47:0/597063963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc009b30 con 0x7f990c0b0810 2024-03-23T09:46:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.291+0000 7f9912183640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f990c0b0810 0x7f990c0ae2b0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f98fc036040 tx=0x7f98fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.291+0000 7f990affd640 1 -- 172.21.15.47:0/597063963 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc009510 con 0x7f990c0b0810 2024-03-23T09:46:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.291+0000 7f990affd640 1 -- 172.21.15.47:0/597063963 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98fc03e450 con 0x7f990c0b0810 2024-03-23T09:46:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.291+0000 7f99143e7640 1 -- 172.21.15.47:0/597063963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990c0ad7d0 con 0x7f990c0b0810 2024-03-23T09:46:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.291+0000 7f990affd640 1 -- 172.21.15.47:0/597063963 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc03e800 con 0x7f990c0b0810 2024-03-23T09:46:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.291+0000 7f99143e7640 1 -- 172.21.15.47:0/597063963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990c0add90 con 0x7f990c0b0810 2024-03-23T09:46:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.292+0000 7f99143e7640 1 -- 172.21.15.47:0/597063963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990c0672f0 con 0x7f990c0b0810 2024-03-23T09:46:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.292+0000 7f990affd640 1 -- 172.21.15.47:0/597063963 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f98fc03e960 con 0x7f990c0b0810 2024-03-23T09:46:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.293+0000 7f990affd640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f98f806ac00 0x7f98f806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.293+0000 7f990affd640 1 -- 172.21.15.47:0/597063963 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f98fc0ac1a0 con 0x7f990c0b0810 2024-03-23T09:46:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.293+0000 7f9911982640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f98f806ac00 0x7f98f806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.294+0000 7f9911982640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f98f806ac00 0x7f98f806d0c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f990c160d90 tx=0x7f9900008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:58.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.297+0000 7f990affd640 1 -- 172.21.15.47:0/597063963 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f98fc0774c0 con 0x7f990c0b0810 2024-03-23T09:46:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.482+0000 7f99143e7640 1 -- 172.21.15.47:0/597063963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f990c0b56a0 con 0x7f990c0b0810 2024-03-23T09:46:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.483+0000 7f990affd640 1 -- 172.21.15.47:0/597063963 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7f98fc07aae0 con 0x7f990c0b0810 2024-03-23T09:46:58.483 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:46:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.485+0000 7f99143e7640 1 -- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f98f806ac00 msgr2=0x7f98f806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.485+0000 7f99143e7640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f98f806ac00 0x7f98f806d0c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f990c160d90 tx=0x7f9900008040 comp rx=0 tx=0).stop 2024-03-23T09:46:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.485+0000 7f99143e7640 1 -- 172.21.15.47:0/597063963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f990c0b0810 msgr2=0x7f990c0ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.485+0000 7f99143e7640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f990c0b0810 0x7f990c0ae2b0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f98fc036040 tx=0x7f98fc004a40 comp rx=0 tx=0).stop 2024-03-23T09:46:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.485+0000 7f99143e7640 1 -- 172.21.15.47:0/597063963 shutdown_connections 2024-03-23T09:46:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.485+0000 7f99143e7640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f98f806ac00 0x7f98f806d0c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.485+0000 7f99143e7640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f990c0cb360 0x7f990c0aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.485+0000 7f99143e7640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f990c0caf80 0x7f990c153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.485+0000 7f99143e7640 1 --2- 172.21.15.47:0/597063963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f990c0b0810 0x7f990c0ae2b0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.485+0000 7f99143e7640 1 -- 172.21.15.47:0/597063963 >> 172.21.15.47:0/597063963 conn(0x7f990c098a00 msgr2=0x7f990c15ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.485+0000 7f99143e7640 1 -- 172.21.15.47:0/597063963 shutdown_connections 2024-03-23T09:46:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.485+0000 7f99143e7640 1 -- 172.21.15.47:0/597063963 wait complete. 2024-03-23T09:46:58.496 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:46:58.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.600+0000 7f3636684640 1 Processor -- start 2024-03-23T09:46:58.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.600+0000 7f3636684640 1 -- start start 2024-03-23T09:46:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.601+0000 7f3636684640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36300cde10 0x7f3630160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.601+0000 7f3636684640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3630161630 con 0x7f363015c990 2024-03-23T09:46:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.601+0000 7f3636684640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36300d1030 con 0x7f36300b8350 2024-03-23T09:46:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.601+0000 7f3636684640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36300cfe10 con 0x7f36300cde10 2024-03-23T09:46:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.601+0000 7f362ffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f363015c990 0x7f36300cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52860/0 (socket says 172.21.15.47:52860) 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.601+0000 7f362ffff640 1 -- 172.21.15.47:0/1365583459 learned_addr learned my addr 172.21.15.47:0/1365583459 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.601+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 162709730 0 0) 0x7f3630161630 con 0x7f363015c990 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.601+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f361c003520 con 0x7f363015c990 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362f7fe640 1 --2- 172.21.15.47:0/1365583459 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36300cde10 0x7f3630160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1211007187 0 0) 0x7f361c003520 con 0x7f363015c990 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36300cde10 msgr2=0x7f3630160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362effd640 1 --2- 172.21.15.47:0/1365583459 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36300cde10 0x7f3630160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 >> v1:172.21.15.175:6789/0 conn(0x7f36300b8350 legacy=0x7f36300d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36300b5b60 con 0x7f363015c990 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f3618001670 con 0x7f363015c990 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 >> v1:172.21.15.47:6789/0 conn(0x7f363015c990 legacy=0x7f36300cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362effd640 1 --2- 172.21.15.47:0/1365583459 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36300cde10 0x7f361c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362effd640 1 --2- 172.21.15.47:0/1365583459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f361c003f20 0x7f361c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362effd640 1 --2- 172.21.15.47:0/1365583459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361c004d00 0x7f361c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36300cfe10 con 0x7f361c004d00 2024-03-23T09:46:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c00bbc0 con 0x7f361c003f20 2024-03-23T09:46:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c00bd80 con 0x7f36300cde10 2024-03-23T09:46:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f3634c21640 1 --2- 172.21.15.47:0/1365583459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361c004d00 0x7f361c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362ffff640 1 --2- 172.21.15.47:0/1365583459 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36300cde10 0x7f361c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f362f7fe640 1 --2- 172.21.15.47:0/1365583459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f361c003f20 0x7f361c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f3634c21640 1 -- 172.21.15.47:0/1365583459 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36300cde10 msgr2=0x7f361c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f3634c21640 1 --2- 172.21.15.47:0/1365583459 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36300cde10 0x7f361c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f3634c21640 1 -- 172.21.15.47:0/1365583459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f361c003f20 msgr2=0x7f361c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f3634c21640 1 --2- 172.21.15.47:0/1365583459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f361c003f20 0x7f361c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.602+0000 7f3634c21640 1 -- 172.21.15.47:0/1365583459 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c003520 con 0x7f361c004d00 2024-03-23T09:46:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.603+0000 7f3634c21640 1 --2- 172.21.15.47:0/1365583459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361c004d00 0x7f361c005180 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3628001e50 tx=0x7f361c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92b955c740ae114 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.603+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628003b70 con 0x7f361c004d00 2024-03-23T09:46:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.603+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3628003cd0 con 0x7f361c004d00 2024-03-23T09:46:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.603+0000 7f362effd640 1 -- 172.21.15.47:0/1365583459 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628005310 con 0x7f361c004d00 2024-03-23T09:46:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.604+0000 7f3636684640 1 -- 172.21.15.47:0/1365583459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361c004d00 msgr2=0x7f361c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.604+0000 7f3636684640 1 --2- 172.21.15.47:0/1365583459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361c004d00 0x7f361c005180 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3628001e50 tx=0x7f361c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.604+0000 7f3634c21640 1 -- 172.21.15.47:0/1365583459 reap_dead start 2024-03-23T09:46:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.604+0000 7f3636684640 1 -- 172.21.15.47:0/1365583459 shutdown_connections 2024-03-23T09:46:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.604+0000 7f3636684640 1 -- 172.21.15.47:0/1365583459 >> 172.21.15.47:0/1365583459 conn(0x7f36300659d0 msgr2=0x7f3630065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.604+0000 7f3636684640 1 -- 172.21.15.47:0/1365583459 shutdown_connections 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.604+0000 7f3636684640 1 -- 172.21.15.47:0/1365583459 wait complete. 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.605+0000 7f3636684640 1 Processor -- start 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.605+0000 7f3636684640 1 -- start start 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f3636684640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361c004d00 0x7f3630153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f3636684640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f363015ee50 0x7f36300c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f3636684640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36301585f0 0x7f3630158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f3636684640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3630159ca0 con 0x7f361c004d00 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f3636684640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3630153a80 con 0x7f363015ee50 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f3636684640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3630159090 con 0x7f36301585f0 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f362ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361c004d00 0x7f3630153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f362ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361c004d00 0x7f3630153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43428/0 (socket says 172.21.15.47:43428) 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f362ffff640 1 -- 172.21.15.47:0/3391975937 learned_addr learned my addr 172.21.15.47:0/3391975937 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f362ffff640 1 -- 172.21.15.47:0/3391975937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36301585f0 msgr2=0x7f3630158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f3634c21640 1 --2- 172.21.15.47:0/3391975937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36301585f0 0x7f3630158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f362ffff640 1 --2- 172.21.15.47:0/3391975937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36301585f0 0x7f3630158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f362ffff640 1 -- 172.21.15.47:0/3391975937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f363015ee50 msgr2=0x7f36300c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f362ffff640 1 --2- 172.21.15.47:0/3391975937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f363015ee50 0x7f36300c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f362ffff640 1 -- 172.21.15.47:0/3391975937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3630159370 con 0x7f361c004d00 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f362ffff640 1 --2- 172.21.15.47:0/3391975937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361c004d00 0x7f3630153540 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f36180072c0 tx=0x7f3618007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f362cff9640 1 -- 172.21.15.47:0/3391975937 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618004ea0 con 0x7f361c004d00 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f362cff9640 1 -- 172.21.15.47:0/3391975937 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361800f960 con 0x7f361c004d00 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f3636684640 1 -- 172.21.15.47:0/3391975937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3630159650 con 0x7f361c004d00 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.606+0000 7f362cff9640 1 -- 172.21.15.47:0/3391975937 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800fd10 con 0x7f361c004d00 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.607+0000 7f3636684640 1 -- 172.21.15.47:0/3391975937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36300c9fd0 con 0x7f361c004d00 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.608+0000 7f3636684640 1 -- 172.21.15.47:0/3391975937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36300ca190 con 0x7f361c004d00 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.608+0000 7f362cff9640 1 -- 172.21.15.47:0/3391975937 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f361801d460 con 0x7f361c004d00 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.608+0000 7f362cff9640 1 --2- 172.21.15.47:0/3391975937 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f361006ac30 0x7f361006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.608+0000 7f362cff9640 1 -- 172.21.15.47:0/3391975937 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f36180895d0 con 0x7f361c004d00 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.609+0000 7f362f7fe640 1 --2- 172.21.15.47:0/3391975937 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f361006ac30 0x7f361006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.609+0000 7f362f7fe640 1 --2- 172.21.15.47:0/3391975937 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f361006ac30 0x7f361006d0f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f36300cdd70 tx=0x7f36200095d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:58.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.613+0000 7f362cff9640 1 -- 172.21.15.47:0/3391975937 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3618054690 con 0x7f361c004d00 2024-03-23T09:46:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.793+0000 7f3636684640 1 -- 172.21.15.47:0/3391975937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3630072360 con 0x7f361c004d00 2024-03-23T09:46:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.793+0000 7f362cff9640 1 -- 172.21.15.47:0/3391975937 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3618057cb0 con 0x7f361c004d00 2024-03-23T09:46:58.793 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:46:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.795+0000 7f3636684640 1 -- 172.21.15.47:0/3391975937 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f361006ac30 msgr2=0x7f361006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.795+0000 7f3636684640 1 --2- 172.21.15.47:0/3391975937 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f361006ac30 0x7f361006d0f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f36300cdd70 tx=0x7f36200095d0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.795+0000 7f3636684640 1 -- 172.21.15.47:0/3391975937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361c004d00 msgr2=0x7f3630153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.795+0000 7f3636684640 1 --2- 172.21.15.47:0/3391975937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361c004d00 0x7f3630153540 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f36180072c0 tx=0x7f3618007780 comp rx=0 tx=0).stop 2024-03-23T09:46:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.795+0000 7f3636684640 1 -- 172.21.15.47:0/3391975937 shutdown_connections 2024-03-23T09:46:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.795+0000 7f3636684640 1 --2- 172.21.15.47:0/3391975937 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f361006ac30 0x7f361006d0f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.795+0000 7f3636684640 1 --2- 172.21.15.47:0/3391975937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36301585f0 0x7f3630158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.795+0000 7f3636684640 1 --2- 172.21.15.47:0/3391975937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f363015ee50 0x7f36300c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.795+0000 7f3636684640 1 --2- 172.21.15.47:0/3391975937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361c004d00 0x7f3630153540 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.795+0000 7f3636684640 1 -- 172.21.15.47:0/3391975937 >> 172.21.15.47:0/3391975937 conn(0x7f36300659d0 msgr2=0x7f36300b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.795+0000 7f3636684640 1 -- 172.21.15.47:0/3391975937 shutdown_connections 2024-03-23T09:46:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.795+0000 7f3636684640 1 -- 172.21.15.47:0/3391975937 wait complete. 2024-03-23T09:46:58.806 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:46:58.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.906+0000 7f8243aab640 1 Processor -- start 2024-03-23T09:46:58.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.907+0000 7f8243aab640 1 -- start start 2024-03-23T09:46:58.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.907+0000 7f8243aab640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f823c15c9d0 0x7f823c071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.907+0000 7f8243aab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f823c15cdb0 0x7f823c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.907+0000 7f8243aab640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f823c0aecc0 0x7f823c06a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.907+0000 7f8243aab640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c0721f0 con 0x7f823c15c9d0 2024-03-23T09:46:58.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.907+0000 7f8243aab640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c06eb80 con 0x7f823c0aecc0 2024-03-23T09:46:58.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.907+0000 7f8243aab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c06edd0 con 0x7f823c15cdb0 2024-03-23T09:46:58.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.908+0000 7f8241847640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f823c15c9d0 0x7f823c071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.908+0000 7f8241847640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f823c15c9d0 0x7f823c071cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43436/0 (socket says 172.21.15.47:43436) 2024-03-23T09:46:58.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.908+0000 7f8241847640 1 -- 172.21.15.47:0/4129547512 learned_addr learned my addr 172.21.15.47:0/4129547512 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:58.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.908+0000 7f8241046640 1 --2- 172.21.15.47:0/4129547512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f823c15cdb0 0x7f823c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.908+0000 7f8242048640 1 --2- 172.21.15.47:0/4129547512 >> v2:172.21.15.175:3300/0 conn(0x7f823c0aecc0 0x7f823c06a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.908+0000 7f8241847640 1 -- 172.21.15.47:0/4129547512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f823c15cdb0 msgr2=0x7f823c069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:58.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.908+0000 7f8241847640 1 --2- 172.21.15.47:0/4129547512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f823c15cdb0 0x7f823c069ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.908+0000 7f8241847640 1 -- 172.21.15.47:0/4129547512 >> v2:172.21.15.175:3300/0 conn(0x7f823c0aecc0 msgr2=0x7f823c06a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:58.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.908+0000 7f8241847640 1 --2- 172.21.15.47:0/4129547512 >> v2:172.21.15.175:3300/0 conn(0x7f823c0aecc0 0x7f823c06a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.908+0000 7f8241847640 1 -- 172.21.15.47:0/4129547512 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c06f0b0 con 0x7f823c15c9d0 2024-03-23T09:46:58.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.909+0000 7f8241847640 1 --2- 172.21.15.47:0/4129547512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f823c15c9d0 0x7f823c071cb0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f823c070f90 tx=0x7f823802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0e37862087fd04 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:58.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.909+0000 7f8240845640 1 -- 172.21.15.47:0/4129547512 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238031870 con 0x7f823c15c9d0 2024-03-23T09:46:58.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.909+0000 7f8240845640 1 -- 172.21.15.47:0/4129547512 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82380319d0 con 0x7f823c15c9d0 2024-03-23T09:46:58.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.909+0000 7f8240845640 1 -- 172.21.15.47:0/4129547512 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238031d60 con 0x7f823c15c9d0 2024-03-23T09:46:58.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.909+0000 7f8243aab640 1 -- 172.21.15.47:0/4129547512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f823c15c9d0 msgr2=0x7f823c071cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.909+0000 7f8243aab640 1 --2- 172.21.15.47:0/4129547512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f823c15c9d0 0x7f823c071cb0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f823c070f90 tx=0x7f823802b910 comp rx=0 tx=0).stop 2024-03-23T09:46:58.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.909+0000 7f8243aab640 1 -- 172.21.15.47:0/4129547512 shutdown_connections 2024-03-23T09:46:58.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.909+0000 7f8243aab640 1 --2- 172.21.15.47:0/4129547512 >> v2:172.21.15.175:3300/0 conn(0x7f823c0aecc0 0x7f823c06a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.909+0000 7f8243aab640 1 --2- 172.21.15.47:0/4129547512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f823c15cdb0 0x7f823c069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.909+0000 7f8243aab640 1 --2- 172.21.15.47:0/4129547512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f823c15c9d0 0x7f823c071cb0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.909+0000 7f8243aab640 1 -- 172.21.15.47:0/4129547512 >> 172.21.15.47:0/4129547512 conn(0x7f823c0b13a0 msgr2=0x7f823c0b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:58.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.909+0000 7f8243aab640 1 -- 172.21.15.47:0/4129547512 shutdown_connections 2024-03-23T09:46:58.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.909+0000 7f8243aab640 1 -- 172.21.15.47:0/4129547512 wait complete. 2024-03-23T09:46:58.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.911+0000 7f8243aab640 1 Processor -- start 2024-03-23T09:46:58.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.911+0000 7f8243aab640 1 -- start start 2024-03-23T09:46:58.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.911+0000 7f8243aab640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f823c0aecc0 0x7f823c14d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.911+0000 7f8243aab640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f823c15c9d0 0x7f823c156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.911+0000 7f8243aab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f823c15cdb0 0x7f823c1571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.911+0000 7f8243aab640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c15d1c0 con 0x7f823c0aecc0 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.911+0000 7f8243aab640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c15d4a0 con 0x7f823c15c9d0 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.911+0000 7f8243aab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c15d780 con 0x7f823c15cdb0 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.911+0000 7f8241847640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f823c0aecc0 0x7f823c14d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.911+0000 7f8241847640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f823c0aecc0 0x7f823c14d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43448/0 (socket says 172.21.15.47:43448) 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.911+0000 7f8241847640 1 -- 172.21.15.47:0/1868519328 learned_addr learned my addr 172.21.15.47:0/1868519328 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.912+0000 7f8242048640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f823c15cdb0 0x7f823c1571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.912+0000 7f8241847640 1 -- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f823c15cdb0 msgr2=0x7f823c1571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.912+0000 7f8241046640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f823c15c9d0 0x7f823c156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.912+0000 7f8241847640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f823c15cdb0 0x7f823c1571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.912+0000 7f8241847640 1 -- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f823c15c9d0 msgr2=0x7f823c156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.912+0000 7f8241847640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f823c15c9d0 0x7f823c156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.912+0000 7f8241847640 1 -- 172.21.15.47:0/1868519328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82380094d0 con 0x7f823c0aecc0 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.912+0000 7f8241847640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f823c0aecc0 0x7f823c14d170 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f823802b950 tx=0x7f82380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:58.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.912+0000 7f82327fc640 1 -- 172.21.15.47:0/1868519328 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823803ac60 con 0x7f823c0aecc0 2024-03-23T09:46:58.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.912+0000 7f82327fc640 1 -- 172.21.15.47:0/1868519328 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823803adc0 con 0x7f823c0aecc0 2024-03-23T09:46:58.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.912+0000 7f8243aab640 1 -- 172.21.15.47:0/1868519328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823c15da60 con 0x7f823c0aecc0 2024-03-23T09:46:58.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.912+0000 7f82327fc640 1 -- 172.21.15.47:0/1868519328 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238044630 con 0x7f823c0aecc0 2024-03-23T09:46:58.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.912+0000 7f8243aab640 1 -- 172.21.15.47:0/1868519328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823c14d6b0 con 0x7f823c0aecc0 2024-03-23T09:46:58.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.913+0000 7f8243aab640 1 -- 172.21.15.47:0/1868519328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823c15df80 con 0x7f823c0aecc0 2024-03-23T09:46:58.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.913+0000 7f82327fc640 1 -- 172.21.15.47:0/1868519328 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f823804d450 con 0x7f823c0aecc0 2024-03-23T09:46:58.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.914+0000 7f82327fc640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f822006ac00 0x7f822006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:58.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.914+0000 7f82327fc640 1 -- 172.21.15.47:0/1868519328 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f823802d040 con 0x7f823c0aecc0 2024-03-23T09:46:58.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.914+0000 7f8241046640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f822006ac00 0x7f822006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:58.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.915+0000 7f8241046640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f822006ac00 0x7f822006d0c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f823c15bab0 tx=0x7f822c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:58.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:58.918+0000 7f82327fc640 1 -- 172.21.15.47:0/1868519328 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f823c15df80 con 0x7f823c0aecc0 2024-03-23T09:46:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.135+0000 7f8243aab640 1 -- 172.21.15.47:0/1868519328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f823c15df80 con 0x7f823c0aecc0 2024-03-23T09:46:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.135+0000 7f82327fc640 1 -- 172.21.15.47:0/1868519328 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f823c15df80 con 0x7f823c0aecc0 2024-03-23T09:46:59.135 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:46:59.135 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3862818105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3862818105},{"type":"v1","addr":"172.21.15.175:6836","nonce":3862818105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3615592850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3615592850},{"type":"v1","addr":"172.21.15.175:6837","nonce":3615592850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/832829684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":832829684},{"type":"v1","addr":"172.21.15.47:6834","nonce":832829684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:45:18.296714+0000","modified":"2024-03-23T09:46:24.891678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7005":{"gid":7005,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":16,"addr":"172.21.15.47:6835/4141889683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4141889683},{"type":"v1","addr":"172.21.15.47:6835","nonce":4141889683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-23T09:46:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-23T09:46:59.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.137+0000 7f8243aab640 1 -- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f822006ac00 msgr2=0x7f822006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.137+0000 7f8243aab640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f822006ac00 0x7f822006d0c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f823c15bab0 tx=0x7f822c008040 comp rx=0 tx=0).stop 2024-03-23T09:46:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.137+0000 7f8243aab640 1 -- 172.21.15.47:0/1868519328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f823c0aecc0 msgr2=0x7f823c14d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.137+0000 7f8243aab640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f823c0aecc0 0x7f823c14d170 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f823802b950 tx=0x7f82380046c0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.137+0000 7f8243aab640 1 -- 172.21.15.47:0/1868519328 shutdown_connections 2024-03-23T09:46:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.137+0000 7f8243aab640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f822006ac00 0x7f822006d0c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.137+0000 7f8243aab640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f823c15cdb0 0x7f823c1571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.137+0000 7f8243aab640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f823c15c9d0 0x7f823c156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.137+0000 7f8243aab640 1 --2- 172.21.15.47:0/1868519328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f823c0aecc0 0x7f823c14d170 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.137+0000 7f8243aab640 1 -- 172.21.15.47:0/1868519328 >> 172.21.15.47:0/1868519328 conn(0x7f823c0b13a0 msgr2=0x7f823c15bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.137+0000 7f8243aab640 1 -- 172.21.15.47:0/1868519328 shutdown_connections 2024-03-23T09:46:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.137+0000 7f8243aab640 1 -- 172.21.15.47:0/1868519328 wait complete. 2024-03-23T09:46:59.148 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:46:59.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.248+0000 7f9a75870640 1 Processor -- start 2024-03-23T09:46:59.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.248+0000 7f9a75870640 1 -- start start 2024-03-23T09:46:59.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.249+0000 7f9a75870640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9a7015cee0 0x7f9a700d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.249+0000 7f9a75870640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a700d2f80 0x7f9a700b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.249+0000 7f9a75870640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a700d1300 con 0x7f9a700d23a0 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.249+0000 7f9a75870640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a700d15e0 con 0x7f9a700d2f80 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.249+0000 7f9a75870640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a700d18c0 con 0x7f9a7015cee0 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.249+0000 7f9a6f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a700d2f80 0x7f9a700b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.249+0000 7f9a6f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a700d2f80 0x7f9a700b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43454/0 (socket says 172.21.15.47:43454) 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.249+0000 7f9a6f7fe640 1 -- 172.21.15.47:0/871961964 learned_addr learned my addr 172.21.15.47:0/871961964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.249+0000 7f9a6effd640 1 --2- 172.21.15.47:0/871961964 >> v2:172.21.15.175:3300/0 conn(0x7f9a7015cee0 0x7f9a700d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.250+0000 7f9a6f7fe640 1 -- 172.21.15.47:0/871961964 >> v2:172.21.15.175:3300/0 conn(0x7f9a7015cee0 msgr2=0x7f9a700d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.250+0000 7f9a6f7fe640 1 --2- 172.21.15.47:0/871961964 >> v2:172.21.15.175:3300/0 conn(0x7f9a7015cee0 0x7f9a700d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.250+0000 7f9a6f7fe640 1 -- 172.21.15.47:0/871961964 >> v1:172.21.15.175:6789/0 conn(0x7f9a700d23a0 legacy=0x7f9a700d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.250+0000 7f9a6f7fe640 1 -- 172.21.15.47:0/871961964 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a700d1ba0 con 0x7f9a700d2f80 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.250+0000 7f9a6effd640 1 --2- 172.21.15.47:0/871961964 >> v2:172.21.15.175:3300/0 conn(0x7f9a7015cee0 0x7f9a700d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.250+0000 7f9a6f7fe640 1 --2- 172.21.15.47:0/871961964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a700d2f80 0x7f9a700b5a80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f9a700d3430 tx=0x7f9a6002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac297980fd66d4a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.250+0000 7f9a6dffb640 1 -- 172.21.15.47:0/871961964 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6002ca80 con 0x7f9a700d2f80 2024-03-23T09:46:59.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.250+0000 7f9a6dffb640 1 -- 172.21.15.47:0/871961964 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6002cbe0 con 0x7f9a700d2f80 2024-03-23T09:46:59.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.250+0000 7f9a6dffb640 1 -- 172.21.15.47:0/871961964 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600343f0 con 0x7f9a700d2f80 2024-03-23T09:46:59.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.251+0000 7f9a75870640 1 -- 172.21.15.47:0/871961964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a700d2f80 msgr2=0x7f9a700b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.251+0000 7f9a75870640 1 --2- 172.21.15.47:0/871961964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a700d2f80 0x7f9a700b5a80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f9a700d3430 tx=0x7f9a6002bee0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.251+0000 7f9a75870640 1 -- 172.21.15.47:0/871961964 shutdown_connections 2024-03-23T09:46:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.251+0000 7f9a75870640 1 --2- 172.21.15.47:0/871961964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a700d2f80 0x7f9a700b5a80 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.251+0000 7f9a75870640 1 --2- 172.21.15.47:0/871961964 >> v2:172.21.15.175:3300/0 conn(0x7f9a7015cee0 0x7f9a700d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.251+0000 7f9a75870640 1 -- 172.21.15.47:0/871961964 >> 172.21.15.47:0/871961964 conn(0x7f9a70065df0 msgr2=0x7f9a70066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.251+0000 7f9a75870640 1 -- 172.21.15.47:0/871961964 shutdown_connections 2024-03-23T09:46:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.251+0000 7f9a75870640 1 -- 172.21.15.47:0/871961964 wait complete. 2024-03-23T09:46:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.252+0000 7f9a75870640 1 Processor -- start 2024-03-23T09:46:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.252+0000 7f9a75870640 1 -- start start 2024-03-23T09:46:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a75870640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a700d23a0 0x7f9a70153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a75870640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a700d2f80 0x7f9a700cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a75870640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a7015cee0 0x7f9a700cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a75870640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700ca6e0 con 0x7f9a700d23a0 2024-03-23T09:46:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a75870640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a70154a80 con 0x7f9a7015cee0 2024-03-23T09:46:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a75870640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a70154d60 con 0x7f9a700d2f80 2024-03-23T09:46:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a6effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a700d23a0 0x7f9a70153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a6effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a700d23a0 0x7f9a70153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43466/0 (socket says 172.21.15.47:43466) 2024-03-23T09:46:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a6effd640 1 -- 172.21.15.47:0/2612203472 learned_addr learned my addr 172.21.15.47:0/2612203472 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a6effd640 1 -- 172.21.15.47:0/2612203472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a700d2f80 msgr2=0x7f9a700cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a6effd640 1 --2- 172.21.15.47:0/2612203472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a700d2f80 0x7f9a700cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a6effd640 1 -- 172.21.15.47:0/2612203472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a7015cee0 msgr2=0x7f9a700cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a6effd640 1 --2- 172.21.15.47:0/2612203472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a7015cee0 0x7f9a700cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.253+0000 7f9a6effd640 1 -- 172.21.15.47:0/2612203472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a60009b30 con 0x7f9a700d23a0 2024-03-23T09:46:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.254+0000 7f9a6effd640 1 --2- 172.21.15.47:0/2612203472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a700d23a0 0x7f9a70153190 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800c7f0 tx=0x7f9a6800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.254+0000 7f9a67fff640 1 -- 172.21.15.47:0/2612203472 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68007820 con 0x7f9a700d23a0 2024-03-23T09:46:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.254+0000 7f9a67fff640 1 -- 172.21.15.47:0/2612203472 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a68007980 con 0x7f9a700d23a0 2024-03-23T09:46:59.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.254+0000 7f9a75870640 1 -- 172.21.15.47:0/2612203472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a70155040 con 0x7f9a700d23a0 2024-03-23T09:46:59.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.254+0000 7f9a67fff640 1 -- 172.21.15.47:0/2612203472 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68007cf0 con 0x7f9a700d23a0 2024-03-23T09:46:59.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.254+0000 7f9a75870640 1 -- 172.21.15.47:0/2612203472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a700ca9c0 con 0x7f9a700d23a0 2024-03-23T09:46:59.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.255+0000 7f9a75870640 1 -- 172.21.15.47:0/2612203472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a70072360 con 0x7f9a700d23a0 2024-03-23T09:46:59.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.255+0000 7f9a67fff640 1 -- 172.21.15.47:0/2612203472 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9a680113f0 con 0x7f9a700d23a0 2024-03-23T09:46:59.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.255+0000 7f9a67fff640 1 --2- 172.21.15.47:0/2612203472 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9a5006ac30 0x7f9a5006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.256+0000 7f9a67fff640 1 -- 172.21.15.47:0/2612203472 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9a68089650 con 0x7f9a700d23a0 2024-03-23T09:46:59.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.256+0000 7f9a6e7fc640 1 --2- 172.21.15.47:0/2612203472 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9a5006ac30 0x7f9a5006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.256+0000 7f9a6e7fc640 1 --2- 172.21.15.47:0/2612203472 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9a5006ac30 0x7f9a5006d0f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9a700cd150 tx=0x7f9a58007990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:59.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.260+0000 7f9a67fff640 1 -- 172.21.15.47:0/2612203472 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9a68054830 con 0x7f9a700d23a0 2024-03-23T09:46:59.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.443+0000 7f9a75870640 1 -- 172.21.15.47:0/2612203472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a70066be0 con 0x7f9a700d23a0 2024-03-23T09:46:59.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.444+0000 7f9a67fff640 1 -- 172.21.15.47:0/2612203472 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16324 (secure 0 0 0) 0x7f9a68057e50 con 0x7f9a700d23a0 2024-03-23T09:46:59.445 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:46:59.445 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":73,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:45:52.838915+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:45:16.609941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-23T09:45:17.261503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.3","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/372771687":"2024-03-24T09:45:33.740376+0000","172.21.15.47:6833/372771687":"2024-03-24T09:45:33.740376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:46:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.447+0000 7f9a75870640 1 -- 172.21.15.47:0/2612203472 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9a5006ac30 msgr2=0x7f9a5006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.447+0000 7f9a75870640 1 --2- 172.21.15.47:0/2612203472 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9a5006ac30 0x7f9a5006d0f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9a700cd150 tx=0x7f9a58007990 comp rx=0 tx=0).stop 2024-03-23T09:46:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.447+0000 7f9a75870640 1 -- 172.21.15.47:0/2612203472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a700d23a0 msgr2=0x7f9a70153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.447+0000 7f9a75870640 1 --2- 172.21.15.47:0/2612203472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a700d23a0 0x7f9a70153190 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800c7f0 tx=0x7f9a6800ccb0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.447+0000 7f9a75870640 1 -- 172.21.15.47:0/2612203472 shutdown_connections 2024-03-23T09:46:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.447+0000 7f9a75870640 1 --2- 172.21.15.47:0/2612203472 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9a5006ac30 0x7f9a5006d0f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.447+0000 7f9a75870640 1 --2- 172.21.15.47:0/2612203472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a7015cee0 0x7f9a700cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.447+0000 7f9a75870640 1 --2- 172.21.15.47:0/2612203472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a700d2f80 0x7f9a700cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.447+0000 7f9a75870640 1 --2- 172.21.15.47:0/2612203472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a700d23a0 0x7f9a70153190 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.447+0000 7f9a75870640 1 -- 172.21.15.47:0/2612203472 >> 172.21.15.47:0/2612203472 conn(0x7f9a70065df0 msgr2=0x7f9a700b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.447+0000 7f9a75870640 1 -- 172.21.15.47:0/2612203472 shutdown_connections 2024-03-23T09:46:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.447+0000 7f9a75870640 1 -- 172.21.15.47:0/2612203472 wait complete. 2024-03-23T09:46:59.458 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T09:46:59.458 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:46:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.558+0000 7fc3997e0640 1 Processor -- start 2024-03-23T09:46:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.559+0000 7fc3997e0640 1 -- start start 2024-03-23T09:46:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.559+0000 7fc3997e0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc39415ca90 0x7fc394160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.559+0000 7fc3997e0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc3940b8350 0x7fc3940cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.559+0000 7fc3997e0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3940ce090 con 0x7fc3940cdc40 2024-03-23T09:46:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.559+0000 7fc3997e0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3940d11e0 con 0x7fc39415ca90 2024-03-23T09:46:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.559+0000 7fc3997e0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3940d2680 con 0x7fc3940b8350 2024-03-23T09:46:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.559+0000 7fc392ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc39415ca90 0x7fc394160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.559+0000 7fc3937fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc3940cdc40 0x7fc3940b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52874/0 (socket says 172.21.15.47:52874) 2024-03-23T09:46:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.559+0000 7fc3937fe640 1 -- 172.21.15.47:0/1327702672 learned_addr learned my addr 172.21.15.47:0/1327702672 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:46:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.560+0000 7fc3927fc640 1 --2- 172.21.15.47:0/1327702672 >> v2:172.21.15.175:3300/0 conn(0x7fc3940b8350 0x7fc3940cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.560+0000 7fc392ffd640 1 -- 172.21.15.47:0/1327702672 >> v2:172.21.15.175:3300/0 conn(0x7fc3940b8350 msgr2=0x7fc3940cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.560+0000 7fc392ffd640 1 --2- 172.21.15.47:0/1327702672 >> v2:172.21.15.175:3300/0 conn(0x7fc3940b8350 0x7fc3940cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.560+0000 7fc392ffd640 1 -- 172.21.15.47:0/1327702672 >> v1:172.21.15.47:6789/0 conn(0x7fc3940cdc40 legacy=0x7fc3940b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.560+0000 7fc391ffb640 1 -- 172.21.15.47:0/1327702672 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629874310 0 0) 0x7fc384001490 con 0x7fc3940cdc40 2024-03-23T09:46:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.560+0000 7fc392ffd640 1 -- 172.21.15.47:0/1327702672 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc394160f60 con 0x7fc39415ca90 2024-03-23T09:46:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.560+0000 7fc3927fc640 1 --2- 172.21.15.47:0/1327702672 >> v2:172.21.15.175:3300/0 conn(0x7fc3940b8350 0x7fc3940cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:46:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.561+0000 7fc392ffd640 1 --2- 172.21.15.47:0/1327702672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc39415ca90 0x7fc394160710 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc394160670 tx=0x7fc38c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c0e47034a9fe26 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.561+0000 7fc391ffb640 1 -- 172.21.15.47:0/1327702672 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c02e070 con 0x7fc39415ca90 2024-03-23T09:46:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.561+0000 7fc391ffb640 1 -- 172.21.15.47:0/1327702672 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c0315a0 con 0x7fc39415ca90 2024-03-23T09:46:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.561+0000 7fc391ffb640 1 -- 172.21.15.47:0/1327702672 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c031930 con 0x7fc39415ca90 2024-03-23T09:46:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.561+0000 7fc3997e0640 1 -- 172.21.15.47:0/1327702672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc39415ca90 msgr2=0x7fc394160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.561+0000 7fc3997e0640 1 --2- 172.21.15.47:0/1327702672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc39415ca90 0x7fc394160710 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc394160670 tx=0x7fc38c02b840 comp rx=0 tx=0).stop 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.561+0000 7fc3997e0640 1 -- 172.21.15.47:0/1327702672 shutdown_connections 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.561+0000 7fc3997e0640 1 --2- 172.21.15.47:0/1327702672 >> v2:172.21.15.175:3300/0 conn(0x7fc3940b8350 0x7fc3940cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.561+0000 7fc3997e0640 1 --2- 172.21.15.47:0/1327702672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc39415ca90 0x7fc394160710 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.561+0000 7fc3997e0640 1 -- 172.21.15.47:0/1327702672 >> 172.21.15.47:0/1327702672 conn(0x7fc3940659d0 msgr2=0x7fc394065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.561+0000 7fc3997e0640 1 -- 172.21.15.47:0/1327702672 shutdown_connections 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.561+0000 7fc3997e0640 1 -- 172.21.15.47:0/1327702672 wait complete. 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.563+0000 7fc3997e0640 1 Processor -- start 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.563+0000 7fc3997e0640 1 -- start start 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.563+0000 7fc3997e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3940b8350 0x7fc394150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.563+0000 7fc3997e0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3940cdc40 0x7fc3940b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.563+0000 7fc3997e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39415ca90 0x7fc3940b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.563+0000 7fc3997e0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3941f2140 con 0x7fc3940cdc40 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.563+0000 7fc3997e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39415bf10 con 0x7fc39415ca90 2024-03-23T09:46:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.563+0000 7fc3997e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3941f6640 con 0x7fc3940b8350 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.563+0000 7fc3927fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3940cdc40 0x7fc3940b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.563+0000 7fc3927fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3940cdc40 0x7fc3940b2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43476/0 (socket says 172.21.15.47:43476) 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.563+0000 7fc3927fc640 1 -- 172.21.15.47:0/4174940092 learned_addr learned my addr 172.21.15.47:0/4174940092 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.564+0000 7fc392ffd640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3940b8350 0x7fc394150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.564+0000 7fc3937fe640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39415ca90 0x7fc3940b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.564+0000 7fc3927fc640 1 -- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3940b8350 msgr2=0x7fc394150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.564+0000 7fc3927fc640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3940b8350 0x7fc394150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.564+0000 7fc3927fc640 1 -- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39415ca90 msgr2=0x7fc3940b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.564+0000 7fc3927fc640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39415ca90 0x7fc3940b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.564+0000 7fc3927fc640 1 -- 172.21.15.47:0/4174940092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c009400 con 0x7fc3940cdc40 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.564+0000 7fc3927fc640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3940cdc40 0x7fc3940b2910 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc38800d7d0 tx=0x7fc38800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.564+0000 7fc37b7fe640 1 -- 172.21.15.47:0/4174940092 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388004030 con 0x7fc3940cdc40 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.564+0000 7fc37b7fe640 1 -- 172.21.15.47:0/4174940092 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc388004190 con 0x7fc3940cdc40 2024-03-23T09:46:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.564+0000 7fc3997e0640 1 -- 172.21.15.47:0/4174940092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3941510e0 con 0x7fc3940cdc40 2024-03-23T09:46:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.564+0000 7fc37b7fe640 1 -- 172.21.15.47:0/4174940092 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388004520 con 0x7fc3940cdc40 2024-03-23T09:46:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.564+0000 7fc3997e0640 1 -- 172.21.15.47:0/4174940092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3941f2330 con 0x7fc3940cdc40 2024-03-23T09:46:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.565+0000 7fc3997e0640 1 -- 172.21.15.47:0/4174940092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc394072360 con 0x7fc3940cdc40 2024-03-23T09:46:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.565+0000 7fc37b7fe640 1 -- 172.21.15.47:0/4174940092 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc3880096a0 con 0x7fc3940cdc40 2024-03-23T09:46:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.566+0000 7fc37b7fe640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc38006ac30 0x7fc38006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.566+0000 7fc37b7fe640 1 -- 172.21.15.47:0/4174940092 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc388089490 con 0x7fc3940cdc40 2024-03-23T09:46:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.566+0000 7fc392ffd640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc38006ac30 0x7fc38006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.566+0000 7fc392ffd640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc38006ac30 0x7fc38006d0f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc38c036000 tx=0x7fc38c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:59.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.570+0000 7fc37b7fe640 1 -- 172.21.15.47:0/4174940092 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc388054670 con 0x7fc3940cdc40 2024-03-23T09:46:59.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.784+0000 7fc3997e0640 1 -- 172.21.15.47:0/4174940092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc394066be0 con 0x7fc3940cdc40 2024-03-23T09:46:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.785+0000 7fc37b7fe640 1 -- 172.21.15.47:0/4174940092 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4741 (secure 0 0 0) 0x7fc388057c90 con 0x7fc3940cdc40 2024-03-23T09:46:59.785 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:46:59.785 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3862818105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3862818105},{"type":"v1","addr":"172.21.15.175:6836","nonce":3862818105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3615592850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3615592850},{"type":"v1","addr":"172.21.15.175:6837","nonce":3615592850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/832829684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":832829684},{"type":"v1","addr":"172.21.15.47:6834","nonce":832829684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:45:18.296714+0000","modified":"2024-03-23T09:46:24.891678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7005":{"gid":7005,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":16,"addr":"172.21.15.47:6835/4141889683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4141889683},{"type":"v1","addr":"172.21.15.47:6835","nonce":4141889683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-23T09:46:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-23T09:46:59.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.787+0000 7fc3997e0640 1 -- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc38006ac30 msgr2=0x7fc38006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.787+0000 7fc3997e0640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc38006ac30 0x7fc38006d0f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc38c036000 tx=0x7fc38c03d040 comp rx=0 tx=0).stop 2024-03-23T09:46:59.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.787+0000 7fc3997e0640 1 -- 172.21.15.47:0/4174940092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3940cdc40 msgr2=0x7fc3940b2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.787+0000 7fc3997e0640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3940cdc40 0x7fc3940b2910 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc38800d7d0 tx=0x7fc38800dc90 comp rx=0 tx=0).stop 2024-03-23T09:46:59.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.787+0000 7fc3997e0640 1 -- 172.21.15.47:0/4174940092 shutdown_connections 2024-03-23T09:46:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.787+0000 7fc3997e0640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc38006ac30 0x7fc38006d0f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.787+0000 7fc3997e0640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39415ca90 0x7fc3940b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.787+0000 7fc3997e0640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3940cdc40 0x7fc3940b2910 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.787+0000 7fc3997e0640 1 --2- 172.21.15.47:0/4174940092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3940b8350 0x7fc394150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.787+0000 7fc3997e0640 1 -- 172.21.15.47:0/4174940092 >> 172.21.15.47:0/4174940092 conn(0x7fc3940659d0 msgr2=0x7fc3940c8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.787+0000 7fc3997e0640 1 -- 172.21.15.47:0/4174940092 shutdown_connections 2024-03-23T09:46:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.787+0000 7fc3997e0640 1 -- 172.21.15.47:0/4174940092 wait complete. 2024-03-23T09:46:59.798 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:46:59.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.898+0000 7f3b2233e640 1 Processor -- start 2024-03-23T09:46:59.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.898+0000 7f3b2233e640 1 -- start start 2024-03-23T09:46:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.899+0000 7f3b2233e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3b1c13d810 0x7f3b1c0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.899+0000 7f3b2233e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b1c140920 0x7f3b1c070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.899+0000 7f3b2233e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1c13ea70 con 0x7f3b1c0701f0 2024-03-23T09:46:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.899+0000 7f3b2233e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c13dbf0 con 0x7f3b1c140920 2024-03-23T09:46:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.899+0000 7f3b2233e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c0a7670 con 0x7f3b1c13d810 2024-03-23T09:46:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.899+0000 7f3b208db640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b1c140920 0x7f3b1c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.899+0000 7f3b208db640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b1c140920 0x7f3b1c070630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43490/0 (socket says 172.21.15.47:43490) 2024-03-23T09:46:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.899+0000 7f3b208db640 1 -- 172.21.15.47:0/1823154930 learned_addr learned my addr 172.21.15.47:0/1823154930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.899+0000 7f3b1bfff640 1 --2- 172.21.15.47:0/1823154930 >> v2:172.21.15.175:3300/0 conn(0x7f3b1c13d810 0x7f3b1c0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.899+0000 7f3b208db640 1 -- 172.21.15.47:0/1823154930 >> v2:172.21.15.175:3300/0 conn(0x7f3b1c13d810 msgr2=0x7f3b1c0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:46:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.899+0000 7f3b208db640 1 --2- 172.21.15.47:0/1823154930 >> v2:172.21.15.175:3300/0 conn(0x7f3b1c13d810 0x7f3b1c0cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.899+0000 7f3b208db640 1 -- 172.21.15.47:0/1823154930 >> v1:172.21.15.175:6789/0 conn(0x7f3b1c0701f0 legacy=0x7f3b1c140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.899+0000 7f3b208db640 1 -- 172.21.15.47:0/1823154930 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c070b70 con 0x7f3b1c140920 2024-03-23T09:46:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.900+0000 7f3b208db640 1 --2- 172.21.15.47:0/1823154930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b1c140920 0x7f3b1c070630 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c06a9e0 tx=0x7f3b1402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ea41f6bf7b3f37 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.900+0000 7f3b1affd640 1 -- 172.21.15.47:0/1823154930 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1402cbb0 con 0x7f3b1c140920 2024-03-23T09:46:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.900+0000 7f3b1affd640 1 -- 172.21.15.47:0/1823154930 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1402cd10 con 0x7f3b1c140920 2024-03-23T09:46:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.900+0000 7f3b1affd640 1 -- 172.21.15.47:0/1823154930 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14034610 con 0x7f3b1c140920 2024-03-23T09:46:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.901+0000 7f3b2233e640 1 -- 172.21.15.47:0/1823154930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b1c140920 msgr2=0x7f3b1c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.901+0000 7f3b2233e640 1 --2- 172.21.15.47:0/1823154930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b1c140920 0x7f3b1c070630 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c06a9e0 tx=0x7f3b1402c010 comp rx=0 tx=0).stop 2024-03-23T09:46:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.901+0000 7f3b2233e640 1 -- 172.21.15.47:0/1823154930 shutdown_connections 2024-03-23T09:46:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.901+0000 7f3b2233e640 1 --2- 172.21.15.47:0/1823154930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b1c140920 0x7f3b1c070630 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.901+0000 7f3b2233e640 1 --2- 172.21.15.47:0/1823154930 >> v2:172.21.15.175:3300/0 conn(0x7f3b1c13d810 0x7f3b1c0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.901+0000 7f3b2233e640 1 -- 172.21.15.47:0/1823154930 >> 172.21.15.47:0/1823154930 conn(0x7f3b1c06a310 msgr2=0x7f3b1c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:46:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.901+0000 7f3b2233e640 1 -- 172.21.15.47:0/1823154930 shutdown_connections 2024-03-23T09:46:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.901+0000 7f3b2233e640 1 -- 172.21.15.47:0/1823154930 wait complete. 2024-03-23T09:46:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.902+0000 7f3b2233e640 1 Processor -- start 2024-03-23T09:46:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.902+0000 7f3b2233e640 1 -- start start 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b2233e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b1c0701f0 0x7f3b1c13c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b2233e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b1c13d810 0x7f3b1c06dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b2233e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b1c140920 0x7f3b1c143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b2233e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c06e400 con 0x7f3b1c0701f0 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b2233e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c142b90 con 0x7f3b1c13d810 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b2233e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c1425c0 con 0x7f3b1c140920 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b1bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b1c0701f0 0x7f3b1c13c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b1bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b1c0701f0 0x7f3b1c13c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43498/0 (socket says 172.21.15.47:43498) 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b1bfff640 1 -- 172.21.15.47:0/2368254388 learned_addr learned my addr 172.21.15.47:0/2368254388 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b208db640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b1c140920 0x7f3b1c143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b1b7fe640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b1c13d810 0x7f3b1c06dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b1bfff640 1 -- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b1c140920 msgr2=0x7f3b1c143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b1bfff640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b1c140920 0x7f3b1c143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b1bfff640 1 -- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b1c13d810 msgr2=0x7f3b1c06dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:46:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b1bfff640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b1c13d810 0x7f3b1c06dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:46:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.903+0000 7f3b1bfff640 1 -- 172.21.15.47:0/2368254388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b14009b30 con 0x7f3b1c0701f0 2024-03-23T09:46:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.904+0000 7f3b1bfff640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b1c0701f0 0x7f3b1c13c800 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00e9b0 tx=0x7f3b0c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.904+0000 7f3b18ff9640 1 -- 172.21.15.47:0/2368254388 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c009670 con 0x7f3b1c0701f0 2024-03-23T09:46:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.904+0000 7f3b18ff9640 1 -- 172.21.15.47:0/2368254388 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0c00c9a0 con 0x7f3b1c0701f0 2024-03-23T09:46:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.904+0000 7f3b2233e640 1 -- 172.21.15.47:0/2368254388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c06d2e0 con 0x7f3b1c0701f0 2024-03-23T09:46:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.904+0000 7f3b18ff9640 1 -- 172.21.15.47:0/2368254388 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c00cd50 con 0x7f3b1c0701f0 2024-03-23T09:46:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.904+0000 7f3b2233e640 1 -- 172.21.15.47:0/2368254388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c06d8a0 con 0x7f3b1c0701f0 2024-03-23T09:46:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.905+0000 7f3b2233e640 1 -- 172.21.15.47:0/2368254388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c073480 con 0x7f3b1c0701f0 2024-03-23T09:46:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.905+0000 7f3b18ff9640 1 -- 172.21.15.47:0/2368254388 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3b0c013470 con 0x7f3b1c0701f0 2024-03-23T09:46:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.906+0000 7f3b18ff9640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3afc06ac00 0x7f3afc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:46:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.906+0000 7f3b18ff9640 1 -- 172.21.15.47:0/2368254388 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f3b0c0895b0 con 0x7f3b1c0701f0 2024-03-23T09:46:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.906+0000 7f3b1b7fe640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3afc06ac00 0x7f3afc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:46:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.906+0000 7f3b1b7fe640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3afc06ac00 0x7f3afc06d0c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c1f5d40 tx=0x7f3b04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:46:59.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:46:59.910+0000 7f3b18ff9640 1 -- 172.21.15.47:0/2368254388 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3b0c054790 con 0x7f3b1c0701f0 2024-03-23T09:47:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.125+0000 7f3b2233e640 1 -- 172.21.15.47:0/2368254388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b1c066be0 con 0x7f3b1c0701f0 2024-03-23T09:47:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.125+0000 7f3b18ff9640 1 -- 172.21.15.47:0/2368254388 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f3b0c057db0 con 0x7f3b1c0701f0 2024-03-23T09:47:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:00.126 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3862818105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3862818105},{"type":"v1","addr":"172.21.15.175:6836","nonce":3862818105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3615592850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3615592850},{"type":"v1","addr":"172.21.15.175:6837","nonce":3615592850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/832829684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":832829684},{"type":"v1","addr":"172.21.15.47:6834","nonce":832829684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:45:18.296714+0000","modified":"2024-03-23T09:46:24.891678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7005":{"gid":7005,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":16,"addr":"172.21.15.47:6835/4141889683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4141889683},{"type":"v1","addr":"172.21.15.47:6835","nonce":4141889683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-23T09:47:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-23T09:47:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.127+0000 7f3b2233e640 1 -- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3afc06ac00 msgr2=0x7f3afc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.127+0000 7f3b2233e640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3afc06ac00 0x7f3afc06d0c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c1f5d40 tx=0x7f3b04009040 comp rx=0 tx=0).stop 2024-03-23T09:47:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.127+0000 7f3b2233e640 1 -- 172.21.15.47:0/2368254388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b1c0701f0 msgr2=0x7f3b1c13c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.127+0000 7f3b2233e640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b1c0701f0 0x7f3b1c13c800 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00e9b0 tx=0x7f3b0c00ee70 comp rx=0 tx=0).stop 2024-03-23T09:47:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.127+0000 7f3b2233e640 1 -- 172.21.15.47:0/2368254388 shutdown_connections 2024-03-23T09:47:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.127+0000 7f3b2233e640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3afc06ac00 0x7f3afc06d0c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.127+0000 7f3b2233e640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b1c140920 0x7f3b1c143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.127+0000 7f3b2233e640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b1c13d810 0x7f3b1c06dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.127+0000 7f3b2233e640 1 --2- 172.21.15.47:0/2368254388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b1c0701f0 0x7f3b1c13c800 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.127+0000 7f3b2233e640 1 -- 172.21.15.47:0/2368254388 >> 172.21.15.47:0/2368254388 conn(0x7f3b1c06a310 msgr2=0x7f3b1c0b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.127+0000 7f3b2233e640 1 -- 172.21.15.47:0/2368254388 shutdown_connections 2024-03-23T09:47:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.127+0000 7f3b2233e640 1 -- 172.21.15.47:0/2368254388 wait complete. 2024-03-23T09:47:00.138 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:47:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.238+0000 7f7bd0100640 1 Processor -- start 2024-03-23T09:47:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.238+0000 7f7bd0100640 1 -- start start 2024-03-23T09:47:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.239+0000 7f7bd0100640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7bc815ca90 0x7f7bc8160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.239+0000 7f7bd0100640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bc80b8350 0x7f7bc80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.239+0000 7f7bd0100640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc80d11e0 con 0x7f7bc80cdc40 2024-03-23T09:47:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.239+0000 7f7bd0100640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc80ce0d0 con 0x7f7bc815ca90 2024-03-23T09:47:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.239+0000 7f7bd0100640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc80d2680 con 0x7f7bc80b8350 2024-03-23T09:47:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.240+0000 7f7bcd69b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bc80b8350 0x7f7bc80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.240+0000 7f7bcde9c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7bc815ca90 0x7f7bc8160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.240+0000 7f7bcde9c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7bc815ca90 0x7f7bc8160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:46430/0 (socket says 172.21.15.47:46430) 2024-03-23T09:47:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.240+0000 7f7bcd69b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bc80b8350 0x7f7bc80cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:53580/0 (socket says 172.21.15.47:53580) 2024-03-23T09:47:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.240+0000 7f7bcde9c640 1 -- 172.21.15.47:0/2780127001 learned_addr learned my addr 172.21.15.47:0/2780127001 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.240+0000 7f7bce69d640 1 --1- 172.21.15.47:0/2780127001 >> v1:172.21.15.175:6789/0 conn(0x7f7bc80cdc40 0x7f7bc80b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:55188/0 (socket says 172.21.15.47:55188) 2024-03-23T09:47:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.240+0000 7f7bcde9c640 1 -- 172.21.15.47:0/2780127001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bc80b8350 msgr2=0x7f7bc80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.240+0000 7f7bcde9c640 1 --2- 172.21.15.47:0/2780127001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bc80b8350 0x7f7bc80cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.240+0000 7f7bcde9c640 1 -- 172.21.15.47:0/2780127001 >> v1:172.21.15.175:6789/0 conn(0x7f7bc80cdc40 legacy=0x7f7bc80b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.240+0000 7f7bcde9c640 1 -- 172.21.15.47:0/2780127001 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc8160f60 con 0x7f7bc815ca90 2024-03-23T09:47:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.240+0000 7f7bcce9a640 1 -- 172.21.15.47:0/2780127001 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938859132 0 0) 0x7f7bc80d11e0 con 0x7f7bc80cdc40 2024-03-23T09:47:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.240+0000 7f7bcd69b640 1 --2- 172.21.15.47:0/2780127001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bc80b8350 0x7f7bc80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:47:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.241+0000 7f7bcde9c640 1 --2- 172.21.15.47:0/2780127001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bc815ca90 0x7f7bc8160710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009870 tx=0x7f7bc002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6a04d7551fd5394 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.241+0000 7f7bcce9a640 1 -- 172.21.15.47:0/2780127001 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0031640 con 0x7f7bc815ca90 2024-03-23T09:47:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.241+0000 7f7bcce9a640 1 -- 172.21.15.47:0/2780127001 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc00317a0 con 0x7f7bc815ca90 2024-03-23T09:47:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.241+0000 7f7bcce9a640 1 -- 172.21.15.47:0/2780127001 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0031b30 con 0x7f7bc815ca90 2024-03-23T09:47:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.241+0000 7f7bd0100640 1 -- 172.21.15.47:0/2780127001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bc815ca90 msgr2=0x7f7bc8160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.241+0000 7f7bd0100640 1 --2- 172.21.15.47:0/2780127001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bc815ca90 0x7f7bc8160710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009870 tx=0x7f7bc002b8f0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.241+0000 7f7bd0100640 1 -- 172.21.15.47:0/2780127001 shutdown_connections 2024-03-23T09:47:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.241+0000 7f7bd0100640 1 --2- 172.21.15.47:0/2780127001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bc80b8350 0x7f7bc80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.241+0000 7f7bd0100640 1 --2- 172.21.15.47:0/2780127001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bc815ca90 0x7f7bc8160710 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.241+0000 7f7bd0100640 1 -- 172.21.15.47:0/2780127001 >> 172.21.15.47:0/2780127001 conn(0x7f7bc80659d0 msgr2=0x7f7bc8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.242+0000 7f7bd0100640 1 -- 172.21.15.47:0/2780127001 shutdown_connections 2024-03-23T09:47:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.242+0000 7f7bd0100640 1 -- 172.21.15.47:0/2780127001 wait complete. 2024-03-23T09:47:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.243+0000 7f7bd0100640 1 Processor -- start 2024-03-23T09:47:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.243+0000 7f7bd0100640 1 -- start start 2024-03-23T09:47:00.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.243+0000 7f7bd0100640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bc80b8350 0x7f7bc814a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.243+0000 7f7bd0100640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bc80cdc40 0x7f7bc8155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.243+0000 7f7bd0100640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bc815ca90 0x7f7bc80b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.243+0000 7f7bd0100640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc815d190 con 0x7f7bc80cdc40 2024-03-23T09:47:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.243+0000 7f7bd0100640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc8156d70 con 0x7f7bc815ca90 2024-03-23T09:47:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.243+0000 7f7bd0100640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc8156160 con 0x7f7bc80b8350 2024-03-23T09:47:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.243+0000 7f7bcd69b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bc80cdc40 0x7f7bc8155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.243+0000 7f7bcd69b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bc80cdc40 0x7f7bc8155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43502/0 (socket says 172.21.15.47:43502) 2024-03-23T09:47:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.243+0000 7f7bcd69b640 1 -- 172.21.15.47:0/1670019095 learned_addr learned my addr 172.21.15.47:0/1670019095 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.244+0000 7f7bce69d640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bc815ca90 0x7f7bc80b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.244+0000 7f7bcde9c640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bc80b8350 0x7f7bc814a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.244+0000 7f7bcd69b640 1 -- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bc80b8350 msgr2=0x7f7bc814a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.244+0000 7f7bcd69b640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bc80b8350 0x7f7bc814a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.244+0000 7f7bcd69b640 1 -- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bc815ca90 msgr2=0x7f7bc80b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.244+0000 7f7bcd69b640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bc815ca90 0x7f7bc80b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.244+0000 7f7bcd69b640 1 -- 172.21.15.47:0/1670019095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc009400 con 0x7f7bc80cdc40 2024-03-23T09:47:00.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.244+0000 7f7bcd69b640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bc80cdc40 0x7f7bc8155710 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00e960 tx=0x7f7bbc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:00.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.244+0000 7f7bc67fc640 1 -- 172.21.15.47:0/1670019095 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc00ca00 con 0x7f7bc80cdc40 2024-03-23T09:47:00.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.244+0000 7f7bc67fc640 1 -- 172.21.15.47:0/1670019095 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc00cb60 con 0x7f7bc80cdc40 2024-03-23T09:47:00.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.244+0000 7f7bd0100640 1 -- 172.21.15.47:0/1670019095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc00094d0 con 0x7f7bc80cdc40 2024-03-23T09:47:00.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.244+0000 7f7bc67fc640 1 -- 172.21.15.47:0/1670019095 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc013470 con 0x7f7bc80cdc40 2024-03-23T09:47:00.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.244+0000 7f7bd0100640 1 -- 172.21.15.47:0/1670019095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc8156720 con 0x7f7bc80cdc40 2024-03-23T09:47:00.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.245+0000 7f7bd0100640 1 -- 172.21.15.47:0/1670019095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc8072360 con 0x7f7bc80cdc40 2024-03-23T09:47:00.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.246+0000 7f7bc67fc640 1 -- 172.21.15.47:0/1670019095 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7bbc0135d0 con 0x7f7bc80cdc40 2024-03-23T09:47:00.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.246+0000 7f7bc67fc640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7bb406ac00 0x7f7bb406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.246+0000 7f7bc67fc640 1 -- 172.21.15.47:0/1670019095 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7bbc0895d0 con 0x7f7bc80cdc40 2024-03-23T09:47:00.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.246+0000 7f7bcde9c640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7bb406ac00 0x7f7bb406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.247+0000 7f7bcde9c640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7bb406ac00 0x7f7bb406d0c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0036000 tx=0x7f7bc003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:00.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.250+0000 7f7bc67fc640 1 -- 172.21.15.47:0/1670019095 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7bbc0547d0 con 0x7f7bc80cdc40 2024-03-23T09:47:00.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.429+0000 7f7bd0100640 1 -- 172.21.15.47:0/1670019095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bc8066be0 con 0x7f7bc80cdc40 2024-03-23T09:47:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.430+0000 7f7bc67fc640 1 -- 172.21.15.47:0/1670019095 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16324 (secure 0 0 0) 0x7f7bbc057df0 con 0x7f7bc80cdc40 2024-03-23T09:47:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:00.430 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":73,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:45:52.838915+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:45:16.609941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-23T09:45:17.261503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.3","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/372771687":"2024-03-24T09:45:33.740376+0000","172.21.15.47:6833/372771687":"2024-03-24T09:45:33.740376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:47:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.432+0000 7f7bd0100640 1 -- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7bb406ac00 msgr2=0x7f7bb406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.432+0000 7f7bd0100640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7bb406ac00 0x7f7bb406d0c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0036000 tx=0x7f7bc003d040 comp rx=0 tx=0).stop 2024-03-23T09:47:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.432+0000 7f7bd0100640 1 -- 172.21.15.47:0/1670019095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bc80cdc40 msgr2=0x7f7bc8155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.432+0000 7f7bd0100640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bc80cdc40 0x7f7bc8155710 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00e960 tx=0x7f7bbc00ee20 comp rx=0 tx=0).stop 2024-03-23T09:47:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.432+0000 7f7bd0100640 1 -- 172.21.15.47:0/1670019095 shutdown_connections 2024-03-23T09:47:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.432+0000 7f7bd0100640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7bb406ac00 0x7f7bb406d0c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.432+0000 7f7bd0100640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bc815ca90 0x7f7bc80b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.432+0000 7f7bd0100640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bc80cdc40 0x7f7bc8155710 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.432+0000 7f7bd0100640 1 --2- 172.21.15.47:0/1670019095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bc80b8350 0x7f7bc814a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.432+0000 7f7bd0100640 1 -- 172.21.15.47:0/1670019095 >> 172.21.15.47:0/1670019095 conn(0x7f7bc80659d0 msgr2=0x7f7bc80b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.432+0000 7f7bd0100640 1 -- 172.21.15.47:0/1670019095 shutdown_connections 2024-03-23T09:47:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.432+0000 7f7bd0100640 1 -- 172.21.15.47:0/1670019095 wait complete. 2024-03-23T09:47:00.443 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:47:00.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.545+0000 7fc308d00640 1 Processor -- start 2024-03-23T09:47:00.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.545+0000 7fc308d00640 1 -- start start 2024-03-23T09:47:00.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.545+0000 7fc308d00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc30415d0c0 0x7fc30415d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.545+0000 7fc308d00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc3040c9dc0 0x7fc3040ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.545+0000 7fc308d00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3040ca710 con 0x7fc3041571e0 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.545+0000 7fc308d00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30406f190 con 0x7fc3040c9dc0 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.545+0000 7fc308d00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30406eba0 con 0x7fc30415d0c0 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.545+0000 7fc302d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc3040c9dc0 0x7fc3040ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.545+0000 7fc30259c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc3041571e0 0x7fc30406f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52888/0 (socket says 172.21.15.47:52888) 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.545+0000 7fc30259c640 1 -- 172.21.15.47:0/4040399494 learned_addr learned my addr 172.21.15.47:0/4040399494 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.546+0000 7fc2fbfff640 1 --2- 172.21.15.47:0/4040399494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc30415d0c0 0x7fc30415d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.546+0000 7fc302d9d640 1 -- 172.21.15.47:0/4040399494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc30415d0c0 msgr2=0x7fc30415d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.546+0000 7fc302d9d640 1 --2- 172.21.15.47:0/4040399494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc30415d0c0 0x7fc30415d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.546+0000 7fc302d9d640 1 -- 172.21.15.47:0/4040399494 >> v1:172.21.15.47:6789/0 conn(0x7fc3041571e0 legacy=0x7fc30406f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.546+0000 7fc301d9b640 1 -- 172.21.15.47:0/4040399494 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549355249 0 0) 0x7fc3040ca710 con 0x7fc3041571e0 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.546+0000 7fc302d9d640 1 -- 172.21.15.47:0/4040399494 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3041575c0 con 0x7fc3040c9dc0 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.546+0000 7fc2fbfff640 1 --2- 172.21.15.47:0/4040399494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc30415d0c0 0x7fc30415d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.547+0000 7fc302d9d640 1 --2- 172.21.15.47:0/4040399494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3040c9dc0 0x7fc3040ca1d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009d90 tx=0x7fc2fc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1409a15638583cb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.547+0000 7fc301d9b640 1 -- 172.21.15.47:0/4040399494 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc02cc50 con 0x7fc3040c9dc0 2024-03-23T09:47:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.547+0000 7fc301d9b640 1 -- 172.21.15.47:0/4040399494 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc02cdb0 con 0x7fc3040c9dc0 2024-03-23T09:47:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.547+0000 7fc301d9b640 1 -- 172.21.15.47:0/4040399494 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc034660 con 0x7fc3040c9dc0 2024-03-23T09:47:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.547+0000 7fc308d00640 1 -- 172.21.15.47:0/4040399494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3040c9dc0 msgr2=0x7fc3040ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.547+0000 7fc308d00640 1 --2- 172.21.15.47:0/4040399494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3040c9dc0 0x7fc3040ca1d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009d90 tx=0x7fc2fc02bea0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.547+0000 7fc308d00640 1 -- 172.21.15.47:0/4040399494 shutdown_connections 2024-03-23T09:47:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.547+0000 7fc308d00640 1 --2- 172.21.15.47:0/4040399494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3040c9dc0 0x7fc3040ca1d0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.547+0000 7fc308d00640 1 --2- 172.21.15.47:0/4040399494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc30415d0c0 0x7fc30415d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.547+0000 7fc308d00640 1 -- 172.21.15.47:0/4040399494 >> 172.21.15.47:0/4040399494 conn(0x7fc304071400 msgr2=0x7fc304071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.547+0000 7fc308d00640 1 -- 172.21.15.47:0/4040399494 shutdown_connections 2024-03-23T09:47:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.547+0000 7fc308d00640 1 -- 172.21.15.47:0/4040399494 wait complete. 2024-03-23T09:47:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.549+0000 7fc308d00640 1 Processor -- start 2024-03-23T09:47:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.549+0000 7fc308d00640 1 -- start start 2024-03-23T09:47:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.549+0000 7fc308d00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3040c9dc0 0x7fc3041f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.549+0000 7fc308d00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3041571e0 0x7fc304158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.549+0000 7fc308d00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc30415d0c0 0x7fc304157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.549+0000 7fc308d00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc304154740 con 0x7fc3041571e0 2024-03-23T09:47:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.549+0000 7fc308d00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040d0190 con 0x7fc30415d0c0 2024-03-23T09:47:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.549+0000 7fc308d00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3041542b0 con 0x7fc3040c9dc0 2024-03-23T09:47:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.549+0000 7fc2fbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3041571e0 0x7fc304158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.549+0000 7fc2fbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3041571e0 0x7fc304158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43520/0 (socket says 172.21.15.47:43520) 2024-03-23T09:47:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc2fbfff640 1 -- 172.21.15.47:0/2563458227 learned_addr learned my addr 172.21.15.47:0/2563458227 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc302d9d640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc30415d0c0 0x7fc304157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc30259c640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3040c9dc0 0x7fc3041f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc2fbfff640 1 -- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3040c9dc0 msgr2=0x7fc3041f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc2fbfff640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3040c9dc0 0x7fc3041f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc2fbfff640 1 -- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc30415d0c0 msgr2=0x7fc304157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc2fbfff640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc30415d0c0 0x7fc304157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc2fbfff640 1 -- 172.21.15.47:0/2563458227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc009a70 con 0x7fc3041571e0 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc30259c640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3040c9dc0 0x7fc3041f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc302d9d640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc30415d0c0 0x7fc304157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc2fbfff640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3041571e0 0x7fc304158c70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00c7f0 tx=0x7fc2ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc2faffd640 1 -- 172.21.15.47:0/2563458227 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec007820 con 0x7fc3041571e0 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc2faffd640 1 -- 172.21.15.47:0/2563458227 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec007980 con 0x7fc3041571e0 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc2faffd640 1 -- 172.21.15.47:0/2563458227 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec007cf0 con 0x7fc3041571e0 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc308d00640 1 -- 172.21.15.47:0/2563458227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3041591b0 con 0x7fc3041571e0 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.550+0000 7fc308d00640 1 -- 172.21.15.47:0/2563458227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc304153980 con 0x7fc3041571e0 2024-03-23T09:47:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.551+0000 7fc308d00640 1 -- 172.21.15.47:0/2563458227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc304154420 con 0x7fc3041571e0 2024-03-23T09:47:00.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.552+0000 7fc2faffd640 1 -- 172.21.15.47:0/2563458227 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc2ec0113f0 con 0x7fc3041571e0 2024-03-23T09:47:00.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.552+0000 7fc2faffd640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc2e406ac00 0x7fc2e406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.552+0000 7fc2faffd640 1 -- 172.21.15.47:0/2563458227 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc2ec089670 con 0x7fc3041571e0 2024-03-23T09:47:00.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.552+0000 7fc30259c640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc2e406ac00 0x7fc2e406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.553+0000 7fc30259c640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc2e406ac00 0x7fc2e406d0c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4000f60 tx=0x7fc2f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:00.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.556+0000 7fc2faffd640 1 -- 172.21.15.47:0/2563458227 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc2ec054850 con 0x7fc3041571e0 2024-03-23T09:47:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.771+0000 7fc308d00640 1 -- 172.21.15.47:0/2563458227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc3040672f0 con 0x7fc3041571e0 2024-03-23T09:47:00.772 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:47:00.772+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:47:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.779+0000 7fc2faffd640 1 -- 172.21.15.47:0/2563458227 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc2ec057e70 con 0x7fc3041571e0 2024-03-23T09:47:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:47:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.781+0000 7fc308d00640 1 -- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc2e406ac00 msgr2=0x7fc2e406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.781+0000 7fc308d00640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc2e406ac00 0x7fc2e406d0c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4000f60 tx=0x7fc2f4009040 comp rx=0 tx=0).stop 2024-03-23T09:47:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.781+0000 7fc308d00640 1 -- 172.21.15.47:0/2563458227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3041571e0 msgr2=0x7fc304158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.781+0000 7fc308d00640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3041571e0 0x7fc304158c70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00c7f0 tx=0x7fc2ec00ccb0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.781+0000 7fc308d00640 1 -- 172.21.15.47:0/2563458227 shutdown_connections 2024-03-23T09:47:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.781+0000 7fc308d00640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc2e406ac00 0x7fc2e406d0c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.781+0000 7fc308d00640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc30415d0c0 0x7fc304157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.781+0000 7fc308d00640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3041571e0 0x7fc304158c70 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.781+0000 7fc308d00640 1 --2- 172.21.15.47:0/2563458227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3040c9dc0 0x7fc3041f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.781+0000 7fc308d00640 1 -- 172.21.15.47:0/2563458227 >> 172.21.15.47:0/2563458227 conn(0x7fc304071400 msgr2=0x7fc3040717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.781+0000 7fc308d00640 1 -- 172.21.15.47:0/2563458227 shutdown_connections 2024-03-23T09:47:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.781+0000 7fc308d00640 1 -- 172.21.15.47:0/2563458227 wait complete. 2024-03-23T09:47:00.782 INFO:tasks.ceph.mds.c.smithi047.stderr: -4032> 2024-03-23T09:45:39.754+0000 7f4fbb675640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:47:00.792 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T09:47:00.810 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:47:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.896+0000 7f1416981640 1 Processor -- start 2024-03-23T09:47:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.896+0000 7f1416981640 1 -- start start 2024-03-23T09:47:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.896+0000 7f1416981640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f141015cc30 0x7f14100ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.896+0000 7f1416981640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14100cd900 con 0x7f14100cad40 2024-03-23T09:47:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.896+0000 7f1416981640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141015d010 con 0x7f14100cd4b0 2024-03-23T09:47:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.896+0000 7f1416981640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14100b13d0 con 0x7f141015cc30 2024-03-23T09:47:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.896+0000 7f140ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f141015cc30 0x7f14100ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.896+0000 7f140f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f14100cad40 0x7f14100b0610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52906/0 (socket says 172.21.15.47:52906) 2024-03-23T09:47:00.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.896+0000 7f140f7fe640 1 -- 172.21.15.47:0/1798808042 learned_addr learned my addr 172.21.15.47:0/1798808042 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:00.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.897+0000 7f140ffff640 1 -- 172.21.15.47:0/1798808042 >> v1:172.21.15.175:6789/0 conn(0x7f14100cd4b0 legacy=0x7f14100cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:00.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.897+0000 7f140ffff640 1 -- 172.21.15.47:0/1798808042 >> v1:172.21.15.47:6789/0 conn(0x7f14100cad40 legacy=0x7f14100b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.897+0000 7f140ffff640 1 -- 172.21.15.47:0/1798808042 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14100b16b0 con 0x7f141015cc30 2024-03-23T09:47:00.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.897+0000 7f140ffff640 1 --2- 172.21.15.47:0/1798808042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141015cc30 0x7f14100ccf70 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f13f80097a0 tx=0x7f13f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4421ceb77e8ef407 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:00.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.898+0000 7f140effd640 1 -- 172.21.15.47:0/1798808042 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8031830 con 0x7f141015cc30 2024-03-23T09:47:00.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.898+0000 7f140effd640 1 -- 172.21.15.47:0/1798808042 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f8031990 con 0x7f141015cc30 2024-03-23T09:47:00.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.898+0000 7f140effd640 1 -- 172.21.15.47:0/1798808042 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8031d20 con 0x7f141015cc30 2024-03-23T09:47:00.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.898+0000 7f1416981640 1 -- 172.21.15.47:0/1798808042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141015cc30 msgr2=0x7f14100ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.898+0000 7f1416981640 1 --2- 172.21.15.47:0/1798808042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141015cc30 0x7f14100ccf70 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f13f80097a0 tx=0x7f13f802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.898+0000 7f1416981640 1 -- 172.21.15.47:0/1798808042 shutdown_connections 2024-03-23T09:47:00.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.898+0000 7f1416981640 1 --2- 172.21.15.47:0/1798808042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141015cc30 0x7f14100ccf70 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.898+0000 7f1416981640 1 -- 172.21.15.47:0/1798808042 >> 172.21.15.47:0/1798808042 conn(0x7f14100d06f0 msgr2=0x7f14100d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:00.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.898+0000 7f1416981640 1 -- 172.21.15.47:0/1798808042 shutdown_connections 2024-03-23T09:47:00.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.898+0000 7f1416981640 1 -- 172.21.15.47:0/1798808042 wait complete. 2024-03-23T09:47:00.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.899+0000 7f1416981640 1 Processor -- start 2024-03-23T09:47:00.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.900+0000 7f1416981640 1 -- start start 2024-03-23T09:47:00.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.900+0000 7f1416981640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14100cad40 0x7f1410150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.900+0000 7f1416981640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14100cd4b0 0x7f141015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.900+0000 7f1416981640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141015cc30 0x7f141015db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.900+0000 7f1416981640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141015e260 con 0x7f141015cc30 2024-03-23T09:47:00.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.900+0000 7f1416981640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14100c8300 con 0x7f14100cd4b0 2024-03-23T09:47:00.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.900+0000 7f1416981640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141015e050 con 0x7f14100cad40 2024-03-23T09:47:00.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.900+0000 7f1414f1e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141015cc30 0x7f141015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.900+0000 7f1414f1e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141015cc30 0x7f141015db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43550/0 (socket says 172.21.15.47:43550) 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.900+0000 7f1414f1e640 1 -- 172.21.15.47:0/526475325 learned_addr learned my addr 172.21.15.47:0/526475325 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.901+0000 7f140f7fe640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14100cd4b0 0x7f141015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.901+0000 7f140ffff640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14100cad40 0x7f1410150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.901+0000 7f1414f1e640 1 -- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14100cad40 msgr2=0x7f1410150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.901+0000 7f1414f1e640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14100cad40 0x7f1410150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.901+0000 7f1414f1e640 1 -- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14100cd4b0 msgr2=0x7f141015d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.901+0000 7f1414f1e640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14100cd4b0 0x7f141015d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.901+0000 7f1414f1e640 1 -- 172.21.15.47:0/526475325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f8009400 con 0x7f141015cc30 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.901+0000 7f1414f1e640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141015cc30 0x7f141015db10 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f140800c780 tx=0x7f140800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.901+0000 7f140cff9640 1 -- 172.21.15.47:0/526475325 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408003d80 con 0x7f141015cc30 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.901+0000 7f140cff9640 1 -- 172.21.15.47:0/526475325 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140800ce90 con 0x7f141015cc30 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.901+0000 7f1416981640 1 -- 172.21.15.47:0/526475325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14100c8510 con 0x7f141015cc30 2024-03-23T09:47:00.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.901+0000 7f140cff9640 1 -- 172.21.15.47:0/526475325 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14080052b0 con 0x7f141015cc30 2024-03-23T09:47:00.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.901+0000 7f1416981640 1 -- 172.21.15.47:0/526475325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14100c8ad0 con 0x7f141015cc30 2024-03-23T09:47:00.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.902+0000 7f1416981640 1 -- 172.21.15.47:0/526475325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1410066be0 con 0x7f141015cc30 2024-03-23T09:47:00.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.903+0000 7f140cff9640 1 -- 172.21.15.47:0/526475325 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1408005410 con 0x7f141015cc30 2024-03-23T09:47:00.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.903+0000 7f140cff9640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13f006ac00 0x7f13f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:00.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.903+0000 7f140cff9640 1 -- 172.21.15.47:0/526475325 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f1408088c00 con 0x7f141015cc30 2024-03-23T09:47:00.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.903+0000 7f140ffff640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13f006ac00 0x7f13f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:00.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.904+0000 7f140ffff640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13f006ac00 0x7f13f006d0c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f13f8036000 tx=0x7f13f803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:00.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:00.907+0000 7f140cff9640 1 -- 172.21.15.47:0/526475325 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f140808f080 con 0x7f141015cc30 2024-03-23T09:47:01.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.126+0000 7f1416981640 1 -- 172.21.15.47:0/526475325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f14100b1b20 con 0x7f141015cc30 2024-03-23T09:47:01.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.781+0000 7f140cff9640 1 -- 172.21.15.47:0/526475325 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1408017020 con 0x7f141015cc30 2024-03-23T09:47:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.784+0000 7f1416981640 1 -- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13f006ac00 msgr2=0x7f13f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.784+0000 7f1416981640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13f006ac00 0x7f13f006d0c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f13f8036000 tx=0x7f13f803c040 comp rx=0 tx=0).stop 2024-03-23T09:47:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.784+0000 7f1416981640 1 -- 172.21.15.47:0/526475325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141015cc30 msgr2=0x7f141015db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.784+0000 7f1416981640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141015cc30 0x7f141015db10 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f140800c780 tx=0x7f140800cc40 comp rx=0 tx=0).stop 2024-03-23T09:47:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.784+0000 7f1416981640 1 -- 172.21.15.47:0/526475325 shutdown_connections 2024-03-23T09:47:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.784+0000 7f1416981640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13f006ac00 0x7f13f006d0c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.784+0000 7f1416981640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141015cc30 0x7f141015db10 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.784+0000 7f1416981640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14100cd4b0 0x7f141015d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.784+0000 7f1416981640 1 --2- 172.21.15.47:0/526475325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14100cad40 0x7f1410150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.784+0000 7f1416981640 1 -- 172.21.15.47:0/526475325 >> 172.21.15.47:0/526475325 conn(0x7f14100d06f0 msgr2=0x7f141006e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.784+0000 7f1416981640 1 -- 172.21.15.47:0/526475325 shutdown_connections 2024-03-23T09:47:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.784+0000 7f1416981640 1 -- 172.21.15.47:0/526475325 wait complete. 2024-03-23T09:47:01.798 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T09:47:01.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.963+0000 7fd8ed0db640 1 Processor -- start 2024-03-23T09:47:01.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.963+0000 7fd8ed0db640 1 -- start start 2024-03-23T09:47:01.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.964+0000 7fd8ed0db640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd8e815a900 0x7fd8e815ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:01.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.964+0000 7fd8ed0db640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd8e80ce820 0x7fd8e80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:01.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.964+0000 7fd8ed0db640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e80cf1b0 con 0x7fd8e80b1880 2024-03-23T09:47:01.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.964+0000 7fd8ed0db640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e815e400 con 0x7fd8e815a900 2024-03-23T09:47:01.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.964+0000 7fd8ed0db640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e80cb9c0 con 0x7fd8e80ce820 2024-03-23T09:47:01.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.964+0000 7fd8e6d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd8e815a900 0x7fd8e815ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.964+0000 7fd8e6d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd8e815a900 0x7fd8e815ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43564/0 (socket says 172.21.15.47:43564) 2024-03-23T09:47:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.964+0000 7fd8e6d9d640 1 -- 172.21.15.47:0/2210194287 learned_addr learned my addr 172.21.15.47:0/2210194287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.964+0000 7fd8e759e640 1 --2- 172.21.15.47:0/2210194287 >> v2:172.21.15.175:3300/0 conn(0x7fd8e80ce820 0x7fd8e80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.964+0000 7fd8e6d9d640 1 -- 172.21.15.47:0/2210194287 >> v2:172.21.15.175:3300/0 conn(0x7fd8e80ce820 msgr2=0x7fd8e80cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.964+0000 7fd8e6d9d640 1 --2- 172.21.15.47:0/2210194287 >> v2:172.21.15.175:3300/0 conn(0x7fd8e80ce820 0x7fd8e80cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.964+0000 7fd8e6d9d640 1 -- 172.21.15.47:0/2210194287 >> v1:172.21.15.175:6789/0 conn(0x7fd8e80b1880 legacy=0x7fd8e80cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.964+0000 7fd8e6d9d640 1 -- 172.21.15.47:0/2210194287 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e815e670 con 0x7fd8e815a900 2024-03-23T09:47:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.965+0000 7fd8e6d9d640 1 --2- 172.21.15.47:0/2210194287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8e815a900 0x7fd8e815ee70 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8072340 tx=0x7fd8d802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9e248510694e5d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.965+0000 7fd8e659c640 1 -- 172.21.15.47:0/2210194287 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8030860 con 0x7fd8e815a900 2024-03-23T09:47:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.965+0000 7fd8e659c640 1 -- 172.21.15.47:0/2210194287 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d80309c0 con 0x7fd8e815a900 2024-03-23T09:47:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.965+0000 7fd8e659c640 1 -- 172.21.15.47:0/2210194287 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8030d50 con 0x7fd8e815a900 2024-03-23T09:47:01.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.966+0000 7fd8ed0db640 1 -- 172.21.15.47:0/2210194287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8e815a900 msgr2=0x7fd8e815ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:01.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.966+0000 7fd8ed0db640 1 --2- 172.21.15.47:0/2210194287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8e815a900 0x7fd8e815ee70 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8072340 tx=0x7fd8d802b9a0 comp rx=0 tx=0).stop 2024-03-23T09:47:01.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.966+0000 7fd8ed0db640 1 -- 172.21.15.47:0/2210194287 shutdown_connections 2024-03-23T09:47:01.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.966+0000 7fd8ed0db640 1 --2- 172.21.15.47:0/2210194287 >> v2:172.21.15.175:3300/0 conn(0x7fd8e80ce820 0x7fd8e80cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:01.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.966+0000 7fd8ed0db640 1 --2- 172.21.15.47:0/2210194287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8e815a900 0x7fd8e815ee70 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:01.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.966+0000 7fd8ed0db640 1 -- 172.21.15.47:0/2210194287 >> 172.21.15.47:0/2210194287 conn(0x7fd8e8098a00 msgr2=0x7fd8e80b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:01.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.966+0000 7fd8ed0db640 1 -- 172.21.15.47:0/2210194287 shutdown_connections 2024-03-23T09:47:01.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.966+0000 7fd8ed0db640 1 -- 172.21.15.47:0/2210194287 wait complete. 2024-03-23T09:47:01.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.968+0000 7fd8ed0db640 1 Processor -- start 2024-03-23T09:47:01.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.968+0000 7fd8ed0db640 1 -- start start 2024-03-23T09:47:01.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.968+0000 7fd8ed0db640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8e80b1880 0x7fd8e80cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:01.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.968+0000 7fd8ed0db640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e80ce820 0x7fd8e81f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:01.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.968+0000 7fd8ed0db640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e815a900 0x7fd8e80c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:01.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.968+0000 7fd8ed0db640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80c80b0 con 0x7fd8e80b1880 2024-03-23T09:47:01.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.968+0000 7fd8ed0db640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80c8390 con 0x7fd8e815a900 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.968+0000 7fd8ed0db640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80c8670 con 0x7fd8e80ce820 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8e6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8e80b1880 0x7fd8e80cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8e6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8e80b1880 0x7fd8e80cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43578/0 (socket says 172.21.15.47:43578) 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8e6d9d640 1 -- 172.21.15.47:0/162007838 learned_addr learned my addr 172.21.15.47:0/162007838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8e759e640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e815a900 0x7fd8e80c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8dffff640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e80ce820 0x7fd8e81f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8e6d9d640 1 -- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e80ce820 msgr2=0x7fd8e81f3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8e6d9d640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e80ce820 0x7fd8e81f3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8e6d9d640 1 -- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e815a900 msgr2=0x7fd8e80c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8e6d9d640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e815a900 0x7fd8e80c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8e6d9d640 1 -- 172.21.15.47:0/162007838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d80094d0 con 0x7fd8e80b1880 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8e6d9d640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8e80b1880 0x7fd8e80cdf70 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd8d802b9e0 tx=0x7fd8d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8df7fe640 1 -- 172.21.15.47:0/162007838 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8039b60 con 0x7fd8e80b1880 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8df7fe640 1 -- 172.21.15.47:0/162007838 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d8039ce0 con 0x7fd8e80b1880 2024-03-23T09:47:01.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8ed0db640 1 -- 172.21.15.47:0/162007838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e81f3a50 con 0x7fd8e80b1880 2024-03-23T09:47:01.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8df7fe640 1 -- 172.21.15.47:0/162007838 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8043650 con 0x7fd8e80b1880 2024-03-23T09:47:01.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.969+0000 7fd8ed0db640 1 -- 172.21.15.47:0/162007838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e80c7da0 con 0x7fd8e80b1880 2024-03-23T09:47:01.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.970+0000 7fd8ed0db640 1 -- 172.21.15.47:0/162007838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e80672f0 con 0x7fd8e80b1880 2024-03-23T09:47:01.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.971+0000 7fd8df7fe640 1 -- 172.21.15.47:0/162007838 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd8d804c3b0 con 0x7fd8e80b1880 2024-03-23T09:47:01.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.971+0000 7fd8df7fe640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd8c806ac00 0x7fd8c806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:01.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.971+0000 7fd8df7fe640 1 -- 172.21.15.47:0/162007838 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd8d80ab930 con 0x7fd8e80b1880 2024-03-23T09:47:01.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.971+0000 7fd8dffff640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd8c806ac00 0x7fd8c806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:01.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.972+0000 7fd8dffff640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd8c806ac00 0x7fd8c806d0c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80ce4b0 tx=0x7fd8d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:01.976+0000 7fd8df7fe640 1 -- 172.21.15.47:0/162007838 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd8d8076ab0 con 0x7fd8e80b1880 2024-03-23T09:47:02.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.169+0000 7fd8ed0db640 1 -- 172.21.15.47:0/162007838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd8e806cfb0 con 0x7fd8e80b1880 2024-03-23T09:47:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.962+0000 7fd8df7fe640 1 -- 172.21.15.47:0/162007838 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v75) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd8d80340d0 con 0x7fd8e80b1880 2024-03-23T09:47:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-03-23T09:47:02.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.966+0000 7fd8ed0db640 1 -- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd8c806ac00 msgr2=0x7fd8c806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:02.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.966+0000 7fd8ed0db640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd8c806ac00 0x7fd8c806d0c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80ce4b0 tx=0x7fd8d0009040 comp rx=0 tx=0).stop 2024-03-23T09:47:02.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.966+0000 7fd8ed0db640 1 -- 172.21.15.47:0/162007838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8e80b1880 msgr2=0x7fd8e80cdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:02.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.966+0000 7fd8ed0db640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8e80b1880 0x7fd8e80cdf70 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd8d802b9e0 tx=0x7fd8d8004730 comp rx=0 tx=0).stop 2024-03-23T09:47:02.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.966+0000 7fd8ed0db640 1 -- 172.21.15.47:0/162007838 shutdown_connections 2024-03-23T09:47:02.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.966+0000 7fd8ed0db640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd8c806ac00 0x7fd8c806d0c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:02.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.966+0000 7fd8ed0db640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e815a900 0x7fd8e80c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:02.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.966+0000 7fd8ed0db640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e80ce820 0x7fd8e81f3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:02.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.966+0000 7fd8ed0db640 1 --2- 172.21.15.47:0/162007838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8e80b1880 0x7fd8e80cdf70 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:02.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.966+0000 7fd8ed0db640 1 -- 172.21.15.47:0/162007838 >> 172.21.15.47:0/162007838 conn(0x7fd8e8098a00 msgr2=0x7fd8e80cbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:02.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.966+0000 7fd8ed0db640 1 -- 172.21.15.47:0/162007838 shutdown_connections 2024-03-23T09:47:02.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:02.966+0000 7fd8ed0db640 1 -- 172.21.15.47:0/162007838 wait complete. 2024-03-23T09:47:02.980 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T09:47:03.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.092+0000 7f70ed773640 1 Processor -- start 2024-03-23T09:47:03.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.092+0000 7f70ed773640 1 -- start start 2024-03-23T09:47:03.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70ed773640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70e815a860 0x7f70e8161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70ed773640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f70e80cf8a0 0x7f70e815d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70ed773640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e8160bd0 con 0x7f70e815d0f0 2024-03-23T09:47:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70ed773640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e80d2790 con 0x7f70e80cf8a0 2024-03-23T09:47:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70ed773640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e815dbb0 con 0x7f70e815a860 2024-03-23T09:47:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70e77fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f70e80cf8a0 0x7f70e815d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70e67fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f70e815d0f0 0x7f70e80cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52912/0 (socket says 172.21.15.47:52912) 2024-03-23T09:47:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70e77fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f70e80cf8a0 0x7f70e815d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43594/0 (socket says 172.21.15.47:43594) 2024-03-23T09:47:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70e67fc640 1 -- 172.21.15.47:0/2437610148 learned_addr learned my addr 172.21.15.47:0/2437610148 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70e6ffd640 1 --2- 172.21.15.47:0/2437610148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70e815a860 0x7f70e8161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70e77fe640 1 -- 172.21.15.47:0/2437610148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70e815a860 msgr2=0x7f70e8161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:03.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70e77fe640 1 --2- 172.21.15.47:0/2437610148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70e815a860 0x7f70e8161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:03.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70e77fe640 1 -- 172.21.15.47:0/2437610148 >> v1:172.21.15.47:6789/0 conn(0x7f70e815d0f0 legacy=0x7f70e80cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:03.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70e5ffb640 1 -- 172.21.15.47:0/2437610148 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287875563 0 0) 0x7f70e8160bd0 con 0x7f70e815d0f0 2024-03-23T09:47:03.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.093+0000 7f70e77fe640 1 -- 172.21.15.47:0/2437610148 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e815de90 con 0x7f70e80cf8a0 2024-03-23T09:47:03.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.094+0000 7f70e77fe640 1 --2- 172.21.15.47:0/2437610148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70e80cf8a0 0x7f70e815d550 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009d90 tx=0x7f70d802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf34e34495c384c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:03.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.094+0000 7f70e5ffb640 1 -- 172.21.15.47:0/2437610148 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d802cc50 con 0x7f70e80cf8a0 2024-03-23T09:47:03.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.094+0000 7f70e5ffb640 1 -- 172.21.15.47:0/2437610148 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d802cdb0 con 0x7f70e80cf8a0 2024-03-23T09:47:03.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.094+0000 7f70e5ffb640 1 -- 172.21.15.47:0/2437610148 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8034660 con 0x7f70e80cf8a0 2024-03-23T09:47:03.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.095+0000 7f70ed773640 1 -- 172.21.15.47:0/2437610148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70e80cf8a0 msgr2=0x7f70e815d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:03.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.095+0000 7f70ed773640 1 --2- 172.21.15.47:0/2437610148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70e80cf8a0 0x7f70e815d550 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009d90 tx=0x7f70d802bea0 comp rx=0 tx=0).stop 2024-03-23T09:47:03.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.095+0000 7f70ed773640 1 -- 172.21.15.47:0/2437610148 shutdown_connections 2024-03-23T09:47:03.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.095+0000 7f70ed773640 1 --2- 172.21.15.47:0/2437610148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70e80cf8a0 0x7f70e815d550 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:03.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.095+0000 7f70ed773640 1 --2- 172.21.15.47:0/2437610148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70e815a860 0x7f70e8161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:03.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.095+0000 7f70ed773640 1 -- 172.21.15.47:0/2437610148 >> 172.21.15.47:0/2437610148 conn(0x7f70e80659d0 msgr2=0x7f70e8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:03.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.095+0000 7f70ed773640 1 -- 172.21.15.47:0/2437610148 shutdown_connections 2024-03-23T09:47:03.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.095+0000 7f70ed773640 1 -- 172.21.15.47:0/2437610148 wait complete. 2024-03-23T09:47:03.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.096+0000 7f70ed773640 1 Processor -- start 2024-03-23T09:47:03.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.096+0000 7f70ed773640 1 -- start start 2024-03-23T09:47:03.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70ed773640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70e80cf8a0 0x7f70e80cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:03.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70ed773640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70e815a860 0x7f70e814ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:03.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70ed773640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70e815d0f0 0x7f70e80b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:03.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70ed773640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e80c6000 con 0x7f70e815a860 2024-03-23T09:47:03.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70ed773640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e80ccbf0 con 0x7f70e80cf8a0 2024-03-23T09:47:03.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70ed773640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e80cc910 con 0x7f70e815d0f0 2024-03-23T09:47:03.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70e67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70e815a860 0x7f70e814ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:03.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70e67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70e815a860 0x7f70e814ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43610/0 (socket says 172.21.15.47:43610) 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70e67fc640 1 -- 172.21.15.47:0/1475209855 learned_addr learned my addr 172.21.15.47:0/1475209855 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70e6ffd640 1 --2- 172.21.15.47:0/1475209855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70e80cf8a0 0x7f70e80cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70e67fc640 1 -- 172.21.15.47:0/1475209855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70e815d0f0 msgr2=0x7f70e80b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70e67fc640 1 --2- 172.21.15.47:0/1475209855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70e815d0f0 0x7f70e80b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70e67fc640 1 -- 172.21.15.47:0/1475209855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70e80cf8a0 msgr2=0x7f70e80cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70e67fc640 1 --2- 172.21.15.47:0/1475209855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70e80cf8a0 0x7f70e80cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70e67fc640 1 -- 172.21.15.47:0/1475209855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d8009a70 con 0x7f70e815a860 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.097+0000 7f70e67fc640 1 --2- 172.21.15.47:0/1475209855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70e815a860 0x7f70e814ed90 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00c430 tx=0x7f70dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.098+0000 7f70cf7fe640 1 -- 172.21.15.47:0/1475209855 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0074e0 con 0x7f70e815a860 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.098+0000 7f70cf7fe640 1 -- 172.21.15.47:0/1475209855 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc007660 con 0x7f70e815a860 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.098+0000 7f70ed773640 1 -- 172.21.15.47:0/1475209855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e80c7030 con 0x7f70e815a860 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.098+0000 7f70cf7fe640 1 -- 172.21.15.47:0/1475209855 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc004a20 con 0x7f70e815a860 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.098+0000 7f70ed773640 1 -- 172.21.15.47:0/1475209855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e80c75f0 con 0x7f70e815a860 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.099+0000 7f70ed773640 1 -- 172.21.15.47:0/1475209855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e8072360 con 0x7f70e815a860 2024-03-23T09:47:03.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.099+0000 7f70cf7fe640 1 -- 172.21.15.47:0/1475209855 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f70dc004b80 con 0x7f70e815a860 2024-03-23T09:47:03.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.100+0000 7f70cf7fe640 1 --2- 172.21.15.47:0/1475209855 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f70d406ac00 0x7f70d406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:03.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.100+0000 7f70cf7fe640 1 -- 172.21.15.47:0/1475209855 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f70dc089090 con 0x7f70e815a860 2024-03-23T09:47:03.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.100+0000 7f70e6ffd640 1 --2- 172.21.15.47:0/1475209855 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f70d406ac00 0x7f70d406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:03.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.101+0000 7f70e6ffd640 1 --2- 172.21.15.47:0/1475209855 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f70d406ac00 0x7f70d406d0c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f70e0006fd0 tx=0x7f70e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:03.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.104+0000 7f70cf7fe640 1 -- 172.21.15.47:0/1475209855 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f70dc054340 con 0x7f70e815a860 2024-03-23T09:47:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.282+0000 7f70ed773640 1 -- 172.21.15.47:0/1475209855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f70e80c7810 con 0x7f70e815a860 2024-03-23T09:47:03.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.965+0000 7f70cf7fe640 1 -- 172.21.15.47:0/1475209855 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v76) v1 ==== 169+0+0 (secure 0 0 0) 0x7f70dc057960 con 0x7f70e815a860 2024-03-23T09:47:03.967 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-03-23T09:47:03.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.971+0000 7f70ed773640 1 -- 172.21.15.47:0/1475209855 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f70d406ac00 msgr2=0x7f70d406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:03.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.971+0000 7f70ed773640 1 --2- 172.21.15.47:0/1475209855 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f70d406ac00 0x7f70d406d0c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f70e0006fd0 tx=0x7f70e0008040 comp rx=0 tx=0).stop 2024-03-23T09:47:03.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.971+0000 7f70ed773640 1 -- 172.21.15.47:0/1475209855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70e815a860 msgr2=0x7f70e814ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:03.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.971+0000 7f70ed773640 1 --2- 172.21.15.47:0/1475209855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70e815a860 0x7f70e814ed90 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00c430 tx=0x7f70dc00c8f0 comp rx=0 tx=0).stop 2024-03-23T09:47:03.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.972+0000 7f70ed773640 1 -- 172.21.15.47:0/1475209855 shutdown_connections 2024-03-23T09:47:03.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.972+0000 7f70ed773640 1 --2- 172.21.15.47:0/1475209855 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f70d406ac00 0x7f70d406d0c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:03.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.972+0000 7f70ed773640 1 --2- 172.21.15.47:0/1475209855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70e815d0f0 0x7f70e80b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:03.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.972+0000 7f70ed773640 1 --2- 172.21.15.47:0/1475209855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70e815a860 0x7f70e814ed90 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:03.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.972+0000 7f70ed773640 1 --2- 172.21.15.47:0/1475209855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70e80cf8a0 0x7f70e80cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:03.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.972+0000 7f70ed773640 1 -- 172.21.15.47:0/1475209855 >> 172.21.15.47:0/1475209855 conn(0x7f70e80659d0 msgr2=0x7f70e80d1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:03.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.972+0000 7f70ed773640 1 -- 172.21.15.47:0/1475209855 shutdown_connections 2024-03-23T09:47:03.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:03.972+0000 7f70ed773640 1 -- 172.21.15.47:0/1475209855 wait complete. 2024-03-23T09:47:03.985 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-23T09:47:04.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.100+0000 7f1ebaa95640 1 Processor -- start 2024-03-23T09:47:04.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.100+0000 7f1ebaa95640 1 -- start start 2024-03-23T09:47:04.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.101+0000 7f1ebaa95640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40cde10 0x7f1eb4160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:04.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.101+0000 7f1ebaa95640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1eb40b8350 0x7f1eb40d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:04.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.101+0000 7f1ebaa95640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb4161630 con 0x7f1eb415c990 2024-03-23T09:47:04.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.101+0000 7f1ebaa95640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb40d1030 con 0x7f1eb40b8350 2024-03-23T09:47:04.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.101+0000 7f1ebaa95640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40cfe10 con 0x7f1eb40cde10 2024-03-23T09:47:04.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.101+0000 7f1eb3fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40cde10 0x7f1eb4160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:04.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.101+0000 7f1eb9032640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1eb40b8350 0x7f1eb40d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:04.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.101+0000 7f1eb3fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40cde10 0x7f1eb4160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:53678/0 (socket says 172.21.15.47:53678) 2024-03-23T09:47:04.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.101+0000 7f1eb9032640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1eb40b8350 0x7f1eb40d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:46526/0 (socket says 172.21.15.47:46526) 2024-03-23T09:47:04.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.101+0000 7f1eb3fff640 1 -- 172.21.15.47:0/827589473 learned_addr learned my addr 172.21.15.47:0/827589473 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:04.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.101+0000 7f1eb8831640 1 --1- 172.21.15.47:0/827589473 >> v1:172.21.15.175:6789/0 conn(0x7f1eb415c990 0x7f1eb40cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:55224/0 (socket says 172.21.15.47:55224) 2024-03-23T09:47:04.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.102+0000 7f1eb3fff640 1 -- 172.21.15.47:0/827589473 >> v2:172.21.15.175:3300/0 conn(0x7f1eb40b8350 msgr2=0x7f1eb40d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:04.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.102+0000 7f1eb3fff640 1 --2- 172.21.15.47:0/827589473 >> v2:172.21.15.175:3300/0 conn(0x7f1eb40b8350 0x7f1eb40d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.102+0000 7f1eb3fff640 1 -- 172.21.15.47:0/827589473 >> v1:172.21.15.175:6789/0 conn(0x7f1eb415c990 legacy=0x7f1eb40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:04.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.102+0000 7f1eb3fff640 1 -- 172.21.15.47:0/827589473 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb40b5b60 con 0x7f1eb40cde10 2024-03-23T09:47:04.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.102+0000 7f1eb3fff640 1 --2- 172.21.15.47:0/827589473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40cde10 0x7f1eb4160690 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009870 tx=0x7f1ea002b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e2d36473bda62e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:04.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.103+0000 7f1eb37fe640 1 -- 172.21.15.47:0/827589473 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00317c0 con 0x7f1eb40cde10 2024-03-23T09:47:04.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.103+0000 7f1eb37fe640 1 -- 172.21.15.47:0/827589473 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea0031920 con 0x7f1eb40cde10 2024-03-23T09:47:04.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.103+0000 7f1eb37fe640 1 -- 172.21.15.47:0/827589473 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0031cb0 con 0x7f1eb40cde10 2024-03-23T09:47:04.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.103+0000 7f1ebaa95640 1 -- 172.21.15.47:0/827589473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40cde10 msgr2=0x7f1eb4160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:04.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.103+0000 7f1ebaa95640 1 --2- 172.21.15.47:0/827589473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40cde10 0x7f1eb4160690 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009870 tx=0x7f1ea002b860 comp rx=0 tx=0).stop 2024-03-23T09:47:04.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.103+0000 7f1ebaa95640 1 -- 172.21.15.47:0/827589473 shutdown_connections 2024-03-23T09:47:04.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.103+0000 7f1ebaa95640 1 --2- 172.21.15.47:0/827589473 >> v2:172.21.15.175:3300/0 conn(0x7f1eb40b8350 0x7f1eb40d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.103+0000 7f1ebaa95640 1 --2- 172.21.15.47:0/827589473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40cde10 0x7f1eb4160690 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.103+0000 7f1ebaa95640 1 -- 172.21.15.47:0/827589473 >> 172.21.15.47:0/827589473 conn(0x7f1eb40659d0 msgr2=0x7f1eb4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:04.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.103+0000 7f1ebaa95640 1 -- 172.21.15.47:0/827589473 shutdown_connections 2024-03-23T09:47:04.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.103+0000 7f1ebaa95640 1 -- 172.21.15.47:0/827589473 wait complete. 2024-03-23T09:47:04.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.104+0000 7f1ebaa95640 1 Processor -- start 2024-03-23T09:47:04.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.104+0000 7f1ebaa95640 1 -- start start 2024-03-23T09:47:04.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.105+0000 7f1ebaa95640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb40b8350 0x7f1eb41f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:04.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.105+0000 7f1ebaa95640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40cde10 0x7f1eb414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:04.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.105+0000 7f1ebaa95640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb415c990 0x7f1eb40b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:04.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.105+0000 7f1ebaa95640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40c7ad0 con 0x7f1eb40b8350 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.105+0000 7f1ebaa95640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb414c200 con 0x7f1eb415c990 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.105+0000 7f1ebaa95640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40c72c0 con 0x7f1eb40cde10 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.105+0000 7f1eb8831640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb40b8350 0x7f1eb41f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.105+0000 7f1eb8831640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb40b8350 0x7f1eb41f25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43616/0 (socket says 172.21.15.47:43616) 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.105+0000 7f1eb8831640 1 -- 172.21.15.47:0/1269022934 learned_addr learned my addr 172.21.15.47:0/1269022934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.105+0000 7f1eb3fff640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40cde10 0x7f1eb414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.106+0000 7f1eb9032640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb415c990 0x7f1eb40b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.106+0000 7f1eb8831640 1 -- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40cde10 msgr2=0x7f1eb414bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.106+0000 7f1eb8831640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40cde10 0x7f1eb414bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.106+0000 7f1eb8831640 1 -- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb415c990 msgr2=0x7f1eb40b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.106+0000 7f1eb8831640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb415c990 0x7f1eb40b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.106+0000 7f1eb8831640 1 -- 172.21.15.47:0/1269022934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea00094d0 con 0x7f1eb40b8350 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.106+0000 7f1eb8831640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb40b8350 0x7f1eb41f25b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800c4d0 tx=0x7f1ea800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.106+0000 7f1eb17fa640 1 -- 172.21.15.47:0/1269022934 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8007480 con 0x7f1eb40b8350 2024-03-23T09:47:04.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.106+0000 7f1eb17fa640 1 -- 172.21.15.47:0/1269022934 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea80075e0 con 0x7f1eb40b8350 2024-03-23T09:47:04.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.106+0000 7f1ebaa95640 1 -- 172.21.15.47:0/1269022934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb40c7d70 con 0x7f1eb40b8350 2024-03-23T09:47:04.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.106+0000 7f1eb17fa640 1 -- 172.21.15.47:0/1269022934 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea80049e0 con 0x7f1eb40b8350 2024-03-23T09:47:04.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.106+0000 7f1ebaa95640 1 -- 172.21.15.47:0/1269022934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb40c75a0 con 0x7f1eb40b8350 2024-03-23T09:47:04.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.107+0000 7f1ebaa95640 1 -- 172.21.15.47:0/1269022934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb4072360 con 0x7f1eb40b8350 2024-03-23T09:47:04.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.108+0000 7f1eb17fa640 1 -- 172.21.15.47:0/1269022934 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1ea8004b40 con 0x7f1eb40b8350 2024-03-23T09:47:04.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.108+0000 7f1eb17fa640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1e9806ac00 0x7f1e9806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:04.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.108+0000 7f1eb17fa640 1 -- 172.21.15.47:0/1269022934 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f1ea8088130 con 0x7f1eb40b8350 2024-03-23T09:47:04.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.108+0000 7f1eb3fff640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1e9806ac00 0x7f1e9806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:04.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.109+0000 7f1eb3fff640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1e9806ac00 0x7f1e9806d0c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0036000 tx=0x7f1ea003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:04.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.112+0000 7f1eb17fa640 1 -- 172.21.15.47:0/1269022934 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1ea8053670 con 0x7f1eb40b8350 2024-03-23T09:47:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.290+0000 7f1ebaa95640 1 -- 172.21.15.47:0/1269022934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1eb40c8290 con 0x7f1eb40b8350 2024-03-23T09:47:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.294+0000 7f1eb17fa640 1 -- 172.21.15.47:0/1269022934 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1eb40c8290 con 0x7f1eb40b8350 2024-03-23T09:47:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.296+0000 7f1ebaa95640 1 -- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1e9806ac00 msgr2=0x7f1e9806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.296+0000 7f1ebaa95640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1e9806ac00 0x7f1e9806d0c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0036000 tx=0x7f1ea003d040 comp rx=0 tx=0).stop 2024-03-23T09:47:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.296+0000 7f1ebaa95640 1 -- 172.21.15.47:0/1269022934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb40b8350 msgr2=0x7f1eb41f25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.296+0000 7f1ebaa95640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb40b8350 0x7f1eb41f25b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800c4d0 tx=0x7f1ea800c990 comp rx=0 tx=0).stop 2024-03-23T09:47:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.296+0000 7f1ebaa95640 1 -- 172.21.15.47:0/1269022934 shutdown_connections 2024-03-23T09:47:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.296+0000 7f1ebaa95640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1e9806ac00 0x7f1e9806d0c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.296+0000 7f1ebaa95640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb415c990 0x7f1eb40b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.296+0000 7f1ebaa95640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40cde10 0x7f1eb414bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.296+0000 7f1ebaa95640 1 --2- 172.21.15.47:0/1269022934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb40b8350 0x7f1eb41f25b0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.296+0000 7f1ebaa95640 1 -- 172.21.15.47:0/1269022934 >> 172.21.15.47:0/1269022934 conn(0x7f1eb40659d0 msgr2=0x7f1eb40b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.296+0000 7f1ebaa95640 1 -- 172.21.15.47:0/1269022934 shutdown_connections 2024-03-23T09:47:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.296+0000 7f1ebaa95640 1 -- 172.21.15.47:0/1269022934 wait complete. 2024-03-23T09:47:04.307 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-03-23T09:47:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.408+0000 7f3c25bc4640 1 Processor -- start 2024-03-23T09:47:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.408+0000 7f3c25bc4640 1 -- start start 2024-03-23T09:47:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c25bc4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3c200ce290 0x7f3c200d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c25bc4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c200b0cb0 0x7f3c2015e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c25bc4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c200b1140 con 0x7f3c200cdeb0 2024-03-23T09:47:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c25bc4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c200b0510 con 0x7f3c200ce290 2024-03-23T09:47:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c25bc4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200aff50 con 0x7f3c200b0cb0 2024-03-23T09:47:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c17fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3c200ce290 0x7f3c200d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c1f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3c200cdeb0 0x7f3c2015da50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52920/0 (socket says 172.21.15.47:52920) 2024-03-23T09:47:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c1f7fe640 1 -- 172.21.15.47:0/1145673467 learned_addr learned my addr 172.21.15.47:0/1145673467 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c1ffff640 1 --2- 172.21.15.47:0/1145673467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c200b0cb0 0x7f3c2015e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:04.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c17fff640 1 -- 172.21.15.47:0/1145673467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c200b0cb0 msgr2=0x7f3c2015e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:04.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c17fff640 1 --2- 172.21.15.47:0/1145673467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c200b0cb0 0x7f3c2015e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c17fff640 1 -- 172.21.15.47:0/1145673467 >> v1:172.21.15.47:6789/0 conn(0x7f3c200cdeb0 legacy=0x7f3c2015da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:04.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c17fff640 1 -- 172.21.15.47:0/1145673467 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c200b0230 con 0x7f3c200ce290 2024-03-23T09:47:04.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.409+0000 7f3c1effd640 1 -- 172.21.15.47:0/1145673467 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824234417 0 0) 0x7f3c200b1140 con 0x7f3c200cdeb0 2024-03-23T09:47:04.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.410+0000 7f3c17fff640 1 --2- 172.21.15.47:0/1145673467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c200ce290 0x7f3c200d01a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800ba40 tx=0x7f3c0802e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809d367ba05541ab server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:04.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.410+0000 7f3c1effd640 1 -- 172.21.15.47:0/1145673467 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0802ee30 con 0x7f3c200ce290 2024-03-23T09:47:04.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.410+0000 7f3c1effd640 1 -- 172.21.15.47:0/1145673467 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c08036440 con 0x7f3c200ce290 2024-03-23T09:47:04.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.410+0000 7f3c1effd640 1 -- 172.21.15.47:0/1145673467 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c080367d0 con 0x7f3c200ce290 2024-03-23T09:47:04.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.410+0000 7f3c25bc4640 1 -- 172.21.15.47:0/1145673467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c200ce290 msgr2=0x7f3c200d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:04.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.410+0000 7f3c25bc4640 1 --2- 172.21.15.47:0/1145673467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c200ce290 0x7f3c200d01a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800ba40 tx=0x7f3c0802e080 comp rx=0 tx=0).stop 2024-03-23T09:47:04.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.411+0000 7f3c25bc4640 1 -- 172.21.15.47:0/1145673467 shutdown_connections 2024-03-23T09:47:04.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.411+0000 7f3c25bc4640 1 --2- 172.21.15.47:0/1145673467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c200b0cb0 0x7f3c2015e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.411+0000 7f3c25bc4640 1 --2- 172.21.15.47:0/1145673467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c200ce290 0x7f3c200d01a0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.411+0000 7f3c25bc4640 1 -- 172.21.15.47:0/1145673467 >> 172.21.15.47:0/1145673467 conn(0x7f3c20098a00 msgr2=0x7f3c200cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:04.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.411+0000 7f3c25bc4640 1 -- 172.21.15.47:0/1145673467 shutdown_connections 2024-03-23T09:47:04.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.411+0000 7f3c25bc4640 1 -- 172.21.15.47:0/1145673467 wait complete. 2024-03-23T09:47:04.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.412+0000 7f3c25bc4640 1 Processor -- start 2024-03-23T09:47:04.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.412+0000 7f3c25bc4640 1 -- start start 2024-03-23T09:47:04.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.412+0000 7f3c25bc4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c200b0cb0 0x7f3c201536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:04.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.412+0000 7f3c25bc4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c200cdeb0 0x7f3c20155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:04.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.412+0000 7f3c25bc4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c200ce290 0x7f3c200ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:04.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c25bc4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200b4310 con 0x7f3c200ce290 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c25bc4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200b4f60 con 0x7f3c200b0cb0 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c25bc4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200b5240 con 0x7f3c200cdeb0 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c1ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c200ce290 0x7f3c200ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c1ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c200ce290 0x7f3c200ad9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43636/0 (socket says 172.21.15.47:43636) 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c1ffff640 1 -- 172.21.15.47:0/3546837954 learned_addr learned my addr 172.21.15.47:0/3546837954 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c17fff640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c200cdeb0 0x7f3c20155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c1f7fe640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c200b0cb0 0x7f3c201536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c1ffff640 1 -- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c200cdeb0 msgr2=0x7f3c20155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c1ffff640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c200cdeb0 0x7f3c20155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c1ffff640 1 -- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c200b0cb0 msgr2=0x7f3c201536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c1ffff640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c200b0cb0 0x7f3c201536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c1ffff640 1 -- 172.21.15.47:0/3546837954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0800b6a0 con 0x7f3c200ce290 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c1ffff640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c200ce290 0x7f3c200ad9e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800efc0 tx=0x7f3c1800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c1cff9640 1 -- 172.21.15.47:0/3546837954 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18014480 con 0x7f3c200ce290 2024-03-23T09:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c1cff9640 1 -- 172.21.15.47:0/3546837954 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1800f040 con 0x7f3c200ce290 2024-03-23T09:47:04.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c25bc4640 1 -- 172.21.15.47:0/3546837954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c200b5520 con 0x7f3c200ce290 2024-03-23T09:47:04.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c1cff9640 1 -- 172.21.15.47:0/3546837954 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18014820 con 0x7f3c200ce290 2024-03-23T09:47:04.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.413+0000 7f3c25bc4640 1 -- 172.21.15.47:0/3546837954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c20157360 con 0x7f3c200ce290 2024-03-23T09:47:04.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.414+0000 7f3c25bc4640 1 -- 172.21.15.47:0/3546837954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c201578f0 con 0x7f3c200ce290 2024-03-23T09:47:04.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.415+0000 7f3c1cff9640 1 -- 172.21.15.47:0/3546837954 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3c18014980 con 0x7f3c200ce290 2024-03-23T09:47:04.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.415+0000 7f3c1cff9640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3c0006ac00 0x7f3c0006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:04.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.415+0000 7f3c1cff9640 1 -- 172.21.15.47:0/3546837954 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3c18089d40 con 0x7f3c200ce290 2024-03-23T09:47:04.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.415+0000 7f3c1f7fe640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3c0006ac00 0x7f3c0006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:04.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.416+0000 7f3c1f7fe640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3c0006ac00 0x7f3c0006d0c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3c10000f60 tx=0x7f3c10007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:04.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.419+0000 7f3c1cff9640 1 -- 172.21.15.47:0/3546837954 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3c201578f0 con 0x7f3c200ce290 2024-03-23T09:47:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.668+0000 7f3c25bc4640 1 -- 172.21.15.47:0/3546837954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7f3c201578f0 con 0x7f3c200ce290 2024-03-23T09:47:04.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.970+0000 7f3c1cff9640 1 -- 172.21.15.47:0/3546837954 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v282) v1 ==== 166+0+0 (secure 0 0 0) 0x7f3c201578f0 con 0x7f3c200ce290 2024-03-23T09:47:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.972+0000 7f3c25bc4640 1 -- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3c0006ac00 msgr2=0x7f3c0006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.972+0000 7f3c25bc4640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3c0006ac00 0x7f3c0006d0c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3c10000f60 tx=0x7f3c10007d50 comp rx=0 tx=0).stop 2024-03-23T09:47:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.972+0000 7f3c25bc4640 1 -- 172.21.15.47:0/3546837954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c200ce290 msgr2=0x7f3c200ad9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.972+0000 7f3c25bc4640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c200ce290 0x7f3c200ad9e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800efc0 tx=0x7f3c1800c490 comp rx=0 tx=0).stop 2024-03-23T09:47:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.972+0000 7f3c25bc4640 1 -- 172.21.15.47:0/3546837954 shutdown_connections 2024-03-23T09:47:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.972+0000 7f3c25bc4640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3c0006ac00 0x7f3c0006d0c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.972+0000 7f3c25bc4640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c200ce290 0x7f3c200ad9e0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.972+0000 7f3c25bc4640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c200cdeb0 0x7f3c20155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.972+0000 7f3c25bc4640 1 --2- 172.21.15.47:0/3546837954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c200b0cb0 0x7f3c201536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.972+0000 7f3c25bc4640 1 -- 172.21.15.47:0/3546837954 >> 172.21.15.47:0/3546837954 conn(0x7f3c20098a00 msgr2=0x7f3c2006d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:04.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.972+0000 7f3c25bc4640 1 -- 172.21.15.47:0/3546837954 shutdown_connections 2024-03-23T09:47:04.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:04.972+0000 7f3c25bc4640 1 -- 172.21.15.47:0/3546837954 wait complete. 2024-03-23T09:47:04.984 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-23T09:47:04.984 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-23T09:47:04.984 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-23T09:47:04.984 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-03-23T09:47:05.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.087+0000 7fd04b72e640 1 Processor -- start 2024-03-23T09:47:05.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.087+0000 7fd04b72e640 1 -- start start 2024-03-23T09:47:05.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd04b72e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd04415ca90 0x7fd044160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:05.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd04b72e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0440b8350 0x7fd0440cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd04b72e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0440d11e0 con 0x7fd0440cdc40 2024-03-23T09:47:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd04b72e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0440ce0d0 con 0x7fd04415ca90 2024-03-23T09:47:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd04b72e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0440d2680 con 0x7fd0440b8350 2024-03-23T09:47:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd049ccb640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd0440cdc40 0x7fd0440b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48682/0 (socket says 172.21.15.47:48682) 2024-03-23T09:47:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd049ccb640 1 -- 172.21.15.47:0/2370750306 learned_addr learned my addr 172.21.15.47:0/2370750306 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408500856 0 0) 0x7fd0440d11e0 con 0x7fd0440cdc40 2024-03-23T09:47:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd048cc9640 1 --2- 172.21.15.47:0/2370750306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0440b8350 0x7fd0440cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd0494ca640 1 --2- 172.21.15.47:0/2370750306 >> v2:172.21.15.175:3300/0 conn(0x7fd04415ca90 0x7fd044160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd030003520 con 0x7fd0440cdc40 2024-03-23T09:47:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3660169009 0 0) 0x7fd030003520 con 0x7fd0440cdc40 2024-03-23T09:47:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0440b8350 msgr2=0x7fd0440cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 --2- 172.21.15.47:0/2370750306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0440b8350 0x7fd0440cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 >> v2:172.21.15.175:3300/0 conn(0x7fd04415ca90 msgr2=0x7fd044160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 --2- 172.21.15.47:0/2370750306 >> v2:172.21.15.175:3300/0 conn(0x7fd04415ca90 0x7fd044160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd044160f60 con 0x7fd0440cdc40 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fd03c0019f0 con 0x7fd0440cdc40 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 >> v1:172.21.15.47:6789/0 conn(0x7fd0440cdc40 legacy=0x7fd0440b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 --2- 172.21.15.47:0/2370750306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0440b8350 0x7fd030003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 --2- 172.21.15.47:0/2370750306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd030003ef0 0x7fd0300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 --2- 172.21.15.47:0/2370750306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd030004d00 0x7fd030005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0440ce0d0 con 0x7fd030004d00 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0440d2680 con 0x7fd030003ef0 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.088+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03000b9d0 con 0x7fd0440b8350 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.089+0000 7fd049ccb640 1 --2- 172.21.15.47:0/2370750306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd030004d00 0x7fd030005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.089+0000 7fd0494ca640 1 --2- 172.21.15.47:0/2370750306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0440b8350 0x7fd030003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.089+0000 7fd048cc9640 1 --2- 172.21.15.47:0/2370750306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd030003ef0 0x7fd0300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.089+0000 7fd049ccb640 1 -- 172.21.15.47:0/2370750306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0440b8350 msgr2=0x7fd030003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.089+0000 7fd049ccb640 1 --2- 172.21.15.47:0/2370750306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0440b8350 0x7fd030003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.089+0000 7fd049ccb640 1 -- 172.21.15.47:0/2370750306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd030003ef0 msgr2=0x7fd0300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.089+0000 7fd049ccb640 1 --2- 172.21.15.47:0/2370750306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd030003ef0 0x7fd0300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.089+0000 7fd049ccb640 1 -- 172.21.15.47:0/2370750306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd030003520 con 0x7fd030004d00 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.089+0000 7fd048cc9640 1 --2- 172.21.15.47:0/2370750306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd030003ef0 0x7fd0300047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.090+0000 7fd049ccb640 1 --2- 172.21.15.47:0/2370750306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd030004d00 0x7fd030005180 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00a890 tx=0x7fd0300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2af33488e0664 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.090+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c034630 con 0x7fd030004d00 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.090+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c02f070 con 0x7fd030004d00 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.090+0000 7fd03bfff640 1 -- 172.21.15.47:0/2370750306 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c034a30 con 0x7fd030004d00 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.090+0000 7fd04b72e640 1 -- 172.21.15.47:0/2370750306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd030004d00 msgr2=0x7fd030005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.090+0000 7fd04b72e640 1 --2- 172.21.15.47:0/2370750306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd030004d00 0x7fd030005180 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00a890 tx=0x7fd0300057f0 comp rx=0 tx=0).stop 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.090+0000 7fd049ccb640 1 -- 172.21.15.47:0/2370750306 reap_dead start 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.090+0000 7fd04b72e640 1 -- 172.21.15.47:0/2370750306 shutdown_connections 2024-03-23T09:47:05.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.090+0000 7fd04b72e640 1 -- 172.21.15.47:0/2370750306 >> 172.21.15.47:0/2370750306 conn(0x7fd0440659d0 msgr2=0x7fd044065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:05.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.090+0000 7fd04b72e640 1 -- 172.21.15.47:0/2370750306 shutdown_connections 2024-03-23T09:47:05.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.090+0000 7fd04b72e640 1 -- 172.21.15.47:0/2370750306 wait complete. 2024-03-23T09:47:05.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.091+0000 7fd04b72e640 1 Processor -- start 2024-03-23T09:47:05.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.092+0000 7fd04b72e640 1 -- start start 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.092+0000 7fd04b72e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd030004d00 0x7fd044150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.092+0000 7fd04b72e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd044153040 0x7fd0441534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.092+0000 7fd04b72e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0440b6170 0x7fd0440b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.092+0000 7fd04b72e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0440b6b30 con 0x7fd044153040 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.092+0000 7fd04b72e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd044152380 con 0x7fd0440b6170 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.092+0000 7fd04b72e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0440b2600 con 0x7fd030004d00 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.092+0000 7fd048cc9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd044153040 0x7fd0441534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.092+0000 7fd048cc9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd044153040 0x7fd0441534c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39166/0 (socket says 172.21.15.47:39166) 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.092+0000 7fd048cc9640 1 -- 172.21.15.47:0/3215738809 learned_addr learned my addr 172.21.15.47:0/3215738809 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd0494ca640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd030004d00 0x7fd044150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd049ccb640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0440b6170 0x7fd0440b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd048cc9640 1 -- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd030004d00 msgr2=0x7fd044150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd048cc9640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd030004d00 0x7fd044150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd048cc9640 1 -- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0440b6170 msgr2=0x7fd0440b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:05.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd048cc9640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0440b6170 0x7fd0440b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd048cc9640 1 -- 172.21.15.47:0/3215738809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0441517b0 con 0x7fd044153040 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd0494ca640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd030004d00 0x7fd044150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd048cc9640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd044153040 0x7fd0441534c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd034011570 tx=0x7fd034011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd039ffb640 1 -- 172.21.15.47:0/3215738809 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034015870 con 0x7fd044153040 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd039ffb640 1 -- 172.21.15.47:0/3215738809 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0340159d0 con 0x7fd044153040 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd039ffb640 1 -- 172.21.15.47:0/3215738809 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034015d40 con 0x7fd044153040 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd04b72e640 1 -- 172.21.15.47:0/3215738809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd044151a50 con 0x7fd044153040 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.093+0000 7fd04b72e640 1 -- 172.21.15.47:0/3215738809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd044152010 con 0x7fd044153040 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.094+0000 7fd04b72e640 1 -- 172.21.15.47:0/3215738809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd044072360 con 0x7fd044153040 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.095+0000 7fd039ffb640 1 -- 172.21.15.47:0/3215738809 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd03401d400 con 0x7fd044153040 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.095+0000 7fd039ffb640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd02806ac00 0x7fd02806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.095+0000 7fd039ffb640 1 -- 172.21.15.47:0/3215738809 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd0340898e0 con 0x7fd044153040 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.095+0000 7fd0494ca640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd02806ac00 0x7fd02806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:05.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.096+0000 7fd0494ca640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd02806ac00 0x7fd02806d0c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd040009c40 tx=0x7fd040008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:05.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.099+0000 7fd039ffb640 1 -- 172.21.15.47:0/3215738809 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd034054d60 con 0x7fd044153040 2024-03-23T09:47:05.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.347+0000 7fd04b72e640 1 -- 172.21.15.47:0/3215738809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7fd044066be0 con 0x7fd044153040 2024-03-23T09:47:05.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.971+0000 7fd039ffb640 1 -- 172.21.15.47:0/3215738809 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v283) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd034058380 con 0x7fd044153040 2024-03-23T09:47:05.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.973+0000 7fd04b72e640 1 -- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd02806ac00 msgr2=0x7fd02806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:05.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.973+0000 7fd04b72e640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd02806ac00 0x7fd02806d0c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd040009c40 tx=0x7fd040008040 comp rx=0 tx=0).stop 2024-03-23T09:47:05.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.973+0000 7fd04b72e640 1 -- 172.21.15.47:0/3215738809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd044153040 msgr2=0x7fd0441534c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:05.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.973+0000 7fd04b72e640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd044153040 0x7fd0441534c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd034011570 tx=0x7fd034011a30 comp rx=0 tx=0).stop 2024-03-23T09:47:05.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.973+0000 7fd04b72e640 1 -- 172.21.15.47:0/3215738809 shutdown_connections 2024-03-23T09:47:05.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.973+0000 7fd04b72e640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd02806ac00 0x7fd02806d0c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:05.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.973+0000 7fd04b72e640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0440b6170 0x7fd0440b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:05.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.973+0000 7fd04b72e640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd044153040 0x7fd0441534c0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:05.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.973+0000 7fd04b72e640 1 --2- 172.21.15.47:0/3215738809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd030004d00 0x7fd044150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:05.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.973+0000 7fd04b72e640 1 -- 172.21.15.47:0/3215738809 >> 172.21.15.47:0/3215738809 conn(0x7fd0440659d0 msgr2=0x7fd0440b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:05.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.973+0000 7fd04b72e640 1 -- 172.21.15.47:0/3215738809 shutdown_connections 2024-03-23T09:47:05.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:05.973+0000 7fd04b72e640 1 -- 172.21.15.47:0/3215738809 wait complete. 2024-03-23T09:47:05.985 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-23T09:47:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.091+0000 7f3276607640 1 Processor -- start 2024-03-23T09:47:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.091+0000 7f3276607640 1 -- start start 2024-03-23T09:47:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f3276607640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f32700cde10 0x7f3270160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f3276607640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f32700b8350 0x7f32700d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f3276607640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3270161630 con 0x7f327015c990 2024-03-23T09:47:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f3276607640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32700d1030 con 0x7f32700cde10 2024-03-23T09:47:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f3276607640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32700cfe10 con 0x7f32700b8350 2024-03-23T09:47:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f326f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f32700cde10 0x7f3270160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f326ffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f327015c990 0x7f32700cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48694/0 (socket says 172.21.15.47:48694) 2024-03-23T09:47:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f326ffff640 1 -- 172.21.15.47:0/1425200990 learned_addr learned my addr 172.21.15.47:0/1425200990 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f3274ba4640 1 --2- 172.21.15.47:0/1425200990 >> v2:172.21.15.175:3300/0 conn(0x7f32700b8350 0x7f32700d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f326f7fe640 1 -- 172.21.15.47:0/1425200990 >> v2:172.21.15.175:3300/0 conn(0x7f32700b8350 msgr2=0x7f32700d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f326f7fe640 1 --2- 172.21.15.47:0/1425200990 >> v2:172.21.15.175:3300/0 conn(0x7f32700b8350 0x7f32700d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f326f7fe640 1 -- 172.21.15.47:0/1425200990 >> v1:172.21.15.47:6789/0 conn(0x7f327015c990 legacy=0x7f32700cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f326f7fe640 1 -- 172.21.15.47:0/1425200990 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32700b5b60 con 0x7f32700cde10 2024-03-23T09:47:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.092+0000 7f326effd640 1 -- 172.21.15.47:0/1425200990 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2032880941 0 0) 0x7f3270161630 con 0x7f327015c990 2024-03-23T09:47:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.093+0000 7f3274ba4640 1 --2- 172.21.15.47:0/1425200990 >> v2:172.21.15.175:3300/0 conn(0x7f32700b8350 0x7f32700d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.093+0000 7f326f7fe640 1 --2- 172.21.15.47:0/1425200990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32700cde10 0x7f3270160690 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f32700d0fa0 tx=0x7f326002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ae93b0c80cb487 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.093+0000 7f326effd640 1 -- 172.21.15.47:0/1425200990 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326002e070 con 0x7f32700cde10 2024-03-23T09:47:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.093+0000 7f326effd640 1 -- 172.21.15.47:0/1425200990 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32600315a0 con 0x7f32700cde10 2024-03-23T09:47:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.093+0000 7f326effd640 1 -- 172.21.15.47:0/1425200990 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260031930 con 0x7f32700cde10 2024-03-23T09:47:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.094+0000 7f3276607640 1 -- 172.21.15.47:0/1425200990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32700cde10 msgr2=0x7f3270160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.094+0000 7f3276607640 1 --2- 172.21.15.47:0/1425200990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32700cde10 0x7f3270160690 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f32700d0fa0 tx=0x7f326002b840 comp rx=0 tx=0).stop 2024-03-23T09:47:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.094+0000 7f3276607640 1 -- 172.21.15.47:0/1425200990 shutdown_connections 2024-03-23T09:47:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.094+0000 7f3276607640 1 --2- 172.21.15.47:0/1425200990 >> v2:172.21.15.175:3300/0 conn(0x7f32700b8350 0x7f32700d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.094+0000 7f3276607640 1 --2- 172.21.15.47:0/1425200990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32700cde10 0x7f3270160690 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.094+0000 7f3276607640 1 -- 172.21.15.47:0/1425200990 >> 172.21.15.47:0/1425200990 conn(0x7f32700659d0 msgr2=0x7f3270065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.094+0000 7f3276607640 1 -- 172.21.15.47:0/1425200990 shutdown_connections 2024-03-23T09:47:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.094+0000 7f3276607640 1 -- 172.21.15.47:0/1425200990 wait complete. 2024-03-23T09:47:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.095+0000 7f3276607640 1 Processor -- start 2024-03-23T09:47:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.095+0000 7f3276607640 1 -- start start 2024-03-23T09:47:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3276607640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32700b8350 0x7f3270142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3276607640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32700cde10 0x7f32701429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3276607640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327015c990 0x7f3270148540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3276607640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32700c9b20 con 0x7f327015c990 2024-03-23T09:47:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3276607640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327015df00 con 0x7f32700b8350 2024-03-23T09:47:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3276607640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327015e1e0 con 0x7f32700cde10 2024-03-23T09:47:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3274ba4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327015c990 0x7f3270148540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3274ba4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327015c990 0x7f3270148540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39180/0 (socket says 172.21.15.47:39180) 2024-03-23T09:47:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3274ba4640 1 -- 172.21.15.47:0/3046766350 learned_addr learned my addr 172.21.15.47:0/3046766350 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3274ba4640 1 -- 172.21.15.47:0/3046766350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32700cde10 msgr2=0x7f32701429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3274ba4640 1 --2- 172.21.15.47:0/3046766350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32700cde10 0x7f32701429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3274ba4640 1 -- 172.21.15.47:0/3046766350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32700b8350 msgr2=0x7f3270142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3274ba4640 1 --2- 172.21.15.47:0/3046766350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32700b8350 0x7f3270142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.096+0000 7f3274ba4640 1 -- 172.21.15.47:0/3046766350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3260009400 con 0x7f327015c990 2024-03-23T09:47:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.097+0000 7f3274ba4640 1 --2- 172.21.15.47:0/3046766350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327015c990 0x7f3270148540 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f326800cd50 tx=0x7f326800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.097+0000 7f326cff9640 1 -- 172.21.15.47:0/3046766350 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326800f680 con 0x7f327015c990 2024-03-23T09:47:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.097+0000 7f326cff9640 1 -- 172.21.15.47:0/3046766350 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326800f800 con 0x7f327015c990 2024-03-23T09:47:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.097+0000 7f3276607640 1 -- 172.21.15.47:0/3046766350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327015e4c0 con 0x7f327015c990 2024-03-23T09:47:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.097+0000 7f326cff9640 1 -- 172.21.15.47:0/3046766350 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326800fbb0 con 0x7f327015c990 2024-03-23T09:47:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.097+0000 7f3276607640 1 -- 172.21.15.47:0/3046766350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327015ea80 con 0x7f327015c990 2024-03-23T09:47:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.098+0000 7f3276607640 1 -- 172.21.15.47:0/3046766350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3270072360 con 0x7f327015c990 2024-03-23T09:47:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.098+0000 7f326cff9640 1 -- 172.21.15.47:0/3046766350 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f326800b040 con 0x7f327015c990 2024-03-23T09:47:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.098+0000 7f326cff9640 1 --2- 172.21.15.47:0/3046766350 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f325c06ac00 0x7f325c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.099+0000 7f326cff9640 1 -- 172.21.15.47:0/3046766350 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3268089bc0 con 0x7f327015c990 2024-03-23T09:47:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.099+0000 7f326ffff640 1 --2- 172.21.15.47:0/3046766350 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f325c06ac00 0x7f325c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.099+0000 7f326ffff640 1 --2- 172.21.15.47:0/3046766350 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f325c06ac00 0x7f325c06d0c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3264007fd0 tx=0x7f3264007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.103+0000 7f326cff9640 1 -- 172.21.15.47:0/3046766350 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3268054fb0 con 0x7f327015c990 2024-03-23T09:47:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.280+0000 7f3276607640 1 -- 172.21.15.47:0/3046766350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3270066be0 con 0x7f327015c990 2024-03-23T09:47:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.284+0000 7f326cff9640 1 -- 172.21.15.47:0/3046766350 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f32680585d0 con 0x7f327015c990 2024-03-23T09:47:06.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.286+0000 7f3276607640 1 -- 172.21.15.47:0/3046766350 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f325c06ac00 msgr2=0x7f325c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:06.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.286+0000 7f3276607640 1 --2- 172.21.15.47:0/3046766350 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f325c06ac00 0x7f325c06d0c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3264007fd0 tx=0x7f3264007920 comp rx=0 tx=0).stop 2024-03-23T09:47:06.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.286+0000 7f3276607640 1 -- 172.21.15.47:0/3046766350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327015c990 msgr2=0x7f3270148540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:06.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.286+0000 7f3276607640 1 --2- 172.21.15.47:0/3046766350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327015c990 0x7f3270148540 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f326800cd50 tx=0x7f326800a650 comp rx=0 tx=0).stop 2024-03-23T09:47:06.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.286+0000 7f3276607640 1 -- 172.21.15.47:0/3046766350 shutdown_connections 2024-03-23T09:47:06.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.286+0000 7f3276607640 1 --2- 172.21.15.47:0/3046766350 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f325c06ac00 0x7f325c06d0c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.286+0000 7f3276607640 1 --2- 172.21.15.47:0/3046766350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327015c990 0x7f3270148540 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.286+0000 7f3276607640 1 --2- 172.21.15.47:0/3046766350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32700cde10 0x7f32701429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.286+0000 7f3276607640 1 --2- 172.21.15.47:0/3046766350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32700b8350 0x7f3270142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.286+0000 7f3276607640 1 -- 172.21.15.47:0/3046766350 >> 172.21.15.47:0/3046766350 conn(0x7f32700659d0 msgr2=0x7f327015b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:06.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.286+0000 7f3276607640 1 -- 172.21.15.47:0/3046766350 shutdown_connections 2024-03-23T09:47:06.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.286+0000 7f3276607640 1 -- 172.21.15.47:0/3046766350 wait complete. 2024-03-23T09:47:06.297 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:47:06.297 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:47:06.336 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:47:06.383 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:47:06.383 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:47:06.383 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:47:06.398 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:47:06.398 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:47:06.398 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:47:06.415 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:47:06.415 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:47:06.415 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:47:06.452 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:47:06.452 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:47:06.452 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:47:06.472 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:47:06.473 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-23T09:47:06.473 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:47:06.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.593+0000 7f948beea640 1 Processor -- start 2024-03-23T09:47:06.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.593+0000 7f948beea640 1 -- start start 2024-03-23T09:47:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f948beea640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f948415cee0 0x7f94840d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f948beea640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94840d1300 con 0x7f94840d2f80 2024-03-23T09:47:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f948beea640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94840d15e0 con 0x7f94840d23a0 2024-03-23T09:47:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f948beea640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94840d18c0 con 0x7f948415cee0 2024-03-23T09:47:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f948a487640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f94840d2f80 0x7f94840b5a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48708/0 (socket says 172.21.15.47:48708) 2024-03-23T09:47:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f948a487640 1 -- 172.21.15.47:0/3241969957 learned_addr learned my addr 172.21.15.47:0/3241969957 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f9489c86640 1 --2- 172.21.15.47:0/3241969957 >> v2:172.21.15.175:3300/0 conn(0x7f948415cee0 0x7f94840d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318207054 0 0) 0x7f94840d1300 con 0x7f94840d2f80 2024-03-23T09:47:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9470003520 con 0x7f94840d2f80 2024-03-23T09:47:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3788628374 0 0) 0x7f9470003520 con 0x7f94840d2f80 2024-03-23T09:47:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 >> v2:172.21.15.175:3300/0 conn(0x7f948415cee0 msgr2=0x7f94840d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f9488c84640 1 --2- 172.21.15.47:0/3241969957 >> v2:172.21.15.175:3300/0 conn(0x7f948415cee0 0x7f94840d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 >> v1:172.21.15.175:6789/0 conn(0x7f94840d23a0 legacy=0x7f94840d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.594+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94840d1ba0 con 0x7f94840d2f80 2024-03-23T09:47:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f947c0019f0 con 0x7f94840d2f80 2024-03-23T09:47:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9489c86640 1 --2- 172.21.15.47:0/3241969957 >> v2:172.21.15.175:3300/0 conn(0x7f948415cee0 0x7f94840d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:47:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 >> v1:172.21.15.47:6789/0 conn(0x7f94840d2f80 legacy=0x7f94840b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9488c84640 1 --2- 172.21.15.47:0/3241969957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9470003970 0x7f9470003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9488c84640 1 --2- 172.21.15.47:0/3241969957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94700047c0 0x7f9470004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9488c84640 1 --2- 172.21.15.47:0/3241969957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9470005180 0x7f947000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94840d18c0 con 0x7f9470003970 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947000c020 con 0x7f94700047c0 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947000c1e0 con 0x7f9470005180 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9489c86640 1 --2- 172.21.15.47:0/3241969957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9470003970 0x7f9470003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9489485640 1 --2- 172.21.15.47:0/3241969957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94700047c0 0x7f9470004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f948a487640 1 --2- 172.21.15.47:0/3241969957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9470005180 0x7f947000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9489c86640 1 -- 172.21.15.47:0/3241969957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9470005180 msgr2=0x7f947000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9489c86640 1 --2- 172.21.15.47:0/3241969957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9470005180 0x7f947000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9489c86640 1 -- 172.21.15.47:0/3241969957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94700047c0 msgr2=0x7f9470004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9489c86640 1 --2- 172.21.15.47:0/3241969957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94700047c0 0x7f9470004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f9489c86640 1 -- 172.21.15.47:0/3241969957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9470003520 con 0x7f9470003970 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f948a487640 1 --2- 172.21.15.47:0/3241969957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9470005180 0x7f947000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.595+0000 7f948a487640 1 -- 172.21.15.47:0/3241969957 reap_dead start 2024-03-23T09:47:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.596+0000 7f9489c86640 1 --2- 172.21.15.47:0/3241969957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9470003970 0x7f9470003d90 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f9480009c90 tx=0x7f94700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9c7b078c34f29 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.596+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948002c8f0 con 0x7f9470003970 2024-03-23T09:47:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.596+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948002ca50 con 0x7f9470003970 2024-03-23T09:47:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.596+0000 7f9488c84640 1 -- 172.21.15.47:0/3241969957 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948002ce40 con 0x7f9470003970 2024-03-23T09:47:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.596+0000 7f948beea640 1 -- 172.21.15.47:0/3241969957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9470003970 msgr2=0x7f9470003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.596+0000 7f948beea640 1 --2- 172.21.15.47:0/3241969957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9470003970 0x7f9470003d90 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f9480009c90 tx=0x7f94700057f0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.596+0000 7f948beea640 1 -- 172.21.15.47:0/3241969957 shutdown_connections 2024-03-23T09:47:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.596+0000 7f948beea640 1 --2- 172.21.15.47:0/3241969957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9470003970 0x7f9470003d90 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.596+0000 7f948beea640 1 -- 172.21.15.47:0/3241969957 >> 172.21.15.47:0/3241969957 conn(0x7f9484065df0 msgr2=0x7f9484066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.596+0000 7f948beea640 1 -- 172.21.15.47:0/3241969957 shutdown_connections 2024-03-23T09:47:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.596+0000 7f948beea640 1 -- 172.21.15.47:0/3241969957 wait complete. 2024-03-23T09:47:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.598+0000 7f948beea640 1 Processor -- start 2024-03-23T09:47:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.598+0000 7f948beea640 1 -- start start 2024-03-23T09:47:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.598+0000 7f948beea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9470003970 0x7f948414faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.598+0000 7f948beea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948415b680 0x7f948415bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.598+0000 7f948beea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94840c8220 0x7f94840c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.598+0000 7f948beea640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948415c040 con 0x7f948415b680 2024-03-23T09:47:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.598+0000 7f948beea640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948415e120 con 0x7f94840c8220 2024-03-23T09:47:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.598+0000 7f948beea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948415d360 con 0x7f9470003970 2024-03-23T09:47:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.598+0000 7f9489485640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948415b680 0x7f948415bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.598+0000 7f9489485640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948415b680 0x7f948415bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39208/0 (socket says 172.21.15.47:39208) 2024-03-23T09:47:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.598+0000 7f9489485640 1 -- 172.21.15.47:0/3519268402 learned_addr learned my addr 172.21.15.47:0/3519268402 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.599+0000 7f9489c86640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9470003970 0x7f948414faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.599+0000 7f948a487640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94840c8220 0x7f94840c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.599+0000 7f9489485640 1 -- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9470003970 msgr2=0x7f948414faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.599+0000 7f9489485640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9470003970 0x7f948414faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.599+0000 7f9489485640 1 -- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94840c8220 msgr2=0x7f94840c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.599+0000 7f9489485640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94840c8220 0x7f94840c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.599+0000 7f9489485640 1 -- 172.21.15.47:0/3519268402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948415d640 con 0x7f948415b680 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.599+0000 7f9489485640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948415b680 0x7f948415bb00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f947400e490 tx=0x7f947400e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.599+0000 7f947a7fc640 1 -- 172.21.15.47:0/3519268402 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94740053a0 con 0x7f948415b680 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.599+0000 7f947a7fc640 1 -- 172.21.15.47:0/3519268402 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94740035f0 con 0x7f948415b680 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.599+0000 7f948beea640 1 -- 172.21.15.47:0/3519268402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948415d920 con 0x7f948415b680 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.599+0000 7f947a7fc640 1 -- 172.21.15.47:0/3519268402 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474003980 con 0x7f948415b680 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.599+0000 7f948beea640 1 -- 172.21.15.47:0/3519268402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9484150030 con 0x7f948415b680 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.600+0000 7f948beea640 1 -- 172.21.15.47:0/3519268402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9484072360 con 0x7f948415b680 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.600+0000 7f947a7fc640 1 -- 172.21.15.47:0/3519268402 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9474003ae0 con 0x7f948415b680 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.601+0000 7f947a7fc640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f946806ac30 0x7f946806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.601+0000 7f947a7fc640 1 -- 172.21.15.47:0/3519268402 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f9474088db0 con 0x7f948415b680 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.601+0000 7f9489c86640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f946806ac30 0x7f946806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.601+0000 7f9489c86640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f946806ac30 0x7f946806d0f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9480036000 tx=0x7f9480030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:06.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.605+0000 7f947a7fc640 1 -- 172.21.15.47:0/3519268402 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f94740542c0 con 0x7f948415b680 2024-03-23T09:47:06.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.821+0000 7f948beea640 1 -- 172.21.15.47:0/3519268402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9484066be0 con 0x7f948415b680 2024-03-23T09:47:06.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.821+0000 7f947a7fc640 1 -- 172.21.15.47:0/3519268402 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f94740578e0 con 0x7f948415b680 2024-03-23T09:47:06.822 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:06.822 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3862818105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3862818105},{"type":"v1","addr":"172.21.15.175:6836","nonce":3862818105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3615592850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3615592850},{"type":"v1","addr":"172.21.15.175:6837","nonce":3615592850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/832829684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":832829684},{"type":"v1","addr":"172.21.15.47:6834","nonce":832829684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":7452,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4052995919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4052995919},{"type":"v1","addr":"172.21.15.47:6835","nonce":4052995919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[]} 2024-03-23T09:47:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 42 2024-03-23T09:47:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.823+0000 7f948beea640 1 -- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f946806ac30 msgr2=0x7f946806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.823+0000 7f948beea640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f946806ac30 0x7f946806d0f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9480036000 tx=0x7f9480030000 comp rx=0 tx=0).stop 2024-03-23T09:47:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.823+0000 7f948beea640 1 -- 172.21.15.47:0/3519268402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948415b680 msgr2=0x7f948415bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.823+0000 7f948beea640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948415b680 0x7f948415bb00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f947400e490 tx=0x7f947400e950 comp rx=0 tx=0).stop 2024-03-23T09:47:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.823+0000 7f948beea640 1 -- 172.21.15.47:0/3519268402 shutdown_connections 2024-03-23T09:47:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.823+0000 7f948beea640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f946806ac30 0x7f946806d0f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.823+0000 7f948beea640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94840c8220 0x7f94840c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.823+0000 7f948beea640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948415b680 0x7f948415bb00 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.823+0000 7f948beea640 1 --2- 172.21.15.47:0/3519268402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9470003970 0x7f948414faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.823+0000 7f948beea640 1 -- 172.21.15.47:0/3519268402 >> 172.21.15.47:0/3519268402 conn(0x7f9484065df0 msgr2=0x7f9484156530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.824+0000 7f948beea640 1 -- 172.21.15.47:0/3519268402 shutdown_connections 2024-03-23T09:47:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.824+0000 7f948beea640 1 -- 172.21.15.47:0/3519268402 wait complete. 2024-03-23T09:47:06.834 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-23T09:47:06.835 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-23T09:47:06.835 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-23T09:47:06.835 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:47:06.835 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-23T09:47:06.835 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-23T09:47:06.835 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-23T09:47:06.835 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:47:06.836 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-23T09:47:06.836 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-23T09:47:06.836 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-23T09:47:06.836 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:47:06.836 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-23T09:47:06.836 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-23T09:47:06.836 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-23T09:47:06.837 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:47:06.844 INFO:tasks.ceph.mds.a:Stopped 2024-03-23T09:47:06.844 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-23T09:47:06.845 INFO:tasks.ceph.mds.c:Stopped 2024-03-23T09:47:06.845 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-23T09:47:06.848 INFO:tasks.ceph.mds.d:Stopped 2024-03-23T09:47:06.848 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-23T09:47:06.848 INFO:tasks.ceph.mds.b:Stopped 2024-03-23T09:47:06.849 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-23T09:47:06.886 INFO:tasks.ceph.mds.a:Started 2024-03-23T09:47:06.887 DEBUG:teuthology.parallel:result is None 2024-03-23T09:47:06.887 INFO:tasks.ceph.mds.c:Started 2024-03-23T09:47:06.887 DEBUG:teuthology.parallel:result is None 2024-03-23T09:47:06.890 INFO:tasks.ceph.mds.d:Started 2024-03-23T09:47:06.890 DEBUG:teuthology.parallel:result is None 2024-03-23T09:47:06.891 INFO:tasks.ceph.mds.b:Started 2024-03-23T09:47:06.891 DEBUG:teuthology.parallel:result is None 2024-03-23T09:47:06.891 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-23T09:47:06.961 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-03-23T09:47:06.962 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:47:06.973 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-03-23T09:47:06.973 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-03-23T09:47:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.998+0000 7ff6e3ae7640 1 Processor -- start 2024-03-23T09:47:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.998+0000 7ff6e3ae7640 1 -- start start 2024-03-23T09:47:07.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.999+0000 7ff6e3ae7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6dc0cf690 0x7ff6dc0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:07.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.999+0000 7ff6e3ae7640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6dc15d0e0 con 0x7ff6dc0cf210 2024-03-23T09:47:07.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.999+0000 7ff6e3ae7640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6dc0d1540 con 0x7ff6dc15cd00 2024-03-23T09:47:07.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:06.999+0000 7ff6e3ae7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0d1820 con 0x7ff6dc0cf690 2024-03-23T09:47:07.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.000+0000 7ff6e1082640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff6dc0cf210 0x7ff6dc0b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48734/0 (socket says 172.21.15.47:48734) 2024-03-23T09:47:07.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.000+0000 7ff6e1082640 1 -- 172.21.15.47:0/2727299850 learned_addr learned my addr 172.21.15.47:0/2727299850 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:07.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.000+0000 7ff6e2084640 1 --2- 172.21.15.47:0/2727299850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6dc0cf690 0x7ff6dc0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:07.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.000+0000 7ff6e2084640 1 -- 172.21.15.47:0/2727299850 >> v1:172.21.15.175:6789/0 conn(0x7ff6dc15cd00 legacy=0x7ff6dc0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:07.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.000+0000 7ff6e2084640 1 -- 172.21.15.47:0/2727299850 >> v1:172.21.15.47:6789/0 conn(0x7ff6dc0cf210 legacy=0x7ff6dc0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:07.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.000+0000 7ff6e2084640 1 -- 172.21.15.47:0/2727299850 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc0d1b00 con 0x7ff6dc0cf690 2024-03-23T09:47:07.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.001+0000 7ff6e2084640 1 --2- 172.21.15.47:0/2727299850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6dc0cf690 0x7ff6dc0b61a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009dc0 tx=0x7ff6d402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bce53141e29ecca server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:07.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.001+0000 7ff6e0881640 1 -- 172.21.15.47:0/2727299850 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d402cc40 con 0x7ff6dc0cf690 2024-03-23T09:47:07.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.001+0000 7ff6e0881640 1 -- 172.21.15.47:0/2727299850 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d402cda0 con 0x7ff6dc0cf690 2024-03-23T09:47:07.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.001+0000 7ff6e0881640 1 -- 172.21.15.47:0/2727299850 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4034660 con 0x7ff6dc0cf690 2024-03-23T09:47:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.002+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2727299850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6dc0cf690 msgr2=0x7ff6dc0b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.002+0000 7ff6e3ae7640 1 --2- 172.21.15.47:0/2727299850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6dc0cf690 0x7ff6dc0b61a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009dc0 tx=0x7ff6d402be90 comp rx=0 tx=0).stop 2024-03-23T09:47:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.002+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2727299850 shutdown_connections 2024-03-23T09:47:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.002+0000 7ff6e3ae7640 1 --2- 172.21.15.47:0/2727299850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6dc0cf690 0x7ff6dc0b61a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.002+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2727299850 >> 172.21.15.47:0/2727299850 conn(0x7ff6dc06b160 msgr2=0x7ff6dc06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.002+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2727299850 shutdown_connections 2024-03-23T09:47:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.002+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2727299850 wait complete. 2024-03-23T09:47:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.003+0000 7ff6e3ae7640 1 Processor -- start 2024-03-23T09:47:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.003+0000 7ff6e3ae7640 1 -- start start 2024-03-23T09:47:07.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.004+0000 7ff6e3ae7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6dc0cf210 0x7ff6dc14f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:07.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.004+0000 7ff6e3ae7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6dc0cf690 0x7ff6dc0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:07.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.004+0000 7ff6e3ae7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6dc15cd00 0x7ff6dc0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:07.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.004+0000 7ff6e3ae7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0cb5e0 con 0x7ff6dc0cf210 2024-03-23T09:47:07.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.004+0000 7ff6e3ae7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0ca9d0 con 0x7ff6dc15cd00 2024-03-23T09:47:07.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.004+0000 7ff6e3ae7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0ca6f0 con 0x7ff6dc0cf690 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.004+0000 7ff6e1883640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6dc0cf210 0x7ff6dc14f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.004+0000 7ff6e1883640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6dc0cf210 0x7ff6dc14f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39246/0 (socket says 172.21.15.47:39246) 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.004+0000 7ff6e1883640 1 -- 172.21.15.47:0/2846838693 learned_addr learned my addr 172.21.15.47:0/2846838693 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.004+0000 7ff6e1082640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6dc0cf690 0x7ff6dc0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.004+0000 7ff6e2084640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6dc15cd00 0x7ff6dc0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.005+0000 7ff6e1883640 1 -- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6dc0cf690 msgr2=0x7ff6dc0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.005+0000 7ff6e1883640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6dc0cf690 0x7ff6dc0b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.005+0000 7ff6e1883640 1 -- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6dc15cd00 msgr2=0x7ff6dc0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.005+0000 7ff6e1883640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6dc15cd00 0x7ff6dc0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.005+0000 7ff6e1883640 1 -- 172.21.15.47:0/2846838693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4009a70 con 0x7ff6dc0cf210 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.005+0000 7ff6e1883640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6dc0cf210 0x7ff6dc14f150 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800c3c0 tx=0x7ff6d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.005+0000 7ff6d27fc640 1 -- 172.21.15.47:0/2846838693 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d80052d0 con 0x7ff6dc0cf210 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.005+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2846838693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6dc0caf20 con 0x7ff6dc0cf210 2024-03-23T09:47:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.005+0000 7ff6d27fc640 1 -- 172.21.15.47:0/2846838693 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d8005450 con 0x7ff6dc0cf210 2024-03-23T09:47:07.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.005+0000 7ff6d27fc640 1 -- 172.21.15.47:0/2846838693 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8003830 con 0x7ff6dc0cf210 2024-03-23T09:47:07.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.005+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2846838693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6dc14f690 con 0x7ff6dc0cf210 2024-03-23T09:47:07.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.007+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2846838693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6dc072360 con 0x7ff6dc0cf210 2024-03-23T09:47:07.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.007+0000 7ff6d27fc640 1 -- 172.21.15.47:0/2846838693 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff6d8003990 con 0x7ff6dc0cf210 2024-03-23T09:47:07.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.007+0000 7ff6d27fc640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff6c006ac00 0x7ff6c006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:07.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.007+0000 7ff6d27fc640 1 -- 172.21.15.47:0/2846838693 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff6d8088530 con 0x7ff6dc0cf210 2024-03-23T09:47:07.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.007+0000 7ff6e1082640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff6c006ac00 0x7ff6c006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:07.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.008+0000 7ff6e1082640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff6c006ac00 0x7ff6c006d0c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0b5340 tx=0x7ff6cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:07.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.011+0000 7ff6d27fc640 1 -- 172.21.15.47:0/2846838693 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff6d8053a70 con 0x7ff6dc0cf210 2024-03-23T09:47:07.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:07.205+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2846838693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff6dc15afd0 con 0x7ff6dc0cf210 2024-03-23T09:47:07.996 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:47:07.999 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:47:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.007+0000 7ff6d27fc640 1 -- 172.21.15.47:0/2846838693 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v81) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff6d8057090 con 0x7ff6dc0cf210 2024-03-23T09:47:08.008 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blocklist entries 2024-03-23T09:47:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.010+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff6c006ac00 msgr2=0x7ff6c006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.010+0000 7ff6e3ae7640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff6c006ac00 0x7ff6c006d0c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0b5340 tx=0x7ff6cc009040 comp rx=0 tx=0).stop 2024-03-23T09:47:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.010+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2846838693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6dc0cf210 msgr2=0x7ff6dc14f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.010+0000 7ff6e3ae7640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6dc0cf210 0x7ff6dc14f150 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800c3c0 tx=0x7ff6d800c880 comp rx=0 tx=0).stop 2024-03-23T09:47:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.010+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2846838693 shutdown_connections 2024-03-23T09:47:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.010+0000 7ff6e3ae7640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff6c006ac00 0x7ff6c006d0c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.010+0000 7ff6e3ae7640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6dc15cd00 0x7ff6dc0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.010+0000 7ff6e3ae7640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6dc0cf690 0x7ff6dc0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.010+0000 7ff6e3ae7640 1 --2- 172.21.15.47:0/2846838693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6dc0cf210 0x7ff6dc14f150 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.010+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2846838693 >> 172.21.15.47:0/2846838693 conn(0x7ff6dc06b160 msgr2=0x7ff6dc15da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.011+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2846838693 shutdown_connections 2024-03-23T09:47:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.011+0000 7ff6e3ae7640 1 -- 172.21.15.47:0/2846838693 wait complete. 2024-03-23T09:47:08.024 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-23T09:47:08.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.203+0000 7f5e78f5c640 1 Processor -- start 2024-03-23T09:47:08.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.203+0000 7f5e78f5c640 1 -- start start 2024-03-23T09:47:08.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.203+0000 7f5e78f5c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5e7415cee0 0x7f5e740d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e78f5c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e740d23a0 0x7f5e740d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e78f5c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e740d1300 con 0x7f5e740b8300 2024-03-23T09:47:08.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e78f5c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e740d15e0 con 0x7f5e7415cee0 2024-03-23T09:47:08.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e78f5c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740d18c0 con 0x7f5e740d23a0 2024-03-23T09:47:08.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e7259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5e7415cee0 0x7f5e740d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e7259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5e7415cee0 0x7f5e740d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39252/0 (socket says 172.21.15.47:39252) 2024-03-23T09:47:08.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e7259c640 1 -- 172.21.15.47:0/2907660583 learned_addr learned my addr 172.21.15.47:0/2907660583 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:08.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e72d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5e740b8300 0x7f5e740b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48746/0 (socket says 172.21.15.47:48746) 2024-03-23T09:47:08.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e69d9b640 1 --2- 172.21.15.47:0/2907660583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e740d23a0 0x7f5e740d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e7259c640 1 -- 172.21.15.47:0/2907660583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e740d23a0 msgr2=0x7f5e740d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:08.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e7259c640 1 --2- 172.21.15.47:0/2907660583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e740d23a0 0x7f5e740d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e7259c640 1 -- 172.21.15.47:0/2907660583 >> v1:172.21.15.47:6789/0 conn(0x7f5e740b8300 legacy=0x7f5e740b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e7259c640 1 -- 172.21.15.47:0/2907660583 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e740d1ba0 con 0x7f5e7415cee0 2024-03-23T09:47:08.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e71d9b640 1 -- 172.21.15.47:0/2907660583 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441362725 0 0) 0x7f5e740d1300 con 0x7f5e740b8300 2024-03-23T09:47:08.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.204+0000 7f5e69d9b640 1 --2- 172.21.15.47:0/2907660583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e740d23a0 0x7f5e740d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:08.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.205+0000 7f5e7259c640 1 --2- 172.21.15.47:0/2907660583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e7415cee0 0x7f5e740d0990 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009870 tx=0x7f5e6402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d19751bb049f011 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:08.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.205+0000 7f5e71d9b640 1 -- 172.21.15.47:0/2907660583 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e64030840 con 0x7f5e7415cee0 2024-03-23T09:47:08.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.205+0000 7f5e71d9b640 1 -- 172.21.15.47:0/2907660583 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e640309a0 con 0x7f5e7415cee0 2024-03-23T09:47:08.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.205+0000 7f5e71d9b640 1 -- 172.21.15.47:0/2907660583 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e64030d30 con 0x7f5e7415cee0 2024-03-23T09:47:08.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.205+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2907660583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e7415cee0 msgr2=0x7f5e740d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.205+0000 7f5e78f5c640 1 --2- 172.21.15.47:0/2907660583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e7415cee0 0x7f5e740d0990 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009870 tx=0x7f5e6402b980 comp rx=0 tx=0).stop 2024-03-23T09:47:08.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.205+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2907660583 shutdown_connections 2024-03-23T09:47:08.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.205+0000 7f5e78f5c640 1 --2- 172.21.15.47:0/2907660583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e740d23a0 0x7f5e740d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.205+0000 7f5e78f5c640 1 --2- 172.21.15.47:0/2907660583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e7415cee0 0x7f5e740d0990 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.205+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2907660583 >> 172.21.15.47:0/2907660583 conn(0x7f5e74065df0 msgr2=0x7f5e74066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:08.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.206+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2907660583 shutdown_connections 2024-03-23T09:47:08.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.206+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2907660583 wait complete. 2024-03-23T09:47:08.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.207+0000 7f5e78f5c640 1 Processor -- start 2024-03-23T09:47:08.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.207+0000 7f5e78f5c640 1 -- start start 2024-03-23T09:47:08.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e78f5c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e740b8300 0x7f5e7414e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e78f5c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e740d23a0 0x7f5e740c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e78f5c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e7415cee0 0x7f5e741504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e78f5c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740c6390 con 0x7f5e7415cee0 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e78f5c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74150f20 con 0x7f5e740d23a0 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e78f5c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740c65e0 con 0x7f5e740b8300 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e72d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e7415cee0 0x7f5e741504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e72d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e7415cee0 0x7f5e741504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39262/0 (socket says 172.21.15.47:39262) 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e72d9d640 1 -- 172.21.15.47:0/2877315877 learned_addr learned my addr 172.21.15.47:0/2877315877 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e69d9b640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e740d23a0 0x7f5e740c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e72d9d640 1 -- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e740b8300 msgr2=0x7f5e7414e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e72d9d640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e740b8300 0x7f5e7414e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e72d9d640 1 -- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e740d23a0 msgr2=0x7f5e740c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e72d9d640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e740d23a0 0x7f5e740c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e72d9d640 1 -- 172.21.15.47:0/2877315877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e640094d0 con 0x7f5e7415cee0 2024-03-23T09:47:08.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e69d9b640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e740d23a0 0x7f5e740c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:08.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e72d9d640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e7415cee0 0x7f5e741504d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00c770 tx=0x7f5e6c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:08.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.208+0000 7f5e6b7fe640 1 -- 172.21.15.47:0/2877315877 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00ce80 con 0x7f5e7415cee0 2024-03-23T09:47:08.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.209+0000 7f5e6b7fe640 1 -- 172.21.15.47:0/2877315877 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c00a640 con 0x7f5e7415cee0 2024-03-23T09:47:08.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.209+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2877315877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e740c68c0 con 0x7f5e7415cee0 2024-03-23T09:47:08.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.209+0000 7f5e6b7fe640 1 -- 172.21.15.47:0/2877315877 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00a9f0 con 0x7f5e7415cee0 2024-03-23T09:47:08.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.209+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2877315877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e740c8e80 con 0x7f5e7415cee0 2024-03-23T09:47:08.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.210+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2877315877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e74072360 con 0x7f5e7415cee0 2024-03-23T09:47:08.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.210+0000 7f5e6b7fe640 1 -- 172.21.15.47:0/2877315877 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5e6c00ab50 con 0x7f5e7415cee0 2024-03-23T09:47:08.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.210+0000 7f5e6b7fe640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5e5406ac00 0x7f5e5406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.210+0000 7f5e6b7fe640 1 -- 172.21.15.47:0/2877315877 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e6c088770 con 0x7f5e7415cee0 2024-03-23T09:47:08.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.211+0000 7f5e7259c640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5e5406ac00 0x7f5e5406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.211+0000 7f5e7259c640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5e5406ac00 0x7f5e5406d0c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5e64036000 tx=0x7f5e6403b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:08.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.215+0000 7f5e6b7fe640 1 -- 172.21.15.47:0/2877315877 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5e6c053d30 con 0x7f5e7415cee0 2024-03-23T09:47:08.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.466+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2877315877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5e740c6de0 con 0x7f5e7415cee0 2024-03-23T09:47:08.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.466+0000 7f5e6b7fe640 1 -- 172.21.15.47:0/2877315877 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5e740c6de0 con 0x7f5e7415cee0 2024-03-23T09:47:08.467 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:08.467 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:47:08.467 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2024-03-23T09:47:08.467 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.467 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2024-03-23T09:47:08.467 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lo+xoDhAAyxqr1H6eiINwt2X7LUlrSg==", 2024-03-23T09:47:08.467 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.467 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:47:08.467 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:47:08.467 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:47:08.467 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.467 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.468 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.468 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2024-03-23T09:47:08.468 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZ5OlGhAAuUElzRghNRFGKREdCpxKvA==", 2024-03-23T09:47:08.468 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.468 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:47:08.468 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:47:08.468 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:47:08.468 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.468 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.468 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.468 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2024-03-23T09:47:08.468 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lFwyzFRAAKldMOA7oSblA8YMrvntY8w==", 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.d", 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lpiD3IRAApXAdUQ4AkxWHzsFV0qcBng==", 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:47:08.469 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB7o/5lLvoADBAAgdixkm+jHwQilYw0IAn6EA==", 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2024-03-23T09:47:08.470 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB+o/5lOBfGCRAApglzalO1ztnJzOKg/UAyWA==", 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCBo/5lRBjlBxAAd45w8/kTMPqxmNyfCS3XYA==", 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.471 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.472 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2024-03-23T09:47:08.472 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCEo/5l+JxrBhAAbR/XqDyPDnN8eLpmvBZ7Mg==", 2024-03-23T09:47:08.472 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.472 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:47:08.472 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:47:08.472 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.472 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.472 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2024-03-23T09:47:08.472 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCKo/5ltsqwDRAAkqcZ/sEf2oozWFN4AXg/+w==", 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCNo/5lpLHiCxAAB2axUv9gjcYVlij9f+AEHg==", 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCQo/5l2io3ChAAu+lfYcWWBQf43aYzWjCBNg==", 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCTo/5lA7jjCBAAjJgBkp89H8lx6fidl/Z7IA==", 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.474 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZFc3JhAAZWmDZOnkW9eo9hqEDYynQw==", 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.475 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lIrJSLxAAPJer3ZP3h8VkImkiJmnKVw==", 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l1+DyKxAAZLKZNbUpUJ0fWPIYNvWK+g==", 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:47:08.476 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lWqkiNRAAL1q0lEcM/NKW3Pf9LcP21g==", 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.477 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB4o/5lyeuKLRAARrfTpPc5BTVYZe/LUREDkg==", 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l/0EOCxAAIrgIXRiFSQ1G7Jat5hNm4A==", 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:47:08.478 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lywMFBxAAIVwjCe6B9EtBMAG2n7cZJw==", 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-23T09:47:08.479 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:47:08.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.468+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5e5406ac00 msgr2=0x7f5e5406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.468+0000 7f5e78f5c640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5e5406ac00 0x7f5e5406d0c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5e64036000 tx=0x7f5e6403b040 comp rx=0 tx=0).stop 2024-03-23T09:47:08.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.468+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2877315877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e7415cee0 msgr2=0x7f5e741504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.468+0000 7f5e78f5c640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e7415cee0 0x7f5e741504d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00c770 tx=0x7f5e6c00cc30 comp rx=0 tx=0).stop 2024-03-23T09:47:08.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.468+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2877315877 shutdown_connections 2024-03-23T09:47:08.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.468+0000 7f5e78f5c640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5e5406ac00 0x7f5e5406d0c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.468+0000 7f5e78f5c640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e7415cee0 0x7f5e741504d0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.468+0000 7f5e78f5c640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e740d23a0 0x7f5e740c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.468+0000 7f5e78f5c640 1 --2- 172.21.15.47:0/2877315877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e740b8300 0x7f5e7414e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.468+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2877315877 >> 172.21.15.47:0/2877315877 conn(0x7f5e74065df0 msgr2=0x7f5e7415c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.468+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2877315877 shutdown_connections 2024-03-23T09:47:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.468+0000 7f5e78f5c640 1 -- 172.21.15.47:0/2877315877 wait complete. 2024-03-23T09:47:08.482 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:47:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.584+0000 7fd2426c4640 1 Processor -- start 2024-03-23T09:47:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.584+0000 7fd2426c4640 1 -- start start 2024-03-23T09:47:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.584+0000 7fd2426c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd23c15c9d0 0x7fd23c071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.584+0000 7fd2426c4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23c0721f0 con 0x7fd23c06a3f0 2024-03-23T09:47:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.584+0000 7fd2426c4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23c06eb80 con 0x7fd23c15cdb0 2024-03-23T09:47:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.584+0000 7fd2426c4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23c06edd0 con 0x7fd23c15c9d0 2024-03-23T09:47:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.585+0000 7fd23bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd23c15c9d0 0x7fd23c071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.585+0000 7fd240c61640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd23c06a3f0 0x7fd23c160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48750/0 (socket says 172.21.15.47:48750) 2024-03-23T09:47:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.585+0000 7fd240c61640 1 -- 172.21.15.47:0/1418897558 learned_addr learned my addr 172.21.15.47:0/1418897558 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.585+0000 7fd23bfff640 1 -- 172.21.15.47:0/1418897558 >> v1:172.21.15.175:6789/0 conn(0x7fd23c15cdb0 legacy=0x7fd23c069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.585+0000 7fd23bfff640 1 -- 172.21.15.47:0/1418897558 >> v1:172.21.15.47:6789/0 conn(0x7fd23c06a3f0 legacy=0x7fd23c160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.585+0000 7fd23affd640 1 -- 172.21.15.47:0/1418897558 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469074020 0 0) 0x7fd23c0721f0 con 0x7fd23c06a3f0 2024-03-23T09:47:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.585+0000 7fd23bfff640 1 -- 172.21.15.47:0/1418897558 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c06f0b0 con 0x7fd23c15c9d0 2024-03-23T09:47:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.586+0000 7fd23bfff640 1 --2- 172.21.15.47:0/1418897558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd23c15c9d0 0x7fd23c071cb0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0097a0 tx=0x7fd22c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71e29637de93dff server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.586+0000 7fd23affd640 1 -- 172.21.15.47:0/1418897558 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c030610 con 0x7fd23c15c9d0 2024-03-23T09:47:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.586+0000 7fd23affd640 1 -- 172.21.15.47:0/1418897558 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c030770 con 0x7fd23c15c9d0 2024-03-23T09:47:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.586+0000 7fd23affd640 1 -- 172.21.15.47:0/1418897558 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c030b00 con 0x7fd23c15c9d0 2024-03-23T09:47:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.586+0000 7fd2426c4640 1 -- 172.21.15.47:0/1418897558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd23c15c9d0 msgr2=0x7fd23c071cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.586+0000 7fd2426c4640 1 --2- 172.21.15.47:0/1418897558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd23c15c9d0 0x7fd23c071cb0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0097a0 tx=0x7fd22c02b960 comp rx=0 tx=0).stop 2024-03-23T09:47:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.586+0000 7fd2426c4640 1 -- 172.21.15.47:0/1418897558 shutdown_connections 2024-03-23T09:47:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.586+0000 7fd2426c4640 1 --2- 172.21.15.47:0/1418897558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd23c15c9d0 0x7fd23c071cb0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.586+0000 7fd2426c4640 1 -- 172.21.15.47:0/1418897558 >> 172.21.15.47:0/1418897558 conn(0x7fd23c0b13a0 msgr2=0x7fd23c0b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.586+0000 7fd2426c4640 1 -- 172.21.15.47:0/1418897558 shutdown_connections 2024-03-23T09:47:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.587+0000 7fd2426c4640 1 -- 172.21.15.47:0/1418897558 wait complete. 2024-03-23T09:47:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.588+0000 7fd2426c4640 1 Processor -- start 2024-03-23T09:47:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.588+0000 7fd2426c4640 1 -- start start 2024-03-23T09:47:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.588+0000 7fd2426c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd23c06a3f0 0x7fd23c1545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.588+0000 7fd2426c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd23c15c9d0 0x7fd23c154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.588+0000 7fd2426c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd23c15cdb0 0x7fd23c0b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.588+0000 7fd2426c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c14fa60 con 0x7fd23c15cdb0 2024-03-23T09:47:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.588+0000 7fd2426c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c155150 con 0x7fd23c06a3f0 2024-03-23T09:47:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.588+0000 7fd2426c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0caa80 con 0x7fd23c15c9d0 2024-03-23T09:47:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd240c61640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd23c15cdb0 0x7fd23c0b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd240c61640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd23c15cdb0 0x7fd23c0b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39278/0 (socket says 172.21.15.47:39278) 2024-03-23T09:47:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd240c61640 1 -- 172.21.15.47:0/349346858 learned_addr learned my addr 172.21.15.47:0/349346858 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd23bfff640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd23c06a3f0 0x7fd23c1545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd23b7fe640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd23c15c9d0 0x7fd23c154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd240c61640 1 -- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd23c15c9d0 msgr2=0x7fd23c154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd240c61640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd23c15c9d0 0x7fd23c154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd240c61640 1 -- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd23c06a3f0 msgr2=0x7fd23c1545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd240c61640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd23c06a3f0 0x7fd23c1545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd240c61640 1 -- 172.21.15.47:0/349346858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c009400 con 0x7fd23c15cdb0 2024-03-23T09:47:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd240c61640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd23c15cdb0 0x7fd23c0b0140 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c780 tx=0x7fd23400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd238ff9640 1 -- 172.21.15.47:0/349346858 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23400ce90 con 0x7fd23c15cdb0 2024-03-23T09:47:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd238ff9640 1 -- 172.21.15.47:0/349346858 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23400a640 con 0x7fd23c15cdb0 2024-03-23T09:47:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd2426c4640 1 -- 172.21.15.47:0/349346858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23c0cad60 con 0x7fd23c15cdb0 2024-03-23T09:47:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd238ff9640 1 -- 172.21.15.47:0/349346858 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23400aa70 con 0x7fd23c15cdb0 2024-03-23T09:47:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.589+0000 7fd2426c4640 1 -- 172.21.15.47:0/349346858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23c0cb320 con 0x7fd23c15cdb0 2024-03-23T09:47:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.590+0000 7fd2426c4640 1 -- 172.21.15.47:0/349346858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23c0cb510 con 0x7fd23c15cdb0 2024-03-23T09:47:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.591+0000 7fd238ff9640 1 -- 172.21.15.47:0/349346858 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd23400abd0 con 0x7fd23c15cdb0 2024-03-23T09:47:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.591+0000 7fd238ff9640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd22806ac00 0x7fd22806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.591+0000 7fd238ff9640 1 -- 172.21.15.47:0/349346858 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd234088890 con 0x7fd23c15cdb0 2024-03-23T09:47:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.591+0000 7fd23bfff640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd22806ac00 0x7fd22806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.592+0000 7fd23bfff640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd22806ac00 0x7fd22806d0c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd22c036000 tx=0x7fd22c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:08.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.595+0000 7fd238ff9640 1 -- 172.21.15.47:0/349346858 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd234053d30 con 0x7fd23c15cdb0 2024-03-23T09:47:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.777+0000 7fd2426c4640 1 -- 172.21.15.47:0/349346858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd23c066be0 con 0x7fd23c15cdb0 2024-03-23T09:47:08.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.778+0000 7fd238ff9640 1 -- 172.21.15.47:0/349346858 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+12390 (secure 0 0 0) 0x7fd234018070 con 0x7fd23c15cdb0 2024-03-23T09:47:08.778 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:08.778 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":81,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:47:08.005208+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:47:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.780+0000 7fd2426c4640 1 -- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd22806ac00 msgr2=0x7fd22806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.780+0000 7fd2426c4640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd22806ac00 0x7fd22806d0c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd22c036000 tx=0x7fd22c008030 comp rx=0 tx=0).stop 2024-03-23T09:47:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.780+0000 7fd2426c4640 1 -- 172.21.15.47:0/349346858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd23c15cdb0 msgr2=0x7fd23c0b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.780+0000 7fd2426c4640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd23c15cdb0 0x7fd23c0b0140 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c780 tx=0x7fd23400cc40 comp rx=0 tx=0).stop 2024-03-23T09:47:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.780+0000 7fd2426c4640 1 -- 172.21.15.47:0/349346858 shutdown_connections 2024-03-23T09:47:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.780+0000 7fd2426c4640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd22806ac00 0x7fd22806d0c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.780+0000 7fd2426c4640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd23c15cdb0 0x7fd23c0b0140 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.780+0000 7fd2426c4640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd23c15c9d0 0x7fd23c154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.780+0000 7fd2426c4640 1 --2- 172.21.15.47:0/349346858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd23c06a3f0 0x7fd23c1545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.780+0000 7fd2426c4640 1 -- 172.21.15.47:0/349346858 >> 172.21.15.47:0/349346858 conn(0x7fd23c0b13a0 msgr2=0x7fd23c0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.780+0000 7fd2426c4640 1 -- 172.21.15.47:0/349346858 shutdown_connections 2024-03-23T09:47:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.780+0000 7fd2426c4640 1 -- 172.21.15.47:0/349346858 wait complete. 2024-03-23T09:47:08.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:47:08.791 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:47:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.895+0000 7f320fa81640 1 Processor -- start 2024-03-23T09:47:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.896+0000 7f320fa81640 1 -- start start 2024-03-23T09:47:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.896+0000 7f320fa81640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3208143010 0x7f32080b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.896+0000 7f320fa81640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32080ccd00 con 0x7f32080b40e0 2024-03-23T09:47:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.896+0000 7f320fa81640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32080ccfe0 con 0x7f3208142a90 2024-03-23T09:47:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.896+0000 7f320fa81640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32080b27c0 con 0x7f3208143010 2024-03-23T09:47:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320e01e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f32080b40e0 0x7f320806c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48756/0 (socket says 172.21.15.47:48756) 2024-03-23T09:47:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320e01e640 1 -- 172.21.15.47:0/2958494015 learned_addr learned my addr 172.21.15.47:0/2958494015 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320d01c640 1 --2- 172.21.15.47:0/2958494015 >> v2:172.21.15.175:3300/0 conn(0x7f3208143010 0x7f32080b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561862469 0 0) 0x7f32080ccd00 con 0x7f32080b40e0 2024-03-23T09:47:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31f4003520 con 0x7f32080b40e0 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1690650635 0 0) 0x7f31f4003520 con 0x7f32080b40e0 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 >> v2:172.21.15.175:3300/0 conn(0x7f3208143010 msgr2=0x7f32080b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 --2- 172.21.15.47:0/2958494015 >> v2:172.21.15.175:3300/0 conn(0x7f3208143010 0x7f32080b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 >> v1:172.21.15.175:6789/0 conn(0x7f3208142a90 legacy=0x7f32080cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32080cd2c0 con 0x7f32080b40e0 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f32000019f0 con 0x7f32080b40e0 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 >> v1:172.21.15.47:6789/0 conn(0x7f32080b40e0 legacy=0x7f320806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 --2- 172.21.15.47:0/2958494015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f4003970 0x7f31f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 --2- 172.21.15.47:0/2958494015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f40047c0 0x7f31f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 --2- 172.21.15.47:0/2958494015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31f4005180 0x7f31f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32080b27c0 con 0x7f31f4005180 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f400c020 con 0x7f31f40047c0 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f400c1e0 con 0x7f31f4003970 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320e01e640 1 --2- 172.21.15.47:0/2958494015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31f4005180 0x7f31f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320d01c640 1 --2- 172.21.15.47:0/2958494015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f40047c0 0x7f31f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.897+0000 7f320d81d640 1 --2- 172.21.15.47:0/2958494015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f4003970 0x7f31f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.898+0000 7f320e01e640 1 -- 172.21.15.47:0/2958494015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f4003970 msgr2=0x7f31f4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:08.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.898+0000 7f320e01e640 1 --2- 172.21.15.47:0/2958494015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f4003970 0x7f31f4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.898+0000 7f320e01e640 1 -- 172.21.15.47:0/2958494015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f40047c0 msgr2=0x7f31f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:08.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.898+0000 7f320e01e640 1 --2- 172.21.15.47:0/2958494015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f40047c0 0x7f31f4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.898+0000 7f320e01e640 1 -- 172.21.15.47:0/2958494015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f4003520 con 0x7f31f4005180 2024-03-23T09:47:08.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.898+0000 7f320d81d640 1 --2- 172.21.15.47:0/2958494015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f4003970 0x7f31f4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:08.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.898+0000 7f320e01e640 1 -- 172.21.15.47:0/2958494015 reap_dead start 2024-03-23T09:47:08.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.898+0000 7f320e01e640 1 --2- 172.21.15.47:0/2958494015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31f4005180 0x7f31f400b980 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f320000a830 tx=0x7f31f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425e6d9e816515fe server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:08.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.899+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32000072d0 con 0x7f31f4005180 2024-03-23T09:47:08.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.899+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320002f070 con 0x7f31f4005180 2024-03-23T09:47:08.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.899+0000 7f320c81b640 1 -- 172.21.15.47:0/2958494015 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32000076b0 con 0x7f31f4005180 2024-03-23T09:47:08.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.899+0000 7f320fa81640 1 -- 172.21.15.47:0/2958494015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31f4005180 msgr2=0x7f31f400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.899+0000 7f320fa81640 1 --2- 172.21.15.47:0/2958494015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31f4005180 0x7f31f400b980 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f320000a830 tx=0x7f31f40057f0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.899+0000 7f320fa81640 1 -- 172.21.15.47:0/2958494015 shutdown_connections 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.899+0000 7f320fa81640 1 --2- 172.21.15.47:0/2958494015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31f4005180 0x7f31f400b980 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.899+0000 7f320fa81640 1 -- 172.21.15.47:0/2958494015 >> 172.21.15.47:0/2958494015 conn(0x7f320806a310 msgr2=0x7f32080b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.899+0000 7f320fa81640 1 -- 172.21.15.47:0/2958494015 shutdown_connections 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.899+0000 7f320fa81640 1 -- 172.21.15.47:0/2958494015 wait complete. 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.900+0000 7f320fa81640 1 Processor -- start 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.900+0000 7f320fa81640 1 -- start start 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320fa81640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f4005180 0x7f3208159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320fa81640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32080af650 0x7f32080afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320fa81640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32080abc10 0x7f32080ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320fa81640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32080b0010 con 0x7f32080af650 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320fa81640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3208140020 con 0x7f32080abc10 2024-03-23T09:47:08.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320fa81640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32081595a0 con 0x7f31f4005180 2024-03-23T09:47:08.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320d01c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32080af650 0x7f32080afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320d01c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32080af650 0x7f32080afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39302/0 (socket says 172.21.15.47:39302) 2024-03-23T09:47:08.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320d01c640 1 -- 172.21.15.47:0/845531273 learned_addr learned my addr 172.21.15.47:0/845531273 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:08.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320e01e640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32080abc10 0x7f32080ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320d81d640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f4005180 0x7f3208159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320d01c640 1 -- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f4005180 msgr2=0x7f3208159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320d01c640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f4005180 0x7f3208159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320d01c640 1 -- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32080abc10 msgr2=0x7f32080ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:08.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320d01c640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32080abc10 0x7f32080ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:08.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.901+0000 7f320d01c640 1 -- 172.21.15.47:0/845531273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320813fa10 con 0x7f32080af650 2024-03-23T09:47:08.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.902+0000 7f320d81d640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f4005180 0x7f3208159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:08.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.902+0000 7f320d01c640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32080af650 0x7f32080afad0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f31f80118e0 tx=0x7f31f8011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:08.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.902+0000 7f31fe7fc640 1 -- 172.21.15.47:0/845531273 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f800faf0 con 0x7f32080af650 2024-03-23T09:47:08.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.902+0000 7f31fe7fc640 1 -- 172.21.15.47:0/845531273 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f800fc50 con 0x7f32080af650 2024-03-23T09:47:08.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.902+0000 7f320fa81640 1 -- 172.21.15.47:0/845531273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32080ac9d0 con 0x7f32080af650 2024-03-23T09:47:08.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.902+0000 7f31fe7fc640 1 -- 172.21.15.47:0/845531273 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8014440 con 0x7f32080af650 2024-03-23T09:47:08.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.902+0000 7f320fa81640 1 -- 172.21.15.47:0/845531273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32080accb0 con 0x7f32080af650 2024-03-23T09:47:08.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.903+0000 7f320fa81640 1 -- 172.21.15.47:0/845531273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3208073480 con 0x7f32080af650 2024-03-23T09:47:08.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.903+0000 7f31fe7fc640 1 -- 172.21.15.47:0/845531273 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f31f80145a0 con 0x7f32080af650 2024-03-23T09:47:08.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.904+0000 7f31fe7fc640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f31ec06ac00 0x7f31ec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:08.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.904+0000 7f31fe7fc640 1 -- 172.21.15.47:0/845531273 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f31f8089810 con 0x7f32080af650 2024-03-23T09:47:08.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.904+0000 7f320d81d640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f31ec06ac00 0x7f31ec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:08.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.904+0000 7f320d81d640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f31ec06ac00 0x7f31ec06d0c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3204000c00 tx=0x7f3204009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:08.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:08.908+0000 7f31fe7fc640 1 -- 172.21.15.47:0/845531273 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f31f8054dd0 con 0x7f32080af650 2024-03-23T09:47:09.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.086+0000 7f320fa81640 1 -- 172.21.15.47:0/845531273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3208140190 con 0x7f32080af650 2024-03-23T09:47:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.087+0000 7f31fe7fc640 1 -- 172.21.15.47:0/845531273 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31f80583f0 con 0x7f32080af650 2024-03-23T09:47:09.087 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:47:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.089+0000 7f320fa81640 1 -- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f31ec06ac00 msgr2=0x7f31ec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.089+0000 7f320fa81640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f31ec06ac00 0x7f31ec06d0c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3204000c00 tx=0x7f3204009040 comp rx=0 tx=0).stop 2024-03-23T09:47:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.089+0000 7f320fa81640 1 -- 172.21.15.47:0/845531273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32080af650 msgr2=0x7f32080afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.089+0000 7f320fa81640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32080af650 0x7f32080afad0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f31f80118e0 tx=0x7f31f8011da0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.089+0000 7f320fa81640 1 -- 172.21.15.47:0/845531273 shutdown_connections 2024-03-23T09:47:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.089+0000 7f320fa81640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f31ec06ac00 0x7f31ec06d0c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.089+0000 7f320fa81640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32080abc10 0x7f32080ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.089+0000 7f320fa81640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32080af650 0x7f32080afad0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.089+0000 7f320fa81640 1 --2- 172.21.15.47:0/845531273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f4005180 0x7f3208159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.089+0000 7f320fa81640 1 -- 172.21.15.47:0/845531273 >> 172.21.15.47:0/845531273 conn(0x7f320806a310 msgr2=0x7f32080b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.089+0000 7f320fa81640 1 -- 172.21.15.47:0/845531273 shutdown_connections 2024-03-23T09:47:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.089+0000 7f320fa81640 1 -- 172.21.15.47:0/845531273 wait complete. 2024-03-23T09:47:09.099 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T09:47:09.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.200+0000 7fc83a3d2640 1 Processor -- start 2024-03-23T09:47:09.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.200+0000 7fc83a3d2640 1 -- start start 2024-03-23T09:47:09.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.201+0000 7fc83a3d2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8341516d0 0x7fc834069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.201+0000 7fc83a3d2640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc8340cdcb0 0x7fc83406a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.201+0000 7fc83a3d2640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8340b1cd0 con 0x7fc83415aca0 2024-03-23T09:47:09.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.201+0000 7fc83a3d2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83415bc10 con 0x7fc8341516d0 2024-03-23T09:47:09.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.201+0000 7fc83a3d2640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83415b930 con 0x7fc8340cdcb0 2024-03-23T09:47:09.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.201+0000 7fc833fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8341516d0 0x7fc834069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:09.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.201+0000 7fc833fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8341516d0 0x7fc834069ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39308/0 (socket says 172.21.15.47:39308) 2024-03-23T09:47:09.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.201+0000 7fc833fff640 1 -- 172.21.15.47:0/993950420 learned_addr learned my addr 172.21.15.47:0/993950420 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:09.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.201+0000 7fc833fff640 1 -- 172.21.15.47:0/993950420 >> v2:172.21.15.175:3300/0 conn(0x7fc8340cdcb0 msgr2=0x7fc83406a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:09.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.201+0000 7fc833fff640 1 --2- 172.21.15.47:0/993950420 >> v2:172.21.15.175:3300/0 conn(0x7fc8340cdcb0 0x7fc83406a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.201+0000 7fc833fff640 1 -- 172.21.15.47:0/993950420 >> v1:172.21.15.175:6789/0 conn(0x7fc83415aca0 legacy=0x7fc83415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:09.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.201+0000 7fc833fff640 1 -- 172.21.15.47:0/993950420 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8340cd1d0 con 0x7fc8341516d0 2024-03-23T09:47:09.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.202+0000 7fc833fff640 1 --2- 172.21.15.47:0/993950420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8341516d0 0x7fc834069ce0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc83406d060 tx=0x7fc81c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9835949d6a8bf5b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:09.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.202+0000 7fc832ffd640 1 -- 172.21.15.47:0/993950420 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0317e0 con 0x7fc8341516d0 2024-03-23T09:47:09.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.202+0000 7fc832ffd640 1 -- 172.21.15.47:0/993950420 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c031940 con 0x7fc8341516d0 2024-03-23T09:47:09.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.202+0000 7fc832ffd640 1 -- 172.21.15.47:0/993950420 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c031cd0 con 0x7fc8341516d0 2024-03-23T09:47:09.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.203+0000 7fc83a3d2640 1 -- 172.21.15.47:0/993950420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8341516d0 msgr2=0x7fc834069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:09.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.203+0000 7fc83a3d2640 1 --2- 172.21.15.47:0/993950420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8341516d0 0x7fc834069ce0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc83406d060 tx=0x7fc81c02b880 comp rx=0 tx=0).stop 2024-03-23T09:47:09.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.203+0000 7fc83a3d2640 1 -- 172.21.15.47:0/993950420 shutdown_connections 2024-03-23T09:47:09.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.203+0000 7fc83a3d2640 1 --2- 172.21.15.47:0/993950420 >> v2:172.21.15.175:3300/0 conn(0x7fc8340cdcb0 0x7fc83406a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.203+0000 7fc83a3d2640 1 --2- 172.21.15.47:0/993950420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8341516d0 0x7fc834069ce0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.203+0000 7fc83a3d2640 1 -- 172.21.15.47:0/993950420 >> 172.21.15.47:0/993950420 conn(0x7fc834098a00 msgr2=0x7fc8340b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:09.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.203+0000 7fc83a3d2640 1 -- 172.21.15.47:0/993950420 shutdown_connections 2024-03-23T09:47:09.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.203+0000 7fc83a3d2640 1 -- 172.21.15.47:0/993950420 wait complete. 2024-03-23T09:47:09.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.204+0000 7fc83a3d2640 1 Processor -- start 2024-03-23T09:47:09.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.204+0000 7fc83a3d2640 1 -- start start 2024-03-23T09:47:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc83a3d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340cdcb0 0x7fc83414acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc83a3d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8341516d0 0x7fc834157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc83a3d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc83415aca0 0x7fc8341e1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc83a3d2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc834157780 con 0x7fc8340cdcb0 2024-03-23T09:47:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc83a3d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc834156de0 con 0x7fc8341516d0 2024-03-23T09:47:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc83a3d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8341570c0 con 0x7fc83415aca0 2024-03-23T09:47:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc833fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340cdcb0 0x7fc83414acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc833fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340cdcb0 0x7fc83414acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39314/0 (socket says 172.21.15.47:39314) 2024-03-23T09:47:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc833fff640 1 -- 172.21.15.47:0/34163814 learned_addr learned my addr 172.21.15.47:0/34163814 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc8337fe640 1 --2- 172.21.15.47:0/34163814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8341516d0 0x7fc834157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc833fff640 1 -- 172.21.15.47:0/34163814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc83415aca0 msgr2=0x7fc8341e1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc833fff640 1 --2- 172.21.15.47:0/34163814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc83415aca0 0x7fc8341e1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc833fff640 1 -- 172.21.15.47:0/34163814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8341516d0 msgr2=0x7fc834157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc833fff640 1 --2- 172.21.15.47:0/34163814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8341516d0 0x7fc834157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc833fff640 1 -- 172.21.15.47:0/34163814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c0094d0 con 0x7fc8340cdcb0 2024-03-23T09:47:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.205+0000 7fc833fff640 1 --2- 172.21.15.47:0/34163814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340cdcb0 0x7fc83414acb0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc81c02b8c0 tx=0x7fc81c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.206+0000 7fc830ff9640 1 -- 172.21.15.47:0/34163814 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c032940 con 0x7fc8340cdcb0 2024-03-23T09:47:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.206+0000 7fc830ff9640 1 -- 172.21.15.47:0/34163814 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c032aa0 con 0x7fc8340cdcb0 2024-03-23T09:47:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.206+0000 7fc830ff9640 1 -- 172.21.15.47:0/34163814 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c032e50 con 0x7fc8340cdcb0 2024-03-23T09:47:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.206+0000 7fc83a3d2640 1 -- 172.21.15.47:0/34163814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8341573a0 con 0x7fc8340cdcb0 2024-03-23T09:47:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.206+0000 7fc83a3d2640 1 -- 172.21.15.47:0/34163814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8341e19a0 con 0x7fc8340cdcb0 2024-03-23T09:47:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.207+0000 7fc83a3d2640 1 -- 172.21.15.47:0/34163814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8341579f0 con 0x7fc8340cdcb0 2024-03-23T09:47:09.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.207+0000 7fc830ff9640 1 -- 172.21.15.47:0/34163814 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc81c0453f0 con 0x7fc8340cdcb0 2024-03-23T09:47:09.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.207+0000 7fc830ff9640 1 --2- 172.21.15.47:0/34163814 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc81406ac00 0x7fc81406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.208+0000 7fc830ff9640 1 -- 172.21.15.47:0/34163814 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc81c0ab4c0 con 0x7fc8340cdcb0 2024-03-23T09:47:09.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.208+0000 7fc8337fe640 1 --2- 172.21.15.47:0/34163814 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc81406ac00 0x7fc81406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:09.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.208+0000 7fc8337fe640 1 --2- 172.21.15.47:0/34163814 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc81406ac00 0x7fc81406d0c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc834150d70 tx=0x7fc824008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:09.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.212+0000 7fc830ff9640 1 -- 172.21.15.47:0/34163814 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc81c076b00 con 0x7fc8340cdcb0 2024-03-23T09:47:09.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.392+0000 7fc83a3d2640 1 -- 172.21.15.47:0/34163814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc834066be0 con 0x7fc8340cdcb0 2024-03-23T09:47:09.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.392+0000 7fc830ff9640 1 -- 172.21.15.47:0/34163814 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc81c07a120 con 0x7fc8340cdcb0 2024-03-23T09:47:09.393 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:09.393 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-23T09:47:09.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:47:09.393 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-03-23T09:47:09.393 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-03-23T09:47:09.393 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:09.393 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-23T09:47:09.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.394+0000 7fc83a3d2640 1 -- 172.21.15.47:0/34163814 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc81406ac00 msgr2=0x7fc81406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.394+0000 7fc83a3d2640 1 --2- 172.21.15.47:0/34163814 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc81406ac00 0x7fc81406d0c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc834150d70 tx=0x7fc824008040 comp rx=0 tx=0).stop 2024-03-23T09:47:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.394+0000 7fc83a3d2640 1 -- 172.21.15.47:0/34163814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340cdcb0 msgr2=0x7fc83414acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.394+0000 7fc83a3d2640 1 --2- 172.21.15.47:0/34163814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340cdcb0 0x7fc83414acb0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc81c02b8c0 tx=0x7fc81c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.394+0000 7fc83a3d2640 1 -- 172.21.15.47:0/34163814 shutdown_connections 2024-03-23T09:47:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.394+0000 7fc83a3d2640 1 --2- 172.21.15.47:0/34163814 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc81406ac00 0x7fc81406d0c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.394+0000 7fc83a3d2640 1 --2- 172.21.15.47:0/34163814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc83415aca0 0x7fc8341e1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.394+0000 7fc83a3d2640 1 --2- 172.21.15.47:0/34163814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8341516d0 0x7fc834157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.394+0000 7fc83a3d2640 1 --2- 172.21.15.47:0/34163814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340cdcb0 0x7fc83414acb0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.394+0000 7fc83a3d2640 1 -- 172.21.15.47:0/34163814 >> 172.21.15.47:0/34163814 conn(0x7fc834098a00 msgr2=0x7fc8340b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.394+0000 7fc83a3d2640 1 -- 172.21.15.47:0/34163814 shutdown_connections 2024-03-23T09:47:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.394+0000 7fc83a3d2640 1 -- 172.21.15.47:0/34163814 wait complete. 2024-03-23T09:47:09.405 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T09:47:09.405 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T09:47:09.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.510+0000 7f379ba26640 1 Processor -- start 2024-03-23T09:47:09.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.510+0000 7f379ba26640 1 -- start start 2024-03-23T09:47:09.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f379ba26640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f37940cf8a0 0x7f379415d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f379ba26640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3794160bd0 con 0x7f379415d0f0 2024-03-23T09:47:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f379ba26640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379415dbb0 con 0x7f379415a860 2024-03-23T09:47:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f379ba26640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37940d2790 con 0x7f37940cf8a0 2024-03-23T09:47:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f3798fc1640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f379415d0f0 0x7f37940cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48758/0 (socket says 172.21.15.47:48758) 2024-03-23T09:47:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f3798fc1640 1 -- 172.21.15.47:0/3239337711 learned_addr learned my addr 172.21.15.47:0/3239337711 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f3799fc3640 1 --2- 172.21.15.47:0/3239337711 >> v2:172.21.15.175:3300/0 conn(0x7f37940cf8a0 0x7f379415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145959618 0 0) 0x7f3794160bd0 con 0x7f379415d0f0 2024-03-23T09:47:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3780003520 con 0x7f379415d0f0 2024-03-23T09:47:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 904576365 0 0) 0x7f3780003520 con 0x7f379415d0f0 2024-03-23T09:47:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 >> v2:172.21.15.175:3300/0 conn(0x7f37940cf8a0 msgr2=0x7f379415d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f378bfff640 1 --2- 172.21.15.47:0/3239337711 >> v2:172.21.15.175:3300/0 conn(0x7f37940cf8a0 0x7f379415d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 >> v1:172.21.15.175:6789/0 conn(0x7f379415a860 legacy=0x7f3794161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379415de90 con 0x7f379415d0f0 2024-03-23T09:47:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.511+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f3784001670 con 0x7f379415d0f0 2024-03-23T09:47:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 >> v1:172.21.15.47:6789/0 conn(0x7f379415d0f0 legacy=0x7f37940cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f378bfff640 1 --2- 172.21.15.47:0/3239337711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3780003970 0x7f3780003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f378bfff640 1 --2- 172.21.15.47:0/3239337711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37800047c0 0x7f3780004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f378bfff640 1 --2- 172.21.15.47:0/3239337711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3780005180 0x7f378000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37940d2790 con 0x7f3780003970 2024-03-23T09:47:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378000c020 con 0x7f37800047c0 2024-03-23T09:47:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378000c1e0 con 0x7f3780005180 2024-03-23T09:47:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f37997c2640 1 --2- 172.21.15.47:0/3239337711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3780003970 0x7f3780003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f3798fc1640 1 --2- 172.21.15.47:0/3239337711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37800047c0 0x7f3780004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f3799fc3640 1 --2- 172.21.15.47:0/3239337711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3780005180 0x7f378000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f37997c2640 1 -- 172.21.15.47:0/3239337711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3780005180 msgr2=0x7f378000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f37997c2640 1 --2- 172.21.15.47:0/3239337711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3780005180 0x7f378000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f37997c2640 1 -- 172.21.15.47:0/3239337711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37800047c0 msgr2=0x7f3780004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f37997c2640 1 --2- 172.21.15.47:0/3239337711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37800047c0 0x7f3780004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f37997c2640 1 -- 172.21.15.47:0/3239337711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3780003520 con 0x7f3780003970 2024-03-23T09:47:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.512+0000 7f3799fc3640 1 -- 172.21.15.47:0/3239337711 reap_dead start 2024-03-23T09:47:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.513+0000 7f37997c2640 1 --2- 172.21.15.47:0/3239337711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3780003970 0x7f3780003d90 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3790001750 tx=0x7f37800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5151df776706190e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.513+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37900052d0 con 0x7f3780003970 2024-03-23T09:47:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.513+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379002e050 con 0x7f3780003970 2024-03-23T09:47:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.513+0000 7f378bfff640 1 -- 172.21.15.47:0/3239337711 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3790003830 con 0x7f3780003970 2024-03-23T09:47:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.513+0000 7f379ba26640 1 -- 172.21.15.47:0/3239337711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3780003970 msgr2=0x7f3780003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.513+0000 7f379ba26640 1 --2- 172.21.15.47:0/3239337711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3780003970 0x7f3780003d90 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3790001750 tx=0x7f37800057f0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.513+0000 7f379ba26640 1 -- 172.21.15.47:0/3239337711 shutdown_connections 2024-03-23T09:47:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.513+0000 7f379ba26640 1 --2- 172.21.15.47:0/3239337711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3780003970 0x7f3780003d90 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.513+0000 7f379ba26640 1 -- 172.21.15.47:0/3239337711 >> 172.21.15.47:0/3239337711 conn(0x7f37940659d0 msgr2=0x7f3794065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.514+0000 7f379ba26640 1 -- 172.21.15.47:0/3239337711 shutdown_connections 2024-03-23T09:47:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.514+0000 7f379ba26640 1 -- 172.21.15.47:0/3239337711 wait complete. 2024-03-23T09:47:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.515+0000 7f379ba26640 1 Processor -- start 2024-03-23T09:47:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.515+0000 7f379ba26640 1 -- start start 2024-03-23T09:47:09.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.515+0000 7f379ba26640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3780003970 0x7f3794152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.515+0000 7f379ba26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37940c7220 0x7f37941596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.515+0000 7f379ba26640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3794159c30 0x7f37940cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.515+0000 7f379ba26640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379415a0b0 con 0x7f37940c7220 2024-03-23T09:47:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.515+0000 7f379ba26640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37941533a0 con 0x7f3780003970 2024-03-23T09:47:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.515+0000 7f379ba26640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37940c64f0 con 0x7f3794159c30 2024-03-23T09:47:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f3798fc1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37940c7220 0x7f37941596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f3798fc1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37940c7220 0x7f37941596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39338/0 (socket says 172.21.15.47:39338) 2024-03-23T09:47:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f3798fc1640 1 -- 172.21.15.47:0/3544192233 learned_addr learned my addr 172.21.15.47:0/3544192233 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f37997c2640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3780003970 0x7f3794152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f3799fc3640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3794159c30 0x7f37940cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f3798fc1640 1 -- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3794159c30 msgr2=0x7f37940cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f3798fc1640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3794159c30 0x7f37940cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f3798fc1640 1 -- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3780003970 msgr2=0x7f3794152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:09.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f3798fc1640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3780003970 0x7f3794152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:09.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f3798fc1640 1 -- 172.21.15.47:0/3544192233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37940c5e00 con 0x7f37940c7220 2024-03-23T09:47:09.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f3798fc1640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37940c7220 0x7f37941596f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f37840072f0 tx=0x7f37840077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:09.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f3789ffb640 1 -- 172.21.15.47:0/3544192233 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784001650 con 0x7f37940c7220 2024-03-23T09:47:09.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f3789ffb640 1 -- 172.21.15.47:0/3544192233 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378400f930 con 0x7f37940c7220 2024-03-23T09:47:09.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f379ba26640 1 -- 172.21.15.47:0/3544192233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37940cc6f0 con 0x7f37940c7220 2024-03-23T09:47:09.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f3789ffb640 1 -- 172.21.15.47:0/3544192233 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400fca0 con 0x7f37940c7220 2024-03-23T09:47:09.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.516+0000 7f379ba26640 1 -- 172.21.15.47:0/3544192233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37940c6a20 con 0x7f37940c7220 2024-03-23T09:47:09.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.517+0000 7f379ba26640 1 -- 172.21.15.47:0/3544192233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3794072360 con 0x7f37940c7220 2024-03-23T09:47:09.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.518+0000 7f3789ffb640 1 -- 172.21.15.47:0/3544192233 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f378401e3b0 con 0x7f37940c7220 2024-03-23T09:47:09.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.518+0000 7f3789ffb640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f377806ac00 0x7f377806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:09.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.518+0000 7f3789ffb640 1 -- 172.21.15.47:0/3544192233 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3784088b90 con 0x7f37940c7220 2024-03-23T09:47:09.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.518+0000 7f37997c2640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f377806ac00 0x7f377806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:09.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.519+0000 7f37997c2640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f377806ac00 0x7f377806d0c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3790033000 tx=0x7f3790009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:09.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.522+0000 7f3789ffb640 1 -- 172.21.15.47:0/3544192233 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3784054170 con 0x7f37940c7220 2024-03-23T09:47:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:09.702+0000 7f379ba26640 1 -- 172.21.15.47:0/3544192233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3794066be0 con 0x7f37940c7220 2024-03-23T09:47:10.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.016+0000 7f3789ffb640 1 -- 172.21.15.47:0/3544192233 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f378401a070 con 0x7f37940c7220 2024-03-23T09:47:10.016 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-03-23T09:47:10.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.020+0000 7f379ba26640 1 -- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f377806ac00 msgr2=0x7f377806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:10.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.020+0000 7f379ba26640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f377806ac00 0x7f377806d0c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3790033000 tx=0x7f3790009030 comp rx=0 tx=0).stop 2024-03-23T09:47:10.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.020+0000 7f379ba26640 1 -- 172.21.15.47:0/3544192233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37940c7220 msgr2=0x7f37941596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:10.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.020+0000 7f379ba26640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37940c7220 0x7f37941596f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f37840072f0 tx=0x7f37840077b0 comp rx=0 tx=0).stop 2024-03-23T09:47:10.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.020+0000 7f379ba26640 1 -- 172.21.15.47:0/3544192233 shutdown_connections 2024-03-23T09:47:10.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.020+0000 7f379ba26640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f377806ac00 0x7f377806d0c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:10.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.020+0000 7f379ba26640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3794159c30 0x7f37940cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:10.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.020+0000 7f379ba26640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37940c7220 0x7f37941596f0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:10.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.020+0000 7f379ba26640 1 --2- 172.21.15.47:0/3544192233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3780003970 0x7f3794152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:10.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.020+0000 7f379ba26640 1 -- 172.21.15.47:0/3544192233 >> 172.21.15.47:0/3544192233 conn(0x7f37940659d0 msgr2=0x7f379415f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:10.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.020+0000 7f379ba26640 1 -- 172.21.15.47:0/3544192233 shutdown_connections 2024-03-23T09:47:10.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.020+0000 7f379ba26640 1 -- 172.21.15.47:0/3544192233 wait complete. 2024-03-23T09:47:10.033 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T09:47:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.141+0000 7f9be073c640 1 Processor -- start 2024-03-23T09:47:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.141+0000 7f9be073c640 1 -- start start 2024-03-23T09:47:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.142+0000 7f9be073c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9bd815a890 0x7f9bd80cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.142+0000 7f9be073c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9bd80cebe0 0x7f9bd80cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.142+0000 7f9be073c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd80cf5d0 con 0x7f9bd815e4f0 2024-03-23T09:47:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.142+0000 7f9be073c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd80ce510 con 0x7f9bd80cebe0 2024-03-23T09:47:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.142+0000 7f9be073c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd80cb530 con 0x7f9bd815a890 2024-03-23T09:47:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.142+0000 7f9bddcd7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9bd80cebe0 0x7f9bd80cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.142+0000 7f9bddcd7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9bd80cebe0 0x7f9bd80cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39342/0 (socket says 172.21.15.47:39342) 2024-03-23T09:47:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.142+0000 7f9bddcd7640 1 -- 172.21.15.47:0/3409894188 learned_addr learned my addr 172.21.15.47:0/3409894188 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.143+0000 7f9bde4d8640 1 --2- 172.21.15.47:0/3409894188 >> v2:172.21.15.175:3300/0 conn(0x7f9bd815a890 0x7f9bd80cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.143+0000 7f9bddcd7640 1 -- 172.21.15.47:0/3409894188 >> v2:172.21.15.175:3300/0 conn(0x7f9bd815a890 msgr2=0x7f9bd80cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.143+0000 7f9bddcd7640 1 --2- 172.21.15.47:0/3409894188 >> v2:172.21.15.175:3300/0 conn(0x7f9bd815a890 0x7f9bd80cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.143+0000 7f9bddcd7640 1 -- 172.21.15.47:0/3409894188 >> v1:172.21.15.175:6789/0 conn(0x7f9bd815e4f0 legacy=0x7f9bd815e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.143+0000 7f9bddcd7640 1 -- 172.21.15.47:0/3409894188 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd80cb810 con 0x7f9bd80cebe0 2024-03-23T09:47:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.144+0000 7f9bddcd7640 1 --2- 172.21.15.47:0/3409894188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bd80cebe0 0x7f9bd80cf090 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80caf40 tx=0x7f9bcc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a681df0496fa31 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.144+0000 7f9bdd4d6640 1 -- 172.21.15.47:0/3409894188 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc030860 con 0x7f9bd80cebe0 2024-03-23T09:47:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.144+0000 7f9bdd4d6640 1 -- 172.21.15.47:0/3409894188 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bcc0309c0 con 0x7f9bd80cebe0 2024-03-23T09:47:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.144+0000 7f9bdd4d6640 1 -- 172.21.15.47:0/3409894188 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc030d50 con 0x7f9bd80cebe0 2024-03-23T09:47:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.144+0000 7f9be073c640 1 -- 172.21.15.47:0/3409894188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bd80cebe0 msgr2=0x7f9bd80cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.144+0000 7f9be073c640 1 --2- 172.21.15.47:0/3409894188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bd80cebe0 0x7f9bd80cf090 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80caf40 tx=0x7f9bcc02b9a0 comp rx=0 tx=0).stop 2024-03-23T09:47:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.144+0000 7f9be073c640 1 -- 172.21.15.47:0/3409894188 shutdown_connections 2024-03-23T09:47:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.144+0000 7f9be073c640 1 --2- 172.21.15.47:0/3409894188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bd80cebe0 0x7f9bd80cf090 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.144+0000 7f9be073c640 1 --2- 172.21.15.47:0/3409894188 >> v2:172.21.15.175:3300/0 conn(0x7f9bd815a890 0x7f9bd80cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.144+0000 7f9be073c640 1 -- 172.21.15.47:0/3409894188 >> 172.21.15.47:0/3409894188 conn(0x7f9bd80d0680 msgr2=0x7f9bd80d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.144+0000 7f9be073c640 1 -- 172.21.15.47:0/3409894188 shutdown_connections 2024-03-23T09:47:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.144+0000 7f9be073c640 1 -- 172.21.15.47:0/3409894188 wait complete. 2024-03-23T09:47:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.145+0000 7f9be073c640 1 Processor -- start 2024-03-23T09:47:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.146+0000 7f9be073c640 1 -- start start 2024-03-23T09:47:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.146+0000 7f9be073c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bd80cebe0 0x7f9bd80c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.146+0000 7f9be073c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bd815a890 0x7f9bd80c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.146+0000 7f9be073c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd815e4f0 0x7f9bd80c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.146+0000 7f9be073c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80c8710 con 0x7f9bd815a890 2024-03-23T09:47:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.146+0000 7f9be073c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd8150180 con 0x7f9bd815e4f0 2024-03-23T09:47:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.146+0000 7f9be073c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80b3cf0 con 0x7f9bd80cebe0 2024-03-23T09:47:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.146+0000 7f9bddcd7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bd815a890 0x7f9bd80c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.146+0000 7f9bddcd7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bd815a890 0x7f9bd80c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39354/0 (socket says 172.21.15.47:39354) 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.146+0000 7f9bddcd7640 1 -- 172.21.15.47:0/1186033562 learned_addr learned my addr 172.21.15.47:0/1186033562 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.146+0000 7f9bdecd9640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd815e4f0 0x7f9bd80c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.146+0000 7f9bde4d8640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bd80cebe0 0x7f9bd80c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.147+0000 7f9bddcd7640 1 -- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bd80cebe0 msgr2=0x7f9bd80c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.147+0000 7f9bddcd7640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bd80cebe0 0x7f9bd80c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.147+0000 7f9bddcd7640 1 -- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd815e4f0 msgr2=0x7f9bd80c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.147+0000 7f9bddcd7640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd815e4f0 0x7f9bd80c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.147+0000 7f9bddcd7640 1 -- 172.21.15.47:0/1186033562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc0094d0 con 0x7f9bd815a890 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.147+0000 7f9bddcd7640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bd815a890 0x7f9bd80c7c90 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc02b9e0 tx=0x7f9bcc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.147+0000 7f9bd6ffd640 1 -- 172.21.15.47:0/1186033562 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc039b60 con 0x7f9bd815a890 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.147+0000 7f9bd6ffd640 1 -- 172.21.15.47:0/1186033562 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bcc039ce0 con 0x7f9bd815a890 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.147+0000 7f9be073c640 1 -- 172.21.15.47:0/1186033562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd80b3fd0 con 0x7f9bd815a890 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.147+0000 7f9bd6ffd640 1 -- 172.21.15.47:0/1186033562 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc043650 con 0x7f9bd815a890 2024-03-23T09:47:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.147+0000 7f9be073c640 1 -- 172.21.15.47:0/1186033562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd80b4590 con 0x7f9bd815a890 2024-03-23T09:47:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.148+0000 7f9be073c640 1 -- 172.21.15.47:0/1186033562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd8066be0 con 0x7f9bd815a890 2024-03-23T09:47:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.149+0000 7f9bd6ffd640 1 -- 172.21.15.47:0/1186033562 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9bcc04c3b0 con 0x7f9bd815a890 2024-03-23T09:47:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.149+0000 7f9bd6ffd640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9bc406ac00 0x7f9bc406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.149+0000 7f9bd6ffd640 1 -- 172.21.15.47:0/1186033562 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9bcc0ab630 con 0x7f9bd815a890 2024-03-23T09:47:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.149+0000 7f9bde4d8640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9bc406ac00 0x7f9bc406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.150+0000 7f9bde4d8640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9bc406ac00 0x7f9bc406d0c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00079a0 tx=0x7f9bd0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:10.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.154+0000 7f9bd6ffd640 1 -- 172.21.15.47:0/1186033562 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9bcc076b30 con 0x7f9bd815a890 2024-03-23T09:47:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:10.334+0000 7f9be073c640 1 -- 172.21.15.47:0/1186033562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9bd80cf0c0 con 0x7f9bd815a890 2024-03-23T09:47:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.023+0000 7f9bd6ffd640 1 -- 172.21.15.47:0/1186033562 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9bcc0480d0 con 0x7f9bd815a890 2024-03-23T09:47:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-03-23T09:47:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.028+0000 7f9be073c640 1 -- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9bc406ac00 msgr2=0x7f9bc406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.029+0000 7f9be073c640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9bc406ac00 0x7f9bc406d0c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00079a0 tx=0x7f9bd0008040 comp rx=0 tx=0).stop 2024-03-23T09:47:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.029+0000 7f9be073c640 1 -- 172.21.15.47:0/1186033562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bd815a890 msgr2=0x7f9bd80c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.029+0000 7f9be073c640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bd815a890 0x7f9bd80c7c90 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc02b9e0 tx=0x7f9bcc004730 comp rx=0 tx=0).stop 2024-03-23T09:47:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.030+0000 7f9be073c640 1 -- 172.21.15.47:0/1186033562 shutdown_connections 2024-03-23T09:47:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.030+0000 7f9be073c640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9bc406ac00 0x7f9bc406d0c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.030+0000 7f9be073c640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd815e4f0 0x7f9bd80c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.030+0000 7f9be073c640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bd815a890 0x7f9bd80c7c90 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.030+0000 7f9be073c640 1 --2- 172.21.15.47:0/1186033562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bd80cebe0 0x7f9bd80c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.030+0000 7f9be073c640 1 -- 172.21.15.47:0/1186033562 >> 172.21.15.47:0/1186033562 conn(0x7f9bd80d0680 msgr2=0x7f9bd806df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.034+0000 7f9be073c640 1 -- 172.21.15.47:0/1186033562 shutdown_connections 2024-03-23T09:47:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.036+0000 7f9be073c640 1 -- 172.21.15.47:0/1186033562 wait complete. 2024-03-23T09:47:11.052 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T09:47:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.157+0000 7f484b795640 1 Processor -- start 2024-03-23T09:47:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.157+0000 7f484b795640 1 -- start start 2024-03-23T09:47:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.158+0000 7f484b795640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f484413d810 0x7f48440cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.158+0000 7f484b795640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f48440701f0 0x7f4844070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.158+0000 7f484b795640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4844140db0 con 0x7f4844140210 2024-03-23T09:47:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.158+0000 7f484b795640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484413dbf0 con 0x7f48440701f0 2024-03-23T09:47:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.158+0000 7f484b795640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48440a7670 con 0x7f484413d810 2024-03-23T09:47:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.158+0000 7f4849531640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f484413d810 0x7f48440cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.158+0000 7f4849531640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f484413d810 0x7f48440cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:49478/0 (socket says 172.21.15.47:49478) 2024-03-23T09:47:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.158+0000 7f4848d30640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f48440701f0 0x7f4844070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.158+0000 7f4849531640 1 -- 172.21.15.47:0/4231419393 learned_addr learned my addr 172.21.15.47:0/4231419393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.159+0000 7f4849531640 1 -- 172.21.15.47:0/4231419393 >> v2:172.21.15.175:3300/0 conn(0x7f48440701f0 msgr2=0x7f4844070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.159+0000 7f4849531640 1 --2- 172.21.15.47:0/4231419393 >> v2:172.21.15.175:3300/0 conn(0x7f48440701f0 0x7f4844070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.159+0000 7f4849531640 1 -- 172.21.15.47:0/4231419393 >> v1:172.21.15.175:6789/0 conn(0x7f4844140210 legacy=0x7f48441406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.159+0000 7f4849531640 1 -- 172.21.15.47:0/4231419393 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484413ea70 con 0x7f484413d810 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.160+0000 7f4849531640 1 --2- 172.21.15.47:0/4231419393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f484413d810 0x7f48440cc4b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f483c009870 tx=0x7f483c02ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9ad29eb6884ffef server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.160+0000 7f483bfff640 1 -- 172.21.15.47:0/4231419393 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c02f840 con 0x7f484413d810 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.160+0000 7f483bfff640 1 -- 172.21.15.47:0/4231419393 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c02f9a0 con 0x7f484413d810 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.160+0000 7f483bfff640 1 -- 172.21.15.47:0/4231419393 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c02fd30 con 0x7f484413d810 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.160+0000 7f484b795640 1 -- 172.21.15.47:0/4231419393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f484413d810 msgr2=0x7f48440cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.160+0000 7f484b795640 1 --2- 172.21.15.47:0/4231419393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f484413d810 0x7f48440cc4b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f483c009870 tx=0x7f483c02ba20 comp rx=0 tx=0).stop 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.160+0000 7f484b795640 1 -- 172.21.15.47:0/4231419393 shutdown_connections 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.160+0000 7f484b795640 1 --2- 172.21.15.47:0/4231419393 >> v2:172.21.15.175:3300/0 conn(0x7f48440701f0 0x7f4844070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.160+0000 7f484b795640 1 --2- 172.21.15.47:0/4231419393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f484413d810 0x7f48440cc4b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.160+0000 7f484b795640 1 -- 172.21.15.47:0/4231419393 >> 172.21.15.47:0/4231419393 conn(0x7f484406a310 msgr2=0x7f48440b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.160+0000 7f484b795640 1 -- 172.21.15.47:0/4231419393 shutdown_connections 2024-03-23T09:47:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.160+0000 7f484b795640 1 -- 172.21.15.47:0/4231419393 wait complete. 2024-03-23T09:47:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.161+0000 7f484b795640 1 Processor -- start 2024-03-23T09:47:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.162+0000 7f484b795640 1 -- start start 2024-03-23T09:47:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.162+0000 7f484b795640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48440701f0 0x7f4844131720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.162+0000 7f484b795640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f484413d810 0x7f48441ef270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.162+0000 7f484b795640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4844140210 0x7f48441ef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.162+0000 7f484b795640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484412fb40 con 0x7f4844140210 2024-03-23T09:47:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.162+0000 7f484b795640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48441329f0 con 0x7f484413d810 2024-03-23T09:47:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.162+0000 7f484b795640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484415df30 con 0x7f48440701f0 2024-03-23T09:47:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.162+0000 7f4849d32640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4844140210 0x7f48441ef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.162+0000 7f4849d32640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4844140210 0x7f48441ef7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39370/0 (socket says 172.21.15.47:39370) 2024-03-23T09:47:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.162+0000 7f4849d32640 1 -- 172.21.15.47:0/1075545007 learned_addr learned my addr 172.21.15.47:0/1075545007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.162+0000 7f4849531640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48440701f0 0x7f4844131720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.162+0000 7f4848d30640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f484413d810 0x7f48441ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.163+0000 7f4849d32640 1 -- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48440701f0 msgr2=0x7f4844131720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.163+0000 7f4849d32640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48440701f0 0x7f4844131720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.163+0000 7f4849d32640 1 -- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f484413d810 msgr2=0x7f48441ef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.163+0000 7f4849d32640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f484413d810 0x7f48441ef270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.163+0000 7f4849d32640 1 -- 172.21.15.47:0/1075545007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c0094d0 con 0x7f4844140210 2024-03-23T09:47:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.163+0000 7f4849d32640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4844140210 0x7f48441ef7b0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f484000c7a0 tx=0x7f484000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.163+0000 7f4839ffb640 1 -- 172.21.15.47:0/1075545007 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840003b70 con 0x7f4844140210 2024-03-23T09:47:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.163+0000 7f4839ffb640 1 -- 172.21.15.47:0/1075545007 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4840003cf0 con 0x7f4844140210 2024-03-23T09:47:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.163+0000 7f484b795640 1 -- 172.21.15.47:0/1075545007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484415e210 con 0x7f4844140210 2024-03-23T09:47:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.163+0000 7f4839ffb640 1 -- 172.21.15.47:0/1075545007 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840003ea0 con 0x7f4844140210 2024-03-23T09:47:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.163+0000 7f484b795640 1 -- 172.21.15.47:0/1075545007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484415e7d0 con 0x7f4844140210 2024-03-23T09:47:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.164+0000 7f484b795640 1 -- 172.21.15.47:0/1075545007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4844073480 con 0x7f4844140210 2024-03-23T09:47:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.165+0000 7f4839ffb640 1 -- 172.21.15.47:0/1075545007 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f484000aa90 con 0x7f4844140210 2024-03-23T09:47:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.165+0000 7f4839ffb640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f482806ac00 0x7f482806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.165+0000 7f4839ffb640 1 -- 172.21.15.47:0/1075545007 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f48400890d0 con 0x7f4844140210 2024-03-23T09:47:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.165+0000 7f4849531640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f482806ac00 0x7f482806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.166+0000 7f4849531640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f482806ac00 0x7f482806d0c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f483c036000 tx=0x7f483c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:11.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.170+0000 7f4839ffb640 1 -- 172.21.15.47:0/1075545007 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4840054310 con 0x7f4844140210 2024-03-23T09:47:11.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.388+0000 7f484b795640 1 -- 172.21.15.47:0/1075545007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4844066be0 con 0x7f4844140210 2024-03-23T09:47:11.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.394+0000 7f4839ffb640 1 -- 172.21.15.47:0/1075545007 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v44) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4840057930 con 0x7f4844140210 2024-03-23T09:47:11.394 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 12 and data pool 13 2024-03-23T09:47:11.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.399+0000 7f484b795640 1 -- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f482806ac00 msgr2=0x7f482806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.399+0000 7f484b795640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f482806ac00 0x7f482806d0c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f483c036000 tx=0x7f483c008030 comp rx=0 tx=0).stop 2024-03-23T09:47:11.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.399+0000 7f484b795640 1 -- 172.21.15.47:0/1075545007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4844140210 msgr2=0x7f48441ef7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.399+0000 7f484b795640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4844140210 0x7f48441ef7b0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f484000c7a0 tx=0x7f484000cc60 comp rx=0 tx=0).stop 2024-03-23T09:47:11.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.400+0000 7f484b795640 1 -- 172.21.15.47:0/1075545007 shutdown_connections 2024-03-23T09:47:11.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.400+0000 7f484b795640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f482806ac00 0x7f482806d0c0 secure :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f483c036000 tx=0x7f483c008030 comp rx=0 tx=0).stop 2024-03-23T09:47:11.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.400+0000 7f484b795640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4844140210 0x7f48441ef7b0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.400+0000 7f484b795640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f484413d810 0x7f48441ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.400+0000 7f484b795640 1 --2- 172.21.15.47:0/1075545007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48440701f0 0x7f4844131720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.400+0000 7f484b795640 1 -- 172.21.15.47:0/1075545007 >> 172.21.15.47:0/1075545007 conn(0x7f484406a310 msgr2=0x7f48440aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.402+0000 7f484b795640 1 -- 172.21.15.47:0/1075545007 shutdown_connections 2024-03-23T09:47:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.403+0000 7f484b795640 1 -- 172.21.15.47:0/1075545007 wait complete. 2024-03-23T09:47:11.424 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:47:11.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.524+0000 7fabe2f89640 1 Processor -- start 2024-03-23T09:47:11.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.524+0000 7fabe2f89640 1 -- start start 2024-03-23T09:47:11.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.525+0000 7fabe2f89640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fabdc143010 0x7fabdc0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.525+0000 7fabe2f89640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc0b40e0 0x7fabdc06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.525+0000 7fabe2f89640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabdc0ccd00 con 0x7fabdc142a90 2024-03-23T09:47:11.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.525+0000 7fabe2f89640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabdc0b27c0 con 0x7fabdc143010 2024-03-23T09:47:11.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.525+0000 7fabe2f89640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc142e70 con 0x7fabdc0b40e0 2024-03-23T09:47:11.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.525+0000 7fabdbfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fabdc143010 0x7fabdc0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.525+0000 7fabdbfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fabdc143010 0x7fabdc0b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39380/0 (socket says 172.21.15.47:39380) 2024-03-23T09:47:11.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.525+0000 7fabdbfff640 1 -- 172.21.15.47:0/1353478422 learned_addr learned my addr 172.21.15.47:0/1353478422 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:11.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.525+0000 7fabe0d25640 1 --1- 172.21.15.47:0/1353478422 >> v1:172.21.15.47:6789/0 conn(0x7fabdc142a90 0x7fabdc0cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48760/0 (socket says 172.21.15.47:48760) 2024-03-23T09:47:11.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.526+0000 7fabe1526640 1 --2- 172.21.15.47:0/1353478422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc0b40e0 0x7fabdc06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.526+0000 7fabdbfff640 1 -- 172.21.15.47:0/1353478422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc0b40e0 msgr2=0x7fabdc06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:11.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.526+0000 7fabdbfff640 1 --2- 172.21.15.47:0/1353478422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc0b40e0 0x7fabdc06c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.526+0000 7fabdbfff640 1 -- 172.21.15.47:0/1353478422 >> v1:172.21.15.47:6789/0 conn(0x7fabdc142a90 legacy=0x7fabdc0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.526+0000 7fabdbfff640 1 -- 172.21.15.47:0/1353478422 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc0ccfe0 con 0x7fabdc143010 2024-03-23T09:47:11.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.526+0000 7fabdb7fe640 1 -- 172.21.15.47:0/1353478422 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527290692 0 0) 0x7fabdc0ccd00 con 0x7fabdc142a90 2024-03-23T09:47:11.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.526+0000 7fabdbfff640 1 --2- 172.21.15.47:0/1353478422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabdc143010 0x7fabdc0b3ba0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fabd00098a0 tx=0x7fabd002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bebc0b5b5eaf47 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:11.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.526+0000 7fabdb7fe640 1 -- 172.21.15.47:0/1353478422 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0030840 con 0x7fabdc143010 2024-03-23T09:47:11.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.527+0000 7fabdb7fe640 1 -- 172.21.15.47:0/1353478422 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd00309a0 con 0x7fabdc143010 2024-03-23T09:47:11.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.527+0000 7fabdb7fe640 1 -- 172.21.15.47:0/1353478422 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0030d30 con 0x7fabdc143010 2024-03-23T09:47:11.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.527+0000 7fabe2f89640 1 -- 172.21.15.47:0/1353478422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabdc143010 msgr2=0x7fabdc0b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.527+0000 7fabe2f89640 1 --2- 172.21.15.47:0/1353478422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabdc143010 0x7fabdc0b3ba0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fabd00098a0 tx=0x7fabd002b980 comp rx=0 tx=0).stop 2024-03-23T09:47:11.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.527+0000 7fabe2f89640 1 -- 172.21.15.47:0/1353478422 shutdown_connections 2024-03-23T09:47:11.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.527+0000 7fabe2f89640 1 --2- 172.21.15.47:0/1353478422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc0b40e0 0x7fabdc06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.527+0000 7fabe2f89640 1 --2- 172.21.15.47:0/1353478422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabdc143010 0x7fabdc0b3ba0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.527+0000 7fabe2f89640 1 -- 172.21.15.47:0/1353478422 >> 172.21.15.47:0/1353478422 conn(0x7fabdc06a310 msgr2=0x7fabdc0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:11.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.527+0000 7fabe2f89640 1 -- 172.21.15.47:0/1353478422 shutdown_connections 2024-03-23T09:47:11.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.527+0000 7fabe2f89640 1 -- 172.21.15.47:0/1353478422 wait complete. 2024-03-23T09:47:11.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.528+0000 7fabe2f89640 1 Processor -- start 2024-03-23T09:47:11.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.528+0000 7fabe2f89640 1 -- start start 2024-03-23T09:47:11.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe2f89640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabdc0b40e0 0x7fabdc1586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe2f89640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc142a90 0x7fabdc13aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe2f89640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabdc143010 0x7fabdc13df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe2f89640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc158c10 con 0x7fabdc0b40e0 2024-03-23T09:47:11.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe2f89640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc138040 con 0x7fabdc143010 2024-03-23T09:47:11.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe2f89640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc138320 con 0x7fabdc142a90 2024-03-23T09:47:11.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe0d25640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabdc0b40e0 0x7fabdc1586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe0d25640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabdc0b40e0 0x7fabdc1586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39392/0 (socket says 172.21.15.47:39392) 2024-03-23T09:47:11.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe0d25640 1 -- 172.21.15.47:0/775904185 learned_addr learned my addr 172.21.15.47:0/775904185 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe1526640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabdc143010 0x7fabdc13df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabdbfff640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc142a90 0x7fabdc13aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe0d25640 1 -- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc142a90 msgr2=0x7fabdc13aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe0d25640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc142a90 0x7fabdc13aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe0d25640 1 -- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabdc143010 msgr2=0x7fabdc13df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe0d25640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabdc143010 0x7fabdc13df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe0d25640 1 -- 172.21.15.47:0/775904185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd00094d0 con 0x7fabdc0b40e0 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabe1526640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabdc143010 0x7fabdc13df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.529+0000 7fabdbfff640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc142a90 0x7fabdc13aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.530+0000 7fabe0d25640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabdc0b40e0 0x7fabdc1586d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fabd400c8c0 tx=0x7fabd400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.530+0000 7fabd97fa640 1 -- 172.21.15.47:0/775904185 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd400a640 con 0x7fabdc0b40e0 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.530+0000 7fabd97fa640 1 -- 172.21.15.47:0/775904185 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd400a7c0 con 0x7fabdc0b40e0 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.530+0000 7fabd97fa640 1 -- 172.21.15.47:0/775904185 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd400ab70 con 0x7fabdc0b40e0 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.530+0000 7fabe2f89640 1 -- 172.21.15.47:0/775904185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc13e920 con 0x7fabdc0b40e0 2024-03-23T09:47:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.530+0000 7fabe2f89640 1 -- 172.21.15.47:0/775904185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc13eee0 con 0x7fabdc0b40e0 2024-03-23T09:47:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.531+0000 7fabe2f89640 1 -- 172.21.15.47:0/775904185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc073480 con 0x7fabdc0b40e0 2024-03-23T09:47:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.531+0000 7fabd97fa640 1 -- 172.21.15.47:0/775904185 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fabd400acd0 con 0x7fabdc0b40e0 2024-03-23T09:47:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.532+0000 7fabd97fa640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fabc806ac00 0x7fabc806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.532+0000 7fabd97fa640 1 -- 172.21.15.47:0/775904185 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabd4088940 con 0x7fabdc0b40e0 2024-03-23T09:47:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.532+0000 7fabdbfff640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fabc806ac00 0x7fabc806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.532+0000 7fabdbfff640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fabc806ac00 0x7fabc806d0c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fabd0036000 tx=0x7fabd003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:11.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.536+0000 7fabd97fa640 1 -- 172.21.15.47:0/775904185 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fabd4053bb0 con 0x7fabdc0b40e0 2024-03-23T09:47:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.716+0000 7fabe2f89640 1 -- 172.21.15.47:0/775904185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabdc066be0 con 0x7fabdc0b40e0 2024-03-23T09:47:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.717+0000 7fabd97fa640 1 -- 172.21.15.47:0/775904185 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16105 (secure 0 0 0) 0x7fabd4018070 con 0x7fabdc0b40e0 2024-03-23T09:47:11.717 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:11.717 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":84,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:47:11.390099+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:47:09.722391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-23T09:47:10.354054+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:47:11.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.719+0000 7fabe2f89640 1 -- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fabc806ac00 msgr2=0x7fabc806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.719+0000 7fabe2f89640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fabc806ac00 0x7fabc806d0c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fabd0036000 tx=0x7fabd003b040 comp rx=0 tx=0).stop 2024-03-23T09:47:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.719+0000 7fabe2f89640 1 -- 172.21.15.47:0/775904185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabdc0b40e0 msgr2=0x7fabdc1586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.719+0000 7fabe2f89640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabdc0b40e0 0x7fabdc1586d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fabd400c8c0 tx=0x7fabd400cd80 comp rx=0 tx=0).stop 2024-03-23T09:47:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.719+0000 7fabe2f89640 1 -- 172.21.15.47:0/775904185 shutdown_connections 2024-03-23T09:47:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.719+0000 7fabe2f89640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fabc806ac00 0x7fabc806d0c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.719+0000 7fabe2f89640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabdc143010 0x7fabdc13df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.719+0000 7fabe2f89640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc142a90 0x7fabdc13aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.719+0000 7fabe2f89640 1 --2- 172.21.15.47:0/775904185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabdc0b40e0 0x7fabdc1586d0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.719+0000 7fabe2f89640 1 -- 172.21.15.47:0/775904185 >> 172.21.15.47:0/775904185 conn(0x7fabdc06a310 msgr2=0x7fabdc0af760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.719+0000 7fabe2f89640 1 -- 172.21.15.47:0/775904185 shutdown_connections 2024-03-23T09:47:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.719+0000 7fabe2f89640 1 -- 172.21.15.47:0/775904185 wait complete. 2024-03-23T09:47:11.730 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:47:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.830+0000 7fa9bb3b9640 1 Processor -- start 2024-03-23T09:47:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.831+0000 7fa9bb3b9640 1 -- start start 2024-03-23T09:47:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.831+0000 7fa9bb3b9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9b40afe10 0x7fa9b4069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.831+0000 7fa9bb3b9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa9b40b2390 0x7fa9b406a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.831+0000 7fa9bb3b9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b40cd110 0x7fa9b415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.831+0000 7fa9bb3b9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b40cf320 con 0x7fa9b40afe10 2024-03-23T09:47:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.831+0000 7fa9bb3b9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b40ced60 con 0x7fa9b40b2390 2024-03-23T09:47:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.831+0000 7fa9bb3b9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b40cf040 con 0x7fa9b40cd110 2024-03-23T09:47:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.832+0000 7fa9b9155640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9b40afe10 0x7fa9b4069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.832+0000 7fa9b9155640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9b40afe10 0x7fa9b4069ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39406/0 (socket says 172.21.15.47:39406) 2024-03-23T09:47:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.832+0000 7fa9b9155640 1 -- 172.21.15.47:0/961543441 learned_addr learned my addr 172.21.15.47:0/961543441 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.832+0000 7fa9b8954640 1 --2- 172.21.15.47:0/961543441 >> v2:172.21.15.175:3300/0 conn(0x7fa9b40b2390 0x7fa9b406a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.832+0000 7fa9b9956640 1 --2- 172.21.15.47:0/961543441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b40cd110 0x7fa9b415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.832+0000 7fa9b9155640 1 -- 172.21.15.47:0/961543441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b40cd110 msgr2=0x7fa9b415c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.832+0000 7fa9b9155640 1 --2- 172.21.15.47:0/961543441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b40cd110 0x7fa9b415c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.832+0000 7fa9b9155640 1 -- 172.21.15.47:0/961543441 >> v2:172.21.15.175:3300/0 conn(0x7fa9b40b2390 msgr2=0x7fa9b406a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.832+0000 7fa9b9155640 1 --2- 172.21.15.47:0/961543441 >> v2:172.21.15.175:3300/0 conn(0x7fa9b40b2390 0x7fa9b406a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.832+0000 7fa9b9155640 1 -- 172.21.15.47:0/961543441 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b40c9c30 con 0x7fa9b40afe10 2024-03-23T09:47:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.832+0000 7fa9b8954640 1 --2- 172.21.15.47:0/961543441 >> v2:172.21.15.175:3300/0 conn(0x7fa9b40b2390 0x7fa9b406a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.833+0000 7fa9b9155640 1 --2- 172.21.15.47:0/961543441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9b40afe10 0x7fa9b4069ce0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fa9b406cb40 tx=0x7fa9b002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe60ec43212aaed server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.833+0000 7fa9abfff640 1 -- 172.21.15.47:0/961543441 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0031870 con 0x7fa9b40afe10 2024-03-23T09:47:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.833+0000 7fa9abfff640 1 -- 172.21.15.47:0/961543441 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b00319d0 con 0x7fa9b40afe10 2024-03-23T09:47:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.833+0000 7fa9abfff640 1 -- 172.21.15.47:0/961543441 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0031d60 con 0x7fa9b40afe10 2024-03-23T09:47:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.833+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/961543441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9b40afe10 msgr2=0x7fa9b4069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.833+0000 7fa9bb3b9640 1 --2- 172.21.15.47:0/961543441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9b40afe10 0x7fa9b4069ce0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fa9b406cb40 tx=0x7fa9b002b910 comp rx=0 tx=0).stop 2024-03-23T09:47:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.833+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/961543441 shutdown_connections 2024-03-23T09:47:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.833+0000 7fa9bb3b9640 1 --2- 172.21.15.47:0/961543441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b40cd110 0x7fa9b415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.833+0000 7fa9bb3b9640 1 --2- 172.21.15.47:0/961543441 >> v2:172.21.15.175:3300/0 conn(0x7fa9b40b2390 0x7fa9b406a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.833+0000 7fa9bb3b9640 1 --2- 172.21.15.47:0/961543441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9b40afe10 0x7fa9b4069ce0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.833+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/961543441 >> 172.21.15.47:0/961543441 conn(0x7fa9b40cf540 msgr2=0x7fa9b40cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.833+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/961543441 shutdown_connections 2024-03-23T09:47:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.833+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/961543441 wait complete. 2024-03-23T09:47:11.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.835+0000 7fa9bb3b9640 1 Processor -- start 2024-03-23T09:47:11.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.835+0000 7fa9bb3b9640 1 -- start start 2024-03-23T09:47:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.835+0000 7fa9bb3b9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9b40afe10 0x7fa9b41f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.835+0000 7fa9bb3b9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b40b2390 0x7fa9b40c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.835+0000 7fa9bb3b9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b40cd110 0x7fa9b40c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.835+0000 7fa9bb3b9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b40c7520 con 0x7fa9b40afe10 2024-03-23T09:47:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.835+0000 7fa9bb3b9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b40c7800 con 0x7fa9b40b2390 2024-03-23T09:47:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.835+0000 7fa9bb3b9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b40c7ae0 con 0x7fa9b40cd110 2024-03-23T09:47:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.835+0000 7fa9b9155640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9b40afe10 0x7fa9b41f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.835+0000 7fa9b9155640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9b40afe10 0x7fa9b41f1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39414/0 (socket says 172.21.15.47:39414) 2024-03-23T09:47:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.835+0000 7fa9b9155640 1 -- 172.21.15.47:0/64672076 learned_addr learned my addr 172.21.15.47:0/64672076 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.835+0000 7fa9b8954640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b40b2390 0x7fa9b40c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.836+0000 7fa9b9956640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b40cd110 0x7fa9b40c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.836+0000 7fa9b9155640 1 -- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b40cd110 msgr2=0x7fa9b40c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.836+0000 7fa9b9155640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b40cd110 0x7fa9b40c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.836+0000 7fa9b9155640 1 -- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b40b2390 msgr2=0x7fa9b40c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.836+0000 7fa9b9155640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b40b2390 0x7fa9b40c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.836+0000 7fa9b9155640 1 -- 172.21.15.47:0/64672076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b00094d0 con 0x7fa9b40afe10 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.836+0000 7fa9b9956640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b40cd110 0x7fa9b40c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.836+0000 7fa9b9155640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9b40afe10 0x7fa9b41f1a10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00097c0 tx=0x7fa9b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.836+0000 7fa9a9ffb640 1 -- 172.21.15.47:0/64672076 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b003ac60 con 0x7fa9b40afe10 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.836+0000 7fa9a9ffb640 1 -- 172.21.15.47:0/64672076 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b003adc0 con 0x7fa9b40afe10 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.836+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/64672076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b40c7dc0 con 0x7fa9b40afe10 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.836+0000 7fa9a9ffb640 1 -- 172.21.15.47:0/64672076 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0044610 con 0x7fa9b40afe10 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.836+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/64672076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b41f1f50 con 0x7fa9b40afe10 2024-03-23T09:47:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.837+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/64672076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b4066be0 con 0x7fa9b40afe10 2024-03-23T09:47:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.838+0000 7fa9a9ffb640 1 -- 172.21.15.47:0/64672076 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa9b004d450 con 0x7fa9b40afe10 2024-03-23T09:47:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.838+0000 7fa9a9ffb640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa99806ac00 0x7fa99806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.838+0000 7fa9a9ffb640 1 -- 172.21.15.47:0/64672076 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9b002d040 con 0x7fa9b40afe10 2024-03-23T09:47:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.838+0000 7fa9b8954640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa99806ac00 0x7fa99806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.839+0000 7fa9b8954640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa99806ac00 0x7fa99806d0c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40c6130 tx=0x7fa9a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:11.842+0000 7fa9a9ffb640 1 -- 172.21.15.47:0/64672076 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa9b0076ac0 con 0x7fa9b40afe10 2024-03-23T09:47:12.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.022+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/64672076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9b40b5180 con 0x7fa9b40afe10 2024-03-23T09:47:12.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.024+0000 7fa9a9ffb640 1 -- 172.21.15.47:0/64672076 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16105 (secure 0 0 0) 0x7fa9b0035090 con 0x7fa9b40afe10 2024-03-23T09:47:12.024 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:12.025 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":84,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:47:11.390099+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:47:09.722391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-23T09:47:10.354054+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:47:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.026+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa99806ac00 msgr2=0x7fa99806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.026+0000 7fa9bb3b9640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa99806ac00 0x7fa99806d0c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40c6130 tx=0x7fa9a4008040 comp rx=0 tx=0).stop 2024-03-23T09:47:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.026+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/64672076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9b40afe10 msgr2=0x7fa9b41f1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.026+0000 7fa9bb3b9640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9b40afe10 0x7fa9b41f1a10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00097c0 tx=0x7fa9b00046c0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.026+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/64672076 shutdown_connections 2024-03-23T09:47:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.026+0000 7fa9bb3b9640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa99806ac00 0x7fa99806d0c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.026+0000 7fa9bb3b9640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b40cd110 0x7fa9b40c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.026+0000 7fa9bb3b9640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b40b2390 0x7fa9b40c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.026+0000 7fa9bb3b9640 1 --2- 172.21.15.47:0/64672076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9b40afe10 0x7fa9b41f1a10 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.026+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/64672076 >> 172.21.15.47:0/64672076 conn(0x7fa9b40cf540 msgr2=0x7fa9b40b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:12.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.026+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/64672076 shutdown_connections 2024-03-23T09:47:12.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.026+0000 7fa9bb3b9640 1 -- 172.21.15.47:0/64672076 wait complete. 2024-03-23T09:47:12.037 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T09:47:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.141+0000 7f4d0398b640 1 Processor -- start 2024-03-23T09:47:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.141+0000 7f4d0398b640 1 -- start start 2024-03-23T09:47:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.142+0000 7f4d0398b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4cfc0cde10 0x7f4cfc160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.142+0000 7f4d0398b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4cfc0b8350 0x7f4cfc0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.142+0000 7f4d0398b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cfc161630 con 0x7f4cfc15c990 2024-03-23T09:47:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.142+0000 7f4d0398b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cfc0d1030 con 0x7f4cfc0b8350 2024-03-23T09:47:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.142+0000 7f4d0398b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cfc0cfe10 con 0x7f4cfc0cde10 2024-03-23T09:47:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.142+0000 7f4d01f28640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4cfc0b8350 0x7f4cfc0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.142+0000 7f4d01f28640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4cfc0b8350 0x7f4cfc0d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39426/0 (socket says 172.21.15.47:39426) 2024-03-23T09:47:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.142+0000 7f4d01727640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4cfc15c990 0x7f4cfc0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48776/0 (socket says 172.21.15.47:48776) 2024-03-23T09:47:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.142+0000 7f4d01f28640 1 -- 172.21.15.47:0/1495048627 learned_addr learned my addr 172.21.15.47:0/1495048627 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:12.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.142+0000 7f4d01f28640 1 -- 172.21.15.47:0/1495048627 >> v2:172.21.15.175:3300/0 conn(0x7f4cfc0cde10 msgr2=0x7f4cfc160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:12.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.142+0000 7f4d00f26640 1 --2- 172.21.15.47:0/1495048627 >> v2:172.21.15.175:3300/0 conn(0x7f4cfc0cde10 0x7f4cfc160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:12.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.142+0000 7f4d01f28640 1 --2- 172.21.15.47:0/1495048627 >> v2:172.21.15.175:3300/0 conn(0x7f4cfc0cde10 0x7f4cfc160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.143+0000 7f4d01f28640 1 -- 172.21.15.47:0/1495048627 >> v1:172.21.15.47:6789/0 conn(0x7f4cfc15c990 legacy=0x7f4cfc0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:12.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.142+0000 7f4cfbfff640 1 -- 172.21.15.47:0/1495048627 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132576047 0 0) 0x7f4cfc161630 con 0x7f4cfc15c990 2024-03-23T09:47:12.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.143+0000 7f4d01f28640 1 -- 172.21.15.47:0/1495048627 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc0b5b60 con 0x7f4cfc0b8350 2024-03-23T09:47:12.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.143+0000 7f4d00f26640 1 --2- 172.21.15.47:0/1495048627 >> v2:172.21.15.175:3300/0 conn(0x7f4cfc0cde10 0x7f4cfc160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:12.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.143+0000 7f4d01f28640 1 --2- 172.21.15.47:0/1495048627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cfc0b8350 0x7f4cfc0d14c0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc08ac70 tx=0x7f4cf402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e57322483fb1ad server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:12.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.143+0000 7f4cfbfff640 1 -- 172.21.15.47:0/1495048627 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4031870 con 0x7f4cfc0b8350 2024-03-23T09:47:12.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.143+0000 7f4cfbfff640 1 -- 172.21.15.47:0/1495048627 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf40319d0 con 0x7f4cfc0b8350 2024-03-23T09:47:12.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.143+0000 7f4cfbfff640 1 -- 172.21.15.47:0/1495048627 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4031d60 con 0x7f4cfc0b8350 2024-03-23T09:47:12.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.144+0000 7f4d0398b640 1 -- 172.21.15.47:0/1495048627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cfc0b8350 msgr2=0x7f4cfc0d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:12.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.144+0000 7f4d0398b640 1 --2- 172.21.15.47:0/1495048627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cfc0b8350 0x7f4cfc0d14c0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc08ac70 tx=0x7f4cf402b910 comp rx=0 tx=0).stop 2024-03-23T09:47:12.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.144+0000 7f4d0398b640 1 -- 172.21.15.47:0/1495048627 shutdown_connections 2024-03-23T09:47:12.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.144+0000 7f4d0398b640 1 --2- 172.21.15.47:0/1495048627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cfc0b8350 0x7f4cfc0d14c0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.144+0000 7f4d0398b640 1 --2- 172.21.15.47:0/1495048627 >> v2:172.21.15.175:3300/0 conn(0x7f4cfc0cde10 0x7f4cfc160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.144+0000 7f4d0398b640 1 -- 172.21.15.47:0/1495048627 >> 172.21.15.47:0/1495048627 conn(0x7f4cfc0659d0 msgr2=0x7f4cfc065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.144+0000 7f4d0398b640 1 -- 172.21.15.47:0/1495048627 shutdown_connections 2024-03-23T09:47:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.144+0000 7f4d0398b640 1 -- 172.21.15.47:0/1495048627 wait complete. 2024-03-23T09:47:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.145+0000 7f4d0398b640 1 Processor -- start 2024-03-23T09:47:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.145+0000 7f4d0398b640 1 -- start start 2024-03-23T09:47:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d0398b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc0b8350 0x7f4cfc15faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d0398b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cfc0cde10 0x7f4cfc15b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d0398b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cfc15c990 0x7f4cfc0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d0398b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc15ede0 con 0x7f4cfc15c990 2024-03-23T09:47:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d0398b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc150910 con 0x7f4cfc0b8350 2024-03-23T09:47:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d0398b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc15e160 con 0x7f4cfc0cde10 2024-03-23T09:47:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d01f28640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cfc15c990 0x7f4cfc0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d01f28640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cfc15c990 0x7f4cfc0d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39440/0 (socket says 172.21.15.47:39440) 2024-03-23T09:47:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d01f28640 1 -- 172.21.15.47:0/2030496978 learned_addr learned my addr 172.21.15.47:0/2030496978 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d00f26640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cfc0cde10 0x7f4cfc15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d01727640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc0b8350 0x7f4cfc15faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d01f28640 1 -- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cfc0cde10 msgr2=0x7f4cfc15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d01f28640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cfc0cde10 0x7f4cfc15b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d01f28640 1 -- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc0b8350 msgr2=0x7f4cfc15faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d01f28640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc0b8350 0x7f4cfc15faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d01f28640 1 -- 172.21.15.47:0/2030496978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf40094d0 con 0x7f4cfc15c990 2024-03-23T09:47:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.146+0000 7f4d01727640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc0b8350 0x7f4cfc15faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.147+0000 7f4d01f28640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cfc15c990 0x7f4cfc0d1950 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40090b0 tx=0x7f4cf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.147+0000 7f4cf9ffb640 1 -- 172.21.15.47:0/2030496978 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4031800 con 0x7f4cfc15c990 2024-03-23T09:47:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.147+0000 7f4cf9ffb640 1 -- 172.21.15.47:0/2030496978 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf40443f0 con 0x7f4cfc15c990 2024-03-23T09:47:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.147+0000 7f4d0398b640 1 -- 172.21.15.47:0/2030496978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cfc15e440 con 0x7f4cfc15c990 2024-03-23T09:47:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.147+0000 7f4cf9ffb640 1 -- 172.21.15.47:0/2030496978 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf40447a0 con 0x7f4cfc15c990 2024-03-23T09:47:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.147+0000 7f4d0398b640 1 -- 172.21.15.47:0/2030496978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cfc15ea00 con 0x7f4cfc15c990 2024-03-23T09:47:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.148+0000 7f4d0398b640 1 -- 172.21.15.47:0/2030496978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cfc072360 con 0x7f4cfc15c990 2024-03-23T09:47:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.148+0000 7f4cf9ffb640 1 -- 172.21.15.47:0/2030496978 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4cf404d450 con 0x7f4cfc15c990 2024-03-23T09:47:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.149+0000 7f4cf9ffb640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ce806ac00 0x7f4ce806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:12.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.149+0000 7f4cf9ffb640 1 -- 172.21.15.47:0/2030496978 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4cf402d040 con 0x7f4cfc15c990 2024-03-23T09:47:12.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.149+0000 7f4d01727640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ce806ac00 0x7f4ce806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:12.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.149+0000 7f4d01727640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ce806ac00 0x7f4ce806d0c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0012e0 tx=0x7f4cec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:12.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.153+0000 7f4cf9ffb640 1 -- 172.21.15.47:0/2030496978 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4cf4076c00 con 0x7f4cfc15c990 2024-03-23T09:47:12.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.369+0000 7f4d0398b640 1 -- 172.21.15.47:0/2030496978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4cfc15ef20 con 0x7f4cfc15c990 2024-03-23T09:47:12.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.394+0000 7f4cf9ffb640 1 -- 172.21.15.47:0/2030496978 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4cfc15ef20 con 0x7f4cfc15c990 2024-03-23T09:47:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.396+0000 7f4d0398b640 1 -- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ce806ac00 msgr2=0x7f4ce806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.396+0000 7f4d0398b640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ce806ac00 0x7f4ce806d0c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0012e0 tx=0x7f4cec009040 comp rx=0 tx=0).stop 2024-03-23T09:47:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.396+0000 7f4d0398b640 1 -- 172.21.15.47:0/2030496978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cfc15c990 msgr2=0x7f4cfc0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.396+0000 7f4d0398b640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cfc15c990 0x7f4cfc0d1950 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40090b0 tx=0x7f4cf40046c0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.396+0000 7f4d0398b640 1 -- 172.21.15.47:0/2030496978 shutdown_connections 2024-03-23T09:47:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.396+0000 7f4d0398b640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ce806ac00 0x7f4ce806d0c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.396+0000 7f4d0398b640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cfc15c990 0x7f4cfc0d1950 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.396+0000 7f4d0398b640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cfc0cde10 0x7f4cfc15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.396+0000 7f4d0398b640 1 --2- 172.21.15.47:0/2030496978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc0b8350 0x7f4cfc15faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.396+0000 7f4d0398b640 1 -- 172.21.15.47:0/2030496978 >> 172.21.15.47:0/2030496978 conn(0x7f4cfc0659d0 msgr2=0x7f4cfc156dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.396+0000 7f4d0398b640 1 -- 172.21.15.47:0/2030496978 shutdown_connections 2024-03-23T09:47:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.396+0000 7f4d0398b640 1 -- 172.21.15.47:0/2030496978 wait complete. 2024-03-23T09:47:12.408 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-23T09:47:12.408 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-23T09:47:12.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.514+0000 7f316fd00640 1 Processor -- start 2024-03-23T09:47:12.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.514+0000 7f316fd00640 1 -- start start 2024-03-23T09:47:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.515+0000 7f316fd00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316815c990 0x7f3168160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.515+0000 7f316fd00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31680cde10 0x7f31680d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.515+0000 7f316fd00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3168161630 con 0x7f31680b8350 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.515+0000 7f316fd00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31680d1030 con 0x7f31680cde10 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.515+0000 7f316fd00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31680cfe10 con 0x7f316815c990 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.515+0000 7f316e29d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31680cde10 0x7f31680d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.515+0000 7f316d29b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f31680b8350 0x7f31680cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48786/0 (socket says 172.21.15.47:48786) 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.515+0000 7f316e29d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31680cde10 0x7f31680d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39452/0 (socket says 172.21.15.47:39452) 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.515+0000 7f316d29b640 1 -- 172.21.15.47:0/2689649070 learned_addr learned my addr 172.21.15.47:0/2689649070 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.515+0000 7f316da9c640 1 --2- 172.21.15.47:0/2689649070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316815c990 0x7f3168160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.516+0000 7f316e29d640 1 -- 172.21.15.47:0/2689649070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316815c990 msgr2=0x7f3168160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.516+0000 7f316e29d640 1 --2- 172.21.15.47:0/2689649070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316815c990 0x7f3168160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.516+0000 7f316e29d640 1 -- 172.21.15.47:0/2689649070 >> v1:172.21.15.47:6789/0 conn(0x7f31680b8350 legacy=0x7f31680cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.516+0000 7f316ca9a640 1 -- 172.21.15.47:0/2689649070 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858418996 0 0) 0x7f3168161630 con 0x7f31680b8350 2024-03-23T09:47:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.516+0000 7f316e29d640 1 -- 172.21.15.47:0/2689649070 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31680b5b60 con 0x7f31680cde10 2024-03-23T09:47:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.516+0000 7f316e29d640 1 --2- 172.21.15.47:0/2689649070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31680cde10 0x7f31680d14c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3160009d90 tx=0x7f316002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fb61f991d06ca7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.517+0000 7f316ca9a640 1 -- 172.21.15.47:0/2689649070 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316002cbc0 con 0x7f31680cde10 2024-03-23T09:47:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.517+0000 7f316ca9a640 1 -- 172.21.15.47:0/2689649070 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316002cd20 con 0x7f31680cde10 2024-03-23T09:47:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.517+0000 7f316ca9a640 1 -- 172.21.15.47:0/2689649070 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160034660 con 0x7f31680cde10 2024-03-23T09:47:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.517+0000 7f316fd00640 1 -- 172.21.15.47:0/2689649070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31680cde10 msgr2=0x7f31680d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.517+0000 7f316fd00640 1 --2- 172.21.15.47:0/2689649070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31680cde10 0x7f31680d14c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3160009d90 tx=0x7f316002be10 comp rx=0 tx=0).stop 2024-03-23T09:47:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.517+0000 7f316fd00640 1 -- 172.21.15.47:0/2689649070 shutdown_connections 2024-03-23T09:47:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.517+0000 7f316fd00640 1 --2- 172.21.15.47:0/2689649070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31680cde10 0x7f31680d14c0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.517+0000 7f316fd00640 1 --2- 172.21.15.47:0/2689649070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316815c990 0x7f3168160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.517+0000 7f316fd00640 1 -- 172.21.15.47:0/2689649070 >> 172.21.15.47:0/2689649070 conn(0x7f31680659d0 msgr2=0x7f3168065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.517+0000 7f316fd00640 1 -- 172.21.15.47:0/2689649070 shutdown_connections 2024-03-23T09:47:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.517+0000 7f316fd00640 1 -- 172.21.15.47:0/2689649070 wait complete. 2024-03-23T09:47:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.518+0000 7f316fd00640 1 Processor -- start 2024-03-23T09:47:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.518+0000 7f316fd00640 1 -- start start 2024-03-23T09:47:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316fd00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31680b8350 0x7f3168153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316fd00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31680cde10 0x7f31680cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316fd00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316815c990 0x7f31680cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316fd00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316815eda0 con 0x7f316815c990 2024-03-23T09:47:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316fd00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316815e190 con 0x7f31680b8350 2024-03-23T09:47:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316fd00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316815e470 con 0x7f31680cde10 2024-03-23T09:47:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316e29d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316815c990 0x7f31680cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316e29d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316815c990 0x7f31680cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39456/0 (socket says 172.21.15.47:39456) 2024-03-23T09:47:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316e29d640 1 -- 172.21.15.47:0/132408722 learned_addr learned my addr 172.21.15.47:0/132408722 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316d29b640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31680cde10 0x7f31680cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316e29d640 1 -- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31680cde10 msgr2=0x7f31680cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316e29d640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31680cde10 0x7f31680cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316da9c640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31680b8350 0x7f3168153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316e29d640 1 -- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31680b8350 msgr2=0x7f3168153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316e29d640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31680b8350 0x7f3168153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.519+0000 7f316e29d640 1 -- 172.21.15.47:0/132408722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3160009a70 con 0x7f316815c990 2024-03-23T09:47:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.520+0000 7f316e29d640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316815c990 0x7f31680cf7d0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3160009d90 tx=0x7f3160004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.520+0000 7f315e7fc640 1 -- 172.21.15.47:0/132408722 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160040650 con 0x7f316815c990 2024-03-23T09:47:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.520+0000 7f315e7fc640 1 -- 172.21.15.47:0/132408722 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316002ce90 con 0x7f316815c990 2024-03-23T09:47:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.520+0000 7f315e7fc640 1 -- 172.21.15.47:0/132408722 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31600409f0 con 0x7f316815c990 2024-03-23T09:47:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.520+0000 7f316fd00640 1 -- 172.21.15.47:0/132408722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316815e750 con 0x7f316815c990 2024-03-23T09:47:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.520+0000 7f316fd00640 1 -- 172.21.15.47:0/132408722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3168154210 con 0x7f316815c990 2024-03-23T09:47:12.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.521+0000 7f316fd00640 1 -- 172.21.15.47:0/132408722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3168072360 con 0x7f316815c990 2024-03-23T09:47:12.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.521+0000 7f315e7fc640 1 -- 172.21.15.47:0/132408722 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3160040b50 con 0x7f316815c990 2024-03-23T09:47:12.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.521+0000 7f315e7fc640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f314c06ac00 0x7f314c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:12.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.522+0000 7f315e7fc640 1 -- 172.21.15.47:0/132408722 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31600ac0a0 con 0x7f316815c990 2024-03-23T09:47:12.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.522+0000 7f316da9c640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f314c06ac00 0x7f314c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:12.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.522+0000 7f316da9c640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f314c06ac00 0x7f314c06d0c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f31680d0cd0 tx=0x7f3164008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:12.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.526+0000 7f315e7fc640 1 -- 172.21.15.47:0/132408722 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f31600773a0 con 0x7f316815c990 2024-03-23T09:47:12.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:12.742+0000 7f316fd00640 1 -- 172.21.15.47:0/132408722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3168071f40 con 0x7f316815c990 2024-03-23T09:47:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.405+0000 7f315e7fc640 1 -- 172.21.15.47:0/132408722 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3160009450 con 0x7f316815c990 2024-03-23T09:47:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.408+0000 7f316fd00640 1 -- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f314c06ac00 msgr2=0x7f314c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.408+0000 7f316fd00640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f314c06ac00 0x7f314c06d0c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f31680d0cd0 tx=0x7f3164008040 comp rx=0 tx=0).stop 2024-03-23T09:47:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.408+0000 7f316fd00640 1 -- 172.21.15.47:0/132408722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316815c990 msgr2=0x7f31680cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.408+0000 7f316fd00640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316815c990 0x7f31680cf7d0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3160009d90 tx=0x7f3160004a40 comp rx=0 tx=0).stop 2024-03-23T09:47:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.408+0000 7f316fd00640 1 -- 172.21.15.47:0/132408722 shutdown_connections 2024-03-23T09:47:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.408+0000 7f316fd00640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f314c06ac00 0x7f314c06d0c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.408+0000 7f316fd00640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316815c990 0x7f31680cf7d0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.408+0000 7f316fd00640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31680cde10 0x7f31680cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.408+0000 7f316fd00640 1 --2- 172.21.15.47:0/132408722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31680b8350 0x7f3168153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.408+0000 7f316fd00640 1 -- 172.21.15.47:0/132408722 >> 172.21.15.47:0/132408722 conn(0x7f31680659d0 msgr2=0x7f3168065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.408+0000 7f316fd00640 1 -- 172.21.15.47:0/132408722 shutdown_connections 2024-03-23T09:47:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.408+0000 7f316fd00640 1 -- 172.21.15.47:0/132408722 wait complete. 2024-03-23T09:47:13.421 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-23T09:47:13.422 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:47:13.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.528+0000 7f9c0e2f7640 1 Processor -- start 2024-03-23T09:47:13.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.528+0000 7f9c0e2f7640 1 -- start start 2024-03-23T09:47:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.529+0000 7f9c0e2f7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c080cde10 0x7f9c08167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.529+0000 7f9c0e2f7640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c080d11e0 con 0x7f9c080b8350 2024-03-23T09:47:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.529+0000 7f9c0e2f7640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c080d2680 con 0x7f9c0815ca90 2024-03-23T09:47:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.529+0000 7f9c0e2f7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08160f60 con 0x7f9c080cde10 2024-03-23T09:47:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.529+0000 7f9c077fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9c080b8350 0x7f9c080cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48802/0 (socket says 172.21.15.47:48802) 2024-03-23T09:47:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.529+0000 7f9c077fe640 1 -- 172.21.15.47:0/2130298212 learned_addr learned my addr 172.21.15.47:0/2130298212 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.529+0000 7f9c0c894640 1 --2- 172.21.15.47:0/2130298212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c080cde10 0x7f9c08167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.529+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397364403 0 0) 0x7f9c080d11e0 con 0x7f9c080b8350 2024-03-23T09:47:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.529+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bf0003520 con 0x7f9c080b8350 2024-03-23T09:47:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4153489123 0 0) 0x7f9bf0003520 con 0x7f9c080b8350 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c080cde10 msgr2=0x7f9c08167aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c06ffd640 1 --2- 172.21.15.47:0/2130298212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c080cde10 0x7f9c08167aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 >> v1:172.21.15.175:6789/0 conn(0x7f9c0815ca90 legacy=0x7f9c08160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c08161240 con 0x7f9c080b8350 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f9bf8001670 con 0x7f9c080b8350 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 >> v1:172.21.15.47:6789/0 conn(0x7f9c080b8350 legacy=0x7f9c080cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c06ffd640 1 --2- 172.21.15.47:0/2130298212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c080cde10 0x7f9bf00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c06ffd640 1 --2- 172.21.15.47:0/2130298212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bf0003ee0 0x7f9bf00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c06ffd640 1 --2- 172.21.15.47:0/2130298212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bf0004d00 0x7f9bf0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08160f60 con 0x7f9bf0004d00 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf000ba90 con 0x7f9bf0003ee0 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf000bc50 con 0x7f9c080cde10 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c0c894640 1 --2- 172.21.15.47:0/2130298212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bf0004d00 0x7f9bf0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c077fe640 1 --2- 172.21.15.47:0/2130298212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bf0003ee0 0x7f9bf00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c07fff640 1 --2- 172.21.15.47:0/2130298212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c080cde10 0x7f9bf00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c0c894640 1 -- 172.21.15.47:0/2130298212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c080cde10 msgr2=0x7f9bf00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c0c894640 1 --2- 172.21.15.47:0/2130298212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c080cde10 0x7f9bf00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c0c894640 1 -- 172.21.15.47:0/2130298212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bf0003ee0 msgr2=0x7f9bf00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c0c894640 1 --2- 172.21.15.47:0/2130298212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bf0003ee0 0x7f9bf00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.530+0000 7f9c0c894640 1 -- 172.21.15.47:0/2130298212 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf0003520 con 0x7f9bf0004d00 2024-03-23T09:47:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.531+0000 7f9c0c894640 1 --2- 172.21.15.47:0/2130298212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bf0004d00 0x7f9bf0005180 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9c0808bd50 tx=0x7f9bf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36132d6cf1caa346 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.531+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0002cb90 con 0x7f9bf0004d00 2024-03-23T09:47:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.531+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0000f040 con 0x7f9bf0004d00 2024-03-23T09:47:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.531+0000 7f9c06ffd640 1 -- 172.21.15.47:0/2130298212 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00034440 con 0x7f9bf0004d00 2024-03-23T09:47:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.532+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/2130298212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bf0004d00 msgr2=0x7f9bf0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.532+0000 7f9c0e2f7640 1 --2- 172.21.15.47:0/2130298212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bf0004d00 0x7f9bf0005180 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9c0808bd50 tx=0x7f9bf00057f0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.532+0000 7f9c0c894640 1 -- 172.21.15.47:0/2130298212 reap_dead start 2024-03-23T09:47:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.532+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/2130298212 shutdown_connections 2024-03-23T09:47:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.532+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/2130298212 >> 172.21.15.47:0/2130298212 conn(0x7f9c080659d0 msgr2=0x7f9c08065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.532+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/2130298212 shutdown_connections 2024-03-23T09:47:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.532+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/2130298212 wait complete. 2024-03-23T09:47:13.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.533+0000 7f9c0e2f7640 1 Processor -- start 2024-03-23T09:47:13.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.533+0000 7f9c0e2f7640 1 -- start start 2024-03-23T09:47:13.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0e2f7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c080cbed0 0x7f9c081f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:13.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0e2f7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c080cc2b0 0x7f9c0815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:13.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0e2f7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0815b3e0 0x7f9c08155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:13.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0e2f7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080b3760 con 0x7f9c0815b3e0 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0e2f7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080b3a40 con 0x7f9c080cc2b0 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0e2f7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080b3d20 con 0x7f9c080cbed0 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0c894640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0815b3e0 0x7f9c08155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0c894640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0815b3e0 0x7f9c08155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39472/0 (socket says 172.21.15.47:39472) 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0c894640 1 -- 172.21.15.47:0/1251869655 learned_addr learned my addr 172.21.15.47:0/1251869655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c077fe640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c080cc2b0 0x7f9c0815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c07fff640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c080cbed0 0x7f9c081f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0c894640 1 -- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c080cbed0 msgr2=0x7f9c081f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0c894640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c080cbed0 0x7f9c081f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0c894640 1 -- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c080cc2b0 msgr2=0x7f9c0815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0c894640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c080cc2b0 0x7f9c0815aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0c894640 1 -- 172.21.15.47:0/1251869655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c080b4000 con 0x7f9c0815b3e0 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c077fe640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c080cc2b0 0x7f9c0815aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.534+0000 7f9c0c894640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0815b3e0 0x7f9c08155d90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9c00002230 tx=0x7f9c00009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.535+0000 7f9c04ff9640 1 -- 172.21.15.47:0/1251869655 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00043070 con 0x7f9c0815b3e0 2024-03-23T09:47:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.535+0000 7f9c04ff9640 1 -- 172.21.15.47:0/1251869655 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c00003990 con 0x7f9c0815b3e0 2024-03-23T09:47:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.535+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/1251869655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c080b42e0 con 0x7f9c0815b3e0 2024-03-23T09:47:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.535+0000 7f9c04ff9640 1 -- 172.21.15.47:0/1251869655 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00003d40 con 0x7f9c0815b3e0 2024-03-23T09:47:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.535+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/1251869655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c081f2a00 con 0x7f9c0815b3e0 2024-03-23T09:47:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.536+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/1251869655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c08072360 con 0x7f9c0815b3e0 2024-03-23T09:47:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.536+0000 7f9c04ff9640 1 -- 172.21.15.47:0/1251869655 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9c00034880 con 0x7f9c0815b3e0 2024-03-23T09:47:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.536+0000 7f9c04ff9640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9bf406ac30 0x7f9bf406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.537+0000 7f9c04ff9640 1 -- 172.21.15.47:0/1251869655 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c000aaae0 con 0x7f9c0815b3e0 2024-03-23T09:47:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.537+0000 7f9c07fff640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9bf406ac30 0x7f9bf406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.537+0000 7f9c07fff640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9bf406ac30 0x7f9bf406d0f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc003700 tx=0x7f9bfc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:13.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.541+0000 7f9c04ff9640 1 -- 172.21.15.47:0/1251869655 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9c00075cb0 con 0x7f9c0815b3e0 2024-03-23T09:47:13.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.756+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/1251869655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c080b4500 con 0x7f9c0815b3e0 2024-03-23T09:47:13.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.757+0000 7f9c04ff9640 1 -- 172.21.15.47:0/1251869655 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f9c00039020 con 0x7f9c0815b3e0 2024-03-23T09:47:13.757 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:13.757 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:47:13.401354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/1927629460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1927629460},{"type":"v1","addr":"172.21.15.47:6835","nonce":1927629460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:47:13.757 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 46 2024-03-23T09:47:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.759+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9bf406ac30 msgr2=0x7f9bf406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.759+0000 7f9c0e2f7640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9bf406ac30 0x7f9bf406d0f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc003700 tx=0x7f9bfc009040 comp rx=0 tx=0).stop 2024-03-23T09:47:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.759+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/1251869655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0815b3e0 msgr2=0x7f9c08155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.759+0000 7f9c0e2f7640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0815b3e0 0x7f9c08155d90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9c00002230 tx=0x7f9c00009750 comp rx=0 tx=0).stop 2024-03-23T09:47:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.759+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/1251869655 shutdown_connections 2024-03-23T09:47:13.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.759+0000 7f9c0e2f7640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9bf406ac30 0x7f9bf406d0f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.759+0000 7f9c0e2f7640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0815b3e0 0x7f9c08155d90 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.759+0000 7f9c0e2f7640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c080cc2b0 0x7f9c0815aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.759+0000 7f9c0e2f7640 1 --2- 172.21.15.47:0/1251869655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c080cbed0 0x7f9c081f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.759+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/1251869655 >> 172.21.15.47:0/1251869655 conn(0x7f9c080659d0 msgr2=0x7f9c080d2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.759+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/1251869655 shutdown_connections 2024-03-23T09:47:13.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.759+0000 7f9c0e2f7640 1 -- 172.21.15.47:0/1251869655 wait complete. 2024-03-23T09:47:13.770 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:47:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.871+0000 7f9e89cd5640 1 Processor -- start 2024-03-23T09:47:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.871+0000 7f9e89cd5640 1 -- start start 2024-03-23T09:47:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.871+0000 7f9e89cd5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9e840cf690 0x7f9e840b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.871+0000 7f9e89cd5640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8415d0e0 con 0x7f9e840cf210 2024-03-23T09:47:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.871+0000 7f9e89cd5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e840d1540 con 0x7f9e8415cd00 2024-03-23T09:47:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.871+0000 7f9e89cd5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e840d1820 con 0x7f9e840cf690 2024-03-23T09:47:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.872+0000 7f9e83fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9e840cf690 0x7f9e840b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.872+0000 7f9e83fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9e840cf690 0x7f9e840b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39482/0 (socket says 172.21.15.47:39482) 2024-03-23T09:47:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.872+0000 7f9e83fff640 1 -- 172.21.15.47:0/2904260403 learned_addr learned my addr 172.21.15.47:0/2904260403 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.872+0000 7f9e83fff640 1 -- 172.21.15.47:0/2904260403 >> v1:172.21.15.175:6789/0 conn(0x7f9e8415cd00 legacy=0x7f9e840d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.872+0000 7f9e83fff640 1 -- 172.21.15.47:0/2904260403 >> v1:172.21.15.47:6789/0 conn(0x7f9e840cf210 legacy=0x7f9e840b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.872+0000 7f9e83fff640 1 -- 172.21.15.47:0/2904260403 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e840d1b00 con 0x7f9e840cf690 2024-03-23T09:47:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.872+0000 7f9e827fc640 1 -- 172.21.15.47:0/2904260403 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654894574 0 0) 0x7f9e8415d0e0 con 0x7f9e840cf210 2024-03-23T09:47:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.873+0000 7f9e83fff640 1 --2- 172.21.15.47:0/2904260403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e840cf690 0x7f9e840b61a0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009e50 tx=0x7f9e7c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67bf4028db58ec7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.873+0000 7f9e827fc640 1 -- 172.21.15.47:0/2904260403 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c02ca80 con 0x7f9e840cf690 2024-03-23T09:47:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.873+0000 7f9e827fc640 1 -- 172.21.15.47:0/2904260403 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7c02cbe0 con 0x7f9e840cf690 2024-03-23T09:47:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.873+0000 7f9e827fc640 1 -- 172.21.15.47:0/2904260403 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c0343f0 con 0x7f9e840cf690 2024-03-23T09:47:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.873+0000 7f9e89cd5640 1 -- 172.21.15.47:0/2904260403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e840cf690 msgr2=0x7f9e840b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.873+0000 7f9e89cd5640 1 --2- 172.21.15.47:0/2904260403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e840cf690 0x7f9e840b61a0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009e50 tx=0x7f9e7c02bee0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.873+0000 7f9e89cd5640 1 -- 172.21.15.47:0/2904260403 shutdown_connections 2024-03-23T09:47:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.873+0000 7f9e89cd5640 1 --2- 172.21.15.47:0/2904260403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e840cf690 0x7f9e840b61a0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.873+0000 7f9e89cd5640 1 -- 172.21.15.47:0/2904260403 >> 172.21.15.47:0/2904260403 conn(0x7f9e8406b160 msgr2=0x7f9e8406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.873+0000 7f9e89cd5640 1 -- 172.21.15.47:0/2904260403 shutdown_connections 2024-03-23T09:47:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.873+0000 7f9e89cd5640 1 -- 172.21.15.47:0/2904260403 wait complete. 2024-03-23T09:47:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.875+0000 7f9e89cd5640 1 Processor -- start 2024-03-23T09:47:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.875+0000 7f9e89cd5640 1 -- start start 2024-03-23T09:47:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.875+0000 7f9e89cd5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e840cf210 0x7f9e840b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.875+0000 7f9e89cd5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e840cf690 0x7f9e8415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.875+0000 7f9e89cd5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e8415cd00 0x7f9e840b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.875+0000 7f9e89cd5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8415b120 con 0x7f9e840cf690 2024-03-23T09:47:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.875+0000 7f9e89cd5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e840b52d0 con 0x7f9e840cf210 2024-03-23T09:47:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.875+0000 7f9e89cd5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e840b2de0 con 0x7f9e8415cd00 2024-03-23T09:47:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e82ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e840cf690 0x7f9e8415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e82ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e840cf690 0x7f9e8415abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39492/0 (socket says 172.21.15.47:39492) 2024-03-23T09:47:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e82ffd640 1 -- 172.21.15.47:0/601888309 learned_addr learned my addr 172.21.15.47:0/601888309 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e837fe640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e840cf210 0x7f9e840b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e83fff640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e8415cd00 0x7f9e840b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e82ffd640 1 -- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e8415cd00 msgr2=0x7f9e840b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e82ffd640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e8415cd00 0x7f9e840b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e82ffd640 1 -- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e840cf210 msgr2=0x7f9e840b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e82ffd640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e840cf210 0x7f9e840b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e82ffd640 1 -- 172.21.15.47:0/601888309 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7c009b30 con 0x7f9e840cf690 2024-03-23T09:47:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e83fff640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e8415cd00 0x7f9e840b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e82ffd640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e840cf690 0x7f9e8415abe0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800c4e0 tx=0x7f9e7800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e6bfff640 1 -- 172.21.15.47:0/601888309 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78005480 con 0x7f9e840cf690 2024-03-23T09:47:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e6bfff640 1 -- 172.21.15.47:0/601888309 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e780035f0 con 0x7f9e840cf690 2024-03-23T09:47:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e89cd5640 1 -- 172.21.15.47:0/601888309 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e840b5000 con 0x7f9e840cf690 2024-03-23T09:47:13.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e6bfff640 1 -- 172.21.15.47:0/601888309 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e780039c0 con 0x7f9e840cf690 2024-03-23T09:47:13.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.876+0000 7f9e89cd5640 1 -- 172.21.15.47:0/601888309 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e840b2790 con 0x7f9e840cf690 2024-03-23T09:47:13.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.877+0000 7f9e89cd5640 1 -- 172.21.15.47:0/601888309 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e84072360 con 0x7f9e840cf690 2024-03-23T09:47:13.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.878+0000 7f9e6bfff640 1 -- 172.21.15.47:0/601888309 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9e78003b20 con 0x7f9e840cf690 2024-03-23T09:47:13.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.878+0000 7f9e6bfff640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e7006ac00 0x7f9e7006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:13.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.878+0000 7f9e6bfff640 1 -- 172.21.15.47:0/601888309 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e780886d0 con 0x7f9e840cf690 2024-03-23T09:47:13.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.878+0000 7f9e837fe640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e7006ac00 0x7f9e7006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:13.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.879+0000 7f9e837fe640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e7006ac00 0x7f9e7006d0c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9e74006fd0 tx=0x7f9e74009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:13.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:13.882+0000 7f9e6bfff640 1 -- 172.21.15.47:0/601888309 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9e780539f0 con 0x7f9e840cf690 2024-03-23T09:47:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.064+0000 7f9e89cd5640 1 -- 172.21.15.47:0/601888309 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e84066be0 con 0x7f9e840cf690 2024-03-23T09:47:14.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.065+0000 7f9e6bfff640 1 -- 172.21.15.47:0/601888309 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16092 (secure 0 0 0) 0x7f9e78017020 con 0x7f9e840cf690 2024-03-23T09:47:14.066 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:14.066 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":85,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:47:12.392230+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:47:09.722391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-23T09:47:10.354054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:47:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.067+0000 7f9e89cd5640 1 -- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e7006ac00 msgr2=0x7f9e7006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.067+0000 7f9e89cd5640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e7006ac00 0x7f9e7006d0c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9e74006fd0 tx=0x7f9e74009060 comp rx=0 tx=0).stop 2024-03-23T09:47:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.067+0000 7f9e89cd5640 1 -- 172.21.15.47:0/601888309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e840cf690 msgr2=0x7f9e8415abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.067+0000 7f9e89cd5640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e840cf690 0x7f9e8415abe0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800c4e0 tx=0x7f9e7800c9a0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.067+0000 7f9e89cd5640 1 -- 172.21.15.47:0/601888309 shutdown_connections 2024-03-23T09:47:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.067+0000 7f9e89cd5640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e7006ac00 0x7f9e7006d0c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.067+0000 7f9e89cd5640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e8415cd00 0x7f9e840b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.067+0000 7f9e89cd5640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e840cf690 0x7f9e8415abe0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.067+0000 7f9e89cd5640 1 --2- 172.21.15.47:0/601888309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e840cf210 0x7f9e840b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.067+0000 7f9e89cd5640 1 -- 172.21.15.47:0/601888309 >> 172.21.15.47:0/601888309 conn(0x7f9e8406b160 msgr2=0x7f9e8415bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:14.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.068+0000 7f9e89cd5640 1 -- 172.21.15.47:0/601888309 shutdown_connections 2024-03-23T09:47:14.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.068+0000 7f9e89cd5640 1 -- 172.21.15.47:0/601888309 wait complete. 2024-03-23T09:47:14.079 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T09:47:14.079 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:47:14.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.180+0000 7f4c1d769640 1 Processor -- start 2024-03-23T09:47:14.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.180+0000 7f4c1d769640 1 -- start start 2024-03-23T09:47:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c1d769640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4c1815a8d0 0x7f4c180cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c1d769640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4c180cb730 0x7f4c180cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c1d769640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c180cc610 con 0x7f4c180cf190 2024-03-23T09:47:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c1d769640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c180c41c0 con 0x7f4c180cb730 2024-03-23T09:47:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c1d769640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c180cf600 con 0x7f4c1815a8d0 2024-03-23T09:47:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c0ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4c180cb730 0x7f4c180cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c177fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4c180cf190 0x7f4c18098a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48822/0 (socket says 172.21.15.47:48822) 2024-03-23T09:47:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c177fe640 1 -- 172.21.15.47:0/3210028240 learned_addr learned my addr 172.21.15.47:0/3210028240 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c16ffd640 1 --2- 172.21.15.47:0/3210028240 >> v2:172.21.15.175:3300/0 conn(0x7f4c1815a8d0 0x7f4c180cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c0ffff640 1 -- 172.21.15.47:0/3210028240 >> v2:172.21.15.175:3300/0 conn(0x7f4c1815a8d0 msgr2=0x7f4c180cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c0ffff640 1 --2- 172.21.15.47:0/3210028240 >> v2:172.21.15.175:3300/0 conn(0x7f4c1815a8d0 0x7f4c180cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c0ffff640 1 -- 172.21.15.47:0/3210028240 >> v1:172.21.15.47:6789/0 conn(0x7f4c180cf190 legacy=0x7f4c18098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c0ffff640 1 -- 172.21.15.47:0/3210028240 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c180b0700 con 0x7f4c180cb730 2024-03-23T09:47:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.181+0000 7f4c167fc640 1 -- 172.21.15.47:0/3210028240 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853914436 0 0) 0x7f4c180cc610 con 0x7f4c180cf190 2024-03-23T09:47:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.182+0000 7f4c0ffff640 1 --2- 172.21.15.47:0/3210028240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c180cb730 0x7f4c180cec50 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f4c180d1410 tx=0x7f4c0002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a260297825b266 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.182+0000 7f4c167fc640 1 -- 172.21.15.47:0/3210028240 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0002e070 con 0x7f4c180cb730 2024-03-23T09:47:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.182+0000 7f4c167fc640 1 -- 172.21.15.47:0/3210028240 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c000315a0 con 0x7f4c180cb730 2024-03-23T09:47:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.182+0000 7f4c167fc640 1 -- 172.21.15.47:0/3210028240 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00031930 con 0x7f4c180cb730 2024-03-23T09:47:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.183+0000 7f4c1d769640 1 -- 172.21.15.47:0/3210028240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c180cb730 msgr2=0x7f4c180cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.183+0000 7f4c1d769640 1 --2- 172.21.15.47:0/3210028240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c180cb730 0x7f4c180cec50 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f4c180d1410 tx=0x7f4c0002b840 comp rx=0 tx=0).stop 2024-03-23T09:47:14.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.183+0000 7f4c1d769640 1 -- 172.21.15.47:0/3210028240 shutdown_connections 2024-03-23T09:47:14.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.183+0000 7f4c1d769640 1 --2- 172.21.15.47:0/3210028240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c180cb730 0x7f4c180cec50 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.183+0000 7f4c1d769640 1 --2- 172.21.15.47:0/3210028240 >> v2:172.21.15.175:3300/0 conn(0x7f4c1815a8d0 0x7f4c180cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.183+0000 7f4c1d769640 1 -- 172.21.15.47:0/3210028240 >> 172.21.15.47:0/3210028240 conn(0x7f4c180d0670 msgr2=0x7f4c180d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:14.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.183+0000 7f4c1d769640 1 -- 172.21.15.47:0/3210028240 shutdown_connections 2024-03-23T09:47:14.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.183+0000 7f4c1d769640 1 -- 172.21.15.47:0/3210028240 wait complete. 2024-03-23T09:47:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.184+0000 7f4c1d769640 1 Processor -- start 2024-03-23T09:47:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.184+0000 7f4c1d769640 1 -- start start 2024-03-23T09:47:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c1d769640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c180cb730 0x7f4c18156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c1d769640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c180cf190 0x7f4c180c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c1d769640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c1815a8d0 0x7f4c180c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c1d769640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c180ae2b0 con 0x7f4c180cf190 2024-03-23T09:47:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c1d769640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c180b5b00 con 0x7f4c180cb730 2024-03-23T09:47:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c1d769640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c180b5de0 con 0x7f4c1815a8d0 2024-03-23T09:47:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c0ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c180cf190 0x7f4c180c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c0ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c180cf190 0x7f4c180c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39514/0 (socket says 172.21.15.47:39514) 2024-03-23T09:47:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c0ffff640 1 -- 172.21.15.47:0/2638606514 learned_addr learned my addr 172.21.15.47:0/2638606514 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c16ffd640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c180cb730 0x7f4c18156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c177fe640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c1815a8d0 0x7f4c180c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c0ffff640 1 -- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c1815a8d0 msgr2=0x7f4c180c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c0ffff640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c1815a8d0 0x7f4c180c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c0ffff640 1 -- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c180cb730 msgr2=0x7f4c18156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c0ffff640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c180cb730 0x7f4c18156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c0ffff640 1 -- 172.21.15.47:0/2638606514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c00009400 con 0x7f4c180cf190 2024-03-23T09:47:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c16ffd640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c180cb730 0x7f4c18156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c0ffff640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c180cf190 0x7f4c180c89d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009720 tx=0x7f4c000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c0f7fe640 1 -- 172.21.15.47:0/2638606514 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00004230 con 0x7f4c180cf190 2024-03-23T09:47:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c0f7fe640 1 -- 172.21.15.47:0/2638606514 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c00031f00 con 0x7f4c180cf190 2024-03-23T09:47:14.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c1d769640 1 -- 172.21.15.47:0/2638606514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c180b60c0 con 0x7f4c180cf190 2024-03-23T09:47:14.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.185+0000 7f4c0f7fe640 1 -- 172.21.15.47:0/2638606514 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00032640 con 0x7f4c180cf190 2024-03-23T09:47:14.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.186+0000 7f4c1d769640 1 -- 172.21.15.47:0/2638606514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c18157040 con 0x7f4c180cf190 2024-03-23T09:47:14.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.187+0000 7f4c1d769640 1 -- 172.21.15.47:0/2638606514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c180ae420 con 0x7f4c180cf190 2024-03-23T09:47:14.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.187+0000 7f4c0f7fe640 1 -- 172.21.15.47:0/2638606514 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4c000327a0 con 0x7f4c180cf190 2024-03-23T09:47:14.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.187+0000 7f4c0f7fe640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4bf806ac30 0x7f4bf806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.187+0000 7f4c0f7fe640 1 -- 172.21.15.47:0/2638606514 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c000ab310 con 0x7f4c180cf190 2024-03-23T09:47:14.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.187+0000 7f4c16ffd640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4bf806ac30 0x7f4bf806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.188+0000 7f4c16ffd640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4bf806ac30 0x7f4bf806d0f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006fd0 tx=0x7f4c08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:14.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.192+0000 7f4c0f7fe640 1 -- 172.21.15.47:0/2638606514 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4c180ae420 con 0x7f4c180cf190 2024-03-23T09:47:14.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.368+0000 7f4c1d769640 1 -- 172.21.15.47:0/2638606514 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c180cc330 con 0x7f4bf806ac30 2024-03-23T09:47:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.376+0000 7f4c0f7fe640 1 -- 172.21.15.47:0/2638606514 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289491 (secure 0 0 0) 0x7f4c08009e10 con 0x7f4bf806ac30 2024-03-23T09:47:14.379 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:14.381 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:47:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.379+0000 7f4c1d769640 1 -- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4bf806ac30 msgr2=0x7f4bf806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.379+0000 7f4c1d769640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4bf806ac30 0x7f4bf806d0f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006fd0 tx=0x7f4c08008040 comp rx=0 tx=0).stop 2024-03-23T09:47:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.379+0000 7f4c1d769640 1 -- 172.21.15.47:0/2638606514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c180cf190 msgr2=0x7f4c180c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.379+0000 7f4c1d769640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c180cf190 0x7f4c180c89d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009720 tx=0x7f4c000046c0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.379+0000 7f4c1d769640 1 -- 172.21.15.47:0/2638606514 shutdown_connections 2024-03-23T09:47:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.379+0000 7f4c1d769640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4bf806ac30 0x7f4bf806d0f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.379+0000 7f4c1d769640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c1815a8d0 0x7f4c180c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.379+0000 7f4c1d769640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c180cf190 0x7f4c180c89d0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.379+0000 7f4c1d769640 1 --2- 172.21.15.47:0/2638606514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c180cb730 0x7f4c18156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.379+0000 7f4c1d769640 1 -- 172.21.15.47:0/2638606514 >> 172.21.15.47:0/2638606514 conn(0x7f4c180d0670 msgr2=0x7f4c181577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:14.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.379+0000 7f4c1d769640 1 -- 172.21.15.47:0/2638606514 shutdown_connections 2024-03-23T09:47:14.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.379+0000 7f4c1d769640 1 -- 172.21.15.47:0/2638606514 wait complete. 2024-03-23T09:47:14.390 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-03-23T09:47:12.429702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250056,"kb_used_data":9280,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754724664,"statfs":{"total":773094113280,"available":772838055936,"internally_reserved":0,"allocated":9502720,"data_stored":4682158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":246402616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000828"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424221+0000","last_change":"2024-03-23T09:47:11.420454+0000","last_active":"2024-03-23T09:47:11.424221+0000","last_peered":"2024-03-23T09:47:11.424221+0000","last_clean":"2024-03-23T09:47:11.424221+0000","last_became_active":"2024-03-23T09:47:11.418812+0000","last_became_peered":"2024-03-23T09:47:11.418812+0000","last_unstale":"2024-03-23T09:47:11.424221+0000","last_undegraded":"2024-03-23T09:47:11.424221+0000","last_fullsized":"2024-03-23T09:47:11.424221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:26:29.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421936+0000","last_change":"2024-03-23T09:47:11.420152+0000","last_active":"2024-03-23T09:47:11.421936+0000","last_peered":"2024-03-23T09:47:11.421936+0000","last_clean":"2024-03-23T09:47:11.421936+0000","last_became_active":"2024-03-23T09:47:11.419635+0000","last_became_peered":"2024-03-23T09:47:11.419635+0000","last_unstale":"2024-03-23T09:47:11.421936+0000","last_undegraded":"2024-03-23T09:47:11.421936+0000","last_fullsized":"2024-03-23T09:47:11.421936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:15:28.768150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.413487+0000","last_change":"2024-03-23T09:47:11.413059+0000","last_active":"2024-03-23T09:47:11.413487+0000","last_peered":"2024-03-23T09:47:11.413487+0000","last_clean":"2024-03-23T09:47:11.413487+0000","last_became_active":"2024-03-23T09:47:11.412801+0000","last_became_peered":"2024-03-23T09:47:11.412801+0000","last_unstale":"2024-03-23T09:47:11.413487+0000","last_undegraded":"2024-03-23T09:47:11.413487+0000","last_fullsized":"2024-03-23T09:47:11.413487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:21:53.365087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.402806+0000","last_change":"2024-03-23T09:47:11.402058+0000","last_active":"2024-03-23T09:47:11.402806+0000","last_peered":"2024-03-23T09:47:11.402806+0000","last_clean":"2024-03-23T09:47:11.402806+0000","last_became_active":"2024-03-23T09:47:11.401797+0000","last_became_peered":"2024-03-23T09:47:11.401797+0000","last_unstale":"2024-03-23T09:47:11.402806+0000","last_undegraded":"2024-03-23T09:47:11.402806+0000","last_fullsized":"2024-03-23T09:47:11.402806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:52:51.537440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416097+0000","last_change":"2024-03-23T09:47:11.405570+0000","last_active":"2024-03-23T09:47:11.416097+0000","last_peered":"2024-03-23T09:47:11.416097+0000","last_clean":"2024-03-23T09:47:11.416097+0000","last_became_active":"2024-03-23T09:47:11.405323+0000","last_became_peered":"2024-03-23T09:47:11.405323+0000","last_unstale":"2024-03-23T09:47:11.416097+0000","last_undegraded":"2024-03-23T09:47:11.416097+0000","last_fullsized":"2024-03-23T09:47:11.416097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:59:56.765267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.417663+0000","last_change":"2024-03-23T09:47:11.413134+0000","last_active":"2024-03-23T09:47:11.417663+0000","last_peered":"2024-03-23T09:47:11.417663+0000","last_clean":"2024-03-23T09:47:11.417663+0000","last_became_active":"2024-03-23T09:47:11.412923+0000","last_became_peered":"2024-03-23T09:47:11.412923+0000","last_unstale":"2024-03-23T09:47:11.417663+0000","last_undegraded":"2024-03-23T09:47:11.417663+0000","last_fullsized":"2024-03-23T09:47:11.417663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:07:33.336490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406596+0000","last_change":"2024-03-23T09:47:11.404392+0000","last_active":"2024-03-23T09:47:11.406596+0000","last_peered":"2024-03-23T09:47:11.406596+0000","last_clean":"2024-03-23T09:47:11.406596+0000","last_became_active":"2024-03-23T09:47:11.404177+0000","last_became_peered":"2024-03-23T09:47:11.404177+0000","last_unstale":"2024-03-23T09:47:11.406596+0000","last_undegraded":"2024-03-23T09:47:11.406596+0000","last_fullsized":"2024-03-23T09:47:11.406596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:47:30.427023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.418247+0000","last_change":"2024-03-23T09:47:11.416697+0000","last_active":"2024-03-23T09:47:11.418247+0000","last_peered":"2024-03-23T09:47:11.418247+0000","last_clean":"2024-03-23T09:47:11.418247+0000","last_became_active":"2024-03-23T09:47:11.416430+0000","last_became_peered":"2024-03-23T09:47:11.416430+0000","last_unstale":"2024-03-23T09:47:11.418247+0000","last_undegraded":"2024-03-23T09:47:11.418247+0000","last_fullsized":"2024-03-23T09:47:11.418247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:10:02.020675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.423529+0000","last_change":"2024-03-23T09:47:11.418822+0000","last_active":"2024-03-23T09:47:11.423529+0000","last_peered":"2024-03-23T09:47:11.423529+0000","last_clean":"2024-03-23T09:47:11.423529+0000","last_became_active":"2024-03-23T09:47:11.418544+0000","last_became_peered":"2024-03-23T09:47:11.418544+0000","last_unstale":"2024-03-23T09:47:11.423529+0000","last_undegraded":"2024-03-23T09:47:11.423529+0000","last_fullsized":"2024-03-23T09:47:11.423529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:15.479312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424183+0000","last_change":"2024-03-23T09:47:11.420068+0000","last_active":"2024-03-23T09:47:11.424183+0000","last_peered":"2024-03-23T09:47:11.424183+0000","last_clean":"2024-03-23T09:47:11.424183+0000","last_became_active":"2024-03-23T09:47:11.418780+0000","last_became_peered":"2024-03-23T09:47:11.418780+0000","last_unstale":"2024-03-23T09:47:11.424183+0000","last_undegraded":"2024-03-23T09:47:11.424183+0000","last_fullsized":"2024-03-23T09:47:11.424183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:03:19.975566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406084+0000","last_change":"2024-03-23T09:47:11.404487+0000","last_active":"2024-03-23T09:47:11.406084+0000","last_peered":"2024-03-23T09:47:11.406084+0000","last_clean":"2024-03-23T09:47:11.406084+0000","last_became_active":"2024-03-23T09:47:11.404234+0000","last_became_peered":"2024-03-23T09:47:11.404234+0000","last_unstale":"2024-03-23T09:47:11.406084+0000","last_undegraded":"2024-03-23T09:47:11.406084+0000","last_fullsized":"2024-03-23T09:47:11.406084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:10:02.020675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421269+0000","last_change":"2024-03-23T09:47:11.413423+0000","last_active":"2024-03-23T09:47:11.421269+0000","last_peered":"2024-03-23T09:47:11.421269+0000","last_clean":"2024-03-23T09:47:11.421269+0000","last_became_active":"2024-03-23T09:47:11.412787+0000","last_became_peered":"2024-03-23T09:47:11.412787+0000","last_unstale":"2024-03-23T09:47:11.421269+0000","last_undegraded":"2024-03-23T09:47:11.421269+0000","last_fullsized":"2024-03-23T09:47:11.421269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:53:52.661871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412487+0000","last_change":"2024-03-23T09:47:11.407990+0000","last_active":"2024-03-23T09:47:11.412487+0000","last_peered":"2024-03-23T09:47:11.412487+0000","last_clean":"2024-03-23T09:47:11.412487+0000","last_became_active":"2024-03-23T09:47:11.407356+0000","last_became_peered":"2024-03-23T09:47:11.407356+0000","last_unstale":"2024-03-23T09:47:11.412487+0000","last_undegraded":"2024-03-23T09:47:11.412487+0000","last_fullsized":"2024-03-23T09:47:11.412487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:08:58.274779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406651+0000","last_change":"2024-03-23T09:47:11.403189+0000","last_active":"2024-03-23T09:47:11.406651+0000","last_peered":"2024-03-23T09:47:11.406651+0000","last_clean":"2024-03-23T09:47:11.406651+0000","last_became_active":"2024-03-23T09:47:11.402767+0000","last_became_peered":"2024-03-23T09:47:11.402767+0000","last_unstale":"2024-03-23T09:47:11.406651+0000","last_undegraded":"2024-03-23T09:47:11.406651+0000","last_fullsized":"2024-03-23T09:47:11.406651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:22.023655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412561+0000","last_change":"2024-03-23T09:47:11.411012+0000","last_active":"2024-03-23T09:47:11.412561+0000","last_peered":"2024-03-23T09:47:11.412561+0000","last_clean":"2024-03-23T09:47:11.412561+0000","last_became_active":"2024-03-23T09:47:11.410764+0000","last_became_peered":"2024-03-23T09:47:11.410764+0000","last_unstale":"2024-03-23T09:47:11.412561+0000","last_undegraded":"2024-03-23T09:47:11.412561+0000","last_fullsized":"2024-03-23T09:47:11.412561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:48:02.922170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.423997+0000","last_change":"2024-03-23T09:47:11.414350+0000","last_active":"2024-03-23T09:47:11.423997+0000","last_peered":"2024-03-23T09:47:11.423997+0000","last_clean":"2024-03-23T09:47:11.423997+0000","last_became_active":"2024-03-23T09:47:11.413980+0000","last_became_peered":"2024-03-23T09:47:11.413980+0000","last_unstale":"2024-03-23T09:47:11.423997+0000","last_undegraded":"2024-03-23T09:47:11.423997+0000","last_fullsized":"2024-03-23T09:47:11.423997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:53:52.661871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.414898+0000","last_change":"2024-03-23T09:47:11.404275+0000","last_active":"2024-03-23T09:47:11.414898+0000","last_peered":"2024-03-23T09:47:11.414898+0000","last_clean":"2024-03-23T09:47:11.414898+0000","last_became_active":"2024-03-23T09:47:11.403931+0000","last_became_peered":"2024-03-23T09:47:11.403931+0000","last_unstale":"2024-03-23T09:47:11.414898+0000","last_undegraded":"2024-03-23T09:47:11.414898+0000","last_fullsized":"2024-03-23T09:47:11.414898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:03:19.975566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.418367+0000","last_change":"2024-03-23T09:47:11.413532+0000","last_active":"2024-03-23T09:47:11.418367+0000","last_peered":"2024-03-23T09:47:11.418367+0000","last_clean":"2024-03-23T09:47:11.418367+0000","last_became_active":"2024-03-23T09:47:11.413352+0000","last_became_peered":"2024-03-23T09:47:11.413352+0000","last_unstale":"2024-03-23T09:47:11.418367+0000","last_undegraded":"2024-03-23T09:47:11.418367+0000","last_fullsized":"2024-03-23T09:47:11.418367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:14:34.499019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.411934+0000","last_change":"2024-03-23T09:47:11.410786+0000","last_active":"2024-03-23T09:47:11.411934+0000","last_peered":"2024-03-23T09:47:11.411934+0000","last_clean":"2024-03-23T09:47:11.411934+0000","last_became_active":"2024-03-23T09:47:11.410560+0000","last_became_peered":"2024-03-23T09:47:11.410560+0000","last_unstale":"2024-03-23T09:47:11.411934+0000","last_undegraded":"2024-03-23T09:47:11.411934+0000","last_fullsized":"2024-03-23T09:47:11.411934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:08:58.274779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412540+0000","last_change":"2024-03-23T09:47:11.408366+0000","last_active":"2024-03-23T09:47:11.412540+0000","last_peered":"2024-03-23T09:47:11.412540+0000","last_clean":"2024-03-23T09:47:11.412540+0000","last_became_active":"2024-03-23T09:47:11.407952+0000","last_became_peered":"2024-03-23T09:47:11.407952+0000","last_unstale":"2024-03-23T09:47:11.412540+0000","last_undegraded":"2024-03-23T09:47:11.412540+0000","last_fullsized":"2024-03-23T09:47:11.412540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:41:39.284406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409033+0000","last_change":"2024-03-23T09:47:11.408177+0000","last_active":"2024-03-23T09:47:11.409033+0000","last_peered":"2024-03-23T09:47:11.409033+0000","last_clean":"2024-03-23T09:47:11.409033+0000","last_became_active":"2024-03-23T09:47:11.407922+0000","last_became_peered":"2024-03-23T09:47:11.407922+0000","last_unstale":"2024-03-23T09:47:11.409033+0000","last_undegraded":"2024-03-23T09:47:11.409033+0000","last_fullsized":"2024-03-23T09:47:11.409033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:20:54.064158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409754+0000","last_change":"2024-03-23T09:47:11.408673+0000","last_active":"2024-03-23T09:47:11.409754+0000","last_peered":"2024-03-23T09:47:11.409754+0000","last_clean":"2024-03-23T09:47:11.409754+0000","last_became_active":"2024-03-23T09:47:11.408014+0000","last_became_peered":"2024-03-23T09:47:11.408014+0000","last_unstale":"2024-03-23T09:47:11.409754+0000","last_undegraded":"2024-03-23T09:47:11.409754+0000","last_fullsized":"2024-03-23T09:47:11.409754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:07:37.588894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421310+0000","last_change":"2024-03-23T09:47:11.413587+0000","last_active":"2024-03-23T09:47:11.421310+0000","last_peered":"2024-03-23T09:47:11.421310+0000","last_clean":"2024-03-23T09:47:11.421310+0000","last_became_active":"2024-03-23T09:47:11.413292+0000","last_became_peered":"2024-03-23T09:47:11.413292+0000","last_unstale":"2024-03-23T09:47:11.421310+0000","last_undegraded":"2024-03-23T09:47:11.421310+0000","last_fullsized":"2024-03-23T09:47:11.421310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:26:43.651127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412186+0000","last_change":"2024-03-23T09:47:11.409156+0000","last_active":"2024-03-23T09:47:11.412186+0000","last_peered":"2024-03-23T09:47:11.412186+0000","last_clean":"2024-03-23T09:47:11.412186+0000","last_became_active":"2024-03-23T09:47:11.408797+0000","last_became_peered":"2024-03-23T09:47:11.408797+0000","last_unstale":"2024-03-23T09:47:11.412186+0000","last_undegraded":"2024-03-23T09:47:11.412186+0000","last_fullsized":"2024-03-23T09:47:11.412186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:42:20.730126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.419832+0000","last_change":"2024-03-23T09:47:11.416759+0000","last_active":"2024-03-23T09:47:11.419832+0000","last_peered":"2024-03-23T09:47:11.419832+0000","last_clean":"2024-03-23T09:47:11.419832+0000","last_became_active":"2024-03-23T09:47:11.416484+0000","last_became_peered":"2024-03-23T09:47:11.416484+0000","last_unstale":"2024-03-23T09:47:11.419832+0000","last_undegraded":"2024-03-23T09:47:11.419832+0000","last_fullsized":"2024-03-23T09:47:11.419832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:15.479312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412487+0000","last_change":"2024-03-23T09:47:11.408214+0000","last_active":"2024-03-23T09:47:11.412487+0000","last_peered":"2024-03-23T09:47:11.412487+0000","last_clean":"2024-03-23T09:47:11.412487+0000","last_became_active":"2024-03-23T09:47:11.407947+0000","last_became_peered":"2024-03-23T09:47:11.407947+0000","last_unstale":"2024-03-23T09:47:11.412487+0000","last_undegraded":"2024-03-23T09:47:11.412487+0000","last_fullsized":"2024-03-23T09:47:11.412487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:59:33.183165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416189+0000","last_change":"2024-03-23T09:47:11.404766+0000","last_active":"2024-03-23T09:47:11.416189+0000","last_peered":"2024-03-23T09:47:11.416189+0000","last_clean":"2024-03-23T09:47:11.416189+0000","last_became_active":"2024-03-23T09:47:11.404503+0000","last_became_peered":"2024-03-23T09:47:11.404503+0000","last_unstale":"2024-03-23T09:47:11.416189+0000","last_undegraded":"2024-03-23T09:47:11.416189+0000","last_fullsized":"2024-03-23T09:47:11.416189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:08:58.274779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424244+0000","last_change":"2024-03-23T09:47:11.419656+0000","last_active":"2024-03-23T09:47:11.424244+0000","last_peered":"2024-03-23T09:47:11.424244+0000","last_clean":"2024-03-23T09:47:11.424244+0000","last_became_active":"2024-03-23T09:47:11.419067+0000","last_became_peered":"2024-03-23T09:47:11.419067+0000","last_unstale":"2024-03-23T09:47:11.424244+0000","last_undegraded":"2024-03-23T09:47:11.424244+0000","last_fullsized":"2024-03-23T09:47:11.424244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:45:12.520727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409089+0000","last_change":"2024-03-23T09:47:11.405310+0000","last_active":"2024-03-23T09:47:11.409089+0000","last_peered":"2024-03-23T09:47:11.409089+0000","last_clean":"2024-03-23T09:47:11.409089+0000","last_became_active":"2024-03-23T09:47:11.404830+0000","last_became_peered":"2024-03-23T09:47:11.404830+0000","last_unstale":"2024-03-23T09:47:11.409089+0000","last_undegraded":"2024-03-23T09:47:11.409089+0000","last_fullsized":"2024-03-23T09:47:11.409089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:35:54.438265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.410472+0000","last_change":"2024-03-23T09:47:11.408966+0000","last_active":"2024-03-23T09:47:11.410472+0000","last_peered":"2024-03-23T09:47:11.410472+0000","last_clean":"2024-03-23T09:47:11.410472+0000","last_became_active":"2024-03-23T09:47:11.408716+0000","last_became_peered":"2024-03-23T09:47:11.408716+0000","last_unstale":"2024-03-23T09:47:11.410472+0000","last_undegraded":"2024-03-23T09:47:11.410472+0000","last_fullsized":"2024-03-23T09:47:11.410472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:22:33.792384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.423976+0000","last_change":"2024-03-23T09:47:11.415729+0000","last_active":"2024-03-23T09:47:11.423976+0000","last_peered":"2024-03-23T09:47:11.423976+0000","last_clean":"2024-03-23T09:47:11.423976+0000","last_became_active":"2024-03-23T09:47:11.415477+0000","last_became_peered":"2024-03-23T09:47:11.415477+0000","last_unstale":"2024-03-23T09:47:11.423976+0000","last_undegraded":"2024-03-23T09:47:11.423976+0000","last_fullsized":"2024-03-23T09:47:11.423976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:30.983075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421529+0000","last_change":"2024-03-23T09:47:11.418138+0000","last_active":"2024-03-23T09:47:11.421529+0000","last_peered":"2024-03-23T09:47:11.421529+0000","last_clean":"2024-03-23T09:47:11.421529+0000","last_became_active":"2024-03-23T09:47:11.417863+0000","last_became_peered":"2024-03-23T09:47:11.417863+0000","last_unstale":"2024-03-23T09:47:11.421529+0000","last_undegraded":"2024-03-23T09:47:11.421529+0000","last_fullsized":"2024-03-23T09:47:11.421529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:21:53.365087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.418448+0000","last_change":"2024-03-23T09:47:11.413398+0000","last_active":"2024-03-23T09:47:11.418448+0000","last_peered":"2024-03-23T09:47:11.418448+0000","last_clean":"2024-03-23T09:47:11.418448+0000","last_became_active":"2024-03-23T09:47:11.413121+0000","last_became_peered":"2024-03-23T09:47:11.413121+0000","last_unstale":"2024-03-23T09:47:11.418448+0000","last_undegraded":"2024-03-23T09:47:11.418448+0000","last_fullsized":"2024-03-23T09:47:11.418448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:26:43.651127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"84'4","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421264+0000","last_change":"2024-03-23T09:47:11.034751+0000","last_active":"2024-03-23T09:47:11.421264+0000","last_peered":"2024-03-23T09:47:11.421264+0000","last_clean":"2024-03-23T09:47:11.421264+0000","last_became_active":"2024-03-23T09:47:11.034458+0000","last_became_peered":"2024-03-23T09:47:11.034458+0000","last_unstale":"2024-03-23T09:47:11.421264+0000","last_undegraded":"2024-03-23T09:47:11.421264+0000","last_fullsized":"2024-03-23T09:47:11.421264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:27:13.329590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421488+0000","last_change":"2024-03-23T09:47:11.417835+0000","last_active":"2024-03-23T09:47:11.421488+0000","last_peered":"2024-03-23T09:47:11.421488+0000","last_clean":"2024-03-23T09:47:11.421488+0000","last_became_active":"2024-03-23T09:47:11.417267+0000","last_became_peered":"2024-03-23T09:47:11.417267+0000","last_unstale":"2024-03-23T09:47:11.421488+0000","last_undegraded":"2024-03-23T09:47:11.421488+0000","last_fullsized":"2024-03-23T09:47:11.421488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:30.983075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416547+0000","last_change":"2024-03-23T09:47:11.033030+0000","last_active":"2024-03-23T09:47:11.416547+0000","last_peered":"2024-03-23T09:47:11.416547+0000","last_clean":"2024-03-23T09:47:11.416547+0000","last_became_active":"2024-03-23T09:47:11.032579+0000","last_became_peered":"2024-03-23T09:47:11.032579+0000","last_unstale":"2024-03-23T09:47:11.416547+0000","last_undegraded":"2024-03-23T09:47:11.416547+0000","last_fullsized":"2024-03-23T09:47:11.416547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:07:36.583568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421288+0000","last_change":"2024-03-23T09:47:11.413661+0000","last_active":"2024-03-23T09:47:11.421288+0000","last_peered":"2024-03-23T09:47:11.421288+0000","last_clean":"2024-03-23T09:47:11.421288+0000","last_became_active":"2024-03-23T09:47:11.413120+0000","last_became_peered":"2024-03-23T09:47:11.413120+0000","last_unstale":"2024-03-23T09:47:11.421288+0000","last_undegraded":"2024-03-23T09:47:11.421288+0000","last_fullsized":"2024-03-23T09:47:11.421288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:59:33.183165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.420004+0000","last_change":"2024-03-23T09:47:11.039090+0000","last_active":"2024-03-23T09:47:11.420004+0000","last_peered":"2024-03-23T09:47:11.420004+0000","last_clean":"2024-03-23T09:47:11.420004+0000","last_became_active":"2024-03-23T09:47:11.038823+0000","last_became_peered":"2024-03-23T09:47:11.038823+0000","last_unstale":"2024-03-23T09:47:11.420004+0000","last_undegraded":"2024-03-23T09:47:11.420004+0000","last_fullsized":"2024-03-23T09:47:11.420004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:26:28.173693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412491+0000","last_change":"2024-03-23T09:47:11.409787+0000","last_active":"2024-03-23T09:47:11.412491+0000","last_peered":"2024-03-23T09:47:11.412491+0000","last_clean":"2024-03-23T09:47:11.412491+0000","last_became_active":"2024-03-23T09:47:11.409509+0000","last_became_peered":"2024-03-23T09:47:11.409509+0000","last_unstale":"2024-03-23T09:47:11.412491+0000","last_undegraded":"2024-03-23T09:47:11.412491+0000","last_fullsized":"2024-03-23T09:47:11.412491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:03:19.975566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.420150+0000","last_change":"2024-03-23T09:47:11.038814+0000","last_active":"2024-03-23T09:47:11.420150+0000","last_peered":"2024-03-23T09:47:11.420150+0000","last_clean":"2024-03-23T09:47:11.420150+0000","last_became_active":"2024-03-23T09:47:11.038374+0000","last_became_peered":"2024-03-23T09:47:11.038374+0000","last_unstale":"2024-03-23T09:47:11.420150+0000","last_undegraded":"2024-03-23T09:47:11.420150+0000","last_fullsized":"2024-03-23T09:47:11.420150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:06:10.478448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409770+0000","last_change":"2024-03-23T09:47:11.408837+0000","last_active":"2024-03-23T09:47:11.409770+0000","last_peered":"2024-03-23T09:47:11.409770+0000","last_clean":"2024-03-23T09:47:11.409770+0000","last_became_active":"2024-03-23T09:47:11.408412+0000","last_became_peered":"2024-03-23T09:47:11.408412+0000","last_unstale":"2024-03-23T09:47:11.409770+0000","last_undegraded":"2024-03-23T09:47:11.409770+0000","last_fullsized":"2024-03-23T09:47:11.409770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:07:33.336490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.417279+0000","last_change":"2024-03-23T09:47:11.038235+0000","last_active":"2024-03-23T09:47:11.417279+0000","last_peered":"2024-03-23T09:47:11.417279+0000","last_clean":"2024-03-23T09:47:11.417279+0000","last_became_active":"2024-03-23T09:47:11.037820+0000","last_became_peered":"2024-03-23T09:47:11.037820+0000","last_unstale":"2024-03-23T09:47:11.417279+0000","last_undegraded":"2024-03-23T09:47:11.417279+0000","last_fullsized":"2024-03-23T09:47:11.417279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:17.326242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416077+0000","last_change":"2024-03-23T09:47:11.405285+0000","last_active":"2024-03-23T09:47:11.416077+0000","last_peered":"2024-03-23T09:47:11.416077+0000","last_clean":"2024-03-23T09:47:11.416077+0000","last_became_active":"2024-03-23T09:47:11.404560+0000","last_became_peered":"2024-03-23T09:47:11.404560+0000","last_unstale":"2024-03-23T09:47:11.416077+0000","last_undegraded":"2024-03-23T09:47:11.416077+0000","last_fullsized":"2024-03-23T09:47:11.416077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:45:12.520727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416906+0000","last_change":"2024-03-23T09:47:11.035230+0000","last_active":"2024-03-23T09:47:11.416906+0000","last_peered":"2024-03-23T09:47:11.416906+0000","last_clean":"2024-03-23T09:47:11.416906+0000","last_became_active":"2024-03-23T09:47:11.034777+0000","last_became_peered":"2024-03-23T09:47:11.034777+0000","last_unstale":"2024-03-23T09:47:11.416906+0000","last_undegraded":"2024-03-23T09:47:11.416906+0000","last_fullsized":"2024-03-23T09:47:11.416906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:26:08.320253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412317+0000","last_change":"2024-03-23T09:47:11.409098+0000","last_active":"2024-03-23T09:47:11.412317+0000","last_peered":"2024-03-23T09:47:11.412317+0000","last_clean":"2024-03-23T09:47:11.412317+0000","last_became_active":"2024-03-23T09:47:11.408784+0000","last_became_peered":"2024-03-23T09:47:11.408784+0000","last_unstale":"2024-03-23T09:47:11.412317+0000","last_undegraded":"2024-03-23T09:47:11.412317+0000","last_fullsized":"2024-03-23T09:47:11.412317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:16:21.633326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406081+0000","last_change":"2024-03-23T09:47:11.040129+0000","last_active":"2024-03-23T09:47:11.406081+0000","last_peered":"2024-03-23T09:47:11.406081+0000","last_clean":"2024-03-23T09:47:11.406081+0000","last_became_active":"2024-03-23T09:47:11.039770+0000","last_became_peered":"2024-03-23T09:47:11.039770+0000","last_unstale":"2024-03-23T09:47:11.406081+0000","last_undegraded":"2024-03-23T09:47:11.406081+0000","last_fullsized":"2024-03-23T09:47:11.406081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:45:35.363288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409731+0000","last_change":"2024-03-23T09:47:11.409151+0000","last_active":"2024-03-23T09:47:11.409731+0000","last_peered":"2024-03-23T09:47:11.409731+0000","last_clean":"2024-03-23T09:47:11.409731+0000","last_became_active":"2024-03-23T09:47:11.408897+0000","last_became_peered":"2024-03-23T09:47:11.408897+0000","last_unstale":"2024-03-23T09:47:11.409731+0000","last_undegraded":"2024-03-23T09:47:11.409731+0000","last_fullsized":"2024-03-23T09:47:11.409731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:08:20.711770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.396433+0000","last_change":"2024-03-23T09:47:11.035672+0000","last_active":"2024-03-23T09:47:11.396433+0000","last_peered":"2024-03-23T09:47:11.396433+0000","last_clean":"2024-03-23T09:47:11.396433+0000","last_became_active":"2024-03-23T09:47:11.035424+0000","last_became_peered":"2024-03-23T09:47:11.035424+0000","last_unstale":"2024-03-23T09:47:11.396433+0000","last_undegraded":"2024-03-23T09:47:11.396433+0000","last_fullsized":"2024-03-23T09:47:11.396433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:52:50.532114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.407440+0000","last_change":"2024-03-23T09:47:11.404596+0000","last_active":"2024-03-23T09:47:11.407440+0000","last_peered":"2024-03-23T09:47:11.407440+0000","last_clean":"2024-03-23T09:47:11.407440+0000","last_became_active":"2024-03-23T09:47:11.404362+0000","last_became_peered":"2024-03-23T09:47:11.404362+0000","last_unstale":"2024-03-23T09:47:11.407440+0000","last_undegraded":"2024-03-23T09:47:11.407440+0000","last_fullsized":"2024-03-23T09:47:11.407440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:28:40.156425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.408857+0000","last_change":"2024-03-23T09:47:11.037991+0000","last_active":"2024-03-23T09:47:11.408857+0000","last_peered":"2024-03-23T09:47:11.408857+0000","last_clean":"2024-03-23T09:47:11.408857+0000","last_became_active":"2024-03-23T09:47:11.037694+0000","last_became_peered":"2024-03-23T09:47:11.037694+0000","last_unstale":"2024-03-23T09:47:11.408857+0000","last_undegraded":"2024-03-23T09:47:11.408857+0000","last_fullsized":"2024-03-23T09:47:11.408857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:52:08.866579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.417692+0000","last_change":"2024-03-23T09:47:11.413118+0000","last_active":"2024-03-23T09:47:11.417692+0000","last_peered":"2024-03-23T09:47:11.417692+0000","last_clean":"2024-03-23T09:47:11.417692+0000","last_became_active":"2024-03-23T09:47:11.412874+0000","last_became_peered":"2024-03-23T09:47:11.412874+0000","last_unstale":"2024-03-23T09:47:11.417692+0000","last_undegraded":"2024-03-23T09:47:11.417692+0000","last_fullsized":"2024-03-23T09:47:11.417692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:22:33.792384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409552+0000","last_change":"2024-03-23T09:47:11.036496+0000","last_active":"2024-03-23T09:47:11.409552+0000","last_peered":"2024-03-23T09:47:11.409552+0000","last_clean":"2024-03-23T09:47:11.409552+0000","last_became_active":"2024-03-23T09:47:11.036242+0000","last_became_peered":"2024-03-23T09:47:11.036242+0000","last_unstale":"2024-03-23T09:47:11.409552+0000","last_undegraded":"2024-03-23T09:47:11.409552+0000","last_fullsized":"2024-03-23T09:47:11.409552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:28:41.141201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.418667+0000","last_change":"2024-03-23T09:47:11.416306+0000","last_active":"2024-03-23T09:47:11.418667+0000","last_peered":"2024-03-23T09:47:11.418667+0000","last_clean":"2024-03-23T09:47:11.418667+0000","last_became_active":"2024-03-23T09:47:11.416072+0000","last_became_peered":"2024-03-23T09:47:11.416072+0000","last_unstale":"2024-03-23T09:47:11.418667+0000","last_undegraded":"2024-03-23T09:47:11.418667+0000","last_fullsized":"2024-03-23T09:47:11.418667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:33:16.231177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"84'3","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.420081+0000","last_change":"2024-03-23T09:47:11.038881+0000","last_active":"2024-03-23T09:47:11.420081+0000","last_peered":"2024-03-23T09:47:11.420081+0000","last_clean":"2024-03-23T09:47:11.420081+0000","last_became_active":"2024-03-23T09:47:11.038582+0000","last_became_peered":"2024-03-23T09:47:11.038582+0000","last_unstale":"2024-03-23T09:47:11.420081+0000","last_undegraded":"2024-03-23T09:47:11.420081+0000","last_fullsized":"2024-03-23T09:47:11.420081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:08.501328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.423946+0000","last_change":"2024-03-23T09:47:11.414821+0000","last_active":"2024-03-23T09:47:11.423946+0000","last_peered":"2024-03-23T09:47:11.423946+0000","last_clean":"2024-03-23T09:47:11.423946+0000","last_became_active":"2024-03-23T09:47:11.414581+0000","last_became_peered":"2024-03-23T09:47:11.414581+0000","last_unstale":"2024-03-23T09:47:11.423946+0000","last_undegraded":"2024-03-23T09:47:11.423946+0000","last_fullsized":"2024-03-23T09:47:11.423946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:16:21.633326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.423828+0000","last_change":"2024-03-23T09:47:11.037998+0000","last_active":"2024-03-23T09:47:11.423828+0000","last_peered":"2024-03-23T09:47:11.423828+0000","last_clean":"2024-03-23T09:47:11.423828+0000","last_became_active":"2024-03-23T09:47:11.037749+0000","last_became_peered":"2024-03-23T09:47:11.037749+0000","last_unstale":"2024-03-23T09:47:11.423828+0000","last_undegraded":"2024-03-23T09:47:11.423828+0000","last_fullsized":"2024-03-23T09:47:11.423828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:41:38.279080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424897+0000","last_change":"2024-03-23T09:47:11.424435+0000","last_active":"2024-03-23T09:47:11.424897+0000","last_peered":"2024-03-23T09:47:11.424897+0000","last_clean":"2024-03-23T09:47:11.424897+0000","last_became_active":"2024-03-23T09:47:11.424169+0000","last_became_peered":"2024-03-23T09:47:11.424169+0000","last_unstale":"2024-03-23T09:47:11.424897+0000","last_undegraded":"2024-03-23T09:47:11.424897+0000","last_fullsized":"2024-03-23T09:47:11.424897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:16:21.633326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.422731+0000","last_change":"2024-03-23T09:47:11.041094+0000","last_active":"2024-03-23T09:47:11.422731+0000","last_peered":"2024-03-23T09:47:11.422731+0000","last_clean":"2024-03-23T09:47:11.422731+0000","last_became_active":"2024-03-23T09:47:11.040738+0000","last_became_peered":"2024-03-23T09:47:11.040738+0000","last_unstale":"2024-03-23T09:47:11.422731+0000","last_undegraded":"2024-03-23T09:47:11.422731+0000","last_fullsized":"2024-03-23T09:47:11.422731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:28:41.141201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412509+0000","last_change":"2024-03-23T09:47:11.409127+0000","last_active":"2024-03-23T09:47:11.412509+0000","last_peered":"2024-03-23T09:47:11.412509+0000","last_clean":"2024-03-23T09:47:11.412509+0000","last_became_active":"2024-03-23T09:47:11.408622+0000","last_became_peered":"2024-03-23T09:47:11.408622+0000","last_unstale":"2024-03-23T09:47:11.412509+0000","last_undegraded":"2024-03-23T09:47:11.412509+0000","last_fullsized":"2024-03-23T09:47:11.412509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:59:56.765267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.396673+0000","last_change":"2024-03-23T09:47:11.038063+0000","last_active":"2024-03-23T09:47:11.396673+0000","last_peered":"2024-03-23T09:47:11.396673+0000","last_clean":"2024-03-23T09:47:11.396673+0000","last_became_active":"2024-03-23T09:47:11.037488+0000","last_became_peered":"2024-03-23T09:47:11.037488+0000","last_unstale":"2024-03-23T09:47:11.396673+0000","last_undegraded":"2024-03-23T09:47:11.396673+0000","last_fullsized":"2024-03-23T09:47:11.396673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:59:55.759941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409794+0000","last_change":"2024-03-23T09:47:11.407676+0000","last_active":"2024-03-23T09:47:11.409794+0000","last_peered":"2024-03-23T09:47:11.409794+0000","last_clean":"2024-03-23T09:47:11.409794+0000","last_became_active":"2024-03-23T09:47:11.407442+0000","last_became_peered":"2024-03-23T09:47:11.407442+0000","last_unstale":"2024-03-23T09:47:11.409794+0000","last_undegraded":"2024-03-23T09:47:11.409794+0000","last_fullsized":"2024-03-23T09:47:11.409794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:52:59.938205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.402747+0000","last_change":"2024-03-23T09:47:11.039946+0000","last_active":"2024-03-23T09:47:11.402747+0000","last_peered":"2024-03-23T09:47:11.402747+0000","last_clean":"2024-03-23T09:47:11.402747+0000","last_became_active":"2024-03-23T09:47:11.039660+0000","last_became_peered":"2024-03-23T09:47:11.039660+0000","last_unstale":"2024-03-23T09:47:11.402747+0000","last_undegraded":"2024-03-23T09:47:11.402747+0000","last_fullsized":"2024-03-23T09:47:11.402747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:20:53.058832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.419889+0000","last_change":"2024-03-23T09:47:11.416706+0000","last_active":"2024-03-23T09:47:11.419889+0000","last_peered":"2024-03-23T09:47:11.419889+0000","last_clean":"2024-03-23T09:47:11.419889+0000","last_became_active":"2024-03-23T09:47:11.416460+0000","last_became_peered":"2024-03-23T09:47:11.416460+0000","last_unstale":"2024-03-23T09:47:11.419889+0000","last_undegraded":"2024-03-23T09:47:11.419889+0000","last_fullsized":"2024-03-23T09:47:11.419889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:40.160543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421846+0000","last_change":"2024-03-23T09:47:11.042502+0000","last_active":"2024-03-23T09:47:11.421846+0000","last_peered":"2024-03-23T09:47:11.421846+0000","last_clean":"2024-03-23T09:47:11.421846+0000","last_became_active":"2024-03-23T09:47:11.042193+0000","last_became_peered":"2024-03-23T09:47:11.042193+0000","last_unstale":"2024-03-23T09:47:11.421846+0000","last_undegraded":"2024-03-23T09:47:11.421846+0000","last_fullsized":"2024-03-23T09:47:11.421846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:35:53.432938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412511+0000","last_change":"2024-03-23T09:47:11.409298+0000","last_active":"2024-03-23T09:47:11.412511+0000","last_peered":"2024-03-23T09:47:11.412511+0000","last_clean":"2024-03-23T09:47:11.412511+0000","last_became_active":"2024-03-23T09:47:11.408207+0000","last_became_peered":"2024-03-23T09:47:11.408207+0000","last_unstale":"2024-03-23T09:47:11.412511+0000","last_undegraded":"2024-03-23T09:47:11.412511+0000","last_fullsized":"2024-03-23T09:47:11.412511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:38:15.086915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"84'1","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.420368+0000","last_change":"2024-03-23T09:47:11.036587+0000","last_active":"2024-03-23T09:47:11.420368+0000","last_peered":"2024-03-23T09:47:11.420368+0000","last_clean":"2024-03-23T09:47:11.420368+0000","last_became_active":"2024-03-23T09:47:11.036143+0000","last_became_peered":"2024-03-23T09:47:11.036143+0000","last_unstale":"2024-03-23T09:47:11.420368+0000","last_undegraded":"2024-03-23T09:47:11.420368+0000","last_fullsized":"2024-03-23T09:47:11.420368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:04.866753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421363+0000","last_change":"2024-03-23T09:47:11.414510+0000","last_active":"2024-03-23T09:47:11.421363+0000","last_peered":"2024-03-23T09:47:11.421363+0000","last_clean":"2024-03-23T09:47:11.421363+0000","last_became_active":"2024-03-23T09:47:11.414251+0000","last_became_peered":"2024-03-23T09:47:11.414251+0000","last_unstale":"2024-03-23T09:47:11.421363+0000","last_undegraded":"2024-03-23T09:47:11.421363+0000","last_fullsized":"2024-03-23T09:47:11.421363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:46:56.811017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424100+0000","last_change":"2024-03-23T09:47:11.043123+0000","last_active":"2024-03-23T09:47:11.424100+0000","last_peered":"2024-03-23T09:47:11.424100+0000","last_clean":"2024-03-23T09:47:11.424100+0000","last_became_active":"2024-03-23T09:47:11.042890+0000","last_became_peered":"2024-03-23T09:47:11.042890+0000","last_unstale":"2024-03-23T09:47:11.424100+0000","last_undegraded":"2024-03-23T09:47:11.424100+0000","last_fullsized":"2024-03-23T09:47:11.424100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:10:20.426205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416208+0000","last_change":"2024-03-23T09:47:11.404906+0000","last_active":"2024-03-23T09:47:11.416208+0000","last_peered":"2024-03-23T09:47:11.416208+0000","last_clean":"2024-03-23T09:47:11.416208+0000","last_became_active":"2024-03-23T09:47:11.404521+0000","last_became_peered":"2024-03-23T09:47:11.404521+0000","last_unstale":"2024-03-23T09:47:11.416208+0000","last_undegraded":"2024-03-23T09:47:11.416208+0000","last_fullsized":"2024-03-23T09:47:11.416208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:21:53.365087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406640+0000","last_change":"2024-03-23T09:47:11.040175+0000","last_active":"2024-03-23T09:47:11.406640+0000","last_peered":"2024-03-23T09:47:11.406640+0000","last_clean":"2024-03-23T09:47:11.406640+0000","last_became_active":"2024-03-23T09:47:11.039927+0000","last_became_peered":"2024-03-23T09:47:11.039927+0000","last_unstale":"2024-03-23T09:47:11.406640+0000","last_undegraded":"2024-03-23T09:47:11.406640+0000","last_fullsized":"2024-03-23T09:47:11.406640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:10:01.015349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406633+0000","last_change":"2024-03-23T09:47:11.403033+0000","last_active":"2024-03-23T09:47:11.406633+0000","last_peered":"2024-03-23T09:47:11.406633+0000","last_clean":"2024-03-23T09:47:11.406633+0000","last_became_active":"2024-03-23T09:47:11.402436+0000","last_became_peered":"2024-03-23T09:47:11.402436+0000","last_unstale":"2024-03-23T09:47:11.406633+0000","last_undegraded":"2024-03-23T09:47:11.406633+0000","last_fullsized":"2024-03-23T09:47:11.406633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:28:42.146527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.407703+0000","last_change":"2024-03-23T09:47:11.030291+0000","last_active":"2024-03-23T09:47:11.407703+0000","last_peered":"2024-03-23T09:47:11.407703+0000","last_clean":"2024-03-23T09:47:11.407703+0000","last_became_active":"2024-03-23T09:47:11.030047+0000","last_became_peered":"2024-03-23T09:47:11.030047+0000","last_unstale":"2024-03-23T09:47:11.407703+0000","last_undegraded":"2024-03-23T09:47:11.407703+0000","last_fullsized":"2024-03-23T09:47:11.407703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:06:15.483933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.411968+0000","last_change":"2024-03-23T09:47:11.410537+0000","last_active":"2024-03-23T09:47:11.411968+0000","last_peered":"2024-03-23T09:47:11.411968+0000","last_clean":"2024-03-23T09:47:11.411968+0000","last_became_active":"2024-03-23T09:47:11.410278+0000","last_became_peered":"2024-03-23T09:47:11.410278+0000","last_unstale":"2024-03-23T09:47:11.411968+0000","last_undegraded":"2024-03-23T09:47:11.411968+0000","last_fullsized":"2024-03-23T09:47:11.411968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:52:59.938205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.394248+0000","last_change":"2024-03-23T09:47:11.035723+0000","last_active":"2024-03-23T09:47:11.394248+0000","last_peered":"2024-03-23T09:47:11.394248+0000","last_clean":"2024-03-23T09:47:11.394248+0000","last_became_active":"2024-03-23T09:47:11.035455+0000","last_became_peered":"2024-03-23T09:47:11.035455+0000","last_unstale":"2024-03-23T09:47:11.394248+0000","last_undegraded":"2024-03-23T09:47:11.394248+0000","last_fullsized":"2024-03-23T09:47:11.394248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:43:34.477625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412972+0000","last_change":"2024-03-23T09:47:11.409615+0000","last_active":"2024-03-23T09:47:11.412972+0000","last_peered":"2024-03-23T09:47:11.412972+0000","last_clean":"2024-03-23T09:47:11.412972+0000","last_became_active":"2024-03-23T09:47:11.409329+0000","last_became_peered":"2024-03-23T09:47:11.409329+0000","last_unstale":"2024-03-23T09:47:11.412972+0000","last_undegraded":"2024-03-23T09:47:11.412972+0000","last_fullsized":"2024-03-23T09:47:11.412972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:38.307851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.393880+0000","last_change":"2024-03-23T09:47:11.036698+0000","last_active":"2024-03-23T09:47:11.393880+0000","last_peered":"2024-03-23T09:47:11.393880+0000","last_clean":"2024-03-23T09:47:11.393880+0000","last_became_active":"2024-03-23T09:47:11.036444+0000","last_became_peered":"2024-03-23T09:47:11.036444+0000","last_unstale":"2024-03-23T09:47:11.393880+0000","last_undegraded":"2024-03-23T09:47:11.393880+0000","last_fullsized":"2024-03-23T09:47:11.393880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:27:12.582469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.423795+0000","last_change":"2024-03-23T09:47:11.419859+0000","last_active":"2024-03-23T09:47:11.423795+0000","last_peered":"2024-03-23T09:47:11.423795+0000","last_clean":"2024-03-23T09:47:11.423795+0000","last_became_active":"2024-03-23T09:47:11.419360+0000","last_became_peered":"2024-03-23T09:47:11.419360+0000","last_unstale":"2024-03-23T09:47:11.423795+0000","last_undegraded":"2024-03-23T09:47:11.423795+0000","last_fullsized":"2024-03-23T09:47:11.423795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:40.160543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.397198+0000","last_change":"2024-03-23T09:47:11.030195+0000","last_active":"2024-03-23T09:47:11.397198+0000","last_peered":"2024-03-23T09:47:11.397198+0000","last_clean":"2024-03-23T09:47:11.397198+0000","last_became_active":"2024-03-23T09:47:11.029904+0000","last_became_peered":"2024-03-23T09:47:11.029904+0000","last_unstale":"2024-03-23T09:47:11.397198+0000","last_undegraded":"2024-03-23T09:47:11.397198+0000","last_fullsized":"2024-03-23T09:47:11.397198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:38:14.081589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416622+0000","last_change":"2024-03-23T09:47:11.411692+0000","last_active":"2024-03-23T09:47:11.416622+0000","last_peered":"2024-03-23T09:47:11.416622+0000","last_clean":"2024-03-23T09:47:11.416622+0000","last_became_active":"2024-03-23T09:47:11.411455+0000","last_became_peered":"2024-03-23T09:47:11.411455+0000","last_unstale":"2024-03-23T09:47:11.416622+0000","last_undegraded":"2024-03-23T09:47:11.416622+0000","last_fullsized":"2024-03-23T09:47:11.416622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:59:33.183165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424350+0000","last_change":"2024-03-23T09:47:11.042468+0000","last_active":"2024-03-23T09:47:11.424350+0000","last_peered":"2024-03-23T09:47:11.424350+0000","last_clean":"2024-03-23T09:47:11.424350+0000","last_became_active":"2024-03-23T09:47:11.042161+0000","last_became_peered":"2024-03-23T09:47:11.042161+0000","last_unstale":"2024-03-23T09:47:11.424350+0000","last_undegraded":"2024-03-23T09:47:11.424350+0000","last_fullsized":"2024-03-23T09:47:11.424350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:14:33.493692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424087+0000","last_change":"2024-03-23T09:47:11.419426+0000","last_active":"2024-03-23T09:47:11.424087+0000","last_peered":"2024-03-23T09:47:11.424087+0000","last_clean":"2024-03-23T09:47:11.424087+0000","last_became_active":"2024-03-23T09:47:11.419138+0000","last_became_peered":"2024-03-23T09:47:11.419138+0000","last_unstale":"2024-03-23T09:47:11.424087+0000","last_undegraded":"2024-03-23T09:47:11.424087+0000","last_fullsized":"2024-03-23T09:47:11.424087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:33:16.231177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.398091+0000","last_change":"2024-03-23T09:47:11.035761+0000","last_active":"2024-03-23T09:47:11.398091+0000","last_peered":"2024-03-23T09:47:11.398091+0000","last_clean":"2024-03-23T09:47:11.398091+0000","last_became_active":"2024-03-23T09:47:11.035526+0000","last_became_peered":"2024-03-23T09:47:11.035526+0000","last_unstale":"2024-03-23T09:47:11.398091+0000","last_undegraded":"2024-03-23T09:47:11.398091+0000","last_fullsized":"2024-03-23T09:47:11.398091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:21.018329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424209+0000","last_change":"2024-03-23T09:47:11.420249+0000","last_active":"2024-03-23T09:47:11.424209+0000","last_peered":"2024-03-23T09:47:11.424209+0000","last_clean":"2024-03-23T09:47:11.424209+0000","last_became_active":"2024-03-23T09:47:11.419410+0000","last_became_peered":"2024-03-23T09:47:11.419410+0000","last_unstale":"2024-03-23T09:47:11.424209+0000","last_undegraded":"2024-03-23T09:47:11.424209+0000","last_fullsized":"2024-03-23T09:47:11.424209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:10:21.431532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.405708+0000","last_change":"2024-03-23T09:47:11.035478+0000","last_active":"2024-03-23T09:47:11.405708+0000","last_peered":"2024-03-23T09:47:11.405708+0000","last_clean":"2024-03-23T09:47:11.405708+0000","last_became_active":"2024-03-23T09:47:11.034616+0000","last_became_peered":"2024-03-23T09:47:11.034616+0000","last_unstale":"2024-03-23T09:47:11.405708+0000","last_undegraded":"2024-03-23T09:47:11.405708+0000","last_fullsized":"2024-03-23T09:47:11.405708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:41:38.279080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424024+0000","last_change":"2024-03-23T09:47:11.416738+0000","last_active":"2024-03-23T09:47:11.424024+0000","last_peered":"2024-03-23T09:47:11.424024+0000","last_clean":"2024-03-23T09:47:11.424024+0000","last_became_active":"2024-03-23T09:47:11.415878+0000","last_became_peered":"2024-03-23T09:47:11.415878+0000","last_unstale":"2024-03-23T09:47:11.424024+0000","last_undegraded":"2024-03-23T09:47:11.424024+0000","last_fullsized":"2024-03-23T09:47:11.424024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:33:16.231177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409142+0000","last_change":"2024-03-23T09:47:11.038803+0000","last_active":"2024-03-23T09:47:11.409142+0000","last_peered":"2024-03-23T09:47:11.409142+0000","last_clean":"2024-03-23T09:47:11.409142+0000","last_became_active":"2024-03-23T09:47:11.038534+0000","last_became_peered":"2024-03-23T09:47:11.038534+0000","last_unstale":"2024-03-23T09:47:11.409142+0000","last_undegraded":"2024-03-23T09:47:11.409142+0000","last_fullsized":"2024-03-23T09:47:11.409142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:52:50.532114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.408715+0000","last_change":"2024-03-23T09:47:11.406913+0000","last_active":"2024-03-23T09:47:11.408715+0000","last_peered":"2024-03-23T09:47:11.408715+0000","last_clean":"2024-03-23T09:47:11.408715+0000","last_became_active":"2024-03-23T09:47:11.406649+0000","last_became_peered":"2024-03-23T09:47:11.406649+0000","last_unstale":"2024-03-23T09:47:11.408715+0000","last_undegraded":"2024-03-23T09:47:11.408715+0000","last_fullsized":"2024-03-23T09:47:11.408715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:45:36.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.407074+0000","last_change":"2024-03-23T09:47:11.042912+0000","last_active":"2024-03-23T09:47:11.407074+0000","last_peered":"2024-03-23T09:47:11.407074+0000","last_clean":"2024-03-23T09:47:11.407074+0000","last_became_active":"2024-03-23T09:47:11.042517+0000","last_became_peered":"2024-03-23T09:47:11.042517+0000","last_unstale":"2024-03-23T09:47:11.407074+0000","last_undegraded":"2024-03-23T09:47:11.407074+0000","last_fullsized":"2024-03-23T09:47:11.407074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:46:55.805690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.419868+0000","last_change":"2024-03-23T09:47:11.419080+0000","last_active":"2024-03-23T09:47:11.419868+0000","last_peered":"2024-03-23T09:47:11.419868+0000","last_clean":"2024-03-23T09:47:11.419868+0000","last_became_active":"2024-03-23T09:47:11.418788+0000","last_became_peered":"2024-03-23T09:47:11.418788+0000","last_unstale":"2024-03-23T09:47:11.419868+0000","last_undegraded":"2024-03-23T09:47:11.419868+0000","last_fullsized":"2024-03-23T09:47:11.419868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:21:53.365087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.398216+0000","last_change":"2024-03-23T09:47:11.032898+0000","last_active":"2024-03-23T09:47:11.398216+0000","last_peered":"2024-03-23T09:47:11.398216+0000","last_clean":"2024-03-23T09:47:11.398216+0000","last_became_active":"2024-03-23T09:47:11.032593+0000","last_became_peered":"2024-03-23T09:47:11.032593+0000","last_unstale":"2024-03-23T09:47:11.398216+0000","last_undegraded":"2024-03-23T09:47:11.398216+0000","last_fullsized":"2024-03-23T09:47:11.398216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:15:27.762824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412557+0000","last_change":"2024-03-23T09:47:11.409994+0000","last_active":"2024-03-23T09:47:11.412557+0000","last_peered":"2024-03-23T09:47:11.412557+0000","last_clean":"2024-03-23T09:47:11.412557+0000","last_became_active":"2024-03-23T09:47:11.409056+0000","last_became_peered":"2024-03-23T09:47:11.409056+0000","last_unstale":"2024-03-23T09:47:11.412557+0000","last_undegraded":"2024-03-23T09:47:11.412557+0000","last_fullsized":"2024-03-23T09:47:11.412557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:30.983075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406791+0000","last_change":"2024-03-23T09:47:11.037345+0000","last_active":"2024-03-23T09:47:11.406791+0000","last_peered":"2024-03-23T09:47:11.406791+0000","last_clean":"2024-03-23T09:47:11.406791+0000","last_became_active":"2024-03-23T09:47:11.037044+0000","last_became_peered":"2024-03-23T09:47:11.037044+0000","last_unstale":"2024-03-23T09:47:11.406791+0000","last_undegraded":"2024-03-23T09:47:11.406791+0000","last_fullsized":"2024-03-23T09:47:11.406791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:21.018329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421329+0000","last_change":"2024-03-23T09:47:11.414063+0000","last_active":"2024-03-23T09:47:11.421329+0000","last_peered":"2024-03-23T09:47:11.421329+0000","last_clean":"2024-03-23T09:47:11.421329+0000","last_became_active":"2024-03-23T09:47:11.413828+0000","last_became_peered":"2024-03-23T09:47:11.413828+0000","last_unstale":"2024-03-23T09:47:11.421329+0000","last_undegraded":"2024-03-23T09:47:11.421329+0000","last_fullsized":"2024-03-23T09:47:11.421329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:06:11.483774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.398600+0000","last_change":"2024-03-23T09:47:11.036536+0000","last_active":"2024-03-23T09:47:11.398600+0000","last_peered":"2024-03-23T09:47:11.398600+0000","last_clean":"2024-03-23T09:47:11.398600+0000","last_became_active":"2024-03-23T09:47:11.036119+0000","last_became_peered":"2024-03-23T09:47:11.036119+0000","last_unstale":"2024-03-23T09:47:11.398600+0000","last_undegraded":"2024-03-23T09:47:11.398600+0000","last_fullsized":"2024-03-23T09:47:11.398600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:08:29.870983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409402+0000","last_change":"2024-03-23T09:47:11.406594+0000","last_active":"2024-03-23T09:47:11.409402+0000","last_peered":"2024-03-23T09:47:11.409402+0000","last_clean":"2024-03-23T09:47:11.409402+0000","last_became_active":"2024-03-23T09:47:11.406362+0000","last_became_peered":"2024-03-23T09:47:11.406362+0000","last_unstale":"2024-03-23T09:47:11.409402+0000","last_undegraded":"2024-03-23T09:47:11.409402+0000","last_fullsized":"2024-03-23T09:47:11.409402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:26:09.325579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409300+0000","last_change":"2024-03-23T09:47:11.042365+0000","last_active":"2024-03-23T09:47:11.409300+0000","last_peered":"2024-03-23T09:47:11.409300+0000","last_clean":"2024-03-23T09:47:11.409300+0000","last_became_active":"2024-03-23T09:47:11.041966+0000","last_became_peered":"2024-03-23T09:47:11.041966+0000","last_unstale":"2024-03-23T09:47:11.409300+0000","last_undegraded":"2024-03-23T09:47:11.409300+0000","last_fullsized":"2024-03-23T09:47:11.409300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:14.473986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":230,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.034578+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:47:11.034578+0000","last_peered":"2024-03-23T09:47:11.034578+0000","last_clean":"2024-03-23T09:47:11.034578+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:47:11.034578+0000","last_undegraded":"2024-03-23T09:47:11.034578+0000","last_fullsized":"2024-03-23T09:47:11.034578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1630208,"data_stored":1027680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.499},"min":{"1min":0.35799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67600000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58699999999999997,"15min":0.52300000000000002},"min":{"1min":0.42899999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71999999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45400000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57299999999999995,"15min":0.52300000000000002},"min":{"1min":0.38300000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69699999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.52200000000000002},"min":{"1min":0.33200000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76000000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40600000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.53900000000000003},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.34699999999999998},"max":{"1min":0.73199999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.45900000000000002,"5min":0.42299999999999999,"15min":0.39900000000000002},"max":{"1min":0.74199999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.41899999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57799999999999996,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.373},"max":{"1min":0.73899999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.436},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.45400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69699999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.39100000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.499,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76300000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.51900000000000002},"min":{"1min":0.38200000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74399999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34999999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.77300000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.56299999999999994},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.36899999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.58299999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.753,"5min":0.874,"15min":0.874},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.38600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76200000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.32200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1040384,"data_stored":437317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55800000000000005,"15min":0.52900000000000003},"min":{"1min":0.48299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.37}]},{"osd":1,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54900000000000004,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.432},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.40999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54600000000000004,"15min":0.52500000000000002},"min":{"1min":0.39000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.39400000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.42299999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.44600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40799999999999997}]},{"osd":4,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48299999999999998,"15min":0.503},"min":{"1min":0.33300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69799999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57199999999999995}]},{"osd":5,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.503,"15min":0.52900000000000003},"min":{"1min":0.39800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.378,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.746,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51600000000000001}]},{"osd":7,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55300000000000005,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78400000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":438828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54900000000000004,"15min":0.49299999999999999},"min":{"1min":0.42799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.752,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.628,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.27400000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54700000000000004,"15min":0.499},"min":{"1min":0.371,"5min":0.371,"15min":0.35199999999999998},"max":{"1min":0.623,"5min":0.746,"15min":0.746},"last":0.313},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.51200000000000001},"min":{"1min":0.44400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72499999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.38300000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57199999999999995,"15min":0.53600000000000003},"min":{"1min":0.41399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72699999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56599999999999995,"15min":0.52600000000000002},"min":{"1min":0.434,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.435}]},{"osd":3,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56899999999999995,"15min":0.51400000000000001},"min":{"1min":0.45500000000000002,"5min":0.376,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.371},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.54100000000000004},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":0.71199999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.35899999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.45000000000000001,"15min":0.46700000000000003},"min":{"1min":0.318,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.57399999999999995,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.49099999999999999,"15min":0.51500000000000001},"min":{"1min":0.36199999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.93600000000000005},"last":0.41699999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48999999999999999,"15min":0.498},"min":{"1min":0.36299999999999999,"5min":0.33500000000000002,"15min":0.31900000000000001},"max":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.749},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48799999999999999,"15min":0.50800000000000001},"min":{"1min":0.38200000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.70099999999999996,"15min":0.72399999999999998},"last":0.49099999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.44400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.78200000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.504,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.47999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":1032192,"data_stored":437242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58099999999999996,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70399999999999996,"5min":0.78500000000000003,"15min":0.84599999999999997},"last":0.313},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.754,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.47199999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.48599999999999999},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72599999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53600000000000003,"15min":0.51000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.77400000000000002},"last":0.36299999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59099999999999997,"15min":0.54800000000000004},"min":{"1min":0.42799999999999999,"5min":0.40200000000000002,"15min":0.39400000000000002},"max":{"1min":0.77100000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.499},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59499999999999997,"15min":0.54900000000000004},"min":{"1min":0.48999999999999999,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.76600000000000001,"5min":0.81200000000000006,"15min":0.82199999999999995},"last":0.51500000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.753,"5min":0.80900000000000005,"15min":1.0369999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56100000000000005,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.38500000000000001,"15min":0.374},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.438}]},{"osd":5,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46000000000000002,"15min":0.46800000000000003},"min":{"1min":0.34599999999999997,"5min":0.30199999999999999,"15min":0.26500000000000001},"max":{"1min":0.58099999999999996,"5min":0.73099999999999998,"15min":0.75800000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46500000000000002,"15min":0.51300000000000001},"min":{"1min":0.33000000000000002,"5min":0.308,"15min":0.253},"max":{"1min":0.58099999999999996,"5min":0.76100000000000001,"15min":1.008},"last":0.52900000000000003}]},{"osd":6,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35499999999999998,"5min":0.307,"15min":0.28100000000000003},"max":{"1min":0.60699999999999998,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.497,"15min":0.51700000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.314},"max":{"1min":0.627,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54200000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.502,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.65100000000000002,"5min":0.71799999999999997,"15min":1.0069999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.57399999999999995},"min":{"1min":0.40699999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.68000000000000005,"5min":0.80400000000000005,"15min":0.93400000000000005},"last":0.55300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":1044480,"data_stored":438806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35499999999999998,"15min":0.32100000000000001},"min":{"1min":0.22600000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.56499999999999995,"5min":1.45,"15min":1.45},"last":0.249},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.36099999999999999,"15min":0.32300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.45700000000000002,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.372}]},{"osd":1,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.35299999999999998,"15min":0.318},"min":{"1min":0.24299999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.499,"5min":1.323,"15min":1.323},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38900000000000001,"15min":0.35599999999999998},"min":{"1min":0.28399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.28899999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42799999999999999,"15min":0.38900000000000001},"min":{"1min":0.314,"5min":0.251,"15min":0.251},"max":{"1min":0.57299999999999995,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.44900000000000001,"15min":0.41899999999999998},"min":{"1min":0.32800000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.56499999999999995,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.38800000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71099999999999997,"5min":1.337,"15min":1.337},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.55100000000000005},"min":{"1min":0.44500000000000001,"5min":0.39700000000000002,"15min":0.34699999999999998},"max":{"1min":0.751,"5min":2.758,"15min":2.758},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60699999999999998,"15min":0.59699999999999998},"min":{"1min":0.51000000000000001,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.77000000000000002,"5min":1.276,"15min":1.276},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.751,"5min":1.895,"15min":1.895},"last":0.628}]},{"osd":6,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.62,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.71999999999999997,"5min":5.4539999999999997,"15min":5.4539999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.59099999999999997},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79100000000000004,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.54900000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64500000000000002,"15min":0.61299999999999999},"min":{"1min":0.49299999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76700000000000002,"5min":3.7400000000000002,"15min":3.7400000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66000000000000003,"15min":0.622},"min":{"1min":0.45800000000000002,"5min":0.41399999999999998,"15min":0.40300000000000002},"max":{"1min":0.78600000000000003,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.60099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":1036288,"data_stored":437824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34599999999999997},"min":{"1min":0.219,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.316},{"interface":"front","average":{"1min":0.376,"5min":0.35799999999999998,"15min":0.375},"min":{"1min":0.26400000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.60799999999999998,"5min":0.64400000000000002,"15min":0.872},"last":0.30099999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35499999999999998,"15min":0.36799999999999999},"min":{"1min":0.24199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52100000000000002,"5min":0.61099999999999999,"15min":0.746},"last":0.35499999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.29199999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.57699999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39100000000000001,"15min":0.40300000000000002},"min":{"1min":0.32300000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.72499999999999998},"last":0.379}]},{"osd":4,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61199999999999999,"15min":0.63200000000000001},"min":{"1min":0.5,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77200000000000002,"5min":0.84699999999999998,"15min":1.216},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.501,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81599999999999995,"5min":0.89200000000000002,"15min":0.99299999999999999},"last":0.55100000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.621,"15min":0.63100000000000001},"min":{"1min":0.52600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.79200000000000004,"5min":0.90400000000000003,"15min":0.95699999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.47899999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80600000000000005,"5min":0.93700000000000006,"15min":1.101},"last":0.73399999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.51400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":0.89100000000000001,"15min":0.97299999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.628,"15min":0.64400000000000002},"min":{"1min":0.52600000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.84099999999999997,"5min":0.89900000000000002,"15min":1.0780000000000001},"last":0.52600000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.52900000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":1.155},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64000000000000001,"15min":0.65200000000000002},"min":{"1min":0.51400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":1044480,"data_stored":437436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35499999999999998,"15min":0.33900000000000002},"min":{"1min":0.214,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.52000000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.31},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.371,"15min":0.46000000000000002},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.62,"5min":0.71999999999999997,"15min":2.7069999999999999},"last":0.26500000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36199999999999999,"15min":0.34799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.40100000000000002,"15min":0.379},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.52700000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.28299999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.41299999999999998},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.76900000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.48999999999999999},"min":{"1min":0.25600000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.56200000000000006,"5min":0.73899999999999999,"15min":2.75},"last":0.34399999999999997}]},{"osd":4,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59999999999999998,"15min":0.58799999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.53300000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.625,"15min":0.69099999999999995},"min":{"1min":0.45600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82099999999999995,"5min":1.0900000000000001,"15min":2.778},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63,"15min":0.60999999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88700000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63300000000000001,"15min":0.69899999999999995},"min":{"1min":0.46600000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93100000000000005,"5min":1.026,"15min":2.8210000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.64300000000000002,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79100000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.74399999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64800000000000002,"15min":0.64500000000000002},"min":{"1min":0.46400000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.65000000000000002,"15min":0.74099999999999999},"min":{"1min":0.435,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80500000000000005,"5min":0.91600000000000004,"15min":2.7989999999999999},"last":0.54100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1626112,"data_stored":1027025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34399999999999997,"15min":0.32600000000000001},"min":{"1min":0.23799999999999999,"5min":0.184,"15min":0.184},"max":{"1min":0.49399999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.377,"15min":0.35499999999999998},"min":{"1min":0.23899999999999999,"5min":0.23300000000000001,"15min":0.217},"max":{"1min":0.57999999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.47699999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.27300000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.56999999999999995,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.63},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.371},"min":{"1min":0.32700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53800000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.53300000000000003}]},{"osd":3,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.38400000000000001},"min":{"1min":0.25800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.52600000000000002,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.42799999999999999},"min":{"1min":0.32700000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59599999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.65200000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61499999999999999,"15min":0.58699999999999997},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.625,"15min":0.59399999999999997},"min":{"1min":0.43099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.69999999999999996}]},{"osd":5,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.39500000000000002,"15min":0.36099999999999999},"max":{"1min":0.80900000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.626,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.379},"max":{"1min":0.88,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.622},"min":{"1min":0.49199999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91000000000000003,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.46800000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89100000000000001,"5min":0.91600000000000004,"15min":0.92900000000000005},"last":0.83599999999999997}]},{"osd":7,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64600000000000002,"15min":0.60799999999999998},"min":{"1min":0.498,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.64300000000000002,"15min":0.60499999999999998},"min":{"1min":0.45700000000000002,"5min":0.44700000000000001,"15min":0.41899999999999998},"max":{"1min":0.84299999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:47:14.396 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:47:14.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.531+0000 7f9273a2c640 1 Processor -- start 2024-03-23T09:47:14.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.531+0000 7f9273a2c640 1 -- start start 2024-03-23T09:47:14.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.531+0000 7f9273a2c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f926c0b1680 0x7f926c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.531+0000 7f9273a2c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f926c06bc50 0x7f926c0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.531+0000 7f9273a2c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926c0b12c0 con 0x7f926c098a00 2024-03-23T09:47:14.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.531+0000 7f9273a2c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926c06ae90 con 0x7f926c06bc50 2024-03-23T09:47:14.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.531+0000 7f9273a2c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c06b170 con 0x7f926c0b1680 2024-03-23T09:47:14.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.532+0000 7f9271fc9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f926c06bc50 0x7f926c0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.532+0000 7f9271fc9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f926c06bc50 0x7f926c0ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39516/0 (socket says 172.21.15.47:39516) 2024-03-23T09:47:14.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.532+0000 7f9271fc9640 1 -- 172.21.15.47:0/70510931 learned_addr learned my addr 172.21.15.47:0/70510931 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:14.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.532+0000 7f9270fc7640 1 --2- 172.21.15.47:0/70510931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f926c0b1680 0x7f926c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.532+0000 7f9271fc9640 1 -- 172.21.15.47:0/70510931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f926c0b1680 msgr2=0x7f926c0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:14.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.532+0000 7f9271fc9640 1 --2- 172.21.15.47:0/70510931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f926c0b1680 0x7f926c0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.532+0000 7f9271fc9640 1 -- 172.21.15.47:0/70510931 >> v1:172.21.15.175:6789/0 conn(0x7f926c098a00 legacy=0x7f926c098de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.532+0000 7f9271fc9640 1 -- 172.21.15.47:0/70510931 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c06b450 con 0x7f926c06bc50 2024-03-23T09:47:14.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.532+0000 7f9270fc7640 1 --2- 172.21.15.47:0/70510931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f926c0b1680 0x7f926c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:14.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.533+0000 7f9271fc9640 1 --2- 172.21.15.47:0/70510931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c06bc50 0x7f926c0ceac0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f9264009870 tx=0x7f926402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be85d0240c33a8ac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:14.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.533+0000 7f926bfff640 1 -- 172.21.15.47:0/70510931 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264031640 con 0x7f926c06bc50 2024-03-23T09:47:14.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.533+0000 7f926bfff640 1 -- 172.21.15.47:0/70510931 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92640317a0 con 0x7f926c06bc50 2024-03-23T09:47:14.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.533+0000 7f926bfff640 1 -- 172.21.15.47:0/70510931 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264031b30 con 0x7f926c06bc50 2024-03-23T09:47:14.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.534+0000 7f9273a2c640 1 -- 172.21.15.47:0/70510931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c06bc50 msgr2=0x7f926c0ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.534+0000 7f9273a2c640 1 --2- 172.21.15.47:0/70510931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c06bc50 0x7f926c0ceac0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f9264009870 tx=0x7f926402b8f0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.534+0000 7f9273a2c640 1 -- 172.21.15.47:0/70510931 shutdown_connections 2024-03-23T09:47:14.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.534+0000 7f9273a2c640 1 --2- 172.21.15.47:0/70510931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c06bc50 0x7f926c0ceac0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.534+0000 7f9273a2c640 1 --2- 172.21.15.47:0/70510931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f926c0b1680 0x7f926c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.534+0000 7f9273a2c640 1 -- 172.21.15.47:0/70510931 >> 172.21.15.47:0/70510931 conn(0x7f926c0b2140 msgr2=0x7f926c0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:14.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.534+0000 7f9273a2c640 1 -- 172.21.15.47:0/70510931 shutdown_connections 2024-03-23T09:47:14.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.534+0000 7f9273a2c640 1 -- 172.21.15.47:0/70510931 wait complete. 2024-03-23T09:47:14.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.535+0000 7f9273a2c640 1 Processor -- start 2024-03-23T09:47:14.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.535+0000 7f9273a2c640 1 -- start start 2024-03-23T09:47:14.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9273a2c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c06bc50 0x7f926c154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9273a2c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f926c098a00 0x7f926c0b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9273a2c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f926c0b1680 0x7f926c0c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9273a2c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0c5f40 con 0x7f926c06bc50 2024-03-23T09:47:14.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9273a2c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0c53c0 con 0x7f926c0b1680 2024-03-23T09:47:14.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9273a2c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0c56a0 con 0x7f926c098a00 2024-03-23T09:47:14.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f92717c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c06bc50 0x7f926c154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f92717c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c06bc50 0x7f926c154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39520/0 (socket says 172.21.15.47:39520) 2024-03-23T09:47:14.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f92717c8640 1 -- 172.21.15.47:0/2970963843 learned_addr learned my addr 172.21.15.47:0/2970963843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:14.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9271fc9640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f926c0b1680 0x7f926c0c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9270fc7640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f926c098a00 0x7f926c0b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f92717c8640 1 -- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f926c098a00 msgr2=0x7f926c0b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f92717c8640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f926c098a00 0x7f926c0b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f92717c8640 1 -- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f926c0b1680 msgr2=0x7f926c0c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f92717c8640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f926c0b1680 0x7f926c0c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f92717c8640 1 -- 172.21.15.47:0/2970963843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92640094d0 con 0x7f926c06bc50 2024-03-23T09:47:14.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9271fc9640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f926c0b1680 0x7f926c0c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:14.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9270fc7640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f926c098a00 0x7f926c0b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:14.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f92717c8640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c06bc50 0x7f926c154e50 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f925c00c870 tx=0x7f925c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:14.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9269ffb640 1 -- 172.21.15.47:0/2970963843 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c003b70 con 0x7f926c06bc50 2024-03-23T09:47:14.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9269ffb640 1 -- 172.21.15.47:0/2970963843 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925c003cf0 con 0x7f926c06bc50 2024-03-23T09:47:14.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9273a2c640 1 -- 172.21.15.47:0/2970963843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c0c5980 con 0x7f926c06bc50 2024-03-23T09:47:14.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.536+0000 7f9269ffb640 1 -- 172.21.15.47:0/2970963843 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c00aa90 con 0x7f926c06bc50 2024-03-23T09:47:14.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.537+0000 7f9273a2c640 1 -- 172.21.15.47:0/2970963843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c0c50a0 con 0x7f926c06bc50 2024-03-23T09:47:14.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.538+0000 7f9273a2c640 1 -- 172.21.15.47:0/2970963843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c0b1d20 con 0x7f926c06bc50 2024-03-23T09:47:14.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.538+0000 7f9269ffb640 1 -- 172.21.15.47:0/2970963843 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f925c00abf0 con 0x7f926c06bc50 2024-03-23T09:47:14.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.538+0000 7f9269ffb640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f925806ac00 0x7f925806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.538+0000 7f9269ffb640 1 -- 172.21.15.47:0/2970963843 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f925c0891a0 con 0x7f926c06bc50 2024-03-23T09:47:14.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.538+0000 7f9270fc7640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f925806ac00 0x7f925806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.539+0000 7f9270fc7640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f925806ac00 0x7f925806d0c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f926c0b4a50 tx=0x7f9260008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:14.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.542+0000 7f9269ffb640 1 -- 172.21.15.47:0/2970963843 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f925c054410 con 0x7f926c06bc50 2024-03-23T09:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.723+0000 7f9273a2c640 1 -- 172.21.15.47:0/2970963843 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f926c0afcc0 con 0x7f925806ac00 2024-03-23T09:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.730+0000 7f9269ffb640 1 -- 172.21.15.47:0/2970963843 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289491 (secure 0 0 0) 0x7f9260009600 con 0x7f925806ac00 2024-03-23T09:47:14.734 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.733+0000 7f9273a2c640 1 -- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f925806ac00 msgr2=0x7f925806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.733+0000 7f9273a2c640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f925806ac00 0x7f925806d0c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f926c0b4a50 tx=0x7f9260008040 comp rx=0 tx=0).stop 2024-03-23T09:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.733+0000 7f9273a2c640 1 -- 172.21.15.47:0/2970963843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c06bc50 msgr2=0x7f926c154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.733+0000 7f9273a2c640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c06bc50 0x7f926c154e50 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f925c00c870 tx=0x7f925c00cd30 comp rx=0 tx=0).stop 2024-03-23T09:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.733+0000 7f9273a2c640 1 -- 172.21.15.47:0/2970963843 shutdown_connections 2024-03-23T09:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.733+0000 7f9273a2c640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f925806ac00 0x7f925806d0c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.734+0000 7f9273a2c640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f926c0b1680 0x7f926c0c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.734+0000 7f9273a2c640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f926c098a00 0x7f926c0b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.734+0000 7f9273a2c640 1 --2- 172.21.15.47:0/2970963843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c06bc50 0x7f926c154e50 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.734+0000 7f9273a2c640 1 -- 172.21.15.47:0/2970963843 >> 172.21.15.47:0/2970963843 conn(0x7f926c0b2140 msgr2=0x7f926c0cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.734+0000 7f9273a2c640 1 -- 172.21.15.47:0/2970963843 shutdown_connections 2024-03-23T09:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.734+0000 7f9273a2c640 1 -- 172.21.15.47:0/2970963843 wait complete. 2024-03-23T09:47:14.744 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-03-23T09:47:14.430202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250056,"kb_used_data":9280,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754724664,"statfs":{"total":773094113280,"available":772838055936,"internally_reserved":0,"allocated":9502720,"data_stored":4682158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":246402616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.412862"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424221+0000","last_change":"2024-03-23T09:47:11.420454+0000","last_active":"2024-03-23T09:47:11.424221+0000","last_peered":"2024-03-23T09:47:11.424221+0000","last_clean":"2024-03-23T09:47:11.424221+0000","last_became_active":"2024-03-23T09:47:11.418812+0000","last_became_peered":"2024-03-23T09:47:11.418812+0000","last_unstale":"2024-03-23T09:47:11.424221+0000","last_undegraded":"2024-03-23T09:47:11.424221+0000","last_fullsized":"2024-03-23T09:47:11.424221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:26:29.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421936+0000","last_change":"2024-03-23T09:47:11.420152+0000","last_active":"2024-03-23T09:47:11.421936+0000","last_peered":"2024-03-23T09:47:11.421936+0000","last_clean":"2024-03-23T09:47:11.421936+0000","last_became_active":"2024-03-23T09:47:11.419635+0000","last_became_peered":"2024-03-23T09:47:11.419635+0000","last_unstale":"2024-03-23T09:47:11.421936+0000","last_undegraded":"2024-03-23T09:47:11.421936+0000","last_fullsized":"2024-03-23T09:47:11.421936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:15:28.768150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.413487+0000","last_change":"2024-03-23T09:47:11.413059+0000","last_active":"2024-03-23T09:47:11.413487+0000","last_peered":"2024-03-23T09:47:11.413487+0000","last_clean":"2024-03-23T09:47:11.413487+0000","last_became_active":"2024-03-23T09:47:11.412801+0000","last_became_peered":"2024-03-23T09:47:11.412801+0000","last_unstale":"2024-03-23T09:47:11.413487+0000","last_undegraded":"2024-03-23T09:47:11.413487+0000","last_fullsized":"2024-03-23T09:47:11.413487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:21:53.365087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.402806+0000","last_change":"2024-03-23T09:47:11.402058+0000","last_active":"2024-03-23T09:47:11.402806+0000","last_peered":"2024-03-23T09:47:11.402806+0000","last_clean":"2024-03-23T09:47:11.402806+0000","last_became_active":"2024-03-23T09:47:11.401797+0000","last_became_peered":"2024-03-23T09:47:11.401797+0000","last_unstale":"2024-03-23T09:47:11.402806+0000","last_undegraded":"2024-03-23T09:47:11.402806+0000","last_fullsized":"2024-03-23T09:47:11.402806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:52:51.537440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416097+0000","last_change":"2024-03-23T09:47:11.405570+0000","last_active":"2024-03-23T09:47:11.416097+0000","last_peered":"2024-03-23T09:47:11.416097+0000","last_clean":"2024-03-23T09:47:11.416097+0000","last_became_active":"2024-03-23T09:47:11.405323+0000","last_became_peered":"2024-03-23T09:47:11.405323+0000","last_unstale":"2024-03-23T09:47:11.416097+0000","last_undegraded":"2024-03-23T09:47:11.416097+0000","last_fullsized":"2024-03-23T09:47:11.416097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:59:56.765267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.417663+0000","last_change":"2024-03-23T09:47:11.413134+0000","last_active":"2024-03-23T09:47:11.417663+0000","last_peered":"2024-03-23T09:47:11.417663+0000","last_clean":"2024-03-23T09:47:11.417663+0000","last_became_active":"2024-03-23T09:47:11.412923+0000","last_became_peered":"2024-03-23T09:47:11.412923+0000","last_unstale":"2024-03-23T09:47:11.417663+0000","last_undegraded":"2024-03-23T09:47:11.417663+0000","last_fullsized":"2024-03-23T09:47:11.417663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:07:33.336490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406596+0000","last_change":"2024-03-23T09:47:11.404392+0000","last_active":"2024-03-23T09:47:11.406596+0000","last_peered":"2024-03-23T09:47:11.406596+0000","last_clean":"2024-03-23T09:47:11.406596+0000","last_became_active":"2024-03-23T09:47:11.404177+0000","last_became_peered":"2024-03-23T09:47:11.404177+0000","last_unstale":"2024-03-23T09:47:11.406596+0000","last_undegraded":"2024-03-23T09:47:11.406596+0000","last_fullsized":"2024-03-23T09:47:11.406596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:47:30.427023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.418247+0000","last_change":"2024-03-23T09:47:11.416697+0000","last_active":"2024-03-23T09:47:11.418247+0000","last_peered":"2024-03-23T09:47:11.418247+0000","last_clean":"2024-03-23T09:47:11.418247+0000","last_became_active":"2024-03-23T09:47:11.416430+0000","last_became_peered":"2024-03-23T09:47:11.416430+0000","last_unstale":"2024-03-23T09:47:11.418247+0000","last_undegraded":"2024-03-23T09:47:11.418247+0000","last_fullsized":"2024-03-23T09:47:11.418247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:10:02.020675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.423529+0000","last_change":"2024-03-23T09:47:11.418822+0000","last_active":"2024-03-23T09:47:11.423529+0000","last_peered":"2024-03-23T09:47:11.423529+0000","last_clean":"2024-03-23T09:47:11.423529+0000","last_became_active":"2024-03-23T09:47:11.418544+0000","last_became_peered":"2024-03-23T09:47:11.418544+0000","last_unstale":"2024-03-23T09:47:11.423529+0000","last_undegraded":"2024-03-23T09:47:11.423529+0000","last_fullsized":"2024-03-23T09:47:11.423529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:15.479312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424183+0000","last_change":"2024-03-23T09:47:11.420068+0000","last_active":"2024-03-23T09:47:11.424183+0000","last_peered":"2024-03-23T09:47:11.424183+0000","last_clean":"2024-03-23T09:47:11.424183+0000","last_became_active":"2024-03-23T09:47:11.418780+0000","last_became_peered":"2024-03-23T09:47:11.418780+0000","last_unstale":"2024-03-23T09:47:11.424183+0000","last_undegraded":"2024-03-23T09:47:11.424183+0000","last_fullsized":"2024-03-23T09:47:11.424183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:03:19.975566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406084+0000","last_change":"2024-03-23T09:47:11.404487+0000","last_active":"2024-03-23T09:47:11.406084+0000","last_peered":"2024-03-23T09:47:11.406084+0000","last_clean":"2024-03-23T09:47:11.406084+0000","last_became_active":"2024-03-23T09:47:11.404234+0000","last_became_peered":"2024-03-23T09:47:11.404234+0000","last_unstale":"2024-03-23T09:47:11.406084+0000","last_undegraded":"2024-03-23T09:47:11.406084+0000","last_fullsized":"2024-03-23T09:47:11.406084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:10:02.020675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421269+0000","last_change":"2024-03-23T09:47:11.413423+0000","last_active":"2024-03-23T09:47:11.421269+0000","last_peered":"2024-03-23T09:47:11.421269+0000","last_clean":"2024-03-23T09:47:11.421269+0000","last_became_active":"2024-03-23T09:47:11.412787+0000","last_became_peered":"2024-03-23T09:47:11.412787+0000","last_unstale":"2024-03-23T09:47:11.421269+0000","last_undegraded":"2024-03-23T09:47:11.421269+0000","last_fullsized":"2024-03-23T09:47:11.421269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:53:52.661871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412487+0000","last_change":"2024-03-23T09:47:11.407990+0000","last_active":"2024-03-23T09:47:11.412487+0000","last_peered":"2024-03-23T09:47:11.412487+0000","last_clean":"2024-03-23T09:47:11.412487+0000","last_became_active":"2024-03-23T09:47:11.407356+0000","last_became_peered":"2024-03-23T09:47:11.407356+0000","last_unstale":"2024-03-23T09:47:11.412487+0000","last_undegraded":"2024-03-23T09:47:11.412487+0000","last_fullsized":"2024-03-23T09:47:11.412487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:08:58.274779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406651+0000","last_change":"2024-03-23T09:47:11.403189+0000","last_active":"2024-03-23T09:47:11.406651+0000","last_peered":"2024-03-23T09:47:11.406651+0000","last_clean":"2024-03-23T09:47:11.406651+0000","last_became_active":"2024-03-23T09:47:11.402767+0000","last_became_peered":"2024-03-23T09:47:11.402767+0000","last_unstale":"2024-03-23T09:47:11.406651+0000","last_undegraded":"2024-03-23T09:47:11.406651+0000","last_fullsized":"2024-03-23T09:47:11.406651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:22.023655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412561+0000","last_change":"2024-03-23T09:47:11.411012+0000","last_active":"2024-03-23T09:47:11.412561+0000","last_peered":"2024-03-23T09:47:11.412561+0000","last_clean":"2024-03-23T09:47:11.412561+0000","last_became_active":"2024-03-23T09:47:11.410764+0000","last_became_peered":"2024-03-23T09:47:11.410764+0000","last_unstale":"2024-03-23T09:47:11.412561+0000","last_undegraded":"2024-03-23T09:47:11.412561+0000","last_fullsized":"2024-03-23T09:47:11.412561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:48:02.922170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.423997+0000","last_change":"2024-03-23T09:47:11.414350+0000","last_active":"2024-03-23T09:47:11.423997+0000","last_peered":"2024-03-23T09:47:11.423997+0000","last_clean":"2024-03-23T09:47:11.423997+0000","last_became_active":"2024-03-23T09:47:11.413980+0000","last_became_peered":"2024-03-23T09:47:11.413980+0000","last_unstale":"2024-03-23T09:47:11.423997+0000","last_undegraded":"2024-03-23T09:47:11.423997+0000","last_fullsized":"2024-03-23T09:47:11.423997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:53:52.661871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.414898+0000","last_change":"2024-03-23T09:47:11.404275+0000","last_active":"2024-03-23T09:47:11.414898+0000","last_peered":"2024-03-23T09:47:11.414898+0000","last_clean":"2024-03-23T09:47:11.414898+0000","last_became_active":"2024-03-23T09:47:11.403931+0000","last_became_peered":"2024-03-23T09:47:11.403931+0000","last_unstale":"2024-03-23T09:47:11.414898+0000","last_undegraded":"2024-03-23T09:47:11.414898+0000","last_fullsized":"2024-03-23T09:47:11.414898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:03:19.975566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.418367+0000","last_change":"2024-03-23T09:47:11.413532+0000","last_active":"2024-03-23T09:47:11.418367+0000","last_peered":"2024-03-23T09:47:11.418367+0000","last_clean":"2024-03-23T09:47:11.418367+0000","last_became_active":"2024-03-23T09:47:11.413352+0000","last_became_peered":"2024-03-23T09:47:11.413352+0000","last_unstale":"2024-03-23T09:47:11.418367+0000","last_undegraded":"2024-03-23T09:47:11.418367+0000","last_fullsized":"2024-03-23T09:47:11.418367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:14:34.499019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.411934+0000","last_change":"2024-03-23T09:47:11.410786+0000","last_active":"2024-03-23T09:47:11.411934+0000","last_peered":"2024-03-23T09:47:11.411934+0000","last_clean":"2024-03-23T09:47:11.411934+0000","last_became_active":"2024-03-23T09:47:11.410560+0000","last_became_peered":"2024-03-23T09:47:11.410560+0000","last_unstale":"2024-03-23T09:47:11.411934+0000","last_undegraded":"2024-03-23T09:47:11.411934+0000","last_fullsized":"2024-03-23T09:47:11.411934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:08:58.274779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412540+0000","last_change":"2024-03-23T09:47:11.408366+0000","last_active":"2024-03-23T09:47:11.412540+0000","last_peered":"2024-03-23T09:47:11.412540+0000","last_clean":"2024-03-23T09:47:11.412540+0000","last_became_active":"2024-03-23T09:47:11.407952+0000","last_became_peered":"2024-03-23T09:47:11.407952+0000","last_unstale":"2024-03-23T09:47:11.412540+0000","last_undegraded":"2024-03-23T09:47:11.412540+0000","last_fullsized":"2024-03-23T09:47:11.412540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:41:39.284406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409033+0000","last_change":"2024-03-23T09:47:11.408177+0000","last_active":"2024-03-23T09:47:11.409033+0000","last_peered":"2024-03-23T09:47:11.409033+0000","last_clean":"2024-03-23T09:47:11.409033+0000","last_became_active":"2024-03-23T09:47:11.407922+0000","last_became_peered":"2024-03-23T09:47:11.407922+0000","last_unstale":"2024-03-23T09:47:11.409033+0000","last_undegraded":"2024-03-23T09:47:11.409033+0000","last_fullsized":"2024-03-23T09:47:11.409033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:20:54.064158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409754+0000","last_change":"2024-03-23T09:47:11.408673+0000","last_active":"2024-03-23T09:47:11.409754+0000","last_peered":"2024-03-23T09:47:11.409754+0000","last_clean":"2024-03-23T09:47:11.409754+0000","last_became_active":"2024-03-23T09:47:11.408014+0000","last_became_peered":"2024-03-23T09:47:11.408014+0000","last_unstale":"2024-03-23T09:47:11.409754+0000","last_undegraded":"2024-03-23T09:47:11.409754+0000","last_fullsized":"2024-03-23T09:47:11.409754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:07:37.588894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421310+0000","last_change":"2024-03-23T09:47:11.413587+0000","last_active":"2024-03-23T09:47:11.421310+0000","last_peered":"2024-03-23T09:47:11.421310+0000","last_clean":"2024-03-23T09:47:11.421310+0000","last_became_active":"2024-03-23T09:47:11.413292+0000","last_became_peered":"2024-03-23T09:47:11.413292+0000","last_unstale":"2024-03-23T09:47:11.421310+0000","last_undegraded":"2024-03-23T09:47:11.421310+0000","last_fullsized":"2024-03-23T09:47:11.421310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:26:43.651127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412186+0000","last_change":"2024-03-23T09:47:11.409156+0000","last_active":"2024-03-23T09:47:11.412186+0000","last_peered":"2024-03-23T09:47:11.412186+0000","last_clean":"2024-03-23T09:47:11.412186+0000","last_became_active":"2024-03-23T09:47:11.408797+0000","last_became_peered":"2024-03-23T09:47:11.408797+0000","last_unstale":"2024-03-23T09:47:11.412186+0000","last_undegraded":"2024-03-23T09:47:11.412186+0000","last_fullsized":"2024-03-23T09:47:11.412186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:42:20.730126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.419832+0000","last_change":"2024-03-23T09:47:11.416759+0000","last_active":"2024-03-23T09:47:11.419832+0000","last_peered":"2024-03-23T09:47:11.419832+0000","last_clean":"2024-03-23T09:47:11.419832+0000","last_became_active":"2024-03-23T09:47:11.416484+0000","last_became_peered":"2024-03-23T09:47:11.416484+0000","last_unstale":"2024-03-23T09:47:11.419832+0000","last_undegraded":"2024-03-23T09:47:11.419832+0000","last_fullsized":"2024-03-23T09:47:11.419832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:15.479312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412487+0000","last_change":"2024-03-23T09:47:11.408214+0000","last_active":"2024-03-23T09:47:11.412487+0000","last_peered":"2024-03-23T09:47:11.412487+0000","last_clean":"2024-03-23T09:47:11.412487+0000","last_became_active":"2024-03-23T09:47:11.407947+0000","last_became_peered":"2024-03-23T09:47:11.407947+0000","last_unstale":"2024-03-23T09:47:11.412487+0000","last_undegraded":"2024-03-23T09:47:11.412487+0000","last_fullsized":"2024-03-23T09:47:11.412487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:59:33.183165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416189+0000","last_change":"2024-03-23T09:47:11.404766+0000","last_active":"2024-03-23T09:47:11.416189+0000","last_peered":"2024-03-23T09:47:11.416189+0000","last_clean":"2024-03-23T09:47:11.416189+0000","last_became_active":"2024-03-23T09:47:11.404503+0000","last_became_peered":"2024-03-23T09:47:11.404503+0000","last_unstale":"2024-03-23T09:47:11.416189+0000","last_undegraded":"2024-03-23T09:47:11.416189+0000","last_fullsized":"2024-03-23T09:47:11.416189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:08:58.274779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424244+0000","last_change":"2024-03-23T09:47:11.419656+0000","last_active":"2024-03-23T09:47:11.424244+0000","last_peered":"2024-03-23T09:47:11.424244+0000","last_clean":"2024-03-23T09:47:11.424244+0000","last_became_active":"2024-03-23T09:47:11.419067+0000","last_became_peered":"2024-03-23T09:47:11.419067+0000","last_unstale":"2024-03-23T09:47:11.424244+0000","last_undegraded":"2024-03-23T09:47:11.424244+0000","last_fullsized":"2024-03-23T09:47:11.424244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:45:12.520727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409089+0000","last_change":"2024-03-23T09:47:11.405310+0000","last_active":"2024-03-23T09:47:11.409089+0000","last_peered":"2024-03-23T09:47:11.409089+0000","last_clean":"2024-03-23T09:47:11.409089+0000","last_became_active":"2024-03-23T09:47:11.404830+0000","last_became_peered":"2024-03-23T09:47:11.404830+0000","last_unstale":"2024-03-23T09:47:11.409089+0000","last_undegraded":"2024-03-23T09:47:11.409089+0000","last_fullsized":"2024-03-23T09:47:11.409089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:35:54.438265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.410472+0000","last_change":"2024-03-23T09:47:11.408966+0000","last_active":"2024-03-23T09:47:11.410472+0000","last_peered":"2024-03-23T09:47:11.410472+0000","last_clean":"2024-03-23T09:47:11.410472+0000","last_became_active":"2024-03-23T09:47:11.408716+0000","last_became_peered":"2024-03-23T09:47:11.408716+0000","last_unstale":"2024-03-23T09:47:11.410472+0000","last_undegraded":"2024-03-23T09:47:11.410472+0000","last_fullsized":"2024-03-23T09:47:11.410472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:22:33.792384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.423976+0000","last_change":"2024-03-23T09:47:11.415729+0000","last_active":"2024-03-23T09:47:11.423976+0000","last_peered":"2024-03-23T09:47:11.423976+0000","last_clean":"2024-03-23T09:47:11.423976+0000","last_became_active":"2024-03-23T09:47:11.415477+0000","last_became_peered":"2024-03-23T09:47:11.415477+0000","last_unstale":"2024-03-23T09:47:11.423976+0000","last_undegraded":"2024-03-23T09:47:11.423976+0000","last_fullsized":"2024-03-23T09:47:11.423976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:30.983075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421529+0000","last_change":"2024-03-23T09:47:11.418138+0000","last_active":"2024-03-23T09:47:11.421529+0000","last_peered":"2024-03-23T09:47:11.421529+0000","last_clean":"2024-03-23T09:47:11.421529+0000","last_became_active":"2024-03-23T09:47:11.417863+0000","last_became_peered":"2024-03-23T09:47:11.417863+0000","last_unstale":"2024-03-23T09:47:11.421529+0000","last_undegraded":"2024-03-23T09:47:11.421529+0000","last_fullsized":"2024-03-23T09:47:11.421529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:21:53.365087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.418448+0000","last_change":"2024-03-23T09:47:11.413398+0000","last_active":"2024-03-23T09:47:11.418448+0000","last_peered":"2024-03-23T09:47:11.418448+0000","last_clean":"2024-03-23T09:47:11.418448+0000","last_became_active":"2024-03-23T09:47:11.413121+0000","last_became_peered":"2024-03-23T09:47:11.413121+0000","last_unstale":"2024-03-23T09:47:11.418448+0000","last_undegraded":"2024-03-23T09:47:11.418448+0000","last_fullsized":"2024-03-23T09:47:11.418448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:26:43.651127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"84'4","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421264+0000","last_change":"2024-03-23T09:47:11.034751+0000","last_active":"2024-03-23T09:47:11.421264+0000","last_peered":"2024-03-23T09:47:11.421264+0000","last_clean":"2024-03-23T09:47:11.421264+0000","last_became_active":"2024-03-23T09:47:11.034458+0000","last_became_peered":"2024-03-23T09:47:11.034458+0000","last_unstale":"2024-03-23T09:47:11.421264+0000","last_undegraded":"2024-03-23T09:47:11.421264+0000","last_fullsized":"2024-03-23T09:47:11.421264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:27:13.329590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421488+0000","last_change":"2024-03-23T09:47:11.417835+0000","last_active":"2024-03-23T09:47:11.421488+0000","last_peered":"2024-03-23T09:47:11.421488+0000","last_clean":"2024-03-23T09:47:11.421488+0000","last_became_active":"2024-03-23T09:47:11.417267+0000","last_became_peered":"2024-03-23T09:47:11.417267+0000","last_unstale":"2024-03-23T09:47:11.421488+0000","last_undegraded":"2024-03-23T09:47:11.421488+0000","last_fullsized":"2024-03-23T09:47:11.421488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:30.983075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416547+0000","last_change":"2024-03-23T09:47:11.033030+0000","last_active":"2024-03-23T09:47:11.416547+0000","last_peered":"2024-03-23T09:47:11.416547+0000","last_clean":"2024-03-23T09:47:11.416547+0000","last_became_active":"2024-03-23T09:47:11.032579+0000","last_became_peered":"2024-03-23T09:47:11.032579+0000","last_unstale":"2024-03-23T09:47:11.416547+0000","last_undegraded":"2024-03-23T09:47:11.416547+0000","last_fullsized":"2024-03-23T09:47:11.416547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:07:36.583568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421288+0000","last_change":"2024-03-23T09:47:11.413661+0000","last_active":"2024-03-23T09:47:11.421288+0000","last_peered":"2024-03-23T09:47:11.421288+0000","last_clean":"2024-03-23T09:47:11.421288+0000","last_became_active":"2024-03-23T09:47:11.413120+0000","last_became_peered":"2024-03-23T09:47:11.413120+0000","last_unstale":"2024-03-23T09:47:11.421288+0000","last_undegraded":"2024-03-23T09:47:11.421288+0000","last_fullsized":"2024-03-23T09:47:11.421288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:59:33.183165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.420004+0000","last_change":"2024-03-23T09:47:11.039090+0000","last_active":"2024-03-23T09:47:11.420004+0000","last_peered":"2024-03-23T09:47:11.420004+0000","last_clean":"2024-03-23T09:47:11.420004+0000","last_became_active":"2024-03-23T09:47:11.038823+0000","last_became_peered":"2024-03-23T09:47:11.038823+0000","last_unstale":"2024-03-23T09:47:11.420004+0000","last_undegraded":"2024-03-23T09:47:11.420004+0000","last_fullsized":"2024-03-23T09:47:11.420004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:26:28.173693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412491+0000","last_change":"2024-03-23T09:47:11.409787+0000","last_active":"2024-03-23T09:47:11.412491+0000","last_peered":"2024-03-23T09:47:11.412491+0000","last_clean":"2024-03-23T09:47:11.412491+0000","last_became_active":"2024-03-23T09:47:11.409509+0000","last_became_peered":"2024-03-23T09:47:11.409509+0000","last_unstale":"2024-03-23T09:47:11.412491+0000","last_undegraded":"2024-03-23T09:47:11.412491+0000","last_fullsized":"2024-03-23T09:47:11.412491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:03:19.975566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.420150+0000","last_change":"2024-03-23T09:47:11.038814+0000","last_active":"2024-03-23T09:47:11.420150+0000","last_peered":"2024-03-23T09:47:11.420150+0000","last_clean":"2024-03-23T09:47:11.420150+0000","last_became_active":"2024-03-23T09:47:11.038374+0000","last_became_peered":"2024-03-23T09:47:11.038374+0000","last_unstale":"2024-03-23T09:47:11.420150+0000","last_undegraded":"2024-03-23T09:47:11.420150+0000","last_fullsized":"2024-03-23T09:47:11.420150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:06:10.478448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409770+0000","last_change":"2024-03-23T09:47:11.408837+0000","last_active":"2024-03-23T09:47:11.409770+0000","last_peered":"2024-03-23T09:47:11.409770+0000","last_clean":"2024-03-23T09:47:11.409770+0000","last_became_active":"2024-03-23T09:47:11.408412+0000","last_became_peered":"2024-03-23T09:47:11.408412+0000","last_unstale":"2024-03-23T09:47:11.409770+0000","last_undegraded":"2024-03-23T09:47:11.409770+0000","last_fullsized":"2024-03-23T09:47:11.409770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:07:33.336490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.417279+0000","last_change":"2024-03-23T09:47:11.038235+0000","last_active":"2024-03-23T09:47:11.417279+0000","last_peered":"2024-03-23T09:47:11.417279+0000","last_clean":"2024-03-23T09:47:11.417279+0000","last_became_active":"2024-03-23T09:47:11.037820+0000","last_became_peered":"2024-03-23T09:47:11.037820+0000","last_unstale":"2024-03-23T09:47:11.417279+0000","last_undegraded":"2024-03-23T09:47:11.417279+0000","last_fullsized":"2024-03-23T09:47:11.417279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:17.326242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416077+0000","last_change":"2024-03-23T09:47:11.405285+0000","last_active":"2024-03-23T09:47:11.416077+0000","last_peered":"2024-03-23T09:47:11.416077+0000","last_clean":"2024-03-23T09:47:11.416077+0000","last_became_active":"2024-03-23T09:47:11.404560+0000","last_became_peered":"2024-03-23T09:47:11.404560+0000","last_unstale":"2024-03-23T09:47:11.416077+0000","last_undegraded":"2024-03-23T09:47:11.416077+0000","last_fullsized":"2024-03-23T09:47:11.416077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:45:12.520727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416906+0000","last_change":"2024-03-23T09:47:11.035230+0000","last_active":"2024-03-23T09:47:11.416906+0000","last_peered":"2024-03-23T09:47:11.416906+0000","last_clean":"2024-03-23T09:47:11.416906+0000","last_became_active":"2024-03-23T09:47:11.034777+0000","last_became_peered":"2024-03-23T09:47:11.034777+0000","last_unstale":"2024-03-23T09:47:11.416906+0000","last_undegraded":"2024-03-23T09:47:11.416906+0000","last_fullsized":"2024-03-23T09:47:11.416906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:26:08.320253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412317+0000","last_change":"2024-03-23T09:47:11.409098+0000","last_active":"2024-03-23T09:47:11.412317+0000","last_peered":"2024-03-23T09:47:11.412317+0000","last_clean":"2024-03-23T09:47:11.412317+0000","last_became_active":"2024-03-23T09:47:11.408784+0000","last_became_peered":"2024-03-23T09:47:11.408784+0000","last_unstale":"2024-03-23T09:47:11.412317+0000","last_undegraded":"2024-03-23T09:47:11.412317+0000","last_fullsized":"2024-03-23T09:47:11.412317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:16:21.633326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406081+0000","last_change":"2024-03-23T09:47:11.040129+0000","last_active":"2024-03-23T09:47:11.406081+0000","last_peered":"2024-03-23T09:47:11.406081+0000","last_clean":"2024-03-23T09:47:11.406081+0000","last_became_active":"2024-03-23T09:47:11.039770+0000","last_became_peered":"2024-03-23T09:47:11.039770+0000","last_unstale":"2024-03-23T09:47:11.406081+0000","last_undegraded":"2024-03-23T09:47:11.406081+0000","last_fullsized":"2024-03-23T09:47:11.406081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:45:35.363288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409731+0000","last_change":"2024-03-23T09:47:11.409151+0000","last_active":"2024-03-23T09:47:11.409731+0000","last_peered":"2024-03-23T09:47:11.409731+0000","last_clean":"2024-03-23T09:47:11.409731+0000","last_became_active":"2024-03-23T09:47:11.408897+0000","last_became_peered":"2024-03-23T09:47:11.408897+0000","last_unstale":"2024-03-23T09:47:11.409731+0000","last_undegraded":"2024-03-23T09:47:11.409731+0000","last_fullsized":"2024-03-23T09:47:11.409731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:08:20.711770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.396433+0000","last_change":"2024-03-23T09:47:11.035672+0000","last_active":"2024-03-23T09:47:11.396433+0000","last_peered":"2024-03-23T09:47:11.396433+0000","last_clean":"2024-03-23T09:47:11.396433+0000","last_became_active":"2024-03-23T09:47:11.035424+0000","last_became_peered":"2024-03-23T09:47:11.035424+0000","last_unstale":"2024-03-23T09:47:11.396433+0000","last_undegraded":"2024-03-23T09:47:11.396433+0000","last_fullsized":"2024-03-23T09:47:11.396433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:52:50.532114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.407440+0000","last_change":"2024-03-23T09:47:11.404596+0000","last_active":"2024-03-23T09:47:11.407440+0000","last_peered":"2024-03-23T09:47:11.407440+0000","last_clean":"2024-03-23T09:47:11.407440+0000","last_became_active":"2024-03-23T09:47:11.404362+0000","last_became_peered":"2024-03-23T09:47:11.404362+0000","last_unstale":"2024-03-23T09:47:11.407440+0000","last_undegraded":"2024-03-23T09:47:11.407440+0000","last_fullsized":"2024-03-23T09:47:11.407440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:28:40.156425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.408857+0000","last_change":"2024-03-23T09:47:11.037991+0000","last_active":"2024-03-23T09:47:11.408857+0000","last_peered":"2024-03-23T09:47:11.408857+0000","last_clean":"2024-03-23T09:47:11.408857+0000","last_became_active":"2024-03-23T09:47:11.037694+0000","last_became_peered":"2024-03-23T09:47:11.037694+0000","last_unstale":"2024-03-23T09:47:11.408857+0000","last_undegraded":"2024-03-23T09:47:11.408857+0000","last_fullsized":"2024-03-23T09:47:11.408857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:52:08.866579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.417692+0000","last_change":"2024-03-23T09:47:11.413118+0000","last_active":"2024-03-23T09:47:11.417692+0000","last_peered":"2024-03-23T09:47:11.417692+0000","last_clean":"2024-03-23T09:47:11.417692+0000","last_became_active":"2024-03-23T09:47:11.412874+0000","last_became_peered":"2024-03-23T09:47:11.412874+0000","last_unstale":"2024-03-23T09:47:11.417692+0000","last_undegraded":"2024-03-23T09:47:11.417692+0000","last_fullsized":"2024-03-23T09:47:11.417692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:22:33.792384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409552+0000","last_change":"2024-03-23T09:47:11.036496+0000","last_active":"2024-03-23T09:47:11.409552+0000","last_peered":"2024-03-23T09:47:11.409552+0000","last_clean":"2024-03-23T09:47:11.409552+0000","last_became_active":"2024-03-23T09:47:11.036242+0000","last_became_peered":"2024-03-23T09:47:11.036242+0000","last_unstale":"2024-03-23T09:47:11.409552+0000","last_undegraded":"2024-03-23T09:47:11.409552+0000","last_fullsized":"2024-03-23T09:47:11.409552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:28:41.141201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.418667+0000","last_change":"2024-03-23T09:47:11.416306+0000","last_active":"2024-03-23T09:47:11.418667+0000","last_peered":"2024-03-23T09:47:11.418667+0000","last_clean":"2024-03-23T09:47:11.418667+0000","last_became_active":"2024-03-23T09:47:11.416072+0000","last_became_peered":"2024-03-23T09:47:11.416072+0000","last_unstale":"2024-03-23T09:47:11.418667+0000","last_undegraded":"2024-03-23T09:47:11.418667+0000","last_fullsized":"2024-03-23T09:47:11.418667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:33:16.231177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"84'3","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.420081+0000","last_change":"2024-03-23T09:47:11.038881+0000","last_active":"2024-03-23T09:47:11.420081+0000","last_peered":"2024-03-23T09:47:11.420081+0000","last_clean":"2024-03-23T09:47:11.420081+0000","last_became_active":"2024-03-23T09:47:11.038582+0000","last_became_peered":"2024-03-23T09:47:11.038582+0000","last_unstale":"2024-03-23T09:47:11.420081+0000","last_undegraded":"2024-03-23T09:47:11.420081+0000","last_fullsized":"2024-03-23T09:47:11.420081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:08.501328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.423946+0000","last_change":"2024-03-23T09:47:11.414821+0000","last_active":"2024-03-23T09:47:11.423946+0000","last_peered":"2024-03-23T09:47:11.423946+0000","last_clean":"2024-03-23T09:47:11.423946+0000","last_became_active":"2024-03-23T09:47:11.414581+0000","last_became_peered":"2024-03-23T09:47:11.414581+0000","last_unstale":"2024-03-23T09:47:11.423946+0000","last_undegraded":"2024-03-23T09:47:11.423946+0000","last_fullsized":"2024-03-23T09:47:11.423946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:16:21.633326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.423828+0000","last_change":"2024-03-23T09:47:11.037998+0000","last_active":"2024-03-23T09:47:11.423828+0000","last_peered":"2024-03-23T09:47:11.423828+0000","last_clean":"2024-03-23T09:47:11.423828+0000","last_became_active":"2024-03-23T09:47:11.037749+0000","last_became_peered":"2024-03-23T09:47:11.037749+0000","last_unstale":"2024-03-23T09:47:11.423828+0000","last_undegraded":"2024-03-23T09:47:11.423828+0000","last_fullsized":"2024-03-23T09:47:11.423828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:41:38.279080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424897+0000","last_change":"2024-03-23T09:47:11.424435+0000","last_active":"2024-03-23T09:47:11.424897+0000","last_peered":"2024-03-23T09:47:11.424897+0000","last_clean":"2024-03-23T09:47:11.424897+0000","last_became_active":"2024-03-23T09:47:11.424169+0000","last_became_peered":"2024-03-23T09:47:11.424169+0000","last_unstale":"2024-03-23T09:47:11.424897+0000","last_undegraded":"2024-03-23T09:47:11.424897+0000","last_fullsized":"2024-03-23T09:47:11.424897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:16:21.633326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.422731+0000","last_change":"2024-03-23T09:47:11.041094+0000","last_active":"2024-03-23T09:47:11.422731+0000","last_peered":"2024-03-23T09:47:11.422731+0000","last_clean":"2024-03-23T09:47:11.422731+0000","last_became_active":"2024-03-23T09:47:11.040738+0000","last_became_peered":"2024-03-23T09:47:11.040738+0000","last_unstale":"2024-03-23T09:47:11.422731+0000","last_undegraded":"2024-03-23T09:47:11.422731+0000","last_fullsized":"2024-03-23T09:47:11.422731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:28:41.141201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412509+0000","last_change":"2024-03-23T09:47:11.409127+0000","last_active":"2024-03-23T09:47:11.412509+0000","last_peered":"2024-03-23T09:47:11.412509+0000","last_clean":"2024-03-23T09:47:11.412509+0000","last_became_active":"2024-03-23T09:47:11.408622+0000","last_became_peered":"2024-03-23T09:47:11.408622+0000","last_unstale":"2024-03-23T09:47:11.412509+0000","last_undegraded":"2024-03-23T09:47:11.412509+0000","last_fullsized":"2024-03-23T09:47:11.412509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:59:56.765267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.396673+0000","last_change":"2024-03-23T09:47:11.038063+0000","last_active":"2024-03-23T09:47:11.396673+0000","last_peered":"2024-03-23T09:47:11.396673+0000","last_clean":"2024-03-23T09:47:11.396673+0000","last_became_active":"2024-03-23T09:47:11.037488+0000","last_became_peered":"2024-03-23T09:47:11.037488+0000","last_unstale":"2024-03-23T09:47:11.396673+0000","last_undegraded":"2024-03-23T09:47:11.396673+0000","last_fullsized":"2024-03-23T09:47:11.396673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:59:55.759941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409794+0000","last_change":"2024-03-23T09:47:11.407676+0000","last_active":"2024-03-23T09:47:11.409794+0000","last_peered":"2024-03-23T09:47:11.409794+0000","last_clean":"2024-03-23T09:47:11.409794+0000","last_became_active":"2024-03-23T09:47:11.407442+0000","last_became_peered":"2024-03-23T09:47:11.407442+0000","last_unstale":"2024-03-23T09:47:11.409794+0000","last_undegraded":"2024-03-23T09:47:11.409794+0000","last_fullsized":"2024-03-23T09:47:11.409794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:52:59.938205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.402747+0000","last_change":"2024-03-23T09:47:11.039946+0000","last_active":"2024-03-23T09:47:11.402747+0000","last_peered":"2024-03-23T09:47:11.402747+0000","last_clean":"2024-03-23T09:47:11.402747+0000","last_became_active":"2024-03-23T09:47:11.039660+0000","last_became_peered":"2024-03-23T09:47:11.039660+0000","last_unstale":"2024-03-23T09:47:11.402747+0000","last_undegraded":"2024-03-23T09:47:11.402747+0000","last_fullsized":"2024-03-23T09:47:11.402747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:20:53.058832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.419889+0000","last_change":"2024-03-23T09:47:11.416706+0000","last_active":"2024-03-23T09:47:11.419889+0000","last_peered":"2024-03-23T09:47:11.419889+0000","last_clean":"2024-03-23T09:47:11.419889+0000","last_became_active":"2024-03-23T09:47:11.416460+0000","last_became_peered":"2024-03-23T09:47:11.416460+0000","last_unstale":"2024-03-23T09:47:11.419889+0000","last_undegraded":"2024-03-23T09:47:11.419889+0000","last_fullsized":"2024-03-23T09:47:11.419889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:40.160543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421846+0000","last_change":"2024-03-23T09:47:11.042502+0000","last_active":"2024-03-23T09:47:11.421846+0000","last_peered":"2024-03-23T09:47:11.421846+0000","last_clean":"2024-03-23T09:47:11.421846+0000","last_became_active":"2024-03-23T09:47:11.042193+0000","last_became_peered":"2024-03-23T09:47:11.042193+0000","last_unstale":"2024-03-23T09:47:11.421846+0000","last_undegraded":"2024-03-23T09:47:11.421846+0000","last_fullsized":"2024-03-23T09:47:11.421846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:35:53.432938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412511+0000","last_change":"2024-03-23T09:47:11.409298+0000","last_active":"2024-03-23T09:47:11.412511+0000","last_peered":"2024-03-23T09:47:11.412511+0000","last_clean":"2024-03-23T09:47:11.412511+0000","last_became_active":"2024-03-23T09:47:11.408207+0000","last_became_peered":"2024-03-23T09:47:11.408207+0000","last_unstale":"2024-03-23T09:47:11.412511+0000","last_undegraded":"2024-03-23T09:47:11.412511+0000","last_fullsized":"2024-03-23T09:47:11.412511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:38:15.086915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"84'1","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.420368+0000","last_change":"2024-03-23T09:47:11.036587+0000","last_active":"2024-03-23T09:47:11.420368+0000","last_peered":"2024-03-23T09:47:11.420368+0000","last_clean":"2024-03-23T09:47:11.420368+0000","last_became_active":"2024-03-23T09:47:11.036143+0000","last_became_peered":"2024-03-23T09:47:11.036143+0000","last_unstale":"2024-03-23T09:47:11.420368+0000","last_undegraded":"2024-03-23T09:47:11.420368+0000","last_fullsized":"2024-03-23T09:47:11.420368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:04.866753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421363+0000","last_change":"2024-03-23T09:47:11.414510+0000","last_active":"2024-03-23T09:47:11.421363+0000","last_peered":"2024-03-23T09:47:11.421363+0000","last_clean":"2024-03-23T09:47:11.421363+0000","last_became_active":"2024-03-23T09:47:11.414251+0000","last_became_peered":"2024-03-23T09:47:11.414251+0000","last_unstale":"2024-03-23T09:47:11.421363+0000","last_undegraded":"2024-03-23T09:47:11.421363+0000","last_fullsized":"2024-03-23T09:47:11.421363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:46:56.811017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424100+0000","last_change":"2024-03-23T09:47:11.043123+0000","last_active":"2024-03-23T09:47:11.424100+0000","last_peered":"2024-03-23T09:47:11.424100+0000","last_clean":"2024-03-23T09:47:11.424100+0000","last_became_active":"2024-03-23T09:47:11.042890+0000","last_became_peered":"2024-03-23T09:47:11.042890+0000","last_unstale":"2024-03-23T09:47:11.424100+0000","last_undegraded":"2024-03-23T09:47:11.424100+0000","last_fullsized":"2024-03-23T09:47:11.424100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:10:20.426205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416208+0000","last_change":"2024-03-23T09:47:11.404906+0000","last_active":"2024-03-23T09:47:11.416208+0000","last_peered":"2024-03-23T09:47:11.416208+0000","last_clean":"2024-03-23T09:47:11.416208+0000","last_became_active":"2024-03-23T09:47:11.404521+0000","last_became_peered":"2024-03-23T09:47:11.404521+0000","last_unstale":"2024-03-23T09:47:11.416208+0000","last_undegraded":"2024-03-23T09:47:11.416208+0000","last_fullsized":"2024-03-23T09:47:11.416208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:21:53.365087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406640+0000","last_change":"2024-03-23T09:47:11.040175+0000","last_active":"2024-03-23T09:47:11.406640+0000","last_peered":"2024-03-23T09:47:11.406640+0000","last_clean":"2024-03-23T09:47:11.406640+0000","last_became_active":"2024-03-23T09:47:11.039927+0000","last_became_peered":"2024-03-23T09:47:11.039927+0000","last_unstale":"2024-03-23T09:47:11.406640+0000","last_undegraded":"2024-03-23T09:47:11.406640+0000","last_fullsized":"2024-03-23T09:47:11.406640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:10:01.015349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406633+0000","last_change":"2024-03-23T09:47:11.403033+0000","last_active":"2024-03-23T09:47:11.406633+0000","last_peered":"2024-03-23T09:47:11.406633+0000","last_clean":"2024-03-23T09:47:11.406633+0000","last_became_active":"2024-03-23T09:47:11.402436+0000","last_became_peered":"2024-03-23T09:47:11.402436+0000","last_unstale":"2024-03-23T09:47:11.406633+0000","last_undegraded":"2024-03-23T09:47:11.406633+0000","last_fullsized":"2024-03-23T09:47:11.406633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:28:42.146527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.407703+0000","last_change":"2024-03-23T09:47:11.030291+0000","last_active":"2024-03-23T09:47:11.407703+0000","last_peered":"2024-03-23T09:47:11.407703+0000","last_clean":"2024-03-23T09:47:11.407703+0000","last_became_active":"2024-03-23T09:47:11.030047+0000","last_became_peered":"2024-03-23T09:47:11.030047+0000","last_unstale":"2024-03-23T09:47:11.407703+0000","last_undegraded":"2024-03-23T09:47:11.407703+0000","last_fullsized":"2024-03-23T09:47:11.407703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:06:15.483933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.411968+0000","last_change":"2024-03-23T09:47:11.410537+0000","last_active":"2024-03-23T09:47:11.411968+0000","last_peered":"2024-03-23T09:47:11.411968+0000","last_clean":"2024-03-23T09:47:11.411968+0000","last_became_active":"2024-03-23T09:47:11.410278+0000","last_became_peered":"2024-03-23T09:47:11.410278+0000","last_unstale":"2024-03-23T09:47:11.411968+0000","last_undegraded":"2024-03-23T09:47:11.411968+0000","last_fullsized":"2024-03-23T09:47:11.411968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:52:59.938205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.394248+0000","last_change":"2024-03-23T09:47:11.035723+0000","last_active":"2024-03-23T09:47:11.394248+0000","last_peered":"2024-03-23T09:47:11.394248+0000","last_clean":"2024-03-23T09:47:11.394248+0000","last_became_active":"2024-03-23T09:47:11.035455+0000","last_became_peered":"2024-03-23T09:47:11.035455+0000","last_unstale":"2024-03-23T09:47:11.394248+0000","last_undegraded":"2024-03-23T09:47:11.394248+0000","last_fullsized":"2024-03-23T09:47:11.394248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:43:34.477625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412972+0000","last_change":"2024-03-23T09:47:11.409615+0000","last_active":"2024-03-23T09:47:11.412972+0000","last_peered":"2024-03-23T09:47:11.412972+0000","last_clean":"2024-03-23T09:47:11.412972+0000","last_became_active":"2024-03-23T09:47:11.409329+0000","last_became_peered":"2024-03-23T09:47:11.409329+0000","last_unstale":"2024-03-23T09:47:11.412972+0000","last_undegraded":"2024-03-23T09:47:11.412972+0000","last_fullsized":"2024-03-23T09:47:11.412972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:38.307851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.393880+0000","last_change":"2024-03-23T09:47:11.036698+0000","last_active":"2024-03-23T09:47:11.393880+0000","last_peered":"2024-03-23T09:47:11.393880+0000","last_clean":"2024-03-23T09:47:11.393880+0000","last_became_active":"2024-03-23T09:47:11.036444+0000","last_became_peered":"2024-03-23T09:47:11.036444+0000","last_unstale":"2024-03-23T09:47:11.393880+0000","last_undegraded":"2024-03-23T09:47:11.393880+0000","last_fullsized":"2024-03-23T09:47:11.393880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:27:12.582469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.423795+0000","last_change":"2024-03-23T09:47:11.419859+0000","last_active":"2024-03-23T09:47:11.423795+0000","last_peered":"2024-03-23T09:47:11.423795+0000","last_clean":"2024-03-23T09:47:11.423795+0000","last_became_active":"2024-03-23T09:47:11.419360+0000","last_became_peered":"2024-03-23T09:47:11.419360+0000","last_unstale":"2024-03-23T09:47:11.423795+0000","last_undegraded":"2024-03-23T09:47:11.423795+0000","last_fullsized":"2024-03-23T09:47:11.423795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:40.160543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.397198+0000","last_change":"2024-03-23T09:47:11.030195+0000","last_active":"2024-03-23T09:47:11.397198+0000","last_peered":"2024-03-23T09:47:11.397198+0000","last_clean":"2024-03-23T09:47:11.397198+0000","last_became_active":"2024-03-23T09:47:11.029904+0000","last_became_peered":"2024-03-23T09:47:11.029904+0000","last_unstale":"2024-03-23T09:47:11.397198+0000","last_undegraded":"2024-03-23T09:47:11.397198+0000","last_fullsized":"2024-03-23T09:47:11.397198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:38:14.081589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.416622+0000","last_change":"2024-03-23T09:47:11.411692+0000","last_active":"2024-03-23T09:47:11.416622+0000","last_peered":"2024-03-23T09:47:11.416622+0000","last_clean":"2024-03-23T09:47:11.416622+0000","last_became_active":"2024-03-23T09:47:11.411455+0000","last_became_peered":"2024-03-23T09:47:11.411455+0000","last_unstale":"2024-03-23T09:47:11.416622+0000","last_undegraded":"2024-03-23T09:47:11.416622+0000","last_fullsized":"2024-03-23T09:47:11.416622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:59:33.183165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424350+0000","last_change":"2024-03-23T09:47:11.042468+0000","last_active":"2024-03-23T09:47:11.424350+0000","last_peered":"2024-03-23T09:47:11.424350+0000","last_clean":"2024-03-23T09:47:11.424350+0000","last_became_active":"2024-03-23T09:47:11.042161+0000","last_became_peered":"2024-03-23T09:47:11.042161+0000","last_unstale":"2024-03-23T09:47:11.424350+0000","last_undegraded":"2024-03-23T09:47:11.424350+0000","last_fullsized":"2024-03-23T09:47:11.424350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:14:33.493692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424087+0000","last_change":"2024-03-23T09:47:11.419426+0000","last_active":"2024-03-23T09:47:11.424087+0000","last_peered":"2024-03-23T09:47:11.424087+0000","last_clean":"2024-03-23T09:47:11.424087+0000","last_became_active":"2024-03-23T09:47:11.419138+0000","last_became_peered":"2024-03-23T09:47:11.419138+0000","last_unstale":"2024-03-23T09:47:11.424087+0000","last_undegraded":"2024-03-23T09:47:11.424087+0000","last_fullsized":"2024-03-23T09:47:11.424087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:33:16.231177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.398091+0000","last_change":"2024-03-23T09:47:11.035761+0000","last_active":"2024-03-23T09:47:11.398091+0000","last_peered":"2024-03-23T09:47:11.398091+0000","last_clean":"2024-03-23T09:47:11.398091+0000","last_became_active":"2024-03-23T09:47:11.035526+0000","last_became_peered":"2024-03-23T09:47:11.035526+0000","last_unstale":"2024-03-23T09:47:11.398091+0000","last_undegraded":"2024-03-23T09:47:11.398091+0000","last_fullsized":"2024-03-23T09:47:11.398091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:21.018329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424209+0000","last_change":"2024-03-23T09:47:11.420249+0000","last_active":"2024-03-23T09:47:11.424209+0000","last_peered":"2024-03-23T09:47:11.424209+0000","last_clean":"2024-03-23T09:47:11.424209+0000","last_became_active":"2024-03-23T09:47:11.419410+0000","last_became_peered":"2024-03-23T09:47:11.419410+0000","last_unstale":"2024-03-23T09:47:11.424209+0000","last_undegraded":"2024-03-23T09:47:11.424209+0000","last_fullsized":"2024-03-23T09:47:11.424209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:10:21.431532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.405708+0000","last_change":"2024-03-23T09:47:11.035478+0000","last_active":"2024-03-23T09:47:11.405708+0000","last_peered":"2024-03-23T09:47:11.405708+0000","last_clean":"2024-03-23T09:47:11.405708+0000","last_became_active":"2024-03-23T09:47:11.034616+0000","last_became_peered":"2024-03-23T09:47:11.034616+0000","last_unstale":"2024-03-23T09:47:11.405708+0000","last_undegraded":"2024-03-23T09:47:11.405708+0000","last_fullsized":"2024-03-23T09:47:11.405708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:41:38.279080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.424024+0000","last_change":"2024-03-23T09:47:11.416738+0000","last_active":"2024-03-23T09:47:11.424024+0000","last_peered":"2024-03-23T09:47:11.424024+0000","last_clean":"2024-03-23T09:47:11.424024+0000","last_became_active":"2024-03-23T09:47:11.415878+0000","last_became_peered":"2024-03-23T09:47:11.415878+0000","last_unstale":"2024-03-23T09:47:11.424024+0000","last_undegraded":"2024-03-23T09:47:11.424024+0000","last_fullsized":"2024-03-23T09:47:11.424024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:33:16.231177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409142+0000","last_change":"2024-03-23T09:47:11.038803+0000","last_active":"2024-03-23T09:47:11.409142+0000","last_peered":"2024-03-23T09:47:11.409142+0000","last_clean":"2024-03-23T09:47:11.409142+0000","last_became_active":"2024-03-23T09:47:11.038534+0000","last_became_peered":"2024-03-23T09:47:11.038534+0000","last_unstale":"2024-03-23T09:47:11.409142+0000","last_undegraded":"2024-03-23T09:47:11.409142+0000","last_fullsized":"2024-03-23T09:47:11.409142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:52:50.532114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.408715+0000","last_change":"2024-03-23T09:47:11.406913+0000","last_active":"2024-03-23T09:47:11.408715+0000","last_peered":"2024-03-23T09:47:11.408715+0000","last_clean":"2024-03-23T09:47:11.408715+0000","last_became_active":"2024-03-23T09:47:11.406649+0000","last_became_peered":"2024-03-23T09:47:11.406649+0000","last_unstale":"2024-03-23T09:47:11.408715+0000","last_undegraded":"2024-03-23T09:47:11.408715+0000","last_fullsized":"2024-03-23T09:47:11.408715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:45:36.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.407074+0000","last_change":"2024-03-23T09:47:11.042912+0000","last_active":"2024-03-23T09:47:11.407074+0000","last_peered":"2024-03-23T09:47:11.407074+0000","last_clean":"2024-03-23T09:47:11.407074+0000","last_became_active":"2024-03-23T09:47:11.042517+0000","last_became_peered":"2024-03-23T09:47:11.042517+0000","last_unstale":"2024-03-23T09:47:11.407074+0000","last_undegraded":"2024-03-23T09:47:11.407074+0000","last_fullsized":"2024-03-23T09:47:11.407074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:46:55.805690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.419868+0000","last_change":"2024-03-23T09:47:11.419080+0000","last_active":"2024-03-23T09:47:11.419868+0000","last_peered":"2024-03-23T09:47:11.419868+0000","last_clean":"2024-03-23T09:47:11.419868+0000","last_became_active":"2024-03-23T09:47:11.418788+0000","last_became_peered":"2024-03-23T09:47:11.418788+0000","last_unstale":"2024-03-23T09:47:11.419868+0000","last_undegraded":"2024-03-23T09:47:11.419868+0000","last_fullsized":"2024-03-23T09:47:11.419868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:21:53.365087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.398216+0000","last_change":"2024-03-23T09:47:11.032898+0000","last_active":"2024-03-23T09:47:11.398216+0000","last_peered":"2024-03-23T09:47:11.398216+0000","last_clean":"2024-03-23T09:47:11.398216+0000","last_became_active":"2024-03-23T09:47:11.032593+0000","last_became_peered":"2024-03-23T09:47:11.032593+0000","last_unstale":"2024-03-23T09:47:11.398216+0000","last_undegraded":"2024-03-23T09:47:11.398216+0000","last_fullsized":"2024-03-23T09:47:11.398216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:15:27.762824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.412557+0000","last_change":"2024-03-23T09:47:11.409994+0000","last_active":"2024-03-23T09:47:11.412557+0000","last_peered":"2024-03-23T09:47:11.412557+0000","last_clean":"2024-03-23T09:47:11.412557+0000","last_became_active":"2024-03-23T09:47:11.409056+0000","last_became_peered":"2024-03-23T09:47:11.409056+0000","last_unstale":"2024-03-23T09:47:11.412557+0000","last_undegraded":"2024-03-23T09:47:11.412557+0000","last_fullsized":"2024-03-23T09:47:11.412557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:30.983075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.406791+0000","last_change":"2024-03-23T09:47:11.037345+0000","last_active":"2024-03-23T09:47:11.406791+0000","last_peered":"2024-03-23T09:47:11.406791+0000","last_clean":"2024-03-23T09:47:11.406791+0000","last_became_active":"2024-03-23T09:47:11.037044+0000","last_became_peered":"2024-03-23T09:47:11.037044+0000","last_unstale":"2024-03-23T09:47:11.406791+0000","last_undegraded":"2024-03-23T09:47:11.406791+0000","last_fullsized":"2024-03-23T09:47:11.406791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:15:21.018329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.421329+0000","last_change":"2024-03-23T09:47:11.414063+0000","last_active":"2024-03-23T09:47:11.421329+0000","last_peered":"2024-03-23T09:47:11.421329+0000","last_clean":"2024-03-23T09:47:11.421329+0000","last_became_active":"2024-03-23T09:47:11.413828+0000","last_became_peered":"2024-03-23T09:47:11.413828+0000","last_unstale":"2024-03-23T09:47:11.421329+0000","last_undegraded":"2024-03-23T09:47:11.421329+0000","last_fullsized":"2024-03-23T09:47:11.421329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:06:11.483774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.398600+0000","last_change":"2024-03-23T09:47:11.036536+0000","last_active":"2024-03-23T09:47:11.398600+0000","last_peered":"2024-03-23T09:47:11.398600+0000","last_clean":"2024-03-23T09:47:11.398600+0000","last_became_active":"2024-03-23T09:47:11.036119+0000","last_became_peered":"2024-03-23T09:47:11.036119+0000","last_unstale":"2024-03-23T09:47:11.398600+0000","last_undegraded":"2024-03-23T09:47:11.398600+0000","last_fullsized":"2024-03-23T09:47:11.398600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:08:29.870983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409402+0000","last_change":"2024-03-23T09:47:11.406594+0000","last_active":"2024-03-23T09:47:11.409402+0000","last_peered":"2024-03-23T09:47:11.409402+0000","last_clean":"2024-03-23T09:47:11.409402+0000","last_became_active":"2024-03-23T09:47:11.406362+0000","last_became_peered":"2024-03-23T09:47:11.406362+0000","last_unstale":"2024-03-23T09:47:11.409402+0000","last_undegraded":"2024-03-23T09:47:11.409402+0000","last_fullsized":"2024-03-23T09:47:11.409402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:11.016146+0000","last_clean_scrub_stamp":"2024-03-23T09:47:11.016146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:26:09.325579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.409300+0000","last_change":"2024-03-23T09:47:11.042365+0000","last_active":"2024-03-23T09:47:11.409300+0000","last_peered":"2024-03-23T09:47:11.409300+0000","last_clean":"2024-03-23T09:47:11.409300+0000","last_became_active":"2024-03-23T09:47:11.041966+0000","last_became_peered":"2024-03-23T09:47:11.041966+0000","last_unstale":"2024-03-23T09:47:11.409300+0000","last_undegraded":"2024-03-23T09:47:11.409300+0000","last_fullsized":"2024-03-23T09:47:11.409300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:47:10.010820+0000","last_clean_scrub_stamp":"2024-03-23T09:47:10.010820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:14.473986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":230,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-23T09:47:11.034578+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:47:11.034578+0000","last_peered":"2024-03-23T09:47:11.034578+0000","last_clean":"2024-03-23T09:47:11.034578+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:47:11.034578+0000","last_undegraded":"2024-03-23T09:47:11.034578+0000","last_fullsized":"2024-03-23T09:47:11.034578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1630208,"data_stored":1027680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.499},"min":{"1min":0.35799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67600000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58699999999999997,"15min":0.52300000000000002},"min":{"1min":0.42899999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71999999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45400000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57299999999999995,"15min":0.52300000000000002},"min":{"1min":0.38300000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69699999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.52200000000000002},"min":{"1min":0.33200000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76000000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40600000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.53900000000000003},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.34699999999999998},"max":{"1min":0.73199999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.45900000000000002,"5min":0.42299999999999999,"15min":0.39900000000000002},"max":{"1min":0.74199999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.41899999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57799999999999996,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.373},"max":{"1min":0.73899999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.436},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.45400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69699999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.39100000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.499,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76300000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.51900000000000002},"min":{"1min":0.38200000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74399999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34999999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.77300000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.56299999999999994},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.36899999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.58299999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.753,"5min":0.874,"15min":0.874},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.38600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76200000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.32200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1040384,"data_stored":437317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55800000000000005,"15min":0.52900000000000003},"min":{"1min":0.48299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.37}]},{"osd":1,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54900000000000004,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.432},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.40999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54600000000000004,"15min":0.52500000000000002},"min":{"1min":0.39000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.39400000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.42299999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.44600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40799999999999997}]},{"osd":4,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48299999999999998,"15min":0.503},"min":{"1min":0.33300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69799999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57199999999999995}]},{"osd":5,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.503,"15min":0.52900000000000003},"min":{"1min":0.39800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.378,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.746,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51600000000000001}]},{"osd":7,"last update":"Sat Mar 23 09:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55300000000000005,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78400000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":438828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54900000000000004,"15min":0.49299999999999999},"min":{"1min":0.42799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.752,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.628,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.27400000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54700000000000004,"15min":0.499},"min":{"1min":0.371,"5min":0.371,"15min":0.35199999999999998},"max":{"1min":0.623,"5min":0.746,"15min":0.746},"last":0.313},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.51200000000000001},"min":{"1min":0.44400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72499999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.38300000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57199999999999995,"15min":0.53600000000000003},"min":{"1min":0.41399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72699999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56599999999999995,"15min":0.52600000000000002},"min":{"1min":0.434,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.435}]},{"osd":3,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56899999999999995,"15min":0.51400000000000001},"min":{"1min":0.45500000000000002,"5min":0.376,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.371},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.54100000000000004},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":0.71199999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.35899999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.45000000000000001,"15min":0.46700000000000003},"min":{"1min":0.318,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.57399999999999995,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.49099999999999999,"15min":0.51500000000000001},"min":{"1min":0.36199999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.93600000000000005},"last":0.41699999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48999999999999999,"15min":0.498},"min":{"1min":0.36299999999999999,"5min":0.33500000000000002,"15min":0.31900000000000001},"max":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.749},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48799999999999999,"15min":0.50800000000000001},"min":{"1min":0.38200000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.70099999999999996,"15min":0.72399999999999998},"last":0.49099999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.44400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.78200000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.504,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.47999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":1032192,"data_stored":437242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58099999999999996,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70399999999999996,"5min":0.78500000000000003,"15min":0.84599999999999997},"last":0.313},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.754,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.47199999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.48599999999999999},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72599999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53600000000000003,"15min":0.51000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.77400000000000002},"last":0.36299999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59099999999999997,"15min":0.54800000000000004},"min":{"1min":0.42799999999999999,"5min":0.40200000000000002,"15min":0.39400000000000002},"max":{"1min":0.77100000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.499},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59499999999999997,"15min":0.54900000000000004},"min":{"1min":0.48999999999999999,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.76600000000000001,"5min":0.81200000000000006,"15min":0.82199999999999995},"last":0.51500000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.753,"5min":0.80900000000000005,"15min":1.0369999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56100000000000005,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.38500000000000001,"15min":0.374},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.438}]},{"osd":5,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46000000000000002,"15min":0.46800000000000003},"min":{"1min":0.34599999999999997,"5min":0.30199999999999999,"15min":0.26500000000000001},"max":{"1min":0.58099999999999996,"5min":0.73099999999999998,"15min":0.75800000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46500000000000002,"15min":0.51300000000000001},"min":{"1min":0.33000000000000002,"5min":0.308,"15min":0.253},"max":{"1min":0.58099999999999996,"5min":0.76100000000000001,"15min":1.008},"last":0.52900000000000003}]},{"osd":6,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35499999999999998,"5min":0.307,"15min":0.28100000000000003},"max":{"1min":0.60699999999999998,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.497,"15min":0.51700000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.314},"max":{"1min":0.627,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54200000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.502,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.65100000000000002,"5min":0.71799999999999997,"15min":1.0069999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.57399999999999995},"min":{"1min":0.40699999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.68000000000000005,"5min":0.80400000000000005,"15min":0.93400000000000005},"last":0.55300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":1044480,"data_stored":438806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35499999999999998,"15min":0.32100000000000001},"min":{"1min":0.22600000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.56499999999999995,"5min":1.45,"15min":1.45},"last":0.249},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.36099999999999999,"15min":0.32300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.45700000000000002,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.372}]},{"osd":1,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.35299999999999998,"15min":0.318},"min":{"1min":0.24299999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.499,"5min":1.323,"15min":1.323},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38900000000000001,"15min":0.35599999999999998},"min":{"1min":0.28399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.28899999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42799999999999999,"15min":0.38900000000000001},"min":{"1min":0.314,"5min":0.251,"15min":0.251},"max":{"1min":0.57299999999999995,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.44900000000000001,"15min":0.41899999999999998},"min":{"1min":0.32800000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.56499999999999995,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.38800000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71099999999999997,"5min":1.337,"15min":1.337},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.55100000000000005},"min":{"1min":0.44500000000000001,"5min":0.39700000000000002,"15min":0.34699999999999998},"max":{"1min":0.751,"5min":2.758,"15min":2.758},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60699999999999998,"15min":0.59699999999999998},"min":{"1min":0.51000000000000001,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.77000000000000002,"5min":1.276,"15min":1.276},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.751,"5min":1.895,"15min":1.895},"last":0.628}]},{"osd":6,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.62,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.71999999999999997,"5min":5.4539999999999997,"15min":5.4539999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.59099999999999997},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79100000000000004,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.54900000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64500000000000002,"15min":0.61299999999999999},"min":{"1min":0.49299999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76700000000000002,"5min":3.7400000000000002,"15min":3.7400000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66000000000000003,"15min":0.622},"min":{"1min":0.45800000000000002,"5min":0.41399999999999998,"15min":0.40300000000000002},"max":{"1min":0.78600000000000003,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.60099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":1036288,"data_stored":437824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34599999999999997},"min":{"1min":0.219,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.316},{"interface":"front","average":{"1min":0.376,"5min":0.35799999999999998,"15min":0.375},"min":{"1min":0.26400000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.60799999999999998,"5min":0.64400000000000002,"15min":0.872},"last":0.30099999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35499999999999998,"15min":0.36799999999999999},"min":{"1min":0.24199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52100000000000002,"5min":0.61099999999999999,"15min":0.746},"last":0.35499999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.29199999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.57699999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39100000000000001,"15min":0.40300000000000002},"min":{"1min":0.32300000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.72499999999999998},"last":0.379}]},{"osd":4,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61199999999999999,"15min":0.63200000000000001},"min":{"1min":0.5,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77200000000000002,"5min":0.84699999999999998,"15min":1.216},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.501,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81599999999999995,"5min":0.89200000000000002,"15min":0.99299999999999999},"last":0.55100000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.621,"15min":0.63100000000000001},"min":{"1min":0.52600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.79200000000000004,"5min":0.90400000000000003,"15min":0.95699999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.47899999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80600000000000005,"5min":0.93700000000000006,"15min":1.101},"last":0.73399999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.51400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":0.89100000000000001,"15min":0.97299999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.628,"15min":0.64400000000000002},"min":{"1min":0.52600000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.84099999999999997,"5min":0.89900000000000002,"15min":1.0780000000000001},"last":0.52600000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.52900000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":1.155},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64000000000000001,"15min":0.65200000000000002},"min":{"1min":0.51400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":1044480,"data_stored":437436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35499999999999998,"15min":0.33900000000000002},"min":{"1min":0.214,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.52000000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.31},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.371,"15min":0.46000000000000002},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.62,"5min":0.71999999999999997,"15min":2.7069999999999999},"last":0.26500000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36199999999999999,"15min":0.34799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.40100000000000002,"15min":0.379},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.52700000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.28299999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.41299999999999998},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.76900000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.48999999999999999},"min":{"1min":0.25600000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.56200000000000006,"5min":0.73899999999999999,"15min":2.75},"last":0.34399999999999997}]},{"osd":4,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59999999999999998,"15min":0.58799999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.53300000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.625,"15min":0.69099999999999995},"min":{"1min":0.45600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82099999999999995,"5min":1.0900000000000001,"15min":2.778},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63,"15min":0.60999999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88700000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63300000000000001,"15min":0.69899999999999995},"min":{"1min":0.46600000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93100000000000005,"5min":1.026,"15min":2.8210000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.64300000000000002,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79100000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.74399999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:47:11 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64800000000000002,"15min":0.64500000000000002},"min":{"1min":0.46400000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.65000000000000002,"15min":0.74099999999999999},"min":{"1min":0.435,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80500000000000005,"5min":0.91600000000000004,"15min":2.7989999999999999},"last":0.54100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1626112,"data_stored":1027025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34399999999999997,"15min":0.32600000000000001},"min":{"1min":0.23799999999999999,"5min":0.184,"15min":0.184},"max":{"1min":0.49399999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.377,"15min":0.35499999999999998},"min":{"1min":0.23899999999999999,"5min":0.23300000000000001,"15min":0.217},"max":{"1min":0.57999999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.47699999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.27300000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.56999999999999995,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.63},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.371},"min":{"1min":0.32700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53800000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.53300000000000003}]},{"osd":3,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.38400000000000001},"min":{"1min":0.25800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.52600000000000002,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.42799999999999999},"min":{"1min":0.32700000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59599999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.65200000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61499999999999999,"15min":0.58699999999999997},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.625,"15min":0.59399999999999997},"min":{"1min":0.43099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.69999999999999996}]},{"osd":5,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.39500000000000002,"15min":0.36099999999999999},"max":{"1min":0.80900000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.626,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.379},"max":{"1min":0.88,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.622},"min":{"1min":0.49199999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91000000000000003,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.46800000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89100000000000001,"5min":0.91600000000000004,"15min":0.92900000000000005},"last":0.83599999999999997}]},{"osd":7,"last update":"Sat Mar 23 09:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64600000000000002,"15min":0.60799999999999998},"min":{"1min":0.498,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.64300000000000002,"15min":0.60499999999999998},"min":{"1min":0.45700000000000002,"5min":0.44700000000000001,"15min":0.41899999999999998},"max":{"1min":0.84299999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:47:14.748 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T09:47:14.749 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-23T09:47:14.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.893+0000 7f6668e0b640 1 Processor -- start 2024-03-23T09:47:14.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.893+0000 7f6668e0b640 1 -- start start 2024-03-23T09:47:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.893+0000 7f6668e0b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f666415cbb0 0x7f66640b8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.893+0000 7f6668e0b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66640d1df0 0x7f66640b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.893+0000 7f6668e0b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666415cf90 con 0x7f66640b48f0 2024-03-23T09:47:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.893+0000 7f6668e0b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66640b4da0 con 0x7f666415cbb0 2024-03-23T09:47:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.893+0000 7f6668e0b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66640b5210 con 0x7f66640d1df0 2024-03-23T09:47:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.894+0000 7f666259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f666415cbb0 0x7f66640b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.894+0000 7f6662d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f66640b48f0 0x7f66640d22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51308/0 (socket says 172.21.15.47:51308) 2024-03-23T09:47:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.894+0000 7f6662d9d640 1 -- 172.21.15.47:0/1638669761 learned_addr learned my addr 172.21.15.47:0/1638669761 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.894+0000 7f666259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f666415cbb0 0x7f66640b8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38448/0 (socket says 172.21.15.47:38448) 2024-03-23T09:47:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.894+0000 7f666259c640 1 -- 172.21.15.47:0/1638669761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66640d1df0 msgr2=0x7f66640b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.894+0000 7f666259c640 1 --2- 172.21.15.47:0/1638669761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66640d1df0 0x7f66640b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.894+0000 7f666259c640 1 -- 172.21.15.47:0/1638669761 >> v1:172.21.15.47:6789/0 conn(0x7f66640b48f0 legacy=0x7f66640d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.894+0000 7f6661d9b640 1 -- 172.21.15.47:0/1638669761 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849497107 0 0) 0x7f666415cf90 con 0x7f66640b48f0 2024-03-23T09:47:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.894+0000 7f666259c640 1 -- 172.21.15.47:0/1638669761 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66640d0d50 con 0x7f666415cbb0 2024-03-23T09:47:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.895+0000 7f666259c640 1 --2- 172.21.15.47:0/1638669761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f666415cbb0 0x7f66640b8250 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f66540097a0 tx=0x7f665402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8629b0c99d868be8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.895+0000 7f6661d9b640 1 -- 172.21.15.47:0/1638669761 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654031830 con 0x7f666415cbb0 2024-03-23T09:47:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.895+0000 7f6661d9b640 1 -- 172.21.15.47:0/1638669761 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6654031990 con 0x7f666415cbb0 2024-03-23T09:47:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.895+0000 7f6661d9b640 1 -- 172.21.15.47:0/1638669761 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654031d20 con 0x7f666415cbb0 2024-03-23T09:47:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.895+0000 7f6668e0b640 1 -- 172.21.15.47:0/1638669761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f666415cbb0 msgr2=0x7f66640b8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.895+0000 7f6668e0b640 1 --2- 172.21.15.47:0/1638669761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f666415cbb0 0x7f66640b8250 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f66540097a0 tx=0x7f665402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.895+0000 7f6668e0b640 1 -- 172.21.15.47:0/1638669761 shutdown_connections 2024-03-23T09:47:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.895+0000 7f6668e0b640 1 --2- 172.21.15.47:0/1638669761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66640d1df0 0x7f66640b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.895+0000 7f6668e0b640 1 --2- 172.21.15.47:0/1638669761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f666415cbb0 0x7f66640b8250 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.895+0000 7f6668e0b640 1 -- 172.21.15.47:0/1638669761 >> 172.21.15.47:0/1638669761 conn(0x7f6664065dc0 msgr2=0x7f66640661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.895+0000 7f6668e0b640 1 -- 172.21.15.47:0/1638669761 shutdown_connections 2024-03-23T09:47:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.895+0000 7f6668e0b640 1 -- 172.21.15.47:0/1638669761 wait complete. 2024-03-23T09:47:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.897+0000 7f6668e0b640 1 Processor -- start 2024-03-23T09:47:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.897+0000 7f6668e0b640 1 -- start start 2024-03-23T09:47:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.897+0000 7f6668e0b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66640b48f0 0x7f666415b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.897+0000 7f6668e0b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66640d1df0 0x7f666415f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6668e0b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f666415cbb0 0x7f666415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6668e0b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66641f56f0 con 0x7f666415cbb0 2024-03-23T09:47:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6668e0b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66641f5990 con 0x7f66640b48f0 2024-03-23T09:47:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6668e0b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66641f5c70 con 0x7f66640d1df0 2024-03-23T09:47:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6662d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f666415cbb0 0x7f666415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6662d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f666415cbb0 0x7f666415fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38454/0 (socket says 172.21.15.47:38454) 2024-03-23T09:47:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6662d9d640 1 -- 172.21.15.47:0/2047918762 learned_addr learned my addr 172.21.15.47:0/2047918762 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f666259c640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66640b48f0 0x7f666415b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6659d9b640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66640d1df0 0x7f666415f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6662d9d640 1 -- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66640d1df0 msgr2=0x7f666415f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6662d9d640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66640d1df0 0x7f666415f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6662d9d640 1 -- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66640b48f0 msgr2=0x7f666415b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6662d9d640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66640b48f0 0x7f666415b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6662d9d640 1 -- 172.21.15.47:0/2047918762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6654009400 con 0x7f666415cbb0 2024-03-23T09:47:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6662d9d640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f666415cbb0 0x7f666415fe60 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f665c00c700 tx=0x7f665c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f665b7fe640 1 -- 172.21.15.47:0/2047918762 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c0070d0 con 0x7f666415cbb0 2024-03-23T09:47:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f665b7fe640 1 -- 172.21.15.47:0/2047918762 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c007230 con 0x7f666415cbb0 2024-03-23T09:47:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f665b7fe640 1 -- 172.21.15.47:0/2047918762 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c007660 con 0x7f666415cbb0 2024-03-23T09:47:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6668e0b640 1 -- 172.21.15.47:0/2047918762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66641f5f50 con 0x7f666415cbb0 2024-03-23T09:47:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.898+0000 7f6668e0b640 1 -- 172.21.15.47:0/2047918762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666415b9e0 con 0x7f666415cbb0 2024-03-23T09:47:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.899+0000 7f6668e0b640 1 -- 172.21.15.47:0/2047918762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6664072360 con 0x7f666415cbb0 2024-03-23T09:47:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.900+0000 7f665b7fe640 1 -- 172.21.15.47:0/2047918762 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f665c0077c0 con 0x7f666415cbb0 2024-03-23T09:47:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.900+0000 7f665b7fe640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f664406ac00 0x7f664406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.900+0000 7f665b7fe640 1 -- 172.21.15.47:0/2047918762 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f665c088750 con 0x7f666415cbb0 2024-03-23T09:47:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.900+0000 7f666259c640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f664406ac00 0x7f664406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.901+0000 7f666259c640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f664406ac00 0x7f664406d0c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6654036000 tx=0x7f665403e040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:14.904+0000 7f665b7fe640 1 -- 172.21.15.47:0/2047918762 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f665c0539e0 con 0x7f666415cbb0 2024-03-23T09:47:15.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.152+0000 7f6668e0b640 1 -- 172.21.15.47:0/2047918762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6664066be0 con 0x7f666415cbb0 2024-03-23T09:47:15.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.155+0000 7f665b7fe640 1 -- 172.21.15.47:0/2047918762 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f665c018020 con 0x7f666415cbb0 2024-03-23T09:47:15.155 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-03-23T09:47:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.157+0000 7f6668e0b640 1 -- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f664406ac00 msgr2=0x7f664406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.157+0000 7f6668e0b640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f664406ac00 0x7f664406d0c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6654036000 tx=0x7f665403e040 comp rx=0 tx=0).stop 2024-03-23T09:47:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.157+0000 7f6668e0b640 1 -- 172.21.15.47:0/2047918762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f666415cbb0 msgr2=0x7f666415fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.157+0000 7f6668e0b640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f666415cbb0 0x7f666415fe60 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f665c00c700 tx=0x7f665c00cb00 comp rx=0 tx=0).stop 2024-03-23T09:47:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.157+0000 7f6668e0b640 1 -- 172.21.15.47:0/2047918762 shutdown_connections 2024-03-23T09:47:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.157+0000 7f6668e0b640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f664406ac00 0x7f664406d0c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.157+0000 7f6668e0b640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f666415cbb0 0x7f666415fe60 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.157+0000 7f6668e0b640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66640d1df0 0x7f666415f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.157+0000 7f6668e0b640 1 --2- 172.21.15.47:0/2047918762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66640b48f0 0x7f666415b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.157+0000 7f6668e0b640 1 -- 172.21.15.47:0/2047918762 >> 172.21.15.47:0/2047918762 conn(0x7f6664065dc0 msgr2=0x7f66640b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:15.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.157+0000 7f6668e0b640 1 -- 172.21.15.47:0/2047918762 shutdown_connections 2024-03-23T09:47:15.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.157+0000 7f6668e0b640 1 -- 172.21.15.47:0/2047918762 wait complete. 2024-03-23T09:47:15.168 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:47:15.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.268+0000 7fc6a8651640 1 Processor -- start 2024-03-23T09:47:15.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.268+0000 7fc6a8651640 1 -- start start 2024-03-23T09:47:15.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.269+0000 7fc6a8651640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc6a015ca90 0x7fc6a0160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:15.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.269+0000 7fc6a8651640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6a00b8350 0x7fc6a00cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:15.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.269+0000 7fc6a8651640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a00cdc40 0x7fc6a00b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:15.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.269+0000 7fc6a8651640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a00ce0d0 con 0x7fc6a00b8350 2024-03-23T09:47:15.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.269+0000 7fc6a8651640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a00d11e0 con 0x7fc6a015ca90 2024-03-23T09:47:15.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.269+0000 7fc6a8651640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a0160f60 con 0x7fc6a00cdc40 2024-03-23T09:47:15.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.269+0000 7fc6a5bec640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6a00b8350 0x7fc6a00cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:15.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.269+0000 7fc6a5bec640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6a00b8350 0x7fc6a00cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38456/0 (socket says 172.21.15.47:38456) 2024-03-23T09:47:15.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.269+0000 7fc6a5bec640 1 -- 172.21.15.47:0/966997304 learned_addr learned my addr 172.21.15.47:0/966997304 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:15.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.269+0000 7fc6a63ed640 1 --2- 172.21.15.47:0/966997304 >> v2:172.21.15.175:3300/0 conn(0x7fc6a015ca90 0x7fc6a0160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:15.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.269+0000 7fc6a6bee640 1 --2- 172.21.15.47:0/966997304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a00cdc40 0x7fc6a00b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:15.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.270+0000 7fc6a5bec640 1 -- 172.21.15.47:0/966997304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a00cdc40 msgr2=0x7fc6a00b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:15.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.270+0000 7fc6a5bec640 1 --2- 172.21.15.47:0/966997304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a00cdc40 0x7fc6a00b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.270+0000 7fc6a5bec640 1 -- 172.21.15.47:0/966997304 >> v2:172.21.15.175:3300/0 conn(0x7fc6a015ca90 msgr2=0x7fc6a0160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:15.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.270+0000 7fc6a5bec640 1 --2- 172.21.15.47:0/966997304 >> v2:172.21.15.175:3300/0 conn(0x7fc6a015ca90 0x7fc6a0160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.270+0000 7fc6a5bec640 1 -- 172.21.15.47:0/966997304 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a0161240 con 0x7fc6a00b8350 2024-03-23T09:47:15.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.270+0000 7fc6a5bec640 1 --2- 172.21.15.47:0/966997304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a00b8350 0x7fc6a00cd700 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0066680 tx=0x7fc69402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4faaa9f35c80980 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:15.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.270+0000 7fc6a53eb640 1 -- 172.21.15.47:0/966997304 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6940317e0 con 0x7fc6a00b8350 2024-03-23T09:47:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.271+0000 7fc6a53eb640 1 -- 172.21.15.47:0/966997304 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc694031940 con 0x7fc6a00b8350 2024-03-23T09:47:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.271+0000 7fc6a53eb640 1 -- 172.21.15.47:0/966997304 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc694031cd0 con 0x7fc6a00b8350 2024-03-23T09:47:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.271+0000 7fc6a8651640 1 -- 172.21.15.47:0/966997304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a00b8350 msgr2=0x7fc6a00cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.271+0000 7fc6a8651640 1 --2- 172.21.15.47:0/966997304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a00b8350 0x7fc6a00cd700 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0066680 tx=0x7fc69402b880 comp rx=0 tx=0).stop 2024-03-23T09:47:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.271+0000 7fc6a8651640 1 -- 172.21.15.47:0/966997304 shutdown_connections 2024-03-23T09:47:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.271+0000 7fc6a8651640 1 --2- 172.21.15.47:0/966997304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a00cdc40 0x7fc6a00b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.271+0000 7fc6a8651640 1 --2- 172.21.15.47:0/966997304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a00b8350 0x7fc6a00cd700 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.271+0000 7fc6a8651640 1 --2- 172.21.15.47:0/966997304 >> v2:172.21.15.175:3300/0 conn(0x7fc6a015ca90 0x7fc6a0160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.271+0000 7fc6a8651640 1 -- 172.21.15.47:0/966997304 >> 172.21.15.47:0/966997304 conn(0x7fc6a00659d0 msgr2=0x7fc6a0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.271+0000 7fc6a8651640 1 -- 172.21.15.47:0/966997304 shutdown_connections 2024-03-23T09:47:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.271+0000 7fc6a8651640 1 -- 172.21.15.47:0/966997304 wait complete. 2024-03-23T09:47:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.272+0000 7fc6a8651640 1 Processor -- start 2024-03-23T09:47:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.272+0000 7fc6a8651640 1 -- start start 2024-03-23T09:47:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.273+0000 7fc6a8651640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a00b8350 0x7fc6a00caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.273+0000 7fc6a8651640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a00cdc40 0x7fc6a015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.273+0000 7fc6a8651640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6a015ca90 0x7fc6a015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.273+0000 7fc6a8651640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00cbfb0 con 0x7fc6a00b8350 2024-03-23T09:47:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.273+0000 7fc6a8651640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a01f6230 con 0x7fc6a015ca90 2024-03-23T09:47:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.273+0000 7fc6a8651640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a01f6510 con 0x7fc6a00cdc40 2024-03-23T09:47:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.273+0000 7fc6a63ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a00b8350 0x7fc6a00caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.273+0000 7fc6a63ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a00b8350 0x7fc6a00caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38464/0 (socket says 172.21.15.47:38464) 2024-03-23T09:47:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.273+0000 7fc6a63ed640 1 -- 172.21.15.47:0/3495629452 learned_addr learned my addr 172.21.15.47:0/3495629452 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.273+0000 7fc6a5bec640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a00cdc40 0x7fc6a015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.273+0000 7fc6a6bee640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6a015ca90 0x7fc6a015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.273+0000 7fc6a63ed640 1 -- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a00cdc40 msgr2=0x7fc6a015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.273+0000 7fc6a63ed640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a00cdc40 0x7fc6a015ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.274+0000 7fc6a63ed640 1 -- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6a015ca90 msgr2=0x7fc6a015f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.274+0000 7fc6a63ed640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6a015ca90 0x7fc6a015f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.274+0000 7fc6a63ed640 1 -- 172.21.15.47:0/3495629452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6940094d0 con 0x7fc6a00b8350 2024-03-23T09:47:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.274+0000 7fc6a5bec640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a00cdc40 0x7fc6a015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.274+0000 7fc6a63ed640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a00b8350 0x7fc6a00caae0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fc69800e9b0 tx=0x7fc69800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.274+0000 7fc69effd640 1 -- 172.21.15.47:0/3495629452 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc698009670 con 0x7fc6a00b8350 2024-03-23T09:47:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.274+0000 7fc69effd640 1 -- 172.21.15.47:0/3495629452 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69800c9a0 con 0x7fc6a00b8350 2024-03-23T09:47:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.274+0000 7fc6a8651640 1 -- 172.21.15.47:0/3495629452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a01f67f0 con 0x7fc6a00b8350 2024-03-23T09:47:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.274+0000 7fc69effd640 1 -- 172.21.15.47:0/3495629452 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69800cd50 con 0x7fc6a00b8350 2024-03-23T09:47:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.274+0000 7fc6a8651640 1 -- 172.21.15.47:0/3495629452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a01f6ad0 con 0x7fc6a00b8350 2024-03-23T09:47:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.275+0000 7fc6a8651640 1 -- 172.21.15.47:0/3495629452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a0072360 con 0x7fc6a00b8350 2024-03-23T09:47:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.275+0000 7fc69effd640 1 -- 172.21.15.47:0/3495629452 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc698013470 con 0x7fc6a00b8350 2024-03-23T09:47:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.276+0000 7fc69effd640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc68c06ac00 0x7fc68c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.276+0000 7fc69effd640 1 -- 172.21.15.47:0/3495629452 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc698089650 con 0x7fc6a00b8350 2024-03-23T09:47:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.276+0000 7fc6a5bec640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc68c06ac00 0x7fc68c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.277+0000 7fc6a5bec640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc68c06ac00 0x7fc68c06d0c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc694036000 tx=0x7fc69403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.280+0000 7fc69effd640 1 -- 172.21.15.47:0/3495629452 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc698054830 con 0x7fc6a00b8350 2024-03-23T09:47:15.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.498+0000 7fc6a8651640 1 -- 172.21.15.47:0/3495629452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6a0066be0 con 0x7fc6a00b8350 2024-03-23T09:47:15.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.499+0000 7fc69effd640 1 -- 172.21.15.47:0/3495629452 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fc698057e50 con 0x7fc6a00b8350 2024-03-23T09:47:15.499 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:15.499 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:47:13.401354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/1927629460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1927629460},{"type":"v1","addr":"172.21.15.47:6835","nonce":1927629460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:47:15.499 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 46 2024-03-23T09:47:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.500+0000 7fc6a8651640 1 -- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc68c06ac00 msgr2=0x7fc68c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.500+0000 7fc6a8651640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc68c06ac00 0x7fc68c06d0c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc694036000 tx=0x7fc69403d040 comp rx=0 tx=0).stop 2024-03-23T09:47:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.500+0000 7fc6a8651640 1 -- 172.21.15.47:0/3495629452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a00b8350 msgr2=0x7fc6a00caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.500+0000 7fc6a8651640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a00b8350 0x7fc6a00caae0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fc69800e9b0 tx=0x7fc69800ee70 comp rx=0 tx=0).stop 2024-03-23T09:47:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.500+0000 7fc6a8651640 1 -- 172.21.15.47:0/3495629452 shutdown_connections 2024-03-23T09:47:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.500+0000 7fc6a8651640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc68c06ac00 0x7fc68c06d0c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.500+0000 7fc6a8651640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6a015ca90 0x7fc6a015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.500+0000 7fc6a8651640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a00cdc40 0x7fc6a015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.500+0000 7fc6a8651640 1 --2- 172.21.15.47:0/3495629452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a00b8350 0x7fc6a00caae0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.500+0000 7fc6a8651640 1 -- 172.21.15.47:0/3495629452 >> 172.21.15.47:0/3495629452 conn(0x7fc6a00659d0 msgr2=0x7fc6a00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.501+0000 7fc6a8651640 1 -- 172.21.15.47:0/3495629452 shutdown_connections 2024-03-23T09:47:15.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:15.501+0000 7fc6a8651640 1 -- 172.21.15.47:0/3495629452 wait complete. 2024-03-23T09:47:15.914 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:47:11.390397+0000', 'modified': '2024-03-23T09:47:13.401354+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7524}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7524': {'gid': 7524, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/1927629460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 1927629460}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 1927629460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:47:15.914 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:47:15.914 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T09:47:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.019+0000 7fab59ece640 1 Processor -- start 2024-03-23T09:47:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.019+0000 7fab59ece640 1 -- start start 2024-03-23T09:47:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.019+0000 7fab59ece640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fab5415cd00 0x7fab540d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.020+0000 7fab59ece640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fab540cf920 0x7fab540b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.020+0000 7fab59ece640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5415d0e0 con 0x7fab540d1020 2024-03-23T09:47:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.020+0000 7fab59ece640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab540d1540 con 0x7fab540cf920 2024-03-23T09:47:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.020+0000 7fab59ece640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab540d1820 con 0x7fab5415cd00 2024-03-23T09:47:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.020+0000 7fab53fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fab540cf920 0x7fab540b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.020+0000 7fab53fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fab540cf920 0x7fab540b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38466/0 (socket says 172.21.15.47:38466) 2024-03-23T09:47:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.020+0000 7fab53fff640 1 -- 172.21.15.47:0/2210587215 learned_addr learned my addr 172.21.15.47:0/2210587215 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.020+0000 7fab537fe640 1 --2- 172.21.15.47:0/2210587215 >> v2:172.21.15.175:3300/0 conn(0x7fab5415cd00 0x7fab540d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.020+0000 7fab53fff640 1 -- 172.21.15.47:0/2210587215 >> v2:172.21.15.175:3300/0 conn(0x7fab5415cd00 msgr2=0x7fab540d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.020+0000 7fab53fff640 1 --2- 172.21.15.47:0/2210587215 >> v2:172.21.15.175:3300/0 conn(0x7fab5415cd00 0x7fab540d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.020+0000 7fab53fff640 1 -- 172.21.15.47:0/2210587215 >> v1:172.21.15.175:6789/0 conn(0x7fab540d1020 legacy=0x7fab540cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.020+0000 7fab53fff640 1 -- 172.21.15.47:0/2210587215 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab540d1b00 con 0x7fab540cf920 2024-03-23T09:47:16.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.021+0000 7fab53fff640 1 --2- 172.21.15.47:0/2210587215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab540cf920 0x7fab540b5a90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fab5406c1a0 tx=0x7fab4c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8597511c8c415c50 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.021+0000 7fab527fc640 1 -- 172.21.15.47:0/2210587215 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c02c9f0 con 0x7fab540cf920 2024-03-23T09:47:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.021+0000 7fab527fc640 1 -- 172.21.15.47:0/2210587215 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c02cb50 con 0x7fab540cf920 2024-03-23T09:47:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.021+0000 7fab527fc640 1 -- 172.21.15.47:0/2210587215 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0343f0 con 0x7fab540cf920 2024-03-23T09:47:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.021+0000 7fab59ece640 1 -- 172.21.15.47:0/2210587215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab540cf920 msgr2=0x7fab540b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.021+0000 7fab59ece640 1 --2- 172.21.15.47:0/2210587215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab540cf920 0x7fab540b5a90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fab5406c1a0 tx=0x7fab4c02be50 comp rx=0 tx=0).stop 2024-03-23T09:47:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.021+0000 7fab59ece640 1 -- 172.21.15.47:0/2210587215 shutdown_connections 2024-03-23T09:47:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.021+0000 7fab59ece640 1 --2- 172.21.15.47:0/2210587215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab540cf920 0x7fab540b5a90 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.021+0000 7fab59ece640 1 --2- 172.21.15.47:0/2210587215 >> v2:172.21.15.175:3300/0 conn(0x7fab5415cd00 0x7fab540d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.021+0000 7fab59ece640 1 -- 172.21.15.47:0/2210587215 >> 172.21.15.47:0/2210587215 conn(0x7fab5406b160 msgr2=0x7fab5406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.022+0000 7fab59ece640 1 -- 172.21.15.47:0/2210587215 shutdown_connections 2024-03-23T09:47:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.022+0000 7fab59ece640 1 -- 172.21.15.47:0/2210587215 wait complete. 2024-03-23T09:47:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.023+0000 7fab59ece640 1 Processor -- start 2024-03-23T09:47:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.023+0000 7fab59ece640 1 -- start start 2024-03-23T09:47:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.023+0000 7fab59ece640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab540cf920 0x7fab5414f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.023+0000 7fab59ece640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fab540d1020 0x7fab5415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.023+0000 7fab59ece640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab5415cd00 0x7fab5415ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab59ece640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab540cc300 con 0x7fab540cf920 2024-03-23T09:47:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab59ece640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab540cb6f0 con 0x7fab5415cd00 2024-03-23T09:47:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab59ece640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab540cb9d0 con 0x7fab540d1020 2024-03-23T09:47:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab537fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab540cf920 0x7fab5414f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab537fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab540cf920 0x7fab5414f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38470/0 (socket says 172.21.15.47:38470) 2024-03-23T09:47:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab537fe640 1 -- 172.21.15.47:0/1161915224 learned_addr learned my addr 172.21.15.47:0/1161915224 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab53fff640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab5415cd00 0x7fab5415ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab537fe640 1 -- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fab540d1020 msgr2=0x7fab5415f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab52ffd640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fab540d1020 0x7fab5415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab537fe640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fab540d1020 0x7fab5415f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab537fe640 1 -- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab5415cd00 msgr2=0x7fab5415ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab537fe640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab5415cd00 0x7fab5415ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab537fe640 1 -- 172.21.15.47:0/1161915224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c009b30 con 0x7fab540cf920 2024-03-23T09:47:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab53fff640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab5415cd00 0x7fab5415ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab537fe640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab540cf920 0x7fab5414f430 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fab4400d5f0 tx=0x7fab4400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab3bfff640 1 -- 172.21.15.47:0/1161915224 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab440125f0 con 0x7fab540cf920 2024-03-23T09:47:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab3bfff640 1 -- 172.21.15.47:0/1161915224 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab44012750 con 0x7fab540cf920 2024-03-23T09:47:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab59ece640 1 -- 172.21.15.47:0/1161915224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab540cbcb0 con 0x7fab540cf920 2024-03-23T09:47:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab3bfff640 1 -- 172.21.15.47:0/1161915224 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44012b00 con 0x7fab540cf920 2024-03-23T09:47:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.024+0000 7fab59ece640 1 -- 172.21.15.47:0/1161915224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5414f970 con 0x7fab540cf920 2024-03-23T09:47:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.026+0000 7fab3bfff640 1 -- 172.21.15.47:0/1161915224 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fab44012c60 con 0x7fab540cf920 2024-03-23T09:47:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.026+0000 7fab3bfff640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fab4006ac00 0x7fab4006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.027+0000 7fab3bfff640 1 -- 172.21.15.47:0/1161915224 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab440898e0 con 0x7fab540cf920 2024-03-23T09:47:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.027+0000 7fab52ffd640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fab4006ac00 0x7fab4006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.027+0000 7fab52ffd640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fab4006ac00 0x7fab4006d0c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fab540ccac0 tx=0x7fab48009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.027+0000 7fab5ca41740 1 Processor -- start 2024-03-23T09:47:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.027+0000 7fab5ca41740 1 -- start start 2024-03-23T09:47:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.028+0000 7fab5ca41740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6ccb1750 0x559b6cc68d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.028+0000 7fab5ca41740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 0x559b6cc692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.028+0000 7fab5ca41740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc63c70 0x559b6cc93a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.028+0000 7fab5ca41740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6cbd49b0 con 0x559b6cc677d0 2024-03-23T09:47:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.028+0000 7fab5ca41740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6cbd2140 con 0x559b6ccb1750 2024-03-23T09:47:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.028+0000 7fab5ca41740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b6cbcda30 con 0x559b6cc63c70 2024-03-23T09:47:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.028+0000 7fab52ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 0x559b6cc692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.028+0000 7fab52ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 0x559b6cc692c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38482/0 (socket says 172.21.15.47:38482) 2024-03-23T09:47:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.028+0000 7fab52ffd640 1 -- 172.21.15.47:0/585206218 learned_addr learned my addr 172.21.15.47:0/585206218 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.028+0000 7fab537fe640 1 --2- 172.21.15.47:0/585206218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6ccb1750 0x559b6cc68d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.028+0000 7fab53fff640 1 --2- 172.21.15.47:0/585206218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc63c70 0x559b6cc93a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.029+0000 7fab52ffd640 1 -- 172.21.15.47:0/585206218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc63c70 msgr2=0x559b6cc93a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.029+0000 7fab52ffd640 1 --2- 172.21.15.47:0/585206218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc63c70 0x559b6cc93a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.029+0000 7fab52ffd640 1 -- 172.21.15.47:0/585206218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6ccb1750 msgr2=0x559b6cc68d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.029+0000 7fab52ffd640 1 --2- 172.21.15.47:0/585206218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6ccb1750 0x559b6cc68d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.029+0000 7fab52ffd640 1 -- 172.21.15.47:0/585206218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b6cb5a2f0 con 0x559b6cc677d0 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.029+0000 7fab52ffd640 1 --2- 172.21.15.47:0/585206218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 0x559b6cc692c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fab48005400 tx=0x7fab48010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500dd98eba562767 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.029+0000 7fab397fa640 1 -- 172.21.15.47:0/585206218 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4801c070 con 0x559b6cc677d0 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.029+0000 7fab397fa640 1 -- 172.21.15.47:0/585206218 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4800d6a0 con 0x559b6cc677d0 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.029+0000 7fab397fa640 1 -- 172.21.15.47:0/585206218 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab480054f0 con 0x559b6cc677d0 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.030+0000 7fab5ca41740 1 -- 172.21.15.47:0/585206218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 msgr2=0x559b6cc692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.030+0000 7fab5ca41740 1 --2- 172.21.15.47:0/585206218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 0x559b6cc692c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fab48005400 tx=0x7fab48010000 comp rx=0 tx=0).stop 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.030+0000 7fab5ca41740 1 -- 172.21.15.47:0/585206218 shutdown_connections 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.030+0000 7fab5ca41740 1 --2- 172.21.15.47:0/585206218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc63c70 0x559b6cc93a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.030+0000 7fab5ca41740 1 --2- 172.21.15.47:0/585206218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 0x559b6cc692c0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.030+0000 7fab5ca41740 1 --2- 172.21.15.47:0/585206218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6ccb1750 0x559b6cc68d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.030+0000 7fab5ca41740 1 -- 172.21.15.47:0/585206218 >> 172.21.15.47:0/585206218 conn(0x559b6cc65ee0 msgr2=0x559b6ccaea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.030+0000 7fab5ca41740 1 -- 172.21.15.47:0/585206218 shutdown_connections 2024-03-23T09:47:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.030+0000 7fab5ca41740 1 -- 172.21.15.47:0/585206218 wait complete. 2024-03-23T09:47:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.031+0000 7fab5ca41740 1 -- start start 2024-03-23T09:47:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.031+0000 7fab5ca41740 1 Processor -- start 2024-03-23T09:47:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab5ca41740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc677d0 0x559b6cc8a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab5ca41740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6ccb1750 0x559b6ccb31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab5ca41740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6cc75880 0x559b6ccafc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab5ca41740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6cb8c090 con 0x559b6ccb1750 2024-03-23T09:47:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab5ca41740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6cb8bd30 con 0x559b6cc75880 2024-03-23T09:47:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab5ca41740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b6cb512a0 con 0x559b6cc677d0 2024-03-23T09:47:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab5ca41740 10 client.0 _ll_register_callbacks cb 0x559b6cb87050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:47:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab52ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6ccb1750 0x559b6ccb31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab52ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6ccb1750 0x559b6ccb31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38490/0 (socket says 172.21.15.47:38490) 2024-03-23T09:47:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab52ffd640 1 -- 172.21.15.47:0/3763704073 learned_addr learned my addr 172.21.15.47:0/3763704073 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab537fe640 1 --2- 172.21.15.47:0/3763704073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc677d0 0x559b6cc8a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab53fff640 1 --2- 172.21.15.47:0/3763704073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6cc75880 0x559b6ccafc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab52ffd640 1 -- 172.21.15.47:0/3763704073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc677d0 msgr2=0x559b6cc8a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab52ffd640 1 --2- 172.21.15.47:0/3763704073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc677d0 0x559b6cc8a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab52ffd640 1 -- 172.21.15.47:0/3763704073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6cc75880 msgr2=0x559b6ccafc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab52ffd640 1 --2- 172.21.15.47:0/3763704073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6cc75880 0x559b6ccafc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.032+0000 7fab52ffd640 1 -- 172.21.15.47:0/3763704073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab480050b0 con 0x559b6ccb1750 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab52ffd640 1 --2- 172.21.15.47:0/3763704073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6ccb1750 0x559b6ccb31e0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x559b6cc64120 tx=0x7fab480036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89feff0ee03cc06e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab327fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab327fc640 1 -- 172.21.15.47:0/3763704073 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab480054f0 con 0x559b6ccb1750 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab327fc640 1 -- 172.21.15.47:0/3763704073 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab4800de50 con 0x559b6ccb1750 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab5ca41740 1 -- 172.21.15.47:0/3763704073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b6c8c13c0 con 0x559b6ccb1750 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab327fc640 1 -- 172.21.15.47:0/3763704073 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab48029640 con 0x559b6ccb1750 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab5ca41740 10 client.7671 fetch_fsmap learned FSMap version 46 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab5ca41740 1 -- 172.21.15.47:0/3763704073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x559b6cbcc0b0 con 0x559b6ccb1750 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab327fc640 1 -- 172.21.15.47:0/3763704073 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559b6c8c13c0 con 0x559b6ccb1750 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab327fc640 1 -- 172.21.15.47:0/3763704073 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fab480297a0 con 0x559b6ccb1750 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab5ca41740 10 client.7671 fetch_fsmap finished waiting for FSMap version 46 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab5ca41740 10 client.7671 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:47:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab5ca41740 20 client.7671 populate_metadata read hostname 'smithi047' 2024-03-23T09:47:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab5ca41740 1 --2- 172.21.15.47:0/3763704073 >> [v2:172.21.15.47:6834/1927629460,v1:172.21.15.47:6835/1927629460] conn(0x559b6ccc9b90 0x559b6cccbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab5ca41740 4 client.7671 mds_command: new command op to 7524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:47:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab5ca41740 1 -- 172.21.15.47:0/3763704073 --> [v2:172.21.15.47:6834/1927629460,v1:172.21.15.47:6835/1927629460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b6c8d26c0 con 0x559b6ccc9b90 2024-03-23T09:47:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.033+0000 7fab537fe640 1 --2- 172.21.15.47:0/3763704073 >> [v2:172.21.15.47:6834/1927629460,v1:172.21.15.47:6835/1927629460] conn(0x559b6ccc9b90 0x559b6cccbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab537fe640 1 --2- 172.21.15.47:0/3763704073 >> [v2:172.21.15.47:6834/1927629460,v1:172.21.15.47:6835/1927629460] conn(0x559b6ccc9b90 0x559b6cccbf80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=567736427f3b1603 server_cookie=a4649f8902bbee6b in_seq=0 out_seq=0 2024-03-23T09:47:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab327fc640 10 client.7671 ms_handle_connect on v2:172.21.15.47:6834/1927629460 2024-03-23T09:47:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab327fc640 1 -- 172.21.15.47:0/3763704073 <== mds.0 v2:172.21.15.47:6834/1927629460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559b6c8d26c0 con 0x559b6ccc9b90 2024-03-23T09:47:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab327fc640 10 client.7671 handle_command_reply: tid=0 2024-03-23T09:47:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab5ca41740 1 client.7671 shutdown 2024-03-23T09:47:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab5ca41740 1 -- 172.21.15.47:0/3763704073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6ccb1750 msgr2=0x559b6ccb31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab5ca41740 1 --2- 172.21.15.47:0/3763704073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6ccb1750 0x559b6ccb31e0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x559b6cc64120 tx=0x7fab480036f0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab5ca41740 1 -- 172.21.15.47:0/3763704073 shutdown_connections 2024-03-23T09:47:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab5ca41740 1 --2- 172.21.15.47:0/3763704073 >> [v2:172.21.15.47:6834/1927629460,v1:172.21.15.47:6835/1927629460] conn(0x559b6ccc9b90 0x559b6cccbf80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab5ca41740 1 --2- 172.21.15.47:0/3763704073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6cc75880 0x559b6ccafc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab5ca41740 1 --2- 172.21.15.47:0/3763704073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6ccb1750 0x559b6ccb31e0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab327fc640 0 client.7671 ms_handle_reset on v2:172.21.15.47:6834/1927629460 2024-03-23T09:47:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab5ca41740 1 --2- 172.21.15.47:0/3763704073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc677d0 0x559b6cc8a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.034+0000 7fab5ca41740 1 -- 172.21.15.47:0/3763704073 >> 172.21.15.47:0/3763704073 conn(0x559b6cc63c70 msgr2=0x559b6cc67f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.035+0000 7fab5ca41740 1 -- 172.21.15.47:0/3763704073 shutdown_connections 2024-03-23T09:47:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.035+0000 7fab5ca41740 1 -- 172.21.15.47:0/3763704073 wait complete. 2024-03-23T09:47:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.035+0000 7fab5ca41740 20 client.7671 trim_cache size 0 max 16384 2024-03-23T09:47:16.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.050+0000 7fab5ca41740 1 Processor -- start 2024-03-23T09:47:16.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.050+0000 7fab5ca41740 1 -- start start 2024-03-23T09:47:16.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab5ca41740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 0x559b6cc62a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab5ca41740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc75880 0x559b6cc7b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab5ca41740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6ccb1750 0x559b6cc7b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab5ca41740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6ccd08d0 con 0x559b6cc677d0 2024-03-23T09:47:16.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab5ca41740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6ccb42c0 con 0x559b6ccb1750 2024-03-23T09:47:16.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab5ca41740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b6ccd00d0 con 0x559b6cc75880 2024-03-23T09:47:16.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab537fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 0x559b6cc62a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab537fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 0x559b6cc62a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38496/0 (socket says 172.21.15.47:38496) 2024-03-23T09:47:16.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab537fe640 1 -- 172.21.15.47:0/1510760265 learned_addr learned my addr 172.21.15.47:0/1510760265 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:16.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab537fe640 1 -- 172.21.15.47:0/1510760265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc75880 msgr2=0x559b6cc7b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:16.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab537fe640 1 --2- 172.21.15.47:0/1510760265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc75880 0x559b6cc7b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab537fe640 1 -- 172.21.15.47:0/1510760265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6ccb1750 msgr2=0x559b6cc7b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:16.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab537fe640 1 --2- 172.21.15.47:0/1510760265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6ccb1750 0x559b6cc7b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab537fe640 1 -- 172.21.15.47:0/1510760265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b6cb6e430 con 0x559b6cc677d0 2024-03-23T09:47:16.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab537fe640 1 --2- 172.21.15.47:0/1510760265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 0x559b6cc62a60 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fab44087d70 tx=0x7fab4405b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e26fbe66a581f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:16.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab39ffb640 1 -- 172.21.15.47:0/1510760265 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44060ad0 con 0x559b6cc677d0 2024-03-23T09:47:16.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab39ffb640 1 -- 172.21.15.47:0/1510760265 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab44056820 con 0x559b6cc677d0 2024-03-23T09:47:16.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.051+0000 7fab39ffb640 1 -- 172.21.15.47:0/1510760265 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44056a80 con 0x559b6cc677d0 2024-03-23T09:47:16.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.052+0000 7fab5ca41740 1 -- 172.21.15.47:0/1510760265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 msgr2=0x559b6cc62a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.052+0000 7fab5ca41740 1 --2- 172.21.15.47:0/1510760265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 0x559b6cc62a60 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fab44087d70 tx=0x7fab4405b040 comp rx=0 tx=0).stop 2024-03-23T09:47:16.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.052+0000 7fab5ca41740 1 -- 172.21.15.47:0/1510760265 shutdown_connections 2024-03-23T09:47:16.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.052+0000 7fab5ca41740 1 --2- 172.21.15.47:0/1510760265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6ccb1750 0x559b6cc7b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.052+0000 7fab5ca41740 1 --2- 172.21.15.47:0/1510760265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6cc75880 0x559b6cc7b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.052+0000 7fab5ca41740 1 --2- 172.21.15.47:0/1510760265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc677d0 0x559b6cc62a60 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.052+0000 7fab5ca41740 1 -- 172.21.15.47:0/1510760265 >> 172.21.15.47:0/1510760265 conn(0x559b6cc63c70 msgr2=0x559b6ccad880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:16.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.052+0000 7fab5ca41740 1 -- 172.21.15.47:0/1510760265 shutdown_connections 2024-03-23T09:47:16.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.052+0000 7fab5ca41740 1 -- 172.21.15.47:0/1510760265 wait complete. 2024-03-23T09:47:16.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.053+0000 7fab5ca41740 1 -- start start 2024-03-23T09:47:16.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.053+0000 7fab5ca41740 1 Processor -- start 2024-03-23T09:47:16.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.053+0000 7fab5ca41740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6cc677d0 0x559b6cc89b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.053+0000 7fab5ca41740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc75880 0x559b6cd153d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.053+0000 7fab5ca41740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6ccb1750 0x559b6ccb1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.053+0000 7fab5ca41740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6cd15910 con 0x559b6cc75880 2024-03-23T09:47:16.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.053+0000 7fab5ca41740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6cc67ee0 con 0x559b6cc677d0 2024-03-23T09:47:16.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.053+0000 7fab5ca41740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b6cc686e0 con 0x559b6ccb1750 2024-03-23T09:47:16.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.053+0000 7fab5ca41740 10 client.0 _ll_register_callbacks cb 0x7fab48003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:47:16.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.053+0000 7fab52ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc75880 0x559b6cd153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab52ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc75880 0x559b6cd153d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38512/0 (socket says 172.21.15.47:38512) 2024-03-23T09:47:16.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab52ffd640 1 -- 172.21.15.47:0/3281275165 learned_addr learned my addr 172.21.15.47:0/3281275165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:16.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab537fe640 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6cc677d0 0x559b6cc89b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab53fff640 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6ccb1750 0x559b6ccb1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab52ffd640 1 -- 172.21.15.47:0/3281275165 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6ccb1750 msgr2=0x559b6ccb1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:16.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab52ffd640 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6ccb1750 0x559b6ccb1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab52ffd640 1 -- 172.21.15.47:0/3281275165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6cc677d0 msgr2=0x559b6cc89b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:16.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab52ffd640 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6cc677d0 0x559b6cc89b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab52ffd640 1 -- 172.21.15.47:0/3281275165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5414f970 con 0x559b6cc75880 2024-03-23T09:47:16.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab53fff640 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6ccb1750 0x559b6ccb1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:16.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab52ffd640 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc75880 0x559b6cd153d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fab48022d70 tx=0x7fab48003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9313330d56af0ccf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:16.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab327fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:47:16.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab327fc640 1 -- 172.21.15.47:0/3281275165 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab480052d0 con 0x559b6cc75880 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab327fc640 1 -- 172.21.15.47:0/3281275165 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab48021770 con 0x559b6cc75880 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab5ca41740 1 -- 172.21.15.47:0/3281275165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b6cc5f670 con 0x559b6cc75880 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab327fc640 1 -- 172.21.15.47:0/3281275165 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab48007b50 con 0x559b6cc75880 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab5ca41740 10 client.7677 fetch_fsmap learned FSMap version 46 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab5ca41740 1 -- 172.21.15.47:0/3281275165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x559b6cb93b40 con 0x559b6cc75880 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.054+0000 7fab327fc640 1 -- 172.21.15.47:0/3281275165 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559b6cc5f670 con 0x559b6cc75880 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab327fc640 1 -- 172.21.15.47:0/3281275165 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fab48038440 con 0x559b6cc75880 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 10 client.7677 fetch_fsmap finished waiting for FSMap version 46 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 10 client.7677 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 20 client.7677 populate_metadata read hostname 'smithi047' 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.47:6834/1927629460,v1:172.21.15.47:6835/1927629460] conn(0x559b6ccb2070 0x559b6cc90120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 4 client.7677 mds_command: new command op to 7524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 1 -- 172.21.15.47:0/3281275165 --> [v2:172.21.15.47:6834/1927629460,v1:172.21.15.47:6835/1927629460] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fab4800d6a0 con 0x559b6ccb2070 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab537fe640 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.47:6834/1927629460,v1:172.21.15.47:6835/1927629460] conn(0x559b6ccb2070 0x559b6cc90120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab327fc640 10 client.7677 ms_handle_connect on v2:172.21.15.47:6834/1927629460 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab537fe640 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.47:6834/1927629460,v1:172.21.15.47:6835/1927629460] conn(0x559b6ccb2070 0x559b6cc90120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56476f777eb513a server_cookie=5e805806fe266e8e in_seq=0 out_seq=0 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab327fc640 1 -- 172.21.15.47:0/3281275165 <== mds.0 v2:172.21.15.47:6834/1927629460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7fab4800d6a0 con 0x559b6ccb2070 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab327fc640 10 client.7677 handle_command_reply: tid=0 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 1 client.7677 shutdown 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 1 -- 172.21.15.47:0/3281275165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc75880 msgr2=0x559b6cd153d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc75880 0x559b6cd153d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fab48022d70 tx=0x7fab48003760 comp rx=0 tx=0).stop 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 1 -- 172.21.15.47:0/3281275165 shutdown_connections 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.47:6834/1927629460,v1:172.21.15.47:6835/1927629460] conn(0x559b6ccb2070 0x559b6cc90120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b6ccb1750 0x559b6ccb1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b6cc75880 0x559b6cd153d0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab327fc640 0 client.7677 ms_handle_reset on v2:172.21.15.47:6834/1927629460 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 1 --2- 172.21.15.47:0/3281275165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b6cc677d0 0x559b6cc89b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.055+0000 7fab5ca41740 1 -- 172.21.15.47:0/3281275165 >> 172.21.15.47:0/3281275165 conn(0x559b6cc63c70 msgr2=0x559b6ccb62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:16.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab5ca41740 1 -- 172.21.15.47:0/3281275165 shutdown_connections 2024-03-23T09:47:16.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab5ca41740 1 -- 172.21.15.47:0/3281275165 wait complete. 2024-03-23T09:47:16.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab5ca41740 20 client.7677 trim_cache size 0 max 16384 2024-03-23T09:47:16.065 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:47:16.065 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:47:16.065 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:47:16.065 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 7524, 2024-03-23T09:47:16.065 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:47:16.065 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:47:16.065 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:47:16.065 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 4.6643469770000001, 2024-03-23T09:47:16.066 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 46, 2024-03-23T09:47:16.066 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 85, 2024-03-23T09:47:16.066 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T09:47:16.066 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 9.0910763580000005 2024-03-23T09:47:16.066 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:47:16.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab59ece640 1 -- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fab4006ac00 msgr2=0x7fab4006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab59ece640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fab4006ac00 0x7fab4006d0c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fab540ccac0 tx=0x7fab48009040 comp rx=0 tx=0).stop 2024-03-23T09:47:16.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab59ece640 1 -- 172.21.15.47:0/1161915224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab540cf920 msgr2=0x7fab5414f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab59ece640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab540cf920 0x7fab5414f430 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fab4400d5f0 tx=0x7fab4400dab0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab59ece640 1 -- 172.21.15.47:0/1161915224 shutdown_connections 2024-03-23T09:47:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab59ece640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fab4006ac00 0x7fab4006d0c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab59ece640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab5415cd00 0x7fab5415ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab59ece640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fab540d1020 0x7fab5415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab59ece640 1 --2- 172.21.15.47:0/1161915224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab540cf920 0x7fab5414f430 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab59ece640 1 -- 172.21.15.47:0/1161915224 >> 172.21.15.47:0/1161915224 conn(0x7fab5406b160 msgr2=0x7fab5415dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab59ece640 1 -- 172.21.15.47:0/1161915224 shutdown_connections 2024-03-23T09:47:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.056+0000 7fab59ece640 1 -- 172.21.15.47:0/1161915224 wait complete. 2024-03-23T09:47:16.068 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:47:16.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.169+0000 7f59bf173640 1 Processor -- start 2024-03-23T09:47:16.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.169+0000 7f59bf173640 1 -- start start 2024-03-23T09:47:16.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.170+0000 7f59bf173640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59b80cf210 0x7f59b80cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.170+0000 7f59bf173640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b80cfbd0 con 0x7f59b80b5a90 2024-03-23T09:47:16.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.170+0000 7f59bf173640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b815d0e0 con 0x7f59b815cd00 2024-03-23T09:47:16.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.170+0000 7f59bf173640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b80d1540 con 0x7f59b80cf210 2024-03-23T09:47:16.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.170+0000 7f59b7fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59b80cf210 0x7f59b80cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.170+0000 7f59b7fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59b80cf210 0x7f59b80cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38528/0 (socket says 172.21.15.47:38528) 2024-03-23T09:47:16.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.170+0000 7f59bd710640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f59b80b5a90 0x7f59b80b5ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51320/0 (socket says 172.21.15.47:51320) 2024-03-23T09:47:16.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.170+0000 7f59b7fff640 1 -- 172.21.15.47:0/1807445048 learned_addr learned my addr 172.21.15.47:0/1807445048 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:16.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.170+0000 7f59b7fff640 1 -- 172.21.15.47:0/1807445048 >> v1:172.21.15.175:6789/0 conn(0x7f59b815cd00 legacy=0x7f59b80d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:16.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.170+0000 7f59b7fff640 1 -- 172.21.15.47:0/1807445048 >> v1:172.21.15.47:6789/0 conn(0x7f59b80b5a90 legacy=0x7f59b80b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.170+0000 7f59b7fff640 1 -- 172.21.15.47:0/1807445048 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b80d1820 con 0x7f59b80cf210 2024-03-23T09:47:16.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.171+0000 7f59b7fff640 1 --2- 172.21.15.47:0/1807445048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b80cf210 0x7f59b80cf690 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009870 tx=0x7f59ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c2dd99bfbaec1a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:16.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.171+0000 7f59b77fe640 1 -- 172.21.15.47:0/1807445048 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac031870 con 0x7f59b80cf210 2024-03-23T09:47:16.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.171+0000 7f59b77fe640 1 -- 172.21.15.47:0/1807445048 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac0319d0 con 0x7f59b80cf210 2024-03-23T09:47:16.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.171+0000 7f59b77fe640 1 -- 172.21.15.47:0/1807445048 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac031d60 con 0x7f59b80cf210 2024-03-23T09:47:16.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.172+0000 7f59bf173640 1 -- 172.21.15.47:0/1807445048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b80cf210 msgr2=0x7f59b80cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.172+0000 7f59bf173640 1 --2- 172.21.15.47:0/1807445048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b80cf210 0x7f59b80cf690 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009870 tx=0x7f59ac02b910 comp rx=0 tx=0).stop 2024-03-23T09:47:16.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.172+0000 7f59bf173640 1 -- 172.21.15.47:0/1807445048 shutdown_connections 2024-03-23T09:47:16.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.172+0000 7f59bf173640 1 --2- 172.21.15.47:0/1807445048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b80cf210 0x7f59b80cf690 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.172+0000 7f59bf173640 1 -- 172.21.15.47:0/1807445048 >> 172.21.15.47:0/1807445048 conn(0x7f59b806b160 msgr2=0x7f59b806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:16.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.172+0000 7f59bf173640 1 -- 172.21.15.47:0/1807445048 shutdown_connections 2024-03-23T09:47:16.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.172+0000 7f59bf173640 1 -- 172.21.15.47:0/1807445048 wait complete. 2024-03-23T09:47:16.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.173+0000 7f59bf173640 1 Processor -- start 2024-03-23T09:47:16.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.173+0000 7f59bf173640 1 -- start start 2024-03-23T09:47:16.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bf173640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b80b5a90 0x7f59b8159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bf173640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59b80cf210 0x7f59b80ab910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bf173640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59b815cd00 0x7f59b80a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bf173640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b80b2d90 con 0x7f59b80b5a90 2024-03-23T09:47:16.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bf173640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b80abe50 con 0x7f59b80cf210 2024-03-23T09:47:16.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bf173640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b80b2fe0 con 0x7f59b815cd00 2024-03-23T09:47:16.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bcf0f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b80b5a90 0x7f59b8159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bcf0f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b80b5a90 0x7f59b8159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38534/0 (socket says 172.21.15.47:38534) 2024-03-23T09:47:16.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bcf0f640 1 -- 172.21.15.47:0/3290371781 learned_addr learned my addr 172.21.15.47:0/3290371781 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59b7fff640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59b80cf210 0x7f59b80ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bd710640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59b815cd00 0x7f59b80a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bcf0f640 1 -- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59b815cd00 msgr2=0x7f59b80a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bcf0f640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59b815cd00 0x7f59b80a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bcf0f640 1 -- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59b80cf210 msgr2=0x7f59b80ab910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bcf0f640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59b80cf210 0x7f59b80ab910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bcf0f640 1 -- 172.21.15.47:0/3290371781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac0094d0 con 0x7f59b80b5a90 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59b7fff640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59b80cf210 0x7f59b80ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.174+0000 7f59bcf0f640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b80b5a90 0x7f59b8159400 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f59b000ba50 tx=0x7f59b000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.175+0000 7f59b57fa640 1 -- 172.21.15.47:0/3290371781 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b000cbe0 con 0x7f59b80b5a90 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.175+0000 7f59b57fa640 1 -- 172.21.15.47:0/3290371781 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b000cd80 con 0x7f59b80b5a90 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.175+0000 7f59b57fa640 1 -- 172.21.15.47:0/3290371781 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0014680 con 0x7f59b80b5a90 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.175+0000 7f59bf173640 1 -- 172.21.15.47:0/3290371781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b80b32c0 con 0x7f59b80b5a90 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.175+0000 7f59bf173640 1 -- 172.21.15.47:0/3290371781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b80b3880 con 0x7f59b80b5a90 2024-03-23T09:47:16.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.176+0000 7f59bf173640 1 -- 172.21.15.47:0/3290371781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b8072360 con 0x7f59b80b5a90 2024-03-23T09:47:16.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.176+0000 7f59b57fa640 1 -- 172.21.15.47:0/3290371781 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f59b00147e0 con 0x7f59b80b5a90 2024-03-23T09:47:16.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.176+0000 7f59b57fa640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f59a406ac30 0x7f59a406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:16.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.176+0000 7f59b57fa640 1 -- 172.21.15.47:0/3290371781 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59b0089180 con 0x7f59b80b5a90 2024-03-23T09:47:16.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.177+0000 7f59b7fff640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f59a406ac30 0x7f59a406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:16.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.177+0000 7f59b7fff640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f59a406ac30 0x7f59a406d0f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f59ac036000 tx=0x7f59ac03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:16.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.181+0000 7f59b57fa640 1 -- 172.21.15.47:0/3290371781 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f59b00544a0 con 0x7f59b80b5a90 2024-03-23T09:47:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.397+0000 7f59bf173640 1 -- 172.21.15.47:0/3290371781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f59b8066be0 con 0x7f59b80b5a90 2024-03-23T09:47:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.398+0000 7f59b57fa640 1 -- 172.21.15.47:0/3290371781 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f59b0057ac0 con 0x7f59b80b5a90 2024-03-23T09:47:16.398 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:47:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.399+0000 7f59bf173640 1 -- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f59a406ac30 msgr2=0x7f59a406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.399+0000 7f59bf173640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f59a406ac30 0x7f59a406d0f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f59ac036000 tx=0x7f59ac03c040 comp rx=0 tx=0).stop 2024-03-23T09:47:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.399+0000 7f59bf173640 1 -- 172.21.15.47:0/3290371781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b80b5a90 msgr2=0x7f59b8159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.399+0000 7f59bf173640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b80b5a90 0x7f59b8159400 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f59b000ba50 tx=0x7f59b000bf10 comp rx=0 tx=0).stop 2024-03-23T09:47:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.400+0000 7f59bf173640 1 -- 172.21.15.47:0/3290371781 shutdown_connections 2024-03-23T09:47:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.400+0000 7f59bf173640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f59a406ac30 0x7f59a406d0f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.400+0000 7f59bf173640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59b815cd00 0x7f59b80a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.400+0000 7f59bf173640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59b80cf210 0x7f59b80ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.400+0000 7f59bf173640 1 --2- 172.21.15.47:0/3290371781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b80b5a90 0x7f59b8159400 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.400+0000 7f59bf173640 1 -- 172.21.15.47:0/3290371781 >> 172.21.15.47:0/3290371781 conn(0x7f59b806b160 msgr2=0x7f59b815d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.400+0000 7f59bf173640 1 -- 172.21.15.47:0/3290371781 shutdown_connections 2024-03-23T09:47:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:16.400+0000 7f59bf173640 1 -- 172.21.15.47:0/3290371781 wait complete. 2024-03-23T09:47:16.410 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:47:16.410 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:47:16.410 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:47:16.411 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:47:16.411 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:47:16.411 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:47:16.411 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:47:16.411 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:47:16.411 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:47:16.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:16.411 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:47:16.426 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:47:16.427 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:47:16.427 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:47:16.427 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:16.427 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:47:16.427 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:16.427 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:47:16.427 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:47:16.427 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:47:16.427 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:47:16.427 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:47:16.427 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:47:16.427 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:47:16.427 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:47:16.428 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:47:16.428 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:47:16.428 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42302sec preferred_lft 42302sec 2024-03-23T09:47:16.428 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:47:16.428 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:16.428 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:47:16.428 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:47:16.428 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:47:16.428 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:47:16.428 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:47:16.428 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:47:16.429 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:16.429 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:47:16.429 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:16.429 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:47:16.429 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:47:16.429 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:47:16.429 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:16.429 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:47:16.429 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:47:16.429 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:47:16.429 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:16.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:16.430 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:47:16.492 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:47:16.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:16.493 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:47:16.549 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:47:16.549 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:47:16.549 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:47:16.549 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:47:16.549 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:47:16.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:16.550 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:47:16.550 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:47:16.550 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:47:16.550 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:47:16.550 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:47:16.550 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:47:16.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:16.644 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:47:16.658 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:47:16.658 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:47:16.659 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:47:16.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:16.659 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:47:16.725 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:47:16.726 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:47:16.726 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:47:16.780 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:47:16.781 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:47:16.781 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:47:16.835 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:47:16.901 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:47:16.901 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:47:16.967 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:47:16.968 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:47:16.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:16.969 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:47:17.023 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:47:17.023 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:47:17.065 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:47:17.091 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:47:17.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:47:17.150 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:47:17.150+0000 7f835e438480 -1 init, newargv = 0x560071838ca0 newargc=15 2024-03-23T09:47:17.150 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[67189]: starting ceph client 2024-03-23T09:47:17.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[67189]: starting fuse 2024-03-23T09:47:17.161 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:47:17.162 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:47:17.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:17.162 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:47:17.217 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:47:17.217 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:47:17.217 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:47:17.217 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:47:17.217 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:47:17.218 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:47:17.219 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:47:17.219 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:47:17.219 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:47:17.219 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:47:17.219 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:47:17.219 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:47:17.219 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:47:17.219 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:47:17.290 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:47:17.290+0000 7f245cad0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:47:17.333 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.67189.asok 2024-03-23T09:47:17.519 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.67189.asok 2024-03-23T09:47:17.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:17.519 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.67189.asok status 2024-03-23T09:47:17.656 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:47:17.656 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:47:17.656 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:47:17.656 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:47:17.656 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:47:17.656 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:47:17.656 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:47:17.656 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "67189", 2024-03-23T09:47:17.656 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 7695, 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7695 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 247773926 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:17.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:47:17.658 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:47:17.658 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:47:17.658 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 247773926 2024-03-23T09:47:17.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:17.658 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.7695 192.168.0.1:0/247773926", 2024-03-23T09:47:17.658 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/247773926", 2024-03-23T09:47:17.658 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:47:17.658 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 46, 2024-03-23T09:47:17.658 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 85, 2024-03-23T09:47:17.658 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:47:17.658 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:47:17.658 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:47:17.658 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:47:17.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:17.664 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:47:17.664 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:47:17.678 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:47:17.679 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:47:17.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:17.679 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:47:17.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:17.745 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-03-23T09:47:17.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:17.804 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-03-23T09:47:17.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:17.861 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-03-23T09:47:17.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:17.918 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-03-23T09:47:17.977 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:47:17.977 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:47:17.977 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:47:17.977 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:47:17.978 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-03-23T09:47:17.978 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:47:18.080 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627776 2024-03-23T09:47:18.269 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:47:18.270 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:47:18.325 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:47:18.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.467+0000 7fceae34e640 1 Processor -- start 2024-03-23T09:47:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.467+0000 7fceae34e640 1 -- start start 2024-03-23T09:47:18.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.468+0000 7fceae34e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcea80ccb80 0x7fcea80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.468+0000 7fceae34e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea815ae50 con 0x7fcea80cd7f0 2024-03-23T09:47:18.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.468+0000 7fceae34e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea80cf010 con 0x7fcea815aa70 2024-03-23T09:47:18.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.468+0000 7fceae34e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea80cf2f0 con 0x7fcea80ccb80 2024-03-23T09:47:18.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.468+0000 7fcea77fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcea80cd7f0 0x7fcea80ce2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51326/0 (socket says 172.21.15.47:51326) 2024-03-23T09:47:18.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.468+0000 7fcea77fe640 1 -- 172.21.15.47:0/4021789469 learned_addr learned my addr 172.21.15.47:0/4021789469 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:18.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fceac8eb640 1 --2- 172.21.15.47:0/4021789469 >> v2:172.21.15.175:3300/0 conn(0x7fcea80ccb80 0x7fcea80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693331307 0 0) 0x7fcea815ae50 con 0x7fcea80cd7f0 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce90003520 con 0x7fcea80cd7f0 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 515384299 0 0) 0x7fce90003520 con 0x7fcea80cd7f0 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 >> v2:172.21.15.175:3300/0 conn(0x7fcea80ccb80 msgr2=0x7fcea80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 --2- 172.21.15.47:0/4021789469 >> v2:172.21.15.175:3300/0 conn(0x7fcea80ccb80 0x7fcea80ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 >> v1:172.21.15.175:6789/0 conn(0x7fcea815aa70 legacy=0x7fcea80cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea80cf5d0 con 0x7fcea80cd7f0 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fce9c001690 con 0x7fcea80cd7f0 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fceac8eb640 1 --2- 172.21.15.47:0/4021789469 >> v2:172.21.15.175:3300/0 conn(0x7fcea80ccb80 0x7fcea80ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 >> v1:172.21.15.47:6789/0 conn(0x7fcea80cd7f0 legacy=0x7fcea80ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 --2- 172.21.15.47:0/4021789469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce90003970 0x7fce90003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 --2- 172.21.15.47:0/4021789469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce900047c0 0x7fce90004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 --2- 172.21.15.47:0/4021789469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce90005180 0x7fce9000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea80cf2f0 con 0x7fce900047c0 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9000c030 con 0x7fce90003970 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9000c1f0 con 0x7fce90005180 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea77fe640 1 --2- 172.21.15.47:0/4021789469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce900047c0 0x7fce90004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea7fff640 1 --2- 172.21.15.47:0/4021789469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce90003970 0x7fce90003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fceac8eb640 1 --2- 172.21.15.47:0/4021789469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce90005180 0x7fce9000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea77fe640 1 -- 172.21.15.47:0/4021789469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce90005180 msgr2=0x7fce9000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea77fe640 1 --2- 172.21.15.47:0/4021789469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce90005180 0x7fce9000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea77fe640 1 -- 172.21.15.47:0/4021789469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce90003970 msgr2=0x7fce90003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea77fe640 1 --2- 172.21.15.47:0/4021789469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce90003970 0x7fce90003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fcea77fe640 1 -- 172.21.15.47:0/4021789469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce90003520 con 0x7fce900047c0 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.469+0000 7fceac8eb640 1 --2- 172.21.15.47:0/4021789469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce90005180 0x7fce9000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.470+0000 7fceac8eb640 1 -- 172.21.15.47:0/4021789469 reap_dead start 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.470+0000 7fcea77fe640 1 --2- 172.21.15.47:0/4021789469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce900047c0 0x7fce90004c40 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00c210 tx=0x7fce900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abed88ae3f9d49d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.470+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c005ad0 con 0x7fce900047c0 2024-03-23T09:47:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.470+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c005c30 con 0x7fce900047c0 2024-03-23T09:47:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.470+0000 7fcea6ffd640 1 -- 172.21.15.47:0/4021789469 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c02c810 con 0x7fce900047c0 2024-03-23T09:47:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.471+0000 7fceae34e640 1 -- 172.21.15.47:0/4021789469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce900047c0 msgr2=0x7fce90004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.471+0000 7fceae34e640 1 --2- 172.21.15.47:0/4021789469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce900047c0 0x7fce90004c40 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00c210 tx=0x7fce900057f0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.471+0000 7fceae34e640 1 -- 172.21.15.47:0/4021789469 shutdown_connections 2024-03-23T09:47:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.471+0000 7fceae34e640 1 --2- 172.21.15.47:0/4021789469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce900047c0 0x7fce90004c40 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.471+0000 7fceae34e640 1 -- 172.21.15.47:0/4021789469 >> 172.21.15.47:0/4021789469 conn(0x7fcea80d06f0 msgr2=0x7fcea80d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.471+0000 7fceae34e640 1 -- 172.21.15.47:0/4021789469 shutdown_connections 2024-03-23T09:47:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.471+0000 7fceae34e640 1 -- 172.21.15.47:0/4021789469 wait complete. 2024-03-23T09:47:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.472+0000 7fceae34e640 1 Processor -- start 2024-03-23T09:47:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.472+0000 7fceae34e640 1 -- start start 2024-03-23T09:47:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fceae34e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce900047c0 0x7fcea80c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fceae34e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcea8150b00 0x7fcea815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fceae34e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcea815bf90 0x7fcea81f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fceae34e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea8150f80 con 0x7fce900047c0 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fceae34e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea815de40 con 0x7fcea815bf90 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fceae34e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea815c410 con 0x7fcea8150b00 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fcea7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce900047c0 0x7fcea80c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fcea7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce900047c0 0x7fcea80c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38560/0 (socket says 172.21.15.47:38560) 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fcea7fff640 1 -- 172.21.15.47:0/553021358 learned_addr learned my addr 172.21.15.47:0/553021358 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fcea77fe640 1 --2- 172.21.15.47:0/553021358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcea8150b00 0x7fcea815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fcea7fff640 1 -- 172.21.15.47:0/553021358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcea8150b00 msgr2=0x7fcea815ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fcea7fff640 1 --2- 172.21.15.47:0/553021358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcea8150b00 0x7fcea815ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fcea7fff640 1 -- 172.21.15.47:0/553021358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcea815bf90 msgr2=0x7fcea81f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fcea7fff640 1 --2- 172.21.15.47:0/553021358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcea815bf90 0x7fcea81f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fcea7fff640 1 -- 172.21.15.47:0/553021358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea80c9910 con 0x7fce900047c0 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.473+0000 7fcea7fff640 1 --2- 172.21.15.47:0/553021358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce900047c0 0x7fcea80c9da0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fce9800f5f0 tx=0x7fce9800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.474+0000 7fcea4ff9640 1 -- 172.21.15.47:0/553021358 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce980052e0 con 0x7fce900047c0 2024-03-23T09:47:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.474+0000 7fcea4ff9640 1 -- 172.21.15.47:0/553021358 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce98005440 con 0x7fce900047c0 2024-03-23T09:47:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.474+0000 7fcea4ff9640 1 -- 172.21.15.47:0/553021358 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98003810 con 0x7fce900047c0 2024-03-23T09:47:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.474+0000 7fceae34e640 1 -- 172.21.15.47:0/553021358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea806b5b0 con 0x7fce900047c0 2024-03-23T09:47:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.474+0000 7fceae34e640 1 -- 172.21.15.47:0/553021358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea80c92c0 con 0x7fce900047c0 2024-03-23T09:47:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.475+0000 7fceae34e640 1 -- 172.21.15.47:0/553021358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea8066be0 con 0x7fce900047c0 2024-03-23T09:47:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.475+0000 7fcea4ff9640 1 -- 172.21.15.47:0/553021358 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fce98003970 con 0x7fce900047c0 2024-03-23T09:47:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.475+0000 7fcea4ff9640 1 --2- 172.21.15.47:0/553021358 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce9406ac00 0x7fce9406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.476+0000 7fcea4ff9640 1 -- 172.21.15.47:0/553021358 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce98089110 con 0x7fce900047c0 2024-03-23T09:47:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.476+0000 7fcea77fe640 1 --2- 172.21.15.47:0/553021358 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce9406ac00 0x7fce9406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.476+0000 7fcea77fe640 1 --2- 172.21.15.47:0/553021358 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce9406ac00 0x7fce9406d0c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fce9c034000 tx=0x7fce9c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.480+0000 7fcea4ff9640 1 -- 172.21.15.47:0/553021358 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fce98054380 con 0x7fce900047c0 2024-03-23T09:47:18.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.659+0000 7fceae34e640 1 -- 172.21.15.47:0/553021358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcea8151d30 con 0x7fce900047c0 2024-03-23T09:47:18.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.660+0000 7fcea4ff9640 1 -- 172.21.15.47:0/553021358 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16092 (secure 0 0 0) 0x7fce980579a0 con 0x7fce900047c0 2024-03-23T09:47:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:18.660 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":85,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:47:12.392230+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:47:09.722391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-23T09:47:10.354054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:47:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.662+0000 7fceae34e640 1 -- 172.21.15.47:0/553021358 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce9406ac00 msgr2=0x7fce9406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.662+0000 7fceae34e640 1 --2- 172.21.15.47:0/553021358 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce9406ac00 0x7fce9406d0c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fce9c034000 tx=0x7fce9c009030 comp rx=0 tx=0).stop 2024-03-23T09:47:18.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.662+0000 7fceae34e640 1 -- 172.21.15.47:0/553021358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce900047c0 msgr2=0x7fcea80c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.662+0000 7fceae34e640 1 --2- 172.21.15.47:0/553021358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce900047c0 0x7fcea80c9da0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fce9800f5f0 tx=0x7fce9800fab0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.662+0000 7fceae34e640 1 -- 172.21.15.47:0/553021358 shutdown_connections 2024-03-23T09:47:18.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.662+0000 7fceae34e640 1 --2- 172.21.15.47:0/553021358 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fce9406ac00 0x7fce9406d0c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.662+0000 7fceae34e640 1 --2- 172.21.15.47:0/553021358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcea815bf90 0x7fcea81f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.662+0000 7fceae34e640 1 --2- 172.21.15.47:0/553021358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcea8150b00 0x7fcea815ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.662+0000 7fceae34e640 1 --2- 172.21.15.47:0/553021358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce900047c0 0x7fcea80c9da0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.662+0000 7fceae34e640 1 -- 172.21.15.47:0/553021358 >> 172.21.15.47:0/553021358 conn(0x7fcea80d06f0 msgr2=0x7fcea8158180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:18.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.662+0000 7fceae34e640 1 -- 172.21.15.47:0/553021358 shutdown_connections 2024-03-23T09:47:18.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.662+0000 7fceae34e640 1 -- 172.21.15.47:0/553021358 wait complete. 2024-03-23T09:47:18.673 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:47:09.722391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:47:10.354054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:47:18.673 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:47:18.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.777+0000 7f5204899640 1 Processor -- start 2024-03-23T09:47:18.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.777+0000 7f5204899640 1 -- start start 2024-03-23T09:47:18.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.778+0000 7f5204899640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520015c990 0x7f5200160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.778+0000 7f5204899640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5200161630 con 0x7f52000b8350 2024-03-23T09:47:18.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.778+0000 7f5204899640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52000d1030 con 0x7f52000cde10 2024-03-23T09:47:18.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.778+0000 7f5204899640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52000cfe10 con 0x7f520015c990 2024-03-23T09:47:18.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.778+0000 7f51fdd9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f52000b8350 0x7f52000cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51336/0 (socket says 172.21.15.47:51336) 2024-03-23T09:47:18.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.778+0000 7f51fdd9b640 1 -- 172.21.15.47:0/454398073 learned_addr learned my addr 172.21.15.47:0/454398073 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.778+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121281079 0 0) 0x7f5200161630 con 0x7f52000b8350 2024-03-23T09:47:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.778+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51ec003520 con 0x7f52000b8350 2024-03-23T09:47:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.778+0000 7f51fe59c640 1 --2- 172.21.15.47:0/454398073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520015c990 0x7f5200160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1408567339 0 0) 0x7f51ec003520 con 0x7f52000b8350 2024-03-23T09:47:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520015c990 msgr2=0x7f5200160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fd59a640 1 --2- 172.21.15.47:0/454398073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520015c990 0x7f5200160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 >> v1:172.21.15.175:6789/0 conn(0x7f52000cde10 legacy=0x7f52000d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52000b5b60 con 0x7f52000b8350 2024-03-23T09:47:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f51e8001670 con 0x7f52000b8350 2024-03-23T09:47:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 >> v1:172.21.15.47:6789/0 conn(0x7f52000b8350 legacy=0x7f52000cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fd59a640 1 --2- 172.21.15.47:0/454398073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520015c990 0x7f51ec0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fd59a640 1 --2- 172.21.15.47:0/454398073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51ec003fd0 0x7f51ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fd59a640 1 --2- 172.21.15.47:0/454398073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec004d00 0x7f51ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52000cfe10 con 0x7f51ec003fd0 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec00bbc0 con 0x7f51ec004d00 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec00bd80 con 0x7f520015c990 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fdd9b640 1 --2- 172.21.15.47:0/454398073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51ec003fd0 0x7f51ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fe59c640 1 --2- 172.21.15.47:0/454398073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520015c990 0x7f51ec0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fed9d640 1 --2- 172.21.15.47:0/454398073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec004d00 0x7f51ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fdd9b640 1 -- 172.21.15.47:0/454398073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520015c990 msgr2=0x7f51ec0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fdd9b640 1 --2- 172.21.15.47:0/454398073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520015c990 0x7f51ec0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fdd9b640 1 -- 172.21.15.47:0/454398073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec004d00 msgr2=0x7f51ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fdd9b640 1 --2- 172.21.15.47:0/454398073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec004d00 0x7f51ec005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fdd9b640 1 -- 172.21.15.47:0/454398073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec003520 con 0x7f51ec003fd0 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.779+0000 7f51fed9d640 1 --2- 172.21.15.47:0/454398073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec004d00 0x7f51ec005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.780+0000 7f51fdd9b640 1 --2- 172.21.15.47:0/454398073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51ec003fd0 0x7f51ec0047c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f51e800c240 tx=0x7f51ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d54b24891dcd56 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.780+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8005ad0 con 0x7f51ec003fd0 2024-03-23T09:47:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.780+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e8005c30 con 0x7f51ec003fd0 2024-03-23T09:47:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.780+0000 7f51fd59a640 1 -- 172.21.15.47:0/454398073 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e802caf0 con 0x7f51ec003fd0 2024-03-23T09:47:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.781+0000 7f5204899640 1 -- 172.21.15.47:0/454398073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51ec003fd0 msgr2=0x7f51ec0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.781+0000 7f5204899640 1 --2- 172.21.15.47:0/454398073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51ec003fd0 0x7f51ec0047c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f51e800c240 tx=0x7f51ec0057f0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.781+0000 7f51fed9d640 1 -- 172.21.15.47:0/454398073 reap_dead start 2024-03-23T09:47:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.781+0000 7f5204899640 1 -- 172.21.15.47:0/454398073 shutdown_connections 2024-03-23T09:47:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.781+0000 7f5204899640 1 -- 172.21.15.47:0/454398073 >> 172.21.15.47:0/454398073 conn(0x7f52000659d0 msgr2=0x7f5200065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.781+0000 7f5204899640 1 -- 172.21.15.47:0/454398073 shutdown_connections 2024-03-23T09:47:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.781+0000 7f5204899640 1 -- 172.21.15.47:0/454398073 wait complete. 2024-03-23T09:47:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.782+0000 7f5204899640 1 Processor -- start 2024-03-23T09:47:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.782+0000 7f5204899640 1 -- start start 2024-03-23T09:47:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f5204899640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec003fd0 0x7f52000c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f5204899640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52000b28f0 0x7f5200153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f5204899640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52000b2d70 0x7f52001544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f5204899640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52001601e0 con 0x7f52000b28f0 2024-03-23T09:47:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f5204899640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52000c7400 con 0x7f51ec003fd0 2024-03-23T09:47:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f5204899640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52000c7ca0 con 0x7f52000b2d70 2024-03-23T09:47:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f51fdd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52000b28f0 0x7f5200153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f51fdd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52000b28f0 0x7f5200153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38586/0 (socket says 172.21.15.47:38586) 2024-03-23T09:47:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f51fdd9b640 1 -- 172.21.15.47:0/802701509 learned_addr learned my addr 172.21.15.47:0/802701509 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f51fe59c640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec003fd0 0x7f52000c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f51fed9d640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52000b2d70 0x7f52001544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f51fdd9b640 1 -- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52000b2d70 msgr2=0x7f52001544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f51fdd9b640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52000b2d70 0x7f52001544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f51fdd9b640 1 -- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec003fd0 msgr2=0x7f52000c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f51fdd9b640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec003fd0 0x7f52000c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.783+0000 7f51fdd9b640 1 -- 172.21.15.47:0/802701509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520015ff00 con 0x7f52000b28f0 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.784+0000 7f51fdd9b640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52000b28f0 0x7f5200153fa0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f51e800c240 tx=0x7f51e8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.784+0000 7f51f6ffd640 1 -- 172.21.15.47:0/802701509 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8039070 con 0x7f52000b28f0 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.784+0000 7f51f6ffd640 1 -- 172.21.15.47:0/802701509 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e802ccc0 con 0x7f52000b28f0 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.784+0000 7f5204899640 1 -- 172.21.15.47:0/802701509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52000c7650 con 0x7f52000b28f0 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.784+0000 7f51f6ffd640 1 -- 172.21.15.47:0/802701509 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e803c040 con 0x7f52000b28f0 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.784+0000 7f5204899640 1 -- 172.21.15.47:0/802701509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520015f1d0 con 0x7f52000b28f0 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.785+0000 7f5204899640 1 -- 172.21.15.47:0/802701509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5200072360 con 0x7f52000b28f0 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.785+0000 7f51f6ffd640 1 -- 172.21.15.47:0/802701509 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f51e8032070 con 0x7f52000b28f0 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.785+0000 7f51f6ffd640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f51e006ac00 0x7f51e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.785+0000 7f51f6ffd640 1 -- 172.21.15.47:0/802701509 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51e80aa590 con 0x7f52000b28f0 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.786+0000 7f51fe59c640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f51e006ac00 0x7f51e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.786+0000 7f51fe59c640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f51e006ac00 0x7f51e006d0c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f51f0003b30 tx=0x7f51f0009760 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:18.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.790+0000 7f51f6ffd640 1 -- 172.21.15.47:0/802701509 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f51e8075820 con 0x7f52000b28f0 2024-03-23T09:47:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.968+0000 7f5204899640 1 -- 172.21.15.47:0/802701509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5200066be0 con 0x7f52000b28f0 2024-03-23T09:47:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.968+0000 7f51f6ffd640 1 -- 172.21.15.47:0/802701509 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51e8078e40 con 0x7f52000b28f0 2024-03-23T09:47:18.968 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:47:18.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.970+0000 7f5204899640 1 -- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f51e006ac00 msgr2=0x7f51e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.970+0000 7f5204899640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f51e006ac00 0x7f51e006d0c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f51f0003b30 tx=0x7f51f0009760 comp rx=0 tx=0).stop 2024-03-23T09:47:18.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.970+0000 7f5204899640 1 -- 172.21.15.47:0/802701509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52000b28f0 msgr2=0x7f5200153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:18.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.970+0000 7f5204899640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52000b28f0 0x7f5200153fa0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f51e800c240 tx=0x7f51e8001a00 comp rx=0 tx=0).stop 2024-03-23T09:47:18.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.970+0000 7f5204899640 1 -- 172.21.15.47:0/802701509 shutdown_connections 2024-03-23T09:47:18.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.970+0000 7f5204899640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f51e006ac00 0x7f51e006d0c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.970+0000 7f5204899640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52000b2d70 0x7f52001544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.970+0000 7f5204899640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52000b28f0 0x7f5200153fa0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.970+0000 7f5204899640 1 --2- 172.21.15.47:0/802701509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec003fd0 0x7f52000c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.970+0000 7f5204899640 1 -- 172.21.15.47:0/802701509 >> 172.21.15.47:0/802701509 conn(0x7f52000659d0 msgr2=0x7f52000b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.970+0000 7f5204899640 1 -- 172.21.15.47:0/802701509 shutdown_connections 2024-03-23T09:47:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:18.970+0000 7f5204899640 1 -- 172.21.15.47:0/802701509 wait complete. 2024-03-23T09:47:18.980 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:47:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.086+0000 7fc8fc8f3640 1 Processor -- start 2024-03-23T09:47:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.086+0000 7fc8fc8f3640 1 -- start start 2024-03-23T09:47:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8fc8f3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8f815ce10 0x7fc8f80cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8fc8f3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8f80b85d0 0x7fc8f80cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8fc8f3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f8161600 con 0x7fc8f815ca30 2024-03-23T09:47:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8fc8f3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f8160e50 con 0x7fc8f815ce10 2024-03-23T09:47:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8fc8f3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f80d25e0 con 0x7fc8f80b85d0 2024-03-23T09:47:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8f5d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8f815ce10 0x7fc8f80cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8f5d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8f815ce10 0x7fc8f80cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38596/0 (socket says 172.21.15.47:38596) 2024-03-23T09:47:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8f5d9b640 1 -- 172.21.15.47:0/170995840 learned_addr learned my addr 172.21.15.47:0/170995840 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8f6d9d640 1 --2- 172.21.15.47:0/170995840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8f80b85d0 0x7fc8f80cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8f5d9b640 1 -- 172.21.15.47:0/170995840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8f80b85d0 msgr2=0x7fc8f80cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8f5d9b640 1 --2- 172.21.15.47:0/170995840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8f80b85d0 0x7fc8f80cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8f5d9b640 1 -- 172.21.15.47:0/170995840 >> v1:172.21.15.175:6789/0 conn(0x7fc8f815ca30 legacy=0x7fc8f8160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8f5d9b640 1 -- 172.21.15.47:0/170995840 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f8161040 con 0x7fc8f815ce10 2024-03-23T09:47:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.087+0000 7fc8f6d9d640 1 --2- 172.21.15.47:0/170995840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8f80b85d0 0x7fc8f80cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:19.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.088+0000 7fc8f5d9b640 1 --2- 172.21.15.47:0/170995840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f815ce10 0x7fc8f80cd730 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009870 tx=0x7fc8ec02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e2993a67ebb01c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:19.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.088+0000 7fc8f559a640 1 -- 172.21.15.47:0/170995840 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec030840 con 0x7fc8f815ce10 2024-03-23T09:47:19.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.088+0000 7fc8f559a640 1 -- 172.21.15.47:0/170995840 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ec0309a0 con 0x7fc8f815ce10 2024-03-23T09:47:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.088+0000 7fc8f559a640 1 -- 172.21.15.47:0/170995840 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec030d30 con 0x7fc8f815ce10 2024-03-23T09:47:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.089+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/170995840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f815ce10 msgr2=0x7fc8f80cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.089+0000 7fc8fc8f3640 1 --2- 172.21.15.47:0/170995840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f815ce10 0x7fc8f80cd730 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009870 tx=0x7fc8ec02b980 comp rx=0 tx=0).stop 2024-03-23T09:47:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.089+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/170995840 shutdown_connections 2024-03-23T09:47:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.089+0000 7fc8fc8f3640 1 --2- 172.21.15.47:0/170995840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8f80b85d0 0x7fc8f80cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.089+0000 7fc8fc8f3640 1 --2- 172.21.15.47:0/170995840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f815ce10 0x7fc8f80cd730 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.089+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/170995840 >> 172.21.15.47:0/170995840 conn(0x7fc8f806b160 msgr2=0x7fc8f806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.089+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/170995840 shutdown_connections 2024-03-23T09:47:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.089+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/170995840 wait complete. 2024-03-23T09:47:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.090+0000 7fc8fc8f3640 1 Processor -- start 2024-03-23T09:47:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.090+0000 7fc8fc8f3640 1 -- start start 2024-03-23T09:47:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8fc8f3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8f80b85d0 0x7fc8f815d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8fc8f3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8f815ca30 0x7fc8f8150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8fc8f3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f815ce10 0x7fc8f80cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8fc8f3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f80caa90 con 0x7fc8f815ce10 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8fc8f3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f80cc6d0 con 0x7fc8f80b85d0 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8fc8f3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f8151320 con 0x7fc8f815ca30 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8f6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f815ce10 0x7fc8f80cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8f6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f815ce10 0x7fc8f80cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38610/0 (socket says 172.21.15.47:38610) 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8f6d9d640 1 -- 172.21.15.47:0/1054131373 learned_addr learned my addr 172.21.15.47:0/1054131373 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8f5d9b640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8f815ca30 0x7fc8f8150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8f659c640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8f80b85d0 0x7fc8f815d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8f6d9d640 1 -- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8f815ca30 msgr2=0x7fc8f8150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8f6d9d640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8f815ca30 0x7fc8f8150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8f6d9d640 1 -- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8f80b85d0 msgr2=0x7fc8f815d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8f6d9d640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8f80b85d0 0x7fc8f815d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.091+0000 7fc8f6d9d640 1 -- 172.21.15.47:0/1054131373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec0094d0 con 0x7fc8f815ce10 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.092+0000 7fc8f6d9d640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f815ce10 0x7fc8f80cb310 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8007f20 tx=0x7fc8e800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.092+0000 7fc8e6ffd640 1 -- 172.21.15.47:0/1054131373 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e800dee0 con 0x7fc8f815ce10 2024-03-23T09:47:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.092+0000 7fc8e6ffd640 1 -- 172.21.15.47:0/1054131373 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e800f040 con 0x7fc8f815ce10 2024-03-23T09:47:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.092+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/1054131373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f80c9e10 con 0x7fc8f815ce10 2024-03-23T09:47:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.092+0000 7fc8e6ffd640 1 -- 172.21.15.47:0/1054131373 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8003dd0 con 0x7fc8f815ce10 2024-03-23T09:47:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.092+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/1054131373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f80ca3d0 con 0x7fc8f815ce10 2024-03-23T09:47:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.093+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/1054131373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f8072360 con 0x7fc8f815ce10 2024-03-23T09:47:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.093+0000 7fc8e6ffd640 1 -- 172.21.15.47:0/1054131373 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc8e8004030 con 0x7fc8f815ce10 2024-03-23T09:47:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.094+0000 7fc8e6ffd640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc8d806ac00 0x7fc8d806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.094+0000 7fc8e6ffd640 1 -- 172.21.15.47:0/1054131373 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8e80896c0 con 0x7fc8f815ce10 2024-03-23T09:47:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.094+0000 7fc8f659c640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc8d806ac00 0x7fc8d806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.094+0000 7fc8f659c640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc8d806ac00 0x7fc8d806d0c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80cb2a0 tx=0x7fc8f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.098+0000 7fc8e6ffd640 1 -- 172.21.15.47:0/1054131373 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc8e80548a0 con 0x7fc8f815ce10 2024-03-23T09:47:19.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.275+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/1054131373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8f80c6490 con 0x7fc8f815ce10 2024-03-23T09:47:19.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.275+0000 7fc8e6ffd640 1 -- 172.21.15.47:0/1054131373 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc8f80c6490 con 0x7fc8f815ce10 2024-03-23T09:47:19.276 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:47:19.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.277+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc8d806ac00 msgr2=0x7fc8d806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.278+0000 7fc8fc8f3640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc8d806ac00 0x7fc8d806d0c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80cb2a0 tx=0x7fc8f0009040 comp rx=0 tx=0).stop 2024-03-23T09:47:19.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.278+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/1054131373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f815ce10 msgr2=0x7fc8f80cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.278+0000 7fc8fc8f3640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f815ce10 0x7fc8f80cb310 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8007f20 tx=0x7fc8e800d560 comp rx=0 tx=0).stop 2024-03-23T09:47:19.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.278+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/1054131373 shutdown_connections 2024-03-23T09:47:19.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.278+0000 7fc8fc8f3640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc8d806ac00 0x7fc8d806d0c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.278+0000 7fc8fc8f3640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f815ce10 0x7fc8f80cb310 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.278+0000 7fc8fc8f3640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8f815ca30 0x7fc8f8150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.278+0000 7fc8fc8f3640 1 --2- 172.21.15.47:0/1054131373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8f80b85d0 0x7fc8f815d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.278+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/1054131373 >> 172.21.15.47:0/1054131373 conn(0x7fc8f806b160 msgr2=0x7fc8f8154250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:19.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.278+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/1054131373 shutdown_connections 2024-03-23T09:47:19.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.278+0000 7fc8fc8f3640 1 -- 172.21.15.47:0/1054131373 wait complete. 2024-03-23T09:47:19.288 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:47:19.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.391+0000 7f3c2a626640 1 Processor -- start 2024-03-23T09:47:19.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.391+0000 7f3c2a626640 1 -- start start 2024-03-23T09:47:19.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.392+0000 7f3c2a626640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c2415ce10 0x7f3c240cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.392+0000 7f3c2a626640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3c240b85d0 0x7f3c240cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.392+0000 7f3c2a626640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c24161600 con 0x7f3c2415ca30 2024-03-23T09:47:19.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.392+0000 7f3c2a626640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c24160e50 con 0x7f3c240b85d0 2024-03-23T09:47:19.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.392+0000 7f3c2a626640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c240d25e0 con 0x7f3c2415ce10 2024-03-23T09:47:19.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.392+0000 7f3c237fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c2415ce10 0x7f3c240cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.392+0000 7f3c237fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c2415ce10 0x7f3c240cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:58862/0 (socket says 172.21.15.47:58862) 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.392+0000 7f3c237fe640 1 -- 172.21.15.47:0/1102520481 learned_addr learned my addr 172.21.15.47:0/1102520481 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.392+0000 7f3c23fff640 1 --1- 172.21.15.47:0/1102520481 >> v1:172.21.15.175:6789/0 conn(0x7f3c2415ca30 0x7f3c24160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:49836/0 (socket says 172.21.15.47:49836) 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.392+0000 7f3c28bc3640 1 --2- 172.21.15.47:0/1102520481 >> v2:172.21.15.175:3300/0 conn(0x7f3c240b85d0 0x7f3c240cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.393+0000 7f3c22ffd640 1 -- 172.21.15.47:0/1102520481 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330979647 0 0) 0x7f3c24161600 con 0x7f3c2415ca30 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.393+0000 7f3c22ffd640 1 -- 172.21.15.47:0/1102520481 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c0c003520 con 0x7f3c2415ca30 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.393+0000 7f3c237fe640 1 -- 172.21.15.47:0/1102520481 >> v2:172.21.15.175:3300/0 conn(0x7f3c240b85d0 msgr2=0x7f3c240cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.393+0000 7f3c237fe640 1 --2- 172.21.15.47:0/1102520481 >> v2:172.21.15.175:3300/0 conn(0x7f3c240b85d0 0x7f3c240cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.393+0000 7f3c237fe640 1 -- 172.21.15.47:0/1102520481 >> v1:172.21.15.175:6789/0 conn(0x7f3c2415ca30 legacy=0x7f3c24160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.393+0000 7f3c237fe640 1 -- 172.21.15.47:0/1102520481 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c24161040 con 0x7f3c2415ce10 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.394+0000 7f3c237fe640 1 --2- 172.21.15.47:0/1102520481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c2415ce10 0x7f3c240cd730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009870 tx=0x7f3c1402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa9a48b7e3d30061 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.394+0000 7f3c22ffd640 1 -- 172.21.15.47:0/1102520481 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14030840 con 0x7f3c2415ce10 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.394+0000 7f3c22ffd640 1 -- 172.21.15.47:0/1102520481 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c140309a0 con 0x7f3c2415ce10 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.394+0000 7f3c22ffd640 1 -- 172.21.15.47:0/1102520481 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14030d10 con 0x7f3c2415ce10 2024-03-23T09:47:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.394+0000 7f3c2a626640 1 -- 172.21.15.47:0/1102520481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c2415ce10 msgr2=0x7f3c240cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.394+0000 7f3c2a626640 1 --2- 172.21.15.47:0/1102520481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c2415ce10 0x7f3c240cd730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009870 tx=0x7f3c1402b980 comp rx=0 tx=0).stop 2024-03-23T09:47:19.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.394+0000 7f3c2a626640 1 -- 172.21.15.47:0/1102520481 shutdown_connections 2024-03-23T09:47:19.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.394+0000 7f3c2a626640 1 --2- 172.21.15.47:0/1102520481 >> v2:172.21.15.175:3300/0 conn(0x7f3c240b85d0 0x7f3c240cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.394+0000 7f3c2a626640 1 --2- 172.21.15.47:0/1102520481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c2415ce10 0x7f3c240cd730 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.394+0000 7f3c2a626640 1 -- 172.21.15.47:0/1102520481 >> 172.21.15.47:0/1102520481 conn(0x7f3c2406b160 msgr2=0x7f3c2406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:19.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.394+0000 7f3c2a626640 1 -- 172.21.15.47:0/1102520481 shutdown_connections 2024-03-23T09:47:19.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.394+0000 7f3c2a626640 1 -- 172.21.15.47:0/1102520481 wait complete. 2024-03-23T09:47:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.395+0000 7f3c2a626640 1 Processor -- start 2024-03-23T09:47:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.396+0000 7f3c2a626640 1 -- start start 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.396+0000 7f3c2a626640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c240b85d0 0x7f3c240ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.396+0000 7f3c2a626640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c2415ca30 0x7f3c24154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.396+0000 7f3c2a626640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c2415ce10 0x7f3c2414acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.396+0000 7f3c2a626640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c240ca650 con 0x7f3c2415ce10 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.396+0000 7f3c2a626640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c240ca930 con 0x7f3c2415ca30 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.396+0000 7f3c2a626640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c240cac10 con 0x7f3c240b85d0 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c28bc3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c2415ce10 0x7f3c2414acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c28bc3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c2415ce10 0x7f3c2414acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38620/0 (socket says 172.21.15.47:38620) 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c28bc3640 1 -- 172.21.15.47:0/4259299218 learned_addr learned my addr 172.21.15.47:0/4259299218 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c237fe640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c2415ca30 0x7f3c24154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c23fff640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c240b85d0 0x7f3c240ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c28bc3640 1 -- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c240b85d0 msgr2=0x7f3c240ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c28bc3640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c240b85d0 0x7f3c240ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c28bc3640 1 -- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c2415ca30 msgr2=0x7f3c24154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c28bc3640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c2415ca30 0x7f3c24154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c28bc3640 1 -- 172.21.15.47:0/4259299218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c140094d0 con 0x7f3c2415ce10 2024-03-23T09:47:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c28bc3640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c2415ce10 0x7f3c2414acf0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00efc0 tx=0x7f3c1c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c20ff9640 1 -- 172.21.15.47:0/4259299218 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c014480 con 0x7f3c2415ce10 2024-03-23T09:47:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c20ff9640 1 -- 172.21.15.47:0/4259299218 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1c00f040 con 0x7f3c2415ce10 2024-03-23T09:47:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c2a626640 1 -- 172.21.15.47:0/4259299218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c240caef0 con 0x7f3c2415ce10 2024-03-23T09:47:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c20ff9640 1 -- 172.21.15.47:0/4259299218 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c014820 con 0x7f3c2415ce10 2024-03-23T09:47:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.397+0000 7f3c2a626640 1 -- 172.21.15.47:0/4259299218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c24155500 con 0x7f3c2415ce10 2024-03-23T09:47:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.398+0000 7f3c2a626640 1 -- 172.21.15.47:0/4259299218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c24072360 con 0x7f3c2415ce10 2024-03-23T09:47:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.399+0000 7f3c20ff9640 1 -- 172.21.15.47:0/4259299218 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3c1c014980 con 0x7f3c2415ce10 2024-03-23T09:47:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.399+0000 7f3c20ff9640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3c1006ac00 0x7f3c1006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.399+0000 7f3c20ff9640 1 -- 172.21.15.47:0/4259299218 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c1c089db0 con 0x7f3c2415ce10 2024-03-23T09:47:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.399+0000 7f3c23fff640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3c1006ac00 0x7f3c1006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.400+0000 7f3c23fff640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3c1006ac00 0x7f3c1006d0c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000f60 tx=0x7f3c18007cf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.404+0000 7f3c20ff9640 1 -- 172.21.15.47:0/4259299218 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3c1c055040 con 0x7f3c2415ce10 2024-03-23T09:47:19.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.583+0000 7f3c2a626640 1 -- 172.21.15.47:0/4259299218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3c240cb410 con 0x7f3c2415ce10 2024-03-23T09:47:19.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.584+0000 7f3c20ff9640 1 -- 172.21.15.47:0/4259299218 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3c240cb410 con 0x7f3c2415ce10 2024-03-23T09:47:19.584 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:47:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.585+0000 7f3c2a626640 1 -- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3c1006ac00 msgr2=0x7f3c1006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.585+0000 7f3c2a626640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3c1006ac00 0x7f3c1006d0c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000f60 tx=0x7f3c18007cf0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.585+0000 7f3c2a626640 1 -- 172.21.15.47:0/4259299218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c2415ce10 msgr2=0x7f3c2414acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.585+0000 7f3c2a626640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c2415ce10 0x7f3c2414acf0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00efc0 tx=0x7f3c1c00c490 comp rx=0 tx=0).stop 2024-03-23T09:47:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.585+0000 7f3c2a626640 1 -- 172.21.15.47:0/4259299218 shutdown_connections 2024-03-23T09:47:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.585+0000 7f3c2a626640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3c1006ac00 0x7f3c1006d0c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.585+0000 7f3c2a626640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c2415ce10 0x7f3c2414acf0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.585+0000 7f3c2a626640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c2415ca30 0x7f3c24154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.585+0000 7f3c2a626640 1 --2- 172.21.15.47:0/4259299218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c240b85d0 0x7f3c240ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.585+0000 7f3c2a626640 1 -- 172.21.15.47:0/4259299218 >> 172.21.15.47:0/4259299218 conn(0x7f3c2406b160 msgr2=0x7f3c24157080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.586+0000 7f3c2a626640 1 -- 172.21.15.47:0/4259299218 shutdown_connections 2024-03-23T09:47:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.586+0000 7f3c2a626640 1 -- 172.21.15.47:0/4259299218 wait complete. 2024-03-23T09:47:19.596 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:47:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.697+0000 7f9b928e0640 1 Processor -- start 2024-03-23T09:47:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.697+0000 7f9b928e0640 1 -- start start 2024-03-23T09:47:19.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.698+0000 7f9b928e0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9b8c0b10f0 0x7f9b8c0affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.698+0000 7f9b928e0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9b8c099110 0x7f9b8c0ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.698+0000 7f9b928e0640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8c0ce8f0 con 0x7f9b8c0b0520 2024-03-23T09:47:19.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.698+0000 7f9b928e0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8c0cc300 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.698+0000 7f9b928e0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8c0cc5c0 con 0x7f9b8c099110 2024-03-23T09:47:19.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.698+0000 7f9b8bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9b8c0b10f0 0x7f9b8c0affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.698+0000 7f9b8bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9b8c0b10f0 0x7f9b8c0affe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38626/0 (socket says 172.21.15.47:38626) 2024-03-23T09:47:19.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.698+0000 7f9b8bfff640 1 -- 172.21.15.47:0/4251076227 learned_addr learned my addr 172.21.15.47:0/4251076227 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.699+0000 7f9b90e7d640 1 --2- 172.21.15.47:0/4251076227 >> v2:172.21.15.175:3300/0 conn(0x7f9b8c099110 0x7f9b8c0ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.699+0000 7f9b8bfff640 1 -- 172.21.15.47:0/4251076227 >> v2:172.21.15.175:3300/0 conn(0x7f9b8c099110 msgr2=0x7f9b8c0ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.699+0000 7f9b8bfff640 1 --2- 172.21.15.47:0/4251076227 >> v2:172.21.15.175:3300/0 conn(0x7f9b8c099110 0x7f9b8c0ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.699+0000 7f9b8bfff640 1 -- 172.21.15.47:0/4251076227 >> v1:172.21.15.175:6789/0 conn(0x7f9b8c0b0520 legacy=0x7f9b8c098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.699+0000 7f9b8bfff640 1 -- 172.21.15.47:0/4251076227 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c0ce2a0 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.699+0000 7f9b90e7d640 1 --2- 172.21.15.47:0/4251076227 >> v2:172.21.15.175:3300/0 conn(0x7f9b8c099110 0x7f9b8c0ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.699+0000 7f9b8bfff640 1 --2- 172.21.15.47:0/4251076227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c0b10f0 0x7f9b8c0affe0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0722f0 tx=0x7f9b8002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4ca4521c402231 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.700+0000 7f9b8affd640 1 -- 172.21.15.47:0/4251076227 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b800317e0 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.700+0000 7f9b8affd640 1 -- 172.21.15.47:0/4251076227 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b80031940 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.700+0000 7f9b8affd640 1 -- 172.21.15.47:0/4251076227 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80031cd0 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.700+0000 7f9b928e0640 1 -- 172.21.15.47:0/4251076227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c0b10f0 msgr2=0x7f9b8c0affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.700+0000 7f9b928e0640 1 --2- 172.21.15.47:0/4251076227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c0b10f0 0x7f9b8c0affe0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0722f0 tx=0x7f9b8002b880 comp rx=0 tx=0).stop 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.700+0000 7f9b928e0640 1 -- 172.21.15.47:0/4251076227 shutdown_connections 2024-03-23T09:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.700+0000 7f9b928e0640 1 --2- 172.21.15.47:0/4251076227 >> v2:172.21.15.175:3300/0 conn(0x7f9b8c099110 0x7f9b8c0ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.700+0000 7f9b928e0640 1 --2- 172.21.15.47:0/4251076227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c0b10f0 0x7f9b8c0affe0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.700+0000 7f9b928e0640 1 -- 172.21.15.47:0/4251076227 >> 172.21.15.47:0/4251076227 conn(0x7f9b8c0b21c0 msgr2=0x7f9b8c0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.700+0000 7f9b928e0640 1 -- 172.21.15.47:0/4251076227 shutdown_connections 2024-03-23T09:47:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.700+0000 7f9b928e0640 1 -- 172.21.15.47:0/4251076227 wait complete. 2024-03-23T09:47:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.701+0000 7f9b928e0640 1 Processor -- start 2024-03-23T09:47:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.702+0000 7f9b928e0640 1 -- start start 2024-03-23T09:47:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.702+0000 7f9b928e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c099110 0x7f9b8c0c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.702+0000 7f9b928e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c0b0520 0x7f9b8c0b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.702+0000 7f9b928e0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c0b10f0 0x7f9b8c1e0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.702+0000 7f9b928e0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c0c9390 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.702+0000 7f9b928e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c1e3880 con 0x7f9b8c0b0520 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.702+0000 7f9b928e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c1e3b60 con 0x7f9b8c099110 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.702+0000 7f9b90e7d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c0b10f0 0x7f9b8c1e0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.702+0000 7f9b90e7d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c0b10f0 0x7f9b8c1e0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38642/0 (socket says 172.21.15.47:38642) 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.702+0000 7f9b90e7d640 1 -- 172.21.15.47:0/2385247347 learned_addr learned my addr 172.21.15.47:0/2385247347 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.702+0000 7f9b8bfff640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c099110 0x7f9b8c0c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.702+0000 7f9b8b7fe640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c0b0520 0x7f9b8c0b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.703+0000 7f9b90e7d640 1 -- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c099110 msgr2=0x7f9b8c0c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.703+0000 7f9b90e7d640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c099110 0x7f9b8c0c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.703+0000 7f9b90e7d640 1 -- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c0b0520 msgr2=0x7f9b8c0b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.703+0000 7f9b90e7d640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c0b0520 0x7f9b8c0b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.703+0000 7f9b90e7d640 1 -- 172.21.15.47:0/2385247347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b800094d0 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.703+0000 7f9b8b7fe640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c0b0520 0x7f9b8c0b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.703+0000 7f9b90e7d640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c0b10f0 0x7f9b8c1e0260 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400cef0 tx=0x7f9b8400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.703+0000 7f9b88ff9640 1 -- 172.21.15.47:0/2385247347 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8400d840 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.703+0000 7f9b88ff9640 1 -- 172.21.15.47:0/2385247347 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8400d9c0 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.703+0000 7f9b928e0640 1 -- 172.21.15.47:0/2385247347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c0b1650 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.703+0000 7f9b88ff9640 1 -- 172.21.15.47:0/2385247347 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8400dd70 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.703+0000 7f9b928e0640 1 -- 172.21.15.47:0/2385247347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c1e4120 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.704+0000 7f9b928e0640 1 -- 172.21.15.47:0/2385247347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8c066be0 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.704+0000 7f9b88ff9640 1 -- 172.21.15.47:0/2385247347 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9b8401e3b0 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.705+0000 7f9b88ff9640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b7806ac00 0x7f9b7806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.705+0000 7f9b88ff9640 1 -- 172.21.15.47:0/2385247347 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b84089de0 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.705+0000 7f9b8bfff640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b7806ac00 0x7f9b7806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.705+0000 7f9b8bfff640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b7806ac00 0x7f9b7806d0c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9b80036000 tx=0x7f9b8003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:19.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.709+0000 7f9b88ff9640 1 -- 172.21.15.47:0/2385247347 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9b840550a0 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.899+0000 7f9b928e0640 1 -- 172.21.15.47:0/2385247347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9b8c0b6650 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.899+0000 7f9b88ff9640 1 -- 172.21.15.47:0/2385247347 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v85) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9b8401a070 con 0x7f9b8c0b10f0 2024-03-23T09:47:19.899 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:47:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.901+0000 7f9b928e0640 1 -- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b7806ac00 msgr2=0x7f9b7806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.901+0000 7f9b928e0640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b7806ac00 0x7f9b7806d0c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9b80036000 tx=0x7f9b8003d040 comp rx=0 tx=0).stop 2024-03-23T09:47:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.901+0000 7f9b928e0640 1 -- 172.21.15.47:0/2385247347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c0b10f0 msgr2=0x7f9b8c1e0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.901+0000 7f9b928e0640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c0b10f0 0x7f9b8c1e0260 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400cef0 tx=0x7f9b8400a710 comp rx=0 tx=0).stop 2024-03-23T09:47:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.901+0000 7f9b928e0640 1 -- 172.21.15.47:0/2385247347 shutdown_connections 2024-03-23T09:47:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.901+0000 7f9b928e0640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b7806ac00 0x7f9b7806d0c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.901+0000 7f9b928e0640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c0b10f0 0x7f9b8c1e0260 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.901+0000 7f9b928e0640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c0b0520 0x7f9b8c0b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.901+0000 7f9b928e0640 1 --2- 172.21.15.47:0/2385247347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c099110 0x7f9b8c0c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.901+0000 7f9b928e0640 1 -- 172.21.15.47:0/2385247347 >> 172.21.15.47:0/2385247347 conn(0x7f9b8c0b21c0 msgr2=0x7f9b8c1e2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.902+0000 7f9b928e0640 1 -- 172.21.15.47:0/2385247347 shutdown_connections 2024-03-23T09:47:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:19.902+0000 7f9b928e0640 1 -- 172.21.15.47:0/2385247347 wait complete. 2024-03-23T09:47:19.912 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:47:19.912 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:47:19.927 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:47:20.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.067+0000 7f0ebd73e640 1 Processor -- start 2024-03-23T09:47:20.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.067+0000 7f0ebd73e640 1 -- start start 2024-03-23T09:47:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0ebd73e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0eb815a940 0x7f0eb806ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0ebd73e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb80cc010 0x7f0eb806f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0ebd73e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb80cd0b0 con 0x7f0eb806fb00 2024-03-23T09:47:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0ebd73e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb80cc4a0 con 0x7f0eb815a940 2024-03-23T09:47:20.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0ebd73e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb80cc780 con 0x7f0eb80cc010 2024-03-23T09:47:20.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0eb6ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0eb815a940 0x7f0eb806ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0eb6ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0eb815a940 0x7f0eb806ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38656/0 (socket says 172.21.15.47:38656) 2024-03-23T09:47:20.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0eb6ffd640 1 -- 172.21.15.47:0/560908894 learned_addr learned my addr 172.21.15.47:0/560908894 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:20.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0eb67fc640 1 --2- 172.21.15.47:0/560908894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb80cc010 0x7f0eb806f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0eb6ffd640 1 -- 172.21.15.47:0/560908894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb80cc010 msgr2=0x7f0eb806f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:20.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0eb6ffd640 1 --2- 172.21.15.47:0/560908894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb80cc010 0x7f0eb806f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0eb6ffd640 1 -- 172.21.15.47:0/560908894 >> v1:172.21.15.175:6789/0 conn(0x7f0eb806fb00 legacy=0x7f0eb815e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0eb6ffd640 1 -- 172.21.15.47:0/560908894 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb80cca60 con 0x7f0eb815a940 2024-03-23T09:47:20.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.068+0000 7f0eb67fc640 1 --2- 172.21.15.47:0/560908894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb80cc010 0x7f0eb806f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:20.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.069+0000 7f0eb6ffd640 1 --2- 172.21.15.47:0/560908894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb815a940 0x7f0eb806ed90 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009870 tx=0x7f0eb002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de40753bbaa30bcf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.069+0000 7f0eb5ffb640 1 -- 172.21.15.47:0/560908894 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0030840 con 0x7f0eb815a940 2024-03-23T09:47:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.069+0000 7f0eb5ffb640 1 -- 172.21.15.47:0/560908894 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb00309a0 con 0x7f0eb815a940 2024-03-23T09:47:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.069+0000 7f0eb5ffb640 1 -- 172.21.15.47:0/560908894 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0030d30 con 0x7f0eb815a940 2024-03-23T09:47:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.070+0000 7f0ebd73e640 1 -- 172.21.15.47:0/560908894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb815a940 msgr2=0x7f0eb806ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.070+0000 7f0ebd73e640 1 --2- 172.21.15.47:0/560908894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb815a940 0x7f0eb806ed90 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009870 tx=0x7f0eb002b980 comp rx=0 tx=0).stop 2024-03-23T09:47:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.070+0000 7f0ebd73e640 1 -- 172.21.15.47:0/560908894 shutdown_connections 2024-03-23T09:47:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.070+0000 7f0ebd73e640 1 --2- 172.21.15.47:0/560908894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb80cc010 0x7f0eb806f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.070+0000 7f0ebd73e640 1 --2- 172.21.15.47:0/560908894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb815a940 0x7f0eb806ed90 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.070+0000 7f0ebd73e640 1 -- 172.21.15.47:0/560908894 >> 172.21.15.47:0/560908894 conn(0x7f0eb80b13a0 msgr2=0x7f0eb80b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.070+0000 7f0ebd73e640 1 -- 172.21.15.47:0/560908894 shutdown_connections 2024-03-23T09:47:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.070+0000 7f0ebd73e640 1 -- 172.21.15.47:0/560908894 wait complete. 2024-03-23T09:47:20.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.071+0000 7f0ebd73e640 1 Processor -- start 2024-03-23T09:47:20.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.071+0000 7f0ebd73e640 1 -- start start 2024-03-23T09:47:20.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0ebd73e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb806fb00 0x7f0eb814afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0ebd73e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb815a940 0x7f0eb814db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0ebd73e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb806d390 0x7f0eb806d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0ebd73e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb806dd60 con 0x7f0eb815a940 2024-03-23T09:47:20.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0ebd73e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb8157a80 con 0x7f0eb806fb00 2024-03-23T09:47:20.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0ebd73e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb81f3880 con 0x7f0eb806d390 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0eb67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb815a940 0x7f0eb814db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0eb67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb815a940 0x7f0eb814db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38672/0 (socket says 172.21.15.47:38672) 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0eb67fc640 1 -- 172.21.15.47:0/3806375782 learned_addr learned my addr 172.21.15.47:0/3806375782 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0eb6ffd640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb806fb00 0x7f0eb814afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0eb67fc640 1 -- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb806d390 msgr2=0x7f0eb806d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0eb67fc640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb806d390 0x7f0eb806d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0eb67fc640 1 -- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb806fb00 msgr2=0x7f0eb814afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0eb67fc640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb806fb00 0x7f0eb814afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0eb67fc640 1 -- 172.21.15.47:0/3806375782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb00094d0 con 0x7f0eb815a940 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0eb6ffd640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb806fb00 0x7f0eb814afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.072+0000 7f0eb67fc640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb815a940 0x7f0eb814db80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c7f0 tx=0x7f0eac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.073+0000 7f0e9f7fe640 1 -- 172.21.15.47:0/3806375782 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac007a30 con 0x7f0eb815a940 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.073+0000 7f0e9f7fe640 1 -- 172.21.15.47:0/3806375782 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac007b90 con 0x7f0eb815a940 2024-03-23T09:47:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.073+0000 7f0e9f7fe640 1 -- 172.21.15.47:0/3806375782 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00ce40 con 0x7f0eb815a940 2024-03-23T09:47:20.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.073+0000 7f0ebd73e640 1 -- 172.21.15.47:0/3806375782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb815aea0 con 0x7f0eb815a940 2024-03-23T09:47:20.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.073+0000 7f0ebd73e640 1 -- 172.21.15.47:0/3806375782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb81f3e40 con 0x7f0eb815a940 2024-03-23T09:47:20.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.074+0000 7f0ebd73e640 1 -- 172.21.15.47:0/3806375782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb814d770 con 0x7f0eb815a940 2024-03-23T09:47:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.074+0000 7f0e9f7fe640 1 -- 172.21.15.47:0/3806375782 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0eac0113f0 con 0x7f0eb815a940 2024-03-23T09:47:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.074+0000 7f0e9f7fe640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ea406ac00 0x7f0ea406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.074+0000 7f0e9f7fe640 1 -- 172.21.15.47:0/3806375782 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0eac089480 con 0x7f0eb815a940 2024-03-23T09:47:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.075+0000 7f0eb6ffd640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ea406ac00 0x7f0ea406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.075+0000 7f0eb6ffd640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ea406ac00 0x7f0ea406d0c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0036000 tx=0x7f0eb003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:20.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.079+0000 7f0e9f7fe640 1 -- 172.21.15.47:0/3806375782 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0eac0546f0 con 0x7f0eb815a940 2024-03-23T09:47:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.261+0000 7f0ebd73e640 1 -- 172.21.15.47:0/3806375782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0eb8066be0 con 0x7f0eb815a940 2024-03-23T09:47:20.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.262+0000 7f0e9f7fe640 1 -- 172.21.15.47:0/3806375782 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16092 (secure 0 0 0) 0x7f0eac057d10 con 0x7f0eb815a940 2024-03-23T09:47:20.262 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:20.262 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":85,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:47:12.392230+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:47:09.722391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-23T09:47:10.354054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:47:20.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.264+0000 7f0ebd73e640 1 -- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ea406ac00 msgr2=0x7f0ea406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.264+0000 7f0ebd73e640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ea406ac00 0x7f0ea406d0c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0036000 tx=0x7f0eb003b040 comp rx=0 tx=0).stop 2024-03-23T09:47:20.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.264+0000 7f0ebd73e640 1 -- 172.21.15.47:0/3806375782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb815a940 msgr2=0x7f0eb814db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.264+0000 7f0ebd73e640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb815a940 0x7f0eb814db80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c7f0 tx=0x7f0eac00cbf0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.264+0000 7f0ebd73e640 1 -- 172.21.15.47:0/3806375782 shutdown_connections 2024-03-23T09:47:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.264+0000 7f0ebd73e640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0ea406ac00 0x7f0ea406d0c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.264+0000 7f0ebd73e640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0eb806d390 0x7f0eb806d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.264+0000 7f0ebd73e640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eb815a940 0x7f0eb814db80 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.264+0000 7f0ebd73e640 1 --2- 172.21.15.47:0/3806375782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb806fb00 0x7f0eb814afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.264+0000 7f0ebd73e640 1 -- 172.21.15.47:0/3806375782 >> 172.21.15.47:0/3806375782 conn(0x7f0eb80b13a0 msgr2=0x7f0eb814ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.264+0000 7f0ebd73e640 1 -- 172.21.15.47:0/3806375782 shutdown_connections 2024-03-23T09:47:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.264+0000 7f0ebd73e640 1 -- 172.21.15.47:0/3806375782 wait complete. 2024-03-23T09:47:20.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:47:09.722391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:47:10.354054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:47:20.275 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:47:20.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.379+0000 7ff84d127640 1 Processor -- start 2024-03-23T09:47:20.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.379+0000 7ff84d127640 1 -- start start 2024-03-23T09:47:20.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.380+0000 7ff84d127640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8480cdcb0 0x7ff8480b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.380+0000 7ff84d127640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff8480d2020 0x7ff8480b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.380+0000 7ff84d127640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8480b72f0 con 0x7ff84815cb50 2024-03-23T09:47:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.380+0000 7ff84d127640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8480b7e70 con 0x7ff8480d2020 2024-03-23T09:47:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.380+0000 7ff84d127640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84815cf30 con 0x7ff8480cdcb0 2024-03-23T09:47:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.380+0000 7ff84759e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff8480d2020 0x7ff8480b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.380+0000 7ff84759e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff8480d2020 0x7ff8480b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38684/0 (socket says 172.21.15.47:38684) 2024-03-23T09:47:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.380+0000 7ff84759e640 1 -- 172.21.15.47:0/3819824469 learned_addr learned my addr 172.21.15.47:0/3819824469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.380+0000 7ff84659c640 1 --2- 172.21.15.47:0/3819824469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8480cdcb0 0x7ff8480b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.380+0000 7ff84759e640 1 -- 172.21.15.47:0/3819824469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8480cdcb0 msgr2=0x7ff8480b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.380+0000 7ff84759e640 1 --2- 172.21.15.47:0/3819824469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8480cdcb0 0x7ff8480b84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.380+0000 7ff84759e640 1 -- 172.21.15.47:0/3819824469 >> v1:172.21.15.175:6789/0 conn(0x7ff84815cb50 legacy=0x7ff8480cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.380+0000 7ff84759e640 1 -- 172.21.15.47:0/3819824469 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8480b75d0 con 0x7ff8480d2020 2024-03-23T09:47:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.381+0000 7ff84759e640 1 --2- 172.21.15.47:0/3819824469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8480d2020 0x7ff8480b5a00 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff838009e50 tx=0x7ff83802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31238e46e6162b3a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.381+0000 7ff845d9b640 1 -- 172.21.15.47:0/3819824469 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83802cb00 con 0x7ff8480d2020 2024-03-23T09:47:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.381+0000 7ff845d9b640 1 -- 172.21.15.47:0/3819824469 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83802cc60 con 0x7ff8480d2020 2024-03-23T09:47:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.381+0000 7ff845d9b640 1 -- 172.21.15.47:0/3819824469 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8380343f0 con 0x7ff8480d2020 2024-03-23T09:47:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.382+0000 7ff84d127640 1 -- 172.21.15.47:0/3819824469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8480d2020 msgr2=0x7ff8480b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.382+0000 7ff84d127640 1 --2- 172.21.15.47:0/3819824469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8480d2020 0x7ff8480b5a00 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff838009e50 tx=0x7ff83802bf60 comp rx=0 tx=0).stop 2024-03-23T09:47:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.382+0000 7ff84d127640 1 -- 172.21.15.47:0/3819824469 shutdown_connections 2024-03-23T09:47:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.382+0000 7ff84d127640 1 --2- 172.21.15.47:0/3819824469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8480d2020 0x7ff8480b5a00 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.382+0000 7ff84d127640 1 --2- 172.21.15.47:0/3819824469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8480cdcb0 0x7ff8480b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.382+0000 7ff84d127640 1 -- 172.21.15.47:0/3819824469 >> 172.21.15.47:0/3819824469 conn(0x7ff848065df0 msgr2=0x7ff848066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.382+0000 7ff84d127640 1 -- 172.21.15.47:0/3819824469 shutdown_connections 2024-03-23T09:47:20.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.382+0000 7ff84d127640 1 -- 172.21.15.47:0/3819824469 wait complete. 2024-03-23T09:47:20.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.383+0000 7ff84d127640 1 Processor -- start 2024-03-23T09:47:20.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.383+0000 7ff84d127640 1 -- start start 2024-03-23T09:47:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84d127640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8480cdcb0 0x7ff8480ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84d127640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8480d2020 0x7ff8480b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84d127640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff84815cb50 0x7ff8481f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84d127640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8480c99a0 con 0x7ff84815cb50 2024-03-23T09:47:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84d127640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8481f2340 con 0x7ff8480cdcb0 2024-03-23T09:47:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84d127640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8480ca8f0 con 0x7ff8480d2020 2024-03-23T09:47:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff84815cb50 0x7ff8481f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff84815cb50 0x7ff8481f27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38688/0 (socket says 172.21.15.47:38688) 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84759e640 1 -- 172.21.15.47:0/773954720 learned_addr learned my addr 172.21.15.47:0/773954720 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff846d9d640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8480cdcb0 0x7ff8480ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84659c640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8480d2020 0x7ff8480b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84759e640 1 -- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8480d2020 msgr2=0x7ff8480b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84759e640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8480d2020 0x7ff8480b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84759e640 1 -- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8480cdcb0 msgr2=0x7ff8480ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84759e640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8480cdcb0 0x7ff8480ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.384+0000 7ff84759e640 1 -- 172.21.15.47:0/773954720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff838009b30 con 0x7ff84815cb50 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.385+0000 7ff84759e640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff84815cb50 0x7ff8481f27d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff838036040 tx=0x7ff838004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.385+0000 7ff82f7fe640 1 -- 172.21.15.47:0/773954720 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838009510 con 0x7ff84815cb50 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.385+0000 7ff82f7fe640 1 -- 172.21.15.47:0/773954720 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83803e450 con 0x7ff84815cb50 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.385+0000 7ff84d127640 1 -- 172.21.15.47:0/773954720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8481f1ff0 con 0x7ff84815cb50 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.385+0000 7ff82f7fe640 1 -- 172.21.15.47:0/773954720 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83803e800 con 0x7ff84815cb50 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.385+0000 7ff84d127640 1 -- 172.21.15.47:0/773954720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8480c9ed0 con 0x7ff84815cb50 2024-03-23T09:47:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.386+0000 7ff84d127640 1 -- 172.21.15.47:0/773954720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff848072360 con 0x7ff84815cb50 2024-03-23T09:47:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.386+0000 7ff82f7fe640 1 -- 172.21.15.47:0/773954720 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff83803e960 con 0x7ff84815cb50 2024-03-23T09:47:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.387+0000 7ff82f7fe640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff83406ac30 0x7ff83406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.387+0000 7ff82f7fe640 1 -- 172.21.15.47:0/773954720 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8380abfa0 con 0x7ff84815cb50 2024-03-23T09:47:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.387+0000 7ff846d9d640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff83406ac30 0x7ff83406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.387+0000 7ff846d9d640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff83406ac30 0x7ff83406d0f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff8481f5540 tx=0x7ff840009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.391+0000 7ff82f7fe640 1 -- 172.21.15.47:0/773954720 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff8380df580 con 0x7ff84815cb50 2024-03-23T09:47:20.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.569+0000 7ff84d127640 1 -- 172.21.15.47:0/773954720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff848066be0 con 0x7ff84815cb50 2024-03-23T09:47:20.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.569+0000 7ff82f7fe640 1 -- 172.21.15.47:0/773954720 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff8380772c0 con 0x7ff84815cb50 2024-03-23T09:47:20.569 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:47:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.571+0000 7ff84d127640 1 -- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff83406ac30 msgr2=0x7ff83406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.571+0000 7ff84d127640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff83406ac30 0x7ff83406d0f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff8481f5540 tx=0x7ff840009060 comp rx=0 tx=0).stop 2024-03-23T09:47:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.571+0000 7ff84d127640 1 -- 172.21.15.47:0/773954720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff84815cb50 msgr2=0x7ff8481f27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.571+0000 7ff84d127640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff84815cb50 0x7ff8481f27d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff838036040 tx=0x7ff838004a40 comp rx=0 tx=0).stop 2024-03-23T09:47:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.571+0000 7ff84d127640 1 -- 172.21.15.47:0/773954720 shutdown_connections 2024-03-23T09:47:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.571+0000 7ff84d127640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff83406ac30 0x7ff83406d0f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.571+0000 7ff84d127640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff84815cb50 0x7ff8481f27d0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.571+0000 7ff84d127640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8480d2020 0x7ff8480b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.571+0000 7ff84d127640 1 --2- 172.21.15.47:0/773954720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8480cdcb0 0x7ff8480ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.571+0000 7ff84d127640 1 -- 172.21.15.47:0/773954720 >> 172.21.15.47:0/773954720 conn(0x7ff848065df0 msgr2=0x7ff8480b81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.571+0000 7ff84d127640 1 -- 172.21.15.47:0/773954720 shutdown_connections 2024-03-23T09:47:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.571+0000 7ff84d127640 1 -- 172.21.15.47:0/773954720 wait complete. 2024-03-23T09:47:20.582 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:47:20.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.686+0000 7f2d7dd3c640 1 Processor -- start 2024-03-23T09:47:20.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.686+0000 7f2d7dd3c640 1 -- start start 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d7dd3c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2d780b8350 0x7f2d78160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d7dd3c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d78161630 con 0x7f2d780cde10 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d7dd3c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d780d1030 con 0x7f2d7815c990 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d7dd3c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d780cfe10 con 0x7f2d780b8350 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d76ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2d780cde10 0x7f2d780d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51344/0 (socket says 172.21.15.47:51344) 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d76ffd640 1 -- 172.21.15.47:0/2463066851 learned_addr learned my addr 172.21.15.47:0/2463066851 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802360193 0 0) 0x7f2d78161630 con 0x7f2d780cde10 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d60003520 con 0x7f2d780cde10 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d77fff640 1 --2- 172.21.15.47:0/2463066851 >> v2:172.21.15.175:3300/0 conn(0x7f2d780b8350 0x7f2d78160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2595778974 0 0) 0x7f2d60003520 con 0x7f2d780cde10 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 >> v2:172.21.15.175:3300/0 conn(0x7f2d780b8350 msgr2=0x7f2d78160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d767fc640 1 --2- 172.21.15.47:0/2463066851 >> v2:172.21.15.175:3300/0 conn(0x7f2d780b8350 0x7f2d78160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 >> v1:172.21.15.175:6789/0 conn(0x7f2d7815c990 legacy=0x7f2d780cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d780b5b60 con 0x7f2d780cde10 2024-03-23T09:47:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f2d6c001670 con 0x7f2d780cde10 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 >> v1:172.21.15.47:6789/0 conn(0x7f2d780cde10 legacy=0x7f2d780d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d767fc640 1 --2- 172.21.15.47:0/2463066851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d60003970 0x7f2d60003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d767fc640 1 --2- 172.21.15.47:0/2463066851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d600047c0 0x7f2d60004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.687+0000 7f2d767fc640 1 --2- 172.21.15.47:0/2463066851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60005180 0x7f2d6000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.688+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780cfe10 con 0x7f2d60003970 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.688+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6000c120 con 0x7f2d600047c0 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.688+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6000c2e0 con 0x7f2d60005180 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.688+0000 7f2d777fe640 1 --2- 172.21.15.47:0/2463066851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d60003970 0x7f2d60003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.688+0000 7f2d76ffd640 1 --2- 172.21.15.47:0/2463066851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d600047c0 0x7f2d60004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.688+0000 7f2d77fff640 1 --2- 172.21.15.47:0/2463066851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60005180 0x7f2d6000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.688+0000 7f2d777fe640 1 -- 172.21.15.47:0/2463066851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60005180 msgr2=0x7f2d6000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.688+0000 7f2d777fe640 1 --2- 172.21.15.47:0/2463066851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60005180 0x7f2d6000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.688+0000 7f2d777fe640 1 -- 172.21.15.47:0/2463066851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d600047c0 msgr2=0x7f2d60004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.688+0000 7f2d777fe640 1 --2- 172.21.15.47:0/2463066851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d600047c0 0x7f2d60004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.688+0000 7f2d777fe640 1 -- 172.21.15.47:0/2463066851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d60003520 con 0x7f2d60003970 2024-03-23T09:47:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.688+0000 7f2d77fff640 1 -- 172.21.15.47:0/2463066851 reap_dead start 2024-03-23T09:47:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.689+0000 7f2d777fe640 1 --2- 172.21.15.47:0/2463066851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d60003970 0x7f2d60003d90 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f2d68001750 tx=0x7f2d600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4de51d64c27d658 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.689+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d680052d0 con 0x7f2d60003970 2024-03-23T09:47:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.689+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6802e050 con 0x7f2d60003970 2024-03-23T09:47:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.689+0000 7f2d767fc640 1 -- 172.21.15.47:0/2463066851 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d68003830 con 0x7f2d60003970 2024-03-23T09:47:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.689+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/2463066851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d60003970 msgr2=0x7f2d60003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.689+0000 7f2d7dd3c640 1 --2- 172.21.15.47:0/2463066851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d60003970 0x7f2d60003d90 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f2d68001750 tx=0x7f2d600057f0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.689+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/2463066851 shutdown_connections 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.689+0000 7f2d7dd3c640 1 --2- 172.21.15.47:0/2463066851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d60003970 0x7f2d60003d90 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.689+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/2463066851 >> 172.21.15.47:0/2463066851 conn(0x7f2d780659d0 msgr2=0x7f2d78065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.689+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/2463066851 shutdown_connections 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.689+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/2463066851 wait complete. 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.691+0000 7f2d7dd3c640 1 Processor -- start 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.691+0000 7f2d7dd3c640 1 -- start start 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.691+0000 7f2d7dd3c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d60003970 0x7f2d780cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.691+0000 7f2d7dd3c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d781463c0 0x7f2d780b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.691+0000 7f2d7dd3c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d780cc9c0 0x7f2d780b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.691+0000 7f2d7dd3c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d781f6940 con 0x7f2d781463c0 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.691+0000 7f2d7dd3c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d78145f30 con 0x7f2d60003970 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.691+0000 7f2d7dd3c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d781f6660 con 0x7f2d780cc9c0 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d76ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d781463c0 0x7f2d780b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d76ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d781463c0 0x7f2d780b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38714/0 (socket says 172.21.15.47:38714) 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d76ffd640 1 -- 172.21.15.47:0/4207535482 learned_addr learned my addr 172.21.15.47:0/4207535482 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d777fe640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d60003970 0x7f2d780cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d77fff640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d780cc9c0 0x7f2d780b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d76ffd640 1 -- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d780cc9c0 msgr2=0x7f2d780b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d76ffd640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d780cc9c0 0x7f2d780b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d76ffd640 1 -- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d60003970 msgr2=0x7f2d780cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d76ffd640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d60003970 0x7f2d780cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d76ffd640 1 -- 172.21.15.47:0/4207535482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d781458e0 con 0x7f2d781463c0 2024-03-23T09:47:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d76ffd640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d781463c0 0x7f2d780b3900 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0072c0 tx=0x7f2d6c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d5ffff640 1 -- 172.21.15.47:0/4207535482 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0016a0 con 0x7f2d781463c0 2024-03-23T09:47:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d5ffff640 1 -- 172.21.15.47:0/4207535482 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c00f960 con 0x7f2d781463c0 2024-03-23T09:47:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d5ffff640 1 -- 172.21.15.47:0/4207535482 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c00fd10 con 0x7f2d781463c0 2024-03-23T09:47:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/4207535482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d78145bc0 con 0x7f2d781463c0 2024-03-23T09:47:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.692+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/4207535482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d781f5ba0 con 0x7f2d781463c0 2024-03-23T09:47:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.693+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/4207535482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d78072360 con 0x7f2d781463c0 2024-03-23T09:47:20.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.694+0000 7f2d5ffff640 1 -- 172.21.15.47:0/4207535482 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2d6c01d460 con 0x7f2d781463c0 2024-03-23T09:47:20.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.694+0000 7f2d5ffff640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d6406ac00 0x7f2d6406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.694+0000 7f2d5ffff640 1 -- 172.21.15.47:0/4207535482 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d6c0892a0 con 0x7f2d781463c0 2024-03-23T09:47:20.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.694+0000 7f2d777fe640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d6406ac00 0x7f2d6406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.695+0000 7f2d777fe640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d6406ac00 0x7f2d6406d0c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2d68033000 tx=0x7f2d68009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:20.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.698+0000 7f2d5ffff640 1 -- 172.21.15.47:0/4207535482 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2d6c054530 con 0x7f2d781463c0 2024-03-23T09:47:20.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.875+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/4207535482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d78145740 con 0x7f2d781463c0 2024-03-23T09:47:20.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.876+0000 7f2d5ffff640 1 -- 172.21.15.47:0/4207535482 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d78145740 con 0x7f2d781463c0 2024-03-23T09:47:20.876 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:47:20.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.878+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d6406ac00 msgr2=0x7f2d6406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.878+0000 7f2d7dd3c640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d6406ac00 0x7f2d6406d0c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2d68033000 tx=0x7f2d68009030 comp rx=0 tx=0).stop 2024-03-23T09:47:20.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.878+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/4207535482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d781463c0 msgr2=0x7f2d780b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.878+0000 7f2d7dd3c640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d781463c0 0x7f2d780b3900 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0072c0 tx=0x7f2d6c007780 comp rx=0 tx=0).stop 2024-03-23T09:47:20.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.878+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/4207535482 shutdown_connections 2024-03-23T09:47:20.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.878+0000 7f2d7dd3c640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2d6406ac00 0x7f2d6406d0c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.878+0000 7f2d7dd3c640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d780cc9c0 0x7f2d780b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.878+0000 7f2d7dd3c640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d781463c0 0x7f2d780b3900 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.878+0000 7f2d7dd3c640 1 --2- 172.21.15.47:0/4207535482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d60003970 0x7f2d780cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.878+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/4207535482 >> 172.21.15.47:0/4207535482 conn(0x7f2d780659d0 msgr2=0x7f2d78065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:20.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.878+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/4207535482 shutdown_connections 2024-03-23T09:47:20.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.878+0000 7f2d7dd3c640 1 -- 172.21.15.47:0/4207535482 wait complete. 2024-03-23T09:47:20.888 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:47:20.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.992+0000 7f7e36aab640 1 Processor -- start 2024-03-23T09:47:20.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.992+0000 7f7e36aab640 1 -- start start 2024-03-23T09:47:20.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.992+0000 7f7e36aab640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e3015cc30 0x7f7e300ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.992+0000 7f7e36aab640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7e300cad40 0x7f7e300cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.992+0000 7f7e36aab640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e300b1170 con 0x7f7e300b0610 2024-03-23T09:47:20.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.992+0000 7f7e36aab640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3015d010 con 0x7f7e3015cc30 2024-03-23T09:47:20.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.992+0000 7f7e36aab640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e300b13d0 con 0x7f7e300cad40 2024-03-23T09:47:20.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.992+0000 7f7e34847640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e3015cc30 0x7f7e300ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.993+0000 7f7e34847640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e3015cc30 0x7f7e300ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38730/0 (socket says 172.21.15.47:38730) 2024-03-23T09:47:20.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.993+0000 7f7e34847640 1 -- 172.21.15.47:0/1366667843 learned_addr learned my addr 172.21.15.47:0/1366667843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:20.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.993+0000 7f7e2ffff640 1 --2- 172.21.15.47:0/1366667843 >> v2:172.21.15.175:3300/0 conn(0x7f7e300cad40 0x7f7e300cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.993+0000 7f7e34847640 1 -- 172.21.15.47:0/1366667843 >> v2:172.21.15.175:3300/0 conn(0x7f7e300cad40 msgr2=0x7f7e300cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:20.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.993+0000 7f7e34847640 1 --2- 172.21.15.47:0/1366667843 >> v2:172.21.15.175:3300/0 conn(0x7f7e300cad40 0x7f7e300cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.993+0000 7f7e34847640 1 -- 172.21.15.47:0/1366667843 >> v1:172.21.15.175:6789/0 conn(0x7f7e300b0610 legacy=0x7f7e300b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:20.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.993+0000 7f7e34847640 1 -- 172.21.15.47:0/1366667843 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e300b16b0 con 0x7f7e3015cc30 2024-03-23T09:47:20.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.993+0000 7f7e2ffff640 1 --2- 172.21.15.47:0/1366667843 >> v2:172.21.15.175:3300/0 conn(0x7f7e300cad40 0x7f7e300cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:20.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.994+0000 7f7e34847640 1 --2- 172.21.15.47:0/1366667843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e3015cc30 0x7f7e300ccf70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f7e300ce240 tx=0x7f7e2802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9b208f46835493 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:20.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.994+0000 7f7e2f7fe640 1 -- 172.21.15.47:0/1366667843 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2802ca80 con 0x7f7e3015cc30 2024-03-23T09:47:20.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.994+0000 7f7e2f7fe640 1 -- 172.21.15.47:0/1366667843 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2802cbe0 con 0x7f7e3015cc30 2024-03-23T09:47:20.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.994+0000 7f7e2f7fe640 1 -- 172.21.15.47:0/1366667843 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e280343f0 con 0x7f7e3015cc30 2024-03-23T09:47:20.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.994+0000 7f7e36aab640 1 -- 172.21.15.47:0/1366667843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e3015cc30 msgr2=0x7f7e300ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.994+0000 7f7e36aab640 1 --2- 172.21.15.47:0/1366667843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e3015cc30 0x7f7e300ccf70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f7e300ce240 tx=0x7f7e2802bee0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.994+0000 7f7e36aab640 1 -- 172.21.15.47:0/1366667843 shutdown_connections 2024-03-23T09:47:20.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.994+0000 7f7e36aab640 1 --2- 172.21.15.47:0/1366667843 >> v2:172.21.15.175:3300/0 conn(0x7f7e300cad40 0x7f7e300cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.994+0000 7f7e36aab640 1 --2- 172.21.15.47:0/1366667843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e3015cc30 0x7f7e300ccf70 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.994+0000 7f7e36aab640 1 -- 172.21.15.47:0/1366667843 >> 172.21.15.47:0/1366667843 conn(0x7f7e300d06f0 msgr2=0x7f7e300d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:20.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.994+0000 7f7e36aab640 1 -- 172.21.15.47:0/1366667843 shutdown_connections 2024-03-23T09:47:20.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.994+0000 7f7e36aab640 1 -- 172.21.15.47:0/1366667843 wait complete. 2024-03-23T09:47:20.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.995+0000 7f7e36aab640 1 Processor -- start 2024-03-23T09:47:20.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.995+0000 7f7e36aab640 1 -- start start 2024-03-23T09:47:20.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.996+0000 7f7e36aab640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e300b0610 0x7f7e30144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.996+0000 7f7e36aab640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e300cad40 0x7f7e300c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.996+0000 7f7e36aab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e3015cc30 0x7f7e301f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:20.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.996+0000 7f7e36aab640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300c4720 con 0x7f7e300cad40 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.996+0000 7f7e36aab640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300ae500 con 0x7f7e300b0610 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.996+0000 7f7e36aab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e301f2210 con 0x7f7e3015cc30 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.996+0000 7f7e2ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e300cad40 0x7f7e300c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.996+0000 7f7e2ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e300cad40 0x7f7e300c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38744/0 (socket says 172.21.15.47:38744) 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.996+0000 7f7e2ffff640 1 -- 172.21.15.47:0/3967445637 learned_addr learned my addr 172.21.15.47:0/3967445637 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.997+0000 7f7e34847640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e300b0610 0x7f7e30144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.997+0000 7f7e35048640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e3015cc30 0x7f7e301f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.997+0000 7f7e2ffff640 1 -- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e3015cc30 msgr2=0x7f7e301f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.997+0000 7f7e2ffff640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e3015cc30 0x7f7e301f1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.997+0000 7f7e2ffff640 1 -- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e300b0610 msgr2=0x7f7e30144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.997+0000 7f7e2ffff640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e300b0610 0x7f7e30144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.997+0000 7f7e2ffff640 1 -- 172.21.15.47:0/3967445637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e28009b30 con 0x7f7e300cad40 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.997+0000 7f7e2ffff640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e300cad40 0x7f7e300c41e0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c820 tx=0x7f7e2400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.997+0000 7f7e2d7fa640 1 -- 172.21.15.47:0/3967445637 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24007a30 con 0x7f7e300cad40 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.997+0000 7f7e2d7fa640 1 -- 172.21.15.47:0/3967445637 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e24007b90 con 0x7f7e300cad40 2024-03-23T09:47:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.997+0000 7f7e36aab640 1 -- 172.21.15.47:0/3967445637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e300c7090 con 0x7f7e300cad40 2024-03-23T09:47:21.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.997+0000 7f7e2d7fa640 1 -- 172.21.15.47:0/3967445637 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2400ce70 con 0x7f7e300cad40 2024-03-23T09:47:21.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.997+0000 7f7e36aab640 1 -- 172.21.15.47:0/3967445637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e300c6ad0 con 0x7f7e300cad40 2024-03-23T09:47:21.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.998+0000 7f7e36aab640 1 -- 172.21.15.47:0/3967445637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e30066be0 con 0x7f7e300cad40 2024-03-23T09:47:21.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.998+0000 7f7e2d7fa640 1 -- 172.21.15.47:0/3967445637 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7e240113f0 con 0x7f7e300cad40 2024-03-23T09:47:21.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.999+0000 7f7e2d7fa640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e1c06ac00 0x7f7e1c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:21.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.999+0000 7f7e2d7fa640 1 -- 172.21.15.47:0/3967445637 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e24089400 con 0x7f7e300cad40 2024-03-23T09:47:21.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:20.999+0000 7f7e34847640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e1c06ac00 0x7f7e1c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:21.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.000+0000 7f7e34847640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e1c06ac00 0x7f7e1c06d0c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7e28035000 tx=0x7f7e28030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.003+0000 7f7e2d7fa640 1 -- 172.21.15.47:0/3967445637 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7e24054720 con 0x7f7e300cad40 2024-03-23T09:47:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.180+0000 7f7e36aab640 1 -- 172.21.15.47:0/3967445637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e3006b860 con 0x7f7e300cad40 2024-03-23T09:47:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.181+0000 7f7e2d7fa640 1 -- 172.21.15.47:0/3967445637 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e24057d40 con 0x7f7e300cad40 2024-03-23T09:47:21.181 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:47:21.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.183+0000 7f7e36aab640 1 -- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e1c06ac00 msgr2=0x7f7e1c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:21.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.183+0000 7f7e36aab640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e1c06ac00 0x7f7e1c06d0c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7e28035000 tx=0x7f7e28030000 comp rx=0 tx=0).stop 2024-03-23T09:47:21.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.183+0000 7f7e36aab640 1 -- 172.21.15.47:0/3967445637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e300cad40 msgr2=0x7f7e300c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:21.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.183+0000 7f7e36aab640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e300cad40 0x7f7e300c41e0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c820 tx=0x7f7e2400cc20 comp rx=0 tx=0).stop 2024-03-23T09:47:21.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.183+0000 7f7e36aab640 1 -- 172.21.15.47:0/3967445637 shutdown_connections 2024-03-23T09:47:21.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.183+0000 7f7e36aab640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e1c06ac00 0x7f7e1c06d0c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.183+0000 7f7e36aab640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e3015cc30 0x7f7e301f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.183+0000 7f7e36aab640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e300cad40 0x7f7e300c41e0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.183+0000 7f7e36aab640 1 --2- 172.21.15.47:0/3967445637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e300b0610 0x7f7e30144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.183+0000 7f7e36aab640 1 -- 172.21.15.47:0/3967445637 >> 172.21.15.47:0/3967445637 conn(0x7f7e300d06f0 msgr2=0x7f7e300cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.183+0000 7f7e36aab640 1 -- 172.21.15.47:0/3967445637 shutdown_connections 2024-03-23T09:47:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.183+0000 7f7e36aab640 1 -- 172.21.15.47:0/3967445637 wait complete. 2024-03-23T09:47:21.193 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:47:21.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.295+0000 7f77ab363640 1 Processor -- start 2024-03-23T09:47:21.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.295+0000 7f77ab363640 1 -- start start 2024-03-23T09:47:21.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f77ab363640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f77a415cc40 0x7f77a4160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:21.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f77ab363640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f77a4160c90 0x7f77a40cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:21.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f77ab363640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a40d09d0 con 0x7f77a40cdbf0 2024-03-23T09:47:21.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f77ab363640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a40b5a80 con 0x7f77a4160c90 2024-03-23T09:47:21.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f77ab363640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a40ce000 con 0x7f77a415cc40 2024-03-23T09:47:21.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f77a88fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f77a4160c90 0x7f77a40cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:21.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f77a9900640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f77a40cdbf0 0x7f77a40b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51352/0 (socket says 172.21.15.47:51352) 2024-03-23T09:47:21.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f77a9900640 1 -- 172.21.15.47:0/1745618806 learned_addr learned my addr 172.21.15.47:0/1745618806 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:21.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f77a90ff640 1 --2- 172.21.15.47:0/1745618806 >> v2:172.21.15.175:3300/0 conn(0x7f77a415cc40 0x7f77a4160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:21.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f77a88fe640 1 -- 172.21.15.47:0/1745618806 >> v2:172.21.15.175:3300/0 conn(0x7f77a415cc40 msgr2=0x7f77a4160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f77a88fe640 1 --2- 172.21.15.47:0/1745618806 >> v2:172.21.15.175:3300/0 conn(0x7f77a415cc40 0x7f77a4160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f77a88fe640 1 -- 172.21.15.47:0/1745618806 >> v1:172.21.15.47:6789/0 conn(0x7f77a40cdbf0 legacy=0x7f77a40b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f77a88fe640 1 -- 172.21.15.47:0/1745618806 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a415d020 con 0x7f77a4160c90 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.296+0000 7f779bfff640 1 -- 172.21.15.47:0/1745618806 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338699617 0 0) 0x7f77a40d09d0 con 0x7f77a40cdbf0 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.297+0000 7f77a88fe640 1 --2- 172.21.15.47:0/1745618806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4160c90 0x7f77a40cd6b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f77a408b280 tx=0x7f779402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686e92e3452251fb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.297+0000 7f779bfff640 1 -- 172.21.15.47:0/1745618806 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794031830 con 0x7f77a4160c90 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.297+0000 7f779bfff640 1 -- 172.21.15.47:0/1745618806 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7794031990 con 0x7f77a4160c90 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.297+0000 7f779bfff640 1 -- 172.21.15.47:0/1745618806 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794031d20 con 0x7f77a4160c90 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.297+0000 7f77ab363640 1 -- 172.21.15.47:0/1745618806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4160c90 msgr2=0x7f77a40cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.297+0000 7f77ab363640 1 --2- 172.21.15.47:0/1745618806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4160c90 0x7f77a40cd6b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f77a408b280 tx=0x7f779402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.298+0000 7f77ab363640 1 -- 172.21.15.47:0/1745618806 shutdown_connections 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.298+0000 7f77ab363640 1 --2- 172.21.15.47:0/1745618806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4160c90 0x7f77a40cd6b0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.298+0000 7f77ab363640 1 --2- 172.21.15.47:0/1745618806 >> v2:172.21.15.175:3300/0 conn(0x7f77a415cc40 0x7f77a4160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.298+0000 7f77ab363640 1 -- 172.21.15.47:0/1745618806 >> 172.21.15.47:0/1745618806 conn(0x7f77a406b160 msgr2=0x7f77a406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.298+0000 7f77ab363640 1 -- 172.21.15.47:0/1745618806 shutdown_connections 2024-03-23T09:47:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.298+0000 7f77ab363640 1 -- 172.21.15.47:0/1745618806 wait complete. 2024-03-23T09:47:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.299+0000 7f77ab363640 1 Processor -- start 2024-03-23T09:47:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.299+0000 7f77ab363640 1 -- start start 2024-03-23T09:47:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.299+0000 7f77ab363640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77a40cdbf0 0x7f77a41516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.299+0000 7f77ab363640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a415cc40 0x7f77a4155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77ab363640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4160c90 0x7f77a4155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77ab363640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a4151c30 con 0x7f77a4160c90 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77ab363640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a4154a70 con 0x7f77a415cc40 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77ab363640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a4156210 con 0x7f77a40cdbf0 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77a9900640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4160c90 0x7f77a4155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77a9900640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4160c90 0x7f77a4155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38760/0 (socket says 172.21.15.47:38760) 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77a9900640 1 -- 172.21.15.47:0/3348598298 learned_addr learned my addr 172.21.15.47:0/3348598298 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77a90ff640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77a40cdbf0 0x7f77a41516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77a88fe640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a415cc40 0x7f77a4155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77a9900640 1 -- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77a40cdbf0 msgr2=0x7f77a41516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77a9900640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77a40cdbf0 0x7f77a41516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77a9900640 1 -- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a415cc40 msgr2=0x7f77a4155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77a9900640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a415cc40 0x7f77a4155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77a9900640 1 -- 172.21.15.47:0/3348598298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7794009400 con 0x7f77a4160c90 2024-03-23T09:47:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77a90ff640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77a40cdbf0 0x7f77a41516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77a9900640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4160c90 0x7f77a4155cd0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f779c00c700 tx=0x7f779c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f7799ffb640 1 -- 172.21.15.47:0/3348598298 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c00ce10 con 0x7f77a4160c90 2024-03-23T09:47:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f7799ffb640 1 -- 172.21.15.47:0/3348598298 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c00a640 con 0x7f77a4160c90 2024-03-23T09:47:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f77ab363640 1 -- 172.21.15.47:0/3348598298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a41544b0 con 0x7f77a4160c90 2024-03-23T09:47:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.300+0000 7f7799ffb640 1 -- 172.21.15.47:0/3348598298 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c00aa30 con 0x7f77a4160c90 2024-03-23T09:47:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.301+0000 7f77ab363640 1 -- 172.21.15.47:0/3348598298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a4154f90 con 0x7f77a4160c90 2024-03-23T09:47:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.301+0000 7f77ab363640 1 -- 172.21.15.47:0/3348598298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a4072360 con 0x7f77a4160c90 2024-03-23T09:47:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.302+0000 7f7799ffb640 1 -- 172.21.15.47:0/3348598298 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f779c00ab90 con 0x7f77a4160c90 2024-03-23T09:47:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.302+0000 7f7799ffb640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f778806ac00 0x7f778806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.302+0000 7f7799ffb640 1 -- 172.21.15.47:0/3348598298 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f779c088a70 con 0x7f77a4160c90 2024-03-23T09:47:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.302+0000 7f77a90ff640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f778806ac00 0x7f778806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.303+0000 7f77a90ff640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f778806ac00 0x7f778806d0c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f77a0008890 tx=0x7f77a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.306+0000 7f7799ffb640 1 -- 172.21.15.47:0/3348598298 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f779c053ce0 con 0x7f77a4160c90 2024-03-23T09:47:21.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.496+0000 7f77ab363640 1 -- 172.21.15.47:0/3348598298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f77a4066be0 con 0x7f77a4160c90 2024-03-23T09:47:21.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.497+0000 7f7799ffb640 1 -- 172.21.15.47:0/3348598298 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v85) v1 ==== 80+0+0 (secure 0 0 0) 0x7f779c018020 con 0x7f77a4160c90 2024-03-23T09:47:21.497 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:47:21.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.499+0000 7f77ab363640 1 -- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f778806ac00 msgr2=0x7f778806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:21.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.499+0000 7f77ab363640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f778806ac00 0x7f778806d0c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f77a0008890 tx=0x7f77a0008040 comp rx=0 tx=0).stop 2024-03-23T09:47:21.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.499+0000 7f77ab363640 1 -- 172.21.15.47:0/3348598298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4160c90 msgr2=0x7f77a4155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:21.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.499+0000 7f77ab363640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4160c90 0x7f77a4155cd0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f779c00c700 tx=0x7f779c00cbc0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.499+0000 7f77ab363640 1 -- 172.21.15.47:0/3348598298 shutdown_connections 2024-03-23T09:47:21.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.499+0000 7f77ab363640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f778806ac00 0x7f778806d0c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.499+0000 7f77ab363640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4160c90 0x7f77a4155cd0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.499+0000 7f77ab363640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a415cc40 0x7f77a4155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.499+0000 7f77ab363640 1 --2- 172.21.15.47:0/3348598298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77a40cdbf0 0x7f77a41516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:21.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.499+0000 7f77ab363640 1 -- 172.21.15.47:0/3348598298 >> 172.21.15.47:0/3348598298 conn(0x7f77a406b160 msgr2=0x7f77a40cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:21.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.499+0000 7f77ab363640 1 -- 172.21.15.47:0/3348598298 shutdown_connections 2024-03-23T09:47:21.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:21.499+0000 7f77ab363640 1 -- 172.21.15.47:0/3348598298 wait complete. 2024-03-23T09:47:21.509 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:47:21.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:21.510 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:47:21.534 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[67189]: fuse finished with error 0 and tester_r 0 2024-03-23T09:47:21.535 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:47:27.637 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:47:27.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:27.638 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:47:27.656 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:47:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.809+0000 7f20ef010640 1 Processor -- start 2024-03-23T09:47:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.809+0000 7f20ef010640 1 -- start start 2024-03-23T09:47:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.810+0000 7f20ef010640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f20e80b8350 0x7f20e80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.810+0000 7f20ef010640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20e80cdc40 0x7f20e80d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.810+0000 7f20ef010640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e80ce090 con 0x7f20e815ca90 2024-03-23T09:47:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.810+0000 7f20ef010640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e80d11e0 con 0x7f20e80cdc40 2024-03-23T09:47:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.810+0000 7f20ef010640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e8160f60 con 0x7f20e80b8350 2024-03-23T09:47:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.810+0000 7f20ed5ad640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20e80cdc40 0x7f20e80d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.810+0000 7f20ed5ad640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20e80cdc40 0x7f20e80d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58020/0 (socket says 172.21.15.47:58020) 2024-03-23T09:47:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.810+0000 7f20ed5ad640 1 -- 172.21.15.47:0/1753810483 learned_addr learned my addr 172.21.15.47:0/1753810483 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.810+0000 7f20dffff640 1 --2- 172.21.15.47:0/1753810483 >> v2:172.21.15.175:3300/0 conn(0x7f20e80b8350 0x7f20e80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.810+0000 7f20ed5ad640 1 -- 172.21.15.47:0/1753810483 >> v2:172.21.15.175:3300/0 conn(0x7f20e80b8350 msgr2=0x7f20e80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.810+0000 7f20ed5ad640 1 --2- 172.21.15.47:0/1753810483 >> v2:172.21.15.175:3300/0 conn(0x7f20e80b8350 0x7f20e80cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.810+0000 7f20ed5ad640 1 -- 172.21.15.47:0/1753810483 >> v1:172.21.15.175:6789/0 conn(0x7f20e815ca90 legacy=0x7f20e8160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.810+0000 7f20ed5ad640 1 -- 172.21.15.47:0/1753810483 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e8161240 con 0x7f20e80cdc40 2024-03-23T09:47:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.811+0000 7f20ed5ad640 1 --2- 172.21.15.47:0/1753810483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20e80cdc40 0x7f20e80d1670 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f20e8160670 tx=0x7f20e002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88af8643d473b1c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.811+0000 7f20df7fe640 1 -- 172.21.15.47:0/1753810483 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e002c9f0 con 0x7f20e80cdc40 2024-03-23T09:47:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.811+0000 7f20df7fe640 1 -- 172.21.15.47:0/1753810483 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e002cb50 con 0x7f20e80cdc40 2024-03-23T09:47:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.811+0000 7f20df7fe640 1 -- 172.21.15.47:0/1753810483 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e00343f0 con 0x7f20e80cdc40 2024-03-23T09:47:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.812+0000 7f20ef010640 1 -- 172.21.15.47:0/1753810483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20e80cdc40 msgr2=0x7f20e80d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.812+0000 7f20ef010640 1 --2- 172.21.15.47:0/1753810483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20e80cdc40 0x7f20e80d1670 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f20e8160670 tx=0x7f20e002be50 comp rx=0 tx=0).stop 2024-03-23T09:47:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.812+0000 7f20ef010640 1 -- 172.21.15.47:0/1753810483 shutdown_connections 2024-03-23T09:47:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.812+0000 7f20ef010640 1 --2- 172.21.15.47:0/1753810483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20e80cdc40 0x7f20e80d1670 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.812+0000 7f20ef010640 1 --2- 172.21.15.47:0/1753810483 >> v2:172.21.15.175:3300/0 conn(0x7f20e80b8350 0x7f20e80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:27.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.812+0000 7f20ef010640 1 -- 172.21.15.47:0/1753810483 >> 172.21.15.47:0/1753810483 conn(0x7f20e80659d0 msgr2=0x7f20e8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:27.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.812+0000 7f20ef010640 1 -- 172.21.15.47:0/1753810483 shutdown_connections 2024-03-23T09:47:27.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.812+0000 7f20ef010640 1 -- 172.21.15.47:0/1753810483 wait complete. 2024-03-23T09:47:27.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.813+0000 7f20ef010640 1 Processor -- start 2024-03-23T09:47:27.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.813+0000 7f20ef010640 1 -- start start 2024-03-23T09:47:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.814+0000 7f20ef010640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20e80b8350 0x7f20e80b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.814+0000 7f20ef010640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20e80cdc40 0x7f20e80c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.814+0000 7f20ef010640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815ca90 0x7f20e80ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.814+0000 7f20ef010640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e80b3610 con 0x7f20e80cdc40 2024-03-23T09:47:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.814+0000 7f20ef010640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e80b2990 con 0x7f20e80b8350 2024-03-23T09:47:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.814+0000 7f20ef010640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e80b2c70 con 0x7f20e815ca90 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.814+0000 7f20dffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20e80cdc40 0x7f20e80c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.814+0000 7f20dffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20e80cdc40 0x7f20e80c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58032/0 (socket says 172.21.15.47:58032) 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.814+0000 7f20dffff640 1 -- 172.21.15.47:0/1690079499 learned_addr learned my addr 172.21.15.47:0/1690079499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.814+0000 7f20ecdac640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20e80b8350 0x7f20e80b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.814+0000 7f20ed5ad640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815ca90 0x7f20e80ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.814+0000 7f20dffff640 1 -- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815ca90 msgr2=0x7f20e80ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.815+0000 7f20dffff640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815ca90 0x7f20e80ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.815+0000 7f20dffff640 1 -- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20e80b8350 msgr2=0x7f20e80b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.815+0000 7f20dffff640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20e80b8350 0x7f20e80b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.815+0000 7f20dffff640 1 -- 172.21.15.47:0/1690079499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e0009b30 con 0x7f20e80cdc40 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.815+0000 7f20ed5ad640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815ca90 0x7f20e80ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.815+0000 7f20dffff640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20e80cdc40 0x7f20e80c8e10 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f20d800d5f0 tx=0x7f20d800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.815+0000 7f20dd7fa640 1 -- 172.21.15.47:0/1690079499 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d80125f0 con 0x7f20e80cdc40 2024-03-23T09:47:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.815+0000 7f20dd7fa640 1 -- 172.21.15.47:0/1690079499 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8012750 con 0x7f20e80cdc40 2024-03-23T09:47:27.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.815+0000 7f20ef010640 1 -- 172.21.15.47:0/1690079499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e80b2f50 con 0x7f20e80cdc40 2024-03-23T09:47:27.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.815+0000 7f20dd7fa640 1 -- 172.21.15.47:0/1690079499 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8012b00 con 0x7f20e80cdc40 2024-03-23T09:47:27.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.815+0000 7f20ef010640 1 -- 172.21.15.47:0/1690079499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e80caca0 con 0x7f20e80cdc40 2024-03-23T09:47:27.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.816+0000 7f20ef010640 1 -- 172.21.15.47:0/1690079499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e8072360 con 0x7f20e80cdc40 2024-03-23T09:47:27.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.816+0000 7f20dd7fa640 1 -- 172.21.15.47:0/1690079499 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f20d8012c60 con 0x7f20e80cdc40 2024-03-23T09:47:27.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.817+0000 7f20dd7fa640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f20d406ac00 0x7f20d406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:27.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.817+0000 7f20dd7fa640 1 -- 172.21.15.47:0/1690079499 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20d8089940 con 0x7f20e80cdc40 2024-03-23T09:47:27.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.817+0000 7f20ecdac640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f20d406ac00 0x7f20d406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:27.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.817+0000 7f20ecdac640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f20d406ac00 0x7f20d406d0c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f20e4004ea0 tx=0x7f20e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:27.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:27.821+0000 7f20dd7fa640 1 -- 172.21.15.47:0/1690079499 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f20d8054a00 con 0x7f20e80cdc40 2024-03-23T09:47:28.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.037+0000 7f20ef010640 1 -- 172.21.15.47:0/1690079499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20e80c9350 con 0x7f20e80cdc40 2024-03-23T09:47:28.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.037+0000 7f20dd7fa640 1 -- 172.21.15.47:0/1690079499 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f20d8058020 con 0x7f20e80cdc40 2024-03-23T09:47:28.037 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:28.037 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:47:13.401354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/1927629460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1927629460},{"type":"v1","addr":"172.21.15.47:6835","nonce":1927629460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:47:28.038 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 46 2024-03-23T09:47:28.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.039+0000 7f20ef010640 1 -- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f20d406ac00 msgr2=0x7f20d406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.039+0000 7f20ef010640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f20d406ac00 0x7f20d406d0c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f20e4004ea0 tx=0x7f20e4009040 comp rx=0 tx=0).stop 2024-03-23T09:47:28.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.039+0000 7f20ef010640 1 -- 172.21.15.47:0/1690079499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20e80cdc40 msgr2=0x7f20e80c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.039+0000 7f20ef010640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20e80cdc40 0x7f20e80c8e10 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f20d800d5f0 tx=0x7f20d800dab0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.039+0000 7f20ef010640 1 -- 172.21.15.47:0/1690079499 shutdown_connections 2024-03-23T09:47:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.039+0000 7f20ef010640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f20d406ac00 0x7f20d406d0c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.039+0000 7f20ef010640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815ca90 0x7f20e80ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.039+0000 7f20ef010640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20e80cdc40 0x7f20e80c8e10 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.039+0000 7f20ef010640 1 --2- 172.21.15.47:0/1690079499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20e80b8350 0x7f20e80b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.039+0000 7f20ef010640 1 -- 172.21.15.47:0/1690079499 >> 172.21.15.47:0/1690079499 conn(0x7f20e80659d0 msgr2=0x7f20e8157780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.039+0000 7f20ef010640 1 -- 172.21.15.47:0/1690079499 shutdown_connections 2024-03-23T09:47:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.039+0000 7f20ef010640 1 -- 172.21.15.47:0/1690079499 wait complete. 2024-03-23T09:47:28.050 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-23T09:47:28.174 INFO:teuthology.orchestra.run.smithi047.stdout:{"message":"","return_code":0} 2024-03-23T09:47:28.174 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-23T09:47:28.174 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:47:28.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.275+0000 7f85a72c4640 1 Processor -- start 2024-03-23T09:47:28.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.275+0000 7f85a72c4640 1 -- start start 2024-03-23T09:47:28.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a72c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f85a015cc40 0x7f85a0160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a72c4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f85a0160c90 0x7f85a00cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a72c4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a00d09d0 con 0x7f85a00cdbf0 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a72c4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a00b5a80 con 0x7f85a015cc40 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a72c4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a00ce000 con 0x7f85a0160c90 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a5060640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f85a015cc40 0x7f85a0160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a5861640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f85a00cdbf0 0x7f85a00b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53446/0 (socket says 172.21.15.47:53446) 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a5060640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f85a015cc40 0x7f85a0160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58048/0 (socket says 172.21.15.47:58048) 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a5861640 1 -- 172.21.15.47:0/3455650247 learned_addr learned my addr 172.21.15.47:0/3455650247 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a485f640 1 --2- 172.21.15.47:0/3455650247 >> v2:172.21.15.175:3300/0 conn(0x7f85a0160c90 0x7f85a00cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a5060640 1 -- 172.21.15.47:0/3455650247 >> v2:172.21.15.175:3300/0 conn(0x7f85a0160c90 msgr2=0x7f85a00cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f859ffff640 1 -- 172.21.15.47:0/3455650247 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709905935 0 0) 0x7f85a00d09d0 con 0x7f85a00cdbf0 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a5060640 1 --2- 172.21.15.47:0/3455650247 >> v2:172.21.15.175:3300/0 conn(0x7f85a0160c90 0x7f85a00cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a5060640 1 -- 172.21.15.47:0/3455650247 >> v1:172.21.15.47:6789/0 conn(0x7f85a00cdbf0 legacy=0x7f85a00b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a5060640 1 -- 172.21.15.47:0/3455650247 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a015d020 con 0x7f85a015cc40 2024-03-23T09:47:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.276+0000 7f85a485f640 1 --2- 172.21.15.47:0/3455650247 >> v2:172.21.15.175:3300/0 conn(0x7f85a0160c90 0x7f85a00cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.277+0000 7f85a5060640 1 --2- 172.21.15.47:0/3455650247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85a015cc40 0x7f85a0160750 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f85a00cd2f0 tx=0x7f859802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff09c82dba6b678 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.277+0000 7f859ffff640 1 -- 172.21.15.47:0/3455650247 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598031830 con 0x7f85a015cc40 2024-03-23T09:47:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.277+0000 7f859ffff640 1 -- 172.21.15.47:0/3455650247 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8598031990 con 0x7f85a015cc40 2024-03-23T09:47:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.277+0000 7f859ffff640 1 -- 172.21.15.47:0/3455650247 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598031d20 con 0x7f85a015cc40 2024-03-23T09:47:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.278+0000 7f85a72c4640 1 -- 172.21.15.47:0/3455650247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85a015cc40 msgr2=0x7f85a0160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.278+0000 7f85a72c4640 1 --2- 172.21.15.47:0/3455650247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85a015cc40 0x7f85a0160750 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f85a00cd2f0 tx=0x7f859802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.278+0000 7f85a72c4640 1 -- 172.21.15.47:0/3455650247 shutdown_connections 2024-03-23T09:47:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.278+0000 7f85a72c4640 1 --2- 172.21.15.47:0/3455650247 >> v2:172.21.15.175:3300/0 conn(0x7f85a0160c90 0x7f85a00cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.278+0000 7f85a72c4640 1 --2- 172.21.15.47:0/3455650247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85a015cc40 0x7f85a0160750 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.278+0000 7f85a72c4640 1 -- 172.21.15.47:0/3455650247 >> 172.21.15.47:0/3455650247 conn(0x7f85a006b160 msgr2=0x7f85a006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.278+0000 7f85a72c4640 1 -- 172.21.15.47:0/3455650247 shutdown_connections 2024-03-23T09:47:28.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.278+0000 7f85a72c4640 1 -- 172.21.15.47:0/3455650247 wait complete. 2024-03-23T09:47:28.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.279+0000 7f85a72c4640 1 Processor -- start 2024-03-23T09:47:28.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.279+0000 7f85a72c4640 1 -- start start 2024-03-23T09:47:28.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a72c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85a00cdbf0 0x7f85a0153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a72c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85a015cc40 0x7f85a00c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a72c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85a0160c90 0x7f85a0154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a72c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0142680 con 0x7f85a015cc40 2024-03-23T09:47:28.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a72c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0151720 con 0x7f85a00cdbf0 2024-03-23T09:47:28.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a72c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00c8710 con 0x7f85a0160c90 2024-03-23T09:47:28.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a485f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85a015cc40 0x7f85a00c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a485f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85a015cc40 0x7f85a00c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58064/0 (socket says 172.21.15.47:58064) 2024-03-23T09:47:28.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a485f640 1 -- 172.21.15.47:0/2695864796 learned_addr learned my addr 172.21.15.47:0/2695864796 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:28.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a5060640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85a00cdbf0 0x7f85a0153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a485f640 1 -- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85a0160c90 msgr2=0x7f85a0154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a485f640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85a0160c90 0x7f85a0154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a485f640 1 -- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85a00cdbf0 msgr2=0x7f85a0153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a485f640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85a00cdbf0 0x7f85a0153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a485f640 1 -- 172.21.15.47:0/2695864796 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8598009400 con 0x7f85a015cc40 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a5060640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85a00cdbf0 0x7f85a0153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a485f640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85a015cc40 0x7f85a00c81d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f859400c7f0 tx=0x7f859400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f859dffb640 1 -- 172.21.15.47:0/2695864796 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8594007a30 con 0x7f85a015cc40 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f859dffb640 1 -- 172.21.15.47:0/2695864796 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8594007b90 con 0x7f85a015cc40 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.280+0000 7f85a72c4640 1 -- 172.21.15.47:0/2695864796 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a0151440 con 0x7f85a015cc40 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.281+0000 7f859dffb640 1 -- 172.21.15.47:0/2695864796 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859400ce40 con 0x7f85a015cc40 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.281+0000 7f85a72c4640 1 -- 172.21.15.47:0/2695864796 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a01422a0 con 0x7f85a015cc40 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.282+0000 7f85a72c4640 1 -- 172.21.15.47:0/2695864796 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a0072360 con 0x7f85a015cc40 2024-03-23T09:47:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.282+0000 7f859dffb640 1 -- 172.21.15.47:0/2695864796 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f85940113f0 con 0x7f85a015cc40 2024-03-23T09:47:28.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.282+0000 7f859dffb640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f858c06ac00 0x7f858c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.283+0000 7f859dffb640 1 -- 172.21.15.47:0/2695864796 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8594089540 con 0x7f85a015cc40 2024-03-23T09:47:28.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.283+0000 7f85a5060640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f858c06ac00 0x7f858c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.283+0000 7f85a5060640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f858c06ac00 0x7f858c06d0c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8598036000 tx=0x7f8598008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.287+0000 7f859dffb640 1 -- 172.21.15.47:0/2695864796 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8594054720 con 0x7f85a015cc40 2024-03-23T09:47:28.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.504+0000 7f85a72c4640 1 -- 172.21.15.47:0/2695864796 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f85a01427c0 con 0x7f85a015cc40 2024-03-23T09:47:28.505 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:47:28.505+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:47:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.511+0000 7f859dffb640 1 -- 172.21.15.47:0/2695864796 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8594057d40 con 0x7f85a015cc40 2024-03-23T09:47:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:47:28.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.513+0000 7f85a72c4640 1 -- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f858c06ac00 msgr2=0x7f858c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.513+0000 7f85a72c4640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f858c06ac00 0x7f858c06d0c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8598036000 tx=0x7f8598008030 comp rx=0 tx=0).stop 2024-03-23T09:47:28.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.513+0000 7f85a72c4640 1 -- 172.21.15.47:0/2695864796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85a015cc40 msgr2=0x7f85a00c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.513+0000 7f85a72c4640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85a015cc40 0x7f85a00c81d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f859400c7f0 tx=0x7f859400cbf0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.513+0000 7f85a72c4640 1 -- 172.21.15.47:0/2695864796 shutdown_connections 2024-03-23T09:47:28.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.513+0000 7f85a72c4640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f858c06ac00 0x7f858c06d0c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.513+0000 7f85a72c4640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85a0160c90 0x7f85a0154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.513+0000 7f85a72c4640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85a015cc40 0x7f85a00c81d0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.513+0000 7f85a72c4640 1 --2- 172.21.15.47:0/2695864796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85a00cdbf0 0x7f85a0153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.513+0000 7f85a72c4640 1 -- 172.21.15.47:0/2695864796 >> 172.21.15.47:0/2695864796 conn(0x7f85a006b160 msgr2=0x7f85a00c9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:28.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.514+0000 7f85a72c4640 1 -- 172.21.15.47:0/2695864796 shutdown_connections 2024-03-23T09:47:28.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.514+0000 7f85a72c4640 1 -- 172.21.15.47:0/2695864796 wait complete. 2024-03-23T09:47:28.517 INFO:tasks.ceph.mds.c.smithi047.stderr: -3593> 2024-03-23T09:47:17.290+0000 7f245cad0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:47:28.524 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-03-23T09:47:28.543 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:47:28.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.565+0000 7fa79164af40 1 Processor -- start 2024-03-23T09:47:28.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.565+0000 7fa79164af40 1 -- start start 2024-03-23T09:47:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.566+0000 7fa79164af40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560969404e10 0x5609694c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.566+0000 7fa79164af40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609694d7140 0x5609694d5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.566+0000 7fa79164af40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609694d7a60 0x5609694cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.566+0000 7fa79164af40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096934a800 con 0x5609694d7a60 2024-03-23T09:47:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.566+0000 7fa79164af40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096932d520 con 0x5609694d7140 2024-03-23T09:47:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.566+0000 7fa79164af40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609694d78f0 con 0x560969404e10 2024-03-23T09:47:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.566+0000 7fa78f0ca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609694d7a60 0x5609694cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.566+0000 7fa78f0ca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609694d7a60 0x5609694cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58106/0 (socket says 172.21.15.47:58106) 2024-03-23T09:47:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.566+0000 7fa78f0ca640 1 -- 172.21.15.47:0/1174914832 learned_addr learned my addr 172.21.15.47:0/1174914832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:28.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.566+0000 7fa78e0c8640 1 --2- 172.21.15.47:0/1174914832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609694d7140 0x5609694d5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.566+0000 7fa78e8c9640 1 --2- 172.21.15.47:0/1174914832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560969404e10 0x5609694c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.566+0000 7fa78f0ca640 1 -- 172.21.15.47:0/1174914832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560969404e10 msgr2=0x5609694c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:28.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa78f0ca640 1 --2- 172.21.15.47:0/1174914832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560969404e10 0x5609694c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa78f0ca640 1 -- 172.21.15.47:0/1174914832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609694d7140 msgr2=0x5609694d5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:28.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa78f0ca640 1 --2- 172.21.15.47:0/1174914832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609694d7140 0x5609694d5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa78f0ca640 1 -- 172.21.15.47:0/1174914832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78800a1c0 con 0x5609694d7a60 2024-03-23T09:47:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa78e0c8640 1 --2- 172.21.15.47:0/1174914832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609694d7140 0x5609694d5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa78e8c9640 1 --2- 172.21.15.47:0/1174914832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560969404e10 0x5609694c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa78f0ca640 1 --2- 172.21.15.47:0/1174914832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609694d7a60 0x5609694cf570 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa78801ceb0 tx=0x7fa78804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68aa7fd800b6300b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa78d0c6640 1 -- 172.21.15.47:0/1174914832 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788052ae0 con 0x5609694d7a60 2024-03-23T09:47:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa78d0c6640 1 -- 172.21.15.47:0/1174914832 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa788052c40 con 0x5609694d7a60 2024-03-23T09:47:28.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa78d0c6640 1 -- 172.21.15.47:0/1174914832 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7880525d0 con 0x5609694d7a60 2024-03-23T09:47:28.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa79164af40 1 -- 172.21.15.47:0/1174914832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609694d7a60 msgr2=0x5609694cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa79164af40 1 --2- 172.21.15.47:0/1174914832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609694d7a60 0x5609694cf570 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa78801ceb0 tx=0x7fa78804c000 comp rx=0 tx=0).stop 2024-03-23T09:47:28.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa79164af40 1 -- 172.21.15.47:0/1174914832 shutdown_connections 2024-03-23T09:47:28.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa79164af40 1 --2- 172.21.15.47:0/1174914832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609694d7a60 0x5609694cf570 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa79164af40 1 --2- 172.21.15.47:0/1174914832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609694d7140 0x5609694d5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa79164af40 1 --2- 172.21.15.47:0/1174914832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560969404e10 0x5609694c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa79164af40 1 -- 172.21.15.47:0/1174914832 >> 172.21.15.47:0/1174914832 conn(0x56096942de40 msgr2=0x5609694d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:28.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa79164af40 1 -- 172.21.15.47:0/1174914832 shutdown_connections 2024-03-23T09:47:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.567+0000 7fa79164af40 1 -- 172.21.15.47:0/1174914832 wait complete. 2024-03-23T09:47:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.568+0000 7fa79164af40 1 Processor -- start 2024-03-23T09:47:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.568+0000 7fa79164af40 1 -- start start 2024-03-23T09:47:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa79164af40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560969404e10 0x5609694c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa79164af40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609694d7140 0x5609694becb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa79164af40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5609694d7a60 0x5609694d5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa79164af40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609694d0180 con 0x560969404e10 2024-03-23T09:47:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa79164af40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609694c8830 con 0x5609694d7140 2024-03-23T09:47:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa79164af40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609694cf570 con 0x5609694d7a60 2024-03-23T09:47:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa78e8c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560969404e10 0x5609694c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa78e8c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560969404e10 0x5609694c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58112/0 (socket says 172.21.15.47:58112) 2024-03-23T09:47:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa78e8c9640 1 -- 172.21.15.47:0/672696737 learned_addr learned my addr 172.21.15.47:0/672696737 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa78f0ca640 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5609694d7a60 0x5609694d5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa78e0c8640 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609694d7140 0x5609694becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa78e8c9640 1 -- 172.21.15.47:0/672696737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5609694d7a60 msgr2=0x5609694d5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa78e8c9640 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5609694d7a60 0x5609694d5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa78e8c9640 1 -- 172.21.15.47:0/672696737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609694d7140 msgr2=0x5609694becb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa78e8c9640 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609694d7140 0x5609694becb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa78e8c9640 1 -- 172.21.15.47:0/672696737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56096934ccd0 con 0x560969404e10 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa78f0ca640 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5609694d7a60 0x5609694d5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa78e0c8640 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609694d7140 0x5609694becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa78e8c9640 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560969404e10 0x5609694c82f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fa78000bf50 tx=0x7fa7800074b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa777fff640 1 -- 172.21.15.47:0/672696737 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780010070 con 0x560969404e10 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa777fff640 1 -- 172.21.15.47:0/672696737 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa780007c60 con 0x560969404e10 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa777fff640 1 -- 172.21.15.47:0/672696737 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780014410 con 0x560969404e10 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.569+0000 7fa79164af40 1 -- 172.21.15.47:0/672696737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560969373dc0 con 0x560969404e10 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.570+0000 7fa79164af40 1 -- 172.21.15.47:0/672696737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560969359c20 con 0x560969404e10 2024-03-23T09:47:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.571+0000 7fa777fff640 1 -- 172.21.15.47:0/672696737 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa780014570 con 0x560969404e10 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.571+0000 7fa777fff640 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa77c0695e0 0x7fa77c06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.571+0000 7fa777fff640 1 -- 172.21.15.47:0/672696737 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa7800872e0 con 0x560969404e10 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.571+0000 7fa79164af40 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x5609694c6540 0x5609694cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.571+0000 7fa79164af40 1 -- 172.21.15.47:0/672696737 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e86) v8 -- 0x5609694cba10 con 0x5609694c6540 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.571+0000 7fa78e0c8640 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa77c0695e0 0x7fa77c06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.572+0000 7fa78f0ca640 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x5609694c6540 0x5609694cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.572+0000 7fa78f0ca640 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x5609694c6540 0x5609694cb4d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.572+0000 7fa78e0c8640 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa77c0695e0 0x7fa77c06baa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa7840055f0 tx=0x7fa784005780 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.575+0000 7fa78f0ca640 1 -- 172.21.15.47:0/672696737 <== osd.0 v2:172.21.15.47:6816/37679 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v86'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa788052db0 con 0x5609694c6540 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.575+0000 7fa79164af40 1 -- 172.21.15.47:0/672696737 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x5609694c6540 msgr2=0x5609694cb4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.575+0000 7fa79164af40 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x5609694c6540 0x5609694cb4d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.575+0000 7fa79164af40 1 -- 172.21.15.47:0/672696737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa77c0695e0 msgr2=0x7fa77c06baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.575+0000 7fa79164af40 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa77c0695e0 0x7fa77c06baa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa7840055f0 tx=0x7fa784005780 comp rx=0 tx=0).stop 2024-03-23T09:47:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.575+0000 7fa79164af40 1 -- 172.21.15.47:0/672696737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560969404e10 msgr2=0x5609694c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.575+0000 7fa79164af40 1 --2- 172.21.15.47:0/672696737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560969404e10 0x5609694c82f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fa78000bf50 tx=0x7fa7800074b0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.575+0000 7fa78f0ca640 1 -- 172.21.15.47:0/672696737 reap_dead start 2024-03-23T09:47:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.575+0000 7fa79164af40 1 -- 172.21.15.47:0/672696737 shutdown_connections 2024-03-23T09:47:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.575+0000 7fa79164af40 1 -- 172.21.15.47:0/672696737 >> 172.21.15.47:0/672696737 conn(0x56096942de40 msgr2=0x5609693f0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.575+0000 7fa79164af40 1 -- 172.21.15.47:0/672696737 shutdown_connections 2024-03-23T09:47:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.575+0000 7fa79164af40 1 -- 172.21.15.47:0/672696737 wait complete. 2024-03-23T09:47:28.589 DEBUG:teuthology.orchestra.run.smithi047:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-03-23T09:47:28.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.659+0000 7f4fa9c3d280 1 -- start start 2024-03-23T09:47:28.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.659+0000 7f4fa9c3d280 1 Processor -- start 2024-03-23T09:47:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa9c3d280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55735685a000 0x557356781080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa9c3d280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55735685bc00 0x557356780b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa9c3d280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575aa000 0x557356781b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa9c3d280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356717e00 con 0x55735685a000 2024-03-23T09:47:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa9c3d280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356778a00 con 0x5573575aa000 2024-03-23T09:47:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa9c3d280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557356778280 con 0x55735685bc00 2024-03-23T09:47:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa7cca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55735685a000 0x557356781080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa7cca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55735685a000 0x557356781080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58126/0 (socket says 172.21.15.47:58126) 2024-03-23T09:47:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa7cca640 1 -- 172.21.15.47:0/721837959 learned_addr learned my addr 172.21.15.47:0/721837959 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa74c9640 1 --2- 172.21.15.47:0/721837959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55735685bc00 0x557356780b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa84cb640 1 --2- 172.21.15.47:0/721837959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575aa000 0x557356781b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa7cca640 1 -- 172.21.15.47:0/721837959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55735685bc00 msgr2=0x557356780b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/721837959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55735685bc00 0x557356780b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa7cca640 1 -- 172.21.15.47:0/721837959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575aa000 msgr2=0x557356781b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/721837959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575aa000 0x557356781b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa7cca640 1 -- 172.21.15.47:0/721837959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55735670c820 con 0x55735685a000 2024-03-23T09:47:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa74c9640 1 --2- 172.21.15.47:0/721837959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55735685bc00 0x557356780b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa84cb640 1 --2- 172.21.15.47:0/721837959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575aa000 0x557356781b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/721837959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55735685a000 0x557356781080 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x557357512180 tx=0x5573574e6c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aeec3c3f7344d1f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa54c5640 1 -- 172.21.15.47:0/721837959 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5573567ab080 con 0x55735685a000 2024-03-23T09:47:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa54c5640 1 -- 172.21.15.47:0/721837959 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557357510180 con 0x55735685a000 2024-03-23T09:47:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/721837959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55735670c680 con 0x55735685a000 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/721837959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55735670c4e0 con 0x55735685a000 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa9c3d280 4 waiting for MDS map... 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.660+0000 7f4fa54c5640 1 -- 172.21.15.47:0/721837959 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5573567ab600 con 0x55735685a000 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.661+0000 7f4fa54c5640 1 -- 172.21.15.47:0/721837959 <== mon.0 v2:172.21.15.47:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x557356712380 con 0x55735685a000 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.661+0000 7f4fa54c5640 1 -- 172.21.15.47:0/721837959 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 47) v1 ==== 2573+0+0 (secure 0 0 0) 0x557356738f00 con 0x55735685a000 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.661+0000 7f4fa9c3d280 4 Got MDS map 47 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.661+0000 7f4fa9c3d280 10 main: JournalTool::main 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.661+0000 7f4fa9c3d280 4 main: JournalTool: connecting to RADOS... 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.661+0000 7f4fa9c3d280 1 Processor -- start 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.661+0000 7f4fa9c3d280 1 -- start start 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa9c3d280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5573575aa800 0x557356780580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa9c3d280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 0x557356782100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa9c3d280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575ab000 0x557356782680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa9c3d280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573567783c0 con 0x5573575aac00 2024-03-23T09:47:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa9c3d280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356778500 con 0x5573575ab000 2024-03-23T09:47:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa9c3d280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557356778780 con 0x5573575aa800 2024-03-23T09:47:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa7cca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 0x557356782100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa7cca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 0x557356782100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58128/0 (socket says 172.21.15.47:58128) 2024-03-23T09:47:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa7cca640 1 -- 172.21.15.47:0/3875721751 learned_addr learned my addr 172.21.15.47:0/3875721751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa84cb640 1 --2- 172.21.15.47:0/3875721751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5573575aa800 0x557356780580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa74c9640 1 --2- 172.21.15.47:0/3875721751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575ab000 0x557356782680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa7cca640 1 -- 172.21.15.47:0/3875721751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5573575aa800 msgr2=0x557356780580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/3875721751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5573575aa800 0x557356780580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa7cca640 1 -- 172.21.15.47:0/3875721751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575ab000 msgr2=0x557356782680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/3875721751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575ab000 0x557356782680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa7cca640 1 -- 172.21.15.47:0/3875721751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55735670c1a0 con 0x5573575aac00 2024-03-23T09:47:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa84cb640 1 --2- 172.21.15.47:0/3875721751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5573575aa800 0x557356780580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa74c9640 1 --2- 172.21.15.47:0/3875721751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575ab000 0x557356782680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/3875721751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 0x557356782100 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x5573575130e0 tx=0x5573574e6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7da5dacc31112ba server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.662+0000 7f4fa34c1640 1 -- 172.21.15.47:0/3875721751 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5573567ab1e0 con 0x5573575aac00 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.663+0000 7f4fa34c1640 1 -- 172.21.15.47:0/3875721751 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557357510a80 con 0x5573575aac00 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.663+0000 7f4fa34c1640 1 -- 172.21.15.47:0/3875721751 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5573567ab080 con 0x5573575aac00 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.663+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3875721751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 msgr2=0x557356782100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.663+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3875721751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 0x557356782100 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x5573575130e0 tx=0x5573574e6cc0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.663+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3875721751 shutdown_connections 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.663+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3875721751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575ab000 0x557356782680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.663+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3875721751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 0x557356782100 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.663+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3875721751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5573575aa800 0x557356780580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.663+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3875721751 >> 172.21.15.47:0/3875721751 conn(0x5573575aa400 msgr2=0x5573567d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.663+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3875721751 shutdown_connections 2024-03-23T09:47:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.663+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3875721751 wait complete. 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.663+0000 7f4fa9c3d280 1 Processor -- start 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.663+0000 7f4fa9c3d280 1 -- start start 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa9c3d280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575aa800 0x557356783700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa9c3d280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 0x557356783180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa9c3d280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5573575ab000 0x557356782c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa9c3d280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356778280 con 0x5573575aac00 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa9c3d280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356778a00 con 0x5573575aa800 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa9c3d280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557356778640 con 0x5573575ab000 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa7cca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 0x557356783180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa7cca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 0x557356783180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58138/0 (socket says 172.21.15.47:58138) 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa7cca640 1 -- 172.21.15.47:0/2294744153 learned_addr learned my addr 172.21.15.47:0/2294744153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa84cb640 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575aa800 0x557356783700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa74c9640 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5573575ab000 0x557356782c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa7cca640 1 -- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5573575ab000 msgr2=0x557356782c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5573575ab000 0x557356782c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa7cca640 1 -- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575aa800 msgr2=0x557356783700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575aa800 0x557356783700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.664+0000 7f4fa7cca640 1 -- 172.21.15.47:0/2294744153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55735670cb60 con 0x5573575aac00 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.665+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 0x557356783180 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x55735761a240 tx=0x5573574e6b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.665+0000 7f4fa1cbe640 1 -- 172.21.15.47:0/2294744153 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5573567ab600 con 0x5573575aac00 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.665+0000 7f4fa1cbe640 1 -- 172.21.15.47:0/2294744153 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557357510c00 con 0x5573575aac00 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.665+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/2294744153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55735670cd00 con 0x5573575aac00 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.665+0000 7f4fa1cbe640 1 -- 172.21.15.47:0/2294744153 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5573567aaf20 con 0x5573575aac00 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.665+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/2294744153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55735670c820 con 0x5573575aac00 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.665+0000 7f4fa9c3d280 4 main: JournalTool: resolving pool 12 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.666+0000 7f4fa1cbe640 1 -- 172.21.15.47:0/2294744153 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x557356757340 con 0x5573575aac00 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.666+0000 7f4fa1cbe640 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x557356a42000 0x557356a48000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.666+0000 7f4fa1cbe640 1 -- 172.21.15.47:0/2294744153 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x557356712540 con 0x5573575aac00 2024-03-23T09:47:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.666+0000 7f4fa9c3d280 4 main: JournalTool: creating IoCtx.. 2024-03-23T09:47:28.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.666+0000 7f4fa9c3d280 4 main: Executing for rank 0 2024-03-23T09:47:28.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.666+0000 7f4fa84cb640 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x557356a42000 0x557356a48000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.667+0000 7f4fa9c3d280 1 -- start start 2024-03-23T09:47:28.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.667+0000 7f4fa9c3d280 1 Processor -- start 2024-03-23T09:47:28.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.667+0000 7f4fa84cb640 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x557356a42000 0x557356a48000 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55735674fa70 tx=0x557356a54c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.667+0000 7f4fa9c3d280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557356a42800 0x557356782680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.667+0000 7f4fa9c3d280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557356a42c00 0x557356782100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.667+0000 7f4fa9c3d280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557356a43000 0x557356780580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa9c3d280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356778140 con 0x557356a42c00 2024-03-23T09:47:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa9c3d280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356778500 con 0x557356a43000 2024-03-23T09:47:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa9c3d280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573567783c0 con 0x557356a42800 2024-03-23T09:47:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa7cca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557356a42c00 0x557356782100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa7cca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557356a42c00 0x557356782100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58144/0 (socket says 172.21.15.47:58144) 2024-03-23T09:47:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa7cca640 1 -- 172.21.15.47:0/3755205428 learned_addr learned my addr 172.21.15.47:0/3755205428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa84cb640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557356a42800 0x557356782680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa74c9640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557356a43000 0x557356780580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa7cca640 1 -- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557356a42800 msgr2=0x557356782680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557356a42800 0x557356782680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa7cca640 1 -- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557356a43000 msgr2=0x557356780580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557356a43000 0x557356780580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa7cca640 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573576504e0 con 0x557356a42c00 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557356a42c00 0x557356782100 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x557356a10d80 tx=0x5573574e1380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e90a2b6391e88b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4f9ecb8640 1 -- 172.21.15.47:0/3755205428 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5573567aadc0 con 0x557356a42c00 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4f9ecb8640 1 -- 172.21.15.47:0/3755205428 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557357510180 con 0x557356a42c00 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4f9ecb8640 1 -- 172.21.15.47:0/3755205428 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5573567ab080 con 0x557356a42c00 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557357650340 con 0x557356a42c00 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55735764fba0 con 0x557356a42c00 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4fa9c3d280 4 waiting for MDS map... 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.668+0000 7f4f9ecb8640 1 -- 172.21.15.47:0/3755205428 <== mon.0 v2:172.21.15.47:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x557356712a80 con 0x557356a42c00 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.669+0000 7f4f9ecb8640 1 -- 172.21.15.47:0/3755205428 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 47) v1 ==== 2573+0+0 (secure 0 0 0) 0x557356ab6900 con 0x557356a42c00 2024-03-23T09:47:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.669+0000 7f4fa9c3d280 4 Got MDS map 47 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.669+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x557356a43400 0x557356a48580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.669+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x557356a43800 con 0x557356a43400 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.669+0000 7f4fa84cb640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x557356a43400 0x557356a48580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.669+0000 7f4fa84cb640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x557356a43400 0x557356a48580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=3beb4588b697f55d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.672+0000 7f4fa84cb640 1 -- 172.21.15.47:0/3755205428 <== osd.6 v2:172.21.15.175:6808/38160 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x5573575a4b40 con 0x557356a43400 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.673+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x5573575c5c00 0x557356a48b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.673+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x557356aca000 con 0x5573575c5c00 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.673+0000 7f4fa74c9640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x5573575c5c00 0x557356a48b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.673+0000 7f4fa74c9640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x5573575c5c00 0x557356a48b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=92a42281e11ce18a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.676+0000 7f4fa74c9640 1 -- 172.21.15.47:0/3755205428 <== osd.5 v2:172.21.15.175:6816/38161 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv19 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x5573575a4d80 con 0x5573575c5c00 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.676+0000 7f4f9d4b5640 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e86) v8 -- 0x55735685a400 con 0x5573575c5c00 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.676+0000 7f4f9d4b5640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x557356aca000 0x557356a4ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.676+0000 7f4f9d4b5640 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e86) v8 -- 0x557356ad0000 con 0x557356aca000 2024-03-23T09:47:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.676+0000 7f4fa74c9640 1 -- 172.21.15.47:0/3755205428 <== osd.5 v2:172.21.15.175:6816/38161 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x5573575a4d80 con 0x5573575c5c00 2024-03-23T09:47:28.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.676+0000 7f4fa84cb640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x557356aca000 0x557356a4ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.676+0000 7f4fa84cb640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x557356aca000 0x557356a4ac00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=b38538f810ac2505 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.677+0000 7f4fa84cb640 1 -- 172.21.15.47:0/3755205428 <== osd.4 v2:172.21.15.175:6800/38159 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5573575a4b40 con 0x557356aca000 2024-03-23T09:47:28.681 INFO:teuthology.orchestra.run.smithi047.stdout:old journal was 4209787~892 2024-03-23T09:47:28.681 INFO:teuthology.orchestra.run.smithi047.stdout:new journal start will be 8388608 (4177929 bytes past old end) 2024-03-23T09:47:28.681 INFO:teuthology.orchestra.run.smithi047.stdout:writing journal head 2024-03-23T09:47:28.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.677+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x557356aca400 con 0x5573575c5c00 2024-03-23T09:47:28.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.678+0000 7f4fa74c9640 1 -- 172.21.15.47:0/3755205428 <== osd.5 v2:172.21.15.175:6816/38161 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v86'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5573575a4d80 con 0x5573575c5c00 2024-03-23T09:47:28.682 INFO:teuthology.orchestra.run.smithi047.stdout:writing EResetJournal entry 2024-03-23T09:47:28.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.678+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x557356aca800 con 0x557356aca000 2024-03-23T09:47:28.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.678+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x557356acac00 0x557356a49080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.678+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x557356acb000 con 0x557356acac00 2024-03-23T09:47:28.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.678+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x557356acb400 con 0x557356a43400 2024-03-23T09:47:28.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.678+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x557356acb800 con 0x557356aca000 2024-03-23T09:47:28.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.678+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x557356acbc00 con 0x557356aca000 2024-03-23T09:47:28.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.678+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x557356ad0000 0x557356a49600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.678+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x557356ad0c00 con 0x557356ad0000 2024-03-23T09:47:28.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.678+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x557356acac00 0x557356a49080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.679+0000 7f4fa74c9640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x557356ad0000 0x557356a49600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.679+0000 7f4fa74c9640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x557356ad0000 0x557356a49600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=8efbbcd4267c36 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.679+0000 7f4fa7cca640 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x557356acac00 0x557356a49080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=e323fc8b92a61df3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.680+0000 7f4fa84cb640 1 -- 172.21.15.47:0/3755205428 <== osd.4 v2:172.21.15.175:6800/38159 2 ==== osd_op_reply(10 200.00000006 [delete] v86'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5573575a4b40 con 0x557356aca000 2024-03-23T09:47:28.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.680+0000 7f4fa84cb640 1 -- 172.21.15.47:0/3755205428 <== osd.4 v2:172.21.15.175:6800/38159 3 ==== osd_op_reply(6 200.00000002 [delete] v86'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5573575a4b40 con 0x557356aca000 2024-03-23T09:47:28.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.680+0000 7f4fa84cb640 1 -- 172.21.15.47:0/3755205428 <== osd.4 v2:172.21.15.175:6800/38159 4 ==== osd_op_reply(9 200.00000005 [delete] v86'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5573575a4b40 con 0x557356aca000 2024-03-23T09:47:28.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.680+0000 7f4fa84cb640 1 -- 172.21.15.47:0/3755205428 <== osd.6 v2:172.21.15.175:6808/38160 2 ==== osd_op_reply(8 200.00000004 [delete] v86'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5573575a4b40 con 0x557356a43400 2024-03-23T09:47:28.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.680+0000 7f4fa74c9640 1 -- 172.21.15.47:0/3755205428 <== osd.0 v2:172.21.15.47:6816/37679 1 ==== osd_op_reply(11 200.00000007 [delete] v86'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5573575a4d80 con 0x557356ad0000 2024-03-23T09:47:28.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.681+0000 7f4fa7cca640 1 -- 172.21.15.47:0/3755205428 <== osd.3 v2:172.21.15.47:6808/37683 1 ==== osd_op_reply(7 200.00000003 [delete] v86'4 uv3 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5573575a58c0 con 0x557356acac00 2024-03-23T09:47:28.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.681+0000 7f4f9d4b5640 1 -- 172.21.15.47:0/3755205428 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x557356ad0400 con 0x557356aca000 2024-03-23T09:47:28.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa84cb640 1 -- 172.21.15.47:0/3755205428 <== osd.4 v2:172.21.15.175:6800/38159 5 ==== osd_op_reply(12 200.00000002 [write 0~48] v86'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5573575a4b40 con 0x557356aca000 2024-03-23T09:47:28.685 INFO:teuthology.orchestra.run.smithi047.stdout:done 2024-03-23T09:47:28.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x557356ad0000 msgr2=0x557356a49600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x557356ad0000 0x557356a49600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x557356acac00 msgr2=0x557356a49080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x557356acac00 0x557356a49080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x557356aca000 msgr2=0x557356a4ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x557356aca000 0x557356a4ac00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x5573575c5c00 msgr2=0x557356a48b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x5573575c5c00 0x557356a48b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x557356a43400 msgr2=0x557356a48580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x557356a43400 0x557356a48580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa74c9640 1 -- 172.21.15.47:0/3755205428 reap_dead start 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557356a42c00 msgr2=0x557356782100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/3755205428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557356a42c00 0x557356782100 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x557356a10d80 tx=0x5573574e1380 comp rx=0 tx=0).stop 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa74c9640 1 -- 172.21.15.47:0/3755205428 reap_dead start 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa74c9640 1 -- 172.21.15.47:0/3755205428 reap_dead start 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 shutdown_connections 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 >> 172.21.15.47:0/3755205428 conn(0x557356a42400 msgr2=0x5573567d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 shutdown_connections 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.682+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/3755205428 wait complete. 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x557356a42000 msgr2=0x557356a48000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x557356a42000 0x557356a48000 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55735674fa70 tx=0x557356a54c00 comp rx=0 tx=0).stop 2024-03-23T09:47:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/2294744153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 msgr2=0x557356783180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 0x557356783180 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x55735761a240 tx=0x5573574e6b40 comp rx=0 tx=0).stop 2024-03-23T09:47:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/2294744153 shutdown_connections 2024-03-23T09:47:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x557356a42000 0x557356a48000 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5573575ab000 0x557356782c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573575aac00 0x557356783180 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/2294744153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575aa800 0x557356783700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/2294744153 >> 172.21.15.47:0/2294744153 conn(0x5573575aa400 msgr2=0x5573567d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/2294744153 shutdown_connections 2024-03-23T09:47:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/2294744153 wait complete. 2024-03-23T09:47:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/721837959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55735685a000 msgr2=0x557356781080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/721837959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55735685a000 0x557356781080 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x557357512180 tx=0x5573574e6c00 comp rx=0 tx=0).stop 2024-03-23T09:47:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/721837959 shutdown_connections 2024-03-23T09:47:28.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/721837959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573575aa000 0x557356781b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/721837959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55735685bc00 0x557356780b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 --2- 172.21.15.47:0/721837959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55735685a000 0x557356781080 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/721837959 >> 172.21.15.47:0/721837959 conn(0x5573566e7000 msgr2=0x5573567d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:28.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/721837959 shutdown_connections 2024-03-23T09:47:28.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.683+0000 7f4fa9c3d280 1 -- 172.21.15.47:0/721837959 wait complete. 2024-03-23T09:47:28.691 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.102005, result: old journal was 4209787~892 new journal start will be 8388608 (4177929 bytes past old end) writing journal head writing EResetJournal entry done 2024-03-23T09:47:28.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-23T09:47:28.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.832+0000 7fe5e7e8e640 1 Processor -- start 2024-03-23T09:47:28.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.832+0000 7fe5e7e8e640 1 -- start start 2024-03-23T09:47:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.833+0000 7fe5e7e8e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe5e00cde10 0x7fe5e0160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.833+0000 7fe5e7e8e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5e00b8350 0x7fe5e00d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.833+0000 7fe5e7e8e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e0161630 con 0x7fe5e015c990 2024-03-23T09:47:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.833+0000 7fe5e7e8e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e00d1030 con 0x7fe5e00b8350 2024-03-23T09:47:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.833+0000 7fe5e7e8e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e00cfe10 con 0x7fe5e00cde10 2024-03-23T09:47:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.833+0000 7fe5e642b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5e00b8350 0x7fe5e00d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.833+0000 7fe5e5c2a640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe5e015c990 0x7fe5e00cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53464/0 (socket says 172.21.15.47:53464) 2024-03-23T09:47:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.833+0000 7fe5e5c2a640 1 -- 172.21.15.47:0/4244340683 learned_addr learned my addr 172.21.15.47:0/4244340683 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.833+0000 7fe5e5429640 1 --2- 172.21.15.47:0/4244340683 >> v2:172.21.15.175:3300/0 conn(0x7fe5e00cde10 0x7fe5e0160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.834+0000 7fe5e642b640 1 -- 172.21.15.47:0/4244340683 >> v2:172.21.15.175:3300/0 conn(0x7fe5e00cde10 msgr2=0x7fe5e0160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.834+0000 7fe5e642b640 1 --2- 172.21.15.47:0/4244340683 >> v2:172.21.15.175:3300/0 conn(0x7fe5e00cde10 0x7fe5e0160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.834+0000 7fe5e642b640 1 -- 172.21.15.47:0/4244340683 >> v1:172.21.15.47:6789/0 conn(0x7fe5e015c990 legacy=0x7fe5e00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.834+0000 7fe5e642b640 1 -- 172.21.15.47:0/4244340683 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e00b5b60 con 0x7fe5e00b8350 2024-03-23T09:47:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.834+0000 7fe5e4c28640 1 -- 172.21.15.47:0/4244340683 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513264231 0 0) 0x7fe5e0161630 con 0x7fe5e015c990 2024-03-23T09:47:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.834+0000 7fe5e5429640 1 --2- 172.21.15.47:0/4244340683 >> v2:172.21.15.175:3300/0 conn(0x7fe5e00cde10 0x7fe5e0160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.834+0000 7fe5e642b640 1 --2- 172.21.15.47:0/4244340683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5e00b8350 0x7fe5e00d14c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe5e008ac70 tx=0x7fe5d002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f76e75cd0cb7ae server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.834+0000 7fe5e4c28640 1 -- 172.21.15.47:0/4244340683 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d002cbf0 con 0x7fe5e00b8350 2024-03-23T09:47:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.835+0000 7fe5e4c28640 1 -- 172.21.15.47:0/4244340683 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d002cd50 con 0x7fe5e00b8350 2024-03-23T09:47:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.835+0000 7fe5e4c28640 1 -- 172.21.15.47:0/4244340683 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0034660 con 0x7fe5e00b8350 2024-03-23T09:47:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.835+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/4244340683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5e00b8350 msgr2=0x7fe5e00d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.835+0000 7fe5e7e8e640 1 --2- 172.21.15.47:0/4244340683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5e00b8350 0x7fe5e00d14c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe5e008ac70 tx=0x7fe5d002be40 comp rx=0 tx=0).stop 2024-03-23T09:47:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.835+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/4244340683 shutdown_connections 2024-03-23T09:47:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.835+0000 7fe5e7e8e640 1 --2- 172.21.15.47:0/4244340683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5e00b8350 0x7fe5e00d14c0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.835+0000 7fe5e7e8e640 1 --2- 172.21.15.47:0/4244340683 >> v2:172.21.15.175:3300/0 conn(0x7fe5e00cde10 0x7fe5e0160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.835+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/4244340683 >> 172.21.15.47:0/4244340683 conn(0x7fe5e00659d0 msgr2=0x7fe5e0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.835+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/4244340683 shutdown_connections 2024-03-23T09:47:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.835+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/4244340683 wait complete. 2024-03-23T09:47:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.836+0000 7fe5e7e8e640 1 Processor -- start 2024-03-23T09:47:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.836+0000 7fe5e7e8e640 1 -- start start 2024-03-23T09:47:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e7e8e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5e00b8350 0x7fe5e0153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e7e8e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5e00cde10 0x7fe5e00b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e7e8e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5e015c990 0x7fe5e00b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e7e8e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e015e300 con 0x7fe5e015c990 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e7e8e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e015e5e0 con 0x7fe5e00cde10 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e7e8e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e015e8c0 con 0x7fe5e00b8350 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e642b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5e015c990 0x7fe5e00b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e642b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5e015c990 0x7fe5e00b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58164/0 (socket says 172.21.15.47:58164) 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e642b640 1 -- 172.21.15.47:0/2039298915 learned_addr learned my addr 172.21.15.47:0/2039298915 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e5429640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5e00cde10 0x7fe5e00b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e5c2a640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5e00b8350 0x7fe5e0153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e642b640 1 -- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5e00b8350 msgr2=0x7fe5e0153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e642b640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5e00b8350 0x7fe5e0153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e642b640 1 -- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5e00cde10 msgr2=0x7fe5e00b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e642b640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5e00cde10 0x7fe5e00b3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.837+0000 7fe5e642b640 1 -- 172.21.15.47:0/2039298915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d0009a70 con 0x7fe5e015c990 2024-03-23T09:47:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.838+0000 7fe5e642b640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5e015c990 0x7fe5e00b43f0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009e80 tx=0x7fe5d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.838+0000 7fe5de7fc640 1 -- 172.21.15.47:0/2039298915 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0040650 con 0x7fe5e015c990 2024-03-23T09:47:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.838+0000 7fe5de7fc640 1 -- 172.21.15.47:0/2039298915 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d00407d0 con 0x7fe5e015c990 2024-03-23T09:47:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.838+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/2039298915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e015eba0 con 0x7fe5e015c990 2024-03-23T09:47:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.838+0000 7fe5de7fc640 1 -- 172.21.15.47:0/2039298915 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0040b80 con 0x7fe5e015c990 2024-03-23T09:47:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.838+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/2039298915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e015a3b0 con 0x7fe5e015c990 2024-03-23T09:47:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.839+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/2039298915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e0072360 con 0x7fe5e015c990 2024-03-23T09:47:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.839+0000 7fe5de7fc640 1 -- 172.21.15.47:0/2039298915 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe5d0040ce0 con 0x7fe5e015c990 2024-03-23T09:47:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.840+0000 7fe5de7fc640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5cc06ac00 0x7fe5cc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.840+0000 7fe5de7fc640 1 -- 172.21.15.47:0/2039298915 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe5d00ac360 con 0x7fe5e015c990 2024-03-23T09:47:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.840+0000 7fe5e5c2a640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5cc06ac00 0x7fe5cc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.840+0000 7fe5e5c2a640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5cc06ac00 0x7fe5cc06d0c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8000f60 tx=0x7fe5d8007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:28.844+0000 7fe5de7fc640 1 -- 172.21.15.47:0/2039298915 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe5d0077570 con 0x7fe5e015c990 2024-03-23T09:47:29.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.064+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/2039298915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fe5e015f0c0 con 0x7fe5e015c990 2024-03-23T09:47:29.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.513+0000 7fe5de7fc640 1 -- 172.21.15.47:0/2039298915 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v48) v1 ==== 162+0+0 (secure 0 0 0) 0x7fe5e015f0c0 con 0x7fe5e015c990 2024-03-23T09:47:29.513 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-23T09:47:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.516+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5cc06ac00 msgr2=0x7fe5cc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.516+0000 7fe5e7e8e640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5cc06ac00 0x7fe5cc06d0c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8000f60 tx=0x7fe5d8007b50 comp rx=0 tx=0).stop 2024-03-23T09:47:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.516+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/2039298915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5e015c990 msgr2=0x7fe5e00b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.516+0000 7fe5e7e8e640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5e015c990 0x7fe5e00b43f0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009e80 tx=0x7fe5d0004a40 comp rx=0 tx=0).stop 2024-03-23T09:47:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.516+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/2039298915 shutdown_connections 2024-03-23T09:47:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.516+0000 7fe5e7e8e640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5cc06ac00 0x7fe5cc06d0c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.516+0000 7fe5e7e8e640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5e015c990 0x7fe5e00b43f0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.516+0000 7fe5e7e8e640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5e00cde10 0x7fe5e00b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.516+0000 7fe5e7e8e640 1 --2- 172.21.15.47:0/2039298915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5e00b8350 0x7fe5e0153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.516+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/2039298915 >> 172.21.15.47:0/2039298915 conn(0x7fe5e00659d0 msgr2=0x7fe5e00b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:29.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.516+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/2039298915 shutdown_connections 2024-03-23T09:47:29.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.516+0000 7fe5e7e8e640 1 -- 172.21.15.47:0/2039298915 wait complete. 2024-03-23T09:47:29.532 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:47:29.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.635+0000 7fa51c682640 1 Processor -- start 2024-03-23T09:47:29.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.635+0000 7fa51c682640 1 -- start start 2024-03-23T09:47:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.635+0000 7fa51c682640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa514098a00 0x7fa514098de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.635+0000 7fa51c682640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa51406bc50 0x7fa5140ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.635+0000 7fa51c682640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5140b1060 con 0x7fa5140b1680 2024-03-23T09:47:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.635+0000 7fa51c682640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa514099320 con 0x7fa51406bc50 2024-03-23T09:47:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.635+0000 7fa51c682640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5140b12c0 con 0x7fa514098a00 2024-03-23T09:47:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.636+0000 7fa51a41e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa514098a00 0x7fa514098de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.636+0000 7fa51ac1f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa51406bc50 0x7fa5140ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.636+0000 7fa51a41e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa514098a00 0x7fa514098de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:48674/0 (socket says 172.21.15.47:48674) 2024-03-23T09:47:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.636+0000 7fa51ac1f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa51406bc50 0x7fa5140ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40200/0 (socket says 172.21.15.47:40200) 2024-03-23T09:47:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.636+0000 7fa519c1d640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fa5140b1680 0x7fa5140b0950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:38594/0 (socket says 172.21.15.47:38594) 2024-03-23T09:47:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.636+0000 7fa51a41e640 1 -- 172.21.15.47:0/3853731452 learned_addr learned my addr 172.21.15.47:0/3853731452 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.636+0000 7fa51a41e640 1 -- 172.21.15.47:0/3853731452 >> v2:172.21.15.175:3300/0 conn(0x7fa51406bc50 msgr2=0x7fa5140ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.636+0000 7fa51a41e640 1 --2- 172.21.15.47:0/3853731452 >> v2:172.21.15.175:3300/0 conn(0x7fa51406bc50 0x7fa5140ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.636+0000 7fa51a41e640 1 -- 172.21.15.47:0/3853731452 >> v1:172.21.15.175:6789/0 conn(0x7fa5140b1680 legacy=0x7fa5140b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.636+0000 7fa51a41e640 1 -- 172.21.15.47:0/3853731452 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51406ae90 con 0x7fa514098a00 2024-03-23T09:47:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.636+0000 7fa51ac1f640 1 --2- 172.21.15.47:0/3853731452 >> v2:172.21.15.175:3300/0 conn(0x7fa51406bc50 0x7fa5140ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:47:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.637+0000 7fa51a41e640 1 --2- 172.21.15.47:0/3853731452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa514098a00 0x7fa514098de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa504009e50 tx=0x7fa50402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54c53bd4eea2a81f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.637+0000 7fa51941c640 1 -- 172.21.15.47:0/3853731452 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50402ca70 con 0x7fa514098a00 2024-03-23T09:47:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.637+0000 7fa51941c640 1 -- 172.21.15.47:0/3853731452 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50402cbd0 con 0x7fa514098a00 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.637+0000 7fa51941c640 1 -- 172.21.15.47:0/3853731452 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5040343f0 con 0x7fa514098a00 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.638+0000 7fa51c682640 1 -- 172.21.15.47:0/3853731452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa514098a00 msgr2=0x7fa514098de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.638+0000 7fa51c682640 1 --2- 172.21.15.47:0/3853731452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa514098a00 0x7fa514098de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa504009e50 tx=0x7fa50402bed0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.638+0000 7fa51c682640 1 -- 172.21.15.47:0/3853731452 shutdown_connections 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.638+0000 7fa51c682640 1 --2- 172.21.15.47:0/3853731452 >> v2:172.21.15.175:3300/0 conn(0x7fa51406bc50 0x7fa5140ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.638+0000 7fa51c682640 1 --2- 172.21.15.47:0/3853731452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa514098a00 0x7fa514098de0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.638+0000 7fa51c682640 1 -- 172.21.15.47:0/3853731452 >> 172.21.15.47:0/3853731452 conn(0x7fa5140b2140 msgr2=0x7fa5140b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.638+0000 7fa51c682640 1 -- 172.21.15.47:0/3853731452 shutdown_connections 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.638+0000 7fa51c682640 1 -- 172.21.15.47:0/3853731452 wait complete. 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.639+0000 7fa51c682640 1 Processor -- start 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.639+0000 7fa51c682640 1 -- start start 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51c682640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51406bc50 0x7fa5140c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51c682640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa514098a00 0x7fa5141f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51c682640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5140b1680 0x7fa51409bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51c682640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51409a4e0 con 0x7fa51406bc50 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51c682640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5140c61b0 con 0x7fa514098a00 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51c682640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5141f3330 con 0x7fa5140b1680 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51a41e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51406bc50 0x7fa5140c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51a41e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51406bc50 0x7fa5140c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58166/0 (socket says 172.21.15.47:58166) 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51a41e640 1 -- 172.21.15.47:0/755716828 learned_addr learned my addr 172.21.15.47:0/755716828 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa519c1d640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa514098a00 0x7fa5141f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51ac1f640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5140b1680 0x7fa51409bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51a41e640 1 -- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5140b1680 msgr2=0x7fa51409bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51a41e640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5140b1680 0x7fa51409bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51a41e640 1 -- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa514098a00 msgr2=0x7fa5141f2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51a41e640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa514098a00 0x7fa5141f2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51a41e640 1 -- 172.21.15.47:0/755716828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c009400 con 0x7fa51406bc50 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa51a41e640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51406bc50 0x7fa5140c5c70 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa504036040 tx=0x7fa504004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa512ffd640 1 -- 172.21.15.47:0/755716828 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504009510 con 0x7fa51406bc50 2024-03-23T09:47:29.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.640+0000 7fa512ffd640 1 -- 172.21.15.47:0/755716828 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50403e450 con 0x7fa51406bc50 2024-03-23T09:47:29.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.641+0000 7fa51c682640 1 -- 172.21.15.47:0/755716828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa504009b30 con 0x7fa51406bc50 2024-03-23T09:47:29.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.641+0000 7fa512ffd640 1 -- 172.21.15.47:0/755716828 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50403e800 con 0x7fa51406bc50 2024-03-23T09:47:29.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.641+0000 7fa51c682640 1 -- 172.21.15.47:0/755716828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5140c6660 con 0x7fa51406bc50 2024-03-23T09:47:29.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.641+0000 7fa51c682640 1 -- 172.21.15.47:0/755716828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5140b1d20 con 0x7fa51406bc50 2024-03-23T09:47:29.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.642+0000 7fa512ffd640 1 -- 172.21.15.47:0/755716828 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa50403e960 con 0x7fa51406bc50 2024-03-23T09:47:29.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.642+0000 7fa512ffd640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa50006ac00 0x7fa50006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:29.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.642+0000 7fa512ffd640 1 -- 172.21.15.47:0/755716828 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa5040abf70 con 0x7fa51406bc50 2024-03-23T09:47:29.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.643+0000 7fa519c1d640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa50006ac00 0x7fa50006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:29.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.643+0000 7fa519c1d640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa50006ac00 0x7fa50006d0c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa51406cd50 tx=0x7fa508007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:29.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.647+0000 7fa512ffd640 1 -- 172.21.15.47:0/755716828 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa5040770e0 con 0x7fa51406bc50 2024-03-23T09:47:29.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.863+0000 7fa51c682640 1 -- 172.21.15.47:0/755716828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa514066be0 con 0x7fa51406bc50 2024-03-23T09:47:29.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.864+0000 7fa512ffd640 1 -- 172.21.15.47:0/755716828 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fa50407a700 con 0x7fa51406bc50 2024-03-23T09:47:29.864 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:29.864 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:47:29.514189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7776":{"gid":7776,"name":"c","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.47:6835/4278516621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4278516621},{"type":"v1","addr":"172.21.15.47:6835","nonce":4278516621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:47:29.865 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 49 2024-03-23T09:47:29.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.866+0000 7fa51c682640 1 -- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa50006ac00 msgr2=0x7fa50006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:29.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.866+0000 7fa51c682640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa50006ac00 0x7fa50006d0c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa51406cd50 tx=0x7fa508007e80 comp rx=0 tx=0).stop 2024-03-23T09:47:29.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.866+0000 7fa51c682640 1 -- 172.21.15.47:0/755716828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51406bc50 msgr2=0x7fa5140c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:29.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.866+0000 7fa51c682640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51406bc50 0x7fa5140c5c70 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa504036040 tx=0x7fa504004a40 comp rx=0 tx=0).stop 2024-03-23T09:47:29.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.866+0000 7fa51c682640 1 -- 172.21.15.47:0/755716828 shutdown_connections 2024-03-23T09:47:29.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.866+0000 7fa51c682640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa50006ac00 0x7fa50006d0c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.866+0000 7fa51c682640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5140b1680 0x7fa51409bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.866+0000 7fa51c682640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa514098a00 0x7fa5141f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.866+0000 7fa51c682640 1 --2- 172.21.15.47:0/755716828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51406bc50 0x7fa5140c5c70 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:29.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.866+0000 7fa51c682640 1 -- 172.21.15.47:0/755716828 >> 172.21.15.47:0/755716828 conn(0x7fa5140b2140 msgr2=0x7fa5140cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:29.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.866+0000 7fa51c682640 1 -- 172.21.15.47:0/755716828 shutdown_connections 2024-03-23T09:47:29.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:29.866+0000 7fa51c682640 1 -- 172.21.15.47:0/755716828 wait complete. 2024-03-23T09:47:30.279 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:47:11.390397+0000', 'modified': '2024-03-23T09:47:29.514189+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7776}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7776': {'gid': 7776, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.47:6835/4278516621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 4278516621}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 4278516621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:47:30.279 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7776:up:replay 2024-03-23T09:47:31.281 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:47:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.394+0000 7f4095eb6640 1 Processor -- start 2024-03-23T09:47:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.394+0000 7f4095eb6640 1 -- start start 2024-03-23T09:47:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f4095eb6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f409015ca90 0x7f4090160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f4095eb6640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f40900cde10 0x7f40900d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f4095eb6640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40900d11e0 con 0x7f40900b8350 2024-03-23T09:47:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f4095eb6640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4090160f60 con 0x7f409015ca90 2024-03-23T09:47:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f4095eb6640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4090161240 con 0x7f40900cde10 2024-03-23T09:47:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f408f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f409015ca90 0x7f4090160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f408f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f409015ca90 0x7f4090160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58170/0 (socket says 172.21.15.47:58170) 2024-03-23T09:47:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f408effd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f40900b8350 0x7f40900cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53480/0 (socket says 172.21.15.47:53480) 2024-03-23T09:47:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f408f7fe640 1 -- 172.21.15.47:0/2548697800 learned_addr learned my addr 172.21.15.47:0/2548697800 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f408ffff640 1 --2- 172.21.15.47:0/2548697800 >> v2:172.21.15.175:3300/0 conn(0x7f40900cde10 0x7f40900d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f408f7fe640 1 -- 172.21.15.47:0/2548697800 >> v2:172.21.15.175:3300/0 conn(0x7f40900cde10 msgr2=0x7f40900d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f408f7fe640 1 --2- 172.21.15.47:0/2548697800 >> v2:172.21.15.175:3300/0 conn(0x7f40900cde10 0x7f40900d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f408f7fe640 1 -- 172.21.15.47:0/2548697800 >> v1:172.21.15.47:6789/0 conn(0x7f40900b8350 legacy=0x7f40900cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f408e7fc640 1 -- 172.21.15.47:0/2548697800 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383191440 0 0) 0x7f40900d11e0 con 0x7f40900b8350 2024-03-23T09:47:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f408f7fe640 1 -- 172.21.15.47:0/2548697800 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4090161520 con 0x7f409015ca90 2024-03-23T09:47:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.395+0000 7f408ffff640 1 --2- 172.21.15.47:0/2548697800 >> v2:172.21.15.175:3300/0 conn(0x7f40900cde10 0x7f40900d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.396+0000 7f408f7fe640 1 --2- 172.21.15.47:0/2548697800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409015ca90 0x7f4090160710 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f40900b5930 tx=0x7f408002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b455ec8c1a8f82 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.396+0000 7f408e7fc640 1 -- 172.21.15.47:0/2548697800 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4080031870 con 0x7f409015ca90 2024-03-23T09:47:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.396+0000 7f408e7fc640 1 -- 172.21.15.47:0/2548697800 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40800319d0 con 0x7f409015ca90 2024-03-23T09:47:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.396+0000 7f408e7fc640 1 -- 172.21.15.47:0/2548697800 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4080031d60 con 0x7f409015ca90 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.397+0000 7f4095eb6640 1 -- 172.21.15.47:0/2548697800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409015ca90 msgr2=0x7f4090160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.397+0000 7f4095eb6640 1 --2- 172.21.15.47:0/2548697800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409015ca90 0x7f4090160710 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f40900b5930 tx=0x7f408002b910 comp rx=0 tx=0).stop 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.397+0000 7f4095eb6640 1 -- 172.21.15.47:0/2548697800 shutdown_connections 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.397+0000 7f4095eb6640 1 --2- 172.21.15.47:0/2548697800 >> v2:172.21.15.175:3300/0 conn(0x7f40900cde10 0x7f40900d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.397+0000 7f4095eb6640 1 --2- 172.21.15.47:0/2548697800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409015ca90 0x7f4090160710 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.397+0000 7f4095eb6640 1 -- 172.21.15.47:0/2548697800 >> 172.21.15.47:0/2548697800 conn(0x7f40900659d0 msgr2=0x7f4090065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.397+0000 7f4095eb6640 1 -- 172.21.15.47:0/2548697800 shutdown_connections 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.397+0000 7f4095eb6640 1 -- 172.21.15.47:0/2548697800 wait complete. 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.398+0000 7f4095eb6640 1 Processor -- start 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.398+0000 7f4095eb6640 1 -- start start 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f4095eb6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40900b8350 0x7f40901560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f4095eb6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40900cde10 0x7f40901f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f4095eb6640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f409015ca90 0x7f409015e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f4095eb6640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40900c9f10 con 0x7f40900cde10 2024-03-23T09:47:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f4095eb6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40901f34c0 con 0x7f40900b8350 2024-03-23T09:47:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f4095eb6640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40901f2f00 con 0x7f409015ca90 2024-03-23T09:47:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f408effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40900cde10 0x7f40901f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f408effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40900cde10 0x7f40901f2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58178/0 (socket says 172.21.15.47:58178) 2024-03-23T09:47:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f408effd640 1 -- 172.21.15.47:0/75406344 learned_addr learned my addr 172.21.15.47:0/75406344 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f408f7fe640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40900b8350 0x7f40901560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f408ffff640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f409015ca90 0x7f409015e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f408effd640 1 -- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f409015ca90 msgr2=0x7f409015e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f408effd640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f409015ca90 0x7f409015e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f408effd640 1 -- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40900b8350 msgr2=0x7f40901560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f408effd640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40900b8350 0x7f40901560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f408effd640 1 -- 172.21.15.47:0/75406344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40800094d0 con 0x7f40900cde10 2024-03-23T09:47:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f408f7fe640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40900b8350 0x7f40901560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:31.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.399+0000 7f408effd640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40900cde10 0x7f40901f2510 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f408400c4d0 tx=0x7f408400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:31.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.400+0000 7f4077fff640 1 -- 172.21.15.47:0/75406344 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40840074e0 con 0x7f40900cde10 2024-03-23T09:47:31.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.400+0000 7f4077fff640 1 -- 172.21.15.47:0/75406344 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4084007660 con 0x7f40900cde10 2024-03-23T09:47:31.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.400+0000 7f4077fff640 1 -- 172.21.15.47:0/75406344 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084004a20 con 0x7f40900cde10 2024-03-23T09:47:31.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.400+0000 7f4095eb6640 1 -- 172.21.15.47:0/75406344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40901f31e0 con 0x7f40900cde10 2024-03-23T09:47:31.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.400+0000 7f4095eb6640 1 -- 172.21.15.47:0/75406344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40901f2a50 con 0x7f40900cde10 2024-03-23T09:47:31.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.401+0000 7f4095eb6640 1 -- 172.21.15.47:0/75406344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40901f2cd0 con 0x7f40900cde10 2024-03-23T09:47:31.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.401+0000 7f4077fff640 1 -- 172.21.15.47:0/75406344 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4084004b80 con 0x7f40900cde10 2024-03-23T09:47:31.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.401+0000 7f4077fff640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f407c06ac00 0x7f407c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:31.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.402+0000 7f4077fff640 1 -- 172.21.15.47:0/75406344 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f40840885d0 con 0x7f40900cde10 2024-03-23T09:47:31.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.402+0000 7f408f7fe640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f407c06ac00 0x7f407c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:31.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.402+0000 7f408f7fe640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f407c06ac00 0x7f407c06d0c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4080036000 tx=0x7f408003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:31.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.406+0000 7f4077fff640 1 -- 172.21.15.47:0/75406344 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4084053740 con 0x7f40900cde10 2024-03-23T09:47:31.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.622+0000 7f4095eb6640 1 -- 172.21.15.47:0/75406344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4090072360 con 0x7f40900cde10 2024-03-23T09:47:31.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.622+0000 7f4077fff640 1 -- 172.21.15.47:0/75406344 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f4084056d60 con 0x7f40900cde10 2024-03-23T09:47:31.623 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:31.623 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:47:31.521222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7776":{"gid":7776,"name":"c","rank":0,"incarnation":49,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.47:6835/4278516621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4278516621},{"type":"v1","addr":"172.21.15.47:6835","nonce":4278516621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:47:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 51 2024-03-23T09:47:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.624+0000 7f4095eb6640 1 -- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f407c06ac00 msgr2=0x7f407c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.624+0000 7f4095eb6640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f407c06ac00 0x7f407c06d0c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4080036000 tx=0x7f408003c040 comp rx=0 tx=0).stop 2024-03-23T09:47:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.624+0000 7f4095eb6640 1 -- 172.21.15.47:0/75406344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40900cde10 msgr2=0x7f40901f2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.624+0000 7f4095eb6640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40900cde10 0x7f40901f2510 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f408400c4d0 tx=0x7f408400c990 comp rx=0 tx=0).stop 2024-03-23T09:47:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.624+0000 7f4095eb6640 1 -- 172.21.15.47:0/75406344 shutdown_connections 2024-03-23T09:47:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.624+0000 7f4095eb6640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f407c06ac00 0x7f407c06d0c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.624+0000 7f4095eb6640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f409015ca90 0x7f409015e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.624+0000 7f4095eb6640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40900cde10 0x7f40901f2510 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.624+0000 7f4095eb6640 1 --2- 172.21.15.47:0/75406344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40900b8350 0x7f40901560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.624+0000 7f4095eb6640 1 -- 172.21.15.47:0/75406344 >> 172.21.15.47:0/75406344 conn(0x7f40900659d0 msgr2=0x7f409015f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.624+0000 7f4095eb6640 1 -- 172.21.15.47:0/75406344 shutdown_connections 2024-03-23T09:47:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:31.624+0000 7f4095eb6640 1 -- 172.21.15.47:0/75406344 wait complete. 2024-03-23T09:47:32.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:47:11.390397+0000', 'modified': '2024-03-23T09:47:31.521222+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7776}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7776': {'gid': 7776, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.47:6835/4278516621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 4278516621}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 4278516621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:47:32.038 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7776:up:rejoin 2024-03-23T09:47:33.039 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:47:33.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.152+0000 7fa3f462c640 1 Processor -- start 2024-03-23T09:47:33.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.152+0000 7fa3f462c640 1 -- start start 2024-03-23T09:47:33.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f462c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3ec0b8350 0x7fa3ec0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f462c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa3ec0cdc40 0x7fa3ec0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f462c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec0ce090 con 0x7fa3ec15ca90 2024-03-23T09:47:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f462c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ec0d11e0 con 0x7fa3ec0b8350 2024-03-23T09:47:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f462c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ec160f60 con 0x7fa3ec0cdc40 2024-03-23T09:47:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f1bc7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3ec0b8350 0x7fa3ec0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f23c8640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa3ec15ca90 0x7fa3ec160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53492/0 (socket says 172.21.15.47:53492) 2024-03-23T09:47:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f23c8640 1 -- 172.21.15.47:0/3642754711 learned_addr learned my addr 172.21.15.47:0/3642754711 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f1bc7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3ec0b8350 0x7fa3ec0cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58184/0 (socket says 172.21.15.47:58184) 2024-03-23T09:47:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f2bc9640 1 --2- 172.21.15.47:0/3642754711 >> v2:172.21.15.175:3300/0 conn(0x7fa3ec0cdc40 0x7fa3ec0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f1bc7640 1 -- 172.21.15.47:0/3642754711 >> v2:172.21.15.175:3300/0 conn(0x7fa3ec0cdc40 msgr2=0x7fa3ec0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f1bc7640 1 --2- 172.21.15.47:0/3642754711 >> v2:172.21.15.175:3300/0 conn(0x7fa3ec0cdc40 0x7fa3ec0d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f1bc7640 1 -- 172.21.15.47:0/3642754711 >> v1:172.21.15.47:6789/0 conn(0x7fa3ec15ca90 legacy=0x7fa3ec160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f13c6640 1 -- 172.21.15.47:0/3642754711 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315392304 0 0) 0x7fa3e4001110 con 0x7fa3ec15ca90 2024-03-23T09:47:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.153+0000 7fa3f1bc7640 1 -- 172.21.15.47:0/3642754711 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec161240 con 0x7fa3ec0b8350 2024-03-23T09:47:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.154+0000 7fa3f1bc7640 1 --2- 172.21.15.47:0/3642754711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ec0b8350 0x7fa3ec0cd700 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0b5930 tx=0x7fa3e002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0ddfcd9f8a921 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.154+0000 7fa3f13c6640 1 -- 172.21.15.47:0/3642754711 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e002e070 con 0x7fa3ec0b8350 2024-03-23T09:47:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.154+0000 7fa3f13c6640 1 -- 172.21.15.47:0/3642754711 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e00315a0 con 0x7fa3ec0b8350 2024-03-23T09:47:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.154+0000 7fa3f13c6640 1 -- 172.21.15.47:0/3642754711 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0031930 con 0x7fa3ec0b8350 2024-03-23T09:47:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.155+0000 7fa3f462c640 1 -- 172.21.15.47:0/3642754711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ec0b8350 msgr2=0x7fa3ec0cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.155+0000 7fa3f462c640 1 --2- 172.21.15.47:0/3642754711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ec0b8350 0x7fa3ec0cd700 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0b5930 tx=0x7fa3e002b840 comp rx=0 tx=0).stop 2024-03-23T09:47:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.155+0000 7fa3f462c640 1 -- 172.21.15.47:0/3642754711 shutdown_connections 2024-03-23T09:47:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.155+0000 7fa3f462c640 1 --2- 172.21.15.47:0/3642754711 >> v2:172.21.15.175:3300/0 conn(0x7fa3ec0cdc40 0x7fa3ec0d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.155+0000 7fa3f462c640 1 --2- 172.21.15.47:0/3642754711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ec0b8350 0x7fa3ec0cd700 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.155+0000 7fa3f462c640 1 -- 172.21.15.47:0/3642754711 >> 172.21.15.47:0/3642754711 conn(0x7fa3ec0659d0 msgr2=0x7fa3ec065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.155+0000 7fa3f462c640 1 -- 172.21.15.47:0/3642754711 shutdown_connections 2024-03-23T09:47:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.155+0000 7fa3f462c640 1 -- 172.21.15.47:0/3642754711 wait complete. 2024-03-23T09:47:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.156+0000 7fa3f462c640 1 Processor -- start 2024-03-23T09:47:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.156+0000 7fa3f462c640 1 -- start start 2024-03-23T09:47:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.157+0000 7fa3f462c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ec0b8350 0x7fa3ec148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.157+0000 7fa3f462c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3ec0cdc40 0x7fa3ec1470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.157+0000 7fa3f462c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3ec15ca90 0x7fa3ec0cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.157+0000 7fa3f462c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec147a40 con 0x7fa3ec0b8350 2024-03-23T09:47:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.157+0000 7fa3f462c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec147d20 con 0x7fa3ec0cdc40 2024-03-23T09:47:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.157+0000 7fa3f462c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec148000 con 0x7fa3ec15ca90 2024-03-23T09:47:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.157+0000 7fa3f23c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ec0b8350 0x7fa3ec148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.157+0000 7fa3f23c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ec0b8350 0x7fa3ec148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58200/0 (socket says 172.21.15.47:58200) 2024-03-23T09:47:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.157+0000 7fa3f23c8640 1 -- 172.21.15.47:0/2228631849 learned_addr learned my addr 172.21.15.47:0/2228631849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.157+0000 7fa3f1bc7640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3ec0cdc40 0x7fa3ec1470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.157+0000 7fa3f23c8640 1 -- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3ec15ca90 msgr2=0x7fa3ec0cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.158+0000 7fa3f2bc9640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3ec15ca90 0x7fa3ec0cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.158+0000 7fa3f23c8640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3ec15ca90 0x7fa3ec0cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.158+0000 7fa3f23c8640 1 -- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3ec0cdc40 msgr2=0x7fa3ec1470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.158+0000 7fa3f23c8640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3ec0cdc40 0x7fa3ec1470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.158+0000 7fa3f23c8640 1 -- 172.21.15.47:0/2228631849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e0009400 con 0x7fa3ec0b8350 2024-03-23T09:47:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.158+0000 7fa3f1bc7640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3ec0cdc40 0x7fa3ec1470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.158+0000 7fa3f23c8640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ec0b8350 0x7fa3ec148ae0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400c780 tx=0x7fa3e400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.158+0000 7fa3eaffd640 1 -- 172.21.15.47:0/2228631849 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40072d0 con 0x7fa3ec0b8350 2024-03-23T09:47:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.158+0000 7fa3eaffd640 1 -- 172.21.15.47:0/2228631849 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e4007450 con 0x7fa3ec0b8350 2024-03-23T09:47:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.158+0000 7fa3eaffd640 1 -- 172.21.15.47:0/2228631849 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40047e0 con 0x7fa3ec0b8350 2024-03-23T09:47:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.158+0000 7fa3f462c640 1 -- 172.21.15.47:0/2228631849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec1482e0 con 0x7fa3ec0b8350 2024-03-23T09:47:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.158+0000 7fa3f462c640 1 -- 172.21.15.47:0/2228631849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec149020 con 0x7fa3ec0b8350 2024-03-23T09:47:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.159+0000 7fa3f462c640 1 -- 172.21.15.47:0/2228631849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec072360 con 0x7fa3ec0b8350 2024-03-23T09:47:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.159+0000 7fa3eaffd640 1 -- 172.21.15.47:0/2228631849 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa3e4004960 con 0x7fa3ec0b8350 2024-03-23T09:47:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.160+0000 7fa3eaffd640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa3d806ac00 0x7fa3d806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.160+0000 7fa3eaffd640 1 -- 172.21.15.47:0/2228631849 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa3e4088480 con 0x7fa3ec0b8350 2024-03-23T09:47:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.160+0000 7fa3f1bc7640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa3d806ac00 0x7fa3d806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.161+0000 7fa3f1bc7640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa3d806ac00 0x7fa3d806d0c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0036000 tx=0x7fa3e003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:33.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.164+0000 7fa3eaffd640 1 -- 172.21.15.47:0/2228631849 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa3e4053690 con 0x7fa3ec0b8350 2024-03-23T09:47:33.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.381+0000 7fa3f462c640 1 -- 172.21.15.47:0/2228631849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3ec0ce090 con 0x7fa3ec0b8350 2024-03-23T09:47:33.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.382+0000 7fa3eaffd640 1 -- 172.21.15.47:0/2228631849 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fa3ec0ce090 con 0x7fa3ec0b8350 2024-03-23T09:47:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:47:33.382 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:47:32.528346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7776":{"gid":7776,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6835/4278516621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4278516621},{"type":"v1","addr":"172.21.15.47:6835","nonce":4278516621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:47:33.382 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 52 2024-03-23T09:47:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.383+0000 7fa3f462c640 1 -- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa3d806ac00 msgr2=0x7fa3d806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.383+0000 7fa3f462c640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa3d806ac00 0x7fa3d806d0c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0036000 tx=0x7fa3e003d040 comp rx=0 tx=0).stop 2024-03-23T09:47:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.383+0000 7fa3f462c640 1 -- 172.21.15.47:0/2228631849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ec0b8350 msgr2=0x7fa3ec148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.383+0000 7fa3f462c640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ec0b8350 0x7fa3ec148ae0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400c780 tx=0x7fa3e400cc40 comp rx=0 tx=0).stop 2024-03-23T09:47:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.383+0000 7fa3f462c640 1 -- 172.21.15.47:0/2228631849 shutdown_connections 2024-03-23T09:47:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.383+0000 7fa3f462c640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa3d806ac00 0x7fa3d806d0c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.383+0000 7fa3f462c640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3ec15ca90 0x7fa3ec0cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.383+0000 7fa3f462c640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3ec0cdc40 0x7fa3ec1470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.383+0000 7fa3f462c640 1 --2- 172.21.15.47:0/2228631849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3ec0b8350 0x7fa3ec148ae0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.384+0000 7fa3f462c640 1 -- 172.21.15.47:0/2228631849 >> 172.21.15.47:0/2228631849 conn(0x7fa3ec0659d0 msgr2=0x7fa3ec0b74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.384+0000 7fa3f462c640 1 -- 172.21.15.47:0/2228631849 shutdown_connections 2024-03-23T09:47:33.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.384+0000 7fa3f462c640 1 -- 172.21.15.47:0/2228631849 wait complete. 2024-03-23T09:47:33.798 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:47:11.390397+0000', 'modified': '2024-03-23T09:47:32.528346+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7776}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7776': {'gid': 7776, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.47:6835/4278516621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 4278516621}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 4278516621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:47:33.798 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:47:33.798 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T09:47:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.904+0000 7f106c436640 1 Processor -- start 2024-03-23T09:47:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.904+0000 7f106c436640 1 -- start start 2024-03-23T09:47:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.905+0000 7f106c436640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f10640cf210 0x7f10640cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.905+0000 7f106c436640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10640b5a90 0x7f10640b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.905+0000 7f106c436640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10640cfbd0 con 0x7f106415cd00 2024-03-23T09:47:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.905+0000 7f106c436640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106415d0e0 con 0x7f10640b5a90 2024-03-23T09:47:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.905+0000 7f106c436640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10640d1540 con 0x7f10640cf210 2024-03-23T09:47:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.906+0000 7f106a9d3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10640b5a90 0x7f10640b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.906+0000 7f106a1d2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f106415cd00 0x7f10640d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53500/0 (socket says 172.21.15.47:53500) 2024-03-23T09:47:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.906+0000 7f106a1d2640 1 -- 172.21.15.47:0/656506094 learned_addr learned my addr 172.21.15.47:0/656506094 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.906+0000 7f10699d1640 1 --2- 172.21.15.47:0/656506094 >> v2:172.21.15.175:3300/0 conn(0x7f10640cf210 0x7f10640cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.906+0000 7f106a9d3640 1 -- 172.21.15.47:0/656506094 >> v2:172.21.15.175:3300/0 conn(0x7f10640cf210 msgr2=0x7f10640cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.906+0000 7f106a9d3640 1 --2- 172.21.15.47:0/656506094 >> v2:172.21.15.175:3300/0 conn(0x7f10640cf210 0x7f10640cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.906+0000 7f106a9d3640 1 -- 172.21.15.47:0/656506094 >> v1:172.21.15.47:6789/0 conn(0x7f106415cd00 legacy=0x7f10640d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.906+0000 7f10691d0640 1 -- 172.21.15.47:0/656506094 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294480870 0 0) 0x7f10640cfbd0 con 0x7f106415cd00 2024-03-23T09:47:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.906+0000 7f106a9d3640 1 -- 172.21.15.47:0/656506094 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10640d1820 con 0x7f10640b5a90 2024-03-23T09:47:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.906+0000 7f10699d1640 1 --2- 172.21.15.47:0/656506094 >> v2:172.21.15.175:3300/0 conn(0x7f10640cf210 0x7f10640cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.907+0000 7f106a9d3640 1 --2- 172.21.15.47:0/656506094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10640b5a90 0x7f10640b5ee0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f10640b5520 tx=0x7f105c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59c574d7fe0ac61 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.907+0000 7f10691d0640 1 -- 172.21.15.47:0/656506094 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c02cbf0 con 0x7f10640b5a90 2024-03-23T09:47:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.907+0000 7f10691d0640 1 -- 172.21.15.47:0/656506094 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c02cd50 con 0x7f10640b5a90 2024-03-23T09:47:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.907+0000 7f10691d0640 1 -- 172.21.15.47:0/656506094 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c034660 con 0x7f10640b5a90 2024-03-23T09:47:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.907+0000 7f106c436640 1 -- 172.21.15.47:0/656506094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10640b5a90 msgr2=0x7f10640b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.907+0000 7f106c436640 1 --2- 172.21.15.47:0/656506094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10640b5a90 0x7f10640b5ee0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f10640b5520 tx=0x7f105c02be40 comp rx=0 tx=0).stop 2024-03-23T09:47:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.907+0000 7f106c436640 1 -- 172.21.15.47:0/656506094 shutdown_connections 2024-03-23T09:47:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.907+0000 7f106c436640 1 --2- 172.21.15.47:0/656506094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10640b5a90 0x7f10640b5ee0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.907+0000 7f106c436640 1 --2- 172.21.15.47:0/656506094 >> v2:172.21.15.175:3300/0 conn(0x7f10640cf210 0x7f10640cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.907+0000 7f106c436640 1 -- 172.21.15.47:0/656506094 >> 172.21.15.47:0/656506094 conn(0x7f106406b160 msgr2=0x7f106406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.907+0000 7f106c436640 1 -- 172.21.15.47:0/656506094 shutdown_connections 2024-03-23T09:47:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.907+0000 7f106c436640 1 -- 172.21.15.47:0/656506094 wait complete. 2024-03-23T09:47:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.908+0000 7f106c436640 1 Processor -- start 2024-03-23T09:47:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.909+0000 7f106c436640 1 -- start start 2024-03-23T09:47:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.909+0000 7f106c436640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10640b5a90 0x7f10641571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.909+0000 7f106c436640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10640cf210 0x7f10640ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.909+0000 7f106c436640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f106415cd00 0x7f10640caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.909+0000 7f106c436640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106415a900 con 0x7f10640cf210 2024-03-23T09:47:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.909+0000 7f106c436640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1064159690 con 0x7f106415cd00 2024-03-23T09:47:33.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.909+0000 7f106c436640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106415abc0 con 0x7f10640b5a90 2024-03-23T09:47:33.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.909+0000 7f10699d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10640cf210 0x7f10640ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.909+0000 7f10699d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10640cf210 0x7f10640ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58210/0 (socket says 172.21.15.47:58210) 2024-03-23T09:47:33.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.909+0000 7f10699d1640 1 -- 172.21.15.47:0/2509682807 learned_addr learned my addr 172.21.15.47:0/2509682807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:33.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f106a9d3640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f106415cd00 0x7f10640caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f106a1d2640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10640b5a90 0x7f10641571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f10699d1640 1 -- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10640b5a90 msgr2=0x7f10641571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f10699d1640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10640b5a90 0x7f10641571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f10699d1640 1 -- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f106415cd00 msgr2=0x7f10640caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f10699d1640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f106415cd00 0x7f10640caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f10699d1640 1 -- 172.21.15.47:0/2509682807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c009a70 con 0x7f10640cf210 2024-03-23T09:47:33.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f106a1d2640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10640b5a90 0x7f10641571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:33.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f10699d1640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10640cf210 0x7f10640ca500 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f105400c7f0 tx=0x7f105400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:33.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f105affd640 1 -- 172.21.15.47:0/2509682807 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054007a30 con 0x7f10640cf210 2024-03-23T09:47:33.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f105affd640 1 -- 172.21.15.47:0/2509682807 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1054007b90 con 0x7f10640cf210 2024-03-23T09:47:33.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f105affd640 1 -- 172.21.15.47:0/2509682807 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105400ce40 con 0x7f10640cf210 2024-03-23T09:47:33.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f106c436640 1 -- 172.21.15.47:0/2509682807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106415aea0 con 0x7f10640cf210 2024-03-23T09:47:33.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.910+0000 7f106c436640 1 -- 172.21.15.47:0/2509682807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1064157730 con 0x7f10640cf210 2024-03-23T09:47:33.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.911+0000 7f105affd640 1 -- 172.21.15.47:0/2509682807 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f10540113f0 con 0x7f10640cf210 2024-03-23T09:47:33.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.912+0000 7f105affd640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f104806ac00 0x7f104806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.912+0000 7f105affd640 1 -- 172.21.15.47:0/2509682807 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1054089460 con 0x7f10640cf210 2024-03-23T09:47:33.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.912+0000 7f106a1d2640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f104806ac00 0x7f104806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.913+0000 7f106a1d2640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f104806ac00 0x7f104806d0c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1060000f60 tx=0x7f1060007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.913+0000 7f106efa8740 1 Processor -- start 2024-03-23T09:47:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.913+0000 7f106efa8740 1 -- start start 2024-03-23T09:47:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106efa8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f646750 0x55746f5fdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106efa8740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f5fc7d0 0x55746f5fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106efa8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f5f8c70 0x55746f628a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106efa8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746f5699b0 con 0x55746f5f8c70 2024-03-23T09:47:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106efa8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746f567140 con 0x55746f646750 2024-03-23T09:47:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106efa8740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55746f562a30 con 0x55746f5fc7d0 2024-03-23T09:47:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106a9d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f5f8c70 0x55746f628a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106a9d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f5f8c70 0x55746f628a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58220/0 (socket says 172.21.15.47:58220) 2024-03-23T09:47:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106a9d3640 1 -- 172.21.15.47:0/365826745 learned_addr learned my addr 172.21.15.47:0/365826745 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106a1d2640 1 --2- 172.21.15.47:0/365826745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f646750 0x55746f5fdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f10699d1640 1 --2- 172.21.15.47:0/365826745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f5fc7d0 0x55746f5fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106a9d3640 1 -- 172.21.15.47:0/365826745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f5fc7d0 msgr2=0x55746f5fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106a9d3640 1 --2- 172.21.15.47:0/365826745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f5fc7d0 0x55746f5fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106a9d3640 1 -- 172.21.15.47:0/365826745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f646750 msgr2=0x55746f5fdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106a9d3640 1 --2- 172.21.15.47:0/365826745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f646750 0x55746f5fdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106a9d3640 1 -- 172.21.15.47:0/365826745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55746f4ef2f0 con 0x55746f5f8c70 2024-03-23T09:47:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f10699d1640 1 --2- 172.21.15.47:0/365826745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f5fc7d0 0x55746f5fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.914+0000 7f106a9d3640 1 --2- 172.21.15.47:0/365826745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f5f8c70 0x55746f628a60 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f105c009e80 tx=0x7f105c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d3252f99115f87 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.915+0000 7f1047fff640 1 -- 172.21.15.47:0/365826745 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c034640 con 0x55746f5f8c70 2024-03-23T09:47:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.915+0000 7f1047fff640 1 -- 172.21.15.47:0/365826745 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c0347a0 con 0x55746f5f8c70 2024-03-23T09:47:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.915+0000 7f1047fff640 1 -- 172.21.15.47:0/365826745 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c034b30 con 0x55746f5f8c70 2024-03-23T09:47:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.915+0000 7f106efa8740 1 -- 172.21.15.47:0/365826745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f5f8c70 msgr2=0x55746f628a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.915+0000 7f106efa8740 1 --2- 172.21.15.47:0/365826745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f5f8c70 0x55746f628a60 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f105c009e80 tx=0x7f105c02e000 comp rx=0 tx=0).stop 2024-03-23T09:47:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.915+0000 7f106efa8740 1 -- 172.21.15.47:0/365826745 shutdown_connections 2024-03-23T09:47:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.915+0000 7f106efa8740 1 --2- 172.21.15.47:0/365826745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f5f8c70 0x55746f628a60 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.915+0000 7f106efa8740 1 --2- 172.21.15.47:0/365826745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f5fc7d0 0x55746f5fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.915+0000 7f106efa8740 1 --2- 172.21.15.47:0/365826745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f646750 0x55746f5fdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.915+0000 7f106efa8740 1 -- 172.21.15.47:0/365826745 >> 172.21.15.47:0/365826745 conn(0x55746f5faee0 msgr2=0x55746f643a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.915+0000 7f106efa8740 1 -- 172.21.15.47:0/365826745 shutdown_connections 2024-03-23T09:47:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.915+0000 7f106efa8740 1 -- 172.21.15.47:0/365826745 wait complete. 2024-03-23T09:47:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.916+0000 7f106efa8740 1 -- start start 2024-03-23T09:47:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.916+0000 7f106efa8740 1 Processor -- start 2024-03-23T09:47:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106efa8740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f5fc7d0 0x55746f64b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106efa8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f646750 0x55746f62cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106efa8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f6210b0 0x55746f6254f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106efa8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746f521090 con 0x55746f6210b0 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106efa8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746f520d30 con 0x55746f646750 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106efa8740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55746f4e62a0 con 0x55746f5fc7d0 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106efa8740 10 client.0 _ll_register_callbacks cb 0x55746f51c050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106a9d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f6210b0 0x55746f6254f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106a9d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f6210b0 0x55746f6254f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58228/0 (socket says 172.21.15.47:58228) 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106a9d3640 1 -- 172.21.15.47:0/1648580115 learned_addr learned my addr 172.21.15.47:0/1648580115 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f10699d1640 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f646750 0x55746f62cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106a1d2640 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f5fc7d0 0x55746f64b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106a9d3640 1 -- 172.21.15.47:0/1648580115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f5fc7d0 msgr2=0x55746f64b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106a9d3640 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f5fc7d0 0x55746f64b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106a9d3640 1 -- 172.21.15.47:0/1648580115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f646750 msgr2=0x55746f62cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106a9d3640 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f646750 0x55746f62cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106a9d3640 1 -- 172.21.15.47:0/1648580115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10640d1820 con 0x55746f6210b0 2024-03-23T09:47:33.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f10699d1640 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f646750 0x55746f62cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:33.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f106a9d3640 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f6210b0 0x55746f6254f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f105c02c450 tx=0x7f105c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbc4c2bbf699b5b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:33.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f10457fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:47:33.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f10457fa640 1 -- 172.21.15.47:0/1648580115 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c003da0 con 0x55746f6210b0 2024-03-23T09:47:33.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.917+0000 7f10457fa640 1 -- 172.21.15.47:0/1648580115 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f105c04f030 con 0x55746f6210b0 2024-03-23T09:47:33.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f106efa8740 1 -- 172.21.15.47:0/1648580115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55746f2563c0 con 0x55746f6210b0 2024-03-23T09:47:33.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f10457fa640 1 -- 172.21.15.47:0/1648580115 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f105c004030 con 0x55746f6210b0 2024-03-23T09:47:33.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f106efa8740 10 client.7845 fetch_fsmap learned FSMap version 52 2024-03-23T09:47:33.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f106efa8740 1 -- 172.21.15.47:0/1648580115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55746f5610b0 con 0x55746f6210b0 2024-03-23T09:47:33.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f10457fa640 1 -- 172.21.15.47:0/1648580115 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55746f2563c0 con 0x55746f6210b0 2024-03-23T09:47:33.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f10457fa640 1 -- 172.21.15.47:0/1648580115 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 52) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f105c004190 con 0x55746f6210b0 2024-03-23T09:47:33.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f106efa8740 10 client.7845 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T09:47:33.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f106efa8740 10 client.7845 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:47:33.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f106efa8740 20 client.7845 populate_metadata read hostname 'smithi047' 2024-03-23T09:47:33.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f106efa8740 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55746f60c9e0 0x55746f60da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f106efa8740 4 client.7845 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:47:33.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f106efa8740 1 -- 172.21.15.47:0/1648580115 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55746f2676c0 con 0x55746f60c9e0 2024-03-23T09:47:33.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f106a1d2640 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55746f60c9e0 0x55746f60da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f10457fa640 10 client.7845 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:47:33.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.918+0000 7f106a1d2640 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55746f60c9e0 0x55746f60da00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff6f4255c538ed8 server_cookie=9dbd68be055bee44 in_seq=0 out_seq=0 2024-03-23T09:47:33.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f10457fa640 1 -- 172.21.15.47:0/1648580115 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55746f2676c0 con 0x55746f60c9e0 2024-03-23T09:47:33.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f10457fa640 10 client.7845 handle_command_reply: tid=0 2024-03-23T09:47:33.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f106efa8740 1 client.7845 shutdown 2024-03-23T09:47:33.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f106efa8740 1 -- 172.21.15.47:0/1648580115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f6210b0 msgr2=0x55746f6254f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f106efa8740 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f6210b0 0x55746f6254f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f105c02c450 tx=0x7f105c004ab0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f106efa8740 1 -- 172.21.15.47:0/1648580115 shutdown_connections 2024-03-23T09:47:33.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f106efa8740 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55746f60c9e0 0x55746f60da00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f106efa8740 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f6210b0 0x55746f6254f0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f106efa8740 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f646750 0x55746f62cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f10457fa640 0 client.7845 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:47:33.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f106efa8740 1 --2- 172.21.15.47:0/1648580115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f5fc7d0 0x55746f64b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f106efa8740 1 -- 172.21.15.47:0/1648580115 >> 172.21.15.47:0/1648580115 conn(0x55746f5f8c70 msgr2=0x55746f5fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:33.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f106efa8740 1 -- 172.21.15.47:0/1648580115 shutdown_connections 2024-03-23T09:47:33.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f106efa8740 1 -- 172.21.15.47:0/1648580115 wait complete. 2024-03-23T09:47:33.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.919+0000 7f106efa8740 20 client.7845 trim_cache size 0 max 16384 2024-03-23T09:47:33.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.934+0000 7f106efa8740 1 Processor -- start 2024-03-23T09:47:33.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.934+0000 7f106efa8740 1 -- start start 2024-03-23T09:47:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106efa8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f5fc7d0 0x55746f6a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106efa8740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f6210b0 0x55746f5fe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106efa8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 0x55746f5f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106efa8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746f643860 con 0x55746f646750 2024-03-23T09:47:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106efa8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746f5fefe0 con 0x55746f5fc7d0 2024-03-23T09:47:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106efa8740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55746f625470 con 0x55746f6210b0 2024-03-23T09:47:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106a9d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 0x55746f5f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106a9d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 0x55746f5f3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58242/0 (socket says 172.21.15.47:58242) 2024-03-23T09:47:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106a9d3640 1 -- 172.21.15.47:0/436571852 learned_addr learned my addr 172.21.15.47:0/436571852 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106a9d3640 1 -- 172.21.15.47:0/436571852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f6210b0 msgr2=0x55746f5fe930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106a9d3640 1 --2- 172.21.15.47:0/436571852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f6210b0 0x55746f5fe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106a9d3640 1 -- 172.21.15.47:0/436571852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f5fc7d0 msgr2=0x55746f6a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106a9d3640 1 --2- 172.21.15.47:0/436571852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f5fc7d0 0x55746f6a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.935+0000 7f106a9d3640 1 -- 172.21.15.47:0/436571852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55746f503430 con 0x55746f646750 2024-03-23T09:47:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.936+0000 7f106a9d3640 1 --2- 172.21.15.47:0/436571852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 0x55746f5f3f90 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f105c02be80 tx=0x7f105c04b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a08177f6a83bc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.936+0000 7f1058e0c640 1 -- 172.21.15.47:0/436571852 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c003da0 con 0x55746f646750 2024-03-23T09:47:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.936+0000 7f1058e0c640 1 -- 172.21.15.47:0/436571852 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c003f20 con 0x55746f646750 2024-03-23T09:47:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.936+0000 7f1058e0c640 1 -- 172.21.15.47:0/436571852 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c0042d0 con 0x55746f646750 2024-03-23T09:47:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.936+0000 7f106efa8740 1 -- 172.21.15.47:0/436571852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 msgr2=0x55746f5f3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.936+0000 7f106efa8740 1 --2- 172.21.15.47:0/436571852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 0x55746f5f3f90 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f105c02be80 tx=0x7f105c04b070 comp rx=0 tx=0).stop 2024-03-23T09:47:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.936+0000 7f106efa8740 1 -- 172.21.15.47:0/436571852 shutdown_connections 2024-03-23T09:47:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.936+0000 7f106efa8740 1 --2- 172.21.15.47:0/436571852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 0x55746f5f3f90 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.936+0000 7f106efa8740 1 --2- 172.21.15.47:0/436571852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f6210b0 0x55746f5fe930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.936+0000 7f106efa8740 1 --2- 172.21.15.47:0/436571852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f5fc7d0 0x55746f6a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.936+0000 7f106efa8740 1 -- 172.21.15.47:0/436571852 >> 172.21.15.47:0/436571852 conn(0x55746f5f8c70 msgr2=0x55746f621490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.936+0000 7f106efa8740 1 -- 172.21.15.47:0/436571852 shutdown_connections 2024-03-23T09:47:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.936+0000 7f106efa8740 1 -- 172.21.15.47:0/436571852 wait complete. 2024-03-23T09:47:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.937+0000 7f106efa8740 1 -- start start 2024-03-23T09:47:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.937+0000 7f106efa8740 1 Processor -- start 2024-03-23T09:47:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106efa8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f5fc7d0 0x55746f69d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106efa8740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f6210b0 0x55746f6452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106efa8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 0x55746f5fcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106efa8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746f5fd990 con 0x55746f646750 2024-03-23T09:47:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106efa8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746f5fdc70 con 0x55746f5fc7d0 2024-03-23T09:47:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106efa8740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55746f5fdf50 con 0x55746f6210b0 2024-03-23T09:47:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106efa8740 10 client.0 _ll_register_callbacks cb 0x7f105c009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:47:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106a9d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 0x55746f5fcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106a9d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 0x55746f5fcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58246/0 (socket says 172.21.15.47:58246) 2024-03-23T09:47:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106a9d3640 1 -- 172.21.15.47:0/2916519938 learned_addr learned my addr 172.21.15.47:0/2916519938 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106a1d2640 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f5fc7d0 0x55746f69d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f10699d1640 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f6210b0 0x55746f6452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106a9d3640 1 -- 172.21.15.47:0/2916519938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f6210b0 msgr2=0x55746f6452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106a9d3640 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f6210b0 0x55746f6452c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106a9d3640 1 -- 172.21.15.47:0/2916519938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f5fc7d0 msgr2=0x55746f69d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106a9d3640 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f5fc7d0 0x55746f69d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106a9d3640 1 -- 172.21.15.47:0/2916519938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55746f5610b0 con 0x55746f646750 2024-03-23T09:47:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106a1d2640 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f5fc7d0 0x55746f69d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f106a9d3640 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 0x55746f5fcbb0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f105c009920 tx=0x7f105c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc08e7b16947f8d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.938+0000 7f10457fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:47:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f10457fa640 1 -- 172.21.15.47:0/2916519938 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c071070 con 0x55746f646750 2024-03-23T09:47:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f10457fa640 1 -- 172.21.15.47:0/2916519938 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f105c04f050 con 0x55746f646750 2024-03-23T09:47:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f106efa8740 1 -- 172.21.15.47:0/2916519938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f105c0347a0 con 0x55746f646750 2024-03-23T09:47:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f10457fa640 1 -- 172.21.15.47:0/2916519938 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f105c035b30 con 0x55746f646750 2024-03-23T09:47:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f106efa8740 10 client.7851 fetch_fsmap learned FSMap version 52 2024-03-23T09:47:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f106efa8740 1 -- 172.21.15.47:0/2916519938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55746f528b40 con 0x55746f646750 2024-03-23T09:47:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f10457fa640 1 -- 172.21.15.47:0/2916519938 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c0347a0 con 0x55746f646750 2024-03-23T09:47:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f10457fa640 1 -- 172.21.15.47:0/2916519938 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 52) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f105c054470 con 0x55746f646750 2024-03-23T09:47:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f106efa8740 10 client.7851 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T09:47:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f106efa8740 10 client.7851 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:47:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f106efa8740 20 client.7851 populate_metadata read hostname 'smithi047' 2024-03-23T09:47:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f106efa8740 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55746f5f79b0 0x55746f5f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f106efa8740 4 client.7851 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:47:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f106efa8740 1 -- 172.21.15.47:0/2916519938 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55746f2bad10 con 0x55746f5f79b0 2024-03-23T09:47:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.939+0000 7f106a1d2640 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55746f5f79b0 0x55746f5f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f10457fa640 10 client.7851 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:47:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f106a1d2640 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55746f5f79b0 0x55746f5f7d90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4da8ace2433af595 server_cookie=e172b65e5e7e8a8f in_seq=0 out_seq=0 2024-03-23T09:47:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f10457fa640 1 -- 172.21.15.47:0/2916519938 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55746f2bad10 con 0x55746f5f79b0 2024-03-23T09:47:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f10457fa640 10 client.7851 handle_command_reply: tid=0 2024-03-23T09:47:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f106efa8740 1 client.7851 shutdown 2024-03-23T09:47:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f106efa8740 1 -- 172.21.15.47:0/2916519938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 msgr2=0x55746f5fcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f106efa8740 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 0x55746f5fcbb0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f105c009920 tx=0x7f105c040000 comp rx=0 tx=0).stop 2024-03-23T09:47:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f106efa8740 1 -- 172.21.15.47:0/2916519938 shutdown_connections 2024-03-23T09:47:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f106efa8740 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55746f5f79b0 0x55746f5f7d90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f106efa8740 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55746f646750 0x55746f5fcbb0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f106efa8740 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55746f6210b0 0x55746f6452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f10457fa640 0 client.7851 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:47:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f106efa8740 1 --2- 172.21.15.47:0/2916519938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55746f5fc7d0 0x55746f69d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f106efa8740 1 -- 172.21.15.47:0/2916519938 >> 172.21.15.47:0/2916519938 conn(0x55746f5f8c70 msgr2=0x55746f5f3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f106efa8740 1 -- 172.21.15.47:0/2916519938 shutdown_connections 2024-03-23T09:47:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f106efa8740 1 -- 172.21.15.47:0/2916519938 wait complete. 2024-03-23T09:47:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.940+0000 7f106efa8740 20 client.7851 trim_cache size 0 max 16384 2024-03-23T09:47:33.948 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:47:33.948 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:47:33.948 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:47:33.948 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 7776, 2024-03-23T09:47:33.948 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:47:33.948 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:47:33.948 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:47:33.948 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 4.4226145069999996, 2024-03-23T09:47:33.949 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 52, 2024-03-23T09:47:33.949 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 86, 2024-03-23T09:47:33.949 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 86, 2024-03-23T09:47:33.949 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 5.3958952509999998 2024-03-23T09:47:33.949 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:47:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.941+0000 7f106c436640 1 -- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f104806ac00 msgr2=0x7f104806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.941+0000 7f106c436640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f104806ac00 0x7f104806d0c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1060000f60 tx=0x7f1060007ca0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.941+0000 7f106c436640 1 -- 172.21.15.47:0/2509682807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10640cf210 msgr2=0x7f10640ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.941+0000 7f106c436640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10640cf210 0x7f10640ca500 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f105400c7f0 tx=0x7f105400cbf0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.941+0000 7f106c436640 1 -- 172.21.15.47:0/2509682807 shutdown_connections 2024-03-23T09:47:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.941+0000 7f106c436640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f104806ac00 0x7f104806d0c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.941+0000 7f106c436640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f106415cd00 0x7f10640caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.941+0000 7f106c436640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10640cf210 0x7f10640ca500 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.941+0000 7f106c436640 1 --2- 172.21.15.47:0/2509682807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10640b5a90 0x7f10641571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.941+0000 7f106c436640 1 -- 172.21.15.47:0/2509682807 >> 172.21.15.47:0/2509682807 conn(0x7f106406b160 msgr2=0x7f10640b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.941+0000 7f106c436640 1 -- 172.21.15.47:0/2509682807 shutdown_connections 2024-03-23T09:47:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:33.941+0000 7f106c436640 1 -- 172.21.15.47:0/2509682807 wait complete. 2024-03-23T09:47:33.952 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:47:34.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.053+0000 7f05027fd640 1 Processor -- start 2024-03-23T09:47:34.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.053+0000 7f05027fd640 1 -- start start 2024-03-23T09:47:34.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.054+0000 7f05027fd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f04fc0cf690 0x7f04fc0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:34.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.054+0000 7f05027fd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04fc15d0e0 con 0x7f04fc15cd00 2024-03-23T09:47:34.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.054+0000 7f05027fd640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04fc0d1540 con 0x7f04fc0cf210 2024-03-23T09:47:34.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.054+0000 7f05027fd640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04fc0d1820 con 0x7f04fc0cf690 2024-03-23T09:47:34.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.054+0000 7f04fbfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f04fc15cd00 0x7f04fc0d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53502/0 (socket says 172.21.15.47:53502) 2024-03-23T09:47:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.054+0000 7f04fbfff640 1 -- 172.21.15.47:0/2983897592 learned_addr learned my addr 172.21.15.47:0/2983897592 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:47:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.054+0000 7f0500d9a640 1 --2- 172.21.15.47:0/2983897592 >> v2:172.21.15.47:3300/0 conn(0x7f04fc0cf690 0x7f04fc0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.054+0000 7f04faffd640 1 -- 172.21.15.47:0/2983897592 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714157217 0 0) 0x7f04fc15d0e0 con 0x7f04fc15cd00 2024-03-23T09:47:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.054+0000 7f04faffd640 1 -- 172.21.15.47:0/2983897592 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04e8003520 con 0x7f04fc15cd00 2024-03-23T09:47:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.054+0000 7f0500d9a640 1 -- 172.21.15.47:0/2983897592 >> v1:172.21.15.175:6789/0 conn(0x7f04fc0cf210 legacy=0x7f04fc0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.054+0000 7f0500d9a640 1 -- 172.21.15.47:0/2983897592 >> v1:172.21.15.47:6789/0 conn(0x7f04fc15cd00 legacy=0x7f04fc0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.054+0000 7f0500d9a640 1 -- 172.21.15.47:0/2983897592 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc0d1b00 con 0x7f04fc0cf690 2024-03-23T09:47:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.055+0000 7f0500d9a640 1 --2- 172.21.15.47:0/2983897592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04fc0cf690 0x7f04fc0b61a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009d90 tx=0x7f04f402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b3f2b09d351f6a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.055+0000 7f04faffd640 1 -- 172.21.15.47:0/2983897592 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f402cc60 con 0x7f04fc0cf690 2024-03-23T09:47:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.055+0000 7f04faffd640 1 -- 172.21.15.47:0/2983897592 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f402cdc0 con 0x7f04fc0cf690 2024-03-23T09:47:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.055+0000 7f04faffd640 1 -- 172.21.15.47:0/2983897592 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f4034660 con 0x7f04fc0cf690 2024-03-23T09:47:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.056+0000 7f05027fd640 1 -- 172.21.15.47:0/2983897592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04fc0cf690 msgr2=0x7f04fc0b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:34.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.056+0000 7f05027fd640 1 --2- 172.21.15.47:0/2983897592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04fc0cf690 0x7f04fc0b61a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009d90 tx=0x7f04f402beb0 comp rx=0 tx=0).stop 2024-03-23T09:47:34.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.056+0000 7f05027fd640 1 -- 172.21.15.47:0/2983897592 shutdown_connections 2024-03-23T09:47:34.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.056+0000 7f05027fd640 1 --2- 172.21.15.47:0/2983897592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04fc0cf690 0x7f04fc0b61a0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:34.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.056+0000 7f05027fd640 1 -- 172.21.15.47:0/2983897592 >> 172.21.15.47:0/2983897592 conn(0x7f04fc06b160 msgr2=0x7f04fc06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:34.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.056+0000 7f05027fd640 1 -- 172.21.15.47:0/2983897592 shutdown_connections 2024-03-23T09:47:34.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.056+0000 7f05027fd640 1 -- 172.21.15.47:0/2983897592 wait complete. 2024-03-23T09:47:34.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.057+0000 7f05027fd640 1 Processor -- start 2024-03-23T09:47:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.057+0000 7f05027fd640 1 -- start start 2024-03-23T09:47:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f05027fd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04fc0cf210 0x7f04fc0b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f05027fd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04fc0cf690 0x7f04fc15acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f05027fd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04fc15cd00 0x7f04fc1f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f05027fd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc0aa670 con 0x7f04fc0cf690 2024-03-23T09:47:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f05027fd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc0b4ea0 con 0x7f04fc0cf210 2024-03-23T09:47:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f05027fd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc0aa0b0 con 0x7f04fc15cd00 2024-03-23T09:47:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f04fb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04fc0cf690 0x7f04fc15acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f04fb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04fc0cf690 0x7f04fc15acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58276/0 (socket says 172.21.15.47:58276) 2024-03-23T09:47:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f04fb7fe640 1 -- 172.21.15.47:0/1908969424 learned_addr learned my addr 172.21.15.47:0/1908969424 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f04fbfff640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04fc0cf210 0x7f04fc0b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f0500d9a640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04fc15cd00 0x7f04fc1f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:34.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f04fb7fe640 1 -- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04fc15cd00 msgr2=0x7f04fc1f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:34.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f04fb7fe640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04fc15cd00 0x7f04fc1f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:34.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f04fb7fe640 1 -- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04fc0cf210 msgr2=0x7f04fc0b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:34.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f04fb7fe640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04fc0cf210 0x7f04fc0b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:34.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f04fb7fe640 1 -- 172.21.15.47:0/1908969424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f4009a70 con 0x7f04fc0cf690 2024-03-23T09:47:34.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.058+0000 7f04fb7fe640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04fc0cf690 0x7f04fc15acc0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00c450 tx=0x7f04ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:34.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.059+0000 7f04f8ff9640 1 -- 172.21.15.47:0/1908969424 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec0052d0 con 0x7f04fc0cf690 2024-03-23T09:47:34.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.059+0000 7f04f8ff9640 1 -- 172.21.15.47:0/1908969424 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ec005430 con 0x7f04fc0cf690 2024-03-23T09:47:34.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.059+0000 7f05027fd640 1 -- 172.21.15.47:0/1908969424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04fc0aa390 con 0x7f04fc0cf690 2024-03-23T09:47:34.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.059+0000 7f04f8ff9640 1 -- 172.21.15.47:0/1908969424 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec012870 con 0x7f04fc0cf690 2024-03-23T09:47:34.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.059+0000 7f05027fd640 1 -- 172.21.15.47:0/1908969424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04fc0b24f0 con 0x7f04fc0cf690 2024-03-23T09:47:34.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.060+0000 7f05027fd640 1 -- 172.21.15.47:0/1908969424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04fc072360 con 0x7f04fc0cf690 2024-03-23T09:47:34.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.060+0000 7f04f8ff9640 1 -- 172.21.15.47:0/1908969424 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f04ec0129d0 con 0x7f04fc0cf690 2024-03-23T09:47:34.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.060+0000 7f04f8ff9640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f04dc06ac00 0x7f04dc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:34.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.061+0000 7f04f8ff9640 1 -- 172.21.15.47:0/1908969424 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f04ec088e90 con 0x7f04fc0cf690 2024-03-23T09:47:34.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.061+0000 7f04fbfff640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f04dc06ac00 0x7f04dc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:34.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.061+0000 7f04fbfff640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f04dc06ac00 0x7f04dc06d0c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f04e40011e0 tx=0x7f04e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:34.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.065+0000 7f04f8ff9640 1 -- 172.21.15.47:0/1908969424 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f04ec0540a0 con 0x7f04fc0cf690 2024-03-23T09:47:34.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.282+0000 7f05027fd640 1 -- 172.21.15.47:0/1908969424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f04fc0b26e0 con 0x7f04fc0cf690 2024-03-23T09:47:34.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.282+0000 7f04f8ff9640 1 -- 172.21.15.47:0/1908969424 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7f04ec0576c0 con 0x7f04fc0cf690 2024-03-23T09:47:34.283 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:47:34.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.284+0000 7f05027fd640 1 -- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f04dc06ac00 msgr2=0x7f04dc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:34.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.284+0000 7f05027fd640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f04dc06ac00 0x7f04dc06d0c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f04e40011e0 tx=0x7f04e4009040 comp rx=0 tx=0).stop 2024-03-23T09:47:34.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.284+0000 7f05027fd640 1 -- 172.21.15.47:0/1908969424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04fc0cf690 msgr2=0x7f04fc15acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:34.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.284+0000 7f05027fd640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04fc0cf690 0x7f04fc15acc0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00c450 tx=0x7f04ec00c910 comp rx=0 tx=0).stop 2024-03-23T09:47:34.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.284+0000 7f05027fd640 1 -- 172.21.15.47:0/1908969424 shutdown_connections 2024-03-23T09:47:34.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.284+0000 7f05027fd640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f04dc06ac00 0x7f04dc06d0c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:34.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.284+0000 7f05027fd640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04fc15cd00 0x7f04fc1f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:34.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.284+0000 7f05027fd640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04fc0cf690 0x7f04fc15acc0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:34.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.284+0000 7f05027fd640 1 --2- 172.21.15.47:0/1908969424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04fc0cf210 0x7f04fc0b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:34.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.284+0000 7f05027fd640 1 -- 172.21.15.47:0/1908969424 >> 172.21.15.47:0/1908969424 conn(0x7f04fc06b160 msgr2=0x7f04fc15d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:34.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.284+0000 7f05027fd640 1 -- 172.21.15.47:0/1908969424 shutdown_connections 2024-03-23T09:47:34.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:34.284+0000 7f05027fd640 1 -- 172.21.15.47:0/1908969424 wait complete. 2024-03-23T09:47:34.295 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:47:34.295 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:47:34.295 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:47:34.295 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:47:34.295 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:47:34.295 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:47:34.296 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:47:34.296 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:47:34.296 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:47:34.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:34.296 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42284sec preferred_lft 42284sec 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:34.312 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:47:34.313 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:47:34.313 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:47:34.313 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:34.313 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:47:34.313 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:47:34.313 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:47:34.313 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:47:34.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:34.313 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:47:34.376 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:47:34.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:34.377 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:47:34.433 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:47:34.433 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:47:34.433 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:47:34.433 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:47:34.433 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:47:34.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:34.433 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:47:34.433 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:47:34.433 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:47:34.434 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:47:34.434 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:47:34.434 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:47:34.518 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:47:34.518+0000 7f3d23d07640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:47:34.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:34.528 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:47:34.542 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:47:34.542 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:47:34.542 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:47:34.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:34.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:47:34.609 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:47:34.609 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:47:34.609 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:47:34.664 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:47:34.664 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:47:34.665 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:47:34.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:47:34.786 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:47:34.786 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:47:34.852 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:47:34.853 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:47:34.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:34.854 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:47:34.908 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:47:34.908 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:47:34.951 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:47:34.977 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:47:34.977 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:47:35.037 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:47:35.037+0000 7fbcc2041480 -1 init, newargv = 0x55c019fc6e70 newargc=15 2024-03-23T09:47:35.038 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[68636]: starting ceph client 2024-03-23T09:47:35.042 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[68636]: starting fuse 2024-03-23T09:47:35.046 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:47:35.048 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:47:35.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:35.048 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:47:35.106 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:47:35.106 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:47:35.106 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:47:35.106 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:47:35.106 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:47:35.106 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:47:35.106 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:47:35.106 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:47:35.107 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:47:35.108 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:47:35.108 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:47:35.108 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:47:35.108 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:47:35.222 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.68636.asok 2024-03-23T09:47:35.409 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.68636.asok 2024-03-23T09:47:35.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:35.409 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.68636.asok status 2024-03-23T09:47:35.545 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:47:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:47:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:47:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:47:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:47:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:47:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:47:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "68636", 2024-03-23T09:47:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:47:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:47:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:47:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 7869, 2024-03-23T09:47:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:47:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:47:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:47:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7869 2024-03-23T09:47:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:47:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:47:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:47:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1664844574 2024-03-23T09:47:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:47:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:47:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:47:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1664844574 2024-03-23T09:47:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:47:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.7869 192.168.0.1:0/1664844574", 2024-03-23T09:47:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/1664844574", 2024-03-23T09:47:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:47:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 52, 2024-03-23T09:47:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 86, 2024-03-23T09:47:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:47:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:47:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:47:35.548 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:47:35.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:35.554 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:47:35.554 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:47:35.569 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:47:35.569 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:47:35.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:35.569 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:47:35.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:35.636 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-03-23T09:47:35.693 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:47:35.693+0000 7f3d20500640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-23T09:47:35.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:47:35.694 INFO:teuthology.orchestra.run.smithi047.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-03-23T09:47:35.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:35.695 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-03-23T09:47:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.841+0000 7f89de989640 1 Processor -- start 2024-03-23T09:47:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.841+0000 7f89de989640 1 -- start start 2024-03-23T09:47:35.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.842+0000 7f89de989640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f89d80b4340 0x7f89d80cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.842+0000 7f89de989640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f89d80cf6b0 0x7f89d80d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.842+0000 7f89de989640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d80d0b40 con 0x7f89d815cc40 2024-03-23T09:47:35.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.842+0000 7f89de989640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d80cfb40 con 0x7f89d80cf6b0 2024-03-23T09:47:35.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.842+0000 7f89de989640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d80b48b0 con 0x7f89d80b4340 2024-03-23T09:47:35.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.842+0000 7f89dcf26640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f89d80cf6b0 0x7f89d80d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.842+0000 7f89dcf26640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f89d80cf6b0 0x7f89d80d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53382/0 (socket says 172.21.15.47:53382) 2024-03-23T09:47:35.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.842+0000 7f89dcf26640 1 -- 172.21.15.47:0/2334950152 learned_addr learned my addr 172.21.15.47:0/2334950152 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:35.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.842+0000 7f89d7fff640 1 --1- 172.21.15.47:0/2334950152 >> v1:172.21.15.47:6789/0 conn(0x7f89d815cc40 0x7f89d80b8250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36826/0 (socket says 172.21.15.47:36826) 2024-03-23T09:47:35.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.842+0000 7f89d77fe640 1 --2- 172.21.15.47:0/2334950152 >> v2:172.21.15.175:3300/0 conn(0x7f89d80b4340 0x7f89d80cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.843+0000 7f89dcf26640 1 -- 172.21.15.47:0/2334950152 >> v2:172.21.15.175:3300/0 conn(0x7f89d80b4340 msgr2=0x7f89d80cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:35.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.843+0000 7f89dcf26640 1 --2- 172.21.15.47:0/2334950152 >> v2:172.21.15.175:3300/0 conn(0x7f89d80b4340 0x7f89d80cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.843+0000 7f89dcf26640 1 -- 172.21.15.47:0/2334950152 >> v1:172.21.15.47:6789/0 conn(0x7f89d815cc40 legacy=0x7f89d80b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.843+0000 7f89dcf26640 1 -- 172.21.15.47:0/2334950152 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d80b4b90 con 0x7f89d80cf6b0 2024-03-23T09:47:35.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.843+0000 7f89d6ffd640 1 -- 172.21.15.47:0/2334950152 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277653734 0 0) 0x7f89d80d0b40 con 0x7f89d815cc40 2024-03-23T09:47:35.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.843+0000 7f89dcf26640 1 --2- 172.21.15.47:0/2334950152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89d80cf6b0 0x7f89d80d0fd0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f89d80d27a0 tx=0x7f89d002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcc31f5ce990e08 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:35.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.843+0000 7f89d6ffd640 1 -- 172.21.15.47:0/2334950152 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d002cb20 con 0x7f89d80cf6b0 2024-03-23T09:47:35.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.843+0000 7f89d6ffd640 1 -- 172.21.15.47:0/2334950152 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d002cc80 con 0x7f89d80cf6b0 2024-03-23T09:47:35.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.843+0000 7f89d6ffd640 1 -- 172.21.15.47:0/2334950152 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d00343f0 con 0x7f89d80cf6b0 2024-03-23T09:47:35.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.844+0000 7f89de989640 1 -- 172.21.15.47:0/2334950152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89d80cf6b0 msgr2=0x7f89d80d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.844+0000 7f89de989640 1 --2- 172.21.15.47:0/2334950152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89d80cf6b0 0x7f89d80d0fd0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f89d80d27a0 tx=0x7f89d002bf80 comp rx=0 tx=0).stop 2024-03-23T09:47:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.844+0000 7f89de989640 1 -- 172.21.15.47:0/2334950152 shutdown_connections 2024-03-23T09:47:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.844+0000 7f89de989640 1 --2- 172.21.15.47:0/2334950152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89d80cf6b0 0x7f89d80d0fd0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.844+0000 7f89de989640 1 --2- 172.21.15.47:0/2334950152 >> v2:172.21.15.175:3300/0 conn(0x7f89d80b4340 0x7f89d80cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.844+0000 7f89de989640 1 -- 172.21.15.47:0/2334950152 >> 172.21.15.47:0/2334950152 conn(0x7f89d80659d0 msgr2=0x7f89d8065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.844+0000 7f89de989640 1 -- 172.21.15.47:0/2334950152 shutdown_connections 2024-03-23T09:47:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.844+0000 7f89de989640 1 -- 172.21.15.47:0/2334950152 wait complete. 2024-03-23T09:47:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.845+0000 7f89de989640 1 Processor -- start 2024-03-23T09:47:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.845+0000 7f89de989640 1 -- start start 2024-03-23T09:47:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89de989640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89d80b4340 0x7f89d815f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89de989640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89d80cf6b0 0x7f89d81f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89de989640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89d815cc40 0x7f89d80ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89de989640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d81f35c0 con 0x7f89d80b4340 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89de989640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d8160070 con 0x7f89d80cf6b0 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89de989640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d815fd90 con 0x7f89d815cc40 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89d7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89d80b4340 0x7f89d815f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89d7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89d80b4340 0x7f89d815f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53384/0 (socket says 172.21.15.47:53384) 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89d7fff640 1 -- 172.21.15.47:0/161333579 learned_addr learned my addr 172.21.15.47:0/161333579 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89d77fe640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89d80cf6b0 0x7f89d81f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89dcf26640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89d815cc40 0x7f89d80ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89d7fff640 1 -- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89d815cc40 msgr2=0x7f89d80ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89d7fff640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89d815cc40 0x7f89d80ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89d7fff640 1 -- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89d80cf6b0 msgr2=0x7f89d81f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89d7fff640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89d80cf6b0 0x7f89d81f3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.846+0000 7f89d7fff640 1 -- 172.21.15.47:0/161333579 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d0009b30 con 0x7f89d80b4340 2024-03-23T09:47:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.847+0000 7f89d7fff640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89d80b4340 0x7f89d815f7b0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c430 tx=0x7f89c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:35.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.847+0000 7f89d4ff9640 1 -- 172.21.15.47:0/161333579 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c00074e0 con 0x7f89d80b4340 2024-03-23T09:47:35.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.847+0000 7f89d4ff9640 1 -- 172.21.15.47:0/161333579 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c0007660 con 0x7f89d80b4340 2024-03-23T09:47:35.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.847+0000 7f89de989640 1 -- 172.21.15.47:0/161333579 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d81f25a0 con 0x7f89d80b4340 2024-03-23T09:47:35.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.847+0000 7f89d4ff9640 1 -- 172.21.15.47:0/161333579 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c0004a20 con 0x7f89d80b4340 2024-03-23T09:47:35.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.847+0000 7f89de989640 1 -- 172.21.15.47:0/161333579 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d81f2b60 con 0x7f89d80b4340 2024-03-23T09:47:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.848+0000 7f89d4ff9640 1 -- 172.21.15.47:0/161333579 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f89c0004b80 con 0x7f89d80b4340 2024-03-23T09:47:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.849+0000 7f89d4ff9640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f89b806ac00 0x7f89b806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.849+0000 7f89d4ff9640 1 -- 172.21.15.47:0/161333579 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f89c0089240 con 0x7f89d80b4340 2024-03-23T09:47:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.849+0000 7f89d77fe640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f89b806ac00 0x7f89b806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.849+0000 7f89d77fe640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f89b806ac00 0x7f89b806d0c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f89d80c6c20 tx=0x7f89c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.850+0000 7f89e15d5740 1 Processor -- start 2024-03-23T09:47:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.850+0000 7f89e15d5740 1 -- start start 2024-03-23T09:47:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.850+0000 7f89e15d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e665c0 0x55f562e39560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.850+0000 7f89e15d5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e669a0 0x55f562e69350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.850+0000 7f89e15d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e39aa0 0x55f562e69890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.850+0000 7f89e15d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f562daac20 con 0x55f562e39aa0 2024-03-23T09:47:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.850+0000 7f89e15d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f562da83b0 con 0x55f562e665c0 2024-03-23T09:47:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.850+0000 7f89e15d5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f562da3ca0 con 0x55f562e669a0 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.850+0000 7f89dcf26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e39aa0 0x55f562e69890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.850+0000 7f89dcf26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e39aa0 0x55f562e69890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53390/0 (socket says 172.21.15.47:53390) 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.850+0000 7f89dcf26640 1 -- 172.21.15.47:0/432054497 learned_addr learned my addr 172.21.15.47:0/432054497 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.850+0000 7f89d77fe640 1 --2- 172.21.15.47:0/432054497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e669a0 0x55f562e69350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.851+0000 7f89d7fff640 1 --2- 172.21.15.47:0/432054497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e665c0 0x55f562e39560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.851+0000 7f89dcf26640 1 -- 172.21.15.47:0/432054497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e669a0 msgr2=0x55f562e69350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.851+0000 7f89dcf26640 1 --2- 172.21.15.47:0/432054497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e669a0 0x55f562e69350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.851+0000 7f89dcf26640 1 -- 172.21.15.47:0/432054497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e665c0 msgr2=0x55f562e39560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.851+0000 7f89dcf26640 1 --2- 172.21.15.47:0/432054497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e665c0 0x55f562e39560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.851+0000 7f89dcf26640 1 -- 172.21.15.47:0/432054497 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f562d4a710 con 0x55f562e39aa0 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.851+0000 7f89dcf26640 1 --2- 172.21.15.47:0/432054497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e39aa0 0x55f562e69890 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f89d002c590 tx=0x7f89d002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c624eba22b4a097 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.851+0000 7f89cdffb640 1 -- 172.21.15.47:0/432054497 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d002cb20 con 0x55f562e39aa0 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.851+0000 7f89cdffb640 1 -- 172.21.15.47:0/432054497 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d0040470 con 0x55f562e39aa0 2024-03-23T09:47:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.851+0000 7f89cdffb640 1 -- 172.21.15.47:0/432054497 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0040800 con 0x55f562e39aa0 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.852+0000 7f89e15d5740 1 -- 172.21.15.47:0/432054497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e39aa0 msgr2=0x55f562e69890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.852+0000 7f89e15d5740 1 --2- 172.21.15.47:0/432054497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e39aa0 0x55f562e69890 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f89d002c590 tx=0x7f89d002e000 comp rx=0 tx=0).stop 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.852+0000 7f89e15d5740 1 -- 172.21.15.47:0/432054497 shutdown_connections 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.852+0000 7f89e15d5740 1 --2- 172.21.15.47:0/432054497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e39aa0 0x55f562e69890 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.852+0000 7f89e15d5740 1 --2- 172.21.15.47:0/432054497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e669a0 0x55f562e69350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.852+0000 7f89e15d5740 1 --2- 172.21.15.47:0/432054497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e665c0 0x55f562e39560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.852+0000 7f89e15d5740 1 -- 172.21.15.47:0/432054497 >> 172.21.15.47:0/432054497 conn(0x55f562e3b7d0 msgr2=0x55f562e84350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.852+0000 7f89e15d5740 1 -- 172.21.15.47:0/432054497 shutdown_connections 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.852+0000 7f89e15d5740 1 -- 172.21.15.47:0/432054497 wait complete. 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.852+0000 7f89e15d5740 1 -- start start 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.852+0000 7f89e15d5740 1 Processor -- start 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.853+0000 7f89e15d5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e665c0 0x55f562e61590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.853+0000 7f89e15d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e669a0 0x55f562e6a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.853+0000 7f89e15d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e6ca60 0x55f562e6cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.853+0000 7f89e15d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f562d61e70 con 0x55f562e6ca60 2024-03-23T09:47:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.853+0000 7f89e15d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f562d61b10 con 0x55f562e669a0 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.853+0000 7f89e15d5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f562d27060 con 0x55f562e665c0 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.853+0000 7f89e15d5740 10 client.0 _ll_register_callbacks cb 0x55f562d9d310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.853+0000 7f89dcf26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e6ca60 0x55f562e6cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.853+0000 7f89dcf26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e6ca60 0x55f562e6cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53396/0 (socket says 172.21.15.47:53396) 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.853+0000 7f89dcf26640 1 -- 172.21.15.47:0/1788904372 learned_addr learned my addr 172.21.15.47:0/1788904372 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.853+0000 7f89d77fe640 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e669a0 0x55f562e6a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.853+0000 7f89d7fff640 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e665c0 0x55f562e61590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89dcf26640 1 -- 172.21.15.47:0/1788904372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e665c0 msgr2=0x55f562e61590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89dcf26640 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e665c0 0x55f562e61590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89dcf26640 1 -- 172.21.15.47:0/1788904372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e669a0 msgr2=0x55f562e6a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89dcf26640 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e669a0 0x55f562e6a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89dcf26640 1 -- 172.21.15.47:0/1788904372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d80b4b90 con 0x55f562e6ca60 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89d77fe640 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e669a0 0x55f562e6a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89dcf26640 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e6ca60 0x55f562e6cee0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f89d003f000 tx=0x7f89d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f2a1c3e8ecda95 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89b6ffd640 1 -- 172.21.15.47:0/1788904372 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0003da0 con 0x55f562e6ca60 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89b6ffd640 1 -- 172.21.15.47:0/1788904372 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f89d0004030 con 0x55f562e6ca60 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89e15d5740 1 -- 172.21.15.47:0/1788904372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f562d94fc0 con 0x55f562e6ca60 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89b6ffd640 1 -- 172.21.15.47:0/1788904372 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89d00041f0 con 0x55f562e6ca60 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89e15d5740 10 client.7884 fetch_fsmap learned FSMap version 52 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89e15d5740 1 -- 172.21.15.47:0/1788904372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55f562da2320 con 0x55f562e6ca60 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89b6ffd640 1 -- 172.21.15.47:0/1788904372 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f562d94fc0 con 0x55f562e6ca60 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89b6ffd640 1 -- 172.21.15.47:0/1788904372 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 52) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f89d0047070 con 0x55f562e6ca60 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89e15d5740 10 client.7884 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89e15d5740 10 client.7884 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89e15d5740 20 client.7884 populate_metadata read hostname 'smithi047' 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89e15d5740 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55f562e94010 0x55f562e94420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89e15d5740 4 client.7884 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:47:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.854+0000 7f89e15d5740 1 -- 172.21.15.47:0/1788904372 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f562a97490 con 0x55f562e94010 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.855+0000 7f89d7fff640 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55f562e94010 0x55f562e94420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.855+0000 7f89b6ffd640 10 client.7884 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.855+0000 7f89d7fff640 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55f562e94010 0x55f562e94420 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31a6fd444ddff988 server_cookie=e07cb54fb4d7cfe2 in_seq=0 out_seq=0 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.855+0000 7f89b6ffd640 1 -- 172.21.15.47:0/1788904372 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f562a97490 con 0x55f562e94010 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.855+0000 7f89b6ffd640 10 client.7884 handle_command_reply: tid=0 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.855+0000 7f89e15d5740 1 client.7884 shutdown 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.856+0000 7f89e15d5740 1 -- 172.21.15.47:0/1788904372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e6ca60 msgr2=0x55f562e6cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.856+0000 7f89e15d5740 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e6ca60 0x55f562e6cee0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f89d003f000 tx=0x7f89d0004ab0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.856+0000 7f89e15d5740 1 -- 172.21.15.47:0/1788904372 shutdown_connections 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.856+0000 7f89e15d5740 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55f562e94010 0x55f562e94420 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.856+0000 7f89e15d5740 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e6ca60 0x55f562e6cee0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.856+0000 7f89e15d5740 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e669a0 0x55f562e6a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.856+0000 7f89b6ffd640 0 client.7884 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.856+0000 7f89e15d5740 1 --2- 172.21.15.47:0/1788904372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e665c0 0x55f562e61590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.856+0000 7f89e15d5740 1 -- 172.21.15.47:0/1788904372 >> 172.21.15.47:0/1788904372 conn(0x55f562e39aa0 msgr2=0x55f562e38d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.856+0000 7f89e15d5740 1 -- 172.21.15.47:0/1788904372 shutdown_connections 2024-03-23T09:47:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.856+0000 7f89e15d5740 1 -- 172.21.15.47:0/1788904372 wait complete. 2024-03-23T09:47:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.856+0000 7f89e15d5740 20 client.7884 trim_cache size 0 max 16384 2024-03-23T09:47:35.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.870+0000 7f89e15d5740 1 Processor -- start 2024-03-23T09:47:35.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.870+0000 7f89e15d5740 1 -- start start 2024-03-23T09:47:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89e15d5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e665c0 0x55f562e40580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89e15d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e669a0 0x55f562e84970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89e15d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e6ca60 0x55f562e84eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89e15d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f562e87f80 con 0x55f562e669a0 2024-03-23T09:47:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89e15d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f562e7b500 con 0x55f562e6ca60 2024-03-23T09:47:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89e15d5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f562e87420 con 0x55f562e665c0 2024-03-23T09:47:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89d77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e669a0 0x55f562e84970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89d77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e669a0 0x55f562e84970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53406/0 (socket says 172.21.15.47:53406) 2024-03-23T09:47:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89d77fe640 1 -- 172.21.15.47:0/101111584 learned_addr learned my addr 172.21.15.47:0/101111584 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89dcf26640 1 --2- 172.21.15.47:0/101111584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e6ca60 0x55f562e84eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89d7fff640 1 --2- 172.21.15.47:0/101111584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e665c0 0x55f562e40580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89d77fe640 1 -- 172.21.15.47:0/101111584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e665c0 msgr2=0x55f562e40580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89d77fe640 1 --2- 172.21.15.47:0/101111584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e665c0 0x55f562e40580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89d77fe640 1 -- 172.21.15.47:0/101111584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e6ca60 msgr2=0x55f562e84eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89d77fe640 1 --2- 172.21.15.47:0/101111584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e6ca60 0x55f562e84eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89d77fe640 1 -- 172.21.15.47:0/101111584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f562d44100 con 0x55f562e669a0 2024-03-23T09:47:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89dcf26640 1 --2- 172.21.15.47:0/101111584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e6ca60 0x55f562e84eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.871+0000 7f89d7fff640 1 --2- 172.21.15.47:0/101111584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e665c0 0x55f562e40580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.872+0000 7f89d77fe640 1 --2- 172.21.15.47:0/101111584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e669a0 0x55f562e84970 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f89c800fcf0 tx=0x7f89c8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1ef6cc033e7cf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.872+0000 7f89ce7fc640 1 -- 172.21.15.47:0/101111584 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8023470 con 0x55f562e669a0 2024-03-23T09:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.872+0000 7f89ce7fc640 1 -- 172.21.15.47:0/101111584 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c80235d0 con 0x55f562e669a0 2024-03-23T09:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.872+0000 7f89ce7fc640 1 -- 172.21.15.47:0/101111584 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8023980 con 0x55f562e669a0 2024-03-23T09:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.872+0000 7f89e15d5740 1 -- 172.21.15.47:0/101111584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e669a0 msgr2=0x55f562e84970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.872+0000 7f89e15d5740 1 --2- 172.21.15.47:0/101111584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e669a0 0x55f562e84970 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f89c800fcf0 tx=0x7f89c8015000 comp rx=0 tx=0).stop 2024-03-23T09:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.872+0000 7f89e15d5740 1 -- 172.21.15.47:0/101111584 shutdown_connections 2024-03-23T09:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.872+0000 7f89e15d5740 1 --2- 172.21.15.47:0/101111584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e6ca60 0x55f562e84eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.872+0000 7f89e15d5740 1 --2- 172.21.15.47:0/101111584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e669a0 0x55f562e84970 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.872+0000 7f89e15d5740 1 --2- 172.21.15.47:0/101111584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e665c0 0x55f562e40580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.872+0000 7f89e15d5740 1 -- 172.21.15.47:0/101111584 >> 172.21.15.47:0/101111584 conn(0x55f562e39aa0 msgr2=0x55f562e613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.872+0000 7f89e15d5740 1 -- 172.21.15.47:0/101111584 shutdown_connections 2024-03-23T09:47:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.872+0000 7f89e15d5740 1 -- 172.21.15.47:0/101111584 wait complete. 2024-03-23T09:47:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.873+0000 7f89e15d5740 1 -- start start 2024-03-23T09:47:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.873+0000 7f89e15d5740 1 Processor -- start 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89e15d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e665c0 0x55f562e645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89e15d5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e669a0 0x55f562e64b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89e15d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e6ca60 0x55f562ed12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89e15d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f562ed23e0 con 0x55f562e665c0 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89e15d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f562e60ce0 con 0x55f562e6ca60 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89e15d5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f562ea0490 con 0x55f562e669a0 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89e15d5740 10 client.0 _ll_register_callbacks cb 0x7f89d0043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89d7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e665c0 0x55f562e645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89d7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e665c0 0x55f562e645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53420/0 (socket says 172.21.15.47:53420) 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89d7fff640 1 -- 172.21.15.47:0/2339677038 learned_addr learned my addr 172.21.15.47:0/2339677038 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89dcf26640 1 --2- 172.21.15.47:0/2339677038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e6ca60 0x55f562ed12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89d77fe640 1 --2- 172.21.15.47:0/2339677038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e669a0 0x55f562e64b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89d7fff640 1 -- 172.21.15.47:0/2339677038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e669a0 msgr2=0x55f562e64b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89d7fff640 1 --2- 172.21.15.47:0/2339677038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e669a0 0x55f562e64b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89d7fff640 1 -- 172.21.15.47:0/2339677038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e6ca60 msgr2=0x55f562ed12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89d7fff640 1 --2- 172.21.15.47:0/2339677038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e6ca60 0x55f562ed12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89d7fff640 1 -- 172.21.15.47:0/2339677038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c800e040 con 0x55f562e665c0 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89d7fff640 1 --2- 172.21.15.47:0/2339677038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e665c0 0x55f562e645d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f89c0058010 tx=0x7f89c0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a33d606b9a815a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.874+0000 7f89b6ffd640 1 -- 172.21.15.47:0/2339677038 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c005e0b0 con 0x55f562e665c0 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89b6ffd640 1 -- 172.21.15.47:0/2339677038 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f89c0054790 con 0x55f562e665c0 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89e15d5740 1 -- 172.21.15.47:0/2339677038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89d0040470 con 0x55f562e665c0 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89b6ffd640 1 -- 172.21.15.47:0/2339677038 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89c0054950 con 0x55f562e665c0 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89e15d5740 10 client.7890 fetch_fsmap learned FSMap version 52 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89e15d5740 1 -- 172.21.15.47:0/2339677038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55f562d698d0 con 0x55f562e665c0 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89b6ffd640 1 -- 172.21.15.47:0/2339677038 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d0040470 con 0x55f562e665c0 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89b6ffd640 1 -- 172.21.15.47:0/2339677038 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 52) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f89c001b060 con 0x55f562e665c0 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89e15d5740 10 client.7890 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T09:47:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89e15d5740 10 client.7890 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:47:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89e15d5740 20 client.7890 populate_metadata read hostname 'smithi047' 2024-03-23T09:47:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89e15d5740 1 --2- 172.21.15.47:0/2339677038 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55f562e3f010 0x55f562ed1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89e15d5740 4 client.7890 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-03-23T09:47:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89e15d5740 1 -- 172.21.15.47:0/2339677038 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) v1 -- 0x55f562aa8790 con 0x55f562e3f010 2024-03-23T09:47:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89d77fe640 1 --2- 172.21.15.47:0/2339677038 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55f562e3f010 0x55f562ed1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89b6ffd640 10 client.7890 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:47:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.875+0000 7f89d77fe640 1 --2- 172.21.15.47:0/2339677038 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55f562e3f010 0x55f562ed1810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df30fb0494a9d5d server_cookie=7ced7a79a2539da8 in_seq=0 out_seq=0 2024-03-23T09:47:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89b6ffd640 1 -- 172.21.15.47:0/2339677038 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f562aa8790 con 0x55f562e3f010 2024-03-23T09:47:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89b6ffd640 10 client.7890 handle_command_reply: tid=0 2024-03-23T09:47:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89e15d5740 1 client.7890 shutdown 2024-03-23T09:47:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89e15d5740 1 -- 172.21.15.47:0/2339677038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e665c0 msgr2=0x55f562e645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89e15d5740 1 --2- 172.21.15.47:0/2339677038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e665c0 0x55f562e645d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f89c0058010 tx=0x7f89c0009040 comp rx=0 tx=0).stop 2024-03-23T09:47:35.882 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f3d20500640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-03-23T09:47:17.860649+0000 1=1+0) n(v0 rc2024-03-23T09:47:17.976300+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x5650412f2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-23T09:47:17.860649+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-23T09:47:17.976300+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-03-23T09:47:35.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89e15d5740 1 -- 172.21.15.47:0/2339677038 shutdown_connections 2024-03-23T09:47:35.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89e15d5740 1 --2- 172.21.15.47:0/2339677038 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55f562e3f010 0x55f562ed1810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89e15d5740 1 --2- 172.21.15.47:0/2339677038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f562e6ca60 0x55f562ed12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89b6ffd640 0 client.7890 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:47:35.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89e15d5740 1 --2- 172.21.15.47:0/2339677038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f562e669a0 0x55f562e64b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89e15d5740 1 --2- 172.21.15.47:0/2339677038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f562e665c0 0x55f562e645d0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89e15d5740 1 -- 172.21.15.47:0/2339677038 >> 172.21.15.47:0/2339677038 conn(0x55f562e39aa0 msgr2=0x55f562e9ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:35.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89e15d5740 1 -- 172.21.15.47:0/2339677038 shutdown_connections 2024-03-23T09:47:35.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89e15d5740 1 -- 172.21.15.47:0/2339677038 wait complete. 2024-03-23T09:47:35.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.876+0000 7f89e15d5740 20 client.7890 trim_cache size 0 max 16384 2024-03-23T09:47:35.883 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:47:35.883 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-23T09:47:35.883 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "5791d269-e2a5-4e59-a335-896386ef4535", 2024-03-23T09:47:35.884 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-23T09:47:35.884 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:47:35.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.877+0000 7f89de989640 1 -- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f89b806ac00 msgr2=0x7f89b806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.877+0000 7f89de989640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f89b806ac00 0x7f89b806d0c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f89d80c6c20 tx=0x7f89c8008040 comp rx=0 tx=0).stop 2024-03-23T09:47:35.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.877+0000 7f89de989640 1 -- 172.21.15.47:0/161333579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89d80b4340 msgr2=0x7f89d815f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.877+0000 7f89de989640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89d80b4340 0x7f89d815f7b0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c430 tx=0x7f89c000c8f0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.877+0000 7f89de989640 1 -- 172.21.15.47:0/161333579 shutdown_connections 2024-03-23T09:47:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.877+0000 7f89de989640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f89b806ac00 0x7f89b806d0c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.877+0000 7f89de989640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89d815cc40 0x7f89d80ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.877+0000 7f89de989640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89d80cf6b0 0x7f89d81f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.877+0000 7f89de989640 1 --2- 172.21.15.47:0/161333579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89d80b4340 0x7f89d815f7b0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.877+0000 7f89de989640 1 -- 172.21.15.47:0/161333579 >> 172.21.15.47:0/161333579 conn(0x7f89d80659d0 msgr2=0x7f89d815bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.877+0000 7f89de989640 1 -- 172.21.15.47:0/161333579 shutdown_connections 2024-03-23T09:47:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.877+0000 7f89de989640 1 -- 172.21.15.47:0/161333579 wait complete. 2024-03-23T09:47:35.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:47:35.888 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-23T09:47:35.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.990+0000 7fe5c7670640 1 Processor -- start 2024-03-23T09:47:35.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.990+0000 7fe5c7670640 1 -- start start 2024-03-23T09:47:35.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.991+0000 7fe5c7670640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5c015ce10 0x7fe5c00cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.991+0000 7fe5c7670640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c0161600 con 0x7fe5c00b85d0 2024-03-23T09:47:35.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.991+0000 7fe5c7670640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c0161040 con 0x7fe5c015ca30 2024-03-23T09:47:35.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.991+0000 7fe5c7670640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c0160e50 con 0x7fe5c015ce10 2024-03-23T09:47:35.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.991+0000 7fe5c4c0b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5c015ce10 0x7fe5c00cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.991+0000 7fe5c4c0b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5c015ce10 0x7fe5c00cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53426/0 (socket says 172.21.15.47:53426) 2024-03-23T09:47:35.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.991+0000 7fe5c4c0b640 1 -- 172.21.15.47:0/1487740125 learned_addr learned my addr 172.21.15.47:0/1487740125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:35.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.991+0000 7fe5c5c0d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe5c00b85d0 0x7fe5c00d15d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36828/0 (socket says 172.21.15.47:36828) 2024-03-23T09:47:35.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.991+0000 7fe5c4c0b640 1 -- 172.21.15.47:0/1487740125 >> v1:172.21.15.175:6789/0 conn(0x7fe5c015ca30 legacy=0x7fe5c0160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:35.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.991+0000 7fe5c4c0b640 1 -- 172.21.15.47:0/1487740125 >> v1:172.21.15.47:6789/0 conn(0x7fe5c00b85d0 legacy=0x7fe5c00d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.991+0000 7fe5c4c0b640 1 -- 172.21.15.47:0/1487740125 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c0161320 con 0x7fe5c015ce10 2024-03-23T09:47:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.992+0000 7fe5c4c0b640 1 --2- 172.21.15.47:0/1487740125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5c015ce10 0x7fe5c00cd730 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009870 tx=0x7fe5b402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2226d62dbf0afa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.992+0000 7fe5bffff640 1 -- 172.21.15.47:0/1487740125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4030860 con 0x7fe5c015ce10 2024-03-23T09:47:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.992+0000 7fe5bffff640 1 -- 172.21.15.47:0/1487740125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b40309c0 con 0x7fe5c015ce10 2024-03-23T09:47:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.992+0000 7fe5bffff640 1 -- 172.21.15.47:0/1487740125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4030d50 con 0x7fe5c015ce10 2024-03-23T09:47:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.993+0000 7fe5c7670640 1 -- 172.21.15.47:0/1487740125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5c015ce10 msgr2=0x7fe5c00cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.993+0000 7fe5c7670640 1 --2- 172.21.15.47:0/1487740125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5c015ce10 0x7fe5c00cd730 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009870 tx=0x7fe5b402b9a0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.993+0000 7fe5c7670640 1 -- 172.21.15.47:0/1487740125 shutdown_connections 2024-03-23T09:47:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.993+0000 7fe5c7670640 1 --2- 172.21.15.47:0/1487740125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5c015ce10 0x7fe5c00cd730 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.993+0000 7fe5c7670640 1 -- 172.21.15.47:0/1487740125 >> 172.21.15.47:0/1487740125 conn(0x7fe5c006b160 msgr2=0x7fe5c006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.993+0000 7fe5c7670640 1 -- 172.21.15.47:0/1487740125 shutdown_connections 2024-03-23T09:47:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.993+0000 7fe5c7670640 1 -- 172.21.15.47:0/1487740125 wait complete. 2024-03-23T09:47:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.994+0000 7fe5c7670640 1 Processor -- start 2024-03-23T09:47:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.994+0000 7fe5c7670640 1 -- start start 2024-03-23T09:47:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c7670640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5c00b85d0 0x7fe5c01539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c7670640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c015ca30 0x7fe5c00b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c7670640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c015ce10 0x7fe5c00c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c7670640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c01f55e0 con 0x7fe5c00b85d0 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c7670640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c01f58c0 con 0x7fe5c015ca30 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c7670640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c01f5ba0 con 0x7fe5c015ce10 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c540c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5c00b85d0 0x7fe5c01539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c540c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5c00b85d0 0x7fe5c01539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53436/0 (socket says 172.21.15.47:53436) 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c540c640 1 -- 172.21.15.47:0/437800519 learned_addr learned my addr 172.21.15.47:0/437800519 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c4c0b640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c015ca30 0x7fe5c00b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c5c0d640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c015ce10 0x7fe5c00c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c540c640 1 -- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c015ce10 msgr2=0x7fe5c00c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c540c640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c015ce10 0x7fe5c00c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c540c640 1 -- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c015ca30 msgr2=0x7fe5c00b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c540c640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c015ca30 0x7fe5c00b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c540c640 1 -- 172.21.15.47:0/437800519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b40094d0 con 0x7fe5c00b85d0 2024-03-23T09:47:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c4c0b640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c015ca30 0x7fe5c00b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:47:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c540c640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5c00b85d0 0x7fe5c01539d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800c430 tx=0x7fe5b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5bdffb640 1 -- 172.21.15.47:0/437800519 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8011790 con 0x7fe5c00b85d0 2024-03-23T09:47:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5bdffb640 1 -- 172.21.15.47:0/437800519 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b80118f0 con 0x7fe5c00b85d0 2024-03-23T09:47:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.995+0000 7fe5c7670640 1 -- 172.21.15.47:0/437800519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c01f5e80 con 0x7fe5c00b85d0 2024-03-23T09:47:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.996+0000 7fe5bdffb640 1 -- 172.21.15.47:0/437800519 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8011ce0 con 0x7fe5c00b85d0 2024-03-23T09:47:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.996+0000 7fe5c7670640 1 -- 172.21.15.47:0/437800519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c0153f10 con 0x7fe5c00b85d0 2024-03-23T09:47:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.997+0000 7fe5bdffb640 1 -- 172.21.15.47:0/437800519 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe5b801c460 con 0x7fe5c00b85d0 2024-03-23T09:47:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.997+0000 7fe5bdffb640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5ac06ac00 0x7fe5ac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.997+0000 7fe5bdffb640 1 -- 172.21.15.47:0/437800519 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe5b80893b0 con 0x7fe5c00b85d0 2024-03-23T09:47:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.998+0000 7fe5c4c0b640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5ac06ac00 0x7fe5ac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.998+0000 7fe5c4c0b640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5ac06ac00 0x7fe5ac06d0c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4036000 tx=0x7fe5b4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.999+0000 7fe5ca1e3740 1 Processor -- start 2024-03-23T09:47:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.999+0000 7fe5ca1e3740 1 -- start start 2024-03-23T09:47:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.999+0000 7fe5ca1e3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.999+0000 7fe5ca1e3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c749c850 0x55e6c74c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.999+0000 7fe5ca1e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 0x55e6c74c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.999+0000 7fe5ca1e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6c740cdd0 con 0x55e6c74a03b0 2024-03-23T09:47:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.999+0000 7fe5ca1e3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6c740a560 con 0x55e6c74cc830 2024-03-23T09:47:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.999+0000 7fe5ca1e3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6c7405e50 con 0x55e6c749c850 2024-03-23T09:47:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.999+0000 7fe5c5c0d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 0x55e6c74c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.999+0000 7fe5c5c0d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 0x55e6c74c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53446/0 (socket says 172.21.15.47:53446) 2024-03-23T09:47:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.999+0000 7fe5c5c0d640 1 -- 172.21.15.47:0/2395518362 learned_addr learned my addr 172.21.15.47:0/2395518362 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.999+0000 7fe5c540c640 1 --2- 172.21.15.47:0/2395518362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:35.999+0000 7fe5c4c0b640 1 --2- 172.21.15.47:0/2395518362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c749c850 0x55e6c74c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.000+0000 7fe5c5c0d640 1 -- 172.21.15.47:0/2395518362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c749c850 msgr2=0x55e6c74c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.000+0000 7fe5c5c0d640 1 --2- 172.21.15.47:0/2395518362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c749c850 0x55e6c74c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.000+0000 7fe5c5c0d640 1 -- 172.21.15.47:0/2395518362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 msgr2=0x55e6c74ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.000+0000 7fe5c5c0d640 1 --2- 172.21.15.47:0/2395518362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.000+0000 7fe5c5c0d640 1 -- 172.21.15.47:0/2395518362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6c73cd470 con 0x55e6c74a03b0 2024-03-23T09:47:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.000+0000 7fe5c5c0d640 1 --2- 172.21.15.47:0/2395518362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 0x55e6c74c9fe0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00073e0 tx=0x7fe5b00078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d827c515d55ca892 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.000+0000 7fe5a6ffd640 1 -- 172.21.15.47:0/2395518362 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0001c60 con 0x55e6c74a03b0 2024-03-23T09:47:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.000+0000 7fe5a6ffd640 1 -- 172.21.15.47:0/2395518362 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b0001dc0 con 0x55e6c74a03b0 2024-03-23T09:47:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.000+0000 7fe5a6ffd640 1 -- 172.21.15.47:0/2395518362 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0014480 con 0x55e6c74a03b0 2024-03-23T09:47:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.001+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/2395518362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 msgr2=0x55e6c74c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.001+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/2395518362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 0x55e6c74c9fe0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00073e0 tx=0x7fe5b00078a0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.001+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/2395518362 shutdown_connections 2024-03-23T09:47:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.001+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/2395518362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 0x55e6c74c9fe0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.001+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/2395518362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c749c850 0x55e6c74c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.001+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/2395518362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.001+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/2395518362 >> 172.21.15.47:0/2395518362 conn(0x55e6c749eac0 msgr2=0x55e6c74e7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.001+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/2395518362 shutdown_connections 2024-03-23T09:47:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.001+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/2395518362 wait complete. 2024-03-23T09:47:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.001+0000 7fe5ca1e3740 1 -- start start 2024-03-23T09:47:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.001+0000 7fe5ca1e3740 1 Processor -- start 2024-03-23T09:47:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.002+0000 7fe5ca1e3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74a03b0 0x55e6c74b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.002+0000 7fe5ca1e3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74dda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.002+0000 7fe5ca1e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 0x55e6c74c3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.002+0000 7fe5ca1e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6c73c4280 con 0x55e6c74c6430 2024-03-23T09:47:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.002+0000 7fe5ca1e3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6c73c3f20 con 0x55e6c74cc830 2024-03-23T09:47:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.002+0000 7fe5ca1e3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6c73893c0 con 0x55e6c74a03b0 2024-03-23T09:47:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.002+0000 7fe5ca1e3740 10 client.0 _ll_register_callbacks cb 0x55e6c73db380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:47:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.002+0000 7fe5c5c0d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 0x55e6c74c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.002+0000 7fe5c5c0d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 0x55e6c74c3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53458/0 (socket says 172.21.15.47:53458) 2024-03-23T09:47:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.002+0000 7fe5c5c0d640 1 -- 172.21.15.47:0/449889901 learned_addr learned my addr 172.21.15.47:0/449889901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.002+0000 7fe5c4c0b640 1 --2- 172.21.15.47:0/449889901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74dda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.002+0000 7fe5c540c640 1 --2- 172.21.15.47:0/449889901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74a03b0 0x55e6c74b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5c5c0d640 1 -- 172.21.15.47:0/449889901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74a03b0 msgr2=0x55e6c74b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5c5c0d640 1 --2- 172.21.15.47:0/449889901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74a03b0 0x55e6c74b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5c5c0d640 1 -- 172.21.15.47:0/449889901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 msgr2=0x55e6c74dda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5c5c0d640 1 --2- 172.21.15.47:0/449889901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74dda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5c5c0d640 1 -- 172.21.15.47:0/449889901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b000f040 con 0x55e6c74c6430 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5c5c0d640 1 --2- 172.21.15.47:0/449889901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 0x55e6c74c3810 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0001840 tx=0x7fe5b0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee1418771a60b77 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe59ffff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe59ffff640 1 -- 172.21.15.47:0/449889901 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0001c60 con 0x55e6c74c6430 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe59ffff640 1 -- 172.21.15.47:0/449889901 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe5b0014690 con 0x55e6c74c6430 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/449889901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6c742f690 con 0x55e6c74c6430 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe59ffff640 1 -- 172.21.15.47:0/449889901 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5b0014850 con 0x55e6c74c6430 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5ca1e3740 10 client.7905 fetch_fsmap learned FSMap version 52 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/449889901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55e6c74044d0 con 0x55e6c74c6430 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe59ffff640 1 -- 172.21.15.47:0/449889901 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e6c742f690 con 0x55e6c74c6430 2024-03-23T09:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe59ffff640 1 -- 172.21.15.47:0/449889901 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 52) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe5b00149b0 con 0x55e6c74c6430 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5ca1e3740 10 client.7905 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5ca1e3740 10 client.7905 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5ca1e3740 20 client.7905 populate_metadata read hostname 'smithi047' 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/449889901 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55e6c74cd0e0 0x55e6c74fa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5ca1e3740 4 client.7905 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/449889901 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6c73f7eb0 con 0x55e6c74cd0e0 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.003+0000 7fe5c540c640 1 --2- 172.21.15.47:0/449889901 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55e6c74cd0e0 0x55e6c74fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe5c540c640 1 --2- 172.21.15.47:0/449889901 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55e6c74cd0e0 0x55e6c74fa800 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47c56852c373af90 server_cookie=90fd599a0e1814c8 in_seq=0 out_seq=0 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe59ffff640 10 client.7905 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe59ffff640 1 -- 172.21.15.47:0/449889901 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e6c73f7eb0 con 0x55e6c74cd0e0 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe59ffff640 10 client.7905 handle_command_reply: tid=0 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe5ca1e3740 1 client.7905 shutdown 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/449889901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 msgr2=0x55e6c74c3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/449889901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 0x55e6c74c3810 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0001840 tx=0x7fe5b0001250 comp rx=0 tx=0).stop 2024-03-23T09:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/449889901 shutdown_connections 2024-03-23T09:47:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/449889901 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55e6c74cd0e0 0x55e6c74fa800 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/449889901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 0x55e6c74c3810 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/449889901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74dda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe59ffff640 0 client.7905 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:47:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/449889901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74a03b0 0x55e6c74b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.004+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/449889901 >> 172.21.15.47:0/449889901 conn(0x55e6c749c850 msgr2=0x55e6c74eab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.005+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/449889901 shutdown_connections 2024-03-23T09:47:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.005+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/449889901 wait complete. 2024-03-23T09:47:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.005+0000 7fe5ca1e3740 20 client.7905 trim_cache size 0 max 16384 2024-03-23T09:47:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.019+0000 7fe5ca1e3740 1 Processor -- start 2024-03-23T09:47:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.019+0000 7fe5ca1e3740 1 -- start start 2024-03-23T09:47:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5ca1e3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74a03b0 0x55e6c74f6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5ca1e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 0x55e6c74c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5ca1e3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5ca1e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6c74f5e40 con 0x55e6c74c6430 2024-03-23T09:47:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5ca1e3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6c74f6d00 con 0x55e6c74cc830 2024-03-23T09:47:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5ca1e3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6c74fa4a0 con 0x55e6c74a03b0 2024-03-23T09:47:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5c4c0b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 0x55e6c74c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5c4c0b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 0x55e6c74c3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53474/0 (socket says 172.21.15.47:53474) 2024-03-23T09:47:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5c4c0b640 1 -- 172.21.15.47:0/2284417240 learned_addr learned my addr 172.21.15.47:0/2284417240 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5c4c0b640 1 -- 172.21.15.47:0/2284417240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74a03b0 msgr2=0x55e6c74f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5c4c0b640 1 --2- 172.21.15.47:0/2284417240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74a03b0 0x55e6c74f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5c4c0b640 1 -- 172.21.15.47:0/2284417240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 msgr2=0x55e6c74c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5c4c0b640 1 --2- 172.21.15.47:0/2284417240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5c4c0b640 1 -- 172.21.15.47:0/2284417240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6c74389e0 con 0x55e6c74c6430 2024-03-23T09:47:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5c4c0b640 1 --2- 172.21.15.47:0/2284417240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 0x55e6c74c3750 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009990 tx=0x7fe5b4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529868555d734489 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5a77fe640 1 -- 172.21.15.47:0/2284417240 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b404b040 con 0x55e6c74c6430 2024-03-23T09:47:36.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5a77fe640 1 -- 172.21.15.47:0/2284417240 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b4039710 con 0x55e6c74c6430 2024-03-23T09:47:36.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.020+0000 7fe5a77fe640 1 -- 172.21.15.47:0/2284417240 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4039a80 con 0x55e6c74c6430 2024-03-23T09:47:36.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.021+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/2284417240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 msgr2=0x55e6c74c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:36.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.021+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/2284417240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 0x55e6c74c3750 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009990 tx=0x7fe5b4030000 comp rx=0 tx=0).stop 2024-03-23T09:47:36.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.021+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/2284417240 shutdown_connections 2024-03-23T09:47:36.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.021+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/2284417240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.021+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/2284417240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74c6430 0x55e6c74c3750 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.021+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/2284417240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74a03b0 0x55e6c74f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.021+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/2284417240 >> 172.21.15.47:0/2284417240 conn(0x55e6c749c850 msgr2=0x55e6c74981f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:36.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.021+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/2284417240 shutdown_connections 2024-03-23T09:47:36.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.021+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/2284417240 wait complete. 2024-03-23T09:47:36.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.021+0000 7fe5ca1e3740 1 -- start start 2024-03-23T09:47:36.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.022+0000 7fe5ca1e3740 1 Processor -- start 2024-03-23T09:47:36.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.022+0000 7fe5ca1e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 0x55e6c7533430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.022+0000 7fe5ca1e3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74c6430 0x55e6c74c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.022+0000 7fe5ca1e3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.022+0000 7fe5ca1e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6c7501720 con 0x55e6c74a03b0 2024-03-23T09:47:36.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.022+0000 7fe5ca1e3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6c75511d0 con 0x55e6c74cc830 2024-03-23T09:47:36.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.022+0000 7fe5ca1e3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6c74f6510 con 0x55e6c74c6430 2024-03-23T09:47:36.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.022+0000 7fe5ca1e3740 10 client.0 _ll_register_callbacks cb 0x7fe5b0005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5c540c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 0x55e6c7533430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5c540c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 0x55e6c7533430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53476/0 (socket says 172.21.15.47:53476) 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5c540c640 1 -- 172.21.15.47:0/743952780 learned_addr learned my addr 172.21.15.47:0/743952780 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5c5c0d640 1 --2- 172.21.15.47:0/743952780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5c4c0b640 1 --2- 172.21.15.47:0/743952780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74c6430 0x55e6c74c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5c540c640 1 -- 172.21.15.47:0/743952780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74c6430 msgr2=0x55e6c74c36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5c540c640 1 --2- 172.21.15.47:0/743952780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74c6430 0x55e6c74c36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5c540c640 1 -- 172.21.15.47:0/743952780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 msgr2=0x55e6c74c3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5c540c640 1 --2- 172.21.15.47:0/743952780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74c3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5c540c640 1 -- 172.21.15.47:0/743952780 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c0161320 con 0x55e6c74a03b0 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5c540c640 1 --2- 172.21.15.47:0/743952780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 0x55e6c7533430 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8057ee0 tx=0x7fe5b8003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc37dbe79dd92f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe59ffff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe59ffff640 1 -- 172.21.15.47:0/743952780 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80605d0 con 0x55e6c74a03b0 2024-03-23T09:47:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe59ffff640 1 -- 172.21.15.47:0/743952780 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe5b8056580 con 0x55e6c74a03b0 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/743952780 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5b0001dc0 con 0x55e6c74a03b0 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe59ffff640 1 -- 172.21.15.47:0/743952780 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5b80890b0 con 0x55e6c74a03b0 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5ca1e3740 10 client.7911 fetch_fsmap learned FSMap version 52 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/743952780 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55e6c73a65c0 con 0x55e6c74a03b0 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.023+0000 7fe59ffff640 1 -- 172.21.15.47:0/743952780 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b0001dc0 con 0x55e6c74a03b0 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe59ffff640 1 -- 172.21.15.47:0/743952780 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 52) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe5b80813f0 con 0x55e6c74a03b0 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 10 client.7911 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 10 client.7911 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 20 client.7911 populate_metadata read hostname 'smithi047' 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/743952780 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55e6c7550a60 0x55e6c74e9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 4 client.7911 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/743952780 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e6c70f94e0 con 0x55e6c7550a60 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5c4c0b640 1 --2- 172.21.15.47:0/743952780 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55e6c7550a60 0x55e6c74e9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe59ffff640 10 client.7911 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5c4c0b640 1 --2- 172.21.15.47:0/743952780 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55e6c7550a60 0x55e6c74e9210 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=260eca7f91e671b1 server_cookie=588cd06ba75ce4a6 in_seq=0 out_seq=0 2024-03-23T09:47:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe59ffff640 1 -- 172.21.15.47:0/743952780 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+256 (crc 0 0 0) 0x55e6c70f94e0 con 0x55e6c7550a60 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe59ffff640 10 client.7911 handle_command_reply: tid=0 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 1 client.7911 shutdown 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/743952780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 msgr2=0x55e6c7533430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/743952780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 0x55e6c7533430 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8057ee0 tx=0x7fe5b8003980 comp rx=0 tx=0).stop 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/743952780 shutdown_connections 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/743952780 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55e6c7550a60 0x55e6c74e9210 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/743952780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6c74cc830 0x55e6c74c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/743952780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e6c74c6430 0x55e6c74c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe59ffff640 0 client.7911 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 1 --2- 172.21.15.47:0/743952780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6c74a03b0 0x55e6c7533430 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.024+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/743952780 >> 172.21.15.47:0/743952780 conn(0x55e6c749c850 msgr2=0x55e6c74ea260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/743952780 shutdown_connections 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5ca1e3740 1 -- 172.21.15.47:0/743952780 wait complete. 2024-03-23T09:47:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5ca1e3740 20 client.7911 trim_cache size 0 max 16384 2024-03-23T09:47:36.031 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:47:36.031 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T09:47:36.031 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-23T09:47:36.031 INFO:teuthology.orchestra.run.smithi047.stdout: "5791d269-e2a5-4e59-a335-896386ef4535": { 2024-03-23T09:47:36.031 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/dir", 2024-03-23T09:47:36.031 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "5791d269-e2a5-4e59-a335-896386ef4535", 2024-03-23T09:47:36.031 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive" 2024-03-23T09:47:36.031 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:36.031 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:47:36.031 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:47:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5c7670640 1 -- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5ac06ac00 msgr2=0x7fe5ac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5c7670640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5ac06ac00 0x7fe5ac06d0c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4036000 tx=0x7fe5b4008030 comp rx=0 tx=0).stop 2024-03-23T09:47:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5c7670640 1 -- 172.21.15.47:0/437800519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5c00b85d0 msgr2=0x7fe5c01539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:47:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5c7670640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5c00b85d0 0x7fe5c01539d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800c430 tx=0x7fe5b800c8f0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5c7670640 1 -- 172.21.15.47:0/437800519 shutdown_connections 2024-03-23T09:47:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5c7670640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe5ac06ac00 0x7fe5ac06d0c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5c7670640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c015ce10 0x7fe5c00c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5c7670640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c015ca30 0x7fe5c00b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5c7670640 1 --2- 172.21.15.47:0/437800519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5c00b85d0 0x7fe5c01539d0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:47:36.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5c7670640 1 -- 172.21.15.47:0/437800519 >> 172.21.15.47:0/437800519 conn(0x7fe5c006b160 msgr2=0x7fe5c015a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:47:36.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5c7670640 1 -- 172.21.15.47:0/437800519 shutdown_connections 2024-03-23T09:47:36.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:47:36.025+0000 7fe5c7670640 1 -- 172.21.15.47:0/437800519 wait complete. 2024-03-23T09:47:36.036 INFO:tasks.cephfs.filesystem:scrub status for tag:5791d269-e2a5-4e59-a335-896386ef4535 - {'path': '/dir', 'tag': '5791d269-e2a5-4e59-a335-896386ef4535', 'options': 'recursive'} 2024-03-23T09:47:38.105 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:47:38.108 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:47:39.842 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:47:39.842+0000 7fefe5743640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T09:48:06.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:06.037 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-23T09:48:06.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.154+0000 7f57333d2640 1 Processor -- start 2024-03-23T09:48:06.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.154+0000 7f57333d2640 1 -- start start 2024-03-23T09:48:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f57333d2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f572c06e080 0x7f572c159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f57333d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f572c0b0f00 0x7f572c0701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f57333d2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572c159ac0 con 0x7f572c0b0a70 2024-03-23T09:48:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f57333d2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f572c15db20 con 0x7f572c06e080 2024-03-23T09:48:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f57333d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572c1594e0 con 0x7f572c0b0f00 2024-03-23T09:48:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f573116e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f572c06e080 0x7f572c159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f573096d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f572c0b0a70 0x7f572c15cb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42438/0 (socket says 172.21.15.47:42438) 2024-03-23T09:48:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f573096d640 1 -- 172.21.15.47:0/1749899601 learned_addr learned my addr 172.21.15.47:0/1749899601 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f573196f640 1 --2- 172.21.15.47:0/1749899601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f572c0b0f00 0x7f572c0701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f573116e640 1 -- 172.21.15.47:0/1749899601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f572c0b0f00 msgr2=0x7f572c0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f573116e640 1 --2- 172.21.15.47:0/1749899601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f572c0b0f00 0x7f572c0701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f573116e640 1 -- 172.21.15.47:0/1749899601 >> v1:172.21.15.47:6789/0 conn(0x7f572c0b0a70 legacy=0x7f572c15cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f5723fff640 1 -- 172.21.15.47:0/1749899601 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81894648 0 0) 0x7f572c159ac0 con 0x7f572c0b0a70 2024-03-23T09:48:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f573116e640 1 -- 172.21.15.47:0/1749899601 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c15a490 con 0x7f572c06e080 2024-03-23T09:48:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.155+0000 7f573196f640 1 --2- 172.21.15.47:0/1749899601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f572c0b0f00 0x7f572c0701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.156+0000 7f573116e640 1 --2- 172.21.15.47:0/1749899601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c06e080 0x7f572c159f50 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f57280097a0 tx=0x7f572802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c922998a58955eb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.156+0000 7f5723fff640 1 -- 172.21.15.47:0/1749899601 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572802d040 con 0x7f572c06e080 2024-03-23T09:48:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.156+0000 7f5723fff640 1 -- 172.21.15.47:0/1749899601 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5728031810 con 0x7f572c06e080 2024-03-23T09:48:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.156+0000 7f5723fff640 1 -- 172.21.15.47:0/1749899601 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728031ba0 con 0x7f572c06e080 2024-03-23T09:48:06.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.157+0000 7f57333d2640 1 -- 172.21.15.47:0/1749899601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c06e080 msgr2=0x7f572c159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.157+0000 7f57333d2640 1 --2- 172.21.15.47:0/1749899601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c06e080 0x7f572c159f50 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f57280097a0 tx=0x7f572802b8b0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.157+0000 7f57333d2640 1 -- 172.21.15.47:0/1749899601 shutdown_connections 2024-03-23T09:48:06.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.157+0000 7f57333d2640 1 --2- 172.21.15.47:0/1749899601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f572c0b0f00 0x7f572c0701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.157+0000 7f57333d2640 1 --2- 172.21.15.47:0/1749899601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c06e080 0x7f572c159f50 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.157+0000 7f57333d2640 1 -- 172.21.15.47:0/1749899601 >> 172.21.15.47:0/1749899601 conn(0x7f572c06a310 msgr2=0x7f572c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.157+0000 7f57333d2640 1 -- 172.21.15.47:0/1749899601 shutdown_connections 2024-03-23T09:48:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.157+0000 7f57333d2640 1 -- 172.21.15.47:0/1749899601 wait complete. 2024-03-23T09:48:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.158+0000 7f57333d2640 1 Processor -- start 2024-03-23T09:48:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.158+0000 7f57333d2640 1 -- start start 2024-03-23T09:48:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f57333d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f572c06e080 0x7f572c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f57333d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f572c0b0a70 0x7f572c14abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f57333d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c0b0f00 0x7f572c14b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f57333d2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c14bce0 con 0x7f572c0b0f00 2024-03-23T09:48:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f57333d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c14bfc0 con 0x7f572c0b0a70 2024-03-23T09:48:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f57333d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572c14c2a0 con 0x7f572c06e080 2024-03-23T09:48:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f573196f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c0b0f00 0x7f572c14b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f573196f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c0b0f00 0x7f572c14b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38536/0 (socket says 172.21.15.47:38536) 2024-03-23T09:48:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f573196f640 1 -- 172.21.15.47:0/2816112230 learned_addr learned my addr 172.21.15.47:0/2816112230 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f573096d640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f572c0b0a70 0x7f572c14abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f573196f640 1 -- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f572c06e080 msgr2=0x7f572c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f573116e640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f572c06e080 0x7f572c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f573196f640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f572c06e080 0x7f572c0ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f573196f640 1 -- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f572c0b0a70 msgr2=0x7f572c14abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f573196f640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f572c0b0a70 0x7f572c14abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f573196f640 1 -- 172.21.15.47:0/2816112230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5728009400 con 0x7f572c0b0f00 2024-03-23T09:48:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f573096d640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f572c0b0a70 0x7f572c14abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.159+0000 7f573196f640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c0b0f00 0x7f572c14b100 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f572400cea0 tx=0x7f572400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.160+0000 7f5721ffb640 1 -- 172.21.15.47:0/2816112230 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724003b90 con 0x7f572c0b0f00 2024-03-23T09:48:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.160+0000 7f5721ffb640 1 -- 172.21.15.47:0/2816112230 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5724003d10 con 0x7f572c0b0f00 2024-03-23T09:48:06.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.160+0000 7f5721ffb640 1 -- 172.21.15.47:0/2816112230 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724004260 con 0x7f572c0b0f00 2024-03-23T09:48:06.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.160+0000 7f57333d2640 1 -- 172.21.15.47:0/2816112230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572c14c580 con 0x7f572c0b0f00 2024-03-23T09:48:06.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.160+0000 7f57333d2640 1 -- 172.21.15.47:0/2816112230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572c14cd80 con 0x7f572c0b0f00 2024-03-23T09:48:06.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.161+0000 7f5721ffb640 1 -- 172.21.15.47:0/2816112230 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f57240043c0 con 0x7f572c0b0f00 2024-03-23T09:48:06.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.162+0000 7f5721ffb640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f571006ac00 0x7f571006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.162+0000 7f5721ffb640 1 -- 172.21.15.47:0/2816112230 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5724089d30 con 0x7f572c0b0f00 2024-03-23T09:48:06.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.162+0000 7f573116e640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f571006ac00 0x7f571006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.163+0000 7f573116e640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f571006ac00 0x7f571006d0c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5728036000 tx=0x7f572803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.163+0000 7f5736026740 1 Processor -- start 2024-03-23T09:48:06.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.163+0000 7f5736026740 1 -- start start 2024-03-23T09:48:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f5736026740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d9190dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f5736026740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d918dd850 0x560d9190aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f5736026740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 0x560d9190afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f5736026740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d9184ddd0 con 0x560d9190d830 2024-03-23T09:48:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f5736026740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d9184b560 con 0x560d918dd850 2024-03-23T09:48:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f5736026740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d91846e50 con 0x560d918e13b0 2024-03-23T09:48:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f573116e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d9190dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f573116e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d9190dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38548/0 (socket says 172.21.15.47:38548) 2024-03-23T09:48:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f573116e640 1 -- 172.21.15.47:0/3795077945 learned_addr learned my addr 172.21.15.47:0/3795077945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f573096d640 1 --2- 172.21.15.47:0/3795077945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d918dd850 0x560d9190aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f573196f640 1 --2- 172.21.15.47:0/3795077945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 0x560d9190afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f573116e640 1 -- 172.21.15.47:0/3795077945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 msgr2=0x560d9190afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f573116e640 1 --2- 172.21.15.47:0/3795077945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 0x560d9190afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f573116e640 1 -- 172.21.15.47:0/3795077945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d918dd850 msgr2=0x560d9190aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f573116e640 1 --2- 172.21.15.47:0/3795077945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d918dd850 0x560d9190aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f573116e640 1 -- 172.21.15.47:0/3795077945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d9180e470 con 0x560d9190d830 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f573096d640 1 --2- 172.21.15.47:0/3795077945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d918dd850 0x560d9190aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.164+0000 7f573116e640 1 --2- 172.21.15.47:0/3795077945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d9190dc10 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f5728009890 tx=0x7f5728030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33014b186802883 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.165+0000 7f570effd640 1 -- 172.21.15.47:0/3795077945 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728050480 con 0x560d9190d830 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.165+0000 7f570effd640 1 -- 172.21.15.47:0/3795077945 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5728049070 con 0x560d9190d830 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.165+0000 7f570effd640 1 -- 172.21.15.47:0/3795077945 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280507e0 con 0x560d9190d830 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.165+0000 7f5736026740 1 -- 172.21.15.47:0/3795077945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 msgr2=0x560d9190dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.165+0000 7f5736026740 1 --2- 172.21.15.47:0/3795077945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d9190dc10 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f5728009890 tx=0x7f5728030000 comp rx=0 tx=0).stop 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.165+0000 7f5736026740 1 -- 172.21.15.47:0/3795077945 shutdown_connections 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.165+0000 7f5736026740 1 --2- 172.21.15.47:0/3795077945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 0x560d9190afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.165+0000 7f5736026740 1 --2- 172.21.15.47:0/3795077945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d918dd850 0x560d9190aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.165+0000 7f5736026740 1 --2- 172.21.15.47:0/3795077945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d9190dc10 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.165+0000 7f5736026740 1 -- 172.21.15.47:0/3795077945 >> 172.21.15.47:0/3795077945 conn(0x560d918dfac0 msgr2=0x560d91928640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.165+0000 7f5736026740 1 -- 172.21.15.47:0/3795077945 shutdown_connections 2024-03-23T09:48:06.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.165+0000 7f5736026740 1 -- 172.21.15.47:0/3795077945 wait complete. 2024-03-23T09:48:06.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.166+0000 7f5736026740 1 -- start start 2024-03-23T09:48:06.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.166+0000 7f5736026740 1 Processor -- start 2024-03-23T09:48:06.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f5736026740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d918e13b0 0x560d9192bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f5736026740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d9190d830 0x560d91910b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f5736026740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190f4f0 0x560d919047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f5736026740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d91805280 con 0x560d9190f4f0 2024-03-23T09:48:06.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f5736026740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d91804f20 con 0x560d918e13b0 2024-03-23T09:48:06.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f5736026740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d917ca3c0 con 0x560d9190d830 2024-03-23T09:48:06.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f5736026740 10 client.0 _ll_register_callbacks cb 0x560d9181c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:06.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f573196f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190f4f0 0x560d919047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f573196f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190f4f0 0x560d919047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38552/0 (socket says 172.21.15.47:38552) 2024-03-23T09:48:06.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f573196f640 1 -- 172.21.15.47:0/1548156285 learned_addr learned my addr 172.21.15.47:0/1548156285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:06.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f573116e640 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d918e13b0 0x560d9192bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f573096d640 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d9190d830 0x560d91910b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f573196f640 1 -- 172.21.15.47:0/1548156285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d9190d830 msgr2=0x560d91910b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f573196f640 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d9190d830 0x560d91910b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f573196f640 1 -- 172.21.15.47:0/1548156285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d918e13b0 msgr2=0x560d9192bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f573196f640 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d918e13b0 0x560d9192bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f573196f640 1 -- 172.21.15.47:0/1548156285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572c15a490 con 0x560d9190f4f0 2024-03-23T09:48:06.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f573096d640 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d9190d830 0x560d91910b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:06.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f573196f640 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190f4f0 0x560d919047d0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f572400ce70 tx=0x7f5724058c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5670107f54d7629e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.167+0000 7f5707fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:06.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5707fff640 1 -- 172.21.15.47:0/1548156285 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724004870 con 0x560d9190f4f0 2024-03-23T09:48:06.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5707fff640 1 -- 172.21.15.47:0/1548156285 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5724058020 con 0x560d9190f4f0 2024-03-23T09:48:06.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5736026740 1 -- 172.21.15.47:0/1548156285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d91870690 con 0x560d9190f4f0 2024-03-23T09:48:06.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5707fff640 1 -- 172.21.15.47:0/1548156285 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57240581e0 con 0x560d9190f4f0 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5736026740 10 client.7926 fetch_fsmap learned FSMap version 53 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5736026740 1 -- 172.21.15.47:0/1548156285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x560d918454d0 con 0x560d9190f4f0 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5707fff640 1 -- 172.21.15.47:0/1548156285 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5724074070 con 0x560d9190f4f0 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5707fff640 1 -- 172.21.15.47:0/1548156285 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 53) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5724008040 con 0x560d9190f4f0 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5736026740 10 client.7926 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5736026740 10 client.7926 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5736026740 20 client.7926 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5736026740 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x560d91901f20 0x560d91902300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5736026740 4 client.7926 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f5736026740 1 -- 172.21.15.47:0/1548156285 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d91838eb0 con 0x560d91901f20 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.168+0000 7f573116e640 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x560d91901f20 0x560d91902300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f573116e640 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x560d91901f20 0x560d91902300 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9624d566bfdbca54 server_cookie=ecf1426ef2bfc644 in_seq=0 out_seq=0 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5707fff640 10 client.7926 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5707fff640 1 -- 172.21.15.47:0/1548156285 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560d91838eb0 con 0x560d91901f20 2024-03-23T09:48:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5707fff640 10 client.7926 handle_command_reply: tid=0 2024-03-23T09:48:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5736026740 1 client.7926 shutdown 2024-03-23T09:48:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5736026740 1 -- 172.21.15.47:0/1548156285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190f4f0 msgr2=0x560d919047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5736026740 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190f4f0 0x560d919047d0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f572400ce70 tx=0x7f5724058c70 comp rx=0 tx=0).stop 2024-03-23T09:48:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5736026740 1 -- 172.21.15.47:0/1548156285 shutdown_connections 2024-03-23T09:48:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5736026740 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x560d91901f20 0x560d91902300 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5736026740 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190f4f0 0x560d919047d0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5707fff640 0 client.7926 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5736026740 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d9190d830 0x560d91910b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5736026740 1 --2- 172.21.15.47:0/1548156285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d918e13b0 0x560d9192bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5736026740 1 -- 172.21.15.47:0/1548156285 >> 172.21.15.47:0/1548156285 conn(0x560d918dd850 msgr2=0x560d918e1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5736026740 1 -- 172.21.15.47:0/1548156285 shutdown_connections 2024-03-23T09:48:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.169+0000 7f5736026740 1 -- 172.21.15.47:0/1548156285 wait complete. 2024-03-23T09:48:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.170+0000 7f5736026740 20 client.7926 trim_cache size 0 max 16384 2024-03-23T09:48:06.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.184+0000 7f5736026740 1 Processor -- start 2024-03-23T09:48:06.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.184+0000 7f5736026740 1 -- start start 2024-03-23T09:48:06.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f5736026740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 0x560d91902300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f5736026740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d91902840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f5736026740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d9190f4f0 0x560d919413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f5736026740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d91903ec0 con 0x560d9190d830 2024-03-23T09:48:06.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f5736026740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d91910bc0 con 0x560d9190f4f0 2024-03-23T09:48:06.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f5736026740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d9190ad20 con 0x560d918e13b0 2024-03-23T09:48:06.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f573096d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d91902840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f573096d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d91902840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38566/0 (socket says 172.21.15.47:38566) 2024-03-23T09:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f573096d640 1 -- 172.21.15.47:0/1530772516 learned_addr learned my addr 172.21.15.47:0/1530772516 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f573196f640 1 --2- 172.21.15.47:0/1530772516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d9190f4f0 0x560d919413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f573116e640 1 --2- 172.21.15.47:0/1530772516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 0x560d91902300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f573096d640 1 -- 172.21.15.47:0/1530772516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 msgr2=0x560d91902300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f573096d640 1 --2- 172.21.15.47:0/1530772516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 0x560d91902300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f573096d640 1 -- 172.21.15.47:0/1530772516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d9190f4f0 msgr2=0x560d919413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f573096d640 1 --2- 172.21.15.47:0/1530772516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d9190f4f0 0x560d919413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f573096d640 1 -- 172.21.15.47:0/1530772516 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d918799e0 con 0x560d9190d830 2024-03-23T09:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f573196f640 1 --2- 172.21.15.47:0/1530772516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d9190f4f0 0x560d919413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f573116e640 1 --2- 172.21.15.47:0/1530772516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 0x560d91902300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f573096d640 1 --2- 172.21.15.47:0/1530772516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d91902840 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f571c0073b0 tx=0x7f571c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61668a97a0914272 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.185+0000 7f570f7fe640 1 -- 172.21.15.47:0/1530772516 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c0074a0 con 0x560d9190d830 2024-03-23T09:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.186+0000 7f570f7fe640 1 -- 172.21.15.47:0/1530772516 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571c00d070 con 0x560d9190d830 2024-03-23T09:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.186+0000 7f570f7fe640 1 -- 172.21.15.47:0/1530772516 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c001890 con 0x560d9190d830 2024-03-23T09:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.186+0000 7f5736026740 1 -- 172.21.15.47:0/1530772516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 msgr2=0x560d91902840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.186+0000 7f5736026740 1 --2- 172.21.15.47:0/1530772516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d91902840 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f571c0073b0 tx=0x7f571c005bc0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.186+0000 7f5736026740 1 -- 172.21.15.47:0/1530772516 shutdown_connections 2024-03-23T09:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.186+0000 7f5736026740 1 --2- 172.21.15.47:0/1530772516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d9190f4f0 0x560d919413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.186+0000 7f5736026740 1 --2- 172.21.15.47:0/1530772516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d91902840 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.186+0000 7f5736026740 1 --2- 172.21.15.47:0/1530772516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 0x560d91902300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.186+0000 7f5736026740 1 -- 172.21.15.47:0/1530772516 >> 172.21.15.47:0/1530772516 conn(0x560d918dd850 msgr2=0x560d918e1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.186+0000 7f5736026740 1 -- 172.21.15.47:0/1530772516 shutdown_connections 2024-03-23T09:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.186+0000 7f5736026740 1 -- 172.21.15.47:0/1530772516 wait complete. 2024-03-23T09:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.187+0000 7f5736026740 1 -- start start 2024-03-23T09:48:06.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.187+0000 7f5736026740 1 Processor -- start 2024-03-23T09:48:06.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f5736026740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 0x560d919930d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f5736026740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d91978d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f5736026740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d9190f4f0 0x560d91902300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f5736026740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d918e38e0 con 0x560d9190d830 2024-03-23T09:48:06.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f5736026740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d91993970 con 0x560d9190f4f0 2024-03-23T09:48:06.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f5736026740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d918e3690 con 0x560d918e13b0 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f5736026740 10 client.0 _ll_register_callbacks cb 0x7f5728009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f573096d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d91978d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f573096d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d91978d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38578/0 (socket says 172.21.15.47:38578) 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f573096d640 1 -- 172.21.15.47:0/3491485707 learned_addr learned my addr 172.21.15.47:0/3491485707 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f573196f640 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d9190f4f0 0x560d91902300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f573116e640 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 0x560d919930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f573096d640 1 -- 172.21.15.47:0/3491485707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 msgr2=0x560d919930d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f573096d640 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 0x560d919930d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f573096d640 1 -- 172.21.15.47:0/3491485707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d9190f4f0 msgr2=0x560d91902300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f573096d640 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d9190f4f0 0x560d91902300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f573096d640 1 -- 172.21.15.47:0/3491485707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c00f040 con 0x560d9190d830 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f573196f640 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d9190f4f0 0x560d91902300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f573096d640 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d91978d80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f571c019040 tx=0x7f571c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87e1cf4dd488841 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f5707fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f5707fff640 1 -- 172.21.15.47:0/3491485707 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c001d20 con 0x560d9190d830 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f5707fff640 1 -- 172.21.15.47:0/3491485707 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f571c015460 con 0x560d9190d830 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.188+0000 7f5736026740 1 -- 172.21.15.47:0/3491485707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5728049070 con 0x560d9190d830 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5707fff640 1 -- 172.21.15.47:0/3491485707 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f571c015620 con 0x560d9190d830 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5736026740 10 client.7932 fetch_fsmap learned FSMap version 53 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5736026740 1 -- 172.21.15.47:0/3491485707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x560d917e75c0 con 0x560d9190d830 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5707fff640 1 -- 172.21.15.47:0/3491485707 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5728049070 con 0x560d9190d830 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5707fff640 1 -- 172.21.15.47:0/3491485707 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 53) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f571c028060 con 0x560d9190d830 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5736026740 10 client.7932 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5736026740 10 client.7932 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5736026740 20 client.7932 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5736026740 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x560d918e3170 0x560d91902840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5736026740 4 client.7932 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5736026740 1 -- 172.21.15.47:0/3491485707 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560d9153a4e0 con 0x560d918e3170 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f573116e640 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x560d918e3170 0x560d91902840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5707fff640 10 client.7932 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f573116e640 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x560d918e3170 0x560d91902840 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=753108bcf7a6fe16 server_cookie=d5780d75a39a6feb in_seq=0 out_seq=0 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5707fff640 1 -- 172.21.15.47:0/3491485707 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560d9153a4e0 con 0x560d918e3170 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5707fff640 10 client.7932 handle_command_reply: tid=0 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.189+0000 7f5736026740 1 client.7932 shutdown 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f5736026740 1 -- 172.21.15.47:0/3491485707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 msgr2=0x560d91978d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f5736026740 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d91978d80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f571c019040 tx=0x7f571c000ed0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f5736026740 1 -- 172.21.15.47:0/3491485707 shutdown_connections 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f5736026740 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x560d918e3170 0x560d91902840 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f5736026740 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d9190f4f0 0x560d91902300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f5736026740 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d9190d830 0x560d91978d80 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f5736026740 1 --2- 172.21.15.47:0/3491485707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560d918e13b0 0x560d919930d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f5707fff640 0 client.7932 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f5736026740 1 -- 172.21.15.47:0/3491485707 >> 172.21.15.47:0/3491485707 conn(0x560d918dd850 msgr2=0x560d918dc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f5736026740 1 -- 172.21.15.47:0/3491485707 shutdown_connections 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f5736026740 1 -- 172.21.15.47:0/3491485707 wait complete. 2024-03-23T09:48:06.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f5736026740 20 client.7932 trim_cache size 0 max 16384 2024-03-23T09:48:06.197 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:48:06.197 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-23T09:48:06.197 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-23T09:48:06.197 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:48:06.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f57333d2640 1 -- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f571006ac00 msgr2=0x7f571006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f57333d2640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f571006ac00 0x7f571006d0c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5728036000 tx=0x7f572803c040 comp rx=0 tx=0).stop 2024-03-23T09:48:06.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f57333d2640 1 -- 172.21.15.47:0/2816112230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c0b0f00 msgr2=0x7f572c14b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f57333d2640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c0b0f00 0x7f572c14b100 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f572400cea0 tx=0x7f572400cf90 comp rx=0 tx=0).stop 2024-03-23T09:48:06.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f57333d2640 1 -- 172.21.15.47:0/2816112230 shutdown_connections 2024-03-23T09:48:06.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f57333d2640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f571006ac00 0x7f571006d0c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f57333d2640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c0b0f00 0x7f572c14b100 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f57333d2640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f572c0b0a70 0x7f572c14abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f57333d2640 1 --2- 172.21.15.47:0/2816112230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f572c06e080 0x7f572c0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.190+0000 7f57333d2640 1 -- 172.21.15.47:0/2816112230 >> 172.21.15.47:0/2816112230 conn(0x7f572c06a310 msgr2=0x7f572c151bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.191+0000 7f57333d2640 1 -- 172.21.15.47:0/2816112230 shutdown_connections 2024-03-23T09:48:06.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.191+0000 7f57333d2640 1 -- 172.21.15.47:0/2816112230 wait complete. 2024-03-23T09:48:06.201 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T09:48:06.201 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:48:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.302+0000 7f0a8b90a640 1 Processor -- start 2024-03-23T09:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.303+0000 7f0a8b90a640 1 -- start start 2024-03-23T09:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.303+0000 7f0a8b90a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0a8415ca90 0x7f0a84160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.303+0000 7f0a8b90a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a840d11e0 con 0x7f0a840b8350 2024-03-23T09:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.303+0000 7f0a8b90a640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a84160f60 con 0x7f0a840cde10 2024-03-23T09:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.303+0000 7f0a8b90a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a84161240 con 0x7f0a8415ca90 2024-03-23T09:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.303+0000 7f0a88ea5640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0a840b8350 0x7f0a840cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42452/0 (socket says 172.21.15.47:42452) 2024-03-23T09:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.303+0000 7f0a88ea5640 1 -- 172.21.15.47:0/1689099226 learned_addr learned my addr 172.21.15.47:0/1689099226 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320212375 0 0) 0x7f0a840d11e0 con 0x7f0a840b8350 2024-03-23T09:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a896a6640 1 --2- 172.21.15.47:0/1689099226 >> v2:172.21.15.175:3300/0 conn(0x7f0a8415ca90 0x7f0a84160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a70003520 con 0x7f0a840b8350 2024-03-23T09:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 606596329 0 0) 0x7f0a70003520 con 0x7f0a840b8350 2024-03-23T09:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 >> v2:172.21.15.175:3300/0 conn(0x7f0a8415ca90 msgr2=0x7f0a84160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 --2- 172.21.15.47:0/1689099226 >> v2:172.21.15.175:3300/0 conn(0x7f0a8415ca90 0x7f0a84160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 >> v1:172.21.15.175:6789/0 conn(0x7f0a840cde10 legacy=0x7f0a840d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a84161520 con 0x7f0a840b8350 2024-03-23T09:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0a74001670 con 0x7f0a840b8350 2024-03-23T09:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 >> v1:172.21.15.47:6789/0 conn(0x7f0a840b8350 legacy=0x7f0a840cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 --2- 172.21.15.47:0/1689099226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a70003970 0x7f0a70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 --2- 172.21.15.47:0/1689099226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a700047c0 0x7f0a70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 --2- 172.21.15.47:0/1689099226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a70005180 0x7f0a7000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a84161240 con 0x7f0a70005180 2024-03-23T09:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7000c120 con 0x7f0a70003970 2024-03-23T09:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7000c2e0 con 0x7f0a700047c0 2024-03-23T09:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a89ea7640 1 --2- 172.21.15.47:0/1689099226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a70005180 0x7f0a7000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a896a6640 1 --2- 172.21.15.47:0/1689099226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a70003970 0x7f0a70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.304+0000 7f0a88ea5640 1 --2- 172.21.15.47:0/1689099226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a700047c0 0x7f0a70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.305+0000 7f0a89ea7640 1 -- 172.21.15.47:0/1689099226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a700047c0 msgr2=0x7f0a70004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.305+0000 7f0a89ea7640 1 --2- 172.21.15.47:0/1689099226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a700047c0 0x7f0a70004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.305+0000 7f0a89ea7640 1 -- 172.21.15.47:0/1689099226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a70003970 msgr2=0x7f0a70003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.305+0000 7f0a89ea7640 1 --2- 172.21.15.47:0/1689099226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a70003970 0x7f0a70003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.305+0000 7f0a89ea7640 1 -- 172.21.15.47:0/1689099226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a70003520 con 0x7f0a70005180 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.305+0000 7f0a88ea5640 1 --2- 172.21.15.47:0/1689099226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a700047c0 0x7f0a70004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.305+0000 7f0a89ea7640 1 -- 172.21.15.47:0/1689099226 reap_dead start 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.305+0000 7f0a89ea7640 1 --2- 172.21.15.47:0/1689099226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a70005180 0x7f0a7000b980 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c001e20 tx=0x7f0a700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b550a56eb15ba1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.305+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c003b70 con 0x7f0a70005180 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.305+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c003cd0 con 0x7f0a70005180 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.305+0000 7f0a7bfff640 1 -- 172.21.15.47:0/1689099226 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c005310 con 0x7f0a70005180 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.306+0000 7f0a8b90a640 1 -- 172.21.15.47:0/1689099226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a70005180 msgr2=0x7f0a7000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.306+0000 7f0a8b90a640 1 --2- 172.21.15.47:0/1689099226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a70005180 0x7f0a7000b980 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c001e20 tx=0x7f0a700057f0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.306+0000 7f0a8b90a640 1 -- 172.21.15.47:0/1689099226 shutdown_connections 2024-03-23T09:48:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.306+0000 7f0a8b90a640 1 --2- 172.21.15.47:0/1689099226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a70005180 0x7f0a7000b980 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.306+0000 7f0a8b90a640 1 -- 172.21.15.47:0/1689099226 >> 172.21.15.47:0/1689099226 conn(0x7f0a840659d0 msgr2=0x7f0a84065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.306+0000 7f0a8b90a640 1 -- 172.21.15.47:0/1689099226 shutdown_connections 2024-03-23T09:48:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.306+0000 7f0a8b90a640 1 -- 172.21.15.47:0/1689099226 wait complete. 2024-03-23T09:48:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.307+0000 7f0a8b90a640 1 Processor -- start 2024-03-23T09:48:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.307+0000 7f0a8b90a640 1 -- start start 2024-03-23T09:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a8b90a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a70005180 0x7f0a840b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a8b90a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a84148980 0x7f0a8415f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a8b90a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a841f1620 0x7f0a841f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a8b90a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840c9ff0 con 0x7f0a841f1620 2024-03-23T09:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a8b90a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840b43c0 con 0x7f0a70005180 2024-03-23T09:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a8b90a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840b4640 con 0x7f0a84148980 2024-03-23T09:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a89ea7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a841f1620 0x7f0a841f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a89ea7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a841f1620 0x7f0a841f1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38594/0 (socket says 172.21.15.47:38594) 2024-03-23T09:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a89ea7640 1 -- 172.21.15.47:0/3322224666 learned_addr learned my addr 172.21.15.47:0/3322224666 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a896a6640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a70005180 0x7f0a840b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a89ea7640 1 -- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a84148980 msgr2=0x7f0a8415f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a88ea5640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a84148980 0x7f0a8415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a89ea7640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a84148980 0x7f0a8415f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a89ea7640 1 -- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a70005180 msgr2=0x7f0a840b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a89ea7640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a70005180 0x7f0a840b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a89ea7640 1 -- 172.21.15.47:0/3322224666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a840b4920 con 0x7f0a841f1620 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a896a6640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a70005180 0x7f0a840b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a88ea5640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a84148980 0x7f0a8415f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.308+0000 7f0a89ea7640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a841f1620 0x7f0a841f1aa0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c001f40 tx=0x7f0a7c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.309+0000 7f0a79ffb640 1 -- 172.21.15.47:0/3322224666 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c006d50 con 0x7f0a841f1620 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.309+0000 7f0a79ffb640 1 -- 172.21.15.47:0/3322224666 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c007030 con 0x7f0a841f1620 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.309+0000 7f0a8b90a640 1 -- 172.21.15.47:0/3322224666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a840c9c10 con 0x7f0a841f1620 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.309+0000 7f0a79ffb640 1 -- 172.21.15.47:0/3322224666 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c0073c0 con 0x7f0a841f1620 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.309+0000 7f0a8b90a640 1 -- 172.21.15.47:0/3322224666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8415fe80 con 0x7f0a841f1620 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.310+0000 7f0a8b90a640 1 -- 172.21.15.47:0/3322224666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a84072360 con 0x7f0a841f1620 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.310+0000 7f0a79ffb640 1 -- 172.21.15.47:0/3322224666 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0a7c007520 con 0x7f0a841f1620 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.311+0000 7f0a79ffb640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0a6806ac00 0x7f0a6806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.311+0000 7f0a79ffb640 1 -- 172.21.15.47:0/3322224666 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0a7c0aac90 con 0x7f0a841f1620 2024-03-23T09:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.311+0000 7f0a896a6640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0a6806ac00 0x7f0a6806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.311+0000 7f0a896a6640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0a6806ac00 0x7f0a6806d0c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009a40 tx=0x7f0a80008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.315+0000 7f0a79ffb640 1 -- 172.21.15.47:0/3322224666 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0a7c076010 con 0x7f0a841f1620 2024-03-23T09:48:06.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.530+0000 7f0a8b90a640 1 -- 172.21.15.47:0/3322224666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a840b4b40 con 0x7f0a841f1620 2024-03-23T09:48:06.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.531+0000 7f0a79ffb640 1 -- 172.21.15.47:0/3322224666 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f0a7c047090 con 0x7f0a841f1620 2024-03-23T09:48:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:48:06.531 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:47:39.843531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7776":{"gid":7776,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":6,"addr":"172.21.15.47:6835/4278516621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4278516621},{"type":"v1","addr":"172.21.15.47:6835","nonce":4278516621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:48:06.532 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 53 2024-03-23T09:48:06.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.533+0000 7f0a8b90a640 1 -- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0a6806ac00 msgr2=0x7f0a6806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.533+0000 7f0a8b90a640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0a6806ac00 0x7f0a6806d0c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009a40 tx=0x7f0a80008040 comp rx=0 tx=0).stop 2024-03-23T09:48:06.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.533+0000 7f0a8b90a640 1 -- 172.21.15.47:0/3322224666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a841f1620 msgr2=0x7f0a841f1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.533+0000 7f0a8b90a640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a841f1620 0x7f0a841f1aa0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c001f40 tx=0x7f0a7c0014a0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.533+0000 7f0a8b90a640 1 -- 172.21.15.47:0/3322224666 shutdown_connections 2024-03-23T09:48:06.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.533+0000 7f0a8b90a640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0a6806ac00 0x7f0a6806d0c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.533+0000 7f0a8b90a640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a841f1620 0x7f0a841f1aa0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.533+0000 7f0a8b90a640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a84148980 0x7f0a8415f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.533+0000 7f0a8b90a640 1 --2- 172.21.15.47:0/3322224666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a70005180 0x7f0a840b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.533+0000 7f0a8b90a640 1 -- 172.21.15.47:0/3322224666 >> 172.21.15.47:0/3322224666 conn(0x7f0a840659d0 msgr2=0x7f0a84154020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.533+0000 7f0a8b90a640 1 -- 172.21.15.47:0/3322224666 shutdown_connections 2024-03-23T09:48:06.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.533+0000 7f0a8b90a640 1 -- 172.21.15.47:0/3322224666 wait complete. 2024-03-23T09:48:06.544 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-23T09:48:06.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.647+0000 7f535b3ea640 1 Processor -- start 2024-03-23T09:48:06.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.648+0000 7f535b3ea640 1 -- start start 2024-03-23T09:48:06.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.648+0000 7f535b3ea640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f535415c990 0x7f5354160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.648+0000 7f535b3ea640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53540b8350 0x7f53540cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.648+0000 7f535b3ea640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5354161630 con 0x7f53540cdc40 2024-03-23T09:48:06.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.648+0000 7f535b3ea640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53540d1030 con 0x7f53540b8350 2024-03-23T09:48:06.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.648+0000 7f535b3ea640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53540ce070 con 0x7f535415c990 2024-03-23T09:48:06.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.649+0000 7f5358985640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53540b8350 0x7f53540cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.649+0000 7f5358985640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53540b8350 0x7f53540cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38600/0 (socket says 172.21.15.47:38600) 2024-03-23T09:48:06.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.649+0000 7f5358985640 1 -- 172.21.15.47:0/4256952611 learned_addr learned my addr 172.21.15.47:0/4256952611 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:06.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.649+0000 7f5359186640 1 --2- 172.21.15.47:0/4256952611 >> v2:172.21.15.175:3300/0 conn(0x7f535415c990 0x7f5354160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.649+0000 7f5358985640 1 -- 172.21.15.47:0/4256952611 >> v2:172.21.15.175:3300/0 conn(0x7f535415c990 msgr2=0x7f5354160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.649+0000 7f5358985640 1 --2- 172.21.15.47:0/4256952611 >> v2:172.21.15.175:3300/0 conn(0x7f535415c990 0x7f5354160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.649+0000 7f5358985640 1 -- 172.21.15.47:0/4256952611 >> v1:172.21.15.175:6789/0 conn(0x7f53540cdc40 legacy=0x7f53540b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.649+0000 7f5358985640 1 -- 172.21.15.47:0/4256952611 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53540cfe10 con 0x7f53540b8350 2024-03-23T09:48:06.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.650+0000 7f5358985640 1 --2- 172.21.15.47:0/4256952611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53540b8350 0x7f53540cd700 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f53540b8b30 tx=0x7f534402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a637f873305361ee server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.650+0000 7f534bfff640 1 -- 172.21.15.47:0/4256952611 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344031870 con 0x7f53540b8350 2024-03-23T09:48:06.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.650+0000 7f534bfff640 1 -- 172.21.15.47:0/4256952611 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53440319d0 con 0x7f53540b8350 2024-03-23T09:48:06.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.650+0000 7f534bfff640 1 -- 172.21.15.47:0/4256952611 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344031d60 con 0x7f53540b8350 2024-03-23T09:48:06.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.650+0000 7f535b3ea640 1 -- 172.21.15.47:0/4256952611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53540b8350 msgr2=0x7f53540cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.650+0000 7f535b3ea640 1 --2- 172.21.15.47:0/4256952611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53540b8350 0x7f53540cd700 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f53540b8b30 tx=0x7f534402b910 comp rx=0 tx=0).stop 2024-03-23T09:48:06.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.650+0000 7f535b3ea640 1 -- 172.21.15.47:0/4256952611 shutdown_connections 2024-03-23T09:48:06.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.650+0000 7f535b3ea640 1 --2- 172.21.15.47:0/4256952611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53540b8350 0x7f53540cd700 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.650+0000 7f535b3ea640 1 --2- 172.21.15.47:0/4256952611 >> v2:172.21.15.175:3300/0 conn(0x7f535415c990 0x7f5354160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.650+0000 7f535b3ea640 1 -- 172.21.15.47:0/4256952611 >> 172.21.15.47:0/4256952611 conn(0x7f53540659d0 msgr2=0x7f5354065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.650+0000 7f535b3ea640 1 -- 172.21.15.47:0/4256952611 shutdown_connections 2024-03-23T09:48:06.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.650+0000 7f535b3ea640 1 -- 172.21.15.47:0/4256952611 wait complete. 2024-03-23T09:48:06.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.652+0000 7f535b3ea640 1 Processor -- start 2024-03-23T09:48:06.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.652+0000 7f535b3ea640 1 -- start start 2024-03-23T09:48:06.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.652+0000 7f535b3ea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53540b8350 0x7f53541f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.652+0000 7f535b3ea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53540cdc40 0x7f5354159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.652+0000 7f535b3ea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f535415c990 0x7f5354155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.652+0000 7f535b3ea640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5354159c60 con 0x7f53540b8350 2024-03-23T09:48:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.652+0000 7f535b3ea640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535415a6f0 con 0x7f535415c990 2024-03-23T09:48:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.652+0000 7f535b3ea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5354159e50 con 0x7f53540cdc40 2024-03-23T09:48:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.652+0000 7f5359186640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53540b8350 0x7f53541f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.652+0000 7f5359186640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53540b8350 0x7f53541f65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38602/0 (socket says 172.21.15.47:38602) 2024-03-23T09:48:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.652+0000 7f5359186640 1 -- 172.21.15.47:0/1963671277 learned_addr learned my addr 172.21.15.47:0/1963671277 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f5359987640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f535415c990 0x7f5354155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f5358985640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53540cdc40 0x7f5354159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f5359186640 1 -- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53540cdc40 msgr2=0x7f5354159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f5359186640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53540cdc40 0x7f5354159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f5359186640 1 -- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f535415c990 msgr2=0x7f5354155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f5359186640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f535415c990 0x7f5354155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f5359186640 1 -- 172.21.15.47:0/1963671277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53440094d0 con 0x7f53540b8350 2024-03-23T09:48:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f5359987640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f535415c990 0x7f5354155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f5358985640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53540cdc40 0x7f5354159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f5359186640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53540b8350 0x7f53541f65f0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f535000d5f0 tx=0x7f535000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f5349ffb640 1 -- 172.21.15.47:0/1963671277 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53500125f0 con 0x7f53540b8350 2024-03-23T09:48:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f5349ffb640 1 -- 172.21.15.47:0/1963671277 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5350012750 con 0x7f53540b8350 2024-03-23T09:48:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f535b3ea640 1 -- 172.21.15.47:0/1963671277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535415a130 con 0x7f53540b8350 2024-03-23T09:48:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f5349ffb640 1 -- 172.21.15.47:0/1963671277 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350012b00 con 0x7f53540b8350 2024-03-23T09:48:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.653+0000 7f535b3ea640 1 -- 172.21.15.47:0/1963671277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53540c64c0 con 0x7f53540b8350 2024-03-23T09:48:06.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.655+0000 7f5349ffb640 1 -- 172.21.15.47:0/1963671277 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5350012c60 con 0x7f53540b8350 2024-03-23T09:48:06.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.655+0000 7f5349ffb640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f533806ac00 0x7f533806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.655+0000 7f5349ffb640 1 -- 172.21.15.47:0/1963671277 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5350089870 con 0x7f53540b8350 2024-03-23T09:48:06.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.655+0000 7f5358985640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f533806ac00 0x7f533806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.656+0000 7f5358985640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f533806ac00 0x7f533806d0c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5344036000 tx=0x7f534403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.656+0000 7f535e034740 1 Processor -- start 2024-03-23T09:48:06.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.656+0000 7f535e034740 1 -- start start 2024-03-23T09:48:06.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f535e034740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33e26b0 0x5642b33e2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f535e034740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b26d0 0x5642b33df920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f535e034740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33b6230 0x5642b33dfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f535e034740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b33236a0 con 0x5642b33b26d0 2024-03-23T09:48:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f535e034740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b3320c70 con 0x5642b33b6230 2024-03-23T09:48:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f535e034740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642b331c560 con 0x5642b33e26b0 2024-03-23T09:48:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f5358985640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b26d0 0x5642b33df920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f5358985640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b26d0 0x5642b33df920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38604/0 (socket says 172.21.15.47:38604) 2024-03-23T09:48:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f5358985640 1 -- 172.21.15.47:0/324580556 learned_addr learned my addr 172.21.15.47:0/324580556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f5359186640 1 --2- 172.21.15.47:0/324580556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33e26b0 0x5642b33e2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f5358985640 1 -- 172.21.15.47:0/324580556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33e26b0 msgr2=0x5642b33e2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f5359987640 1 --2- 172.21.15.47:0/324580556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33b6230 0x5642b33dfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f5358985640 1 --2- 172.21.15.47:0/324580556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33e26b0 0x5642b33e2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f5358985640 1 -- 172.21.15.47:0/324580556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33b6230 msgr2=0x5642b33dfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f5358985640 1 --2- 172.21.15.47:0/324580556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33b6230 0x5642b33dfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.657+0000 7f5358985640 1 -- 172.21.15.47:0/324580556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642b32e3d50 con 0x5642b33b26d0 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.658+0000 7f5358985640 1 --2- 172.21.15.47:0/324580556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b26d0 0x5642b33df920 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f534402bf20 tx=0x7f5344030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07e9806ec669b51 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.658+0000 7f533effd640 1 -- 172.21.15.47:0/324580556 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534403acb0 con 0x5642b33b26d0 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.658+0000 7f533effd640 1 -- 172.21.15.47:0/324580556 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534404a070 con 0x5642b33b26d0 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.658+0000 7f533effd640 1 -- 172.21.15.47:0/324580556 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534404f630 con 0x5642b33b26d0 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.658+0000 7f535e034740 1 -- 172.21.15.47:0/324580556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b26d0 msgr2=0x5642b33df920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.658+0000 7f535e034740 1 --2- 172.21.15.47:0/324580556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b26d0 0x5642b33df920 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f534402bf20 tx=0x7f5344030000 comp rx=0 tx=0).stop 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.658+0000 7f535e034740 1 -- 172.21.15.47:0/324580556 shutdown_connections 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.658+0000 7f535e034740 1 --2- 172.21.15.47:0/324580556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33b6230 0x5642b33dfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.658+0000 7f535e034740 1 --2- 172.21.15.47:0/324580556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b26d0 0x5642b33df920 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.658+0000 7f535e034740 1 --2- 172.21.15.47:0/324580556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33e26b0 0x5642b33e2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.658+0000 7f535e034740 1 -- 172.21.15.47:0/324580556 >> 172.21.15.47:0/324580556 conn(0x5642b33b4940 msgr2=0x5642b33fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.658+0000 7f535e034740 1 -- 172.21.15.47:0/324580556 shutdown_connections 2024-03-23T09:48:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.658+0000 7f535e034740 1 -- 172.21.15.47:0/324580556 wait complete. 2024-03-23T09:48:06.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.659+0000 7f535e034740 1 -- start start 2024-03-23T09:48:06.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.659+0000 7f535e034740 1 Processor -- start 2024-03-23T09:48:06.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f535e034740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33b6230 0x5642b33e4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f535e034740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e26b0 0x5642b33e9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f535e034740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e4fb0 0x5642b33ea190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f535e034740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b2ff53f0 con 0x5642b33e26b0 2024-03-23T09:48:06.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f535e034740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b30a53d0 con 0x5642b33e4fb0 2024-03-23T09:48:06.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f535e034740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642b329fa10 con 0x5642b33b6230 2024-03-23T09:48:06.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f535e034740 10 client.0 _ll_register_callbacks cb 0x5642b3315d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:06.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f5358985640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e26b0 0x5642b33e9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f5358985640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e26b0 0x5642b33e9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38622/0 (socket says 172.21.15.47:38622) 2024-03-23T09:48:06.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f5358985640 1 -- 172.21.15.47:0/2182162333 learned_addr learned my addr 172.21.15.47:0/2182162333 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:06.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f5359186640 1 --2- 172.21.15.47:0/2182162333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33b6230 0x5642b33e4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f5359987640 1 --2- 172.21.15.47:0/2182162333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e4fb0 0x5642b33ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f5358985640 1 -- 172.21.15.47:0/2182162333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33b6230 msgr2=0x5642b33e4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f5358985640 1 --2- 172.21.15.47:0/2182162333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33b6230 0x5642b33e4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f5358985640 1 -- 172.21.15.47:0/2182162333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e4fb0 msgr2=0x5642b33ea190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f5358985640 1 --2- 172.21.15.47:0/2182162333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e4fb0 0x5642b33ea190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f5358985640 1 -- 172.21.15.47:0/2182162333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53540cfe10 con 0x5642b33e26b0 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f5358985640 1 --2- 172.21.15.47:0/2182162333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e26b0 0x5642b33e9c50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5344009930 tx=0x7f5344004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccb942f1fae9a6d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.660+0000 7f5333fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f5333fff640 1 -- 172.21.15.47:0/2182162333 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344003810 con 0x5642b33e26b0 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f5333fff640 1 -- 172.21.15.47:0/2182162333 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f534402d040 con 0x5642b33e26b0 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f535e034740 1 -- 172.21.15.47:0/2182162333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642b3345150 con 0x5642b33e26b0 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f5333fff640 1 -- 172.21.15.47:0/2182162333 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5344003970 con 0x5642b33e26b0 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f535e034740 10 client.7950 fetch_fsmap learned FSMap version 53 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f535e034740 1 -- 172.21.15.47:0/2182162333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x5642b32de760 con 0x5642b33e26b0 2024-03-23T09:48:06.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f5333fff640 1 -- 172.21.15.47:0/2182162333 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5642b3345150 con 0x5642b33e26b0 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f5333fff640 1 -- 172.21.15.47:0/2182162333 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 53) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5344003ad0 con 0x5642b33e26b0 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f535e034740 10 client.7950 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f535e034740 10 client.7950 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f535e034740 20 client.7950 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f535e034740 1 --2- 172.21.15.47:0/2182162333 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x5642b3411080 0x5642b33e7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f535e034740 4 client.7950 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f535e034740 1 -- 172.21.15.47:0/2182162333 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642b3327650 con 0x5642b3411080 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.661+0000 7f5359186640 1 --2- 172.21.15.47:0/2182162333 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x5642b3411080 0x5642b33e7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f5333fff640 10 client.7950 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f5359186640 1 --2- 172.21.15.47:0/2182162333 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x5642b3411080 0x5642b33e7f90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8418a08ccbaaf5 server_cookie=cfaf96d362f33929 in_seq=0 out_seq=0 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f5333fff640 1 -- 172.21.15.47:0/2182162333 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5642b3327650 con 0x5642b3411080 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f5333fff640 10 client.7950 handle_command_reply: tid=0 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f535e034740 1 client.7950 shutdown 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f535e034740 1 -- 172.21.15.47:0/2182162333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e26b0 msgr2=0x5642b33e9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f535e034740 1 --2- 172.21.15.47:0/2182162333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e26b0 0x5642b33e9c50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5344009930 tx=0x7f5344004730 comp rx=0 tx=0).stop 2024-03-23T09:48:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f535e034740 1 -- 172.21.15.47:0/2182162333 shutdown_connections 2024-03-23T09:48:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f535e034740 1 --2- 172.21.15.47:0/2182162333 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x5642b3411080 0x5642b33e7f90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f535e034740 1 --2- 172.21.15.47:0/2182162333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e4fb0 0x5642b33ea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f535e034740 1 --2- 172.21.15.47:0/2182162333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e26b0 0x5642b33e9c50 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f5333fff640 0 client.7950 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f535e034740 1 --2- 172.21.15.47:0/2182162333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33b6230 0x5642b33e4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f535e034740 1 -- 172.21.15.47:0/2182162333 >> 172.21.15.47:0/2182162333 conn(0x5642b33b26d0 msgr2=0x5642b34009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f535e034740 1 -- 172.21.15.47:0/2182162333 shutdown_connections 2024-03-23T09:48:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f535e034740 1 -- 172.21.15.47:0/2182162333 wait complete. 2024-03-23T09:48:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.662+0000 7f535e034740 20 client.7950 trim_cache size 0 max 16384 2024-03-23T09:48:06.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.674+0000 7f535e034740 1 Processor -- start 2024-03-23T09:48:06.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.674+0000 7f535e034740 1 -- start start 2024-03-23T09:48:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.674+0000 7f535e034740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33b6230 0x5642b33ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.674+0000 7f535e034740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e26b0 0x5642b33ff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.674+0000 7f535e034740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e4fb0 0x5642b340b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.674+0000 7f535e034740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b33e7220 con 0x5642b33e4fb0 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.674+0000 7f535e034740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b32d5c30 con 0x5642b33e26b0 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.674+0000 7f535e034740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642b33de600 con 0x5642b33b6230 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.674+0000 7f5359987640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e4fb0 0x5642b340b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.674+0000 7f5359987640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e4fb0 0x5642b340b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38632/0 (socket says 172.21.15.47:38632) 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.674+0000 7f5359987640 1 -- 172.21.15.47:0/803696952 learned_addr learned my addr 172.21.15.47:0/803696952 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.675+0000 7f5358985640 1 --2- 172.21.15.47:0/803696952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e26b0 0x5642b33ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.675+0000 7f5359186640 1 --2- 172.21.15.47:0/803696952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33b6230 0x5642b33ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.675+0000 7f5359987640 1 -- 172.21.15.47:0/803696952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33b6230 msgr2=0x5642b33ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.675+0000 7f5359987640 1 --2- 172.21.15.47:0/803696952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33b6230 0x5642b33ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.675+0000 7f5359987640 1 -- 172.21.15.47:0/803696952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e26b0 msgr2=0x5642b33ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.675+0000 7f5359987640 1 --2- 172.21.15.47:0/803696952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e26b0 0x5642b33ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.675+0000 7f5359987640 1 -- 172.21.15.47:0/803696952 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642b32bcad0 con 0x5642b33e4fb0 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.675+0000 7f5359186640 1 --2- 172.21.15.47:0/803696952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33b6230 0x5642b33ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.675+0000 7f5359987640 1 --2- 172.21.15.47:0/803696952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e4fb0 0x5642b340b7a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f534c001ba0 tx=0x7f534c002060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b422cc61e0b9707 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.675+0000 7f533f7fe640 1 -- 172.21.15.47:0/803696952 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c003d80 con 0x5642b33e4fb0 2024-03-23T09:48:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.675+0000 7f533f7fe640 1 -- 172.21.15.47:0/803696952 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c00d040 con 0x5642b33e4fb0 2024-03-23T09:48:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.675+0000 7f533f7fe640 1 -- 172.21.15.47:0/803696952 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c011b70 con 0x5642b33e4fb0 2024-03-23T09:48:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.676+0000 7f535e034740 1 -- 172.21.15.47:0/803696952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e4fb0 msgr2=0x5642b340b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.676+0000 7f535e034740 1 --2- 172.21.15.47:0/803696952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e4fb0 0x5642b340b7a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f534c001ba0 tx=0x7f534c002060 comp rx=0 tx=0).stop 2024-03-23T09:48:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.676+0000 7f535e034740 1 -- 172.21.15.47:0/803696952 shutdown_connections 2024-03-23T09:48:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.676+0000 7f535e034740 1 --2- 172.21.15.47:0/803696952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33e4fb0 0x5642b340b7a0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.676+0000 7f535e034740 1 --2- 172.21.15.47:0/803696952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e26b0 0x5642b33ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.676+0000 7f535e034740 1 --2- 172.21.15.47:0/803696952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33b6230 0x5642b33ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.676+0000 7f535e034740 1 -- 172.21.15.47:0/803696952 >> 172.21.15.47:0/803696952 conn(0x5642b33b26d0 msgr2=0x5642b34009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.676+0000 7f535e034740 1 -- 172.21.15.47:0/803696952 shutdown_connections 2024-03-23T09:48:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.676+0000 7f535e034740 1 -- 172.21.15.47:0/803696952 wait complete. 2024-03-23T09:48:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.676+0000 7f535e034740 1 -- start start 2024-03-23T09:48:06.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.676+0000 7f535e034740 1 Processor -- start 2024-03-23T09:48:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.677+0000 7f535e034740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b6230 0x5642b33e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.677+0000 7f535e034740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e26b0 0x5642b340f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.677+0000 7f535e034740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33e4fb0 0x5642b340b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.677+0000 7f535e034740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b33e99f0 con 0x5642b33b6230 2024-03-23T09:48:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.677+0000 7f535e034740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b33e9e20 con 0x5642b33e26b0 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.677+0000 7f535e034740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642b34170b0 con 0x5642b33e4fb0 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.677+0000 7f535e034740 10 client.0 _ll_register_callbacks cb 0x7f534403a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.677+0000 7f5359186640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b6230 0x5642b33e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.677+0000 7f5359186640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b6230 0x5642b33e94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38638/0 (socket says 172.21.15.47:38638) 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.677+0000 7f5359186640 1 -- 172.21.15.47:0/1256491714 learned_addr learned my addr 172.21.15.47:0/1256491714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.677+0000 7f5358985640 1 --2- 172.21.15.47:0/1256491714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e26b0 0x5642b340f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.677+0000 7f5359987640 1 --2- 172.21.15.47:0/1256491714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33e4fb0 0x5642b340b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f5359186640 1 -- 172.21.15.47:0/1256491714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33e4fb0 msgr2=0x5642b340b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f5359186640 1 --2- 172.21.15.47:0/1256491714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33e4fb0 0x5642b340b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f5359186640 1 -- 172.21.15.47:0/1256491714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e26b0 msgr2=0x5642b340f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f5359186640 1 --2- 172.21.15.47:0/1256491714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e26b0 0x5642b340f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f5359186640 1 -- 172.21.15.47:0/1256491714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5354161630 con 0x5642b33b6230 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f5359186640 1 --2- 172.21.15.47:0/1256491714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b6230 0x5642b33e94b0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f535000dfb0 tx=0x7f5350012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2214218426f8e269 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f5333fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f5333fff640 1 -- 172.21.15.47:0/1256491714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350089040 con 0x5642b33b6230 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f5333fff640 1 -- 172.21.15.47:0/1256491714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5350054e20 con 0x5642b33b6230 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f535e034740 1 -- 172.21.15.47:0/1256491714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f534404a070 con 0x5642b33b6230 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f5333fff640 1 -- 172.21.15.47:0/1256491714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5350054fe0 con 0x5642b33b6230 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f535e034740 10 client.7956 fetch_fsmap learned FSMap version 53 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f535e034740 1 -- 172.21.15.47:0/1256491714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x5642b32e25b0 con 0x5642b33b6230 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f5333fff640 1 -- 172.21.15.47:0/1256491714 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535007b080 con 0x5642b33b6230 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f5333fff640 1 -- 172.21.15.47:0/1256491714 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 53) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5350057c00 con 0x5642b33b6230 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f535e034740 10 client.7956 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f535e034740 10 client.7956 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f535e034740 20 client.7956 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f535e034740 1 --2- 172.21.15.47:0/1256491714 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x5642b340bce0 0x5642b340c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f535e034740 4 client.7956 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f535e034740 1 -- 172.21.15.47:0/1256491714 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5642b2fcd8d0 con 0x5642b340bce0 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.678+0000 7f5358985640 1 --2- 172.21.15.47:0/1256491714 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x5642b340bce0 0x5642b340c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f5333fff640 10 client.7956 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f5358985640 1 --2- 172.21.15.47:0/1256491714 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x5642b340bce0 0x5642b340c0c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16394995bd375fcf server_cookie=26e18a24a6e3473b in_seq=0 out_seq=0 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f5333fff640 1 -- 172.21.15.47:0/1256491714 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+419 (crc 0 0 0) 0x5642b2fcd8d0 con 0x5642b340bce0 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f5333fff640 10 client.7956 handle_command_reply: tid=0 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f535e034740 1 client.7956 shutdown 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f535e034740 1 -- 172.21.15.47:0/1256491714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b6230 msgr2=0x5642b33e94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f535e034740 1 --2- 172.21.15.47:0/1256491714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b6230 0x5642b33e94b0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f535000dfb0 tx=0x7f5350012f90 comp rx=0 tx=0).stop 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f535e034740 1 -- 172.21.15.47:0/1256491714 shutdown_connections 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f535e034740 1 --2- 172.21.15.47:0/1256491714 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x5642b340bce0 0x5642b340c0c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f535e034740 1 --2- 172.21.15.47:0/1256491714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642b33e4fb0 0x5642b340b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f535e034740 1 --2- 172.21.15.47:0/1256491714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642b33e26b0 0x5642b340f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f5333fff640 0 client.7956 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f535e034740 1 --2- 172.21.15.47:0/1256491714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b33b6230 0x5642b33e94b0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f535e034740 1 -- 172.21.15.47:0/1256491714 >> 172.21.15.47:0/1256491714 conn(0x5642b33b26d0 msgr2=0x5642b33b8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f535e034740 1 -- 172.21.15.47:0/1256491714 shutdown_connections 2024-03-23T09:48:06.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f535e034740 1 -- 172.21.15.47:0/1256491714 wait complete. 2024-03-23T09:48:06.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.679+0000 7f535e034740 20 client.7956 trim_cache size 0 max 16384 2024-03-23T09:48:06.686 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:48:06.686 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-23T09:48:06.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:48:06.686 INFO:teuthology.orchestra.run.smithi047.stdout: "damage_type": "dir_frag", 2024-03-23T09:48:06.686 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 129085381, 2024-03-23T09:48:06.686 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 1099511627776, 2024-03-23T09:48:06.686 INFO:teuthology.orchestra.run.smithi047.stdout: "frag": "*", 2024-03-23T09:48:06.686 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/dir" 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout: "damage_type": "backtrace", 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 1352690682, 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 1099511627777, 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/testdir/hardlink" 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout: "damage_type": "backtrace", 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 3186197532, 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 1099511627776, 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/dir" 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:48:06.687 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-23T09:48:06.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.680+0000 7f535b3ea640 1 -- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f533806ac00 msgr2=0x7f533806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.680+0000 7f535b3ea640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f533806ac00 0x7f533806d0c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5344036000 tx=0x7f534403c040 comp rx=0 tx=0).stop 2024-03-23T09:48:06.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.680+0000 7f535b3ea640 1 -- 172.21.15.47:0/1963671277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53540b8350 msgr2=0x7f53541f65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.680+0000 7f535b3ea640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53540b8350 0x7f53541f65f0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f535000d5f0 tx=0x7f535000dab0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.680+0000 7f535b3ea640 1 -- 172.21.15.47:0/1963671277 shutdown_connections 2024-03-23T09:48:06.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.680+0000 7f535b3ea640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f533806ac00 0x7f533806d0c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.680+0000 7f535b3ea640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f535415c990 0x7f5354155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.680+0000 7f535b3ea640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53540cdc40 0x7f5354159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.680+0000 7f535b3ea640 1 --2- 172.21.15.47:0/1963671277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53540b8350 0x7f53541f65f0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.680+0000 7f535b3ea640 1 -- 172.21.15.47:0/1963671277 >> 172.21.15.47:0/1963671277 conn(0x7f53540659d0 msgr2=0x7f53540b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.680+0000 7f535b3ea640 1 -- 172.21.15.47:0/1963671277 shutdown_connections 2024-03-23T09:48:06.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.680+0000 7f535b3ea640 1 -- 172.21.15.47:0/1963671277 wait complete. 2024-03-23T09:48:06.690 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-23T09:48:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.790+0000 7f919dcea640 1 Processor -- start 2024-03-23T09:48:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.790+0000 7f919dcea640 1 -- start start 2024-03-23T09:48:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.791+0000 7f919dcea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91980cdeb0 0x7f91980d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.791+0000 7f919dcea640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919815e870 con 0x7f91980b0cb0 2024-03-23T09:48:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.791+0000 7f919dcea640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91980b1140 con 0x7f91980ce290 2024-03-23T09:48:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.791+0000 7f919dcea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91980b0510 con 0x7f91980cdeb0 2024-03-23T09:48:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.791+0000 7f9197fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f91980b0cb0 0x7f919815e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42460/0 (socket says 172.21.15.47:42460) 2024-03-23T09:48:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.791+0000 7f9197fff640 1 -- 172.21.15.47:0/2018352083 learned_addr learned my addr 172.21.15.47:0/2018352083 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.791+0000 7f91977fe640 1 --2- 172.21.15.47:0/2018352083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91980cdeb0 0x7f91980d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.792+0000 7f91967fc640 1 -- 172.21.15.47:0/2018352083 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475172522 0 0) 0x7f919815e870 con 0x7f91980b0cb0 2024-03-23T09:48:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.792+0000 7f91967fc640 1 -- 172.21.15.47:0/2018352083 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9180003520 con 0x7f91980b0cb0 2024-03-23T09:48:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.792+0000 7f91967fc640 1 -- 172.21.15.47:0/2018352083 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3444560458 0 0) 0x7f9180003520 con 0x7f91980b0cb0 2024-03-23T09:48:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.792+0000 7f91977fe640 1 -- 172.21.15.47:0/2018352083 >> v1:172.21.15.175:6789/0 conn(0x7f91980ce290 legacy=0x7f919815da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.792+0000 7f91977fe640 1 -- 172.21.15.47:0/2018352083 >> v1:172.21.15.47:6789/0 conn(0x7f91980b0cb0 legacy=0x7f919815e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.792+0000 7f91977fe640 1 -- 172.21.15.47:0/2018352083 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91980aff50 con 0x7f91980cdeb0 2024-03-23T09:48:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.793+0000 7f91977fe640 1 --2- 172.21.15.47:0/2018352083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91980cdeb0 0x7f91980d01a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f91880097d0 tx=0x7f918802b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6866a2f0b348d98 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.793+0000 7f91967fc640 1 -- 172.21.15.47:0/2018352083 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91900019f0 con 0x7f91980cdeb0 2024-03-23T09:48:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.793+0000 7f91967fc640 1 -- 172.21.15.47:0/2018352083 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918802e050 con 0x7f91980cdeb0 2024-03-23T09:48:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.793+0000 7f91967fc640 1 -- 172.21.15.47:0/2018352083 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188030710 con 0x7f91980cdeb0 2024-03-23T09:48:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.793+0000 7f919dcea640 1 -- 172.21.15.47:0/2018352083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91980cdeb0 msgr2=0x7f91980d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.793+0000 7f919dcea640 1 --2- 172.21.15.47:0/2018352083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91980cdeb0 0x7f91980d01a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f91880097d0 tx=0x7f918802b660 comp rx=0 tx=0).stop 2024-03-23T09:48:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.793+0000 7f919dcea640 1 -- 172.21.15.47:0/2018352083 shutdown_connections 2024-03-23T09:48:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.793+0000 7f919dcea640 1 --2- 172.21.15.47:0/2018352083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91980cdeb0 0x7f91980d01a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.793+0000 7f919dcea640 1 -- 172.21.15.47:0/2018352083 >> 172.21.15.47:0/2018352083 conn(0x7f9198098a00 msgr2=0x7f91980cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.793+0000 7f919dcea640 1 -- 172.21.15.47:0/2018352083 shutdown_connections 2024-03-23T09:48:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.793+0000 7f919dcea640 1 -- 172.21.15.47:0/2018352083 wait complete. 2024-03-23T09:48:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.794+0000 7f919dcea640 1 Processor -- start 2024-03-23T09:48:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.795+0000 7f919dcea640 1 -- start start 2024-03-23T09:48:06.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.795+0000 7f919dcea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91980b0cb0 0x7f919815a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.795+0000 7f919dcea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91980cdeb0 0x7f91980ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.795+0000 7f919dcea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91980ce290 0x7f91980ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.795+0000 7f919dcea640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91980c69f0 con 0x7f91980ce290 2024-03-23T09:48:06.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.795+0000 7f919dcea640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919815ac90 con 0x7f91980cdeb0 2024-03-23T09:48:06.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.795+0000 7f919dcea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91981f5a10 con 0x7f91980b0cb0 2024-03-23T09:48:06.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.795+0000 7f9197fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91980ce290 0x7f91980ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f9197fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91980ce290 0x7f91980ad830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38646/0 (socket says 172.21.15.47:38646) 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f9197fff640 1 -- 172.21.15.47:0/3627087417 learned_addr learned my addr 172.21.15.47:0/3627087417 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f9196ffd640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91980cdeb0 0x7f91980ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f91977fe640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91980b0cb0 0x7f919815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f9197fff640 1 -- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91980b0cb0 msgr2=0x7f919815a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f9197fff640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91980b0cb0 0x7f919815a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f9197fff640 1 -- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91980cdeb0 msgr2=0x7f91980ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f9197fff640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91980cdeb0 0x7f91980ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f9197fff640 1 -- 172.21.15.47:0/3627087417 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9188009400 con 0x7f91980ce290 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f9196ffd640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91980cdeb0 0x7f91980ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f9197fff640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91980ce290 0x7f91980ad830 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f919000c8c0 tx=0x7f919000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f917ffff640 1 -- 172.21.15.47:0/3627087417 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919000a640 con 0x7f91980ce290 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f917ffff640 1 -- 172.21.15.47:0/3627087417 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919000a7c0 con 0x7f91980ce290 2024-03-23T09:48:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f919dcea640 1 -- 172.21.15.47:0/3627087417 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91981f5cf0 con 0x7f91980ce290 2024-03-23T09:48:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f917ffff640 1 -- 172.21.15.47:0/3627087417 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919000ab70 con 0x7f91980ce290 2024-03-23T09:48:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.796+0000 7f919dcea640 1 -- 172.21.15.47:0/3627087417 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91981f62b0 con 0x7f91980ce290 2024-03-23T09:48:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.797+0000 7f919dcea640 1 -- 172.21.15.47:0/3627087417 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91980c6bc0 con 0x7f91980ce290 2024-03-23T09:48:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.798+0000 7f917ffff640 1 -- 172.21.15.47:0/3627087417 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f919000acd0 con 0x7f91980ce290 2024-03-23T09:48:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.798+0000 7f917ffff640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f918406ac00 0x7f918406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.798+0000 7f917ffff640 1 -- 172.21.15.47:0/3627087417 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9190088dd0 con 0x7f91980ce290 2024-03-23T09:48:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.798+0000 7f91977fe640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f918406ac00 0x7f918406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.799+0000 7f91977fe640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f918406ac00 0x7f918406d0c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9188034000 tx=0x7f9188008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:06.802+0000 7f917ffff640 1 -- 172.21.15.47:0/3627087417 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f91980c6bc0 con 0x7f91980ce290 2024-03-23T09:48:07.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.019+0000 7f919dcea640 1 -- 172.21.15.47:0/3627087417 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f91980672f0 con 0x7f91980ce290 2024-03-23T09:48:07.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.019+0000 7f917ffff640 1 -- 172.21.15.47:0/3627087417 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9190018080 con 0x7f91980ce290 2024-03-23T09:48:07.020 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:48:07.020 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-23T09:48:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.021+0000 7f919dcea640 1 -- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f918406ac00 msgr2=0x7f918406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.021+0000 7f919dcea640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f918406ac00 0x7f918406d0c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9188034000 tx=0x7f9188008030 comp rx=0 tx=0).stop 2024-03-23T09:48:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.021+0000 7f919dcea640 1 -- 172.21.15.47:0/3627087417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91980ce290 msgr2=0x7f91980ad830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.021+0000 7f919dcea640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91980ce290 0x7f91980ad830 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f919000c8c0 tx=0x7f919000cd80 comp rx=0 tx=0).stop 2024-03-23T09:48:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.021+0000 7f919dcea640 1 -- 172.21.15.47:0/3627087417 shutdown_connections 2024-03-23T09:48:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.021+0000 7f919dcea640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f918406ac00 0x7f918406d0c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.021+0000 7f919dcea640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91980ce290 0x7f91980ad830 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.021+0000 7f919dcea640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91980cdeb0 0x7f91980ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.021+0000 7f919dcea640 1 --2- 172.21.15.47:0/3627087417 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91980b0cb0 0x7f919815a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.021+0000 7f919dcea640 1 -- 172.21.15.47:0/3627087417 >> 172.21.15.47:0/3627087417 conn(0x7f9198098a00 msgr2=0x7f91980cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.021+0000 7f919dcea640 1 -- 172.21.15.47:0/3627087417 shutdown_connections 2024-03-23T09:48:07.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.021+0000 7f919dcea640 1 -- 172.21.15.47:0/3627087417 wait complete. 2024-03-23T09:48:07.032 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-23T09:48:07.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:07.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-03-23T09:48:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.136+0000 7f963d1c4640 1 Processor -- start 2024-03-23T09:48:07.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.137+0000 7f963d1c4640 1 -- start start 2024-03-23T09:48:07.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.137+0000 7f963d1c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f963815a700 0x7f96380cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.137+0000 7f963d1c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96380d2f00 0x7f963815d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.137+0000 7f963d1c4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963815dbd0 con 0x7f963815d0c0 2024-03-23T09:48:07.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.137+0000 7f963d1c4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963815aae0 con 0x7f963815a700 2024-03-23T09:48:07.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.137+0000 7f963d1c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963815deb0 con 0x7f96380d2f00 2024-03-23T09:48:07.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.137+0000 7f9636d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f963815a700 0x7f96380cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.137+0000 7f963659c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f963815d0c0 0x7f96380d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42462/0 (socket says 172.21.15.47:42462) 2024-03-23T09:48:07.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.137+0000 7f963659c640 1 -- 172.21.15.47:0/3149031306 learned_addr learned my addr 172.21.15.47:0/3149031306 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:07.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.138+0000 7f963759e640 1 --2- 172.21.15.47:0/3149031306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96380d2f00 0x7f963815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.138+0000 7f9636d9d640 1 -- 172.21.15.47:0/3149031306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96380d2f00 msgr2=0x7f963815d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.138+0000 7f9636d9d640 1 --2- 172.21.15.47:0/3149031306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96380d2f00 0x7f963815d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.138+0000 7f9636d9d640 1 -- 172.21.15.47:0/3149031306 >> v1:172.21.15.47:6789/0 conn(0x7f963815d0c0 legacy=0x7f96380d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.138+0000 7f9636d9d640 1 -- 172.21.15.47:0/3149031306 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96380cf340 con 0x7f963815a700 2024-03-23T09:48:07.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.138+0000 7f9635d9b640 1 -- 172.21.15.47:0/3149031306 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362812241 0 0) 0x7f963815dbd0 con 0x7f963815d0c0 2024-03-23T09:48:07.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.138+0000 7f9636d9d640 1 --2- 172.21.15.47:0/3149031306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963815a700 0x7f96380cfb40 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f96300097a0 tx=0x7f963002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4011976385903de server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.139+0000 7f9635d9b640 1 -- 172.21.15.47:0/3149031306 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630031830 con 0x7f963815a700 2024-03-23T09:48:07.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.139+0000 7f9635d9b640 1 -- 172.21.15.47:0/3149031306 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9630031990 con 0x7f963815a700 2024-03-23T09:48:07.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.139+0000 7f9635d9b640 1 -- 172.21.15.47:0/3149031306 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630031d20 con 0x7f963815a700 2024-03-23T09:48:07.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.139+0000 7f963d1c4640 1 -- 172.21.15.47:0/3149031306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963815a700 msgr2=0x7f96380cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.139+0000 7f963d1c4640 1 --2- 172.21.15.47:0/3149031306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963815a700 0x7f96380cfb40 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f96300097a0 tx=0x7f963002b8d0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.139+0000 7f963d1c4640 1 -- 172.21.15.47:0/3149031306 shutdown_connections 2024-03-23T09:48:07.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.139+0000 7f963d1c4640 1 --2- 172.21.15.47:0/3149031306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96380d2f00 0x7f963815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.139+0000 7f963d1c4640 1 --2- 172.21.15.47:0/3149031306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963815a700 0x7f96380cfb40 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.139+0000 7f963d1c4640 1 -- 172.21.15.47:0/3149031306 >> 172.21.15.47:0/3149031306 conn(0x7f963806afd0 msgr2=0x7f963806b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:07.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.139+0000 7f963d1c4640 1 -- 172.21.15.47:0/3149031306 shutdown_connections 2024-03-23T09:48:07.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.139+0000 7f963d1c4640 1 -- 172.21.15.47:0/3149031306 wait complete. 2024-03-23T09:48:07.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.141+0000 7f963d1c4640 1 Processor -- start 2024-03-23T09:48:07.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.141+0000 7f963d1c4640 1 -- start start 2024-03-23T09:48:07.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.141+0000 7f963d1c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96380d2f00 0x7f963815e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.141+0000 7f963d1c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f963815a700 0x7f96380c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.141+0000 7f963d1c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963815d0c0 0x7f9638158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.141+0000 7f963d1c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96380c8210 con 0x7f963815d0c0 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.141+0000 7f963d1c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963815e790 con 0x7f96380d2f00 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.141+0000 7f963d1c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963815ea70 con 0x7f963815a700 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f963759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963815d0c0 0x7f9638158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f963759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963815d0c0 0x7f9638158300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38668/0 (socket says 172.21.15.47:38668) 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f963759e640 1 -- 172.21.15.47:0/3072075740 learned_addr learned my addr 172.21.15.47:0/3072075740 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f9636d9d640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96380d2f00 0x7f963815e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f963659c640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f963815a700 0x7f96380c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f963759e640 1 -- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f963815a700 msgr2=0x7f96380c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f963759e640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f963815a700 0x7f96380c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f963759e640 1 -- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96380d2f00 msgr2=0x7f963815e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f963759e640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96380d2f00 0x7f963815e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f963759e640 1 -- 172.21.15.47:0/3072075740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9630009400 con 0x7f963815d0c0 2024-03-23T09:48:07.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f963759e640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963815d0c0 0x7f9638158300 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f9628007e40 tx=0x7f962800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f962f7fe640 1 -- 172.21.15.47:0/3072075740 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628014620 con 0x7f963815d0c0 2024-03-23T09:48:07.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f962f7fe640 1 -- 172.21.15.47:0/3072075740 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962800ee90 con 0x7f963815d0c0 2024-03-23T09:48:07.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f963d1c4640 1 -- 172.21.15.47:0/3072075740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963815ed50 con 0x7f963815d0c0 2024-03-23T09:48:07.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f962f7fe640 1 -- 172.21.15.47:0/3072075740 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96280149e0 con 0x7f963815d0c0 2024-03-23T09:48:07.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.142+0000 7f963d1c4640 1 -- 172.21.15.47:0/3072075740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9638156670 con 0x7f963815d0c0 2024-03-23T09:48:07.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.144+0000 7f962f7fe640 1 -- 172.21.15.47:0/3072075740 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9628014b40 con 0x7f963815d0c0 2024-03-23T09:48:07.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.144+0000 7f962f7fe640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f961806ac00 0x7f961806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.144+0000 7f962f7fe640 1 -- 172.21.15.47:0/3072075740 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9628089c10 con 0x7f963815d0c0 2024-03-23T09:48:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.145+0000 7f9636d9d640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f961806ac00 0x7f961806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.145+0000 7f9636d9d640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f961806ac00 0x7f961806d0c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9630036000 tx=0x7f963003e040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.145+0000 7f963fe15740 1 Processor -- start 2024-03-23T09:48:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.145+0000 7f963fe15740 1 -- start start 2024-03-23T09:48:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.146+0000 7f963fe15740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7a52f0 0x55d81e7a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.146+0000 7f963fe15740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7a9340 0x55d81e7aade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.146+0000 7f963fe15740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7d5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.146+0000 7f963fe15740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81e716940 con 0x55d81e7a52f0 2024-03-23T09:48:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.146+0000 7f963fe15740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81e7140d0 con 0x55d81e7a9340 2024-03-23T09:48:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.146+0000 7f963fe15740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d81e70f9c0 con 0x55d81e7d5080 2024-03-23T09:48:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.146+0000 7f9636d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7a52f0 0x55d81e7a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.146+0000 7f9636d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7a52f0 0x55d81e7a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38682/0 (socket says 172.21.15.47:38682) 2024-03-23T09:48:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.146+0000 7f9636d9d640 1 -- 172.21.15.47:0/3248234257 learned_addr learned my addr 172.21.15.47:0/3248234257 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.146+0000 7f963659c640 1 --2- 172.21.15.47:0/3248234257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7a9340 0x55d81e7aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.146+0000 7f963759e640 1 --2- 172.21.15.47:0/3248234257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7d5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.146+0000 7f9636d9d640 1 -- 172.21.15.47:0/3248234257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 msgr2=0x55d81e7d5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.146+0000 7f9636d9d640 1 --2- 172.21.15.47:0/3248234257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7d5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.147+0000 7f9636d9d640 1 -- 172.21.15.47:0/3248234257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7a9340 msgr2=0x55d81e7aade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.147+0000 7f9636d9d640 1 --2- 172.21.15.47:0/3248234257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7a9340 0x55d81e7aade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.147+0000 7f9636d9d640 1 -- 172.21.15.47:0/3248234257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d81e6db8a0 con 0x55d81e7a52f0 2024-03-23T09:48:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.147+0000 7f963759e640 1 --2- 172.21.15.47:0/3248234257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7d5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.147+0000 7f9636d9d640 1 --2- 172.21.15.47:0/3248234257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7a52f0 0x55d81e7a56d0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f963002bee0 tx=0x7f9630030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fd9b0cd5a8f09e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.147+0000 7f962cff9640 1 -- 172.21.15.47:0/3248234257 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963004e070 con 0x55d81e7a52f0 2024-03-23T09:48:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.147+0000 7f962cff9640 1 -- 172.21.15.47:0/3248234257 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9630031d10 con 0x55d81e7a52f0 2024-03-23T09:48:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.147+0000 7f962cff9640 1 -- 172.21.15.47:0/3248234257 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630041040 con 0x55d81e7a52f0 2024-03-23T09:48:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.147+0000 7f963fe15740 1 -- 172.21.15.47:0/3248234257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7a52f0 msgr2=0x55d81e7a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.147+0000 7f963fe15740 1 --2- 172.21.15.47:0/3248234257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7a52f0 0x55d81e7a56d0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f963002bee0 tx=0x7f9630030000 comp rx=0 tx=0).stop 2024-03-23T09:48:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.148+0000 7f963fe15740 1 -- 172.21.15.47:0/3248234257 shutdown_connections 2024-03-23T09:48:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.148+0000 7f963fe15740 1 --2- 172.21.15.47:0/3248234257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7d5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.148+0000 7f963fe15740 1 --2- 172.21.15.47:0/3248234257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7a9340 0x55d81e7aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.148+0000 7f963fe15740 1 --2- 172.21.15.47:0/3248234257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7a52f0 0x55d81e7a56d0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.148+0000 7f963fe15740 1 -- 172.21.15.47:0/3248234257 >> 172.21.15.47:0/3248234257 conn(0x55d81e7a7370 msgr2=0x55d81e7a8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.148+0000 7f963fe15740 1 -- 172.21.15.47:0/3248234257 shutdown_connections 2024-03-23T09:48:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.148+0000 7f963fe15740 1 -- 172.21.15.47:0/3248234257 wait complete. 2024-03-23T09:48:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.148+0000 7f963fe15740 1 -- start start 2024-03-23T09:48:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.148+0000 7f963fe15740 1 Processor -- start 2024-03-23T09:48:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963fe15740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963fe15740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 0x55d81e7d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963fe15740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 0x55d81e7f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963fe15740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81e6b0a90 con 0x55d81e7d18f0 2024-03-23T09:48:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963fe15740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81e6b0730 con 0x55d81e7f35f0 2024-03-23T09:48:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963fe15740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d81e692e00 con 0x55d81e7d5080 2024-03-23T09:48:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963fe15740 10 client.0 _ll_register_callbacks cb 0x55d81e6c8b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 0x55d81e7d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 0x55d81e7d1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38698/0 (socket says 172.21.15.47:38698) 2024-03-23T09:48:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963659c640 1 -- 172.21.15.47:0/2653740491 learned_addr learned my addr 172.21.15.47:0/2653740491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963759e640 1 --2- 172.21.15.47:0/2653740491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 0x55d81e7f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f9636d9d640 1 --2- 172.21.15.47:0/2653740491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963659c640 1 -- 172.21.15.47:0/2653740491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 msgr2=0x55d81e7c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963659c640 1 --2- 172.21.15.47:0/2653740491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7c4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963659c640 1 -- 172.21.15.47:0/2653740491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 msgr2=0x55d81e7f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963659c640 1 --2- 172.21.15.47:0/2653740491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 0x55d81e7f3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.149+0000 7f963659c640 1 -- 172.21.15.47:0/2653740491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96380cf340 con 0x55d81e7d18f0 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f963659c640 1 --2- 172.21.15.47:0/2653740491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 0x55d81e7d1cd0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f9620004e00 tx=0x7f96200015e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6c0590b41539e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f9615ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f9615ffb640 1 -- 172.21.15.47:0/2653740491 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620015650 con 0x55d81e7d18f0 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f9615ffb640 1 -- 172.21.15.47:0/2653740491 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f96200198b0 con 0x55d81e7d18f0 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f963fe15740 1 -- 172.21.15.47:0/2653740491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d81e403560 con 0x55d81e7d18f0 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f9615ffb640 1 -- 172.21.15.47:0/2653740491 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9620019a70 con 0x55d81e7d18f0 2024-03-23T09:48:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f963fe15740 10 client.7977 fetch_fsmap learned FSMap version 53 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f963fe15740 1 -- 172.21.15.47:0/2653740491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55d81e70e040 con 0x55d81e7d18f0 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f9615ffb640 1 -- 172.21.15.47:0/2653740491 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d81e403560 con 0x55d81e7d18f0 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f9615ffb640 1 -- 172.21.15.47:0/2653740491 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 53) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9620012050 con 0x55d81e7d18f0 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f963fe15740 10 client.7977 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f963fe15740 10 client.7977 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f963fe15740 20 client.7977 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f963fe15740 1 --2- 172.21.15.47:0/2653740491 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55d81e7d7a40 0x55d81e7d7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f963fe15740 4 client.7977 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f963fe15740 1 -- 172.21.15.47:0/2653740491 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d81e414860 con 0x55d81e7d7a40 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.150+0000 7f9636d9d640 1 --2- 172.21.15.47:0/2653740491 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55d81e7d7a40 0x55d81e7d7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f9615ffb640 10 client.7977 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f9636d9d640 1 --2- 172.21.15.47:0/2653740491 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55d81e7d7a40 0x55d81e7d7e20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e24ab477a55044 server_cookie=2c2ebe5930eb4158 in_seq=0 out_seq=0 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f9615ffb640 1 -- 172.21.15.47:0/2653740491 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d81e414860 con 0x55d81e7d7a40 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f9615ffb640 10 client.7977 handle_command_reply: tid=0 2024-03-23T09:48:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f963fe15740 1 client.7977 shutdown 2024-03-23T09:48:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f963fe15740 1 -- 172.21.15.47:0/2653740491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 msgr2=0x55d81e7d1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f963fe15740 1 --2- 172.21.15.47:0/2653740491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 0x55d81e7d1cd0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f9620004e00 tx=0x7f96200015e0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f963fe15740 1 -- 172.21.15.47:0/2653740491 shutdown_connections 2024-03-23T09:48:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f963fe15740 1 --2- 172.21.15.47:0/2653740491 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55d81e7d7a40 0x55d81e7d7e20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f963fe15740 1 --2- 172.21.15.47:0/2653740491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 0x55d81e7f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f9615ffb640 0 client.7977 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f963fe15740 1 --2- 172.21.15.47:0/2653740491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 0x55d81e7d1cd0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f963fe15740 1 --2- 172.21.15.47:0/2653740491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f963fe15740 1 -- 172.21.15.47:0/2653740491 >> 172.21.15.47:0/2653740491 conn(0x55d81e7a52f0 msgr2=0x55d81e7ab100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f963fe15740 1 -- 172.21.15.47:0/2653740491 shutdown_connections 2024-03-23T09:48:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.151+0000 7f963fe15740 1 -- 172.21.15.47:0/2653740491 wait complete. 2024-03-23T09:48:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.152+0000 7f963fe15740 20 client.7977 trim_cache size 0 max 16384 2024-03-23T09:48:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.166+0000 7f963fe15740 1 Processor -- start 2024-03-23T09:48:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.166+0000 7f963fe15740 1 -- start start 2024-03-23T09:48:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.166+0000 7f963fe15740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 0x55d81e7d2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.166+0000 7f963fe15740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7aa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.166+0000 7f963fe15740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 0x55d81e7d7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.166+0000 7f963fe15740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81e7ceff0 con 0x55d81e7d18f0 2024-03-23T09:48:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.166+0000 7f963fe15740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81e7a8f60 con 0x55d81e7f35f0 2024-03-23T09:48:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.166+0000 7f963fe15740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d81e7a5130 con 0x55d81e7d5080 2024-03-23T09:48:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f9636d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 0x55d81e7d2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f9636d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 0x55d81e7d2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38704/0 (socket says 172.21.15.47:38704) 2024-03-23T09:48:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f9636d9d640 1 -- 172.21.15.47:0/3033110487 learned_addr learned my addr 172.21.15.47:0/3033110487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f963759e640 1 --2- 172.21.15.47:0/3033110487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 0x55d81e7d7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f963659c640 1 --2- 172.21.15.47:0/3033110487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f9636d9d640 1 -- 172.21.15.47:0/3033110487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 msgr2=0x55d81e7aa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f9636d9d640 1 --2- 172.21.15.47:0/3033110487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7aa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f9636d9d640 1 -- 172.21.15.47:0/3033110487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 msgr2=0x55d81e7d7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f9636d9d640 1 --2- 172.21.15.47:0/3033110487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 0x55d81e7d7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f9636d9d640 1 -- 172.21.15.47:0/3033110487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d81e6afe20 con 0x55d81e7d18f0 2024-03-23T09:48:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f963759e640 1 --2- 172.21.15.47:0/3033110487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 0x55d81e7d7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f963659c640 1 --2- 172.21.15.47:0/3033110487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7aa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f9636d9d640 1 --2- 172.21.15.47:0/3033110487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 0x55d81e7d2240 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f963003d000 tx=0x7f9630009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349eaa7b671b3379 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f962d7fa640 1 -- 172.21.15.47:0/3033110487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630041030 con 0x55d81e7d18f0 2024-03-23T09:48:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f962d7fa640 1 -- 172.21.15.47:0/3033110487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963004b040 con 0x55d81e7d18f0 2024-03-23T09:48:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.167+0000 7f962d7fa640 1 -- 172.21.15.47:0/3033110487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963004e040 con 0x55d81e7d18f0 2024-03-23T09:48:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.168+0000 7f963fe15740 1 -- 172.21.15.47:0/3033110487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 msgr2=0x55d81e7d2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.168+0000 7f963fe15740 1 --2- 172.21.15.47:0/3033110487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 0x55d81e7d2240 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f963003d000 tx=0x7f9630009190 comp rx=0 tx=0).stop 2024-03-23T09:48:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.168+0000 7f963fe15740 1 -- 172.21.15.47:0/3033110487 shutdown_connections 2024-03-23T09:48:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.168+0000 7f963fe15740 1 --2- 172.21.15.47:0/3033110487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 0x55d81e7d7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.168+0000 7f963fe15740 1 --2- 172.21.15.47:0/3033110487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d5080 0x55d81e7aa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.168+0000 7f963fe15740 1 --2- 172.21.15.47:0/3033110487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d18f0 0x55d81e7d2240 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.168+0000 7f963fe15740 1 -- 172.21.15.47:0/3033110487 >> 172.21.15.47:0/3033110487 conn(0x55d81e7a52f0 msgr2=0x55d81e7c4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.168+0000 7f963fe15740 1 -- 172.21.15.47:0/3033110487 shutdown_connections 2024-03-23T09:48:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.168+0000 7f963fe15740 1 -- 172.21.15.47:0/3033110487 wait complete. 2024-03-23T09:48:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.168+0000 7f963fe15740 1 -- start start 2024-03-23T09:48:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.168+0000 7f963fe15740 1 Processor -- start 2024-03-23T09:48:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963fe15740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d18f0 0x55d81e7d8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963fe15740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d5080 0x55d81e808b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963fe15740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 0x55d81e809060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963fe15740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81e7efdb0 con 0x55d81e7d5080 2024-03-23T09:48:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963fe15740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81e857ad0 con 0x55d81e7f35f0 2024-03-23T09:48:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963fe15740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d81e7d8760 con 0x55d81e7d18f0 2024-03-23T09:48:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963fe15740 10 client.0 _ll_register_callbacks cb 0x7f9630006c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d5080 0x55d81e808b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d5080 0x55d81e808b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38708/0 (socket says 172.21.15.47:38708) 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963659c640 1 -- 172.21.15.47:0/486868730 learned_addr learned my addr 172.21.15.47:0/486868730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963659c640 1 -- 172.21.15.47:0/486868730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d18f0 msgr2=0x55d81e7d8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963659c640 1 --2- 172.21.15.47:0/486868730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d18f0 0x55d81e7d8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963659c640 1 -- 172.21.15.47:0/486868730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 msgr2=0x55d81e809060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963659c640 1 --2- 172.21.15.47:0/486868730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 0x55d81e809060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.169+0000 7f963659c640 1 -- 172.21.15.47:0/486868730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d81e6db8a0 con 0x55d81e7d5080 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f963659c640 1 --2- 172.21.15.47:0/486868730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d5080 0x55d81e808b20 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f9620005fd0 tx=0x7f9620001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb8f0f506c66747 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f9615ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f9615ffb640 1 -- 172.21.15.47:0/486868730 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620001ae0 con 0x55d81e7d5080 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f9615ffb640 1 -- 172.21.15.47:0/486868730 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9620007aa0 con 0x55d81e7d5080 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f963fe15740 1 -- 172.21.15.47:0/486868730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9630031d10 con 0x55d81e7d5080 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f9615ffb640 1 -- 172.21.15.47:0/486868730 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9620019db0 con 0x55d81e7d5080 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f963fe15740 10 client.7983 fetch_fsmap learned FSMap version 53 2024-03-23T09:48:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f963fe15740 1 -- 172.21.15.47:0/486868730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55d81e6d55f0 con 0x55d81e7d5080 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f9615ffb640 1 -- 172.21.15.47:0/486868730 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9630031d10 con 0x55d81e7d5080 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f9615ffb640 1 -- 172.21.15.47:0/486868730 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 53) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9620012040 con 0x55d81e7d5080 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f963fe15740 10 client.7983 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f963fe15740 10 client.7983 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f963fe15740 20 client.7983 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f963fe15740 1 --2- 172.21.15.47:0/486868730 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55d81e7d2210 0x55d81e8095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f963fe15740 4 client.7983 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f963fe15740 1 -- 172.21.15.47:0/486868730 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x55d81e467eb0 con 0x55d81e7d2210 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.170+0000 7f9636d9d640 1 --2- 172.21.15.47:0/486868730 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55d81e7d2210 0x55d81e8095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f9615ffb640 10 client.7983 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f9636d9d640 1 --2- 172.21.15.47:0/486868730 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55d81e7d2210 0x55d81e8095a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bcc2b49232c94eb server_cookie=cadca3f78bb26c9e in_seq=0 out_seq=0 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f9615ffb640 1 -- 172.21.15.47:0/486868730 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d81e467eb0 con 0x55d81e7d2210 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f9615ffb640 10 client.7983 handle_command_reply: tid=0 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f963fe15740 1 client.7983 shutdown 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f963fe15740 1 -- 172.21.15.47:0/486868730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d5080 msgr2=0x55d81e808b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f963fe15740 1 --2- 172.21.15.47:0/486868730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d5080 0x55d81e808b20 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f9620005fd0 tx=0x7f9620001080 comp rx=0 tx=0).stop 2024-03-23T09:48:07.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f963fe15740 1 -- 172.21.15.47:0/486868730 shutdown_connections 2024-03-23T09:48:07.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f963fe15740 1 --2- 172.21.15.47:0/486868730 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55d81e7d2210 0x55d81e8095a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f9615ffb640 0 client.7983 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:07.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f963fe15740 1 --2- 172.21.15.47:0/486868730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d81e7f35f0 0x55d81e809060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f963fe15740 1 --2- 172.21.15.47:0/486868730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d81e7d5080 0x55d81e808b20 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f963fe15740 1 --2- 172.21.15.47:0/486868730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d81e7d18f0 0x55d81e7d8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f963fe15740 1 -- 172.21.15.47:0/486868730 >> 172.21.15.47:0/486868730 conn(0x55d81e7a52f0 msgr2=0x55d81e7d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:07.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f963fe15740 1 -- 172.21.15.47:0/486868730 shutdown_connections 2024-03-23T09:48:07.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f963fe15740 1 -- 172.21.15.47:0/486868730 wait complete. 2024-03-23T09:48:07.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.171+0000 7f963fe15740 20 client.7983 trim_cache size 0 max 16384 2024-03-23T09:48:07.177 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:48:07.172+0000 7f3d20500640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-03-23T09:47:17.860649+0000 1=1+0)->f(v1 m2024-03-23T09:47:17.860649+0000) n(v0 rc2024-03-23T09:47:17.976300+0000 2=1+1)->n(v0 rc2024-03-23T09:47:17.976300+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5650412f2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-23T09:47:17.860649+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-23T09:47:17.976300+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-03-23T09:48:07.178 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:48:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-23T09:48:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "52c61cce-5a53-469c-8b21-3e9563d7346b", 2024-03-23T09:48:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-23T09:48:07.178 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:48:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.172+0000 7f963d1c4640 1 -- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f961806ac00 msgr2=0x7f961806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.172+0000 7f963d1c4640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f961806ac00 0x7f961806d0c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9630036000 tx=0x7f963003e040 comp rx=0 tx=0).stop 2024-03-23T09:48:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.172+0000 7f963d1c4640 1 -- 172.21.15.47:0/3072075740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963815d0c0 msgr2=0x7f9638158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.172+0000 7f963d1c4640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963815d0c0 0x7f9638158300 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f9628007e40 tx=0x7f962800e4b0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.172+0000 7f963d1c4640 1 -- 172.21.15.47:0/3072075740 shutdown_connections 2024-03-23T09:48:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.172+0000 7f963d1c4640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f961806ac00 0x7f961806d0c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.172+0000 7f963d1c4640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963815d0c0 0x7f9638158300 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.172+0000 7f963d1c4640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f963815a700 0x7f96380c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.172+0000 7f963d1c4640 1 --2- 172.21.15.47:0/3072075740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96380d2f00 0x7f963815e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.172+0000 7f963d1c4640 1 -- 172.21.15.47:0/3072075740 >> 172.21.15.47:0/3072075740 conn(0x7f963806afd0 msgr2=0x7f96380d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:07.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.172+0000 7f963d1c4640 1 -- 172.21.15.47:0/3072075740 shutdown_connections 2024-03-23T09:48:07.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.172+0000 7f963d1c4640 1 -- 172.21.15.47:0/3072075740 wait complete. 2024-03-23T09:48:07.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:07.183 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-23T09:48:07.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.286+0000 7fc6ac799640 1 Processor -- start 2024-03-23T09:48:07.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.286+0000 7fc6ac799640 1 -- start start 2024-03-23T09:48:07.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.287+0000 7fc6ac799640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc6a415c990 0x7fc6a4160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.287+0000 7fc6ac799640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6a40b8350 0x7fc6a40cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.287+0000 7fc6ac799640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a4161630 con 0x7fc6a40cdc40 2024-03-23T09:48:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.287+0000 7fc6ac799640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a40d1030 con 0x7fc6a40b8350 2024-03-23T09:48:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.287+0000 7fc6ac799640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a40ce070 con 0x7fc6a415c990 2024-03-23T09:48:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.287+0000 7fc6a9d34640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6a40b8350 0x7fc6a40cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.287+0000 7fc6a9d34640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6a40b8350 0x7fc6a40cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38716/0 (socket says 172.21.15.47:38716) 2024-03-23T09:48:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.287+0000 7fc6a9d34640 1 -- 172.21.15.47:0/2942432697 learned_addr learned my addr 172.21.15.47:0/2942432697 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.287+0000 7fc6aa535640 1 --2- 172.21.15.47:0/2942432697 >> v2:172.21.15.175:3300/0 conn(0x7fc6a415c990 0x7fc6a4160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.287+0000 7fc6a9d34640 1 -- 172.21.15.47:0/2942432697 >> v2:172.21.15.175:3300/0 conn(0x7fc6a415c990 msgr2=0x7fc6a4160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.287+0000 7fc6a9d34640 1 --2- 172.21.15.47:0/2942432697 >> v2:172.21.15.175:3300/0 conn(0x7fc6a415c990 0x7fc6a4160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.287+0000 7fc6a9d34640 1 -- 172.21.15.47:0/2942432697 >> v1:172.21.15.175:6789/0 conn(0x7fc6a40cdc40 legacy=0x7fc6a40b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.287+0000 7fc6a9d34640 1 -- 172.21.15.47:0/2942432697 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a40cfe10 con 0x7fc6a40b8350 2024-03-23T09:48:07.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.288+0000 7fc6a9d34640 1 --2- 172.21.15.47:0/2942432697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a40b8350 0x7fc6a40cd700 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40b8b30 tx=0x7fc69402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a20e1aae0fc18e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.288+0000 7fc6a9533640 1 -- 172.21.15.47:0/2942432697 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc694031870 con 0x7fc6a40b8350 2024-03-23T09:48:07.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.288+0000 7fc6a9533640 1 -- 172.21.15.47:0/2942432697 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6940319d0 con 0x7fc6a40b8350 2024-03-23T09:48:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.288+0000 7fc6a9533640 1 -- 172.21.15.47:0/2942432697 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc694031d60 con 0x7fc6a40b8350 2024-03-23T09:48:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.289+0000 7fc6ac799640 1 -- 172.21.15.47:0/2942432697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a40b8350 msgr2=0x7fc6a40cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.289+0000 7fc6ac799640 1 --2- 172.21.15.47:0/2942432697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a40b8350 0x7fc6a40cd700 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40b8b30 tx=0x7fc69402b910 comp rx=0 tx=0).stop 2024-03-23T09:48:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.289+0000 7fc6ac799640 1 -- 172.21.15.47:0/2942432697 shutdown_connections 2024-03-23T09:48:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.289+0000 7fc6ac799640 1 --2- 172.21.15.47:0/2942432697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a40b8350 0x7fc6a40cd700 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.289+0000 7fc6ac799640 1 --2- 172.21.15.47:0/2942432697 >> v2:172.21.15.175:3300/0 conn(0x7fc6a415c990 0x7fc6a4160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.289+0000 7fc6ac799640 1 -- 172.21.15.47:0/2942432697 >> 172.21.15.47:0/2942432697 conn(0x7fc6a40659d0 msgr2=0x7fc6a4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.289+0000 7fc6ac799640 1 -- 172.21.15.47:0/2942432697 shutdown_connections 2024-03-23T09:48:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.289+0000 7fc6ac799640 1 -- 172.21.15.47:0/2942432697 wait complete. 2024-03-23T09:48:07.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.290+0000 7fc6ac799640 1 Processor -- start 2024-03-23T09:48:07.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.290+0000 7fc6ac799640 1 -- start start 2024-03-23T09:48:07.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6ac799640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a40b8350 0x7fc6a41f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6ac799640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6a40cdc40 0x7fc6a4159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6ac799640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a415c990 0x7fc6a4155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6ac799640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a4159c60 con 0x7fc6a415c990 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6ac799640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a415a6f0 con 0x7fc6a40cdc40 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6ac799640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a4159e50 con 0x7fc6a40b8350 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6aad36640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a415c990 0x7fc6a4155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6aad36640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a415c990 0x7fc6a4155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38724/0 (socket says 172.21.15.47:38724) 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6aad36640 1 -- 172.21.15.47:0/2070604645 learned_addr learned my addr 172.21.15.47:0/2070604645 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6a9d34640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6a40cdc40 0x7fc6a4159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6aa535640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a40b8350 0x7fc6a41f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6aad36640 1 -- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a40b8350 msgr2=0x7fc6a41f65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6aad36640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a40b8350 0x7fc6a41f65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6aad36640 1 -- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6a40cdc40 msgr2=0x7fc6a4159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6aad36640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6a40cdc40 0x7fc6a4159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6aad36640 1 -- 172.21.15.47:0/2070604645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6940094d0 con 0x7fc6a415c990 2024-03-23T09:48:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6aad36640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a415c990 0x7fc6a4155b00 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c7a0 tx=0x7fc69c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc69affd640 1 -- 172.21.15.47:0/2070604645 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c003b70 con 0x7fc6a415c990 2024-03-23T09:48:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc69affd640 1 -- 172.21.15.47:0/2070604645 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69c003cf0 con 0x7fc6a415c990 2024-03-23T09:48:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc69affd640 1 -- 172.21.15.47:0/2070604645 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c003ea0 con 0x7fc6a415c990 2024-03-23T09:48:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6ac799640 1 -- 172.21.15.47:0/2070604645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a415a130 con 0x7fc6a415c990 2024-03-23T09:48:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.291+0000 7fc6ac799640 1 -- 172.21.15.47:0/2070604645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a40c64c0 con 0x7fc6a415c990 2024-03-23T09:48:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.293+0000 7fc69affd640 1 -- 172.21.15.47:0/2070604645 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc69c00aa90 con 0x7fc6a415c990 2024-03-23T09:48:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.293+0000 7fc69affd640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc68806ac00 0x7fc68806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.293+0000 7fc69affd640 1 -- 172.21.15.47:0/2070604645 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc69c0891c0 con 0x7fc6a415c990 2024-03-23T09:48:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.294+0000 7fc6aa535640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc68806ac00 0x7fc68806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.294+0000 7fc6aa535640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc68806ac00 0x7fc68806d0c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0006fd0 tx=0x7fc6a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.294+0000 7fc6af3e3740 1 Processor -- start 2024-03-23T09:48:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.295+0000 7fc6af3e3740 1 -- start start 2024-03-23T09:48:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.295+0000 7fc6af3e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 0x55b3fa087c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.295+0000 7fc6af3e3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa057850 0x55b3fa084aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.295+0000 7fc6af3e3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 0x55b3fa084fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.295+0000 7fc6af3e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f9fc7dd0 con 0x55b3fa087830 2024-03-23T09:48:07.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.295+0000 7fc6af3e3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f9fc5560 con 0x55b3fa05b3b0 2024-03-23T09:48:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.295+0000 7fc6af3e3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3f9fc0e50 con 0x55b3fa057850 2024-03-23T09:48:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.295+0000 7fc6aa535640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 0x55b3fa087c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.295+0000 7fc6aa535640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 0x55b3fa087c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38728/0 (socket says 172.21.15.47:38728) 2024-03-23T09:48:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.295+0000 7fc6aa535640 1 -- 172.21.15.47:0/2167683762 learned_addr learned my addr 172.21.15.47:0/2167683762 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.295+0000 7fc6a9d34640 1 --2- 172.21.15.47:0/2167683762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa057850 0x55b3fa084aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.295+0000 7fc6aad36640 1 --2- 172.21.15.47:0/2167683762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 0x55b3fa084fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.296+0000 7fc6aa535640 1 -- 172.21.15.47:0/2167683762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa057850 msgr2=0x55b3fa084aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.296+0000 7fc6aa535640 1 --2- 172.21.15.47:0/2167683762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa057850 0x55b3fa084aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.296+0000 7fc6aa535640 1 -- 172.21.15.47:0/2167683762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 msgr2=0x55b3fa084fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.296+0000 7fc6aa535640 1 --2- 172.21.15.47:0/2167683762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 0x55b3fa084fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.296+0000 7fc6aa535640 1 -- 172.21.15.47:0/2167683762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3f9f88470 con 0x55b3fa087830 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.296+0000 7fc6aa535640 1 --2- 172.21.15.47:0/2167683762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 0x55b3fa087c10 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000f9c0 tx=0x7fc6a0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97109e21abbb8254 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.296+0000 7fc687fff640 1 -- 172.21.15.47:0/2167683762 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0014be0 con 0x55b3fa087830 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.296+0000 7fc687fff640 1 -- 172.21.15.47:0/2167683762 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a0019070 con 0x55b3fa087830 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.296+0000 7fc687fff640 1 -- 172.21.15.47:0/2167683762 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a000b5e0 con 0x55b3fa087830 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.297+0000 7fc6af3e3740 1 -- 172.21.15.47:0/2167683762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 msgr2=0x55b3fa087c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.297+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/2167683762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 0x55b3fa087c10 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000f9c0 tx=0x7fc6a0012000 comp rx=0 tx=0).stop 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.297+0000 7fc6af3e3740 1 -- 172.21.15.47:0/2167683762 shutdown_connections 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.297+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/2167683762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 0x55b3fa084fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.297+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/2167683762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa057850 0x55b3fa084aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.297+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/2167683762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 0x55b3fa087c10 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.297+0000 7fc6af3e3740 1 -- 172.21.15.47:0/2167683762 >> 172.21.15.47:0/2167683762 conn(0x55b3fa059ac0 msgr2=0x55b3fa0a2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.297+0000 7fc6af3e3740 1 -- 172.21.15.47:0/2167683762 shutdown_connections 2024-03-23T09:48:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.297+0000 7fc6af3e3740 1 -- 172.21.15.47:0/2167683762 wait complete. 2024-03-23T09:48:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.297+0000 7fc6af3e3740 1 -- start start 2024-03-23T09:48:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.297+0000 7fc6af3e3740 1 Processor -- start 2024-03-23T09:48:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.298+0000 7fc6af3e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa05b3b0 0x55b3fa080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.298+0000 7fc6af3e3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa087830 0x55b3fa06d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.298+0000 7fc6af3e3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa07c870 0x55b3fa06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.298+0000 7fc6af3e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f9f7f280 con 0x55b3fa05b3b0 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.298+0000 7fc6af3e3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f9f7ef20 con 0x55b3fa07c870 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.298+0000 7fc6af3e3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3f9f443c0 con 0x55b3fa087830 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.298+0000 7fc6af3e3740 10 client.0 _ll_register_callbacks cb 0x55b3f9f96380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.298+0000 7fc6aa535640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa05b3b0 0x55b3fa080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.298+0000 7fc6aa535640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa05b3b0 0x55b3fa080310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38736/0 (socket says 172.21.15.47:38736) 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.298+0000 7fc6aa535640 1 -- 172.21.15.47:0/4062483048 learned_addr learned my addr 172.21.15.47:0/4062483048 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.298+0000 7fc6a9d34640 1 --2- 172.21.15.47:0/4062483048 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa087830 0x55b3fa06d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.298+0000 7fc6aad36640 1 --2- 172.21.15.47:0/4062483048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa07c870 0x55b3fa06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6aa535640 1 -- 172.21.15.47:0/4062483048 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa087830 msgr2=0x55b3fa06d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6aa535640 1 --2- 172.21.15.47:0/4062483048 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa087830 0x55b3fa06d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6aa535640 1 -- 172.21.15.47:0/4062483048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa07c870 msgr2=0x55b3fa06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6aa535640 1 --2- 172.21.15.47:0/4062483048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa07c870 0x55b3fa06dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6aa535640 1 -- 172.21.15.47:0/4062483048 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a000e040 con 0x55b3fa05b3b0 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6aa535640 1 --2- 172.21.15.47:0/4062483048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa05b3b0 0x55b3fa080310 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0022040 tx=0x7fc6a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9c35db6ab94f1c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6857fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6857fa640 1 -- 172.21.15.47:0/4062483048 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0020850 con 0x55b3fa05b3b0 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6857fa640 1 -- 172.21.15.47:0/4062483048 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc6a0035b80 con 0x55b3fa05b3b0 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6af3e3740 1 -- 172.21.15.47:0/4062483048 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3f9fea690 con 0x55b3fa05b3b0 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6857fa640 1 -- 172.21.15.47:0/4062483048 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6a0035d40 con 0x55b3fa05b3b0 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6af3e3740 10 client.7995 fetch_fsmap learned FSMap version 53 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6af3e3740 1 -- 172.21.15.47:0/4062483048 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55b3f9fbf4d0 con 0x55b3fa05b3b0 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6857fa640 1 -- 172.21.15.47:0/4062483048 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3f9fea690 con 0x55b3fa05b3b0 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.299+0000 7fc6857fa640 1 -- 172.21.15.47:0/4062483048 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 53) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc6a00209b0 con 0x55b3fa05b3b0 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.300+0000 7fc6af3e3740 10 client.7995 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.300+0000 7fc6af3e3740 10 client.7995 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.300+0000 7fc6af3e3740 20 client.7995 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.300+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/4062483048 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55b3fa08ab80 0x55b3fa08af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.300+0000 7fc6af3e3740 4 client.7995 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.300+0000 7fc6af3e3740 1 -- 172.21.15.47:0/4062483048 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3f9fb2eb0 con 0x55b3fa08ab80 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.300+0000 7fc6a9d34640 1 --2- 172.21.15.47:0/4062483048 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55b3fa08ab80 0x55b3fa08af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.300+0000 7fc6a9d34640 1 --2- 172.21.15.47:0/4062483048 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55b3fa08ab80 0x55b3fa08af90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8493c80f721a1684 server_cookie=f5fcbb626ef5606e in_seq=0 out_seq=0 2024-03-23T09:48:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.300+0000 7fc6857fa640 10 client.7995 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.300+0000 7fc6857fa640 1 -- 172.21.15.47:0/4062483048 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b3f9fb2eb0 con 0x55b3fa08ab80 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.300+0000 7fc6857fa640 10 client.7995 handle_command_reply: tid=0 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.300+0000 7fc6af3e3740 1 client.7995 shutdown 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.301+0000 7fc6af3e3740 1 -- 172.21.15.47:0/4062483048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa05b3b0 msgr2=0x55b3fa080310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.301+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/4062483048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa05b3b0 0x55b3fa080310 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0022040 tx=0x7fc6a0004730 comp rx=0 tx=0).stop 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.301+0000 7fc6af3e3740 1 -- 172.21.15.47:0/4062483048 shutdown_connections 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.301+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/4062483048 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55b3fa08ab80 0x55b3fa08af90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.301+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/4062483048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa07c870 0x55b3fa06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.301+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/4062483048 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa087830 0x55b3fa06d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.301+0000 7fc6857fa640 0 client.7995 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.301+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/4062483048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa05b3b0 0x55b3fa080310 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.301+0000 7fc6af3e3740 1 -- 172.21.15.47:0/4062483048 >> 172.21.15.47:0/4062483048 conn(0x55b3fa057850 msgr2=0x55b3fa05bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.301+0000 7fc6af3e3740 1 -- 172.21.15.47:0/4062483048 shutdown_connections 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.301+0000 7fc6af3e3740 1 -- 172.21.15.47:0/4062483048 wait complete. 2024-03-23T09:48:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.301+0000 7fc6af3e3740 20 client.7995 trim_cache size 0 max 16384 2024-03-23T09:48:07.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.315+0000 7fc6af3e3740 1 Processor -- start 2024-03-23T09:48:07.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.315+0000 7fc6af3e3740 1 -- start start 2024-03-23T09:48:07.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6af3e3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 0x55b3fa082610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6af3e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa07c870 0x55b3fa084ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6af3e3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa087830 0x55b3fa088c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6af3e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3fa0891b0 con 0x55b3fa07c870 2024-03-23T09:48:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6af3e3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3fa0b4790 con 0x55b3fa05b3b0 2024-03-23T09:48:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6af3e3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3fa082cb0 con 0x55b3fa087830 2024-03-23T09:48:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6a9d34640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa07c870 0x55b3fa084ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6a9d34640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa07c870 0x55b3fa084ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38748/0 (socket says 172.21.15.47:38748) 2024-03-23T09:48:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6a9d34640 1 -- 172.21.15.47:0/657592738 learned_addr learned my addr 172.21.15.47:0/657592738 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6a9d34640 1 -- 172.21.15.47:0/657592738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa087830 msgr2=0x55b3fa088c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6a9d34640 1 --2- 172.21.15.47:0/657592738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa087830 0x55b3fa088c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6a9d34640 1 -- 172.21.15.47:0/657592738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 msgr2=0x55b3fa082610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6aa535640 1 --2- 172.21.15.47:0/657592738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 0x55b3fa082610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6a9d34640 1 --2- 172.21.15.47:0/657592738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 0x55b3fa082610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6a9d34640 1 -- 172.21.15.47:0/657592738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3f9ff39e0 con 0x55b3fa07c870 2024-03-23T09:48:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6aa535640 1 --2- 172.21.15.47:0/657592738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 0x55b3fa082610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc6a9d34640 1 --2- 172.21.15.47:0/657592738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa07c870 0x55b3fa084ef0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fc694009960 tx=0x7fc694008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4dea22375669b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc698ff9640 1 -- 172.21.15.47:0/657592738 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69404a300 con 0x55b3fa07c870 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc698ff9640 1 -- 172.21.15.47:0/657592738 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc694004040 con 0x55b3fa07c870 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.316+0000 7fc698ff9640 1 -- 172.21.15.47:0/657592738 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6940043d0 con 0x55b3fa07c870 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.317+0000 7fc6af3e3740 1 -- 172.21.15.47:0/657592738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa07c870 msgr2=0x55b3fa084ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.317+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/657592738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa07c870 0x55b3fa084ef0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fc694009960 tx=0x7fc694008030 comp rx=0 tx=0).stop 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.317+0000 7fc6af3e3740 1 -- 172.21.15.47:0/657592738 shutdown_connections 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.317+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/657592738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa087830 0x55b3fa088c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.317+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/657592738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa07c870 0x55b3fa084ef0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.317+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/657592738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 0x55b3fa082610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.317+0000 7fc6af3e3740 1 -- 172.21.15.47:0/657592738 >> 172.21.15.47:0/657592738 conn(0x55b3fa057850 msgr2=0x55b3fa053030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.317+0000 7fc6af3e3740 1 -- 172.21.15.47:0/657592738 shutdown_connections 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.317+0000 7fc6af3e3740 1 -- 172.21.15.47:0/657592738 wait complete. 2024-03-23T09:48:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.318+0000 7fc6af3e3740 1 -- start start 2024-03-23T09:48:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.318+0000 7fc6af3e3740 1 Processor -- start 2024-03-23T09:48:07.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.318+0000 7fc6af3e3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 0x55b3fa05d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.318+0000 7fc6af3e3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa07c870 0x55b3fa0eb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.318+0000 7fc6af3e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 0x55b3fa0ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6af3e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3fa100540 con 0x55b3fa087830 2024-03-23T09:48:07.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6af3e3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3fa0a2020 con 0x55b3fa05b3b0 2024-03-23T09:48:07.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6af3e3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3fa0eab90 con 0x55b3fa07c870 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6af3e3740 10 client.0 _ll_register_callbacks cb 0x7fc6a000b220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6aad36640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 0x55b3fa0ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6aad36640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 0x55b3fa0ff9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38758/0 (socket says 172.21.15.47:38758) 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6aad36640 1 -- 172.21.15.47:0/340147297 learned_addr learned my addr 172.21.15.47:0/340147297 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6aa535640 1 --2- 172.21.15.47:0/340147297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 0x55b3fa05d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6a9d34640 1 --2- 172.21.15.47:0/340147297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa07c870 0x55b3fa0eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6aad36640 1 -- 172.21.15.47:0/340147297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa07c870 msgr2=0x55b3fa0eb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6aad36640 1 --2- 172.21.15.47:0/340147297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa07c870 0x55b3fa0eb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6aad36640 1 -- 172.21.15.47:0/340147297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 msgr2=0x55b3fa05d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6aad36640 1 --2- 172.21.15.47:0/340147297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 0x55b3fa05d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6aad36640 1 -- 172.21.15.47:0/340147297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a40cfe10 con 0x55b3fa087830 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6aad36640 1 --2- 172.21.15.47:0/340147297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 0x55b3fa0ff9a0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fc69c057d60 tx=0x7fc69c0875c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8529a500d150ee server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6857fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6857fa640 1 -- 172.21.15.47:0/340147297 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c06bea0 con 0x55b3fa087830 2024-03-23T09:48:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6857fa640 1 -- 172.21.15.47:0/340147297 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc69c05be90 con 0x55b3fa087830 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6af3e3740 1 -- 172.21.15.47:0/340147297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6a0019070 con 0x55b3fa087830 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6857fa640 1 -- 172.21.15.47:0/340147297 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc69c060450 con 0x55b3fa087830 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6af3e3740 10 client.8001 fetch_fsmap learned FSMap version 53 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.319+0000 7fc6af3e3740 1 -- 172.21.15.47:0/340147297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55b3f9f615c0 con 0x55b3fa087830 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6857fa640 1 -- 172.21.15.47:0/340147297 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69c076020 con 0x55b3fa087830 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6857fa640 1 -- 172.21.15.47:0/340147297 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 53) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc69c019020 con 0x55b3fa087830 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 10 client.8001 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 10 client.8001 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 20 client.8001 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/340147297 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55b3fa0e80d0 0x55b3fa0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 4 client.8001 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 1 -- 172.21.15.47:0/340147297 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b3f9cb44e0 con 0x55b3fa0e80d0 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6aa535640 1 --2- 172.21.15.47:0/340147297 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55b3fa0e80d0 0x55b3fa0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6857fa640 10 client.8001 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6aa535640 1 --2- 172.21.15.47:0/340147297 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55b3fa0e80d0 0x55b3fa0a5b50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61367d497bde7fb5 server_cookie=825b4333e1c718a4 in_seq=0 out_seq=0 2024-03-23T09:48:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6857fa640 1 -- 172.21.15.47:0/340147297 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x55b3f9cb44e0 con 0x55b3fa0e80d0 2024-03-23T09:48:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6857fa640 10 client.8001 handle_command_reply: tid=0 2024-03-23T09:48:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 1 client.8001 shutdown 2024-03-23T09:48:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 1 -- 172.21.15.47:0/340147297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 msgr2=0x55b3fa0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/340147297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 0x55b3fa0ff9a0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fc69c057d60 tx=0x7fc69c0875c0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 1 -- 172.21.15.47:0/340147297 shutdown_connections 2024-03-23T09:48:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/340147297 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55b3fa0e80d0 0x55b3fa0a5b50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/340147297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3fa087830 0x55b3fa0ff9a0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/340147297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3fa07c870 0x55b3fa0eb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6857fa640 0 client.8001 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 1 --2- 172.21.15.47:0/340147297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3fa05b3b0 0x55b3fa05d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.320+0000 7fc6af3e3740 1 -- 172.21.15.47:0/340147297 >> 172.21.15.47:0/340147297 conn(0x55b3fa057850 msgr2=0x55b3fa057110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6af3e3740 1 -- 172.21.15.47:0/340147297 shutdown_connections 2024-03-23T09:48:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6af3e3740 1 -- 172.21.15.47:0/340147297 wait complete. 2024-03-23T09:48:07.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6af3e3740 20 client.8001 trim_cache size 0 max 16384 2024-03-23T09:48:07.327 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:48:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T09:48:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-23T09:48:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "52c61cce-5a53-469c-8b21-3e9563d7346b": { 2024-03-23T09:48:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/dir", 2024-03-23T09:48:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "52c61cce-5a53-469c-8b21-3e9563d7346b", 2024-03-23T09:48:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,repair" 2024-03-23T09:48:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:48:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:48:07.327 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:48:07.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6ac799640 1 -- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc68806ac00 msgr2=0x7fc68806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6ac799640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc68806ac00 0x7fc68806d0c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0006fd0 tx=0x7fc6a0008040 comp rx=0 tx=0).stop 2024-03-23T09:48:07.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6ac799640 1 -- 172.21.15.47:0/2070604645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a415c990 msgr2=0x7fc6a4155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:07.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6ac799640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a415c990 0x7fc6a4155b00 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c7a0 tx=0x7fc69c00cc60 comp rx=0 tx=0).stop 2024-03-23T09:48:07.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6ac799640 1 -- 172.21.15.47:0/2070604645 shutdown_connections 2024-03-23T09:48:07.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6ac799640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc68806ac00 0x7fc68806d0c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6ac799640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6a415c990 0x7fc6a4155b00 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6ac799640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6a40cdc40 0x7fc6a4159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6ac799640 1 --2- 172.21.15.47:0/2070604645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6a40b8350 0x7fc6a41f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:07.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6ac799640 1 -- 172.21.15.47:0/2070604645 >> 172.21.15.47:0/2070604645 conn(0x7fc6a40659d0 msgr2=0x7fc6a40b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:07.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6ac799640 1 -- 172.21.15.47:0/2070604645 shutdown_connections 2024-03-23T09:48:07.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:07.321+0000 7fc6ac799640 1 -- 172.21.15.47:0/2070604645 wait complete. 2024-03-23T09:48:07.331 INFO:tasks.cephfs.filesystem:scrub status for tag:52c61cce-5a53-469c-8b21-3e9563d7346b - {'path': '/dir', 'tag': '52c61cce-5a53-469c-8b21-3e9563d7346b', 'options': 'recursive,repair'} 2024-03-23T09:48:08.177 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:48:08.180 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:48:37.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:37.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-23T09:48:37.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.448+0000 7fa2b0f2e640 1 Processor -- start 2024-03-23T09:48:37.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.448+0000 7fa2b0f2e640 1 -- start start 2024-03-23T09:48:37.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.449+0000 7fa2b0f2e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa2ac15ca30 0x7fa2ac160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.449+0000 7fa2b0f2e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa2ac15ce10 0x7fa2ac0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.449+0000 7fa2b0f2e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2ac0b85d0 0x7fa2ac0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.449+0000 7fa2b0f2e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ac161600 con 0x7fa2ac15ce10 2024-03-23T09:48:37.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.449+0000 7fa2b0f2e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ac161040 con 0x7fa2ac15ca30 2024-03-23T09:48:37.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.449+0000 7fa2b0f2e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac161320 con 0x7fa2ac0b85d0 2024-03-23T09:48:37.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.449+0000 7fa2a9d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa2ac15ce10 0x7fa2ac0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.449+0000 7fa2a9d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa2ac15ce10 0x7fa2ac0cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52496/0 (socket says 172.21.15.47:52496) 2024-03-23T09:48:37.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.449+0000 7fa2a9d9b640 1 -- 172.21.15.47:0/2475951331 learned_addr learned my addr 172.21.15.47:0/2475951331 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:37.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.449+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/2475951331 >> v2:172.21.15.175:3300/0 conn(0x7fa2ac15ca30 0x7fa2ac160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.449+0000 7fa2aad9d640 1 --2- 172.21.15.47:0/2475951331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2ac0b85d0 0x7fa2ac0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.450+0000 7fa2a9d9b640 1 -- 172.21.15.47:0/2475951331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2ac0b85d0 msgr2=0x7fa2ac0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.450+0000 7fa2a9d9b640 1 --2- 172.21.15.47:0/2475951331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2ac0b85d0 0x7fa2ac0cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.450+0000 7fa2a9d9b640 1 -- 172.21.15.47:0/2475951331 >> v2:172.21.15.175:3300/0 conn(0x7fa2ac15ca30 msgr2=0x7fa2ac160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.450+0000 7fa2a9d9b640 1 --2- 172.21.15.47:0/2475951331 >> v2:172.21.15.175:3300/0 conn(0x7fa2ac15ca30 0x7fa2ac160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.450+0000 7fa2a9d9b640 1 -- 172.21.15.47:0/2475951331 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac0d0dd0 con 0x7fa2ac15ce10 2024-03-23T09:48:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.450+0000 7fa2aad9d640 1 --2- 172.21.15.47:0/2475951331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2ac0b85d0 0x7fa2ac0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.450+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/2475951331 >> v2:172.21.15.175:3300/0 conn(0x7fa2ac15ca30 0x7fa2ac160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.450+0000 7fa2a9d9b640 1 --2- 172.21.15.47:0/2475951331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2ac15ce10 0x7fa2ac0cd730 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac06c1a0 tx=0x7fa2a002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d5fed54946625b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.450+0000 7fa2a959a640 1 -- 172.21.15.47:0/2475951331 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0031860 con 0x7fa2ac15ce10 2024-03-23T09:48:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.450+0000 7fa2a959a640 1 -- 172.21.15.47:0/2475951331 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a00319c0 con 0x7fa2ac15ce10 2024-03-23T09:48:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.450+0000 7fa2a959a640 1 -- 172.21.15.47:0/2475951331 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0031d50 con 0x7fa2ac15ce10 2024-03-23T09:48:37.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.451+0000 7fa2b0f2e640 1 -- 172.21.15.47:0/2475951331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2ac15ce10 msgr2=0x7fa2ac0cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.451+0000 7fa2b0f2e640 1 --2- 172.21.15.47:0/2475951331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2ac15ce10 0x7fa2ac0cd730 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac06c1a0 tx=0x7fa2a002b900 comp rx=0 tx=0).stop 2024-03-23T09:48:37.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.451+0000 7fa2b0f2e640 1 -- 172.21.15.47:0/2475951331 shutdown_connections 2024-03-23T09:48:37.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.451+0000 7fa2b0f2e640 1 --2- 172.21.15.47:0/2475951331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2ac0b85d0 0x7fa2ac0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.451+0000 7fa2b0f2e640 1 --2- 172.21.15.47:0/2475951331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2ac15ce10 0x7fa2ac0cd730 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.451+0000 7fa2b0f2e640 1 --2- 172.21.15.47:0/2475951331 >> v2:172.21.15.175:3300/0 conn(0x7fa2ac15ca30 0x7fa2ac160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.451+0000 7fa2b0f2e640 1 -- 172.21.15.47:0/2475951331 >> 172.21.15.47:0/2475951331 conn(0x7fa2ac06b160 msgr2=0x7fa2ac06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.451+0000 7fa2b0f2e640 1 -- 172.21.15.47:0/2475951331 shutdown_connections 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.451+0000 7fa2b0f2e640 1 -- 172.21.15.47:0/2475951331 wait complete. 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.453+0000 7fa2b0f2e640 1 Processor -- start 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.453+0000 7fa2b0f2e640 1 -- start start 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.453+0000 7fa2b0f2e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2ac0b85d0 0x7fa2ac15d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.453+0000 7fa2b0f2e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2ac15ca30 0x7fa2ac15d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.453+0000 7fa2b0f2e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2ac15ce10 0x7fa2ac15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.453+0000 7fa2b0f2e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac159240 con 0x7fa2ac15ce10 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.453+0000 7fa2b0f2e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac158630 con 0x7fa2ac0b85d0 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.453+0000 7fa2b0f2e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac158910 con 0x7fa2ac15ca30 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.453+0000 7fa2aad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2ac15ce10 0x7fa2ac15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.453+0000 7fa2aad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2ac15ce10 0x7fa2ac15dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52498/0 (socket says 172.21.15.47:52498) 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa2aad9d640 1 -- 172.21.15.47:0/2002872951 learned_addr learned my addr 172.21.15.47:0/2002872951 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:37.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2ac0b85d0 0x7fa2ac15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa2a9d9b640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2ac15ca30 0x7fa2ac15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa2aad9d640 1 -- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2ac15ca30 msgr2=0x7fa2ac15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa2aad9d640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2ac15ca30 0x7fa2ac15d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa2aad9d640 1 -- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2ac0b85d0 msgr2=0x7fa2ac15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa2aad9d640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2ac0b85d0 0x7fa2ac15d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa2aad9d640 1 -- 172.21.15.47:0/2002872951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a00094d0 con 0x7fa2ac15ce10 2024-03-23T09:48:37.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa2aad9d640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2ac15ce10 0x7fa2ac15dca0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00c7f0 tx=0x7fa29c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa29affd640 1 -- 172.21.15.47:0/2002872951 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c007820 con 0x7fa2ac15ce10 2024-03-23T09:48:37.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa29affd640 1 -- 172.21.15.47:0/2002872951 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29c007980 con 0x7fa2ac15ce10 2024-03-23T09:48:37.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa29affd640 1 -- 172.21.15.47:0/2002872951 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c007d30 con 0x7fa2ac15ce10 2024-03-23T09:48:37.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa2b0f2e640 1 -- 172.21.15.47:0/2002872951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ac158bf0 con 0x7fa2ac15ce10 2024-03-23T09:48:37.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.454+0000 7fa2b0f2e640 1 -- 172.21.15.47:0/2002872951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac15e1e0 con 0x7fa2ac15ce10 2024-03-23T09:48:37.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.456+0000 7fa29affd640 1 -- 172.21.15.47:0/2002872951 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa29c0113f0 con 0x7fa2ac15ce10 2024-03-23T09:48:37.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.456+0000 7fa29affd640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa28c06ac00 0x7fa28c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.456+0000 7fa29affd640 1 -- 172.21.15.47:0/2002872951 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa29c0893d0 con 0x7fa2ac15ce10 2024-03-23T09:48:37.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.456+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa28c06ac00 0x7fa28c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.457+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa28c06ac00 0x7fa28c06d0c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400d620 tx=0x7fa2a4007810 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.457+0000 7fa2b3b7c740 1 Processor -- start 2024-03-23T09:48:37.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.457+0000 7fa2b3b7c740 1 -- start start 2024-03-23T09:48:37.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.458+0000 7fa2b3b7c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 0x55a4b9d73c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.458+0000 7fa2b3b7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d43850 0x55a4b9d70aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.458+0000 7fa2b3b7c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d473b0 0x55a4b9d70fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.458+0000 7fa2b3b7c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b9cb3dd0 con 0x55a4b9d73830 2024-03-23T09:48:37.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.458+0000 7fa2b3b7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b9cb1560 con 0x55a4b9d43850 2024-03-23T09:48:37.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.458+0000 7fa2b3b7c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4b9cace50 con 0x55a4b9d473b0 2024-03-23T09:48:37.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.458+0000 7fa2aa59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 0x55a4b9d73c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.458+0000 7fa2aa59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 0x55a4b9d73c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52506/0 (socket says 172.21.15.47:52506) 2024-03-23T09:48:37.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.458+0000 7fa2aa59c640 1 -- 172.21.15.47:0/3335964981 learned_addr learned my addr 172.21.15.47:0/3335964981 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:37.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.458+0000 7fa2a9d9b640 1 --2- 172.21.15.47:0/3335964981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d43850 0x55a4b9d70aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.458+0000 7fa2aad9d640 1 --2- 172.21.15.47:0/3335964981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d473b0 0x55a4b9d70fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.459+0000 7fa2aa59c640 1 -- 172.21.15.47:0/3335964981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d473b0 msgr2=0x55a4b9d70fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.459+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/3335964981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d473b0 0x55a4b9d70fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.459+0000 7fa2aa59c640 1 -- 172.21.15.47:0/3335964981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d43850 msgr2=0x55a4b9d70aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.459+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/3335964981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d43850 0x55a4b9d70aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.459+0000 7fa2aa59c640 1 -- 172.21.15.47:0/3335964981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4b9c74470 con 0x55a4b9d73830 2024-03-23T09:48:37.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.459+0000 7fa2aad9d640 1 --2- 172.21.15.47:0/3335964981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d473b0 0x55a4b9d70fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:37.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.459+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/3335964981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 0x55a4b9d73c10 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400c8c0 tx=0x7fa2a4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f2a2fbc18ddf43 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.459+0000 7fa28bfff640 1 -- 172.21.15.47:0/3335964981 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4012070 con 0x55a4b9d73830 2024-03-23T09:48:37.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.459+0000 7fa28bfff640 1 -- 172.21.15.47:0/3335964981 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a400ce40 con 0x55a4b9d73830 2024-03-23T09:48:37.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.459+0000 7fa28bfff640 1 -- 172.21.15.47:0/3335964981 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a401b9b0 con 0x55a4b9d73830 2024-03-23T09:48:37.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.460+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/3335964981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 msgr2=0x55a4b9d73c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.460+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/3335964981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 0x55a4b9d73c10 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400c8c0 tx=0x7fa2a4012000 comp rx=0 tx=0).stop 2024-03-23T09:48:37.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.460+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/3335964981 shutdown_connections 2024-03-23T09:48:37.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.460+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/3335964981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d473b0 0x55a4b9d70fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.460+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/3335964981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d43850 0x55a4b9d70aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.460+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/3335964981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 0x55a4b9d73c10 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.460+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/3335964981 >> 172.21.15.47:0/3335964981 conn(0x55a4b9d45ac0 msgr2=0x55a4b9d8e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.460+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/3335964981 shutdown_connections 2024-03-23T09:48:37.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.460+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/3335964981 wait complete. 2024-03-23T09:48:37.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.460+0000 7fa2b3b7c740 1 -- start start 2024-03-23T09:48:37.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.461+0000 7fa2b3b7c740 1 Processor -- start 2024-03-23T09:48:37.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.461+0000 7fa2b3b7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d473b0 0x55a4b9d68da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.461+0000 7fa2b3b7c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d73830 0x55a4b9d6a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.461+0000 7fa2b3b7c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d75600 0x55a4b9d6bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.461+0000 7fa2b3b7c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b9c6b280 con 0x55a4b9d75600 2024-03-23T09:48:37.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2b3b7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b9c6af20 con 0x55a4b9d473b0 2024-03-23T09:48:37.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2b3b7c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4b9c303c0 con 0x55a4b9d73830 2024-03-23T09:48:37.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2b3b7c740 10 client.0 _ll_register_callbacks cb 0x55a4b9c82380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:37.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2aad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d75600 0x55a4b9d6bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2aad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d75600 0x55a4b9d6bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52522/0 (socket says 172.21.15.47:52522) 2024-03-23T09:48:37.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2aad9d640 1 -- 172.21.15.47:0/2122131225 learned_addr learned my addr 172.21.15.47:0/2122131225 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:37.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/2122131225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d473b0 0x55a4b9d68da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2a9d9b640 1 --2- 172.21.15.47:0/2122131225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d73830 0x55a4b9d6a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2aad9d640 1 -- 172.21.15.47:0/2122131225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d73830 msgr2=0x55a4b9d6a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2aad9d640 1 --2- 172.21.15.47:0/2122131225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d73830 0x55a4b9d6a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2aad9d640 1 -- 172.21.15.47:0/2122131225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d473b0 msgr2=0x55a4b9d68da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2aad9d640 1 --2- 172.21.15.47:0/2122131225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d473b0 0x55a4b9d68da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2aad9d640 1 -- 172.21.15.47:0/2122131225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a400c5a0 con 0x55a4b9d75600 2024-03-23T09:48:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2aad9d640 1 --2- 172.21.15.47:0/2122131225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d75600 0x55a4b9d6bd80 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0580e0 tx=0x7fa29c058520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a791e99a1c66d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2897fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2897fa640 1 -- 172.21.15.47:0/2122131225 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c060490 con 0x55a4b9d75600 2024-03-23T09:48:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2897fa640 1 -- 172.21.15.47:0/2122131225 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa29c05e100 con 0x55a4b9d75600 2024-03-23T09:48:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2122131225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4b9cd6690 con 0x55a4b9d75600 2024-03-23T09:48:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2897fa640 1 -- 172.21.15.47:0/2122131225 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa29c05e2e0 con 0x55a4b9d75600 2024-03-23T09:48:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2b3b7c740 10 client.8013 fetch_fsmap learned FSMap version 53 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.462+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2122131225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55a4b9cab4d0 con 0x55a4b9d75600 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.463+0000 7fa2897fa640 1 -- 172.21.15.47:0/2122131225 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a4b9cd6690 con 0x55a4b9d75600 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.463+0000 7fa2897fa640 1 -- 172.21.15.47:0/2122131225 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 53) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa29c0541d0 con 0x55a4b9d75600 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.463+0000 7fa2b3b7c740 10 client.8013 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.463+0000 7fa2b3b7c740 10 client.8013 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.463+0000 7fa2b3b7c740 20 client.8013 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.463+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/2122131225 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55a4b9d6e020 0x55a4b9da1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.463+0000 7fa2b3b7c740 4 client.8013 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.463+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2122131225 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4b9c9eeb0 con 0x55a4b9d6e020 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.463+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/2122131225 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55a4b9d6e020 0x55a4b9da1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.463+0000 7fa2897fa640 10 client.8013 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.463+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/2122131225 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55a4b9d6e020 0x55a4b9da1350 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86c765c97ebbde8a server_cookie=5f556e87c642f3b3 in_seq=0 out_seq=0 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2897fa640 1 -- 172.21.15.47:0/2122131225 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a4b9c9eeb0 con 0x55a4b9d6e020 2024-03-23T09:48:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2897fa640 10 client.8013 handle_command_reply: tid=0 2024-03-23T09:48:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2b3b7c740 1 client.8013 shutdown 2024-03-23T09:48:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2122131225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d75600 msgr2=0x55a4b9d6bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/2122131225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d75600 0x55a4b9d6bd80 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0580e0 tx=0x7fa29c058520 comp rx=0 tx=0).stop 2024-03-23T09:48:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2122131225 shutdown_connections 2024-03-23T09:48:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/2122131225 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55a4b9d6e020 0x55a4b9da1350 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/2122131225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d75600 0x55a4b9d6bd80 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/2122131225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d73830 0x55a4b9d6a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2897fa640 0 client.8013 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/2122131225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d473b0 0x55a4b9d68da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2122131225 >> 172.21.15.47:0/2122131225 conn(0x55a4b9d43850 msgr2=0x55a4b9d47c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2122131225 shutdown_connections 2024-03-23T09:48:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2122131225 wait complete. 2024-03-23T09:48:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.464+0000 7fa2b3b7c740 20 client.8013 trim_cache size 0 max 16384 2024-03-23T09:48:37.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.479+0000 7fa2b3b7c740 1 Processor -- start 2024-03-23T09:48:37.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.479+0000 7fa2b3b7c740 1 -- start start 2024-03-23T09:48:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.479+0000 7fa2b3b7c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d473b0 0x55a4b9d70ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.479+0000 7fa2b3b7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d73830 0x55a4b9d91b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.479+0000 7fa2b3b7c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d75600 0x55a4b9d92090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.479+0000 7fa2b3b7c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b9dee7e0 con 0x55a4b9d473b0 2024-03-23T09:48:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.479+0000 7fa2b3b7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b9d706a0 con 0x55a4b9d73830 2024-03-23T09:48:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.479+0000 7fa2b3b7c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4b9d6abc0 con 0x55a4b9d75600 2024-03-23T09:48:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.479+0000 7fa2aa59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d473b0 0x55a4b9d70ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.479+0000 7fa2aa59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d473b0 0x55a4b9d70ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52526/0 (socket says 172.21.15.47:52526) 2024-03-23T09:48:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.479+0000 7fa2aa59c640 1 -- 172.21.15.47:0/966823616 learned_addr learned my addr 172.21.15.47:0/966823616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.480+0000 7fa2a9d9b640 1 --2- 172.21.15.47:0/966823616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d73830 0x55a4b9d91b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.480+0000 7fa2aad9d640 1 --2- 172.21.15.47:0/966823616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d75600 0x55a4b9d92090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.480+0000 7fa2aa59c640 1 -- 172.21.15.47:0/966823616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d75600 msgr2=0x55a4b9d92090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.480+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/966823616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d75600 0x55a4b9d92090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.480+0000 7fa2aa59c640 1 -- 172.21.15.47:0/966823616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d73830 msgr2=0x55a4b9d91b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.480+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/966823616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d73830 0x55a4b9d91b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.480+0000 7fa2aa59c640 1 -- 172.21.15.47:0/966823616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4b9cdf9e0 con 0x55a4b9d473b0 2024-03-23T09:48:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.480+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/966823616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d473b0 0x55a4b9d70ea0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400cfd0 tx=0x7fa2a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c8f01db4f7db5e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.480+0000 7fa298ff9640 1 -- 172.21.15.47:0/966823616 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4012040 con 0x55a4b9d473b0 2024-03-23T09:48:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.480+0000 7fa298ff9640 1 -- 172.21.15.47:0/966823616 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a40073a0 con 0x55a4b9d473b0 2024-03-23T09:48:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.480+0000 7fa298ff9640 1 -- 172.21.15.47:0/966823616 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4033410 con 0x55a4b9d473b0 2024-03-23T09:48:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.481+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/966823616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d473b0 msgr2=0x55a4b9d70ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.481+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/966823616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d473b0 0x55a4b9d70ea0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400cfd0 tx=0x7fa2a4014000 comp rx=0 tx=0).stop 2024-03-23T09:48:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.481+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/966823616 shutdown_connections 2024-03-23T09:48:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.481+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/966823616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d75600 0x55a4b9d92090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.481+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/966823616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d73830 0x55a4b9d91b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.481+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/966823616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d473b0 0x55a4b9d70ea0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.481+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/966823616 >> 172.21.15.47:0/966823616 conn(0x55a4b9d43850 msgr2=0x55a4b9d775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.481+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/966823616 shutdown_connections 2024-03-23T09:48:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.481+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/966823616 wait complete. 2024-03-23T09:48:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.481+0000 7fa2b3b7c740 1 -- start start 2024-03-23T09:48:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.481+0000 7fa2b3b7c740 1 Processor -- start 2024-03-23T09:48:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.482+0000 7fa2b3b7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d473b0 0x55a4b9d91b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.482+0000 7fa2b3b7c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 0x55a4b9d92090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.482+0000 7fa2b3b7c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d75600 0x55a4b9d903f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.482+0000 7fa2b3b7c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b9d91420 con 0x55a4b9d73830 2024-03-23T09:48:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.482+0000 7fa2b3b7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b9d6a340 con 0x55a4b9d473b0 2024-03-23T09:48:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.482+0000 7fa2b3b7c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4b9d6f1b0 con 0x55a4b9d75600 2024-03-23T09:48:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.482+0000 7fa2b3b7c740 10 client.0 _ll_register_callbacks cb 0x7fa2a4007d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.482+0000 7fa2a9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 0x55a4b9d92090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.482+0000 7fa2a9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 0x55a4b9d92090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52542/0 (socket says 172.21.15.47:52542) 2024-03-23T09:48:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.482+0000 7fa2a9d9b640 1 -- 172.21.15.47:0/2400546910 learned_addr learned my addr 172.21.15.47:0/2400546910 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.482+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/2400546910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d473b0 0x55a4b9d91b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.482+0000 7fa2aad9d640 1 --2- 172.21.15.47:0/2400546910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d75600 0x55a4b9d903f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2a9d9b640 1 -- 172.21.15.47:0/2400546910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d75600 msgr2=0x55a4b9d903f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2a9d9b640 1 --2- 172.21.15.47:0/2400546910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d75600 0x55a4b9d903f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2a9d9b640 1 -- 172.21.15.47:0/2400546910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d473b0 msgr2=0x55a4b9d91b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2a9d9b640 1 --2- 172.21.15.47:0/2400546910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d473b0 0x55a4b9d91b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2a9d9b640 1 -- 172.21.15.47:0/2400546910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4b9c74470 con 0x55a4b9d73830 2024-03-23T09:48:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2a9d9b640 1 --2- 172.21.15.47:0/2400546910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 0x55a4b9d92090 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0000c00 tx=0x7fa2a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0e5ff543436e61 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2897fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2897fa640 1 -- 172.21.15.47:0/2400546910 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0044870 con 0x55a4b9d73830 2024-03-23T09:48:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2897fa640 1 -- 172.21.15.47:0/2400546910 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa2a003f040 con 0x55a4b9d73830 2024-03-23T09:48:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2400546910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2a400ce40 con 0x55a4b9d73830 2024-03-23T09:48:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2897fa640 1 -- 172.21.15.47:0/2400546910 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2a00449d0 con 0x55a4b9d73830 2024-03-23T09:48:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2b3b7c740 10 client.8019 fetch_fsmap learned FSMap version 53 2024-03-23T09:48:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2400546910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55a4b9c4d5c0 con 0x55a4b9d73830 2024-03-23T09:48:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2897fa640 1 -- 172.21.15.47:0/2400546910 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a400ce40 con 0x55a4b9d73830 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2897fa640 1 -- 172.21.15.47:0/2400546910 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 53) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa2a0044b30 con 0x55a4b9d73830 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2b3b7c740 10 client.8019 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2b3b7c740 10 client.8019 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2b3b7c740 20 client.8019 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.483+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/2400546910 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55a4b9d68c40 0x55a4b9d90930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2b3b7c740 4 client.8019 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2400546910 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a4b99a04e0 con 0x55a4b9d68c40 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/2400546910 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55a4b9d68c40 0x55a4b9d90930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2897fa640 10 client.8019 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2aa59c640 1 --2- 172.21.15.47:0/2400546910 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55a4b9d68c40 0x55a4b9d90930 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab5e7a3b61de84ac server_cookie=c25e4abc6f832634 in_seq=0 out_seq=0 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2897fa640 1 -- 172.21.15.47:0/2400546910 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a4b99a04e0 con 0x55a4b9d68c40 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2897fa640 10 client.8019 handle_command_reply: tid=0 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2b3b7c740 1 client.8019 shutdown 2024-03-23T09:48:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2400546910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 msgr2=0x55a4b9d92090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/2400546910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 0x55a4b9d92090 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0000c00 tx=0x7fa2a0004730 comp rx=0 tx=0).stop 2024-03-23T09:48:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2400546910 shutdown_connections 2024-03-23T09:48:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/2400546910 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x55a4b9d68c40 0x55a4b9d90930 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/2400546910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a4b9d75600 0x55a4b9d903f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/2400546910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4b9d73830 0x55a4b9d92090 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2897fa640 0 client.8019 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2b3b7c740 1 --2- 172.21.15.47:0/2400546910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4b9d473b0 0x55a4b9d91b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2400546910 >> 172.21.15.47:0/2400546910 conn(0x55a4b9d43850 msgr2=0x55a4b9d77590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2400546910 shutdown_connections 2024-03-23T09:48:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.484+0000 7fa2b3b7c740 1 -- 172.21.15.47:0/2400546910 wait complete. 2024-03-23T09:48:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.485+0000 7fa2b3b7c740 20 client.8019 trim_cache size 0 max 16384 2024-03-23T09:48:37.493 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:48:37.493 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-23T09:48:37.493 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-23T09:48:37.493 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:48:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.485+0000 7fa2b0f2e640 1 -- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa28c06ac00 msgr2=0x7fa28c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.485+0000 7fa2b0f2e640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa28c06ac00 0x7fa28c06d0c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400d620 tx=0x7fa2a4007810 comp rx=0 tx=0).stop 2024-03-23T09:48:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.485+0000 7fa2b0f2e640 1 -- 172.21.15.47:0/2002872951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2ac15ce10 msgr2=0x7fa2ac15dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.485+0000 7fa2b0f2e640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2ac15ce10 0x7fa2ac15dca0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00c7f0 tx=0x7fa29c00ccb0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.485+0000 7fa2b0f2e640 1 -- 172.21.15.47:0/2002872951 shutdown_connections 2024-03-23T09:48:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.485+0000 7fa2b0f2e640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa28c06ac00 0x7fa28c06d0c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.485+0000 7fa2b0f2e640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2ac15ce10 0x7fa2ac15dca0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.485+0000 7fa2b0f2e640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2ac15ca30 0x7fa2ac15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.485+0000 7fa2b0f2e640 1 --2- 172.21.15.47:0/2002872951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2ac0b85d0 0x7fa2ac15d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.485+0000 7fa2b0f2e640 1 -- 172.21.15.47:0/2002872951 >> 172.21.15.47:0/2002872951 conn(0x7fa2ac06b160 msgr2=0x7fa2ac15a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.485+0000 7fa2b0f2e640 1 -- 172.21.15.47:0/2002872951 shutdown_connections 2024-03-23T09:48:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.485+0000 7fa2b0f2e640 1 -- 172.21.15.47:0/2002872951 wait complete. 2024-03-23T09:48:37.496 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T09:48:37.496 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:48:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.597+0000 7fe82eec4640 1 Processor -- start 2024-03-23T09:48:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.597+0000 7fe82eec4640 1 -- start start 2024-03-23T09:48:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.598+0000 7fe82eec4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe8280b1680 0x7fe8280b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.598+0000 7fe82eec4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe82806bc50 0x7fe8280ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.598+0000 7fe82eec4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8280b12c0 con 0x7fe828098a00 2024-03-23T09:48:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.598+0000 7fe82eec4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82806ae90 con 0x7fe8280b1680 2024-03-23T09:48:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.598+0000 7fe82eec4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82806b170 con 0x7fe82806bc50 2024-03-23T09:48:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.598+0000 7fe827fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe8280b1680 0x7fe8280b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.598+0000 7fe827fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe8280b1680 0x7fe8280b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52558/0 (socket says 172.21.15.47:52558) 2024-03-23T09:48:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.598+0000 7fe827fff640 1 -- 172.21.15.47:0/2121254380 learned_addr learned my addr 172.21.15.47:0/2121254380 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:37.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.598+0000 7fe82d461640 1 --2- 172.21.15.47:0/2121254380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe82806bc50 0x7fe8280ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.598+0000 7fe827fff640 1 -- 172.21.15.47:0/2121254380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe82806bc50 msgr2=0x7fe8280ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.598+0000 7fe827fff640 1 --2- 172.21.15.47:0/2121254380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe82806bc50 0x7fe8280ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.598+0000 7fe827fff640 1 -- 172.21.15.47:0/2121254380 >> v1:172.21.15.175:6789/0 conn(0x7fe828098a00 legacy=0x7fe828098de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.598+0000 7fe827fff640 1 -- 172.21.15.47:0/2121254380 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82806b450 con 0x7fe8280b1680 2024-03-23T09:48:37.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.599+0000 7fe827fff640 1 --2- 172.21.15.47:0/2121254380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8280b1680 0x7fe8280b0950 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe814009870 tx=0x7fe81402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998599433e0e4108 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.599+0000 7fe8277fe640 1 -- 172.21.15.47:0/2121254380 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814031640 con 0x7fe8280b1680 2024-03-23T09:48:37.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.599+0000 7fe8277fe640 1 -- 172.21.15.47:0/2121254380 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8140317a0 con 0x7fe8280b1680 2024-03-23T09:48:37.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.599+0000 7fe8277fe640 1 -- 172.21.15.47:0/2121254380 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814031b30 con 0x7fe8280b1680 2024-03-23T09:48:37.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.600+0000 7fe82eec4640 1 -- 172.21.15.47:0/2121254380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8280b1680 msgr2=0x7fe8280b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.600+0000 7fe82eec4640 1 --2- 172.21.15.47:0/2121254380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8280b1680 0x7fe8280b0950 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe814009870 tx=0x7fe81402b8f0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.600+0000 7fe82eec4640 1 -- 172.21.15.47:0/2121254380 shutdown_connections 2024-03-23T09:48:37.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.600+0000 7fe82eec4640 1 --2- 172.21.15.47:0/2121254380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe82806bc50 0x7fe8280ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.600+0000 7fe82eec4640 1 --2- 172.21.15.47:0/2121254380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8280b1680 0x7fe8280b0950 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.600+0000 7fe82eec4640 1 -- 172.21.15.47:0/2121254380 >> 172.21.15.47:0/2121254380 conn(0x7fe8280b2140 msgr2=0x7fe8280b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.600+0000 7fe82eec4640 1 -- 172.21.15.47:0/2121254380 shutdown_connections 2024-03-23T09:48:37.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.600+0000 7fe82eec4640 1 -- 172.21.15.47:0/2121254380 wait complete. 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.601+0000 7fe82eec4640 1 Processor -- start 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.601+0000 7fe82eec4640 1 -- start start 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe82eec4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe82806bc50 0x7fe8280b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe82eec4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe828098a00 0x7fe828154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe82eec4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8280b1680 0x7fe8280ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe82eec4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280b4040 con 0x7fe828098a00 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe82eec4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280b4320 con 0x7fe82806bc50 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe82eec4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280b4600 con 0x7fe8280b1680 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe827fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe828098a00 0x7fe828154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe827fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe828098a00 0x7fe828154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52574/0 (socket says 172.21.15.47:52574) 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe827fff640 1 -- 172.21.15.47:0/607227022 learned_addr learned my addr 172.21.15.47:0/607227022 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe82cc60640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe82806bc50 0x7fe8280b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe82d461640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8280b1680 0x7fe8280ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe827fff640 1 -- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8280b1680 msgr2=0x7fe8280ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe827fff640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8280b1680 0x7fe8280ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe827fff640 1 -- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe82806bc50 msgr2=0x7fe8280b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe827fff640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe82806bc50 0x7fe8280b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe827fff640 1 -- 172.21.15.47:0/607227022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8140094d0 con 0x7fe828098a00 2024-03-23T09:48:37.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe82d461640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8280b1680 0x7fe8280ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:37.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe827fff640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe828098a00 0x7fe828154890 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fe814000c00 tx=0x7fe8140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.602+0000 7fe8257fa640 1 -- 172.21.15.47:0/607227022 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81403a930 con 0x7fe828098a00 2024-03-23T09:48:37.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.603+0000 7fe8257fa640 1 -- 172.21.15.47:0/607227022 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81403aa90 con 0x7fe828098a00 2024-03-23T09:48:37.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.603+0000 7fe82eec4640 1 -- 172.21.15.47:0/607227022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8280b48e0 con 0x7fe828098a00 2024-03-23T09:48:37.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.603+0000 7fe8257fa640 1 -- 172.21.15.47:0/607227022 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81403ae40 con 0x7fe828098a00 2024-03-23T09:48:37.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.603+0000 7fe82eec4640 1 -- 172.21.15.47:0/607227022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe828154dd0 con 0x7fe828098a00 2024-03-23T09:48:37.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.604+0000 7fe82eec4640 1 -- 172.21.15.47:0/607227022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8280b1d20 con 0x7fe828098a00 2024-03-23T09:48:37.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.604+0000 7fe8257fa640 1 -- 172.21.15.47:0/607227022 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe81402f050 con 0x7fe828098a00 2024-03-23T09:48:37.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.604+0000 7fe8257fa640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe80c06ac00 0x7fe80c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.604+0000 7fe8257fa640 1 -- 172.21.15.47:0/607227022 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe8140ab110 con 0x7fe828098a00 2024-03-23T09:48:37.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.605+0000 7fe82cc60640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe80c06ac00 0x7fe80c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.605+0000 7fe82cc60640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe80c06ac00 0x7fe80c06d0c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe81c000f60 tx=0x7fe81c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.609+0000 7fe8257fa640 1 -- 172.21.15.47:0/607227022 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe8140762f0 con 0x7fe828098a00 2024-03-23T09:48:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.824+0000 7fe82eec4640 1 -- 172.21.15.47:0/607227022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe828066be0 con 0x7fe828098a00 2024-03-23T09:48:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.825+0000 7fe8257fa640 1 -- 172.21.15.47:0/607227022 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fe81402d040 con 0x7fe828098a00 2024-03-23T09:48:37.825 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:48:37.825 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:47:39.843531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7776":{"gid":7776,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":6,"addr":"172.21.15.47:6835/4278516621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4278516621},{"type":"v1","addr":"172.21.15.47:6835","nonce":4278516621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:48:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 53 2024-03-23T09:48:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.827+0000 7fe82eec4640 1 -- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe80c06ac00 msgr2=0x7fe80c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.827+0000 7fe82eec4640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe80c06ac00 0x7fe80c06d0c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe81c000f60 tx=0x7fe81c009040 comp rx=0 tx=0).stop 2024-03-23T09:48:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.827+0000 7fe82eec4640 1 -- 172.21.15.47:0/607227022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe828098a00 msgr2=0x7fe828154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.827+0000 7fe82eec4640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe828098a00 0x7fe828154890 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fe814000c00 tx=0x7fe8140046c0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.827+0000 7fe82eec4640 1 -- 172.21.15.47:0/607227022 shutdown_connections 2024-03-23T09:48:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.827+0000 7fe82eec4640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe80c06ac00 0x7fe80c06d0c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.827+0000 7fe82eec4640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8280b1680 0x7fe8280ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.827+0000 7fe82eec4640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe828098a00 0x7fe828154890 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.827+0000 7fe82eec4640 1 --2- 172.21.15.47:0/607227022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe82806bc50 0x7fe8280b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.827+0000 7fe82eec4640 1 -- 172.21.15.47:0/607227022 >> 172.21.15.47:0/607227022 conn(0x7fe8280b2140 msgr2=0x7fe82806f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.827+0000 7fe82eec4640 1 -- 172.21.15.47:0/607227022 shutdown_connections 2024-03-23T09:48:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.827+0000 7fe82eec4640 1 -- 172.21.15.47:0/607227022 wait complete. 2024-03-23T09:48:37.838 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-23T09:48:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.940+0000 7fcb9cd63640 1 Processor -- start 2024-03-23T09:48:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.940+0000 7fcb9cd63640 1 -- start start 2024-03-23T09:48:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.940+0000 7fcb9cd63640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb9815ca30 0x7fcb98160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.940+0000 7fcb9cd63640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcb980cde40 0x7fcb980d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.940+0000 7fcb9cd63640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb98161600 con 0x7fcb9815ce10 2024-03-23T09:48:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.940+0000 7fcb9cd63640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb98161040 con 0x7fcb980cde40 2024-03-23T09:48:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.940+0000 7fcb9cd63640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb98161320 con 0x7fcb9815ca30 2024-03-23T09:48:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.941+0000 7fcb9659c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb9815ca30 0x7fcb98160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.941+0000 7fcb96d9d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcb980cde40 0x7fcb980d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.941+0000 7fcb96d9d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcb980cde40 0x7fcb980d15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:37890/0 (socket says 172.21.15.47:37890) 2024-03-23T09:48:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.941+0000 7fcb8dd9b640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fcb9815ce10 0x7fcb980cd730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:51416/0 (socket says 172.21.15.47:51416) 2024-03-23T09:48:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.941+0000 7fcb9659c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb9815ca30 0x7fcb98160740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:58030/0 (socket says 172.21.15.47:58030) 2024-03-23T09:48:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.941+0000 7fcb8dd9b640 1 -- 172.21.15.47:0/2015899153 learned_addr learned my addr 172.21.15.47:0/2015899153 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.941+0000 7fcb9659c640 1 -- 172.21.15.47:0/2015899153 >> v2:172.21.15.175:3300/0 conn(0x7fcb980cde40 msgr2=0x7fcb980d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.941+0000 7fcb9659c640 1 --2- 172.21.15.47:0/2015899153 >> v2:172.21.15.175:3300/0 conn(0x7fcb980cde40 0x7fcb980d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.941+0000 7fcb9659c640 1 -- 172.21.15.47:0/2015899153 >> v1:172.21.15.175:6789/0 conn(0x7fcb9815ce10 legacy=0x7fcb980cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.941+0000 7fcb9659c640 1 -- 172.21.15.47:0/2015899153 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb980d0dd0 con 0x7fcb9815ca30 2024-03-23T09:48:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.942+0000 7fcb9659c640 1 --2- 172.21.15.47:0/2015899153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb9815ca30 0x7fcb98160740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcb880097a0 tx=0x7fcb8802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=276f41c554d17db1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.942+0000 7fcb95d9b640 1 -- 172.21.15.47:0/2015899153 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88031830 con 0x7fcb9815ca30 2024-03-23T09:48:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.942+0000 7fcb95d9b640 1 -- 172.21.15.47:0/2015899153 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb88031990 con 0x7fcb9815ca30 2024-03-23T09:48:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.942+0000 7fcb95d9b640 1 -- 172.21.15.47:0/2015899153 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88031d20 con 0x7fcb9815ca30 2024-03-23T09:48:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.943+0000 7fcb9cd63640 1 -- 172.21.15.47:0/2015899153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb9815ca30 msgr2=0x7fcb98160740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.943+0000 7fcb9cd63640 1 --2- 172.21.15.47:0/2015899153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb9815ca30 0x7fcb98160740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcb880097a0 tx=0x7fcb8802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.943+0000 7fcb9cd63640 1 -- 172.21.15.47:0/2015899153 shutdown_connections 2024-03-23T09:48:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.943+0000 7fcb9cd63640 1 --2- 172.21.15.47:0/2015899153 >> v2:172.21.15.175:3300/0 conn(0x7fcb980cde40 0x7fcb980d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.943+0000 7fcb9cd63640 1 --2- 172.21.15.47:0/2015899153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb9815ca30 0x7fcb98160740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.943+0000 7fcb9cd63640 1 -- 172.21.15.47:0/2015899153 >> 172.21.15.47:0/2015899153 conn(0x7fcb9806b160 msgr2=0x7fcb9806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.943+0000 7fcb9cd63640 1 -- 172.21.15.47:0/2015899153 shutdown_connections 2024-03-23T09:48:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.943+0000 7fcb9cd63640 1 -- 172.21.15.47:0/2015899153 wait complete. 2024-03-23T09:48:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.944+0000 7fcb9cd63640 1 Processor -- start 2024-03-23T09:48:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.944+0000 7fcb9cd63640 1 -- start start 2024-03-23T09:48:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb9cd63640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb980cde40 0x7fcb9814cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb9cd63640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb9815ca30 0x7fcb9815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb9cd63640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9815ce10 0x7fcb980b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb9cd63640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb980c72b0 con 0x7fcb9815ce10 2024-03-23T09:48:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb9cd63640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb980c8310 con 0x7fcb9815ca30 2024-03-23T09:48:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb9cd63640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb980c7550 con 0x7fcb980cde40 2024-03-23T09:48:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb96d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9815ce10 0x7fcb980b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb96d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9815ce10 0x7fcb980b6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52582/0 (socket says 172.21.15.47:52582) 2024-03-23T09:48:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb96d9d640 1 -- 172.21.15.47:0/1462822388 learned_addr learned my addr 172.21.15.47:0/1462822388 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb9659c640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb980cde40 0x7fcb9814cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb8dd9b640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb9815ca30 0x7fcb9815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb96d9d640 1 -- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb980cde40 msgr2=0x7fcb9814cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb96d9d640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb980cde40 0x7fcb9814cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb96d9d640 1 -- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb9815ca30 msgr2=0x7fcb9815f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb96d9d640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb9815ca30 0x7fcb9815f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb96d9d640 1 -- 172.21.15.47:0/1462822388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb88009400 con 0x7fcb9815ce10 2024-03-23T09:48:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.945+0000 7fcb8dd9b640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb9815ca30 0x7fcb9815f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.946+0000 7fcb96d9d640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9815ce10 0x7fcb980b6e70 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000efc0 tx=0x7fcb9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.946+0000 7fcb8f7fe640 1 -- 172.21.15.47:0/1462822388 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90014480 con 0x7fcb9815ce10 2024-03-23T09:48:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.946+0000 7fcb8f7fe640 1 -- 172.21.15.47:0/1462822388 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9000f040 con 0x7fcb9815ce10 2024-03-23T09:48:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.946+0000 7fcb9cd63640 1 -- 172.21.15.47:0/1462822388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb980c7830 con 0x7fcb9815ce10 2024-03-23T09:48:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.946+0000 7fcb8f7fe640 1 -- 172.21.15.47:0/1462822388 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90014820 con 0x7fcb9815ce10 2024-03-23T09:48:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.946+0000 7fcb9cd63640 1 -- 172.21.15.47:0/1462822388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb980c7df0 con 0x7fcb9815ce10 2024-03-23T09:48:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.948+0000 7fcb8f7fe640 1 -- 172.21.15.47:0/1462822388 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcb90014980 con 0x7fcb9815ce10 2024-03-23T09:48:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.948+0000 7fcb8f7fe640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcb7806ac00 0x7fcb7806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.948+0000 7fcb8f7fe640 1 -- 172.21.15.47:0/1462822388 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fcb90089fe0 con 0x7fcb9815ce10 2024-03-23T09:48:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.948+0000 7fcb9659c640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcb7806ac00 0x7fcb7806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.949+0000 7fcb9659c640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcb7806ac00 0x7fcb7806d0c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcb88036000 tx=0x7fcb8803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.949+0000 7fcb9f9b5740 1 Processor -- start 2024-03-23T09:48:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.949+0000 7fcb9f9b5740 1 -- start start 2024-03-23T09:48:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb9f9b5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c56e6b0 0x559c7c56ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb9f9b5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c53e6d0 0x559c7c56b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb9f9b5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 0x559c7c56be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb9f9b5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7c4af6a0 con 0x559c7c542230 2024-03-23T09:48:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb9f9b5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7c4acc70 con 0x559c7c53e6d0 2024-03-23T09:48:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb9f9b5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c7c4a8560 con 0x559c7c56e6b0 2024-03-23T09:48:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb96d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 0x559c7c56be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb96d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 0x559c7c56be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52588/0 (socket says 172.21.15.47:52588) 2024-03-23T09:48:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb96d9d640 1 -- 172.21.15.47:0/4160164684 learned_addr learned my addr 172.21.15.47:0/4160164684 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb8dd9b640 1 --2- 172.21.15.47:0/4160164684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c53e6d0 0x559c7c56b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb9659c640 1 --2- 172.21.15.47:0/4160164684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c56e6b0 0x559c7c56ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb96d9d640 1 -- 172.21.15.47:0/4160164684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c56e6b0 msgr2=0x559c7c56ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb96d9d640 1 --2- 172.21.15.47:0/4160164684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c56e6b0 0x559c7c56ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb96d9d640 1 -- 172.21.15.47:0/4160164684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c53e6d0 msgr2=0x559c7c56b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb96d9d640 1 --2- 172.21.15.47:0/4160164684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c53e6d0 0x559c7c56b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb96d9d640 1 -- 172.21.15.47:0/4160164684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c7c46fd50 con 0x559c7c542230 2024-03-23T09:48:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.950+0000 7fcb96d9d640 1 --2- 172.21.15.47:0/4160164684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 0x559c7c56be60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fcb90060cb0 tx=0x7fcb900595b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9688ee759903d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.951+0000 7fcb77fff640 1 -- 172.21.15.47:0/4160164684 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb900611e0 con 0x559c7c542230 2024-03-23T09:48:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.951+0000 7fcb77fff640 1 -- 172.21.15.47:0/4160164684 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb90056f30 con 0x559c7c542230 2024-03-23T09:48:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.951+0000 7fcb77fff640 1 -- 172.21.15.47:0/4160164684 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90055500 con 0x559c7c542230 2024-03-23T09:48:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.951+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/4160164684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 msgr2=0x559c7c56be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.951+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/4160164684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 0x559c7c56be60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fcb90060cb0 tx=0x7fcb900595b0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.951+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/4160164684 shutdown_connections 2024-03-23T09:48:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.951+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/4160164684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 0x559c7c56be60 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.951+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/4160164684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c53e6d0 0x559c7c56b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.951+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/4160164684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c56e6b0 0x559c7c56ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.951+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/4160164684 >> 172.21.15.47:0/4160164684 conn(0x559c7c540940 msgr2=0x559c7c5894c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.951+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/4160164684 shutdown_connections 2024-03-23T09:48:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.951+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/4160164684 wait complete. 2024-03-23T09:48:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.952+0000 7fcb9f9b5740 1 -- start start 2024-03-23T09:48:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.952+0000 7fcb9f9b5740 1 Processor -- start 2024-03-23T09:48:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9f9b5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 0x559c7c55bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9f9b5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c56e6b0 0x559c7c570fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9f9b5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c571cf0 0x559c7c55c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9f9b5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7c1813f0 con 0x559c7c542230 2024-03-23T09:48:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9f9b5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7c2313d0 con 0x559c7c56e6b0 2024-03-23T09:48:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9f9b5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c7c42ba10 con 0x559c7c571cf0 2024-03-23T09:48:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9f9b5740 10 client.0 _ll_register_callbacks cb 0x559c7c4a1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 0x559c7c55bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 0x559c7c55bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52602/0 (socket says 172.21.15.47:52602) 2024-03-23T09:48:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9659c640 1 -- 172.21.15.47:0/3040401564 learned_addr learned my addr 172.21.15.47:0/3040401564 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb8dd9b640 1 --2- 172.21.15.47:0/3040401564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c56e6b0 0x559c7c570fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb96d9d640 1 --2- 172.21.15.47:0/3040401564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c571cf0 0x559c7c55c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9659c640 1 -- 172.21.15.47:0/3040401564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c571cf0 msgr2=0x559c7c55c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9659c640 1 --2- 172.21.15.47:0/3040401564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c571cf0 0x559c7c55c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9659c640 1 -- 172.21.15.47:0/3040401564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c56e6b0 msgr2=0x559c7c570fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9659c640 1 --2- 172.21.15.47:0/3040401564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c56e6b0 0x559c7c570fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9659c640 1 -- 172.21.15.47:0/3040401564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb980c7df0 con 0x559c7c542230 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb9659c640 1 --2- 172.21.15.47:0/3040401564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 0x559c7c55bb70 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fcb8802bee0 tx=0x7fcb88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5402ad97a7d1a9a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.953+0000 7fcb757fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb757fa640 1 -- 172.21.15.47:0/3040401564 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8804a070 con 0x559c7c542230 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb757fa640 1 -- 172.21.15.47:0/3040401564 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fcb880458e0 con 0x559c7c542230 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/3040401564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c7c4d1150 con 0x559c7c542230 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb757fa640 1 -- 172.21.15.47:0/3040401564 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb8803f040 con 0x559c7c542230 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb9f9b5740 10 client.8034 fetch_fsmap learned FSMap version 53 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/3040401564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x559c7c46a760 con 0x559c7c542230 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb757fa640 1 -- 172.21.15.47:0/3040401564 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559c7c4d1150 con 0x559c7c542230 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb757fa640 1 -- 172.21.15.47:0/3040401564 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 53) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcb88045aa0 con 0x559c7c542230 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb9f9b5740 10 client.8034 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb9f9b5740 10 client.8034 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb9f9b5740 20 client.8034 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/3040401564 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x559c7c5a47d0 0x559c7c5a6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb9f9b5740 4 client.8034 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:48:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/3040401564 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c7c4b3650 con 0x559c7c5a47d0 2024-03-23T09:48:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb8dd9b640 1 --2- 172.21.15.47:0/3040401564 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x559c7c5a47d0 0x559c7c5a6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb8dd9b640 1 --2- 172.21.15.47:0/3040401564 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x559c7c5a47d0 0x559c7c5a6bc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=216fd436e3ebe0d9 server_cookie=d1effce9fdce4179 in_seq=0 out_seq=0 2024-03-23T09:48:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.954+0000 7fcb757fa640 10 client.8034 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb757fa640 1 -- 172.21.15.47:0/3040401564 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559c7c4b3650 con 0x559c7c5a47d0 2024-03-23T09:48:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb757fa640 10 client.8034 handle_command_reply: tid=0 2024-03-23T09:48:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb9f9b5740 1 client.8034 shutdown 2024-03-23T09:48:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/3040401564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 msgr2=0x559c7c55bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/3040401564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 0x559c7c55bb70 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fcb8802bee0 tx=0x7fcb88004730 comp rx=0 tx=0).stop 2024-03-23T09:48:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/3040401564 shutdown_connections 2024-03-23T09:48:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/3040401564 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x559c7c5a47d0 0x559c7c5a6bc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/3040401564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c571cf0 0x559c7c55c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/3040401564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c56e6b0 0x559c7c570fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/3040401564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c542230 0x559c7c55bb70 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb757fa640 0 client.8034 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/3040401564 >> 172.21.15.47:0/3040401564 conn(0x559c7c53e6d0 msgr2=0x559c7c542ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/3040401564 shutdown_connections 2024-03-23T09:48:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/3040401564 wait complete. 2024-03-23T09:48:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.955+0000 7fcb9f9b5740 20 client.8034 trim_cache size 0 max 16384 2024-03-23T09:48:37.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.967+0000 7fcb9f9b5740 1 Processor -- start 2024-03-23T09:48:37.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.967+0000 7fcb9f9b5740 1 -- start start 2024-03-23T09:48:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.967+0000 7fcb9f9b5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c542230 0x559c7c565090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.967+0000 7fcb9f9b5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c56e6b0 0x559c7c53d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.967+0000 7fcb9f9b5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c571cf0 0x559c7c56a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.967+0000 7fcb9f9b5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7c543280 con 0x559c7c571cf0 2024-03-23T09:48:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.967+0000 7fcb9f9b5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7c56f9e0 con 0x559c7c542230 2024-03-23T09:48:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.967+0000 7fcb9f9b5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c7c4ddd60 con 0x559c7c56e6b0 2024-03-23T09:48:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.967+0000 7fcb96d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c571cf0 0x559c7c56a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.967+0000 7fcb96d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c571cf0 0x559c7c56a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52612/0 (socket says 172.21.15.47:52612) 2024-03-23T09:48:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.967+0000 7fcb96d9d640 1 -- 172.21.15.47:0/695055339 learned_addr learned my addr 172.21.15.47:0/695055339 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.968+0000 7fcb96d9d640 1 -- 172.21.15.47:0/695055339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c56e6b0 msgr2=0x559c7c53d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.968+0000 7fcb96d9d640 1 --2- 172.21.15.47:0/695055339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c56e6b0 0x559c7c53d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.968+0000 7fcb96d9d640 1 -- 172.21.15.47:0/695055339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c542230 msgr2=0x559c7c565090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.968+0000 7fcb9659c640 1 --2- 172.21.15.47:0/695055339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c542230 0x559c7c565090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.968+0000 7fcb96d9d640 1 --2- 172.21.15.47:0/695055339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c542230 0x559c7c565090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.968+0000 7fcb96d9d640 1 -- 172.21.15.47:0/695055339 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c7c448ad0 con 0x559c7c571cf0 2024-03-23T09:48:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.968+0000 7fcb96d9d640 1 --2- 172.21.15.47:0/695055339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c571cf0 0x559c7c56a4b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c990 tx=0x7fcb9005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08ef476106dbdfe server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.968+0000 7fcb8cd99640 1 -- 172.21.15.47:0/695055339 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb900881d0 con 0x559c7c571cf0 2024-03-23T09:48:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.968+0000 7fcb8cd99640 1 -- 172.21.15.47:0/695055339 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb900611e0 con 0x559c7c571cf0 2024-03-23T09:48:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.968+0000 7fcb8cd99640 1 -- 172.21.15.47:0/695055339 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90055500 con 0x559c7c571cf0 2024-03-23T09:48:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.969+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/695055339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c571cf0 msgr2=0x559c7c56a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.969+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/695055339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c571cf0 0x559c7c56a4b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c990 tx=0x7fcb9005b000 comp rx=0 tx=0).stop 2024-03-23T09:48:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.969+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/695055339 shutdown_connections 2024-03-23T09:48:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.969+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/695055339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c571cf0 0x559c7c56a4b0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.969+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/695055339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c56e6b0 0x559c7c53d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.969+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/695055339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c542230 0x559c7c565090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.969+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/695055339 >> 172.21.15.47:0/695055339 conn(0x559c7c53e6d0 msgr2=0x559c7c566c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.969+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/695055339 shutdown_connections 2024-03-23T09:48:37.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.969+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/695055339 wait complete. 2024-03-23T09:48:37.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.969+0000 7fcb9f9b5740 1 -- start start 2024-03-23T09:48:37.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.969+0000 7fcb9f9b5740 1 Processor -- start 2024-03-23T09:48:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.970+0000 7fcb9f9b5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c542230 0x559c7c588700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.970+0000 7fcb9f9b5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c56e6b0 0x559c7c569a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.970+0000 7fcb9f9b5740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c571cf0 0x559c7c565cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.970+0000 7fcb9f9b5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7c569650 con 0x559c7c56e6b0 2024-03-23T09:48:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.970+0000 7fcb9f9b5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7c56b680 con 0x559c7c542230 2024-03-23T09:48:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.970+0000 7fcb9f9b5740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c7c589180 con 0x559c7c571cf0 2024-03-23T09:48:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.970+0000 7fcb9f9b5740 10 client.0 _ll_register_callbacks cb 0x7fcb88009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.970+0000 7fcb8dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c56e6b0 0x559c7c569a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.970+0000 7fcb8dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c56e6b0 0x559c7c569a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52620/0 (socket says 172.21.15.47:52620) 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.970+0000 7fcb8dd9b640 1 -- 172.21.15.47:0/1744658292 learned_addr learned my addr 172.21.15.47:0/1744658292 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.970+0000 7fcb9659c640 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c542230 0x559c7c588700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb96d9d640 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c571cf0 0x559c7c565cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb8dd9b640 1 -- 172.21.15.47:0/1744658292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c571cf0 msgr2=0x559c7c565cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb8dd9b640 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c571cf0 0x559c7c565cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb8dd9b640 1 -- 172.21.15.47:0/1744658292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c542230 msgr2=0x559c7c588700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb8dd9b640 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c542230 0x559c7c588700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb8dd9b640 1 -- 172.21.15.47:0/1744658292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c7c46fd50 con 0x559c7c56e6b0 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb96d9d640 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c571cf0 0x559c7c565cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb8dd9b640 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c56e6b0 0x559c7c569a70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fcb980c94b0 tx=0x7fcb800193a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb44f27b0a2d642 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb757fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb757fa640 1 -- 172.21.15.47:0/1744658292 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb80019d90 con 0x559c7c56e6b0 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb757fa640 1 -- 172.21.15.47:0/1744658292 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fcb80007710 con 0x559c7c56e6b0 2024-03-23T09:48:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/1744658292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb90056f30 con 0x559c7c56e6b0 2024-03-23T09:48:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb757fa640 1 -- 172.21.15.47:0/1744658292 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb80015440 con 0x559c7c56e6b0 2024-03-23T09:48:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb9f9b5740 10 client.8040 fetch_fsmap learned FSMap version 53 2024-03-23T09:48:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/1744658292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x559c7c46e5b0 con 0x559c7c56e6b0 2024-03-23T09:48:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb757fa640 1 -- 172.21.15.47:0/1744658292 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb90056f30 con 0x559c7c56e6b0 2024-03-23T09:48:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb757fa640 1 -- 172.21.15.47:0/1744658292 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 53) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcb80012070 con 0x559c7c56e6b0 2024-03-23T09:48:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.971+0000 7fcb9f9b5740 10 client.8040 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T09:48:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 10 client.8040 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:48:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 20 client.8040 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x559c7c5662e0 0x559c7c5eec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 4 client.8040 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-23T09:48:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/1744658292 --> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x559c7c1598d0 con 0x559c7c5662e0 2024-03-23T09:48:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9659c640 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x559c7c5662e0 0x559c7c5eec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb757fa640 10 client.8040 ms_handle_connect on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9659c640 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x559c7c5662e0 0x559c7c5eec40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48b6baf286b07ae server_cookie=3f45539543d06dea in_seq=0 out_seq=0 2024-03-23T09:48:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb757fa640 1 -- 172.21.15.47:0/1744658292 <== mds.0 v2:172.21.15.47:6834/4278516621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+144 (crc 0 0 0) 0x559c7c1598d0 con 0x559c7c5662e0 2024-03-23T09:48:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb757fa640 10 client.8040 handle_command_reply: tid=0 2024-03-23T09:48:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 1 client.8040 shutdown 2024-03-23T09:48:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/1744658292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c56e6b0 msgr2=0x559c7c569a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c56e6b0 0x559c7c569a70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fcb980c94b0 tx=0x7fcb800193a0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/1744658292 shutdown_connections 2024-03-23T09:48:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.47:6834/4278516621,v1:172.21.15.47:6835/4278516621] conn(0x559c7c5662e0 0x559c7c5eec40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559c7c571cf0 0x559c7c565cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c7c56e6b0 0x559c7c569a70 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 1 --2- 172.21.15.47:0/1744658292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c7c542230 0x559c7c588700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb757fa640 0 client.8040 ms_handle_reset on v2:172.21.15.47:6834/4278516621 2024-03-23T09:48:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.972+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/1744658292 >> 172.21.15.47:0/1744658292 conn(0x559c7c53e6d0 msgr2=0x559c7c565090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/1744658292 shutdown_connections 2024-03-23T09:48:37.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9f9b5740 1 -- 172.21.15.47:0/1744658292 wait complete. 2024-03-23T09:48:37.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9f9b5740 20 client.8040 trim_cache size 0 max 16384 2024-03-23T09:48:37.980 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:48:37.980 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-23T09:48:37.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:48:37.980 INFO:teuthology.orchestra.run.smithi047.stdout: "damage_type": "backtrace", 2024-03-23T09:48:37.980 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 1352690682, 2024-03-23T09:48:37.980 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 1099511627777, 2024-03-23T09:48:37.981 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/testdir/hardlink" 2024-03-23T09:48:37.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:48:37.981 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-23T09:48:37.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9cd63640 1 -- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcb7806ac00 msgr2=0x7fcb7806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9cd63640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcb7806ac00 0x7fcb7806d0c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcb88036000 tx=0x7fcb8803c040 comp rx=0 tx=0).stop 2024-03-23T09:48:37.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9cd63640 1 -- 172.21.15.47:0/1462822388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9815ce10 msgr2=0x7fcb980b6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:37.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9cd63640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9815ce10 0x7fcb980b6e70 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000efc0 tx=0x7fcb9000c490 comp rx=0 tx=0).stop 2024-03-23T09:48:37.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9cd63640 1 -- 172.21.15.47:0/1462822388 shutdown_connections 2024-03-23T09:48:37.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9cd63640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcb7806ac00 0x7fcb7806d0c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9cd63640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9815ce10 0x7fcb980b6e70 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9cd63640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb9815ca30 0x7fcb9815f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9cd63640 1 --2- 172.21.15.47:0/1462822388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb980cde40 0x7fcb9814cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:37.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9cd63640 1 -- 172.21.15.47:0/1462822388 >> 172.21.15.47:0/1462822388 conn(0x7fcb9806b160 msgr2=0x7fcb9814d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:37.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9cd63640 1 -- 172.21.15.47:0/1462822388 shutdown_connections 2024-03-23T09:48:37.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:37.973+0000 7fcb9cd63640 1 -- 172.21.15.47:0/1462822388 wait complete. 2024-03-23T09:48:37.985 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:48:37.985 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:48:38.001 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:48:38.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.141+0000 7f27b3d88640 1 Processor -- start 2024-03-23T09:48:38.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.142+0000 7f27b3d88640 1 -- start start 2024-03-23T09:48:38.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.142+0000 7f27b3d88640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27ac15a8d0 0x7f27ac0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.142+0000 7f27b3d88640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f27ac0cf360 0x7f27ac0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.142+0000 7f27b3d88640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ac0cc610 con 0x7f27ac0cb730 2024-03-23T09:48:38.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.142+0000 7f27b3d88640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ac0c41c0 con 0x7f27ac15a8d0 2024-03-23T09:48:38.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.142+0000 7f27b3d88640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ac0b0700 con 0x7f27ac0cf360 2024-03-23T09:48:38.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.142+0000 7f27b1b24640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27ac15a8d0 0x7f27ac0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.143+0000 7f27b1b24640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27ac15a8d0 0x7f27ac0cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52624/0 (socket says 172.21.15.47:52624) 2024-03-23T09:48:38.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.143+0000 7f27b1b24640 1 -- 172.21.15.47:0/2874344861 learned_addr learned my addr 172.21.15.47:0/2874344861 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:38.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.143+0000 7f27b2325640 1 --2- 172.21.15.47:0/2874344861 >> v2:172.21.15.175:3300/0 conn(0x7f27ac0cf360 0x7f27ac0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.143+0000 7f27b1b24640 1 -- 172.21.15.47:0/2874344861 >> v2:172.21.15.175:3300/0 conn(0x7f27ac0cf360 msgr2=0x7f27ac0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:38.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.143+0000 7f27b1b24640 1 --2- 172.21.15.47:0/2874344861 >> v2:172.21.15.175:3300/0 conn(0x7f27ac0cf360 0x7f27ac0b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.143+0000 7f27b1b24640 1 -- 172.21.15.47:0/2874344861 >> v1:172.21.15.175:6789/0 conn(0x7f27ac0cb730 legacy=0x7f27ac0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:38.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.143+0000 7f27b1b24640 1 -- 172.21.15.47:0/2874344861 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0cbd70 con 0x7f27ac15a8d0 2024-03-23T09:48:38.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.143+0000 7f27b2325640 1 --2- 172.21.15.47:0/2874344861 >> v2:172.21.15.175:3300/0 conn(0x7f27ac0cf360 0x7f27ac0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:38.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.144+0000 7f27b1b24640 1 --2- 172.21.15.47:0/2874344861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27ac15a8d0 0x7f27ac0cb1f0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0c4170 tx=0x7f27a402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40a64485c55e2eb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:38.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.144+0000 7f27b0b22640 1 -- 172.21.15.47:0/2874344861 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a40317e0 con 0x7f27ac15a8d0 2024-03-23T09:48:38.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.144+0000 7f27b0b22640 1 -- 172.21.15.47:0/2874344861 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a4031940 con 0x7f27ac15a8d0 2024-03-23T09:48:38.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.144+0000 7f27b0b22640 1 -- 172.21.15.47:0/2874344861 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a4031cd0 con 0x7f27ac15a8d0 2024-03-23T09:48:38.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.144+0000 7f27b3d88640 1 -- 172.21.15.47:0/2874344861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27ac15a8d0 msgr2=0x7f27ac0cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.144+0000 7f27b3d88640 1 --2- 172.21.15.47:0/2874344861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27ac15a8d0 0x7f27ac0cb1f0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0c4170 tx=0x7f27a402b880 comp rx=0 tx=0).stop 2024-03-23T09:48:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.144+0000 7f27b3d88640 1 -- 172.21.15.47:0/2874344861 shutdown_connections 2024-03-23T09:48:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.144+0000 7f27b3d88640 1 --2- 172.21.15.47:0/2874344861 >> v2:172.21.15.175:3300/0 conn(0x7f27ac0cf360 0x7f27ac0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.144+0000 7f27b3d88640 1 --2- 172.21.15.47:0/2874344861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27ac15a8d0 0x7f27ac0cb1f0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.144+0000 7f27b3d88640 1 -- 172.21.15.47:0/2874344861 >> 172.21.15.47:0/2874344861 conn(0x7f27ac0d0670 msgr2=0x7f27ac0d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.144+0000 7f27b3d88640 1 -- 172.21.15.47:0/2874344861 shutdown_connections 2024-03-23T09:48:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.144+0000 7f27b3d88640 1 -- 172.21.15.47:0/2874344861 wait complete. 2024-03-23T09:48:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.145+0000 7f27b3d88640 1 Processor -- start 2024-03-23T09:48:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.146+0000 7f27b3d88640 1 -- start start 2024-03-23T09:48:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.146+0000 7f27b3d88640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27ac0cb730 0x7f27ac15da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.146+0000 7f27b3d88640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27ac0cf360 0x7f27ac0b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.146+0000 7f27b3d88640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27ac15a8d0 0x7f27ac1495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.146+0000 7f27b3d88640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0afa70 con 0x7f27ac0cf360 2024-03-23T09:48:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.146+0000 7f27b3d88640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0aedf0 con 0x7f27ac15a8d0 2024-03-23T09:48:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.146+0000 7f27b3d88640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0af0d0 con 0x7f27ac0cb730 2024-03-23T09:48:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.146+0000 7f27b1323640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27ac0cf360 0x7f27ac0b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.146+0000 7f27b1323640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27ac0cf360 0x7f27ac0b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52628/0 (socket says 172.21.15.47:52628) 2024-03-23T09:48:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.146+0000 7f27b1323640 1 -- 172.21.15.47:0/3282916304 learned_addr learned my addr 172.21.15.47:0/3282916304 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.147+0000 7f27b2325640 1 --2- 172.21.15.47:0/3282916304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27ac15a8d0 0x7f27ac1495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.147+0000 7f27b1323640 1 -- 172.21.15.47:0/3282916304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27ac0cb730 msgr2=0x7f27ac15da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.147+0000 7f27b1323640 1 --2- 172.21.15.47:0/3282916304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27ac0cb730 0x7f27ac15da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.147+0000 7f27b1323640 1 -- 172.21.15.47:0/3282916304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27ac15a8d0 msgr2=0x7f27ac1495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.147+0000 7f27b1323640 1 --2- 172.21.15.47:0/3282916304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27ac15a8d0 0x7f27ac1495e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.147+0000 7f27b1323640 1 -- 172.21.15.47:0/3282916304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a40094d0 con 0x7f27ac0cf360 2024-03-23T09:48:38.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.147+0000 7f27b1323640 1 --2- 172.21.15.47:0/3282916304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27ac0cf360 0x7f27ac0b3220 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f27a000c430 tx=0x7f27a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:38.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.147+0000 7f27aa7fc640 1 -- 172.21.15.47:0/3282916304 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0011790 con 0x7f27ac0cf360 2024-03-23T09:48:38.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.147+0000 7f27aa7fc640 1 -- 172.21.15.47:0/3282916304 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a00118f0 con 0x7f27ac0cf360 2024-03-23T09:48:38.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.147+0000 7f27b3d88640 1 -- 172.21.15.47:0/3282916304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ac0af3b0 con 0x7f27ac0cf360 2024-03-23T09:48:38.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.147+0000 7f27aa7fc640 1 -- 172.21.15.47:0/3282916304 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0011ce0 con 0x7f27ac0cf360 2024-03-23T09:48:38.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.147+0000 7f27b3d88640 1 -- 172.21.15.47:0/3282916304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ac15df40 con 0x7f27ac0cf360 2024-03-23T09:48:38.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.148+0000 7f27b3d88640 1 -- 172.21.15.47:0/3282916304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ac0672f0 con 0x7f27ac0cf360 2024-03-23T09:48:38.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.149+0000 7f27aa7fc640 1 -- 172.21.15.47:0/3282916304 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f27a001c460 con 0x7f27ac0cf360 2024-03-23T09:48:38.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.149+0000 7f27aa7fc640 1 --2- 172.21.15.47:0/3282916304 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f279806ac00 0x7f279806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.149+0000 7f27aa7fc640 1 -- 172.21.15.47:0/3282916304 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f27a00893e0 con 0x7f27ac0cf360 2024-03-23T09:48:38.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.149+0000 7f27b1b24640 1 --2- 172.21.15.47:0/3282916304 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f279806ac00 0x7f279806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.150+0000 7f27b1b24640 1 --2- 172.21.15.47:0/3282916304 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f279806ac00 0x7f279806d0c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f27a4036000 tx=0x7f27a4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:38.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.153+0000 7f27aa7fc640 1 -- 172.21.15.47:0/3282916304 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f27a0054490 con 0x7f27ac0cf360 2024-03-23T09:48:38.323 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:48:38.326 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:48:38.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.333+0000 7f27b3d88640 1 -- 172.21.15.47:0/3282916304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27ac0b0320 con 0x7f27ac0cf360 2024-03-23T09:48:38.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.335+0000 7f27aa7fc640 1 -- 172.21.15.47:0/3282916304 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16319 (secure 0 0 0) 0x7f27a0057ab0 con 0x7f27ac0cf360 2024-03-23T09:48:38.335 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:48:38.335 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":88,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:47:52.879597+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:47:09.722391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-23T09:47:10.354054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.30","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/1927629460":"2024-03-24T09:47:28.505920+0000","172.21.15.47:6834/1927629460":"2024-03-24T09:47:28.505920+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:48:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.337+0000 7f27b3d88640 1 -- 172.21.15.47:0/3282916304 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f279806ac00 msgr2=0x7f279806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.337+0000 7f27b3d88640 1 --2- 172.21.15.47:0/3282916304 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f279806ac00 0x7f279806d0c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f27a4036000 tx=0x7f27a4030000 comp rx=0 tx=0).stop 2024-03-23T09:48:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.337+0000 7f27b3d88640 1 -- 172.21.15.47:0/3282916304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27ac0cf360 msgr2=0x7f27ac0b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.337+0000 7f27b3d88640 1 --2- 172.21.15.47:0/3282916304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27ac0cf360 0x7f27ac0b3220 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f27a000c430 tx=0x7f27a000c8f0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.337+0000 7f27b3d88640 1 -- 172.21.15.47:0/3282916304 shutdown_connections 2024-03-23T09:48:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.337+0000 7f27b3d88640 1 --2- 172.21.15.47:0/3282916304 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f279806ac00 0x7f279806d0c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.337+0000 7f27b3d88640 1 --2- 172.21.15.47:0/3282916304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27ac15a8d0 0x7f27ac1495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.337+0000 7f27b3d88640 1 --2- 172.21.15.47:0/3282916304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27ac0cf360 0x7f27ac0b3220 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.337+0000 7f27b3d88640 1 --2- 172.21.15.47:0/3282916304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27ac0cb730 0x7f27ac15da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.337+0000 7f27b3d88640 1 -- 172.21.15.47:0/3282916304 >> 172.21.15.47:0/3282916304 conn(0x7f27ac0d0670 msgr2=0x7f27ac157540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.337+0000 7f27b3d88640 1 -- 172.21.15.47:0/3282916304 shutdown_connections 2024-03-23T09:48:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.337+0000 7f27b3d88640 1 -- 172.21.15.47:0/3282916304 wait complete. 2024-03-23T09:48:38.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:47:09.722391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:47:10.354054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:48:38.348 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:48:38.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.532+0000 7f511a36d640 1 Processor -- start 2024-03-23T09:48:38.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.532+0000 7f511a36d640 1 -- start start 2024-03-23T09:48:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f511a36d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511415a940 0x7f51140ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f511a36d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f511415f300 0x7f51140cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f511a36d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511415e5d0 con 0x7f511406ee20 2024-03-23T09:48:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f511a36d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f511415eee0 con 0x7f511415f300 2024-03-23T09:48:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f511a36d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51140cf330 con 0x7f511415a940 2024-03-23T09:48:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f51137fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f511415f300 0x7f51140cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f511890a640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f511406ee20 0x7f51140cf5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47368/0 (socket says 172.21.15.47:47368) 2024-03-23T09:48:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f51137fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f511415f300 0x7f51140cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52640/0 (socket says 172.21.15.47:52640) 2024-03-23T09:48:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f511890a640 1 -- 172.21.15.47:0/3976565612 learned_addr learned my addr 172.21.15.47:0/3976565612 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f51137fe640 1 -- 172.21.15.47:0/3976565612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511415a940 msgr2=0x7f51140ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f5113fff640 1 --2- 172.21.15.47:0/3976565612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511415a940 0x7f51140ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f51137fe640 1 --2- 172.21.15.47:0/3976565612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511415a940 0x7f51140ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f5112ffd640 1 -- 172.21.15.47:0/3976565612 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847055454 0 0) 0x7f511415e5d0 con 0x7f511406ee20 2024-03-23T09:48:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f51137fe640 1 -- 172.21.15.47:0/3976565612 >> v1:172.21.15.47:6789/0 conn(0x7f511406ee20 legacy=0x7f51140cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.533+0000 7f51137fe640 1 -- 172.21.15.47:0/3976565612 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511406f680 con 0x7f511415f300 2024-03-23T09:48:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.534+0000 7f51137fe640 1 --2- 172.21.15.47:0/3976565612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f511415f300 0x7f51140cedf0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0097a0 tx=0x7f50fc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e5ecd2dca77aa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.534+0000 7f5112ffd640 1 -- 172.21.15.47:0/3976565612 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc02e070 con 0x7f511415f300 2024-03-23T09:48:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.534+0000 7f5112ffd640 1 -- 172.21.15.47:0/3976565612 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc0315a0 con 0x7f511415f300 2024-03-23T09:48:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.534+0000 7f5112ffd640 1 -- 172.21.15.47:0/3976565612 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc031930 con 0x7f511415f300 2024-03-23T09:48:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.535+0000 7f511a36d640 1 -- 172.21.15.47:0/3976565612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f511415f300 msgr2=0x7f51140cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.535+0000 7f511a36d640 1 --2- 172.21.15.47:0/3976565612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f511415f300 0x7f51140cedf0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0097a0 tx=0x7f50fc02b840 comp rx=0 tx=0).stop 2024-03-23T09:48:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.535+0000 7f511a36d640 1 -- 172.21.15.47:0/3976565612 shutdown_connections 2024-03-23T09:48:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.535+0000 7f511a36d640 1 --2- 172.21.15.47:0/3976565612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f511415f300 0x7f51140cedf0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.535+0000 7f511a36d640 1 --2- 172.21.15.47:0/3976565612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511415a940 0x7f51140ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.535+0000 7f511a36d640 1 -- 172.21.15.47:0/3976565612 >> 172.21.15.47:0/3976565612 conn(0x7f51140b13a0 msgr2=0x7f51140b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.535+0000 7f511a36d640 1 -- 172.21.15.47:0/3976565612 shutdown_connections 2024-03-23T09:48:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.535+0000 7f511a36d640 1 -- 172.21.15.47:0/3976565612 wait complete. 2024-03-23T09:48:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.536+0000 7f511a36d640 1 Processor -- start 2024-03-23T09:48:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.536+0000 7f511a36d640 1 -- start start 2024-03-23T09:48:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f511a36d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f511406ee20 0x7f51141efc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f511a36d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511415a940 0x7f5114154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f511a36d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f511415f300 0x7f5114150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f511a36d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511414fa40 con 0x7f511406ee20 2024-03-23T09:48:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f511a36d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511414fd20 con 0x7f511415f300 2024-03-23T09:48:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f511a36d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5114150000 con 0x7f511415a940 2024-03-23T09:48:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f5113fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f511406ee20 0x7f51141efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f5113fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f511406ee20 0x7f51141efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52656/0 (socket says 172.21.15.47:52656) 2024-03-23T09:48:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f5113fff640 1 -- 172.21.15.47:0/3092606056 learned_addr learned my addr 172.21.15.47:0/3092606056 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f5113fff640 1 -- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511415a940 msgr2=0x7f5114154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f51137fe640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511415a940 0x7f5114154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f511890a640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f511415f300 0x7f5114150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f5113fff640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511415a940 0x7f5114154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f5113fff640 1 -- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f511415f300 msgr2=0x7f5114150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f5113fff640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f511415f300 0x7f5114150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f5113fff640 1 -- 172.21.15.47:0/3092606056 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc009400 con 0x7f511406ee20 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f511890a640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f511415f300 0x7f5114150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f51137fe640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511415a940 0x7f5114154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.537+0000 7f5113fff640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f511406ee20 0x7f51141efc60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f510400c7f0 tx=0x7f510400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.538+0000 7f5110ff9640 1 -- 172.21.15.47:0/3092606056 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104004220 con 0x7f511406ee20 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.538+0000 7f5110ff9640 1 -- 172.21.15.47:0/3092606056 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5104004380 con 0x7f511406ee20 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.538+0000 7f511a36d640 1 -- 172.21.15.47:0/3092606056 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51141502e0 con 0x7f511406ee20 2024-03-23T09:48:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.538+0000 7f5110ff9640 1 -- 172.21.15.47:0/3092606056 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104007840 con 0x7f511406ee20 2024-03-23T09:48:38.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.538+0000 7f511a36d640 1 -- 172.21.15.47:0/3092606056 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51141508a0 con 0x7f511406ee20 2024-03-23T09:48:38.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.539+0000 7f511a36d640 1 -- 172.21.15.47:0/3092606056 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51140713c0 con 0x7f511406ee20 2024-03-23T09:48:38.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.539+0000 7f5110ff9640 1 -- 172.21.15.47:0/3092606056 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5104007bb0 con 0x7f511406ee20 2024-03-23T09:48:38.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.539+0000 7f5110ff9640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f50f406ac00 0x7f50f406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.540+0000 7f5110ff9640 1 -- 172.21.15.47:0/3092606056 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5104089240 con 0x7f511406ee20 2024-03-23T09:48:38.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.540+0000 7f51137fe640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f50f406ac00 0x7f50f406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.540+0000 7f51137fe640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f50f406ac00 0x7f50f406d0c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f50fc036000 tx=0x7f50fc03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:38.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.544+0000 7f5110ff9640 1 -- 172.21.15.47:0/3092606056 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5104054420 con 0x7f511406ee20 2024-03-23T09:48:38.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.726+0000 7f511a36d640 1 -- 172.21.15.47:0/3092606056 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5114150a90 con 0x7f511406ee20 2024-03-23T09:48:38.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.726+0000 7f5110ff9640 1 -- 172.21.15.47:0/3092606056 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5104007e60 con 0x7f511406ee20 2024-03-23T09:48:38.726 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:48:38.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.728+0000 7f511a36d640 1 -- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f50f406ac00 msgr2=0x7f50f406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.728+0000 7f511a36d640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f50f406ac00 0x7f50f406d0c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f50fc036000 tx=0x7f50fc03d040 comp rx=0 tx=0).stop 2024-03-23T09:48:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.728+0000 7f511a36d640 1 -- 172.21.15.47:0/3092606056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f511406ee20 msgr2=0x7f51141efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.728+0000 7f511a36d640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f511406ee20 0x7f51141efc60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f510400c7f0 tx=0x7f510400ccb0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.728+0000 7f511a36d640 1 -- 172.21.15.47:0/3092606056 shutdown_connections 2024-03-23T09:48:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.728+0000 7f511a36d640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f50f406ac00 0x7f50f406d0c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.728+0000 7f511a36d640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f511415f300 0x7f5114150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.728+0000 7f511a36d640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511415a940 0x7f5114154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.728+0000 7f511a36d640 1 --2- 172.21.15.47:0/3092606056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f511406ee20 0x7f51141efc60 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.728+0000 7f511a36d640 1 -- 172.21.15.47:0/3092606056 >> 172.21.15.47:0/3092606056 conn(0x7f51140b13a0 msgr2=0x7f51141598c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.728+0000 7f511a36d640 1 -- 172.21.15.47:0/3092606056 shutdown_connections 2024-03-23T09:48:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.728+0000 7f511a36d640 1 -- 172.21.15.47:0/3092606056 wait complete. 2024-03-23T09:48:38.739 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:48:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.843+0000 7f8e3725c640 1 Processor -- start 2024-03-23T09:48:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.843+0000 7f8e3725c640 1 -- start start 2024-03-23T09:48:38.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.843+0000 7f8e3725c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e300b8350 0x7f8e30160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.843+0000 7f8e3725c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e30161630 con 0x7f8e300cde10 2024-03-23T09:48:38.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.843+0000 7f8e3725c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e300d1030 con 0x7f8e3015c990 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.843+0000 7f8e3725c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300cfe10 con 0x7f8e300b8350 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2ffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8e300cde10 0x7f8e300d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47378/0 (socket says 172.21.15.47:47378) 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2ffff640 1 -- 172.21.15.47:0/4158393477 learned_addr learned my addr 172.21.15.47:0/4158393477 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e357f9640 1 --2- 172.21.15.47:0/4158393477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e300b8350 0x7f8e30160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901270591 0 0) 0x7f8e30161630 con 0x7f8e300cde10 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e20003520 con 0x7f8e300cde10 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1625339301 0 0) 0x7f8e20003520 con 0x7f8e300cde10 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e300b8350 msgr2=0x7f8e30160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 --2- 172.21.15.47:0/4158393477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e300b8350 0x7f8e30160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 >> v1:172.21.15.175:6789/0 conn(0x7f8e3015c990 legacy=0x7f8e300cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e300b5b60 con 0x7f8e300cde10 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f8e1c001670 con 0x7f8e300cde10 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 >> v1:172.21.15.47:6789/0 conn(0x7f8e300cde10 legacy=0x7f8e300d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 --2- 172.21.15.47:0/4158393477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e200039a0 0x7f8e20003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 --2- 172.21.15.47:0/4158393477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e300b8350 0x7f8e200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 --2- 172.21.15.47:0/4158393477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e20004d00 0x7f8e20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300cfe10 con 0x7f8e20004d00 2024-03-23T09:48:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2000ba60 con 0x7f8e200039a0 2024-03-23T09:48:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2000bc20 con 0x7f8e300b8350 2024-03-23T09:48:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.844+0000 7f8e357f9640 1 --2- 172.21.15.47:0/4158393477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e20004d00 0x7f8e20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.845+0000 7f8e2ffff640 1 --2- 172.21.15.47:0/4158393477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e300b8350 0x7f8e200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.845+0000 7f8e34ff8640 1 --2- 172.21.15.47:0/4158393477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e200039a0 0x7f8e20003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.845+0000 7f8e357f9640 1 -- 172.21.15.47:0/4158393477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e300b8350 msgr2=0x7f8e200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.845+0000 7f8e357f9640 1 --2- 172.21.15.47:0/4158393477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e300b8350 0x7f8e200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.845+0000 7f8e357f9640 1 -- 172.21.15.47:0/4158393477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e200039a0 msgr2=0x7f8e20003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.845+0000 7f8e357f9640 1 --2- 172.21.15.47:0/4158393477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e200039a0 0x7f8e20003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.845+0000 7f8e357f9640 1 -- 172.21.15.47:0/4158393477 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e20003520 con 0x7f8e20004d00 2024-03-23T09:48:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.845+0000 7f8e2ffff640 1 --2- 172.21.15.47:0/4158393477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e300b8350 0x7f8e200047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.845+0000 7f8e357f9640 1 --2- 172.21.15.47:0/4158393477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e20004d00 0x7f8e20005180 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009fc0 tx=0x7f8e200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa8bc402d1eed56 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.846+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2802cc20 con 0x7f8e20004d00 2024-03-23T09:48:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.846+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2800f040 con 0x7f8e20004d00 2024-03-23T09:48:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.846+0000 7f8e2f7fe640 1 -- 172.21.15.47:0/4158393477 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28034440 con 0x7f8e20004d00 2024-03-23T09:48:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.846+0000 7f8e3725c640 1 -- 172.21.15.47:0/4158393477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e20004d00 msgr2=0x7f8e20005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.846+0000 7f8e3725c640 1 --2- 172.21.15.47:0/4158393477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e20004d00 0x7f8e20005180 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009fc0 tx=0x7f8e200057f0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.846+0000 7f8e357f9640 1 -- 172.21.15.47:0/4158393477 reap_dead start 2024-03-23T09:48:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.846+0000 7f8e3725c640 1 -- 172.21.15.47:0/4158393477 shutdown_connections 2024-03-23T09:48:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.846+0000 7f8e3725c640 1 -- 172.21.15.47:0/4158393477 >> 172.21.15.47:0/4158393477 conn(0x7f8e300659d0 msgr2=0x7f8e30065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.846+0000 7f8e3725c640 1 -- 172.21.15.47:0/4158393477 shutdown_connections 2024-03-23T09:48:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.846+0000 7f8e3725c640 1 -- 172.21.15.47:0/4158393477 wait complete. 2024-03-23T09:48:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.847+0000 7f8e3725c640 1 Processor -- start 2024-03-23T09:48:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.847+0000 7f8e3725c640 1 -- start start 2024-03-23T09:48:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.848+0000 7f8e3725c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30153e50 0x7f8e300cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.848+0000 7f8e3725c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e3015b580 0x7f8e3015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.848+0000 7f8e3725c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e3015d5e0 0x7f8e300ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.848+0000 7f8e3725c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e301585b0 con 0x7f8e30153e50 2024-03-23T09:48:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.848+0000 7f8e3725c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300cc820 con 0x7f8e3015d5e0 2024-03-23T09:48:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.848+0000 7f8e3725c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300cc010 con 0x7f8e3015b580 2024-03-23T09:48:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.848+0000 7f8e34ff8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30153e50 0x7f8e300cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.848+0000 7f8e34ff8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30153e50 0x7f8e300cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52674/0 (socket says 172.21.15.47:52674) 2024-03-23T09:48:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.848+0000 7f8e34ff8640 1 -- 172.21.15.47:0/3582495924 learned_addr learned my addr 172.21.15.47:0/3582495924 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.848+0000 7f8e357f9640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e3015d5e0 0x7f8e300ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.848+0000 7f8e2ffff640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e3015b580 0x7f8e3015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.849+0000 7f8e34ff8640 1 -- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e3015b580 msgr2=0x7f8e3015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.849+0000 7f8e34ff8640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e3015b580 0x7f8e3015d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.849+0000 7f8e34ff8640 1 -- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e3015d5e0 msgr2=0x7f8e300ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.849+0000 7f8e34ff8640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e3015d5e0 0x7f8e300ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.849+0000 7f8e34ff8640 1 -- 172.21.15.47:0/3582495924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e300c9c30 con 0x7f8e30153e50 2024-03-23T09:48:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.849+0000 7f8e357f9640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e3015d5e0 0x7f8e300ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.849+0000 7f8e34ff8640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30153e50 0x7f8e300cb680 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400e5b0 tx=0x7f8e2400ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.849+0000 7f8e2d7fa640 1 -- 172.21.15.47:0/3582495924 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e240053a0 con 0x7f8e30153e50 2024-03-23T09:48:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.849+0000 7f8e2d7fa640 1 -- 172.21.15.47:0/3582495924 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e240035f0 con 0x7f8e30153e50 2024-03-23T09:48:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.849+0000 7f8e3725c640 1 -- 172.21.15.47:0/3582495924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e300cbbc0 con 0x7f8e30153e50 2024-03-23T09:48:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.849+0000 7f8e2d7fa640 1 -- 172.21.15.47:0/3582495924 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e240039a0 con 0x7f8e30153e50 2024-03-23T09:48:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.849+0000 7f8e3725c640 1 -- 172.21.15.47:0/3582495924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e300c9ed0 con 0x7f8e30153e50 2024-03-23T09:48:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.850+0000 7f8e3725c640 1 -- 172.21.15.47:0/3582495924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e30072360 con 0x7f8e30153e50 2024-03-23T09:48:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.850+0000 7f8e2d7fa640 1 -- 172.21.15.47:0/3582495924 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8e24003b00 con 0x7f8e30153e50 2024-03-23T09:48:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.851+0000 7f8e2d7fa640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e1406ac00 0x7f8e1406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.851+0000 7f8e2d7fa640 1 -- 172.21.15.47:0/3582495924 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f8e24089250 con 0x7f8e30153e50 2024-03-23T09:48:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.851+0000 7f8e2ffff640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e1406ac00 0x7f8e1406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.851+0000 7f8e2ffff640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e1406ac00 0x7f8e1406d0c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0010e0 tx=0x7f8e1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:38.855+0000 7f8e2d7fa640 1 -- 172.21.15.47:0/3582495924 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8e240543a0 con 0x7f8e30153e50 2024-03-23T09:48:39.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.036+0000 7f8e3725c640 1 -- 172.21.15.47:0/3582495924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8e30066be0 con 0x7f8e30153e50 2024-03-23T09:48:39.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.036+0000 7f8e2d7fa640 1 -- 172.21.15.47:0/3582495924 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8e24003db0 con 0x7f8e30153e50 2024-03-23T09:48:39.036 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:48:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.038+0000 7f8e3725c640 1 -- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e1406ac00 msgr2=0x7f8e1406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.038+0000 7f8e3725c640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e1406ac00 0x7f8e1406d0c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0010e0 tx=0x7f8e1c009040 comp rx=0 tx=0).stop 2024-03-23T09:48:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.038+0000 7f8e3725c640 1 -- 172.21.15.47:0/3582495924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30153e50 msgr2=0x7f8e300cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.038+0000 7f8e3725c640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30153e50 0x7f8e300cb680 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400e5b0 tx=0x7f8e2400ea70 comp rx=0 tx=0).stop 2024-03-23T09:48:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.038+0000 7f8e3725c640 1 -- 172.21.15.47:0/3582495924 shutdown_connections 2024-03-23T09:48:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.038+0000 7f8e3725c640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e1406ac00 0x7f8e1406d0c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.038+0000 7f8e3725c640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e3015d5e0 0x7f8e300ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.038+0000 7f8e3725c640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e3015b580 0x7f8e3015d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.038+0000 7f8e3725c640 1 --2- 172.21.15.47:0/3582495924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30153e50 0x7f8e300cb680 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.038+0000 7f8e3725c640 1 -- 172.21.15.47:0/3582495924 >> 172.21.15.47:0/3582495924 conn(0x7f8e300659d0 msgr2=0x7f8e3015bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.038+0000 7f8e3725c640 1 -- 172.21.15.47:0/3582495924 shutdown_connections 2024-03-23T09:48:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.038+0000 7f8e3725c640 1 -- 172.21.15.47:0/3582495924 wait complete. 2024-03-23T09:48:39.049 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:48:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.153+0000 7f8674c69640 1 Processor -- start 2024-03-23T09:48:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.153+0000 7f8674c69640 1 -- start start 2024-03-23T09:48:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f8674c69640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86700cde10 0x7f8670160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f8674c69640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f86700b8350 0x7f86700d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f8674c69640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8670161630 con 0x7f867015c990 2024-03-23T09:48:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f8674c69640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86700d1030 con 0x7f86700cde10 2024-03-23T09:48:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f8674c69640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86700cfe10 con 0x7f86700b8350 2024-03-23T09:48:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f866dd9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86700cde10 0x7f8670160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f866e59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f867015c990 0x7f86700cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47380/0 (socket says 172.21.15.47:47380) 2024-03-23T09:48:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f866e59c640 1 -- 172.21.15.47:0/4098081757 learned_addr learned my addr 172.21.15.47:0/4098081757 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f866ed9d640 1 --2- 172.21.15.47:0/4098081757 >> v2:172.21.15.175:3300/0 conn(0x7f86700b8350 0x7f86700d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f866dd9b640 1 -- 172.21.15.47:0/4098081757 >> v2:172.21.15.175:3300/0 conn(0x7f86700b8350 msgr2=0x7f86700d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f866dd9b640 1 --2- 172.21.15.47:0/4098081757 >> v2:172.21.15.175:3300/0 conn(0x7f86700b8350 0x7f86700d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f866dd9b640 1 -- 172.21.15.47:0/4098081757 >> v1:172.21.15.47:6789/0 conn(0x7f867015c990 legacy=0x7f86700cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f866d59a640 1 -- 172.21.15.47:0/4098081757 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221686117 0 0) 0x7f8670161630 con 0x7f867015c990 2024-03-23T09:48:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.154+0000 7f866dd9b640 1 -- 172.21.15.47:0/4098081757 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86700b5b60 con 0x7f86700cde10 2024-03-23T09:48:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.155+0000 7f866dd9b640 1 --2- 172.21.15.47:0/4098081757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700cde10 0x7f8670160690 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f86700d0fa0 tx=0x7f865802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb62051d72f6f423 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.155+0000 7f866d59a640 1 -- 172.21.15.47:0/4098081757 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865802e070 con 0x7f86700cde10 2024-03-23T09:48:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.155+0000 7f866d59a640 1 -- 172.21.15.47:0/4098081757 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86580315a0 con 0x7f86700cde10 2024-03-23T09:48:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.155+0000 7f866d59a640 1 -- 172.21.15.47:0/4098081757 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658031930 con 0x7f86700cde10 2024-03-23T09:48:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.156+0000 7f8674c69640 1 -- 172.21.15.47:0/4098081757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700cde10 msgr2=0x7f8670160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.156+0000 7f8674c69640 1 --2- 172.21.15.47:0/4098081757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700cde10 0x7f8670160690 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f86700d0fa0 tx=0x7f865802b840 comp rx=0 tx=0).stop 2024-03-23T09:48:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.156+0000 7f8674c69640 1 -- 172.21.15.47:0/4098081757 shutdown_connections 2024-03-23T09:48:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.156+0000 7f8674c69640 1 --2- 172.21.15.47:0/4098081757 >> v2:172.21.15.175:3300/0 conn(0x7f86700b8350 0x7f86700d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.156+0000 7f8674c69640 1 --2- 172.21.15.47:0/4098081757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700cde10 0x7f8670160690 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.156+0000 7f8674c69640 1 -- 172.21.15.47:0/4098081757 >> 172.21.15.47:0/4098081757 conn(0x7f86700659d0 msgr2=0x7f8670065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.156+0000 7f8674c69640 1 -- 172.21.15.47:0/4098081757 shutdown_connections 2024-03-23T09:48:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.156+0000 7f8674c69640 1 -- 172.21.15.47:0/4098081757 wait complete. 2024-03-23T09:48:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.157+0000 7f8674c69640 1 Processor -- start 2024-03-23T09:48:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.157+0000 7f8674c69640 1 -- start start 2024-03-23T09:48:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f8674c69640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86700b8350 0x7f86700d2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f8674c69640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700cde10 0x7f8670152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f8674c69640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f867015c990 0x7f86700ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f8674c69640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867015eb40 con 0x7f86700cde10 2024-03-23T09:48:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f8674c69640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8670159730 con 0x7f867015c990 2024-03-23T09:48:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f8674c69640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8670159a10 con 0x7f86700b8350 2024-03-23T09:48:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f866dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700cde10 0x7f8670152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f866dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700cde10 0x7f8670152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52690/0 (socket says 172.21.15.47:52690) 2024-03-23T09:48:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f866dd9b640 1 -- 172.21.15.47:0/2770012773 learned_addr learned my addr 172.21.15.47:0/2770012773 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f866ed9d640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f867015c990 0x7f86700ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f866e59c640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86700b8350 0x7f86700d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f866dd9b640 1 -- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86700b8350 msgr2=0x7f86700d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f866dd9b640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86700b8350 0x7f86700d2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f866dd9b640 1 -- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f867015c990 msgr2=0x7f86700ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f866dd9b640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f867015c990 0x7f86700ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f866dd9b640 1 -- 172.21.15.47:0/2770012773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8658009400 con 0x7f86700cde10 2024-03-23T09:48:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.158+0000 7f866e59c640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86700b8350 0x7f86700d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.159+0000 7f866dd9b640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700cde10 0x7f8670152f90 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f8658000c00 tx=0x7f86580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.159+0000 7f8666ffd640 1 -- 172.21.15.47:0/2770012773 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658032900 con 0x7f86700cde10 2024-03-23T09:48:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.159+0000 7f8666ffd640 1 -- 172.21.15.47:0/2770012773 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8658032a60 con 0x7f86700cde10 2024-03-23T09:48:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.159+0000 7f8674c69640 1 -- 172.21.15.47:0/2770012773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867015edc0 con 0x7f86700cde10 2024-03-23T09:48:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.159+0000 7f8666ffd640 1 -- 172.21.15.47:0/2770012773 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658032dd0 con 0x7f86700cde10 2024-03-23T09:48:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.159+0000 7f8674c69640 1 -- 172.21.15.47:0/2770012773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86700cae30 con 0x7f86700cde10 2024-03-23T09:48:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.160+0000 7f8674c69640 1 -- 172.21.15.47:0/2770012773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8670159bd0 con 0x7f86700cde10 2024-03-23T09:48:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.160+0000 7f8666ffd640 1 -- 172.21.15.47:0/2770012773 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f865804d3f0 con 0x7f86700cde10 2024-03-23T09:48:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.161+0000 7f8666ffd640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f865006ac00 0x7f865006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.161+0000 7f8666ffd640 1 -- 172.21.15.47:0/2770012773 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f86580ab1c0 con 0x7f86700cde10 2024-03-23T09:48:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.161+0000 7f866e59c640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f865006ac00 0x7f865006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.161+0000 7f866e59c640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f865006ac00 0x7f865006d0c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8668000f60 tx=0x7f8668009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:39.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.165+0000 7f8666ffd640 1 -- 172.21.15.47:0/2770012773 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8658076280 con 0x7f86700cde10 2024-03-23T09:48:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.349+0000 7f8674c69640 1 -- 172.21.15.47:0/2770012773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8670072360 con 0x7f86700cde10 2024-03-23T09:48:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.349+0000 7f8666ffd640 1 -- 172.21.15.47:0/2770012773 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f86580798a0 con 0x7f86700cde10 2024-03-23T09:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:48:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.351+0000 7f8674c69640 1 -- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f865006ac00 msgr2=0x7f865006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.351+0000 7f8674c69640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f865006ac00 0x7f865006d0c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8668000f60 tx=0x7f8668009040 comp rx=0 tx=0).stop 2024-03-23T09:48:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.351+0000 7f8674c69640 1 -- 172.21.15.47:0/2770012773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700cde10 msgr2=0x7f8670152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.351+0000 7f8674c69640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700cde10 0x7f8670152f90 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f8658000c00 tx=0x7f86580046c0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.351+0000 7f8674c69640 1 -- 172.21.15.47:0/2770012773 shutdown_connections 2024-03-23T09:48:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.351+0000 7f8674c69640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f865006ac00 0x7f865006d0c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.351+0000 7f8674c69640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f867015c990 0x7f86700ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.351+0000 7f8674c69640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700cde10 0x7f8670152f90 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.351+0000 7f8674c69640 1 --2- 172.21.15.47:0/2770012773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86700b8350 0x7f86700d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.351+0000 7f8674c69640 1 -- 172.21.15.47:0/2770012773 >> 172.21.15.47:0/2770012773 conn(0x7f86700659d0 msgr2=0x7f867015baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.352+0000 7f8674c69640 1 -- 172.21.15.47:0/2770012773 shutdown_connections 2024-03-23T09:48:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.352+0000 7f8674c69640 1 -- 172.21.15.47:0/2770012773 wait complete. 2024-03-23T09:48:39.362 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:48:39.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.465+0000 7fda1fe5b640 1 Processor -- start 2024-03-23T09:48:39.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.466+0000 7fda1fe5b640 1 -- start start 2024-03-23T09:48:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.466+0000 7fda1fe5b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fda1815cb40 0x7fda18160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.466+0000 7fda1fe5b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda18160cb0 0x7fda180d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.466+0000 7fda1fe5b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda180d0ee0 con 0x7fda180b8300 2024-03-23T09:48:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.466+0000 7fda1fe5b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda180b5a10 con 0x7fda1815cb40 2024-03-23T09:48:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.466+0000 7fda1fe5b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda181616f0 con 0x7fda18160cb0 2024-03-23T09:48:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.466+0000 7fda1dbf7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fda1815cb40 0x7fda18160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.466+0000 7fda1d3f6640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fda180b8300 0x7fda180cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47392/0 (socket says 172.21.15.47:47392) 2024-03-23T09:48:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.466+0000 7fda1d3f6640 1 -- 172.21.15.47:0/3711805870 learned_addr learned my addr 172.21.15.47:0/3711805870 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.467+0000 7fda1dbf7640 1 -- 172.21.15.47:0/3711805870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda18160cb0 msgr2=0x7fda180d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.467+0000 7fda1dbf7640 1 --2- 172.21.15.47:0/3711805870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda18160cb0 0x7fda180d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.467+0000 7fda1dbf7640 1 -- 172.21.15.47:0/3711805870 >> v1:172.21.15.47:6789/0 conn(0x7fda180b8300 legacy=0x7fda180cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.467+0000 7fda1dbf7640 1 -- 172.21.15.47:0/3711805870 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda18161130 con 0x7fda1815cb40 2024-03-23T09:48:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.467+0000 7fda1cbf5640 1 -- 172.21.15.47:0/3711805870 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665732264 0 0) 0x7fda180d0ee0 con 0x7fda180b8300 2024-03-23T09:48:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.467+0000 7fda1dbf7640 1 --2- 172.21.15.47:0/3711805870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1815cb40 0x7fda18160770 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fda140097d0 tx=0x7fda1402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98301e029d83e78 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.468+0000 7fda1cbf5640 1 -- 172.21.15.47:0/3711805870 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14031800 con 0x7fda1815cb40 2024-03-23T09:48:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.468+0000 7fda1cbf5640 1 -- 172.21.15.47:0/3711805870 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda14031960 con 0x7fda1815cb40 2024-03-23T09:48:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.468+0000 7fda1cbf5640 1 -- 172.21.15.47:0/3711805870 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14031cf0 con 0x7fda1815cb40 2024-03-23T09:48:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.468+0000 7fda1fe5b640 1 -- 172.21.15.47:0/3711805870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1815cb40 msgr2=0x7fda18160770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.468+0000 7fda1fe5b640 1 --2- 172.21.15.47:0/3711805870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1815cb40 0x7fda18160770 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fda140097d0 tx=0x7fda1402b8a0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.468+0000 7fda1fe5b640 1 -- 172.21.15.47:0/3711805870 shutdown_connections 2024-03-23T09:48:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.468+0000 7fda1fe5b640 1 --2- 172.21.15.47:0/3711805870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda18160cb0 0x7fda180d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.468+0000 7fda1fe5b640 1 --2- 172.21.15.47:0/3711805870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1815cb40 0x7fda18160770 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.468+0000 7fda1fe5b640 1 -- 172.21.15.47:0/3711805870 >> 172.21.15.47:0/3711805870 conn(0x7fda1806b160 msgr2=0x7fda1806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.468+0000 7fda1fe5b640 1 -- 172.21.15.47:0/3711805870 shutdown_connections 2024-03-23T09:48:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.468+0000 7fda1fe5b640 1 -- 172.21.15.47:0/3711805870 wait complete. 2024-03-23T09:48:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.469+0000 7fda1fe5b640 1 Processor -- start 2024-03-23T09:48:39.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.469+0000 7fda1fe5b640 1 -- start start 2024-03-23T09:48:39.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.470+0000 7fda1fe5b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda180b8300 0x7fda180cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.470+0000 7fda1fe5b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda1815cb40 0x7fda181f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.470+0000 7fda1fe5b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda18160cb0 0x7fda180b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.470+0000 7fda1fe5b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda180cb950 con 0x7fda180b8300 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.470+0000 7fda1fe5b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda18155bf0 con 0x7fda18160cb0 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.470+0000 7fda1fe5b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda18155e90 con 0x7fda1815cb40 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.470+0000 7fda1dbf7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda180b8300 0x7fda180cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.470+0000 7fda1dbf7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda180b8300 0x7fda180cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52716/0 (socket says 172.21.15.47:52716) 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.470+0000 7fda1dbf7640 1 -- 172.21.15.47:0/2179925846 learned_addr learned my addr 172.21.15.47:0/2179925846 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.470+0000 7fda1e3f8640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda18160cb0 0x7fda180b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda1d3f6640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda1815cb40 0x7fda181f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda1dbf7640 1 -- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda1815cb40 msgr2=0x7fda181f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda1dbf7640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda1815cb40 0x7fda181f1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda1dbf7640 1 -- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda18160cb0 msgr2=0x7fda180b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda1dbf7640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda18160cb0 0x7fda180b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda1dbf7640 1 -- 172.21.15.47:0/2179925846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda14009400 con 0x7fda180b8300 2024-03-23T09:48:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda1e3f8640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda18160cb0 0x7fda180b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda1d3f6640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda1815cb40 0x7fda181f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda1dbf7640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda180b8300 0x7fda180cb410 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fda140098f0 tx=0x7fda140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda0e7fc640 1 -- 172.21.15.47:0/2179925846 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1403bd80 con 0x7fda180b8300 2024-03-23T09:48:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda0e7fc640 1 -- 172.21.15.47:0/2179925846 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda140453f0 con 0x7fda180b8300 2024-03-23T09:48:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda1fe5b640 1 -- 172.21.15.47:0/2179925846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda18156170 con 0x7fda180b8300 2024-03-23T09:48:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda0e7fc640 1 -- 172.21.15.47:0/2179925846 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140457a0 con 0x7fda180b8300 2024-03-23T09:48:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.471+0000 7fda1fe5b640 1 -- 172.21.15.47:0/2179925846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda18156730 con 0x7fda180b8300 2024-03-23T09:48:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.472+0000 7fda1fe5b640 1 -- 172.21.15.47:0/2179925846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda18072360 con 0x7fda180b8300 2024-03-23T09:48:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.472+0000 7fda0e7fc640 1 -- 172.21.15.47:0/2179925846 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fda1404e450 con 0x7fda180b8300 2024-03-23T09:48:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.473+0000 7fda0e7fc640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd9fc06ac00 0x7fd9fc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.473+0000 7fda0e7fc640 1 -- 172.21.15.47:0/2179925846 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fda1402e070 con 0x7fda180b8300 2024-03-23T09:48:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.473+0000 7fda1d3f6640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd9fc06ac00 0x7fd9fc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.473+0000 7fda1d3f6640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd9fc06ac00 0x7fd9fc06d0c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fda180ce190 tx=0x7fda08009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:39.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.477+0000 7fda0e7fc640 1 -- 172.21.15.47:0/2179925846 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fda14076ba0 con 0x7fda180b8300 2024-03-23T09:48:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.669+0000 7fda1fe5b640 1 -- 172.21.15.47:0/2179925846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fda18066be0 con 0x7fda180b8300 2024-03-23T09:48:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.669+0000 7fda0e7fc640 1 -- 172.21.15.47:0/2179925846 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v88) v1 ==== 80+0+122 (secure 0 0 0) 0x7fda1407a1c0 con 0x7fda180b8300 2024-03-23T09:48:39.670 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6835/1927629460 2024-03-24T09:47:28.505920+0000 2024-03-23T09:48:39.670 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6834/1927629460 2024-03-24T09:47:28.505920+0000 2024-03-23T09:48:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:listed 2 entries 2024-03-23T09:48:39.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.671+0000 7fda1fe5b640 1 -- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd9fc06ac00 msgr2=0x7fd9fc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.671+0000 7fda1fe5b640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd9fc06ac00 0x7fd9fc06d0c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fda180ce190 tx=0x7fda08009040 comp rx=0 tx=0).stop 2024-03-23T09:48:39.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.671+0000 7fda1fe5b640 1 -- 172.21.15.47:0/2179925846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda180b8300 msgr2=0x7fda180cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.671+0000 7fda1fe5b640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda180b8300 0x7fda180cb410 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fda140098f0 tx=0x7fda140046c0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.671+0000 7fda1fe5b640 1 -- 172.21.15.47:0/2179925846 shutdown_connections 2024-03-23T09:48:39.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.671+0000 7fda1fe5b640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd9fc06ac00 0x7fd9fc06d0c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.671+0000 7fda1fe5b640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda18160cb0 0x7fda180b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.671+0000 7fda1fe5b640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda1815cb40 0x7fda181f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.671+0000 7fda1fe5b640 1 --2- 172.21.15.47:0/2179925846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda180b8300 0x7fda180cb410 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.671+0000 7fda1fe5b640 1 -- 172.21.15.47:0/2179925846 >> 172.21.15.47:0/2179925846 conn(0x7fda1806b160 msgr2=0x7fda180d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:39.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.672+0000 7fda1fe5b640 1 -- 172.21.15.47:0/2179925846 shutdown_connections 2024-03-23T09:48:39.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.672+0000 7fda1fe5b640 1 -- 172.21.15.47:0/2179925846 wait complete. 2024-03-23T09:48:39.682 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:48:39.682 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:48:39.697 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:48:39.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.837+0000 7f75eaa77640 1 Processor -- start 2024-03-23T09:48:39.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.837+0000 7f75eaa77640 1 -- start start 2024-03-23T09:48:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75eaa77640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f75e40b8350 0x7f75e40cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75eaa77640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f75e40cdc40 0x7f75e40d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75eaa77640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e40ce090 con 0x7f75e415ca90 2024-03-23T09:48:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75eaa77640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e40d11e0 con 0x7f75e40b8350 2024-03-23T09:48:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75eaa77640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e4160f60 con 0x7f75e40cdc40 2024-03-23T09:48:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75e3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f75e40b8350 0x7f75e40cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75e8813640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f75e415ca90 0x7f75e4160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47398/0 (socket says 172.21.15.47:47398) 2024-03-23T09:48:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75e3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f75e40b8350 0x7f75e40cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52732/0 (socket says 172.21.15.47:52732) 2024-03-23T09:48:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75e8813640 1 -- 172.21.15.47:0/441422942 learned_addr learned my addr 172.21.15.47:0/441422942 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75e9014640 1 --2- 172.21.15.47:0/441422942 >> v2:172.21.15.175:3300/0 conn(0x7f75e40cdc40 0x7f75e40d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75e3fff640 1 -- 172.21.15.47:0/441422942 >> v2:172.21.15.175:3300/0 conn(0x7f75e40cdc40 msgr2=0x7f75e40d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75e3fff640 1 --2- 172.21.15.47:0/441422942 >> v2:172.21.15.175:3300/0 conn(0x7f75e40cdc40 0x7f75e40d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75e3fff640 1 -- 172.21.15.47:0/441422942 >> v1:172.21.15.47:6789/0 conn(0x7f75e415ca90 legacy=0x7f75e4160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.838+0000 7f75e3fff640 1 -- 172.21.15.47:0/441422942 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e4161240 con 0x7f75e40b8350 2024-03-23T09:48:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.839+0000 7f75e3fff640 1 --2- 172.21.15.47:0/441422942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e40b8350 0x7f75e40cd700 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f75e40b5930 tx=0x7f75d002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec833c9785f4cf08 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.839+0000 7f75e37fe640 1 -- 172.21.15.47:0/441422942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d002e070 con 0x7f75e40b8350 2024-03-23T09:48:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.839+0000 7f75e37fe640 1 -- 172.21.15.47:0/441422942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d00315a0 con 0x7f75e40b8350 2024-03-23T09:48:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.839+0000 7f75e37fe640 1 -- 172.21.15.47:0/441422942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0031930 con 0x7f75e40b8350 2024-03-23T09:48:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.840+0000 7f75eaa77640 1 -- 172.21.15.47:0/441422942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e40b8350 msgr2=0x7f75e40cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.840+0000 7f75eaa77640 1 --2- 172.21.15.47:0/441422942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e40b8350 0x7f75e40cd700 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f75e40b5930 tx=0x7f75d002b840 comp rx=0 tx=0).stop 2024-03-23T09:48:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.840+0000 7f75eaa77640 1 -- 172.21.15.47:0/441422942 shutdown_connections 2024-03-23T09:48:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.840+0000 7f75eaa77640 1 --2- 172.21.15.47:0/441422942 >> v2:172.21.15.175:3300/0 conn(0x7f75e40cdc40 0x7f75e40d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.840+0000 7f75eaa77640 1 --2- 172.21.15.47:0/441422942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e40b8350 0x7f75e40cd700 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.840+0000 7f75eaa77640 1 -- 172.21.15.47:0/441422942 >> 172.21.15.47:0/441422942 conn(0x7f75e40659d0 msgr2=0x7f75e4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.840+0000 7f75eaa77640 1 -- 172.21.15.47:0/441422942 shutdown_connections 2024-03-23T09:48:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.840+0000 7f75eaa77640 1 -- 172.21.15.47:0/441422942 wait complete. 2024-03-23T09:48:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.841+0000 7f75eaa77640 1 Processor -- start 2024-03-23T09:48:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.841+0000 7f75eaa77640 1 -- start start 2024-03-23T09:48:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75eaa77640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75e40b8350 0x7f75e40c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75eaa77640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e40cdc40 0x7f75e40b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75eaa77640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e415ca90 0x7f75e4152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75eaa77640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e40c92e0 con 0x7f75e40cdc40 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75eaa77640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e40c9a90 con 0x7f75e415ca90 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75eaa77640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e4153580 con 0x7f75e40b8350 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75e3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e40cdc40 0x7f75e40b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75e3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e40cdc40 0x7f75e40b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52738/0 (socket says 172.21.15.47:52738) 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75e3fff640 1 -- 172.21.15.47:0/3727419492 learned_addr learned my addr 172.21.15.47:0/3727419492 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75e9014640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e415ca90 0x7f75e4152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75e8813640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75e40b8350 0x7f75e40c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75e3fff640 1 -- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75e40b8350 msgr2=0x7f75e40c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75e3fff640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75e40b8350 0x7f75e40c8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75e3fff640 1 -- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e415ca90 msgr2=0x7f75e4152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75e3fff640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e415ca90 0x7f75e4152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75e3fff640 1 -- 172.21.15.47:0/3727419492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d0009400 con 0x7f75e40cdc40 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75e9014640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e415ca90 0x7f75e4152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75e3fff640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e40cdc40 0x7f75e40b4c80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009890 tx=0x7f75d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.842+0000 7f75e17fa640 1 -- 172.21.15.47:0/3727419492 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0031f10 con 0x7f75e40cdc40 2024-03-23T09:48:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.843+0000 7f75e17fa640 1 -- 172.21.15.47:0/3727419492 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d0032840 con 0x7f75e40cdc40 2024-03-23T09:48:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.843+0000 7f75e17fa640 1 -- 172.21.15.47:0/3727419492 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0032bd0 con 0x7f75e40cdc40 2024-03-23T09:48:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.843+0000 7f75eaa77640 1 -- 172.21.15.47:0/3727419492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e4153860 con 0x7f75e40cdc40 2024-03-23T09:48:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.843+0000 7f75eaa77640 1 -- 172.21.15.47:0/3727419492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e4153e20 con 0x7f75e40cdc40 2024-03-23T09:48:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.844+0000 7f75eaa77640 1 -- 172.21.15.47:0/3727419492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e4072360 con 0x7f75e40cdc40 2024-03-23T09:48:39.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.844+0000 7f75e17fa640 1 -- 172.21.15.47:0/3727419492 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f75d0032d30 con 0x7f75e40cdc40 2024-03-23T09:48:39.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.844+0000 7f75e17fa640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75c806ac30 0x7f75c806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:39.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.845+0000 7f75e17fa640 1 -- 172.21.15.47:0/3727419492 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f75d00ab550 con 0x7f75e40cdc40 2024-03-23T09:48:39.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.845+0000 7f75e8813640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75c806ac30 0x7f75c806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:39.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.845+0000 7f75e8813640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75c806ac30 0x7f75c806d0f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f75d8000f60 tx=0x7f75d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:39.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:39.849+0000 7f75e17fa640 1 -- 172.21.15.47:0/3727419492 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f75d00766c0 con 0x7f75e40cdc40 2024-03-23T09:48:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.033+0000 7f75eaa77640 1 -- 172.21.15.47:0/3727419492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75e40c9c20 con 0x7f75e40cdc40 2024-03-23T09:48:40.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.034+0000 7f75e17fa640 1 -- 172.21.15.47:0/3727419492 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16319 (secure 0 0 0) 0x7f75d0079ce0 con 0x7f75e40cdc40 2024-03-23T09:48:40.034 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:48:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":88,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:47:52.879597+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:47:09.722391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-23T09:47:10.354054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.30","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/1927629460":"2024-03-24T09:47:28.505920+0000","172.21.15.47:6834/1927629460":"2024-03-24T09:47:28.505920+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:48:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.036+0000 7f75eaa77640 1 -- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75c806ac30 msgr2=0x7f75c806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.036+0000 7f75eaa77640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75c806ac30 0x7f75c806d0f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f75d8000f60 tx=0x7f75d8009040 comp rx=0 tx=0).stop 2024-03-23T09:48:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.036+0000 7f75eaa77640 1 -- 172.21.15.47:0/3727419492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e40cdc40 msgr2=0x7f75e40b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.036+0000 7f75eaa77640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e40cdc40 0x7f75e40b4c80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009890 tx=0x7f75d00046c0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.036+0000 7f75eaa77640 1 -- 172.21.15.47:0/3727419492 shutdown_connections 2024-03-23T09:48:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.037+0000 7f75eaa77640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75c806ac30 0x7f75c806d0f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.037+0000 7f75eaa77640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e415ca90 0x7f75e4152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.037+0000 7f75eaa77640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e40cdc40 0x7f75e40b4c80 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.037+0000 7f75eaa77640 1 --2- 172.21.15.47:0/3727419492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75e40b8350 0x7f75e40c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.037+0000 7f75eaa77640 1 -- 172.21.15.47:0/3727419492 >> 172.21.15.47:0/3727419492 conn(0x7f75e40659d0 msgr2=0x7f75e415fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:40.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.037+0000 7f75eaa77640 1 -- 172.21.15.47:0/3727419492 shutdown_connections 2024-03-23T09:48:40.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.037+0000 7f75eaa77640 1 -- 172.21.15.47:0/3727419492 wait complete. 2024-03-23T09:48:40.048 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:47:09.722391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:47:10.354054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:48:40.048 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:48:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.151+0000 7f026f7fb640 1 Processor -- start 2024-03-23T09:48:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.151+0000 7f026f7fb640 1 -- start start 2024-03-23T09:48:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.152+0000 7f026f7fb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f026815cc30 0x7f02680ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.152+0000 7f026f7fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02680cad40 0x7f02680cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.152+0000 7f026f7fb640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026815d010 con 0x7f02680b0610 2024-03-23T09:48:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.152+0000 7f026f7fb640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02680b11b0 con 0x7f026815cc30 2024-03-23T09:48:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.152+0000 7f026f7fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02680b13d0 con 0x7f02680cad40 2024-03-23T09:48:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.152+0000 7f026d597640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f026815cc30 0x7f02680ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.152+0000 7f026dd98640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f02680b0610 0x7f02680b0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47412/0 (socket says 172.21.15.47:47412) 2024-03-23T09:48:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.152+0000 7f026dd98640 1 -- 172.21.15.47:0/265704874 learned_addr learned my addr 172.21.15.47:0/265704874 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.153+0000 7f026d597640 1 -- 172.21.15.47:0/265704874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02680cad40 msgr2=0x7f02680cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:40.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.153+0000 7f026cd96640 1 --2- 172.21.15.47:0/265704874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02680cad40 0x7f02680cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.153+0000 7f026d597640 1 --2- 172.21.15.47:0/265704874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02680cad40 0x7f02680cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.153+0000 7f0267fff640 1 -- 172.21.15.47:0/265704874 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541044233 0 0) 0x7f026815d010 con 0x7f02680b0610 2024-03-23T09:48:40.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.153+0000 7f026d597640 1 -- 172.21.15.47:0/265704874 >> v1:172.21.15.47:6789/0 conn(0x7f02680b0610 legacy=0x7f02680b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.153+0000 7f026d597640 1 -- 172.21.15.47:0/265704874 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680b16b0 con 0x7f026815cc30 2024-03-23T09:48:40.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.153+0000 7f026cd96640 1 --2- 172.21.15.47:0/265704874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02680cad40 0x7f02680cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:40.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.153+0000 7f026d597640 1 --2- 172.21.15.47:0/265704874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f026815cc30 0x7f02680ccf70 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f02600097a0 tx=0x7f026002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a205001864810251 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:40.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.154+0000 7f0267fff640 1 -- 172.21.15.47:0/265704874 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260031830 con 0x7f026815cc30 2024-03-23T09:48:40.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.154+0000 7f0267fff640 1 -- 172.21.15.47:0/265704874 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0260031990 con 0x7f026815cc30 2024-03-23T09:48:40.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.154+0000 7f0267fff640 1 -- 172.21.15.47:0/265704874 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260031d20 con 0x7f026815cc30 2024-03-23T09:48:40.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.154+0000 7f026f7fb640 1 -- 172.21.15.47:0/265704874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f026815cc30 msgr2=0x7f02680ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.154+0000 7f026f7fb640 1 --2- 172.21.15.47:0/265704874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f026815cc30 0x7f02680ccf70 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f02600097a0 tx=0x7f026002b8d0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.154+0000 7f026f7fb640 1 -- 172.21.15.47:0/265704874 shutdown_connections 2024-03-23T09:48:40.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.154+0000 7f026f7fb640 1 --2- 172.21.15.47:0/265704874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02680cad40 0x7f02680cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.154+0000 7f026f7fb640 1 --2- 172.21.15.47:0/265704874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f026815cc30 0x7f02680ccf70 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.154+0000 7f026f7fb640 1 -- 172.21.15.47:0/265704874 >> 172.21.15.47:0/265704874 conn(0x7f02680d06f0 msgr2=0x7f02680d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:40.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.154+0000 7f026f7fb640 1 -- 172.21.15.47:0/265704874 shutdown_connections 2024-03-23T09:48:40.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.154+0000 7f026f7fb640 1 -- 172.21.15.47:0/265704874 wait complete. 2024-03-23T09:48:40.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.155+0000 7f026f7fb640 1 Processor -- start 2024-03-23T09:48:40.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.155+0000 7f026f7fb640 1 -- start start 2024-03-23T09:48:40.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026f7fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02680b0610 0x7f02680c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026f7fb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02680cad40 0x7f02680c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026f7fb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f026815cc30 0x7f02681f5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026f7fb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680ae340 con 0x7f026815cc30 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026f7fb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02681f5650 con 0x7f02680cad40 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026f7fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02680ca6b0 con 0x7f02680b0610 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026dd98640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f026815cc30 0x7f02681f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026dd98640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f026815cc30 0x7f02681f5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52754/0 (socket says 172.21.15.47:52754) 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026dd98640 1 -- 172.21.15.47:0/354385852 learned_addr learned my addr 172.21.15.47:0/354385852 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026dd98640 1 -- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02680b0610 msgr2=0x7f02680c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026d597640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02680b0610 0x7f02680c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026cd96640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02680cad40 0x7f02680c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026dd98640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02680b0610 0x7f02680c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026dd98640 1 -- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02680cad40 msgr2=0x7f02680c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026dd98640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02680cad40 0x7f02680c6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.156+0000 7f026dd98640 1 -- 172.21.15.47:0/354385852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0260009400 con 0x7f026815cc30 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.157+0000 7f026dd98640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f026815cc30 0x7f02681f5880 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f025800c7a0 tx=0x7f025800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:40.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.157+0000 7f0265ffb640 1 -- 172.21.15.47:0/354385852 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025800a640 con 0x7f026815cc30 2024-03-23T09:48:40.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.157+0000 7f0265ffb640 1 -- 172.21.15.47:0/354385852 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025800a7a0 con 0x7f026815cc30 2024-03-23T09:48:40.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.157+0000 7f026f7fb640 1 -- 172.21.15.47:0/354385852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02680c7860 con 0x7f026815cc30 2024-03-23T09:48:40.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.157+0000 7f0265ffb640 1 -- 172.21.15.47:0/354385852 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025800abd0 con 0x7f026815cc30 2024-03-23T09:48:40.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.157+0000 7f026f7fb640 1 -- 172.21.15.47:0/354385852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02680c7e20 con 0x7f026815cc30 2024-03-23T09:48:40.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.158+0000 7f026f7fb640 1 -- 172.21.15.47:0/354385852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02680ae4f0 con 0x7f026815cc30 2024-03-23T09:48:40.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.158+0000 7f0265ffb640 1 -- 172.21.15.47:0/354385852 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f025800ad30 con 0x7f026815cc30 2024-03-23T09:48:40.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.159+0000 7f0265ffb640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f025406ac30 0x7f025406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.159+0000 7f0265ffb640 1 -- 172.21.15.47:0/354385852 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0258088af0 con 0x7f026815cc30 2024-03-23T09:48:40.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.159+0000 7f026d597640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f025406ac30 0x7f025406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.160+0000 7f026d597640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f025406ac30 0x7f025406d0f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0260036000 tx=0x7f0260008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:40.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.163+0000 7f0265ffb640 1 -- 172.21.15.47:0/354385852 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f02680ae4f0 con 0x7f026815cc30 2024-03-23T09:48:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.341+0000 7f026f7fb640 1 -- 172.21.15.47:0/354385852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02680ae4f0 con 0x7f026815cc30 2024-03-23T09:48:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.342+0000 7f0265ffb640 1 -- 172.21.15.47:0/354385852 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f02680ae4f0 con 0x7f026815cc30 2024-03-23T09:48:40.342 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:48:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.344+0000 7f026f7fb640 1 -- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f025406ac30 msgr2=0x7f025406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.344+0000 7f026f7fb640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f025406ac30 0x7f025406d0f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0260036000 tx=0x7f0260008030 comp rx=0 tx=0).stop 2024-03-23T09:48:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.344+0000 7f026f7fb640 1 -- 172.21.15.47:0/354385852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f026815cc30 msgr2=0x7f02681f5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.344+0000 7f026f7fb640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f026815cc30 0x7f02681f5880 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f025800c7a0 tx=0x7f025800cc60 comp rx=0 tx=0).stop 2024-03-23T09:48:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.344+0000 7f026f7fb640 1 -- 172.21.15.47:0/354385852 shutdown_connections 2024-03-23T09:48:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.344+0000 7f026f7fb640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f025406ac30 0x7f025406d0f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.344+0000 7f026f7fb640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f026815cc30 0x7f02681f5880 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.344+0000 7f026f7fb640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02680cad40 0x7f02680c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.344+0000 7f026f7fb640 1 --2- 172.21.15.47:0/354385852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02680b0610 0x7f02680c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.344+0000 7f026f7fb640 1 -- 172.21.15.47:0/354385852 >> 172.21.15.47:0/354385852 conn(0x7f02680d06f0 msgr2=0x7f02680cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.344+0000 7f026f7fb640 1 -- 172.21.15.47:0/354385852 shutdown_connections 2024-03-23T09:48:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.344+0000 7f026f7fb640 1 -- 172.21.15.47:0/354385852 wait complete. 2024-03-23T09:48:40.354 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:48:40.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.458+0000 7fa8406e0640 1 Processor -- start 2024-03-23T09:48:40.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.458+0000 7fa8406e0640 1 -- start start 2024-03-23T09:48:40.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa8406e0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa83815c990 0x7fa838160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa8406e0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa8380b8350 0x7fa8380cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa8406e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8380cdc40 0x7fa8380b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa8406e0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8380ce0b0 con 0x7fa83815c990 2024-03-23T09:48:40.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa8406e0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa838161630 con 0x7fa8380b8350 2024-03-23T09:48:40.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa8406e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8380d1030 con 0x7fa8380cdc40 2024-03-23T09:48:40.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa83e47c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa83815c990 0x7fa838160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa83e47c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa83815c990 0x7fa838160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52756/0 (socket says 172.21.15.47:52756) 2024-03-23T09:48:40.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa83e47c640 1 -- 172.21.15.47:0/3008795630 learned_addr learned my addr 172.21.15.47:0/3008795630 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:40.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa83dc7b640 1 --2- 172.21.15.47:0/3008795630 >> v2:172.21.15.175:3300/0 conn(0x7fa8380b8350 0x7fa8380cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa83e47c640 1 -- 172.21.15.47:0/3008795630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8380cdc40 msgr2=0x7fa8380b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:40.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa83ec7d640 1 --2- 172.21.15.47:0/3008795630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8380cdc40 0x7fa8380b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa83e47c640 1 --2- 172.21.15.47:0/3008795630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8380cdc40 0x7fa8380b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa83e47c640 1 -- 172.21.15.47:0/3008795630 >> v2:172.21.15.175:3300/0 conn(0x7fa8380b8350 msgr2=0x7fa8380cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:40.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa83e47c640 1 --2- 172.21.15.47:0/3008795630 >> v2:172.21.15.175:3300/0 conn(0x7fa8380b8350 0x7fa8380cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa83e47c640 1 -- 172.21.15.47:0/3008795630 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8380cfe10 con 0x7fa83815c990 2024-03-23T09:48:40.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa83dc7b640 1 --2- 172.21.15.47:0/3008795630 >> v2:172.21.15.175:3300/0 conn(0x7fa8380b8350 0x7fa8380cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:40.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.459+0000 7fa83ec7d640 1 --2- 172.21.15.47:0/3008795630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8380cdc40 0x7fa8380b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:40.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.460+0000 7fa83e47c640 1 --2- 172.21.15.47:0/3008795630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83815c990 0x7fa838160690 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa83808ac70 tx=0x7fa83402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a29b22f66418f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:40.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.460+0000 7fa83d47a640 1 -- 172.21.15.47:0/3008795630 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8340317e0 con 0x7fa83815c990 2024-03-23T09:48:40.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.460+0000 7fa83d47a640 1 -- 172.21.15.47:0/3008795630 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa834031940 con 0x7fa83815c990 2024-03-23T09:48:40.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.460+0000 7fa83d47a640 1 -- 172.21.15.47:0/3008795630 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834031cd0 con 0x7fa83815c990 2024-03-23T09:48:40.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.460+0000 7fa8406e0640 1 -- 172.21.15.47:0/3008795630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83815c990 msgr2=0x7fa838160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.460+0000 7fa8406e0640 1 --2- 172.21.15.47:0/3008795630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83815c990 0x7fa838160690 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa83808ac70 tx=0x7fa83402b880 comp rx=0 tx=0).stop 2024-03-23T09:48:40.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.461+0000 7fa8406e0640 1 -- 172.21.15.47:0/3008795630 shutdown_connections 2024-03-23T09:48:40.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.461+0000 7fa8406e0640 1 --2- 172.21.15.47:0/3008795630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8380cdc40 0x7fa8380b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.461+0000 7fa8406e0640 1 --2- 172.21.15.47:0/3008795630 >> v2:172.21.15.175:3300/0 conn(0x7fa8380b8350 0x7fa8380cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.461+0000 7fa8406e0640 1 --2- 172.21.15.47:0/3008795630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83815c990 0x7fa838160690 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.461+0000 7fa8406e0640 1 -- 172.21.15.47:0/3008795630 >> 172.21.15.47:0/3008795630 conn(0x7fa8380659d0 msgr2=0x7fa838065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:40.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.461+0000 7fa8406e0640 1 -- 172.21.15.47:0/3008795630 shutdown_connections 2024-03-23T09:48:40.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.461+0000 7fa8406e0640 1 -- 172.21.15.47:0/3008795630 wait complete. 2024-03-23T09:48:40.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.462+0000 7fa8406e0640 1 Processor -- start 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.462+0000 7fa8406e0640 1 -- start start 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.462+0000 7fa8406e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8380b8350 0x7fa8381f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.462+0000 7fa8406e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8380cdc40 0x7fa8381f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa8406e0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83815c990 0x7fa8380b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa8406e0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8380c7510 con 0x7fa83815c990 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa8406e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8380c77f0 con 0x7fa8380b8350 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa8406e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8380c7ad0 con 0x7fa8380cdc40 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa83ec7d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83815c990 0x7fa8380b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa83ec7d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83815c990 0x7fa8380b45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52772/0 (socket says 172.21.15.47:52772) 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa83ec7d640 1 -- 172.21.15.47:0/1592141584 learned_addr learned my addr 172.21.15.47:0/1592141584 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa83dc7b640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8380cdc40 0x7fa8381f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa83e47c640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8380b8350 0x7fa8381f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa83ec7d640 1 -- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8380cdc40 msgr2=0x7fa8381f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa83ec7d640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8380cdc40 0x7fa8381f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa83ec7d640 1 -- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8380b8350 msgr2=0x7fa8381f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa83ec7d640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8380b8350 0x7fa8381f1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa83ec7d640 1 -- 172.21.15.47:0/1592141584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8340094d0 con 0x7fa83815c990 2024-03-23T09:48:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa83ec7d640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83815c990 0x7fa8380b45a0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa83000cd50 tx=0x7fa83000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa82effd640 1 -- 172.21.15.47:0/1592141584 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83000f680 con 0x7fa83815c990 2024-03-23T09:48:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa82effd640 1 -- 172.21.15.47:0/1592141584 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83000f800 con 0x7fa83815c990 2024-03-23T09:48:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa82effd640 1 -- 172.21.15.47:0/1592141584 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83000fbb0 con 0x7fa83815c990 2024-03-23T09:48:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa8406e0640 1 -- 172.21.15.47:0/1592141584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8380c7db0 con 0x7fa83815c990 2024-03-23T09:48:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.463+0000 7fa8406e0640 1 -- 172.21.15.47:0/1592141584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8380c82d0 con 0x7fa83815c990 2024-03-23T09:48:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.464+0000 7fa8406e0640 1 -- 172.21.15.47:0/1592141584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa838072360 con 0x7fa83815c990 2024-03-23T09:48:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.465+0000 7fa82effd640 1 -- 172.21.15.47:0/1592141584 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa83000b040 con 0x7fa83815c990 2024-03-23T09:48:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.466+0000 7fa82effd640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa81c06ac00 0x7fa81c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.466+0000 7fa82effd640 1 -- 172.21.15.47:0/1592141584 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa830089c50 con 0x7fa83815c990 2024-03-23T09:48:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.466+0000 7fa83e47c640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa81c06ac00 0x7fa81c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.466+0000 7fa83e47c640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa81c06ac00 0x7fa81c06d0c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa834036000 tx=0x7fa83403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:40.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.470+0000 7fa82effd640 1 -- 172.21.15.47:0/1592141584 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa83008f020 con 0x7fa83815c990 2024-03-23T09:48:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.646+0000 7fa8406e0640 1 -- 172.21.15.47:0/1592141584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8380c7ff0 con 0x7fa83815c990 2024-03-23T09:48:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.647+0000 7fa82effd640 1 -- 172.21.15.47:0/1592141584 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa8380c7ff0 con 0x7fa83815c990 2024-03-23T09:48:40.647 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:48:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.649+0000 7fa8406e0640 1 -- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa81c06ac00 msgr2=0x7fa81c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.649+0000 7fa8406e0640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa81c06ac00 0x7fa81c06d0c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa834036000 tx=0x7fa83403d040 comp rx=0 tx=0).stop 2024-03-23T09:48:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.649+0000 7fa8406e0640 1 -- 172.21.15.47:0/1592141584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83815c990 msgr2=0x7fa8380b45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.649+0000 7fa8406e0640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83815c990 0x7fa8380b45a0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa83000cd50 tx=0x7fa83000a650 comp rx=0 tx=0).stop 2024-03-23T09:48:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.649+0000 7fa8406e0640 1 -- 172.21.15.47:0/1592141584 shutdown_connections 2024-03-23T09:48:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.649+0000 7fa8406e0640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa81c06ac00 0x7fa81c06d0c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.649+0000 7fa8406e0640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83815c990 0x7fa8380b45a0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.649+0000 7fa8406e0640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8380cdc40 0x7fa8381f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.649+0000 7fa8406e0640 1 --2- 172.21.15.47:0/1592141584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8380b8350 0x7fa8381f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.649+0000 7fa8406e0640 1 -- 172.21.15.47:0/1592141584 >> 172.21.15.47:0/1592141584 conn(0x7fa8380659d0 msgr2=0x7fa8380b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.649+0000 7fa8406e0640 1 -- 172.21.15.47:0/1592141584 shutdown_connections 2024-03-23T09:48:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.649+0000 7fa8406e0640 1 -- 172.21.15.47:0/1592141584 wait complete. 2024-03-23T09:48:40.659 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:48:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.763+0000 7f47567e3640 1 Processor -- start 2024-03-23T09:48:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.763+0000 7f47567e3640 1 -- start start 2024-03-23T09:48:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.764+0000 7f47567e3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47500cdcb0 0x7f47500b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.764+0000 7f47567e3640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47500b72f0 con 0x7f47500d2020 2024-03-23T09:48:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.764+0000 7f47567e3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47500b75d0 con 0x7f475015cb50 2024-03-23T09:48:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.764+0000 7f47567e3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47500b7e70 con 0x7f47500cdcb0 2024-03-23T09:48:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.764+0000 7f474f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47500cdcb0 0x7f47500b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.764+0000 7f4754d80640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f47500d2020 0x7f47500b5a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47416/0 (socket says 172.21.15.47:47416) 2024-03-23T09:48:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.764+0000 7f4754d80640 1 -- 172.21.15.47:0/3442918511 learned_addr learned my addr 172.21.15.47:0/3442918511 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.764+0000 7f474effd640 1 -- 172.21.15.47:0/3442918511 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764981043 0 0) 0x7f47500b72f0 con 0x7f47500d2020 2024-03-23T09:48:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.764+0000 7f474effd640 1 -- 172.21.15.47:0/3442918511 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4738003520 con 0x7f47500d2020 2024-03-23T09:48:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.764+0000 7f474f7fe640 1 -- 172.21.15.47:0/3442918511 >> v1:172.21.15.175:6789/0 conn(0x7f475015cb50 legacy=0x7f47500cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.764+0000 7f474f7fe640 1 -- 172.21.15.47:0/3442918511 >> v1:172.21.15.47:6789/0 conn(0x7f47500d2020 legacy=0x7f47500b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.764+0000 7f474f7fe640 1 -- 172.21.15.47:0/3442918511 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47500b78b0 con 0x7f47500cdcb0 2024-03-23T09:48:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.765+0000 7f474f7fe640 1 --2- 172.21.15.47:0/3442918511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47500cdcb0 0x7f47500b84a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f47400097a0 tx=0x7f474002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d552e34317cd1e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.765+0000 7f474effd640 1 -- 172.21.15.47:0/3442918511 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740030610 con 0x7f47500cdcb0 2024-03-23T09:48:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.765+0000 7f474effd640 1 -- 172.21.15.47:0/3442918511 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4740030770 con 0x7f47500cdcb0 2024-03-23T09:48:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.765+0000 7f474effd640 1 -- 172.21.15.47:0/3442918511 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740030b00 con 0x7f47500cdcb0 2024-03-23T09:48:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.765+0000 7f47567e3640 1 -- 172.21.15.47:0/3442918511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47500cdcb0 msgr2=0x7f47500b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.765+0000 7f47567e3640 1 --2- 172.21.15.47:0/3442918511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47500cdcb0 0x7f47500b84a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f47400097a0 tx=0x7f474002b960 comp rx=0 tx=0).stop 2024-03-23T09:48:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.766+0000 7f47567e3640 1 -- 172.21.15.47:0/3442918511 shutdown_connections 2024-03-23T09:48:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.766+0000 7f47567e3640 1 --2- 172.21.15.47:0/3442918511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47500cdcb0 0x7f47500b84a0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.766+0000 7f47567e3640 1 -- 172.21.15.47:0/3442918511 >> 172.21.15.47:0/3442918511 conn(0x7f4750065df0 msgr2=0x7f4750066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.766+0000 7f47567e3640 1 -- 172.21.15.47:0/3442918511 shutdown_connections 2024-03-23T09:48:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.766+0000 7f47567e3640 1 -- 172.21.15.47:0/3442918511 wait complete. 2024-03-23T09:48:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.767+0000 7f47567e3640 1 Processor -- start 2024-03-23T09:48:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.767+0000 7f47567e3640 1 -- start start 2024-03-23T09:48:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f47567e3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47500cdcb0 0x7f4750155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f47567e3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47500d2020 0x7f4750152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f47567e3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f475015cb50 0x7f475014db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f47567e3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475014fb50 con 0x7f47500d2020 2024-03-23T09:48:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f47567e3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475014fe30 con 0x7f47500cdcb0 2024-03-23T09:48:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f47567e3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4750150110 con 0x7f475015cb50 2024-03-23T09:48:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47500d2020 0x7f4750152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47500d2020 0x7f4750152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52788/0 (socket says 172.21.15.47:52788) 2024-03-23T09:48:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474f7fe640 1 -- 172.21.15.47:0/746212561 learned_addr learned my addr 172.21.15.47:0/746212561 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474ffff640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47500cdcb0 0x7f4750155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474f7fe640 1 -- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f475015cb50 msgr2=0x7f475014db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f4754d80640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f475015cb50 0x7f475014db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474f7fe640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f475015cb50 0x7f475014db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474f7fe640 1 -- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47500cdcb0 msgr2=0x7f4750155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474f7fe640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47500cdcb0 0x7f4750155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474f7fe640 1 -- 172.21.15.47:0/746212561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4740009400 con 0x7f47500d2020 2024-03-23T09:48:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474ffff640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47500cdcb0 0x7f4750155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474f7fe640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47500d2020 0x7f4750152f90 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f47400098c0 tx=0x7f4740004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474cff9640 1 -- 172.21.15.47:0/746212561 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474003e040 con 0x7f47500d2020 2024-03-23T09:48:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474cff9640 1 -- 172.21.15.47:0/746212561 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4740039cb0 con 0x7f47500d2020 2024-03-23T09:48:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f47567e3640 1 -- 172.21.15.47:0/746212561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47501503f0 con 0x7f47500d2020 2024-03-23T09:48:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f474cff9640 1 -- 172.21.15.47:0/746212561 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740044630 con 0x7f47500d2020 2024-03-23T09:48:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.768+0000 7f47567e3640 1 -- 172.21.15.47:0/746212561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475014f7a0 con 0x7f47500d2020 2024-03-23T09:48:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.769+0000 7f47567e3640 1 -- 172.21.15.47:0/746212561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4750072360 con 0x7f47500d2020 2024-03-23T09:48:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.770+0000 7f474cff9640 1 -- 172.21.15.47:0/746212561 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f474004d3b0 con 0x7f47500d2020 2024-03-23T09:48:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.770+0000 7f474cff9640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f473c06ac30 0x7f473c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.770+0000 7f474cff9640 1 -- 172.21.15.47:0/746212561 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f47400ab770 con 0x7f47500d2020 2024-03-23T09:48:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.770+0000 7f474ffff640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f473c06ac30 0x7f473c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.771+0000 7f474ffff640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f473c06ac30 0x7f473c06d0f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f474400a830 tx=0x7f4744008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:40.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.775+0000 7f474cff9640 1 -- 172.21.15.47:0/746212561 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4740076950 con 0x7f47500d2020 2024-03-23T09:48:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.954+0000 7f47567e3640 1 -- 172.21.15.47:0/746212561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4750150910 con 0x7f47500d2020 2024-03-23T09:48:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.955+0000 7f474cff9640 1 -- 172.21.15.47:0/746212561 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4750150910 con 0x7f47500d2020 2024-03-23T09:48:40.955 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:48:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.957+0000 7f47567e3640 1 -- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f473c06ac30 msgr2=0x7f473c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.957+0000 7f47567e3640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f473c06ac30 0x7f473c06d0f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f474400a830 tx=0x7f4744008040 comp rx=0 tx=0).stop 2024-03-23T09:48:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.957+0000 7f47567e3640 1 -- 172.21.15.47:0/746212561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47500d2020 msgr2=0x7f4750152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.957+0000 7f47567e3640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47500d2020 0x7f4750152f90 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f47400098c0 tx=0x7f4740004730 comp rx=0 tx=0).stop 2024-03-23T09:48:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.957+0000 7f47567e3640 1 -- 172.21.15.47:0/746212561 shutdown_connections 2024-03-23T09:48:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.957+0000 7f47567e3640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f473c06ac30 0x7f473c06d0f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.957+0000 7f47567e3640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f475015cb50 0x7f475014db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.957+0000 7f47567e3640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47500d2020 0x7f4750152f90 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.957+0000 7f47567e3640 1 --2- 172.21.15.47:0/746212561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47500cdcb0 0x7f4750155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.957+0000 7f47567e3640 1 -- 172.21.15.47:0/746212561 >> 172.21.15.47:0/746212561 conn(0x7f4750065df0 msgr2=0x7f47500cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.957+0000 7f47567e3640 1 -- 172.21.15.47:0/746212561 shutdown_connections 2024-03-23T09:48:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:40.957+0000 7f47567e3640 1 -- 172.21.15.47:0/746212561 wait complete. 2024-03-23T09:48:40.968 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:48:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.070+0000 7f591b3a4640 1 Processor -- start 2024-03-23T09:48:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.070+0000 7f591b3a4640 1 -- start start 2024-03-23T09:48:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.071+0000 7f591b3a4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5914153b70 0x7f59140d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.071+0000 7f591b3a4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f59140cfdc0 0x7f59140b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.071+0000 7f591b3a4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591415cf50 con 0x7f59140b1840 2024-03-23T09:48:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.071+0000 7f591b3a4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59140d26c0 con 0x7f59140cfdc0 2024-03-23T09:48:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.071+0000 7f591b3a4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59140b2200 con 0x7f5914153b70 2024-03-23T09:48:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.071+0000 7f591893f640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f59140b1840 0x7f59140cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47422/0 (socket says 172.21.15.47:47422) 2024-03-23T09:48:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.071+0000 7f591893f640 1 -- 172.21.15.47:0/4211256259 learned_addr learned my addr 172.21.15.47:0/4211256259 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473122660 0 0) 0x7f591415cf50 con 0x7f59140b1840 2024-03-23T09:48:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5900003520 con 0x7f59140b1840 2024-03-23T09:48:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5919941640 1 --2- 172.21.15.47:0/4211256259 >> v2:172.21.15.175:3300/0 conn(0x7f59140cfdc0 0x7f59140b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5919140640 1 --2- 172.21.15.47:0/4211256259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5914153b70 0x7f59140d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669734974 0 0) 0x7f5900003520 con 0x7f59140b1840 2024-03-23T09:48:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5914153b70 msgr2=0x7f59140d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 --2- 172.21.15.47:0/4211256259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5914153b70 0x7f59140d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 >> v2:172.21.15.175:3300/0 conn(0x7f59140cfdc0 msgr2=0x7f59140b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 --2- 172.21.15.47:0/4211256259 >> v2:172.21.15.175:3300/0 conn(0x7f59140cfdc0 0x7f59140b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591415d210 con 0x7f59140b1840 2024-03-23T09:48:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f5908001670 con 0x7f59140b1840 2024-03-23T09:48:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5919140640 1 --2- 172.21.15.47:0/4211256259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5914153b70 0x7f59140d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 >> v1:172.21.15.47:6789/0 conn(0x7f59140b1840 legacy=0x7f59140cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 --2- 172.21.15.47:0/4211256259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5900003970 0x7f5900003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 --2- 172.21.15.47:0/4211256259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59000047c0 0x7f5900004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 --2- 172.21.15.47:0/4211256259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5914153b70 0x7f5900005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59140b2200 con 0x7f59000047c0 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590000b980 con 0x7f5900003970 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590000bac0 con 0x7f5914153b70 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f591893f640 1 --2- 172.21.15.47:0/4211256259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59000047c0 0x7f5900004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5919140640 1 --2- 172.21.15.47:0/4211256259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5900003970 0x7f5900003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f5919941640 1 --2- 172.21.15.47:0/4211256259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5914153b70 0x7f5900005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f591893f640 1 -- 172.21.15.47:0/4211256259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5914153b70 msgr2=0x7f5900005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f591893f640 1 --2- 172.21.15.47:0/4211256259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5914153b70 0x7f5900005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f591893f640 1 -- 172.21.15.47:0/4211256259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5900003970 msgr2=0x7f5900003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f591893f640 1 --2- 172.21.15.47:0/4211256259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5900003970 0x7f5900003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.072+0000 7f591893f640 1 -- 172.21.15.47:0/4211256259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900003520 con 0x7f59000047c0 2024-03-23T09:48:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.073+0000 7f591893f640 1 --2- 172.21.15.47:0/4211256259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59000047c0 0x7f5900004c40 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f590800c240 tx=0x7f59000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476ececed9dcc30f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.073+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908005ad0 con 0x7f59000047c0 2024-03-23T09:48:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.073+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5908005c30 con 0x7f59000047c0 2024-03-23T09:48:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.073+0000 7f5907fff640 1 -- 172.21.15.47:0/4211256259 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590802c8d0 con 0x7f59000047c0 2024-03-23T09:48:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.074+0000 7f591b3a4640 1 -- 172.21.15.47:0/4211256259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59000047c0 msgr2=0x7f5900004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.074+0000 7f591b3a4640 1 --2- 172.21.15.47:0/4211256259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59000047c0 0x7f5900004c40 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f590800c240 tx=0x7f59000057f0 comp rx=0 tx=0).stop 2024-03-23T09:48:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.074+0000 7f5919941640 1 -- 172.21.15.47:0/4211256259 reap_dead start 2024-03-23T09:48:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.074+0000 7f591b3a4640 1 -- 172.21.15.47:0/4211256259 shutdown_connections 2024-03-23T09:48:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.074+0000 7f591b3a4640 1 -- 172.21.15.47:0/4211256259 >> 172.21.15.47:0/4211256259 conn(0x7f5914065bb0 msgr2=0x7f5914065fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.074+0000 7f591b3a4640 1 -- 172.21.15.47:0/4211256259 shutdown_connections 2024-03-23T09:48:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.074+0000 7f591b3a4640 1 -- 172.21.15.47:0/4211256259 wait complete. 2024-03-23T09:48:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.075+0000 7f591b3a4640 1 Processor -- start 2024-03-23T09:48:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.075+0000 7f591b3a4640 1 -- start start 2024-03-23T09:48:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f591b3a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59000047c0 0x7f59140b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f591b3a4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59141f6180 0x7f59141f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f591b3a4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59141f1410 0x7f59141f1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f591b3a4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59140b6120 con 0x7f59141f1410 2024-03-23T09:48:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f591b3a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59140c5e90 con 0x7f59000047c0 2024-03-23T09:48:41.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f591b3a4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59140cc390 con 0x7f59141f6180 2024-03-23T09:48:41.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f5919941640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59141f1410 0x7f59141f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:41.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f5919941640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59141f1410 0x7f59141f1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52802/0 (socket says 172.21.15.47:52802) 2024-03-23T09:48:41.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f5919941640 1 -- 172.21.15.47:0/2969331481 learned_addr learned my addr 172.21.15.47:0/2969331481 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:41.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f5919140640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59000047c0 0x7f59140b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:41.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f591893f640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59141f6180 0x7f59141f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:41.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f5919941640 1 -- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59141f6180 msgr2=0x7f59141f57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:41.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f5919941640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59141f6180 0x7f59141f57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:41.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f5919941640 1 -- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59000047c0 msgr2=0x7f59140b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:41.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f5919941640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59000047c0 0x7f59140b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:41.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f5919941640 1 -- 172.21.15.47:0/2969331481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59140cb5d0 con 0x7f59141f1410 2024-03-23T09:48:41.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.076+0000 7f5919941640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59141f1410 0x7f59141f1890 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f591408bbf0 tx=0x7f590c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:41.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.077+0000 7f5905ffb640 1 -- 172.21.15.47:0/2969331481 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c00e040 con 0x7f59141f1410 2024-03-23T09:48:41.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.077+0000 7f5905ffb640 1 -- 172.21.15.47:0/2969331481 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c00f040 con 0x7f59141f1410 2024-03-23T09:48:41.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.077+0000 7f591b3a4640 1 -- 172.21.15.47:0/2969331481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59140cb8b0 con 0x7f59141f1410 2024-03-23T09:48:41.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.077+0000 7f5905ffb640 1 -- 172.21.15.47:0/2969331481 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c015670 con 0x7f59141f1410 2024-03-23T09:48:41.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.077+0000 7f591b3a4640 1 -- 172.21.15.47:0/2969331481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59140cbe70 con 0x7f59141f1410 2024-03-23T09:48:41.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.078+0000 7f591b3a4640 1 -- 172.21.15.47:0/2969331481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59140cc090 con 0x7f59141f1410 2024-03-23T09:48:41.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.078+0000 7f5905ffb640 1 -- 172.21.15.47:0/2969331481 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f590c0157d0 con 0x7f59141f1410 2024-03-23T09:48:41.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.078+0000 7f5905ffb640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f58f806ac30 0x7f58f806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:41.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.079+0000 7f5905ffb640 1 -- 172.21.15.47:0/2969331481 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f590c089d40 con 0x7f59141f1410 2024-03-23T09:48:41.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.079+0000 7f5919140640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f58f806ac30 0x7f58f806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:41.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.079+0000 7f5919140640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f58f806ac30 0x7f58f806d0f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5910006fa0 tx=0x7f5910008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:41.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.083+0000 7f5905ffb640 1 -- 172.21.15.47:0/2969331481 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f590c054fd0 con 0x7f59141f1410 2024-03-23T09:48:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.273+0000 7f591b3a4640 1 -- 172.21.15.47:0/2969331481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f59140672f0 con 0x7f59141f1410 2024-03-23T09:48:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.273+0000 7f5905ffb640 1 -- 172.21.15.47:0/2969331481 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v88) v1 ==== 80+0+122 (secure 0 0 0) 0x7f590c0585f0 con 0x7f59141f1410 2024-03-23T09:48:41.274 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6835/1927629460 2024-03-24T09:47:28.505920+0000 2024-03-23T09:48:41.274 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6834/1927629460 2024-03-24T09:47:28.505920+0000 2024-03-23T09:48:41.274 INFO:teuthology.orchestra.run.smithi047.stderr:listed 2 entries 2024-03-23T09:48:41.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.275+0000 7f591b3a4640 1 -- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f58f806ac30 msgr2=0x7f58f806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.275+0000 7f591b3a4640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f58f806ac30 0x7f58f806d0f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5910006fa0 tx=0x7f5910008040 comp rx=0 tx=0).stop 2024-03-23T09:48:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.275+0000 7f591b3a4640 1 -- 172.21.15.47:0/2969331481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59141f1410 msgr2=0x7f59141f1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.275+0000 7f591b3a4640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59141f1410 0x7f59141f1890 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f591408bbf0 tx=0x7f590c00f580 comp rx=0 tx=0).stop 2024-03-23T09:48:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.275+0000 7f591b3a4640 1 -- 172.21.15.47:0/2969331481 shutdown_connections 2024-03-23T09:48:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.275+0000 7f591b3a4640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f58f806ac30 0x7f58f806d0f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.275+0000 7f591b3a4640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59141f1410 0x7f59141f1890 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.275+0000 7f591b3a4640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59141f6180 0x7f59141f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.275+0000 7f591b3a4640 1 --2- 172.21.15.47:0/2969331481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59000047c0 0x7f59140b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.275+0000 7f591b3a4640 1 -- 172.21.15.47:0/2969331481 >> 172.21.15.47:0/2969331481 conn(0x7f5914065bb0 msgr2=0x7f59141520c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.275+0000 7f591b3a4640 1 -- 172.21.15.47:0/2969331481 shutdown_connections 2024-03-23T09:48:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:41.275+0000 7f591b3a4640 1 -- 172.21.15.47:0/2969331481 wait complete. 2024-03-23T09:48:41.286 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:48:41.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:41.286 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:48:41.311 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[68636]: fuse finished with error 0 and tester_r 0 2024-03-23T09:48:41.313 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:48:47.414 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:48:47.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:47.415 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:48:47.433 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:48:47.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.582+0000 7f549d296640 1 Processor -- start 2024-03-23T09:48:47.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.582+0000 7f549d296640 1 -- start start 2024-03-23T09:48:47.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f549d296640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f549815ca90 0x7f5498160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:47.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f549d296640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f54980cde10 0x7f54980d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:47.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f549d296640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54980d11e0 con 0x7f54980b8350 2024-03-23T09:48:47.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f549d296640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5498160f60 con 0x7f54980cde10 2024-03-23T09:48:47.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f549d296640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5498161240 con 0x7f549815ca90 2024-03-23T09:48:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f5496ffd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f549815ca90 0x7f5498160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f54977fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f54980cde10 0x7f54980d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f5496ffd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f549815ca90 0x7f5498160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:56276/0 (socket says 172.21.15.47:56276) 2024-03-23T09:48:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f5496ffd640 1 -- 172.21.15.47:0/4108935832 learned_addr learned my addr 172.21.15.47:0/4108935832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f54967fc640 1 --1- 172.21.15.47:0/4108935832 >> v1:172.21.15.175:6789/0 conn(0x7f54980b8350 0x7f54980cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:58630/0 (socket says 172.21.15.47:58630) 2024-03-23T09:48:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f5496ffd640 1 -- 172.21.15.47:0/4108935832 >> v2:172.21.15.175:3300/0 conn(0x7f54980cde10 msgr2=0x7f54980d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f5496ffd640 1 --2- 172.21.15.47:0/4108935832 >> v2:172.21.15.175:3300/0 conn(0x7f54980cde10 0x7f54980d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f5496ffd640 1 -- 172.21.15.47:0/4108935832 >> v1:172.21.15.175:6789/0 conn(0x7f54980b8350 legacy=0x7f54980cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.583+0000 7f5496ffd640 1 -- 172.21.15.47:0/4108935832 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5498161520 con 0x7f549815ca90 2024-03-23T09:48:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.584+0000 7f5496ffd640 1 --2- 172.21.15.47:0/4108935832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f549815ca90 0x7f5498160710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5488009870 tx=0x7f548802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6198f025791a060c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.584+0000 7f5495ffb640 1 -- 172.21.15.47:0/4108935832 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488031640 con 0x7f549815ca90 2024-03-23T09:48:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.584+0000 7f5495ffb640 1 -- 172.21.15.47:0/4108935832 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54880317a0 con 0x7f549815ca90 2024-03-23T09:48:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.584+0000 7f5495ffb640 1 -- 172.21.15.47:0/4108935832 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488031b30 con 0x7f549815ca90 2024-03-23T09:48:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.585+0000 7f549d296640 1 -- 172.21.15.47:0/4108935832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f549815ca90 msgr2=0x7f5498160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.585+0000 7f549d296640 1 --2- 172.21.15.47:0/4108935832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f549815ca90 0x7f5498160710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5488009870 tx=0x7f548802b8f0 comp rx=0 tx=0).stop 2024-03-23T09:48:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.585+0000 7f549d296640 1 -- 172.21.15.47:0/4108935832 shutdown_connections 2024-03-23T09:48:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.585+0000 7f549d296640 1 --2- 172.21.15.47:0/4108935832 >> v2:172.21.15.175:3300/0 conn(0x7f54980cde10 0x7f54980d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.585+0000 7f549d296640 1 --2- 172.21.15.47:0/4108935832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f549815ca90 0x7f5498160710 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.585+0000 7f549d296640 1 -- 172.21.15.47:0/4108935832 >> 172.21.15.47:0/4108935832 conn(0x7f54980659d0 msgr2=0x7f5498065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.585+0000 7f549d296640 1 -- 172.21.15.47:0/4108935832 shutdown_connections 2024-03-23T09:48:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.585+0000 7f549d296640 1 -- 172.21.15.47:0/4108935832 wait complete. 2024-03-23T09:48:47.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.586+0000 7f549d296640 1 Processor -- start 2024-03-23T09:48:47.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.586+0000 7f549d296640 1 -- start start 2024-03-23T09:48:47.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.587+0000 7f549d296640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54980b8350 0x7f549815a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.587+0000 7f549d296640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54980cde10 0x7f54981503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.587+0000 7f549d296640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f549815ca90 0x7f5498157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.587+0000 7f549d296640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549815b850 con 0x7f549815ca90 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.587+0000 7f549d296640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54980b25e0 con 0x7f54980cde10 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.587+0000 7f549d296640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5498151430 con 0x7f54980b8350 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.587+0000 7f54977fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f549815ca90 0x7f5498157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.587+0000 7f54977fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f549815ca90 0x7f5498157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49166/0 (socket says 172.21.15.47:49166) 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.587+0000 7f54977fe640 1 -- 172.21.15.47:0/2565503442 learned_addr learned my addr 172.21.15.47:0/2565503442 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.587+0000 7f5496ffd640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54980b8350 0x7f549815a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.587+0000 7f54967fc640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54980cde10 0x7f54981503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.587+0000 7f54977fe640 1 -- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54980b8350 msgr2=0x7f549815a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.588+0000 7f54977fe640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54980b8350 0x7f549815a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.588+0000 7f54977fe640 1 -- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54980cde10 msgr2=0x7f54981503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:47.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.588+0000 7f54977fe640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54980cde10 0x7f54981503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.588+0000 7f54977fe640 1 -- 172.21.15.47:0/2565503442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54880094d0 con 0x7f549815ca90 2024-03-23T09:48:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.588+0000 7f54977fe640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f549815ca90 0x7f5498157250 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f549000efc0 tx=0x7f549000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.588+0000 7f547f7fe640 1 -- 172.21.15.47:0/2565503442 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5490009b20 con 0x7f549815ca90 2024-03-23T09:48:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.588+0000 7f549d296640 1 -- 172.21.15.47:0/2565503442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549815afb0 con 0x7f549815ca90 2024-03-23T09:48:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.588+0000 7f547f7fe640 1 -- 172.21.15.47:0/2565503442 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549000f040 con 0x7f549815ca90 2024-03-23T09:48:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.588+0000 7f547f7fe640 1 -- 172.21.15.47:0/2565503442 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54900146c0 con 0x7f549815ca90 2024-03-23T09:48:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.588+0000 7f549d296640 1 -- 172.21.15.47:0/2565503442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549815b570 con 0x7f549815ca90 2024-03-23T09:48:47.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.589+0000 7f549d296640 1 -- 172.21.15.47:0/2565503442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5498072360 con 0x7f549815ca90 2024-03-23T09:48:47.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.590+0000 7f547f7fe640 1 -- 172.21.15.47:0/2565503442 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5490014820 con 0x7f549815ca90 2024-03-23T09:48:47.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.590+0000 7f547f7fe640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f548406ac30 0x7f548406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:47.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.590+0000 7f547f7fe640 1 -- 172.21.15.47:0/2565503442 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5490089fa0 con 0x7f549815ca90 2024-03-23T09:48:47.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.590+0000 7f5496ffd640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f548406ac30 0x7f548406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:47.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.591+0000 7f5496ffd640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f548406ac30 0x7f548406d0f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5488036000 tx=0x7f548803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:47.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.594+0000 7f547f7fe640 1 -- 172.21.15.47:0/2565503442 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f54900bd580 con 0x7f549815ca90 2024-03-23T09:48:47.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.811+0000 7f549d296640 1 -- 172.21.15.47:0/2565503442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5498066be0 con 0x7f549815ca90 2024-03-23T09:48:47.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.812+0000 7f547f7fe640 1 -- 172.21.15.47:0/2565503442 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f54900551b0 con 0x7f549815ca90 2024-03-23T09:48:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:48:47.812 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:47:39.843531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7776":{"gid":7776,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":6,"addr":"172.21.15.47:6835/4278516621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4278516621},{"type":"v1","addr":"172.21.15.47:6835","nonce":4278516621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:48:47.812 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 53 2024-03-23T09:48:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.814+0000 7f549d296640 1 -- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f548406ac30 msgr2=0x7f548406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.814+0000 7f549d296640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f548406ac30 0x7f548406d0f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5488036000 tx=0x7f548803c040 comp rx=0 tx=0).stop 2024-03-23T09:48:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.814+0000 7f549d296640 1 -- 172.21.15.47:0/2565503442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f549815ca90 msgr2=0x7f5498157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.814+0000 7f549d296640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f549815ca90 0x7f5498157250 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f549000efc0 tx=0x7f549000c490 comp rx=0 tx=0).stop 2024-03-23T09:48:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.814+0000 7f549d296640 1 -- 172.21.15.47:0/2565503442 shutdown_connections 2024-03-23T09:48:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.814+0000 7f549d296640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f548406ac30 0x7f548406d0f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.814+0000 7f549d296640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f549815ca90 0x7f5498157250 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.814+0000 7f549d296640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54980cde10 0x7f54981503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.814+0000 7f549d296640 1 --2- 172.21.15.47:0/2565503442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54980b8350 0x7f549815a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.814+0000 7f549d296640 1 -- 172.21.15.47:0/2565503442 >> 172.21.15.47:0/2565503442 conn(0x7f54980659d0 msgr2=0x7f54980b7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.814+0000 7f549d296640 1 -- 172.21.15.47:0/2565503442 shutdown_connections 2024-03-23T09:48:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:47.814+0000 7f549d296640 1 -- 172.21.15.47:0/2565503442 wait complete. 2024-03-23T09:48:47.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-23T09:48:47.952 INFO:teuthology.orchestra.run.smithi047.stdout:{"message":"","return_code":0} 2024-03-23T09:48:47.952 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-23T09:48:47.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:48:48.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.056+0000 7ffb0c5bf640 1 Processor -- start 2024-03-23T09:48:48.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.056+0000 7ffb0c5bf640 1 -- start start 2024-03-23T09:48:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb0c5bf640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb0415ccc0 0x7ffb040cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb0c5bf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffb0415d0a0 0x7ffb040d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb0c5bf640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffb0406b210 0x7ffb0406b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb0c5bf640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0406c0e0 con 0x7ffb0415d0a0 2024-03-23T09:48:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb0c5bf640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0406bbe0 con 0x7ffb0406b210 2024-03-23T09:48:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb0c5bf640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040d0f00 con 0x7ffb0415ccc0 2024-03-23T09:48:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb09b5a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffb0415d0a0 0x7ffb040d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb09b5a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffb0415d0a0 0x7ffb040d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49172/0 (socket says 172.21.15.47:49172) 2024-03-23T09:48:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb09b5a640 1 -- 172.21.15.47:0/449182491 learned_addr learned my addr 172.21.15.47:0/449182491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb0a35b640 1 --2- 172.21.15.47:0/449182491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb0415ccc0 0x7ffb040cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb0ab5c640 1 --2- 172.21.15.47:0/449182491 >> v2:172.21.15.175:3300/0 conn(0x7ffb0406b210 0x7ffb0406b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb09b5a640 1 -- 172.21.15.47:0/449182491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb0415ccc0 msgr2=0x7ffb040cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb09b5a640 1 --2- 172.21.15.47:0/449182491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb0415ccc0 0x7ffb040cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb09b5a640 1 -- 172.21.15.47:0/449182491 >> v2:172.21.15.175:3300/0 conn(0x7ffb0406b210 msgr2=0x7ffb0406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb09b5a640 1 --2- 172.21.15.47:0/449182491 >> v2:172.21.15.175:3300/0 conn(0x7ffb0406b210 0x7ffb0406b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb09b5a640 1 -- 172.21.15.47:0/449182491 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb040ce2d0 con 0x7ffb0415d0a0 2024-03-23T09:48:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb0a35b640 1 --2- 172.21.15.47:0/449182491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb0415ccc0 0x7ffb040cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.057+0000 7ffb0ab5c640 1 --2- 172.21.15.47:0/449182491 >> v2:172.21.15.175:3300/0 conn(0x7ffb0406b210 0x7ffb0406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.058+0000 7ffb09b5a640 1 --2- 172.21.15.47:0/449182491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb0415d0a0 0x7ffb040d1c30 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7ffb040d2950 tx=0x7ffaf402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66c86e64469ba71 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.058+0000 7ffb09359640 1 -- 172.21.15.47:0/449182491 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4030860 con 0x7ffb0415d0a0 2024-03-23T09:48:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.058+0000 7ffb09359640 1 -- 172.21.15.47:0/449182491 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf40309c0 con 0x7ffb0415d0a0 2024-03-23T09:48:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.058+0000 7ffb09359640 1 -- 172.21.15.47:0/449182491 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4030d50 con 0x7ffb0415d0a0 2024-03-23T09:48:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.059+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/449182491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb0415d0a0 msgr2=0x7ffb040d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.059+0000 7ffb0c5bf640 1 --2- 172.21.15.47:0/449182491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb0415d0a0 0x7ffb040d1c30 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7ffb040d2950 tx=0x7ffaf402b9a0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.059+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/449182491 shutdown_connections 2024-03-23T09:48:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.059+0000 7ffb0c5bf640 1 --2- 172.21.15.47:0/449182491 >> v2:172.21.15.175:3300/0 conn(0x7ffb0406b210 0x7ffb0406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.059+0000 7ffb0c5bf640 1 --2- 172.21.15.47:0/449182491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb0415d0a0 0x7ffb040d1c30 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.059+0000 7ffb0c5bf640 1 --2- 172.21.15.47:0/449182491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb0415ccc0 0x7ffb040cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.059+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/449182491 >> 172.21.15.47:0/449182491 conn(0x7ffb040cf430 msgr2=0x7ffb040cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.059+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/449182491 shutdown_connections 2024-03-23T09:48:48.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.059+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/449182491 wait complete. 2024-03-23T09:48:48.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.060+0000 7ffb0c5bf640 1 Processor -- start 2024-03-23T09:48:48.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.060+0000 7ffb0c5bf640 1 -- start start 2024-03-23T09:48:48.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.060+0000 7ffb0c5bf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb0406b210 0x7ffb0414d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0c5bf640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb0415ccc0 0x7ffb040b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0c5bf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb0415d0a0 0x7ffb040b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0c5bf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040b5250 con 0x7ffb0415d0a0 2024-03-23T09:48:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0c5bf640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb04153e50 con 0x7ffb0406b210 2024-03-23T09:48:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0c5bf640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040acfb0 con 0x7ffb0415ccc0 2024-03-23T09:48:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0ab5c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb0415d0a0 0x7ffb040b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0ab5c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb0415d0a0 0x7ffb040b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49174/0 (socket says 172.21.15.47:49174) 2024-03-23T09:48:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0ab5c640 1 -- 172.21.15.47:0/2931841843 learned_addr learned my addr 172.21.15.47:0/2931841843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb09b5a640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb0415ccc0 0x7ffb040b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0a35b640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb0406b210 0x7ffb0414d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0ab5c640 1 -- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb0415ccc0 msgr2=0x7ffb040b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0ab5c640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb0415ccc0 0x7ffb040b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0ab5c640 1 -- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb0406b210 msgr2=0x7ffb0414d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0ab5c640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb0406b210 0x7ffb0414d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0ab5c640 1 -- 172.21.15.47:0/2931841843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf40094d0 con 0x7ffb0415d0a0 2024-03-23T09:48:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0ab5c640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb0415d0a0 0x7ffb040b45b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00cf20 tx=0x7ffafc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffafaffd640 1 -- 172.21.15.47:0/2931841843 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00d840 con 0x7ffb0415d0a0 2024-03-23T09:48:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffafaffd640 1 -- 172.21.15.47:0/2931841843 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc00d9a0 con 0x7ffb0415d0a0 2024-03-23T09:48:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/2931841843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb040ad290 con 0x7ffb0415d0a0 2024-03-23T09:48:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffafaffd640 1 -- 172.21.15.47:0/2931841843 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00dd50 con 0x7ffb0415d0a0 2024-03-23T09:48:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.061+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/2931841843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb040ad850 con 0x7ffb0415d0a0 2024-03-23T09:48:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.062+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/2931841843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb04072360 con 0x7ffb0415d0a0 2024-03-23T09:48:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.063+0000 7ffafaffd640 1 -- 172.21.15.47:0/2931841843 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ffafc01e3b0 con 0x7ffb0415d0a0 2024-03-23T09:48:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.063+0000 7ffafaffd640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffae806ac00 0x7ffae806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.064+0000 7ffafaffd640 1 -- 172.21.15.47:0/2931841843 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ffafc089d70 con 0x7ffb0415d0a0 2024-03-23T09:48:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.064+0000 7ffb0a35b640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffae806ac00 0x7ffae806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.064+0000 7ffb0a35b640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffae806ac00 0x7ffae806d0c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000ab60 tx=0x7ffb00008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.068+0000 7ffafaffd640 1 -- 172.21.15.47:0/2931841843 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ffafc054f70 con 0x7ffb0415d0a0 2024-03-23T09:48:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.284+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/2931841843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ffb04066be0 con 0x7ffb0415d0a0 2024-03-23T09:48:48.285 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:48:48.285+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:48:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.293+0000 7ffafaffd640 1 -- 172.21.15.47:0/2931841843 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7ffafc01a020 con 0x7ffb0415d0a0 2024-03-23T09:48:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:48:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.295+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffae806ac00 msgr2=0x7ffae806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.295+0000 7ffb0c5bf640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffae806ac00 0x7ffae806d0c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000ab60 tx=0x7ffb00008040 comp rx=0 tx=0).stop 2024-03-23T09:48:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.295+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/2931841843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb0415d0a0 msgr2=0x7ffb040b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.295+0000 7ffb0c5bf640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb0415d0a0 0x7ffb040b45b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00cf20 tx=0x7ffafc00a710 comp rx=0 tx=0).stop 2024-03-23T09:48:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.295+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/2931841843 shutdown_connections 2024-03-23T09:48:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.295+0000 7ffb0c5bf640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffae806ac00 0x7ffae806d0c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.295+0000 7ffb0c5bf640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb0415d0a0 0x7ffb040b45b0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.295+0000 7ffb0c5bf640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb0415ccc0 0x7ffb040b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.295+0000 7ffb0c5bf640 1 --2- 172.21.15.47:0/2931841843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb0406b210 0x7ffb0414d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.295+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/2931841843 >> 172.21.15.47:0/2931841843 conn(0x7ffb040cf430 msgr2=0x7ffb040b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.295+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/2931841843 shutdown_connections 2024-03-23T09:48:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.295+0000 7ffb0c5bf640 1 -- 172.21.15.47:0/2931841843 wait complete. 2024-03-23T09:48:48.297 INFO:tasks.ceph.mds.c.smithi047.stderr: -4379> 2024-03-23T09:47:34.518+0000 7f3d23d07640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:48:48.297 INFO:tasks.ceph.mds.c.smithi047.stderr: -4049> 2024-03-23T09:47:35.693+0000 7f3d20500640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-23T09:48:48.297 INFO:tasks.ceph.mds.c.smithi047.stderr: -3891> 2024-03-23T09:47:35.876+0000 7f3d20500640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-03-23T09:47:17.860649+0000 1=1+0) n(v0 rc2024-03-23T09:47:17.976300+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x5650412f2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-23T09:47:17.860649+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-23T09:47:17.976300+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-03-23T09:48:48.298 INFO:tasks.ceph.mds.c.smithi047.stderr: -2299> 2024-03-23T09:48:07.172+0000 7f3d20500640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-03-23T09:47:17.860649+0000 1=1+0)->f(v1 m2024-03-23T09:47:17.860649+0000) n(v0 rc2024-03-23T09:47:17.976300+0000 2=1+1)->n(v0 rc2024-03-23T09:47:17.976300+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5650412f2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-23T09:47:17.860649+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-23T09:47:17.976300+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-03-23T09:48:48.306 DEBUG:teuthology.orchestra.run.smithi047:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-03-23T09:48:48.324 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:48:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.344+0000 7fc71222f740 1 -- start start 2024-03-23T09:48:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.344+0000 7fc71222f740 1 Processor -- start 2024-03-23T09:48:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc71222f740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c402e2cc00 0x55c402ec6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc71222f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c402e2d800 0x55c402ec6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc71222f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c402e2dc00 0x55c402ec7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc71222f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c402ebea00 con 0x55c402e2d800 2024-03-23T09:48:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc71222f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c402ebe780 con 0x55c402e2dc00 2024-03-23T09:48:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc71222f740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c402ebe640 con 0x55c402e2cc00 2024-03-23T09:48:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc70f1d0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c402e2d800 0x55c402ec6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc70f1d0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c402e2d800 0x55c402ec6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49220/0 (socket says 172.21.15.47:49220) 2024-03-23T09:48:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc70f1d0640 1 -- 172.21.15.47:0/412350209 learned_addr learned my addr 172.21.15.47:0/412350209 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc7101d2640 1 --2- 172.21.15.47:0/412350209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c402e2dc00 0x55c402ec7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc70f9d1640 1 --2- 172.21.15.47:0/412350209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c402e2cc00 0x55c402ec6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc70f1d0640 1 -- 172.21.15.47:0/412350209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c402e2cc00 msgr2=0x55c402ec6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc70f1d0640 1 --2- 172.21.15.47:0/412350209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c402e2cc00 0x55c402ec6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc70f1d0640 1 -- 172.21.15.47:0/412350209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c402e2dc00 msgr2=0x55c402ec7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.345+0000 7fc70f1d0640 1 --2- 172.21.15.47:0/412350209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c402e2dc00 0x55c402ec7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc70f1d0640 1 -- 172.21.15.47:0/412350209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c402e529c0 con 0x55c402e2d800 2024-03-23T09:48:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc7101d2640 1 --2- 172.21.15.47:0/412350209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c402e2dc00 0x55c402ec7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc70f1d0640 1 --2- 172.21.15.47:0/412350209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c402e2d800 0x55c402ec6b00 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x55c403c43440 tx=0x55c403cf8120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e45b12429f3232 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc70d1cc640 1 -- 172.21.15.47:0/412350209 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c402eef600 con 0x55c402e2d800 2024-03-23T09:48:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc70d1cc640 1 -- 172.21.15.47:0/412350209 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c402e8a480 con 0x55c402e2d800 2024-03-23T09:48:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc70d1cc640 1 -- 172.21.15.47:0/412350209 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c402eef1e0 con 0x55c402e2d800 2024-03-23T09:48:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc71222f740 1 -- 172.21.15.47:0/412350209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c402e52b60 con 0x55c402e2d800 2024-03-23T09:48:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc71222f740 1 -- 172.21.15.47:0/412350209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55c402e52820 con 0x55c402e2d800 2024-03-23T09:48:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc71222f740 4 waiting for MDS map... 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc70d1cc640 1 -- 172.21.15.47:0/412350209 <== mon.0 v2:172.21.15.47:3300/0 4 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x55c402e74c40 con 0x55c402e2d800 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc70d1cc640 1 -- 172.21.15.47:0/412350209 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 54) v1 ==== 2573+0+0 (secure 0 0 0) 0x55c402e7cf00 con 0x55c402e2d800 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc71222f740 4 Got MDS map 54 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc71222f740 4 datascan.main: Using metadata pool output 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.346+0000 7fc71222f740 4 datascan.main: connecting to RADOS... 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.347+0000 7fc71222f740 1 Processor -- start 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.347+0000 7fc71222f740 1 -- start start 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.347+0000 7fc71222f740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c403d01000 0x55c402ec7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.347+0000 7fc71222f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c403d01400 0x55c402ec7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.347+0000 7fc71222f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c403d01800 0x55c402ec8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.347+0000 7fc71222f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c402e5be00 con 0x55c403d01400 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.347+0000 7fc71222f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c402ebe780 con 0x55c403d01800 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.347+0000 7fc71222f740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c402ebe500 con 0x55c403d01000 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.347+0000 7fc70f9d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c403d01400 0x55c402ec7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.347+0000 7fc70f9d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c403d01400 0x55c402ec7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49230/0 (socket says 172.21.15.47:49230) 2024-03-23T09:48:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.347+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2088601954 learned_addr learned my addr 172.21.15.47:0/2088601954 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.347+0000 7fc7101d2640 1 --2- 172.21.15.47:0/2088601954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c403d01000 0x55c402ec7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc70f1d0640 1 --2- 172.21.15.47:0/2088601954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c403d01800 0x55c402ec8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2088601954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c403d01000 msgr2=0x55c402ec7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc70f9d1640 1 --2- 172.21.15.47:0/2088601954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c403d01000 0x55c402ec7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2088601954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c403d01800 msgr2=0x55c402ec8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc70f9d1640 1 --2- 172.21.15.47:0/2088601954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c403d01800 0x55c402ec8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2088601954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c402e524e0 con 0x55c403d01400 2024-03-23T09:48:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc7101d2640 1 --2- 172.21.15.47:0/2088601954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c403d01000 0x55c402ec7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc70f9d1640 1 --2- 172.21.15.47:0/2088601954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c403d01400 0x55c402ec7b80 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x55c403d0a8d0 tx=0x55c403d0cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f765d0f2a1926 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2088601954 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c402eef600 con 0x55c403d01400 2024-03-23T09:48:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2088601954 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c403c06600 con 0x55c403d01400 2024-03-23T09:48:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2088601954 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c402eef1e0 con 0x55c403d01400 2024-03-23T09:48:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc71222f740 1 -- 172.21.15.47:0/2088601954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c403d01400 msgr2=0x55c402ec7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc71222f740 1 --2- 172.21.15.47:0/2088601954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c403d01400 0x55c402ec7b80 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x55c403d0a8d0 tx=0x55c403d0cc00 comp rx=0 tx=0).stop 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc71222f740 1 -- 172.21.15.47:0/2088601954 shutdown_connections 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc71222f740 1 --2- 172.21.15.47:0/2088601954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c403d01800 0x55c402ec8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc71222f740 1 --2- 172.21.15.47:0/2088601954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c403d01400 0x55c402ec7b80 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc71222f740 1 --2- 172.21.15.47:0/2088601954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c403d01000 0x55c402ec7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc71222f740 1 -- 172.21.15.47:0/2088601954 >> 172.21.15.47:0/2088601954 conn(0x55c403d00c00 msgr2=0x55c402f15000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc71222f740 1 -- 172.21.15.47:0/2088601954 shutdown_connections 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.348+0000 7fc71222f740 1 -- 172.21.15.47:0/2088601954 wait complete. 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.349+0000 7fc71222f740 1 Processor -- start 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.349+0000 7fc71222f740 1 -- start start 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.349+0000 7fc71222f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c403d01000 0x55c402ec9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.349+0000 7fc71222f740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c403d01400 0x55c402ec8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc71222f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c403d01800 0x55c402ec8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc71222f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c402ebe3c0 con 0x55c403d01800 2024-03-23T09:48:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc71222f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c402ebe280 con 0x55c403d01000 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc71222f740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c402ebe140 con 0x55c403d01400 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc70f1d0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c403d01800 0x55c402ec8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc70f1d0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c403d01800 0x55c402ec8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49238/0 (socket says 172.21.15.47:49238) 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 learned_addr learned my addr 172.21.15.47:0/2266166782 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc7101d2640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c403d01000 0x55c402ec9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc70f9d1640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c403d01400 0x55c402ec8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c403d01400 msgr2=0x55c402ec8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc70f1d0640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c403d01400 0x55c402ec8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c403d01000 msgr2=0x55c402ec9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc70f1d0640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c403d01000 0x55c402ec9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c402e529c0 con 0x55c403d01800 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc70f1d0640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c403d01800 0x55c402ec8680 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x55c403d0b5c0 tx=0x55c403bc3140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc7099c5640 1 -- 172.21.15.47:0/2266166782 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c402eeef20 con 0x55c403d01800 2024-03-23T09:48:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc7099c5640 1 -- 172.21.15.47:0/2266166782 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c403c06900 con 0x55c403d01800 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c402e52b60 con 0x55c403d01800 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc7099c5640 1 -- 172.21.15.47:0/2266166782 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c402eef600 con 0x55c403d01800 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c402e521a0 con 0x55c403d01800 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.350+0000 7fc71222f740 4 datascan.init: resolving metadata pool 12 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.351+0000 7fc7099c5640 1 -- 172.21.15.47:0/2266166782 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x55c402e92840 con 0x55c403d01800 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.352+0000 7fc7099c5640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55c403d65800 0x55c402ec9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.352+0000 7fc7099c5640 1 -- 172.21.15.47:0/2266166782 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x55c402e74a80 con 0x55c403d01800 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.352+0000 7fc71222f740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.352+0000 7fc71222f740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.352+0000 7fc71222f740 4 datascan.main: opening data pool 'cephfs_data' 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.352+0000 7fc71222f740 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55c403d65c00 0x55c402ec8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.352+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403184000 con 0x55c403d65c00 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.352+0000 7fc7101d2640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55c403d65800 0x55c402ec9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.352+0000 7fc70f9d1640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55c403d65c00 0x55c402ec8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.353+0000 7fc7101d2640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55c403d65800 0x55c402ec9700 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55c402e9da70 tx=0x55c403d0cf60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.353+0000 7fc70f9d1640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55c403d65c00 0x55c402ec8100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.353+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.7 v2:172.21.15.175:6824/38174 1 ==== osd_op_reply(1 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403d65c00 2024-03-23T09:48:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.353+0000 7fc70b1c8640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55c403d01c00 0x55c403198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.353+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403d64400 con 0x55c403d01c00 2024-03-23T09:48:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.353+0000 7fc7101d2640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55c403d01c00 0x55c403198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.354+0000 7fc7101d2640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55c403d01c00 0x55c403198000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.356+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.6 v2:172.21.15.175:6808/38160 1 ==== osd_op_reply(2 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d01c00 2024-03-23T09:48:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.356+0000 7fc70a9c7640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55c403184000 0x55c403198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.356+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403185800 con 0x55c403184000 2024-03-23T09:48:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.357+0000 7fc70f9d1640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55c403184000 0x55c403198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.357+0000 7fc70f9d1640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55c403184000 0x55c403198580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.357+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== osd_op_reply(3 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403184000 2024-03-23T09:48:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.357+0000 7fc70b1c8640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55c403d64000 0x55c403198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.357+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b0000 con 0x55c403d64000 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.358+0000 7fc70f1d0640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55c403d64000 0x55c403198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.358+0000 7fc70f1d0640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55c403d64000 0x55c403198b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.358+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.4 v2:172.21.15.175:6800/38159 1 ==== osd_op_reply(4 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c403d64000 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.358+0000 7fc70a9c7640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55c403185c00 0x55c403199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.358+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b0c00 con 0x55c403185c00 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.358+0000 7fc7101d2640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55c403185c00 0x55c403199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.359+0000 7fc7101d2640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55c403185c00 0x55c403199600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.360+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.2 v2:172.21.15.47:6800/37678 1 ==== osd_op_reply(5 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403185c00 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.361+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b0400 con 0x55c403185c00 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.361+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.2 v2:172.21.15.47:6800/37678 2 ==== osd_op_reply(6 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403185c00 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.361+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b1000 con 0x55c403d65c00 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.361+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.7 v2:172.21.15.175:6824/38174 2 ==== osd_op_reply(7 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403d65c00 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.361+0000 7fc70b1c8640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55c4031b0800 0x55c403199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.361+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b0c00 con 0x55c4031b0800 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.362+0000 7fc70f9d1640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55c4031b0800 0x55c403199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.362+0000 7fc70f9d1640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55c4031b0800 0x55c403199080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.365+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.5 v2:172.21.15.175:6816/38161 1 ==== osd_op_reply(8 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c4031b0800 2024-03-23T09:48:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.365+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b1400 con 0x55c403d01c00 2024-03-23T09:48:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.365+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.6 v2:172.21.15.175:6808/38160 2 ==== osd_op_reply(9 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d01c00 2024-03-23T09:48:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.365+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403d64400 con 0x55c403d01c00 2024-03-23T09:48:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.366+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.6 v2:172.21.15.175:6808/38160 3 ==== osd_op_reply(10 [pgnls start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d01c00 2024-03-23T09:48:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.366+0000 7fc70a9c7640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55c4031b0400 0x55c403199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.366+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b1800 con 0x55c4031b0400 2024-03-23T09:48:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.366+0000 7fc70f1d0640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55c4031b0400 0x55c403199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.366+0000 7fc70f1d0640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55c4031b0400 0x55c403199b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.368+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.0 v2:172.21.15.47:6816/37679 1 ==== osd_op_reply(11 [pgnls start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c4031b0400 2024-03-23T09:48:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.368+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403c09400 con 0x55c403d01c00 2024-03-23T09:48:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.369+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.6 v2:172.21.15.175:6808/38160 4 ==== osd_op_reply(12 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d01c00 2024-03-23T09:48:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.369+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b1c00 con 0x55c403184000 2024-03-23T09:48:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.369+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.1 v2:172.21.15.47:6824/37680 2 ==== osd_op_reply(13 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403184000 2024-03-23T09:48:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.369+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403c09000 con 0x55c403185c00 2024-03-23T09:48:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.369+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.2 v2:172.21.15.47:6800/37678 3 ==== osd_op_reply(14 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403185c00 2024-03-23T09:48:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.369+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031be000 con 0x55c403d01c00 2024-03-23T09:48:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.370+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.6 v2:172.21.15.175:6808/38160 5 ==== osd_op_reply(15 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d01c00 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.370+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bf400 con 0x55c403185c00 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.370+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.2 v2:172.21.15.47:6800/37678 4 ==== osd_op_reply(16 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403185c00 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.370+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031be400 con 0x55c4031b0800 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.370+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.5 v2:172.21.15.175:6816/38161 2 ==== osd_op_reply(17 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c4031b0800 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.370+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bf800 con 0x55c403184000 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.370+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.1 v2:172.21.15.47:6824/37680 3 ==== osd_op_reply(18 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403184000 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.370+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031be800 con 0x55c4031b0400 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.371+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.0 v2:172.21.15.47:6816/37679 2 ==== osd_op_reply(19 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c4031b0400 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.371+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bfc00 con 0x55c403185c00 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.371+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.2 v2:172.21.15.47:6800/37678 5 ==== osd_op_reply(20 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403185c00 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.371+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bec00 con 0x55c403185c00 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.371+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.2 v2:172.21.15.47:6800/37678 6 ==== osd_op_reply(21 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403185c00 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.371+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b0000 con 0x55c403184000 2024-03-23T09:48:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.371+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.1 v2:172.21.15.47:6824/37680 4 ==== osd_op_reply(22 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403184000 2024-03-23T09:48:48.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.371+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bf000 con 0x55c403184000 2024-03-23T09:48:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.372+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.1 v2:172.21.15.47:6824/37680 5 ==== osd_op_reply(23 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403184000 2024-03-23T09:48:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.372+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b1800 con 0x55c4031b0800 2024-03-23T09:48:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.372+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.5 v2:172.21.15.175:6816/38161 3 ==== osd_op_reply(24 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c4031b0800 2024-03-23T09:48:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.372+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403c09400 con 0x55c403d64000 2024-03-23T09:48:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.372+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.4 v2:172.21.15.175:6800/38159 2 ==== osd_op_reply(25 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c403d64000 2024-03-23T09:48:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.372+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b1000 con 0x55c403d65c00 2024-03-23T09:48:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.373+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.7 v2:172.21.15.175:6824/38174 3 ==== osd_op_reply(26 [pgnls start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403d65c00 2024-03-23T09:48:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.373+0000 7fc70a9c7640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55c403d65400 0x55c40319a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.373+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403d65000 con 0x55c403d65400 2024-03-23T09:48:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.373+0000 7fc7101d2640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55c403d65400 0x55c40319a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.373+0000 7fc7101d2640 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55c403d65400 0x55c40319a100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.375+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.3 v2:172.21.15.47:6808/37683 1 ==== osd_op_reply(27 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d65400 2024-03-23T09:48:48.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.375+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b0c00 con 0x55c403d01c00 2024-03-23T09:48:48.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.376+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.6 v2:172.21.15.175:6808/38160 6 ==== osd_op_reply(28 [pgnls start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d01c00 2024-03-23T09:48:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.376+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403d64c00 con 0x55c403d65c00 2024-03-23T09:48:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.376+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.7 v2:172.21.15.175:6824/38174 4 ==== osd_op_reply(29 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403d65c00 2024-03-23T09:48:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.376+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b1c00 con 0x55c403185c00 2024-03-23T09:48:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.376+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.2 v2:172.21.15.47:6800/37678 7 ==== osd_op_reply(30 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403185c00 2024-03-23T09:48:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.376+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403d64400 con 0x55c403d64000 2024-03-23T09:48:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.376+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.4 v2:172.21.15.175:6800/38159 3 ==== osd_op_reply(31 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c403d64000 2024-03-23T09:48:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.376+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403185800 con 0x55c403184000 2024-03-23T09:48:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.377+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.1 v2:172.21.15.47:6824/37680 6 ==== osd_op_reply(32 [pgnls start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403184000 2024-03-23T09:48:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.377+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b1400 con 0x55c403d01c00 2024-03-23T09:48:48.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.377+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.6 v2:172.21.15.175:6808/38160 7 ==== osd_op_reply(33 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d01c00 2024-03-23T09:48:48.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.377+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bf800 con 0x55c403d64000 2024-03-23T09:48:48.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.377+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.4 v2:172.21.15.175:6800/38159 4 ==== osd_op_reply(34 [pgnls start_epoch 89 out=49b] v85'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c403d64000 2024-03-23T09:48:48.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.377+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031be800 con 0x55c403184000 2024-03-23T09:48:48.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.378+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.1 v2:172.21.15.47:6824/37680 7 ==== osd_op_reply(35 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403184000 2024-03-23T09:48:48.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.378+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bf000 con 0x55c403d65400 2024-03-23T09:48:48.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.378+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.3 v2:172.21.15.47:6808/37683 2 ==== osd_op_reply(36 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d65400 2024-03-23T09:48:48.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.378+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031be400 con 0x55c4031b0800 2024-03-23T09:48:48.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.378+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.5 v2:172.21.15.175:6816/38161 4 ==== osd_op_reply(37 [pgnls start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c4031b0800 2024-03-23T09:48:48.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.378+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403c09000 con 0x55c403d01c00 2024-03-23T09:48:48.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.379+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.6 v2:172.21.15.175:6808/38160 8 ==== osd_op_reply(38 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d01c00 2024-03-23T09:48:48.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.379+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031be000 con 0x55c403d65400 2024-03-23T09:48:48.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.379+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.3 v2:172.21.15.47:6808/37683 3 ==== osd_op_reply(39 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d65400 2024-03-23T09:48:48.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.379+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403d64c00 con 0x55c403185c00 2024-03-23T09:48:48.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.379+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.2 v2:172.21.15.47:6800/37678 8 ==== osd_op_reply(40 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403185c00 2024-03-23T09:48:48.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.379+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bf400 con 0x55c403d64000 2024-03-23T09:48:48.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.380+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.4 v2:172.21.15.175:6800/38159 5 ==== osd_op_reply(41 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c403d64000 2024-03-23T09:48:48.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.380+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b0000 con 0x55c403d64000 2024-03-23T09:48:48.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.380+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.4 v2:172.21.15.175:6800/38159 6 ==== osd_op_reply(42 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c403d64000 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.380+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bfc00 con 0x55c403d64000 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.380+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.4 v2:172.21.15.175:6800/38159 7 ==== osd_op_reply(43 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c403d64000 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.380+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b1800 con 0x55c403184000 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.380+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.1 v2:172.21.15.47:6824/37680 8 ==== osd_op_reply(44 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403184000 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.380+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bec00 con 0x55c403d64000 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.381+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.4 v2:172.21.15.175:6800/38159 8 ==== osd_op_reply(45 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c403d64000 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.381+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b1000 con 0x55c4031b0400 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.381+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.0 v2:172.21.15.47:6816/37679 3 ==== osd_op_reply(46 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c4031b0400 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.381+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bf000 con 0x55c403d65c00 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.381+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.7 v2:172.21.15.175:6824/38174 5 ==== osd_op_reply(47 [pgnls start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403d65c00 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.381+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bf800 con 0x55c403d65c00 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.382+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.7 v2:172.21.15.175:6824/38174 6 ==== osd_op_reply(48 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403d65c00 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.382+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b0c00 con 0x55c4031b0400 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.382+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.0 v2:172.21.15.47:6816/37679 4 ==== osd_op_reply(49 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c4031b0400 2024-03-23T09:48:48.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.382+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bf400 con 0x55c403184000 2024-03-23T09:48:48.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.382+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.1 v2:172.21.15.47:6824/37680 9 ==== osd_op_reply(50 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403184000 2024-03-23T09:48:48.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.382+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b1c00 con 0x55c403d65c00 2024-03-23T09:48:48.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.382+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.7 v2:172.21.15.175:6824/38174 7 ==== osd_op_reply(51 [pgnls start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403d65c00 2024-03-23T09:48:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.383+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bfc00 con 0x55c403d65c00 2024-03-23T09:48:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.383+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.7 v2:172.21.15.175:6824/38174 8 ==== osd_op_reply(52 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403d65c00 2024-03-23T09:48:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.383+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b1400 con 0x55c4031b0800 2024-03-23T09:48:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.383+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.5 v2:172.21.15.175:6816/38161 5 ==== osd_op_reply(53 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c4031b0800 2024-03-23T09:48:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.383+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bec00 con 0x55c403185c00 2024-03-23T09:48:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.383+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.2 v2:172.21.15.47:6800/37678 9 ==== osd_op_reply(54 [pgnls start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403185c00 2024-03-23T09:48:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.383+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403d65000 con 0x55c403d64000 2024-03-23T09:48:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.384+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.4 v2:172.21.15.175:6800/38159 9 ==== osd_op_reply(55 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c403d64000 2024-03-23T09:48:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.384+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031b0000 con 0x55c403d64000 2024-03-23T09:48:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.384+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.4 v2:172.21.15.175:6800/38159 10 ==== osd_op_reply(56 [pgnls start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c403d64000 2024-03-23T09:48:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.384+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031be800 con 0x55c403d65c00 2024-03-23T09:48:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.385+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.7 v2:172.21.15.175:6824/38174 9 ==== osd_op_reply(57 [pgnls start_epoch 89 out=81b] v85'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403d65c00 2024-03-23T09:48:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.385+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403184400 con 0x55c403d65c00 2024-03-23T09:48:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.385+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.7 v2:172.21.15.175:6824/38174 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403d65c00 2024-03-23T09:48:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.385+0000 7fc71222f740 10 datascan.operator(): handling object 1099511627777.0 2024-03-23T09:48:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.385+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403184800 con 0x55c403d65c00 2024-03-23T09:48:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.387+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.7 v2:172.21.15.175:6824/38174 11 ==== osd_op_reply(59 10000000001.00000000 [call] v89'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403d65c00 2024-03-23T09:48:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.387+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403184c00 con 0x55c403184000 2024-03-23T09:48:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.387+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.1 v2:172.21.15.47:6824/37680 10 ==== osd_op_reply(60 [pgnls start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c403184000 2024-03-23T09:48:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.387+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031be400 con 0x55c4031b0800 2024-03-23T09:48:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.387+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.5 v2:172.21.15.175:6816/38161 6 ==== osd_op_reply(61 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c4031b0800 2024-03-23T09:48:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.387+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403d64400 con 0x55c4031b0800 2024-03-23T09:48:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.388+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 <== osd.5 v2:172.21.15.175:6816/38161 7 ==== osd_op_reply(62 [pgnls start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2b40 con 0x55c4031b0800 2024-03-23T09:48:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.388+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bf000 con 0x55c403d01c00 2024-03-23T09:48:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.388+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.6 v2:172.21.15.175:6808/38160 9 ==== osd_op_reply(63 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d01c00 2024-03-23T09:48:48.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.388+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c403c09400 con 0x55c403d64000 2024-03-23T09:48:48.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.388+0000 7fc70f1d0640 1 -- 172.21.15.47:0/2266166782 <== osd.4 v2:172.21.15.175:6800/38159 11 ==== osd_op_reply(64 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2fc0 con 0x55c403d64000 2024-03-23T09:48:48.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.389+0000 7fc70a9c7640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bf800 con 0x55c403d01c00 2024-03-23T09:48:48.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.389+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.6 v2:172.21.15.175:6808/38160 10 ==== osd_op_reply(65 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d01c00 2024-03-23T09:48:48.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.389+0000 7fc70b1c8640 1 -- 172.21.15.47:0/2266166782 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55c4031bfc00 con 0x55c403d01c00 2024-03-23T09:48:48.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc7101d2640 1 -- 172.21.15.47:0/2266166782 <== osd.6 v2:172.21.15.175:6808/38160 11 ==== osd_op_reply(66 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c403ce2d80 con 0x55c403d01c00 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55c4031b0400 msgr2=0x55c403199b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55c4031b0400 0x55c403199b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55c403184000 msgr2=0x55c403198580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55c403184000 0x55c403198580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55c403185c00 msgr2=0x55c403199600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55c403185c00 0x55c403199600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55c403d65400 msgr2=0x55c40319a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55c403d65400 0x55c40319a100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55c403d64000 msgr2=0x55c403198b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55c403d64000 0x55c403198b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55c4031b0800 msgr2=0x55c403199080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 reap_dead start 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55c4031b0800 0x55c403199080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55c403d01c00 msgr2=0x55c403198000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55c403d01c00 0x55c403198000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55c403d65c00 msgr2=0x55c402ec8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55c403d65c00 0x55c402ec8100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55c403d65800 msgr2=0x55c402ec9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55c403d65800 0x55c402ec9700 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55c402e9da70 tx=0x55c403d0cf60 comp rx=0 tx=0).stop 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c403d01800 msgr2=0x55c402ec8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/2266166782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c403d01800 0x55c402ec8680 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x55c403d0b5c0 tx=0x55c403bc3140 comp rx=0 tx=0).stop 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc70f9d1640 1 -- 172.21.15.47:0/2266166782 reap_dead start 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 shutdown_connections 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 >> 172.21.15.47:0/2266166782 conn(0x55c403d00c00 msgr2=0x55c402f15000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 shutdown_connections 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/2266166782 wait complete. 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/412350209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c402e2d800 msgr2=0x55c402ec6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/412350209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c402e2d800 0x55c402ec6b00 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x55c403c43440 tx=0x55c403cf8120 comp rx=0 tx=0).stop 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/412350209 shutdown_connections 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/412350209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c402e2dc00 0x55c402ec7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/412350209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c402e2d800 0x55c402ec6b00 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 --2- 172.21.15.47:0/412350209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c402e2cc00 0x55c402ec6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/412350209 >> 172.21.15.47:0/412350209 conn(0x55c402e2d000 msgr2=0x55c402f14800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/412350209 shutdown_connections 2024-03-23T09:48:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.390+0000 7fc71222f740 1 -- 172.21.15.47:0/412350209 wait complete. 2024-03-23T09:48:48.394 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.088325, result: 2024-03-23T09:48:48.395 DEBUG:teuthology.orchestra.run.smithi047:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-03-23T09:48:48.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.473+0000 7f4cc0e0d740 1 -- start start 2024-03-23T09:48:48.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.473+0000 7f4cc0e0d740 1 Processor -- start 2024-03-23T09:48:48.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.474+0000 7f4cc0e0d740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f353bf0c00 0x55f353c8a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.474+0000 7f4cc0e0d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f353bf1800 0x55f353c8ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.474+0000 7f4cc0e0d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f353bf1c00 0x55f353c8b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.474+0000 7f4cc0e0d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f353c82a00 con 0x55f353bf1c00 2024-03-23T09:48:48.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.474+0000 7f4cc0e0d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f353c82780 con 0x55f353bf1800 2024-03-23T09:48:48.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.474+0000 7f4cc0e0d740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f353c82640 con 0x55f353bf0c00 2024-03-23T09:48:48.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.474+0000 7f4cbf1f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f353bf1c00 0x55f353c8b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.474+0000 7f4cbf1f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f353bf1c00 0x55f353c8b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49254/0 (socket says 172.21.15.47:49254) 2024-03-23T09:48:48.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.474+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/3214003736 learned_addr learned my addr 172.21.15.47:0/3214003736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:48.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.474+0000 7f4cbe9ef640 1 --2- 172.21.15.47:0/3214003736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f353bf0c00 0x55f353c8a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.474+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/3214003736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f353bf1800 0x55f353c8ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.474+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/3214003736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f353bf0c00 msgr2=0x55f353c8a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cbf1f0640 1 --2- 172.21.15.47:0/3214003736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f353bf0c00 0x55f353c8a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/3214003736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f353bf1800 msgr2=0x55f353c8ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cbf1f0640 1 --2- 172.21.15.47:0/3214003736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f353bf1800 0x55f353c8ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/3214003736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f353c169c0 con 0x55f353bf1c00 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cbe9ef640 1 --2- 172.21.15.47:0/3214003736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f353bf0c00 0x55f353c8a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/3214003736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f353bf1800 0x55f353c8ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cbf1f0640 1 --2- 172.21.15.47:0/3214003736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f353bf1c00 0x55f353c8b080 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x55f354a09290 tx=0x55f354ab4120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a442ef6ebbe1771f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cbc1ea640 1 -- 172.21.15.47:0/3214003736 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f353cb3080 con 0x55f353bf1c00 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cbc1ea640 1 -- 172.21.15.47:0/3214003736 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f3549c2300 con 0x55f353bf1c00 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cbc1ea640 1 -- 172.21.15.47:0/3214003736 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f353cb3600 con 0x55f353bf1c00 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/3214003736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f353c16d00 con 0x55f353bf1c00 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/3214003736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f353c169c0 con 0x55f353bf1c00 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cc0e0d740 4 waiting for MDS map... 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cbc1ea640 1 -- 172.21.15.47:0/3214003736 <== mon.0 v2:172.21.15.47:3300/0 4 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x55f353c38c40 con 0x55f353bf1c00 2024-03-23T09:48:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cbc1ea640 1 -- 172.21.15.47:0/3214003736 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 54) v1 ==== 2573+0+0 (secure 0 0 0) 0x55f353c41800 con 0x55f353bf1c00 2024-03-23T09:48:48.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cc0e0d740 4 Got MDS map 54 2024-03-23T09:48:48.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cc0e0d740 4 datascan.main: Using metadata pool output 2024-03-23T09:48:48.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.475+0000 7f4cc0e0d740 4 datascan.main: connecting to RADOS... 2024-03-23T09:48:48.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.476+0000 7f4cc0e0d740 1 Processor -- start 2024-03-23T09:48:48.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.476+0000 7f4cc0e0d740 1 -- start start 2024-03-23T09:48:48.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.476+0000 7f4cc0e0d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3549cfc00 0x55f353c8b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.476+0000 7f4cc0e0d740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f354b06000 0x55f353c8bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.476+0000 7f4cc0e0d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f354b06400 0x55f353c8c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.476+0000 7f4cc0e0d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f353c1fe00 con 0x55f354b06400 2024-03-23T09:48:48.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.476+0000 7f4cc0e0d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f353c82780 con 0x55f3549cfc00 2024-03-23T09:48:48.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.476+0000 7f4cc0e0d740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f353c82a00 con 0x55f354b06000 2024-03-23T09:48:48.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.476+0000 7f4cbe1ee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f354b06400 0x55f353c8c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.476+0000 7f4cbe1ee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f354b06400 0x55f353c8c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49266/0 (socket says 172.21.15.47:49266) 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/4159321604 learned_addr learned my addr 172.21.15.47:0/4159321604 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cbe9ef640 1 --2- 172.21.15.47:0/4159321604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f354b06000 0x55f353c8bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/4159321604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3549cfc00 0x55f353c8b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/4159321604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f354b06000 msgr2=0x55f353c8bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/4159321604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f354b06000 0x55f353c8bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/4159321604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3549cfc00 msgr2=0x55f353c8b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/4159321604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3549cfc00 0x55f353c8b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/4159321604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f353c16b60 con 0x55f354b06400 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/4159321604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f354b06400 0x55f353c8c100 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x55f354aae930 tx=0x55f354acecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14629f545c0aa27d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cba1e6640 1 -- 172.21.15.47:0/4159321604 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f353cb3080 con 0x55f354b06400 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cba1e6640 1 -- 172.21.15.47:0/4159321604 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f353c4e900 con 0x55f354b06400 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cba1e6640 1 -- 172.21.15.47:0/4159321604 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f353cb3600 con 0x55f354b06400 2024-03-23T09:48:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/4159321604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f354b06400 msgr2=0x55f353c8c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/4159321604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f354b06400 0x55f353c8c100 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x55f354aae930 tx=0x55f354acecc0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/4159321604 shutdown_connections 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/4159321604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f354b06400 0x55f353c8c100 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/4159321604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f354b06000 0x55f353c8bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/4159321604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3549cfc00 0x55f353c8b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.477+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/4159321604 >> 172.21.15.47:0/4159321604 conn(0x55f3549cf800 msgr2=0x55f353cd9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.478+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/4159321604 shutdown_connections 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.478+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/4159321604 wait complete. 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.478+0000 7f4cc0e0d740 1 Processor -- start 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.478+0000 7f4cc0e0d740 1 -- start start 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cc0e0d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3549cf800 0x55f353c8d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cc0e0d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f354b07000 0x55f353c8cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cc0e0d740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f354b06c00 0x55f353c8c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cc0e0d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f353c82500 con 0x55f3549cf800 2024-03-23T09:48:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cc0e0d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f353c823c0 con 0x55f354b07000 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cc0e0d740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f353c82280 con 0x55f354b06c00 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cbe1ee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3549cf800 0x55f353c8d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cbe1ee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3549cf800 0x55f353c8d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49268/0 (socket says 172.21.15.47:49268) 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 learned_addr learned my addr 172.21.15.47:0/118777466 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cbe9ef640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f354b07000 0x55f353c8cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f354b06c00 0x55f353c8c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f354b06c00 msgr2=0x55f353c8c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f354b06c00 0x55f353c8c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f354b07000 msgr2=0x55f353c8cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f354b07000 0x55f353c8cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f353c16820 con 0x55f3549cf800 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3549cf800 0x55f353c8d180 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x55f354aaf5f0 tx=0x55f354acec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cb89e3640 1 -- 172.21.15.47:0/118777466 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f353cb2dc0 con 0x55f3549cf800 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cb89e3640 1 -- 172.21.15.47:0/118777466 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f353c4e480 con 0x55f3549cf800 2024-03-23T09:48:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f353c164e0 con 0x55f3549cf800 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cb89e3640 1 -- 172.21.15.47:0/118777466 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f353cb3080 con 0x55f3549cf800 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f353c161a0 con 0x55f3549cf800 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.479+0000 7f4cc0e0d740 4 datascan.init: resolving metadata pool 12 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.481+0000 7f4cb89e3640 1 -- 172.21.15.47:0/118777466 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x55f353c57080 con 0x55f3549cf800 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.481+0000 7f4cb89e3640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55f353f4e000 0x55f353c8d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.481+0000 7f4cb89e3640 1 -- 172.21.15.47:0/118777466 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x55f353c38a80 con 0x55f3549cf800 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.481+0000 7f4cc0e0d740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.481+0000 7f4cc0e0d740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.481+0000 7f4cc0e0d740 4 datascan.main: opening data pool 'cephfs_data' 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.481+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55f354b06800 0x55f353c8c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.481+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f354b06400 con 0x55f354b06800 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.481+0000 7f4cbe9ef640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55f353f4e000 0x55f353c8d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.481+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55f354b06800 0x55f353c8c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.482+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55f354b06800 0x55f353c8c100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.482+0000 7f4cbe9ef640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55f353f4e000 0x55f353c8d700 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55f35496e7e0 tx=0x55f354acfda0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.482+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.5 v2:172.21.15.175:6816/38161 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv24 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55f354aa6b40 con 0x55f354b06800 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.482+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55f354b06000 0x55f353c8bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.482+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4f000 con 0x55f354b06000 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.482+0000 7f4cbf1f0640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55f354b06000 0x55f353c8bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.482+0000 7f4cbf1f0640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55f354b06000 0x55f353c8bb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.483+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.483+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55f353f4f400 0x55f353c8b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.483+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4f800 con 0x55f353f4f400 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.483+0000 7f4cbe9ef640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55f353f4f400 0x55f353c8b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.483+0000 7f4cbe9ef640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55f353f4f400 0x55f353c8b600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.484+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.484+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6c000 con 0x55f354b06000 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.484+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.484+0000 7f4cba1e6640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55f353f6c400 0x55f353f72000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.484+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6c800 con 0x55f353f6c400 2024-03-23T09:48:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.484+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55f353f6c400 0x55f353f72000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.484+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55f353f6c400 0x55f353f72000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.485+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.485+0000 7f4cb99e5640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55f353f6d000 0x55f353f72580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.485+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6d400 con 0x55f353f6d000 2024-03-23T09:48:48.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.485+0000 7f4cbf1f0640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55f353f6d000 0x55f353f72580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.485+0000 7f4cbf1f0640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55f353f6d000 0x55f353f72580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.486+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.486+0000 7f4cba1e6640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55f353f6cc00 0x55f353f72b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.486+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6d800 con 0x55f353f6cc00 2024-03-23T09:48:48.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.486+0000 7f4cbe9ef640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55f353f6cc00 0x55f353f72b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.486+0000 7f4cbe9ef640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55f353f6cc00 0x55f353f72b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.486+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.486+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e000 con 0x55f353f6cc00 2024-03-23T09:48:48.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.487+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.487+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6dc00 con 0x55f353f4f400 2024-03-23T09:48:48.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.487+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.487+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e400 con 0x55f354b06800 2024-03-23T09:48:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.487+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.5 v2:172.21.15.175:6816/38161 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f354b06800 2024-03-23T09:48:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.487+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e000 con 0x55f354b06000 2024-03-23T09:48:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.488+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.488+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e800 con 0x55f354b06000 2024-03-23T09:48:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.488+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.488+0000 7f4cba1e6640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55f353f6d800 0x55f353f73080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.488+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4ec00 con 0x55f353f6d800 2024-03-23T09:48:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.488+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55f353f6d800 0x55f353f73080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.488+0000 7f4cbe1ee640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55f353f6d800 0x55f353f73080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.489+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.0 v2:172.21.15.47:6816/37679 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6d800 2024-03-23T09:48:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.489+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4ec00 con 0x55f354b06000 2024-03-23T09:48:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.489+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.489+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4f800 con 0x55f353f6c400 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.489+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.489+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4f000 con 0x55f353f6cc00 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.490+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.490+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e400 con 0x55f354b06000 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.490+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.490+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f3549cf400 con 0x55f353f6cc00 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.490+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.490+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e000 con 0x55f354b06800 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.490+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.5 v2:172.21.15.175:6816/38161 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f354b06800 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.490+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f354b06400 con 0x55f353f6c400 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.491+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.491+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6c800 con 0x55f353f6d800 2024-03-23T09:48:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.491+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.0 v2:172.21.15.47:6816/37679 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6d800 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.491+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7ec00 con 0x55f353f6cc00 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.491+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.491+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6c000 con 0x55f353f6cc00 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.491+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.491+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7f000 con 0x55f353f6c400 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.492+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.492+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6d400 con 0x55f353f6c400 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.492+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.492+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7f400 con 0x55f354b06800 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.492+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.5 v2:172.21.15.175:6816/38161 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f354b06800 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.492+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e000 con 0x55f353f6d000 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.493+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.493+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7f800 con 0x55f353f4f400 2024-03-23T09:48:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.493+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.493+0000 7f4cba1e6640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55f353f7f000 0x55f353f73600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.493+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f3549cf400 con 0x55f353f7f000 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.493+0000 7f4cbf1f0640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55f353f7f000 0x55f353f73600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.493+0000 7f4cbf1f0640 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55f353f7f000 0x55f353f73600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.494+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.3 v2:172.21.15.47:6808/37683 1 ==== osd_op_reply(29 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f7f000 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.494+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7fc00 con 0x55f354b06000 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.494+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.494+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4f800 con 0x55f353f4f400 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.494+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.494+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e800 con 0x55f353f6cc00 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.494+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 7 ==== osd_op_reply(32 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.494+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4f000 con 0x55f353f6d000 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.495+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.495+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e400 con 0x55f353f6c400 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.495+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.495+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f354b06400 con 0x55f354b06000 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.495+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 8 ==== osd_op_reply(35 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.495+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e000 con 0x55f353f6d000 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.496+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 89 out=49b] v85'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.496+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6c800 con 0x55f353f6c400 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.496+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.496+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7fc00 con 0x55f353f7f000 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.496+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.3 v2:172.21.15.47:6808/37683 2 ==== osd_op_reply(38 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f7f000 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.496+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6dc00 con 0x55f354b06800 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.497+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.5 v2:172.21.15.175:6816/38161 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f354b06800 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.497+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f3549cf400 con 0x55f354b06000 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.497+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 9 ==== osd_op_reply(40 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.497+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6c000 con 0x55f353f7f000 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.497+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.3 v2:172.21.15.47:6808/37683 3 ==== osd_op_reply(41 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f7f000 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.497+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4ec00 con 0x55f353f6cc00 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.497+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.497+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7ec00 con 0x55f353f6d000 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.498+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.498+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e000 con 0x55f353f6d000 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.498+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.498+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f90000 con 0x55f353f6d000 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.498+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.498+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7fc00 con 0x55f353f6c400 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.498+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.498+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f90400 con 0x55f353f6d000 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.499+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 8 ==== osd_op_reply(47 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.499+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7ec00 con 0x55f353f6d800 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.499+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.0 v2:172.21.15.47:6816/37679 3 ==== osd_op_reply(48 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6d800 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.499+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f90800 con 0x55f353f4f400 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.499+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.499+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6c000 con 0x55f353f4f400 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.500+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 6 ==== osd_op_reply(50 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.500+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f90c00 con 0x55f353f6d800 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.500+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.0 v2:172.21.15.47:6816/37679 4 ==== osd_op_reply(51 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6d800 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.500+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f3549cf400 con 0x55f353f6c400 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.500+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 9 ==== osd_op_reply(52 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.500+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f91000 con 0x55f353f4f400 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.500+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.500+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f354b06400 con 0x55f353f4f400 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.501+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.501+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f91400 con 0x55f354b06800 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.501+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.5 v2:172.21.15.175:6816/38161 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f354b06800 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.501+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4f000 con 0x55f353f6cc00 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.501+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.501+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f91800 con 0x55f353f6d000 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.502+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.502+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7f800 con 0x55f353f6d000 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.502+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 10 ==== osd_op_reply(58 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.502+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f91c00 con 0x55f353f4f400 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.502+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 9 ==== osd_op_reply(59 [pgnls-filter start_epoch 89 out=81b] v89'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.502+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4fc00 con 0x55f353f4f400 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.502+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.502+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e800 con 0x55f354b06000 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.503+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 10 ==== osd_op_reply(61 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.503+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6c000 con 0x55f353f6c400 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.503+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 10 ==== osd_op_reply(62 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.503+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7f400 con 0x55f353f6d000 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.503+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 11 ==== osd_op_reply(63 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.503+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6d400 con 0x55f353f6cc00 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.504+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.504+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e400 con 0x55f353f6cc00 2024-03-23T09:48:48.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.504+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 11 ==== osd_op_reply(65 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.504+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6c800 con 0x55f353f4f400 2024-03-23T09:48:48.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.504+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.504+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7fc00 con 0x55f354b06800 2024-03-23T09:48:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.504+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.5 v2:172.21.15.175:6816/38161 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f354b06800 2024-03-23T09:48:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.504+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6dc00 con 0x55f354b06000 2024-03-23T09:48:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.505+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.505+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6dc00 con 0x55f354b06000 2024-03-23T09:48:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.505+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.505+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f90800 con 0x55f353f6d800 2024-03-23T09:48:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.505+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.0 v2:172.21.15.47:6816/37679 5 ==== osd_op_reply(70 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6d800 2024-03-23T09:48:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.505+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6d400 con 0x55f354b06000 2024-03-23T09:48:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.506+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 13 ==== osd_op_reply(71 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.506+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4f800 con 0x55f353f6c400 2024-03-23T09:48:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.506+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.506+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4f000 con 0x55f353f6cc00 2024-03-23T09:48:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.506+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.506+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4ec00 con 0x55f354b06000 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.506+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.507+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4fc00 con 0x55f353f6cc00 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.507+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.507+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7ec00 con 0x55f354b06800 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.507+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.5 v2:172.21.15.175:6816/38161 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f354b06800 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.507+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f90000 con 0x55f353f6c400 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.507+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 12 ==== osd_op_reply(77 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.507+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7fc00 con 0x55f353f6d800 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.508+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.0 v2:172.21.15.47:6816/37679 6 ==== osd_op_reply(78 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6d800 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.508+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f90400 con 0x55f353f6cc00 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.508+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.508+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e000 con 0x55f353f6cc00 2024-03-23T09:48:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.508+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 15 ==== osd_op_reply(80 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.508+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f91800 con 0x55f353f6c400 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.508+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 13 ==== osd_op_reply(81 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.508+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7f800 con 0x55f353f6c400 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.509+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 14 ==== osd_op_reply(82 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.509+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f91000 con 0x55f354b06800 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.509+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.5 v2:172.21.15.175:6816/38161 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f354b06800 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.509+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f90c00 con 0x55f353f6d000 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.509+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.509+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f91c00 con 0x55f353f4f400 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.509+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.510+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f91400 con 0x55f353f7f000 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.510+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.3 v2:172.21.15.47:6808/37683 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f7f000 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.510+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f354b06400 con 0x55f354b06000 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.510+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 15 ==== osd_op_reply(87 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.510+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f90800 con 0x55f353f4f400 2024-03-23T09:48:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.510+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.510+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e800 con 0x55f353f6cc00 2024-03-23T09:48:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.511+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.511+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6c000 con 0x55f353f6d000 2024-03-23T09:48:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.511+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.511+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7f400 con 0x55f353f6c400 2024-03-23T09:48:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.511+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 15 ==== osd_op_reply(91 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.511+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f3549cf400 con 0x55f354b06000 2024-03-23T09:48:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.511+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 16 ==== osd_op_reply(92 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.511+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e400 con 0x55f353f6d000 2024-03-23T09:48:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.512+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 14 ==== osd_op_reply(93 [pgnls-filter start_epoch 89 out=49b] v85'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.512+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4ec00 con 0x55f353f6c400 2024-03-23T09:48:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.512+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 16 ==== osd_op_reply(94 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.512+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e400 con 0x55f353f7f000 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.512+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.3 v2:172.21.15.47:6808/37683 5 ==== osd_op_reply(95 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f7f000 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.512+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4f800 con 0x55f354b06800 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.512+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.5 v2:172.21.15.175:6816/38161 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f354b06800 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.513+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7ec00 con 0x55f354b06000 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.513+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 17 ==== osd_op_reply(97 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.513+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4f000 con 0x55f353f7f000 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.513+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.3 v2:172.21.15.47:6808/37683 6 ==== osd_op_reply(98 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f7f000 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.513+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7fc00 con 0x55f353f6cc00 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.513+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.513+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f4fc00 con 0x55f353f6d000 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.514+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 15 ==== osd_op_reply(100 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.514+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7f800 con 0x55f353f6d000 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.514+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 16 ==== osd_op_reply(101 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.514+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6dc00 con 0x55f353f6d000 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.514+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 17 ==== osd_op_reply(102 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.514+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7f400 con 0x55f353f6c400 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.514+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 17 ==== osd_op_reply(103 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.514+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6d400 con 0x55f353f6d000 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.515+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 18 ==== osd_op_reply(104 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.515+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e000 con 0x55f353f6d800 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.515+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.0 v2:172.21.15.47:6816/37679 7 ==== osd_op_reply(105 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6d800 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.515+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6c000 con 0x55f353f4f400 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.515+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 14 ==== osd_op_reply(106 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.515+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f7e800 con 0x55f353f4f400 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.516+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.516+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f6c800 con 0x55f353f6d800 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.516+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.0 v2:172.21.15.47:6816/37679 8 ==== osd_op_reply(108 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6d800 2024-03-23T09:48:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.516+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f9e000 con 0x55f353f6c400 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.516+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 18 ==== osd_op_reply(109 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.516+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f3549cf400 con 0x55f353f4f400 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.516+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 16 ==== osd_op_reply(110 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.516+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f9e400 con 0x55f353f4f400 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.517+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.517+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f354b06400 con 0x55f354b06800 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.517+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.5 v2:172.21.15.175:6816/38161 11 ==== osd_op_reply(112 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f354b06800 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.517+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f9e800 con 0x55f353f6cc00 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.517+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.2 v2:172.21.15.47:6800/37678 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f6cc00 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.517+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353fa0000 con 0x55f353f6d000 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.517+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 19 ==== osd_op_reply(114 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.517+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f9ec00 con 0x55f353f6d000 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.518+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 20 ==== osd_op_reply(115 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.518+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353fa0400 con 0x55f353f4f400 2024-03-23T09:48:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.518+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 18 ==== osd_op_reply(116 [pgnls-filter start_epoch 89 out=81b] v89'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.518+0000 7f4cc0e0d740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.518+0000 7f4cc0e0d740 10 datascan.operator(): handling object 10000000001.0 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.518+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353fa6000 con 0x55f353f4f400 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.518+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 <== osd.7 v2:172.21.15.175:6824/38174 19 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x55f354aa6fc0 con 0x55f353f4f400 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.518+0000 7f4cc0e0d740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.518+0000 7f4cc0e0d740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.518+0000 7f4cc0e0d740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.518+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353fa6400 con 0x55f353f6d800 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.519+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.0 v2:172.21.15.47:6816/37679 9 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+59 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6d800 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.519+0000 7f4cc0e0d740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.519+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353fa6800 con 0x55f354b06000 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.519+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 18 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+520 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.519+0000 7f4cc0e0d740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-03-23T09:48:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.519+0000 7f4cc0e0d740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.519+0000 7f4cc0e0d740 20 datascan.get_frag_of: frag is * 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.519+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353fa6c00 con 0x55f353f6d800 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.519+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.0 v2:172.21.15.47:6816/37679 10 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6d800 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.519+0000 7f4cc0e0d740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.519+0000 7f4cc0e0d740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.519+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353fa7000 con 0x55f353f6d800 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.520+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.0 v2:172.21.15.47:6816/37679 11 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6d800 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.520+0000 7f4cc0e0d740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.520+0000 7f4cc0e0d740 20 datascan.read_dentry: file_head not found in result 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.520+0000 7f4cc0e0d740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.520+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353fa7400 con 0x55f353f6d800 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.521+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.0 v2:172.21.15.47:6816/37679 12 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v89'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6d800 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.521+0000 7f4cc0e0d740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.521+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353fa7800 con 0x55f353f6c400 2024-03-23T09:48:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.521+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.1 v2:172.21.15.47:6824/37680 19 ==== osd_op_reply(123 [pgnls-filter start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f353f6c400 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.521+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353fa0800 con 0x55f354b06800 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.521+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.5 v2:172.21.15.175:6816/38161 12 ==== osd_op_reply(124 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f354b06800 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.521+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f9f000 con 0x55f354b06800 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.522+0000 7f4cbe1ee640 1 -- 172.21.15.47:0/118777466 <== osd.5 v2:172.21.15.175:6816/38161 13 ==== osd_op_reply(125 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6b40 con 0x55f354b06800 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.522+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353fa0c00 con 0x55f354b06000 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.522+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 19 ==== osd_op_reply(126 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.522+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f9f400 con 0x55f353f6d000 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.522+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.4 v2:172.21.15.175:6800/38159 21 ==== osd_op_reply(127 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f353f6d000 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.522+0000 7f4cb99e5640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353fa1000 con 0x55f354b06000 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.522+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 20 ==== osd_op_reply(128 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cba1e6640 1 -- 172.21.15.47:0/118777466 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55f353f9f800 con 0x55f354b06000 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cbf1f0640 1 -- 172.21.15.47:0/118777466 <== osd.6 v2:172.21.15.175:6808/38160 21 ==== osd_op_reply(129 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f354aa6d80 con 0x55f354b06000 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55f353f6d800 msgr2=0x55f353f73080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55f353f6d800 0x55f353f73080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55f353f6c400 msgr2=0x55f353f72000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55f353f6c400 0x55f353f72000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55f353f6cc00 msgr2=0x55f353f72b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55f353f6cc00 0x55f353f72b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55f353f7f000 msgr2=0x55f353f73600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55f353f7f000 0x55f353f73600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55f353f6d000 msgr2=0x55f353f72580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 reap_dead start 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55f353f6d000 0x55f353f72580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55f354b06800 msgr2=0x55f353c8c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55f354b06800 0x55f353c8c100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55f354b06000 msgr2=0x55f353c8bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55f354b06000 0x55f353c8bb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55f353f4f400 msgr2=0x55f353c8b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55f353f4f400 0x55f353c8b600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cbe9ef640 1 -- 172.21.15.47:0/118777466 reap_dead start 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55f353f4e000 msgr2=0x55f353c8d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55f353f4e000 0x55f353c8d700 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55f35496e7e0 tx=0x55f354acfda0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3549cf800 msgr2=0x55f353c8d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3549cf800 0x55f353c8d180 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x55f354aaf5f0 tx=0x55f354acec00 comp rx=0 tx=0).stop 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 shutdown_connections 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55f353f4e000 0x55f353c8d700 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/118777466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3549cf800 0x55f353c8d180 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 >> 172.21.15.47:0/118777466 conn(0x55f3549cfc00 msgr2=0x55f353cd9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 shutdown_connections 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/118777466 wait complete. 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/3214003736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f353bf1c00 msgr2=0x55f353c8b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/3214003736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f353bf1c00 0x55f353c8b080 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x55f354a09290 tx=0x55f354ab4120 comp rx=0 tx=0).stop 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/3214003736 shutdown_connections 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/3214003736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f353bf1c00 0x55f353c8b080 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/3214003736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f353bf1800 0x55f353c8ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.523+0000 7f4cc0e0d740 1 --2- 172.21.15.47:0/3214003736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f353bf0c00 0x55f353c8a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.524+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/3214003736 >> 172.21.15.47:0/3214003736 conn(0x55f353bf1000 msgr2=0x55f353cd8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.524+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/3214003736 shutdown_connections 2024-03-23T09:48:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.524+0000 7f4cc0e0d740 1 -- 172.21.15.47:0/3214003736 wait complete. 2024-03-23T09:48:48.539 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.144288, result: 2024-03-23T09:48:48.539 DEBUG:teuthology.orchestra.run.smithi047:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-03-23T09:48:48.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.603+0000 7fecaca0c740 1 -- start start 2024-03-23T09:48:48.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.603+0000 7fecaca0c740 1 Processor -- start 2024-03-23T09:48:48.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaca0c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc0373cc00 0x55fc037d6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaca0c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc0373d800 0x55fc037d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaca0c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc0373dc00 0x55fc037d7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaca0c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc037cea00 con 0x55fc0373cc00 2024-03-23T09:48:48.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaca0c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc037ce780 con 0x55fc0373d800 2024-03-23T09:48:48.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaca0c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc037ce640 con 0x55fc0373dc00 2024-03-23T09:48:48.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaa5c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc0373cc00 0x55fc037d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaa5c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc0373cc00 0x55fc037d6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49296/0 (socket says 172.21.15.47:49296) 2024-03-23T09:48:48.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaa5c8640 1 -- 172.21.15.47:0/4092760081 learned_addr learned my addr 172.21.15.47:0/4092760081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:48.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7feca9dc7640 1 --2- 172.21.15.47:0/4092760081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc0373d800 0x55fc037d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaadc9640 1 --2- 172.21.15.47:0/4092760081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc0373dc00 0x55fc037d7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaa5c8640 1 -- 172.21.15.47:0/4092760081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc0373dc00 msgr2=0x55fc037d7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaa5c8640 1 --2- 172.21.15.47:0/4092760081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc0373dc00 0x55fc037d7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaa5c8640 1 -- 172.21.15.47:0/4092760081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc0373d800 msgr2=0x55fc037d6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaa5c8640 1 --2- 172.21.15.47:0/4092760081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc0373d800 0x55fc037d6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaa5c8640 1 -- 172.21.15.47:0/4092760081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc037629c0 con 0x55fc0373cc00 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7feca9dc7640 1 --2- 172.21.15.47:0/4092760081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc0373d800 0x55fc037d6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaa5c8640 1 --2- 172.21.15.47:0/4092760081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc0373cc00 0x55fc037d6580 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x55fc045533e0 tx=0x55fc045fc120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3f193ab42843bf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7feca7dc3640 1 -- 172.21.15.47:0/4092760081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fc037ff600 con 0x55fc0373cc00 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7feca7dc3640 1 -- 172.21.15.47:0/4092760081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fc0379a180 con 0x55fc0373cc00 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7feca7dc3640 1 -- 172.21.15.47:0/4092760081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fc037ff1e0 con 0x55fc0373cc00 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaca0c740 1 -- 172.21.15.47:0/4092760081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc03762b60 con 0x55fc0373cc00 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaca0c740 1 -- 172.21.15.47:0/4092760081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55fc03762820 con 0x55fc0373cc00 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.604+0000 7fecaca0c740 4 waiting for MDS map... 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.605+0000 7feca7dc3640 1 -- 172.21.15.47:0/4092760081 <== mon.0 v2:172.21.15.47:3300/0 4 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x55fc03784c40 con 0x55fc0373cc00 2024-03-23T09:48:48.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.605+0000 7feca7dc3640 1 -- 172.21.15.47:0/4092760081 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 54) v1 ==== 2573+0+0 (secure 0 0 0) 0x55fc0378d800 con 0x55fc0373cc00 2024-03-23T09:48:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.605+0000 7fecaca0c740 4 Got MDS map 54 2024-03-23T09:48:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.605+0000 7fecaca0c740 4 datascan.main: Using metadata pool output 2024-03-23T09:48:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.605+0000 7fecaca0c740 4 datascan.main: connecting to RADOS... 2024-03-23T09:48:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.605+0000 7fecaca0c740 1 Processor -- start 2024-03-23T09:48:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.605+0000 7fecaca0c740 1 -- start start 2024-03-23T09:48:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7fecaca0c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc04611000 0x55fc037d7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7fecaca0c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc04611400 0x55fc037d7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7fecaca0c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611800 0x55fc037d8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7fecaca0c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc0376be00 con 0x55fc04611800 2024-03-23T09:48:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7fecaca0c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc037ce780 con 0x55fc04611000 2024-03-23T09:48:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7fecaca0c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc037ce500 con 0x55fc04611400 2024-03-23T09:48:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7feca9dc7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611800 0x55fc037d8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7feca9dc7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611800 0x55fc037d8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49310/0 (socket says 172.21.15.47:49310) 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7feca9dc7640 1 -- 172.21.15.47:0/1392759110 learned_addr learned my addr 172.21.15.47:0/1392759110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7fecaadc9640 1 --2- 172.21.15.47:0/1392759110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc04611000 0x55fc037d7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7fecaa5c8640 1 --2- 172.21.15.47:0/1392759110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc04611400 0x55fc037d7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7feca9dc7640 1 -- 172.21.15.47:0/1392759110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc04611400 msgr2=0x55fc037d7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7feca9dc7640 1 --2- 172.21.15.47:0/1392759110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc04611400 0x55fc037d7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7feca9dc7640 1 -- 172.21.15.47:0/1392759110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc04611000 msgr2=0x55fc037d7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7feca9dc7640 1 --2- 172.21.15.47:0/1392759110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc04611000 0x55fc037d7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.606+0000 7feca9dc7640 1 -- 172.21.15.47:0/1392759110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc037624e0 con 0x55fc04611800 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.607+0000 7feca9dc7640 1 --2- 172.21.15.47:0/1392759110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611800 0x55fc037d8100 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x55fc04624660 tx=0x55fc04626f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f548c2c822b8738c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.607+0000 7feca5dbf640 1 -- 172.21.15.47:0/1392759110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fc037ff600 con 0x55fc04611800 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.607+0000 7feca5dbf640 1 -- 172.21.15.47:0/1392759110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fc0379a480 con 0x55fc04611800 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.607+0000 7feca5dbf640 1 -- 172.21.15.47:0/1392759110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fc037ff1e0 con 0x55fc04611800 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.607+0000 7fecaca0c740 1 -- 172.21.15.47:0/1392759110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611800 msgr2=0x55fc037d8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.607+0000 7fecaca0c740 1 --2- 172.21.15.47:0/1392759110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611800 0x55fc037d8100 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x55fc04624660 tx=0x55fc04626f00 comp rx=0 tx=0).stop 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.607+0000 7fecaca0c740 1 -- 172.21.15.47:0/1392759110 shutdown_connections 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.607+0000 7fecaca0c740 1 --2- 172.21.15.47:0/1392759110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611800 0x55fc037d8100 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.607+0000 7fecaca0c740 1 --2- 172.21.15.47:0/1392759110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc04611400 0x55fc037d7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.607+0000 7fecaca0c740 1 --2- 172.21.15.47:0/1392759110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc04611000 0x55fc037d7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.607+0000 7fecaca0c740 1 -- 172.21.15.47:0/1392759110 >> 172.21.15.47:0/1392759110 conn(0x55fc04610c00 msgr2=0x55fc03825000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.607+0000 7fecaca0c740 1 -- 172.21.15.47:0/1392759110 shutdown_connections 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.607+0000 7fecaca0c740 1 -- 172.21.15.47:0/1392759110 wait complete. 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.608+0000 7fecaca0c740 1 Processor -- start 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.608+0000 7fecaca0c740 1 -- start start 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.608+0000 7fecaca0c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc04611000 0x55fc037d9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.608+0000 7fecaca0c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611400 0x55fc037d8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.608+0000 7fecaca0c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc04611800 0x55fc037d8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.608+0000 7fecaca0c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc037ce3c0 con 0x55fc04611400 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.608+0000 7fecaca0c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc037ce280 con 0x55fc04611000 2024-03-23T09:48:48.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.608+0000 7fecaca0c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc037ce140 con 0x55fc04611800 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.608+0000 7fecaa5c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611400 0x55fc037d8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.608+0000 7fecaa5c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611400 0x55fc037d8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49326/0 (socket says 172.21.15.47:49326) 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.608+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 learned_addr learned my addr 172.21.15.47:0/2783165441 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.608+0000 7fecaadc9640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc04611000 0x55fc037d9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.608+0000 7feca9dc7640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc04611800 0x55fc037d8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.609+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc04611800 msgr2=0x55fc037d8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.609+0000 7fecaa5c8640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc04611800 0x55fc037d8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.609+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc04611000 msgr2=0x55fc037d9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.609+0000 7fecaa5c8640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc04611000 0x55fc037d9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.609+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc037629c0 con 0x55fc04611400 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.609+0000 7fecaa5c8640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611400 0x55fc037d8c00 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x55fc04625380 tx=0x55fc045fc060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.609+0000 7feca45bc640 1 -- 172.21.15.47:0/2783165441 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fc037fef20 con 0x55fc04611400 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.609+0000 7feca45bc640 1 -- 172.21.15.47:0/2783165441 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fc04517680 con 0x55fc04611400 2024-03-23T09:48:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.609+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fc03762b60 con 0x55fc04611400 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.609+0000 7feca45bc640 1 -- 172.21.15.47:0/2783165441 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fc037ff600 con 0x55fc04611400 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.609+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc037621a0 con 0x55fc04611400 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.609+0000 7fecaca0c740 4 datascan.init: resolving metadata pool 12 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.610+0000 7feca45bc640 1 -- 172.21.15.47:0/2783165441 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x55fc037a3080 con 0x55fc04611400 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.611+0000 7feca45bc640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55fc045ff800 0x55fc037d9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.611+0000 7feca45bc640 1 -- 172.21.15.47:0/2783165441 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x55fc03784a80 con 0x55fc04611400 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.611+0000 7fecaca0c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.611+0000 7fecaca0c740 4 datascan.main: resolving metadata pool 12 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.611+0000 7fecaca0c740 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55fc045ffc00 0x55fc037d8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.611+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa0000 con 0x55fc045ffc00 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.611+0000 7fecaadc9640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55fc045ff800 0x55fc037d9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.611+0000 7feca9dc7640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55fc045ffc00 0x55fc037d8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.611+0000 7fecaadc9640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55fc045ff800 0x55fc037d9700 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x55fc037ad740 tx=0x55fc04626d80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.611+0000 7feca9dc7640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55fc045ffc00 0x55fc037d8100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.612+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.4 v2:172.21.15.175:6800/38159 1 ==== osd_op_reply(1 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc045ffc00 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.612+0000 7feca9dc7640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55fc03aa0000 0x55fc03aa4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.612+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045fe400 con 0x55fc03aa0000 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.612+0000 7fecaadc9640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55fc03aa0000 0x55fc03aa4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.612+0000 7fecaadc9640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55fc03aa0000 0x55fc03aa4000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.612+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 1 ==== osd_op_reply(2 [pgnls start_epoch 89 out=177b] v89'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.612+0000 7fecaadc9640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55fc045fe400 0x55fc03aa4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.612+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc04519000 con 0x55fc045fe400 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.613+0000 7fecaa5c8640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55fc045fe400 0x55fc03aa4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.613+0000 7fecaa5c8640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55fc045fe400 0x55fc03aa4580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.613+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 <== osd.3 v2:172.21.15.47:6808/37683 1 ==== osd_op_reply(3 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2fc0 con 0x55fc045fe400 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.613+0000 7fecaa5c8640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55fc04519000 0x55fc03aa4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.613+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045fec00 con 0x55fc04519000 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.613+0000 7feca9dc7640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55fc04519000 0x55fc03aa4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.614+0000 7feca9dc7640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55fc04519000 0x55fc03aa4b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.614+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== osd_op_reply(4 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.614+0000 7feca9dc7640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55fc045fec00 0x55fc03aa5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.614+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045fe000 con 0x55fc045fec00 2024-03-23T09:48:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.614+0000 7fecaadc9640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55fc045fec00 0x55fc03aa5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.614+0000 7fecaadc9640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55fc045fec00 0x55fc03aa5080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.615+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 1 ==== osd_op_reply(5 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.615+0000 7fecaadc9640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55fc045fe000 0x55fc03aa5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.615+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc04519400 con 0x55fc045fe000 2024-03-23T09:48:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.615+0000 7fecaa5c8640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55fc045fe000 0x55fc03aa5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.615+0000 7fecaa5c8640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55fc045fe000 0x55fc03aa5b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.616+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 <== osd.5 v2:172.21.15.175:6816/38161 1 ==== osd_op_reply(6 [pgnls start_epoch 89 out=149b] v88'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55fc045f2fc0 con 0x55fc045fe000 2024-03-23T09:48:48.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.616+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc04519400 con 0x55fc03aa0000 2024-03-23T09:48:48.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.616+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 2 ==== osd_op_reply(7 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.616+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc04519400 con 0x55fc03aa0000 2024-03-23T09:48:48.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.616+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 3 ==== osd_op_reply(8 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.616+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc04519400 con 0x55fc045ffc00 2024-03-23T09:48:48.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.616+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.4 v2:172.21.15.175:6800/38159 2 ==== osd_op_reply(9 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc045ffc00 2024-03-23T09:48:48.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.616+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc04519400 con 0x55fc045fe400 2024-03-23T09:48:48.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.617+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 <== osd.3 v2:172.21.15.47:6808/37683 2 ==== osd_op_reply(10 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2fc0 con 0x55fc045fe400 2024-03-23T09:48:48.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.617+0000 7fecaa5c8640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55fc04519400 0x55fc03aa6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.617+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045ff000 con 0x55fc04519400 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.617+0000 7feca9dc7640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55fc04519400 0x55fc03aa6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.617+0000 7feca9dc7640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55fc04519400 0x55fc03aa6680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.617+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.2 v2:172.21.15.47:6800/37678 1 ==== osd_op_reply(11 [pgnls start_epoch 89 out=81b] v88'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519400 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.617+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045ff000 con 0x55fc045ffc00 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.618+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.4 v2:172.21.15.175:6800/38159 3 ==== osd_op_reply(12 [pgnls start_epoch 89 out=49b] v86'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc045ffc00 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.618+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045ff000 con 0x55fc04519000 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.618+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 2 ==== osd_op_reply(13 [pgnls start_epoch 89 out=77b] v88'6 uv6 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.618+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045ff000 con 0x55fc045fec00 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.618+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 2 ==== osd_op_reply(14 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.618+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045ff000 con 0x55fc03aa0000 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.618+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 4 ==== osd_op_reply(15 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.619+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045ff000 con 0x55fc045ffc00 2024-03-23T09:48:48.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.619+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.4 v2:172.21.15.175:6800/38159 4 ==== osd_op_reply(16 [pgnls start_epoch 89 out=49b] v86'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc045ffc00 2024-03-23T09:48:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.619+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045ff000 con 0x55fc045fec00 2024-03-23T09:48:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.619+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 3 ==== osd_op_reply(17 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.619+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045ff000 con 0x55fc045fe000 2024-03-23T09:48:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.619+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 <== osd.5 v2:172.21.15.175:6816/38161 2 ==== osd_op_reply(18 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2fc0 con 0x55fc045fe000 2024-03-23T09:48:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.619+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045ff000 con 0x55fc04519400 2024-03-23T09:48:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.620+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.2 v2:172.21.15.47:6800/37678 2 ==== osd_op_reply(19 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519400 2024-03-23T09:48:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.620+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045ff000 con 0x55fc03aa0000 2024-03-23T09:48:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.620+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 5 ==== osd_op_reply(20 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.620+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045ff000 con 0x55fc045fec00 2024-03-23T09:48:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.620+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 4 ==== osd_op_reply(21 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.620+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc045ff000 con 0x55fc045fec00 2024-03-23T09:48:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.620+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 5 ==== osd_op_reply(22 [pgnls start_epoch 89 out=97b] v88'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.620+0000 7fecaadc9640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55fc045ff000 0x55fc03aa6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.620+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1800 con 0x55fc045ff000 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.621+0000 7fecaadc9640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55fc045ff000 0x55fc03aa6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.621+0000 7fecaadc9640 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55fc045ff000 0x55fc03aa6100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.621+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.7 v2:172.21.15.175:6824/38174 1 ==== osd_op_reply(23 [pgnls start_epoch 89 out=74b] v84'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045ff000 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.621+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1800 con 0x55fc03aa0000 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.621+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 6 ==== osd_op_reply(24 [pgnls start_epoch 89 out=73b] v88'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.622+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1800 con 0x55fc03aa0000 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.622+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 7 ==== osd_op_reply(25 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.622+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1800 con 0x55fc04519000 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.622+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 3 ==== osd_op_reply(26 [pgnls start_epoch 89 out=98b] v88'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.622+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1800 con 0x55fc045fe400 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.622+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 <== osd.3 v2:172.21.15.47:6808/37683 3 ==== osd_op_reply(27 [pgnls start_epoch 89 out=76b] v88'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55fc045f2fc0 con 0x55fc045fe400 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.622+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1800 con 0x55fc045ffc00 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.623+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.4 v2:172.21.15.175:6800/38159 5 ==== osd_op_reply(28 [pgnls start_epoch 89 out=73b] v88'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc045ffc00 2024-03-23T09:48:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.623+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1800 con 0x55fc04519000 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.623+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 4 ==== osd_op_reply(29 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.623+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1800 con 0x55fc045fec00 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.623+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 6 ==== osd_op_reply(30 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.623+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1800 con 0x55fc045fec00 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.624+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 7 ==== osd_op_reply(31 [pgnls start_epoch 89 out=73b] v84'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.624+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1800 con 0x55fc045fec00 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.624+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 8 ==== osd_op_reply(32 [pgnls start_epoch 89 out=101b] v88'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.624+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.624+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa0400 con 0x55fc03aa0000 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.624+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.624+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.624+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa0800 con 0x55fc03aa0000 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.624+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.624+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-03-23T09:48:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.624+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa0c00 con 0x55fc03aa0000 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 20 datascan.scan_links: injected ino detected 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1000 con 0x55fc03aa0000 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1400 con 0x55fc03aa0000 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-03-23T09:48:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1c00 con 0x55fc045fe000 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 <== osd.5 v2:172.21.15.175:6816/38161 3 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2fc0 con 0x55fc045fe000 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.625+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1800 con 0x55fc03aa0000 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.626+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.626+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.626+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa0400 con 0x55fc04519400 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.626+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.2 v2:172.21.15.47:6800/37678 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519400 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.626+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.626+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.626+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.626+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa0800 con 0x55fc03aa0000 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.626+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.626+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.626+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa0c00 con 0x55fc045fec00 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1000 con 0x55fc03aa0000 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4000 con 0x55fc04519000 2024-03-23T09:48:48.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.627+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4400 con 0x55fc04519000 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.628+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 6 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.628+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.628+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.628+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.628+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.628+0000 7fecaca0c740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.628+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fec00 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.628+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 10 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.628+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4c00 con 0x55fc045ffc00 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.628+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.4 v2:172.21.15.175:6800/38159 6 ==== osd_op_reply(47 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc045ffc00 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.628+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4c00 con 0x55fc03aa0000 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.628+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 16 ==== osd_op_reply(48 [pgnls start_epoch 89 out=177b] v89'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.628+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fe400 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.629+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 <== osd.3 v2:172.21.15.47:6808/37683 4 ==== osd_op_reply(49 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2fc0 con 0x55fc045fe400 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.629+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc04519000 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.629+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 7 ==== osd_op_reply(50 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.629+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fec00 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.629+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 11 ==== osd_op_reply(51 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.629+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fe000 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.630+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 <== osd.5 v2:172.21.15.175:6816/38161 4 ==== osd_op_reply(52 [pgnls start_epoch 89 out=149b] v88'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55fc045f2fc0 con 0x55fc045fe000 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.630+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc03aa0000 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.630+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 17 ==== osd_op_reply(53 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.630+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc03aa0000 2024-03-23T09:48:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.630+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 18 ==== osd_op_reply(54 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.630+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045ffc00 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.630+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.4 v2:172.21.15.175:6800/38159 7 ==== osd_op_reply(55 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc045ffc00 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.630+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fe400 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.631+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 <== osd.3 v2:172.21.15.47:6808/37683 5 ==== osd_op_reply(56 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2fc0 con 0x55fc045fe400 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.631+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc04519400 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.631+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.2 v2:172.21.15.47:6800/37678 4 ==== osd_op_reply(57 [pgnls start_epoch 89 out=81b] v88'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519400 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.631+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045ffc00 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.632+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.4 v2:172.21.15.175:6800/38159 8 ==== osd_op_reply(58 [pgnls start_epoch 89 out=49b] v86'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc045ffc00 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.632+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc04519000 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.632+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 8 ==== osd_op_reply(59 [pgnls start_epoch 89 out=77b] v88'6 uv6 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.632+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fec00 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.632+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 12 ==== osd_op_reply(60 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.632+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc03aa0000 2024-03-23T09:48:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.632+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 19 ==== osd_op_reply(61 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.632+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045ffc00 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.633+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.4 v2:172.21.15.175:6800/38159 9 ==== osd_op_reply(62 [pgnls start_epoch 89 out=49b] v86'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc045ffc00 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.633+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fec00 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.633+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 13 ==== osd_op_reply(63 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.633+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fe000 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.633+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 <== osd.5 v2:172.21.15.175:6816/38161 5 ==== osd_op_reply(64 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2fc0 con 0x55fc045fe000 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.633+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc04519400 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.633+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.2 v2:172.21.15.47:6800/37678 5 ==== osd_op_reply(65 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519400 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.633+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc03aa0000 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.634+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 20 ==== osd_op_reply(66 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.634+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fec00 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.634+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 14 ==== osd_op_reply(67 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.634+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fec00 2024-03-23T09:48:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.634+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 15 ==== osd_op_reply(68 [pgnls start_epoch 89 out=97b] v88'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.634+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045ff000 2024-03-23T09:48:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.635+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.7 v2:172.21.15.175:6824/38174 2 ==== osd_op_reply(69 [pgnls start_epoch 89 out=74b] v84'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045ff000 2024-03-23T09:48:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.635+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc03aa0000 2024-03-23T09:48:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.635+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 21 ==== osd_op_reply(70 [pgnls start_epoch 89 out=73b] v88'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.635+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc03aa0000 2024-03-23T09:48:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.635+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 22 ==== osd_op_reply(71 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.635+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc04519000 2024-03-23T09:48:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.635+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 9 ==== osd_op_reply(72 [pgnls start_epoch 89 out=98b] v88'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.635+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fe400 2024-03-23T09:48:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.635+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 <== osd.3 v2:172.21.15.47:6808/37683 6 ==== osd_op_reply(73 [pgnls start_epoch 89 out=76b] v88'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55fc045f2fc0 con 0x55fc045fe400 2024-03-23T09:48:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.635+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045ffc00 2024-03-23T09:48:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.636+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.4 v2:172.21.15.175:6800/38159 10 ==== osd_op_reply(74 [pgnls start_epoch 89 out=73b] v88'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc045ffc00 2024-03-23T09:48:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.636+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc04519000 2024-03-23T09:48:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.636+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 10 ==== osd_op_reply(75 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.636+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fec00 2024-03-23T09:48:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.637+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 16 ==== osd_op_reply(76 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.637+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fec00 2024-03-23T09:48:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.637+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 17 ==== osd_op_reply(77 [pgnls start_epoch 89 out=73b] v84'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.637+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fec00 2024-03-23T09:48:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.637+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 18 ==== osd_op_reply(78 [pgnls start_epoch 89 out=101b] v88'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.637+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.637+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad5000 con 0x55fc03aa0000 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.638+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 23 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.638+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.638+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad5400 con 0x55fc03aa0000 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.638+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 24 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.638+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.638+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad5800 con 0x55fc03aa0000 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.638+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 25 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.638+0000 7fecaca0c740 20 datascan.scan_links: injected ino detected 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.638+0000 7fecaca0c740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.638+0000 7fecaca0c740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.638+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-03-23T09:48:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.638+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad5c00 con 0x55fc03aa0000 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 26 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4c00 con 0x55fc03aa0000 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 27 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4800 con 0x55fc045fe000 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaa5c8640 1 -- 172.21.15.47:0/2783165441 <== osd.5 v2:172.21.15.175:6816/38161 6 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2fc0 con 0x55fc045fe000 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-03-23T09:48:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad5000 con 0x55fc03aa0000 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 28 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.639+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad5400 con 0x55fc04519400 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.2 v2:172.21.15.47:6800/37678 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519400 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1000 con 0x55fc03aa0000 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 29 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa0c00 con 0x55fc045fec00 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 19 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-23T09:48:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.640+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa0800 con 0x55fc03aa0000 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 30 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1800 con 0x55fc04519000 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 11 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-03-23T09:48:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1400 con 0x55fc04519000 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 12 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.641+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa0400 con 0x55fc045fec00 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.642+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.6 v2:172.21.15.175:6808/38160 20 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045fec00 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.642+0000 7fecaca0c740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.642+0000 7fecaca0c740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.642+0000 7fecaca0c740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.642+0000 7fecaca0c740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.642+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03aa1800 con 0x55fc03aa0000 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.642+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 31 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.642+0000 7fecaca0c740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.642+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4000 con 0x55fc03aa0000 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.643+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 32 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v89'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.643+0000 7fecaca0c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.643+0000 7fecaca0c740 10 datascan.scan_links: processing 1 injected_inos 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.643+0000 7fecaca0c740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.643+0000 7fecaca0c740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.643+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad4400 con 0x55fc03aa0000 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.644+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 33 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.644+0000 7fecaca0c740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.644+0000 7fecaca0c740 20 datascan.scan_links: first is now 2 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.644+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad5400 con 0x55fc03aa0000 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.645+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.0 v2:172.21.15.47:6816/37679 34 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v89'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc03aa0000 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.645+0000 7fecaca0c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-23T09:48:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.645+0000 7fecaca0c740 10 datascan.scan_links: updating inotable 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.645+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc04611c00 con 0x55fc04519000 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.645+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 13 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.645+0000 7fecaca0c740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.645+0000 7fecaca0c740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.645+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad5800 con 0x55fc04519000 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.646+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 <== osd.1 v2:172.21.15.47:6824/37680 14 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v89'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55fc045f2b40 con 0x55fc04519000 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.646+0000 7fecaca0c740 10 datascan.scan_links: updating snaptable 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.646+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) v8 -- 0x55fc03ad5c00 con 0x55fc045ff000 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.646+0000 7fecaadc9640 1 -- 172.21.15.47:0/2783165441 <== osd.7 v2:172.21.15.175:6824/38174 3 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55fc045f2d80 con 0x55fc045ff000 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.646+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55fc03aa0000 msgr2=0x55fc03aa4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.646+0000 7fecaca0c740 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55fc03aa0000 0x55fc03aa4000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.646+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55fc04519000 msgr2=0x55fc03aa4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.646+0000 7fecaca0c740 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55fc04519000 0x55fc03aa4b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.646+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55fc04519400 msgr2=0x55fc03aa6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.646+0000 7fecaca0c740 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55fc04519400 0x55fc03aa6680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.646+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55fc045fe400 msgr2=0x55fc03aa4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.646+0000 7fecaca0c740 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55fc045fe400 0x55fc03aa4580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.646+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55fc045ffc00 msgr2=0x55fc037d8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55fc045ffc00 0x55fc037d8100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55fc045fe000 msgr2=0x55fc03aa5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55fc045fe000 0x55fc03aa5b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55fc045fec00 msgr2=0x55fc03aa5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55fc045fec00 0x55fc03aa5080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 reap_dead start 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 reap_dead start 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 reap_dead start 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55fc045ff000 msgr2=0x55fc03aa6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55fc045ff000 0x55fc03aa6100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7feca9dc7640 1 -- 172.21.15.47:0/2783165441 reap_dead start 2024-03-23T09:48:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55fc045ff800 msgr2=0x55fc037d9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55fc045ff800 0x55fc037d9700 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x55fc037ad740 tx=0x55fc04626d80 comp rx=0 tx=0).stop 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611400 msgr2=0x55fc037d8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611400 0x55fc037d8c00 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x55fc04625380 tx=0x55fc045fc060 comp rx=0 tx=0).stop 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 shutdown_connections 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55fc045ff800 0x55fc037d9700 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 --2- 172.21.15.47:0/2783165441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc04611400 0x55fc037d8c00 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 >> 172.21.15.47:0/2783165441 conn(0x55fc04610c00 msgr2=0x55fc03825000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 shutdown_connections 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/2783165441 wait complete. 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/4092760081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc0373cc00 msgr2=0x55fc037d6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 --2- 172.21.15.47:0/4092760081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc0373cc00 0x55fc037d6580 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x55fc045533e0 tx=0x55fc045fc120 comp rx=0 tx=0).stop 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/4092760081 shutdown_connections 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 --2- 172.21.15.47:0/4092760081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fc0373dc00 0x55fc037d7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 --2- 172.21.15.47:0/4092760081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc0373d800 0x55fc037d6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 --2- 172.21.15.47:0/4092760081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc0373cc00 0x55fc037d6580 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/4092760081 >> 172.21.15.47:0/4092760081 conn(0x55fc0373d000 msgr2=0x55fc03824800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/4092760081 shutdown_connections 2024-03-23T09:48:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.647+0000 7fecaca0c740 1 -- 172.21.15.47:0/4092760081 wait complete. 2024-03-23T09:48:48.667 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.127777, result: 2024-03-23T09:48:48.667 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-23T09:48:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.802+0000 7fdd6ad2e640 1 Processor -- start 2024-03-23T09:48:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.802+0000 7fdd6ad2e640 1 -- start start 2024-03-23T09:48:48.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.802+0000 7fdd6ad2e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdd6415cee0 0x7fdd640d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.802+0000 7fdd6ad2e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd640d1300 con 0x7fdd640d2f80 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.802+0000 7fdd6ad2e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd640d15e0 con 0x7fdd640d23a0 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.802+0000 7fdd6ad2e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd640d18c0 con 0x7fdd6415cee0 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd692cb640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdd640d2f80 0x7fdd640b5a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44988/0 (socket says 172.21.15.47:44988) 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd692cb640 1 -- 172.21.15.47:0/2078231588 learned_addr learned my addr 172.21.15.47:0/2078231588 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd68aca640 1 --2- 172.21.15.47:0/2078231588 >> v2:172.21.15.175:3300/0 conn(0x7fdd6415cee0 0x7fdd640d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841371625 0 0) 0x7fdd640d1300 con 0x7fdd640d2f80 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd54003520 con 0x7fdd640d2f80 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4055871276 0 0) 0x7fdd54003520 con 0x7fdd640d2f80 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 >> v2:172.21.15.175:3300/0 conn(0x7fdd6415cee0 msgr2=0x7fdd640d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 --2- 172.21.15.47:0/2078231588 >> v2:172.21.15.175:3300/0 conn(0x7fdd6415cee0 0x7fdd640d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 >> v1:172.21.15.175:6789/0 conn(0x7fdd640d23a0 legacy=0x7fdd640d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd640d1ba0 con 0x7fdd640d2f80 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fdd5c0019f0 con 0x7fdd640d2f80 2024-03-23T09:48:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 >> v1:172.21.15.47:6789/0 conn(0x7fdd640d2f80 legacy=0x7fdd640b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 --2- 172.21.15.47:0/2078231588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd54003970 0x7fdd54003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 --2- 172.21.15.47:0/2078231588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd540047c0 0x7fdd54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 --2- 172.21.15.47:0/2078231588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd54005180 0x7fdd5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd640d18c0 con 0x7fdd54005180 2024-03-23T09:48:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5400c020 con 0x7fdd540047c0 2024-03-23T09:48:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5400c1e0 con 0x7fdd54003970 2024-03-23T09:48:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd692cb640 1 --2- 172.21.15.47:0/2078231588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd54005180 0x7fdd5400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd63fff640 1 --2- 172.21.15.47:0/2078231588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd540047c0 0x7fdd54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.803+0000 7fdd68aca640 1 --2- 172.21.15.47:0/2078231588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd54003970 0x7fdd54003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.804+0000 7fdd692cb640 1 -- 172.21.15.47:0/2078231588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd54003970 msgr2=0x7fdd54003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.804+0000 7fdd692cb640 1 --2- 172.21.15.47:0/2078231588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd54003970 0x7fdd54003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.804+0000 7fdd692cb640 1 -- 172.21.15.47:0/2078231588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd540047c0 msgr2=0x7fdd54004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.804+0000 7fdd692cb640 1 --2- 172.21.15.47:0/2078231588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd540047c0 0x7fdd54004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.804+0000 7fdd692cb640 1 -- 172.21.15.47:0/2078231588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd54003520 con 0x7fdd54005180 2024-03-23T09:48:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.804+0000 7fdd692cb640 1 -- 172.21.15.47:0/2078231588 reap_dead start 2024-03-23T09:48:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.804+0000 7fdd692cb640 1 --2- 172.21.15.47:0/2078231588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd54005180 0x7fdd5400b980 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00a830 tx=0x7fdd540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6a5c1e79e2ec26 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.804+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c0072d0 con 0x7fdd54005180 2024-03-23T09:48:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.804+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c02f070 con 0x7fdd54005180 2024-03-23T09:48:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.804+0000 7fdd637fe640 1 -- 172.21.15.47:0/2078231588 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c0076b0 con 0x7fdd54005180 2024-03-23T09:48:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.805+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2078231588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd54005180 msgr2=0x7fdd5400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.805+0000 7fdd6ad2e640 1 --2- 172.21.15.47:0/2078231588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd54005180 0x7fdd5400b980 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00a830 tx=0x7fdd540057f0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.805+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2078231588 shutdown_connections 2024-03-23T09:48:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.805+0000 7fdd6ad2e640 1 --2- 172.21.15.47:0/2078231588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd54005180 0x7fdd5400b980 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.805+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2078231588 >> 172.21.15.47:0/2078231588 conn(0x7fdd64065df0 msgr2=0x7fdd64066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.805+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2078231588 shutdown_connections 2024-03-23T09:48:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.805+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2078231588 wait complete. 2024-03-23T09:48:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.806+0000 7fdd6ad2e640 1 Processor -- start 2024-03-23T09:48:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.806+0000 7fdd6ad2e640 1 -- start start 2024-03-23T09:48:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd6ad2e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd54005180 0x7fdd641f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd6ad2e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd6414ec30 0x7fdd6415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd6ad2e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd6415ad20 0x7fdd6415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd6ad2e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6415b6e0 con 0x7fdd6415ad20 2024-03-23T09:48:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd6ad2e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6415e690 con 0x7fdd6414ec30 2024-03-23T09:48:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd6ad2e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd641f1fb0 con 0x7fdd54005180 2024-03-23T09:48:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd692cb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd6415ad20 0x7fdd6415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd692cb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd6415ad20 0x7fdd6415b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49348/0 (socket says 172.21.15.47:49348) 2024-03-23T09:48:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd692cb640 1 -- 172.21.15.47:0/2247471160 learned_addr learned my addr 172.21.15.47:0/2247471160 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd63fff640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd6414ec30 0x7fdd6415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd68aca640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd54005180 0x7fdd641f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd692cb640 1 -- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd54005180 msgr2=0x7fdd641f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd692cb640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd54005180 0x7fdd641f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd692cb640 1 -- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd6414ec30 msgr2=0x7fdd6415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd692cb640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd6414ec30 0x7fdd6415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd692cb640 1 -- 172.21.15.47:0/2247471160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6415d800 con 0x7fdd6415ad20 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.807+0000 7fdd63fff640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd6414ec30 0x7fdd6415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.808+0000 7fdd692cb640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd6415ad20 0x7fdd6415b1a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c02cc40 tx=0x7fdd5c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.808+0000 7fdd617fa640 1 -- 172.21.15.47:0/2247471160 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c0019b0 con 0x7fdd6415ad20 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.808+0000 7fdd617fa640 1 -- 172.21.15.47:0/2247471160 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c004b60 con 0x7fdd6415ad20 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.808+0000 7fdd617fa640 1 -- 172.21.15.47:0/2247471160 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c007400 con 0x7fdd6415ad20 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.808+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2247471160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6415da80 con 0x7fdd6415ad20 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.808+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2247471160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6415e040 con 0x7fdd6415ad20 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.809+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2247471160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd64072360 con 0x7fdd6415ad20 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.809+0000 7fdd617fa640 1 -- 172.21.15.47:0/2247471160 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdd5c031030 con 0x7fdd6415ad20 2024-03-23T09:48:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.810+0000 7fdd617fa640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdd4806ac00 0x7fdd4806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:48.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.810+0000 7fdd617fa640 1 -- 172.21.15.47:0/2247471160 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fdd5c0aa640 con 0x7fdd6415ad20 2024-03-23T09:48:48.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.810+0000 7fdd68aca640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdd4806ac00 0x7fdd4806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:48.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.810+0000 7fdd68aca640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdd4806ac00 0x7fdd4806d0c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009a80 tx=0x7fdd58008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:48.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:48.814+0000 7fdd617fa640 1 -- 172.21.15.47:0/2247471160 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdd5c0757e0 con 0x7fdd6415ad20 2024-03-23T09:48:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.033+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2247471160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fdd64066be0 con 0x7fdd6415ad20 2024-03-23T09:48:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.301+0000 7fdd617fa640 1 -- 172.21.15.47:0/2247471160 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v55) v1 ==== 162+0+0 (secure 0 0 0) 0x7fdd5c078e00 con 0x7fdd6415ad20 2024-03-23T09:48:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-23T09:48:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.303+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdd4806ac00 msgr2=0x7fdd4806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.304+0000 7fdd6ad2e640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdd4806ac00 0x7fdd4806d0c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009a80 tx=0x7fdd58008040 comp rx=0 tx=0).stop 2024-03-23T09:48:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.304+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2247471160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd6415ad20 msgr2=0x7fdd6415b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.304+0000 7fdd6ad2e640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd6415ad20 0x7fdd6415b1a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c02cc40 tx=0x7fdd5c001d80 comp rx=0 tx=0).stop 2024-03-23T09:48:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.304+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2247471160 shutdown_connections 2024-03-23T09:48:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.304+0000 7fdd6ad2e640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdd4806ac00 0x7fdd4806d0c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.304+0000 7fdd6ad2e640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd6415ad20 0x7fdd6415b1a0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.304+0000 7fdd6ad2e640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd6414ec30 0x7fdd6415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.304+0000 7fdd6ad2e640 1 --2- 172.21.15.47:0/2247471160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd54005180 0x7fdd641f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.304+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2247471160 >> 172.21.15.47:0/2247471160 conn(0x7fdd64065df0 msgr2=0x7fdd640c83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.304+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2247471160 shutdown_connections 2024-03-23T09:48:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.304+0000 7fdd6ad2e640 1 -- 172.21.15.47:0/2247471160 wait complete. 2024-03-23T09:48:49.316 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:48:49.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.416+0000 7f38d7448640 1 Processor -- start 2024-03-23T09:48:49.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.416+0000 7f38d7448640 1 -- start start 2024-03-23T09:48:49.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38d7448640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f38d015ca90 0x7f38d0160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:49.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38d7448640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38d00b8350 0x7f38d00cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38d7448640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d00d11e0 con 0x7f38d00cdc40 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38d7448640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d00ce0d0 con 0x7f38d015ca90 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38d7448640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00d2680 con 0x7f38d00b8350 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38d59e5640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f38d00cdc40 0x7f38d00b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44990/0 (socket says 172.21.15.47:44990) 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38d59e5640 1 -- 172.21.15.47:0/887530396 learned_addr learned my addr 172.21.15.47:0/887530396 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2755344087 0 0) 0x7f38d00d11e0 con 0x7f38d00cdc40 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38bc003520 con 0x7f38d00cdc40 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38d51e4640 1 --2- 172.21.15.47:0/887530396 >> v2:172.21.15.175:3300/0 conn(0x7f38d015ca90 0x7f38d0160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38d49e3640 1 --2- 172.21.15.47:0/887530396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38d00b8350 0x7f38d00cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 742337605 0 0) 0x7f38bc003520 con 0x7f38d00cdc40 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38d00b8350 msgr2=0x7f38d00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38c7fff640 1 --2- 172.21.15.47:0/887530396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38d00b8350 0x7f38d00cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 >> v2:172.21.15.175:3300/0 conn(0x7f38d015ca90 msgr2=0x7f38d0160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.417+0000 7f38c7fff640 1 --2- 172.21.15.47:0/887530396 >> v2:172.21.15.175:3300/0 conn(0x7f38d015ca90 0x7f38d0160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d0160f60 con 0x7f38d00cdc40 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f38c80019f0 con 0x7f38d00cdc40 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 >> v1:172.21.15.47:6789/0 conn(0x7f38d00cdc40 legacy=0x7f38d00b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38c7fff640 1 --2- 172.21.15.47:0/887530396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38d00b8350 0x7f38bc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38c7fff640 1 --2- 172.21.15.47:0/887530396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38bc003ef0 0x7f38bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38c7fff640 1 --2- 172.21.15.47:0/887530396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38bc004d00 0x7f38bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00ce0d0 con 0x7f38bc004d00 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00d2680 con 0x7f38bc003ef0 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc00b9d0 con 0x7f38d00b8350 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38d59e5640 1 --2- 172.21.15.47:0/887530396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38bc004d00 0x7f38bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38d51e4640 1 --2- 172.21.15.47:0/887530396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38d00b8350 0x7f38bc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38d49e3640 1 --2- 172.21.15.47:0/887530396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38bc003ef0 0x7f38bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38d59e5640 1 -- 172.21.15.47:0/887530396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38d00b8350 msgr2=0x7f38bc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38d59e5640 1 --2- 172.21.15.47:0/887530396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38d00b8350 0x7f38bc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38d59e5640 1 -- 172.21.15.47:0/887530396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38bc003ef0 msgr2=0x7f38bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38d59e5640 1 --2- 172.21.15.47:0/887530396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38bc003ef0 0x7f38bc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.418+0000 7f38d59e5640 1 -- 172.21.15.47:0/887530396 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc003520 con 0x7f38bc004d00 2024-03-23T09:48:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.419+0000 7f38d59e5640 1 --2- 172.21.15.47:0/887530396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38bc004d00 0x7f38bc005180 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f38c800a900 tx=0x7f38bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e120b61ad0a7ec38 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.419+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8034630 con 0x7f38bc004d00 2024-03-23T09:48:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.419+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c802f070 con 0x7f38bc004d00 2024-03-23T09:48:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.419+0000 7f38c7fff640 1 -- 172.21.15.47:0/887530396 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c80349d0 con 0x7f38bc004d00 2024-03-23T09:48:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.419+0000 7f38d7448640 1 -- 172.21.15.47:0/887530396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38bc004d00 msgr2=0x7f38bc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.419+0000 7f38d7448640 1 --2- 172.21.15.47:0/887530396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38bc004d00 0x7f38bc005180 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f38c800a900 tx=0x7f38bc0057f0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.419+0000 7f38d59e5640 1 -- 172.21.15.47:0/887530396 reap_dead start 2024-03-23T09:48:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.419+0000 7f38d7448640 1 -- 172.21.15.47:0/887530396 shutdown_connections 2024-03-23T09:48:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.419+0000 7f38d7448640 1 -- 172.21.15.47:0/887530396 >> 172.21.15.47:0/887530396 conn(0x7f38d00659d0 msgr2=0x7f38d0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.420+0000 7f38d7448640 1 -- 172.21.15.47:0/887530396 shutdown_connections 2024-03-23T09:48:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.420+0000 7f38d7448640 1 -- 172.21.15.47:0/887530396 wait complete. 2024-03-23T09:48:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.421+0000 7f38d7448640 1 Processor -- start 2024-03-23T09:48:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.421+0000 7f38d7448640 1 -- start start 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.421+0000 7f38d7448640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38bc004d00 0x7f38d0157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.421+0000 7f38d7448640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38d0156640 0x7f38d01f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.421+0000 7f38d7448640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d01549e0 0x7f38d0154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.421+0000 7f38d7448640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00b3880 con 0x7f38d01549e0 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.421+0000 7f38d7448640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d01f29e0 con 0x7f38d0156640 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.421+0000 7f38d7448640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d01f2cc0 con 0x7f38bc004d00 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38d59e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d01549e0 0x7f38d0154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38d59e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d01549e0 0x7f38d0154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49364/0 (socket says 172.21.15.47:49364) 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38d59e5640 1 -- 172.21.15.47:0/2730336936 learned_addr learned my addr 172.21.15.47:0/2730336936 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38d49e3640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38d0156640 0x7f38d01f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38d51e4640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38bc004d00 0x7f38d0157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38d59e5640 1 -- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38bc004d00 msgr2=0x7f38d0157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38d59e5640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38bc004d00 0x7f38d0157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38d59e5640 1 -- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38d0156640 msgr2=0x7f38d01f2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38d59e5640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38d0156640 0x7f38d01f2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38d59e5640 1 -- 172.21.15.47:0/2730336936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d00b3510 con 0x7f38d01549e0 2024-03-23T09:48:49.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38d59e5640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d01549e0 0x7f38d0154e60 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f38c8000f80 tx=0x7f38c8001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:49.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38c5ffb640 1 -- 172.21.15.47:0/2730336936 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8043070 con 0x7f38d01549e0 2024-03-23T09:48:49.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38c5ffb640 1 -- 172.21.15.47:0/2730336936 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c8002000 con 0x7f38d01549e0 2024-03-23T09:48:49.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38d7448640 1 -- 172.21.15.47:0/2730336936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d01f2fa0 con 0x7f38d01549e0 2024-03-23T09:48:49.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38c5ffb640 1 -- 172.21.15.47:0/2730336936 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8034640 con 0x7f38d01549e0 2024-03-23T09:48:49.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.422+0000 7f38d7448640 1 -- 172.21.15.47:0/2730336936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d01f27a0 con 0x7f38d01549e0 2024-03-23T09:48:49.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.423+0000 7f38d7448640 1 -- 172.21.15.47:0/2730336936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d0072360 con 0x7f38d01549e0 2024-03-23T09:48:49.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.424+0000 7f38c5ffb640 1 -- 172.21.15.47:0/2730336936 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f38c8031030 con 0x7f38d01549e0 2024-03-23T09:48:49.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.424+0000 7f38c5ffb640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f38b406ac00 0x7f38b406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:49.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.424+0000 7f38c5ffb640 1 -- 172.21.15.47:0/2730336936 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f38c80aad30 con 0x7f38d01549e0 2024-03-23T09:48:49.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.424+0000 7f38d51e4640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f38b406ac00 0x7f38b406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:49.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.425+0000 7f38d51e4640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f38b406ac00 0x7f38b406d0c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009c40 tx=0x7f38cc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:49.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.428+0000 7f38c5ffb640 1 -- 172.21.15.47:0/2730336936 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f38c8076090 con 0x7f38d01549e0 2024-03-23T09:48:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.647+0000 7f38d7448640 1 -- 172.21.15.47:0/2730336936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38d01f27a0 con 0x7f38d01549e0 2024-03-23T09:48:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.648+0000 7f38c5ffb640 1 -- 172.21.15.47:0/2730336936 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f38d01f27a0 con 0x7f38d01549e0 2024-03-23T09:48:49.648 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:48:49.648 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:48:49.302818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.47:6835/3671387340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3671387340},{"type":"v1","addr":"172.21.15.47:6835","nonce":3671387340}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:48:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 56 2024-03-23T09:48:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.650+0000 7f38d7448640 1 -- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f38b406ac00 msgr2=0x7f38b406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.650+0000 7f38d7448640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f38b406ac00 0x7f38b406d0c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009c40 tx=0x7f38cc009190 comp rx=0 tx=0).stop 2024-03-23T09:48:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.650+0000 7f38d7448640 1 -- 172.21.15.47:0/2730336936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d01549e0 msgr2=0x7f38d0154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.650+0000 7f38d7448640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d01549e0 0x7f38d0154e60 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f38c8000f80 tx=0x7f38c8001360 comp rx=0 tx=0).stop 2024-03-23T09:48:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.650+0000 7f38d7448640 1 -- 172.21.15.47:0/2730336936 shutdown_connections 2024-03-23T09:48:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.650+0000 7f38d7448640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f38b406ac00 0x7f38b406d0c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.650+0000 7f38d7448640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d01549e0 0x7f38d0154e60 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.650+0000 7f38d7448640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38d0156640 0x7f38d01f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.650+0000 7f38d7448640 1 --2- 172.21.15.47:0/2730336936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38bc004d00 0x7f38d0157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.650+0000 7f38d7448640 1 -- 172.21.15.47:0/2730336936 >> 172.21.15.47:0/2730336936 conn(0x7f38d00659d0 msgr2=0x7f38d015ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.650+0000 7f38d7448640 1 -- 172.21.15.47:0/2730336936 shutdown_connections 2024-03-23T09:48:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:49.650+0000 7f38d7448640 1 -- 172.21.15.47:0/2730336936 wait complete. 2024-03-23T09:48:50.063 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:47:11.390397+0000', 'modified': '2024-03-23T09:48:49.302818+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8133}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8133': {'gid': 8133, 'name': 'c', 'rank': 0, 'incarnation': 56, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.47:6835/3671387340', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 3671387340}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 3671387340}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:48:50.063 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8133:up:replay 2024-03-23T09:48:51.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:48:51.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.178+0000 7ff08b084640 1 Processor -- start 2024-03-23T09:48:51.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.178+0000 7ff08b084640 1 -- start start 2024-03-23T09:48:51.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.178+0000 7ff08b084640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0840b1c40 0x7ff0840cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.178+0000 7ff08b084640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff084099110 0x7ff08415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.178+0000 7ff08b084640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0840cf8f0 con 0x7ff0840ce430 2024-03-23T09:48:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.178+0000 7ff08b084640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0840b0fa0 con 0x7ff084099110 2024-03-23T09:48:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.178+0000 7ff08b084640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0840cbd90 con 0x7ff0840b1c40 2024-03-23T09:48:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.179+0000 7ff07bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff0840ce430 0x7ff084098a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44994/0 (socket says 172.21.15.47:44994) 2024-03-23T09:48:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.179+0000 7ff07bfff640 1 -- 172.21.15.47:0/3223365541 learned_addr learned my addr 172.21.15.47:0/3223365541 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.179+0000 7ff089621640 1 --2- 172.21.15.47:0/3223365541 >> v2:172.21.15.175:3300/0 conn(0x7ff084099110 0x7ff08415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.179+0000 7ff088e20640 1 --2- 172.21.15.47:0/3223365541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0840b1c40 0x7ff0840cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.179+0000 7ff07b7fe640 1 -- 172.21.15.47:0/3223365541 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612805278 0 0) 0x7ff0840cf8f0 con 0x7ff0840ce430 2024-03-23T09:48:51.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.179+0000 7ff07b7fe640 1 -- 172.21.15.47:0/3223365541 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff06c003520 con 0x7ff0840ce430 2024-03-23T09:48:51.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.179+0000 7ff089621640 1 -- 172.21.15.47:0/3223365541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0840b1c40 msgr2=0x7ff0840cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:51.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.179+0000 7ff089621640 1 --2- 172.21.15.47:0/3223365541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0840b1c40 0x7ff0840cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:51.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.179+0000 7ff089621640 1 -- 172.21.15.47:0/3223365541 >> v1:172.21.15.47:6789/0 conn(0x7ff0840ce430 legacy=0x7ff084098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:51.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.179+0000 7ff07b7fe640 1 -- 172.21.15.47:0/3223365541 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 594698783 0 0) 0x7ff06c003520 con 0x7ff0840ce430 2024-03-23T09:48:51.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.179+0000 7ff089621640 1 -- 172.21.15.47:0/3223365541 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0840cc070 con 0x7ff084099110 2024-03-23T09:48:51.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.179+0000 7ff088e20640 1 --2- 172.21.15.47:0/3223365541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0840b1c40 0x7ff0840cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:48:51.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.180+0000 7ff089621640 1 --2- 172.21.15.47:0/3223365541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084099110 0x7ff08415a2a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009d90 tx=0x7ff07c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5be8e1e4bd46af server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:51.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.180+0000 7ff07b7fe640 1 -- 172.21.15.47:0/3223365541 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c02cc50 con 0x7ff084099110 2024-03-23T09:48:51.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.180+0000 7ff07b7fe640 1 -- 172.21.15.47:0/3223365541 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c02cdb0 con 0x7ff084099110 2024-03-23T09:48:51.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.180+0000 7ff07b7fe640 1 -- 172.21.15.47:0/3223365541 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c034640 con 0x7ff084099110 2024-03-23T09:48:51.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.181+0000 7ff08b084640 1 -- 172.21.15.47:0/3223365541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084099110 msgr2=0x7ff08415a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:51.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.181+0000 7ff08b084640 1 --2- 172.21.15.47:0/3223365541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084099110 0x7ff08415a2a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009d90 tx=0x7ff07c02bea0 comp rx=0 tx=0).stop 2024-03-23T09:48:51.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.181+0000 7ff08b084640 1 -- 172.21.15.47:0/3223365541 shutdown_connections 2024-03-23T09:48:51.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.181+0000 7ff08b084640 1 --2- 172.21.15.47:0/3223365541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084099110 0x7ff08415a2a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:51.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.181+0000 7ff08b084640 1 --2- 172.21.15.47:0/3223365541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0840b1c40 0x7ff0840cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:51.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.181+0000 7ff08b084640 1 -- 172.21.15.47:0/3223365541 >> 172.21.15.47:0/3223365541 conn(0x7ff0840b2140 msgr2=0x7ff0840b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:51.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.181+0000 7ff08b084640 1 -- 172.21.15.47:0/3223365541 shutdown_connections 2024-03-23T09:48:51.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.181+0000 7ff08b084640 1 -- 172.21.15.47:0/3223365541 wait complete. 2024-03-23T09:48:51.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.182+0000 7ff08b084640 1 Processor -- start 2024-03-23T09:48:51.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.182+0000 7ff08b084640 1 -- start start 2024-03-23T09:48:51.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.183+0000 7ff08b084640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084099110 0x7ff0840afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:51.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.183+0000 7ff08b084640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0840b1c40 0x7ff0840ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:51.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.183+0000 7ff08b084640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840ce430 0x7ff0841f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:51.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.183+0000 7ff08b084640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0840ad400 con 0x7ff0840ce430 2024-03-23T09:48:51.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.183+0000 7ff08b084640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0840ad6e0 con 0x7ff084099110 2024-03-23T09:48:51.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.183+0000 7ff08b084640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0840ad9c0 con 0x7ff0840b1c40 2024-03-23T09:48:51.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.183+0000 7ff089621640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840ce430 0x7ff0841f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:51.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.183+0000 7ff089621640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840ce430 0x7ff0841f17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49368/0 (socket says 172.21.15.47:49368) 2024-03-23T09:48:51.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.183+0000 7ff089621640 1 -- 172.21.15.47:0/2345929868 learned_addr learned my addr 172.21.15.47:0/2345929868 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:51.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.183+0000 7ff088e20640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084099110 0x7ff0840afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:51.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.183+0000 7ff07bfff640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0840b1c40 0x7ff0840ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:51.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.183+0000 7ff089621640 1 -- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0840b1c40 msgr2=0x7ff0840ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:51.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.183+0000 7ff089621640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0840b1c40 0x7ff0840ae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:51.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.184+0000 7ff089621640 1 -- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084099110 msgr2=0x7ff0840afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:51.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.184+0000 7ff089621640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084099110 0x7ff0840afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:51.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.184+0000 7ff089621640 1 -- 172.21.15.47:0/2345929868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff080009400 con 0x7ff0840ce430 2024-03-23T09:48:51.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.184+0000 7ff088e20640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084099110 0x7ff0840afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:51.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.184+0000 7ff089621640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840ce430 0x7ff0841f17d0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009eb0 tx=0x7ff07c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:51.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.184+0000 7ff0797fa640 1 -- 172.21.15.47:0/2345929868 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c009450 con 0x7ff0840ce430 2024-03-23T09:48:51.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.184+0000 7ff0797fa640 1 -- 172.21.15.47:0/2345929868 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c03f420 con 0x7ff0840ce430 2024-03-23T09:48:51.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.184+0000 7ff08b084640 1 -- 172.21.15.47:0/2345929868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07c009a70 con 0x7ff0840ce430 2024-03-23T09:48:51.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.184+0000 7ff0797fa640 1 -- 172.21.15.47:0/2345929868 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c03f7d0 con 0x7ff0840ce430 2024-03-23T09:48:51.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.184+0000 7ff08b084640 1 -- 172.21.15.47:0/2345929868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0840adf80 con 0x7ff0840ce430 2024-03-23T09:48:51.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.185+0000 7ff08b084640 1 -- 172.21.15.47:0/2345929868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0840ae170 con 0x7ff0840ce430 2024-03-23T09:48:51.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.186+0000 7ff0797fa640 1 -- 172.21.15.47:0/2345929868 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff07c03f930 con 0x7ff0840ce430 2024-03-23T09:48:51.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.186+0000 7ff0797fa640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff07006ac00 0x7ff07006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:51.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.186+0000 7ff0797fa640 1 -- 172.21.15.47:0/2345929868 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff07c0abfc0 con 0x7ff0840ce430 2024-03-23T09:48:51.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.186+0000 7ff088e20640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff07006ac00 0x7ff07006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:51.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.187+0000 7ff088e20640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff07006ac00 0x7ff07006d0c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff080009bb0 tx=0x7ff080008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:51.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.190+0000 7ff0797fa640 1 -- 172.21.15.47:0/2345929868 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff07c0df580 con 0x7ff0840ce430 2024-03-23T09:48:51.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.410+0000 7ff08b084640 1 -- 172.21.15.47:0/2345929868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff084066be0 con 0x7ff0840ce430 2024-03-23T09:48:51.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.411+0000 7ff0797fa640 1 -- 172.21.15.47:0/2345929868 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4740 (secure 0 0 0) 0x7ff07c0770c0 con 0x7ff0840ce430 2024-03-23T09:48:51.411 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:48:51.411 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:48:51.310290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":56,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.47:6835/3671387340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3671387340},{"type":"v1","addr":"172.21.15.47:6835","nonce":3671387340}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:48:51.411 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 58 2024-03-23T09:48:51.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.412+0000 7ff08b084640 1 -- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff07006ac00 msgr2=0x7ff07006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:51.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.412+0000 7ff08b084640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff07006ac00 0x7ff07006d0c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff080009bb0 tx=0x7ff080008040 comp rx=0 tx=0).stop 2024-03-23T09:48:51.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.412+0000 7ff08b084640 1 -- 172.21.15.47:0/2345929868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840ce430 msgr2=0x7ff0841f17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:51.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.412+0000 7ff08b084640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840ce430 0x7ff0841f17d0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009eb0 tx=0x7ff07c004a40 comp rx=0 tx=0).stop 2024-03-23T09:48:51.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.413+0000 7ff08b084640 1 -- 172.21.15.47:0/2345929868 shutdown_connections 2024-03-23T09:48:51.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.413+0000 7ff08b084640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff07006ac00 0x7ff07006d0c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:51.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.413+0000 7ff08b084640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840ce430 0x7ff0841f17d0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:51.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.413+0000 7ff08b084640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0840b1c40 0x7ff0840ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:51.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.413+0000 7ff08b084640 1 --2- 172.21.15.47:0/2345929868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084099110 0x7ff0840afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:51.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.413+0000 7ff08b084640 1 -- 172.21.15.47:0/2345929868 >> 172.21.15.47:0/2345929868 conn(0x7ff0840b2140 msgr2=0x7ff08406fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:51.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.413+0000 7ff08b084640 1 -- 172.21.15.47:0/2345929868 shutdown_connections 2024-03-23T09:48:51.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:51.413+0000 7ff08b084640 1 -- 172.21.15.47:0/2345929868 wait complete. 2024-03-23T09:48:51.826 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:47:11.390397+0000', 'modified': '2024-03-23T09:48:51.310290+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8133}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8133': {'gid': 8133, 'name': 'c', 'rank': 0, 'incarnation': 56, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.47:6835/3671387340', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 3671387340}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 3671387340}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:48:51.826 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8133:up:rejoin 2024-03-23T09:48:52.827 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:48:52.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.940+0000 7f0c74a8b640 1 Processor -- start 2024-03-23T09:48:52.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.940+0000 7f0c74a8b640 1 -- start start 2024-03-23T09:48:52.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.941+0000 7f0c74a8b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0c7015c9a0 0x7f0c7006b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.941+0000 7f0c74a8b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c7015d940 0x7f0c700ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.941+0000 7f0c74a8b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7015df60 con 0x7f0c7006bd40 2024-03-23T09:48:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.941+0000 7f0c74a8b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c7006c1d0 con 0x7f0c7015c9a0 2024-03-23T09:48:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.941+0000 7f0c74a8b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7015dd60 con 0x7f0c7015d940 2024-03-23T09:48:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.942+0000 7f0c6e59c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0c7015c9a0 0x7f0c7006b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.942+0000 7f0c6ed9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c7015d940 0x7f0c700ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.942+0000 7f0c6dd9b640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f0c7006bd40 0x7f0c7015d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:58676/0 (socket says 172.21.15.47:58676) 2024-03-23T09:48:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.942+0000 7f0c6e59c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0c7015c9a0 0x7f0c7006b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:57210/0 (socket says 172.21.15.47:57210) 2024-03-23T09:48:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.942+0000 7f0c6ed9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c7015d940 0x7f0c700ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:56502/0 (socket says 172.21.15.47:56502) 2024-03-23T09:48:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.942+0000 7f0c6e59c640 1 -- 172.21.15.47:0/1798392563 learned_addr learned my addr 172.21.15.47:0/1798392563 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.942+0000 7f0c6e59c640 1 -- 172.21.15.47:0/1798392563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c7015d940 msgr2=0x7f0c700ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.942+0000 7f0c6e59c640 1 --2- 172.21.15.47:0/1798392563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c7015d940 0x7f0c700ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.942+0000 7f0c6e59c640 1 -- 172.21.15.47:0/1798392563 >> v1:172.21.15.175:6789/0 conn(0x7f0c7006bd40 legacy=0x7f0c7015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.942+0000 7f0c6e59c640 1 -- 172.21.15.47:0/1798392563 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c700b1530 con 0x7f0c7015c9a0 2024-03-23T09:48:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.942+0000 7f0c6ed9d640 1 --2- 172.21.15.47:0/1798392563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c7015d940 0x7f0c700ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:48:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.943+0000 7f0c6e59c640 1 --2- 172.21.15.47:0/1798392563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c7015c9a0 0x7f0c7006b800 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009870 tx=0x7f0c6802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d982291dfaadac45 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.943+0000 7f0c6d59a640 1 -- 172.21.15.47:0/1798392563 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68030840 con 0x7f0c7015c9a0 2024-03-23T09:48:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.943+0000 7f0c6d59a640 1 -- 172.21.15.47:0/1798392563 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c680309a0 con 0x7f0c7015c9a0 2024-03-23T09:48:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.943+0000 7f0c6d59a640 1 -- 172.21.15.47:0/1798392563 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68030d30 con 0x7f0c7015c9a0 2024-03-23T09:48:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.943+0000 7f0c74a8b640 1 -- 172.21.15.47:0/1798392563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c7015c9a0 msgr2=0x7f0c7006b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:52.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.943+0000 7f0c74a8b640 1 --2- 172.21.15.47:0/1798392563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c7015c9a0 0x7f0c7006b800 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009870 tx=0x7f0c6802b980 comp rx=0 tx=0).stop 2024-03-23T09:48:52.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.944+0000 7f0c74a8b640 1 -- 172.21.15.47:0/1798392563 shutdown_connections 2024-03-23T09:48:52.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.944+0000 7f0c74a8b640 1 --2- 172.21.15.47:0/1798392563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c7015d940 0x7f0c700ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:52.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.944+0000 7f0c74a8b640 1 --2- 172.21.15.47:0/1798392563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c7015c9a0 0x7f0c7006b800 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:52.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.944+0000 7f0c74a8b640 1 -- 172.21.15.47:0/1798392563 >> 172.21.15.47:0/1798392563 conn(0x7f0c700d0670 msgr2=0x7f0c700d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:52.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.944+0000 7f0c74a8b640 1 -- 172.21.15.47:0/1798392563 shutdown_connections 2024-03-23T09:48:52.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.944+0000 7f0c74a8b640 1 -- 172.21.15.47:0/1798392563 wait complete. 2024-03-23T09:48:52.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.945+0000 7f0c74a8b640 1 Processor -- start 2024-03-23T09:48:52.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.945+0000 7f0c74a8b640 1 -- start start 2024-03-23T09:48:52.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c74a8b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c7006bd40 0x7f0c700c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:52.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c74a8b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c7015c9a0 0x7f0c700b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:52.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c74a8b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c7015d940 0x7f0c70159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:52.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c74a8b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7015ef50 con 0x7f0c7015c9a0 2024-03-23T09:48:52.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c74a8b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7015e340 con 0x7f0c7006bd40 2024-03-23T09:48:52.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c74a8b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7015e620 con 0x7f0c7015d940 2024-03-23T09:48:52.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c6dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c7015c9a0 0x7f0c700b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:52.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c6dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c7015c9a0 0x7f0c700b3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49376/0 (socket says 172.21.15.47:49376) 2024-03-23T09:48:52.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c6dd9b640 1 -- 172.21.15.47:0/3146352727 learned_addr learned my addr 172.21.15.47:0/3146352727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:52.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c6e59c640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c7006bd40 0x7f0c700c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:52.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c6ed9d640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c7015d940 0x7f0c70159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:52.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c6dd9b640 1 -- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c7015d940 msgr2=0x7f0c70159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:52.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c6dd9b640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c7015d940 0x7f0c70159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:52.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c6dd9b640 1 -- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c7006bd40 msgr2=0x7f0c700c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:52.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c6dd9b640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c7006bd40 0x7f0c700c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:52.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c6dd9b640 1 -- 172.21.15.47:0/3146352727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c60009a70 con 0x7f0c7015c9a0 2024-03-23T09:48:52.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.946+0000 7f0c6dd9b640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c7015c9a0 0x7f0c700b3bf0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800c400 tx=0x7f0c5800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:52.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.947+0000 7f0c66ffd640 1 -- 172.21.15.47:0/3146352727 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c580074e0 con 0x7f0c7015c9a0 2024-03-23T09:48:52.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.947+0000 7f0c66ffd640 1 -- 172.21.15.47:0/3146352727 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c58007640 con 0x7f0c7015c9a0 2024-03-23T09:48:52.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.947+0000 7f0c66ffd640 1 -- 172.21.15.47:0/3146352727 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58004a00 con 0x7f0c7015c9a0 2024-03-23T09:48:52.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.947+0000 7f0c74a8b640 1 -- 172.21.15.47:0/3146352727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c680094d0 con 0x7f0c7015c9a0 2024-03-23T09:48:52.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.947+0000 7f0c74a8b640 1 -- 172.21.15.47:0/3146352727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7015ebe0 con 0x7f0c7015c9a0 2024-03-23T09:48:52.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.948+0000 7f0c74a8b640 1 -- 172.21.15.47:0/3146352727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7015eda0 con 0x7f0c7015c9a0 2024-03-23T09:48:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.948+0000 7f0c66ffd640 1 -- 172.21.15.47:0/3146352727 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0c58004b60 con 0x7f0c7015c9a0 2024-03-23T09:48:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.948+0000 7f0c66ffd640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c5006ac00 0x7f0c5006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.949+0000 7f0c66ffd640 1 -- 172.21.15.47:0/3146352727 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f0c58088e60 con 0x7f0c7015c9a0 2024-03-23T09:48:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.949+0000 7f0c6e59c640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c5006ac00 0x7f0c5006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:52.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.949+0000 7f0c6e59c640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c5006ac00 0x7f0c5006d0c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0c68036000 tx=0x7f0c68008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:52.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:52.953+0000 7f0c66ffd640 1 -- 172.21.15.47:0/3146352727 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0c58053f50 con 0x7f0c7015c9a0 2024-03-23T09:48:53.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.168+0000 7f0c74a8b640 1 -- 172.21.15.47:0/3146352727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c700672f0 con 0x7f0c7015c9a0 2024-03-23T09:48:53.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.168+0000 7f0c66ffd640 1 -- 172.21.15.47:0/3146352727 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f0c58057570 con 0x7f0c7015c9a0 2024-03-23T09:48:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:48:53.169 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:48:52.314987+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6835/3671387340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3671387340},{"type":"v1","addr":"172.21.15.47:6835","nonce":3671387340}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:48:53.169 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 59 2024-03-23T09:48:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.170+0000 7f0c74a8b640 1 -- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c5006ac00 msgr2=0x7f0c5006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.170+0000 7f0c74a8b640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c5006ac00 0x7f0c5006d0c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0c68036000 tx=0x7f0c68008030 comp rx=0 tx=0).stop 2024-03-23T09:48:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.170+0000 7f0c74a8b640 1 -- 172.21.15.47:0/3146352727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c7015c9a0 msgr2=0x7f0c700b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.170+0000 7f0c74a8b640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c7015c9a0 0x7f0c700b3bf0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800c400 tx=0x7f0c5800c8c0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.170+0000 7f0c74a8b640 1 -- 172.21.15.47:0/3146352727 shutdown_connections 2024-03-23T09:48:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.170+0000 7f0c74a8b640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0c5006ac00 0x7f0c5006d0c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.170+0000 7f0c74a8b640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c7015d940 0x7f0c70159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.170+0000 7f0c74a8b640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c7015c9a0 0x7f0c700b3bf0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.170+0000 7f0c74a8b640 1 --2- 172.21.15.47:0/3146352727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c7006bd40 0x7f0c700c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.170+0000 7f0c74a8b640 1 -- 172.21.15.47:0/3146352727 >> 172.21.15.47:0/3146352727 conn(0x7f0c700d0670 msgr2=0x7f0c700b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.170+0000 7f0c74a8b640 1 -- 172.21.15.47:0/3146352727 shutdown_connections 2024-03-23T09:48:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.170+0000 7f0c74a8b640 1 -- 172.21.15.47:0/3146352727 wait complete. 2024-03-23T09:48:53.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:47:11.390397+0000', 'modified': '2024-03-23T09:48:52.314987+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8133}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8133': {'gid': 8133, 'name': 'c', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.47:6835/3671387340', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 3671387340}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 3671387340}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:48:53.584 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:48:53.584 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T09:48:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.687+0000 7ff6477fa640 1 Processor -- start 2024-03-23T09:48:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.687+0000 7ff6477fa640 1 -- start start 2024-03-23T09:48:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff6477fa640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff64006d6b0 0x7ff64006a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff6477fa640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64015cec0 con 0x7ff64006aa20 2024-03-23T09:48:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff6477fa640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff640070de0 con 0x7ff64015cae0 2024-03-23T09:48:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff6477fa640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6400710c0 con 0x7ff64006d6b0 2024-03-23T09:48:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff645d97640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff64006aa20 0x7ff6400b1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45006/0 (socket says 172.21.15.47:45006) 2024-03-23T09:48:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff645d97640 1 -- 172.21.15.47:0/709582630 learned_addr learned my addr 172.21.15.47:0/709582630 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff644d95640 1 --2- 172.21.15.47:0/709582630 >> v2:172.21.15.175:3300/0 conn(0x7ff64006d6b0 0x7ff64006a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394392577 0 0) 0x7ff64015cec0 con 0x7ff64006aa20 2024-03-23T09:48:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff62c003520 con 0x7ff64006aa20 2024-03-23T09:48:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3809671938 0 0) 0x7ff62c003520 con 0x7ff64006aa20 2024-03-23T09:48:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 >> v2:172.21.15.175:3300/0 conn(0x7ff64006d6b0 msgr2=0x7ff64006a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 --2- 172.21.15.47:0/709582630 >> v2:172.21.15.175:3300/0 conn(0x7ff64006d6b0 0x7ff64006a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 >> v1:172.21.15.175:6789/0 conn(0x7ff64015cae0 legacy=0x7ff6400705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400713a0 con 0x7ff64006aa20 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7ff63c001670 con 0x7ff64006aa20 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 >> v1:172.21.15.47:6789/0 conn(0x7ff64006aa20 legacy=0x7ff6400b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 --2- 172.21.15.47:0/709582630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62c003970 0x7ff62c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 --2- 172.21.15.47:0/709582630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff62c0047c0 0x7ff62c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 --2- 172.21.15.47:0/709582630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62c005180 0x7ff62c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6400710c0 con 0x7ff62c0047c0 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c00c020 con 0x7ff62c003970 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c00c1e0 con 0x7ff62c005180 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.688+0000 7ff644d95640 1 --2- 172.21.15.47:0/709582630 >> v2:172.21.15.175:3300/0 conn(0x7ff64006d6b0 0x7ff64006a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.689+0000 7ff645596640 1 --2- 172.21.15.47:0/709582630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62c003970 0x7ff62c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.689+0000 7ff644d95640 1 --2- 172.21.15.47:0/709582630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff62c0047c0 0x7ff62c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.689+0000 7ff645d97640 1 --2- 172.21.15.47:0/709582630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62c005180 0x7ff62c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.689+0000 7ff644d95640 1 -- 172.21.15.47:0/709582630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62c005180 msgr2=0x7ff62c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:53.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.689+0000 7ff644d95640 1 --2- 172.21.15.47:0/709582630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62c005180 0x7ff62c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.689+0000 7ff644d95640 1 -- 172.21.15.47:0/709582630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62c003970 msgr2=0x7ff62c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:53.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.689+0000 7ff644d95640 1 --2- 172.21.15.47:0/709582630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62c003970 0x7ff62c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.689+0000 7ff644d95640 1 -- 172.21.15.47:0/709582630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c003520 con 0x7ff62c0047c0 2024-03-23T09:48:53.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.689+0000 7ff645d97640 1 -- 172.21.15.47:0/709582630 reap_dead start 2024-03-23T09:48:53.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.690+0000 7ff644d95640 1 --2- 172.21.15.47:0/709582630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff62c0047c0 0x7ff62c004c40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ff630009c70 tx=0x7ff62c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9dffcd059219837 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:53.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.690+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630004020 con 0x7ff62c0047c0 2024-03-23T09:48:53.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.690+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff630004180 con 0x7ff62c0047c0 2024-03-23T09:48:53.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.690+0000 7ff637fff640 1 -- 172.21.15.47:0/709582630 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63002c8f0 con 0x7ff62c0047c0 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.690+0000 7ff6477fa640 1 -- 172.21.15.47:0/709582630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff62c0047c0 msgr2=0x7ff62c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.690+0000 7ff6477fa640 1 --2- 172.21.15.47:0/709582630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff62c0047c0 0x7ff62c004c40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ff630009c70 tx=0x7ff62c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.690+0000 7ff6477fa640 1 -- 172.21.15.47:0/709582630 shutdown_connections 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.690+0000 7ff6477fa640 1 --2- 172.21.15.47:0/709582630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff62c0047c0 0x7ff62c004c40 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.690+0000 7ff6477fa640 1 -- 172.21.15.47:0/709582630 >> 172.21.15.47:0/709582630 conn(0x7ff6400cb3c0 msgr2=0x7ff6400cb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.690+0000 7ff6477fa640 1 -- 172.21.15.47:0/709582630 shutdown_connections 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.690+0000 7ff6477fa640 1 -- 172.21.15.47:0/709582630 wait complete. 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.691+0000 7ff6477fa640 1 Processor -- start 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.692+0000 7ff6477fa640 1 -- start start 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.692+0000 7ff6477fa640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62c0047c0 0x7ff6400ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.692+0000 7ff6477fa640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64014f190 0x7ff6400af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.692+0000 7ff6477fa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6400af600 0x7ff64015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.692+0000 7ff6477fa640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6400afab0 con 0x7ff6400af600 2024-03-23T09:48:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.692+0000 7ff6477fa640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401f2240 con 0x7ff62c0047c0 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.692+0000 7ff6477fa640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401f2520 con 0x7ff64014f190 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.692+0000 7ff645d97640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6400af600 0x7ff64015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.692+0000 7ff645d97640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6400af600 0x7ff64015d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49394/0 (socket says 172.21.15.47:49394) 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.692+0000 7ff645d97640 1 -- 172.21.15.47:0/2692812224 learned_addr learned my addr 172.21.15.47:0/2692812224 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.693+0000 7ff645d97640 1 -- 172.21.15.47:0/2692812224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64014f190 msgr2=0x7ff6400af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.693+0000 7ff645d97640 1 --2- 172.21.15.47:0/2692812224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64014f190 0x7ff6400af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.693+0000 7ff645d97640 1 -- 172.21.15.47:0/2692812224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62c0047c0 msgr2=0x7ff6400ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.693+0000 7ff645d97640 1 --2- 172.21.15.47:0/2692812224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62c0047c0 0x7ff6400ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.693+0000 7ff645d97640 1 -- 172.21.15.47:0/2692812224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6401f2800 con 0x7ff6400af600 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.693+0000 7ff645d97640 1 --2- 172.21.15.47:0/2692812224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6400af600 0x7ff64015d0b0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff63c000c00 tx=0x7ff63c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.693+0000 7ff635ffb640 1 -- 172.21.15.47:0/2692812224 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c007770 con 0x7ff6400af600 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.693+0000 7ff635ffb640 1 -- 172.21.15.47:0/2692812224 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c00f6a0 con 0x7ff6400af600 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.693+0000 7ff635ffb640 1 -- 172.21.15.47:0/2692812224 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c00fa50 con 0x7ff6400af600 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.693+0000 7ff6477fa640 1 -- 172.21.15.47:0/2692812224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6401f2ae0 con 0x7ff6400af600 2024-03-23T09:48:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.693+0000 7ff6477fa640 1 -- 172.21.15.47:0/2692812224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6400adda0 con 0x7ff6400af600 2024-03-23T09:48:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.694+0000 7ff635ffb640 1 -- 172.21.15.47:0/2692812224 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff63c00fbb0 con 0x7ff6400af600 2024-03-23T09:48:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.695+0000 7ff635ffb640 1 --2- 172.21.15.47:0/2692812224 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff62406ac00 0x7ff62406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.695+0000 7ff635ffb640 1 -- 172.21.15.47:0/2692812224 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff63c089230 con 0x7ff6400af600 2024-03-23T09:48:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.695+0000 7ff645596640 1 --2- 172.21.15.47:0/2692812224 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff62406ac00 0x7ff62406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.696+0000 7ff645596640 1 --2- 172.21.15.47:0/2692812224 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff62406ac00 0x7ff62406d0c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff638000f80 tx=0x7ff638009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.696+0000 7ff64a368740 1 Processor -- start 2024-03-23T09:48:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.696+0000 7ff64a368740 1 -- start start 2024-03-23T09:48:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff64a368740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 0x557a8e5e1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff64a368740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e5e07d0 0x557a8e5e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff64a368740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5dcc70 0x557a8e60ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff64a368740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8e54d9b0 con 0x557a8e62a750 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff64a368740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8e54b140 con 0x557a8e5dcc70 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff64a368740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a8e546a30 con 0x557a8e5e07d0 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff645596640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 0x557a8e5e1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff645596640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 0x557a8e5e1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49408/0 (socket says 172.21.15.47:49408) 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff645596640 1 -- 172.21.15.47:0/2885448655 learned_addr learned my addr 172.21.15.47:0/2885448655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff645d97640 1 --2- 172.21.15.47:0/2885448655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5dcc70 0x557a8e60ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff644d95640 1 --2- 172.21.15.47:0/2885448655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e5e07d0 0x557a8e5e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff645596640 1 -- 172.21.15.47:0/2885448655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e5e07d0 msgr2=0x557a8e5e22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff645596640 1 --2- 172.21.15.47:0/2885448655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e5e07d0 0x557a8e5e22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff645596640 1 -- 172.21.15.47:0/2885448655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5dcc70 msgr2=0x557a8e60ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff645596640 1 --2- 172.21.15.47:0/2885448655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5dcc70 0x557a8e60ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff645596640 1 -- 172.21.15.47:0/2885448655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a8e4d32f0 con 0x557a8e62a750 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.697+0000 7ff645d97640 1 --2- 172.21.15.47:0/2885448655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5dcc70 0x557a8e60ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.698+0000 7ff645596640 1 --2- 172.21.15.47:0/2885448655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 0x557a8e5e1d80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff63800fa50 tx=0x7ff638012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4dccd9aa50d3b0d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:53.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.698+0000 7ff622ffd640 1 -- 172.21.15.47:0/2885448655 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638014be0 con 0x557a8e62a750 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.698+0000 7ff622ffd640 1 -- 172.21.15.47:0/2885448655 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff638014d40 con 0x557a8e62a750 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.698+0000 7ff622ffd640 1 -- 172.21.15.47:0/2885448655 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638020610 con 0x557a8e62a750 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.698+0000 7ff64a368740 1 -- 172.21.15.47:0/2885448655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 msgr2=0x557a8e5e1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.698+0000 7ff64a368740 1 --2- 172.21.15.47:0/2885448655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 0x557a8e5e1d80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff63800fa50 tx=0x7ff638012000 comp rx=0 tx=0).stop 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.698+0000 7ff64a368740 1 -- 172.21.15.47:0/2885448655 shutdown_connections 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.698+0000 7ff64a368740 1 --2- 172.21.15.47:0/2885448655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5dcc70 0x557a8e60ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.698+0000 7ff64a368740 1 --2- 172.21.15.47:0/2885448655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e5e07d0 0x557a8e5e22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.698+0000 7ff64a368740 1 --2- 172.21.15.47:0/2885448655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 0x557a8e5e1d80 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.698+0000 7ff64a368740 1 -- 172.21.15.47:0/2885448655 >> 172.21.15.47:0/2885448655 conn(0x557a8e5deee0 msgr2=0x557a8e627a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.698+0000 7ff64a368740 1 -- 172.21.15.47:0/2885448655 shutdown_connections 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.699+0000 7ff64a368740 1 -- 172.21.15.47:0/2885448655 wait complete. 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.699+0000 7ff64a368740 1 -- start start 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.699+0000 7ff64a368740 1 Processor -- start 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.700+0000 7ff64a368740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5e07d0 0x557a8e5f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.700+0000 7ff64a368740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 0x557a8e608160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.700+0000 7ff64a368740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e60a9b0 0x557a8e604500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.700+0000 7ff64a368740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8e505090 con 0x557a8e62a750 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.700+0000 7ff64a368740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8e504d30 con 0x557a8e5e07d0 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.700+0000 7ff64a368740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a8e4ca2a0 con 0x557a8e60a9b0 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.700+0000 7ff64a368740 10 client.0 _ll_register_callbacks cb 0x557a8e500050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.700+0000 7ff644d95640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 0x557a8e608160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.700+0000 7ff644d95640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 0x557a8e608160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49412/0 (socket says 172.21.15.47:49412) 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.700+0000 7ff644d95640 1 -- 172.21.15.47:0/2801221566 learned_addr learned my addr 172.21.15.47:0/2801221566 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.700+0000 7ff645596640 1 --2- 172.21.15.47:0/2801221566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5e07d0 0x557a8e5f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.700+0000 7ff645d97640 1 --2- 172.21.15.47:0/2801221566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e60a9b0 0x557a8e604500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff644d95640 1 -- 172.21.15.47:0/2801221566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e60a9b0 msgr2=0x557a8e604500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff644d95640 1 --2- 172.21.15.47:0/2801221566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e60a9b0 0x557a8e604500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff644d95640 1 -- 172.21.15.47:0/2801221566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5e07d0 msgr2=0x557a8e5f04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff644d95640 1 --2- 172.21.15.47:0/2801221566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5e07d0 0x557a8e5f04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff644d95640 1 -- 172.21.15.47:0/2801221566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff640070de0 con 0x557a8e62a750 2024-03-23T09:48:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff644d95640 1 --2- 172.21.15.47:0/2801221566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 0x557a8e608160 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ff630009d90 tx=0x7ff630004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e9af5f424ae422 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff61bfff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff61bfff640 1 -- 172.21.15.47:0/2801221566 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630005d60 con 0x557a8e62a750 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff61bfff640 1 -- 172.21.15.47:0/2801221566 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff63002c8f0 con 0x557a8e62a750 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff64a368740 1 -- 172.21.15.47:0/2801221566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a8e23a3c0 con 0x557a8e62a750 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff61bfff640 1 -- 172.21.15.47:0/2801221566 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff63002cab0 con 0x557a8e62a750 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff64a368740 10 client.8205 fetch_fsmap learned FSMap version 59 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff64a368740 1 -- 172.21.15.47:0/2801221566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x557a8e5450b0 con 0x557a8e62a750 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff61bfff640 1 -- 172.21.15.47:0/2801221566 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63002cc10 con 0x557a8e62a750 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff61bfff640 1 -- 172.21.15.47:0/2801221566 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 59) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff630006040 con 0x557a8e62a750 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff64a368740 10 client.8205 fetch_fsmap finished waiting for FSMap version 59 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff64a368740 10 client.8205 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff64a368740 20 client.8205 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff64a368740 1 --2- 172.21.15.47:0/2801221566 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x557a8e5f2620 0x557a8e5f3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff64a368740 4 client.8205 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.701+0000 7ff64a368740 1 -- 172.21.15.47:0/2801221566 --> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a8e23a3c0 con 0x557a8e5f2620 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.702+0000 7ff645596640 1 --2- 172.21.15.47:0/2801221566 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x557a8e5f2620 0x557a8e5f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.702+0000 7ff61bfff640 10 client.8205 ms_handle_connect on v2:172.21.15.47:6834/3671387340 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.702+0000 7ff645596640 1 --2- 172.21.15.47:0/2801221566 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x557a8e5f2620 0x557a8e5f3260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1cc83c96c90a641 server_cookie=48adf01a042606aa in_seq=0 out_seq=0 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.702+0000 7ff61bfff640 1 -- 172.21.15.47:0/2801221566 <== mds.0 v2:172.21.15.47:6834/3671387340 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557a8e23a3c0 con 0x557a8e5f2620 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.702+0000 7ff61bfff640 10 client.8205 handle_command_reply: tid=0 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.702+0000 7ff64a368740 1 client.8205 shutdown 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.703+0000 7ff64a368740 1 -- 172.21.15.47:0/2801221566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 msgr2=0x557a8e608160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.703+0000 7ff64a368740 1 --2- 172.21.15.47:0/2801221566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 0x557a8e608160 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ff630009d90 tx=0x7ff630004730 comp rx=0 tx=0).stop 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.703+0000 7ff64a368740 1 -- 172.21.15.47:0/2801221566 shutdown_connections 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.703+0000 7ff64a368740 1 --2- 172.21.15.47:0/2801221566 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x557a8e5f2620 0x557a8e5f3260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.703+0000 7ff64a368740 1 --2- 172.21.15.47:0/2801221566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e60a9b0 0x557a8e604500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.703+0000 7ff64a368740 1 --2- 172.21.15.47:0/2801221566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e62a750 0x557a8e608160 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.703+0000 7ff64a368740 1 --2- 172.21.15.47:0/2801221566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5e07d0 0x557a8e5f04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.703+0000 7ff64a368740 1 -- 172.21.15.47:0/2801221566 >> 172.21.15.47:0/2801221566 conn(0x557a8e5dcc70 msgr2=0x557a8e5e0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.703+0000 7ff61bfff640 0 client.8205 ms_handle_reset on v2:172.21.15.47:6834/3671387340 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.703+0000 7ff64a368740 1 -- 172.21.15.47:0/2801221566 shutdown_connections 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.703+0000 7ff64a368740 1 -- 172.21.15.47:0/2801221566 wait complete. 2024-03-23T09:48:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.703+0000 7ff64a368740 20 client.8205 trim_cache size 0 max 16384 2024-03-23T09:48:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.718+0000 7ff64a368740 1 Processor -- start 2024-03-23T09:48:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.718+0000 7ff64a368740 1 -- start start 2024-03-23T09:48:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.719+0000 7ff64a368740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e5e07d0 0x557a8e640f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.719+0000 7ff64a368740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e60a9b0 0x557a8e5e18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.719+0000 7ff64a368740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e62a750 0x557a8e5e1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.719+0000 7ff64a368740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8e608010 con 0x557a8e5e07d0 2024-03-23T09:48:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.719+0000 7ff64a368740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8e5f1640 con 0x557a8e62a750 2024-03-23T09:48:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.719+0000 7ff64a368740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a8e6414d0 con 0x557a8e60a9b0 2024-03-23T09:48:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.719+0000 7ff645596640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e5e07d0 0x557a8e640f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.719+0000 7ff645596640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e5e07d0 0x557a8e640f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49426/0 (socket says 172.21.15.47:49426) 2024-03-23T09:48:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.719+0000 7ff645596640 1 -- 172.21.15.47:0/3019136292 learned_addr learned my addr 172.21.15.47:0/3019136292 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.719+0000 7ff645596640 1 -- 172.21.15.47:0/3019136292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e60a9b0 msgr2=0x557a8e5e18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.719+0000 7ff645596640 1 --2- 172.21.15.47:0/3019136292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e60a9b0 0x557a8e5e18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.719+0000 7ff645596640 1 -- 172.21.15.47:0/3019136292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e62a750 msgr2=0x557a8e5e1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.720+0000 7ff645596640 1 --2- 172.21.15.47:0/3019136292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e62a750 0x557a8e5e1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.720+0000 7ff645596640 1 -- 172.21.15.47:0/3019136292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a8e4e7430 con 0x557a8e5e07d0 2024-03-23T09:48:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.720+0000 7ff645596640 1 --2- 172.21.15.47:0/3019136292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e5e07d0 0x557a8e640f90 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7ff638020000 tx=0x7ff638014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc4e774695e428f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.720+0000 7ff6237fe640 1 -- 172.21.15.47:0/3019136292 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638031040 con 0x557a8e5e07d0 2024-03-23T09:48:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.720+0000 7ff6237fe640 1 -- 172.21.15.47:0/3019136292 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff638001a50 con 0x557a8e5e07d0 2024-03-23T09:48:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.720+0000 7ff6237fe640 1 -- 172.21.15.47:0/3019136292 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638014be0 con 0x557a8e5e07d0 2024-03-23T09:48:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.720+0000 7ff64a368740 1 -- 172.21.15.47:0/3019136292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e5e07d0 msgr2=0x557a8e640f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.720+0000 7ff64a368740 1 --2- 172.21.15.47:0/3019136292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e5e07d0 0x557a8e640f90 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7ff638020000 tx=0x7ff638014000 comp rx=0 tx=0).stop 2024-03-23T09:48:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.721+0000 7ff64a368740 1 -- 172.21.15.47:0/3019136292 shutdown_connections 2024-03-23T09:48:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.721+0000 7ff64a368740 1 --2- 172.21.15.47:0/3019136292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e62a750 0x557a8e5e1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.721+0000 7ff64a368740 1 --2- 172.21.15.47:0/3019136292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e60a9b0 0x557a8e5e18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.721+0000 7ff64a368740 1 --2- 172.21.15.47:0/3019136292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e5e07d0 0x557a8e640f90 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.721+0000 7ff64a368740 1 -- 172.21.15.47:0/3019136292 >> 172.21.15.47:0/3019136292 conn(0x557a8e5dcc70 msgr2=0x557a8e5e2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.721+0000 7ff64a368740 1 -- 172.21.15.47:0/3019136292 shutdown_connections 2024-03-23T09:48:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.721+0000 7ff64a368740 1 -- 172.21.15.47:0/3019136292 wait complete. 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.721+0000 7ff64a368740 1 -- start start 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.721+0000 7ff64a368740 1 Processor -- start 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff64a368740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5e07d0 0x557a8e628cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff64a368740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e60a9b0 0x557a8e629210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff64a368740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e62a750 0x557a8e639910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff64a368740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8e68f570 con 0x557a8e60a9b0 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff64a368740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8e68f6f0 con 0x557a8e5e07d0 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff64a368740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a8e68fb50 con 0x557a8e62a750 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff64a368740 10 client.0 _ll_register_callbacks cb 0x7ff638017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff644d95640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e60a9b0 0x557a8e629210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff644d95640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e60a9b0 0x557a8e629210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49440/0 (socket says 172.21.15.47:49440) 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff644d95640 1 -- 172.21.15.47:0/812050896 learned_addr learned my addr 172.21.15.47:0/812050896 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff645596640 1 --2- 172.21.15.47:0/812050896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5e07d0 0x557a8e628cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff645d97640 1 --2- 172.21.15.47:0/812050896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e62a750 0x557a8e639910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff644d95640 1 -- 172.21.15.47:0/812050896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e62a750 msgr2=0x557a8e639910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff644d95640 1 --2- 172.21.15.47:0/812050896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e62a750 0x557a8e639910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff644d95640 1 -- 172.21.15.47:0/812050896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5e07d0 msgr2=0x557a8e628cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff644d95640 1 --2- 172.21.15.47:0/812050896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5e07d0 0x557a8e628cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.722+0000 7ff644d95640 1 -- 172.21.15.47:0/812050896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a8e4d32f0 con 0x557a8e60a9b0 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff644d95640 1 --2- 172.21.15.47:0/812050896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e60a9b0 0x557a8e629210 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff63002c3d0 tx=0x7ff630030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0f70870f0f5e3f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff61bfff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff61bfff640 1 -- 172.21.15.47:0/812050896 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63002cd00 con 0x557a8e60a9b0 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff61bfff640 1 -- 172.21.15.47:0/812050896 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff6300065f0 con 0x557a8e60a9b0 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff64a368740 1 -- 172.21.15.47:0/812050896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a8e5ddc90 con 0x557a8e60a9b0 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff61bfff640 1 -- 172.21.15.47:0/812050896 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6300099e0 con 0x557a8e60a9b0 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff64a368740 10 client.8211 fetch_fsmap learned FSMap version 59 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff64a368740 1 -- 172.21.15.47:0/812050896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x557a8e50cb40 con 0x557a8e60a9b0 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff61bfff640 1 -- 172.21.15.47:0/812050896 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557a8e5ddc90 con 0x557a8e60a9b0 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff61bfff640 1 -- 172.21.15.47:0/812050896 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 59) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff630003610 con 0x557a8e60a9b0 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff64a368740 10 client.8211 fetch_fsmap finished waiting for FSMap version 59 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff64a368740 10 client.8211 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:48:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff64a368740 20 client.8211 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff64a368740 1 --2- 172.21.15.47:0/812050896 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x557a8e68bd70 0x557a8e639e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff64a368740 4 client.8211 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:48:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff64a368740 1 -- 172.21.15.47:0/812050896 --> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7ff638014d40 con 0x557a8e68bd70 2024-03-23T09:48:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.723+0000 7ff645596640 1 --2- 172.21.15.47:0/812050896 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x557a8e68bd70 0x557a8e639e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff61bfff640 10 client.8211 ms_handle_connect on v2:172.21.15.47:6834/3671387340 2024-03-23T09:48:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff645596640 1 --2- 172.21.15.47:0/812050896 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x557a8e68bd70 0x557a8e639e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df60b892106bc778 server_cookie=53b216a0d31049fd in_seq=0 out_seq=0 2024-03-23T09:48:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff61bfff640 1 -- 172.21.15.47:0/812050896 <== mds.0 v2:172.21.15.47:6834/3671387340 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7ff638014d40 con 0x557a8e68bd70 2024-03-23T09:48:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff61bfff640 10 client.8211 handle_command_reply: tid=0 2024-03-23T09:48:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff64a368740 1 client.8211 shutdown 2024-03-23T09:48:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff64a368740 1 -- 172.21.15.47:0/812050896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e60a9b0 msgr2=0x557a8e629210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff64a368740 1 --2- 172.21.15.47:0/812050896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e60a9b0 0x557a8e629210 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff63002c3d0 tx=0x7ff630030000 comp rx=0 tx=0).stop 2024-03-23T09:48:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff64a368740 1 -- 172.21.15.47:0/812050896 shutdown_connections 2024-03-23T09:48:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff64a368740 1 --2- 172.21.15.47:0/812050896 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x557a8e68bd70 0x557a8e639e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff64a368740 1 --2- 172.21.15.47:0/812050896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557a8e62a750 0x557a8e639910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff64a368740 1 --2- 172.21.15.47:0/812050896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a8e60a9b0 0x557a8e629210 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff61bfff640 0 client.8211 ms_handle_reset on v2:172.21.15.47:6834/3671387340 2024-03-23T09:48:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff64a368740 1 --2- 172.21.15.47:0/812050896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a8e5e07d0 0x557a8e628cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff64a368740 1 -- 172.21.15.47:0/812050896 >> 172.21.15.47:0/812050896 conn(0x557a8e5dcc70 msgr2=0x557a8e5e18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff64a368740 1 -- 172.21.15.47:0/812050896 shutdown_connections 2024-03-23T09:48:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff64a368740 1 -- 172.21.15.47:0/812050896 wait complete. 2024-03-23T09:48:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.724+0000 7ff64a368740 20 client.8211 trim_cache size 0 max 16384 2024-03-23T09:48:53.730 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:48:53.731 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:48:53.731 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:48:53.731 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 8133, 2024-03-23T09:48:53.731 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:48:53.731 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:48:53.731 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:48:53.731 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 4.4159481090000003, 2024-03-23T09:48:53.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 59, 2024-03-23T09:48:53.731 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 89, 2024-03-23T09:48:53.731 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 89, 2024-03-23T09:48:53.731 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 5.3985214030000002 2024-03-23T09:48:53.731 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:48:53.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.725+0000 7ff6477fa640 1 -- 172.21.15.47:0/2692812224 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff62406ac00 msgr2=0x7ff62406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.725+0000 7ff6477fa640 1 --2- 172.21.15.47:0/2692812224 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff62406ac00 0x7ff62406d0c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff638000f80 tx=0x7ff638009040 comp rx=0 tx=0).stop 2024-03-23T09:48:53.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.725+0000 7ff6477fa640 1 -- 172.21.15.47:0/2692812224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6400af600 msgr2=0x7ff64015d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.725+0000 7ff6477fa640 1 --2- 172.21.15.47:0/2692812224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6400af600 0x7ff64015d0b0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff63c000c00 tx=0x7ff63c007430 comp rx=0 tx=0).stop 2024-03-23T09:48:53.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.725+0000 7ff6477fa640 1 -- 172.21.15.47:0/2692812224 shutdown_connections 2024-03-23T09:48:53.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.725+0000 7ff6477fa640 1 --2- 172.21.15.47:0/2692812224 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff62406ac00 0x7ff62406d0c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.725+0000 7ff6477fa640 1 --2- 172.21.15.47:0/2692812224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6400af600 0x7ff64015d0b0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.725+0000 7ff6477fa640 1 --2- 172.21.15.47:0/2692812224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64014f190 0x7ff6400af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.725+0000 7ff6477fa640 1 --2- 172.21.15.47:0/2692812224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62c0047c0 0x7ff6400ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.725+0000 7ff6477fa640 1 -- 172.21.15.47:0/2692812224 >> 172.21.15.47:0/2692812224 conn(0x7ff6400cb3c0 msgr2=0x7ff64015a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:53.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.725+0000 7ff6477fa640 1 -- 172.21.15.47:0/2692812224 shutdown_connections 2024-03-23T09:48:53.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.725+0000 7ff6477fa640 1 -- 172.21.15.47:0/2692812224 wait complete. 2024-03-23T09:48:53.735 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:48:53.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.835+0000 7f0d21c19640 1 Processor -- start 2024-03-23T09:48:53.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.836+0000 7f0d21c19640 1 -- start start 2024-03-23T09:48:53.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.836+0000 7f0d21c19640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0d1c15d0a0 0x7f0d1c0d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.836+0000 7f0d21c19640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1c0ce090 con 0x7f0d1c15ccc0 2024-03-23T09:48:53.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.836+0000 7f0d21c19640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1c0d0f00 con 0x7f0d1c06b210 2024-03-23T09:48:53.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.836+0000 7f0d21c19640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1c06c0e0 con 0x7f0d1c15d0a0 2024-03-23T09:48:53.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.836+0000 7f0d1b7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0d1c15ccc0 0x7f0d1c0cd980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45016/0 (socket says 172.21.15.47:45016) 2024-03-23T09:48:53.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.836+0000 7f0d1b7fe640 1 -- 172.21.15.47:0/44811661 learned_addr learned my addr 172.21.15.47:0/44811661 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:53.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.836+0000 7f0d1affd640 1 --2- 172.21.15.47:0/44811661 >> v2:172.21.15.47:3300/0 conn(0x7f0d1c15d0a0 0x7f0d1c0d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.837+0000 7f0d1affd640 1 -- 172.21.15.47:0/44811661 >> v1:172.21.15.175:6789/0 conn(0x7f0d1c06b210 legacy=0x7f0d1c0b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:53.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.837+0000 7f0d1affd640 1 -- 172.21.15.47:0/44811661 >> v1:172.21.15.47:6789/0 conn(0x7f0d1c15ccc0 legacy=0x7f0d1c0cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.837+0000 7f0d1affd640 1 -- 172.21.15.47:0/44811661 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0ce2d0 con 0x7f0d1c15d0a0 2024-03-23T09:48:53.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.837+0000 7f0d1a7fc640 1 -- 172.21.15.47:0/44811661 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174593925 0 0) 0x7f0d0c001110 con 0x7f0d1c15ccc0 2024-03-23T09:48:53.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.837+0000 7f0d1affd640 1 --2- 172.21.15.47:0/44811661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d1c15d0a0 0x7f0d1c0d1c30 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f0d100097a0 tx=0x7f0d1002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55ca09052f42916 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:53.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.838+0000 7f0d1a7fc640 1 -- 172.21.15.47:0/44811661 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10030610 con 0x7f0d1c15d0a0 2024-03-23T09:48:53.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.838+0000 7f0d1a7fc640 1 -- 172.21.15.47:0/44811661 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d10030770 con 0x7f0d1c15d0a0 2024-03-23T09:48:53.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.838+0000 7f0d1a7fc640 1 -- 172.21.15.47:0/44811661 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10030b00 con 0x7f0d1c15d0a0 2024-03-23T09:48:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.838+0000 7f0d21c19640 1 -- 172.21.15.47:0/44811661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d1c15d0a0 msgr2=0x7f0d1c0d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.838+0000 7f0d21c19640 1 --2- 172.21.15.47:0/44811661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d1c15d0a0 0x7f0d1c0d1c30 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f0d100097a0 tx=0x7f0d1002b960 comp rx=0 tx=0).stop 2024-03-23T09:48:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.838+0000 7f0d21c19640 1 -- 172.21.15.47:0/44811661 shutdown_connections 2024-03-23T09:48:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.838+0000 7f0d21c19640 1 --2- 172.21.15.47:0/44811661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d1c15d0a0 0x7f0d1c0d1c30 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.838+0000 7f0d21c19640 1 -- 172.21.15.47:0/44811661 >> 172.21.15.47:0/44811661 conn(0x7f0d1c0cf430 msgr2=0x7f0d1c0cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.838+0000 7f0d21c19640 1 -- 172.21.15.47:0/44811661 shutdown_connections 2024-03-23T09:48:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.838+0000 7f0d21c19640 1 -- 172.21.15.47:0/44811661 wait complete. 2024-03-23T09:48:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.839+0000 7f0d21c19640 1 Processor -- start 2024-03-23T09:48:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.840+0000 7f0d21c19640 1 -- start start 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.840+0000 7f0d21c19640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d1c06b210 0x7f0d1c1e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.840+0000 7f0d21c19640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d1c15ccc0 0x7f0d1c1594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.840+0000 7f0d21c19640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d1c15d0a0 0x7f0d1c1e4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.840+0000 7f0d21c19640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c1e4d90 con 0x7f0d1c15ccc0 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.840+0000 7f0d21c19640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c1e5070 con 0x7f0d1c15d0a0 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.840+0000 7f0d21c19640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c1e5350 con 0x7f0d1c06b210 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.840+0000 7f0d1affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d1c15ccc0 0x7f0d1c1594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.840+0000 7f0d1affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d1c15ccc0 0x7f0d1c1594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49460/0 (socket says 172.21.15.47:49460) 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.840+0000 7f0d1affd640 1 -- 172.21.15.47:0/4277672294 learned_addr learned my addr 172.21.15.47:0/4277672294 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.841+0000 7f0d1bfff640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d1c15d0a0 0x7f0d1c1e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.841+0000 7f0d1b7fe640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d1c06b210 0x7f0d1c1e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.841+0000 7f0d1affd640 1 -- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d1c06b210 msgr2=0x7f0d1c1e5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.841+0000 7f0d1affd640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d1c06b210 0x7f0d1c1e5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.841+0000 7f0d1affd640 1 -- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d1c15d0a0 msgr2=0x7f0d1c1e4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.841+0000 7f0d1affd640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d1c15d0a0 0x7f0d1c1e4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.841+0000 7f0d1affd640 1 -- 172.21.15.47:0/4277672294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d10009400 con 0x7f0d1c15ccc0 2024-03-23T09:48:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.841+0000 7f0d1affd640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d1c15ccc0 0x7f0d1c1594d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f0d100098c0 tx=0x7f0d100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.841+0000 7f0d03fff640 1 -- 172.21.15.47:0/4277672294 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10039c10 con 0x7f0d1c15ccc0 2024-03-23T09:48:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.841+0000 7f0d03fff640 1 -- 172.21.15.47:0/4277672294 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1003f040 con 0x7f0d1c15ccc0 2024-03-23T09:48:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.841+0000 7f0d21c19640 1 -- 172.21.15.47:0/4277672294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1c1e5630 con 0x7f0d1c15ccc0 2024-03-23T09:48:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.841+0000 7f0d03fff640 1 -- 172.21.15.47:0/4277672294 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d100433f0 con 0x7f0d1c15ccc0 2024-03-23T09:48:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.841+0000 7f0d21c19640 1 -- 172.21.15.47:0/4277672294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1c1565f0 con 0x7f0d1c15ccc0 2024-03-23T09:48:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.842+0000 7f0d21c19640 1 -- 172.21.15.47:0/4277672294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1c156780 con 0x7f0d1c15ccc0 2024-03-23T09:48:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.843+0000 7f0d03fff640 1 -- 172.21.15.47:0/4277672294 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0d10043550 con 0x7f0d1c15ccc0 2024-03-23T09:48:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.843+0000 7f0d03fff640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d0806ac00 0x7f0d0806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.843+0000 7f0d03fff640 1 -- 172.21.15.47:0/4277672294 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f0d100abda0 con 0x7f0d1c15ccc0 2024-03-23T09:48:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.843+0000 7f0d1b7fe640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d0806ac00 0x7f0d0806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.844+0000 7f0d1b7fe640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d0806ac00 0x7f0d0806d0c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000f60 tx=0x7f0d0c007b90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:53.847+0000 7f0d03fff640 1 -- 172.21.15.47:0/4277672294 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0d10076fd0 con 0x7f0d1c15ccc0 2024-03-23T09:48:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.067+0000 7f0d21c19640 1 -- 172.21.15.47:0/4277672294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0d1c072360 con 0x7f0d1c15ccc0 2024-03-23T09:48:54.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.068+0000 7f0d03fff640 1 -- 172.21.15.47:0/4277672294 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0d10034070 con 0x7f0d1c15ccc0 2024-03-23T09:48:54.068 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:48:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.069+0000 7f0d21c19640 1 -- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d0806ac00 msgr2=0x7f0d0806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.069+0000 7f0d21c19640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d0806ac00 0x7f0d0806d0c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000f60 tx=0x7f0d0c007b90 comp rx=0 tx=0).stop 2024-03-23T09:48:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.070+0000 7f0d21c19640 1 -- 172.21.15.47:0/4277672294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d1c15ccc0 msgr2=0x7f0d1c1594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.070+0000 7f0d21c19640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d1c15ccc0 0x7f0d1c1594d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f0d100098c0 tx=0x7f0d100046c0 comp rx=0 tx=0).stop 2024-03-23T09:48:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.070+0000 7f0d21c19640 1 -- 172.21.15.47:0/4277672294 shutdown_connections 2024-03-23T09:48:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.070+0000 7f0d21c19640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0d0806ac00 0x7f0d0806d0c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.070+0000 7f0d21c19640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d1c15d0a0 0x7f0d1c1e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.070+0000 7f0d21c19640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d1c15ccc0 0x7f0d1c1594d0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.070+0000 7f0d21c19640 1 --2- 172.21.15.47:0/4277672294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d1c06b210 0x7f0d1c1e5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.070+0000 7f0d21c19640 1 -- 172.21.15.47:0/4277672294 >> 172.21.15.47:0/4277672294 conn(0x7f0d1c0cf430 msgr2=0x7f0d1c0b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.070+0000 7f0d21c19640 1 -- 172.21.15.47:0/4277672294 shutdown_connections 2024-03-23T09:48:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:54.070+0000 7f0d21c19640 1 -- 172.21.15.47:0/4277672294 wait complete. 2024-03-23T09:48:54.080 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:48:54.080 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:48:54.081 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:48:54.081 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:48:54.081 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:48:54.081 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:48:54.081 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:48:54.081 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:48:54.081 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:48:54.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:54.081 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:48:54.096 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:48:54.096 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:48:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:48:54.098 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42204sec preferred_lft 42204sec 2024-03-23T09:48:54.098 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:48:54.098 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:48:54.098 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:48:54.098 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:48:54.098 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:48:54.098 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:48:54.098 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:48:54.098 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:48:54.098 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:48:54.098 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:48:54.098 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:48:54.098 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:48:54.099 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:48:54.099 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:48:54.099 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:48:54.099 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:48:54.099 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:48:54.099 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:48:54.099 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:48:54.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:54.099 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:48:54.161 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:48:54.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:54.163 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:48:54.218 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:48:54.218 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:48:54.218 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:48:54.218 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:48:54.218 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:48:54.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:54.219 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:48:54.219 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:48:54.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:48:54.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:48:54.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:48:54.219 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:48:54.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:54.313 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:48:54.327 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:48:54.327 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:48:54.327 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:48:54.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:54.328 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:48:54.394 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:48:54.394 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:48:54.394 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:48:54.449 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:48:54.449 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:48:54.450 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:48:54.503 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:48:54.571 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:48:54.571 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:48:54.636 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:48:54.638 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:48:54.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:54.638 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:48:54.693 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:48:54.693 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:48:54.736 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:48:54.762 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:48:54.762 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:48:54.821 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:48:54.821+0000 7efd76c37480 -1 init, newargv = 0x5616461f01a0 newargc=15 2024-03-23T09:48:54.821 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[70776]: starting ceph client 2024-03-23T09:48:54.825 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[70776]: starting fuse 2024-03-23T09:48:54.830 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:48:54.831 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:48:54.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:54.831 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:48:54.886 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:48:54.886 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:48:54.887 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:48:54.888 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:48:54.888 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:48:54.888 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:48:54.888 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:48:54.888 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:48:54.888 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:48:54.888 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:48:54.888 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:48:54.888 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:48:54.888 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:48:54.888 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:48:54.888 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:48:54.888 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:48:55.003 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.70776.asok 2024-03-23T09:48:55.189 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.70776.asok 2024-03-23T09:48:55.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:55.189 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.70776.asok status 2024-03-23T09:48:55.305 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:48:55.305+0000 7f793c60c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:48:55.326 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:48:55.326 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:48:55.326 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "70776", 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 8226, 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:48:55.327 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 8226 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 379832080 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 379832080 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.8226 192.168.0.1:0/379832080", 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/379832080", 2024-03-23T09:48:55.328 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:48:55.329 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 59, 2024-03-23T09:48:55.329 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 89, 2024-03-23T09:48:55.329 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:48:55.329 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:48:55.329 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:48:55.329 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:48:55.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:55.335 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:48:55.335 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:48:55.350 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:48:55.350 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:48:55.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:55.350 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:48:55.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:55.419 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-03-23T09:48:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.563+0000 7f5177900640 1 Processor -- start 2024-03-23T09:48:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.563+0000 7f5177900640 1 -- start start 2024-03-23T09:48:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.564+0000 7f5177900640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f517015a860 0x7f5170161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.564+0000 7f5177900640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f517015d0f0 0x7f517015d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.564+0000 7f5177900640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5170160bd0 con 0x7f51700cf190 2024-03-23T09:48:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.564+0000 7f5177900640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51700d2790 con 0x7f517015d0f0 2024-03-23T09:48:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.564+0000 7f5177900640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517015dbb0 con 0x7f517015a860 2024-03-23T09:48:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.564+0000 7f5175e9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f51700cf190 0x7f51700cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:49036/0 (socket says 172.21.15.47:49036) 2024-03-23T09:48:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.564+0000 7f5175e9d640 1 -- 172.21.15.47:0/2075565640 learned_addr learned my addr 172.21.15.47:0/2075565640 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.565+0000 7f517569c640 1 --2- 172.21.15.47:0/2075565640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f517015a860 0x7f5170161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.565+0000 7f5174e9b640 1 --2- 172.21.15.47:0/2075565640 >> v2:172.21.15.175:3300/0 conn(0x7f517015d0f0 0x7f517015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.565+0000 7f516ffff640 1 -- 172.21.15.47:0/2075565640 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461531845 0 0) 0x7f5170160bd0 con 0x7f51700cf190 2024-03-23T09:48:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.565+0000 7f516ffff640 1 -- 172.21.15.47:0/2075565640 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5158003520 con 0x7f51700cf190 2024-03-23T09:48:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.565+0000 7f516ffff640 1 -- 172.21.15.47:0/2075565640 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3033116240 0 0) 0x7f5158003520 con 0x7f51700cf190 2024-03-23T09:48:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.565+0000 7f5174e9b640 1 -- 172.21.15.47:0/2075565640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f517015a860 msgr2=0x7f5170161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.565+0000 7f5174e9b640 1 --2- 172.21.15.47:0/2075565640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f517015a860 0x7f5170161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.565+0000 7f5174e9b640 1 -- 172.21.15.47:0/2075565640 >> v1:172.21.15.47:6789/0 conn(0x7f51700cf190 legacy=0x7f51700cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.565+0000 7f5174e9b640 1 -- 172.21.15.47:0/2075565640 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517015de90 con 0x7f517015d0f0 2024-03-23T09:48:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.565+0000 7f517569c640 1 --2- 172.21.15.47:0/2075565640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f517015a860 0x7f5170161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:48:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.566+0000 7f5174e9b640 1 --2- 172.21.15.47:0/2075565640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f517015d0f0 0x7f517015d550 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f51640097a0 tx=0x7f516402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f657945206410dc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.566+0000 7f516ffff640 1 -- 172.21.15.47:0/2075565640 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51600019f0 con 0x7f517015d0f0 2024-03-23T09:48:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.566+0000 7f516ffff640 1 -- 172.21.15.47:0/2075565640 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516402e050 con 0x7f517015d0f0 2024-03-23T09:48:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.566+0000 7f516ffff640 1 -- 172.21.15.47:0/2075565640 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5164030710 con 0x7f517015d0f0 2024-03-23T09:48:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.566+0000 7f5177900640 1 -- 172.21.15.47:0/2075565640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f517015d0f0 msgr2=0x7f517015d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.566+0000 7f5177900640 1 --2- 172.21.15.47:0/2075565640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f517015d0f0 0x7f517015d550 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f51640097a0 tx=0x7f516402b790 comp rx=0 tx=0).stop 2024-03-23T09:48:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.567+0000 7f5177900640 1 -- 172.21.15.47:0/2075565640 shutdown_connections 2024-03-23T09:48:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.567+0000 7f5177900640 1 --2- 172.21.15.47:0/2075565640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f517015d0f0 0x7f517015d550 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.567+0000 7f5177900640 1 --2- 172.21.15.47:0/2075565640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f517015a860 0x7f5170161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.567+0000 7f5177900640 1 -- 172.21.15.47:0/2075565640 >> 172.21.15.47:0/2075565640 conn(0x7f51700659d0 msgr2=0x7f5170065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.567+0000 7f5177900640 1 -- 172.21.15.47:0/2075565640 shutdown_connections 2024-03-23T09:48:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.567+0000 7f5177900640 1 -- 172.21.15.47:0/2075565640 wait complete. 2024-03-23T09:48:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.568+0000 7f5177900640 1 Processor -- start 2024-03-23T09:48:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.568+0000 7f5177900640 1 -- start start 2024-03-23T09:48:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.568+0000 7f5177900640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51700cf190 0x7f51700b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.568+0000 7f5177900640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f517015a860 0x7f51701f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.568+0000 7f5177900640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f517015d0f0 0x7f517015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.568+0000 7f5177900640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517015cc10 con 0x7f517015a860 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.568+0000 7f5177900640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51700b5260 con 0x7f51700cf190 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.568+0000 7f5177900640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51700cc550 con 0x7f517015d0f0 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f5174e9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f517015a860 0x7f51701f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f5174e9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f517015a860 0x7f51701f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37510/0 (socket says 172.21.15.47:37510) 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f5174e9b640 1 -- 172.21.15.47:0/1894777176 learned_addr learned my addr 172.21.15.47:0/1894777176 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f517569c640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51700cf190 0x7f51700b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f5175e9d640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f517015d0f0 0x7f517015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f5174e9b640 1 -- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f517015d0f0 msgr2=0x7f517015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f5174e9b640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f517015d0f0 0x7f517015c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f5174e9b640 1 -- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51700cf190 msgr2=0x7f51700b7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f5174e9b640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51700cf190 0x7f51700b7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f5174e9b640 1 -- 172.21.15.47:0/1894777176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5168009400 con 0x7f517015a860 2024-03-23T09:48:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f5175e9d640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f517015d0f0 0x7f517015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f5174e9b640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f517015a860 0x7f51701f1ff0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f51640098c0 tx=0x7f51640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f516dffb640 1 -- 172.21.15.47:0/1894777176 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516403c620 con 0x7f517015a860 2024-03-23T09:48:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f516dffb640 1 -- 172.21.15.47:0/1894777176 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516403c7a0 con 0x7f517015a860 2024-03-23T09:48:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f5177900640 1 -- 172.21.15.47:0/1894777176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5164009400 con 0x7f517015a860 2024-03-23T09:48:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.569+0000 7f516dffb640 1 -- 172.21.15.47:0/1894777176 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516403cb50 con 0x7f517015a860 2024-03-23T09:48:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.570+0000 7f5177900640 1 -- 172.21.15.47:0/1894777176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51700b4f80 con 0x7f517015a860 2024-03-23T09:48:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.571+0000 7f516dffb640 1 -- 172.21.15.47:0/1894777176 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5164039070 con 0x7f517015a860 2024-03-23T09:48:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.572+0000 7f516dffb640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515c06ac00 0x7f515c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.572+0000 7f516dffb640 1 -- 172.21.15.47:0/1894777176 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f51640aaab0 con 0x7f517015a860 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.572+0000 7f517569c640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515c06ac00 0x7f515c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.572+0000 7f517569c640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515c06ac00 0x7f515c06d0c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5168009cd0 tx=0x7f5168009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.573+0000 7f517a47a740 1 Processor -- start 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.573+0000 7f517a47a740 1 -- start start 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.573+0000 7f517a47a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7add92f0 0x556d7add96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.573+0000 7f517a47a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7addd340 0x556d7addede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.573+0000 7f517a47a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae09080 0x556d7ae09500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.573+0000 7f517a47a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7ad4a940 con 0x556d7addd340 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.573+0000 7f517a47a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7ad480d0 con 0x556d7add92f0 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.573+0000 7f517a47a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d7ad439c0 con 0x556d7ae09080 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.573+0000 7f5174e9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7addd340 0x556d7addede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f5174e9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7addd340 0x556d7addede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37524/0 (socket says 172.21.15.47:37524) 2024-03-23T09:48:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f5174e9b640 1 -- 172.21.15.47:0/533585771 learned_addr learned my addr 172.21.15.47:0/533585771 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f5175e9d640 1 --2- 172.21.15.47:0/533585771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae09080 0x556d7ae09500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f517569c640 1 --2- 172.21.15.47:0/533585771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7add92f0 0x556d7add96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f5174e9b640 1 -- 172.21.15.47:0/533585771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae09080 msgr2=0x556d7ae09500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f5174e9b640 1 --2- 172.21.15.47:0/533585771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae09080 0x556d7ae09500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f5174e9b640 1 -- 172.21.15.47:0/533585771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7add92f0 msgr2=0x556d7add96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f5174e9b640 1 --2- 172.21.15.47:0/533585771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7add92f0 0x556d7add96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f5174e9b640 1 -- 172.21.15.47:0/533585771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7ad0f8a0 con 0x556d7addd340 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f5175e9d640 1 --2- 172.21.15.47:0/533585771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae09080 0x556d7ae09500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f517569c640 1 --2- 172.21.15.47:0/533585771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7add92f0 0x556d7add96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f5174e9b640 1 --2- 172.21.15.47:0/533585771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7addd340 0x556d7addede0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5164006dc0 tx=0x7f5164079900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd5bd46d3a7e415 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f5156ffd640 1 -- 172.21.15.47:0/533585771 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5164077ca0 con 0x556d7addd340 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f5156ffd640 1 -- 172.21.15.47:0/533585771 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5164091170 con 0x556d7addd340 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.574+0000 7f5156ffd640 1 -- 172.21.15.47:0/533585771 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516403ccb0 con 0x556d7addd340 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.575+0000 7f517a47a740 1 -- 172.21.15.47:0/533585771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7addd340 msgr2=0x556d7addede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.575+0000 7f517a47a740 1 --2- 172.21.15.47:0/533585771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7addd340 0x556d7addede0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5164006dc0 tx=0x7f5164079900 comp rx=0 tx=0).stop 2024-03-23T09:48:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.575+0000 7f517a47a740 1 -- 172.21.15.47:0/533585771 shutdown_connections 2024-03-23T09:48:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.575+0000 7f517a47a740 1 --2- 172.21.15.47:0/533585771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae09080 0x556d7ae09500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.575+0000 7f517a47a740 1 --2- 172.21.15.47:0/533585771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7addd340 0x556d7addede0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.575+0000 7f517a47a740 1 --2- 172.21.15.47:0/533585771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7add92f0 0x556d7add96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.575+0000 7f517a47a740 1 -- 172.21.15.47:0/533585771 >> 172.21.15.47:0/533585771 conn(0x556d7addb370 msgr2=0x556d7addcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.575+0000 7f517a47a740 1 -- 172.21.15.47:0/533585771 shutdown_connections 2024-03-23T09:48:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.575+0000 7f517a47a740 1 -- 172.21.15.47:0/533585771 wait complete. 2024-03-23T09:48:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.575+0000 7f517a47a740 1 -- start start 2024-03-23T09:48:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.575+0000 7f517a47a740 1 Processor -- start 2024-03-23T09:48:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.576+0000 7f517a47a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7ae09080 0x556d7ae240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.576+0000 7f517a47a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7adfcf60 0x556d7adddd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.576+0000 7f517a47a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae24640 0x556d7ae051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.576+0000 7f517a47a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7ace4a90 con 0x556d7ae09080 2024-03-23T09:48:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.576+0000 7f517a47a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7ace4730 con 0x556d7ae24640 2024-03-23T09:48:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.576+0000 7f517a47a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d7acc6e00 con 0x556d7adfcf60 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.576+0000 7f517a47a740 10 client.0 _ll_register_callbacks cb 0x556d7acfcb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.576+0000 7f517569c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7ae09080 0x556d7ae240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.576+0000 7f517569c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7ae09080 0x556d7ae240b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37528/0 (socket says 172.21.15.47:37528) 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.576+0000 7f517569c640 1 -- 172.21.15.47:0/270374552 learned_addr learned my addr 172.21.15.47:0/270374552 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.576+0000 7f5174e9b640 1 --2- 172.21.15.47:0/270374552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7adfcf60 0x556d7adddd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.576+0000 7f5175e9d640 1 --2- 172.21.15.47:0/270374552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae24640 0x556d7ae051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517569c640 1 -- 172.21.15.47:0/270374552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7adfcf60 msgr2=0x556d7adddd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517569c640 1 --2- 172.21.15.47:0/270374552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7adfcf60 0x556d7adddd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517569c640 1 -- 172.21.15.47:0/270374552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae24640 msgr2=0x556d7ae051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517569c640 1 --2- 172.21.15.47:0/270374552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae24640 0x556d7ae051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517569c640 1 -- 172.21.15.47:0/270374552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51700b4f80 con 0x556d7ae09080 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517569c640 1 --2- 172.21.15.47:0/270374552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7ae09080 0x556d7ae240b0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5168014e90 tx=0x7f51680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0203ba9e7dbfefe server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f514ffff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f514ffff640 1 -- 172.21.15.47:0/270374552 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168004020 con 0x556d7ae09080 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f514ffff640 1 -- 172.21.15.47:0/270374552 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f5168004180 con 0x556d7ae09080 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517a47a740 1 -- 172.21.15.47:0/270374552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d7aa37560 con 0x556d7ae09080 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f514ffff640 1 -- 172.21.15.47:0/270374552 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5168004340 con 0x556d7ae09080 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517a47a740 10 client.8238 fetch_fsmap learned FSMap version 59 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517a47a740 1 -- 172.21.15.47:0/270374552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x556d7ad42040 con 0x556d7ae09080 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f514ffff640 1 -- 172.21.15.47:0/270374552 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d7aa37560 con 0x556d7ae09080 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f514ffff640 1 -- 172.21.15.47:0/270374552 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 59) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f51680138a0 con 0x556d7ae09080 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517a47a740 10 client.8238 fetch_fsmap finished waiting for FSMap version 59 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517a47a740 10 client.8238 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517a47a740 20 client.8238 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517a47a740 1 --2- 172.21.15.47:0/270374552 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x556d7ae27620 0x556d7ae260a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517a47a740 4 client.8238 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f517a47a740 1 -- 172.21.15.47:0/270374552 --> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d7aa48860 con 0x556d7ae27620 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.577+0000 7f5174e9b640 1 --2- 172.21.15.47:0/270374552 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x556d7ae27620 0x556d7ae260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f514ffff640 10 client.8238 ms_handle_connect on v2:172.21.15.47:6834/3671387340 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f5174e9b640 1 --2- 172.21.15.47:0/270374552 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x556d7ae27620 0x556d7ae260a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27fe7d78c9c59aa7 server_cookie=6dc03841965b8041 in_seq=0 out_seq=0 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f514ffff640 1 -- 172.21.15.47:0/270374552 <== mds.0 v2:172.21.15.47:6834/3671387340 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556d7aa48860 con 0x556d7ae27620 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f514ffff640 10 client.8238 handle_command_reply: tid=0 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f517a47a740 1 client.8238 shutdown 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f517a47a740 1 -- 172.21.15.47:0/270374552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7ae09080 msgr2=0x556d7ae240b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f517a47a740 1 --2- 172.21.15.47:0/270374552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7ae09080 0x556d7ae240b0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5168014e90 tx=0x7f51680046c0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f517a47a740 1 -- 172.21.15.47:0/270374552 shutdown_connections 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f517a47a740 1 --2- 172.21.15.47:0/270374552 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x556d7ae27620 0x556d7ae260a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f517a47a740 1 --2- 172.21.15.47:0/270374552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae24640 0x556d7ae051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f517a47a740 1 --2- 172.21.15.47:0/270374552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7adfcf60 0x556d7adddd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f517a47a740 1 --2- 172.21.15.47:0/270374552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7ae09080 0x556d7ae240b0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f514ffff640 0 client.8238 ms_handle_reset on v2:172.21.15.47:6834/3671387340 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.578+0000 7f517a47a740 1 -- 172.21.15.47:0/270374552 >> 172.21.15.47:0/270374552 conn(0x556d7add92f0 msgr2=0x556d7ae255b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.579+0000 7f517a47a740 1 -- 172.21.15.47:0/270374552 shutdown_connections 2024-03-23T09:48:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.579+0000 7f517a47a740 1 -- 172.21.15.47:0/270374552 wait complete. 2024-03-23T09:48:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.579+0000 7f517a47a740 20 client.8238 trim_cache size 0 max 16384 2024-03-23T09:48:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.593+0000 7f517a47a740 1 Processor -- start 2024-03-23T09:48:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.593+0000 7f517a47a740 1 -- start start 2024-03-23T09:48:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517a47a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 0x556d7ae034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517a47a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae09080 0x556d7ae08ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517a47a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae24640 0x556d7adfae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517a47a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7ae08930 con 0x556d7adfcf60 2024-03-23T09:48:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517a47a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7ae23f30 con 0x556d7ae09080 2024-03-23T09:48:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517a47a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d7ae24f70 con 0x556d7ae24640 2024-03-23T09:48:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517569c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 0x556d7ae034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517569c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 0x556d7ae034c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37530/0 (socket says 172.21.15.47:37530) 2024-03-23T09:48:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517569c640 1 -- 172.21.15.47:0/1549822730 learned_addr learned my addr 172.21.15.47:0/1549822730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f5174e9b640 1 --2- 172.21.15.47:0/1549822730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae09080 0x556d7ae08ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f5175e9d640 1 --2- 172.21.15.47:0/1549822730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae24640 0x556d7adfae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517569c640 1 -- 172.21.15.47:0/1549822730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae24640 msgr2=0x556d7adfae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517569c640 1 --2- 172.21.15.47:0/1549822730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae24640 0x556d7adfae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517569c640 1 -- 172.21.15.47:0/1549822730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae09080 msgr2=0x556d7ae08ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517569c640 1 --2- 172.21.15.47:0/1549822730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae09080 0x556d7ae08ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517569c640 1 -- 172.21.15.47:0/1549822730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7ae040c0 con 0x556d7adfcf60 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f517569c640 1 --2- 172.21.15.47:0/1549822730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 0x556d7ae034c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f5168014e60 tx=0x7f516801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304b2426e9391d1c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f51577fe640 1 -- 172.21.15.47:0/1549822730 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168038030 con 0x556d7adfcf60 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f51577fe640 1 -- 172.21.15.47:0/1549822730 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5168013770 con 0x556d7adfcf60 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.594+0000 7f51577fe640 1 -- 172.21.15.47:0/1549822730 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168013900 con 0x556d7adfcf60 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.595+0000 7f517a47a740 1 -- 172.21.15.47:0/1549822730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 msgr2=0x556d7ae034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.595+0000 7f517a47a740 1 --2- 172.21.15.47:0/1549822730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 0x556d7ae034c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f5168014e60 tx=0x7f516801c000 comp rx=0 tx=0).stop 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.595+0000 7f517a47a740 1 -- 172.21.15.47:0/1549822730 shutdown_connections 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.595+0000 7f517a47a740 1 --2- 172.21.15.47:0/1549822730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae24640 0x556d7adfae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.595+0000 7f517a47a740 1 --2- 172.21.15.47:0/1549822730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae09080 0x556d7ae08ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.595+0000 7f517a47a740 1 --2- 172.21.15.47:0/1549822730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 0x556d7ae034c0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.595+0000 7f517a47a740 1 -- 172.21.15.47:0/1549822730 >> 172.21.15.47:0/1549822730 conn(0x556d7add92f0 msgr2=0x556d7ae36b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.595+0000 7f517a47a740 1 -- 172.21.15.47:0/1549822730 shutdown_connections 2024-03-23T09:48:55.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.595+0000 7f517a47a740 1 -- 172.21.15.47:0/1549822730 wait complete. 2024-03-23T09:48:55.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.595+0000 7f517a47a740 1 -- start start 2024-03-23T09:48:55.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.596+0000 7f517a47a740 1 Processor -- start 2024-03-23T09:48:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.596+0000 7f517a47a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 0x556d7adfb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.596+0000 7f517a47a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae09080 0x556d7adfc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.596+0000 7f517a47a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae24640 0x556d7adfd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.596+0000 7f517a47a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7adfe1d0 con 0x556d7adfcf60 2024-03-23T09:48:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.596+0000 7f517a47a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7adfc320 con 0x556d7ae24640 2024-03-23T09:48:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.596+0000 7f517a47a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d7adfb710 con 0x556d7ae09080 2024-03-23T09:48:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.596+0000 7f517a47a740 10 client.0 _ll_register_callbacks cb 0x7f5164009110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.596+0000 7f517569c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 0x556d7adfb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.596+0000 7f517569c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 0x556d7adfb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37532/0 (socket says 172.21.15.47:37532) 2024-03-23T09:48:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517569c640 1 -- 172.21.15.47:0/3630928356 learned_addr learned my addr 172.21.15.47:0/3630928356 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517569c640 1 -- 172.21.15.47:0/3630928356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae09080 msgr2=0x556d7adfc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517569c640 1 --2- 172.21.15.47:0/3630928356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae09080 0x556d7adfc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f5175e9d640 1 --2- 172.21.15.47:0/3630928356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae24640 0x556d7adfd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517569c640 1 -- 172.21.15.47:0/3630928356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae24640 msgr2=0x556d7adfd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517569c640 1 --2- 172.21.15.47:0/3630928356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae24640 0x556d7adfd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517569c640 1 -- 172.21.15.47:0/3630928356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d7ad42040 con 0x556d7adfcf60 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517569c640 1 --2- 172.21.15.47:0/3630928356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 0x556d7adfb1d0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f5168039040 tx=0x7f5168019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77642e2b46975dc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f514ffff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f514ffff640 1 -- 172.21.15.47:0/3630928356 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168003610 con 0x556d7adfcf60 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f514ffff640 1 -- 172.21.15.47:0/3630928356 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f5168039070 con 0x556d7adfcf60 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517a47a740 1 -- 172.21.15.47:0/3630928356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5164091170 con 0x556d7adfcf60 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f514ffff640 1 -- 172.21.15.47:0/3630928356 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5168003770 con 0x556d7adfcf60 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517a47a740 10 client.8244 fetch_fsmap learned FSMap version 59 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517a47a740 1 -- 172.21.15.47:0/3630928356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x556d7ace3e20 con 0x556d7adfcf60 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f514ffff640 1 -- 172.21.15.47:0/3630928356 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5164091170 con 0x556d7adfcf60 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f514ffff640 1 -- 172.21.15.47:0/3630928356 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 59) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f51680038d0 con 0x556d7adfcf60 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517a47a740 10 client.8244 fetch_fsmap finished waiting for FSMap version 59 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517a47a740 10 client.8244 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:48:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517a47a740 20 client.8244 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.597+0000 7f517a47a740 1 --2- 172.21.15.47:0/3630928356 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x556d7ae07550 0x556d7adde7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.598+0000 7f517a47a740 4 client.8244 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.598+0000 7f517a47a740 1 -- 172.21.15.47:0/3630928356 --> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x556d7aa9beb0 con 0x556d7ae07550 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.598+0000 7f5174e9b640 1 --2- 172.21.15.47:0/3630928356 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x556d7ae07550 0x556d7adde7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.598+0000 7f514ffff640 10 client.8244 ms_handle_connect on v2:172.21.15.47:6834/3671387340 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.598+0000 7f5174e9b640 1 --2- 172.21.15.47:0/3630928356 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x556d7ae07550 0x556d7adde7e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4318bfef8f605527 server_cookie=f351ed52c223deb7 in_seq=0 out_seq=0 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.598+0000 7f514ffff640 1 -- 172.21.15.47:0/3630928356 <== mds.0 v2:172.21.15.47:6834/3671387340 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556d7aa9beb0 con 0x556d7ae07550 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.598+0000 7f514ffff640 10 client.8244 handle_command_reply: tid=0 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.598+0000 7f517a47a740 1 client.8244 shutdown 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.598+0000 7f517a47a740 1 -- 172.21.15.47:0/3630928356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 msgr2=0x556d7adfb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.598+0000 7f517a47a740 1 --2- 172.21.15.47:0/3630928356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 0x556d7adfb1d0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f5168039040 tx=0x7f5168019000 comp rx=0 tx=0).stop 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f517a47a740 1 -- 172.21.15.47:0/3630928356 shutdown_connections 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f517a47a740 1 --2- 172.21.15.47:0/3630928356 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x556d7ae07550 0x556d7adde7e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f517a47a740 1 --2- 172.21.15.47:0/3630928356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7ae24640 0x556d7adfd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f517a47a740 1 --2- 172.21.15.47:0/3630928356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556d7ae09080 0x556d7adfc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f514ffff640 0 client.8244 ms_handle_reset on v2:172.21.15.47:6834/3671387340 2024-03-23T09:48:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f517a47a740 1 --2- 172.21.15.47:0/3630928356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d7adfcf60 0x556d7adfb1d0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f517a47a740 1 -- 172.21.15.47:0/3630928356 >> 172.21.15.47:0/3630928356 conn(0x556d7add92f0 msgr2=0x556d7ae268a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f517a47a740 1 -- 172.21.15.47:0/3630928356 shutdown_connections 2024-03-23T09:48:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f517a47a740 1 -- 172.21.15.47:0/3630928356 wait complete. 2024-03-23T09:48:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f517a47a740 20 client.8244 trim_cache size 0 max 16384 2024-03-23T09:48:55.605 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:48:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-23T09:48:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "594b351b-5ba8-4d0d-8c88-d2b9f240e490", 2024-03-23T09:48:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-23T09:48:55.605 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:48:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f5177900640 1 -- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515c06ac00 msgr2=0x7f515c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f5177900640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515c06ac00 0x7f515c06d0c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5168009cd0 tx=0x7f5168009190 comp rx=0 tx=0).stop 2024-03-23T09:48:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f5177900640 1 -- 172.21.15.47:0/1894777176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f517015a860 msgr2=0x7f51701f1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f5177900640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f517015a860 0x7f51701f1ff0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f51640098c0 tx=0x7f51640046c0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f5177900640 1 -- 172.21.15.47:0/1894777176 shutdown_connections 2024-03-23T09:48:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f5177900640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515c06ac00 0x7f515c06d0c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f5177900640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f517015d0f0 0x7f517015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f5177900640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f517015a860 0x7f51701f1ff0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f5177900640 1 --2- 172.21.15.47:0/1894777176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51700cf190 0x7f51700b7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f5177900640 1 -- 172.21.15.47:0/1894777176 >> 172.21.15.47:0/1894777176 conn(0x7f51700659d0 msgr2=0x7f5170159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f5177900640 1 -- 172.21.15.47:0/1894777176 shutdown_connections 2024-03-23T09:48:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.599+0000 7f5177900640 1 -- 172.21.15.47:0/1894777176 wait complete. 2024-03-23T09:48:55.607 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:48:55.600+0000 7f7938e05640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5628a5710680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T09:48:55.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:48:55.610 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-23T09:48:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.712+0000 7fd06cb55640 1 Processor -- start 2024-03-23T09:48:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.712+0000 7fd06cb55640 1 -- start start 2024-03-23T09:48:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.713+0000 7fd06cb55640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd06815a9b0 0x7fd0680cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.713+0000 7fd06cb55640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0680cd180 con 0x7fd06815bc70 2024-03-23T09:48:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.713+0000 7fd06cb55640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06815b7e0 con 0x7fd06806a3f0 2024-03-23T09:48:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.713+0000 7fd06cb55640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680ccea0 con 0x7fd06815a9b0 2024-03-23T09:48:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.713+0000 7fd065d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd06815bc70 0x7fd068069ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:49044/0 (socket says 172.21.15.47:49044) 2024-03-23T09:48:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.713+0000 7fd065d9b640 1 -- 172.21.15.47:0/3349062859 learned_addr learned my addr 172.21.15.47:0/3349062859 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:48:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.714+0000 7fd06659c640 1 --2- 172.21.15.47:0/3349062859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd06815a9b0 0x7fd0680cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.714+0000 7fd06559a640 1 -- 172.21.15.47:0/3349062859 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573157531 0 0) 0x7fd050001110 con 0x7fd06815bc70 2024-03-23T09:48:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.714+0000 7fd06559a640 1 -- 172.21.15.47:0/3349062859 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd054003520 con 0x7fd06815bc70 2024-03-23T09:48:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.714+0000 7fd06559a640 1 -- 172.21.15.47:0/3349062859 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177684662 0 0) 0x7fd06815b7e0 con 0x7fd06806a3f0 2024-03-23T09:48:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.714+0000 7fd06559a640 1 -- 172.21.15.47:0/3349062859 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd050001110 con 0x7fd06806a3f0 2024-03-23T09:48:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.714+0000 7fd06559a640 1 -- 172.21.15.47:0/3349062859 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2710346428 0 0) 0x7fd054003520 con 0x7fd06815bc70 2024-03-23T09:48:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.714+0000 7fd06659c640 1 -- 172.21.15.47:0/3349062859 >> v1:172.21.15.175:6789/0 conn(0x7fd06806a3f0 legacy=0x7fd0680cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.714+0000 7fd06659c640 1 -- 172.21.15.47:0/3349062859 >> v1:172.21.15.47:6789/0 conn(0x7fd06815bc70 legacy=0x7fd068069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.714+0000 7fd06659c640 1 -- 172.21.15.47:0/3349062859 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06815b190 con 0x7fd06815a9b0 2024-03-23T09:48:55.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.715+0000 7fd06659c640 1 --2- 172.21.15.47:0/3349062859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd06815a9b0 0x7fd0680cb970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd058009dc0 tx=0x7fd05802bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1690bd4016aecd4c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.715+0000 7fd06559a640 1 -- 172.21.15.47:0/3349062859 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050001800 con 0x7fd06815a9b0 2024-03-23T09:48:55.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.715+0000 7fd06559a640 1 -- 172.21.15.47:0/3349062859 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05802c910 con 0x7fd06815a9b0 2024-03-23T09:48:55.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.715+0000 7fd06559a640 1 -- 172.21.15.47:0/3349062859 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05802cca0 con 0x7fd06815a9b0 2024-03-23T09:48:55.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.715+0000 7fd06cb55640 1 -- 172.21.15.47:0/3349062859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd06815a9b0 msgr2=0x7fd0680cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.715+0000 7fd06cb55640 1 --2- 172.21.15.47:0/3349062859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd06815a9b0 0x7fd0680cb970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd058009dc0 tx=0x7fd05802bde0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.716+0000 7fd06cb55640 1 -- 172.21.15.47:0/3349062859 shutdown_connections 2024-03-23T09:48:55.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.716+0000 7fd06cb55640 1 --2- 172.21.15.47:0/3349062859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd06815a9b0 0x7fd0680cb970 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.716+0000 7fd06cb55640 1 -- 172.21.15.47:0/3349062859 >> 172.21.15.47:0/3349062859 conn(0x7fd068071400 msgr2=0x7fd068071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:55.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.716+0000 7fd06cb55640 1 -- 172.21.15.47:0/3349062859 shutdown_connections 2024-03-23T09:48:55.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.716+0000 7fd06cb55640 1 -- 172.21.15.47:0/3349062859 wait complete. 2024-03-23T09:48:55.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.717+0000 7fd06cb55640 1 Processor -- start 2024-03-23T09:48:55.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.717+0000 7fd06cb55640 1 -- start start 2024-03-23T09:48:55.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd06cb55640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd06806a3f0 0x7fd06814c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd06cb55640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd06815a9b0 0x7fd068159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd06cb55640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd06815bc70 0x7fd0680ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd06cb55640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06814c370 con 0x7fd06815a9b0 2024-03-23T09:48:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd06cb55640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd068157b60 con 0x7fd06815bc70 2024-03-23T09:48:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd06cb55640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06814cd40 con 0x7fd06806a3f0 2024-03-23T09:48:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd065d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd06815a9b0 0x7fd068159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd065d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd06815a9b0 0x7fd068159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37534/0 (socket says 172.21.15.47:37534) 2024-03-23T09:48:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd065d9b640 1 -- 172.21.15.47:0/1200121829 learned_addr learned my addr 172.21.15.47:0/1200121829 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd06659c640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd06806a3f0 0x7fd06814c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd066d9d640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd06815bc70 0x7fd0680ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd065d9b640 1 -- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd06806a3f0 msgr2=0x7fd06814c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd065d9b640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd06806a3f0 0x7fd06814c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd065d9b640 1 -- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd06815bc70 msgr2=0x7fd0680ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd065d9b640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd06815bc70 0x7fd0680ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd065d9b640 1 -- 172.21.15.47:0/1200121829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd058009a70 con 0x7fd06815a9b0 2024-03-23T09:48:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd065d9b640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd06815a9b0 0x7fd068159b90 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fd05000c740 tx=0x7fd05000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd05effd640 1 -- 172.21.15.47:0/1200121829 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050005ce0 con 0x7fd06815a9b0 2024-03-23T09:48:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd05effd640 1 -- 172.21.15.47:0/1200121829 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd050005e40 con 0x7fd06815a9b0 2024-03-23T09:48:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.718+0000 7fd06cb55640 1 -- 172.21.15.47:0/1200121829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd068157880 con 0x7fd06815a9b0 2024-03-23T09:48:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.719+0000 7fd05effd640 1 -- 172.21.15.47:0/1200121829 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500072b0 con 0x7fd06815a9b0 2024-03-23T09:48:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.719+0000 7fd06cb55640 1 -- 172.21.15.47:0/1200121829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06814c000 con 0x7fd06815a9b0 2024-03-23T09:48:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.720+0000 7fd05effd640 1 -- 172.21.15.47:0/1200121829 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd050007410 con 0x7fd06815a9b0 2024-03-23T09:48:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.720+0000 7fd05effd640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd04806ac00 0x7fd04806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.721+0000 7fd05effd640 1 -- 172.21.15.47:0/1200121829 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd050089070 con 0x7fd06815a9b0 2024-03-23T09:48:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.721+0000 7fd06659c640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd04806ac00 0x7fd04806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.721+0000 7fd06659c640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd04806ac00 0x7fd04806d0c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd058036000 tx=0x7fd058030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.722+0000 7fd06f7a1740 1 Processor -- start 2024-03-23T09:48:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.722+0000 7fd06f7a1740 1 -- start start 2024-03-23T09:48:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.722+0000 7fd06f7a1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63ce0830 0x559a63ce0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.722+0000 7fd06f7a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb0850 0x559a63cddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.722+0000 7fd06f7a1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cb43b0 0x559a63cddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.722+0000 7fd06f7a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63c20dd0 con 0x559a63cb0850 2024-03-23T09:48:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.722+0000 7fd06f7a1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63c1e560 con 0x559a63ce0830 2024-03-23T09:48:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.722+0000 7fd06f7a1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a63c19e50 con 0x559a63cb43b0 2024-03-23T09:48:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd065d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb0850 0x559a63cddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd065d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb0850 0x559a63cddaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37542/0 (socket says 172.21.15.47:37542) 2024-03-23T09:48:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd065d9b640 1 -- 172.21.15.47:0/3514703531 learned_addr learned my addr 172.21.15.47:0/3514703531 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd06659c640 1 --2- 172.21.15.47:0/3514703531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63ce0830 0x559a63ce0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd066d9d640 1 --2- 172.21.15.47:0/3514703531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cb43b0 0x559a63cddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd065d9b640 1 -- 172.21.15.47:0/3514703531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cb43b0 msgr2=0x559a63cddfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd065d9b640 1 --2- 172.21.15.47:0/3514703531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cb43b0 0x559a63cddfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd065d9b640 1 -- 172.21.15.47:0/3514703531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63ce0830 msgr2=0x559a63ce0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd065d9b640 1 --2- 172.21.15.47:0/3514703531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63ce0830 0x559a63ce0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd065d9b640 1 -- 172.21.15.47:0/3514703531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a63be1470 con 0x559a63cb0850 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd065d9b640 1 --2- 172.21.15.47:0/3514703531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb0850 0x559a63cddaa0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fd050007040 tx=0x7fd05005bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797784f9c59a231b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd047fff640 1 -- 172.21.15.47:0/3514703531 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500561d0 con 0x559a63cb0850 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd047fff640 1 -- 172.21.15.47:0/3514703531 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd050054540 con 0x559a63cb0850 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.723+0000 7fd047fff640 1 -- 172.21.15.47:0/3514703531 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050057520 con 0x559a63cb0850 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.724+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3514703531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb0850 msgr2=0x559a63cddaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.724+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3514703531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb0850 0x559a63cddaa0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fd050007040 tx=0x7fd05005bbb0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.724+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3514703531 shutdown_connections 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.724+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3514703531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cb43b0 0x559a63cddfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.724+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3514703531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb0850 0x559a63cddaa0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.724+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3514703531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63ce0830 0x559a63ce0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.724+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3514703531 >> 172.21.15.47:0/3514703531 conn(0x559a63cb2ac0 msgr2=0x559a63cfb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.724+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3514703531 shutdown_connections 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.724+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3514703531 wait complete. 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.725+0000 7fd06f7a1740 1 -- start start 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.725+0000 7fd06f7a1740 1 Processor -- start 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.725+0000 7fd06f7a1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63cb43b0 0x559a63ccdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.725+0000 7fd06f7a1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63ce0830 0x559a63cda270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.725+0000 7fd06f7a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cd9300 0x559a63ce3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.725+0000 7fd06f7a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63bd8280 con 0x559a63cd9300 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.725+0000 7fd06f7a1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63bd7f20 con 0x559a63cb43b0 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.725+0000 7fd06f7a1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a63b9d3c0 con 0x559a63ce0830 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd06f7a1740 10 client.0 _ll_register_callbacks cb 0x559a63bef380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd066d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cd9300 0x559a63ce3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd066d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cd9300 0x559a63ce3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37550/0 (socket says 172.21.15.47:37550) 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd066d9d640 1 -- 172.21.15.47:0/4115509345 learned_addr learned my addr 172.21.15.47:0/4115509345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd06659c640 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63cb43b0 0x559a63ccdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd065d9b640 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63ce0830 0x559a63cda270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd066d9d640 1 -- 172.21.15.47:0/4115509345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63ce0830 msgr2=0x559a63cda270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd066d9d640 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63ce0830 0x559a63cda270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd066d9d640 1 -- 172.21.15.47:0/4115509345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63cb43b0 msgr2=0x559a63ccdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd066d9d640 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63cb43b0 0x559a63ccdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd066d9d640 1 -- 172.21.15.47:0/4115509345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06814c000 con 0x559a63cd9300 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd065d9b640 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63ce0830 0x559a63cda270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd066d9d640 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cd9300 0x559a63ce3580 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fd060001e90 tx=0x7fd060012750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99d6763336a98d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd0457fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd0457fa640 1 -- 172.21.15.47:0/4115509345 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06001a030 con 0x559a63cd9300 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd0457fa640 1 -- 172.21.15.47:0/4115509345 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd06001b070 con 0x559a63cd9300 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd06f7a1740 1 -- 172.21.15.47:0/4115509345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a63c43690 con 0x559a63cd9300 2024-03-23T09:48:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd0457fa640 1 -- 172.21.15.47:0/4115509345 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd060012ec0 con 0x559a63cd9300 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd06f7a1740 10 client.8256 fetch_fsmap learned FSMap version 59 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.726+0000 7fd06f7a1740 1 -- 172.21.15.47:0/4115509345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x559a63c184d0 con 0x559a63cd9300 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.727+0000 7fd0457fa640 1 -- 172.21.15.47:0/4115509345 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd060025020 con 0x559a63cd9300 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.727+0000 7fd0457fa640 1 -- 172.21.15.47:0/4115509345 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 59) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd060005320 con 0x559a63cd9300 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.727+0000 7fd06f7a1740 10 client.8256 fetch_fsmap finished waiting for FSMap version 59 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.727+0000 7fd06f7a1740 10 client.8256 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.727+0000 7fd06f7a1740 20 client.8256 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.727+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x559a63d0eb80 0x559a63d0ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.727+0000 7fd06f7a1740 4 client.8256 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.727+0000 7fd06f7a1740 1 -- 172.21.15.47:0/4115509345 --> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a63c0beb0 con 0x559a63d0eb80 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.727+0000 7fd06659c640 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x559a63d0eb80 0x559a63d0ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.727+0000 7fd06659c640 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x559a63d0eb80 0x559a63d0ef60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17aaf73d53f04a35 server_cookie=de07bfb5255abf34 in_seq=0 out_seq=0 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.727+0000 7fd0457fa640 10 client.8256 ms_handle_connect on v2:172.21.15.47:6834/3671387340 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd0457fa640 1 -- 172.21.15.47:0/4115509345 <== mds.0 v2:172.21.15.47:6834/3671387340 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559a63c0beb0 con 0x559a63d0eb80 2024-03-23T09:48:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd0457fa640 10 client.8256 handle_command_reply: tid=0 2024-03-23T09:48:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd06f7a1740 1 client.8256 shutdown 2024-03-23T09:48:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd06f7a1740 1 -- 172.21.15.47:0/4115509345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cd9300 msgr2=0x559a63ce3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cd9300 0x559a63ce3580 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fd060001e90 tx=0x7fd060012750 comp rx=0 tx=0).stop 2024-03-23T09:48:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd06f7a1740 1 -- 172.21.15.47:0/4115509345 shutdown_connections 2024-03-23T09:48:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x559a63d0eb80 0x559a63d0ef60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cd9300 0x559a63ce3580 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63ce0830 0x559a63cda270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd0457fa640 0 client.8256 ms_handle_reset on v2:172.21.15.47:6834/3671387340 2024-03-23T09:48:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/4115509345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63cb43b0 0x559a63ccdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd06f7a1740 1 -- 172.21.15.47:0/4115509345 >> 172.21.15.47:0/4115509345 conn(0x559a63cb0850 msgr2=0x559a63ce43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd06f7a1740 1 -- 172.21.15.47:0/4115509345 shutdown_connections 2024-03-23T09:48:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd06f7a1740 1 -- 172.21.15.47:0/4115509345 wait complete. 2024-03-23T09:48:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.728+0000 7fd06f7a1740 20 client.8256 trim_cache size 0 max 16384 2024-03-23T09:48:55.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.742+0000 7fd06f7a1740 1 Processor -- start 2024-03-23T09:48:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.742+0000 7fd06f7a1740 1 -- start start 2024-03-23T09:48:55.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06f7a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb43b0 0x559a63d1db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06f7a1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cd9300 0x559a63cfc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06f7a1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63ce0830 0x559a63cafdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06f7a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63cdb930 con 0x559a63cb43b0 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06f7a1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63cde3b0 con 0x559a63ce0830 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06f7a1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a63cfe1f0 con 0x559a63cd9300 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb43b0 0x559a63d1db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb43b0 0x559a63d1db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37562/0 (socket says 172.21.15.47:37562) 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06659c640 1 -- 172.21.15.47:0/3394210837 learned_addr learned my addr 172.21.15.47:0/3394210837 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06659c640 1 -- 172.21.15.47:0/3394210837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cd9300 msgr2=0x559a63cfc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06659c640 1 --2- 172.21.15.47:0/3394210837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cd9300 0x559a63cfc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06659c640 1 -- 172.21.15.47:0/3394210837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63ce0830 msgr2=0x559a63cafdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06659c640 1 --2- 172.21.15.47:0/3394210837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63ce0830 0x559a63cafdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.743+0000 7fd06659c640 1 -- 172.21.15.47:0/3394210837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a63c4c9e0 con 0x559a63cb43b0 2024-03-23T09:48:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.744+0000 7fd06659c640 1 --2- 172.21.15.47:0/3394210837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb43b0 0x559a63d1db00 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fd05802c3f0 tx=0x7fd05802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbe9ef9fa759a0e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.744+0000 7fd05cff9640 1 -- 172.21.15.47:0/3394210837 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058050040 con 0x559a63cb43b0 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.744+0000 7fd05cff9640 1 -- 172.21.15.47:0/3394210837 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd058043070 con 0x559a63cb43b0 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.744+0000 7fd05cff9640 1 -- 172.21.15.47:0/3394210837 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0580569f0 con 0x559a63cb43b0 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.744+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3394210837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb43b0 msgr2=0x559a63d1db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.744+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3394210837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb43b0 0x559a63d1db00 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fd05802c3f0 tx=0x7fd05802e000 comp rx=0 tx=0).stop 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.744+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3394210837 shutdown_connections 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.744+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3394210837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63ce0830 0x559a63cafdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.744+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3394210837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cd9300 0x559a63cfc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.744+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3394210837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63cb43b0 0x559a63d1db00 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.744+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3394210837 >> 172.21.15.47:0/3394210837 conn(0x559a63cb0850 msgr2=0x559a63ce43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.744+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3394210837 shutdown_connections 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.744+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3394210837 wait complete. 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.745+0000 7fd06f7a1740 1 -- start start 2024-03-23T09:48:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.745+0000 7fd06f7a1740 1 Processor -- start 2024-03-23T09:48:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd06f7a1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cb43b0 0x559a63cd8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd06f7a1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63cd9300 0x559a63d60460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd06f7a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63ce0830 0x559a63ccc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd06f7a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63ccce50 con 0x559a63ce0830 2024-03-23T09:48:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd06f7a1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63cff2e0 con 0x559a63cd9300 2024-03-23T09:48:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd06f7a1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a63cff530 con 0x559a63cb43b0 2024-03-23T09:48:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd06f7a1740 10 client.0 _ll_register_callbacks cb 0x7fd0500541f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:48:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd066d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63ce0830 0x559a63ccc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd066d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63ce0830 0x559a63ccc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37568/0 (socket says 172.21.15.47:37568) 2024-03-23T09:48:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd066d9d640 1 -- 172.21.15.47:0/3769217297 learned_addr learned my addr 172.21.15.47:0/3769217297 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:48:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd065d9b640 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63cd9300 0x559a63d60460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd06659c640 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cb43b0 0x559a63cd8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd066d9d640 1 -- 172.21.15.47:0/3769217297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cb43b0 msgr2=0x559a63cd8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd066d9d640 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cb43b0 0x559a63cd8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd066d9d640 1 -- 172.21.15.47:0/3769217297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63cd9300 msgr2=0x559a63d60460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:48:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd066d9d640 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63cd9300 0x559a63d60460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd066d9d640 1 -- 172.21.15.47:0/3769217297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06815b190 con 0x559a63ce0830 2024-03-23T09:48:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd065d9b640 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63cd9300 0x559a63d60460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:48:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd066d9d640 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63ce0830 0x559a63ccc130 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fd060001b00 tx=0x7fd060000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4184b167d760485 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:48:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd0457fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:48:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.746+0000 7fd0457fa640 1 -- 172.21.15.47:0/3769217297 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06001a040 con 0x559a63ce0830 2024-03-23T09:48:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd0457fa640 1 -- 172.21.15.47:0/3769217297 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd060026080 con 0x559a63ce0830 2024-03-23T09:48:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3769217297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a63cb21f0 con 0x559a63ce0830 2024-03-23T09:48:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd0457fa640 1 -- 172.21.15.47:0/3769217297 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd060012c50 con 0x559a63ce0830 2024-03-23T09:48:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd06f7a1740 10 client.8262 fetch_fsmap learned FSMap version 59 2024-03-23T09:48:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3769217297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x559a63bba5c0 con 0x559a63ce0830 2024-03-23T09:48:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd0457fa640 1 -- 172.21.15.47:0/3769217297 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559a63cb21f0 con 0x559a63ce0830 2024-03-23T09:48:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd0457fa640 1 -- 172.21.15.47:0/3769217297 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 59) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd0600069d0 con 0x559a63ce0830 2024-03-23T09:48:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd06f7a1740 10 client.8262 fetch_fsmap finished waiting for FSMap version 59 2024-03-23T09:48:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd06f7a1740 10 client.8262 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:48:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd06f7a1740 20 client.8262 populate_metadata read hostname 'smithi047' 2024-03-23T09:48:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x559a63ccc670 0x559a63cb55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:48:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd06f7a1740 4 client.8262 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:48:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3769217297 --> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fd050054540 con 0x559a63ccc670 2024-03-23T09:48:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd06659c640 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x559a63ccc670 0x559a63cb55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:48:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd0457fa640 10 client.8262 ms_handle_connect on v2:172.21.15.47:6834/3671387340 2024-03-23T09:48:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.747+0000 7fd06659c640 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x559a63ccc670 0x559a63cb55f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6910eca307fd8422 server_cookie=4f78bdc842058bed in_seq=0 out_seq=0 2024-03-23T09:48:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd0457fa640 1 -- 172.21.15.47:0/3769217297 <== mds.0 v2:172.21.15.47:6834/3671387340 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x7fd050054540 con 0x559a63ccc670 2024-03-23T09:48:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd0457fa640 10 client.8262 handle_command_reply: tid=0 2024-03-23T09:48:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06f7a1740 1 client.8262 shutdown 2024-03-23T09:48:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3769217297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63ce0830 msgr2=0x559a63ccc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63ce0830 0x559a63ccc130 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fd060001b00 tx=0x7fd060000ed0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3769217297 shutdown_connections 2024-03-23T09:48:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x559a63ccc670 0x559a63cb55f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a63ce0830 0x559a63ccc130 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a63cd9300 0x559a63d60460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06f7a1740 1 --2- 172.21.15.47:0/3769217297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559a63cb43b0 0x559a63cd8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd0457fa640 0 client.8262 ms_handle_reset on v2:172.21.15.47:6834/3671387340 2024-03-23T09:48:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3769217297 >> 172.21.15.47:0/3769217297 conn(0x559a63cb0850 msgr2=0x559a63cdd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3769217297 shutdown_connections 2024-03-23T09:48:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06f7a1740 1 -- 172.21.15.47:0/3769217297 wait complete. 2024-03-23T09:48:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06f7a1740 20 client.8262 trim_cache size 0 max 16384 2024-03-23T09:48:55.756 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:48:55.756 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T09:48:55.756 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-23T09:48:55.756 INFO:teuthology.orchestra.run.smithi047.stdout: "594b351b-5ba8-4d0d-8c88-d2b9f240e490": { 2024-03-23T09:48:55.756 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/dir", 2024-03-23T09:48:55.756 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "594b351b-5ba8-4d0d-8c88-d2b9f240e490", 2024-03-23T09:48:55.756 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,repair" 2024-03-23T09:48:55.756 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:48:55.756 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:48:55.756 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:48:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06cb55640 1 -- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd04806ac00 msgr2=0x7fd04806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06cb55640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd04806ac00 0x7fd04806d0c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd058036000 tx=0x7fd058030000 comp rx=0 tx=0).stop 2024-03-23T09:48:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06cb55640 1 -- 172.21.15.47:0/1200121829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd06815a9b0 msgr2=0x7fd068159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:48:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06cb55640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd06815a9b0 0x7fd068159b90 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fd05000c740 tx=0x7fd05000cc00 comp rx=0 tx=0).stop 2024-03-23T09:48:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06cb55640 1 -- 172.21.15.47:0/1200121829 shutdown_connections 2024-03-23T09:48:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06cb55640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd04806ac00 0x7fd04806d0c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06cb55640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd06815bc70 0x7fd0680ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06cb55640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd06815a9b0 0x7fd068159b90 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06cb55640 1 --2- 172.21.15.47:0/1200121829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd06806a3f0 0x7fd06814c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:48:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.748+0000 7fd06cb55640 1 -- 172.21.15.47:0/1200121829 >> 172.21.15.47:0/1200121829 conn(0x7fd068071400 msgr2=0x7fd0680cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:48:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.749+0000 7fd06cb55640 1 -- 172.21.15.47:0/1200121829 shutdown_connections 2024-03-23T09:48:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:48:55.749+0000 7fd06cb55640 1 -- 172.21.15.47:0/1200121829 wait complete. 2024-03-23T09:48:55.759 INFO:tasks.cephfs.filesystem:scrub status for tag:594b351b-5ba8-4d0d-8c88-d2b9f240e490 - {'path': '/dir', 'tag': '594b351b-5ba8-4d0d-8c88-d2b9f240e490', 'options': 'recursive,repair'} 2024-03-23T09:49:08.390 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:49:08.394 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:49:25.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:25.760 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-23T09:49:25.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.878+0000 7fd97b389640 1 Processor -- start 2024-03-23T09:49:25.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.878+0000 7fd97b389640 1 -- start start 2024-03-23T09:49:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd97b389640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd97415ca30 0x7fd974160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd97b389640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd97415ce10 0x7fd9740cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd97b389640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9740b85d0 0x7fd9740cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd97b389640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd974161600 con 0x7fd97415ca30 2024-03-23T09:49:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd97b389640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd974161040 con 0x7fd97415ce10 2024-03-23T09:49:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd97b389640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd974161320 con 0x7fd9740b85d0 2024-03-23T09:49:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd979125640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd97415ca30 0x7fd974160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd979125640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd97415ca30 0x7fd974160740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37244/0 (socket says 172.21.15.47:37244) 2024-03-23T09:49:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd979125640 1 -- 172.21.15.47:0/2158950686 learned_addr learned my addr 172.21.15.47:0/2158950686 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd978924640 1 --2- 172.21.15.47:0/2158950686 >> v2:172.21.15.175:3300/0 conn(0x7fd97415ce10 0x7fd9740cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd979926640 1 --2- 172.21.15.47:0/2158950686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9740b85d0 0x7fd9740cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd979125640 1 -- 172.21.15.47:0/2158950686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9740b85d0 msgr2=0x7fd9740cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd979125640 1 --2- 172.21.15.47:0/2158950686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9740b85d0 0x7fd9740cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd979125640 1 -- 172.21.15.47:0/2158950686 >> v2:172.21.15.175:3300/0 conn(0x7fd97415ce10 msgr2=0x7fd9740cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd979125640 1 --2- 172.21.15.47:0/2158950686 >> v2:172.21.15.175:3300/0 conn(0x7fd97415ce10 0x7fd9740cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd979125640 1 -- 172.21.15.47:0/2158950686 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9740d0dd0 con 0x7fd97415ca30 2024-03-23T09:49:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd978924640 1 --2- 172.21.15.47:0/2158950686 >> v2:172.21.15.175:3300/0 conn(0x7fd97415ce10 0x7fd9740cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.879+0000 7fd979926640 1 --2- 172.21.15.47:0/2158950686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9740b85d0 0x7fd9740cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.880+0000 7fd979125640 1 --2- 172.21.15.47:0/2158950686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd97415ca30 0x7fd974160740 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fd974066930 tx=0x7fd96c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2003e749407bda3d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.880+0000 7fd973fff640 1 -- 172.21.15.47:0/2158950686 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c031870 con 0x7fd97415ca30 2024-03-23T09:49:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.880+0000 7fd973fff640 1 -- 172.21.15.47:0/2158950686 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c0319d0 con 0x7fd97415ca30 2024-03-23T09:49:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.880+0000 7fd973fff640 1 -- 172.21.15.47:0/2158950686 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c031d60 con 0x7fd97415ca30 2024-03-23T09:49:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.881+0000 7fd97b389640 1 -- 172.21.15.47:0/2158950686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd97415ca30 msgr2=0x7fd974160740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.881+0000 7fd97b389640 1 --2- 172.21.15.47:0/2158950686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd97415ca30 0x7fd974160740 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fd974066930 tx=0x7fd96c02b910 comp rx=0 tx=0).stop 2024-03-23T09:49:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.881+0000 7fd97b389640 1 -- 172.21.15.47:0/2158950686 shutdown_connections 2024-03-23T09:49:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.881+0000 7fd97b389640 1 --2- 172.21.15.47:0/2158950686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9740b85d0 0x7fd9740cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.881+0000 7fd97b389640 1 --2- 172.21.15.47:0/2158950686 >> v2:172.21.15.175:3300/0 conn(0x7fd97415ce10 0x7fd9740cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.881+0000 7fd97b389640 1 --2- 172.21.15.47:0/2158950686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd97415ca30 0x7fd974160740 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.881+0000 7fd97b389640 1 -- 172.21.15.47:0/2158950686 >> 172.21.15.47:0/2158950686 conn(0x7fd97406b160 msgr2=0x7fd97406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.881+0000 7fd97b389640 1 -- 172.21.15.47:0/2158950686 shutdown_connections 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.881+0000 7fd97b389640 1 -- 172.21.15.47:0/2158950686 wait complete. 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.882+0000 7fd97b389640 1 Processor -- start 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.882+0000 7fd97b389640 1 -- start start 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd97b389640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740b85d0 0x7fd974157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd97b389640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd97415ca30 0x7fd97413f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd97b389640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd97415ce10 0x7fd97413fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd97b389640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9741f6490 con 0x7fd9740b85d0 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd97b389640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9741f5af0 con 0x7fd97415ca30 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd97b389640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9741f5dd0 con 0x7fd97415ce10 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd979125640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740b85d0 0x7fd974157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd979125640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740b85d0 0x7fd974157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37252/0 (socket says 172.21.15.47:37252) 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd979125640 1 -- 172.21.15.47:0/2929464438 learned_addr learned my addr 172.21.15.47:0/2929464438 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd979926640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd97415ce10 0x7fd97413fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd978924640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd97415ca30 0x7fd97413f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd979125640 1 -- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd97415ce10 msgr2=0x7fd97413fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd979125640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd97415ce10 0x7fd97413fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd979125640 1 -- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd97415ca30 msgr2=0x7fd97413f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd979125640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd97415ca30 0x7fd97413f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.883+0000 7fd979125640 1 -- 172.21.15.47:0/2929464438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c0094d0 con 0x7fd9740b85d0 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.884+0000 7fd979125640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740b85d0 0x7fd974157020 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd96c02b950 tx=0x7fd96c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.884+0000 7fd971ffb640 1 -- 172.21.15.47:0/2929464438 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c03ad30 con 0x7fd9740b85d0 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.884+0000 7fd971ffb640 1 -- 172.21.15.47:0/2929464438 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c0443f0 con 0x7fd9740b85d0 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.884+0000 7fd97b389640 1 -- 172.21.15.47:0/2929464438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9741f60b0 con 0x7fd9740b85d0 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.884+0000 7fd971ffb640 1 -- 172.21.15.47:0/2929464438 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0447a0 con 0x7fd9740b85d0 2024-03-23T09:49:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.884+0000 7fd97b389640 1 -- 172.21.15.47:0/2929464438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd974157560 con 0x7fd9740b85d0 2024-03-23T09:49:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.885+0000 7fd971ffb640 1 -- 172.21.15.47:0/2929464438 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd96c04d450 con 0x7fd9740b85d0 2024-03-23T09:49:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.886+0000 7fd971ffb640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd96006ac00 0x7fd96006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.886+0000 7fd971ffb640 1 -- 172.21.15.47:0/2929464438 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd96c02d040 con 0x7fd9740b85d0 2024-03-23T09:49:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.886+0000 7fd978924640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd96006ac00 0x7fd96006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.887+0000 7fd978924640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd96006ac00 0x7fd96006d0c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd9741f59b0 tx=0x7fd968008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.887+0000 7fd97dfd4740 1 Processor -- start 2024-03-23T09:49:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.887+0000 7fd97dfd4740 1 -- start start 2024-03-23T09:49:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd97dfd4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c66830 0x565072c66c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd97dfd4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c36850 0x565072c63aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd97dfd4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c3a3b0 0x565072c63fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd97dfd4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x565072ba6dd0 con 0x565072c36850 2024-03-23T09:49:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd97dfd4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x565072ba4560 con 0x565072c3a3b0 2024-03-23T09:49:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd97dfd4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x565072b9fe50 con 0x565072c66830 2024-03-23T09:49:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd978924640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c36850 0x565072c63aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd978924640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c36850 0x565072c63aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37256/0 (socket says 172.21.15.47:37256) 2024-03-23T09:49:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd978924640 1 -- 172.21.15.47:0/3329956496 learned_addr learned my addr 172.21.15.47:0/3329956496 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd979125640 1 --2- 172.21.15.47:0/3329956496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c66830 0x565072c66c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd979926640 1 --2- 172.21.15.47:0/3329956496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c3a3b0 0x565072c63fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd978924640 1 -- 172.21.15.47:0/3329956496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c66830 msgr2=0x565072c66c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd978924640 1 --2- 172.21.15.47:0/3329956496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c66830 0x565072c66c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd978924640 1 -- 172.21.15.47:0/3329956496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c3a3b0 msgr2=0x565072c63fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd978924640 1 --2- 172.21.15.47:0/3329956496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c3a3b0 0x565072c63fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd978924640 1 -- 172.21.15.47:0/3329956496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565072b67470 con 0x565072c36850 2024-03-23T09:49:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd979926640 1 --2- 172.21.15.47:0/3329956496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c3a3b0 0x565072c63fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd978924640 1 --2- 172.21.15.47:0/3329956496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c36850 0x565072c63aa0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd96800f9f0 tx=0x7fd968012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addc3ca574b4f7de server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd95affd640 1 -- 172.21.15.47:0/3329956496 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968014be0 con 0x565072c36850 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd95affd640 1 -- 172.21.15.47:0/3329956496 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd968019070 con 0x565072c36850 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.888+0000 7fd95affd640 1 -- 172.21.15.47:0/3329956496 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968020440 con 0x565072c36850 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.889+0000 7fd97dfd4740 1 -- 172.21.15.47:0/3329956496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c36850 msgr2=0x565072c63aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.889+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/3329956496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c36850 0x565072c63aa0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd96800f9f0 tx=0x7fd968012000 comp rx=0 tx=0).stop 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.889+0000 7fd97dfd4740 1 -- 172.21.15.47:0/3329956496 shutdown_connections 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.889+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/3329956496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c3a3b0 0x565072c63fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.889+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/3329956496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c36850 0x565072c63aa0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.889+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/3329956496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c66830 0x565072c66c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.889+0000 7fd97dfd4740 1 -- 172.21.15.47:0/3329956496 >> 172.21.15.47:0/3329956496 conn(0x565072c38ac0 msgr2=0x565072c81640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.889+0000 7fd97dfd4740 1 -- 172.21.15.47:0/3329956496 shutdown_connections 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.889+0000 7fd97dfd4740 1 -- 172.21.15.47:0/3329956496 wait complete. 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.890+0000 7fd97dfd4740 1 -- start start 2024-03-23T09:49:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.890+0000 7fd97dfd4740 1 Processor -- start 2024-03-23T09:49:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd97dfd4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c3a3b0 0x565072c52bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd97dfd4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c66830 0x565072c6ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd97dfd4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c53510 0x565072c6af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd97dfd4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x565072b5e280 con 0x565072c66830 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd97dfd4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x565072b5df20 con 0x565072c53510 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd97dfd4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x565072b233c0 con 0x565072c3a3b0 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd97dfd4740 10 client.0 _ll_register_callbacks cb 0x565072b75380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd978924640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c66830 0x565072c6ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd978924640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c66830 0x565072c6ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37260/0 (socket says 172.21.15.47:37260) 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd978924640 1 -- 172.21.15.47:0/3171939364 learned_addr learned my addr 172.21.15.47:0/3171939364 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd979125640 1 --2- 172.21.15.47:0/3171939364 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c3a3b0 0x565072c52bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd979926640 1 --2- 172.21.15.47:0/3171939364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c53510 0x565072c6af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd978924640 1 -- 172.21.15.47:0/3171939364 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c3a3b0 msgr2=0x565072c52bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd978924640 1 --2- 172.21.15.47:0/3171939364 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c3a3b0 0x565072c52bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd978924640 1 -- 172.21.15.47:0/3171939364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c53510 msgr2=0x565072c6af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd978924640 1 --2- 172.21.15.47:0/3171939364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c53510 0x565072c6af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd978924640 1 -- 172.21.15.47:0/3171939364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96800e040 con 0x565072c66830 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd978924640 1 --2- 172.21.15.47:0/3171939364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c66830 0x565072c6ccd0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fd968014830 tx=0x7fd968004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c02cd6948216e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd953fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd953fff640 1 -- 172.21.15.47:0/3171939364 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968004230 con 0x565072c66830 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd97dfd4740 1 -- 172.21.15.47:0/3171939364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565072bc9690 con 0x565072c66830 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.891+0000 7fd953fff640 1 -- 172.21.15.47:0/3171939364 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd968004390 con 0x565072c66830 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd953fff640 1 -- 172.21.15.47:0/3171939364 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd968004550 con 0x565072c66830 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd97dfd4740 10 client.8274 fetch_fsmap learned FSMap version 59 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd97dfd4740 1 -- 172.21.15.47:0/3171939364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x565072b9e4d0 con 0x565072c66830 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd953fff640 1 -- 172.21.15.47:0/3171939364 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565072bc9690 con 0x565072c66830 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd953fff640 1 -- 172.21.15.47:0/3171939364 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 59) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd9680208a0 con 0x565072c66830 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd97dfd4740 10 client.8274 fetch_fsmap finished waiting for FSMap version 59 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd97dfd4740 10 client.8274 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd97dfd4740 20 client.8274 populate_metadata read hostname 'smithi047' 2024-03-23T09:49:25.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/3171939364 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x565072c53990 0x565072c69420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd97dfd4740 4 client.8274 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd97dfd4740 1 -- 172.21.15.47:0/3171939364 --> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565072b91eb0 con 0x565072c53990 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd979125640 1 --2- 172.21.15.47:0/3171939364 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x565072c53990 0x565072c69420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd953fff640 10 client.8274 ms_handle_connect on v2:172.21.15.47:6834/3671387340 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.892+0000 7fd979125640 1 --2- 172.21.15.47:0/3171939364 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x565072c53990 0x565072c69420 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa48694bc6d2ce2e server_cookie=10fcc98ec436c36a in_seq=0 out_seq=0 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd953fff640 1 -- 172.21.15.47:0/3171939364 <== mds.0 v2:172.21.15.47:6834/3671387340 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x565072b91eb0 con 0x565072c53990 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd953fff640 10 client.8274 handle_command_reply: tid=0 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd97dfd4740 1 client.8274 shutdown 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd97dfd4740 1 -- 172.21.15.47:0/3171939364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c66830 msgr2=0x565072c6ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/3171939364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c66830 0x565072c6ccd0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fd968014830 tx=0x7fd968004730 comp rx=0 tx=0).stop 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd97dfd4740 1 -- 172.21.15.47:0/3171939364 shutdown_connections 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/3171939364 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x565072c53990 0x565072c69420 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/3171939364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c53510 0x565072c6af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd953fff640 0 client.8274 ms_handle_reset on v2:172.21.15.47:6834/3671387340 2024-03-23T09:49:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/3171939364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c66830 0x565072c6ccd0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/3171939364 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c3a3b0 0x565072c52bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd97dfd4740 1 -- 172.21.15.47:0/3171939364 >> 172.21.15.47:0/3171939364 conn(0x565072c36850 msgr2=0x565072c6a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:25.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd97dfd4740 1 -- 172.21.15.47:0/3171939364 shutdown_connections 2024-03-23T09:49:25.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd97dfd4740 1 -- 172.21.15.47:0/3171939364 wait complete. 2024-03-23T09:49:25.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.893+0000 7fd97dfd4740 20 client.8274 trim_cache size 0 max 16384 2024-03-23T09:49:25.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.908+0000 7fd97dfd4740 1 Processor -- start 2024-03-23T09:49:25.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.908+0000 7fd97dfd4740 1 -- start start 2024-03-23T09:49:25.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd97dfd4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c3a3b0 0x565072c51f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd97dfd4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 0x565072c69420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd97dfd4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c66830 0x565072c36240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd97dfd4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x565072c6b140 con 0x565072c53510 2024-03-23T09:49:25.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd97dfd4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x565072b9efe0 con 0x565072c3a3b0 2024-03-23T09:49:25.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd97dfd4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x565072c63d70 con 0x565072c66830 2024-03-23T09:49:25.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd978924640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 0x565072c69420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd978924640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 0x565072c69420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37276/0 (socket says 172.21.15.47:37276) 2024-03-23T09:49:25.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd979125640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c3a3b0 0x565072c51f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd978924640 1 -- 172.21.15.47:0/1732311147 learned_addr learned my addr 172.21.15.47:0/1732311147 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:25.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd979926640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c66830 0x565072c36240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd978924640 1 -- 172.21.15.47:0/1732311147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c66830 msgr2=0x565072c36240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:25.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd978924640 1 --2- 172.21.15.47:0/1732311147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c66830 0x565072c36240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd978924640 1 -- 172.21.15.47:0/1732311147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c3a3b0 msgr2=0x565072c51f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd978924640 1 --2- 172.21.15.47:0/1732311147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c3a3b0 0x565072c51f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd978924640 1 -- 172.21.15.47:0/1732311147 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565072bd29e0 con 0x565072c53510 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd978924640 1 --2- 172.21.15.47:0/1732311147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 0x565072c69420 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fd96800f780 tx=0x7fd96802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af37b2b4a68ec698 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.909+0000 7fd95b7fe640 1 -- 172.21.15.47:0/1732311147 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9680208f0 con 0x565072c53510 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.910+0000 7fd95b7fe640 1 -- 172.21.15.47:0/1732311147 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96802f070 con 0x565072c53510 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.910+0000 7fd95b7fe640 1 -- 172.21.15.47:0/1732311147 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968004270 con 0x565072c53510 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.910+0000 7fd97dfd4740 1 -- 172.21.15.47:0/1732311147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 msgr2=0x565072c69420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.910+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/1732311147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 0x565072c69420 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fd96800f780 tx=0x7fd96802d070 comp rx=0 tx=0).stop 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.910+0000 7fd97dfd4740 1 -- 172.21.15.47:0/1732311147 shutdown_connections 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.910+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/1732311147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c66830 0x565072c36240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.910+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/1732311147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 0x565072c69420 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.910+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/1732311147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c3a3b0 0x565072c51f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.910+0000 7fd97dfd4740 1 -- 172.21.15.47:0/1732311147 >> 172.21.15.47:0/1732311147 conn(0x565072c36850 msgr2=0x565072c32030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:25.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.910+0000 7fd97dfd4740 1 -- 172.21.15.47:0/1732311147 shutdown_connections 2024-03-23T09:49:25.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.910+0000 7fd97dfd4740 1 -- 172.21.15.47:0/1732311147 wait complete. 2024-03-23T09:49:25.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.911+0000 7fd97dfd4740 1 -- start start 2024-03-23T09:49:25.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.911+0000 7fd97dfd4740 1 Processor -- start 2024-03-23T09:49:25.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd97dfd4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c3a3b0 0x565072c9f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd97dfd4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 0x565072cea690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd97dfd4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c66830 0x565072c62690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd97dfd4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x565072c52280 con 0x565072c53510 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd97dfd4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x565072ca0220 con 0x565072c66830 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd97dfd4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x565072c52030 con 0x565072c3a3b0 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd97dfd4740 10 client.0 _ll_register_callbacks cb 0x7fd96800b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd978924640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 0x565072cea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd978924640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 0x565072cea690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37292/0 (socket says 172.21.15.47:37292) 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd978924640 1 -- 172.21.15.47:0/1750047458 learned_addr learned my addr 172.21.15.47:0/1750047458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd979926640 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c66830 0x565072c62690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd979125640 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c3a3b0 0x565072c9f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd978924640 1 -- 172.21.15.47:0/1750047458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c3a3b0 msgr2=0x565072c9f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd978924640 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c3a3b0 0x565072c9f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd978924640 1 -- 172.21.15.47:0/1750047458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c66830 msgr2=0x565072c62690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd978924640 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c66830 0x565072c62690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd978924640 1 -- 172.21.15.47:0/1750047458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565072b9e4d0 con 0x565072c53510 2024-03-23T09:49:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd979125640 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c3a3b0 0x565072c9f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd978924640 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 0x565072cea690 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd968006a90 tx=0x7fd968015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96654e50d05e7c3a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd953fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd953fff640 1 -- 172.21.15.47:0/1750047458 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968004440 con 0x565072c53510 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd953fff640 1 -- 172.21.15.47:0/1750047458 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd968003610 con 0x565072c53510 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.912+0000 7fd97dfd4740 1 -- 172.21.15.47:0/1750047458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd968019070 con 0x565072c53510 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd953fff640 1 -- 172.21.15.47:0/1750047458 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd968014cf0 con 0x565072c53510 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd97dfd4740 10 client.8280 fetch_fsmap learned FSMap version 59 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd97dfd4740 1 -- 172.21.15.47:0/1750047458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x565072b405c0 con 0x565072c53510 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd953fff640 1 -- 172.21.15.47:0/1750047458 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd968019070 con 0x565072c53510 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd953fff640 1 -- 172.21.15.47:0/1750047458 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 59) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd9680037d0 con 0x565072c53510 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd97dfd4740 10 client.8280 fetch_fsmap finished waiting for FSMap version 59 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd97dfd4740 10 client.8280 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd97dfd4740 20 client.8280 populate_metadata read hostname 'smithi047' 2024-03-23T09:49:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x565072c35910 0x565072c62bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd97dfd4740 4 client.8280 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd97dfd4740 1 -- 172.21.15.47:0/1750047458 --> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5650728934e0 con 0x565072c35910 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd979125640 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x565072c35910 0x565072c62bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd953fff640 10 client.8280 ms_handle_connect on v2:172.21.15.47:6834/3671387340 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.913+0000 7fd979125640 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x565072c35910 0x565072c62bd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8165ee93908ec169 server_cookie=cff8538cb08c71ed in_seq=0 out_seq=0 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd953fff640 1 -- 172.21.15.47:0/1750047458 <== mds.0 v2:172.21.15.47:6834/3671387340 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5650728934e0 con 0x565072c35910 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd953fff640 10 client.8280 handle_command_reply: tid=0 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97dfd4740 1 client.8280 shutdown 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97dfd4740 1 -- 172.21.15.47:0/1750047458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 msgr2=0x565072cea690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 0x565072cea690 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd968006a90 tx=0x7fd968015000 comp rx=0 tx=0).stop 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97dfd4740 1 -- 172.21.15.47:0/1750047458 shutdown_connections 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x565072c35910 0x565072c62bd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565072c66830 0x565072c62690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565072c53510 0x565072cea690 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd953fff640 0 client.8280 ms_handle_reset on v2:172.21.15.47:6834/3671387340 2024-03-23T09:49:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97dfd4740 1 --2- 172.21.15.47:0/1750047458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x565072c3a3b0 0x565072c9f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97dfd4740 1 -- 172.21.15.47:0/1750047458 >> 172.21.15.47:0/1750047458 conn(0x565072c36850 msgr2=0x565072c3a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97dfd4740 1 -- 172.21.15.47:0/1750047458 shutdown_connections 2024-03-23T09:49:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97dfd4740 1 -- 172.21.15.47:0/1750047458 wait complete. 2024-03-23T09:49:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97dfd4740 20 client.8280 trim_cache size 0 max 16384 2024-03-23T09:49:25.920 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:49:25.920 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-23T09:49:25.920 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-23T09:49:25.920 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:49:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97b389640 1 -- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd96006ac00 msgr2=0x7fd96006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.914+0000 7fd97b389640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd96006ac00 0x7fd96006d0c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd9741f59b0 tx=0x7fd968008040 comp rx=0 tx=0).stop 2024-03-23T09:49:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.915+0000 7fd97b389640 1 -- 172.21.15.47:0/2929464438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740b85d0 msgr2=0x7fd974157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.915+0000 7fd97b389640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740b85d0 0x7fd974157020 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd96c02b950 tx=0x7fd96c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.915+0000 7fd97b389640 1 -- 172.21.15.47:0/2929464438 shutdown_connections 2024-03-23T09:49:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.915+0000 7fd97b389640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd96006ac00 0x7fd96006d0c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.915+0000 7fd97b389640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd97415ce10 0x7fd97413fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.915+0000 7fd97b389640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd97415ca30 0x7fd97413f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.915+0000 7fd97b389640 1 --2- 172.21.15.47:0/2929464438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740b85d0 0x7fd974157020 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.915+0000 7fd97b389640 1 -- 172.21.15.47:0/2929464438 >> 172.21.15.47:0/2929464438 conn(0x7fd97406b160 msgr2=0x7fd9740ca480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.915+0000 7fd97b389640 1 -- 172.21.15.47:0/2929464438 shutdown_connections 2024-03-23T09:49:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:25.915+0000 7fd97b389640 1 -- 172.21.15.47:0/2929464438 wait complete. 2024-03-23T09:49:25.925 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T09:49:25.926 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:49:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.026+0000 7fe5a6a8f640 1 Processor -- start 2024-03-23T09:49:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.027+0000 7fe5a6a8f640 1 -- start start 2024-03-23T09:49:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.027+0000 7fe5a6a8f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe5a00cec50 0x7fe5a00cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.027+0000 7fe5a6a8f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a00cc610 con 0x7fe5a015a8d0 2024-03-23T09:49:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.027+0000 7fe5a6a8f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a00b0700 con 0x7fe5a00b0b90 2024-03-23T09:49:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.027+0000 7fe5a6a8f640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a00cf640 con 0x7fe5a00cec50 2024-03-23T09:49:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.027+0000 7fe5a482b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe5a015a8d0 0x7fe5a00cb1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59190/0 (socket says 172.21.15.47:59190) 2024-03-23T09:49:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5a482b640 1 -- 172.21.15.47:0/3485625655 learned_addr learned my addr 172.21.15.47:0/3485625655 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe597fff640 1 --2- 172.21.15.47:0/3485625655 >> v2:172.21.15.175:3300/0 conn(0x7fe5a00cec50 0x7fe5a00cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214326261 0 0) 0x7fe5a00cc610 con 0x7fe5a015a8d0 2024-03-23T09:49:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe588003520 con 0x7fe5a015a8d0 2024-03-23T09:49:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 193022171 0 0) 0x7fe588003520 con 0x7fe5a015a8d0 2024-03-23T09:49:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 >> v2:172.21.15.175:3300/0 conn(0x7fe5a00cec50 msgr2=0x7fe5a00cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 --2- 172.21.15.47:0/3485625655 >> v2:172.21.15.175:3300/0 conn(0x7fe5a00cec50 0x7fe5a00cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 >> v1:172.21.15.175:6789/0 conn(0x7fe5a00b0b90 legacy=0x7fe5a00b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a00cbd70 con 0x7fe5a015a8d0 2024-03-23T09:49:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fe59c001670 con 0x7fe5a015a8d0 2024-03-23T09:49:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 >> v1:172.21.15.47:6789/0 conn(0x7fe5a015a8d0 legacy=0x7fe5a00cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 --2- 172.21.15.47:0/3485625655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe588003970 0x7fe588003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 --2- 172.21.15.47:0/3485625655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5880047c0 0x7fe588004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 --2- 172.21.15.47:0/3485625655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe588005180 0x7fe58800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00cf640 con 0x7fe588003970 2024-03-23T09:49:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58800c120 con 0x7fe5880047c0 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58800c2e0 con 0x7fe588005180 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5a482b640 1 --2- 172.21.15.47:0/3485625655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe588003970 0x7fe588003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe597fff640 1 --2- 172.21.15.47:0/3485625655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5880047c0 0x7fe588004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5a502c640 1 --2- 172.21.15.47:0/3485625655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe588005180 0x7fe58800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5a482b640 1 -- 172.21.15.47:0/3485625655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe588005180 msgr2=0x7fe58800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5a482b640 1 --2- 172.21.15.47:0/3485625655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe588005180 0x7fe58800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5a482b640 1 -- 172.21.15.47:0/3485625655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5880047c0 msgr2=0x7fe588004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5a482b640 1 --2- 172.21.15.47:0/3485625655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5880047c0 0x7fe588004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5a482b640 1 -- 172.21.15.47:0/3485625655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe588003520 con 0x7fe588003970 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe597fff640 1 --2- 172.21.15.47:0/3485625655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5880047c0 0x7fe588004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5a502c640 1 --2- 172.21.15.47:0/3485625655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe588005180 0x7fe58800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.028+0000 7fe5a502c640 1 -- 172.21.15.47:0/3485625655 reap_dead start 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.029+0000 7fe5a482b640 1 --2- 172.21.15.47:0/3485625655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe588003970 0x7fe588003d90 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00c240 tx=0x7fe5880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1653d232d8cb344c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.029+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c005ad0 con 0x7fe588003970 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.029+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59c005c30 con 0x7fe588003970 2024-03-23T09:49:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.029+0000 7fe5977fe640 1 -- 172.21.15.47:0/3485625655 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c02c8b0 con 0x7fe588003970 2024-03-23T09:49:26.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.030+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/3485625655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe588003970 msgr2=0x7fe588003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.030+0000 7fe5a6a8f640 1 --2- 172.21.15.47:0/3485625655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe588003970 0x7fe588003d90 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00c240 tx=0x7fe5880057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.030+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/3485625655 shutdown_connections 2024-03-23T09:49:26.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.030+0000 7fe5a6a8f640 1 --2- 172.21.15.47:0/3485625655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe588003970 0x7fe588003d90 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.030+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/3485625655 >> 172.21.15.47:0/3485625655 conn(0x7fe5a00d0670 msgr2=0x7fe5a00d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:26.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.030+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/3485625655 shutdown_connections 2024-03-23T09:49:26.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.030+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/3485625655 wait complete. 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.031+0000 7fe5a6a8f640 1 Processor -- start 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.031+0000 7fe5a6a8f640 1 -- start start 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a6a8f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe588003970 0x7fe5a01560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a6a8f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5a00ad260 0x7fe5a015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a6a8f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5a015b590 0x7fe5a00b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a6a8f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00c8b60 con 0x7fe5a015b590 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a6a8f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00c8e40 con 0x7fe588003970 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a6a8f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00c9120 con 0x7fe5a00ad260 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a502c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5a015b590 0x7fe5a00b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a502c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5a015b590 0x7fe5a00b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37304/0 (socket says 172.21.15.47:37304) 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a502c640 1 -- 172.21.15.47:0/1687686333 learned_addr learned my addr 172.21.15.47:0/1687686333 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe597fff640 1 --2- 172.21.15.47:0/1687686333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5a00ad260 0x7fe5a015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a502c640 1 -- 172.21.15.47:0/1687686333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5a00ad260 msgr2=0x7fe5a015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a502c640 1 --2- 172.21.15.47:0/1687686333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5a00ad260 0x7fe5a015b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a502c640 1 -- 172.21.15.47:0/1687686333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe588003970 msgr2=0x7fe5a01560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a502c640 1 --2- 172.21.15.47:0/1687686333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe588003970 0x7fe5a01560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a502c640 1 -- 172.21.15.47:0/1687686333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a00c9400 con 0x7fe5a015b590 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.032+0000 7fe5a502c640 1 --2- 172.21.15.47:0/1687686333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5a015b590 0x7fe5a00b2140 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00c76a0 tx=0x7fe59800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.033+0000 7fe5957fa640 1 -- 172.21.15.47:0/1687686333 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598003b70 con 0x7fe5a015b590 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.033+0000 7fe5957fa640 1 -- 172.21.15.47:0/1687686333 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe598003cf0 con 0x7fe5a015b590 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.033+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/1687686333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a00c96e0 con 0x7fe5a015b590 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.033+0000 7fe5957fa640 1 -- 172.21.15.47:0/1687686333 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5980052b0 con 0x7fe5a015b590 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.033+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/1687686333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a00ad6e0 con 0x7fe5a015b590 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.034+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/1687686333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a00672f0 con 0x7fe5a015b590 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.034+0000 7fe5957fa640 1 -- 172.21.15.47:0/1687686333 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe598005410 con 0x7fe5a015b590 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.034+0000 7fe5957fa640 1 --2- 172.21.15.47:0/1687686333 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe58c06ac00 0x7fe58c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.034+0000 7fe5957fa640 1 -- 172.21.15.47:0/1687686333 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fe598088c00 con 0x7fe5a015b590 2024-03-23T09:49:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.035+0000 7fe5a482b640 1 --2- 172.21.15.47:0/1687686333 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe58c06ac00 0x7fe58c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.035+0000 7fe5a482b640 1 --2- 172.21.15.47:0/1687686333 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe58c06ac00 0x7fe58c06d0c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe59c034000 tx=0x7fe59c039040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:26.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.039+0000 7fe5957fa640 1 -- 172.21.15.47:0/1687686333 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe59808f080 con 0x7fe5a015b590 2024-03-23T09:49:26.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.253+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/1687686333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5a00ad930 con 0x7fe5a015b590 2024-03-23T09:49:26.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.253+0000 7fe5957fa640 1 -- 172.21.15.47:0/1687686333 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fe598018070 con 0x7fe5a015b590 2024-03-23T09:49:26.254 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:26.254 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:48:52.314987+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6835/3671387340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3671387340},{"type":"v1","addr":"172.21.15.47:6835","nonce":3671387340}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:49:26.254 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 59 2024-03-23T09:49:26.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.255+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/1687686333 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe58c06ac00 msgr2=0x7fe58c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.255+0000 7fe5a6a8f640 1 --2- 172.21.15.47:0/1687686333 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe58c06ac00 0x7fe58c06d0c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe59c034000 tx=0x7fe59c039040 comp rx=0 tx=0).stop 2024-03-23T09:49:26.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.255+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/1687686333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5a015b590 msgr2=0x7fe5a00b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.255+0000 7fe5a6a8f640 1 --2- 172.21.15.47:0/1687686333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5a015b590 0x7fe5a00b2140 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00c76a0 tx=0x7fe59800db80 comp rx=0 tx=0).stop 2024-03-23T09:49:26.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.256+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/1687686333 shutdown_connections 2024-03-23T09:49:26.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.256+0000 7fe5a6a8f640 1 --2- 172.21.15.47:0/1687686333 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe58c06ac00 0x7fe58c06d0c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.256+0000 7fe5a6a8f640 1 --2- 172.21.15.47:0/1687686333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5a015b590 0x7fe5a00b2140 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.256+0000 7fe5a6a8f640 1 --2- 172.21.15.47:0/1687686333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5a00ad260 0x7fe5a015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.256+0000 7fe5a6a8f640 1 --2- 172.21.15.47:0/1687686333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe588003970 0x7fe5a01560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.256+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/1687686333 >> 172.21.15.47:0/1687686333 conn(0x7fe5a00d0670 msgr2=0x7fe5a006e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:26.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.256+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/1687686333 shutdown_connections 2024-03-23T09:49:26.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.256+0000 7fe5a6a8f640 1 -- 172.21.15.47:0/1687686333 wait complete. 2024-03-23T09:49:26.266 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-23T09:49:26.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.368+0000 7f092bce0640 1 Processor -- start 2024-03-23T09:49:26.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.368+0000 7f092bce0640 1 -- start start 2024-03-23T09:49:26.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.369+0000 7f092bce0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f092415cb60 0x7f09240b5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.369+0000 7f092bce0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f09240cf7f0 0x7f09240cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.369+0000 7f092bce0640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09240b6310 con 0x7f09240b8500 2024-03-23T09:49:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.369+0000 7f092bce0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09240b6f90 con 0x7f09240cf7f0 2024-03-23T09:49:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.369+0000 7f092bce0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09240d01a0 con 0x7f092415cb60 2024-03-23T09:49:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.369+0000 7f092a27d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f09240cf7f0 0x7f09240cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.369+0000 7f092a27d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f09240cf7f0 0x7f09240cfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37314/0 (socket says 172.21.15.47:37314) 2024-03-23T09:49:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.369+0000 7f092a27d640 1 -- 172.21.15.47:0/3131268151 learned_addr learned my addr 172.21.15.47:0/3131268151 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.369+0000 7f0929a7c640 1 --2- 172.21.15.47:0/3131268151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f092415cb60 0x7f09240b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.369+0000 7f092a27d640 1 -- 172.21.15.47:0/3131268151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f092415cb60 msgr2=0x7f09240b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.369+0000 7f092a27d640 1 --2- 172.21.15.47:0/3131268151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f092415cb60 0x7f09240b5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.369+0000 7f092a27d640 1 -- 172.21.15.47:0/3131268151 >> v1:172.21.15.175:6789/0 conn(0x7f09240b8500 legacy=0x7f09240d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.369+0000 7f092a27d640 1 -- 172.21.15.47:0/3131268151 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09240b65f0 con 0x7f09240cf7f0 2024-03-23T09:49:26.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.370+0000 7f092a27d640 1 --2- 172.21.15.47:0/3131268151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09240cf7f0 0x7f09240cfc60 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f09140098a0 tx=0x7f091402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c15d2e4c93b8a40 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:26.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.370+0000 7f0928a7a640 1 -- 172.21.15.47:0/3131268151 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140317c0 con 0x7f09240cf7f0 2024-03-23T09:49:26.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.370+0000 7f0928a7a640 1 -- 172.21.15.47:0/3131268151 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0914031920 con 0x7f09240cf7f0 2024-03-23T09:49:26.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.370+0000 7f0928a7a640 1 -- 172.21.15.47:0/3131268151 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914031cb0 con 0x7f09240cf7f0 2024-03-23T09:49:26.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.371+0000 7f092bce0640 1 -- 172.21.15.47:0/3131268151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09240cf7f0 msgr2=0x7f09240cfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.371+0000 7f092bce0640 1 --2- 172.21.15.47:0/3131268151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09240cf7f0 0x7f09240cfc60 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f09140098a0 tx=0x7f091402b860 comp rx=0 tx=0).stop 2024-03-23T09:49:26.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.371+0000 7f092bce0640 1 -- 172.21.15.47:0/3131268151 shutdown_connections 2024-03-23T09:49:26.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.371+0000 7f092bce0640 1 --2- 172.21.15.47:0/3131268151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09240cf7f0 0x7f09240cfc60 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.371+0000 7f092bce0640 1 --2- 172.21.15.47:0/3131268151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f092415cb60 0x7f09240b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.371+0000 7f092bce0640 1 -- 172.21.15.47:0/3131268151 >> 172.21.15.47:0/3131268151 conn(0x7f0924065df0 msgr2=0x7f0924066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:26.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.371+0000 7f092bce0640 1 -- 172.21.15.47:0/3131268151 shutdown_connections 2024-03-23T09:49:26.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.371+0000 7f092bce0640 1 -- 172.21.15.47:0/3131268151 wait complete. 2024-03-23T09:49:26.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.372+0000 7f092bce0640 1 Processor -- start 2024-03-23T09:49:26.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.372+0000 7f092bce0640 1 -- start start 2024-03-23T09:49:26.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f092bce0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09240b8500 0x7f0924156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f092bce0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09240cf7f0 0x7f09240c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f092bce0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f092415cb60 0x7f092415d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f092bce0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0924157f10 con 0x7f09240b8500 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f092bce0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09241573b0 con 0x7f092415cb60 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f092bce0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092415f900 con 0x7f09240cf7f0 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f0929a7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09240b8500 0x7f0924156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f0929a7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09240b8500 0x7f0924156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37324/0 (socket says 172.21.15.47:37324) 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f0929a7c640 1 -- 172.21.15.47:0/1732330887 learned_addr learned my addr 172.21.15.47:0/1732330887 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f092927b640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09240cf7f0 0x7f09240c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f0929a7c640 1 -- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09240cf7f0 msgr2=0x7f09240c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f092a27d640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f092415cb60 0x7f092415d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f0929a7c640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09240cf7f0 0x7f09240c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f0929a7c640 1 -- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f092415cb60 msgr2=0x7f092415d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f0929a7c640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f092415cb60 0x7f092415d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f0929a7c640 1 -- 172.21.15.47:0/1732330887 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09140094d0 con 0x7f09240b8500 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f0929a7c640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09240b8500 0x7f0924156a20 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f091c00ce10 tx=0x7f091c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f09227fc640 1 -- 172.21.15.47:0/1732330887 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c003b90 con 0x7f09240b8500 2024-03-23T09:49:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f09227fc640 1 -- 172.21.15.47:0/1732330887 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c003cf0 con 0x7f09240b8500 2024-03-23T09:49:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f092bce0640 1 -- 172.21.15.47:0/1732330887 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0924157670 con 0x7f09240b8500 2024-03-23T09:49:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f09227fc640 1 -- 172.21.15.47:0/1732330887 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c003ea0 con 0x7f09240b8500 2024-03-23T09:49:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.373+0000 7f092bce0640 1 -- 172.21.15.47:0/1732330887 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0924157c30 con 0x7f09240b8500 2024-03-23T09:49:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.375+0000 7f09227fc640 1 -- 172.21.15.47:0/1732330887 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f091c004240 con 0x7f09240b8500 2024-03-23T09:49:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.375+0000 7f09227fc640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f091006ac00 0x7f091006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.375+0000 7f09227fc640 1 -- 172.21.15.47:0/1732330887 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f091c012070 con 0x7f09240b8500 2024-03-23T09:49:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.376+0000 7f092927b640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f091006ac00 0x7f091006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.376+0000 7f092927b640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f091006ac00 0x7f091006d0c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f09240c9030 tx=0x7f0918008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.376+0000 7f092e855740 1 Processor -- start 2024-03-23T09:49:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.376+0000 7f092e855740 1 -- start start 2024-03-23T09:49:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092e855740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 0x5575317c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092e855740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317966d0 0x5575317c3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092e855740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55753179a230 0x5575317c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092e855740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575317076a0 con 0x5575317966d0 2024-03-23T09:49:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092e855740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557531704c70 con 0x55753179a230 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092e855740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557531700560 con 0x5575317c66b0 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092927b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317966d0 0x5575317c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092927b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317966d0 0x5575317c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37332/0 (socket says 172.21.15.47:37332) 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092927b640 1 -- 172.21.15.47:0/3813916728 learned_addr learned my addr 172.21.15.47:0/3813916728 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f0929a7c640 1 --2- 172.21.15.47:0/3813916728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 0x5575317c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092927b640 1 -- 172.21.15.47:0/3813916728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 msgr2=0x5575317c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092927b640 1 --2- 172.21.15.47:0/3813916728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 0x5575317c6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092927b640 1 -- 172.21.15.47:0/3813916728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55753179a230 msgr2=0x5575317c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092a27d640 1 --2- 172.21.15.47:0/3813916728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55753179a230 0x5575317c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092927b640 1 --2- 172.21.15.47:0/3813916728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55753179a230 0x5575317c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092927b640 1 -- 172.21.15.47:0/3813916728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575316c7d50 con 0x5575317966d0 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.377+0000 7f092a27d640 1 --2- 172.21.15.47:0/3813916728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55753179a230 0x5575317c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.378+0000 7f092927b640 1 --2- 172.21.15.47:0/3813916728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317966d0 0x5575317c3920 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f091800c900 tx=0x7f091800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892a4ed54c9420cf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.378+0000 7f090b7fe640 1 -- 172.21.15.47:0/3813916728 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918013b80 con 0x5575317966d0 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.378+0000 7f090b7fe640 1 -- 172.21.15.47:0/3813916728 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0918013ce0 con 0x5575317966d0 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.378+0000 7f090b7fe640 1 -- 172.21.15.47:0/3813916728 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091801d630 con 0x5575317966d0 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.378+0000 7f092e855740 1 -- 172.21.15.47:0/3813916728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317966d0 msgr2=0x5575317c3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.378+0000 7f092e855740 1 --2- 172.21.15.47:0/3813916728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317966d0 0x5575317c3920 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f091800c900 tx=0x7f091800f000 comp rx=0 tx=0).stop 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.378+0000 7f092e855740 1 -- 172.21.15.47:0/3813916728 shutdown_connections 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.378+0000 7f092e855740 1 --2- 172.21.15.47:0/3813916728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55753179a230 0x5575317c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.378+0000 7f092e855740 1 --2- 172.21.15.47:0/3813916728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317966d0 0x5575317c3920 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.378+0000 7f092e855740 1 --2- 172.21.15.47:0/3813916728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 0x5575317c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.378+0000 7f092e855740 1 -- 172.21.15.47:0/3813916728 >> 172.21.15.47:0/3813916728 conn(0x557531798940 msgr2=0x5575317e14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.378+0000 7f092e855740 1 -- 172.21.15.47:0/3813916728 shutdown_connections 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.378+0000 7f092e855740 1 -- 172.21.15.47:0/3813916728 wait complete. 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.379+0000 7f092e855740 1 -- start start 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.379+0000 7f092e855740 1 Processor -- start 2024-03-23T09:49:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092e855740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55753179a230 0x5575317c1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092e855740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317c66b0 0x5575317c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092e855740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c1570 0x5575317ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092e855740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575313d93f0 con 0x5575317c66b0 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092e855740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575314893d0 con 0x55753179a230 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092e855740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557531683a10 con 0x5575317c1570 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092e855740 10 client.0 _ll_register_callbacks cb 0x5575316f9d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092927b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317c66b0 0x5575317c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092927b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317c66b0 0x5575317c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37344/0 (socket says 172.21.15.47:37344) 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092927b640 1 -- 172.21.15.47:0/1758039511 learned_addr learned my addr 172.21.15.47:0/1758039511 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092a27d640 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c1570 0x5575317ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f0929a7c640 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55753179a230 0x5575317c1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092927b640 1 -- 172.21.15.47:0/1758039511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c1570 msgr2=0x5575317ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092927b640 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c1570 0x5575317ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092927b640 1 -- 172.21.15.47:0/1758039511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55753179a230 msgr2=0x5575317c1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092927b640 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55753179a230 0x5575317c1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092927b640 1 -- 172.21.15.47:0/1758039511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091800c5b0 con 0x5575317c66b0 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f0929a7c640 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55753179a230 0x5575317c1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f092927b640 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317c66b0 0x5575317c9c30 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f091801e000 tx=0x7f0918004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748834abe254fd7f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.380+0000 7f0908ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f0908ff9640 1 -- 172.21.15.47:0/1758039511 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918004240 con 0x5575317c66b0 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f0908ff9640 1 -- 172.21.15.47:0/1758039511 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f09180043a0 con 0x5575317c66b0 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f092e855740 1 -- 172.21.15.47:0/1758039511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557531729150 con 0x5575317c66b0 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f0908ff9640 1 -- 172.21.15.47:0/1758039511 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09180035f0 con 0x5575317c66b0 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f092e855740 10 client.8298 fetch_fsmap learned FSMap version 59 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f092e855740 1 -- 172.21.15.47:0/1758039511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x5575316c2760 con 0x5575317c66b0 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f0908ff9640 1 -- 172.21.15.47:0/1758039511 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557531729150 con 0x5575317c66b0 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f0908ff9640 1 -- 172.21.15.47:0/1758039511 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 59) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0918003750 con 0x5575317c66b0 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f092e855740 10 client.8298 fetch_fsmap finished waiting for FSMap version 59 2024-03-23T09:49:26.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f092e855740 10 client.8298 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f092e855740 20 client.8298 populate_metadata read hostname 'smithi047' 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f092e855740 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x5575317ae2f0 0x5575317ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f092e855740 4 client.8298 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f092e855740 1 -- 172.21.15.47:0/1758039511 --> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55753170b650 con 0x5575317ae2f0 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f0929a7c640 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x5575317ae2f0 0x5575317ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f0929a7c640 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x5575317ae2f0 0x5575317ae6d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63718438557a8e6f server_cookie=8bcbc4dd2ea659d2 in_seq=0 out_seq=0 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.381+0000 7f0908ff9640 10 client.8298 ms_handle_connect on v2:172.21.15.47:6834/3671387340 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f0908ff9640 1 -- 172.21.15.47:0/1758039511 <== mds.0 v2:172.21.15.47:6834/3671387340 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55753170b650 con 0x5575317ae2f0 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f0908ff9640 10 client.8298 handle_command_reply: tid=0 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f092e855740 1 client.8298 shutdown 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f092e855740 1 -- 172.21.15.47:0/1758039511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317c66b0 msgr2=0x5575317c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f092e855740 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317c66b0 0x5575317c9c30 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f091801e000 tx=0x7f0918004680 comp rx=0 tx=0).stop 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f092e855740 1 -- 172.21.15.47:0/1758039511 shutdown_connections 2024-03-23T09:49:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f092e855740 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x5575317ae2f0 0x5575317ae6d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f092e855740 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c1570 0x5575317ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f092e855740 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575317c66b0 0x5575317c9c30 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f092e855740 1 --2- 172.21.15.47:0/1758039511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55753179a230 0x5575317c1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f0908ff9640 0 client.8298 ms_handle_reset on v2:172.21.15.47:6834/3671387340 2024-03-23T09:49:26.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f092e855740 1 -- 172.21.15.47:0/1758039511 >> 172.21.15.47:0/1758039511 conn(0x5575317966d0 msgr2=0x5575317e49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:26.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f092e855740 1 -- 172.21.15.47:0/1758039511 shutdown_connections 2024-03-23T09:49:26.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f092e855740 1 -- 172.21.15.47:0/1758039511 wait complete. 2024-03-23T09:49:26.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.382+0000 7f092e855740 20 client.8298 trim_cache size 0 max 16384 2024-03-23T09:49:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.394+0000 7f092e855740 1 Processor -- start 2024-03-23T09:49:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.394+0000 7f092e855740 1 -- start start 2024-03-23T09:49:26.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.394+0000 7f092e855740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 0x5575317c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.394+0000 7f092e855740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575317c1570 0x5575317c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.394+0000 7f092e855740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 0x5575317ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.394+0000 7f092e855740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557531799e80 con 0x55753179a230 2024-03-23T09:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.394+0000 7f092e855740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557531798270 con 0x5575317c1570 2024-03-23T09:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.394+0000 7f092e855740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575317adaf0 con 0x5575317c66b0 2024-03-23T09:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f0929a7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 0x5575317c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f0929a7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 0x5575317c6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37358/0 (socket says 172.21.15.47:37358) 2024-03-23T09:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f0929a7c640 1 -- 172.21.15.47:0/3927951869 learned_addr learned my addr 172.21.15.47:0/3927951869 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f092a27d640 1 --2- 172.21.15.47:0/3927951869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 0x5575317ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f092927b640 1 --2- 172.21.15.47:0/3927951869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575317c1570 0x5575317c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f0929a7c640 1 -- 172.21.15.47:0/3927951869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 msgr2=0x5575317ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f0929a7c640 1 --2- 172.21.15.47:0/3927951869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 0x5575317ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f0929a7c640 1 -- 172.21.15.47:0/3927951869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575317c1570 msgr2=0x5575317c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f0929a7c640 1 --2- 172.21.15.47:0/3927951869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575317c1570 0x5575317c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f0929a7c640 1 -- 172.21.15.47:0/3927951869 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575316a0ad0 con 0x55753179a230 2024-03-23T09:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f0929a7c640 1 --2- 172.21.15.47:0/3927951869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 0x5575317c6a90 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f091c004af0 tx=0x7f091c058b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1df639abaa3ed4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f090bfff640 1 -- 172.21.15.47:0/3927951869 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c05cad0 con 0x55753179a230 2024-03-23T09:49:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f090bfff640 1 -- 172.21.15.47:0/3927951869 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c061090 con 0x55753179a230 2024-03-23T09:49:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.395+0000 7f090bfff640 1 -- 172.21.15.47:0/3927951869 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c057040 con 0x55753179a230 2024-03-23T09:49:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.396+0000 7f092e855740 1 -- 172.21.15.47:0/3927951869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 msgr2=0x5575317c6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.396+0000 7f092e855740 1 --2- 172.21.15.47:0/3927951869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 0x5575317c6a90 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f091c004af0 tx=0x7f091c058b80 comp rx=0 tx=0).stop 2024-03-23T09:49:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.396+0000 7f092e855740 1 -- 172.21.15.47:0/3927951869 shutdown_connections 2024-03-23T09:49:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.396+0000 7f092e855740 1 --2- 172.21.15.47:0/3927951869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 0x5575317ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.396+0000 7f092e855740 1 --2- 172.21.15.47:0/3927951869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575317c1570 0x5575317c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.396+0000 7f092e855740 1 --2- 172.21.15.47:0/3927951869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 0x5575317c6a90 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.396+0000 7f092e855740 1 -- 172.21.15.47:0/3927951869 >> 172.21.15.47:0/3927951869 conn(0x5575317966d0 msgr2=0x557531795800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.396+0000 7f092e855740 1 -- 172.21.15.47:0/3927951869 shutdown_connections 2024-03-23T09:49:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.396+0000 7f092e855740 1 -- 172.21.15.47:0/3927951869 wait complete. 2024-03-23T09:49:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.396+0000 7f092e855740 1 -- start start 2024-03-23T09:49:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.396+0000 7f092e855740 1 Processor -- start 2024-03-23T09:49:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f092e855740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 0x5575317e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f092e855740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575317c1570 0x5575317abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f092e855740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 0x5575317c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f092e855740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575317c0600 con 0x55753179a230 2024-03-23T09:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f092e855740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575317e0b80 con 0x5575317c1570 2024-03-23T09:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f092e855740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575317ac790 con 0x5575317c66b0 2024-03-23T09:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f092e855740 10 client.0 _ll_register_callbacks cb 0x7f091800c7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f0929a7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 0x5575317e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f0929a7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 0x5575317e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37362/0 (socket says 172.21.15.47:37362) 2024-03-23T09:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f0929a7c640 1 -- 172.21.15.47:0/3119526657 learned_addr learned my addr 172.21.15.47:0/3119526657 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f092a27d640 1 --2- 172.21.15.47:0/3119526657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 0x5575317c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f092927b640 1 --2- 172.21.15.47:0/3119526657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575317c1570 0x5575317abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f0929a7c640 1 -- 172.21.15.47:0/3119526657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 msgr2=0x5575317c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f0929a7c640 1 --2- 172.21.15.47:0/3119526657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 0x5575317c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f0929a7c640 1 -- 172.21.15.47:0/3119526657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575317c1570 msgr2=0x5575317abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f0929a7c640 1 --2- 172.21.15.47:0/3119526657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575317c1570 0x5575317abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.397+0000 7f0929a7c640 1 -- 172.21.15.47:0/3119526657 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0924157c30 con 0x55753179a230 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f0929a7c640 1 --2- 172.21.15.47:0/3119526657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 0x5575317e0090 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f091c0551a0 tx=0x7f091c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc72e5db84d93690 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f0908ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f0908ff9640 1 -- 172.21.15.47:0/3119526657 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c0044f0 con 0x55753179a230 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f0908ff9640 1 -- 172.21.15.47:0/3119526657 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f091c004650 con 0x55753179a230 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f092e855740 1 -- 172.21.15.47:0/3119526657 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0918013ce0 con 0x55753179a230 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f0908ff9640 1 -- 172.21.15.47:0/3119526657 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f091c058190 con 0x55753179a230 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f092e855740 10 client.8304 fetch_fsmap learned FSMap version 59 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f092e855740 1 -- 172.21.15.47:0/3119526657 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x5575316c65b0 con 0x55753179a230 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f0908ff9640 1 -- 172.21.15.47:0/3119526657 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0918013ce0 con 0x55753179a230 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f0908ff9640 1 -- 172.21.15.47:0/3119526657 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 59) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f091c01d060 con 0x55753179a230 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f092e855740 10 client.8304 fetch_fsmap finished waiting for FSMap version 59 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f092e855740 10 client.8304 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f092e855740 20 client.8304 populate_metadata read hostname 'smithi047' 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f092e855740 1 --2- 172.21.15.47:0/3119526657 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x557531795dc0 0x5575317ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f092e855740 4 client.8304 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f092e855740 1 -- 172.21.15.47:0/3119526657 --> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5575313b18d0 con 0x557531795dc0 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.398+0000 7f092927b640 1 --2- 172.21.15.47:0/3119526657 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x557531795dc0 0x5575317ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f092927b640 1 --2- 172.21.15.47:0/3119526657 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x557531795dc0 0x5575317ca100 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ba4c5828eb4a64b server_cookie=2a9828502ca0dd4 in_seq=0 out_seq=0 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f0908ff9640 10 client.8304 ms_handle_connect on v2:172.21.15.47:6834/3671387340 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f0908ff9640 1 -- 172.21.15.47:0/3119526657 <== mds.0 v2:172.21.15.47:6834/3671387340 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5575313b18d0 con 0x557531795dc0 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f0908ff9640 10 client.8304 handle_command_reply: tid=0 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f092e855740 1 client.8304 shutdown 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f092e855740 1 -- 172.21.15.47:0/3119526657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 msgr2=0x5575317e0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f092e855740 1 --2- 172.21.15.47:0/3119526657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 0x5575317e0090 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f091c0551a0 tx=0x7f091c004a40 comp rx=0 tx=0).stop 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f092e855740 1 -- 172.21.15.47:0/3119526657 shutdown_connections 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f092e855740 1 --2- 172.21.15.47:0/3119526657 >> [v2:172.21.15.47:6834/3671387340,v1:172.21.15.47:6835/3671387340] conn(0x557531795dc0 0x5575317ca100 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f092e855740 1 --2- 172.21.15.47:0/3119526657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5575317c66b0 0x5575317c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f092e855740 1 --2- 172.21.15.47:0/3119526657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575317c1570 0x5575317abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f0908ff9640 0 client.8304 ms_handle_reset on v2:172.21.15.47:6834/3671387340 2024-03-23T09:49:26.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f092e855740 1 --2- 172.21.15.47:0/3119526657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55753179a230 0x5575317e0090 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f092e855740 1 -- 172.21.15.47:0/3119526657 >> 172.21.15.47:0/3119526657 conn(0x5575317966d0 msgr2=0x5575317c0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:26.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f092e855740 1 -- 172.21.15.47:0/3119526657 shutdown_connections 2024-03-23T09:49:26.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f092e855740 1 -- 172.21.15.47:0/3119526657 wait complete. 2024-03-23T09:49:26.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.399+0000 7f092e855740 20 client.8304 trim_cache size 0 max 16384 2024-03-23T09:49:26.406 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:26.406 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-23T09:49:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.400+0000 7f092bce0640 1 -- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f091006ac00 msgr2=0x7f091006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.400+0000 7f092bce0640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f091006ac00 0x7f091006d0c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f09240c9030 tx=0x7f0918008040 comp rx=0 tx=0).stop 2024-03-23T09:49:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.400+0000 7f092bce0640 1 -- 172.21.15.47:0/1732330887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09240b8500 msgr2=0x7f0924156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.400+0000 7f092bce0640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09240b8500 0x7f0924156a20 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f091c00ce10 tx=0x7f091c00a660 comp rx=0 tx=0).stop 2024-03-23T09:49:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.400+0000 7f092bce0640 1 -- 172.21.15.47:0/1732330887 shutdown_connections 2024-03-23T09:49:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.400+0000 7f092bce0640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f091006ac00 0x7f091006d0c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.400+0000 7f092bce0640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f092415cb60 0x7f092415d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.400+0000 7f092bce0640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09240cf7f0 0x7f09240c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.400+0000 7f092bce0640 1 --2- 172.21.15.47:0/1732330887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09240b8500 0x7f0924156a20 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.400+0000 7f092bce0640 1 -- 172.21.15.47:0/1732330887 >> 172.21.15.47:0/1732330887 conn(0x7f0924065df0 msgr2=0x7f09240d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.400+0000 7f092bce0640 1 -- 172.21.15.47:0/1732330887 shutdown_connections 2024-03-23T09:49:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.400+0000 7f092bce0640 1 -- 172.21.15.47:0/1732330887 wait complete. 2024-03-23T09:49:26.410 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-23T09:49:26.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.510+0000 7fb185438640 1 Processor -- start 2024-03-23T09:49:26.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.510+0000 7fb185438640 1 -- start start 2024-03-23T09:49:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.511+0000 7fb185438640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1801601e0 0x7fb1800cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.511+0000 7fb185438640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18006d580 con 0x7fb18006e620 2024-03-23T09:49:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.511+0000 7fb185438640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18006d860 con 0x7fb1800cce50 2024-03-23T09:49:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.511+0000 7fb185438640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800cd300 con 0x7fb1801601e0 2024-03-23T09:49:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.511+0000 7fb17e7fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb18006e620 0x7fb1800c9390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59200/0 (socket says 172.21.15.47:59200) 2024-03-23T09:49:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.511+0000 7fb17e7fc640 1 -- 172.21.15.47:0/3350719720 learned_addr learned my addr 172.21.15.47:0/3350719720 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.511+0000 7fb17effd640 1 --2- 172.21.15.47:0/3350719720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1801601e0 0x7fb1800cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.511+0000 7fb17dffb640 1 -- 172.21.15.47:0/3350719720 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455479411 0 0) 0x7fb18006d580 con 0x7fb18006e620 2024-03-23T09:49:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.511+0000 7fb17dffb640 1 -- 172.21.15.47:0/3350719720 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb16c003520 con 0x7fb18006e620 2024-03-23T09:49:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.512+0000 7fb17dffb640 1 -- 172.21.15.47:0/3350719720 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514550730 0 0) 0x7fb18006d860 con 0x7fb1800cce50 2024-03-23T09:49:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.512+0000 7fb17dffb640 1 -- 172.21.15.47:0/3350719720 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb18006d580 con 0x7fb1800cce50 2024-03-23T09:49:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.512+0000 7fb17effd640 1 -- 172.21.15.47:0/3350719720 >> v1:172.21.15.175:6789/0 conn(0x7fb1800cce50 legacy=0x7fb1800c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.512+0000 7fb17effd640 1 -- 172.21.15.47:0/3350719720 >> v1:172.21.15.47:6789/0 conn(0x7fb18006e620 legacy=0x7fb1800c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.512+0000 7fb17effd640 1 -- 172.21.15.47:0/3350719720 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18006db40 con 0x7fb1801601e0 2024-03-23T09:49:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.512+0000 7fb17effd640 1 --2- 172.21.15.47:0/3350719720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1801601e0 0x7fb1800cc910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb170009dc0 tx=0x7fb17002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17ec1e8d3de39798 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.513+0000 7fb17dffb640 1 -- 172.21.15.47:0/3350719720 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17002ccd0 con 0x7fb1801601e0 2024-03-23T09:49:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.513+0000 7fb17dffb640 1 -- 172.21.15.47:0/3350719720 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17002ce30 con 0x7fb1801601e0 2024-03-23T09:49:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.513+0000 7fb17dffb640 1 -- 172.21.15.47:0/3350719720 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170034660 con 0x7fb1801601e0 2024-03-23T09:49:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.513+0000 7fb185438640 1 -- 172.21.15.47:0/3350719720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1801601e0 msgr2=0x7fb1800cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.513+0000 7fb185438640 1 --2- 172.21.15.47:0/3350719720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1801601e0 0x7fb1800cc910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb170009dc0 tx=0x7fb17002bf20 comp rx=0 tx=0).stop 2024-03-23T09:49:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.513+0000 7fb185438640 1 -- 172.21.15.47:0/3350719720 shutdown_connections 2024-03-23T09:49:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.513+0000 7fb185438640 1 --2- 172.21.15.47:0/3350719720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1801601e0 0x7fb1800cc910 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.513+0000 7fb185438640 1 -- 172.21.15.47:0/3350719720 >> 172.21.15.47:0/3350719720 conn(0x7fb1800cb7c0 msgr2=0x7fb1800cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.513+0000 7fb185438640 1 -- 172.21.15.47:0/3350719720 shutdown_connections 2024-03-23T09:49:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.513+0000 7fb185438640 1 -- 172.21.15.47:0/3350719720 wait complete. 2024-03-23T09:49:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.514+0000 7fb185438640 1 Processor -- start 2024-03-23T09:49:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.515+0000 7fb185438640 1 -- start start 2024-03-23T09:49:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.515+0000 7fb185438640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb18006e620 0x7fb1800b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.515+0000 7fb185438640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1800cce50 0x7fb1801f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.515+0000 7fb185438640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1801601e0 0x7fb1800c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.515+0000 7fb185438640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800c4490 con 0x7fb18006e620 2024-03-23T09:49:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.515+0000 7fb185438640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800c3880 con 0x7fb1801601e0 2024-03-23T09:49:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.515+0000 7fb185438640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800c3b60 con 0x7fb1800cce50 2024-03-23T09:49:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.515+0000 7fb17effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb18006e620 0x7fb1800b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.515+0000 7fb17effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb18006e620 0x7fb1800b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37378/0 (socket says 172.21.15.47:37378) 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.515+0000 7fb17effd640 1 -- 172.21.15.47:0/2897031801 learned_addr learned my addr 172.21.15.47:0/2897031801 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb17f7fe640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1801601e0 0x7fb1800c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb17e7fc640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1800cce50 0x7fb1801f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb17effd640 1 -- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1800cce50 msgr2=0x7fb1801f2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb17effd640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1800cce50 0x7fb1801f2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb17effd640 1 -- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1801601e0 msgr2=0x7fb1800c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb17effd640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1801601e0 0x7fb1800c4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb17effd640 1 -- 172.21.15.47:0/2897031801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb170009a70 con 0x7fb18006e620 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb17f7fe640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1801601e0 0x7fb1800c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb17e7fc640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1800cce50 0x7fb1801f2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb17effd640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb18006e620 0x7fb1800b4180 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fb170000f80 tx=0x7fb170004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb1777fe640 1 -- 172.21.15.47:0/2897031801 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17003d600 con 0x7fb18006e620 2024-03-23T09:49:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb1777fe640 1 -- 172.21.15.47:0/2897031801 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17003d760 con 0x7fb18006e620 2024-03-23T09:49:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb1777fe640 1 -- 172.21.15.47:0/2897031801 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17003db10 con 0x7fb18006e620 2024-03-23T09:49:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb185438640 1 -- 172.21.15.47:0/2897031801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1800c3e40 con 0x7fb18006e620 2024-03-23T09:49:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.516+0000 7fb185438640 1 -- 172.21.15.47:0/2897031801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1800b46c0 con 0x7fb18006e620 2024-03-23T09:49:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.517+0000 7fb185438640 1 -- 172.21.15.47:0/2897031801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1800672f0 con 0x7fb18006e620 2024-03-23T09:49:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.518+0000 7fb1777fe640 1 -- 172.21.15.47:0/2897031801 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb17003b040 con 0x7fb18006e620 2024-03-23T09:49:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.518+0000 7fb1777fe640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb16006ac00 0x7fb16006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.518+0000 7fb1777fe640 1 -- 172.21.15.47:0/2897031801 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fb1700ab4e0 con 0x7fb18006e620 2024-03-23T09:49:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.518+0000 7fb17e7fc640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb16006ac00 0x7fb16006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.519+0000 7fb17e7fc640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb16006ac00 0x7fb16006d0c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb168000f60 tx=0x7fb168009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.522+0000 7fb1777fe640 1 -- 172.21.15.47:0/2897031801 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb170076660 con 0x7fb18006e620 2024-03-23T09:49:26.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.737+0000 7fb185438640 1 -- 172.21.15.47:0/2897031801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb180065dd0 con 0x7fb18006e620 2024-03-23T09:49:26.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.738+0000 7fb1777fe640 1 -- 172.21.15.47:0/2897031801 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb170009450 con 0x7fb18006e620 2024-03-23T09:49:26.738 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:26.738 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-23T09:49:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.739+0000 7fb185438640 1 -- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb16006ac00 msgr2=0x7fb16006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.739+0000 7fb185438640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb16006ac00 0x7fb16006d0c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb168000f60 tx=0x7fb168009040 comp rx=0 tx=0).stop 2024-03-23T09:49:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.739+0000 7fb185438640 1 -- 172.21.15.47:0/2897031801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb18006e620 msgr2=0x7fb1800b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.739+0000 7fb185438640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb18006e620 0x7fb1800b4180 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fb170000f80 tx=0x7fb170004a40 comp rx=0 tx=0).stop 2024-03-23T09:49:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.739+0000 7fb185438640 1 -- 172.21.15.47:0/2897031801 shutdown_connections 2024-03-23T09:49:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.739+0000 7fb185438640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb16006ac00 0x7fb16006d0c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.739+0000 7fb185438640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1801601e0 0x7fb1800c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.739+0000 7fb185438640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1800cce50 0x7fb1801f2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.739+0000 7fb185438640 1 --2- 172.21.15.47:0/2897031801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb18006e620 0x7fb1800b4180 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.739+0000 7fb185438640 1 -- 172.21.15.47:0/2897031801 >> 172.21.15.47:0/2897031801 conn(0x7fb1800cb7c0 msgr2=0x7fb1800c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.739+0000 7fb185438640 1 -- 172.21.15.47:0/2897031801 shutdown_connections 2024-03-23T09:49:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.739+0000 7fb185438640 1 -- 172.21.15.47:0/2897031801 wait complete. 2024-03-23T09:49:26.750 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-23T09:49:26.750 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:49:26.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:49:26.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:49:26.837 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:49:26.838 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:49:26.838 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:49:26.853 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:49:26.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.994+0000 7fb995cc4640 1 Processor -- start 2024-03-23T09:49:26.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.994+0000 7fb995cc4640 1 -- start start 2024-03-23T09:49:26.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.995+0000 7fb995cc4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb9900b8350 0x7fb9900cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:26.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.995+0000 7fb995cc4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9900ce090 con 0x7fb99015ca90 2024-03-23T09:49:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.995+0000 7fb995cc4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9900d11e0 con 0x7fb9900cdc40 2024-03-23T09:49:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.995+0000 7fb995cc4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb990160f60 con 0x7fb9900b8350 2024-03-23T09:49:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.995+0000 7fb98f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb99015ca90 0x7fb990160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59210/0 (socket says 172.21.15.47:59210) 2024-03-23T09:49:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.995+0000 7fb98f7fe640 1 -- 172.21.15.47:0/3450343394 learned_addr learned my addr 172.21.15.47:0/3450343394 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.995+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958318678 0 0) 0x7fb980001110 con 0x7fb99015ca90 2024-03-23T09:49:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.995+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb978003520 con 0x7fb99015ca90 2024-03-23T09:49:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98effd640 1 --2- 172.21.15.47:0/3450343394 >> v2:172.21.15.175:3300/0 conn(0x7fb9900b8350 0x7fb9900cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3645130215 0 0) 0x7fb978003520 con 0x7fb99015ca90 2024-03-23T09:49:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 >> v2:172.21.15.175:3300/0 conn(0x7fb9900b8350 msgr2=0x7fb9900cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98e7fc640 1 --2- 172.21.15.47:0/3450343394 >> v2:172.21.15.175:3300/0 conn(0x7fb9900b8350 0x7fb9900cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 >> v1:172.21.15.175:6789/0 conn(0x7fb9900cdc40 legacy=0x7fb9900d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb990161240 con 0x7fb99015ca90 2024-03-23T09:49:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fb980001800 con 0x7fb99015ca90 2024-03-23T09:49:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 >> v1:172.21.15.47:6789/0 conn(0x7fb99015ca90 legacy=0x7fb990160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98e7fc640 1 --2- 172.21.15.47:0/3450343394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb978003970 0x7fb978003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98e7fc640 1 --2- 172.21.15.47:0/3450343394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780047c0 0x7fb978004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98e7fc640 1 --2- 172.21.15.47:0/3450343394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9780052a0 0x7fb97800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb990160f60 con 0x7fb9780047c0 2024-03-23T09:49:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97800c150 con 0x7fb9780052a0 2024-03-23T09:49:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97800c310 con 0x7fb978003970 2024-03-23T09:49:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98effd640 1 --2- 172.21.15.47:0/3450343394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780047c0 0x7fb978004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98f7fe640 1 --2- 172.21.15.47:0/3450343394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb978003970 0x7fb978003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98ffff640 1 --2- 172.21.15.47:0/3450343394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9780052a0 0x7fb97800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98effd640 1 -- 172.21.15.47:0/3450343394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb978003970 msgr2=0x7fb978003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98effd640 1 --2- 172.21.15.47:0/3450343394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb978003970 0x7fb978003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98effd640 1 -- 172.21.15.47:0/3450343394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9780052a0 msgr2=0x7fb97800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98effd640 1 --2- 172.21.15.47:0/3450343394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9780052a0 0x7fb97800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98effd640 1 -- 172.21.15.47:0/3450343394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb978003520 con 0x7fb9780047c0 2024-03-23T09:49:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.996+0000 7fb98ffff640 1 -- 172.21.15.47:0/3450343394 reap_dead start 2024-03-23T09:49:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.997+0000 7fb98effd640 1 --2- 172.21.15.47:0/3450343394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780047c0 0x7fb978004c40 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb98400ba00 tx=0x7fb9780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0040d8d9448759 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.997+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984032620 con 0x7fb9780047c0 2024-03-23T09:49:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.997+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb984032780 con 0x7fb9780047c0 2024-03-23T09:49:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.997+0000 7fb98e7fc640 1 -- 172.21.15.47:0/3450343394 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984032b10 con 0x7fb9780047c0 2024-03-23T09:49:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.998+0000 7fb995cc4640 1 -- 172.21.15.47:0/3450343394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780047c0 msgr2=0x7fb978004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.998+0000 7fb995cc4640 1 --2- 172.21.15.47:0/3450343394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780047c0 0x7fb978004c40 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb98400ba00 tx=0x7fb9780057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.998+0000 7fb995cc4640 1 -- 172.21.15.47:0/3450343394 shutdown_connections 2024-03-23T09:49:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.998+0000 7fb995cc4640 1 --2- 172.21.15.47:0/3450343394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780047c0 0x7fb978004c40 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.998+0000 7fb995cc4640 1 -- 172.21.15.47:0/3450343394 >> 172.21.15.47:0/3450343394 conn(0x7fb9900659d0 msgr2=0x7fb990065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.998+0000 7fb995cc4640 1 -- 172.21.15.47:0/3450343394 shutdown_connections 2024-03-23T09:49:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.998+0000 7fb995cc4640 1 -- 172.21.15.47:0/3450343394 wait complete. 2024-03-23T09:49:27.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.999+0000 7fb995cc4640 1 Processor -- start 2024-03-23T09:49:27.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:26.999+0000 7fb995cc4640 1 -- start start 2024-03-23T09:49:27.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb995cc4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780047c0 0x7fb9900b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb995cc4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9900cb520 0x7fb990153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb995cc4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9900b7560 0x7fb9900b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb995cc4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9900b7f30 con 0x7fb9780047c0 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb995cc4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9900b4e10 con 0x7fb9900cb520 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb995cc4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9900b4290 con 0x7fb9900b7560 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb98f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780047c0 0x7fb9900b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb98f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780047c0 0x7fb9900b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37382/0 (socket says 172.21.15.47:37382) 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb98f7fe640 1 -- 172.21.15.47:0/3425700137 learned_addr learned my addr 172.21.15.47:0/3425700137 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb98effd640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9900cb520 0x7fb990153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb98ffff640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9900b7560 0x7fb9900b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb98f7fe640 1 -- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9900b7560 msgr2=0x7fb9900b79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb98f7fe640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9900b7560 0x7fb9900b79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb98f7fe640 1 -- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9900cb520 msgr2=0x7fb990153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb98f7fe640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9900cb520 0x7fb990153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb98f7fe640 1 -- 172.21.15.47:0/3425700137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9900b4570 con 0x7fb9780047c0 2024-03-23T09:49:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb98ffff640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9900b7560 0x7fb9900b79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:27.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb98effd640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9900cb520 0x7fb990153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:27.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb98f7fe640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780047c0 0x7fb9900b3cb0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb980007290 tx=0x7fb980007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:27.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb977fff640 1 -- 172.21.15.47:0/3425700137 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98000f9c0 con 0x7fb9780047c0 2024-03-23T09:49:27.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb977fff640 1 -- 172.21.15.47:0/3425700137 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98000fb20 con 0x7fb9780047c0 2024-03-23T09:49:27.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb995cc4640 1 -- 172.21.15.47:0/3425700137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9900b4850 con 0x7fb9780047c0 2024-03-23T09:49:27.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb977fff640 1 -- 172.21.15.47:0/3425700137 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98000feb0 con 0x7fb9780047c0 2024-03-23T09:49:27.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.000+0000 7fb995cc4640 1 -- 172.21.15.47:0/3425700137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb990159280 con 0x7fb9780047c0 2024-03-23T09:49:27.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.001+0000 7fb995cc4640 1 -- 172.21.15.47:0/3425700137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb990072360 con 0x7fb9780047c0 2024-03-23T09:49:27.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.002+0000 7fb977fff640 1 -- 172.21.15.47:0/3425700137 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb98001e3b0 con 0x7fb9780047c0 2024-03-23T09:49:27.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.002+0000 7fb977fff640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb97c06ac00 0x7fb97c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.002+0000 7fb977fff640 1 -- 172.21.15.47:0/3425700137 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fb980088f30 con 0x7fb9780047c0 2024-03-23T09:49:27.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.003+0000 7fb98effd640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb97c06ac00 0x7fb97c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.003+0000 7fb98effd640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb97c06ac00 0x7fb97c06d0c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb9780057a0 tx=0x7fb984030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:27.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.006+0000 7fb977fff640 1 -- 172.21.15.47:0/3425700137 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb9800540b0 con 0x7fb9780047c0 2024-03-23T09:49:27.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.189+0000 7fb995cc4640 1 -- 172.21.15.47:0/3425700137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9900ce090 con 0x7fb9780047c0 2024-03-23T09:49:27.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.190+0000 7fb977fff640 1 -- 172.21.15.47:0/3425700137 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16449 (secure 0 0 0) 0x7fb9900ce090 con 0x7fb9780047c0 2024-03-23T09:49:27.190 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:27.190 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":89,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:48:48.285538+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:47:09.722391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-23T09:47:10.354054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.30","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6834/4278516621":"2024-03-24T09:48:48.285514+0000","172.21.15.47:6835/4278516621":"2024-03-24T09:48:48.285514+0000","172.21.15.47:6835/1927629460":"2024-03-24T09:47:28.505920+0000","172.21.15.47:6834/1927629460":"2024-03-24T09:47:28.505920+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:49:27.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.192+0000 7fb995cc4640 1 -- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb97c06ac00 msgr2=0x7fb97c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.192+0000 7fb995cc4640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb97c06ac00 0x7fb97c06d0c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb9780057a0 tx=0x7fb984030000 comp rx=0 tx=0).stop 2024-03-23T09:49:27.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.192+0000 7fb995cc4640 1 -- 172.21.15.47:0/3425700137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780047c0 msgr2=0x7fb9900b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.192+0000 7fb995cc4640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780047c0 0x7fb9900b3cb0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb980007290 tx=0x7fb980007750 comp rx=0 tx=0).stop 2024-03-23T09:49:27.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.192+0000 7fb995cc4640 1 -- 172.21.15.47:0/3425700137 shutdown_connections 2024-03-23T09:49:27.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.192+0000 7fb995cc4640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb97c06ac00 0x7fb97c06d0c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.192+0000 7fb995cc4640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9900b7560 0x7fb9900b79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.192+0000 7fb995cc4640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9900cb520 0x7fb990153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.192+0000 7fb995cc4640 1 --2- 172.21.15.47:0/3425700137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780047c0 0x7fb9900b3cb0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.192+0000 7fb995cc4640 1 -- 172.21.15.47:0/3425700137 >> 172.21.15.47:0/3425700137 conn(0x7fb9900659d0 msgr2=0x7fb990072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:27.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.192+0000 7fb995cc4640 1 -- 172.21.15.47:0/3425700137 shutdown_connections 2024-03-23T09:49:27.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.192+0000 7fb995cc4640 1 -- 172.21.15.47:0/3425700137 wait complete. 2024-03-23T09:49:27.203 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:47:09.722391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:47:10.354054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:49:27.203 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:49:27.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.306+0000 7ff31bb20640 1 Processor -- start 2024-03-23T09:49:27.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.306+0000 7ff31bb20640 1 -- start start 2024-03-23T09:49:27.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.307+0000 7ff31bb20640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3140cde10 0x7ff314160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.307+0000 7ff31bb20640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff3140b8350 0x7ff3140d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.307+0000 7ff31bb20640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff314161630 con 0x7ff31415c990 2024-03-23T09:49:27.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.307+0000 7ff31bb20640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3140d1030 con 0x7ff3140b8350 2024-03-23T09:49:27.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.307+0000 7ff31bb20640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140cfe10 con 0x7ff3140cde10 2024-03-23T09:49:27.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.307+0000 7ff3198bc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff31415c990 0x7ff3140cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59212/0 (socket says 172.21.15.47:59212) 2024-03-23T09:49:27.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.307+0000 7ff3198bc640 1 -- 172.21.15.47:0/2244508796 learned_addr learned my addr 172.21.15.47:0/2244508796 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:27.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.307+0000 7ff31a0bd640 1 --2- 172.21.15.47:0/2244508796 >> v2:172.21.15.47:3300/0 conn(0x7ff3140b8350 0x7ff3140d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.308+0000 7ff3190bb640 1 --2- 172.21.15.47:0/2244508796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3140cde10 0x7ff314160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.308+0000 7ff31a0bd640 1 -- 172.21.15.47:0/2244508796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3140cde10 msgr2=0x7ff314160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:27.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.308+0000 7ff31a0bd640 1 --2- 172.21.15.47:0/2244508796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3140cde10 0x7ff314160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.308+0000 7ff31a0bd640 1 -- 172.21.15.47:0/2244508796 >> v1:172.21.15.47:6789/0 conn(0x7ff31415c990 legacy=0x7ff3140cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.308+0000 7ff31a0bd640 1 -- 172.21.15.47:0/2244508796 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3140b5b60 con 0x7ff3140b8350 2024-03-23T09:49:27.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.308+0000 7ff3188ba640 1 -- 172.21.15.47:0/2244508796 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613459776 0 0) 0x7ff314161630 con 0x7ff31415c990 2024-03-23T09:49:27.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.308+0000 7ff31a0bd640 1 --2- 172.21.15.47:0/2244508796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3140b8350 0x7ff3140d14c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff304009d90 tx=0x7ff30402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc6d0402c09f8dd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:27.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.308+0000 7ff3188ba640 1 -- 172.21.15.47:0/2244508796 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30402cb30 con 0x7ff3140b8350 2024-03-23T09:49:27.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.309+0000 7ff3188ba640 1 -- 172.21.15.47:0/2244508796 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30402cc90 con 0x7ff3140b8350 2024-03-23T09:49:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.309+0000 7ff3188ba640 1 -- 172.21.15.47:0/2244508796 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304034440 con 0x7ff3140b8350 2024-03-23T09:49:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.309+0000 7ff31bb20640 1 -- 172.21.15.47:0/2244508796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3140b8350 msgr2=0x7ff3140d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.309+0000 7ff31bb20640 1 --2- 172.21.15.47:0/2244508796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3140b8350 0x7ff3140d14c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff304009d90 tx=0x7ff30402bd80 comp rx=0 tx=0).stop 2024-03-23T09:49:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.309+0000 7ff31bb20640 1 -- 172.21.15.47:0/2244508796 shutdown_connections 2024-03-23T09:49:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.309+0000 7ff31bb20640 1 --2- 172.21.15.47:0/2244508796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3140b8350 0x7ff3140d14c0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.309+0000 7ff31bb20640 1 --2- 172.21.15.47:0/2244508796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3140cde10 0x7ff314160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.309+0000 7ff31bb20640 1 -- 172.21.15.47:0/2244508796 >> 172.21.15.47:0/2244508796 conn(0x7ff3140659d0 msgr2=0x7ff314065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.309+0000 7ff31bb20640 1 -- 172.21.15.47:0/2244508796 shutdown_connections 2024-03-23T09:49:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.309+0000 7ff31bb20640 1 -- 172.21.15.47:0/2244508796 wait complete. 2024-03-23T09:49:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.310+0000 7ff31bb20640 1 Processor -- start 2024-03-23T09:49:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.310+0000 7ff31bb20640 1 -- start start 2024-03-23T09:49:27.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31bb20640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3140b8350 0x7ff31415e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31bb20640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3140cde10 0x7ff314153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31bb20640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31415c990 0x7ff3140cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31bb20640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140c6880 con 0x7ff31415c990 2024-03-23T09:49:27.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31bb20640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31415e680 con 0x7ff3140b8350 2024-03-23T09:49:27.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31bb20640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31415e920 con 0x7ff3140cde10 2024-03-23T09:49:27.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31a0bd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31415c990 0x7ff3140cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31a0bd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31415c990 0x7ff3140cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37408/0 (socket says 172.21.15.47:37408) 2024-03-23T09:49:27.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31a0bd640 1 -- 172.21.15.47:0/3087081134 learned_addr learned my addr 172.21.15.47:0/3087081134 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:27.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff3190bb640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3140cde10 0x7ff314153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff3198bc640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3140b8350 0x7ff31415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31a0bd640 1 -- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3140cde10 msgr2=0x7ff314153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31a0bd640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3140cde10 0x7ff314153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31a0bd640 1 -- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3140b8350 msgr2=0x7ff31415e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31a0bd640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3140b8350 0x7ff31415e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.311+0000 7ff31a0bd640 1 -- 172.21.15.47:0/3087081134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff304009a70 con 0x7ff31415c990 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.312+0000 7ff31a0bd640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31415c990 0x7ff3140cad60 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff304009d90 tx=0x7ff304004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.312+0000 7ff3127fc640 1 -- 172.21.15.47:0/3087081134 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304003990 con 0x7ff31415c990 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.312+0000 7ff3127fc640 1 -- 172.21.15.47:0/3087081134 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff304003af0 con 0x7ff31415c990 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.312+0000 7ff31bb20640 1 -- 172.21.15.47:0/3087081134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31415ec00 con 0x7ff31415c990 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.312+0000 7ff3127fc640 1 -- 172.21.15.47:0/3087081134 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304003ea0 con 0x7ff31415c990 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.312+0000 7ff31bb20640 1 -- 172.21.15.47:0/3087081134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3140c6ad0 con 0x7ff31415c990 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.313+0000 7ff31bb20640 1 -- 172.21.15.47:0/3087081134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff314072360 con 0x7ff31415c990 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.313+0000 7ff3127fc640 1 -- 172.21.15.47:0/3087081134 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff30403b050 con 0x7ff31415c990 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.314+0000 7ff3127fc640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff30006ac00 0x7ff30006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.314+0000 7ff3127fc640 1 -- 172.21.15.47:0/3087081134 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff3040ab7e0 con 0x7ff31415c990 2024-03-23T09:49:27.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.314+0000 7ff3198bc640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff30006ac00 0x7ff30006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.314+0000 7ff3198bc640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff30006ac00 0x7ff30006d0c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff30c000f60 tx=0x7ff30c007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:27.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.318+0000 7ff3127fc640 1 -- 172.21.15.47:0/3087081134 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff304076960 con 0x7ff31415c990 2024-03-23T09:49:27.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.498+0000 7ff31bb20640 1 -- 172.21.15.47:0/3087081134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3140c6cf0 con 0x7ff31415c990 2024-03-23T09:49:27.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.498+0000 7ff3127fc640 1 -- 172.21.15.47:0/3087081134 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3040390a0 con 0x7ff31415c990 2024-03-23T09:49:27.498 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:49:27.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.500+0000 7ff31bb20640 1 -- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff30006ac00 msgr2=0x7ff30006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.500+0000 7ff31bb20640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff30006ac00 0x7ff30006d0c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff30c000f60 tx=0x7ff30c007b50 comp rx=0 tx=0).stop 2024-03-23T09:49:27.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.500+0000 7ff31bb20640 1 -- 172.21.15.47:0/3087081134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31415c990 msgr2=0x7ff3140cad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.500+0000 7ff31bb20640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31415c990 0x7ff3140cad60 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff304009d90 tx=0x7ff304004a40 comp rx=0 tx=0).stop 2024-03-23T09:49:27.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.500+0000 7ff31bb20640 1 -- 172.21.15.47:0/3087081134 shutdown_connections 2024-03-23T09:49:27.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.500+0000 7ff31bb20640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff30006ac00 0x7ff30006d0c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.500+0000 7ff31bb20640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31415c990 0x7ff3140cad60 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.500+0000 7ff31bb20640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3140cde10 0x7ff314153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.500+0000 7ff31bb20640 1 --2- 172.21.15.47:0/3087081134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3140b8350 0x7ff31415e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.500+0000 7ff31bb20640 1 -- 172.21.15.47:0/3087081134 >> 172.21.15.47:0/3087081134 conn(0x7ff3140659d0 msgr2=0x7ff3140b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:27.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.500+0000 7ff31bb20640 1 -- 172.21.15.47:0/3087081134 shutdown_connections 2024-03-23T09:49:27.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.500+0000 7ff31bb20640 1 -- 172.21.15.47:0/3087081134 wait complete. 2024-03-23T09:49:27.511 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:49:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.615+0000 7f3efd918640 1 Processor -- start 2024-03-23T09:49:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.615+0000 7f3efd918640 1 -- start start 2024-03-23T09:49:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3efd918640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ef815cee0 0x7f3ef80d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3efd918640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef80d23a0 0x7f3ef80d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3efd918640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3ef80b8300 0x7f3ef80d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3efd918640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef80d1300 con 0x7f3ef815cee0 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3efd918640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef80d15e0 con 0x7f3ef80b8300 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3efd918640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef80d18c0 con 0x7f3ef80d23a0 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3ef6ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ef815cee0 0x7f3ef80d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3ef6ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ef815cee0 0x7f3ef80d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37412/0 (socket says 172.21.15.47:37412) 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3ef6ffd640 1 -- 172.21.15.47:0/3198783760 learned_addr learned my addr 172.21.15.47:0/3198783760 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3ef77fe640 1 --2- 172.21.15.47:0/3198783760 >> v2:172.21.15.175:3300/0 conn(0x7f3ef80b8300 0x7f3ef80d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3ef6ffd640 1 -- 172.21.15.47:0/3198783760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef80d23a0 msgr2=0x7f3ef80d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3ef6ffd640 1 --2- 172.21.15.47:0/3198783760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef80d23a0 0x7f3ef80d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3ef6ffd640 1 -- 172.21.15.47:0/3198783760 >> v2:172.21.15.175:3300/0 conn(0x7f3ef80b8300 msgr2=0x7f3ef80d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3ef6ffd640 1 --2- 172.21.15.47:0/3198783760 >> v2:172.21.15.175:3300/0 conn(0x7f3ef80b8300 0x7f3ef80d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3ef6ffd640 1 -- 172.21.15.47:0/3198783760 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef80d1ba0 con 0x7f3ef815cee0 2024-03-23T09:49:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.616+0000 7f3ef77fe640 1 --2- 172.21.15.47:0/3198783760 >> v2:172.21.15.175:3300/0 conn(0x7f3ef80b8300 0x7f3ef80d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:27.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.617+0000 7f3ef6ffd640 1 --2- 172.21.15.47:0/3198783760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef815cee0 0x7f3ef80d0990 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80d0250 tx=0x7f3ee802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217c734d5cfa60e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:27.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.617+0000 7f3ef67fc640 1 -- 172.21.15.47:0/3198783760 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8031870 con 0x7f3ef815cee0 2024-03-23T09:49:27.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.617+0000 7f3ef67fc640 1 -- 172.21.15.47:0/3198783760 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee80319d0 con 0x7f3ef815cee0 2024-03-23T09:49:27.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.617+0000 7f3ef67fc640 1 -- 172.21.15.47:0/3198783760 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8031d60 con 0x7f3ef815cee0 2024-03-23T09:49:27.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.618+0000 7f3efd918640 1 -- 172.21.15.47:0/3198783760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef815cee0 msgr2=0x7f3ef80d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.618+0000 7f3efd918640 1 --2- 172.21.15.47:0/3198783760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef815cee0 0x7f3ef80d0990 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80d0250 tx=0x7f3ee802b910 comp rx=0 tx=0).stop 2024-03-23T09:49:27.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.618+0000 7f3efd918640 1 -- 172.21.15.47:0/3198783760 shutdown_connections 2024-03-23T09:49:27.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.618+0000 7f3efd918640 1 --2- 172.21.15.47:0/3198783760 >> v2:172.21.15.175:3300/0 conn(0x7f3ef80b8300 0x7f3ef80d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.618+0000 7f3efd918640 1 --2- 172.21.15.47:0/3198783760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef80d23a0 0x7f3ef80d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.618+0000 7f3efd918640 1 --2- 172.21.15.47:0/3198783760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef815cee0 0x7f3ef80d0990 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.618+0000 7f3efd918640 1 -- 172.21.15.47:0/3198783760 >> 172.21.15.47:0/3198783760 conn(0x7f3ef8065df0 msgr2=0x7f3ef8066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:27.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.618+0000 7f3efd918640 1 -- 172.21.15.47:0/3198783760 shutdown_connections 2024-03-23T09:49:27.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.618+0000 7f3efd918640 1 -- 172.21.15.47:0/3198783760 wait complete. 2024-03-23T09:49:27.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.619+0000 7f3efd918640 1 Processor -- start 2024-03-23T09:49:27.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.619+0000 7f3efd918640 1 -- start start 2024-03-23T09:49:27.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3efd918640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef80b8300 0x7f3ef8150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3efd918640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef80d23a0 0x7f3ef80c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3efd918640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef815cee0 0x7f3ef815e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3efd918640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef80ca910 con 0x7f3ef80b8300 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3efd918640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef80cabf0 con 0x7f3ef815cee0 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3efd918640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef81f35d0 con 0x7f3ef80d23a0 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3ef6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef80b8300 0x7f3ef8150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3ef6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef80b8300 0x7f3ef8150e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37424/0 (socket says 172.21.15.47:37424) 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3ef6ffd640 1 -- 172.21.15.47:0/2353275292 learned_addr learned my addr 172.21.15.47:0/2353275292 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3eeffff640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef80d23a0 0x7f3ef80c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3ef77fe640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef815cee0 0x7f3ef815e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3ef6ffd640 1 -- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef80d23a0 msgr2=0x7f3ef80c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3ef6ffd640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef80d23a0 0x7f3ef80c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3ef6ffd640 1 -- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef815cee0 msgr2=0x7f3ef815e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3ef6ffd640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef815cee0 0x7f3ef815e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3ef6ffd640 1 -- 172.21.15.47:0/2353275292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee80094d0 con 0x7f3ef80b8300 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3eeffff640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef80d23a0 0x7f3ef80c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3ef6ffd640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef80b8300 0x7f3ef8150e50 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8000c00 tx=0x7f3ee80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3ef4ff9640 1 -- 172.21.15.47:0/2353275292 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee803ad10 con 0x7f3ef80b8300 2024-03-23T09:49:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3ef4ff9640 1 -- 172.21.15.47:0/2353275292 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee803ae70 con 0x7f3ef80b8300 2024-03-23T09:49:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.620+0000 7f3efd918640 1 -- 172.21.15.47:0/2353275292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef81f38b0 con 0x7f3ef80b8300 2024-03-23T09:49:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.621+0000 7f3ef4ff9640 1 -- 172.21.15.47:0/2353275292 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8044610 con 0x7f3ef80b8300 2024-03-23T09:49:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.621+0000 7f3efd918640 1 -- 172.21.15.47:0/2353275292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef80caed0 con 0x7f3ef80b8300 2024-03-23T09:49:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.622+0000 7f3efd918640 1 -- 172.21.15.47:0/2353275292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef8072360 con 0x7f3ef80b8300 2024-03-23T09:49:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.622+0000 7f3ef4ff9640 1 -- 172.21.15.47:0/2353275292 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3ee804d450 con 0x7f3ef80b8300 2024-03-23T09:49:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.622+0000 7f3ef4ff9640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ed806ac00 0x7f3ed806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.622+0000 7f3ef4ff9640 1 -- 172.21.15.47:0/2353275292 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f3ee802d040 con 0x7f3ef80b8300 2024-03-23T09:49:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.623+0000 7f3eeffff640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ed806ac00 0x7f3ed806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.623+0000 7f3eeffff640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ed806ac00 0x7f3ed806d0c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8159ef0 tx=0x7f3ee0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:27.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.627+0000 7f3ef4ff9640 1 -- 172.21.15.47:0/2353275292 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3ee8076b60 con 0x7f3ef80b8300 2024-03-23T09:49:27.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.807+0000 7f3efd918640 1 -- 172.21.15.47:0/2353275292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ef8066be0 con 0x7f3ef80b8300 2024-03-23T09:49:27.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.807+0000 7f3ef4ff9640 1 -- 172.21.15.47:0/2353275292 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ee807a180 con 0x7f3ef80b8300 2024-03-23T09:49:27.807 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:49:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.809+0000 7f3efd918640 1 -- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ed806ac00 msgr2=0x7f3ed806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.809+0000 7f3efd918640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ed806ac00 0x7f3ed806d0c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8159ef0 tx=0x7f3ee0008040 comp rx=0 tx=0).stop 2024-03-23T09:49:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.809+0000 7f3efd918640 1 -- 172.21.15.47:0/2353275292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef80b8300 msgr2=0x7f3ef8150e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.809+0000 7f3efd918640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef80b8300 0x7f3ef8150e50 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8000c00 tx=0x7f3ee80046c0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.809+0000 7f3efd918640 1 -- 172.21.15.47:0/2353275292 shutdown_connections 2024-03-23T09:49:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.809+0000 7f3efd918640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ed806ac00 0x7f3ed806d0c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.809+0000 7f3efd918640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef815cee0 0x7f3ef815e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.809+0000 7f3efd918640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef80d23a0 0x7f3ef80c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.809+0000 7f3efd918640 1 --2- 172.21.15.47:0/2353275292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef80b8300 0x7f3ef8150e50 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.809+0000 7f3efd918640 1 -- 172.21.15.47:0/2353275292 >> 172.21.15.47:0/2353275292 conn(0x7f3ef8065df0 msgr2=0x7f3ef80b7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.809+0000 7f3efd918640 1 -- 172.21.15.47:0/2353275292 shutdown_connections 2024-03-23T09:49:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.809+0000 7f3efd918640 1 -- 172.21.15.47:0/2353275292 wait complete. 2024-03-23T09:49:27.820 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:49:27.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.924+0000 7f6028ee9640 1 Processor -- start 2024-03-23T09:49:27.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.924+0000 7f6028ee9640 1 -- start start 2024-03-23T09:49:27.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.924+0000 7f6028ee9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f602415c990 0x7f6024160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.924+0000 7f6028ee9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60240b8350 0x7f60240cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.924+0000 7f6028ee9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cdc40 0x7f60240b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.924+0000 7f6028ee9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60240ce0b0 con 0x7f60240b8350 2024-03-23T09:49:27.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.924+0000 7f6028ee9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6024161630 con 0x7f602415c990 2024-03-23T09:49:27.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.924+0000 7f6028ee9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60240d1030 con 0x7f60240cdc40 2024-03-23T09:49:27.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.925+0000 7f6021d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60240b8350 0x7f60240cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.925+0000 7f6021d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60240b8350 0x7f60240cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37436/0 (socket says 172.21.15.47:37436) 2024-03-23T09:49:27.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.925+0000 7f6021d9b640 1 -- 172.21.15.47:0/3303431007 learned_addr learned my addr 172.21.15.47:0/3303431007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:27.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.925+0000 7f602259c640 1 --2- 172.21.15.47:0/3303431007 >> v2:172.21.15.175:3300/0 conn(0x7f602415c990 0x7f6024160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.925+0000 7f6022d9d640 1 --2- 172.21.15.47:0/3303431007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cdc40 0x7f60240b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.925+0000 7f6021d9b640 1 -- 172.21.15.47:0/3303431007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cdc40 msgr2=0x7f60240b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:27.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.925+0000 7f6021d9b640 1 --2- 172.21.15.47:0/3303431007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cdc40 0x7f60240b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.925+0000 7f6021d9b640 1 -- 172.21.15.47:0/3303431007 >> v2:172.21.15.175:3300/0 conn(0x7f602415c990 msgr2=0x7f6024160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:27.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.925+0000 7f6021d9b640 1 --2- 172.21.15.47:0/3303431007 >> v2:172.21.15.175:3300/0 conn(0x7f602415c990 0x7f6024160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.925+0000 7f6021d9b640 1 -- 172.21.15.47:0/3303431007 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60240cfe10 con 0x7f60240b8350 2024-03-23T09:49:27.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.925+0000 7f602259c640 1 --2- 172.21.15.47:0/3303431007 >> v2:172.21.15.175:3300/0 conn(0x7f602415c990 0x7f6024160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:27.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.925+0000 7f6022d9d640 1 --2- 172.21.15.47:0/3303431007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cdc40 0x7f60240b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:27.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.926+0000 7f6021d9b640 1 --2- 172.21.15.47:0/3303431007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240b8350 0x7f60240cd700 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f60240b8b30 tx=0x7f600c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8145c42e121838 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:27.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.926+0000 7f602159a640 1 -- 172.21.15.47:0/3303431007 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c0317e0 con 0x7f60240b8350 2024-03-23T09:49:27.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.926+0000 7f602159a640 1 -- 172.21.15.47:0/3303431007 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c031940 con 0x7f60240b8350 2024-03-23T09:49:27.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.926+0000 7f602159a640 1 -- 172.21.15.47:0/3303431007 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c031cd0 con 0x7f60240b8350 2024-03-23T09:49:27.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.926+0000 7f6028ee9640 1 -- 172.21.15.47:0/3303431007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240b8350 msgr2=0x7f60240cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.926+0000 7f6028ee9640 1 --2- 172.21.15.47:0/3303431007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240b8350 0x7f60240cd700 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f60240b8b30 tx=0x7f600c02b880 comp rx=0 tx=0).stop 2024-03-23T09:49:27.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.926+0000 7f6028ee9640 1 -- 172.21.15.47:0/3303431007 shutdown_connections 2024-03-23T09:49:27.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.927+0000 7f6028ee9640 1 --2- 172.21.15.47:0/3303431007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cdc40 0x7f60240b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.927+0000 7f6028ee9640 1 --2- 172.21.15.47:0/3303431007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240b8350 0x7f60240cd700 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.927+0000 7f6028ee9640 1 --2- 172.21.15.47:0/3303431007 >> v2:172.21.15.175:3300/0 conn(0x7f602415c990 0x7f6024160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.927+0000 7f6028ee9640 1 -- 172.21.15.47:0/3303431007 >> 172.21.15.47:0/3303431007 conn(0x7f60240659d0 msgr2=0x7f6024065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:27.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.927+0000 7f6028ee9640 1 -- 172.21.15.47:0/3303431007 shutdown_connections 2024-03-23T09:49:27.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.927+0000 7f6028ee9640 1 -- 172.21.15.47:0/3303431007 wait complete. 2024-03-23T09:49:27.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.928+0000 7f6028ee9640 1 Processor -- start 2024-03-23T09:49:27.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.928+0000 7f6028ee9640 1 -- start start 2024-03-23T09:49:27.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f6028ee9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240b8350 0x7f6024157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f6028ee9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cdc40 0x7f60240b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f6028ee9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f602415c990 0x7f60240b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f6028ee9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60241f25e0 con 0x7f60240b8350 2024-03-23T09:49:27.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f6028ee9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60241f28c0 con 0x7f602415c990 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f6028ee9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60241f2ba0 con 0x7f60240cdc40 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f602259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240b8350 0x7f6024157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f602259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240b8350 0x7f6024157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37448/0 (socket says 172.21.15.47:37448) 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f602259c640 1 -- 172.21.15.47:0/223468886 learned_addr learned my addr 172.21.15.47:0/223468886 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f6022d9d640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f602415c990 0x7f60240b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f6021d9b640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cdc40 0x7f60240b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f602259c640 1 -- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cdc40 msgr2=0x7f60240b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f602259c640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cdc40 0x7f60240b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f602259c640 1 -- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f602415c990 msgr2=0x7f60240b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f602259c640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f602415c990 0x7f60240b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f602259c640 1 -- 172.21.15.47:0/223468886 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c0094d0 con 0x7f60240b8350 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f6022d9d640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f602415c990 0x7f60240b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f602259c640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240b8350 0x7f6024157f40 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f601c00c7f0 tx=0x7f601c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f601affd640 1 -- 172.21.15.47:0/223468886 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c007a30 con 0x7f60240b8350 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f601affd640 1 -- 172.21.15.47:0/223468886 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c007b90 con 0x7f60240b8350 2024-03-23T09:49:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f6028ee9640 1 -- 172.21.15.47:0/223468886 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60241f2e80 con 0x7f60240b8350 2024-03-23T09:49:27.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f601affd640 1 -- 172.21.15.47:0/223468886 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00ce40 con 0x7f60240b8350 2024-03-23T09:49:27.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.929+0000 7f6028ee9640 1 -- 172.21.15.47:0/223468886 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602415f110 con 0x7f60240b8350 2024-03-23T09:49:27.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.930+0000 7f6028ee9640 1 -- 172.21.15.47:0/223468886 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6024072360 con 0x7f60240b8350 2024-03-23T09:49:27.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.931+0000 7f601affd640 1 -- 172.21.15.47:0/223468886 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f601c0113f0 con 0x7f60240b8350 2024-03-23T09:49:27.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.931+0000 7f601affd640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f600406ac30 0x7f600406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:27.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.931+0000 7f601affd640 1 -- 172.21.15.47:0/223468886 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f601c089600 con 0x7f60240b8350 2024-03-23T09:49:27.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.931+0000 7f6021d9b640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f600406ac30 0x7f600406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:27.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.932+0000 7f6021d9b640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f600406ac30 0x7f600406d0f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f600c036000 tx=0x7f600c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:27.936+0000 7f601affd640 1 -- 172.21.15.47:0/223468886 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f601c0546f0 con 0x7f60240b8350 2024-03-23T09:49:28.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.116+0000 7f6028ee9640 1 -- 172.21.15.47:0/223468886 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6024066be0 con 0x7f60240b8350 2024-03-23T09:49:28.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.116+0000 7f601affd640 1 -- 172.21.15.47:0/223468886 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f601c087e60 con 0x7f60240b8350 2024-03-23T09:49:28.117 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:49:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.118+0000 7f6028ee9640 1 -- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f600406ac30 msgr2=0x7f600406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.118+0000 7f6028ee9640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f600406ac30 0x7f600406d0f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f600c036000 tx=0x7f600c03d040 comp rx=0 tx=0).stop 2024-03-23T09:49:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.118+0000 7f6028ee9640 1 -- 172.21.15.47:0/223468886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240b8350 msgr2=0x7f6024157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.118+0000 7f6028ee9640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240b8350 0x7f6024157f40 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f601c00c7f0 tx=0x7f601c00cbf0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.118+0000 7f6028ee9640 1 -- 172.21.15.47:0/223468886 shutdown_connections 2024-03-23T09:49:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.118+0000 7f6028ee9640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f600406ac30 0x7f600406d0f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.118+0000 7f6028ee9640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f602415c990 0x7f60240b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.118+0000 7f6028ee9640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cdc40 0x7f60240b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.118+0000 7f6028ee9640 1 --2- 172.21.15.47:0/223468886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240b8350 0x7f6024157f40 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.118+0000 7f6028ee9640 1 -- 172.21.15.47:0/223468886 >> 172.21.15.47:0/223468886 conn(0x7f60240659d0 msgr2=0x7f60240d13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.118+0000 7f6028ee9640 1 -- 172.21.15.47:0/223468886 shutdown_connections 2024-03-23T09:49:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.118+0000 7f6028ee9640 1 -- 172.21.15.47:0/223468886 wait complete. 2024-03-23T09:49:28.129 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:49:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.231+0000 7fd86008b640 1 Processor -- start 2024-03-23T09:49:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.231+0000 7fd86008b640 1 -- start start 2024-03-23T09:49:28.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.232+0000 7fd86008b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd8580cf210 0x7fd8580cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.232+0000 7fd86008b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8580b5a90 0x7fd8580b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.232+0000 7fd86008b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8580cfbd0 con 0x7fd85815cd00 2024-03-23T09:49:28.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.232+0000 7fd86008b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85815d0e0 con 0x7fd8580cf210 2024-03-23T09:49:28.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.232+0000 7fd86008b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580d1540 con 0x7fd8580b5a90 2024-03-23T09:49:28.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.232+0000 7fd85de27640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd85815cd00 0x7fd8580d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59218/0 (socket says 172.21.15.47:59218) 2024-03-23T09:49:28.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.232+0000 7fd85de27640 1 -- 172.21.15.47:0/2495212171 learned_addr learned my addr 172.21.15.47:0/2495212171 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:28.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.232+0000 7fd85d626640 1 --2- 172.21.15.47:0/2495212171 >> v2:172.21.15.175:3300/0 conn(0x7fd8580cf210 0x7fd8580cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.232+0000 7fd85e628640 1 --2- 172.21.15.47:0/2495212171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8580b5a90 0x7fd8580b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.232+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243279250 0 0) 0x7fd8580cfbd0 con 0x7fd85815cd00 2024-03-23T09:49:28.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.232+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd840003520 con 0x7fd85815cd00 2024-03-23T09:49:28.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3477870012 0 0) 0x7fd840003520 con 0x7fd85815cd00 2024-03-23T09:49:28.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8580b5a90 msgr2=0x7fd8580b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:28.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 --2- 172.21.15.47:0/2495212171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8580b5a90 0x7fd8580b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 >> v2:172.21.15.175:3300/0 conn(0x7fd8580cf210 msgr2=0x7fd8580cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 --2- 172.21.15.47:0/2495212171 >> v2:172.21.15.175:3300/0 conn(0x7fd8580cf210 0x7fd8580cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8580d1820 con 0x7fd85815cd00 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fd8480019f0 con 0x7fd85815cd00 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 >> v1:172.21.15.47:6789/0 conn(0x7fd85815cd00 legacy=0x7fd8580d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 --2- 172.21.15.47:0/2495212171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd840003970 0x7fd840003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 --2- 172.21.15.47:0/2495212171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8580b5a90 0x7fd8400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 --2- 172.21.15.47:0/2495212171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd840004d00 0x7fd840005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85815d0e0 con 0x7fd840003970 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580d1540 con 0x7fd840004d00 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84000b9b0 con 0x7fd8580b5a90 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85de27640 1 --2- 172.21.15.47:0/2495212171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd840003970 0x7fd840003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85e628640 1 --2- 172.21.15.47:0/2495212171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd840004d00 0x7fd840005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85d626640 1 --2- 172.21.15.47:0/2495212171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8580b5a90 0x7fd8400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85de27640 1 -- 172.21.15.47:0/2495212171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8580b5a90 msgr2=0x7fd8400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:28.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85de27640 1 --2- 172.21.15.47:0/2495212171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8580b5a90 0x7fd8400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85de27640 1 -- 172.21.15.47:0/2495212171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd840004d00 msgr2=0x7fd840005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85de27640 1 --2- 172.21.15.47:0/2495212171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd840004d00 0x7fd840005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85de27640 1 -- 172.21.15.47:0/2495212171 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd840003520 con 0x7fd840003970 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.233+0000 7fd85d626640 1 --2- 172.21.15.47:0/2495212171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8580b5a90 0x7fd8400047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.234+0000 7fd85de27640 1 --2- 172.21.15.47:0/2495212171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd840003970 0x7fd840003d90 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd84800a790 tx=0x7fd8400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844770b84b57106a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.234+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848034630 con 0x7fd840003970 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.234+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84802f070 con 0x7fd840003970 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.234+0000 7fd85ce25640 1 -- 172.21.15.47:0/2495212171 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480349f0 con 0x7fd840003970 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.234+0000 7fd86008b640 1 -- 172.21.15.47:0/2495212171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd840003970 msgr2=0x7fd840003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.234+0000 7fd86008b640 1 --2- 172.21.15.47:0/2495212171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd840003970 0x7fd840003d90 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd84800a790 tx=0x7fd8400057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.235+0000 7fd85e628640 1 -- 172.21.15.47:0/2495212171 reap_dead start 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.235+0000 7fd86008b640 1 -- 172.21.15.47:0/2495212171 shutdown_connections 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.235+0000 7fd86008b640 1 -- 172.21.15.47:0/2495212171 >> 172.21.15.47:0/2495212171 conn(0x7fd85806b160 msgr2=0x7fd85806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.235+0000 7fd86008b640 1 -- 172.21.15.47:0/2495212171 shutdown_connections 2024-03-23T09:49:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.235+0000 7fd86008b640 1 -- 172.21.15.47:0/2495212171 wait complete. 2024-03-23T09:49:28.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.236+0000 7fd86008b640 1 Processor -- start 2024-03-23T09:49:28.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.236+0000 7fd86008b640 1 -- start start 2024-03-23T09:49:28.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.236+0000 7fd86008b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd840003970 0x7fd8580cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.236+0000 7fd86008b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8580cc780 0x7fd858142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd86008b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd858142c80 0x7fd8580b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd86008b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580c74e0 con 0x7fd858142c80 2024-03-23T09:49:28.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd86008b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580c77c0 con 0x7fd8580cc780 2024-03-23T09:49:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd86008b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580c7aa0 con 0x7fd840003970 2024-03-23T09:49:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd85e628640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd858142c80 0x7fd8580b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd85e628640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd858142c80 0x7fd8580b49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37478/0 (socket says 172.21.15.47:37478) 2024-03-23T09:49:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd85e628640 1 -- 172.21.15.47:0/2462320599 learned_addr learned my addr 172.21.15.47:0/2462320599 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd85d626640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8580cc780 0x7fd858142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd85de27640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd840003970 0x7fd8580cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd85e628640 1 -- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd840003970 msgr2=0x7fd8580cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd85e628640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd840003970 0x7fd8580cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd85e628640 1 -- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8580cc780 msgr2=0x7fd858142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd85e628640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8580cc780 0x7fd858142740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd85e628640 1 -- 172.21.15.47:0/2462320599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8580c7d80 con 0x7fd858142c80 2024-03-23T09:49:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd85e628640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd858142c80 0x7fd8580b49f0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd85806cbf0 tx=0x7fd850011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd8567fc640 1 -- 172.21.15.47:0/2462320599 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850011ee0 con 0x7fd858142c80 2024-03-23T09:49:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd8567fc640 1 -- 172.21.15.47:0/2462320599 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd850004240 con 0x7fd858142c80 2024-03-23T09:49:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd86008b640 1 -- 172.21.15.47:0/2462320599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8580c8060 con 0x7fd858142c80 2024-03-23T09:49:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd8567fc640 1 -- 172.21.15.47:0/2462320599 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000f770 con 0x7fd858142c80 2024-03-23T09:49:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.237+0000 7fd86008b640 1 -- 172.21.15.47:0/2462320599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8580b4f30 con 0x7fd858142c80 2024-03-23T09:49:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.238+0000 7fd86008b640 1 -- 172.21.15.47:0/2462320599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd858072360 con 0x7fd858142c80 2024-03-23T09:49:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.239+0000 7fd8567fc640 1 -- 172.21.15.47:0/2462320599 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd85000f8d0 con 0x7fd858142c80 2024-03-23T09:49:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.239+0000 7fd8567fc640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd84406ac30 0x7fd84406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.239+0000 7fd8567fc640 1 -- 172.21.15.47:0/2462320599 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd8500894e0 con 0x7fd858142c80 2024-03-23T09:49:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.239+0000 7fd85de27640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd84406ac30 0x7fd84406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.240+0000 7fd85de27640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd84406ac30 0x7fd84406d0f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd848034000 tx=0x7fd848009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:28.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.243+0000 7fd8567fc640 1 -- 172.21.15.47:0/2462320599 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd8500545d0 con 0x7fd858142c80 2024-03-23T09:49:28.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.436+0000 7fd86008b640 1 -- 172.21.15.47:0/2462320599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd858066be0 con 0x7fd858142c80 2024-03-23T09:49:28.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.436+0000 7fd8567fc640 1 -- 172.21.15.47:0/2462320599 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v89) v1 ==== 80+0+244 (secure 0 0 0) 0x7fd850057bf0 con 0x7fd858142c80 2024-03-23T09:49:28.437 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6834/4278516621 2024-03-24T09:48:48.285514+0000 2024-03-23T09:49:28.437 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6835/4278516621 2024-03-24T09:48:48.285514+0000 2024-03-23T09:49:28.437 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6835/1927629460 2024-03-24T09:47:28.505920+0000 2024-03-23T09:49:28.437 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6834/1927629460 2024-03-24T09:47:28.505920+0000 2024-03-23T09:49:28.437 INFO:teuthology.orchestra.run.smithi047.stderr:listed 4 entries 2024-03-23T09:49:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.438+0000 7fd86008b640 1 -- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd84406ac30 msgr2=0x7fd84406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.438+0000 7fd86008b640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd84406ac30 0x7fd84406d0f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd848034000 tx=0x7fd848009030 comp rx=0 tx=0).stop 2024-03-23T09:49:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.438+0000 7fd86008b640 1 -- 172.21.15.47:0/2462320599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd858142c80 msgr2=0x7fd8580b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.438+0000 7fd86008b640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd858142c80 0x7fd8580b49f0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd85806cbf0 tx=0x7fd850011cf0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.439+0000 7fd86008b640 1 -- 172.21.15.47:0/2462320599 shutdown_connections 2024-03-23T09:49:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.439+0000 7fd86008b640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd84406ac30 0x7fd84406d0f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.439+0000 7fd86008b640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd858142c80 0x7fd8580b49f0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.439+0000 7fd86008b640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8580cc780 0x7fd858142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.439+0000 7fd86008b640 1 --2- 172.21.15.47:0/2462320599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd840003970 0x7fd8580cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.439+0000 7fd86008b640 1 -- 172.21.15.47:0/2462320599 >> 172.21.15.47:0/2462320599 conn(0x7fd85806b160 msgr2=0x7fd8580d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.439+0000 7fd86008b640 1 -- 172.21.15.47:0/2462320599 shutdown_connections 2024-03-23T09:49:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.439+0000 7fd86008b640 1 -- 172.21.15.47:0/2462320599 wait complete. 2024-03-23T09:49:28.449 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:49:28.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:28.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:49:28.474 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[70776]: fuse finished with error 0 and tester_r 0 2024-03-23T09:49:28.476 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:49:28.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:28.476 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:49:28.640 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:49:28.640 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:49:28.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:49:28.659 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:49:28.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:28.660 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-23T09:49:28.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:49:28.717 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-23T09:49:28.718 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:49:28.718 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:49:28.733 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:49:28.733 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:49:28.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:28.733 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-23T09:49:28.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:49:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-23T09:49:28.787 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:49:28.788 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:49:28.805 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:49:28.805 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:49:28.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:28.805 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-23T09:49:28.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:49:28.863 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-23T09:49:28.863 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:49:28.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.965+0000 7f504c23e640 1 Processor -- start 2024-03-23T09:49:28.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.965+0000 7f504c23e640 1 -- start start 2024-03-23T09:49:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.966+0000 7f504c23e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50440d23a0 0x7f50440d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.966+0000 7f504c23e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f50440b8300 0x7f50440d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.966+0000 7f504c23e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50440d10a0 con 0x7f504415cee0 2024-03-23T09:49:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.966+0000 7f504c23e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50440d1300 con 0x7f50440b8300 2024-03-23T09:49:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.966+0000 7f504c23e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50440d15e0 con 0x7f50440d23a0 2024-03-23T09:49:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.966+0000 7f5049fda640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f504415cee0 0x7f50440d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59222/0 (socket says 172.21.15.47:59222) 2024-03-23T09:49:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.966+0000 7f5049fda640 1 -- 172.21.15.47:0/3944119686 learned_addr learned my addr 172.21.15.47:0/3944119686 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.966+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265456881 0 0) 0x7f50440d10a0 con 0x7f504415cee0 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.966+0000 7f504a7db640 1 --2- 172.21.15.47:0/3944119686 >> v2:172.21.15.175:3300/0 conn(0x7f50440b8300 0x7f50440d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.966+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f502c003520 con 0x7f504415cee0 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.966+0000 7f50497d9640 1 --2- 172.21.15.47:0/3944119686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50440d23a0 0x7f50440d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4026583435 0 0) 0x7f502c003520 con 0x7f504415cee0 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50440d23a0 msgr2=0x7f50440d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 --2- 172.21.15.47:0/3944119686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50440d23a0 0x7f50440d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 >> v2:172.21.15.175:3300/0 conn(0x7f50440b8300 msgr2=0x7f50440d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 --2- 172.21.15.47:0/3944119686 >> v2:172.21.15.175:3300/0 conn(0x7f50440b8300 0x7f50440d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50440d18c0 con 0x7f504415cee0 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f503c001690 con 0x7f504415cee0 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 >> v1:172.21.15.47:6789/0 conn(0x7f504415cee0 legacy=0x7f50440d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 --2- 172.21.15.47:0/3944119686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50440d23a0 0x7f502c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 --2- 172.21.15.47:0/3944119686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f502c003ef0 0x7f502c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 --2- 172.21.15.47:0/3944119686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f502c004d00 0x7f502c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50440d1300 con 0x7f502c003ef0 2024-03-23T09:49:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50440d15e0 con 0x7f502c004d00 2024-03-23T09:49:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c00b980 con 0x7f50440d23a0 2024-03-23T09:49:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f50497d9640 1 --2- 172.21.15.47:0/3944119686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f502c003ef0 0x7f502c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f5049fda640 1 --2- 172.21.15.47:0/3944119686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50440d23a0 0x7f502c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f504a7db640 1 --2- 172.21.15.47:0/3944119686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f502c004d00 0x7f502c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f50497d9640 1 -- 172.21.15.47:0/3944119686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50440d23a0 msgr2=0x7f502c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f50497d9640 1 --2- 172.21.15.47:0/3944119686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50440d23a0 0x7f502c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f50497d9640 1 -- 172.21.15.47:0/3944119686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f502c004d00 msgr2=0x7f502c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f50497d9640 1 --2- 172.21.15.47:0/3944119686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f502c004d00 0x7f502c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.967+0000 7f50497d9640 1 -- 172.21.15.47:0/3944119686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c003520 con 0x7f502c003ef0 2024-03-23T09:49:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.968+0000 7f50497d9640 1 --2- 172.21.15.47:0/3944119686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f502c003ef0 0x7f502c0047c0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f5038009a90 tx=0x7f502c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12282a0ecd22299d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.968+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503802c6b0 con 0x7f502c003ef0 2024-03-23T09:49:28.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.968+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503802c810 con 0x7f502c003ef0 2024-03-23T09:49:28.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.968+0000 7f5048fd8640 1 -- 172.21.15.47:0/3944119686 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503802cba0 con 0x7f502c003ef0 2024-03-23T09:49:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.969+0000 7f504c23e640 1 -- 172.21.15.47:0/3944119686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f502c003ef0 msgr2=0x7f502c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.969+0000 7f504c23e640 1 --2- 172.21.15.47:0/3944119686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f502c003ef0 0x7f502c0047c0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f5038009a90 tx=0x7f502c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.969+0000 7f504a7db640 1 -- 172.21.15.47:0/3944119686 reap_dead start 2024-03-23T09:49:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.969+0000 7f504c23e640 1 -- 172.21.15.47:0/3944119686 shutdown_connections 2024-03-23T09:49:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.969+0000 7f504c23e640 1 -- 172.21.15.47:0/3944119686 >> 172.21.15.47:0/3944119686 conn(0x7f5044065df0 msgr2=0x7f5044066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.969+0000 7f504c23e640 1 -- 172.21.15.47:0/3944119686 shutdown_connections 2024-03-23T09:49:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.969+0000 7f504c23e640 1 -- 172.21.15.47:0/3944119686 wait complete. 2024-03-23T09:49:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.970+0000 7f504c23e640 1 Processor -- start 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.970+0000 7f504c23e640 1 -- start start 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504c23e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f502c003ef0 0x7f50440c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504c23e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5044146da0 0x7f5044153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504c23e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50441536c0 0x7f50440c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504c23e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5044153b70 con 0x7f50441536c0 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504c23e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50440c63e0 con 0x7f502c003ef0 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504c23e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50440c66c0 con 0x7f5044146da0 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504a7db640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50441536c0 0x7f50440c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504a7db640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50441536c0 0x7f50440c7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37486/0 (socket says 172.21.15.47:37486) 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504a7db640 1 -- 172.21.15.47:0/2348019959 learned_addr learned my addr 172.21.15.47:0/2348019959 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f5049fda640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f502c003ef0 0x7f50440c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f50497d9640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5044146da0 0x7f5044153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504a7db640 1 -- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5044146da0 msgr2=0x7f5044153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504a7db640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5044146da0 0x7f5044153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504a7db640 1 -- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f502c003ef0 msgr2=0x7f50440c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504a7db640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f502c003ef0 0x7f50440c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504a7db640 1 -- 172.21.15.47:0/2348019959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50440c69a0 con 0x7f50441536c0 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504a7db640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50441536c0 0x7f50440c7d00 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f50440b8720 tx=0x7f503400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f50427fc640 1 -- 172.21.15.47:0/2348019959 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503400e040 con 0x7f50441536c0 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f50427fc640 1 -- 172.21.15.47:0/2348019959 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503400f040 con 0x7f50441536c0 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504c23e640 1 -- 172.21.15.47:0/2348019959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50440c6c80 con 0x7f50441536c0 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f50427fc640 1 -- 172.21.15.47:0/2348019959 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034015430 con 0x7f50441536c0 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.971+0000 7f504c23e640 1 -- 172.21.15.47:0/2348019959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5044147220 con 0x7f50441536c0 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.972+0000 7f504c23e640 1 -- 172.21.15.47:0/2348019959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5044072360 con 0x7f50441536c0 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.973+0000 7f50427fc640 1 -- 172.21.15.47:0/2348019959 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5034015590 con 0x7f50441536c0 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.973+0000 7f50427fc640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f503006ac30 0x7f503006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.973+0000 7f50427fc640 1 -- 172.21.15.47:0/2348019959 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f5034089d50 con 0x7f50441536c0 2024-03-23T09:49:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.973+0000 7f5049fda640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f503006ac30 0x7f503006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.974+0000 7f5049fda640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f503006ac30 0x7f503006d0f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f50440d1230 tx=0x7f503c007e50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:28.977+0000 7f50427fc640 1 -- 172.21.15.47:0/2348019959 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f503408d030 con 0x7f50441536c0 2024-03-23T09:49:29.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.194+0000 7f504c23e640 1 -- 172.21.15.47:0/2348019959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50440c71a0 con 0x7f50441536c0 2024-03-23T09:49:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.195+0000 7f50427fc640 1 -- 172.21.15.47:0/2348019959 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f50440c71a0 con 0x7f50441536c0 2024-03-23T09:49:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:29.195 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:48:52.314987+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6835/3671387340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3671387340},{"type":"v1","addr":"172.21.15.47:6835","nonce":3671387340}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:49:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 59 2024-03-23T09:49:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.196+0000 7f504c23e640 1 -- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f503006ac30 msgr2=0x7f503006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.196+0000 7f504c23e640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f503006ac30 0x7f503006d0f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f50440d1230 tx=0x7f503c007e50 comp rx=0 tx=0).stop 2024-03-23T09:49:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.196+0000 7f504c23e640 1 -- 172.21.15.47:0/2348019959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50441536c0 msgr2=0x7f50440c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.196+0000 7f504c23e640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50441536c0 0x7f50440c7d00 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f50440b8720 tx=0x7f503400f1b0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.196+0000 7f504c23e640 1 -- 172.21.15.47:0/2348019959 shutdown_connections 2024-03-23T09:49:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.196+0000 7f504c23e640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f503006ac30 0x7f503006d0f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.196+0000 7f504c23e640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50441536c0 0x7f50440c7d00 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.196+0000 7f504c23e640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5044146da0 0x7f5044153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.196+0000 7f504c23e640 1 --2- 172.21.15.47:0/2348019959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f502c003ef0 0x7f50440c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.196+0000 7f504c23e640 1 -- 172.21.15.47:0/2348019959 >> 172.21.15.47:0/2348019959 conn(0x7f5044065df0 msgr2=0x7f504415b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.196+0000 7f504c23e640 1 -- 172.21.15.47:0/2348019959 shutdown_connections 2024-03-23T09:49:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.196+0000 7f504c23e640 1 -- 172.21.15.47:0/2348019959 wait complete. 2024-03-23T09:49:29.208 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:49:29.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.308+0000 7fcfeb5d2640 1 Processor -- start 2024-03-23T09:49:29.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.308+0000 7fcfeb5d2640 1 -- start start 2024-03-23T09:49:29.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.309+0000 7fcfeb5d2640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcfe415cb40 0x7fcfe4160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.309+0000 7fcfeb5d2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcfe40b8300 0x7fcfe40cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.309+0000 7fcfeb5d2640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe40d0ee0 con 0x7fcfe4160cb0 2024-03-23T09:49:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.309+0000 7fcfeb5d2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe40b5a10 con 0x7fcfe40b8300 2024-03-23T09:49:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.309+0000 7fcfeb5d2640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe41616f0 con 0x7fcfe415cb40 2024-03-23T09:49:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.309+0000 7fcfe8b6d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcfe40b8300 0x7fcfe40cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.309+0000 7fcfe8b6d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcfe40b8300 0x7fcfe40cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37496/0 (socket says 172.21.15.47:37496) 2024-03-23T09:49:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.309+0000 7fcfe8b6d640 1 -- 172.21.15.47:0/2742719788 learned_addr learned my addr 172.21.15.47:0/2742719788 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.309+0000 7fcfe936e640 1 --2- 172.21.15.47:0/2742719788 >> v2:172.21.15.175:3300/0 conn(0x7fcfe415cb40 0x7fcfe4160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.309+0000 7fcfe8b6d640 1 -- 172.21.15.47:0/2742719788 >> v2:172.21.15.175:3300/0 conn(0x7fcfe415cb40 msgr2=0x7fcfe4160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.309+0000 7fcfe8b6d640 1 --2- 172.21.15.47:0/2742719788 >> v2:172.21.15.175:3300/0 conn(0x7fcfe415cb40 0x7fcfe4160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.309+0000 7fcfe8b6d640 1 -- 172.21.15.47:0/2742719788 >> v1:172.21.15.175:6789/0 conn(0x7fcfe4160cb0 legacy=0x7fcfe40b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.309+0000 7fcfe8b6d640 1 -- 172.21.15.47:0/2742719788 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe4161130 con 0x7fcfe40b8300 2024-03-23T09:49:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.310+0000 7fcfe8b6d640 1 --2- 172.21.15.47:0/2742719788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfe40b8300 0x7fcfe40cf190 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009870 tx=0x7fcfd402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04e9ffb729b575c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.310+0000 7fcfdbfff640 1 -- 172.21.15.47:0/2742719788 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd40317e0 con 0x7fcfe40b8300 2024-03-23T09:49:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.310+0000 7fcfdbfff640 1 -- 172.21.15.47:0/2742719788 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd4031940 con 0x7fcfe40b8300 2024-03-23T09:49:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.310+0000 7fcfdbfff640 1 -- 172.21.15.47:0/2742719788 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4031cd0 con 0x7fcfe40b8300 2024-03-23T09:49:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.311+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/2742719788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfe40b8300 msgr2=0x7fcfe40cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.311+0000 7fcfeb5d2640 1 --2- 172.21.15.47:0/2742719788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfe40b8300 0x7fcfe40cf190 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009870 tx=0x7fcfd402b880 comp rx=0 tx=0).stop 2024-03-23T09:49:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.311+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/2742719788 shutdown_connections 2024-03-23T09:49:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.311+0000 7fcfeb5d2640 1 --2- 172.21.15.47:0/2742719788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfe40b8300 0x7fcfe40cf190 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.311+0000 7fcfeb5d2640 1 --2- 172.21.15.47:0/2742719788 >> v2:172.21.15.175:3300/0 conn(0x7fcfe415cb40 0x7fcfe4160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.311+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/2742719788 >> 172.21.15.47:0/2742719788 conn(0x7fcfe406b160 msgr2=0x7fcfe406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.311+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/2742719788 shutdown_connections 2024-03-23T09:49:29.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.311+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/2742719788 wait complete. 2024-03-23T09:49:29.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.312+0000 7fcfeb5d2640 1 Processor -- start 2024-03-23T09:49:29.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.312+0000 7fcfeb5d2640 1 -- start start 2024-03-23T09:49:29.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfeb5d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfe40b8300 0x7fcfe41f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfeb5d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfe415cb40 0x7fcfe415a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfeb5d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfe4160cb0 0x7fcfe40b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfeb5d2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4159780 con 0x7fcfe40b8300 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfeb5d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe41f34f0 con 0x7fcfe4160cb0 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfeb5d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4159c10 con 0x7fcfe415cb40 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfe936e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfe40b8300 0x7fcfe41f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfe936e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfe40b8300 0x7fcfe41f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37512/0 (socket says 172.21.15.47:37512) 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfe936e640 1 -- 172.21.15.47:0/1125192781 learned_addr learned my addr 172.21.15.47:0/1125192781 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfe936e640 1 -- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfe415cb40 msgr2=0x7fcfe415a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfe8b6d640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfe415cb40 0x7fcfe415a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfe9b6f640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfe4160cb0 0x7fcfe40b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfe936e640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfe415cb40 0x7fcfe415a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfe936e640 1 -- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfe4160cb0 msgr2=0x7fcfe40b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfe936e640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfe4160cb0 0x7fcfe40b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfe936e640 1 -- 172.21.15.47:0/1125192781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd40094d0 con 0x7fcfe40b8300 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfe8b6d640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfe415cb40 0x7fcfe415a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfe9b6f640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfe4160cb0 0x7fcfe40b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfe936e640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfe40b8300 0x7fcfe41f2fb0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000d830 tx=0x7fcfe000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfd9ffb640 1 -- 172.21.15.47:0/1125192781 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe000b6c0 con 0x7fcfe40b8300 2024-03-23T09:49:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfd9ffb640 1 -- 172.21.15.47:0/1125192781 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe000b820 con 0x7fcfe40b8300 2024-03-23T09:49:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/1125192781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe4158b70 con 0x7fcfe40b8300 2024-03-23T09:49:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.313+0000 7fcfd9ffb640 1 -- 172.21.15.47:0/1125192781 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe000bbb0 con 0x7fcfe40b8300 2024-03-23T09:49:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.314+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/1125192781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe4159130 con 0x7fcfe40b8300 2024-03-23T09:49:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.315+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/1125192781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe4072360 con 0x7fcfe40b8300 2024-03-23T09:49:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.315+0000 7fcfd9ffb640 1 -- 172.21.15.47:0/1125192781 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcfe001e460 con 0x7fcfe40b8300 2024-03-23T09:49:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.315+0000 7fcfd9ffb640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcfc806ac00 0x7fcfc806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.315+0000 7fcfd9ffb640 1 -- 172.21.15.47:0/1125192781 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fcfe0089c40 con 0x7fcfe40b8300 2024-03-23T09:49:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.316+0000 7fcfe8b6d640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcfc806ac00 0x7fcfc806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.316+0000 7fcfe8b6d640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcfc806ac00 0x7fcfc806d0c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4036000 tx=0x7fcfd403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:29.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.320+0000 7fcfd9ffb640 1 -- 172.21.15.47:0/1125192781 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcfe0054d30 con 0x7fcfe40b8300 2024-03-23T09:49:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.503+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/1125192781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfe4066be0 con 0x7fcfe40b8300 2024-03-23T09:49:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.504+0000 7fcfd9ffb640 1 -- 172.21.15.47:0/1125192781 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16449 (secure 0 0 0) 0x7fcfe0058350 con 0x7fcfe40b8300 2024-03-23T09:49:29.506 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:29.506 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":89,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:48:48.285538+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:47:09.722391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-23T09:47:10.354054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.30","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6834/4278516621":"2024-03-24T09:48:48.285514+0000","172.21.15.47:6835/4278516621":"2024-03-24T09:48:48.285514+0000","172.21.15.47:6835/1927629460":"2024-03-24T09:47:28.505920+0000","172.21.15.47:6834/1927629460":"2024-03-24T09:47:28.505920+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:49:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.506+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcfc806ac00 msgr2=0x7fcfc806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.506+0000 7fcfeb5d2640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcfc806ac00 0x7fcfc806d0c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4036000 tx=0x7fcfd403d040 comp rx=0 tx=0).stop 2024-03-23T09:49:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.506+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/1125192781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfe40b8300 msgr2=0x7fcfe41f2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.506+0000 7fcfeb5d2640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfe40b8300 0x7fcfe41f2fb0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000d830 tx=0x7fcfe000dcf0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.506+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/1125192781 shutdown_connections 2024-03-23T09:49:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.506+0000 7fcfeb5d2640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcfc806ac00 0x7fcfc806d0c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.506+0000 7fcfeb5d2640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfe4160cb0 0x7fcfe40b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.506+0000 7fcfeb5d2640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfe415cb40 0x7fcfe415a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.506+0000 7fcfeb5d2640 1 --2- 172.21.15.47:0/1125192781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfe40b8300 0x7fcfe41f2fb0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.506+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/1125192781 >> 172.21.15.47:0/1125192781 conn(0x7fcfe406b160 msgr2=0x7fcfe40c6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.506+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/1125192781 shutdown_connections 2024-03-23T09:49:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.506+0000 7fcfeb5d2640 1 -- 172.21.15.47:0/1125192781 wait complete. 2024-03-23T09:49:29.517 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:47:09.722391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:47:10.354054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:49:29.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:49:29.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.621+0000 7f6449ba4640 1 Processor -- start 2024-03-23T09:49:29.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.621+0000 7f6449ba4640 1 -- start start 2024-03-23T09:49:29.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.622+0000 7f6449ba4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f644415ca30 0x7f6444160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.622+0000 7f6449ba4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f644415ce10 0x7f64440cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.622+0000 7f6449ba4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f64440b85d0 0x7f64440cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.622+0000 7f6449ba4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6444161600 con 0x7f644415ce10 2024-03-23T09:49:29.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.622+0000 7f6449ba4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6444161040 con 0x7f64440b85d0 2024-03-23T09:49:29.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.622+0000 7f6449ba4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6444161320 con 0x7f644415ca30 2024-03-23T09:49:29.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.622+0000 7f6442ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f644415ce10 0x7f64440cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.622+0000 7f6442ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f644415ce10 0x7f64440cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37514/0 (socket says 172.21.15.47:37514) 2024-03-23T09:49:29.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.622+0000 7f6442ffd640 1 -- 172.21.15.47:0/3776893051 learned_addr learned my addr 172.21.15.47:0/3776893051 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:29.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.622+0000 7f6443fff640 1 --2- 172.21.15.47:0/3776893051 >> v2:172.21.15.175:3300/0 conn(0x7f64440b85d0 0x7f64440cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.622+0000 7f64437fe640 1 --2- 172.21.15.47:0/3776893051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f644415ca30 0x7f6444160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.623+0000 7f6442ffd640 1 -- 172.21.15.47:0/3776893051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f644415ca30 msgr2=0x7f6444160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:29.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.623+0000 7f6442ffd640 1 --2- 172.21.15.47:0/3776893051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f644415ca30 0x7f6444160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.623+0000 7f6442ffd640 1 -- 172.21.15.47:0/3776893051 >> v2:172.21.15.175:3300/0 conn(0x7f64440b85d0 msgr2=0x7f64440cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:29.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.623+0000 7f6442ffd640 1 --2- 172.21.15.47:0/3776893051 >> v2:172.21.15.175:3300/0 conn(0x7f64440b85d0 0x7f64440cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.623+0000 7f6442ffd640 1 -- 172.21.15.47:0/3776893051 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64440d0dd0 con 0x7f644415ce10 2024-03-23T09:49:29.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.623+0000 7f6443fff640 1 --2- 172.21.15.47:0/3776893051 >> v2:172.21.15.175:3300/0 conn(0x7f64440b85d0 0x7f64440cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:29.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.623+0000 7f6442ffd640 1 --2- 172.21.15.47:0/3776893051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644415ce10 0x7f64440cd730 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f6444160680 tx=0x7f643802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fe40c904a7f55f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:29.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.623+0000 7f64427fc640 1 -- 172.21.15.47:0/3776893051 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438031870 con 0x7f644415ce10 2024-03-23T09:49:29.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.623+0000 7f64427fc640 1 -- 172.21.15.47:0/3776893051 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64380319d0 con 0x7f644415ce10 2024-03-23T09:49:29.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.623+0000 7f64427fc640 1 -- 172.21.15.47:0/3776893051 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438031d60 con 0x7f644415ce10 2024-03-23T09:49:29.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.624+0000 7f6449ba4640 1 -- 172.21.15.47:0/3776893051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644415ce10 msgr2=0x7f64440cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.624+0000 7f6449ba4640 1 --2- 172.21.15.47:0/3776893051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644415ce10 0x7f64440cd730 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f6444160680 tx=0x7f643802b910 comp rx=0 tx=0).stop 2024-03-23T09:49:29.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.624+0000 7f6449ba4640 1 -- 172.21.15.47:0/3776893051 shutdown_connections 2024-03-23T09:49:29.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.624+0000 7f6449ba4640 1 --2- 172.21.15.47:0/3776893051 >> v2:172.21.15.175:3300/0 conn(0x7f64440b85d0 0x7f64440cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.624+0000 7f6449ba4640 1 --2- 172.21.15.47:0/3776893051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644415ce10 0x7f64440cd730 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.624+0000 7f6449ba4640 1 --2- 172.21.15.47:0/3776893051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f644415ca30 0x7f6444160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.624+0000 7f6449ba4640 1 -- 172.21.15.47:0/3776893051 >> 172.21.15.47:0/3776893051 conn(0x7f644406b160 msgr2=0x7f644406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:29.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.624+0000 7f6449ba4640 1 -- 172.21.15.47:0/3776893051 shutdown_connections 2024-03-23T09:49:29.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.624+0000 7f6449ba4640 1 -- 172.21.15.47:0/3776893051 wait complete. 2024-03-23T09:49:29.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.625+0000 7f6449ba4640 1 Processor -- start 2024-03-23T09:49:29.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.625+0000 7f6449ba4640 1 -- start start 2024-03-23T09:49:29.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6449ba4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64440b85d0 0x7f644414e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6449ba4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644415ca30 0x7f644415f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6449ba4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644415ce10 0x7f64440cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6449ba4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64440c8dd0 con 0x7f644415ca30 2024-03-23T09:49:29.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6449ba4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6444160180 con 0x7f644415ce10 2024-03-23T09:49:29.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6449ba4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644415fbd0 con 0x7f64440b85d0 2024-03-23T09:49:29.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6442ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644415ca30 0x7f644415f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6442ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644415ca30 0x7f644415f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37518/0 (socket says 172.21.15.47:37518) 2024-03-23T09:49:29.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6442ffd640 1 -- 172.21.15.47:0/3071376418 learned_addr learned my addr 172.21.15.47:0/3071376418 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:29.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f64437fe640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64440b85d0 0x7f644414e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6442ffd640 1 -- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64440b85d0 msgr2=0x7f644414e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6443fff640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644415ce10 0x7f64440cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6442ffd640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64440b85d0 0x7f644414e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6442ffd640 1 -- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644415ce10 msgr2=0x7f64440cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6442ffd640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644415ce10 0x7f64440cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6442ffd640 1 -- 172.21.15.47:0/3071376418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64380094d0 con 0x7f644415ca30 2024-03-23T09:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.626+0000 7f6443fff640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644415ce10 0x7f64440cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.627+0000 7f6442ffd640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644415ca30 0x7f644415f690 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f643802b950 tx=0x7f64380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.627+0000 7f642bfff640 1 -- 172.21.15.47:0/3071376418 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643803ad30 con 0x7f644415ca30 2024-03-23T09:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.627+0000 7f642bfff640 1 -- 172.21.15.47:0/3071376418 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64380443f0 con 0x7f644415ca30 2024-03-23T09:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.627+0000 7f6449ba4640 1 -- 172.21.15.47:0/3071376418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644406bae0 con 0x7f644415ca30 2024-03-23T09:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.627+0000 7f642bfff640 1 -- 172.21.15.47:0/3071376418 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64380447a0 con 0x7f644415ca30 2024-03-23T09:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.627+0000 7f6449ba4640 1 -- 172.21.15.47:0/3071376418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644415fea0 con 0x7f644415ca30 2024-03-23T09:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.628+0000 7f6449ba4640 1 -- 172.21.15.47:0/3071376418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644415fd10 con 0x7f644415ca30 2024-03-23T09:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.628+0000 7f642bfff640 1 -- 172.21.15.47:0/3071376418 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f643804d450 con 0x7f644415ca30 2024-03-23T09:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.629+0000 7f642bfff640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f643006ac30 0x7f643006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.629+0000 7f642bfff640 1 -- 172.21.15.47:0/3071376418 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f643802d040 con 0x7f644415ca30 2024-03-23T09:49:29.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.629+0000 7f64437fe640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f643006ac30 0x7f643006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.630+0000 7f64437fe640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f643006ac30 0x7f643006d0f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6434006fd0 tx=0x7f6434008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:29.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.633+0000 7f642bfff640 1 -- 172.21.15.47:0/3071376418 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6438076aa0 con 0x7f644415ca30 2024-03-23T09:49:29.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.811+0000 7f6449ba4640 1 -- 172.21.15.47:0/3071376418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6444072360 con 0x7f644415ca30 2024-03-23T09:49:29.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.811+0000 7f642bfff640 1 -- 172.21.15.47:0/3071376418 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6438035020 con 0x7f644415ca30 2024-03-23T09:49:29.812 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:49:29.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.813+0000 7f6449ba4640 1 -- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f643006ac30 msgr2=0x7f643006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.813+0000 7f6449ba4640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f643006ac30 0x7f643006d0f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6434006fd0 tx=0x7f6434008040 comp rx=0 tx=0).stop 2024-03-23T09:49:29.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.813+0000 7f6449ba4640 1 -- 172.21.15.47:0/3071376418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644415ca30 msgr2=0x7f644415f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.813+0000 7f6449ba4640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644415ca30 0x7f644415f690 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f643802b950 tx=0x7f64380046c0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.813+0000 7f6449ba4640 1 -- 172.21.15.47:0/3071376418 shutdown_connections 2024-03-23T09:49:29.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.813+0000 7f6449ba4640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f643006ac30 0x7f643006d0f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.813+0000 7f6449ba4640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644415ce10 0x7f64440cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.813+0000 7f6449ba4640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644415ca30 0x7f644415f690 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.813+0000 7f6449ba4640 1 --2- 172.21.15.47:0/3071376418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64440b85d0 0x7f644414e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.813+0000 7f6449ba4640 1 -- 172.21.15.47:0/3071376418 >> 172.21.15.47:0/3071376418 conn(0x7f644406b160 msgr2=0x7f64440b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:29.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.813+0000 7f6449ba4640 1 -- 172.21.15.47:0/3071376418 shutdown_connections 2024-03-23T09:49:29.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.813+0000 7f6449ba4640 1 -- 172.21.15.47:0/3071376418 wait complete. 2024-03-23T09:49:29.824 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:49:29.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.928+0000 7ff53fd31640 1 Processor -- start 2024-03-23T09:49:29.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.928+0000 7ff53fd31640 1 -- start start 2024-03-23T09:49:29.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.929+0000 7ff53fd31640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff53815c990 0x7ff538160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.929+0000 7ff53fd31640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff5380cde10 0x7ff5380d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.929+0000 7ff53fd31640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff538161630 con 0x7ff5380b8350 2024-03-23T09:49:29.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.929+0000 7ff53fd31640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5380d1030 con 0x7ff5380cde10 2024-03-23T09:49:29.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.929+0000 7ff53fd31640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5380cfe10 con 0x7ff53815c990 2024-03-23T09:49:29.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.929+0000 7ff53d2cc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff5380b8350 0x7ff5380cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59234/0 (socket says 172.21.15.47:59234) 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.929+0000 7ff53d2cc640 1 -- 172.21.15.47:0/1105988005 learned_addr learned my addr 172.21.15.47:0/1105988005 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.929+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143258654 0 0) 0x7ff538161630 con 0x7ff5380b8350 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.929+0000 7ff53e2ce640 1 --2- 172.21.15.47:0/1105988005 >> v2:172.21.15.175:3300/0 conn(0x7ff5380cde10 0x7ff5380d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.929+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff524003520 con 0x7ff5380b8350 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.929+0000 7ff53dacd640 1 --2- 172.21.15.47:0/1105988005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff53815c990 0x7ff538160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2686046320 0 0) 0x7ff524003520 con 0x7ff5380b8350 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff53815c990 msgr2=0x7ff538160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 --2- 172.21.15.47:0/1105988005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff53815c990 0x7ff538160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 >> v2:172.21.15.175:3300/0 conn(0x7ff5380cde10 msgr2=0x7ff5380d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 --2- 172.21.15.47:0/1105988005 >> v2:172.21.15.175:3300/0 conn(0x7ff5380cde10 0x7ff5380d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5380b5b60 con 0x7ff5380b8350 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7ff528001670 con 0x7ff5380b8350 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 >> v1:172.21.15.47:6789/0 conn(0x7ff5380b8350 legacy=0x7ff5380cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 --2- 172.21.15.47:0/1105988005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff524003970 0x7ff524003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 --2- 172.21.15.47:0/1105988005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff53815c990 0x7ff5240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 --2- 172.21.15.47:0/1105988005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff524004d00 0x7ff524005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5380d1030 con 0x7ff524004d00 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5380cfe10 con 0x7ff524003970 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52400b9b0 con 0x7ff53815c990 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53e2ce640 1 --2- 172.21.15.47:0/1105988005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff524004d00 0x7ff524005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53dacd640 1 --2- 172.21.15.47:0/1105988005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff524003970 0x7ff524003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53d2cc640 1 --2- 172.21.15.47:0/1105988005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff53815c990 0x7ff5240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53e2ce640 1 -- 172.21.15.47:0/1105988005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff53815c990 msgr2=0x7ff5240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53e2ce640 1 --2- 172.21.15.47:0/1105988005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff53815c990 0x7ff5240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53e2ce640 1 -- 172.21.15.47:0/1105988005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff524003970 msgr2=0x7ff524003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53e2ce640 1 --2- 172.21.15.47:0/1105988005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff524003970 0x7ff524003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.930+0000 7ff53e2ce640 1 -- 172.21.15.47:0/1105988005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff524003520 con 0x7ff524004d00 2024-03-23T09:49:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.931+0000 7ff53e2ce640 1 --2- 172.21.15.47:0/1105988005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff524004d00 0x7ff524005180 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff534009f90 tx=0x7ff5240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192714cc44f10411 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:29.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.931+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53402cc20 con 0x7ff524004d00 2024-03-23T09:49:29.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.931+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53400f040 con 0x7ff524004d00 2024-03-23T09:49:29.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.931+0000 7ff53cacb640 1 -- 172.21.15.47:0/1105988005 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534034440 con 0x7ff524004d00 2024-03-23T09:49:29.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.932+0000 7ff53fd31640 1 -- 172.21.15.47:0/1105988005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff524004d00 msgr2=0x7ff524005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.932+0000 7ff53fd31640 1 --2- 172.21.15.47:0/1105988005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff524004d00 0x7ff524005180 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff534009f90 tx=0x7ff5240057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.932+0000 7ff53e2ce640 1 -- 172.21.15.47:0/1105988005 reap_dead start 2024-03-23T09:49:29.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.932+0000 7ff53fd31640 1 -- 172.21.15.47:0/1105988005 shutdown_connections 2024-03-23T09:49:29.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.932+0000 7ff53fd31640 1 -- 172.21.15.47:0/1105988005 >> 172.21.15.47:0/1105988005 conn(0x7ff5380659d0 msgr2=0x7ff538065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:29.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.932+0000 7ff53fd31640 1 -- 172.21.15.47:0/1105988005 shutdown_connections 2024-03-23T09:49:29.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.932+0000 7ff53fd31640 1 -- 172.21.15.47:0/1105988005 wait complete. 2024-03-23T09:49:29.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.933+0000 7ff53fd31640 1 Processor -- start 2024-03-23T09:49:29.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.933+0000 7ff53fd31640 1 -- start start 2024-03-23T09:49:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53fd31640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff538150e40 0x7ff53815f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53fd31640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5380c8d60 0x7ff5381560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53fd31640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381565f0 0x7ff5380caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53fd31640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff538156a70 con 0x7ff538150e40 2024-03-23T09:49:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53fd31640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff538160110 con 0x7ff5381565f0 2024-03-23T09:49:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53fd31640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53815f870 con 0x7ff5380c8d60 2024-03-23T09:49:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53dacd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff538150e40 0x7ff53815f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53dacd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff538150e40 0x7ff53815f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37538/0 (socket says 172.21.15.47:37538) 2024-03-23T09:49:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53dacd640 1 -- 172.21.15.47:0/2522019374 learned_addr learned my addr 172.21.15.47:0/2522019374 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53dacd640 1 -- 172.21.15.47:0/2522019374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5380c8d60 msgr2=0x7ff5381560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53dacd640 1 --2- 172.21.15.47:0/2522019374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5380c8d60 0x7ff5381560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53dacd640 1 -- 172.21.15.47:0/2522019374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381565f0 msgr2=0x7ff5380caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53dacd640 1 --2- 172.21.15.47:0/2522019374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381565f0 0x7ff5380caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53dacd640 1 -- 172.21.15.47:0/2522019374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53815fb50 con 0x7ff538150e40 2024-03-23T09:49:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53dacd640 1 --2- 172.21.15.47:0/2522019374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff538150e40 0x7ff53815f200 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ff5300119a0 tx=0x7ff530011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff52e7fc640 1 -- 172.21.15.47:0/2522019374 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53000fa70 con 0x7ff538150e40 2024-03-23T09:49:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff52e7fc640 1 -- 172.21.15.47:0/2522019374 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53000fbd0 con 0x7ff538150e40 2024-03-23T09:49:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53fd31640 1 -- 172.21.15.47:0/2522019374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53815fe30 con 0x7ff538150e40 2024-03-23T09:49:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff52e7fc640 1 -- 172.21.15.47:0/2522019374 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530014440 con 0x7ff538150e40 2024-03-23T09:49:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.934+0000 7ff53fd31640 1 -- 172.21.15.47:0/2522019374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5380c91c0 con 0x7ff538150e40 2024-03-23T09:49:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.936+0000 7ff53fd31640 1 -- 172.21.15.47:0/2522019374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff538072360 con 0x7ff538150e40 2024-03-23T09:49:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.936+0000 7ff52e7fc640 1 -- 172.21.15.47:0/2522019374 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff5300145a0 con 0x7ff538150e40 2024-03-23T09:49:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.936+0000 7ff52e7fc640 1 --2- 172.21.15.47:0/2522019374 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff51c06ac30 0x7ff51c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.936+0000 7ff52e7fc640 1 -- 172.21.15.47:0/2522019374 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff530089c10 con 0x7ff538150e40 2024-03-23T09:49:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.937+0000 7ff53d2cc640 1 --2- 172.21.15.47:0/2522019374 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff51c06ac30 0x7ff51c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.937+0000 7ff53d2cc640 1 --2- 172.21.15.47:0/2522019374 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff51c06ac30 0x7ff51c06d0f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff528000c00 tx=0x7ff5280047e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:29.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:29.940+0000 7ff52e7fc640 1 -- 172.21.15.47:0/2522019374 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff530054d90 con 0x7ff538150e40 2024-03-23T09:49:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.120+0000 7ff53fd31640 1 -- 172.21.15.47:0/2522019374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff538161310 con 0x7ff538150e40 2024-03-23T09:49:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.120+0000 7ff52e7fc640 1 -- 172.21.15.47:0/2522019374 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5300583b0 con 0x7ff538150e40 2024-03-23T09:49:30.121 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:49:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.122+0000 7ff53fd31640 1 -- 172.21.15.47:0/2522019374 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff51c06ac30 msgr2=0x7ff51c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.122+0000 7ff53fd31640 1 --2- 172.21.15.47:0/2522019374 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff51c06ac30 0x7ff51c06d0f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff528000c00 tx=0x7ff5280047e0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.122+0000 7ff53fd31640 1 -- 172.21.15.47:0/2522019374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff538150e40 msgr2=0x7ff53815f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.122+0000 7ff53fd31640 1 --2- 172.21.15.47:0/2522019374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff538150e40 0x7ff53815f200 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ff5300119a0 tx=0x7ff530011e60 comp rx=0 tx=0).stop 2024-03-23T09:49:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.122+0000 7ff53fd31640 1 -- 172.21.15.47:0/2522019374 shutdown_connections 2024-03-23T09:49:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.122+0000 7ff53fd31640 1 --2- 172.21.15.47:0/2522019374 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff51c06ac30 0x7ff51c06d0f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.122+0000 7ff53fd31640 1 --2- 172.21.15.47:0/2522019374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381565f0 0x7ff5380caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.122+0000 7ff53fd31640 1 --2- 172.21.15.47:0/2522019374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5380c8d60 0x7ff5381560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.122+0000 7ff53fd31640 1 --2- 172.21.15.47:0/2522019374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff538150e40 0x7ff53815f200 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.122+0000 7ff53fd31640 1 -- 172.21.15.47:0/2522019374 >> 172.21.15.47:0/2522019374 conn(0x7ff5380659d0 msgr2=0x7ff5380b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.122+0000 7ff53fd31640 1 -- 172.21.15.47:0/2522019374 shutdown_connections 2024-03-23T09:49:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.122+0000 7ff53fd31640 1 -- 172.21.15.47:0/2522019374 wait complete. 2024-03-23T09:49:30.133 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:49:30.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.237+0000 7fcaa55ea640 1 Processor -- start 2024-03-23T09:49:30.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.237+0000 7fcaa55ea640 1 -- start start 2024-03-23T09:49:30.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fcaa55ea640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcaa00cde10 0x7fcaa0160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fcaa55ea640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa0161630 con 0x7fcaa00b8350 2024-03-23T09:49:30.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fcaa55ea640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa00d1030 con 0x7fcaa015c990 2024-03-23T09:49:30.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fcaa55ea640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa00cfe10 con 0x7fcaa00cde10 2024-03-23T09:49:30.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fca9f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcaa00b8350 0x7fcaa00d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59242/0 (socket says 172.21.15.47:59242) 2024-03-23T09:49:30.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fca9f7fe640 1 -- 172.21.15.47:0/3190839223 learned_addr learned my addr 172.21.15.47:0/3190839223 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:30.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309690497 0 0) 0x7fcaa0161630 con 0x7fcaa00b8350 2024-03-23T09:49:30.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca8c003520 con 0x7fcaa00b8350 2024-03-23T09:49:30.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fca97fff640 1 --2- 172.21.15.47:0/3190839223 >> v2:172.21.15.175:3300/0 conn(0x7fcaa00cde10 0x7fcaa0160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 824673895 0 0) 0x7fca8c003520 con 0x7fcaa00b8350 2024-03-23T09:49:30.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 >> v2:172.21.15.175:3300/0 conn(0x7fcaa00cde10 msgr2=0x7fcaa0160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:30.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fca9e7fc640 1 --2- 172.21.15.47:0/3190839223 >> v2:172.21.15.175:3300/0 conn(0x7fcaa00cde10 0x7fcaa0160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 >> v1:172.21.15.175:6789/0 conn(0x7fcaa015c990 legacy=0x7fcaa00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa00b5b60 con 0x7fcaa00b8350 2024-03-23T09:49:30.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fca980019f0 con 0x7fcaa00b8350 2024-03-23T09:49:30.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 >> v1:172.21.15.47:6789/0 conn(0x7fcaa00b8350 legacy=0x7fcaa00d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.238+0000 7fca9e7fc640 1 --2- 172.21.15.47:0/3190839223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8c003970 0x7fca8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca9e7fc640 1 --2- 172.21.15.47:0/3190839223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca8c0047c0 0x7fca8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca9e7fc640 1 --2- 172.21.15.47:0/3190839223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca8c005180 0x7fca8c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa00cfe10 con 0x7fca8c0047c0 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c00c120 con 0x7fca8c003970 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c00c2e0 con 0x7fca8c005180 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca97fff640 1 --2- 172.21.15.47:0/3190839223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca8c0047c0 0x7fca8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca9effd640 1 --2- 172.21.15.47:0/3190839223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8c003970 0x7fca8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca9f7fe640 1 --2- 172.21.15.47:0/3190839223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca8c005180 0x7fca8c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca97fff640 1 -- 172.21.15.47:0/3190839223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca8c005180 msgr2=0x7fca8c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca97fff640 1 --2- 172.21.15.47:0/3190839223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca8c005180 0x7fca8c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca97fff640 1 -- 172.21.15.47:0/3190839223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8c003970 msgr2=0x7fca8c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca97fff640 1 --2- 172.21.15.47:0/3190839223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8c003970 0x7fca8c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca97fff640 1 -- 172.21.15.47:0/3190839223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c003520 con 0x7fca8c0047c0 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca9f7fe640 1 --2- 172.21.15.47:0/3190839223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca8c005180 0x7fca8c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.239+0000 7fca9f7fe640 1 -- 172.21.15.47:0/3190839223 reap_dead start 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.240+0000 7fca97fff640 1 --2- 172.21.15.47:0/3190839223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca8c0047c0 0x7fca8c004c40 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fca88009ac0 tx=0x7fca8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d90d851aaa4196 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.240+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8802c6b0 con 0x7fca8c0047c0 2024-03-23T09:49:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.240+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8802c810 con 0x7fca8c0047c0 2024-03-23T09:49:30.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.240+0000 7fca9e7fc640 1 -- 172.21.15.47:0/3190839223 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8802cba0 con 0x7fca8c0047c0 2024-03-23T09:49:30.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.240+0000 7fcaa55ea640 1 -- 172.21.15.47:0/3190839223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca8c0047c0 msgr2=0x7fca8c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.240+0000 7fcaa55ea640 1 --2- 172.21.15.47:0/3190839223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca8c0047c0 0x7fca8c004c40 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fca88009ac0 tx=0x7fca8c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.240+0000 7fcaa55ea640 1 -- 172.21.15.47:0/3190839223 shutdown_connections 2024-03-23T09:49:30.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.240+0000 7fcaa55ea640 1 --2- 172.21.15.47:0/3190839223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca8c0047c0 0x7fca8c004c40 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.240+0000 7fcaa55ea640 1 -- 172.21.15.47:0/3190839223 >> 172.21.15.47:0/3190839223 conn(0x7fcaa00659d0 msgr2=0x7fcaa0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:30.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.240+0000 7fcaa55ea640 1 -- 172.21.15.47:0/3190839223 shutdown_connections 2024-03-23T09:49:30.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.240+0000 7fcaa55ea640 1 -- 172.21.15.47:0/3190839223 wait complete. 2024-03-23T09:49:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.242+0000 7fcaa55ea640 1 Processor -- start 2024-03-23T09:49:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.242+0000 7fcaa55ea640 1 -- start start 2024-03-23T09:49:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.242+0000 7fcaa55ea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca8c0047c0 0x7fcaa015f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.242+0000 7fcaa55ea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaa015e290 0x7fcaa01f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.242+0000 7fcaa55ea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcaa01f1960 0x7fcaa01f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.242+0000 7fcaa55ea640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa015f480 con 0x7fca8c0047c0 2024-03-23T09:49:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.242+0000 7fcaa55ea640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa00c8ff0 con 0x7fcaa015e290 2024-03-23T09:49:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.242+0000 7fcaa55ea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa015fde0 con 0x7fcaa01f1960 2024-03-23T09:49:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.242+0000 7fca9effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca8c0047c0 0x7fcaa015f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.242+0000 7fca9effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca8c0047c0 0x7fcaa015f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37548/0 (socket says 172.21.15.47:37548) 2024-03-23T09:49:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.242+0000 7fca9effd640 1 -- 172.21.15.47:0/1789679901 learned_addr learned my addr 172.21.15.47:0/1789679901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fca97fff640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaa015e290 0x7fcaa01f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fca9f7fe640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcaa01f1960 0x7fcaa01f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fca9effd640 1 -- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcaa01f1960 msgr2=0x7fcaa01f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fca9effd640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcaa01f1960 0x7fcaa01f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fca9effd640 1 -- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaa015e290 msgr2=0x7fcaa01f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fca9effd640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaa015e290 0x7fcaa01f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fca9effd640 1 -- 172.21.15.47:0/1789679901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa015e800 con 0x7fca8c0047c0 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fca9f7fe640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcaa01f1960 0x7fcaa01f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fca9effd640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca8c0047c0 0x7fcaa015f8a0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fca9000f820 tx=0x7fca9000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fca977fe640 1 -- 172.21.15.47:0/1789679901 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca900050d0 con 0x7fca8c0047c0 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fca977fe640 1 -- 172.21.15.47:0/1789679901 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca90005230 con 0x7fca8c0047c0 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fcaa55ea640 1 -- 172.21.15.47:0/1789679901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa015eae0 con 0x7fca8c0047c0 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fca977fe640 1 -- 172.21.15.47:0/1789679901 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9000d730 con 0x7fca8c0047c0 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.243+0000 7fcaa55ea640 1 -- 172.21.15.47:0/1789679901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa015f0a0 con 0x7fca8c0047c0 2024-03-23T09:49:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.244+0000 7fcaa55ea640 1 -- 172.21.15.47:0/1789679901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa0072360 con 0x7fca8c0047c0 2024-03-23T09:49:30.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.244+0000 7fca977fe640 1 -- 172.21.15.47:0/1789679901 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fca9000d890 con 0x7fca8c0047c0 2024-03-23T09:49:30.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.245+0000 7fca977fe640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca8006ac30 0x7fca8006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.245+0000 7fca977fe640 1 -- 172.21.15.47:0/1789679901 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fca90089290 con 0x7fca8c0047c0 2024-03-23T09:49:30.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.245+0000 7fca97fff640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca8006ac30 0x7fca8006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.246+0000 7fca97fff640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca8006ac30 0x7fca8006d0f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fca88036000 tx=0x7fca88030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:30.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.249+0000 7fca977fe640 1 -- 172.21.15.47:0/1789679901 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fca90054410 con 0x7fca8c0047c0 2024-03-23T09:49:30.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.429+0000 7fcaa55ea640 1 -- 172.21.15.47:0/1789679901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcaa0066be0 con 0x7fca8c0047c0 2024-03-23T09:49:30.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.429+0000 7fca977fe640 1 -- 172.21.15.47:0/1789679901 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca90057a30 con 0x7fca8c0047c0 2024-03-23T09:49:30.429 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:49:30.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.431+0000 7fcaa55ea640 1 -- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca8006ac30 msgr2=0x7fca8006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.431+0000 7fcaa55ea640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca8006ac30 0x7fca8006d0f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fca88036000 tx=0x7fca88030000 comp rx=0 tx=0).stop 2024-03-23T09:49:30.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.431+0000 7fcaa55ea640 1 -- 172.21.15.47:0/1789679901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca8c0047c0 msgr2=0x7fcaa015f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.431+0000 7fcaa55ea640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca8c0047c0 0x7fcaa015f8a0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fca9000f820 tx=0x7fca9000fce0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.431+0000 7fcaa55ea640 1 -- 172.21.15.47:0/1789679901 shutdown_connections 2024-03-23T09:49:30.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.431+0000 7fcaa55ea640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca8006ac30 0x7fca8006d0f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.431+0000 7fcaa55ea640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcaa01f1960 0x7fcaa01f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.431+0000 7fcaa55ea640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaa015e290 0x7fcaa01f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.431+0000 7fcaa55ea640 1 --2- 172.21.15.47:0/1789679901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca8c0047c0 0x7fcaa015f8a0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.431+0000 7fcaa55ea640 1 -- 172.21.15.47:0/1789679901 >> 172.21.15.47:0/1789679901 conn(0x7fcaa00659d0 msgr2=0x7fcaa00b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:30.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.431+0000 7fcaa55ea640 1 -- 172.21.15.47:0/1789679901 shutdown_connections 2024-03-23T09:49:30.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.431+0000 7fcaa55ea640 1 -- 172.21.15.47:0/1789679901 wait complete. 2024-03-23T09:49:30.443 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:49:30.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.544+0000 7fba820e9640 1 Processor -- start 2024-03-23T09:49:30.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.544+0000 7fba820e9640 1 -- start start 2024-03-23T09:49:30.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.544+0000 7fba820e9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba7c0b8300 0x7fba7c0cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.544+0000 7fba820e9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7c0d0ee0 con 0x7fba7c15cb40 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.544+0000 7fba820e9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7c0cfb20 con 0x7fba7c0cf6d0 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.544+0000 7fba820e9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c0b5a10 con 0x7fba7c0b8300 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7b7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fba7c15cb40 0x7fba7c160770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59254/0 (socket says 172.21.15.47:59254) 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7b7fe640 1 -- 172.21.15.47:0/2077393184 learned_addr learned my addr 172.21.15.47:0/2077393184 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89555778 0 0) 0x7fba7c0d0ee0 con 0x7fba7c15cb40 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7affd640 1 --2- 172.21.15.47:0/2077393184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba7c0b8300 0x7fba7c0cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba64003520 con 0x7fba7c15cb40 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3446944144 0 0) 0x7fba64003520 con 0x7fba7c15cb40 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba7c0b8300 msgr2=0x7fba7c0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 --2- 172.21.15.47:0/2077393184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba7c0b8300 0x7fba7c0cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 >> v1:172.21.15.175:6789/0 conn(0x7fba7c0cf6d0 legacy=0x7fba7c0d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c1616f0 con 0x7fba7c15cb40 2024-03-23T09:49:30.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fba6c001670 con 0x7fba7c15cb40 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 >> v1:172.21.15.47:6789/0 conn(0x7fba7c15cb40 legacy=0x7fba7c160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 --2- 172.21.15.47:0/2077393184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba640039a0 0x7fba64003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 --2- 172.21.15.47:0/2077393184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba64004350 0x7fba64004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 --2- 172.21.15.47:0/2077393184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba7c0b8300 0x7fba64005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c0b5a10 con 0x7fba64004350 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6400b9b0 con 0x7fba640039a0 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6400bb70 con 0x7fba7c0b8300 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7affd640 1 --2- 172.21.15.47:0/2077393184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba64004350 0x7fba64004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7b7fe640 1 --2- 172.21.15.47:0/2077393184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba640039a0 0x7fba64003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.545+0000 7fba7bfff640 1 --2- 172.21.15.47:0/2077393184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba7c0b8300 0x7fba64005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.546+0000 7fba7affd640 1 -- 172.21.15.47:0/2077393184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba7c0b8300 msgr2=0x7fba64005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.546+0000 7fba7affd640 1 --2- 172.21.15.47:0/2077393184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba7c0b8300 0x7fba64005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.546+0000 7fba7affd640 1 -- 172.21.15.47:0/2077393184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba640039a0 msgr2=0x7fba64003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.546+0000 7fba7affd640 1 --2- 172.21.15.47:0/2077393184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba640039a0 0x7fba64003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.546+0000 7fba7affd640 1 -- 172.21.15.47:0/2077393184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba64003520 con 0x7fba64004350 2024-03-23T09:49:30.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.546+0000 7fba7bfff640 1 --2- 172.21.15.47:0/2077393184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba7c0b8300 0x7fba64005080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:30.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.546+0000 7fba7affd640 1 --2- 172.21.15.47:0/2077393184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba64004350 0x7fba64004820 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fba7c06c3f0 tx=0x7fba640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5671a80f25dba09a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:30.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.546+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7002c660 con 0x7fba64004350 2024-03-23T09:49:30.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.546+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7002c7c0 con 0x7fba64004350 2024-03-23T09:49:30.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.546+0000 7fba7a7fc640 1 -- 172.21.15.47:0/2077393184 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7002cb70 con 0x7fba64004350 2024-03-23T09:49:30.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.547+0000 7fba820e9640 1 -- 172.21.15.47:0/2077393184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba64004350 msgr2=0x7fba64004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.547+0000 7fba820e9640 1 --2- 172.21.15.47:0/2077393184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba64004350 0x7fba64004820 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fba7c06c3f0 tx=0x7fba640057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.547+0000 7fba7bfff640 1 -- 172.21.15.47:0/2077393184 reap_dead start 2024-03-23T09:49:30.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.547+0000 7fba820e9640 1 -- 172.21.15.47:0/2077393184 shutdown_connections 2024-03-23T09:49:30.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.547+0000 7fba820e9640 1 -- 172.21.15.47:0/2077393184 >> 172.21.15.47:0/2077393184 conn(0x7fba7c06b160 msgr2=0x7fba7c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:30.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.547+0000 7fba820e9640 1 -- 172.21.15.47:0/2077393184 shutdown_connections 2024-03-23T09:49:30.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.547+0000 7fba820e9640 1 -- 172.21.15.47:0/2077393184 wait complete. 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.548+0000 7fba820e9640 1 Processor -- start 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.548+0000 7fba820e9640 1 -- start start 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba820e9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba64004350 0x7fba7c14d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba820e9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba7c0b6fd0 0x7fba7c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba820e9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba7c15f410 0x7fba7c0cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba820e9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c159010 con 0x7fba7c0b6fd0 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba820e9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c1592b0 con 0x7fba7c15f410 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba820e9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c159590 con 0x7fba64004350 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba7affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba7c0b6fd0 0x7fba7c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba7affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba7c0b6fd0 0x7fba7c159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37572/0 (socket says 172.21.15.47:37572) 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba7affd640 1 -- 172.21.15.47:0/3984161942 learned_addr learned my addr 172.21.15.47:0/3984161942 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba7bfff640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba7c15f410 0x7fba7c0cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba7b7fe640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba64004350 0x7fba7c14d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba7affd640 1 -- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba64004350 msgr2=0x7fba7c14d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba7affd640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba64004350 0x7fba7c14d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba7affd640 1 -- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba7c15f410 msgr2=0x7fba7c0cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba7affd640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba7c15f410 0x7fba7c0cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba7affd640 1 -- 172.21.15.47:0/3984161942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c159870 con 0x7fba7c0b6fd0 2024-03-23T09:49:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba7bfff640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba7c15f410 0x7fba7c0cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.549+0000 7fba7b7fe640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba64004350 0x7fba7c14d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.550+0000 7fba7affd640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba7c0b6fd0 0x7fba7c159d90 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fba70009b80 tx=0x7fba70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.550+0000 7fba63fff640 1 -- 172.21.15.47:0/3984161942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70034a10 con 0x7fba7c0b6fd0 2024-03-23T09:49:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.550+0000 7fba63fff640 1 -- 172.21.15.47:0/3984161942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba70034b70 con 0x7fba7c0b6fd0 2024-03-23T09:49:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.550+0000 7fba820e9640 1 -- 172.21.15.47:0/3984161942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7c159010 con 0x7fba7c0b6fd0 2024-03-23T09:49:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.550+0000 7fba63fff640 1 -- 172.21.15.47:0/3984161942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7003e400 con 0x7fba7c0b6fd0 2024-03-23T09:49:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.550+0000 7fba820e9640 1 -- 172.21.15.47:0/3984161942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7c15a2d0 con 0x7fba7c0b6fd0 2024-03-23T09:49:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.551+0000 7fba820e9640 1 -- 172.21.15.47:0/3984161942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7c072360 con 0x7fba7c0b6fd0 2024-03-23T09:49:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.551+0000 7fba63fff640 1 -- 172.21.15.47:0/3984161942 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fba7003e560 con 0x7fba7c0b6fd0 2024-03-23T09:49:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.552+0000 7fba63fff640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba6806ac00 0x7fba6806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.552+0000 7fba63fff640 1 -- 172.21.15.47:0/3984161942 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fba700ab3e0 con 0x7fba7c0b6fd0 2024-03-23T09:49:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.552+0000 7fba7b7fe640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba6806ac00 0x7fba6806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.552+0000 7fba7b7fe640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba6806ac00 0x7fba6806d0c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fba6c001eb0 tx=0x7fba6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:30.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.556+0000 7fba63fff640 1 -- 172.21.15.47:0/3984161942 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fba700764f0 con 0x7fba7c0b6fd0 2024-03-23T09:49:30.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.771+0000 7fba820e9640 1 -- 172.21.15.47:0/3984161942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba7c159a00 con 0x7fba7c0b6fd0 2024-03-23T09:49:30.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.771+0000 7fba63fff640 1 -- 172.21.15.47:0/3984161942 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fba70079b10 con 0x7fba7c0b6fd0 2024-03-23T09:49:30.772 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:30.772 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:48:52.314987+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6835/3671387340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3671387340},{"type":"v1","addr":"172.21.15.47:6835","nonce":3671387340}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:49:30.772 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 59 2024-03-23T09:49:30.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.774+0000 7fba820e9640 1 -- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba6806ac00 msgr2=0x7fba6806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.774+0000 7fba820e9640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba6806ac00 0x7fba6806d0c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fba6c001eb0 tx=0x7fba6c009040 comp rx=0 tx=0).stop 2024-03-23T09:49:30.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.774+0000 7fba820e9640 1 -- 172.21.15.47:0/3984161942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba7c0b6fd0 msgr2=0x7fba7c159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.774+0000 7fba820e9640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba7c0b6fd0 0x7fba7c159d90 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fba70009b80 tx=0x7fba70004730 comp rx=0 tx=0).stop 2024-03-23T09:49:30.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.774+0000 7fba820e9640 1 -- 172.21.15.47:0/3984161942 shutdown_connections 2024-03-23T09:49:30.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.774+0000 7fba820e9640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fba6806ac00 0x7fba6806d0c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.774+0000 7fba820e9640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba7c15f410 0x7fba7c0cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.774+0000 7fba820e9640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba7c0b6fd0 0x7fba7c159d90 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.774+0000 7fba820e9640 1 --2- 172.21.15.47:0/3984161942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba64004350 0x7fba7c14d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.774+0000 7fba820e9640 1 -- 172.21.15.47:0/3984161942 >> 172.21.15.47:0/3984161942 conn(0x7fba7c06b160 msgr2=0x7fba7c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:30.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.774+0000 7fba820e9640 1 -- 172.21.15.47:0/3984161942 shutdown_connections 2024-03-23T09:49:30.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.774+0000 7fba820e9640 1 -- 172.21.15.47:0/3984161942 wait complete. 2024-03-23T09:49:30.785 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:49:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.886+0000 7f93f091b640 1 Processor -- start 2024-03-23T09:49:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.886+0000 7f93f091b640 1 -- start start 2024-03-23T09:49:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.887+0000 7f93f091b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f93ec0b8350 0x7f93ec0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.887+0000 7f93f091b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f93ec0cdc40 0x7f93ec0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.887+0000 7f93f091b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93ec0ce090 con 0x7f93ec15ca90 2024-03-23T09:49:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.887+0000 7f93f091b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ec0d11e0 con 0x7f93ec0cdc40 2024-03-23T09:49:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.887+0000 7f93f091b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ec160f60 con 0x7f93ec0b8350 2024-03-23T09:49:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.887+0000 7f93ead9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f93ec0cdc40 0x7f93ec0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.887+0000 7f93ead9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f93ec0cdc40 0x7f93ec0d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37586/0 (socket says 172.21.15.47:37586) 2024-03-23T09:49:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.887+0000 7f93ead9d640 1 -- 172.21.15.47:0/2026061723 learned_addr learned my addr 172.21.15.47:0/2026061723 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.887+0000 7f93ead9d640 1 -- 172.21.15.47:0/2026061723 >> v2:172.21.15.175:3300/0 conn(0x7f93ec0b8350 msgr2=0x7f93ec0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.887+0000 7f93ead9d640 1 --2- 172.21.15.47:0/2026061723 >> v2:172.21.15.175:3300/0 conn(0x7f93ec0b8350 0x7f93ec0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.887+0000 7f93ead9d640 1 -- 172.21.15.47:0/2026061723 >> v1:172.21.15.175:6789/0 conn(0x7f93ec15ca90 legacy=0x7f93ec160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.887+0000 7f93ead9d640 1 -- 172.21.15.47:0/2026061723 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec161240 con 0x7f93ec0cdc40 2024-03-23T09:49:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.888+0000 7f93ead9d640 1 --2- 172.21.15.47:0/2026061723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec0cdc40 0x7f93ec0d1670 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f93ec160670 tx=0x7f93dc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e990438dd648b11d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.888+0000 7f93e959a640 1 -- 172.21.15.47:0/2026061723 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc02c9f0 con 0x7f93ec0cdc40 2024-03-23T09:49:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.888+0000 7f93e959a640 1 -- 172.21.15.47:0/2026061723 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc02cb50 con 0x7f93ec0cdc40 2024-03-23T09:49:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.888+0000 7f93e959a640 1 -- 172.21.15.47:0/2026061723 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0343f0 con 0x7f93ec0cdc40 2024-03-23T09:49:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.889+0000 7f93f091b640 1 -- 172.21.15.47:0/2026061723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec0cdc40 msgr2=0x7f93ec0d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.889+0000 7f93f091b640 1 --2- 172.21.15.47:0/2026061723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec0cdc40 0x7f93ec0d1670 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f93ec160670 tx=0x7f93dc02be50 comp rx=0 tx=0).stop 2024-03-23T09:49:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.889+0000 7f93f091b640 1 -- 172.21.15.47:0/2026061723 shutdown_connections 2024-03-23T09:49:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.889+0000 7f93f091b640 1 --2- 172.21.15.47:0/2026061723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec0cdc40 0x7f93ec0d1670 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.889+0000 7f93f091b640 1 --2- 172.21.15.47:0/2026061723 >> v2:172.21.15.175:3300/0 conn(0x7f93ec0b8350 0x7f93ec0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.889+0000 7f93f091b640 1 -- 172.21.15.47:0/2026061723 >> 172.21.15.47:0/2026061723 conn(0x7f93ec0659d0 msgr2=0x7f93ec065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.889+0000 7f93f091b640 1 -- 172.21.15.47:0/2026061723 shutdown_connections 2024-03-23T09:49:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.889+0000 7f93f091b640 1 -- 172.21.15.47:0/2026061723 wait complete. 2024-03-23T09:49:30.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.890+0000 7f93f091b640 1 Processor -- start 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.890+0000 7f93f091b640 1 -- start start 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93f091b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ec0b8350 0x7f93ec0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93f091b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93ec0cdc40 0x7f93ec15a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93f091b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec15ca90 0x7f93ec0b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93f091b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec14b860 con 0x7f93ec15ca90 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93f091b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec15aea0 con 0x7f93ec0b8350 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93f091b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec15b0f0 con 0x7f93ec0cdc40 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93ead9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec15ca90 0x7f93ec0b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93ead9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec15ca90 0x7f93ec0b6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37596/0 (socket says 172.21.15.47:37596) 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93ead9d640 1 -- 172.21.15.47:0/3040760938 learned_addr learned my addr 172.21.15.47:0/3040760938 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93ead9d640 1 -- 172.21.15.47:0/3040760938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93ec0cdc40 msgr2=0x7f93ec15a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93ead9d640 1 --2- 172.21.15.47:0/3040760938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93ec0cdc40 0x7f93ec15a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93ea59c640 1 --2- 172.21.15.47:0/3040760938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ec0b8350 0x7f93ec0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93ead9d640 1 -- 172.21.15.47:0/3040760938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ec0b8350 msgr2=0x7f93ec0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93ead9d640 1 --2- 172.21.15.47:0/3040760938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ec0b8350 0x7f93ec0d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93ead9d640 1 -- 172.21.15.47:0/3040760938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc009b30 con 0x7f93ec15ca90 2024-03-23T09:49:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.891+0000 7f93ead9d640 1 --2- 172.21.15.47:0/3040760938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec15ca90 0x7f93ec0b6e20 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f93dc036040 tx=0x7f93dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.892+0000 7f93e2ffd640 1 -- 172.21.15.47:0/3040760938 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc03e450 con 0x7f93ec15ca90 2024-03-23T09:49:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.892+0000 7f93e2ffd640 1 -- 172.21.15.47:0/3040760938 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc03e5b0 con 0x7f93ec15ca90 2024-03-23T09:49:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.892+0000 7f93f091b640 1 -- 172.21.15.47:0/3040760938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ec15b3d0 con 0x7f93ec15ca90 2024-03-23T09:49:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.892+0000 7f93e2ffd640 1 -- 172.21.15.47:0/3040760938 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc03e960 con 0x7f93ec15ca90 2024-03-23T09:49:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.892+0000 7f93f091b640 1 -- 172.21.15.47:0/3040760938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec15b990 con 0x7f93ec15ca90 2024-03-23T09:49:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.893+0000 7f93f091b640 1 -- 172.21.15.47:0/3040760938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ec072360 con 0x7f93ec15ca90 2024-03-23T09:49:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.893+0000 7f93e2ffd640 1 -- 172.21.15.47:0/3040760938 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f93dc03eac0 con 0x7f93ec15ca90 2024-03-23T09:49:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.893+0000 7f93e2ffd640 1 --2- 172.21.15.47:0/3040760938 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f93cc06ac30 0x7f93cc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.893+0000 7f93e2ffd640 1 -- 172.21.15.47:0/3040760938 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f93dc0abe70 con 0x7f93ec15ca90 2024-03-23T09:49:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.894+0000 7f93ea59c640 1 --2- 172.21.15.47:0/3040760938 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f93cc06ac30 0x7f93cc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.894+0000 7f93ea59c640 1 --2- 172.21.15.47:0/3040760938 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f93cc06ac30 0x7f93cc06d0f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f93e40015d0 tx=0x7f93e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:30.898+0000 7f93e2ffd640 1 -- 172.21.15.47:0/3040760938 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f93dc076ff0 con 0x7f93ec15ca90 2024-03-23T09:49:31.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.080+0000 7f93f091b640 1 -- 172.21.15.47:0/3040760938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93ec0ce090 con 0x7f93ec15ca90 2024-03-23T09:49:31.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.081+0000 7f93e2ffd640 1 -- 172.21.15.47:0/3040760938 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16449 (secure 0 0 0) 0x7f93ec0ce090 con 0x7f93ec15ca90 2024-03-23T09:49:31.081 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:31.081 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":89,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:48:48.285538+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:47:09.722391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-23T09:47:10.354054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.30","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6834/4278516621":"2024-03-24T09:48:48.285514+0000","172.21.15.47:6835/4278516621":"2024-03-24T09:48:48.285514+0000","172.21.15.47:6835/1927629460":"2024-03-24T09:47:28.505920+0000","172.21.15.47:6834/1927629460":"2024-03-24T09:47:28.505920+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:49:31.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.082+0000 7f93f091b640 1 -- 172.21.15.47:0/3040760938 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f93cc06ac30 msgr2=0x7f93cc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.082+0000 7f93f091b640 1 --2- 172.21.15.47:0/3040760938 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f93cc06ac30 0x7f93cc06d0f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f93e40015d0 tx=0x7f93e4009040 comp rx=0 tx=0).stop 2024-03-23T09:49:31.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.082+0000 7f93f091b640 1 -- 172.21.15.47:0/3040760938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec15ca90 msgr2=0x7f93ec0b6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.082+0000 7f93f091b640 1 --2- 172.21.15.47:0/3040760938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec15ca90 0x7f93ec0b6e20 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f93dc036040 tx=0x7f93dc004a40 comp rx=0 tx=0).stop 2024-03-23T09:49:31.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.083+0000 7f93f091b640 1 -- 172.21.15.47:0/3040760938 shutdown_connections 2024-03-23T09:49:31.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.083+0000 7f93f091b640 1 --2- 172.21.15.47:0/3040760938 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f93cc06ac30 0x7f93cc06d0f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.083+0000 7f93f091b640 1 --2- 172.21.15.47:0/3040760938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec15ca90 0x7f93ec0b6e20 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.083+0000 7f93f091b640 1 --2- 172.21.15.47:0/3040760938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93ec0cdc40 0x7f93ec15a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.083+0000 7f93f091b640 1 --2- 172.21.15.47:0/3040760938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ec0b8350 0x7f93ec0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.083+0000 7f93f091b640 1 -- 172.21.15.47:0/3040760938 >> 172.21.15.47:0/3040760938 conn(0x7f93ec0659d0 msgr2=0x7f93ec15d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:31.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.083+0000 7f93f091b640 1 -- 172.21.15.47:0/3040760938 shutdown_connections 2024-03-23T09:49:31.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.083+0000 7f93f091b640 1 -- 172.21.15.47:0/3040760938 wait complete. 2024-03-23T09:49:31.094 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T09:49:31.094 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:49:31.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.194+0000 7fdbc45d2640 1 Processor -- start 2024-03-23T09:49:31.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.194+0000 7fdbc45d2640 1 -- start start 2024-03-23T09:49:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.195+0000 7fdbc45d2640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdbbc0ce560 0x7fdbbc0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.195+0000 7fdbc45d2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbbc0cebf0 con 0x7fdbbc06a330 2024-03-23T09:49:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.195+0000 7fdbc45d2640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbbc0ce9c0 con 0x7fdbbc160610 2024-03-23T09:49:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.195+0000 7fdbc45d2640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbbc1614d0 con 0x7fdbbc0ce560 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc2b6f640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdbbc06a330 0x7fdbbc06a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59270/0 (socket says 172.21.15.47:59270) 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc2b6f640 1 -- 172.21.15.47:0/525036581 learned_addr learned my addr 172.21.15.47:0/525036581 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819492853 0 0) 0x7fdbbc0cebf0 con 0x7fdbbc06a330 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdba8003520 con 0x7fdbbc06a330 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc1b6d640 1 --2- 172.21.15.47:0/525036581 >> v2:172.21.15.175:3300/0 conn(0x7fdbbc0ce560 0x7fdbbc0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 387358926 0 0) 0x7fdba8003520 con 0x7fdbbc06a330 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 >> v2:172.21.15.175:3300/0 conn(0x7fdbbc0ce560 msgr2=0x7fdbbc0ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 --2- 172.21.15.47:0/525036581 >> v2:172.21.15.175:3300/0 conn(0x7fdbbc0ce560 0x7fdbbc0ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 >> v1:172.21.15.175:6789/0 conn(0x7fdbbc160610 legacy=0x7fdbbc1609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc069ea0 con 0x7fdbbc06a330 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fdbb40019f0 con 0x7fdbbc06a330 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 >> v1:172.21.15.47:6789/0 conn(0x7fdbbc06a330 legacy=0x7fdbbc06a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 --2- 172.21.15.47:0/525036581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdba8003970 0x7fdba8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 --2- 172.21.15.47:0/525036581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdba80047c0 0x7fdba8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 --2- 172.21.15.47:0/525036581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdba80052a0 0x7fdba800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba800c140 con 0x7fdba8003970 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba800c2b0 con 0x7fdba80052a0 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba800c470 con 0x7fdba80047c0 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc236e640 1 --2- 172.21.15.47:0/525036581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdba8003970 0x7fdba8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc1b6d640 1 --2- 172.21.15.47:0/525036581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdba80047c0 0x7fdba8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.196+0000 7fdbc2b6f640 1 --2- 172.21.15.47:0/525036581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdba80052a0 0x7fdba800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.197+0000 7fdbc236e640 1 -- 172.21.15.47:0/525036581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdba80047c0 msgr2=0x7fdba8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.197+0000 7fdbc236e640 1 --2- 172.21.15.47:0/525036581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdba80047c0 0x7fdba8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.197+0000 7fdbc236e640 1 -- 172.21.15.47:0/525036581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdba80052a0 msgr2=0x7fdba800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.197+0000 7fdbc236e640 1 --2- 172.21.15.47:0/525036581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdba80052a0 0x7fdba800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.197+0000 7fdbc2b6f640 1 -- 172.21.15.47:0/525036581 reap_dead start 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.197+0000 7fdbc236e640 1 -- 172.21.15.47:0/525036581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba8003520 con 0x7fdba8003970 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.197+0000 7fdbc236e640 1 --2- 172.21.15.47:0/525036581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdba8003970 0x7fdba8003d90 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8001720 tx=0x7fdba80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a812c43a30b39d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.198+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80052d0 con 0x7fdba8003970 2024-03-23T09:49:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.198+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb802e050 con 0x7fdba8003970 2024-03-23T09:49:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.198+0000 7fdbc136c640 1 -- 172.21.15.47:0/525036581 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8003830 con 0x7fdba8003970 2024-03-23T09:49:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.198+0000 7fdbc45d2640 1 -- 172.21.15.47:0/525036581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdba8003970 msgr2=0x7fdba8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.198+0000 7fdbc45d2640 1 --2- 172.21.15.47:0/525036581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdba8003970 0x7fdba8003d90 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8001720 tx=0x7fdba80057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.198+0000 7fdbc45d2640 1 -- 172.21.15.47:0/525036581 shutdown_connections 2024-03-23T09:49:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.198+0000 7fdbc45d2640 1 --2- 172.21.15.47:0/525036581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdba8003970 0x7fdba8003d90 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.198+0000 7fdbc45d2640 1 -- 172.21.15.47:0/525036581 >> 172.21.15.47:0/525036581 conn(0x7fdbbc0ced80 msgr2=0x7fdbbc0cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.198+0000 7fdbc45d2640 1 -- 172.21.15.47:0/525036581 shutdown_connections 2024-03-23T09:49:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.198+0000 7fdbc45d2640 1 -- 172.21.15.47:0/525036581 wait complete. 2024-03-23T09:49:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.199+0000 7fdbc45d2640 1 Processor -- start 2024-03-23T09:49:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.199+0000 7fdbc45d2640 1 -- start start 2024-03-23T09:49:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc45d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdba8003970 0x7fdbbc152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc45d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbbc15f7c0 0x7fdbbc15d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc45d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbbc15df00 0x7fdbbc1f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc45d2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc15e720 con 0x7fdba8003970 2024-03-23T09:49:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc45d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc15ea00 con 0x7fdbbc15df00 2024-03-23T09:49:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc45d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc15ece0 con 0x7fdbbc15f7c0 2024-03-23T09:49:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc236e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdba8003970 0x7fdbbc152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc236e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdba8003970 0x7fdbbc152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37604/0 (socket says 172.21.15.47:37604) 2024-03-23T09:49:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc236e640 1 -- 172.21.15.47:0/895611766 learned_addr learned my addr 172.21.15.47:0/895611766 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc1b6d640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbbc15f7c0 0x7fdbbc15d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc2b6f640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbbc15df00 0x7fdbbc1f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc236e640 1 -- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbbc15f7c0 msgr2=0x7fdbbc15d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc236e640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbbc15f7c0 0x7fdbbc15d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc236e640 1 -- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbbc15df00 msgr2=0x7fdbbc1f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc236e640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbbc15df00 0x7fdbbc1f21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.200+0000 7fdbc236e640 1 -- 172.21.15.47:0/895611766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc15efc0 con 0x7fdba8003970 2024-03-23T09:49:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.201+0000 7fdbc236e640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdba8003970 0x7fdbbc152cf0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8005040 tx=0x7fdbb8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.201+0000 7fdbb2ffd640 1 -- 172.21.15.47:0/895611766 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80076c0 con 0x7fdba8003970 2024-03-23T09:49:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.201+0000 7fdbc45d2640 1 -- 172.21.15.47:0/895611766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbbc15f2a0 con 0x7fdba8003970 2024-03-23T09:49:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.201+0000 7fdbb2ffd640 1 -- 172.21.15.47:0/895611766 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb8007a30 con 0x7fdba8003970 2024-03-23T09:49:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.201+0000 7fdbb2ffd640 1 -- 172.21.15.47:0/895611766 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8006be0 con 0x7fdba8003970 2024-03-23T09:49:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.201+0000 7fdbc45d2640 1 -- 172.21.15.47:0/895611766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbbc15e380 con 0x7fdba8003970 2024-03-23T09:49:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.202+0000 7fdbc45d2640 1 -- 172.21.15.47:0/895611766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc066be0 con 0x7fdba8003970 2024-03-23T09:49:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.202+0000 7fdbb2ffd640 1 -- 172.21.15.47:0/895611766 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdbb8006d40 con 0x7fdba8003970 2024-03-23T09:49:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.202+0000 7fdbb2ffd640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdba006ac00 0x7fdba006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.203+0000 7fdbb2ffd640 1 -- 172.21.15.47:0/895611766 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fdbb80ab5a0 con 0x7fdba8003970 2024-03-23T09:49:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.203+0000 7fdbc1b6d640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdba006ac00 0x7fdba006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.203+0000 7fdbc1b6d640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdba006ac00 0x7fdba006d0c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc06b4a0 tx=0x7fdbac009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.207+0000 7fdbb2ffd640 1 -- 172.21.15.47:0/895611766 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdbb8076630 con 0x7fdba8003970 2024-03-23T09:49:31.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.423+0000 7fdbc45d2640 1 -- 172.21.15.47:0/895611766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbbc09cea0 con 0x7fdba8003970 2024-03-23T09:49:31.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.423+0000 7fdbb2ffd640 1 -- 172.21.15.47:0/895611766 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fdbb8079c50 con 0x7fdba8003970 2024-03-23T09:49:31.424 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:31.424 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:48:52.314987+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6835/3671387340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3671387340},{"type":"v1","addr":"172.21.15.47:6835","nonce":3671387340}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:49:31.424 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 59 2024-03-23T09:49:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.425+0000 7fdbc45d2640 1 -- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdba006ac00 msgr2=0x7fdba006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.425+0000 7fdbc45d2640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdba006ac00 0x7fdba006d0c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc06b4a0 tx=0x7fdbac009190 comp rx=0 tx=0).stop 2024-03-23T09:49:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.425+0000 7fdbc45d2640 1 -- 172.21.15.47:0/895611766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdba8003970 msgr2=0x7fdbbc152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.425+0000 7fdbc45d2640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdba8003970 0x7fdbbc152cf0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8005040 tx=0x7fdbb8001120 comp rx=0 tx=0).stop 2024-03-23T09:49:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.425+0000 7fdbc45d2640 1 -- 172.21.15.47:0/895611766 shutdown_connections 2024-03-23T09:49:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.425+0000 7fdbc45d2640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdba006ac00 0x7fdba006d0c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.425+0000 7fdbc45d2640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbbc15df00 0x7fdbbc1f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.425+0000 7fdbc45d2640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbbc15f7c0 0x7fdbbc15d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.425+0000 7fdbc45d2640 1 --2- 172.21.15.47:0/895611766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdba8003970 0x7fdbbc152cf0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.425+0000 7fdbc45d2640 1 -- 172.21.15.47:0/895611766 >> 172.21.15.47:0/895611766 conn(0x7fdbbc0ced80 msgr2=0x7fdbbc0add50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.425+0000 7fdbc45d2640 1 -- 172.21.15.47:0/895611766 shutdown_connections 2024-03-23T09:49:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.425+0000 7fdbc45d2640 1 -- 172.21.15.47:0/895611766 wait complete. 2024-03-23T09:49:31.436 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:49:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.539+0000 7fb806b00640 1 Processor -- start 2024-03-23T09:49:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.539+0000 7fb806b00640 1 -- start start 2024-03-23T09:49:31.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.539+0000 7fb806b00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb80015cb50 0x7fb8000b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb806b00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8000b72f0 con 0x7fb8000cd9e0 2024-03-23T09:49:31.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb806b00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8000b75d0 con 0x7fb8000cd5a0 2024-03-23T09:49:31.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb806b00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8000b7e70 con 0x7fb80015cb50 2024-03-23T09:49:31.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb80509d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb8000cd9e0 0x7fb8000b6a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59272/0 (socket says 172.21.15.47:59272) 2024-03-23T09:49:31.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb80509d640 1 -- 172.21.15.47:0/2053032736 learned_addr learned my addr 172.21.15.47:0/2053032736 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:31.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb80489c640 1 --2- 172.21.15.47:0/2053032736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb80015cb50 0x7fb8000b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167198471 0 0) 0x7fb8000b72f0 con 0x7fb8000cd9e0 2024-03-23T09:49:31.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f0003520 con 0x7fb8000cd9e0 2024-03-23T09:49:31.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969026357 0 0) 0x7fb8000b75d0 con 0x7fb8000cd5a0 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8000b72f0 con 0x7fb8000cd5a0 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1094547198 0 0) 0x7fb7f0003520 con 0x7fb8000cd9e0 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb80015cb50 msgr2=0x7fb8000b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 --2- 172.21.15.47:0/2053032736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb80015cb50 0x7fb8000b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 >> v1:172.21.15.175:6789/0 conn(0x7fb8000cd5a0 legacy=0x7fb8000d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8000b78b0 con 0x7fb8000cd9e0 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fb7f8001970 con 0x7fb8000cd9e0 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 >> v1:172.21.15.47:6789/0 conn(0x7fb8000cd9e0 legacy=0x7fb8000b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 --2- 172.21.15.47:0/2053032736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7f0003b30 0x7fb7f0003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 --2- 172.21.15.47:0/2053032736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb80015cb50 0x7fb7f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 --2- 172.21.15.47:0/2053032736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7f0004d00 0x7fb7f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8000b7e70 con 0x7fb7f0004d00 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f000ba30 con 0x7fb7f0003b30 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.540+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f000bbf0 con 0x7fb80015cb50 2024-03-23T09:49:31.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.541+0000 7fb80509d640 1 --2- 172.21.15.47:0/2053032736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7f0004d00 0x7fb7f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.541+0000 7fb80489c640 1 --2- 172.21.15.47:0/2053032736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7f0003b30 0x7fb7f0003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.541+0000 7fb7fffff640 1 --2- 172.21.15.47:0/2053032736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb80015cb50 0x7fb7f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.541+0000 7fb80509d640 1 -- 172.21.15.47:0/2053032736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb80015cb50 msgr2=0x7fb7f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.541+0000 7fb80509d640 1 --2- 172.21.15.47:0/2053032736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb80015cb50 0x7fb7f00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.541+0000 7fb80509d640 1 -- 172.21.15.47:0/2053032736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7f0003b30 msgr2=0x7fb7f0003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.541+0000 7fb80509d640 1 --2- 172.21.15.47:0/2053032736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7f0003b30 0x7fb7f0003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.541+0000 7fb80509d640 1 -- 172.21.15.47:0/2053032736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f0003520 con 0x7fb7f0004d00 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.542+0000 7fb80509d640 1 --2- 172.21.15.47:0/2053032736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7f0004d00 0x7fb7f0005180 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800a680 tx=0x7fb7f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9239dee8e1ce0be3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.542+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80072d0 con 0x7fb7f0004d00 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.542+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f802f070 con 0x7fb7f0004d00 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.542+0000 7fb7ff7fe640 1 -- 172.21.15.47:0/2053032736 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8007690 con 0x7fb7f0004d00 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.542+0000 7fb806b00640 1 -- 172.21.15.47:0/2053032736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7f0004d00 msgr2=0x7fb7f0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.542+0000 7fb806b00640 1 --2- 172.21.15.47:0/2053032736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7f0004d00 0x7fb7f0005180 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800a680 tx=0x7fb7f00057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.542+0000 7fb80509d640 1 -- 172.21.15.47:0/2053032736 reap_dead start 2024-03-23T09:49:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.542+0000 7fb806b00640 1 -- 172.21.15.47:0/2053032736 shutdown_connections 2024-03-23T09:49:31.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.542+0000 7fb806b00640 1 -- 172.21.15.47:0/2053032736 >> 172.21.15.47:0/2053032736 conn(0x7fb800065df0 msgr2=0x7fb800066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:31.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.542+0000 7fb806b00640 1 -- 172.21.15.47:0/2053032736 shutdown_connections 2024-03-23T09:49:31.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.542+0000 7fb806b00640 1 -- 172.21.15.47:0/2053032736 wait complete. 2024-03-23T09:49:31.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.543+0000 7fb806b00640 1 Processor -- start 2024-03-23T09:49:31.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.544+0000 7fb806b00640 1 -- start start 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.544+0000 7fb806b00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7f0004d00 0x7fb80015e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.544+0000 7fb806b00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8001f6230 0x7fb8000c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.544+0000 7fb806b00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80014d8f0 0x7fb80014dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.544+0000 7fb806b00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8000a92c0 con 0x7fb80014d8f0 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.544+0000 7fb806b00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80014e2b0 con 0x7fb8001f6230 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.544+0000 7fb806b00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8001f66b0 con 0x7fb7f0004d00 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb80509d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80014d8f0 0x7fb80014dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb80509d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80014d8f0 0x7fb80014dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37616/0 (socket says 172.21.15.47:37616) 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb80509d640 1 -- 172.21.15.47:0/3116041616 learned_addr learned my addr 172.21.15.47:0/3116041616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb7fffff640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8001f6230 0x7fb8000c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb80489c640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7f0004d00 0x7fb80015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb80509d640 1 -- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7f0004d00 msgr2=0x7fb80015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb80509d640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7f0004d00 0x7fb80015e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb80509d640 1 -- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8001f6230 msgr2=0x7fb8000c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb80509d640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8001f6230 0x7fb8000c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb80509d640 1 -- 172.21.15.47:0/3116041616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80015f010 con 0x7fb80014d8f0 2024-03-23T09:49:31.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb80509d640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80014d8f0 0x7fb80014dd70 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fb7f802ca80 tx=0x7fb7f8001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:31.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb7fd7fa640 1 -- 172.21.15.47:0/3116041616 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8004d60 con 0x7fb80014d8f0 2024-03-23T09:49:31.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb7fd7fa640 1 -- 172.21.15.47:0/3116041616 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f80070d0 con 0x7fb80014d8f0 2024-03-23T09:49:31.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb806b00640 1 -- 172.21.15.47:0/3116041616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80015f2f0 con 0x7fb80014d8f0 2024-03-23T09:49:31.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb7fd7fa640 1 -- 172.21.15.47:0/3116041616 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8007480 con 0x7fb80014d8f0 2024-03-23T09:49:31.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.545+0000 7fb806b00640 1 -- 172.21.15.47:0/3116041616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80015f8b0 con 0x7fb80014d8f0 2024-03-23T09:49:31.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.546+0000 7fb806b00640 1 -- 172.21.15.47:0/3116041616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb800072360 con 0x7fb80014d8f0 2024-03-23T09:49:31.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.547+0000 7fb7fd7fa640 1 -- 172.21.15.47:0/3116041616 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb7f8031030 con 0x7fb80014d8f0 2024-03-23T09:49:31.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.547+0000 7fb7fd7fa640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb7e406ac00 0x7fb7e406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.547+0000 7fb7fd7fa640 1 -- 172.21.15.47:0/3116041616 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fb7f80ab030 con 0x7fb80014d8f0 2024-03-23T09:49:31.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.547+0000 7fb80489c640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb7e406ac00 0x7fb7e406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.548+0000 7fb80489c640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb7e406ac00 0x7fb7e406d0c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009bc0 tx=0x7fb7f4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:31.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.551+0000 7fb7fd7fa640 1 -- 172.21.15.47:0/3116041616 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb7f8076120 con 0x7fb80014d8f0 2024-03-23T09:49:31.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.766+0000 7fb806b00640 1 -- 172.21.15.47:0/3116041616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb800072360 con 0x7fb80014d8f0 2024-03-23T09:49:31.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.766+0000 7fb7fd7fa640 1 -- 172.21.15.47:0/3116041616 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fb7f8079740 con 0x7fb80014d8f0 2024-03-23T09:49:31.767 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:47:11.390397+0000","modified":"2024-03-23T09:48:52.314987+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6835/3671387340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3671387340},{"type":"v1","addr":"172.21.15.47:6835","nonce":3671387340}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-23T09:49:31.767 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 59 2024-03-23T09:49:31.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.768+0000 7fb806b00640 1 -- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb7e406ac00 msgr2=0x7fb7e406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.768+0000 7fb806b00640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb7e406ac00 0x7fb7e406d0c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009bc0 tx=0x7fb7f4009190 comp rx=0 tx=0).stop 2024-03-23T09:49:31.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.768+0000 7fb806b00640 1 -- 172.21.15.47:0/3116041616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80014d8f0 msgr2=0x7fb80014dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.768+0000 7fb806b00640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80014d8f0 0x7fb80014dd70 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fb7f802ca80 tx=0x7fb7f8001c30 comp rx=0 tx=0).stop 2024-03-23T09:49:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.768+0000 7fb806b00640 1 -- 172.21.15.47:0/3116041616 shutdown_connections 2024-03-23T09:49:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.768+0000 7fb806b00640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb7e406ac00 0x7fb7e406d0c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.768+0000 7fb806b00640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80014d8f0 0x7fb80014dd70 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.768+0000 7fb806b00640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8001f6230 0x7fb8000c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.768+0000 7fb806b00640 1 --2- 172.21.15.47:0/3116041616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7f0004d00 0x7fb80015e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.768+0000 7fb806b00640 1 -- 172.21.15.47:0/3116041616 >> 172.21.15.47:0/3116041616 conn(0x7fb800065df0 msgr2=0x7fb8000d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.768+0000 7fb806b00640 1 -- 172.21.15.47:0/3116041616 shutdown_connections 2024-03-23T09:49:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.768+0000 7fb806b00640 1 -- 172.21.15.47:0/3116041616 wait complete. 2024-03-23T09:49:31.779 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:49:31.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.880+0000 7f5bd2de7640 1 Processor -- start 2024-03-23T09:49:31.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.880+0000 7f5bd2de7640 1 -- start start 2024-03-23T09:49:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.881+0000 7f5bd2de7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5bcc15cb50 0x7f5bcc0b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.881+0000 7f5bd2de7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bcc0cd5a0 0x7f5bcc0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.881+0000 7f5bd2de7640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bcc0cdf20 con 0x7f5bcc0d1010 2024-03-23T09:49:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.881+0000 7f5bd2de7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bcc0b7e70 con 0x7f5bcc15cb50 2024-03-23T09:49:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.881+0000 7f5bd2de7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc15cf30 con 0x7f5bcc0cd5a0 2024-03-23T09:49:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.881+0000 7f5bd0b83640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5bcc15cb50 0x7f5bcc0b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.881+0000 7f5bd0b83640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5bcc15cb50 0x7f5bcc0b84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37626/0 (socket says 172.21.15.47:37626) 2024-03-23T09:49:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.881+0000 7f5bd0b83640 1 -- 172.21.15.47:0/4122248230 learned_addr learned my addr 172.21.15.47:0/4122248230 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.881+0000 7f5bcbfff640 1 --2- 172.21.15.47:0/4122248230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bcc0cd5a0 0x7f5bcc0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.881+0000 7f5bd0b83640 1 -- 172.21.15.47:0/4122248230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bcc0cd5a0 msgr2=0x7f5bcc0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.881+0000 7f5bd0b83640 1 --2- 172.21.15.47:0/4122248230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bcc0cd5a0 0x7f5bcc0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.881+0000 7f5bd0b83640 1 -- 172.21.15.47:0/4122248230 >> v1:172.21.15.175:6789/0 conn(0x7f5bcc0d1010 legacy=0x7f5bcc0d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.881+0000 7f5bd0b83640 1 -- 172.21.15.47:0/4122248230 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc0b72f0 con 0x7f5bcc15cb50 2024-03-23T09:49:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.882+0000 7f5bd0b83640 1 --2- 172.21.15.47:0/4122248230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc15cb50 0x7f5bcc0b84a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009870 tx=0x7f5bc002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e267c8a48dafe76c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.882+0000 7f5bcb7fe640 1 -- 172.21.15.47:0/4122248230 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc002f840 con 0x7f5bcc15cb50 2024-03-23T09:49:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.882+0000 7f5bcb7fe640 1 -- 172.21.15.47:0/4122248230 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc002f9a0 con 0x7f5bcc15cb50 2024-03-23T09:49:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.882+0000 7f5bcb7fe640 1 -- 172.21.15.47:0/4122248230 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc002fd30 con 0x7f5bcc15cb50 2024-03-23T09:49:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.883+0000 7f5bd2de7640 1 -- 172.21.15.47:0/4122248230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc15cb50 msgr2=0x7f5bcc0b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.883+0000 7f5bd2de7640 1 --2- 172.21.15.47:0/4122248230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc15cb50 0x7f5bcc0b84a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009870 tx=0x7f5bc002ba20 comp rx=0 tx=0).stop 2024-03-23T09:49:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.883+0000 7f5bd2de7640 1 -- 172.21.15.47:0/4122248230 shutdown_connections 2024-03-23T09:49:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.883+0000 7f5bd2de7640 1 --2- 172.21.15.47:0/4122248230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bcc0cd5a0 0x7f5bcc0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.883+0000 7f5bd2de7640 1 --2- 172.21.15.47:0/4122248230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc15cb50 0x7f5bcc0b84a0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.883+0000 7f5bd2de7640 1 -- 172.21.15.47:0/4122248230 >> 172.21.15.47:0/4122248230 conn(0x7f5bcc065df0 msgr2=0x7f5bcc066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.883+0000 7f5bd2de7640 1 -- 172.21.15.47:0/4122248230 shutdown_connections 2024-03-23T09:49:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.883+0000 7f5bd2de7640 1 -- 172.21.15.47:0/4122248230 wait complete. 2024-03-23T09:49:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.884+0000 7f5bd2de7640 1 Processor -- start 2024-03-23T09:49:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.884+0000 7f5bd2de7640 1 -- start start 2024-03-23T09:49:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd2de7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bcc0cd5a0 0x7f5bcc154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd2de7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bcc0d1010 0x7f5bcc15a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd2de7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc15cb50 0x7f5bcc0cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd2de7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc15a450 con 0x7f5bcc15cb50 2024-03-23T09:49:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd2de7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc159840 con 0x7f5bcc0cd5a0 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd2de7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc159b20 con 0x7f5bcc0d1010 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd1384640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc15cb50 0x7f5bcc0cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd1384640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc15cb50 0x7f5bcc0cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37638/0 (socket says 172.21.15.47:37638) 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd1384640 1 -- 172.21.15.47:0/629718229 learned_addr learned my addr 172.21.15.47:0/629718229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd0b83640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bcc0cd5a0 0x7f5bcc154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bcbfff640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bcc0d1010 0x7f5bcc15a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd1384640 1 -- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bcc0d1010 msgr2=0x7f5bcc15a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd1384640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bcc0d1010 0x7f5bcc15a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd1384640 1 -- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bcc0cd5a0 msgr2=0x7f5bcc154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd1384640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bcc0cd5a0 0x7f5bcc154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd1384640 1 -- 172.21.15.47:0/629718229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc00094d0 con 0x7f5bcc15cb50 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd1384640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc15cb50 0x7f5bcc0cb9b0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c7a0 tx=0x7f5bc400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bc97fa640 1 -- 172.21.15.47:0/629718229 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4003b70 con 0x7f5bcc15cb50 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bc97fa640 1 -- 172.21.15.47:0/629718229 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc4003cf0 con 0x7f5bcc15cb50 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd2de7640 1 -- 172.21.15.47:0/629718229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bcc159e00 con 0x7f5bcc15cb50 2024-03-23T09:49:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bc97fa640 1 -- 172.21.15.47:0/629718229 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4003ea0 con 0x7f5bcc15cb50 2024-03-23T09:49:31.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.885+0000 7f5bd2de7640 1 -- 172.21.15.47:0/629718229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bcc155360 con 0x7f5bcc15cb50 2024-03-23T09:49:31.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.886+0000 7f5bd2de7640 1 -- 172.21.15.47:0/629718229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bcc072360 con 0x7f5bcc15cb50 2024-03-23T09:49:31.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.887+0000 7f5bc97fa640 1 -- 172.21.15.47:0/629718229 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5bc400aa90 con 0x7f5bcc15cb50 2024-03-23T09:49:31.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.887+0000 7f5bc97fa640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5bb006ac00 0x7f5bb006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:31.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.887+0000 7f5bc97fa640 1 -- 172.21.15.47:0/629718229 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f5bc40892b0 con 0x7f5bcc15cb50 2024-03-23T09:49:31.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.887+0000 7f5bd0b83640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5bb006ac00 0x7f5bb006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:31.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.888+0000 7f5bd0b83640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5bb006ac00 0x7f5bb006d0c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0036000 tx=0x7f5bc0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:31.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:31.891+0000 7f5bc97fa640 1 -- 172.21.15.47:0/629718229 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5bc40543a0 con 0x7f5bcc15cb50 2024-03-23T09:49:32.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.071+0000 7f5bd2de7640 1 -- 172.21.15.47:0/629718229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bcc066be0 con 0x7f5bcc15cb50 2024-03-23T09:49:32.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.072+0000 7f5bc97fa640 1 -- 172.21.15.47:0/629718229 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16449 (secure 0 0 0) 0x7f5bc40579c0 con 0x7f5bcc15cb50 2024-03-23T09:49:32.072 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:32.072 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":89,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:48:48.285538+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:47:09.722391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-23T09:47:10.354054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.30","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6834/4278516621":"2024-03-24T09:48:48.285514+0000","172.21.15.47:6835/4278516621":"2024-03-24T09:48:48.285514+0000","172.21.15.47:6835/1927629460":"2024-03-24T09:47:28.505920+0000","172.21.15.47:6834/1927629460":"2024-03-24T09:47:28.505920+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:49:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.073+0000 7f5bd2de7640 1 -- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5bb006ac00 msgr2=0x7f5bb006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.073+0000 7f5bd2de7640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5bb006ac00 0x7f5bb006d0c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0036000 tx=0x7f5bc0008030 comp rx=0 tx=0).stop 2024-03-23T09:49:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.073+0000 7f5bd2de7640 1 -- 172.21.15.47:0/629718229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc15cb50 msgr2=0x7f5bcc0cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.073+0000 7f5bd2de7640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc15cb50 0x7f5bcc0cb9b0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c7a0 tx=0x7f5bc400cc60 comp rx=0 tx=0).stop 2024-03-23T09:49:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.074+0000 7f5bd2de7640 1 -- 172.21.15.47:0/629718229 shutdown_connections 2024-03-23T09:49:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.074+0000 7f5bd2de7640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5bb006ac00 0x7f5bb006d0c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.074+0000 7f5bd2de7640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc15cb50 0x7f5bcc0cb9b0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.074+0000 7f5bd2de7640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bcc0d1010 0x7f5bcc15a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.074+0000 7f5bd2de7640 1 --2- 172.21.15.47:0/629718229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bcc0cd5a0 0x7f5bcc154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.074+0000 7f5bd2de7640 1 -- 172.21.15.47:0/629718229 >> 172.21.15.47:0/629718229 conn(0x7f5bcc065df0 msgr2=0x7f5bcc0b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:32.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.074+0000 7f5bd2de7640 1 -- 172.21.15.47:0/629718229 shutdown_connections 2024-03-23T09:49:32.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.074+0000 7f5bd2de7640 1 -- 172.21.15.47:0/629718229 wait complete. 2024-03-23T09:49:32.085 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:49:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.186+0000 7fdff9077640 1 Processor -- start 2024-03-23T09:49:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.186+0000 7fdff9077640 1 -- start start 2024-03-23T09:49:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff9077640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdff406a3f0 0x7fdff4071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff9077640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff40721f0 con 0x7fdff415c9d0 2024-03-23T09:49:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff9077640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff406eb80 con 0x7fdff415cdb0 2024-03-23T09:49:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff9077640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff406edd0 con 0x7fdff406a3f0 2024-03-23T09:49:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff2d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdff415c9d0 0x7fdff4069ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59282/0 (socket says 172.21.15.47:59282) 2024-03-23T09:49:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff2d9d640 1 -- 172.21.15.47:0/3384816301 learned_addr learned my addr 172.21.15.47:0/3384816301 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140106735 0 0) 0x7fdff40721f0 con 0x7fdff415c9d0 2024-03-23T09:49:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe0003520 con 0x7fdff415c9d0 2024-03-23T09:49:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff359e640 1 --2- 172.21.15.47:0/3384816301 >> v2:172.21.15.175:3300/0 conn(0x7fdff406a3f0 0x7fdff4071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1062134824 0 0) 0x7fdfe0003520 con 0x7fdff415c9d0 2024-03-23T09:49:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 >> v2:172.21.15.175:3300/0 conn(0x7fdff406a3f0 msgr2=0x7fdff4071cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff259c640 1 --2- 172.21.15.47:0/3384816301 >> v2:172.21.15.175:3300/0 conn(0x7fdff406a3f0 0x7fdff4071cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 >> v1:172.21.15.175:6789/0 conn(0x7fdff415cdb0 legacy=0x7fdff4160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff406f0b0 con 0x7fdff415c9d0 2024-03-23T09:49:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fdfe4001670 con 0x7fdff415c9d0 2024-03-23T09:49:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 >> v1:172.21.15.47:6789/0 conn(0x7fdff415c9d0 legacy=0x7fdff4069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff259c640 1 --2- 172.21.15.47:0/3384816301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfe0003970 0x7fdfe0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.187+0000 7fdff259c640 1 --2- 172.21.15.47:0/3384816301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe00047c0 0x7fdfe0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdff259c640 1 --2- 172.21.15.47:0/3384816301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfe0005180 0x7fdfe000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff406edd0 con 0x7fdfe00047c0 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe000c0d0 con 0x7fdfe0003970 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe000c290 con 0x7fdfe0005180 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdfebfff640 1 --2- 172.21.15.47:0/3384816301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe00047c0 0x7fdfe0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdff2d9d640 1 --2- 172.21.15.47:0/3384816301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfe0003970 0x7fdfe0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdff359e640 1 --2- 172.21.15.47:0/3384816301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfe0005180 0x7fdfe000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdfebfff640 1 -- 172.21.15.47:0/3384816301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfe0005180 msgr2=0x7fdfe000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdfebfff640 1 --2- 172.21.15.47:0/3384816301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfe0005180 0x7fdfe000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdfebfff640 1 -- 172.21.15.47:0/3384816301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfe0003970 msgr2=0x7fdfe0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdfebfff640 1 --2- 172.21.15.47:0/3384816301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfe0003970 0x7fdfe0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdfebfff640 1 -- 172.21.15.47:0/3384816301 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe0003520 con 0x7fdfe00047c0 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdff359e640 1 --2- 172.21.15.47:0/3384816301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfe0005180 0x7fdfe000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.188+0000 7fdff359e640 1 -- 172.21.15.47:0/3384816301 reap_dead start 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.189+0000 7fdfebfff640 1 --2- 172.21.15.47:0/3384816301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe00047c0 0x7fdfe0004c40 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc001720 tx=0x7fdfe00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653555c38c4f8512 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.189+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0052d0 con 0x7fdfe00047c0 2024-03-23T09:49:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.189+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfdc02e050 con 0x7fdfe00047c0 2024-03-23T09:49:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.189+0000 7fdff259c640 1 -- 172.21.15.47:0/3384816301 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc003830 con 0x7fdfe00047c0 2024-03-23T09:49:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.189+0000 7fdff9077640 1 -- 172.21.15.47:0/3384816301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe00047c0 msgr2=0x7fdfe0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.189+0000 7fdff9077640 1 --2- 172.21.15.47:0/3384816301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe00047c0 0x7fdfe0004c40 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc001720 tx=0x7fdfe00057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.189+0000 7fdff9077640 1 -- 172.21.15.47:0/3384816301 shutdown_connections 2024-03-23T09:49:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.189+0000 7fdff9077640 1 --2- 172.21.15.47:0/3384816301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe00047c0 0x7fdfe0004c40 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.189+0000 7fdff9077640 1 -- 172.21.15.47:0/3384816301 >> 172.21.15.47:0/3384816301 conn(0x7fdff40b13a0 msgr2=0x7fdff40b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.189+0000 7fdff9077640 1 -- 172.21.15.47:0/3384816301 shutdown_connections 2024-03-23T09:49:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.189+0000 7fdff9077640 1 -- 172.21.15.47:0/3384816301 wait complete. 2024-03-23T09:49:32.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.191+0000 7fdff9077640 1 Processor -- start 2024-03-23T09:49:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.191+0000 7fdff9077640 1 -- start start 2024-03-23T09:49:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.191+0000 7fdff9077640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfe00047c0 0x7fdff4150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.191+0000 7fdff9077640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdff40b0050 0x7fdff41f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.191+0000 7fdff9077640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdff40c9350 0x7fdff40c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.191+0000 7fdff9077640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41f6a10 con 0x7fdff40c9350 2024-03-23T09:49:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.191+0000 7fdff9077640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff40c8ec0 con 0x7fdfe00047c0 2024-03-23T09:49:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.191+0000 7fdff9077640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41f6450 con 0x7fdff40b0050 2024-03-23T09:49:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.191+0000 7fdff359e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdff40c9350 0x7fdff40c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.191+0000 7fdff359e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdff40c9350 0x7fdff40c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37658/0 (socket says 172.21.15.47:37658) 2024-03-23T09:49:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.191+0000 7fdff359e640 1 -- 172.21.15.47:0/856563182 learned_addr learned my addr 172.21.15.47:0/856563182 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdff2d9d640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfe00047c0 0x7fdff4150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdfebfff640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdff40b0050 0x7fdff41f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdff359e640 1 -- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdff40b0050 msgr2=0x7fdff41f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdff359e640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdff40b0050 0x7fdff41f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdff359e640 1 -- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfe00047c0 msgr2=0x7fdff4150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdff359e640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfe00047c0 0x7fdff4150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdff359e640 1 -- 172.21.15.47:0/856563182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff41f6730 con 0x7fdff40c9350 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdfebfff640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdff40b0050 0x7fdff41f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdff359e640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdff40c9350 0x7fdff40c9800 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fdff40b2af0 tx=0x7fdfec0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdfeb7fe640 1 -- 172.21.15.47:0/856563182 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec007cd0 con 0x7fdff40c9350 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdfeb7fe640 1 -- 172.21.15.47:0/856563182 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfec00f050 con 0x7fdff40c9350 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdfeb7fe640 1 -- 172.21.15.47:0/856563182 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec014650 con 0x7fdff40c9350 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdff9077640 1 -- 172.21.15.47:0/856563182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff40c8b50 con 0x7fdff40c9350 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.192+0000 7fdff9077640 1 -- 172.21.15.47:0/856563182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff41f6200 con 0x7fdff40c9350 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.193+0000 7fdff9077640 1 -- 172.21.15.47:0/856563182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff406aa40 con 0x7fdff40c9350 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.194+0000 7fdfeb7fe640 1 -- 172.21.15.47:0/856563182 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdfec0147b0 con 0x7fdff40c9350 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.194+0000 7fdfeb7fe640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdfd406ac30 0x7fdfd406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.194+0000 7fdfeb7fe640 1 -- 172.21.15.47:0/856563182 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fdfec089920 con 0x7fdff40c9350 2024-03-23T09:49:32.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.194+0000 7fdff2d9d640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdfd406ac30 0x7fdfd406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.195+0000 7fdff2d9d640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdfd406ac30 0x7fdfd406d0f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdff40ceee0 tx=0x7fdfe4000f80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:32.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.198+0000 7fdfeb7fe640 1 -- 172.21.15.47:0/856563182 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdfec054a00 con 0x7fdff40c9350 2024-03-23T09:49:32.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.414+0000 7fdff9077640 1 -- 172.21.15.47:0/856563182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdff4066be0 con 0x7fdff40c9350 2024-03-23T09:49:32.414 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:49:32.415+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:49:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.426+0000 7fdfeb7fe640 1 -- 172.21.15.47:0/856563182 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdfec058020 con 0x7fdff40c9350 2024-03-23T09:49:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:49:32.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.429+0000 7fdff9077640 1 -- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdfd406ac30 msgr2=0x7fdfd406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:32.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.429+0000 7fdff9077640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdfd406ac30 0x7fdfd406d0f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdff40ceee0 tx=0x7fdfe4000f80 comp rx=0 tx=0).stop 2024-03-23T09:49:32.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.429+0000 7fdff9077640 1 -- 172.21.15.47:0/856563182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdff40c9350 msgr2=0x7fdff40c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:32.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.429+0000 7fdff9077640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdff40c9350 0x7fdff40c9800 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fdff40b2af0 tx=0x7fdfec0047d0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.429+0000 7fdff9077640 1 -- 172.21.15.47:0/856563182 shutdown_connections 2024-03-23T09:49:32.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.429+0000 7fdff9077640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdfd406ac30 0x7fdfd406d0f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.429+0000 7fdff9077640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdff40c9350 0x7fdff40c9800 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.429+0000 7fdff9077640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdff40b0050 0x7fdff41f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.429+0000 7fdff9077640 1 --2- 172.21.15.47:0/856563182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfe00047c0 0x7fdff4150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.429+0000 7fdff9077640 1 -- 172.21.15.47:0/856563182 >> 172.21.15.47:0/856563182 conn(0x7fdff40b13a0 msgr2=0x7fdff40cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.429+0000 7fdff9077640 1 -- 172.21.15.47:0/856563182 shutdown_connections 2024-03-23T09:49:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.429+0000 7fdff9077640 1 -- 172.21.15.47:0/856563182 wait complete. 2024-03-23T09:49:32.430 INFO:tasks.ceph.mds.c.smithi047.stderr: -1989> 2024-03-23T09:48:55.305+0000 7f793c60c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:49:32.430 INFO:tasks.ceph.mds.c.smithi047.stderr: -1756> 2024-03-23T09:48:55.600+0000 7f7938e05640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5628a5710680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T09:49:32.440 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T09:49:32.456 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:49:32.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.545+0000 7efdab448640 1 Processor -- start 2024-03-23T09:49:32.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.545+0000 7efdab448640 1 -- start start 2024-03-23T09:49:32.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efdab448640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efda415cb50 0x7efda40b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efdab448640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efda40cd5a0 0x7efda40cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efdab448640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efda40d1010 0x7efda40d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efdab448640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda40b7e70 con 0x7efda40cd5a0 2024-03-23T09:49:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efdab448640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda415cf30 con 0x7efda40d1010 2024-03-23T09:49:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efdab448640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda40cdf20 con 0x7efda415cb50 2024-03-23T09:49:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efda89e3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efda40cd5a0 0x7efda40cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efda89e3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efda40cd5a0 0x7efda40cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37664/0 (socket says 172.21.15.47:37664) 2024-03-23T09:49:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efda89e3640 1 -- 172.21.15.47:0/558307832 learned_addr learned my addr 172.21.15.47:0/558307832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efda99e5640 1 --2- 172.21.15.47:0/558307832 >> v2:172.21.15.175:3300/0 conn(0x7efda40d1010 0x7efda40d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efda91e4640 1 --2- 172.21.15.47:0/558307832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efda415cb50 0x7efda40b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efda89e3640 1 -- 172.21.15.47:0/558307832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efda415cb50 msgr2=0x7efda40b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efda89e3640 1 --2- 172.21.15.47:0/558307832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efda415cb50 0x7efda40b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efda89e3640 1 -- 172.21.15.47:0/558307832 >> v2:172.21.15.175:3300/0 conn(0x7efda40d1010 msgr2=0x7efda40d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:32.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efda89e3640 1 --2- 172.21.15.47:0/558307832 >> v2:172.21.15.175:3300/0 conn(0x7efda40d1010 0x7efda40d14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.546+0000 7efda89e3640 1 -- 172.21.15.47:0/558307832 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda40b72f0 con 0x7efda40cd5a0 2024-03-23T09:49:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.547+0000 7efda89e3640 1 --2- 172.21.15.47:0/558307832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda40cd5a0 0x7efda40cd9e0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7efda40d3890 tx=0x7efd9802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159477683605557f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.547+0000 7efda3fff640 1 -- 172.21.15.47:0/558307832 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98030860 con 0x7efda40cd5a0 2024-03-23T09:49:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.547+0000 7efda3fff640 1 -- 172.21.15.47:0/558307832 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd980309c0 con 0x7efda40cd5a0 2024-03-23T09:49:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.547+0000 7efda3fff640 1 -- 172.21.15.47:0/558307832 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98030d50 con 0x7efda40cd5a0 2024-03-23T09:49:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.548+0000 7efdab448640 1 -- 172.21.15.47:0/558307832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda40cd5a0 msgr2=0x7efda40cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.548+0000 7efdab448640 1 --2- 172.21.15.47:0/558307832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda40cd5a0 0x7efda40cd9e0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7efda40d3890 tx=0x7efd9802b9a0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.548+0000 7efdab448640 1 -- 172.21.15.47:0/558307832 shutdown_connections 2024-03-23T09:49:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.548+0000 7efdab448640 1 --2- 172.21.15.47:0/558307832 >> v2:172.21.15.175:3300/0 conn(0x7efda40d1010 0x7efda40d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.548+0000 7efdab448640 1 --2- 172.21.15.47:0/558307832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda40cd5a0 0x7efda40cd9e0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.548+0000 7efdab448640 1 --2- 172.21.15.47:0/558307832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efda415cb50 0x7efda40b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.548+0000 7efdab448640 1 -- 172.21.15.47:0/558307832 >> 172.21.15.47:0/558307832 conn(0x7efda4065df0 msgr2=0x7efda4066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.548+0000 7efdab448640 1 -- 172.21.15.47:0/558307832 shutdown_connections 2024-03-23T09:49:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.548+0000 7efdab448640 1 -- 172.21.15.47:0/558307832 wait complete. 2024-03-23T09:49:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.549+0000 7efdab448640 1 Processor -- start 2024-03-23T09:49:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.549+0000 7efdab448640 1 -- start start 2024-03-23T09:49:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efdab448640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efda40cd5a0 0x7efda4153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efdab448640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efda40d1010 0x7efda41f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efdab448640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda415cb50 0x7efda40b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efdab448640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda41f5a70 con 0x7efda415cb50 2024-03-23T09:49:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efdab448640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda41570f0 con 0x7efda40cd5a0 2024-03-23T09:49:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efdab448640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda4157390 con 0x7efda40d1010 2024-03-23T09:49:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efda99e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda415cb50 0x7efda40b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efda99e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda415cb50 0x7efda40b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37668/0 (socket says 172.21.15.47:37668) 2024-03-23T09:49:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efda99e5640 1 -- 172.21.15.47:0/3707238101 learned_addr learned my addr 172.21.15.47:0/3707238101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efda91e4640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efda40cd5a0 0x7efda4153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efda89e3640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efda40d1010 0x7efda41f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efda99e5640 1 -- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efda40d1010 msgr2=0x7efda41f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efda99e5640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efda40d1010 0x7efda41f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efda99e5640 1 -- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efda40cd5a0 msgr2=0x7efda4153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efda99e5640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efda40cd5a0 0x7efda4153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.550+0000 7efda99e5640 1 -- 172.21.15.47:0/3707238101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd980094d0 con 0x7efda415cb50 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.551+0000 7efda99e5640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda415cb50 0x7efda40b2cb0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7efd94007e40 tx=0x7efd9400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.551+0000 7efda1ffb640 1 -- 172.21.15.47:0/3707238101 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94014620 con 0x7efda415cb50 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.551+0000 7efda1ffb640 1 -- 172.21.15.47:0/3707238101 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9400ee90 con 0x7efda415cb50 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.551+0000 7efdab448640 1 -- 172.21.15.47:0/3707238101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda41f5700 con 0x7efda415cb50 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.551+0000 7efda1ffb640 1 -- 172.21.15.47:0/3707238101 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd940149e0 con 0x7efda415cb50 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.551+0000 7efdab448640 1 -- 172.21.15.47:0/3707238101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda4156350 con 0x7efda415cb50 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.552+0000 7efdab448640 1 -- 172.21.15.47:0/3707238101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda4072360 con 0x7efda415cb50 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.552+0000 7efda1ffb640 1 -- 172.21.15.47:0/3707238101 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efd94014b40 con 0x7efda415cb50 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.553+0000 7efda1ffb640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efd9006ac00 0x7efd9006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.553+0000 7efda1ffb640 1 -- 172.21.15.47:0/3707238101 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6025+0+0 (secure 0 0 0) 0x7efd94089d40 con 0x7efda415cb50 2024-03-23T09:49:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.553+0000 7efda91e4640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efd9006ac00 0x7efd9006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:32.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.554+0000 7efda91e4640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efd9006ac00 0x7efd9006d0c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7efd9c007920 tx=0x7efd9c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:32.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.557+0000 7efda1ffb640 1 -- 172.21.15.47:0/3707238101 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efd94054e90 con 0x7efda415cb50 2024-03-23T09:49:32.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:32.773+0000 7efdab448640 1 -- 172.21.15.47:0/3707238101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efda4066be0 con 0x7efda415cb50 2024-03-23T09:49:33.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.433+0000 7efda1ffb640 1 -- 172.21.15.47:0/3707238101 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7efd940584b0 con 0x7efda415cb50 2024-03-23T09:49:33.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.436+0000 7efdab448640 1 -- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efd9006ac00 msgr2=0x7efd9006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:33.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.436+0000 7efdab448640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efd9006ac00 0x7efd9006d0c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7efd9c007920 tx=0x7efd9c008040 comp rx=0 tx=0).stop 2024-03-23T09:49:33.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.436+0000 7efdab448640 1 -- 172.21.15.47:0/3707238101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda415cb50 msgr2=0x7efda40b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:33.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.436+0000 7efdab448640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda415cb50 0x7efda40b2cb0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7efd94007e40 tx=0x7efd9400e4b0 comp rx=0 tx=0).stop 2024-03-23T09:49:33.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.436+0000 7efdab448640 1 -- 172.21.15.47:0/3707238101 shutdown_connections 2024-03-23T09:49:33.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.436+0000 7efdab448640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efd9006ac00 0x7efd9006d0c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:33.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.436+0000 7efdab448640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda415cb50 0x7efda40b2cb0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:33.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.436+0000 7efdab448640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efda40d1010 0x7efda41f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:33.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.436+0000 7efdab448640 1 --2- 172.21.15.47:0/3707238101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efda40cd5a0 0x7efda4153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:33.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.436+0000 7efdab448640 1 -- 172.21.15.47:0/3707238101 >> 172.21.15.47:0/3707238101 conn(0x7efda4065df0 msgr2=0x7efda415be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:33.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.436+0000 7efdab448640 1 -- 172.21.15.47:0/3707238101 shutdown_connections 2024-03-23T09:49:33.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.436+0000 7efdab448640 1 -- 172.21.15.47:0/3707238101 wait complete. 2024-03-23T09:49:33.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T09:49:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.603+0000 7f1c7b4e9640 1 Processor -- start 2024-03-23T09:49:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.603+0000 7f1c7b4e9640 1 -- start start 2024-03-23T09:49:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c7b4e9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1c7415d0f0 0x7f1c7415d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c7b4e9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c74160bd0 con 0x7f1c7415a860 2024-03-23T09:49:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c7b4e9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7415dbb0 con 0x7f1c740cf190 2024-03-23T09:49:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c7b4e9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c740d2790 con 0x7f1c7415d0f0 2024-03-23T09:49:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c79285640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1c7415a860 0x7f1c74161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59304/0 (socket says 172.21.15.47:59304) 2024-03-23T09:49:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c79285640 1 -- 172.21.15.47:0/1171596650 learned_addr learned my addr 172.21.15.47:0/1171596650 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998801453 0 0) 0x7f1c74160bd0 con 0x7f1c7415a860 2024-03-23T09:49:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c60003520 con 0x7f1c7415a860 2024-03-23T09:49:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c78a84640 1 --2- 172.21.15.47:0/1171596650 >> v2:172.21.15.175:3300/0 conn(0x7f1c7415d0f0 0x7f1c7415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2710333321 0 0) 0x7f1c60003520 con 0x7f1c7415a860 2024-03-23T09:49:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 >> v2:172.21.15.175:3300/0 conn(0x7f1c7415d0f0 msgr2=0x7f1c7415d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 --2- 172.21.15.47:0/1171596650 >> v2:172.21.15.175:3300/0 conn(0x7f1c7415d0f0 0x7f1c7415d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 >> v1:172.21.15.175:6789/0 conn(0x7f1c740cf190 legacy=0x7f1c740cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7415de90 con 0x7f1c7415a860 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f1c70001670 con 0x7f1c7415a860 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 >> v1:172.21.15.47:6789/0 conn(0x7f1c7415a860 legacy=0x7f1c74161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 --2- 172.21.15.47:0/1171596650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c600039c0 0x7f1c60003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 --2- 172.21.15.47:0/1171596650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c600047c0 0x7f1c60004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 --2- 172.21.15.47:0/1171596650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c60005180 0x7f1c6000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740d2790 con 0x7f1c600047c0 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6000c080 con 0x7f1c60005180 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.604+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6000c240 con 0x7f1c600039c0 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.605+0000 7f1c78a84640 1 --2- 172.21.15.47:0/1171596650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c600047c0 0x7f1c60004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.605+0000 7f1c79285640 1 --2- 172.21.15.47:0/1171596650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c600039c0 0x7f1c60003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.605+0000 7f1c79a86640 1 --2- 172.21.15.47:0/1171596650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c60005180 0x7f1c6000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.605+0000 7f1c78a84640 1 -- 172.21.15.47:0/1171596650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c600039c0 msgr2=0x7f1c60003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.605+0000 7f1c78a84640 1 --2- 172.21.15.47:0/1171596650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c600039c0 0x7f1c60003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.605+0000 7f1c78a84640 1 -- 172.21.15.47:0/1171596650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c60005180 msgr2=0x7f1c6000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.605+0000 7f1c78a84640 1 --2- 172.21.15.47:0/1171596650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c60005180 0x7f1c6000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.605+0000 7f1c78a84640 1 -- 172.21.15.47:0/1171596650 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c60003520 con 0x7f1c600047c0 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.605+0000 7f1c79a86640 1 -- 172.21.15.47:0/1171596650 reap_dead start 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.606+0000 7f1c78a84640 1 --2- 172.21.15.47:0/1171596650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c600047c0 0x7f1c60004c40 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009ac0 tx=0x7f1c600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315a5c6181654e9b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.606+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6402c6b0 con 0x7f1c600047c0 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.606+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6402c810 con 0x7f1c600047c0 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.606+0000 7f1c6bfff640 1 -- 172.21.15.47:0/1171596650 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6402cba0 con 0x7f1c600047c0 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.606+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/1171596650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c600047c0 msgr2=0x7f1c60004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.606+0000 7f1c7b4e9640 1 --2- 172.21.15.47:0/1171596650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c600047c0 0x7f1c60004c40 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009ac0 tx=0x7f1c600057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.606+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/1171596650 shutdown_connections 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.606+0000 7f1c7b4e9640 1 --2- 172.21.15.47:0/1171596650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c600047c0 0x7f1c60004c40 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.606+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/1171596650 >> 172.21.15.47:0/1171596650 conn(0x7f1c740659d0 msgr2=0x7f1c74065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.606+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/1171596650 shutdown_connections 2024-03-23T09:49:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.606+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/1171596650 wait complete. 2024-03-23T09:49:33.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.608+0000 7f1c7b4e9640 1 Processor -- start 2024-03-23T09:49:33.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.608+0000 7f1c7b4e9640 1 -- start start 2024-03-23T09:49:33.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.608+0000 7f1c7b4e9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c600047c0 0x7f1c740b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:33.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.608+0000 7f1c7b4e9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c741f68c0 0x7f1c740c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:33.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.608+0000 7f1c7b4e9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c740c9fd0 0x7f1c741596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.608+0000 7f1c7b4e9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c5a90 con 0x7f1c741f68c0 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.608+0000 7f1c7b4e9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740ca550 con 0x7f1c740c9fd0 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.608+0000 7f1c7b4e9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c741f5de0 con 0x7f1c600047c0 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.608+0000 7f1c78a84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c741f68c0 0x7f1c740c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.608+0000 7f1c78a84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c741f68c0 0x7f1c740c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37684/0 (socket says 172.21.15.47:37684) 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.608+0000 7f1c78a84640 1 -- 172.21.15.47:0/3978173662 learned_addr learned my addr 172.21.15.47:0/3978173662 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.609+0000 7f1c79a86640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c740c9fd0 0x7f1c741596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.609+0000 7f1c79285640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c600047c0 0x7f1c740b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.609+0000 7f1c78a84640 1 -- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c600047c0 msgr2=0x7f1c740b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.609+0000 7f1c78a84640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c600047c0 0x7f1c740b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.609+0000 7f1c78a84640 1 -- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c740c9fd0 msgr2=0x7f1c741596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.609+0000 7f1c78a84640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c740c9fd0 0x7f1c741596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.609+0000 7f1c78a84640 1 -- 172.21.15.47:0/3978173662 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c741f60c0 con 0x7f1c741f68c0 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.609+0000 7f1c78a84640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c741f68c0 0x7f1c740c9a90 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f1c6402c250 tx=0x7f1c64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.609+0000 7f1c69ffb640 1 -- 172.21.15.47:0/3978173662 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c64003810 con 0x7f1c741f68c0 2024-03-23T09:49:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.609+0000 7f1c69ffb640 1 -- 172.21.15.47:0/3978173662 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c64003970 con 0x7f1c741f68c0 2024-03-23T09:49:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.609+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/3978173662 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c741f63a0 con 0x7f1c741f68c0 2024-03-23T09:49:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.609+0000 7f1c69ffb640 1 -- 172.21.15.47:0/3978173662 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c64003d00 con 0x7f1c741f68c0 2024-03-23T09:49:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.609+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/3978173662 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c74148300 con 0x7f1c741f68c0 2024-03-23T09:49:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.610+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/3978173662 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c74072360 con 0x7f1c741f68c0 2024-03-23T09:49:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.611+0000 7f1c69ffb640 1 -- 172.21.15.47:0/3978173662 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1c64047450 con 0x7f1c741f68c0 2024-03-23T09:49:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.611+0000 7f1c69ffb640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1c5806ac30 0x7f1c5806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.611+0000 7f1c69ffb640 1 -- 172.21.15.47:0/3978173662 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f1c640ab8d0 con 0x7f1c741f68c0 2024-03-23T09:49:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.612+0000 7f1c79285640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1c5806ac30 0x7f1c5806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.612+0000 7f1c79285640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1c5806ac30 0x7f1c5806d0f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1c70001050 tx=0x7f1c70000f80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.616+0000 7f1c69ffb640 1 -- 172.21.15.47:0/3978173662 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1c64076b40 con 0x7f1c741f68c0 2024-03-23T09:49:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:33.794+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/3978173662 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1c7408a370 con 0x7f1c741f68c0 2024-03-23T09:49:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.435+0000 7f1c69ffb640 1 -- 172.21.15.47:0/3978173662 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1c64003e60 con 0x7f1c741f68c0 2024-03-23T09:49:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-03-23T09:49:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.439+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1c5806ac30 msgr2=0x7f1c5806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.439+0000 7f1c7b4e9640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1c5806ac30 0x7f1c5806d0f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1c70001050 tx=0x7f1c70000f80 comp rx=0 tx=0).stop 2024-03-23T09:49:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.439+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/3978173662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c741f68c0 msgr2=0x7f1c740c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.439+0000 7f1c7b4e9640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c741f68c0 0x7f1c740c9a90 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f1c6402c250 tx=0x7f1c64004730 comp rx=0 tx=0).stop 2024-03-23T09:49:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.440+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/3978173662 shutdown_connections 2024-03-23T09:49:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.440+0000 7f1c7b4e9640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1c5806ac30 0x7f1c5806d0f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.440+0000 7f1c7b4e9640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c740c9fd0 0x7f1c741596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.440+0000 7f1c7b4e9640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c741f68c0 0x7f1c740c9a90 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.440+0000 7f1c7b4e9640 1 --2- 172.21.15.47:0/3978173662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c600047c0 0x7f1c740b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.440+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/3978173662 >> 172.21.15.47:0/3978173662 conn(0x7f1c740659d0 msgr2=0x7f1c74065f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:34.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.440+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/3978173662 shutdown_connections 2024-03-23T09:49:34.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.440+0000 7f1c7b4e9640 1 -- 172.21.15.47:0/3978173662 wait complete. 2024-03-23T09:49:34.453 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T09:49:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.565+0000 7fac6cfa9640 1 Processor -- start 2024-03-23T09:49:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.565+0000 7fac6cfa9640 1 -- start start 2024-03-23T09:49:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.565+0000 7fac6cfa9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac6815cb50 0x7fac680cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.565+0000 7fac6cfa9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac680cde50 0x7fac680cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.565+0000 7fac6cfa9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fac680cfd80 0x7fac680d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.565+0000 7fac6cfa9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6815cf30 con 0x7fac680cde50 2024-03-23T09:49:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.565+0000 7fac6cfa9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac680d2330 con 0x7fac680cfd80 2024-03-23T09:49:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.565+0000 7fac6cfa9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac680d01f0 con 0x7fac6815cb50 2024-03-23T09:49:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.565+0000 7fac5ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac680cde50 0x7fac680cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.565+0000 7fac5ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac680cde50 0x7fac680cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37698/0 (socket says 172.21.15.47:37698) 2024-03-23T09:49:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.566+0000 7fac5ffff640 1 -- 172.21.15.47:0/4147030990 learned_addr learned my addr 172.21.15.47:0/4147030990 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.566+0000 7fac6659c640 1 --2- 172.21.15.47:0/4147030990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac6815cb50 0x7fac680cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.566+0000 7fac66d9d640 1 --2- 172.21.15.47:0/4147030990 >> v2:172.21.15.175:3300/0 conn(0x7fac680cfd80 0x7fac680d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.566+0000 7fac5ffff640 1 -- 172.21.15.47:0/4147030990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac6815cb50 msgr2=0x7fac680cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.566+0000 7fac5ffff640 1 --2- 172.21.15.47:0/4147030990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac6815cb50 0x7fac680cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.566+0000 7fac5ffff640 1 -- 172.21.15.47:0/4147030990 >> v2:172.21.15.175:3300/0 conn(0x7fac680cfd80 msgr2=0x7fac680d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.566+0000 7fac5ffff640 1 --2- 172.21.15.47:0/4147030990 >> v2:172.21.15.175:3300/0 conn(0x7fac680cfd80 0x7fac680d2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.566+0000 7fac5ffff640 1 -- 172.21.15.47:0/4147030990 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac680b1d90 con 0x7fac680cde50 2024-03-23T09:49:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.567+0000 7fac5ffff640 1 --2- 172.21.15.47:0/4147030990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac680cde50 0x7fac680cf840 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fac680b52a0 tx=0x7fac5002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7743324ddebe7f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.567+0000 7fac65d9b640 1 -- 172.21.15.47:0/4147030990 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50031640 con 0x7fac680cde50 2024-03-23T09:49:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.567+0000 7fac65d9b640 1 -- 172.21.15.47:0/4147030990 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac500317a0 con 0x7fac680cde50 2024-03-23T09:49:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.567+0000 7fac65d9b640 1 -- 172.21.15.47:0/4147030990 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50031b30 con 0x7fac680cde50 2024-03-23T09:49:34.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.567+0000 7fac6cfa9640 1 -- 172.21.15.47:0/4147030990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac680cde50 msgr2=0x7fac680cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:34.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.567+0000 7fac6cfa9640 1 --2- 172.21.15.47:0/4147030990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac680cde50 0x7fac680cf840 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fac680b52a0 tx=0x7fac5002b8f0 comp rx=0 tx=0).stop 2024-03-23T09:49:34.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.567+0000 7fac6cfa9640 1 -- 172.21.15.47:0/4147030990 shutdown_connections 2024-03-23T09:49:34.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.567+0000 7fac6cfa9640 1 --2- 172.21.15.47:0/4147030990 >> v2:172.21.15.175:3300/0 conn(0x7fac680cfd80 0x7fac680d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:34.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.567+0000 7fac6cfa9640 1 --2- 172.21.15.47:0/4147030990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac680cde50 0x7fac680cf840 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:34.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.567+0000 7fac6cfa9640 1 --2- 172.21.15.47:0/4147030990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac6815cb50 0x7fac680cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:34.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.567+0000 7fac6cfa9640 1 -- 172.21.15.47:0/4147030990 >> 172.21.15.47:0/4147030990 conn(0x7fac68065df0 msgr2=0x7fac68066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:34.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.567+0000 7fac6cfa9640 1 -- 172.21.15.47:0/4147030990 shutdown_connections 2024-03-23T09:49:34.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.567+0000 7fac6cfa9640 1 -- 172.21.15.47:0/4147030990 wait complete. 2024-03-23T09:49:34.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.569+0000 7fac6cfa9640 1 Processor -- start 2024-03-23T09:49:34.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.569+0000 7fac6cfa9640 1 -- start start 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.569+0000 7fac6cfa9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac680cde50 0x7fac680c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.569+0000 7fac6cfa9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac680cfd80 0x7fac681f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.569+0000 7fac6cfa9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac6815cb50 0x7fac68158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.569+0000 7fac6cfa9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac68154b90 con 0x7fac680cfd80 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.569+0000 7fac6cfa9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac68153f80 con 0x7fac680cde50 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.569+0000 7fac6cfa9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac68154260 con 0x7fac6815cb50 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.569+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac680cfd80 0x7fac681f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.569+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac680cfd80 0x7fac681f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37702/0 (socket says 172.21.15.47:37702) 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac5ffff640 1 -- 172.21.15.47:0/610902737 learned_addr learned my addr 172.21.15.47:0/610902737 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac6659c640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac680cde50 0x7fac680c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac66d9d640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac6815cb50 0x7fac68158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac5ffff640 1 -- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac6815cb50 msgr2=0x7fac68158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac5ffff640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac6815cb50 0x7fac68158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:34.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac5ffff640 1 -- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac680cde50 msgr2=0x7fac680c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:34.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac5ffff640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac680cde50 0x7fac680c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:34.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac5ffff640 1 -- 172.21.15.47:0/610902737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac500094d0 con 0x7fac680cfd80 2024-03-23T09:49:34.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac5ffff640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac680cfd80 0x7fac681f27e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fac50000c00 tx=0x7fac500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:34.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac5effd640 1 -- 172.21.15.47:0/610902737 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5003ac60 con 0x7fac680cfd80 2024-03-23T09:49:34.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac5effd640 1 -- 172.21.15.47:0/610902737 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5003adc0 con 0x7fac680cfd80 2024-03-23T09:49:34.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac6cfa9640 1 -- 172.21.15.47:0/610902737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac68154540 con 0x7fac680cfd80 2024-03-23T09:49:34.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac5effd640 1 -- 172.21.15.47:0/610902737 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50044630 con 0x7fac680cfd80 2024-03-23T09:49:34.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.570+0000 7fac6cfa9640 1 -- 172.21.15.47:0/610902737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac681f2d20 con 0x7fac680cfd80 2024-03-23T09:49:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.571+0000 7fac6cfa9640 1 -- 172.21.15.47:0/610902737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac68072360 con 0x7fac680cfd80 2024-03-23T09:49:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.572+0000 7fac5effd640 1 -- 172.21.15.47:0/610902737 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fac5002f050 con 0x7fac680cfd80 2024-03-23T09:49:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.572+0000 7fac5effd640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fac4806ac30 0x7fac4806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.572+0000 7fac5effd640 1 -- 172.21.15.47:0/610902737 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5587+0+0 (secure 0 0 0) 0x7fac500aaf40 con 0x7fac680cfd80 2024-03-23T09:49:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.572+0000 7fac6659c640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fac4806ac30 0x7fac4806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.573+0000 7fac6659c640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fac4806ac30 0x7fac4806d0f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fac580079a0 tx=0x7fac58008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:34.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.576+0000 7fac5effd640 1 -- 172.21.15.47:0/610902737 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fac500762d0 con 0x7fac680cfd80 2024-03-23T09:49:34.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:34.763+0000 7fac6cfa9640 1 -- 172.21.15.47:0/610902737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fac68066be0 con 0x7fac680cfd80 2024-03-23T09:49:35.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.437+0000 7fac5effd640 1 -- 172.21.15.47:0/610902737 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) v1 ==== 169+0+0 (secure 0 0 0) 0x7fac50031e60 con 0x7fac680cfd80 2024-03-23T09:49:35.437 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-03-23T09:49:35.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.441+0000 7fac6cfa9640 1 -- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fac4806ac30 msgr2=0x7fac4806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:35.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.441+0000 7fac6cfa9640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fac4806ac30 0x7fac4806d0f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fac580079a0 tx=0x7fac58008040 comp rx=0 tx=0).stop 2024-03-23T09:49:35.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.441+0000 7fac6cfa9640 1 -- 172.21.15.47:0/610902737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac680cfd80 msgr2=0x7fac681f27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:35.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.441+0000 7fac6cfa9640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac680cfd80 0x7fac681f27e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fac50000c00 tx=0x7fac500046c0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.441+0000 7fac6cfa9640 1 -- 172.21.15.47:0/610902737 shutdown_connections 2024-03-23T09:49:35.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.441+0000 7fac6cfa9640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fac4806ac30 0x7fac4806d0f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.441+0000 7fac6cfa9640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac6815cb50 0x7fac68158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.441+0000 7fac6cfa9640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac680cfd80 0x7fac681f27e0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.441+0000 7fac6cfa9640 1 --2- 172.21.15.47:0/610902737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac680cde50 0x7fac680c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.441+0000 7fac6cfa9640 1 -- 172.21.15.47:0/610902737 >> 172.21.15.47:0/610902737 conn(0x7fac68065df0 msgr2=0x7fac6815e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:35.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.441+0000 7fac6cfa9640 1 -- 172.21.15.47:0/610902737 shutdown_connections 2024-03-23T09:49:35.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.441+0000 7fac6cfa9640 1 -- 172.21.15.47:0/610902737 wait complete. 2024-03-23T09:49:35.454 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-23T09:49:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.564+0000 7f2c7a626640 1 Processor -- start 2024-03-23T09:49:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.564+0000 7f2c7a626640 1 -- start start 2024-03-23T09:49:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c7a626640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c74161020 0x7f2c740b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c7a626640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2c74161400 0x7f2c7406b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c7a626640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7406bd40 0x7f2c740ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c7a626640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c7406c1d0 con 0x7f2c74161020 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c7a626640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c740cdad0 con 0x7f2c74161400 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c7a626640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740b03c0 con 0x7f2c7406bd40 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c73fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c74161020 0x7f2c740b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c73fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c74161020 0x7f2c740b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43816/0 (socket says 172.21.15.47:43816) 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c73fff640 1 -- 172.21.15.47:0/2116392393 learned_addr learned my addr 172.21.15.47:0/2116392393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c737fe640 1 --2- 172.21.15.47:0/2116392393 >> v2:172.21.15.175:3300/0 conn(0x7f2c74161400 0x7f2c7406b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c78bc3640 1 --2- 172.21.15.47:0/2116392393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7406bd40 0x7f2c740ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c73fff640 1 -- 172.21.15.47:0/2116392393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7406bd40 msgr2=0x7f2c740ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c73fff640 1 --2- 172.21.15.47:0/2116392393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7406bd40 0x7f2c740ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c73fff640 1 -- 172.21.15.47:0/2116392393 >> v2:172.21.15.175:3300/0 conn(0x7f2c74161400 msgr2=0x7f2c7406b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c73fff640 1 --2- 172.21.15.47:0/2116392393 >> v2:172.21.15.175:3300/0 conn(0x7f2c74161400 0x7f2c7406b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c73fff640 1 -- 172.21.15.47:0/2116392393 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c740b06a0 con 0x7f2c74161020 2024-03-23T09:49:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.565+0000 7f2c78bc3640 1 --2- 172.21.15.47:0/2116392393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7406bd40 0x7f2c740ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.566+0000 7f2c73fff640 1 --2- 172.21.15.47:0/2116392393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74161020 0x7f2c740b1180 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2c740d1340 tx=0x7f2c6402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52280d063e86ff server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.566+0000 7f2c72ffd640 1 -- 172.21.15.47:0/2116392393 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64030860 con 0x7f2c74161020 2024-03-23T09:49:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.566+0000 7f2c72ffd640 1 -- 172.21.15.47:0/2116392393 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c640309c0 con 0x7f2c74161020 2024-03-23T09:49:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.566+0000 7f2c72ffd640 1 -- 172.21.15.47:0/2116392393 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64030d50 con 0x7f2c74161020 2024-03-23T09:49:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.567+0000 7f2c7a626640 1 -- 172.21.15.47:0/2116392393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74161020 msgr2=0x7f2c740b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.567+0000 7f2c7a626640 1 --2- 172.21.15.47:0/2116392393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74161020 0x7f2c740b1180 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2c740d1340 tx=0x7f2c6402b9a0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.567+0000 7f2c7a626640 1 -- 172.21.15.47:0/2116392393 shutdown_connections 2024-03-23T09:49:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.567+0000 7f2c7a626640 1 --2- 172.21.15.47:0/2116392393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7406bd40 0x7f2c740ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.567+0000 7f2c7a626640 1 --2- 172.21.15.47:0/2116392393 >> v2:172.21.15.175:3300/0 conn(0x7f2c74161400 0x7f2c7406b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.567+0000 7f2c7a626640 1 --2- 172.21.15.47:0/2116392393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74161020 0x7f2c740b1180 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.567+0000 7f2c7a626640 1 -- 172.21.15.47:0/2116392393 >> 172.21.15.47:0/2116392393 conn(0x7f2c740d0670 msgr2=0x7f2c740d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.567+0000 7f2c7a626640 1 -- 172.21.15.47:0/2116392393 shutdown_connections 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.567+0000 7f2c7a626640 1 -- 172.21.15.47:0/2116392393 wait complete. 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.568+0000 7f2c7a626640 1 Processor -- start 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.568+0000 7f2c7a626640 1 -- start start 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.569+0000 7f2c7a626640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7406bd40 0x7f2c740c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.569+0000 7f2c7a626640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c74161020 0x7f2c740b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.569+0000 7f2c7a626640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c74161400 0x7f2c7409dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.569+0000 7f2c7a626640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7414e8a0 con 0x7f2c7406bd40 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.569+0000 7f2c7a626640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740c9780 con 0x7f2c74161020 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.569+0000 7f2c7a626640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740c9a60 con 0x7f2c74161400 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.569+0000 7f2c73fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7406bd40 0x7f2c740c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.569+0000 7f2c73fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7406bd40 0x7f2c740c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43824/0 (socket says 172.21.15.47:43824) 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.569+0000 7f2c73fff640 1 -- 172.21.15.47:0/3634023031 learned_addr learned my addr 172.21.15.47:0/3634023031 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.569+0000 7f2c78bc3640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c74161400 0x7f2c7409dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.569+0000 7f2c737fe640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c74161020 0x7f2c740b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.570+0000 7f2c73fff640 1 -- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c74161400 msgr2=0x7f2c7409dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.570+0000 7f2c73fff640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c74161400 0x7f2c7409dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.570+0000 7f2c73fff640 1 -- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c74161020 msgr2=0x7f2c740b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.570+0000 7f2c73fff640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c74161020 0x7f2c740b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.570+0000 7f2c73fff640 1 -- 172.21.15.47:0/3634023031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c640094d0 con 0x7f2c7406bd40 2024-03-23T09:49:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.570+0000 7f2c73fff640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7406bd40 0x7f2c740c91a0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2c6402b9e0 tx=0x7f2c64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.570+0000 7f2c70ff9640 1 -- 172.21.15.47:0/3634023031 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64039c90 con 0x7f2c7406bd40 2024-03-23T09:49:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.570+0000 7f2c70ff9640 1 -- 172.21.15.47:0/3634023031 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c64039e10 con 0x7f2c7406bd40 2024-03-23T09:49:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.570+0000 7f2c7a626640 1 -- 172.21.15.47:0/3634023031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c740c9d40 con 0x7f2c7406bd40 2024-03-23T09:49:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.570+0000 7f2c70ff9640 1 -- 172.21.15.47:0/3634023031 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64043640 con 0x7f2c7406bd40 2024-03-23T09:49:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.570+0000 7f2c7a626640 1 -- 172.21.15.47:0/3634023031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c740ca020 con 0x7f2c7406bd40 2024-03-23T09:49:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.571+0000 7f2c7a626640 1 -- 172.21.15.47:0/3634023031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c740672f0 con 0x7f2c7406bd40 2024-03-23T09:49:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.571+0000 7f2c70ff9640 1 -- 172.21.15.47:0/3634023031 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2c640437a0 con 0x7f2c7406bd40 2024-03-23T09:49:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.572+0000 7f2c70ff9640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c6006ac00 0x7f2c6006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.572+0000 7f2c70ff9640 1 -- 172.21.15.47:0/3634023031 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f2c6403e070 con 0x7f2c7406bd40 2024-03-23T09:49:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.572+0000 7f2c737fe640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c6006ac00 0x7f2c6006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.573+0000 7f2c737fe640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c6006ac00 0x7f2c6006d0c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2c740ca6f0 tx=0x7f2c68008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.576+0000 7f2c70ff9640 1 -- 172.21.15.47:0/3634023031 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2c64077610 con 0x7f2c7406bd40 2024-03-23T09:49:35.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.756+0000 7f2c7a626640 1 -- 172.21.15.47:0/3634023031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2c740c74b0 con 0x7f2c7406bd40 2024-03-23T09:49:35.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.759+0000 7f2c70ff9640 1 -- 172.21.15.47:0/3634023031 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2c6407ac30 con 0x7f2c7406bd40 2024-03-23T09:49:35.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.761+0000 7f2c7a626640 1 -- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c6006ac00 msgr2=0x7f2c6006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:35.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.761+0000 7f2c7a626640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c6006ac00 0x7f2c6006d0c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2c740ca6f0 tx=0x7f2c68008040 comp rx=0 tx=0).stop 2024-03-23T09:49:35.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.761+0000 7f2c7a626640 1 -- 172.21.15.47:0/3634023031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7406bd40 msgr2=0x7f2c740c91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:35.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.761+0000 7f2c7a626640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7406bd40 0x7f2c740c91a0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2c6402b9e0 tx=0x7f2c64004730 comp rx=0 tx=0).stop 2024-03-23T09:49:35.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.761+0000 7f2c7a626640 1 -- 172.21.15.47:0/3634023031 shutdown_connections 2024-03-23T09:49:35.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.761+0000 7f2c7a626640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2c6006ac00 0x7f2c6006d0c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.761+0000 7f2c7a626640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c74161400 0x7f2c7409dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.761+0000 7f2c7a626640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c74161020 0x7f2c740b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.761+0000 7f2c7a626640 1 --2- 172.21.15.47:0/3634023031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7406bd40 0x7f2c740c91a0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.761+0000 7f2c7a626640 1 -- 172.21.15.47:0/3634023031 >> 172.21.15.47:0/3634023031 conn(0x7f2c740d0670 msgr2=0x7f2c7406eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:35.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.761+0000 7f2c7a626640 1 -- 172.21.15.47:0/3634023031 shutdown_connections 2024-03-23T09:49:35.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.761+0000 7f2c7a626640 1 -- 172.21.15.47:0/3634023031 wait complete. 2024-03-23T09:49:35.772 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-03-23T09:49:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.873+0000 7f6bb1d00640 1 Processor -- start 2024-03-23T09:49:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.873+0000 7f6bb1d00640 1 -- start start 2024-03-23T09:49:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.874+0000 7f6bb1d00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bac0cf210 0x7f6bac0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.874+0000 7f6bb1d00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac0cfbd0 con 0x7f6bac0b5a90 2024-03-23T09:49:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.874+0000 7f6bb1d00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac15d0e0 con 0x7f6bac15cd00 2024-03-23T09:49:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.874+0000 7f6bb1d00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bac0d1540 con 0x7f6bac0cf210 2024-03-23T09:49:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.874+0000 7f6baaffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bac0cf210 0x7f6bac0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.874+0000 7f6babfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6bac0b5a90 0x7f6bac0b5ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44552/0 (socket says 172.21.15.47:44552) 2024-03-23T09:49:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.874+0000 7f6babfff640 1 -- 172.21.15.47:0/608741693 learned_addr learned my addr 172.21.15.47:0/608741693 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.874+0000 7f6baaffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bac0cf210 0x7f6bac0cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43828/0 (socket says 172.21.15.47:43828) 2024-03-23T09:49:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.874+0000 7f6baaffd640 1 -- 172.21.15.47:0/608741693 >> v1:172.21.15.175:6789/0 conn(0x7f6bac15cd00 legacy=0x7f6bac0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.874+0000 7f6baaffd640 1 -- 172.21.15.47:0/608741693 >> v1:172.21.15.47:6789/0 conn(0x7f6bac0b5a90 legacy=0x7f6bac0b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.874+0000 7f6baaffd640 1 -- 172.21.15.47:0/608741693 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac0d1820 con 0x7f6bac0cf210 2024-03-23T09:49:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.874+0000 7f6baa7fc640 1 -- 172.21.15.47:0/608741693 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009536890 0 0) 0x7f6bac0cfbd0 con 0x7f6bac0b5a90 2024-03-23T09:49:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.875+0000 7f6baaffd640 1 --2- 172.21.15.47:0/608741693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bac0cf210 0x7f6bac0cf690 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00097a0 tx=0x7f6ba002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa1d0f1a0b4b686 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.875+0000 7f6baa7fc640 1 -- 172.21.15.47:0/608741693 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba002e070 con 0x7f6bac0cf210 2024-03-23T09:49:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.875+0000 7f6baa7fc640 1 -- 172.21.15.47:0/608741693 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba00315a0 con 0x7f6bac0cf210 2024-03-23T09:49:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.875+0000 7f6baa7fc640 1 -- 172.21.15.47:0/608741693 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0031930 con 0x7f6bac0cf210 2024-03-23T09:49:35.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.876+0000 7f6bb1d00640 1 -- 172.21.15.47:0/608741693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bac0cf210 msgr2=0x7f6bac0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:35.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.876+0000 7f6bb1d00640 1 --2- 172.21.15.47:0/608741693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bac0cf210 0x7f6bac0cf690 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00097a0 tx=0x7f6ba002b840 comp rx=0 tx=0).stop 2024-03-23T09:49:35.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.876+0000 7f6bb1d00640 1 -- 172.21.15.47:0/608741693 shutdown_connections 2024-03-23T09:49:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.876+0000 7f6bb1d00640 1 --2- 172.21.15.47:0/608741693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bac0cf210 0x7f6bac0cf690 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.876+0000 7f6bb1d00640 1 -- 172.21.15.47:0/608741693 >> 172.21.15.47:0/608741693 conn(0x7f6bac06b160 msgr2=0x7f6bac06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.876+0000 7f6bb1d00640 1 -- 172.21.15.47:0/608741693 shutdown_connections 2024-03-23T09:49:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.876+0000 7f6bb1d00640 1 -- 172.21.15.47:0/608741693 wait complete. 2024-03-23T09:49:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.877+0000 7f6bb1d00640 1 Processor -- start 2024-03-23T09:49:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.877+0000 7f6bb1d00640 1 -- start start 2024-03-23T09:49:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bb1d00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bac0b5a90 0x7f6bac158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:35.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bb1d00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bac0cf210 0x7f6bac156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bb1d00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bac15cd00 0x7f6bac0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bb1d00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac1574f0 con 0x7f6bac0b5a90 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bb1d00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac1577d0 con 0x7f6bac15cd00 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bb1d00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac157ab0 con 0x7f6bac0cf210 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bac0b5a90 0x7f6bac158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bac0b5a90 0x7f6bac158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43844/0 (socket says 172.21.15.47:43844) 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bab7fe640 1 -- 172.21.15.47:0/1109737487 learned_addr learned my addr 172.21.15.47:0/1109737487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6babfff640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bac15cd00 0x7f6bac0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6baaffd640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bac0cf210 0x7f6bac156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bab7fe640 1 -- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bac0cf210 msgr2=0x7f6bac156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bab7fe640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bac0cf210 0x7f6bac156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bab7fe640 1 -- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bac15cd00 msgr2=0x7f6bac0c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bab7fe640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bac15cd00 0x7f6bac0c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bab7fe640 1 -- 172.21.15.47:0/1109737487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba0009400 con 0x7f6bac0b5a90 2024-03-23T09:49:35.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6babfff640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bac15cd00 0x7f6bac0c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:35.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6bab7fe640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bac0b5a90 0x7f6bac158590 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00b3f0 tx=0x7f6b9c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:35.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.878+0000 7f6b93fff640 1 -- 172.21.15.47:0/1109737487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c011830 con 0x7f6bac0b5a90 2024-03-23T09:49:35.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.879+0000 7f6b93fff640 1 -- 172.21.15.47:0/1109737487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c011990 con 0x7f6bac0b5a90 2024-03-23T09:49:35.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.879+0000 7f6bb1d00640 1 -- 172.21.15.47:0/1109737487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac157d90 con 0x7f6bac0b5a90 2024-03-23T09:49:35.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.879+0000 7f6b93fff640 1 -- 172.21.15.47:0/1109737487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c011d40 con 0x7f6bac0b5a90 2024-03-23T09:49:35.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.879+0000 7f6bb1d00640 1 -- 172.21.15.47:0/1109737487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac0c9140 con 0x7f6bac0b5a90 2024-03-23T09:49:35.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.879+0000 7f6bb1d00640 1 -- 172.21.15.47:0/1109737487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac072360 con 0x7f6bac0b5a90 2024-03-23T09:49:35.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.880+0000 7f6b93fff640 1 -- 172.21.15.47:0/1109737487 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6b9c01a400 con 0x7f6bac0b5a90 2024-03-23T09:49:35.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.880+0000 7f6b93fff640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b9806ac00 0x7f6b9806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:35.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.880+0000 7f6b93fff640 1 -- 172.21.15.47:0/1109737487 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f6b9c088f50 con 0x7f6bac0b5a90 2024-03-23T09:49:35.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.880+0000 7f6baaffd640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b9806ac00 0x7f6b9806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:35.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.881+0000 7f6baaffd640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b9806ac00 0x7f6b9806d0c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0036000 tx=0x7f6ba0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:35.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:35.885+0000 7f6b93fff640 1 -- 172.21.15.47:0/1109737487 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6b9c054380 con 0x7f6bac0b5a90 2024-03-23T09:49:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.133+0000 7f6bb1d00640 1 -- 172.21.15.47:0/1109737487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f6bac066be0 con 0x7f6bac0b5a90 2024-03-23T09:49:36.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.445+0000 7f6b93fff640 1 -- 172.21.15.47:0/1109737487 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v386) v1 ==== 167+0+0 (secure 0 0 0) 0x7f6b9c0579a0 con 0x7f6bac0b5a90 2024-03-23T09:49:36.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.447+0000 7f6bb1d00640 1 -- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b9806ac00 msgr2=0x7f6b9806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:36.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.447+0000 7f6bb1d00640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b9806ac00 0x7f6b9806d0c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0036000 tx=0x7f6ba0008030 comp rx=0 tx=0).stop 2024-03-23T09:49:36.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.447+0000 7f6bb1d00640 1 -- 172.21.15.47:0/1109737487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bac0b5a90 msgr2=0x7f6bac158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:36.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.447+0000 7f6bb1d00640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bac0b5a90 0x7f6bac158590 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00b3f0 tx=0x7f6b9c00b8b0 comp rx=0 tx=0).stop 2024-03-23T09:49:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.447+0000 7f6bb1d00640 1 -- 172.21.15.47:0/1109737487 shutdown_connections 2024-03-23T09:49:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.447+0000 7f6bb1d00640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b9806ac00 0x7f6b9806d0c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.447+0000 7f6bb1d00640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bac15cd00 0x7f6bac0c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.447+0000 7f6bb1d00640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bac0cf210 0x7f6bac156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.447+0000 7f6bb1d00640 1 --2- 172.21.15.47:0/1109737487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bac0b5a90 0x7f6bac158590 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.447+0000 7f6bb1d00640 1 -- 172.21.15.47:0/1109737487 >> 172.21.15.47:0/1109737487 conn(0x7f6bac06b160 msgr2=0x7f6bac0cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.447+0000 7f6bb1d00640 1 -- 172.21.15.47:0/1109737487 shutdown_connections 2024-03-23T09:49:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.447+0000 7f6bb1d00640 1 -- 172.21.15.47:0/1109737487 wait complete. 2024-03-23T09:49:36.458 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-23T09:49:36.458 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-23T09:49:36.459 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-03-23T09:49:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.560+0000 7f9f2285b640 1 Processor -- start 2024-03-23T09:49:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.560+0000 7f9f2285b640 1 -- start start 2024-03-23T09:49:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.560+0000 7f9f2285b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f1c0ae200 0x7f9f1c0cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.561+0000 7f9f2285b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9f1c15e520 0x7f9f1c15e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.561+0000 7f9f2285b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f1c06a6b0 con 0x7f9f1c0b3b70 2024-03-23T09:49:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.561+0000 7f9f2285b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1c0cb6b0 con 0x7f9f1c0ae200 2024-03-23T09:49:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.561+0000 7f9f2285b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1c15ee70 con 0x7f9f1c15e520 2024-03-23T09:49:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.561+0000 7f9f1bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f1c0ae200 0x7f9f1c0cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.561+0000 7f9f1bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f1c0ae200 0x7f9f1c0cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43850/0 (socket says 172.21.15.47:43850) 2024-03-23T09:49:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.561+0000 7f9f1bfff640 1 -- 172.21.15.47:0/571711327 learned_addr learned my addr 172.21.15.47:0/571711327 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.561+0000 7f9f1bfff640 1 -- 172.21.15.47:0/571711327 >> v2:172.21.15.175:3300/0 conn(0x7f9f1c15e520 msgr2=0x7f9f1c15e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.561+0000 7f9f20df8640 1 --2- 172.21.15.47:0/571711327 >> v2:172.21.15.175:3300/0 conn(0x7f9f1c15e520 0x7f9f1c15e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.561+0000 7f9f1bfff640 1 --2- 172.21.15.47:0/571711327 >> v2:172.21.15.175:3300/0 conn(0x7f9f1c15e520 0x7f9f1c15e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.561+0000 7f9f1bfff640 1 -- 172.21.15.47:0/571711327 >> v1:172.21.15.175:6789/0 conn(0x7f9f1c0b3b70 legacy=0x7f9f1c06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.561+0000 7f9f1bfff640 1 -- 172.21.15.47:0/571711327 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c069d80 con 0x7f9f1c0ae200 2024-03-23T09:49:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.562+0000 7f9f1bfff640 1 --2- 172.21.15.47:0/571711327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f1c0ae200 0x7f9f1c0cb170 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c158a70 tx=0x7f9f0c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358a9e316905c907 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.562+0000 7f9f1affd640 1 -- 172.21.15.47:0/571711327 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c0317e0 con 0x7f9f1c0ae200 2024-03-23T09:49:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.562+0000 7f9f1affd640 1 -- 172.21.15.47:0/571711327 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c031940 con 0x7f9f1c0ae200 2024-03-23T09:49:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.562+0000 7f9f1affd640 1 -- 172.21.15.47:0/571711327 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c031cd0 con 0x7f9f1c0ae200 2024-03-23T09:49:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.562+0000 7f9f2285b640 1 -- 172.21.15.47:0/571711327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f1c0ae200 msgr2=0x7f9f1c0cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.562+0000 7f9f2285b640 1 --2- 172.21.15.47:0/571711327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f1c0ae200 0x7f9f1c0cb170 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c158a70 tx=0x7f9f0c02b880 comp rx=0 tx=0).stop 2024-03-23T09:49:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.562+0000 7f9f2285b640 1 -- 172.21.15.47:0/571711327 shutdown_connections 2024-03-23T09:49:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.562+0000 7f9f2285b640 1 --2- 172.21.15.47:0/571711327 >> v2:172.21.15.175:3300/0 conn(0x7f9f1c15e520 0x7f9f1c15e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.563+0000 7f9f2285b640 1 --2- 172.21.15.47:0/571711327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f1c0ae200 0x7f9f1c0cb170 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.563+0000 7f9f2285b640 1 -- 172.21.15.47:0/571711327 >> 172.21.15.47:0/571711327 conn(0x7f9f1c0ced80 msgr2=0x7f9f1c0cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.563+0000 7f9f2285b640 1 -- 172.21.15.47:0/571711327 shutdown_connections 2024-03-23T09:49:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.563+0000 7f9f2285b640 1 -- 172.21.15.47:0/571711327 wait complete. 2024-03-23T09:49:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.564+0000 7f9f2285b640 1 Processor -- start 2024-03-23T09:49:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.564+0000 7f9f2285b640 1 -- start start 2024-03-23T09:49:36.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f2285b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f1c0ae200 0x7f9f1c1518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:36.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f2285b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f1c0b3b70 0x7f9f1c15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:36.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f2285b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f1c15e520 0x7f9f1c0af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f2285b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c0cd490 con 0x7f9f1c0b3b70 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f2285b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c0cd770 con 0x7f9f1c15e520 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f2285b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c0cda50 con 0x7f9f1c0ae200 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f1b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f1c0b3b70 0x7f9f1c15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f1b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f1c0b3b70 0x7f9f1c15a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43866/0 (socket says 172.21.15.47:43866) 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f1b7fe640 1 -- 172.21.15.47:0/1374254461 learned_addr learned my addr 172.21.15.47:0/1374254461 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f20df8640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f1c15e520 0x7f9f1c0af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f1bfff640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f1c0ae200 0x7f9f1c1518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f1b7fe640 1 -- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f1c0ae200 msgr2=0x7f9f1c1518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f1b7fe640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f1c0ae200 0x7f9f1c1518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f1b7fe640 1 -- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f1c15e520 msgr2=0x7f9f1c0af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f1b7fe640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f1c15e520 0x7f9f1c0af880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f1b7fe640 1 -- 172.21.15.47:0/1374254461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c0094d0 con 0x7f9f1c0b3b70 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f20df8640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f1c15e520 0x7f9f1c0af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f1bfff640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f1c0ae200 0x7f9f1c1518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f1b7fe640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f1c0b3b70 0x7f9f1c15a210 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000c410 tx=0x7f9f1000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f18ff9640 1 -- 172.21.15.47:0/1374254461 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f100035f0 con 0x7f9f1c0b3b70 2024-03-23T09:49:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f18ff9640 1 -- 172.21.15.47:0/1374254461 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f10003750 con 0x7f9f1c0b3b70 2024-03-23T09:49:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f2285b640 1 -- 172.21.15.47:0/1374254461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1c0cdd30 con 0x7f9f1c0b3b70 2024-03-23T09:49:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f18ff9640 1 -- 172.21.15.47:0/1374254461 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10003b40 con 0x7f9f1c0b3b70 2024-03-23T09:49:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.565+0000 7f9f2285b640 1 -- 172.21.15.47:0/1374254461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1c151fb0 con 0x7f9f1c0b3b70 2024-03-23T09:49:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.566+0000 7f9f2285b640 1 -- 172.21.15.47:0/1374254461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1c0cdf70 con 0x7f9f1c0b3b70 2024-03-23T09:49:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.567+0000 7f9f18ff9640 1 -- 172.21.15.47:0/1374254461 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9f10003ca0 con 0x7f9f1c0b3b70 2024-03-23T09:49:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.567+0000 7f9f18ff9640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f0806ac30 0x7f9f0806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.567+0000 7f9f18ff9640 1 -- 172.21.15.47:0/1374254461 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f9f10088770 con 0x7f9f1c0b3b70 2024-03-23T09:49:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.568+0000 7f9f1bfff640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f0806ac30 0x7f9f0806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:36.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.568+0000 7f9f1bfff640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f0806ac30 0x7f9f0806d0f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c036000 tx=0x7f9f0c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:36.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.571+0000 7f9f18ff9640 1 -- 172.21.15.47:0/1374254461 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9f1c0cdf70 con 0x7f9f1c0b3b70 2024-03-23T09:49:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:36.820+0000 7f9f2285b640 1 -- 172.21.15.47:0/1374254461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f9f1c0cd5d0 con 0x7f9f1c0b3b70 2024-03-23T09:49:37.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.449+0000 7f9f18ff9640 1 -- 172.21.15.47:0/1374254461 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v387) v1 ==== 149+0+0 (secure 0 0 0) 0x7f9f1c0cd5d0 con 0x7f9f1c0b3b70 2024-03-23T09:49:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.452+0000 7f9f2285b640 1 -- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f0806ac30 msgr2=0x7f9f0806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.452+0000 7f9f2285b640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f0806ac30 0x7f9f0806d0f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c036000 tx=0x7f9f0c03d040 comp rx=0 tx=0).stop 2024-03-23T09:49:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.452+0000 7f9f2285b640 1 -- 172.21.15.47:0/1374254461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f1c0b3b70 msgr2=0x7f9f1c15a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:37.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.452+0000 7f9f2285b640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f1c0b3b70 0x7f9f1c15a210 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000c410 tx=0x7f9f1000c8d0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.452+0000 7f9f2285b640 1 -- 172.21.15.47:0/1374254461 shutdown_connections 2024-03-23T09:49:37.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.452+0000 7f9f2285b640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9f0806ac30 0x7f9f0806d0f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.452+0000 7f9f2285b640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f1c15e520 0x7f9f1c0af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.452+0000 7f9f2285b640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f1c0b3b70 0x7f9f1c15a210 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.452+0000 7f9f2285b640 1 --2- 172.21.15.47:0/1374254461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f1c0ae200 0x7f9f1c1518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.452+0000 7f9f2285b640 1 -- 172.21.15.47:0/1374254461 >> 172.21.15.47:0/1374254461 conn(0x7f9f1c0ced80 msgr2=0x7f9f1c0b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:37.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.452+0000 7f9f2285b640 1 -- 172.21.15.47:0/1374254461 shutdown_connections 2024-03-23T09:49:37.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.452+0000 7f9f2285b640 1 -- 172.21.15.47:0/1374254461 wait complete. 2024-03-23T09:49:37.465 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-23T09:49:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.575+0000 7fa7bf699640 1 Processor -- start 2024-03-23T09:49:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.576+0000 7fa7bf699640 1 -- start start 2024-03-23T09:49:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.576+0000 7fa7bf699640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7b815ca30 0x7fa7b8160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.576+0000 7fa7bf699640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa7b815ce10 0x7fa7b80cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.576+0000 7fa7bf699640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b80d25e0 con 0x7fa7b80b85d0 2024-03-23T09:49:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.576+0000 7fa7bf699640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b8161600 con 0x7fa7b815ca30 2024-03-23T09:49:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.576+0000 7fa7bf699640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b8161040 con 0x7fa7b815ce10 2024-03-23T09:49:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.577+0000 7fa7bd435640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7b815ca30 0x7fa7b8160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.577+0000 7fa7bd435640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7b815ca30 0x7fa7b8160740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43878/0 (socket says 172.21.15.47:43878) 2024-03-23T09:49:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.577+0000 7fa7bd435640 1 -- 172.21.15.47:0/1490579212 learned_addr learned my addr 172.21.15.47:0/1490579212 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.577+0000 7fa7bcc34640 1 --2- 172.21.15.47:0/1490579212 >> v2:172.21.15.175:3300/0 conn(0x7fa7b815ce10 0x7fa7b80cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.577+0000 7fa7bd435640 1 -- 172.21.15.47:0/1490579212 >> v2:172.21.15.175:3300/0 conn(0x7fa7b815ce10 msgr2=0x7fa7b80cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.577+0000 7fa7bd435640 1 --2- 172.21.15.47:0/1490579212 >> v2:172.21.15.175:3300/0 conn(0x7fa7b815ce10 0x7fa7b80cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.577+0000 7fa7bd435640 1 -- 172.21.15.47:0/1490579212 >> v1:172.21.15.175:6789/0 conn(0x7fa7b80b85d0 legacy=0x7fa7b80b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.577+0000 7fa7bd435640 1 -- 172.21.15.47:0/1490579212 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b8161320 con 0x7fa7b815ca30 2024-03-23T09:49:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.578+0000 7fa7bd435640 1 --2- 172.21.15.47:0/1490579212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7b815ca30 0x7fa7b8160740 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8066930 tx=0x7fa7b002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bede997114fd3a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.578+0000 7fa7b7fff640 1 -- 172.21.15.47:0/1490579212 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0031870 con 0x7fa7b815ca30 2024-03-23T09:49:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.578+0000 7fa7b7fff640 1 -- 172.21.15.47:0/1490579212 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b00319d0 con 0x7fa7b815ca30 2024-03-23T09:49:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.578+0000 7fa7b7fff640 1 -- 172.21.15.47:0/1490579212 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0031d60 con 0x7fa7b815ca30 2024-03-23T09:49:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.578+0000 7fa7bf699640 1 -- 172.21.15.47:0/1490579212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7b815ca30 msgr2=0x7fa7b8160740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.578+0000 7fa7bf699640 1 --2- 172.21.15.47:0/1490579212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7b815ca30 0x7fa7b8160740 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8066930 tx=0x7fa7b002b910 comp rx=0 tx=0).stop 2024-03-23T09:49:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.578+0000 7fa7bf699640 1 -- 172.21.15.47:0/1490579212 shutdown_connections 2024-03-23T09:49:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.578+0000 7fa7bf699640 1 --2- 172.21.15.47:0/1490579212 >> v2:172.21.15.175:3300/0 conn(0x7fa7b815ce10 0x7fa7b80cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.578+0000 7fa7bf699640 1 --2- 172.21.15.47:0/1490579212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7b815ca30 0x7fa7b8160740 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.578+0000 7fa7bf699640 1 -- 172.21.15.47:0/1490579212 >> 172.21.15.47:0/1490579212 conn(0x7fa7b806b160 msgr2=0x7fa7b806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.578+0000 7fa7bf699640 1 -- 172.21.15.47:0/1490579212 shutdown_connections 2024-03-23T09:49:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.578+0000 7fa7bf699640 1 -- 172.21.15.47:0/1490579212 wait complete. 2024-03-23T09:49:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.580+0000 7fa7bf699640 1 Processor -- start 2024-03-23T09:49:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.580+0000 7fa7bf699640 1 -- start start 2024-03-23T09:49:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.580+0000 7fa7bf699640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b80b85d0 0x7fa7b8153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.580+0000 7fa7bf699640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7b815ca30 0x7fa7b815f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.580+0000 7fa7bf699640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b815ce10 0x7fa7b80b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.580+0000 7fa7bf699640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b815db50 con 0x7fa7b815ca30 2024-03-23T09:49:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.580+0000 7fa7bf699640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b815d240 con 0x7fa7b815ce10 2024-03-23T09:49:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.580+0000 7fa7bf699640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b81541b0 con 0x7fa7b80b85d0 2024-03-23T09:49:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.580+0000 7fa7bcc34640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7b815ca30 0x7fa7b815f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.580+0000 7fa7bcc34640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7b815ca30 0x7fa7b815f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43884/0 (socket says 172.21.15.47:43884) 2024-03-23T09:49:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.580+0000 7fa7bcc34640 1 -- 172.21.15.47:0/3985114714 learned_addr learned my addr 172.21.15.47:0/3985114714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7bdc36640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b815ce10 0x7fa7b80b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7bd435640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b80b85d0 0x7fa7b8153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7bcc34640 1 -- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b80b85d0 msgr2=0x7fa7b8153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7bcc34640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b80b85d0 0x7fa7b8153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7bcc34640 1 -- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b815ce10 msgr2=0x7fa7b80b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7bcc34640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b815ce10 0x7fa7b80b60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7bcc34640 1 -- 172.21.15.47:0/3985114714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b00094d0 con 0x7fa7b815ca30 2024-03-23T09:49:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7bdc36640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b815ce10 0x7fa7b80b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7bcc34640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7b815ca30 0x7fa7b815f320 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00d5f0 tx=0x7fa7ac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7b5ffb640 1 -- 172.21.15.47:0/3985114714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac0125f0 con 0x7fa7b815ca30 2024-03-23T09:49:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7b5ffb640 1 -- 172.21.15.47:0/3985114714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac012750 con 0x7fa7b815ca30 2024-03-23T09:49:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7bf699640 1 -- 172.21.15.47:0/3985114714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b8158f40 con 0x7fa7b815ca30 2024-03-23T09:49:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7b5ffb640 1 -- 172.21.15.47:0/3985114714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac012b00 con 0x7fa7b815ca30 2024-03-23T09:49:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.581+0000 7fa7bf699640 1 -- 172.21.15.47:0/3985114714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b815d490 con 0x7fa7b815ca30 2024-03-23T09:49:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.582+0000 7fa7bf699640 1 -- 172.21.15.47:0/3985114714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b8072360 con 0x7fa7b815ca30 2024-03-23T09:49:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.583+0000 7fa7b5ffb640 1 -- 172.21.15.47:0/3985114714 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa7ac012c60 con 0x7fa7b815ca30 2024-03-23T09:49:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.583+0000 7fa7b5ffb640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa7a406ac00 0x7fa7a406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.583+0000 7fa7b5ffb640 1 -- 172.21.15.47:0/3985114714 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa7ac0895b0 con 0x7fa7b815ca30 2024-03-23T09:49:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.583+0000 7fa7bd435640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa7a406ac00 0x7fa7a406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.584+0000 7fa7bd435640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa7a406ac00 0x7fa7a406d0c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0036000 tx=0x7fa7b003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.587+0000 7fa7b5ffb640 1 -- 172.21.15.47:0/3985114714 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa7ac0549e0 con 0x7fa7b815ca30 2024-03-23T09:49:37.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.766+0000 7fa7bf699640 1 -- 172.21.15.47:0/3985114714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa7b815dc90 con 0x7fa7b815ca30 2024-03-23T09:49:37.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.769+0000 7fa7b5ffb640 1 -- 172.21.15.47:0/3985114714 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa7b815dc90 con 0x7fa7b815ca30 2024-03-23T09:49:37.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.772+0000 7fa7bf699640 1 -- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa7a406ac00 msgr2=0x7fa7a406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:37.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.772+0000 7fa7bf699640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa7a406ac00 0x7fa7a406d0c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0036000 tx=0x7fa7b003c040 comp rx=0 tx=0).stop 2024-03-23T09:49:37.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.772+0000 7fa7bf699640 1 -- 172.21.15.47:0/3985114714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7b815ca30 msgr2=0x7fa7b815f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:37.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.772+0000 7fa7bf699640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7b815ca30 0x7fa7b815f320 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00d5f0 tx=0x7fa7ac00dab0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.772+0000 7fa7bf699640 1 -- 172.21.15.47:0/3985114714 shutdown_connections 2024-03-23T09:49:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.772+0000 7fa7bf699640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa7a406ac00 0x7fa7a406d0c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.772+0000 7fa7bf699640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b815ce10 0x7fa7b80b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.772+0000 7fa7bf699640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7b815ca30 0x7fa7b815f320 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.772+0000 7fa7bf699640 1 --2- 172.21.15.47:0/3985114714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b80b85d0 0x7fa7b8153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.772+0000 7fa7bf699640 1 -- 172.21.15.47:0/3985114714 >> 172.21.15.47:0/3985114714 conn(0x7fa7b806b160 msgr2=0x7fa7b815bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.772+0000 7fa7bf699640 1 -- 172.21.15.47:0/3985114714 shutdown_connections 2024-03-23T09:49:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:37.772+0000 7fa7bf699640 1 -- 172.21.15.47:0/3985114714 wait complete. 2024-03-23T09:49:37.783 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:49:37.783 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:49:37.822 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:49:37.897 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:49:37.897 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:49:37.897 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:49:37.912 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:49:37.913 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:49:37.913 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:49:37.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:49:37.930 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:49:37.930 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:49:37.967 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:49:37.968 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:49:37.968 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:49:37.986 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:49:37.987 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-23T09:49:37.987 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:49:38.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.109+0000 7f7880e70640 1 Processor -- start 2024-03-23T09:49:38.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.109+0000 7f7880e70640 1 -- start start 2024-03-23T09:49:38.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.110+0000 7f7880e70640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f787c15c9d0 0x7f787c071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:38.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.110+0000 7f7880e70640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f787c06a3f0 0x7f787c160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:38.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.110+0000 7f7880e70640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c0721f0 con 0x7f787c15cdb0 2024-03-23T09:49:38.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.110+0000 7f7880e70640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787c06eb80 con 0x7f787c15c9d0 2024-03-23T09:49:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.110+0000 7f7880e70640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787c06edd0 con 0x7f787c06a3f0 2024-03-23T09:49:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.110+0000 7f787a59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f787c15c9d0 0x7f787c071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.110+0000 7f787a59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f787c15c9d0 0x7f787c071cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43894/0 (socket says 172.21.15.47:43894) 2024-03-23T09:49:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.110+0000 7f787a59c640 1 -- 172.21.15.47:0/693077229 learned_addr learned my addr 172.21.15.47:0/693077229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.110+0000 7f787ad9d640 1 --2- 172.21.15.47:0/693077229 >> v2:172.21.15.175:3300/0 conn(0x7f787c06a3f0 0x7f787c160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.110+0000 7f787a59c640 1 -- 172.21.15.47:0/693077229 >> v2:172.21.15.175:3300/0 conn(0x7f787c06a3f0 msgr2=0x7f787c160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.110+0000 7f787a59c640 1 --2- 172.21.15.47:0/693077229 >> v2:172.21.15.175:3300/0 conn(0x7f787c06a3f0 0x7f787c160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.110+0000 7f787a59c640 1 -- 172.21.15.47:0/693077229 >> v1:172.21.15.175:6789/0 conn(0x7f787c15cdb0 legacy=0x7f787c069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.110+0000 7f787a59c640 1 -- 172.21.15.47:0/693077229 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c06f0b0 con 0x7f787c15c9d0 2024-03-23T09:49:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.111+0000 7f787a59c640 1 --2- 172.21.15.47:0/693077229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c15c9d0 0x7f787c071cb0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f787c06e460 tx=0x7f786c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2422e0accb0ddfba server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.111+0000 7f787959a640 1 -- 172.21.15.47:0/693077229 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c02ca80 con 0x7f787c15c9d0 2024-03-23T09:49:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.111+0000 7f787959a640 1 -- 172.21.15.47:0/693077229 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c02cbe0 con 0x7f787c15c9d0 2024-03-23T09:49:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.111+0000 7f787959a640 1 -- 172.21.15.47:0/693077229 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c0343f0 con 0x7f787c15c9d0 2024-03-23T09:49:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.112+0000 7f7880e70640 1 -- 172.21.15.47:0/693077229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c15c9d0 msgr2=0x7f787c071cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.112+0000 7f7880e70640 1 --2- 172.21.15.47:0/693077229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c15c9d0 0x7f787c071cb0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f787c06e460 tx=0x7f786c02bee0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.112+0000 7f7880e70640 1 -- 172.21.15.47:0/693077229 shutdown_connections 2024-03-23T09:49:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.112+0000 7f7880e70640 1 --2- 172.21.15.47:0/693077229 >> v2:172.21.15.175:3300/0 conn(0x7f787c06a3f0 0x7f787c160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.112+0000 7f7880e70640 1 --2- 172.21.15.47:0/693077229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c15c9d0 0x7f787c071cb0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.112+0000 7f7880e70640 1 -- 172.21.15.47:0/693077229 >> 172.21.15.47:0/693077229 conn(0x7f787c0b13a0 msgr2=0x7f787c0b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:38.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.112+0000 7f7880e70640 1 -- 172.21.15.47:0/693077229 shutdown_connections 2024-03-23T09:49:38.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.112+0000 7f7880e70640 1 -- 172.21.15.47:0/693077229 wait complete. 2024-03-23T09:49:38.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.113+0000 7f7880e70640 1 Processor -- start 2024-03-23T09:49:38.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.113+0000 7f7880e70640 1 -- start start 2024-03-23T09:49:38.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f7880e70640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c06a3f0 0x7f787c14e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:38.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f7880e70640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f787c15c9d0 0x7f787c0ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f7880e70640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f787c15cdb0 0x7f787c0ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f7880e70640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c14ed30 con 0x7f787c06a3f0 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f7880e70640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c15efb0 con 0x7f787c15c9d0 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f7880e70640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c15f290 con 0x7f787c15cdb0 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f787a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c06a3f0 0x7f787c14e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f787a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c06a3f0 0x7f787c14e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43900/0 (socket says 172.21.15.47:43900) 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f787a59c640 1 -- 172.21.15.47:0/2472215997 learned_addr learned my addr 172.21.15.47:0/2472215997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f787a59c640 1 -- 172.21.15.47:0/2472215997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f787c15cdb0 msgr2=0x7f787c0ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f787a59c640 1 --2- 172.21.15.47:0/2472215997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f787c15cdb0 0x7f787c0ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f787a59c640 1 -- 172.21.15.47:0/2472215997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f787c15c9d0 msgr2=0x7f787c0ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f787a59c640 1 --2- 172.21.15.47:0/2472215997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f787c15c9d0 0x7f787c0ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f787a59c640 1 -- 172.21.15.47:0/2472215997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c009b30 con 0x7f787c06a3f0 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.114+0000 7f787a59c640 1 --2- 172.21.15.47:0/2472215997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c06a3f0 0x7f787c14e7f0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f786c000f80 tx=0x7f786c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.115+0000 7f7872ffd640 1 -- 172.21.15.47:0/2472215997 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c009510 con 0x7f787c06a3f0 2024-03-23T09:49:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.115+0000 7f7872ffd640 1 -- 172.21.15.47:0/2472215997 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c03e450 con 0x7f787c06a3f0 2024-03-23T09:49:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.115+0000 7f7880e70640 1 -- 172.21.15.47:0/2472215997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c15f570 con 0x7f787c06a3f0 2024-03-23T09:49:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.115+0000 7f7872ffd640 1 -- 172.21.15.47:0/2472215997 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c03e800 con 0x7f787c06a3f0 2024-03-23T09:49:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.115+0000 7f7880e70640 1 -- 172.21.15.47:0/2472215997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c15fb30 con 0x7f787c06a3f0 2024-03-23T09:49:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.116+0000 7f7880e70640 1 -- 172.21.15.47:0/2472215997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c066be0 con 0x7f787c06a3f0 2024-03-23T09:49:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.116+0000 7f7872ffd640 1 -- 172.21.15.47:0/2472215997 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f786c02cd50 con 0x7f787c06a3f0 2024-03-23T09:49:38.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.117+0000 7f7872ffd640 1 --2- 172.21.15.47:0/2472215997 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f785c06ac00 0x7f785c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:38.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.117+0000 7f7872ffd640 1 -- 172.21.15.47:0/2472215997 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f786c0abc50 con 0x7f787c06a3f0 2024-03-23T09:49:38.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.117+0000 7f7879d9b640 1 --2- 172.21.15.47:0/2472215997 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f785c06ac00 0x7f785c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:38.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.118+0000 7f7879d9b640 1 --2- 172.21.15.47:0/2472215997 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f785c06ac00 0x7f785c06d0c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7864006fd0 tx=0x7f7864006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:38.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.121+0000 7f7872ffd640 1 -- 172.21.15.47:0/2472215997 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f786c0b1020 con 0x7f787c06a3f0 2024-03-23T09:49:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.337+0000 7f7880e70640 1 -- 172.21.15.47:0/2472215997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f787c0717c0 con 0x7f787c06a3f0 2024-03-23T09:49:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.337+0000 7f7872ffd640 1 -- 172.21.15.47:0/2472215997 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f786c077110 con 0x7f787c06a3f0 2024-03-23T09:49:38.338 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:38.338 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":61,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/1369976949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1369976949},{"type":"v1","addr":"172.21.15.47:6833","nonce":1369976949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1488187609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1488187609},{"type":"v1","addr":"172.21.15.175:6836","nonce":1488187609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/275195813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":275195813},{"type":"v1","addr":"172.21.15.175:6837","nonce":275195813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1475945190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1475945190},{"type":"v1","addr":"172.21.15.47:6835","nonce":1475945190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[]} 2024-03-23T09:49:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 61 2024-03-23T09:49:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.339+0000 7f7880e70640 1 -- 172.21.15.47:0/2472215997 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f785c06ac00 msgr2=0x7f785c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.339+0000 7f7880e70640 1 --2- 172.21.15.47:0/2472215997 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f785c06ac00 0x7f785c06d0c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7864006fd0 tx=0x7f7864006bc0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.339+0000 7f7880e70640 1 -- 172.21.15.47:0/2472215997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c06a3f0 msgr2=0x7f787c14e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.339+0000 7f7880e70640 1 --2- 172.21.15.47:0/2472215997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c06a3f0 0x7f787c14e7f0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f786c000f80 tx=0x7f786c004a40 comp rx=0 tx=0).stop 2024-03-23T09:49:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.339+0000 7f7880e70640 1 -- 172.21.15.47:0/2472215997 shutdown_connections 2024-03-23T09:49:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.339+0000 7f7880e70640 1 --2- 172.21.15.47:0/2472215997 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f785c06ac00 0x7f785c06d0c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.339+0000 7f7880e70640 1 --2- 172.21.15.47:0/2472215997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f787c15cdb0 0x7f787c0ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.339+0000 7f7880e70640 1 --2- 172.21.15.47:0/2472215997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f787c15c9d0 0x7f787c0ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.339+0000 7f7880e70640 1 --2- 172.21.15.47:0/2472215997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f787c06a3f0 0x7f787c14e7f0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.339+0000 7f7880e70640 1 -- 172.21.15.47:0/2472215997 >> 172.21.15.47:0/2472215997 conn(0x7f787c0b13a0 msgr2=0x7f787c0cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.339+0000 7f7880e70640 1 -- 172.21.15.47:0/2472215997 shutdown_connections 2024-03-23T09:49:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.339+0000 7f7880e70640 1 -- 172.21.15.47:0/2472215997 wait complete. 2024-03-23T09:49:38.350 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-23T09:49:38.350 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-23T09:49:38.351 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-23T09:49:38.351 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:49:38.351 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-23T09:49:38.351 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-23T09:49:38.351 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-23T09:49:38.351 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:49:38.352 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-23T09:49:38.352 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-23T09:49:38.352 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-23T09:49:38.352 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:49:38.352 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-23T09:49:38.352 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-23T09:49:38.352 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-23T09:49:38.352 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:49:38.359 INFO:tasks.ceph.mds.a:Stopped 2024-03-23T09:49:38.360 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-23T09:49:38.360 INFO:tasks.ceph.mds.c:Stopped 2024-03-23T09:49:38.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-23T09:49:38.363 INFO:tasks.ceph.mds.b:Stopped 2024-03-23T09:49:38.363 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-23T09:49:38.364 INFO:tasks.ceph.mds.d:Stopped 2024-03-23T09:49:38.365 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-23T09:49:38.402 INFO:tasks.ceph.mds.a:Started 2024-03-23T09:49:38.403 DEBUG:teuthology.parallel:result is None 2024-03-23T09:49:38.403 INFO:tasks.ceph.mds.c:Started 2024-03-23T09:49:38.403 DEBUG:teuthology.parallel:result is None 2024-03-23T09:49:38.406 INFO:tasks.ceph.mds.b:Started 2024-03-23T09:49:38.406 DEBUG:teuthology.parallel:result is None 2024-03-23T09:49:38.407 INFO:tasks.ceph.mds.d:Started 2024-03-23T09:49:38.407 DEBUG:teuthology.parallel:result is None 2024-03-23T09:49:38.407 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-23T09:49:38.479 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-03-23T09:49:38.479 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:49:38.494 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-03-23T09:49:38.496 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-03-23T09:49:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.520+0000 7f553fe2c640 1 Processor -- start 2024-03-23T09:49:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.520+0000 7f553fe2c640 1 -- start start 2024-03-23T09:49:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.521+0000 7f553fe2c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f553806bbe0 0x7f55380cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.521+0000 7f553fe2c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f553806ae90 con 0x7f5538098a00 2024-03-23T09:49:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.521+0000 7f553fe2c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f553806b100 con 0x7f55380b1600 2024-03-23T09:49:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.521+0000 7f553fe2c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553806b3e0 con 0x7f553806bbe0 2024-03-23T09:49:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.521+0000 7f553dbc8640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5538098a00 0x7f5538098de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44582/0 (socket says 172.21.15.47:44582) 2024-03-23T09:49:38.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.521+0000 7f553dbc8640 1 -- 172.21.15.47:0/4078430637 learned_addr learned my addr 172.21.15.47:0/4078430637 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:38.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.521+0000 7f553e3c9640 1 --2- 172.21.15.47:0/4078430637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f553806bbe0 0x7f55380cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:38.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.521+0000 7f553e3c9640 1 -- 172.21.15.47:0/4078430637 >> v1:172.21.15.175:6789/0 conn(0x7f55380b1600 legacy=0x7f55380b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:38.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.521+0000 7f553e3c9640 1 -- 172.21.15.47:0/4078430637 >> v1:172.21.15.47:6789/0 conn(0x7f5538098a00 legacy=0x7f5538098de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:38.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.521+0000 7f553e3c9640 1 -- 172.21.15.47:0/4078430637 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553806b6c0 con 0x7f553806bbe0 2024-03-23T09:49:38.522 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:49:38.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.522+0000 7f553e3c9640 1 --2- 172.21.15.47:0/4078430637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f553806bbe0 0x7f55380cea40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5530009dc0 tx=0x7f553002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f5b0cda47e63a89 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:38.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:49:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.529+0000 7f553cbc6640 1 -- 172.21.15.47:0/4078430637 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553002ccd0 con 0x7f553806bbe0 2024-03-23T09:49:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.529+0000 7f553cbc6640 1 -- 172.21.15.47:0/4078430637 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553002ce30 con 0x7f553806bbe0 2024-03-23T09:49:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.529+0000 7f553cbc6640 1 -- 172.21.15.47:0/4078430637 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530034660 con 0x7f553806bbe0 2024-03-23T09:49:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.529+0000 7f553fe2c640 1 -- 172.21.15.47:0/4078430637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f553806bbe0 msgr2=0x7f55380cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.529+0000 7f553fe2c640 1 --2- 172.21.15.47:0/4078430637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f553806bbe0 0x7f55380cea40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5530009dc0 tx=0x7f553002bf20 comp rx=0 tx=0).stop 2024-03-23T09:49:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.530+0000 7f553fe2c640 1 -- 172.21.15.47:0/4078430637 shutdown_connections 2024-03-23T09:49:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.530+0000 7f553fe2c640 1 --2- 172.21.15.47:0/4078430637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f553806bbe0 0x7f55380cea40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.530+0000 7f553fe2c640 1 -- 172.21.15.47:0/4078430637 >> 172.21.15.47:0/4078430637 conn(0x7f55380b21c0 msgr2=0x7f55380b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.530+0000 7f553fe2c640 1 -- 172.21.15.47:0/4078430637 shutdown_connections 2024-03-23T09:49:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.530+0000 7f553fe2c640 1 -- 172.21.15.47:0/4078430637 wait complete. 2024-03-23T09:49:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.531+0000 7f553fe2c640 1 Processor -- start 2024-03-23T09:49:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.531+0000 7f553fe2c640 1 -- start start 2024-03-23T09:49:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553fe2c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553806bbe0 0x7f553814caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553fe2c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5538098a00 0x7f55380b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553fe2c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55380b1600 0x7f553815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553fe2c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380b5d90 con 0x7f553806bbe0 2024-03-23T09:49:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553fe2c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380b5ab0 con 0x7f5538098a00 2024-03-23T09:49:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553fe2c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55380c5030 con 0x7f55380b1600 2024-03-23T09:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553dbc8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553806bbe0 0x7f553814caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553dbc8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553806bbe0 0x7f553814caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43938/0 (socket says 172.21.15.47:43938) 2024-03-23T09:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553dbc8640 1 -- 172.21.15.47:0/1069572088 learned_addr learned my addr 172.21.15.47:0/1069572088 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553d3c7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5538098a00 0x7f55380b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553e3c9640 1 --2- 172.21.15.47:0/1069572088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55380b1600 0x7f553815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553dbc8640 1 -- 172.21.15.47:0/1069572088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55380b1600 msgr2=0x7f553815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553dbc8640 1 --2- 172.21.15.47:0/1069572088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55380b1600 0x7f553815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553dbc8640 1 -- 172.21.15.47:0/1069572088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5538098a00 msgr2=0x7f55380b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553dbc8640 1 --2- 172.21.15.47:0/1069572088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5538098a00 0x7f55380b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.532+0000 7f553dbc8640 1 -- 172.21.15.47:0/1069572088 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5530009a70 con 0x7f553806bbe0 2024-03-23T09:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.533+0000 7f553dbc8640 1 --2- 172.21.15.47:0/1069572088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553806bbe0 0x7f553814caa0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f553400c4a0 tx=0x7f553400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.533+0000 7f552e7fc640 1 -- 172.21.15.47:0/1069572088 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534005ce0 con 0x7f553806bbe0 2024-03-23T09:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.533+0000 7f552e7fc640 1 -- 172.21.15.47:0/1069572088 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5534005e40 con 0x7f553806bbe0 2024-03-23T09:49:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.533+0000 7f553fe2c640 1 -- 172.21.15.47:0/1069572088 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55380c5310 con 0x7f553806bbe0 2024-03-23T09:49:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.533+0000 7f552e7fc640 1 -- 172.21.15.47:0/1069572088 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534007270 con 0x7f553806bbe0 2024-03-23T09:49:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.533+0000 7f553fe2c640 1 -- 172.21.15.47:0/1069572088 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553814cfe0 con 0x7f553806bbe0 2024-03-23T09:49:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.534+0000 7f553fe2c640 1 -- 172.21.15.47:0/1069572088 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55380c4e90 con 0x7f553806bbe0 2024-03-23T09:49:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.535+0000 7f552e7fc640 1 -- 172.21.15.47:0/1069572088 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5534007490 con 0x7f553806bbe0 2024-03-23T09:49:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.535+0000 7f552e7fc640 1 --2- 172.21.15.47:0/1069572088 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f551c06ac00 0x7f551c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.535+0000 7f552e7fc640 1 -- 172.21.15.47:0/1069572088 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f5534088550 con 0x7f553806bbe0 2024-03-23T09:49:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.535+0000 7f553d3c7640 1 --2- 172.21.15.47:0/1069572088 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f551c06ac00 0x7f551c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.536+0000 7f553d3c7640 1 --2- 172.21.15.47:0/1069572088 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f551c06ac00 0x7f551c06d0c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f553815ffa0 tx=0x7f5528009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.539+0000 7f552e7fc640 1 -- 172.21.15.47:0/1069572088 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f55380c4e90 con 0x7f553806bbe0 2024-03-23T09:49:38.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:38.731+0000 7f553fe2c640 1 -- 172.21.15.47:0/1069572088 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f55380c4e90 con 0x7f553806bbe0 2024-03-23T09:49:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.525+0000 7f552e7fc640 1 -- 172.21.15.47:0/1069572088 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) v1 ==== 97+0+0 (secure 0 0 0) 0x7f55380c4e90 con 0x7f553806bbe0 2024-03-23T09:49:39.525 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blocklist entries 2024-03-23T09:49:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.527+0000 7f553fe2c640 1 -- 172.21.15.47:0/1069572088 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f551c06ac00 msgr2=0x7f551c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.527+0000 7f553fe2c640 1 --2- 172.21.15.47:0/1069572088 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f551c06ac00 0x7f551c06d0c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f553815ffa0 tx=0x7f5528009040 comp rx=0 tx=0).stop 2024-03-23T09:49:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.527+0000 7f553fe2c640 1 -- 172.21.15.47:0/1069572088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553806bbe0 msgr2=0x7f553814caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.527+0000 7f553fe2c640 1 --2- 172.21.15.47:0/1069572088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553806bbe0 0x7f553814caa0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f553400c4a0 tx=0x7f553400c960 comp rx=0 tx=0).stop 2024-03-23T09:49:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.527+0000 7f553fe2c640 1 -- 172.21.15.47:0/1069572088 shutdown_connections 2024-03-23T09:49:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.527+0000 7f553fe2c640 1 --2- 172.21.15.47:0/1069572088 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f551c06ac00 0x7f551c06d0c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.527+0000 7f553fe2c640 1 --2- 172.21.15.47:0/1069572088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55380b1600 0x7f553815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.527+0000 7f553fe2c640 1 --2- 172.21.15.47:0/1069572088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5538098a00 0x7f55380b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.527+0000 7f553fe2c640 1 --2- 172.21.15.47:0/1069572088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553806bbe0 0x7f553814caa0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.527+0000 7f553fe2c640 1 -- 172.21.15.47:0/1069572088 >> 172.21.15.47:0/1069572088 conn(0x7f55380b21c0 msgr2=0x7f553806f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.528+0000 7f553fe2c640 1 -- 172.21.15.47:0/1069572088 shutdown_connections 2024-03-23T09:49:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.528+0000 7f553fe2c640 1 -- 172.21.15.47:0/1069572088 wait complete. 2024-03-23T09:49:39.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-23T09:49:39.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.647+0000 7f785b0b6640 1 Processor -- start 2024-03-23T09:49:39.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.647+0000 7f785b0b6640 1 -- start start 2024-03-23T09:49:39.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.648+0000 7f785b0b6640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78540b8300 0x7f78540cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:39.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.648+0000 7f785b0b6640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78540d0ee0 con 0x7f78540cf6d0 2024-03-23T09:49:39.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.648+0000 7f785b0b6640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78540cfb20 con 0x7f785415cb40 2024-03-23T09:49:39.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.648+0000 7f785b0b6640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78540b5a10 con 0x7f78540b8300 2024-03-23T09:49:39.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.648+0000 7f7859653640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f78540cf6d0 0x7f78540d1370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44588/0 (socket says 172.21.15.47:44588) 2024-03-23T09:49:39.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.648+0000 7f7859653640 1 -- 172.21.15.47:0/1979202709 learned_addr learned my addr 172.21.15.47:0/1979202709 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:39.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.648+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641391669 0 0) 0x7f78540d0ee0 con 0x7f78540cf6d0 2024-03-23T09:49:39.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.648+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f783c003520 con 0x7f78540cf6d0 2024-03-23T09:49:39.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.648+0000 7f7853fff640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78540b8300 0x7f78540cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:39.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2605969929 0 0) 0x7f783c003520 con 0x7f78540cf6d0 2024-03-23T09:49:39.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78540b8300 msgr2=0x7f78540cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:39.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f78537fe640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78540b8300 0x7f78540cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 >> v1:172.21.15.175:6789/0 conn(0x7f785415cb40 legacy=0x7f7854160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:39.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78541616f0 con 0x7f78540cf6d0 2024-03-23T09:49:39.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f7844001670 con 0x7f78540cf6d0 2024-03-23T09:49:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f7853fff640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78540b8300 0x7f78540cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 >> v1:172.21.15.47:6789/0 conn(0x7f78540cf6d0 legacy=0x7f78540d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f78537fe640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f783c0039a0 0x7f783c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f78537fe640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78540b8300 0x7f783c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f78537fe640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c004d00 0x7f783c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78540b5a10 con 0x7f783c004d00 2024-03-23T09:49:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c00bac0 con 0x7f783c0039a0 2024-03-23T09:49:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783c00bc80 con 0x7f78540b8300 2024-03-23T09:49:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f7859653640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c004d00 0x7f783c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f7858e52640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f783c0039a0 0x7f783c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f7853fff640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78540b8300 0x7f783c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f7859653640 1 -- 172.21.15.47:0/1979202709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78540b8300 msgr2=0x7f783c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f7859653640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78540b8300 0x7f783c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f7859653640 1 -- 172.21.15.47:0/1979202709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f783c0039a0 msgr2=0x7f783c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f7859653640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f783c0039a0 0x7f783c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f7859653640 1 -- 172.21.15.47:0/1979202709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c003520 con 0x7f783c004d00 2024-03-23T09:49:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.649+0000 7f7853fff640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78540b8300 0x7f783c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.650+0000 7f7859653640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c004d00 0x7f783c005180 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f784400c210 tx=0x7f783c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a95d1003966a3f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.650+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844005ad0 con 0x7f783c004d00 2024-03-23T09:49:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.650+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7844005c30 con 0x7f783c004d00 2024-03-23T09:49:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.650+0000 7f78537fe640 1 -- 172.21.15.47:0/1979202709 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784402c890 con 0x7f783c004d00 2024-03-23T09:49:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.651+0000 7f785b0b6640 1 -- 172.21.15.47:0/1979202709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c004d00 msgr2=0x7f783c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.651+0000 7f785b0b6640 1 --2- 172.21.15.47:0/1979202709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c004d00 0x7f783c005180 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f784400c210 tx=0x7f783c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.651+0000 7f7859653640 1 -- 172.21.15.47:0/1979202709 reap_dead start 2024-03-23T09:49:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.651+0000 7f785b0b6640 1 -- 172.21.15.47:0/1979202709 shutdown_connections 2024-03-23T09:49:39.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.651+0000 7f785b0b6640 1 -- 172.21.15.47:0/1979202709 >> 172.21.15.47:0/1979202709 conn(0x7f785406b160 msgr2=0x7f785406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:39.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.651+0000 7f785b0b6640 1 -- 172.21.15.47:0/1979202709 shutdown_connections 2024-03-23T09:49:39.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.651+0000 7f785b0b6640 1 -- 172.21.15.47:0/1979202709 wait complete. 2024-03-23T09:49:39.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.652+0000 7f785b0b6640 1 Processor -- start 2024-03-23T09:49:39.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.652+0000 7f785b0b6640 1 -- start start 2024-03-23T09:49:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f785b0b6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78541537d0 0x7f7854151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f785b0b6640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7854154930 0x7f785414ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f785b0b6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7854153bb0 0x7f7854154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f785b0b6640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78540cb410 con 0x7f7854153bb0 2024-03-23T09:49:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f785b0b6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7854154db0 con 0x7f78541537d0 2024-03-23T09:49:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f785b0b6640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785414e280 con 0x7f7854154930 2024-03-23T09:49:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f7859653640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7854153bb0 0x7f7854154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f7859653640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7854153bb0 0x7f7854154030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43970/0 (socket says 172.21.15.47:43970) 2024-03-23T09:49:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f7859653640 1 -- 172.21.15.47:0/96106518 learned_addr learned my addr 172.21.15.47:0/96106518 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f7858e52640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78541537d0 0x7f7854151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f7853fff640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7854154930 0x7f785414ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f7859653640 1 -- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7854154930 msgr2=0x7f785414ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f7859653640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7854154930 0x7f785414ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f7859653640 1 -- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78541537d0 msgr2=0x7f7854151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f7859653640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78541537d0 0x7f7854151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f7859653640 1 -- 172.21.15.47:0/96106518 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7854152140 con 0x7f7854153bb0 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.653+0000 7f7859653640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7854153bb0 0x7f7854154030 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f784400c210 tx=0x7f7844001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.654+0000 7f78517fa640 1 -- 172.21.15.47:0/96106518 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844004aa0 con 0x7f7854153bb0 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.654+0000 7f78517fa640 1 -- 172.21.15.47:0/96106518 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7844004c00 con 0x7f7854153bb0 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.654+0000 7f785b0b6640 1 -- 172.21.15.47:0/96106518 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78541510b0 con 0x7f7854153bb0 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.654+0000 7f78517fa640 1 -- 172.21.15.47:0/96106518 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844001640 con 0x7f7854153bb0 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.654+0000 7f785b0b6640 1 -- 172.21.15.47:0/96106518 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7854151670 con 0x7f7854153bb0 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.655+0000 7f785b0b6640 1 -- 172.21.15.47:0/96106518 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7854072360 con 0x7f7854153bb0 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.655+0000 7f78517fa640 1 -- 172.21.15.47:0/96106518 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7844032070 con 0x7f7854153bb0 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.655+0000 7f78517fa640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f784006ac00 0x7f784006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.655+0000 7f78517fa640 1 -- 172.21.15.47:0/96106518 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f784403c070 con 0x7f7854153bb0 2024-03-23T09:49:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.656+0000 7f7858e52640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f784006ac00 0x7f784006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.656+0000 7f7858e52640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f784006ac00 0x7f784006d0c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f784c001e60 tx=0x7f784c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.659+0000 7f78517fa640 1 -- 172.21.15.47:0/96106518 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f78440756b0 con 0x7f7854153bb0 2024-03-23T09:49:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.905+0000 7f785b0b6640 1 -- 172.21.15.47:0/96106518 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f78540cedf0 con 0x7f7854153bb0 2024-03-23T09:49:39.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.906+0000 7f78517fa640 1 -- 172.21.15.47:0/96106518 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7844039020 con 0x7f7854153bb0 2024-03-23T09:49:39.906 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:39.906 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:49:39.906 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2024-03-23T09:49:39.906 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lo+xoDhAAyxqr1H6eiINwt2X7LUlrSg==", 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZ5OlGhAAuUElzRghNRFGKREdCpxKvA==", 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.907 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lFwyzFRAAKldMOA7oSblA8YMrvntY8w==", 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.d", 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lpiD3IRAApXAdUQ4AkxWHzsFV0qcBng==", 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB7o/5lLvoADBAAgdixkm+jHwQilYw0IAn6EA==", 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:49:39.909 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:49:39.910 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.910 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.910 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.910 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2024-03-23T09:49:39.910 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB+o/5lOBfGCRAApglzalO1ztnJzOKg/UAyWA==", 2024-03-23T09:49:39.910 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.910 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:49:39.910 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:49:39.910 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.910 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.910 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCBo/5lRBjlBxAAd45w8/kTMPqxmNyfCS3XYA==", 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCEo/5l+JxrBhAAbR/XqDyPDnN8eLpmvBZ7Mg==", 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:49:39.911 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCKo/5ltsqwDRAAkqcZ/sEf2oozWFN4AXg/+w==", 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCNo/5lpLHiCxAAB2axUv9gjcYVlij9f+AEHg==", 2024-03-23T09:49:39.912 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCQo/5l2io3ChAAu+lfYcWWBQf43aYzWjCBNg==", 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2024-03-23T09:49:39.913 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCTo/5lA7jjCBAAjJgBkp89H8lx6fidl/Z7IA==", 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZFc3JhAAZWmDZOnkW9eo9hqEDYynQw==", 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lIrJSLxAAPJer3ZP3h8VkImkiJmnKVw==", 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l1+DyKxAAZLKZNbUpUJ0fWPIYNvWK+g==", 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:49:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lWqkiNRAAL1q0lEcM/NKW3Pf9LcP21g==", 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2024-03-23T09:49:39.916 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB4o/5lyeuKLRAARrfTpPc5BTVYZe/LUREDkg==", 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l/0EOCxAAIrgIXRiFSQ1G7Jat5hNm4A==", 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:39.917 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:39.918 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2024-03-23T09:49:39.918 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lywMFBxAAIVwjCe6B9EtBMAG2n7cZJw==", 2024-03-23T09:49:39.918 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:49:39.918 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:49:39.918 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:49:39.918 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:49:39.918 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.918 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:39.918 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-23T09:49:39.918 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:49:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.908+0000 7f785b0b6640 1 -- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f784006ac00 msgr2=0x7f784006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.908+0000 7f785b0b6640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f784006ac00 0x7f784006d0c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f784c001e60 tx=0x7f784c009040 comp rx=0 tx=0).stop 2024-03-23T09:49:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.908+0000 7f785b0b6640 1 -- 172.21.15.47:0/96106518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7854153bb0 msgr2=0x7f7854154030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.908+0000 7f785b0b6640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7854153bb0 0x7f7854154030 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f784400c210 tx=0x7f7844001a00 comp rx=0 tx=0).stop 2024-03-23T09:49:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.908+0000 7f785b0b6640 1 -- 172.21.15.47:0/96106518 shutdown_connections 2024-03-23T09:49:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.908+0000 7f785b0b6640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f784006ac00 0x7f784006d0c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.908+0000 7f785b0b6640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7854153bb0 0x7f7854154030 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.908+0000 7f785b0b6640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7854154930 0x7f785414ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.908+0000 7f785b0b6640 1 --2- 172.21.15.47:0/96106518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78541537d0 0x7f7854151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.908+0000 7f785b0b6640 1 -- 172.21.15.47:0/96106518 >> 172.21.15.47:0/96106518 conn(0x7f785406b160 msgr2=0x7f78540b7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.908+0000 7f785b0b6640 1 -- 172.21.15.47:0/96106518 shutdown_connections 2024-03-23T09:49:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:39.908+0000 7f785b0b6640 1 -- 172.21.15.47:0/96106518 wait complete. 2024-03-23T09:49:39.921 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:49:40.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.022+0000 7f0a12807640 1 Processor -- start 2024-03-23T09:49:40.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.022+0000 7f0a12807640 1 -- start start 2024-03-23T09:49:40.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.022+0000 7f0a12807640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0c0cf210 0x7f0a0c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.022+0000 7f0a12807640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c0cfbd0 con 0x7f0a0c0b5a90 2024-03-23T09:49:40.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.022+0000 7f0a12807640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c15d0e0 con 0x7f0a0c15cd00 2024-03-23T09:49:40.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.022+0000 7f0a12807640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0d1540 con 0x7f0a0c0cf210 2024-03-23T09:49:40.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a10da4640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0a0c0b5a90 0x7f0a0c0b5f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44592/0 (socket says 172.21.15.47:44592) 2024-03-23T09:49:40.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a10da4640 1 -- 172.21.15.47:0/1857653386 learned_addr learned my addr 172.21.15.47:0/1857653386 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:40.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27690324 0 0) 0x7f0a0c0cfbd0 con 0x7f0a0c0b5a90 2024-03-23T09:49:40.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f4003520 con 0x7f0a0c0b5a90 2024-03-23T09:49:40.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0b7fe640 1 --2- 172.21.15.47:0/1857653386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0c0cf210 0x7f0a0c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1178256568 0 0) 0x7f09f4003520 con 0x7f0a0c0b5a90 2024-03-23T09:49:40.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0c0cf210 msgr2=0x7f0a0c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 --2- 172.21.15.47:0/1857653386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0c0cf210 0x7f0a0c0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 >> v1:172.21.15.175:6789/0 conn(0x7f0a0c15cd00 legacy=0x7f0a0c0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0d1820 con 0x7f0a0c0b5a90 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0a040019f0 con 0x7f0a0c0b5a90 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 >> v1:172.21.15.47:6789/0 conn(0x7f0a0c0b5a90 legacy=0x7f0a0c0b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 --2- 172.21.15.47:0/1857653386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f40039a0 0x7f09f4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 --2- 172.21.15.47:0/1857653386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0c0cf210 0x7f09f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 --2- 172.21.15.47:0/1857653386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f4004d00 0x7f09f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0d1540 con 0x7f09f4004d00 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f400bb60 con 0x7f09f40039a0 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f400bd20 con 0x7f0a0c0cf210 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a10da4640 1 --2- 172.21.15.47:0/1857653386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f4004d00 0x7f09f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0bfff640 1 --2- 172.21.15.47:0/1857653386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f40039a0 0x7f09f4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.023+0000 7f0a0b7fe640 1 --2- 172.21.15.47:0/1857653386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0c0cf210 0x7f09f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.024+0000 7f0a10da4640 1 -- 172.21.15.47:0/1857653386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0c0cf210 msgr2=0x7f09f40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.024+0000 7f0a10da4640 1 --2- 172.21.15.47:0/1857653386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0c0cf210 0x7f09f40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.024+0000 7f0a10da4640 1 -- 172.21.15.47:0/1857653386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f40039a0 msgr2=0x7f09f4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.024+0000 7f0a10da4640 1 --2- 172.21.15.47:0/1857653386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f40039a0 0x7f09f4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.024+0000 7f0a10da4640 1 -- 172.21.15.47:0/1857653386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f4003520 con 0x7f09f4004d00 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.025+0000 7f0a10da4640 1 --2- 172.21.15.47:0/1857653386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f4004d00 0x7f09f4005180 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400a830 tx=0x7f09f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf9c19c044788b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.025+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a040072d0 con 0x7f09f4004d00 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.025+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0402f070 con 0x7f09f4004d00 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.025+0000 7f0a0affd640 1 -- 172.21.15.47:0/1857653386 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a040076b0 con 0x7f09f4004d00 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.025+0000 7f0a12807640 1 -- 172.21.15.47:0/1857653386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f4004d00 msgr2=0x7f09f4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.025+0000 7f0a12807640 1 --2- 172.21.15.47:0/1857653386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f4004d00 0x7f09f4005180 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400a830 tx=0x7f09f40057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.025+0000 7f0a10da4640 1 -- 172.21.15.47:0/1857653386 reap_dead start 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.025+0000 7f0a12807640 1 -- 172.21.15.47:0/1857653386 shutdown_connections 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.025+0000 7f0a12807640 1 -- 172.21.15.47:0/1857653386 >> 172.21.15.47:0/1857653386 conn(0x7f0a0c06b160 msgr2=0x7f0a0c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.025+0000 7f0a12807640 1 -- 172.21.15.47:0/1857653386 shutdown_connections 2024-03-23T09:49:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.025+0000 7f0a12807640 1 -- 172.21.15.47:0/1857653386 wait complete. 2024-03-23T09:49:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.026+0000 7f0a12807640 1 Processor -- start 2024-03-23T09:49:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.027+0000 7f0a12807640 1 -- start start 2024-03-23T09:49:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.027+0000 7f0a12807640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4004d00 0x7f0a0c159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.027+0000 7f0a12807640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0c0cb880 0x7f0a0c0cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.027+0000 7f0a12807640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a0c1f2040 0x7f0a0c1f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.027+0000 7f0a12807640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0a9e70 con 0x7f0a0c1f2040 2024-03-23T09:49:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.027+0000 7f0a12807640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0aa110 con 0x7f09f4004d00 2024-03-23T09:49:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.027+0000 7f0a12807640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0aa3f0 con 0x7f0a0c0cb880 2024-03-23T09:49:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.027+0000 7f0a10da4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a0c1f2040 0x7f0a0c1f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.027+0000 7f0a10da4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a0c1f2040 0x7f0a0c1f24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43996/0 (socket says 172.21.15.47:43996) 2024-03-23T09:49:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.027+0000 7f0a10da4640 1 -- 172.21.15.47:0/3172258165 learned_addr learned my addr 172.21.15.47:0/3172258165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.028+0000 7f0a0b7fe640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0c0cb880 0x7f0a0c0cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.028+0000 7f0a0bfff640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4004d00 0x7f0a0c159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.028+0000 7f0a10da4640 1 -- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0c0cb880 msgr2=0x7f0a0c0cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.028+0000 7f0a10da4640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0c0cb880 0x7f0a0c0cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.028+0000 7f0a10da4640 1 -- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4004d00 msgr2=0x7f0a0c159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.028+0000 7f0a10da4640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4004d00 0x7f0a0c159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.028+0000 7f0a10da4640 1 -- 172.21.15.47:0/3172258165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0aa6d0 con 0x7f0a0c1f2040 2024-03-23T09:49:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.028+0000 7f0a10da4640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a0c1f2040 0x7f0a0c1f24c0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f0a04001580 tx=0x7f0a04001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.028+0000 7f0a08ff9640 1 -- 172.21.15.47:0/3172258165 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04043070 con 0x7f0a0c1f2040 2024-03-23T09:49:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.028+0000 7f0a08ff9640 1 -- 172.21.15.47:0/3172258165 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a04001250 con 0x7f0a0c1f2040 2024-03-23T09:49:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.028+0000 7f0a08ff9640 1 -- 172.21.15.47:0/3172258165 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a040074e0 con 0x7f0a0c1f2040 2024-03-23T09:49:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.028+0000 7f0a12807640 1 -- 172.21.15.47:0/3172258165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c0aa9b0 con 0x7f0a0c1f2040 2024-03-23T09:49:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.028+0000 7f0a12807640 1 -- 172.21.15.47:0/3172258165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c0aaed0 con 0x7f0a0c1f2040 2024-03-23T09:49:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.029+0000 7f0a12807640 1 -- 172.21.15.47:0/3172258165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0c072360 con 0x7f0a0c1f2040 2024-03-23T09:49:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.030+0000 7f0a08ff9640 1 -- 172.21.15.47:0/3172258165 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0a04031030 con 0x7f0a0c1f2040 2024-03-23T09:49:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.030+0000 7f0a08ff9640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09f806ac30 0x7f09f806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.030+0000 7f0a08ff9640 1 -- 172.21.15.47:0/3172258165 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0a040aa5b0 con 0x7f0a0c1f2040 2024-03-23T09:49:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.030+0000 7f0a0bfff640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09f806ac30 0x7f09f806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.031+0000 7f0a0bfff640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09f806ac30 0x7f09f806d0f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0017e0 tx=0x7f09fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:40.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.034+0000 7f0a08ff9640 1 -- 172.21.15.47:0/3172258165 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0a04075c20 con 0x7f0a0c1f2040 2024-03-23T09:49:40.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.215+0000 7f0a12807640 1 -- 172.21.15.47:0/3172258165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a0c066be0 con 0x7f0a0c1f2040 2024-03-23T09:49:40.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.216+0000 7f0a08ff9640 1 -- 172.21.15.47:0/3172258165 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+12390 (secure 0 0 0) 0x7f0a04079240 con 0x7f0a0c1f2040 2024-03-23T09:49:40.216 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:40.216 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":97,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:49:39.522136+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:49:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.218+0000 7f0a12807640 1 -- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09f806ac30 msgr2=0x7f09f806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.218+0000 7f0a12807640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09f806ac30 0x7f09f806d0f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0017e0 tx=0x7f09fc009040 comp rx=0 tx=0).stop 2024-03-23T09:49:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.218+0000 7f0a12807640 1 -- 172.21.15.47:0/3172258165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a0c1f2040 msgr2=0x7f0a0c1f24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.218+0000 7f0a12807640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a0c1f2040 0x7f0a0c1f24c0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f0a04001580 tx=0x7f0a04001d80 comp rx=0 tx=0).stop 2024-03-23T09:49:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.218+0000 7f0a12807640 1 -- 172.21.15.47:0/3172258165 shutdown_connections 2024-03-23T09:49:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.218+0000 7f0a12807640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f09f806ac30 0x7f09f806d0f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.218+0000 7f0a12807640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a0c1f2040 0x7f0a0c1f24c0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.218+0000 7f0a12807640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0c0cb880 0x7f0a0c0cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.218+0000 7f0a12807640 1 --2- 172.21.15.47:0/3172258165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4004d00 0x7f0a0c159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.218+0000 7f0a12807640 1 -- 172.21.15.47:0/3172258165 >> 172.21.15.47:0/3172258165 conn(0x7f0a0c06b160 msgr2=0x7f0a0c15da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.218+0000 7f0a12807640 1 -- 172.21.15.47:0/3172258165 shutdown_connections 2024-03-23T09:49:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.218+0000 7f0a12807640 1 -- 172.21.15.47:0/3172258165 wait complete. 2024-03-23T09:49:40.229 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:49:40.230 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:49:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.333+0000 7f4f3154e640 1 Processor -- start 2024-03-23T09:49:40.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.334+0000 7f4f3154e640 1 -- start start 2024-03-23T09:49:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.334+0000 7f4f3154e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4f2c0cde10 0x7f4f2c160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.334+0000 7f4f3154e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4f2c0b8350 0x7f4f2c0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.334+0000 7f4f3154e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2c161630 con 0x7f4f2c15c990 2024-03-23T09:49:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.334+0000 7f4f3154e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f2c0d1030 con 0x7f4f2c0b8350 2024-03-23T09:49:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.334+0000 7f4f3154e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f2c0cfe10 con 0x7f4f2c0cde10 2024-03-23T09:49:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.334+0000 7f4f2b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4f2c0b8350 0x7f4f2c0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.334+0000 7f4f2b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4f2c0b8350 0x7f4f2c0d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44008/0 (socket says 172.21.15.47:44008) 2024-03-23T09:49:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.334+0000 7f4f2b7fe640 1 -- 172.21.15.47:0/3132034688 learned_addr learned my addr 172.21.15.47:0/3132034688 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.335+0000 7f4f2a7fc640 1 --2- 172.21.15.47:0/3132034688 >> v2:172.21.15.175:3300/0 conn(0x7f4f2c0cde10 0x7f4f2c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.335+0000 7f4f2b7fe640 1 -- 172.21.15.47:0/3132034688 >> v2:172.21.15.175:3300/0 conn(0x7f4f2c0cde10 msgr2=0x7f4f2c160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.335+0000 7f4f2b7fe640 1 --2- 172.21.15.47:0/3132034688 >> v2:172.21.15.175:3300/0 conn(0x7f4f2c0cde10 0x7f4f2c160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.335+0000 7f4f2b7fe640 1 -- 172.21.15.47:0/3132034688 >> v1:172.21.15.175:6789/0 conn(0x7f4f2c15c990 legacy=0x7f4f2c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.335+0000 7f4f2b7fe640 1 -- 172.21.15.47:0/3132034688 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2c0b5b60 con 0x7f4f2c0b8350 2024-03-23T09:49:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.335+0000 7f4f2b7fe640 1 --2- 172.21.15.47:0/3132034688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f2c0b8350 0x7f4f2c0d14c0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c08ac70 tx=0x7f4f2402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e5e53858c345ab server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.336+0000 7f4f29ffb640 1 -- 172.21.15.47:0/3132034688 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2402c9f0 con 0x7f4f2c0b8350 2024-03-23T09:49:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.336+0000 7f4f29ffb640 1 -- 172.21.15.47:0/3132034688 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2402cb50 con 0x7f4f2c0b8350 2024-03-23T09:49:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.336+0000 7f4f29ffb640 1 -- 172.21.15.47:0/3132034688 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f240343f0 con 0x7f4f2c0b8350 2024-03-23T09:49:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.336+0000 7f4f3154e640 1 -- 172.21.15.47:0/3132034688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f2c0b8350 msgr2=0x7f4f2c0d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.336+0000 7f4f3154e640 1 --2- 172.21.15.47:0/3132034688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f2c0b8350 0x7f4f2c0d14c0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c08ac70 tx=0x7f4f2402be50 comp rx=0 tx=0).stop 2024-03-23T09:49:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.336+0000 7f4f3154e640 1 -- 172.21.15.47:0/3132034688 shutdown_connections 2024-03-23T09:49:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.336+0000 7f4f3154e640 1 --2- 172.21.15.47:0/3132034688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f2c0b8350 0x7f4f2c0d14c0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.336+0000 7f4f3154e640 1 --2- 172.21.15.47:0/3132034688 >> v2:172.21.15.175:3300/0 conn(0x7f4f2c0cde10 0x7f4f2c160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.336+0000 7f4f3154e640 1 -- 172.21.15.47:0/3132034688 >> 172.21.15.47:0/3132034688 conn(0x7f4f2c0659d0 msgr2=0x7f4f2c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.336+0000 7f4f3154e640 1 -- 172.21.15.47:0/3132034688 shutdown_connections 2024-03-23T09:49:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.336+0000 7f4f3154e640 1 -- 172.21.15.47:0/3132034688 wait complete. 2024-03-23T09:49:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.337+0000 7f4f3154e640 1 Processor -- start 2024-03-23T09:49:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.338+0000 7f4f3154e640 1 -- start start 2024-03-23T09:49:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.338+0000 7f4f3154e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f2c0b8350 0x7f4f2c153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.338+0000 7f4f3154e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f2c0cde10 0x7f4f2c15eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.338+0000 7f4f3154e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f2c15c990 0x7f4f2c0b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.338+0000 7f4f3154e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c159920 con 0x7f4f2c0b8350 2024-03-23T09:49:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.338+0000 7f4f3154e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c159c00 con 0x7f4f2c15c990 2024-03-23T09:49:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.338+0000 7f4f3154e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c159ee0 con 0x7f4f2c0cde10 2024-03-23T09:49:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.338+0000 7f4f2affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f2c0b8350 0x7f4f2c153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.338+0000 7f4f2affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f2c0b8350 0x7f4f2c153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44020/0 (socket says 172.21.15.47:44020) 2024-03-23T09:49:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.338+0000 7f4f2affd640 1 -- 172.21.15.47:0/612987199 learned_addr learned my addr 172.21.15.47:0/612987199 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f2b7fe640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f2c15c990 0x7f4f2c0b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f2a7fc640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f2c0cde10 0x7f4f2c15eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f2affd640 1 -- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f2c0cde10 msgr2=0x7f4f2c15eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f2affd640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f2c0cde10 0x7f4f2c15eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f2affd640 1 -- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f2c15c990 msgr2=0x7f4f2c0b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f2affd640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f2c15c990 0x7f4f2c0b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f2affd640 1 -- 172.21.15.47:0/612987199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f24009b30 con 0x7f4f2c0b8350 2024-03-23T09:49:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f2b7fe640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f2c15c990 0x7f4f2c0b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f2affd640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f2c0b8350 0x7f4f2c153cb0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00c430 tx=0x7f4f1c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f137fe640 1 -- 172.21.15.47:0/612987199 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c011790 con 0x7f4f2c0b8350 2024-03-23T09:49:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f137fe640 1 -- 172.21.15.47:0/612987199 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1c0118f0 con 0x7f4f2c0b8350 2024-03-23T09:49:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f137fe640 1 -- 172.21.15.47:0/612987199 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c011ce0 con 0x7f4f2c0b8350 2024-03-23T09:49:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f3154e640 1 -- 172.21.15.47:0/612987199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2c15a1c0 con 0x7f4f2c0b8350 2024-03-23T09:49:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.339+0000 7f4f3154e640 1 -- 172.21.15.47:0/612987199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2c15f800 con 0x7f4f2c0b8350 2024-03-23T09:49:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.340+0000 7f4f3154e640 1 -- 172.21.15.47:0/612987199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2c072360 con 0x7f4f2c0b8350 2024-03-23T09:49:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.341+0000 7f4f137fe640 1 -- 172.21.15.47:0/612987199 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4f1c01c460 con 0x7f4f2c0b8350 2024-03-23T09:49:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.341+0000 7f4f137fe640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4f1806ac00 0x7f4f1806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.341+0000 7f4f137fe640 1 -- 172.21.15.47:0/612987199 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4f1c088ed0 con 0x7f4f2c0b8350 2024-03-23T09:49:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.341+0000 7f4f2a7fc640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4f1806ac00 0x7f4f1806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.342+0000 7f4f2a7fc640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4f1806ac00 0x7f4f1806d0c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4f20006fd0 tx=0x7f4f20008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.345+0000 7f4f137fe640 1 -- 172.21.15.47:0/612987199 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4f1c054490 con 0x7f4f2c0b8350 2024-03-23T09:49:40.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.524+0000 7f4f3154e640 1 -- 172.21.15.47:0/612987199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f2c15f990 con 0x7f4f2c0b8350 2024-03-23T09:49:40.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.524+0000 7f4f137fe640 1 -- 172.21.15.47:0/612987199 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f1c057ab0 con 0x7f4f2c0b8350 2024-03-23T09:49:40.524 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:49:40.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.526+0000 7f4f3154e640 1 -- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4f1806ac00 msgr2=0x7f4f1806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.526+0000 7f4f3154e640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4f1806ac00 0x7f4f1806d0c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4f20006fd0 tx=0x7f4f20008040 comp rx=0 tx=0).stop 2024-03-23T09:49:40.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.526+0000 7f4f3154e640 1 -- 172.21.15.47:0/612987199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f2c0b8350 msgr2=0x7f4f2c153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.526+0000 7f4f3154e640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f2c0b8350 0x7f4f2c153cb0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00c430 tx=0x7f4f1c00c8f0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.526+0000 7f4f3154e640 1 -- 172.21.15.47:0/612987199 shutdown_connections 2024-03-23T09:49:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.526+0000 7f4f3154e640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4f1806ac00 0x7f4f1806d0c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.526+0000 7f4f3154e640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f2c15c990 0x7f4f2c0b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.526+0000 7f4f3154e640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f2c0cde10 0x7f4f2c15eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.526+0000 7f4f3154e640 1 --2- 172.21.15.47:0/612987199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f2c0b8350 0x7f4f2c153cb0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.526+0000 7f4f3154e640 1 -- 172.21.15.47:0/612987199 >> 172.21.15.47:0/612987199 conn(0x7f4f2c0659d0 msgr2=0x7f4f2c0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.526+0000 7f4f3154e640 1 -- 172.21.15.47:0/612987199 shutdown_connections 2024-03-23T09:49:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.526+0000 7f4f3154e640 1 -- 172.21.15.47:0/612987199 wait complete. 2024-03-23T09:49:40.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T09:49:40.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.638+0000 7f7810448640 1 Processor -- start 2024-03-23T09:49:40.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.638+0000 7f7810448640 1 -- start start 2024-03-23T09:49:40.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.638+0000 7f7810448640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f780815cd00 0x7f78080d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.638+0000 7f7810448640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780815d0e0 con 0x7f78080cf920 2024-03-23T09:49:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.638+0000 7f7810448640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78080d1540 con 0x7f78080d1020 2024-03-23T09:49:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.638+0000 7f7810448640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78080d1820 con 0x7f780815cd00 2024-03-23T09:49:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780e9e5640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f78080cf920 0x7f78080b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44606/0 (socket says 172.21.15.47:44606) 2024-03-23T09:49:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780e9e5640 1 -- 172.21.15.47:0/4205976465 learned_addr learned my addr 172.21.15.47:0/4205976465 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282876262 0 0) 0x7f780815d0e0 con 0x7f78080cf920 2024-03-23T09:49:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780e1e4640 1 --2- 172.21.15.47:0/4205976465 >> v2:172.21.15.175:3300/0 conn(0x7f780815cd00 0x7f78080d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77f4003520 con 0x7f78080cf920 2024-03-23T09:49:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2911494220 0 0) 0x7f77f4003520 con 0x7f78080cf920 2024-03-23T09:49:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 >> v2:172.21.15.175:3300/0 conn(0x7f780815cd00 msgr2=0x7f78080d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 --2- 172.21.15.47:0/4205976465 >> v2:172.21.15.175:3300/0 conn(0x7f780815cd00 0x7f78080d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 >> v1:172.21.15.175:6789/0 conn(0x7f78080d1020 legacy=0x7f78080cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78080d1b00 con 0x7f78080cf920 2024-03-23T09:49:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f78000019f0 con 0x7f78080cf920 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 >> v1:172.21.15.47:6789/0 conn(0x7f78080cf920 legacy=0x7f78080b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 --2- 172.21.15.47:0/4205976465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f4003970 0x7f77f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 --2- 172.21.15.47:0/4205976465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77f40047c0 0x7f77f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 --2- 172.21.15.47:0/4205976465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f4005180 0x7f77f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78080d1820 con 0x7f77f40047c0 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f400c120 con 0x7f77f4005180 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f400c2e0 con 0x7f77f4003970 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780d9e3640 1 --2- 172.21.15.47:0/4205976465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77f40047c0 0x7f77f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.639+0000 7f780e1e4640 1 --2- 172.21.15.47:0/4205976465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f4003970 0x7f77f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.640+0000 7f780e9e5640 1 --2- 172.21.15.47:0/4205976465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f4005180 0x7f77f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.640+0000 7f780d9e3640 1 -- 172.21.15.47:0/4205976465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f4003970 msgr2=0x7f77f4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.640+0000 7f780d9e3640 1 --2- 172.21.15.47:0/4205976465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f4003970 0x7f77f4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.640+0000 7f780d9e3640 1 -- 172.21.15.47:0/4205976465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f4005180 msgr2=0x7f77f400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.640+0000 7f780d9e3640 1 --2- 172.21.15.47:0/4205976465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f4005180 0x7f77f400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.640+0000 7f780d9e3640 1 -- 172.21.15.47:0/4205976465 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4003520 con 0x7f77f40047c0 2024-03-23T09:49:40.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.640+0000 7f780e9e5640 1 -- 172.21.15.47:0/4205976465 reap_dead start 2024-03-23T09:49:40.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.640+0000 7f780d9e3640 1 --2- 172.21.15.47:0/4205976465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77f40047c0 0x7f77f4004c40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f77f8001750 tx=0x7f77f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727c8cff54293e09 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:40.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.641+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f80052d0 con 0x7f77f40047c0 2024-03-23T09:49:40.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.641+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f802e050 con 0x7f77f40047c0 2024-03-23T09:49:40.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.641+0000 7f780d1e2640 1 -- 172.21.15.47:0/4205976465 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f8003830 con 0x7f77f40047c0 2024-03-23T09:49:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.641+0000 7f7810448640 1 -- 172.21.15.47:0/4205976465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77f40047c0 msgr2=0x7f77f4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.641+0000 7f7810448640 1 --2- 172.21.15.47:0/4205976465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77f40047c0 0x7f77f4004c40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f77f8001750 tx=0x7f77f40057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.641+0000 7f7810448640 1 -- 172.21.15.47:0/4205976465 shutdown_connections 2024-03-23T09:49:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.641+0000 7f7810448640 1 --2- 172.21.15.47:0/4205976465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77f40047c0 0x7f77f4004c40 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.641+0000 7f7810448640 1 -- 172.21.15.47:0/4205976465 >> 172.21.15.47:0/4205976465 conn(0x7f780806b160 msgr2=0x7f780806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.641+0000 7f7810448640 1 -- 172.21.15.47:0/4205976465 shutdown_connections 2024-03-23T09:49:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.641+0000 7f7810448640 1 -- 172.21.15.47:0/4205976465 wait complete. 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.642+0000 7f7810448640 1 Processor -- start 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.642+0000 7f7810448640 1 -- start start 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f7810448640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f40047c0 0x7f78080ab660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f7810448640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78080abba0 0x7f780814d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f7810448640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f780814da70 0x7f780814def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f7810448640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78080ac020 con 0x7f78080abba0 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f7810448640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78080a93d0 con 0x7f780814da70 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f7810448640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78080a96b0 con 0x7f77f40047c0 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f780d9e3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78080abba0 0x7f780814d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f780d9e3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78080abba0 0x7f780814d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44026/0 (socket says 172.21.15.47:44026) 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f780d9e3640 1 -- 172.21.15.47:0/1533809984 learned_addr learned my addr 172.21.15.47:0/1533809984 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f780e1e4640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f40047c0 0x7f78080ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f780e9e5640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f780814da70 0x7f780814def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f780d9e3640 1 -- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f40047c0 msgr2=0x7f78080ab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f780d9e3640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f40047c0 0x7f78080ab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f780d9e3640 1 -- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f780814da70 msgr2=0x7f780814def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.643+0000 7f780d9e3640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f780814da70 0x7f780814def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.644+0000 7f780d9e3640 1 -- 172.21.15.47:0/1533809984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78080a9990 con 0x7f78080abba0 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.644+0000 7f780e1e4640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f40047c0 0x7f78080ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.644+0000 7f780e9e5640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f780814da70 0x7f780814def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.644+0000 7f780d9e3640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78080abba0 0x7f780814d530 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005040 tx=0x7f77f8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.644+0000 7f77feffd640 1 -- 172.21.15.47:0/1533809984 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f8007a30 con 0x7f78080abba0 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.644+0000 7f77feffd640 1 -- 172.21.15.47:0/1533809984 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f80069e0 con 0x7f78080abba0 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.644+0000 7f7810448640 1 -- 172.21.15.47:0/1533809984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78080a9c70 con 0x7f78080abba0 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.644+0000 7f77feffd640 1 -- 172.21.15.47:0/1533809984 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f8006d70 con 0x7f78080abba0 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.644+0000 7f7810448640 1 -- 172.21.15.47:0/1533809984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780814e990 con 0x7f78080abba0 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.645+0000 7f7810448640 1 -- 172.21.15.47:0/1533809984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7808072360 con 0x7f78080abba0 2024-03-23T09:49:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.645+0000 7f77feffd640 1 -- 172.21.15.47:0/1533809984 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f77f80035f0 con 0x7f78080abba0 2024-03-23T09:49:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.645+0000 7f77feffd640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f77ec06ac00 0x7f77ec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.646+0000 7f77feffd640 1 -- 172.21.15.47:0/1533809984 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f77f80ab100 con 0x7f78080abba0 2024-03-23T09:49:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.646+0000 7f780e1e4640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f77ec06ac00 0x7f77ec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.646+0000 7f780e1e4640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f77ec06ac00 0x7f77ec06d0c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7804009ab0 tx=0x7f7804008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.650+0000 7f77feffd640 1 -- 172.21.15.47:0/1533809984 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f77f8076630 con 0x7f78080abba0 2024-03-23T09:49:40.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.829+0000 7f7810448640 1 -- 172.21.15.47:0/1533809984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7808066be0 con 0x7f78080abba0 2024-03-23T09:49:40.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.830+0000 7f77feffd640 1 -- 172.21.15.47:0/1533809984 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v97) v1 ==== 84+0+65 (secure 0 0 0) 0x7f77f804b060 con 0x7f78080abba0 2024-03-23T09:49:40.830 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:40.830 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-23T09:49:40.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:49:40.830 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-03-23T09:49:40.830 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-03-23T09:49:40.830 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:40.830 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-23T09:49:40.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.831+0000 7f7810448640 1 -- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f77ec06ac00 msgr2=0x7f77ec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.831+0000 7f7810448640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f77ec06ac00 0x7f77ec06d0c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7804009ab0 tx=0x7f7804008040 comp rx=0 tx=0).stop 2024-03-23T09:49:40.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.831+0000 7f7810448640 1 -- 172.21.15.47:0/1533809984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78080abba0 msgr2=0x7f780814d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.831+0000 7f7810448640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78080abba0 0x7f780814d530 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005040 tx=0x7f77f8001120 comp rx=0 tx=0).stop 2024-03-23T09:49:40.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.831+0000 7f7810448640 1 -- 172.21.15.47:0/1533809984 shutdown_connections 2024-03-23T09:49:40.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.831+0000 7f7810448640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f77ec06ac00 0x7f77ec06d0c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.831+0000 7f7810448640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f780814da70 0x7f780814def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.831+0000 7f7810448640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78080abba0 0x7f780814d530 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.831+0000 7f7810448640 1 --2- 172.21.15.47:0/1533809984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f40047c0 0x7f78080ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.831+0000 7f7810448640 1 -- 172.21.15.47:0/1533809984 >> 172.21.15.47:0/1533809984 conn(0x7f780806b160 msgr2=0x7f78080c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:40.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.831+0000 7f7810448640 1 -- 172.21.15.47:0/1533809984 shutdown_connections 2024-03-23T09:49:40.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.832+0000 7f7810448640 1 -- 172.21.15.47:0/1533809984 wait complete. 2024-03-23T09:49:40.842 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T09:49:40.842 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T09:49:40.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.947+0000 7fbbfb920640 1 Processor -- start 2024-03-23T09:49:40.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.947+0000 7fbbfb920640 1 -- start start 2024-03-23T09:49:40.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.948+0000 7fbbfb920640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf415a860 0x7fbbf4161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.948+0000 7fbbfb920640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbbf415d0f0 0x7fbbf415d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.948+0000 7fbbfb920640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf4160bd0 con 0x7fbbf40cf190 2024-03-23T09:49:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.948+0000 7fbbfb920640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf40d2790 con 0x7fbbf415d0f0 2024-03-23T09:49:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.948+0000 7fbbfb920640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf415dbb0 con 0x7fbbf415a860 2024-03-23T09:49:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.948+0000 7fbbf8ebb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbbf415d0f0 0x7fbbf415d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.948+0000 7fbbf8ebb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbbf415d0f0 0x7fbbf415d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44032/0 (socket says 172.21.15.47:44032) 2024-03-23T09:49:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.948+0000 7fbbf8ebb640 1 -- 172.21.15.47:0/1253784638 learned_addr learned my addr 172.21.15.47:0/1253784638 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.948+0000 7fbbf96bc640 1 --2- 172.21.15.47:0/1253784638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf415a860 0x7fbbf4161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.949+0000 7fbbf8ebb640 1 -- 172.21.15.47:0/1253784638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf415a860 msgr2=0x7fbbf4161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.949+0000 7fbbf8ebb640 1 --2- 172.21.15.47:0/1253784638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf415a860 0x7fbbf4161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.949+0000 7fbbf8ebb640 1 -- 172.21.15.47:0/1253784638 >> v1:172.21.15.175:6789/0 conn(0x7fbbf40cf190 legacy=0x7fbbf40cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.949+0000 7fbbf8ebb640 1 -- 172.21.15.47:0/1253784638 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf415de90 con 0x7fbbf415d0f0 2024-03-23T09:49:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.949+0000 7fbbf8ebb640 1 --2- 172.21.15.47:0/1253784638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf415d0f0 0x7fbbf415d550 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009870 tx=0x7fbbe802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99121e6904ceede8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.949+0000 7fbbf3fff640 1 -- 172.21.15.47:0/1253784638 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8030840 con 0x7fbbf415d0f0 2024-03-23T09:49:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.950+0000 7fbbf3fff640 1 -- 172.21.15.47:0/1253784638 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe80309a0 con 0x7fbbf415d0f0 2024-03-23T09:49:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.950+0000 7fbbf3fff640 1 -- 172.21.15.47:0/1253784638 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8030d30 con 0x7fbbf415d0f0 2024-03-23T09:49:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.950+0000 7fbbfb920640 1 -- 172.21.15.47:0/1253784638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf415d0f0 msgr2=0x7fbbf415d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.950+0000 7fbbfb920640 1 --2- 172.21.15.47:0/1253784638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf415d0f0 0x7fbbf415d550 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009870 tx=0x7fbbe802b980 comp rx=0 tx=0).stop 2024-03-23T09:49:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.950+0000 7fbbfb920640 1 -- 172.21.15.47:0/1253784638 shutdown_connections 2024-03-23T09:49:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.950+0000 7fbbfb920640 1 --2- 172.21.15.47:0/1253784638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf415d0f0 0x7fbbf415d550 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.950+0000 7fbbfb920640 1 --2- 172.21.15.47:0/1253784638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf415a860 0x7fbbf4161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.950+0000 7fbbfb920640 1 -- 172.21.15.47:0/1253784638 >> 172.21.15.47:0/1253784638 conn(0x7fbbf40659d0 msgr2=0x7fbbf4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.950+0000 7fbbfb920640 1 -- 172.21.15.47:0/1253784638 shutdown_connections 2024-03-23T09:49:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.950+0000 7fbbfb920640 1 -- 172.21.15.47:0/1253784638 wait complete. 2024-03-23T09:49:40.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.951+0000 7fbbfb920640 1 Processor -- start 2024-03-23T09:49:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.951+0000 7fbbfb920640 1 -- start start 2024-03-23T09:49:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbfb920640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf40cf190 0x7fbbf40b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbfb920640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf415a860 0x7fbbf41f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbfb920640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf415d0f0 0x7fbbf40cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbfb920640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf40b77a0 con 0x7fbbf415a860 2024-03-23T09:49:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbfb920640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf40c7ec0 con 0x7fbbf40cf190 2024-03-23T09:49:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbfb920640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf40b74c0 con 0x7fbbf415d0f0 2024-03-23T09:49:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbf8ebb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf415a860 0x7fbbf41f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbf8ebb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf415a860 0x7fbbf41f1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44038/0 (socket says 172.21.15.47:44038) 2024-03-23T09:49:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbf8ebb640 1 -- 172.21.15.47:0/1944692658 learned_addr learned my addr 172.21.15.47:0/1944692658 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbf8ebb640 1 -- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf415d0f0 msgr2=0x7fbbf40cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbf9ebd640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf415d0f0 0x7fbbf40cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbf8ebb640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf415d0f0 0x7fbbf40cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbf96bc640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf40cf190 0x7fbbf40b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.952+0000 7fbbf8ebb640 1 -- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf40cf190 msgr2=0x7fbbf40b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.953+0000 7fbbf8ebb640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf40cf190 0x7fbbf40b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.953+0000 7fbbf8ebb640 1 -- 172.21.15.47:0/1944692658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe80094d0 con 0x7fbbf415a860 2024-03-23T09:49:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.953+0000 7fbbf8ebb640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf415a860 0x7fbbf41f1b10 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8000c00 tx=0x7fbbe8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.953+0000 7fbbf1ffb640 1 -- 172.21.15.47:0/1944692658 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8039a90 con 0x7fbbf415a860 2024-03-23T09:49:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.953+0000 7fbbf1ffb640 1 -- 172.21.15.47:0/1944692658 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe8039bf0 con 0x7fbbf415a860 2024-03-23T09:49:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.953+0000 7fbbfb920640 1 -- 172.21.15.47:0/1944692658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf40c7870 con 0x7fbbf415a860 2024-03-23T09:49:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.953+0000 7fbbf1ffb640 1 -- 172.21.15.47:0/1944692658 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8043410 con 0x7fbbf415a860 2024-03-23T09:49:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.953+0000 7fbbfb920640 1 -- 172.21.15.47:0/1944692658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf40b7120 con 0x7fbbf415a860 2024-03-23T09:49:40.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.954+0000 7fbbfb920640 1 -- 172.21.15.47:0/1944692658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf4072360 con 0x7fbbf415a860 2024-03-23T09:49:40.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.955+0000 7fbbf1ffb640 1 -- 172.21.15.47:0/1944692658 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbbe804c3b0 con 0x7fbbf415a860 2024-03-23T09:49:40.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.955+0000 7fbbf1ffb640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbbe006ac30 0x7fbbe006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:40.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.955+0000 7fbbf1ffb640 1 -- 172.21.15.47:0/1944692658 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbbe80ab4b0 con 0x7fbbf415a860 2024-03-23T09:49:40.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.955+0000 7fbbf96bc640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbbe006ac30 0x7fbbe006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:40.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.956+0000 7fbbf96bc640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbbe006ac30 0x7fbbe006d0f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbbec006fd0 tx=0x7fbbec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:40.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:40.959+0000 7fbbf1ffb640 1 -- 172.21.15.47:0/1944692658 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbbe80769e0 con 0x7fbbf415a860 2024-03-23T09:49:41.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.138+0000 7fbbfb920640 1 -- 172.21.15.47:0/1944692658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbbf40c76d0 con 0x7fbbf415a860 2024-03-23T09:49:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.866+0000 7fbbf1ffb640 1 -- 172.21.15.47:0/1944692658 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v98) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbbf40c76d0 con 0x7fbbf415a860 2024-03-23T09:49:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-03-23T09:49:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.871+0000 7fbbfb920640 1 -- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbbe006ac30 msgr2=0x7fbbe006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.871+0000 7fbbfb920640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbbe006ac30 0x7fbbe006d0f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbbec006fd0 tx=0x7fbbec008040 comp rx=0 tx=0).stop 2024-03-23T09:49:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.871+0000 7fbbfb920640 1 -- 172.21.15.47:0/1944692658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf415a860 msgr2=0x7fbbf41f1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.871+0000 7fbbfb920640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf415a860 0x7fbbf41f1b10 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8000c00 tx=0x7fbbe8004730 comp rx=0 tx=0).stop 2024-03-23T09:49:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.872+0000 7fbbfb920640 1 -- 172.21.15.47:0/1944692658 shutdown_connections 2024-03-23T09:49:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.872+0000 7fbbfb920640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbbe006ac30 0x7fbbe006d0f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.872+0000 7fbbfb920640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf415d0f0 0x7fbbf40cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.872+0000 7fbbfb920640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf415a860 0x7fbbf41f1b10 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.872+0000 7fbbfb920640 1 --2- 172.21.15.47:0/1944692658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf40cf190 0x7fbbf40b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.872+0000 7fbbfb920640 1 -- 172.21.15.47:0/1944692658 >> 172.21.15.47:0/1944692658 conn(0x7fbbf40659d0 msgr2=0x7fbbf415e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.873+0000 7fbbfb920640 1 -- 172.21.15.47:0/1944692658 shutdown_connections 2024-03-23T09:49:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.873+0000 7fbbfb920640 1 -- 172.21.15.47:0/1944692658 wait complete. 2024-03-23T09:49:41.886 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T09:49:41.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.996+0000 7f2b67c1e640 1 Processor -- start 2024-03-23T09:49:41.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.996+0000 7f2b67c1e640 1 -- start start 2024-03-23T09:49:41.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.997+0000 7f2b67c1e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6015ce80 0x7f2b600d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.997+0000 7f2b67c1e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2b600b83d0 0x7f2b600cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.997+0000 7f2b67c1e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2b600d2460 0x7f2b600cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.997+0000 7f2b67c1e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b600d13c0 con 0x7f2b600b83d0 2024-03-23T09:49:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.997+0000 7f2b67c1e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b600d16a0 con 0x7f2b600d2460 2024-03-23T09:49:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.997+0000 7f2b67c1e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b600d1980 con 0x7f2b6015ce80 2024-03-23T09:49:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.997+0000 7f2b651b9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2b600b83d0 0x7f2b600cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.997+0000 7f2b651b9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2b600b83d0 0x7f2b600cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44046/0 (socket says 172.21.15.47:44046) 2024-03-23T09:49:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.997+0000 7f2b651b9640 1 -- 172.21.15.47:0/3271744746 learned_addr learned my addr 172.21.15.47:0/3271744746 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.997+0000 7f2b651b9640 1 -- 172.21.15.47:0/3271744746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6015ce80 msgr2=0x7f2b600d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.997+0000 7f2b661bb640 1 --2- 172.21.15.47:0/3271744746 >> v2:172.21.15.175:3300/0 conn(0x7f2b600d2460 0x7f2b600cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.997+0000 7f2b659ba640 1 --2- 172.21.15.47:0/3271744746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6015ce80 0x7f2b600d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.998+0000 7f2b651b9640 1 --2- 172.21.15.47:0/3271744746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6015ce80 0x7f2b600d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.998+0000 7f2b651b9640 1 -- 172.21.15.47:0/3271744746 >> v2:172.21.15.175:3300/0 conn(0x7f2b600d2460 msgr2=0x7f2b600cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.998+0000 7f2b651b9640 1 --2- 172.21.15.47:0/3271744746 >> v2:172.21.15.175:3300/0 conn(0x7f2b600d2460 0x7f2b600cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.998+0000 7f2b651b9640 1 -- 172.21.15.47:0/3271744746 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b600d1c60 con 0x7f2b600b83d0 2024-03-23T09:49:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.998+0000 7f2b661bb640 1 --2- 172.21.15.47:0/3271744746 >> v2:172.21.15.175:3300/0 conn(0x7f2b600d2460 0x7f2b600cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.998+0000 7f2b659ba640 1 --2- 172.21.15.47:0/3271744746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6015ce80 0x7f2b600d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.998+0000 7f2b651b9640 1 --2- 172.21.15.47:0/3271744746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b600b83d0 0x7f2b600cd730 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f2b600cfbd0 tx=0x7f2b5402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3683cf5b35367e2b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.998+0000 7f2b649b8640 1 -- 172.21.15.47:0/3271744746 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54030850 con 0x7f2b600b83d0 2024-03-23T09:49:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.999+0000 7f2b649b8640 1 -- 172.21.15.47:0/3271744746 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b540309b0 con 0x7f2b600b83d0 2024-03-23T09:49:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.999+0000 7f2b649b8640 1 -- 172.21.15.47:0/3271744746 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54030d40 con 0x7f2b600b83d0 2024-03-23T09:49:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.999+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3271744746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b600b83d0 msgr2=0x7f2b600cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.999+0000 7f2b67c1e640 1 --2- 172.21.15.47:0/3271744746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b600b83d0 0x7f2b600cd730 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f2b600cfbd0 tx=0x7f2b5402b990 comp rx=0 tx=0).stop 2024-03-23T09:49:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.999+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3271744746 shutdown_connections 2024-03-23T09:49:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.999+0000 7f2b67c1e640 1 --2- 172.21.15.47:0/3271744746 >> v2:172.21.15.175:3300/0 conn(0x7f2b600d2460 0x7f2b600cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.999+0000 7f2b67c1e640 1 --2- 172.21.15.47:0/3271744746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b600b83d0 0x7f2b600cd730 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.999+0000 7f2b67c1e640 1 --2- 172.21.15.47:0/3271744746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6015ce80 0x7f2b600d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.999+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3271744746 >> 172.21.15.47:0/3271744746 conn(0x7f2b60065df0 msgr2=0x7f2b60066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.999+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3271744746 shutdown_connections 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:41.999+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3271744746 wait complete. 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.000+0000 7f2b67c1e640 1 Processor -- start 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.000+0000 7f2b67c1e640 1 -- start start 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b67c1e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b600b83d0 0x7f2b601f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b67c1e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b600d2460 0x7f2b60157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b67c1e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6015ce80 0x7f2b600cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b67c1e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b600b3080 con 0x7f2b600b83d0 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b67c1e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b600b3320 con 0x7f2b600d2460 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b67c1e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b600b3600 con 0x7f2b6015ce80 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b659ba640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b600b83d0 0x7f2b601f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b659ba640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b600b83d0 0x7f2b601f1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44054/0 (socket says 172.21.15.47:44054) 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b659ba640 1 -- 172.21.15.47:0/3451344064 learned_addr learned my addr 172.21.15.47:0/3451344064 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b651b9640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b600d2460 0x7f2b60157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b661bb640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6015ce80 0x7f2b600cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b659ba640 1 -- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6015ce80 msgr2=0x7f2b600cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b659ba640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6015ce80 0x7f2b600cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b659ba640 1 -- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b600d2460 msgr2=0x7f2b60157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b659ba640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b600d2460 0x7f2b60157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.001+0000 7f2b659ba640 1 -- 172.21.15.47:0/3451344064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b540094d0 con 0x7f2b600b83d0 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.002+0000 7f2b659ba640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b600b83d0 0x7f2b601f1db0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800c7f0 tx=0x7f2b5800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.002+0000 7f2b5e7fc640 1 -- 172.21.15.47:0/3451344064 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58007a30 con 0x7f2b600b83d0 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.002+0000 7f2b5e7fc640 1 -- 172.21.15.47:0/3451344064 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b58007b90 con 0x7f2b600b83d0 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.002+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3451344064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b600b38e0 con 0x7f2b600b83d0 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.002+0000 7f2b5e7fc640 1 -- 172.21.15.47:0/3451344064 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5800ce40 con 0x7f2b600b83d0 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.002+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3451344064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b600b3ea0 con 0x7f2b600b83d0 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.003+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3451344064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b60072360 con 0x7f2b600b83d0 2024-03-23T09:49:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.003+0000 7f2b5e7fc640 1 -- 172.21.15.47:0/3451344064 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2b580113f0 con 0x7f2b600b83d0 2024-03-23T09:49:42.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.004+0000 7f2b5e7fc640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b4c06ac00 0x7f2b4c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:42.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.004+0000 7f2b5e7fc640 1 -- 172.21.15.47:0/3451344064 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2b580893d0 con 0x7f2b600b83d0 2024-03-23T09:49:42.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.004+0000 7f2b651b9640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b4c06ac00 0x7f2b4c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:42.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.004+0000 7f2b651b9640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b4c06ac00 0x7f2b4c06d0c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2b54036000 tx=0x7f2b5403b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:42.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.008+0000 7f2b5e7fc640 1 -- 172.21.15.47:0/3451344064 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2b580546f0 con 0x7f2b600b83d0 2024-03-23T09:49:42.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.190+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3451344064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2b600b4090 con 0x7f2b600b83d0 2024-03-23T09:49:42.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.875+0000 7f2b5e7fc640 1 -- 172.21.15.47:0/3451344064 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v99) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2b58087e60 con 0x7f2b600b83d0 2024-03-23T09:49:42.875 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-03-23T09:49:42.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.879+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b4c06ac00 msgr2=0x7f2b4c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:42.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.879+0000 7f2b67c1e640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b4c06ac00 0x7f2b4c06d0c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2b54036000 tx=0x7f2b5403b040 comp rx=0 tx=0).stop 2024-03-23T09:49:42.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.879+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3451344064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b600b83d0 msgr2=0x7f2b601f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:42.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.879+0000 7f2b67c1e640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b600b83d0 0x7f2b601f1db0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800c7f0 tx=0x7f2b5800cbf0 comp rx=0 tx=0).stop 2024-03-23T09:49:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.884+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3451344064 shutdown_connections 2024-03-23T09:49:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.884+0000 7f2b67c1e640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2b4c06ac00 0x7f2b4c06d0c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.884+0000 7f2b67c1e640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6015ce80 0x7f2b600cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:42.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.884+0000 7f2b67c1e640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b600d2460 0x7f2b60157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:42.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.884+0000 7f2b67c1e640 1 --2- 172.21.15.47:0/3451344064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b600b83d0 0x7f2b601f1db0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:42.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.885+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3451344064 >> 172.21.15.47:0/3451344064 conn(0x7f2b60065df0 msgr2=0x7f2b6015d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:42.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.891+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3451344064 shutdown_connections 2024-03-23T09:49:42.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:42.891+0000 7f2b67c1e640 1 -- 172.21.15.47:0/3451344064 wait complete. 2024-03-23T09:49:42.907 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T09:49:43.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.016+0000 7ff3f718b640 1 Processor -- start 2024-03-23T09:49:43.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.016+0000 7ff3f718b640 1 -- start start 2024-03-23T09:49:43.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.016+0000 7ff3f718b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff3f006b090 0x7ff3f00d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.016+0000 7ff3f718b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f0099760 con 0x7ff3f0099110 2024-03-23T09:49:43.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.016+0000 7ff3f718b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f00cf020 con 0x7ff3f015cd40 2024-03-23T09:49:43.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.016+0000 7ff3f718b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f015d120 con 0x7ff3f006b090 2024-03-23T09:49:43.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.017+0000 7ff3f5728640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff3f006b090 0x7ff3f00d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.017+0000 7ff3e7fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff3f0099110 0x7ff3f00caca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44616/0 (socket says 172.21.15.47:44616) 2024-03-23T09:49:43.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.017+0000 7ff3e7fff640 1 -- 172.21.15.47:0/3058360942 learned_addr learned my addr 172.21.15.47:0/3058360942 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:43.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.017+0000 7ff3f5728640 1 -- 172.21.15.47:0/3058360942 >> v1:172.21.15.175:6789/0 conn(0x7ff3f015cd40 legacy=0x7ff3f0098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:43.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.017+0000 7ff3f5728640 1 -- 172.21.15.47:0/3058360942 >> v1:172.21.15.47:6789/0 conn(0x7ff3f0099110 legacy=0x7ff3f00caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.017+0000 7ff3f5728640 1 -- 172.21.15.47:0/3058360942 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f00cf300 con 0x7ff3f006b090 2024-03-23T09:49:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.018+0000 7ff3f5728640 1 --2- 172.21.15.47:0/3058360942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f006b090 0x7ff3f00d00c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009d90 tx=0x7ff3e802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf002ffc4ecfe08 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.018+0000 7ff3e77fe640 1 -- 172.21.15.47:0/3058360942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e802cc60 con 0x7ff3f006b090 2024-03-23T09:49:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.018+0000 7ff3e77fe640 1 -- 172.21.15.47:0/3058360942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e802cdc0 con 0x7ff3f006b090 2024-03-23T09:49:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.018+0000 7ff3e77fe640 1 -- 172.21.15.47:0/3058360942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8034660 con 0x7ff3f006b090 2024-03-23T09:49:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.018+0000 7ff3f718b640 1 -- 172.21.15.47:0/3058360942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f006b090 msgr2=0x7ff3f00d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.018+0000 7ff3f718b640 1 --2- 172.21.15.47:0/3058360942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f006b090 0x7ff3f00d00c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009d90 tx=0x7ff3e802beb0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.018+0000 7ff3f718b640 1 -- 172.21.15.47:0/3058360942 shutdown_connections 2024-03-23T09:49:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.018+0000 7ff3f718b640 1 --2- 172.21.15.47:0/3058360942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f006b090 0x7ff3f00d00c0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.018+0000 7ff3f718b640 1 -- 172.21.15.47:0/3058360942 >> 172.21.15.47:0/3058360942 conn(0x7ff3f00b2140 msgr2=0x7ff3f00b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.018+0000 7ff3f718b640 1 -- 172.21.15.47:0/3058360942 shutdown_connections 2024-03-23T09:49:43.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.018+0000 7ff3f718b640 1 -- 172.21.15.47:0/3058360942 wait complete. 2024-03-23T09:49:43.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.020+0000 7ff3f718b640 1 Processor -- start 2024-03-23T09:49:43.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.020+0000 7ff3f718b640 1 -- start start 2024-03-23T09:49:43.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.020+0000 7ff3f718b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f006b090 0x7ff3f01f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.020+0000 7ff3f718b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3f0099110 0x7ff3f009c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.020+0000 7ff3f718b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3f015cd40 0x7ff3f0146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.020+0000 7ff3f718b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f009c460 con 0x7ff3f006b090 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.020+0000 7ff3f718b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f009b850 con 0x7ff3f0099110 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.020+0000 7ff3f718b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f009bb30 con 0x7ff3f015cd40 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3f4f27640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f006b090 0x7ff3f01f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3f4f27640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f006b090 0x7ff3f01f1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44060/0 (socket says 172.21.15.47:44060) 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3f4f27640 1 -- 172.21.15.47:0/1887488177 learned_addr learned my addr 172.21.15.47:0/1887488177 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3f5728640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3f015cd40 0x7ff3f0146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3e7fff640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3f0099110 0x7ff3f009c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3f4f27640 1 -- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3f015cd40 msgr2=0x7ff3f0146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3f4f27640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3f015cd40 0x7ff3f0146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3f4f27640 1 -- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3f0099110 msgr2=0x7ff3f009c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3f4f27640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3f0099110 0x7ff3f009c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3f4f27640 1 -- 172.21.15.47:0/1887488177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e8009a70 con 0x7ff3f006b090 2024-03-23T09:49:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3e7fff640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3f0099110 0x7ff3f009c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3f4f27640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f006b090 0x7ff3f01f1e50 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00b3e0 tx=0x7ff3ec00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3e57fa640 1 -- 172.21.15.47:0/1887488177 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec011830 con 0x7ff3f006b090 2024-03-23T09:49:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3e57fa640 1 -- 172.21.15.47:0/1887488177 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec0119b0 con 0x7ff3f006b090 2024-03-23T09:49:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3f718b640 1 -- 172.21.15.47:0/1887488177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f009be10 con 0x7ff3f006b090 2024-03-23T09:49:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3e57fa640 1 -- 172.21.15.47:0/1887488177 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec011d60 con 0x7ff3f006b090 2024-03-23T09:49:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.021+0000 7ff3f718b640 1 -- 172.21.15.47:0/1887488177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f01f2390 con 0x7ff3f006b090 2024-03-23T09:49:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.022+0000 7ff3f718b640 1 -- 172.21.15.47:0/1887488177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f0066be0 con 0x7ff3f006b090 2024-03-23T09:49:43.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.023+0000 7ff3e57fa640 1 -- 172.21.15.47:0/1887488177 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff3ec01a400 con 0x7ff3f006b090 2024-03-23T09:49:43.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.023+0000 7ff3e57fa640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff3dc06ac00 0x7ff3dc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.023+0000 7ff3e57fa640 1 -- 172.21.15.47:0/1887488177 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ff3ec089480 con 0x7ff3f006b090 2024-03-23T09:49:43.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.023+0000 7ff3e7fff640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff3dc06ac00 0x7ff3dc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.024+0000 7ff3e7fff640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff3dc06ac00 0x7ff3dc06d0c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0146360 tx=0x7ff3e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:43.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.028+0000 7ff3e57fa640 1 -- 172.21.15.47:0/1887488177 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff3ec054630 con 0x7ff3f006b090 2024-03-23T09:49:43.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.245+0000 7ff3f718b640 1 -- 172.21.15.47:0/1887488177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff3f00b12a0 con 0x7ff3f006b090 2024-03-23T09:49:43.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.254+0000 7ff3e57fa640 1 -- 172.21.15.47:0/1887488177 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v63) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff3ec0150c0 con 0x7ff3f006b090 2024-03-23T09:49:43.254 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 14 and data pool 15 2024-03-23T09:49:43.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.262+0000 7ff3f718b640 1 -- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff3dc06ac00 msgr2=0x7ff3dc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.262+0000 7ff3f718b640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff3dc06ac00 0x7ff3dc06d0c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0146360 tx=0x7ff3e0009040 comp rx=0 tx=0).stop 2024-03-23T09:49:43.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.262+0000 7ff3f718b640 1 -- 172.21.15.47:0/1887488177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f006b090 msgr2=0x7ff3f01f1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.262+0000 7ff3f718b640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f006b090 0x7ff3f01f1e50 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00b3e0 tx=0x7ff3ec00b8a0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.263+0000 7ff3f718b640 1 -- 172.21.15.47:0/1887488177 shutdown_connections 2024-03-23T09:49:43.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.263+0000 7ff3f718b640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff3dc06ac00 0x7ff3dc06d0c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.263+0000 7ff3f718b640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3f015cd40 0x7ff3f0146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.263+0000 7ff3f718b640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3f0099110 0x7ff3f009c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.263+0000 7ff3f718b640 1 --2- 172.21.15.47:0/1887488177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3f006b090 0x7ff3f01f1e50 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.263+0000 7ff3f718b640 1 -- 172.21.15.47:0/1887488177 >> 172.21.15.47:0/1887488177 conn(0x7ff3f00b2140 msgr2=0x7ff3f006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:43.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.264+0000 7ff3f718b640 1 -- 172.21.15.47:0/1887488177 shutdown_connections 2024-03-23T09:49:43.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.264+0000 7ff3f718b640 1 -- 172.21.15.47:0/1887488177 wait complete. 2024-03-23T09:49:43.282 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:49:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.382+0000 7f0136d55640 1 Processor -- start 2024-03-23T09:49:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.382+0000 7f0136d55640 1 -- start start 2024-03-23T09:49:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.383+0000 7f0136d55640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015ca90 0x7f0130160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.383+0000 7f0136d55640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01300d11e0 con 0x7f01300cde10 2024-03-23T09:49:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.383+0000 7f0136d55640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0130160f60 con 0x7f01300b8350 2024-03-23T09:49:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.383+0000 7f0136d55640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0130161240 con 0x7f013015ca90 2024-03-23T09:49:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.383+0000 7f01352f2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f01300cde10 0x7f01300d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44620/0 (socket says 172.21.15.47:44620) 2024-03-23T09:49:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.383+0000 7f01352f2640 1 -- 172.21.15.47:0/1691750447 learned_addr learned my addr 172.21.15.47:0/1691750447 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f0134af1640 1 --2- 172.21.15.47:0/1691750447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015ca90 0x7f0130160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519562301 0 0) 0x7f01300d11e0 con 0x7f01300cde10 2024-03-23T09:49:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f011c003520 con 0x7f01300cde10 2024-03-23T09:49:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2061747774 0 0) 0x7f011c003520 con 0x7f01300cde10 2024-03-23T09:49:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015ca90 msgr2=0x7f0130160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 --2- 172.21.15.47:0/1691750447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015ca90 0x7f0130160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 >> v1:172.21.15.175:6789/0 conn(0x7f01300b8350 legacy=0x7f01300cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0130161520 con 0x7f01300cde10 2024-03-23T09:49:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0118001670 con 0x7f01300cde10 2024-03-23T09:49:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 >> v1:172.21.15.47:6789/0 conn(0x7f01300cde10 legacy=0x7f01300d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 --2- 172.21.15.47:0/1691750447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015ca90 0x7f011c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 --2- 172.21.15.47:0/1691750447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f011c003ee0 0x7f011c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 --2- 172.21.15.47:0/1691750447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011c004d00 0x7f011c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0130161240 con 0x7f011c003ee0 2024-03-23T09:49:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c00ba90 con 0x7f011c004d00 2024-03-23T09:49:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011c00bc50 con 0x7f013015ca90 2024-03-23T09:49:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012ffff640 1 --2- 172.21.15.47:0/1691750447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f011c003ee0 0x7f011c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f0134af1640 1 --2- 172.21.15.47:0/1691750447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015ca90 0x7f011c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f01352f2640 1 --2- 172.21.15.47:0/1691750447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011c004d00 0x7f011c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012ffff640 1 -- 172.21.15.47:0/1691750447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015ca90 msgr2=0x7f011c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012ffff640 1 --2- 172.21.15.47:0/1691750447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015ca90 0x7f011c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012ffff640 1 -- 172.21.15.47:0/1691750447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011c004d00 msgr2=0x7f011c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012ffff640 1 --2- 172.21.15.47:0/1691750447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011c004d00 0x7f011c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.384+0000 7f012ffff640 1 -- 172.21.15.47:0/1691750447 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c003520 con 0x7f011c003ee0 2024-03-23T09:49:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.385+0000 7f012ffff640 1 --2- 172.21.15.47:0/1691750447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f011c003ee0 0x7f011c0047c0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0120001f60 tx=0x7f011c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5473d717b08ed22a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.385+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012002d6f0 con 0x7f011c003ee0 2024-03-23T09:49:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.385+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012002d870 con 0x7f011c003ee0 2024-03-23T09:49:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.385+0000 7f012f7fe640 1 -- 172.21.15.47:0/1691750447 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012002dc20 con 0x7f011c003ee0 2024-03-23T09:49:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.386+0000 7f0136d55640 1 -- 172.21.15.47:0/1691750447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f011c003ee0 msgr2=0x7f011c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.386+0000 7f0136d55640 1 --2- 172.21.15.47:0/1691750447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f011c003ee0 0x7f011c0047c0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0120001f60 tx=0x7f011c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.386+0000 7f01352f2640 1 -- 172.21.15.47:0/1691750447 reap_dead start 2024-03-23T09:49:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.386+0000 7f0136d55640 1 -- 172.21.15.47:0/1691750447 shutdown_connections 2024-03-23T09:49:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.386+0000 7f0136d55640 1 -- 172.21.15.47:0/1691750447 >> 172.21.15.47:0/1691750447 conn(0x7f01300659d0 msgr2=0x7f0130065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.386+0000 7f0136d55640 1 -- 172.21.15.47:0/1691750447 shutdown_connections 2024-03-23T09:49:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.386+0000 7f0136d55640 1 -- 172.21.15.47:0/1691750447 wait complete. 2024-03-23T09:49:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.387+0000 7f0136d55640 1 Processor -- start 2024-03-23T09:49:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.387+0000 7f0136d55640 1 -- start start 2024-03-23T09:49:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f0136d55640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f011c003ee0 0x7f01300c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f0136d55640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01300b3610 0x7f01300d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f0136d55640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01300b3a90 0x7f01300d3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f0136d55640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01300ca1f0 con 0x7f01300b3610 2024-03-23T09:49:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f0136d55640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0130152610 con 0x7f01300b3a90 2024-03-23T09:49:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f0136d55640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01301528f0 con 0x7f011c003ee0 2024-03-23T09:49:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f012ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01300b3610 0x7f01300d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f012ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01300b3610 0x7f01300d2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44076/0 (socket says 172.21.15.47:44076) 2024-03-23T09:49:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f012ffff640 1 -- 172.21.15.47:0/97484060 learned_addr learned my addr 172.21.15.47:0/97484060 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f012ffff640 1 -- 172.21.15.47:0/97484060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f011c003ee0 msgr2=0x7f01300c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f012ffff640 1 --2- 172.21.15.47:0/97484060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f011c003ee0 0x7f01300c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f01352f2640 1 --2- 172.21.15.47:0/97484060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01300b3a90 0x7f01300d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f012ffff640 1 -- 172.21.15.47:0/97484060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01300b3a90 msgr2=0x7f01300d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f012ffff640 1 --2- 172.21.15.47:0/97484060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01300b3a90 0x7f01300d3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f012ffff640 1 -- 172.21.15.47:0/97484060 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01300c9b30 con 0x7f01300b3610 2024-03-23T09:49:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.388+0000 7f012ffff640 1 --2- 172.21.15.47:0/97484060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01300b3610 0x7f01300d2c50 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f012002c080 tx=0x7f01200015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.389+0000 7f012d7fa640 1 -- 172.21.15.47:0/97484060 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120003d80 con 0x7f01300b3610 2024-03-23T09:49:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.389+0000 7f012d7fa640 1 -- 172.21.15.47:0/97484060 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012002de10 con 0x7f01300b3610 2024-03-23T09:49:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.389+0000 7f0136d55640 1 -- 172.21.15.47:0/97484060 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01300c9e10 con 0x7f01300b3610 2024-03-23T09:49:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.389+0000 7f012d7fa640 1 -- 172.21.15.47:0/97484060 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01200052b0 con 0x7f01300b3610 2024-03-23T09:49:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.389+0000 7f0136d55640 1 -- 172.21.15.47:0/97484060 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0130151b00 con 0x7f01300b3610 2024-03-23T09:49:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.390+0000 7f0136d55640 1 -- 172.21.15.47:0/97484060 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01300ca460 con 0x7f01300b3610 2024-03-23T09:49:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.390+0000 7f012d7fa640 1 -- 172.21.15.47:0/97484060 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0120005410 con 0x7f01300b3610 2024-03-23T09:49:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.390+0000 7f012d7fa640 1 --2- 172.21.15.47:0/97484060 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f011006ac00 0x7f011006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.391+0000 7f012d7fa640 1 -- 172.21.15.47:0/97484060 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0120031070 con 0x7f01300b3610 2024-03-23T09:49:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.391+0000 7f0134af1640 1 --2- 172.21.15.47:0/97484060 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f011006ac00 0x7f011006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.391+0000 7f0134af1640 1 --2- 172.21.15.47:0/97484060 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f011006ac00 0x7f011006d0c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0128009ab0 tx=0x7f01280095d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.395+0000 7f012d7fa640 1 -- 172.21.15.47:0/97484060 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0120076c60 con 0x7f01300b3610 2024-03-23T09:49:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.575+0000 7f0136d55640 1 -- 172.21.15.47:0/97484060 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0130072360 con 0x7f01300b3610 2024-03-23T09:49:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.576+0000 7f012d7fa640 1 -- 172.21.15.47:0/97484060 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16056 (secure 0 0 0) 0x7f0120043020 con 0x7f01300b3610 2024-03-23T09:49:43.576 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:43.576 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":100,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:49:43.246738+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:49:41.158056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-23T09:49:42.210388+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:49:43.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.578+0000 7f0136d55640 1 -- 172.21.15.47:0/97484060 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f011006ac00 msgr2=0x7f011006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.578+0000 7f0136d55640 1 --2- 172.21.15.47:0/97484060 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f011006ac00 0x7f011006d0c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0128009ab0 tx=0x7f01280095d0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.578+0000 7f0136d55640 1 -- 172.21.15.47:0/97484060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01300b3610 msgr2=0x7f01300d2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.578+0000 7f0136d55640 1 --2- 172.21.15.47:0/97484060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01300b3610 0x7f01300d2c50 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f012002c080 tx=0x7f01200015b0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.578+0000 7f0136d55640 1 -- 172.21.15.47:0/97484060 shutdown_connections 2024-03-23T09:49:43.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.578+0000 7f0136d55640 1 --2- 172.21.15.47:0/97484060 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f011006ac00 0x7f011006d0c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.578+0000 7f0136d55640 1 --2- 172.21.15.47:0/97484060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01300b3a90 0x7f01300d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.578+0000 7f0136d55640 1 --2- 172.21.15.47:0/97484060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01300b3610 0x7f01300d2c50 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.578+0000 7f0136d55640 1 --2- 172.21.15.47:0/97484060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f011c003ee0 0x7f01300c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.578+0000 7f0136d55640 1 -- 172.21.15.47:0/97484060 >> 172.21.15.47:0/97484060 conn(0x7f01300659d0 msgr2=0x7f01300ce3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.578+0000 7f0136d55640 1 -- 172.21.15.47:0/97484060 shutdown_connections 2024-03-23T09:49:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.578+0000 7f0136d55640 1 -- 172.21.15.47:0/97484060 wait complete. 2024-03-23T09:49:43.589 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:49:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.690+0000 7ff879977640 1 Processor -- start 2024-03-23T09:49:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.690+0000 7ff879977640 1 -- start start 2024-03-23T09:49:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff879977640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740cdcb0 0x7ff8740b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff879977640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8740b72f0 con 0x7ff87415cb50 2024-03-23T09:49:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff879977640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8740b75d0 con 0x7ff8740d2020 2024-03-23T09:49:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff879977640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740b7e70 con 0x7ff8740cdcb0 2024-03-23T09:49:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff872ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff87415cb50 0x7ff8740cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44624/0 (socket says 172.21.15.47:44624) 2024-03-23T09:49:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff872ffd640 1 -- 172.21.15.47:0/891068302 learned_addr learned my addr 172.21.15.47:0/891068302 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff86a7fc640 1 --2- 172.21.15.47:0/891068302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740cdcb0 0x7ff8740b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382649782 0 0) 0x7ff8740b72f0 con 0x7ff87415cb50 2024-03-23T09:49:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff860003520 con 0x7ff87415cb50 2024-03-23T09:49:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220985348 0 0) 0x7ff8740b75d0 con 0x7ff8740d2020 2024-03-23T09:49:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8740b72f0 con 0x7ff8740d2020 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 650388698 0 0) 0x7ff860003520 con 0x7ff87415cb50 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740cdcb0 msgr2=0x7ff8740b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 --2- 172.21.15.47:0/891068302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740cdcb0 0x7ff8740b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 >> v1:172.21.15.175:6789/0 conn(0x7ff8740d2020 legacy=0x7ff8740b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8740b78b0 con 0x7ff87415cb50 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7ff864001670 con 0x7ff87415cb50 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 >> v1:172.21.15.47:6789/0 conn(0x7ff87415cb50 legacy=0x7ff8740cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 --2- 172.21.15.47:0/891068302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740cdcb0 0x7ff860003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 --2- 172.21.15.47:0/891068302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff860004090 0x7ff8600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 --2- 172.21.15.47:0/891068302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d00 0x7ff860005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740b7e70 con 0x7ff860004d00 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86000ba60 con 0x7ff860004090 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.691+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86000bc20 con 0x7ff8740cdcb0 2024-03-23T09:49:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.692+0000 7ff8737fe640 1 --2- 172.21.15.47:0/891068302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d00 0x7ff860005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.692+0000 7ff872ffd640 1 --2- 172.21.15.47:0/891068302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740cdcb0 0x7ff860003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.692+0000 7ff86a7fc640 1 --2- 172.21.15.47:0/891068302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff860004090 0x7ff8600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.692+0000 7ff8737fe640 1 -- 172.21.15.47:0/891068302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740cdcb0 msgr2=0x7ff860003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.692+0000 7ff8737fe640 1 --2- 172.21.15.47:0/891068302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740cdcb0 0x7ff860003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.692+0000 7ff8737fe640 1 -- 172.21.15.47:0/891068302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff860004090 msgr2=0x7ff8600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.692+0000 7ff8737fe640 1 --2- 172.21.15.47:0/891068302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff860004090 0x7ff8600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.692+0000 7ff8737fe640 1 -- 172.21.15.47:0/891068302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff860003520 con 0x7ff860004d00 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.692+0000 7ff86a7fc640 1 --2- 172.21.15.47:0/891068302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff860004090 0x7ff8600047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.693+0000 7ff8737fe640 1 --2- 172.21.15.47:0/891068302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d00 0x7ff860005180 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff86c001fa0 tx=0x7ff8600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f5ae494f361421 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.693+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c003b70 con 0x7ff860004d00 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.693+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c003cd0 con 0x7ff860004d00 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.693+0000 7ff8727fc640 1 -- 172.21.15.47:0/891068302 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0052f0 con 0x7ff860004d00 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.693+0000 7ff879977640 1 -- 172.21.15.47:0/891068302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d00 msgr2=0x7ff860005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.693+0000 7ff879977640 1 --2- 172.21.15.47:0/891068302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d00 0x7ff860005180 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff86c001fa0 tx=0x7ff8600057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.693+0000 7ff8737fe640 1 -- 172.21.15.47:0/891068302 reap_dead start 2024-03-23T09:49:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.693+0000 7ff879977640 1 -- 172.21.15.47:0/891068302 shutdown_connections 2024-03-23T09:49:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.693+0000 7ff879977640 1 -- 172.21.15.47:0/891068302 >> 172.21.15.47:0/891068302 conn(0x7ff874065df0 msgr2=0x7ff874066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.693+0000 7ff879977640 1 -- 172.21.15.47:0/891068302 shutdown_connections 2024-03-23T09:49:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.693+0000 7ff879977640 1 -- 172.21.15.47:0/891068302 wait complete. 2024-03-23T09:49:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.694+0000 7ff879977640 1 Processor -- start 2024-03-23T09:49:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.695+0000 7ff879977640 1 -- start start 2024-03-23T09:49:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.695+0000 7ff879977640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8741f1a50 0x7ff874145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.695+0000 7ff879977640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87415fd30 0x7ff8740b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.695+0000 7ff879977640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87415e740 0x7ff8740b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.695+0000 7ff879977640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87415f8a0 con 0x7ff87415fd30 2024-03-23T09:49:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.695+0000 7ff879977640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740c8f50 con 0x7ff8741f1a50 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.695+0000 7ff879977640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff874145990 con 0x7ff87415e740 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.695+0000 7ff86a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87415fd30 0x7ff8740b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.695+0000 7ff86a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87415fd30 0x7ff8740b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44094/0 (socket says 172.21.15.47:44094) 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.695+0000 7ff86a7fc640 1 -- 172.21.15.47:0/2736796399 learned_addr learned my addr 172.21.15.47:0/2736796399 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.696+0000 7ff872ffd640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8741f1a50 0x7ff874145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.696+0000 7ff86a7fc640 1 -- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87415e740 msgr2=0x7ff8740b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.696+0000 7ff8737fe640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87415e740 0x7ff8740b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.696+0000 7ff86a7fc640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87415e740 0x7ff8740b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.696+0000 7ff86a7fc640 1 -- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8741f1a50 msgr2=0x7ff874145450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.696+0000 7ff86a7fc640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8741f1a50 0x7ff874145450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.696+0000 7ff86a7fc640 1 -- 172.21.15.47:0/2736796399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87415ec90 con 0x7ff87415fd30 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.696+0000 7ff86a7fc640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87415fd30 0x7ff8740b5c60 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7ff85c011600 tx=0x7ff85c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.696+0000 7ff870ff9640 1 -- 172.21.15.47:0/2736796399 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c00f920 con 0x7ff87415fd30 2024-03-23T09:49:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.696+0000 7ff870ff9640 1 -- 172.21.15.47:0/2736796399 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85c00fa80 con 0x7ff87415fd30 2024-03-23T09:49:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.696+0000 7ff879977640 1 -- 172.21.15.47:0/2736796399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87415ef70 con 0x7ff87415fd30 2024-03-23T09:49:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.696+0000 7ff870ff9640 1 -- 172.21.15.47:0/2736796399 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c00fe10 con 0x7ff87415fd30 2024-03-23T09:49:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.696+0000 7ff879977640 1 -- 172.21.15.47:0/2736796399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87415f530 con 0x7ff87415fd30 2024-03-23T09:49:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.697+0000 7ff879977640 1 -- 172.21.15.47:0/2736796399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff874072360 con 0x7ff87415fd30 2024-03-23T09:49:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.697+0000 7ff870ff9640 1 -- 172.21.15.47:0/2736796399 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff85c015460 con 0x7ff87415fd30 2024-03-23T09:49:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.698+0000 7ff870ff9640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff85406ac30 0x7ff85406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.698+0000 7ff870ff9640 1 -- 172.21.15.47:0/2736796399 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff85c089bc0 con 0x7ff87415fd30 2024-03-23T09:49:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.698+0000 7ff872ffd640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff85406ac30 0x7ff85406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.698+0000 7ff872ffd640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff85406ac30 0x7ff85406d0f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff864001eb0 tx=0x7ff864001110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.702+0000 7ff870ff9640 1 -- 172.21.15.47:0/2736796399 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff85c054da0 con 0x7ff87415fd30 2024-03-23T09:49:43.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.887+0000 7ff879977640 1 -- 172.21.15.47:0/2736796399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff874066be0 con 0x7ff87415fd30 2024-03-23T09:49:43.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.888+0000 7ff870ff9640 1 -- 172.21.15.47:0/2736796399 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16056 (secure 0 0 0) 0x7ff85c0583c0 con 0x7ff87415fd30 2024-03-23T09:49:43.888 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:43.888 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":100,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:49:43.246738+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:49:41.158056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-23T09:49:42.210388+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:49:43.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.890+0000 7ff879977640 1 -- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff85406ac30 msgr2=0x7ff85406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.890+0000 7ff879977640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff85406ac30 0x7ff85406d0f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff864001eb0 tx=0x7ff864001110 comp rx=0 tx=0).stop 2024-03-23T09:49:43.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.890+0000 7ff879977640 1 -- 172.21.15.47:0/2736796399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87415fd30 msgr2=0x7ff8740b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:43.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.890+0000 7ff879977640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87415fd30 0x7ff8740b5c60 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7ff85c011600 tx=0x7ff85c011ac0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.890+0000 7ff879977640 1 -- 172.21.15.47:0/2736796399 shutdown_connections 2024-03-23T09:49:43.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.890+0000 7ff879977640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff85406ac30 0x7ff85406d0f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.890+0000 7ff879977640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87415e740 0x7ff8740b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.890+0000 7ff879977640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87415fd30 0x7ff8740b5c60 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.890+0000 7ff879977640 1 --2- 172.21.15.47:0/2736796399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8741f1a50 0x7ff874145450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:43.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.890+0000 7ff879977640 1 -- 172.21.15.47:0/2736796399 >> 172.21.15.47:0/2736796399 conn(0x7ff874065df0 msgr2=0x7ff8740b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:43.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.890+0000 7ff879977640 1 -- 172.21.15.47:0/2736796399 shutdown_connections 2024-03-23T09:49:43.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:43.890+0000 7ff879977640 1 -- 172.21.15.47:0/2736796399 wait complete. 2024-03-23T09:49:43.901 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T09:49:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.005+0000 7f285044a640 1 Processor -- start 2024-03-23T09:49:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.005+0000 7f285044a640 1 -- start start 2024-03-23T09:49:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.006+0000 7f285044a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f284815cb50 0x7f28480b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.006+0000 7f285044a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f28480cd9e0 0x7f28480b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.006+0000 7f285044a640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28480b72f0 con 0x7f28480cd5a0 2024-03-23T09:49:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.006+0000 7f285044a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28480b7e70 con 0x7f28480cd9e0 2024-03-23T09:49:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.006+0000 7f285044a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284815cf30 con 0x7f284815cb50 2024-03-23T09:49:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.006+0000 7f284e9e7640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f28480cd9e0 0x7f28480b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.006+0000 7f284e1e6640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f284815cb50 0x7f28480b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.006+0000 7f284d9e5640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f28480cd5a0 0x7f28480d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:35464/0 (socket says 172.21.15.47:35464) 2024-03-23T09:49:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.006+0000 7f284d9e5640 1 -- 172.21.15.47:0/2444373112 learned_addr learned my addr 172.21.15.47:0/2444373112 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.007+0000 7f284e9e7640 1 -- 172.21.15.47:0/2444373112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f284815cb50 msgr2=0x7f28480b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.007+0000 7f284e9e7640 1 --2- 172.21.15.47:0/2444373112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f284815cb50 0x7f28480b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.007+0000 7f284e9e7640 1 -- 172.21.15.47:0/2444373112 >> v1:172.21.15.175:6789/0 conn(0x7f28480cd5a0 legacy=0x7f28480d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.007+0000 7f284d1e4640 1 -- 172.21.15.47:0/2444373112 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985624310 0 0) 0x7f28480b72f0 con 0x7f28480cd5a0 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.007+0000 7f284e9e7640 1 -- 172.21.15.47:0/2444373112 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28480b75d0 con 0x7f28480cd9e0 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.007+0000 7f284e1e6640 1 --2- 172.21.15.47:0/2444373112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f284815cb50 0x7f28480b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.008+0000 7f284e9e7640 1 --2- 172.21.15.47:0/2444373112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28480cd9e0 0x7f28480b6a10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2838009d90 tx=0x7f283802bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6efb8c3a4d7a327f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.008+0000 7f284d1e4640 1 -- 172.21.15.47:0/2444373112 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283802cce0 con 0x7f28480cd9e0 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.008+0000 7f284d1e4640 1 -- 172.21.15.47:0/2444373112 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283802ce40 con 0x7f28480cd9e0 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.008+0000 7f284d1e4640 1 -- 172.21.15.47:0/2444373112 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838034660 con 0x7f28480cd9e0 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.008+0000 7f285044a640 1 -- 172.21.15.47:0/2444373112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28480cd9e0 msgr2=0x7f28480b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.008+0000 7f285044a640 1 --2- 172.21.15.47:0/2444373112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28480cd9e0 0x7f28480b6a10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2838009d90 tx=0x7f283802bf30 comp rx=0 tx=0).stop 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.008+0000 7f285044a640 1 -- 172.21.15.47:0/2444373112 shutdown_connections 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.008+0000 7f285044a640 1 --2- 172.21.15.47:0/2444373112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28480cd9e0 0x7f28480b6a10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.008+0000 7f285044a640 1 --2- 172.21.15.47:0/2444373112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f284815cb50 0x7f28480b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.008+0000 7f285044a640 1 -- 172.21.15.47:0/2444373112 >> 172.21.15.47:0/2444373112 conn(0x7f2848065df0 msgr2=0x7f2848066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.008+0000 7f285044a640 1 -- 172.21.15.47:0/2444373112 shutdown_connections 2024-03-23T09:49:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.008+0000 7f285044a640 1 -- 172.21.15.47:0/2444373112 wait complete. 2024-03-23T09:49:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.009+0000 7f285044a640 1 Processor -- start 2024-03-23T09:49:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.010+0000 7f285044a640 1 -- start start 2024-03-23T09:49:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.010+0000 7f285044a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28480cd5a0 0x7f2848156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.010+0000 7f285044a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28480cd9e0 0x7f28480a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.010+0000 7f285044a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f284815cb50 0x7f284815f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.010+0000 7f285044a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2848156c70 con 0x7f284815cb50 2024-03-23T09:49:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.010+0000 7f285044a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28481589c0 con 0x7f28480cd9e0 2024-03-23T09:49:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.010+0000 7f285044a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2848157db0 con 0x7f28480cd5a0 2024-03-23T09:49:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.010+0000 7f284e9e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f284815cb50 0x7f284815f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.010+0000 7f284e9e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f284815cb50 0x7f284815f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44100/0 (socket says 172.21.15.47:44100) 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.010+0000 7f284e9e7640 1 -- 172.21.15.47:0/3368110747 learned_addr learned my addr 172.21.15.47:0/3368110747 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.010+0000 7f284d9e5640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28480cd9e0 0x7f28480a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.011+0000 7f284e1e6640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28480cd5a0 0x7f2848156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.011+0000 7f284e9e7640 1 -- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28480cd5a0 msgr2=0x7f2848156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.011+0000 7f284e9e7640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28480cd5a0 0x7f2848156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.011+0000 7f284e9e7640 1 -- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28480cd9e0 msgr2=0x7f28480a9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.011+0000 7f284e9e7640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28480cd9e0 0x7f28480a9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.011+0000 7f284e9e7640 1 -- 172.21.15.47:0/3368110747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2840009400 con 0x7f284815cb50 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.011+0000 7f284d9e5640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28480cd9e0 0x7f28480a9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.011+0000 7f284e9e7640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f284815cb50 0x7f284815f620 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f2838009d90 tx=0x7f2838004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.011+0000 7f2846ffd640 1 -- 172.21.15.47:0/3368110747 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838003b90 con 0x7f284815cb50 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.011+0000 7f2846ffd640 1 -- 172.21.15.47:0/3368110747 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2838003cf0 con 0x7f284815cb50 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.011+0000 7f285044a640 1 -- 172.21.15.47:0/3368110747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2838009a70 con 0x7f284815cb50 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.011+0000 7f2846ffd640 1 -- 172.21.15.47:0/3368110747 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838004250 con 0x7f284815cb50 2024-03-23T09:49:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.011+0000 7f285044a640 1 -- 172.21.15.47:0/3368110747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2848158370 con 0x7f284815cb50 2024-03-23T09:49:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.012+0000 7f285044a640 1 -- 172.21.15.47:0/3368110747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2848072360 con 0x7f284815cb50 2024-03-23T09:49:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.012+0000 7f2846ffd640 1 -- 172.21.15.47:0/3368110747 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f283803b050 con 0x7f284815cb50 2024-03-23T09:49:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.013+0000 7f2846ffd640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f283406ac30 0x7f283406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.013+0000 7f2846ffd640 1 -- 172.21.15.47:0/3368110747 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28380ab7e0 con 0x7f284815cb50 2024-03-23T09:49:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.013+0000 7f284e1e6640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f283406ac30 0x7f283406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.014+0000 7f284e1e6640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f283406ac30 0x7f283406d0f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2840009bc0 tx=0x7f2840008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:44.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.017+0000 7f2846ffd640 1 -- 172.21.15.47:0/3368110747 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2838076a50 con 0x7f284815cb50 2024-03-23T09:49:44.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.236+0000 7f285044a640 1 -- 172.21.15.47:0/3368110747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2848066be0 con 0x7f284815cb50 2024-03-23T09:49:44.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.251+0000 7f2846ffd640 1 -- 172.21.15.47:0/3368110747 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) v1 ==== 118+0+0 (secure 0 0 0) 0x7f28380390b0 con 0x7f284815cb50 2024-03-23T09:49:44.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.253+0000 7f285044a640 1 -- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f283406ac30 msgr2=0x7f283406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:44.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.253+0000 7f285044a640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f283406ac30 0x7f283406d0f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2840009bc0 tx=0x7f2840008040 comp rx=0 tx=0).stop 2024-03-23T09:49:44.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.253+0000 7f285044a640 1 -- 172.21.15.47:0/3368110747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f284815cb50 msgr2=0x7f284815f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:44.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.253+0000 7f285044a640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f284815cb50 0x7f284815f620 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f2838009d90 tx=0x7f2838004a40 comp rx=0 tx=0).stop 2024-03-23T09:49:44.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.253+0000 7f285044a640 1 -- 172.21.15.47:0/3368110747 shutdown_connections 2024-03-23T09:49:44.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.253+0000 7f285044a640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f283406ac30 0x7f283406d0f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.253+0000 7f285044a640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f284815cb50 0x7f284815f620 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.253+0000 7f285044a640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28480cd9e0 0x7f28480a9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.253+0000 7f285044a640 1 --2- 172.21.15.47:0/3368110747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28480cd5a0 0x7f2848156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.253+0000 7f285044a640 1 -- 172.21.15.47:0/3368110747 >> 172.21.15.47:0/3368110747 conn(0x7f2848065df0 msgr2=0x7f28480cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:44.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.253+0000 7f285044a640 1 -- 172.21.15.47:0/3368110747 shutdown_connections 2024-03-23T09:49:44.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.253+0000 7f285044a640 1 -- 172.21.15.47:0/3368110747 wait complete. 2024-03-23T09:49:44.264 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-23T09:49:44.265 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-23T09:49:44.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.368+0000 7feaf8ae0640 1 Processor -- start 2024-03-23T09:49:44.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.368+0000 7feaf8ae0640 1 -- start start 2024-03-23T09:49:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf8ae0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feaf415cbf0 0x7feaf40b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf8ae0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf4161570 con 0x7feaf40cf9c0 2024-03-23T09:49:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf8ae0640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf4160960 con 0x7feaf40cd140 2024-03-23T09:49:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf8ae0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf4160c40 con 0x7feaf415cbf0 2024-03-23T09:49:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf2d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feaf40cf9c0 0x7feaf40d0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44638/0 (socket says 172.21.15.47:44638) 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf2d9d640 1 -- 172.21.15.47:0/842604090 learned_addr learned my addr 172.21.15.47:0/842604090 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176431249 0 0) 0x7feaf4161570 con 0x7feaf40cf9c0 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feae0003520 con 0x7feaf40cf9c0 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf259c640 1 --2- 172.21.15.47:0/842604090 >> v2:172.21.15.175:3300/0 conn(0x7feaf415cbf0 0x7feaf40b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 286646820 0 0) 0x7feae0003520 con 0x7feaf40cf9c0 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 >> v2:172.21.15.175:3300/0 conn(0x7feaf415cbf0 msgr2=0x7feaf40b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf1d9b640 1 --2- 172.21.15.47:0/842604090 >> v2:172.21.15.175:3300/0 conn(0x7feaf415cbf0 0x7feaf40b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 >> v1:172.21.15.175:6789/0 conn(0x7feaf40cd140 legacy=0x7feaf40cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf4160f20 con 0x7feaf40cf9c0 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7feaec0019f0 con 0x7feaf40cf9c0 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 >> v1:172.21.15.47:6789/0 conn(0x7feaf40cf9c0 legacy=0x7feaf40d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.369+0000 7feaf1d9b640 1 --2- 172.21.15.47:0/842604090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feae0003970 0x7feae0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.370+0000 7feaf1d9b640 1 --2- 172.21.15.47:0/842604090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feae00047c0 0x7feae0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.370+0000 7feaf1d9b640 1 --2- 172.21.15.47:0/842604090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feae0005180 0x7feae000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.370+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4160c40 con 0x7feae0005180 2024-03-23T09:49:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.370+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae000c0d0 con 0x7feae0003970 2024-03-23T09:49:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.370+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae000c290 con 0x7feae00047c0 2024-03-23T09:49:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.370+0000 7feaf2d9d640 1 --2- 172.21.15.47:0/842604090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feae0005180 0x7feae000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.370+0000 7feaf259c640 1 --2- 172.21.15.47:0/842604090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feae0003970 0x7feae0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.370+0000 7feaebfff640 1 --2- 172.21.15.47:0/842604090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feae00047c0 0x7feae0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.370+0000 7feaf2d9d640 1 -- 172.21.15.47:0/842604090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feae00047c0 msgr2=0x7feae0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.370+0000 7feaf2d9d640 1 --2- 172.21.15.47:0/842604090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feae00047c0 0x7feae0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.370+0000 7feaf2d9d640 1 -- 172.21.15.47:0/842604090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feae0003970 msgr2=0x7feae0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.370+0000 7feaf2d9d640 1 --2- 172.21.15.47:0/842604090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feae0003970 0x7feae0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.370+0000 7feaf2d9d640 1 -- 172.21.15.47:0/842604090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae0003520 con 0x7feae0005180 2024-03-23T09:49:44.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.371+0000 7feaf2d9d640 1 -- 172.21.15.47:0/842604090 reap_dead start 2024-03-23T09:49:44.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.371+0000 7feaf2d9d640 1 --2- 172.21.15.47:0/842604090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feae0005180 0x7feae000b980 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7feaec00a830 tx=0x7feae00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7101d7621bc1247d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:44.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.371+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec0072d0 con 0x7feae0005180 2024-03-23T09:49:44.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.371+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec02f070 con 0x7feae0005180 2024-03-23T09:49:44.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.371+0000 7feaf1d9b640 1 -- 172.21.15.47:0/842604090 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec0076b0 con 0x7feae0005180 2024-03-23T09:49:44.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.371+0000 7feaf8ae0640 1 -- 172.21.15.47:0/842604090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feae0005180 msgr2=0x7feae000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:44.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.371+0000 7feaf8ae0640 1 --2- 172.21.15.47:0/842604090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feae0005180 0x7feae000b980 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7feaec00a830 tx=0x7feae00057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.371+0000 7feaf8ae0640 1 -- 172.21.15.47:0/842604090 shutdown_connections 2024-03-23T09:49:44.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.371+0000 7feaf8ae0640 1 --2- 172.21.15.47:0/842604090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feae0005180 0x7feae000b980 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.371+0000 7feaf8ae0640 1 -- 172.21.15.47:0/842604090 >> 172.21.15.47:0/842604090 conn(0x7feaf406b160 msgr2=0x7feaf406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:44.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.371+0000 7feaf8ae0640 1 -- 172.21.15.47:0/842604090 shutdown_connections 2024-03-23T09:49:44.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.371+0000 7feaf8ae0640 1 -- 172.21.15.47:0/842604090 wait complete. 2024-03-23T09:49:44.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.373+0000 7feaf8ae0640 1 Processor -- start 2024-03-23T09:49:44.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.373+0000 7feaf8ae0640 1 -- start start 2024-03-23T09:49:44.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.373+0000 7feaf8ae0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feae0005180 0x7feaf40cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.373+0000 7feaf8ae0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf40b41e0 0x7feaf4156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.373+0000 7feaf8ae0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf40d1a30 0x7feaf40d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.373+0000 7feaf8ae0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf41f2000 con 0x7feae0005180 2024-03-23T09:49:44.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.373+0000 7feaf8ae0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf40d23f0 con 0x7feaf40b41e0 2024-03-23T09:49:44.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.373+0000 7feaf8ae0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf40b2dd0 con 0x7feaf40d1a30 2024-03-23T09:49:44.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaf259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feae0005180 0x7feaf40cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaf259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feae0005180 0x7feaf40cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44122/0 (socket says 172.21.15.47:44122) 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaf259c640 1 -- 172.21.15.47:0/2286836832 learned_addr learned my addr 172.21.15.47:0/2286836832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaebfff640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf40b41e0 0x7feaf4156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaf2d9d640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf40d1a30 0x7feaf40d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaf259c640 1 -- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf40d1a30 msgr2=0x7feaf40d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaf259c640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf40d1a30 0x7feaf40d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaf259c640 1 -- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf40b41e0 msgr2=0x7feaf4156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaf259c640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf40b41e0 0x7feaf4156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaf259c640 1 -- 172.21.15.47:0/2286836832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf40cc990 con 0x7feae0005180 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaebfff640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf40b41e0 0x7feaf4156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaf2d9d640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf40d1a30 0x7feaf40d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaf259c640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feae0005180 0x7feaf40cc450 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7feae400b7f0 tx=0x7feae400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaeaffd640 1 -- 172.21.15.47:0/2286836832 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4010830 con 0x7feae0005180 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaeaffd640 1 -- 172.21.15.47:0/2286836832 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae4010990 con 0x7feae0005180 2024-03-23T09:49:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaf8ae0640 1 -- 172.21.15.47:0/2286836832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf40b5210 con 0x7feae0005180 2024-03-23T09:49:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaeaffd640 1 -- 172.21.15.47:0/2286836832 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4010d60 con 0x7feae0005180 2024-03-23T09:49:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.374+0000 7feaf8ae0640 1 -- 172.21.15.47:0/2286836832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf40b4c50 con 0x7feae0005180 2024-03-23T09:49:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.375+0000 7feaf8ae0640 1 -- 172.21.15.47:0/2286836832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf4072360 con 0x7feae0005180 2024-03-23T09:49:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.376+0000 7feaeaffd640 1 -- 172.21.15.47:0/2286836832 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7feae4014430 con 0x7feae0005180 2024-03-23T09:49:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.376+0000 7feaeaffd640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fead406ac00 0x7fead406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.376+0000 7feaeaffd640 1 -- 172.21.15.47:0/2286836832 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feae4089c30 con 0x7feae0005180 2024-03-23T09:49:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.376+0000 7feaebfff640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fead406ac00 0x7fead406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.377+0000 7feaebfff640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fead406ac00 0x7fead406d0c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7feadc000c00 tx=0x7feadc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.380+0000 7feaeaffd640 1 -- 172.21.15.47:0/2286836832 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7feae4054ea0 con 0x7feae0005180 2024-03-23T09:49:44.462 INFO:tasks.ceph.mds.a.smithi047.stderr:2024-03-23T09:49:44.462+0000 7f499437d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:49:44.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:44.599+0000 7feaf8ae0640 1 -- 172.21.15.47:0/2286836832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7feaf40cc990 con 0x7feae0005180 2024-03-23T09:49:45.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.266+0000 7feaeaffd640 1 -- 172.21.15.47:0/2286836832 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) v1 ==== 122+0+0 (secure 0 0 0) 0x7feaf40cc990 con 0x7feae0005180 2024-03-23T09:49:45.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.268+0000 7feaf8ae0640 1 -- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fead406ac00 msgr2=0x7fead406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:45.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.268+0000 7feaf8ae0640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fead406ac00 0x7fead406d0c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7feadc000c00 tx=0x7feadc009040 comp rx=0 tx=0).stop 2024-03-23T09:49:45.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.268+0000 7feaf8ae0640 1 -- 172.21.15.47:0/2286836832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feae0005180 msgr2=0x7feaf40cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:45.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.268+0000 7feaf8ae0640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feae0005180 0x7feaf40cc450 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7feae400b7f0 tx=0x7feae400bcb0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.268+0000 7feaf8ae0640 1 -- 172.21.15.47:0/2286836832 shutdown_connections 2024-03-23T09:49:45.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.268+0000 7feaf8ae0640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fead406ac00 0x7fead406d0c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.268+0000 7feaf8ae0640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf40d1a30 0x7feaf40d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.268+0000 7feaf8ae0640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf40b41e0 0x7feaf4156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.268+0000 7feaf8ae0640 1 --2- 172.21.15.47:0/2286836832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feae0005180 0x7feaf40cc450 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.268+0000 7feaf8ae0640 1 -- 172.21.15.47:0/2286836832 >> 172.21.15.47:0/2286836832 conn(0x7feaf406b160 msgr2=0x7feaf415a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:45.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.268+0000 7feaf8ae0640 1 -- 172.21.15.47:0/2286836832 shutdown_connections 2024-03-23T09:49:45.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.268+0000 7feaf8ae0640 1 -- 172.21.15.47:0/2286836832 wait complete. 2024-03-23T09:49:45.282 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-23T09:49:45.282 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:49:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.389+0000 7febda341640 1 Processor -- start 2024-03-23T09:49:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.389+0000 7febda341640 1 -- start start 2024-03-23T09:49:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.389+0000 7febda341640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7febd40b1b00 0x7febd406b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.389+0000 7febda341640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febd40b0020 0x7febd4098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.389+0000 7febda341640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7febd406bd40 0x7febd4098f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.389+0000 7febda341640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd406c2c0 con 0x7febd40b1b00 2024-03-23T09:49:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.389+0000 7febda341640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd40caed0 con 0x7febd406bd40 2024-03-23T09:49:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.389+0000 7febda341640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd40cb1b0 con 0x7febd40b0020 2024-03-23T09:49:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.390+0000 7febd3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7febd40b1b00 0x7febd406b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.390+0000 7febd3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7febd40b1b00 0x7febd406b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44178/0 (socket says 172.21.15.47:44178) 2024-03-23T09:49:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.390+0000 7febd3fff640 1 -- 172.21.15.47:0/984861163 learned_addr learned my addr 172.21.15.47:0/984861163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.390+0000 7febd88de640 1 --2- 172.21.15.47:0/984861163 >> v2:172.21.15.175:3300/0 conn(0x7febd406bd40 0x7febd4098f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.390+0000 7febd37fe640 1 --2- 172.21.15.47:0/984861163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febd40b0020 0x7febd4098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.390+0000 7febd3fff640 1 -- 172.21.15.47:0/984861163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febd40b0020 msgr2=0x7febd4098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.390+0000 7febd3fff640 1 --2- 172.21.15.47:0/984861163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febd40b0020 0x7febd4098a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.390+0000 7febd3fff640 1 -- 172.21.15.47:0/984861163 >> v2:172.21.15.175:3300/0 conn(0x7febd406bd40 msgr2=0x7febd4098f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.390+0000 7febd3fff640 1 --2- 172.21.15.47:0/984861163 >> v2:172.21.15.175:3300/0 conn(0x7febd406bd40 0x7febd4098f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.390+0000 7febd3fff640 1 -- 172.21.15.47:0/984861163 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd40cb490 con 0x7febd40b1b00 2024-03-23T09:49:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.391+0000 7febd3fff640 1 --2- 172.21.15.47:0/984861163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd40b1b00 0x7febd406b800 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7febd4071ae0 tx=0x7febc402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73325a9490e9ead5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.391+0000 7febd2ffd640 1 -- 172.21.15.47:0/984861163 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4031870 con 0x7febd40b1b00 2024-03-23T09:49:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.391+0000 7febd2ffd640 1 -- 172.21.15.47:0/984861163 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc40319d0 con 0x7febd40b1b00 2024-03-23T09:49:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.391+0000 7febd2ffd640 1 -- 172.21.15.47:0/984861163 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4031d60 con 0x7febd40b1b00 2024-03-23T09:49:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.391+0000 7febda341640 1 -- 172.21.15.47:0/984861163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd40b1b00 msgr2=0x7febd406b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.391+0000 7febda341640 1 --2- 172.21.15.47:0/984861163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd40b1b00 0x7febd406b800 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7febd4071ae0 tx=0x7febc402b910 comp rx=0 tx=0).stop 2024-03-23T09:49:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.391+0000 7febda341640 1 -- 172.21.15.47:0/984861163 shutdown_connections 2024-03-23T09:49:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.391+0000 7febda341640 1 --2- 172.21.15.47:0/984861163 >> v2:172.21.15.175:3300/0 conn(0x7febd406bd40 0x7febd4098f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.391+0000 7febda341640 1 --2- 172.21.15.47:0/984861163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febd40b0020 0x7febd4098a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.391+0000 7febda341640 1 --2- 172.21.15.47:0/984861163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd40b1b00 0x7febd406b800 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.391+0000 7febda341640 1 -- 172.21.15.47:0/984861163 >> 172.21.15.47:0/984861163 conn(0x7febd40b21d0 msgr2=0x7febd40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.391+0000 7febda341640 1 -- 172.21.15.47:0/984861163 shutdown_connections 2024-03-23T09:49:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.392+0000 7febda341640 1 -- 172.21.15.47:0/984861163 wait complete. 2024-03-23T09:49:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.393+0000 7febda341640 1 Processor -- start 2024-03-23T09:49:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.393+0000 7febda341640 1 -- start start 2024-03-23T09:49:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febda341640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febd406bd40 0x7febd40ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febda341640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd40b0020 0x7febd40c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febda341640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febd40b1b00 0x7febd40c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febda341640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd41f33f0 con 0x7febd40b0020 2024-03-23T09:49:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febda341640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40c95b0 con 0x7febd40b1b00 2024-03-23T09:49:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febda341640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd40c9830 con 0x7febd406bd40 2024-03-23T09:49:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd40b0020 0x7febd40c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd40b0020 0x7febd40c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44182/0 (socket says 172.21.15.47:44182) 2024-03-23T09:49:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd37fe640 1 -- 172.21.15.47:0/1257442392 learned_addr learned my addr 172.21.15.47:0/1257442392 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd3fff640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febd406bd40 0x7febd40ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd88de640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febd40b1b00 0x7febd40c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd37fe640 1 -- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febd406bd40 msgr2=0x7febd40ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd37fe640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febd406bd40 0x7febd40ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd37fe640 1 -- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febd40b1b00 msgr2=0x7febd40c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd37fe640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febd40b1b00 0x7febd40c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd37fe640 1 -- 172.21.15.47:0/1257442392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc40094d0 con 0x7febd40b0020 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd3fff640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febd406bd40 0x7febd40ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd88de640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febd40b1b00 0x7febd40c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd37fe640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd40b0020 0x7febd40c67d0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7febc800e8b0 tx=0x7febc800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd0ff9640 1 -- 172.21.15.47:0/1257442392 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc800c940 con 0x7febd40b0020 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd0ff9640 1 -- 172.21.15.47:0/1257442392 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc800caa0 con 0x7febd40b0020 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febda341640 1 -- 172.21.15.47:0/1257442392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd40c9b10 con 0x7febd40b0020 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febd0ff9640 1 -- 172.21.15.47:0/1257442392 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc800ce30 con 0x7febd40b0020 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.394+0000 7febda341640 1 -- 172.21.15.47:0/1257442392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd40ca0d0 con 0x7febd40b0020 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.395+0000 7febda341640 1 -- 172.21.15.47:0/1257442392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd40cab80 con 0x7febd40b0020 2024-03-23T09:49:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.396+0000 7febd0ff9640 1 -- 172.21.15.47:0/1257442392 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7febc8013470 con 0x7febd40b0020 2024-03-23T09:49:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.396+0000 7febd0ff9640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7febc006ac00 0x7febc006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.396+0000 7febd0ff9640 1 -- 172.21.15.47:0/1257442392 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febc8089510 con 0x7febd40b0020 2024-03-23T09:49:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.396+0000 7febd3fff640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7febc006ac00 0x7febc006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.397+0000 7febd3fff640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7febc006ac00 0x7febc006d0c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7febc4036000 tx=0x7febc403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.400+0000 7febd0ff9640 1 -- 172.21.15.47:0/1257442392 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7febc80546e0 con 0x7febd40b0020 2024-03-23T09:49:45.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.614+0000 7febda341640 1 -- 172.21.15.47:0/1257442392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febd40672f0 con 0x7febd40b0020 2024-03-23T09:49:45.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.615+0000 7febd0ff9640 1 -- 172.21.15.47:0/1257442392 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4739 (secure 0 0 0) 0x7febc8057d00 con 0x7febd40b0020 2024-03-23T09:49:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:45.615 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/249416109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":249416109},{"type":"v1","addr":"172.21.15.175:6837","nonce":249416109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3603846321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3603846321},{"type":"v1","addr":"172.21.15.175:6835","nonce":3603846321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1921094794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1921094794},{"type":"v1","addr":"172.21.15.47:6834","nonce":1921094794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:49:43.247119+0000","modified":"2024-03-23T09:49:45.258459+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8484":{"gid":8484,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/2388298413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2388298413},{"type":"v1","addr":"172.21.15.47:6835","nonce":2388298413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-23T09:49:45.615 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 65 2024-03-23T09:49:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.617+0000 7febda341640 1 -- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7febc006ac00 msgr2=0x7febc006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.617+0000 7febda341640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7febc006ac00 0x7febc006d0c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7febc4036000 tx=0x7febc403c040 comp rx=0 tx=0).stop 2024-03-23T09:49:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.617+0000 7febda341640 1 -- 172.21.15.47:0/1257442392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd40b0020 msgr2=0x7febd40c67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.617+0000 7febda341640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd40b0020 0x7febd40c67d0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7febc800e8b0 tx=0x7febc800ed70 comp rx=0 tx=0).stop 2024-03-23T09:49:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.617+0000 7febda341640 1 -- 172.21.15.47:0/1257442392 shutdown_connections 2024-03-23T09:49:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.617+0000 7febda341640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7febc006ac00 0x7febc006d0c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.617+0000 7febda341640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febd40b1b00 0x7febd40c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.617+0000 7febda341640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd40b0020 0x7febd40c67d0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.617+0000 7febda341640 1 --2- 172.21.15.47:0/1257442392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febd406bd40 0x7febd40ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.617+0000 7febda341640 1 -- 172.21.15.47:0/1257442392 >> 172.21.15.47:0/1257442392 conn(0x7febd40b21d0 msgr2=0x7febd40cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.617+0000 7febda341640 1 -- 172.21.15.47:0/1257442392 shutdown_connections 2024-03-23T09:49:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.617+0000 7febda341640 1 -- 172.21.15.47:0/1257442392 wait complete. 2024-03-23T09:49:45.628 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:49:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.728+0000 7f40ef366640 1 Processor -- start 2024-03-23T09:49:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.728+0000 7f40ef366640 1 -- start start 2024-03-23T09:49:45.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.729+0000 7f40ef366640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40e806f210 0x7f40e806f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:45.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.729+0000 7f40ef366640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e80cf2f0 con 0x7f40e80ce780 2024-03-23T09:49:45.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.729+0000 7f40ef366640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e806fb30 con 0x7f40e80b28e0 2024-03-23T09:49:45.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.729+0000 7f40ef366640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e80d01c0 con 0x7f40e806f210 2024-03-23T09:49:45.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.729+0000 7f40ed102640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40e806f210 0x7f40e806f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:45.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.729+0000 7f40ec901640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f40e80ce780 0x7f40e80cebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47484/0 (socket says 172.21.15.47:47484) 2024-03-23T09:49:45.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.729+0000 7f40ec901640 1 -- 172.21.15.47:0/4171533625 learned_addr learned my addr 172.21.15.47:0/4171533625 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:45.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.730+0000 7f40ed102640 1 -- 172.21.15.47:0/4171533625 >> v1:172.21.15.175:6789/0 conn(0x7f40e80b28e0 legacy=0x7f40e80b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:45.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.730+0000 7f40ed102640 1 -- 172.21.15.47:0/4171533625 >> v1:172.21.15.47:6789/0 conn(0x7f40e80ce780 legacy=0x7f40e80cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:45.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.730+0000 7f40ed102640 1 -- 172.21.15.47:0/4171533625 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e80cf540 con 0x7f40e806f210 2024-03-23T09:49:45.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.730+0000 7f40e7fff640 1 -- 172.21.15.47:0/4171533625 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538069886 0 0) 0x7f40e80cf2f0 con 0x7f40e80ce780 2024-03-23T09:49:45.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.730+0000 7f40ed102640 1 --2- 172.21.15.47:0/4171533625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e806f210 0x7f40e806f5f0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f40e00097a0 tx=0x7f40e002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9b9929505be609 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:45.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.731+0000 7f40e7fff640 1 -- 172.21.15.47:0/4171533625 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e002e070 con 0x7f40e806f210 2024-03-23T09:49:45.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.731+0000 7f40e7fff640 1 -- 172.21.15.47:0/4171533625 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e00315a0 con 0x7f40e806f210 2024-03-23T09:49:45.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.731+0000 7f40e7fff640 1 -- 172.21.15.47:0/4171533625 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0031930 con 0x7f40e806f210 2024-03-23T09:49:45.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.731+0000 7f40ef366640 1 -- 172.21.15.47:0/4171533625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e806f210 msgr2=0x7f40e806f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:45.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.731+0000 7f40ef366640 1 --2- 172.21.15.47:0/4171533625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e806f210 0x7f40e806f5f0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f40e00097a0 tx=0x7f40e002b840 comp rx=0 tx=0).stop 2024-03-23T09:49:45.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.731+0000 7f40ef366640 1 -- 172.21.15.47:0/4171533625 shutdown_connections 2024-03-23T09:49:45.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.731+0000 7f40ef366640 1 --2- 172.21.15.47:0/4171533625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e806f210 0x7f40e806f5f0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.731+0000 7f40ef366640 1 -- 172.21.15.47:0/4171533625 >> 172.21.15.47:0/4171533625 conn(0x7f40e8071400 msgr2=0x7f40e8071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:45.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.731+0000 7f40ef366640 1 -- 172.21.15.47:0/4171533625 shutdown_connections 2024-03-23T09:49:45.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.731+0000 7f40ef366640 1 -- 172.21.15.47:0/4171533625 wait complete. 2024-03-23T09:49:45.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.732+0000 7f40ef366640 1 Processor -- start 2024-03-23T09:49:45.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.732+0000 7f40ef366640 1 -- start start 2024-03-23T09:49:45.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ef366640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e806f210 0x7f40e81ef870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:45.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ef366640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e80b28e0 0x7f40e8146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:45.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ef366640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40e80ce780 0x7f40e8144c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:45.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ef366640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80cac60 con 0x7f40e806f210 2024-03-23T09:49:45.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ef366640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80caf40 con 0x7f40e80b28e0 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ef366640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e81474d0 con 0x7f40e80ce780 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ed102640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e806f210 0x7f40e81ef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ed102640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e806f210 0x7f40e81ef870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44196/0 (socket says 172.21.15.47:44196) 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ed102640 1 -- 172.21.15.47:0/1521019395 learned_addr learned my addr 172.21.15.47:0/1521019395 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ed102640 1 -- 172.21.15.47:0/1521019395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40e80ce780 msgr2=0x7f40e8144c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ed102640 1 --2- 172.21.15.47:0/1521019395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40e80ce780 0x7f40e8144c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ed102640 1 -- 172.21.15.47:0/1521019395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e80b28e0 msgr2=0x7f40e8146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ed102640 1 --2- 172.21.15.47:0/1521019395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e80b28e0 0x7f40e8146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.733+0000 7f40ed102640 1 -- 172.21.15.47:0/1521019395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e0009400 con 0x7f40e806f210 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.734+0000 7f40ed102640 1 --2- 172.21.15.47:0/1521019395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e806f210 0x7f40e81ef870 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f40e0034040 tx=0x7f40e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.734+0000 7f40e5ffb640 1 -- 172.21.15.47:0/1521019395 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0004230 con 0x7f40e806f210 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.734+0000 7f40e5ffb640 1 -- 172.21.15.47:0/1521019395 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e0004390 con 0x7f40e806f210 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.734+0000 7f40ef366640 1 -- 172.21.15.47:0/1521019395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e81477b0 con 0x7f40e806f210 2024-03-23T09:49:45.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.734+0000 7f40e5ffb640 1 -- 172.21.15.47:0/1521019395 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0032880 con 0x7f40e806f210 2024-03-23T09:49:45.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.734+0000 7f40ef366640 1 -- 172.21.15.47:0/1521019395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e81efdb0 con 0x7f40e806f210 2024-03-23T09:49:45.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.735+0000 7f40ef366640 1 -- 172.21.15.47:0/1521019395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e80672f0 con 0x7f40e806f210 2024-03-23T09:49:45.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.735+0000 7f40e5ffb640 1 -- 172.21.15.47:0/1521019395 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f40e00329e0 con 0x7f40e806f210 2024-03-23T09:49:45.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.736+0000 7f40e5ffb640 1 --2- 172.21.15.47:0/1521019395 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f40d406ac30 0x7f40d406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.736+0000 7f40e5ffb640 1 -- 172.21.15.47:0/1521019395 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40e00ab560 con 0x7f40e806f210 2024-03-23T09:49:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.736+0000 7f40ec901640 1 --2- 172.21.15.47:0/1521019395 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f40d406ac30 0x7f40d406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.737+0000 7f40ec901640 1 --2- 172.21.15.47:0/1521019395 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f40d406ac30 0x7f40d406d0f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f40dc007fd0 tx=0x7f40dc007960 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:45.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.740+0000 7f40e5ffb640 1 -- 172.21.15.47:0/1521019395 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f40e0076860 con 0x7f40e806f210 2024-03-23T09:49:45.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.921+0000 7f40ef366640 1 -- 172.21.15.47:0/1521019395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40e80672f0 con 0x7f40e806f210 2024-03-23T09:49:45.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.922+0000 7f40e5ffb640 1 -- 172.21.15.47:0/1521019395 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16043 (secure 0 0 0) 0x7f40e0079e80 con 0x7f40e806f210 2024-03-23T09:49:45.922 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:45.923 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":101,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:49:44.249270+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:49:41.158056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-23T09:49:42.210388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:49:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.924+0000 7f40ef366640 1 -- 172.21.15.47:0/1521019395 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f40d406ac30 msgr2=0x7f40d406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.924+0000 7f40ef366640 1 --2- 172.21.15.47:0/1521019395 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f40d406ac30 0x7f40d406d0f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f40dc007fd0 tx=0x7f40dc007960 comp rx=0 tx=0).stop 2024-03-23T09:49:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.924+0000 7f40ef366640 1 -- 172.21.15.47:0/1521019395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e806f210 msgr2=0x7f40e81ef870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.924+0000 7f40ef366640 1 --2- 172.21.15.47:0/1521019395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e806f210 0x7f40e81ef870 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f40e0034040 tx=0x7f40e00046c0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.924+0000 7f40ef366640 1 -- 172.21.15.47:0/1521019395 shutdown_connections 2024-03-23T09:49:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.924+0000 7f40ef366640 1 --2- 172.21.15.47:0/1521019395 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f40d406ac30 0x7f40d406d0f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.924+0000 7f40ef366640 1 --2- 172.21.15.47:0/1521019395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40e80ce780 0x7f40e8144c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.924+0000 7f40ef366640 1 --2- 172.21.15.47:0/1521019395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e80b28e0 0x7f40e8146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.924+0000 7f40ef366640 1 --2- 172.21.15.47:0/1521019395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e806f210 0x7f40e81ef870 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.924+0000 7f40ef366640 1 -- 172.21.15.47:0/1521019395 >> 172.21.15.47:0/1521019395 conn(0x7f40e8071400 msgr2=0x7f40e806a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.924+0000 7f40ef366640 1 -- 172.21.15.47:0/1521019395 shutdown_connections 2024-03-23T09:49:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:45.924+0000 7f40ef366640 1 -- 172.21.15.47:0/1521019395 wait complete. 2024-03-23T09:49:45.936 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T09:49:45.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:49:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.036+0000 7ff189d9f640 1 Processor -- start 2024-03-23T09:49:46.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.036+0000 7ff189d9f640 1 -- start start 2024-03-23T09:49:46.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.037+0000 7ff189d9f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff18415ab50 0x7ff1840cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.037+0000 7ff189d9f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1840cb6f0 0x7ff1840af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.037+0000 7ff189d9f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1840cbf50 con 0x7ff1840cd090 2024-03-23T09:49:46.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.037+0000 7ff189d9f640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1840cc230 con 0x7ff18415ab50 2024-03-23T09:49:46.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.037+0000 7ff189d9f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840cc510 con 0x7ff1840cb6f0 2024-03-23T09:49:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.037+0000 7ff1837fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff1840cd090 0x7ff1840cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47490/0 (socket says 172.21.15.47:47490) 2024-03-23T09:49:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.037+0000 7ff1837fe640 1 -- 172.21.15.47:0/2776193589 learned_addr learned my addr 172.21.15.47:0/2776193589 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587319370 0 0) 0x7ff1840cbf50 con 0x7ff1840cd090 2024-03-23T09:49:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff182ffd640 1 --2- 172.21.15.47:0/2776193589 >> v2:172.21.15.175:3300/0 conn(0x7ff18415ab50 0x7ff1840cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff183fff640 1 --2- 172.21.15.47:0/2776193589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1840cb6f0 0x7ff1840af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff16c003520 con 0x7ff1840cd090 2024-03-23T09:49:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1266223470 0 0) 0x7ff16c003520 con 0x7ff1840cd090 2024-03-23T09:49:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1840cb6f0 msgr2=0x7ff1840af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 --2- 172.21.15.47:0/2776193589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1840cb6f0 0x7ff1840af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 >> v2:172.21.15.175:3300/0 conn(0x7ff18415ab50 msgr2=0x7ff1840cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 --2- 172.21.15.47:0/2776193589 >> v2:172.21.15.175:3300/0 conn(0x7ff18415ab50 0x7ff1840cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1840cc7f0 con 0x7ff1840cd090 2024-03-23T09:49:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7ff174001670 con 0x7ff1840cd090 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff182ffd640 1 --2- 172.21.15.47:0/2776193589 >> v2:172.21.15.175:3300/0 conn(0x7ff18415ab50 0x7ff1840cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 >> v1:172.21.15.47:6789/0 conn(0x7ff1840cd090 legacy=0x7ff1840cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 --2- 172.21.15.47:0/2776193589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff16c003970 0x7ff16c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 --2- 172.21.15.47:0/2776193589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff16c0047c0 0x7ff16c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 --2- 172.21.15.47:0/2776193589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff16c005180 0x7ff16c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840cc230 con 0x7ff16c005180 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840cc510 con 0x7ff16c003970 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c00bfa0 con 0x7ff16c0047c0 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff183fff640 1 --2- 172.21.15.47:0/2776193589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1840cb6f0 0x7ff1840af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff183fff640 1 --2- 172.21.15.47:0/2776193589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff16c005180 0x7ff16c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff1837fe640 1 --2- 172.21.15.47:0/2776193589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff16c003970 0x7ff16c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff182ffd640 1 --2- 172.21.15.47:0/2776193589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff16c0047c0 0x7ff16c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff183fff640 1 -- 172.21.15.47:0/2776193589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff16c0047c0 msgr2=0x7ff16c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff183fff640 1 --2- 172.21.15.47:0/2776193589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff16c0047c0 0x7ff16c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff183fff640 1 -- 172.21.15.47:0/2776193589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff16c003970 msgr2=0x7ff16c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff183fff640 1 --2- 172.21.15.47:0/2776193589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff16c003970 0x7ff16c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.038+0000 7ff183fff640 1 -- 172.21.15.47:0/2776193589 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c003520 con 0x7ff16c005180 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.039+0000 7ff183fff640 1 --2- 172.21.15.47:0/2776193589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff16c005180 0x7ff16c00b980 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009f00 tx=0x7ff16c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75e20836bdbdc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.039+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c02cc70 con 0x7ff16c005180 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.039+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c02cdd0 con 0x7ff16c005180 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.039+0000 7ff1827fc640 1 -- 172.21.15.47:0/2776193589 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c003bb0 con 0x7ff16c005180 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.040+0000 7ff189d9f640 1 -- 172.21.15.47:0/2776193589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff16c005180 msgr2=0x7ff16c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.040+0000 7ff189d9f640 1 --2- 172.21.15.47:0/2776193589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff16c005180 0x7ff16c00b980 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009f00 tx=0x7ff16c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.040+0000 7ff183fff640 1 -- 172.21.15.47:0/2776193589 reap_dead start 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.040+0000 7ff189d9f640 1 -- 172.21.15.47:0/2776193589 shutdown_connections 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.040+0000 7ff189d9f640 1 -- 172.21.15.47:0/2776193589 >> 172.21.15.47:0/2776193589 conn(0x7ff1840d06a0 msgr2=0x7ff1840d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.040+0000 7ff189d9f640 1 -- 172.21.15.47:0/2776193589 shutdown_connections 2024-03-23T09:49:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.040+0000 7ff189d9f640 1 -- 172.21.15.47:0/2776193589 wait complete. 2024-03-23T09:49:46.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.041+0000 7ff189d9f640 1 Processor -- start 2024-03-23T09:49:46.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.041+0000 7ff189d9f640 1 -- start start 2024-03-23T09:49:46.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff189d9f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff18414a100 0x7ff18415b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff189d9f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1840ae510 0x7ff1841595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff189d9f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff184159b20 0x7ff1840ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff189d9f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18406adf0 con 0x7ff1840ae510 2024-03-23T09:49:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff189d9f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18415b810 con 0x7ff184159b20 2024-03-23T09:49:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff189d9f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff184159fb0 con 0x7ff18414a100 2024-03-23T09:49:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff182ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1840ae510 0x7ff1841595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff182ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1840ae510 0x7ff1841595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44200/0 (socket says 172.21.15.47:44200) 2024-03-23T09:49:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff182ffd640 1 -- 172.21.15.47:0/2726680110 learned_addr learned my addr 172.21.15.47:0/2726680110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff183fff640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff184159b20 0x7ff1840ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff1837fe640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff18414a100 0x7ff18415b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff182ffd640 1 -- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff18414a100 msgr2=0x7ff18415b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff182ffd640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff18414a100 0x7ff18415b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff182ffd640 1 -- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff184159b20 msgr2=0x7ff1840ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff182ffd640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff184159b20 0x7ff1840ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff182ffd640 1 -- 172.21.15.47:0/2726680110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18406b620 con 0x7ff1840ae510 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff1837fe640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff18414a100 0x7ff18415b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff183fff640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff184159b20 0x7ff1840ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.042+0000 7ff182ffd640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1840ae510 0x7ff1841595e0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7ff178011700 tx=0x7ff178011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.043+0000 7ff16bfff640 1 -- 172.21.15.47:0/2726680110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17800c070 con 0x7ff1840ae510 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.043+0000 7ff16bfff640 1 -- 172.21.15.47:0/2726680110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff178004220 con 0x7ff1840ae510 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.043+0000 7ff189d9f640 1 -- 172.21.15.47:0/2726680110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1840ae990 con 0x7ff1840ae510 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.043+0000 7ff16bfff640 1 -- 172.21.15.47:0/2726680110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17800f7c0 con 0x7ff1840ae510 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.043+0000 7ff189d9f640 1 -- 172.21.15.47:0/2726680110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18406afd0 con 0x7ff1840ae510 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.044+0000 7ff189d9f640 1 -- 172.21.15.47:0/2726680110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18406b7b0 con 0x7ff1840ae510 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.044+0000 7ff16bfff640 1 -- 172.21.15.47:0/2726680110 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff17800f920 con 0x7ff1840ae510 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.044+0000 7ff16bfff640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff17006ac00 0x7ff17006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.044+0000 7ff16bfff640 1 -- 172.21.15.47:0/2726680110 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff178089370 con 0x7ff1840ae510 2024-03-23T09:49:46.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.044+0000 7ff1837fe640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff17006ac00 0x7ff17006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.045+0000 7ff1837fe640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff17006ac00 0x7ff17006d0c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff174001cc0 tx=0x7ff174009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:46.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.048+0000 7ff16bfff640 1 -- 172.21.15.47:0/2726680110 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff1780545e0 con 0x7ff1840ae510 2024-03-23T09:49:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.226+0000 7ff189d9f640 1 -- 172.21.15.47:0/2726680110 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1840af9f0 con 0x7ff17006ac00 2024-03-23T09:49:46.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.234+0000 7ff16bfff640 1 -- 172.21.15.47:0/2726680110 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278936 (secure 0 0 0) 0x7ff174007d10 con 0x7ff17006ac00 2024-03-23T09:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:46.239 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:49:46.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.237+0000 7ff189d9f640 1 -- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff17006ac00 msgr2=0x7ff17006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.237+0000 7ff189d9f640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff17006ac00 0x7ff17006d0c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff174001cc0 tx=0x7ff174009040 comp rx=0 tx=0).stop 2024-03-23T09:49:46.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.237+0000 7ff189d9f640 1 -- 172.21.15.47:0/2726680110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1840ae510 msgr2=0x7ff1841595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.237+0000 7ff189d9f640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1840ae510 0x7ff1841595e0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7ff178011700 tx=0x7ff178011bc0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.237+0000 7ff189d9f640 1 -- 172.21.15.47:0/2726680110 shutdown_connections 2024-03-23T09:49:46.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.237+0000 7ff189d9f640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff17006ac00 0x7ff17006d0c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.237+0000 7ff189d9f640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff184159b20 0x7ff1840ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.237+0000 7ff189d9f640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1840ae510 0x7ff1841595e0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.237+0000 7ff189d9f640 1 --2- 172.21.15.47:0/2726680110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff18414a100 0x7ff18415b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.237+0000 7ff189d9f640 1 -- 172.21.15.47:0/2726680110 >> 172.21.15.47:0/2726680110 conn(0x7ff1840d06a0 msgr2=0x7ff184098a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:46.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.237+0000 7ff189d9f640 1 -- 172.21.15.47:0/2726680110 shutdown_connections 2024-03-23T09:49:46.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.237+0000 7ff189d9f640 1 -- 172.21.15.47:0/2726680110 wait complete. 2024-03-23T09:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-03-23T09:49:44.470415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259576,"kb_used_data":10624,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754715144,"statfs":{"total":773094113280,"available":772828307456,"internally_reserved":0,"allocated":10878976,"data_stored":5328994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001078"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870278+0000","last_change":"2024-03-23T09:49:41.870278+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870278+0000","last_undegraded":"2024-03-23T09:49:41.870278+0000","last_fullsized":"2024-03-23T09:49:41.870278+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:25:47.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870832+0000","last_change":"2024-03-23T09:49:41.870832+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870832+0000","last_undegraded":"2024-03-23T09:49:41.870832+0000","last_fullsized":"2024-03-23T09:49:41.870832+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870647+0000","last_change":"2024-03-23T09:49:41.870647+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870647+0000","last_undegraded":"2024-03-23T09:49:41.870647+0000","last_fullsized":"2024-03-23T09:49:41.870647+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:56:58.937373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870392+0000","last_change":"2024-03-23T09:49:41.870392+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870392+0000","last_undegraded":"2024-03-23T09:49:41.870392+0000","last_fullsized":"2024-03-23T09:49:41.870392+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870487+0000","last_change":"2024-03-23T09:49:41.870487+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870487+0000","last_undegraded":"2024-03-23T09:49:41.870487+0000","last_fullsized":"2024-03-23T09:49:41.870487+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:09:41.124566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869903+0000","last_change":"2024-03-23T09:49:41.869903+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869903+0000","last_undegraded":"2024-03-23T09:49:41.869903+0000","last_fullsized":"2024-03-23T09:49:41.869903+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:04:23.374836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872359+0000","last_change":"2024-03-23T09:49:41.872359+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872359+0000","last_undegraded":"2024-03-23T09:49:41.872359+0000","last_fullsized":"2024-03-23T09:49:41.872359+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871434+0000","last_change":"2024-03-23T09:49:41.871434+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871434+0000","last_undegraded":"2024-03-23T09:49:41.871434+0000","last_fullsized":"2024-03-23T09:49:41.871434+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:29:10.221541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870644+0000","last_change":"2024-03-23T09:49:41.870644+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870644+0000","last_undegraded":"2024-03-23T09:49:41.870644+0000","last_fullsized":"2024-03-23T09:49:41.870644+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:39.896584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.868045+0000","last_change":"2024-03-23T09:49:41.868045+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.868045+0000","last_undegraded":"2024-03-23T09:49:41.868045+0000","last_fullsized":"2024-03-23T09:49:41.868045+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:21:22.073815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871230+0000","last_change":"2024-03-23T09:49:41.871230+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871230+0000","last_undegraded":"2024-03-23T09:49:41.871230+0000","last_fullsized":"2024-03-23T09:49:41.871230+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869789+0000","last_change":"2024-03-23T09:49:41.869789+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869789+0000","last_undegraded":"2024-03-23T09:49:41.869789+0000","last_fullsized":"2024-03-23T09:49:41.869789+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.873527+0000","last_change":"2024-03-23T09:49:41.873527+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.873527+0000","last_undegraded":"2024-03-23T09:49:41.873527+0000","last_fullsized":"2024-03-23T09:49:41.873527+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:29.138334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.873607+0000","last_change":"2024-03-23T09:49:41.873607+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.873607+0000","last_undegraded":"2024-03-23T09:49:41.873607+0000","last_fullsized":"2024-03-23T09:49:41.873607+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:22:32.641742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869776+0000","last_change":"2024-03-23T09:49:41.869776+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869776+0000","last_undegraded":"2024-03-23T09:49:41.869776+0000","last_fullsized":"2024-03-23T09:49:41.869776+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871016+0000","last_change":"2024-03-23T09:49:41.871016+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871016+0000","last_undegraded":"2024-03-23T09:49:41.871016+0000","last_fullsized":"2024-03-23T09:49:41.871016+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:24:49.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869713+0000","last_change":"2024-03-23T09:49:41.869713+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869713+0000","last_undegraded":"2024-03-23T09:49:41.869713+0000","last_fullsized":"2024-03-23T09:49:41.869713+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872429+0000","last_change":"2024-03-23T09:49:41.872429+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872429+0000","last_undegraded":"2024-03-23T09:49:41.872429+0000","last_fullsized":"2024-03-23T09:49:41.872429+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:25:47.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869932+0000","last_change":"2024-03-23T09:49:41.869932+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869932+0000","last_undegraded":"2024-03-23T09:49:41.869932+0000","last_fullsized":"2024-03-23T09:49:41.869932+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:14:31.482221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871303+0000","last_change":"2024-03-23T09:49:41.871303+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871303+0000","last_undegraded":"2024-03-23T09:49:41.871303+0000","last_fullsized":"2024-03-23T09:49:41.871303+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:21:22.073815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.874136+0000","last_change":"2024-03-23T09:49:41.874136+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.874136+0000","last_undegraded":"2024-03-23T09:49:41.874136+0000","last_fullsized":"2024-03-23T09:49:41.874136+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:24:49.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.874193+0000","last_change":"2024-03-23T09:49:41.874193+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.874193+0000","last_undegraded":"2024-03-23T09:49:41.874193+0000","last_fullsized":"2024-03-23T09:49:41.874193+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:09:35.927315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869882+0000","last_change":"2024-03-23T09:49:41.869882+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869882+0000","last_undegraded":"2024-03-23T09:49:41.869882+0000","last_fullsized":"2024-03-23T09:49:41.869882+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:09.171978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.874150+0000","last_change":"2024-03-23T09:49:41.874150+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.874150+0000","last_undegraded":"2024-03-23T09:49:41.874150+0000","last_fullsized":"2024-03-23T09:49:41.874150+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:39.896584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870470+0000","last_change":"2024-03-23T09:49:41.870470+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870470+0000","last_undegraded":"2024-03-23T09:49:41.870470+0000","last_fullsized":"2024-03-23T09:49:41.870470+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:15:46.255074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869958+0000","last_change":"2024-03-23T09:49:41.869958+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869958+0000","last_undegraded":"2024-03-23T09:49:41.869958+0000","last_fullsized":"2024-03-23T09:49:41.869958+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872465+0000","last_change":"2024-03-23T09:49:41.872465+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872465+0000","last_undegraded":"2024-03-23T09:49:41.872465+0000","last_fullsized":"2024-03-23T09:49:41.872465+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:04:23.374836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869975+0000","last_change":"2024-03-23T09:49:41.869975+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869975+0000","last_undegraded":"2024-03-23T09:49:41.869975+0000","last_fullsized":"2024-03-23T09:49:41.869975+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:33:50.023224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872851+0000","last_change":"2024-03-23T09:49:41.872851+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872851+0000","last_undegraded":"2024-03-23T09:49:41.872851+0000","last_fullsized":"2024-03-23T09:49:41.872851+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:33:50.023224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.868124+0000","last_change":"2024-03-23T09:49:41.868124+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.868124+0000","last_undegraded":"2024-03-23T09:49:41.868124+0000","last_fullsized":"2024-03-23T09:49:41.868124+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:58:35.947175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.873525+0000","last_change":"2024-03-23T09:49:41.873525+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.873525+0000","last_undegraded":"2024-03-23T09:49:41.873525+0000","last_fullsized":"2024-03-23T09:49:41.873525+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:41:54.983525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870759+0000","last_change":"2024-03-23T09:49:41.870759+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870759+0000","last_undegraded":"2024-03-23T09:49:41.870759+0000","last_fullsized":"2024-03-23T09:49:41.870759+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":244,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-03-23T09:49:39.852581+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:49:39.852581+0000","last_peered":"2024-03-23T09:49:39.852581+0000","last_clean":"2024-03-23T09:49:39.852581+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:49:39.852581+0000","last_undegraded":"2024-03-23T09:49:39.852581+0000","last_fullsized":"2024-03-23T09:49:39.852581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1806336,"data_stored":1109055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55600000000000005,"15min":0.51100000000000001},"min":{"1min":0.374,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.68000000000000005,"5min":0.76800000000000002,"15min":0.77500000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.53200000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28199999999999997},"max":{"1min":0.71799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57699999999999996,"15min":0.53000000000000003},"min":{"1min":0.41299999999999998,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.76300000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.503},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.30199999999999999},"max":{"1min":0.747,"5min":0.81000000000000005,"15min":0.81299999999999994},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57399999999999995,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58999999999999997,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73299999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55100000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57299999999999995,"15min":0.54000000000000004},"min":{"1min":0.47199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.73199999999999998,"5min":0.84699999999999998,"15min":0.86199999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.59699999999999998,"15min":0.57999999999999996},"min":{"1min":0.503,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.82699999999999996,"15min":0.83699999999999997},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.30399999999999999,"15min":0.29599999999999999},"max":{"1min":0.67500000000000004,"5min":0.76300000000000001,"15min":0.78200000000000003},"last":0.629},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.48999999999999999,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.29299999999999998},"max":{"1min":0.67500000000000004,"5min":0.754,"15min":0.80100000000000005},"last":0.45100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.5,"15min":0.51400000000000001},"min":{"1min":0.35599999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66200000000000003,"5min":0.80100000000000005,"15min":0.81499999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52600000000000002,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.64300000000000002,"5min":0.78000000000000003,"15min":0.84999999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53500000000000003,"15min":0.56999999999999995},"min":{"1min":0.36199999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71499999999999997,"5min":0.75700000000000001,"15min":0.874},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54900000000000004,"15min":0.57299999999999995},"min":{"1min":0.39600000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65400000000000003,"5min":0.79000000000000004,"15min":0.90200000000000002},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":1208320,"data_stored":518068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.52200000000000002},"min":{"1min":0.441,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.81000000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.45200000000000001,"5min":0.37,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.51100000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58999999999999997,"15min":0.53000000000000003},"min":{"1min":0.43099999999999999,"5min":0.315,"15min":0.311},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.44},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59699999999999998,"15min":0.55100000000000005},"min":{"1min":0.495,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42699999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.36299999999999999,"15min":0.29699999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.81599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.56299999999999994},"min":{"1min":0.435,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47299999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.57299999999999995},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.378},"max":{"1min":0.73299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57899999999999996,"15min":0.54500000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":0.79000000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46000000000000002,"15min":0.49199999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.62,"5min":0.69799999999999995,"15min":0.72499999999999998},"last":0.313},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.499,"15min":0.53400000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.66800000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.33200000000000002}]},{"osd":5,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49199999999999999,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.34499999999999997,"15min":0.316},"max":{"1min":0.61399999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52500000000000002,"15min":0.54800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70399999999999996,"5min":0.76200000000000001,"15min":0.77700000000000002},"last":0.371}]},{"osd":7,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55400000000000005,"15min":0.57699999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70099999999999996,"5min":0.78400000000000003,"15min":0.86199999999999999},"last":0.499}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54700000000000004,"15min":0.502},"min":{"1min":0.28799999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.70899999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.34899999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.75800000000000001},"last":0.47699999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.50600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71799999999999997,"5min":0.746,"15min":0.746},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.39200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57199999999999995}]},{"osd":2,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65500000000000003,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56000000000000005,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60299999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.52600000000000002},"min":{"1min":0.41199999999999998,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.74099999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.54600000000000004},"min":{"1min":0.42299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69299999999999995,"5min":0.71199999999999997,"15min":0.77400000000000002},"last":0.66400000000000003}]},{"osd":4,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45000000000000001,"15min":0.46500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73699999999999999},"last":0.627},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47799999999999998,"15min":0.50800000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68600000000000005,"5min":0.72499999999999998,"15min":0.93600000000000005},"last":0.71399999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.749},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48399999999999999,"15min":0.505},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66800000000000004,"5min":0.71799999999999997,"15min":0.72399999999999998},"last":0.68400000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52900000000000003,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.64600000000000002,"5min":0.76200000000000001,"15min":0.78200000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.76500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.52800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71499999999999997,"5min":0.77100000000000002,"15min":0.84599999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73599999999999999,"5min":0.76500000000000001,"15min":0.78600000000000003},"last":0.61799999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52300000000000002,"15min":0.49199999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66800000000000004,"5min":0.72699999999999998,"15min":0.72999999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.34100000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.77400000000000002},"last":0.40699999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59499999999999997,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.40200000000000002,"15min":0.39400000000000002},"max":{"1min":0.69899999999999995,"5min":0.77900000000000003,"15min":0.80200000000000005},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59999999999999998,"15min":0.55800000000000005},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.68500000000000005,"5min":0.79900000000000004,"15min":0.82199999999999995},"last":0.49099999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68400000000000005,"5min":0.80900000000000005,"15min":1.0369999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55500000000000005,"15min":0.52900000000000003},"min":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.374},"max":{"1min":0.72599999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.505}]},{"osd":5,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44,"15min":0.46000000000000002},"min":{"1min":0.315,"5min":0.309,"15min":0.26500000000000001},"max":{"1min":0.65800000000000003,"5min":0.73099999999999998,"15min":0.75800000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45900000000000002,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.253},"max":{"1min":0.59199999999999997,"5min":0.76100000000000001,"15min":1.008},"last":0.51800000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.315,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64100000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48899999999999999,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67700000000000005,"5min":0.79200000000000004,"15min":0.82399999999999995},"last":0.58599999999999997}]},{"osd":7,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.501,"15min":0.53700000000000003},"min":{"1min":0.33500000000000002,"5min":0.312,"15min":0.30099999999999999},"max":{"1min":0.67300000000000004,"5min":0.71799999999999997,"15min":1.0069999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.36499999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69099999999999995,"5min":0.77700000000000002,"15min":0.93400000000000005},"last":0.63200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36396,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335444,"statfs":{"total":96636764160,"available":96599494656,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35999999999999999,"15min":0.32700000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.183},"max":{"1min":0.54800000000000004,"5min":0.67800000000000005,"15min":1.45},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36599999999999999,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54400000000000004,"5min":0.621,"15min":1.4099999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35299999999999998,"15min":0.32500000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46600000000000003,"5min":0.54300000000000004,"15min":1.323},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39900000000000002,"15min":0.36299999999999999},"min":{"1min":0.309,"5min":0.22800000000000001,"15min":0.218},"max":{"1min":0.58299999999999996,"5min":0.67800000000000005,"15min":0.94599999999999995},"last":0.34300000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.437,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.251},"max":{"1min":0.60699999999999998,"5min":0.68999999999999995,"15min":1.1619999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46000000000000002,"15min":0.42599999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":0.70899999999999996,"15min":1.2989999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.80600000000000005,"15min":1.337},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75900000000000001,"5min":0.83899999999999997,"15min":2.758},"last":0.57499999999999996}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.874,"15min":1.276},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.60699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.88700000000000001,"15min":1.895},"last":0.56000000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.40699999999999997},"max":{"1min":0.84199999999999997,"5min":0.872,"15min":5.4539999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82099999999999995,"5min":0.93600000000000005,"15min":2.8159999999999998},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.501,"5min":0.44600000000000001,"15min":0.41299999999999998},"max":{"1min":0.85699999999999998,"5min":0.89800000000000002,"15min":3.7400000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.443,"5min":0.443,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":5.4900000000000002},"last":0.60699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34999999999999998,"15min":0.34599999999999997},"min":{"1min":0.249,"5min":0.191,"15min":0.191},"max":{"1min":0.49099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.379},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.379},"min":{"1min":0.247,"5min":0.245,"15min":0.22},"max":{"1min":0.58199999999999996,"5min":0.64400000000000002,"15min":0.872},"last":0.36499999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.371,"15min":0.36899999999999999},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.20999999999999999},"max":{"1min":0.49399999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.30299999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.746},"last":0.40000000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41899999999999998,"15min":0.41199999999999998},"min":{"1min":0.32300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.54200000000000004,"5min":0.65200000000000002,"15min":0.67100000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39900000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.248},"max":{"1min":0.56299999999999994,"5min":0.66700000000000004,"15min":0.72499999999999998},"last":0.33300000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.62,"15min":0.629},"min":{"1min":0.47599999999999998,"5min":0.43099999999999999,"15min":0.38500000000000001},"max":{"1min":0.80100000000000005,"5min":0.87,"15min":1.216},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.61699999999999999},"min":{"1min":0.503,"5min":0.41099999999999998,"15min":0.40300000000000002},"max":{"1min":0.80000000000000004,"5min":0.89600000000000002,"15min":0.99299999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.90400000000000003,"15min":0.95699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.63900000000000001,"15min":0.63200000000000001},"min":{"1min":0.47899999999999998,"5min":0.42099999999999999,"15min":0.40300000000000002},"max":{"1min":0.81499999999999995,"5min":0.93700000000000006,"15min":1.101},"last":0.58799999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65300000000000002,"15min":0.63600000000000001},"min":{"1min":0.46400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83199999999999996,"5min":0.89100000000000001,"15min":0.97299999999999998},"last":0.622},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64800000000000002,"15min":0.64500000000000002},"min":{"1min":0.48799999999999999,"5min":0.45800000000000002,"15min":0.436},"max":{"1min":0.78600000000000003,"5min":0.89500000000000002,"15min":1.0780000000000001},"last":0.64500000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.54100000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80000000000000004,"5min":0.88500000000000001,"15min":1.155},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.41599999999999998},"max":{"1min":0.81399999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.34200000000000003},"min":{"1min":0.22500000000000001,"5min":0.214,"15min":0.19400000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.64400000000000002},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.44700000000000001},"min":{"1min":0.25800000000000001,"5min":0.218,"15min":0.20699999999999999},"max":{"1min":0.56499999999999995,"5min":0.71999999999999997,"15min":2.7069999999999999},"last":0.314}]},{"osd":2,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.36699999999999999,"15min":0.35099999999999998},"min":{"1min":0.22800000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57299999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.221},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.38500000000000001},"min":{"1min":0.28799999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.53300000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34599999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.437,"15min":0.41899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.76900000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40799999999999997,"15min":0.47299999999999998},"min":{"1min":0.27300000000000002,"5min":0.25600000000000001,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.73899999999999999,"15min":2.75},"last":0.36299999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60599999999999998,"15min":0.59299999999999997},"min":{"1min":0.44900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.95899999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83299999999999996,"5min":1.002,"15min":1.002},"last":0.45100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.0900000000000001,"15min":2.778},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.438,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88800000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.50700000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.68700000000000006},"min":{"1min":0.46700000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86299999999999999,"5min":1.026,"15min":2.8210000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.51400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93300000000000005,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.438}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84999999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65600000000000003,"15min":0.72399999999999998},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91400000000000003,"5min":0.91600000000000004,"15min":2.7989999999999999},"last":0.46500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1802240,"data_stored":1108436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"min":{"1min":0.254,"5min":0.192,"15min":0.184},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.375,"15min":0.35799999999999998},"min":{"1min":0.28199999999999997,"5min":0.23300000000000001,"15min":0.217},"max":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.22900000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.374,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.25,"15min":0.25},"max":{"1min":0.56799999999999995,"5min":0.63,"15min":0.63},"last":0.246},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.377},"min":{"1min":0.26500000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.59399999999999997},"last":0.28699999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.38700000000000001},"min":{"1min":0.27100000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.67600000000000005},"last":0.252},{"interface":"front","average":{"1min":0.496,"5min":0.44700000000000001,"15min":0.432},"min":{"1min":0.35899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.35899999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60599999999999998,"15min":0.59099999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.89100000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61699999999999999,"15min":0.59799999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91200000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.90000000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.59499999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.95999999999999996},"last":0.56799999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65000000000000002,"15min":0.629},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.93600000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63}]},{"osd":7,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64400000000000002,"15min":0.61299999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.878,"15min":0.93600000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63600000000000001,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.46500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:49:46.253 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:49:46.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.389+0000 7f6bc4651640 1 Processor -- start 2024-03-23T09:49:46.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.389+0000 7f6bc4651640 1 -- start start 2024-03-23T09:49:46.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc4651640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bbc0b8350 0x7f6bbc0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc4651640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6bbc0cdc40 0x7f6bbc0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc4651640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bbc0ce090 con 0x7f6bbc15ca90 2024-03-23T09:49:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc4651640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bbc0d11e0 con 0x7f6bbc0b8350 2024-03-23T09:49:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc4651640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bbc160f60 con 0x7f6bbc0cdc40 2024-03-23T09:49:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc1bec640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bbc0b8350 0x7f6bbc0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc1bec640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bbc0b8350 0x7f6bbc0cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44206/0 (socket says 172.21.15.47:44206) 2024-03-23T09:49:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc1bec640 1 -- 172.21.15.47:0/1571468340 learned_addr learned my addr 172.21.15.47:0/1571468340 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc2bee640 1 --2- 172.21.15.47:0/1571468340 >> v2:172.21.15.175:3300/0 conn(0x7f6bbc0cdc40 0x7f6bbc0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc1bec640 1 -- 172.21.15.47:0/1571468340 >> v2:172.21.15.175:3300/0 conn(0x7f6bbc0cdc40 msgr2=0x7f6bbc0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc1bec640 1 --2- 172.21.15.47:0/1571468340 >> v2:172.21.15.175:3300/0 conn(0x7f6bbc0cdc40 0x7f6bbc0d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc1bec640 1 -- 172.21.15.47:0/1571468340 >> v1:172.21.15.175:6789/0 conn(0x7f6bbc15ca90 legacy=0x7f6bbc160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc1bec640 1 -- 172.21.15.47:0/1571468340 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc161240 con 0x7f6bbc0b8350 2024-03-23T09:49:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.390+0000 7f6bc2bee640 1 --2- 172.21.15.47:0/1571468340 >> v2:172.21.15.175:3300/0 conn(0x7f6bbc0cdc40 0x7f6bbc0d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.391+0000 7f6bc1bec640 1 --2- 172.21.15.47:0/1571468340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bbc0b8350 0x7f6bbc0cd700 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0b5930 tx=0x7f6bac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146db0e2093bf02 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.391+0000 7f6bc13eb640 1 -- 172.21.15.47:0/1571468340 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac031870 con 0x7f6bbc0b8350 2024-03-23T09:49:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.391+0000 7f6bc13eb640 1 -- 172.21.15.47:0/1571468340 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac0319d0 con 0x7f6bbc0b8350 2024-03-23T09:49:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.391+0000 7f6bc13eb640 1 -- 172.21.15.47:0/1571468340 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac031d60 con 0x7f6bbc0b8350 2024-03-23T09:49:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.392+0000 7f6bc4651640 1 -- 172.21.15.47:0/1571468340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bbc0b8350 msgr2=0x7f6bbc0cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.392+0000 7f6bc4651640 1 --2- 172.21.15.47:0/1571468340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bbc0b8350 0x7f6bbc0cd700 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0b5930 tx=0x7f6bac02b910 comp rx=0 tx=0).stop 2024-03-23T09:49:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.392+0000 7f6bc4651640 1 -- 172.21.15.47:0/1571468340 shutdown_connections 2024-03-23T09:49:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.392+0000 7f6bc4651640 1 --2- 172.21.15.47:0/1571468340 >> v2:172.21.15.175:3300/0 conn(0x7f6bbc0cdc40 0x7f6bbc0d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.392+0000 7f6bc4651640 1 --2- 172.21.15.47:0/1571468340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bbc0b8350 0x7f6bbc0cd700 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.392+0000 7f6bc4651640 1 -- 172.21.15.47:0/1571468340 >> 172.21.15.47:0/1571468340 conn(0x7f6bbc0659d0 msgr2=0x7f6bbc065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.392+0000 7f6bc4651640 1 -- 172.21.15.47:0/1571468340 shutdown_connections 2024-03-23T09:49:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.392+0000 7f6bc4651640 1 -- 172.21.15.47:0/1571468340 wait complete. 2024-03-23T09:49:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.393+0000 7f6bc4651640 1 Processor -- start 2024-03-23T09:49:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.393+0000 7f6bc4651640 1 -- start start 2024-03-23T09:49:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc4651640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc0b8350 0x7f6bbc1f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc4651640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bbc0cdc40 0x7f6bbc15d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc4651640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bbc15ca90 0x7f6bbc15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc4651640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc0ca100 con 0x7f6bbc15ca90 2024-03-23T09:49:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc4651640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc151bd0 con 0x7f6bbc0b8350 2024-03-23T09:49:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc4651640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc151eb0 con 0x7f6bbc0cdc40 2024-03-23T09:49:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc2bee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bbc15ca90 0x7f6bbc15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc2bee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bbc15ca90 0x7f6bbc15dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44216/0 (socket says 172.21.15.47:44216) 2024-03-23T09:49:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc2bee640 1 -- 172.21.15.47:0/3701716644 learned_addr learned my addr 172.21.15.47:0/3701716644 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc23ed640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc0b8350 0x7f6bbc1f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc1bec640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bbc0cdc40 0x7f6bbc15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc2bee640 1 -- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bbc0cdc40 msgr2=0x7f6bbc15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc2bee640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bbc0cdc40 0x7f6bbc15d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc2bee640 1 -- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc0b8350 msgr2=0x7f6bbc1f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc2bee640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc0b8350 0x7f6bbc1f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc2bee640 1 -- 172.21.15.47:0/3701716644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac0094d0 con 0x7f6bbc15ca90 2024-03-23T09:49:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc1bec640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bbc0cdc40 0x7f6bbc15d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc2bee640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bbc15ca90 0x7f6bbc15dcb0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4007ef0 tx=0x7f6bb400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bb2ffd640 1 -- 172.21.15.47:0/3701716644 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4003b90 con 0x7f6bbc15ca90 2024-03-23T09:49:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bb2ffd640 1 -- 172.21.15.47:0/3701716644 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb400f040 con 0x7f6bbc15ca90 2024-03-23T09:49:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc4651640 1 -- 172.21.15.47:0/3701716644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc152190 con 0x7f6bbc15ca90 2024-03-23T09:49:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bb2ffd640 1 -- 172.21.15.47:0/3701716644 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb400de80 con 0x7f6bbc15ca90 2024-03-23T09:49:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.394+0000 7f6bc4651640 1 -- 172.21.15.47:0/3701716644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc0b5140 con 0x7f6bbc15ca90 2024-03-23T09:49:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.395+0000 7f6bc4651640 1 -- 172.21.15.47:0/3701716644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc072360 con 0x7f6bbc15ca90 2024-03-23T09:49:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.396+0000 7f6bb2ffd640 1 -- 172.21.15.47:0/3701716644 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6bb4004030 con 0x7f6bbc15ca90 2024-03-23T09:49:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.396+0000 7f6bb2ffd640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6ba006ac00 0x7f6ba006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.396+0000 7f6bb2ffd640 1 -- 172.21.15.47:0/3701716644 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bb4089550 con 0x7f6bbc15ca90 2024-03-23T09:49:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.396+0000 7f6bc23ed640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6ba006ac00 0x7f6ba006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.397+0000 7f6bc23ed640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6ba006ac00 0x7f6ba006d0c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8000f60 tx=0x7f6bb8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.401+0000 7f6bb2ffd640 1 -- 172.21.15.47:0/3701716644 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6bb4054870 con 0x7f6bbc15ca90 2024-03-23T09:49:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.579+0000 7f6bc4651640 1 -- 172.21.15.47:0/3701716644 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6bbc0d0e70 con 0x7f6ba006ac00 2024-03-23T09:49:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.586+0000 7f6bb2ffd640 1 -- 172.21.15.47:0/3701716644 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278936 (secure 0 0 0) 0x7f6bb80014a0 con 0x7f6ba006ac00 2024-03-23T09:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:49:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.589+0000 7f6bc4651640 1 -- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6ba006ac00 msgr2=0x7f6ba006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.589+0000 7f6bc4651640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6ba006ac00 0x7f6ba006d0c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8000f60 tx=0x7f6bb8009040 comp rx=0 tx=0).stop 2024-03-23T09:49:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.589+0000 7f6bc4651640 1 -- 172.21.15.47:0/3701716644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bbc15ca90 msgr2=0x7f6bbc15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.589+0000 7f6bc4651640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bbc15ca90 0x7f6bbc15dcb0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4007ef0 tx=0x7f6bb400d4a0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.589+0000 7f6bc4651640 1 -- 172.21.15.47:0/3701716644 shutdown_connections 2024-03-23T09:49:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.589+0000 7f6bc4651640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6ba006ac00 0x7f6ba006d0c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.589+0000 7f6bc4651640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bbc15ca90 0x7f6bbc15dcb0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.589+0000 7f6bc4651640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bbc0cdc40 0x7f6bbc15d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.589+0000 7f6bc4651640 1 --2- 172.21.15.47:0/3701716644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc0b8350 0x7f6bbc1f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.589+0000 7f6bc4651640 1 -- 172.21.15.47:0/3701716644 >> 172.21.15.47:0/3701716644 conn(0x7f6bbc0659d0 msgr2=0x7f6bbc15f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.589+0000 7f6bc4651640 1 -- 172.21.15.47:0/3701716644 shutdown_connections 2024-03-23T09:49:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.589+0000 7f6bc4651640 1 -- 172.21.15.47:0/3701716644 wait complete. 2024-03-23T09:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-03-23T09:49:46.470979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259576,"kb_used_data":10624,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754715144,"statfs":{"total":773094113280,"available":772828307456,"internally_reserved":0,"allocated":10878976,"data_stored":5328994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.602065"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870278+0000","last_change":"2024-03-23T09:49:41.870278+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870278+0000","last_undegraded":"2024-03-23T09:49:41.870278+0000","last_fullsized":"2024-03-23T09:49:41.870278+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:25:47.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870832+0000","last_change":"2024-03-23T09:49:41.870832+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870832+0000","last_undegraded":"2024-03-23T09:49:41.870832+0000","last_fullsized":"2024-03-23T09:49:41.870832+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870647+0000","last_change":"2024-03-23T09:49:41.870647+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870647+0000","last_undegraded":"2024-03-23T09:49:41.870647+0000","last_fullsized":"2024-03-23T09:49:41.870647+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:56:58.937373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870392+0000","last_change":"2024-03-23T09:49:41.870392+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870392+0000","last_undegraded":"2024-03-23T09:49:41.870392+0000","last_fullsized":"2024-03-23T09:49:41.870392+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870487+0000","last_change":"2024-03-23T09:49:41.870487+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870487+0000","last_undegraded":"2024-03-23T09:49:41.870487+0000","last_fullsized":"2024-03-23T09:49:41.870487+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:09:41.124566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869903+0000","last_change":"2024-03-23T09:49:41.869903+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869903+0000","last_undegraded":"2024-03-23T09:49:41.869903+0000","last_fullsized":"2024-03-23T09:49:41.869903+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:04:23.374836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872359+0000","last_change":"2024-03-23T09:49:41.872359+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872359+0000","last_undegraded":"2024-03-23T09:49:41.872359+0000","last_fullsized":"2024-03-23T09:49:41.872359+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871434+0000","last_change":"2024-03-23T09:49:41.871434+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871434+0000","last_undegraded":"2024-03-23T09:49:41.871434+0000","last_fullsized":"2024-03-23T09:49:41.871434+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:29:10.221541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870644+0000","last_change":"2024-03-23T09:49:41.870644+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870644+0000","last_undegraded":"2024-03-23T09:49:41.870644+0000","last_fullsized":"2024-03-23T09:49:41.870644+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:39.896584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.868045+0000","last_change":"2024-03-23T09:49:41.868045+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.868045+0000","last_undegraded":"2024-03-23T09:49:41.868045+0000","last_fullsized":"2024-03-23T09:49:41.868045+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:21:22.073815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871230+0000","last_change":"2024-03-23T09:49:41.871230+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871230+0000","last_undegraded":"2024-03-23T09:49:41.871230+0000","last_fullsized":"2024-03-23T09:49:41.871230+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869789+0000","last_change":"2024-03-23T09:49:41.869789+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869789+0000","last_undegraded":"2024-03-23T09:49:41.869789+0000","last_fullsized":"2024-03-23T09:49:41.869789+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.873527+0000","last_change":"2024-03-23T09:49:41.873527+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.873527+0000","last_undegraded":"2024-03-23T09:49:41.873527+0000","last_fullsized":"2024-03-23T09:49:41.873527+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:29.138334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.873607+0000","last_change":"2024-03-23T09:49:41.873607+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.873607+0000","last_undegraded":"2024-03-23T09:49:41.873607+0000","last_fullsized":"2024-03-23T09:49:41.873607+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:22:32.641742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869776+0000","last_change":"2024-03-23T09:49:41.869776+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869776+0000","last_undegraded":"2024-03-23T09:49:41.869776+0000","last_fullsized":"2024-03-23T09:49:41.869776+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871016+0000","last_change":"2024-03-23T09:49:41.871016+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871016+0000","last_undegraded":"2024-03-23T09:49:41.871016+0000","last_fullsized":"2024-03-23T09:49:41.871016+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:24:49.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869713+0000","last_change":"2024-03-23T09:49:41.869713+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869713+0000","last_undegraded":"2024-03-23T09:49:41.869713+0000","last_fullsized":"2024-03-23T09:49:41.869713+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872429+0000","last_change":"2024-03-23T09:49:41.872429+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872429+0000","last_undegraded":"2024-03-23T09:49:41.872429+0000","last_fullsized":"2024-03-23T09:49:41.872429+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:25:47.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869932+0000","last_change":"2024-03-23T09:49:41.869932+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869932+0000","last_undegraded":"2024-03-23T09:49:41.869932+0000","last_fullsized":"2024-03-23T09:49:41.869932+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:14:31.482221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871303+0000","last_change":"2024-03-23T09:49:41.871303+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871303+0000","last_undegraded":"2024-03-23T09:49:41.871303+0000","last_fullsized":"2024-03-23T09:49:41.871303+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:21:22.073815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.874136+0000","last_change":"2024-03-23T09:49:41.874136+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.874136+0000","last_undegraded":"2024-03-23T09:49:41.874136+0000","last_fullsized":"2024-03-23T09:49:41.874136+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:24:49.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.874193+0000","last_change":"2024-03-23T09:49:41.874193+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.874193+0000","last_undegraded":"2024-03-23T09:49:41.874193+0000","last_fullsized":"2024-03-23T09:49:41.874193+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:09:35.927315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869882+0000","last_change":"2024-03-23T09:49:41.869882+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869882+0000","last_undegraded":"2024-03-23T09:49:41.869882+0000","last_fullsized":"2024-03-23T09:49:41.869882+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:09.171978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.874150+0000","last_change":"2024-03-23T09:49:41.874150+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.874150+0000","last_undegraded":"2024-03-23T09:49:41.874150+0000","last_fullsized":"2024-03-23T09:49:41.874150+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:39.896584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870470+0000","last_change":"2024-03-23T09:49:41.870470+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870470+0000","last_undegraded":"2024-03-23T09:49:41.870470+0000","last_fullsized":"2024-03-23T09:49:41.870470+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:15:46.255074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869958+0000","last_change":"2024-03-23T09:49:41.869958+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869958+0000","last_undegraded":"2024-03-23T09:49:41.869958+0000","last_fullsized":"2024-03-23T09:49:41.869958+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872465+0000","last_change":"2024-03-23T09:49:41.872465+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872465+0000","last_undegraded":"2024-03-23T09:49:41.872465+0000","last_fullsized":"2024-03-23T09:49:41.872465+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:04:23.374836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869975+0000","last_change":"2024-03-23T09:49:41.869975+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869975+0000","last_undegraded":"2024-03-23T09:49:41.869975+0000","last_fullsized":"2024-03-23T09:49:41.869975+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:33:50.023224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872851+0000","last_change":"2024-03-23T09:49:41.872851+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872851+0000","last_undegraded":"2024-03-23T09:49:41.872851+0000","last_fullsized":"2024-03-23T09:49:41.872851+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:33:50.023224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.868124+0000","last_change":"2024-03-23T09:49:41.868124+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.868124+0000","last_undegraded":"2024-03-23T09:49:41.868124+0000","last_fullsized":"2024-03-23T09:49:41.868124+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:58:35.947175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.873525+0000","last_change":"2024-03-23T09:49:41.873525+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.873525+0000","last_undegraded":"2024-03-23T09:49:41.873525+0000","last_fullsized":"2024-03-23T09:49:41.873525+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:41:54.983525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870759+0000","last_change":"2024-03-23T09:49:41.870759+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870759+0000","last_undegraded":"2024-03-23T09:49:41.870759+0000","last_fullsized":"2024-03-23T09:49:41.870759+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":244,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-03-23T09:49:39.852581+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:49:39.852581+0000","last_peered":"2024-03-23T09:49:39.852581+0000","last_clean":"2024-03-23T09:49:39.852581+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:49:39.852581+0000","last_undegraded":"2024-03-23T09:49:39.852581+0000","last_fullsized":"2024-03-23T09:49:39.852581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1806336,"data_stored":1109055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55600000000000005,"15min":0.51100000000000001},"min":{"1min":0.374,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.68000000000000005,"5min":0.76800000000000002,"15min":0.77500000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.53200000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28199999999999997},"max":{"1min":0.71799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57699999999999996,"15min":0.53000000000000003},"min":{"1min":0.41299999999999998,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.76300000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.503},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.30199999999999999},"max":{"1min":0.747,"5min":0.81000000000000005,"15min":0.81299999999999994},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57399999999999995,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58999999999999997,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73299999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55100000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57299999999999995,"15min":0.54000000000000004},"min":{"1min":0.47199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.73199999999999998,"5min":0.84699999999999998,"15min":0.86199999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.59699999999999998,"15min":0.57999999999999996},"min":{"1min":0.503,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.82699999999999996,"15min":0.83699999999999997},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.30399999999999999,"15min":0.29599999999999999},"max":{"1min":0.67500000000000004,"5min":0.76300000000000001,"15min":0.78200000000000003},"last":0.629},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.48999999999999999,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.29299999999999998},"max":{"1min":0.67500000000000004,"5min":0.754,"15min":0.80100000000000005},"last":0.45100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.5,"15min":0.51400000000000001},"min":{"1min":0.35599999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66200000000000003,"5min":0.80100000000000005,"15min":0.81499999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52600000000000002,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.64300000000000002,"5min":0.78000000000000003,"15min":0.84999999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53500000000000003,"15min":0.56999999999999995},"min":{"1min":0.36199999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71499999999999997,"5min":0.75700000000000001,"15min":0.874},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54900000000000004,"15min":0.57299999999999995},"min":{"1min":0.39600000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65400000000000003,"5min":0.79000000000000004,"15min":0.90200000000000002},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":1208320,"data_stored":518068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.52200000000000002},"min":{"1min":0.441,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.81000000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.45200000000000001,"5min":0.37,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.51100000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58999999999999997,"15min":0.53000000000000003},"min":{"1min":0.43099999999999999,"5min":0.315,"15min":0.311},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.44},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59699999999999998,"15min":0.55100000000000005},"min":{"1min":0.495,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42699999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.36299999999999999,"15min":0.29699999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.81599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.56299999999999994},"min":{"1min":0.435,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47299999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.57299999999999995},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.378},"max":{"1min":0.73299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57899999999999996,"15min":0.54500000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":0.79000000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46000000000000002,"15min":0.49199999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.62,"5min":0.69799999999999995,"15min":0.72499999999999998},"last":0.313},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.499,"15min":0.53400000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.66800000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.33200000000000002}]},{"osd":5,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49199999999999999,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.34499999999999997,"15min":0.316},"max":{"1min":0.61399999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52500000000000002,"15min":0.54800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70399999999999996,"5min":0.76200000000000001,"15min":0.77700000000000002},"last":0.371}]},{"osd":7,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55400000000000005,"15min":0.57699999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70099999999999996,"5min":0.78400000000000003,"15min":0.86199999999999999},"last":0.499}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54700000000000004,"15min":0.502},"min":{"1min":0.28799999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.70899999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.34899999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.75800000000000001},"last":0.47699999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.50600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71799999999999997,"5min":0.746,"15min":0.746},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.39200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57199999999999995}]},{"osd":2,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65500000000000003,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56000000000000005,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60299999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.52600000000000002},"min":{"1min":0.41199999999999998,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.74099999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.54600000000000004},"min":{"1min":0.42299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69299999999999995,"5min":0.71199999999999997,"15min":0.77400000000000002},"last":0.66400000000000003}]},{"osd":4,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45000000000000001,"15min":0.46500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73699999999999999},"last":0.627},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47799999999999998,"15min":0.50800000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68600000000000005,"5min":0.72499999999999998,"15min":0.93600000000000005},"last":0.71399999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.749},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48399999999999999,"15min":0.505},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66800000000000004,"5min":0.71799999999999997,"15min":0.72399999999999998},"last":0.68400000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52900000000000003,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.64600000000000002,"5min":0.76200000000000001,"15min":0.78200000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.76500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.52800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71499999999999997,"5min":0.77100000000000002,"15min":0.84599999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73599999999999999,"5min":0.76500000000000001,"15min":0.78600000000000003},"last":0.61799999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52300000000000002,"15min":0.49199999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66800000000000004,"5min":0.72699999999999998,"15min":0.72999999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.34100000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.77400000000000002},"last":0.40699999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59499999999999997,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.40200000000000002,"15min":0.39400000000000002},"max":{"1min":0.69899999999999995,"5min":0.77900000000000003,"15min":0.80200000000000005},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59999999999999998,"15min":0.55800000000000005},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.68500000000000005,"5min":0.79900000000000004,"15min":0.82199999999999995},"last":0.49099999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68400000000000005,"5min":0.80900000000000005,"15min":1.0369999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55500000000000005,"15min":0.52900000000000003},"min":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.374},"max":{"1min":0.72599999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.505}]},{"osd":5,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44,"15min":0.46000000000000002},"min":{"1min":0.315,"5min":0.309,"15min":0.26500000000000001},"max":{"1min":0.65800000000000003,"5min":0.73099999999999998,"15min":0.75800000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45900000000000002,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.253},"max":{"1min":0.59199999999999997,"5min":0.76100000000000001,"15min":1.008},"last":0.51800000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.315,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64100000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48899999999999999,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67700000000000005,"5min":0.79200000000000004,"15min":0.82399999999999995},"last":0.58599999999999997}]},{"osd":7,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.501,"15min":0.53700000000000003},"min":{"1min":0.33500000000000002,"5min":0.312,"15min":0.30099999999999999},"max":{"1min":0.67300000000000004,"5min":0.71799999999999997,"15min":1.0069999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.36499999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69099999999999995,"5min":0.77700000000000002,"15min":0.93400000000000005},"last":0.63200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36396,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335444,"statfs":{"total":96636764160,"available":96599494656,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35999999999999999,"15min":0.32700000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.183},"max":{"1min":0.54800000000000004,"5min":0.67800000000000005,"15min":1.45},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36599999999999999,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54400000000000004,"5min":0.621,"15min":1.4099999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35299999999999998,"15min":0.32500000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46600000000000003,"5min":0.54300000000000004,"15min":1.323},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39900000000000002,"15min":0.36299999999999999},"min":{"1min":0.309,"5min":0.22800000000000001,"15min":0.218},"max":{"1min":0.58299999999999996,"5min":0.67800000000000005,"15min":0.94599999999999995},"last":0.34300000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.437,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.251},"max":{"1min":0.60699999999999998,"5min":0.68999999999999995,"15min":1.1619999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46000000000000002,"15min":0.42599999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":0.70899999999999996,"15min":1.2989999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.80600000000000005,"15min":1.337},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75900000000000001,"5min":0.83899999999999997,"15min":2.758},"last":0.57499999999999996}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.874,"15min":1.276},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.60699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.88700000000000001,"15min":1.895},"last":0.56000000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.40699999999999997},"max":{"1min":0.84199999999999997,"5min":0.872,"15min":5.4539999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82099999999999995,"5min":0.93600000000000005,"15min":2.8159999999999998},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.501,"5min":0.44600000000000001,"15min":0.41299999999999998},"max":{"1min":0.85699999999999998,"5min":0.89800000000000002,"15min":3.7400000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.443,"5min":0.443,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":5.4900000000000002},"last":0.60699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34999999999999998,"15min":0.34599999999999997},"min":{"1min":0.249,"5min":0.191,"15min":0.191},"max":{"1min":0.49099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.379},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.379},"min":{"1min":0.247,"5min":0.245,"15min":0.22},"max":{"1min":0.58199999999999996,"5min":0.64400000000000002,"15min":0.872},"last":0.36499999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.371,"15min":0.36899999999999999},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.20999999999999999},"max":{"1min":0.49399999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.30299999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.746},"last":0.40000000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41899999999999998,"15min":0.41199999999999998},"min":{"1min":0.32300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.54200000000000004,"5min":0.65200000000000002,"15min":0.67100000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39900000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.248},"max":{"1min":0.56299999999999994,"5min":0.66700000000000004,"15min":0.72499999999999998},"last":0.33300000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.62,"15min":0.629},"min":{"1min":0.47599999999999998,"5min":0.43099999999999999,"15min":0.38500000000000001},"max":{"1min":0.80100000000000005,"5min":0.87,"15min":1.216},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.61699999999999999},"min":{"1min":0.503,"5min":0.41099999999999998,"15min":0.40300000000000002},"max":{"1min":0.80000000000000004,"5min":0.89600000000000002,"15min":0.99299999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.90400000000000003,"15min":0.95699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.63900000000000001,"15min":0.63200000000000001},"min":{"1min":0.47899999999999998,"5min":0.42099999999999999,"15min":0.40300000000000002},"max":{"1min":0.81499999999999995,"5min":0.93700000000000006,"15min":1.101},"last":0.58799999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65300000000000002,"15min":0.63600000000000001},"min":{"1min":0.46400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83199999999999996,"5min":0.89100000000000001,"15min":0.97299999999999998},"last":0.622},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64800000000000002,"15min":0.64500000000000002},"min":{"1min":0.48799999999999999,"5min":0.45800000000000002,"15min":0.436},"max":{"1min":0.78600000000000003,"5min":0.89500000000000002,"15min":1.0780000000000001},"last":0.64500000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.54100000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80000000000000004,"5min":0.88500000000000001,"15min":1.155},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.41599999999999998},"max":{"1min":0.81399999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.34200000000000003},"min":{"1min":0.22500000000000001,"5min":0.214,"15min":0.19400000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.64400000000000002},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.44700000000000001},"min":{"1min":0.25800000000000001,"5min":0.218,"15min":0.20699999999999999},"max":{"1min":0.56499999999999995,"5min":0.71999999999999997,"15min":2.7069999999999999},"last":0.314}]},{"osd":2,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.36699999999999999,"15min":0.35099999999999998},"min":{"1min":0.22800000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57299999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.221},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.38500000000000001},"min":{"1min":0.28799999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.53300000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34599999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.437,"15min":0.41899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.76900000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40799999999999997,"15min":0.47299999999999998},"min":{"1min":0.27300000000000002,"5min":0.25600000000000001,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.73899999999999999,"15min":2.75},"last":0.36299999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60599999999999998,"15min":0.59299999999999997},"min":{"1min":0.44900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.95899999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83299999999999996,"5min":1.002,"15min":1.002},"last":0.45100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.0900000000000001,"15min":2.778},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.438,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88800000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.50700000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.68700000000000006},"min":{"1min":0.46700000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86299999999999999,"5min":1.026,"15min":2.8210000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.51400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93300000000000005,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.438}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84999999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65600000000000003,"15min":0.72399999999999998},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91400000000000003,"5min":0.91600000000000004,"15min":2.7989999999999999},"last":0.46500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1802240,"data_stored":1108436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"min":{"1min":0.254,"5min":0.192,"15min":0.184},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.375,"15min":0.35799999999999998},"min":{"1min":0.28199999999999997,"5min":0.23300000000000001,"15min":0.217},"max":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.22900000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.374,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.25,"15min":0.25},"max":{"1min":0.56799999999999995,"5min":0.63,"15min":0.63},"last":0.246},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.377},"min":{"1min":0.26500000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.59399999999999997},"last":0.28699999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.38700000000000001},"min":{"1min":0.27100000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.67600000000000005},"last":0.252},{"interface":"front","average":{"1min":0.496,"5min":0.44700000000000001,"15min":0.432},"min":{"1min":0.35899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.35899999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60599999999999998,"15min":0.59099999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.89100000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61699999999999999,"15min":0.59799999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91200000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.90000000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.59499999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.95999999999999996},"last":0.56799999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65000000000000002,"15min":0.629},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.93600000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63}]},{"osd":7,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64400000000000002,"15min":0.61299999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.878,"15min":0.93600000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63600000000000001,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.46500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:49:46.604 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:49:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.740+0000 7fe268c81640 1 Processor -- start 2024-03-23T09:49:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.740+0000 7fe268c81640 1 -- start start 2024-03-23T09:49:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.741+0000 7fe268c81640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2640ccb80 0x7fe2640ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.741+0000 7fe268c81640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26415ae50 con 0x7fe2640cd7f0 2024-03-23T09:49:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.741+0000 7fe268c81640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2640cf010 con 0x7fe26415aa70 2024-03-23T09:49:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.741+0000 7fe268c81640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2640cf2f0 con 0x7fe2640ccb80 2024-03-23T09:49:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.741+0000 7fe261d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe2640cd7f0 0x7fe2640ce2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47506/0 (socket says 172.21.15.47:47506) 2024-03-23T09:49:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.741+0000 7fe261d9b640 1 -- 172.21.15.47:0/1013095474 learned_addr learned my addr 172.21.15.47:0/1013095474 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.741+0000 7fe262d9d640 1 --2- 172.21.15.47:0/1013095474 >> v2:172.21.15.47:3300/0 conn(0x7fe2640ccb80 0x7fe2640ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.742+0000 7fe262d9d640 1 -- 172.21.15.47:0/1013095474 >> v1:172.21.15.175:6789/0 conn(0x7fe26415aa70 legacy=0x7fe2640cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.742+0000 7fe262d9d640 1 -- 172.21.15.47:0/1013095474 >> v1:172.21.15.47:6789/0 conn(0x7fe2640cd7f0 legacy=0x7fe2640ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.742+0000 7fe26159a640 1 -- 172.21.15.47:0/1013095474 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3544457575 0 0) 0x7fe26415ae50 con 0x7fe2640cd7f0 2024-03-23T09:49:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.742+0000 7fe262d9d640 1 -- 172.21.15.47:0/1013095474 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2640cf5d0 con 0x7fe2640ccb80 2024-03-23T09:49:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.742+0000 7fe262d9d640 1 --2- 172.21.15.47:0/1013095474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2640ccb80 0x7fe2640ce9d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0097a0 tx=0x7fe25c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f67915cf37b71d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.742+0000 7fe26159a640 1 -- 172.21.15.47:0/1013095474 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c030610 con 0x7fe2640ccb80 2024-03-23T09:49:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.743+0000 7fe26159a640 1 -- 172.21.15.47:0/1013095474 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25c030770 con 0x7fe2640ccb80 2024-03-23T09:49:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.743+0000 7fe26159a640 1 -- 172.21.15.47:0/1013095474 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c030b00 con 0x7fe2640ccb80 2024-03-23T09:49:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.743+0000 7fe268c81640 1 -- 172.21.15.47:0/1013095474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2640ccb80 msgr2=0x7fe2640ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.743+0000 7fe268c81640 1 --2- 172.21.15.47:0/1013095474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2640ccb80 0x7fe2640ce9d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0097a0 tx=0x7fe25c02b960 comp rx=0 tx=0).stop 2024-03-23T09:49:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.743+0000 7fe268c81640 1 -- 172.21.15.47:0/1013095474 shutdown_connections 2024-03-23T09:49:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.743+0000 7fe268c81640 1 --2- 172.21.15.47:0/1013095474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2640ccb80 0x7fe2640ce9d0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.743+0000 7fe268c81640 1 -- 172.21.15.47:0/1013095474 >> 172.21.15.47:0/1013095474 conn(0x7fe2640d06f0 msgr2=0x7fe2640d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.743+0000 7fe268c81640 1 -- 172.21.15.47:0/1013095474 shutdown_connections 2024-03-23T09:49:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.743+0000 7fe268c81640 1 -- 172.21.15.47:0/1013095474 wait complete. 2024-03-23T09:49:46.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.744+0000 7fe268c81640 1 Processor -- start 2024-03-23T09:49:46.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.745+0000 7fe268c81640 1 -- start start 2024-03-23T09:49:46.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.745+0000 7fe268c81640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2640ccb80 0x7fe264163460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.745+0000 7fe268c81640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2640cd7f0 0x7fe2641f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.745+0000 7fe268c81640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe26415aa70 0x7fe2641f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.745+0000 7fe268c81640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26406b800 con 0x7fe26415aa70 2024-03-23T09:49:46.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.745+0000 7fe268c81640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26406bae0 con 0x7fe2640cd7f0 2024-03-23T09:49:46.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.745+0000 7fe268c81640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26406bdc0 con 0x7fe2640ccb80 2024-03-23T09:49:46.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.745+0000 7fe262d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe26415aa70 0x7fe2641f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.745+0000 7fe262d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe26415aa70 0x7fe2641f62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44234/0 (socket says 172.21.15.47:44234) 2024-03-23T09:49:46.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.745+0000 7fe262d9d640 1 -- 172.21.15.47:0/3676535121 learned_addr learned my addr 172.21.15.47:0/3676535121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe26259c640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2640ccb80 0x7fe264163460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe261d9b640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2640cd7f0 0x7fe2641f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe262d9d640 1 -- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2640ccb80 msgr2=0x7fe264163460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe262d9d640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2640ccb80 0x7fe264163460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe262d9d640 1 -- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2640cd7f0 msgr2=0x7fe2641f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe262d9d640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2640cd7f0 0x7fe2641f5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe262d9d640 1 -- 172.21.15.47:0/3676535121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c009400 con 0x7fe26415aa70 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe26259c640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2640ccb80 0x7fe264163460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe261d9b640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2640cd7f0 0x7fe2641f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe262d9d640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe26415aa70 0x7fe2641f62d0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0098c0 tx=0x7fe25c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe25affd640 1 -- 172.21.15.47:0/3676535121 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c039c10 con 0x7fe26415aa70 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe25affd640 1 -- 172.21.15.47:0/3676535121 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25c03f040 con 0x7fe26415aa70 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe268c81640 1 -- 172.21.15.47:0/3676535121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26406c0a0 con 0x7fe26415aa70 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe25affd640 1 -- 172.21.15.47:0/3676535121 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c006e10 con 0x7fe26415aa70 2024-03-23T09:49:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.746+0000 7fe268c81640 1 -- 172.21.15.47:0/3676535121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26406c660 con 0x7fe26415aa70 2024-03-23T09:49:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.747+0000 7fe268c81640 1 -- 172.21.15.47:0/3676535121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26406c880 con 0x7fe26415aa70 2024-03-23T09:49:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.748+0000 7fe25affd640 1 -- 172.21.15.47:0/3676535121 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe25c004030 con 0x7fe26415aa70 2024-03-23T09:49:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.748+0000 7fe25affd640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe24406ac00 0x7fe24406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.748+0000 7fe25affd640 1 -- 172.21.15.47:0/3676535121 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe25c0ab300 con 0x7fe26415aa70 2024-03-23T09:49:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.748+0000 7fe26259c640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe24406ac00 0x7fe24406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.749+0000 7fe26259c640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe24406ac00 0x7fe24406d0c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe254007e90 tx=0x7fe2540073a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:46.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.752+0000 7fe25affd640 1 -- 172.21.15.47:0/3676535121 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe25c076590 con 0x7fe26415aa70 2024-03-23T09:49:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.928+0000 7fe268c81640 1 -- 172.21.15.47:0/3676535121 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2641655a0 con 0x7fe24406ac00 2024-03-23T09:49:46.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.936+0000 7fe25affd640 1 -- 172.21.15.47:0/3676535121 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278936 (secure 0 0 0) 0x7fe25400cdb0 con 0x7fe24406ac00 2024-03-23T09:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:46.940 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:49:46.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.938+0000 7fe268c81640 1 -- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe24406ac00 msgr2=0x7fe24406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.938+0000 7fe268c81640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe24406ac00 0x7fe24406d0c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe254007e90 tx=0x7fe2540073a0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.938+0000 7fe268c81640 1 -- 172.21.15.47:0/3676535121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe26415aa70 msgr2=0x7fe2641f62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:46.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.938+0000 7fe268c81640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe26415aa70 0x7fe2641f62d0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0098c0 tx=0x7fe25c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.939+0000 7fe268c81640 1 -- 172.21.15.47:0/3676535121 shutdown_connections 2024-03-23T09:49:46.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.939+0000 7fe268c81640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe24406ac00 0x7fe24406d0c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.939+0000 7fe268c81640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe26415aa70 0x7fe2641f62d0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.939+0000 7fe268c81640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2640cd7f0 0x7fe2641f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.939+0000 7fe268c81640 1 --2- 172.21.15.47:0/3676535121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2640ccb80 0x7fe264163460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:46.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.939+0000 7fe268c81640 1 -- 172.21.15.47:0/3676535121 >> 172.21.15.47:0/3676535121 conn(0x7fe2640d06f0 msgr2=0x7fe264159900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:46.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.939+0000 7fe268c81640 1 -- 172.21.15.47:0/3676535121 shutdown_connections 2024-03-23T09:49:46.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:46.939+0000 7fe268c81640 1 -- 172.21.15.47:0/3676535121 wait complete. 2024-03-23T09:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-03-23T09:49:46.470979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259576,"kb_used_data":10624,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754715144,"statfs":{"total":773094113280,"available":772828307456,"internally_reserved":0,"allocated":10878976,"data_stored":5328994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.602065"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870278+0000","last_change":"2024-03-23T09:49:41.870278+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870278+0000","last_undegraded":"2024-03-23T09:49:41.870278+0000","last_fullsized":"2024-03-23T09:49:41.870278+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:25:47.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870832+0000","last_change":"2024-03-23T09:49:41.870832+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870832+0000","last_undegraded":"2024-03-23T09:49:41.870832+0000","last_fullsized":"2024-03-23T09:49:41.870832+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870647+0000","last_change":"2024-03-23T09:49:41.870647+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870647+0000","last_undegraded":"2024-03-23T09:49:41.870647+0000","last_fullsized":"2024-03-23T09:49:41.870647+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:56:58.937373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870392+0000","last_change":"2024-03-23T09:49:41.870392+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870392+0000","last_undegraded":"2024-03-23T09:49:41.870392+0000","last_fullsized":"2024-03-23T09:49:41.870392+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870487+0000","last_change":"2024-03-23T09:49:41.870487+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870487+0000","last_undegraded":"2024-03-23T09:49:41.870487+0000","last_fullsized":"2024-03-23T09:49:41.870487+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:09:41.124566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869903+0000","last_change":"2024-03-23T09:49:41.869903+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869903+0000","last_undegraded":"2024-03-23T09:49:41.869903+0000","last_fullsized":"2024-03-23T09:49:41.869903+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:04:23.374836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872359+0000","last_change":"2024-03-23T09:49:41.872359+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872359+0000","last_undegraded":"2024-03-23T09:49:41.872359+0000","last_fullsized":"2024-03-23T09:49:41.872359+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871434+0000","last_change":"2024-03-23T09:49:41.871434+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871434+0000","last_undegraded":"2024-03-23T09:49:41.871434+0000","last_fullsized":"2024-03-23T09:49:41.871434+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:29:10.221541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870644+0000","last_change":"2024-03-23T09:49:41.870644+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870644+0000","last_undegraded":"2024-03-23T09:49:41.870644+0000","last_fullsized":"2024-03-23T09:49:41.870644+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:39.896584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.868045+0000","last_change":"2024-03-23T09:49:41.868045+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.868045+0000","last_undegraded":"2024-03-23T09:49:41.868045+0000","last_fullsized":"2024-03-23T09:49:41.868045+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:21:22.073815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871230+0000","last_change":"2024-03-23T09:49:41.871230+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871230+0000","last_undegraded":"2024-03-23T09:49:41.871230+0000","last_fullsized":"2024-03-23T09:49:41.871230+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869789+0000","last_change":"2024-03-23T09:49:41.869789+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869789+0000","last_undegraded":"2024-03-23T09:49:41.869789+0000","last_fullsized":"2024-03-23T09:49:41.869789+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.873527+0000","last_change":"2024-03-23T09:49:41.873527+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.873527+0000","last_undegraded":"2024-03-23T09:49:41.873527+0000","last_fullsized":"2024-03-23T09:49:41.873527+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:29.138334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.873607+0000","last_change":"2024-03-23T09:49:41.873607+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.873607+0000","last_undegraded":"2024-03-23T09:49:41.873607+0000","last_fullsized":"2024-03-23T09:49:41.873607+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:22:32.641742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869776+0000","last_change":"2024-03-23T09:49:41.869776+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869776+0000","last_undegraded":"2024-03-23T09:49:41.869776+0000","last_fullsized":"2024-03-23T09:49:41.869776+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871016+0000","last_change":"2024-03-23T09:49:41.871016+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871016+0000","last_undegraded":"2024-03-23T09:49:41.871016+0000","last_fullsized":"2024-03-23T09:49:41.871016+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:24:49.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869713+0000","last_change":"2024-03-23T09:49:41.869713+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869713+0000","last_undegraded":"2024-03-23T09:49:41.869713+0000","last_fullsized":"2024-03-23T09:49:41.869713+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872429+0000","last_change":"2024-03-23T09:49:41.872429+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872429+0000","last_undegraded":"2024-03-23T09:49:41.872429+0000","last_fullsized":"2024-03-23T09:49:41.872429+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:25:47.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869932+0000","last_change":"2024-03-23T09:49:41.869932+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869932+0000","last_undegraded":"2024-03-23T09:49:41.869932+0000","last_fullsized":"2024-03-23T09:49:41.869932+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:14:31.482221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871303+0000","last_change":"2024-03-23T09:49:41.871303+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871303+0000","last_undegraded":"2024-03-23T09:49:41.871303+0000","last_fullsized":"2024-03-23T09:49:41.871303+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:21:22.073815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.874136+0000","last_change":"2024-03-23T09:49:41.874136+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.874136+0000","last_undegraded":"2024-03-23T09:49:41.874136+0000","last_fullsized":"2024-03-23T09:49:41.874136+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:24:49.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.874193+0000","last_change":"2024-03-23T09:49:41.874193+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.874193+0000","last_undegraded":"2024-03-23T09:49:41.874193+0000","last_fullsized":"2024-03-23T09:49:41.874193+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:09:35.927315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869882+0000","last_change":"2024-03-23T09:49:41.869882+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869882+0000","last_undegraded":"2024-03-23T09:49:41.869882+0000","last_fullsized":"2024-03-23T09:49:41.869882+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:09.171978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.874150+0000","last_change":"2024-03-23T09:49:41.874150+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.874150+0000","last_undegraded":"2024-03-23T09:49:41.874150+0000","last_fullsized":"2024-03-23T09:49:41.874150+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:39.896584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870470+0000","last_change":"2024-03-23T09:49:41.870470+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870470+0000","last_undegraded":"2024-03-23T09:49:41.870470+0000","last_fullsized":"2024-03-23T09:49:41.870470+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:15:46.255074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869958+0000","last_change":"2024-03-23T09:49:41.869958+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869958+0000","last_undegraded":"2024-03-23T09:49:41.869958+0000","last_fullsized":"2024-03-23T09:49:41.869958+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872465+0000","last_change":"2024-03-23T09:49:41.872465+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872465+0000","last_undegraded":"2024-03-23T09:49:41.872465+0000","last_fullsized":"2024-03-23T09:49:41.872465+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:04:23.374836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869975+0000","last_change":"2024-03-23T09:49:41.869975+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869975+0000","last_undegraded":"2024-03-23T09:49:41.869975+0000","last_fullsized":"2024-03-23T09:49:41.869975+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:33:50.023224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872851+0000","last_change":"2024-03-23T09:49:41.872851+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872851+0000","last_undegraded":"2024-03-23T09:49:41.872851+0000","last_fullsized":"2024-03-23T09:49:41.872851+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:33:50.023224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.868124+0000","last_change":"2024-03-23T09:49:41.868124+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.868124+0000","last_undegraded":"2024-03-23T09:49:41.868124+0000","last_fullsized":"2024-03-23T09:49:41.868124+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:58:35.947175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.873525+0000","last_change":"2024-03-23T09:49:41.873525+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.873525+0000","last_undegraded":"2024-03-23T09:49:41.873525+0000","last_fullsized":"2024-03-23T09:49:41.873525+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:41:54.983525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870759+0000","last_change":"2024-03-23T09:49:41.870759+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870759+0000","last_undegraded":"2024-03-23T09:49:41.870759+0000","last_fullsized":"2024-03-23T09:49:41.870759+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":244,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-03-23T09:49:39.852581+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:49:39.852581+0000","last_peered":"2024-03-23T09:49:39.852581+0000","last_clean":"2024-03-23T09:49:39.852581+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:49:39.852581+0000","last_undegraded":"2024-03-23T09:49:39.852581+0000","last_fullsized":"2024-03-23T09:49:39.852581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1806336,"data_stored":1109055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55600000000000005,"15min":0.51100000000000001},"min":{"1min":0.374,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.68000000000000005,"5min":0.76800000000000002,"15min":0.77500000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.53200000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28199999999999997},"max":{"1min":0.71799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57699999999999996,"15min":0.53000000000000003},"min":{"1min":0.41299999999999998,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.76300000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.503},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.30199999999999999},"max":{"1min":0.747,"5min":0.81000000000000005,"15min":0.81299999999999994},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57399999999999995,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58999999999999997,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73299999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55100000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57299999999999995,"15min":0.54000000000000004},"min":{"1min":0.47199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.73199999999999998,"5min":0.84699999999999998,"15min":0.86199999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.59699999999999998,"15min":0.57999999999999996},"min":{"1min":0.503,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.82699999999999996,"15min":0.83699999999999997},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.30399999999999999,"15min":0.29599999999999999},"max":{"1min":0.67500000000000004,"5min":0.76300000000000001,"15min":0.78200000000000003},"last":0.629},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.48999999999999999,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.29299999999999998},"max":{"1min":0.67500000000000004,"5min":0.754,"15min":0.80100000000000005},"last":0.45100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.5,"15min":0.51400000000000001},"min":{"1min":0.35599999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66200000000000003,"5min":0.80100000000000005,"15min":0.81499999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52600000000000002,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.64300000000000002,"5min":0.78000000000000003,"15min":0.84999999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53500000000000003,"15min":0.56999999999999995},"min":{"1min":0.36199999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71499999999999997,"5min":0.75700000000000001,"15min":0.874},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54900000000000004,"15min":0.57299999999999995},"min":{"1min":0.39600000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65400000000000003,"5min":0.79000000000000004,"15min":0.90200000000000002},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":1208320,"data_stored":518068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.52200000000000002},"min":{"1min":0.441,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.81000000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.45200000000000001,"5min":0.37,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.51100000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58999999999999997,"15min":0.53000000000000003},"min":{"1min":0.43099999999999999,"5min":0.315,"15min":0.311},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.44},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59699999999999998,"15min":0.55100000000000005},"min":{"1min":0.495,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42699999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.36299999999999999,"15min":0.29699999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.81599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.56299999999999994},"min":{"1min":0.435,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47299999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.57299999999999995},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.378},"max":{"1min":0.73299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57899999999999996,"15min":0.54500000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":0.79000000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46000000000000002,"15min":0.49199999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.62,"5min":0.69799999999999995,"15min":0.72499999999999998},"last":0.313},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.499,"15min":0.53400000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.66800000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.33200000000000002}]},{"osd":5,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49199999999999999,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.34499999999999997,"15min":0.316},"max":{"1min":0.61399999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52500000000000002,"15min":0.54800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70399999999999996,"5min":0.76200000000000001,"15min":0.77700000000000002},"last":0.371}]},{"osd":7,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55400000000000005,"15min":0.57699999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70099999999999996,"5min":0.78400000000000003,"15min":0.86199999999999999},"last":0.499}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54700000000000004,"15min":0.502},"min":{"1min":0.28799999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.70899999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.34899999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.75800000000000001},"last":0.47699999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.50600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71799999999999997,"5min":0.746,"15min":0.746},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.39200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57199999999999995}]},{"osd":2,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65500000000000003,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56000000000000005,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60299999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.52600000000000002},"min":{"1min":0.41199999999999998,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.74099999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.54600000000000004},"min":{"1min":0.42299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69299999999999995,"5min":0.71199999999999997,"15min":0.77400000000000002},"last":0.66400000000000003}]},{"osd":4,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45000000000000001,"15min":0.46500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73699999999999999},"last":0.627},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47799999999999998,"15min":0.50800000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68600000000000005,"5min":0.72499999999999998,"15min":0.93600000000000005},"last":0.71399999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.749},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48399999999999999,"15min":0.505},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66800000000000004,"5min":0.71799999999999997,"15min":0.72399999999999998},"last":0.68400000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52900000000000003,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.64600000000000002,"5min":0.76200000000000001,"15min":0.78200000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.76500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.52800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71499999999999997,"5min":0.77100000000000002,"15min":0.84599999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73599999999999999,"5min":0.76500000000000001,"15min":0.78600000000000003},"last":0.61799999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52300000000000002,"15min":0.49199999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66800000000000004,"5min":0.72699999999999998,"15min":0.72999999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.34100000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.77400000000000002},"last":0.40699999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59499999999999997,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.40200000000000002,"15min":0.39400000000000002},"max":{"1min":0.69899999999999995,"5min":0.77900000000000003,"15min":0.80200000000000005},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59999999999999998,"15min":0.55800000000000005},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.68500000000000005,"5min":0.79900000000000004,"15min":0.82199999999999995},"last":0.49099999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68400000000000005,"5min":0.80900000000000005,"15min":1.0369999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55500000000000005,"15min":0.52900000000000003},"min":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.374},"max":{"1min":0.72599999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.505}]},{"osd":5,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44,"15min":0.46000000000000002},"min":{"1min":0.315,"5min":0.309,"15min":0.26500000000000001},"max":{"1min":0.65800000000000003,"5min":0.73099999999999998,"15min":0.75800000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45900000000000002,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.253},"max":{"1min":0.59199999999999997,"5min":0.76100000000000001,"15min":1.008},"last":0.51800000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.315,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64100000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48899999999999999,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67700000000000005,"5min":0.79200000000000004,"15min":0.82399999999999995},"last":0.58599999999999997}]},{"osd":7,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.501,"15min":0.53700000000000003},"min":{"1min":0.33500000000000002,"5min":0.312,"15min":0.30099999999999999},"max":{"1min":0.67300000000000004,"5min":0.71799999999999997,"15min":1.0069999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.36499999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69099999999999995,"5min":0.77700000000000002,"15min":0.93400000000000005},"last":0.63200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36396,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335444,"statfs":{"total":96636764160,"available":96599494656,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35999999999999999,"15min":0.32700000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.183},"max":{"1min":0.54800000000000004,"5min":0.67800000000000005,"15min":1.45},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36599999999999999,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54400000000000004,"5min":0.621,"15min":1.4099999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35299999999999998,"15min":0.32500000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46600000000000003,"5min":0.54300000000000004,"15min":1.323},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39900000000000002,"15min":0.36299999999999999},"min":{"1min":0.309,"5min":0.22800000000000001,"15min":0.218},"max":{"1min":0.58299999999999996,"5min":0.67800000000000005,"15min":0.94599999999999995},"last":0.34300000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.437,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.251},"max":{"1min":0.60699999999999998,"5min":0.68999999999999995,"15min":1.1619999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46000000000000002,"15min":0.42599999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":0.70899999999999996,"15min":1.2989999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.80600000000000005,"15min":1.337},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75900000000000001,"5min":0.83899999999999997,"15min":2.758},"last":0.57499999999999996}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.874,"15min":1.276},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.60699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.88700000000000001,"15min":1.895},"last":0.56000000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.40699999999999997},"max":{"1min":0.84199999999999997,"5min":0.872,"15min":5.4539999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82099999999999995,"5min":0.93600000000000005,"15min":2.8159999999999998},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.501,"5min":0.44600000000000001,"15min":0.41299999999999998},"max":{"1min":0.85699999999999998,"5min":0.89800000000000002,"15min":3.7400000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.443,"5min":0.443,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":5.4900000000000002},"last":0.60699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34999999999999998,"15min":0.34599999999999997},"min":{"1min":0.249,"5min":0.191,"15min":0.191},"max":{"1min":0.49099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.379},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.379},"min":{"1min":0.247,"5min":0.245,"15min":0.22},"max":{"1min":0.58199999999999996,"5min":0.64400000000000002,"15min":0.872},"last":0.36499999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.371,"15min":0.36899999999999999},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.20999999999999999},"max":{"1min":0.49399999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.30299999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.746},"last":0.40000000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41899999999999998,"15min":0.41199999999999998},"min":{"1min":0.32300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.54200000000000004,"5min":0.65200000000000002,"15min":0.67100000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39900000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.248},"max":{"1min":0.56299999999999994,"5min":0.66700000000000004,"15min":0.72499999999999998},"last":0.33300000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.62,"15min":0.629},"min":{"1min":0.47599999999999998,"5min":0.43099999999999999,"15min":0.38500000000000001},"max":{"1min":0.80100000000000005,"5min":0.87,"15min":1.216},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.61699999999999999},"min":{"1min":0.503,"5min":0.41099999999999998,"15min":0.40300000000000002},"max":{"1min":0.80000000000000004,"5min":0.89600000000000002,"15min":0.99299999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.90400000000000003,"15min":0.95699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.63900000000000001,"15min":0.63200000000000001},"min":{"1min":0.47899999999999998,"5min":0.42099999999999999,"15min":0.40300000000000002},"max":{"1min":0.81499999999999995,"5min":0.93700000000000006,"15min":1.101},"last":0.58799999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65300000000000002,"15min":0.63600000000000001},"min":{"1min":0.46400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83199999999999996,"5min":0.89100000000000001,"15min":0.97299999999999998},"last":0.622},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64800000000000002,"15min":0.64500000000000002},"min":{"1min":0.48799999999999999,"5min":0.45800000000000002,"15min":0.436},"max":{"1min":0.78600000000000003,"5min":0.89500000000000002,"15min":1.0780000000000001},"last":0.64500000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.54100000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80000000000000004,"5min":0.88500000000000001,"15min":1.155},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.41599999999999998},"max":{"1min":0.81399999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.34200000000000003},"min":{"1min":0.22500000000000001,"5min":0.214,"15min":0.19400000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.64400000000000002},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.44700000000000001},"min":{"1min":0.25800000000000001,"5min":0.218,"15min":0.20699999999999999},"max":{"1min":0.56499999999999995,"5min":0.71999999999999997,"15min":2.7069999999999999},"last":0.314}]},{"osd":2,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.36699999999999999,"15min":0.35099999999999998},"min":{"1min":0.22800000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57299999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.221},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.38500000000000001},"min":{"1min":0.28799999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.53300000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34599999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.437,"15min":0.41899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.76900000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40799999999999997,"15min":0.47299999999999998},"min":{"1min":0.27300000000000002,"5min":0.25600000000000001,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.73899999999999999,"15min":2.75},"last":0.36299999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60599999999999998,"15min":0.59299999999999997},"min":{"1min":0.44900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.95899999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83299999999999996,"5min":1.002,"15min":1.002},"last":0.45100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.0900000000000001,"15min":2.778},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.438,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88800000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.50700000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.68700000000000006},"min":{"1min":0.46700000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86299999999999999,"5min":1.026,"15min":2.8210000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.51400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93300000000000005,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.438}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84999999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65600000000000003,"15min":0.72399999999999998},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91400000000000003,"5min":0.91600000000000004,"15min":2.7989999999999999},"last":0.46500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1802240,"data_stored":1108436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"min":{"1min":0.254,"5min":0.192,"15min":0.184},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.375,"15min":0.35799999999999998},"min":{"1min":0.28199999999999997,"5min":0.23300000000000001,"15min":0.217},"max":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.22900000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.374,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.25,"15min":0.25},"max":{"1min":0.56799999999999995,"5min":0.63,"15min":0.63},"last":0.246},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.377},"min":{"1min":0.26500000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.59399999999999997},"last":0.28699999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.38700000000000001},"min":{"1min":0.27100000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.67600000000000005},"last":0.252},{"interface":"front","average":{"1min":0.496,"5min":0.44700000000000001,"15min":0.432},"min":{"1min":0.35899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.35899999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60599999999999998,"15min":0.59099999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.89100000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61699999999999999,"15min":0.59799999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91200000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.90000000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.59499999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.95999999999999996},"last":0.56799999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65000000000000002,"15min":0.629},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.93600000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63}]},{"osd":7,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64400000000000002,"15min":0.61299999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.878,"15min":0.93600000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63600000000000001,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.46500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:49:46.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2024-03-23T09:49:46.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2024-03-23T09:49:46.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2024-03-23T09:49:46.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2024-03-23T09:49:46.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2024-03-23T09:49:46.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2024-03-23T09:49:46.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2024-03-23T09:49:46.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2024-03-23T09:49:46.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2024-03-23T09:49:46.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2024-03-23T09:49:46.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2024-03-23T09:49:46.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2024-03-23T09:49:46.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2024-03-23T09:49:46.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2024-03-23T09:49:46.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2024-03-23T09:49:46.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2024-03-23T09:49:46.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2024-03-23T09:49:46.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2024-03-23T09:49:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2024-03-23T09:49:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2024-03-23T09:49:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2024-03-23T09:49:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2024-03-23T09:49:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2024-03-23T09:49:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2024-03-23T09:49:46.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2024-03-23T09:49:46.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2024-03-23T09:49:46.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2024-03-23T09:49:46.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2024-03-23T09:49:46.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2024-03-23T09:49:46.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2024-03-23T09:49:46.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2024-03-23T09:49:46.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2024-03-23T09:49:46.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2024-03-23T09:49:46.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-03-23T09:49:46.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.870278+0000', 'last_change': '2024-03-23T09:49:41.870278+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.870278+0000', 'last_undegraded': '2024-03-23T09:49:41.870278+0000', 'last_fullsized': '2024-03-23T09:49:41.870278+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T13:25:47.364729+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:49:46.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2024-03-23T09:49:46.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2024-03-23T09:49:46.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.870832+0000', 'last_change': '2024-03-23T09:49:41.870832+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.870832+0000', 'last_undegraded': '2024-03-23T09:49:41.870832+0000', 'last_fullsized': '2024-03-23T09:49:41.870832+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:49:46.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2024-03-23T09:49:46.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-03-23T09:49:46.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.870647+0000', 'last_change': '2024-03-23T09:49:41.870647+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.870647+0000', 'last_undegraded': '2024-03-23T09:49:41.870647+0000', 'last_fullsized': '2024-03-23T09:49:41.870647+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T10:56:58.937373+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:49:46.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2024-03-23T09:49:46.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-03-23T09:49:46.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.870392+0000', 'last_change': '2024-03-23T09:49:41.870392+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.870392+0000', 'last_undegraded': '2024-03-23T09:49:41.870392+0000', 'last_fullsized': '2024-03-23T09:49:41.870392+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:49:46.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2024-03-23T09:49:46.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2024-03-23T09:49:46.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.870487+0000', 'last_change': '2024-03-23T09:49:41.870487+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.870487+0000', 'last_undegraded': '2024-03-23T09:49:41.870487+0000', 'last_fullsized': '2024-03-23T09:49:41.870487+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T13:09:41.124566+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:49:46.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2024-03-23T09:49:46.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-03-23T09:49:46.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.869903+0000', 'last_change': '2024-03-23T09:49:41.869903+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.869903+0000', 'last_undegraded': '2024-03-23T09:49:41.869903+0000', 'last_fullsized': '2024-03-23T09:49:41.869903+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:04:23.374836+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:49:46.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2024-03-23T09:49:46.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-03-23T09:49:46.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.872359+0000', 'last_change': '2024-03-23T09:49:41.872359+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.872359+0000', 'last_undegraded': '2024-03-23T09:49:41.872359+0000', 'last_fullsized': '2024-03-23T09:49:41.872359+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:49:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2024-03-23T09:49:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2024-03-23T09:49:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.871434+0000', 'last_change': '2024-03-23T09:49:41.871434+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.871434+0000', 'last_undegraded': '2024-03-23T09:49:41.871434+0000', 'last_fullsized': '2024-03-23T09:49:41.871434+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T15:29:10.221541+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:49:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2024-03-23T09:49:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2024-03-23T09:49:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.870644+0000', 'last_change': '2024-03-23T09:49:41.870644+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.870644+0000', 'last_undegraded': '2024-03-23T09:49:41.870644+0000', 'last_fullsized': '2024-03-23T09:49:41.870644+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:59:39.896584+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:49:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2024-03-23T09:49:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2024-03-23T09:49:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.868045+0000', 'last_change': '2024-03-23T09:49:41.868045+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.868045+0000', 'last_undegraded': '2024-03-23T09:49:41.868045+0000', 'last_fullsized': '2024-03-23T09:49:41.868045+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:21:22.073815+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:49:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2024-03-23T09:49:46.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2024-03-23T09:49:46.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.871230+0000', 'last_change': '2024-03-23T09:49:41.871230+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.871230+0000', 'last_undegraded': '2024-03-23T09:49:41.871230+0000', 'last_fullsized': '2024-03-23T09:49:41.871230+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:49:46.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2024-03-23T09:49:46.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2024-03-23T09:49:46.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.869789+0000', 'last_change': '2024-03-23T09:49:41.869789+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.869789+0000', 'last_undegraded': '2024-03-23T09:49:41.869789+0000', 'last_fullsized': '2024-03-23T09:49:41.869789+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:49:46.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2024-03-23T09:49:46.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2024-03-23T09:49:46.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.873527+0000', 'last_change': '2024-03-23T09:49:41.873527+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.873527+0000', 'last_undegraded': '2024-03-23T09:49:41.873527+0000', 'last_fullsized': '2024-03-23T09:49:41.873527+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T17:06:29.138334+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:49:46.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2024-03-23T09:49:46.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2024-03-23T09:49:46.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.873607+0000', 'last_change': '2024-03-23T09:49:41.873607+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.873607+0000', 'last_undegraded': '2024-03-23T09:49:41.873607+0000', 'last_fullsized': '2024-03-23T09:49:41.873607+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T15:22:32.641742+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:49:46.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2024-03-23T09:49:46.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2024-03-23T09:49:46.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.869776+0000', 'last_change': '2024-03-23T09:49:41.869776+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.869776+0000', 'last_undegraded': '2024-03-23T09:49:41.869776+0000', 'last_fullsized': '2024-03-23T09:49:41.869776+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:49:46.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2024-03-23T09:49:46.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2024-03-23T09:49:46.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.871016+0000', 'last_change': '2024-03-23T09:49:41.871016+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.871016+0000', 'last_undegraded': '2024-03-23T09:49:41.871016+0000', 'last_fullsized': '2024-03-23T09:49:41.871016+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:24:49.878448+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-23T09:49:46.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2024-03-23T09:49:46.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.869713+0000', 'last_change': '2024-03-23T09:49:41.869713+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.869713+0000', 'last_undegraded': '2024-03-23T09:49:41.869713+0000', 'last_fullsized': '2024-03-23T09:49:41.869713+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.872429+0000', 'last_change': '2024-03-23T09:49:41.872429+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.872429+0000', 'last_undegraded': '2024-03-23T09:49:41.872429+0000', 'last_fullsized': '2024-03-23T09:49:41.872429+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T13:25:47.364729+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.869932+0000', 'last_change': '2024-03-23T09:49:41.869932+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.869932+0000', 'last_undegraded': '2024-03-23T09:49:41.869932+0000', 'last_fullsized': '2024-03-23T09:49:41.869932+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T21:14:31.482221+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-03-23T09:49:46.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.871303+0000', 'last_change': '2024-03-23T09:49:41.871303+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.871303+0000', 'last_undegraded': '2024-03-23T09:49:41.871303+0000', 'last_fullsized': '2024-03-23T09:49:41.871303+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:21:22.073815+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:49:46.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2024-03-23T09:49:46.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-03-23T09:49:46.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.874136+0000', 'last_change': '2024-03-23T09:49:41.874136+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.874136+0000', 'last_undegraded': '2024-03-23T09:49:41.874136+0000', 'last_fullsized': '2024-03-23T09:49:41.874136+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:24:49.878448+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:49:46.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2024-03-23T09:49:46.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-03-23T09:49:46.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.874193+0000', 'last_change': '2024-03-23T09:49:41.874193+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.874193+0000', 'last_undegraded': '2024-03-23T09:49:41.874193+0000', 'last_fullsized': '2024-03-23T09:49:41.874193+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:09:35.927315+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:49:46.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2024-03-23T09:49:46.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-03-23T09:49:46.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.869882+0000', 'last_change': '2024-03-23T09:49:41.869882+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.869882+0000', 'last_undegraded': '2024-03-23T09:49:41.869882+0000', 'last_fullsized': '2024-03-23T09:49:41.869882+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T21:38:09.171978+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:49:46.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2024-03-23T09:49:46.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-03-23T09:49:46.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.874150+0000', 'last_change': '2024-03-23T09:49:41.874150+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.874150+0000', 'last_undegraded': '2024-03-23T09:49:41.874150+0000', 'last_fullsized': '2024-03-23T09:49:41.874150+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T19:59:39.896584+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-23T09:49:46.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2024-03-23T09:49:46.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-03-23T09:49:46.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.870470+0000', 'last_change': '2024-03-23T09:49:41.870470+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.870470+0000', 'last_undegraded': '2024-03-23T09:49:41.870470+0000', 'last_fullsized': '2024-03-23T09:49:41.870470+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T13:15:46.255074+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:49:46.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2024-03-23T09:49:46.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-03-23T09:49:46.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.869958+0000', 'last_change': '2024-03-23T09:49:41.869958+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.869958+0000', 'last_undegraded': '2024-03-23T09:49:41.869958+0000', 'last_fullsized': '2024-03-23T09:49:41.869958+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:49:46.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2024-03-23T09:49:46.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-03-23T09:49:46.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.872465+0000', 'last_change': '2024-03-23T09:49:41.872465+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.872465+0000', 'last_undegraded': '2024-03-23T09:49:41.872465+0000', 'last_fullsized': '2024-03-23T09:49:41.872465+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:04:23.374836+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:49:46.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2024-03-23T09:49:46.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-03-23T09:49:46.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.869975+0000', 'last_change': '2024-03-23T09:49:41.869975+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.869975+0000', 'last_undegraded': '2024-03-23T09:49:41.869975+0000', 'last_fullsized': '2024-03-23T09:49:41.869975+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:33:50.023224+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-23T09:49:46.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2024-03-23T09:49:46.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-03-23T09:49:46.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.872851+0000', 'last_change': '2024-03-23T09:49:41.872851+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.872851+0000', 'last_undegraded': '2024-03-23T09:49:41.872851+0000', 'last_fullsized': '2024-03-23T09:49:41.872851+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T20:33:50.023224+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-23T09:49:46.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2024-03-23T09:49:46.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2024-03-23T09:49:46.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.868124+0000', 'last_change': '2024-03-23T09:49:41.868124+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.868124+0000', 'last_undegraded': '2024-03-23T09:49:41.868124+0000', 'last_fullsized': '2024-03-23T09:49:41.868124+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:58:35.947175+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-23T09:49:46.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2024-03-23T09:49:46.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2024-03-23T09:49:46.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.873525+0000', 'last_change': '2024-03-23T09:49:41.873525+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.873525+0000', 'last_undegraded': '2024-03-23T09:49:41.873525+0000', 'last_fullsized': '2024-03-23T09:49:41.873525+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T16:41:54.983525+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-23T09:49:46.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2024-03-23T09:49:46.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-23T09:49:42.865767+0000', 'last_change': '2024-03-23T09:49:42.865767+0000', 'last_active': '2024-03-23T09:49:42.865767+0000', 'last_peered': '2024-03-23T09:49:42.865767+0000', 'last_clean': '2024-03-23T09:49:42.865767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:42.865767+0000', 'last_undegraded': '2024-03-23T09:49:42.865767+0000', 'last_fullsized': '2024-03-23T09:49:42.865767+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:42.865767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-23T09:49:46.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2024-03-23T09:49:46.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-03-23T09:49:41.870759+0000', 'last_change': '2024-03-23T09:49:41.870759+0000', 'last_active': '2024-03-23T09:49:41.863654+0000', 'last_peered': '2024-03-23T09:49:41.863654+0000', 'last_clean': '2024-03-23T09:49:41.863654+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-23T09:49:41.870759+0000', 'last_undegraded': '2024-03-23T09:49:41.870759+0000', 'last_fullsized': '2024-03-23T09:49:41.870759+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'last_clean_scrub_stamp': '2024-03-23T09:49:41.863654+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-23T09:49:46.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-23T09:49:47.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.090+0000 7f203fbcb640 1 Processor -- start 2024-03-23T09:49:47.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.090+0000 7f203fbcb640 1 -- start start 2024-03-23T09:49:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.091+0000 7f203fbcb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20380b8740 0x7f203815d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.091+0000 7f203fbcb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20380cfb30 0x7f20380cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.091+0000 7f203fbcb640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20380b45e0 con 0x7f20380b8360 2024-03-23T09:49:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.091+0000 7f203fbcb640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20380b48c0 con 0x7f20380cfb30 2024-03-23T09:49:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.091+0000 7f203fbcb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20380b4ba0 con 0x7f20380b8740 2024-03-23T09:49:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.091+0000 7f203e168640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20380cfb30 0x7f20380cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.091+0000 7f203e168640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20380cfb30 0x7f20380cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44240/0 (socket says 172.21.15.47:44240) 2024-03-23T09:49:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.091+0000 7f203e168640 1 -- 172.21.15.47:0/2542547274 learned_addr learned my addr 172.21.15.47:0/2542547274 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.091+0000 7f203e168640 1 -- 172.21.15.47:0/2542547274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20380b8740 msgr2=0x7f203815d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.091+0000 7f203e168640 1 --2- 172.21.15.47:0/2542547274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20380b8740 0x7f203815d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.091+0000 7f203e168640 1 -- 172.21.15.47:0/2542547274 >> v1:172.21.15.175:6789/0 conn(0x7f20380b8360 legacy=0x7f20380cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.091+0000 7f203e168640 1 -- 172.21.15.47:0/2542547274 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20380b4e80 con 0x7f20380cfb30 2024-03-23T09:49:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.092+0000 7f203e168640 1 --2- 172.21.15.47:0/2542547274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380cfb30 0x7f20380cd800 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f2030009e50 tx=0x7f203002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592cbc2bab3b517 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.092+0000 7f203c965640 1 -- 172.21.15.47:0/2542547274 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203002c9e0 con 0x7f20380cfb30 2024-03-23T09:49:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.092+0000 7f203c965640 1 -- 172.21.15.47:0/2542547274 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203002cb40 con 0x7f20380cfb30 2024-03-23T09:49:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.092+0000 7f203c965640 1 -- 172.21.15.47:0/2542547274 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20300343f0 con 0x7f20380cfb30 2024-03-23T09:49:47.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.093+0000 7f203fbcb640 1 -- 172.21.15.47:0/2542547274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380cfb30 msgr2=0x7f20380cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:47.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.093+0000 7f203fbcb640 1 --2- 172.21.15.47:0/2542547274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380cfb30 0x7f20380cd800 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f2030009e50 tx=0x7f203002be40 comp rx=0 tx=0).stop 2024-03-23T09:49:47.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.093+0000 7f203fbcb640 1 -- 172.21.15.47:0/2542547274 shutdown_connections 2024-03-23T09:49:47.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.093+0000 7f203fbcb640 1 --2- 172.21.15.47:0/2542547274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380cfb30 0x7f20380cd800 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.093+0000 7f203fbcb640 1 --2- 172.21.15.47:0/2542547274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20380b8740 0x7f203815d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.093+0000 7f203fbcb640 1 -- 172.21.15.47:0/2542547274 >> 172.21.15.47:0/2542547274 conn(0x7f20380659d0 msgr2=0x7f2038065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:47.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.093+0000 7f203fbcb640 1 -- 172.21.15.47:0/2542547274 shutdown_connections 2024-03-23T09:49:47.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.093+0000 7f203fbcb640 1 -- 172.21.15.47:0/2542547274 wait complete. 2024-03-23T09:49:47.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.094+0000 7f203fbcb640 1 Processor -- start 2024-03-23T09:49:47.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.094+0000 7f203fbcb640 1 -- start start 2024-03-23T09:49:47.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203fbcb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20380b8360 0x7f20380b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203fbcb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20380b8740 0x7f203814ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203fbcb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380cfb30 0x7f20381f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203fbcb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20380b3d20 con 0x7f20380cfb30 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203fbcb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203814dea0 con 0x7f20380b8740 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203fbcb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203814e180 con 0x7f20380b8360 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203e168640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380cfb30 0x7f20381f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203e168640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380cfb30 0x7f20381f2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44246/0 (socket says 172.21.15.47:44246) 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203e168640 1 -- 172.21.15.47:0/3650894745 learned_addr learned my addr 172.21.15.47:0/3650894745 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203d967640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20380b8360 0x7f20380b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203d166640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20380b8740 0x7f203814ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203e168640 1 -- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20380b8360 msgr2=0x7f20380b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203e168640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20380b8360 0x7f20380b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203e168640 1 -- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20380b8740 msgr2=0x7f203814ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203e168640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20380b8740 0x7f203814ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203e168640 1 -- 172.21.15.47:0/3650894745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2030009b30 con 0x7f20380cfb30 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203d166640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20380b8740 0x7f203814ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203e168640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380cfb30 0x7f20381f2950 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f2030036040 tx=0x7f2030004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f202e7fc640 1 -- 172.21.15.47:0/3650894745 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203003e450 con 0x7f20380cfb30 2024-03-23T09:49:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f202e7fc640 1 -- 172.21.15.47:0/3650894745 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203003e5d0 con 0x7f20380cfb30 2024-03-23T09:49:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203fbcb640 1 -- 172.21.15.47:0/3650894745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203814e460 con 0x7f20380cfb30 2024-03-23T09:49:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f202e7fc640 1 -- 172.21.15.47:0/3650894745 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203003e980 con 0x7f20380cfb30 2024-03-23T09:49:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.095+0000 7f203fbcb640 1 -- 172.21.15.47:0/3650894745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203814ea20 con 0x7f20380cfb30 2024-03-23T09:49:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.096+0000 7f203fbcb640 1 -- 172.21.15.47:0/3650894745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2038072360 con 0x7f20380cfb30 2024-03-23T09:49:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.097+0000 7f202e7fc640 1 -- 172.21.15.47:0/3650894745 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f203003eae0 con 0x7f20380cfb30 2024-03-23T09:49:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.097+0000 7f202e7fc640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f201c06ac00 0x7f201c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.097+0000 7f202e7fc640 1 -- 172.21.15.47:0/3650894745 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20300ac220 con 0x7f20380cfb30 2024-03-23T09:49:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.098+0000 7f203d967640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f201c06ac00 0x7f201c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.098+0000 7f203d967640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f201c06ac00 0x7f201c06d0c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2034007fd0 tx=0x7f2034009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.102+0000 7f202e7fc640 1 -- 172.21.15.47:0/3650894745 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2030077400 con 0x7f20380cfb30 2024-03-23T09:49:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.354+0000 7f203fbcb640 1 -- 172.21.15.47:0/3650894745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2038066be0 con 0x7f20380cfb30 2024-03-23T09:49:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.355+0000 7f202e7fc640 1 -- 172.21.15.47:0/3650894745 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1110 (secure 0 0 0) 0x7f203007aa20 con 0x7f20380cfb30 2024-03-23T09:49:47.355 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:47.355 INFO:teuthology.orchestra.run.smithi047.stdout:{"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":541,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":101,"num_osds":8,"num_up_osds":8,"osd_up_since":1711186856,"num_in_osds":8,"osd_in_since":1711186848,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":265805824,"bytes_avail":772828307456,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8484}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-03-23T09:49:32.467805+0000","services":{}},"progress_events":{}} 2024-03-23T09:49:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.357+0000 7f203fbcb640 1 -- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f201c06ac00 msgr2=0x7f201c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.357+0000 7f203fbcb640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f201c06ac00 0x7f201c06d0c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2034007fd0 tx=0x7f2034009040 comp rx=0 tx=0).stop 2024-03-23T09:49:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.357+0000 7f203fbcb640 1 -- 172.21.15.47:0/3650894745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380cfb30 msgr2=0x7f20381f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.357+0000 7f203fbcb640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380cfb30 0x7f20381f2950 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f2030036040 tx=0x7f2030004a40 comp rx=0 tx=0).stop 2024-03-23T09:49:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.357+0000 7f203fbcb640 1 -- 172.21.15.47:0/3650894745 shutdown_connections 2024-03-23T09:49:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.357+0000 7f203fbcb640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f201c06ac00 0x7f201c06d0c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.357+0000 7f203fbcb640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380cfb30 0x7f20381f2950 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.357+0000 7f203fbcb640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20380b8740 0x7f203814ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.357+0000 7f203fbcb640 1 --2- 172.21.15.47:0/3650894745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20380b8360 0x7f20380b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.357+0000 7f203fbcb640 1 -- 172.21.15.47:0/3650894745 >> 172.21.15.47:0/3650894745 conn(0x7f20380659d0 msgr2=0x7f20380d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.357+0000 7f203fbcb640 1 -- 172.21.15.47:0/3650894745 shutdown_connections 2024-03-23T09:49:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.357+0000 7f203fbcb640 1 -- 172.21.15.47:0/3650894745 wait complete. 2024-03-23T09:49:47.368 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-23T09:49:47.368 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:49:47.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.469+0000 7f68f3638640 1 Processor -- start 2024-03-23T09:49:47.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.469+0000 7f68f3638640 1 -- start start 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68f3638640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0cde10 0x7f68ec167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68f3638640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ec0d11e0 con 0x7f68ec0b8350 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68f3638640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ec0d2680 con 0x7f68ec15ca90 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68f3638640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec160f60 con 0x7f68ec0cde10 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68f0bd3640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f68ec0b8350 0x7f68ec0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47520/0 (socket says 172.21.15.47:47520) 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68f0bd3640 1 -- 172.21.15.47:0/2580543276 learned_addr learned my addr 172.21.15.47:0/2580543276 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68f1bd5640 1 --2- 172.21.15.47:0/2580543276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0cde10 0x7f68ec167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872887529 0 0) 0x7f68ec0d11e0 con 0x7f68ec0b8350 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d8003520 con 0x7f68ec0b8350 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1874589048 0 0) 0x7f68d8003520 con 0x7f68ec0b8350 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0cde10 msgr2=0x7f68ec167aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 --2- 172.21.15.47:0/2580543276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0cde10 0x7f68ec167aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 >> v1:172.21.15.175:6789/0 conn(0x7f68ec15ca90 legacy=0x7f68ec160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec161240 con 0x7f68ec0b8350 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f68dc001670 con 0x7f68ec0b8350 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 >> v1:172.21.15.47:6789/0 conn(0x7f68ec0b8350 legacy=0x7f68ec0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 --2- 172.21.15.47:0/2580543276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0cde10 0x7f68d80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 --2- 172.21.15.47:0/2580543276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d8003ee0 0x7f68d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 --2- 172.21.15.47:0/2580543276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68d8004d00 0x7f68d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec160f60 con 0x7f68d8004d00 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d800ba90 con 0x7f68d8003ee0 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d800bc50 con 0x7f68ec0cde10 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.470+0000 7f68f1bd5640 1 --2- 172.21.15.47:0/2580543276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68d8004d00 0x7f68d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.471+0000 7f68f13d4640 1 --2- 172.21.15.47:0/2580543276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0cde10 0x7f68d80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.471+0000 7f68f0bd3640 1 --2- 172.21.15.47:0/2580543276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d8003ee0 0x7f68d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.471+0000 7f68f1bd5640 1 -- 172.21.15.47:0/2580543276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0cde10 msgr2=0x7f68d80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.471+0000 7f68f1bd5640 1 --2- 172.21.15.47:0/2580543276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0cde10 0x7f68d80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.471+0000 7f68f1bd5640 1 -- 172.21.15.47:0/2580543276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d8003ee0 msgr2=0x7f68d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.471+0000 7f68f1bd5640 1 --2- 172.21.15.47:0/2580543276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d8003ee0 0x7f68d80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.471+0000 7f68f1bd5640 1 -- 172.21.15.47:0/2580543276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d8003520 con 0x7f68d8004d00 2024-03-23T09:49:47.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.471+0000 7f68f0bd3640 1 --2- 172.21.15.47:0/2580543276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d8003ee0 0x7f68d80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:47.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.471+0000 7f68f1bd5640 1 --2- 172.21.15.47:0/2580543276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68d8004d00 0x7f68d8005180 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f68ec08bd50 tx=0x7f68d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145f3fd52132c91e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:47.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.472+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e402cb90 con 0x7f68d8004d00 2024-03-23T09:49:47.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.472+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e400f040 con 0x7f68d8004d00 2024-03-23T09:49:47.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.472+0000 7f68e3fff640 1 -- 172.21.15.47:0/2580543276 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4034440 con 0x7f68d8004d00 2024-03-23T09:49:47.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.472+0000 7f68f3638640 1 -- 172.21.15.47:0/2580543276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68d8004d00 msgr2=0x7f68d8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:47.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.472+0000 7f68f3638640 1 --2- 172.21.15.47:0/2580543276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68d8004d00 0x7f68d8005180 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f68ec08bd50 tx=0x7f68d80057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.472+0000 7f68f1bd5640 1 -- 172.21.15.47:0/2580543276 reap_dead start 2024-03-23T09:49:47.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.472+0000 7f68f3638640 1 -- 172.21.15.47:0/2580543276 shutdown_connections 2024-03-23T09:49:47.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.472+0000 7f68f3638640 1 -- 172.21.15.47:0/2580543276 >> 172.21.15.47:0/2580543276 conn(0x7f68ec0659d0 msgr2=0x7f68ec065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:47.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.472+0000 7f68f3638640 1 -- 172.21.15.47:0/2580543276 shutdown_connections 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.472+0000 7f68f3638640 1 -- 172.21.15.47:0/2580543276 wait complete. 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.473+0000 7f68f3638640 1 Processor -- start 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.473+0000 7f68f3638640 1 -- start start 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.474+0000 7f68f3638640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d8004d00 0x7f68ec153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.474+0000 7f68f3638640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0d2c70 0x7f68ec0d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.474+0000 7f68f3638640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ec15a380 0x7f68ec15a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.474+0000 7f68f3638640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec0d3610 con 0x7f68ec15a380 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.474+0000 7f68f3638640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec0b4120 con 0x7f68d8004d00 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.474+0000 7f68f3638640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec0b3510 con 0x7f68ec0d2c70 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.474+0000 7f68f1bd5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ec15a380 0x7f68ec15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.474+0000 7f68f1bd5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ec15a380 0x7f68ec15a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44268/0 (socket says 172.21.15.47:44268) 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.474+0000 7f68f1bd5640 1 -- 172.21.15.47:0/4271336196 learned_addr learned my addr 172.21.15.47:0/4271336196 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.474+0000 7f68f1bd5640 1 -- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0d2c70 msgr2=0x7f68ec0d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:47.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.474+0000 7f68f0bd3640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0d2c70 0x7f68ec0d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.474+0000 7f68f13d4640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d8004d00 0x7f68ec153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.475+0000 7f68f1bd5640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0d2c70 0x7f68ec0d30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.475+0000 7f68f1bd5640 1 -- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d8004d00 msgr2=0x7f68ec153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.475+0000 7f68f1bd5640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d8004d00 0x7f68ec153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.475+0000 7f68f1bd5640 1 -- 172.21.15.47:0/4271336196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec0b37f0 con 0x7f68ec15a380 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.475+0000 7f68f13d4640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d8004d00 0x7f68ec153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.475+0000 7f68f0bd3640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0d2c70 0x7f68ec0d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.475+0000 7f68f1bd5640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ec15a380 0x7f68ec15a800 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f68e4002230 tx=0x7f68e4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.475+0000 7f68e1ffb640 1 -- 172.21.15.47:0/4271336196 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4043070 con 0x7f68ec15a380 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.475+0000 7f68e1ffb640 1 -- 172.21.15.47:0/4271336196 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e4003990 con 0x7f68ec15a380 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.475+0000 7f68f3638640 1 -- 172.21.15.47:0/4271336196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ec0b3ad0 con 0x7f68ec15a380 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.475+0000 7f68e1ffb640 1 -- 172.21.15.47:0/4271336196 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4003d40 con 0x7f68ec15a380 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.475+0000 7f68f3638640 1 -- 172.21.15.47:0/4271336196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ec15ad40 con 0x7f68ec15a380 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.476+0000 7f68f3638640 1 -- 172.21.15.47:0/4271336196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ec072360 con 0x7f68ec15a380 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.476+0000 7f68e1ffb640 1 -- 172.21.15.47:0/4271336196 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f68e4034940 con 0x7f68ec15a380 2024-03-23T09:49:47.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.477+0000 7f68e1ffb640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f68d006ac30 0x7f68d006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:47.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.477+0000 7f68e1ffb640 1 -- 172.21.15.47:0/4271336196 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68e40aab00 con 0x7f68ec15a380 2024-03-23T09:49:47.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.477+0000 7f68f13d4640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f68d006ac30 0x7f68d006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:47.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.478+0000 7f68f13d4640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f68d006ac30 0x7f68d006d0f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f68e8003700 tx=0x7f68e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:47.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.481+0000 7f68e1ffb640 1 -- 172.21.15.47:0/4271336196 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f68e4075d70 con 0x7f68ec15a380 2024-03-23T09:49:47.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.656+0000 7f68f3638640 1 -- 172.21.15.47:0/4271336196 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f68ec0d0e70 con 0x7f68d006ac30 2024-03-23T09:49:47.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.664+0000 7f68e1ffb640 1 -- 172.21.15.47:0/4271336196 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278936 (secure 0 0 0) 0x7f68e80015a0 con 0x7f68d006ac30 2024-03-23T09:49:47.667 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:47.668 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:49:47.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.667+0000 7f68f3638640 1 -- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f68d006ac30 msgr2=0x7f68d006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:47.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.667+0000 7f68f3638640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f68d006ac30 0x7f68d006d0f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f68e8003700 tx=0x7f68e8009040 comp rx=0 tx=0).stop 2024-03-23T09:49:47.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.667+0000 7f68f3638640 1 -- 172.21.15.47:0/4271336196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ec15a380 msgr2=0x7f68ec15a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:47.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.667+0000 7f68f3638640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ec15a380 0x7f68ec15a800 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f68e4002230 tx=0x7f68e4009750 comp rx=0 tx=0).stop 2024-03-23T09:49:47.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.667+0000 7f68f3638640 1 -- 172.21.15.47:0/4271336196 shutdown_connections 2024-03-23T09:49:47.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.667+0000 7f68f3638640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f68d006ac30 0x7f68d006d0f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.667+0000 7f68f3638640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ec15a380 0x7f68ec15a800 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.667+0000 7f68f3638640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68ec0d2c70 0x7f68ec0d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.667+0000 7f68f3638640 1 --2- 172.21.15.47:0/4271336196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d8004d00 0x7f68ec153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:47.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.667+0000 7f68f3638640 1 -- 172.21.15.47:0/4271336196 >> 172.21.15.47:0/4271336196 conn(0x7f68ec0659d0 msgr2=0x7f68ec160460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:47.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.667+0000 7f68f3638640 1 -- 172.21.15.47:0/4271336196 shutdown_connections 2024-03-23T09:49:47.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:47.667+0000 7f68f3638640 1 -- 172.21.15.47:0/4271336196 wait complete. 2024-03-23T09:49:47.678 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-03-23T09:49:46.470979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259576,"kb_used_data":10624,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754715144,"statfs":{"total":773094113280,"available":772828307456,"internally_reserved":0,"allocated":10878976,"data_stored":5328994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.602065"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870278+0000","last_change":"2024-03-23T09:49:41.870278+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870278+0000","last_undegraded":"2024-03-23T09:49:41.870278+0000","last_fullsized":"2024-03-23T09:49:41.870278+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:25:47.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870832+0000","last_change":"2024-03-23T09:49:41.870832+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870832+0000","last_undegraded":"2024-03-23T09:49:41.870832+0000","last_fullsized":"2024-03-23T09:49:41.870832+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870647+0000","last_change":"2024-03-23T09:49:41.870647+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870647+0000","last_undegraded":"2024-03-23T09:49:41.870647+0000","last_fullsized":"2024-03-23T09:49:41.870647+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:56:58.937373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870392+0000","last_change":"2024-03-23T09:49:41.870392+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870392+0000","last_undegraded":"2024-03-23T09:49:41.870392+0000","last_fullsized":"2024-03-23T09:49:41.870392+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870487+0000","last_change":"2024-03-23T09:49:41.870487+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870487+0000","last_undegraded":"2024-03-23T09:49:41.870487+0000","last_fullsized":"2024-03-23T09:49:41.870487+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:09:41.124566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869903+0000","last_change":"2024-03-23T09:49:41.869903+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869903+0000","last_undegraded":"2024-03-23T09:49:41.869903+0000","last_fullsized":"2024-03-23T09:49:41.869903+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:04:23.374836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872359+0000","last_change":"2024-03-23T09:49:41.872359+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872359+0000","last_undegraded":"2024-03-23T09:49:41.872359+0000","last_fullsized":"2024-03-23T09:49:41.872359+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871434+0000","last_change":"2024-03-23T09:49:41.871434+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871434+0000","last_undegraded":"2024-03-23T09:49:41.871434+0000","last_fullsized":"2024-03-23T09:49:41.871434+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:29:10.221541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870644+0000","last_change":"2024-03-23T09:49:41.870644+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870644+0000","last_undegraded":"2024-03-23T09:49:41.870644+0000","last_fullsized":"2024-03-23T09:49:41.870644+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:39.896584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.868045+0000","last_change":"2024-03-23T09:49:41.868045+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.868045+0000","last_undegraded":"2024-03-23T09:49:41.868045+0000","last_fullsized":"2024-03-23T09:49:41.868045+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:21:22.073815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871230+0000","last_change":"2024-03-23T09:49:41.871230+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871230+0000","last_undegraded":"2024-03-23T09:49:41.871230+0000","last_fullsized":"2024-03-23T09:49:41.871230+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869789+0000","last_change":"2024-03-23T09:49:41.869789+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869789+0000","last_undegraded":"2024-03-23T09:49:41.869789+0000","last_fullsized":"2024-03-23T09:49:41.869789+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.873527+0000","last_change":"2024-03-23T09:49:41.873527+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.873527+0000","last_undegraded":"2024-03-23T09:49:41.873527+0000","last_fullsized":"2024-03-23T09:49:41.873527+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:29.138334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.873607+0000","last_change":"2024-03-23T09:49:41.873607+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.873607+0000","last_undegraded":"2024-03-23T09:49:41.873607+0000","last_fullsized":"2024-03-23T09:49:41.873607+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:22:32.641742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869776+0000","last_change":"2024-03-23T09:49:41.869776+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869776+0000","last_undegraded":"2024-03-23T09:49:41.869776+0000","last_fullsized":"2024-03-23T09:49:41.869776+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871016+0000","last_change":"2024-03-23T09:49:41.871016+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871016+0000","last_undegraded":"2024-03-23T09:49:41.871016+0000","last_fullsized":"2024-03-23T09:49:41.871016+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:24:49.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869713+0000","last_change":"2024-03-23T09:49:41.869713+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869713+0000","last_undegraded":"2024-03-23T09:49:41.869713+0000","last_fullsized":"2024-03-23T09:49:41.869713+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872429+0000","last_change":"2024-03-23T09:49:41.872429+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872429+0000","last_undegraded":"2024-03-23T09:49:41.872429+0000","last_fullsized":"2024-03-23T09:49:41.872429+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:25:47.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869932+0000","last_change":"2024-03-23T09:49:41.869932+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869932+0000","last_undegraded":"2024-03-23T09:49:41.869932+0000","last_fullsized":"2024-03-23T09:49:41.869932+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:14:31.482221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.871303+0000","last_change":"2024-03-23T09:49:41.871303+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.871303+0000","last_undegraded":"2024-03-23T09:49:41.871303+0000","last_fullsized":"2024-03-23T09:49:41.871303+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:21:22.073815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.874136+0000","last_change":"2024-03-23T09:49:41.874136+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.874136+0000","last_undegraded":"2024-03-23T09:49:41.874136+0000","last_fullsized":"2024-03-23T09:49:41.874136+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:24:49.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.874193+0000","last_change":"2024-03-23T09:49:41.874193+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.874193+0000","last_undegraded":"2024-03-23T09:49:41.874193+0000","last_fullsized":"2024-03-23T09:49:41.874193+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:09:35.927315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869882+0000","last_change":"2024-03-23T09:49:41.869882+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869882+0000","last_undegraded":"2024-03-23T09:49:41.869882+0000","last_fullsized":"2024-03-23T09:49:41.869882+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:09.171978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.874150+0000","last_change":"2024-03-23T09:49:41.874150+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.874150+0000","last_undegraded":"2024-03-23T09:49:41.874150+0000","last_fullsized":"2024-03-23T09:49:41.874150+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:39.896584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870470+0000","last_change":"2024-03-23T09:49:41.870470+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870470+0000","last_undegraded":"2024-03-23T09:49:41.870470+0000","last_fullsized":"2024-03-23T09:49:41.870470+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:15:46.255074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869958+0000","last_change":"2024-03-23T09:49:41.869958+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869958+0000","last_undegraded":"2024-03-23T09:49:41.869958+0000","last_fullsized":"2024-03-23T09:49:41.869958+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872465+0000","last_change":"2024-03-23T09:49:41.872465+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872465+0000","last_undegraded":"2024-03-23T09:49:41.872465+0000","last_fullsized":"2024-03-23T09:49:41.872465+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:04:23.374836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.869975+0000","last_change":"2024-03-23T09:49:41.869975+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.869975+0000","last_undegraded":"2024-03-23T09:49:41.869975+0000","last_fullsized":"2024-03-23T09:49:41.869975+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:33:50.023224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.872851+0000","last_change":"2024-03-23T09:49:41.872851+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.872851+0000","last_undegraded":"2024-03-23T09:49:41.872851+0000","last_fullsized":"2024-03-23T09:49:41.872851+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:33:50.023224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.868124+0000","last_change":"2024-03-23T09:49:41.868124+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.868124+0000","last_undegraded":"2024-03-23T09:49:41.868124+0000","last_fullsized":"2024-03-23T09:49:41.868124+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:58:35.947175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.873525+0000","last_change":"2024-03-23T09:49:41.873525+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.873525+0000","last_undegraded":"2024-03-23T09:49:41.873525+0000","last_fullsized":"2024-03-23T09:49:41.873525+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:41:54.983525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T09:49:42.865767+0000","last_change":"2024-03-23T09:49:42.865767+0000","last_active":"2024-03-23T09:49:42.865767+0000","last_peered":"2024-03-23T09:49:42.865767+0000","last_clean":"2024-03-23T09:49:42.865767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:42.865767+0000","last_undegraded":"2024-03-23T09:49:42.865767+0000","last_fullsized":"2024-03-23T09:49:42.865767+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-03-23T09:49:41.870759+0000","last_change":"2024-03-23T09:49:41.870759+0000","last_active":"2024-03-23T09:49:41.863654+0000","last_peered":"2024-03-23T09:49:41.863654+0000","last_clean":"2024-03-23T09:49:41.863654+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T09:49:41.870759+0000","last_undegraded":"2024-03-23T09:49:41.870759+0000","last_fullsized":"2024-03-23T09:49:41.870759+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":244,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-03-23T09:49:39.852581+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:49:39.852581+0000","last_peered":"2024-03-23T09:49:39.852581+0000","last_clean":"2024-03-23T09:49:39.852581+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:49:39.852581+0000","last_undegraded":"2024-03-23T09:49:39.852581+0000","last_fullsized":"2024-03-23T09:49:39.852581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1806336,"data_stored":1109055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55600000000000005,"15min":0.51100000000000001},"min":{"1min":0.374,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.68000000000000005,"5min":0.76800000000000002,"15min":0.77500000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.53200000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28199999999999997},"max":{"1min":0.71799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57699999999999996,"15min":0.53000000000000003},"min":{"1min":0.41299999999999998,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.76300000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.503},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.30199999999999999},"max":{"1min":0.747,"5min":0.81000000000000005,"15min":0.81299999999999994},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57399999999999995,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58999999999999997,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73299999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55100000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57299999999999995,"15min":0.54000000000000004},"min":{"1min":0.47199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.73199999999999998,"5min":0.84699999999999998,"15min":0.86199999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.59699999999999998,"15min":0.57999999999999996},"min":{"1min":0.503,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.82699999999999996,"15min":0.83699999999999997},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.30399999999999999,"15min":0.29599999999999999},"max":{"1min":0.67500000000000004,"5min":0.76300000000000001,"15min":0.78200000000000003},"last":0.629},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.48999999999999999,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.29299999999999998},"max":{"1min":0.67500000000000004,"5min":0.754,"15min":0.80100000000000005},"last":0.45100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.5,"15min":0.51400000000000001},"min":{"1min":0.35599999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66200000000000003,"5min":0.80100000000000005,"15min":0.81499999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52600000000000002,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.64300000000000002,"5min":0.78000000000000003,"15min":0.84999999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53500000000000003,"15min":0.56999999999999995},"min":{"1min":0.36199999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71499999999999997,"5min":0.75700000000000001,"15min":0.874},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54900000000000004,"15min":0.57299999999999995},"min":{"1min":0.39600000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65400000000000003,"5min":0.79000000000000004,"15min":0.90200000000000002},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":1208320,"data_stored":518068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.52200000000000002},"min":{"1min":0.441,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.81000000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.45200000000000001,"5min":0.37,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.51100000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58999999999999997,"15min":0.53000000000000003},"min":{"1min":0.43099999999999999,"5min":0.315,"15min":0.311},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.44},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59699999999999998,"15min":0.55100000000000005},"min":{"1min":0.495,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42699999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.36299999999999999,"15min":0.29699999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.81599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.56299999999999994},"min":{"1min":0.435,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47299999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.57299999999999995},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.378},"max":{"1min":0.73299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57899999999999996,"15min":0.54500000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":0.79000000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46000000000000002,"15min":0.49199999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.62,"5min":0.69799999999999995,"15min":0.72499999999999998},"last":0.313},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.499,"15min":0.53400000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.66800000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.33200000000000002}]},{"osd":5,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49199999999999999,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.34499999999999997,"15min":0.316},"max":{"1min":0.61399999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52500000000000002,"15min":0.54800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70399999999999996,"5min":0.76200000000000001,"15min":0.77700000000000002},"last":0.371}]},{"osd":7,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55400000000000005,"15min":0.57699999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70099999999999996,"5min":0.78400000000000003,"15min":0.86199999999999999},"last":0.499}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54700000000000004,"15min":0.502},"min":{"1min":0.28799999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.70899999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.34899999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.75800000000000001},"last":0.47699999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.50600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71799999999999997,"5min":0.746,"15min":0.746},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.39200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57199999999999995}]},{"osd":2,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65500000000000003,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56000000000000005,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60299999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.52600000000000002},"min":{"1min":0.41199999999999998,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.74099999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.54600000000000004},"min":{"1min":0.42299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69299999999999995,"5min":0.71199999999999997,"15min":0.77400000000000002},"last":0.66400000000000003}]},{"osd":4,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45000000000000001,"15min":0.46500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73699999999999999},"last":0.627},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47799999999999998,"15min":0.50800000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68600000000000005,"5min":0.72499999999999998,"15min":0.93600000000000005},"last":0.71399999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.749},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48399999999999999,"15min":0.505},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66800000000000004,"5min":0.71799999999999997,"15min":0.72399999999999998},"last":0.68400000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52900000000000003,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.64600000000000002,"5min":0.76200000000000001,"15min":0.78200000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.76500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.52800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71499999999999997,"5min":0.77100000000000002,"15min":0.84599999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73599999999999999,"5min":0.76500000000000001,"15min":0.78600000000000003},"last":0.61799999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52300000000000002,"15min":0.49199999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66800000000000004,"5min":0.72699999999999998,"15min":0.72999999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.34100000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.77400000000000002},"last":0.40699999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59499999999999997,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.40200000000000002,"15min":0.39400000000000002},"max":{"1min":0.69899999999999995,"5min":0.77900000000000003,"15min":0.80200000000000005},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59999999999999998,"15min":0.55800000000000005},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.68500000000000005,"5min":0.79900000000000004,"15min":0.82199999999999995},"last":0.49099999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68400000000000005,"5min":0.80900000000000005,"15min":1.0369999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55500000000000005,"15min":0.52900000000000003},"min":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.374},"max":{"1min":0.72599999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.505}]},{"osd":5,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44,"15min":0.46000000000000002},"min":{"1min":0.315,"5min":0.309,"15min":0.26500000000000001},"max":{"1min":0.65800000000000003,"5min":0.73099999999999998,"15min":0.75800000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45900000000000002,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.253},"max":{"1min":0.59199999999999997,"5min":0.76100000000000001,"15min":1.008},"last":0.51800000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.315,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64100000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48899999999999999,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67700000000000005,"5min":0.79200000000000004,"15min":0.82399999999999995},"last":0.58599999999999997}]},{"osd":7,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.501,"15min":0.53700000000000003},"min":{"1min":0.33500000000000002,"5min":0.312,"15min":0.30099999999999999},"max":{"1min":0.67300000000000004,"5min":0.71799999999999997,"15min":1.0069999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.36499999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69099999999999995,"5min":0.77700000000000002,"15min":0.93400000000000005},"last":0.63200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36396,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335444,"statfs":{"total":96636764160,"available":96599494656,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35999999999999999,"15min":0.32700000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.183},"max":{"1min":0.54800000000000004,"5min":0.67800000000000005,"15min":1.45},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36599999999999999,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54400000000000004,"5min":0.621,"15min":1.4099999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35299999999999998,"15min":0.32500000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46600000000000003,"5min":0.54300000000000004,"15min":1.323},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39900000000000002,"15min":0.36299999999999999},"min":{"1min":0.309,"5min":0.22800000000000001,"15min":0.218},"max":{"1min":0.58299999999999996,"5min":0.67800000000000005,"15min":0.94599999999999995},"last":0.34300000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.437,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.251},"max":{"1min":0.60699999999999998,"5min":0.68999999999999995,"15min":1.1619999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46000000000000002,"15min":0.42599999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":0.70899999999999996,"15min":1.2989999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.80600000000000005,"15min":1.337},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75900000000000001,"5min":0.83899999999999997,"15min":2.758},"last":0.57499999999999996}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.874,"15min":1.276},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.60699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.88700000000000001,"15min":1.895},"last":0.56000000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.40699999999999997},"max":{"1min":0.84199999999999997,"5min":0.872,"15min":5.4539999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82099999999999995,"5min":0.93600000000000005,"15min":2.8159999999999998},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.501,"5min":0.44600000000000001,"15min":0.41299999999999998},"max":{"1min":0.85699999999999998,"5min":0.89800000000000002,"15min":3.7400000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.443,"5min":0.443,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":5.4900000000000002},"last":0.60699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34999999999999998,"15min":0.34599999999999997},"min":{"1min":0.249,"5min":0.191,"15min":0.191},"max":{"1min":0.49099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.379},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.379},"min":{"1min":0.247,"5min":0.245,"15min":0.22},"max":{"1min":0.58199999999999996,"5min":0.64400000000000002,"15min":0.872},"last":0.36499999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.371,"15min":0.36899999999999999},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.20999999999999999},"max":{"1min":0.49399999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.30299999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.746},"last":0.40000000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41899999999999998,"15min":0.41199999999999998},"min":{"1min":0.32300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.54200000000000004,"5min":0.65200000000000002,"15min":0.67100000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39900000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.248},"max":{"1min":0.56299999999999994,"5min":0.66700000000000004,"15min":0.72499999999999998},"last":0.33300000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.62,"15min":0.629},"min":{"1min":0.47599999999999998,"5min":0.43099999999999999,"15min":0.38500000000000001},"max":{"1min":0.80100000000000005,"5min":0.87,"15min":1.216},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.61699999999999999},"min":{"1min":0.503,"5min":0.41099999999999998,"15min":0.40300000000000002},"max":{"1min":0.80000000000000004,"5min":0.89600000000000002,"15min":0.99299999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.90400000000000003,"15min":0.95699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.63900000000000001,"15min":0.63200000000000001},"min":{"1min":0.47899999999999998,"5min":0.42099999999999999,"15min":0.40300000000000002},"max":{"1min":0.81499999999999995,"5min":0.93700000000000006,"15min":1.101},"last":0.58799999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65300000000000002,"15min":0.63600000000000001},"min":{"1min":0.46400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83199999999999996,"5min":0.89100000000000001,"15min":0.97299999999999998},"last":0.622},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64800000000000002,"15min":0.64500000000000002},"min":{"1min":0.48799999999999999,"5min":0.45800000000000002,"15min":0.436},"max":{"1min":0.78600000000000003,"5min":0.89500000000000002,"15min":1.0780000000000001},"last":0.64500000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.54100000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80000000000000004,"5min":0.88500000000000001,"15min":1.155},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.41599999999999998},"max":{"1min":0.81399999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":518687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.34200000000000003},"min":{"1min":0.22500000000000001,"5min":0.214,"15min":0.19400000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.64400000000000002},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.44700000000000001},"min":{"1min":0.25800000000000001,"5min":0.218,"15min":0.20699999999999999},"max":{"1min":0.56499999999999995,"5min":0.71999999999999997,"15min":2.7069999999999999},"last":0.314}]},{"osd":2,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.36699999999999999,"15min":0.35099999999999998},"min":{"1min":0.22800000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57299999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.221},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.38500000000000001},"min":{"1min":0.28799999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.53300000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34599999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.437,"15min":0.41899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.76900000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40799999999999997,"15min":0.47299999999999998},"min":{"1min":0.27300000000000002,"5min":0.25600000000000001,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.73899999999999999,"15min":2.75},"last":0.36299999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60599999999999998,"15min":0.59299999999999997},"min":{"1min":0.44900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.95899999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83299999999999996,"5min":1.002,"15min":1.002},"last":0.45100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.0900000000000001,"15min":2.778},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.438,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88800000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.50700000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.68700000000000006},"min":{"1min":0.46700000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86299999999999999,"5min":1.026,"15min":2.8210000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.51400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93300000000000005,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.438}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84999999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65600000000000003,"15min":0.72399999999999998},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91400000000000003,"5min":0.91600000000000004,"15min":2.7989999999999999},"last":0.46500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1802240,"data_stored":1108436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"min":{"1min":0.254,"5min":0.192,"15min":0.184},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.375,"15min":0.35799999999999998},"min":{"1min":0.28199999999999997,"5min":0.23300000000000001,"15min":0.217},"max":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.22900000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.374,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.25,"15min":0.25},"max":{"1min":0.56799999999999995,"5min":0.63,"15min":0.63},"last":0.246},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.377},"min":{"1min":0.26500000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.59399999999999997},"last":0.28699999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.38700000000000001},"min":{"1min":0.27100000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.67600000000000005},"last":0.252},{"interface":"front","average":{"1min":0.496,"5min":0.44700000000000001,"15min":0.432},"min":{"1min":0.35899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.35899999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60599999999999998,"15min":0.59099999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.89100000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61699999999999999,"15min":0.59799999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91200000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.90000000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.59499999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.95999999999999996},"last":0.56799999999999995}]},{"osd":6,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65000000000000002,"15min":0.629},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.93600000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63}]},{"osd":7,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64400000000000002,"15min":0.61299999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.878,"15min":0.93600000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63600000000000001,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.46500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:49:50.683 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:49:50.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.795+0000 7fdc9be51640 1 Processor -- start 2024-03-23T09:49:50.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.796+0000 7fdc9be51640 1 -- start start 2024-03-23T09:49:50.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.796+0000 7fdc9be51640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc94160250 0x7fdc940cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:50.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.796+0000 7fdc9be51640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdc94099110 0x7fdc9406bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.796+0000 7fdc9be51640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc940cea50 con 0x7fdc9406bb30 2024-03-23T09:49:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.796+0000 7fdc9be51640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc940cf890 con 0x7fdc94099110 2024-03-23T09:49:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.796+0000 7fdc9be51640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc940d0260 con 0x7fdc94160250 2024-03-23T09:49:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.797+0000 7fdc99bed640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc94160250 0x7fdc940cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.797+0000 7fdc9a3ee640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdc94099110 0x7fdc9406bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.797+0000 7fdc993ec640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fdc9406bb30 0x7fdc94098a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:39658/0 (socket says 172.21.15.47:39658) 2024-03-23T09:49:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.797+0000 7fdc99bed640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc94160250 0x7fdc940cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:38220/0 (socket says 172.21.15.47:38220) 2024-03-23T09:49:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.797+0000 7fdc993ec640 1 -- 172.21.15.47:0/3573600983 learned_addr learned my addr 172.21.15.47:0/3573600983 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:50.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.797+0000 7fdc99bed640 1 -- 172.21.15.47:0/3573600983 >> v2:172.21.15.175:3300/0 conn(0x7fdc94099110 msgr2=0x7fdc9406bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:50.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.797+0000 7fdc99bed640 1 --2- 172.21.15.47:0/3573600983 >> v2:172.21.15.175:3300/0 conn(0x7fdc94099110 0x7fdc9406bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:50.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.797+0000 7fdc99bed640 1 -- 172.21.15.47:0/3573600983 >> v1:172.21.15.175:6789/0 conn(0x7fdc9406bb30 legacy=0x7fdc94098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:50.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.797+0000 7fdc99bed640 1 -- 172.21.15.47:0/3573600983 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc940cec80 con 0x7fdc94160250 2024-03-23T09:49:50.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.798+0000 7fdc99bed640 1 --2- 172.21.15.47:0/3573600983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc94160250 0x7fdc940cfd20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009d90 tx=0x7fdc9002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f64095863b7482 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:50.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.798+0000 7fdc98beb640 1 -- 172.21.15.47:0/3573600983 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9002cc50 con 0x7fdc94160250 2024-03-23T09:49:50.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.798+0000 7fdc98beb640 1 -- 172.21.15.47:0/3573600983 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9002cdb0 con 0x7fdc94160250 2024-03-23T09:49:50.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.798+0000 7fdc98beb640 1 -- 172.21.15.47:0/3573600983 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90034660 con 0x7fdc94160250 2024-03-23T09:49:50.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.799+0000 7fdc9be51640 1 -- 172.21.15.47:0/3573600983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc94160250 msgr2=0x7fdc940cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:50.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.799+0000 7fdc9be51640 1 --2- 172.21.15.47:0/3573600983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc94160250 0x7fdc940cfd20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009d90 tx=0x7fdc9002bea0 comp rx=0 tx=0).stop 2024-03-23T09:49:50.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.799+0000 7fdc9be51640 1 -- 172.21.15.47:0/3573600983 shutdown_connections 2024-03-23T09:49:50.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.799+0000 7fdc9be51640 1 --2- 172.21.15.47:0/3573600983 >> v2:172.21.15.175:3300/0 conn(0x7fdc94099110 0x7fdc9406bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:50.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.799+0000 7fdc9be51640 1 --2- 172.21.15.47:0/3573600983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc94160250 0x7fdc940cfd20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:50.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.799+0000 7fdc9be51640 1 -- 172.21.15.47:0/3573600983 >> 172.21.15.47:0/3573600983 conn(0x7fdc940b2140 msgr2=0x7fdc940b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:50.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.799+0000 7fdc9be51640 1 -- 172.21.15.47:0/3573600983 shutdown_connections 2024-03-23T09:49:50.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.799+0000 7fdc9be51640 1 -- 172.21.15.47:0/3573600983 wait complete. 2024-03-23T09:49:50.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.800+0000 7fdc9be51640 1 Processor -- start 2024-03-23T09:49:50.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.800+0000 7fdc9be51640 1 -- start start 2024-03-23T09:49:50.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9be51640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc9406bb30 0x7fdc9409cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:50.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9be51640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc94099110 0x7fdc94159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9be51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc94160250 0x7fdc9409b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9be51640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9409c800 con 0x7fdc94160250 2024-03-23T09:49:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9be51640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9415a150 con 0x7fdc94099110 2024-03-23T09:49:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9be51640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9409bbf0 con 0x7fdc9406bb30 2024-03-23T09:49:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9a3ee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc94160250 0x7fdc9409b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9a3ee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc94160250 0x7fdc9409b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44284/0 (socket says 172.21.15.47:44284) 2024-03-23T09:49:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9a3ee640 1 -- 172.21.15.47:0/190994174 learned_addr learned my addr 172.21.15.47:0/190994174 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc99bed640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc9406bb30 0x7fdc9409cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc993ec640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc94099110 0x7fdc94159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9a3ee640 1 -- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc9406bb30 msgr2=0x7fdc9409cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9a3ee640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc9406bb30 0x7fdc9409cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9a3ee640 1 -- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc94099110 msgr2=0x7fdc94159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9a3ee640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc94099110 0x7fdc94159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9a3ee640 1 -- 172.21.15.47:0/190994174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc90009a70 con 0x7fdc94160250 2024-03-23T09:49:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.801+0000 7fdc9a3ee640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc94160250 0x7fdc9409b280 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00e930 tx=0x7fdc8c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.802+0000 7fdc8a7fc640 1 -- 172.21.15.47:0/190994174 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c009690 con 0x7fdc94160250 2024-03-23T09:49:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.802+0000 7fdc8a7fc640 1 -- 172.21.15.47:0/190994174 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c00c980 con 0x7fdc94160250 2024-03-23T09:49:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.802+0000 7fdc9be51640 1 -- 172.21.15.47:0/190994174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9409bed0 con 0x7fdc94160250 2024-03-23T09:49:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.802+0000 7fdc8a7fc640 1 -- 172.21.15.47:0/190994174 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c00cd30 con 0x7fdc94160250 2024-03-23T09:49:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.802+0000 7fdc9be51640 1 -- 172.21.15.47:0/190994174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9409c490 con 0x7fdc94160250 2024-03-23T09:49:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.803+0000 7fdc9be51640 1 -- 172.21.15.47:0/190994174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc94066be0 con 0x7fdc94160250 2024-03-23T09:49:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.803+0000 7fdc8a7fc640 1 -- 172.21.15.47:0/190994174 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdc8c013470 con 0x7fdc94160250 2024-03-23T09:49:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.804+0000 7fdc8a7fc640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdc7806ac30 0x7fdc7806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.804+0000 7fdc8a7fc640 1 -- 172.21.15.47:0/190994174 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc8c089480 con 0x7fdc94160250 2024-03-23T09:49:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.804+0000 7fdc99bed640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdc7806ac30 0x7fdc7806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.804+0000 7fdc99bed640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdc7806ac30 0x7fdc7806d0f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdc90036000 tx=0x7fdc90030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.808+0000 7fdc8a7fc640 1 -- 172.21.15.47:0/190994174 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdc8c054860 con 0x7fdc94160250 2024-03-23T09:49:50.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.988+0000 7fdc9be51640 1 -- 172.21.15.47:0/190994174 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc940cf520 con 0x7fdc7806ac30 2024-03-23T09:49:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.996+0000 7fdc8a7fc640 1 -- 172.21.15.47:0/190994174 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289880 (secure 0 0 0) 0x7fdc90043070 con 0x7fdc7806ac30 2024-03-23T09:49:50.999 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:51.001 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:49:51.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.999+0000 7fdc9be51640 1 -- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdc7806ac30 msgr2=0x7fdc7806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.999+0000 7fdc9be51640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdc7806ac30 0x7fdc7806d0f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdc90036000 tx=0x7fdc90030000 comp rx=0 tx=0).stop 2024-03-23T09:49:51.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.999+0000 7fdc9be51640 1 -- 172.21.15.47:0/190994174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc94160250 msgr2=0x7fdc9409b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.999+0000 7fdc9be51640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc94160250 0x7fdc9409b280 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00e930 tx=0x7fdc8c00edf0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.999+0000 7fdc9be51640 1 -- 172.21.15.47:0/190994174 shutdown_connections 2024-03-23T09:49:51.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.999+0000 7fdc9be51640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdc7806ac30 0x7fdc7806d0f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.999+0000 7fdc9be51640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc94160250 0x7fdc9409b280 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.999+0000 7fdc9be51640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc94099110 0x7fdc94159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.999+0000 7fdc9be51640 1 --2- 172.21.15.47:0/190994174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc9406bb30 0x7fdc9409cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.999+0000 7fdc9be51640 1 -- 172.21.15.47:0/190994174 >> 172.21.15.47:0/190994174 conn(0x7fdc940b2140 msgr2=0x7fdc940c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:51.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.999+0000 7fdc9be51640 1 -- 172.21.15.47:0/190994174 shutdown_connections 2024-03-23T09:49:51.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:50.999+0000 7fdc9be51640 1 -- 172.21.15.47:0/190994174 wait complete. 2024-03-23T09:49:51.010 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-03-23T09:49:50.471961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":272252,"kb_used_data":10976,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754702468,"statfs":{"total":773094113280,"available":772815327232,"internally_reserved":0,"allocated":11239424,"data_stored":5488518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":267374136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.597455"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.755454+0000","last_change":"2024-03-23T09:49:43.269200+0000","last_active":"2024-03-23T09:49:44.755454+0000","last_peered":"2024-03-23T09:49:44.755454+0000","last_clean":"2024-03-23T09:49:44.755454+0000","last_became_active":"2024-03-23T09:49:43.268958+0000","last_became_peered":"2024-03-23T09:49:43.268958+0000","last_unstale":"2024-03-23T09:49:44.755454+0000","last_undegraded":"2024-03-23T09:49:44.755454+0000","last_fullsized":"2024-03-23T09:49:44.755454+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:13.736292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.552633+0000","last_change":"2024-03-23T09:49:43.278285+0000","last_active":"2024-03-23T09:49:44.552633+0000","last_peered":"2024-03-23T09:49:44.552633+0000","last_clean":"2024-03-23T09:49:44.552633+0000","last_became_active":"2024-03-23T09:49:43.277983+0000","last_became_peered":"2024-03-23T09:49:43.277983+0000","last_unstale":"2024-03-23T09:49:44.552633+0000","last_undegraded":"2024-03-23T09:49:44.552633+0000","last_fullsized":"2024-03-23T09:49:44.552633+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:15:53.067159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.755411+0000","last_change":"2024-03-23T09:49:43.270541+0000","last_active":"2024-03-23T09:49:44.755411+0000","last_peered":"2024-03-23T09:49:44.755411+0000","last_clean":"2024-03-23T09:49:44.755411+0000","last_became_active":"2024-03-23T09:49:43.270218+0000","last_became_peered":"2024-03-23T09:49:43.270218+0000","last_unstale":"2024-03-23T09:49:44.755411+0000","last_undegraded":"2024-03-23T09:49:44.755411+0000","last_fullsized":"2024-03-23T09:49:44.755411+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:44:24.587964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.260183+0000","last_change":"2024-03-23T09:49:43.267517+0000","last_active":"2024-03-23T09:49:44.260183+0000","last_peered":"2024-03-23T09:49:44.260183+0000","last_clean":"2024-03-23T09:49:44.260183+0000","last_became_active":"2024-03-23T09:49:43.267243+0000","last_became_peered":"2024-03-23T09:49:43.267243+0000","last_unstale":"2024-03-23T09:49:44.260183+0000","last_undegraded":"2024-03-23T09:49:44.260183+0000","last_fullsized":"2024-03-23T09:49:44.260183+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:12:20.855071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.526642+0000","last_change":"2024-03-23T09:49:43.275740+0000","last_active":"2024-03-23T09:49:45.526642+0000","last_peered":"2024-03-23T09:49:45.526642+0000","last_clean":"2024-03-23T09:49:45.526642+0000","last_became_active":"2024-03-23T09:49:43.275437+0000","last_became_peered":"2024-03-23T09:49:43.275437+0000","last_unstale":"2024-03-23T09:49:45.526642+0000","last_undegraded":"2024-03-23T09:49:45.526642+0000","last_fullsized":"2024-03-23T09:49:45.526642+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:28:34.243110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265328+0000","last_change":"2024-03-23T09:49:43.269257+0000","last_active":"2024-03-23T09:49:44.265328+0000","last_peered":"2024-03-23T09:49:44.265328+0000","last_clean":"2024-03-23T09:49:44.265328+0000","last_became_active":"2024-03-23T09:49:43.268994+0000","last_became_peered":"2024-03-23T09:49:43.268994+0000","last_unstale":"2024-03-23T09:49:44.265328+0000","last_undegraded":"2024-03-23T09:49:44.265328+0000","last_fullsized":"2024-03-23T09:49:44.265328+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:37:40.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265285+0000","last_change":"2024-03-23T09:49:43.269389+0000","last_active":"2024-03-23T09:49:44.265285+0000","last_peered":"2024-03-23T09:49:44.265285+0000","last_clean":"2024-03-23T09:49:44.265285+0000","last_became_active":"2024-03-23T09:49:43.269124+0000","last_became_peered":"2024-03-23T09:49:43.269124+0000","last_unstale":"2024-03-23T09:49:44.265285+0000","last_undegraded":"2024-03-23T09:49:44.265285+0000","last_fullsized":"2024-03-23T09:49:44.265285+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:28:34.243110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.552531+0000","last_change":"2024-03-23T09:49:43.272572+0000","last_active":"2024-03-23T09:49:44.552531+0000","last_peered":"2024-03-23T09:49:44.552531+0000","last_clean":"2024-03-23T09:49:44.552531+0000","last_became_active":"2024-03-23T09:49:43.272324+0000","last_became_peered":"2024-03-23T09:49:43.272324+0000","last_unstale":"2024-03-23T09:49:44.552531+0000","last_undegraded":"2024-03-23T09:49:44.552531+0000","last_fullsized":"2024-03-23T09:49:44.552531+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:09.721273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265687+0000","last_change":"2024-03-23T09:49:43.262055+0000","last_active":"2024-03-23T09:49:44.265687+0000","last_peered":"2024-03-23T09:49:44.265687+0000","last_clean":"2024-03-23T09:49:44.265687+0000","last_became_active":"2024-03-23T09:49:43.261774+0000","last_became_peered":"2024-03-23T09:49:43.261774+0000","last_unstale":"2024-03-23T09:49:44.265687+0000","last_undegraded":"2024-03-23T09:49:44.265687+0000","last_fullsized":"2024-03-23T09:49:44.265687+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:28:34.243110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.526741+0000","last_change":"2024-03-23T09:49:43.268204+0000","last_active":"2024-03-23T09:49:45.526741+0000","last_peered":"2024-03-23T09:49:45.526741+0000","last_clean":"2024-03-23T09:49:45.526741+0000","last_became_active":"2024-03-23T09:49:43.267377+0000","last_became_peered":"2024-03-23T09:49:43.267377+0000","last_unstale":"2024-03-23T09:49:45.526741+0000","last_undegraded":"2024-03-23T09:49:45.526741+0000","last_fullsized":"2024-03-23T09:49:45.526741+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:07:02.577248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265458+0000","last_change":"2024-03-23T09:49:43.267281+0000","last_active":"2024-03-23T09:49:44.265458+0000","last_peered":"2024-03-23T09:49:44.265458+0000","last_clean":"2024-03-23T09:49:44.265458+0000","last_became_active":"2024-03-23T09:49:43.266661+0000","last_became_peered":"2024-03-23T09:49:43.266661+0000","last_unstale":"2024-03-23T09:49:44.265458+0000","last_undegraded":"2024-03-23T09:49:44.265458+0000","last_fullsized":"2024-03-23T09:49:44.265458+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:12:49.012393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265842+0000","last_change":"2024-03-23T09:49:43.266510+0000","last_active":"2024-03-23T09:49:44.265842+0000","last_peered":"2024-03-23T09:49:44.265842+0000","last_clean":"2024-03-23T09:49:44.265842+0000","last_became_active":"2024-03-23T09:49:43.266252+0000","last_became_peered":"2024-03-23T09:49:43.266252+0000","last_unstale":"2024-03-23T09:49:44.265842+0000","last_undegraded":"2024-03-23T09:49:44.265842+0000","last_fullsized":"2024-03-23T09:49:44.265842+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:36:46.470298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.260111+0000","last_change":"2024-03-23T09:49:43.272715+0000","last_active":"2024-03-23T09:49:44.260111+0000","last_peered":"2024-03-23T09:49:44.260111+0000","last_clean":"2024-03-23T09:49:44.260111+0000","last_became_active":"2024-03-23T09:49:43.271756+0000","last_became_peered":"2024-03-23T09:49:43.271756+0000","last_unstale":"2024-03-23T09:49:44.260111+0000","last_undegraded":"2024-03-23T09:49:44.260111+0000","last_fullsized":"2024-03-23T09:49:44.260111+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:20:54.940535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.839371+0000","last_change":"2024-03-23T09:49:43.264677+0000","last_active":"2024-03-23T09:49:45.839371+0000","last_peered":"2024-03-23T09:49:45.839371+0000","last_clean":"2024-03-23T09:49:45.839371+0000","last_became_active":"2024-03-23T09:49:43.264227+0000","last_became_peered":"2024-03-23T09:49:43.264227+0000","last_unstale":"2024-03-23T09:49:45.839371+0000","last_undegraded":"2024-03-23T09:49:45.839371+0000","last_fullsized":"2024-03-23T09:49:45.839371+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:57:55.092936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.259895+0000","last_change":"2024-03-23T09:49:43.261964+0000","last_active":"2024-03-23T09:49:44.259895+0000","last_peered":"2024-03-23T09:49:44.259895+0000","last_clean":"2024-03-23T09:49:44.259895+0000","last_became_active":"2024-03-23T09:49:43.261731+0000","last_became_peered":"2024-03-23T09:49:43.261731+0000","last_unstale":"2024-03-23T09:49:44.259895+0000","last_undegraded":"2024-03-23T09:49:44.259895+0000","last_fullsized":"2024-03-23T09:49:44.259895+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:14:48.608570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265954+0000","last_change":"2024-03-23T09:49:43.276455+0000","last_active":"2024-03-23T09:49:44.265954+0000","last_peered":"2024-03-23T09:49:44.265954+0000","last_clean":"2024-03-23T09:49:44.265954+0000","last_became_active":"2024-03-23T09:49:43.275898+0000","last_became_peered":"2024-03-23T09:49:43.275898+0000","last_unstale":"2024-03-23T09:49:44.265954+0000","last_undegraded":"2024-03-23T09:49:44.265954+0000","last_fullsized":"2024-03-23T09:49:44.265954+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:28:52.260374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.552705+0000","last_change":"2024-03-23T09:49:43.276517+0000","last_active":"2024-03-23T09:49:44.552705+0000","last_peered":"2024-03-23T09:49:44.552705+0000","last_clean":"2024-03-23T09:49:44.552705+0000","last_became_active":"2024-03-23T09:49:43.276142+0000","last_became_peered":"2024-03-23T09:49:43.276142+0000","last_unstale":"2024-03-23T09:49:44.552705+0000","last_undegraded":"2024-03-23T09:49:44.552705+0000","last_fullsized":"2024-03-23T09:49:44.552705+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:20:54.940535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.260217+0000","last_change":"2024-03-23T09:49:43.261445+0000","last_active":"2024-03-23T09:49:44.260217+0000","last_peered":"2024-03-23T09:49:44.260217+0000","last_clean":"2024-03-23T09:49:44.260217+0000","last_became_active":"2024-03-23T09:49:43.261137+0000","last_became_peered":"2024-03-23T09:49:43.261137+0000","last_unstale":"2024-03-23T09:49:44.260217+0000","last_undegraded":"2024-03-23T09:49:44.260217+0000","last_fullsized":"2024-03-23T09:49:44.260217+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:59:01.417727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.839651+0000","last_change":"2024-03-23T09:49:43.261513+0000","last_active":"2024-03-23T09:49:45.839651+0000","last_peered":"2024-03-23T09:49:45.839651+0000","last_clean":"2024-03-23T09:49:45.839651+0000","last_became_active":"2024-03-23T09:49:43.261259+0000","last_became_peered":"2024-03-23T09:49:43.261259+0000","last_unstale":"2024-03-23T09:49:45.839651+0000","last_undegraded":"2024-03-23T09:49:45.839651+0000","last_fullsized":"2024-03-23T09:49:45.839651+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:08:27.528039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265960+0000","last_change":"2024-03-23T09:49:43.274035+0000","last_active":"2024-03-23T09:49:44.265960+0000","last_peered":"2024-03-23T09:49:44.265960+0000","last_clean":"2024-03-23T09:49:44.265960+0000","last_became_active":"2024-03-23T09:49:43.273234+0000","last_became_peered":"2024-03-23T09:49:43.273234+0000","last_unstale":"2024-03-23T09:49:44.265960+0000","last_undegraded":"2024-03-23T09:49:44.265960+0000","last_fullsized":"2024-03-23T09:49:44.265960+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:37:40.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.839599+0000","last_change":"2024-03-23T09:49:43.261548+0000","last_active":"2024-03-23T09:49:45.839599+0000","last_peered":"2024-03-23T09:49:45.839599+0000","last_clean":"2024-03-23T09:49:45.839599+0000","last_became_active":"2024-03-23T09:49:43.261288+0000","last_became_peered":"2024-03-23T09:49:43.261288+0000","last_unstale":"2024-03-23T09:49:45.839599+0000","last_undegraded":"2024-03-23T09:49:45.839599+0000","last_fullsized":"2024-03-23T09:49:45.839599+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:29:11.223654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.552601+0000","last_change":"2024-03-23T09:49:43.274513+0000","last_active":"2024-03-23T09:49:44.552601+0000","last_peered":"2024-03-23T09:49:44.552601+0000","last_clean":"2024-03-23T09:49:44.552601+0000","last_became_active":"2024-03-23T09:49:43.272834+0000","last_became_peered":"2024-03-23T09:49:43.272834+0000","last_unstale":"2024-03-23T09:49:44.552601+0000","last_undegraded":"2024-03-23T09:49:44.552601+0000","last_fullsized":"2024-03-23T09:49:44.552601+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:07:02.577248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.552520+0000","last_change":"2024-03-23T09:49:43.272384+0000","last_active":"2024-03-23T09:49:44.552520+0000","last_peered":"2024-03-23T09:49:44.552520+0000","last_clean":"2024-03-23T09:49:44.552520+0000","last_became_active":"2024-03-23T09:49:43.272071+0000","last_became_peered":"2024-03-23T09:49:43.272071+0000","last_unstale":"2024-03-23T09:49:44.552520+0000","last_undegraded":"2024-03-23T09:49:44.552520+0000","last_fullsized":"2024-03-23T09:49:44.552520+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:12:20.855071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.266052+0000","last_change":"2024-03-23T09:49:43.262331+0000","last_active":"2024-03-23T09:49:44.266052+0000","last_peered":"2024-03-23T09:49:44.266052+0000","last_clean":"2024-03-23T09:49:44.266052+0000","last_became_active":"2024-03-23T09:49:43.261792+0000","last_became_peered":"2024-03-23T09:49:43.261792+0000","last_unstale":"2024-03-23T09:49:44.266052+0000","last_undegraded":"2024-03-23T09:49:44.266052+0000","last_fullsized":"2024-03-23T09:49:44.266052+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:03:48.447165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.526849+0000","last_change":"2024-03-23T09:49:43.278149+0000","last_active":"2024-03-23T09:49:45.526849+0000","last_peered":"2024-03-23T09:49:45.526849+0000","last_clean":"2024-03-23T09:49:45.526849+0000","last_became_active":"2024-03-23T09:49:43.277882+0000","last_became_peered":"2024-03-23T09:49:43.277882+0000","last_unstale":"2024-03-23T09:49:45.526849+0000","last_undegraded":"2024-03-23T09:49:45.526849+0000","last_fullsized":"2024-03-23T09:49:45.526849+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:20:54.940535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.552811+0000","last_change":"2024-03-23T09:49:43.272380+0000","last_active":"2024-03-23T09:49:44.552811+0000","last_peered":"2024-03-23T09:49:44.552811+0000","last_clean":"2024-03-23T09:49:44.552811+0000","last_became_active":"2024-03-23T09:49:43.272077+0000","last_became_peered":"2024-03-23T09:49:43.272077+0000","last_unstale":"2024-03-23T09:49:44.552811+0000","last_undegraded":"2024-03-23T09:49:44.552811+0000","last_fullsized":"2024-03-23T09:49:44.552811+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:59:01.417727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265557+0000","last_change":"2024-03-23T09:49:43.263241+0000","last_active":"2024-03-23T09:49:44.265557+0000","last_peered":"2024-03-23T09:49:44.265557+0000","last_clean":"2024-03-23T09:49:44.265557+0000","last_became_active":"2024-03-23T09:49:43.262678+0000","last_became_peered":"2024-03-23T09:49:43.262678+0000","last_unstale":"2024-03-23T09:49:44.265557+0000","last_undegraded":"2024-03-23T09:49:44.265557+0000","last_fullsized":"2024-03-23T09:49:44.265557+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:12:20.855071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.526785+0000","last_change":"2024-03-23T09:49:43.275748+0000","last_active":"2024-03-23T09:49:45.526785+0000","last_peered":"2024-03-23T09:49:45.526785+0000","last_clean":"2024-03-23T09:49:45.526785+0000","last_became_active":"2024-03-23T09:49:43.275478+0000","last_became_peered":"2024-03-23T09:49:43.275478+0000","last_unstale":"2024-03-23T09:49:45.526785+0000","last_undegraded":"2024-03-23T09:49:45.526785+0000","last_fullsized":"2024-03-23T09:49:45.526785+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:36:46.470298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.552727+0000","last_change":"2024-03-23T09:49:43.275219+0000","last_active":"2024-03-23T09:49:44.552727+0000","last_peered":"2024-03-23T09:49:44.552727+0000","last_clean":"2024-03-23T09:49:44.552727+0000","last_became_active":"2024-03-23T09:49:43.272879+0000","last_became_peered":"2024-03-23T09:49:43.272879+0000","last_unstale":"2024-03-23T09:49:44.552727+0000","last_undegraded":"2024-03-23T09:49:44.552727+0000","last_fullsized":"2024-03-23T09:49:44.552727+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:03:04.313653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.839467+0000","last_change":"2024-03-23T09:49:43.262615+0000","last_active":"2024-03-23T09:49:45.839467+0000","last_peered":"2024-03-23T09:49:45.839467+0000","last_clean":"2024-03-23T09:49:45.839467+0000","last_became_active":"2024-03-23T09:49:43.262282+0000","last_became_peered":"2024-03-23T09:49:43.262282+0000","last_unstale":"2024-03-23T09:49:45.839467+0000","last_undegraded":"2024-03-23T09:49:45.839467+0000","last_fullsized":"2024-03-23T09:49:45.839467+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:33:11.460484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.259991+0000","last_change":"2024-03-23T09:49:43.271050+0000","last_active":"2024-03-23T09:49:44.259991+0000","last_peered":"2024-03-23T09:49:44.259991+0000","last_clean":"2024-03-23T09:49:44.259991+0000","last_became_active":"2024-03-23T09:49:43.270746+0000","last_became_peered":"2024-03-23T09:49:43.270746+0000","last_unstale":"2024-03-23T09:49:44.259991+0000","last_undegraded":"2024-03-23T09:49:44.259991+0000","last_fullsized":"2024-03-23T09:49:44.259991+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:07:02.577248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.755210+0000","last_change":"2024-03-23T09:49:43.279313+0000","last_active":"2024-03-23T09:49:44.755210+0000","last_peered":"2024-03-23T09:49:44.755210+0000","last_clean":"2024-03-23T09:49:44.755210+0000","last_became_active":"2024-03-23T09:49:43.279041+0000","last_became_peered":"2024-03-23T09:49:43.279041+0000","last_unstale":"2024-03-23T09:49:44.755210+0000","last_undegraded":"2024-03-23T09:49:44.755210+0000","last_fullsized":"2024-03-23T09:49:44.755210+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:52:26.388494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.839812+0000","last_change":"2024-03-23T09:49:43.257293+0000","last_active":"2024-03-23T09:49:45.839812+0000","last_peered":"2024-03-23T09:49:45.839812+0000","last_clean":"2024-03-23T09:49:45.839812+0000","last_became_active":"2024-03-23T09:49:43.257008+0000","last_became_peered":"2024-03-23T09:49:43.257008+0000","last_unstale":"2024-03-23T09:49:45.839812+0000","last_undegraded":"2024-03-23T09:49:45.839812+0000","last_fullsized":"2024-03-23T09:49:45.839812+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:44:24.587964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"100'4","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.280084+0000","last_change":"2024-03-23T09:49:42.892014+0000","last_active":"2024-03-23T09:49:43.280084+0000","last_peered":"2024-03-23T09:49:43.280084+0000","last_clean":"2024-03-23T09:49:43.280084+0000","last_became_active":"2024-03-23T09:49:42.891739+0000","last_became_peered":"2024-03-23T09:49:42.891739+0000","last_unstale":"2024-03-23T09:49:43.280084+0000","last_undegraded":"2024-03-23T09:49:43.280084+0000","last_fullsized":"2024-03-23T09:49:43.280084+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:25:47.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.260337+0000","last_change":"2024-03-23T09:49:43.271069+0000","last_active":"2024-03-23T09:49:44.260337+0000","last_peered":"2024-03-23T09:49:44.260337+0000","last_clean":"2024-03-23T09:49:44.260337+0000","last_became_active":"2024-03-23T09:49:43.270831+0000","last_became_peered":"2024-03-23T09:49:43.270831+0000","last_unstale":"2024-03-23T09:49:44.260337+0000","last_undegraded":"2024-03-23T09:49:44.260337+0000","last_fullsized":"2024-03-23T09:49:44.260337+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:28:52.360503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"100'1","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.264436+0000","last_change":"2024-03-23T09:49:42.892006+0000","last_active":"2024-03-23T09:49:43.264436+0000","last_peered":"2024-03-23T09:49:43.264436+0000","last_clean":"2024-03-23T09:49:43.264436+0000","last_became_active":"2024-03-23T09:49:42.891789+0000","last_became_peered":"2024-03-23T09:49:42.891789+0000","last_unstale":"2024-03-23T09:49:43.264436+0000","last_undegraded":"2024-03-23T09:49:43.264436+0000","last_fullsized":"2024-03-23T09:49:43.264436+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.755352+0000","last_change":"2024-03-23T09:49:43.266593+0000","last_active":"2024-03-23T09:49:44.755352+0000","last_peered":"2024-03-23T09:49:44.755352+0000","last_clean":"2024-03-23T09:49:44.755352+0000","last_became_active":"2024-03-23T09:49:43.266244+0000","last_became_peered":"2024-03-23T09:49:43.266244+0000","last_unstale":"2024-03-23T09:49:44.755352+0000","last_undegraded":"2024-03-23T09:49:44.755352+0000","last_fullsized":"2024-03-23T09:49:44.755352+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:41:41.706246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"100'2","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.275407+0000","last_change":"2024-03-23T09:49:42.897531+0000","last_active":"2024-03-23T09:49:43.275407+0000","last_peered":"2024-03-23T09:49:43.275407+0000","last_clean":"2024-03-23T09:49:43.275407+0000","last_became_active":"2024-03-23T09:49:42.897234+0000","last_became_peered":"2024-03-23T09:49:42.897234+0000","last_unstale":"2024-03-23T09:49:43.275407+0000","last_undegraded":"2024-03-23T09:49:43.275407+0000","last_fullsized":"2024-03-23T09:49:43.275407+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:56:58.937373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.266263+0000","last_change":"2024-03-23T09:49:43.262900+0000","last_active":"2024-03-23T09:49:44.266263+0000","last_peered":"2024-03-23T09:49:44.266263+0000","last_clean":"2024-03-23T09:49:44.266263+0000","last_became_active":"2024-03-23T09:49:43.262637+0000","last_became_peered":"2024-03-23T09:49:43.262637+0000","last_unstale":"2024-03-23T09:49:44.266263+0000","last_undegraded":"2024-03-23T09:49:44.266263+0000","last_fullsized":"2024-03-23T09:49:44.266263+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:09.721273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"100'2","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.278843+0000","last_change":"2024-03-23T09:49:42.891074+0000","last_active":"2024-03-23T09:49:43.278843+0000","last_peered":"2024-03-23T09:49:43.278843+0000","last_clean":"2024-03-23T09:49:43.278843+0000","last_became_active":"2024-03-23T09:49:42.890799+0000","last_became_peered":"2024-03-23T09:49:42.890799+0000","last_unstale":"2024-03-23T09:49:43.278843+0000","last_undegraded":"2024-03-23T09:49:43.278843+0000","last_fullsized":"2024-03-23T09:49:43.278843+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.927315+0000","last_change":"2024-03-23T09:49:43.271080+0000","last_active":"2024-03-23T09:49:44.927315+0000","last_peered":"2024-03-23T09:49:44.927315+0000","last_clean":"2024-03-23T09:49:44.927315+0000","last_became_active":"2024-03-23T09:49:43.270803+0000","last_became_peered":"2024-03-23T09:49:43.270803+0000","last_unstale":"2024-03-23T09:49:44.927315+0000","last_undegraded":"2024-03-23T09:49:44.927315+0000","last_fullsized":"2024-03-23T09:49:44.927315+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:08:27.528039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"100'1","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.265411+0000","last_change":"2024-03-23T09:49:42.895046+0000","last_active":"2024-03-23T09:49:43.265411+0000","last_peered":"2024-03-23T09:49:43.265411+0000","last_clean":"2024-03-23T09:49:43.265411+0000","last_became_active":"2024-03-23T09:49:42.894556+0000","last_became_peered":"2024-03-23T09:49:42.894556+0000","last_unstale":"2024-03-23T09:49:43.265411+0000","last_undegraded":"2024-03-23T09:49:43.265411+0000","last_fullsized":"2024-03-23T09:49:43.265411+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:09:41.124566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265499+0000","last_change":"2024-03-23T09:49:43.264274+0000","last_active":"2024-03-23T09:49:44.265499+0000","last_peered":"2024-03-23T09:49:44.265499+0000","last_clean":"2024-03-23T09:49:44.265499+0000","last_became_active":"2024-03-23T09:49:43.264022+0000","last_became_peered":"2024-03-23T09:49:43.264022+0000","last_unstale":"2024-03-23T09:49:44.265499+0000","last_undegraded":"2024-03-23T09:49:44.265499+0000","last_fullsized":"2024-03-23T09:49:44.265499+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:03.784913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"100'1","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.273022+0000","last_change":"2024-03-23T09:49:42.893727+0000","last_active":"2024-03-23T09:49:43.273022+0000","last_peered":"2024-03-23T09:49:43.273022+0000","last_clean":"2024-03-23T09:49:43.273022+0000","last_became_active":"2024-03-23T09:49:42.893453+0000","last_became_peered":"2024-03-23T09:49:42.893453+0000","last_unstale":"2024-03-23T09:49:43.273022+0000","last_undegraded":"2024-03-23T09:49:43.273022+0000","last_fullsized":"2024-03-23T09:49:43.273022+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:04:23.374836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.259900+0000","last_change":"2024-03-23T09:49:43.262064+0000","last_active":"2024-03-23T09:49:44.259900+0000","last_peered":"2024-03-23T09:49:44.259900+0000","last_clean":"2024-03-23T09:49:44.259900+0000","last_became_active":"2024-03-23T09:49:43.261797+0000","last_became_peered":"2024-03-23T09:49:43.261797+0000","last_unstale":"2024-03-23T09:49:44.259900+0000","last_undegraded":"2024-03-23T09:49:44.259900+0000","last_fullsized":"2024-03-23T09:49:44.259900+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:15:53.067159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.262381+0000","last_change":"2024-03-23T09:49:42.888788+0000","last_active":"2024-03-23T09:49:43.262381+0000","last_peered":"2024-03-23T09:49:43.262381+0000","last_clean":"2024-03-23T09:49:43.262381+0000","last_became_active":"2024-03-23T09:49:42.888517+0000","last_became_peered":"2024-03-23T09:49:42.888517+0000","last_unstale":"2024-03-23T09:49:43.262381+0000","last_undegraded":"2024-03-23T09:49:43.262381+0000","last_fullsized":"2024-03-23T09:49:43.262381+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265434+0000","last_change":"2024-03-23T09:49:43.263488+0000","last_active":"2024-03-23T09:49:44.265434+0000","last_peered":"2024-03-23T09:49:44.265434+0000","last_clean":"2024-03-23T09:49:44.265434+0000","last_became_active":"2024-03-23T09:49:43.263222+0000","last_became_peered":"2024-03-23T09:49:43.263222+0000","last_unstale":"2024-03-23T09:49:44.265434+0000","last_undegraded":"2024-03-23T09:49:44.265434+0000","last_fullsized":"2024-03-23T09:49:44.265434+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:18:36.210200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.257208+0000","last_change":"2024-03-23T09:49:42.894713+0000","last_active":"2024-03-23T09:49:43.257208+0000","last_peered":"2024-03-23T09:49:43.257208+0000","last_clean":"2024-03-23T09:49:43.257208+0000","last_became_active":"2024-03-23T09:49:42.894405+0000","last_became_peered":"2024-03-23T09:49:42.894405+0000","last_unstale":"2024-03-23T09:49:43.257208+0000","last_undegraded":"2024-03-23T09:49:43.257208+0000","last_fullsized":"2024-03-23T09:49:43.257208+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:29:10.221541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.552820+0000","last_change":"2024-03-23T09:49:43.275195+0000","last_active":"2024-03-23T09:49:44.552820+0000","last_peered":"2024-03-23T09:49:44.552820+0000","last_clean":"2024-03-23T09:49:44.552820+0000","last_became_active":"2024-03-23T09:49:43.272887+0000","last_became_peered":"2024-03-23T09:49:43.272887+0000","last_unstale":"2024-03-23T09:49:44.552820+0000","last_undegraded":"2024-03-23T09:49:44.552820+0000","last_fullsized":"2024-03-23T09:49:44.552820+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:14:48.608570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.259392+0000","last_change":"2024-03-23T09:49:42.894059+0000","last_active":"2024-03-23T09:49:43.259392+0000","last_peered":"2024-03-23T09:49:43.259392+0000","last_clean":"2024-03-23T09:49:43.259392+0000","last_became_active":"2024-03-23T09:49:42.893263+0000","last_became_peered":"2024-03-23T09:49:42.893263+0000","last_unstale":"2024-03-23T09:49:43.259392+0000","last_undegraded":"2024-03-23T09:49:43.259392+0000","last_fullsized":"2024-03-23T09:49:43.259392+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:39.896584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265725+0000","last_change":"2024-03-23T09:49:43.265115+0000","last_active":"2024-03-23T09:49:44.265725+0000","last_peered":"2024-03-23T09:49:44.265725+0000","last_clean":"2024-03-23T09:49:44.265725+0000","last_became_active":"2024-03-23T09:49:43.264848+0000","last_became_peered":"2024-03-23T09:49:43.264848+0000","last_unstale":"2024-03-23T09:49:44.265725+0000","last_undegraded":"2024-03-23T09:49:44.265725+0000","last_fullsized":"2024-03-23T09:49:44.265725+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:03.784913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.256217+0000","last_change":"2024-03-23T09:49:42.888001+0000","last_active":"2024-03-23T09:49:43.256217+0000","last_peered":"2024-03-23T09:49:43.256217+0000","last_clean":"2024-03-23T09:49:43.256217+0000","last_became_active":"2024-03-23T09:49:42.887720+0000","last_became_peered":"2024-03-23T09:49:42.887720+0000","last_unstale":"2024-03-23T09:49:43.256217+0000","last_undegraded":"2024-03-23T09:49:43.256217+0000","last_fullsized":"2024-03-23T09:49:43.256217+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:21:22.073815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.526780+0000","last_change":"2024-03-23T09:49:43.279649+0000","last_active":"2024-03-23T09:49:45.526780+0000","last_peered":"2024-03-23T09:49:45.526780+0000","last_clean":"2024-03-23T09:49:45.526780+0000","last_became_active":"2024-03-23T09:49:43.279374+0000","last_became_peered":"2024-03-23T09:49:43.279374+0000","last_unstale":"2024-03-23T09:49:45.526780+0000","last_undegraded":"2024-03-23T09:49:45.526780+0000","last_fullsized":"2024-03-23T09:49:45.526780+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:12:20.855071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"100'3","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.267026+0000","last_change":"2024-03-23T09:49:42.887836+0000","last_active":"2024-03-23T09:49:43.267026+0000","last_peered":"2024-03-23T09:49:43.267026+0000","last_clean":"2024-03-23T09:49:43.267026+0000","last_became_active":"2024-03-23T09:49:42.887568+0000","last_became_peered":"2024-03-23T09:49:42.887568+0000","last_unstale":"2024-03-23T09:49:43.267026+0000","last_undegraded":"2024-03-23T09:49:43.267026+0000","last_fullsized":"2024-03-23T09:49:43.267026+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265653+0000","last_change":"2024-03-23T09:49:43.276519+0000","last_active":"2024-03-23T09:49:44.265653+0000","last_peered":"2024-03-23T09:49:44.265653+0000","last_clean":"2024-03-23T09:49:44.265653+0000","last_became_active":"2024-03-23T09:49:43.276171+0000","last_became_peered":"2024-03-23T09:49:43.276171+0000","last_unstale":"2024-03-23T09:49:44.265653+0000","last_undegraded":"2024-03-23T09:49:44.265653+0000","last_fullsized":"2024-03-23T09:49:44.265653+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:36:46.470298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"100'6","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.282513+0000","last_change":"2024-03-23T09:49:42.887945+0000","last_active":"2024-03-23T09:49:43.282513+0000","last_peered":"2024-03-23T09:49:43.282513+0000","last_clean":"2024-03-23T09:49:43.282513+0000","last_became_active":"2024-03-23T09:49:42.887664+0000","last_became_peered":"2024-03-23T09:49:42.887664+0000","last_unstale":"2024-03-23T09:49:43.282513+0000","last_undegraded":"2024-03-23T09:49:43.282513+0000","last_fullsized":"2024-03-23T09:49:43.282513+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.259999+0000","last_change":"2024-03-23T09:49:43.275190+0000","last_active":"2024-03-23T09:49:44.259999+0000","last_peered":"2024-03-23T09:49:44.259999+0000","last_clean":"2024-03-23T09:49:44.259999+0000","last_became_active":"2024-03-23T09:49:43.274918+0000","last_became_peered":"2024-03-23T09:49:43.274918+0000","last_unstale":"2024-03-23T09:49:44.259999+0000","last_undegraded":"2024-03-23T09:49:44.259999+0000","last_fullsized":"2024-03-23T09:49:44.259999+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:03:48.447165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"100'2","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.267942+0000","last_change":"2024-03-23T09:49:42.890974+0000","last_active":"2024-03-23T09:49:43.267942+0000","last_peered":"2024-03-23T09:49:43.267942+0000","last_clean":"2024-03-23T09:49:43.267942+0000","last_became_active":"2024-03-23T09:49:42.890719+0000","last_became_peered":"2024-03-23T09:49:42.890719+0000","last_unstale":"2024-03-23T09:49:43.267942+0000","last_undegraded":"2024-03-23T09:49:43.267942+0000","last_fullsized":"2024-03-23T09:49:43.267942+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:29.138334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.755359+0000","last_change":"2024-03-23T09:49:43.264771+0000","last_active":"2024-03-23T09:49:44.755359+0000","last_peered":"2024-03-23T09:49:44.755359+0000","last_clean":"2024-03-23T09:49:44.755359+0000","last_became_active":"2024-03-23T09:49:43.264498+0000","last_became_peered":"2024-03-23T09:49:43.264498+0000","last_unstale":"2024-03-23T09:49:44.755359+0000","last_undegraded":"2024-03-23T09:49:44.755359+0000","last_fullsized":"2024-03-23T09:49:44.755359+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:35:39.113834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.251697+0000","last_change":"2024-03-23T09:49:42.893293+0000","last_active":"2024-03-23T09:49:43.251697+0000","last_peered":"2024-03-23T09:49:43.251697+0000","last_clean":"2024-03-23T09:49:43.251697+0000","last_became_active":"2024-03-23T09:49:42.893016+0000","last_became_peered":"2024-03-23T09:49:42.893016+0000","last_unstale":"2024-03-23T09:49:43.251697+0000","last_undegraded":"2024-03-23T09:49:43.251697+0000","last_fullsized":"2024-03-23T09:49:43.251697+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:22:32.641742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265951+0000","last_change":"2024-03-23T09:49:43.260670+0000","last_active":"2024-03-23T09:49:44.265951+0000","last_peered":"2024-03-23T09:49:44.265951+0000","last_clean":"2024-03-23T09:49:44.265951+0000","last_became_active":"2024-03-23T09:49:43.260386+0000","last_became_peered":"2024-03-23T09:49:43.260386+0000","last_unstale":"2024-03-23T09:49:44.265951+0000","last_undegraded":"2024-03-23T09:49:44.265951+0000","last_fullsized":"2024-03-23T09:49:44.265951+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:12:49.012393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.254493+0000","last_change":"2024-03-23T09:49:42.887212+0000","last_active":"2024-03-23T09:49:43.254493+0000","last_peered":"2024-03-23T09:49:43.254493+0000","last_clean":"2024-03-23T09:49:43.254493+0000","last_became_active":"2024-03-23T09:49:42.886933+0000","last_became_peered":"2024-03-23T09:49:42.886933+0000","last_unstale":"2024-03-23T09:49:43.254493+0000","last_undegraded":"2024-03-23T09:49:43.254493+0000","last_fullsized":"2024-03-23T09:49:43.254493+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:35.833734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.839359+0000","last_change":"2024-03-23T09:49:43.268185+0000","last_active":"2024-03-23T09:49:45.839359+0000","last_peered":"2024-03-23T09:49:45.839359+0000","last_clean":"2024-03-23T09:49:45.839359+0000","last_became_active":"2024-03-23T09:49:43.267925+0000","last_became_peered":"2024-03-23T09:49:43.267925+0000","last_unstale":"2024-03-23T09:49:45.839359+0000","last_undegraded":"2024-03-23T09:49:45.839359+0000","last_fullsized":"2024-03-23T09:49:45.839359+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:40.898697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"100'8","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.269323+0000","last_change":"2024-03-23T09:49:42.894936+0000","last_active":"2024-03-23T09:49:43.269323+0000","last_peered":"2024-03-23T09:49:43.269323+0000","last_clean":"2024-03-23T09:49:43.269323+0000","last_became_active":"2024-03-23T09:49:42.894617+0000","last_became_peered":"2024-03-23T09:49:42.894617+0000","last_unstale":"2024-03-23T09:49:43.269323+0000","last_undegraded":"2024-03-23T09:49:43.269323+0000","last_fullsized":"2024-03-23T09:49:43.269323+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:24:49.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.526879+0000","last_change":"2024-03-23T09:49:43.274729+0000","last_active":"2024-03-23T09:49:45.526879+0000","last_peered":"2024-03-23T09:49:45.526879+0000","last_clean":"2024-03-23T09:49:45.526879+0000","last_became_active":"2024-03-23T09:49:43.273681+0000","last_became_peered":"2024-03-23T09:49:43.273681+0000","last_unstale":"2024-03-23T09:49:45.526879+0000","last_undegraded":"2024-03-23T09:49:45.526879+0000","last_fullsized":"2024-03-23T09:49:45.526879+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:28:52.260374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"100'2","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.274226+0000","last_change":"2024-03-23T09:49:42.897130+0000","last_active":"2024-03-23T09:49:43.274226+0000","last_peered":"2024-03-23T09:49:43.274226+0000","last_clean":"2024-03-23T09:49:43.274226+0000","last_became_active":"2024-03-23T09:49:42.896909+0000","last_became_peered":"2024-03-23T09:49:42.896909+0000","last_unstale":"2024-03-23T09:49:43.274226+0000","last_undegraded":"2024-03-23T09:49:43.274226+0000","last_fullsized":"2024-03-23T09:49:43.274226+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:13.446270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.927240+0000","last_change":"2024-03-23T09:49:43.269964+0000","last_active":"2024-03-23T09:49:44.927240+0000","last_peered":"2024-03-23T09:49:44.927240+0000","last_clean":"2024-03-23T09:49:44.927240+0000","last_became_active":"2024-03-23T09:49:43.269408+0000","last_became_peered":"2024-03-23T09:49:43.269408+0000","last_unstale":"2024-03-23T09:49:44.927240+0000","last_undegraded":"2024-03-23T09:49:44.927240+0000","last_fullsized":"2024-03-23T09:49:44.927240+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:33:11.460484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"100'1","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.272835+0000","last_change":"2024-03-23T09:49:42.890004+0000","last_active":"2024-03-23T09:49:43.272835+0000","last_peered":"2024-03-23T09:49:43.272835+0000","last_clean":"2024-03-23T09:49:43.272835+0000","last_became_active":"2024-03-23T09:49:42.889611+0000","last_became_peered":"2024-03-23T09:49:42.889611+0000","last_unstale":"2024-03-23T09:49:43.272835+0000","last_undegraded":"2024-03-23T09:49:43.272835+0000","last_fullsized":"2024-03-23T09:49:43.272835+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:25:47.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.265753+0000","last_change":"2024-03-23T09:49:43.267737+0000","last_active":"2024-03-23T09:49:44.265753+0000","last_peered":"2024-03-23T09:49:44.265753+0000","last_clean":"2024-03-23T09:49:44.265753+0000","last_became_active":"2024-03-23T09:49:43.267450+0000","last_became_peered":"2024-03-23T09:49:43.267450+0000","last_unstale":"2024-03-23T09:49:44.265753+0000","last_undegraded":"2024-03-23T09:49:44.265753+0000","last_fullsized":"2024-03-23T09:49:44.265753+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:25:09.721273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.262455+0000","last_change":"2024-03-23T09:49:42.890436+0000","last_active":"2024-03-23T09:49:43.262455+0000","last_peered":"2024-03-23T09:49:43.262455+0000","last_clean":"2024-03-23T09:49:43.262455+0000","last_became_active":"2024-03-23T09:49:42.890155+0000","last_became_peered":"2024-03-23T09:49:42.890155+0000","last_unstale":"2024-03-23T09:49:43.262455+0000","last_undegraded":"2024-03-23T09:49:43.262455+0000","last_fullsized":"2024-03-23T09:49:43.262455+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:14:31.482221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.839456+0000","last_change":"2024-03-23T09:49:43.259677+0000","last_active":"2024-03-23T09:49:45.839456+0000","last_peered":"2024-03-23T09:49:45.839456+0000","last_clean":"2024-03-23T09:49:45.839456+0000","last_became_active":"2024-03-23T09:49:43.259421+0000","last_became_peered":"2024-03-23T09:49:43.259421+0000","last_unstale":"2024-03-23T09:49:45.839456+0000","last_undegraded":"2024-03-23T09:49:45.839456+0000","last_fullsized":"2024-03-23T09:49:45.839456+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:33:51.336024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"100'2","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.277973+0000","last_change":"2024-03-23T09:49:42.896164+0000","last_active":"2024-03-23T09:49:43.277973+0000","last_peered":"2024-03-23T09:49:43.277973+0000","last_clean":"2024-03-23T09:49:43.277973+0000","last_became_active":"2024-03-23T09:49:42.895700+0000","last_became_peered":"2024-03-23T09:49:42.895700+0000","last_unstale":"2024-03-23T09:49:43.277973+0000","last_undegraded":"2024-03-23T09:49:43.277973+0000","last_fullsized":"2024-03-23T09:49:43.277973+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:21:22.073815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.839649+0000","last_change":"2024-03-23T09:49:43.276140+0000","last_active":"2024-03-23T09:49:45.839649+0000","last_peered":"2024-03-23T09:49:45.839649+0000","last_clean":"2024-03-23T09:49:45.839649+0000","last_became_active":"2024-03-23T09:49:43.275775+0000","last_became_peered":"2024-03-23T09:49:43.275775+0000","last_unstale":"2024-03-23T09:49:45.839649+0000","last_undegraded":"2024-03-23T09:49:45.839649+0000","last_fullsized":"2024-03-23T09:49:45.839649+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:24:50.880561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.259121+0000","last_change":"2024-03-23T09:49:42.887576+0000","last_active":"2024-03-23T09:49:43.259121+0000","last_peered":"2024-03-23T09:49:43.259121+0000","last_clean":"2024-03-23T09:49:43.259121+0000","last_became_active":"2024-03-23T09:49:42.887334+0000","last_became_peered":"2024-03-23T09:49:42.887334+0000","last_unstale":"2024-03-23T09:49:43.259121+0000","last_undegraded":"2024-03-23T09:49:43.259121+0000","last_fullsized":"2024-03-23T09:49:43.259121+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:24:49.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.839344+0000","last_change":"2024-03-23T09:49:43.260271+0000","last_active":"2024-03-23T09:49:45.839344+0000","last_peered":"2024-03-23T09:49:45.839344+0000","last_clean":"2024-03-23T09:49:45.839344+0000","last_became_active":"2024-03-23T09:49:43.260006+0000","last_became_peered":"2024-03-23T09:49:43.260006+0000","last_unstale":"2024-03-23T09:49:45.839344+0000","last_undegraded":"2024-03-23T09:49:45.839344+0000","last_fullsized":"2024-03-23T09:49:45.839344+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:39:41.756112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.258926+0000","last_change":"2024-03-23T09:49:42.885766+0000","last_active":"2024-03-23T09:49:43.258926+0000","last_peered":"2024-03-23T09:49:43.258926+0000","last_clean":"2024-03-23T09:49:43.258926+0000","last_became_active":"2024-03-23T09:49:42.885475+0000","last_became_peered":"2024-03-23T09:49:42.885475+0000","last_unstale":"2024-03-23T09:49:43.258926+0000","last_undegraded":"2024-03-23T09:49:43.258926+0000","last_fullsized":"2024-03-23T09:49:43.258926+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:09:35.927315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.526759+0000","last_change":"2024-03-23T09:49:43.269531+0000","last_active":"2024-03-23T09:49:45.526759+0000","last_peered":"2024-03-23T09:49:45.526759+0000","last_clean":"2024-03-23T09:49:45.526759+0000","last_became_active":"2024-03-23T09:49:43.268566+0000","last_became_peered":"2024-03-23T09:49:43.268566+0000","last_unstale":"2024-03-23T09:49:45.526759+0000","last_undegraded":"2024-03-23T09:49:45.526759+0000","last_fullsized":"2024-03-23T09:49:45.526759+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:28:52.360503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.260831+0000","last_change":"2024-03-23T09:49:42.894354+0000","last_active":"2024-03-23T09:49:43.260831+0000","last_peered":"2024-03-23T09:49:43.260831+0000","last_clean":"2024-03-23T09:49:43.260831+0000","last_became_active":"2024-03-23T09:49:42.894093+0000","last_became_peered":"2024-03-23T09:49:42.894093+0000","last_unstale":"2024-03-23T09:49:43.260831+0000","last_undegraded":"2024-03-23T09:49:43.260831+0000","last_fullsized":"2024-03-23T09:49:43.260831+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:09.171978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.552876+0000","last_change":"2024-03-23T09:49:43.275680+0000","last_active":"2024-03-23T09:49:44.552876+0000","last_peered":"2024-03-23T09:49:44.552876+0000","last_clean":"2024-03-23T09:49:44.552876+0000","last_became_active":"2024-03-23T09:49:43.275436+0000","last_became_peered":"2024-03-23T09:49:43.275436+0000","last_unstale":"2024-03-23T09:49:44.552876+0000","last_undegraded":"2024-03-23T09:49:44.552876+0000","last_fullsized":"2024-03-23T09:49:44.552876+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:28:52.360503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"100'2","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.271507+0000","last_change":"2024-03-23T09:49:42.889337+0000","last_active":"2024-03-23T09:49:43.271507+0000","last_peered":"2024-03-23T09:49:43.271507+0000","last_clean":"2024-03-23T09:49:43.271507+0000","last_became_active":"2024-03-23T09:49:42.889038+0000","last_became_peered":"2024-03-23T09:49:42.889038+0000","last_unstale":"2024-03-23T09:49:43.271507+0000","last_undegraded":"2024-03-23T09:49:43.271507+0000","last_fullsized":"2024-03-23T09:49:43.271507+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:39.896584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.755395+0000","last_change":"2024-03-23T09:49:43.267778+0000","last_active":"2024-03-23T09:49:44.755395+0000","last_peered":"2024-03-23T09:49:44.755395+0000","last_clean":"2024-03-23T09:49:44.755395+0000","last_became_active":"2024-03-23T09:49:43.267521+0000","last_became_peered":"2024-03-23T09:49:43.267521+0000","last_unstale":"2024-03-23T09:49:44.755395+0000","last_undegraded":"2024-03-23T09:49:44.755395+0000","last_fullsized":"2024-03-23T09:49:44.755395+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:14:32.484334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.261989+0000","last_change":"2024-03-23T09:49:42.897628+0000","last_active":"2024-03-23T09:49:43.261989+0000","last_peered":"2024-03-23T09:49:43.261989+0000","last_clean":"2024-03-23T09:49:43.261989+0000","last_became_active":"2024-03-23T09:49:42.897289+0000","last_became_peered":"2024-03-23T09:49:42.897289+0000","last_unstale":"2024-03-23T09:49:43.261989+0000","last_undegraded":"2024-03-23T09:49:43.261989+0000","last_fullsized":"2024-03-23T09:49:43.261989+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:15:46.255074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.755448+0000","last_change":"2024-03-23T09:49:43.265979+0000","last_active":"2024-03-23T09:49:44.755448+0000","last_peered":"2024-03-23T09:49:44.755448+0000","last_clean":"2024-03-23T09:49:44.755448+0000","last_became_active":"2024-03-23T09:49:43.265590+0000","last_became_peered":"2024-03-23T09:49:43.265590+0000","last_unstale":"2024-03-23T09:49:44.755448+0000","last_undegraded":"2024-03-23T09:49:44.755448+0000","last_fullsized":"2024-03-23T09:49:44.755448+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:36:54.112262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.262354+0000","last_change":"2024-03-23T09:49:42.891768+0000","last_active":"2024-03-23T09:49:43.262354+0000","last_peered":"2024-03-23T09:49:43.262354+0000","last_clean":"2024-03-23T09:49:43.262354+0000","last_became_active":"2024-03-23T09:49:42.891464+0000","last_became_peered":"2024-03-23T09:49:42.891464+0000","last_unstale":"2024-03-23T09:49:43.262354+0000","last_undegraded":"2024-03-23T09:49:43.262354+0000","last_fullsized":"2024-03-23T09:49:43.262354+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.526729+0000","last_change":"2024-03-23T09:49:43.271576+0000","last_active":"2024-03-23T09:49:45.526729+0000","last_peered":"2024-03-23T09:49:45.526729+0000","last_clean":"2024-03-23T09:49:45.526729+0000","last_became_active":"2024-03-23T09:49:43.271258+0000","last_became_peered":"2024-03-23T09:49:43.271258+0000","last_unstale":"2024-03-23T09:49:45.526729+0000","last_undegraded":"2024-03-23T09:49:45.526729+0000","last_fullsized":"2024-03-23T09:49:45.526729+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:03.784913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.263060+0000","last_change":"2024-03-23T09:49:42.886908+0000","last_active":"2024-03-23T09:49:43.263060+0000","last_peered":"2024-03-23T09:49:43.263060+0000","last_clean":"2024-03-23T09:49:43.263060+0000","last_became_active":"2024-03-23T09:49:42.886628+0000","last_became_peered":"2024-03-23T09:49:42.886628+0000","last_unstale":"2024-03-23T09:49:43.263060+0000","last_undegraded":"2024-03-23T09:49:43.263060+0000","last_fullsized":"2024-03-23T09:49:43.263060+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:04:23.374836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.839610+0000","last_change":"2024-03-23T09:49:43.263136+0000","last_active":"2024-03-23T09:49:45.839610+0000","last_peered":"2024-03-23T09:49:45.839610+0000","last_clean":"2024-03-23T09:49:45.839610+0000","last_became_active":"2024-03-23T09:49:43.262846+0000","last_became_peered":"2024-03-23T09:49:43.262846+0000","last_unstale":"2024-03-23T09:49:45.839610+0000","last_undegraded":"2024-03-23T09:49:45.839610+0000","last_fullsized":"2024-03-23T09:49:45.839610+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:05:46.113728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.262777+0000","last_change":"2024-03-23T09:49:42.887797+0000","last_active":"2024-03-23T09:49:43.262777+0000","last_peered":"2024-03-23T09:49:43.262777+0000","last_clean":"2024-03-23T09:49:43.262777+0000","last_became_active":"2024-03-23T09:49:42.887516+0000","last_became_peered":"2024-03-23T09:49:42.887516+0000","last_unstale":"2024-03-23T09:49:43.262777+0000","last_undegraded":"2024-03-23T09:49:43.262777+0000","last_fullsized":"2024-03-23T09:49:43.262777+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:33:50.023224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:44.266151+0000","last_change":"2024-03-23T09:49:43.267798+0000","last_active":"2024-03-23T09:49:44.266151+0000","last_peered":"2024-03-23T09:49:44.266151+0000","last_clean":"2024-03-23T09:49:44.266151+0000","last_became_active":"2024-03-23T09:49:43.267550+0000","last_became_peered":"2024-03-23T09:49:43.267550+0000","last_unstale":"2024-03-23T09:49:44.266151+0000","last_undegraded":"2024-03-23T09:49:44.266151+0000","last_fullsized":"2024-03-23T09:49:44.266151+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:28:52.260374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.260461+0000","last_change":"2024-03-23T09:49:42.893737+0000","last_active":"2024-03-23T09:49:43.260461+0000","last_peered":"2024-03-23T09:49:43.260461+0000","last_clean":"2024-03-23T09:49:43.260461+0000","last_became_active":"2024-03-23T09:49:42.893458+0000","last_became_peered":"2024-03-23T09:49:42.893458+0000","last_unstale":"2024-03-23T09:49:43.260461+0000","last_undegraded":"2024-03-23T09:49:43.260461+0000","last_fullsized":"2024-03-23T09:49:43.260461+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:33:50.023224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.839342+0000","last_change":"2024-03-23T09:49:43.262761+0000","last_active":"2024-03-23T09:49:45.839342+0000","last_peered":"2024-03-23T09:49:45.839342+0000","last_clean":"2024-03-23T09:49:45.839342+0000","last_became_active":"2024-03-23T09:49:43.262528+0000","last_became_peered":"2024-03-23T09:49:43.262528+0000","last_unstale":"2024-03-23T09:49:45.839342+0000","last_undegraded":"2024-03-23T09:49:45.839342+0000","last_fullsized":"2024-03-23T09:49:45.839342+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:09:36.929428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.255819+0000","last_change":"2024-03-23T09:49:42.891595+0000","last_active":"2024-03-23T09:49:43.255819+0000","last_peered":"2024-03-23T09:49:43.255819+0000","last_clean":"2024-03-23T09:49:43.255819+0000","last_became_active":"2024-03-23T09:49:42.891321+0000","last_became_peered":"2024-03-23T09:49:42.891321+0000","last_unstale":"2024-03-23T09:49:43.255819+0000","last_undegraded":"2024-03-23T09:49:43.255819+0000","last_fullsized":"2024-03-23T09:49:43.255819+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:58:35.947175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.526601+0000","last_change":"2024-03-23T09:49:43.275987+0000","last_active":"2024-03-23T09:49:45.526601+0000","last_peered":"2024-03-23T09:49:45.526601+0000","last_clean":"2024-03-23T09:49:45.526601+0000","last_became_active":"2024-03-23T09:49:43.275492+0000","last_became_peered":"2024-03-23T09:49:43.275492+0000","last_unstale":"2024-03-23T09:49:45.526601+0000","last_undegraded":"2024-03-23T09:49:45.526601+0000","last_fullsized":"2024-03-23T09:49:45.526601+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:03:48.447165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.251000+0000","last_change":"2024-03-23T09:49:42.888501+0000","last_active":"2024-03-23T09:49:43.251000+0000","last_peered":"2024-03-23T09:49:43.251000+0000","last_clean":"2024-03-23T09:49:43.251000+0000","last_became_active":"2024-03-23T09:49:42.887110+0000","last_became_peered":"2024-03-23T09:49:42.887110+0000","last_unstale":"2024-03-23T09:49:43.251000+0000","last_undegraded":"2024-03-23T09:49:43.251000+0000","last_fullsized":"2024-03-23T09:49:43.251000+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:41:54.983525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-03-23T09:49:45.839812+0000","last_change":"2024-03-23T09:49:43.260781+0000","last_active":"2024-03-23T09:49:45.839812+0000","last_peered":"2024-03-23T09:49:45.839812+0000","last_clean":"2024-03-23T09:49:45.839812+0000","last_became_active":"2024-03-23T09:49:43.260536+0000","last_became_peered":"2024-03-23T09:49:43.260536+0000","last_unstale":"2024-03-23T09:49:45.839812+0000","last_undegraded":"2024-03-23T09:49:45.839812+0000","last_fullsized":"2024-03-23T09:49:45.839812+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:42.865767+0000","last_clean_scrub_stamp":"2024-03-23T09:49:42.865767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:14:46.779105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"100'1","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.271990+0000","last_change":"2024-03-23T09:49:42.887464+0000","last_active":"2024-03-23T09:49:43.271990+0000","last_peered":"2024-03-23T09:49:43.271990+0000","last_clean":"2024-03-23T09:49:43.271990+0000","last_became_active":"2024-03-23T09:49:42.887176+0000","last_became_peered":"2024-03-23T09:49:42.887176+0000","last_unstale":"2024-03-23T09:49:43.271990+0000","last_undegraded":"2024-03-23T09:49:43.271990+0000","last_fullsized":"2024-03-23T09:49:43.271990+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:49:41.863654+0000","last_clean_scrub_stamp":"2024-03-23T09:49:41.863654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:03:12.734179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":247,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-03-23T09:49:43.256049+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:49:43.256049+0000","last_peered":"2024-03-23T09:49:43.256049+0000","last_clean":"2024-03-23T09:49:43.256049+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:49:43.256049+0000","last_undegraded":"2024-03-23T09:49:43.256049+0000","last_fullsized":"2024-03-23T09:49:43.256049+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1851392,"data_stored":1128517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55600000000000005,"15min":0.51100000000000001},"min":{"1min":0.374,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.68000000000000005,"5min":0.76800000000000002,"15min":0.77500000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.53200000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28199999999999997},"max":{"1min":0.71799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.49199999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57699999999999996,"15min":0.53000000000000003},"min":{"1min":0.41299999999999998,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.76300000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.30199999999999999},"max":{"1min":0.747,"5min":0.81000000000000005,"15min":0.81299999999999994},"last":0.501}]},{"osd":2,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57399999999999995,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58999999999999997,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73299999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51100000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57299999999999995,"15min":0.54000000000000004},"min":{"1min":0.47199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.73199999999999998,"5min":0.84699999999999998,"15min":0.86199999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.59699999999999998,"15min":0.57999999999999996},"min":{"1min":0.503,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.82699999999999996,"15min":0.83699999999999997},"last":0.59499999999999997}]},{"osd":4,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.30399999999999999,"15min":0.29599999999999999},"max":{"1min":0.67500000000000004,"5min":0.76300000000000001,"15min":0.78200000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.48999999999999999,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.29299999999999998},"max":{"1min":0.67500000000000004,"5min":0.754,"15min":0.80100000000000005},"last":0.40999999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.5,"15min":0.51400000000000001},"min":{"1min":0.35599999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66200000000000003,"5min":0.80100000000000005,"15min":0.81499999999999995},"last":0.433},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52600000000000002,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.64300000000000002,"5min":0.78000000000000003,"15min":0.84999999999999998},"last":0.57699999999999996}]},{"osd":6,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53500000000000003,"15min":0.56999999999999995},"min":{"1min":0.36199999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71499999999999997,"5min":0.75700000000000001,"15min":0.874},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54900000000000004,"15min":0.57299999999999995},"min":{"1min":0.39600000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65400000000000003,"5min":0.79000000000000004,"15min":0.90200000000000002},"last":0.47799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1253376,"data_stored":538070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.52200000000000002},"min":{"1min":0.441,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.81000000000000005},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.45200000000000001,"5min":0.37,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.66100000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58999999999999997,"15min":0.53000000000000003},"min":{"1min":0.43099999999999999,"5min":0.315,"15min":0.311},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59699999999999998,"15min":0.55100000000000005},"min":{"1min":0.495,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56599999999999995}]},{"osd":2,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.36299999999999999,"15min":0.29699999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.81599999999999995},"last":0.435},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.56299999999999994},"min":{"1min":0.435,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.59399999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.57299999999999995},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.378},"max":{"1min":0.73299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.623},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57899999999999996,"15min":0.54500000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":0.79000000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.64100000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46000000000000002,"15min":0.49199999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.62,"5min":0.69799999999999995,"15min":0.72499999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.499,"15min":0.53400000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.66800000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.78900000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49199999999999999,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.34499999999999997,"15min":0.316},"max":{"1min":0.61399999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52500000000000002,"15min":0.54800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70399999999999996,"5min":0.76200000000000001,"15min":0.77700000000000002},"last":0.80500000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55400000000000005,"15min":0.57699999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70099999999999996,"5min":0.78400000000000003,"15min":0.86199999999999999},"last":0.69499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574956,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1253376,"data_stored":538107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54700000000000004,"15min":0.502},"min":{"1min":0.28799999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.70899999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.34899999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.75800000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.50600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71799999999999997,"5min":0.746,"15min":0.746},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.39200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.65700000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65500000000000003,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56000000000000005,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.40100000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.52600000000000002},"min":{"1min":0.41199999999999998,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.74099999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.54600000000000004},"min":{"1min":0.42299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69299999999999995,"5min":0.71199999999999997,"15min":0.77400000000000002},"last":0.51400000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45000000000000001,"15min":0.46500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73699999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47799999999999998,"15min":0.50800000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68600000000000005,"5min":0.72499999999999998,"15min":0.93600000000000005},"last":0.59999999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.749},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48399999999999999,"15min":0.505},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66800000000000004,"5min":0.71799999999999997,"15min":0.72399999999999998},"last":0.49099999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52900000000000003,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.64600000000000002,"5min":0.76200000000000001,"15min":0.78200000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1269760,"data_stored":539725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.52800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71499999999999997,"5min":0.77100000000000002,"15min":0.84599999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73599999999999999,"5min":0.76500000000000001,"15min":0.78600000000000003},"last":0.54600000000000004}]},{"osd":1,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52300000000000002,"15min":0.49199999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66800000000000004,"5min":0.72699999999999998,"15min":0.72999999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.34100000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.77400000000000002},"last":0.46899999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59499999999999997,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.40200000000000002,"15min":0.39400000000000002},"max":{"1min":0.69899999999999995,"5min":0.77900000000000003,"15min":0.80200000000000005},"last":0.437},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59999999999999998,"15min":0.55800000000000005},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.68500000000000005,"5min":0.79900000000000004,"15min":0.82199999999999995},"last":0.624}]},{"osd":3,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68400000000000005,"5min":0.80900000000000005,"15min":1.0369999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55500000000000005,"15min":0.52900000000000003},"min":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.374},"max":{"1min":0.72599999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.60799999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44,"15min":0.46000000000000002},"min":{"1min":0.315,"5min":0.309,"15min":0.26500000000000001},"max":{"1min":0.65800000000000003,"5min":0.73099999999999998,"15min":0.75800000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45900000000000002,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.253},"max":{"1min":0.59199999999999997,"5min":0.76100000000000001,"15min":1.008},"last":0.59199999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.315,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64100000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48899999999999999,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67700000000000005,"5min":0.79200000000000004,"15min":0.82399999999999995},"last":0.66500000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.501,"15min":0.53700000000000003},"min":{"1min":0.33500000000000002,"5min":0.312,"15min":0.30099999999999999},"max":{"1min":0.67300000000000004,"5min":0.71799999999999997,"15min":1.0069999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.36499999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69099999999999995,"5min":0.77700000000000002,"15min":0.93400000000000005},"last":0.65100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36460,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335380,"statfs":{"total":96636764160,"available":96599429120,"internally_reserved":0,"allocated":1265664,"data_stored":539623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35999999999999999,"15min":0.32700000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.183},"max":{"1min":0.54800000000000004,"5min":0.67800000000000005,"15min":1.45},"last":0.247},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36599999999999999,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54400000000000004,"5min":0.621,"15min":1.4099999999999999},"last":0.29499999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35299999999999998,"15min":0.32500000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46600000000000003,"5min":0.54300000000000004,"15min":1.323},"last":0.31},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39900000000000002,"15min":0.36299999999999999},"min":{"1min":0.309,"5min":0.22800000000000001,"15min":0.218},"max":{"1min":0.58299999999999996,"5min":0.67800000000000005,"15min":0.94599999999999995},"last":0.28199999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.437,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.251},"max":{"1min":0.60699999999999998,"5min":0.68999999999999995,"15min":1.1619999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46000000000000002,"15min":0.42599999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":0.70899999999999996,"15min":1.2989999999999999},"last":0.32800000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.80600000000000005,"15min":1.337},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75900000000000001,"5min":0.83899999999999997,"15min":2.758},"last":0.49099999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.874,"15min":1.276},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.60699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.88700000000000001,"15min":1.895},"last":0.64400000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.40699999999999997},"max":{"1min":0.84199999999999997,"5min":0.872,"15min":5.4539999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82099999999999995,"5min":0.93600000000000005,"15min":2.8159999999999998},"last":0.69699999999999995}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.501,"5min":0.44600000000000001,"15min":0.41299999999999998},"max":{"1min":0.85699999999999998,"5min":0.89800000000000002,"15min":3.7400000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.443,"5min":0.443,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":5.4900000000000002},"last":0.61399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1253376,"data_stored":538071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34999999999999998,"15min":0.34599999999999997},"min":{"1min":0.249,"5min":0.191,"15min":0.191},"max":{"1min":0.49099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.379},"min":{"1min":0.247,"5min":0.245,"15min":0.22},"max":{"1min":0.58199999999999996,"5min":0.64400000000000002,"15min":0.872},"last":0.35699999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.371,"15min":0.36899999999999999},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.20999999999999999},"max":{"1min":0.49399999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.30299999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.746},"last":0.31900000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41899999999999998,"15min":0.41199999999999998},"min":{"1min":0.32300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.54200000000000004,"5min":0.65200000000000002,"15min":0.67100000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39900000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.248},"max":{"1min":0.56299999999999994,"5min":0.66700000000000004,"15min":0.72499999999999998},"last":0.38300000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.62,"15min":0.629},"min":{"1min":0.47599999999999998,"5min":0.43099999999999999,"15min":0.38500000000000001},"max":{"1min":0.80100000000000005,"5min":0.87,"15min":1.216},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.61699999999999999},"min":{"1min":0.503,"5min":0.41099999999999998,"15min":0.40300000000000002},"max":{"1min":0.80000000000000004,"5min":0.89600000000000002,"15min":0.99299999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.90400000000000003,"15min":0.95699999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.63900000000000001,"15min":0.63200000000000001},"min":{"1min":0.47899999999999998,"5min":0.42099999999999999,"15min":0.40300000000000002},"max":{"1min":0.81499999999999995,"5min":0.93700000000000006,"15min":1.101},"last":0.54100000000000004}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65300000000000002,"15min":0.63600000000000001},"min":{"1min":0.46400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83199999999999996,"5min":0.89100000000000001,"15min":0.97299999999999998},"last":0.628},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64800000000000002,"15min":0.64500000000000002},"min":{"1min":0.48799999999999999,"5min":0.45800000000000002,"15min":0.436},"max":{"1min":0.78600000000000003,"5min":0.89500000000000002,"15min":1.0780000000000001},"last":0.52200000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.54100000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80000000000000004,"5min":0.88500000000000001,"15min":1.155},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.41599999999999998},"max":{"1min":0.81399999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36444,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335396,"statfs":{"total":96636764160,"available":96599445504,"internally_reserved":0,"allocated":1249280,"data_stored":538037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.34200000000000003},"min":{"1min":0.22500000000000001,"5min":0.214,"15min":0.19400000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.64400000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.44700000000000001},"min":{"1min":0.25800000000000001,"5min":0.218,"15min":0.20699999999999999},"max":{"1min":0.56499999999999995,"5min":0.71999999999999997,"15min":2.7069999999999999},"last":0.252}]},{"osd":2,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.36699999999999999,"15min":0.35099999999999998},"min":{"1min":0.22800000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57299999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.38500000000000001},"min":{"1min":0.28799999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.53300000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.311}]},{"osd":3,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.437,"15min":0.41899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.76900000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40799999999999997,"15min":0.47299999999999998},"min":{"1min":0.27300000000000002,"5min":0.25600000000000001,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.73899999999999999,"15min":2.75},"last":0.29399999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60599999999999998,"15min":0.59299999999999997},"min":{"1min":0.44900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.95899999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83299999999999996,"5min":1.002,"15min":1.002},"last":0.50900000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.0900000000000001,"15min":2.778},"last":0.498},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.438,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88800000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56100000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.68700000000000006},"min":{"1min":0.46700000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86299999999999999,"5min":1.026,"15min":2.8210000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.51400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93300000000000005,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.53400000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84999999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65600000000000003,"15min":0.72399999999999998},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91400000000000003,"5min":0.91600000000000004,"15min":2.7989999999999999},"last":0.57899999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574956,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37024,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334816,"statfs":{"total":96636764160,"available":96598851584,"internally_reserved":0,"allocated":1843200,"data_stored":1128368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"min":{"1min":0.254,"5min":0.192,"15min":0.184},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.375,"15min":0.35799999999999998},"min":{"1min":0.28199999999999997,"5min":0.23300000000000001,"15min":0.217},"max":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.25600000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.374,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.25,"15min":0.25},"max":{"1min":0.56799999999999995,"5min":0.63,"15min":0.63},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.377},"min":{"1min":0.26500000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.59399999999999997},"last":0.32600000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.38700000000000001},"min":{"1min":0.27100000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.67600000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.44700000000000001,"15min":0.432},"min":{"1min":0.35899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.38}]},{"osd":4,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60599999999999998,"15min":0.59099999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.89100000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61699999999999999,"15min":0.59799999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91200000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.73599999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.90000000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.59499999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.95999999999999996},"last":0.68400000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65000000000000002,"15min":0.629},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.93600000000000005},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.59099999999999997}]},{"osd":7,"last update":"Sat Mar 23 09:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64400000000000002,"15min":0.61299999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.878,"15min":0.93600000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63600000000000001,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.75}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:49:51.015 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T09:49:51.015 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-23T09:49:51.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.159+0000 7f2044e4a640 1 Processor -- start 2024-03-23T09:49:51.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.160+0000 7f2044e4a640 1 -- start start 2024-03-23T09:49:51.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.160+0000 7f2044e4a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f20400ae300 0x7f20400cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.160+0000 7f2044e4a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f204015d6e0 0x7f20400ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.160+0000 7f2044e4a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204006a330 con 0x7f20401401e0 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.160+0000 7f2044e4a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204015dbb0 con 0x7f20400ae300 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.160+0000 7f2044e4a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204015de90 con 0x7f204015d6e0 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.160+0000 7f2037fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f20401401e0 0x7f204015cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47534/0 (socket says 172.21.15.47:47534) 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.160+0000 7f2037fff640 1 -- 172.21.15.47:0/2961515108 learned_addr learned my addr 172.21.15.47:0/2961515108 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204608649 0 0) 0x7f204006a330 con 0x7f20401401e0 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203ed9d640 1 --2- 172.21.15.47:0/2961515108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f204015d6e0 0x7f20400ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f202c003520 con 0x7f20401401e0 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203e59c640 1 --2- 172.21.15.47:0/2961515108 >> v2:172.21.15.175:3300/0 conn(0x7f20400ae300 0x7f20400cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 397814037 0 0) 0x7f202c003520 con 0x7f20401401e0 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f204015d6e0 msgr2=0x7f20400ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 --2- 172.21.15.47:0/2961515108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f204015d6e0 0x7f20400ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 >> v2:172.21.15.175:3300/0 conn(0x7f20400ae300 msgr2=0x7f20400cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 --2- 172.21.15.47:0/2961515108 >> v2:172.21.15.175:3300/0 conn(0x7f20400ae300 0x7f20400cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2040069ce0 con 0x7f20401401e0 2024-03-23T09:49:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f2028001670 con 0x7f20401401e0 2024-03-23T09:49:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 >> v1:172.21.15.47:6789/0 conn(0x7f20401401e0 legacy=0x7f204015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 --2- 172.21.15.47:0/2961515108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f204015d6e0 0x7f202c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 --2- 172.21.15.47:0/2961515108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c003ef0 0x7f202c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 --2- 172.21.15.47:0/2961515108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f202c004d00 0x7f202c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204015dbb0 con 0x7f202c004d00 2024-03-23T09:49:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204015de90 con 0x7f202c003ef0 2024-03-23T09:49:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c00b980 con 0x7f204015d6e0 2024-03-23T09:49:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203ed9d640 1 --2- 172.21.15.47:0/2961515108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f202c004d00 0x7f202c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f2037fff640 1 --2- 172.21.15.47:0/2961515108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c003ef0 0x7f202c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203e59c640 1 --2- 172.21.15.47:0/2961515108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f204015d6e0 0x7f202c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203ed9d640 1 -- 172.21.15.47:0/2961515108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f204015d6e0 msgr2=0x7f202c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203ed9d640 1 --2- 172.21.15.47:0/2961515108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f204015d6e0 0x7f202c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203ed9d640 1 -- 172.21.15.47:0/2961515108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c003ef0 msgr2=0x7f202c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:51.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203ed9d640 1 --2- 172.21.15.47:0/2961515108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c003ef0 0x7f202c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.161+0000 7f203ed9d640 1 -- 172.21.15.47:0/2961515108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c003520 con 0x7f202c004d00 2024-03-23T09:49:51.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.162+0000 7f203ed9d640 1 --2- 172.21.15.47:0/2961515108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f202c004d00 0x7f202c005180 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f2038009f90 tx=0x7f202c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae247f8503fc438 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:51.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.162+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203802cbc0 con 0x7f202c004d00 2024-03-23T09:49:51.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.162+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203800f040 con 0x7f202c004d00 2024-03-23T09:49:51.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.162+0000 7f203dd9b640 1 -- 172.21.15.47:0/2961515108 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038034440 con 0x7f202c004d00 2024-03-23T09:49:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.163+0000 7f2044e4a640 1 -- 172.21.15.47:0/2961515108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f202c004d00 msgr2=0x7f202c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.163+0000 7f2044e4a640 1 --2- 172.21.15.47:0/2961515108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f202c004d00 0x7f202c005180 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f2038009f90 tx=0x7f202c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.163+0000 7f203ed9d640 1 -- 172.21.15.47:0/2961515108 reap_dead start 2024-03-23T09:49:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.163+0000 7f2044e4a640 1 -- 172.21.15.47:0/2961515108 shutdown_connections 2024-03-23T09:49:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.163+0000 7f2044e4a640 1 -- 172.21.15.47:0/2961515108 >> 172.21.15.47:0/2961515108 conn(0x7f20400cb810 msgr2=0x7f20400cbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.163+0000 7f2044e4a640 1 -- 172.21.15.47:0/2961515108 shutdown_connections 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.163+0000 7f2044e4a640 1 -- 172.21.15.47:0/2961515108 wait complete. 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.164+0000 7f2044e4a640 1 Processor -- start 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.164+0000 7f2044e4a640 1 -- start start 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f2044e4a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c004d00 0x7f20401fa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f2044e4a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20400ca7b0 0x7f20401f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f2044e4a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20400cac30 0x7f204015be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f2044e4a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20401f9cb0 con 0x7f20400cac30 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f2044e4a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20401f71a0 con 0x7f202c004d00 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f2044e4a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20401f7480 con 0x7f20400ca7b0 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f203ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20400cac30 0x7f204015be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f203ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20400cac30 0x7f204015be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44292/0 (socket says 172.21.15.47:44292) 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f203ed9d640 1 -- 172.21.15.47:0/164414290 learned_addr learned my addr 172.21.15.47:0/164414290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f2037fff640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20400ca7b0 0x7f20401f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f203e59c640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c004d00 0x7f20401fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f203ed9d640 1 -- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20400ca7b0 msgr2=0x7f20401f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f203ed9d640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20400ca7b0 0x7f20401f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f203ed9d640 1 -- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c004d00 msgr2=0x7f20401fa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f203ed9d640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c004d00 0x7f20401fa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.165+0000 7f203ed9d640 1 -- 172.21.15.47:0/164414290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20401f9f00 con 0x7f20400cac30 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.166+0000 7f203ed9d640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20400cac30 0x7f204015be90 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2038037040 tx=0x7f2038009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.166+0000 7f2036ffd640 1 -- 172.21.15.47:0/164414290 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038034ad0 con 0x7f20400cac30 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.166+0000 7f2036ffd640 1 -- 172.21.15.47:0/164414290 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2038034c30 con 0x7f20400cac30 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.166+0000 7f2044e4a640 1 -- 172.21.15.47:0/164414290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20401fa1e0 con 0x7f20400cac30 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.166+0000 7f2036ffd640 1 -- 172.21.15.47:0/164414290 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038003990 con 0x7f20400cac30 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.166+0000 7f2044e4a640 1 -- 172.21.15.47:0/164414290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20401f6bd0 con 0x7f20400cac30 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.167+0000 7f2044e4a640 1 -- 172.21.15.47:0/164414290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20401fa3d0 con 0x7f20400cac30 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.167+0000 7f2036ffd640 1 -- 172.21.15.47:0/164414290 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2038047460 con 0x7f20400cac30 2024-03-23T09:49:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.168+0000 7f2036ffd640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f202006ac00 0x7f202006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.168+0000 7f2036ffd640 1 -- 172.21.15.47:0/164414290 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20380ab0c0 con 0x7f20400cac30 2024-03-23T09:49:51.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.168+0000 7f203e59c640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f202006ac00 0x7f202006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.168+0000 7f203e59c640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f202006ac00 0x7f202006d0c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f204006b680 tx=0x7f2030008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:51.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.172+0000 7f2036ffd640 1 -- 172.21.15.47:0/164414290 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2038076330 con 0x7f20400cac30 2024-03-23T09:49:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.420+0000 7f2044e4a640 1 -- 172.21.15.47:0/164414290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f20400672f0 con 0x7f20400cac30 2024-03-23T09:49:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.424+0000 7f2036ffd640 1 -- 172.21.15.47:0/164414290 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2038079950 con 0x7f20400cac30 2024-03-23T09:49:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-03-23T09:49:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.426+0000 7f2044e4a640 1 -- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f202006ac00 msgr2=0x7f202006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.426+0000 7f2044e4a640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f202006ac00 0x7f202006d0c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f204006b680 tx=0x7f2030008040 comp rx=0 tx=0).stop 2024-03-23T09:49:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.426+0000 7f2044e4a640 1 -- 172.21.15.47:0/164414290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20400cac30 msgr2=0x7f204015be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.426+0000 7f2044e4a640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20400cac30 0x7f204015be90 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2038037040 tx=0x7f2038009750 comp rx=0 tx=0).stop 2024-03-23T09:49:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.426+0000 7f2044e4a640 1 -- 172.21.15.47:0/164414290 shutdown_connections 2024-03-23T09:49:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.426+0000 7f2044e4a640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f202006ac00 0x7f202006d0c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.426+0000 7f2044e4a640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20400cac30 0x7f204015be90 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.426+0000 7f2044e4a640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20400ca7b0 0x7f20401f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.426+0000 7f2044e4a640 1 --2- 172.21.15.47:0/164414290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c004d00 0x7f20401fa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.426+0000 7f2044e4a640 1 -- 172.21.15.47:0/164414290 >> 172.21.15.47:0/164414290 conn(0x7f20400cb810 msgr2=0x7f204013e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.426+0000 7f2044e4a640 1 -- 172.21.15.47:0/164414290 shutdown_connections 2024-03-23T09:49:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.426+0000 7f2044e4a640 1 -- 172.21.15.47:0/164414290 wait complete. 2024-03-23T09:49:51.437 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:49:51.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.538+0000 7fe7c482c640 1 Processor -- start 2024-03-23T09:49:51.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.538+0000 7fe7c482c640 1 -- start start 2024-03-23T09:49:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.539+0000 7fe7c482c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe7c01601e0 0x7fe7c00cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.539+0000 7fe7c482c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c006e620 0x7fe7c00cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.539+0000 7fe7c482c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c006d580 con 0x7fe7c00c9390 2024-03-23T09:49:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.539+0000 7fe7c482c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c00ce160 con 0x7fe7c01601e0 2024-03-23T09:49:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.539+0000 7fe7c482c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c006d860 con 0x7fe7c006e620 2024-03-23T09:49:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.539+0000 7fe7bed9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe7c00c9390 0x7fe7c00c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47538/0 (socket says 172.21.15.47:47538) 2024-03-23T09:49:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.539+0000 7fe7bed9d640 1 -- 172.21.15.47:0/3841426809 learned_addr learned my addr 172.21.15.47:0/3841426809 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7be59c640 1 --2- 172.21.15.47:0/3841426809 >> v2:172.21.15.175:3300/0 conn(0x7fe7c01601e0 0x7fe7c00cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746367540 0 0) 0x7fe7c006d580 con 0x7fe7c00c9390 2024-03-23T09:49:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bdd9b640 1 --2- 172.21.15.47:0/3841426809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c006e620 0x7fe7c00cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7ac003520 con 0x7fe7c00c9390 2024-03-23T09:49:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3124901402 0 0) 0x7fe7ac003520 con 0x7fe7c00c9390 2024-03-23T09:49:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c006e620 msgr2=0x7fe7c00cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 --2- 172.21.15.47:0/3841426809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c006e620 0x7fe7c00cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 >> v2:172.21.15.175:3300/0 conn(0x7fe7c01601e0 msgr2=0x7fe7c00cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 --2- 172.21.15.47:0/3841426809 >> v2:172.21.15.175:3300/0 conn(0x7fe7c01601e0 0x7fe7c00cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c006db40 con 0x7fe7c00c9390 2024-03-23T09:49:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fe7b0001670 con 0x7fe7c00c9390 2024-03-23T09:49:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 >> v1:172.21.15.47:6789/0 conn(0x7fe7c00c9390 legacy=0x7fe7c00c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 --2- 172.21.15.47:0/3841426809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7ac003970 0x7fe7ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 --2- 172.21.15.47:0/3841426809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7ac0047c0 0x7fe7ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 --2- 172.21.15.47:0/3841426809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c006e620 0x7fe7ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c006d860 con 0x7fe7ac003970 2024-03-23T09:49:51.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac00b980 con 0x7fe7ac0047c0 2024-03-23T09:49:51.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac00baf0 con 0x7fe7c006e620 2024-03-23T09:49:51.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7be59c640 1 --2- 172.21.15.47:0/3841426809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7ac003970 0x7fe7ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bed9d640 1 --2- 172.21.15.47:0/3841426809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c006e620 0x7fe7ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7bdd9b640 1 --2- 172.21.15.47:0/3841426809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7ac0047c0 0x7fe7ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7be59c640 1 -- 172.21.15.47:0/3841426809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c006e620 msgr2=0x7fe7ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7be59c640 1 --2- 172.21.15.47:0/3841426809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c006e620 0x7fe7ac005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7be59c640 1 -- 172.21.15.47:0/3841426809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7ac0047c0 msgr2=0x7fe7ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7be59c640 1 --2- 172.21.15.47:0/3841426809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7ac0047c0 0x7fe7ac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.540+0000 7fe7be59c640 1 -- 172.21.15.47:0/3841426809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac003520 con 0x7fe7ac003970 2024-03-23T09:49:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.541+0000 7fe7be59c640 1 --2- 172.21.15.47:0/3841426809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7ac003970 0x7fe7ac003d90 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009fc0 tx=0x7fe7ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac98de781e337cb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.541+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b802cc60 con 0x7fe7ac003970 2024-03-23T09:49:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.541+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b800f040 con 0x7fe7ac003970 2024-03-23T09:49:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.541+0000 7fe7bd59a640 1 -- 172.21.15.47:0/3841426809 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8034440 con 0x7fe7ac003970 2024-03-23T09:49:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.542+0000 7fe7c482c640 1 -- 172.21.15.47:0/3841426809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7ac003970 msgr2=0x7fe7ac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.542+0000 7fe7c482c640 1 --2- 172.21.15.47:0/3841426809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7ac003970 0x7fe7ac003d90 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009fc0 tx=0x7fe7ac0057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.542+0000 7fe7bed9d640 1 -- 172.21.15.47:0/3841426809 reap_dead start 2024-03-23T09:49:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.542+0000 7fe7c482c640 1 -- 172.21.15.47:0/3841426809 shutdown_connections 2024-03-23T09:49:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.542+0000 7fe7c482c640 1 -- 172.21.15.47:0/3841426809 >> 172.21.15.47:0/3841426809 conn(0x7fe7c00cb7c0 msgr2=0x7fe7c00cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.542+0000 7fe7c482c640 1 -- 172.21.15.47:0/3841426809 shutdown_connections 2024-03-23T09:49:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.542+0000 7fe7c482c640 1 -- 172.21.15.47:0/3841426809 wait complete. 2024-03-23T09:49:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.544+0000 7fe7c482c640 1 Processor -- start 2024-03-23T09:49:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.544+0000 7fe7c482c640 1 -- start start 2024-03-23T09:49:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.544+0000 7fe7c482c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7ac003970 0x7fe7c015ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.544+0000 7fe7c482c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c00c7fe0 0x7fe7c00ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.544+0000 7fe7c482c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7c015cc40 0x7fe7c00ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.544+0000 7fe7c482c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c00c3110 con 0x7fe7ac003970 2024-03-23T09:49:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.544+0000 7fe7c482c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c015da50 con 0x7fe7c015cc40 2024-03-23T09:49:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.544+0000 7fe7c482c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c015d1b0 con 0x7fe7c00c7fe0 2024-03-23T09:49:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.544+0000 7fe7be59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7ac003970 0x7fe7c015ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.544+0000 7fe7be59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7ac003970 0x7fe7c015ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44300/0 (socket says 172.21.15.47:44300) 2024-03-23T09:49:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.544+0000 7fe7be59c640 1 -- 172.21.15.47:0/267595483 learned_addr learned my addr 172.21.15.47:0/267595483 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7bdd9b640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c00c7fe0 0x7fe7c00ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7bed9d640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7c015cc40 0x7fe7c00ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7be59c640 1 -- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c00c7fe0 msgr2=0x7fe7c00ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7be59c640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c00c7fe0 0x7fe7c00ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7be59c640 1 -- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7c015cc40 msgr2=0x7fe7c00ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7be59c640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7c015cc40 0x7fe7c00ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7be59c640 1 -- 172.21.15.47:0/267595483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c015d490 con 0x7fe7ac003970 2024-03-23T09:49:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7bdd9b640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c00c7fe0 0x7fe7c00ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7be59c640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7ac003970 0x7fe7c015ebb0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e230 tx=0x7fe7b8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7b6ffd640 1 -- 172.21.15.47:0/267595483 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8034c50 con 0x7fe7ac003970 2024-03-23T09:49:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7b6ffd640 1 -- 172.21.15.47:0/267595483 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b8034db0 con 0x7fe7ac003970 2024-03-23T09:49:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7c482c640 1 -- 172.21.15.47:0/267595483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c015d770 con 0x7fe7ac003970 2024-03-23T09:49:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7b6ffd640 1 -- 172.21.15.47:0/267595483 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8003bd0 con 0x7fe7ac003970 2024-03-23T09:49:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.545+0000 7fe7c482c640 1 -- 172.21.15.47:0/267595483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c015f0f0 con 0x7fe7ac003970 2024-03-23T09:49:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.546+0000 7fe7c482c640 1 -- 172.21.15.47:0/267595483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c00672f0 con 0x7fe7ac003970 2024-03-23T09:49:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.546+0000 7fe7b6ffd640 1 -- 172.21.15.47:0/267595483 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe7b8040450 con 0x7fe7ac003970 2024-03-23T09:49:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.547+0000 7fe7b6ffd640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe7a006ac30 0x7fe7a006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.547+0000 7fe7b6ffd640 1 -- 172.21.15.47:0/267595483 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7b80abaa0 con 0x7fe7ac003970 2024-03-23T09:49:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.547+0000 7fe7bdd9b640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe7a006ac30 0x7fe7a006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.548+0000 7fe7bdd9b640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe7a006ac30 0x7fe7a006d0f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00add10 tx=0x7fe7a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.551+0000 7fe7b6ffd640 1 -- 172.21.15.47:0/267595483 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe7b8076c80 con 0x7fe7ac003970 2024-03-23T09:49:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.769+0000 7fe7c482c640 1 -- 172.21.15.47:0/267595483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7c006a4f0 con 0x7fe7ac003970 2024-03-23T09:49:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.769+0000 7fe7b6ffd640 1 -- 172.21.15.47:0/267595483 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fe7b807a2a0 con 0x7fe7ac003970 2024-03-23T09:49:51.770 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:51.770 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/249416109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":249416109},{"type":"v1","addr":"172.21.15.175:6837","nonce":249416109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3603846321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3603846321},{"type":"v1","addr":"172.21.15.175:6835","nonce":3603846321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1921094794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1921094794},{"type":"v1","addr":"172.21.15.47:6834","nonce":1921094794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:49:43.247119+0000","modified":"2024-03-23T09:49:45.258459+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8484":{"gid":8484,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/2388298413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2388298413},{"type":"v1","addr":"172.21.15.47:6835","nonce":2388298413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-23T09:49:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 65 2024-03-23T09:49:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.771+0000 7fe7c482c640 1 -- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe7a006ac30 msgr2=0x7fe7a006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.771+0000 7fe7c482c640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe7a006ac30 0x7fe7a006d0f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00add10 tx=0x7fe7a8008040 comp rx=0 tx=0).stop 2024-03-23T09:49:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.771+0000 7fe7c482c640 1 -- 172.21.15.47:0/267595483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7ac003970 msgr2=0x7fe7c015ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.771+0000 7fe7c482c640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7ac003970 0x7fe7c015ebb0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e230 tx=0x7fe7b8009750 comp rx=0 tx=0).stop 2024-03-23T09:49:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.772+0000 7fe7c482c640 1 -- 172.21.15.47:0/267595483 shutdown_connections 2024-03-23T09:49:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.772+0000 7fe7c482c640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe7a006ac30 0x7fe7a006d0f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.772+0000 7fe7c482c640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7c015cc40 0x7fe7c00ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.772+0000 7fe7c482c640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c00c7fe0 0x7fe7c00ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.772+0000 7fe7c482c640 1 --2- 172.21.15.47:0/267595483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7ac003970 0x7fe7c015ebb0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.772+0000 7fe7c482c640 1 -- 172.21.15.47:0/267595483 >> 172.21.15.47:0/267595483 conn(0x7fe7c00cb7c0 msgr2=0x7fe7c00ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.772+0000 7fe7c482c640 1 -- 172.21.15.47:0/267595483 shutdown_connections 2024-03-23T09:49:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:51.772+0000 7fe7c482c640 1 -- 172.21.15.47:0/267595483 wait complete. 2024-03-23T09:49:52.185 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:49:43.247119+0000', 'modified': '2024-03-23T09:49:45.258459+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8484}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8484': {'gid': 8484, 'name': 'a', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/2388298413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6833', 'nonce': 2388298413}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 2388298413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:49:52.185 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:49:52.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-23T09:49:52.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.290+0000 7f10f7199640 1 Processor -- start 2024-03-23T09:49:52.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.290+0000 7f10f7199640 1 -- start start 2024-03-23T09:49:52.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.290+0000 7f10f7199640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10f00cf8a0 0x7f10f0167b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.291+0000 7f10f7199640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f00d0ee0 con 0x7f10f00b8300 2024-03-23T09:49:52.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.291+0000 7f10f7199640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f01616f0 con 0x7f10f015cb40 2024-03-23T09:49:52.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.291+0000 7f10f7199640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f00b5a10 con 0x7f10f00cf8a0 2024-03-23T09:49:52.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.291+0000 7f10f5736640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10f00cf8a0 0x7f10f0167b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.291+0000 7f10effff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f10f00b8300 0x7f10f00cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47552/0 (socket says 172.21.15.47:47552) 2024-03-23T09:49:52.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.291+0000 7f10effff640 1 -- 172.21.15.47:0/3608053115 learned_addr learned my addr 172.21.15.47:0/3608053115 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:52.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.291+0000 7f10f5736640 1 -- 172.21.15.47:0/3608053115 >> v1:172.21.15.175:6789/0 conn(0x7f10f015cb40 legacy=0x7f10f0160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:52.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.291+0000 7f10f5736640 1 -- 172.21.15.47:0/3608053115 >> v1:172.21.15.47:6789/0 conn(0x7f10f00b8300 legacy=0x7f10f00cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.291+0000 7f10ef7fe640 1 -- 172.21.15.47:0/3608053115 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127836770 0 0) 0x7f10f00d0ee0 con 0x7f10f00b8300 2024-03-23T09:49:52.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.291+0000 7f10f5736640 1 -- 172.21.15.47:0/3608053115 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f0160e80 con 0x7f10f00cf8a0 2024-03-23T09:49:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.292+0000 7f10f5736640 1 --2- 172.21.15.47:0/3608053115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f00cf8a0 0x7f10f0167b00 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009dc0 tx=0x7f10e802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17dd7ea13dfda7b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.292+0000 7f10ef7fe640 1 -- 172.21.15.47:0/3608053115 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e802cc40 con 0x7f10f00cf8a0 2024-03-23T09:49:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.292+0000 7f10ef7fe640 1 -- 172.21.15.47:0/3608053115 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e802cda0 con 0x7f10f00cf8a0 2024-03-23T09:49:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.292+0000 7f10ef7fe640 1 -- 172.21.15.47:0/3608053115 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8034660 con 0x7f10f00cf8a0 2024-03-23T09:49:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.292+0000 7f10f7199640 1 -- 172.21.15.47:0/3608053115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f00cf8a0 msgr2=0x7f10f0167b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.292+0000 7f10f7199640 1 --2- 172.21.15.47:0/3608053115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f00cf8a0 0x7f10f0167b00 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009dc0 tx=0x7f10e802be90 comp rx=0 tx=0).stop 2024-03-23T09:49:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.292+0000 7f10f7199640 1 -- 172.21.15.47:0/3608053115 shutdown_connections 2024-03-23T09:49:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.292+0000 7f10f7199640 1 --2- 172.21.15.47:0/3608053115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f00cf8a0 0x7f10f0167b00 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.292+0000 7f10f7199640 1 -- 172.21.15.47:0/3608053115 >> 172.21.15.47:0/3608053115 conn(0x7f10f006b160 msgr2=0x7f10f006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:52.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.292+0000 7f10f7199640 1 -- 172.21.15.47:0/3608053115 shutdown_connections 2024-03-23T09:49:52.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.292+0000 7f10f7199640 1 -- 172.21.15.47:0/3608053115 wait complete. 2024-03-23T09:49:52.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.294+0000 7f10f7199640 1 Processor -- start 2024-03-23T09:49:52.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.294+0000 7f10f7199640 1 -- start start 2024-03-23T09:49:52.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.294+0000 7f10f7199640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f00b8300 0x7f10f0150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.294+0000 7f10f7199640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10f00cf8a0 0x7f10f00c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.294+0000 7f10f7199640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f015cb40 0x7f10f00c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.294+0000 7f10f7199640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00cb130 con 0x7f10f00b8300 2024-03-23T09:49:52.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.294+0000 7f10f7199640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00ca520 con 0x7f10f015cb40 2024-03-23T09:49:52.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.294+0000 7f10f7199640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00ca800 con 0x7f10f00cf8a0 2024-03-23T09:49:52.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.294+0000 7f10f4f35640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f00b8300 0x7f10f0150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10f4f35640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f00b8300 0x7f10f0150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44324/0 (socket says 172.21.15.47:44324) 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10f4f35640 1 -- 172.21.15.47:0/3031228363 learned_addr learned my addr 172.21.15.47:0/3031228363 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10f5736640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f015cb40 0x7f10f00c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10effff640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10f00cf8a0 0x7f10f00c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10f4f35640 1 -- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10f00cf8a0 msgr2=0x7f10f00c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10f4f35640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10f00cf8a0 0x7f10f00c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10f4f35640 1 -- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f015cb40 msgr2=0x7f10f00c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10f4f35640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f015cb40 0x7f10f00c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10f4f35640 1 -- 172.21.15.47:0/3031228363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e8009a70 con 0x7f10f00b8300 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10f4f35640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f00b8300 0x7f10f0150cb0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f10e000c410 tx=0x7f10e000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10ed7fa640 1 -- 172.21.15.47:0/3031228363 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e00035f0 con 0x7f10f00b8300 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10ed7fa640 1 -- 172.21.15.47:0/3031228363 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e0003750 con 0x7f10f00b8300 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10f7199640 1 -- 172.21.15.47:0/3031228363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f00caae0 con 0x7f10f00b8300 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10ed7fa640 1 -- 172.21.15.47:0/3031228363 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0003b40 con 0x7f10f00b8300 2024-03-23T09:49:52.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.295+0000 7f10f7199640 1 -- 172.21.15.47:0/3031228363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f0159e00 con 0x7f10f00b8300 2024-03-23T09:49:52.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.297+0000 7f10ed7fa640 1 -- 172.21.15.47:0/3031228363 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f10e0003ca0 con 0x7f10f00b8300 2024-03-23T09:49:52.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.297+0000 7f10ed7fa640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f10d006ac00 0x7f10d006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.297+0000 7f10ed7fa640 1 -- 172.21.15.47:0/3031228363 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10e0088570 con 0x7f10f00b8300 2024-03-23T09:49:52.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.298+0000 7f10effff640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f10d006ac00 0x7f10d006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.298+0000 7f10effff640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f10d006ac00 0x7f10d006d0c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f10f01f5770 tx=0x7f10d8007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:52.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.298+0000 7f10f9de4740 1 Processor -- start 2024-03-23T09:49:52.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.298+0000 7f10f9de4740 1 -- start start 2024-03-23T09:49:52.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.299+0000 7f10f9de4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b73750 0x55df77b2ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.299+0000 7f10f9de4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b297d0 0x55df77b2b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.299+0000 7f10f9de4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b25c70 0x55df77b55a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.299+0000 7f10f9de4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df77a969b0 con 0x55df77b25c70 2024-03-23T09:49:52.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.299+0000 7f10f9de4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df77a94140 con 0x55df77b73750 2024-03-23T09:49:52.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.299+0000 7f10f9de4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df77a8fa30 con 0x55df77b297d0 2024-03-23T09:49:52.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.299+0000 7f10f5736640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b25c70 0x55df77b55a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.299+0000 7f10f5736640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b25c70 0x55df77b55a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44332/0 (socket says 172.21.15.47:44332) 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.299+0000 7f10f5736640 1 -- 172.21.15.47:0/927254764 learned_addr learned my addr 172.21.15.47:0/927254764 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.299+0000 7f10effff640 1 --2- 172.21.15.47:0/927254764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b297d0 0x55df77b2b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.299+0000 7f10f4f35640 1 --2- 172.21.15.47:0/927254764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b73750 0x55df77b2ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.299+0000 7f10f5736640 1 -- 172.21.15.47:0/927254764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b297d0 msgr2=0x55df77b2b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f5736640 1 --2- 172.21.15.47:0/927254764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b297d0 0x55df77b2b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f5736640 1 -- 172.21.15.47:0/927254764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b73750 msgr2=0x55df77b2ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f5736640 1 --2- 172.21.15.47:0/927254764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b73750 0x55df77b2ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f5736640 1 -- 172.21.15.47:0/927254764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df77a1c2f0 con 0x55df77b25c70 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f5736640 1 --2- 172.21.15.47:0/927254764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b25c70 0x55df77b55a60 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009dc0 tx=0x7f10e802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f218b1489e00fc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10e67fc640 1 -- 172.21.15.47:0/927254764 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e803f650 con 0x55df77b25c70 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10e67fc640 1 -- 172.21.15.47:0/927254764 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e803f7d0 con 0x55df77b25c70 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10e67fc640 1 -- 172.21.15.47:0/927254764 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e803fb80 con 0x55df77b25c70 2024-03-23T09:49:52.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f9de4740 1 -- 172.21.15.47:0/927254764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b25c70 msgr2=0x55df77b55a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f9de4740 1 --2- 172.21.15.47:0/927254764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b25c70 0x55df77b55a60 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009dc0 tx=0x7f10e802e000 comp rx=0 tx=0).stop 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f9de4740 1 -- 172.21.15.47:0/927254764 shutdown_connections 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f9de4740 1 --2- 172.21.15.47:0/927254764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b25c70 0x55df77b55a60 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f9de4740 1 --2- 172.21.15.47:0/927254764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b297d0 0x55df77b2b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f9de4740 1 --2- 172.21.15.47:0/927254764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b73750 0x55df77b2ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f9de4740 1 -- 172.21.15.47:0/927254764 >> 172.21.15.47:0/927254764 conn(0x55df77b27ee0 msgr2=0x55df77b70a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f9de4740 1 -- 172.21.15.47:0/927254764 shutdown_connections 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.300+0000 7f10f9de4740 1 -- 172.21.15.47:0/927254764 wait complete. 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.301+0000 7f10f9de4740 1 -- start start 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.301+0000 7f10f9de4740 1 Processor -- start 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f9de4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b297d0 0x55df77b4c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f9de4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b73750 0x55df77b78150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f9de4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 0x55df77b3a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f9de4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df77a4e090 con 0x55df77b39b80 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f9de4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df77a4dd30 con 0x55df77b297d0 2024-03-23T09:49:52.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f9de4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df77a132a0 con 0x55df77b73750 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f9de4740 10 client.0 _ll_register_callbacks cb 0x55df77a49050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f5736640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 0x55df77b3a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f5736640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 0x55df77b3a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44336/0 (socket says 172.21.15.47:44336) 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f5736640 1 -- 172.21.15.47:0/3295982853 learned_addr learned my addr 172.21.15.47:0/3295982853 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10effff640 1 --2- 172.21.15.47:0/3295982853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b73750 0x55df77b78150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f4f35640 1 --2- 172.21.15.47:0/3295982853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b297d0 0x55df77b4c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f5736640 1 -- 172.21.15.47:0/3295982853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b73750 msgr2=0x55df77b78150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f5736640 1 --2- 172.21.15.47:0/3295982853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b73750 0x55df77b78150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f5736640 1 -- 172.21.15.47:0/3295982853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b297d0 msgr2=0x55df77b4c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f5736640 1 --2- 172.21.15.47:0/3295982853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b297d0 0x55df77b4c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.302+0000 7f10f5736640 1 -- 172.21.15.47:0/3295982853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f0160e80 con 0x55df77b39b80 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10f5736640 1 --2- 172.21.15.47:0/3295982853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 0x55df77b3a000 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f10e802c4a0 tx=0x7f10e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cece806d736c8cc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:49:52.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10cf7fe640 1 -- 172.21.15.47:0/3295982853 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8003990 con 0x55df77b39b80 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10cf7fe640 1 -- 172.21.15.47:0/3295982853 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10e8003af0 con 0x55df77b39b80 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10f9de4740 1 -- 172.21.15.47:0/3295982853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df777833c0 con 0x55df77b39b80 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10cf7fe640 1 -- 172.21.15.47:0/3295982853 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10e8003cd0 con 0x55df77b39b80 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10f9de4740 10 client.8640 fetch_fsmap learned FSMap version 65 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10f9de4740 1 -- 172.21.15.47:0/3295982853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55df77a8e0b0 con 0x55df77b39b80 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10cf7fe640 1 -- 172.21.15.47:0/3295982853 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55df777833c0 con 0x55df77b39b80 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10cf7fe640 1 -- 172.21.15.47:0/3295982853 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f10e8004030 con 0x55df77b39b80 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10f9de4740 10 client.8640 fetch_fsmap finished waiting for FSMap version 65 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10f9de4740 10 client.8640 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10f9de4740 20 client.8640 populate_metadata read hostname 'smithi047' 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10f9de4740 1 --2- 172.21.15.47:0/3295982853 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55df77b73fe0 0x55df77b743c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10f9de4740 4 client.8640 mds_command: new command op to 8484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10f9de4740 1 -- 172.21.15.47:0/3295982853 --> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df777946c0 con 0x55df77b73fe0 2024-03-23T09:49:52.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.303+0000 7f10f4f35640 1 --2- 172.21.15.47:0/3295982853 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55df77b73fe0 0x55df77b743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.304+0000 7f10cf7fe640 10 client.8640 ms_handle_connect on v2:172.21.15.47:6833/2388298413 2024-03-23T09:49:52.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.304+0000 7f10f4f35640 1 --2- 172.21.15.47:0/3295982853 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55df77b73fe0 0x55df77b743c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35e91c5c5ee642d6 server_cookie=8acbd3106beece2f in_seq=0 out_seq=0 2024-03-23T09:49:52.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.304+0000 7f10cf7fe640 1 -- 172.21.15.47:0/3295982853 <== mds.0 v2:172.21.15.47:6833/2388298413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55df777946c0 con 0x55df77b73fe0 2024-03-23T09:49:52.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.304+0000 7f10cf7fe640 10 client.8640 handle_command_reply: tid=0 2024-03-23T09:49:52.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.304+0000 7f10f9de4740 1 client.8640 shutdown 2024-03-23T09:49:52.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.304+0000 7f10f9de4740 1 -- 172.21.15.47:0/3295982853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 msgr2=0x55df77b3a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.304+0000 7f10f9de4740 1 --2- 172.21.15.47:0/3295982853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 0x55df77b3a000 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f10e802c4a0 tx=0x7f10e8004ab0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.305+0000 7f10f9de4740 1 -- 172.21.15.47:0/3295982853 shutdown_connections 2024-03-23T09:49:52.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.305+0000 7f10f9de4740 1 --2- 172.21.15.47:0/3295982853 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55df77b73fe0 0x55df77b743c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.305+0000 7f10f9de4740 1 --2- 172.21.15.47:0/3295982853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 0x55df77b3a000 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.305+0000 7f10f9de4740 1 --2- 172.21.15.47:0/3295982853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b73750 0x55df77b78150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.305+0000 7f10cf7fe640 0 client.8640 ms_handle_reset on v2:172.21.15.47:6833/2388298413 2024-03-23T09:49:52.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.305+0000 7f10f9de4740 1 --2- 172.21.15.47:0/3295982853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b297d0 0x55df77b4c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.305+0000 7f10f9de4740 1 -- 172.21.15.47:0/3295982853 >> 172.21.15.47:0/3295982853 conn(0x55df77b25c70 msgr2=0x55df77b25320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:52.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.305+0000 7f10f9de4740 1 -- 172.21.15.47:0/3295982853 shutdown_connections 2024-03-23T09:49:52.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.305+0000 7f10f9de4740 1 -- 172.21.15.47:0/3295982853 wait complete. 2024-03-23T09:49:52.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.305+0000 7f10f9de4740 20 client.8640 trim_cache size 0 max 16384 2024-03-23T09:49:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.320+0000 7f10f9de4740 1 Processor -- start 2024-03-23T09:49:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.320+0000 7f10f9de4740 1 -- start start 2024-03-23T09:49:52.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.320+0000 7f10f9de4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b297d0 0x55df77b59c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.320+0000 7f10f9de4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 0x55df77b53940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.320+0000 7f10f9de4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b73750 0x55df77b29e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.320+0000 7f10f9de4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df77b2b4a0 con 0x55df77b39b80 2024-03-23T09:49:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.320+0000 7f10f9de4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df77b24cf0 con 0x55df77b297d0 2024-03-23T09:49:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.320+0000 7f10f9de4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df77b6ffd0 con 0x55df77b73750 2024-03-23T09:49:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.320+0000 7f10effff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 0x55df77b53940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.321+0000 7f10effff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 0x55df77b53940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44344/0 (socket says 172.21.15.47:44344) 2024-03-23T09:49:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.321+0000 7f10effff640 1 -- 172.21.15.47:0/3333728135 learned_addr learned my addr 172.21.15.47:0/3333728135 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.321+0000 7f10effff640 1 -- 172.21.15.47:0/3333728135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b73750 msgr2=0x55df77b29e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.321+0000 7f10effff640 1 --2- 172.21.15.47:0/3333728135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b73750 0x55df77b29e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.321+0000 7f10effff640 1 -- 172.21.15.47:0/3333728135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b297d0 msgr2=0x55df77b59c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.321+0000 7f10effff640 1 --2- 172.21.15.47:0/3333728135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b297d0 0x55df77b59c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.321+0000 7f10effff640 1 -- 172.21.15.47:0/3333728135 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df77a30430 con 0x55df77b39b80 2024-03-23T09:49:52.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.321+0000 7f10effff640 1 --2- 172.21.15.47:0/3333728135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 0x55df77b53940 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f10f006c3f0 tx=0x7f10d8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e0b9e32818a186 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:52.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.321+0000 7f10e6ffd640 1 -- 172.21.15.47:0/3333728135 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d8001cf0 con 0x55df77b39b80 2024-03-23T09:49:52.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.321+0000 7f10e6ffd640 1 -- 172.21.15.47:0/3333728135 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d801b030 con 0x55df77b39b80 2024-03-23T09:49:52.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.321+0000 7f10e6ffd640 1 -- 172.21.15.47:0/3333728135 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d8015110 con 0x55df77b39b80 2024-03-23T09:49:52.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.322+0000 7f10f9de4740 1 -- 172.21.15.47:0/3333728135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 msgr2=0x55df77b53940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.322+0000 7f10f9de4740 1 --2- 172.21.15.47:0/3333728135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 0x55df77b53940 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f10f006c3f0 tx=0x7f10d8012000 comp rx=0 tx=0).stop 2024-03-23T09:49:52.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.322+0000 7f10f9de4740 1 -- 172.21.15.47:0/3333728135 shutdown_connections 2024-03-23T09:49:52.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.322+0000 7f10f9de4740 1 --2- 172.21.15.47:0/3333728135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b73750 0x55df77b29e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.322+0000 7f10f9de4740 1 --2- 172.21.15.47:0/3333728135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b39b80 0x55df77b53940 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.322+0000 7f10f9de4740 1 --2- 172.21.15.47:0/3333728135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b297d0 0x55df77b59c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.322+0000 7f10f9de4740 1 -- 172.21.15.47:0/3333728135 >> 172.21.15.47:0/3333728135 conn(0x55df77b25c70 msgr2=0x55df77b77f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:52.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.322+0000 7f10f9de4740 1 -- 172.21.15.47:0/3333728135 shutdown_connections 2024-03-23T09:49:52.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.322+0000 7f10f9de4740 1 -- 172.21.15.47:0/3333728135 wait complete. 2024-03-23T09:49:52.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.322+0000 7f10f9de4740 1 -- start start 2024-03-23T09:49:52.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.322+0000 7f10f9de4740 1 Processor -- start 2024-03-23T09:49:52.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.323+0000 7f10f9de4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b297d0 0x55df77bd6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.323+0000 7f10f9de4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b39b80 0x55df77b26050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.323+0000 7f10f9de4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b73750 0x55df77b26590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.323+0000 7f10f9de4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df77bd83c0 con 0x55df77b297d0 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.323+0000 7f10f9de4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df77b52780 con 0x55df77b73750 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.323+0000 7f10f9de4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df77b89400 con 0x55df77b39b80 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.323+0000 7f10f9de4740 10 client.0 _ll_register_callbacks cb 0x7f10e8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.323+0000 7f10f4f35640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b297d0 0x55df77bd6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.323+0000 7f10f4f35640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b297d0 0x55df77bd6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44360/0 (socket says 172.21.15.47:44360) 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.323+0000 7f10f4f35640 1 -- 172.21.15.47:0/2723191754 learned_addr learned my addr 172.21.15.47:0/2723191754 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.323+0000 7f10effff640 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b39b80 0x55df77b26050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.323+0000 7f10f5736640 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b73750 0x55df77b26590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f4f35640 1 -- 172.21.15.47:0/2723191754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b39b80 msgr2=0x55df77b26050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f4f35640 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b39b80 0x55df77b26050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f4f35640 1 -- 172.21.15.47:0/2723191754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b73750 msgr2=0x55df77b26590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:52.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f4f35640 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b73750 0x55df77b26590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f4f35640 1 -- 172.21.15.47:0/2723191754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d800f040 con 0x55df77b297d0 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f5736640 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b73750 0x55df77b26590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f4f35640 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b297d0 0x55df77bd6d40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f10e005f2d0 tx=0x7f10e0005560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9044a94e567e7bb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10cf7fe640 1 -- 172.21.15.47:0/2723191754 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e005b240 con 0x55df77b297d0 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10cf7fe640 1 -- 172.21.15.47:0/2723191754 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10e005f800 con 0x55df77b297d0 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f9de4740 1 -- 172.21.15.47:0/2723191754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10e803f7d0 con 0x55df77b297d0 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10cf7fe640 1 -- 172.21.15.47:0/2723191754 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10e0053b20 con 0x55df77b297d0 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f9de4740 10 client.8646 fetch_fsmap learned FSMap version 65 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f9de4740 1 -- 172.21.15.47:0/2723191754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55df77a55b40 con 0x55df77b297d0 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10cf7fe640 1 -- 172.21.15.47:0/2723191754 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e803f7d0 con 0x55df77b297d0 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10cf7fe640 1 -- 172.21.15.47:0/2723191754 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f10e007a080 con 0x55df77b297d0 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f9de4740 10 client.8646 fetch_fsmap finished waiting for FSMap version 65 2024-03-23T09:49:52.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f9de4740 10 client.8646 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f9de4740 20 client.8646 populate_metadata read hostname 'smithi047' 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f9de4740 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55df77b66820 0x55df77b66c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f9de4740 4 client.8646 mds_command: new command op to 8484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.324+0000 7f10f9de4740 1 -- 172.21.15.47:0/2723191754 --> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55df777e7d10 con 0x55df77b66820 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10effff640 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55df77b66820 0x55df77b66c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10cf7fe640 10 client.8646 ms_handle_connect on v2:172.21.15.47:6833/2388298413 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10effff640 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55df77b66820 0x55df77b66c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e971a505224f0409 server_cookie=a4c365139d36e855 in_seq=0 out_seq=0 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10cf7fe640 1 -- 172.21.15.47:0/2723191754 <== mds.0 v2:172.21.15.47:6833/2388298413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55df777e7d10 con 0x55df77b66820 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10cf7fe640 10 client.8646 handle_command_reply: tid=0 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10f9de4740 1 client.8646 shutdown 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10f9de4740 1 -- 172.21.15.47:0/2723191754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b297d0 msgr2=0x55df77bd6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10f9de4740 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b297d0 0x55df77bd6d40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f10e005f2d0 tx=0x7f10e0005560 comp rx=0 tx=0).stop 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10f9de4740 1 -- 172.21.15.47:0/2723191754 shutdown_connections 2024-03-23T09:49:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10f9de4740 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55df77b66820 0x55df77b66c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10f9de4740 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df77b73750 0x55df77b26590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10f9de4740 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55df77b39b80 0x55df77b26050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10cf7fe640 0 client.8646 ms_handle_reset on v2:172.21.15.47:6833/2388298413 2024-03-23T09:49:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10f9de4740 1 --2- 172.21.15.47:0/2723191754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df77b297d0 0x55df77bd6d40 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10f9de4740 1 -- 172.21.15.47:0/2723191754 >> 172.21.15.47:0/2723191754 conn(0x55df77b25c70 msgr2=0x55df77b29e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10f9de4740 1 -- 172.21.15.47:0/2723191754 shutdown_connections 2024-03-23T09:49:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10f9de4740 1 -- 172.21.15.47:0/2723191754 wait complete. 2024-03-23T09:49:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.325+0000 7f10f9de4740 20 client.8646 trim_cache size 0 max 16384 2024-03-23T09:49:52.332 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:49:52.332 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:49:52.333 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:49:52.333 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 8484, 2024-03-23T09:49:52.333 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:49:52.333 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:49:52.333 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:49:52.333 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 9.0771298419999997, 2024-03-23T09:49:52.333 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 65, 2024-03-23T09:49:52.333 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 101, 2024-03-23T09:49:52.333 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 101, 2024-03-23T09:49:52.333 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 13.844757981000001 2024-03-23T09:49:52.333 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:49:52.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.326+0000 7f10f7199640 1 -- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f10d006ac00 msgr2=0x7f10d006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.326+0000 7f10f7199640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f10d006ac00 0x7f10d006d0c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f10f01f5770 tx=0x7f10d8007c00 comp rx=0 tx=0).stop 2024-03-23T09:49:52.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.326+0000 7f10f7199640 1 -- 172.21.15.47:0/3031228363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f00b8300 msgr2=0x7f10f0150cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.326+0000 7f10f7199640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f00b8300 0x7f10f0150cb0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f10e000c410 tx=0x7f10e000c8d0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.326+0000 7f10f7199640 1 -- 172.21.15.47:0/3031228363 shutdown_connections 2024-03-23T09:49:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.326+0000 7f10f7199640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f10d006ac00 0x7f10d006d0c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.326+0000 7f10f7199640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f015cb40 0x7f10f00c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.326+0000 7f10f7199640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10f00cf8a0 0x7f10f00c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.326+0000 7f10f7199640 1 --2- 172.21.15.47:0/3031228363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f00b8300 0x7f10f0150cb0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.326+0000 7f10f7199640 1 -- 172.21.15.47:0/3031228363 >> 172.21.15.47:0/3031228363 conn(0x7f10f006b160 msgr2=0x7f10f00c6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.326+0000 7f10f7199640 1 -- 172.21.15.47:0/3031228363 shutdown_connections 2024-03-23T09:49:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.326+0000 7f10f7199640 1 -- 172.21.15.47:0/3031228363 wait complete. 2024-03-23T09:49:52.337 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:49:52.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.438+0000 7fb5ec7b9640 1 Processor -- start 2024-03-23T09:49:52.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.438+0000 7fb5ec7b9640 1 -- start start 2024-03-23T09:49:52.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ec7b9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5e415cc40 0x7fb5e4160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ec7b9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb5e4160c90 0x7fb5e40cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ec7b9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e40cdbf0 0x7fb5e40b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ec7b9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e40ce040 con 0x7fb5e415cc40 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ec7b9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e40d09d0 con 0x7fb5e4160c90 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ec7b9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40b5a80 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ea555640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5e415cc40 0x7fb5e4160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ea555640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5e415cc40 0x7fb5e4160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44374/0 (socket says 172.21.15.47:44374) 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ea555640 1 -- 172.21.15.47:0/2133393731 learned_addr learned my addr 172.21.15.47:0/2133393731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ead56640 1 --2- 172.21.15.47:0/2133393731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e40cdbf0 0x7fb5e40b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5e9d54640 1 --2- 172.21.15.47:0/2133393731 >> v2:172.21.15.175:3300/0 conn(0x7fb5e4160c90 0x7fb5e40cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ea555640 1 -- 172.21.15.47:0/2133393731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e40cdbf0 msgr2=0x7fb5e40b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ea555640 1 --2- 172.21.15.47:0/2133393731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e40cdbf0 0x7fb5e40b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ea555640 1 -- 172.21.15.47:0/2133393731 >> v2:172.21.15.175:3300/0 conn(0x7fb5e4160c90 msgr2=0x7fb5e40cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:52.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ea555640 1 --2- 172.21.15.47:0/2133393731 >> v2:172.21.15.175:3300/0 conn(0x7fb5e4160c90 0x7fb5e40cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.439+0000 7fb5ea555640 1 -- 172.21.15.47:0/2133393731 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e415d020 con 0x7fb5e415cc40 2024-03-23T09:49:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.440+0000 7fb5ea555640 1 --2- 172.21.15.47:0/2133393731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e415cc40 0x7fb5e4160750 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40cd2f0 tx=0x7fb5e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a197b37dd6b5cdb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.440+0000 7fb5e9553640 1 -- 172.21.15.47:0/2133393731 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0031870 con 0x7fb5e415cc40 2024-03-23T09:49:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.440+0000 7fb5e9553640 1 -- 172.21.15.47:0/2133393731 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e00319d0 con 0x7fb5e415cc40 2024-03-23T09:49:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.440+0000 7fb5e9553640 1 -- 172.21.15.47:0/2133393731 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0031d60 con 0x7fb5e415cc40 2024-03-23T09:49:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.441+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2133393731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e415cc40 msgr2=0x7fb5e4160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.441+0000 7fb5ec7b9640 1 --2- 172.21.15.47:0/2133393731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e415cc40 0x7fb5e4160750 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40cd2f0 tx=0x7fb5e002b910 comp rx=0 tx=0).stop 2024-03-23T09:49:52.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.441+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2133393731 shutdown_connections 2024-03-23T09:49:52.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.441+0000 7fb5ec7b9640 1 --2- 172.21.15.47:0/2133393731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e40cdbf0 0x7fb5e40b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.441+0000 7fb5ec7b9640 1 --2- 172.21.15.47:0/2133393731 >> v2:172.21.15.175:3300/0 conn(0x7fb5e4160c90 0x7fb5e40cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.441+0000 7fb5ec7b9640 1 --2- 172.21.15.47:0/2133393731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e415cc40 0x7fb5e4160750 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.441+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2133393731 >> 172.21.15.47:0/2133393731 conn(0x7fb5e406b160 msgr2=0x7fb5e406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:52.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.441+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2133393731 shutdown_connections 2024-03-23T09:49:52.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.441+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2133393731 wait complete. 2024-03-23T09:49:52.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.442+0000 7fb5ec7b9640 1 Processor -- start 2024-03-23T09:49:52.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.442+0000 7fb5ec7b9640 1 -- start start 2024-03-23T09:49:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ec7b9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e40cdbf0 0x7fb5e415f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ec7b9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5e415cc40 0x7fb5e4150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ec7b9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e4160c90 0x7fb5e41434d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ec7b9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e415f090 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ec7b9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e415e010 con 0x7fb5e415cc40 2024-03-23T09:49:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ec7b9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e415ea40 con 0x7fb5e4160c90 2024-03-23T09:49:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ea555640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e40cdbf0 0x7fb5e415f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ea555640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e40cdbf0 0x7fb5e415f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44386/0 (socket says 172.21.15.47:44386) 2024-03-23T09:49:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ea555640 1 -- 172.21.15.47:0/2463247964 learned_addr learned my addr 172.21.15.47:0/2463247964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ead56640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e4160c90 0x7fb5e41434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5e9d54640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5e415cc40 0x7fb5e4150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ea555640 1 -- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e4160c90 msgr2=0x7fb5e41434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ea555640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e4160c90 0x7fb5e41434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ea555640 1 -- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5e415cc40 msgr2=0x7fb5e4150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ea555640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5e415cc40 0x7fb5e4150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ea555640 1 -- 172.21.15.47:0/2463247964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e00094d0 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ead56640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e4160c90 0x7fb5e41434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5e9d54640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5e415cc40 0x7fb5e4150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ea555640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e40cdbf0 0x7fb5e415f520 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb5e002b950 tx=0x7fb5e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5daffd640 1 -- 172.21.15.47:0/2463247964 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e003ad30 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5daffd640 1 -- 172.21.15.47:0/2463247964 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e00443f0 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2463247964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e415ed20 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5daffd640 1 -- 172.21.15.47:0/2463247964 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e00447a0 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.443+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2463247964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e4150d80 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.444+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2463247964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e4072360 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.445+0000 7fb5daffd640 1 -- 172.21.15.47:0/2463247964 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb5e004d450 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.445+0000 7fb5daffd640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5c806ac00 0x7fb5c806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.445+0000 7fb5daffd640 1 -- 172.21.15.47:0/2463247964 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5e002d040 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.445+0000 7fb5e9d54640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5c806ac00 0x7fb5c806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.446+0000 7fb5e9d54640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5c806ac00 0x7fb5c806d0c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4006fd0 tx=0x7fb5d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:52.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.450+0000 7fb5daffd640 1 -- 172.21.15.47:0/2463247964 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb5e0076bd0 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.669+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2463247964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb5e4066be0 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.670+0000 7fb5daffd640 1 -- 172.21.15.47:0/2463247964 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb5e0035030 con 0x7fb5e40cdbf0 2024-03-23T09:49:52.670 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:49:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.671+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5c806ac00 msgr2=0x7fb5c806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.671+0000 7fb5ec7b9640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5c806ac00 0x7fb5c806d0c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4006fd0 tx=0x7fb5d4008040 comp rx=0 tx=0).stop 2024-03-23T09:49:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.671+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2463247964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e40cdbf0 msgr2=0x7fb5e415f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.671+0000 7fb5ec7b9640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e40cdbf0 0x7fb5e415f520 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb5e002b950 tx=0x7fb5e00046c0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.671+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2463247964 shutdown_connections 2024-03-23T09:49:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.671+0000 7fb5ec7b9640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb5c806ac00 0x7fb5c806d0c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.671+0000 7fb5ec7b9640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e4160c90 0x7fb5e41434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.671+0000 7fb5ec7b9640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5e415cc40 0x7fb5e4150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.671+0000 7fb5ec7b9640 1 --2- 172.21.15.47:0/2463247964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e40cdbf0 0x7fb5e415f520 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.671+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2463247964 >> 172.21.15.47:0/2463247964 conn(0x7fb5e406b160 msgr2=0x7fb5e40c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.671+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2463247964 shutdown_connections 2024-03-23T09:49:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:52.671+0000 7fb5ec7b9640 1 -- 172.21.15.47:0/2463247964 wait complete. 2024-03-23T09:49:52.682 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:49:52.682 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:49:52.682 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:49:52.682 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:49:52.682 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:49:52.683 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:49:52.683 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:49:52.683 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:49:52.683 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:49:52.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:52.683 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:49:52.698 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:49:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:49:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:49:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:49:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:49:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:49:52.698 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:49:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:49:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:49:52.698 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:49:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:49:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:49:52.698 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42146sec preferred_lft 42146sec 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:49:52.699 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:49:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:49:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:49:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:49:52.700 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:49:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:49:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:49:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:49:52.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:52.700 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:49:52.763 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:49:52.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:52.764 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:49:52.820 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:49:52.820 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:49:52.820 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:49:52.820 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:49:52.820 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:49:52.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:52.821 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:49:52.821 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:49:52.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:49:52.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:49:52.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:49:52.821 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:49:52.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:52.917 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:49:52.931 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:49:52.931 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:49:52.931 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:49:52.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:52.931 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:49:52.997 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:49:52.997 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:49:52.997 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:49:53.051 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:49:53.051 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:49:53.052 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:49:53.106 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:49:53.173 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:49:53.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:49:53.238 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:49:53.239 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:49:53.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:53.240 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:49:53.294 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:49:53.294 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:49:53.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:49:53.361 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:49:53.362 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:49:53.420 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:49:53.420+0000 7fa793849480 -1 init, newargv = 0x562e26f09d10 newargc=15 2024-03-23T09:49:53.420 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[73846]: starting ceph client 2024-03-23T09:49:53.425 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[73846]: starting fuse 2024-03-23T09:49:53.429 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:49:53.430 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:49:53.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:53.431 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:49:53.485 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:49:53.485 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:49:53.485 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:49:53.485 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:49:53.485 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:49:53.486 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:49:53.487 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:49:53.487 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:49:53.487 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:49:53.487 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:49:53.487 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:49:53.487 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:49:53.487 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:49:53.487 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:49:53.601 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.73846.asok 2024-03-23T09:49:53.787 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.73846.asok 2024-03-23T09:49:53.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:53.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.73846.asok status 2024-03-23T09:49:53.924 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "73846", 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 8655, 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:49:53.925 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 8655 2024-03-23T09:49:53.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:53.926 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:49:53.926 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:49:53.926 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:49:53.926 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1470040494 2024-03-23T09:49:53.926 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:49:53.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:53.926 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:49:53.926 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:49:53.926 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:49:53.926 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1470040494 2024-03-23T09:49:53.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:49:53.926 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.8655 192.168.0.1:0/1470040494", 2024-03-23T09:49:53.927 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/1470040494", 2024-03-23T09:49:53.927 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:49:53.927 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 65, 2024-03-23T09:49:53.927 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 101, 2024-03-23T09:49:53.927 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:49:53.927 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:49:53.927 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:49:53.927 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:49:53.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:53.933 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:49:53.933 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:49:53.948 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:49:53.948 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:49:53.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:53.948 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:49:54.015 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:49:54.015 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:49:54.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:54.070 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-23T09:49:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:6+0 records in 2024-03-23T09:49:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:6+0 records out 2024-03-23T09:49:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0428471 s, 147 MB/s 2024-03-23T09:49:54.172 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:49:54.173 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:49:54.187 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:49:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.328+0000 7fc0f6b7d640 1 Processor -- start 2024-03-23T09:49:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.328+0000 7fc0f6b7d640 1 -- start start 2024-03-23T09:49:54.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.328+0000 7fc0f6b7d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0f015ca90 0x7fc0f0160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.328+0000 7fc0f6b7d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0f00b8350 0x7fc0f00cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.329+0000 7fc0f6b7d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f00d11e0 con 0x7fc0f00cdc40 2024-03-23T09:49:54.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.329+0000 7fc0f6b7d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f00ce0d0 con 0x7fc0f00b8350 2024-03-23T09:49:54.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.329+0000 7fc0f6b7d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f00d2680 con 0x7fc0f015ca90 2024-03-23T09:49:54.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.329+0000 7fc0effff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0f00b8350 0x7fc0f00cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.329+0000 7fc0f511a640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc0f00cdc40 0x7fc0f00b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47558/0 (socket says 172.21.15.47:47558) 2024-03-23T09:49:54.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.329+0000 7fc0f511a640 1 -- 172.21.15.47:0/1431136023 learned_addr learned my addr 172.21.15.47:0/1431136023 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:54.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.329+0000 7fc0f4919640 1 --2- 172.21.15.47:0/1431136023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0f015ca90 0x7fc0f0160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.329+0000 7fc0effff640 1 -- 172.21.15.47:0/1431136023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0f015ca90 msgr2=0x7fc0f0160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:54.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.329+0000 7fc0effff640 1 --2- 172.21.15.47:0/1431136023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0f015ca90 0x7fc0f0160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.329+0000 7fc0effff640 1 -- 172.21.15.47:0/1431136023 >> v1:172.21.15.47:6789/0 conn(0x7fc0f00cdc40 legacy=0x7fc0f00b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.329+0000 7fc0effff640 1 -- 172.21.15.47:0/1431136023 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f0160f60 con 0x7fc0f00b8350 2024-03-23T09:49:54.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.329+0000 7fc0ef7fe640 1 -- 172.21.15.47:0/1431136023 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763033709 0 0) 0x7fc0f00d11e0 con 0x7fc0f00cdc40 2024-03-23T09:49:54.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.329+0000 7fc0f4919640 1 --2- 172.21.15.47:0/1431136023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0f015ca90 0x7fc0f0160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:49:54.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.330+0000 7fc0effff640 1 --2- 172.21.15.47:0/1431136023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f00b8350 0x7fc0f00cd700 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0097a0 tx=0x7fc0dc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6f2ec2091126d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:54.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.330+0000 7fc0ef7fe640 1 -- 172.21.15.47:0/1431136023 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc02e070 con 0x7fc0f00b8350 2024-03-23T09:49:54.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.330+0000 7fc0ef7fe640 1 -- 172.21.15.47:0/1431136023 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0dc0315a0 con 0x7fc0f00b8350 2024-03-23T09:49:54.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.330+0000 7fc0ef7fe640 1 -- 172.21.15.47:0/1431136023 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc031930 con 0x7fc0f00b8350 2024-03-23T09:49:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.330+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/1431136023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f00b8350 msgr2=0x7fc0f00cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.330+0000 7fc0f6b7d640 1 --2- 172.21.15.47:0/1431136023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f00b8350 0x7fc0f00cd700 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0097a0 tx=0x7fc0dc02b840 comp rx=0 tx=0).stop 2024-03-23T09:49:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.330+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/1431136023 shutdown_connections 2024-03-23T09:49:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.330+0000 7fc0f6b7d640 1 --2- 172.21.15.47:0/1431136023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f00b8350 0x7fc0f00cd700 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.330+0000 7fc0f6b7d640 1 --2- 172.21.15.47:0/1431136023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0f015ca90 0x7fc0f0160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.330+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/1431136023 >> 172.21.15.47:0/1431136023 conn(0x7fc0f00659d0 msgr2=0x7fc0f0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.330+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/1431136023 shutdown_connections 2024-03-23T09:49:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.330+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/1431136023 wait complete. 2024-03-23T09:49:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.331+0000 7fc0f6b7d640 1 Processor -- start 2024-03-23T09:49:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.332+0000 7fc0f6b7d640 1 -- start start 2024-03-23T09:49:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.332+0000 7fc0f6b7d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f00b8350 0x7fc0f015af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.332+0000 7fc0f6b7d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0f00cdc40 0x7fc0f00b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.332+0000 7fc0f6b7d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0f015ca90 0x7fc0f01466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.332+0000 7fc0f6b7d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f00cac90 con 0x7fc0f00b8350 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.332+0000 7fc0f6b7d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f00caf30 con 0x7fc0f015ca90 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.332+0000 7fc0f6b7d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f00cb210 con 0x7fc0f00cdc40 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.332+0000 7fc0f4919640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f00b8350 0x7fc0f015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.332+0000 7fc0f4919640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f00b8350 0x7fc0f015af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44392/0 (socket says 172.21.15.47:44392) 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.332+0000 7fc0f4919640 1 -- 172.21.15.47:0/2545539024 learned_addr learned my addr 172.21.15.47:0/2545539024 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.333+0000 7fc0effff640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0f00cdc40 0x7fc0f00b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.333+0000 7fc0f511a640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0f015ca90 0x7fc0f01466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.333+0000 7fc0f4919640 1 -- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0f00cdc40 msgr2=0x7fc0f00b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.333+0000 7fc0f4919640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0f00cdc40 0x7fc0f00b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.333+0000 7fc0f4919640 1 -- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0f015ca90 msgr2=0x7fc0f01466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.333+0000 7fc0f4919640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0f015ca90 0x7fc0f01466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.333+0000 7fc0f4919640 1 -- 172.21.15.47:0/2545539024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc009400 con 0x7fc0f00b8350 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.333+0000 7fc0f4919640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f00b8350 0x7fc0f015af60 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400c7f0 tx=0x7fc0e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.333+0000 7fc0ed7fa640 1 -- 172.21.15.47:0/2545539024 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4007a30 con 0x7fc0f00b8350 2024-03-23T09:49:54.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.333+0000 7fc0ed7fa640 1 -- 172.21.15.47:0/2545539024 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e4007b90 con 0x7fc0f00b8350 2024-03-23T09:49:54.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.333+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/2545539024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f00cb4f0 con 0x7fc0f00b8350 2024-03-23T09:49:54.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.333+0000 7fc0ed7fa640 1 -- 172.21.15.47:0/2545539024 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400ce40 con 0x7fc0f00b8350 2024-03-23T09:49:54.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.333+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/2545539024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f00cbab0 con 0x7fc0f00b8350 2024-03-23T09:49:54.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.334+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/2545539024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f0072360 con 0x7fc0f00b8350 2024-03-23T09:49:54.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.335+0000 7fc0ed7fa640 1 -- 172.21.15.47:0/2545539024 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc0e40113f0 con 0x7fc0f00b8350 2024-03-23T09:49:54.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.335+0000 7fc0ed7fa640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc0d406ac00 0x7fc0d406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.335+0000 7fc0ed7fa640 1 -- 172.21.15.47:0/2545539024 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0e4089480 con 0x7fc0f00b8350 2024-03-23T09:49:54.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.335+0000 7fc0effff640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc0d406ac00 0x7fc0d406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.336+0000 7fc0effff640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc0d406ac00 0x7fc0d406d0c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc036000 tx=0x7fc0dc03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.339+0000 7fc0ed7fa640 1 -- 172.21.15.47:0/2545539024 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc0e40546f0 con 0x7fc0f00b8350 2024-03-23T09:49:54.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.519+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/2545539024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0f0066be0 con 0x7fc0f00b8350 2024-03-23T09:49:54.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.520+0000 7fc0ed7fa640 1 -- 172.21.15.47:0/2545539024 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16043 (secure 0 0 0) 0x7fc0e4057d10 con 0x7fc0f00b8350 2024-03-23T09:49:54.520 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:54.520 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":101,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:49:44.249270+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:49:41.158056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-23T09:49:42.210388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:49:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.522+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc0d406ac00 msgr2=0x7fc0d406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.522+0000 7fc0f6b7d640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc0d406ac00 0x7fc0d406d0c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc036000 tx=0x7fc0dc03d040 comp rx=0 tx=0).stop 2024-03-23T09:49:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.522+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/2545539024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f00b8350 msgr2=0x7fc0f015af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.522+0000 7fc0f6b7d640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f00b8350 0x7fc0f015af60 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400c7f0 tx=0x7fc0e400cbf0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.522+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/2545539024 shutdown_connections 2024-03-23T09:49:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.522+0000 7fc0f6b7d640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc0d406ac00 0x7fc0d406d0c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.522+0000 7fc0f6b7d640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0f015ca90 0x7fc0f01466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.522+0000 7fc0f6b7d640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0f00cdc40 0x7fc0f00b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.522+0000 7fc0f6b7d640 1 --2- 172.21.15.47:0/2545539024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f00b8350 0x7fc0f015af60 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.522+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/2545539024 >> 172.21.15.47:0/2545539024 conn(0x7fc0f00659d0 msgr2=0x7fc0f0156e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.522+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/2545539024 shutdown_connections 2024-03-23T09:49:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.522+0000 7fc0f6b7d640 1 -- 172.21.15.47:0/2545539024 wait complete. 2024-03-23T09:49:54.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:49:41.158056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:49:42.210388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:49:54.533 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:49:54.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.637+0000 7f5257682640 1 Processor -- start 2024-03-23T09:49:54.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.637+0000 7f5257682640 1 -- start start 2024-03-23T09:49:54.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f5257682640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f525015c990 0x7f5250160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f5257682640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f52500b8350 0x7f52500cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f5257682640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52500cdc40 0x7f52500b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f5257682640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52500ce0b0 con 0x7f525015c990 2024-03-23T09:49:54.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f5257682640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5250161630 con 0x7f52500b8350 2024-03-23T09:49:54.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f5257682640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52500d1030 con 0x7f52500cdc40 2024-03-23T09:49:54.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f525541e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f525015c990 0x7f5250160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f525541e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f525015c990 0x7f5250160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55152/0 (socket says 172.21.15.47:55152) 2024-03-23T09:49:54.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f525541e640 1 -- 172.21.15.47:0/3423574233 learned_addr learned my addr 172.21.15.47:0/3423574233 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:54.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f5254c1d640 1 --2- 172.21.15.47:0/3423574233 >> v2:172.21.15.175:3300/0 conn(0x7f52500b8350 0x7f52500cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f5255c1f640 1 --2- 172.21.15.47:0/3423574233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52500cdc40 0x7f52500b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f525541e640 1 -- 172.21.15.47:0/3423574233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52500cdc40 msgr2=0x7f52500b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:54.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f525541e640 1 --2- 172.21.15.47:0/3423574233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52500cdc40 0x7f52500b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f525541e640 1 -- 172.21.15.47:0/3423574233 >> v2:172.21.15.175:3300/0 conn(0x7f52500b8350 msgr2=0x7f52500cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:54.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f525541e640 1 --2- 172.21.15.47:0/3423574233 >> v2:172.21.15.175:3300/0 conn(0x7f52500b8350 0x7f52500cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.638+0000 7f525541e640 1 -- 172.21.15.47:0/3423574233 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52500cfe10 con 0x7f525015c990 2024-03-23T09:49:54.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.639+0000 7f525541e640 1 --2- 172.21.15.47:0/3423574233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f525015c990 0x7f5250160690 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f525008ac70 tx=0x7f524c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8586a388805cb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:54.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.639+0000 7f5247fff640 1 -- 172.21.15.47:0/3423574233 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c0317e0 con 0x7f525015c990 2024-03-23T09:49:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.639+0000 7f5247fff640 1 -- 172.21.15.47:0/3423574233 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c031940 con 0x7f525015c990 2024-03-23T09:49:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.639+0000 7f5247fff640 1 -- 172.21.15.47:0/3423574233 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c031cd0 con 0x7f525015c990 2024-03-23T09:49:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.639+0000 7f5257682640 1 -- 172.21.15.47:0/3423574233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f525015c990 msgr2=0x7f5250160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.639+0000 7f5257682640 1 --2- 172.21.15.47:0/3423574233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f525015c990 0x7f5250160690 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f525008ac70 tx=0x7f524c02b880 comp rx=0 tx=0).stop 2024-03-23T09:49:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.639+0000 7f5257682640 1 -- 172.21.15.47:0/3423574233 shutdown_connections 2024-03-23T09:49:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.639+0000 7f5257682640 1 --2- 172.21.15.47:0/3423574233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52500cdc40 0x7f52500b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.639+0000 7f5257682640 1 --2- 172.21.15.47:0/3423574233 >> v2:172.21.15.175:3300/0 conn(0x7f52500b8350 0x7f52500cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.639+0000 7f5257682640 1 --2- 172.21.15.47:0/3423574233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f525015c990 0x7f5250160690 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.639+0000 7f5257682640 1 -- 172.21.15.47:0/3423574233 >> 172.21.15.47:0/3423574233 conn(0x7f52500659d0 msgr2=0x7f5250065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.639+0000 7f5257682640 1 -- 172.21.15.47:0/3423574233 shutdown_connections 2024-03-23T09:49:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.640+0000 7f5257682640 1 -- 172.21.15.47:0/3423574233 wait complete. 2024-03-23T09:49:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.641+0000 7f5257682640 1 Processor -- start 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.641+0000 7f5257682640 1 -- start start 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.641+0000 7f5257682640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52500b8350 0x7f52500c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.641+0000 7f5257682640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52500cdc40 0x7f52501f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.641+0000 7f5257682640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f525015c990 0x7f525015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.641+0000 7f5257682640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52500c83c0 con 0x7f52500b8350 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.641+0000 7f5257682640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52500c73e0 con 0x7f52500cdc40 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.641+0000 7f5257682640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525015efa0 con 0x7f525015c990 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.641+0000 7f525541e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52500b8350 0x7f52500c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.641+0000 7f525541e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52500b8350 0x7f52500c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55162/0 (socket says 172.21.15.47:55162) 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.641+0000 7f525541e640 1 -- 172.21.15.47:0/2780762842 learned_addr learned my addr 172.21.15.47:0/2780762842 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.642+0000 7f525541e640 1 -- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f525015c990 msgr2=0x7f525015e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.642+0000 7f5255c1f640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f525015c990 0x7f525015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.642+0000 7f5254c1d640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52500cdc40 0x7f52501f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.642+0000 7f525541e640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f525015c990 0x7f525015e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.642+0000 7f525541e640 1 -- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52500cdc40 msgr2=0x7f52501f1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.642+0000 7f525541e640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52500cdc40 0x7f52501f1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.642+0000 7f525541e640 1 -- 172.21.15.47:0/2780762842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c0094d0 con 0x7f52500b8350 2024-03-23T09:49:54.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.642+0000 7f525541e640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52500b8350 0x7f52500c7e80 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f524c02b8c0 tx=0x7f524c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:54.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.642+0000 7f5245ffb640 1 -- 172.21.15.47:0/2780762842 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c032880 con 0x7f52500b8350 2024-03-23T09:49:54.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.642+0000 7f5245ffb640 1 -- 172.21.15.47:0/2780762842 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c032a00 con 0x7f52500b8350 2024-03-23T09:49:54.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.642+0000 7f5257682640 1 -- 172.21.15.47:0/2780762842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525015f280 con 0x7f52500b8350 2024-03-23T09:49:54.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.642+0000 7f5245ffb640 1 -- 172.21.15.47:0/2780762842 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c032db0 con 0x7f52500b8350 2024-03-23T09:49:54.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.642+0000 7f5257682640 1 -- 172.21.15.47:0/2780762842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52500c7960 con 0x7f52500b8350 2024-03-23T09:49:54.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.643+0000 7f5257682640 1 -- 172.21.15.47:0/2780762842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5250072360 con 0x7f52500b8350 2024-03-23T09:49:54.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.644+0000 7f5245ffb640 1 -- 172.21.15.47:0/2780762842 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f524c04e450 con 0x7f52500b8350 2024-03-23T09:49:54.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.644+0000 7f5245ffb640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f523406ac00 0x7f523406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.644+0000 7f5245ffb640 1 -- 172.21.15.47:0/2780762842 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f524c0ab510 con 0x7f52500b8350 2024-03-23T09:49:54.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.644+0000 7f5254c1d640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f523406ac00 0x7f523406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.645+0000 7f5254c1d640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f523406ac00 0x7f523406d0c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5240006fd0 tx=0x7f5240008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:54.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.648+0000 7f5245ffb640 1 -- 172.21.15.47:0/2780762842 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f524c0766f0 con 0x7f52500b8350 2024-03-23T09:49:54.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.829+0000 7f5257682640 1 -- 172.21.15.47:0/2780762842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52500c7b50 con 0x7f52500b8350 2024-03-23T09:49:54.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.829+0000 7f5245ffb640 1 -- 172.21.15.47:0/2780762842 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f524c0a9e60 con 0x7f52500b8350 2024-03-23T09:49:54.830 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:49:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.831+0000 7f5257682640 1 -- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f523406ac00 msgr2=0x7f523406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.831+0000 7f5257682640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f523406ac00 0x7f523406d0c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5240006fd0 tx=0x7f5240008040 comp rx=0 tx=0).stop 2024-03-23T09:49:54.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.831+0000 7f5257682640 1 -- 172.21.15.47:0/2780762842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52500b8350 msgr2=0x7f52500c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.831+0000 7f5257682640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52500b8350 0x7f52500c7e80 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f524c02b8c0 tx=0x7f524c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.831+0000 7f5257682640 1 -- 172.21.15.47:0/2780762842 shutdown_connections 2024-03-23T09:49:54.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.831+0000 7f5257682640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f523406ac00 0x7f523406d0c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.831+0000 7f5257682640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f525015c990 0x7f525015e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.831+0000 7f5257682640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52500cdc40 0x7f52501f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.831+0000 7f5257682640 1 --2- 172.21.15.47:0/2780762842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52500b8350 0x7f52500c7e80 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.831+0000 7f5257682640 1 -- 172.21.15.47:0/2780762842 >> 172.21.15.47:0/2780762842 conn(0x7f52500659d0 msgr2=0x7f52500cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:54.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.831+0000 7f5257682640 1 -- 172.21.15.47:0/2780762842 shutdown_connections 2024-03-23T09:49:54.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.831+0000 7f5257682640 1 -- 172.21.15.47:0/2780762842 wait complete. 2024-03-23T09:49:54.842 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:49:54.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.946+0000 7f13192f3640 1 Processor -- start 2024-03-23T09:49:54.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.946+0000 7f13192f3640 1 -- start start 2024-03-23T09:49:54.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.947+0000 7f13192f3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140caf80 0x7f13140cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.947+0000 7f13192f3640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f13140cb360 0x7f1314161000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.947+0000 7f13192f3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13140d0810 con 0x7f13140b0810 2024-03-23T09:49:54.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.947+0000 7f13192f3640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1314160b70 con 0x7f13140cb360 2024-03-23T09:49:54.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.947+0000 7f13192f3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13140cef80 con 0x7f13140caf80 2024-03-23T09:49:54.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.947+0000 7f13127fc640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f13140cb360 0x7f1314161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.947+0000 7f13137fe640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f13140b0810 0x7f13140c9ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:53934/0 (socket says 172.21.15.47:53934) 2024-03-23T09:49:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.947+0000 7f1312ffd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140caf80 0x7f13140cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.948+0000 7f13127fc640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f13140cb360 0x7f1314161000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:52898/0 (socket says 172.21.15.47:52898) 2024-03-23T09:49:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.948+0000 7f13137fe640 1 -- 172.21.15.47:0/450401806 learned_addr learned my addr 172.21.15.47:0/450401806 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.948+0000 7f1311ffb640 1 -- 172.21.15.47:0/450401806 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1873242158 0 0) 0x7f1304001490 con 0x7f13140b0810 2024-03-23T09:49:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.948+0000 7f1311ffb640 1 -- 172.21.15.47:0/450401806 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1300003520 con 0x7f13140b0810 2024-03-23T09:49:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.948+0000 7f13127fc640 1 -- 172.21.15.47:0/450401806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140caf80 msgr2=0x7f13140cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.948+0000 7f13127fc640 1 --2- 172.21.15.47:0/450401806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140caf80 0x7f13140cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.948+0000 7f13127fc640 1 -- 172.21.15.47:0/450401806 >> v1:172.21.15.175:6789/0 conn(0x7f13140b0810 legacy=0x7f13140c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.948+0000 7f13127fc640 1 -- 172.21.15.47:0/450401806 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13140cf260 con 0x7f13140cb360 2024-03-23T09:49:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.948+0000 7f1312ffd640 1 --2- 172.21.15.47:0/450401806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140caf80 0x7f13140cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:49:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.949+0000 7f13127fc640 1 --2- 172.21.15.47:0/450401806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140cb360 0x7f1314161000 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0097a0 tx=0x7f12fc02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e74b770cf9bfd025 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.949+0000 7f1311ffb640 1 -- 172.21.15.47:0/450401806 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc02d040 con 0x7f13140cb360 2024-03-23T09:49:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.949+0000 7f1311ffb640 1 -- 172.21.15.47:0/450401806 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc031810 con 0x7f13140cb360 2024-03-23T09:49:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.949+0000 7f1311ffb640 1 -- 172.21.15.47:0/450401806 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc031b80 con 0x7f13140cb360 2024-03-23T09:49:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.949+0000 7f13192f3640 1 -- 172.21.15.47:0/450401806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140cb360 msgr2=0x7f1314161000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.949+0000 7f13192f3640 1 --2- 172.21.15.47:0/450401806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140cb360 0x7f1314161000 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0097a0 tx=0x7f12fc02b8b0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.949+0000 7f13192f3640 1 -- 172.21.15.47:0/450401806 shutdown_connections 2024-03-23T09:49:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.949+0000 7f13192f3640 1 --2- 172.21.15.47:0/450401806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140cb360 0x7f1314161000 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.949+0000 7f13192f3640 1 --2- 172.21.15.47:0/450401806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140caf80 0x7f13140cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.949+0000 7f13192f3640 1 -- 172.21.15.47:0/450401806 >> 172.21.15.47:0/450401806 conn(0x7f1314098a00 msgr2=0x7f13140cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.950+0000 7f13192f3640 1 -- 172.21.15.47:0/450401806 shutdown_connections 2024-03-23T09:49:54.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.950+0000 7f13192f3640 1 -- 172.21.15.47:0/450401806 wait complete. 2024-03-23T09:49:54.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.951+0000 7f13192f3640 1 Processor -- start 2024-03-23T09:49:54.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.951+0000 7f13192f3640 1 -- start start 2024-03-23T09:49:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.951+0000 7f13192f3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140b0810 0x7f13140c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.951+0000 7f13192f3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140caf80 0x7f13140aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13192f3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13140cb360 0x7f13141548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13192f3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13141f22c0 con 0x7f13140cb360 2024-03-23T09:49:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13192f3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13140af950 con 0x7f13140caf80 2024-03-23T09:49:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13192f3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13140c4a00 con 0x7f13140b0810 2024-03-23T09:49:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13137fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13140cb360 0x7f13141548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13137fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13140cb360 0x7f13141548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55172/0 (socket says 172.21.15.47:55172) 2024-03-23T09:49:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13137fe640 1 -- 172.21.15.47:0/3067028222 learned_addr learned my addr 172.21.15.47:0/3067028222 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13127fc640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140caf80 0x7f13140aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f1312ffd640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140b0810 0x7f13140c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13137fe640 1 -- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140b0810 msgr2=0x7f13140c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13137fe640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140b0810 0x7f13140c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13137fe640 1 -- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140caf80 msgr2=0x7f13140aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13137fe640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140caf80 0x7f13140aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13137fe640 1 -- 172.21.15.47:0/3067028222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c009400 con 0x7f13140cb360 2024-03-23T09:49:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13127fc640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140caf80 0x7f13140aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:49:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13137fe640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13140cb360 0x7f13141548b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f130400ca70 tx=0x7f130400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f130b7fe640 1 -- 172.21.15.47:0/3067028222 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130400abd0 con 0x7f13140cb360 2024-03-23T09:49:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f130b7fe640 1 -- 172.21.15.47:0/3067028222 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130400ad50 con 0x7f13140cb360 2024-03-23T09:49:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13192f3640 1 -- 172.21.15.47:0/3067028222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc009400 con 0x7f13140cb360 2024-03-23T09:49:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f130b7fe640 1 -- 172.21.15.47:0/3067028222 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304012640 con 0x7f13140cb360 2024-03-23T09:49:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.952+0000 7f13192f3640 1 -- 172.21.15.47:0/3067028222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13140c4fc0 con 0x7f13140cb360 2024-03-23T09:49:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.953+0000 7f13192f3640 1 -- 172.21.15.47:0/3067028222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13140c51b0 con 0x7f13140cb360 2024-03-23T09:49:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.954+0000 7f130b7fe640 1 -- 172.21.15.47:0/3067028222 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1304007070 con 0x7f13140cb360 2024-03-23T09:49:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.954+0000 7f130b7fe640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f12f406ac00 0x7f12f406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.954+0000 7f130b7fe640 1 -- 172.21.15.47:0/3067028222 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1304089110 con 0x7f13140cb360 2024-03-23T09:49:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.955+0000 7f1312ffd640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f12f406ac00 0x7f12f406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.955+0000 7f1312ffd640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f12f406ac00 0x7f12f406d0c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f130c009cd0 tx=0x7f130c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:54.958+0000 7f130b7fe640 1 -- 172.21.15.47:0/3067028222 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1304054290 con 0x7f13140cb360 2024-03-23T09:49:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.136+0000 7f13192f3640 1 -- 172.21.15.47:0/3067028222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13140d0810 con 0x7f13140cb360 2024-03-23T09:49:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.136+0000 7f130b7fe640 1 -- 172.21.15.47:0/3067028222 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1304017020 con 0x7f13140cb360 2024-03-23T09:49:55.137 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:49:55.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.138+0000 7f13192f3640 1 -- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f12f406ac00 msgr2=0x7f12f406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.138+0000 7f13192f3640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f12f406ac00 0x7f12f406d0c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f130c009cd0 tx=0x7f130c009190 comp rx=0 tx=0).stop 2024-03-23T09:49:55.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.138+0000 7f13192f3640 1 -- 172.21.15.47:0/3067028222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13140cb360 msgr2=0x7f13141548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.138+0000 7f13192f3640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13140cb360 0x7f13141548b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f130400ca70 tx=0x7f130400cf30 comp rx=0 tx=0).stop 2024-03-23T09:49:55.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.138+0000 7f13192f3640 1 -- 172.21.15.47:0/3067028222 shutdown_connections 2024-03-23T09:49:55.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.138+0000 7f13192f3640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f12f406ac00 0x7f12f406d0c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.138+0000 7f13192f3640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13140cb360 0x7f13141548b0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.138+0000 7f13192f3640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140caf80 0x7f13140aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.138+0000 7f13192f3640 1 --2- 172.21.15.47:0/3067028222 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140b0810 0x7f13140c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.138+0000 7f13192f3640 1 -- 172.21.15.47:0/3067028222 >> 172.21.15.47:0/3067028222 conn(0x7f1314098a00 msgr2=0x7f131406e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:55.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.138+0000 7f13192f3640 1 -- 172.21.15.47:0/3067028222 shutdown_connections 2024-03-23T09:49:55.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.138+0000 7f13192f3640 1 -- 172.21.15.47:0/3067028222 wait complete. 2024-03-23T09:49:55.149 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:49:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.252+0000 7f259ed77640 1 Processor -- start 2024-03-23T09:49:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.252+0000 7f259ed77640 1 -- start start 2024-03-23T09:49:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259ed77640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f259815c990 0x7f2598160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259ed77640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f25980b8350 0x7f25980cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259ed77640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f25980cdc40 0x7f25980b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259ed77640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25980ce0b0 con 0x7f25980cdc40 2024-03-23T09:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259ed77640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2598161630 con 0x7f25980b8350 2024-03-23T09:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259ed77640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25980d1030 con 0x7f259815c990 2024-03-23T09:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259d314640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f25980cdc40 0x7f25980b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259d314640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f25980cdc40 0x7f25980b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55188/0 (socket says 172.21.15.47:55188) 2024-03-23T09:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259d314640 1 -- 172.21.15.47:0/2793643384 learned_addr learned my addr 172.21.15.47:0/2793643384 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f2597fff640 1 --2- 172.21.15.47:0/2793643384 >> v2:172.21.15.175:3300/0 conn(0x7f25980b8350 0x7f25980cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259d314640 1 -- 172.21.15.47:0/2793643384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f259815c990 msgr2=0x7f2598160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259d314640 1 --2- 172.21.15.47:0/2793643384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f259815c990 0x7f2598160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259d314640 1 -- 172.21.15.47:0/2793643384 >> v2:172.21.15.175:3300/0 conn(0x7f25980b8350 msgr2=0x7f25980cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259d314640 1 --2- 172.21.15.47:0/2793643384 >> v2:172.21.15.175:3300/0 conn(0x7f25980b8350 0x7f25980cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.253+0000 7f259d314640 1 -- 172.21.15.47:0/2793643384 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25980cfe10 con 0x7f25980cdc40 2024-03-23T09:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.254+0000 7f259d314640 1 --2- 172.21.15.47:0/2793643384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25980cdc40 0x7f25980b4440 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f25980b5540 tx=0x7f258802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d92b76de06d899 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.254+0000 7f25977fe640 1 -- 172.21.15.47:0/2793643384 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25880317e0 con 0x7f25980cdc40 2024-03-23T09:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.254+0000 7f25977fe640 1 -- 172.21.15.47:0/2793643384 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2588031940 con 0x7f25980cdc40 2024-03-23T09:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.254+0000 7f25977fe640 1 -- 172.21.15.47:0/2793643384 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588031cd0 con 0x7f25980cdc40 2024-03-23T09:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.255+0000 7f259ed77640 1 -- 172.21.15.47:0/2793643384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25980cdc40 msgr2=0x7f25980b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.255+0000 7f259ed77640 1 --2- 172.21.15.47:0/2793643384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25980cdc40 0x7f25980b4440 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f25980b5540 tx=0x7f258802b880 comp rx=0 tx=0).stop 2024-03-23T09:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.255+0000 7f259ed77640 1 -- 172.21.15.47:0/2793643384 shutdown_connections 2024-03-23T09:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.255+0000 7f259ed77640 1 --2- 172.21.15.47:0/2793643384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25980cdc40 0x7f25980b4440 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.255+0000 7f259ed77640 1 --2- 172.21.15.47:0/2793643384 >> v2:172.21.15.175:3300/0 conn(0x7f25980b8350 0x7f25980cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.255+0000 7f259ed77640 1 --2- 172.21.15.47:0/2793643384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f259815c990 0x7f2598160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.255+0000 7f259ed77640 1 -- 172.21.15.47:0/2793643384 >> 172.21.15.47:0/2793643384 conn(0x7f25980659d0 msgr2=0x7f2598065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.255+0000 7f259ed77640 1 -- 172.21.15.47:0/2793643384 shutdown_connections 2024-03-23T09:49:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.255+0000 7f259ed77640 1 -- 172.21.15.47:0/2793643384 wait complete. 2024-03-23T09:49:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.256+0000 7f259ed77640 1 Processor -- start 2024-03-23T09:49:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.256+0000 7f259ed77640 1 -- start start 2024-03-23T09:49:55.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259ed77640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25980b8350 0x7f259815d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259ed77640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25980cdc40 0x7f25980ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259ed77640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259815c990 0x7f25980cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259ed77640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259815dcc0 con 0x7f259815c990 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259ed77640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2598158750 con 0x7f25980cdc40 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259ed77640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259815df20 con 0x7f25980b8350 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259d314640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259815c990 0x7f25980cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259d314640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259815c990 0x7f25980cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55196/0 (socket says 172.21.15.47:55196) 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259d314640 1 -- 172.21.15.47:0/1475038707 learned_addr learned my addr 172.21.15.47:0/1475038707 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f2597fff640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25980cdc40 0x7f25980ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259cb13640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25980b8350 0x7f259815d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259d314640 1 -- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25980b8350 msgr2=0x7f259815d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259d314640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25980b8350 0x7f259815d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259d314640 1 -- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25980cdc40 msgr2=0x7f25980ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259d314640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25980cdc40 0x7f25980ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.257+0000 7f259d314640 1 -- 172.21.15.47:0/1475038707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25880094d0 con 0x7f259815c990 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.258+0000 7f259d314640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259815c990 0x7f25980cd430 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f25880090b0 tx=0x7f25880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.258+0000 7f25957fa640 1 -- 172.21.15.47:0/1475038707 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588032940 con 0x7f259815c990 2024-03-23T09:49:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.258+0000 7f25957fa640 1 -- 172.21.15.47:0/1475038707 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2588032ac0 con 0x7f259815c990 2024-03-23T09:49:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.258+0000 7f259ed77640 1 -- 172.21.15.47:0/1475038707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259815e200 con 0x7f259815c990 2024-03-23T09:49:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.258+0000 7f25957fa640 1 -- 172.21.15.47:0/1475038707 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588032e70 con 0x7f259815c990 2024-03-23T09:49:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.258+0000 7f259ed77640 1 -- 172.21.15.47:0/1475038707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259815e7c0 con 0x7f259815c990 2024-03-23T09:49:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.259+0000 7f259ed77640 1 -- 172.21.15.47:0/1475038707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2598072360 con 0x7f259815c990 2024-03-23T09:49:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.259+0000 7f25957fa640 1 -- 172.21.15.47:0/1475038707 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f258804d3f0 con 0x7f259815c990 2024-03-23T09:49:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.259+0000 7f25957fa640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f257806ac00 0x7f257806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.259+0000 7f25957fa640 1 -- 172.21.15.47:0/1475038707 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25880ab1f0 con 0x7f259815c990 2024-03-23T09:49:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.260+0000 7f259cb13640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f257806ac00 0x7f257806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.260+0000 7f259cb13640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f257806ac00 0x7f257806d0c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2590009f20 tx=0x7f25900094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:55.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.264+0000 7f25957fa640 1 -- 172.21.15.47:0/1475038707 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f25880763d0 con 0x7f259815c990 2024-03-23T09:49:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.441+0000 7f259ed77640 1 -- 172.21.15.47:0/1475038707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f25981588c0 con 0x7f259815c990 2024-03-23T09:49:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.442+0000 7f25957fa640 1 -- 172.21.15.47:0/1475038707 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+11 (secure 0 0 0) 0x7f25880799f0 con 0x7f259815c990 2024-03-23T09:49:55.442 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:49:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.443+0000 7f259ed77640 1 -- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f257806ac00 msgr2=0x7f257806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.443+0000 7f259ed77640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f257806ac00 0x7f257806d0c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2590009f20 tx=0x7f25900094f0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.443+0000 7f259ed77640 1 -- 172.21.15.47:0/1475038707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259815c990 msgr2=0x7f25980cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.443+0000 7f259ed77640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259815c990 0x7f25980cd430 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f25880090b0 tx=0x7f25880046c0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.444+0000 7f259ed77640 1 -- 172.21.15.47:0/1475038707 shutdown_connections 2024-03-23T09:49:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.444+0000 7f259ed77640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f257806ac00 0x7f257806d0c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.444+0000 7f259ed77640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259815c990 0x7f25980cd430 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.444+0000 7f259ed77640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25980cdc40 0x7f25980ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.444+0000 7f259ed77640 1 --2- 172.21.15.47:0/1475038707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25980b8350 0x7f259815d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.444+0000 7f259ed77640 1 -- 172.21.15.47:0/1475038707 >> 172.21.15.47:0/1475038707 conn(0x7f25980659d0 msgr2=0x7f25980b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:55.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.444+0000 7f259ed77640 1 -- 172.21.15.47:0/1475038707 shutdown_connections 2024-03-23T09:49:55.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.444+0000 7f259ed77640 1 -- 172.21.15.47:0/1475038707 wait complete. 2024-03-23T09:49:55.454 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:49:55.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.556+0000 7fd6b40f3640 1 Processor -- start 2024-03-23T09:49:55.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.556+0000 7fd6b40f3640 1 -- start start 2024-03-23T09:49:55.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b40f3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6ac15cd00 0x7fd6ac0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b40f3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6ac0d1020 0x7fd6ac0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b40f3640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ac15d0e0 con 0x7fd6ac0cf750 2024-03-23T09:49:55.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b40f3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ac0cfbe0 con 0x7fd6ac15cd00 2024-03-23T09:49:55.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b40f3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0d1540 con 0x7fd6ac0d1020 2024-03-23T09:49:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b1e8f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6ac15cd00 0x7fd6ac0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b2690640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd6ac0cf750 0x7fd6ac0b43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59778/0 (socket says 172.21.15.47:59778) 2024-03-23T09:49:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b2690640 1 -- 172.21.15.47:0/1388220135 learned_addr learned my addr 172.21.15.47:0/1388220135 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b168e640 1 --2- 172.21.15.47:0/1388220135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6ac0d1020 0x7fd6ac0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b1e8f640 1 -- 172.21.15.47:0/1388220135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6ac0d1020 msgr2=0x7fd6ac0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b1e8f640 1 --2- 172.21.15.47:0/1388220135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6ac0d1020 0x7fd6ac0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b1e8f640 1 -- 172.21.15.47:0/1388220135 >> v1:172.21.15.47:6789/0 conn(0x7fd6ac0cf750 legacy=0x7fd6ac0b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b0e8d640 1 -- 172.21.15.47:0/1388220135 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960452537 0 0) 0x7fd6ac15d0e0 con 0x7fd6ac0cf750 2024-03-23T09:49:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.557+0000 7fd6b1e8f640 1 -- 172.21.15.47:0/1388220135 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac0d1820 con 0x7fd6ac15cd00 2024-03-23T09:49:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.558+0000 7fd6b1e8f640 1 --2- 172.21.15.47:0/1388220135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ac15cd00 0x7fd6ac0d0ae0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80097a0 tx=0x7fd6a802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321b733d07e83d54 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.558+0000 7fd6b0e8d640 1 -- 172.21.15.47:0/1388220135 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a802e070 con 0x7fd6ac15cd00 2024-03-23T09:49:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.558+0000 7fd6b0e8d640 1 -- 172.21.15.47:0/1388220135 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a80315a0 con 0x7fd6ac15cd00 2024-03-23T09:49:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.558+0000 7fd6b0e8d640 1 -- 172.21.15.47:0/1388220135 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8031930 con 0x7fd6ac15cd00 2024-03-23T09:49:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.558+0000 7fd6b40f3640 1 -- 172.21.15.47:0/1388220135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ac15cd00 msgr2=0x7fd6ac0d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.558+0000 7fd6b40f3640 1 --2- 172.21.15.47:0/1388220135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ac15cd00 0x7fd6ac0d0ae0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80097a0 tx=0x7fd6a802b840 comp rx=0 tx=0).stop 2024-03-23T09:49:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.558+0000 7fd6b40f3640 1 -- 172.21.15.47:0/1388220135 shutdown_connections 2024-03-23T09:49:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.558+0000 7fd6b40f3640 1 --2- 172.21.15.47:0/1388220135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6ac0d1020 0x7fd6ac0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.558+0000 7fd6b40f3640 1 --2- 172.21.15.47:0/1388220135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ac15cd00 0x7fd6ac0d0ae0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.558+0000 7fd6b40f3640 1 -- 172.21.15.47:0/1388220135 >> 172.21.15.47:0/1388220135 conn(0x7fd6ac06b160 msgr2=0x7fd6ac06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.559+0000 7fd6b40f3640 1 -- 172.21.15.47:0/1388220135 shutdown_connections 2024-03-23T09:49:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.559+0000 7fd6b40f3640 1 -- 172.21.15.47:0/1388220135 wait complete. 2024-03-23T09:49:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.560+0000 7fd6b40f3640 1 Processor -- start 2024-03-23T09:49:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.560+0000 7fd6b40f3640 1 -- start start 2024-03-23T09:49:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.560+0000 7fd6b40f3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ac0cf750 0x7fd6ac1f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.560+0000 7fd6b40f3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6ac0d1020 0x7fd6ac0c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.560+0000 7fd6b40f3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6ac15cd00 0x7fd6ac0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.560+0000 7fd6b40f3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1f2fd0 con 0x7fd6ac0cf750 2024-03-23T09:49:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.560+0000 7fd6b40f3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0c8ed0 con 0x7fd6ac0d1020 2024-03-23T09:49:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.560+0000 7fd6b40f3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0b2a30 con 0x7fd6ac15cd00 2024-03-23T09:49:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.560+0000 7fd6b1e8f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ac0cf750 0x7fd6ac1f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.560+0000 7fd6b1e8f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ac0cf750 0x7fd6ac1f2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55218/0 (socket says 172.21.15.47:55218) 2024-03-23T09:49:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.560+0000 7fd6b1e8f640 1 -- 172.21.15.47:0/542003495 learned_addr learned my addr 172.21.15.47:0/542003495 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6b168e640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6ac0d1020 0x7fd6ac0c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6b2690640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6ac15cd00 0x7fd6ac0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6b1e8f640 1 -- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6ac15cd00 msgr2=0x7fd6ac0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6b1e8f640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6ac15cd00 0x7fd6ac0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6b1e8f640 1 -- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6ac0d1020 msgr2=0x7fd6ac0c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6b1e8f640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6ac0d1020 0x7fd6ac0c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6b1e8f640 1 -- 172.21.15.47:0/542003495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a8009400 con 0x7fd6ac0cf750 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6b168e640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6ac0d1020 0x7fd6ac0c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6b1e8f640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ac0cf750 0x7fd6ac1f2a90 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8034040 tx=0x7fd6a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6a27fc640 1 -- 172.21.15.47:0/542003495 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8032840 con 0x7fd6ac0cf750 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6a27fc640 1 -- 172.21.15.47:0/542003495 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a80329a0 con 0x7fd6ac0cf750 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6b40f3640 1 -- 172.21.15.47:0/542003495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac0b2d10 con 0x7fd6ac0cf750 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6a27fc640 1 -- 172.21.15.47:0/542003495 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8032d50 con 0x7fd6ac0cf750 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.561+0000 7fd6b40f3640 1 -- 172.21.15.47:0/542003495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac0b32d0 con 0x7fd6ac0cf750 2024-03-23T09:49:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.562+0000 7fd6b40f3640 1 -- 172.21.15.47:0/542003495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac072360 con 0x7fd6ac0cf750 2024-03-23T09:49:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.562+0000 7fd6a27fc640 1 -- 172.21.15.47:0/542003495 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd6a804e430 con 0x7fd6ac0cf750 2024-03-23T09:49:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.563+0000 7fd6a27fc640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd69006ac00 0x7fd69006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.563+0000 7fd6a27fc640 1 -- 172.21.15.47:0/542003495 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6a80ab3c0 con 0x7fd6ac0cf750 2024-03-23T09:49:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.563+0000 7fd6b168e640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd69006ac00 0x7fd69006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.564+0000 7fd6b168e640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd69006ac00 0x7fd69006d0c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac160390 tx=0x7fd69c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.567+0000 7fd6a27fc640 1 -- 172.21.15.47:0/542003495 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd6a80765a0 con 0x7fd6ac0cf750 2024-03-23T09:49:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.759+0000 7fd6b40f3640 1 -- 172.21.15.47:0/542003495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd6ac066be0 con 0x7fd6ac0cf750 2024-03-23T09:49:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.759+0000 7fd6a27fc640 1 -- 172.21.15.47:0/542003495 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd6a8079bc0 con 0x7fd6ac0cf750 2024-03-23T09:49:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:49:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.761+0000 7fd6b40f3640 1 -- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd69006ac00 msgr2=0x7fd69006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.761+0000 7fd6b40f3640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd69006ac00 0x7fd69006d0c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac160390 tx=0x7fd69c008040 comp rx=0 tx=0).stop 2024-03-23T09:49:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.761+0000 7fd6b40f3640 1 -- 172.21.15.47:0/542003495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ac0cf750 msgr2=0x7fd6ac1f2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.761+0000 7fd6b40f3640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ac0cf750 0x7fd6ac1f2a90 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8034040 tx=0x7fd6a80046c0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.761+0000 7fd6b40f3640 1 -- 172.21.15.47:0/542003495 shutdown_connections 2024-03-23T09:49:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.761+0000 7fd6b40f3640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd69006ac00 0x7fd69006d0c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.761+0000 7fd6b40f3640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6ac15cd00 0x7fd6ac0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.761+0000 7fd6b40f3640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6ac0d1020 0x7fd6ac0c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.761+0000 7fd6b40f3640 1 --2- 172.21.15.47:0/542003495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ac0cf750 0x7fd6ac1f2a90 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.761+0000 7fd6b40f3640 1 -- 172.21.15.47:0/542003495 >> 172.21.15.47:0/542003495 conn(0x7fd6ac06b160 msgr2=0x7fd6ac15ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.762+0000 7fd6b40f3640 1 -- 172.21.15.47:0/542003495 shutdown_connections 2024-03-23T09:49:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.762+0000 7fd6b40f3640 1 -- 172.21.15.47:0/542003495 wait complete. 2024-03-23T09:49:55.772 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:49:55.772 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:49:55.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:49:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.927+0000 7f3e78038640 1 Processor -- start 2024-03-23T09:49:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.928+0000 7f3e78038640 1 -- start start 2024-03-23T09:49:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.928+0000 7f3e78038640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3e700b8350 0x7f3e700cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.928+0000 7f3e78038640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3e700cdc40 0x7f3e700d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.928+0000 7f3e78038640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e700ce090 con 0x7f3e7015ca90 2024-03-23T09:49:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.928+0000 7f3e78038640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e700d11e0 con 0x7f3e700b8350 2024-03-23T09:49:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.928+0000 7f3e78038640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e70160f60 con 0x7f3e700cdc40 2024-03-23T09:49:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.929+0000 7f3e755d3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3e700b8350 0x7f3e700cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.929+0000 7f3e75dd4640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3e7015ca90 0x7f3e70160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59786/0 (socket says 172.21.15.47:59786) 2024-03-23T09:49:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.929+0000 7f3e75dd4640 1 -- 172.21.15.47:0/916334937 learned_addr learned my addr 172.21.15.47:0/916334937 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.929+0000 7f3e74dd2640 1 -- 172.21.15.47:0/916334937 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538947984 0 0) 0x7f3e68001110 con 0x7f3e7015ca90 2024-03-23T09:49:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.929+0000 7f3e74dd2640 1 -- 172.21.15.47:0/916334937 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e58003520 con 0x7f3e7015ca90 2024-03-23T09:49:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.929+0000 7f3e765d5640 1 --2- 172.21.15.47:0/916334937 >> v2:172.21.15.175:3300/0 conn(0x7f3e700cdc40 0x7f3e700d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.929+0000 7f3e755d3640 1 -- 172.21.15.47:0/916334937 >> v2:172.21.15.175:3300/0 conn(0x7f3e700cdc40 msgr2=0x7f3e700d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.929+0000 7f3e755d3640 1 --2- 172.21.15.47:0/916334937 >> v2:172.21.15.175:3300/0 conn(0x7f3e700cdc40 0x7f3e700d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.929+0000 7f3e755d3640 1 -- 172.21.15.47:0/916334937 >> v1:172.21.15.47:6789/0 conn(0x7f3e7015ca90 legacy=0x7f3e70160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.929+0000 7f3e755d3640 1 -- 172.21.15.47:0/916334937 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e70161240 con 0x7f3e700b8350 2024-03-23T09:49:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.930+0000 7f3e755d3640 1 --2- 172.21.15.47:0/916334937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e700b8350 0x7f3e700cd700 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f3e700b5930 tx=0x7f3e6402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8592333c6cb47da server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.930+0000 7f3e74dd2640 1 -- 172.21.15.47:0/916334937 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6402e070 con 0x7f3e700b8350 2024-03-23T09:49:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.930+0000 7f3e74dd2640 1 -- 172.21.15.47:0/916334937 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e640315a0 con 0x7f3e700b8350 2024-03-23T09:49:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.930+0000 7f3e74dd2640 1 -- 172.21.15.47:0/916334937 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64031930 con 0x7f3e700b8350 2024-03-23T09:49:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.930+0000 7f3e78038640 1 -- 172.21.15.47:0/916334937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e700b8350 msgr2=0x7f3e700cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.930+0000 7f3e78038640 1 --2- 172.21.15.47:0/916334937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e700b8350 0x7f3e700cd700 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f3e700b5930 tx=0x7f3e6402b840 comp rx=0 tx=0).stop 2024-03-23T09:49:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.930+0000 7f3e78038640 1 -- 172.21.15.47:0/916334937 shutdown_connections 2024-03-23T09:49:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.930+0000 7f3e78038640 1 --2- 172.21.15.47:0/916334937 >> v2:172.21.15.175:3300/0 conn(0x7f3e700cdc40 0x7f3e700d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.930+0000 7f3e78038640 1 --2- 172.21.15.47:0/916334937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e700b8350 0x7f3e700cd700 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.930+0000 7f3e78038640 1 -- 172.21.15.47:0/916334937 >> 172.21.15.47:0/916334937 conn(0x7f3e700659d0 msgr2=0x7f3e70065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.930+0000 7f3e78038640 1 -- 172.21.15.47:0/916334937 shutdown_connections 2024-03-23T09:49:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.931+0000 7f3e78038640 1 -- 172.21.15.47:0/916334937 wait complete. 2024-03-23T09:49:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.932+0000 7f3e78038640 1 Processor -- start 2024-03-23T09:49:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.932+0000 7f3e78038640 1 -- start start 2024-03-23T09:49:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.932+0000 7f3e78038640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e700b8350 0x7f3e700c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.932+0000 7f3e78038640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e700cdc40 0x7f3e700b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.932+0000 7f3e78038640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e7015ca90 0x7f3e700b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.932+0000 7f3e78038640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7015b110 con 0x7f3e700cdc40 2024-03-23T09:49:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.932+0000 7f3e78038640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e70158b50 con 0x7f3e700b8350 2024-03-23T09:49:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.932+0000 7f3e78038640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700b34a0 con 0x7f3e7015ca90 2024-03-23T09:49:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.932+0000 7f3e755d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e700cdc40 0x7f3e700b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.932+0000 7f3e755d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e700cdc40 0x7f3e700b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55228/0 (socket says 172.21.15.47:55228) 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.932+0000 7f3e755d3640 1 -- 172.21.15.47:0/3194556491 learned_addr learned my addr 172.21.15.47:0/3194556491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e75dd4640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e700b8350 0x7f3e700c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e765d5640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e7015ca90 0x7f3e700b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e755d3640 1 -- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e7015ca90 msgr2=0x7f3e700b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e755d3640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e7015ca90 0x7f3e700b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e755d3640 1 -- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e700b8350 msgr2=0x7f3e700c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e755d3640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e700b8350 0x7f3e700c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e755d3640 1 -- 172.21.15.47:0/3194556491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e64009400 con 0x7f3e700cdc40 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e765d5640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e7015ca90 0x7f3e700b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e75dd4640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e700b8350 0x7f3e700c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e755d3640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e700cdc40 0x7f3e700b2420 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f3e6402be50 tx=0x7f3e640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e6e7fc640 1 -- 172.21.15.47:0/3194556491 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64032840 con 0x7f3e700cdc40 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e6e7fc640 1 -- 172.21.15.47:0/3194556491 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e64031f00 con 0x7f3e700cdc40 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e6e7fc640 1 -- 172.21.15.47:0/3194556491 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64032be0 con 0x7f3e700cdc40 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e78038640 1 -- 172.21.15.47:0/3194556491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e700b2ee0 con 0x7f3e700cdc40 2024-03-23T09:49:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.933+0000 7f3e78038640 1 -- 172.21.15.47:0/3194556491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e70158770 con 0x7f3e700cdc40 2024-03-23T09:49:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.934+0000 7f3e78038640 1 -- 172.21.15.47:0/3194556491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e70072360 con 0x7f3e700cdc40 2024-03-23T09:49:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.935+0000 7f3e6e7fc640 1 -- 172.21.15.47:0/3194556491 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3e64032d40 con 0x7f3e700cdc40 2024-03-23T09:49:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.935+0000 7f3e6e7fc640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3e5c06ac00 0x7f3e5c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.935+0000 7f3e6e7fc640 1 -- 172.21.15.47:0/3194556491 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e640ab220 con 0x7f3e700cdc40 2024-03-23T09:49:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.935+0000 7f3e75dd4640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3e5c06ac00 0x7f3e5c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.936+0000 7f3e75dd4640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3e5c06ac00 0x7f3e5c06d0c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3e68001270 tx=0x7f3e68009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:55.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:55.939+0000 7f3e6e7fc640 1 -- 172.21.15.47:0/3194556491 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3e64076540 con 0x7f3e700cdc40 2024-03-23T09:49:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.121+0000 7f3e78038640 1 -- 172.21.15.47:0/3194556491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e70158c90 con 0x7f3e700cdc40 2024-03-23T09:49:56.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.122+0000 7f3e6e7fc640 1 -- 172.21.15.47:0/3194556491 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16043 (secure 0 0 0) 0x7f3e70158c90 con 0x7f3e700cdc40 2024-03-23T09:49:56.123 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:49:56.123 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":101,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:49:44.249270+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:49:41.158056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-23T09:49:42.210388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:49:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.124+0000 7f3e78038640 1 -- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3e5c06ac00 msgr2=0x7f3e5c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.124+0000 7f3e78038640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3e5c06ac00 0x7f3e5c06d0c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3e68001270 tx=0x7f3e68009040 comp rx=0 tx=0).stop 2024-03-23T09:49:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.125+0000 7f3e78038640 1 -- 172.21.15.47:0/3194556491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e700cdc40 msgr2=0x7f3e700b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.125+0000 7f3e78038640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e700cdc40 0x7f3e700b2420 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f3e6402be50 tx=0x7f3e640046c0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.125+0000 7f3e78038640 1 -- 172.21.15.47:0/3194556491 shutdown_connections 2024-03-23T09:49:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.125+0000 7f3e78038640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3e5c06ac00 0x7f3e5c06d0c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.125+0000 7f3e78038640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e7015ca90 0x7f3e700b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.125+0000 7f3e78038640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e700cdc40 0x7f3e700b2420 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.125+0000 7f3e78038640 1 --2- 172.21.15.47:0/3194556491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e700b8350 0x7f3e700c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.125+0000 7f3e78038640 1 -- 172.21.15.47:0/3194556491 >> 172.21.15.47:0/3194556491 conn(0x7f3e700659d0 msgr2=0x7f3e700c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.125+0000 7f3e78038640 1 -- 172.21.15.47:0/3194556491 shutdown_connections 2024-03-23T09:49:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.125+0000 7f3e78038640 1 -- 172.21.15.47:0/3194556491 wait complete. 2024-03-23T09:49:56.135 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:49:41.158056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:49:42.210388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:49:56.136 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:49:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.239+0000 7f962d8c4640 1 Processor -- start 2024-03-23T09:49:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.239+0000 7f962d8c4640 1 -- start start 2024-03-23T09:49:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.240+0000 7f962d8c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f962815ff60 0x7f96280cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.240+0000 7f962d8c4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9628160d60 con 0x7f96280b1b20 2024-03-23T09:49:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.240+0000 7f962d8c4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9628160340 con 0x7f96280b81f0 2024-03-23T09:49:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.240+0000 7f962d8c4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96280d3680 con 0x7f962815ff60 2024-03-23T09:49:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.240+0000 7f9626ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f962815ff60 0x7f96280cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.240+0000 7f96277fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f96280b1b20 0x7f96280d2f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59800/0 (socket says 172.21.15.47:59800) 2024-03-23T09:49:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.240+0000 7f96277fe640 1 -- 172.21.15.47:0/3265101189 learned_addr learned my addr 172.21.15.47:0/3265101189 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.240+0000 7f9626ffd640 1 -- 172.21.15.47:0/3265101189 >> v1:172.21.15.175:6789/0 conn(0x7f96280b81f0 legacy=0x7f96280d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.240+0000 7f9626ffd640 1 -- 172.21.15.47:0/3265101189 >> v1:172.21.15.47:6789/0 conn(0x7f96280b1b20 legacy=0x7f96280d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.240+0000 7f9626ffd640 1 -- 172.21.15.47:0/3265101189 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96280b8680 con 0x7f962815ff60 2024-03-23T09:49:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.240+0000 7f9625ffb640 1 -- 172.21.15.47:0/3265101189 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 787185435 0 0) 0x7f9628160d60 con 0x7f96280b1b20 2024-03-23T09:49:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.241+0000 7f9626ffd640 1 --2- 172.21.15.47:0/3265101189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f962815ff60 0x7f96280cf820 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f9618009d90 tx=0x7f961802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fc2c6bf1a3d736 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.241+0000 7f9625ffb640 1 -- 172.21.15.47:0/3265101189 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961802cbd0 con 0x7f962815ff60 2024-03-23T09:49:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.241+0000 7f9625ffb640 1 -- 172.21.15.47:0/3265101189 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961802cd30 con 0x7f962815ff60 2024-03-23T09:49:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.241+0000 7f9625ffb640 1 -- 172.21.15.47:0/3265101189 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961802cea0 con 0x7f962815ff60 2024-03-23T09:49:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.242+0000 7f962d8c4640 1 -- 172.21.15.47:0/3265101189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f962815ff60 msgr2=0x7f96280cf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.242+0000 7f962d8c4640 1 --2- 172.21.15.47:0/3265101189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f962815ff60 0x7f96280cf820 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f9618009d90 tx=0x7f961802be20 comp rx=0 tx=0).stop 2024-03-23T09:49:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.242+0000 7f962d8c4640 1 -- 172.21.15.47:0/3265101189 shutdown_connections 2024-03-23T09:49:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.242+0000 7f962d8c4640 1 --2- 172.21.15.47:0/3265101189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f962815ff60 0x7f96280cf820 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.242+0000 7f962d8c4640 1 -- 172.21.15.47:0/3265101189 >> 172.21.15.47:0/3265101189 conn(0x7f9628065df0 msgr2=0x7f9628066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.242+0000 7f962d8c4640 1 -- 172.21.15.47:0/3265101189 shutdown_connections 2024-03-23T09:49:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.242+0000 7f962d8c4640 1 -- 172.21.15.47:0/3265101189 wait complete. 2024-03-23T09:49:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.243+0000 7f962d8c4640 1 Processor -- start 2024-03-23T09:49:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.243+0000 7f962d8c4640 1 -- start start 2024-03-23T09:49:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f962d8c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96280b1b20 0x7f96280b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f962d8c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f962815ff60 0x7f962815dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f962d8c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96281f5180 0x7f962815e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f962d8c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96280b7980 con 0x7f962815ff60 2024-03-23T09:49:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f962d8c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96280b7770 con 0x7f96280b1b20 2024-03-23T09:49:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f962d8c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96281546c0 con 0x7f96281f5180 2024-03-23T09:49:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f96267fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f962815ff60 0x7f962815dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f96267fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f962815ff60 0x7f962815dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55248/0 (socket says 172.21.15.47:55248) 2024-03-23T09:49:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f96267fc640 1 -- 172.21.15.47:0/1341090283 learned_addr learned my addr 172.21.15.47:0/1341090283 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f96277fe640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96281f5180 0x7f962815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f9626ffd640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96280b1b20 0x7f96280b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f96267fc640 1 -- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96281f5180 msgr2=0x7f962815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f96267fc640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96281f5180 0x7f962815e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f96267fc640 1 -- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96280b1b20 msgr2=0x7f96280b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f96267fc640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96280b1b20 0x7f96280b7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.244+0000 7f96267fc640 1 -- 172.21.15.47:0/1341090283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9618009a70 con 0x7f962815ff60 2024-03-23T09:49:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.245+0000 7f96267fc640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f962815ff60 0x7f962815dd00 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f961000c440 tx=0x7f961000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.245+0000 7f961ffff640 1 -- 172.21.15.47:0/1341090283 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610005480 con 0x7f962815ff60 2024-03-23T09:49:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.245+0000 7f961ffff640 1 -- 172.21.15.47:0/1341090283 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96100035f0 con 0x7f962815ff60 2024-03-23T09:49:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.245+0000 7f962d8c4640 1 -- 172.21.15.47:0/1341090283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9628155ca0 con 0x7f962815ff60 2024-03-23T09:49:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.245+0000 7f961ffff640 1 -- 172.21.15.47:0/1341090283 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96100039c0 con 0x7f962815ff60 2024-03-23T09:49:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.245+0000 7f962d8c4640 1 -- 172.21.15.47:0/1341090283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9628156260 con 0x7f962815ff60 2024-03-23T09:49:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.246+0000 7f962d8c4640 1 -- 172.21.15.47:0/1341090283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96280672f0 con 0x7f962815ff60 2024-03-23T09:49:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.246+0000 7f961ffff640 1 -- 172.21.15.47:0/1341090283 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9610003b20 con 0x7f962815ff60 2024-03-23T09:49:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.247+0000 7f961ffff640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f960806ac00 0x7f960806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.247+0000 7f961ffff640 1 -- 172.21.15.47:0/1341090283 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9610088650 con 0x7f962815ff60 2024-03-23T09:49:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.247+0000 7f9626ffd640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f960806ac00 0x7f960806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.247+0000 7f9626ffd640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f960806ac00 0x7f960806d0c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9618036000 tx=0x7f9618030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.251+0000 7f961ffff640 1 -- 172.21.15.47:0/1341090283 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9610053710 con 0x7f962815ff60 2024-03-23T09:49:56.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.429+0000 7f962d8c4640 1 -- 172.21.15.47:0/1341090283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96280b3020 con 0x7f962815ff60 2024-03-23T09:49:56.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.429+0000 7f961ffff640 1 -- 172.21.15.47:0/1341090283 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9610056d30 con 0x7f962815ff60 2024-03-23T09:49:56.429 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:49:56.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.431+0000 7f962d8c4640 1 -- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f960806ac00 msgr2=0x7f960806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.431+0000 7f962d8c4640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f960806ac00 0x7f960806d0c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9618036000 tx=0x7f9618030000 comp rx=0 tx=0).stop 2024-03-23T09:49:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.431+0000 7f962d8c4640 1 -- 172.21.15.47:0/1341090283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f962815ff60 msgr2=0x7f962815dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.431+0000 7f962d8c4640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f962815ff60 0x7f962815dd00 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f961000c440 tx=0x7f961000c900 comp rx=0 tx=0).stop 2024-03-23T09:49:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.431+0000 7f962d8c4640 1 -- 172.21.15.47:0/1341090283 shutdown_connections 2024-03-23T09:49:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.431+0000 7f962d8c4640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f960806ac00 0x7f960806d0c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.431+0000 7f962d8c4640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96281f5180 0x7f962815e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.431+0000 7f962d8c4640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f962815ff60 0x7f962815dd00 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.431+0000 7f962d8c4640 1 --2- 172.21.15.47:0/1341090283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96280b1b20 0x7f96280b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.431+0000 7f962d8c4640 1 -- 172.21.15.47:0/1341090283 >> 172.21.15.47:0/1341090283 conn(0x7f9628065df0 msgr2=0x7f96280661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.431+0000 7f962d8c4640 1 -- 172.21.15.47:0/1341090283 shutdown_connections 2024-03-23T09:49:56.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.431+0000 7f962d8c4640 1 -- 172.21.15.47:0/1341090283 wait complete. 2024-03-23T09:49:56.442 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:49:56.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.545+0000 7f9ded55c640 1 Processor -- start 2024-03-23T09:49:56.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.545+0000 7f9ded55c640 1 -- start start 2024-03-23T09:49:56.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.546+0000 7f9ded55c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9de80d01b0 0x7f9de80cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.546+0000 7f9ded55c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de80cc100 0x7f9de80c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.546+0000 7f9ded55c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de806f260 con 0x7f9de815cc50 2024-03-23T09:49:56.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.546+0000 7f9ded55c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de80cc590 con 0x7f9de80d01b0 2024-03-23T09:49:56.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.546+0000 7f9ded55c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80d0590 con 0x7f9de80cc100 2024-03-23T09:49:56.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.546+0000 7f9de77fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de80cc100 0x7f9de80c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.546+0000 7f9de67fc640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9de80d01b0 0x7f9de80cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.546+0000 7f9de6ffd640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f9de815cc50 0x7f9de806eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:53952/0 (socket says 172.21.15.47:53952) 2024-03-23T09:49:56.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.546+0000 7f9de6ffd640 1 -- 172.21.15.47:0/1838637742 learned_addr learned my addr 172.21.15.47:0/1838637742 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.546+0000 7f9de77fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de80cc100 0x7f9de80c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:36716/0 (socket says 172.21.15.47:36716) 2024-03-23T09:49:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.547+0000 7f9de67fc640 1 -- 172.21.15.47:0/1838637742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de80cc100 msgr2=0x7f9de80c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.547+0000 7f9de67fc640 1 --2- 172.21.15.47:0/1838637742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de80cc100 0x7f9de80c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.547+0000 7f9de67fc640 1 -- 172.21.15.47:0/1838637742 >> v1:172.21.15.175:6789/0 conn(0x7f9de815cc50 legacy=0x7f9de806eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.547+0000 7f9de5ffb640 1 -- 172.21.15.47:0/1838637742 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164539211 0 0) 0x7f9de806f260 con 0x7f9de815cc50 2024-03-23T09:49:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.547+0000 7f9de67fc640 1 -- 172.21.15.47:0/1838637742 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de806e360 con 0x7f9de80d01b0 2024-03-23T09:49:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.547+0000 7f9de77fe640 1 --2- 172.21.15.47:0/1838637742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de80cc100 0x7f9de80c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:49:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.548+0000 7f9de67fc640 1 --2- 172.21.15.47:0/1838637742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de80d01b0 0x7f9de80cbbc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00097d0 tx=0x7f9dd002b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fab8d0bd0c541ca server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.548+0000 7f9de5ffb640 1 -- 172.21.15.47:0/1838637742 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0031890 con 0x7f9de80d01b0 2024-03-23T09:49:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.548+0000 7f9de5ffb640 1 -- 172.21.15.47:0/1838637742 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd00319f0 con 0x7f9de80d01b0 2024-03-23T09:49:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.548+0000 7f9de5ffb640 1 -- 172.21.15.47:0/1838637742 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0031d80 con 0x7f9de80d01b0 2024-03-23T09:49:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.548+0000 7f9ded55c640 1 -- 172.21.15.47:0/1838637742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de80d01b0 msgr2=0x7f9de80cbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.548+0000 7f9ded55c640 1 --2- 172.21.15.47:0/1838637742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de80d01b0 0x7f9de80cbbc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00097d0 tx=0x7f9dd002b930 comp rx=0 tx=0).stop 2024-03-23T09:49:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.548+0000 7f9ded55c640 1 -- 172.21.15.47:0/1838637742 shutdown_connections 2024-03-23T09:49:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.548+0000 7f9ded55c640 1 --2- 172.21.15.47:0/1838637742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de80cc100 0x7f9de80c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.548+0000 7f9ded55c640 1 --2- 172.21.15.47:0/1838637742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de80d01b0 0x7f9de80cbbc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.548+0000 7f9ded55c640 1 -- 172.21.15.47:0/1838637742 >> 172.21.15.47:0/1838637742 conn(0x7f9de8098a00 msgr2=0x7f9de80cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:56.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.548+0000 7f9ded55c640 1 -- 172.21.15.47:0/1838637742 shutdown_connections 2024-03-23T09:49:56.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.548+0000 7f9ded55c640 1 -- 172.21.15.47:0/1838637742 wait complete. 2024-03-23T09:49:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.550+0000 7f9ded55c640 1 Processor -- start 2024-03-23T09:49:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.550+0000 7f9ded55c640 1 -- start start 2024-03-23T09:49:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.550+0000 7f9ded55c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de80cc100 0x7f9de81552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.550+0000 7f9ded55c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9de80d01b0 0x7f9de80c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.550+0000 7f9ded55c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de815cc50 0x7f9de80c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.550+0000 7f9ded55c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de815a9f0 con 0x7f9de80d01b0 2024-03-23T09:49:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.550+0000 7f9ded55c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de81557f0 con 0x7f9de80cc100 2024-03-23T09:49:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.550+0000 7f9ded55c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de8159d70 con 0x7f9de815cc50 2024-03-23T09:49:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.550+0000 7f9de67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9de80d01b0 0x7f9de80c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.550+0000 7f9de67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9de80d01b0 0x7f9de80c8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55260/0 (socket says 172.21.15.47:55260) 2024-03-23T09:49:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.550+0000 7f9de67fc640 1 -- 172.21.15.47:0/2383231281 learned_addr learned my addr 172.21.15.47:0/2383231281 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.551+0000 7f9de6ffd640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de80cc100 0x7f9de81552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.551+0000 7f9de77fe640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de815cc50 0x7f9de80c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.551+0000 7f9de67fc640 1 -- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de815cc50 msgr2=0x7f9de80c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.551+0000 7f9de67fc640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de815cc50 0x7f9de80c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.551+0000 7f9de67fc640 1 -- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de80cc100 msgr2=0x7f9de81552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.551+0000 7f9de67fc640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de80cc100 0x7f9de81552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.551+0000 7f9de67fc640 1 -- 172.21.15.47:0/2383231281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd800bc80 con 0x7f9de80d01b0 2024-03-23T09:49:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.551+0000 7f9de67fc640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9de80d01b0 0x7f9de80c8880 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009720 tx=0x7f9dd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.551+0000 7f9ddf7fe640 1 -- 172.21.15.47:0/2383231281 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd003cc00 con 0x7f9de80d01b0 2024-03-23T09:49:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.551+0000 7f9ddf7fe640 1 -- 172.21.15.47:0/2383231281 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd003cd60 con 0x7f9de80d01b0 2024-03-23T09:49:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.551+0000 7f9ddf7fe640 1 -- 172.21.15.47:0/2383231281 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0004250 con 0x7f9de80d01b0 2024-03-23T09:49:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.551+0000 7f9ded55c640 1 -- 172.21.15.47:0/2383231281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd0009400 con 0x7f9de80d01b0 2024-03-23T09:49:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.551+0000 7f9ded55c640 1 -- 172.21.15.47:0/2383231281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de815a330 con 0x7f9de80d01b0 2024-03-23T09:49:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.552+0000 7f9ded55c640 1 -- 172.21.15.47:0/2383231281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de815ab30 con 0x7f9de80d01b0 2024-03-23T09:49:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.552+0000 7f9ddf7fe640 1 -- 172.21.15.47:0/2383231281 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9dd00043b0 con 0x7f9de80d01b0 2024-03-23T09:49:56.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.553+0000 7f9ddf7fe640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9dc806ac00 0x7f9dc806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.553+0000 7f9ddf7fe640 1 -- 172.21.15.47:0/2383231281 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9dd00abd20 con 0x7f9de80d01b0 2024-03-23T09:49:56.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.553+0000 7f9de6ffd640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9dc806ac00 0x7f9dc806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.554+0000 7f9de6ffd640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9dc806ac00 0x7f9dc806d0c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000f60 tx=0x7f9de0007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:56.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.557+0000 7f9ddf7fe640 1 -- 172.21.15.47:0/2383231281 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9de815ab30 con 0x7f9de80d01b0 2024-03-23T09:49:56.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.738+0000 7f9ded55c640 1 -- 172.21.15.47:0/2383231281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9de815ab30 con 0x7f9de80d01b0 2024-03-23T09:49:56.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.739+0000 7f9ddf7fe640 1 -- 172.21.15.47:0/2383231281 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9de815ab30 con 0x7f9de80d01b0 2024-03-23T09:49:56.739 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:49:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.740+0000 7f9ded55c640 1 -- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9dc806ac00 msgr2=0x7f9dc806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.740+0000 7f9ded55c640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9dc806ac00 0x7f9dc806d0c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000f60 tx=0x7f9de0007d50 comp rx=0 tx=0).stop 2024-03-23T09:49:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.740+0000 7f9ded55c640 1 -- 172.21.15.47:0/2383231281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9de80d01b0 msgr2=0x7f9de80c8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.740+0000 7f9ded55c640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9de80d01b0 0x7f9de80c8880 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009720 tx=0x7f9dd00046c0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.741+0000 7f9ded55c640 1 -- 172.21.15.47:0/2383231281 shutdown_connections 2024-03-23T09:49:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.741+0000 7f9ded55c640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9dc806ac00 0x7f9dc806d0c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.741+0000 7f9ded55c640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de815cc50 0x7f9de80c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.741+0000 7f9ded55c640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9de80d01b0 0x7f9de80c8880 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.741+0000 7f9ded55c640 1 --2- 172.21.15.47:0/2383231281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de80cc100 0x7f9de81552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.741+0000 7f9ded55c640 1 -- 172.21.15.47:0/2383231281 >> 172.21.15.47:0/2383231281 conn(0x7f9de8098a00 msgr2=0x7f9de80cac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.741+0000 7f9ded55c640 1 -- 172.21.15.47:0/2383231281 shutdown_connections 2024-03-23T09:49:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.741+0000 7f9ded55c640 1 -- 172.21.15.47:0/2383231281 wait complete. 2024-03-23T09:49:56.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:49:56.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.854+0000 7fe6c612e640 1 Processor -- start 2024-03-23T09:49:56.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.854+0000 7fe6c612e640 1 -- start start 2024-03-23T09:49:56.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.855+0000 7fe6c612e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6c015c990 0x7fe6c0160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.855+0000 7fe6c612e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c0161630 con 0x7fe6c00cde10 2024-03-23T09:49:56.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.855+0000 7fe6c612e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c00d1030 con 0x7fe6c00b8350 2024-03-23T09:49:56.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.855+0000 7fe6c612e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00cfe10 con 0x7fe6c015c990 2024-03-23T09:49:56.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.855+0000 7fe6bffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe6c00cde10 0x7fe6c00d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59810/0 (socket says 172.21.15.47:59810) 2024-03-23T09:49:56.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.855+0000 7fe6bffff640 1 -- 172.21.15.47:0/1276112947 learned_addr learned my addr 172.21.15.47:0/1276112947 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924854448 0 0) 0x7fe6c0161630 con 0x7fe6c00cde10 2024-03-23T09:49:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a8003520 con 0x7fe6c00cde10 2024-03-23T09:49:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6bf7fe640 1 --2- 172.21.15.47:0/1276112947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6c015c990 0x7fe6c0160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2181809577 0 0) 0x7fe6a8003520 con 0x7fe6c00cde10 2024-03-23T09:49:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6c015c990 msgr2=0x7fe6c0160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 --2- 172.21.15.47:0/1276112947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6c015c990 0x7fe6c0160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 >> v1:172.21.15.175:6789/0 conn(0x7fe6c00b8350 legacy=0x7fe6c00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c00b5b60 con 0x7fe6c00cde10 2024-03-23T09:49:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fe6b80019f0 con 0x7fe6c00cde10 2024-03-23T09:49:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 >> v1:172.21.15.47:6789/0 conn(0x7fe6c00cde10 legacy=0x7fe6c00d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 --2- 172.21.15.47:0/1276112947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6c015c990 0x7fe6a80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 --2- 172.21.15.47:0/1276112947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6a8003fd0 0x7fe6a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 --2- 172.21.15.47:0/1276112947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a8004d00 0x7fe6a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00cfe10 con 0x7fe6a8003fd0 2024-03-23T09:49:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a800bbc0 con 0x7fe6a8004d00 2024-03-23T09:49:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a800bd80 con 0x7fe6c015c990 2024-03-23T09:49:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6beffd640 1 --2- 172.21.15.47:0/1276112947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6a8003fd0 0x7fe6a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6bf7fe640 1 --2- 172.21.15.47:0/1276112947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6c015c990 0x7fe6a80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6bffff640 1 --2- 172.21.15.47:0/1276112947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a8004d00 0x7fe6a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6beffd640 1 -- 172.21.15.47:0/1276112947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6c015c990 msgr2=0x7fe6a80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6beffd640 1 --2- 172.21.15.47:0/1276112947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6c015c990 0x7fe6a80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6beffd640 1 -- 172.21.15.47:0/1276112947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a8004d00 msgr2=0x7fe6a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6beffd640 1 --2- 172.21.15.47:0/1276112947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a8004d00 0x7fe6a8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.856+0000 7fe6beffd640 1 -- 172.21.15.47:0/1276112947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a8003520 con 0x7fe6a8003fd0 2024-03-23T09:49:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.857+0000 7fe6beffd640 1 --2- 172.21.15.47:0/1276112947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6a8003fd0 0x7fe6a80047c0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4001750 tx=0x7fe6a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c538f3e32762b57 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.857+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40052d0 con 0x7fe6a8003fd0 2024-03-23T09:49:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.857+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b402e050 con 0x7fe6a8003fd0 2024-03-23T09:49:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.857+0000 7fe6be7fc640 1 -- 172.21.15.47:0/1276112947 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4003830 con 0x7fe6a8003fd0 2024-03-23T09:49:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.858+0000 7fe6c612e640 1 -- 172.21.15.47:0/1276112947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6a8003fd0 msgr2=0x7fe6a80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.858+0000 7fe6c612e640 1 --2- 172.21.15.47:0/1276112947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6a8003fd0 0x7fe6a80047c0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4001750 tx=0x7fe6a80057f0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.858+0000 7fe6bffff640 1 -- 172.21.15.47:0/1276112947 reap_dead start 2024-03-23T09:49:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.858+0000 7fe6c612e640 1 -- 172.21.15.47:0/1276112947 shutdown_connections 2024-03-23T09:49:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.858+0000 7fe6c612e640 1 -- 172.21.15.47:0/1276112947 >> 172.21.15.47:0/1276112947 conn(0x7fe6c00659d0 msgr2=0x7fe6c0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.858+0000 7fe6c612e640 1 -- 172.21.15.47:0/1276112947 shutdown_connections 2024-03-23T09:49:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.858+0000 7fe6c612e640 1 -- 172.21.15.47:0/1276112947 wait complete. 2024-03-23T09:49:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.859+0000 7fe6c612e640 1 Processor -- start 2024-03-23T09:49:56.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.859+0000 7fe6c612e640 1 -- start start 2024-03-23T09:49:56.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6c612e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a8003fd0 0x7fe6c0155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6c612e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6c00c75e0 0x7fe6c0159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6c612e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6c00c8b80 0x7fe6c00b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6c612e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00c86f0 con 0x7fe6c00c75e0 2024-03-23T09:49:56.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6c612e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00c7ae0 con 0x7fe6a8003fd0 2024-03-23T09:49:56.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6c612e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00c7dc0 con 0x7fe6c00c8b80 2024-03-23T09:49:56.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6beffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6c00c75e0 0x7fe6c0159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6beffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6c00c75e0 0x7fe6c0159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55282/0 (socket says 172.21.15.47:55282) 2024-03-23T09:49:56.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6beffd640 1 -- 172.21.15.47:0/447476317 learned_addr learned my addr 172.21.15.47:0/447476317 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6bffff640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6c00c8b80 0x7fe6c00b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6bf7fe640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a8003fd0 0x7fe6c0155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6beffd640 1 -- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6c00c8b80 msgr2=0x7fe6c00b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6beffd640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6c00c8b80 0x7fe6c00b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6beffd640 1 -- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a8003fd0 msgr2=0x7fe6c0155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6beffd640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a8003fd0 0x7fe6c0155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.860+0000 7fe6beffd640 1 -- 172.21.15.47:0/447476317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c00c80a0 con 0x7fe6c00c75e0 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.861+0000 7fe6beffd640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6c00c75e0 0x7fe6c0159740 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4008fd0 tx=0x7fe6b4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.861+0000 7fe6a7fff640 1 -- 172.21.15.47:0/447476317 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40076c0 con 0x7fe6c00c75e0 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.861+0000 7fe6a7fff640 1 -- 172.21.15.47:0/447476317 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b4007a30 con 0x7fe6c00c75e0 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.861+0000 7fe6c612e640 1 -- 172.21.15.47:0/447476317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c00c8380 con 0x7fe6c00c75e0 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.861+0000 7fe6a7fff640 1 -- 172.21.15.47:0/447476317 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4006c00 con 0x7fe6c00c75e0 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.861+0000 7fe6c612e640 1 -- 172.21.15.47:0/447476317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c0159e60 con 0x7fe6c00c75e0 2024-03-23T09:49:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.862+0000 7fe6c612e640 1 -- 172.21.15.47:0/447476317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c0072360 con 0x7fe6c00c75e0 2024-03-23T09:49:56.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.862+0000 7fe6a7fff640 1 -- 172.21.15.47:0/447476317 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe6b40035f0 con 0x7fe6c00c75e0 2024-03-23T09:49:56.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.862+0000 7fe6a7fff640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe6ac06ac00 0x7fe6ac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:56.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.863+0000 7fe6a7fff640 1 -- 172.21.15.47:0/447476317 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6b40ab480 con 0x7fe6c00c75e0 2024-03-23T09:49:56.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.863+0000 7fe6bf7fe640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe6ac06ac00 0x7fe6ac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:56.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.863+0000 7fe6bf7fe640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe6ac06ac00 0x7fe6ac06d0c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0006fd0 tx=0x7fe6b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:56.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:56.867+0000 7fe6a7fff640 1 -- 172.21.15.47:0/447476317 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe6b407a0f0 con 0x7fe6c00c75e0 2024-03-23T09:49:57.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.047+0000 7fe6c612e640 1 -- 172.21.15.47:0/447476317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe6c0066be0 con 0x7fe6c00c75e0 2024-03-23T09:49:57.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.047+0000 7fe6a7fff640 1 -- 172.21.15.47:0/447476317 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe6b4079c80 con 0x7fe6c00c75e0 2024-03-23T09:49:57.048 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:49:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.049+0000 7fe6c612e640 1 -- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe6ac06ac00 msgr2=0x7fe6ac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.049+0000 7fe6c612e640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe6ac06ac00 0x7fe6ac06d0c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0006fd0 tx=0x7fe6b0008040 comp rx=0 tx=0).stop 2024-03-23T09:49:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.049+0000 7fe6c612e640 1 -- 172.21.15.47:0/447476317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6c00c75e0 msgr2=0x7fe6c0159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.049+0000 7fe6c612e640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6c00c75e0 0x7fe6c0159740 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4008fd0 tx=0x7fe6b4001120 comp rx=0 tx=0).stop 2024-03-23T09:49:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.049+0000 7fe6c612e640 1 -- 172.21.15.47:0/447476317 shutdown_connections 2024-03-23T09:49:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.049+0000 7fe6c612e640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe6ac06ac00 0x7fe6ac06d0c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.049+0000 7fe6c612e640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6c00c8b80 0x7fe6c00b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.049+0000 7fe6c612e640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6c00c75e0 0x7fe6c0159740 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.049+0000 7fe6c612e640 1 --2- 172.21.15.47:0/447476317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a8003fd0 0x7fe6c0155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.049+0000 7fe6c612e640 1 -- 172.21.15.47:0/447476317 >> 172.21.15.47:0/447476317 conn(0x7fe6c00659d0 msgr2=0x7fe6c00d13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:57.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.049+0000 7fe6c612e640 1 -- 172.21.15.47:0/447476317 shutdown_connections 2024-03-23T09:49:57.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.050+0000 7fe6c612e640 1 -- 172.21.15.47:0/447476317 wait complete. 2024-03-23T09:49:57.060 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:49:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.162+0000 7f773d155640 1 Processor -- start 2024-03-23T09:49:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.162+0000 7f773d155640 1 -- start start 2024-03-23T09:49:57.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.162+0000 7f773d155640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f773815cc40 0x7f7738160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:57.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.162+0000 7f773d155640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7738160c90 0x7f77380cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:57.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.162+0000 7f773d155640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77380d09d0 con 0x7f77380cdbf0 2024-03-23T09:49:57.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.162+0000 7f773d155640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77380ce060 con 0x7f7738160c90 2024-03-23T09:49:57.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.162+0000 7f773d155640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77380b5a80 con 0x7f773815cc40 2024-03-23T09:49:57.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.163+0000 7f7736d9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f773815cc40 0x7f7738160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:57.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.163+0000 7f773659c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7738160c90 0x7f77380cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:57.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.163+0000 7f773759e640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f77380cdbf0 0x7f77380b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:53972/0 (socket says 172.21.15.47:53972) 2024-03-23T09:49:57.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.163+0000 7f7736d9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f773815cc40 0x7f7738160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:36752/0 (socket says 172.21.15.47:36752) 2024-03-23T09:49:57.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.163+0000 7f773759e640 1 -- 172.21.15.47:0/1078071879 learned_addr learned my addr 172.21.15.47:0/1078071879 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:49:57.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.163+0000 7f7736d9d640 1 -- 172.21.15.47:0/1078071879 >> v2:172.21.15.175:3300/0 conn(0x7f7738160c90 msgr2=0x7f77380cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:49:57.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.163+0000 7f7736d9d640 1 --2- 172.21.15.47:0/1078071879 >> v2:172.21.15.175:3300/0 conn(0x7f7738160c90 0x7f77380cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.163+0000 7f7736d9d640 1 -- 172.21.15.47:0/1078071879 >> v1:172.21.15.175:6789/0 conn(0x7f77380cdbf0 legacy=0x7f77380b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:57.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.163+0000 7f7736d9d640 1 -- 172.21.15.47:0/1078071879 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773815d020 con 0x7f773815cc40 2024-03-23T09:49:57.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.163+0000 7f773659c640 1 --2- 172.21.15.47:0/1078071879 >> v2:172.21.15.175:3300/0 conn(0x7f7738160c90 0x7f77380cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:49:57.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.164+0000 7f7736d9d640 1 --2- 172.21.15.47:0/1078071879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f773815cc40 0x7f7738160750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77280097a0 tx=0x7f772802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=888c6fce55d11327 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:57.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.164+0000 7f7735d9b640 1 -- 172.21.15.47:0/1078071879 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772802d040 con 0x7f773815cc40 2024-03-23T09:49:57.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.164+0000 7f7735d9b640 1 -- 172.21.15.47:0/1078071879 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7728031810 con 0x7f773815cc40 2024-03-23T09:49:57.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.164+0000 7f7735d9b640 1 -- 172.21.15.47:0/1078071879 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728031ba0 con 0x7f773815cc40 2024-03-23T09:49:57.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.164+0000 7f773d155640 1 -- 172.21.15.47:0/1078071879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f773815cc40 msgr2=0x7f7738160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:57.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.164+0000 7f773d155640 1 --2- 172.21.15.47:0/1078071879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f773815cc40 0x7f7738160750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77280097a0 tx=0x7f772802b8b0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.165+0000 7f773d155640 1 -- 172.21.15.47:0/1078071879 shutdown_connections 2024-03-23T09:49:57.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.165+0000 7f773d155640 1 --2- 172.21.15.47:0/1078071879 >> v2:172.21.15.175:3300/0 conn(0x7f7738160c90 0x7f77380cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.165+0000 7f773d155640 1 --2- 172.21.15.47:0/1078071879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f773815cc40 0x7f7738160750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.165+0000 7f773d155640 1 -- 172.21.15.47:0/1078071879 >> 172.21.15.47:0/1078071879 conn(0x7f773806b160 msgr2=0x7f773806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.165+0000 7f773d155640 1 -- 172.21.15.47:0/1078071879 shutdown_connections 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.165+0000 7f773d155640 1 -- 172.21.15.47:0/1078071879 wait complete. 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.166+0000 7f773d155640 1 Processor -- start 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.166+0000 7f773d155640 1 -- start start 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773d155640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77380cdbf0 0x7f77381f6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773d155640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f773815cc40 0x7f773815f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773d155640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7738160c90 0x7f77381507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773d155640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77381f6ca0 con 0x7f773815cc40 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773d155640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7738159270 con 0x7f7738160c90 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773d155640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7738159550 con 0x7f77380cdbf0 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f773815cc40 0x7f773815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f773815cc40 0x7f773815f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55292/0 (socket says 172.21.15.47:55292) 2024-03-23T09:49:57.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773659c640 1 -- 172.21.15.47:0/3962663534 learned_addr learned my addr 172.21.15.47:0/3962663534 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:49:57.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f7736d9d640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77380cdbf0 0x7f77381f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:57.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773759e640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7738160c90 0x7f77381507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:57.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773659c640 1 -- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77380cdbf0 msgr2=0x7f77381f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:57.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773659c640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77380cdbf0 0x7f77381f6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773659c640 1 -- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7738160c90 msgr2=0x7f77381507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:57.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773659c640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7738160c90 0x7f77381507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.167+0000 7f773659c640 1 -- 172.21.15.47:0/3962663534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c009400 con 0x7f773815cc40 2024-03-23T09:49:57.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.168+0000 7f773659c640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f773815cc40 0x7f773815f650 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f772c00e900 tx=0x7f772c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:57.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.168+0000 7f771f7fe640 1 -- 172.21.15.47:0/3962663534 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c009660 con 0x7f773815cc40 2024-03-23T09:49:57.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.168+0000 7f771f7fe640 1 -- 172.21.15.47:0/3962663534 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c00ca00 con 0x7f773815cc40 2024-03-23T09:49:57.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.168+0000 7f773d155640 1 -- 172.21.15.47:0/3962663534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7728009400 con 0x7f773815cc40 2024-03-23T09:49:57.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.168+0000 7f771f7fe640 1 -- 172.21.15.47:0/3962663534 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c00cdb0 con 0x7f773815cc40 2024-03-23T09:49:57.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.168+0000 7f773d155640 1 -- 172.21.15.47:0/3962663534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77381598e0 con 0x7f773815cc40 2024-03-23T09:49:57.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.169+0000 7f773d155640 1 -- 172.21.15.47:0/3962663534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7738072360 con 0x7f773815cc40 2024-03-23T09:49:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.169+0000 7f771f7fe640 1 -- 172.21.15.47:0/3962663534 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f772c013470 con 0x7f773815cc40 2024-03-23T09:49:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.169+0000 7f771f7fe640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f772406ac00 0x7f772406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:49:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.170+0000 7f771f7fe640 1 -- 172.21.15.47:0/3962663534 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f772c089150 con 0x7f773815cc40 2024-03-23T09:49:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.170+0000 7f7736d9d640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f772406ac00 0x7f772406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:49:57.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.170+0000 7f7736d9d640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f772406ac00 0x7f772406d0c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7728036000 tx=0x7f772803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:49:57.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.174+0000 7f771f7fe640 1 -- 172.21.15.47:0/3962663534 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f772c0543f0 con 0x7f773815cc40 2024-03-23T09:49:57.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.369+0000 7f773d155640 1 -- 172.21.15.47:0/3962663534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7738159eb0 con 0x7f773815cc40 2024-03-23T09:49:57.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.369+0000 7f771f7fe640 1 -- 172.21.15.47:0/3962663534 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) v1 ==== 80+0+0 (secure 0 0 0) 0x7f772c057a10 con 0x7f773815cc40 2024-03-23T09:49:57.369 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:49:57.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.371+0000 7f773d155640 1 -- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f772406ac00 msgr2=0x7f772406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.371+0000 7f773d155640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f772406ac00 0x7f772406d0c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7728036000 tx=0x7f772803c040 comp rx=0 tx=0).stop 2024-03-23T09:49:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.371+0000 7f773d155640 1 -- 172.21.15.47:0/3962663534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f773815cc40 msgr2=0x7f773815f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:49:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.371+0000 7f773d155640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f773815cc40 0x7f773815f650 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f772c00e900 tx=0x7f772c00edc0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.371+0000 7f773d155640 1 -- 172.21.15.47:0/3962663534 shutdown_connections 2024-03-23T09:49:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.371+0000 7f773d155640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f772406ac00 0x7f772406d0c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.371+0000 7f773d155640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7738160c90 0x7f77381507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.371+0000 7f773d155640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f773815cc40 0x7f773815f650 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.371+0000 7f773d155640 1 --2- 172.21.15.47:0/3962663534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77380cdbf0 0x7f77381f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:49:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.371+0000 7f773d155640 1 -- 172.21.15.47:0/3962663534 >> 172.21.15.47:0/3962663534 conn(0x7f773806b160 msgr2=0x7f77380d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:49:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.371+0000 7f773d155640 1 -- 172.21.15.47:0/3962663534 shutdown_connections 2024-03-23T09:49:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:49:57.371+0000 7f773d155640 1 -- 172.21.15.47:0/3962663534 wait complete. 2024-03-23T09:49:57.382 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:49:57.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:49:57.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:49:57.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[73846]: fuse finished with error 0 and tester_r 0 2024-03-23T09:49:57.409 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:50:03.511 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:50:03.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:03.511 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:50:03.530 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:50:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.683+0000 7f7aabb00640 1 Processor -- start 2024-03-23T09:50:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.683+0000 7f7aabb00640 1 -- start start 2024-03-23T09:50:03.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.684+0000 7f7aabb00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7aa4142a90 0x7f7aa40cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:03.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.684+0000 7f7aabb00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7aa4143010 0x7f7aa40b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:03.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.684+0000 7f7aabb00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7aa40b40e0 0x7f7aa415d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:03.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.684+0000 7f7aabb00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa40b27c0 con 0x7f7aa40b40e0 2024-03-23T09:50:03.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.684+0000 7f7aabb00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa4142e70 con 0x7f7aa4143010 2024-03-23T09:50:03.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.684+0000 7f7aabb00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40ccd00 con 0x7f7aa4142a90 2024-03-23T09:50:03.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.684+0000 7f7aaa09d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7aa40b40e0 0x7f7aa415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:03.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.684+0000 7f7aaa09d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7aa40b40e0 0x7f7aa415d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55300/0 (socket says 172.21.15.47:55300) 2024-03-23T09:50:03.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.684+0000 7f7aaa09d640 1 -- 172.21.15.47:0/127552007 learned_addr learned my addr 172.21.15.47:0/127552007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:03.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.684+0000 7f7aa989c640 1 --2- 172.21.15.47:0/127552007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7aa4142a90 0x7f7aa40cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:03.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.684+0000 7f7aa909b640 1 --2- 172.21.15.47:0/127552007 >> v2:172.21.15.175:3300/0 conn(0x7f7aa4143010 0x7f7aa40b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:03.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.685+0000 7f7aaa09d640 1 -- 172.21.15.47:0/127552007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7aa4142a90 msgr2=0x7f7aa40cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:03.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.685+0000 7f7aaa09d640 1 --2- 172.21.15.47:0/127552007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7aa4142a90 0x7f7aa40cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:03.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.685+0000 7f7aaa09d640 1 -- 172.21.15.47:0/127552007 >> v2:172.21.15.175:3300/0 conn(0x7f7aa4143010 msgr2=0x7f7aa40b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:03.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.685+0000 7f7aaa09d640 1 --2- 172.21.15.47:0/127552007 >> v2:172.21.15.175:3300/0 conn(0x7f7aa4143010 0x7f7aa40b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:03.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.685+0000 7f7aaa09d640 1 -- 172.21.15.47:0/127552007 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa40ccfe0 con 0x7f7aa40b40e0 2024-03-23T09:50:03.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.685+0000 7f7aa909b640 1 --2- 172.21.15.47:0/127552007 >> v2:172.21.15.175:3300/0 conn(0x7f7aa4143010 0x7f7aa40b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:03.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.685+0000 7f7aa989c640 1 --2- 172.21.15.47:0/127552007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7aa4142a90 0x7f7aa40cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:03.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.685+0000 7f7aaa09d640 1 --2- 172.21.15.47:0/127552007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa40b40e0 0x7f7aa415d310 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f7aa406abe0 tx=0x7f7a9402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbf72403cd0d5eb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:03.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.686+0000 7f7aa889a640 1 -- 172.21.15.47:0/127552007 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9402ca70 con 0x7f7aa40b40e0 2024-03-23T09:50:03.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.686+0000 7f7aa889a640 1 -- 172.21.15.47:0/127552007 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9402cbd0 con 0x7f7aa40b40e0 2024-03-23T09:50:03.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.686+0000 7f7aa889a640 1 -- 172.21.15.47:0/127552007 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a940343f0 con 0x7f7aa40b40e0 2024-03-23T09:50:03.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.686+0000 7f7aabb00640 1 -- 172.21.15.47:0/127552007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa40b40e0 msgr2=0x7f7aa415d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:03.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.686+0000 7f7aabb00640 1 --2- 172.21.15.47:0/127552007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa40b40e0 0x7f7aa415d310 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f7aa406abe0 tx=0x7f7a9402bed0 comp rx=0 tx=0).stop 2024-03-23T09:50:03.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.686+0000 7f7aabb00640 1 -- 172.21.15.47:0/127552007 shutdown_connections 2024-03-23T09:50:03.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.686+0000 7f7aabb00640 1 --2- 172.21.15.47:0/127552007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa40b40e0 0x7f7aa415d310 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:03.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.686+0000 7f7aabb00640 1 --2- 172.21.15.47:0/127552007 >> v2:172.21.15.175:3300/0 conn(0x7f7aa4143010 0x7f7aa40b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:03.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.686+0000 7f7aabb00640 1 --2- 172.21.15.47:0/127552007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7aa4142a90 0x7f7aa40cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:03.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.686+0000 7f7aabb00640 1 -- 172.21.15.47:0/127552007 >> 172.21.15.47:0/127552007 conn(0x7f7aa406a310 msgr2=0x7f7aa40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:03.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.686+0000 7f7aabb00640 1 -- 172.21.15.47:0/127552007 shutdown_connections 2024-03-23T09:50:03.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.686+0000 7f7aabb00640 1 -- 172.21.15.47:0/127552007 wait complete. 2024-03-23T09:50:03.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.687+0000 7f7aabb00640 1 Processor -- start 2024-03-23T09:50:03.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.688+0000 7f7aabb00640 1 -- start start 2024-03-23T09:50:03.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.688+0000 7f7aabb00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa40b40e0 0x7f7aa4158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:03.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.688+0000 7f7aabb00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7aa4142a90 0x7f7aa40a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:03.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.688+0000 7f7aabb00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7aa4143010 0x7f7aa40aa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:03.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.688+0000 7f7aabb00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40a95d0 con 0x7f7aa40b40e0 2024-03-23T09:50:03.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.688+0000 7f7aabb00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4159240 con 0x7f7aa4143010 2024-03-23T09:50:03.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.688+0000 7f7aabb00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40aa7d0 con 0x7f7aa4142a90 2024-03-23T09:50:03.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.688+0000 7f7aa989c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa40b40e0 0x7f7aa4158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:03.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.688+0000 7f7aa989c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa40b40e0 0x7f7aa4158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55304/0 (socket says 172.21.15.47:55304) 2024-03-23T09:50:03.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.688+0000 7f7aa989c640 1 -- 172.21.15.47:0/1219408924 learned_addr learned my addr 172.21.15.47:0/1219408924 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:03.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aa909b640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7aa4142a90 0x7f7aa40a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aaa09d640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7aa4143010 0x7f7aa40aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aa989c640 1 -- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7aa4142a90 msgr2=0x7f7aa40a9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aa989c640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7aa4142a90 0x7f7aa40a9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aa989c640 1 -- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7aa4143010 msgr2=0x7f7aa40aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aa989c640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7aa4143010 0x7f7aa40aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aa989c640 1 -- 172.21.15.47:0/1219408924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a94009b30 con 0x7f7aa40b40e0 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aa909b640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7aa4142a90 0x7f7aa40a9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aaa09d640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7aa4143010 0x7f7aa40aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aa989c640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa40b40e0 0x7f7aa4158d00 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e810 tx=0x7f7a9c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aa27fc640 1 -- 172.21.15.47:0/1219408924 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c004030 con 0x7f7aa40b40e0 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aa27fc640 1 -- 172.21.15.47:0/1219408924 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c004190 con 0x7f7aa40b40e0 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aabb00640 1 -- 172.21.15.47:0/1219408924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa40aaab0 con 0x7f7aa40b40e0 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aa27fc640 1 -- 172.21.15.47:0/1219408924 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c004520 con 0x7f7aa40b40e0 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.689+0000 7f7aabb00640 1 -- 172.21.15.47:0/1219408924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa40ab070 con 0x7f7aa40b40e0 2024-03-23T09:50:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.690+0000 7f7aabb00640 1 -- 172.21.15.47:0/1219408924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa4073480 con 0x7f7aa40b40e0 2024-03-23T09:50:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.691+0000 7f7aa27fc640 1 -- 172.21.15.47:0/1219408924 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7a9c00c6d0 con 0x7f7aa40b40e0 2024-03-23T09:50:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.691+0000 7f7aa27fc640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a9006ac30 0x7f7a9006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.691+0000 7f7aa27fc640 1 -- 172.21.15.47:0/1219408924 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a9c088f10 con 0x7f7aa40b40e0 2024-03-23T09:50:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.691+0000 7f7aa909b640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a9006ac30 0x7f7a9006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.692+0000 7f7aa909b640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a9006ac30 0x7f7a9006d0f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7aa41f3c60 tx=0x7f7a98008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.696+0000 7f7aa27fc640 1 -- 172.21.15.47:0/1219408924 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7a9c054060 con 0x7f7aa40b40e0 2024-03-23T09:50:03.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.911+0000 7f7aabb00640 1 -- 172.21.15.47:0/1219408924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7aa4066be0 con 0x7f7aa40b40e0 2024-03-23T09:50:03.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.912+0000 7f7aa27fc640 1 -- 172.21.15.47:0/1219408924 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f7a9c020020 con 0x7f7aa40b40e0 2024-03-23T09:50:03.913 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:03.913 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/249416109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":249416109},{"type":"v1","addr":"172.21.15.175:6837","nonce":249416109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3603846321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3603846321},{"type":"v1","addr":"172.21.15.175:6835","nonce":3603846321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1921094794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1921094794},{"type":"v1","addr":"172.21.15.47:6834","nonce":1921094794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:49:43.247119+0000","modified":"2024-03-23T09:49:45.258459+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8484":{"gid":8484,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/2388298413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2388298413},{"type":"v1","addr":"172.21.15.47:6835","nonce":2388298413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-23T09:50:03.914 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 65 2024-03-23T09:50:03.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.914+0000 7f7aabb00640 1 -- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a9006ac30 msgr2=0x7f7a9006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.914+0000 7f7aabb00640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a9006ac30 0x7f7a9006d0f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7aa41f3c60 tx=0x7f7a98008040 comp rx=0 tx=0).stop 2024-03-23T09:50:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.914+0000 7f7aabb00640 1 -- 172.21.15.47:0/1219408924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa40b40e0 msgr2=0x7f7aa4158d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.914+0000 7f7aabb00640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa40b40e0 0x7f7aa4158d00 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e810 tx=0x7f7a9c00ecd0 comp rx=0 tx=0).stop 2024-03-23T09:50:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.914+0000 7f7aabb00640 1 -- 172.21.15.47:0/1219408924 shutdown_connections 2024-03-23T09:50:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.914+0000 7f7aabb00640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a9006ac30 0x7f7a9006d0f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.914+0000 7f7aabb00640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7aa4143010 0x7f7aa40aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.914+0000 7f7aabb00640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7aa4142a90 0x7f7aa40a9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.914+0000 7f7aabb00640 1 --2- 172.21.15.47:0/1219408924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa40b40e0 0x7f7aa4158d00 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.914+0000 7f7aabb00640 1 -- 172.21.15.47:0/1219408924 >> 172.21.15.47:0/1219408924 conn(0x7f7aa406a310 msgr2=0x7f7aa406d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.914+0000 7f7aabb00640 1 -- 172.21.15.47:0/1219408924 shutdown_connections 2024-03-23T09:50:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:03.914+0000 7f7aabb00640 1 -- 172.21.15.47:0/1219408924 wait complete. 2024-03-23T09:50:03.926 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-03-23T09:50:04.051 INFO:teuthology.orchestra.run.smithi047.stdout:{"message":"","return_code":0} 2024-03-23T09:50:04.051 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-23T09:50:04.052 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:50:04.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.151+0000 7f341ca09640 1 Processor -- start 2024-03-23T09:50:04.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.151+0000 7f341ca09640 1 -- start start 2024-03-23T09:50:04.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.152+0000 7f341ca09640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f341815aa70 0x7f34180cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.152+0000 7f341ca09640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f34180cd620 0x7f34180ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.152+0000 7f341ca09640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341815ae50 con 0x7f34180ce800 2024-03-23T09:50:04.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.152+0000 7f341ca09640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34180cda90 con 0x7f34180cd620 2024-03-23T09:50:04.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.152+0000 7f341ca09640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34180cf010 con 0x7f341815aa70 2024-03-23T09:50:04.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.152+0000 7f3415d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f34180cd620 0x7f34180ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.152+0000 7f3415d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f34180cd620 0x7f34180ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55306/0 (socket says 172.21.15.47:55306) 2024-03-23T09:50:04.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.152+0000 7f3415d9b640 1 -- 172.21.15.47:0/727883398 learned_addr learned my addr 172.21.15.47:0/727883398 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:04.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.153+0000 7f341659c640 1 --2- 172.21.15.47:0/727883398 >> v2:172.21.15.175:3300/0 conn(0x7f341815aa70 0x7f34180cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.153+0000 7f3415d9b640 1 -- 172.21.15.47:0/727883398 >> v2:172.21.15.175:3300/0 conn(0x7f341815aa70 msgr2=0x7f34180cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:04.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.153+0000 7f3415d9b640 1 --2- 172.21.15.47:0/727883398 >> v2:172.21.15.175:3300/0 conn(0x7f341815aa70 0x7f34180cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.153+0000 7f3415d9b640 1 -- 172.21.15.47:0/727883398 >> v1:172.21.15.175:6789/0 conn(0x7f34180ce800 legacy=0x7f3418098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:04.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.153+0000 7f3415d9b640 1 -- 172.21.15.47:0/727883398 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34180cf2f0 con 0x7f34180cd620 2024-03-23T09:50:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.153+0000 7f3415d9b640 1 --2- 172.21.15.47:0/727883398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34180cd620 0x7f34180ce2c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f34180b32b0 tx=0x7f340c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad0ad32c8f3e95f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.154+0000 7f341559a640 1 -- 172.21.15.47:0/727883398 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c030840 con 0x7f34180cd620 2024-03-23T09:50:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.154+0000 7f341559a640 1 -- 172.21.15.47:0/727883398 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c0309a0 con 0x7f34180cd620 2024-03-23T09:50:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.154+0000 7f341559a640 1 -- 172.21.15.47:0/727883398 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c030d30 con 0x7f34180cd620 2024-03-23T09:50:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.154+0000 7f341ca09640 1 -- 172.21.15.47:0/727883398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34180cd620 msgr2=0x7f34180ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.154+0000 7f341ca09640 1 --2- 172.21.15.47:0/727883398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34180cd620 0x7f34180ce2c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f34180b32b0 tx=0x7f340c02b980 comp rx=0 tx=0).stop 2024-03-23T09:50:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.154+0000 7f341ca09640 1 -- 172.21.15.47:0/727883398 shutdown_connections 2024-03-23T09:50:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.154+0000 7f341ca09640 1 --2- 172.21.15.47:0/727883398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34180cd620 0x7f34180ce2c0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.154+0000 7f341ca09640 1 --2- 172.21.15.47:0/727883398 >> v2:172.21.15.175:3300/0 conn(0x7f341815aa70 0x7f34180cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.154+0000 7f341ca09640 1 -- 172.21.15.47:0/727883398 >> 172.21.15.47:0/727883398 conn(0x7f34180d06f0 msgr2=0x7f34180d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.154+0000 7f341ca09640 1 -- 172.21.15.47:0/727883398 shutdown_connections 2024-03-23T09:50:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.154+0000 7f341ca09640 1 -- 172.21.15.47:0/727883398 wait complete. 2024-03-23T09:50:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.155+0000 7f341ca09640 1 Processor -- start 2024-03-23T09:50:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.156+0000 7f341ca09640 1 -- start start 2024-03-23T09:50:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.156+0000 7f341ca09640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34180cd620 0x7f34180b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.156+0000 7f341ca09640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34180ce800 0x7f34181ed5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.156+0000 7f341ca09640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341815aa70 0x7f34181f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.156+0000 7f341ca09640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341806fc20 con 0x7f341815aa70 2024-03-23T09:50:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.156+0000 7f341ca09640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34181edb20 con 0x7f34180cd620 2024-03-23T09:50:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.156+0000 7f341ca09640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34180b3980 con 0x7f34180ce800 2024-03-23T09:50:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.156+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341815aa70 0x7f34181f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.156+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341815aa70 0x7f34181f12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55308/0 (socket says 172.21.15.47:55308) 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.156+0000 7f3416d9d640 1 -- 172.21.15.47:0/4198319714 learned_addr learned my addr 172.21.15.47:0/4198319714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.156+0000 7f3415d9b640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34180ce800 0x7f34181ed5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.157+0000 7f341659c640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34180cd620 0x7f34180b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.157+0000 7f3416d9d640 1 -- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34180ce800 msgr2=0x7f34181ed5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.157+0000 7f3416d9d640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34180ce800 0x7f34181ed5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.157+0000 7f3416d9d640 1 -- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34180cd620 msgr2=0x7f34180b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.157+0000 7f3416d9d640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34180cd620 0x7f34180b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.157+0000 7f3416d9d640 1 -- 172.21.15.47:0/4198319714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c0094d0 con 0x7f341815aa70 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.157+0000 7f3416d9d640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341815aa70 0x7f34181f12e0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f341000c410 tx=0x7f341000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.157+0000 7f3406ffd640 1 -- 172.21.15.47:0/4198319714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410011790 con 0x7f341815aa70 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.157+0000 7f3406ffd640 1 -- 172.21.15.47:0/4198319714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34100118f0 con 0x7f341815aa70 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.157+0000 7f341ca09640 1 -- 172.21.15.47:0/4198319714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34180b3c60 con 0x7f341815aa70 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.157+0000 7f3406ffd640 1 -- 172.21.15.47:0/4198319714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410011ce0 con 0x7f341815aa70 2024-03-23T09:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.157+0000 7f341ca09640 1 -- 172.21.15.47:0/4198319714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34180b4220 con 0x7f341815aa70 2024-03-23T09:50:04.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.158+0000 7f341ca09640 1 -- 172.21.15.47:0/4198319714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3418066be0 con 0x7f341815aa70 2024-03-23T09:50:04.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.158+0000 7f3406ffd640 1 -- 172.21.15.47:0/4198319714 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f341001c460 con 0x7f341815aa70 2024-03-23T09:50:04.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.159+0000 7f3406ffd640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f33f806ac00 0x7f33f806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.159+0000 7f3406ffd640 1 -- 172.21.15.47:0/4198319714 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3410089000 con 0x7f341815aa70 2024-03-23T09:50:04.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.159+0000 7f341659c640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f33f806ac00 0x7f33f806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.160+0000 7f341659c640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f33f806ac00 0x7f33f806d0c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f34080099c0 tx=0x7f3408003990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.163+0000 7f3406ffd640 1 -- 172.21.15.47:0/4198319714 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f34100541e0 con 0x7f341815aa70 2024-03-23T09:50:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.379+0000 7f341ca09640 1 -- 172.21.15.47:0/4198319714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f341806ee70 con 0x7f341815aa70 2024-03-23T09:50:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.380+0000 7f3406ffd640 1 -- 172.21.15.47:0/4198319714 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f3410057800 con 0x7f341815aa70 2024-03-23T09:50:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:04.381 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/249416109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":249416109},{"type":"v1","addr":"172.21.15.175:6837","nonce":249416109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3603846321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3603846321},{"type":"v1","addr":"172.21.15.175:6835","nonce":3603846321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1921094794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1921094794},{"type":"v1","addr":"172.21.15.47:6834","nonce":1921094794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:49:43.247119+0000","modified":"2024-03-23T09:49:45.258459+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8484":{"gid":8484,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/2388298413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2388298413},{"type":"v1","addr":"172.21.15.47:6835","nonce":2388298413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-23T09:50:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 65 2024-03-23T09:50:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.381+0000 7f341ca09640 1 -- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f33f806ac00 msgr2=0x7f33f806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.381+0000 7f341ca09640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f33f806ac00 0x7f33f806d0c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f34080099c0 tx=0x7f3408003990 comp rx=0 tx=0).stop 2024-03-23T09:50:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.382+0000 7f341ca09640 1 -- 172.21.15.47:0/4198319714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341815aa70 msgr2=0x7f34181f12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.382+0000 7f341ca09640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341815aa70 0x7f34181f12e0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f341000c410 tx=0x7f341000c8d0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.382+0000 7f341ca09640 1 -- 172.21.15.47:0/4198319714 shutdown_connections 2024-03-23T09:50:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.382+0000 7f341ca09640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f33f806ac00 0x7f33f806d0c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.382+0000 7f341ca09640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341815aa70 0x7f34181f12e0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.382+0000 7f341ca09640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34180ce800 0x7f34181ed5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.382+0000 7f341ca09640 1 --2- 172.21.15.47:0/4198319714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34180cd620 0x7f34180b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.382+0000 7f341ca09640 1 -- 172.21.15.47:0/4198319714 >> 172.21.15.47:0/4198319714 conn(0x7f34180d06f0 msgr2=0x7f3418158850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:04.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.382+0000 7f341ca09640 1 -- 172.21.15.47:0/4198319714 shutdown_connections 2024-03-23T09:50:04.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.382+0000 7f341ca09640 1 -- 172.21.15.47:0/4198319714 wait complete. 2024-03-23T09:50:04.394 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-03-23T09:50:04.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.433+0000 7f5e1e839f40 1 Processor -- start 2024-03-23T09:50:04.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.433+0000 7f5e1e839f40 1 -- start start 2024-03-23T09:50:04.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1e839f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56060e96ce10 0x56060ea35180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1e839f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56060ea3f140 0x56060ea37c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1e839f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56060ea3fa60 0x56060ea381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1e839f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060e8b2800 con 0x56060ea3f140 2024-03-23T09:50:04.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1e839f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060e895520 con 0x56060e96ce10 2024-03-23T09:50:04.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1e839f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x56060ea28c30 con 0x56060ea3fa60 2024-03-23T09:50:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1b41f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56060ea3f140 0x56060ea37c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1b41f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56060ea3f140 0x56060ea37c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55320/0 (socket says 172.21.15.47:55320) 2024-03-23T09:50:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1b41f640 1 -- 172.21.15.47:0/2419091964 learned_addr learned my addr 172.21.15.47:0/2419091964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1c421640 1 --2- 172.21.15.47:0/2419091964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56060ea3fa60 0x56060ea381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1bc20640 1 --2- 172.21.15.47:0/2419091964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56060e96ce10 0x56060ea35180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1b41f640 1 -- 172.21.15.47:0/2419091964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56060ea3fa60 msgr2=0x56060ea381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1b41f640 1 --2- 172.21.15.47:0/2419091964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56060ea3fa60 0x56060ea381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1b41f640 1 -- 172.21.15.47:0/2419091964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56060e96ce10 msgr2=0x56060ea35180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1b41f640 1 --2- 172.21.15.47:0/2419091964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56060e96ce10 0x56060ea35180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1b41f640 1 -- 172.21.15.47:0/2419091964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e04005de0 con 0x56060ea3f140 2024-03-23T09:50:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.434+0000 7f5e1b41f640 1 --2- 172.21.15.47:0/2419091964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56060ea3f140 0x56060ea37c70 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f5e04055040 tx=0x7f5e0404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712535ec3f5b582b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.435+0000 7f5e1a41d640 1 -- 172.21.15.47:0/2419091964 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e04052910 con 0x56060ea3f140 2024-03-23T09:50:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.435+0000 7f5e1a41d640 1 -- 172.21.15.47:0/2419091964 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e04052a90 con 0x56060ea3f140 2024-03-23T09:50:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.435+0000 7f5e1a41d640 1 -- 172.21.15.47:0/2419091964 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e04052e40 con 0x56060ea3f140 2024-03-23T09:50:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.435+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2419091964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56060ea3f140 msgr2=0x56060ea37c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.435+0000 7f5e1e839f40 1 --2- 172.21.15.47:0/2419091964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56060ea3f140 0x56060ea37c70 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f5e04055040 tx=0x7f5e0404c000 comp rx=0 tx=0).stop 2024-03-23T09:50:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.435+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2419091964 shutdown_connections 2024-03-23T09:50:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.435+0000 7f5e1e839f40 1 --2- 172.21.15.47:0/2419091964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56060ea3fa60 0x56060ea381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.435+0000 7f5e1e839f40 1 --2- 172.21.15.47:0/2419091964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56060ea3f140 0x56060ea37c70 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.435+0000 7f5e1e839f40 1 --2- 172.21.15.47:0/2419091964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56060e96ce10 0x56060ea35180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.435+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2419091964 >> 172.21.15.47:0/2419091964 conn(0x56060e995e40 msgr2=0x56060e958c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.435+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2419091964 shutdown_connections 2024-03-23T09:50:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.435+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2419091964 wait complete. 2024-03-23T09:50:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.436+0000 7f5e1e839f40 1 Processor -- start 2024-03-23T09:50:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.436+0000 7f5e1e839f40 1 -- start start 2024-03-23T09:50:04.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.436+0000 7f5e1e839f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56060e96ce10 0x56060ea2b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.436+0000 7f5e1e839f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56060ea3f140 0x56060ea2e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.436+0000 7f5e1e839f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56060ea3fa60 0x56060ea2eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.436+0000 7f5e1e839f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060ea27f90 con 0x56060ea3f140 2024-03-23T09:50:04.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.436+0000 7f5e1e839f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060ea28270 con 0x56060ea3fa60 2024-03-23T09:50:04.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.436+0000 7f5e1e839f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x56060ea2a870 con 0x56060e96ce10 2024-03-23T09:50:04.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1b41f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56060ea3f140 0x56060ea2e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1b41f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56060ea3f140 0x56060ea2e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55326/0 (socket says 172.21.15.47:55326) 2024-03-23T09:50:04.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1b41f640 1 -- 172.21.15.47:0/2960491630 learned_addr learned my addr 172.21.15.47:0/2960491630 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:04.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1bc20640 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56060e96ce10 0x56060ea2b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1c421640 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56060ea3fa60 0x56060ea2eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1b41f640 1 -- 172.21.15.47:0/2960491630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56060e96ce10 msgr2=0x56060ea2b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1b41f640 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56060e96ce10 0x56060ea2b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1b41f640 1 -- 172.21.15.47:0/2960491630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56060ea3fa60 msgr2=0x56060ea2eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1b41f640 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56060ea3fa60 0x56060ea2eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1b41f640 1 -- 172.21.15.47:0/2960491630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56060e8b4cd0 con 0x56060ea3f140 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1b41f640 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56060ea3f140 0x56060ea2e630 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f5e0401d200 tx=0x7f5e04002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1941b640 1 -- 172.21.15.47:0/2960491630 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0406a070 con 0x56060ea3f140 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2960491630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56060e8dbdc0 con 0x56060ea3f140 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1941b640 1 -- 172.21.15.47:0/2960491630 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e04052e90 con 0x56060ea3f140 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1941b640 1 -- 172.21.15.47:0/2960491630 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e04065690 con 0x56060ea3f140 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.437+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2960491630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56060e8c1c20 con 0x56060ea3f140 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.438+0000 7f5e1941b640 1 -- 172.21.15.47:0/2960491630 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5e04059070 con 0x56060ea3f140 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.439+0000 7f5e1941b640 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5dfc0695e0 0x7f5dfc06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.439+0000 7f5e1941b640 1 -- 172.21.15.47:0/2960491630 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e040cb210 con 0x56060ea3f140 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.439+0000 7f5e1bc20640 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5dfc0695e0 0x7f5dfc06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.439+0000 7f5e1e839f40 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x56060ea27a90 0x56060ea36b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.439+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2960491630 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) v8 -- 0x56060ea23630 con 0x56060ea27a90 2024-03-23T09:50:04.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.439+0000 7f5e1c421640 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x56060ea27a90 0x56060ea36b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1c421640 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x56060ea27a90 0x56060ea36b30 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:04.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1bc20640 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5dfc0695e0 0x7f5dfc06baa0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c005fd0 tx=0x7f5e0c007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:04.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1c421640 1 -- 172.21.15.47:0/2960491630 <== osd.2 v2:172.21.15.47:6800/37678 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7f5e14007780 con 0x56060ea27a90 2024-03-23T09:50:04.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2960491630 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x56060ea27a90 msgr2=0x56060ea36b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1e839f40 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x56060ea27a90 0x56060ea36b30 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2960491630 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5dfc0695e0 msgr2=0x7f5dfc06baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1e839f40 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5dfc0695e0 0x7f5dfc06baa0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c005fd0 tx=0x7f5e0c007040 comp rx=0 tx=0).stop 2024-03-23T09:50:04.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2960491630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56060ea3f140 msgr2=0x56060ea2e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1e839f40 1 --2- 172.21.15.47:0/2960491630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56060ea3f140 0x56060ea2e630 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f5e0401d200 tx=0x7f5e04002fb0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1c421640 1 -- 172.21.15.47:0/2960491630 reap_dead start 2024-03-23T09:50:04.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2960491630 shutdown_connections 2024-03-23T09:50:04.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2960491630 >> 172.21.15.47:0/2960491630 conn(0x56060e995e40 msgr2=0x56060ea35180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:04.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2960491630 shutdown_connections 2024-03-23T09:50:04.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.440+0000 7f5e1e839f40 1 -- 172.21.15.47:0/2960491630 wait complete. 2024-03-23T09:50:04.447 INFO:teuthology.orchestra.run.smithi047.stdout:����� 2024-03-23T09:50:04.448 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:50:04.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.586+0000 7f3628f55640 1 Processor -- start 2024-03-23T09:50:04.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.586+0000 7f3628f55640 1 -- start start 2024-03-23T09:50:04.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.587+0000 7f3628f55640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f362415cc40 0x7f3624160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.587+0000 7f3628f55640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f36240cddc0 0x7f36240d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.587+0000 7f3628f55640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36240d09d0 con 0x7f3624160c90 2024-03-23T09:50:04.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.587+0000 7f3628f55640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36240b5a80 con 0x7f362415cc40 2024-03-23T09:50:04.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.587+0000 7f3628f55640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362415d020 con 0x7f36240cddc0 2024-03-23T09:50:04.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.587+0000 7f362259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f362415cc40 0x7f3624160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.587+0000 7f3621d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3624160c90 0x7f36240cd6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59816/0 (socket says 172.21.15.47:59816) 2024-03-23T09:50:04.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.587+0000 7f362259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f362415cc40 0x7f3624160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55338/0 (socket says 172.21.15.47:55338) 2024-03-23T09:50:04.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.587+0000 7f3621d9b640 1 -- 172.21.15.47:0/3664442314 learned_addr learned my addr 172.21.15.47:0/3664442314 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:04.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.587+0000 7f3622d9d640 1 --2- 172.21.15.47:0/3664442314 >> v2:172.21.15.175:3300/0 conn(0x7f36240cddc0 0x7f36240d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.587+0000 7f362259c640 1 -- 172.21.15.47:0/3664442314 >> v2:172.21.15.175:3300/0 conn(0x7f36240cddc0 msgr2=0x7f36240d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:04.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.588+0000 7f362259c640 1 --2- 172.21.15.47:0/3664442314 >> v2:172.21.15.175:3300/0 conn(0x7f36240cddc0 0x7f36240d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.588+0000 7f362259c640 1 -- 172.21.15.47:0/3664442314 >> v1:172.21.15.47:6789/0 conn(0x7f3624160c90 legacy=0x7f36240cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.588+0000 7f362259c640 1 -- 172.21.15.47:0/3664442314 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36241611a0 con 0x7f362415cc40 2024-03-23T09:50:04.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.588+0000 7f362259c640 1 --2- 172.21.15.47:0/3664442314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362415cc40 0x7f3624160750 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f36240b5850 tx=0x7f361402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa8d272d8807ae6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:04.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.588+0000 7f362159a640 1 -- 172.21.15.47:0/3664442314 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361402cc80 con 0x7f362415cc40 2024-03-23T09:50:04.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.588+0000 7f362159a640 1 -- 172.21.15.47:0/3664442314 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361402cde0 con 0x7f362415cc40 2024-03-23T09:50:04.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.588+0000 7f362159a640 1 -- 172.21.15.47:0/3664442314 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614034660 con 0x7f362415cc40 2024-03-23T09:50:04.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.589+0000 7f3628f55640 1 -- 172.21.15.47:0/3664442314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362415cc40 msgr2=0x7f3624160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.589+0000 7f3628f55640 1 --2- 172.21.15.47:0/3664442314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362415cc40 0x7f3624160750 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f36240b5850 tx=0x7f361402bed0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.589+0000 7f3628f55640 1 -- 172.21.15.47:0/3664442314 shutdown_connections 2024-03-23T09:50:04.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.589+0000 7f3628f55640 1 --2- 172.21.15.47:0/3664442314 >> v2:172.21.15.175:3300/0 conn(0x7f36240cddc0 0x7f36240d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.589+0000 7f3628f55640 1 --2- 172.21.15.47:0/3664442314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362415cc40 0x7f3624160750 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.589+0000 7f3628f55640 1 -- 172.21.15.47:0/3664442314 >> 172.21.15.47:0/3664442314 conn(0x7f362406b160 msgr2=0x7f362406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:04.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.589+0000 7f3628f55640 1 -- 172.21.15.47:0/3664442314 shutdown_connections 2024-03-23T09:50:04.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.589+0000 7f3628f55640 1 -- 172.21.15.47:0/3664442314 wait complete. 2024-03-23T09:50:04.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.590+0000 7f3628f55640 1 Processor -- start 2024-03-23T09:50:04.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.590+0000 7f3628f55640 1 -- start start 2024-03-23T09:50:04.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3628f55640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36240cddc0 0x7f3624153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3628f55640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f362415cc40 0x7f3624154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3628f55640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3624160c90 0x7f3624151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3628f55640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3624153d30 con 0x7f3624160c90 2024-03-23T09:50:04.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3628f55640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3624154010 con 0x7f36240cddc0 2024-03-23T09:50:04.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3628f55640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36241542f0 con 0x7f362415cc40 2024-03-23T09:50:04.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3622d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3624160c90 0x7f3624151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3622d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3624160c90 0x7f3624151dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55352/0 (socket says 172.21.15.47:55352) 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3622d9d640 1 -- 172.21.15.47:0/526988118 learned_addr learned my addr 172.21.15.47:0/526988118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3622d9d640 1 -- 172.21.15.47:0/526988118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f362415cc40 msgr2=0x7f3624154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3622d9d640 1 --2- 172.21.15.47:0/526988118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f362415cc40 0x7f3624154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3622d9d640 1 -- 172.21.15.47:0/526988118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36240cddc0 msgr2=0x7f3624153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3622d9d640 1 --2- 172.21.15.47:0/526988118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36240cddc0 0x7f3624153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.591+0000 7f3622d9d640 1 -- 172.21.15.47:0/526988118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614009a70 con 0x7f3624160c90 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.592+0000 7f3622d9d640 1 --2- 172.21.15.47:0/526988118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3624160c90 0x7f3624151dd0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c7f0 tx=0x7f361c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.592+0000 7f361affd640 1 -- 172.21.15.47:0/526988118 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c004220 con 0x7f3624160c90 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.592+0000 7f361affd640 1 -- 172.21.15.47:0/526988118 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c004380 con 0x7f3624160c90 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.592+0000 7f3628f55640 1 -- 172.21.15.47:0/526988118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36241545d0 con 0x7f3624160c90 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.592+0000 7f361affd640 1 -- 172.21.15.47:0/526988118 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c007860 con 0x7f3624160c90 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.592+0000 7f3628f55640 1 -- 172.21.15.47:0/526988118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3624155310 con 0x7f3624160c90 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.593+0000 7f3628f55640 1 -- 172.21.15.47:0/526988118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3624072360 con 0x7f3624160c90 2024-03-23T09:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.593+0000 7f361affd640 1 -- 172.21.15.47:0/526988118 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f361c007bd0 con 0x7f3624160c90 2024-03-23T09:50:04.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.594+0000 7f361affd640 1 --2- 172.21.15.47:0/526988118 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f360406ac00 0x7f360406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:04.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.594+0000 7f361affd640 1 -- 172.21.15.47:0/526988118 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f361c089270 con 0x7f3624160c90 2024-03-23T09:50:04.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.594+0000 7f362259c640 1 --2- 172.21.15.47:0/526988118 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f360406ac00 0x7f360406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:04.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.594+0000 7f362259c640 1 --2- 172.21.15.47:0/526988118 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f360406ac00 0x7f360406d0c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3614036000 tx=0x7f361402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:04.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.598+0000 7f361affd640 1 -- 172.21.15.47:0/526988118 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f361c0544e0 con 0x7f3624160c90 2024-03-23T09:50:04.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.816+0000 7f3628f55640 1 -- 172.21.15.47:0/526988118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3624154150 con 0x7f3624160c90 2024-03-23T09:50:04.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.816+0000 7f361affd640 1 -- 172.21.15.47:0/526988118 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3624154150 con 0x7f3624160c90 2024-03-23T09:50:04.817 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:50:04.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.818+0000 7f3628f55640 1 -- 172.21.15.47:0/526988118 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f360406ac00 msgr2=0x7f360406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.818+0000 7f3628f55640 1 --2- 172.21.15.47:0/526988118 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f360406ac00 0x7f360406d0c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3614036000 tx=0x7f361402e000 comp rx=0 tx=0).stop 2024-03-23T09:50:04.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.818+0000 7f3628f55640 1 -- 172.21.15.47:0/526988118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3624160c90 msgr2=0x7f3624151dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:04.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.818+0000 7f3628f55640 1 --2- 172.21.15.47:0/526988118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3624160c90 0x7f3624151dd0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c7f0 tx=0x7f361c00ccb0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.818+0000 7f3628f55640 1 -- 172.21.15.47:0/526988118 shutdown_connections 2024-03-23T09:50:04.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.818+0000 7f3628f55640 1 --2- 172.21.15.47:0/526988118 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f360406ac00 0x7f360406d0c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.818+0000 7f3628f55640 1 --2- 172.21.15.47:0/526988118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3624160c90 0x7f3624151dd0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.818+0000 7f3628f55640 1 --2- 172.21.15.47:0/526988118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f362415cc40 0x7f3624154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.818+0000 7f3628f55640 1 --2- 172.21.15.47:0/526988118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36240cddc0 0x7f3624153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:04.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.818+0000 7f3628f55640 1 -- 172.21.15.47:0/526988118 >> 172.21.15.47:0/526988118 conn(0x7f362406b160 msgr2=0x7f36240cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:04.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.818+0000 7f3628f55640 1 -- 172.21.15.47:0/526988118 shutdown_connections 2024-03-23T09:50:04.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:04.818+0000 7f3628f55640 1 -- 172.21.15.47:0/526988118 wait complete. 2024-03-23T09:50:04.830 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:50:04.830 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:50:04.830 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:50:04.830 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:50:04.830 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:50:04.830 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:50:04.831 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:50:04.831 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:50:04.831 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:50:04.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:04.831 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:50:04.846 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:50:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:50:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:50:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:50:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:50:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42134sec preferred_lft 42134sec 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:50:04.847 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:50:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:50:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:50:04.848 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:50:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:50:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:50:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:50:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:50:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:50:04.848 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:50:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:50:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:50:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:50:04.848 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:50:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:50:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:50:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:50:04.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:04.849 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:50:04.912 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:50:04.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:04.913 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:50:04.969 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:50:04.969 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:50:04.969 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:50:04.970 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:50:04.970 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:50:04.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:04.970 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:50:04.970 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:50:04.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:50:04.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:50:04.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:50:04.970 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:50:05.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:05.064 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:50:05.079 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:50:05.079 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:50:05.079 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:50:05.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:05.079 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:50:05.146 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:50:05.146 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:50:05.146 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:50:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:50:05.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:50:05.201 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:50:05.255 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:50:05.322 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:50:05.322 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:50:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:50:05.389 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:50:05.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:05.390 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:50:05.444 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:50:05.444 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:50:05.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:50:05.513 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:50:05.513 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:50:05.572 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:50:05.571+0000 7ff66544b480 -1 init, newargv = 0x563b0aac9f20 newargc=15 2024-03-23T09:50:05.573 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[74927]: starting ceph client 2024-03-23T09:50:05.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[74927]: starting fuse 2024-03-23T09:50:05.583 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:50:05.584 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:50:05.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:05.585 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:50:05.639 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:50:05.640 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:50:05.640 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:50:05.640 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:50:05.640 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:50:05.640 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:50:05.640 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:50:05.640 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:50:05.640 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:50:05.640 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:50:05.640 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:50:05.640 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:50:05.640 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:50:05.641 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:50:05.642 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:50:05.756 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.74927.asok 2024-03-23T09:50:05.943 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.74927.asok 2024-03-23T09:50:05.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:05.943 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.74927.asok status 2024-03-23T09:50:06.079 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:50:06.079 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:50:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:50:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:50:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:50:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:50:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:50:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "74927", 2024-03-23T09:50:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:50:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:50:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:50:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:50:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 8754, 2024-03-23T09:50:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:50:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 8754 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3068025187 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3068025187 2024-03-23T09:50:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:50:06.082 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.8754 192.168.0.1:0/3068025187", 2024-03-23T09:50:06.082 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/3068025187", 2024-03-23T09:50:06.082 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:50:06.082 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 65, 2024-03-23T09:50:06.082 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 101, 2024-03-23T09:50:06.082 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:50:06.082 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:50:06.082 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:50:06.082 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:50:06.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:06.088 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:50:06.088 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:50:06.103 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:50:06.103 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:50:06.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:06.103 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:50:06.169 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:50:06.170 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:50:06.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:06.225 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-23T09:50:06.320 INFO:teuthology.orchestra.run.smithi047.stderr:6+0 records in 2024-03-23T09:50:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:6+0 records out 2024-03-23T09:50:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0372951 s, 169 MB/s 2024-03-23T09:50:06.321 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:50:06.322 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:50:06.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:06.336 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-23T09:50:06.434 INFO:teuthology.orchestra.run.smithi047.stderr:6+0 records in 2024-03-23T09:50:06.434 INFO:teuthology.orchestra.run.smithi047.stderr:6+0 records out 2024-03-23T09:50:06.434 INFO:teuthology.orchestra.run.smithi047.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.039766 s, 158 MB/s 2024-03-23T09:50:06.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:06.435 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-23T09:50:06.452 INFO:teuthology.orchestra.run.smithi047.stdout:./ 2024-03-23T09:50:06.452 INFO:teuthology.orchestra.run.smithi047.stdout:./file2_sixmegs 2024-03-23T09:50:06.452 INFO:teuthology.orchestra.run.smithi047.stdout:./file3_sixmegs 2024-03-23T09:50:06.452 INFO:teuthology.orchestra.run.smithi047.stdout:./file1_sixmegs 2024-03-23T09:50:06.452 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:50:06.452 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:50:06.453 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:50:06.453 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:50:06.453 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-23T09:50:06.453 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:50:06.554 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:50:06.742 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:50:06.742 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:50:06.742 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:50:06.742 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:50:06.742 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-03-23T09:50:06.742 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:50:06.843 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627777 2024-03-23T09:50:07.033 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:50:07.033 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:50:07.033 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:50:07.033 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:50:07.033 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-03-23T09:50:07.033 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:50:07.135 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627778 2024-03-23T09:50:07.323 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:50:07.324 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:50:07.324 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:50:07.324 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:50:07.324 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-03-23T09:50:07.324 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:50:07.425 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627776 2024-03-23T09:50:07.614 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:50:07.614 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:50:07.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:50:07.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.809+0000 7ff792916640 1 Processor -- start 2024-03-23T09:50:07.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.809+0000 7ff792916640 1 -- start start 2024-03-23T09:50:07.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff792916640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78c15cd00 0x7ff78c0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:07.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff792916640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff78c0cf920 0x7ff78c0b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:07.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff792916640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c15d0e0 con 0x7ff78c0d1020 2024-03-23T09:50:07.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff792916640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78c0d1540 con 0x7ff78c0cf920 2024-03-23T09:50:07.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff792916640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0d1820 con 0x7ff78c15cd00 2024-03-23T09:50:07.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff790eb3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff78c0cf920 0x7ff78c0b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:07.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff790eb3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff78c0cf920 0x7ff78c0b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53456/0 (socket says 172.21.15.47:53456) 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff790eb3640 1 -- 172.21.15.47:0/3748665487 learned_addr learned my addr 172.21.15.47:0/3748665487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff78bfff640 1 --2- 172.21.15.47:0/3748665487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78c15cd00 0x7ff78c0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff790eb3640 1 -- 172.21.15.47:0/3748665487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78c15cd00 msgr2=0x7ff78c0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff790eb3640 1 --2- 172.21.15.47:0/3748665487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78c15cd00 0x7ff78c0d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff790eb3640 1 -- 172.21.15.47:0/3748665487 >> v1:172.21.15.175:6789/0 conn(0x7ff78c0d1020 legacy=0x7ff78c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff790eb3640 1 -- 172.21.15.47:0/3748665487 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c0d1b00 con 0x7ff78c0cf920 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.810+0000 7ff78bfff640 1 --2- 172.21.15.47:0/3748665487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78c15cd00 0x7ff78c0d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.811+0000 7ff790eb3640 1 --2- 172.21.15.47:0/3748665487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff78c0cf920 0x7ff78c0b5a90 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7ff784009e50 tx=0x7ff78402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ade503a4752e6b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.811+0000 7ff78affd640 1 -- 172.21.15.47:0/3748665487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78402c9e0 con 0x7ff78c0cf920 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.811+0000 7ff78affd640 1 -- 172.21.15.47:0/3748665487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78402cb40 con 0x7ff78c0cf920 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.811+0000 7ff78affd640 1 -- 172.21.15.47:0/3748665487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7840343f0 con 0x7ff78c0cf920 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.812+0000 7ff792916640 1 -- 172.21.15.47:0/3748665487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff78c0cf920 msgr2=0x7ff78c0b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.812+0000 7ff792916640 1 --2- 172.21.15.47:0/3748665487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff78c0cf920 0x7ff78c0b5a90 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7ff784009e50 tx=0x7ff78402be40 comp rx=0 tx=0).stop 2024-03-23T09:50:07.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.812+0000 7ff792916640 1 -- 172.21.15.47:0/3748665487 shutdown_connections 2024-03-23T09:50:07.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.812+0000 7ff792916640 1 --2- 172.21.15.47:0/3748665487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff78c0cf920 0x7ff78c0b5a90 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:07.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.812+0000 7ff792916640 1 --2- 172.21.15.47:0/3748665487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78c15cd00 0x7ff78c0d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:07.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.812+0000 7ff792916640 1 -- 172.21.15.47:0/3748665487 >> 172.21.15.47:0/3748665487 conn(0x7ff78c06b160 msgr2=0x7ff78c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:07.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.812+0000 7ff792916640 1 -- 172.21.15.47:0/3748665487 shutdown_connections 2024-03-23T09:50:07.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.812+0000 7ff792916640 1 -- 172.21.15.47:0/3748665487 wait complete. 2024-03-23T09:50:07.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.813+0000 7ff792916640 1 Processor -- start 2024-03-23T09:50:07.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.813+0000 7ff792916640 1 -- start start 2024-03-23T09:50:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff792916640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff78c0cf920 0x7ff78c0b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff792916640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff78c0d1020 0x7ff78c0c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff792916640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78c15cd00 0x7ff78c1f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff792916640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c1f5fb0 con 0x7ff78c0d1020 2024-03-23T09:50:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff792916640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c1f6290 con 0x7ff78c0cf920 2024-03-23T09:50:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff792916640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0b2130 con 0x7ff78c15cd00 2024-03-23T09:50:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff78b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff78c0d1020 0x7ff78c0c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff78b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff78c0d1020 0x7ff78c0c7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53472/0 (socket says 172.21.15.47:53472) 2024-03-23T09:50:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff78b7fe640 1 -- 172.21.15.47:0/3805247207 learned_addr learned my addr 172.21.15.47:0/3805247207 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff790eb3640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78c15cd00 0x7ff78c1f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff78bfff640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff78c0cf920 0x7ff78c0b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff78b7fe640 1 -- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78c15cd00 msgr2=0x7ff78c1f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff78b7fe640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78c15cd00 0x7ff78c1f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff78b7fe640 1 -- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff78c0cf920 msgr2=0x7ff78c0b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff78b7fe640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff78c0cf920 0x7ff78c0b2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff78b7fe640 1 -- 172.21.15.47:0/3805247207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff784009b30 con 0x7ff78c0d1020 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff78b7fe640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff78c0d1020 0x7ff78c0c7970 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00c430 tx=0x7ff77c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.814+0000 7ff788ff9640 1 -- 172.21.15.47:0/3805247207 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c011790 con 0x7ff78c0d1020 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.815+0000 7ff788ff9640 1 -- 172.21.15.47:0/3805247207 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c0118f0 con 0x7ff78c0d1020 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.815+0000 7ff792916640 1 -- 172.21.15.47:0/3805247207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c0b2410 con 0x7ff78c0d1020 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.815+0000 7ff788ff9640 1 -- 172.21.15.47:0/3805247207 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c011ce0 con 0x7ff78c0d1020 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.815+0000 7ff792916640 1 -- 172.21.15.47:0/3805247207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c0b3bc0 con 0x7ff78c0d1020 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.815+0000 7ff792916640 1 -- 172.21.15.47:0/3805247207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c072360 con 0x7ff78c0d1020 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.816+0000 7ff788ff9640 1 -- 172.21.15.47:0/3805247207 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff77c01c460 con 0x7ff78c0d1020 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.816+0000 7ff788ff9640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff77806ac00 0x7ff77806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.816+0000 7ff788ff9640 1 -- 172.21.15.47:0/3805247207 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff77c0892b0 con 0x7ff78c0d1020 2024-03-23T09:50:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.817+0000 7ff78bfff640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff77806ac00 0x7ff77806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:07.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.817+0000 7ff78bfff640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff77806ac00 0x7ff77806d0c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff78000b840 tx=0x7ff780008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.821+0000 7ff788ff9640 1 -- 172.21.15.47:0/3805247207 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff77c054520 con 0x7ff78c0d1020 2024-03-23T09:50:07.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:07.999+0000 7ff792916640 1 -- 172.21.15.47:0/3805247207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff78c066be0 con 0x7ff78c0d1020 2024-03-23T09:50:08.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.000+0000 7ff788ff9640 1 -- 172.21.15.47:0/3805247207 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16043 (secure 0 0 0) 0x7ff77c057b40 con 0x7ff78c0d1020 2024-03-23T09:50:08.001 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:08.001 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":101,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:49:44.249270+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:49:41.158056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-23T09:49:42.210388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:50:08.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.002+0000 7ff792916640 1 -- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff77806ac00 msgr2=0x7ff77806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.002+0000 7ff792916640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff77806ac00 0x7ff77806d0c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff78000b840 tx=0x7ff780008040 comp rx=0 tx=0).stop 2024-03-23T09:50:08.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.002+0000 7ff792916640 1 -- 172.21.15.47:0/3805247207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff78c0d1020 msgr2=0x7ff78c0c7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.002+0000 7ff792916640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff78c0d1020 0x7ff78c0c7970 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00c430 tx=0x7ff77c00c8f0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.002+0000 7ff792916640 1 -- 172.21.15.47:0/3805247207 shutdown_connections 2024-03-23T09:50:08.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.002+0000 7ff792916640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff77806ac00 0x7ff77806d0c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.002+0000 7ff792916640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78c15cd00 0x7ff78c1f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.002+0000 7ff792916640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff78c0d1020 0x7ff78c0c7970 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.002+0000 7ff792916640 1 --2- 172.21.15.47:0/3805247207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff78c0cf920 0x7ff78c0b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.002+0000 7ff792916640 1 -- 172.21.15.47:0/3805247207 >> 172.21.15.47:0/3805247207 conn(0x7ff78c06b160 msgr2=0x7ff78c15af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:08.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.002+0000 7ff792916640 1 -- 172.21.15.47:0/3805247207 shutdown_connections 2024-03-23T09:50:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.002+0000 7ff792916640 1 -- 172.21.15.47:0/3805247207 wait complete. 2024-03-23T09:50:08.013 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:49:41.158056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:49:42.210388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:50:08.014 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:50:08.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.117+0000 7f2f663c4640 1 Processor -- start 2024-03-23T09:50:08.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.117+0000 7f2f663c4640 1 -- start start 2024-03-23T09:50:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f663c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2f6015ca30 0x7f2f60160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f663c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f600cde40 0x7f2f600d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f663c4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f60161600 con 0x7f2f6015ce10 2024-03-23T09:50:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f663c4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f60161040 con 0x7f2f6015ca30 2024-03-23T09:50:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f663c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f60161320 con 0x7f2f600cde40 2024-03-23T09:50:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f5ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2f6015ca30 0x7f2f60160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f5f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2f6015ce10 0x7f2f600cd730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50716/0 (socket says 172.21.15.47:50716) 2024-03-23T09:50:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f5f7fe640 1 -- 172.21.15.47:0/1559421838 learned_addr learned my addr 172.21.15.47:0/1559421838 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f64961640 1 --2- 172.21.15.47:0/1559421838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f600cde40 0x7f2f600d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f5ffff640 1 -- 172.21.15.47:0/1559421838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f600cde40 msgr2=0x7f2f600d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f5ffff640 1 --2- 172.21.15.47:0/1559421838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f600cde40 0x7f2f600d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f5ffff640 1 -- 172.21.15.47:0/1559421838 >> v1:172.21.15.47:6789/0 conn(0x7f2f6015ce10 legacy=0x7f2f600cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f5effd640 1 -- 172.21.15.47:0/1559421838 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251710861 0 0) 0x7f2f60161600 con 0x7f2f6015ce10 2024-03-23T09:50:08.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f5ffff640 1 -- 172.21.15.47:0/1559421838 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f600d0dd0 con 0x7f2f6015ca30 2024-03-23T09:50:08.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.118+0000 7f2f64961640 1 --2- 172.21.15.47:0/1559421838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f600cde40 0x7f2f600d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.119+0000 7f2f5ffff640 1 --2- 172.21.15.47:0/1559421838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f6015ca30 0x7f2f60160740 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f2f540097a0 tx=0x7f2f5402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9863aca10e5cc01 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.119+0000 7f2f5effd640 1 -- 172.21.15.47:0/1559421838 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f54031830 con 0x7f2f6015ca30 2024-03-23T09:50:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.119+0000 7f2f5effd640 1 -- 172.21.15.47:0/1559421838 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f54031990 con 0x7f2f6015ca30 2024-03-23T09:50:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.119+0000 7f2f5effd640 1 -- 172.21.15.47:0/1559421838 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f54031d20 con 0x7f2f6015ca30 2024-03-23T09:50:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.120+0000 7f2f663c4640 1 -- 172.21.15.47:0/1559421838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f6015ca30 msgr2=0x7f2f60160740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.120+0000 7f2f663c4640 1 --2- 172.21.15.47:0/1559421838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f6015ca30 0x7f2f60160740 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f2f540097a0 tx=0x7f2f5402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.120+0000 7f2f663c4640 1 -- 172.21.15.47:0/1559421838 shutdown_connections 2024-03-23T09:50:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.120+0000 7f2f663c4640 1 --2- 172.21.15.47:0/1559421838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f600cde40 0x7f2f600d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.120+0000 7f2f663c4640 1 --2- 172.21.15.47:0/1559421838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f6015ca30 0x7f2f60160740 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.120+0000 7f2f663c4640 1 -- 172.21.15.47:0/1559421838 >> 172.21.15.47:0/1559421838 conn(0x7f2f6006b160 msgr2=0x7f2f6006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:08.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.120+0000 7f2f663c4640 1 -- 172.21.15.47:0/1559421838 shutdown_connections 2024-03-23T09:50:08.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.120+0000 7f2f663c4640 1 -- 172.21.15.47:0/1559421838 wait complete. 2024-03-23T09:50:08.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.121+0000 7f2f663c4640 1 Processor -- start 2024-03-23T09:50:08.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.121+0000 7f2f663c4640 1 -- start start 2024-03-23T09:50:08.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f663c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f600cde40 0x7f2f601f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f663c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f6015ca30 0x7f2f60150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f663c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f6015ce10 0x7f2f6015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f663c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f600c8090 con 0x7f2f6015ca30 2024-03-23T09:50:08.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f663c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f60151590 con 0x7f2f600cde40 2024-03-23T09:50:08.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f663c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f60151850 con 0x7f2f6015ce10 2024-03-23T09:50:08.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f5f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f6015ca30 0x7f2f60150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f5f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f6015ca30 0x7f2f60150ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53484/0 (socket says 172.21.15.47:53484) 2024-03-23T09:50:08.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f5f7fe640 1 -- 172.21.15.47:0/2713978924 learned_addr learned my addr 172.21.15.47:0/2713978924 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:08.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f5ffff640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f600cde40 0x7f2f601f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f64961640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f6015ce10 0x7f2f6015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f5f7fe640 1 -- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f6015ce10 msgr2=0x7f2f6015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f5f7fe640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f6015ce10 0x7f2f6015e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f5f7fe640 1 -- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f600cde40 msgr2=0x7f2f601f6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f5f7fe640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f600cde40 0x7f2f601f6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f5f7fe640 1 -- 172.21.15.47:0/2713978924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f54009400 con 0x7f2f6015ca30 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f5ffff640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f600cde40 0x7f2f601f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.122+0000 7f2f5f7fe640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f6015ca30 0x7f2f60150ce0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000c4d0 tx=0x7f2f5000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.123+0000 7f2f5cff9640 1 -- 172.21.15.47:0/2713978924 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f500074e0 con 0x7f2f6015ca30 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.123+0000 7f2f5cff9640 1 -- 172.21.15.47:0/2713978924 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f50007660 con 0x7f2f6015ca30 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.123+0000 7f2f5cff9640 1 -- 172.21.15.47:0/2713978924 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f50004a20 con 0x7f2f6015ca30 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.123+0000 7f2f663c4640 1 -- 172.21.15.47:0/2713978924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f60151b30 con 0x7f2f6015ca30 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.123+0000 7f2f663c4640 1 -- 172.21.15.47:0/2713978924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f601520f0 con 0x7f2f6015ca30 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.124+0000 7f2f663c4640 1 -- 172.21.15.47:0/2713978924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f60072360 con 0x7f2f6015ca30 2024-03-23T09:50:08.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.124+0000 7f2f5cff9640 1 -- 172.21.15.47:0/2713978924 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2f50004b80 con 0x7f2f6015ca30 2024-03-23T09:50:08.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.124+0000 7f2f5cff9640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2f4c06ac00 0x7f2f4c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.125+0000 7f2f5cff9640 1 -- 172.21.15.47:0/2713978924 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f50088560 con 0x7f2f6015ca30 2024-03-23T09:50:08.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.125+0000 7f2f5ffff640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2f4c06ac00 0x7f2f4c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.125+0000 7f2f5ffff640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2f4c06ac00 0x7f2f4c06d0c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2f54036000 tx=0x7f2f54008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.129+0000 7f2f5cff9640 1 -- 172.21.15.47:0/2713978924 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2f50053740 con 0x7f2f6015ca30 2024-03-23T09:50:08.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.311+0000 7f2f663c4640 1 -- 172.21.15.47:0/2713978924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f60152310 con 0x7f2f6015ca30 2024-03-23T09:50:08.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.311+0000 7f2f5cff9640 1 -- 172.21.15.47:0/2713978924 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f50056d60 con 0x7f2f6015ca30 2024-03-23T09:50:08.311 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:50:08.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.313+0000 7f2f663c4640 1 -- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2f4c06ac00 msgr2=0x7f2f4c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.313+0000 7f2f663c4640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2f4c06ac00 0x7f2f4c06d0c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2f54036000 tx=0x7f2f54008030 comp rx=0 tx=0).stop 2024-03-23T09:50:08.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.313+0000 7f2f663c4640 1 -- 172.21.15.47:0/2713978924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f6015ca30 msgr2=0x7f2f60150ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.313+0000 7f2f663c4640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f6015ca30 0x7f2f60150ce0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000c4d0 tx=0x7f2f5000c990 comp rx=0 tx=0).stop 2024-03-23T09:50:08.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.313+0000 7f2f663c4640 1 -- 172.21.15.47:0/2713978924 shutdown_connections 2024-03-23T09:50:08.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.313+0000 7f2f663c4640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2f4c06ac00 0x7f2f4c06d0c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.313+0000 7f2f663c4640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f6015ce10 0x7f2f6015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.313+0000 7f2f663c4640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f6015ca30 0x7f2f60150ce0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.313+0000 7f2f663c4640 1 --2- 172.21.15.47:0/2713978924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f600cde40 0x7f2f601f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.313+0000 7f2f663c4640 1 -- 172.21.15.47:0/2713978924 >> 172.21.15.47:0/2713978924 conn(0x7f2f6006b160 msgr2=0x7f2f60156b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:08.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.313+0000 7f2f663c4640 1 -- 172.21.15.47:0/2713978924 shutdown_connections 2024-03-23T09:50:08.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.313+0000 7f2f663c4640 1 -- 172.21.15.47:0/2713978924 wait complete. 2024-03-23T09:50:08.324 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:50:08.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.427+0000 7f448d1fb640 1 Processor -- start 2024-03-23T09:50:08.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.427+0000 7f448d1fb640 1 -- start start 2024-03-23T09:50:08.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f448d1fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f448815cee0 0x7f44880d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f448d1fb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44880d2f80 0x7f44880b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f448d1fb640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44880d1300 con 0x7f44880d23a0 2024-03-23T09:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f448d1fb640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44880d15e0 con 0x7f44880d2f80 2024-03-23T09:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f448d1fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44880d18c0 con 0x7f448815cee0 2024-03-23T09:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f448759e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44880d2f80 0x7f44880b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f448659c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f44880d23a0 0x7f44880d2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50720/0 (socket says 172.21.15.47:50720) 2024-03-23T09:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f448659c640 1 -- 172.21.15.47:0/2671268796 learned_addr learned my addr 172.21.15.47:0/2671268796 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f4486d9d640 1 --2- 172.21.15.47:0/2671268796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f448815cee0 0x7f44880d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f448759e640 1 -- 172.21.15.47:0/2671268796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f448815cee0 msgr2=0x7f44880d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f448759e640 1 --2- 172.21.15.47:0/2671268796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f448815cee0 0x7f44880d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f448759e640 1 -- 172.21.15.47:0/2671268796 >> v1:172.21.15.47:6789/0 conn(0x7f44880d23a0 legacy=0x7f44880d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f448759e640 1 -- 172.21.15.47:0/2671268796 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44880d1ba0 con 0x7f44880d2f80 2024-03-23T09:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.428+0000 7f4485d9b640 1 -- 172.21.15.47:0/2671268796 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934848657 0 0) 0x7f44880d1300 con 0x7f44880d23a0 2024-03-23T09:50:08.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.429+0000 7f448759e640 1 --2- 172.21.15.47:0/2671268796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44880d2f80 0x7f44880b5a80 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f44800097a0 tx=0x7f448002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7397f86e2e4c1c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:08.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.429+0000 7f4485d9b640 1 -- 172.21.15.47:0/2671268796 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480031830 con 0x7f44880d2f80 2024-03-23T09:50:08.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.429+0000 7f4485d9b640 1 -- 172.21.15.47:0/2671268796 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4480031990 con 0x7f44880d2f80 2024-03-23T09:50:08.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.429+0000 7f4485d9b640 1 -- 172.21.15.47:0/2671268796 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480031d20 con 0x7f44880d2f80 2024-03-23T09:50:08.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.430+0000 7f448d1fb640 1 -- 172.21.15.47:0/2671268796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44880d2f80 msgr2=0x7f44880b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.430+0000 7f448d1fb640 1 --2- 172.21.15.47:0/2671268796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44880d2f80 0x7f44880b5a80 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f44800097a0 tx=0x7f448002b8d0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.430+0000 7f448d1fb640 1 -- 172.21.15.47:0/2671268796 shutdown_connections 2024-03-23T09:50:08.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.430+0000 7f448d1fb640 1 --2- 172.21.15.47:0/2671268796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44880d2f80 0x7f44880b5a80 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.430+0000 7f448d1fb640 1 --2- 172.21.15.47:0/2671268796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f448815cee0 0x7f44880d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.430+0000 7f448d1fb640 1 -- 172.21.15.47:0/2671268796 >> 172.21.15.47:0/2671268796 conn(0x7f4488065df0 msgr2=0x7f4488066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:08.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.430+0000 7f448d1fb640 1 -- 172.21.15.47:0/2671268796 shutdown_connections 2024-03-23T09:50:08.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.430+0000 7f448d1fb640 1 -- 172.21.15.47:0/2671268796 wait complete. 2024-03-23T09:50:08.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.431+0000 7f448d1fb640 1 Processor -- start 2024-03-23T09:50:08.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.431+0000 7f448d1fb640 1 -- start start 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448d1fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44880d23a0 0x7f448815f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448d1fb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44880d2f80 0x7f448814d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448d1fb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448815cee0 0x7f448814dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448d1fb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448813db30 con 0x7f448815cee0 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448d1fb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44881409d0 con 0x7f44880d2f80 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448d1fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448813ddd0 con 0x7f44880d23a0 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448815cee0 0x7f448814dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448815cee0 0x7f448814dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53498/0 (socket says 172.21.15.47:53498) 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448759e640 1 -- 172.21.15.47:0/2082655034 learned_addr learned my addr 172.21.15.47:0/2082655034 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f4486d9d640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44880d23a0 0x7f448815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448659c640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44880d2f80 0x7f448814d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448759e640 1 -- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44880d23a0 msgr2=0x7f448815f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448759e640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44880d23a0 0x7f448815f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448759e640 1 -- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44880d2f80 msgr2=0x7f448814d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448759e640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44880d2f80 0x7f448814d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448759e640 1 -- 172.21.15.47:0/2082655034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4480009400 con 0x7f448815cee0 2024-03-23T09:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f4486d9d640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44880d23a0 0x7f448815f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.432+0000 7f448659c640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44880d2f80 0x7f448814d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.433+0000 7f448759e640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448815cee0 0x7f448814dd40 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f44800098c0 tx=0x7f44800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.433+0000 7f447f7fe640 1 -- 172.21.15.47:0/2082655034 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448003a630 con 0x7f448815cee0 2024-03-23T09:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.433+0000 7f447f7fe640 1 -- 172.21.15.47:0/2082655034 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448003a790 con 0x7f448815cee0 2024-03-23T09:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.433+0000 7f447f7fe640 1 -- 172.21.15.47:0/2082655034 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448003ab20 con 0x7f448815cee0 2024-03-23T09:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.433+0000 7f448d1fb640 1 -- 172.21.15.47:0/2082655034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448813e0b0 con 0x7f448815cee0 2024-03-23T09:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.433+0000 7f448d1fb640 1 -- 172.21.15.47:0/2082655034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448815fe60 con 0x7f448815cee0 2024-03-23T09:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.434+0000 7f448d1fb640 1 -- 172.21.15.47:0/2082655034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4488072360 con 0x7f448815cee0 2024-03-23T09:50:08.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.434+0000 7f447f7fe640 1 -- 172.21.15.47:0/2082655034 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f448003ac80 con 0x7f448815cee0 2024-03-23T09:50:08.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.434+0000 7f447f7fe640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f446806ac00 0x7f446806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.435+0000 7f447f7fe640 1 -- 172.21.15.47:0/2082655034 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44800ab0e0 con 0x7f448815cee0 2024-03-23T09:50:08.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.435+0000 7f4486d9d640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f446806ac00 0x7f446806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.435+0000 7f4486d9d640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f446806ac00 0x7f446806d0c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4478007440 tx=0x7f447800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:08.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.439+0000 7f447f7fe640 1 -- 172.21.15.47:0/2082655034 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4480076350 con 0x7f448815cee0 2024-03-23T09:50:08.586 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:50:08.588 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:50:08.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.618+0000 7f448d1fb640 1 -- 172.21.15.47:0/2082655034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f448813e5d0 con 0x7f448815cee0 2024-03-23T09:50:08.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.619+0000 7f447f7fe640 1 -- 172.21.15.47:0/2082655034 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+11 (secure 0 0 0) 0x7f448813e5d0 con 0x7f448815cee0 2024-03-23T09:50:08.619 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:50:08.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.620+0000 7f448d1fb640 1 -- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f446806ac00 msgr2=0x7f446806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.620+0000 7f448d1fb640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f446806ac00 0x7f446806d0c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4478007440 tx=0x7f447800b040 comp rx=0 tx=0).stop 2024-03-23T09:50:08.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.620+0000 7f448d1fb640 1 -- 172.21.15.47:0/2082655034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448815cee0 msgr2=0x7f448814dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.620+0000 7f448d1fb640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448815cee0 0x7f448814dd40 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f44800098c0 tx=0x7f44800046c0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.621+0000 7f448d1fb640 1 -- 172.21.15.47:0/2082655034 shutdown_connections 2024-03-23T09:50:08.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.621+0000 7f448d1fb640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f446806ac00 0x7f446806d0c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.621+0000 7f448d1fb640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448815cee0 0x7f448814dd40 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.621+0000 7f448d1fb640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44880d2f80 0x7f448814d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.621+0000 7f448d1fb640 1 --2- 172.21.15.47:0/2082655034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44880d23a0 0x7f448815f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.621+0000 7f448d1fb640 1 -- 172.21.15.47:0/2082655034 >> 172.21.15.47:0/2082655034 conn(0x7f4488065df0 msgr2=0x7f44880b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:08.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.621+0000 7f448d1fb640 1 -- 172.21.15.47:0/2082655034 shutdown_connections 2024-03-23T09:50:08.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.621+0000 7f448d1fb640 1 -- 172.21.15.47:0/2082655034 wait complete. 2024-03-23T09:50:08.631 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:50:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.777+0000 7ff5bba48640 1 Processor -- start 2024-03-23T09:50:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.777+0000 7ff5bba48640 1 -- start start 2024-03-23T09:50:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.778+0000 7ff5bba48640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff5b40ceb20 0x7ff5b41655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.778+0000 7ff5bba48640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b40cb270 con 0x7ff5b40cd7f0 2024-03-23T09:50:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.778+0000 7ff5bba48640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b40cb510 con 0x7ff5b415ab80 2024-03-23T09:50:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.778+0000 7ff5bba48640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b40cef90 con 0x7ff5b40ceb20 2024-03-23T09:50:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.778+0000 7ff5b8fe3640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff5b40cd7f0 0x7ff5b40ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50734/0 (socket says 172.21.15.47:50734) 2024-03-23T09:50:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.778+0000 7ff5b8fe3640 1 -- 172.21.15.47:0/583512188 learned_addr learned my addr 172.21.15.47:0/583512188 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b9fe5640 1 --2- 172.21.15.47:0/583512188 >> v2:172.21.15.175:3300/0 conn(0x7ff5b40ceb20 0x7ff5b41655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336480340 0 0) 0x7ff5b40cb270 con 0x7ff5b40cd7f0 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff59c003520 con 0x7ff5b40cd7f0 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2280496783 0 0) 0x7ff59c003520 con 0x7ff5b40cd7f0 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 >> v2:172.21.15.175:3300/0 conn(0x7ff5b40ceb20 msgr2=0x7ff5b41655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 --2- 172.21.15.47:0/583512188 >> v2:172.21.15.175:3300/0 conn(0x7ff5b40ceb20 0x7ff5b41655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 >> v1:172.21.15.175:6789/0 conn(0x7ff5b415ab80 legacy=0x7ff5b40cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b40c4240 con 0x7ff5b40cd7f0 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7ff5a8001670 con 0x7ff5b40cd7f0 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 >> v1:172.21.15.47:6789/0 conn(0x7ff5b40cd7f0 legacy=0x7ff5b40ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 --2- 172.21.15.47:0/583512188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff59c003970 0x7ff59c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 --2- 172.21.15.47:0/583512188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59c0047c0 0x7ff59c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 --2- 172.21.15.47:0/583512188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59c005180 0x7ff59c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c00c0f0 con 0x7ff59c0047c0 2024-03-23T09:50:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c00c260 con 0x7ff59c005180 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c00c420 con 0x7ff59c003970 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b8fe3640 1 --2- 172.21.15.47:0/583512188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59c0047c0 0x7ff59c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b9fe5640 1 --2- 172.21.15.47:0/583512188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59c005180 0x7ff59c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b97e4640 1 --2- 172.21.15.47:0/583512188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff59c003970 0x7ff59c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b8fe3640 1 -- 172.21.15.47:0/583512188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff59c003970 msgr2=0x7ff59c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b8fe3640 1 --2- 172.21.15.47:0/583512188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff59c003970 0x7ff59c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b8fe3640 1 -- 172.21.15.47:0/583512188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59c005180 msgr2=0x7ff59c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b8fe3640 1 --2- 172.21.15.47:0/583512188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59c005180 0x7ff59c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b9fe5640 1 -- 172.21.15.47:0/583512188 reap_dead start 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.779+0000 7ff5b8fe3640 1 -- 172.21.15.47:0/583512188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c003520 con 0x7ff59c0047c0 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.780+0000 7ff5b8fe3640 1 --2- 172.21.15.47:0/583512188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59c0047c0 0x7ff59c004c40 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800c240 tx=0x7ff59c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab9af887e8263ae server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.780+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8005ad0 con 0x7ff59c0047c0 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.780+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a8005c30 con 0x7ff59c0047c0 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.780+0000 7ff5b3fff640 1 -- 172.21.15.47:0/583512188 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a802ca90 con 0x7ff59c0047c0 2024-03-23T09:50:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.781+0000 7ff5bba48640 1 -- 172.21.15.47:0/583512188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59c0047c0 msgr2=0x7ff59c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.781+0000 7ff5bba48640 1 --2- 172.21.15.47:0/583512188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59c0047c0 0x7ff59c004c40 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800c240 tx=0x7ff59c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.781+0000 7ff5bba48640 1 -- 172.21.15.47:0/583512188 shutdown_connections 2024-03-23T09:50:08.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.781+0000 7ff5bba48640 1 --2- 172.21.15.47:0/583512188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff59c0047c0 0x7ff59c004c40 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.781+0000 7ff5bba48640 1 -- 172.21.15.47:0/583512188 >> 172.21.15.47:0/583512188 conn(0x7ff5b40d06f0 msgr2=0x7ff5b40d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:08.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.781+0000 7ff5bba48640 1 -- 172.21.15.47:0/583512188 shutdown_connections 2024-03-23T09:50:08.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.781+0000 7ff5bba48640 1 -- 172.21.15.47:0/583512188 wait complete. 2024-03-23T09:50:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.782+0000 7ff5bba48640 1 Processor -- start 2024-03-23T09:50:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.782+0000 7ff5bba48640 1 -- start start 2024-03-23T09:50:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5bba48640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59c0047c0 0x7ff5b40c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5bba48640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5b415d920 0x7ff5b40aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5bba48640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b414c480 0x7ff5b414c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5bba48640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b40c89b0 con 0x7ff5b414c480 2024-03-23T09:50:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5bba48640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b40af440 con 0x7ff59c0047c0 2024-03-23T09:50:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5bba48640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b415dda0 con 0x7ff5b415d920 2024-03-23T09:50:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5b9fe5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b414c480 0x7ff5b414c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5b9fe5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b414c480 0x7ff5b414c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53522/0 (socket says 172.21.15.47:53522) 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5b9fe5640 1 -- 172.21.15.47:0/3187701038 learned_addr learned my addr 172.21.15.47:0/3187701038 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5b8fe3640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5b415d920 0x7ff5b40aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5b97e4640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59c0047c0 0x7ff5b40c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5b9fe5640 1 -- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5b415d920 msgr2=0x7ff5b40aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5b9fe5640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5b415d920 0x7ff5b40aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5b9fe5640 1 -- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59c0047c0 msgr2=0x7ff5b40c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5b9fe5640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59c0047c0 0x7ff5b40c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5b9fe5640 1 -- 172.21.15.47:0/3187701038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b40c7da0 con 0x7ff5b414c480 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5b8fe3640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5b415d920 0x7ff5b40aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.783+0000 7ff5b97e4640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59c0047c0 0x7ff5b40c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.784+0000 7ff5b9fe5640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b414c480 0x7ff5b414c900 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40cb040 tx=0x7ff5a400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.784+0000 7ff5b1ffb640 1 -- 172.21.15.47:0/3187701038 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400e040 con 0x7ff5b414c480 2024-03-23T09:50:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.784+0000 7ff5b1ffb640 1 -- 172.21.15.47:0/3187701038 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a400f040 con 0x7ff5b414c480 2024-03-23T09:50:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.784+0000 7ff5bba48640 1 -- 172.21.15.47:0/3187701038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b40c8080 con 0x7ff5b414c480 2024-03-23T09:50:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.784+0000 7ff5b1ffb640 1 -- 172.21.15.47:0/3187701038 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4015430 con 0x7ff5b414c480 2024-03-23T09:50:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.784+0000 7ff5bba48640 1 -- 172.21.15.47:0/3187701038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b40c8640 con 0x7ff5b414c480 2024-03-23T09:50:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.785+0000 7ff5bba48640 1 -- 172.21.15.47:0/3187701038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b4066be0 con 0x7ff5b414c480 2024-03-23T09:50:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.785+0000 7ff5b1ffb640 1 -- 172.21.15.47:0/3187701038 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff5a4015590 con 0x7ff5b414c480 2024-03-23T09:50:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.785+0000 7ff5b1ffb640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5a006ac30 0x7ff5a006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.786+0000 7ff5b1ffb640 1 -- 172.21.15.47:0/3187701038 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5a4089bc0 con 0x7ff5b414c480 2024-03-23T09:50:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.786+0000 7ff5b97e4640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5a006ac30 0x7ff5a006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.786+0000 7ff5b97e4640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5a006ac30 0x7ff5a006d0f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac005580 tx=0x7ff5ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:08.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.790+0000 7ff5b1ffb640 1 -- 172.21.15.47:0/3187701038 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff5a4054da0 con 0x7ff5b414c480 2024-03-23T09:50:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.968+0000 7ff5bba48640 1 -- 172.21.15.47:0/3187701038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5b406e540 con 0x7ff5b414c480 2024-03-23T09:50:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.968+0000 7ff5b1ffb640 1 -- 172.21.15.47:0/3187701038 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5a40583c0 con 0x7ff5b414c480 2024-03-23T09:50:08.969 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:50:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.970+0000 7ff5bba48640 1 -- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5a006ac30 msgr2=0x7ff5a006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.970+0000 7ff5bba48640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5a006ac30 0x7ff5a006d0f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac005580 tx=0x7ff5ac009040 comp rx=0 tx=0).stop 2024-03-23T09:50:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.970+0000 7ff5bba48640 1 -- 172.21.15.47:0/3187701038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b414c480 msgr2=0x7ff5b414c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.970+0000 7ff5bba48640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b414c480 0x7ff5b414c900 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40cb040 tx=0x7ff5a400f580 comp rx=0 tx=0).stop 2024-03-23T09:50:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.970+0000 7ff5bba48640 1 -- 172.21.15.47:0/3187701038 shutdown_connections 2024-03-23T09:50:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.970+0000 7ff5bba48640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff5a006ac30 0x7ff5a006d0f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.970+0000 7ff5bba48640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b414c480 0x7ff5b414c900 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.970+0000 7ff5bba48640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5b415d920 0x7ff5b40aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.970+0000 7ff5bba48640 1 --2- 172.21.15.47:0/3187701038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59c0047c0 0x7ff5b40c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.970+0000 7ff5bba48640 1 -- 172.21.15.47:0/3187701038 >> 172.21.15.47:0/3187701038 conn(0x7ff5b40d06f0 msgr2=0x7ff5b40d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.970+0000 7ff5bba48640 1 -- 172.21.15.47:0/3187701038 shutdown_connections 2024-03-23T09:50:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:08.970+0000 7ff5bba48640 1 -- 172.21.15.47:0/3187701038 wait complete. 2024-03-23T09:50:08.981 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:50:09.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.082+0000 7efe53626640 1 Processor -- start 2024-03-23T09:50:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.082+0000 7efe53626640 1 -- start start 2024-03-23T09:50:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.083+0000 7efe53626640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe4c0cbdf0 0x7efe4c098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.083+0000 7efe53626640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efe4c099740 0x7efe4c098f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.083+0000 7efe53626640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c0ce8f0 con 0x7efe4c0b10f0 2024-03-23T09:50:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.083+0000 7efe53626640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4c0cc300 con 0x7efe4c099740 2024-03-23T09:50:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.083+0000 7efe53626640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0cc5c0 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.083+0000 7efe513c2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efe4c0b10f0 0x7efe4c0affe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50740/0 (socket says 172.21.15.47:50740) 2024-03-23T09:50:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.083+0000 7efe513c2640 1 -- 172.21.15.47:0/1686320619 learned_addr learned my addr 172.21.15.47:0/1686320619 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.083+0000 7efe51bc3640 1 --2- 172.21.15.47:0/1686320619 >> v2:172.21.15.47:3300/0 conn(0x7efe4c099740 0x7efe4c098f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.084+0000 7efe4bfff640 1 -- 172.21.15.47:0/1686320619 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449851015 0 0) 0x7efe4c0ce8f0 con 0x7efe4c0b10f0 2024-03-23T09:50:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.084+0000 7efe50bc1640 1 --2- 172.21.15.47:0/1686320619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe4c0cbdf0 0x7efe4c098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.084+0000 7efe4bfff640 1 -- 172.21.15.47:0/1686320619 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe34003520 con 0x7efe4c0b10f0 2024-03-23T09:50:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.084+0000 7efe51bc3640 1 -- 172.21.15.47:0/1686320619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe4c0cbdf0 msgr2=0x7efe4c098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.084+0000 7efe51bc3640 1 --2- 172.21.15.47:0/1686320619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe4c0cbdf0 0x7efe4c098a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.084+0000 7efe51bc3640 1 -- 172.21.15.47:0/1686320619 >> v1:172.21.15.47:6789/0 conn(0x7efe4c0b10f0 legacy=0x7efe4c0affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.084+0000 7efe51bc3640 1 -- 172.21.15.47:0/1686320619 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0ce2a0 con 0x7efe4c099740 2024-03-23T09:50:09.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.084+0000 7efe51bc3640 1 --2- 172.21.15.47:0/1686320619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe4c099740 0x7efe4c098f40 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0097a0 tx=0x7efe3c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36ab6eab8768a48 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:09.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.085+0000 7efe4bfff640 1 -- 172.21.15.47:0/1686320619 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c02e070 con 0x7efe4c099740 2024-03-23T09:50:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.085+0000 7efe4bfff640 1 -- 172.21.15.47:0/1686320619 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c0315a0 con 0x7efe4c099740 2024-03-23T09:50:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.085+0000 7efe4bfff640 1 -- 172.21.15.47:0/1686320619 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c031910 con 0x7efe4c099740 2024-03-23T09:50:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.085+0000 7efe53626640 1 -- 172.21.15.47:0/1686320619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe4c099740 msgr2=0x7efe4c098f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.085+0000 7efe53626640 1 --2- 172.21.15.47:0/1686320619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe4c099740 0x7efe4c098f40 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0097a0 tx=0x7efe3c02b840 comp rx=0 tx=0).stop 2024-03-23T09:50:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.085+0000 7efe53626640 1 -- 172.21.15.47:0/1686320619 shutdown_connections 2024-03-23T09:50:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.085+0000 7efe53626640 1 --2- 172.21.15.47:0/1686320619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe4c099740 0x7efe4c098f40 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.085+0000 7efe53626640 1 --2- 172.21.15.47:0/1686320619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe4c0cbdf0 0x7efe4c098a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.085+0000 7efe53626640 1 -- 172.21.15.47:0/1686320619 >> 172.21.15.47:0/1686320619 conn(0x7efe4c0b21c0 msgr2=0x7efe4c0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.085+0000 7efe53626640 1 -- 172.21.15.47:0/1686320619 shutdown_connections 2024-03-23T09:50:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.085+0000 7efe53626640 1 -- 172.21.15.47:0/1686320619 wait complete. 2024-03-23T09:50:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.086+0000 7efe53626640 1 Processor -- start 2024-03-23T09:50:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.086+0000 7efe53626640 1 -- start start 2024-03-23T09:50:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.087+0000 7efe53626640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe4c099740 0x7efe4c1e1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.087+0000 7efe53626640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe4c0b10f0 0x7efe4c1ddd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.087+0000 7efe53626640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe4c0cbdf0 0x7efe4c0aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.087+0000 7efe53626640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0c93f0 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.087+0000 7efe53626640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c1e15a0 con 0x7efe4c099740 2024-03-23T09:50:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.087+0000 7efe53626640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0c9600 con 0x7efe4c0b10f0 2024-03-23T09:50:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.087+0000 7efe51bc3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe4c0cbdf0 0x7efe4c0aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.087+0000 7efe51bc3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe4c0cbdf0 0x7efe4c0aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53536/0 (socket says 172.21.15.47:53536) 2024-03-23T09:50:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.087+0000 7efe51bc3640 1 -- 172.21.15.47:0/236203754 learned_addr learned my addr 172.21.15.47:0/236203754 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.088+0000 7efe50bc1640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe4c0b10f0 0x7efe4c1ddd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.088+0000 7efe51bc3640 1 -- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe4c0b10f0 msgr2=0x7efe4c1ddd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.088+0000 7efe51bc3640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe4c0b10f0 0x7efe4c1ddd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.088+0000 7efe51bc3640 1 -- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe4c099740 msgr2=0x7efe4c1e1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.088+0000 7efe513c2640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe4c099740 0x7efe4c1e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.088+0000 7efe51bc3640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe4c099740 0x7efe4c1e1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.088+0000 7efe51bc3640 1 -- 172.21.15.47:0/236203754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c009400 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.088+0000 7efe51bc3640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe4c0cbdf0 0x7efe4c0aef70 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0097a0 tx=0x7efe3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.088+0000 7efe49ffb640 1 -- 172.21.15.47:0/236203754 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c032630 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.088+0000 7efe49ffb640 1 -- 172.21.15.47:0/236203754 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c032790 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.088+0000 7efe49ffb640 1 -- 172.21.15.47:0/236203754 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c032b00 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.088+0000 7efe53626640 1 -- 172.21.15.47:0/236203754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4c0c98e0 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.088+0000 7efe53626640 1 -- 172.21.15.47:0/236203754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4c0c9ea0 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.089+0000 7efe53626640 1 -- 172.21.15.47:0/236203754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4c066be0 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.090+0000 7efe49ffb640 1 -- 172.21.15.47:0/236203754 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efe3c032c60 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.090+0000 7efe49ffb640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efe3806ac30 0x7efe3806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.090+0000 7efe49ffb640 1 -- 172.21.15.47:0/236203754 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe3c0ab270 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.090+0000 7efe513c2640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efe3806ac30 0x7efe3806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.091+0000 7efe513c2640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efe3806ac30 0x7efe3806d0f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7efe440012e0 tx=0x7efe44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:09.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.094+0000 7efe49ffb640 1 -- 172.21.15.47:0/236203754 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efe3c0763c0 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.285+0000 7efe53626640 1 -- 172.21.15.47:0/236203754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efe4c0b6650 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.286+0000 7efe49ffb640 1 -- 172.21.15.47:0/236203754 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) v1 ==== 80+0+0 (secure 0 0 0) 0x7efe3c0799e0 con 0x7efe4c0cbdf0 2024-03-23T09:50:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:50:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.287+0000 7efe53626640 1 -- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efe3806ac30 msgr2=0x7efe3806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.287+0000 7efe53626640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efe3806ac30 0x7efe3806d0f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7efe440012e0 tx=0x7efe44009040 comp rx=0 tx=0).stop 2024-03-23T09:50:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.287+0000 7efe53626640 1 -- 172.21.15.47:0/236203754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe4c0cbdf0 msgr2=0x7efe4c0aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.287+0000 7efe53626640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe4c0cbdf0 0x7efe4c0aef70 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0097a0 tx=0x7efe3c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.288+0000 7efe53626640 1 -- 172.21.15.47:0/236203754 shutdown_connections 2024-03-23T09:50:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.288+0000 7efe53626640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7efe3806ac30 0x7efe3806d0f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.288+0000 7efe53626640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe4c0cbdf0 0x7efe4c0aef70 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.288+0000 7efe53626640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe4c0b10f0 0x7efe4c1ddd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.288+0000 7efe53626640 1 --2- 172.21.15.47:0/236203754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe4c099740 0x7efe4c1e1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.288+0000 7efe53626640 1 -- 172.21.15.47:0/236203754 >> 172.21.15.47:0/236203754 conn(0x7efe4c0b21c0 msgr2=0x7efe4c1e1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.288+0000 7efe53626640 1 -- 172.21.15.47:0/236203754 shutdown_connections 2024-03-23T09:50:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.288+0000 7efe53626640 1 -- 172.21.15.47:0/236203754 wait complete. 2024-03-23T09:50:09.298 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:50:09.298 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:50:09.313 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:50:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.453+0000 7f39fd155640 1 Processor -- start 2024-03-23T09:50:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.453+0000 7f39fd155640 1 -- start start 2024-03-23T09:50:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.453+0000 7f39fd155640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39f815cd00 0x7f39f80d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.453+0000 7f39fd155640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f815d0e0 con 0x7f39f80cf920 2024-03-23T09:50:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.453+0000 7f39fd155640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f80d1540 con 0x7f39f80d1020 2024-03-23T09:50:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.453+0000 7f39fd155640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f80d1820 con 0x7f39f815cd00 2024-03-23T09:50:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.454+0000 7f39f6d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39f815cd00 0x7f39f80d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.454+0000 7f39f759e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f39f80cf920 0x7f39f80b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50756/0 (socket says 172.21.15.47:50756) 2024-03-23T09:50:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.454+0000 7f39f759e640 1 -- 172.21.15.47:0/3483931032 learned_addr learned my addr 172.21.15.47:0/3483931032 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.454+0000 7f39f6d9d640 1 -- 172.21.15.47:0/3483931032 >> v1:172.21.15.175:6789/0 conn(0x7f39f80d1020 legacy=0x7f39f80cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.454+0000 7f39f6d9d640 1 -- 172.21.15.47:0/3483931032 >> v1:172.21.15.47:6789/0 conn(0x7f39f80cf920 legacy=0x7f39f80b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.454+0000 7f39f6d9d640 1 -- 172.21.15.47:0/3483931032 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f80d1b00 con 0x7f39f815cd00 2024-03-23T09:50:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.455+0000 7f39f6d9d640 1 --2- 172.21.15.47:0/3483931032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39f815cd00 0x7f39f80d0ae0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f39e80097a0 tx=0x7f39e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c857c5a3d38529c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.455+0000 7f39f659c640 1 -- 172.21.15.47:0/3483931032 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8031830 con 0x7f39f815cd00 2024-03-23T09:50:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.455+0000 7f39f659c640 1 -- 172.21.15.47:0/3483931032 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e8031990 con 0x7f39f815cd00 2024-03-23T09:50:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.455+0000 7f39f659c640 1 -- 172.21.15.47:0/3483931032 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8031d20 con 0x7f39f815cd00 2024-03-23T09:50:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.455+0000 7f39fd155640 1 -- 172.21.15.47:0/3483931032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39f815cd00 msgr2=0x7f39f80d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.455+0000 7f39fd155640 1 --2- 172.21.15.47:0/3483931032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39f815cd00 0x7f39f80d0ae0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f39e80097a0 tx=0x7f39e802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.455+0000 7f39fd155640 1 -- 172.21.15.47:0/3483931032 shutdown_connections 2024-03-23T09:50:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.455+0000 7f39fd155640 1 --2- 172.21.15.47:0/3483931032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39f815cd00 0x7f39f80d0ae0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.455+0000 7f39fd155640 1 -- 172.21.15.47:0/3483931032 >> 172.21.15.47:0/3483931032 conn(0x7f39f806b160 msgr2=0x7f39f806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.455+0000 7f39fd155640 1 -- 172.21.15.47:0/3483931032 shutdown_connections 2024-03-23T09:50:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.455+0000 7f39fd155640 1 -- 172.21.15.47:0/3483931032 wait complete. 2024-03-23T09:50:09.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.457+0000 7f39fd155640 1 Processor -- start 2024-03-23T09:50:09.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.457+0000 7f39fd155640 1 -- start start 2024-03-23T09:50:09.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39fd155640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39f80cf920 0x7f39f80b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39fd155640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39f80d1020 0x7f39f8156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39fd155640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39f815cd00 0x7f39f80cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39fd155640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8156ea0 con 0x7f39f815cd00 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39fd155640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f80cbbe0 con 0x7f39f80d1020 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39fd155640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f80cbeb0 con 0x7f39f80cf920 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39f759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39f815cd00 0x7f39f80cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39f759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39f815cd00 0x7f39f80cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53560/0 (socket says 172.21.15.47:53560) 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39f759e640 1 -- 172.21.15.47:0/920033484 learned_addr learned my addr 172.21.15.47:0/920033484 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39ee59c640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39f80d1020 0x7f39f8156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39f6d9d640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39f80cf920 0x7f39f80b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39f759e640 1 -- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39f80cf920 msgr2=0x7f39f80b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39f759e640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39f80cf920 0x7f39f80b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39f759e640 1 -- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39f80d1020 msgr2=0x7f39f8156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39f759e640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39f80d1020 0x7f39f8156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39f759e640 1 -- 172.21.15.47:0/920033484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e8009400 con 0x7f39f815cd00 2024-03-23T09:50:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39f6d9d640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39f80cf920 0x7f39f80b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39ee59c640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39f80d1020 0x7f39f8156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39f759e640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39f815cd00 0x7f39f80cb6a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f39f000caf0 tx=0x7f39f000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39effff640 1 -- 172.21.15.47:0/920033484 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f000aca0 con 0x7f39f815cd00 2024-03-23T09:50:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39effff640 1 -- 172.21.15.47:0/920033484 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f000ae00 con 0x7f39f815cd00 2024-03-23T09:50:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39fd155640 1 -- 172.21.15.47:0/920033484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f80cc190 con 0x7f39f815cd00 2024-03-23T09:50:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39effff640 1 -- 172.21.15.47:0/920033484 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0012600 con 0x7f39f815cd00 2024-03-23T09:50:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.458+0000 7f39fd155640 1 -- 172.21.15.47:0/920033484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f80cc750 con 0x7f39f815cd00 2024-03-23T09:50:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.459+0000 7f39fd155640 1 -- 172.21.15.47:0/920033484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f8072360 con 0x7f39f815cd00 2024-03-23T09:50:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.460+0000 7f39effff640 1 -- 172.21.15.47:0/920033484 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f39f0007070 con 0x7f39f815cd00 2024-03-23T09:50:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.460+0000 7f39effff640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f39d806ac00 0x7f39d806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.460+0000 7f39effff640 1 -- 172.21.15.47:0/920033484 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39f0089260 con 0x7f39f815cd00 2024-03-23T09:50:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.460+0000 7f39f6d9d640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f39d806ac00 0x7f39d806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.461+0000 7f39f6d9d640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f39d806ac00 0x7f39d806d0c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f39e8036000 tx=0x7f39e803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:09.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.464+0000 7f39effff640 1 -- 172.21.15.47:0/920033484 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f39f0054470 con 0x7f39f815cd00 2024-03-23T09:50:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.645+0000 7f39fd155640 1 -- 172.21.15.47:0/920033484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39f8066be0 con 0x7f39f815cd00 2024-03-23T09:50:09.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.646+0000 7f39effff640 1 -- 172.21.15.47:0/920033484 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16043 (secure 0 0 0) 0x7f39f0057a90 con 0x7f39f815cd00 2024-03-23T09:50:09.647 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:09.647 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":101,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:49:44.249270+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:49:41.158056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-23T09:49:42.210388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:50:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.648+0000 7f39fd155640 1 -- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f39d806ac00 msgr2=0x7f39d806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.648+0000 7f39fd155640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f39d806ac00 0x7f39d806d0c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f39e8036000 tx=0x7f39e803c040 comp rx=0 tx=0).stop 2024-03-23T09:50:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.648+0000 7f39fd155640 1 -- 172.21.15.47:0/920033484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39f815cd00 msgr2=0x7f39f80cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.648+0000 7f39fd155640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39f815cd00 0x7f39f80cb6a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f39f000caf0 tx=0x7f39f000a060 comp rx=0 tx=0).stop 2024-03-23T09:50:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.648+0000 7f39fd155640 1 -- 172.21.15.47:0/920033484 shutdown_connections 2024-03-23T09:50:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.648+0000 7f39fd155640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f39d806ac00 0x7f39d806d0c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.648+0000 7f39fd155640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39f815cd00 0x7f39f80cb6a0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.648+0000 7f39fd155640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39f80d1020 0x7f39f8156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.648+0000 7f39fd155640 1 --2- 172.21.15.47:0/920033484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39f80cf920 0x7f39f80b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.648+0000 7f39fd155640 1 -- 172.21.15.47:0/920033484 >> 172.21.15.47:0/920033484 conn(0x7f39f806b160 msgr2=0x7f39f80cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.648+0000 7f39fd155640 1 -- 172.21.15.47:0/920033484 shutdown_connections 2024-03-23T09:50:09.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.648+0000 7f39fd155640 1 -- 172.21.15.47:0/920033484 wait complete. 2024-03-23T09:50:09.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:49:41.158056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:49:42.210388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:50:09.660 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:50:09.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.763+0000 7ffaeb2e0640 1 Processor -- start 2024-03-23T09:50:09.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.763+0000 7ffaeb2e0640 1 -- start start 2024-03-23T09:50:09.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffaeb2e0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffae415c990 0x7ffae4160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffaeb2e0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffae40cde10 0x7ffae40d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffaeb2e0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae4161630 con 0x7ffae40b8350 2024-03-23T09:50:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffaeb2e0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae40d1030 con 0x7ffae40cde10 2024-03-23T09:50:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffaeb2e0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae40cfe10 con 0x7ffae415c990 2024-03-23T09:50:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffae987d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffae40cde10 0x7ffae40d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffae887b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ffae40b8350 0x7ffae40cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50762/0 (socket says 172.21.15.47:50762) 2024-03-23T09:50:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffae887b640 1 -- 172.21.15.47:0/1948372605 learned_addr learned my addr 172.21.15.47:0/1948372605 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffae907c640 1 --2- 172.21.15.47:0/1948372605 >> v2:172.21.15.175:3300/0 conn(0x7ffae415c990 0x7ffae4160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffae987d640 1 -- 172.21.15.47:0/1948372605 >> v2:172.21.15.175:3300/0 conn(0x7ffae415c990 msgr2=0x7ffae4160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffae987d640 1 --2- 172.21.15.47:0/1948372605 >> v2:172.21.15.175:3300/0 conn(0x7ffae415c990 0x7ffae4160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffae987d640 1 -- 172.21.15.47:0/1948372605 >> v1:172.21.15.47:6789/0 conn(0x7ffae40b8350 legacy=0x7ffae40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffae3fff640 1 -- 172.21.15.47:0/1948372605 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700809706 0 0) 0x7ffae4161630 con 0x7ffae40b8350 2024-03-23T09:50:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.764+0000 7ffae987d640 1 -- 172.21.15.47:0/1948372605 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae40b5b60 con 0x7ffae40cde10 2024-03-23T09:50:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.765+0000 7ffae987d640 1 --2- 172.21.15.47:0/1948372605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae40cde10 0x7ffae40d14c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7ffae40b8b30 tx=0x7ffad402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab07ae7c5ec1258 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.765+0000 7ffae3fff640 1 -- 172.21.15.47:0/1948372605 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad402e070 con 0x7ffae40cde10 2024-03-23T09:50:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.765+0000 7ffae3fff640 1 -- 172.21.15.47:0/1948372605 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad40315a0 con 0x7ffae40cde10 2024-03-23T09:50:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.765+0000 7ffae3fff640 1 -- 172.21.15.47:0/1948372605 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad4031930 con 0x7ffae40cde10 2024-03-23T09:50:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.766+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/1948372605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae40cde10 msgr2=0x7ffae40d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.766+0000 7ffaeb2e0640 1 --2- 172.21.15.47:0/1948372605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae40cde10 0x7ffae40d14c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7ffae40b8b30 tx=0x7ffad402b840 comp rx=0 tx=0).stop 2024-03-23T09:50:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.766+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/1948372605 shutdown_connections 2024-03-23T09:50:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.766+0000 7ffaeb2e0640 1 --2- 172.21.15.47:0/1948372605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae40cde10 0x7ffae40d14c0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.766+0000 7ffaeb2e0640 1 --2- 172.21.15.47:0/1948372605 >> v2:172.21.15.175:3300/0 conn(0x7ffae415c990 0x7ffae4160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.766+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/1948372605 >> 172.21.15.47:0/1948372605 conn(0x7ffae40659d0 msgr2=0x7ffae4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.766+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/1948372605 shutdown_connections 2024-03-23T09:50:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.766+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/1948372605 wait complete. 2024-03-23T09:50:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.767+0000 7ffaeb2e0640 1 Processor -- start 2024-03-23T09:50:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.767+0000 7ffaeb2e0640 1 -- start start 2024-03-23T09:50:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffaeb2e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffae40b8350 0x7ffae4155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffaeb2e0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae40cde10 0x7ffae415f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffaeb2e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffae415c990 0x7ffae415eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffaeb2e0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae415e6d0 con 0x7ffae40cde10 2024-03-23T09:50:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffaeb2e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae415da90 con 0x7ffae415c990 2024-03-23T09:50:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffaeb2e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae415dd30 con 0x7ffae40b8350 2024-03-23T09:50:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffae887b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae40cde10 0x7ffae415f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffae887b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae40cde10 0x7ffae415f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53576/0 (socket says 172.21.15.47:53576) 2024-03-23T09:50:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffae887b640 1 -- 172.21.15.47:0/909178606 learned_addr learned my addr 172.21.15.47:0/909178606 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffae987d640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffae415c990 0x7ffae415eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffae907c640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffae40b8350 0x7ffae4155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffae887b640 1 -- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffae40b8350 msgr2=0x7ffae4155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffae887b640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffae40b8350 0x7ffae4155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffae887b640 1 -- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffae415c990 msgr2=0x7ffae415eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffae887b640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffae415c990 0x7ffae415eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffae887b640 1 -- 172.21.15.47:0/909178606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad4009400 con 0x7ffae40cde10 2024-03-23T09:50:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.768+0000 7ffae887b640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae40cde10 0x7ffae415f8f0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7ffad800c4d0 tx=0x7ffad800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.769+0000 7ffae1ffb640 1 -- 172.21.15.47:0/909178606 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad80074e0 con 0x7ffae40cde10 2024-03-23T09:50:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.769+0000 7ffae1ffb640 1 -- 172.21.15.47:0/909178606 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad8007660 con 0x7ffae40cde10 2024-03-23T09:50:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.769+0000 7ffae1ffb640 1 -- 172.21.15.47:0/909178606 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8004a20 con 0x7ffae40cde10 2024-03-23T09:50:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.769+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/909178606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae415e010 con 0x7ffae40cde10 2024-03-23T09:50:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.769+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/909178606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae415fe30 con 0x7ffae40cde10 2024-03-23T09:50:09.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.770+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/909178606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae4072360 con 0x7ffae40cde10 2024-03-23T09:50:09.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.770+0000 7ffae1ffb640 1 -- 172.21.15.47:0/909178606 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ffad8004b80 con 0x7ffae40cde10 2024-03-23T09:50:09.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.771+0000 7ffae1ffb640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffad006ac00 0x7ffad006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:09.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.771+0000 7ffae1ffb640 1 -- 172.21.15.47:0/909178606 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffad8088550 con 0x7ffae40cde10 2024-03-23T09:50:09.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.771+0000 7ffae907c640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffad006ac00 0x7ffad006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:09.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.771+0000 7ffae907c640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffad006ac00 0x7ffad006d0c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00f360 tx=0x7ffadc007710 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.775+0000 7ffae1ffb640 1 -- 172.21.15.47:0/909178606 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ffad8053740 con 0x7ffae40cde10 2024-03-23T09:50:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.955+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/909178606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffae415e810 con 0x7ffae40cde10 2024-03-23T09:50:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.956+0000 7ffae1ffb640 1 -- 172.21.15.47:0/909178606 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffad8018020 con 0x7ffae40cde10 2024-03-23T09:50:09.956 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:50:09.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.958+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffad006ac00 msgr2=0x7ffad006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.958+0000 7ffaeb2e0640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffad006ac00 0x7ffad006d0c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00f360 tx=0x7ffadc007710 comp rx=0 tx=0).stop 2024-03-23T09:50:09.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.958+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/909178606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae40cde10 msgr2=0x7ffae415f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:09.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.958+0000 7ffaeb2e0640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae40cde10 0x7ffae415f8f0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7ffad800c4d0 tx=0x7ffad800c990 comp rx=0 tx=0).stop 2024-03-23T09:50:09.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.958+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/909178606 shutdown_connections 2024-03-23T09:50:09.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.958+0000 7ffaeb2e0640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ffad006ac00 0x7ffad006d0c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.958+0000 7ffaeb2e0640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffae415c990 0x7ffae415eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.958+0000 7ffaeb2e0640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae40cde10 0x7ffae415f8f0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.958+0000 7ffaeb2e0640 1 --2- 172.21.15.47:0/909178606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffae40b8350 0x7ffae4155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:09.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.958+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/909178606 >> 172.21.15.47:0/909178606 conn(0x7ffae40659d0 msgr2=0x7ffae40d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:09.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.958+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/909178606 shutdown_connections 2024-03-23T09:50:09.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:09.958+0000 7ffaeb2e0640 1 -- 172.21.15.47:0/909178606 wait complete. 2024-03-23T09:50:09.969 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:50:10.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.073+0000 7f67e28c4640 1 Processor -- start 2024-03-23T09:50:10.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.073+0000 7f67e28c4640 1 -- start start 2024-03-23T09:50:10.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.073+0000 7f67e28c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67dc15ab90 0x7f67dc099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.073+0000 7f67e28c4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67dc099f80 con 0x7f67dc0cf340 2024-03-23T09:50:10.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.073+0000 7f67e28c4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67dc0cc250 con 0x7f67dc0cb160 2024-03-23T09:50:10.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.073+0000 7f67e28c4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67dc0cb640 con 0x7f67dc15ab90 2024-03-23T09:50:10.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.074+0000 7f67dbfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67dc15ab90 0x7f67dc099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.074+0000 7f67dbfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67dc15ab90 0x7f67dc099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53584/0 (socket says 172.21.15.47:53584) 2024-03-23T09:50:10.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.074+0000 7f67dbfff640 1 -- 172.21.15.47:0/3045593092 learned_addr learned my addr 172.21.15.47:0/3045593092 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:10.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.074+0000 7f67e0e61640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f67dc0cf340 0x7f67dc0afad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50778/0 (socket says 172.21.15.47:50778) 2024-03-23T09:50:10.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.074+0000 7f67dbfff640 1 -- 172.21.15.47:0/3045593092 >> v1:172.21.15.175:6789/0 conn(0x7f67dc0cb160 legacy=0x7f67dc0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.074+0000 7f67dbfff640 1 -- 172.21.15.47:0/3045593092 >> v1:172.21.15.47:6789/0 conn(0x7f67dc0cf340 legacy=0x7f67dc0afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.074+0000 7f67dbfff640 1 -- 172.21.15.47:0/3045593092 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc0cb920 con 0x7f67dc15ab90 2024-03-23T09:50:10.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.074+0000 7f67daffd640 1 -- 172.21.15.47:0/3045593092 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934241808 0 0) 0x7f67d4001490 con 0x7f67dc0cf340 2024-03-23T09:50:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.075+0000 7f67dbfff640 1 --2- 172.21.15.47:0/3045593092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67dc15ab90 0x7f67dc099a40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009870 tx=0x7f67cc02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706af7723b8c9882 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.075+0000 7f67daffd640 1 -- 172.21.15.47:0/3045593092 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc02f840 con 0x7f67dc15ab90 2024-03-23T09:50:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.075+0000 7f67daffd640 1 -- 172.21.15.47:0/3045593092 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67cc02f9a0 con 0x7f67dc15ab90 2024-03-23T09:50:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.075+0000 7f67daffd640 1 -- 172.21.15.47:0/3045593092 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc02fd30 con 0x7f67dc15ab90 2024-03-23T09:50:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.075+0000 7f67e28c4640 1 -- 172.21.15.47:0/3045593092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67dc15ab90 msgr2=0x7f67dc099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.075+0000 7f67e28c4640 1 --2- 172.21.15.47:0/3045593092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67dc15ab90 0x7f67dc099a40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009870 tx=0x7f67cc02ba20 comp rx=0 tx=0).stop 2024-03-23T09:50:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.075+0000 7f67e28c4640 1 -- 172.21.15.47:0/3045593092 shutdown_connections 2024-03-23T09:50:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.075+0000 7f67e28c4640 1 --2- 172.21.15.47:0/3045593092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67dc15ab90 0x7f67dc099a40 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.075+0000 7f67e28c4640 1 -- 172.21.15.47:0/3045593092 >> 172.21.15.47:0/3045593092 conn(0x7f67dc0b2140 msgr2=0x7f67dc0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.075+0000 7f67e28c4640 1 -- 172.21.15.47:0/3045593092 shutdown_connections 2024-03-23T09:50:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.075+0000 7f67e28c4640 1 -- 172.21.15.47:0/3045593092 wait complete. 2024-03-23T09:50:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.077+0000 7f67e28c4640 1 Processor -- start 2024-03-23T09:50:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.077+0000 7f67e28c4640 1 -- start start 2024-03-23T09:50:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.077+0000 7f67e28c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67dc0cb160 0x7f67dc150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67e28c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67dc0cf340 0x7f67dc14dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67e28c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67dc15ab90 0x7f67dc14ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67e28c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc150bb0 con 0x7f67dc0cf340 2024-03-23T09:50:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67e28c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc1501e0 con 0x7f67dc15ab90 2024-03-23T09:50:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67e28c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc14e100 con 0x7f67dc0cb160 2024-03-23T09:50:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67db7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67dc0cf340 0x7f67dc14dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67db7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67dc0cf340 0x7f67dc14dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53600/0 (socket says 172.21.15.47:53600) 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67db7fe640 1 -- 172.21.15.47:0/2671402948 learned_addr learned my addr 172.21.15.47:0/2671402948 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67e0e61640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67dc15ab90 0x7f67dc14ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67dbfff640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67dc0cb160 0x7f67dc150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67db7fe640 1 -- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67dc0cb160 msgr2=0x7f67dc150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67db7fe640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67dc0cb160 0x7f67dc150670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67db7fe640 1 -- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67dc15ab90 msgr2=0x7f67dc14ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67db7fe640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67dc15ab90 0x7f67dc14ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67db7fe640 1 -- 172.21.15.47:0/2671402948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc0094d0 con 0x7f67dc0cf340 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67db7fe640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67dc0cf340 0x7f67dc14dbc0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f67d000c400 tx=0x7f67d000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67d8ff9640 1 -- 172.21.15.47:0/2671402948 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d00052d0 con 0x7f67dc0cf340 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67d8ff9640 1 -- 172.21.15.47:0/2671402948 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d0005430 con 0x7f67dc0cf340 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67e28c4640 1 -- 172.21.15.47:0/2671402948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67dc14f5d0 con 0x7f67dc0cf340 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67d8ff9640 1 -- 172.21.15.47:0/2671402948 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0003830 con 0x7f67dc0cf340 2024-03-23T09:50:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.078+0000 7f67e28c4640 1 -- 172.21.15.47:0/2671402948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67dc14fb90 con 0x7f67dc0cf340 2024-03-23T09:50:10.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.079+0000 7f67e28c4640 1 -- 172.21.15.47:0/2671402948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67dc150390 con 0x7f67dc0cf340 2024-03-23T09:50:10.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.080+0000 7f67d8ff9640 1 -- 172.21.15.47:0/2671402948 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f67d0003990 con 0x7f67dc0cf340 2024-03-23T09:50:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.080+0000 7f67d8ff9640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f67c806ac00 0x7f67c806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.080+0000 7f67d8ff9640 1 -- 172.21.15.47:0/2671402948 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67d00888b0 con 0x7f67dc0cf340 2024-03-23T09:50:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.080+0000 7f67dbfff640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f67c806ac00 0x7f67c806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.081+0000 7f67dbfff640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f67c806ac00 0x7f67c806d0c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f67cc036000 tx=0x7f67cc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.084+0000 7f67d8ff9640 1 -- 172.21.15.47:0/2671402948 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f67dc150390 con 0x7f67dc0cf340 2024-03-23T09:50:10.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.265+0000 7f67e28c4640 1 -- 172.21.15.47:0/2671402948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67dc150350 con 0x7f67dc0cf340 2024-03-23T09:50:10.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.266+0000 7f67d8ff9640 1 -- 172.21.15.47:0/2671402948 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67d0017020 con 0x7f67dc0cf340 2024-03-23T09:50:10.266 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:50:10.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.268+0000 7f67e28c4640 1 -- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f67c806ac00 msgr2=0x7f67c806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.268+0000 7f67e28c4640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f67c806ac00 0x7f67c806d0c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f67cc036000 tx=0x7f67cc008030 comp rx=0 tx=0).stop 2024-03-23T09:50:10.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.268+0000 7f67e28c4640 1 -- 172.21.15.47:0/2671402948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67dc0cf340 msgr2=0x7f67dc14dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.268+0000 7f67e28c4640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67dc0cf340 0x7f67dc14dbc0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f67d000c400 tx=0x7f67d000c8c0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.268+0000 7f67e28c4640 1 -- 172.21.15.47:0/2671402948 shutdown_connections 2024-03-23T09:50:10.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.268+0000 7f67e28c4640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f67c806ac00 0x7f67c806d0c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.268+0000 7f67e28c4640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67dc15ab90 0x7f67dc14ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.268+0000 7f67e28c4640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67dc0cf340 0x7f67dc14dbc0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.268+0000 7f67e28c4640 1 --2- 172.21.15.47:0/2671402948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67dc0cb160 0x7f67dc150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.268+0000 7f67e28c4640 1 -- 172.21.15.47:0/2671402948 >> 172.21.15.47:0/2671402948 conn(0x7f67dc0b2140 msgr2=0x7f67dc0b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:10.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.268+0000 7f67e28c4640 1 -- 172.21.15.47:0/2671402948 shutdown_connections 2024-03-23T09:50:10.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.268+0000 7f67e28c4640 1 -- 172.21.15.47:0/2671402948 wait complete. 2024-03-23T09:50:10.279 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:50:10.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.382+0000 7f9b9ed38640 1 Processor -- start 2024-03-23T09:50:10.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.382+0000 7f9b9ed38640 1 -- start start 2024-03-23T09:50:10.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.383+0000 7f9b9ed38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b980b81f0 0x7f9b980d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.383+0000 7f9b9ed38640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b98160d60 con 0x7f9b980b1b20 2024-03-23T09:50:10.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.383+0000 7f9b9ed38640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b98160340 con 0x7f9b9815ff60 2024-03-23T09:50:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.383+0000 7f9b9ed38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980b8680 con 0x7f9b980b81f0 2024-03-23T09:50:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.383+0000 7f9b9d2d5640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9b980b1b20 0x7f9b980d0f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50780/0 (socket says 172.21.15.47:50780) 2024-03-23T09:50:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.383+0000 7f9b9d2d5640 1 -- 172.21.15.47:0/537398851 learned_addr learned my addr 172.21.15.47:0/537398851 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348777294 0 0) 0x7f9b98160d60 con 0x7f9b980b1b20 2024-03-23T09:50:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b80003520 con 0x7f9b980b1b20 2024-03-23T09:50:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b97fff640 1 --2- 172.21.15.47:0/537398851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b980b81f0 0x7f9b980d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3654059001 0 0) 0x7f9b80003520 con 0x7f9b980b1b20 2024-03-23T09:50:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b980b81f0 msgr2=0x7f9b980d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 --2- 172.21.15.47:0/537398851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b980b81f0 0x7f9b980d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 >> v1:172.21.15.175:6789/0 conn(0x7f9b9815ff60 legacy=0x7f9b980cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b98160710 con 0x7f9b980b1b20 2024-03-23T09:50:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f9b88001670 con 0x7f9b980b1b20 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 >> v1:172.21.15.47:6789/0 conn(0x7f9b980b1b20 legacy=0x7f9b980d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 --2- 172.21.15.47:0/537398851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b800039a0 0x7f9b80003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 --2- 172.21.15.47:0/537398851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b800047c0 0x7f9b80004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 --2- 172.21.15.47:0/537398851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b80005140 0x7f9b8000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980b8680 con 0x7f9b80005140 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8000bec0 con 0x7f9b800039a0 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8000c050 con 0x7f9b800047c0 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b9d2d5640 1 --2- 172.21.15.47:0/537398851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b80005140 0x7f9b8000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b9cad4640 1 --2- 172.21.15.47:0/537398851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b800039a0 0x7f9b80003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b97fff640 1 --2- 172.21.15.47:0/537398851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b800047c0 0x7f9b80004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b9d2d5640 1 -- 172.21.15.47:0/537398851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b800047c0 msgr2=0x7f9b80004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b9d2d5640 1 --2- 172.21.15.47:0/537398851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b800047c0 0x7f9b80004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b9d2d5640 1 -- 172.21.15.47:0/537398851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b800039a0 msgr2=0x7f9b80003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b9d2d5640 1 --2- 172.21.15.47:0/537398851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b800039a0 0x7f9b80003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b9d2d5640 1 -- 172.21.15.47:0/537398851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b80003520 con 0x7f9b80005140 2024-03-23T09:50:10.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.384+0000 7f9b97fff640 1 --2- 172.21.15.47:0/537398851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b800047c0 0x7f9b80004c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:10.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.385+0000 7f9b9d2d5640 1 --2- 172.21.15.47:0/537398851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b80005140 0x7f9b8000b980 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800c240 tx=0x7f9b800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883d54e4442d8d32 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:10.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.385+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88005ad0 con 0x7f9b80005140 2024-03-23T09:50:10.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.385+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b88005c30 con 0x7f9b80005140 2024-03-23T09:50:10.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.385+0000 7f9b977fe640 1 -- 172.21.15.47:0/537398851 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8802c8b0 con 0x7f9b80005140 2024-03-23T09:50:10.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.386+0000 7f9b9ed38640 1 -- 172.21.15.47:0/537398851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b80005140 msgr2=0x7f9b8000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.386+0000 7f9b9ed38640 1 --2- 172.21.15.47:0/537398851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b80005140 0x7f9b8000b980 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800c240 tx=0x7f9b800057f0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.386+0000 7f9b9d2d5640 1 -- 172.21.15.47:0/537398851 reap_dead start 2024-03-23T09:50:10.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.386+0000 7f9b9ed38640 1 -- 172.21.15.47:0/537398851 shutdown_connections 2024-03-23T09:50:10.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.386+0000 7f9b9ed38640 1 -- 172.21.15.47:0/537398851 >> 172.21.15.47:0/537398851 conn(0x7f9b98065df0 msgr2=0x7f9b98066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:10.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.386+0000 7f9b9ed38640 1 -- 172.21.15.47:0/537398851 shutdown_connections 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.386+0000 7f9b9ed38640 1 -- 172.21.15.47:0/537398851 wait complete. 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.387+0000 7f9b9ed38640 1 Processor -- start 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.387+0000 7f9b9ed38640 1 -- start start 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b9ed38640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b80005140 0x7f9b981548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b9ed38640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b9815ab30 0x7f9b981577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b9ed38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b980b6360 0x7f9b980b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b9ed38640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98157d10 con 0x7f9b9815ab30 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b9ed38640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980b6d20 con 0x7f9b80005140 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b9ed38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98150680 con 0x7f9b980b6360 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b97fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b9815ab30 0x7f9b981577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b97fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b9815ab30 0x7f9b981577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53610/0 (socket says 172.21.15.47:53610) 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b97fff640 1 -- 172.21.15.47:0/3485399943 learned_addr learned my addr 172.21.15.47:0/3485399943 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b9cad4640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b80005140 0x7f9b981548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b9d2d5640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b980b6360 0x7f9b980b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b97fff640 1 -- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b980b6360 msgr2=0x7f9b980b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b97fff640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b980b6360 0x7f9b980b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b97fff640 1 -- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b80005140 msgr2=0x7f9b981548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b97fff640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b80005140 0x7f9b981548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.388+0000 7f9b97fff640 1 -- 172.21.15.47:0/3485399943 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b980ca970 con 0x7f9b9815ab30 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.389+0000 7f9b97fff640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b9815ab30 0x7f9b981577d0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0116d0 tx=0x7f9b8c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.389+0000 7f9b957fa640 1 -- 172.21.15.47:0/3485399943 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00f9c0 con 0x7f9b9815ab30 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.389+0000 7f9b957fa640 1 -- 172.21.15.47:0/3485399943 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c00fb20 con 0x7f9b9815ab30 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.389+0000 7f9b9ed38640 1 -- 172.21.15.47:0/3485399943 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b981540b0 con 0x7f9b9815ab30 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.389+0000 7f9b957fa640 1 -- 172.21.15.47:0/3485399943 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00fed0 con 0x7f9b9815ab30 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.389+0000 7f9b9ed38640 1 -- 172.21.15.47:0/3485399943 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b981552b0 con 0x7f9b9815ab30 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.390+0000 7f9b9ed38640 1 -- 172.21.15.47:0/3485399943 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b980cab00 con 0x7f9b9815ab30 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.390+0000 7f9b957fa640 1 -- 172.21.15.47:0/3485399943 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9b8c015460 con 0x7f9b9815ab30 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.391+0000 7f9b957fa640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b8406ac30 0x7f9b8406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.391+0000 7f9b957fa640 1 -- 172.21.15.47:0/3485399943 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b8c089ad0 con 0x7f9b9815ab30 2024-03-23T09:50:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.391+0000 7f9b9cad4640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b8406ac30 0x7f9b8406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.391+0000 7f9b9cad4640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b8406ac30 0x7f9b8406d0f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9b90001e60 tx=0x7f9b90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:10.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.395+0000 7f9b957fa640 1 -- 172.21.15.47:0/3485399943 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9b8c054d60 con 0x7f9b9815ab30 2024-03-23T09:50:10.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.573+0000 7f9b9ed38640 1 -- 172.21.15.47:0/3485399943 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9b980672f0 con 0x7f9b9815ab30 2024-03-23T09:50:10.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.574+0000 7f9b957fa640 1 -- 172.21.15.47:0/3485399943 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9b8c0220a0 con 0x7f9b9815ab30 2024-03-23T09:50:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:50:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.575+0000 7f9b9ed38640 1 -- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b8406ac30 msgr2=0x7f9b8406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.575+0000 7f9b9ed38640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b8406ac30 0x7f9b8406d0f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9b90001e60 tx=0x7f9b90009040 comp rx=0 tx=0).stop 2024-03-23T09:50:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.575+0000 7f9b9ed38640 1 -- 172.21.15.47:0/3485399943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b9815ab30 msgr2=0x7f9b981577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.575+0000 7f9b9ed38640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b9815ab30 0x7f9b981577d0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0116d0 tx=0x7f9b8c011b90 comp rx=0 tx=0).stop 2024-03-23T09:50:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.575+0000 7f9b9ed38640 1 -- 172.21.15.47:0/3485399943 shutdown_connections 2024-03-23T09:50:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.575+0000 7f9b9ed38640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9b8406ac30 0x7f9b8406d0f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.575+0000 7f9b9ed38640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b980b6360 0x7f9b980b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.575+0000 7f9b9ed38640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b9815ab30 0x7f9b981577d0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.575+0000 7f9b9ed38640 1 --2- 172.21.15.47:0/3485399943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b80005140 0x7f9b981548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.575+0000 7f9b9ed38640 1 -- 172.21.15.47:0/3485399943 >> 172.21.15.47:0/3485399943 conn(0x7f9b98065df0 msgr2=0x7f9b980d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.576+0000 7f9b9ed38640 1 -- 172.21.15.47:0/3485399943 shutdown_connections 2024-03-23T09:50:10.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.576+0000 7f9b9ed38640 1 -- 172.21.15.47:0/3485399943 wait complete. 2024-03-23T09:50:10.586 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:50:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.688+0000 7f9e6cba9640 1 Processor -- start 2024-03-23T09:50:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.688+0000 7f9e6cba9640 1 -- start start 2024-03-23T09:50:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.689+0000 7f9e6cba9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9e6815c9d0 0x7f9e68071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.689+0000 7f9e6cba9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9e6806a3f0 0x7f9e68160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.689+0000 7f9e6cba9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e680721f0 con 0x7f9e6815cdb0 2024-03-23T09:50:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.689+0000 7f9e6cba9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6806eb80 con 0x7f9e6815c9d0 2024-03-23T09:50:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.689+0000 7f9e6cba9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6806edd0 con 0x7f9e6806a3f0 2024-03-23T09:50:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.689+0000 7f9e6659c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9e6815c9d0 0x7f9e68071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.689+0000 7f9e5ffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9e6815cdb0 0x7f9e68069ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50792/0 (socket says 172.21.15.47:50792) 2024-03-23T09:50:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.689+0000 7f9e5ffff640 1 -- 172.21.15.47:0/1419813432 learned_addr learned my addr 172.21.15.47:0/1419813432 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.689+0000 7f9e6659c640 1 -- 172.21.15.47:0/1419813432 >> v2:172.21.15.175:3300/0 conn(0x7f9e6806a3f0 msgr2=0x7f9e68160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.689+0000 7f9e6659c640 1 --2- 172.21.15.47:0/1419813432 >> v2:172.21.15.175:3300/0 conn(0x7f9e6806a3f0 0x7f9e68160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.689+0000 7f9e6659c640 1 -- 172.21.15.47:0/1419813432 >> v1:172.21.15.47:6789/0 conn(0x7f9e6815cdb0 legacy=0x7f9e68069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.689+0000 7f9e6659c640 1 -- 172.21.15.47:0/1419813432 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6806f0b0 con 0x7f9e6815c9d0 2024-03-23T09:50:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.689+0000 7f9e65d9b640 1 -- 172.21.15.47:0/1419813432 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007667702 0 0) 0x7f9e680721f0 con 0x7f9e6815cdb0 2024-03-23T09:50:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.690+0000 7f9e6659c640 1 --2- 172.21.15.47:0/1419813432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e6815c9d0 0x7f9e68071cb0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f9e6806e460 tx=0x7f9e5802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0905e762a17a71 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.690+0000 7f9e65d9b640 1 -- 172.21.15.47:0/1419813432 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58031830 con 0x7f9e6815c9d0 2024-03-23T09:50:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.690+0000 7f9e65d9b640 1 -- 172.21.15.47:0/1419813432 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e58031990 con 0x7f9e6815c9d0 2024-03-23T09:50:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.690+0000 7f9e65d9b640 1 -- 172.21.15.47:0/1419813432 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58031d20 con 0x7f9e6815c9d0 2024-03-23T09:50:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.690+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1419813432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e6815c9d0 msgr2=0x7f9e68071cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.690+0000 7f9e6cba9640 1 --2- 172.21.15.47:0/1419813432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e6815c9d0 0x7f9e68071cb0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f9e6806e460 tx=0x7f9e5802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.691+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1419813432 shutdown_connections 2024-03-23T09:50:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.691+0000 7f9e6cba9640 1 --2- 172.21.15.47:0/1419813432 >> v2:172.21.15.175:3300/0 conn(0x7f9e6806a3f0 0x7f9e68160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.691+0000 7f9e6cba9640 1 --2- 172.21.15.47:0/1419813432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e6815c9d0 0x7f9e68071cb0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.691+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1419813432 >> 172.21.15.47:0/1419813432 conn(0x7f9e680b13a0 msgr2=0x7f9e680b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.691+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1419813432 shutdown_connections 2024-03-23T09:50:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.691+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1419813432 wait complete. 2024-03-23T09:50:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.692+0000 7f9e6cba9640 1 Processor -- start 2024-03-23T09:50:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.692+0000 7f9e6cba9640 1 -- start start 2024-03-23T09:50:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e6cba9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e6806a3f0 0x7f9e681587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e6cba9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e6815c9d0 0x7f9e68158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e6cba9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e6815cdb0 0x7f9e6815e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e6cba9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6815f090 con 0x7f9e6815cdb0 2024-03-23T09:50:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e6cba9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6815f370 con 0x7f9e6815c9d0 2024-03-23T09:50:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e6cba9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6815f650 con 0x7f9e6806a3f0 2024-03-23T09:50:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e66d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e6815cdb0 0x7f9e6815e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e66d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e6815cdb0 0x7f9e6815e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53624/0 (socket says 172.21.15.47:53624) 2024-03-23T09:50:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e66d9d640 1 -- 172.21.15.47:0/1976386575 learned_addr learned my addr 172.21.15.47:0/1976386575 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e6659c640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e6806a3f0 0x7f9e681587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e5ffff640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e6815c9d0 0x7f9e68158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e66d9d640 1 -- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e6806a3f0 msgr2=0x7f9e681587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e66d9d640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e6806a3f0 0x7f9e681587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e66d9d640 1 -- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e6815c9d0 msgr2=0x7f9e68158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e66d9d640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e6815c9d0 0x7f9e68158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e66d9d640 1 -- 172.21.15.47:0/1976386575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e58009400 con 0x7f9e6815cdb0 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e5ffff640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e6815c9d0 0x7f9e68158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e66d9d640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e6815cdb0 0x7f9e6815e360 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000b9e0 tx=0x7f9e6000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e5effd640 1 -- 172.21.15.47:0/1976386575 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6000cb20 con 0x7f9e6815cdb0 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e5effd640 1 -- 172.21.15.47:0/1976386575 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6000cc80 con 0x7f9e6815cdb0 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.693+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1976386575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6815f930 con 0x7f9e6815cdb0 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.694+0000 7f9e5effd640 1 -- 172.21.15.47:0/1976386575 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60014680 con 0x7f9e6815cdb0 2024-03-23T09:50:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.694+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1976386575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6815ea80 con 0x7f9e6815cdb0 2024-03-23T09:50:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.695+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1976386575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e68066be0 con 0x7f9e6815cdb0 2024-03-23T09:50:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.695+0000 7f9e5effd640 1 -- 172.21.15.47:0/1976386575 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9e6001d3b0 con 0x7f9e6815cdb0 2024-03-23T09:50:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.695+0000 7f9e5effd640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e4806ac00 0x7f9e4806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.695+0000 7f9e5effd640 1 -- 172.21.15.47:0/1976386575 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e60089280 con 0x7f9e6815cdb0 2024-03-23T09:50:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.696+0000 7f9e6659c640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e4806ac00 0x7f9e4806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.696+0000 7f9e6659c640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e4806ac00 0x7f9e4806d0c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9e58036000 tx=0x7f9e5803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:10.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.700+0000 7f9e5effd640 1 -- 172.21.15.47:0/1976386575 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9e60054480 con 0x7f9e6815cdb0 2024-03-23T09:50:10.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.894+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1976386575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9e680af390 con 0x7f9e6815cdb0 2024-03-23T09:50:10.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.894+0000 7f9e5effd640 1 -- 172.21.15.47:0/1976386575 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9e60019080 con 0x7f9e6815cdb0 2024-03-23T09:50:10.895 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:50:10.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.897+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e4806ac00 msgr2=0x7f9e4806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.897+0000 7f9e6cba9640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e4806ac00 0x7f9e4806d0c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9e58036000 tx=0x7f9e5803d040 comp rx=0 tx=0).stop 2024-03-23T09:50:10.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.897+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1976386575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e6815cdb0 msgr2=0x7f9e6815e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.897+0000 7f9e6cba9640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e6815cdb0 0x7f9e6815e360 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000b9e0 tx=0x7f9e6000bea0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.897+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1976386575 shutdown_connections 2024-03-23T09:50:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.897+0000 7f9e6cba9640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9e4806ac00 0x7f9e4806d0c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.897+0000 7f9e6cba9640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e6815cdb0 0x7f9e6815e360 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.897+0000 7f9e6cba9640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e6815c9d0 0x7f9e68158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.897+0000 7f9e6cba9640 1 --2- 172.21.15.47:0/1976386575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e6806a3f0 0x7f9e681587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.897+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1976386575 >> 172.21.15.47:0/1976386575 conn(0x7f9e680b13a0 msgr2=0x7f9e680cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.897+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1976386575 shutdown_connections 2024-03-23T09:50:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:10.897+0000 7f9e6cba9640 1 -- 172.21.15.47:0/1976386575 wait complete. 2024-03-23T09:50:10.907 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:50:10.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:10.907 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:50:10.932 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[74927]: fuse finished with error 0 and tester_r 0 2024-03-23T09:50:10.935 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:50:17.036 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:50:17.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:17.037 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:50:17.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:50:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.206+0000 7f16b0f89640 1 Processor -- start 2024-03-23T09:50:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.206+0000 7f16b0f89640 1 -- start start 2024-03-23T09:50:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.207+0000 7f16b0f89640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16ac0ce560 0x7f16ac0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.207+0000 7f16b0f89640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f16ac06a330 0x7f16ac06a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.207+0000 7f16b0f89640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ac0cebf0 con 0x7f16ac160610 2024-03-23T09:50:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.207+0000 7f16b0f89640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ac1614d0 con 0x7f16ac06a330 2024-03-23T09:50:17.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.207+0000 7f16b0f89640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac069ea0 con 0x7f16ac0ce560 2024-03-23T09:50:17.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.207+0000 7f16aad9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f16ac06a330 0x7f16ac06a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.207+0000 7f16aa59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f16ac160610 0x7f16ac1609f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59134/0 (socket says 172.21.15.47:59134) 2024-03-23T09:50:17.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.207+0000 7f16aa59c640 1 -- 172.21.15.47:0/2004971942 learned_addr learned my addr 172.21.15.47:0/2004971942 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:17.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.207+0000 7f16a3fff640 1 --2- 172.21.15.47:0/2004971942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16ac0ce560 0x7f16ac0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.207+0000 7f16aad9d640 1 -- 172.21.15.47:0/2004971942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16ac0ce560 msgr2=0x7f16ac0ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.207+0000 7f16aad9d640 1 --2- 172.21.15.47:0/2004971942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16ac0ce560 0x7f16ac0ae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.207+0000 7f16aad9d640 1 -- 172.21.15.47:0/2004971942 >> v1:172.21.15.47:6789/0 conn(0x7f16ac160610 legacy=0x7f16ac1609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.207+0000 7f16aad9d640 1 -- 172.21.15.47:0/2004971942 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac0ce9c0 con 0x7f16ac06a330 2024-03-23T09:50:17.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.208+0000 7f16aad9d640 1 --2- 172.21.15.47:0/2004971942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16ac06a330 0x7f16ac06a7c0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009d90 tx=0x7f16a402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181e94d91f026c1e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.208+0000 7f16a9d9b640 1 -- 172.21.15.47:0/2004971942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a402cb30 con 0x7f16ac06a330 2024-03-23T09:50:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.208+0000 7f16a9d9b640 1 -- 172.21.15.47:0/2004971942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a402cc90 con 0x7f16ac06a330 2024-03-23T09:50:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.208+0000 7f16a9d9b640 1 -- 172.21.15.47:0/2004971942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4034440 con 0x7f16ac06a330 2024-03-23T09:50:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.209+0000 7f16b0f89640 1 -- 172.21.15.47:0/2004971942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16ac06a330 msgr2=0x7f16ac06a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.209+0000 7f16b0f89640 1 --2- 172.21.15.47:0/2004971942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16ac06a330 0x7f16ac06a7c0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009d90 tx=0x7f16a402bd80 comp rx=0 tx=0).stop 2024-03-23T09:50:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.209+0000 7f16b0f89640 1 -- 172.21.15.47:0/2004971942 shutdown_connections 2024-03-23T09:50:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.209+0000 7f16b0f89640 1 --2- 172.21.15.47:0/2004971942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16ac06a330 0x7f16ac06a7c0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.209+0000 7f16b0f89640 1 --2- 172.21.15.47:0/2004971942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16ac0ce560 0x7f16ac0ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.209+0000 7f16b0f89640 1 -- 172.21.15.47:0/2004971942 >> 172.21.15.47:0/2004971942 conn(0x7f16ac0ced80 msgr2=0x7f16ac0cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.209+0000 7f16b0f89640 1 -- 172.21.15.47:0/2004971942 shutdown_connections 2024-03-23T09:50:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.209+0000 7f16b0f89640 1 -- 172.21.15.47:0/2004971942 wait complete. 2024-03-23T09:50:17.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.210+0000 7f16b0f89640 1 Processor -- start 2024-03-23T09:50:17.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.210+0000 7f16b0f89640 1 -- start start 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16b0f89640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16ac06a330 0x7f16ac0b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16b0f89640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ac0ce560 0x7f16ac143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16b0f89640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16ac160610 0x7f16ac143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16b0f89640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac143160 con 0x7f16ac160610 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16b0f89640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac142cd0 con 0x7f16ac0ce560 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16b0f89640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac0b3970 con 0x7f16ac06a330 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16aad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16ac160610 0x7f16ac143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16aad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16ac160610 0x7f16ac143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37002/0 (socket says 172.21.15.47:37002) 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16aad9d640 1 -- 172.21.15.47:0/3588140779 learned_addr learned my addr 172.21.15.47:0/3588140779 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16a3fff640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ac0ce560 0x7f16ac143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16aa59c640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16ac06a330 0x7f16ac0b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16aad9d640 1 -- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16ac06a330 msgr2=0x7f16ac0b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16aad9d640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16ac06a330 0x7f16ac0b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16aad9d640 1 -- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ac0ce560 msgr2=0x7f16ac143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16aad9d640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ac0ce560 0x7f16ac143320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16aad9d640 1 -- 172.21.15.47:0/3588140779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a4009a70 con 0x7f16ac160610 2024-03-23T09:50:17.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16a3fff640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ac0ce560 0x7f16ac143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:17.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16aa59c640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16ac06a330 0x7f16ac0b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:17.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.211+0000 7f16aad9d640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16ac160610 0x7f16ac143860 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f16a402c390 tx=0x7f16a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.212+0000 7f16a2ffd640 1 -- 172.21.15.47:0/3588140779 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4040450 con 0x7f16ac160610 2024-03-23T09:50:17.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.212+0000 7f16a2ffd640 1 -- 172.21.15.47:0/3588140779 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a40405b0 con 0x7f16ac160610 2024-03-23T09:50:17.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.212+0000 7f16b0f89640 1 -- 172.21.15.47:0/3588140779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ac1420c0 con 0x7f16ac160610 2024-03-23T09:50:17.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.212+0000 7f16a2ffd640 1 -- 172.21.15.47:0/3588140779 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4040960 con 0x7f16ac160610 2024-03-23T09:50:17.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.212+0000 7f16b0f89640 1 -- 172.21.15.47:0/3588140779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ac142680 con 0x7f16ac160610 2024-03-23T09:50:17.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.213+0000 7f16b0f89640 1 -- 172.21.15.47:0/3588140779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ac066be0 con 0x7f16ac160610 2024-03-23T09:50:17.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.213+0000 7f16a2ffd640 1 -- 172.21.15.47:0/3588140779 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f16a4040cd0 con 0x7f16ac160610 2024-03-23T09:50:17.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.213+0000 7f16a2ffd640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f168c06ac00 0x7f168c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.213+0000 7f16a2ffd640 1 -- 172.21.15.47:0/3588140779 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16a40ac360 con 0x7f16ac160610 2024-03-23T09:50:17.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.214+0000 7f16aa59c640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f168c06ac00 0x7f168c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.214+0000 7f16aa59c640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f168c06ac00 0x7f168c06d0c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f169c005880 tx=0x7f169c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.218+0000 7f16a2ffd640 1 -- 172.21.15.47:0/3588140779 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f16a40775c0 con 0x7f16ac160610 2024-03-23T09:50:17.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.436+0000 7f16b0f89640 1 -- 172.21.15.47:0/3588140779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16ac142e10 con 0x7f16ac160610 2024-03-23T09:50:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.437+0000 7f16a2ffd640 1 -- 172.21.15.47:0/3588140779 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f16a407abe0 con 0x7f16ac160610 2024-03-23T09:50:17.437 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:17.437 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/249416109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":249416109},{"type":"v1","addr":"172.21.15.175:6837","nonce":249416109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3603846321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3603846321},{"type":"v1","addr":"172.21.15.175:6835","nonce":3603846321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1921094794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1921094794},{"type":"v1","addr":"172.21.15.47:6834","nonce":1921094794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:49:43.247119+0000","modified":"2024-03-23T09:49:45.258459+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8484":{"gid":8484,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/2388298413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2388298413},{"type":"v1","addr":"172.21.15.47:6835","nonce":2388298413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-23T09:50:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 65 2024-03-23T09:50:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.439+0000 7f16b0f89640 1 -- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f168c06ac00 msgr2=0x7f168c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.439+0000 7f16b0f89640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f168c06ac00 0x7f168c06d0c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f169c005880 tx=0x7f169c009040 comp rx=0 tx=0).stop 2024-03-23T09:50:17.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.439+0000 7f16b0f89640 1 -- 172.21.15.47:0/3588140779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16ac160610 msgr2=0x7f16ac143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.439+0000 7f16b0f89640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16ac160610 0x7f16ac143860 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f16a402c390 tx=0x7f16a4004a40 comp rx=0 tx=0).stop 2024-03-23T09:50:17.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.439+0000 7f16b0f89640 1 -- 172.21.15.47:0/3588140779 shutdown_connections 2024-03-23T09:50:17.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.439+0000 7f16b0f89640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f168c06ac00 0x7f168c06d0c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.439+0000 7f16b0f89640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16ac160610 0x7f16ac143860 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.439+0000 7f16b0f89640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ac0ce560 0x7f16ac143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.439+0000 7f16b0f89640 1 --2- 172.21.15.47:0/3588140779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16ac06a330 0x7f16ac0b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.439+0000 7f16b0f89640 1 -- 172.21.15.47:0/3588140779 >> 172.21.15.47:0/3588140779 conn(0x7f16ac0ced80 msgr2=0x7f16ac1599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.439+0000 7f16b0f89640 1 -- 172.21.15.47:0/3588140779 shutdown_connections 2024-03-23T09:50:17.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.439+0000 7f16b0f89640 1 -- 172.21.15.47:0/3588140779 wait complete. 2024-03-23T09:50:17.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-03-23T09:50:17.572 INFO:teuthology.orchestra.run.smithi047.stdout:{"message":"","return_code":0} 2024-03-23T09:50:17.572 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-23T09:50:17.572 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:50:17.573 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:50:17.587 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:50:17.587 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-03-23T09:50:17.587 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:50:17.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:17.588 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:50:17.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:50:17.643 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-23T09:50:17.644 DEBUG:teuthology.orchestra.run.smithi047:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-23T09:50:17.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:17.686 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-03-23T09:50:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.791+0000 7fef1f851640 1 Processor -- start 2024-03-23T09:50:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.792+0000 7fef1f851640 1 -- start start 2024-03-23T09:50:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.792+0000 7fef1f851640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fef1815ad10 0x7fef180cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.792+0000 7fef1f851640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef180ca340 0x7fef180ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.792+0000 7fef1f851640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef180cdeb0 con 0x7fef180b0a80 2024-03-23T09:50:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef1f851640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef180b0f80 con 0x7fef1815ad10 2024-03-23T09:50:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef1f851640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef180aebd0 con 0x7fef180ca340 2024-03-23T09:50:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef1cdec640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fef180b0a80 0x7fef180cc2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59138/0 (socket says 172.21.15.47:59138) 2024-03-23T09:50:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef1cdec640 1 -- 172.21.15.47:0/699931581 learned_addr learned my addr 172.21.15.47:0/699931581 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/699931581 >> v2:172.21.15.175:3300/0 conn(0x7fef1815ad10 0x7fef180cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef1ddee640 1 --2- 172.21.15.47:0/699931581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef180ca340 0x7fef180ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355502629 0 0) 0x7fef180cdeb0 con 0x7fef180b0a80 2024-03-23T09:50:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef04003520 con 0x7fef180b0a80 2024-03-23T09:50:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2971974853 0 0) 0x7fef04003520 con 0x7fef180b0a80 2024-03-23T09:50:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef180ca340 msgr2=0x7fef180ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 --2- 172.21.15.47:0/699931581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef180ca340 0x7fef180ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 >> v2:172.21.15.175:3300/0 conn(0x7fef1815ad10 msgr2=0x7fef180cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 --2- 172.21.15.47:0/699931581 >> v2:172.21.15.175:3300/0 conn(0x7fef1815ad10 0x7fef180cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef180b1bb0 con 0x7fef180b0a80 2024-03-23T09:50:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fef08001670 con 0x7fef180b0a80 2024-03-23T09:50:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/699931581 >> v2:172.21.15.175:3300/0 conn(0x7fef1815ad10 0x7fef180cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:50:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 >> v1:172.21.15.47:6789/0 conn(0x7fef180b0a80 legacy=0x7fef180cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 --2- 172.21.15.47:0/699931581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef04003970 0x7fef04003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 --2- 172.21.15.47:0/699931581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef180ca340 0x7fef040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 --2- 172.21.15.47:0/699931581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef04004d00 0x7fef04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef180b0f80 con 0x7fef04004d00 2024-03-23T09:50:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef180aebd0 con 0x7fef04003970 2024-03-23T09:50:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.793+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0400b980 con 0x7fef180ca340 2024-03-23T09:50:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.794+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/699931581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef04003970 0x7fef04003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.794+0000 7fef1ddee640 1 --2- 172.21.15.47:0/699931581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef04004d00 0x7fef04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.794+0000 7fef1cdec640 1 --2- 172.21.15.47:0/699931581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef180ca340 0x7fef040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.794+0000 7fef1d5ed640 1 -- 172.21.15.47:0/699931581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef180ca340 msgr2=0x7fef040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.794+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/699931581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef180ca340 0x7fef040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.794+0000 7fef1d5ed640 1 -- 172.21.15.47:0/699931581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef04004d00 msgr2=0x7fef04005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.794+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/699931581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef04004d00 0x7fef04005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.794+0000 7fef1d5ed640 1 -- 172.21.15.47:0/699931581 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef04003520 con 0x7fef04003970 2024-03-23T09:50:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.794+0000 7fef1ddee640 1 --2- 172.21.15.47:0/699931581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef04004d00 0x7fef04005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:50:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.795+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/699931581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef04003970 0x7fef04003d90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fef1400e350 tx=0x7fef040057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cef05670cdb93b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.795+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1402ce00 con 0x7fef04003970 2024-03-23T09:50:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.795+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1400f040 con 0x7fef04003970 2024-03-23T09:50:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.795+0000 7fef0ffff640 1 -- 172.21.15.47:0/699931581 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14034640 con 0x7fef04003970 2024-03-23T09:50:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.795+0000 7fef1f851640 1 -- 172.21.15.47:0/699931581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef04003970 msgr2=0x7fef04003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.795+0000 7fef1f851640 1 --2- 172.21.15.47:0/699931581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef04003970 0x7fef04003d90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fef1400e350 tx=0x7fef040057f0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.795+0000 7fef1ddee640 1 -- 172.21.15.47:0/699931581 reap_dead start 2024-03-23T09:50:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.796+0000 7fef1f851640 1 -- 172.21.15.47:0/699931581 shutdown_connections 2024-03-23T09:50:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.796+0000 7fef1f851640 1 -- 172.21.15.47:0/699931581 >> 172.21.15.47:0/699931581 conn(0x7fef18098a00 msgr2=0x7fef180b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.796+0000 7fef1f851640 1 -- 172.21.15.47:0/699931581 shutdown_connections 2024-03-23T09:50:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.796+0000 7fef1f851640 1 -- 172.21.15.47:0/699931581 wait complete. 2024-03-23T09:50:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.797+0000 7fef1f851640 1 Processor -- start 2024-03-23T09:50:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.797+0000 7fef1f851640 1 -- start start 2024-03-23T09:50:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1f851640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef181e4480 0x7fef180c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1f851640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef181f2bf0 0x7fef180c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1f851640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef181e4860 0x7fef18156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1f851640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef180c5c80 con 0x7fef181e4480 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1f851640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef181f30c0 con 0x7fef181e4860 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1f851640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef181f33a0 con 0x7fef181f2bf0 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1d5ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef181e4480 0x7fef180c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1d5ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef181e4480 0x7fef180c5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37024/0 (socket says 172.21.15.47:37024) 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1d5ed640 1 -- 172.21.15.47:0/2046798480 learned_addr learned my addr 172.21.15.47:0/2046798480 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1ddee640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef181e4860 0x7fef18156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1cdec640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef181f2bf0 0x7fef180c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1d5ed640 1 -- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef181f2bf0 msgr2=0x7fef180c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef181f2bf0 0x7fef180c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1d5ed640 1 -- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef181e4860 msgr2=0x7fef18156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef181e4860 0x7fef18156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1d5ed640 1 -- 172.21.15.47:0/2046798480 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef181f3680 con 0x7fef181e4480 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1ddee640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef181e4860 0x7fef18156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef181e4480 0x7fef180c5200 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fef14037040 tx=0x7fef14009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef0dffb640 1 -- 172.21.15.47:0/2046798480 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1403e850 con 0x7fef181e4480 2024-03-23T09:50:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef0dffb640 1 -- 172.21.15.47:0/2046798480 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1403e9b0 con 0x7fef181e4480 2024-03-23T09:50:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef1f851640 1 -- 172.21.15.47:0/2046798480 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef181f3960 con 0x7fef181e4480 2024-03-23T09:50:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.798+0000 7fef0dffb640 1 -- 172.21.15.47:0/2046798480 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1403ed60 con 0x7fef181e4480 2024-03-23T09:50:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.799+0000 7fef1f851640 1 -- 172.21.15.47:0/2046798480 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef180c93c0 con 0x7fef181e4480 2024-03-23T09:50:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.800+0000 7fef0dffb640 1 -- 172.21.15.47:0/2046798480 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fef1403d3a0 con 0x7fef181e4480 2024-03-23T09:50:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.801+0000 7fef0dffb640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feefc06ac30 0x7feefc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.801+0000 7fef0dffb640 1 -- 172.21.15.47:0/2046798480 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef140abae0 con 0x7fef181e4480 2024-03-23T09:50:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.801+0000 7fef1cdec640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feefc06ac30 0x7feefc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.801+0000 7fef223c6740 1 Processor -- start 2024-03-23T09:50:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.801+0000 7fef223c6740 1 -- start start 2024-03-23T09:50:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.802+0000 7fef1cdec640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feefc06ac30 0x7feefc06d0f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fef181f2ff0 tx=0x7fef08009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.802+0000 7fef223c6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5589669be2f0 0x5589669be6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.802+0000 7fef223c6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669c2340 0x5589669c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.802+0000 7fef223c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669ee500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.802+0000 7fef223c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55896692f940 con 0x5589669ee080 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.802+0000 7fef223c6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55896692d0d0 con 0x5589669c2340 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.802+0000 7fef223c6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589669289c0 con 0x5589669be2f0 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.802+0000 7fef1ddee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669ee500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.802+0000 7fef1ddee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669ee500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37026/0 (socket says 172.21.15.47:37026) 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.802+0000 7fef1ddee640 1 -- 172.21.15.47:0/4204727979 learned_addr learned my addr 172.21.15.47:0/4204727979 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.802+0000 7fef1cdec640 1 --2- 172.21.15.47:0/4204727979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669c2340 0x5589669c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.802+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/4204727979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5589669be2f0 0x5589669be6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.803+0000 7fef1ddee640 1 -- 172.21.15.47:0/4204727979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5589669be2f0 msgr2=0x5589669be6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.803+0000 7fef1ddee640 1 --2- 172.21.15.47:0/4204727979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5589669be2f0 0x5589669be6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.803+0000 7fef1ddee640 1 -- 172.21.15.47:0/4204727979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669c2340 msgr2=0x5589669c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.803+0000 7fef1ddee640 1 --2- 172.21.15.47:0/4204727979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669c2340 0x5589669c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.803+0000 7fef1ddee640 1 -- 172.21.15.47:0/4204727979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589668f48a0 con 0x5589669ee080 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.803+0000 7fef1ddee640 1 --2- 172.21.15.47:0/4204727979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669ee500 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fef1806ccd0 tx=0x7fef10009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7106daaf3088bb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.803+0000 7fef02ffd640 1 -- 172.21.15.47:0/4204727979 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1001d070 con 0x5589669ee080 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.803+0000 7fef02ffd640 1 -- 172.21.15.47:0/4204727979 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef10014c00 con 0x5589669ee080 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.803+0000 7fef02ffd640 1 -- 172.21.15.47:0/4204727979 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10014d70 con 0x5589669ee080 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.804+0000 7fef223c6740 1 -- 172.21.15.47:0/4204727979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 msgr2=0x5589669ee500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.804+0000 7fef223c6740 1 --2- 172.21.15.47:0/4204727979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669ee500 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fef1806ccd0 tx=0x7fef10009190 comp rx=0 tx=0).stop 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.804+0000 7fef223c6740 1 -- 172.21.15.47:0/4204727979 shutdown_connections 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.804+0000 7fef223c6740 1 --2- 172.21.15.47:0/4204727979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669ee500 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.804+0000 7fef223c6740 1 --2- 172.21.15.47:0/4204727979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669c2340 0x5589669c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.804+0000 7fef223c6740 1 --2- 172.21.15.47:0/4204727979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5589669be2f0 0x5589669be6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.804+0000 7fef223c6740 1 -- 172.21.15.47:0/4204727979 >> 172.21.15.47:0/4204727979 conn(0x5589669c0370 msgr2=0x5589669c1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.804+0000 7fef223c6740 1 -- 172.21.15.47:0/4204727979 shutdown_connections 2024-03-23T09:50:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.804+0000 7fef223c6740 1 -- 172.21.15.47:0/4204727979 wait complete. 2024-03-23T09:50:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.804+0000 7fef223c6740 1 -- start start 2024-03-23T09:50:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.804+0000 7fef223c6740 1 Processor -- start 2024-03-23T09:50:17.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.805+0000 7fef223c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669ed690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.805+0000 7fef223c6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558966a09720 0x5589669ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.805+0000 7fef223c6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669ef730 0x5589669e0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.805+0000 7fef223c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589668c9a90 con 0x5589669ee080 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.805+0000 7fef223c6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589668c9730 con 0x5589669ef730 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.805+0000 7fef223c6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589668abe00 con 0x558966a09720 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.805+0000 7fef223c6740 10 client.0 _ll_register_callbacks cb 0x5589668e1b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.805+0000 7fef1d5ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.805+0000 7fef1d5ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669ed690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37042/0 (socket says 172.21.15.47:37042) 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.805+0000 7fef1d5ed640 1 -- 172.21.15.47:0/3958202705 learned_addr learned my addr 172.21.15.47:0/3958202705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7fef1cdec640 1 --2- 172.21.15.47:0/3958202705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558966a09720 0x5589669ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7fef1ddee640 1 --2- 172.21.15.47:0/3958202705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669ef730 0x5589669e0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7fef1d5ed640 1 -- 172.21.15.47:0/3958202705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558966a09720 msgr2=0x5589669ef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/3958202705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558966a09720 0x5589669ef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7fef1d5ed640 1 -- 172.21.15.47:0/3958202705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669ef730 msgr2=0x5589669e0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/3958202705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669ef730 0x5589669e0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7fef1d5ed640 1 -- 172.21.15.47:0/3958202705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef14009a70 con 0x5589669ee080 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/3958202705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669ed690 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fef1407a600 tx=0x7fef14009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3a207e7b719481 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7feef7fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7feef7fff640 1 -- 172.21.15.47:0/3958202705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1400e020 con 0x5589669ee080 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7feef7fff640 1 -- 172.21.15.47:0/3958202705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef1407e730 con 0x5589669ee080 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7fef223c6740 1 -- 172.21.15.47:0/3958202705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55896661c560 con 0x5589669ee080 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7feef7fff640 1 -- 172.21.15.47:0/3958202705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef14082cf0 con 0x5589669ee080 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7fef223c6740 10 client.8856 fetch_fsmap learned FSMap version 65 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7fef223c6740 1 -- 172.21.15.47:0/3958202705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x558966927040 con 0x5589669ee080 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7feef7fff640 1 -- 172.21.15.47:0/3958202705 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef140a7020 con 0x5589669ee080 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7feef7fff640 1 -- 172.21.15.47:0/3958202705 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fef1407ab60 con 0x5589669ee080 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.806+0000 7fef223c6740 10 client.8856 fetch_fsmap finished waiting for FSMap version 65 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.807+0000 7fef223c6740 10 client.8856 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.807+0000 7fef223c6740 20 client.8856 populate_metadata read hostname 'smithi047' 2024-03-23T09:50:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.807+0000 7fef223c6740 1 --2- 172.21.15.47:0/3958202705 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x558966a24160 0x558966a26550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.807+0000 7fef223c6740 4 client.8856 mds_command: new command op to 8484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.807+0000 7fef223c6740 1 -- 172.21.15.47:0/3958202705 --> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55896662d860 con 0x558966a24160 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.807+0000 7fef1cdec640 1 --2- 172.21.15.47:0/3958202705 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x558966a24160 0x558966a26550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.807+0000 7feef7fff640 10 client.8856 ms_handle_connect on v2:172.21.15.47:6833/2388298413 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.807+0000 7fef1cdec640 1 --2- 172.21.15.47:0/3958202705 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x558966a24160 0x558966a26550 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b1a8769c909eb6 server_cookie=88ed9556a81ac680 in_seq=0 out_seq=0 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.807+0000 7feef7fff640 1 -- 172.21.15.47:0/3958202705 <== mds.0 v2:172.21.15.47:6833/2388298413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55896662d860 con 0x558966a24160 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.807+0000 7feef7fff640 10 client.8856 handle_command_reply: tid=0 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.808+0000 7fef223c6740 1 client.8856 shutdown 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.808+0000 7fef223c6740 1 -- 172.21.15.47:0/3958202705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 msgr2=0x5589669ed690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.808+0000 7fef223c6740 1 --2- 172.21.15.47:0/3958202705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669ed690 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fef1407a600 tx=0x7fef14009de0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.808+0000 7fef223c6740 1 -- 172.21.15.47:0/3958202705 shutdown_connections 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.808+0000 7fef223c6740 1 --2- 172.21.15.47:0/3958202705 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x558966a24160 0x558966a26550 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.808+0000 7fef223c6740 1 --2- 172.21.15.47:0/3958202705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669ef730 0x5589669e0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.808+0000 7feef7fff640 0 client.8856 ms_handle_reset on v2:172.21.15.47:6833/2388298413 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.808+0000 7fef223c6740 1 --2- 172.21.15.47:0/3958202705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558966a09720 0x5589669ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.808+0000 7fef223c6740 1 --2- 172.21.15.47:0/3958202705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669ed690 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.808+0000 7fef223c6740 1 -- 172.21.15.47:0/3958202705 >> 172.21.15.47:0/3958202705 conn(0x5589669be2f0 msgr2=0x558966a0a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.808+0000 7fef223c6740 1 -- 172.21.15.47:0/3958202705 shutdown_connections 2024-03-23T09:50:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.808+0000 7fef223c6740 1 -- 172.21.15.47:0/3958202705 wait complete. 2024-03-23T09:50:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.808+0000 7fef223c6740 20 client.8856 trim_cache size 0 max 16384 2024-03-23T09:50:17.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.811+0000 7f8d2d106640 1 Processor -- start 2024-03-23T09:50:17.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.811+0000 7f8d2d106640 1 -- start start 2024-03-23T09:50:17.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d2d106640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d2815aa10 0x7f8d280b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d2d106640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8d2806f410 0x7f8d28164980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d2d106640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d280cb4d0 con 0x7f8d2806e880 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d2d106640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d280cb770 con 0x7f8d2806f410 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d2d106640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d280cba50 con 0x7f8d2815aa10 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d2659c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8d2806e880 0x7f8d2806ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59144/0 (socket says 172.21.15.47:59144) 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d2659c640 1 -- 172.21.15.47:0/172586544 learned_addr learned my addr 172.21.15.47:0/172586544 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d2759e640 1 --2- 172.21.15.47:0/172586544 >> v2:172.21.15.175:3300/0 conn(0x7f8d2806f410 0x7f8d28164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d26d9d640 1 --2- 172.21.15.47:0/172586544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d2815aa10 0x7f8d280b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275131469 0 0) 0x7f8d280cb4d0 con 0x7f8d2806e880 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d10003520 con 0x7f8d2806e880 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726093742 0 0) 0x7f8d10003520 con 0x7f8d2806e880 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d2815aa10 msgr2=0x7f8d280b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 --2- 172.21.15.47:0/172586544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d2815aa10 0x7f8d280b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 >> v2:172.21.15.175:3300/0 conn(0x7f8d2806f410 msgr2=0x7f8d28164980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 --2- 172.21.15.47:0/172586544 >> v2:172.21.15.175:3300/0 conn(0x7f8d2806f410 0x7f8d28164980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d280cbd30 con 0x7f8d2806e880 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f8d1c001670 con 0x7f8d2806e880 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 >> v1:172.21.15.47:6789/0 conn(0x7f8d2806e880 legacy=0x7f8d2806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 --2- 172.21.15.47:0/172586544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d2815aa10 0x7f8d10003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 --2- 172.21.15.47:0/172586544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d10003ef0 0x7f8d100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 --2- 172.21.15.47:0/172586544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d10004d00 0x7f8d10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d280cb770 con 0x7f8d10003ef0 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d280cba50 con 0x7f8d10004d00 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1000b980 con 0x7f8d2815aa10 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.812+0000 7f8d2659c640 1 --2- 172.21.15.47:0/172586544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d10003ef0 0x7f8d100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.813+0000 7f8d26d9d640 1 --2- 172.21.15.47:0/172586544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d2815aa10 0x7f8d10003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.813+0000 7f8d2759e640 1 --2- 172.21.15.47:0/172586544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d10004d00 0x7f8d10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.813+0000 7f8d2659c640 1 -- 172.21.15.47:0/172586544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d2815aa10 msgr2=0x7f8d10003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.813+0000 7f8d2659c640 1 --2- 172.21.15.47:0/172586544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d2815aa10 0x7f8d10003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.813+0000 7f8d2659c640 1 -- 172.21.15.47:0/172586544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d10004d00 msgr2=0x7f8d10005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.813+0000 7f8d2659c640 1 --2- 172.21.15.47:0/172586544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d10004d00 0x7f8d10005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.813+0000 7f8d2659c640 1 -- 172.21.15.47:0/172586544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d10003520 con 0x7f8d10003ef0 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.813+0000 7f8d2659c640 1 --2- 172.21.15.47:0/172586544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d10003ef0 0x7f8d100047c0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c290 tx=0x7f8d100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3473119d0b47277b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.814+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c005ad0 con 0x7f8d10003ef0 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.814+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c005c30 con 0x7f8d10003ef0 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.814+0000 7f8d25d9b640 1 -- 172.21.15.47:0/172586544 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c02c8c0 con 0x7f8d10003ef0 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.814+0000 7f8d2d106640 1 -- 172.21.15.47:0/172586544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d10003ef0 msgr2=0x7f8d100047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.814+0000 7f8d2d106640 1 --2- 172.21.15.47:0/172586544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d10003ef0 0x7f8d100047c0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c290 tx=0x7f8d100057f0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.814+0000 7f8d2759e640 1 -- 172.21.15.47:0/172586544 reap_dead start 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.814+0000 7f8d2d106640 1 -- 172.21.15.47:0/172586544 shutdown_connections 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.814+0000 7f8d2d106640 1 -- 172.21.15.47:0/172586544 >> 172.21.15.47:0/172586544 conn(0x7f8d280d0770 msgr2=0x7f8d280d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.814+0000 7f8d2d106640 1 -- 172.21.15.47:0/172586544 shutdown_connections 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.814+0000 7f8d2d106640 1 -- 172.21.15.47:0/172586544 wait complete. 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.816+0000 7f8d2d106640 1 Processor -- start 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.817+0000 7f8d2d106640 1 -- start start 2024-03-23T09:50:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d2d106640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d10003ef0 0x7f8d28147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d2d106640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d281477d0 0x7f8d28149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d2d106640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d2814a290 0x7f8d2814a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d2d106640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d28148520 con 0x7f8d10003ef0 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d2d106640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d280c41f0 con 0x7f8d2814a290 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d2d106640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2814ac60 con 0x7f8d281477d0 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d26d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d10003ef0 0x7f8d28147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d26d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d10003ef0 0x7f8d28147290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37064/0 (socket says 172.21.15.47:37064) 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d26d9d640 1 -- 172.21.15.47:0/139136251 learned_addr learned my addr 172.21.15.47:0/139136251 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d2659c640 1 --2- 172.21.15.47:0/139136251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d281477d0 0x7f8d28149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d2759e640 1 --2- 172.21.15.47:0/139136251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d2814a290 0x7f8d2814a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d26d9d640 1 -- 172.21.15.47:0/139136251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d281477d0 msgr2=0x7f8d28149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d26d9d640 1 --2- 172.21.15.47:0/139136251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d281477d0 0x7f8d28149d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d26d9d640 1 -- 172.21.15.47:0/139136251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d2814a290 msgr2=0x7f8d2814a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d26d9d640 1 --2- 172.21.15.47:0/139136251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d2814a290 0x7f8d2814a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d26d9d640 1 -- 172.21.15.47:0/139136251 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d280c9f10 con 0x7f8d10003ef0 2024-03-23T09:50:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d2659c640 1 --2- 172.21.15.47:0/139136251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d281477d0 0x7f8d28149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d2759e640 1 --2- 172.21.15.47:0/139136251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d2814a290 0x7f8d2814a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.818+0000 7f8d26d9d640 1 --2- 172.21.15.47:0/139136251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d10003ef0 0x7f8d28147290 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f8d180119a0 tx=0x7f8d18011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.819+0000 7f8d0f7fe640 1 -- 172.21.15.47:0/139136251 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1800faf0 con 0x7f8d10003ef0 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.819+0000 7f8d0f7fe640 1 -- 172.21.15.47:0/139136251 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1800fc50 con 0x7f8d10003ef0 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.819+0000 7f8d2d106640 1 -- 172.21.15.47:0/139136251 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d28147e60 con 0x7f8d10003ef0 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.819+0000 7f8d0f7fe640 1 -- 172.21.15.47:0/139136251 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d18014440 con 0x7f8d10003ef0 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.819+0000 7f8d2d106640 1 -- 172.21.15.47:0/139136251 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d28148940 con 0x7f8d10003ef0 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.820+0000 7f8d2d106640 1 -- 172.21.15.47:0/139136251 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f8d28147cc0 con 0x7f8d10003ef0 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.820+0000 7f8d0f7fe640 1 -- 172.21.15.47:0/139136251 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8d180145e0 con 0x7f8d10003ef0 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.820+0000 7f8d0f7fe640 1 --2- 172.21.15.47:0/139136251 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d1406ac00 0x7f8d1406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.821+0000 7f8d0f7fe640 1 -- 172.21.15.47:0/139136251 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d180891c0 con 0x7f8d10003ef0 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.821+0000 7f8d0f7fe640 1 -- 172.21.15.47:0/139136251 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+511 (secure 0 0 0) 0x7f8d28147cc0 con 0x7f8d10003ef0 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.821+0000 7f8d2659c640 1 --2- 172.21.15.47:0/139136251 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d1406ac00 0x7f8d1406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.821+0000 7f8d2d106640 1 -- 172.21.15.47:0/139136251 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f8d28147cc0 con 0x7f8d10003ef0 2024-03-23T09:50:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.821+0000 7f8d2659c640 1 --2- 172.21.15.47:0/139136251 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d1406ac00 0x7f8d1406d0c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c034000 tx=0x7f8d1c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.826+0000 7fef223c6740 1 Processor -- start 2024-03-23T09:50:17.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.826+0000 7fef223c6740 1 -- start start 2024-03-23T09:50:17.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef223c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef223c6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669ef730 0x5589669ee460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef223c6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558966a09720 0x5589669ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef223c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589669c4cb0 con 0x5589669ee080 2024-03-23T09:50:17.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef223c6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589669ef3d0 con 0x5589669ef730 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef223c6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589669bf670 con 0x558966a09720 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef1d5ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef1d5ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669c38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37074/0 (socket says 172.21.15.47:37074) 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef1d5ed640 1 -- 172.21.15.47:0/4156540081 learned_addr learned my addr 172.21.15.47:0/4156540081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef1cdec640 1 --2- 172.21.15.47:0/4156540081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669ef730 0x5589669ee460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef1d5ed640 1 -- 172.21.15.47:0/4156540081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558966a09720 msgr2=0x5589669ee9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/4156540081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558966a09720 0x5589669ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef1d5ed640 1 -- 172.21.15.47:0/4156540081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669ef730 msgr2=0x5589669ee460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/4156540081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669ef730 0x5589669ee460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef1d5ed640 1 -- 172.21.15.47:0/4156540081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589669efec0 con 0x5589669ee080 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/4156540081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669c38f0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fef14009f30 tx=0x7fef1407c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d6ff6736d881de server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef037fe640 1 -- 172.21.15.47:0/4156540081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1400e020 con 0x5589669ee080 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef037fe640 1 -- 172.21.15.47:0/4156540081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef140a7070 con 0x5589669ee080 2024-03-23T09:50:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.827+0000 7fef037fe640 1 -- 172.21.15.47:0/4156540081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1407b000 con 0x5589669ee080 2024-03-23T09:50:17.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.828+0000 7fef223c6740 1 -- 172.21.15.47:0/4156540081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 msgr2=0x5589669c38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.828+0000 7fef223c6740 1 --2- 172.21.15.47:0/4156540081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669c38f0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fef14009f30 tx=0x7fef1407c000 comp rx=0 tx=0).stop 2024-03-23T09:50:17.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.828+0000 7fef223c6740 1 -- 172.21.15.47:0/4156540081 shutdown_connections 2024-03-23T09:50:17.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.828+0000 7fef223c6740 1 --2- 172.21.15.47:0/4156540081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558966a09720 0x5589669ee9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.828+0000 7fef223c6740 1 --2- 172.21.15.47:0/4156540081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589669ef730 0x5589669ee460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.828+0000 7fef223c6740 1 --2- 172.21.15.47:0/4156540081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ee080 0x5589669c38f0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.828+0000 7fef223c6740 1 -- 172.21.15.47:0/4156540081 >> 172.21.15.47:0/4156540081 conn(0x5589669be2f0 msgr2=0x5589669e17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.828+0000 7fef223c6740 1 -- 172.21.15.47:0/4156540081 shutdown_connections 2024-03-23T09:50:17.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.828+0000 7fef223c6740 1 -- 172.21.15.47:0/4156540081 wait complete. 2024-03-23T09:50:17.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.829+0000 7fef223c6740 1 -- start start 2024-03-23T09:50:17.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.829+0000 7fef223c6740 1 Processor -- start 2024-03-23T09:50:17.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef223c6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5589669ee080 0x5589669e2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef223c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ef730 0x5589669e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef223c6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558966a09720 0x558966a26090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef223c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589669bdd80 con 0x5589669ef730 2024-03-23T09:50:17.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef223c6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558966a272c0 con 0x558966a09720 2024-03-23T09:50:17.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef223c6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589669eea00 con 0x5589669ee080 2024-03-23T09:50:17.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef223c6740 10 client.0 _ll_register_callbacks cb 0x7fef10006a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:50:17.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef1cdec640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ef730 0x5589669e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef1d5ed640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5589669ee080 0x5589669e2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef1cdec640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ef730 0x5589669e28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37086/0 (socket says 172.21.15.47:37086) 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef1d5ed640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5589669ee080 0x5589669e2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:52616/0 (socket says 172.21.15.47:52616) 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef1cdec640 1 -- 172.21.15.47:0/2310116743 learned_addr learned my addr 172.21.15.47:0/2310116743 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef1ddee640 1 --2- 172.21.15.47:0/2310116743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558966a09720 0x558966a26090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef1cdec640 1 -- 172.21.15.47:0/2310116743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5589669ee080 msgr2=0x5589669e2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef1cdec640 1 --2- 172.21.15.47:0/2310116743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5589669ee080 0x5589669e2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef1cdec640 1 -- 172.21.15.47:0/2310116743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558966a09720 msgr2=0x558966a26090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef1cdec640 1 --2- 172.21.15.47:0/2310116743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558966a09720 0x558966a26090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef1cdec640 1 -- 172.21.15.47:0/2310116743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558966927040 con 0x5589669ef730 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef1cdec640 1 --2- 172.21.15.47:0/2310116743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ef730 0x5589669e28a0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fef08004840 tx=0x7fef08001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7042413e5ea37 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7feef7fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7feef7fff640 1 -- 172.21.15.47:0/2310116743 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08027070 con 0x5589669ef730 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7feef7fff640 1 -- 172.21.15.47:0/2310116743 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef0801a040 con 0x5589669ef730 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7fef223c6740 1 -- 172.21.15.47:0/2310116743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef10014c00 con 0x5589669ef730 2024-03-23T09:50:17.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.830+0000 7feef7fff640 1 -- 172.21.15.47:0/2310116743 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef0802a620 con 0x5589669ef730 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.831+0000 7fef223c6740 10 client.8868 fetch_fsmap learned FSMap version 65 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.831+0000 7fef223c6740 1 -- 172.21.15.47:0/2310116743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x558966a11890 con 0x5589669ef730 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.831+0000 7feef7fff640 1 -- 172.21.15.47:0/2310116743 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef10014c00 con 0x5589669ef730 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.831+0000 7feef7fff640 1 -- 172.21.15.47:0/2310116743 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fef0802a780 con 0x5589669ef730 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.831+0000 7fef223c6740 10 client.8868 fetch_fsmap finished waiting for FSMap version 65 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.831+0000 7fef223c6740 10 client.8868 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.831+0000 7fef223c6740 20 client.8868 populate_metadata read hostname 'smithi047' 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.831+0000 7fef223c6740 1 --2- 172.21.15.47:0/2310116743 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x558966a24160 0x558966a265d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.831+0000 7fef223c6740 4 client.8868 mds_command: new command op to 8484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.831+0000 7fef223c6740 1 -- 172.21.15.47:0/2310116743 --> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x558966680eb0 con 0x558966a24160 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.831+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/2310116743 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x558966a24160 0x558966a265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.831+0000 7feef7fff640 10 client.8868 ms_handle_connect on v2:172.21.15.47:6833/2388298413 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.831+0000 7fef1d5ed640 1 --2- 172.21.15.47:0/2310116743 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x558966a24160 0x558966a265d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b477c26d7629e server_cookie=1c88bde038b12351 in_seq=0 out_seq=0 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7feef7fff640 1 -- 172.21.15.47:0/2310116743 <== mds.0 v2:172.21.15.47:6833/2388298413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558966680eb0 con 0x558966a24160 2024-03-23T09:50:17.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7feef7fff640 10 client.8868 handle_command_reply: tid=0 2024-03-23T09:50:17.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7fef223c6740 1 client.8868 shutdown 2024-03-23T09:50:17.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7fef223c6740 1 -- 172.21.15.47:0/2310116743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ef730 msgr2=0x5589669e28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7fef223c6740 1 --2- 172.21.15.47:0/2310116743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ef730 0x5589669e28a0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fef08004840 tx=0x7fef08001a00 comp rx=0 tx=0).stop 2024-03-23T09:50:17.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7fef223c6740 1 -- 172.21.15.47:0/2310116743 shutdown_connections 2024-03-23T09:50:17.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7fef223c6740 1 --2- 172.21.15.47:0/2310116743 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x558966a24160 0x558966a265d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7fef223c6740 1 --2- 172.21.15.47:0/2310116743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558966a09720 0x558966a26090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7fef223c6740 1 --2- 172.21.15.47:0/2310116743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5589669ef730 0x5589669e28a0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7fef223c6740 1 --2- 172.21.15.47:0/2310116743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5589669ee080 0x5589669e2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7fef223c6740 1 -- 172.21.15.47:0/2310116743 >> 172.21.15.47:0/2310116743 conn(0x5589669be2f0 msgr2=0x5589669c37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7feef7fff640 0 client.8868 ms_handle_reset on v2:172.21.15.47:6833/2388298413 2024-03-23T09:50:17.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7fef223c6740 1 -- 172.21.15.47:0/2310116743 shutdown_connections 2024-03-23T09:50:17.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7fef223c6740 1 -- 172.21.15.47:0/2310116743 wait complete. 2024-03-23T09:50:17.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.832+0000 7fef223c6740 20 client.8868 trim_cache size 0 max 16384 2024-03-23T09:50:17.839 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:50:17.839 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-23T09:50:17.839 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "a29b5fa3-22e4-43b4-aa4d-e09be3350a5e", 2024-03-23T09:50:17.839 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-23T09:50:17.839 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:50:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.833+0000 7fef1f851640 1 -- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feefc06ac30 msgr2=0x7feefc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.833+0000 7fef1f851640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feefc06ac30 0x7feefc06d0f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fef181f2ff0 tx=0x7fef08009040 comp rx=0 tx=0).stop 2024-03-23T09:50:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.833+0000 7fef1f851640 1 -- 172.21.15.47:0/2046798480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef181e4480 msgr2=0x7fef180c5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.833+0000 7fef1f851640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef181e4480 0x7fef180c5200 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fef14037040 tx=0x7fef14009750 comp rx=0 tx=0).stop 2024-03-23T09:50:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.833+0000 7fef1f851640 1 -- 172.21.15.47:0/2046798480 shutdown_connections 2024-03-23T09:50:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.833+0000 7fef1f851640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7feefc06ac30 0x7feefc06d0f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.833+0000 7fef1f851640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef181e4860 0x7fef18156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.833+0000 7fef1f851640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef181f2bf0 0x7fef180c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.833+0000 7fef1f851640 1 --2- 172.21.15.47:0/2046798480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef181e4480 0x7fef180c5200 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.833+0000 7fef1f851640 1 -- 172.21.15.47:0/2046798480 >> 172.21.15.47:0/2046798480 conn(0x7fef18098a00 msgr2=0x7fef180ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.833+0000 7fef1f851640 1 -- 172.21.15.47:0/2046798480 shutdown_connections 2024-03-23T09:50:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.833+0000 7fef1f851640 1 -- 172.21.15.47:0/2046798480 wait complete. 2024-03-23T09:50:17.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:17.844 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-23T09:50:17.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.947+0000 7fa42db99640 1 Processor -- start 2024-03-23T09:50:17.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.947+0000 7fa42db99640 1 -- start start 2024-03-23T09:50:17.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.948+0000 7fa42db99640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa42815c990 0x7fa428160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.948+0000 7fa42db99640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa4280cde10 0x7fa4280d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.948+0000 7fa42db99640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa428161630 con 0x7fa4280b8350 2024-03-23T09:50:17.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.948+0000 7fa42db99640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4280d1030 con 0x7fa4280cde10 2024-03-23T09:50:17.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.948+0000 7fa42db99640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4280cfe10 con 0x7fa42815c990 2024-03-23T09:50:17.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.948+0000 7fa427fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa4280cde10 0x7fa4280d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.948+0000 7fa427fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa4280cde10 0x7fa4280d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37094/0 (socket says 172.21.15.47:37094) 2024-03-23T09:50:17.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.948+0000 7fa427fff640 1 -- 172.21.15.47:0/2857367919 learned_addr learned my addr 172.21.15.47:0/2857367919 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:17.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.948+0000 7fa4277fe640 1 --2- 172.21.15.47:0/2857367919 >> v2:172.21.15.175:3300/0 conn(0x7fa42815c990 0x7fa428160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.948+0000 7fa427fff640 1 -- 172.21.15.47:0/2857367919 >> v2:172.21.15.175:3300/0 conn(0x7fa42815c990 msgr2=0x7fa428160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.948+0000 7fa427fff640 1 --2- 172.21.15.47:0/2857367919 >> v2:172.21.15.175:3300/0 conn(0x7fa42815c990 0x7fa428160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.948+0000 7fa427fff640 1 -- 172.21.15.47:0/2857367919 >> v1:172.21.15.175:6789/0 conn(0x7fa4280b8350 legacy=0x7fa4280cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.948+0000 7fa427fff640 1 -- 172.21.15.47:0/2857367919 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4280b5b60 con 0x7fa4280cde10 2024-03-23T09:50:17.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.949+0000 7fa427fff640 1 --2- 172.21.15.47:0/2857367919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4280cde10 0x7fa4280d14c0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa4280b8b30 tx=0x7fa42002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cb9d95927f67c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.949+0000 7fa4267fc640 1 -- 172.21.15.47:0/2857367919 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42002c9f0 con 0x7fa4280cde10 2024-03-23T09:50:17.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.949+0000 7fa4267fc640 1 -- 172.21.15.47:0/2857367919 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42002cb50 con 0x7fa4280cde10 2024-03-23T09:50:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.949+0000 7fa4267fc640 1 -- 172.21.15.47:0/2857367919 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4200343f0 con 0x7fa4280cde10 2024-03-23T09:50:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.950+0000 7fa42db99640 1 -- 172.21.15.47:0/2857367919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4280cde10 msgr2=0x7fa4280d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.950+0000 7fa42db99640 1 --2- 172.21.15.47:0/2857367919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4280cde10 0x7fa4280d14c0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa4280b8b30 tx=0x7fa42002be50 comp rx=0 tx=0).stop 2024-03-23T09:50:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.950+0000 7fa42db99640 1 -- 172.21.15.47:0/2857367919 shutdown_connections 2024-03-23T09:50:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.950+0000 7fa42db99640 1 --2- 172.21.15.47:0/2857367919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4280cde10 0x7fa4280d14c0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.950+0000 7fa42db99640 1 --2- 172.21.15.47:0/2857367919 >> v2:172.21.15.175:3300/0 conn(0x7fa42815c990 0x7fa428160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.950+0000 7fa42db99640 1 -- 172.21.15.47:0/2857367919 >> 172.21.15.47:0/2857367919 conn(0x7fa4280659d0 msgr2=0x7fa428065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.950+0000 7fa42db99640 1 -- 172.21.15.47:0/2857367919 shutdown_connections 2024-03-23T09:50:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.950+0000 7fa42db99640 1 -- 172.21.15.47:0/2857367919 wait complete. 2024-03-23T09:50:17.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.951+0000 7fa42db99640 1 Processor -- start 2024-03-23T09:50:17.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.951+0000 7fa42db99640 1 -- start start 2024-03-23T09:50:17.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa42db99640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4280b8350 0x7fa428153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa42db99640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4280cde10 0x7fa4280c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa42db99640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa42815c990 0x7fa4280b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa42db99640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42815d510 con 0x7fa4280cde10 2024-03-23T09:50:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa42db99640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4281544b0 con 0x7fa4280b8350 2024-03-23T09:50:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa42db99640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4281598f0 con 0x7fa42815c990 2024-03-23T09:50:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa426ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4280cde10 0x7fa4280c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa426ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4280cde10 0x7fa4280c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37100/0 (socket says 172.21.15.47:37100) 2024-03-23T09:50:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa426ffd640 1 -- 172.21.15.47:0/1094558217 learned_addr learned my addr 172.21.15.47:0/1094558217 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa4277fe640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4280b8350 0x7fa428153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa427fff640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa42815c990 0x7fa4280b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa426ffd640 1 -- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa42815c990 msgr2=0x7fa4280b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa426ffd640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa42815c990 0x7fa4280b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa426ffd640 1 -- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4280b8350 msgr2=0x7fa428153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa426ffd640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4280b8350 0x7fa428153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa426ffd640 1 -- 172.21.15.47:0/1094558217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420009b30 con 0x7fa4280cde10 2024-03-23T09:50:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa426ffd640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4280cde10 0x7fa4280c7db0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00c430 tx=0x7fa41c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.952+0000 7fa40ffff640 1 -- 172.21.15.47:0/1094558217 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c011790 con 0x7fa4280cde10 2024-03-23T09:50:17.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.953+0000 7fa40ffff640 1 -- 172.21.15.47:0/1094558217 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c0118f0 con 0x7fa4280cde10 2024-03-23T09:50:17.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.953+0000 7fa42db99640 1 -- 172.21.15.47:0/1094558217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa428159bd0 con 0x7fa4280cde10 2024-03-23T09:50:17.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.953+0000 7fa40ffff640 1 -- 172.21.15.47:0/1094558217 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c011ce0 con 0x7fa4280cde10 2024-03-23T09:50:17.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.953+0000 7fa42db99640 1 -- 172.21.15.47:0/1094558217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42815a190 con 0x7fa4280cde10 2024-03-23T09:50:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.954+0000 7fa40ffff640 1 -- 172.21.15.47:0/1094558217 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa41c01c460 con 0x7fa4280cde10 2024-03-23T09:50:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.954+0000 7fa40ffff640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa41406ac30 0x7fa41406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.954+0000 7fa40ffff640 1 -- 172.21.15.47:0/1094558217 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa41c0891f0 con 0x7fa4280cde10 2024-03-23T09:50:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.955+0000 7fa4277fe640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa41406ac30 0x7fa41406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.955+0000 7fa4277fe640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa41406ac30 0x7fa41406d0f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa41800aa20 tx=0x7fa418008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.955+0000 7fa4307e6740 1 Processor -- start 2024-03-23T09:50:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.955+0000 7fa4307e6740 1 -- start start 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.956+0000 7fa4307e6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a310830 0x55b83a310c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.956+0000 7fa4307e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a2e0850 0x55b83a30daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.956+0000 7fa4307e6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a2e43b0 0x55b83a30dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.956+0000 7fa4307e6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83a250dd0 con 0x55b83a310830 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.956+0000 7fa4307e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83a24e560 con 0x55b83a2e0850 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.956+0000 7fa4307e6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b83a249e50 con 0x55b83a2e43b0 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.956+0000 7fa4277fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a310830 0x55b83a310c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.956+0000 7fa4277fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a310830 0x55b83a310c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37108/0 (socket says 172.21.15.47:37108) 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.956+0000 7fa4277fe640 1 -- 172.21.15.47:0/1111447917 learned_addr learned my addr 172.21.15.47:0/1111447917 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.957+0000 7fa4277fe640 1 -- 172.21.15.47:0/1111447917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a2e43b0 msgr2=0x55b83a30dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.957+0000 7fa4277fe640 1 --2- 172.21.15.47:0/1111447917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a2e43b0 0x55b83a30dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.957+0000 7fa426ffd640 1 --2- 172.21.15.47:0/1111447917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a2e0850 0x55b83a30daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.957+0000 7fa4277fe640 1 -- 172.21.15.47:0/1111447917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a2e0850 msgr2=0x55b83a30daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.957+0000 7fa4277fe640 1 --2- 172.21.15.47:0/1111447917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a2e0850 0x55b83a30daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.957+0000 7fa4277fe640 1 -- 172.21.15.47:0/1111447917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b83a211470 con 0x55b83a310830 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.957+0000 7fa4277fe640 1 --2- 172.21.15.47:0/1111447917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a310830 0x55b83a310c10 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fa41800fc90 tx=0x7fa418012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d508ecdd279da5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.957+0000 7fa40d7fa640 1 -- 172.21.15.47:0/1111447917 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418013620 con 0x55b83a310830 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.957+0000 7fa40d7fa640 1 -- 172.21.15.47:0/1111447917 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41800c040 con 0x55b83a310830 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.957+0000 7fa40d7fa640 1 -- 172.21.15.47:0/1111447917 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180139a0 con 0x55b83a310830 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.957+0000 7fa4307e6740 1 -- 172.21.15.47:0/1111447917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a310830 msgr2=0x55b83a310c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.957+0000 7fa4307e6740 1 --2- 172.21.15.47:0/1111447917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a310830 0x55b83a310c10 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fa41800fc90 tx=0x7fa418012000 comp rx=0 tx=0).stop 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.958+0000 7fa4307e6740 1 -- 172.21.15.47:0/1111447917 shutdown_connections 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.958+0000 7fa4307e6740 1 --2- 172.21.15.47:0/1111447917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a2e43b0 0x55b83a30dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.958+0000 7fa4307e6740 1 --2- 172.21.15.47:0/1111447917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a2e0850 0x55b83a30daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.958+0000 7fa4307e6740 1 --2- 172.21.15.47:0/1111447917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a310830 0x55b83a310c10 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.958+0000 7fa4307e6740 1 -- 172.21.15.47:0/1111447917 >> 172.21.15.47:0/1111447917 conn(0x55b83a2e2ac0 msgr2=0x55b83a32b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.958+0000 7fa4307e6740 1 -- 172.21.15.47:0/1111447917 shutdown_connections 2024-03-23T09:50:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.958+0000 7fa4307e6740 1 -- 172.21.15.47:0/1111447917 wait complete. 2024-03-23T09:50:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.958+0000 7fa4307e6740 1 -- start start 2024-03-23T09:50:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.958+0000 7fa4307e6740 1 Processor -- start 2024-03-23T09:50:17.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4307e6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 0x55b83a30b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4307e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a310830 0x55b83a3128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4307e6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a311170 0x55b83a2fdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4307e6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83a208280 con 0x55b83a2e43b0 2024-03-23T09:50:17.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4307e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83a207f20 con 0x55b83a310830 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4307e6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b83a1cd3c0 con 0x55b83a311170 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4307e6740 10 client.0 _ll_register_callbacks cb 0x55b83a21f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4277fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 0x55b83a30b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4277fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 0x55b83a30b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37112/0 (socket says 172.21.15.47:37112) 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4277fe640 1 -- 172.21.15.47:0/4019510271 learned_addr learned my addr 172.21.15.47:0/4019510271 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa426ffd640 1 --2- 172.21.15.47:0/4019510271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a310830 0x55b83a3128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa427fff640 1 --2- 172.21.15.47:0/4019510271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a311170 0x55b83a2fdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4277fe640 1 -- 172.21.15.47:0/4019510271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a311170 msgr2=0x55b83a2fdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4277fe640 1 --2- 172.21.15.47:0/4019510271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a311170 0x55b83a2fdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4277fe640 1 -- 172.21.15.47:0/4019510271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a310830 msgr2=0x55b83a3128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4277fe640 1 --2- 172.21.15.47:0/4019510271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a310830 0x55b83a3128f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4277fe640 1 -- 172.21.15.47:0/4019510271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41800f940 con 0x55b83a2e43b0 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.959+0000 7fa4277fe640 1 --2- 172.21.15.47:0/4019510271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 0x55b83a30b710 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa41800a800 tx=0x7fa418017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738f48813986f8b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4067fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4067fc640 1 -- 172.21.15.47:0/4019510271 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418004020 con 0x55b83a2e43b0 2024-03-23T09:50:17.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4067fc640 1 -- 172.21.15.47:0/4019510271 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa41801a040 con 0x55b83a2e43b0 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4307e6740 1 -- 172.21.15.47:0/4019510271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b83a273690 con 0x55b83a2e43b0 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4067fc640 1 -- 172.21.15.47:0/4019510271 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa418004180 con 0x55b83a2e43b0 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4307e6740 10 client.8880 fetch_fsmap learned FSMap version 65 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4307e6740 1 -- 172.21.15.47:0/4019510271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55b83a2484d0 con 0x55b83a2e43b0 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4067fc640 1 -- 172.21.15.47:0/4019510271 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41803c070 con 0x55b83a2e43b0 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4067fc640 1 -- 172.21.15.47:0/4019510271 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa418011040 con 0x55b83a2e43b0 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4307e6740 10 client.8880 fetch_fsmap finished waiting for FSMap version 65 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4307e6740 10 client.8880 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4307e6740 20 client.8880 populate_metadata read hostname 'smithi047' 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4307e6740 1 --2- 172.21.15.47:0/4019510271 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b83a32f020 0x55b83a309860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4307e6740 4 client.8880 mds_command: new command op to 8484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4307e6740 1 -- 172.21.15.47:0/4019510271 --> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b83a23beb0 con 0x55b83a32f020 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa426ffd640 1 --2- 172.21.15.47:0/4019510271 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b83a32f020 0x55b83a309860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa4067fc640 10 client.8880 ms_handle_connect on v2:172.21.15.47:6833/2388298413 2024-03-23T09:50:17.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.960+0000 7fa426ffd640 1 --2- 172.21.15.47:0/4019510271 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b83a32f020 0x55b83a309860 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8102fdf2d2d2cf79 server_cookie=80e2a02539778971 in_seq=0 out_seq=0 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4067fc640 1 -- 172.21.15.47:0/4019510271 <== mds.0 v2:172.21.15.47:6833/2388298413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b83a23beb0 con 0x55b83a32f020 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4067fc640 10 client.8880 handle_command_reply: tid=0 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4307e6740 1 client.8880 shutdown 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4307e6740 1 -- 172.21.15.47:0/4019510271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 msgr2=0x55b83a30b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4307e6740 1 --2- 172.21.15.47:0/4019510271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 0x55b83a30b710 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa41800a800 tx=0x7fa418017000 comp rx=0 tx=0).stop 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4307e6740 1 -- 172.21.15.47:0/4019510271 shutdown_connections 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4307e6740 1 --2- 172.21.15.47:0/4019510271 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b83a32f020 0x55b83a309860 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4307e6740 1 --2- 172.21.15.47:0/4019510271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a311170 0x55b83a2fdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4307e6740 1 --2- 172.21.15.47:0/4019510271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a310830 0x55b83a3128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4067fc640 0 client.8880 ms_handle_reset on v2:172.21.15.47:6833/2388298413 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4307e6740 1 --2- 172.21.15.47:0/4019510271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 0x55b83a30b710 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4307e6740 1 -- 172.21.15.47:0/4019510271 >> 172.21.15.47:0/4019510271 conn(0x55b83a2e0850 msgr2=0x55b83a3143b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4307e6740 1 -- 172.21.15.47:0/4019510271 shutdown_connections 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4307e6740 1 -- 172.21.15.47:0/4019510271 wait complete. 2024-03-23T09:50:17.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.961+0000 7fa4307e6740 20 client.8880 trim_cache size 0 max 16384 2024-03-23T09:50:17.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.976+0000 7fa4307e6740 1 Processor -- start 2024-03-23T09:50:17.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.976+0000 7fa4307e6740 1 -- start start 2024-03-23T09:50:17.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.976+0000 7fa4307e6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 0x55b83a2e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.976+0000 7fa4307e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a310830 0x55b83a3443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa4307e6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a311170 0x55b83a344900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa4307e6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83a30a180 con 0x55b83a2e43b0 2024-03-23T09:50:17.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa4307e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83a33acc0 con 0x55b83a310830 2024-03-23T09:50:17.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa4307e6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b83a1d2fa0 con 0x55b83a311170 2024-03-23T09:50:17.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa4277fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 0x55b83a2e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa4277fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 0x55b83a2e4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37126/0 (socket says 172.21.15.47:37126) 2024-03-23T09:50:17.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa4277fe640 1 -- 172.21.15.47:0/779062052 learned_addr learned my addr 172.21.15.47:0/779062052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:17.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa426ffd640 1 --2- 172.21.15.47:0/779062052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a310830 0x55b83a3443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa427fff640 1 --2- 172.21.15.47:0/779062052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a311170 0x55b83a344900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa4277fe640 1 -- 172.21.15.47:0/779062052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a311170 msgr2=0x55b83a344900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa4277fe640 1 --2- 172.21.15.47:0/779062052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a311170 0x55b83a344900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa4277fe640 1 -- 172.21.15.47:0/779062052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a310830 msgr2=0x55b83a3443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa4277fe640 1 --2- 172.21.15.47:0/779062052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a310830 0x55b83a3443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa4277fe640 1 -- 172.21.15.47:0/779062052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b83a27c9e0 con 0x55b83a2e43b0 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa4277fe640 1 --2- 172.21.15.47:0/779062052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 0x55b83a2e4aa0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fa418006e30 tx=0x7fa418019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816973865e9e9c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa40dffb640 1 -- 172.21.15.47:0/779062052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418004420 con 0x55b83a2e43b0 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa40dffb640 1 -- 172.21.15.47:0/779062052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41803c070 con 0x55b83a2e43b0 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.977+0000 7fa40dffb640 1 -- 172.21.15.47:0/779062052 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418004240 con 0x55b83a2e43b0 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.978+0000 7fa4307e6740 1 -- 172.21.15.47:0/779062052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 msgr2=0x55b83a2e4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.978+0000 7fa4307e6740 1 --2- 172.21.15.47:0/779062052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 0x55b83a2e4aa0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fa418006e30 tx=0x7fa418019000 comp rx=0 tx=0).stop 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.978+0000 7fa4307e6740 1 -- 172.21.15.47:0/779062052 shutdown_connections 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.978+0000 7fa4307e6740 1 --2- 172.21.15.47:0/779062052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a311170 0x55b83a344900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.978+0000 7fa4307e6740 1 --2- 172.21.15.47:0/779062052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a310830 0x55b83a3443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.978+0000 7fa4307e6740 1 --2- 172.21.15.47:0/779062052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a2e43b0 0x55b83a2e4aa0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.978+0000 7fa4307e6740 1 -- 172.21.15.47:0/779062052 >> 172.21.15.47:0/779062052 conn(0x55b83a2e0850 msgr2=0x55b83a3157d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.978+0000 7fa4307e6740 1 -- 172.21.15.47:0/779062052 shutdown_connections 2024-03-23T09:50:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.978+0000 7fa4307e6740 1 -- 172.21.15.47:0/779062052 wait complete. 2024-03-23T09:50:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.979+0000 7fa4307e6740 1 -- start start 2024-03-23T09:50:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.979+0000 7fa4307e6740 1 Processor -- start 2024-03-23T09:50:17.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.979+0000 7fa4307e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a2e43b0 0x55b83a315770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.979+0000 7fa4307e6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a310830 0x55b83a2fc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.979+0000 7fa4307e6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a311170 0x55b83a2fc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.979+0000 7fa4307e6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83a34b220 con 0x55b83a311170 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.979+0000 7fa4307e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83a3133e0 con 0x55b83a2e43b0 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.979+0000 7fa4307e6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b83a30cd10 con 0x55b83a310830 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.979+0000 7fa4307e6740 10 client.0 _ll_register_callbacks cb 0x7fa41800a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.979+0000 7fa427fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a311170 0x55b83a2fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa427fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a311170 0x55b83a2fc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37138/0 (socket says 172.21.15.47:37138) 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa427fff640 1 -- 172.21.15.47:0/3220635319 learned_addr learned my addr 172.21.15.47:0/3220635319 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa4277fe640 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a2e43b0 0x55b83a315770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa426ffd640 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a310830 0x55b83a2fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa427fff640 1 -- 172.21.15.47:0/3220635319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a310830 msgr2=0x55b83a2fc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa427fff640 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a310830 0x55b83a2fc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa427fff640 1 -- 172.21.15.47:0/3220635319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a2e43b0 msgr2=0x55b83a315770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa427fff640 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a2e43b0 0x55b83a315770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa427fff640 1 -- 172.21.15.47:0/3220635319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b83a2484d0 con 0x55b83a311170 2024-03-23T09:50:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa4277fe640 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a2e43b0 0x55b83a315770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:50:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa427fff640 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a311170 0x55b83a2fc770 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fa420009f40 tx=0x7fa420004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8e9083d52b09c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa4067fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:50:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa4067fc640 1 -- 172.21.15.47:0/3220635319 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420034600 con 0x55b83a311170 2024-03-23T09:50:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa4067fc640 1 -- 172.21.15.47:0/3220635319 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa420043070 con 0x55b83a311170 2024-03-23T09:50:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa4307e6740 1 -- 172.21.15.47:0/3220635319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa41800c040 con 0x55b83a311170 2024-03-23T09:50:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa4067fc640 1 -- 172.21.15.47:0/3220635319 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa420034760 con 0x55b83a311170 2024-03-23T09:50:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa4307e6740 10 client.8886 fetch_fsmap learned FSMap version 65 2024-03-23T09:50:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa4307e6740 1 -- 172.21.15.47:0/3220635319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55b83a1ea5c0 con 0x55b83a311170 2024-03-23T09:50:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.980+0000 7fa4067fc640 1 -- 172.21.15.47:0/3220635319 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41800c040 con 0x55b83a311170 2024-03-23T09:50:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4067fc640 1 -- 172.21.15.47:0/3220635319 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa42003c040 con 0x55b83a311170 2024-03-23T09:50:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 10 client.8886 fetch_fsmap finished waiting for FSMap version 65 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 10 client.8886 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 20 client.8886 populate_metadata read hostname 'smithi047' 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b83a37a730 0x55b83a3443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 4 client.8886 mds_command: new command op to 8484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 1 -- 172.21.15.47:0/3220635319 --> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b839f3d4e0 con 0x55b83a37a730 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4277fe640 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b83a37a730 0x55b83a3443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4067fc640 10 client.8886 ms_handle_connect on v2:172.21.15.47:6833/2388298413 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4277fe640 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b83a37a730 0x55b83a3443c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0c1119363d051a2 server_cookie=d20b62790ab1c9eb in_seq=0 out_seq=0 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4067fc640 1 -- 172.21.15.47:0/3220635319 <== mds.0 v2:172.21.15.47:6833/2388298413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55b839f3d4e0 con 0x55b83a37a730 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4067fc640 10 client.8886 handle_command_reply: tid=0 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 1 client.8886 shutdown 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 1 -- 172.21.15.47:0/3220635319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a311170 msgr2=0x55b83a2fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a311170 0x55b83a2fc770 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fa420009f40 tx=0x7fa420004ab0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 1 -- 172.21.15.47:0/3220635319 shutdown_connections 2024-03-23T09:50:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b83a37a730 0x55b83a3443c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b83a311170 0x55b83a2fc770 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b83a310830 0x55b83a2fc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 1 --2- 172.21.15.47:0/3220635319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b83a2e43b0 0x55b83a315770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4067fc640 0 client.8886 ms_handle_reset on v2:172.21.15.47:6833/2388298413 2024-03-23T09:50:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 1 -- 172.21.15.47:0/3220635319 >> 172.21.15.47:0/3220635319 conn(0x55b83a2e0850 msgr2=0x55b83a30d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 1 -- 172.21.15.47:0/3220635319 shutdown_connections 2024-03-23T09:50:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.981+0000 7fa4307e6740 1 -- 172.21.15.47:0/3220635319 wait complete. 2024-03-23T09:50:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.982+0000 7fa4307e6740 20 client.8886 trim_cache size 0 max 16384 2024-03-23T09:50:17.988 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:50:17.988 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T09:50:17.988 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-23T09:50:17.989 INFO:teuthology.orchestra.run.smithi047.stdout: "a29b5fa3-22e4-43b4-aa4d-e09be3350a5e": { 2024-03-23T09:50:17.989 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-23T09:50:17.989 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "a29b5fa3-22e4-43b4-aa4d-e09be3350a5e", 2024-03-23T09:50:17.989 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,repair" 2024-03-23T09:50:17.989 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:50:17.989 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:50:17.989 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:50:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.982+0000 7fa42db99640 1 -- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa41406ac30 msgr2=0x7fa41406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.982+0000 7fa42db99640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa41406ac30 0x7fa41406d0f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa41800aa20 tx=0x7fa418008040 comp rx=0 tx=0).stop 2024-03-23T09:50:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.982+0000 7fa42db99640 1 -- 172.21.15.47:0/1094558217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4280cde10 msgr2=0x7fa4280c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.982+0000 7fa42db99640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4280cde10 0x7fa4280c7db0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00c430 tx=0x7fa41c00c8f0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.982+0000 7fa42db99640 1 -- 172.21.15.47:0/1094558217 shutdown_connections 2024-03-23T09:50:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.982+0000 7fa42db99640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa41406ac30 0x7fa41406d0f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.982+0000 7fa42db99640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa42815c990 0x7fa4280b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.982+0000 7fa42db99640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4280cde10 0x7fa4280c7db0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.982+0000 7fa42db99640 1 --2- 172.21.15.47:0/1094558217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4280b8350 0x7fa428153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.982+0000 7fa42db99640 1 -- 172.21.15.47:0/1094558217 >> 172.21.15.47:0/1094558217 conn(0x7fa4280659d0 msgr2=0x7fa4280b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.982+0000 7fa42db99640 1 -- 172.21.15.47:0/1094558217 shutdown_connections 2024-03-23T09:50:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:17.982+0000 7fa42db99640 1 -- 172.21.15.47:0/1094558217 wait complete. 2024-03-23T09:50:17.993 INFO:tasks.cephfs.filesystem:scrub status for tag:a29b5fa3-22e4-43b4-aa4d-e09be3350a5e - {'path': '/', 'tag': 'a29b5fa3-22e4-43b4-aa4d-e09be3350a5e', 'options': 'recursive,repair'} 2024-03-23T09:50:18.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:18.978+0000 7f8d0f7fe640 1 -- 172.21.15.47:0/139136251 <== mon.0 v2:172.21.15.47:3300/0 7 ==== log(5 entries from seq 1 at 2024-03-23T09:50:17.832820+0000) v1 ==== 846+0+0 (secure 0 0 0) 0x7f8d1801a2b0 con 0x7f8d10003ef0 2024-03-23T09:50:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: cluster: 2024-03-23T09:50:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: id: f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:50:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: health: HEALTH_OK 2024-03-23T09:50:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: services: 2024-03-23T09:50:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-03-23T09:50:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: mgr: x(active, since 9m), standbys: y 2024-03-23T09:50:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: mds: 1/1 daemons up, 3 standby 2024-03-23T09:50:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-03-23T09:50:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: data: 2024-03-23T09:50:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: volumes: 1/1 healthy 2024-03-23T09:50:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: pools: 3 pools, 97 pgs 2024-03-23T09:50:18.980 INFO:teuthology.orchestra.run.smithi047.stdout: objects: 30 objects, 19 MiB 2024-03-23T09:50:18.980 INFO:teuthology.orchestra.run.smithi047.stdout: usage: 302 MiB used, 720 GiB / 720 GiB avail 2024-03-23T09:50:18.980 INFO:teuthology.orchestra.run.smithi047.stdout: pgs: 97 active+clean 2024-03-23T09:50:18.980 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:18.980 INFO:teuthology.orchestra.run.smithi047.stdout: io: 2024-03-23T09:50:18.980 INFO:teuthology.orchestra.run.smithi047.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 2 op/s wr 2024-03-23T09:50:18.980 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:18.980 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:18.980 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:17.832820+0000 mds.a [INF] scrub queued for path: / 2024-03-23T09:50:18.980 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:17.832824+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-03-23T09:50:18.980 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:17.832828+0000 mds.a [INF] scrub summary: active paths [/] 2024-03-23T09:50:18.980 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:17.833986+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-03-23T09:50:18.980 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:18.484671+0000 mds.a [INF] scrub summary: idle 2024-03-23T09:50:38.749 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:50:38.752 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:50:47.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:47.994 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-23T09:50:48.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.108+0000 7f90e883e640 1 Processor -- start 2024-03-23T09:50:48.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.108+0000 7f90e883e640 1 -- start start 2024-03-23T09:50:48.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.109+0000 7f90e883e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f90e415c990 0x7f90e4160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.109+0000 7f90e883e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e40b8350 0x7f90e40cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.109+0000 7f90e883e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e4161630 con 0x7f90e40cdc40 2024-03-23T09:50:48.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.109+0000 7f90e883e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e40ce0b0 con 0x7f90e415c990 2024-03-23T09:50:48.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.109+0000 7f90e883e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40d1030 con 0x7f90e40b8350 2024-03-23T09:50:48.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.109+0000 7f90e2d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f90e40cdc40 0x7f90e40b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45816/0 (socket says 172.21.15.47:45816) 2024-03-23T09:50:48.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.109+0000 7f90e2d9d640 1 -- 172.21.15.47:0/345807701 learned_addr learned my addr 172.21.15.47:0/345807701 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:48.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544389788 0 0) 0x7f90e4161630 con 0x7f90e40cdc40 2024-03-23T09:50:48.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90d0003520 con 0x7f90e40cdc40 2024-03-23T09:50:48.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e259c640 1 --2- 172.21.15.47:0/345807701 >> v2:172.21.15.175:3300/0 conn(0x7f90e415c990 0x7f90e4160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/345807701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e40b8350 0x7f90e40cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3864668465 0 0) 0x7f90d0003520 con 0x7f90e40cdc40 2024-03-23T09:50:48.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e40b8350 msgr2=0x7f90e40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 --2- 172.21.15.47:0/345807701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e40b8350 0x7f90e40cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 >> v2:172.21.15.175:3300/0 conn(0x7f90e415c990 msgr2=0x7f90e4160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 --2- 172.21.15.47:0/345807701 >> v2:172.21.15.175:3300/0 conn(0x7f90e415c990 0x7f90e4160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e40cfe10 con 0x7f90e40cdc40 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f90dc001670 con 0x7f90e40cdc40 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e259c640 1 --2- 172.21.15.47:0/345807701 >> v2:172.21.15.175:3300/0 conn(0x7f90e415c990 0x7f90e4160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 >> v1:172.21.15.47:6789/0 conn(0x7f90e40cdc40 legacy=0x7f90e40b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 --2- 172.21.15.47:0/345807701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e40b8350 0x7f90d0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 --2- 172.21.15.47:0/345807701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90d0003fa0 0x7f90d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 --2- 172.21.15.47:0/345807701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90d0004d00 0x7f90d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40ce0b0 con 0x7f90d0003fa0 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40d1030 con 0x7f90d0004d00 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d000bb40 con 0x7f90e40b8350 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/345807701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90d0003fa0 0x7f90d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e259c640 1 --2- 172.21.15.47:0/345807701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e40b8350 0x7f90d0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.110+0000 7f90e2d9d640 1 --2- 172.21.15.47:0/345807701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90d0004d00 0x7f90d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.111+0000 7f90e1d9b640 1 -- 172.21.15.47:0/345807701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e40b8350 msgr2=0x7f90d0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.111+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/345807701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e40b8350 0x7f90d0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.111+0000 7f90e1d9b640 1 -- 172.21.15.47:0/345807701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90d0004d00 msgr2=0x7f90d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.111+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/345807701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90d0004d00 0x7f90d0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.111+0000 7f90e1d9b640 1 -- 172.21.15.47:0/345807701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d0003520 con 0x7f90d0003fa0 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.111+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/345807701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90d0003fa0 0x7f90d00047c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00ba30 tx=0x7f90d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab559ea6d6f20e36 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.111+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc032640 con 0x7f90d0003fa0 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.111+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90cc0327a0 con 0x7f90d0003fa0 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.112+0000 7f90e159a640 1 -- 172.21.15.47:0/345807701 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc032b30 con 0x7f90d0003fa0 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.112+0000 7f90e883e640 1 -- 172.21.15.47:0/345807701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90d0003fa0 msgr2=0x7f90d00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.112+0000 7f90e883e640 1 --2- 172.21.15.47:0/345807701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90d0003fa0 0x7f90d00047c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00ba30 tx=0x7f90d00057f0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.112+0000 7f90e2d9d640 1 -- 172.21.15.47:0/345807701 reap_dead start 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.112+0000 7f90e883e640 1 -- 172.21.15.47:0/345807701 shutdown_connections 2024-03-23T09:50:48.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.112+0000 7f90e883e640 1 -- 172.21.15.47:0/345807701 >> 172.21.15.47:0/345807701 conn(0x7f90e40659d0 msgr2=0x7f90e4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.112+0000 7f90e883e640 1 -- 172.21.15.47:0/345807701 shutdown_connections 2024-03-23T09:50:48.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.112+0000 7f90e883e640 1 -- 172.21.15.47:0/345807701 wait complete. 2024-03-23T09:50:48.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.113+0000 7f90e883e640 1 Processor -- start 2024-03-23T09:50:48.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.114+0000 7f90e883e640 1 -- start start 2024-03-23T09:50:48.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.114+0000 7f90e883e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90d0003fa0 0x7f90e40c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.114+0000 7f90e883e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e415ac10 0x7f90e4155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.114+0000 7f90e883e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90e4156230 0x7f90e4159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.114+0000 7f90e883e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40c7d10 con 0x7f90e4156230 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.114+0000 7f90e883e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e41566b0 con 0x7f90e415ac10 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.114+0000 7f90e883e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40c6390 con 0x7f90d0003fa0 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.114+0000 7f90e2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90e4156230 0x7f90e4159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.114+0000 7f90e2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90e4156230 0x7f90e4159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59386/0 (socket says 172.21.15.47:59386) 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.114+0000 7f90e2d9d640 1 -- 172.21.15.47:0/3764869992 learned_addr learned my addr 172.21.15.47:0/3764869992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e415ac10 0x7f90e4155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90e259c640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90d0003fa0 0x7f90e40c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90e2d9d640 1 -- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90d0003fa0 msgr2=0x7f90e40c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90e2d9d640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90d0003fa0 0x7f90e40c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90e2d9d640 1 -- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e415ac10 msgr2=0x7f90e4155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90e2d9d640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e415ac10 0x7f90e4155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90e2d9d640 1 -- 172.21.15.47:0/3764869992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e40c65e0 con 0x7f90e4156230 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90e259c640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90d0003fa0 0x7f90e40c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90e2d9d640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90e4156230 0x7f90e4159690 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f90e40cd210 tx=0x7f90dc007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90daffd640 1 -- 172.21.15.47:0/3764869992 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc00f680 con 0x7f90e4156230 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90daffd640 1 -- 172.21.15.47:0/3764869992 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc00f820 con 0x7f90e4156230 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90e883e640 1 -- 172.21.15.47:0/3764869992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e40c68c0 con 0x7f90e4156230 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90daffd640 1 -- 172.21.15.47:0/3764869992 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc00fbd0 con 0x7f90e4156230 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.115+0000 7f90e883e640 1 -- 172.21.15.47:0/3764869992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e40c6e80 con 0x7f90e4156230 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.117+0000 7f90daffd640 1 -- 172.21.15.47:0/3764869992 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f90dc00fd30 con 0x7f90e4156230 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.117+0000 7f90daffd640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90c406ac30 0x7f90c406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.117+0000 7f90daffd640 1 -- 172.21.15.47:0/3764869992 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90dc089190 con 0x7f90e4156230 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.117+0000 7f90e259c640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90c406ac30 0x7f90c406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.118+0000 7f90eb3b1740 1 Processor -- start 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.118+0000 7f90e259c640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90c406ac30 0x7f90c406d0f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f90d4007520 tx=0x7f90d400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.118+0000 7f90eb3b1740 1 -- start start 2024-03-23T09:50:48.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90eb3b1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994a0830 0x55b8994a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90eb3b1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b899470850 0x55b89949daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90eb3b1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994743b0 0x55b89949dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90eb3b1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8993e0dd0 con 0x55b8994743b0 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90eb3b1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8993de560 con 0x55b899470850 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90eb3b1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8993d9e50 con 0x55b8994a0830 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90e2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994743b0 0x55b89949dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90e2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994743b0 0x55b89949dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59390/0 (socket says 172.21.15.47:59390) 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90e2d9d640 1 -- 172.21.15.47:0/1680733202 learned_addr learned my addr 172.21.15.47:0/1680733202 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90e259c640 1 --2- 172.21.15.47:0/1680733202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994a0830 0x55b8994a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/1680733202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b899470850 0x55b89949daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90e2d9d640 1 -- 172.21.15.47:0/1680733202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994a0830 msgr2=0x55b8994a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90e2d9d640 1 --2- 172.21.15.47:0/1680733202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994a0830 0x55b8994a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90e2d9d640 1 -- 172.21.15.47:0/1680733202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b899470850 msgr2=0x55b89949daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90e2d9d640 1 --2- 172.21.15.47:0/1680733202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b899470850 0x55b89949daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90e2d9d640 1 -- 172.21.15.47:0/1680733202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8993a1470 con 0x55b8994743b0 2024-03-23T09:50:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/1680733202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b899470850 0x55b89949daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.119+0000 7f90e2d9d640 1 --2- 172.21.15.47:0/1680733202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994743b0 0x55b89949dfe0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f90dc057f60 tx=0x7f90dc0579f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4086d3a4a547a3d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.120+0000 7f90c3fff640 1 -- 172.21.15.47:0/1680733202 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc05be00 con 0x55b8994743b0 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.120+0000 7f90c3fff640 1 -- 172.21.15.47:0/1680733202 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc0603c0 con 0x55b8994743b0 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.120+0000 7f90c3fff640 1 -- 172.21.15.47:0/1680733202 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc056370 con 0x55b8994743b0 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.120+0000 7f90eb3b1740 1 -- 172.21.15.47:0/1680733202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994743b0 msgr2=0x55b89949dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.120+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/1680733202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994743b0 0x55b89949dfe0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f90dc057f60 tx=0x7f90dc0579f0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.120+0000 7f90eb3b1740 1 -- 172.21.15.47:0/1680733202 shutdown_connections 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.120+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/1680733202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994743b0 0x55b89949dfe0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.120+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/1680733202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b899470850 0x55b89949daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.120+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/1680733202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994a0830 0x55b8994a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.120+0000 7f90eb3b1740 1 -- 172.21.15.47:0/1680733202 >> 172.21.15.47:0/1680733202 conn(0x55b899472ac0 msgr2=0x55b8994bb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.120+0000 7f90eb3b1740 1 -- 172.21.15.47:0/1680733202 shutdown_connections 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.120+0000 7f90eb3b1740 1 -- 172.21.15.47:0/1680733202 wait complete. 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.121+0000 7f90eb3b1740 1 -- start start 2024-03-23T09:50:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.121+0000 7f90eb3b1740 1 Processor -- start 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90eb3b1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994743b0 0x55b8994a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90eb3b1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994a0830 0x55b8994a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90eb3b1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994cae30 0x55b89949a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90eb3b1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b899398280 con 0x55b8994cae30 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90eb3b1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b899397f20 con 0x55b8994743b0 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90eb3b1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b89935d3c0 con 0x55b8994a0830 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90eb3b1740 10 client.0 _ll_register_callbacks cb 0x55b8993af380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90e2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994cae30 0x55b89949a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90e2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994cae30 0x55b89949a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59400/0 (socket says 172.21.15.47:59400) 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90e2d9d640 1 -- 172.21.15.47:0/3564492031 learned_addr learned my addr 172.21.15.47:0/3564492031 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90e259c640 1 --2- 172.21.15.47:0/3564492031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994743b0 0x55b8994a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/3564492031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994a0830 0x55b8994a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90e2d9d640 1 -- 172.21.15.47:0/3564492031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994a0830 msgr2=0x55b8994a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90e2d9d640 1 --2- 172.21.15.47:0/3564492031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994a0830 0x55b8994a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90e2d9d640 1 -- 172.21.15.47:0/3564492031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994743b0 msgr2=0x55b8994a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90e2d9d640 1 --2- 172.21.15.47:0/3564492031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994743b0 0x55b8994a2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.122+0000 7f90e2d9d640 1 -- 172.21.15.47:0/3564492031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e40c6e80 con 0x55b8994cae30 2024-03-23T09:50:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90e2d9d640 1 --2- 172.21.15.47:0/3564492031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994cae30 0x55b89949a920 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f90dc057f60 tx=0x7f90dc015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339915fb4080c56d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:50:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90c17fa640 1 -- 172.21.15.47:0/3564492031 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc05be00 con 0x55b8994cae30 2024-03-23T09:50:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90c17fa640 1 -- 172.21.15.47:0/3564492031 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90dc056370 con 0x55b8994cae30 2024-03-23T09:50:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90eb3b1740 1 -- 172.21.15.47:0/3564492031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b899403690 con 0x55b8994cae30 2024-03-23T09:50:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90c17fa640 1 -- 172.21.15.47:0/3564492031 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90dc06fbe0 con 0x55b8994cae30 2024-03-23T09:50:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90eb3b1740 10 client.8898 fetch_fsmap learned FSMap version 65 2024-03-23T09:50:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90eb3b1740 1 -- 172.21.15.47:0/3564492031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55b8993d84d0 con 0x55b8994cae30 2024-03-23T09:50:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90c17fa640 1 -- 172.21.15.47:0/3564492031 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b899403690 con 0x55b8994cae30 2024-03-23T09:50:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90c17fa640 1 -- 172.21.15.47:0/3564492031 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f90dc076d20 con 0x55b8994cae30 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90eb3b1740 10 client.8898 fetch_fsmap finished waiting for FSMap version 65 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90eb3b1740 10 client.8898 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90eb3b1740 20 client.8898 populate_metadata read hostname 'smithi047' 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/3564492031 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b8994cb2b0 0x55b89949d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90eb3b1740 4 client.8898 mds_command: new command op to 8484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.123+0000 7f90eb3b1740 1 -- 172.21.15.47:0/3564492031 --> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8993cbeb0 con 0x55b8994cb2b0 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.124+0000 7f90e259c640 1 --2- 172.21.15.47:0/3564492031 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b8994cb2b0 0x55b89949d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.124+0000 7f90e259c640 1 --2- 172.21.15.47:0/3564492031 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b8994cb2b0 0x55b89949d870 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32259830bbbfc7e7 server_cookie=b89b169239644866 in_seq=0 out_seq=0 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.124+0000 7f90c17fa640 10 client.8898 ms_handle_connect on v2:172.21.15.47:6833/2388298413 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.124+0000 7f90c17fa640 1 -- 172.21.15.47:0/3564492031 <== mds.0 v2:172.21.15.47:6833/2388298413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b8993cbeb0 con 0x55b8994cb2b0 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.124+0000 7f90c17fa640 10 client.8898 handle_command_reply: tid=0 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.124+0000 7f90eb3b1740 1 client.8898 shutdown 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.125+0000 7f90eb3b1740 1 -- 172.21.15.47:0/3564492031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994cae30 msgr2=0x55b89949a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.125+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/3564492031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994cae30 0x55b89949a920 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f90dc057f60 tx=0x7f90dc015040 comp rx=0 tx=0).stop 2024-03-23T09:50:48.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.125+0000 7f90eb3b1740 1 -- 172.21.15.47:0/3564492031 shutdown_connections 2024-03-23T09:50:48.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.125+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/3564492031 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b8994cb2b0 0x55b89949d870 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.125+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/3564492031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994cae30 0x55b89949a920 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.125+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/3564492031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994a0830 0x55b8994a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.125+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/3564492031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994743b0 0x55b8994a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.125+0000 7f90c17fa640 0 client.8898 ms_handle_reset on v2:172.21.15.47:6833/2388298413 2024-03-23T09:50:48.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.125+0000 7f90eb3b1740 1 -- 172.21.15.47:0/3564492031 >> 172.21.15.47:0/3564492031 conn(0x55b899470850 msgr2=0x55b8994a43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.125+0000 7f90eb3b1740 1 -- 172.21.15.47:0/3564492031 shutdown_connections 2024-03-23T09:50:48.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.125+0000 7f90eb3b1740 1 -- 172.21.15.47:0/3564492031 wait complete. 2024-03-23T09:50:48.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.125+0000 7f90eb3b1740 20 client.8898 trim_cache size 0 max 16384 2024-03-23T09:50:48.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.139+0000 7f90eb3b1740 1 Processor -- start 2024-03-23T09:50:48.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.139+0000 7f90eb3b1740 1 -- start start 2024-03-23T09:50:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90eb3b1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994743b0 0x55b899471610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90eb3b1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 0x55b8994a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90eb3b1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994cae30 0x55b8994beb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90eb3b1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8994a40d0 con 0x55b8994a0830 2024-03-23T09:50:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90eb3b1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89949e710 con 0x55b8994743b0 2024-03-23T09:50:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90eb3b1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b89949ce00 con 0x55b8994cae30 2024-03-23T09:50:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90e1d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 0x55b8994a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90e1d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 0x55b8994a43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59416/0 (socket says 172.21.15.47:59416) 2024-03-23T09:50:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90e1d9b640 1 -- 172.21.15.47:0/2649403892 learned_addr learned my addr 172.21.15.47:0/2649403892 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90e2d9d640 1 --2- 172.21.15.47:0/2649403892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994cae30 0x55b8994beb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90e259c640 1 --2- 172.21.15.47:0/2649403892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994743b0 0x55b899471610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90e1d9b640 1 -- 172.21.15.47:0/2649403892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994cae30 msgr2=0x55b8994beb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/2649403892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994cae30 0x55b8994beb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90e1d9b640 1 -- 172.21.15.47:0/2649403892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994743b0 msgr2=0x55b899471610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/2649403892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994743b0 0x55b899471610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90e1d9b640 1 -- 172.21.15.47:0/2649403892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b89940c9e0 con 0x55b8994a0830 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90e2d9d640 1 --2- 172.21.15.47:0/2649403892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994cae30 0x55b8994beb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.140+0000 7f90e259c640 1 --2- 172.21.15.47:0/2649403892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994743b0 0x55b899471610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.141+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/2649403892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 0x55b8994a43c0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f90cc006b10 tx=0x7f90cc032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe703443f731bec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.141+0000 7f90d8ff9640 1 -- 172.21.15.47:0/2649403892 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc040040 con 0x55b8994a0830 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.141+0000 7f90d8ff9640 1 -- 172.21.15.47:0/2649403892 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90cc004020 con 0x55b8994a0830 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.141+0000 7f90d8ff9640 1 -- 172.21.15.47:0/2649403892 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc004390 con 0x55b8994a0830 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.141+0000 7f90eb3b1740 1 -- 172.21.15.47:0/2649403892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 msgr2=0x55b8994a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.141+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/2649403892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 0x55b8994a43c0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f90cc006b10 tx=0x7f90cc032000 comp rx=0 tx=0).stop 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.141+0000 7f90eb3b1740 1 -- 172.21.15.47:0/2649403892 shutdown_connections 2024-03-23T09:50:48.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.141+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/2649403892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994cae30 0x55b8994beb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.141+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/2649403892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 0x55b8994a43c0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.141+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/2649403892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994743b0 0x55b899471610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.141+0000 7f90eb3b1740 1 -- 172.21.15.47:0/2649403892 >> 172.21.15.47:0/2649403892 conn(0x55b899470850 msgr2=0x55b8994cdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.141+0000 7f90eb3b1740 1 -- 172.21.15.47:0/2649403892 shutdown_connections 2024-03-23T09:50:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.141+0000 7f90eb3b1740 1 -- 172.21.15.47:0/2649403892 wait complete. 2024-03-23T09:50:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.142+0000 7f90eb3b1740 1 -- start start 2024-03-23T09:50:48.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.142+0000 7f90eb3b1740 1 Processor -- start 2024-03-23T09:50:48.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90eb3b1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994743b0 0x55b8994bcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90eb3b1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 0x55b8995240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90eb3b1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994cae30 0x55b899499ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90eb3b1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b899497350 con 0x55b8994a0830 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90eb3b1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8994bd4b0 con 0x55b8994cae30 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90eb3b1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b899499a10 con 0x55b8994743b0 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90eb3b1740 10 client.0 _ll_register_callbacks cb 0x7f90dc089050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90e1d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 0x55b8995240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90e1d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 0x55b8995240b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59422/0 (socket says 172.21.15.47:59422) 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90e1d9b640 1 -- 172.21.15.47:0/2946715118 learned_addr learned my addr 172.21.15.47:0/2946715118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90e2d9d640 1 --2- 172.21.15.47:0/2946715118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994cae30 0x55b899499ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90e1d9b640 1 -- 172.21.15.47:0/2946715118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994743b0 msgr2=0x55b8994bcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/2946715118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994743b0 0x55b8994bcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90e1d9b640 1 -- 172.21.15.47:0/2946715118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994cae30 msgr2=0x55b899499ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/2946715118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994cae30 0x55b899499ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90e1d9b640 1 -- 172.21.15.47:0/2946715118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d0003520 con 0x55b8994a0830 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90e1d9b640 1 --2- 172.21.15.47:0/2946715118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 0x55b8995240b0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00b850 tx=0x7f90cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff93902463a3874 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.143+0000 7f90c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90c17fa640 1 -- 172.21.15.47:0/2946715118 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc003810 con 0x55b8994a0830 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90c17fa640 1 -- 172.21.15.47:0/2946715118 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90cc040040 con 0x55b8994a0830 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90eb3b1740 1 -- 172.21.15.47:0/2946715118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90dc0603c0 con 0x55b8994a0830 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90c17fa640 1 -- 172.21.15.47:0/2946715118 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90cc003970 con 0x55b8994a0830 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90eb3b1740 10 client.8904 fetch_fsmap learned FSMap version 65 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90eb3b1740 1 -- 172.21.15.47:0/2946715118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55b89937a5c0 con 0x55b8994a0830 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90c17fa640 1 -- 172.21.15.47:0/2946715118 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc0603c0 con 0x55b8994a0830 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90c17fa640 1 -- 172.21.15.47:0/2946715118 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f90cc003ad0 con 0x55b8994a0830 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90eb3b1740 10 client.8904 fetch_fsmap finished waiting for FSMap version 65 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90eb3b1740 10 client.8904 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-03-23T09:50:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90eb3b1740 20 client.8904 populate_metadata read hostname 'smithi047' 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/2946715118 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b89949a3e0 0x55b8994a4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90eb3b1740 4 client.8904 mds_command: new command op to 8484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90eb3b1740 1 -- 172.21.15.47:0/2946715118 --> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b8990cd4e0 con 0x55b89949a3e0 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90e259c640 1 --2- 172.21.15.47:0/2946715118 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b89949a3e0 0x55b8994a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90c17fa640 10 client.8904 ms_handle_connect on v2:172.21.15.47:6833/2388298413 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.144+0000 7f90e259c640 1 --2- 172.21.15.47:0/2946715118 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b89949a3e0 0x55b8994a4210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c1c1f930478640 server_cookie=4fb5fb85aa515d73 in_seq=0 out_seq=0 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90c17fa640 1 -- 172.21.15.47:0/2946715118 <== mds.0 v2:172.21.15.47:6833/2388298413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b8990cd4e0 con 0x55b89949a3e0 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90c17fa640 10 client.8904 handle_command_reply: tid=0 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90eb3b1740 1 client.8904 shutdown 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90eb3b1740 1 -- 172.21.15.47:0/2946715118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 msgr2=0x55b8995240b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/2946715118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 0x55b8995240b0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00b850 tx=0x7f90cc0046c0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90eb3b1740 1 -- 172.21.15.47:0/2946715118 shutdown_connections 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/2946715118 >> [v2:172.21.15.47:6833/2388298413,v1:172.21.15.47:6835/2388298413] conn(0x55b89949a3e0 0x55b8994a4210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/2946715118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8994cae30 0x55b899499ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/2946715118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8994a0830 0x55b8995240b0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90c17fa640 0 client.8904 ms_handle_reset on v2:172.21.15.47:6833/2388298413 2024-03-23T09:50:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90eb3b1740 1 --2- 172.21.15.47:0/2946715118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b8994743b0 0x55b8994bcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90eb3b1740 1 -- 172.21.15.47:0/2946715118 >> 172.21.15.47:0/2946715118 conn(0x55b899470850 msgr2=0x55b8994cdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90eb3b1740 1 -- 172.21.15.47:0/2946715118 shutdown_connections 2024-03-23T09:50:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90eb3b1740 1 -- 172.21.15.47:0/2946715118 wait complete. 2024-03-23T09:50:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90eb3b1740 20 client.8904 trim_cache size 0 max 16384 2024-03-23T09:50:48.152 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:50:48.152 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-23T09:50:48.152 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-23T09:50:48.152 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:50:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90e883e640 1 -- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90c406ac30 msgr2=0x7f90c406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90e883e640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90c406ac30 0x7f90c406d0f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f90d4007520 tx=0x7f90d400b040 comp rx=0 tx=0).stop 2024-03-23T09:50:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90e883e640 1 -- 172.21.15.47:0/3764869992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90e4156230 msgr2=0x7f90e4159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.145+0000 7f90e883e640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90e4156230 0x7f90e4159690 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f90e40cd210 tx=0x7f90dc007430 comp rx=0 tx=0).stop 2024-03-23T09:50:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.146+0000 7f90e883e640 1 -- 172.21.15.47:0/3764869992 shutdown_connections 2024-03-23T09:50:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.146+0000 7f90e883e640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90c406ac30 0x7f90c406d0f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.146+0000 7f90e883e640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90e4156230 0x7f90e4159690 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.146+0000 7f90e883e640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e415ac10 0x7f90e4155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.146+0000 7f90e883e640 1 --2- 172.21.15.47:0/3764869992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90d0003fa0 0x7f90e40c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.146+0000 7f90e883e640 1 -- 172.21.15.47:0/3764869992 >> 172.21.15.47:0/3764869992 conn(0x7f90e40659d0 msgr2=0x7f90e40b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.146+0000 7f90e883e640 1 -- 172.21.15.47:0/3764869992 shutdown_connections 2024-03-23T09:50:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.146+0000 7f90e883e640 1 -- 172.21.15.47:0/3764869992 wait complete. 2024-03-23T09:50:48.157 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T09:50:48.259 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:50:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.359+0000 7f6e985fd640 1 Processor -- start 2024-03-23T09:50:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.359+0000 7f6e985fd640 1 -- start start 2024-03-23T09:50:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.360+0000 7f6e985fd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e900cf210 0x7f6e900cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.360+0000 7f6e985fd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e900b5a90 0x7f6e900b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.360+0000 7f6e985fd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e900cfbd0 con 0x7f6e9015cd00 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.360+0000 7f6e985fd640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9015d0e0 con 0x7f6e900cf210 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.360+0000 7f6e985fd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900d1540 con 0x7f6e900b5a90 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.360+0000 7f6e96399640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6e9015cd00 0x7f6e900d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45830/0 (socket says 172.21.15.47:45830) 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.360+0000 7f6e96399640 1 -- 172.21.15.47:0/1785736671 learned_addr learned my addr 172.21.15.47:0/1785736671 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e96b9a640 1 --2- 172.21.15.47:0/1785736671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e900b5a90 0x7f6e900b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021937105 0 0) 0x7f6e900cfbd0 con 0x7f6e9015cd00 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e78003520 con 0x7f6e9015cd00 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95b98640 1 --2- 172.21.15.47:0/1785736671 >> v2:172.21.15.175:3300/0 conn(0x7f6e900cf210 0x7f6e900cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2939442877 0 0) 0x7f6e78003520 con 0x7f6e9015cd00 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e900b5a90 msgr2=0x7f6e900b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 --2- 172.21.15.47:0/1785736671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e900b5a90 0x7f6e900b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 >> v2:172.21.15.175:3300/0 conn(0x7f6e900cf210 msgr2=0x7f6e900cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 --2- 172.21.15.47:0/1785736671 >> v2:172.21.15.175:3300/0 conn(0x7f6e900cf210 0x7f6e900cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e900d1820 con 0x7f6e9015cd00 2024-03-23T09:50:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f6e88001670 con 0x7f6e9015cd00 2024-03-23T09:50:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95b98640 1 --2- 172.21.15.47:0/1785736671 >> v2:172.21.15.175:3300/0 conn(0x7f6e900cf210 0x7f6e900cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 >> v1:172.21.15.47:6789/0 conn(0x7f6e9015cd00 legacy=0x7f6e900d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 --2- 172.21.15.47:0/1785736671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e78003970 0x7f6e78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 --2- 172.21.15.47:0/1785736671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e900b5a90 0x7f6e780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 --2- 172.21.15.47:0/1785736671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e78004d00 0x7f6e78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9015d0e0 con 0x7f6e78003970 2024-03-23T09:50:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900d1540 con 0x7f6e78004d00 2024-03-23T09:50:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7800ba30 con 0x7f6e900b5a90 2024-03-23T09:50:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e96399640 1 --2- 172.21.15.47:0/1785736671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e78003970 0x7f6e78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e96b9a640 1 --2- 172.21.15.47:0/1785736671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e78004d00 0x7f6e78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e95b98640 1 --2- 172.21.15.47:0/1785736671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e900b5a90 0x7f6e780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e96399640 1 -- 172.21.15.47:0/1785736671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e900b5a90 msgr2=0x7f6e780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e96399640 1 --2- 172.21.15.47:0/1785736671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e900b5a90 0x7f6e780047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e96399640 1 -- 172.21.15.47:0/1785736671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e78004d00 msgr2=0x7f6e78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.361+0000 7f6e96399640 1 --2- 172.21.15.47:0/1785736671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e78004d00 0x7f6e78005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.362+0000 7f6e96399640 1 -- 172.21.15.47:0/1785736671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e78003520 con 0x7f6e78003970 2024-03-23T09:50:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.362+0000 7f6e96399640 1 --2- 172.21.15.47:0/1785736671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e78003970 0x7f6e78003d90 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c240 tx=0x7f6e780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396e981431cb31a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.362+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8802cb00 con 0x7f6e78003970 2024-03-23T09:50:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.362+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8802cc60 con 0x7f6e78003970 2024-03-23T09:50:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.362+0000 7f6e95397640 1 -- 172.21.15.47:0/1785736671 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88005b10 con 0x7f6e78003970 2024-03-23T09:50:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.363+0000 7f6e985fd640 1 -- 172.21.15.47:0/1785736671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e78003970 msgr2=0x7f6e78003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.363+0000 7f6e985fd640 1 --2- 172.21.15.47:0/1785736671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e78003970 0x7f6e78003d90 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c240 tx=0x7f6e780057f0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.363+0000 7f6e96b9a640 1 -- 172.21.15.47:0/1785736671 reap_dead start 2024-03-23T09:50:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.363+0000 7f6e985fd640 1 -- 172.21.15.47:0/1785736671 shutdown_connections 2024-03-23T09:50:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.363+0000 7f6e985fd640 1 -- 172.21.15.47:0/1785736671 >> 172.21.15.47:0/1785736671 conn(0x7f6e9006b160 msgr2=0x7f6e9006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.363+0000 7f6e985fd640 1 -- 172.21.15.47:0/1785736671 shutdown_connections 2024-03-23T09:50:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.363+0000 7f6e985fd640 1 -- 172.21.15.47:0/1785736671 wait complete. 2024-03-23T09:50:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.364+0000 7f6e985fd640 1 Processor -- start 2024-03-23T09:50:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.364+0000 7f6e985fd640 1 -- start start 2024-03-23T09:50:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e985fd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e78003970 0x7f6e901f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e985fd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e900c9f80 0x7f6e900b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e985fd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e900cc1f0 0x7f6e901446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e985fd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900ccb90 con 0x7f6e78003970 2024-03-23T09:50:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e985fd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900c7fb0 con 0x7f6e900c9f80 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e985fd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900c7680 con 0x7f6e900cc1f0 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e96399640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e78003970 0x7f6e901f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e96399640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e78003970 0x7f6e901f1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59432/0 (socket says 172.21.15.47:59432) 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e96399640 1 -- 172.21.15.47:0/194106385 learned_addr learned my addr 172.21.15.47:0/194106385 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e96b9a640 1 --2- 172.21.15.47:0/194106385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e900cc1f0 0x7f6e901446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e96399640 1 -- 172.21.15.47:0/194106385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e900cc1f0 msgr2=0x7f6e901446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e96399640 1 --2- 172.21.15.47:0/194106385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e900cc1f0 0x7f6e901446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e96399640 1 -- 172.21.15.47:0/194106385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e900c9f80 msgr2=0x7f6e900b27c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e96399640 1 --2- 172.21.15.47:0/194106385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e900c9f80 0x7f6e900b27c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.365+0000 7f6e96399640 1 -- 172.21.15.47:0/194106385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e900c7960 con 0x7f6e78003970 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.366+0000 7f6e96399640 1 --2- 172.21.15.47:0/194106385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e78003970 0x7f6e901f1f20 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f6e8802c4a0 tx=0x7f6e88001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.366+0000 7f6e8effd640 1 -- 172.21.15.47:0/194106385 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88005d40 con 0x7f6e78003970 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.366+0000 7f6e8effd640 1 -- 172.21.15.47:0/194106385 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e88007070 con 0x7f6e78003970 2024-03-23T09:50:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.366+0000 7f6e985fd640 1 -- 172.21.15.47:0/194106385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e900c7c40 con 0x7f6e78003970 2024-03-23T09:50:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.366+0000 7f6e8effd640 1 -- 172.21.15.47:0/194106385 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88005ea0 con 0x7f6e78003970 2024-03-23T09:50:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.366+0000 7f6e985fd640 1 -- 172.21.15.47:0/194106385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e900cc670 con 0x7f6e78003970 2024-03-23T09:50:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.367+0000 7f6e985fd640 1 -- 172.21.15.47:0/194106385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e90072360 con 0x7f6e78003970 2024-03-23T09:50:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.367+0000 7f6e8effd640 1 -- 172.21.15.47:0/194106385 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6e88032070 con 0x7f6e78003970 2024-03-23T09:50:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.367+0000 7f6e8effd640 1 --2- 172.21.15.47:0/194106385 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e7c06ac30 0x7f6e7c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.368+0000 7f6e8effd640 1 -- 172.21.15.47:0/194106385 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e880aa720 con 0x7f6e78003970 2024-03-23T09:50:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.369+0000 7f6e95b98640 1 --2- 172.21.15.47:0/194106385 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e7c06ac30 0x7f6e7c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.370+0000 7f6e95b98640 1 --2- 172.21.15.47:0/194106385 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e7c06ac30 0x7f6e7c06d0f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6e900b2730 tx=0x7f6e84008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.372+0000 7f6e8effd640 1 -- 172.21.15.47:0/194106385 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6e88075900 con 0x7f6e78003970 2024-03-23T09:50:48.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.587+0000 7f6e985fd640 1 -- 172.21.15.47:0/194106385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6e900cc950 con 0x7f6e78003970 2024-03-23T09:50:48.588 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:50:48.588+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:50:48.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.597+0000 7f6e8effd640 1 -- 172.21.15.47:0/194106385 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6e900cc950 con 0x7f6e78003970 2024-03-23T09:50:48.599 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:50:48.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.600+0000 7f6e985fd640 1 -- 172.21.15.47:0/194106385 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e7c06ac30 msgr2=0x7f6e7c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.600+0000 7f6e985fd640 1 --2- 172.21.15.47:0/194106385 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e7c06ac30 0x7f6e7c06d0f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6e900b2730 tx=0x7f6e84008040 comp rx=0 tx=0).stop 2024-03-23T09:50:48.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.600+0000 7f6e985fd640 1 -- 172.21.15.47:0/194106385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e78003970 msgr2=0x7f6e901f1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.600+0000 7f6e985fd640 1 --2- 172.21.15.47:0/194106385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e78003970 0x7f6e901f1f20 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f6e8802c4a0 tx=0x7f6e88001a00 comp rx=0 tx=0).stop 2024-03-23T09:50:48.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.600+0000 7f6e985fd640 1 -- 172.21.15.47:0/194106385 shutdown_connections 2024-03-23T09:50:48.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.600+0000 7f6e985fd640 1 --2- 172.21.15.47:0/194106385 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e7c06ac30 0x7f6e7c06d0f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.600+0000 7f6e985fd640 1 --2- 172.21.15.47:0/194106385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e900cc1f0 0x7f6e901446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.600+0000 7f6e985fd640 1 --2- 172.21.15.47:0/194106385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e900c9f80 0x7f6e900b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.600+0000 7f6e985fd640 1 --2- 172.21.15.47:0/194106385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e78003970 0x7f6e901f1f20 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.600+0000 7f6e985fd640 1 -- 172.21.15.47:0/194106385 >> 172.21.15.47:0/194106385 conn(0x7f6e9006b160 msgr2=0x7f6e9006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.600+0000 7f6e985fd640 1 -- 172.21.15.47:0/194106385 shutdown_connections 2024-03-23T09:50:48.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.600+0000 7f6e985fd640 1 -- 172.21.15.47:0/194106385 wait complete. 2024-03-23T09:50:48.602 INFO:tasks.ceph.mds.a.smithi047.stderr: -6155> 2024-03-23T09:49:44.462+0000 7f499437d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:50:48.612 DEBUG:teuthology.orchestra.run.smithi047:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-03-23T09:50:48.631 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-03-23T09:50:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.653+0000 7f7d21c3d280 1 -- start start 2024-03-23T09:50:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.653+0000 7f7d21c3d280 1 Processor -- start 2024-03-23T09:50:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.653+0000 7f7d21c3d280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294c282800 0x56294c1adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.653+0000 7f7d21c3d280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c113c00 0x56294c1acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.653+0000 7f7d21c3d280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c283c00 0x56294c1ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.653+0000 7f7d21c3d280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c143e00 con 0x56294c113c00 2024-03-23T09:50:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.653+0000 7f7d21c3d280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c1a4a00 con 0x56294c283c00 2024-03-23T09:50:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.653+0000 7f7d21c3d280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x56294c1a4280 con 0x56294c282800 2024-03-23T09:50:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.653+0000 7f7d1f4a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c113c00 0x56294c1acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.653+0000 7f7d1f4a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c113c00 0x56294c1acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59480/0 (socket says 172.21.15.47:59480) 2024-03-23T09:50:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.653+0000 7f7d1f4a6640 1 -- 172.21.15.47:0/1921467346 learned_addr learned my addr 172.21.15.47:0/1921467346 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d1fca7640 1 --2- 172.21.15.47:0/1921467346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294c282800 0x56294c1adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d204a8640 1 --2- 172.21.15.47:0/1921467346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c283c00 0x56294c1ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d1f4a6640 1 -- 172.21.15.47:0/1921467346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294c282800 msgr2=0x56294c1adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d1f4a6640 1 --2- 172.21.15.47:0/1921467346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294c282800 0x56294c1adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d1f4a6640 1 -- 172.21.15.47:0/1921467346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c283c00 msgr2=0x56294c1ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d1f4a6640 1 --2- 172.21.15.47:0/1921467346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c283c00 0x56294c1ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d1f4a6640 1 -- 172.21.15.47:0/1921467346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56294c1384e0 con 0x56294c113c00 2024-03-23T09:50:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d204a8640 1 --2- 172.21.15.47:0/1921467346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c283c00 0x56294c1ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:48.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d1f4a6640 1 --2- 172.21.15.47:0/1921467346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c113c00 0x56294c1acb00 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x56294cf46d80 tx=0x56294cf1eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa41bcd2daa8ee2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d1d4a2640 1 -- 172.21.15.47:0/1921467346 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56294c1d7080 con 0x56294c113c00 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d1d4a2640 1 -- 172.21.15.47:0/1921467346 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56294c172900 con 0x56294c113c00 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d1d4a2640 1 -- 172.21.15.47:0/1921467346 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56294c1d7600 con 0x56294c113c00 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d21c3d280 1 -- 172.21.15.47:0/1921467346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56294c1381a0 con 0x56294c113c00 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d21c3d280 1 -- 172.21.15.47:0/1921467346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56294cfee000 con 0x56294c113c00 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d21c3d280 4 waiting for MDS map... 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d1d4a2640 1 -- 172.21.15.47:0/1921467346 <== mon.0 v2:172.21.15.47:3300/0 4 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x56294c13e380 con 0x56294c113c00 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d1d4a2640 1 -- 172.21.15.47:0/1921467346 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 66) v1 ==== 2573+0+0 (secure 0 0 0) 0x56294c164f00 con 0x56294c113c00 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d21c3d280 4 Got MDS map 66 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d21c3d280 10 main: JournalTool::main 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.654+0000 7f7d21c3d280 4 main: JournalTool: connecting to RADOS... 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.655+0000 7f7d21c3d280 1 Processor -- start 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.655+0000 7f7d21c3d280 1 -- start start 2024-03-23T09:50:48.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.655+0000 7f7d21c3d280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294c282000 0x56294c1ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d21c3d280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294cfe0800 0x56294c1ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d21c3d280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294cfe0c00 0x56294c1ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d21c3d280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c1a4500 con 0x56294cfe0c00 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d21c3d280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c1a43c0 con 0x56294cfe0800 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d21c3d280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x56294c1a4a00 con 0x56294c282000 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1f4a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294cfe0c00 0x56294c1ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1f4a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294cfe0c00 0x56294c1ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59484/0 (socket says 172.21.15.47:59484) 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1f4a6640 1 -- 172.21.15.47:0/3757896338 learned_addr learned my addr 172.21.15.47:0/3757896338 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1fca7640 1 --2- 172.21.15.47:0/3757896338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294cfe0800 0x56294c1ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d204a8640 1 --2- 172.21.15.47:0/3757896338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294c282000 0x56294c1ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1f4a6640 1 -- 172.21.15.47:0/3757896338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294c282000 msgr2=0x56294c1ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1f4a6640 1 --2- 172.21.15.47:0/3757896338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294c282000 0x56294c1ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1f4a6640 1 -- 172.21.15.47:0/3757896338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294cfe0800 msgr2=0x56294c1ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1f4a6640 1 --2- 172.21.15.47:0/3757896338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294cfe0800 0x56294c1ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1f4a6640 1 -- 172.21.15.47:0/3757896338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56294cfee1a0 con 0x56294cfe0c00 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1fca7640 1 --2- 172.21.15.47:0/3757896338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294cfe0800 0x56294c1ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1f4a6640 1 --2- 172.21.15.47:0/3757896338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294cfe0c00 0x56294c1ae680 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x56294cf47170 tx=0x56294cf1efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48c783932234593 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1b49e640 1 -- 172.21.15.47:0/3757896338 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56294c1d71e0 con 0x56294cfe0c00 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1b49e640 1 -- 172.21.15.47:0/3757896338 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56294cfe6300 con 0x56294cfe0c00 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.656+0000 7f7d1b49e640 1 -- 172.21.15.47:0/3757896338 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56294c1d7080 con 0x56294cfe0c00 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.657+0000 7f7d21c3d280 1 -- 172.21.15.47:0/3757896338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294cfe0c00 msgr2=0x56294c1ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.657+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/3757896338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294cfe0c00 0x56294c1ae680 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x56294cf47170 tx=0x56294cf1efc0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.657+0000 7f7d21c3d280 1 -- 172.21.15.47:0/3757896338 shutdown_connections 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.657+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/3757896338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294cfe0c00 0x56294c1ae680 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.657+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/3757896338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294cfe0800 0x56294c1ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.657+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/3757896338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294c282000 0x56294c1ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.657+0000 7f7d21c3d280 1 -- 172.21.15.47:0/3757896338 >> 172.21.15.47:0/3757896338 conn(0x56294c282400 msgr2=0x56294c1fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.657+0000 7f7d21c3d280 1 -- 172.21.15.47:0/3757896338 shutdown_connections 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.657+0000 7f7d21c3d280 1 -- 172.21.15.47:0/3757896338 wait complete. 2024-03-23T09:50:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.657+0000 7f7d21c3d280 1 Processor -- start 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.657+0000 7f7d21c3d280 1 -- start start 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d21c3d280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c282400 0x56294c1af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d21c3d280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294cfe1800 0x56294c1af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d21c3d280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294cfe1400 0x56294c1aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d21c3d280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c1a4640 con 0x56294c282400 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d21c3d280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c1a4780 con 0x56294cfe1400 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d21c3d280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x56294c1a4140 con 0x56294cfe1800 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d204a8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c282400 0x56294c1af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d204a8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c282400 0x56294c1af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59486/0 (socket says 172.21.15.47:59486) 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d204a8640 1 -- 172.21.15.47:0/488776305 learned_addr learned my addr 172.21.15.47:0/488776305 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d1f4a6640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294cfe1400 0x56294c1aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d1fca7640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294cfe1800 0x56294c1af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d204a8640 1 -- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294cfe1800 msgr2=0x56294c1af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d204a8640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294cfe1800 0x56294c1af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d204a8640 1 -- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294cfe1400 msgr2=0x56294c1aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d204a8640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294cfe1400 0x56294c1aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d204a8640 1 -- 172.21.15.47:0/488776305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56294cfee340 con 0x56294c282400 2024-03-23T09:50:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d204a8640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c282400 0x56294c1af700 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x56294c17b8f0 tx=0x56294cf1ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d19c9b640 1 -- 172.21.15.47:0/488776305 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56294c1d6f20 con 0x56294c282400 2024-03-23T09:50:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d19c9b640 1 -- 172.21.15.47:0/488776305 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56294cfe7980 con 0x56294c282400 2024-03-23T09:50:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56294cfee4e0 con 0x56294c282400 2024-03-23T09:50:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.658+0000 7f7d19c9b640 1 -- 172.21.15.47:0/488776305 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56294c1d71e0 con 0x56294c282400 2024-03-23T09:50:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.659+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56294cfee340 con 0x56294c282400 2024-03-23T09:50:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.659+0000 7f7d21c3d280 4 main: JournalTool: resolving pool 14 2024-03-23T09:50:48.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.660+0000 7f7d19c9b640 1 -- 172.21.15.47:0/488776305 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x56294c183340 con 0x56294c282400 2024-03-23T09:50:48.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.660+0000 7f7d19c9b640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x56294d047000 0x56294c472000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.660+0000 7f7d19c9b640 1 -- 172.21.15.47:0/488776305 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x56294c13e540 con 0x56294c282400 2024-03-23T09:50:48.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.660+0000 7f7d21c3d280 4 main: JournalTool: creating IoCtx.. 2024-03-23T09:50:48.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.660+0000 7f7d21c3d280 4 main: Executing for rank 0 2024-03-23T09:50:48.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.660+0000 7f7d21c3d280 4 Filtering by inode '1099511627777' 2024-03-23T09:50:48.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.660+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x56294cfe1000 0x56294c1ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.660+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) v8 -- 0x56294cfe0c00 con 0x56294cfe1000 2024-03-23T09:50:48.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.660+0000 7f7d1fca7640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x56294d047000 0x56294c472000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.660+0000 7f7d1f4a6640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x56294cfe1000 0x56294c1ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.661+0000 7f7d1fca7640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x56294d047000 0x56294c472000 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x56294ce90390 tx=0x56294cffcf00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.661+0000 7f7d1f4a6640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x56294cfe1000 0x56294c1ae680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.661+0000 7f7d1f4a6640 1 -- 172.21.15.47:0/488776305 <== osd.7 v2:172.21.15.175:6824/38174 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x56294cf12b40 con 0x56294cfe1000 2024-03-23T09:50:48.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.661+0000 7f7d21c3d280 4 Pointer 400.00000000 is readable 2024-03-23T09:50:48.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.661+0000 7f7d21c3d280 4 JournalScanner::scan: reading header object '200.00000000' 2024-03-23T09:50:48.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.661+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x56294cfe0800 0x56294c1ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.661+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) v8 -- 0x56294cfe1c00 con 0x56294cfe0800 2024-03-23T09:50:48.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.661+0000 7f7d204a8640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x56294cfe0800 0x56294c1ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.662+0000 7f7d204a8640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x56294cfe0800 0x56294c1ae100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.665+0000 7f7d204a8640 1 -- 172.21.15.47:0/488776305 <== osd.4 v2:172.21.15.175:6800/38159 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv30 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x56294cf12d80 con 0x56294cfe0800 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.665+0000 7f7d21c3d280 10 Header 0x400000 0x405ee0 0x40625c 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.665+0000 7f7d21c3d280 10 Starting journal scan from offset 0x405ee0 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.665+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24288~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) v8 -- 0x56294cfe0c00 con 0x56294cfe0800 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d204a8640 1 -- 172.21.15.47:0/488776305 <== osd.4 v2:172.21.15.175:6800/38159 2 ==== osd_op_reply(3 200.00000001 [read 24288~892 out=892b] v0'0 uv27 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x56294cf12d80 con 0x56294cfe0800 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 10 Parsing data, 0x37c bytes available 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 10 Attempting decode at 0x405ee0 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 10 Consumed 0x37c bytes 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 10 Valid entry at 0x405ee0 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x56294d046c00 0x56294c1ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) v8 -- 0x56294d046800 con 0x56294d046c00 2024-03-23T09:50:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d1fca7640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x56294d046c00 0x56294c1ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d1fca7640 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x56294d046c00 0x56294c1ad080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d1fca7640 1 -- 172.21.15.47:0/488776305 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56294cf12fc0 con 0x56294d046c00 2024-03-23T09:50:48.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 4 Reached end of journal objects 2024-03-23T09:50:48.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 4 Scanned objects, 0 missing, 1 valid 2024-03-23T09:50:48.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 4 Events scanned, 0 gaps 2024-03-23T09:50:48.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 4 Found 1 valid events 2024-03-23T09:50:48.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 4 Selected 0 events events for processing 2024-03-23T09:50:48.672 INFO:teuthology.orchestra.run.smithi047.stdout:Events by type: 2024-03-23T09:50:48.672 INFO:teuthology.orchestra.run.smithi047.stdout:Errors: 0 2024-03-23T09:50:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 4 0 events 2024-03-23T09:50:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x56294d046c00 msgr2=0x56294c1ad080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x56294d046c00 0x56294c1ad080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x56294cfe0800 msgr2=0x56294c1ae100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x56294cfe0800 0x56294c1ae100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x56294cfe1000 msgr2=0x56294c1ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x56294cfe1000 0x56294c1ae680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d1fca7640 1 -- 172.21.15.47:0/488776305 reap_dead start 2024-03-23T09:50:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x56294d047000 msgr2=0x56294c472000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x56294d047000 0x56294c472000 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x56294ce90390 tx=0x56294cffcf00 comp rx=0 tx=0).stop 2024-03-23T09:50:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c282400 msgr2=0x56294c1af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c282400 0x56294c1af700 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x56294c17b8f0 tx=0x56294cf1ed20 comp rx=0 tx=0).stop 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 shutdown_connections 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x56294d047000 0x56294c472000 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/488776305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c282400 0x56294c1af700 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.666+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 >> 172.21.15.47:0/488776305 conn(0x56294c282000 msgr2=0x56294c1fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.667+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 shutdown_connections 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.667+0000 7f7d21c3d280 1 -- 172.21.15.47:0/488776305 wait complete. 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.667+0000 7f7d21c3d280 1 -- 172.21.15.47:0/1921467346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c113c00 msgr2=0x56294c1acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.667+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/1921467346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c113c00 0x56294c1acb00 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x56294cf46d80 tx=0x56294cf1eb40 comp rx=0 tx=0).stop 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.667+0000 7f7d21c3d280 1 -- 172.21.15.47:0/1921467346 shutdown_connections 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.667+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/1921467346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c283c00 0x56294c1ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.667+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/1921467346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56294c113c00 0x56294c1acb00 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.667+0000 7f7d21c3d280 1 --2- 172.21.15.47:0/1921467346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56294c282800 0x56294c1adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.667+0000 7f7d21c3d280 1 -- 172.21.15.47:0/1921467346 >> 172.21.15.47:0/1921467346 conn(0x56294c113000 msgr2=0x56294c1fc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.667+0000 7f7d21c3d280 1 -- 172.21.15.47:0/1921467346 shutdown_connections 2024-03-23T09:50:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.667+0000 7f7d21c3d280 1 -- 172.21.15.47:0/1921467346 wait complete. 2024-03-23T09:50:48.677 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.064457, result: Events by type: Errors: 0 2024-03-23T09:50:48.677 DEBUG:teuthology.orchestra.run.smithi047:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-03-23T09:50:48.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.755+0000 7f8539c39280 1 -- start start 2024-03-23T09:50:48.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.755+0000 7f8539c39280 1 Processor -- start 2024-03-23T09:50:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.755+0000 7f8539c39280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5925cc00 0x55eb592f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.755+0000 7f8539c39280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb5925d800 0x55eb592f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.755+0000 7f8539c39280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5925dc00 0x55eb592f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.755+0000 7f8539c39280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb592eea00 con 0x55eb5925dc00 2024-03-23T09:50:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.755+0000 7f8539c39280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb592ee780 con 0x55eb5925cc00 2024-03-23T09:50:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.755+0000 7f8539c39280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb592ee640 con 0x55eb5925d800 2024-03-23T09:50:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.755+0000 7f8538366640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5925dc00 0x55eb592f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.755+0000 7f8538366640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5925dc00 0x55eb592f7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59508/0 (socket says 172.21.15.47:59508) 2024-03-23T09:50:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.755+0000 7f8538366640 1 -- 172.21.15.47:0/1287830408 learned_addr learned my addr 172.21.15.47:0/1287830408 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8537b65640 1 --2- 172.21.15.47:0/1287830408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5925cc00 0x55eb592f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8538366640 1 -- 172.21.15.47:0/1287830408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb5925d800 msgr2=0x55eb592f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8538366640 1 --2- 172.21.15.47:0/1287830408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb5925d800 0x55eb592f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8538366640 1 -- 172.21.15.47:0/1287830408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5925cc00 msgr2=0x55eb592f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8538366640 1 --2- 172.21.15.47:0/1287830408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5925cc00 0x55eb592f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8538366640 1 -- 172.21.15.47:0/1287830408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb592829c0 con 0x55eb5925dc00 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8538366640 1 --2- 172.21.15.47:0/1287830408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5925dc00 0x55eb592f7080 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x55eb5a07ccc0 tx=0x55eb5a116d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4655b0a42e2a4a26 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8535360640 1 -- 172.21.15.47:0/1287830408 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eb59321600 con 0x55eb5925dc00 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8535360640 1 -- 172.21.15.47:0/1287830408 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb592bc180 con 0x55eb5925dc00 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8535360640 1 -- 172.21.15.47:0/1287830408 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eb593211e0 con 0x55eb5925dc00 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8539c39280 1 -- 172.21.15.47:0/1287830408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb59282b60 con 0x55eb5925dc00 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8539c39280 1 -- 172.21.15.47:0/1287830408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55eb59282820 con 0x55eb5925dc00 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8539c39280 4 waiting for MDS map... 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8535360640 1 -- 172.21.15.47:0/1287830408 <== mon.0 v2:172.21.15.47:3300/0 4 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x55eb59288380 con 0x55eb5925dc00 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8535360640 1 -- 172.21.15.47:0/1287830408 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 66) v1 ==== 2573+0+0 (secure 0 0 0) 0x55eb592af200 con 0x55eb5925dc00 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8539c39280 4 Got MDS map 66 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.756+0000 7f8539c39280 10 main: JournalTool::main 2024-03-23T09:50:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.757+0000 7f8539c39280 4 main: JournalTool: connecting to RADOS... 2024-03-23T09:50:48.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.757+0000 7f8539c39280 1 Processor -- start 2024-03-23T09:50:48.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.757+0000 7f8539c39280 1 -- start start 2024-03-23T09:50:48.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8539c39280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5a12d000 0x55eb592f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8539c39280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d400 0x55eb592f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8539c39280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb5a12d800 0x55eb592f8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8539c39280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb5928de00 con 0x55eb5a12d400 2024-03-23T09:50:48.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8539c39280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb592ee780 con 0x55eb5a12d000 2024-03-23T09:50:48.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8539c39280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb592ee500 con 0x55eb5a12d800 2024-03-23T09:50:48.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8537b65640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d400 0x55eb592f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8537b65640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d400 0x55eb592f7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59518/0 (socket says 172.21.15.47:59518) 2024-03-23T09:50:48.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8537b65640 1 -- 172.21.15.47:0/1842949972 learned_addr learned my addr 172.21.15.47:0/1842949972 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8537364640 1 --2- 172.21.15.47:0/1842949972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5a12d000 0x55eb592f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8537364640 1 --2- 172.21.15.47:0/1842949972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb5a12d800 0x55eb592f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8537b65640 1 -- 172.21.15.47:0/1842949972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb5a12d800 msgr2=0x55eb592f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8537b65640 1 --2- 172.21.15.47:0/1842949972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb5a12d800 0x55eb592f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8537b65640 1 -- 172.21.15.47:0/1842949972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5a12d000 msgr2=0x55eb592f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8537b65640 1 --2- 172.21.15.47:0/1842949972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5a12d000 0x55eb592f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8537b65640 1 -- 172.21.15.47:0/1842949972 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb592824e0 con 0x55eb5a12d400 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8537364640 1 --2- 172.21.15.47:0/1842949972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5a12d000 0x55eb592f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f8537b65640 1 --2- 172.21.15.47:0/1842949972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d400 0x55eb592f7b80 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x55eb5a1980c0 tx=0x55eb5a13ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62ffc3d97f519ef server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f853335c640 1 -- 172.21.15.47:0/1842949972 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eb59321600 con 0x55eb5a12d400 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f853335c640 1 -- 172.21.15.47:0/1842949972 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb592bc480 con 0x55eb5a12d400 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.758+0000 7f853335c640 1 -- 172.21.15.47:0/1842949972 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eb593211e0 con 0x55eb5a12d400 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.759+0000 7f8539c39280 1 -- 172.21.15.47:0/1842949972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d400 msgr2=0x55eb592f7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.759+0000 7f8539c39280 1 --2- 172.21.15.47:0/1842949972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d400 0x55eb592f7b80 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x55eb5a1980c0 tx=0x55eb5a13ccc0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.759+0000 7f8539c39280 1 -- 172.21.15.47:0/1842949972 shutdown_connections 2024-03-23T09:50:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.759+0000 7f8539c39280 1 --2- 172.21.15.47:0/1842949972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb5a12d800 0x55eb592f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.759+0000 7f8539c39280 1 --2- 172.21.15.47:0/1842949972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d400 0x55eb592f7b80 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.759+0000 7f8539c39280 1 --2- 172.21.15.47:0/1842949972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5a12d000 0x55eb592f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.759+0000 7f8539c39280 1 -- 172.21.15.47:0/1842949972 >> 172.21.15.47:0/1842949972 conn(0x55eb5a12cc00 msgr2=0x55eb59347000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.759+0000 7f8539c39280 1 -- 172.21.15.47:0/1842949972 shutdown_connections 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.759+0000 7f8539c39280 1 -- 172.21.15.47:0/1842949972 wait complete. 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.759+0000 7f8539c39280 1 Processor -- start 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.759+0000 7f8539c39280 1 -- start start 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8539c39280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d000 0x55eb592f9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8539c39280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb5a12d400 0x55eb592f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8539c39280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5a12d800 0x55eb592f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8539c39280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb592ee3c0 con 0x55eb5a12d000 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8539c39280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb592ee280 con 0x55eb5a12d800 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8539c39280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb592ee140 con 0x55eb5a12d400 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8537364640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d000 0x55eb592f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8537364640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d000 0x55eb592f9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59522/0 (socket says 172.21.15.47:59522) 2024-03-23T09:50:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8537364640 1 -- 172.21.15.47:0/3922022932 learned_addr learned my addr 172.21.15.47:0/3922022932 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8537364640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5a12d800 0x55eb592f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8537b65640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb5a12d400 0x55eb592f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8537364640 1 -- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb5a12d400 msgr2=0x55eb592f8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8537364640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb5a12d400 0x55eb592f8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8537364640 1 -- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5a12d800 msgr2=0x55eb592f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8537364640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5a12d800 0x55eb592f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8537364640 1 -- 172.21.15.47:0/3922022932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb592829c0 con 0x55eb5a12d000 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8537364640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d000 0x55eb592f9180 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x55eb5a198de0 tx=0x55eb59ff2f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.760+0000 7f8531b59640 1 -- 172.21.15.47:0/3922022932 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eb59321080 con 0x55eb5a12d000 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.761+0000 7f8531b59640 1 -- 172.21.15.47:0/3922022932 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb593d4300 con 0x55eb5a12d000 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.761+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eb59282b60 con 0x55eb5a12d000 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.761+0000 7f8531b59640 1 -- 172.21.15.47:0/3922022932 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eb59321600 con 0x55eb5a12d000 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.761+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb59282820 con 0x55eb5a12d000 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.761+0000 7f8539c39280 4 main: JournalTool: resolving pool 14 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.762+0000 7f8531b59640 1 -- 172.21.15.47:0/3922022932 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x55eb592cd080 con 0x55eb5a12d000 2024-03-23T09:50:48.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.762+0000 7f8531b59640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55eb5a1a6800 0x55eb592f9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.762+0000 7f8531b59640 1 -- 172.21.15.47:0/3922022932 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x55eb59288540 con 0x55eb5a12d000 2024-03-23T09:50:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.762+0000 7f8539c39280 4 main: JournalTool: creating IoCtx.. 2024-03-23T09:50:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.762+0000 7f8539c39280 4 main: Executing for rank 0 2024-03-23T09:50:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.762+0000 7f8539c39280 4 Filtering by inode '1099511627778' 2024-03-23T09:50:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.762+0000 7f8539c39280 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55eb5a1a6c00 0x55eb592f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.762+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) v8 -- 0x55eb5a1a7000 con 0x55eb5a1a6c00 2024-03-23T09:50:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.762+0000 7f8537b65640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55eb5a1a6800 0x55eb592f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.762+0000 7f8537364640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55eb5a1a6c00 0x55eb592f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.763+0000 7f8537b65640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55eb5a1a6800 0x55eb592f9700 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x55eb5a1980c0 tx=0x55eb5a13cc00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.763+0000 7f8537364640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55eb5a1a6c00 0x55eb592f8100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.763+0000 7f8537364640 1 -- 172.21.15.47:0/3922022932 <== osd.7 v2:172.21.15.175:6824/38174 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55eb5a112b40 con 0x55eb5a1a6c00 2024-03-23T09:50:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.763+0000 7f8539c39280 4 Pointer 400.00000000 is readable 2024-03-23T09:50:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.763+0000 7f8539c39280 4 JournalScanner::scan: reading header object '200.00000000' 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.763+0000 7f8539c39280 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55eb5a1a7400 0x55eb592f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.763+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) v8 -- 0x55eb5a1a7800 con 0x55eb5a1a7400 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.763+0000 7f8538366640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55eb5a1a7400 0x55eb592f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.764+0000 7f8538366640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55eb5a1a7400 0x55eb592f7b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.764+0000 7f8538366640 1 -- 172.21.15.47:0/3922022932 <== osd.4 v2:172.21.15.175:6800/38159 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv30 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55eb5a112d80 con 0x55eb5a1a7400 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.764+0000 7f8539c39280 10 Header 0x400000 0x405ee0 0x40625c 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.764+0000 7f8539c39280 10 Starting journal scan from offset 0x405ee0 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.764+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24288~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) v8 -- 0x55eb5a1a7c00 con 0x55eb5a1a7400 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.764+0000 7f8538366640 1 -- 172.21.15.47:0/3922022932 <== osd.4 v2:172.21.15.175:6800/38159 2 ==== osd_op_reply(3 200.00000001 [read 24288~892 out=892b] v0'0 uv27 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x55eb5a112d80 con 0x55eb5a1a7400 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.764+0000 7f8539c39280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.764+0000 7f8539c39280 10 Parsing data, 0x37c bytes available 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.764+0000 7f8539c39280 10 Attempting decode at 0x405ee0 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.764+0000 7f8539c39280 10 Consumed 0x37c bytes 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 10 Valid entry at 0x405ee0 2024-03-23T09:50:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55eb595dc000 0x55eb592f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) v8 -- 0x55eb595dc400 con 0x55eb595dc000 2024-03-23T09:50:48.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8537b65640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55eb595dc000 0x55eb592f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8537b65640 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55eb595dc000 0x55eb592f7600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8537b65640 1 -- 172.21.15.47:0/3922022932 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55eb5a112fc0 con 0x55eb595dc000 2024-03-23T09:50:48.772 INFO:teuthology.orchestra.run.smithi047.stdout:Events by type: 2024-03-23T09:50:48.772 INFO:teuthology.orchestra.run.smithi047.stdout:Errors: 0 2024-03-23T09:50:48.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 4 Reached end of journal objects 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 4 Scanned objects, 0 missing, 1 valid 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 4 Events scanned, 0 gaps 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 4 Found 1 valid events 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 4 Selected 0 events events for processing 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 4 0 events 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55eb595dc000 msgr2=0x55eb592f7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55eb595dc000 0x55eb592f7600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55eb5a1a7400 msgr2=0x55eb592f7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55eb5a1a7400 0x55eb592f7b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55eb5a1a6c00 msgr2=0x55eb592f8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8539c39280 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55eb5a1a6c00 0x55eb592f8100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.765+0000 7f8537b65640 1 -- 172.21.15.47:0/3922022932 reap_dead start 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55eb5a1a6800 msgr2=0x55eb592f9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55eb5a1a6800 0x55eb592f9700 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x55eb5a1980c0 tx=0x55eb5a13cc00 comp rx=0 tx=0).stop 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d000 msgr2=0x55eb592f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d000 0x55eb592f9180 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x55eb5a198de0 tx=0x55eb59ff2f60 comp rx=0 tx=0).stop 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 shutdown_connections 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55eb5a1a6800 0x55eb592f9700 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 --2- 172.21.15.47:0/3922022932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5a12d000 0x55eb592f9180 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 >> 172.21.15.47:0/3922022932 conn(0x55eb5a12cc00 msgr2=0x55eb59347000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 shutdown_connections 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 -- 172.21.15.47:0/3922022932 wait complete. 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 -- 172.21.15.47:0/1287830408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5925dc00 msgr2=0x55eb592f7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 --2- 172.21.15.47:0/1287830408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5925dc00 0x55eb592f7080 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x55eb5a07ccc0 tx=0x55eb5a116d80 comp rx=0 tx=0).stop 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 -- 172.21.15.47:0/1287830408 shutdown_connections 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 --2- 172.21.15.47:0/1287830408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb5925dc00 0x55eb592f7080 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 --2- 172.21.15.47:0/1287830408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55eb5925d800 0x55eb592f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 --2- 172.21.15.47:0/1287830408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb5925cc00 0x55eb592f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 -- 172.21.15.47:0/1287830408 >> 172.21.15.47:0/1287830408 conn(0x55eb5925d000 msgr2=0x55eb59346800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 -- 172.21.15.47:0/1287830408 shutdown_connections 2024-03-23T09:50:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.766+0000 7f8539c39280 1 -- 172.21.15.47:0/1287830408 wait complete. 2024-03-23T09:50:48.777 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.099707, result: Events by type: Errors: 0 2024-03-23T09:50:48.777 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-03-23T09:50:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.854+0000 7fef7223df40 1 Processor -- start 2024-03-23T09:50:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.855+0000 7fef7223df40 1 -- start start 2024-03-23T09:50:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.855+0000 7fef7223df40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606413aaab0 0x560641438aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.855+0000 7fef7223df40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5606413b2b00 0x56064143a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.855+0000 7fef7223df40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560641441b20 0x56064143aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.855+0000 7fef7223df40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606413174d0 con 0x560641441b20 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.855+0000 7fef7223df40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606412b2b60 con 0x5606413aaab0 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.855+0000 7fef7223df40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606412b2800 con 0x5606413b2b00 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.855+0000 7fef6fc9c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560641441b20 0x56064143aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.855+0000 7fef6fc9c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560641441b20 0x56064143aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59528/0 (socket says 172.21.15.47:59528) 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.855+0000 7fef6fc9c640 1 -- 172.21.15.47:0/63239977 learned_addr learned my addr 172.21.15.47:0/63239977 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef6f49b640 1 --2- 172.21.15.47:0/63239977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606413aaab0 0x560641438aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef6ec9a640 1 --2- 172.21.15.47:0/63239977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5606413b2b00 0x56064143a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef6fc9c640 1 -- 172.21.15.47:0/63239977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5606413b2b00 msgr2=0x56064143a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef6fc9c640 1 --2- 172.21.15.47:0/63239977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5606413b2b00 0x56064143a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef6fc9c640 1 -- 172.21.15.47:0/63239977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606413aaab0 msgr2=0x560641438aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef6fc9c640 1 --2- 172.21.15.47:0/63239977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606413aaab0 0x560641438aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef6fc9c640 1 -- 172.21.15.47:0/63239977 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef68005d20 con 0x560641441b20 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef6f49b640 1 --2- 172.21.15.47:0/63239977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606413aaab0 0x560641438aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef6ec9a640 1 --2- 172.21.15.47:0/63239977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5606413b2b00 0x56064143a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef6fc9c640 1 --2- 172.21.15.47:0/63239977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560641441b20 0x56064143aaf0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fef6800a4f0 tx=0x7fef6804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffcdd57645f9a82 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef6dc98640 1 -- 172.21.15.47:0/63239977 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68052b10 con 0x560641441b20 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef6dc98640 1 -- 172.21.15.47:0/63239977 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef68052c90 con 0x560641441b20 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef6dc98640 1 -- 172.21.15.47:0/63239977 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680525c0 con 0x560641441b20 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef7223df40 1 -- 172.21.15.47:0/63239977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560641441b20 msgr2=0x56064143aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.856+0000 7fef7223df40 1 --2- 172.21.15.47:0/63239977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560641441b20 0x56064143aaf0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fef6800a4f0 tx=0x7fef6804c000 comp rx=0 tx=0).stop 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.857+0000 7fef7223df40 1 -- 172.21.15.47:0/63239977 shutdown_connections 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.857+0000 7fef7223df40 1 --2- 172.21.15.47:0/63239977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560641441b20 0x56064143aaf0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.857+0000 7fef7223df40 1 --2- 172.21.15.47:0/63239977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5606413b2b00 0x56064143a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.857+0000 7fef7223df40 1 --2- 172.21.15.47:0/63239977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606413aaab0 0x560641438aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.857+0000 7fef7223df40 1 -- 172.21.15.47:0/63239977 >> 172.21.15.47:0/63239977 conn(0x560641395e40 msgr2=0x5606413b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.857+0000 7fef7223df40 1 -- 172.21.15.47:0/63239977 shutdown_connections 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.857+0000 7fef7223df40 1 -- 172.21.15.47:0/63239977 wait complete. 2024-03-23T09:50:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.857+0000 7fef7223df40 1 Processor -- start 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.857+0000 7fef7223df40 1 -- start start 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef7223df40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5606413aaab0 0x560641444320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef7223df40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5606413b2b00 0x560641434e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef7223df40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560641441b20 0x560641440960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef7223df40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560641295520 con 0x5606413b2b00 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef7223df40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606413982e0 con 0x560641441b20 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef7223df40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606413985c0 con 0x5606413aaab0 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef6ec9a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5606413b2b00 0x560641434e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef6ec9a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5606413b2b00 0x560641434e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59538/0 (socket says 172.21.15.47:59538) 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef6ec9a640 1 -- 172.21.15.47:0/3475094504 learned_addr learned my addr 172.21.15.47:0/3475094504 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef6f49b640 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5606413aaab0 0x560641444320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef6fc9c640 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560641441b20 0x560641440960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef6ec9a640 1 -- 172.21.15.47:0/3475094504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5606413aaab0 msgr2=0x560641444320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef6ec9a640 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5606413aaab0 0x560641444320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef6ec9a640 1 -- 172.21.15.47:0/3475094504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560641441b20 msgr2=0x560641440960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef6ec9a640 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560641441b20 0x560641440960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef6ec9a640 1 -- 172.21.15.47:0/3475094504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606412c1c20 con 0x5606413b2b00 2024-03-23T09:50:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.858+0000 7fef6ec9a640 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5606413b2b00 0x560641434e50 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fef5800cf70 tx=0x7fef5800a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.859+0000 7fef6cc96640 1 -- 172.21.15.47:0/3475094504 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5800ae60 con 0x5606413b2b00 2024-03-23T09:50:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.859+0000 7fef6cc96640 1 -- 172.21.15.47:0/3475094504 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5800f400 con 0x5606413b2b00 2024-03-23T09:50:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.859+0000 7fef7223df40 1 -- 172.21.15.47:0/3475094504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5606412b5060 con 0x5606413b2b00 2024-03-23T09:50:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.859+0000 7fef6cc96640 1 -- 172.21.15.47:0/3475094504 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5800f790 con 0x5606413b2b00 2024-03-23T09:50:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.859+0000 7fef7223df40 1 -- 172.21.15.47:0/3475094504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606412b4ac0 con 0x5606413b2b00 2024-03-23T09:50:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.860+0000 7fef6cc96640 1 -- 172.21.15.47:0/3475094504 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fef5800d040 con 0x5606413b2b00 2024-03-23T09:50:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.860+0000 7fef6cc96640 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef500695e0 0x7fef5006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.860+0000 7fef6cc96640 1 -- 172.21.15.47:0/3475094504 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fef58087ca0 con 0x5606413b2b00 2024-03-23T09:50:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.861+0000 7fef6f49b640 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef500695e0 0x7fef5006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.861+0000 7fef7223df40 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x560641440ea0 0x560641399380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.861+0000 7fef7223df40 1 -- 172.21.15.47:0/3475094504 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5606413998c0 con 0x560641440ea0 2024-03-23T09:50:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.861+0000 7fef6fc9c640 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x560641440ea0 0x560641399380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.861+0000 7fef6f49b640 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef500695e0 0x7fef5006baa0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x560641394b50 tx=0x7fef60007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.861+0000 7fef6fc9c640 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x560641440ea0 0x560641399380 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.864+0000 7fef6fc9c640 1 -- 172.21.15.47:0/3475094504 <== osd.2 v2:172.21.15.47:6800/37678 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v102'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef68052910 con 0x560641440ea0 2024-03-23T09:50:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.864+0000 7fef7223df40 1 -- 172.21.15.47:0/3475094504 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x560641440ea0 msgr2=0x560641399380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.864+0000 7fef7223df40 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x560641440ea0 0x560641399380 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.864+0000 7fef7223df40 1 -- 172.21.15.47:0/3475094504 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef500695e0 msgr2=0x7fef5006baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.864+0000 7fef7223df40 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef500695e0 0x7fef5006baa0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x560641394b50 tx=0x7fef60007040 comp rx=0 tx=0).stop 2024-03-23T09:50:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.865+0000 7fef7223df40 1 -- 172.21.15.47:0/3475094504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5606413b2b00 msgr2=0x560641434e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:48.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.865+0000 7fef7223df40 1 --2- 172.21.15.47:0/3475094504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5606413b2b00 0x560641434e50 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fef5800cf70 tx=0x7fef5800a690 comp rx=0 tx=0).stop 2024-03-23T09:50:48.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.865+0000 7fef6fc9c640 1 -- 172.21.15.47:0/3475094504 reap_dead start 2024-03-23T09:50:48.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.865+0000 7fef7223df40 1 -- 172.21.15.47:0/3475094504 shutdown_connections 2024-03-23T09:50:48.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.865+0000 7fef7223df40 1 -- 172.21.15.47:0/3475094504 >> 172.21.15.47:0/3475094504 conn(0x560641395e40 msgr2=0x560641444ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:48.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.865+0000 7fef7223df40 1 -- 172.21.15.47:0/3475094504 shutdown_connections 2024-03-23T09:50:48.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:48.865+0000 7fef7223df40 1 -- 172.21.15.47:0/3475094504 wait complete. 2024-03-23T09:50:48.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-23T09:50:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.013+0000 7f178508b640 1 Processor -- start 2024-03-23T09:50:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.013+0000 7f178508b640 1 -- start start 2024-03-23T09:50:49.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.013+0000 7f178508b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f178006bf10 0x7f17800ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.013+0000 7f178508b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17800cdad0 con 0x7f1780161020 2024-03-23T09:50:49.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.013+0000 7f178508b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17800b03c0 con 0x7f1780161400 2024-03-23T09:50:49.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.013+0000 7f178508b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17800b06a0 con 0x7f178006bf10 2024-03-23T09:50:49.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177ed9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1780161020 0x7f17800b1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45860/0 (socket says 172.21.15.47:45860) 2024-03-23T09:50:49.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177ed9d640 1 -- 172.21.15.47:0/53703956 learned_addr learned my addr 172.21.15.47:0/53703956 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:49.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177f59e640 1 --2- 172.21.15.47:0/53703956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f178006bf10 0x7f17800ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509312174 0 0) 0x7f17800cdad0 con 0x7f1780161020 2024-03-23T09:50:49.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f176c003520 con 0x7f1780161020 2024-03-23T09:50:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259380150 0 0) 0x7f17800b03c0 con 0x7f1780161400 2024-03-23T09:50:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17800cdad0 con 0x7f1780161400 2024-03-23T09:50:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 138315112 0 0) 0x7f176c003520 con 0x7f1780161020 2024-03-23T09:50:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f178006bf10 msgr2=0x7f17800ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 --2- 172.21.15.47:0/53703956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f178006bf10 0x7f17800ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 >> v1:172.21.15.175:6789/0 conn(0x7f1780161400 legacy=0x7f178006b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17800b0980 con 0x7f1780161020 2024-03-23T09:50:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f1770001670 con 0x7f1780161020 2024-03-23T09:50:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 >> v1:172.21.15.47:6789/0 conn(0x7f1780161020 legacy=0x7f17800b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 --2- 172.21.15.47:0/53703956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f176c003b30 0x7f176c003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 --2- 172.21.15.47:0/53703956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f176c0047c0 0x7f176c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 --2- 172.21.15.47:0/53703956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c005180 0x7f176c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17800b06a0 con 0x7f176c003b30 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c00bfa0 con 0x7f176c005180 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c00c160 con 0x7f176c0047c0 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.014+0000 7f177f59e640 1 --2- 172.21.15.47:0/53703956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f178006bf10 0x7f17800ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.015+0000 7f177ed9d640 1 --2- 172.21.15.47:0/53703956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f176c003b30 0x7f176c003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.015+0000 7f1777fff640 1 --2- 172.21.15.47:0/53703956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f176c0047c0 0x7f176c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.015+0000 7f177f59e640 1 --2- 172.21.15.47:0/53703956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c005180 0x7f176c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.015+0000 7f177ed9d640 1 -- 172.21.15.47:0/53703956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f176c0047c0 msgr2=0x7f176c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.015+0000 7f177ed9d640 1 --2- 172.21.15.47:0/53703956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f176c0047c0 0x7f176c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.015+0000 7f177ed9d640 1 -- 172.21.15.47:0/53703956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c005180 msgr2=0x7f176c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.015+0000 7f177ed9d640 1 --2- 172.21.15.47:0/53703956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c005180 0x7f176c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.015+0000 7f177ed9d640 1 -- 172.21.15.47:0/53703956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c003520 con 0x7f176c003b30 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.016+0000 7f177ed9d640 1 --2- 172.21.15.47:0/53703956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f176c003b30 0x7f176c003f30 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1770007f90 tx=0x7f176c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a45ca5a6ef26a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.016+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1770005ad0 con 0x7f176c003b30 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.016+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1770005c30 con 0x7f176c003b30 2024-03-23T09:50:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.016+0000 7f177e59c640 1 -- 172.21.15.47:0/53703956 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177002c790 con 0x7f176c003b30 2024-03-23T09:50:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.016+0000 7f178508b640 1 -- 172.21.15.47:0/53703956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f176c003b30 msgr2=0x7f176c003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.016+0000 7f178508b640 1 --2- 172.21.15.47:0/53703956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f176c003b30 0x7f176c003f30 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1770007f90 tx=0x7f176c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.016+0000 7f177f59e640 1 -- 172.21.15.47:0/53703956 reap_dead start 2024-03-23T09:50:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.017+0000 7f178508b640 1 -- 172.21.15.47:0/53703956 shutdown_connections 2024-03-23T09:50:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.017+0000 7f178508b640 1 -- 172.21.15.47:0/53703956 >> 172.21.15.47:0/53703956 conn(0x7f17800d0670 msgr2=0x7f17800d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.017+0000 7f178508b640 1 -- 172.21.15.47:0/53703956 shutdown_connections 2024-03-23T09:50:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.017+0000 7f178508b640 1 -- 172.21.15.47:0/53703956 wait complete. 2024-03-23T09:50:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.018+0000 7f178508b640 1 Processor -- start 2024-03-23T09:50:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.018+0000 7f178508b640 1 -- start start 2024-03-23T09:50:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f178508b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c003b30 0x7f178014e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f178508b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17800b5d00 0x7f178009dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f178508b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17800b6180 0x7f178009e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f178508b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17800c8790 con 0x7f17800b5d00 2024-03-23T09:50:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f178508b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17800c89a0 con 0x7f176c003b30 2024-03-23T09:50:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f178508b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17800c8c80 con 0x7f17800b6180 2024-03-23T09:50:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f1777fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17800b5d00 0x7f178009dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f1777fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17800b5d00 0x7f178009dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59558/0 (socket says 172.21.15.47:59558) 2024-03-23T09:50:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f1777fff640 1 -- 172.21.15.47:0/999261181 learned_addr learned my addr 172.21.15.47:0/999261181 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f177f59e640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17800b6180 0x7f178009e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f177ed9d640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c003b30 0x7f178014e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f1777fff640 1 -- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17800b6180 msgr2=0x7f178009e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f1777fff640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17800b6180 0x7f178009e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f1777fff640 1 -- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c003b30 msgr2=0x7f178014e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f1777fff640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c003b30 0x7f178014e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f1777fff640 1 -- 172.21.15.47:0/999261181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17800c8f60 con 0x7f17800b5d00 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.019+0000 7f1777fff640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17800b5d00 0x7f178009dae0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f176800f7a0 tx=0x7f176800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.020+0000 7f17777fe640 1 -- 172.21.15.47:0/999261181 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176800feb0 con 0x7f17800b5d00 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.020+0000 7f17777fe640 1 -- 172.21.15.47:0/999261181 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1768005320 con 0x7f17800b5d00 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.020+0000 7f178508b640 1 -- 172.21.15.47:0/999261181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17800c9240 con 0x7f17800b5d00 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.020+0000 7f17777fe640 1 -- 172.21.15.47:0/999261181 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17680072b0 con 0x7f17800b5d00 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.020+0000 7f178508b640 1 -- 172.21.15.47:0/999261181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1780150630 con 0x7f17800b5d00 2024-03-23T09:50:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.021+0000 7f178508b640 1 -- 172.21.15.47:0/999261181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17800672f0 con 0x7f17800b5d00 2024-03-23T09:50:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.022+0000 7f17777fe640 1 -- 172.21.15.47:0/999261181 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1768007410 con 0x7f17800b5d00 2024-03-23T09:50:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.022+0000 7f17777fe640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f176006ac00 0x7f176006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.022+0000 7f17777fe640 1 -- 172.21.15.47:0/999261181 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1768088b30 con 0x7f17800b5d00 2024-03-23T09:50:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.022+0000 7f177ed9d640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f176006ac00 0x7f176006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.023+0000 7f177ed9d640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f176006ac00 0x7f176006d0c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1770034000 tx=0x7f1770009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:49.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.028+0000 7f17777fe640 1 -- 172.21.15.47:0/999261181 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1768053d60 con 0x7f17800b5d00 2024-03-23T09:50:49.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.248+0000 7f178508b640 1 -- 172.21.15.47:0/999261181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f17800b01a0 con 0x7f17800b5d00 2024-03-23T09:50:49.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.599+0000 7f17777fe640 1 -- 172.21.15.47:0/999261181 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v67) v1 ==== 162+0+0 (secure 0 0 0) 0x7f176801a020 con 0x7f17800b5d00 2024-03-23T09:50:49.601 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-23T09:50:49.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.601+0000 7f178508b640 1 -- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f176006ac00 msgr2=0x7f176006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:49.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.602+0000 7f178508b640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f176006ac00 0x7f176006d0c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1770034000 tx=0x7f1770009030 comp rx=0 tx=0).stop 2024-03-23T09:50:49.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.602+0000 7f178508b640 1 -- 172.21.15.47:0/999261181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17800b5d00 msgr2=0x7f178009dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:49.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.602+0000 7f178508b640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17800b5d00 0x7f178009dae0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f176800f7a0 tx=0x7f176800fc60 comp rx=0 tx=0).stop 2024-03-23T09:50:49.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.602+0000 7f178508b640 1 -- 172.21.15.47:0/999261181 shutdown_connections 2024-03-23T09:50:49.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.602+0000 7f178508b640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f176006ac00 0x7f176006d0c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.602+0000 7f178508b640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17800b6180 0x7f178009e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.602+0000 7f178508b640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17800b5d00 0x7f178009dae0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.602+0000 7f178508b640 1 --2- 172.21.15.47:0/999261181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c003b30 0x7f178014e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.602+0000 7f178508b640 1 -- 172.21.15.47:0/999261181 >> 172.21.15.47:0/999261181 conn(0x7f17800d0670 msgr2=0x7f178006f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:49.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.602+0000 7f178508b640 1 -- 172.21.15.47:0/999261181 shutdown_connections 2024-03-23T09:50:49.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.602+0000 7f178508b640 1 -- 172.21.15.47:0/999261181 wait complete. 2024-03-23T09:50:49.617 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:50:49.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.716+0000 7f885ffb1640 1 Processor -- start 2024-03-23T09:50:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.717+0000 7f885ffb1640 1 -- start start 2024-03-23T09:50:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.717+0000 7f885ffb1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88580cd7f0 0x7f88580ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.717+0000 7f885ffb1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88580ce800 0x7f88580cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.717+0000 7f885ffb1640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885815ae50 con 0x7f885815aa70 2024-03-23T09:50:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.717+0000 7f885ffb1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88580cf010 con 0x7f88580ce800 2024-03-23T09:50:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.717+0000 7f885ffb1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88580cf2f0 con 0x7f88580cd7f0 2024-03-23T09:50:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.717+0000 7f885e54e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88580ce800 0x7f88580cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.718+0000 7f885e54e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88580ce800 0x7f88580cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59562/0 (socket says 172.21.15.47:59562) 2024-03-23T09:50:49.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.718+0000 7f885e54e640 1 -- 172.21.15.47:0/3601388884 learned_addr learned my addr 172.21.15.47:0/3601388884 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:49.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.718+0000 7f885d54c640 1 --2- 172.21.15.47:0/3601388884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88580cd7f0 0x7f88580ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.718+0000 7f885e54e640 1 -- 172.21.15.47:0/3601388884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88580cd7f0 msgr2=0x7f88580ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:49.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.718+0000 7f885e54e640 1 --2- 172.21.15.47:0/3601388884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88580cd7f0 0x7f88580ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.718+0000 7f885e54e640 1 -- 172.21.15.47:0/3601388884 >> v1:172.21.15.175:6789/0 conn(0x7f885815aa70 legacy=0x7f88580cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:49.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.718+0000 7f885e54e640 1 -- 172.21.15.47:0/3601388884 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88580cf5d0 con 0x7f88580ce800 2024-03-23T09:50:49.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.719+0000 7f885e54e640 1 --2- 172.21.15.47:0/3601388884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88580ce800 0x7f88580cbb70 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8850009870 tx=0x7f885002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a15e8cfe08cace server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:49.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.719+0000 7f885cd4b640 1 -- 172.21.15.47:0/3601388884 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850030840 con 0x7f88580ce800 2024-03-23T09:50:49.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.719+0000 7f885cd4b640 1 -- 172.21.15.47:0/3601388884 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88500309a0 con 0x7f88580ce800 2024-03-23T09:50:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.719+0000 7f885cd4b640 1 -- 172.21.15.47:0/3601388884 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850030d30 con 0x7f88580ce800 2024-03-23T09:50:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.719+0000 7f885ffb1640 1 -- 172.21.15.47:0/3601388884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88580ce800 msgr2=0x7f88580cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.719+0000 7f885ffb1640 1 --2- 172.21.15.47:0/3601388884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88580ce800 0x7f88580cbb70 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8850009870 tx=0x7f885002b980 comp rx=0 tx=0).stop 2024-03-23T09:50:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.719+0000 7f885ffb1640 1 -- 172.21.15.47:0/3601388884 shutdown_connections 2024-03-23T09:50:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.719+0000 7f885ffb1640 1 --2- 172.21.15.47:0/3601388884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88580ce800 0x7f88580cbb70 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.719+0000 7f885ffb1640 1 --2- 172.21.15.47:0/3601388884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88580cd7f0 0x7f88580ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.719+0000 7f885ffb1640 1 -- 172.21.15.47:0/3601388884 >> 172.21.15.47:0/3601388884 conn(0x7f88580d06f0 msgr2=0x7f88580d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.719+0000 7f885ffb1640 1 -- 172.21.15.47:0/3601388884 shutdown_connections 2024-03-23T09:50:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.719+0000 7f885ffb1640 1 -- 172.21.15.47:0/3601388884 wait complete. 2024-03-23T09:50:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.720+0000 7f885ffb1640 1 Processor -- start 2024-03-23T09:50:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.720+0000 7f885ffb1640 1 -- start start 2024-03-23T09:50:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.721+0000 7f885ffb1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88580cd7f0 0x7f885815ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.721+0000 7f885ffb1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88580ce800 0x7f885815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.721+0000 7f885ffb1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f885815aa70 0x7f8858153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.721+0000 7f885ffb1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88581521d0 con 0x7f88580cd7f0 2024-03-23T09:50:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.721+0000 7f885ffb1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885815d4e0 con 0x7f885815aa70 2024-03-23T09:50:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.721+0000 7f885ffb1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88580c9110 con 0x7f88580ce800 2024-03-23T09:50:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.721+0000 7f885dd4d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88580cd7f0 0x7f885815ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.721+0000 7f885dd4d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88580cd7f0 0x7f885815ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59572/0 (socket says 172.21.15.47:59572) 2024-03-23T09:50:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.721+0000 7f885dd4d640 1 -- 172.21.15.47:0/3196267256 learned_addr learned my addr 172.21.15.47:0/3196267256 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.721+0000 7f885e54e640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f885815aa70 0x7f8858153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.721+0000 7f885d54c640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88580ce800 0x7f885815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.722+0000 7f885dd4d640 1 -- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88580ce800 msgr2=0x7f885815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.722+0000 7f885dd4d640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88580ce800 0x7f885815cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.722+0000 7f885dd4d640 1 -- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f885815aa70 msgr2=0x7f8858153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.722+0000 7f885dd4d640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f885815aa70 0x7f8858153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.722+0000 7f885dd4d640 1 -- 172.21.15.47:0/3196267256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88500094d0 con 0x7f88580cd7f0 2024-03-23T09:50:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.722+0000 7f885dd4d640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88580cd7f0 0x7f885815ca60 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f884800ba50 tx=0x7f884800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.722+0000 7f88567fc640 1 -- 172.21.15.47:0/3196267256 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884800cbe0 con 0x7f88580cd7f0 2024-03-23T09:50:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.722+0000 7f88567fc640 1 -- 172.21.15.47:0/3196267256 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884800cd80 con 0x7f88580cd7f0 2024-03-23T09:50:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.722+0000 7f885ffb1640 1 -- 172.21.15.47:0/3196267256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88580c8870 con 0x7f88580cd7f0 2024-03-23T09:50:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.722+0000 7f88567fc640 1 -- 172.21.15.47:0/3196267256 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848014680 con 0x7f88580cd7f0 2024-03-23T09:50:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.722+0000 7f885ffb1640 1 -- 172.21.15.47:0/3196267256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88580c8e30 con 0x7f88580cd7f0 2024-03-23T09:50:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.723+0000 7f885ffb1640 1 -- 172.21.15.47:0/3196267256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88580c3fe0 con 0x7f88580cd7f0 2024-03-23T09:50:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.723+0000 7f88567fc640 1 -- 172.21.15.47:0/3196267256 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f88480147e0 con 0x7f88580cd7f0 2024-03-23T09:50:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.724+0000 7f88567fc640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f884406ac30 0x7f884406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.724+0000 7f88567fc640 1 -- 172.21.15.47:0/3196267256 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8848089210 con 0x7f88580cd7f0 2024-03-23T09:50:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.724+0000 7f885d54c640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f884406ac30 0x7f884406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.725+0000 7f885d54c640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f884406ac30 0x7f884406d0f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f88580ca7c0 tx=0x7f884c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.728+0000 7f88567fc640 1 -- 172.21.15.47:0/3196267256 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8848054440 con 0x7f88580cd7f0 2024-03-23T09:50:49.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.944+0000 7f885ffb1640 1 -- 172.21.15.47:0/3196267256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8858066be0 con 0x7f88580cd7f0 2024-03-23T09:50:49.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.945+0000 7f88567fc640 1 -- 172.21.15.47:0/3196267256 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f8848057a60 con 0x7f88580cd7f0 2024-03-23T09:50:49.946 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:49.946 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":68,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/249416109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":249416109},{"type":"v1","addr":"172.21.15.175:6837","nonce":249416109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3603846321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3603846321},{"type":"v1","addr":"172.21.15.175:6835","nonce":3603846321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1921094794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1921094794},{"type":"v1","addr":"172.21.15.47:6834","nonce":1921094794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:49:43.247119+0000","modified":"2024-03-23T09:50:49.601459+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8916},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8916":{"gid":8916,"name":"a","rank":0,"incarnation":68,"state":"up:replay","state_seq":1,"addr":"172.21.15.47:6835/1748845936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1748845936},{"type":"v1","addr":"172.21.15.47:6835","nonce":1748845936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-23T09:50:49.947 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 68 2024-03-23T09:50:49.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.947+0000 7f885ffb1640 1 -- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f884406ac30 msgr2=0x7f884406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:49.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.947+0000 7f885ffb1640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f884406ac30 0x7f884406d0f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f88580ca7c0 tx=0x7f884c008040 comp rx=0 tx=0).stop 2024-03-23T09:50:49.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.947+0000 7f885ffb1640 1 -- 172.21.15.47:0/3196267256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88580cd7f0 msgr2=0x7f885815ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:49.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.947+0000 7f885ffb1640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88580cd7f0 0x7f885815ca60 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f884800ba50 tx=0x7f884800bf10 comp rx=0 tx=0).stop 2024-03-23T09:50:49.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.947+0000 7f885ffb1640 1 -- 172.21.15.47:0/3196267256 shutdown_connections 2024-03-23T09:50:49.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.947+0000 7f885ffb1640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f884406ac30 0x7f884406d0f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.947+0000 7f885ffb1640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f885815aa70 0x7f8858153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.947+0000 7f885ffb1640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88580ce800 0x7f885815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.947+0000 7f885ffb1640 1 --2- 172.21.15.47:0/3196267256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88580cd7f0 0x7f885815ca60 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:49.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.947+0000 7f885ffb1640 1 -- 172.21.15.47:0/3196267256 >> 172.21.15.47:0/3196267256 conn(0x7f88580d06f0 msgr2=0x7f885806e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:49.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.947+0000 7f885ffb1640 1 -- 172.21.15.47:0/3196267256 shutdown_connections 2024-03-23T09:50:49.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:49.947+0000 7f885ffb1640 1 -- 172.21.15.47:0/3196267256 wait complete. 2024-03-23T09:50:50.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:49:43.247119+0000', 'modified': '2024-03-23T09:50:49.601459+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8916}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8916': {'gid': 8916, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.47:6835/1748845936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6833', 'nonce': 1748845936}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 1748845936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:50:50.361 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8916:up:replay 2024-03-23T09:50:51.362 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:50:51.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.475+0000 7f7446251640 1 Processor -- start 2024-03-23T09:50:51.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.475+0000 7f7446251640 1 -- start start 2024-03-23T09:50:51.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.475+0000 7f7446251640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f74400d02e0 0x7f7440159b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:51.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.476+0000 7f7446251640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744015a060 con 0x7f74400cb200 2024-03-23T09:50:51.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.476+0000 7f7446251640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74400ca440 con 0x7f74400cbd20 2024-03-23T09:50:51.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.476+0000 7f7446251640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74400ca720 con 0x7f74400d02e0 2024-03-23T09:50:51.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.476+0000 7f743effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f74400d02e0 0x7f7440159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:51.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.476+0000 7f743ffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f74400cb200 0x7f74400cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45870/0 (socket says 172.21.15.47:45870) 2024-03-23T09:50:51.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.476+0000 7f743ffff640 1 -- 172.21.15.47:0/4290143600 learned_addr learned my addr 172.21.15.47:0/4290143600 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:51.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.476+0000 7f743effd640 1 -- 172.21.15.47:0/4290143600 >> v1:172.21.15.175:6789/0 conn(0x7f74400cbd20 legacy=0x7f74400cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:51.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.476+0000 7f743effd640 1 -- 172.21.15.47:0/4290143600 >> v1:172.21.15.47:6789/0 conn(0x7f74400cb200 legacy=0x7f74400cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:51.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.476+0000 7f743e7fc640 1 -- 172.21.15.47:0/4290143600 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717746843 0 0) 0x7f744015a060 con 0x7f74400cb200 2024-03-23T09:50:51.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.476+0000 7f743effd640 1 -- 172.21.15.47:0/4290143600 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74400caa00 con 0x7f74400d02e0 2024-03-23T09:50:51.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.477+0000 7f743effd640 1 --2- 172.21.15.47:0/4290143600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74400d02e0 0x7f7440159b20 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f74340097a0 tx=0x7f743402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0a694e82e809d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:51.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.477+0000 7f743e7fc640 1 -- 172.21.15.47:0/4290143600 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434030610 con 0x7f74400d02e0 2024-03-23T09:50:51.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.477+0000 7f743e7fc640 1 -- 172.21.15.47:0/4290143600 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7434030770 con 0x7f74400d02e0 2024-03-23T09:50:51.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.477+0000 7f743e7fc640 1 -- 172.21.15.47:0/4290143600 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434030b00 con 0x7f74400d02e0 2024-03-23T09:50:51.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.478+0000 7f7446251640 1 -- 172.21.15.47:0/4290143600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74400d02e0 msgr2=0x7f7440159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:51.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.478+0000 7f7446251640 1 --2- 172.21.15.47:0/4290143600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74400d02e0 0x7f7440159b20 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f74340097a0 tx=0x7f743402b960 comp rx=0 tx=0).stop 2024-03-23T09:50:51.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.478+0000 7f7446251640 1 -- 172.21.15.47:0/4290143600 shutdown_connections 2024-03-23T09:50:51.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.478+0000 7f7446251640 1 --2- 172.21.15.47:0/4290143600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74400d02e0 0x7f7440159b20 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:51.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.478+0000 7f7446251640 1 -- 172.21.15.47:0/4290143600 >> 172.21.15.47:0/4290143600 conn(0x7f7440098a00 msgr2=0x7f74400cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:51.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.478+0000 7f7446251640 1 -- 172.21.15.47:0/4290143600 shutdown_connections 2024-03-23T09:50:51.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.478+0000 7f7446251640 1 -- 172.21.15.47:0/4290143600 wait complete. 2024-03-23T09:50:51.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.479+0000 7f7446251640 1 Processor -- start 2024-03-23T09:50:51.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.479+0000 7f7446251640 1 -- start start 2024-03-23T09:50:51.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f7446251640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74400cb200 0x7f7440151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:51.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f7446251640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74400cbd20 0x7f74400aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:51.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f7446251640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74400d02e0 0x7f74400c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:51.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f7446251640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74401e3df0 con 0x7f74400cbd20 2024-03-23T09:50:51.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f7446251640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74401e40d0 con 0x7f74400d02e0 2024-03-23T09:50:51.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f7446251640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74401e43b0 con 0x7f74400cb200 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f743effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74400cbd20 0x7f74400aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f743effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74400cbd20 0x7f74400aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59578/0 (socket says 172.21.15.47:59578) 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f743effd640 1 -- 172.21.15.47:0/2353893716 learned_addr learned my addr 172.21.15.47:0/2353893716 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f743f7fe640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74400cb200 0x7f7440151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f743ffff640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74400d02e0 0x7f74400c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f743effd640 1 -- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74400cb200 msgr2=0x7f7440151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f743effd640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74400cb200 0x7f7440151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f743effd640 1 -- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74400d02e0 msgr2=0x7f74400c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f743effd640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74400d02e0 0x7f74400c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f743effd640 1 -- 172.21.15.47:0/2353893716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7434009400 con 0x7f74400cbd20 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f743f7fe640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74400cb200 0x7f7440151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.480+0000 7f743effd640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74400cbd20 0x7f74400aef40 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f74340098c0 tx=0x7f74340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.481+0000 7f7427fff640 1 -- 172.21.15.47:0/2353893716 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743403e070 con 0x7f74400cbd20 2024-03-23T09:50:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.481+0000 7f7427fff640 1 -- 172.21.15.47:0/2353893716 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7434039a10 con 0x7f74400cbd20 2024-03-23T09:50:51.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.481+0000 7f7427fff640 1 -- 172.21.15.47:0/2353893716 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434039da0 con 0x7f74400cbd20 2024-03-23T09:50:51.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.481+0000 7f7446251640 1 -- 172.21.15.47:0/2353893716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74401e4690 con 0x7f74400cbd20 2024-03-23T09:50:51.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.481+0000 7f7446251640 1 -- 172.21.15.47:0/2353893716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74400af480 con 0x7f74400cbd20 2024-03-23T09:50:51.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.482+0000 7f7446251640 1 -- 172.21.15.47:0/2353893716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74401e4bb0 con 0x7f74400cbd20 2024-03-23T09:50:51.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.482+0000 7f7427fff640 1 -- 172.21.15.47:0/2353893716 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f743404b400 con 0x7f74400cbd20 2024-03-23T09:50:51.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.482+0000 7f7427fff640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f742c06ac00 0x7f742c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:51.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.483+0000 7f7427fff640 1 -- 172.21.15.47:0/2353893716 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f74340ab8d0 con 0x7f74400cbd20 2024-03-23T09:50:51.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.483+0000 7f743f7fe640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f742c06ac00 0x7f742c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:51.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.483+0000 7f743f7fe640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f742c06ac00 0x7f742c06d0c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f74400c97d0 tx=0x7f7430009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.487+0000 7f7427fff640 1 -- 172.21.15.47:0/2353893716 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f74401e4bb0 con 0x7f74400cbd20 2024-03-23T09:50:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.705+0000 7f7446251640 1 -- 172.21.15.47:0/2353893716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74401e4bb0 con 0x7f74400cbd20 2024-03-23T09:50:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.706+0000 7f7427fff640 1 -- 172.21.15.47:0/2353893716 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f74401e4bb0 con 0x7f74400cbd20 2024-03-23T09:50:51.707 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:51.707 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":70,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/249416109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":249416109},{"type":"v1","addr":"172.21.15.175:6837","nonce":249416109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3603846321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3603846321},{"type":"v1","addr":"172.21.15.175:6835","nonce":3603846321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1921094794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1921094794},{"type":"v1","addr":"172.21.15.47:6834","nonce":1921094794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:49:43.247119+0000","modified":"2024-03-23T09:50:51.604191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8916},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8916":{"gid":8916,"name":"a","rank":0,"incarnation":68,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.47:6835/1748845936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1748845936},{"type":"v1","addr":"172.21.15.47:6835","nonce":1748845936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-23T09:50:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 70 2024-03-23T09:50:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.708+0000 7f7446251640 1 -- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f742c06ac00 msgr2=0x7f742c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.708+0000 7f7446251640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f742c06ac00 0x7f742c06d0c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f74400c97d0 tx=0x7f7430009040 comp rx=0 tx=0).stop 2024-03-23T09:50:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.708+0000 7f7446251640 1 -- 172.21.15.47:0/2353893716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74400cbd20 msgr2=0x7f74400aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.708+0000 7f7446251640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74400cbd20 0x7f74400aef40 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f74340098c0 tx=0x7f74340046c0 comp rx=0 tx=0).stop 2024-03-23T09:50:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.708+0000 7f7446251640 1 -- 172.21.15.47:0/2353893716 shutdown_connections 2024-03-23T09:50:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.708+0000 7f7446251640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f742c06ac00 0x7f742c06d0c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.708+0000 7f7446251640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74400d02e0 0x7f74400c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.708+0000 7f7446251640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74400cbd20 0x7f74400aef40 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.708+0000 7f7446251640 1 --2- 172.21.15.47:0/2353893716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74400cb200 0x7f7440151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.708+0000 7f7446251640 1 -- 172.21.15.47:0/2353893716 >> 172.21.15.47:0/2353893716 conn(0x7f7440098a00 msgr2=0x7f74400cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.708+0000 7f7446251640 1 -- 172.21.15.47:0/2353893716 shutdown_connections 2024-03-23T09:50:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:51.708+0000 7f7446251640 1 -- 172.21.15.47:0/2353893716 wait complete. 2024-03-23T09:50:52.122 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:49:43.247119+0000', 'modified': '2024-03-23T09:50:51.604191+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8916}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8916': {'gid': 8916, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.47:6835/1748845936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6833', 'nonce': 1748845936}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 1748845936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:50:52.123 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8916:up:rejoin 2024-03-23T09:50:53.124 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:50:53.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.237+0000 7f310491b640 1 Processor -- start 2024-03-23T09:50:53.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.237+0000 7f310491b640 1 -- start start 2024-03-23T09:50:53.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.238+0000 7f310491b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3100098a00 0x7f3100098de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:53.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.238+0000 7f310491b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f31000b1680 0x7f31000b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:53.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.238+0000 7f310491b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3100099320 con 0x7f310006bc50 2024-03-23T09:50:53.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.238+0000 7f310491b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31000b12c0 con 0x7f3100098a00 2024-03-23T09:50:53.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.238+0000 7f310491b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310006ae90 con 0x7f31000b1680 2024-03-23T09:50:53.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.238+0000 7f30fe59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3100098a00 0x7f3100098de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:53.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.238+0000 7f30fe59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3100098a00 0x7f3100098de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59584/0 (socket says 172.21.15.47:59584) 2024-03-23T09:50:53.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.238+0000 7f30fe59c640 1 -- 172.21.15.47:0/2835552918 learned_addr learned my addr 172.21.15.47:0/2835552918 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:53.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.238+0000 7f30f7fff640 1 --2- 172.21.15.47:0/2835552918 >> v2:172.21.15.175:3300/0 conn(0x7f31000b1680 0x7f31000b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:53.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.238+0000 7f30fe59c640 1 -- 172.21.15.47:0/2835552918 >> v2:172.21.15.175:3300/0 conn(0x7f31000b1680 msgr2=0x7f31000b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:53.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.238+0000 7f30fe59c640 1 --2- 172.21.15.47:0/2835552918 >> v2:172.21.15.175:3300/0 conn(0x7f31000b1680 0x7f31000b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.238+0000 7f30fe59c640 1 -- 172.21.15.47:0/2835552918 >> v1:172.21.15.175:6789/0 conn(0x7f310006bc50 legacy=0x7f31000ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:53.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.238+0000 7f30fe59c640 1 -- 172.21.15.47:0/2835552918 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310006b170 con 0x7f3100098a00 2024-03-23T09:50:53.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.239+0000 7f30fe59c640 1 --2- 172.21.15.47:0/2835552918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3100098a00 0x7f3100098de0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f31000b3240 tx=0x7f30f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84224caabf27bbdd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:53.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.239+0000 7f30fdd9b640 1 -- 172.21.15.47:0/2835552918 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0031870 con 0x7f3100098a00 2024-03-23T09:50:53.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.239+0000 7f30fdd9b640 1 -- 172.21.15.47:0/2835552918 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f00319d0 con 0x7f3100098a00 2024-03-23T09:50:53.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.239+0000 7f30fdd9b640 1 -- 172.21.15.47:0/2835552918 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0031d60 con 0x7f3100098a00 2024-03-23T09:50:53.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.240+0000 7f310491b640 1 -- 172.21.15.47:0/2835552918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3100098a00 msgr2=0x7f3100098de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:53.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.240+0000 7f310491b640 1 --2- 172.21.15.47:0/2835552918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3100098a00 0x7f3100098de0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f31000b3240 tx=0x7f30f002b910 comp rx=0 tx=0).stop 2024-03-23T09:50:53.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.240+0000 7f310491b640 1 -- 172.21.15.47:0/2835552918 shutdown_connections 2024-03-23T09:50:53.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.240+0000 7f310491b640 1 --2- 172.21.15.47:0/2835552918 >> v2:172.21.15.175:3300/0 conn(0x7f31000b1680 0x7f31000b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.240+0000 7f310491b640 1 --2- 172.21.15.47:0/2835552918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3100098a00 0x7f3100098de0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.240+0000 7f310491b640 1 -- 172.21.15.47:0/2835552918 >> 172.21.15.47:0/2835552918 conn(0x7f31000b2140 msgr2=0x7f31000b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:53.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.240+0000 7f310491b640 1 -- 172.21.15.47:0/2835552918 shutdown_connections 2024-03-23T09:50:53.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.240+0000 7f310491b640 1 -- 172.21.15.47:0/2835552918 wait complete. 2024-03-23T09:50:53.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f310491b640 1 Processor -- start 2024-03-23T09:50:53.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f310491b640 1 -- start start 2024-03-23T09:50:53.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f310491b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f310006bc50 0x7f310014fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:53.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f310491b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3100098a00 0x7f31000c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:53.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f310491b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31000b1680 0x7f31000aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:53.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f310491b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310009a3e0 con 0x7f310006bc50 2024-03-23T09:50:53.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f310491b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310009a6c0 con 0x7f31000b1680 2024-03-23T09:50:53.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f310491b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31000c6b90 con 0x7f3100098a00 2024-03-23T09:50:53.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f30fe59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f310006bc50 0x7f310014fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:53.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f30fe59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f310006bc50 0x7f310014fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59590/0 (socket says 172.21.15.47:59590) 2024-03-23T09:50:53.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f30fe59c640 1 -- 172.21.15.47:0/1996095815 learned_addr learned my addr 172.21.15.47:0/1996095815 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:53.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f30fed9d640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31000b1680 0x7f31000aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f30f7fff640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3100098a00 0x7f31000c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f30fe59c640 1 -- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3100098a00 msgr2=0x7f31000c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f30fe59c640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3100098a00 0x7f31000c6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f30fe59c640 1 -- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31000b1680 msgr2=0x7f31000aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f30fe59c640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31000b1680 0x7f31000aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f30fe59c640 1 -- 172.21.15.47:0/1996095815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f00094d0 con 0x7f310006bc50 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f30fed9d640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31000b1680 0x7f31000aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.241+0000 7f30fe59c640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f310006bc50 0x7f310014fa70 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f30f002b950 tx=0x7f30f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.242+0000 7f30f6ffd640 1 -- 172.21.15.47:0/1996095815 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f003ad30 con 0x7f310006bc50 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.242+0000 7f30f6ffd640 1 -- 172.21.15.47:0/1996095815 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f003ae90 con 0x7f310006bc50 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.242+0000 7f310491b640 1 -- 172.21.15.47:0/1996095815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31000c6e70 con 0x7f310006bc50 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.242+0000 7f30f6ffd640 1 -- 172.21.15.47:0/1996095815 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0044630 con 0x7f310006bc50 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.242+0000 7f310491b640 1 -- 172.21.15.47:0/1996095815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310014ffb0 con 0x7f310006bc50 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.244+0000 7f310491b640 1 -- 172.21.15.47:0/1996095815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31000b1d20 con 0x7f310006bc50 2024-03-23T09:50:53.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.244+0000 7f30f6ffd640 1 -- 172.21.15.47:0/1996095815 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f30f0044790 con 0x7f310006bc50 2024-03-23T09:50:53.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.244+0000 7f30f6ffd640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f30e006ac00 0x7f30e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:53.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.244+0000 7f30f6ffd640 1 -- 172.21.15.47:0/1996095815 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f30f002d040 con 0x7f310006bc50 2024-03-23T09:50:53.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.244+0000 7f30f7fff640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f30e006ac00 0x7f30e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:53.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.244+0000 7f30f7fff640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f30e006ac00 0x7f30e006d0c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3100152b90 tx=0x7f30e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:53.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.249+0000 7f30f6ffd640 1 -- 172.21.15.47:0/1996095815 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f30f0076c30 con 0x7f310006bc50 2024-03-23T09:50:53.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.469+0000 7f310491b640 1 -- 172.21.15.47:0/1996095815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31000afc80 con 0x7f310006bc50 2024-03-23T09:50:53.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.469+0000 7f30f6ffd640 1 -- 172.21.15.47:0/1996095815 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f30f007a250 con 0x7f310006bc50 2024-03-23T09:50:53.470 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:53.470 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/249416109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":249416109},{"type":"v1","addr":"172.21.15.175:6837","nonce":249416109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3603846321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3603846321},{"type":"v1","addr":"172.21.15.175:6835","nonce":3603846321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1921094794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1921094794},{"type":"v1","addr":"172.21.15.47:6834","nonce":1921094794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:49:43.247119+0000","modified":"2024-03-23T09:50:52.610350+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8916},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8916":{"gid":8916,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6835/1748845936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1748845936},{"type":"v1","addr":"172.21.15.47:6835","nonce":1748845936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-23T09:50:53.471 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 71 2024-03-23T09:50:53.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.471+0000 7f310491b640 1 -- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f30e006ac00 msgr2=0x7f30e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:53.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.471+0000 7f310491b640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f30e006ac00 0x7f30e006d0c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3100152b90 tx=0x7f30e8008040 comp rx=0 tx=0).stop 2024-03-23T09:50:53.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.471+0000 7f310491b640 1 -- 172.21.15.47:0/1996095815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f310006bc50 msgr2=0x7f310014fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:53.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.471+0000 7f310491b640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f310006bc50 0x7f310014fa70 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f30f002b950 tx=0x7f30f00046c0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.471+0000 7f310491b640 1 -- 172.21.15.47:0/1996095815 shutdown_connections 2024-03-23T09:50:53.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.471+0000 7f310491b640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f30e006ac00 0x7f30e006d0c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.471+0000 7f310491b640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31000b1680 0x7f31000aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.471+0000 7f310491b640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3100098a00 0x7f31000c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.471+0000 7f310491b640 1 --2- 172.21.15.47:0/1996095815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f310006bc50 0x7f310014fa70 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.471+0000 7f310491b640 1 -- 172.21.15.47:0/1996095815 >> 172.21.15.47:0/1996095815 conn(0x7f31000b2140 msgr2=0x7f31000b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:53.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.471+0000 7f310491b640 1 -- 172.21.15.47:0/1996095815 shutdown_connections 2024-03-23T09:50:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.471+0000 7f310491b640 1 -- 172.21.15.47:0/1996095815 wait complete. 2024-03-23T09:50:53.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:49:43.247119+0000', 'modified': '2024-03-23T09:50:52.610350+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8916}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8916': {'gid': 8916, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.47:6835/1748845936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6833', 'nonce': 1748845936}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 1748845936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:50:53.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:50:53.885 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-23T09:50:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.993+0000 7ff62c589640 1 Processor -- start 2024-03-23T09:50:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.993+0000 7ff62c589640 1 -- start start 2024-03-23T09:50:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff62c589640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62415cc40 0x7ff624160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff62c589640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff6240cddc0 0x7ff6240d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff62c589640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6240d09d0 con 0x7ff624160c90 2024-03-23T09:50:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff62c589640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6240b5a80 con 0x7ff6240cddc0 2024-03-23T09:50:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff62c589640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62415d020 con 0x7ff62415cc40 2024-03-23T09:50:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff629b24640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff624160c90 0x7ff6240cd6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45872/0 (socket says 172.21.15.47:45872) 2024-03-23T09:50:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff629b24640 1 -- 172.21.15.47:0/1580045917 learned_addr learned my addr 172.21.15.47:0/1580045917 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff62ab26640 1 --2- 172.21.15.47:0/1580045917 >> v2:172.21.15.175:3300/0 conn(0x7ff6240cddc0 0x7ff6240d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff62a325640 1 --2- 172.21.15.47:0/1580045917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62415cc40 0x7ff624160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819084740 0 0) 0x7ff6240d09d0 con 0x7ff624160c90 2024-03-23T09:50:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff610003520 con 0x7ff624160c90 2024-03-23T09:50:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2597840572 0 0) 0x7ff610003520 con 0x7ff624160c90 2024-03-23T09:50:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62415cc40 msgr2=0x7ff624160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff629323640 1 --2- 172.21.15.47:0/1580045917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62415cc40 0x7ff624160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.994+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 >> v2:172.21.15.175:3300/0 conn(0x7ff6240cddc0 msgr2=0x7ff6240d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff629323640 1 --2- 172.21.15.47:0/1580045917 >> v2:172.21.15.175:3300/0 conn(0x7ff6240cddc0 0x7ff6240d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6241611a0 con 0x7ff624160c90 2024-03-23T09:50:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7ff614001670 con 0x7ff624160c90 2024-03-23T09:50:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 >> v1:172.21.15.47:6789/0 conn(0x7ff624160c90 legacy=0x7ff6240cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff629323640 1 --2- 172.21.15.47:0/1580045917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62415cc40 0x7ff610003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff629323640 1 --2- 172.21.15.47:0/1580045917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff610003ef0 0x7ff6100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff629323640 1 --2- 172.21.15.47:0/1580045917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff610004d00 0x7ff610005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6240b5a80 con 0x7ff610004d00 2024-03-23T09:50:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62415d020 con 0x7ff610003ef0 2024-03-23T09:50:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61000b980 con 0x7ff62415cc40 2024-03-23T09:50:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff62ab26640 1 --2- 172.21.15.47:0/1580045917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff610004d00 0x7ff610005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff629b24640 1 --2- 172.21.15.47:0/1580045917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff610003ef0 0x7ff6100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff62a325640 1 --2- 172.21.15.47:0/1580045917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62415cc40 0x7ff610003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff62ab26640 1 -- 172.21.15.47:0/1580045917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62415cc40 msgr2=0x7ff610003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff62ab26640 1 --2- 172.21.15.47:0/1580045917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62415cc40 0x7ff610003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff62ab26640 1 -- 172.21.15.47:0/1580045917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff610003ef0 msgr2=0x7ff6100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff62ab26640 1 --2- 172.21.15.47:0/1580045917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff610003ef0 0x7ff6100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.995+0000 7ff62ab26640 1 -- 172.21.15.47:0/1580045917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff610003520 con 0x7ff610004d00 2024-03-23T09:50:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.996+0000 7ff62ab26640 1 --2- 172.21.15.47:0/1580045917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff610004d00 0x7ff610005180 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ff62408bd80 tx=0x7ff6100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa9458a45856242 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.996+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c02cbc0 con 0x7ff610004d00 2024-03-23T09:50:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.996+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61c00f040 con 0x7ff610004d00 2024-03-23T09:50:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.996+0000 7ff629323640 1 -- 172.21.15.47:0/1580045917 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c034440 con 0x7ff610004d00 2024-03-23T09:50:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.996+0000 7ff62c589640 1 -- 172.21.15.47:0/1580045917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff610004d00 msgr2=0x7ff610005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.996+0000 7ff62c589640 1 --2- 172.21.15.47:0/1580045917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff610004d00 0x7ff610005180 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ff62408bd80 tx=0x7ff6100057f0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.996+0000 7ff62ab26640 1 -- 172.21.15.47:0/1580045917 reap_dead start 2024-03-23T09:50:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.997+0000 7ff62c589640 1 -- 172.21.15.47:0/1580045917 shutdown_connections 2024-03-23T09:50:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.997+0000 7ff62c589640 1 -- 172.21.15.47:0/1580045917 >> 172.21.15.47:0/1580045917 conn(0x7ff62406b160 msgr2=0x7ff62406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.997+0000 7ff62c589640 1 -- 172.21.15.47:0/1580045917 shutdown_connections 2024-03-23T09:50:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.997+0000 7ff62c589640 1 -- 172.21.15.47:0/1580045917 wait complete. 2024-03-23T09:50:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.998+0000 7ff62c589640 1 Processor -- start 2024-03-23T09:50:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.998+0000 7ff62c589640 1 -- start start 2024-03-23T09:50:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62c589640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff610004d00 0x7ff62415eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62c589640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62415fc10 0x7ff624155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62c589640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62415bb10 0x7ff62415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62c589640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241588b0 con 0x7ff610004d00 2024-03-23T09:50:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62c589640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff624160070 con 0x7ff62415fc10 2024-03-23T09:50:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62c589640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62415f110 con 0x7ff62415bb10 2024-03-23T09:50:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62a325640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff610004d00 0x7ff62415eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62a325640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff610004d00 0x7ff62415eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59608/0 (socket says 172.21.15.47:59608) 2024-03-23T09:50:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62a325640 1 -- 172.21.15.47:0/168431458 learned_addr learned my addr 172.21.15.47:0/168431458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff629b24640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62415fc10 0x7ff624155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62ab26640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62415bb10 0x7ff62415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62a325640 1 -- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62415bb10 msgr2=0x7ff62415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62a325640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62415bb10 0x7ff62415bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62a325640 1 -- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62415fc10 msgr2=0x7ff624155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62a325640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62415fc10 0x7ff624155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62a325640 1 -- 172.21.15.47:0/168431458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff624158b00 con 0x7ff610004d00 2024-03-23T09:50:54.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff629b24640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62415fc10 0x7ff624155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:54.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62a325640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff610004d00 0x7ff62415eac0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff6200119a0 tx=0x7ff620011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff61affd640 1 -- 172.21.15.47:0/168431458 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62000faf0 con 0x7ff610004d00 2024-03-23T09:50:54.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff61affd640 1 -- 172.21.15.47:0/168431458 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62000fc50 con 0x7ff610004d00 2024-03-23T09:50:54.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62c589640 1 -- 172.21.15.47:0/168431458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff624158de0 con 0x7ff610004d00 2024-03-23T09:50:54.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff61affd640 1 -- 172.21.15.47:0/168431458 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff620014440 con 0x7ff610004d00 2024-03-23T09:50:54.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:53.999+0000 7ff62c589640 1 -- 172.21.15.47:0/168431458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62415c4d0 con 0x7ff610004d00 2024-03-23T09:50:54.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.001+0000 7ff61affd640 1 -- 172.21.15.47:0/168431458 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff6200145a0 con 0x7ff610004d00 2024-03-23T09:50:54.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.001+0000 7ff61affd640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff60806ac30 0x7ff60806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.002+0000 7ff61affd640 1 -- 172.21.15.47:0/168431458 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff620089c40 con 0x7ff610004d00 2024-03-23T09:50:54.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.002+0000 7ff629b24640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff60806ac30 0x7ff60806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.002+0000 7ff629b24640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff60806ac30 0x7ff60806d0f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff6240cfe10 tx=0x7ff614009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.002+0000 7ff62f1d4740 1 Processor -- start 2024-03-23T09:50:54.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.003+0000 7ff62f1d4740 1 -- start start 2024-03-23T09:50:54.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.003+0000 7ff62f1d4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f4d750 0x557584f04d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.003+0000 7ff62f1d4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f037d0 0x557584f052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.003+0000 7ff62f1d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584effc70 0x557584f2fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.003+0000 7ff62f1d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557584e709b0 con 0x557584effc70 2024-03-23T09:50:54.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.003+0000 7ff62f1d4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557584e6e140 con 0x557584f037d0 2024-03-23T09:50:54.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.003+0000 7ff62f1d4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557584e69a30 con 0x557584f4d750 2024-03-23T09:50:54.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.003+0000 7ff62ab26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584effc70 0x557584f2fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.003+0000 7ff62ab26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584effc70 0x557584f2fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59614/0 (socket says 172.21.15.47:59614) 2024-03-23T09:50:54.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.003+0000 7ff62ab26640 1 -- 172.21.15.47:0/1224655396 learned_addr learned my addr 172.21.15.47:0/1224655396 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.004+0000 7ff62a325640 1 --2- 172.21.15.47:0/1224655396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f4d750 0x557584f04d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.004+0000 7ff62ab26640 1 -- 172.21.15.47:0/1224655396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f4d750 msgr2=0x557584f04d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.004+0000 7ff62ab26640 1 --2- 172.21.15.47:0/1224655396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f4d750 0x557584f04d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.004+0000 7ff62ab26640 1 -- 172.21.15.47:0/1224655396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f037d0 msgr2=0x557584f052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.004+0000 7ff62ab26640 1 --2- 172.21.15.47:0/1224655396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f037d0 0x557584f052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.004+0000 7ff62ab26640 1 -- 172.21.15.47:0/1224655396 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557584df62f0 con 0x557584effc70 2024-03-23T09:50:54.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.004+0000 7ff62ab26640 1 --2- 172.21.15.47:0/1224655396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584effc70 0x557584f2fa60 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7ff61c037040 tx=0x7ff61c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b521643675e2b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.004+0000 7ff607fff640 1 -- 172.21.15.47:0/1224655396 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c02cd00 con 0x557584effc70 2024-03-23T09:50:54.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.004+0000 7ff607fff640 1 -- 172.21.15.47:0/1224655396 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61c02ce60 con 0x557584effc70 2024-03-23T09:50:54.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.004+0000 7ff607fff640 1 -- 172.21.15.47:0/1224655396 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c034640 con 0x557584effc70 2024-03-23T09:50:54.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.004+0000 7ff62f1d4740 1 -- 172.21.15.47:0/1224655396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584effc70 msgr2=0x557584f2fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.004+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/1224655396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584effc70 0x557584f2fa60 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7ff61c037040 tx=0x7ff61c02e070 comp rx=0 tx=0).stop 2024-03-23T09:50:54.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.005+0000 7ff62f1d4740 1 -- 172.21.15.47:0/1224655396 shutdown_connections 2024-03-23T09:50:54.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.005+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/1224655396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584effc70 0x557584f2fa60 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.005+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/1224655396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f037d0 0x557584f052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.005+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/1224655396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f4d750 0x557584f04d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.005+0000 7ff62f1d4740 1 -- 172.21.15.47:0/1224655396 >> 172.21.15.47:0/1224655396 conn(0x557584f01ee0 msgr2=0x557584f4aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.005+0000 7ff62f1d4740 1 -- 172.21.15.47:0/1224655396 shutdown_connections 2024-03-23T09:50:54.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.005+0000 7ff62f1d4740 1 -- 172.21.15.47:0/1224655396 wait complete. 2024-03-23T09:50:54.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.005+0000 7ff62f1d4740 1 -- start start 2024-03-23T09:50:54.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.005+0000 7ff62f1d4740 1 Processor -- start 2024-03-23T09:50:54.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.006+0000 7ff62f1d4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f037d0 0x557584f27770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.006+0000 7ff62f1d4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f4d750 0x557584f51480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.006+0000 7ff62f1d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 0x557584f4bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.006+0000 7ff62f1d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557584e28090 con 0x557584f27ce0 2024-03-23T09:50:54.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.006+0000 7ff62f1d4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557584e27d30 con 0x557584f4d750 2024-03-23T09:50:54.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.006+0000 7ff62f1d4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557584ded2a0 con 0x557584f037d0 2024-03-23T09:50:54.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.006+0000 7ff62f1d4740 10 client.0 _ll_register_callbacks cb 0x557584e23050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:50:54.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.006+0000 7ff62ab26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 0x557584f4bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.006+0000 7ff62ab26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 0x557584f4bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59622/0 (socket says 172.21.15.47:59622) 2024-03-23T09:50:54.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.006+0000 7ff62ab26640 1 -- 172.21.15.47:0/3256471670 learned_addr learned my addr 172.21.15.47:0/3256471670 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62ab26640 1 -- 172.21.15.47:0/3256471670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f037d0 msgr2=0x557584f27770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62ab26640 1 --2- 172.21.15.47:0/3256471670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f037d0 0x557584f27770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62ab26640 1 -- 172.21.15.47:0/3256471670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f4d750 msgr2=0x557584f51480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62ab26640 1 --2- 172.21.15.47:0/3256471670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f4d750 0x557584f51480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62ab26640 1 -- 172.21.15.47:0/3256471670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff610003520 con 0x557584f27ce0 2024-03-23T09:50:54.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62ab26640 1 --2- 172.21.15.47:0/3256471670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 0x557584f4bc90 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00e230 tx=0x7ff61c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3f269346ffb23f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff6057fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:50:54.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff6057fa640 1 -- 172.21.15.47:0/3256471670 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c049b60 con 0x557584f27ce0 2024-03-23T09:50:54.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff6057fa640 1 -- 172.21.15.47:0/3256471670 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff61c049cc0 con 0x557584f27ce0 2024-03-23T09:50:54.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62f1d4740 1 -- 172.21.15.47:0/3256471670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557584b5d3c0 con 0x557584f27ce0 2024-03-23T09:50:54.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff6057fa640 1 -- 172.21.15.47:0/3256471670 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff61c049e80 con 0x557584f27ce0 2024-03-23T09:50:54.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62f1d4740 10 client.8994 fetch_fsmap learned FSMap version 71 2024-03-23T09:50:54.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62f1d4740 1 -- 172.21.15.47:0/3256471670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x557584e680b0 con 0x557584f27ce0 2024-03-23T09:50:54.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff6057fa640 1 -- 172.21.15.47:0/3256471670 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557584b5d3c0 con 0x557584f27ce0 2024-03-23T09:50:54.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff6057fa640 1 -- 172.21.15.47:0/3256471670 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff61c034440 con 0x557584f27ce0 2024-03-23T09:50:54.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62f1d4740 10 client.8994 fetch_fsmap finished waiting for FSMap version 71 2024-03-23T09:50:54.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62f1d4740 10 client.8994 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-23T09:50:54.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62f1d4740 20 client.8994 populate_metadata read hostname 'smithi047' 2024-03-23T09:50:54.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/3256471670 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x557584f2fa70 0x557584f4f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62f1d4740 4 client.8994 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:50:54.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62f1d4740 1 -- 172.21.15.47:0/3256471670 --> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557584b6e6c0 con 0x557584f2fa70 2024-03-23T09:50:54.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.007+0000 7ff62a325640 1 --2- 172.21.15.47:0/3256471670 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x557584f2fa70 0x557584f4f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.008+0000 7ff62a325640 1 --2- 172.21.15.47:0/3256471670 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x557584f2fa70 0x557584f4f130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c22218fd90a84966 server_cookie=b544bfce9408e451 in_seq=0 out_seq=0 2024-03-23T09:50:54.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.008+0000 7ff6057fa640 10 client.8994 ms_handle_connect on v2:172.21.15.47:6833/1748845936 2024-03-23T09:50:54.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.008+0000 7ff6057fa640 1 -- 172.21.15.47:0/3256471670 <== mds.0 v2:172.21.15.47:6833/1748845936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557584b6e6c0 con 0x557584f2fa70 2024-03-23T09:50:54.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.008+0000 7ff6057fa640 10 client.8994 handle_command_reply: tid=0 2024-03-23T09:50:54.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.008+0000 7ff62f1d4740 1 client.8994 shutdown 2024-03-23T09:50:54.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.008+0000 7ff62f1d4740 1 -- 172.21.15.47:0/3256471670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 msgr2=0x557584f4bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.008+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/3256471670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 0x557584f4bc90 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00e230 tx=0x7ff61c004a40 comp rx=0 tx=0).stop 2024-03-23T09:50:54.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.009+0000 7ff62f1d4740 1 -- 172.21.15.47:0/3256471670 shutdown_connections 2024-03-23T09:50:54.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.009+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/3256471670 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x557584f2fa70 0x557584f4f130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.009+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/3256471670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 0x557584f4bc90 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.009+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/3256471670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f4d750 0x557584f51480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.009+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/3256471670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f037d0 0x557584f27770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.009+0000 7ff6057fa640 0 client.8994 ms_handle_reset on v2:172.21.15.47:6833/1748845936 2024-03-23T09:50:54.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.009+0000 7ff62f1d4740 1 -- 172.21.15.47:0/3256471670 >> 172.21.15.47:0/3256471670 conn(0x557584effc70 msgr2=0x557584f04080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.009+0000 7ff62f1d4740 1 -- 172.21.15.47:0/3256471670 shutdown_connections 2024-03-23T09:50:54.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.009+0000 7ff62f1d4740 1 -- 172.21.15.47:0/3256471670 wait complete. 2024-03-23T09:50:54.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.009+0000 7ff62f1d4740 20 client.8994 trim_cache size 0 max 16384 2024-03-23T09:50:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.024+0000 7ff62f1d4740 1 Processor -- start 2024-03-23T09:50:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.024+0000 7ff62f1d4740 1 -- start start 2024-03-23T09:50:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.024+0000 7ff62f1d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f037d0 0x557584f4bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.024+0000 7ff62f1d4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f27ce0 0x557584f4b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.024+0000 7ff62f1d4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f4d750 0x557584f4c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.024+0000 7ff62f1d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557584f048f0 con 0x557584f037d0 2024-03-23T09:50:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.024+0000 7ff62f1d4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557584f269a0 con 0x557584f27ce0 2024-03-23T09:50:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.024+0000 7ff62f1d4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557584f072b0 con 0x557584f4d750 2024-03-23T09:50:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.024+0000 7ff62a325640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f037d0 0x557584f4bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff62a325640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f037d0 0x557584f4bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59638/0 (socket says 172.21.15.47:59638) 2024-03-23T09:50:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff62a325640 1 -- 172.21.15.47:0/3974720381 learned_addr learned my addr 172.21.15.47:0/3974720381 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff62a325640 1 -- 172.21.15.47:0/3974720381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f4d750 msgr2=0x557584f4c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff62ab26640 1 --2- 172.21.15.47:0/3974720381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f4d750 0x557584f4c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff629b24640 1 --2- 172.21.15.47:0/3974720381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f27ce0 0x557584f4b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff62a325640 1 --2- 172.21.15.47:0/3974720381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f4d750 0x557584f4c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff62a325640 1 -- 172.21.15.47:0/3974720381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f27ce0 msgr2=0x557584f4b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff62a325640 1 --2- 172.21.15.47:0/3974720381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f27ce0 0x557584f4b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff62a325640 1 -- 172.21.15.47:0/3974720381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557584e0a430 con 0x557584f037d0 2024-03-23T09:50:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff629b24640 1 --2- 172.21.15.47:0/3974720381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f27ce0 0x557584f4b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff62a325640 1 --2- 172.21.15.47:0/3974720381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f037d0 0x557584f4bc00 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7ff620058780 tx=0x7ff620087fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8085d3ada3f12f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff618ff9640 1 -- 172.21.15.47:0/3974720381 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff620060e70 con 0x557584f037d0 2024-03-23T09:50:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff618ff9640 1 -- 172.21.15.47:0/3974720381 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff620056bc0 con 0x557584f037d0 2024-03-23T09:50:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.025+0000 7ff618ff9640 1 -- 172.21.15.47:0/3974720381 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff620055190 con 0x557584f037d0 2024-03-23T09:50:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.026+0000 7ff62f1d4740 1 -- 172.21.15.47:0/3974720381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f037d0 msgr2=0x557584f4bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.026+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/3974720381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f037d0 0x557584f4bc00 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7ff620058780 tx=0x7ff620087fe0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.026+0000 7ff62f1d4740 1 -- 172.21.15.47:0/3974720381 shutdown_connections 2024-03-23T09:50:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.026+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/3974720381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f4d750 0x557584f4c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.026+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/3974720381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f27ce0 0x557584f4b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.026+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/3974720381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f037d0 0x557584f4bc00 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.026+0000 7ff62f1d4740 1 -- 172.21.15.47:0/3974720381 >> 172.21.15.47:0/3974720381 conn(0x557584effc70 msgr2=0x557584f26b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.026+0000 7ff62f1d4740 1 -- 172.21.15.47:0/3974720381 shutdown_connections 2024-03-23T09:50:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.026+0000 7ff62f1d4740 1 -- 172.21.15.47:0/3974720381 wait complete. 2024-03-23T09:50:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.026+0000 7ff62f1d4740 1 -- start start 2024-03-23T09:50:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.026+0000 7ff62f1d4740 1 Processor -- start 2024-03-23T09:50:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.027+0000 7ff62f1d4740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f037d0 0x557584f40e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.027+0000 7ff62f1d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 0x557584f4c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.027+0000 7ff62f1d4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f4d750 0x557584f4cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.027+0000 7ff62f1d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557584fb0f80 con 0x557584f27ce0 2024-03-23T09:50:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.027+0000 7ff62f1d4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557584f51eb0 con 0x557584f4d750 2024-03-23T09:50:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.027+0000 7ff62f1d4740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x557584f41340 con 0x557584f037d0 2024-03-23T09:50:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.027+0000 7ff62f1d4740 10 client.0 _ll_register_callbacks cb 0x7ff61c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:50:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.027+0000 7ff629b24640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 0x557584f4c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff629b24640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 0x557584f4c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59654/0 (socket says 172.21.15.47:59654) 2024-03-23T09:50:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff629b24640 1 -- 172.21.15.47:0/210987997 learned_addr learned my addr 172.21.15.47:0/210987997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff62ab26640 1 --2- 172.21.15.47:0/210987997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f4d750 0x557584f4cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff62a325640 1 --2- 172.21.15.47:0/210987997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f037d0 0x557584f40e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff629b24640 1 -- 172.21.15.47:0/210987997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f037d0 msgr2=0x557584f40e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff629b24640 1 --2- 172.21.15.47:0/210987997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f037d0 0x557584f40e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff629b24640 1 -- 172.21.15.47:0/210987997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f4d750 msgr2=0x557584f4cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff629b24640 1 --2- 172.21.15.47:0/210987997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f4d750 0x557584f4cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff629b24640 1 -- 172.21.15.47:0/210987997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62415c4d0 con 0x557584f27ce0 2024-03-23T09:50:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff629b24640 1 --2- 172.21.15.47:0/210987997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 0x557584f4c9d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7ff6241610d0 tx=0x7ff614015b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b27327588286c9f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff6057fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:50:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff6057fa640 1 -- 172.21.15.47:0/210987997 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614014820 con 0x557584f27ce0 2024-03-23T09:50:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff6057fa640 1 -- 172.21.15.47:0/210987997 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff614014980 con 0x557584f27ce0 2024-03-23T09:50:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff62f1d4740 1 -- 172.21.15.47:0/210987997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff61c02ce60 con 0x557584f27ce0 2024-03-23T09:50:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff6057fa640 1 -- 172.21.15.47:0/210987997 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff614014b40 con 0x557584f27ce0 2024-03-23T09:50:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff62f1d4740 10 client.9000 fetch_fsmap learned FSMap version 71 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.028+0000 7ff62f1d4740 1 -- 172.21.15.47:0/210987997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x557584e2fb40 con 0x557584f27ce0 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.029+0000 7ff6057fa640 1 -- 172.21.15.47:0/210987997 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff614030020 con 0x557584f27ce0 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.029+0000 7ff6057fa640 1 -- 172.21.15.47:0/210987997 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff614014ca0 con 0x557584f27ce0 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.029+0000 7ff62f1d4740 10 client.9000 fetch_fsmap finished waiting for FSMap version 71 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.029+0000 7ff62f1d4740 10 client.9000 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.029+0000 7ff62f1d4740 20 client.9000 populate_metadata read hostname 'smithi047' 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.029+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/210987997 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x557584fb3d50 0x557584fb4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.029+0000 7ff62f1d4740 4 client.9000 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.029+0000 7ff62f1d4740 1 -- 172.21.15.47:0/210987997 --> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557584bc1d10 con 0x557584fb3d50 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.029+0000 7ff62a325640 1 --2- 172.21.15.47:0/210987997 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x557584fb3d50 0x557584fb4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.029+0000 7ff62a325640 1 --2- 172.21.15.47:0/210987997 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x557584fb3d50 0x557584fb4130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da3ae53cb857eaf9 server_cookie=7099345bf215f256 in_seq=0 out_seq=0 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.029+0000 7ff6057fa640 10 client.9000 ms_handle_connect on v2:172.21.15.47:6833/1748845936 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff6057fa640 1 -- 172.21.15.47:0/210987997 <== mds.0 v2:172.21.15.47:6833/1748845936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x557584bc1d10 con 0x557584fb3d50 2024-03-23T09:50:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff6057fa640 10 client.9000 handle_command_reply: tid=0 2024-03-23T09:50:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff62f1d4740 1 client.9000 shutdown 2024-03-23T09:50:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff62f1d4740 1 -- 172.21.15.47:0/210987997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 msgr2=0x557584f4c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/210987997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 0x557584f4c9d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7ff6241610d0 tx=0x7ff614015b40 comp rx=0 tx=0).stop 2024-03-23T09:50:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff62f1d4740 1 -- 172.21.15.47:0/210987997 shutdown_connections 2024-03-23T09:50:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/210987997 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x557584fb3d50 0x557584fb4130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/210987997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557584f4d750 0x557584f4cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/210987997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557584f27ce0 0x557584f4c9d0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff62f1d4740 1 --2- 172.21.15.47:0/210987997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557584f037d0 0x557584f40e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff6057fa640 0 client.9000 ms_handle_reset on v2:172.21.15.47:6833/1748845936 2024-03-23T09:50:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff62f1d4740 1 -- 172.21.15.47:0/210987997 >> 172.21.15.47:0/210987997 conn(0x557584effc70 msgr2=0x557584f2e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff62f1d4740 1 -- 172.21.15.47:0/210987997 shutdown_connections 2024-03-23T09:50:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff62f1d4740 1 -- 172.21.15.47:0/210987997 wait complete. 2024-03-23T09:50:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.030+0000 7ff62f1d4740 20 client.9000 trim_cache size 0 max 16384 2024-03-23T09:50:54.038 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:50:54.038 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:50:54.038 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:50:54.038 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 8916, 2024-03-23T09:50:54.038 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:50:54.038 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:50:54.038 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:50:54.038 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 4.4256595909999996, 2024-03-23T09:50:54.038 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 71, 2024-03-23T09:50:54.039 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 102, 2024-03-23T09:50:54.039 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 102, 2024-03-23T09:50:54.039 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 5.3970364689999997 2024-03-23T09:50:54.039 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:50:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.031+0000 7ff62c589640 1 -- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff60806ac30 msgr2=0x7ff60806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.031+0000 7ff62c589640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff60806ac30 0x7ff60806d0f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff6240cfe10 tx=0x7ff614009040 comp rx=0 tx=0).stop 2024-03-23T09:50:54.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.031+0000 7ff62c589640 1 -- 172.21.15.47:0/168431458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff610004d00 msgr2=0x7ff62415eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.031+0000 7ff62c589640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff610004d00 0x7ff62415eac0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff6200119a0 tx=0x7ff620011e60 comp rx=0 tx=0).stop 2024-03-23T09:50:54.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.031+0000 7ff62c589640 1 -- 172.21.15.47:0/168431458 shutdown_connections 2024-03-23T09:50:54.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.031+0000 7ff62c589640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff60806ac30 0x7ff60806d0f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.031+0000 7ff62c589640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff62415bb10 0x7ff62415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.031+0000 7ff62c589640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff62415fc10 0x7ff624155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.031+0000 7ff62c589640 1 --2- 172.21.15.47:0/168431458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff610004d00 0x7ff62415eac0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.031+0000 7ff62c589640 1 -- 172.21.15.47:0/168431458 >> 172.21.15.47:0/168431458 conn(0x7ff62406b160 msgr2=0x7ff6240cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.031+0000 7ff62c589640 1 -- 172.21.15.47:0/168431458 shutdown_connections 2024-03-23T09:50:54.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.031+0000 7ff62c589640 1 -- 172.21.15.47:0/168431458 wait complete. 2024-03-23T09:50:54.041 DEBUG:teuthology.orchestra.run.smithi047:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-23T09:50:54.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:54.044 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-03-23T09:50:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.149+0000 7fa0061ce640 1 Processor -- start 2024-03-23T09:50:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.149+0000 7fa0061ce640 1 -- start start 2024-03-23T09:50:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7fa0061ce640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0000cb460 0x7fa0000af950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7fa0061ce640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0000b1bf0 0x7fa00006a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7fa0061ce640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0000af4c0 con 0x7fa0000b0280 2024-03-23T09:50:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7fa0061ce640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0000aef40 con 0x7fa0000cb460 2024-03-23T09:50:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7fa0061ce640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000afe90 con 0x7fa0000b1bf0 2024-03-23T09:50:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7f9fff7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0000cb460 0x7fa0000af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7f9ffffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa0000b0280 0x7fa0000cbe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45880/0 (socket says 172.21.15.47:45880) 2024-03-23T09:50:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7f9ffffff640 1 -- 172.21.15.47:0/3625232520 learned_addr learned my addr 172.21.15.47:0/3625232520 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7f9fff7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0000cb460 0x7fa0000af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59670/0 (socket says 172.21.15.47:59670) 2024-03-23T09:50:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7f9ff7fff640 1 --2- 172.21.15.47:0/3625232520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0000b1bf0 0x7fa00006a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7f9fff7fe640 1 -- 172.21.15.47:0/3625232520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0000b1bf0 msgr2=0x7fa00006a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/3625232520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0000b1bf0 0x7fa00006a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7f9fff7fe640 1 -- 172.21.15.47:0/3625232520 >> v1:172.21.15.47:6789/0 conn(0x7fa0000b0280 legacy=0x7fa0000cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7f9fff7fe640 1 -- 172.21.15.47:0/3625232520 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00006ac30 con 0x7fa0000cb460 2024-03-23T09:50:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.150+0000 7f9ffeffd640 1 -- 172.21.15.47:0/3625232520 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439986711 0 0) 0x7fa0000af4c0 con 0x7fa0000b0280 2024-03-23T09:50:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.151+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/3625232520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0000cb460 0x7fa0000af950 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00097a0 tx=0x7f9ff002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988e630cf5da6f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.152+0000 7f9ffeffd640 1 -- 172.21.15.47:0/3625232520 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff002e070 con 0x7fa0000cb460 2024-03-23T09:50:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.152+0000 7f9ffeffd640 1 -- 172.21.15.47:0/3625232520 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff00315a0 con 0x7fa0000cb460 2024-03-23T09:50:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.152+0000 7f9ffeffd640 1 -- 172.21.15.47:0/3625232520 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0031930 con 0x7fa0000cb460 2024-03-23T09:50:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.152+0000 7fa0061ce640 1 -- 172.21.15.47:0/3625232520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0000cb460 msgr2=0x7fa0000af950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.152+0000 7fa0061ce640 1 --2- 172.21.15.47:0/3625232520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0000cb460 0x7fa0000af950 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00097a0 tx=0x7f9ff002b840 comp rx=0 tx=0).stop 2024-03-23T09:50:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.152+0000 7fa0061ce640 1 -- 172.21.15.47:0/3625232520 shutdown_connections 2024-03-23T09:50:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.152+0000 7fa0061ce640 1 --2- 172.21.15.47:0/3625232520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0000b1bf0 0x7fa00006a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.152+0000 7fa0061ce640 1 --2- 172.21.15.47:0/3625232520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0000cb460 0x7fa0000af950 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.152+0000 7fa0061ce640 1 -- 172.21.15.47:0/3625232520 >> 172.21.15.47:0/3625232520 conn(0x7fa000098a00 msgr2=0x7fa0000b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.152+0000 7fa0061ce640 1 -- 172.21.15.47:0/3625232520 shutdown_connections 2024-03-23T09:50:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.152+0000 7fa0061ce640 1 -- 172.21.15.47:0/3625232520 wait complete. 2024-03-23T09:50:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.154+0000 7fa0061ce640 1 Processor -- start 2024-03-23T09:50:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.154+0000 7fa0061ce640 1 -- start start 2024-03-23T09:50:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.154+0000 7fa0061ce640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0000b0280 0x7fa0000b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.154+0000 7fa0061ce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0000b1bf0 0x7fa00015e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.154+0000 7fa0061ce640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0000cb460 0x7fa0001594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.154+0000 7fa0061ce640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00015eb00 con 0x7fa0000b0280 2024-03-23T09:50:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.154+0000 7fa0061ce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa000159a20 con 0x7fa0000b1bf0 2024-03-23T09:50:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.154+0000 7fa0061ce640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00015d520 con 0x7fa0000cb460 2024-03-23T09:50:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.154+0000 7f9fff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0000b0280 0x7fa0000b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.154+0000 7f9fff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0000b0280 0x7fa0000b4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59674/0 (socket says 172.21.15.47:59674) 2024-03-23T09:50:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.154+0000 7f9fff7fe640 1 -- 172.21.15.47:0/1129764422 learned_addr learned my addr 172.21.15.47:0/1129764422 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.155+0000 7f9fff7fe640 1 -- 172.21.15.47:0/1129764422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0000cb460 msgr2=0x7fa0001594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.155+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/1129764422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0000cb460 0x7fa0001594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.155+0000 7f9fff7fe640 1 -- 172.21.15.47:0/1129764422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0000b1bf0 msgr2=0x7fa00015e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.155+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/1129764422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0000b1bf0 0x7fa00015e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.155+0000 7f9fff7fe640 1 -- 172.21.15.47:0/1129764422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff0009400 con 0x7fa0000b0280 2024-03-23T09:50:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.155+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/1129764422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0000b0280 0x7fa0000b4a90 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0034040 tx=0x7f9ff00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.155+0000 7f9ffcff9640 1 -- 172.21.15.47:0/1129764422 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0032840 con 0x7fa0000b0280 2024-03-23T09:50:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.155+0000 7f9ffcff9640 1 -- 172.21.15.47:0/1129764422 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff0031f00 con 0x7fa0000b0280 2024-03-23T09:50:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.155+0000 7fa0061ce640 1 -- 172.21.15.47:0/1129764422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00015d800 con 0x7fa0000b0280 2024-03-23T09:50:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.155+0000 7f9ffcff9640 1 -- 172.21.15.47:0/1129764422 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0032be0 con 0x7fa0000b0280 2024-03-23T09:50:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.155+0000 7fa0061ce640 1 -- 172.21.15.47:0/1129764422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00015ddc0 con 0x7fa0000b0280 2024-03-23T09:50:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.157+0000 7f9ffcff9640 1 -- 172.21.15.47:0/1129764422 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9ff0032d40 con 0x7fa0000b0280 2024-03-23T09:50:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.157+0000 7f9ffcff9640 1 --2- 172.21.15.47:0/1129764422 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9fe006ac30 0x7f9fe006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.157+0000 7f9ffcff9640 1 -- 172.21.15.47:0/1129764422 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9ff00ab320 con 0x7fa0000b0280 2024-03-23T09:50:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.157+0000 7f9ff7fff640 1 --2- 172.21.15.47:0/1129764422 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9fe006ac30 0x7f9fe006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.158+0000 7f9ff7fff640 1 --2- 172.21.15.47:0/1129764422 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9fe006ac30 0x7f9fe006d0f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8006a90 tx=0x7f9fe8006a20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.158+0000 7fa008e17740 1 Processor -- start 2024-03-23T09:50:54.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.158+0000 7fa008e17740 1 -- start start 2024-03-23T09:50:54.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7fa008e17740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b7d2f0 0x556112b7d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7fa008e17740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112b81340 0x556112b82de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7fa008e17740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bad080 0x556112bad500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7fa008e17740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556112aee940 con 0x556112b7d2f0 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7fa008e17740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556112aec0d0 con 0x556112b81340 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7fa008e17740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556112ae79c0 con 0x556112bad080 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7f9fff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b7d2f0 0x556112b7d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7f9fff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b7d2f0 0x556112b7d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59676/0 (socket says 172.21.15.47:59676) 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7f9fff7fe640 1 -- 172.21.15.47:0/1758521292 learned_addr learned my addr 172.21.15.47:0/1758521292 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7f9ff7fff640 1 --2- 172.21.15.47:0/1758521292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112b81340 0x556112b82de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7f9ffffff640 1 --2- 172.21.15.47:0/1758521292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bad080 0x556112bad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7f9fff7fe640 1 -- 172.21.15.47:0/1758521292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bad080 msgr2=0x556112bad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/1758521292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bad080 0x556112bad500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7f9fff7fe640 1 -- 172.21.15.47:0/1758521292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112b81340 msgr2=0x556112b82de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/1758521292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112b81340 0x556112b82de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7f9fff7fe640 1 -- 172.21.15.47:0/1758521292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556112ab38a0 con 0x556112b7d2f0 2024-03-23T09:50:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.159+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/1758521292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b7d2f0 0x556112b7d6d0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00a9810 tx=0x7f9ff0049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed3dc1b52e013b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.160+0000 7f9ff57fa640 1 -- 172.21.15.47:0/1758521292 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0082550 con 0x556112b7d2f0 2024-03-23T09:50:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.160+0000 7f9ff57fa640 1 -- 172.21.15.47:0/1758521292 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff00782a0 con 0x556112b7d2f0 2024-03-23T09:50:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.160+0000 7f9ff57fa640 1 -- 172.21.15.47:0/1758521292 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0078500 con 0x556112b7d2f0 2024-03-23T09:50:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.160+0000 7fa008e17740 1 -- 172.21.15.47:0/1758521292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b7d2f0 msgr2=0x556112b7d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.160+0000 7fa008e17740 1 --2- 172.21.15.47:0/1758521292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b7d2f0 0x556112b7d6d0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00a9810 tx=0x7f9ff0049000 comp rx=0 tx=0).stop 2024-03-23T09:50:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.160+0000 7fa008e17740 1 -- 172.21.15.47:0/1758521292 shutdown_connections 2024-03-23T09:50:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.160+0000 7fa008e17740 1 --2- 172.21.15.47:0/1758521292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bad080 0x556112bad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.160+0000 7fa008e17740 1 --2- 172.21.15.47:0/1758521292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112b81340 0x556112b82de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.160+0000 7fa008e17740 1 --2- 172.21.15.47:0/1758521292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b7d2f0 0x556112b7d6d0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.160+0000 7fa008e17740 1 -- 172.21.15.47:0/1758521292 >> 172.21.15.47:0/1758521292 conn(0x556112b7f370 msgr2=0x556112b80c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.161+0000 7fa008e17740 1 -- 172.21.15.47:0/1758521292 shutdown_connections 2024-03-23T09:50:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.161+0000 7fa008e17740 1 -- 172.21.15.47:0/1758521292 wait complete. 2024-03-23T09:50:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.161+0000 7fa008e17740 1 -- start start 2024-03-23T09:50:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.161+0000 7fa008e17740 1 Processor -- start 2024-03-23T09:50:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7fa008e17740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112bad080 0x556112bc6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7fa008e17740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b9a630 0x556112b9c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7fa008e17740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bc8000 0x556112b9c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7fa008e17740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556112a88a90 con 0x556112b9a630 2024-03-23T09:50:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7fa008e17740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556112a88730 con 0x556112bad080 2024-03-23T09:50:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7fa008e17740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556112a6ae00 con 0x556112bc8000 2024-03-23T09:50:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7fa008e17740 10 client.0 _ll_register_callbacks cb 0x556112aa0b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:50:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7f9ff7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b9a630 0x556112b9c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7f9ff7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b9a630 0x556112b9c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59684/0 (socket says 172.21.15.47:59684) 2024-03-23T09:50:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7f9ff7fff640 1 -- 172.21.15.47:0/2398368942 learned_addr learned my addr 172.21.15.47:0/2398368942 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7f9ffffff640 1 --2- 172.21.15.47:0/2398368942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bc8000 0x556112b9c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/2398368942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112bad080 0x556112bc6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7f9ff7fff640 1 -- 172.21.15.47:0/2398368942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bc8000 msgr2=0x556112b9c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7f9ff7fff640 1 --2- 172.21.15.47:0/2398368942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bc8000 0x556112b9c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7f9ff7fff640 1 -- 172.21.15.47:0/2398368942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112bad080 msgr2=0x556112bc6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7f9ff7fff640 1 --2- 172.21.15.47:0/2398368942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112bad080 0x556112bc6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.162+0000 7f9ff7fff640 1 -- 172.21.15.47:0/2398368942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00015ddc0 con 0x556112b9a630 2024-03-23T09:50:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7f9ff7fff640 1 --2- 172.21.15.47:0/2398368942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b9a630 0x556112b9c2d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8015760 tx=0x7f9fe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26b35640f52f2d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7f9fde7fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:50:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7f9fde7fc640 1 -- 172.21.15.47:0/2398368942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8013860 con 0x556112b9a630 2024-03-23T09:50:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7f9fde7fc640 1 -- 172.21.15.47:0/2398368942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9fe80139c0 con 0x556112b9a630 2024-03-23T09:50:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7fa008e17740 1 -- 172.21.15.47:0/2398368942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561127db560 con 0x556112b9a630 2024-03-23T09:50:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7f9fde7fc640 1 -- 172.21.15.47:0/2398368942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fe8013b80 con 0x556112b9a630 2024-03-23T09:50:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7fa008e17740 10 client.9015 fetch_fsmap learned FSMap version 71 2024-03-23T09:50:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7fa008e17740 1 -- 172.21.15.47:0/2398368942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x556112ae6040 con 0x556112b9a630 2024-03-23T09:50:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7f9fde7fc640 1 -- 172.21.15.47:0/2398368942 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5561127db560 con 0x556112b9a630 2024-03-23T09:50:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7f9fde7fc640 1 -- 172.21.15.47:0/2398368942 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9fe8011070 con 0x556112b9a630 2024-03-23T09:50:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7fa008e17740 10 client.9015 fetch_fsmap finished waiting for FSMap version 71 2024-03-23T09:50:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7fa008e17740 10 client.9015 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-03-23T09:50:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7fa008e17740 20 client.9015 populate_metadata read hostname 'smithi047' 2024-03-23T09:50:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7fa008e17740 1 --2- 172.21.15.47:0/2398368942 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x556112b9d670 0x556112bab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7fa008e17740 4 client.9015 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:50:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7fa008e17740 1 -- 172.21.15.47:0/2398368942 --> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561127ec860 con 0x556112b9d670 2024-03-23T09:50:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.163+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/2398368942 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x556112b9d670 0x556112bab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7f9fde7fc640 10 client.9015 ms_handle_connect on v2:172.21.15.47:6833/1748845936 2024-03-23T09:50:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/2398368942 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x556112b9d670 0x556112bab040 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7ea1b83f5c349ff server_cookie=e4fc5627510c64e in_seq=0 out_seq=0 2024-03-23T09:50:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7f9fde7fc640 1 -- 172.21.15.47:0/2398368942 <== mds.0 v2:172.21.15.47:6833/1748845936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5561127ec860 con 0x556112b9d670 2024-03-23T09:50:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7f9fde7fc640 10 client.9015 handle_command_reply: tid=0 2024-03-23T09:50:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7fa008e17740 1 client.9015 shutdown 2024-03-23T09:50:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7fa008e17740 1 -- 172.21.15.47:0/2398368942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b9a630 msgr2=0x556112b9c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7fa008e17740 1 --2- 172.21.15.47:0/2398368942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b9a630 0x556112b9c2d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8015760 tx=0x7f9fe80046c0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7fa008e17740 1 -- 172.21.15.47:0/2398368942 shutdown_connections 2024-03-23T09:50:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7fa008e17740 1 --2- 172.21.15.47:0/2398368942 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x556112b9d670 0x556112bab040 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7fa008e17740 1 --2- 172.21.15.47:0/2398368942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bc8000 0x556112b9c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7f9fde7fc640 0 client.9015 ms_handle_reset on v2:172.21.15.47:6833/1748845936 2024-03-23T09:50:54.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7fa008e17740 1 --2- 172.21.15.47:0/2398368942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112b9a630 0x556112b9c2d0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7fa008e17740 1 --2- 172.21.15.47:0/2398368942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112bad080 0x556112bc6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7fa008e17740 1 -- 172.21.15.47:0/2398368942 >> 172.21.15.47:0/2398368942 conn(0x556112b7d2f0 msgr2=0x556112bc95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.164+0000 7fa008e17740 1 -- 172.21.15.47:0/2398368942 shutdown_connections 2024-03-23T09:50:54.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.165+0000 7fa008e17740 1 -- 172.21.15.47:0/2398368942 wait complete. 2024-03-23T09:50:54.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.165+0000 7fa008e17740 20 client.9015 trim_cache size 0 max 16384 2024-03-23T09:50:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.169+0000 7f467af27640 1 Processor -- start 2024-03-23T09:50:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.169+0000 7f467af27640 1 -- start start 2024-03-23T09:50:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.170+0000 7f467af27640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f467406a3f0 0x7f46740cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.170+0000 7f467af27640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f46740cd420 0x7f46740b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.170+0000 7f467af27640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4674159be0 con 0x7f4674159800 2024-03-23T09:50:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.170+0000 7f467af27640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46740b16b0 con 0x7f46740cd420 2024-03-23T09:50:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.170+0000 7f467af27640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46740b1990 con 0x7f467406a3f0 2024-03-23T09:50:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.170+0000 7f46794c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f46740cd420 0x7f46740b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.170+0000 7f46794c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f46740cd420 0x7f46740b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59690/0 (socket says 172.21.15.47:59690) 2024-03-23T09:50:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.170+0000 7f46794c4640 1 -- 172.21.15.47:0/2783416838 learned_addr learned my addr 172.21.15.47:0/2783416838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.170+0000 7f4673fff640 1 --2- 172.21.15.47:0/2783416838 >> v2:172.21.15.175:3300/0 conn(0x7f467406a3f0 0x7f46740cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.170+0000 7f46794c4640 1 -- 172.21.15.47:0/2783416838 >> v2:172.21.15.175:3300/0 conn(0x7f467406a3f0 msgr2=0x7f46740cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.170+0000 7f46794c4640 1 --2- 172.21.15.47:0/2783416838 >> v2:172.21.15.175:3300/0 conn(0x7f467406a3f0 0x7f46740cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.170+0000 7f46794c4640 1 -- 172.21.15.47:0/2783416838 >> v1:172.21.15.175:6789/0 conn(0x7f4674159800 legacy=0x7f4674069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.170+0000 7f46794c4640 1 -- 172.21.15.47:0/2783416838 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46740b1c70 con 0x7f46740cd420 2024-03-23T09:50:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.171+0000 7f46794c4640 1 --2- 172.21.15.47:0/2783416838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46740cd420 0x7f46740b2750 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f46740cc1a0 tx=0x7f466c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fde247942fbb50 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.171+0000 7f46737fe640 1 -- 172.21.15.47:0/2783416838 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c02ca80 con 0x7f46740cd420 2024-03-23T09:50:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.171+0000 7f46737fe640 1 -- 172.21.15.47:0/2783416838 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c02cbe0 con 0x7f46740cd420 2024-03-23T09:50:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.171+0000 7f46737fe640 1 -- 172.21.15.47:0/2783416838 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c0343f0 con 0x7f46740cd420 2024-03-23T09:50:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.172+0000 7f467af27640 1 -- 172.21.15.47:0/2783416838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46740cd420 msgr2=0x7f46740b2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.172+0000 7f467af27640 1 --2- 172.21.15.47:0/2783416838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46740cd420 0x7f46740b2750 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f46740cc1a0 tx=0x7f466c02bee0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.172+0000 7f467af27640 1 -- 172.21.15.47:0/2783416838 shutdown_connections 2024-03-23T09:50:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.172+0000 7f467af27640 1 --2- 172.21.15.47:0/2783416838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46740cd420 0x7f46740b2750 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.172+0000 7f467af27640 1 --2- 172.21.15.47:0/2783416838 >> v2:172.21.15.175:3300/0 conn(0x7f467406a3f0 0x7f46740cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.172+0000 7f467af27640 1 -- 172.21.15.47:0/2783416838 >> 172.21.15.47:0/2783416838 conn(0x7f46740b0210 msgr2=0x7f46740b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.172+0000 7f467af27640 1 -- 172.21.15.47:0/2783416838 shutdown_connections 2024-03-23T09:50:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.172+0000 7f467af27640 1 -- 172.21.15.47:0/2783416838 wait complete. 2024-03-23T09:50:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.173+0000 7f467af27640 1 Processor -- start 2024-03-23T09:50:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.173+0000 7f467af27640 1 -- start start 2024-03-23T09:50:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f467af27640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f467406a3f0 0x7f46741effb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f467af27640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46740cd420 0x7f467415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f467af27640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4674159800 0x7f46741ed4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f467af27640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46741f1320 con 0x7f467406a3f0 2024-03-23T09:50:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f467af27640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467415cb70 con 0x7f46740cd420 2024-03-23T09:50:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f467af27640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46741f0680 con 0x7f4674159800 2024-03-23T09:50:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f4678cc3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f467406a3f0 0x7f46741effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f4678cc3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f467406a3f0 0x7f46741effb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59706/0 (socket says 172.21.15.47:59706) 2024-03-23T09:50:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f4678cc3640 1 -- 172.21.15.47:0/2413808955 learned_addr learned my addr 172.21.15.47:0/2413808955 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f4673fff640 1 --2- 172.21.15.47:0/2413808955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46740cd420 0x7f467415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f46794c4640 1 --2- 172.21.15.47:0/2413808955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4674159800 0x7f46741ed4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f4678cc3640 1 -- 172.21.15.47:0/2413808955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4674159800 msgr2=0x7f46741ed4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f4678cc3640 1 --2- 172.21.15.47:0/2413808955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4674159800 0x7f46741ed4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f4678cc3640 1 -- 172.21.15.47:0/2413808955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46740cd420 msgr2=0x7f467415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f4678cc3640 1 --2- 172.21.15.47:0/2413808955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46740cd420 0x7f467415c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f4678cc3640 1 -- 172.21.15.47:0/2413808955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c009b30 con 0x7f467406a3f0 2024-03-23T09:50:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f4673fff640 1 --2- 172.21.15.47:0/2413808955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46740cd420 0x7f467415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:50:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f4678cc3640 1 --2- 172.21.15.47:0/2413808955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f467406a3f0 0x7f46741effb0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f466800b3f0 tx=0x7f466800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.174+0000 7f46717fa640 1 -- 172.21.15.47:0/2413808955 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668011830 con 0x7f467406a3f0 2024-03-23T09:50:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.175+0000 7f46717fa640 1 -- 172.21.15.47:0/2413808955 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4668011990 con 0x7f467406a3f0 2024-03-23T09:50:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.175+0000 7f467af27640 1 -- 172.21.15.47:0/2413808955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46741f0960 con 0x7f467406a3f0 2024-03-23T09:50:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.175+0000 7f46717fa640 1 -- 172.21.15.47:0/2413808955 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668011d40 con 0x7f467406a3f0 2024-03-23T09:50:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.175+0000 7f467af27640 1 -- 172.21.15.47:0/2413808955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46741ed9f0 con 0x7f467406a3f0 2024-03-23T09:50:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.175+0000 7f467af27640 1 -- 172.21.15.47:0/2413808955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f46741f1590 con 0x7f467406a3f0 2024-03-23T09:50:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.176+0000 7f46717fa640 1 -- 172.21.15.47:0/2413808955 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f466801a400 con 0x7f467406a3f0 2024-03-23T09:50:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.176+0000 7f46717fa640 1 --2- 172.21.15.47:0/2413808955 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f465806ac00 0x7f465806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.177+0000 7f46717fa640 1 -- 172.21.15.47:0/2413808955 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4668089950 con 0x7f467406a3f0 2024-03-23T09:50:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.177+0000 7f46717fa640 1 -- 172.21.15.47:0/2413808955 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+515 (secure 0 0 0) 0x7f466808a2e0 con 0x7f467406a3f0 2024-03-23T09:50:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.177+0000 7f4673fff640 1 --2- 172.21.15.47:0/2413808955 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f465806ac00 0x7f465806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.177+0000 7f467af27640 1 -- 172.21.15.47:0/2413808955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f4674066be0 con 0x7f467406a3f0 2024-03-23T09:50:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.177+0000 7f46717fa640 1 -- 172.21.15.47:0/2413808955 <== mon.0 v2:172.21.15.47:3300/0 7 ==== log(2 entries from seq 1028 at 2024-03-23T09:50:52.610405+0000) v1 ==== 389+0+0 (secure 0 0 0) 0x7f4668058840 con 0x7f467406a3f0 2024-03-23T09:50:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: cluster: 2024-03-23T09:50:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: id: f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:50:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: health: HEALTH_OK 2024-03-23T09:50:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: services: 2024-03-23T09:50:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-03-23T09:50:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: mgr: x(active, since 10m), standbys: y 2024-03-23T09:50:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: mds: 1/1 daemons up, 3 standby 2024-03-23T09:50:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 10m) 2024-03-23T09:50:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: data: 2024-03-23T09:50:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: volumes: 1/1 healthy 2024-03-23T09:50:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: pools: 3 pools, 97 pgs 2024-03-23T09:50:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: objects: 30 objects, 19 MiB 2024-03-23T09:50:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: usage: 302 MiB used, 720 GiB / 720 GiB avail 2024-03-23T09:50:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: pgs: 97 active+clean 2024-03-23T09:50:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: io: 2024-03-23T09:50:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-03-23T09:50:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:50:54.185 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:52.610405+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T09:50:54.185 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:52.610434+0000 mon.a [INF] Cluster is now healthy 2024-03-23T09:50:54.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.177+0000 7f4673fff640 1 --2- 172.21.15.47:0/2413808955 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f465806ac00 0x7f465806d0c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f467415c190 tx=0x7f4660008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.181+0000 7fa008e17740 1 Processor -- start 2024-03-23T09:50:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.181+0000 7fa008e17740 1 -- start start 2024-03-23T09:50:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7fa008e17740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112b9a630 0x556112b78990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7fa008e17740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bad080 0x556112bb0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7fa008e17740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112bc8000 0x556112bb0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7fa008e17740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556112b9c940 con 0x556112bad080 2024-03-23T09:50:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7fa008e17740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556112bb12c0 con 0x556112bc8000 2024-03-23T09:50:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7fa008e17740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556112bea7f0 con 0x556112b9a630 2024-03-23T09:50:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7f9ff7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bad080 0x556112bb0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7f9ff7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bad080 0x556112bb0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59708/0 (socket says 172.21.15.47:59708) 2024-03-23T09:50:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7f9ff7fff640 1 -- 172.21.15.47:0/1778956403 learned_addr learned my addr 172.21.15.47:0/1778956403 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/1778956403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112b9a630 0x556112b78990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7f9ffffff640 1 --2- 172.21.15.47:0/1778956403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112bc8000 0x556112bb0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7f9ff7fff640 1 -- 172.21.15.47:0/1778956403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112b9a630 msgr2=0x556112b78990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7f9ff7fff640 1 --2- 172.21.15.47:0/1778956403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112b9a630 0x556112b78990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7f9ff7fff640 1 -- 172.21.15.47:0/1778956403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112bc8000 msgr2=0x556112bb0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7f9ff7fff640 1 --2- 172.21.15.47:0/1778956403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112bc8000 0x556112bb0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7f9ff7fff640 1 -- 172.21.15.47:0/1778956403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556112a87e20 con 0x556112bad080 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7f9ffffff640 1 --2- 172.21.15.47:0/1778956403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112bc8000 0x556112bb0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/1778956403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112b9a630 0x556112b78990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.182+0000 7f9ff7fff640 1 --2- 172.21.15.47:0/1778956403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bad080 0x556112bb0840 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8000c00 tx=0x7f9fe8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964501073f7b885a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.183+0000 7f9ff5ffb640 1 -- 172.21.15.47:0/1778956403 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe802a640 con 0x556112bad080 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.183+0000 7f9ff5ffb640 1 -- 172.21.15.47:0/1778956403 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe8006ac0 con 0x556112bad080 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.183+0000 7f9ff5ffb640 1 -- 172.21.15.47:0/1778956403 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe802aa00 con 0x556112bad080 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.183+0000 7fa008e17740 1 -- 172.21.15.47:0/1778956403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bad080 msgr2=0x556112bb0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.183+0000 7fa008e17740 1 --2- 172.21.15.47:0/1778956403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bad080 0x556112bb0840 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8000c00 tx=0x7f9fe8020000 comp rx=0 tx=0).stop 2024-03-23T09:50:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.183+0000 7fa008e17740 1 -- 172.21.15.47:0/1778956403 shutdown_connections 2024-03-23T09:50:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.183+0000 7fa008e17740 1 --2- 172.21.15.47:0/1778956403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112bc8000 0x556112bb0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.183+0000 7fa008e17740 1 --2- 172.21.15.47:0/1778956403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bad080 0x556112bb0840 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.183+0000 7fa008e17740 1 --2- 172.21.15.47:0/1778956403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112b9a630 0x556112b78990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.183+0000 7fa008e17740 1 -- 172.21.15.47:0/1778956403 >> 172.21.15.47:0/1778956403 conn(0x556112b7d2f0 msgr2=0x556112bab990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.183+0000 7fa008e17740 1 -- 172.21.15.47:0/1778956403 shutdown_connections 2024-03-23T09:50:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.183+0000 7fa008e17740 1 -- 172.21.15.47:0/1778956403 wait complete. 2024-03-23T09:50:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.184+0000 7fa008e17740 1 -- start start 2024-03-23T09:50:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.184+0000 7fa008e17740 1 Processor -- start 2024-03-23T09:50:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7fa008e17740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112b9a630 0x556112c2e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7fa008e17740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bad080 0x556112ba0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7fa008e17740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bc8000 0x556112bb0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7fa008e17740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556112b81970 con 0x556112bc8000 2024-03-23T09:50:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7fa008e17740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556112b9c3e0 con 0x556112b9a630 2024-03-23T09:50:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7fa008e17740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x556112ba0580 con 0x556112bad080 2024-03-23T09:50:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7fa008e17740 10 client.0 _ll_register_callbacks cb 0x7f9ff0076520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:50:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9ffffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bc8000 0x556112bb0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9ffffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bc8000 0x556112bb0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59720/0 (socket says 172.21.15.47:59720) 2024-03-23T09:50:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9ffffff640 1 -- 172.21.15.47:0/1231858738 learned_addr learned my addr 172.21.15.47:0/1231858738 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112b9a630 0x556112c2e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9ff7fff640 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bad080 0x556112ba0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9ffffff640 1 -- 172.21.15.47:0/1231858738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bad080 msgr2=0x556112ba0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9ffffff640 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bad080 0x556112ba0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9ffffff640 1 -- 172.21.15.47:0/1231858738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112b9a630 msgr2=0x556112c2e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9ffffff640 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112b9a630 0x556112c2e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9ffffff640 1 -- 172.21.15.47:0/1231858738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556112ae6040 con 0x556112bc8000 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9ff7fff640 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bad080 0x556112ba0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9ffffff640 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bc8000 0x556112bb0610 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80070a0 tx=0x7f9ff8007560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e294a09191365 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9fde7fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9fde7fc640 1 -- 172.21.15.47:0/1231858738 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff80166f0 con 0x556112bc8000 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7f9fde7fc640 1 -- 172.21.15.47:0/1231858738 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9ff801a880 con 0x556112bc8000 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.185+0000 7fa008e17740 1 -- 172.21.15.47:0/1231858738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ff00782a0 con 0x556112bc8000 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7f9fde7fc640 1 -- 172.21.15.47:0/1231858738 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ff801aa40 con 0x556112bc8000 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7fa008e17740 10 client.9027 fetch_fsmap learned FSMap version 71 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7fa008e17740 1 -- 172.21.15.47:0/1231858738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x556112aad5f0 con 0x556112bc8000 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7f9fde7fc640 1 -- 172.21.15.47:0/1231858738 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff00782a0 con 0x556112bc8000 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7f9fde7fc640 1 -- 172.21.15.47:0/1231858738 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9ff8012050 con 0x556112bc8000 2024-03-23T09:50:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7fa008e17740 10 client.9027 fetch_fsmap finished waiting for FSMap version 71 2024-03-23T09:50:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7fa008e17740 10 client.9027 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-03-23T09:50:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7fa008e17740 20 client.9027 populate_metadata read hostname 'smithi047' 2024-03-23T09:50:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7fa008e17740 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x556112bb0b50 0x556112be8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7fa008e17740 4 client.9027 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-23T09:50:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7fa008e17740 1 -- 172.21.15.47:0/1231858738 --> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55611283feb0 con 0x556112bb0b50 2024-03-23T09:50:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x556112bb0b50 0x556112be8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7f9fde7fc640 10 client.9027 ms_handle_connect on v2:172.21.15.47:6833/1748845936 2024-03-23T09:50:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.186+0000 7f9fff7fe640 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x556112bb0b50 0x556112be8660 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37cc6197b705ba79 server_cookie=a62ef12b8efdb0ba in_seq=0 out_seq=0 2024-03-23T09:50:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.187+0000 7f9fde7fc640 1 -- 172.21.15.47:0/1231858738 <== mds.0 v2:172.21.15.47:6833/1748845936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55611283feb0 con 0x556112bb0b50 2024-03-23T09:50:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.187+0000 7f9fde7fc640 10 client.9027 handle_command_reply: tid=0 2024-03-23T09:50:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.187+0000 7fa008e17740 1 client.9027 shutdown 2024-03-23T09:50:54.198 INFO:tasks.ceph.mds.a.smithi047.stderr:2024-03-23T09:50:54.187+0000 7f757833a640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-03-23T09:50:54.198 INFO:tasks.ceph.mds.a.smithi047.stderr:2024-03-23T09:50:54.187+0000 7f757833a640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-03-23T09:50:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.187+0000 7fa008e17740 1 -- 172.21.15.47:0/1231858738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bc8000 msgr2=0x556112bb0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.187+0000 7fa008e17740 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bc8000 0x556112bb0610 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80070a0 tx=0x7f9ff8007560 comp rx=0 tx=0).stop 2024-03-23T09:50:54.199 INFO:tasks.ceph.mds.a.smithi047.stderr:2024-03-23T09:50:54.187+0000 7f757833a640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-03-23T09:50:54.199 INFO:tasks.ceph.mds.a.smithi047.stderr:2024-03-23T09:50:54.187+0000 7f757833a640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-03-23T09:50:54.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.187+0000 7fa008e17740 1 -- 172.21.15.47:0/1231858738 shutdown_connections 2024-03-23T09:50:54.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.187+0000 7fa008e17740 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x556112bb0b50 0x556112be8660 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.187+0000 7fa008e17740 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556112bc8000 0x556112bb0610 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.187+0000 7fa008e17740 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x556112bad080 0x556112ba0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.187+0000 7fa008e17740 1 --2- 172.21.15.47:0/1231858738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556112b9a630 0x556112c2e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.187+0000 7fa008e17740 1 -- 172.21.15.47:0/1231858738 >> 172.21.15.47:0/1231858738 conn(0x556112b7d2f0 msgr2=0x556112b78990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.187+0000 7f9fde7fc640 0 client.9027 ms_handle_reset on v2:172.21.15.47:6833/1748845936 2024-03-23T09:50:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.188+0000 7fa008e17740 1 -- 172.21.15.47:0/1231858738 shutdown_connections 2024-03-23T09:50:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.188+0000 7fa008e17740 1 -- 172.21.15.47:0/1231858738 wait complete. 2024-03-23T09:50:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.188+0000 7fa008e17740 20 client.9027 trim_cache size 0 max 16384 2024-03-23T09:50:54.200 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:50:54.200 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-23T09:50:54.201 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "68e7cb3a-2c03-4bba-9802-6486b239533a", 2024-03-23T09:50:54.201 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-23T09:50:54.201 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:50:54.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.188+0000 7fa0061ce640 1 -- 172.21.15.47:0/1129764422 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9fe006ac30 msgr2=0x7f9fe006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.188+0000 7fa0061ce640 1 --2- 172.21.15.47:0/1129764422 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9fe006ac30 0x7f9fe006d0f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8006a90 tx=0x7f9fe8006a20 comp rx=0 tx=0).stop 2024-03-23T09:50:54.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.188+0000 7fa0061ce640 1 -- 172.21.15.47:0/1129764422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0000b0280 msgr2=0x7fa0000b4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.188+0000 7fa0061ce640 1 --2- 172.21.15.47:0/1129764422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0000b0280 0x7fa0000b4a90 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0034040 tx=0x7f9ff00046c0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.188+0000 7fa0061ce640 1 -- 172.21.15.47:0/1129764422 shutdown_connections 2024-03-23T09:50:54.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.188+0000 7fa0061ce640 1 --2- 172.21.15.47:0/1129764422 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9fe006ac30 0x7f9fe006d0f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.189+0000 7fa0061ce640 1 --2- 172.21.15.47:0/1129764422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0000cb460 0x7fa0001594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.189+0000 7fa0061ce640 1 --2- 172.21.15.47:0/1129764422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0000b1bf0 0x7fa00015e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.189+0000 7fa0061ce640 1 --2- 172.21.15.47:0/1129764422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0000b0280 0x7fa0000b4a90 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.189+0000 7fa0061ce640 1 -- 172.21.15.47:0/1129764422 >> 172.21.15.47:0/1129764422 conn(0x7fa000098a00 msgr2=0x7fa0000b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.189+0000 7fa0061ce640 1 -- 172.21.15.47:0/1129764422 shutdown_connections 2024-03-23T09:50:54.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.189+0000 7fa0061ce640 1 -- 172.21.15.47:0/1129764422 wait complete. 2024-03-23T09:50:54.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:50:54.204 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-23T09:50:54.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.306+0000 7fb6a7d00640 1 Processor -- start 2024-03-23T09:50:54.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.306+0000 7fb6a7d00640 1 -- start start 2024-03-23T09:50:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.307+0000 7fb6a7d00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb6a015ca30 0x7fb6a0160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.307+0000 7fb6a7d00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6a00cde40 0x7fb6a00d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.307+0000 7fb6a7d00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a0161600 con 0x7fb6a015ce10 2024-03-23T09:50:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.307+0000 7fb6a7d00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a0161040 con 0x7fb6a015ca30 2024-03-23T09:50:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.307+0000 7fb6a7d00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a0161320 con 0x7fb6a00cde40 2024-03-23T09:50:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.307+0000 7fb6a529b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb6a015ce10 0x7fb6a00cd730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45884/0 (socket says 172.21.15.47:45884) 2024-03-23T09:50:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.307+0000 7fb6a529b640 1 -- 172.21.15.47:0/3173315531 learned_addr learned my addr 172.21.15.47:0/3173315531 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:50:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.307+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/3173315531 >> v2:172.21.15.175:3300/0 conn(0x7fb6a015ca30 0x7fb6a0160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.307+0000 7fb6a629d640 1 --2- 172.21.15.47:0/3173315531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6a00cde40 0x7fb6a00d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.307+0000 7fb6a4a9a640 1 -- 172.21.15.47:0/3173315531 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204517289 0 0) 0x7fb6a0161600 con 0x7fb6a015ce10 2024-03-23T09:50:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.307+0000 7fb6a4a9a640 1 -- 172.21.15.47:0/3173315531 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb68c003520 con 0x7fb6a015ce10 2024-03-23T09:50:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.308+0000 7fb6a5a9c640 1 -- 172.21.15.47:0/3173315531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6a00cde40 msgr2=0x7fb6a00d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.308+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/3173315531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6a00cde40 0x7fb6a00d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.308+0000 7fb6a5a9c640 1 -- 172.21.15.47:0/3173315531 >> v1:172.21.15.47:6789/0 conn(0x7fb6a015ce10 legacy=0x7fb6a00cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.308+0000 7fb6a5a9c640 1 -- 172.21.15.47:0/3173315531 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a00d0dd0 con 0x7fb6a015ca30 2024-03-23T09:50:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.308+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/3173315531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a015ca30 0x7fb6a0160740 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb6940097a0 tx=0x7fb69402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ed954ddc14093e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.309+0000 7fb6a4a9a640 1 -- 172.21.15.47:0/3173315531 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694031830 con 0x7fb6a015ca30 2024-03-23T09:50:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.309+0000 7fb6a4a9a640 1 -- 172.21.15.47:0/3173315531 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb694031990 con 0x7fb6a015ca30 2024-03-23T09:50:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.309+0000 7fb6a4a9a640 1 -- 172.21.15.47:0/3173315531 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694031d00 con 0x7fb6a015ca30 2024-03-23T09:50:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.309+0000 7fb6a7d00640 1 -- 172.21.15.47:0/3173315531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a015ca30 msgr2=0x7fb6a0160740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.309+0000 7fb6a7d00640 1 --2- 172.21.15.47:0/3173315531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a015ca30 0x7fb6a0160740 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb6940097a0 tx=0x7fb69402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.309+0000 7fb6a7d00640 1 -- 172.21.15.47:0/3173315531 shutdown_connections 2024-03-23T09:50:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.309+0000 7fb6a7d00640 1 --2- 172.21.15.47:0/3173315531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6a00cde40 0x7fb6a00d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.309+0000 7fb6a7d00640 1 --2- 172.21.15.47:0/3173315531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a015ca30 0x7fb6a0160740 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.309+0000 7fb6a7d00640 1 -- 172.21.15.47:0/3173315531 >> 172.21.15.47:0/3173315531 conn(0x7fb6a006b160 msgr2=0x7fb6a006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.309+0000 7fb6a7d00640 1 -- 172.21.15.47:0/3173315531 shutdown_connections 2024-03-23T09:50:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.309+0000 7fb6a7d00640 1 -- 172.21.15.47:0/3173315531 wait complete. 2024-03-23T09:50:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.310+0000 7fb6a7d00640 1 Processor -- start 2024-03-23T09:50:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.310+0000 7fb6a7d00640 1 -- start start 2024-03-23T09:50:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a7d00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a00cde40 0x7fb6a00c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a7d00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a015ca30 0x7fb6a0156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a7d00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6a015ce10 0x7fb6a015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a7d00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a00c76f0 con 0x7fb6a00cde40 2024-03-23T09:50:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a7d00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a00c79d0 con 0x7fb6a015ca30 2024-03-23T09:50:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a7d00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a00c7cb0 con 0x7fb6a015ce10 2024-03-23T09:50:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a5a9c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a00cde40 0x7fb6a00c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a5a9c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a00cde40 0x7fb6a00c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59734/0 (socket says 172.21.15.47:59734) 2024-03-23T09:50:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a5a9c640 1 -- 172.21.15.47:0/2443992732 learned_addr learned my addr 172.21.15.47:0/2443992732 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a629d640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6a015ce10 0x7fb6a015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a529b640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a015ca30 0x7fb6a0156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a5a9c640 1 -- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6a015ce10 msgr2=0x7fb6a015dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6a015ce10 0x7fb6a015dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a5a9c640 1 -- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a015ca30 msgr2=0x7fb6a0156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a015ca30 0x7fb6a0156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a5a9c640 1 -- 172.21.15.47:0/2443992732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb694009400 con 0x7fb6a00cde40 2024-03-23T09:50:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.311+0000 7fb6a529b640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a015ca30 0x7fb6a0156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.312+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a00cde40 0x7fb6a00c8790 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fb6940098c0 tx=0x7fb6940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.312+0000 7fb6927fc640 1 -- 172.21.15.47:0/2443992732 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69403a630 con 0x7fb6a00cde40 2024-03-23T09:50:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.312+0000 7fb6927fc640 1 -- 172.21.15.47:0/2443992732 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69403a790 con 0x7fb6a00cde40 2024-03-23T09:50:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.312+0000 7fb6a7d00640 1 -- 172.21.15.47:0/2443992732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a00c7f90 con 0x7fb6a00cde40 2024-03-23T09:50:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.312+0000 7fb6927fc640 1 -- 172.21.15.47:0/2443992732 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69403ab40 con 0x7fb6a00cde40 2024-03-23T09:50:54.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.312+0000 7fb6a7d00640 1 -- 172.21.15.47:0/2443992732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a00c8cd0 con 0x7fb6a00cde40 2024-03-23T09:50:54.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.313+0000 7fb6927fc640 1 -- 172.21.15.47:0/2443992732 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb69403aca0 con 0x7fb6a00cde40 2024-03-23T09:50:54.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.314+0000 7fb6927fc640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb68406ac00 0x7fb68406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.314+0000 7fb6927fc640 1 -- 172.21.15.47:0/2443992732 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb6940ab080 con 0x7fb6a00cde40 2024-03-23T09:50:54.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.314+0000 7fb6a529b640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb68406ac00 0x7fb68406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.314+0000 7fb6a529b640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb68406ac00 0x7fb68406d0c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00d2840 tx=0x7fb69c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.315+0000 7fb6aa876740 1 Processor -- start 2024-03-23T09:50:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.315+0000 7fb6aa876740 1 -- start start 2024-03-23T09:50:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.315+0000 7fb6aa876740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 0x55c1ccee3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.315+0000 7fb6aa876740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cceb3850 0x55c1ccee0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.315+0000 7fb6aa876740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccee0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6aa876740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1cce23dd0 con 0x55c1cceb3850 2024-03-23T09:50:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6aa876740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1cce21560 con 0x55c1cceb73b0 2024-03-23T09:50:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6aa876740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1cce1ce50 con 0x55c1ccee3830 2024-03-23T09:50:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6a529b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cceb3850 0x55c1ccee0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6a529b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cceb3850 0x55c1ccee0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59740/0 (socket says 172.21.15.47:59740) 2024-03-23T09:50:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6a529b640 1 -- 172.21.15.47:0/1801652591 learned_addr learned my addr 172.21.15.47:0/1801652591 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/1801652591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 0x55c1ccee3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6a629d640 1 --2- 172.21.15.47:0/1801652591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccee0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6a529b640 1 -- 172.21.15.47:0/1801652591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 msgr2=0x55c1ccee3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6a529b640 1 --2- 172.21.15.47:0/1801652591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 0x55c1ccee3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6a529b640 1 -- 172.21.15.47:0/1801652591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 msgr2=0x55c1ccee0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6a529b640 1 --2- 172.21.15.47:0/1801652591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccee0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6a529b640 1 -- 172.21.15.47:0/1801652591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1ccde4470 con 0x55c1cceb3850 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6a529b640 1 --2- 172.21.15.47:0/1801652591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cceb3850 0x55c1ccee0aa0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00dee0 tx=0x7fb69c0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7c7a9eecbe96c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6837fe640 1 -- 172.21.15.47:0/1801652591 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c012820 con 0x55c1cceb3850 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6837fe640 1 -- 172.21.15.47:0/1801652591 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c012980 con 0x55c1cceb3850 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.316+0000 7fb6837fe640 1 -- 172.21.15.47:0/1801652591 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c012cf0 con 0x55c1cceb3850 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.317+0000 7fb6aa876740 1 -- 172.21.15.47:0/1801652591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cceb3850 msgr2=0x55c1ccee0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.317+0000 7fb6aa876740 1 --2- 172.21.15.47:0/1801652591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cceb3850 0x55c1ccee0aa0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00dee0 tx=0x7fb69c0021f0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.317+0000 7fb6aa876740 1 -- 172.21.15.47:0/1801652591 shutdown_connections 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.317+0000 7fb6aa876740 1 --2- 172.21.15.47:0/1801652591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccee0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.317+0000 7fb6aa876740 1 --2- 172.21.15.47:0/1801652591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cceb3850 0x55c1ccee0aa0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.317+0000 7fb6aa876740 1 --2- 172.21.15.47:0/1801652591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 0x55c1ccee3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.317+0000 7fb6aa876740 1 -- 172.21.15.47:0/1801652591 >> 172.21.15.47:0/1801652591 conn(0x55c1cceb5ac0 msgr2=0x55c1ccefe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.317+0000 7fb6aa876740 1 -- 172.21.15.47:0/1801652591 shutdown_connections 2024-03-23T09:50:54.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.317+0000 7fb6aa876740 1 -- 172.21.15.47:0/1801652591 wait complete. 2024-03-23T09:50:54.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.317+0000 7fb6aa876740 1 -- start start 2024-03-23T09:50:54.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.318+0000 7fb6aa876740 1 Processor -- start 2024-03-23T09:50:54.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.318+0000 7fb6aa876740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccefeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.318+0000 7fb6aa876740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 0x55c1ccee6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.318+0000 7fb6aa876740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 0x55c1ccee69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.318+0000 7fb6aa876740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ccddb280 con 0x55c1cced84e0 2024-03-23T09:50:54.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.318+0000 7fb6aa876740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ccddaf20 con 0x55c1cceb73b0 2024-03-23T09:50:54.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.318+0000 7fb6aa876740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1ccda03c0 con 0x55c1ccee3830 2024-03-23T09:50:54.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.318+0000 7fb6aa876740 10 client.0 _ll_register_callbacks cb 0x55c1ccdf2380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:50:54.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6a629d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 0x55c1ccee69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6a629d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 0x55c1ccee69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59744/0 (socket says 172.21.15.47:59744) 2024-03-23T09:50:54.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6a629d640 1 -- 172.21.15.47:0/465647828 learned_addr learned my addr 172.21.15.47:0/465647828 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccefeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6a529b640 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 0x55c1ccee6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6a629d640 1 -- 172.21.15.47:0/465647828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 msgr2=0x55c1ccee6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6a629d640 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 0x55c1ccee6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6a629d640 1 -- 172.21.15.47:0/465647828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 msgr2=0x55c1ccefeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6a629d640 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccefeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6a629d640 1 -- 172.21.15.47:0/465647828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c003520 con 0x55c1cced84e0 2024-03-23T09:50:54.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccefeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:50:54.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6a629d640 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 0x55c1ccee69c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fb698009c30 tx=0x7fb6980136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52a61f74e656330 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb680ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb680ff9640 1 -- 172.21.15.47:0/465647828 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb698016660 con 0x55c1cced84e0 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb680ff9640 1 -- 172.21.15.47:0/465647828 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb698009e40 con 0x55c1cced84e0 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6aa876740 1 -- 172.21.15.47:0/465647828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1cce46690 con 0x55c1cced84e0 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb680ff9640 1 -- 172.21.15.47:0/465647828 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6980167c0 con 0x55c1cced84e0 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6aa876740 10 client.9039 fetch_fsmap learned FSMap version 71 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.319+0000 7fb6aa876740 1 -- 172.21.15.47:0/465647828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55c1cce1b4d0 con 0x55c1cced84e0 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.320+0000 7fb680ff9640 1 -- 172.21.15.47:0/465647828 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1cce46690 con 0x55c1cced84e0 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.320+0000 7fb680ff9640 1 -- 172.21.15.47:0/465647828 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb698016920 con 0x55c1cced84e0 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.320+0000 7fb6aa876740 10 client.9039 fetch_fsmap finished waiting for FSMap version 71 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.320+0000 7fb6aa876740 10 client.9039 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.320+0000 7fb6aa876740 20 client.9039 populate_metadata read hostname 'smithi047' 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.320+0000 7fb6aa876740 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55c1ccedace0 0x55c1ccee5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.320+0000 7fb6aa876740 4 client.9039 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:50:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.320+0000 7fb6aa876740 1 -- 172.21.15.47:0/465647828 --> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1cce0eeb0 con 0x55c1ccedace0 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.320+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55c1ccedace0 0x55c1ccee5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.320+0000 7fb680ff9640 10 client.9039 ms_handle_connect on v2:172.21.15.47:6833/1748845936 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.320+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55c1ccedace0 0x55c1ccee5640 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=165d74f549ddc80f server_cookie=1e5c87aca02d1fce in_seq=0 out_seq=0 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb680ff9640 1 -- 172.21.15.47:0/465647828 <== mds.0 v2:172.21.15.47:6833/1748845936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c1cce0eeb0 con 0x55c1ccedace0 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb680ff9640 10 client.9039 handle_command_reply: tid=0 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb6aa876740 1 client.9039 shutdown 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb6aa876740 1 -- 172.21.15.47:0/465647828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 msgr2=0x55c1ccee69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb6aa876740 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 0x55c1ccee69c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fb698009c30 tx=0x7fb6980136b0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb6aa876740 1 -- 172.21.15.47:0/465647828 shutdown_connections 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb6aa876740 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55c1ccedace0 0x55c1ccee5640 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb6aa876740 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 0x55c1ccee69c0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb6aa876740 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 0x55c1ccee6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb680ff9640 0 client.9039 ms_handle_reset on v2:172.21.15.47:6833/1748845936 2024-03-23T09:50:54.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb6aa876740 1 --2- 172.21.15.47:0/465647828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccefeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb6aa876740 1 -- 172.21.15.47:0/465647828 >> 172.21.15.47:0/465647828 conn(0x55c1cceb3850 msgr2=0x55c1cceb7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb6aa876740 1 -- 172.21.15.47:0/465647828 shutdown_connections 2024-03-23T09:50:54.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb6aa876740 1 -- 172.21.15.47:0/465647828 wait complete. 2024-03-23T09:50:54.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.321+0000 7fb6aa876740 20 client.9039 trim_cache size 0 max 16384 2024-03-23T09:50:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.335+0000 7fb6aa876740 1 Processor -- start 2024-03-23T09:50:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.335+0000 7fb6aa876740 1 -- start start 2024-03-23T09:50:54.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6aa876740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccedfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6aa876740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 0x55c1cceb78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6aa876740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 0x55c1cceb7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6aa876740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ccef93c0 con 0x55c1cced84e0 2024-03-23T09:50:54.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6aa876740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1cceb5360 con 0x55c1cceb73b0 2024-03-23T09:50:54.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6aa876740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1ccf02a90 con 0x55c1ccee3830 2024-03-23T09:50:54.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6a529b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 0x55c1cceb78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6a529b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 0x55c1cceb78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59758/0 (socket says 172.21.15.47:59758) 2024-03-23T09:50:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6a529b640 1 -- 172.21.15.47:0/1800754056 learned_addr learned my addr 172.21.15.47:0/1800754056 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6a529b640 1 -- 172.21.15.47:0/1800754056 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 msgr2=0x55c1cceb7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6a529b640 1 --2- 172.21.15.47:0/1800754056 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 0x55c1cceb7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6a529b640 1 -- 172.21.15.47:0/1800754056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 msgr2=0x55c1ccedfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6a529b640 1 --2- 172.21.15.47:0/1800754056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccedfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.336+0000 7fb6a529b640 1 -- 172.21.15.47:0/1800754056 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1cce4f9e0 con 0x55c1cced84e0 2024-03-23T09:50:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.337+0000 7fb6a529b640 1 --2- 172.21.15.47:0/1800754056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 0x55c1cceb78b0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0012e0 tx=0x7fb69c007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ec6db0667a674e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.337+0000 7fb683fff640 1 -- 172.21.15.47:0/1800754056 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c012820 con 0x55c1cced84e0 2024-03-23T09:50:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.337+0000 7fb683fff640 1 -- 172.21.15.47:0/1800754056 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c028040 con 0x55c1cced84e0 2024-03-23T09:50:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.337+0000 7fb683fff640 1 -- 172.21.15.47:0/1800754056 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c012d10 con 0x55c1cced84e0 2024-03-23T09:50:54.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.337+0000 7fb6aa876740 1 -- 172.21.15.47:0/1800754056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 msgr2=0x55c1cceb78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.337+0000 7fb6aa876740 1 --2- 172.21.15.47:0/1800754056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 0x55c1cceb78b0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0012e0 tx=0x7fb69c007000 comp rx=0 tx=0).stop 2024-03-23T09:50:54.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.337+0000 7fb6aa876740 1 -- 172.21.15.47:0/1800754056 shutdown_connections 2024-03-23T09:50:54.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.337+0000 7fb6aa876740 1 --2- 172.21.15.47:0/1800754056 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1ccee3830 0x55c1cceb7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.337+0000 7fb6aa876740 1 --2- 172.21.15.47:0/1800754056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1cced84e0 0x55c1cceb78b0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.337+0000 7fb6aa876740 1 --2- 172.21.15.47:0/1800754056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccedfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.337+0000 7fb6aa876740 1 -- 172.21.15.47:0/1800754056 >> 172.21.15.47:0/1800754056 conn(0x55c1cceb3850 msgr2=0x55c1ccee0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.337+0000 7fb6aa876740 1 -- 172.21.15.47:0/1800754056 shutdown_connections 2024-03-23T09:50:54.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.337+0000 7fb6aa876740 1 -- 172.21.15.47:0/1800754056 wait complete. 2024-03-23T09:50:54.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.338+0000 7fb6aa876740 1 -- start start 2024-03-23T09:50:54.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.338+0000 7fb6aa876740 1 Processor -- start 2024-03-23T09:50:54.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6aa876740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccefc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6aa876740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1cced84e0 0x55c1cceb8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6aa876740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1ccee3830 0x55c1ccf0cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6aa876740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ccefbd80 con 0x55c1ccee3830 2024-03-23T09:50:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6aa876740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1cceb9290 con 0x55c1cceb73b0 2024-03-23T09:50:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6aa876740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1ccf17a70 con 0x55c1cced84e0 2024-03-23T09:50:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6aa876740 10 client.0 _ll_register_callbacks cb 0x7fb69c001ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:50:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6a629d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1ccee3830 0x55c1ccf0cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6a629d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1ccee3830 0x55c1ccf0cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59764/0 (socket says 172.21.15.47:59764) 2024-03-23T09:50:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6a629d640 1 -- 172.21.15.47:0/4031782457 learned_addr learned my addr 172.21.15.47:0/4031782457 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:50:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccefc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6a529b640 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1cced84e0 0x55c1cceb8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6a629d640 1 -- 172.21.15.47:0/4031782457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1cced84e0 msgr2=0x55c1cceb8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6a629d640 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1cced84e0 0x55c1cceb8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6a629d640 1 -- 172.21.15.47:0/4031782457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 msgr2=0x55c1ccefc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:50:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6a629d640 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccefc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6a629d640 1 -- 172.21.15.47:0/4031782457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1ccde4470 con 0x55c1ccee3830 2024-03-23T09:50:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccefc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6a529b640 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1cced84e0 0x55c1cceb8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:50:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb6a629d640 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1ccee3830 0x55c1ccf0cd10 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fb698016710 tx=0x7fb698004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79254206158fec3e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:50:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb680ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:50:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.339+0000 7fb680ff9640 1 -- 172.21.15.47:0/4031782457 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb698009450 con 0x55c1ccee3830 2024-03-23T09:50:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb680ff9640 1 -- 172.21.15.47:0/4031782457 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb698009e40 con 0x55c1ccee3830 2024-03-23T09:50:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb6aa876740 1 -- 172.21.15.47:0/4031782457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb69c012980 con 0x55c1ccee3830 2024-03-23T09:50:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb680ff9640 1 -- 172.21.15.47:0/4031782457 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb698006a60 con 0x55c1ccee3830 2024-03-23T09:50:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb6aa876740 10 client.9045 fetch_fsmap learned FSMap version 71 2024-03-23T09:50:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb6aa876740 1 -- 172.21.15.47:0/4031782457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55c1ccdbd5c0 con 0x55c1ccee3830 2024-03-23T09:50:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb680ff9640 1 -- 172.21.15.47:0/4031782457 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c012980 con 0x55c1ccee3830 2024-03-23T09:50:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb680ff9640 1 -- 172.21.15.47:0/4031782457 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb698020640 con 0x55c1ccee3830 2024-03-23T09:50:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb6aa876740 10 client.9045 fetch_fsmap finished waiting for FSMap version 71 2024-03-23T09:50:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb6aa876740 10 client.9045 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-03-23T09:50:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb6aa876740 20 client.9045 populate_metadata read hostname 'smithi047' 2024-03-23T09:50:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb6aa876740 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55c1ccee4a10 0x55c1ccf0d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:50:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb6aa876740 4 client.9045 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:50:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb6aa876740 1 -- 172.21.15.47:0/4031782457 --> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c1ccb104e0 con 0x55c1ccee4a10 2024-03-23T09:50:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55c1ccee4a10 0x55c1ccf0d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:50:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb680ff9640 10 client.9045 ms_handle_connect on v2:172.21.15.47:6833/1748845936 2024-03-23T09:50:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb6a5a9c640 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55c1ccee4a10 0x55c1ccf0d250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac8d9c09d7caf714 server_cookie=4b8f0b1b0a095ae8 in_seq=0 out_seq=0 2024-03-23T09:50:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb680ff9640 1 -- 172.21.15.47:0/4031782457 <== mds.0 v2:172.21.15.47:6833/1748845936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55c1ccb104e0 con 0x55c1ccee4a10 2024-03-23T09:50:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.340+0000 7fb680ff9640 10 client.9045 handle_command_reply: tid=0 2024-03-23T09:50:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6aa876740 1 client.9045 shutdown 2024-03-23T09:50:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6aa876740 1 -- 172.21.15.47:0/4031782457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1ccee3830 msgr2=0x55c1ccf0cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6aa876740 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1ccee3830 0x55c1ccf0cd10 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fb698016710 tx=0x7fb698004730 comp rx=0 tx=0).stop 2024-03-23T09:50:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6aa876740 1 -- 172.21.15.47:0/4031782457 shutdown_connections 2024-03-23T09:50:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6aa876740 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55c1ccee4a10 0x55c1ccf0d250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6aa876740 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1ccee3830 0x55c1ccf0cd10 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6aa876740 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c1cced84e0 0x55c1cceb8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb680ff9640 0 client.9045 ms_handle_reset on v2:172.21.15.47:6833/1748845936 2024-03-23T09:50:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6aa876740 1 --2- 172.21.15.47:0/4031782457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1cceb73b0 0x55c1ccefc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6aa876740 1 -- 172.21.15.47:0/4031782457 >> 172.21.15.47:0/4031782457 conn(0x55c1cceb3850 msgr2=0x55c1ccee0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6aa876740 1 -- 172.21.15.47:0/4031782457 shutdown_connections 2024-03-23T09:50:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6aa876740 1 -- 172.21.15.47:0/4031782457 wait complete. 2024-03-23T09:50:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6aa876740 20 client.9045 trim_cache size 0 max 16384 2024-03-23T09:50:54.349 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:50:54.349 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T09:50:54.349 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-23T09:50:54.349 INFO:teuthology.orchestra.run.smithi047.stdout: "68e7cb3a-2c03-4bba-9802-6486b239533a": { 2024-03-23T09:50:54.349 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-23T09:50:54.349 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "68e7cb3a-2c03-4bba-9802-6486b239533a", 2024-03-23T09:50:54.350 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,repair" 2024-03-23T09:50:54.350 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:50:54.350 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:50:54.350 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:50:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6a7d00640 1 -- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb68406ac00 msgr2=0x7fb68406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6a7d00640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb68406ac00 0x7fb68406d0c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00d2840 tx=0x7fb69c009040 comp rx=0 tx=0).stop 2024-03-23T09:50:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6a7d00640 1 -- 172.21.15.47:0/2443992732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a00cde40 msgr2=0x7fb6a00c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:50:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6a7d00640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a00cde40 0x7fb6a00c8790 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fb6940098c0 tx=0x7fb6940046c0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6a7d00640 1 -- 172.21.15.47:0/2443992732 shutdown_connections 2024-03-23T09:50:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6a7d00640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb68406ac00 0x7fb68406d0c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6a7d00640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6a015ce10 0x7fb6a015dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6a7d00640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a015ca30 0x7fb6a0156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6a7d00640 1 --2- 172.21.15.47:0/2443992732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a00cde40 0x7fb6a00c8790 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:50:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6a7d00640 1 -- 172.21.15.47:0/2443992732 >> 172.21.15.47:0/2443992732 conn(0x7fb6a006b160 msgr2=0x7fb6a00c9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:50:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6a7d00640 1 -- 172.21.15.47:0/2443992732 shutdown_connections 2024-03-23T09:50:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:54.341+0000 7fb6a7d00640 1 -- 172.21.15.47:0/2443992732 wait complete. 2024-03-23T09:50:54.353 INFO:tasks.cephfs.filesystem:scrub status for tag:68e7cb3a-2c03-4bba-9802-6486b239533a - {'path': '/', 'tag': '68e7cb3a-2c03-4bba-9802-6486b239533a', 'options': 'recursive,repair'} 2024-03-23T09:50:54.605 INFO:tasks.ceph.mds.a.smithi047.stderr:2024-03-23T09:50:54.605+0000 7f757bb41640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:50:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:55.618+0000 7f46717fa640 1 -- 172.21.15.47:0/2413808955 <== mon.0 v2:172.21.15.47:3300/0 8 ==== log(8 entries from seq 1 at 2024-03-23T09:50:54.187899+0000) v1 ==== 1407+0+0 (secure 0 0 0) 0x7f466805c5c0 con 0x7f467406a3f0 2024-03-23T09:50:55.618 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:54.187899+0000 mds.a [INF] scrub queued for path: / 2024-03-23T09:50:55.619 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:54.187904+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-03-23T09:50:55.619 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:54.187911+0000 mds.a [INF] scrub summary: active paths [/] 2024-03-23T09:50:55.619 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:54.188634+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-03-23T09:50:55.619 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:54.188641+0000 mds.a [ERR] inode table repaired for inode: 0x10000000001 2024-03-23T09:50:55.619 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:54.188702+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-03-23T09:50:55.619 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:54.188708+0000 mds.a [ERR] inode table repaired for inode: 0x10000000002 2024-03-23T09:50:55.619 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:54.189251+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-03-23T09:50:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:50:59.882+0000 7f46717fa640 1 -- 172.21.15.47:0/2413808955 <== mon.0 v2:172.21.15.47:3300/0 9 ==== log(1 entries from seq 9 at 2024-03-23T09:50:58.635920+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f466808a000 con 0x7f467406a3f0 2024-03-23T09:50:59.883 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:50:58.635920+0000 mds.a [INF] scrub summary: idle 2024-03-23T09:51:08.828 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:51:08.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:51:24.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:24.354 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-23T09:51:24.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.469+0000 7fac141e0640 1 Processor -- start 2024-03-23T09:51:24.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.469+0000 7fac141e0640 1 -- start start 2024-03-23T09:51:24.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac141e0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac0c15ca30 0x7fac0c160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac141e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac0c15ce10 0x7fac0c0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac141e0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0c0d25e0 con 0x7fac0c0b85d0 2024-03-23T09:51:24.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac141e0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac0c161600 con 0x7fac0c15ca30 2024-03-23T09:51:24.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac141e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c161040 con 0x7fac0c15ce10 2024-03-23T09:51:24.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac11f7c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac0c15ca30 0x7fac0c160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac1277d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fac0c0b85d0 0x7fac0c0b43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40038/0 (socket says 172.21.15.47:40038) 2024-03-23T09:51:24.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac1277d640 1 -- 172.21.15.47:0/2596708749 learned_addr learned my addr 172.21.15.47:0/2596708749 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:24.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac1177b640 1 --2- 172.21.15.47:0/2596708749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac0c15ce10 0x7fac0c0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac11f7c640 1 -- 172.21.15.47:0/2596708749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac0c15ce10 msgr2=0x7fac0c0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:24.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac11f7c640 1 --2- 172.21.15.47:0/2596708749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac0c15ce10 0x7fac0c0cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac11f7c640 1 -- 172.21.15.47:0/2596708749 >> v1:172.21.15.47:6789/0 conn(0x7fac0c0b85d0 legacy=0x7fac0c0b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac11f7c640 1 -- 172.21.15.47:0/2596708749 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c161320 con 0x7fac0c15ca30 2024-03-23T09:51:24.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.470+0000 7fac10f7a640 1 -- 172.21.15.47:0/2596708749 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178757228 0 0) 0x7fac0c0d25e0 con 0x7fac0c0b85d0 2024-03-23T09:51:24.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.471+0000 7fac11f7c640 1 --2- 172.21.15.47:0/2596708749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac0c15ca30 0x7fac0c160740 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fac080097a0 tx=0x7fac0802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2362a9ba375fe55c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:24.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.471+0000 7fac10f7a640 1 -- 172.21.15.47:0/2596708749 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08031830 con 0x7fac0c15ca30 2024-03-23T09:51:24.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.471+0000 7fac10f7a640 1 -- 172.21.15.47:0/2596708749 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac08031990 con 0x7fac0c15ca30 2024-03-23T09:51:24.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.471+0000 7fac10f7a640 1 -- 172.21.15.47:0/2596708749 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08031d20 con 0x7fac0c15ca30 2024-03-23T09:51:24.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.472+0000 7fac141e0640 1 -- 172.21.15.47:0/2596708749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac0c15ca30 msgr2=0x7fac0c160740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.472+0000 7fac141e0640 1 --2- 172.21.15.47:0/2596708749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac0c15ca30 0x7fac0c160740 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fac080097a0 tx=0x7fac0802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.472+0000 7fac141e0640 1 -- 172.21.15.47:0/2596708749 shutdown_connections 2024-03-23T09:51:24.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.472+0000 7fac141e0640 1 --2- 172.21.15.47:0/2596708749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac0c15ce10 0x7fac0c0cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.472+0000 7fac141e0640 1 --2- 172.21.15.47:0/2596708749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac0c15ca30 0x7fac0c160740 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.472+0000 7fac141e0640 1 -- 172.21.15.47:0/2596708749 >> 172.21.15.47:0/2596708749 conn(0x7fac0c06b160 msgr2=0x7fac0c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:24.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.472+0000 7fac141e0640 1 -- 172.21.15.47:0/2596708749 shutdown_connections 2024-03-23T09:51:24.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.472+0000 7fac141e0640 1 -- 172.21.15.47:0/2596708749 wait complete. 2024-03-23T09:51:24.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.473+0000 7fac141e0640 1 Processor -- start 2024-03-23T09:51:24.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.473+0000 7fac141e0640 1 -- start start 2024-03-23T09:51:24.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac141e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac0c0b85d0 0x7fac0c0d1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac141e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac0c15ca30 0x7fac0c15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac141e0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac0c15ce10 0x7fac0c15be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac141e0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c0c6560 con 0x7fac0c15ce10 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac141e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c0c6840 con 0x7fac0c0b85d0 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac141e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c0c6b20 con 0x7fac0c15ca30 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac1277d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac0c15ce10 0x7fac0c15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac1277d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac0c15ce10 0x7fac0c15be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37094/0 (socket says 172.21.15.47:37094) 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac1277d640 1 -- 172.21.15.47:0/3950015449 learned_addr learned my addr 172.21.15.47:0/3950015449 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac1177b640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac0c15ca30 0x7fac0c15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac11f7c640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac0c0b85d0 0x7fac0c0d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac1277d640 1 -- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac0c15ca30 msgr2=0x7fac0c15b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac1277d640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac0c15ca30 0x7fac0c15b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac1277d640 1 -- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac0c0b85d0 msgr2=0x7fac0c0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac1277d640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac0c0b85d0 0x7fac0c0d1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.474+0000 7fac1277d640 1 -- 172.21.15.47:0/3950015449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac08009400 con 0x7fac0c15ce10 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.475+0000 7fac1277d640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac0c15ce10 0x7fac0c15be00 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fac0400c7e0 tx=0x7fac0400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.475+0000 7fac027fc640 1 -- 172.21.15.47:0/3950015449 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0400a640 con 0x7fac0c15ce10 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.475+0000 7fac027fc640 1 -- 172.21.15.47:0/3950015449 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0400a7c0 con 0x7fac0c15ce10 2024-03-23T09:51:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.475+0000 7fac141e0640 1 -- 172.21.15.47:0/3950015449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac0c0c6e00 con 0x7fac0c15ce10 2024-03-23T09:51:24.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.475+0000 7fac027fc640 1 -- 172.21.15.47:0/3950015449 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0400ab70 con 0x7fac0c15ce10 2024-03-23T09:51:24.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.475+0000 7fac141e0640 1 -- 172.21.15.47:0/3950015449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0c15f4a0 con 0x7fac0c15ce10 2024-03-23T09:51:24.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.476+0000 7fac027fc640 1 -- 172.21.15.47:0/3950015449 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fac0400acd0 con 0x7fac0c15ce10 2024-03-23T09:51:24.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.477+0000 7fac027fc640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fabf006ac00 0x7fabf006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.477+0000 7fac027fc640 1 -- 172.21.15.47:0/3950015449 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fac040889a0 con 0x7fac0c15ce10 2024-03-23T09:51:24.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.477+0000 7fac11f7c640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fabf006ac00 0x7fabf006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.478+0000 7fac11f7c640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fabf006ac00 0x7fabf006d0c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fac08036000 tx=0x7fac08008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:24.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.478+0000 7fac16e32740 1 Processor -- start 2024-03-23T09:51:24.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.478+0000 7fac16e32740 1 -- start start 2024-03-23T09:51:24.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac16e32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffdf4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac16e32740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdc4850 0x55a6ffdf1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac16e32740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdf1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac16e32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ffd34dd0 con 0x55a6ffdf4830 2024-03-23T09:51:24.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac16e32740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ffd32560 con 0x55a6ffdc83b0 2024-03-23T09:51:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac16e32740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6ffd2de50 con 0x55a6ffdc4850 2024-03-23T09:51:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac11f7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffdf4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac11f7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffdf4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37100/0 (socket says 172.21.15.47:37100) 2024-03-23T09:51:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac11f7c640 1 -- 172.21.15.47:0/3860445587 learned_addr learned my addr 172.21.15.47:0/3860445587 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac1277d640 1 --2- 172.21.15.47:0/3860445587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdf1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac1177b640 1 --2- 172.21.15.47:0/3860445587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdc4850 0x55a6ffdf1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac11f7c640 1 -- 172.21.15.47:0/3860445587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdc4850 msgr2=0x55a6ffdf1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac11f7c640 1 --2- 172.21.15.47:0/3860445587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdc4850 0x55a6ffdf1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac11f7c640 1 -- 172.21.15.47:0/3860445587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdc83b0 msgr2=0x55a6ffdf1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac11f7c640 1 --2- 172.21.15.47:0/3860445587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdf1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac11f7c640 1 -- 172.21.15.47:0/3860445587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6ffcf5470 con 0x55a6ffdf4830 2024-03-23T09:51:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.479+0000 7fac1277d640 1 --2- 172.21.15.47:0/3860445587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdf1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.480+0000 7fac11f7c640 1 --2- 172.21.15.47:0/3860445587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffdf4c10 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fac0802bee0 tx=0x7fac08030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122895da046a334b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.480+0000 7fabef7fe640 1 -- 172.21.15.47:0/3860445587 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0803f070 con 0x55a6ffdf4830 2024-03-23T09:51:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.480+0000 7fabef7fe640 1 -- 172.21.15.47:0/3860445587 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac08047b00 con 0x55a6ffdf4830 2024-03-23T09:51:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.480+0000 7fabef7fe640 1 -- 172.21.15.47:0/3860445587 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08045420 con 0x55a6ffdf4830 2024-03-23T09:51:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.480+0000 7fac16e32740 1 -- 172.21.15.47:0/3860445587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 msgr2=0x55a6ffdf4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.480+0000 7fac16e32740 1 --2- 172.21.15.47:0/3860445587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffdf4c10 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fac0802bee0 tx=0x7fac08030000 comp rx=0 tx=0).stop 2024-03-23T09:51:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.480+0000 7fac16e32740 1 -- 172.21.15.47:0/3860445587 shutdown_connections 2024-03-23T09:51:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.480+0000 7fac16e32740 1 --2- 172.21.15.47:0/3860445587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdf1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.480+0000 7fac16e32740 1 --2- 172.21.15.47:0/3860445587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdc4850 0x55a6ffdf1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.480+0000 7fac16e32740 1 --2- 172.21.15.47:0/3860445587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffdf4c10 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.480+0000 7fac16e32740 1 -- 172.21.15.47:0/3860445587 >> 172.21.15.47:0/3860445587 conn(0x55a6ffdc6ac0 msgr2=0x55a6ffe0f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:24.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.481+0000 7fac16e32740 1 -- 172.21.15.47:0/3860445587 shutdown_connections 2024-03-23T09:51:24.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.481+0000 7fac16e32740 1 -- 172.21.15.47:0/3860445587 wait complete. 2024-03-23T09:51:24.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.481+0000 7fac16e32740 1 -- start start 2024-03-23T09:51:24.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.481+0000 7fac16e32740 1 Processor -- start 2024-03-23T09:51:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.482+0000 7fac16e32740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdc83b0 0x55a6ffe0fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.482+0000 7fac16e32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffdee170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.482+0000 7fac16e32740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdf7dd0 0x55a6ffdf8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.482+0000 7fac16e32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ffcec280 con 0x55a6ffdf4830 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.482+0000 7fac16e32740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ffcebf20 con 0x55a6ffdf7dd0 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.482+0000 7fac16e32740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6ffcb13c0 con 0x55a6ffdc83b0 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.482+0000 7fac16e32740 10 client.0 _ll_register_callbacks cb 0x55a6ffd03380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.482+0000 7fac1177b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffdee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.482+0000 7fac1177b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffdee170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37112/0 (socket says 172.21.15.47:37112) 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.482+0000 7fac1177b640 1 -- 172.21.15.47:0/2202192843 learned_addr learned my addr 172.21.15.47:0/2202192843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.482+0000 7fac11f7c640 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdc83b0 0x55a6ffe0fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac1277d640 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdf7dd0 0x55a6ffdf8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac1177b640 1 -- 172.21.15.47:0/2202192843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdc83b0 msgr2=0x55a6ffe0fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac1177b640 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdc83b0 0x55a6ffe0fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac1177b640 1 -- 172.21.15.47:0/2202192843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdf7dd0 msgr2=0x55a6ffdf8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac1177b640 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdf7dd0 0x55a6ffdf8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac1177b640 1 -- 172.21.15.47:0/2202192843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0c161320 con 0x55a6ffdf4830 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac1277d640 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdf7dd0 0x55a6ffdf8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac11f7c640 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdc83b0 0x55a6ffe0fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac1177b640 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffdee170 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00ddb0 tx=0x7fabfc00f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c56ed746ca58c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fabecff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fabecff9640 1 -- 172.21.15.47:0/2202192843 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc014680 con 0x55a6ffdf4830 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac16e32740 1 -- 172.21.15.47:0/2202192843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6ffd57690 con 0x55a6ffdf4830 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fabecff9640 1 -- 172.21.15.47:0/2202192843 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fabfc00fe70 con 0x55a6ffdf4830 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fabecff9640 1 -- 172.21.15.47:0/2202192843 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabfc0147e0 con 0x55a6ffdf4830 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac16e32740 10 client.9060 fetch_fsmap learned FSMap version 71 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac16e32740 1 -- 172.21.15.47:0/2202192843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55a6ffd2c4d0 con 0x55a6ffdf4830 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fabecff9640 1 -- 172.21.15.47:0/2202192843 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a6ffd57690 con 0x55a6ffdf4830 2024-03-23T09:51:24.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fabecff9640 1 -- 172.21.15.47:0/2202192843 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fabfc014940 con 0x55a6ffdf4830 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac16e32740 10 client.9060 fetch_fsmap finished waiting for FSMap version 71 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac16e32740 10 client.9060 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac16e32740 20 client.9060 populate_metadata read hostname 'smithi047' 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac16e32740 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55a6ffddde60 0x55a6ffdde240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac16e32740 4 client.9060 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.483+0000 7fac16e32740 1 -- 172.21.15.47:0/2202192843 --> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6ffd1feb0 con 0x55a6ffddde60 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.484+0000 7fac11f7c640 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55a6ffddde60 0x55a6ffdde240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.484+0000 7fac11f7c640 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55a6ffddde60 0x55a6ffdde240 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be3a679d5fea5c58 server_cookie=f28a9c6e0a214cb8 in_seq=0 out_seq=0 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.484+0000 7fabecff9640 10 client.9060 ms_handle_connect on v2:172.21.15.47:6833/1748845936 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.484+0000 7fabecff9640 1 -- 172.21.15.47:0/2202192843 <== mds.0 v2:172.21.15.47:6833/1748845936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a6ffd1feb0 con 0x55a6ffddde60 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.484+0000 7fabecff9640 10 client.9060 handle_command_reply: tid=0 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.484+0000 7fac16e32740 1 client.9060 shutdown 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.485+0000 7fac16e32740 1 -- 172.21.15.47:0/2202192843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 msgr2=0x55a6ffdee170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.485+0000 7fac16e32740 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffdee170 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00ddb0 tx=0x7fabfc00f5b0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.485+0000 7fac16e32740 1 -- 172.21.15.47:0/2202192843 shutdown_connections 2024-03-23T09:51:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.485+0000 7fac16e32740 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55a6ffddde60 0x55a6ffdde240 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.485+0000 7fac16e32740 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdf7dd0 0x55a6ffdf8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.485+0000 7fac16e32740 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffdee170 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.485+0000 7fac16e32740 1 --2- 172.21.15.47:0/2202192843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdc83b0 0x55a6ffe0fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.485+0000 7fabecff9640 0 client.9060 ms_handle_reset on v2:172.21.15.47:6833/1748845936 2024-03-23T09:51:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.485+0000 7fac16e32740 1 -- 172.21.15.47:0/2202192843 >> 172.21.15.47:0/2202192843 conn(0x55a6ffdc4850 msgr2=0x55a6ffdc8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.485+0000 7fac16e32740 1 -- 172.21.15.47:0/2202192843 shutdown_connections 2024-03-23T09:51:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.485+0000 7fac16e32740 1 -- 172.21.15.47:0/2202192843 wait complete. 2024-03-23T09:51:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.485+0000 7fac16e32740 20 client.9060 trim_cache size 0 max 16384 2024-03-23T09:51:24.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.499+0000 7fac16e32740 1 Processor -- start 2024-03-23T09:51:24.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.499+0000 7fac16e32740 1 -- start start 2024-03-23T09:51:24.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.500+0000 7fac16e32740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdc0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.500+0000 7fac16e32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffe0f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.500+0000 7fac16e32740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdf7dd0 0x55a6ffe0fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.500+0000 7fac16e32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ffdc0c50 con 0x55a6ffdf4830 2024-03-23T09:51:24.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.500+0000 7fac16e32740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ffdc8a90 con 0x55a6ffdc83b0 2024-03-23T09:51:24.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.500+0000 7fac16e32740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6ffe103c0 con 0x55a6ffdf7dd0 2024-03-23T09:51:24.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.500+0000 7fac1177b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffe0f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.500+0000 7fac1177b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffe0f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37114/0 (socket says 172.21.15.47:37114) 2024-03-23T09:51:24.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.500+0000 7fac1177b640 1 -- 172.21.15.47:0/1322118050 learned_addr learned my addr 172.21.15.47:0/1322118050 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:24.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.500+0000 7fac1277d640 1 --2- 172.21.15.47:0/1322118050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdf7dd0 0x55a6ffe0fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.500+0000 7fac11f7c640 1 --2- 172.21.15.47:0/1322118050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdc0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac1177b640 1 -- 172.21.15.47:0/1322118050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdf7dd0 msgr2=0x55a6ffe0fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:24.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac1177b640 1 --2- 172.21.15.47:0/1322118050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdf7dd0 0x55a6ffe0fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac1177b640 1 -- 172.21.15.47:0/1322118050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdc83b0 msgr2=0x55a6ffdc0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac1177b640 1 --2- 172.21.15.47:0/1322118050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdc0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac1177b640 1 -- 172.21.15.47:0/1322118050 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6ffd609e0 con 0x55a6ffdf4830 2024-03-23T09:51:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac11f7c640 1 --2- 172.21.15.47:0/1322118050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdc0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac1177b640 1 --2- 172.21.15.47:0/1322118050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffe0f940 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00d850 tx=0x7fabfc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f62b62195bd7445 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fabeffff640 1 -- 172.21.15.47:0/1322118050 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc00fe70 con 0x55a6ffdf4830 2024-03-23T09:51:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fabeffff640 1 -- 172.21.15.47:0/1322118050 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc00c030 con 0x55a6ffdf4830 2024-03-23T09:51:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fabeffff640 1 -- 172.21.15.47:0/1322118050 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc004280 con 0x55a6ffdf4830 2024-03-23T09:51:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac16e32740 1 -- 172.21.15.47:0/1322118050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 msgr2=0x55a6ffe0f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac16e32740 1 --2- 172.21.15.47:0/1322118050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffe0f940 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00d850 tx=0x7fabfc014000 comp rx=0 tx=0).stop 2024-03-23T09:51:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac16e32740 1 -- 172.21.15.47:0/1322118050 shutdown_connections 2024-03-23T09:51:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac16e32740 1 --2- 172.21.15.47:0/1322118050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdf7dd0 0x55a6ffe0fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac16e32740 1 --2- 172.21.15.47:0/1322118050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdf4830 0x55a6ffe0f940 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac16e32740 1 --2- 172.21.15.47:0/1322118050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdc0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.501+0000 7fac16e32740 1 -- 172.21.15.47:0/1322118050 >> 172.21.15.47:0/1322118050 conn(0x55a6ffdc4850 msgr2=0x55a6ffdef430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:24.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.502+0000 7fac16e32740 1 -- 172.21.15.47:0/1322118050 shutdown_connections 2024-03-23T09:51:24.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.502+0000 7fac16e32740 1 -- 172.21.15.47:0/1322118050 wait complete. 2024-03-23T09:51:24.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.502+0000 7fac16e32740 1 -- start start 2024-03-23T09:51:24.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.502+0000 7fac16e32740 1 Processor -- start 2024-03-23T09:51:24.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac16e32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac16e32740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdf4830 0x55a6ffdecd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac16e32740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdf7dd0 0x55a6ffe75700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac16e32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ffe76a00 con 0x55a6ffdc83b0 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac16e32740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ffdc3a10 con 0x55a6ffdf4830 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac16e32740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6ffdc3bc0 con 0x55a6ffdf7dd0 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac16e32740 10 client.0 _ll_register_callbacks cb 0x7fac08009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac11f7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac11f7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdec810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37130/0 (socket says 172.21.15.47:37130) 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac11f7c640 1 -- 172.21.15.47:0/2232858273 learned_addr learned my addr 172.21.15.47:0/2232858273 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac1277d640 1 --2- 172.21.15.47:0/2232858273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdf7dd0 0x55a6ffe75700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac1177b640 1 --2- 172.21.15.47:0/2232858273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdf4830 0x55a6ffdecd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac11f7c640 1 -- 172.21.15.47:0/2232858273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdf7dd0 msgr2=0x55a6ffe75700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac11f7c640 1 --2- 172.21.15.47:0/2232858273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdf7dd0 0x55a6ffe75700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac11f7c640 1 -- 172.21.15.47:0/2232858273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdf4830 msgr2=0x55a6ffdecd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:24.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac11f7c640 1 --2- 172.21.15.47:0/2232858273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdf4830 0x55a6ffdecd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.503+0000 7fac11f7c640 1 -- 172.21.15.47:0/2232858273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6ffd2c4d0 con 0x55a6ffdc83b0 2024-03-23T09:51:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fac11f7c640 1 --2- 172.21.15.47:0/2232858273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdec810 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fac080098c0 tx=0x7fac0802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f3bf21d145bc34 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fabecff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:51:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fabecff9640 1 -- 172.21.15.47:0/2232858273 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0805f360 con 0x55a6ffdc83b0 2024-03-23T09:51:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fabecff9640 1 -- 172.21.15.47:0/2232858273 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fac0805f900 con 0x55a6ffdc83b0 2024-03-23T09:51:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fac16e32740 1 -- 172.21.15.47:0/2232858273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6ffdc5ed0 con 0x55a6ffdc83b0 2024-03-23T09:51:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fabecff9640 1 -- 172.21.15.47:0/2232858273 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac0805fac0 con 0x55a6ffdc83b0 2024-03-23T09:51:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fac16e32740 10 client.9066 fetch_fsmap learned FSMap version 71 2024-03-23T09:51:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fac16e32740 1 -- 172.21.15.47:0/2232858273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55a6ffcce5c0 con 0x55a6ffdc83b0 2024-03-23T09:51:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fabecff9640 1 -- 172.21.15.47:0/2232858273 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a6ffdc5ed0 con 0x55a6ffdc83b0 2024-03-23T09:51:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fabecff9640 1 -- 172.21.15.47:0/2232858273 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fac0805a040 con 0x55a6ffdc83b0 2024-03-23T09:51:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fac16e32740 10 client.9066 fetch_fsmap finished waiting for FSMap version 71 2024-03-23T09:51:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fac16e32740 10 client.9066 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fac16e32740 20 client.9066 populate_metadata read hostname 'smithi047' 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fac16e32740 1 --2- 172.21.15.47:0/2232858273 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55a6ffdef6e0 0x55a6ffe75c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fac16e32740 4 client.9066 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fac16e32740 1 -- 172.21.15.47:0/2232858273 --> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fac08047b00 con 0x55a6ffdef6e0 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.504+0000 7fac1177b640 1 --2- 172.21.15.47:0/2232858273 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55a6ffdef6e0 0x55a6ffe75c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fac1177b640 1 --2- 172.21.15.47:0/2232858273 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55a6ffdef6e0 0x55a6ffe75c40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a009b7903bc070f9 server_cookie=9670b6ebaa320a14 in_seq=0 out_seq=0 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fabecff9640 10 client.9066 ms_handle_connect on v2:172.21.15.47:6833/1748845936 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fabecff9640 1 -- 172.21.15.47:0/2232858273 <== mds.0 v2:172.21.15.47:6833/1748845936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fac08047b00 con 0x55a6ffdef6e0 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fabecff9640 10 client.9066 handle_command_reply: tid=0 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fac16e32740 1 client.9066 shutdown 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fac16e32740 1 -- 172.21.15.47:0/2232858273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdc83b0 msgr2=0x55a6ffdec810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fac16e32740 1 --2- 172.21.15.47:0/2232858273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdec810 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fac080098c0 tx=0x7fac0802e000 comp rx=0 tx=0).stop 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fac16e32740 1 -- 172.21.15.47:0/2232858273 shutdown_connections 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fac16e32740 1 --2- 172.21.15.47:0/2232858273 >> [v2:172.21.15.47:6833/1748845936,v1:172.21.15.47:6835/1748845936] conn(0x55a6ffdef6e0 0x55a6ffe75c40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fac16e32740 1 --2- 172.21.15.47:0/2232858273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a6ffdf7dd0 0x55a6ffe75700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fac16e32740 1 --2- 172.21.15.47:0/2232858273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6ffdf4830 0x55a6ffdecd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fabecff9640 0 client.9066 ms_handle_reset on v2:172.21.15.47:6833/1748845936 2024-03-23T09:51:24.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fac16e32740 1 --2- 172.21.15.47:0/2232858273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6ffdc83b0 0x55a6ffdec810 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fac16e32740 1 -- 172.21.15.47:0/2232858273 >> 172.21.15.47:0/2232858273 conn(0x55a6ffdc4850 msgr2=0x55a6ffdedd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:24.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fac16e32740 1 -- 172.21.15.47:0/2232858273 shutdown_connections 2024-03-23T09:51:24.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fac16e32740 1 -- 172.21.15.47:0/2232858273 wait complete. 2024-03-23T09:51:24.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.505+0000 7fac16e32740 20 client.9066 trim_cache size 0 max 16384 2024-03-23T09:51:24.512 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:51:24.512 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-23T09:51:24.512 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-23T09:51:24.512 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:51:24.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.506+0000 7fac141e0640 1 -- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fabf006ac00 msgr2=0x7fabf006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.506+0000 7fac141e0640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fabf006ac00 0x7fabf006d0c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fac08036000 tx=0x7fac08008030 comp rx=0 tx=0).stop 2024-03-23T09:51:24.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.506+0000 7fac141e0640 1 -- 172.21.15.47:0/3950015449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac0c15ce10 msgr2=0x7fac0c15be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.506+0000 7fac141e0640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac0c15ce10 0x7fac0c15be00 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fac0400c7e0 tx=0x7fac0400cca0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.506+0000 7fac141e0640 1 -- 172.21.15.47:0/3950015449 shutdown_connections 2024-03-23T09:51:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.506+0000 7fac141e0640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fabf006ac00 0x7fabf006d0c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.506+0000 7fac141e0640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac0c15ce10 0x7fac0c15be00 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.506+0000 7fac141e0640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac0c15ca30 0x7fac0c15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.506+0000 7fac141e0640 1 --2- 172.21.15.47:0/3950015449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac0c0b85d0 0x7fac0c0d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.506+0000 7fac141e0640 1 -- 172.21.15.47:0/3950015449 >> 172.21.15.47:0/3950015449 conn(0x7fac0c06b160 msgr2=0x7fac0c0cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.506+0000 7fac141e0640 1 -- 172.21.15.47:0/3950015449 shutdown_connections 2024-03-23T09:51:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.506+0000 7fac141e0640 1 -- 172.21.15.47:0/3950015449 wait complete. 2024-03-23T09:51:24.517 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T09:51:24.618 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:51:24.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.720+0000 7f8f31215640 1 Processor -- start 2024-03-23T09:51:24.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.720+0000 7f8f31215640 1 -- start start 2024-03-23T09:51:24.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.721+0000 7f8f31215640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c0cf5c0 0x7f8f2c099740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.721+0000 7f8f31215640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8f2c0ce2e0 0x7f8f2c0b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.721+0000 7f8f31215640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f2c0b1840 con 0x7f8f2c098a00 2024-03-23T09:51:24.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.721+0000 7f8f31215640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2c0cb320 con 0x7f8f2c0ce2e0 2024-03-23T09:51:24.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.721+0000 7f8f31215640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c0cb5a0 con 0x7f8f2c0cf5c0 2024-03-23T09:51:24.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.721+0000 7f8f2b59e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8f2c098a00 0x7f8f2c0cba80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50892/0 (socket says 172.21.15.47:50892) 2024-03-23T09:51:24.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.721+0000 7f8f2b59e640 1 -- 172.21.15.47:0/1839081742 learned_addr learned my addr 172.21.15.47:0/1839081742 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:24.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.721+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627860744 0 0) 0x7f8f2c0b1840 con 0x7f8f2c098a00 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f18003520 con 0x7f8f2c098a00 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f2a59c640 1 --2- 172.21.15.47:0/1839081742 >> v2:172.21.15.175:3300/0 conn(0x7f8f2c0ce2e0 0x7f8f2c0b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f2ad9d640 1 --2- 172.21.15.47:0/1839081742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c0cf5c0 0x7f8f2c099740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 6778831 0 0) 0x7f8f18003520 con 0x7f8f2c098a00 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c0cf5c0 msgr2=0x7f8f2c099740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 --2- 172.21.15.47:0/1839081742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c0cf5c0 0x7f8f2c099740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 >> v2:172.21.15.175:3300/0 conn(0x7f8f2c0ce2e0 msgr2=0x7f8f2c0b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 --2- 172.21.15.47:0/1839081742 >> v2:172.21.15.175:3300/0 conn(0x7f8f2c0ce2e0 0x7f8f2c0b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c0ce740 con 0x7f8f2c098a00 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f8f1c0019f0 con 0x7f8f2c098a00 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 >> v1:172.21.15.47:6789/0 conn(0x7f8f2c098a00 legacy=0x7f8f2c0cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 --2- 172.21.15.47:0/1839081742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c0cf5c0 0x7f8f18003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 --2- 172.21.15.47:0/1839081742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f18003ef0 0x7f8f180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 --2- 172.21.15.47:0/1839081742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f18004d00 0x7f8f18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c0cb320 con 0x7f8f18004d00 2024-03-23T09:51:24.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c0cb5a0 con 0x7f8f18003ef0 2024-03-23T09:51:24.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1800b9d0 con 0x7f8f2c0cf5c0 2024-03-23T09:51:24.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f2b59e640 1 --2- 172.21.15.47:0/1839081742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f18004d00 0x7f8f18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f2a59c640 1 --2- 172.21.15.47:0/1839081742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f18003ef0 0x7f8f180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f2ad9d640 1 --2- 172.21.15.47:0/1839081742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c0cf5c0 0x7f8f18003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f2b59e640 1 -- 172.21.15.47:0/1839081742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c0cf5c0 msgr2=0x7f8f18003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:24.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f2b59e640 1 --2- 172.21.15.47:0/1839081742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c0cf5c0 0x7f8f18003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f2b59e640 1 -- 172.21.15.47:0/1839081742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f18003ef0 msgr2=0x7f8f180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:24.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f2b59e640 1 --2- 172.21.15.47:0/1839081742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f18003ef0 0x7f8f180047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.722+0000 7f8f2b59e640 1 -- 172.21.15.47:0/1839081742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f18003520 con 0x7f8f18004d00 2024-03-23T09:51:24.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.723+0000 7f8f2b59e640 1 --2- 172.21.15.47:0/1839081742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f18004d00 0x7f8f18005180 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00a900 tx=0x7f8f180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c20a8e457df24a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:24.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.723+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c034630 con 0x7f8f18004d00 2024-03-23T09:51:24.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.723+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c02f070 con 0x7f8f18004d00 2024-03-23T09:51:24.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.723+0000 7f8f29d9b640 1 -- 172.21.15.47:0/1839081742 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0349d0 con 0x7f8f18004d00 2024-03-23T09:51:24.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.724+0000 7f8f31215640 1 -- 172.21.15.47:0/1839081742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f18004d00 msgr2=0x7f8f18005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.724+0000 7f8f31215640 1 --2- 172.21.15.47:0/1839081742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f18004d00 0x7f8f18005180 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00a900 tx=0x7f8f180057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.724+0000 7f8f2b59e640 1 -- 172.21.15.47:0/1839081742 reap_dead start 2024-03-23T09:51:24.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.724+0000 7f8f31215640 1 -- 172.21.15.47:0/1839081742 shutdown_connections 2024-03-23T09:51:24.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.724+0000 7f8f31215640 1 -- 172.21.15.47:0/1839081742 >> 172.21.15.47:0/1839081742 conn(0x7f8f2c0b2200 msgr2=0x7f8f2c0b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:24.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.724+0000 7f8f31215640 1 -- 172.21.15.47:0/1839081742 shutdown_connections 2024-03-23T09:51:24.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.724+0000 7f8f31215640 1 -- 172.21.15.47:0/1839081742 wait complete. 2024-03-23T09:51:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.725+0000 7f8f31215640 1 Processor -- start 2024-03-23T09:51:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.725+0000 7f8f31215640 1 -- start start 2024-03-23T09:51:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f31215640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f2c1e3610 0x7f8f2c1e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f31215640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f2c06c130 0x7f8f2c1e6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f31215640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c1e6650 0x7f8f2c1d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f31215640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c1e4350 con 0x7f8f2c1e3610 2024-03-23T09:51:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f31215640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c06adf0 con 0x7f8f2c06c130 2024-03-23T09:51:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f31215640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c1e3f30 con 0x7f8f2c1e6650 2024-03-23T09:51:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f2ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f2c1e3610 0x7f8f2c1e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f2ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f2c1e3610 0x7f8f2c1e39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37362/0 (socket says 172.21.15.47:37362) 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f2ad9d640 1 -- 172.21.15.47:0/3545194318 learned_addr learned my addr 172.21.15.47:0/3545194318 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f2b59e640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c1e6650 0x7f8f2c1d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f2a59c640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f2c06c130 0x7f8f2c1e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f2ad9d640 1 -- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c1e6650 msgr2=0x7f8f2c1d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f2ad9d640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c1e6650 0x7f8f2c1d30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f2ad9d640 1 -- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f2c06c130 msgr2=0x7f8f2c1e6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f2ad9d640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f2c06c130 0x7f8f2c1e6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f2ad9d640 1 -- 172.21.15.47:0/3545194318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c06b090 con 0x7f8f2c1e3610 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f2b59e640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c1e6650 0x7f8f2c1d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.726+0000 7f8f2ad9d640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f2c1e3610 0x7f8f2c1e39f0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f8f240119a0 tx=0x7f8f24011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.727+0000 7f8f237fe640 1 -- 172.21.15.47:0/3545194318 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2400faf0 con 0x7f8f2c1e3610 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.727+0000 7f8f237fe640 1 -- 172.21.15.47:0/3545194318 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2400fc50 con 0x7f8f2c1e3610 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.727+0000 7f8f31215640 1 -- 172.21.15.47:0/3545194318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2c06b370 con 0x7f8f2c1e3610 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.727+0000 7f8f237fe640 1 -- 172.21.15.47:0/3545194318 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24014440 con 0x7f8f2c1e3610 2024-03-23T09:51:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.727+0000 7f8f31215640 1 -- 172.21.15.47:0/3545194318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2c06b930 con 0x7f8f2c1e3610 2024-03-23T09:51:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.728+0000 7f8f31215640 1 -- 172.21.15.47:0/3545194318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f2c0672f0 con 0x7f8f2c1e3610 2024-03-23T09:51:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.728+0000 7f8f237fe640 1 -- 172.21.15.47:0/3545194318 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8f240145a0 con 0x7f8f2c1e3610 2024-03-23T09:51:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.728+0000 7f8f237fe640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8f0c06ac00 0x7f8f0c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.728+0000 7f8f237fe640 1 -- 172.21.15.47:0/3545194318 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8f24089d40 con 0x7f8f2c1e3610 2024-03-23T09:51:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.729+0000 7f8f2a59c640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8f0c06ac00 0x7f8f0c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.729+0000 7f8f2a59c640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8f0c06ac00 0x7f8f0c06d0c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0c4640 tx=0x7f8f14008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.733+0000 7f8f237fe640 1 -- 172.21.15.47:0/3545194318 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8f24054ec0 con 0x7f8f2c1e3610 2024-03-23T09:51:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.948+0000 7f8f31215640 1 -- 172.21.15.47:0/3545194318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8f2c0b3060 con 0x7f8f2c1e3610 2024-03-23T09:51:24.948 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:51:24.949+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:51:24.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.962+0000 7f8f237fe640 1 -- 172.21.15.47:0/3545194318 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8f240584e0 con 0x7f8f2c1e3610 2024-03-23T09:51:24.962 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:51:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.964+0000 7f8f31215640 1 -- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8f0c06ac00 msgr2=0x7f8f0c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.964+0000 7f8f31215640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8f0c06ac00 0x7f8f0c06d0c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0c4640 tx=0x7f8f14008040 comp rx=0 tx=0).stop 2024-03-23T09:51:24.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.964+0000 7f8f31215640 1 -- 172.21.15.47:0/3545194318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f2c1e3610 msgr2=0x7f8f2c1e39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:24.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.964+0000 7f8f31215640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f2c1e3610 0x7f8f2c1e39f0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f8f240119a0 tx=0x7f8f24011e60 comp rx=0 tx=0).stop 2024-03-23T09:51:24.965 INFO:tasks.ceph.mds.a.smithi047.stderr: -1013> 2024-03-23T09:50:54.187+0000 7f757833a640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-03-23T09:51:24.965 INFO:tasks.ceph.mds.a.smithi047.stderr: -1010> 2024-03-23T09:50:54.187+0000 7f757833a640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-03-23T09:51:24.965 INFO:tasks.ceph.mds.a.smithi047.stderr: -994> 2024-03-23T09:50:54.187+0000 7f757833a640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-03-23T09:51:24.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.964+0000 7f8f31215640 1 -- 172.21.15.47:0/3545194318 shutdown_connections 2024-03-23T09:51:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.964+0000 7f8f31215640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8f0c06ac00 0x7f8f0c06d0c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.964+0000 7f8f31215640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f2c1e6650 0x7f8f2c1d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.966 INFO:tasks.ceph.mds.a.smithi047.stderr: -991> 2024-03-23T09:50:54.187+0000 7f757833a640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-03-23T09:51:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.964+0000 7f8f31215640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f2c06c130 0x7f8f2c1e6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.964+0000 7f8f31215640 1 --2- 172.21.15.47:0/3545194318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f2c1e3610 0x7f8f2c1e39f0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.964+0000 7f8f31215640 1 -- 172.21.15.47:0/3545194318 >> 172.21.15.47:0/3545194318 conn(0x7f8f2c0b2200 msgr2=0x7f8f2c0ad670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.964+0000 7f8f31215640 1 -- 172.21.15.47:0/3545194318 shutdown_connections 2024-03-23T09:51:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:24.964+0000 7f8f31215640 1 -- 172.21.15.47:0/3545194318 wait complete. 2024-03-23T09:51:24.967 INFO:tasks.ceph.mds.a.smithi047.stderr: -865> 2024-03-23T09:50:54.605+0000 7f757bb41640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:51:24.976 DEBUG:teuthology.orchestra.run.smithi047:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-03-23T09:51:24.990 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-03-23T09:51:25.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.014+0000 7f3154c42280 1 -- start start 2024-03-23T09:51:25.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.014+0000 7f3154c42280 1 Processor -- start 2024-03-23T09:51:25.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f3154c42280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561787f1cc00 0x561787fb6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f3154c42280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561787f1d800 0x561787fb6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f3154c42280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561787f1dc00 0x561787fb7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f3154c42280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561787faea00 con 0x561787f1d800 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f3154c42280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561787fae780 con 0x561787f1dc00 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f3154c42280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x561787fae640 con 0x561787f1cc00 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f315236b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561787f1d800 0x561787fb6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f315236b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561787f1d800 0x561787fb6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37386/0 (socket says 172.21.15.47:37386) 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f315236b640 1 -- 172.21.15.47:0/4233453631 learned_addr learned my addr 172.21.15.47:0/4233453631 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f3152b6c640 1 --2- 172.21.15.47:0/4233453631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561787f1cc00 0x561787fb6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f315336d640 1 --2- 172.21.15.47:0/4233453631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561787f1dc00 0x561787fb7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f315236b640 1 -- 172.21.15.47:0/4233453631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561787f1cc00 msgr2=0x561787fb6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f315236b640 1 --2- 172.21.15.47:0/4233453631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561787f1cc00 0x561787fb6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f315236b640 1 -- 172.21.15.47:0/4233453631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561787f1dc00 msgr2=0x561787fb7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:25.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f315236b640 1 --2- 172.21.15.47:0/4233453631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561787f1dc00 0x561787fb7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f315236b640 1 -- 172.21.15.47:0/4233453631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561787f429c0 con 0x561787f1d800 2024-03-23T09:51:25.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f315336d640 1 --2- 172.21.15.47:0/4233453631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561787f1dc00 0x561787fb7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:25.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f315236b640 1 --2- 172.21.15.47:0/4233453631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561787f1d800 0x561787fb6b00 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x561788cf2120 tx=0x561788de8060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0168182eaca3aae server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:25.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f3150367640 1 -- 172.21.15.47:0/4233453631 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561787fe1080 con 0x561787f1d800 2024-03-23T09:51:25.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f3150367640 1 -- 172.21.15.47:0/4233453631 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561788c9c300 con 0x561787f1d800 2024-03-23T09:51:25.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f3150367640 1 -- 172.21.15.47:0/4233453631 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561787fe1600 con 0x561787f1d800 2024-03-23T09:51:25.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f3154c42280 1 -- 172.21.15.47:0/4233453631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561787f42d00 con 0x561787f1d800 2024-03-23T09:51:25.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f3154c42280 1 -- 172.21.15.47:0/4233453631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x561787f429c0 con 0x561787f1d800 2024-03-23T09:51:25.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.015+0000 7f3154c42280 4 waiting for MDS map... 2024-03-23T09:51:25.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.016+0000 7f3150367640 1 -- 172.21.15.47:0/4233453631 <== mon.0 v2:172.21.15.47:3300/0 4 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x561787f48380 con 0x561787f1d800 2024-03-23T09:51:25.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.016+0000 7f3150367640 1 -- 172.21.15.47:0/4233453631 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 72) v1 ==== 2573+0+0 (secure 0 0 0) 0x561787f6ef00 con 0x561787f1d800 2024-03-23T09:51:25.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.016+0000 7f3154c42280 4 Got MDS map 72 2024-03-23T09:51:25.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.016+0000 7f3154c42280 10 main: main 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.016+0000 7f3154c42280 4 main: connecting to RADOS... 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.016+0000 7f3154c42280 1 Processor -- start 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.016+0000 7f3154c42280 1 -- start start 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f3154c42280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561788cffc00 0x561787fb7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f3154c42280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561788e44000 0x561787fb7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f3154c42280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561788e44400 0x561787fb8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f3154c42280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561787f4de00 con 0x561788cffc00 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f3154c42280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561787fae780 con 0x561788e44400 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f3154c42280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x561787faea00 con 0x561788e44000 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f315336d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561788cffc00 0x561787fb7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f315336d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561788cffc00 0x561787fb7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37388/0 (socket says 172.21.15.47:37388) 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f315336d640 1 -- 172.21.15.47:0/3254314921 learned_addr learned my addr 172.21.15.47:0/3254314921 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f3152b6c640 1 --2- 172.21.15.47:0/3254314921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561788e44000 0x561787fb7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f315236b640 1 --2- 172.21.15.47:0/3254314921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561788e44400 0x561787fb8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f315336d640 1 -- 172.21.15.47:0/3254314921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561788e44000 msgr2=0x561787fb7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f315336d640 1 --2- 172.21.15.47:0/3254314921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561788e44000 0x561787fb7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f315336d640 1 -- 172.21.15.47:0/3254314921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561788e44400 msgr2=0x561787fb8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f315336d640 1 --2- 172.21.15.47:0/3254314921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561788e44400 0x561787fb8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f315336d640 1 -- 172.21.15.47:0/3254314921 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561787f42b60 con 0x561788cffc00 2024-03-23T09:51:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.017+0000 7f315336d640 1 --2- 172.21.15.47:0/3254314921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561788cffc00 0x561787fb7600 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x561788cf3b30 tx=0x561788d18c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab351857a8d08c99 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.018+0000 7f314e363640 1 -- 172.21.15.47:0/3254314921 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561787fe1080 con 0x561788cffc00 2024-03-23T09:51:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.018+0000 7f314e363640 1 -- 172.21.15.47:0/3254314921 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561788c9ca80 con 0x561788cffc00 2024-03-23T09:51:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.018+0000 7f314e363640 1 -- 172.21.15.47:0/3254314921 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561787fe1600 con 0x561788cffc00 2024-03-23T09:51:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.018+0000 7f3154c42280 1 -- 172.21.15.47:0/3254314921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561788cffc00 msgr2=0x561787fb7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.018+0000 7f3154c42280 1 --2- 172.21.15.47:0/3254314921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561788cffc00 0x561787fb7600 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x561788cf3b30 tx=0x561788d18c00 comp rx=0 tx=0).stop 2024-03-23T09:51:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.018+0000 7f3154c42280 1 -- 172.21.15.47:0/3254314921 shutdown_connections 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.018+0000 7f3154c42280 1 --2- 172.21.15.47:0/3254314921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561788e44400 0x561787fb8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.018+0000 7f3154c42280 1 --2- 172.21.15.47:0/3254314921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561788e44000 0x561787fb7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.018+0000 7f3154c42280 1 --2- 172.21.15.47:0/3254314921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561788cffc00 0x561787fb7600 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.018+0000 7f3154c42280 1 -- 172.21.15.47:0/3254314921 >> 172.21.15.47:0/3254314921 conn(0x561788cff800 msgr2=0x561788007000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.018+0000 7f3154c42280 1 -- 172.21.15.47:0/3254314921 shutdown_connections 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.018+0000 7f3154c42280 1 -- 172.21.15.47:0/3254314921 wait complete. 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.018+0000 7f3154c42280 1 Processor -- start 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f3154c42280 1 -- start start 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f3154c42280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561788cff800 0x561787fb9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f3154c42280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561788e45000 0x561787fb8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f3154c42280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561788e44c00 0x561787fb8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f3154c42280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561787fae500 con 0x561788e44c00 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f3154c42280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561787fae3c0 con 0x561788e45000 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f3154c42280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x561787fae280 con 0x561788cff800 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f315236b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561788e44c00 0x561787fb8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f315236b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561788e44c00 0x561787fb8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37394/0 (socket says 172.21.15.47:37394) 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f315236b640 1 -- 172.21.15.47:0/251409892 learned_addr learned my addr 172.21.15.47:0/251409892 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f3152b6c640 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561788e45000 0x561787fb8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f315336d640 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561788cff800 0x561787fb9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f315236b640 1 -- 172.21.15.47:0/251409892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561788cff800 msgr2=0x561787fb9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f315236b640 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561788cff800 0x561787fb9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f315236b640 1 -- 172.21.15.47:0/251409892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561788e45000 msgr2=0x561787fb8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.019+0000 7f315236b640 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561788e45000 0x561787fb8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.020+0000 7f315236b640 1 -- 172.21.15.47:0/251409892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561787f42820 con 0x561788e44c00 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.020+0000 7f315236b640 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561788e44c00 0x561787fb8680 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x561788cf2e40 tx=0x561788de8120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.020+0000 7f314cb60640 1 -- 172.21.15.47:0/251409892 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561787fe11e0 con 0x561788e44c00 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.020+0000 7f314cb60640 1 -- 172.21.15.47:0/251409892 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561788c9d680 con 0x561788e44c00 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.020+0000 7f3154c42280 1 -- 172.21.15.47:0/251409892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561787f424e0 con 0x561788e44c00 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.020+0000 7f314cb60640 1 -- 172.21.15.47:0/251409892 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561787fe0f20 con 0x561788e44c00 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.020+0000 7f3154c42280 1 -- 172.21.15.47:0/251409892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561787f421a0 con 0x561788e44c00 2024-03-23T09:51:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.020+0000 7f3154c42280 4 main: resolving pool 14 2024-03-23T09:51:25.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.021+0000 7f314cb60640 1 -- 172.21.15.47:0/251409892 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x561787f8d080 con 0x561788e44c00 2024-03-23T09:51:25.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.021+0000 7f314cb60640 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x561788e4f000 0x561787fb9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.021+0000 7f314cb60640 1 -- 172.21.15.47:0/251409892 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x561787f48540 con 0x561788e44c00 2024-03-23T09:51:25.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.021+0000 7f3154c42280 4 main: creating IoCtx.. 2024-03-23T09:51:25.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.021+0000 7f3154c42280 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x561788e44800 0x561787fb8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.021+0000 7f3154c42280 1 -- 172.21.15.47:0/251409892 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103) v8 -- 0x561788e44400 con 0x561788e44800 2024-03-23T09:51:25.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.021+0000 7f315336d640 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x561788e4f000 0x561787fb9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.022+0000 7f3152b6c640 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x561788e44800 0x561787fb8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.022+0000 7f315336d640 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x561788e4f000 0x561787fb9700 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x561788cf3b30 tx=0x561788cb3140 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:25.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.022+0000 7f3152b6c640 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x561788e44800 0x561787fb8100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:25.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.024+0000 7f3152b6c640 1 -- 172.21.15.47:0/251409892 <== osd.2 v2:172.21.15.47:6800/37678 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x561788dd8b40 con 0x561788e44800 2024-03-23T09:51:25.030 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:51:25.030 INFO:teuthology.orchestra.run.smithi047.stdout: "0": { 2024-03-23T09:51:25.030 INFO:teuthology.orchestra.run.smithi047.stdout: "data": { 2024-03-23T09:51:25.030 INFO:teuthology.orchestra.run.smithi047.stdout: "version": 5, 2024-03-23T09:51:25.030 INFO:teuthology.orchestra.run.smithi047.stdout: "inotable": { 2024-03-23T09:51:25.030 INFO:teuthology.orchestra.run.smithi047.stdout: "projected_free": [ 2024-03-23T09:51:25.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:25.030 INFO:teuthology.orchestra.run.smithi047.stdout: "start": 1099511627779, 2024-03-23T09:51:25.030 INFO:teuthology.orchestra.run.smithi047.stdout: "len": 1099511627773 2024-03-23T09:51:25.030 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:25.031 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-23T09:51:25.031 INFO:teuthology.orchestra.run.smithi047.stdout: "free": [ 2024-03-23T09:51:25.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:25.031 INFO:teuthology.orchestra.run.smithi047.stdout: "start": 1099511627779, 2024-03-23T09:51:25.031 INFO:teuthology.orchestra.run.smithi047.stdout: "len": 1099511627773 2024-03-23T09:51:25.031 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:25.031 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-23T09:51:25.031 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:25.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:25.031 INFO:teuthology.orchestra.run.smithi047.stdout: "result": 0 2024-03-23T09:51:25.031 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:25.031 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:51:25.031 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:25.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.024+0000 7f3154c42280 1 -- 172.21.15.47:0/251409892 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x561788e44800 msgr2=0x561787fb8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.024+0000 7f3154c42280 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x561788e44800 0x561787fb8100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.024+0000 7f3154c42280 1 -- 172.21.15.47:0/251409892 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x561788e4f000 msgr2=0x561787fb9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.024+0000 7f3154c42280 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x561788e4f000 0x561787fb9700 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x561788cf3b30 tx=0x561788cb3140 comp rx=0 tx=0).stop 2024-03-23T09:51:25.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.024+0000 7f3154c42280 1 -- 172.21.15.47:0/251409892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561788e44c00 msgr2=0x561787fb8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.024+0000 7f3154c42280 1 --2- 172.21.15.47:0/251409892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561788e44c00 0x561787fb8680 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x561788cf2e40 tx=0x561788de8120 comp rx=0 tx=0).stop 2024-03-23T09:51:25.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.024+0000 7f3152b6c640 1 -- 172.21.15.47:0/251409892 reap_dead start 2024-03-23T09:51:25.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.024+0000 7f3154c42280 1 -- 172.21.15.47:0/251409892 shutdown_connections 2024-03-23T09:51:25.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.024+0000 7f3154c42280 1 -- 172.21.15.47:0/251409892 >> 172.21.15.47:0/251409892 conn(0x561788cffc00 msgr2=0x561788007000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:25.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.024+0000 7f3154c42280 1 -- 172.21.15.47:0/251409892 shutdown_connections 2024-03-23T09:51:25.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.024+0000 7f3154c42280 1 -- 172.21.15.47:0/251409892 wait complete. 2024-03-23T09:51:25.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.025+0000 7f3154c42280 1 -- 172.21.15.47:0/4233453631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561787f1d800 msgr2=0x561787fb6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.025+0000 7f3154c42280 1 --2- 172.21.15.47:0/4233453631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561787f1d800 0x561787fb6b00 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x561788cf2120 tx=0x561788de8060 comp rx=0 tx=0).stop 2024-03-23T09:51:25.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.025+0000 7f3154c42280 1 -- 172.21.15.47:0/4233453631 shutdown_connections 2024-03-23T09:51:25.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.025+0000 7f3154c42280 1 --2- 172.21.15.47:0/4233453631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561787f1dc00 0x561787fb7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.025+0000 7f3154c42280 1 --2- 172.21.15.47:0/4233453631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561787f1d800 0x561787fb6b00 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.025+0000 7f3154c42280 1 --2- 172.21.15.47:0/4233453631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561787f1cc00 0x561787fb6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.025+0000 7f3154c42280 1 -- 172.21.15.47:0/4233453631 >> 172.21.15.47:0/4233453631 conn(0x561787f1d000 msgr2=0x561788006800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:25.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.025+0000 7f3154c42280 1 -- 172.21.15.47:0/4233453631 shutdown_connections 2024-03-23T09:51:25.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.025+0000 7f3154c42280 1 -- 172.21.15.47:0/4233453631 wait complete. 2024-03-23T09:51:25.034 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.058595, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-03-23T09:51:25.035 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:51:25.035 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:51:25.120 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:51:25.163 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:51:25.164 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:51:25.164 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:51:25.179 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:51:25.179 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:51:25.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:25.179 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:51:25.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:51:25.234 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-23T09:51:25.235 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:51:25.235 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:51:25.252 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:51:25.252 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:51:25.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:25.252 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-23T09:51:25.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:51:25.309 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-23T09:51:25.310 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:51:25.310 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:51:25.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:51:25.325 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:51:25.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:25.325 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-23T09:51:25.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:51:25.379 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-23T09:51:25.380 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:51:25.380 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:51:25.397 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:51:25.397 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:51:25.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:25.398 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-23T09:51:25.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:51:25.455 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-23T09:51:25.455 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:51:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.556+0000 7f1edff99640 1 Processor -- start 2024-03-23T09:51:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.556+0000 7f1edff99640 1 -- start start 2024-03-23T09:51:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.557+0000 7f1edff99640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1ed815cb40 0x7f1ed8160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.557+0000 7f1edff99640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed80d0ee0 con 0x7f1ed80b8300 2024-03-23T09:51:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.557+0000 7f1edff99640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed81616f0 con 0x7f1ed8160cb0 2024-03-23T09:51:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.557+0000 7f1edff99640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed80b5a10 con 0x7f1ed815cb40 2024-03-23T09:51:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.557+0000 7f1edd534640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1ed80b8300 0x7f1ed80cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50904/0 (socket says 172.21.15.47:50904) 2024-03-23T09:51:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.557+0000 7f1edd534640 1 -- 172.21.15.47:0/3007018352 learned_addr learned my addr 172.21.15.47:0/3007018352 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.557+0000 7f1eddd35640 1 --2- 172.21.15.47:0/3007018352 >> v2:172.21.15.175:3300/0 conn(0x7f1ed815cb40 0x7f1ed8160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777616401 0 0) 0x7f1ed80d0ee0 con 0x7f1ed80b8300 2024-03-23T09:51:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ec0003520 con 0x7f1ed80b8300 2024-03-23T09:51:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 14758952 0 0) 0x7f1ec0003520 con 0x7f1ed80b8300 2024-03-23T09:51:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 >> v2:172.21.15.175:3300/0 conn(0x7f1ed815cb40 msgr2=0x7f1ed8160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 --2- 172.21.15.47:0/3007018352 >> v2:172.21.15.175:3300/0 conn(0x7f1ed815cb40 0x7f1ed8160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 >> v1:172.21.15.175:6789/0 conn(0x7f1ed8160cb0 legacy=0x7f1ed80d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed8161130 con 0x7f1ed80b8300 2024-03-23T09:51:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f1ecc001670 con 0x7f1ed80b8300 2024-03-23T09:51:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 >> v1:172.21.15.47:6789/0 conn(0x7f1ed80b8300 legacy=0x7f1ed80cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 --2- 172.21.15.47:0/3007018352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec0003970 0x7f1ec0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 --2- 172.21.15.47:0/3007018352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ec00047c0 0x7f1ec0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 --2- 172.21.15.47:0/3007018352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ec00051d0 0x7f1ec000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80b5a10 con 0x7f1ec0003970 2024-03-23T09:51:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec000c100 con 0x7f1ec00051d0 2024-03-23T09:51:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec000c2c0 con 0x7f1ec00047c0 2024-03-23T09:51:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1eddd35640 1 --2- 172.21.15.47:0/3007018352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec0003970 0x7f1ec0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1ede536640 1 --2- 172.21.15.47:0/3007018352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ec00051d0 0x7f1ec000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1edd534640 1 --2- 172.21.15.47:0/3007018352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ec00047c0 0x7f1ec0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1eddd35640 1 -- 172.21.15.47:0/3007018352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ec00047c0 msgr2=0x7f1ec0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1eddd35640 1 --2- 172.21.15.47:0/3007018352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ec00047c0 0x7f1ec0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1eddd35640 1 -- 172.21.15.47:0/3007018352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ec00051d0 msgr2=0x7f1ec000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1eddd35640 1 --2- 172.21.15.47:0/3007018352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ec00051d0 0x7f1ec000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1eddd35640 1 -- 172.21.15.47:0/3007018352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec0003520 con 0x7f1ec0003970 2024-03-23T09:51:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.558+0000 7f1ede536640 1 -- 172.21.15.47:0/3007018352 reap_dead start 2024-03-23T09:51:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.559+0000 7f1eddd35640 1 --2- 172.21.15.47:0/3007018352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec0003970 0x7f1ec0003d90 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009fc0 tx=0x7f1ec00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b161e36c607abf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.559+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec802cc20 con 0x7f1ec0003970 2024-03-23T09:51:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.559+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec800f040 con 0x7f1ec0003970 2024-03-23T09:51:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.559+0000 7f1edcd33640 1 -- 172.21.15.47:0/3007018352 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8034440 con 0x7f1ec0003970 2024-03-23T09:51:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.560+0000 7f1edff99640 1 -- 172.21.15.47:0/3007018352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec0003970 msgr2=0x7f1ec0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.560+0000 7f1edff99640 1 --2- 172.21.15.47:0/3007018352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec0003970 0x7f1ec0003d90 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009fc0 tx=0x7f1ec00057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.560+0000 7f1edff99640 1 -- 172.21.15.47:0/3007018352 shutdown_connections 2024-03-23T09:51:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.560+0000 7f1edff99640 1 --2- 172.21.15.47:0/3007018352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec0003970 0x7f1ec0003d90 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.560+0000 7f1edff99640 1 -- 172.21.15.47:0/3007018352 >> 172.21.15.47:0/3007018352 conn(0x7f1ed806b160 msgr2=0x7f1ed806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.560+0000 7f1edff99640 1 -- 172.21.15.47:0/3007018352 shutdown_connections 2024-03-23T09:51:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.560+0000 7f1edff99640 1 -- 172.21.15.47:0/3007018352 wait complete. 2024-03-23T09:51:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.561+0000 7f1edff99640 1 Processor -- start 2024-03-23T09:51:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.561+0000 7f1edff99640 1 -- start start 2024-03-23T09:51:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edff99640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ec0003970 0x7f1ed8159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edff99640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ed815f0e0 0x7f1ed80cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edff99640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ed80c8460 0x7f1ed80c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edff99640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80c8e20 con 0x7f1ed815f0e0 2024-03-23T09:51:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edff99640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed815a200 con 0x7f1ed80c8460 2024-03-23T09:51:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edff99640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed81595f0 con 0x7f1ec0003970 2024-03-23T09:51:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edd534640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ed815f0e0 0x7f1ed80cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edd534640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ed815f0e0 0x7f1ed80cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37416/0 (socket says 172.21.15.47:37416) 2024-03-23T09:51:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edd534640 1 -- 172.21.15.47:0/3097792131 learned_addr learned my addr 172.21.15.47:0/3097792131 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1eddd35640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ec0003970 0x7f1ed8159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1ede536640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ed80c8460 0x7f1ed80c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edd534640 1 -- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ec0003970 msgr2=0x7f1ed8159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edd534640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ec0003970 0x7f1ed8159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edd534640 1 -- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ed80c8460 msgr2=0x7f1ed80c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edd534640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ed80c8460 0x7f1ed80c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edd534640 1 -- 172.21.15.47:0/3097792131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed81598d0 con 0x7f1ed815f0e0 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1ede536640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ed80c8460 0x7f1ed80c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edd534640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ed815f0e0 0x7f1ed80cd3d0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00e780 tx=0x7f1ecc00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1ed67fc640 1 -- 172.21.15.47:0/3097792131 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00ee90 con 0x7f1ed815f0e0 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1ed67fc640 1 -- 172.21.15.47:0/3097792131 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ecc00c870 con 0x7f1ed815f0e0 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edff99640 1 -- 172.21.15.47:0/3097792131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed8159bb0 con 0x7f1ed815f0e0 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1ed67fc640 1 -- 172.21.15.47:0/3097792131 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00cc20 con 0x7f1ed815f0e0 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.562+0000 7f1edff99640 1 -- 172.21.15.47:0/3097792131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed815f540 con 0x7f1ed815f0e0 2024-03-23T09:51:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.563+0000 7f1edff99640 1 -- 172.21.15.47:0/3097792131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed8072360 con 0x7f1ed815f0e0 2024-03-23T09:51:25.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.564+0000 7f1ed67fc640 1 -- 172.21.15.47:0/3097792131 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1ecc013420 con 0x7f1ed815f0e0 2024-03-23T09:51:25.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.564+0000 7f1ed67fc640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1ec406ac30 0x7f1ec406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.564+0000 7f1ed67fc640 1 -- 172.21.15.47:0/3097792131 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f1ecc0891b0 con 0x7f1ed815f0e0 2024-03-23T09:51:25.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.564+0000 7f1eddd35640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1ec406ac30 0x7f1ec406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.565+0000 7f1eddd35640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1ec406ac30 0x7f1ec406d0f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8036000 tx=0x7f1ec802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:25.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.569+0000 7f1ed67fc640 1 -- 172.21.15.47:0/3097792131 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1ecc057c50 con 0x7f1ed815f0e0 2024-03-23T09:51:25.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.784+0000 7f1edff99640 1 -- 172.21.15.47:0/3097792131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ed80ce2a0 con 0x7f1ed815f0e0 2024-03-23T09:51:25.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.785+0000 7f1ed67fc640 1 -- 172.21.15.47:0/3097792131 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+3976 (secure 0 0 0) 0x7f1ecc0577e0 con 0x7f1ed815f0e0 2024-03-23T09:51:25.785 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:25.785 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":72,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/249416109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":249416109},{"type":"v1","addr":"172.21.15.175:6837","nonce":249416109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3603846321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3603846321},{"type":"v1","addr":"172.21.15.175:6835","nonce":3603846321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1921094794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1921094794},{"type":"v1","addr":"172.21.15.47:6834","nonce":1921094794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":72,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:49:43.247119+0000","modified":"2024-03-23T09:51:24.949801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-23T09:51:25.785 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 72 2024-03-23T09:51:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.786+0000 7f1edff99640 1 -- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1ec406ac30 msgr2=0x7f1ec406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.787+0000 7f1edff99640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1ec406ac30 0x7f1ec406d0f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8036000 tx=0x7f1ec802e000 comp rx=0 tx=0).stop 2024-03-23T09:51:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.787+0000 7f1edff99640 1 -- 172.21.15.47:0/3097792131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ed815f0e0 msgr2=0x7f1ed80cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.787+0000 7f1edff99640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ed815f0e0 0x7f1ed80cd3d0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00e780 tx=0x7f1ecc00ec40 comp rx=0 tx=0).stop 2024-03-23T09:51:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.787+0000 7f1edff99640 1 -- 172.21.15.47:0/3097792131 shutdown_connections 2024-03-23T09:51:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.787+0000 7f1edff99640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1ec406ac30 0x7f1ec406d0f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.787+0000 7f1edff99640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ed80c8460 0x7f1ed80c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.787+0000 7f1edff99640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ed815f0e0 0x7f1ed80cd3d0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.787+0000 7f1edff99640 1 --2- 172.21.15.47:0/3097792131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ec0003970 0x7f1ed8159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.787+0000 7f1edff99640 1 -- 172.21.15.47:0/3097792131 >> 172.21.15.47:0/3097792131 conn(0x7f1ed806b160 msgr2=0x7f1ed80d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.787+0000 7f1edff99640 1 -- 172.21.15.47:0/3097792131 shutdown_connections 2024-03-23T09:51:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.787+0000 7f1edff99640 1 -- 172.21.15.47:0/3097792131 wait complete. 2024-03-23T09:51:25.798 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:51:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.898+0000 7f3d218e0640 1 Processor -- start 2024-03-23T09:51:25.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.898+0000 7f3d218e0640 1 -- start start 2024-03-23T09:51:25.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.899+0000 7f3d218e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c06a3f0 0x7f3d1c0cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.899+0000 7f3d218e0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3d1c15bc70 0x7f3d1c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.899+0000 7f3d218e0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1c0cd180 con 0x7f3d1c15a9b0 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.899+0000 7f3d218e0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1c15b7e0 con 0x7f3d1c15bc70 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.899+0000 7f3d218e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c0ccea0 con 0x7f3d1c06a3f0 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.899+0000 7f3d1affd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3d1c15a9b0 0x7f3d1c069ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50920/0 (socket says 172.21.15.47:50920) 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.899+0000 7f3d1affd640 1 -- 172.21.15.47:0/1048373947 learned_addr learned my addr 172.21.15.47:0/1048373947 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092353343 0 0) 0x7f3d14001110 con 0x7f3d1c15a9b0 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d08003520 con 0x7f3d1c15a9b0 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d1a7fc640 1 --2- 172.21.15.47:0/1048373947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c06a3f0 0x7f3d1c0cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d1b7fe640 1 --2- 172.21.15.47:0/1048373947 >> v2:172.21.15.175:3300/0 conn(0x7f3d1c15bc70 0x7f3d1c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2321169667 0 0) 0x7f3d08003520 con 0x7f3d1c15a9b0 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c06a3f0 msgr2=0x7f3d1c0cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 --2- 172.21.15.47:0/1048373947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c06a3f0 0x7f3d1c0cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 >> v2:172.21.15.175:3300/0 conn(0x7f3d1c15bc70 msgr2=0x7f3d1c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 --2- 172.21.15.47:0/1048373947 >> v2:172.21.15.175:3300/0 conn(0x7f3d1c15bc70 0x7f3d1c0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1c15b190 con 0x7f3d1c15a9b0 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f3d14001800 con 0x7f3d1c15a9b0 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 >> v1:172.21.15.47:6789/0 conn(0x7f3d1c15a9b0 legacy=0x7f3d1c069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 --2- 172.21.15.47:0/1048373947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d08003970 0x7f3d08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 --2- 172.21.15.47:0/1048373947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d080047c0 0x7f3d08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 --2- 172.21.15.47:0/1048373947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c06a3f0 0x7f3d08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c15b7e0 con 0x7f3d080047c0 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c0ccea0 con 0x7f3d08003970 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0800b980 con 0x7f3d1c06a3f0 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d1a7fc640 1 --2- 172.21.15.47:0/1048373947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d080047c0 0x7f3d08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d1affd640 1 --2- 172.21.15.47:0/1048373947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d08003970 0x7f3d08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d1b7fe640 1 --2- 172.21.15.47:0/1048373947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c06a3f0 0x7f3d08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d1a7fc640 1 -- 172.21.15.47:0/1048373947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c06a3f0 msgr2=0x7f3d08005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d1a7fc640 1 --2- 172.21.15.47:0/1048373947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c06a3f0 0x7f3d08005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d1a7fc640 1 -- 172.21.15.47:0/1048373947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d08003970 msgr2=0x7f3d08003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d1a7fc640 1 --2- 172.21.15.47:0/1048373947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d08003970 0x7f3d08003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.900+0000 7f3d1a7fc640 1 -- 172.21.15.47:0/1048373947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d08003520 con 0x7f3d080047c0 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.901+0000 7f3d1a7fc640 1 --2- 172.21.15.47:0/1048373947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d080047c0 0x7f3d08004c40 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009ac0 tx=0x7f3d080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248484ca5e426925 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.901+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0402c6b0 con 0x7f3d080047c0 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.901+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0402c810 con 0x7f3d080047c0 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.901+0000 7f3d19ffb640 1 -- 172.21.15.47:0/1048373947 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0402cba0 con 0x7f3d080047c0 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.902+0000 7f3d218e0640 1 -- 172.21.15.47:0/1048373947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d080047c0 msgr2=0x7f3d08004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.902+0000 7f3d218e0640 1 --2- 172.21.15.47:0/1048373947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d080047c0 0x7f3d08004c40 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009ac0 tx=0x7f3d080057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.902+0000 7f3d1b7fe640 1 -- 172.21.15.47:0/1048373947 reap_dead start 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.902+0000 7f3d218e0640 1 -- 172.21.15.47:0/1048373947 shutdown_connections 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.902+0000 7f3d218e0640 1 -- 172.21.15.47:0/1048373947 >> 172.21.15.47:0/1048373947 conn(0x7f3d1c071400 msgr2=0x7f3d1c071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.902+0000 7f3d218e0640 1 -- 172.21.15.47:0/1048373947 shutdown_connections 2024-03-23T09:51:25.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.902+0000 7f3d218e0640 1 -- 172.21.15.47:0/1048373947 wait complete. 2024-03-23T09:51:25.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.903+0000 7f3d218e0640 1 Processor -- start 2024-03-23T09:51:25.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.903+0000 7f3d218e0640 1 -- start start 2024-03-23T09:51:25.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d218e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d080047c0 0x7f3d1c14ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d218e0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c06ee10 0x7f3d1c1475f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d218e0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d1c06f290 0x7f3d1c147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d218e0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c158b10 con 0x7f3d1c06f290 2024-03-23T09:51:25.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d218e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c14b220 con 0x7f3d080047c0 2024-03-23T09:51:25.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d218e0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c0c9310 con 0x7f3d1c06ee10 2024-03-23T09:51:25.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d1b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d1c06f290 0x7f3d1c147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d1b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d1c06f290 0x7f3d1c147b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37438/0 (socket says 172.21.15.47:37438) 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d1b7fe640 1 -- 172.21.15.47:0/3205340642 learned_addr learned my addr 172.21.15.47:0/3205340642 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d1b7fe640 1 -- 172.21.15.47:0/3205340642 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c06ee10 msgr2=0x7f3d1c1475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d1b7fe640 1 --2- 172.21.15.47:0/3205340642 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c06ee10 0x7f3d1c1475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d1b7fe640 1 -- 172.21.15.47:0/3205340642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d080047c0 msgr2=0x7f3d1c14ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d1b7fe640 1 --2- 172.21.15.47:0/3205340642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d080047c0 0x7f3d1c14ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d1b7fe640 1 -- 172.21.15.47:0/3205340642 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1c0c95f0 con 0x7f3d1c06f290 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d1b7fe640 1 --2- 172.21.15.47:0/3205340642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d1c06f290 0x7f3d1c147b30 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c070d10 tx=0x7f3d0c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d13fff640 1 -- 172.21.15.47:0/3205340642 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00e070 con 0x7f3d1c06f290 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d13fff640 1 -- 172.21.15.47:0/3205340642 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c011e40 con 0x7f3d1c06f290 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d218e0640 1 -- 172.21.15.47:0/3205340642 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c1587a0 con 0x7f3d1c06f290 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.904+0000 7f3d13fff640 1 -- 172.21.15.47:0/3205340642 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c003990 con 0x7f3d1c06f290 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.905+0000 7f3d218e0640 1 -- 172.21.15.47:0/3205340642 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c155a70 con 0x7f3d1c06f290 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.906+0000 7f3d218e0640 1 -- 172.21.15.47:0/3205340642 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1c158930 con 0x7f3d1c06f290 2024-03-23T09:51:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.906+0000 7f3d13fff640 1 -- 172.21.15.47:0/3205340642 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3d0c003af0 con 0x7f3d1c06f290 2024-03-23T09:51:25.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.906+0000 7f3d13fff640 1 --2- 172.21.15.47:0/3205340642 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3cfc06ac00 0x7f3cfc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:25.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.906+0000 7f3d13fff640 1 -- 172.21.15.47:0/3205340642 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f3d0c089d30 con 0x7f3d1c06f290 2024-03-23T09:51:25.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.907+0000 7f3d1affd640 1 --2- 172.21.15.47:0/3205340642 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3cfc06ac00 0x7f3cfc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:25.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.907+0000 7f3d1affd640 1 --2- 172.21.15.47:0/3205340642 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3cfc06ac00 0x7f3cfc06d0c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3d14001050 tx=0x7f3d14001150 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:25.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:25.911+0000 7f3d13fff640 1 -- 172.21.15.47:0/3205340642 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3d0c054e60 con 0x7f3d1c06f290 2024-03-23T09:51:26.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.090+0000 7f3d218e0640 1 -- 172.21.15.47:0/3205340642 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d1c158c50 con 0x7f3d1c06f290 2024-03-23T09:51:26.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.091+0000 7f3d13fff640 1 -- 172.21.15.47:0/3205340642 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f3d1c158c50 con 0x7f3d1c06f290 2024-03-23T09:51:26.092 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:26.092 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":103,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:51:24.949501+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:49:41.158056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-23T09:49:42.210388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/1748845936":"2024-03-24T09:51:24.949481+0000","172.21.15.47:6833/1748845936":"2024-03-24T09:51:24.949481+0000","172.21.15.47:6835/2388298413":"2024-03-24T09:50:48.589519+0000","172.21.15.47:6833/2388298413":"2024-03-24T09:50:48.589519+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:51:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.093+0000 7f3d218e0640 1 -- 172.21.15.47:0/3205340642 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3cfc06ac00 msgr2=0x7f3cfc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.093+0000 7f3d218e0640 1 --2- 172.21.15.47:0/3205340642 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3cfc06ac00 0x7f3cfc06d0c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3d14001050 tx=0x7f3d14001150 comp rx=0 tx=0).stop 2024-03-23T09:51:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.093+0000 7f3d218e0640 1 -- 172.21.15.47:0/3205340642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d1c06f290 msgr2=0x7f3d1c147b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.093+0000 7f3d218e0640 1 --2- 172.21.15.47:0/3205340642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d1c06f290 0x7f3d1c147b30 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c070d10 tx=0x7f3d0c0115a0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.093+0000 7f3d218e0640 1 -- 172.21.15.47:0/3205340642 shutdown_connections 2024-03-23T09:51:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.093+0000 7f3d218e0640 1 --2- 172.21.15.47:0/3205340642 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3cfc06ac00 0x7f3cfc06d0c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.093+0000 7f3d218e0640 1 --2- 172.21.15.47:0/3205340642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d1c06f290 0x7f3d1c147b30 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.093+0000 7f3d218e0640 1 --2- 172.21.15.47:0/3205340642 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c06ee10 0x7f3d1c1475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.093+0000 7f3d218e0640 1 --2- 172.21.15.47:0/3205340642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d080047c0 0x7f3d1c14ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.093+0000 7f3d218e0640 1 -- 172.21.15.47:0/3205340642 >> 172.21.15.47:0/3205340642 conn(0x7f3d1c071400 msgr2=0x7f3d1c0afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.094+0000 7f3d218e0640 1 -- 172.21.15.47:0/3205340642 shutdown_connections 2024-03-23T09:51:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.094+0000 7f3d218e0640 1 -- 172.21.15.47:0/3205340642 wait complete. 2024-03-23T09:51:26.105 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:49:41.158056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:49:42.210388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:51:26.105 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:51:26.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.208+0000 7f7f69042640 1 Processor -- start 2024-03-23T09:51:26.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.209+0000 7f7f69042640 1 -- start start 2024-03-23T09:51:26.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.209+0000 7f7f69042640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7f6415c990 0x7f7f64160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.209+0000 7f7f69042640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f64161630 con 0x7f7f640b8350 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.209+0000 7f7f69042640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f640d1030 con 0x7f7f640cde10 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.209+0000 7f7f69042640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f640cfe10 con 0x7f7f6415c990 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.209+0000 7f7f62d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7f6415c990 0x7f7f64160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.209+0000 7f7f6259c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7f640b8350 0x7f7f640cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50928/0 (socket says 172.21.15.47:50928) 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.209+0000 7f7f6259c640 1 -- 172.21.15.47:0/336135436 learned_addr learned my addr 172.21.15.47:0/336135436 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.210+0000 7f7f62d9d640 1 -- 172.21.15.47:0/336135436 >> v1:172.21.15.175:6789/0 conn(0x7f7f640cde10 legacy=0x7f7f640d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.210+0000 7f7f62d9d640 1 -- 172.21.15.47:0/336135436 >> v1:172.21.15.47:6789/0 conn(0x7f7f640b8350 legacy=0x7f7f640cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.210+0000 7f7f62d9d640 1 -- 172.21.15.47:0/336135436 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f640b5b60 con 0x7f7f6415c990 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.210+0000 7f7f61d9b640 1 -- 172.21.15.47:0/336135436 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622987046 0 0) 0x7f7f64161630 con 0x7f7f640b8350 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.210+0000 7f7f62d9d640 1 --2- 172.21.15.47:0/336135436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f6415c990 0x7f7f64160690 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0097a0 tx=0x7f7f5c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecec578bed21b7a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.211+0000 7f7f61d9b640 1 -- 172.21.15.47:0/336135436 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c031830 con 0x7f7f6415c990 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.211+0000 7f7f61d9b640 1 -- 172.21.15.47:0/336135436 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c031990 con 0x7f7f6415c990 2024-03-23T09:51:26.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.211+0000 7f7f61d9b640 1 -- 172.21.15.47:0/336135436 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c031d20 con 0x7f7f6415c990 2024-03-23T09:51:26.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.211+0000 7f7f69042640 1 -- 172.21.15.47:0/336135436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f6415c990 msgr2=0x7f7f64160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.211+0000 7f7f69042640 1 --2- 172.21.15.47:0/336135436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f6415c990 0x7f7f64160690 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0097a0 tx=0x7f7f5c02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.211+0000 7f7f69042640 1 -- 172.21.15.47:0/336135436 shutdown_connections 2024-03-23T09:51:26.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.211+0000 7f7f69042640 1 --2- 172.21.15.47:0/336135436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f6415c990 0x7f7f64160690 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.211+0000 7f7f69042640 1 -- 172.21.15.47:0/336135436 >> 172.21.15.47:0/336135436 conn(0x7f7f640659d0 msgr2=0x7f7f64065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:26.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.211+0000 7f7f69042640 1 -- 172.21.15.47:0/336135436 shutdown_connections 2024-03-23T09:51:26.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.211+0000 7f7f69042640 1 -- 172.21.15.47:0/336135436 wait complete. 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.213+0000 7f7f69042640 1 Processor -- start 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.213+0000 7f7f69042640 1 -- start start 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.213+0000 7f7f69042640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f640b8350 0x7f7f641543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.213+0000 7f7f69042640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f640cde10 0x7f7f641f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.213+0000 7f7f69042640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f6415c990 0x7f7f6415f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.213+0000 7f7f69042640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f64154930 con 0x7f7f640cde10 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.213+0000 7f7f69042640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640c6e40 con 0x7f7f6415c990 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.213+0000 7f7f69042640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640c7120 con 0x7f7f640b8350 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.213+0000 7f7f6259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f640cde10 0x7f7f641f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.213+0000 7f7f6259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f640cde10 0x7f7f641f1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37450/0 (socket says 172.21.15.47:37450) 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.214+0000 7f7f6259c640 1 -- 172.21.15.47:0/1330366275 learned_addr learned my addr 172.21.15.47:0/1330366275 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.214+0000 7f7f62d9d640 1 --2- 172.21.15.47:0/1330366275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f640b8350 0x7f7f641543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.214+0000 7f7f6259c640 1 -- 172.21.15.47:0/1330366275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f640b8350 msgr2=0x7f7f641543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.214+0000 7f7f6259c640 1 --2- 172.21.15.47:0/1330366275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f640b8350 0x7f7f641543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.214+0000 7f7f6259c640 1 -- 172.21.15.47:0/1330366275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f6415c990 msgr2=0x7f7f6415f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.214+0000 7f7f6259c640 1 --2- 172.21.15.47:0/1330366275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f6415c990 0x7f7f6415f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.214+0000 7f7f6259c640 1 -- 172.21.15.47:0/1330366275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c009400 con 0x7f7f640cde10 2024-03-23T09:51:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.214+0000 7f7f6259c640 1 --2- 172.21.15.47:0/1330366275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f640cde10 0x7f7f641f1a40 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00c4a0 tx=0x7f7f4c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.214+0000 7f7f5b7fe640 1 -- 172.21.15.47:0/1330366275 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0074e0 con 0x7f7f640cde10 2024-03-23T09:51:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.214+0000 7f7f5b7fe640 1 -- 172.21.15.47:0/1330366275 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c007640 con 0x7f7f640cde10 2024-03-23T09:51:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.214+0000 7f7f69042640 1 -- 172.21.15.47:0/1330366275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f640c7400 con 0x7f7f640cde10 2024-03-23T09:51:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.214+0000 7f7f5b7fe640 1 -- 172.21.15.47:0/1330366275 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c004a00 con 0x7f7f640cde10 2024-03-23T09:51:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.214+0000 7f7f69042640 1 -- 172.21.15.47:0/1330366275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f640c79c0 con 0x7f7f640cde10 2024-03-23T09:51:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.215+0000 7f7f69042640 1 -- 172.21.15.47:0/1330366275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f64072360 con 0x7f7f640cde10 2024-03-23T09:51:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.216+0000 7f7f5b7fe640 1 -- 172.21.15.47:0/1330366275 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7f4c004b60 con 0x7f7f640cde10 2024-03-23T09:51:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.216+0000 7f7f5b7fe640 1 --2- 172.21.15.47:0/1330366275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7f4406ac30 0x7f7f4406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.216+0000 7f7f5b7fe640 1 -- 172.21.15.47:0/1330366275 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f7f4c088570 con 0x7f7f640cde10 2024-03-23T09:51:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.216+0000 7f7f62d9d640 1 --2- 172.21.15.47:0/1330366275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7f4406ac30 0x7f7f4406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.217+0000 7f7f62d9d640 1 --2- 172.21.15.47:0/1330366275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7f4406ac30 0x7f7f4406d0f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c036000 tx=0x7f7f5c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.220+0000 7f7f5b7fe640 1 -- 172.21.15.47:0/1330366275 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7f4c053620 con 0x7f7f640cde10 2024-03-23T09:51:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.397+0000 7f7f69042640 1 -- 172.21.15.47:0/1330366275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7f640c7be0 con 0x7f7f640cde10 2024-03-23T09:51:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.398+0000 7f7f5b7fe640 1 -- 172.21.15.47:0/1330366275 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7f4c056c40 con 0x7f7f640cde10 2024-03-23T09:51:26.398 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:51:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.400+0000 7f7f69042640 1 -- 172.21.15.47:0/1330366275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7f4406ac30 msgr2=0x7f7f4406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.400+0000 7f7f69042640 1 --2- 172.21.15.47:0/1330366275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7f4406ac30 0x7f7f4406d0f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c036000 tx=0x7f7f5c03c040 comp rx=0 tx=0).stop 2024-03-23T09:51:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.400+0000 7f7f69042640 1 -- 172.21.15.47:0/1330366275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f640cde10 msgr2=0x7f7f641f1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.400+0000 7f7f69042640 1 --2- 172.21.15.47:0/1330366275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f640cde10 0x7f7f641f1a40 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00c4a0 tx=0x7f7f4c00c960 comp rx=0 tx=0).stop 2024-03-23T09:51:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.400+0000 7f7f69042640 1 -- 172.21.15.47:0/1330366275 shutdown_connections 2024-03-23T09:51:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.400+0000 7f7f69042640 1 --2- 172.21.15.47:0/1330366275 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7f4406ac30 0x7f7f4406d0f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.400+0000 7f7f69042640 1 --2- 172.21.15.47:0/1330366275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f6415c990 0x7f7f6415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.400+0000 7f7f69042640 1 --2- 172.21.15.47:0/1330366275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f640cde10 0x7f7f641f1a40 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.400+0000 7f7f69042640 1 --2- 172.21.15.47:0/1330366275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f640b8350 0x7f7f641543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.400+0000 7f7f69042640 1 -- 172.21.15.47:0/1330366275 >> 172.21.15.47:0/1330366275 conn(0x7f7f640659d0 msgr2=0x7f7f64156fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.400+0000 7f7f69042640 1 -- 172.21.15.47:0/1330366275 shutdown_connections 2024-03-23T09:51:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.400+0000 7f7f69042640 1 -- 172.21.15.47:0/1330366275 wait complete. 2024-03-23T09:51:26.411 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:51:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.514+0000 7ff68a981640 1 Processor -- start 2024-03-23T09:51:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.514+0000 7ff68a981640 1 -- start start 2024-03-23T09:51:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.515+0000 7ff68a981640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6840ae270 0x7ff6840d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.515+0000 7ff68a981640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6840b0d20 0x7ff6840cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.515+0000 7ff68a981640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6840af870 con 0x7ff6840ce210 2024-03-23T09:51:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.515+0000 7ff68a981640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6840ce680 con 0x7ff6840ae270 2024-03-23T09:51:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.515+0000 7ff68a981640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6840afac0 con 0x7ff6840b0d20 2024-03-23T09:51:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.515+0000 7ff683fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6840ae270 0x7ff6840d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.515+0000 7ff683fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6840ae270 0x7ff6840d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37462/0 (socket says 172.21.15.47:37462) 2024-03-23T09:51:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.515+0000 7ff683fff640 1 -- 172.21.15.47:0/3845432644 learned_addr learned my addr 172.21.15.47:0/3845432644 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.515+0000 7ff6837fe640 1 --2- 172.21.15.47:0/3845432644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6840b0d20 0x7ff6840cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.516+0000 7ff683fff640 1 -- 172.21.15.47:0/3845432644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6840b0d20 msgr2=0x7ff6840cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.516+0000 7ff683fff640 1 --2- 172.21.15.47:0/3845432644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6840b0d20 0x7ff6840cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.516+0000 7ff683fff640 1 -- 172.21.15.47:0/3845432644 >> v1:172.21.15.175:6789/0 conn(0x7ff6840ce210 legacy=0x7ff68415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.516+0000 7ff683fff640 1 -- 172.21.15.47:0/3845432644 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6840afda0 con 0x7ff6840ae270 2024-03-23T09:51:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.516+0000 7ff683fff640 1 --2- 172.21.15.47:0/3845432644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6840ae270 0x7ff6840d0230 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7ff674009870 tx=0x7ff67402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba1bdb7a0d711b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.517+0000 7ff682ffd640 1 -- 172.21.15.47:0/3845432644 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674030840 con 0x7ff6840ae270 2024-03-23T09:51:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.517+0000 7ff682ffd640 1 -- 172.21.15.47:0/3845432644 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6740309a0 con 0x7ff6840ae270 2024-03-23T09:51:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.517+0000 7ff682ffd640 1 -- 172.21.15.47:0/3845432644 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674030d30 con 0x7ff6840ae270 2024-03-23T09:51:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.517+0000 7ff68a981640 1 -- 172.21.15.47:0/3845432644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6840ae270 msgr2=0x7ff6840d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.517+0000 7ff68a981640 1 --2- 172.21.15.47:0/3845432644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6840ae270 0x7ff6840d0230 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7ff674009870 tx=0x7ff67402b980 comp rx=0 tx=0).stop 2024-03-23T09:51:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.517+0000 7ff68a981640 1 -- 172.21.15.47:0/3845432644 shutdown_connections 2024-03-23T09:51:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.517+0000 7ff68a981640 1 --2- 172.21.15.47:0/3845432644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6840b0d20 0x7ff6840cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.517+0000 7ff68a981640 1 --2- 172.21.15.47:0/3845432644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6840ae270 0x7ff6840d0230 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.517+0000 7ff68a981640 1 -- 172.21.15.47:0/3845432644 >> 172.21.15.47:0/3845432644 conn(0x7ff684098a00 msgr2=0x7ff6840cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.517+0000 7ff68a981640 1 -- 172.21.15.47:0/3845432644 shutdown_connections 2024-03-23T09:51:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.517+0000 7ff68a981640 1 -- 172.21.15.47:0/3845432644 wait complete. 2024-03-23T09:51:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.518+0000 7ff68a981640 1 Processor -- start 2024-03-23T09:51:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.518+0000 7ff68a981640 1 -- start start 2024-03-23T09:51:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.519+0000 7ff68a981640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6840ae270 0x7ff6841580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.519+0000 7ff68a981640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6840b0d20 0x7ff6841539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.519+0000 7ff68a981640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6840ce210 0x7ff684153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.519+0000 7ff68a981640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68415a3c0 con 0x7ff6840b0d20 2024-03-23T09:51:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.519+0000 7ff68a981640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68415a6a0 con 0x7ff6840ae270 2024-03-23T09:51:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.519+0000 7ff68a981640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68415a980 con 0x7ff6840ce210 2024-03-23T09:51:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.519+0000 7ff6837fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6840b0d20 0x7ff6841539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.519+0000 7ff6837fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6840b0d20 0x7ff6841539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37474/0 (socket says 172.21.15.47:37474) 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.519+0000 7ff6837fe640 1 -- 172.21.15.47:0/292735756 learned_addr learned my addr 172.21.15.47:0/292735756 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.519+0000 7ff683fff640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6840ae270 0x7ff6841580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.519+0000 7ff688f1e640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6840ce210 0x7ff684153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.520+0000 7ff6837fe640 1 -- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6840ce210 msgr2=0x7ff684153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.520+0000 7ff6837fe640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6840ce210 0x7ff684153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.520+0000 7ff6837fe640 1 -- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6840ae270 msgr2=0x7ff6841580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.520+0000 7ff6837fe640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6840ae270 0x7ff6841580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.520+0000 7ff6837fe640 1 -- 172.21.15.47:0/292735756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740094d0 con 0x7ff6840b0d20 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.520+0000 7ff6837fe640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6840b0d20 0x7ff6841539c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7ff67800d800 tx=0x7ff67800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.520+0000 7ff680ff9640 1 -- 172.21.15.47:0/292735756 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800b6c0 con 0x7ff6840b0d20 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.520+0000 7ff680ff9640 1 -- 172.21.15.47:0/292735756 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67800b820 con 0x7ff6840b0d20 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.520+0000 7ff68a981640 1 -- 172.21.15.47:0/292735756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68415ac60 con 0x7ff6840b0d20 2024-03-23T09:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.520+0000 7ff680ff9640 1 -- 172.21.15.47:0/292735756 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800bbb0 con 0x7ff6840b0d20 2024-03-23T09:51:26.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.520+0000 7ff68a981640 1 -- 172.21.15.47:0/292735756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff684158600 con 0x7ff6840b0d20 2024-03-23T09:51:26.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.521+0000 7ff68a981640 1 -- 172.21.15.47:0/292735756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6840b0320 con 0x7ff6840b0d20 2024-03-23T09:51:26.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.521+0000 7ff680ff9640 1 -- 172.21.15.47:0/292735756 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff67800bd10 con 0x7ff6840b0d20 2024-03-23T09:51:26.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.522+0000 7ff680ff9640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff67006ac00 0x7ff67006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.522+0000 7ff680ff9640 1 -- 172.21.15.47:0/292735756 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x7ff6780898f0 con 0x7ff6840b0d20 2024-03-23T09:51:26.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.522+0000 7ff683fff640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff67006ac00 0x7ff67006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.522+0000 7ff683fff640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff67006ac00 0x7ff67006d0c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff674036000 tx=0x7ff674008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:26.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.526+0000 7ff680ff9640 1 -- 172.21.15.47:0/292735756 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff678054a20 con 0x7ff6840b0d20 2024-03-23T09:51:26.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.703+0000 7ff68a981640 1 -- 172.21.15.47:0/292735756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff684158790 con 0x7ff6840b0d20 2024-03-23T09:51:26.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.704+0000 7ff680ff9640 1 -- 172.21.15.47:0/292735756 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff678058040 con 0x7ff6840b0d20 2024-03-23T09:51:26.704 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:51:26.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.705+0000 7ff68a981640 1 -- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff67006ac00 msgr2=0x7ff67006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.705+0000 7ff68a981640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff67006ac00 0x7ff67006d0c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff674036000 tx=0x7ff674008030 comp rx=0 tx=0).stop 2024-03-23T09:51:26.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.705+0000 7ff68a981640 1 -- 172.21.15.47:0/292735756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6840b0d20 msgr2=0x7ff6841539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.705+0000 7ff68a981640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6840b0d20 0x7ff6841539c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7ff67800d800 tx=0x7ff67800dcc0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.705+0000 7ff68a981640 1 -- 172.21.15.47:0/292735756 shutdown_connections 2024-03-23T09:51:26.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.705+0000 7ff68a981640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff67006ac00 0x7ff67006d0c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.705+0000 7ff68a981640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6840ce210 0x7ff684153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.705+0000 7ff68a981640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6840b0d20 0x7ff6841539c0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.705+0000 7ff68a981640 1 --2- 172.21.15.47:0/292735756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6840ae270 0x7ff6841580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.705+0000 7ff68a981640 1 -- 172.21.15.47:0/292735756 >> 172.21.15.47:0/292735756 conn(0x7ff684098a00 msgr2=0x7ff6840aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:26.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.706+0000 7ff68a981640 1 -- 172.21.15.47:0/292735756 shutdown_connections 2024-03-23T09:51:26.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.706+0000 7ff68a981640 1 -- 172.21.15.47:0/292735756 wait complete. 2024-03-23T09:51:26.716 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:51:26.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.820+0000 7f1185096640 1 Processor -- start 2024-03-23T09:51:26.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.820+0000 7f1185096640 1 -- start start 2024-03-23T09:51:26.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.821+0000 7f1185096640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f118015c990 0x7f1180160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.821+0000 7f1185096640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f11800cde10 0x7f11800d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.821+0000 7f1185096640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1180161630 con 0x7f11800b8350 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.821+0000 7f1185096640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11800d1030 con 0x7f118015c990 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.821+0000 7f1185096640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11800cfe10 con 0x7f11800cde10 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.821+0000 7f117ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f118015c990 0x7f1180160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.821+0000 7f117e59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f11800b8350 0x7f11800cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50944/0 (socket says 172.21.15.47:50944) 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.821+0000 7f117ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f118015c990 0x7f1180160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37486/0 (socket says 172.21.15.47:37486) 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.821+0000 7f117e59c640 1 -- 172.21.15.47:0/3270003475 learned_addr learned my addr 172.21.15.47:0/3270003475 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.821+0000 7f117ed9d640 1 -- 172.21.15.47:0/3270003475 >> v2:172.21.15.175:3300/0 conn(0x7f11800cde10 msgr2=0x7f11800d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.822+0000 7f117f59e640 1 --2- 172.21.15.47:0/3270003475 >> v2:172.21.15.175:3300/0 conn(0x7f11800cde10 0x7f11800d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.822+0000 7f117dd9b640 1 -- 172.21.15.47:0/3270003475 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925966456 0 0) 0x7f1180161630 con 0x7f11800b8350 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.822+0000 7f117ed9d640 1 --2- 172.21.15.47:0/3270003475 >> v2:172.21.15.175:3300/0 conn(0x7f11800cde10 0x7f11800d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.822+0000 7f117ed9d640 1 -- 172.21.15.47:0/3270003475 >> v1:172.21.15.47:6789/0 conn(0x7f11800b8350 legacy=0x7f11800cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.822+0000 7f117ed9d640 1 -- 172.21.15.47:0/3270003475 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11800b5b60 con 0x7f118015c990 2024-03-23T09:51:26.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.822+0000 7f117f59e640 1 --2- 172.21.15.47:0/3270003475 >> v2:172.21.15.175:3300/0 conn(0x7f11800cde10 0x7f11800d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:26.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.822+0000 7f117ed9d640 1 --2- 172.21.15.47:0/3270003475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118015c990 0x7f1180160690 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f11800b87e0 tx=0x7f117802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625a910ac667f9aa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:26.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.822+0000 7f117dd9b640 1 -- 172.21.15.47:0/3270003475 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117802e070 con 0x7f118015c990 2024-03-23T09:51:26.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.822+0000 7f117dd9b640 1 -- 172.21.15.47:0/3270003475 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11780315a0 con 0x7f118015c990 2024-03-23T09:51:26.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.822+0000 7f117dd9b640 1 -- 172.21.15.47:0/3270003475 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178031930 con 0x7f118015c990 2024-03-23T09:51:26.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.823+0000 7f1185096640 1 -- 172.21.15.47:0/3270003475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118015c990 msgr2=0x7f1180160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.823+0000 7f1185096640 1 --2- 172.21.15.47:0/3270003475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118015c990 0x7f1180160690 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f11800b87e0 tx=0x7f117802b840 comp rx=0 tx=0).stop 2024-03-23T09:51:26.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.823+0000 7f1185096640 1 -- 172.21.15.47:0/3270003475 shutdown_connections 2024-03-23T09:51:26.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.823+0000 7f1185096640 1 --2- 172.21.15.47:0/3270003475 >> v2:172.21.15.175:3300/0 conn(0x7f11800cde10 0x7f11800d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.823+0000 7f1185096640 1 --2- 172.21.15.47:0/3270003475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118015c990 0x7f1180160690 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.823+0000 7f1185096640 1 -- 172.21.15.47:0/3270003475 >> 172.21.15.47:0/3270003475 conn(0x7f11800659d0 msgr2=0x7f1180065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:26.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.823+0000 7f1185096640 1 -- 172.21.15.47:0/3270003475 shutdown_connections 2024-03-23T09:51:26.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.823+0000 7f1185096640 1 -- 172.21.15.47:0/3270003475 wait complete. 2024-03-23T09:51:26.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.824+0000 7f1185096640 1 Processor -- start 2024-03-23T09:51:26.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.824+0000 7f1185096640 1 -- start start 2024-03-23T09:51:26.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f1185096640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11800b8350 0x7f1180153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f1185096640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11800cde10 0x7f11800d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f1185096640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f118015c990 0x7f11800d2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f1185096640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118015eed0 con 0x7f11800b8350 2024-03-23T09:51:26.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f1185096640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118015e2c0 con 0x7f11800cde10 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f1185096640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118015e5a0 con 0x7f118015c990 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f117ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11800b8350 0x7f1180153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f117ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11800b8350 0x7f1180153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37492/0 (socket says 172.21.15.47:37492) 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f117ed9d640 1 -- 172.21.15.47:0/2135875943 learned_addr learned my addr 172.21.15.47:0/2135875943 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f117ed9d640 1 -- 172.21.15.47:0/2135875943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f118015c990 msgr2=0x7f11800d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f117ed9d640 1 --2- 172.21.15.47:0/2135875943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f118015c990 0x7f11800d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f117ed9d640 1 -- 172.21.15.47:0/2135875943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11800cde10 msgr2=0x7f11800d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f117ed9d640 1 --2- 172.21.15.47:0/2135875943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11800cde10 0x7f11800d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.825+0000 7f117ed9d640 1 -- 172.21.15.47:0/2135875943 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1178009400 con 0x7f11800b8350 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.826+0000 7f117ed9d640 1 --2- 172.21.15.47:0/2135875943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11800b8350 0x7f1180153f70 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f1178000c00 tx=0x7f11780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.826+0000 7f11777fe640 1 -- 172.21.15.47:0/2135875943 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178032900 con 0x7f11800b8350 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.826+0000 7f11777fe640 1 -- 172.21.15.47:0/2135875943 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1178032a60 con 0x7f11800b8350 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.826+0000 7f11777fe640 1 -- 172.21.15.47:0/2135875943 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178032df0 con 0x7f11800b8350 2024-03-23T09:51:26.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.826+0000 7f1185096640 1 -- 172.21.15.47:0/2135875943 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118015e880 con 0x7f11800b8350 2024-03-23T09:51:26.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.826+0000 7f1185096640 1 -- 172.21.15.47:0/2135875943 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11800c8140 con 0x7f11800b8350 2024-03-23T09:51:26.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.827+0000 7f1185096640 1 -- 172.21.15.47:0/2135875943 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1180072360 con 0x7f11800b8350 2024-03-23T09:51:26.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.827+0000 7f11777fe640 1 -- 172.21.15.47:0/2135875943 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f117804d3f0 con 0x7f11800b8350 2024-03-23T09:51:26.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.828+0000 7f11777fe640 1 --2- 172.21.15.47:0/2135875943 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f116006ac00 0x7f116006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:26.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.828+0000 7f11777fe640 1 -- 172.21.15.47:0/2135875943 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f11780ab0c0 con 0x7f11800b8350 2024-03-23T09:51:26.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.828+0000 7f117e59c640 1 --2- 172.21.15.47:0/2135875943 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f116006ac00 0x7f116006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:26.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.828+0000 7f117e59c640 1 --2- 172.21.15.47:0/2135875943 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f116006ac00 0x7f116006d0c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1168007fd0 tx=0x7f1168007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:26.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:26.832+0000 7f11777fe640 1 -- 172.21.15.47:0/2135875943 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1178076280 con 0x7f11800b8350 2024-03-23T09:51:27.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.013+0000 7f1185096640 1 -- 172.21.15.47:0/2135875943 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1180066be0 con 0x7f11800b8350 2024-03-23T09:51:27.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.014+0000 7f11777fe640 1 -- 172.21.15.47:0/2135875943 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11780798a0 con 0x7f11800b8350 2024-03-23T09:51:27.014 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:51:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.016+0000 7f1185096640 1 -- 172.21.15.47:0/2135875943 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f116006ac00 msgr2=0x7f116006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.016+0000 7f1185096640 1 --2- 172.21.15.47:0/2135875943 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f116006ac00 0x7f116006d0c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1168007fd0 tx=0x7f1168007920 comp rx=0 tx=0).stop 2024-03-23T09:51:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.016+0000 7f1185096640 1 -- 172.21.15.47:0/2135875943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11800b8350 msgr2=0x7f1180153f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.016+0000 7f1185096640 1 --2- 172.21.15.47:0/2135875943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11800b8350 0x7f1180153f70 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f1178000c00 tx=0x7f11780046c0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.016+0000 7f1185096640 1 -- 172.21.15.47:0/2135875943 shutdown_connections 2024-03-23T09:51:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.016+0000 7f1185096640 1 --2- 172.21.15.47:0/2135875943 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f116006ac00 0x7f116006d0c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.016+0000 7f1185096640 1 --2- 172.21.15.47:0/2135875943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f118015c990 0x7f11800d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.016+0000 7f1185096640 1 --2- 172.21.15.47:0/2135875943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11800cde10 0x7f11800d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.016+0000 7f1185096640 1 --2- 172.21.15.47:0/2135875943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11800b8350 0x7f1180153f70 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.016+0000 7f1185096640 1 -- 172.21.15.47:0/2135875943 >> 172.21.15.47:0/2135875943 conn(0x7f11800659d0 msgr2=0x7f11800b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.016+0000 7f1185096640 1 -- 172.21.15.47:0/2135875943 shutdown_connections 2024-03-23T09:51:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.016+0000 7f1185096640 1 -- 172.21.15.47:0/2135875943 wait complete. 2024-03-23T09:51:27.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:51:27.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.129+0000 7fef29db9640 1 Processor -- start 2024-03-23T09:51:27.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.129+0000 7fef29db9640 1 -- start start 2024-03-23T09:51:27.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.130+0000 7fef29db9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef2415ca90 0x7fef24160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.130+0000 7fef29db9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef240cde10 0x7fef240d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.130+0000 7fef29db9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef240d11e0 con 0x7fef240b8350 2024-03-23T09:51:27.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.130+0000 7fef29db9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef24160f60 con 0x7fef240cde10 2024-03-23T09:51:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.130+0000 7fef29db9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef24161240 con 0x7fef2415ca90 2024-03-23T09:51:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.130+0000 7fef23fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef240cde10 0x7fef240d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.130+0000 7fef23fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef240cde10 0x7fef240d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37502/0 (socket says 172.21.15.47:37502) 2024-03-23T09:51:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.130+0000 7fef23fff640 1 -- 172.21.15.47:0/3947527167 learned_addr learned my addr 172.21.15.47:0/3947527167 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.130+0000 7fef237fe640 1 --2- 172.21.15.47:0/3947527167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef2415ca90 0x7fef24160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.130+0000 7fef23fff640 1 -- 172.21.15.47:0/3947527167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef2415ca90 msgr2=0x7fef24160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.130+0000 7fef23fff640 1 --2- 172.21.15.47:0/3947527167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef2415ca90 0x7fef24160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.130+0000 7fef23fff640 1 -- 172.21.15.47:0/3947527167 >> v1:172.21.15.175:6789/0 conn(0x7fef240b8350 legacy=0x7fef240cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.130+0000 7fef23fff640 1 -- 172.21.15.47:0/3947527167 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef24161520 con 0x7fef240cde10 2024-03-23T09:51:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.131+0000 7fef23fff640 1 --2- 172.21.15.47:0/3947527167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef240cde10 0x7fef240d1670 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009e50 tx=0x7fef1c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a274dffa44686018 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.131+0000 7fef227fc640 1 -- 172.21.15.47:0/3947527167 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c02ca70 con 0x7fef240cde10 2024-03-23T09:51:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.131+0000 7fef227fc640 1 -- 172.21.15.47:0/3947527167 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c02cbd0 con 0x7fef240cde10 2024-03-23T09:51:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.131+0000 7fef227fc640 1 -- 172.21.15.47:0/3947527167 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0343f0 con 0x7fef240cde10 2024-03-23T09:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.132+0000 7fef29db9640 1 -- 172.21.15.47:0/3947527167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef240cde10 msgr2=0x7fef240d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.132+0000 7fef29db9640 1 --2- 172.21.15.47:0/3947527167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef240cde10 0x7fef240d1670 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009e50 tx=0x7fef1c02bed0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.132+0000 7fef29db9640 1 -- 172.21.15.47:0/3947527167 shutdown_connections 2024-03-23T09:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.132+0000 7fef29db9640 1 --2- 172.21.15.47:0/3947527167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef240cde10 0x7fef240d1670 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.132+0000 7fef29db9640 1 --2- 172.21.15.47:0/3947527167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef2415ca90 0x7fef24160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.132+0000 7fef29db9640 1 -- 172.21.15.47:0/3947527167 >> 172.21.15.47:0/3947527167 conn(0x7fef240659d0 msgr2=0x7fef24065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.132+0000 7fef29db9640 1 -- 172.21.15.47:0/3947527167 shutdown_connections 2024-03-23T09:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.132+0000 7fef29db9640 1 -- 172.21.15.47:0/3947527167 wait complete. 2024-03-23T09:51:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.133+0000 7fef29db9640 1 Processor -- start 2024-03-23T09:51:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.133+0000 7fef29db9640 1 -- start start 2024-03-23T09:51:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef29db9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef240b8350 0x7fef240cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef29db9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef240cde10 0x7fef240b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef29db9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2415ca90 0x7fef24156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef29db9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef24159470 con 0x7fef2415ca90 2024-03-23T09:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef29db9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef241585e0 con 0x7fef240b8350 2024-03-23T09:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef29db9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef24157510 con 0x7fef240cde10 2024-03-23T09:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef23fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2415ca90 0x7fef24156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef23fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2415ca90 0x7fef24156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37516/0 (socket says 172.21.15.47:37516) 2024-03-23T09:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef23fff640 1 -- 172.21.15.47:0/3527684525 learned_addr learned my addr 172.21.15.47:0/3527684525 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef237fe640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef240b8350 0x7fef240cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef22ffd640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef240cde10 0x7fef240b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef23fff640 1 -- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef240cde10 msgr2=0x7fef240b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef23fff640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef240cde10 0x7fef240b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef23fff640 1 -- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef240b8350 msgr2=0x7fef240cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef23fff640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef240b8350 0x7fef240cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.134+0000 7fef23fff640 1 -- 172.21.15.47:0/3527684525 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c009b30 con 0x7fef2415ca90 2024-03-23T09:51:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.135+0000 7fef23fff640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2415ca90 0x7fef24156bc0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009e50 tx=0x7fef1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.135+0000 7fef0bfff640 1 -- 172.21.15.47:0/3527684525 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c009510 con 0x7fef2415ca90 2024-03-23T09:51:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.135+0000 7fef0bfff640 1 -- 172.21.15.47:0/3527684525 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c03e450 con 0x7fef2415ca90 2024-03-23T09:51:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.135+0000 7fef29db9640 1 -- 172.21.15.47:0/3527684525 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef241587f0 con 0x7fef2415ca90 2024-03-23T09:51:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.135+0000 7fef0bfff640 1 -- 172.21.15.47:0/3527684525 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c03e800 con 0x7fef2415ca90 2024-03-23T09:51:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.135+0000 7fef29db9640 1 -- 172.21.15.47:0/3527684525 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef24158db0 con 0x7fef2415ca90 2024-03-23T09:51:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.136+0000 7fef29db9640 1 -- 172.21.15.47:0/3527684525 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef24072360 con 0x7fef2415ca90 2024-03-23T09:51:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.136+0000 7fef0bfff640 1 -- 172.21.15.47:0/3527684525 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fef1c03e960 con 0x7fef2415ca90 2024-03-23T09:51:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.137+0000 7fef0bfff640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef1006ac00 0x7fef1006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.137+0000 7fef0bfff640 1 -- 172.21.15.47:0/3527684525 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fef1c0ac220 con 0x7fef2415ca90 2024-03-23T09:51:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.137+0000 7fef237fe640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef1006ac00 0x7fef1006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.137+0000 7fef237fe640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef1006ac00 0x7fef1006d0c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fef14006fd0 tx=0x7fef14008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.141+0000 7fef0bfff640 1 -- 172.21.15.47:0/3527684525 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fef1c0772c0 con 0x7fef2415ca90 2024-03-23T09:51:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.360+0000 7fef29db9640 1 -- 172.21.15.47:0/3527684525 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef241595b0 con 0x7fef2415ca90 2024-03-23T09:51:27.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.360+0000 7fef0bfff640 1 -- 172.21.15.47:0/3527684525 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4733 (secure 0 0 0) 0x7fef241595b0 con 0x7fef2415ca90 2024-03-23T09:51:27.361 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:27.361 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/249416109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":249416109},{"type":"v1","addr":"172.21.15.175:6837","nonce":249416109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3603846321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3603846321},{"type":"v1","addr":"172.21.15.175:6835","nonce":3603846321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1921094794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1921094794},{"type":"v1","addr":"172.21.15.47:6834","nonce":1921094794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":9078,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3030917369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":3030917369},{"type":"v1","addr":"172.21.15.47:6835","nonce":3030917369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:49:43.247119+0000","modified":"2024-03-23T09:51:24.949801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-23T09:51:27.361 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 73 2024-03-23T09:51:27.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.362+0000 7fef29db9640 1 -- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef1006ac00 msgr2=0x7fef1006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.362+0000 7fef29db9640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef1006ac00 0x7fef1006d0c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fef14006fd0 tx=0x7fef14008040 comp rx=0 tx=0).stop 2024-03-23T09:51:27.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.362+0000 7fef29db9640 1 -- 172.21.15.47:0/3527684525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2415ca90 msgr2=0x7fef24156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.362+0000 7fef29db9640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2415ca90 0x7fef24156bc0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009e50 tx=0x7fef1c004a40 comp rx=0 tx=0).stop 2024-03-23T09:51:27.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.362+0000 7fef29db9640 1 -- 172.21.15.47:0/3527684525 shutdown_connections 2024-03-23T09:51:27.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.362+0000 7fef29db9640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fef1006ac00 0x7fef1006d0c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.362+0000 7fef29db9640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2415ca90 0x7fef24156bc0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.362+0000 7fef29db9640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef240cde10 0x7fef240b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.363+0000 7fef29db9640 1 --2- 172.21.15.47:0/3527684525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef240b8350 0x7fef240cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.363+0000 7fef29db9640 1 -- 172.21.15.47:0/3527684525 >> 172.21.15.47:0/3527684525 conn(0x7fef240659d0 msgr2=0x7fef24157870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:27.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.363+0000 7fef29db9640 1 -- 172.21.15.47:0/3527684525 shutdown_connections 2024-03-23T09:51:27.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.363+0000 7fef29db9640 1 -- 172.21.15.47:0/3527684525 wait complete. 2024-03-23T09:51:27.374 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:51:27.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.474+0000 7f9057e96640 1 Processor -- start 2024-03-23T09:51:27.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.474+0000 7f9057e96640 1 -- start start 2024-03-23T09:51:27.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.475+0000 7f9057e96640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f905015ca90 0x7f9050160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.475+0000 7f9057e96640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f90500b8350 0x7f90500cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.475+0000 7f9057e96640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90500d11e0 con 0x7f90500cdc40 2024-03-23T09:51:27.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.475+0000 7f9057e96640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90500ce0d0 con 0x7f90500b8350 2024-03-23T09:51:27.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.475+0000 7f9057e96640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90500d2680 con 0x7f905015ca90 2024-03-23T09:51:27.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.475+0000 7f9055431640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f90500b8350 0x7f90500cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.475+0000 7f9055431640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f90500b8350 0x7f90500cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37532/0 (socket says 172.21.15.47:37532) 2024-03-23T09:51:27.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.475+0000 7f9055431640 1 -- 172.21.15.47:0/3068587509 learned_addr learned my addr 172.21.15.47:0/3068587509 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:27.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.475+0000 7f9055c32640 1 --2- 172.21.15.47:0/3068587509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f905015ca90 0x7f9050160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.475+0000 7f9055431640 1 -- 172.21.15.47:0/3068587509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f905015ca90 msgr2=0x7f9050160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:27.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.475+0000 7f9055431640 1 --2- 172.21.15.47:0/3068587509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f905015ca90 0x7f9050160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.475+0000 7f9055431640 1 -- 172.21.15.47:0/3068587509 >> v1:172.21.15.175:6789/0 conn(0x7f90500cdc40 legacy=0x7f90500b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:27.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.475+0000 7f9055431640 1 -- 172.21.15.47:0/3068587509 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9050160f60 con 0x7f90500b8350 2024-03-23T09:51:27.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.476+0000 7f9055431640 1 --2- 172.21.15.47:0/3068587509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90500b8350 0x7f90500cd700 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f9040009870 tx=0x7f904002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a757170ff2b71e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:27.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.476+0000 7f9054c30640 1 -- 172.21.15.47:0/3068587509 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400317c0 con 0x7f90500b8350 2024-03-23T09:51:27.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.476+0000 7f9054c30640 1 -- 172.21.15.47:0/3068587509 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9040031920 con 0x7f90500b8350 2024-03-23T09:51:27.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.476+0000 7f9054c30640 1 -- 172.21.15.47:0/3068587509 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040031cb0 con 0x7f90500b8350 2024-03-23T09:51:27.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.477+0000 7f9057e96640 1 -- 172.21.15.47:0/3068587509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90500b8350 msgr2=0x7f90500cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.477+0000 7f9057e96640 1 --2- 172.21.15.47:0/3068587509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90500b8350 0x7f90500cd700 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f9040009870 tx=0x7f904002b860 comp rx=0 tx=0).stop 2024-03-23T09:51:27.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.477+0000 7f9057e96640 1 -- 172.21.15.47:0/3068587509 shutdown_connections 2024-03-23T09:51:27.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.477+0000 7f9057e96640 1 --2- 172.21.15.47:0/3068587509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90500b8350 0x7f90500cd700 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.477+0000 7f9057e96640 1 --2- 172.21.15.47:0/3068587509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f905015ca90 0x7f9050160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.477+0000 7f9057e96640 1 -- 172.21.15.47:0/3068587509 >> 172.21.15.47:0/3068587509 conn(0x7f90500659d0 msgr2=0x7f9050065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:27.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.477+0000 7f9057e96640 1 -- 172.21.15.47:0/3068587509 shutdown_connections 2024-03-23T09:51:27.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.477+0000 7f9057e96640 1 -- 172.21.15.47:0/3068587509 wait complete. 2024-03-23T09:51:27.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.478+0000 7f9057e96640 1 Processor -- start 2024-03-23T09:51:27.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.478+0000 7f9057e96640 1 -- start start 2024-03-23T09:51:27.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9057e96640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90500b8350 0x7f90500b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9057e96640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90500cdc40 0x7f905015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9057e96640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f905015ca90 0x7f905015f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9057e96640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90500cc3e0 con 0x7f90500b8350 2024-03-23T09:51:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9057e96640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90500cb4f0 con 0x7f905015ca90 2024-03-23T09:51:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9057e96640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90500cb7d0 con 0x7f90500cdc40 2024-03-23T09:51:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9055c32640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90500b8350 0x7f90500b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9055c32640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90500b8350 0x7f90500b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37548/0 (socket says 172.21.15.47:37548) 2024-03-23T09:51:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9055c32640 1 -- 172.21.15.47:0/1858746319 learned_addr learned my addr 172.21.15.47:0/1858746319 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9056433640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f905015ca90 0x7f905015f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9055431640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90500cdc40 0x7f905015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9055c32640 1 -- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90500cdc40 msgr2=0x7f905015ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9055c32640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90500cdc40 0x7f905015ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9055c32640 1 -- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f905015ca90 msgr2=0x7f905015f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9055c32640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f905015ca90 0x7f905015f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9055c32640 1 -- 172.21.15.47:0/1858746319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90400094d0 con 0x7f90500b8350 2024-03-23T09:51:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9055c32640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90500b8350 0x7f90500b47a0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c7f0 tx=0x7f904c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f90467fc640 1 -- 172.21.15.47:0/1858746319 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c004220 con 0x7f90500b8350 2024-03-23T09:51:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f90467fc640 1 -- 172.21.15.47:0/1858746319 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c0043a0 con 0x7f90500b8350 2024-03-23T09:51:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f90467fc640 1 -- 172.21.15.47:0/1858746319 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c007860 con 0x7f90500b8350 2024-03-23T09:51:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.479+0000 7f9057e96640 1 -- 172.21.15.47:0/1858746319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90500cbab0 con 0x7f90500b8350 2024-03-23T09:51:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.480+0000 7f9057e96640 1 -- 172.21.15.47:0/1858746319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90500cc070 con 0x7f90500b8350 2024-03-23T09:51:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.480+0000 7f9057e96640 1 -- 172.21.15.47:0/1858746319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9050072360 con 0x7f90500b8350 2024-03-23T09:51:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.481+0000 7f90467fc640 1 -- 172.21.15.47:0/1858746319 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f904c007bd0 con 0x7f90500b8350 2024-03-23T09:51:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.481+0000 7f90467fc640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f903406ac00 0x7f903406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.481+0000 7f90467fc640 1 -- 172.21.15.47:0/1858746319 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f904c0893d0 con 0x7f90500b8350 2024-03-23T09:51:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.482+0000 7f9055431640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f903406ac00 0x7f903406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.482+0000 7f9055431640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f903406ac00 0x7f903406d0c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9040036000 tx=0x7f904003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:27.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.486+0000 7f90467fc640 1 -- 172.21.15.47:0/1858746319 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f904c054510 con 0x7f90500b8350 2024-03-23T09:51:27.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.665+0000 7f9057e96640 1 -- 172.21.15.47:0/1858746319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9050066be0 con 0x7f90500b8350 2024-03-23T09:51:27.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.666+0000 7f90467fc640 1 -- 172.21.15.47:0/1858746319 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f904c057b30 con 0x7f90500b8350 2024-03-23T09:51:27.667 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:27.667 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":103,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:51:24.949501+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:49:41.158056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-23T09:49:42.210388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/1748845936":"2024-03-24T09:51:24.949481+0000","172.21.15.47:6833/1748845936":"2024-03-24T09:51:24.949481+0000","172.21.15.47:6835/2388298413":"2024-03-24T09:50:48.589519+0000","172.21.15.47:6833/2388298413":"2024-03-24T09:50:48.589519+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:51:27.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.668+0000 7f9057e96640 1 -- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f903406ac00 msgr2=0x7f903406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.668+0000 7f9057e96640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f903406ac00 0x7f903406d0c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9040036000 tx=0x7f904003d040 comp rx=0 tx=0).stop 2024-03-23T09:51:27.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.668+0000 7f9057e96640 1 -- 172.21.15.47:0/1858746319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90500b8350 msgr2=0x7f90500b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.668+0000 7f9057e96640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90500b8350 0x7f90500b47a0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c7f0 tx=0x7f904c00ccb0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.668+0000 7f9057e96640 1 -- 172.21.15.47:0/1858746319 shutdown_connections 2024-03-23T09:51:27.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.668+0000 7f9057e96640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f903406ac00 0x7f903406d0c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.668+0000 7f9057e96640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f905015ca90 0x7f905015f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.668+0000 7f9057e96640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90500cdc40 0x7f905015ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.668+0000 7f9057e96640 1 --2- 172.21.15.47:0/1858746319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90500b8350 0x7f90500b47a0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.668+0000 7f9057e96640 1 -- 172.21.15.47:0/1858746319 >> 172.21.15.47:0/1858746319 conn(0x7f90500659d0 msgr2=0x7f90500b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:27.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.669+0000 7f9057e96640 1 -- 172.21.15.47:0/1858746319 shutdown_connections 2024-03-23T09:51:27.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.669+0000 7f9057e96640 1 -- 172.21.15.47:0/1858746319 wait complete. 2024-03-23T09:51:27.680 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T09:51:27.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:51:27.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.781+0000 7f3f0272e640 1 Processor -- start 2024-03-23T09:51:27.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.781+0000 7f3f0272e640 1 -- start start 2024-03-23T09:51:27.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.781+0000 7f3f0272e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc0ce290 0x7f3efc0d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.781+0000 7f3f0272e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc15e870 con 0x7f3efc0cdeb0 2024-03-23T09:51:27.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.781+0000 7f3f0272e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc0b1140 con 0x7f3efc0b0cb0 2024-03-23T09:51:27.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.781+0000 7f3f0272e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0b0510 con 0x7f3efc0ce290 2024-03-23T09:51:27.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.781+0000 7f3efbfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3efc0cdeb0 0x7f3efc15da50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50948/0 (socket says 172.21.15.47:50948) 2024-03-23T09:51:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.781+0000 7f3efbfff640 1 -- 172.21.15.47:0/1773792842 learned_addr learned my addr 172.21.15.47:0/1773792842 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efb7fe640 1 --2- 172.21.15.47:0/1773792842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc0ce290 0x7f3efc0d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395502468 0 0) 0x7f3efc15e870 con 0x7f3efc0cdeb0 2024-03-23T09:51:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ee4003520 con 0x7f3efc0cdeb0 2024-03-23T09:51:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 477421270 0 0) 0x7f3ee4003520 con 0x7f3efc0cdeb0 2024-03-23T09:51:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc0ce290 msgr2=0x7f3efc0d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 --2- 172.21.15.47:0/1773792842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc0ce290 0x7f3efc0d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 >> v1:172.21.15.175:6789/0 conn(0x7f3efc0b0cb0 legacy=0x7f3efc15e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc0aff50 con 0x7f3efc0cdeb0 2024-03-23T09:51:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f3eec001670 con 0x7f3efc0cdeb0 2024-03-23T09:51:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 >> v1:172.21.15.47:6789/0 conn(0x7f3efc0cdeb0 legacy=0x7f3efc15da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 --2- 172.21.15.47:0/1773792842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc0ce290 0x7f3ee40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 --2- 172.21.15.47:0/1773792842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ee4003f20 0x7f3ee40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 --2- 172.21.15.47:0/1773792842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee4004d00 0x7f3ee4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0b0510 con 0x7f3ee4004d00 2024-03-23T09:51:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee400ba60 con 0x7f3ee4003f20 2024-03-23T09:51:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee400bc20 con 0x7f3efc0ce290 2024-03-23T09:51:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3f00ccb640 1 --2- 172.21.15.47:0/1773792842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee4004d00 0x7f3ee4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3efb7fe640 1 --2- 172.21.15.47:0/1773792842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ee4003f20 0x7f3ee40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3f00ccb640 1 -- 172.21.15.47:0/1773792842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc0ce290 msgr2=0x7f3ee40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3f00ccb640 1 --2- 172.21.15.47:0/1773792842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc0ce290 0x7f3ee40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3f00ccb640 1 -- 172.21.15.47:0/1773792842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ee4003f20 msgr2=0x7f3ee40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3f00ccb640 1 --2- 172.21.15.47:0/1773792842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ee4003f20 0x7f3ee40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.782+0000 7f3f00ccb640 1 -- 172.21.15.47:0/1773792842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee4003520 con 0x7f3ee4004d00 2024-03-23T09:51:27.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.783+0000 7f3f00ccb640 1 --2- 172.21.15.47:0/1773792842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee4004d00 0x7f3ee4005180 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4001e50 tx=0x7f3ee40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434c2d18ef19de35 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:27.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.783+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4003b70 con 0x7f3ee4004d00 2024-03-23T09:51:27.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.783+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef4003cd0 con 0x7f3ee4004d00 2024-03-23T09:51:27.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.783+0000 7f3efaffd640 1 -- 172.21.15.47:0/1773792842 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4005310 con 0x7f3ee4004d00 2024-03-23T09:51:27.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.784+0000 7f3f0272e640 1 -- 172.21.15.47:0/1773792842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee4004d00 msgr2=0x7f3ee4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.784+0000 7f3f0272e640 1 --2- 172.21.15.47:0/1773792842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee4004d00 0x7f3ee4005180 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4001e50 tx=0x7f3ee40057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.784+0000 7f3f00ccb640 1 -- 172.21.15.47:0/1773792842 reap_dead start 2024-03-23T09:51:27.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.784+0000 7f3f0272e640 1 -- 172.21.15.47:0/1773792842 shutdown_connections 2024-03-23T09:51:27.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.784+0000 7f3f0272e640 1 -- 172.21.15.47:0/1773792842 >> 172.21.15.47:0/1773792842 conn(0x7f3efc098a00 msgr2=0x7f3efc0cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:27.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.784+0000 7f3f0272e640 1 -- 172.21.15.47:0/1773792842 shutdown_connections 2024-03-23T09:51:27.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.784+0000 7f3f0272e640 1 -- 172.21.15.47:0/1773792842 wait complete. 2024-03-23T09:51:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.785+0000 7f3f0272e640 1 Processor -- start 2024-03-23T09:51:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.785+0000 7f3f0272e640 1 -- start start 2024-03-23T09:51:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f0272e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ee4004d00 0x7f3efc152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f0272e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc152e80 0x7f3efc142800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f0272e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc142d40 0x7f3efc0c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f0272e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0c93d0 con 0x7f3efc142d40 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f0272e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc153300 con 0x7f3ee4004d00 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f0272e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0b50f0 con 0x7f3efc152e80 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f00ccb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc142d40 0x7f3efc0c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f00ccb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc142d40 0x7f3efc0c50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37560/0 (socket says 172.21.15.47:37560) 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f00ccb640 1 -- 172.21.15.47:0/3430425623 learned_addr learned my addr 172.21.15.47:0/3430425623 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3efb7fe640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc152e80 0x7f3efc142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3efbfff640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ee4004d00 0x7f3efc152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f00ccb640 1 -- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc152e80 msgr2=0x7f3efc142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f00ccb640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc152e80 0x7f3efc142800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f00ccb640 1 -- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ee4004d00 msgr2=0x7f3efc152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f00ccb640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ee4004d00 0x7f3efc152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3f00ccb640 1 -- 172.21.15.47:0/3430425623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc0c8750 con 0x7f3efc142d40 2024-03-23T09:51:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.786+0000 7f3efbfff640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ee4004d00 0x7f3efc152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.787+0000 7f3f00ccb640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc142d40 0x7f3efc0c50e0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4001f70 tx=0x7f3ef40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.787+0000 7f3ef8ff9640 1 -- 172.21.15.47:0/3430425623 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef403e070 con 0x7f3efc142d40 2024-03-23T09:51:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.787+0000 7f3ef8ff9640 1 -- 172.21.15.47:0/3430425623 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef4003e40 con 0x7f3efc142d40 2024-03-23T09:51:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.787+0000 7f3f0272e640 1 -- 172.21.15.47:0/3430425623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3efc0c8a30 con 0x7f3efc142d40 2024-03-23T09:51:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.787+0000 7f3ef8ff9640 1 -- 172.21.15.47:0/3430425623 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4006d90 con 0x7f3efc142d40 2024-03-23T09:51:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.787+0000 7f3f0272e640 1 -- 172.21.15.47:0/3430425623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3efc0c8ff0 con 0x7f3efc142d40 2024-03-23T09:51:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.788+0000 7f3f0272e640 1 -- 172.21.15.47:0/3430425623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3efc0672f0 con 0x7f3efc142d40 2024-03-23T09:51:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.788+0000 7f3ef8ff9640 1 -- 172.21.15.47:0/3430425623 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3ef4007030 con 0x7f3efc142d40 2024-03-23T09:51:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.788+0000 7f3ef8ff9640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ee806ac00 0x7f3ee806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.789+0000 7f3ef8ff9640 1 -- 172.21.15.47:0/3430425623 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f3ef40aa5e0 con 0x7f3efc142d40 2024-03-23T09:51:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.789+0000 7f3efbfff640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ee806ac00 0x7f3ee806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.789+0000 7f3efbfff640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ee806ac00 0x7f3ee806d0c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0017a0 tx=0x7f3eec0016f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:27.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:27.793+0000 7f3ef8ff9640 1 -- 172.21.15.47:0/3430425623 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3ef4075710 con 0x7f3efc142d40 2024-03-23T09:51:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.012+0000 7f3f0272e640 1 -- 172.21.15.47:0/3430425623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3efc073340 con 0x7f3efc142d40 2024-03-23T09:51:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.013+0000 7f3ef8ff9640 1 -- 172.21.15.47:0/3430425623 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4733 (secure 0 0 0) 0x7f3ef4048090 con 0x7f3efc142d40 2024-03-23T09:51:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:28.014 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/249416109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":249416109},{"type":"v1","addr":"172.21.15.175:6837","nonce":249416109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3603846321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3603846321},{"type":"v1","addr":"172.21.15.175:6835","nonce":3603846321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1921094794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1921094794},{"type":"v1","addr":"172.21.15.47:6834","nonce":1921094794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":9078,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3030917369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":3030917369},{"type":"v1","addr":"172.21.15.47:6835","nonce":3030917369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:49:43.247119+0000","modified":"2024-03-23T09:51:24.949801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-23T09:51:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 73 2024-03-23T09:51:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.015+0000 7f3f0272e640 1 -- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ee806ac00 msgr2=0x7f3ee806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.015+0000 7f3f0272e640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ee806ac00 0x7f3ee806d0c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0017a0 tx=0x7f3eec0016f0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.015+0000 7f3f0272e640 1 -- 172.21.15.47:0/3430425623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc142d40 msgr2=0x7f3efc0c50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.015+0000 7f3f0272e640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc142d40 0x7f3efc0c50e0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4001f70 tx=0x7f3ef40014a0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.015+0000 7f3f0272e640 1 -- 172.21.15.47:0/3430425623 shutdown_connections 2024-03-23T09:51:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.015+0000 7f3f0272e640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3ee806ac00 0x7f3ee806d0c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.015+0000 7f3f0272e640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3efc142d40 0x7f3efc0c50e0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.015+0000 7f3f0272e640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3efc152e80 0x7f3efc142800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.015+0000 7f3f0272e640 1 --2- 172.21.15.47:0/3430425623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ee4004d00 0x7f3efc152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.015+0000 7f3f0272e640 1 -- 172.21.15.47:0/3430425623 >> 172.21.15.47:0/3430425623 conn(0x7f3efc098a00 msgr2=0x7f3efc0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.015+0000 7f3f0272e640 1 -- 172.21.15.47:0/3430425623 shutdown_connections 2024-03-23T09:51:28.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.015+0000 7f3f0272e640 1 -- 172.21.15.47:0/3430425623 wait complete. 2024-03-23T09:51:28.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:51:28.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.127+0000 7f287e484640 1 Processor -- start 2024-03-23T09:51:28.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.127+0000 7f287e484640 1 -- start start 2024-03-23T09:51:28.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.128+0000 7f287e484640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28780cde10 0x7f2878167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.128+0000 7f287e484640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28780d11e0 con 0x7f287815ca90 2024-03-23T09:51:28.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.128+0000 7f287e484640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28780d2680 con 0x7f28780b8350 2024-03-23T09:51:28.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.128+0000 7f287e484640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2878160f60 con 0x7f28780cde10 2024-03-23T09:51:28.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.128+0000 7f2877fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f287815ca90 0x7f2878160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50952/0 (socket says 172.21.15.47:50952) 2024-03-23T09:51:28.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.128+0000 7f2877fff640 1 -- 172.21.15.47:0/2512534036 learned_addr learned my addr 172.21.15.47:0/2512534036 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:28.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875219066 0 0) 0x7f28780d11e0 con 0x7f287815ca90 2024-03-23T09:51:28.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2864003520 con 0x7f287815ca90 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f287ca21640 1 --2- 172.21.15.47:0/2512534036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28780cde10 0x7f2878167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1145593148 0 0) 0x7f2864003520 con 0x7f287815ca90 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28780cde10 msgr2=0x7f2878167aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 --2- 172.21.15.47:0/2512534036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28780cde10 0x7f2878167aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 >> v1:172.21.15.175:6789/0 conn(0x7f28780b8350 legacy=0x7f28780cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2878161240 con 0x7f287815ca90 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f2868001670 con 0x7f287815ca90 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 >> v1:172.21.15.47:6789/0 conn(0x7f287815ca90 legacy=0x7f2878160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 --2- 172.21.15.47:0/2512534036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28640039a0 0x7f2864003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 --2- 172.21.15.47:0/2512534036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28640047c0 0x7f2864004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 --2- 172.21.15.47:0/2512534036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28780cde10 0x7f28640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2878160f60 con 0x7f28640047c0 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286400bae0 con 0x7f28640039a0 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286400bca0 con 0x7f28780cde10 2024-03-23T09:51:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f28777fe640 1 --2- 172.21.15.47:0/2512534036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28640047c0 0x7f2864004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f287ca21640 1 --2- 172.21.15.47:0/2512534036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28780cde10 0x7f28640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f2877fff640 1 --2- 172.21.15.47:0/2512534036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28640039a0 0x7f2864003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f28777fe640 1 -- 172.21.15.47:0/2512534036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28780cde10 msgr2=0x7f28640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f28777fe640 1 --2- 172.21.15.47:0/2512534036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28780cde10 0x7f28640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f28777fe640 1 -- 172.21.15.47:0/2512534036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28640039a0 msgr2=0x7f2864003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f28777fe640 1 --2- 172.21.15.47:0/2512534036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28640039a0 0x7f2864003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.129+0000 7f28777fe640 1 -- 172.21.15.47:0/2512534036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2864003520 con 0x7f28640047c0 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.130+0000 7f28777fe640 1 --2- 172.21.15.47:0/2512534036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28640047c0 0x7f2864004c40 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f2860001750 tx=0x7f28640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f96cd8fecc9d655 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.130+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600052d0 con 0x7f28640047c0 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.130+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286002e050 con 0x7f28640047c0 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.130+0000 7f2876ffd640 1 -- 172.21.15.47:0/2512534036 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600035f0 con 0x7f28640047c0 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.131+0000 7f287e484640 1 -- 172.21.15.47:0/2512534036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28640047c0 msgr2=0x7f2864004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.131+0000 7f287e484640 1 --2- 172.21.15.47:0/2512534036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28640047c0 0x7f2864004c40 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f2860001750 tx=0x7f28640057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.131+0000 7f287ca21640 1 -- 172.21.15.47:0/2512534036 reap_dead start 2024-03-23T09:51:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.131+0000 7f287e484640 1 -- 172.21.15.47:0/2512534036 shutdown_connections 2024-03-23T09:51:28.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.131+0000 7f287e484640 1 -- 172.21.15.47:0/2512534036 >> 172.21.15.47:0/2512534036 conn(0x7f28780659d0 msgr2=0x7f2878065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:28.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.131+0000 7f287e484640 1 -- 172.21.15.47:0/2512534036 shutdown_connections 2024-03-23T09:51:28.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.131+0000 7f287e484640 1 -- 172.21.15.47:0/2512534036 wait complete. 2024-03-23T09:51:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.132+0000 7f287e484640 1 Processor -- start 2024-03-23T09:51:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.132+0000 7f287e484640 1 -- start start 2024-03-23T09:51:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f287e484640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28640047c0 0x7f2878150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f287e484640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2878154000 0x7f2878152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f287e484640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28780b3d20 0x7f28780b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f287e484640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2878153380 con 0x7f28640047c0 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f287e484640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28780b46e0 con 0x7f28780b3d20 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f287e484640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2878159080 con 0x7f2878154000 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f2877fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28640047c0 0x7f2878150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f2877fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28640047c0 0x7f2878150f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37568/0 (socket says 172.21.15.47:37568) 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f2877fff640 1 -- 172.21.15.47:0/4157733121 learned_addr learned my addr 172.21.15.47:0/4157733121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f2877fff640 1 -- 172.21.15.47:0/4157733121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2878154000 msgr2=0x7f2878152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f2877fff640 1 --2- 172.21.15.47:0/4157733121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2878154000 0x7f2878152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f2877fff640 1 -- 172.21.15.47:0/4157733121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28780b3d20 msgr2=0x7f28780b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f2877fff640 1 --2- 172.21.15.47:0/4157733121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28780b3d20 0x7f28780b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f2877fff640 1 -- 172.21.15.47:0/4157733121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2878152540 con 0x7f28640047c0 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f2877fff640 1 --2- 172.21.15.47:0/4157733121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28640047c0 0x7f2878150f20 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f2868007290 tx=0x7f2868007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.133+0000 7f2874ff9640 1 -- 172.21.15.47:0/4157733121 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868001650 con 0x7f28640047c0 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.134+0000 7f2874ff9640 1 -- 172.21.15.47:0/4157733121 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286800f940 con 0x7f28640047c0 2024-03-23T09:51:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.134+0000 7f287e484640 1 -- 172.21.15.47:0/4157733121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2878154480 con 0x7f28640047c0 2024-03-23T09:51:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.134+0000 7f2874ff9640 1 -- 172.21.15.47:0/4157733121 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286800fcd0 con 0x7f28640047c0 2024-03-23T09:51:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.134+0000 7f287e484640 1 -- 172.21.15.47:0/4157733121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2878152a50 con 0x7f28640047c0 2024-03-23T09:51:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.135+0000 7f287e484640 1 -- 172.21.15.47:0/4157733121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2878072360 con 0x7f28640047c0 2024-03-23T09:51:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.135+0000 7f2874ff9640 1 -- 172.21.15.47:0/4157733121 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f286801d460 con 0x7f28640047c0 2024-03-23T09:51:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.135+0000 7f2874ff9640 1 --2- 172.21.15.47:0/4157733121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f285806ac30 0x7f285806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.135+0000 7f2874ff9640 1 -- 172.21.15.47:0/4157733121 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f2868089300 con 0x7f28640047c0 2024-03-23T09:51:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.136+0000 7f28777fe640 1 --2- 172.21.15.47:0/4157733121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f285806ac30 0x7f285806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.136+0000 7f28777fe640 1 --2- 172.21.15.47:0/4157733121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f285806ac30 0x7f285806d0f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2860033000 tx=0x7f286002f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:28.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.139+0000 7f2874ff9640 1 -- 172.21.15.47:0/4157733121 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2868054430 con 0x7f28640047c0 2024-03-23T09:51:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.355+0000 7f287e484640 1 -- 172.21.15.47:0/4157733121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28780b88c0 con 0x7f28640047c0 2024-03-23T09:51:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.356+0000 7f2874ff9640 1 -- 172.21.15.47:0/4157733121 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4733 (secure 0 0 0) 0x7f2868057a50 con 0x7f28640047c0 2024-03-23T09:51:28.356 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:28.356 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/249416109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":249416109},{"type":"v1","addr":"172.21.15.175:6837","nonce":249416109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3603846321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3603846321},{"type":"v1","addr":"172.21.15.175:6835","nonce":3603846321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1921094794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1921094794},{"type":"v1","addr":"172.21.15.47:6834","nonce":1921094794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":9078,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3030917369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":3030917369},{"type":"v1","addr":"172.21.15.47:6835","nonce":3030917369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:49:43.247119+0000","modified":"2024-03-23T09:51:24.949801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-23T09:51:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 73 2024-03-23T09:51:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.357+0000 7f287e484640 1 -- 172.21.15.47:0/4157733121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f285806ac30 msgr2=0x7f285806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.357+0000 7f287e484640 1 --2- 172.21.15.47:0/4157733121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f285806ac30 0x7f285806d0f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2860033000 tx=0x7f286002f000 comp rx=0 tx=0).stop 2024-03-23T09:51:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.358+0000 7f287e484640 1 -- 172.21.15.47:0/4157733121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28640047c0 msgr2=0x7f2878150f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.358+0000 7f287e484640 1 --2- 172.21.15.47:0/4157733121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28640047c0 0x7f2878150f20 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f2868007290 tx=0x7f2868007750 comp rx=0 tx=0).stop 2024-03-23T09:51:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.358+0000 7f287e484640 1 -- 172.21.15.47:0/4157733121 shutdown_connections 2024-03-23T09:51:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.358+0000 7f287e484640 1 --2- 172.21.15.47:0/4157733121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f285806ac30 0x7f285806d0f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.358+0000 7f287e484640 1 --2- 172.21.15.47:0/4157733121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28780b3d20 0x7f28780b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.358+0000 7f287e484640 1 --2- 172.21.15.47:0/4157733121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2878154000 0x7f2878152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.358+0000 7f287e484640 1 --2- 172.21.15.47:0/4157733121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28640047c0 0x7f2878150f20 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.358+0000 7f287e484640 1 -- 172.21.15.47:0/4157733121 >> 172.21.15.47:0/4157733121 conn(0x7f28780659d0 msgr2=0x7f28780c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.358+0000 7f287e484640 1 -- 172.21.15.47:0/4157733121 shutdown_connections 2024-03-23T09:51:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.358+0000 7f287e484640 1 -- 172.21.15.47:0/4157733121 wait complete. 2024-03-23T09:51:28.368 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:51:28.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.469+0000 7fd52af95640 1 Processor -- start 2024-03-23T09:51:28.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.469+0000 7fd52af95640 1 -- start start 2024-03-23T09:51:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd52af95640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52415ca90 0x7fd524160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd52af95640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5240cde10 0x7fd5240d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd52af95640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5240d11e0 con 0x7fd5240b8350 2024-03-23T09:51:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd52af95640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd524160f60 con 0x7fd5240cde10 2024-03-23T09:51:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd52af95640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd524161240 con 0x7fd52415ca90 2024-03-23T09:51:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd529532640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5240cde10 0x7fd5240d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd523fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd5240b8350 0x7fd5240cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50960/0 (socket says 172.21.15.47:50960) 2024-03-23T09:51:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd523fff640 1 -- 172.21.15.47:0/2065716171 learned_addr learned my addr 172.21.15.47:0/2065716171 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd528d31640 1 --2- 172.21.15.47:0/2065716171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52415ca90 0x7fd524160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd529532640 1 -- 172.21.15.47:0/2065716171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52415ca90 msgr2=0x7fd524160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd529532640 1 --2- 172.21.15.47:0/2065716171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52415ca90 0x7fd524160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd529532640 1 -- 172.21.15.47:0/2065716171 >> v1:172.21.15.47:6789/0 conn(0x7fd5240b8350 legacy=0x7fd5240cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd529532640 1 -- 172.21.15.47:0/2065716171 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd524161520 con 0x7fd5240cde10 2024-03-23T09:51:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.470+0000 7fd5237fe640 1 -- 172.21.15.47:0/2065716171 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667404342 0 0) 0x7fd5240d11e0 con 0x7fd5240b8350 2024-03-23T09:51:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.471+0000 7fd529532640 1 --2- 172.21.15.47:0/2065716171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5240cde10 0x7fd5240d1670 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009d90 tx=0x7fd51c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34220f077f89e771 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.471+0000 7fd5237fe640 1 -- 172.21.15.47:0/2065716171 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c02cbc0 con 0x7fd5240cde10 2024-03-23T09:51:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.471+0000 7fd5237fe640 1 -- 172.21.15.47:0/2065716171 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c02cd20 con 0x7fd5240cde10 2024-03-23T09:51:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.471+0000 7fd5237fe640 1 -- 172.21.15.47:0/2065716171 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c034660 con 0x7fd5240cde10 2024-03-23T09:51:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.471+0000 7fd52af95640 1 -- 172.21.15.47:0/2065716171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5240cde10 msgr2=0x7fd5240d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.471+0000 7fd52af95640 1 --2- 172.21.15.47:0/2065716171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5240cde10 0x7fd5240d1670 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009d90 tx=0x7fd51c02be10 comp rx=0 tx=0).stop 2024-03-23T09:51:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.472+0000 7fd52af95640 1 -- 172.21.15.47:0/2065716171 shutdown_connections 2024-03-23T09:51:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.472+0000 7fd52af95640 1 --2- 172.21.15.47:0/2065716171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5240cde10 0x7fd5240d1670 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.472+0000 7fd52af95640 1 --2- 172.21.15.47:0/2065716171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52415ca90 0x7fd524160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.472+0000 7fd52af95640 1 -- 172.21.15.47:0/2065716171 >> 172.21.15.47:0/2065716171 conn(0x7fd5240659d0 msgr2=0x7fd524065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.472+0000 7fd52af95640 1 -- 172.21.15.47:0/2065716171 shutdown_connections 2024-03-23T09:51:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.472+0000 7fd52af95640 1 -- 172.21.15.47:0/2065716171 wait complete. 2024-03-23T09:51:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.473+0000 7fd52af95640 1 Processor -- start 2024-03-23T09:51:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.473+0000 7fd52af95640 1 -- start start 2024-03-23T09:51:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd52af95640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5240b8350 0x7fd52414d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd52af95640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5240cde10 0x7fd5240b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd52af95640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52415ca90 0x7fd52415f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd52af95640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240b3ea0 con 0x7fd5240b8350 2024-03-23T09:51:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd52af95640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240b4180 con 0x7fd5240cde10 2024-03-23T09:51:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd52af95640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240b4460 con 0x7fd52415ca90 2024-03-23T09:51:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd528d31640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5240b8350 0x7fd52414d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd528d31640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5240b8350 0x7fd52414d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37586/0 (socket says 172.21.15.47:37586) 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd528d31640 1 -- 172.21.15.47:0/850171884 learned_addr learned my addr 172.21.15.47:0/850171884 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd529532640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52415ca90 0x7fd52415f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd523fff640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5240cde10 0x7fd5240b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd528d31640 1 -- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52415ca90 msgr2=0x7fd52415f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd528d31640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52415ca90 0x7fd52415f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd528d31640 1 -- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5240cde10 msgr2=0x7fd5240b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd528d31640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5240cde10 0x7fd5240b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd528d31640 1 -- 172.21.15.47:0/850171884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c009a70 con 0x7fd5240b8350 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd529532640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52415ca90 0x7fd52415f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd523fff640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5240cde10 0x7fd5240b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd528d31640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5240b8350 0x7fd52414d200 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fd51800e9b0 tx=0x7fd51800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd5217fa640 1 -- 172.21.15.47:0/850171884 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518009670 con 0x7fd5240b8350 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd5217fa640 1 -- 172.21.15.47:0/850171884 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51800c9a0 con 0x7fd5240b8350 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd52af95640 1 -- 172.21.15.47:0/850171884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5240b4740 con 0x7fd5240b8350 2024-03-23T09:51:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd5217fa640 1 -- 172.21.15.47:0/850171884 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51800cd50 con 0x7fd5240b8350 2024-03-23T09:51:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.474+0000 7fd52af95640 1 -- 172.21.15.47:0/850171884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5240b4f90 con 0x7fd5240b8350 2024-03-23T09:51:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.475+0000 7fd52af95640 1 -- 172.21.15.47:0/850171884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5240b48d0 con 0x7fd5240b8350 2024-03-23T09:51:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.476+0000 7fd5217fa640 1 -- 172.21.15.47:0/850171884 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd518013470 con 0x7fd5240b8350 2024-03-23T09:51:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.476+0000 7fd5217fa640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd50806ac00 0x7fd50806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.476+0000 7fd5217fa640 1 -- 172.21.15.47:0/850171884 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fd5180896a0 con 0x7fd5240b8350 2024-03-23T09:51:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.477+0000 7fd523fff640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd50806ac00 0x7fd50806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.477+0000 7fd523fff640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd50806ac00 0x7fd50806d0c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd5240b5460 tx=0x7fd510009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:28.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.481+0000 7fd5217fa640 1 -- 172.21.15.47:0/850171884 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd5180547d0 con 0x7fd5240b8350 2024-03-23T09:51:28.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.659+0000 7fd52af95640 1 -- 172.21.15.47:0/850171884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd524072360 con 0x7fd5240b8350 2024-03-23T09:51:28.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.660+0000 7fd5217fa640 1 -- 172.21.15.47:0/850171884 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16302 (secure 0 0 0) 0x7fd518057df0 con 0x7fd5240b8350 2024-03-23T09:51:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:28.661 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":103,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:51:24.949501+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:49:41.158056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-23T09:49:42.210388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/1748845936":"2024-03-24T09:51:24.949481+0000","172.21.15.47:6833/1748845936":"2024-03-24T09:51:24.949481+0000","172.21.15.47:6835/2388298413":"2024-03-24T09:50:48.589519+0000","172.21.15.47:6833/2388298413":"2024-03-24T09:50:48.589519+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:51:28.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.662+0000 7fd52af95640 1 -- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd50806ac00 msgr2=0x7fd50806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.662+0000 7fd52af95640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd50806ac00 0x7fd50806d0c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd5240b5460 tx=0x7fd510009040 comp rx=0 tx=0).stop 2024-03-23T09:51:28.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.662+0000 7fd52af95640 1 -- 172.21.15.47:0/850171884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5240b8350 msgr2=0x7fd52414d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.662+0000 7fd52af95640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5240b8350 0x7fd52414d200 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fd51800e9b0 tx=0x7fd51800ee70 comp rx=0 tx=0).stop 2024-03-23T09:51:28.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.662+0000 7fd52af95640 1 -- 172.21.15.47:0/850171884 shutdown_connections 2024-03-23T09:51:28.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.662+0000 7fd52af95640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd50806ac00 0x7fd50806d0c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.662+0000 7fd52af95640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52415ca90 0x7fd52415f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.662+0000 7fd52af95640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5240cde10 0x7fd5240b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.662+0000 7fd52af95640 1 --2- 172.21.15.47:0/850171884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5240b8350 0x7fd52414d200 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.662+0000 7fd52af95640 1 -- 172.21.15.47:0/850171884 >> 172.21.15.47:0/850171884 conn(0x7fd5240659d0 msgr2=0x7fd52415d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:28.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.662+0000 7fd52af95640 1 -- 172.21.15.47:0/850171884 shutdown_connections 2024-03-23T09:51:28.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.662+0000 7fd52af95640 1 -- 172.21.15.47:0/850171884 wait complete. 2024-03-23T09:51:28.673 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:51:28.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.775+0000 7fc514ce5640 1 Processor -- start 2024-03-23T09:51:28.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.775+0000 7fc514ce5640 1 -- start start 2024-03-23T09:51:28.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.776+0000 7fc514ce5640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc51015c8c0 0x7fc5100cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.776+0000 7fc514ce5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51006b400 0x7fc51006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.776+0000 7fc514ce5640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5100ccea0 con 0x7fc51015cca0 2024-03-23T09:51:28.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.776+0000 7fc514ce5640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5100719a0 con 0x7fc51015c8c0 2024-03-23T09:51:28.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.776+0000 7fc514ce5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5100cb650 con 0x7fc51006b400 2024-03-23T09:51:28.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.776+0000 7fc50dd9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc51015cca0 0x7fc51006acf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50962/0 (socket says 172.21.15.47:50962) 2024-03-23T09:51:28.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.776+0000 7fc50dd9b640 1 -- 172.21.15.47:0/1693761521 learned_addr learned my addr 172.21.15.47:0/1693761521 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.776+0000 7fc50ed9d640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51006b400 0x7fc51006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.776+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491045604 0 0) 0x7fc504001110 con 0x7fc51015cca0 2024-03-23T09:51:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.776+0000 7fc50e59c640 1 --2- 172.21.15.47:0/1693761521 >> v2:172.21.15.175:3300/0 conn(0x7fc51015c8c0 0x7fc5100cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.776+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4f8003520 con 0x7fc51015cca0 2024-03-23T09:51:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1456997280 0 0) 0x7fc4f8003520 con 0x7fc51015cca0 2024-03-23T09:51:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51006b400 msgr2=0x7fc51006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51006b400 0x7fc51006cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 >> v2:172.21.15.175:3300/0 conn(0x7fc51015c8c0 msgr2=0x7fc5100cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 --2- 172.21.15.47:0/1693761521 >> v2:172.21.15.175:3300/0 conn(0x7fc51015c8c0 0x7fc5100cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51006d070 con 0x7fc51015cca0 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fc504001800 con 0x7fc51015cca0 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 >> v1:172.21.15.47:6789/0 conn(0x7fc51015cca0 legacy=0x7fc51006acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f8003970 0x7fc4f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f80047c0 0x7fc4f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51006b400 0x7fc4f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5100719a0 con 0x7fc4f8003970 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5100cb650 con 0x7fc4f80047c0 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f800b980 con 0x7fc51006b400 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50e59c640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f8003970 0x7fc4f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50dd9b640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f80047c0 0x7fc4f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50ed9d640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51006b400 0x7fc4f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50e59c640 1 -- 172.21.15.47:0/1693761521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51006b400 msgr2=0x7fc4f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50e59c640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51006b400 0x7fc4f8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50e59c640 1 -- 172.21.15.47:0/1693761521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f80047c0 msgr2=0x7fc4f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50e59c640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f80047c0 0x7fc4f8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50e59c640 1 -- 172.21.15.47:0/1693761521 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f8003520 con 0x7fc4f8003970 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50ed9d640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51006b400 0x7fc4f8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.777+0000 7fc50dd9b640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f80047c0 0x7fc4f8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.778+0000 7fc50e59c640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f8003970 0x7fc4f8003d90 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fc508009ac0 tx=0x7fc4f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1008a79d1082e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.778+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50802c710 con 0x7fc4f8003970 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.778+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50802c870 con 0x7fc4f8003970 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.778+0000 7fc50d59a640 1 -- 172.21.15.47:0/1693761521 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50802cc00 con 0x7fc4f8003970 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.778+0000 7fc514ce5640 1 -- 172.21.15.47:0/1693761521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f8003970 msgr2=0x7fc4f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.778+0000 7fc514ce5640 1 --2- 172.21.15.47:0/1693761521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f8003970 0x7fc4f8003d90 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fc508009ac0 tx=0x7fc4f80057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.778+0000 7fc50ed9d640 1 -- 172.21.15.47:0/1693761521 reap_dead start 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.779+0000 7fc514ce5640 1 -- 172.21.15.47:0/1693761521 shutdown_connections 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.779+0000 7fc514ce5640 1 -- 172.21.15.47:0/1693761521 >> 172.21.15.47:0/1693761521 conn(0x7fc510070ad0 msgr2=0x7fc510070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.779+0000 7fc514ce5640 1 -- 172.21.15.47:0/1693761521 shutdown_connections 2024-03-23T09:51:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.779+0000 7fc514ce5640 1 -- 172.21.15.47:0/1693761521 wait complete. 2024-03-23T09:51:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.780+0000 7fc514ce5640 1 Processor -- start 2024-03-23T09:51:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.780+0000 7fc514ce5640 1 -- start start 2024-03-23T09:51:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc514ce5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f8003970 0x7fc5100b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc514ce5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc51015ebc0 0x7fc5101f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc514ce5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc510147710 0x7fc510147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc514ce5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc510158b40 con 0x7fc4f8003970 2024-03-23T09:51:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc514ce5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5100b47c0 con 0x7fc51015ebc0 2024-03-23T09:51:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc514ce5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51015de00 con 0x7fc510147710 2024-03-23T09:51:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc50e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f8003970 0x7fc5100b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc50e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f8003970 0x7fc5100b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37606/0 (socket says 172.21.15.47:37606) 2024-03-23T09:51:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc50e59c640 1 -- 172.21.15.47:0/565051499 learned_addr learned my addr 172.21.15.47:0/565051499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc50e59c640 1 -- 172.21.15.47:0/565051499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc510147710 msgr2=0x7fc510147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc50e59c640 1 --2- 172.21.15.47:0/565051499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc510147710 0x7fc510147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc50e59c640 1 -- 172.21.15.47:0/565051499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc51015ebc0 msgr2=0x7fc5101f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc50dd9b640 1 --2- 172.21.15.47:0/565051499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc51015ebc0 0x7fc5101f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc50e59c640 1 --2- 172.21.15.47:0/565051499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc51015ebc0 0x7fc5101f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc50e59c640 1 -- 172.21.15.47:0/565051499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51015e0e0 con 0x7fc4f8003970 2024-03-23T09:51:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc50e59c640 1 --2- 172.21.15.47:0/565051499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f8003970 0x7fc5100b41a0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fc50802c2b0 tx=0x7fc508004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc4feffd640 1 -- 172.21.15.47:0/565051499 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc508003810 con 0x7fc4f8003970 2024-03-23T09:51:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc4feffd640 1 -- 172.21.15.47:0/565051499 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50802de90 con 0x7fc4f8003970 2024-03-23T09:51:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc514ce5640 1 -- 172.21.15.47:0/565051499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51015e3c0 con 0x7fc4f8003970 2024-03-23T09:51:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc4feffd640 1 -- 172.21.15.47:0/565051499 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc508003b70 con 0x7fc4f8003970 2024-03-23T09:51:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.781+0000 7fc514ce5640 1 -- 172.21.15.47:0/565051499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5101f25e0 con 0x7fc4f8003970 2024-03-23T09:51:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.782+0000 7fc514ce5640 1 -- 172.21.15.47:0/565051499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc510071110 con 0x7fc4f8003970 2024-03-23T09:51:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.783+0000 7fc4feffd640 1 -- 172.21.15.47:0/565051499 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc508003cd0 con 0x7fc4f8003970 2024-03-23T09:51:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.783+0000 7fc4feffd640 1 --2- 172.21.15.47:0/565051499 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4f006ac30 0x7fc4f006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.783+0000 7fc4feffd640 1 -- 172.21.15.47:0/565051499 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fc5080ab1f0 con 0x7fc4f8003970 2024-03-23T09:51:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.783+0000 7fc50dd9b640 1 --2- 172.21.15.47:0/565051499 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4f006ac30 0x7fc4f006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.784+0000 7fc50dd9b640 1 --2- 172.21.15.47:0/565051499 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4f006ac30 0x7fc4f006d0f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc504001eb0 tx=0x7fc5040015c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:28.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:28.787+0000 7fc4feffd640 1 -- 172.21.15.47:0/565051499 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc508076320 con 0x7fc4f8003970 2024-03-23T09:51:29.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.007+0000 7fc514ce5640 1 -- 172.21.15.47:0/565051499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc5100672f0 con 0x7fc4f8003970 2024-03-23T09:51:29.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.980+0000 7fc4feffd640 1 -- 172.21.15.47:0/565051499 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc508079940 con 0x7fc4f8003970 2024-03-23T09:51:29.980 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:51:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.983+0000 7fc514ce5640 1 -- 172.21.15.47:0/565051499 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4f006ac30 msgr2=0x7fc4f006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.983+0000 7fc514ce5640 1 --2- 172.21.15.47:0/565051499 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4f006ac30 0x7fc4f006d0f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc504001eb0 tx=0x7fc5040015c0 comp rx=0 tx=0).stop 2024-03-23T09:51:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.983+0000 7fc514ce5640 1 -- 172.21.15.47:0/565051499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f8003970 msgr2=0x7fc5100b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.983+0000 7fc514ce5640 1 --2- 172.21.15.47:0/565051499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f8003970 0x7fc5100b41a0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fc50802c2b0 tx=0x7fc508004730 comp rx=0 tx=0).stop 2024-03-23T09:51:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.983+0000 7fc514ce5640 1 -- 172.21.15.47:0/565051499 shutdown_connections 2024-03-23T09:51:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.983+0000 7fc514ce5640 1 --2- 172.21.15.47:0/565051499 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc4f006ac30 0x7fc4f006d0f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.983+0000 7fc514ce5640 1 --2- 172.21.15.47:0/565051499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc510147710 0x7fc510147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.983+0000 7fc514ce5640 1 --2- 172.21.15.47:0/565051499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc51015ebc0 0x7fc5101f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.983+0000 7fc514ce5640 1 --2- 172.21.15.47:0/565051499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4f8003970 0x7fc5100b41a0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.983+0000 7fc514ce5640 1 -- 172.21.15.47:0/565051499 >> 172.21.15.47:0/565051499 conn(0x7fc510070ad0 msgr2=0x7fc510159510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.983+0000 7fc514ce5640 1 -- 172.21.15.47:0/565051499 shutdown_connections 2024-03-23T09:51:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:29.983+0000 7fc514ce5640 1 -- 172.21.15.47:0/565051499 wait complete. 2024-03-23T09:51:29.997 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T09:51:30.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.107+0000 7fb28bda9640 1 Processor -- start 2024-03-23T09:51:30.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.107+0000 7fb28bda9640 1 -- start start 2024-03-23T09:51:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.108+0000 7fb28bda9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb28415ce10 0x7fb2840cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.108+0000 7fb28bda9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb2840b85d0 0x7fb2840cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.108+0000 7fb28bda9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb284161600 con 0x7fb28415ca30 2024-03-23T09:51:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.108+0000 7fb28bda9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb284160e50 con 0x7fb2840b85d0 2024-03-23T09:51:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.108+0000 7fb28bda9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2840d25e0 con 0x7fb28415ce10 2024-03-23T09:51:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.108+0000 7fb28a346640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb2840b85d0 0x7fb2840cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.108+0000 7fb28a346640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb2840b85d0 0x7fb2840cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37612/0 (socket says 172.21.15.47:37612) 2024-03-23T09:51:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.108+0000 7fb28a346640 1 -- 172.21.15.47:0/1136481859 learned_addr learned my addr 172.21.15.47:0/1136481859 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.108+0000 7fb289344640 1 --2- 172.21.15.47:0/1136481859 >> v2:172.21.15.175:3300/0 conn(0x7fb28415ce10 0x7fb2840cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.109+0000 7fb28a346640 1 -- 172.21.15.47:0/1136481859 >> v2:172.21.15.175:3300/0 conn(0x7fb28415ce10 msgr2=0x7fb2840cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.109+0000 7fb28a346640 1 --2- 172.21.15.47:0/1136481859 >> v2:172.21.15.175:3300/0 conn(0x7fb28415ce10 0x7fb2840cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.109+0000 7fb28a346640 1 -- 172.21.15.47:0/1136481859 >> v1:172.21.15.175:6789/0 conn(0x7fb28415ca30 legacy=0x7fb284160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.109+0000 7fb28a346640 1 -- 172.21.15.47:0/1136481859 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb284161040 con 0x7fb2840b85d0 2024-03-23T09:51:30.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.109+0000 7fb28a346640 1 --2- 172.21.15.47:0/1136481859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2840b85d0 0x7fb2840cdc70 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fb284066930 tx=0x7fb27402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5ca352905b2d89 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:30.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.109+0000 7fb288b43640 1 -- 172.21.15.47:0/1136481859 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27402ca80 con 0x7fb2840b85d0 2024-03-23T09:51:30.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.110+0000 7fb288b43640 1 -- 172.21.15.47:0/1136481859 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27402cbe0 con 0x7fb2840b85d0 2024-03-23T09:51:30.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.110+0000 7fb288b43640 1 -- 172.21.15.47:0/1136481859 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2740343f0 con 0x7fb2840b85d0 2024-03-23T09:51:30.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.110+0000 7fb28bda9640 1 -- 172.21.15.47:0/1136481859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2840b85d0 msgr2=0x7fb2840cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:30.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.110+0000 7fb28bda9640 1 --2- 172.21.15.47:0/1136481859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2840b85d0 0x7fb2840cdc70 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fb284066930 tx=0x7fb27402bee0 comp rx=0 tx=0).stop 2024-03-23T09:51:30.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.110+0000 7fb28bda9640 1 -- 172.21.15.47:0/1136481859 shutdown_connections 2024-03-23T09:51:30.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.110+0000 7fb28bda9640 1 --2- 172.21.15.47:0/1136481859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2840b85d0 0x7fb2840cdc70 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:30.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.110+0000 7fb28bda9640 1 --2- 172.21.15.47:0/1136481859 >> v2:172.21.15.175:3300/0 conn(0x7fb28415ce10 0x7fb2840cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:30.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.110+0000 7fb28bda9640 1 -- 172.21.15.47:0/1136481859 >> 172.21.15.47:0/1136481859 conn(0x7fb28406b160 msgr2=0x7fb28406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:30.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.110+0000 7fb28bda9640 1 -- 172.21.15.47:0/1136481859 shutdown_connections 2024-03-23T09:51:30.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.110+0000 7fb28bda9640 1 -- 172.21.15.47:0/1136481859 wait complete. 2024-03-23T09:51:30.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.111+0000 7fb28bda9640 1 Processor -- start 2024-03-23T09:51:30.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.111+0000 7fb28bda9640 1 -- start start 2024-03-23T09:51:30.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb28bda9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2840b85d0 0x7fb28414afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:30.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb28bda9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb28415ca30 0x7fb2840c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:30.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb28bda9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb28415ce10 0x7fb284157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:30.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb28bda9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840ca330 con 0x7fb28415ca30 2024-03-23T09:51:30.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb28bda9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840ca610 con 0x7fb2840b85d0 2024-03-23T09:51:30.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb28bda9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840ca8f0 con 0x7fb28415ce10 2024-03-23T09:51:30.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb289344640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb28415ca30 0x7fb2840c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:30.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb289344640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb28415ca30 0x7fb2840c9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37616/0 (socket says 172.21.15.47:37616) 2024-03-23T09:51:30.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb289344640 1 -- 172.21.15.47:0/499839949 learned_addr learned my addr 172.21.15.47:0/499839949 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:30.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb289b45640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2840b85d0 0x7fb28414afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:30.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb28a346640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb28415ce10 0x7fb284157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:30.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb289344640 1 -- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb28415ce10 msgr2=0x7fb284157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb289344640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb28415ce10 0x7fb284157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb289344640 1 -- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2840b85d0 msgr2=0x7fb28414afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb289344640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2840b85d0 0x7fb28414afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.112+0000 7fb289344640 1 -- 172.21.15.47:0/499839949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb274009b30 con 0x7fb28415ca30 2024-03-23T09:51:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.113+0000 7fb289344640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb28415ca30 0x7fb2840c9a30 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fb27800d5f0 tx=0x7fb27800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.113+0000 7fb2827fc640 1 -- 172.21.15.47:0/499839949 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2780125f0 con 0x7fb28415ca30 2024-03-23T09:51:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.113+0000 7fb2827fc640 1 -- 172.21.15.47:0/499839949 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb278012750 con 0x7fb28415ca30 2024-03-23T09:51:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.113+0000 7fb2827fc640 1 -- 172.21.15.47:0/499839949 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278012b00 con 0x7fb28415ca30 2024-03-23T09:51:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.113+0000 7fb28bda9640 1 -- 172.21.15.47:0/499839949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb284152670 con 0x7fb28415ca30 2024-03-23T09:51:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.113+0000 7fb28bda9640 1 -- 172.21.15.47:0/499839949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2841f3210 con 0x7fb28415ca30 2024-03-23T09:51:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.114+0000 7fb28bda9640 1 -- 172.21.15.47:0/499839949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb284072360 con 0x7fb28415ca30 2024-03-23T09:51:30.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.114+0000 7fb2827fc640 1 -- 172.21.15.47:0/499839949 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb278012c60 con 0x7fb28415ca30 2024-03-23T09:51:30.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.115+0000 7fb2827fc640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb27006ac00 0x7fb27006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:30.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.115+0000 7fb2827fc640 1 -- 172.21.15.47:0/499839949 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fb2780898f0 con 0x7fb28415ca30 2024-03-23T09:51:30.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.115+0000 7fb289b45640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb27006ac00 0x7fb27006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:30.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.115+0000 7fb289b45640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb27006ac00 0x7fb27006d0c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb28414b9a0 tx=0x7fb27c007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:30.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.119+0000 7fb2827fc640 1 -- 172.21.15.47:0/499839949 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb278054980 con 0x7fb28415ca30 2024-03-23T09:51:30.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.335+0000 7fb28bda9640 1 -- 172.21.15.47:0/499839949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb2840d0d90 con 0x7fb28415ca30 2024-03-23T09:51:30.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.984+0000 7fb2827fc640 1 -- 172.21.15.47:0/499839949 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb278057fa0 con 0x7fb28415ca30 2024-03-23T09:51:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.986+0000 7fb28bda9640 1 -- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb27006ac00 msgr2=0x7fb27006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.986+0000 7fb28bda9640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb27006ac00 0x7fb27006d0c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb28414b9a0 tx=0x7fb27c007e30 comp rx=0 tx=0).stop 2024-03-23T09:51:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.986+0000 7fb28bda9640 1 -- 172.21.15.47:0/499839949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb28415ca30 msgr2=0x7fb2840c9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.986+0000 7fb28bda9640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb28415ca30 0x7fb2840c9a30 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fb27800d5f0 tx=0x7fb27800dab0 comp rx=0 tx=0).stop 2024-03-23T09:51:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.986+0000 7fb28bda9640 1 -- 172.21.15.47:0/499839949 shutdown_connections 2024-03-23T09:51:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.986+0000 7fb28bda9640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb27006ac00 0x7fb27006d0c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.986+0000 7fb28bda9640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb28415ce10 0x7fb284157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.986+0000 7fb28bda9640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb28415ca30 0x7fb2840c9a30 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.986+0000 7fb28bda9640 1 --2- 172.21.15.47:0/499839949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2840b85d0 0x7fb28414afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.986+0000 7fb28bda9640 1 -- 172.21.15.47:0/499839949 >> 172.21.15.47:0/499839949 conn(0x7fb28406b160 msgr2=0x7fb2841551c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.986+0000 7fb28bda9640 1 -- 172.21.15.47:0/499839949 shutdown_connections 2024-03-23T09:51:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:30.986+0000 7fb28bda9640 1 -- 172.21.15.47:0/499839949 wait complete. 2024-03-23T09:51:30.999 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T09:51:31.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.109+0000 7f254ca10640 1 Processor -- start 2024-03-23T09:51:31.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.109+0000 7f254ca10640 1 -- start start 2024-03-23T09:51:31.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.110+0000 7f254ca10640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f254815a860 0x7f2548161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:31.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.110+0000 7f254ca10640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2548160bd0 con 0x7f254815d0f0 2024-03-23T09:51:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.110+0000 7f254ca10640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254815dbb0 con 0x7f25480cf8a0 2024-03-23T09:51:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.110+0000 7f254ca10640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25480d2790 con 0x7f254815a860 2024-03-23T09:51:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.110+0000 7f254659c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f254815a860 0x7f2548161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.110+0000 7f253ffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f254815d0f0 0x7f25480cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50964/0 (socket says 172.21.15.47:50964) 2024-03-23T09:51:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.110+0000 7f253ffff640 1 -- 172.21.15.47:0/1601014602 learned_addr learned my addr 172.21.15.47:0/1601014602 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.110+0000 7f254659c640 1 -- 172.21.15.47:0/1601014602 >> v1:172.21.15.175:6789/0 conn(0x7f25480cf8a0 legacy=0x7f25480d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.110+0000 7f254659c640 1 -- 172.21.15.47:0/1601014602 >> v1:172.21.15.47:6789/0 conn(0x7f254815d0f0 legacy=0x7f25480cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.110+0000 7f254659c640 1 -- 172.21.15.47:0/1601014602 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254815de90 con 0x7f254815a860 2024-03-23T09:51:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.110+0000 7f2545d9b640 1 -- 172.21.15.47:0/1601014602 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416425114 0 0) 0x7f2548160bd0 con 0x7f254815d0f0 2024-03-23T09:51:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.111+0000 7f254659c640 1 --2- 172.21.15.47:0/1601014602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254815a860 0x7f2548161060 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f25380097a0 tx=0x7f253802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed1130de530d25 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.111+0000 7f2545d9b640 1 -- 172.21.15.47:0/1601014602 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538031830 con 0x7f254815a860 2024-03-23T09:51:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.111+0000 7f2545d9b640 1 -- 172.21.15.47:0/1601014602 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2538031990 con 0x7f254815a860 2024-03-23T09:51:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.111+0000 7f2545d9b640 1 -- 172.21.15.47:0/1601014602 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538031d20 con 0x7f254815a860 2024-03-23T09:51:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.112+0000 7f254ca10640 1 -- 172.21.15.47:0/1601014602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254815a860 msgr2=0x7f2548161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.112+0000 7f254ca10640 1 --2- 172.21.15.47:0/1601014602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254815a860 0x7f2548161060 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f25380097a0 tx=0x7f253802b8d0 comp rx=0 tx=0).stop 2024-03-23T09:51:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.112+0000 7f254ca10640 1 -- 172.21.15.47:0/1601014602 shutdown_connections 2024-03-23T09:51:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.112+0000 7f254ca10640 1 --2- 172.21.15.47:0/1601014602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254815a860 0x7f2548161060 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.112+0000 7f254ca10640 1 -- 172.21.15.47:0/1601014602 >> 172.21.15.47:0/1601014602 conn(0x7f25480659d0 msgr2=0x7f2548065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.112+0000 7f254ca10640 1 -- 172.21.15.47:0/1601014602 shutdown_connections 2024-03-23T09:51:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.112+0000 7f254ca10640 1 -- 172.21.15.47:0/1601014602 wait complete. 2024-03-23T09:51:31.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.113+0000 7f254ca10640 1 Processor -- start 2024-03-23T09:51:31.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.113+0000 7f254ca10640 1 -- start start 2024-03-23T09:51:31.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f254ca10640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25480cf8a0 0x7f254814f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:31.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f254ca10640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f254815a860 0x7f25480c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f254ca10640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254815d0f0 0x7f25481f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f254ca10640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25481f4c30 con 0x7f254815d0f0 2024-03-23T09:51:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f254ca10640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25481f4020 con 0x7f25480cf8a0 2024-03-23T09:51:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f254ca10640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25481f4300 con 0x7f254815a860 2024-03-23T09:51:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f2546d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254815d0f0 0x7f25481f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f2546d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254815d0f0 0x7f25481f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37630/0 (socket says 172.21.15.47:37630) 2024-03-23T09:51:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f2546d9d640 1 -- 172.21.15.47:0/1443006463 learned_addr learned my addr 172.21.15.47:0/1443006463 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f254659c640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25480cf8a0 0x7f254814f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f253ffff640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f254815a860 0x7f25480c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f2546d9d640 1 -- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f254815a860 msgr2=0x7f25480c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f2546d9d640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f254815a860 0x7f25480c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f2546d9d640 1 -- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25480cf8a0 msgr2=0x7f254814f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f2546d9d640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25480cf8a0 0x7f254814f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f2546d9d640 1 -- 172.21.15.47:0/1443006463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2538009400 con 0x7f254815d0f0 2024-03-23T09:51:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f254659c640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25480cf8a0 0x7f254814f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f2546d9d640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254815d0f0 0x7f25481f50c0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f254000c780 tx=0x7f254000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f253effd640 1 -- 172.21.15.47:0/1443006463 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540004b60 con 0x7f254815d0f0 2024-03-23T09:51:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.114+0000 7f253effd640 1 -- 172.21.15.47:0/1443006463 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254000ce90 con 0x7f254815d0f0 2024-03-23T09:51:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.115+0000 7f253effd640 1 -- 172.21.15.47:0/1443006463 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000a850 con 0x7f254815d0f0 2024-03-23T09:51:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.115+0000 7f254ca10640 1 -- 172.21.15.47:0/1443006463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25481f45e0 con 0x7f254815d0f0 2024-03-23T09:51:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.115+0000 7f254ca10640 1 -- 172.21.15.47:0/1443006463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25481561d0 con 0x7f254815d0f0 2024-03-23T09:51:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.115+0000 7f254ca10640 1 -- 172.21.15.47:0/1443006463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2548072360 con 0x7f254815d0f0 2024-03-23T09:51:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.116+0000 7f253effd640 1 -- 172.21.15.47:0/1443006463 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f254000a9b0 con 0x7f254815d0f0 2024-03-23T09:51:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.116+0000 7f253effd640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f252806ac30 0x7f252806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.116+0000 7f253effd640 1 -- 172.21.15.47:0/1443006463 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f2540089620 con 0x7f254815d0f0 2024-03-23T09:51:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.117+0000 7f254659c640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f252806ac30 0x7f252806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.117+0000 7f254659c640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f252806ac30 0x7f252806d0f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2538036000 tx=0x7f2538008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.121+0000 7f253effd640 1 -- 172.21.15.47:0/1443006463 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2540054740 con 0x7f254815d0f0 2024-03-23T09:51:31.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.300+0000 7f254ca10640 1 -- 172.21.15.47:0/1443006463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2548066be0 con 0x7f254815d0f0 2024-03-23T09:51:31.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.991+0000 7f253effd640 1 -- 172.21.15.47:0/1443006463 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2540057d60 con 0x7f254815d0f0 2024-03-23T09:51:31.991 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-03-23T09:51:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.995+0000 7f254ca10640 1 -- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f252806ac30 msgr2=0x7f252806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.995+0000 7f254ca10640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f252806ac30 0x7f252806d0f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2538036000 tx=0x7f2538008030 comp rx=0 tx=0).stop 2024-03-23T09:51:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.995+0000 7f254ca10640 1 -- 172.21.15.47:0/1443006463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254815d0f0 msgr2=0x7f25481f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.995+0000 7f254ca10640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254815d0f0 0x7f25481f50c0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f254000c780 tx=0x7f254000cc40 comp rx=0 tx=0).stop 2024-03-23T09:51:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.995+0000 7f254ca10640 1 -- 172.21.15.47:0/1443006463 shutdown_connections 2024-03-23T09:51:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.995+0000 7f254ca10640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f252806ac30 0x7f252806d0f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.995+0000 7f254ca10640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254815d0f0 0x7f25481f50c0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.995+0000 7f254ca10640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f254815a860 0x7f25480c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.995+0000 7f254ca10640 1 --2- 172.21.15.47:0/1443006463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25480cf8a0 0x7f254814f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.995+0000 7f254ca10640 1 -- 172.21.15.47:0/1443006463 >> 172.21.15.47:0/1443006463 conn(0x7f25480659d0 msgr2=0x7f25480d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.995+0000 7f254ca10640 1 -- 172.21.15.47:0/1443006463 shutdown_connections 2024-03-23T09:51:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:31.995+0000 7f254ca10640 1 -- 172.21.15.47:0/1443006463 wait complete. 2024-03-23T09:51:32.009 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T09:51:32.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.120+0000 7f89e37e7640 1 Processor -- start 2024-03-23T09:51:32.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.120+0000 7f89e37e7640 1 -- start start 2024-03-23T09:51:32.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.121+0000 7f89e37e7640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f89dc0cde40 0x7f89dc0b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:32.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.121+0000 7f89e37e7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f89dc160780 0x7f89dc0d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:32.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.121+0000 7f89e37e7640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89dc0b5a90 con 0x7f89dc15cb50 2024-03-23T09:51:32.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.121+0000 7f89e37e7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89dc161790 con 0x7f89dc160780 2024-03-23T09:51:32.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.121+0000 7f89e37e7640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89dc160ef0 con 0x7f89dc0cde40 2024-03-23T09:51:32.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.121+0000 7f89e1d84640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f89dc160780 0x7f89dc0d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:32.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.121+0000 7f89e1583640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f89dc15cb50 0x7f89dc0cd730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50976/0 (socket says 172.21.15.47:50976) 2024-03-23T09:51:32.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.121+0000 7f89e1583640 1 -- 172.21.15.47:0/2930491466 learned_addr learned my addr 172.21.15.47:0/2930491466 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.122+0000 7f89e1d84640 1 -- 172.21.15.47:0/2930491466 >> v2:172.21.15.175:3300/0 conn(0x7f89dc0cde40 msgr2=0x7f89dc0b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.122+0000 7f89e0d82640 1 --2- 172.21.15.47:0/2930491466 >> v2:172.21.15.175:3300/0 conn(0x7f89dc0cde40 0x7f89dc0b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.122+0000 7f89dbfff640 1 -- 172.21.15.47:0/2930491466 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892418774 0 0) 0x7f89dc0b5a90 con 0x7f89dc15cb50 2024-03-23T09:51:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.122+0000 7f89e1d84640 1 --2- 172.21.15.47:0/2930491466 >> v2:172.21.15.175:3300/0 conn(0x7f89dc0cde40 0x7f89dc0b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.122+0000 7f89e1d84640 1 -- 172.21.15.47:0/2930491466 >> v1:172.21.15.47:6789/0 conn(0x7f89dc15cb50 legacy=0x7f89dc0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.122+0000 7f89e1d84640 1 -- 172.21.15.47:0/2930491466 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc1611d0 con 0x7f89dc160780 2024-03-23T09:51:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.122+0000 7f89e0d82640 1 --2- 172.21.15.47:0/2930491466 >> v2:172.21.15.175:3300/0 conn(0x7f89dc0cde40 0x7f89dc0b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.122+0000 7f89e1d84640 1 --2- 172.21.15.47:0/2930491466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89dc160780 0x7f89dc0d1490 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0d39a0 tx=0x7f89d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0645c46d83dc3a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.122+0000 7f89dbfff640 1 -- 172.21.15.47:0/2930491466 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4031830 con 0x7f89dc160780 2024-03-23T09:51:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.122+0000 7f89dbfff640 1 -- 172.21.15.47:0/2930491466 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d4031990 con 0x7f89dc160780 2024-03-23T09:51:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.122+0000 7f89dbfff640 1 -- 172.21.15.47:0/2930491466 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4031d20 con 0x7f89dc160780 2024-03-23T09:51:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.123+0000 7f89e37e7640 1 -- 172.21.15.47:0/2930491466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89dc160780 msgr2=0x7f89dc0d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.123+0000 7f89e37e7640 1 --2- 172.21.15.47:0/2930491466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89dc160780 0x7f89dc0d1490 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0d39a0 tx=0x7f89d402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:51:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.123+0000 7f89e37e7640 1 -- 172.21.15.47:0/2930491466 shutdown_connections 2024-03-23T09:51:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.123+0000 7f89e37e7640 1 --2- 172.21.15.47:0/2930491466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89dc160780 0x7f89dc0d1490 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.123+0000 7f89e37e7640 1 --2- 172.21.15.47:0/2930491466 >> v2:172.21.15.175:3300/0 conn(0x7f89dc0cde40 0x7f89dc0b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.123+0000 7f89e37e7640 1 -- 172.21.15.47:0/2930491466 >> 172.21.15.47:0/2930491466 conn(0x7f89dc06b160 msgr2=0x7f89dc06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.123+0000 7f89e37e7640 1 -- 172.21.15.47:0/2930491466 shutdown_connections 2024-03-23T09:51:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.123+0000 7f89e37e7640 1 -- 172.21.15.47:0/2930491466 wait complete. 2024-03-23T09:51:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.124+0000 7f89e37e7640 1 Processor -- start 2024-03-23T09:51:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.124+0000 7f89e37e7640 1 -- start start 2024-03-23T09:51:32.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e37e7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89dc0cde40 0x7f89dc1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:32.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e37e7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89dc15cb50 0x7f89dc154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:32.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e37e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89dc160780 0x7f89dc151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:32.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e37e7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc154880 con 0x7f89dc160780 2024-03-23T09:51:32.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e37e7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc154b60 con 0x7f89dc15cb50 2024-03-23T09:51:32.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e37e7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc156570 con 0x7f89dc0cde40 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e1d84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89dc160780 0x7f89dc151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e1d84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89dc160780 0x7f89dc151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37650/0 (socket says 172.21.15.47:37650) 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e1d84640 1 -- 172.21.15.47:0/3963212463 learned_addr learned my addr 172.21.15.47:0/3963212463 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e0d82640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89dc15cb50 0x7f89dc154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e1583640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89dc0cde40 0x7f89dc1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e1d84640 1 -- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89dc0cde40 msgr2=0x7f89dc1580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e1d84640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89dc0cde40 0x7f89dc1580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e1d84640 1 -- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89dc15cb50 msgr2=0x7f89dc154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e1d84640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89dc15cb50 0x7f89dc154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e1d84640 1 -- 172.21.15.47:0/3963212463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d4009400 con 0x7f89dc160780 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.125+0000 7f89e1583640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89dc0cde40 0x7f89dc1580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.126+0000 7f89e1d84640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89dc160780 0x7f89dc151880 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f89d4034040 tx=0x7f89d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.126+0000 7f89d9ffb640 1 -- 172.21.15.47:0/3963212463 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d403a9e0 con 0x7f89dc160780 2024-03-23T09:51:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.126+0000 7f89d9ffb640 1 -- 172.21.15.47:0/3963212463 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d403ab40 con 0x7f89dc160780 2024-03-23T09:51:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.126+0000 7f89d9ffb640 1 -- 172.21.15.47:0/3963212463 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d40443f0 con 0x7f89dc160780 2024-03-23T09:51:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.126+0000 7f89e37e7640 1 -- 172.21.15.47:0/3963212463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89dc156850 con 0x7f89dc160780 2024-03-23T09:51:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.126+0000 7f89e37e7640 1 -- 172.21.15.47:0/3963212463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89dc156190 con 0x7f89dc160780 2024-03-23T09:51:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.127+0000 7f89e37e7640 1 -- 172.21.15.47:0/3963212463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89dc1563e0 con 0x7f89dc160780 2024-03-23T09:51:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.127+0000 7f89d9ffb640 1 -- 172.21.15.47:0/3963212463 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f89d4044550 con 0x7f89dc160780 2024-03-23T09:51:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.128+0000 7f89d9ffb640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f89c806ac00 0x7f89c806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.128+0000 7f89d9ffb640 1 -- 172.21.15.47:0/3963212463 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f89d40ab590 con 0x7f89dc160780 2024-03-23T09:51:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.128+0000 7f89e1583640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f89c806ac00 0x7f89c806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.129+0000 7f89e1583640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f89c806ac00 0x7f89c806d0c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0012e0 tx=0x7f89cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:32.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.132+0000 7f89d9ffb640 1 -- 172.21.15.47:0/3963212463 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f89d4076a20 con 0x7f89dc160780 2024-03-23T09:51:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.311+0000 7f89e37e7640 1 -- 172.21.15.47:0/3963212463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f89dc072360 con 0x7f89dc160780 2024-03-23T09:51:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-03-23T09:51:32.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:32.997+0000 7f89d9ffb640 1 -- 172.21.15.47:0/3963212463 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) v1 ==== 169+0+0 (secure 0 0 0) 0x7f89d407a040 con 0x7f89dc160780 2024-03-23T09:51:33.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.002+0000 7f89e37e7640 1 -- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f89c806ac00 msgr2=0x7f89c806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:33.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.002+0000 7f89e37e7640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f89c806ac00 0x7f89c806d0c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0012e0 tx=0x7f89cc009040 comp rx=0 tx=0).stop 2024-03-23T09:51:33.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.002+0000 7f89e37e7640 1 -- 172.21.15.47:0/3963212463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89dc160780 msgr2=0x7f89dc151880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:33.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.002+0000 7f89e37e7640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89dc160780 0x7f89dc151880 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f89d4034040 tx=0x7f89d40046c0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.002+0000 7f89e37e7640 1 -- 172.21.15.47:0/3963212463 shutdown_connections 2024-03-23T09:51:33.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.002+0000 7f89e37e7640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f89c806ac00 0x7f89c806d0c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.002+0000 7f89e37e7640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89dc160780 0x7f89dc151880 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.002+0000 7f89e37e7640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89dc15cb50 0x7f89dc154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.002+0000 7f89e37e7640 1 --2- 172.21.15.47:0/3963212463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89dc0cde40 0x7f89dc1580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.002+0000 7f89e37e7640 1 -- 172.21.15.47:0/3963212463 >> 172.21.15.47:0/3963212463 conn(0x7f89dc06b160 msgr2=0x7f89dc0cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:33.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.002+0000 7f89e37e7640 1 -- 172.21.15.47:0/3963212463 shutdown_connections 2024-03-23T09:51:33.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.002+0000 7f89e37e7640 1 -- 172.21.15.47:0/3963212463 wait complete. 2024-03-23T09:51:33.018 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-23T09:51:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.125+0000 7f8f365ea640 1 Processor -- start 2024-03-23T09:51:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.125+0000 7f8f365ea640 1 -- start start 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.125+0000 7f8f365ea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f3015cc30 0x7f8f300ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.125+0000 7f8f365ea640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8f300cd4b0 0x7f8f300b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.125+0000 7f8f365ea640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3015d010 con 0x7f8f300cad40 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.125+0000 7f8f365ea640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f300cd940 con 0x7f8f300cd4b0 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.125+0000 7f8f365ea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f300b13d0 con 0x7f8f3015cc30 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.126+0000 7f8f34b87640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8f300cd4b0 0x7f8f300b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.126+0000 7f8f34b87640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8f300cd4b0 0x7f8f300b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37658/0 (socket says 172.21.15.47:37658) 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.126+0000 7f8f34b87640 1 -- 172.21.15.47:0/878322946 learned_addr learned my addr 172.21.15.47:0/878322946 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.126+0000 7f8f2ffff640 1 --2- 172.21.15.47:0/878322946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f3015cc30 0x7f8f300ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.126+0000 7f8f34b87640 1 -- 172.21.15.47:0/878322946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f3015cc30 msgr2=0x7f8f300ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.126+0000 7f8f34b87640 1 --2- 172.21.15.47:0/878322946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f3015cc30 0x7f8f300ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.126+0000 7f8f34b87640 1 -- 172.21.15.47:0/878322946 >> v1:172.21.15.175:6789/0 conn(0x7f8f300cad40 legacy=0x7f8f300b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.126+0000 7f8f34b87640 1 -- 172.21.15.47:0/878322946 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f300b16b0 con 0x7f8f300cd4b0 2024-03-23T09:51:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.127+0000 7f8f34b87640 1 --2- 172.21.15.47:0/878322946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f300cd4b0 0x7f8f300b0d20 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009e50 tx=0x7f8f2802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db3d8cc9dedf261 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.127+0000 7f8f2effd640 1 -- 172.21.15.47:0/878322946 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2802ca70 con 0x7f8f300cd4b0 2024-03-23T09:51:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.127+0000 7f8f2effd640 1 -- 172.21.15.47:0/878322946 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2802cbd0 con 0x7f8f300cd4b0 2024-03-23T09:51:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.127+0000 7f8f2effd640 1 -- 172.21.15.47:0/878322946 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f280343f0 con 0x7f8f300cd4b0 2024-03-23T09:51:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.127+0000 7f8f365ea640 1 -- 172.21.15.47:0/878322946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f300cd4b0 msgr2=0x7f8f300b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.127+0000 7f8f365ea640 1 --2- 172.21.15.47:0/878322946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f300cd4b0 0x7f8f300b0d20 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009e50 tx=0x7f8f2802bed0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.127+0000 7f8f365ea640 1 -- 172.21.15.47:0/878322946 shutdown_connections 2024-03-23T09:51:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.127+0000 7f8f365ea640 1 --2- 172.21.15.47:0/878322946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f300cd4b0 0x7f8f300b0d20 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.127+0000 7f8f365ea640 1 --2- 172.21.15.47:0/878322946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f3015cc30 0x7f8f300ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.127+0000 7f8f365ea640 1 -- 172.21.15.47:0/878322946 >> 172.21.15.47:0/878322946 conn(0x7f8f300d06f0 msgr2=0x7f8f300d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.127+0000 7f8f365ea640 1 -- 172.21.15.47:0/878322946 shutdown_connections 2024-03-23T09:51:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.127+0000 7f8f365ea640 1 -- 172.21.15.47:0/878322946 wait complete. 2024-03-23T09:51:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.128+0000 7f8f365ea640 1 Processor -- start 2024-03-23T09:51:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.129+0000 7f8f365ea640 1 -- start start 2024-03-23T09:51:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.129+0000 7f8f365ea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f300cad40 0x7f8f301f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.129+0000 7f8f365ea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f300cd4b0 0x7f8f30158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.129+0000 7f8f365ea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f3015cc30 0x7f8f3006b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.129+0000 7f8f365ea640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f300c46b0 con 0x7f8f300cd4b0 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.129+0000 7f8f365ea640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f300c4990 con 0x7f8f3015cc30 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.129+0000 7f8f365ea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f300c4c70 con 0x7f8f300cad40 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.129+0000 7f8f2f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f300cd4b0 0x7f8f30158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.129+0000 7f8f2f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f300cd4b0 0x7f8f30158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37674/0 (socket says 172.21.15.47:37674) 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.129+0000 7f8f2f7fe640 1 -- 172.21.15.47:0/2297783542 learned_addr learned my addr 172.21.15.47:0/2297783542 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.130+0000 7f8f34b87640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f3015cc30 0x7f8f3006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.130+0000 7f8f2ffff640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f300cad40 0x7f8f301f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.130+0000 7f8f2f7fe640 1 -- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f300cad40 msgr2=0x7f8f301f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.130+0000 7f8f2f7fe640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f300cad40 0x7f8f301f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.130+0000 7f8f2f7fe640 1 -- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f3015cc30 msgr2=0x7f8f3006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.130+0000 7f8f2f7fe640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f3015cc30 0x7f8f3006b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.130+0000 7f8f2f7fe640 1 -- 172.21.15.47:0/2297783542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f28009b30 con 0x7f8f300cd4b0 2024-03-23T09:51:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.130+0000 7f8f2f7fe640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f300cd4b0 0x7f8f30158730 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800c430 tx=0x7f8f1800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.130+0000 7f8f2cff9640 1 -- 172.21.15.47:0/2297783542 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18011790 con 0x7f8f300cd4b0 2024-03-23T09:51:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.130+0000 7f8f2cff9640 1 -- 172.21.15.47:0/2297783542 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f180118f0 con 0x7f8f300cd4b0 2024-03-23T09:51:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.130+0000 7f8f365ea640 1 -- 172.21.15.47:0/2297783542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f300c4f50 con 0x7f8f300cd4b0 2024-03-23T09:51:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.130+0000 7f8f2cff9640 1 -- 172.21.15.47:0/2297783542 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18011ce0 con 0x7f8f300cd4b0 2024-03-23T09:51:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.130+0000 7f8f365ea640 1 -- 172.21.15.47:0/2297783542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f301f1dc0 con 0x7f8f300cd4b0 2024-03-23T09:51:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.131+0000 7f8f365ea640 1 -- 172.21.15.47:0/2297783542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f300c5470 con 0x7f8f300cd4b0 2024-03-23T09:51:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.132+0000 7f8f2cff9640 1 -- 172.21.15.47:0/2297783542 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8f1801c460 con 0x7f8f300cd4b0 2024-03-23T09:51:33.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.132+0000 7f8f2cff9640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8f1006ac30 0x7f8f1006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:33.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.132+0000 7f8f2cff9640 1 -- 172.21.15.47:0/2297783542 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f8f18088f30 con 0x7f8f300cd4b0 2024-03-23T09:51:33.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.132+0000 7f8f2ffff640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8f1006ac30 0x7f8f1006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:33.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.133+0000 7f8f2ffff640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8f1006ac30 0x7f8f1006d0f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8f200079a0 tx=0x7f8f20008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.136+0000 7f8f2cff9640 1 -- 172.21.15.47:0/2297783542 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8f300c5470 con 0x7f8f300cd4b0 2024-03-23T09:51:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.314+0000 7f8f365ea640 1 -- 172.21.15.47:0/2297783542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8f300c5470 con 0x7f8f300cd4b0 2024-03-23T09:51:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.318+0000 7f8f2cff9640 1 -- 172.21.15.47:0/2297783542 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8f180543d0 con 0x7f8f300cd4b0 2024-03-23T09:51:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.320+0000 7f8f365ea640 1 -- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8f1006ac30 msgr2=0x7f8f1006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.320+0000 7f8f365ea640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8f1006ac30 0x7f8f1006d0f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8f200079a0 tx=0x7f8f20008040 comp rx=0 tx=0).stop 2024-03-23T09:51:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.320+0000 7f8f365ea640 1 -- 172.21.15.47:0/2297783542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f300cd4b0 msgr2=0x7f8f30158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.320+0000 7f8f365ea640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f300cd4b0 0x7f8f30158730 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800c430 tx=0x7f8f1800c8f0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.320+0000 7f8f365ea640 1 -- 172.21.15.47:0/2297783542 shutdown_connections 2024-03-23T09:51:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.320+0000 7f8f365ea640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8f1006ac30 0x7f8f1006d0f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.320+0000 7f8f365ea640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f3015cc30 0x7f8f3006b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.320+0000 7f8f365ea640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f300cd4b0 0x7f8f30158730 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.320+0000 7f8f365ea640 1 --2- 172.21.15.47:0/2297783542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f300cad40 0x7f8f301f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.320+0000 7f8f365ea640 1 -- 172.21.15.47:0/2297783542 >> 172.21.15.47:0/2297783542 conn(0x7f8f300d06f0 msgr2=0x7f8f3015a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.320+0000 7f8f365ea640 1 -- 172.21.15.47:0/2297783542 shutdown_connections 2024-03-23T09:51:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.320+0000 7f8f365ea640 1 -- 172.21.15.47:0/2297783542 wait complete. 2024-03-23T09:51:33.331 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-03-23T09:51:33.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.432+0000 7f40351ce640 1 Processor -- start 2024-03-23T09:51:33.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.432+0000 7f40351ce640 1 -- start start 2024-03-23T09:51:33.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f40351ce640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40300cf210 0x7f40300cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:33.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f40351ce640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40300b5a90 0x7f40300b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:33.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f40351ce640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40300cfbd0 con 0x7f403015cd00 2024-03-23T09:51:33.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f40351ce640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403015d0e0 con 0x7f40300b5a90 2024-03-23T09:51:33.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f40351ce640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40300d1540 con 0x7f40300cf210 2024-03-23T09:51:33.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f402f59e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40300b5a90 0x7f40300b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:33.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f402ed9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f403015cd00 0x7f40300d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50978/0 (socket says 172.21.15.47:50978) 2024-03-23T09:51:33.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f402ed9d640 1 -- 172.21.15.47:0/1323164879 learned_addr learned my addr 172.21.15.47:0/1323164879 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:33.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f402f59e640 1 -- 172.21.15.47:0/1323164879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40300cf210 msgr2=0x7f40300cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:33.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f402e59c640 1 --2- 172.21.15.47:0/1323164879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40300cf210 0x7f40300cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:33.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f402f59e640 1 --2- 172.21.15.47:0/1323164879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40300cf210 0x7f40300cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f402f59e640 1 -- 172.21.15.47:0/1323164879 >> v1:172.21.15.47:6789/0 conn(0x7f403015cd00 legacy=0x7f40300d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:33.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f402dd9b640 1 -- 172.21.15.47:0/1323164879 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898654872 0 0) 0x7f40300cfbd0 con 0x7f403015cd00 2024-03-23T09:51:33.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f402f59e640 1 -- 172.21.15.47:0/1323164879 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40300d1820 con 0x7f40300b5a90 2024-03-23T09:51:33.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.433+0000 7f402e59c640 1 --2- 172.21.15.47:0/1323164879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40300cf210 0x7f40300cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:33.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.434+0000 7f402f59e640 1 --2- 172.21.15.47:0/1323164879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40300b5a90 0x7f40300b5f20 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f4028009d90 tx=0x7f402802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843d52e31b2671ce server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:33.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.434+0000 7f402dd9b640 1 -- 172.21.15.47:0/1323164879 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402802cbc0 con 0x7f40300b5a90 2024-03-23T09:51:33.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.434+0000 7f402dd9b640 1 -- 172.21.15.47:0/1323164879 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402802cd20 con 0x7f40300b5a90 2024-03-23T09:51:33.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.434+0000 7f402dd9b640 1 -- 172.21.15.47:0/1323164879 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028034660 con 0x7f40300b5a90 2024-03-23T09:51:33.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.434+0000 7f40351ce640 1 -- 172.21.15.47:0/1323164879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40300b5a90 msgr2=0x7f40300b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:33.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.434+0000 7f40351ce640 1 --2- 172.21.15.47:0/1323164879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40300b5a90 0x7f40300b5f20 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f4028009d90 tx=0x7f402802be10 comp rx=0 tx=0).stop 2024-03-23T09:51:33.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.434+0000 7f40351ce640 1 -- 172.21.15.47:0/1323164879 shutdown_connections 2024-03-23T09:51:33.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.434+0000 7f40351ce640 1 --2- 172.21.15.47:0/1323164879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40300b5a90 0x7f40300b5f20 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.434+0000 7f40351ce640 1 --2- 172.21.15.47:0/1323164879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40300cf210 0x7f40300cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.434+0000 7f40351ce640 1 -- 172.21.15.47:0/1323164879 >> 172.21.15.47:0/1323164879 conn(0x7f403006b160 msgr2=0x7f403006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:33.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.435+0000 7f40351ce640 1 -- 172.21.15.47:0/1323164879 shutdown_connections 2024-03-23T09:51:33.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.435+0000 7f40351ce640 1 -- 172.21.15.47:0/1323164879 wait complete. 2024-03-23T09:51:33.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.436+0000 7f40351ce640 1 Processor -- start 2024-03-23T09:51:33.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.436+0000 7f40351ce640 1 -- start start 2024-03-23T09:51:33.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f40351ce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40300b5a90 0x7f4030153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:33.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f40351ce640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40300cf210 0x7f40300c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f40351ce640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f403015cd00 0x7f403015b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f40351ce640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40301536d0 con 0x7f40300cf210 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f40351ce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300c6490 con 0x7f40300b5a90 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f40351ce640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40300c6770 con 0x7f403015cd00 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f402e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40300cf210 0x7f40300c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f402e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40300cf210 0x7f40300c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37704/0 (socket says 172.21.15.47:37704) 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f402e59c640 1 -- 172.21.15.47:0/3107690571 learned_addr learned my addr 172.21.15.47:0/3107690571 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f402f59e640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f403015cd00 0x7f403015b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f402ed9d640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40300b5a90 0x7f4030153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f402e59c640 1 -- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f403015cd00 msgr2=0x7f403015b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f402e59c640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f403015cd00 0x7f403015b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f402e59c640 1 -- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40300b5a90 msgr2=0x7f4030153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f402e59c640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40300b5a90 0x7f4030153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f402e59c640 1 -- 172.21.15.47:0/3107690571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4028009a70 con 0x7f40300cf210 2024-03-23T09:51:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f402e59c640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40300cf210 0x7f40300c7530 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f402400c7f0 tx=0x7f402400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:33.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f40177fe640 1 -- 172.21.15.47:0/3107690571 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024007a30 con 0x7f40300cf210 2024-03-23T09:51:33.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f40177fe640 1 -- 172.21.15.47:0/3107690571 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4024007b90 con 0x7f40300cf210 2024-03-23T09:51:33.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f40351ce640 1 -- 172.21.15.47:0/3107690571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40300c6a50 con 0x7f40300cf210 2024-03-23T09:51:33.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.437+0000 7f40177fe640 1 -- 172.21.15.47:0/3107690571 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402400ce40 con 0x7f40300cf210 2024-03-23T09:51:33.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.438+0000 7f40351ce640 1 -- 172.21.15.47:0/3107690571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40300c7010 con 0x7f40300cf210 2024-03-23T09:51:33.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.439+0000 7f40351ce640 1 -- 172.21.15.47:0/3107690571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4030072360 con 0x7f40300cf210 2024-03-23T09:51:33.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.439+0000 7f40177fe640 1 -- 172.21.15.47:0/3107690571 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f40240113f0 con 0x7f40300cf210 2024-03-23T09:51:33.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.439+0000 7f40177fe640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f401c06ac00 0x7f401c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:33.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.439+0000 7f40177fe640 1 -- 172.21.15.47:0/3107690571 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f4024089400 con 0x7f40300cf210 2024-03-23T09:51:33.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.440+0000 7f402ed9d640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f401c06ac00 0x7f401c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:33.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.440+0000 7f402ed9d640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f401c06ac00 0x7f401c06d0c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4020000f60 tx=0x7f4020007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:33.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.444+0000 7f40177fe640 1 -- 172.21.15.47:0/3107690571 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4024054760 con 0x7f40300cf210 2024-03-23T09:51:33.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.696+0000 7f40351ce640 1 -- 172.21.15.47:0/3107690571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f4030160510 con 0x7f40300cf210 2024-03-23T09:51:33.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:33.998+0000 7f40177fe640 1 -- 172.21.15.47:0/3107690571 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v466) v1 ==== 146+0+0 (secure 0 0 0) 0x7f402405e070 con 0x7f40300cf210 2024-03-23T09:51:34.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.000+0000 7f40351ce640 1 -- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f401c06ac00 msgr2=0x7f401c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:34.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.000+0000 7f40351ce640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f401c06ac00 0x7f401c06d0c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4020000f60 tx=0x7f4020007b50 comp rx=0 tx=0).stop 2024-03-23T09:51:34.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.000+0000 7f40351ce640 1 -- 172.21.15.47:0/3107690571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40300cf210 msgr2=0x7f40300c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:34.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.000+0000 7f40351ce640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40300cf210 0x7f40300c7530 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f402400c7f0 tx=0x7f402400cbf0 comp rx=0 tx=0).stop 2024-03-23T09:51:34.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.000+0000 7f40351ce640 1 -- 172.21.15.47:0/3107690571 shutdown_connections 2024-03-23T09:51:34.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.000+0000 7f40351ce640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f401c06ac00 0x7f401c06d0c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:34.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.000+0000 7f40351ce640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f403015cd00 0x7f403015b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:34.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.000+0000 7f40351ce640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40300cf210 0x7f40300c7530 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:34.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.000+0000 7f40351ce640 1 --2- 172.21.15.47:0/3107690571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40300b5a90 0x7f4030153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:34.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.000+0000 7f40351ce640 1 -- 172.21.15.47:0/3107690571 >> 172.21.15.47:0/3107690571 conn(0x7f403006b160 msgr2=0x7f40300b5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:34.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.000+0000 7f40351ce640 1 -- 172.21.15.47:0/3107690571 shutdown_connections 2024-03-23T09:51:34.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.000+0000 7f40351ce640 1 -- 172.21.15.47:0/3107690571 wait complete. 2024-03-23T09:51:34.013 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-23T09:51:34.013 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-23T09:51:34.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-03-23T09:51:34.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.115+0000 7fea3c219640 1 Processor -- start 2024-03-23T09:51:34.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.115+0000 7fea3c219640 1 -- start start 2024-03-23T09:51:34.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.116+0000 7fea3c219640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea3415cd00 0x7fea340d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:34.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.116+0000 7fea3c219640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea340d1020 0x7fea340cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:34.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.116+0000 7fea3c219640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3415d0e0 con 0x7fea340cf750 2024-03-23T09:51:34.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.116+0000 7fea3c219640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea340cfbe0 con 0x7fea3415cd00 2024-03-23T09:51:34.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.116+0000 7fea3c219640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea340d1540 con 0x7fea340d1020 2024-03-23T09:51:34.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.116+0000 7fea39fb5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea3415cd00 0x7fea340d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:34.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.116+0000 7fea39fb5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea3415cd00 0x7fea340d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37716/0 (socket says 172.21.15.47:37716) 2024-03-23T09:51:34.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.116+0000 7fea39fb5640 1 -- 172.21.15.47:0/484037340 learned_addr learned my addr 172.21.15.47:0/484037340 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:34.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.116+0000 7fea397b4640 1 --2- 172.21.15.47:0/484037340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea340d1020 0x7fea340cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:34.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.116+0000 7fea39fb5640 1 -- 172.21.15.47:0/484037340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea340d1020 msgr2=0x7fea340cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:34.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.116+0000 7fea39fb5640 1 --2- 172.21.15.47:0/484037340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea340d1020 0x7fea340cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:34.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.116+0000 7fea39fb5640 1 -- 172.21.15.47:0/484037340 >> v1:172.21.15.175:6789/0 conn(0x7fea340cf750 legacy=0x7fea340b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:34.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.116+0000 7fea39fb5640 1 -- 172.21.15.47:0/484037340 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea340d1820 con 0x7fea3415cd00 2024-03-23T09:51:34.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.117+0000 7fea39fb5640 1 --2- 172.21.15.47:0/484037340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415cd00 0x7fea340d0ae0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009870 tx=0x7fea2c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d0479c76da22d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:34.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.117+0000 7fea38fb3640 1 -- 172.21.15.47:0/484037340 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0317c0 con 0x7fea3415cd00 2024-03-23T09:51:34.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.117+0000 7fea38fb3640 1 -- 172.21.15.47:0/484037340 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c031920 con 0x7fea3415cd00 2024-03-23T09:51:34.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.117+0000 7fea38fb3640 1 -- 172.21.15.47:0/484037340 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c031cb0 con 0x7fea3415cd00 2024-03-23T09:51:34.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.118+0000 7fea3c219640 1 -- 172.21.15.47:0/484037340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415cd00 msgr2=0x7fea340d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:34.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.118+0000 7fea3c219640 1 --2- 172.21.15.47:0/484037340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415cd00 0x7fea340d0ae0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009870 tx=0x7fea2c02b860 comp rx=0 tx=0).stop 2024-03-23T09:51:34.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.118+0000 7fea3c219640 1 -- 172.21.15.47:0/484037340 shutdown_connections 2024-03-23T09:51:34.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.118+0000 7fea3c219640 1 --2- 172.21.15.47:0/484037340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea340d1020 0x7fea340cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:34.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.118+0000 7fea3c219640 1 --2- 172.21.15.47:0/484037340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415cd00 0x7fea340d0ae0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:34.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.118+0000 7fea3c219640 1 -- 172.21.15.47:0/484037340 >> 172.21.15.47:0/484037340 conn(0x7fea3406b160 msgr2=0x7fea3406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:34.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.118+0000 7fea3c219640 1 -- 172.21.15.47:0/484037340 shutdown_connections 2024-03-23T09:51:34.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.118+0000 7fea3c219640 1 -- 172.21.15.47:0/484037340 wait complete. 2024-03-23T09:51:34.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.119+0000 7fea3c219640 1 Processor -- start 2024-03-23T09:51:34.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.119+0000 7fea3c219640 1 -- start start 2024-03-23T09:51:34.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea3c219640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea340cf750 0x7fea34145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:34.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea3c219640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea340d1020 0x7fea340cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:34.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea3c219640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3415cd00 0x7fea340cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:34.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea3c219640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea340b3e90 con 0x7fea340d1020 2024-03-23T09:51:34.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea3c219640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea340b4cd0 con 0x7fea3415cd00 2024-03-23T09:51:34.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea3c219640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea341456d0 con 0x7fea340cf750 2024-03-23T09:51:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea397b4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea340d1020 0x7fea340cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea397b4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea340d1020 0x7fea340cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37726/0 (socket says 172.21.15.47:37726) 2024-03-23T09:51:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea397b4640 1 -- 172.21.15.47:0/3198723126 learned_addr learned my addr 172.21.15.47:0/3198723126 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea39fb5640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea340cf750 0x7fea34145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea3a7b6640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3415cd00 0x7fea340cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea397b4640 1 -- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea340cf750 msgr2=0x7fea34145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea397b4640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea340cf750 0x7fea34145190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea397b4640 1 -- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3415cd00 msgr2=0x7fea340cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea397b4640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3415cd00 0x7fea340cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea397b4640 1 -- 172.21.15.47:0/3198723126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c0094d0 con 0x7fea340d1020 2024-03-23T09:51:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea3a7b6640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3415cd00 0x7fea340cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.120+0000 7fea397b4640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea340d1020 0x7fea340cd270 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fea2800c7f0 tx=0x7fea2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:34.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.121+0000 7fea327fc640 1 -- 172.21.15.47:0/3198723126 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28004220 con 0x7fea340d1020 2024-03-23T09:51:34.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.121+0000 7fea327fc640 1 -- 172.21.15.47:0/3198723126 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea280043a0 con 0x7fea340d1020 2024-03-23T09:51:34.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.121+0000 7fea3c219640 1 -- 172.21.15.47:0/3198723126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea340b40c0 con 0x7fea340d1020 2024-03-23T09:51:34.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.121+0000 7fea327fc640 1 -- 172.21.15.47:0/3198723126 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28007860 con 0x7fea340d1020 2024-03-23T09:51:34.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.121+0000 7fea3c219640 1 -- 172.21.15.47:0/3198723126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea340b4680 con 0x7fea340d1020 2024-03-23T09:51:34.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.122+0000 7fea3c219640 1 -- 172.21.15.47:0/3198723126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea34072360 con 0x7fea340d1020 2024-03-23T09:51:34.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.122+0000 7fea327fc640 1 -- 172.21.15.47:0/3198723126 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fea280079c0 con 0x7fea340d1020 2024-03-23T09:51:34.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.122+0000 7fea327fc640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fea2006ac00 0x7fea2006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:34.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.122+0000 7fea327fc640 1 -- 172.21.15.47:0/3198723126 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fea28088da0 con 0x7fea340d1020 2024-03-23T09:51:34.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.123+0000 7fea39fb5640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fea2006ac00 0x7fea2006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:34.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.123+0000 7fea39fb5640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fea2006ac00 0x7fea2006d0c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fea2c036000 tx=0x7fea2c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:34.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.127+0000 7fea327fc640 1 -- 172.21.15.47:0/3198723126 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fea280bc580 con 0x7fea340d1020 2024-03-23T09:51:34.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:34.379+0000 7fea3c219640 1 -- 172.21.15.47:0/3198723126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7fea34066be0 con 0x7fea340d1020 2024-03-23T09:51:35.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.002+0000 7fea327fc640 1 -- 172.21.15.47:0/3198723126 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v467) v1 ==== 147+0+0 (secure 0 0 0) 0x7fea2801e0d0 con 0x7fea340d1020 2024-03-23T09:51:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.004+0000 7fea3c219640 1 -- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fea2006ac00 msgr2=0x7fea2006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.004+0000 7fea3c219640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fea2006ac00 0x7fea2006d0c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fea2c036000 tx=0x7fea2c03d040 comp rx=0 tx=0).stop 2024-03-23T09:51:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.004+0000 7fea3c219640 1 -- 172.21.15.47:0/3198723126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea340d1020 msgr2=0x7fea340cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.004+0000 7fea3c219640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea340d1020 0x7fea340cd270 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fea2800c7f0 tx=0x7fea2800ccb0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.004+0000 7fea3c219640 1 -- 172.21.15.47:0/3198723126 shutdown_connections 2024-03-23T09:51:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.004+0000 7fea3c219640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fea2006ac00 0x7fea2006d0c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.004+0000 7fea3c219640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3415cd00 0x7fea340cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.004+0000 7fea3c219640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea340d1020 0x7fea340cd270 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.004+0000 7fea3c219640 1 --2- 172.21.15.47:0/3198723126 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea340cf750 0x7fea34145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.004+0000 7fea3c219640 1 -- 172.21.15.47:0/3198723126 >> 172.21.15.47:0/3198723126 conn(0x7fea3406b160 msgr2=0x7fea340b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:35.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.004+0000 7fea3c219640 1 -- 172.21.15.47:0/3198723126 shutdown_connections 2024-03-23T09:51:35.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.004+0000 7fea3c219640 1 -- 172.21.15.47:0/3198723126 wait complete. 2024-03-23T09:51:35.018 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-23T09:51:35.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.130+0000 7fe8c2c77640 1 Processor -- start 2024-03-23T09:51:35.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.131+0000 7fe8c2c77640 1 -- start start 2024-03-23T09:51:35.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.131+0000 7fe8c2c77640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8bc0caf80 0x7fe8bc0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:35.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.131+0000 7fe8c2c77640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe8bc0cb360 0x7fe8bc161000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:35.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.131+0000 7fe8c2c77640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc0d0810 con 0x7fe8bc0b0810 2024-03-23T09:51:35.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.131+0000 7fe8c2c77640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc160b70 con 0x7fe8bc0cb360 2024-03-23T09:51:35.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.131+0000 7fe8c2c77640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc0cef80 con 0x7fe8bc0caf80 2024-03-23T09:51:35.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.131+0000 7fe8bbfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe8bc0cb360 0x7fe8bc161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:35.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.131+0000 7fe8c1214640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe8bc0b0810 0x7fe8bc0b0c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52340/0 (socket says 172.21.15.47:52340) 2024-03-23T09:51:35.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.131+0000 7fe8c1214640 1 -- 172.21.15.47:0/3962938984 learned_addr learned my addr 172.21.15.47:0/3962938984 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:35.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.132+0000 7fe8bbfff640 1 -- 172.21.15.47:0/3962938984 >> v2:172.21.15.175:3300/0 conn(0x7fe8bc0caf80 msgr2=0x7fe8bc0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:35.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.132+0000 7fe8bbfff640 1 --2- 172.21.15.47:0/3962938984 >> v2:172.21.15.175:3300/0 conn(0x7fe8bc0caf80 0x7fe8bc0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.132+0000 7fe8bbfff640 1 -- 172.21.15.47:0/3962938984 >> v1:172.21.15.47:6789/0 conn(0x7fe8bc0b0810 legacy=0x7fe8bc0b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.132+0000 7fe8bbfff640 1 -- 172.21.15.47:0/3962938984 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0cf260 con 0x7fe8bc0cb360 2024-03-23T09:51:35.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.132+0000 7fe8bbfff640 1 --2- 172.21.15.47:0/3962938984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8bc0cb360 0x7fe8bc161000 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0cd3a0 tx=0x7fe8b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9f4bf4f3ebaa0a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:35.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.133+0000 7fe8bb7fe640 1 -- 172.21.15.47:0/3962938984 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0031830 con 0x7fe8bc0cb360 2024-03-23T09:51:35.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.133+0000 7fe8bb7fe640 1 -- 172.21.15.47:0/3962938984 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b0031990 con 0x7fe8bc0cb360 2024-03-23T09:51:35.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.133+0000 7fe8bb7fe640 1 -- 172.21.15.47:0/3962938984 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0031d20 con 0x7fe8bc0cb360 2024-03-23T09:51:35.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.133+0000 7fe8c2c77640 1 -- 172.21.15.47:0/3962938984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8bc0cb360 msgr2=0x7fe8bc161000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.133+0000 7fe8c2c77640 1 --2- 172.21.15.47:0/3962938984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8bc0cb360 0x7fe8bc161000 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0cd3a0 tx=0x7fe8b002b8d0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.133+0000 7fe8c2c77640 1 -- 172.21.15.47:0/3962938984 shutdown_connections 2024-03-23T09:51:35.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.133+0000 7fe8c2c77640 1 --2- 172.21.15.47:0/3962938984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8bc0cb360 0x7fe8bc161000 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.133+0000 7fe8c2c77640 1 --2- 172.21.15.47:0/3962938984 >> v2:172.21.15.175:3300/0 conn(0x7fe8bc0caf80 0x7fe8bc0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.133+0000 7fe8c2c77640 1 -- 172.21.15.47:0/3962938984 >> 172.21.15.47:0/3962938984 conn(0x7fe8bc098a00 msgr2=0x7fe8bc0cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:35.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.133+0000 7fe8c2c77640 1 -- 172.21.15.47:0/3962938984 shutdown_connections 2024-03-23T09:51:35.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.133+0000 7fe8c2c77640 1 -- 172.21.15.47:0/3962938984 wait complete. 2024-03-23T09:51:35.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.134+0000 7fe8c2c77640 1 Processor -- start 2024-03-23T09:51:35.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.134+0000 7fe8c2c77640 1 -- start start 2024-03-23T09:51:35.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.135+0000 7fe8c2c77640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8bc0b0810 0x7fe8bc0c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:35.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.135+0000 7fe8c2c77640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8bc0caf80 0x7fe8bc0c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:35.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.135+0000 7fe8c2c77640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8bc0cb360 0x7fe8bc0adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:35.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.135+0000 7fe8c2c77640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0c6e90 con 0x7fe8bc0b0810 2024-03-23T09:51:35.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.135+0000 7fe8c2c77640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc1f2500 con 0x7fe8bc0cb360 2024-03-23T09:51:35.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.135+0000 7fe8c2c77640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc1f27e0 con 0x7fe8bc0caf80 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.135+0000 7fe8c0a13640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8bc0b0810 0x7fe8bc0c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.135+0000 7fe8c0a13640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8bc0b0810 0x7fe8bc0c6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54236/0 (socket says 172.21.15.47:54236) 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.135+0000 7fe8c0a13640 1 -- 172.21.15.47:0/1096285675 learned_addr learned my addr 172.21.15.47:0/1096285675 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.136+0000 7fe8c0a13640 1 -- 172.21.15.47:0/1096285675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8bc0caf80 msgr2=0x7fe8bc0c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.136+0000 7fe8c0a13640 1 --2- 172.21.15.47:0/1096285675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8bc0caf80 0x7fe8bc0c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.136+0000 7fe8c0a13640 1 -- 172.21.15.47:0/1096285675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8bc0cb360 msgr2=0x7fe8bc0adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.136+0000 7fe8c1214640 1 --2- 172.21.15.47:0/1096285675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8bc0cb360 0x7fe8bc0adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.136+0000 7fe8c0a13640 1 --2- 172.21.15.47:0/1096285675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8bc0cb360 0x7fe8bc0adf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.136+0000 7fe8c0a13640 1 -- 172.21.15.47:0/1096285675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b0009400 con 0x7fe8bc0b0810 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.136+0000 7fe8c0a13640 1 --2- 172.21.15.47:0/1096285675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8bc0b0810 0x7fe8bc0c6950 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400ba50 tx=0x7fe8b400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.136+0000 7fe8b97fa640 1 -- 172.21.15.47:0/1096285675 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b400cbe0 con 0x7fe8bc0b0810 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.136+0000 7fe8b97fa640 1 -- 172.21.15.47:0/1096285675 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b400cd60 con 0x7fe8bc0b0810 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.136+0000 7fe8c2c77640 1 -- 172.21.15.47:0/1096285675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc1f2ac0 con 0x7fe8bc0b0810 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.136+0000 7fe8b97fa640 1 -- 172.21.15.47:0/1096285675 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4014680 con 0x7fe8bc0b0810 2024-03-23T09:51:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.136+0000 7fe8c2c77640 1 -- 172.21.15.47:0/1096285675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc1f3080 con 0x7fe8bc0b0810 2024-03-23T09:51:35.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.137+0000 7fe8c2c77640 1 -- 172.21.15.47:0/1096285675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8bc1f5780 con 0x7fe8bc0b0810 2024-03-23T09:51:35.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.137+0000 7fe8b97fa640 1 -- 172.21.15.47:0/1096285675 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe8b401d3b0 con 0x7fe8bc0b0810 2024-03-23T09:51:35.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.138+0000 7fe8b97fa640 1 --2- 172.21.15.47:0/1096285675 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe8a806ac00 0x7fe8a806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:35.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.138+0000 7fe8b97fa640 1 -- 172.21.15.47:0/1096285675 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe8b4089060 con 0x7fe8bc0b0810 2024-03-23T09:51:35.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.138+0000 7fe8bbfff640 1 --2- 172.21.15.47:0/1096285675 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe8a806ac00 0x7fe8a806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:35.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.139+0000 7fe8bbfff640 1 --2- 172.21.15.47:0/1096285675 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe8a806ac00 0x7fe8a806d0c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0036000 tx=0x7fe8b0030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:35.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.142+0000 7fe8b97fa640 1 -- 172.21.15.47:0/1096285675 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe8bc1f5780 con 0x7fe8bc0b0810 2024-03-23T09:51:35.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.322+0000 7fe8c2c77640 1 -- 172.21.15.47:0/1096285675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe8bc1f5780 con 0x7fe8bc0b0810 2024-03-23T09:51:35.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.326+0000 7fe8b97fa640 1 -- 172.21.15.47:0/1096285675 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe8b40544e0 con 0x7fe8bc0b0810 2024-03-23T09:51:35.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.328+0000 7fe8c2c77640 1 -- 172.21.15.47:0/1096285675 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe8a806ac00 msgr2=0x7fe8a806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.328+0000 7fe8c2c77640 1 --2- 172.21.15.47:0/1096285675 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe8a806ac00 0x7fe8a806d0c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0036000 tx=0x7fe8b0030000 comp rx=0 tx=0).stop 2024-03-23T09:51:35.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.328+0000 7fe8c2c77640 1 -- 172.21.15.47:0/1096285675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8bc0b0810 msgr2=0x7fe8bc0c6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.328+0000 7fe8c2c77640 1 --2- 172.21.15.47:0/1096285675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8bc0b0810 0x7fe8bc0c6950 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400ba50 tx=0x7fe8b400bf10 comp rx=0 tx=0).stop 2024-03-23T09:51:35.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.328+0000 7fe8c2c77640 1 -- 172.21.15.47:0/1096285675 shutdown_connections 2024-03-23T09:51:35.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.328+0000 7fe8c2c77640 1 --2- 172.21.15.47:0/1096285675 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe8a806ac00 0x7fe8a806d0c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.328+0000 7fe8c2c77640 1 --2- 172.21.15.47:0/1096285675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8bc0cb360 0x7fe8bc0adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.328+0000 7fe8c2c77640 1 --2- 172.21.15.47:0/1096285675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8bc0caf80 0x7fe8bc0c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.328+0000 7fe8c2c77640 1 --2- 172.21.15.47:0/1096285675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8bc0b0810 0x7fe8bc0c6950 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.328+0000 7fe8c2c77640 1 -- 172.21.15.47:0/1096285675 >> 172.21.15.47:0/1096285675 conn(0x7fe8bc098a00 msgr2=0x7fe8bc0c9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:35.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.328+0000 7fe8c2c77640 1 -- 172.21.15.47:0/1096285675 shutdown_connections 2024-03-23T09:51:35.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.328+0000 7fe8c2c77640 1 -- 172.21.15.47:0/1096285675 wait complete. 2024-03-23T09:51:35.340 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:51:35.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:51:35.384 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:51:35.429 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:51:35.429 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:51:35.429 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:51:35.444 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:51:35.444 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:51:35.444 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:51:35.486 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:51:35.486 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:51:35.486 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:51:35.501 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:51:35.501 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:51:35.501 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:51:35.543 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:51:35.543 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-23T09:51:35.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:51:35.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.643+0000 7f6080377640 1 Processor -- start 2024-03-23T09:51:35.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.643+0000 7f6080377640 1 -- start start 2024-03-23T09:51:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.644+0000 7f6080377640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60780b3c30 0x7f607806d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.644+0000 7f6080377640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f607806c350 0x7f60780b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.644+0000 7f6080377640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60780b4550 con 0x7f6078142aa0 2024-03-23T09:51:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.644+0000 7f6080377640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607806c8c0 con 0x7f60780b3c30 2024-03-23T09:51:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.644+0000 7f6080377640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607806cba0 con 0x7f607806c350 2024-03-23T09:51:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.644+0000 7f607e113640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60780b3c30 0x7f607806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.644+0000 7f607e914640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6078142aa0 0x7f60780cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52352/0 (socket says 172.21.15.47:52352) 2024-03-23T09:51:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.644+0000 7f607e914640 1 -- 172.21.15.47:0/3282934333 learned_addr learned my addr 172.21.15.47:0/3282934333 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.644+0000 7f607d912640 1 --2- 172.21.15.47:0/3282934333 >> v2:172.21.15.175:3300/0 conn(0x7f607806c350 0x7f60780b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.644+0000 7f607e113640 1 -- 172.21.15.47:0/3282934333 >> v2:172.21.15.175:3300/0 conn(0x7f607806c350 msgr2=0x7f60780b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.644+0000 7f607e113640 1 --2- 172.21.15.47:0/3282934333 >> v2:172.21.15.175:3300/0 conn(0x7f607806c350 0x7f60780b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.644+0000 7f607e113640 1 -- 172.21.15.47:0/3282934333 >> v1:172.21.15.47:6789/0 conn(0x7f6078142aa0 legacy=0x7f60780cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.644+0000 7f607e113640 1 -- 172.21.15.47:0/3282934333 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607806ce80 con 0x7f60780b3c30 2024-03-23T09:51:35.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.645+0000 7f607e113640 1 --2- 172.21.15.47:0/3282934333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60780b3c30 0x7f607806d960 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f607806ab80 tx=0x7f607402e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f57171a003984 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:35.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.645+0000 7f607d111640 1 -- 172.21.15.47:0/3282934333 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607402ee50 con 0x7f60780b3c30 2024-03-23T09:51:35.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.645+0000 7f607d111640 1 -- 172.21.15.47:0/3282934333 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6074036440 con 0x7f60780b3c30 2024-03-23T09:51:35.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.645+0000 7f607d111640 1 -- 172.21.15.47:0/3282934333 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60740367d0 con 0x7f60780b3c30 2024-03-23T09:51:35.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.646+0000 7f6080377640 1 -- 172.21.15.47:0/3282934333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60780b3c30 msgr2=0x7f607806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.646+0000 7f6080377640 1 --2- 172.21.15.47:0/3282934333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60780b3c30 0x7f607806d960 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f607806ab80 tx=0x7f607402e0a0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.646+0000 7f6080377640 1 -- 172.21.15.47:0/3282934333 shutdown_connections 2024-03-23T09:51:35.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.646+0000 7f6080377640 1 --2- 172.21.15.47:0/3282934333 >> v2:172.21.15.175:3300/0 conn(0x7f607806c350 0x7f60780b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.646+0000 7f6080377640 1 --2- 172.21.15.47:0/3282934333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60780b3c30 0x7f607806d960 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.646+0000 7f6080377640 1 -- 172.21.15.47:0/3282934333 >> 172.21.15.47:0/3282934333 conn(0x7f607806a310 msgr2=0x7f60780b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:35.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.646+0000 7f6080377640 1 -- 172.21.15.47:0/3282934333 shutdown_connections 2024-03-23T09:51:35.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.646+0000 7f6080377640 1 -- 172.21.15.47:0/3282934333 wait complete. 2024-03-23T09:51:35.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.647+0000 7f6080377640 1 Processor -- start 2024-03-23T09:51:35.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.647+0000 7f6080377640 1 -- start start 2024-03-23T09:51:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f6080377640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f607806c350 0x7f607813b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f6080377640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60780b3c30 0x7f607813f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f6080377640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6078142aa0 0x7f60780ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f6080377640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607814dbd0 con 0x7f60780b3c30 2024-03-23T09:51:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f6080377640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607814cf50 con 0x7f6078142aa0 2024-03-23T09:51:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f6080377640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607814d230 con 0x7f607806c350 2024-03-23T09:51:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f607d912640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60780b3c30 0x7f607813f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f607d912640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60780b3c30 0x7f607813f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54262/0 (socket says 172.21.15.47:54262) 2024-03-23T09:51:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f607d912640 1 -- 172.21.15.47:0/2978042130 learned_addr learned my addr 172.21.15.47:0/2978042130 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f607e914640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6078142aa0 0x7f60780ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f607e113640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f607806c350 0x7f607813b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f607d912640 1 -- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f607806c350 msgr2=0x7f607813b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f607d912640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f607806c350 0x7f607813b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f607d912640 1 -- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6078142aa0 msgr2=0x7f60780ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f607d912640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6078142aa0 0x7f60780ac950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f607d912640 1 -- 172.21.15.47:0/2978042130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607400b6a0 con 0x7f60780b3c30 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f607e914640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6078142aa0 0x7f60780ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f607d912640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60780b3c30 0x7f607813f340 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f606800d5f0 tx=0x7f606800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f606effd640 1 -- 172.21.15.47:0/2978042130 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60680125f0 con 0x7f60780b3c30 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f606effd640 1 -- 172.21.15.47:0/2978042130 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6068012750 con 0x7f60780b3c30 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f6080377640 1 -- 172.21.15.47:0/2978042130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607814d510 con 0x7f60780b3c30 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f606effd640 1 -- 172.21.15.47:0/2978042130 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068012b00 con 0x7f60780b3c30 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.648+0000 7f6080377640 1 -- 172.21.15.47:0/2978042130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607814cc00 con 0x7f60780b3c30 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.649+0000 7f6080377640 1 -- 172.21.15.47:0/2978042130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6078073480 con 0x7f60780b3c30 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.650+0000 7f606effd640 1 -- 172.21.15.47:0/2978042130 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6068012c60 con 0x7f60780b3c30 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.650+0000 7f606effd640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f605c06ac00 0x7f605c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.650+0000 7f606effd640 1 -- 172.21.15.47:0/2978042130 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f6068089510 con 0x7f60780b3c30 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.650+0000 7f607e113640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f605c06ac00 0x7f605c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.651+0000 7f607e113640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f605c06ac00 0x7f605c06d0c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6074038000 tx=0x7f6074008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.654+0000 7f606effd640 1 -- 172.21.15.47:0/2978042130 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6068054a20 con 0x7f60780b3c30 2024-03-23T09:51:35.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.869+0000 7f6080377640 1 -- 172.21.15.47:0/2978042130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6078066be0 con 0x7f60780b3c30 2024-03-23T09:51:35.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.869+0000 7f606effd640 1 -- 172.21.15.47:0/2978042130 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f6068058040 con 0x7f60780b3c30 2024-03-23T09:51:35.871 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:35.871 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/249416109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":249416109},{"type":"v1","addr":"172.21.15.175:6837","nonce":249416109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3603846321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3603846321},{"type":"v1","addr":"172.21.15.175:6835","nonce":3603846321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1921094794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1921094794},{"type":"v1","addr":"172.21.15.47:6834","nonce":1921094794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":9078,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3030917369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":3030917369},{"type":"v1","addr":"172.21.15.47:6835","nonce":3030917369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[]} 2024-03-23T09:51:35.871 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 75 2024-03-23T09:51:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.871+0000 7f6080377640 1 -- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f605c06ac00 msgr2=0x7f605c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.871+0000 7f6080377640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f605c06ac00 0x7f605c06d0c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6074038000 tx=0x7f6074008040 comp rx=0 tx=0).stop 2024-03-23T09:51:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.871+0000 7f6080377640 1 -- 172.21.15.47:0/2978042130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60780b3c30 msgr2=0x7f607813f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.871+0000 7f6080377640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60780b3c30 0x7f607813f340 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f606800d5f0 tx=0x7f606800dab0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.871+0000 7f6080377640 1 -- 172.21.15.47:0/2978042130 shutdown_connections 2024-03-23T09:51:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.871+0000 7f6080377640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f605c06ac00 0x7f605c06d0c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.871+0000 7f6080377640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6078142aa0 0x7f60780ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.871+0000 7f6080377640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60780b3c30 0x7f607813f340 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.871+0000 7f6080377640 1 --2- 172.21.15.47:0/2978042130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f607806c350 0x7f607813b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.871+0000 7f6080377640 1 -- 172.21.15.47:0/2978042130 >> 172.21.15.47:0/2978042130 conn(0x7f607806a310 msgr2=0x7f607815a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.871+0000 7f6080377640 1 -- 172.21.15.47:0/2978042130 shutdown_connections 2024-03-23T09:51:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:35.871+0000 7f6080377640 1 -- 172.21.15.47:0/2978042130 wait complete. 2024-03-23T09:51:35.883 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-23T09:51:35.883 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-23T09:51:35.883 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-23T09:51:35.883 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:51:35.884 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-23T09:51:35.884 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-23T09:51:35.884 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-23T09:51:35.884 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:51:35.884 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-23T09:51:35.884 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-23T09:51:35.884 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-23T09:51:35.885 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:51:35.885 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-23T09:51:35.885 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-23T09:51:35.885 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-23T09:51:35.885 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:51:35.892 INFO:tasks.ceph.mds.a:Stopped 2024-03-23T09:51:35.893 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-23T09:51:35.893 INFO:tasks.ceph.mds.c:Stopped 2024-03-23T09:51:35.893 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-23T09:51:35.896 INFO:tasks.ceph.mds.b:Stopped 2024-03-23T09:51:35.896 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-23T09:51:35.896 INFO:tasks.ceph.mds.d:Stopped 2024-03-23T09:51:35.897 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-23T09:51:35.935 INFO:tasks.ceph.mds.a:Started 2024-03-23T09:51:35.935 DEBUG:teuthology.parallel:result is None 2024-03-23T09:51:35.936 INFO:tasks.ceph.mds.c:Started 2024-03-23T09:51:35.936 DEBUG:teuthology.parallel:result is None 2024-03-23T09:51:35.938 INFO:tasks.ceph.mds.b:Started 2024-03-23T09:51:35.938 DEBUG:teuthology.parallel:result is None 2024-03-23T09:51:35.939 INFO:tasks.ceph.mds.d:Started 2024-03-23T09:51:35.939 DEBUG:teuthology.parallel:result is None 2024-03-23T09:51:35.939 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-23T09:51:36.012 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-03-23T09:51:36.012 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:51:36.020 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-03-23T09:51:36.020 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-03-23T09:51:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.048+0000 7f566df1b640 1 Processor -- start 2024-03-23T09:51:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.048+0000 7f566df1b640 1 -- start start 2024-03-23T09:51:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.048+0000 7f566df1b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56680cf210 0x7f56680cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.048+0000 7f566df1b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56680cfbd0 con 0x7f566815cd00 2024-03-23T09:51:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f566df1b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566815d0e0 con 0x7f56680b5a90 2024-03-23T09:51:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f56677fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f566815cd00 0x7f56680d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52362/0 (socket says 172.21.15.47:52362) 2024-03-23T09:51:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f56677fe640 1 -- 172.21.15.47:0/3409917267 learned_addr learned my addr 172.21.15.47:0/3409917267 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f566df1b640 1 -- 172.21.15.47:0/3409917267 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56680d1540 con 0x7f56680cf210 2024-03-23T09:51:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685909721 0 0) 0x7f56680cfbd0 con 0x7f566815cd00 2024-03-23T09:51:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f565effd640 1 --2- 172.21.15.47:0/3409917267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56680cf210 0x7f56680cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5654003520 con 0x7f566815cd00 2024-03-23T09:51:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3263855795 0 0) 0x7f5654003520 con 0x7f566815cd00 2024-03-23T09:51:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56680cf210 msgr2=0x7f56680cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 --2- 172.21.15.47:0/3409917267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56680cf210 0x7f56680cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 >> v1:172.21.15.175:6789/0 conn(0x7f56680b5a90 legacy=0x7f56680b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56680d1820 con 0x7f566815cd00 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f5658001670 con 0x7f566815cd00 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 >> v1:172.21.15.47:6789/0 conn(0x7f566815cd00 legacy=0x7f56680d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 --2- 172.21.15.47:0/3409917267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56680cf210 0x7f56540039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 --2- 172.21.15.47:0/3409917267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5654003f20 0x7f56540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 --2- 172.21.15.47:0/3409917267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5654004d00 0x7f5654005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56680d1540 con 0x7f5654004d00 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565400bb10 con 0x7f5654003f20 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565400bcd0 con 0x7f56680cf210 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f5667fff640 1 --2- 172.21.15.47:0/3409917267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5654004d00 0x7f5654005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f565effd640 1 --2- 172.21.15.47:0/3409917267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5654003f20 0x7f56540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.049+0000 7f56677fe640 1 --2- 172.21.15.47:0/3409917267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56680cf210 0x7f56540039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.050+0000 7f5667fff640 1 -- 172.21.15.47:0/3409917267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56680cf210 msgr2=0x7f56540039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.050+0000 7f5667fff640 1 --2- 172.21.15.47:0/3409917267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56680cf210 0x7f56540039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.050+0000 7f5667fff640 1 -- 172.21.15.47:0/3409917267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5654003f20 msgr2=0x7f56540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.050+0000 7f5667fff640 1 --2- 172.21.15.47:0/3409917267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5654003f20 0x7f56540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.050+0000 7f5667fff640 1 -- 172.21.15.47:0/3409917267 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5654003520 con 0x7f5654004d00 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.050+0000 7f5667fff640 1 --2- 172.21.15.47:0/3409917267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5654004d00 0x7f5654005180 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f5660001e50 tx=0x7f56540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c1e5c0b8a090ec server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.050+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660003b70 con 0x7f5654004d00 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.050+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5660003cd0 con 0x7f5654004d00 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.050+0000 7f5666ffd640 1 -- 172.21.15.47:0/3409917267 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660005310 con 0x7f5654004d00 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.051+0000 7f566df1b640 1 -- 172.21.15.47:0/3409917267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5654004d00 msgr2=0x7f5654005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.051+0000 7f566df1b640 1 --2- 172.21.15.47:0/3409917267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5654004d00 0x7f5654005180 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f5660001e50 tx=0x7f56540057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.051+0000 7f5667fff640 1 -- 172.21.15.47:0/3409917267 reap_dead start 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.051+0000 7f566df1b640 1 -- 172.21.15.47:0/3409917267 shutdown_connections 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.051+0000 7f566df1b640 1 -- 172.21.15.47:0/3409917267 >> 172.21.15.47:0/3409917267 conn(0x7f566806b160 msgr2=0x7f566806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.051+0000 7f566df1b640 1 -- 172.21.15.47:0/3409917267 shutdown_connections 2024-03-23T09:51:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.051+0000 7f566df1b640 1 -- 172.21.15.47:0/3409917267 wait complete. 2024-03-23T09:51:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.052+0000 7f566df1b640 1 Processor -- start 2024-03-23T09:51:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.052+0000 7f566df1b640 1 -- start start 2024-03-23T09:51:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.053+0000 7f566df1b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56680c8370 0x7f56680b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.053+0000 7f566df1b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56680c8750 0x7f56680b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.053+0000 7f566df1b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b20e0 0x7f56680b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.053+0000 7f566df1b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56680b3ed0 con 0x7f56680b20e0 2024-03-23T09:51:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.053+0000 7f566df1b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56680b2e30 con 0x7f56680c8750 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.053+0000 7f566df1b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56680b3110 con 0x7f56680c8370 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.053+0000 7f5667fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b20e0 0x7f56680b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.053+0000 7f5667fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b20e0 0x7f56680b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54300/0 (socket says 172.21.15.47:54300) 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.053+0000 7f5667fff640 1 -- 172.21.15.47:0/3899769857 learned_addr learned my addr 172.21.15.47:0/3899769857 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.054+0000 7f565effd640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56680c8750 0x7f56680b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.054+0000 7f56677fe640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56680c8370 0x7f56680b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.054+0000 7f5667fff640 1 -- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56680c8370 msgr2=0x7f56680b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.054+0000 7f5667fff640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56680c8370 0x7f56680b72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.054+0000 7f5667fff640 1 -- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56680c8750 msgr2=0x7f56680b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.054+0000 7f5667fff640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56680c8750 0x7f56680b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.054+0000 7f5667fff640 1 -- 172.21.15.47:0/3899769857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56680b33f0 con 0x7f56680b20e0 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.054+0000 7f5667fff640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b20e0 0x7f56680b2560 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f566002c060 tx=0x7f56600014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.054+0000 7f5664ff9640 1 -- 172.21.15.47:0/3899769857 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566003e070 con 0x7f56680b20e0 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.054+0000 7f5664ff9640 1 -- 172.21.15.47:0/3899769857 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5660007db0 con 0x7f56680b20e0 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.054+0000 7f566df1b640 1 -- 172.21.15.47:0/3899769857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56680b36d0 con 0x7f56680b20e0 2024-03-23T09:51:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.054+0000 7f5664ff9640 1 -- 172.21.15.47:0/3899769857 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660006d50 con 0x7f56680b20e0 2024-03-23T09:51:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.054+0000 7f566df1b640 1 -- 172.21.15.47:0/3899769857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56680b2aa0 con 0x7f56680b20e0 2024-03-23T09:51:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.055+0000 7f566df1b640 1 -- 172.21.15.47:0/3899769857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5668072360 con 0x7f56680b20e0 2024-03-23T09:51:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.056+0000 7f5664ff9640 1 -- 172.21.15.47:0/3899769857 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5660007030 con 0x7f56680b20e0 2024-03-23T09:51:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.056+0000 7f5664ff9640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f564806ac00 0x7f564806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.056+0000 7f5664ff9640 1 -- 172.21.15.47:0/3899769857 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f56600aa330 con 0x7f56680b20e0 2024-03-23T09:51:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.056+0000 7f56677fe640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f564806ac00 0x7f564806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.057+0000 7f56677fe640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f564806ac00 0x7f564806d0c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5658000c00 tx=0x7f5658000ef0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.060+0000 7f5664ff9640 1 -- 172.21.15.47:0/3899769857 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5660075760 con 0x7f56680b20e0 2024-03-23T09:51:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:36.256+0000 7f566df1b640 1 -- 172.21.15.47:0/3899769857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f56680b3bf0 con 0x7f56680b20e0 2024-03-23T09:51:37.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.045+0000 7f5664ff9640 1 -- 172.21.15.47:0/3899769857 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v110) v1 ==== 97+0+0 (secure 0 0 0) 0x7f56680b3bf0 con 0x7f56680b20e0 2024-03-23T09:51:37.047 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blocklist entries 2024-03-23T09:51:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.048+0000 7f566df1b640 1 -- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f564806ac00 msgr2=0x7f564806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.048+0000 7f566df1b640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f564806ac00 0x7f564806d0c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5658000c00 tx=0x7f5658000ef0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.048+0000 7f566df1b640 1 -- 172.21.15.47:0/3899769857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b20e0 msgr2=0x7f56680b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.048+0000 7f566df1b640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b20e0 0x7f56680b2560 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f566002c060 tx=0x7f56600014a0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.048+0000 7f566df1b640 1 -- 172.21.15.47:0/3899769857 shutdown_connections 2024-03-23T09:51:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.048+0000 7f566df1b640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f564806ac00 0x7f564806d0c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.048+0000 7f566df1b640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b20e0 0x7f56680b2560 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.048+0000 7f566df1b640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56680c8750 0x7f56680b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.048+0000 7f566df1b640 1 --2- 172.21.15.47:0/3899769857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56680c8370 0x7f56680b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.048+0000 7f566df1b640 1 -- 172.21.15.47:0/3899769857 >> 172.21.15.47:0/3899769857 conn(0x7f566806b160 msgr2=0x7f56680c8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.048+0000 7f566df1b640 1 -- 172.21.15.47:0/3899769857 shutdown_connections 2024-03-23T09:51:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.048+0000 7f566df1b640 1 -- 172.21.15.47:0/3899769857 wait complete. 2024-03-23T09:51:37.063 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-23T09:51:37.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.169+0000 7f45c8b55640 1 Processor -- start 2024-03-23T09:51:37.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.169+0000 7f45c8b55640 1 -- start start 2024-03-23T09:51:37.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.170+0000 7f45c8b55640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45c415e110 0x7f45c40b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.170+0000 7f45c8b55640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c40cf390 con 0x7f45c4099110 2024-03-23T09:51:37.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.170+0000 7f45c8b55640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c40cf670 con 0x7f45c40cee40 2024-03-23T09:51:37.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.170+0000 7f45c8b55640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c40cb4b0 con 0x7f45c415e110 2024-03-23T09:51:37.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.170+0000 7f45c259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45c415e110 0x7f45c40b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.170+0000 7f45c2d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f45c4099110 0x7f45c40cbf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52376/0 (socket says 172.21.15.47:52376) 2024-03-23T09:51:37.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.170+0000 7f45c2d9d640 1 -- 172.21.15.47:0/753874770 learned_addr learned my addr 172.21.15.47:0/753874770 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:37.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.170+0000 7f45c259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45c415e110 0x7f45c40b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54316/0 (socket says 172.21.15.47:54316) 2024-03-23T09:51:37.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.170+0000 7f45c259c640 1 -- 172.21.15.47:0/753874770 >> v1:172.21.15.175:6789/0 conn(0x7f45c40cee40 legacy=0x7f45c4098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:37.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.170+0000 7f45c259c640 1 -- 172.21.15.47:0/753874770 >> v1:172.21.15.47:6789/0 conn(0x7f45c4099110 legacy=0x7f45c40cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.170+0000 7f45c259c640 1 -- 172.21.15.47:0/753874770 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c40cb790 con 0x7f45c415e110 2024-03-23T09:51:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.171+0000 7f45c259c640 1 --2- 172.21.15.47:0/753874770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c415e110 0x7f45c40b1640 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f45b40097a0 tx=0x7f45b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0421fddc96f98c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.171+0000 7f45c159a640 1 -- 172.21.15.47:0/753874770 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4031830 con 0x7f45c415e110 2024-03-23T09:51:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.171+0000 7f45c159a640 1 -- 172.21.15.47:0/753874770 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b4031990 con 0x7f45c415e110 2024-03-23T09:51:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.171+0000 7f45c159a640 1 -- 172.21.15.47:0/753874770 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4031d20 con 0x7f45c415e110 2024-03-23T09:51:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.172+0000 7f45c8b55640 1 -- 172.21.15.47:0/753874770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c415e110 msgr2=0x7f45c40b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.172+0000 7f45c8b55640 1 --2- 172.21.15.47:0/753874770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c415e110 0x7f45c40b1640 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f45b40097a0 tx=0x7f45b402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.172+0000 7f45c8b55640 1 -- 172.21.15.47:0/753874770 shutdown_connections 2024-03-23T09:51:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.172+0000 7f45c8b55640 1 --2- 172.21.15.47:0/753874770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c415e110 0x7f45c40b1640 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.172+0000 7f45c8b55640 1 -- 172.21.15.47:0/753874770 >> 172.21.15.47:0/753874770 conn(0x7f45c40b2140 msgr2=0x7f45c40b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.172+0000 7f45c8b55640 1 -- 172.21.15.47:0/753874770 shutdown_connections 2024-03-23T09:51:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.172+0000 7f45c8b55640 1 -- 172.21.15.47:0/753874770 wait complete. 2024-03-23T09:51:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.173+0000 7f45c8b55640 1 Processor -- start 2024-03-23T09:51:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.173+0000 7f45c8b55640 1 -- start start 2024-03-23T09:51:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c8b55640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c4099110 0x7f45c40b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c8b55640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c40cee40 0x7f45c41e63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c8b55640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c415e110 0x7f45c4159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c8b55640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c41e71f0 con 0x7f45c4099110 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c8b55640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c41e6c30 con 0x7f45c40cee40 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c8b55640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c41e6f10 con 0x7f45c415e110 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c4099110 0x7f45c40b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c4099110 0x7f45c40b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54320/0 (socket says 172.21.15.47:54320) 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c259c640 1 -- 172.21.15.47:0/74876813 learned_addr learned my addr 172.21.15.47:0/74876813 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c1d9b640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c40cee40 0x7f45c41e63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c2d9d640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c415e110 0x7f45c4159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c259c640 1 -- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c415e110 msgr2=0x7f45c4159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c259c640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c415e110 0x7f45c4159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c259c640 1 -- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c40cee40 msgr2=0x7f45c41e63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c259c640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c40cee40 0x7f45c41e63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c259c640 1 -- 172.21.15.47:0/74876813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b4009400 con 0x7f45c4099110 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c1d9b640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c40cee40 0x7f45c41e63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c2d9d640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c415e110 0x7f45c4159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.174+0000 7f45c259c640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c4099110 0x7f45c40b5ca0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f45b4000c00 tx=0x7f45b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.175+0000 7f45b2ffd640 1 -- 172.21.15.47:0/74876813 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b403aa60 con 0x7f45c4099110 2024-03-23T09:51:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.175+0000 7f45b2ffd640 1 -- 172.21.15.47:0/74876813 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b403abc0 con 0x7f45c4099110 2024-03-23T09:51:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.175+0000 7f45c8b55640 1 -- 172.21.15.47:0/74876813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c41e3520 con 0x7f45c4099110 2024-03-23T09:51:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.175+0000 7f45b2ffd640 1 -- 172.21.15.47:0/74876813 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b40443f0 con 0x7f45c4099110 2024-03-23T09:51:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.175+0000 7f45c8b55640 1 -- 172.21.15.47:0/74876813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c41f0aa0 con 0x7f45c4099110 2024-03-23T09:51:37.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.176+0000 7f45c8b55640 1 -- 172.21.15.47:0/74876813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c41e3380 con 0x7f45c4099110 2024-03-23T09:51:37.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.176+0000 7f45b2ffd640 1 -- 172.21.15.47:0/74876813 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f45b404d450 con 0x7f45c4099110 2024-03-23T09:51:37.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.176+0000 7f45b2ffd640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f45a406ac00 0x7f45a406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.177+0000 7f45b2ffd640 1 -- 172.21.15.47:0/74876813 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f45b40ab1f0 con 0x7f45c4099110 2024-03-23T09:51:37.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.177+0000 7f45c1d9b640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f45a406ac00 0x7f45a406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.177+0000 7f45c1d9b640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f45a406ac00 0x7f45a406d0c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f45c409e5b0 tx=0x7f45b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:37.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.181+0000 7f45b2ffd640 1 -- 172.21.15.47:0/74876813 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f45c41e3380 con 0x7f45c4099110 2024-03-23T09:51:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.427+0000 7f45c8b55640 1 -- 172.21.15.47:0/74876813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f45c41e3a40 con 0x7f45c4099110 2024-03-23T09:51:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.427+0000 7f45b2ffd640 1 -- 172.21.15.47:0/74876813 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f45c41e3a40 con 0x7f45c4099110 2024-03-23T09:51:37.429 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:37.429 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:51:37.429 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2024-03-23T09:51:37.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.429 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2024-03-23T09:51:37.429 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lo+xoDhAAyxqr1H6eiINwt2X7LUlrSg==", 2024-03-23T09:51:37.429 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.429 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:51:37.429 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:51:37.429 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:51:37.429 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.429 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.430 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2024-03-23T09:51:37.430 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZ5OlGhAAuUElzRghNRFGKREdCpxKvA==", 2024-03-23T09:51:37.430 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.430 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:51:37.430 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:51:37.430 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:51:37.430 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.430 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lFwyzFRAAKldMOA7oSblA8YMrvntY8w==", 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.d", 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lpiD3IRAApXAdUQ4AkxWHzsFV0qcBng==", 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB7o/5lLvoADBAAgdixkm+jHwQilYw0IAn6EA==", 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB+o/5lOBfGCRAApglzalO1ztnJzOKg/UAyWA==", 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCBo/5lRBjlBxAAd45w8/kTMPqxmNyfCS3XYA==", 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCEo/5l+JxrBhAAbR/XqDyPDnN8eLpmvBZ7Mg==", 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCKo/5ltsqwDRAAkqcZ/sEf2oozWFN4AXg/+w==", 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.434 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCNo/5lpLHiCxAAB2axUv9gjcYVlij9f+AEHg==", 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2024-03-23T09:51:37.435 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCQo/5l2io3ChAAu+lfYcWWBQf43aYzWjCBNg==", 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCTo/5lA7jjCBAAjJgBkp89H8lx6fidl/Z7IA==", 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lZFc3JhAAZWmDZOnkW9eo9hqEDYynQw==", 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lIrJSLxAAPJer3ZP3h8VkImkiJmnKVw==", 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:51:37.437 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l1+DyKxAAZLKZNbUpUJ0fWPIYNvWK+g==", 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2024-03-23T09:51:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lWqkiNRAAL1q0lEcM/NKW3Pf9LcP21g==", 2024-03-23T09:51:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-03-23T09:51:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-03-23T09:51:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-03-23T09:51:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-03-23T09:51:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2024-03-23T09:51:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB4o/5lyeuKLRAARrfTpPc5BTVYZe/LUREDkg==", 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5l/0EOCxAAIrgIXRiFSQ1G7Jat5hNm4A==", 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:37.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:37.441 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2024-03-23T09:51:37.441 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB5o/5lywMFBxAAIVwjCe6B9EtBMAG2n7cZJw==", 2024-03-23T09:51:37.441 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-03-23T09:51:37.441 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-03-23T09:51:37.441 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-03-23T09:51:37.441 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-03-23T09:51:37.441 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.441 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:37.442 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-23T09:51:37.442 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:51:37.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.429+0000 7f45c8b55640 1 -- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f45a406ac00 msgr2=0x7f45a406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.429+0000 7f45c8b55640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f45a406ac00 0x7f45a406d0c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f45c409e5b0 tx=0x7f45b8009040 comp rx=0 tx=0).stop 2024-03-23T09:51:37.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.429+0000 7f45c8b55640 1 -- 172.21.15.47:0/74876813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c4099110 msgr2=0x7f45c40b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.429+0000 7f45c8b55640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c4099110 0x7f45c40b5ca0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f45b4000c00 tx=0x7f45b40046c0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.429+0000 7f45c8b55640 1 -- 172.21.15.47:0/74876813 shutdown_connections 2024-03-23T09:51:37.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.429+0000 7f45c8b55640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f45a406ac00 0x7f45a406d0c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.429+0000 7f45c8b55640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c415e110 0x7f45c4159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.429+0000 7f45c8b55640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c40cee40 0x7f45c41e63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.429+0000 7f45c8b55640 1 --2- 172.21.15.47:0/74876813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c4099110 0x7f45c40b5ca0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.429+0000 7f45c8b55640 1 -- 172.21.15.47:0/74876813 >> 172.21.15.47:0/74876813 conn(0x7f45c40b2140 msgr2=0x7f45c415d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:37.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.429+0000 7f45c8b55640 1 -- 172.21.15.47:0/74876813 shutdown_connections 2024-03-23T09:51:37.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.429+0000 7f45c8b55640 1 -- 172.21.15.47:0/74876813 wait complete. 2024-03-23T09:51:37.444 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:51:37.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.544+0000 7ff14bcaf640 1 Processor -- start 2024-03-23T09:51:37.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.544+0000 7ff14bcaf640 1 -- start start 2024-03-23T09:51:37.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.545+0000 7ff14bcaf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff14415ca90 0x7ff144160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.545+0000 7ff14bcaf640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff1440b8350 0x7ff1440cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.545+0000 7ff14bcaf640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1440ce090 con 0x7ff1440cdc40 2024-03-23T09:51:37.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.545+0000 7ff14bcaf640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1440d11e0 con 0x7ff14415ca90 2024-03-23T09:51:37.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.545+0000 7ff14bcaf640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1440d2680 con 0x7ff1440b8350 2024-03-23T09:51:37.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.545+0000 7ff149a4b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff14415ca90 0x7ff144160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.545+0000 7ff149a4b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff14415ca90 0x7ff144160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54334/0 (socket says 172.21.15.47:54334) 2024-03-23T09:51:37.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.545+0000 7ff149a4b640 1 -- 172.21.15.47:0/519152135 learned_addr learned my addr 172.21.15.47:0/519152135 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:37.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.545+0000 7ff14924a640 1 --2- 172.21.15.47:0/519152135 >> v2:172.21.15.175:3300/0 conn(0x7ff1440b8350 0x7ff1440cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.545+0000 7ff149a4b640 1 -- 172.21.15.47:0/519152135 >> v2:172.21.15.175:3300/0 conn(0x7ff1440b8350 msgr2=0x7ff1440cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:37.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.545+0000 7ff149a4b640 1 --2- 172.21.15.47:0/519152135 >> v2:172.21.15.175:3300/0 conn(0x7ff1440b8350 0x7ff1440cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.545+0000 7ff149a4b640 1 -- 172.21.15.47:0/519152135 >> v1:172.21.15.175:6789/0 conn(0x7ff1440cdc40 legacy=0x7ff1440b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:37.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.545+0000 7ff149a4b640 1 -- 172.21.15.47:0/519152135 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff144160f60 con 0x7ff14415ca90 2024-03-23T09:51:37.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.546+0000 7ff149a4b640 1 --2- 172.21.15.47:0/519152135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff14415ca90 0x7ff144160710 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7ff144160670 tx=0x7ff14002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f969ab112c59c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:37.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.546+0000 7ff148a49640 1 -- 172.21.15.47:0/519152135 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1400317e0 con 0x7ff14415ca90 2024-03-23T09:51:37.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.546+0000 7ff148a49640 1 -- 172.21.15.47:0/519152135 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff140031940 con 0x7ff14415ca90 2024-03-23T09:51:37.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.546+0000 7ff148a49640 1 -- 172.21.15.47:0/519152135 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff140031cd0 con 0x7ff14415ca90 2024-03-23T09:51:37.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.547+0000 7ff14bcaf640 1 -- 172.21.15.47:0/519152135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff14415ca90 msgr2=0x7ff144160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.547+0000 7ff14bcaf640 1 --2- 172.21.15.47:0/519152135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff14415ca90 0x7ff144160710 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7ff144160670 tx=0x7ff14002b880 comp rx=0 tx=0).stop 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.547+0000 7ff14bcaf640 1 -- 172.21.15.47:0/519152135 shutdown_connections 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.547+0000 7ff14bcaf640 1 --2- 172.21.15.47:0/519152135 >> v2:172.21.15.175:3300/0 conn(0x7ff1440b8350 0x7ff1440cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.547+0000 7ff14bcaf640 1 --2- 172.21.15.47:0/519152135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff14415ca90 0x7ff144160710 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.547+0000 7ff14bcaf640 1 -- 172.21.15.47:0/519152135 >> 172.21.15.47:0/519152135 conn(0x7ff1440659d0 msgr2=0x7ff144065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.547+0000 7ff14bcaf640 1 -- 172.21.15.47:0/519152135 shutdown_connections 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.547+0000 7ff14bcaf640 1 -- 172.21.15.47:0/519152135 wait complete. 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.548+0000 7ff14bcaf640 1 Processor -- start 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.548+0000 7ff14bcaf640 1 -- start start 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14bcaf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1440b8350 0x7ff1441528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14bcaf640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1440cdc40 0x7ff144158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14bcaf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff14415ca90 0x7ff14415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14bcaf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14415d010 con 0x7ff14415ca90 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14bcaf640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff144158eb0 con 0x7ff1440b8350 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14bcaf640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14415b730 con 0x7ff1440cdc40 2024-03-23T09:51:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14a24c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff14415ca90 0x7ff14415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14a24c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff14415ca90 0x7ff14415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54350/0 (socket says 172.21.15.47:54350) 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14a24c640 1 -- 172.21.15.47:0/3701625613 learned_addr learned my addr 172.21.15.47:0/3701625613 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14924a640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1440cdc40 0x7ff144158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff149a4b640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1440b8350 0x7ff1441528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14a24c640 1 -- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1440cdc40 msgr2=0x7ff144158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14a24c640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1440cdc40 0x7ff144158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14a24c640 1 -- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1440b8350 msgr2=0x7ff1441528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14a24c640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1440b8350 0x7ff1441528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14a24c640 1 -- 172.21.15.47:0/3701625613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1400094d0 con 0x7ff14415ca90 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14a24c640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff14415ca90 0x7ff14415b1f0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00c780 tx=0x7ff13c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff13a7fc640 1 -- 172.21.15.47:0/3701625613 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c004b60 con 0x7ff14415ca90 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff13a7fc640 1 -- 172.21.15.47:0/3701625613 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13c00ce90 con 0x7ff14415ca90 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff14bcaf640 1 -- 172.21.15.47:0/3701625613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14415b980 con 0x7ff14415ca90 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.549+0000 7ff13a7fc640 1 -- 172.21.15.47:0/3701625613 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c00a850 con 0x7ff14415ca90 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.550+0000 7ff14bcaf640 1 -- 172.21.15.47:0/3701625613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14415bf40 con 0x7ff14415ca90 2024-03-23T09:51:37.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.550+0000 7ff14bcaf640 1 -- 172.21.15.47:0/3701625613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff144072360 con 0x7ff14415ca90 2024-03-23T09:51:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.551+0000 7ff13a7fc640 1 -- 172.21.15.47:0/3701625613 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff13c00a9b0 con 0x7ff14415ca90 2024-03-23T09:51:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.551+0000 7ff13a7fc640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff12806ac30 0x7ff12806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.551+0000 7ff13a7fc640 1 -- 172.21.15.47:0/3701625613 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff13c0894e0 con 0x7ff14415ca90 2024-03-23T09:51:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.552+0000 7ff149a4b640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff12806ac30 0x7ff12806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.552+0000 7ff149a4b640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff12806ac30 0x7ff12806d0f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff140036000 tx=0x7ff14003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.556+0000 7ff13a7fc640 1 -- 172.21.15.47:0/3701625613 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff13c054970 con 0x7ff14415ca90 2024-03-23T09:51:37.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.736+0000 7ff14bcaf640 1 -- 172.21.15.47:0/3701625613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff14415d150 con 0x7ff14415ca90 2024-03-23T09:51:37.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.737+0000 7ff13a7fc640 1 -- 172.21.15.47:0/3701625613 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+12391 (secure 0 0 0) 0x7ff14415d150 con 0x7ff14415ca90 2024-03-23T09:51:37.738 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:37.739 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":110,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:51:37.041581+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:51:37.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.739+0000 7ff14bcaf640 1 -- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff12806ac30 msgr2=0x7ff12806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.739+0000 7ff14bcaf640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff12806ac30 0x7ff12806d0f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff140036000 tx=0x7ff14003d040 comp rx=0 tx=0).stop 2024-03-23T09:51:37.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.739+0000 7ff14bcaf640 1 -- 172.21.15.47:0/3701625613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff14415ca90 msgr2=0x7ff14415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.739+0000 7ff14bcaf640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff14415ca90 0x7ff14415b1f0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00c780 tx=0x7ff13c00cc40 comp rx=0 tx=0).stop 2024-03-23T09:51:37.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.739+0000 7ff14bcaf640 1 -- 172.21.15.47:0/3701625613 shutdown_connections 2024-03-23T09:51:37.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.739+0000 7ff14bcaf640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff12806ac30 0x7ff12806d0f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.739+0000 7ff14bcaf640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff14415ca90 0x7ff14415b1f0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.739+0000 7ff14bcaf640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1440cdc40 0x7ff144158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.739+0000 7ff14bcaf640 1 --2- 172.21.15.47:0/3701625613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1440b8350 0x7ff1441528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.739+0000 7ff14bcaf640 1 -- 172.21.15.47:0/3701625613 >> 172.21.15.47:0/3701625613 conn(0x7ff1440659d0 msgr2=0x7ff1440cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:37.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.739+0000 7ff14bcaf640 1 -- 172.21.15.47:0/3701625613 shutdown_connections 2024-03-23T09:51:37.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.739+0000 7ff14bcaf640 1 -- 172.21.15.47:0/3701625613 wait complete. 2024-03-23T09:51:37.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:51:37.752 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:51:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.854+0000 7f12ec520640 1 Processor -- start 2024-03-23T09:51:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.854+0000 7f12ec520640 1 -- start start 2024-03-23T09:51:37.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.855+0000 7f12ec520640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f12e415c990 0x7f12e4160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.855+0000 7f12ec520640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e4161630 con 0x7f12e40cde10 2024-03-23T09:51:37.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.855+0000 7f12ec520640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e40d1030 con 0x7f12e40b8350 2024-03-23T09:51:37.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.855+0000 7f12ec520640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e40cfe10 con 0x7f12e415c990 2024-03-23T09:51:37.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.855+0000 7f12eaabd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f12e40cde10 0x7f12e40d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52378/0 (socket says 172.21.15.47:52378) 2024-03-23T09:51:37.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.855+0000 7f12eaabd640 1 -- 172.21.15.47:0/1320006741 learned_addr learned my addr 172.21.15.47:0/1320006741 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:37.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.855+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140652912 0 0) 0x7f12e4161630 con 0x7f12e40cde10 2024-03-23T09:51:37.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.855+0000 7f12ea2bc640 1 --2- 172.21.15.47:0/1320006741 >> v2:172.21.15.175:3300/0 conn(0x7f12e415c990 0x7f12e4160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.855+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12cc003520 con 0x7f12e40cde10 2024-03-23T09:51:37.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.855+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2730467314 0 0) 0x7f12cc003520 con 0x7f12e40cde10 2024-03-23T09:51:37.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 >> v2:172.21.15.175:3300/0 conn(0x7f12e415c990 msgr2=0x7f12e4160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:37.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12e92ba640 1 --2- 172.21.15.47:0/1320006741 >> v2:172.21.15.175:3300/0 conn(0x7f12e415c990 0x7f12e4160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 >> v1:172.21.15.175:6789/0 conn(0x7f12e40b8350 legacy=0x7f12e40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e40b5b60 con 0x7f12e40cde10 2024-03-23T09:51:37.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f12d40019f0 con 0x7f12e40cde10 2024-03-23T09:51:37.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 >> v1:172.21.15.47:6789/0 conn(0x7f12e40cde10 legacy=0x7f12e40d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12e92ba640 1 --2- 172.21.15.47:0/1320006741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12cc0039c0 0x7f12cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12e92ba640 1 --2- 172.21.15.47:0/1320006741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12cc0047c0 0x7f12cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12e92ba640 1 --2- 172.21.15.47:0/1320006741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12cc0051d0 0x7f12cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e40cfe10 con 0x7f12cc0051d0 2024-03-23T09:51:37.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc00c120 con 0x7f12cc0039c0 2024-03-23T09:51:37.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc00c2e0 con 0x7f12cc0047c0 2024-03-23T09:51:37.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12eaabd640 1 --2- 172.21.15.47:0/1320006741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12cc0051d0 0x7f12cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12ea2bc640 1 --2- 172.21.15.47:0/1320006741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12cc0039c0 0x7f12cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12e9abb640 1 --2- 172.21.15.47:0/1320006741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12cc0047c0 0x7f12cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12eaabd640 1 -- 172.21.15.47:0/1320006741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12cc0047c0 msgr2=0x7f12cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12eaabd640 1 --2- 172.21.15.47:0/1320006741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12cc0047c0 0x7f12cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12eaabd640 1 -- 172.21.15.47:0/1320006741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12cc0039c0 msgr2=0x7f12cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12eaabd640 1 --2- 172.21.15.47:0/1320006741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12cc0039c0 0x7f12cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.856+0000 7f12eaabd640 1 -- 172.21.15.47:0/1320006741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc003520 con 0x7f12cc0051d0 2024-03-23T09:51:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.857+0000 7f12eaabd640 1 -- 172.21.15.47:0/1320006741 reap_dead start 2024-03-23T09:51:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.857+0000 7f12eaabd640 1 --2- 172.21.15.47:0/1320006741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12cc0051d0 0x7f12cc00b980 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f12d400a720 tx=0x7f12cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f40ca26f064cf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.857+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4034630 con 0x7f12cc0051d0 2024-03-23T09:51:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.857+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d402f070 con 0x7f12cc0051d0 2024-03-23T09:51:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.857+0000 7f12e92ba640 1 -- 172.21.15.47:0/1320006741 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d40349d0 con 0x7f12cc0051d0 2024-03-23T09:51:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.857+0000 7f12ec520640 1 -- 172.21.15.47:0/1320006741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12cc0051d0 msgr2=0x7f12cc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.857+0000 7f12ec520640 1 --2- 172.21.15.47:0/1320006741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12cc0051d0 0x7f12cc00b980 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f12d400a720 tx=0x7f12cc0057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.858+0000 7f12ec520640 1 -- 172.21.15.47:0/1320006741 shutdown_connections 2024-03-23T09:51:37.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.858+0000 7f12ec520640 1 --2- 172.21.15.47:0/1320006741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12cc0051d0 0x7f12cc00b980 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.858+0000 7f12ec520640 1 -- 172.21.15.47:0/1320006741 >> 172.21.15.47:0/1320006741 conn(0x7f12e40659d0 msgr2=0x7f12e4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:37.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.858+0000 7f12ec520640 1 -- 172.21.15.47:0/1320006741 shutdown_connections 2024-03-23T09:51:37.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.858+0000 7f12ec520640 1 -- 172.21.15.47:0/1320006741 wait complete. 2024-03-23T09:51:37.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.859+0000 7f12ec520640 1 Processor -- start 2024-03-23T09:51:37.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.859+0000 7f12ec520640 1 -- start start 2024-03-23T09:51:37.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.859+0000 7f12ec520640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12cc0051d0 0x7f12e4154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.859+0000 7f12ec520640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e40b6d40 0x7f12e40b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.859+0000 7f12ec520640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e41f5410 0x7f12e41f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.859+0000 7f12ec520640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e40b7700 con 0x7f12e40b6d40 2024-03-23T09:51:37.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.859+0000 7f12ec520640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e415aee0 con 0x7f12e41f5410 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.859+0000 7f12ec520640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e41f6370 con 0x7f12cc0051d0 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12e9abb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e40b6d40 0x7f12e40b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12e9abb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e40b6d40 0x7f12e40b71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54372/0 (socket says 172.21.15.47:54372) 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12e9abb640 1 -- 172.21.15.47:0/2790201547 learned_addr learned my addr 172.21.15.47:0/2790201547 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12eaabd640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e41f5410 0x7f12e41f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12ea2bc640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12cc0051d0 0x7f12e4154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12e9abb640 1 -- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12cc0051d0 msgr2=0x7f12e4154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12e9abb640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12cc0051d0 0x7f12e4154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12e9abb640 1 -- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e41f5410 msgr2=0x7f12e41f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12e9abb640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e41f5410 0x7f12e41f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12e9abb640 1 -- 172.21.15.47:0/2790201547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e40c6b30 con 0x7f12e40b6d40 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12ea2bc640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12cc0051d0 0x7f12e4154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12e9abb640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e40b6d40 0x7f12e40b71c0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f12d800f620 tx=0x7f12d800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12e2ffd640 1 -- 172.21.15.47:0/2790201547 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80052e0 con 0x7f12e40b6d40 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12e2ffd640 1 -- 172.21.15.47:0/2790201547 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d8005440 con 0x7f12e40b6d40 2024-03-23T09:51:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12e2ffd640 1 -- 172.21.15.47:0/2790201547 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8003830 con 0x7f12e40b6d40 2024-03-23T09:51:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12ec520640 1 -- 172.21.15.47:0/2790201547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e4154540 con 0x7f12e40b6d40 2024-03-23T09:51:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.860+0000 7f12ec520640 1 -- 172.21.15.47:0/2790201547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e41f6090 con 0x7f12e40b6d40 2024-03-23T09:51:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.861+0000 7f12ec520640 1 -- 172.21.15.47:0/2790201547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e4072360 con 0x7f12e40b6d40 2024-03-23T09:51:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.861+0000 7f12e2ffd640 1 -- 172.21.15.47:0/2790201547 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f12d8003990 con 0x7f12e40b6d40 2024-03-23T09:51:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.862+0000 7f12e2ffd640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f12d006ac30 0x7f12d006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.862+0000 7f12e2ffd640 1 -- 172.21.15.47:0/2790201547 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f12d8088e90 con 0x7f12e40b6d40 2024-03-23T09:51:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.862+0000 7f12ea2bc640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f12d006ac30 0x7f12d006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.862+0000 7f12ea2bc640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f12d006ac30 0x7f12d006d0f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0093d0 tx=0x7f12dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:37.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:37.866+0000 7f12e2ffd640 1 -- 172.21.15.47:0/2790201547 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f12d80543b0 con 0x7f12e40b6d40 2024-03-23T09:51:38.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.043+0000 7f12ec520640 1 -- 172.21.15.47:0/2790201547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f12e40c6cc0 con 0x7f12e40b6d40 2024-03-23T09:51:38.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.044+0000 7f12e2ffd640 1 -- 172.21.15.47:0/2790201547 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f12d80579d0 con 0x7f12e40b6d40 2024-03-23T09:51:38.045 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:51:38.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.045+0000 7f12ec520640 1 -- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f12d006ac30 msgr2=0x7f12d006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.045+0000 7f12ec520640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f12d006ac30 0x7f12d006d0f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0093d0 tx=0x7f12dc008040 comp rx=0 tx=0).stop 2024-03-23T09:51:38.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.045+0000 7f12ec520640 1 -- 172.21.15.47:0/2790201547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e40b6d40 msgr2=0x7f12e40b71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.045+0000 7f12ec520640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e40b6d40 0x7f12e40b71c0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f12d800f620 tx=0x7f12d800fae0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.045+0000 7f12ec520640 1 -- 172.21.15.47:0/2790201547 shutdown_connections 2024-03-23T09:51:38.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.045+0000 7f12ec520640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f12d006ac30 0x7f12d006d0f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.045+0000 7f12ec520640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e41f5410 0x7f12e41f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.045+0000 7f12ec520640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e40b6d40 0x7f12e40b71c0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.045+0000 7f12ec520640 1 --2- 172.21.15.47:0/2790201547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12cc0051d0 0x7f12e4154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.045+0000 7f12ec520640 1 -- 172.21.15.47:0/2790201547 >> 172.21.15.47:0/2790201547 conn(0x7f12e40659d0 msgr2=0x7f12e40b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:38.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.046+0000 7f12ec520640 1 -- 172.21.15.47:0/2790201547 shutdown_connections 2024-03-23T09:51:38.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.046+0000 7f12ec520640 1 -- 172.21.15.47:0/2790201547 wait complete. 2024-03-23T09:51:38.058 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T09:51:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.157+0000 7f661d5d2640 1 Processor -- start 2024-03-23T09:51:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.157+0000 7f661d5d2640 1 -- start start 2024-03-23T09:51:38.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.158+0000 7f661d5d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66180b0cb0 0x7f661815f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.158+0000 7f661d5d2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661806ac50 con 0x7f661815aa50 2024-03-23T09:51:38.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.158+0000 7f661d5d2640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661815e660 con 0x7f661806a810 2024-03-23T09:51:38.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.158+0000 7f661d5d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661815ec20 con 0x7f66180b0cb0 2024-03-23T09:51:38.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.158+0000 7f6616ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f661815aa50 0x7f66180cb1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52390/0 (socket says 172.21.15.47:52390) 2024-03-23T09:51:38.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.158+0000 7f6616ffd640 1 -- 172.21.15.47:0/2974552184 learned_addr learned my addr 172.21.15.47:0/2974552184 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:38.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f66177fe640 1 --2- 172.21.15.47:0/2974552184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66180b0cb0 0x7f661815f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026060942 0 0) 0x7f6608001490 con 0x7f661815aa50 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6604003520 con 0x7f661815aa50 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2642883168 0 0) 0x7f6604003520 con 0x7f661815aa50 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66180b0cb0 msgr2=0x7f661815f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 --2- 172.21.15.47:0/2974552184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66180b0cb0 0x7f661815f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 >> v1:172.21.15.175:6789/0 conn(0x7f661806a810 legacy=0x7f66180b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661815f5f0 con 0x7f661815aa50 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f6608001b80 con 0x7f661815aa50 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 >> v1:172.21.15.47:6789/0 conn(0x7f661815aa50 legacy=0x7f66180cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 --2- 172.21.15.47:0/2974552184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66040039a0 0x7f6604003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 --2- 172.21.15.47:0/2974552184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66180b0cb0 0x7f66040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 --2- 172.21.15.47:0/2974552184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604004d00 0x7f6604005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661815ec20 con 0x7f6604004d00 2024-03-23T09:51:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660400b9b0 con 0x7f66040039a0 2024-03-23T09:51:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660400bb70 con 0x7f66180b0cb0 2024-03-23T09:51:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f66177fe640 1 --2- 172.21.15.47:0/2974552184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604004d00 0x7f6604005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f6616ffd640 1 --2- 172.21.15.47:0/2974552184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66040039a0 0x7f6604003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f66167fc640 1 --2- 172.21.15.47:0/2974552184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66180b0cb0 0x7f66040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f66177fe640 1 -- 172.21.15.47:0/2974552184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66180b0cb0 msgr2=0x7f66040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f66177fe640 1 --2- 172.21.15.47:0/2974552184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66180b0cb0 0x7f66040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f66177fe640 1 -- 172.21.15.47:0/2974552184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66040039a0 msgr2=0x7f6604003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f66177fe640 1 --2- 172.21.15.47:0/2974552184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66040039a0 0x7f6604003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.159+0000 7f66177fe640 1 -- 172.21.15.47:0/2974552184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6604003520 con 0x7f6604004d00 2024-03-23T09:51:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.160+0000 7f66177fe640 1 --2- 172.21.15.47:0/2974552184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604004d00 0x7f6604005180 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f6610009ac0 tx=0x7f66040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae158f795ad0a1a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.160+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661002c6b0 con 0x7f6604004d00 2024-03-23T09:51:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.160+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661002c810 con 0x7f6604004d00 2024-03-23T09:51:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.160+0000 7f6615ffb640 1 -- 172.21.15.47:0/2974552184 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661002cba0 con 0x7f6604004d00 2024-03-23T09:51:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.161+0000 7f661d5d2640 1 -- 172.21.15.47:0/2974552184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604004d00 msgr2=0x7f6604005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.161+0000 7f661d5d2640 1 --2- 172.21.15.47:0/2974552184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604004d00 0x7f6604005180 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f6610009ac0 tx=0x7f66040057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.161+0000 7f66177fe640 1 -- 172.21.15.47:0/2974552184 reap_dead start 2024-03-23T09:51:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.161+0000 7f661d5d2640 1 -- 172.21.15.47:0/2974552184 shutdown_connections 2024-03-23T09:51:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.161+0000 7f661d5d2640 1 -- 172.21.15.47:0/2974552184 >> 172.21.15.47:0/2974552184 conn(0x7f6618098a00 msgr2=0x7f66180b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.161+0000 7f661d5d2640 1 -- 172.21.15.47:0/2974552184 shutdown_connections 2024-03-23T09:51:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.161+0000 7f661d5d2640 1 -- 172.21.15.47:0/2974552184 wait complete. 2024-03-23T09:51:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.162+0000 7f661d5d2640 1 Processor -- start 2024-03-23T09:51:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.162+0000 7f661d5d2640 1 -- start start 2024-03-23T09:51:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f661d5d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6604004d00 0x7f66180cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f661d5d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6618157e50 0x7f66180ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f661d5d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f661815c8a0 0x7f661815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f661d5d2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661815d260 con 0x7f6618157e50 2024-03-23T09:51:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f661d5d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66180c6f10 con 0x7f6604004d00 2024-03-23T09:51:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f661d5d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66180ce0f0 con 0x7f661815c8a0 2024-03-23T09:51:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f66167fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6618157e50 0x7f66180ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f66167fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6618157e50 0x7f66180ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54396/0 (socket says 172.21.15.47:54396) 2024-03-23T09:51:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f66167fc640 1 -- 172.21.15.47:0/3911011270 learned_addr learned my addr 172.21.15.47:0/3911011270 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f66177fe640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f661815c8a0 0x7f661815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f6616ffd640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6604004d00 0x7f66180cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f66167fc640 1 -- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f661815c8a0 msgr2=0x7f661815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f66167fc640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f661815c8a0 0x7f661815cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f66167fc640 1 -- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6604004d00 msgr2=0x7f66180cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f66167fc640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6604004d00 0x7f66180cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f66167fc640 1 -- 172.21.15.47:0/3911011270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180cf260 con 0x7f6618157e50 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f6616ffd640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6604004d00 0x7f66180cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.163+0000 7f66177fe640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f661815c8a0 0x7f661815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.164+0000 7f66167fc640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6618157e50 0x7f66180ced20 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f660000e800 tx=0x7f660000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.164+0000 7f660f7fe640 1 -- 172.21.15.47:0/3911011270 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600017070 con 0x7f6618157e50 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.164+0000 7f660f7fe640 1 -- 172.21.15.47:0/3911011270 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660000eeb0 con 0x7f6618157e50 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.164+0000 7f661d5d2640 1 -- 172.21.15.47:0/3911011270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66180cd850 con 0x7f6618157e50 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.164+0000 7f660f7fe640 1 -- 172.21.15.47:0/3911011270 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66000052e0 con 0x7f6618157e50 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.164+0000 7f661d5d2640 1 -- 172.21.15.47:0/3911011270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66180cde10 con 0x7f6618157e50 2024-03-23T09:51:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.165+0000 7f661d5d2640 1 -- 172.21.15.47:0/3911011270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66180672f0 con 0x7f6618157e50 2024-03-23T09:51:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.165+0000 7f660f7fe640 1 -- 172.21.15.47:0/3911011270 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f660000c6c0 con 0x7f6618157e50 2024-03-23T09:51:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.166+0000 7f660f7fe640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f65f806ac30 0x7f65f806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.166+0000 7f660f7fe640 1 -- 172.21.15.47:0/3911011270 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f66000887b0 con 0x7f6618157e50 2024-03-23T09:51:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.166+0000 7f6616ffd640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f65f806ac30 0x7f65f806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.166+0000 7f6616ffd640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f65f806ac30 0x7f65f806d0f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f66080013d0 tx=0x7f6608006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:38.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.170+0000 7f660f7fe640 1 -- 172.21.15.47:0/3911011270 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6600053c40 con 0x7f6618157e50 2024-03-23T09:51:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.351+0000 7f661d5d2640 1 -- 172.21.15.47:0/3911011270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f66180c9c80 con 0x7f6618157e50 2024-03-23T09:51:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.351+0000 7f660f7fe640 1 -- 172.21.15.47:0/3911011270 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v110) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6600057260 con 0x7f6618157e50 2024-03-23T09:51:38.352 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:38.352 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-23T09:51:38.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-23T09:51:38.352 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-03-23T09:51:38.353 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-03-23T09:51:38.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:38.353 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-23T09:51:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.353+0000 7f661d5d2640 1 -- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f65f806ac30 msgr2=0x7f65f806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.353+0000 7f661d5d2640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f65f806ac30 0x7f65f806d0f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f66080013d0 tx=0x7f6608006040 comp rx=0 tx=0).stop 2024-03-23T09:51:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.353+0000 7f661d5d2640 1 -- 172.21.15.47:0/3911011270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6618157e50 msgr2=0x7f66180ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.353+0000 7f661d5d2640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6618157e50 0x7f66180ced20 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f660000e800 tx=0x7f660000ecc0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.353+0000 7f661d5d2640 1 -- 172.21.15.47:0/3911011270 shutdown_connections 2024-03-23T09:51:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.353+0000 7f661d5d2640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f65f806ac30 0x7f65f806d0f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.353+0000 7f661d5d2640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f661815c8a0 0x7f661815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.353+0000 7f661d5d2640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6618157e50 0x7f66180ced20 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.353+0000 7f661d5d2640 1 --2- 172.21.15.47:0/3911011270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6604004d00 0x7f66180cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.353+0000 7f661d5d2640 1 -- 172.21.15.47:0/3911011270 >> 172.21.15.47:0/3911011270 conn(0x7f6618098a00 msgr2=0x7f6618069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.353+0000 7f661d5d2640 1 -- 172.21.15.47:0/3911011270 shutdown_connections 2024-03-23T09:51:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.353+0000 7f661d5d2640 1 -- 172.21.15.47:0/3911011270 wait complete. 2024-03-23T09:51:38.365 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T09:51:38.365 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T09:51:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.469+0000 7f2208cab640 1 Processor -- start 2024-03-23T09:51:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.469+0000 7f2208cab640 1 -- start start 2024-03-23T09:51:38.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.469+0000 7f2208cab640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f220415a860 0x7f2204161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.469+0000 7f2208cab640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2204160bd0 con 0x7f22040cf8a0 2024-03-23T09:51:38.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.469+0000 7f2208cab640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220415dbb0 con 0x7f220415d0f0 2024-03-23T09:51:38.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.469+0000 7f2208cab640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22040d2790 con 0x7f220415a860 2024-03-23T09:51:38.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2202d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f22040cf8a0 0x7f22040d0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52406/0 (socket says 172.21.15.47:52406) 2024-03-23T09:51:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2202d9d640 1 -- 172.21.15.47:0/1140870989 learned_addr learned my addr 172.21.15.47:0/1140870989 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f220259c640 1 --2- 172.21.15.47:0/1140870989 >> v2:172.21.15.175:3300/0 conn(0x7f220415a860 0x7f2204161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824651940 0 0) 0x7f2204160bd0 con 0x7f22040cf8a0 2024-03-23T09:51:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f0003520 con 0x7f22040cf8a0 2024-03-23T09:51:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2063377178 0 0) 0x7f21f0003520 con 0x7f22040cf8a0 2024-03-23T09:51:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 >> v2:172.21.15.175:3300/0 conn(0x7f220415a860 msgr2=0x7f2204161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 --2- 172.21.15.47:0/1140870989 >> v2:172.21.15.175:3300/0 conn(0x7f220415a860 0x7f2204161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 >> v1:172.21.15.175:6789/0 conn(0x7f220415d0f0 legacy=0x7f22040cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220415de90 con 0x7f22040cf8a0 2024-03-23T09:51:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f21fc0019f0 con 0x7f22040cf8a0 2024-03-23T09:51:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 >> v1:172.21.15.47:6789/0 conn(0x7f22040cf8a0 legacy=0x7f22040d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 --2- 172.21.15.47:0/1140870989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0003970 0x7f21f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 --2- 172.21.15.47:0/1140870989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21f00047c0 0x7f21f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 --2- 172.21.15.47:0/1140870989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21f0005180 0x7f21f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22040d2790 con 0x7f21f0005180 2024-03-23T09:51:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f000c020 con 0x7f21f00047c0 2024-03-23T09:51:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f000c1e0 con 0x7f21f0003970 2024-03-23T09:51:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f2202d9d640 1 --2- 172.21.15.47:0/1140870989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21f0005180 0x7f21f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f220259c640 1 --2- 172.21.15.47:0/1140870989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0003970 0x7f21f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.470+0000 7f21fbfff640 1 --2- 172.21.15.47:0/1140870989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21f00047c0 0x7f21f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.471+0000 7f2202d9d640 1 -- 172.21.15.47:0/1140870989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0003970 msgr2=0x7f21f0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.471+0000 7f2202d9d640 1 --2- 172.21.15.47:0/1140870989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0003970 0x7f21f0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.471+0000 7f2202d9d640 1 -- 172.21.15.47:0/1140870989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21f00047c0 msgr2=0x7f21f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.471+0000 7f2202d9d640 1 --2- 172.21.15.47:0/1140870989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21f00047c0 0x7f21f0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.471+0000 7f2202d9d640 1 -- 172.21.15.47:0/1140870989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f0003520 con 0x7f21f0005180 2024-03-23T09:51:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.471+0000 7f2202d9d640 1 -- 172.21.15.47:0/1140870989 reap_dead start 2024-03-23T09:51:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.471+0000 7f2202d9d640 1 --2- 172.21.15.47:0/1140870989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21f0005180 0x7f21f000b980 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00a830 tx=0x7f21f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebda0af4c0b9899 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.472+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0072d0 con 0x7f21f0005180 2024-03-23T09:51:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.472+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc02f070 con 0x7f21f0005180 2024-03-23T09:51:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.472+0000 7f2201d9b640 1 -- 172.21.15.47:0/1140870989 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0076b0 con 0x7f21f0005180 2024-03-23T09:51:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.472+0000 7f2208cab640 1 -- 172.21.15.47:0/1140870989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21f0005180 msgr2=0x7f21f000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.472+0000 7f2208cab640 1 --2- 172.21.15.47:0/1140870989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21f0005180 0x7f21f000b980 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00a830 tx=0x7f21f00057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.472+0000 7f2208cab640 1 -- 172.21.15.47:0/1140870989 shutdown_connections 2024-03-23T09:51:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.472+0000 7f2208cab640 1 --2- 172.21.15.47:0/1140870989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21f0005180 0x7f21f000b980 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.472+0000 7f2208cab640 1 -- 172.21.15.47:0/1140870989 >> 172.21.15.47:0/1140870989 conn(0x7f22040659d0 msgr2=0x7f2204065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.472+0000 7f2208cab640 1 -- 172.21.15.47:0/1140870989 shutdown_connections 2024-03-23T09:51:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.472+0000 7f2208cab640 1 -- 172.21.15.47:0/1140870989 wait complete. 2024-03-23T09:51:38.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.473+0000 7f2208cab640 1 Processor -- start 2024-03-23T09:51:38.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.473+0000 7f2208cab640 1 -- start start 2024-03-23T09:51:38.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.474+0000 7f2208cab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0005180 0x7f22041f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.474+0000 7f2208cab640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22041530f0 0x7f2204152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.474+0000 7f2208cab640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22040c8510 0x7f22040b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.474+0000 7f2208cab640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22041f60a0 con 0x7f22040c8510 2024-03-23T09:51:38.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.474+0000 7f2208cab640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22040c8e10 con 0x7f22041530f0 2024-03-23T09:51:38.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.474+0000 7f2208cab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22040c90f0 con 0x7f21f0005180 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.474+0000 7f2202d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22040c8510 0x7f22040b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.474+0000 7f2202d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22040c8510 0x7f22040b6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54408/0 (socket says 172.21.15.47:54408) 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.474+0000 7f2202d9d640 1 -- 172.21.15.47:0/3062018146 learned_addr learned my addr 172.21.15.47:0/3062018146 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.474+0000 7f21fbfff640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22041530f0 0x7f2204152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f220259c640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0005180 0x7f22041f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f2202d9d640 1 -- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0005180 msgr2=0x7f22041f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f2202d9d640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0005180 0x7f22041f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f2202d9d640 1 -- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22041530f0 msgr2=0x7f2204152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f2202d9d640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22041530f0 0x7f2204152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f2202d9d640 1 -- 172.21.15.47:0/3062018146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22041f5a50 con 0x7f22040c8510 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f21fbfff640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22041530f0 0x7f2204152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f220259c640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0005180 0x7f22041f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f2202d9d640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22040c8510 0x7f22040b6e30 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f21fc001580 tx=0x7f21fc001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f21faffd640 1 -- 172.21.15.47:0/3062018146 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc004e20 con 0x7f22040c8510 2024-03-23T09:51:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f21faffd640 1 -- 172.21.15.47:0/3062018146 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc0017b0 con 0x7f22040c8510 2024-03-23T09:51:38.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f2208cab640 1 -- 172.21.15.47:0/3062018146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22041f5d30 con 0x7f22040c8510 2024-03-23T09:51:38.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f21faffd640 1 -- 172.21.15.47:0/3062018146 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc001b20 con 0x7f22040c8510 2024-03-23T09:51:38.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.475+0000 7f2208cab640 1 -- 172.21.15.47:0/3062018146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22040c8b70 con 0x7f22040c8510 2024-03-23T09:51:38.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.476+0000 7f2208cab640 1 -- 172.21.15.47:0/3062018146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2204072360 con 0x7f22040c8510 2024-03-23T09:51:38.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.476+0000 7f21faffd640 1 -- 172.21.15.47:0/3062018146 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f21fc031030 con 0x7f22040c8510 2024-03-23T09:51:38.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.477+0000 7f21faffd640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f21e406ac00 0x7f21e406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:38.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.477+0000 7f21faffd640 1 -- 172.21.15.47:0/3062018146 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f21fc0aabb0 con 0x7f22040c8510 2024-03-23T09:51:38.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.477+0000 7f220259c640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f21e406ac00 0x7f21e406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:38.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.477+0000 7f220259c640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f21e406ac00 0x7f21e406d0c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009a40 tx=0x7f21f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:38.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.481+0000 7f21faffd640 1 -- 172.21.15.47:0/3062018146 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f21fc0760e0 con 0x7f22040c8510 2024-03-23T09:51:38.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:38.659+0000 7f2208cab640 1 -- 172.21.15.47:0/3062018146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2204072360 con 0x7f22040c8510 2024-03-23T09:51:38.965 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:51:38.968 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:51:39.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.015+0000 7f21faffd640 1 -- 172.21.15.47:0/3062018146 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v111) v1 ==== 138+0+0 (secure 0 0 0) 0x7f21fc079700 con 0x7f22040c8510 2024-03-23T09:51:39.017 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-03-23T09:51:39.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.020+0000 7f2208cab640 1 -- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f21e406ac00 msgr2=0x7f21e406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:39.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.020+0000 7f2208cab640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f21e406ac00 0x7f21e406d0c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009a40 tx=0x7f21f4008040 comp rx=0 tx=0).stop 2024-03-23T09:51:39.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.020+0000 7f2208cab640 1 -- 172.21.15.47:0/3062018146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22040c8510 msgr2=0x7f22040b6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:39.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.020+0000 7f2208cab640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22040c8510 0x7f22040b6e30 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f21fc001580 tx=0x7f21fc001d80 comp rx=0 tx=0).stop 2024-03-23T09:51:39.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.020+0000 7f2208cab640 1 -- 172.21.15.47:0/3062018146 shutdown_connections 2024-03-23T09:51:39.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.021+0000 7f2208cab640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f21e406ac00 0x7f21e406d0c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:39.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.021+0000 7f2208cab640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22040c8510 0x7f22040b6e30 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:39.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.021+0000 7f2208cab640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22041530f0 0x7f2204152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:39.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.021+0000 7f2208cab640 1 --2- 172.21.15.47:0/3062018146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0005180 0x7f22041f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:39.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.021+0000 7f2208cab640 1 -- 172.21.15.47:0/3062018146 >> 172.21.15.47:0/3062018146 conn(0x7f22040659d0 msgr2=0x7f22040d32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:39.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.021+0000 7f2208cab640 1 -- 172.21.15.47:0/3062018146 shutdown_connections 2024-03-23T09:51:39.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.021+0000 7f2208cab640 1 -- 172.21.15.47:0/3062018146 wait complete. 2024-03-23T09:51:39.035 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T09:51:39.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.184+0000 7fbb69226640 1 Processor -- start 2024-03-23T09:51:39.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.184+0000 7fbb69226640 1 -- start start 2024-03-23T09:51:39.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.184+0000 7fbb69226640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb6415cbb0 0x7fbb640b8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.184+0000 7fbb69226640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6415cf90 con 0x7fbb640d1df0 2024-03-23T09:51:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.184+0000 7fbb69226640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb640d0d50 con 0x7fbb640b4ac0 2024-03-23T09:51:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.184+0000 7fbb69226640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640b5210 con 0x7fbb6415cbb0 2024-03-23T09:51:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.184+0000 7fbb6259c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbb640d1df0 0x7fbb640b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52408/0 (socket says 172.21.15.47:52408) 2024-03-23T09:51:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb6259c640 1 -- 172.21.15.47:0/2270893937 learned_addr learned my addr 172.21.15.47:0/2270893937 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb62d9d640 1 --2- 172.21.15.47:0/2270893937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb6415cbb0 0x7fbb640b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236215804 0 0) 0x7fbb6415cf90 con 0x7fbb640d1df0 2024-03-23T09:51:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb50003520 con 0x7fbb640d1df0 2024-03-23T09:51:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034880670 0 0) 0x7fbb640d0d50 con 0x7fbb640b4ac0 2024-03-23T09:51:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb6415cf90 con 0x7fbb640b4ac0 2024-03-23T09:51:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3245283119 0 0) 0x7fbb50003520 con 0x7fbb640d1df0 2024-03-23T09:51:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb6415cbb0 msgr2=0x7fbb640b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 --2- 172.21.15.47:0/2270893937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb6415cbb0 0x7fbb640b8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 >> v1:172.21.15.175:6789/0 conn(0x7fbb640b4ac0 legacy=0x7fbb640b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb640d1030 con 0x7fbb640d1df0 2024-03-23T09:51:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fbb4c0015f0 con 0x7fbb640d1df0 2024-03-23T09:51:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 >> v1:172.21.15.47:6789/0 conn(0x7fbb640d1df0 legacy=0x7fbb640b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 --2- 172.21.15.47:0/2270893937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb50003b30 0x7fbb50003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 --2- 172.21.15.47:0/2270893937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb6415cbb0 0x7fbb500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 --2- 172.21.15.47:0/2270893937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb50004d00 0x7fbb50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640b5210 con 0x7fbb50004d00 2024-03-23T09:51:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5000ba30 con 0x7fbb50003b30 2024-03-23T09:51:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5000bbf0 con 0x7fbb6415cbb0 2024-03-23T09:51:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb6359e640 1 --2- 172.21.15.47:0/2270893937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb50004d00 0x7fbb50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb62d9d640 1 --2- 172.21.15.47:0/2270893937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb50003b30 0x7fbb50003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.185+0000 7fbb6259c640 1 --2- 172.21.15.47:0/2270893937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb6415cbb0 0x7fbb500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.186+0000 7fbb6359e640 1 -- 172.21.15.47:0/2270893937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb6415cbb0 msgr2=0x7fbb500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.186+0000 7fbb6359e640 1 --2- 172.21.15.47:0/2270893937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb6415cbb0 0x7fbb500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.186+0000 7fbb6359e640 1 -- 172.21.15.47:0/2270893937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb50003b30 msgr2=0x7fbb50003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.186+0000 7fbb6359e640 1 --2- 172.21.15.47:0/2270893937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb50003b30 0x7fbb50003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.186+0000 7fbb6359e640 1 -- 172.21.15.47:0/2270893937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50003520 con 0x7fbb50004d00 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.186+0000 7fbb6359e640 1 --2- 172.21.15.47:0/2270893937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb50004d00 0x7fbb50005180 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fbb640b2360 tx=0x7fbb500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a240d09a119565d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.186+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54003b70 con 0x7fbb50004d00 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.186+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb54003cd0 con 0x7fbb50004d00 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.186+0000 7fbb61d9b640 1 -- 172.21.15.47:0/2270893937 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb540052f0 con 0x7fbb50004d00 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.187+0000 7fbb69226640 1 -- 172.21.15.47:0/2270893937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb50004d00 msgr2=0x7fbb50005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.187+0000 7fbb69226640 1 --2- 172.21.15.47:0/2270893937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb50004d00 0x7fbb50005180 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fbb640b2360 tx=0x7fbb500057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.187+0000 7fbb6359e640 1 -- 172.21.15.47:0/2270893937 reap_dead start 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.187+0000 7fbb69226640 1 -- 172.21.15.47:0/2270893937 shutdown_connections 2024-03-23T09:51:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.187+0000 7fbb69226640 1 -- 172.21.15.47:0/2270893937 >> 172.21.15.47:0/2270893937 conn(0x7fbb64065dc0 msgr2=0x7fbb640661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:39.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.187+0000 7fbb69226640 1 -- 172.21.15.47:0/2270893937 shutdown_connections 2024-03-23T09:51:39.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.187+0000 7fbb69226640 1 -- 172.21.15.47:0/2270893937 wait complete. 2024-03-23T09:51:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.188+0000 7fbb69226640 1 Processor -- start 2024-03-23T09:51:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.189+0000 7fbb69226640 1 -- start start 2024-03-23T09:51:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.189+0000 7fbb69226640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb64145d00 0x7fbb6406af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.189+0000 7fbb69226640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb6406b440 0x7fbb641f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.189+0000 7fbb69226640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb641f2910 0x7fbb64150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.189+0000 7fbb69226640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6406b8c0 con 0x7fbb641f2910 2024-03-23T09:51:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.189+0000 7fbb69226640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb641f3190 con 0x7fbb6406b440 2024-03-23T09:51:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.189+0000 7fbb69226640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb641514d0 con 0x7fbb64145d00 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.189+0000 7fbb6359e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb641f2910 0x7fbb64150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.189+0000 7fbb6359e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb641f2910 0x7fbb64150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54440/0 (socket says 172.21.15.47:54440) 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.189+0000 7fbb6359e640 1 -- 172.21.15.47:0/819435246 learned_addr learned my addr 172.21.15.47:0/819435246 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.190+0000 7fbb62d9d640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb64145d00 0x7fbb6406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.190+0000 7fbb6259c640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb6406b440 0x7fbb641f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.190+0000 7fbb6359e640 1 -- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb64145d00 msgr2=0x7fbb6406af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.190+0000 7fbb6359e640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb64145d00 0x7fbb6406af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.190+0000 7fbb6359e640 1 -- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb6406b440 msgr2=0x7fbb641f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.190+0000 7fbb6359e640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb6406b440 0x7fbb641f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.190+0000 7fbb6359e640 1 -- 172.21.15.47:0/819435246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb641517b0 con 0x7fbb641f2910 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.190+0000 7fbb6359e640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb641f2910 0x7fbb64150330 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fbb54005040 tx=0x7fbb54001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.190+0000 7fbb5b7fe640 1 -- 172.21.15.47:0/819435246 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54041070 con 0x7fbb641f2910 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.190+0000 7fbb5b7fe640 1 -- 172.21.15.47:0/819435246 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb54003e40 con 0x7fbb641f2910 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.190+0000 7fbb69226640 1 -- 172.21.15.47:0/819435246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb64151a90 con 0x7fbb641f2910 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.190+0000 7fbb5b7fe640 1 -- 172.21.15.47:0/819435246 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb540052e0 con 0x7fbb641f2910 2024-03-23T09:51:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.190+0000 7fbb69226640 1 -- 172.21.15.47:0/819435246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb641f3490 con 0x7fbb641f2910 2024-03-23T09:51:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.191+0000 7fbb69226640 1 -- 172.21.15.47:0/819435246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb64072360 con 0x7fbb641f2910 2024-03-23T09:51:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.192+0000 7fbb5b7fe640 1 -- 172.21.15.47:0/819435246 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbb54005440 con 0x7fbb641f2910 2024-03-23T09:51:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.192+0000 7fbb5b7fe640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbb4406ac00 0x7fbb4406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.192+0000 7fbb5b7fe640 1 -- 172.21.15.47:0/819435246 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fbb540aa2b0 con 0x7fbb641f2910 2024-03-23T09:51:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.192+0000 7fbb62d9d640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbb4406ac00 0x7fbb4406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.193+0000 7fbb62d9d640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbb4406ac00 0x7fbb4406d0c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009b00 tx=0x7fbb5c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:39.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.197+0000 7fbb5b7fe640 1 -- 172.21.15.47:0/819435246 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbb54075670 con 0x7fbb641f2910 2024-03-23T09:51:39.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:39.375+0000 7fbb69226640 1 -- 172.21.15.47:0/819435246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbb64072360 con 0x7fbb641f2910 2024-03-23T09:51:40.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.022+0000 7fbb5b7fe640 1 -- 172.21.15.47:0/819435246 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v112) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbb54078c90 con 0x7fbb641f2910 2024-03-23T09:51:40.023 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-03-23T09:51:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.028+0000 7fbb69226640 1 -- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbb4406ac00 msgr2=0x7fbb4406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.028+0000 7fbb69226640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbb4406ac00 0x7fbb4406d0c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009b00 tx=0x7fbb5c008040 comp rx=0 tx=0).stop 2024-03-23T09:51:40.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.028+0000 7fbb69226640 1 -- 172.21.15.47:0/819435246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb641f2910 msgr2=0x7fbb64150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.028+0000 7fbb69226640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb641f2910 0x7fbb64150330 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fbb54005040 tx=0x7fbb54001650 comp rx=0 tx=0).stop 2024-03-23T09:51:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.036+0000 7fbb69226640 1 -- 172.21.15.47:0/819435246 shutdown_connections 2024-03-23T09:51:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.036+0000 7fbb69226640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbb4406ac00 0x7fbb4406d0c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.036+0000 7fbb69226640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb641f2910 0x7fbb64150330 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.036+0000 7fbb69226640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb6406b440 0x7fbb641f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.036+0000 7fbb69226640 1 --2- 172.21.15.47:0/819435246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb64145d00 0x7fbb6406af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.036+0000 7fbb69226640 1 -- 172.21.15.47:0/819435246 >> 172.21.15.47:0/819435246 conn(0x7fbb64065dc0 msgr2=0x7fbb6415a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:40.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.038+0000 7fbb69226640 1 -- 172.21.15.47:0/819435246 shutdown_connections 2024-03-23T09:51:40.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.039+0000 7fbb69226640 1 -- 172.21.15.47:0/819435246 wait complete. 2024-03-23T09:51:40.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T09:51:40.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.164+0000 7f1445dae640 1 Processor -- start 2024-03-23T09:51:40.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.164+0000 7f1445dae640 1 -- start start 2024-03-23T09:51:40.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.165+0000 7f1445dae640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f144015cad0 0x7f144006b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.165+0000 7f1445dae640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f144006bd40 0x7f144015ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.165+0000 7f1445dae640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144006c180 con 0x7f144015d3f0 2024-03-23T09:51:40.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.165+0000 7f1445dae640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14400cee20 con 0x7f144015cad0 2024-03-23T09:51:40.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.165+0000 7f1445dae640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14400cea80 con 0x7f144006bd40 2024-03-23T09:51:40.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.165+0000 7f143f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f144015cad0 0x7f144006b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.165+0000 7f143f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f144015cad0 0x7f144006b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54442/0 (socket says 172.21.15.47:54442) 2024-03-23T09:51:40.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.165+0000 7f143f7fe640 1 -- 172.21.15.47:0/1250273600 learned_addr learned my addr 172.21.15.47:0/1250273600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:40.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.165+0000 7f143effd640 1 --2- 172.21.15.47:0/1250273600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f144006bd40 0x7f144015ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.165+0000 7f143f7fe640 1 -- 172.21.15.47:0/1250273600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f144006bd40 msgr2=0x7f144015ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:40.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.165+0000 7f143f7fe640 1 --2- 172.21.15.47:0/1250273600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f144006bd40 0x7f144015ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.165+0000 7f143f7fe640 1 -- 172.21.15.47:0/1250273600 >> v1:172.21.15.175:6789/0 conn(0x7f144015d3f0 legacy=0x7f1440098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.165+0000 7f143f7fe640 1 -- 172.21.15.47:0/1250273600 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14400afc40 con 0x7f144015cad0 2024-03-23T09:51:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.166+0000 7f143f7fe640 1 --2- 172.21.15.47:0/1250273600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f144015cad0 0x7f144006b800 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f1434009870 tx=0x7f143402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a017b1ce695166 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.166+0000 7f143e7fc640 1 -- 172.21.15.47:0/1250273600 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434031640 con 0x7f144015cad0 2024-03-23T09:51:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.166+0000 7f143e7fc640 1 -- 172.21.15.47:0/1250273600 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14340317a0 con 0x7f144015cad0 2024-03-23T09:51:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.166+0000 7f143e7fc640 1 -- 172.21.15.47:0/1250273600 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434031b30 con 0x7f144015cad0 2024-03-23T09:51:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.167+0000 7f1445dae640 1 -- 172.21.15.47:0/1250273600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f144015cad0 msgr2=0x7f144006b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.167+0000 7f1445dae640 1 --2- 172.21.15.47:0/1250273600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f144015cad0 0x7f144006b800 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f1434009870 tx=0x7f143402b8f0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.167+0000 7f1445dae640 1 -- 172.21.15.47:0/1250273600 shutdown_connections 2024-03-23T09:51:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.167+0000 7f1445dae640 1 --2- 172.21.15.47:0/1250273600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f144006bd40 0x7f144015ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.167+0000 7f1445dae640 1 --2- 172.21.15.47:0/1250273600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f144015cad0 0x7f144006b800 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.167+0000 7f1445dae640 1 -- 172.21.15.47:0/1250273600 >> 172.21.15.47:0/1250273600 conn(0x7f14400d06a0 msgr2=0x7f14400d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.167+0000 7f1445dae640 1 -- 172.21.15.47:0/1250273600 shutdown_connections 2024-03-23T09:51:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.167+0000 7f1445dae640 1 -- 172.21.15.47:0/1250273600 wait complete. 2024-03-23T09:51:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.168+0000 7f1445dae640 1 Processor -- start 2024-03-23T09:51:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.168+0000 7f1445dae640 1 -- start start 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f1445dae640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f144006bd40 0x7f1440155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f1445dae640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f144015cad0 0x7f1440152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f1445dae640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f144015d3f0 0x7f14401530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f1445dae640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1440157d70 con 0x7f144015cad0 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f1445dae640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14401577b0 con 0x7f144015d3f0 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f1445dae640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1440157a90 con 0x7f144006bd40 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f143effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f144015cad0 0x7f1440152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f143effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f144015cad0 0x7f1440152b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54456/0 (socket says 172.21.15.47:54456) 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f143effd640 1 -- 172.21.15.47:0/2203026741 learned_addr learned my addr 172.21.15.47:0/2203026741 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f143f7fe640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f144006bd40 0x7f1440155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f143effd640 1 -- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f144006bd40 msgr2=0x7f1440155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f143effd640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f144006bd40 0x7f1440155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f143effd640 1 -- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f144015d3f0 msgr2=0x7f14401530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f143ffff640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f144015d3f0 0x7f14401530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f143effd640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f144015d3f0 0x7f14401530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.169+0000 7f143effd640 1 -- 172.21.15.47:0/2203026741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14340094d0 con 0x7f144015cad0 2024-03-23T09:51:40.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.170+0000 7f143effd640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f144015cad0 0x7f1440152b60 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f143000e870 tx=0x7f143000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:40.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.170+0000 7f1427fff640 1 -- 172.21.15.47:0/2203026741 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14300094f0 con 0x7f144015cad0 2024-03-23T09:51:40.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.170+0000 7f1427fff640 1 -- 172.21.15.47:0/2203026741 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143000c940 con 0x7f144015cad0 2024-03-23T09:51:40.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.170+0000 7f1445dae640 1 -- 172.21.15.47:0/2203026741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14400b4d60 con 0x7f144015cad0 2024-03-23T09:51:40.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.170+0000 7f1427fff640 1 -- 172.21.15.47:0/2203026741 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143000ccf0 con 0x7f144015cad0 2024-03-23T09:51:40.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.170+0000 7f1445dae640 1 -- 172.21.15.47:0/2203026741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1440157f60 con 0x7f144015cad0 2024-03-23T09:51:40.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.171+0000 7f1445dae640 1 -- 172.21.15.47:0/2203026741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14400672f0 con 0x7f144015cad0 2024-03-23T09:51:40.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.171+0000 7f1427fff640 1 -- 172.21.15.47:0/2203026741 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1430013470 con 0x7f144015cad0 2024-03-23T09:51:40.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.172+0000 7f1427fff640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f142c06ac30 0x7f142c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.172+0000 7f1427fff640 1 -- 172.21.15.47:0/2203026741 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f1430089400 con 0x7f144015cad0 2024-03-23T09:51:40.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.172+0000 7f143f7fe640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f142c06ac30 0x7f142c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.173+0000 7f143f7fe640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f142c06ac30 0x7f142c06d0f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1434036000 tx=0x7f143403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:40.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.176+0000 7f1427fff640 1 -- 172.21.15.47:0/2203026741 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f14300545c0 con 0x7f144015cad0 2024-03-23T09:51:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.393+0000 7f1445dae640 1 -- 172.21.15.47:0/2203026741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f144015d9d0 con 0x7f144015cad0 2024-03-23T09:51:40.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.399+0000 7f1427fff640 1 -- 172.21.15.47:0/2203026741 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v77) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1430057be0 con 0x7f144015cad0 2024-03-23T09:51:40.399 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 16 and data pool 17 2024-03-23T09:51:40.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.403+0000 7f1445dae640 1 -- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f142c06ac30 msgr2=0x7f142c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.403+0000 7f1445dae640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f142c06ac30 0x7f142c06d0f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1434036000 tx=0x7f143403c040 comp rx=0 tx=0).stop 2024-03-23T09:51:40.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.403+0000 7f1445dae640 1 -- 172.21.15.47:0/2203026741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f144015cad0 msgr2=0x7f1440152b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.403+0000 7f1445dae640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f144015cad0 0x7f1440152b60 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f143000e870 tx=0x7f143000ed30 comp rx=0 tx=0).stop 2024-03-23T09:51:40.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.405+0000 7f1445dae640 1 -- 172.21.15.47:0/2203026741 shutdown_connections 2024-03-23T09:51:40.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.405+0000 7f1445dae640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f142c06ac30 0x7f142c06d0f0 secure :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1434036000 tx=0x7f143403c040 comp rx=0 tx=0).stop 2024-03-23T09:51:40.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.405+0000 7f1445dae640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f144015d3f0 0x7f14401530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.405+0000 7f1445dae640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f144015cad0 0x7f1440152b60 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.405+0000 7f1445dae640 1 --2- 172.21.15.47:0/2203026741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f144006bd40 0x7f1440155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.405+0000 7f1445dae640 1 -- 172.21.15.47:0/2203026741 >> 172.21.15.47:0/2203026741 conn(0x7f14400d06a0 msgr2=0x7f144006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:40.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.411+0000 7f1445dae640 1 -- 172.21.15.47:0/2203026741 shutdown_connections 2024-03-23T09:51:40.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.411+0000 7f1445dae640 1 -- 172.21.15.47:0/2203026741 wait complete. 2024-03-23T09:51:40.430 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:51:40.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.530+0000 7f247fdae640 1 Processor -- start 2024-03-23T09:51:40.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.531+0000 7f247fdae640 1 -- start start 2024-03-23T09:51:40.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.531+0000 7f247fdae640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f247815ca90 0x7f2478160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.531+0000 7f247fdae640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24780b8350 0x7f24780cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.531+0000 7f247fdae640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f24780cdc40 0x7f24780b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.531+0000 7f247fdae640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24780ce0d0 con 0x7f24780cdc40 2024-03-23T09:51:40.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.531+0000 7f247fdae640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24780d11e0 con 0x7f247815ca90 2024-03-23T09:51:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.531+0000 7f247fdae640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2478160f60 con 0x7f24780b8350 2024-03-23T09:51:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.532+0000 7f247e34b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f24780cdc40 0x7f24780b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.532+0000 7f247e34b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f24780cdc40 0x7f24780b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54460/0 (socket says 172.21.15.47:54460) 2024-03-23T09:51:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.532+0000 7f247e34b640 1 -- 172.21.15.47:0/1059161759 learned_addr learned my addr 172.21.15.47:0/1059161759 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.532+0000 7f247d349640 1 --2- 172.21.15.47:0/1059161759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24780b8350 0x7f24780cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.532+0000 7f247db4a640 1 --2- 172.21.15.47:0/1059161759 >> v2:172.21.15.175:3300/0 conn(0x7f247815ca90 0x7f2478160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.532+0000 7f247e34b640 1 -- 172.21.15.47:0/1059161759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24780b8350 msgr2=0x7f24780cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.532+0000 7f247e34b640 1 --2- 172.21.15.47:0/1059161759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24780b8350 0x7f24780cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.532+0000 7f247e34b640 1 -- 172.21.15.47:0/1059161759 >> v2:172.21.15.175:3300/0 conn(0x7f247815ca90 msgr2=0x7f2478160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.532+0000 7f247e34b640 1 --2- 172.21.15.47:0/1059161759 >> v2:172.21.15.175:3300/0 conn(0x7f247815ca90 0x7f2478160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.532+0000 7f247e34b640 1 -- 172.21.15.47:0/1059161759 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2478161240 con 0x7f24780cdc40 2024-03-23T09:51:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.532+0000 7f247d349640 1 --2- 172.21.15.47:0/1059161759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24780b8350 0x7f24780cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.532+0000 7f247db4a640 1 --2- 172.21.15.47:0/1059161759 >> v2:172.21.15.175:3300/0 conn(0x7f247815ca90 0x7f2478160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.533+0000 7f247e34b640 1 --2- 172.21.15.47:0/1059161759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24780cdc40 0x7f24780b4440 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f2478066680 tx=0x7f247002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21470f3dfb57c75b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.533+0000 7f247cb48640 1 -- 172.21.15.47:0/1059161759 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247002c9e0 con 0x7f24780cdc40 2024-03-23T09:51:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.533+0000 7f247cb48640 1 -- 172.21.15.47:0/1059161759 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247002cb40 con 0x7f24780cdc40 2024-03-23T09:51:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.533+0000 7f247cb48640 1 -- 172.21.15.47:0/1059161759 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24700343f0 con 0x7f24780cdc40 2024-03-23T09:51:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.533+0000 7f247fdae640 1 -- 172.21.15.47:0/1059161759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24780cdc40 msgr2=0x7f24780b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.533+0000 7f247fdae640 1 --2- 172.21.15.47:0/1059161759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24780cdc40 0x7f24780b4440 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f2478066680 tx=0x7f247002be40 comp rx=0 tx=0).stop 2024-03-23T09:51:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.533+0000 7f247fdae640 1 -- 172.21.15.47:0/1059161759 shutdown_connections 2024-03-23T09:51:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.533+0000 7f247fdae640 1 --2- 172.21.15.47:0/1059161759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24780cdc40 0x7f24780b4440 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.533+0000 7f247fdae640 1 --2- 172.21.15.47:0/1059161759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24780b8350 0x7f24780cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.533+0000 7f247fdae640 1 --2- 172.21.15.47:0/1059161759 >> v2:172.21.15.175:3300/0 conn(0x7f247815ca90 0x7f2478160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.533+0000 7f247fdae640 1 -- 172.21.15.47:0/1059161759 >> 172.21.15.47:0/1059161759 conn(0x7f24780659d0 msgr2=0x7f2478065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:40.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.533+0000 7f247fdae640 1 -- 172.21.15.47:0/1059161759 shutdown_connections 2024-03-23T09:51:40.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.533+0000 7f247fdae640 1 -- 172.21.15.47:0/1059161759 wait complete. 2024-03-23T09:51:40.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.534+0000 7f247fdae640 1 Processor -- start 2024-03-23T09:51:40.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.535+0000 7f247fdae640 1 -- start start 2024-03-23T09:51:40.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.535+0000 7f247fdae640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24780b8350 0x7f247814bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.535+0000 7f247fdae640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24780cdc40 0x7f24781588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.535+0000 7f247fdae640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f247815ca90 0x7f24780b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.535+0000 7f247fdae640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24781592c0 con 0x7f247815ca90 2024-03-23T09:51:40.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.535+0000 7f247fdae640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24781595a0 con 0x7f24780b8350 2024-03-23T09:51:40.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.535+0000 7f247fdae640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2478159880 con 0x7f24780cdc40 2024-03-23T09:51:40.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.535+0000 7f247e34b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f247815ca90 0x7f24780b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.535+0000 7f247e34b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f247815ca90 0x7f24780b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54462/0 (socket says 172.21.15.47:54462) 2024-03-23T09:51:40.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.535+0000 7f247e34b640 1 -- 172.21.15.47:0/4030001922 learned_addr learned my addr 172.21.15.47:0/4030001922 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:40.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.536+0000 7f247db4a640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24780b8350 0x7f247814bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.536+0000 7f247d349640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24780cdc40 0x7f24781588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.536+0000 7f247e34b640 1 -- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24780cdc40 msgr2=0x7f24781588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.536+0000 7f247e34b640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24780cdc40 0x7f24781588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.536+0000 7f247e34b640 1 -- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24780b8350 msgr2=0x7f247814bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.536+0000 7f247e34b640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24780b8350 0x7f247814bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.536+0000 7f247e34b640 1 -- 172.21.15.47:0/4030001922 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2470009b30 con 0x7f247815ca90 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.536+0000 7f247e34b640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f247815ca90 0x7f24780b4430 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f2470009f40 tx=0x7f2470004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.536+0000 7f246e7fc640 1 -- 172.21.15.47:0/4030001922 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247003e450 con 0x7f247815ca90 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.536+0000 7f246e7fc640 1 -- 172.21.15.47:0/4030001922 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247003e5b0 con 0x7f247815ca90 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.536+0000 7f247fdae640 1 -- 172.21.15.47:0/4030001922 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2478159b60 con 0x7f247815ca90 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.536+0000 7f246e7fc640 1 -- 172.21.15.47:0/4030001922 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247003e960 con 0x7f247815ca90 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.536+0000 7f247fdae640 1 -- 172.21.15.47:0/4030001922 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247815bcf0 con 0x7f247815ca90 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.537+0000 7f247fdae640 1 -- 172.21.15.47:0/4030001922 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2478072360 con 0x7f247815ca90 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.537+0000 7f246e7fc640 1 -- 172.21.15.47:0/4030001922 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f247003eac0 con 0x7f247815ca90 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.538+0000 7f246e7fc640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f245c06ac00 0x7f245c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.538+0000 7f246e7fc640 1 -- 172.21.15.47:0/4030001922 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24700ac270 con 0x7f247815ca90 2024-03-23T09:51:40.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.538+0000 7f247db4a640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f245c06ac00 0x7f245c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.538+0000 7f247db4a640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f245c06ac00 0x7f245c06d0c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f247400b840 tx=0x7f2474008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:40.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.542+0000 7f246e7fc640 1 -- 172.21.15.47:0/4030001922 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2470077570 con 0x7f247815ca90 2024-03-23T09:51:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.722+0000 7f247fdae640 1 -- 172.21.15.47:0/4030001922 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f247815beb0 con 0x7f247815ca90 2024-03-23T09:51:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.723+0000 7f246e7fc640 1 -- 172.21.15.47:0/4030001922 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16072 (secure 0 0 0) 0x7f247007ab90 con 0x7f247815ca90 2024-03-23T09:51:40.723 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:40.723 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":113,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:51:40.395177+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:51:38.679890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-23T09:51:39.396299+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:51:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.725+0000 7f247fdae640 1 -- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f245c06ac00 msgr2=0x7f245c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.725+0000 7f247fdae640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f245c06ac00 0x7f245c06d0c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f247400b840 tx=0x7f2474008040 comp rx=0 tx=0).stop 2024-03-23T09:51:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.725+0000 7f247fdae640 1 -- 172.21.15.47:0/4030001922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f247815ca90 msgr2=0x7f24780b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.725+0000 7f247fdae640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f247815ca90 0x7f24780b4430 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f2470009f40 tx=0x7f2470004a40 comp rx=0 tx=0).stop 2024-03-23T09:51:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.725+0000 7f247fdae640 1 -- 172.21.15.47:0/4030001922 shutdown_connections 2024-03-23T09:51:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.725+0000 7f247fdae640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f245c06ac00 0x7f245c06d0c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.725+0000 7f247fdae640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f247815ca90 0x7f24780b4430 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.725+0000 7f247fdae640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24780cdc40 0x7f24781588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.725+0000 7f247fdae640 1 --2- 172.21.15.47:0/4030001922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24780b8350 0x7f247814bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.725+0000 7f247fdae640 1 -- 172.21.15.47:0/4030001922 >> 172.21.15.47:0/4030001922 conn(0x7f24780659d0 msgr2=0x7f24780d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.725+0000 7f247fdae640 1 -- 172.21.15.47:0/4030001922 shutdown_connections 2024-03-23T09:51:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.725+0000 7f247fdae640 1 -- 172.21.15.47:0/4030001922 wait complete. 2024-03-23T09:51:40.736 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:51:40.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.836+0000 7f0b18af7640 1 Processor -- start 2024-03-23T09:51:40.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.836+0000 7f0b18af7640 1 -- start start 2024-03-23T09:51:40.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.837+0000 7f0b18af7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1415cbc0 0x7f0b140cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.837+0000 7f0b18af7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0b1406a3f0 0x7f0b140ca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.837+0000 7f0b18af7640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b140c9ed0 con 0x7f0b14098e10 2024-03-23T09:51:40.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.837+0000 7f0b18af7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b140b0d10 con 0x7f0b1406a3f0 2024-03-23T09:51:40.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.837+0000 7f0b18af7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140b0ff0 con 0x7f0b1415cbc0 2024-03-23T09:51:40.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.837+0000 7f0b12d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0b1406a3f0 0x7f0b140ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.837+0000 7f0b12d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0b1406a3f0 0x7f0b140ca920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54472/0 (socket says 172.21.15.47:54472) 2024-03-23T09:51:40.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.837+0000 7f0b12d9d640 1 -- 172.21.15.47:0/833447774 learned_addr learned my addr 172.21.15.47:0/833447774 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:40.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.838+0000 7f0b1259c640 1 --2- 172.21.15.47:0/833447774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1415cbc0 0x7f0b140cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.838+0000 7f0b12d9d640 1 -- 172.21.15.47:0/833447774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1415cbc0 msgr2=0x7f0b140cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:40.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.838+0000 7f0b12d9d640 1 --2- 172.21.15.47:0/833447774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1415cbc0 0x7f0b140cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.838+0000 7f0b12d9d640 1 -- 172.21.15.47:0/833447774 >> v1:172.21.15.175:6789/0 conn(0x7f0b14098e10 legacy=0x7f0b14069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.838+0000 7f0b12d9d640 1 -- 172.21.15.47:0/833447774 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b140ca120 con 0x7f0b1406a3f0 2024-03-23T09:51:40.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.838+0000 7f0b1259c640 1 --2- 172.21.15.47:0/833447774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1415cbc0 0x7f0b140cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:40.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.839+0000 7f0b12d9d640 1 --2- 172.21.15.47:0/833447774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1406a3f0 0x7f0b140ca920 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009870 tx=0x7f0b0c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ead1fe815840e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:40.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.839+0000 7f0b1159a640 1 -- 172.21.15.47:0/833447774 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c030840 con 0x7f0b1406a3f0 2024-03-23T09:51:40.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.839+0000 7f0b1159a640 1 -- 172.21.15.47:0/833447774 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c0309a0 con 0x7f0b1406a3f0 2024-03-23T09:51:40.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.839+0000 7f0b1159a640 1 -- 172.21.15.47:0/833447774 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c030d30 con 0x7f0b1406a3f0 2024-03-23T09:51:40.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.839+0000 7f0b18af7640 1 -- 172.21.15.47:0/833447774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1406a3f0 msgr2=0x7f0b140ca920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.839+0000 7f0b18af7640 1 --2- 172.21.15.47:0/833447774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1406a3f0 0x7f0b140ca920 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009870 tx=0x7f0b0c02b980 comp rx=0 tx=0).stop 2024-03-23T09:51:40.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.839+0000 7f0b18af7640 1 -- 172.21.15.47:0/833447774 shutdown_connections 2024-03-23T09:51:40.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.839+0000 7f0b18af7640 1 --2- 172.21.15.47:0/833447774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1406a3f0 0x7f0b140ca920 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.839+0000 7f0b18af7640 1 --2- 172.21.15.47:0/833447774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1415cbc0 0x7f0b140cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.839+0000 7f0b18af7640 1 -- 172.21.15.47:0/833447774 >> 172.21.15.47:0/833447774 conn(0x7f0b14098a00 msgr2=0x7f0b140cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:40.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.839+0000 7f0b18af7640 1 -- 172.21.15.47:0/833447774 shutdown_connections 2024-03-23T09:51:40.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.839+0000 7f0b18af7640 1 -- 172.21.15.47:0/833447774 wait complete. 2024-03-23T09:51:40.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.841+0000 7f0b18af7640 1 Processor -- start 2024-03-23T09:51:40.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.841+0000 7f0b18af7640 1 -- start start 2024-03-23T09:51:40.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.841+0000 7f0b18af7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1406a3f0 0x7f0b140ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.841+0000 7f0b18af7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b14098e10 0x7f0b141f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.841+0000 7f0b18af7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1415cbc0 0x7f0b140ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.841+0000 7f0b18af7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140af820 con 0x7f0b1415cbc0 2024-03-23T09:51:40.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.841+0000 7f0b18af7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140aeba0 con 0x7f0b14098e10 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.841+0000 7f0b18af7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140aee80 con 0x7f0b1406a3f0 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.841+0000 7f0b12d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1415cbc0 0x7f0b140ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.841+0000 7f0b12d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1415cbc0 0x7f0b140ac710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54480/0 (socket says 172.21.15.47:54480) 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.841+0000 7f0b12d9d640 1 -- 172.21.15.47:0/536763584 learned_addr learned my addr 172.21.15.47:0/536763584 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b11d9b640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b14098e10 0x7f0b141f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b1259c640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1406a3f0 0x7f0b140ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b12d9d640 1 -- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1406a3f0 msgr2=0x7f0b140ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b12d9d640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1406a3f0 0x7f0b140ae4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b12d9d640 1 -- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b14098e10 msgr2=0x7f0b141f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b12d9d640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b14098e10 0x7f0b141f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b12d9d640 1 -- 172.21.15.47:0/536763584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c0094d0 con 0x7f0b1415cbc0 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b11d9b640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b14098e10 0x7f0b141f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b12d9d640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1415cbc0 0x7f0b140ac710 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009870 tx=0x7f0b0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b0affd640 1 -- 172.21.15.47:0/536763584 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c039a90 con 0x7f0b1415cbc0 2024-03-23T09:51:40.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b0affd640 1 -- 172.21.15.47:0/536763584 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c039c10 con 0x7f0b1415cbc0 2024-03-23T09:51:40.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b18af7640 1 -- 172.21.15.47:0/536763584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b140af160 con 0x7f0b1415cbc0 2024-03-23T09:51:40.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b0affd640 1 -- 172.21.15.47:0/536763584 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c043410 con 0x7f0b1415cbc0 2024-03-23T09:51:40.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.842+0000 7f0b18af7640 1 -- 172.21.15.47:0/536763584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b141f6d70 con 0x7f0b1415cbc0 2024-03-23T09:51:40.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.843+0000 7f0b18af7640 1 -- 172.21.15.47:0/536763584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b140b38a0 con 0x7f0b1415cbc0 2024-03-23T09:51:40.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.844+0000 7f0b0affd640 1 -- 172.21.15.47:0/536763584 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0b0c04c3b0 con 0x7f0b1415cbc0 2024-03-23T09:51:40.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.844+0000 7f0b0affd640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0af406ac00 0x7f0af406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:40.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.844+0000 7f0b0affd640 1 -- 172.21.15.47:0/536763584 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b0c0ab8a0 con 0x7f0b1415cbc0 2024-03-23T09:51:40.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.844+0000 7f0b1259c640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0af406ac00 0x7f0af406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:40.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.845+0000 7f0b1259c640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0af406ac00 0x7f0af406d0c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009f70 tx=0x7f0b0400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:40.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:40.848+0000 7f0b0affd640 1 -- 172.21.15.47:0/536763584 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0b140b38a0 con 0x7f0b1415cbc0 2024-03-23T09:51:41.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.029+0000 7f0b18af7640 1 -- 172.21.15.47:0/536763584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b140b38a0 con 0x7f0b1415cbc0 2024-03-23T09:51:41.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.031+0000 7f0b0affd640 1 -- 172.21.15.47:0/536763584 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16072 (secure 0 0 0) 0x7f0b140b38a0 con 0x7f0b1415cbc0 2024-03-23T09:51:41.031 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:41.031 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":113,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:51:40.395177+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:51:38.679890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-23T09:51:39.396299+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:51:41.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.032+0000 7f0b18af7640 1 -- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0af406ac00 msgr2=0x7f0af406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:41.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.032+0000 7f0b18af7640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0af406ac00 0x7f0af406d0c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009f70 tx=0x7f0b0400b040 comp rx=0 tx=0).stop 2024-03-23T09:51:41.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.032+0000 7f0b18af7640 1 -- 172.21.15.47:0/536763584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1415cbc0 msgr2=0x7f0b140ac710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:41.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.033+0000 7f0b18af7640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1415cbc0 0x7f0b140ac710 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009870 tx=0x7f0b0c004730 comp rx=0 tx=0).stop 2024-03-23T09:51:41.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.033+0000 7f0b18af7640 1 -- 172.21.15.47:0/536763584 shutdown_connections 2024-03-23T09:51:41.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.033+0000 7f0b18af7640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0af406ac00 0x7f0af406d0c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:41.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.033+0000 7f0b18af7640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b1415cbc0 0x7f0b140ac710 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.033+0000 7f0b18af7640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b14098e10 0x7f0b141f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.033+0000 7f0b18af7640 1 --2- 172.21.15.47:0/536763584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1406a3f0 0x7f0b140ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.033+0000 7f0b18af7640 1 -- 172.21.15.47:0/536763584 >> 172.21.15.47:0/536763584 conn(0x7f0b14098a00 msgr2=0x7f0b14158eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.033+0000 7f0b18af7640 1 -- 172.21.15.47:0/536763584 shutdown_connections 2024-03-23T09:51:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.033+0000 7f0b18af7640 1 -- 172.21.15.47:0/536763584 wait complete. 2024-03-23T09:51:41.044 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T09:51:41.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.147+0000 7f170d62c640 1 Processor -- start 2024-03-23T09:51:41.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.147+0000 7f170d62c640 1 -- start start 2024-03-23T09:51:41.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.148+0000 7f170d62c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17080ae270 0x7f17080d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:41.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.148+0000 7f170d62c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f17080b0d20 0x7f17080cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:41.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.148+0000 7f170d62c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f17080ce210 0x7f170815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:41.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.148+0000 7f170d62c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17080af870 con 0x7f17080b0d20 2024-03-23T09:51:41.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.148+0000 7f170d62c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17080afac0 con 0x7f17080ce210 2024-03-23T09:51:41.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.148+0000 7f170d62c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17080afda0 con 0x7f17080ae270 2024-03-23T09:51:41.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.148+0000 7f17067fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f17080b0d20 0x7f17080cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:41.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.148+0000 7f17067fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f17080b0d20 0x7f17080cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54482/0 (socket says 172.21.15.47:54482) 2024-03-23T09:51:41.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.148+0000 7f17067fc640 1 -- 172.21.15.47:0/1799546423 learned_addr learned my addr 172.21.15.47:0/1799546423 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:41.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.148+0000 7f1706ffd640 1 --2- 172.21.15.47:0/1799546423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17080ae270 0x7f17080d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:41.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.148+0000 7f17077fe640 1 --2- 172.21.15.47:0/1799546423 >> v2:172.21.15.175:3300/0 conn(0x7f17080ce210 0x7f170815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:41.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.149+0000 7f17067fc640 1 -- 172.21.15.47:0/1799546423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17080ae270 msgr2=0x7f17080d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:41.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.149+0000 7f17067fc640 1 --2- 172.21.15.47:0/1799546423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17080ae270 0x7f17080d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:41.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.149+0000 7f17067fc640 1 -- 172.21.15.47:0/1799546423 >> v2:172.21.15.175:3300/0 conn(0x7f17080ce210 msgr2=0x7f170815d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:41.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.149+0000 7f17067fc640 1 --2- 172.21.15.47:0/1799546423 >> v2:172.21.15.175:3300/0 conn(0x7f17080ce210 0x7f170815d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:41.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.149+0000 7f17067fc640 1 -- 172.21.15.47:0/1799546423 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17080b0080 con 0x7f17080b0d20 2024-03-23T09:51:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.149+0000 7f17067fc640 1 --2- 172.21.15.47:0/1799546423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17080b0d20 0x7f17080cdcd0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f17080c9f60 tx=0x7f16f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc449f9ca9869ac1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.149+0000 7f1705ffb640 1 -- 172.21.15.47:0/1799546423 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0031870 con 0x7f17080b0d20 2024-03-23T09:51:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.150+0000 7f1705ffb640 1 -- 172.21.15.47:0/1799546423 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f00319d0 con 0x7f17080b0d20 2024-03-23T09:51:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.150+0000 7f1705ffb640 1 -- 172.21.15.47:0/1799546423 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0031d60 con 0x7f17080b0d20 2024-03-23T09:51:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.150+0000 7f170d62c640 1 -- 172.21.15.47:0/1799546423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17080b0d20 msgr2=0x7f17080cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.150+0000 7f170d62c640 1 --2- 172.21.15.47:0/1799546423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17080b0d20 0x7f17080cdcd0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f17080c9f60 tx=0x7f16f002b910 comp rx=0 tx=0).stop 2024-03-23T09:51:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.150+0000 7f170d62c640 1 -- 172.21.15.47:0/1799546423 shutdown_connections 2024-03-23T09:51:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.150+0000 7f170d62c640 1 --2- 172.21.15.47:0/1799546423 >> v2:172.21.15.175:3300/0 conn(0x7f17080ce210 0x7f170815d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.150+0000 7f170d62c640 1 --2- 172.21.15.47:0/1799546423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17080b0d20 0x7f17080cdcd0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.150+0000 7f170d62c640 1 --2- 172.21.15.47:0/1799546423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17080ae270 0x7f17080d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.150+0000 7f170d62c640 1 -- 172.21.15.47:0/1799546423 >> 172.21.15.47:0/1799546423 conn(0x7f1708098a00 msgr2=0x7f17080cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.150+0000 7f170d62c640 1 -- 172.21.15.47:0/1799546423 shutdown_connections 2024-03-23T09:51:41.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.150+0000 7f170d62c640 1 -- 172.21.15.47:0/1799546423 wait complete. 2024-03-23T09:51:41.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.151+0000 7f170d62c640 1 Processor -- start 2024-03-23T09:51:41.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.151+0000 7f170d62c640 1 -- start start 2024-03-23T09:51:41.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.152+0000 7f170d62c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17080ae270 0x7f17080b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:41.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.152+0000 7f170d62c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17080b0d20 0x7f1708158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:41.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.152+0000 7f170d62c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17080ce210 0x7f1708152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:41.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.152+0000 7f170d62c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17080b4c60 con 0x7f17080b0d20 2024-03-23T09:51:41.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.152+0000 7f170d62c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17081534b0 con 0x7f17080ce210 2024-03-23T09:51:41.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.152+0000 7f170d62c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17080b3fe0 con 0x7f17080ae270 2024-03-23T09:51:41.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.152+0000 7f17067fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17080b0d20 0x7f1708158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:41.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.152+0000 7f17067fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17080b0d20 0x7f1708158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54492/0 (socket says 172.21.15.47:54492) 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.152+0000 7f17067fc640 1 -- 172.21.15.47:0/525325843 learned_addr learned my addr 172.21.15.47:0/525325843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.152+0000 7f17067fc640 1 -- 172.21.15.47:0/525325843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17080ae270 msgr2=0x7f17080b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.152+0000 7f17067fc640 1 --2- 172.21.15.47:0/525325843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17080ae270 0x7f17080b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.152+0000 7f17067fc640 1 -- 172.21.15.47:0/525325843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17080ce210 msgr2=0x7f1708152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.152+0000 7f17067fc640 1 --2- 172.21.15.47:0/525325843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17080ce210 0x7f1708152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.153+0000 7f17067fc640 1 -- 172.21.15.47:0/525325843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f00094d0 con 0x7f17080b0d20 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.153+0000 7f17067fc640 1 --2- 172.21.15.47:0/525325843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17080b0d20 0x7f1708158370 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f16f002b950 tx=0x7f16f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.153+0000 7f16ff7fe640 1 -- 172.21.15.47:0/525325843 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f003ad30 con 0x7f17080b0d20 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.153+0000 7f16ff7fe640 1 -- 172.21.15.47:0/525325843 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f00443f0 con 0x7f17080b0d20 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.153+0000 7f170d62c640 1 -- 172.21.15.47:0/525325843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17080b42c0 con 0x7f17080b0d20 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.153+0000 7f16ff7fe640 1 -- 172.21.15.47:0/525325843 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f00447a0 con 0x7f17080b0d20 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.153+0000 7f170d62c640 1 -- 172.21.15.47:0/525325843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17080b4880 con 0x7f17080b0d20 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.154+0000 7f170d62c640 1 -- 172.21.15.47:0/525325843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17080672f0 con 0x7f17080b0d20 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.154+0000 7f16ff7fe640 1 -- 172.21.15.47:0/525325843 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f16f004d450 con 0x7f17080b0d20 2024-03-23T09:51:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.155+0000 7f16ff7fe640 1 --2- 172.21.15.47:0/525325843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f16e806ac00 0x7f16e806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:41.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.155+0000 7f16ff7fe640 1 -- 172.21.15.47:0/525325843 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16f002d040 con 0x7f17080b0d20 2024-03-23T09:51:41.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.155+0000 7f1706ffd640 1 --2- 172.21.15.47:0/525325843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f16e806ac00 0x7f16e806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:41.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.155+0000 7f1706ffd640 1 --2- 172.21.15.47:0/525325843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f16e806ac00 0x7f16e806d0c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1700009870 tx=0x7f1700008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:41.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.159+0000 7f16ff7fe640 1 -- 172.21.15.47:0/525325843 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f16f0076bd0 con 0x7f17080b0d20 2024-03-23T09:51:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:41.374+0000 7f170d62c640 1 -- 172.21.15.47:0/525325843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f17080ad2b0 con 0x7f17080b0d20 2024-03-23T09:51:42.014 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:51:42.013+0000 7f4068e8e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:51:42.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.030+0000 7f16ff7fe640 1 -- 172.21.15.47:0/525325843 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v79) v1 ==== 118+0+0 (secure 0 0 0) 0x7f16f0035020 con 0x7f17080b0d20 2024-03-23T09:51:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.033+0000 7f170d62c640 1 -- 172.21.15.47:0/525325843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f16e806ac00 msgr2=0x7f16e806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.033+0000 7f170d62c640 1 --2- 172.21.15.47:0/525325843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f16e806ac00 0x7f16e806d0c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1700009870 tx=0x7f1700008040 comp rx=0 tx=0).stop 2024-03-23T09:51:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.033+0000 7f170d62c640 1 -- 172.21.15.47:0/525325843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17080b0d20 msgr2=0x7f1708158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.033+0000 7f170d62c640 1 --2- 172.21.15.47:0/525325843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17080b0d20 0x7f1708158370 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f16f002b950 tx=0x7f16f00046c0 comp rx=0 tx=0).stop 2024-03-23T09:51:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.033+0000 7f170d62c640 1 -- 172.21.15.47:0/525325843 shutdown_connections 2024-03-23T09:51:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.033+0000 7f170d62c640 1 --2- 172.21.15.47:0/525325843 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f16e806ac00 0x7f16e806d0c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.033+0000 7f170d62c640 1 --2- 172.21.15.47:0/525325843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17080ce210 0x7f1708152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.033+0000 7f170d62c640 1 --2- 172.21.15.47:0/525325843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17080b0d20 0x7f1708158370 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.033+0000 7f170d62c640 1 --2- 172.21.15.47:0/525325843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17080ae270 0x7f17080b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.033+0000 7f170d62c640 1 -- 172.21.15.47:0/525325843 >> 172.21.15.47:0/525325843 conn(0x7f1708098a00 msgr2=0x7f17080ca150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.033+0000 7f170d62c640 1 -- 172.21.15.47:0/525325843 shutdown_connections 2024-03-23T09:51:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.033+0000 7f170d62c640 1 -- 172.21.15.47:0/525325843 wait complete. 2024-03-23T09:51:42.046 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-23T09:51:42.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-23T09:51:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.156+0000 7f4a8a5f0640 1 Processor -- start 2024-03-23T09:51:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.156+0000 7f4a8a5f0640 1 -- start start 2024-03-23T09:51:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.157+0000 7f4a8a5f0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4a840cd680 0x7f4a840b0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.157+0000 7f4a8a5f0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a840cad40 0x7f4a840b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.157+0000 7f4a8a5f0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8415d010 con 0x7f4a8415cc30 2024-03-23T09:51:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.157+0000 7f4a8a5f0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a840b13d0 con 0x7f4a840cd680 2024-03-23T09:51:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.157+0000 7f4a8a5f0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840b16b0 con 0x7f4a840cad40 2024-03-23T09:51:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.157+0000 7f4a837fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4a840cd680 0x7f4a840b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.157+0000 7f4a83fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4a8415cc30 0x7f4a840ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52418/0 (socket says 172.21.15.47:52418) 2024-03-23T09:51:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.157+0000 7f4a837fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4a840cd680 0x7f4a840b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54500/0 (socket says 172.21.15.47:54500) 2024-03-23T09:51:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.157+0000 7f4a83fff640 1 -- 172.21.15.47:0/2213894796 learned_addr learned my addr 172.21.15.47:0/2213894796 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.158+0000 7f4a88b8d640 1 --2- 172.21.15.47:0/2213894796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a840cad40 0x7f4a840b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.158+0000 7f4a837fe640 1 -- 172.21.15.47:0/2213894796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a840cad40 msgr2=0x7f4a840b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.158+0000 7f4a837fe640 1 --2- 172.21.15.47:0/2213894796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a840cad40 0x7f4a840b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.158+0000 7f4a837fe640 1 -- 172.21.15.47:0/2213894796 >> v1:172.21.15.47:6789/0 conn(0x7f4a8415cc30 legacy=0x7f4a840ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.158+0000 7f4a837fe640 1 -- 172.21.15.47:0/2213894796 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a840cedb0 con 0x7f4a840cd680 2024-03-23T09:51:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.158+0000 7f4a82ffd640 1 -- 172.21.15.47:0/2213894796 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209927005 0 0) 0x7f4a8415d010 con 0x7f4a8415cc30 2024-03-23T09:51:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.158+0000 7f4a88b8d640 1 --2- 172.21.15.47:0/2213894796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a840cad40 0x7f4a840b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.158+0000 7f4a837fe640 1 --2- 172.21.15.47:0/2213894796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a840cd680 0x7f4a840b0610 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0097a0 tx=0x7f4a6c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24d059cb4261cf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.159+0000 7f4a82ffd640 1 -- 172.21.15.47:0/2213894796 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c031830 con 0x7f4a840cd680 2024-03-23T09:51:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.159+0000 7f4a82ffd640 1 -- 172.21.15.47:0/2213894796 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6c031990 con 0x7f4a840cd680 2024-03-23T09:51:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.159+0000 7f4a82ffd640 1 -- 172.21.15.47:0/2213894796 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c031d20 con 0x7f4a840cd680 2024-03-23T09:51:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.159+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/2213894796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a840cd680 msgr2=0x7f4a840b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.159+0000 7f4a8a5f0640 1 --2- 172.21.15.47:0/2213894796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a840cd680 0x7f4a840b0610 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0097a0 tx=0x7f4a6c02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:51:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.159+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/2213894796 shutdown_connections 2024-03-23T09:51:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.159+0000 7f4a8a5f0640 1 --2- 172.21.15.47:0/2213894796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a840cad40 0x7f4a840b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.159+0000 7f4a8a5f0640 1 --2- 172.21.15.47:0/2213894796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a840cd680 0x7f4a840b0610 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.159+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/2213894796 >> 172.21.15.47:0/2213894796 conn(0x7f4a840d06f0 msgr2=0x7f4a840d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.159+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/2213894796 shutdown_connections 2024-03-23T09:51:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.159+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/2213894796 wait complete. 2024-03-23T09:51:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.160+0000 7f4a8a5f0640 1 Processor -- start 2024-03-23T09:51:42.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.161+0000 7f4a8a5f0640 1 -- start start 2024-03-23T09:51:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.161+0000 7f4a8a5f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a840cad40 0x7f4a8414bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.161+0000 7f4a8a5f0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a840cd680 0x7f4a84152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.161+0000 7f4a8a5f0640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a8415cc30 0x7f4a840aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.161+0000 7f4a8a5f0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a841515d0 con 0x7f4a840cad40 2024-03-23T09:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.161+0000 7f4a8a5f0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a84151b90 con 0x7f4a840cd680 2024-03-23T09:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.161+0000 7f4a8a5f0640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840af640 con 0x7f4a8415cc30 2024-03-23T09:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.161+0000 7f4a83fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a840cad40 0x7f4a8414bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.161+0000 7f4a83fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a840cad40 0x7f4a8414bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54514/0 (socket says 172.21.15.47:54514) 2024-03-23T09:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.161+0000 7f4a83fff640 1 -- 172.21.15.47:0/1410311110 learned_addr learned my addr 172.21.15.47:0/1410311110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.162+0000 7f4a88b8d640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a8415cc30 0x7f4a840aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.162+0000 7f4a837fe640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a840cd680 0x7f4a84152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.162+0000 7f4a83fff640 1 -- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a8415cc30 msgr2=0x7f4a840aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.162+0000 7f4a83fff640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a8415cc30 0x7f4a840aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.162+0000 7f4a83fff640 1 -- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a840cd680 msgr2=0x7f4a84152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.162+0000 7f4a83fff640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a840cd680 0x7f4a84152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.162+0000 7f4a83fff640 1 -- 172.21.15.47:0/1410311110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c009400 con 0x7f4a840cad40 2024-03-23T09:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.162+0000 7f4a83fff640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a840cad40 0x7f4a8414bdd0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400c430 tx=0x7f4a7400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.162+0000 7f4a80ff9640 1 -- 172.21.15.47:0/1410311110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a740074e0 con 0x7f4a840cad40 2024-03-23T09:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.162+0000 7f4a80ff9640 1 -- 172.21.15.47:0/1410311110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a74007660 con 0x7f4a840cad40 2024-03-23T09:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.162+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/1410311110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a840af920 con 0x7f4a840cad40 2024-03-23T09:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.162+0000 7f4a80ff9640 1 -- 172.21.15.47:0/1410311110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74004a20 con 0x7f4a840cad40 2024-03-23T09:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.162+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/1410311110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8415f920 con 0x7f4a840cad40 2024-03-23T09:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.163+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/1410311110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a84066be0 con 0x7f4a840cad40 2024-03-23T09:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.163+0000 7f4a80ff9640 1 -- 172.21.15.47:0/1410311110 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4a74004b80 con 0x7f4a840cad40 2024-03-23T09:51:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.164+0000 7f4a80ff9640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4a6406ac00 0x7f4a6406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.164+0000 7f4a80ff9640 1 -- 172.21.15.47:0/1410311110 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a74089200 con 0x7f4a840cad40 2024-03-23T09:51:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.164+0000 7f4a837fe640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4a6406ac00 0x7f4a6406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.165+0000 7f4a837fe640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4a6406ac00 0x7f4a6406d0c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c036000 tx=0x7f4a6c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.168+0000 7f4a80ff9640 1 -- 172.21.15.47:0/1410311110 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4a740543d0 con 0x7f4a840cad40 2024-03-23T09:51:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:42.383+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/1410311110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4a840d0f50 con 0x7f4a840cad40 2024-03-23T09:51:43.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.036+0000 7f4a80ff9640 1 -- 172.21.15.47:0/1410311110 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v80) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4a740579f0 con 0x7f4a840cad40 2024-03-23T09:51:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.038+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4a6406ac00 msgr2=0x7f4a6406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.038+0000 7f4a8a5f0640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4a6406ac00 0x7f4a6406d0c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c036000 tx=0x7f4a6c008030 comp rx=0 tx=0).stop 2024-03-23T09:51:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.039+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/1410311110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a840cad40 msgr2=0x7f4a8414bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.039+0000 7f4a8a5f0640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a840cad40 0x7f4a8414bdd0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400c430 tx=0x7f4a7400c8f0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.039+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/1410311110 shutdown_connections 2024-03-23T09:51:43.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.039+0000 7f4a8a5f0640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4a6406ac00 0x7f4a6406d0c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.039+0000 7f4a8a5f0640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a8415cc30 0x7f4a840aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.039+0000 7f4a8a5f0640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a840cd680 0x7f4a84152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.039+0000 7f4a8a5f0640 1 --2- 172.21.15.47:0/1410311110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a840cad40 0x7f4a8414bdd0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.039+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/1410311110 >> 172.21.15.47:0/1410311110 conn(0x7f4a840d06f0 msgr2=0x7f4a8406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:43.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.039+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/1410311110 shutdown_connections 2024-03-23T09:51:43.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.039+0000 7f4a8a5f0640 1 -- 172.21.15.47:0/1410311110 wait complete. 2024-03-23T09:51:43.052 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-23T09:51:43.052 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:51:43.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.160+0000 7fcbdebb1640 1 Processor -- start 2024-03-23T09:51:43.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.160+0000 7fcbdebb1640 1 -- start start 2024-03-23T09:51:43.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdebb1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd815cd00 0x7fcbd80d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdebb1640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcbd80cf920 0x7fcbd80b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdebb1640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd815d0e0 con 0x7fcbd80d1020 2024-03-23T09:51:43.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdebb1640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd80d1540 con 0x7fcbd80cf920 2024-03-23T09:51:43.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdebb1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80d1820 con 0x7fcbd815cd00 2024-03-23T09:51:43.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdc94d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd815cd00 0x7fcbd80d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdd14e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcbd80cf920 0x7fcbd80b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdc94d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd815cd00 0x7fcbd80d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:59638/0 (socket says 172.21.15.47:59638) 2024-03-23T09:51:43.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdd14e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcbd80cf920 0x7fcbd80b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:59996/0 (socket says 172.21.15.47:59996) 2024-03-23T09:51:43.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdc94d640 1 -- 172.21.15.47:0/4204207671 learned_addr learned my addr 172.21.15.47:0/4204207671 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:43.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdc94d640 1 -- 172.21.15.47:0/4204207671 >> v2:172.21.15.175:3300/0 conn(0x7fcbd80cf920 msgr2=0x7fcbd80b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:43.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdc94d640 1 --2- 172.21.15.47:0/4204207671 >> v2:172.21.15.175:3300/0 conn(0x7fcbd80cf920 0x7fcbd80b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdc94d640 1 -- 172.21.15.47:0/4204207671 >> v1:172.21.15.175:6789/0 conn(0x7fcbd80d1020 legacy=0x7fcbd80cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.161+0000 7fcbdc94d640 1 -- 172.21.15.47:0/4204207671 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd80d1b00 con 0x7fcbd815cd00 2024-03-23T09:51:43.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.162+0000 7fcbdc94d640 1 --2- 172.21.15.47:0/4204207671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd815cd00 0x7fcbd80d0ae0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009870 tx=0x7fcbcc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0203d00e88cedbc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:43.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.162+0000 7fcbd77fe640 1 -- 172.21.15.47:0/4204207671 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc031640 con 0x7fcbd815cd00 2024-03-23T09:51:43.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.162+0000 7fcbd77fe640 1 -- 172.21.15.47:0/4204207671 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc0317a0 con 0x7fcbd815cd00 2024-03-23T09:51:43.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.162+0000 7fcbd77fe640 1 -- 172.21.15.47:0/4204207671 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc031b30 con 0x7fcbd815cd00 2024-03-23T09:51:43.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.163+0000 7fcbdebb1640 1 -- 172.21.15.47:0/4204207671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd815cd00 msgr2=0x7fcbd80d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.163+0000 7fcbdebb1640 1 --2- 172.21.15.47:0/4204207671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd815cd00 0x7fcbd80d0ae0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009870 tx=0x7fcbcc02b8f0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.163+0000 7fcbdebb1640 1 -- 172.21.15.47:0/4204207671 shutdown_connections 2024-03-23T09:51:43.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.163+0000 7fcbdebb1640 1 --2- 172.21.15.47:0/4204207671 >> v2:172.21.15.175:3300/0 conn(0x7fcbd80cf920 0x7fcbd80b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.163+0000 7fcbdebb1640 1 --2- 172.21.15.47:0/4204207671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd815cd00 0x7fcbd80d0ae0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.163+0000 7fcbdebb1640 1 -- 172.21.15.47:0/4204207671 >> 172.21.15.47:0/4204207671 conn(0x7fcbd806b160 msgr2=0x7fcbd806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:43.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.163+0000 7fcbdebb1640 1 -- 172.21.15.47:0/4204207671 shutdown_connections 2024-03-23T09:51:43.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.163+0000 7fcbdebb1640 1 -- 172.21.15.47:0/4204207671 wait complete. 2024-03-23T09:51:43.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.164+0000 7fcbdebb1640 1 Processor -- start 2024-03-23T09:51:43.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.164+0000 7fcbdebb1640 1 -- start start 2024-03-23T09:51:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.165+0000 7fcbdebb1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd80cf920 0x7fcbd81456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.165+0000 7fcbdebb1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd80d1020 0x7fcbd80c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.165+0000 7fcbdebb1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbd815cd00 0x7fcbd8144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.165+0000 7fcbdebb1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd815f170 con 0x7fcbd80d1020 2024-03-23T09:51:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.165+0000 7fcbdebb1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd8160130 con 0x7fcbd815cd00 2024-03-23T09:51:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.165+0000 7fcbdebb1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80ca210 con 0x7fcbd80cf920 2024-03-23T09:51:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.165+0000 7fcbd7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd80d1020 0x7fcbd80c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.165+0000 7fcbd7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd80d1020 0x7fcbd80c9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54516/0 (socket says 172.21.15.47:54516) 2024-03-23T09:51:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.165+0000 7fcbd7fff640 1 -- 172.21.15.47:0/3861957213 learned_addr learned my addr 172.21.15.47:0/3861957213 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbdd14e640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbd815cd00 0x7fcbd8144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbd7fff640 1 -- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd80cf920 msgr2=0x7fcbd81456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbdc94d640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd80cf920 0x7fcbd81456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbd7fff640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd80cf920 0x7fcbd81456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbd7fff640 1 -- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbd815cd00 msgr2=0x7fcbd8144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbd7fff640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbd815cd00 0x7fcbd8144a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbd7fff640 1 -- 172.21.15.47:0/3861957213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc0094d0 con 0x7fcbd80d1020 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbdd14e640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbd815cd00 0x7fcbd8144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbdc94d640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd80cf920 0x7fcbd81456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbd7fff640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd80d1020 0x7fcbd80c9a30 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400c480 tx=0x7fcbc400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbd57fa640 1 -- 172.21.15.47:0/3861957213 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc40052d0 con 0x7fcbd80d1020 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbd57fa640 1 -- 172.21.15.47:0/3861957213 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc4005430 con 0x7fcbd80d1020 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbdebb1640 1 -- 172.21.15.47:0/3861957213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd815f370 con 0x7fcbd80d1020 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbd57fa640 1 -- 172.21.15.47:0/3861957213 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4012870 con 0x7fcbd80d1020 2024-03-23T09:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.166+0000 7fcbdebb1640 1 -- 172.21.15.47:0/3861957213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd815f930 con 0x7fcbd80d1020 2024-03-23T09:51:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.167+0000 7fcbdebb1640 1 -- 172.21.15.47:0/3861957213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd8072360 con 0x7fcbd80d1020 2024-03-23T09:51:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.167+0000 7fcbd57fa640 1 -- 172.21.15.47:0/3861957213 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcbc40129d0 con 0x7fcbd80d1020 2024-03-23T09:51:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.168+0000 7fcbd57fa640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcbbc06ac00 0x7fcbbc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.168+0000 7fcbd57fa640 1 -- 172.21.15.47:0/3861957213 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbc4088e40 con 0x7fcbd80d1020 2024-03-23T09:51:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.168+0000 7fcbdc94d640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcbbc06ac00 0x7fcbbc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.169+0000 7fcbdc94d640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcbbc06ac00 0x7fcbbc06d0c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc036000 tx=0x7fcbcc03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.172+0000 7fcbd57fa640 1 -- 172.21.15.47:0/3861957213 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcbc40540b0 con 0x7fcbd80d1020 2024-03-23T09:51:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.386+0000 7fcbdebb1640 1 -- 172.21.15.47:0/3861957213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbd8066be0 con 0x7fcbd80d1020 2024-03-23T09:51:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.386+0000 7fcbd57fa640 1 -- 172.21.15.47:0/3861957213 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4733 (secure 0 0 0) 0x7fcbc40576d0 con 0x7fcbd80d1020 2024-03-23T09:51:43.387 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:51:40.395539+0000","modified":"2024-03-23T09:51:43.030014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9234":{"gid":9234,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6834/724782441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":724782441},{"type":"v1","addr":"172.21.15.47:6834","nonce":724782441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-23T09:51:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 80 2024-03-23T09:51:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.388+0000 7fcbdebb1640 1 -- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcbbc06ac00 msgr2=0x7fcbbc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.388+0000 7fcbdebb1640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcbbc06ac00 0x7fcbbc06d0c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc036000 tx=0x7fcbcc03c040 comp rx=0 tx=0).stop 2024-03-23T09:51:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.388+0000 7fcbdebb1640 1 -- 172.21.15.47:0/3861957213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd80d1020 msgr2=0x7fcbd80c9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.388+0000 7fcbdebb1640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd80d1020 0x7fcbd80c9a30 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400c480 tx=0x7fcbc400c940 comp rx=0 tx=0).stop 2024-03-23T09:51:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.388+0000 7fcbdebb1640 1 -- 172.21.15.47:0/3861957213 shutdown_connections 2024-03-23T09:51:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.388+0000 7fcbdebb1640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcbbc06ac00 0x7fcbbc06d0c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.388+0000 7fcbdebb1640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbd815cd00 0x7fcbd8144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.388+0000 7fcbdebb1640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd80d1020 0x7fcbd80c9a30 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.388+0000 7fcbdebb1640 1 --2- 172.21.15.47:0/3861957213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd80cf920 0x7fcbd81456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.388+0000 7fcbdebb1640 1 -- 172.21.15.47:0/3861957213 >> 172.21.15.47:0/3861957213 conn(0x7fcbd806b160 msgr2=0x7fcbd80b7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.388+0000 7fcbdebb1640 1 -- 172.21.15.47:0/3861957213 shutdown_connections 2024-03-23T09:51:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.388+0000 7fcbdebb1640 1 -- 172.21.15.47:0/3861957213 wait complete. 2024-03-23T09:51:43.399 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:51:43.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.499+0000 7fbf16de7640 1 Processor -- start 2024-03-23T09:51:43.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.499+0000 7fbf16de7640 1 -- start start 2024-03-23T09:51:43.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf16de7640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbf1015cee0 0x7fbf100d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf16de7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf100d23a0 0x7fbf100d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf16de7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf100b8300 0x7fbf100d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf16de7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf100d1300 con 0x7fbf100d23a0 2024-03-23T09:51:43.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf16de7640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf100d15e0 con 0x7fbf1015cee0 2024-03-23T09:51:43.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf16de7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100d18c0 con 0x7fbf100b8300 2024-03-23T09:51:43.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf0ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf100d23a0 0x7fbf100d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf0ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf100d23a0 0x7fbf100d2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54528/0 (socket says 172.21.15.47:54528) 2024-03-23T09:51:43.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf0ffff640 1 -- 172.21.15.47:0/825916602 learned_addr learned my addr 172.21.15.47:0/825916602 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:43.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf15384640 1 --2- 172.21.15.47:0/825916602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf100b8300 0x7fbf100d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf14b83640 1 --2- 172.21.15.47:0/825916602 >> v2:172.21.15.175:3300/0 conn(0x7fbf1015cee0 0x7fbf100d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf0ffff640 1 -- 172.21.15.47:0/825916602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf100b8300 msgr2=0x7fbf100d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf0ffff640 1 --2- 172.21.15.47:0/825916602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf100b8300 0x7fbf100d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf0ffff640 1 -- 172.21.15.47:0/825916602 >> v2:172.21.15.175:3300/0 conn(0x7fbf1015cee0 msgr2=0x7fbf100d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf0ffff640 1 --2- 172.21.15.47:0/825916602 >> v2:172.21.15.175:3300/0 conn(0x7fbf1015cee0 0x7fbf100d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.500+0000 7fbf0ffff640 1 -- 172.21.15.47:0/825916602 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100d1ba0 con 0x7fbf100d23a0 2024-03-23T09:51:43.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.501+0000 7fbf0ffff640 1 --2- 172.21.15.47:0/825916602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf100d23a0 0x7fbf100d2870 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fbf100d3430 tx=0x7fbf0002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150490e169759ce5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:43.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.501+0000 7fbf0f7fe640 1 -- 172.21.15.47:0/825916602 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0002ca70 con 0x7fbf100d23a0 2024-03-23T09:51:43.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.501+0000 7fbf0f7fe640 1 -- 172.21.15.47:0/825916602 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0002cbd0 con 0x7fbf100d23a0 2024-03-23T09:51:43.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.501+0000 7fbf0f7fe640 1 -- 172.21.15.47:0/825916602 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf000343f0 con 0x7fbf100d23a0 2024-03-23T09:51:43.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.502+0000 7fbf16de7640 1 -- 172.21.15.47:0/825916602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf100d23a0 msgr2=0x7fbf100d2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.502+0000 7fbf16de7640 1 --2- 172.21.15.47:0/825916602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf100d23a0 0x7fbf100d2870 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fbf100d3430 tx=0x7fbf0002bed0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.502+0000 7fbf16de7640 1 -- 172.21.15.47:0/825916602 shutdown_connections 2024-03-23T09:51:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.502+0000 7fbf16de7640 1 --2- 172.21.15.47:0/825916602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf100b8300 0x7fbf100d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.502+0000 7fbf16de7640 1 --2- 172.21.15.47:0/825916602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf100d23a0 0x7fbf100d2870 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.502+0000 7fbf16de7640 1 --2- 172.21.15.47:0/825916602 >> v2:172.21.15.175:3300/0 conn(0x7fbf1015cee0 0x7fbf100d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.502+0000 7fbf16de7640 1 -- 172.21.15.47:0/825916602 >> 172.21.15.47:0/825916602 conn(0x7fbf10065df0 msgr2=0x7fbf10066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.502+0000 7fbf16de7640 1 -- 172.21.15.47:0/825916602 shutdown_connections 2024-03-23T09:51:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.502+0000 7fbf16de7640 1 -- 172.21.15.47:0/825916602 wait complete. 2024-03-23T09:51:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.503+0000 7fbf16de7640 1 Processor -- start 2024-03-23T09:51:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.503+0000 7fbf16de7640 1 -- start start 2024-03-23T09:51:43.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf16de7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf100b8300 0x7fbf10152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf16de7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf100d23a0 0x7fbf1015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf16de7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf1015cee0 0x7fbf100c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf16de7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100c84b0 con 0x7fbf1015cee0 2024-03-23T09:51:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf16de7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf10153460 con 0x7fbf100b8300 2024-03-23T09:51:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf16de7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100c7830 con 0x7fbf100d23a0 2024-03-23T09:51:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf15384640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf1015cee0 0x7fbf100c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf15384640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf1015cee0 0x7fbf100c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54540/0 (socket says 172.21.15.47:54540) 2024-03-23T09:51:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf15384640 1 -- 172.21.15.47:0/1346053685 learned_addr learned my addr 172.21.15.47:0/1346053685 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf14b83640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf100b8300 0x7fbf10152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf0ffff640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf100d23a0 0x7fbf1015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf15384640 1 -- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf100d23a0 msgr2=0x7fbf1015db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf15384640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf100d23a0 0x7fbf1015db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf15384640 1 -- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf100b8300 msgr2=0x7fbf10152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf15384640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf100b8300 0x7fbf10152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.504+0000 7fbf15384640 1 -- 172.21.15.47:0/1346053685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf00009b30 con 0x7fbf1015cee0 2024-03-23T09:51:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.505+0000 7fbf15384640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf1015cee0 0x7fbf100c88d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fbef800d5f0 tx=0x7fbef800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.505+0000 7fbf0d7fa640 1 -- 172.21.15.47:0/1346053685 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80125f0 con 0x7fbf1015cee0 2024-03-23T09:51:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.505+0000 7fbf16de7640 1 -- 172.21.15.47:0/1346053685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf100c7b10 con 0x7fbf1015cee0 2024-03-23T09:51:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.505+0000 7fbf16de7640 1 -- 172.21.15.47:0/1346053685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf100c80d0 con 0x7fbf1015cee0 2024-03-23T09:51:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.505+0000 7fbf0d7fa640 1 -- 172.21.15.47:0/1346053685 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef8012790 con 0x7fbf1015cee0 2024-03-23T09:51:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.505+0000 7fbf0d7fa640 1 -- 172.21.15.47:0/1346053685 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8012b40 con 0x7fbf1015cee0 2024-03-23T09:51:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.506+0000 7fbf16de7640 1 -- 172.21.15.47:0/1346053685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf10072360 con 0x7fbf1015cee0 2024-03-23T09:51:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.506+0000 7fbf0d7fa640 1 -- 172.21.15.47:0/1346053685 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbef8012ca0 con 0x7fbf1015cee0 2024-03-23T09:51:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.507+0000 7fbf0d7fa640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbef006ac00 0x7fbef006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.507+0000 7fbf0d7fa640 1 -- 172.21.15.47:0/1346053685 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbef8089900 con 0x7fbf1015cee0 2024-03-23T09:51:43.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.507+0000 7fbf14b83640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbef006ac00 0x7fbef006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.508+0000 7fbf14b83640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbef006ac00 0x7fbef006d0c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800b7c0 tx=0x7fbf08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:43.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.511+0000 7fbf0d7fa640 1 -- 172.21.15.47:0/1346053685 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbef8054a50 con 0x7fbf1015cee0 2024-03-23T09:51:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.690+0000 7fbf16de7640 1 -- 172.21.15.47:0/1346053685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf100c85f0 con 0x7fbf1015cee0 2024-03-23T09:51:43.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.691+0000 7fbf0d7fa640 1 -- 172.21.15.47:0/1346053685 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16059 (secure 0 0 0) 0x7fbf100c85f0 con 0x7fbf1015cee0 2024-03-23T09:51:43.692 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:43.692 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":114,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:51:41.397088+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:51:38.679890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-23T09:51:39.396299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:51:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.693+0000 7fbf16de7640 1 -- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbef006ac00 msgr2=0x7fbef006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.693+0000 7fbf16de7640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbef006ac00 0x7fbef006d0c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800b7c0 tx=0x7fbf08008040 comp rx=0 tx=0).stop 2024-03-23T09:51:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.693+0000 7fbf16de7640 1 -- 172.21.15.47:0/1346053685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf1015cee0 msgr2=0x7fbf100c88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.693+0000 7fbf16de7640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf1015cee0 0x7fbf100c88d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fbef800d5f0 tx=0x7fbef800dab0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.693+0000 7fbf16de7640 1 -- 172.21.15.47:0/1346053685 shutdown_connections 2024-03-23T09:51:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.693+0000 7fbf16de7640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbef006ac00 0x7fbef006d0c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.693+0000 7fbf16de7640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf1015cee0 0x7fbf100c88d0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.693+0000 7fbf16de7640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf100d23a0 0x7fbf1015db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.693+0000 7fbf16de7640 1 --2- 172.21.15.47:0/1346053685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf100b8300 0x7fbf10152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.693+0000 7fbf16de7640 1 -- 172.21.15.47:0/1346053685 >> 172.21.15.47:0/1346053685 conn(0x7fbf10065df0 msgr2=0x7fbf100b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.693+0000 7fbf16de7640 1 -- 172.21.15.47:0/1346053685 shutdown_connections 2024-03-23T09:51:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.693+0000 7fbf16de7640 1 -- 172.21.15.47:0/1346053685 wait complete. 2024-03-23T09:51:43.705 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T09:51:43.705 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:51:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.806+0000 7f8eaf451640 1 Processor -- start 2024-03-23T09:51:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.806+0000 7f8eaf451640 1 -- start start 2024-03-23T09:51:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.807+0000 7f8eaf451640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea815cb40 0x7f8ea8160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.807+0000 7f8eaf451640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8ea8160cb0 0x7f8ea80d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.807+0000 7f8eaf451640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea80d0ee0 con 0x7f8ea80b8300 2024-03-23T09:51:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.807+0000 7f8eaf451640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea80b5a10 con 0x7f8ea8160cb0 2024-03-23T09:51:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.807+0000 7f8eaf451640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81616f0 con 0x7f8ea815cb40 2024-03-23T09:51:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.807+0000 7f8eac9ec640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8ea80b8300 0x7f8ea80cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52424/0 (socket says 172.21.15.47:52424) 2024-03-23T09:51:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.807+0000 7f8eac9ec640 1 -- 172.21.15.47:0/605407832 learned_addr learned my addr 172.21.15.47:0/605407832 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.807+0000 7f8ead9ee640 1 --2- 172.21.15.47:0/605407832 >> v2:172.21.15.175:3300/0 conn(0x7f8ea8160cb0 0x7f8ea80d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.807+0000 7f8ead1ed640 1 --2- 172.21.15.47:0/605407832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea815cb40 0x7f8ea8160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.807+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970239863 0 0) 0x7f8ea80d0ee0 con 0x7f8ea80b8300 2024-03-23T09:51:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.807+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e94003520 con 0x7f8ea80b8300 2024-03-23T09:51:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.807+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2461712966 0 0) 0x7f8e94003520 con 0x7f8ea80b8300 2024-03-23T09:51:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea815cb40 msgr2=0x7f8ea8160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8e9ffff640 1 --2- 172.21.15.47:0/605407832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea815cb40 0x7f8ea8160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 >> v2:172.21.15.175:3300/0 conn(0x7f8ea8160cb0 msgr2=0x7f8ea80d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8e9ffff640 1 --2- 172.21.15.47:0/605407832 >> v2:172.21.15.175:3300/0 conn(0x7f8ea8160cb0 0x7f8ea80d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea8161130 con 0x7f8ea80b8300 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f8e98001670 con 0x7f8ea80b8300 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8ead9ee640 1 --2- 172.21.15.47:0/605407832 >> v2:172.21.15.175:3300/0 conn(0x7f8ea8160cb0 0x7f8ea80d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 >> v1:172.21.15.47:6789/0 conn(0x7f8ea80b8300 legacy=0x7f8ea80cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8e9ffff640 1 --2- 172.21.15.47:0/605407832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e94003970 0x7f8e94003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8e9ffff640 1 --2- 172.21.15.47:0/605407832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e940047c0 0x7f8e94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8e9ffff640 1 --2- 172.21.15.47:0/605407832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea815cb40 0x7f8e94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea80b5a10 con 0x7f8e940047c0 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81616f0 con 0x7f8e94003970 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9400b980 con 0x7f8ea815cb40 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8eac9ec640 1 --2- 172.21.15.47:0/605407832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e940047c0 0x7f8e94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8ead1ed640 1 --2- 172.21.15.47:0/605407832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e94003970 0x7f8e94003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8ead9ee640 1 --2- 172.21.15.47:0/605407832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea815cb40 0x7f8e94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8eac9ec640 1 -- 172.21.15.47:0/605407832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea815cb40 msgr2=0x7f8e94005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8eac9ec640 1 --2- 172.21.15.47:0/605407832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea815cb40 0x7f8e94005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8eac9ec640 1 -- 172.21.15.47:0/605407832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e94003970 msgr2=0x7f8e94003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8eac9ec640 1 --2- 172.21.15.47:0/605407832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e94003970 0x7f8e94003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.808+0000 7f8eac9ec640 1 -- 172.21.15.47:0/605407832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e94003520 con 0x7f8e940047c0 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.809+0000 7f8eac9ec640 1 --2- 172.21.15.47:0/605407832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e940047c0 0x7f8e94004c40 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80b57e0 tx=0x7f8e940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f1d6b178c738bf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.809+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98005ad0 con 0x7f8e940047c0 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.809+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e98005c30 con 0x7f8e940047c0 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.809+0000 7f8e9ffff640 1 -- 172.21.15.47:0/605407832 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9802c890 con 0x7f8e940047c0 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.809+0000 7f8eaf451640 1 -- 172.21.15.47:0/605407832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e940047c0 msgr2=0x7f8e94004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.809+0000 7f8eaf451640 1 --2- 172.21.15.47:0/605407832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e940047c0 0x7f8e94004c40 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80b57e0 tx=0x7f8e940057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.809+0000 7f8ead9ee640 1 -- 172.21.15.47:0/605407832 reap_dead start 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.810+0000 7f8eaf451640 1 -- 172.21.15.47:0/605407832 shutdown_connections 2024-03-23T09:51:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.810+0000 7f8eaf451640 1 -- 172.21.15.47:0/605407832 >> 172.21.15.47:0/605407832 conn(0x7f8ea806b160 msgr2=0x7f8ea806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:43.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.810+0000 7f8eaf451640 1 -- 172.21.15.47:0/605407832 shutdown_connections 2024-03-23T09:51:43.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.810+0000 7f8eaf451640 1 -- 172.21.15.47:0/605407832 wait complete. 2024-03-23T09:51:43.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.811+0000 7f8eaf451640 1 Processor -- start 2024-03-23T09:51:43.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.811+0000 7f8eaf451640 1 -- start start 2024-03-23T09:51:43.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8eaf451640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e940047c0 0x7f8ea8151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8eaf451640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea8153b50 0x7f8ea815ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8eaf451640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea815f210 0x7f8ea815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8eaf451640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81522f0 con 0x7f8e940047c0 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8eaf451640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea815f690 con 0x7f8ea815f210 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8eaf451640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea8150ea0 con 0x7f8ea8153b50 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8ead1ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e940047c0 0x7f8ea8151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8ead1ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e940047c0 0x7f8ea8151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54568/0 (socket says 172.21.15.47:54568) 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8ead1ed640 1 -- 172.21.15.47:0/2655055535 learned_addr learned my addr 172.21.15.47:0/2655055535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8eac9ec640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea8153b50 0x7f8ea815ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8ead9ee640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea815f210 0x7f8ea815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8ead1ed640 1 -- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea8153b50 msgr2=0x7f8ea815ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8ead1ed640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea8153b50 0x7f8ea815ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8ead1ed640 1 -- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea815f210 msgr2=0x7f8ea815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8ead1ed640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea815f210 0x7f8ea815aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8ead1ed640 1 -- 172.21.15.47:0/2655055535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea815b6f0 con 0x7f8e940047c0 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8eac9ec640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea8153b50 0x7f8ea815ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:43.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8ead9ee640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea815f210 0x7f8ea815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:43.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8ead1ed640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e940047c0 0x7f8ea8151db0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0011910 tx=0x7f8ea0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:43.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8e9dffb640 1 -- 172.21.15.47:0/2655055535 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000faf0 con 0x7f8e940047c0 2024-03-23T09:51:43.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8e9dffb640 1 -- 172.21.15.47:0/2655055535 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea000fc50 con 0x7f8e940047c0 2024-03-23T09:51:43.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8eaf451640 1 -- 172.21.15.47:0/2655055535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea8158660 con 0x7f8e940047c0 2024-03-23T09:51:43.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.812+0000 7f8e9dffb640 1 -- 172.21.15.47:0/2655055535 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0014440 con 0x7f8e940047c0 2024-03-23T09:51:43.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.813+0000 7f8eaf451640 1 -- 172.21.15.47:0/2655055535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea8158c20 con 0x7f8e940047c0 2024-03-23T09:51:43.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.814+0000 7f8eaf451640 1 -- 172.21.15.47:0/2655055535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea8072360 con 0x7f8e940047c0 2024-03-23T09:51:43.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.814+0000 7f8e9dffb640 1 -- 172.21.15.47:0/2655055535 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8ea001d400 con 0x7f8e940047c0 2024-03-23T09:51:43.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.814+0000 7f8e9dffb640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e8c06ac00 0x7f8e8c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:43.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.815+0000 7f8e9dffb640 1 -- 172.21.15.47:0/2655055535 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ea0089b60 con 0x7f8e940047c0 2024-03-23T09:51:43.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.815+0000 7f8eac9ec640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e8c06ac00 0x7f8e8c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:43.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.815+0000 7f8eac9ec640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e8c06ac00 0x7f8e8c06d0c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8e98034000 tx=0x7f8e98007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.819+0000 7f8e9dffb640 1 -- 172.21.15.47:0/2655055535 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8ea0054cb0 con 0x7f8e940047c0 2024-03-23T09:51:43.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:43.994+0000 7f8eaf451640 1 -- 172.21.15.47:0/2655055535 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ea80d0b70 con 0x7f8e8c06ac00 2024-03-23T09:51:44.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.002+0000 7f8e9dffb640 1 -- 172.21.15.47:0/2655055535 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290165 (secure 0 0 0) 0x7f8e98001650 con 0x7f8e8c06ac00 2024-03-23T09:51:44.005 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:51:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.005+0000 7f8eaf451640 1 -- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e8c06ac00 msgr2=0x7f8e8c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.005+0000 7f8eaf451640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e8c06ac00 0x7f8e8c06d0c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8e98034000 tx=0x7f8e98007000 comp rx=0 tx=0).stop 2024-03-23T09:51:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.005+0000 7f8eaf451640 1 -- 172.21.15.47:0/2655055535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e940047c0 msgr2=0x7f8ea8151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.005+0000 7f8eaf451640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e940047c0 0x7f8ea8151db0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0011910 tx=0x7f8ea0011dd0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.005+0000 7f8eaf451640 1 -- 172.21.15.47:0/2655055535 shutdown_connections 2024-03-23T09:51:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.005+0000 7f8eaf451640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8e8c06ac00 0x7f8e8c06d0c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.005+0000 7f8eaf451640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea815f210 0x7f8ea815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.005+0000 7f8eaf451640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ea8153b50 0x7f8ea815ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.005+0000 7f8eaf451640 1 --2- 172.21.15.47:0/2655055535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e940047c0 0x7f8ea8151db0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.005+0000 7f8eaf451640 1 -- 172.21.15.47:0/2655055535 >> 172.21.15.47:0/2655055535 conn(0x7f8ea806b160 msgr2=0x7f8ea815bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.005+0000 7f8eaf451640 1 -- 172.21.15.47:0/2655055535 shutdown_connections 2024-03-23T09:51:44.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.005+0000 7f8eaf451640 1 -- 172.21.15.47:0/2655055535 wait complete. 2024-03-23T09:51:44.016 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-03-23T09:51:42.500407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":273568,"kb_used_data":12288,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754701152,"statfs":{"total":773094113280,"available":772813979648,"internally_reserved":0,"allocated":12582912,"data_stored":6090662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":267374136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000864"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463517+0000","last_change":"2024-03-23T09:51:40.421426+0000","last_active":"2024-03-23T09:51:41.463517+0000","last_peered":"2024-03-23T09:51:41.463517+0000","last_clean":"2024-03-23T09:51:41.463517+0000","last_became_active":"2024-03-23T09:51:40.421186+0000","last_became_peered":"2024-03-23T09:51:40.421186+0000","last_unstale":"2024-03-23T09:51:41.463517+0000","last_undegraded":"2024-03-23T09:51:41.463517+0000","last_fullsized":"2024-03-23T09:51:41.463517+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:52:06.459960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403618+0000","last_change":"2024-03-23T09:51:40.410958+0000","last_active":"2024-03-23T09:51:41.403618+0000","last_peered":"2024-03-23T09:51:41.403618+0000","last_clean":"2024-03-23T09:51:41.403618+0000","last_became_active":"2024-03-23T09:51:40.410685+0000","last_became_peered":"2024-03-23T09:51:40.410685+0000","last_unstale":"2024-03-23T09:51:41.403618+0000","last_undegraded":"2024-03-23T09:51:41.403618+0000","last_fullsized":"2024-03-23T09:51:41.403618+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:08:17.110031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404077+0000","last_change":"2024-03-23T09:51:40.416348+0000","last_active":"2024-03-23T09:51:41.404077+0000","last_peered":"2024-03-23T09:51:41.404077+0000","last_clean":"2024-03-23T09:51:41.404077+0000","last_became_active":"2024-03-23T09:51:40.414713+0000","last_became_peered":"2024-03-23T09:51:40.414713+0000","last_unstale":"2024-03-23T09:51:41.404077+0000","last_undegraded":"2024-03-23T09:51:41.404077+0000","last_fullsized":"2024-03-23T09:51:41.404077+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:15:20.760615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463880+0000","last_change":"2024-03-23T09:51:40.422802+0000","last_active":"2024-03-23T09:51:41.463880+0000","last_peered":"2024-03-23T09:51:41.463880+0000","last_clean":"2024-03-23T09:51:41.463880+0000","last_became_active":"2024-03-23T09:51:40.422513+0000","last_became_peered":"2024-03-23T09:51:40.422513+0000","last_unstale":"2024-03-23T09:51:41.463880+0000","last_undegraded":"2024-03-23T09:51:41.463880+0000","last_fullsized":"2024-03-23T09:51:41.463880+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:08:07.025033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402629+0000","last_change":"2024-03-23T09:51:40.407521+0000","last_active":"2024-03-23T09:51:41.402629+0000","last_peered":"2024-03-23T09:51:41.402629+0000","last_clean":"2024-03-23T09:51:41.402629+0000","last_became_active":"2024-03-23T09:51:40.407256+0000","last_became_peered":"2024-03-23T09:51:40.407256+0000","last_unstale":"2024-03-23T09:51:41.402629+0000","last_undegraded":"2024-03-23T09:51:41.402629+0000","last_fullsized":"2024-03-23T09:51:41.402629+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:36:03.125410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463236+0000","last_change":"2024-03-23T09:51:40.412120+0000","last_active":"2024-03-23T09:51:41.463236+0000","last_peered":"2024-03-23T09:51:41.463236+0000","last_clean":"2024-03-23T09:51:41.463236+0000","last_became_active":"2024-03-23T09:51:40.411874+0000","last_became_peered":"2024-03-23T09:51:40.411874+0000","last_unstale":"2024-03-23T09:51:41.463236+0000","last_undegraded":"2024-03-23T09:51:41.463236+0000","last_fullsized":"2024-03-23T09:51:41.463236+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:52:39.694132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463630+0000","last_change":"2024-03-23T09:51:40.415769+0000","last_active":"2024-03-23T09:51:41.463630+0000","last_peered":"2024-03-23T09:51:41.463630+0000","last_clean":"2024-03-23T09:51:41.463630+0000","last_became_active":"2024-03-23T09:51:40.415505+0000","last_became_peered":"2024-03-23T09:51:40.415505+0000","last_unstale":"2024-03-23T09:51:41.463630+0000","last_undegraded":"2024-03-23T09:51:41.463630+0000","last_fullsized":"2024-03-23T09:51:41.463630+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:59:02.734957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403361+0000","last_change":"2024-03-23T09:51:40.416977+0000","last_active":"2024-03-23T09:51:41.403361+0000","last_peered":"2024-03-23T09:51:41.403361+0000","last_clean":"2024-03-23T09:51:41.403361+0000","last_became_active":"2024-03-23T09:51:40.416548+0000","last_became_peered":"2024-03-23T09:51:40.416548+0000","last_unstale":"2024-03-23T09:51:41.403361+0000","last_undegraded":"2024-03-23T09:51:41.403361+0000","last_fullsized":"2024-03-23T09:51:41.403361+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:21:51.075338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463546+0000","last_change":"2024-03-23T09:51:40.420869+0000","last_active":"2024-03-23T09:51:41.463546+0000","last_peered":"2024-03-23T09:51:41.463546+0000","last_clean":"2024-03-23T09:51:41.463546+0000","last_became_active":"2024-03-23T09:51:40.420615+0000","last_became_peered":"2024-03-23T09:51:40.420615+0000","last_unstale":"2024-03-23T09:51:41.463546+0000","last_undegraded":"2024-03-23T09:51:41.463546+0000","last_fullsized":"2024-03-23T09:51:41.463546+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:40:19.870031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463611+0000","last_change":"2024-03-23T09:51:40.420446+0000","last_active":"2024-03-23T09:51:41.463611+0000","last_peered":"2024-03-23T09:51:41.463611+0000","last_clean":"2024-03-23T09:51:41.463611+0000","last_became_active":"2024-03-23T09:51:40.420182+0000","last_became_peered":"2024-03-23T09:51:40.420182+0000","last_unstale":"2024-03-23T09:51:41.463611+0000","last_undegraded":"2024-03-23T09:51:41.463611+0000","last_fullsized":"2024-03-23T09:51:41.463611+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:48.481687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403560+0000","last_change":"2024-03-23T09:51:40.412690+0000","last_active":"2024-03-23T09:51:41.403560+0000","last_peered":"2024-03-23T09:51:41.403560+0000","last_clean":"2024-03-23T09:51:41.403560+0000","last_became_active":"2024-03-23T09:51:40.412419+0000","last_became_peered":"2024-03-23T09:51:40.412419+0000","last_unstale":"2024-03-23T09:51:41.403560+0000","last_undegraded":"2024-03-23T09:51:41.403560+0000","last_fullsized":"2024-03-23T09:51:41.403560+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:12:46.572433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463952+0000","last_change":"2024-03-23T09:51:40.421571+0000","last_active":"2024-03-23T09:51:41.463952+0000","last_peered":"2024-03-23T09:51:41.463952+0000","last_clean":"2024-03-23T09:51:41.463952+0000","last_became_active":"2024-03-23T09:51:40.421319+0000","last_became_peered":"2024-03-23T09:51:40.421319+0000","last_unstale":"2024-03-23T09:51:41.463952+0000","last_undegraded":"2024-03-23T09:51:41.463952+0000","last_fullsized":"2024-03-23T09:51:41.463952+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:40:36.388768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404063+0000","last_change":"2024-03-23T09:51:40.408493+0000","last_active":"2024-03-23T09:51:41.404063+0000","last_peered":"2024-03-23T09:51:41.404063+0000","last_clean":"2024-03-23T09:51:41.404063+0000","last_became_active":"2024-03-23T09:51:40.408219+0000","last_became_peered":"2024-03-23T09:51:40.408219+0000","last_unstale":"2024-03-23T09:51:41.404063+0000","last_undegraded":"2024-03-23T09:51:41.404063+0000","last_fullsized":"2024-03-23T09:51:41.404063+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:35:26.463429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403856+0000","last_change":"2024-03-23T09:51:40.410894+0000","last_active":"2024-03-23T09:51:41.403856+0000","last_peered":"2024-03-23T09:51:41.403856+0000","last_clean":"2024-03-23T09:51:41.403856+0000","last_became_active":"2024-03-23T09:51:40.410472+0000","last_became_peered":"2024-03-23T09:51:40.410472+0000","last_unstale":"2024-03-23T09:51:41.403856+0000","last_undegraded":"2024-03-23T09:51:41.403856+0000","last_fullsized":"2024-03-23T09:51:41.403856+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:57:09.681811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463018+0000","last_change":"2024-03-23T09:51:40.412237+0000","last_active":"2024-03-23T09:51:41.463018+0000","last_peered":"2024-03-23T09:51:41.463018+0000","last_clean":"2024-03-23T09:51:41.463018+0000","last_became_active":"2024-03-23T09:51:40.411998+0000","last_became_peered":"2024-03-23T09:51:40.411998+0000","last_unstale":"2024-03-23T09:51:41.463018+0000","last_undegraded":"2024-03-23T09:51:41.463018+0000","last_fullsized":"2024-03-23T09:51:41.463018+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:23:50.334771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462889+0000","last_change":"2024-03-23T09:51:40.424475+0000","last_active":"2024-03-23T09:51:41.462889+0000","last_peered":"2024-03-23T09:51:41.462889+0000","last_clean":"2024-03-23T09:51:41.462889+0000","last_became_active":"2024-03-23T09:51:40.424221+0000","last_became_peered":"2024-03-23T09:51:40.424221+0000","last_unstale":"2024-03-23T09:51:41.462889+0000","last_undegraded":"2024-03-23T09:51:41.462889+0000","last_fullsized":"2024-03-23T09:51:41.462889+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:17:25.600706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404075+0000","last_change":"2024-03-23T09:51:40.415022+0000","last_active":"2024-03-23T09:51:41.404075+0000","last_peered":"2024-03-23T09:51:41.404075+0000","last_clean":"2024-03-23T09:51:41.404075+0000","last_became_active":"2024-03-23T09:51:40.414436+0000","last_became_peered":"2024-03-23T09:51:40.414436+0000","last_unstale":"2024-03-23T09:51:41.404075+0000","last_undegraded":"2024-03-23T09:51:41.404075+0000","last_fullsized":"2024-03-23T09:51:41.404075+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:51:23.039272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463344+0000","last_change":"2024-03-23T09:51:40.414029+0000","last_active":"2024-03-23T09:51:41.463344+0000","last_peered":"2024-03-23T09:51:41.463344+0000","last_clean":"2024-03-23T09:51:41.463344+0000","last_became_active":"2024-03-23T09:51:40.413652+0000","last_became_peered":"2024-03-23T09:51:40.413652+0000","last_unstale":"2024-03-23T09:51:41.463344+0000","last_undegraded":"2024-03-23T09:51:41.463344+0000","last_fullsized":"2024-03-23T09:51:41.463344+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:35:26.463429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404224+0000","last_change":"2024-03-23T09:51:40.409377+0000","last_active":"2024-03-23T09:51:41.404224+0000","last_peered":"2024-03-23T09:51:41.404224+0000","last_clean":"2024-03-23T09:51:41.404224+0000","last_became_active":"2024-03-23T09:51:40.409049+0000","last_became_peered":"2024-03-23T09:51:40.409049+0000","last_unstale":"2024-03-23T09:51:41.404224+0000","last_undegraded":"2024-03-23T09:51:41.404224+0000","last_fullsized":"2024-03-23T09:51:41.404224+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:29:13.794262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404074+0000","last_change":"2024-03-23T09:51:40.409221+0000","last_active":"2024-03-23T09:51:41.404074+0000","last_peered":"2024-03-23T09:51:41.404074+0000","last_clean":"2024-03-23T09:51:41.404074+0000","last_became_active":"2024-03-23T09:51:40.408944+0000","last_became_peered":"2024-03-23T09:51:40.408944+0000","last_unstale":"2024-03-23T09:51:41.404074+0000","last_undegraded":"2024-03-23T09:51:41.404074+0000","last_fullsized":"2024-03-23T09:51:41.404074+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:08:01.286016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403914+0000","last_change":"2024-03-23T09:51:40.412732+0000","last_active":"2024-03-23T09:51:41.403914+0000","last_peered":"2024-03-23T09:51:41.403914+0000","last_clean":"2024-03-23T09:51:41.403914+0000","last_became_active":"2024-03-23T09:51:40.412471+0000","last_became_peered":"2024-03-23T09:51:40.412471+0000","last_unstale":"2024-03-23T09:51:41.403914+0000","last_undegraded":"2024-03-23T09:51:41.403914+0000","last_fullsized":"2024-03-23T09:51:41.403914+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:29:13.794262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464103+0000","last_change":"2024-03-23T09:51:40.413442+0000","last_active":"2024-03-23T09:51:41.464103+0000","last_peered":"2024-03-23T09:51:41.464103+0000","last_clean":"2024-03-23T09:51:41.464103+0000","last_became_active":"2024-03-23T09:51:40.412891+0000","last_became_peered":"2024-03-23T09:51:40.412891+0000","last_unstale":"2024-03-23T09:51:41.464103+0000","last_undegraded":"2024-03-23T09:51:41.464103+0000","last_fullsized":"2024-03-23T09:51:41.464103+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:42:30.671652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404192+0000","last_change":"2024-03-23T09:51:40.405226+0000","last_active":"2024-03-23T09:51:41.404192+0000","last_peered":"2024-03-23T09:51:41.404192+0000","last_clean":"2024-03-23T09:51:41.404192+0000","last_became_active":"2024-03-23T09:51:40.404963+0000","last_became_peered":"2024-03-23T09:51:40.404963+0000","last_unstale":"2024-03-23T09:51:41.404192+0000","last_undegraded":"2024-03-23T09:51:41.404192+0000","last_fullsized":"2024-03-23T09:51:41.404192+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:57:09.681811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462633+0000","last_change":"2024-03-23T09:51:40.409602+0000","last_active":"2024-03-23T09:51:41.462633+0000","last_peered":"2024-03-23T09:51:41.462633+0000","last_clean":"2024-03-23T09:51:41.462633+0000","last_became_active":"2024-03-23T09:51:40.409314+0000","last_became_peered":"2024-03-23T09:51:40.409314+0000","last_unstale":"2024-03-23T09:51:41.462633+0000","last_undegraded":"2024-03-23T09:51:41.462633+0000","last_fullsized":"2024-03-23T09:51:41.462633+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:09:00.964135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404163+0000","last_change":"2024-03-23T09:51:40.415280+0000","last_active":"2024-03-23T09:51:41.404163+0000","last_peered":"2024-03-23T09:51:41.404163+0000","last_clean":"2024-03-23T09:51:41.404163+0000","last_became_active":"2024-03-23T09:51:40.414768+0000","last_became_peered":"2024-03-23T09:51:40.414768+0000","last_unstale":"2024-03-23T09:51:41.404163+0000","last_undegraded":"2024-03-23T09:51:41.404163+0000","last_fullsized":"2024-03-23T09:51:41.404163+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:15:20.760615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463664+0000","last_change":"2024-03-23T09:51:40.420428+0000","last_active":"2024-03-23T09:51:41.463664+0000","last_peered":"2024-03-23T09:51:41.463664+0000","last_clean":"2024-03-23T09:51:41.463664+0000","last_became_active":"2024-03-23T09:51:40.420167+0000","last_became_peered":"2024-03-23T09:51:40.420167+0000","last_unstale":"2024-03-23T09:51:41.463664+0000","last_undegraded":"2024-03-23T09:51:41.463664+0000","last_fullsized":"2024-03-23T09:51:41.463664+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:12.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463114+0000","last_change":"2024-03-23T09:51:40.425765+0000","last_active":"2024-03-23T09:51:41.463114+0000","last_peered":"2024-03-23T09:51:41.463114+0000","last_clean":"2024-03-23T09:51:41.463114+0000","last_became_active":"2024-03-23T09:51:40.425534+0000","last_became_peered":"2024-03-23T09:51:40.425534+0000","last_unstale":"2024-03-23T09:51:41.463114+0000","last_undegraded":"2024-03-23T09:51:41.463114+0000","last_fullsized":"2024-03-23T09:51:41.463114+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:45:19.556557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463990+0000","last_change":"2024-03-23T09:51:40.424735+0000","last_active":"2024-03-23T09:51:41.463990+0000","last_peered":"2024-03-23T09:51:41.463990+0000","last_clean":"2024-03-23T09:51:41.463990+0000","last_became_active":"2024-03-23T09:51:40.424488+0000","last_became_peered":"2024-03-23T09:51:40.424488+0000","last_unstale":"2024-03-23T09:51:41.463990+0000","last_undegraded":"2024-03-23T09:51:41.463990+0000","last_fullsized":"2024-03-23T09:51:41.463990+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:10:01.040378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403388+0000","last_change":"2024-03-23T09:51:40.408330+0000","last_active":"2024-03-23T09:51:41.403388+0000","last_peered":"2024-03-23T09:51:41.403388+0000","last_clean":"2024-03-23T09:51:41.403388+0000","last_became_active":"2024-03-23T09:51:40.408067+0000","last_became_peered":"2024-03-23T09:51:40.408067+0000","last_unstale":"2024-03-23T09:51:41.403388+0000","last_undegraded":"2024-03-23T09:51:41.403388+0000","last_fullsized":"2024-03-23T09:51:41.403388+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:45:02.579837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463788+0000","last_change":"2024-03-23T09:51:40.415722+0000","last_active":"2024-03-23T09:51:41.463788+0000","last_peered":"2024-03-23T09:51:41.463788+0000","last_clean":"2024-03-23T09:51:41.463788+0000","last_became_active":"2024-03-23T09:51:40.415454+0000","last_became_peered":"2024-03-23T09:51:40.415454+0000","last_unstale":"2024-03-23T09:51:41.463788+0000","last_undegraded":"2024-03-23T09:51:41.463788+0000","last_fullsized":"2024-03-23T09:51:41.463788+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:46:38.011184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463283+0000","last_change":"2024-03-23T09:51:40.421113+0000","last_active":"2024-03-23T09:51:41.463283+0000","last_peered":"2024-03-23T09:51:41.463283+0000","last_clean":"2024-03-23T09:51:41.463283+0000","last_became_active":"2024-03-23T09:51:40.420849+0000","last_became_peered":"2024-03-23T09:51:40.420849+0000","last_unstale":"2024-03-23T09:51:41.463283+0000","last_undegraded":"2024-03-23T09:51:41.463283+0000","last_fullsized":"2024-03-23T09:51:41.463283+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:38:09.397343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462959+0000","last_change":"2024-03-23T09:51:40.416051+0000","last_active":"2024-03-23T09:51:41.462959+0000","last_peered":"2024-03-23T09:51:41.462959+0000","last_clean":"2024-03-23T09:51:41.462959+0000","last_became_active":"2024-03-23T09:51:40.415621+0000","last_became_peered":"2024-03-23T09:51:40.415621+0000","last_unstale":"2024-03-23T09:51:41.462959+0000","last_undegraded":"2024-03-23T09:51:41.462959+0000","last_fullsized":"2024-03-23T09:51:41.462959+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:44:20.541433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463384+0000","last_change":"2024-03-23T09:51:40.415875+0000","last_active":"2024-03-23T09:51:41.463384+0000","last_peered":"2024-03-23T09:51:41.463384+0000","last_clean":"2024-03-23T09:51:41.463384+0000","last_became_active":"2024-03-23T09:51:40.415628+0000","last_became_peered":"2024-03-23T09:51:40.415628+0000","last_unstale":"2024-03-23T09:51:41.463384+0000","last_undegraded":"2024-03-23T09:51:41.463384+0000","last_fullsized":"2024-03-23T09:51:41.463384+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:43:15.518304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"113'4","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463881+0000","last_change":"2024-03-23T09:51:40.034507+0000","last_active":"2024-03-23T09:51:41.463881+0000","last_peered":"2024-03-23T09:51:41.463881+0000","last_clean":"2024-03-23T09:51:41.463881+0000","last_became_active":"2024-03-23T09:51:40.034258+0000","last_became_peered":"2024-03-23T09:51:40.034258+0000","last_unstale":"2024-03-23T09:51:41.463881+0000","last_undegraded":"2024-03-23T09:51:41.463881+0000","last_fullsized":"2024-03-23T09:51:41.463881+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:14:32.462197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463542+0000","last_change":"2024-03-23T09:51:40.409275+0000","last_active":"2024-03-23T09:51:41.463542+0000","last_peered":"2024-03-23T09:51:41.463542+0000","last_clean":"2024-03-23T09:51:41.463542+0000","last_became_active":"2024-03-23T09:51:40.409033+0000","last_became_peered":"2024-03-23T09:51:40.409033+0000","last_unstale":"2024-03-23T09:51:41.463542+0000","last_undegraded":"2024-03-23T09:51:41.463542+0000","last_fullsized":"2024-03-23T09:51:41.463542+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:08:17.110031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"113'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403756+0000","last_change":"2024-03-23T09:51:40.036209+0000","last_active":"2024-03-23T09:51:41.403756+0000","last_peered":"2024-03-23T09:51:41.403756+0000","last_clean":"2024-03-23T09:51:41.403756+0000","last_became_active":"2024-03-23T09:51:40.035941+0000","last_became_peered":"2024-03-23T09:51:40.035941+0000","last_unstale":"2024-03-23T09:51:41.403756+0000","last_undegraded":"2024-03-23T09:51:41.403756+0000","last_fullsized":"2024-03-23T09:51:41.403756+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:22:39.917877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463341+0000","last_change":"2024-03-23T09:51:40.412139+0000","last_active":"2024-03-23T09:51:41.463341+0000","last_peered":"2024-03-23T09:51:41.463341+0000","last_clean":"2024-03-23T09:51:41.463341+0000","last_became_active":"2024-03-23T09:51:40.411850+0000","last_became_peered":"2024-03-23T09:51:40.411850+0000","last_unstale":"2024-03-23T09:51:41.463341+0000","last_undegraded":"2024-03-23T09:51:41.463341+0000","last_fullsized":"2024-03-23T09:51:41.463341+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:36:03.125410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"113'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402586+0000","last_change":"2024-03-23T09:51:40.042156+0000","last_active":"2024-03-23T09:51:41.402586+0000","last_peered":"2024-03-23T09:51:41.402586+0000","last_clean":"2024-03-23T09:51:41.402586+0000","last_became_active":"2024-03-23T09:51:40.041847+0000","last_became_peered":"2024-03-23T09:51:40.041847+0000","last_unstale":"2024-03-23T09:51:41.402586+0000","last_undegraded":"2024-03-23T09:51:41.402586+0000","last_fullsized":"2024-03-23T09:51:41.402586+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:24:03.629632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404230+0000","last_change":"2024-03-23T09:51:40.410466+0000","last_active":"2024-03-23T09:51:41.404230+0000","last_peered":"2024-03-23T09:51:41.404230+0000","last_clean":"2024-03-23T09:51:41.404230+0000","last_became_active":"2024-03-23T09:51:40.410022+0000","last_became_peered":"2024-03-23T09:51:40.410022+0000","last_unstale":"2024-03-23T09:51:41.404230+0000","last_undegraded":"2024-03-23T09:51:41.404230+0000","last_fullsized":"2024-03-23T09:51:41.404230+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:08:17.110031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"113'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402482+0000","last_change":"2024-03-23T09:51:40.041092+0000","last_active":"2024-03-23T09:51:41.402482+0000","last_peered":"2024-03-23T09:51:41.402482+0000","last_clean":"2024-03-23T09:51:41.402482+0000","last_became_active":"2024-03-23T09:51:40.040799+0000","last_became_peered":"2024-03-23T09:51:40.040799+0000","last_unstale":"2024-03-23T09:51:41.402482+0000","last_undegraded":"2024-03-23T09:51:41.402482+0000","last_fullsized":"2024-03-23T09:51:41.402482+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:27:45.441015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463331+0000","last_change":"2024-03-23T09:51:40.420760+0000","last_active":"2024-03-23T09:51:41.463331+0000","last_peered":"2024-03-23T09:51:41.463331+0000","last_clean":"2024-03-23T09:51:41.463331+0000","last_became_active":"2024-03-23T09:51:40.420258+0000","last_became_peered":"2024-03-23T09:51:40.420258+0000","last_unstale":"2024-03-23T09:51:41.463331+0000","last_undegraded":"2024-03-23T09:51:41.463331+0000","last_fullsized":"2024-03-23T09:51:41.463331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:27:51.398073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"113'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463432+0000","last_change":"2024-03-23T09:51:40.041748+0000","last_active":"2024-03-23T09:51:41.463432+0000","last_peered":"2024-03-23T09:51:41.463432+0000","last_clean":"2024-03-23T09:51:41.463432+0000","last_became_active":"2024-03-23T09:51:40.041452+0000","last_became_peered":"2024-03-23T09:51:40.041452+0000","last_unstale":"2024-03-23T09:51:41.463432+0000","last_undegraded":"2024-03-23T09:51:41.463432+0000","last_fullsized":"2024-03-23T09:51:41.463432+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:37:19.045964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403610+0000","last_change":"2024-03-23T09:51:40.409493+0000","last_active":"2024-03-23T09:51:41.403610+0000","last_peered":"2024-03-23T09:51:41.403610+0000","last_clean":"2024-03-23T09:51:41.403610+0000","last_became_active":"2024-03-23T09:51:40.409251+0000","last_became_peered":"2024-03-23T09:51:40.409251+0000","last_unstale":"2024-03-23T09:51:41.403610+0000","last_undegraded":"2024-03-23T09:51:41.403610+0000","last_fullsized":"2024-03-23T09:51:41.403610+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:23:45.258323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"113'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464215+0000","last_change":"2024-03-23T09:51:40.042878+0000","last_active":"2024-03-23T09:51:41.464215+0000","last_peered":"2024-03-23T09:51:41.464215+0000","last_clean":"2024-03-23T09:51:41.464215+0000","last_became_active":"2024-03-23T09:51:40.042615+0000","last_became_peered":"2024-03-23T09:51:40.042615+0000","last_unstale":"2024-03-23T09:51:41.464215+0000","last_undegraded":"2024-03-23T09:51:41.464215+0000","last_fullsized":"2024-03-23T09:51:41.464215+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:49:12.019134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462907+0000","last_change":"2024-03-23T09:51:40.422062+0000","last_active":"2024-03-23T09:51:41.462907+0000","last_peered":"2024-03-23T09:51:41.462907+0000","last_clean":"2024-03-23T09:51:41.462907+0000","last_became_active":"2024-03-23T09:51:40.421804+0000","last_became_peered":"2024-03-23T09:51:40.421804+0000","last_unstale":"2024-03-23T09:51:41.462907+0000","last_undegraded":"2024-03-23T09:51:41.462907+0000","last_fullsized":"2024-03-23T09:51:41.462907+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:51:53.472429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404261+0000","last_change":"2024-03-23T09:51:40.029982+0000","last_active":"2024-03-23T09:51:41.404261+0000","last_peered":"2024-03-23T09:51:41.404261+0000","last_clean":"2024-03-23T09:51:41.404261+0000","last_became_active":"2024-03-23T09:51:40.029641+0000","last_became_peered":"2024-03-23T09:51:40.029641+0000","last_unstale":"2024-03-23T09:51:41.404261+0000","last_undegraded":"2024-03-23T09:51:41.404261+0000","last_fullsized":"2024-03-23T09:51:41.404261+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:46:38.812091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463549+0000","last_change":"2024-03-23T09:51:40.416691+0000","last_active":"2024-03-23T09:51:41.463549+0000","last_peered":"2024-03-23T09:51:41.463549+0000","last_clean":"2024-03-23T09:51:41.463549+0000","last_became_active":"2024-03-23T09:51:40.416436+0000","last_became_peered":"2024-03-23T09:51:40.416436+0000","last_unstale":"2024-03-23T09:51:41.463549+0000","last_undegraded":"2024-03-23T09:51:41.463549+0000","last_fullsized":"2024-03-23T09:51:41.463549+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:54.096545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462989+0000","last_change":"2024-03-23T09:51:40.041980+0000","last_active":"2024-03-23T09:51:41.462989+0000","last_peered":"2024-03-23T09:51:41.462989+0000","last_clean":"2024-03-23T09:51:41.462989+0000","last_became_active":"2024-03-23T09:51:40.041690+0000","last_became_peered":"2024-03-23T09:51:40.041690+0000","last_unstale":"2024-03-23T09:51:41.462989+0000","last_undegraded":"2024-03-23T09:51:41.462989+0000","last_fullsized":"2024-03-23T09:51:41.462989+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:29:12.788788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463797+0000","last_change":"2024-03-23T09:51:40.415863+0000","last_active":"2024-03-23T09:51:41.463797+0000","last_peered":"2024-03-23T09:51:41.463797+0000","last_clean":"2024-03-23T09:51:41.463797+0000","last_became_active":"2024-03-23T09:51:40.415594+0000","last_became_peered":"2024-03-23T09:51:40.415594+0000","last_unstale":"2024-03-23T09:51:41.463797+0000","last_undegraded":"2024-03-23T09:51:41.463797+0000","last_fullsized":"2024-03-23T09:51:41.463797+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:39:58.810927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402418+0000","last_change":"2024-03-23T09:51:40.044131+0000","last_active":"2024-03-23T09:51:41.402418+0000","last_peered":"2024-03-23T09:51:41.402418+0000","last_clean":"2024-03-23T09:51:41.402418+0000","last_became_active":"2024-03-23T09:51:40.043837+0000","last_became_peered":"2024-03-23T09:51:40.043837+0000","last_unstale":"2024-03-23T09:51:41.402418+0000","last_undegraded":"2024-03-23T09:51:41.402418+0000","last_fullsized":"2024-03-23T09:51:41.402418+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:53.091071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463605+0000","last_change":"2024-03-23T09:51:40.422418+0000","last_active":"2024-03-23T09:51:41.463605+0000","last_peered":"2024-03-23T09:51:41.463605+0000","last_clean":"2024-03-23T09:51:41.463605+0000","last_became_active":"2024-03-23T09:51:40.422098+0000","last_became_peered":"2024-03-23T09:51:40.422098+0000","last_unstale":"2024-03-23T09:51:41.463605+0000","last_undegraded":"2024-03-23T09:51:41.463605+0000","last_fullsized":"2024-03-23T09:51:41.463605+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:23:45.258323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463979+0000","last_change":"2024-03-23T09:51:40.042482+0000","last_active":"2024-03-23T09:51:41.463979+0000","last_peered":"2024-03-23T09:51:41.463979+0000","last_clean":"2024-03-23T09:51:41.463979+0000","last_became_active":"2024-03-23T09:51:40.042210+0000","last_became_peered":"2024-03-23T09:51:40.042210+0000","last_unstale":"2024-03-23T09:51:41.463979+0000","last_undegraded":"2024-03-23T09:51:41.463979+0000","last_fullsized":"2024-03-23T09:51:41.463979+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:42:44.663049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464043+0000","last_change":"2024-03-23T09:51:40.410098+0000","last_active":"2024-03-23T09:51:41.464043+0000","last_peered":"2024-03-23T09:51:41.464043+0000","last_clean":"2024-03-23T09:51:41.464043+0000","last_became_active":"2024-03-23T09:51:40.409868+0000","last_became_peered":"2024-03-23T09:51:40.409868+0000","last_unstale":"2024-03-23T09:51:41.464043+0000","last_undegraded":"2024-03-23T09:51:41.464043+0000","last_fullsized":"2024-03-23T09:51:41.464043+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:15:20.760615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"113'3","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463523+0000","last_change":"2024-03-23T09:51:40.044981+0000","last_active":"2024-03-23T09:51:41.463523+0000","last_peered":"2024-03-23T09:51:41.463523+0000","last_clean":"2024-03-23T09:51:41.463523+0000","last_became_active":"2024-03-23T09:51:40.044497+0000","last_became_peered":"2024-03-23T09:51:40.044497+0000","last_unstale":"2024-03-23T09:51:41.463523+0000","last_undegraded":"2024-03-23T09:51:41.463523+0000","last_fullsized":"2024-03-23T09:51:41.463523+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:32:09.674080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463783+0000","last_change":"2024-03-23T09:51:40.419035+0000","last_active":"2024-03-23T09:51:41.463783+0000","last_peered":"2024-03-23T09:51:41.463783+0000","last_clean":"2024-03-23T09:51:41.463783+0000","last_became_active":"2024-03-23T09:51:40.418772+0000","last_became_peered":"2024-03-23T09:51:40.418772+0000","last_unstale":"2024-03-23T09:51:41.463783+0000","last_undegraded":"2024-03-23T09:51:41.463783+0000","last_fullsized":"2024-03-23T09:51:41.463783+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:27:46.446489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"113'6","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464019+0000","last_change":"2024-03-23T09:51:40.035049+0000","last_active":"2024-03-23T09:51:41.464019+0000","last_peered":"2024-03-23T09:51:41.464019+0000","last_clean":"2024-03-23T09:51:41.464019+0000","last_became_active":"2024-03-23T09:51:40.034609+0000","last_became_peered":"2024-03-23T09:51:40.034609+0000","last_unstale":"2024-03-23T09:51:41.464019+0000","last_undegraded":"2024-03-23T09:51:41.464019+0000","last_fullsized":"2024-03-23T09:51:41.464019+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:52.074712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464157+0000","last_change":"2024-03-23T09:51:40.414165+0000","last_active":"2024-03-23T09:51:41.464157+0000","last_peered":"2024-03-23T09:51:41.464157+0000","last_clean":"2024-03-23T09:51:41.464157+0000","last_became_active":"2024-03-23T09:51:40.413925+0000","last_became_peered":"2024-03-23T09:51:40.413925+0000","last_unstale":"2024-03-23T09:51:41.464157+0000","last_undegraded":"2024-03-23T09:51:41.464157+0000","last_fullsized":"2024-03-23T09:51:41.464157+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:57:09.681811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"113'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463651+0000","last_change":"2024-03-23T09:51:40.038304+0000","last_active":"2024-03-23T09:51:41.463651+0000","last_peered":"2024-03-23T09:51:41.463651+0000","last_clean":"2024-03-23T09:51:41.463651+0000","last_became_active":"2024-03-23T09:51:40.038053+0000","last_became_peered":"2024-03-23T09:51:40.038053+0000","last_unstale":"2024-03-23T09:51:41.463651+0000","last_undegraded":"2024-03-23T09:51:41.463651+0000","last_fullsized":"2024-03-23T09:51:41.463651+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:23:59.943842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464073+0000","last_change":"2024-03-23T09:51:40.409275+0000","last_active":"2024-03-23T09:51:41.464073+0000","last_peered":"2024-03-23T09:51:41.464073+0000","last_clean":"2024-03-23T09:51:41.464073+0000","last_became_active":"2024-03-23T09:51:40.409036+0000","last_became_peered":"2024-03-23T09:51:40.409036+0000","last_unstale":"2024-03-23T09:51:41.464073+0000","last_undegraded":"2024-03-23T09:51:41.464073+0000","last_fullsized":"2024-03-23T09:51:41.464073+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:20.000550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463324+0000","last_change":"2024-03-23T09:51:40.035729+0000","last_active":"2024-03-23T09:51:41.463324+0000","last_peered":"2024-03-23T09:51:41.463324+0000","last_clean":"2024-03-23T09:51:41.463324+0000","last_became_active":"2024-03-23T09:51:40.035430+0000","last_became_peered":"2024-03-23T09:51:40.035430+0000","last_unstale":"2024-03-23T09:51:41.463324+0000","last_undegraded":"2024-03-23T09:51:41.463324+0000","last_fullsized":"2024-03-23T09:51:41.463324+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:10:23.729745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462865+0000","last_change":"2024-03-23T09:51:40.413496+0000","last_active":"2024-03-23T09:51:41.462865+0000","last_peered":"2024-03-23T09:51:41.462865+0000","last_clean":"2024-03-23T09:51:41.462865+0000","last_became_active":"2024-03-23T09:51:40.413169+0000","last_became_peered":"2024-03-23T09:51:40.413169+0000","last_unstale":"2024-03-23T09:51:41.462865+0000","last_undegraded":"2024-03-23T09:51:41.462865+0000","last_fullsized":"2024-03-23T09:51:41.462865+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:20:31.190543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"113'8","reported_seq":19,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402407+0000","last_change":"2024-03-23T09:51:40.035576+0000","last_active":"2024-03-23T09:51:41.402407+0000","last_peered":"2024-03-23T09:51:41.402407+0000","last_clean":"2024-03-23T09:51:41.402407+0000","last_became_active":"2024-03-23T09:51:40.035324+0000","last_became_peered":"2024-03-23T09:51:40.035324+0000","last_unstale":"2024-03-23T09:51:41.402407+0000","last_undegraded":"2024-03-23T09:51:41.402407+0000","last_fullsized":"2024-03-23T09:51:41.402407+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:46:37.005710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402507+0000","last_change":"2024-03-23T09:51:40.421438+0000","last_active":"2024-03-23T09:51:41.402507+0000","last_peered":"2024-03-23T09:51:41.402507+0000","last_clean":"2024-03-23T09:51:41.402507+0000","last_became_active":"2024-03-23T09:51:40.421231+0000","last_became_peered":"2024-03-23T09:51:40.421231+0000","last_unstale":"2024-03-23T09:51:41.402507+0000","last_undegraded":"2024-03-23T09:51:41.402507+0000","last_fullsized":"2024-03-23T09:51:41.402507+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:44:20.541433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"113'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463674+0000","last_change":"2024-03-23T09:51:40.038898+0000","last_active":"2024-03-23T09:51:41.463674+0000","last_peered":"2024-03-23T09:51:41.463674+0000","last_clean":"2024-03-23T09:51:41.463674+0000","last_became_active":"2024-03-23T09:51:40.038438+0000","last_became_peered":"2024-03-23T09:51:40.038438+0000","last_unstale":"2024-03-23T09:51:41.463674+0000","last_undegraded":"2024-03-23T09:51:41.463674+0000","last_fullsized":"2024-03-23T09:51:41.463674+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:23:44.252849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463194+0000","last_change":"2024-03-23T09:51:40.426197+0000","last_active":"2024-03-23T09:51:41.463194+0000","last_peered":"2024-03-23T09:51:41.463194+0000","last_clean":"2024-03-23T09:51:41.463194+0000","last_became_active":"2024-03-23T09:51:40.425939+0000","last_became_peered":"2024-03-23T09:51:40.425939+0000","last_unstale":"2024-03-23T09:51:41.463194+0000","last_undegraded":"2024-03-23T09:51:41.463194+0000","last_fullsized":"2024-03-23T09:51:41.463194+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:22:31.864761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"113'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402555+0000","last_change":"2024-03-23T09:51:40.035425+0000","last_active":"2024-03-23T09:51:41.402555+0000","last_peered":"2024-03-23T09:51:41.402555+0000","last_clean":"2024-03-23T09:51:41.402555+0000","last_became_active":"2024-03-23T09:51:40.034933+0000","last_became_peered":"2024-03-23T09:51:40.034933+0000","last_unstale":"2024-03-23T09:51:41.402555+0000","last_undegraded":"2024-03-23T09:51:41.402555+0000","last_fullsized":"2024-03-23T09:51:41.402555+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:05:27.830265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403492+0000","last_change":"2024-03-23T09:51:40.424246+0000","last_active":"2024-03-23T09:51:41.403492+0000","last_peered":"2024-03-23T09:51:41.403492+0000","last_clean":"2024-03-23T09:51:41.403492+0000","last_became_active":"2024-03-23T09:51:40.423938+0000","last_became_peered":"2024-03-23T09:51:40.423938+0000","last_unstale":"2024-03-23T09:51:41.403492+0000","last_undegraded":"2024-03-23T09:51:41.403492+0000","last_fullsized":"2024-03-23T09:51:41.403492+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:12:46.572433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"113'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463300+0000","last_change":"2024-03-23T09:51:40.042089+0000","last_active":"2024-03-23T09:51:41.463300+0000","last_peered":"2024-03-23T09:51:41.463300+0000","last_clean":"2024-03-23T09:51:41.463300+0000","last_became_active":"2024-03-23T09:51:40.041816+0000","last_became_peered":"2024-03-23T09:51:40.041816+0000","last_unstale":"2024-03-23T09:51:41.463300+0000","last_undegraded":"2024-03-23T09:51:41.463300+0000","last_fullsized":"2024-03-23T09:51:41.463300+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:53:36.094132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404310+0000","last_change":"2024-03-23T09:51:40.414447+0000","last_active":"2024-03-23T09:51:41.404310+0000","last_peered":"2024-03-23T09:51:41.404310+0000","last_clean":"2024-03-23T09:51:41.404310+0000","last_became_active":"2024-03-23T09:51:40.413785+0000","last_became_peered":"2024-03-23T09:51:40.413785+0000","last_unstale":"2024-03-23T09:51:41.404310+0000","last_undegraded":"2024-03-23T09:51:41.404310+0000","last_fullsized":"2024-03-23T09:51:41.404310+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:23:45.258323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463336+0000","last_change":"2024-03-23T09:51:40.040499+0000","last_active":"2024-03-23T09:51:41.463336+0000","last_peered":"2024-03-23T09:51:41.463336+0000","last_clean":"2024-03-23T09:51:41.463336+0000","last_became_active":"2024-03-23T09:51:40.040208+0000","last_became_peered":"2024-03-23T09:51:40.040208+0000","last_unstale":"2024-03-23T09:51:41.463336+0000","last_undegraded":"2024-03-23T09:51:41.463336+0000","last_fullsized":"2024-03-23T09:51:41.463336+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:12:20.739568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403961+0000","last_change":"2024-03-23T09:51:40.415012+0000","last_active":"2024-03-23T09:51:41.403961+0000","last_peered":"2024-03-23T09:51:41.403961+0000","last_clean":"2024-03-23T09:51:41.403961+0000","last_became_active":"2024-03-23T09:51:40.414748+0000","last_became_peered":"2024-03-23T09:51:40.414748+0000","last_unstale":"2024-03-23T09:51:41.403961+0000","last_undegraded":"2024-03-23T09:51:41.403961+0000","last_fullsized":"2024-03-23T09:51:41.403961+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:51:23.039272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"113'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463729+0000","last_change":"2024-03-23T09:51:40.038160+0000","last_active":"2024-03-23T09:51:41.463729+0000","last_peered":"2024-03-23T09:51:41.463729+0000","last_clean":"2024-03-23T09:51:41.463729+0000","last_became_active":"2024-03-23T09:51:40.037899+0000","last_became_peered":"2024-03-23T09:51:40.037899+0000","last_unstale":"2024-03-23T09:51:41.463729+0000","last_undegraded":"2024-03-23T09:51:41.463729+0000","last_fullsized":"2024-03-23T09:51:41.463729+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:24:03.629632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463434+0000","last_change":"2024-03-23T09:51:40.426694+0000","last_active":"2024-03-23T09:51:41.463434+0000","last_peered":"2024-03-23T09:51:41.463434+0000","last_clean":"2024-03-23T09:51:41.463434+0000","last_became_active":"2024-03-23T09:51:40.426440+0000","last_became_peered":"2024-03-23T09:51:40.426440+0000","last_unstale":"2024-03-23T09:51:41.463434+0000","last_undegraded":"2024-03-23T09:51:41.463434+0000","last_fullsized":"2024-03-23T09:51:41.463434+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:33:21.740753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402313+0000","last_change":"2024-03-23T09:51:40.043294+0000","last_active":"2024-03-23T09:51:41.402313+0000","last_peered":"2024-03-23T09:51:41.402313+0000","last_clean":"2024-03-23T09:51:41.402313+0000","last_became_active":"2024-03-23T09:51:40.043044+0000","last_became_peered":"2024-03-23T09:51:40.043044+0000","last_unstale":"2024-03-23T09:51:41.402313+0000","last_undegraded":"2024-03-23T09:51:41.402313+0000","last_fullsized":"2024-03-23T09:51:41.402313+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:59:01.729483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403667+0000","last_change":"2024-03-23T09:51:40.406701+0000","last_active":"2024-03-23T09:51:41.403667+0000","last_peered":"2024-03-23T09:51:41.403667+0000","last_clean":"2024-03-23T09:51:41.403667+0000","last_became_active":"2024-03-23T09:51:40.406433+0000","last_became_peered":"2024-03-23T09:51:40.406433+0000","last_unstale":"2024-03-23T09:51:41.403667+0000","last_undegraded":"2024-03-23T09:51:41.403667+0000","last_fullsized":"2024-03-23T09:51:41.403667+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:21:37.643702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463029+0000","last_change":"2024-03-23T09:51:40.039390+0000","last_active":"2024-03-23T09:51:41.463029+0000","last_peered":"2024-03-23T09:51:41.463029+0000","last_clean":"2024-03-23T09:51:41.463029+0000","last_became_active":"2024-03-23T09:51:40.039105+0000","last_became_peered":"2024-03-23T09:51:40.039105+0000","last_unstale":"2024-03-23T09:51:41.463029+0000","last_undegraded":"2024-03-23T09:51:41.463029+0000","last_fullsized":"2024-03-23T09:51:41.463029+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:39:14.955692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463796+0000","last_change":"2024-03-23T09:51:40.421674+0000","last_active":"2024-03-23T09:51:41.463796+0000","last_peered":"2024-03-23T09:51:41.463796+0000","last_clean":"2024-03-23T09:51:41.463796+0000","last_became_active":"2024-03-23T09:51:40.420840+0000","last_became_peered":"2024-03-23T09:51:40.420840+0000","last_unstale":"2024-03-23T09:51:41.463796+0000","last_undegraded":"2024-03-23T09:51:41.463796+0000","last_fullsized":"2024-03-23T09:51:41.463796+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:51:23.039272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463046+0000","last_change":"2024-03-23T09:51:40.037608+0000","last_active":"2024-03-23T09:51:41.463046+0000","last_peered":"2024-03-23T09:51:41.463046+0000","last_clean":"2024-03-23T09:51:41.463046+0000","last_became_active":"2024-03-23T09:51:40.037351+0000","last_became_peered":"2024-03-23T09:51:40.037351+0000","last_unstale":"2024-03-23T09:51:41.463046+0000","last_undegraded":"2024-03-23T09:51:41.463046+0000","last_fullsized":"2024-03-23T09:51:41.463046+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:35:25.457955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463709+0000","last_change":"2024-03-23T09:51:40.422399+0000","last_active":"2024-03-23T09:51:41.463709+0000","last_peered":"2024-03-23T09:51:41.463709+0000","last_clean":"2024-03-23T09:51:41.463709+0000","last_became_active":"2024-03-23T09:51:40.422078+0000","last_became_peered":"2024-03-23T09:51:40.422078+0000","last_unstale":"2024-03-23T09:51:41.463709+0000","last_undegraded":"2024-03-23T09:51:41.463709+0000","last_fullsized":"2024-03-23T09:51:41.463709+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:29:13.794262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"113'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464070+0000","last_change":"2024-03-23T09:51:40.039375+0000","last_active":"2024-03-23T09:51:41.464070+0000","last_peered":"2024-03-23T09:51:41.464070+0000","last_clean":"2024-03-23T09:51:41.464070+0000","last_became_active":"2024-03-23T09:51:40.039110+0000","last_became_peered":"2024-03-23T09:51:40.039110+0000","last_unstale":"2024-03-23T09:51:41.464070+0000","last_undegraded":"2024-03-23T09:51:41.464070+0000","last_fullsized":"2024-03-23T09:51:41.464070+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:10:23.729745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463589+0000","last_change":"2024-03-23T09:51:40.423720+0000","last_active":"2024-03-23T09:51:41.463589+0000","last_peered":"2024-03-23T09:51:41.463589+0000","last_clean":"2024-03-23T09:51:41.463589+0000","last_became_active":"2024-03-23T09:51:40.423325+0000","last_became_peered":"2024-03-23T09:51:40.423325+0000","last_unstale":"2024-03-23T09:51:41.463589+0000","last_undegraded":"2024-03-23T09:51:41.463589+0000","last_fullsized":"2024-03-23T09:51:41.463589+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:52:38.664859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"113'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463528+0000","last_change":"2024-03-23T09:51:40.038937+0000","last_active":"2024-03-23T09:51:41.463528+0000","last_peered":"2024-03-23T09:51:41.463528+0000","last_clean":"2024-03-23T09:51:41.463528+0000","last_became_active":"2024-03-23T09:51:40.038576+0000","last_became_peered":"2024-03-23T09:51:40.038576+0000","last_unstale":"2024-03-23T09:51:41.463528+0000","last_undegraded":"2024-03-23T09:51:41.463528+0000","last_fullsized":"2024-03-23T09:51:41.463528+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:26:17.535351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463969+0000","last_change":"2024-03-23T09:51:40.422023+0000","last_active":"2024-03-23T09:51:41.463969+0000","last_peered":"2024-03-23T09:51:41.463969+0000","last_clean":"2024-03-23T09:51:41.463969+0000","last_became_active":"2024-03-23T09:51:40.421766+0000","last_became_peered":"2024-03-23T09:51:40.421766+0000","last_unstale":"2024-03-23T09:51:41.463969+0000","last_undegraded":"2024-03-23T09:51:41.463969+0000","last_fullsized":"2024-03-23T09:51:41.463969+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:23:42.776013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463624+0000","last_change":"2024-03-23T09:51:40.037782+0000","last_active":"2024-03-23T09:51:41.463624+0000","last_peered":"2024-03-23T09:51:41.463624+0000","last_clean":"2024-03-23T09:51:41.463624+0000","last_became_active":"2024-03-23T09:51:40.037486+0000","last_became_peered":"2024-03-23T09:51:40.037486+0000","last_unstale":"2024-03-23T09:51:41.463624+0000","last_undegraded":"2024-03-23T09:51:41.463624+0000","last_fullsized":"2024-03-23T09:51:41.463624+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:08:16.104557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463314+0000","last_change":"2024-03-23T09:51:40.422633+0000","last_active":"2024-03-23T09:51:41.463314+0000","last_peered":"2024-03-23T09:51:41.463314+0000","last_clean":"2024-03-23T09:51:41.463314+0000","last_became_active":"2024-03-23T09:51:40.422270+0000","last_became_peered":"2024-03-23T09:51:40.422270+0000","last_unstale":"2024-03-23T09:51:41.463314+0000","last_undegraded":"2024-03-23T09:51:41.463314+0000","last_fullsized":"2024-03-23T09:51:41.463314+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:21:51.075338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464066+0000","last_change":"2024-03-23T09:51:40.040645+0000","last_active":"2024-03-23T09:51:41.464066+0000","last_peered":"2024-03-23T09:51:41.464066+0000","last_clean":"2024-03-23T09:51:41.464066+0000","last_became_active":"2024-03-23T09:51:40.040367+0000","last_became_peered":"2024-03-23T09:51:40.040367+0000","last_unstale":"2024-03-23T09:51:41.464066+0000","last_undegraded":"2024-03-23T09:51:41.464066+0000","last_fullsized":"2024-03-23T09:51:41.464066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:39:16.129578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403744+0000","last_change":"2024-03-23T09:51:40.413956+0000","last_active":"2024-03-23T09:51:41.403744+0000","last_peered":"2024-03-23T09:51:41.403744+0000","last_clean":"2024-03-23T09:51:41.403744+0000","last_became_active":"2024-03-23T09:51:40.413537+0000","last_became_peered":"2024-03-23T09:51:40.413537+0000","last_unstale":"2024-03-23T09:51:41.403744+0000","last_undegraded":"2024-03-23T09:51:41.403744+0000","last_fullsized":"2024-03-23T09:51:41.403744+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:20:31.190543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463340+0000","last_change":"2024-03-23T09:51:40.036680+0000","last_active":"2024-03-23T09:51:41.463340+0000","last_peered":"2024-03-23T09:51:41.463340+0000","last_clean":"2024-03-23T09:51:41.463340+0000","last_became_active":"2024-03-23T09:51:40.036394+0000","last_became_peered":"2024-03-23T09:51:40.036394+0000","last_unstale":"2024-03-23T09:51:41.463340+0000","last_undegraded":"2024-03-23T09:51:41.463340+0000","last_fullsized":"2024-03-23T09:51:41.463340+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:01:21.619133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403762+0000","last_change":"2024-03-23T09:51:40.407626+0000","last_active":"2024-03-23T09:51:41.403762+0000","last_peered":"2024-03-23T09:51:41.403762+0000","last_clean":"2024-03-23T09:51:41.403762+0000","last_became_active":"2024-03-23T09:51:40.407341+0000","last_became_peered":"2024-03-23T09:51:40.407341+0000","last_unstale":"2024-03-23T09:51:41.403762+0000","last_undegraded":"2024-03-23T09:51:41.403762+0000","last_fullsized":"2024-03-23T09:51:41.403762+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:35:26.463429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403826+0000","last_change":"2024-03-23T09:51:40.041649+0000","last_active":"2024-03-23T09:51:41.403826+0000","last_peered":"2024-03-23T09:51:41.403826+0000","last_clean":"2024-03-23T09:51:41.403826+0000","last_became_active":"2024-03-23T09:51:40.041352+0000","last_became_peered":"2024-03-23T09:51:40.041352+0000","last_unstale":"2024-03-23T09:51:41.403826+0000","last_undegraded":"2024-03-23T09:51:41.403826+0000","last_fullsized":"2024-03-23T09:51:41.403826+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:32:54.365123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403468+0000","last_change":"2024-03-23T09:51:40.416853+0000","last_active":"2024-03-23T09:51:41.403468+0000","last_peered":"2024-03-23T09:51:41.403468+0000","last_clean":"2024-03-23T09:51:41.403468+0000","last_became_active":"2024-03-23T09:51:40.416449+0000","last_became_peered":"2024-03-23T09:51:40.416449+0000","last_unstale":"2024-03-23T09:51:41.403468+0000","last_undegraded":"2024-03-23T09:51:41.403468+0000","last_fullsized":"2024-03-23T09:51:41.403468+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:20:31.190543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402309+0000","last_change":"2024-03-23T09:51:40.040229+0000","last_active":"2024-03-23T09:51:41.402309+0000","last_peered":"2024-03-23T09:51:41.402309+0000","last_clean":"2024-03-23T09:51:41.402309+0000","last_became_active":"2024-03-23T09:51:40.039955+0000","last_became_peered":"2024-03-23T09:51:40.039955+0000","last_unstale":"2024-03-23T09:51:41.402309+0000","last_undegraded":"2024-03-23T09:51:41.402309+0000","last_fullsized":"2024-03-23T09:51:41.402309+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:39:57.805453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463307+0000","last_change":"2024-03-23T09:51:40.420521+0000","last_active":"2024-03-23T09:51:41.463307+0000","last_peered":"2024-03-23T09:51:41.463307+0000","last_clean":"2024-03-23T09:51:41.463307+0000","last_became_active":"2024-03-23T09:51:40.420240+0000","last_became_peered":"2024-03-23T09:51:40.420240+0000","last_unstale":"2024-03-23T09:51:41.463307+0000","last_undegraded":"2024-03-23T09:51:41.463307+0000","last_fullsized":"2024-03-23T09:51:41.463307+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:22:49.724185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463125+0000","last_change":"2024-03-23T09:51:40.038163+0000","last_active":"2024-03-23T09:51:41.463125+0000","last_peered":"2024-03-23T09:51:41.463125+0000","last_clean":"2024-03-23T09:51:41.463125+0000","last_became_active":"2024-03-23T09:51:40.037909+0000","last_became_peered":"2024-03-23T09:51:40.037909+0000","last_unstale":"2024-03-23T09:51:41.463125+0000","last_undegraded":"2024-03-23T09:51:41.463125+0000","last_fullsized":"2024-03-23T09:51:41.463125+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:21:50.069864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462725+0000","last_change":"2024-03-23T09:51:40.409724+0000","last_active":"2024-03-23T09:51:41.462725+0000","last_peered":"2024-03-23T09:51:41.462725+0000","last_clean":"2024-03-23T09:51:41.462725+0000","last_became_active":"2024-03-23T09:51:40.409073+0000","last_became_peered":"2024-03-23T09:51:40.409073+0000","last_unstale":"2024-03-23T09:51:41.462725+0000","last_undegraded":"2024-03-23T09:51:41.462725+0000","last_fullsized":"2024-03-23T09:51:41.462725+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:23:45.258323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463630+0000","last_change":"2024-03-23T09:51:40.040498+0000","last_active":"2024-03-23T09:51:41.463630+0000","last_peered":"2024-03-23T09:51:41.463630+0000","last_clean":"2024-03-23T09:51:41.463630+0000","last_became_active":"2024-03-23T09:51:40.040182+0000","last_became_peered":"2024-03-23T09:51:40.040182+0000","last_unstale":"2024-03-23T09:51:41.463630+0000","last_undegraded":"2024-03-23T09:51:41.463630+0000","last_fullsized":"2024-03-23T09:51:41.463630+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:52:30.476534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":261,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-03-23T09:51:40.400826+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:51:40.400826+0000","last_peered":"2024-03-23T09:51:40.400826+0000","last_clean":"2024-03-23T09:51:40.400826+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:51:40.400826+0000","last_undegraded":"2024-03-23T09:51:40.400826+0000","last_fullsized":"2024-03-23T09:51:40.400826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":2019328,"data_stored":1204289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.52200000000000002},"min":{"1min":0.41099999999999998,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56799999999999995,"15min":0.54200000000000004},"min":{"1min":0.47099999999999997,"5min":0.32200000000000001,"15min":0.28199999999999997},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.85999999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56200000000000006,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.77700000000000002,"5min":0.79100000000000004,"15min":0.80400000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55300000000000005,"15min":0.53500000000000003},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.30199999999999999},"max":{"1min":0.68600000000000005,"5min":0.76000000000000001,"15min":0.81299999999999994},"last":0.55600000000000005}]},{"osd":2,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.48899999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68100000000000005,"5min":0.78500000000000003,"15min":0.82499999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80700000000000005,"5min":0.83099999999999996,"15min":0.83599999999999997},"last":0.622}]},{"osd":3,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.54500000000000004},"min":{"1min":0.42799999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71199999999999997,"5min":0.73999999999999999,"15min":0.86199999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.46100000000000002,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.83699999999999997},"last":0.60899999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47699999999999998,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29599999999999999},"max":{"1min":0.64500000000000002,"5min":0.76300000000000001,"15min":0.78200000000000003},"last":0.503},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.47799999999999998,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.29299999999999998},"max":{"1min":0.65600000000000003,"5min":0.74399999999999999,"15min":0.80100000000000005},"last":0.65100000000000002}]},{"osd":5,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48799999999999999,"15min":0.50600000000000001},"min":{"1min":0.32100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69799999999999995,"5min":0.77300000000000002,"15min":0.81499999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.51300000000000001,"15min":0.54300000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.627,"5min":0.78000000000000003,"15min":0.84999999999999998},"last":0.63300000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51600000000000001,"15min":0.55700000000000005},"min":{"1min":0.33900000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75,"5min":0.753,"15min":0.874},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.53200000000000003,"15min":0.56299999999999994},"min":{"1min":0.33900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73999999999999999,"5min":0.76200000000000001,"15min":0.90200000000000002},"last":0.57799999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574980,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1417216,"data_stored":612756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56699999999999995,"15min":0.53000000000000003},"min":{"1min":0.46899999999999997,"5min":0.38,"15min":0.27200000000000002},"max":{"1min":0.73699999999999999,"5min":0.80700000000000005,"15min":0.81000000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.55200000000000005},"min":{"1min":0.439,"5min":0.37,"15min":0.34399999999999997},"max":{"1min":0.73699999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58699999999999997,"15min":0.54200000000000004},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.56000000000000005},"min":{"1min":0.45900000000000002,"5min":0.41299999999999998,"15min":0.32700000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.82999999999999996},"last":0.44700000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55700000000000005,"15min":0.53900000000000003},"min":{"1min":0.45100000000000001,"5min":0.36199999999999999,"15min":0.29699999999999999},"max":{"1min":0.70799999999999996,"5min":0.747,"15min":0.81599999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.56999999999999995},"min":{"1min":0.47499999999999998,"5min":0.39400000000000002,"15min":0.36499999999999999},"max":{"1min":0.78500000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49399999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.435,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.72199999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.505},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.41499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76400000000000001,"5min":0.79000000000000004,"15min":0.81699999999999995},"last":0.51800000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45700000000000002,"15min":0.48499999999999999},"min":{"1min":0.32500000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61099999999999999,"5min":0.69299999999999995,"15min":0.72499999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.47799999999999998,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.623,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59899999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.48399999999999999,"15min":0.50900000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.316},"max":{"1min":0.64400000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.51800000000000002,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70199999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.57699999999999996}]},{"osd":7,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.72199999999999998,"5min":0.754,"15min":0.82699999999999996},"last":0.627},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.56799999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70799999999999996,"5min":0.71099999999999997,"15min":0.86199999999999999},"last":0.69299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574979,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1425408,"data_stored":613909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.51400000000000001},"min":{"1min":0.38,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.432,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.39400000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54900000000000004,"15min":0.52100000000000002},"min":{"1min":0.42099999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55500000000000005,"15min":0.52900000000000003},"min":{"1min":0.47999999999999998,"5min":0.38300000000000001,"15min":0.36599999999999999},"max":{"1min":0.70199999999999996,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46999999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.54700000000000004},"min":{"1min":0.438,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57199999999999995,"15min":0.54400000000000004},"min":{"1min":0.46999999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72499999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.32100000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.54100000000000004},"min":{"1min":0.51400000000000001,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.72799999999999998,"5min":0.77400000000000002,"15min":0.81599999999999995},"last":0.308},{"interface":"front","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.378,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77400000000000002},"last":0.41099999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45600000000000002,"15min":0.46300000000000002},"min":{"1min":0.32800000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66400000000000003,"5min":0.73099999999999998,"15min":0.73699999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.93600000000000005},"last":0.48899999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.497,"15min":0.497},"min":{"1min":0.33700000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72099999999999997,"5min":0.747,"15min":0.749},"last":0.503},{"interface":"front","average":{"1min":0.496,"5min":0.503,"15min":0.505},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.42499999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54400000000000004,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.74099999999999999,"5min":0.76200000000000001,"15min":0.78200000000000003},"last":0.442},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1417216,"data_stored":613305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58299999999999996,"15min":0.54200000000000004},"min":{"1min":0.46400000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.84599999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.45500000000000002,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.73299999999999998,"5min":0.754,"15min":0.78600000000000003},"last":0.51200000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53800000000000003,"15min":0.505},"min":{"1min":0.46800000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72999999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.52400000000000002},"min":{"1min":0.437,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69099999999999995,"5min":0.751,"15min":0.77400000000000002},"last":0.58899999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60499999999999998,"15min":0.56899999999999995},"min":{"1min":0.50900000000000001,"5min":0.42799999999999999,"15min":0.39400000000000002},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.80200000000000005},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.496,"5min":0.39700000000000002,"15min":0.372},"max":{"1min":0.75600000000000001,"5min":0.76600000000000001,"15min":0.82199999999999995},"last":0.54600000000000004}]},{"osd":3,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.52900000000000003},"min":{"1min":0.40400000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71499999999999997,"5min":0.753,"15min":1.0369999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.56599999999999995,"15min":0.54000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40500000000000003,"15min":0.374},"max":{"1min":0.77900000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53500000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.46200000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.26500000000000001},"max":{"1min":0.68600000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46500000000000002,"15min":0.49399999999999999},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.253},"max":{"1min":0.65900000000000003,"5min":0.73199999999999998,"15min":1.008},"last":0.55700000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47399999999999998,"15min":0.47499999999999998},"min":{"1min":0.35299999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68300000000000005,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.505,"15min":0.51300000000000001},"min":{"1min":0.35399999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70499999999999996,"5min":0.74199999999999999,"15min":0.82399999999999995},"last":0.49299999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51000000000000001,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.30099999999999999},"max":{"1min":0.71799999999999997,"5min":0.76000000000000001,"15min":1.0069999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.752,"5min":0.77600000000000002,"15min":0.93400000000000005},"last":0.57499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574979,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36620,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335220,"statfs":{"total":96636764160,"available":96599265280,"internally_reserved":0,"allocated":1433600,"data_stored":614939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33200000000000002},"min":{"1min":0.26100000000000001,"5min":0.192,"15min":0.183},"max":{"1min":0.55100000000000005,"5min":0.67800000000000005,"15min":1.45},"last":0.223},{"interface":"front","average":{"1min":0.374,"5min":0.35499999999999998,"15min":0.33800000000000002},"min":{"1min":0.20699999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":1.4099999999999999},"last":0.27000000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34999999999999998,"15min":0.33200000000000002},"min":{"1min":0.22800000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":1.323},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.379,"15min":0.36599999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.58699999999999997,"5min":0.61299999999999999,"15min":0.94599999999999995},"last":0.28599999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.40600000000000003},"min":{"1min":0.29299999999999998,"5min":0.27100000000000002,"15min":0.251},"max":{"1min":0.66400000000000003,"5min":0.68999999999999995,"15min":1.1619999999999999},"last":0.314},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.44500000000000001,"15min":0.42999999999999999},"min":{"1min":0.30299999999999999,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":1.2989999999999999},"last":0.33600000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58099999999999996,"15min":0.56899999999999995},"min":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.35399999999999998},"max":{"1min":0.872,"5min":0.88600000000000001,"15min":1.337},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.58599999999999997,"15min":0.56599999999999995},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78900000000000003,"5min":0.82199999999999995,"15min":2.758},"last":0.46800000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63700000000000001,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":1.276},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.45700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.85799999999999998,"5min":0.91200000000000003,"15min":1.895},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.42599999999999999,"5min":0.41299999999999998,"15min":0.40699999999999997},"max":{"1min":0.77700000000000002,"5min":0.84199999999999997,"15min":5.4539999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.60199999999999998},"min":{"1min":0.46800000000000003,"5min":0.41599999999999998,"15min":0.38200000000000001},"max":{"1min":0.81699999999999995,"5min":0.96499999999999997,"15min":2.8159999999999998},"last":0.48899999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41299999999999998},"max":{"1min":0.84899999999999998,"5min":0.94899999999999995,"15min":3.7400000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63500000000000001,"15min":0.63200000000000001},"min":{"1min":0.47499999999999998,"5min":0.443,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":5.4900000000000002},"last":0.5}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1417216,"data_stored":613305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34899999999999998,"15min":0.34599999999999997},"min":{"1min":0.23499999999999999,"5min":0.19500000000000001,"15min":0.191},"max":{"1min":0.503,"5min":0.55500000000000005,"15min":0.60399999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.38900000000000001,"15min":0.377},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.22},"max":{"1min":0.57199999999999995,"5min":0.60799999999999998,"15min":0.872},"last":0.42699999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.374,"15min":0.36799999999999999},"min":{"1min":0.249,"5min":0.23699999999999999,"15min":0.20999999999999999},"max":{"1min":0.53600000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40400000000000003,"15min":0.39300000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.20899999999999999},"max":{"1min":0.57399999999999995,"5min":0.61199999999999999,"15min":0.746},"last":0.51500000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42199999999999999,"15min":0.41299999999999998},"min":{"1min":0.315,"5min":0.27600000000000002,"15min":0.22900000000000001},"max":{"1min":0.58999999999999997,"5min":0.65200000000000002,"15min":0.67100000000000004},"last":0.497},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.40000000000000002},"min":{"1min":0.27500000000000002,"5min":0.27000000000000002,"15min":0.248},"max":{"1min":0.59199999999999997,"5min":0.66000000000000003,"15min":0.72499999999999998},"last":0.44900000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.44900000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81399999999999995,"5min":0.87,"15min":1.216},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.99299999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.625,"15min":0.628},"min":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.373},"max":{"1min":0.85399999999999998,"5min":0.878,"15min":0.95699999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.629,"15min":0.629},"min":{"1min":0.48699999999999999,"5min":0.42099999999999999,"15min":0.40300000000000002},"max":{"1min":0.82799999999999996,"5min":0.98499999999999999,"15min":1.101},"last":0.71999999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86199999999999999,"5min":0.86499999999999999,"15min":0.97299999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.49199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89700000000000002,"5min":0.95599999999999996,"15min":1.0780000000000001},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.49399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.872,"5min":0.93200000000000005,"15min":1.155},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.46100000000000002,"15min":0.41599999999999998},"max":{"1min":0.90700000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1425408,"data_stored":613429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.35599999999999998,"15min":0.34699999999999998},"min":{"1min":0.23000000000000001,"5min":0.185,"15min":0.185},"max":{"1min":0.47899999999999998,"5min":0.63500000000000001,"15min":0.64400000000000002},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.432},"min":{"1min":0.24299999999999999,"5min":0.218,"15min":0.20699999999999999},"max":{"1min":0.56000000000000005,"5min":0.62,"15min":2.7069999999999999},"last":0.32600000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.371,"15min":0.35699999999999998},"min":{"1min":0.218,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.38800000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.69799999999999995},"last":0.309}]},{"osd":3,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.442,"15min":0.42399999999999999},"min":{"1min":0.28599999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.71499999999999997,"5min":0.746,"15min":0.76900000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.45700000000000002},"min":{"1min":0.27000000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.628,"5min":0.73999999999999999,"15min":2.75},"last":0.376}]},{"osd":4,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.47699999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81399999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.621,"15min":0.61699999999999999},"min":{"1min":0.49099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":1.002,"15min":1.002},"last":0.48399999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63,"15min":0.66900000000000004},"min":{"1min":0.48199999999999998,"5min":0.41099999999999998,"15min":0.35299999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":2.778},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.52200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82099999999999995,"5min":0.88800000000000001,"15min":1.0649999999999999},"last":0.45800000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64100000000000001,"15min":0.67700000000000005},"min":{"1min":0.501,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":2.8210000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64800000000000002,"15min":0.63900000000000001},"min":{"1min":0.55400000000000005,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.86199999999999999,"5min":0.93300000000000005,"15min":1.1080000000000001},"last":0.51200000000000001}]},{"osd":7,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.50900000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84299999999999997,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64800000000000002,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":2.7989999999999999},"last":0.57099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574979,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37200,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334640,"statfs":{"total":96636764160,"available":96598671360,"internally_reserved":0,"allocated":2027520,"data_stored":1204730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35399999999999998,"15min":0.33600000000000002},"min":{"1min":0.26100000000000001,"5min":0.20599999999999999,"15min":0.184},"max":{"1min":0.52600000000000002,"5min":0.625,"15min":0.625},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.37,"15min":0.36099999999999999},"min":{"1min":0.26300000000000001,"5min":0.22900000000000001,"15min":0.217},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.64900000000000002},"last":0.40799999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38800000000000001,"15min":0.372},"min":{"1min":0.30399999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.58899999999999997,"5min":0.63,"15min":0.63},"last":0.318},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.39600000000000002,"15min":0.38100000000000001},"min":{"1min":0.309,"5min":0.22600000000000001,"15min":0.21199999999999999},"max":{"1min":0.54600000000000004,"5min":0.58199999999999996,"15min":0.59399999999999997},"last":0.35199999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.39800000000000002,"15min":0.38900000000000001},"min":{"1min":0.252,"5min":0.217,"15min":0.217},"max":{"1min":0.53000000000000003,"5min":0.57499999999999996,"15min":0.67600000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.45400000000000001,"15min":0.437},"min":{"1min":0.35699999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.624,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.433}]},{"osd":4,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.44500000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.48699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85399999999999998,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.372}]},{"osd":5,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.46100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.60199999999999998},"min":{"1min":0.51700000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.95999999999999996},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.51500000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86799999999999999,"5min":0.91900000000000004,"15min":0.93600000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65300000000000002,"15min":0.65200000000000002},"min":{"1min":0.52300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.56799999999999995}]},{"osd":7,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.629,"15min":0.61899999999999999},"min":{"1min":0.50900000000000001,"5min":0.40999999999999998,"15min":0.39300000000000002},"max":{"1min":0.79800000000000004,"5min":0.878,"15min":0.93600000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.47999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.53800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:51:44.022 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:51:44.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.157+0000 7fa6436d5640 1 Processor -- start 2024-03-23T09:51:44.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.157+0000 7fa6436d5640 1 -- start start 2024-03-23T09:51:44.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.158+0000 7fa6436d5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa63c06bf10 0x7fa63c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.158+0000 7fa6436d5640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63c0b1530 con 0x7fa63c15c9a0 2024-03-23T09:51:44.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.158+0000 7fa6436d5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63c15df60 con 0x7fa63c15d770 2024-03-23T09:51:44.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.158+0000 7fa6436d5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63c1606e0 con 0x7fa63c06bf10 2024-03-23T09:51:44.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.158+0000 7fa640c70640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa63c06bf10 0x7fa63c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.158+0000 7fa640c70640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa63c06bf10 0x7fa63c15d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54582/0 (socket says 172.21.15.47:54582) 2024-03-23T09:51:44.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.158+0000 7fa641471640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa63c15c9a0 0x7fa63c06b800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52426/0 (socket says 172.21.15.47:52426) 2024-03-23T09:51:44.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.158+0000 7fa640c70640 1 -- 172.21.15.47:0/3424416804 learned_addr learned my addr 172.21.15.47:0/3424416804 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:44.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.159+0000 7fa640c70640 1 -- 172.21.15.47:0/3424416804 >> v1:172.21.15.175:6789/0 conn(0x7fa63c15d770 legacy=0x7fa63c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:44.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.159+0000 7fa640c70640 1 -- 172.21.15.47:0/3424416804 >> v1:172.21.15.47:6789/0 conn(0x7fa63c15c9a0 legacy=0x7fa63c06b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.159+0000 7fa640c70640 1 -- 172.21.15.47:0/3424416804 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c1609c0 con 0x7fa63c06bf10 2024-03-23T09:51:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.159+0000 7fa640c70640 1 --2- 172.21.15.47:0/3424416804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c06bf10 0x7fa63c15d230 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fa630009870 tx=0x7fa63002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e04547ce47e0be server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.160+0000 7fa63bfff640 1 -- 172.21.15.47:0/3424416804 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630030850 con 0x7fa63c06bf10 2024-03-23T09:51:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.160+0000 7fa63bfff640 1 -- 172.21.15.47:0/3424416804 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6300309b0 con 0x7fa63c06bf10 2024-03-23T09:51:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.160+0000 7fa63bfff640 1 -- 172.21.15.47:0/3424416804 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630030d40 con 0x7fa63c06bf10 2024-03-23T09:51:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.160+0000 7fa6436d5640 1 -- 172.21.15.47:0/3424416804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c06bf10 msgr2=0x7fa63c15d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.160+0000 7fa6436d5640 1 --2- 172.21.15.47:0/3424416804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c06bf10 0x7fa63c15d230 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fa630009870 tx=0x7fa63002b990 comp rx=0 tx=0).stop 2024-03-23T09:51:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.160+0000 7fa6436d5640 1 -- 172.21.15.47:0/3424416804 shutdown_connections 2024-03-23T09:51:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.160+0000 7fa6436d5640 1 --2- 172.21.15.47:0/3424416804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c06bf10 0x7fa63c15d230 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.160+0000 7fa6436d5640 1 -- 172.21.15.47:0/3424416804 >> 172.21.15.47:0/3424416804 conn(0x7fa63c0d0670 msgr2=0x7fa63c0d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.160+0000 7fa6436d5640 1 -- 172.21.15.47:0/3424416804 shutdown_connections 2024-03-23T09:51:44.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.160+0000 7fa6436d5640 1 -- 172.21.15.47:0/3424416804 wait complete. 2024-03-23T09:51:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.161+0000 7fa6436d5640 1 Processor -- start 2024-03-23T09:51:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.162+0000 7fa6436d5640 1 -- start start 2024-03-23T09:51:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.162+0000 7fa6436d5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa63c06bf10 0x7fa63c156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.162+0000 7fa6436d5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c15c9a0 0x7fa63c0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.162+0000 7fa6436d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa63c15d770 0x7fa63c15f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.162+0000 7fa6436d5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1f6a70 con 0x7fa63c15c9a0 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.162+0000 7fa6436d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1f5ea0 con 0x7fa63c15d770 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.162+0000 7fa6436d5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1f6140 con 0x7fa63c06bf10 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.162+0000 7fa640c70640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c15c9a0 0x7fa63c0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.162+0000 7fa640c70640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c15c9a0 0x7fa63c0c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54596/0 (socket says 172.21.15.47:54596) 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.162+0000 7fa640c70640 1 -- 172.21.15.47:0/391757101 learned_addr learned my addr 172.21.15.47:0/391757101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa641c72640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa63c15d770 0x7fa63c15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa641471640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa63c06bf10 0x7fa63c156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa640c70640 1 -- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa63c06bf10 msgr2=0x7fa63c156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa640c70640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa63c06bf10 0x7fa63c156020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa640c70640 1 -- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa63c15d770 msgr2=0x7fa63c15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa640c70640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa63c15d770 0x7fa63c15f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa640c70640 1 -- 172.21.15.47:0/391757101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6300094d0 con 0x7fa63c15c9a0 2024-03-23T09:51:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa641c72640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa63c15d770 0x7fa63c15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa640c70640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c15c9a0 0x7fa63c0c8150 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fa630009870 tx=0x7fa630004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa639ffb640 1 -- 172.21.15.47:0/391757101 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630030800 con 0x7fa63c15c9a0 2024-03-23T09:51:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa639ffb640 1 -- 172.21.15.47:0/391757101 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa630043400 con 0x7fa63c15c9a0 2024-03-23T09:51:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa639ffb640 1 -- 172.21.15.47:0/391757101 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6300437b0 con 0x7fa63c15c9a0 2024-03-23T09:51:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa6436d5640 1 -- 172.21.15.47:0/391757101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c1f6420 con 0x7fa63c15c9a0 2024-03-23T09:51:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.163+0000 7fa6436d5640 1 -- 172.21.15.47:0/391757101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c0c8690 con 0x7fa63c15c9a0 2024-03-23T09:51:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.164+0000 7fa6436d5640 1 -- 172.21.15.47:0/391757101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63c15dbc0 con 0x7fa63c15c9a0 2024-03-23T09:51:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.165+0000 7fa639ffb640 1 -- 172.21.15.47:0/391757101 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa63002e070 con 0x7fa63c15c9a0 2024-03-23T09:51:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.165+0000 7fa639ffb640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa62806ac00 0x7fa62806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.165+0000 7fa639ffb640 1 -- 172.21.15.47:0/391757101 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa63003e070 con 0x7fa63c15c9a0 2024-03-23T09:51:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.165+0000 7fa641471640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa62806ac00 0x7fa62806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.166+0000 7fa641471640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa62806ac00 0x7fa62806d0c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa634000f60 tx=0x7fa634009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:44.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.170+0000 7fa639ffb640 1 -- 172.21.15.47:0/391757101 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa630077650 con 0x7fa63c15c9a0 2024-03-23T09:51:44.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.347+0000 7fa6436d5640 1 -- 172.21.15.47:0/391757101 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa63c1614a0 con 0x7fa62806ac00 2024-03-23T09:51:44.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.355+0000 7fa639ffb640 1 -- 172.21.15.47:0/391757101 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290165 (secure 0 0 0) 0x7fa6340012f0 con 0x7fa62806ac00 2024-03-23T09:51:44.358 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:51:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.358+0000 7fa6436d5640 1 -- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa62806ac00 msgr2=0x7fa62806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.358+0000 7fa6436d5640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa62806ac00 0x7fa62806d0c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa634000f60 tx=0x7fa634009040 comp rx=0 tx=0).stop 2024-03-23T09:51:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.358+0000 7fa6436d5640 1 -- 172.21.15.47:0/391757101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c15c9a0 msgr2=0x7fa63c0c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.358+0000 7fa6436d5640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c15c9a0 0x7fa63c0c8150 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fa630009870 tx=0x7fa630004730 comp rx=0 tx=0).stop 2024-03-23T09:51:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.358+0000 7fa6436d5640 1 -- 172.21.15.47:0/391757101 shutdown_connections 2024-03-23T09:51:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.358+0000 7fa6436d5640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa62806ac00 0x7fa62806d0c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.358+0000 7fa6436d5640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa63c15d770 0x7fa63c15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.358+0000 7fa6436d5640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c15c9a0 0x7fa63c0c8150 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.358+0000 7fa6436d5640 1 --2- 172.21.15.47:0/391757101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa63c06bf10 0x7fa63c156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.358+0000 7fa6436d5640 1 -- 172.21.15.47:0/391757101 >> 172.21.15.47:0/391757101 conn(0x7fa63c0d0670 msgr2=0x7fa63c099730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:44.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.358+0000 7fa6436d5640 1 -- 172.21.15.47:0/391757101 shutdown_connections 2024-03-23T09:51:44.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.358+0000 7fa6436d5640 1 -- 172.21.15.47:0/391757101 wait complete. 2024-03-23T09:51:44.369 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-03-23T09:51:42.500407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":273568,"kb_used_data":12288,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754701152,"statfs":{"total":773094113280,"available":772813979648,"internally_reserved":0,"allocated":12582912,"data_stored":6090662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":267374136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000864"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463517+0000","last_change":"2024-03-23T09:51:40.421426+0000","last_active":"2024-03-23T09:51:41.463517+0000","last_peered":"2024-03-23T09:51:41.463517+0000","last_clean":"2024-03-23T09:51:41.463517+0000","last_became_active":"2024-03-23T09:51:40.421186+0000","last_became_peered":"2024-03-23T09:51:40.421186+0000","last_unstale":"2024-03-23T09:51:41.463517+0000","last_undegraded":"2024-03-23T09:51:41.463517+0000","last_fullsized":"2024-03-23T09:51:41.463517+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:52:06.459960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403618+0000","last_change":"2024-03-23T09:51:40.410958+0000","last_active":"2024-03-23T09:51:41.403618+0000","last_peered":"2024-03-23T09:51:41.403618+0000","last_clean":"2024-03-23T09:51:41.403618+0000","last_became_active":"2024-03-23T09:51:40.410685+0000","last_became_peered":"2024-03-23T09:51:40.410685+0000","last_unstale":"2024-03-23T09:51:41.403618+0000","last_undegraded":"2024-03-23T09:51:41.403618+0000","last_fullsized":"2024-03-23T09:51:41.403618+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:08:17.110031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404077+0000","last_change":"2024-03-23T09:51:40.416348+0000","last_active":"2024-03-23T09:51:41.404077+0000","last_peered":"2024-03-23T09:51:41.404077+0000","last_clean":"2024-03-23T09:51:41.404077+0000","last_became_active":"2024-03-23T09:51:40.414713+0000","last_became_peered":"2024-03-23T09:51:40.414713+0000","last_unstale":"2024-03-23T09:51:41.404077+0000","last_undegraded":"2024-03-23T09:51:41.404077+0000","last_fullsized":"2024-03-23T09:51:41.404077+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:15:20.760615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463880+0000","last_change":"2024-03-23T09:51:40.422802+0000","last_active":"2024-03-23T09:51:41.463880+0000","last_peered":"2024-03-23T09:51:41.463880+0000","last_clean":"2024-03-23T09:51:41.463880+0000","last_became_active":"2024-03-23T09:51:40.422513+0000","last_became_peered":"2024-03-23T09:51:40.422513+0000","last_unstale":"2024-03-23T09:51:41.463880+0000","last_undegraded":"2024-03-23T09:51:41.463880+0000","last_fullsized":"2024-03-23T09:51:41.463880+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:08:07.025033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402629+0000","last_change":"2024-03-23T09:51:40.407521+0000","last_active":"2024-03-23T09:51:41.402629+0000","last_peered":"2024-03-23T09:51:41.402629+0000","last_clean":"2024-03-23T09:51:41.402629+0000","last_became_active":"2024-03-23T09:51:40.407256+0000","last_became_peered":"2024-03-23T09:51:40.407256+0000","last_unstale":"2024-03-23T09:51:41.402629+0000","last_undegraded":"2024-03-23T09:51:41.402629+0000","last_fullsized":"2024-03-23T09:51:41.402629+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:36:03.125410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463236+0000","last_change":"2024-03-23T09:51:40.412120+0000","last_active":"2024-03-23T09:51:41.463236+0000","last_peered":"2024-03-23T09:51:41.463236+0000","last_clean":"2024-03-23T09:51:41.463236+0000","last_became_active":"2024-03-23T09:51:40.411874+0000","last_became_peered":"2024-03-23T09:51:40.411874+0000","last_unstale":"2024-03-23T09:51:41.463236+0000","last_undegraded":"2024-03-23T09:51:41.463236+0000","last_fullsized":"2024-03-23T09:51:41.463236+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:52:39.694132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463630+0000","last_change":"2024-03-23T09:51:40.415769+0000","last_active":"2024-03-23T09:51:41.463630+0000","last_peered":"2024-03-23T09:51:41.463630+0000","last_clean":"2024-03-23T09:51:41.463630+0000","last_became_active":"2024-03-23T09:51:40.415505+0000","last_became_peered":"2024-03-23T09:51:40.415505+0000","last_unstale":"2024-03-23T09:51:41.463630+0000","last_undegraded":"2024-03-23T09:51:41.463630+0000","last_fullsized":"2024-03-23T09:51:41.463630+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:59:02.734957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403361+0000","last_change":"2024-03-23T09:51:40.416977+0000","last_active":"2024-03-23T09:51:41.403361+0000","last_peered":"2024-03-23T09:51:41.403361+0000","last_clean":"2024-03-23T09:51:41.403361+0000","last_became_active":"2024-03-23T09:51:40.416548+0000","last_became_peered":"2024-03-23T09:51:40.416548+0000","last_unstale":"2024-03-23T09:51:41.403361+0000","last_undegraded":"2024-03-23T09:51:41.403361+0000","last_fullsized":"2024-03-23T09:51:41.403361+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:21:51.075338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463546+0000","last_change":"2024-03-23T09:51:40.420869+0000","last_active":"2024-03-23T09:51:41.463546+0000","last_peered":"2024-03-23T09:51:41.463546+0000","last_clean":"2024-03-23T09:51:41.463546+0000","last_became_active":"2024-03-23T09:51:40.420615+0000","last_became_peered":"2024-03-23T09:51:40.420615+0000","last_unstale":"2024-03-23T09:51:41.463546+0000","last_undegraded":"2024-03-23T09:51:41.463546+0000","last_fullsized":"2024-03-23T09:51:41.463546+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:40:19.870031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463611+0000","last_change":"2024-03-23T09:51:40.420446+0000","last_active":"2024-03-23T09:51:41.463611+0000","last_peered":"2024-03-23T09:51:41.463611+0000","last_clean":"2024-03-23T09:51:41.463611+0000","last_became_active":"2024-03-23T09:51:40.420182+0000","last_became_peered":"2024-03-23T09:51:40.420182+0000","last_unstale":"2024-03-23T09:51:41.463611+0000","last_undegraded":"2024-03-23T09:51:41.463611+0000","last_fullsized":"2024-03-23T09:51:41.463611+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:48.481687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403560+0000","last_change":"2024-03-23T09:51:40.412690+0000","last_active":"2024-03-23T09:51:41.403560+0000","last_peered":"2024-03-23T09:51:41.403560+0000","last_clean":"2024-03-23T09:51:41.403560+0000","last_became_active":"2024-03-23T09:51:40.412419+0000","last_became_peered":"2024-03-23T09:51:40.412419+0000","last_unstale":"2024-03-23T09:51:41.403560+0000","last_undegraded":"2024-03-23T09:51:41.403560+0000","last_fullsized":"2024-03-23T09:51:41.403560+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:12:46.572433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463952+0000","last_change":"2024-03-23T09:51:40.421571+0000","last_active":"2024-03-23T09:51:41.463952+0000","last_peered":"2024-03-23T09:51:41.463952+0000","last_clean":"2024-03-23T09:51:41.463952+0000","last_became_active":"2024-03-23T09:51:40.421319+0000","last_became_peered":"2024-03-23T09:51:40.421319+0000","last_unstale":"2024-03-23T09:51:41.463952+0000","last_undegraded":"2024-03-23T09:51:41.463952+0000","last_fullsized":"2024-03-23T09:51:41.463952+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:40:36.388768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404063+0000","last_change":"2024-03-23T09:51:40.408493+0000","last_active":"2024-03-23T09:51:41.404063+0000","last_peered":"2024-03-23T09:51:41.404063+0000","last_clean":"2024-03-23T09:51:41.404063+0000","last_became_active":"2024-03-23T09:51:40.408219+0000","last_became_peered":"2024-03-23T09:51:40.408219+0000","last_unstale":"2024-03-23T09:51:41.404063+0000","last_undegraded":"2024-03-23T09:51:41.404063+0000","last_fullsized":"2024-03-23T09:51:41.404063+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:35:26.463429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403856+0000","last_change":"2024-03-23T09:51:40.410894+0000","last_active":"2024-03-23T09:51:41.403856+0000","last_peered":"2024-03-23T09:51:41.403856+0000","last_clean":"2024-03-23T09:51:41.403856+0000","last_became_active":"2024-03-23T09:51:40.410472+0000","last_became_peered":"2024-03-23T09:51:40.410472+0000","last_unstale":"2024-03-23T09:51:41.403856+0000","last_undegraded":"2024-03-23T09:51:41.403856+0000","last_fullsized":"2024-03-23T09:51:41.403856+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:57:09.681811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463018+0000","last_change":"2024-03-23T09:51:40.412237+0000","last_active":"2024-03-23T09:51:41.463018+0000","last_peered":"2024-03-23T09:51:41.463018+0000","last_clean":"2024-03-23T09:51:41.463018+0000","last_became_active":"2024-03-23T09:51:40.411998+0000","last_became_peered":"2024-03-23T09:51:40.411998+0000","last_unstale":"2024-03-23T09:51:41.463018+0000","last_undegraded":"2024-03-23T09:51:41.463018+0000","last_fullsized":"2024-03-23T09:51:41.463018+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:23:50.334771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462889+0000","last_change":"2024-03-23T09:51:40.424475+0000","last_active":"2024-03-23T09:51:41.462889+0000","last_peered":"2024-03-23T09:51:41.462889+0000","last_clean":"2024-03-23T09:51:41.462889+0000","last_became_active":"2024-03-23T09:51:40.424221+0000","last_became_peered":"2024-03-23T09:51:40.424221+0000","last_unstale":"2024-03-23T09:51:41.462889+0000","last_undegraded":"2024-03-23T09:51:41.462889+0000","last_fullsized":"2024-03-23T09:51:41.462889+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:17:25.600706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404075+0000","last_change":"2024-03-23T09:51:40.415022+0000","last_active":"2024-03-23T09:51:41.404075+0000","last_peered":"2024-03-23T09:51:41.404075+0000","last_clean":"2024-03-23T09:51:41.404075+0000","last_became_active":"2024-03-23T09:51:40.414436+0000","last_became_peered":"2024-03-23T09:51:40.414436+0000","last_unstale":"2024-03-23T09:51:41.404075+0000","last_undegraded":"2024-03-23T09:51:41.404075+0000","last_fullsized":"2024-03-23T09:51:41.404075+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:51:23.039272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463344+0000","last_change":"2024-03-23T09:51:40.414029+0000","last_active":"2024-03-23T09:51:41.463344+0000","last_peered":"2024-03-23T09:51:41.463344+0000","last_clean":"2024-03-23T09:51:41.463344+0000","last_became_active":"2024-03-23T09:51:40.413652+0000","last_became_peered":"2024-03-23T09:51:40.413652+0000","last_unstale":"2024-03-23T09:51:41.463344+0000","last_undegraded":"2024-03-23T09:51:41.463344+0000","last_fullsized":"2024-03-23T09:51:41.463344+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:35:26.463429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404224+0000","last_change":"2024-03-23T09:51:40.409377+0000","last_active":"2024-03-23T09:51:41.404224+0000","last_peered":"2024-03-23T09:51:41.404224+0000","last_clean":"2024-03-23T09:51:41.404224+0000","last_became_active":"2024-03-23T09:51:40.409049+0000","last_became_peered":"2024-03-23T09:51:40.409049+0000","last_unstale":"2024-03-23T09:51:41.404224+0000","last_undegraded":"2024-03-23T09:51:41.404224+0000","last_fullsized":"2024-03-23T09:51:41.404224+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:29:13.794262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404074+0000","last_change":"2024-03-23T09:51:40.409221+0000","last_active":"2024-03-23T09:51:41.404074+0000","last_peered":"2024-03-23T09:51:41.404074+0000","last_clean":"2024-03-23T09:51:41.404074+0000","last_became_active":"2024-03-23T09:51:40.408944+0000","last_became_peered":"2024-03-23T09:51:40.408944+0000","last_unstale":"2024-03-23T09:51:41.404074+0000","last_undegraded":"2024-03-23T09:51:41.404074+0000","last_fullsized":"2024-03-23T09:51:41.404074+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:08:01.286016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403914+0000","last_change":"2024-03-23T09:51:40.412732+0000","last_active":"2024-03-23T09:51:41.403914+0000","last_peered":"2024-03-23T09:51:41.403914+0000","last_clean":"2024-03-23T09:51:41.403914+0000","last_became_active":"2024-03-23T09:51:40.412471+0000","last_became_peered":"2024-03-23T09:51:40.412471+0000","last_unstale":"2024-03-23T09:51:41.403914+0000","last_undegraded":"2024-03-23T09:51:41.403914+0000","last_fullsized":"2024-03-23T09:51:41.403914+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:29:13.794262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464103+0000","last_change":"2024-03-23T09:51:40.413442+0000","last_active":"2024-03-23T09:51:41.464103+0000","last_peered":"2024-03-23T09:51:41.464103+0000","last_clean":"2024-03-23T09:51:41.464103+0000","last_became_active":"2024-03-23T09:51:40.412891+0000","last_became_peered":"2024-03-23T09:51:40.412891+0000","last_unstale":"2024-03-23T09:51:41.464103+0000","last_undegraded":"2024-03-23T09:51:41.464103+0000","last_fullsized":"2024-03-23T09:51:41.464103+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:42:30.671652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404192+0000","last_change":"2024-03-23T09:51:40.405226+0000","last_active":"2024-03-23T09:51:41.404192+0000","last_peered":"2024-03-23T09:51:41.404192+0000","last_clean":"2024-03-23T09:51:41.404192+0000","last_became_active":"2024-03-23T09:51:40.404963+0000","last_became_peered":"2024-03-23T09:51:40.404963+0000","last_unstale":"2024-03-23T09:51:41.404192+0000","last_undegraded":"2024-03-23T09:51:41.404192+0000","last_fullsized":"2024-03-23T09:51:41.404192+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:57:09.681811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462633+0000","last_change":"2024-03-23T09:51:40.409602+0000","last_active":"2024-03-23T09:51:41.462633+0000","last_peered":"2024-03-23T09:51:41.462633+0000","last_clean":"2024-03-23T09:51:41.462633+0000","last_became_active":"2024-03-23T09:51:40.409314+0000","last_became_peered":"2024-03-23T09:51:40.409314+0000","last_unstale":"2024-03-23T09:51:41.462633+0000","last_undegraded":"2024-03-23T09:51:41.462633+0000","last_fullsized":"2024-03-23T09:51:41.462633+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:09:00.964135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404163+0000","last_change":"2024-03-23T09:51:40.415280+0000","last_active":"2024-03-23T09:51:41.404163+0000","last_peered":"2024-03-23T09:51:41.404163+0000","last_clean":"2024-03-23T09:51:41.404163+0000","last_became_active":"2024-03-23T09:51:40.414768+0000","last_became_peered":"2024-03-23T09:51:40.414768+0000","last_unstale":"2024-03-23T09:51:41.404163+0000","last_undegraded":"2024-03-23T09:51:41.404163+0000","last_fullsized":"2024-03-23T09:51:41.404163+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:15:20.760615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463664+0000","last_change":"2024-03-23T09:51:40.420428+0000","last_active":"2024-03-23T09:51:41.463664+0000","last_peered":"2024-03-23T09:51:41.463664+0000","last_clean":"2024-03-23T09:51:41.463664+0000","last_became_active":"2024-03-23T09:51:40.420167+0000","last_became_peered":"2024-03-23T09:51:40.420167+0000","last_unstale":"2024-03-23T09:51:41.463664+0000","last_undegraded":"2024-03-23T09:51:41.463664+0000","last_fullsized":"2024-03-23T09:51:41.463664+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:12.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463114+0000","last_change":"2024-03-23T09:51:40.425765+0000","last_active":"2024-03-23T09:51:41.463114+0000","last_peered":"2024-03-23T09:51:41.463114+0000","last_clean":"2024-03-23T09:51:41.463114+0000","last_became_active":"2024-03-23T09:51:40.425534+0000","last_became_peered":"2024-03-23T09:51:40.425534+0000","last_unstale":"2024-03-23T09:51:41.463114+0000","last_undegraded":"2024-03-23T09:51:41.463114+0000","last_fullsized":"2024-03-23T09:51:41.463114+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:45:19.556557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463990+0000","last_change":"2024-03-23T09:51:40.424735+0000","last_active":"2024-03-23T09:51:41.463990+0000","last_peered":"2024-03-23T09:51:41.463990+0000","last_clean":"2024-03-23T09:51:41.463990+0000","last_became_active":"2024-03-23T09:51:40.424488+0000","last_became_peered":"2024-03-23T09:51:40.424488+0000","last_unstale":"2024-03-23T09:51:41.463990+0000","last_undegraded":"2024-03-23T09:51:41.463990+0000","last_fullsized":"2024-03-23T09:51:41.463990+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:10:01.040378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403388+0000","last_change":"2024-03-23T09:51:40.408330+0000","last_active":"2024-03-23T09:51:41.403388+0000","last_peered":"2024-03-23T09:51:41.403388+0000","last_clean":"2024-03-23T09:51:41.403388+0000","last_became_active":"2024-03-23T09:51:40.408067+0000","last_became_peered":"2024-03-23T09:51:40.408067+0000","last_unstale":"2024-03-23T09:51:41.403388+0000","last_undegraded":"2024-03-23T09:51:41.403388+0000","last_fullsized":"2024-03-23T09:51:41.403388+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:45:02.579837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463788+0000","last_change":"2024-03-23T09:51:40.415722+0000","last_active":"2024-03-23T09:51:41.463788+0000","last_peered":"2024-03-23T09:51:41.463788+0000","last_clean":"2024-03-23T09:51:41.463788+0000","last_became_active":"2024-03-23T09:51:40.415454+0000","last_became_peered":"2024-03-23T09:51:40.415454+0000","last_unstale":"2024-03-23T09:51:41.463788+0000","last_undegraded":"2024-03-23T09:51:41.463788+0000","last_fullsized":"2024-03-23T09:51:41.463788+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:46:38.011184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463283+0000","last_change":"2024-03-23T09:51:40.421113+0000","last_active":"2024-03-23T09:51:41.463283+0000","last_peered":"2024-03-23T09:51:41.463283+0000","last_clean":"2024-03-23T09:51:41.463283+0000","last_became_active":"2024-03-23T09:51:40.420849+0000","last_became_peered":"2024-03-23T09:51:40.420849+0000","last_unstale":"2024-03-23T09:51:41.463283+0000","last_undegraded":"2024-03-23T09:51:41.463283+0000","last_fullsized":"2024-03-23T09:51:41.463283+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:38:09.397343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462959+0000","last_change":"2024-03-23T09:51:40.416051+0000","last_active":"2024-03-23T09:51:41.462959+0000","last_peered":"2024-03-23T09:51:41.462959+0000","last_clean":"2024-03-23T09:51:41.462959+0000","last_became_active":"2024-03-23T09:51:40.415621+0000","last_became_peered":"2024-03-23T09:51:40.415621+0000","last_unstale":"2024-03-23T09:51:41.462959+0000","last_undegraded":"2024-03-23T09:51:41.462959+0000","last_fullsized":"2024-03-23T09:51:41.462959+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:44:20.541433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463384+0000","last_change":"2024-03-23T09:51:40.415875+0000","last_active":"2024-03-23T09:51:41.463384+0000","last_peered":"2024-03-23T09:51:41.463384+0000","last_clean":"2024-03-23T09:51:41.463384+0000","last_became_active":"2024-03-23T09:51:40.415628+0000","last_became_peered":"2024-03-23T09:51:40.415628+0000","last_unstale":"2024-03-23T09:51:41.463384+0000","last_undegraded":"2024-03-23T09:51:41.463384+0000","last_fullsized":"2024-03-23T09:51:41.463384+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:43:15.518304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"113'4","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463881+0000","last_change":"2024-03-23T09:51:40.034507+0000","last_active":"2024-03-23T09:51:41.463881+0000","last_peered":"2024-03-23T09:51:41.463881+0000","last_clean":"2024-03-23T09:51:41.463881+0000","last_became_active":"2024-03-23T09:51:40.034258+0000","last_became_peered":"2024-03-23T09:51:40.034258+0000","last_unstale":"2024-03-23T09:51:41.463881+0000","last_undegraded":"2024-03-23T09:51:41.463881+0000","last_fullsized":"2024-03-23T09:51:41.463881+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:14:32.462197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463542+0000","last_change":"2024-03-23T09:51:40.409275+0000","last_active":"2024-03-23T09:51:41.463542+0000","last_peered":"2024-03-23T09:51:41.463542+0000","last_clean":"2024-03-23T09:51:41.463542+0000","last_became_active":"2024-03-23T09:51:40.409033+0000","last_became_peered":"2024-03-23T09:51:40.409033+0000","last_unstale":"2024-03-23T09:51:41.463542+0000","last_undegraded":"2024-03-23T09:51:41.463542+0000","last_fullsized":"2024-03-23T09:51:41.463542+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:08:17.110031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"113'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403756+0000","last_change":"2024-03-23T09:51:40.036209+0000","last_active":"2024-03-23T09:51:41.403756+0000","last_peered":"2024-03-23T09:51:41.403756+0000","last_clean":"2024-03-23T09:51:41.403756+0000","last_became_active":"2024-03-23T09:51:40.035941+0000","last_became_peered":"2024-03-23T09:51:40.035941+0000","last_unstale":"2024-03-23T09:51:41.403756+0000","last_undegraded":"2024-03-23T09:51:41.403756+0000","last_fullsized":"2024-03-23T09:51:41.403756+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:22:39.917877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463341+0000","last_change":"2024-03-23T09:51:40.412139+0000","last_active":"2024-03-23T09:51:41.463341+0000","last_peered":"2024-03-23T09:51:41.463341+0000","last_clean":"2024-03-23T09:51:41.463341+0000","last_became_active":"2024-03-23T09:51:40.411850+0000","last_became_peered":"2024-03-23T09:51:40.411850+0000","last_unstale":"2024-03-23T09:51:41.463341+0000","last_undegraded":"2024-03-23T09:51:41.463341+0000","last_fullsized":"2024-03-23T09:51:41.463341+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:36:03.125410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"113'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402586+0000","last_change":"2024-03-23T09:51:40.042156+0000","last_active":"2024-03-23T09:51:41.402586+0000","last_peered":"2024-03-23T09:51:41.402586+0000","last_clean":"2024-03-23T09:51:41.402586+0000","last_became_active":"2024-03-23T09:51:40.041847+0000","last_became_peered":"2024-03-23T09:51:40.041847+0000","last_unstale":"2024-03-23T09:51:41.402586+0000","last_undegraded":"2024-03-23T09:51:41.402586+0000","last_fullsized":"2024-03-23T09:51:41.402586+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:24:03.629632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404230+0000","last_change":"2024-03-23T09:51:40.410466+0000","last_active":"2024-03-23T09:51:41.404230+0000","last_peered":"2024-03-23T09:51:41.404230+0000","last_clean":"2024-03-23T09:51:41.404230+0000","last_became_active":"2024-03-23T09:51:40.410022+0000","last_became_peered":"2024-03-23T09:51:40.410022+0000","last_unstale":"2024-03-23T09:51:41.404230+0000","last_undegraded":"2024-03-23T09:51:41.404230+0000","last_fullsized":"2024-03-23T09:51:41.404230+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:08:17.110031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"113'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402482+0000","last_change":"2024-03-23T09:51:40.041092+0000","last_active":"2024-03-23T09:51:41.402482+0000","last_peered":"2024-03-23T09:51:41.402482+0000","last_clean":"2024-03-23T09:51:41.402482+0000","last_became_active":"2024-03-23T09:51:40.040799+0000","last_became_peered":"2024-03-23T09:51:40.040799+0000","last_unstale":"2024-03-23T09:51:41.402482+0000","last_undegraded":"2024-03-23T09:51:41.402482+0000","last_fullsized":"2024-03-23T09:51:41.402482+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:27:45.441015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463331+0000","last_change":"2024-03-23T09:51:40.420760+0000","last_active":"2024-03-23T09:51:41.463331+0000","last_peered":"2024-03-23T09:51:41.463331+0000","last_clean":"2024-03-23T09:51:41.463331+0000","last_became_active":"2024-03-23T09:51:40.420258+0000","last_became_peered":"2024-03-23T09:51:40.420258+0000","last_unstale":"2024-03-23T09:51:41.463331+0000","last_undegraded":"2024-03-23T09:51:41.463331+0000","last_fullsized":"2024-03-23T09:51:41.463331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:27:51.398073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"113'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463432+0000","last_change":"2024-03-23T09:51:40.041748+0000","last_active":"2024-03-23T09:51:41.463432+0000","last_peered":"2024-03-23T09:51:41.463432+0000","last_clean":"2024-03-23T09:51:41.463432+0000","last_became_active":"2024-03-23T09:51:40.041452+0000","last_became_peered":"2024-03-23T09:51:40.041452+0000","last_unstale":"2024-03-23T09:51:41.463432+0000","last_undegraded":"2024-03-23T09:51:41.463432+0000","last_fullsized":"2024-03-23T09:51:41.463432+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:37:19.045964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403610+0000","last_change":"2024-03-23T09:51:40.409493+0000","last_active":"2024-03-23T09:51:41.403610+0000","last_peered":"2024-03-23T09:51:41.403610+0000","last_clean":"2024-03-23T09:51:41.403610+0000","last_became_active":"2024-03-23T09:51:40.409251+0000","last_became_peered":"2024-03-23T09:51:40.409251+0000","last_unstale":"2024-03-23T09:51:41.403610+0000","last_undegraded":"2024-03-23T09:51:41.403610+0000","last_fullsized":"2024-03-23T09:51:41.403610+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:23:45.258323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"113'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464215+0000","last_change":"2024-03-23T09:51:40.042878+0000","last_active":"2024-03-23T09:51:41.464215+0000","last_peered":"2024-03-23T09:51:41.464215+0000","last_clean":"2024-03-23T09:51:41.464215+0000","last_became_active":"2024-03-23T09:51:40.042615+0000","last_became_peered":"2024-03-23T09:51:40.042615+0000","last_unstale":"2024-03-23T09:51:41.464215+0000","last_undegraded":"2024-03-23T09:51:41.464215+0000","last_fullsized":"2024-03-23T09:51:41.464215+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:49:12.019134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462907+0000","last_change":"2024-03-23T09:51:40.422062+0000","last_active":"2024-03-23T09:51:41.462907+0000","last_peered":"2024-03-23T09:51:41.462907+0000","last_clean":"2024-03-23T09:51:41.462907+0000","last_became_active":"2024-03-23T09:51:40.421804+0000","last_became_peered":"2024-03-23T09:51:40.421804+0000","last_unstale":"2024-03-23T09:51:41.462907+0000","last_undegraded":"2024-03-23T09:51:41.462907+0000","last_fullsized":"2024-03-23T09:51:41.462907+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:51:53.472429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404261+0000","last_change":"2024-03-23T09:51:40.029982+0000","last_active":"2024-03-23T09:51:41.404261+0000","last_peered":"2024-03-23T09:51:41.404261+0000","last_clean":"2024-03-23T09:51:41.404261+0000","last_became_active":"2024-03-23T09:51:40.029641+0000","last_became_peered":"2024-03-23T09:51:40.029641+0000","last_unstale":"2024-03-23T09:51:41.404261+0000","last_undegraded":"2024-03-23T09:51:41.404261+0000","last_fullsized":"2024-03-23T09:51:41.404261+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:46:38.812091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463549+0000","last_change":"2024-03-23T09:51:40.416691+0000","last_active":"2024-03-23T09:51:41.463549+0000","last_peered":"2024-03-23T09:51:41.463549+0000","last_clean":"2024-03-23T09:51:41.463549+0000","last_became_active":"2024-03-23T09:51:40.416436+0000","last_became_peered":"2024-03-23T09:51:40.416436+0000","last_unstale":"2024-03-23T09:51:41.463549+0000","last_undegraded":"2024-03-23T09:51:41.463549+0000","last_fullsized":"2024-03-23T09:51:41.463549+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:54.096545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462989+0000","last_change":"2024-03-23T09:51:40.041980+0000","last_active":"2024-03-23T09:51:41.462989+0000","last_peered":"2024-03-23T09:51:41.462989+0000","last_clean":"2024-03-23T09:51:41.462989+0000","last_became_active":"2024-03-23T09:51:40.041690+0000","last_became_peered":"2024-03-23T09:51:40.041690+0000","last_unstale":"2024-03-23T09:51:41.462989+0000","last_undegraded":"2024-03-23T09:51:41.462989+0000","last_fullsized":"2024-03-23T09:51:41.462989+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:29:12.788788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463797+0000","last_change":"2024-03-23T09:51:40.415863+0000","last_active":"2024-03-23T09:51:41.463797+0000","last_peered":"2024-03-23T09:51:41.463797+0000","last_clean":"2024-03-23T09:51:41.463797+0000","last_became_active":"2024-03-23T09:51:40.415594+0000","last_became_peered":"2024-03-23T09:51:40.415594+0000","last_unstale":"2024-03-23T09:51:41.463797+0000","last_undegraded":"2024-03-23T09:51:41.463797+0000","last_fullsized":"2024-03-23T09:51:41.463797+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:39:58.810927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402418+0000","last_change":"2024-03-23T09:51:40.044131+0000","last_active":"2024-03-23T09:51:41.402418+0000","last_peered":"2024-03-23T09:51:41.402418+0000","last_clean":"2024-03-23T09:51:41.402418+0000","last_became_active":"2024-03-23T09:51:40.043837+0000","last_became_peered":"2024-03-23T09:51:40.043837+0000","last_unstale":"2024-03-23T09:51:41.402418+0000","last_undegraded":"2024-03-23T09:51:41.402418+0000","last_fullsized":"2024-03-23T09:51:41.402418+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:53.091071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463605+0000","last_change":"2024-03-23T09:51:40.422418+0000","last_active":"2024-03-23T09:51:41.463605+0000","last_peered":"2024-03-23T09:51:41.463605+0000","last_clean":"2024-03-23T09:51:41.463605+0000","last_became_active":"2024-03-23T09:51:40.422098+0000","last_became_peered":"2024-03-23T09:51:40.422098+0000","last_unstale":"2024-03-23T09:51:41.463605+0000","last_undegraded":"2024-03-23T09:51:41.463605+0000","last_fullsized":"2024-03-23T09:51:41.463605+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:23:45.258323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463979+0000","last_change":"2024-03-23T09:51:40.042482+0000","last_active":"2024-03-23T09:51:41.463979+0000","last_peered":"2024-03-23T09:51:41.463979+0000","last_clean":"2024-03-23T09:51:41.463979+0000","last_became_active":"2024-03-23T09:51:40.042210+0000","last_became_peered":"2024-03-23T09:51:40.042210+0000","last_unstale":"2024-03-23T09:51:41.463979+0000","last_undegraded":"2024-03-23T09:51:41.463979+0000","last_fullsized":"2024-03-23T09:51:41.463979+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:42:44.663049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464043+0000","last_change":"2024-03-23T09:51:40.410098+0000","last_active":"2024-03-23T09:51:41.464043+0000","last_peered":"2024-03-23T09:51:41.464043+0000","last_clean":"2024-03-23T09:51:41.464043+0000","last_became_active":"2024-03-23T09:51:40.409868+0000","last_became_peered":"2024-03-23T09:51:40.409868+0000","last_unstale":"2024-03-23T09:51:41.464043+0000","last_undegraded":"2024-03-23T09:51:41.464043+0000","last_fullsized":"2024-03-23T09:51:41.464043+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:15:20.760615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"113'3","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463523+0000","last_change":"2024-03-23T09:51:40.044981+0000","last_active":"2024-03-23T09:51:41.463523+0000","last_peered":"2024-03-23T09:51:41.463523+0000","last_clean":"2024-03-23T09:51:41.463523+0000","last_became_active":"2024-03-23T09:51:40.044497+0000","last_became_peered":"2024-03-23T09:51:40.044497+0000","last_unstale":"2024-03-23T09:51:41.463523+0000","last_undegraded":"2024-03-23T09:51:41.463523+0000","last_fullsized":"2024-03-23T09:51:41.463523+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:32:09.674080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463783+0000","last_change":"2024-03-23T09:51:40.419035+0000","last_active":"2024-03-23T09:51:41.463783+0000","last_peered":"2024-03-23T09:51:41.463783+0000","last_clean":"2024-03-23T09:51:41.463783+0000","last_became_active":"2024-03-23T09:51:40.418772+0000","last_became_peered":"2024-03-23T09:51:40.418772+0000","last_unstale":"2024-03-23T09:51:41.463783+0000","last_undegraded":"2024-03-23T09:51:41.463783+0000","last_fullsized":"2024-03-23T09:51:41.463783+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:27:46.446489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"113'6","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464019+0000","last_change":"2024-03-23T09:51:40.035049+0000","last_active":"2024-03-23T09:51:41.464019+0000","last_peered":"2024-03-23T09:51:41.464019+0000","last_clean":"2024-03-23T09:51:41.464019+0000","last_became_active":"2024-03-23T09:51:40.034609+0000","last_became_peered":"2024-03-23T09:51:40.034609+0000","last_unstale":"2024-03-23T09:51:41.464019+0000","last_undegraded":"2024-03-23T09:51:41.464019+0000","last_fullsized":"2024-03-23T09:51:41.464019+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:52.074712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464157+0000","last_change":"2024-03-23T09:51:40.414165+0000","last_active":"2024-03-23T09:51:41.464157+0000","last_peered":"2024-03-23T09:51:41.464157+0000","last_clean":"2024-03-23T09:51:41.464157+0000","last_became_active":"2024-03-23T09:51:40.413925+0000","last_became_peered":"2024-03-23T09:51:40.413925+0000","last_unstale":"2024-03-23T09:51:41.464157+0000","last_undegraded":"2024-03-23T09:51:41.464157+0000","last_fullsized":"2024-03-23T09:51:41.464157+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:57:09.681811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"113'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463651+0000","last_change":"2024-03-23T09:51:40.038304+0000","last_active":"2024-03-23T09:51:41.463651+0000","last_peered":"2024-03-23T09:51:41.463651+0000","last_clean":"2024-03-23T09:51:41.463651+0000","last_became_active":"2024-03-23T09:51:40.038053+0000","last_became_peered":"2024-03-23T09:51:40.038053+0000","last_unstale":"2024-03-23T09:51:41.463651+0000","last_undegraded":"2024-03-23T09:51:41.463651+0000","last_fullsized":"2024-03-23T09:51:41.463651+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:23:59.943842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464073+0000","last_change":"2024-03-23T09:51:40.409275+0000","last_active":"2024-03-23T09:51:41.464073+0000","last_peered":"2024-03-23T09:51:41.464073+0000","last_clean":"2024-03-23T09:51:41.464073+0000","last_became_active":"2024-03-23T09:51:40.409036+0000","last_became_peered":"2024-03-23T09:51:40.409036+0000","last_unstale":"2024-03-23T09:51:41.464073+0000","last_undegraded":"2024-03-23T09:51:41.464073+0000","last_fullsized":"2024-03-23T09:51:41.464073+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:20.000550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463324+0000","last_change":"2024-03-23T09:51:40.035729+0000","last_active":"2024-03-23T09:51:41.463324+0000","last_peered":"2024-03-23T09:51:41.463324+0000","last_clean":"2024-03-23T09:51:41.463324+0000","last_became_active":"2024-03-23T09:51:40.035430+0000","last_became_peered":"2024-03-23T09:51:40.035430+0000","last_unstale":"2024-03-23T09:51:41.463324+0000","last_undegraded":"2024-03-23T09:51:41.463324+0000","last_fullsized":"2024-03-23T09:51:41.463324+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:10:23.729745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462865+0000","last_change":"2024-03-23T09:51:40.413496+0000","last_active":"2024-03-23T09:51:41.462865+0000","last_peered":"2024-03-23T09:51:41.462865+0000","last_clean":"2024-03-23T09:51:41.462865+0000","last_became_active":"2024-03-23T09:51:40.413169+0000","last_became_peered":"2024-03-23T09:51:40.413169+0000","last_unstale":"2024-03-23T09:51:41.462865+0000","last_undegraded":"2024-03-23T09:51:41.462865+0000","last_fullsized":"2024-03-23T09:51:41.462865+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:20:31.190543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"113'8","reported_seq":19,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402407+0000","last_change":"2024-03-23T09:51:40.035576+0000","last_active":"2024-03-23T09:51:41.402407+0000","last_peered":"2024-03-23T09:51:41.402407+0000","last_clean":"2024-03-23T09:51:41.402407+0000","last_became_active":"2024-03-23T09:51:40.035324+0000","last_became_peered":"2024-03-23T09:51:40.035324+0000","last_unstale":"2024-03-23T09:51:41.402407+0000","last_undegraded":"2024-03-23T09:51:41.402407+0000","last_fullsized":"2024-03-23T09:51:41.402407+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:46:37.005710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402507+0000","last_change":"2024-03-23T09:51:40.421438+0000","last_active":"2024-03-23T09:51:41.402507+0000","last_peered":"2024-03-23T09:51:41.402507+0000","last_clean":"2024-03-23T09:51:41.402507+0000","last_became_active":"2024-03-23T09:51:40.421231+0000","last_became_peered":"2024-03-23T09:51:40.421231+0000","last_unstale":"2024-03-23T09:51:41.402507+0000","last_undegraded":"2024-03-23T09:51:41.402507+0000","last_fullsized":"2024-03-23T09:51:41.402507+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:44:20.541433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"113'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463674+0000","last_change":"2024-03-23T09:51:40.038898+0000","last_active":"2024-03-23T09:51:41.463674+0000","last_peered":"2024-03-23T09:51:41.463674+0000","last_clean":"2024-03-23T09:51:41.463674+0000","last_became_active":"2024-03-23T09:51:40.038438+0000","last_became_peered":"2024-03-23T09:51:40.038438+0000","last_unstale":"2024-03-23T09:51:41.463674+0000","last_undegraded":"2024-03-23T09:51:41.463674+0000","last_fullsized":"2024-03-23T09:51:41.463674+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:23:44.252849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463194+0000","last_change":"2024-03-23T09:51:40.426197+0000","last_active":"2024-03-23T09:51:41.463194+0000","last_peered":"2024-03-23T09:51:41.463194+0000","last_clean":"2024-03-23T09:51:41.463194+0000","last_became_active":"2024-03-23T09:51:40.425939+0000","last_became_peered":"2024-03-23T09:51:40.425939+0000","last_unstale":"2024-03-23T09:51:41.463194+0000","last_undegraded":"2024-03-23T09:51:41.463194+0000","last_fullsized":"2024-03-23T09:51:41.463194+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:22:31.864761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"113'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402555+0000","last_change":"2024-03-23T09:51:40.035425+0000","last_active":"2024-03-23T09:51:41.402555+0000","last_peered":"2024-03-23T09:51:41.402555+0000","last_clean":"2024-03-23T09:51:41.402555+0000","last_became_active":"2024-03-23T09:51:40.034933+0000","last_became_peered":"2024-03-23T09:51:40.034933+0000","last_unstale":"2024-03-23T09:51:41.402555+0000","last_undegraded":"2024-03-23T09:51:41.402555+0000","last_fullsized":"2024-03-23T09:51:41.402555+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:05:27.830265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403492+0000","last_change":"2024-03-23T09:51:40.424246+0000","last_active":"2024-03-23T09:51:41.403492+0000","last_peered":"2024-03-23T09:51:41.403492+0000","last_clean":"2024-03-23T09:51:41.403492+0000","last_became_active":"2024-03-23T09:51:40.423938+0000","last_became_peered":"2024-03-23T09:51:40.423938+0000","last_unstale":"2024-03-23T09:51:41.403492+0000","last_undegraded":"2024-03-23T09:51:41.403492+0000","last_fullsized":"2024-03-23T09:51:41.403492+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:12:46.572433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"113'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463300+0000","last_change":"2024-03-23T09:51:40.042089+0000","last_active":"2024-03-23T09:51:41.463300+0000","last_peered":"2024-03-23T09:51:41.463300+0000","last_clean":"2024-03-23T09:51:41.463300+0000","last_became_active":"2024-03-23T09:51:40.041816+0000","last_became_peered":"2024-03-23T09:51:40.041816+0000","last_unstale":"2024-03-23T09:51:41.463300+0000","last_undegraded":"2024-03-23T09:51:41.463300+0000","last_fullsized":"2024-03-23T09:51:41.463300+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:53:36.094132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.404310+0000","last_change":"2024-03-23T09:51:40.414447+0000","last_active":"2024-03-23T09:51:41.404310+0000","last_peered":"2024-03-23T09:51:41.404310+0000","last_clean":"2024-03-23T09:51:41.404310+0000","last_became_active":"2024-03-23T09:51:40.413785+0000","last_became_peered":"2024-03-23T09:51:40.413785+0000","last_unstale":"2024-03-23T09:51:41.404310+0000","last_undegraded":"2024-03-23T09:51:41.404310+0000","last_fullsized":"2024-03-23T09:51:41.404310+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:23:45.258323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463336+0000","last_change":"2024-03-23T09:51:40.040499+0000","last_active":"2024-03-23T09:51:41.463336+0000","last_peered":"2024-03-23T09:51:41.463336+0000","last_clean":"2024-03-23T09:51:41.463336+0000","last_became_active":"2024-03-23T09:51:40.040208+0000","last_became_peered":"2024-03-23T09:51:40.040208+0000","last_unstale":"2024-03-23T09:51:41.463336+0000","last_undegraded":"2024-03-23T09:51:41.463336+0000","last_fullsized":"2024-03-23T09:51:41.463336+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:12:20.739568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403961+0000","last_change":"2024-03-23T09:51:40.415012+0000","last_active":"2024-03-23T09:51:41.403961+0000","last_peered":"2024-03-23T09:51:41.403961+0000","last_clean":"2024-03-23T09:51:41.403961+0000","last_became_active":"2024-03-23T09:51:40.414748+0000","last_became_peered":"2024-03-23T09:51:40.414748+0000","last_unstale":"2024-03-23T09:51:41.403961+0000","last_undegraded":"2024-03-23T09:51:41.403961+0000","last_fullsized":"2024-03-23T09:51:41.403961+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:51:23.039272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"113'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463729+0000","last_change":"2024-03-23T09:51:40.038160+0000","last_active":"2024-03-23T09:51:41.463729+0000","last_peered":"2024-03-23T09:51:41.463729+0000","last_clean":"2024-03-23T09:51:41.463729+0000","last_became_active":"2024-03-23T09:51:40.037899+0000","last_became_peered":"2024-03-23T09:51:40.037899+0000","last_unstale":"2024-03-23T09:51:41.463729+0000","last_undegraded":"2024-03-23T09:51:41.463729+0000","last_fullsized":"2024-03-23T09:51:41.463729+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:24:03.629632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463434+0000","last_change":"2024-03-23T09:51:40.426694+0000","last_active":"2024-03-23T09:51:41.463434+0000","last_peered":"2024-03-23T09:51:41.463434+0000","last_clean":"2024-03-23T09:51:41.463434+0000","last_became_active":"2024-03-23T09:51:40.426440+0000","last_became_peered":"2024-03-23T09:51:40.426440+0000","last_unstale":"2024-03-23T09:51:41.463434+0000","last_undegraded":"2024-03-23T09:51:41.463434+0000","last_fullsized":"2024-03-23T09:51:41.463434+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:33:21.740753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402313+0000","last_change":"2024-03-23T09:51:40.043294+0000","last_active":"2024-03-23T09:51:41.402313+0000","last_peered":"2024-03-23T09:51:41.402313+0000","last_clean":"2024-03-23T09:51:41.402313+0000","last_became_active":"2024-03-23T09:51:40.043044+0000","last_became_peered":"2024-03-23T09:51:40.043044+0000","last_unstale":"2024-03-23T09:51:41.402313+0000","last_undegraded":"2024-03-23T09:51:41.402313+0000","last_fullsized":"2024-03-23T09:51:41.402313+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:59:01.729483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403667+0000","last_change":"2024-03-23T09:51:40.406701+0000","last_active":"2024-03-23T09:51:41.403667+0000","last_peered":"2024-03-23T09:51:41.403667+0000","last_clean":"2024-03-23T09:51:41.403667+0000","last_became_active":"2024-03-23T09:51:40.406433+0000","last_became_peered":"2024-03-23T09:51:40.406433+0000","last_unstale":"2024-03-23T09:51:41.403667+0000","last_undegraded":"2024-03-23T09:51:41.403667+0000","last_fullsized":"2024-03-23T09:51:41.403667+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:21:37.643702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463029+0000","last_change":"2024-03-23T09:51:40.039390+0000","last_active":"2024-03-23T09:51:41.463029+0000","last_peered":"2024-03-23T09:51:41.463029+0000","last_clean":"2024-03-23T09:51:41.463029+0000","last_became_active":"2024-03-23T09:51:40.039105+0000","last_became_peered":"2024-03-23T09:51:40.039105+0000","last_unstale":"2024-03-23T09:51:41.463029+0000","last_undegraded":"2024-03-23T09:51:41.463029+0000","last_fullsized":"2024-03-23T09:51:41.463029+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:39:14.955692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463796+0000","last_change":"2024-03-23T09:51:40.421674+0000","last_active":"2024-03-23T09:51:41.463796+0000","last_peered":"2024-03-23T09:51:41.463796+0000","last_clean":"2024-03-23T09:51:41.463796+0000","last_became_active":"2024-03-23T09:51:40.420840+0000","last_became_peered":"2024-03-23T09:51:40.420840+0000","last_unstale":"2024-03-23T09:51:41.463796+0000","last_undegraded":"2024-03-23T09:51:41.463796+0000","last_fullsized":"2024-03-23T09:51:41.463796+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:51:23.039272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463046+0000","last_change":"2024-03-23T09:51:40.037608+0000","last_active":"2024-03-23T09:51:41.463046+0000","last_peered":"2024-03-23T09:51:41.463046+0000","last_clean":"2024-03-23T09:51:41.463046+0000","last_became_active":"2024-03-23T09:51:40.037351+0000","last_became_peered":"2024-03-23T09:51:40.037351+0000","last_unstale":"2024-03-23T09:51:41.463046+0000","last_undegraded":"2024-03-23T09:51:41.463046+0000","last_fullsized":"2024-03-23T09:51:41.463046+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:35:25.457955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463709+0000","last_change":"2024-03-23T09:51:40.422399+0000","last_active":"2024-03-23T09:51:41.463709+0000","last_peered":"2024-03-23T09:51:41.463709+0000","last_clean":"2024-03-23T09:51:41.463709+0000","last_became_active":"2024-03-23T09:51:40.422078+0000","last_became_peered":"2024-03-23T09:51:40.422078+0000","last_unstale":"2024-03-23T09:51:41.463709+0000","last_undegraded":"2024-03-23T09:51:41.463709+0000","last_fullsized":"2024-03-23T09:51:41.463709+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:29:13.794262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"113'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464070+0000","last_change":"2024-03-23T09:51:40.039375+0000","last_active":"2024-03-23T09:51:41.464070+0000","last_peered":"2024-03-23T09:51:41.464070+0000","last_clean":"2024-03-23T09:51:41.464070+0000","last_became_active":"2024-03-23T09:51:40.039110+0000","last_became_peered":"2024-03-23T09:51:40.039110+0000","last_unstale":"2024-03-23T09:51:41.464070+0000","last_undegraded":"2024-03-23T09:51:41.464070+0000","last_fullsized":"2024-03-23T09:51:41.464070+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:10:23.729745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463589+0000","last_change":"2024-03-23T09:51:40.423720+0000","last_active":"2024-03-23T09:51:41.463589+0000","last_peered":"2024-03-23T09:51:41.463589+0000","last_clean":"2024-03-23T09:51:41.463589+0000","last_became_active":"2024-03-23T09:51:40.423325+0000","last_became_peered":"2024-03-23T09:51:40.423325+0000","last_unstale":"2024-03-23T09:51:41.463589+0000","last_undegraded":"2024-03-23T09:51:41.463589+0000","last_fullsized":"2024-03-23T09:51:41.463589+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:52:38.664859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"113'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463528+0000","last_change":"2024-03-23T09:51:40.038937+0000","last_active":"2024-03-23T09:51:41.463528+0000","last_peered":"2024-03-23T09:51:41.463528+0000","last_clean":"2024-03-23T09:51:41.463528+0000","last_became_active":"2024-03-23T09:51:40.038576+0000","last_became_peered":"2024-03-23T09:51:40.038576+0000","last_unstale":"2024-03-23T09:51:41.463528+0000","last_undegraded":"2024-03-23T09:51:41.463528+0000","last_fullsized":"2024-03-23T09:51:41.463528+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:26:17.535351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463969+0000","last_change":"2024-03-23T09:51:40.422023+0000","last_active":"2024-03-23T09:51:41.463969+0000","last_peered":"2024-03-23T09:51:41.463969+0000","last_clean":"2024-03-23T09:51:41.463969+0000","last_became_active":"2024-03-23T09:51:40.421766+0000","last_became_peered":"2024-03-23T09:51:40.421766+0000","last_unstale":"2024-03-23T09:51:41.463969+0000","last_undegraded":"2024-03-23T09:51:41.463969+0000","last_fullsized":"2024-03-23T09:51:41.463969+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:23:42.776013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463624+0000","last_change":"2024-03-23T09:51:40.037782+0000","last_active":"2024-03-23T09:51:41.463624+0000","last_peered":"2024-03-23T09:51:41.463624+0000","last_clean":"2024-03-23T09:51:41.463624+0000","last_became_active":"2024-03-23T09:51:40.037486+0000","last_became_peered":"2024-03-23T09:51:40.037486+0000","last_unstale":"2024-03-23T09:51:41.463624+0000","last_undegraded":"2024-03-23T09:51:41.463624+0000","last_fullsized":"2024-03-23T09:51:41.463624+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:08:16.104557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463314+0000","last_change":"2024-03-23T09:51:40.422633+0000","last_active":"2024-03-23T09:51:41.463314+0000","last_peered":"2024-03-23T09:51:41.463314+0000","last_clean":"2024-03-23T09:51:41.463314+0000","last_became_active":"2024-03-23T09:51:40.422270+0000","last_became_peered":"2024-03-23T09:51:40.422270+0000","last_unstale":"2024-03-23T09:51:41.463314+0000","last_undegraded":"2024-03-23T09:51:41.463314+0000","last_fullsized":"2024-03-23T09:51:41.463314+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:21:51.075338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.464066+0000","last_change":"2024-03-23T09:51:40.040645+0000","last_active":"2024-03-23T09:51:41.464066+0000","last_peered":"2024-03-23T09:51:41.464066+0000","last_clean":"2024-03-23T09:51:41.464066+0000","last_became_active":"2024-03-23T09:51:40.040367+0000","last_became_peered":"2024-03-23T09:51:40.040367+0000","last_unstale":"2024-03-23T09:51:41.464066+0000","last_undegraded":"2024-03-23T09:51:41.464066+0000","last_fullsized":"2024-03-23T09:51:41.464066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:39:16.129578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403744+0000","last_change":"2024-03-23T09:51:40.413956+0000","last_active":"2024-03-23T09:51:41.403744+0000","last_peered":"2024-03-23T09:51:41.403744+0000","last_clean":"2024-03-23T09:51:41.403744+0000","last_became_active":"2024-03-23T09:51:40.413537+0000","last_became_peered":"2024-03-23T09:51:40.413537+0000","last_unstale":"2024-03-23T09:51:41.403744+0000","last_undegraded":"2024-03-23T09:51:41.403744+0000","last_fullsized":"2024-03-23T09:51:41.403744+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:20:31.190543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463340+0000","last_change":"2024-03-23T09:51:40.036680+0000","last_active":"2024-03-23T09:51:41.463340+0000","last_peered":"2024-03-23T09:51:41.463340+0000","last_clean":"2024-03-23T09:51:41.463340+0000","last_became_active":"2024-03-23T09:51:40.036394+0000","last_became_peered":"2024-03-23T09:51:40.036394+0000","last_unstale":"2024-03-23T09:51:41.463340+0000","last_undegraded":"2024-03-23T09:51:41.463340+0000","last_fullsized":"2024-03-23T09:51:41.463340+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:01:21.619133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403762+0000","last_change":"2024-03-23T09:51:40.407626+0000","last_active":"2024-03-23T09:51:41.403762+0000","last_peered":"2024-03-23T09:51:41.403762+0000","last_clean":"2024-03-23T09:51:41.403762+0000","last_became_active":"2024-03-23T09:51:40.407341+0000","last_became_peered":"2024-03-23T09:51:40.407341+0000","last_unstale":"2024-03-23T09:51:41.403762+0000","last_undegraded":"2024-03-23T09:51:41.403762+0000","last_fullsized":"2024-03-23T09:51:41.403762+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:35:26.463429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403826+0000","last_change":"2024-03-23T09:51:40.041649+0000","last_active":"2024-03-23T09:51:41.403826+0000","last_peered":"2024-03-23T09:51:41.403826+0000","last_clean":"2024-03-23T09:51:41.403826+0000","last_became_active":"2024-03-23T09:51:40.041352+0000","last_became_peered":"2024-03-23T09:51:40.041352+0000","last_unstale":"2024-03-23T09:51:41.403826+0000","last_undegraded":"2024-03-23T09:51:41.403826+0000","last_fullsized":"2024-03-23T09:51:41.403826+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:32:54.365123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.403468+0000","last_change":"2024-03-23T09:51:40.416853+0000","last_active":"2024-03-23T09:51:41.403468+0000","last_peered":"2024-03-23T09:51:41.403468+0000","last_clean":"2024-03-23T09:51:41.403468+0000","last_became_active":"2024-03-23T09:51:40.416449+0000","last_became_peered":"2024-03-23T09:51:40.416449+0000","last_unstale":"2024-03-23T09:51:41.403468+0000","last_undegraded":"2024-03-23T09:51:41.403468+0000","last_fullsized":"2024-03-23T09:51:41.403468+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:20:31.190543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.402309+0000","last_change":"2024-03-23T09:51:40.040229+0000","last_active":"2024-03-23T09:51:41.402309+0000","last_peered":"2024-03-23T09:51:41.402309+0000","last_clean":"2024-03-23T09:51:41.402309+0000","last_became_active":"2024-03-23T09:51:40.039955+0000","last_became_peered":"2024-03-23T09:51:40.039955+0000","last_unstale":"2024-03-23T09:51:41.402309+0000","last_undegraded":"2024-03-23T09:51:41.402309+0000","last_fullsized":"2024-03-23T09:51:41.402309+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:39:57.805453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463307+0000","last_change":"2024-03-23T09:51:40.420521+0000","last_active":"2024-03-23T09:51:41.463307+0000","last_peered":"2024-03-23T09:51:41.463307+0000","last_clean":"2024-03-23T09:51:41.463307+0000","last_became_active":"2024-03-23T09:51:40.420240+0000","last_became_peered":"2024-03-23T09:51:40.420240+0000","last_unstale":"2024-03-23T09:51:41.463307+0000","last_undegraded":"2024-03-23T09:51:41.463307+0000","last_fullsized":"2024-03-23T09:51:41.463307+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:22:49.724185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463125+0000","last_change":"2024-03-23T09:51:40.038163+0000","last_active":"2024-03-23T09:51:41.463125+0000","last_peered":"2024-03-23T09:51:41.463125+0000","last_clean":"2024-03-23T09:51:41.463125+0000","last_became_active":"2024-03-23T09:51:40.037909+0000","last_became_peered":"2024-03-23T09:51:40.037909+0000","last_unstale":"2024-03-23T09:51:41.463125+0000","last_undegraded":"2024-03-23T09:51:41.463125+0000","last_fullsized":"2024-03-23T09:51:41.463125+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:21:50.069864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.462725+0000","last_change":"2024-03-23T09:51:40.409724+0000","last_active":"2024-03-23T09:51:41.462725+0000","last_peered":"2024-03-23T09:51:41.462725+0000","last_clean":"2024-03-23T09:51:41.462725+0000","last_became_active":"2024-03-23T09:51:40.409073+0000","last_became_peered":"2024-03-23T09:51:40.409073+0000","last_unstale":"2024-03-23T09:51:41.462725+0000","last_undegraded":"2024-03-23T09:51:41.462725+0000","last_fullsized":"2024-03-23T09:51:41.462725+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:40.016012+0000","last_clean_scrub_stamp":"2024-03-23T09:51:40.016012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:23:45.258323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-03-23T09:51:41.463630+0000","last_change":"2024-03-23T09:51:40.040498+0000","last_active":"2024-03-23T09:51:41.463630+0000","last_peered":"2024-03-23T09:51:41.463630+0000","last_clean":"2024-03-23T09:51:41.463630+0000","last_became_active":"2024-03-23T09:51:40.040182+0000","last_became_peered":"2024-03-23T09:51:40.040182+0000","last_unstale":"2024-03-23T09:51:41.463630+0000","last_undegraded":"2024-03-23T09:51:41.463630+0000","last_fullsized":"2024-03-23T09:51:41.463630+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:51:39.010538+0000","last_clean_scrub_stamp":"2024-03-23T09:51:39.010538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:52:30.476534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":261,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-03-23T09:51:40.400826+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:51:40.400826+0000","last_peered":"2024-03-23T09:51:40.400826+0000","last_clean":"2024-03-23T09:51:40.400826+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:51:40.400826+0000","last_undegraded":"2024-03-23T09:51:40.400826+0000","last_fullsized":"2024-03-23T09:51:40.400826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":2019328,"data_stored":1204289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.52200000000000002},"min":{"1min":0.41099999999999998,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56799999999999995,"15min":0.54200000000000004},"min":{"1min":0.47099999999999997,"5min":0.32200000000000001,"15min":0.28199999999999997},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.85999999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56200000000000006,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.77700000000000002,"5min":0.79100000000000004,"15min":0.80400000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55300000000000005,"15min":0.53500000000000003},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.30199999999999999},"max":{"1min":0.68600000000000005,"5min":0.76000000000000001,"15min":0.81299999999999994},"last":0.55600000000000005}]},{"osd":2,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.48899999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68100000000000005,"5min":0.78500000000000003,"15min":0.82499999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80700000000000005,"5min":0.83099999999999996,"15min":0.83599999999999997},"last":0.622}]},{"osd":3,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.54500000000000004},"min":{"1min":0.42799999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71199999999999997,"5min":0.73999999999999999,"15min":0.86199999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.46100000000000002,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.83699999999999997},"last":0.60899999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47699999999999998,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29599999999999999},"max":{"1min":0.64500000000000002,"5min":0.76300000000000001,"15min":0.78200000000000003},"last":0.503},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.47799999999999998,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.29299999999999998},"max":{"1min":0.65600000000000003,"5min":0.74399999999999999,"15min":0.80100000000000005},"last":0.65100000000000002}]},{"osd":5,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48799999999999999,"15min":0.50600000000000001},"min":{"1min":0.32100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69799999999999995,"5min":0.77300000000000002,"15min":0.81499999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.51300000000000001,"15min":0.54300000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.627,"5min":0.78000000000000003,"15min":0.84999999999999998},"last":0.63300000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51600000000000001,"15min":0.55700000000000005},"min":{"1min":0.33900000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75,"5min":0.753,"15min":0.874},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.53200000000000003,"15min":0.56299999999999994},"min":{"1min":0.33900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73999999999999999,"5min":0.76200000000000001,"15min":0.90200000000000002},"last":0.57799999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574980,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1417216,"data_stored":612756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56699999999999995,"15min":0.53000000000000003},"min":{"1min":0.46899999999999997,"5min":0.38,"15min":0.27200000000000002},"max":{"1min":0.73699999999999999,"5min":0.80700000000000005,"15min":0.81000000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.55200000000000005},"min":{"1min":0.439,"5min":0.37,"15min":0.34399999999999997},"max":{"1min":0.73699999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58699999999999997,"15min":0.54200000000000004},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.56000000000000005},"min":{"1min":0.45900000000000002,"5min":0.41299999999999998,"15min":0.32700000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.82999999999999996},"last":0.44700000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55700000000000005,"15min":0.53900000000000003},"min":{"1min":0.45100000000000001,"5min":0.36199999999999999,"15min":0.29699999999999999},"max":{"1min":0.70799999999999996,"5min":0.747,"15min":0.81599999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.56999999999999995},"min":{"1min":0.47499999999999998,"5min":0.39400000000000002,"15min":0.36499999999999999},"max":{"1min":0.78500000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49399999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.435,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.72199999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.505},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.41499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76400000000000001,"5min":0.79000000000000004,"15min":0.81699999999999995},"last":0.51800000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45700000000000002,"15min":0.48499999999999999},"min":{"1min":0.32500000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61099999999999999,"5min":0.69299999999999995,"15min":0.72499999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.47799999999999998,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.623,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59899999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.48399999999999999,"15min":0.50900000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.316},"max":{"1min":0.64400000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.51800000000000002,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70199999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.57699999999999996}]},{"osd":7,"last update":"Sat Mar 23 09:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.72199999999999998,"5min":0.754,"15min":0.82699999999999996},"last":0.627},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.56799999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70799999999999996,"5min":0.71099999999999997,"15min":0.86199999999999999},"last":0.69299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574979,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1425408,"data_stored":613909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.51400000000000001},"min":{"1min":0.38,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.432,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.39400000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54900000000000004,"15min":0.52100000000000002},"min":{"1min":0.42099999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55500000000000005,"15min":0.52900000000000003},"min":{"1min":0.47999999999999998,"5min":0.38300000000000001,"15min":0.36599999999999999},"max":{"1min":0.70199999999999996,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46999999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.54700000000000004},"min":{"1min":0.438,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57199999999999995,"15min":0.54400000000000004},"min":{"1min":0.46999999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72499999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.32100000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.54100000000000004},"min":{"1min":0.51400000000000001,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.72799999999999998,"5min":0.77400000000000002,"15min":0.81599999999999995},"last":0.308},{"interface":"front","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.378,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77400000000000002},"last":0.41099999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45600000000000002,"15min":0.46300000000000002},"min":{"1min":0.32800000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66400000000000003,"5min":0.73099999999999998,"15min":0.73699999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.93600000000000005},"last":0.48899999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.497,"15min":0.497},"min":{"1min":0.33700000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72099999999999997,"5min":0.747,"15min":0.749},"last":0.503},{"interface":"front","average":{"1min":0.496,"5min":0.503,"15min":0.505},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.42499999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:51:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54400000000000004,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.74099999999999999,"5min":0.76200000000000001,"15min":0.78200000000000003},"last":0.442},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1417216,"data_stored":613305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58299999999999996,"15min":0.54200000000000004},"min":{"1min":0.46400000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.84599999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.45500000000000002,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.73299999999999998,"5min":0.754,"15min":0.78600000000000003},"last":0.51200000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53800000000000003,"15min":0.505},"min":{"1min":0.46800000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72999999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.52400000000000002},"min":{"1min":0.437,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69099999999999995,"5min":0.751,"15min":0.77400000000000002},"last":0.58899999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60499999999999998,"15min":0.56899999999999995},"min":{"1min":0.50900000000000001,"5min":0.42799999999999999,"15min":0.39400000000000002},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.80200000000000005},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.496,"5min":0.39700000000000002,"15min":0.372},"max":{"1min":0.75600000000000001,"5min":0.76600000000000001,"15min":0.82199999999999995},"last":0.54600000000000004}]},{"osd":3,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.52900000000000003},"min":{"1min":0.40400000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71499999999999997,"5min":0.753,"15min":1.0369999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.56599999999999995,"15min":0.54000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40500000000000003,"15min":0.374},"max":{"1min":0.77900000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53500000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.46200000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.26500000000000001},"max":{"1min":0.68600000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46500000000000002,"15min":0.49399999999999999},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.253},"max":{"1min":0.65900000000000003,"5min":0.73199999999999998,"15min":1.008},"last":0.55700000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47399999999999998,"15min":0.47499999999999998},"min":{"1min":0.35299999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68300000000000005,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.505,"15min":0.51300000000000001},"min":{"1min":0.35399999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70499999999999996,"5min":0.74199999999999999,"15min":0.82399999999999995},"last":0.49299999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51000000000000001,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.30099999999999999},"max":{"1min":0.71799999999999997,"5min":0.76000000000000001,"15min":1.0069999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.752,"5min":0.77600000000000002,"15min":0.93400000000000005},"last":0.57499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574979,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36620,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335220,"statfs":{"total":96636764160,"available":96599265280,"internally_reserved":0,"allocated":1433600,"data_stored":614939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33200000000000002},"min":{"1min":0.26100000000000001,"5min":0.192,"15min":0.183},"max":{"1min":0.55100000000000005,"5min":0.67800000000000005,"15min":1.45},"last":0.223},{"interface":"front","average":{"1min":0.374,"5min":0.35499999999999998,"15min":0.33800000000000002},"min":{"1min":0.20699999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":1.4099999999999999},"last":0.27000000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34999999999999998,"15min":0.33200000000000002},"min":{"1min":0.22800000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":1.323},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.379,"15min":0.36599999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.58699999999999997,"5min":0.61299999999999999,"15min":0.94599999999999995},"last":0.28599999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.40600000000000003},"min":{"1min":0.29299999999999998,"5min":0.27100000000000002,"15min":0.251},"max":{"1min":0.66400000000000003,"5min":0.68999999999999995,"15min":1.1619999999999999},"last":0.314},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.44500000000000001,"15min":0.42999999999999999},"min":{"1min":0.30299999999999999,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":1.2989999999999999},"last":0.33600000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58099999999999996,"15min":0.56899999999999995},"min":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.35399999999999998},"max":{"1min":0.872,"5min":0.88600000000000001,"15min":1.337},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.58599999999999997,"15min":0.56599999999999995},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78900000000000003,"5min":0.82199999999999995,"15min":2.758},"last":0.46800000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63700000000000001,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":1.276},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.45700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.85799999999999998,"5min":0.91200000000000003,"15min":1.895},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.42599999999999999,"5min":0.41299999999999998,"15min":0.40699999999999997},"max":{"1min":0.77700000000000002,"5min":0.84199999999999997,"15min":5.4539999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.60199999999999998},"min":{"1min":0.46800000000000003,"5min":0.41599999999999998,"15min":0.38200000000000001},"max":{"1min":0.81699999999999995,"5min":0.96499999999999997,"15min":2.8159999999999998},"last":0.48899999999999999}]},{"osd":7,"last update":"Sat Mar 23 09:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41299999999999998},"max":{"1min":0.84899999999999998,"5min":0.94899999999999995,"15min":3.7400000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63500000000000001,"15min":0.63200000000000001},"min":{"1min":0.47499999999999998,"5min":0.443,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":5.4900000000000002},"last":0.5}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1417216,"data_stored":613305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34899999999999998,"15min":0.34599999999999997},"min":{"1min":0.23499999999999999,"5min":0.19500000000000001,"15min":0.191},"max":{"1min":0.503,"5min":0.55500000000000005,"15min":0.60399999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.38900000000000001,"15min":0.377},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.22},"max":{"1min":0.57199999999999995,"5min":0.60799999999999998,"15min":0.872},"last":0.42699999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.374,"15min":0.36799999999999999},"min":{"1min":0.249,"5min":0.23699999999999999,"15min":0.20999999999999999},"max":{"1min":0.53600000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40400000000000003,"15min":0.39300000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.20899999999999999},"max":{"1min":0.57399999999999995,"5min":0.61199999999999999,"15min":0.746},"last":0.51500000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42199999999999999,"15min":0.41299999999999998},"min":{"1min":0.315,"5min":0.27600000000000002,"15min":0.22900000000000001},"max":{"1min":0.58999999999999997,"5min":0.65200000000000002,"15min":0.67100000000000004},"last":0.497},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.40000000000000002},"min":{"1min":0.27500000000000002,"5min":0.27000000000000002,"15min":0.248},"max":{"1min":0.59199999999999997,"5min":0.66000000000000003,"15min":0.72499999999999998},"last":0.44900000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.44900000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81399999999999995,"5min":0.87,"15min":1.216},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.99299999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.625,"15min":0.628},"min":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.373},"max":{"1min":0.85399999999999998,"5min":0.878,"15min":0.95699999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.629,"15min":0.629},"min":{"1min":0.48699999999999999,"5min":0.42099999999999999,"15min":0.40300000000000002},"max":{"1min":0.82799999999999996,"5min":0.98499999999999999,"15min":1.101},"last":0.71999999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86199999999999999,"5min":0.86499999999999999,"15min":0.97299999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.49199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89700000000000002,"5min":0.95599999999999996,"15min":1.0780000000000001},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.49399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.872,"5min":0.93200000000000005,"15min":1.155},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.46100000000000002,"15min":0.41599999999999998},"max":{"1min":0.90700000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1425408,"data_stored":613429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.35599999999999998,"15min":0.34699999999999998},"min":{"1min":0.23000000000000001,"5min":0.185,"15min":0.185},"max":{"1min":0.47899999999999998,"5min":0.63500000000000001,"15min":0.64400000000000002},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.432},"min":{"1min":0.24299999999999999,"5min":0.218,"15min":0.20699999999999999},"max":{"1min":0.56000000000000005,"5min":0.62,"15min":2.7069999999999999},"last":0.32600000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.371,"15min":0.35699999999999998},"min":{"1min":0.218,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.38800000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.69799999999999995},"last":0.309}]},{"osd":3,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.442,"15min":0.42399999999999999},"min":{"1min":0.28599999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.71499999999999997,"5min":0.746,"15min":0.76900000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.45700000000000002},"min":{"1min":0.27000000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.628,"5min":0.73999999999999999,"15min":2.75},"last":0.376}]},{"osd":4,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.47699999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81399999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.621,"15min":0.61699999999999999},"min":{"1min":0.49099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":1.002,"15min":1.002},"last":0.48399999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63,"15min":0.66900000000000004},"min":{"1min":0.48199999999999998,"5min":0.41099999999999998,"15min":0.35299999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":2.778},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.52200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82099999999999995,"5min":0.88800000000000001,"15min":1.0649999999999999},"last":0.45800000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64100000000000001,"15min":0.67700000000000005},"min":{"1min":0.501,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":2.8210000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64800000000000002,"15min":0.63900000000000001},"min":{"1min":0.55400000000000005,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.86199999999999999,"5min":0.93300000000000005,"15min":1.1080000000000001},"last":0.51200000000000001}]},{"osd":7,"last update":"Sat Mar 23 09:51:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.50900000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84299999999999997,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64800000000000002,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":2.7989999999999999},"last":0.57099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574979,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37200,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334640,"statfs":{"total":96636764160,"available":96598671360,"internally_reserved":0,"allocated":2027520,"data_stored":1204730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35399999999999998,"15min":0.33600000000000002},"min":{"1min":0.26100000000000001,"5min":0.20599999999999999,"15min":0.184},"max":{"1min":0.52600000000000002,"5min":0.625,"15min":0.625},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.37,"15min":0.36099999999999999},"min":{"1min":0.26300000000000001,"5min":0.22900000000000001,"15min":0.217},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.64900000000000002},"last":0.40799999999999997}]},{"osd":2,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38800000000000001,"15min":0.372},"min":{"1min":0.30399999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.58899999999999997,"5min":0.63,"15min":0.63},"last":0.318},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.39600000000000002,"15min":0.38100000000000001},"min":{"1min":0.309,"5min":0.22600000000000001,"15min":0.21199999999999999},"max":{"1min":0.54600000000000004,"5min":0.58199999999999996,"15min":0.59399999999999997},"last":0.35199999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.39800000000000002,"15min":0.38900000000000001},"min":{"1min":0.252,"5min":0.217,"15min":0.217},"max":{"1min":0.53000000000000003,"5min":0.57499999999999996,"15min":0.67600000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.45400000000000001,"15min":0.437},"min":{"1min":0.35699999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.624,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.433}]},{"osd":4,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.44500000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.48699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85399999999999998,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.372}]},{"osd":5,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.46100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.60199999999999998},"min":{"1min":0.51700000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.95999999999999996},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.51500000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86799999999999999,"5min":0.91900000000000004,"15min":0.93600000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65300000000000002,"15min":0.65200000000000002},"min":{"1min":0.52300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.56799999999999995}]},{"osd":7,"last update":"Sat Mar 23 09:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.629,"15min":0.61899999999999999},"min":{"1min":0.50900000000000001,"5min":0.40999999999999998,"15min":0.39300000000000002},"max":{"1min":0.79800000000000004,"5min":0.878,"15min":0.93600000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.47999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.53800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:51:44.373 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T09:51:44.373 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-23T09:51:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.517+0000 7fe1578af640 1 Processor -- start 2024-03-23T09:51:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.517+0000 7fe1578af640 1 -- start start 2024-03-23T09:51:44.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.518+0000 7fe1578af640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe150161280 0x7fe1500cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.518+0000 7fe1578af640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe1500cdb30 0x7fe1500b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.518+0000 7fe1578af640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1500d09f0 con 0x7fe15015c960 2024-03-23T09:51:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.518+0000 7fe1578af640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1500cdfc0 con 0x7fe150161280 2024-03-23T09:51:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.518+0000 7fe1578af640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1500b8950 con 0x7fe1500cdb30 2024-03-23T09:51:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.518+0000 7fe154e4a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe150161280 0x7fe1500cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.518+0000 7fe15564b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe15015c960 0x7fe1500b8240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52442/0 (socket says 172.21.15.47:52442) 2024-03-23T09:51:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.518+0000 7fe15564b640 1 -- 172.21.15.47:0/2731009268 learned_addr learned my addr 172.21.15.47:0/2731009268 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.518+0000 7fe155e4c640 1 --2- 172.21.15.47:0/2731009268 >> v2:172.21.15.175:3300/0 conn(0x7fe1500cdb30 0x7fe1500b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.518+0000 7fe154e4a640 1 -- 172.21.15.47:0/2731009268 >> v2:172.21.15.175:3300/0 conn(0x7fe1500cdb30 msgr2=0x7fe1500b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.518+0000 7fe154e4a640 1 --2- 172.21.15.47:0/2731009268 >> v2:172.21.15.175:3300/0 conn(0x7fe1500cdb30 0x7fe1500b6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.518+0000 7fe154e4a640 1 -- 172.21.15.47:0/2731009268 >> v1:172.21.15.47:6789/0 conn(0x7fe15015c960 legacy=0x7fe1500b8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.518+0000 7fe154e4a640 1 -- 172.21.15.47:0/2731009268 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1500b5a50 con 0x7fe150161280 2024-03-23T09:51:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.519+0000 7fe154e4a640 1 --2- 172.21.15.47:0/2731009268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe150161280 0x7fe1500cd5f0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fe1500b2330 tx=0x7fe14002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3759585db80c613f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.519+0000 7fe147fff640 1 -- 172.21.15.47:0/2731009268 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe140031830 con 0x7fe150161280 2024-03-23T09:51:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.519+0000 7fe147fff640 1 -- 172.21.15.47:0/2731009268 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe140031990 con 0x7fe150161280 2024-03-23T09:51:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.519+0000 7fe147fff640 1 -- 172.21.15.47:0/2731009268 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe140031d20 con 0x7fe150161280 2024-03-23T09:51:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.520+0000 7fe1578af640 1 -- 172.21.15.47:0/2731009268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe150161280 msgr2=0x7fe1500cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.520+0000 7fe1578af640 1 --2- 172.21.15.47:0/2731009268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe150161280 0x7fe1500cd5f0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fe1500b2330 tx=0x7fe14002b8d0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.520+0000 7fe1578af640 1 -- 172.21.15.47:0/2731009268 shutdown_connections 2024-03-23T09:51:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.520+0000 7fe1578af640 1 --2- 172.21.15.47:0/2731009268 >> v2:172.21.15.175:3300/0 conn(0x7fe1500cdb30 0x7fe1500b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.520+0000 7fe1578af640 1 --2- 172.21.15.47:0/2731009268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe150161280 0x7fe1500cd5f0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.520+0000 7fe1578af640 1 -- 172.21.15.47:0/2731009268 >> 172.21.15.47:0/2731009268 conn(0x7fe150065dc0 msgr2=0x7fe1500661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.520+0000 7fe1578af640 1 -- 172.21.15.47:0/2731009268 shutdown_connections 2024-03-23T09:51:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.520+0000 7fe1578af640 1 -- 172.21.15.47:0/2731009268 wait complete. 2024-03-23T09:51:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.521+0000 7fe1578af640 1 Processor -- start 2024-03-23T09:51:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.521+0000 7fe1578af640 1 -- start start 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe1578af640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1500cdb30 0x7fe15014a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe1578af640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe15015c960 0x7fe150156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe1578af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe150161280 0x7fe1500b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe1578af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15014ad80 con 0x7fe150161280 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe1578af640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500c8960 con 0x7fe15015c960 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe1578af640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500c7670 con 0x7fe1500cdb30 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe155e4c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe150161280 0x7fe1500b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe155e4c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe150161280 0x7fe1500b7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54616/0 (socket says 172.21.15.47:54616) 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe155e4c640 1 -- 172.21.15.47:0/1259749729 learned_addr learned my addr 172.21.15.47:0/1259749729 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe15564b640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1500cdb30 0x7fe15014a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe154e4a640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe15015c960 0x7fe150156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe155e4c640 1 -- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1500cdb30 msgr2=0x7fe15014a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe155e4c640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1500cdb30 0x7fe15014a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe155e4c640 1 -- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe15015c960 msgr2=0x7fe150156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe155e4c640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe15015c960 0x7fe150156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe155e4c640 1 -- 172.21.15.47:0/1259749729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe140009400 con 0x7fe150161280 2024-03-23T09:51:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe154e4a640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe15015c960 0x7fe150156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe155e4c640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe150161280 0x7fe1500b7d60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fe148007e40 tx=0x7fe14800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe145ffb640 1 -- 172.21.15.47:0/1259749729 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148014620 con 0x7fe150161280 2024-03-23T09:51:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe145ffb640 1 -- 172.21.15.47:0/1259749729 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14800ee90 con 0x7fe150161280 2024-03-23T09:51:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe1578af640 1 -- 172.21.15.47:0/1259749729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1500c78c0 con 0x7fe150161280 2024-03-23T09:51:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.522+0000 7fe145ffb640 1 -- 172.21.15.47:0/1259749729 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480149e0 con 0x7fe150161280 2024-03-23T09:51:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.523+0000 7fe1578af640 1 -- 172.21.15.47:0/1259749729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1500c8160 con 0x7fe150161280 2024-03-23T09:51:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.523+0000 7fe1578af640 1 -- 172.21.15.47:0/1259749729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe150072360 con 0x7fe150161280 2024-03-23T09:51:44.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.524+0000 7fe145ffb640 1 -- 172.21.15.47:0/1259749729 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe148014b40 con 0x7fe150161280 2024-03-23T09:51:44.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.524+0000 7fe145ffb640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe13406ac30 0x7fe13406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.524+0000 7fe145ffb640 1 -- 172.21.15.47:0/1259749729 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe148089c50 con 0x7fe150161280 2024-03-23T09:51:44.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.525+0000 7fe15564b640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe13406ac30 0x7fe13406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.525+0000 7fe15564b640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe13406ac30 0x7fe13406d0f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe14c005880 tx=0x7fe14c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:44.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.529+0000 7fe145ffb640 1 -- 172.21.15.47:0/1259749729 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe14808f020 con 0x7fe150161280 2024-03-23T09:51:44.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.775+0000 7fe1578af640 1 -- 172.21.15.47:0/1259749729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe150066be0 con 0x7fe150161280 2024-03-23T09:51:44.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.779+0000 7fe145ffb640 1 -- 172.21.15.47:0/1259749729 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe148058950 con 0x7fe150161280 2024-03-23T09:51:44.779 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-03-23T09:51:44.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.781+0000 7fe1578af640 1 -- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe13406ac30 msgr2=0x7fe13406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.781+0000 7fe1578af640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe13406ac30 0x7fe13406d0f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe14c005880 tx=0x7fe14c009040 comp rx=0 tx=0).stop 2024-03-23T09:51:44.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.781+0000 7fe1578af640 1 -- 172.21.15.47:0/1259749729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe150161280 msgr2=0x7fe1500b7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.781+0000 7fe1578af640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe150161280 0x7fe1500b7d60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fe148007e40 tx=0x7fe14800e4b0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.781+0000 7fe1578af640 1 -- 172.21.15.47:0/1259749729 shutdown_connections 2024-03-23T09:51:44.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.781+0000 7fe1578af640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe13406ac30 0x7fe13406d0f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.781+0000 7fe1578af640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe150161280 0x7fe1500b7d60 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.781+0000 7fe1578af640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe15015c960 0x7fe150156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.781+0000 7fe1578af640 1 --2- 172.21.15.47:0/1259749729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1500cdb30 0x7fe15014a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.781+0000 7fe1578af640 1 -- 172.21.15.47:0/1259749729 >> 172.21.15.47:0/1259749729 conn(0x7fe150065dc0 msgr2=0x7fe15015b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:44.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.781+0000 7fe1578af640 1 -- 172.21.15.47:0/1259749729 shutdown_connections 2024-03-23T09:51:44.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.781+0000 7fe1578af640 1 -- 172.21.15.47:0/1259749729 wait complete. 2024-03-23T09:51:44.792 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:51:44.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.892+0000 7f416f81e640 1 Processor -- start 2024-03-23T09:51:44.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.892+0000 7f416f81e640 1 -- start start 2024-03-23T09:51:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416f81e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f416815cd00 0x7f41680d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416f81e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f41680d1020 0x7f41680cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416f81e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41680cfba0 con 0x7f41680cf750 2024-03-23T09:51:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416f81e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416815d0e0 con 0x7f416815cd00 2024-03-23T09:51:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416f81e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41680d1540 con 0x7f41680d1020 2024-03-23T09:51:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416d5ba640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f416815cd00 0x7f41680d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416d5ba640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f416815cd00 0x7f41680d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40850/0 (socket says 172.21.15.47:40850) 2024-03-23T09:51:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416d5ba640 1 -- 172.21.15.47:0/1322948160 learned_addr learned my addr 172.21.15.47:0/1322948160 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416cdb9640 1 --2- 172.21.15.47:0/1322948160 >> v2:172.21.15.175:3300/0 conn(0x7f41680d1020 0x7f41680cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416d5ba640 1 -- 172.21.15.47:0/1322948160 >> v2:172.21.15.175:3300/0 conn(0x7f41680d1020 msgr2=0x7f41680cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416d5ba640 1 --2- 172.21.15.47:0/1322948160 >> v2:172.21.15.175:3300/0 conn(0x7f41680d1020 0x7f41680cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416d5ba640 1 -- 172.21.15.47:0/1322948160 >> v1:172.21.15.175:6789/0 conn(0x7f41680cf750 legacy=0x7f41680b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:44.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416d5ba640 1 -- 172.21.15.47:0/1322948160 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41680d1820 con 0x7f416815cd00 2024-03-23T09:51:44.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.893+0000 7f416cdb9640 1 --2- 172.21.15.47:0/1322948160 >> v2:172.21.15.175:3300/0 conn(0x7f41680d1020 0x7f41680cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.894+0000 7f416d5ba640 1 --2- 172.21.15.47:0/1322948160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f416815cd00 0x7f41680d0ae0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f416806c1a0 tx=0x7f416002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f884d8f5d19492a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.894+0000 7f4167fff640 1 -- 172.21.15.47:0/1322948160 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41600317c0 con 0x7f416815cd00 2024-03-23T09:51:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.894+0000 7f4167fff640 1 -- 172.21.15.47:0/1322948160 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4160031920 con 0x7f416815cd00 2024-03-23T09:51:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.894+0000 7f4167fff640 1 -- 172.21.15.47:0/1322948160 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160031cb0 con 0x7f416815cd00 2024-03-23T09:51:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.895+0000 7f416f81e640 1 -- 172.21.15.47:0/1322948160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f416815cd00 msgr2=0x7f41680d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.895+0000 7f416f81e640 1 --2- 172.21.15.47:0/1322948160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f416815cd00 0x7f41680d0ae0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f416806c1a0 tx=0x7f416002b860 comp rx=0 tx=0).stop 2024-03-23T09:51:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.895+0000 7f416f81e640 1 -- 172.21.15.47:0/1322948160 shutdown_connections 2024-03-23T09:51:44.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.895+0000 7f416f81e640 1 --2- 172.21.15.47:0/1322948160 >> v2:172.21.15.175:3300/0 conn(0x7f41680d1020 0x7f41680cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.895+0000 7f416f81e640 1 --2- 172.21.15.47:0/1322948160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f416815cd00 0x7f41680d0ae0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.895+0000 7f416f81e640 1 -- 172.21.15.47:0/1322948160 >> 172.21.15.47:0/1322948160 conn(0x7f416806b160 msgr2=0x7f416806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:44.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.895+0000 7f416f81e640 1 -- 172.21.15.47:0/1322948160 shutdown_connections 2024-03-23T09:51:44.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.895+0000 7f416f81e640 1 -- 172.21.15.47:0/1322948160 wait complete. 2024-03-23T09:51:44.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.896+0000 7f416f81e640 1 Processor -- start 2024-03-23T09:51:44.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.896+0000 7f416f81e640 1 -- start start 2024-03-23T09:51:44.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416f81e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41680cf750 0x7f4168159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416f81e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41680d1020 0x7f41680a9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416f81e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f416815cd00 0x7f41680a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416f81e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41681596a0 con 0x7f41680d1020 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416f81e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41680b1b10 con 0x7f416815cd00 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416f81e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41680b1df0 con 0x7f41680cf750 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416cdb9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41680d1020 0x7f41680a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416cdb9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41680d1020 0x7f41680a9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40854/0 (socket says 172.21.15.47:40854) 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416cdb9640 1 -- 172.21.15.47:0/528515079 learned_addr learned my addr 172.21.15.47:0/528515079 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416d5ba640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41680cf750 0x7f4168159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416ddbb640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f416815cd00 0x7f41680a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416cdb9640 1 -- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41680cf750 msgr2=0x7f4168159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416cdb9640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41680cf750 0x7f4168159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416cdb9640 1 -- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f416815cd00 msgr2=0x7f41680a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416cdb9640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f416815cd00 0x7f41680a9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416cdb9640 1 -- 172.21.15.47:0/528515079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41600094d0 con 0x7f41680d1020 2024-03-23T09:51:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.897+0000 7f416cdb9640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41680d1020 0x7f41680a9330 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f415c00c7f0 tx=0x7f415c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.898+0000 7f4165ffb640 1 -- 172.21.15.47:0/528515079 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c007a30 con 0x7f41680d1020 2024-03-23T09:51:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.898+0000 7f4165ffb640 1 -- 172.21.15.47:0/528515079 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415c007b90 con 0x7f41680d1020 2024-03-23T09:51:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.898+0000 7f4165ffb640 1 -- 172.21.15.47:0/528515079 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c00ce40 con 0x7f41680d1020 2024-03-23T09:51:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.898+0000 7f416f81e640 1 -- 172.21.15.47:0/528515079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41680b20d0 con 0x7f41680d1020 2024-03-23T09:51:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.898+0000 7f416f81e640 1 -- 172.21.15.47:0/528515079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41680b2690 con 0x7f41680d1020 2024-03-23T09:51:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.899+0000 7f416f81e640 1 -- 172.21.15.47:0/528515079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4168072360 con 0x7f41680d1020 2024-03-23T09:51:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.899+0000 7f4165ffb640 1 -- 172.21.15.47:0/528515079 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f415c0113f0 con 0x7f41680d1020 2024-03-23T09:51:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.899+0000 7f4165ffb640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f415406ac00 0x7f415406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.900+0000 7f4165ffb640 1 -- 172.21.15.47:0/528515079 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f415c0895e0 con 0x7f41680d1020 2024-03-23T09:51:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.900+0000 7f416d5ba640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f415406ac00 0x7f415406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:44.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.900+0000 7f416d5ba640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f415406ac00 0x7f415406d0c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4160036000 tx=0x7f416003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:44.904+0000 7f4165ffb640 1 -- 172.21.15.47:0/528515079 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f415c054720 con 0x7f41680d1020 2024-03-23T09:51:45.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.122+0000 7f416f81e640 1 -- 172.21.15.47:0/528515079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41680b28b0 con 0x7f41680d1020 2024-03-23T09:51:45.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.123+0000 7f4165ffb640 1 -- 172.21.15.47:0/528515079 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4733 (secure 0 0 0) 0x7f415c057d40 con 0x7f41680d1020 2024-03-23T09:51:45.123 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:45.123 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:51:40.395539+0000","modified":"2024-03-23T09:51:43.030014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9234":{"gid":9234,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6834/724782441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":724782441},{"type":"v1","addr":"172.21.15.47:6834","nonce":724782441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-23T09:51:45.124 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 80 2024-03-23T09:51:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.125+0000 7f416f81e640 1 -- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f415406ac00 msgr2=0x7f415406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.125+0000 7f416f81e640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f415406ac00 0x7f415406d0c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4160036000 tx=0x7f416003d040 comp rx=0 tx=0).stop 2024-03-23T09:51:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.125+0000 7f416f81e640 1 -- 172.21.15.47:0/528515079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41680d1020 msgr2=0x7f41680a9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.125+0000 7f416f81e640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41680d1020 0x7f41680a9330 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f415c00c7f0 tx=0x7f415c00cbf0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.125+0000 7f416f81e640 1 -- 172.21.15.47:0/528515079 shutdown_connections 2024-03-23T09:51:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.125+0000 7f416f81e640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f415406ac00 0x7f415406d0c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.125+0000 7f416f81e640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f416815cd00 0x7f41680a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.125+0000 7f416f81e640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41680d1020 0x7f41680a9330 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.125+0000 7f416f81e640 1 --2- 172.21.15.47:0/528515079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41680cf750 0x7f4168159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.125+0000 7f416f81e640 1 -- 172.21.15.47:0/528515079 >> 172.21.15.47:0/528515079 conn(0x7f416806b160 msgr2=0x7f41680b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.125+0000 7f416f81e640 1 -- 172.21.15.47:0/528515079 shutdown_connections 2024-03-23T09:51:45.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.125+0000 7f416f81e640 1 -- 172.21.15.47:0/528515079 wait complete. 2024-03-23T09:51:45.538 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:51:40.395539+0000', 'modified': '2024-03-23T09:51:43.030014+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9234}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9234': {'gid': 9234, 'name': 'c', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6834/724782441', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 724782441}, {'type': 'v1', 'addr': '172.21.15.47:6834', 'nonce': 724782441}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:51:45.539 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:51:45.539 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T09:51:45.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.643+0000 7f203ccfc640 1 Processor -- start 2024-03-23T09:51:45.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.643+0000 7f203ccfc640 1 -- start start 2024-03-23T09:51:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.644+0000 7f203ccfc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203815cb40 0x7f2038160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.644+0000 7f203ccfc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20380b8300 0x7f20380cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.644+0000 7f203ccfc640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20380d0ee0 con 0x7f2038160cb0 2024-03-23T09:51:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.644+0000 7f203ccfc640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20380b5a10 con 0x7f20380b8300 2024-03-23T09:51:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.644+0000 7f203ccfc640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20381616f0 con 0x7f203815cb40 2024-03-23T09:51:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.645+0000 7f2035d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20380b8300 0x7f20380cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.645+0000 7f2035d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20380b8300 0x7f20380cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40870/0 (socket says 172.21.15.47:40870) 2024-03-23T09:51:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.645+0000 7f2035d9b640 1 -- 172.21.15.47:0/886605887 learned_addr learned my addr 172.21.15.47:0/886605887 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.645+0000 7f203659c640 1 --2- 172.21.15.47:0/886605887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203815cb40 0x7f2038160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.645+0000 7f2035d9b640 1 -- 172.21.15.47:0/886605887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203815cb40 msgr2=0x7f2038160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.645+0000 7f2035d9b640 1 --2- 172.21.15.47:0/886605887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203815cb40 0x7f2038160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.645+0000 7f2035d9b640 1 -- 172.21.15.47:0/886605887 >> v1:172.21.15.175:6789/0 conn(0x7f2038160cb0 legacy=0x7f20380b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.645+0000 7f2035d9b640 1 -- 172.21.15.47:0/886605887 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2038161130 con 0x7f20380b8300 2024-03-23T09:51:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.646+0000 7f2035d9b640 1 --2- 172.21.15.47:0/886605887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380b8300 0x7f20380cf190 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f2020009e50 tx=0x7f202002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae080c926cc5598d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.646+0000 7f203559a640 1 -- 172.21.15.47:0/886605887 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202002cb00 con 0x7f20380b8300 2024-03-23T09:51:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.646+0000 7f203559a640 1 -- 172.21.15.47:0/886605887 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202002cc60 con 0x7f20380b8300 2024-03-23T09:51:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.646+0000 7f203559a640 1 -- 172.21.15.47:0/886605887 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200343f0 con 0x7f20380b8300 2024-03-23T09:51:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.646+0000 7f203ccfc640 1 -- 172.21.15.47:0/886605887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380b8300 msgr2=0x7f20380cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.646+0000 7f203ccfc640 1 --2- 172.21.15.47:0/886605887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380b8300 0x7f20380cf190 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f2020009e50 tx=0x7f202002bf60 comp rx=0 tx=0).stop 2024-03-23T09:51:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.646+0000 7f203ccfc640 1 -- 172.21.15.47:0/886605887 shutdown_connections 2024-03-23T09:51:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.646+0000 7f203ccfc640 1 --2- 172.21.15.47:0/886605887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20380b8300 0x7f20380cf190 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.646+0000 7f203ccfc640 1 --2- 172.21.15.47:0/886605887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203815cb40 0x7f2038160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.646+0000 7f203ccfc640 1 -- 172.21.15.47:0/886605887 >> 172.21.15.47:0/886605887 conn(0x7f203806b160 msgr2=0x7f203806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.646+0000 7f203ccfc640 1 -- 172.21.15.47:0/886605887 shutdown_connections 2024-03-23T09:51:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.646+0000 7f203ccfc640 1 -- 172.21.15.47:0/886605887 wait complete. 2024-03-23T09:51:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.648+0000 7f203ccfc640 1 Processor -- start 2024-03-23T09:51:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.648+0000 7f203ccfc640 1 -- start start 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.648+0000 7f203ccfc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20380b8300 0x7f2038155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.648+0000 7f203ccfc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203815cb40 0x7f203815d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.648+0000 7f203ccfc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2038160cb0 0x7f203815d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.648+0000 7f203ccfc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203815e3e0 con 0x7f2038160cb0 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.648+0000 7f203ccfc640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203815dd90 con 0x7f20380b8300 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.648+0000 7f203ccfc640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203815e070 con 0x7f203815cb40 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f2036d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2038160cb0 0x7f203815d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f2036d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2038160cb0 0x7f203815d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40886/0 (socket says 172.21.15.47:40886) 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f2036d9d640 1 -- 172.21.15.47:0/849195736 learned_addr learned my addr 172.21.15.47:0/849195736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f203659c640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20380b8300 0x7f2038155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f2035d9b640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203815cb40 0x7f203815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f2036d9d640 1 -- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203815cb40 msgr2=0x7f203815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f2036d9d640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203815cb40 0x7f203815d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f2036d9d640 1 -- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20380b8300 msgr2=0x7f2038155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f2036d9d640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20380b8300 0x7f2038155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f2036d9d640 1 -- 172.21.15.47:0/849195736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2020009b30 con 0x7f2038160cb0 2024-03-23T09:51:45.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f2036d9d640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2038160cb0 0x7f203815d760 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f203000c390 tx=0x7f203000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:45.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f202effd640 1 -- 172.21.15.47:0/849195736 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20300052d0 con 0x7f2038160cb0 2024-03-23T09:51:45.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f202effd640 1 -- 172.21.15.47:0/849195736 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2030005430 con 0x7f2038160cb0 2024-03-23T09:51:45.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f202effd640 1 -- 172.21.15.47:0/849195736 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030003810 con 0x7f2038160cb0 2024-03-23T09:51:45.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f203ccfc640 1 -- 172.21.15.47:0/849195736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203815e6a0 con 0x7f2038160cb0 2024-03-23T09:51:45.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.649+0000 7f203ccfc640 1 -- 172.21.15.47:0/849195736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20380b4d50 con 0x7f2038160cb0 2024-03-23T09:51:45.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.651+0000 7f202effd640 1 -- 172.21.15.47:0/849195736 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2030003970 con 0x7f2038160cb0 2024-03-23T09:51:45.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.651+0000 7f202effd640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f201806ac00 0x7f201806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.651+0000 7f202effd640 1 -- 172.21.15.47:0/849195736 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20300887d0 con 0x7f2038160cb0 2024-03-23T09:51:45.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.651+0000 7f203659c640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f201806ac00 0x7f201806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.652+0000 7f203659c640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f201806ac00 0x7f201806d0c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2028006fd0 tx=0x7f2028008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.652+0000 7f203f86b740 1 Processor -- start 2024-03-23T09:51:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.652+0000 7f203f86b740 1 -- start start 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.653+0000 7f203f86b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 0x564061348d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.653+0000 7f203f86b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 0x5640613492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.653+0000 7f203f86b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564061343c70 0x564061373a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.653+0000 7f203f86b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640612b49b0 con 0x5640613477d0 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.653+0000 7f203f86b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640612b2140 con 0x564061343c70 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.653+0000 7f203f86b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640612ada30 con 0x564061391750 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.653+0000 7f2035d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 0x5640613492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.653+0000 7f2035d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 0x5640613492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40888/0 (socket says 172.21.15.47:40888) 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.653+0000 7f2035d9b640 1 -- 172.21.15.47:0/4038663469 learned_addr learned my addr 172.21.15.47:0/4038663469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.653+0000 7f203659c640 1 --2- 172.21.15.47:0/4038663469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 0x564061348d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.653+0000 7f2036d9d640 1 --2- 172.21.15.47:0/4038663469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564061343c70 0x564061373a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.653+0000 7f2035d9b640 1 -- 172.21.15.47:0/4038663469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 msgr2=0x564061348d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.654+0000 7f2035d9b640 1 --2- 172.21.15.47:0/4038663469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 0x564061348d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.654+0000 7f2035d9b640 1 -- 172.21.15.47:0/4038663469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564061343c70 msgr2=0x564061373a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.654+0000 7f2035d9b640 1 --2- 172.21.15.47:0/4038663469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564061343c70 0x564061373a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.654+0000 7f2035d9b640 1 -- 172.21.15.47:0/4038663469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56406123a2f0 con 0x5640613477d0 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.654+0000 7f203659c640 1 --2- 172.21.15.47:0/4038663469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 0x564061348d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.654+0000 7f2036d9d640 1 --2- 172.21.15.47:0/4038663469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564061343c70 0x564061373a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.654+0000 7f2035d9b640 1 --2- 172.21.15.47:0/4038663469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 0x5640613492c0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f2020000f80 tx=0x7f202002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826da3044e1e7c7c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.654+0000 7f2017fff640 1 -- 172.21.15.47:0/4038663469 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202003f440 con 0x5640613477d0 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.654+0000 7f2017fff640 1 -- 172.21.15.47:0/4038663469 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202003f5a0 con 0x5640613477d0 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.654+0000 7f2017fff640 1 -- 172.21.15.47:0/4038663469 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202003f910 con 0x5640613477d0 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.654+0000 7f203f86b740 1 -- 172.21.15.47:0/4038663469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 msgr2=0x5640613492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.654+0000 7f203f86b740 1 --2- 172.21.15.47:0/4038663469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 0x5640613492c0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f2020000f80 tx=0x7f202002e000 comp rx=0 tx=0).stop 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.655+0000 7f203f86b740 1 -- 172.21.15.47:0/4038663469 shutdown_connections 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.655+0000 7f203f86b740 1 --2- 172.21.15.47:0/4038663469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564061343c70 0x564061373a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.655+0000 7f203f86b740 1 --2- 172.21.15.47:0/4038663469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 0x5640613492c0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.655+0000 7f203f86b740 1 --2- 172.21.15.47:0/4038663469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 0x564061348d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.655+0000 7f203f86b740 1 -- 172.21.15.47:0/4038663469 >> 172.21.15.47:0/4038663469 conn(0x564061345ee0 msgr2=0x56406138ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.655+0000 7f203f86b740 1 -- 172.21.15.47:0/4038663469 shutdown_connections 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.655+0000 7f203f86b740 1 -- 172.21.15.47:0/4038663469 wait complete. 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.655+0000 7f203f86b740 1 -- start start 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.655+0000 7f203f86b740 1 Processor -- start 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.656+0000 7f203f86b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640613477d0 0x564061377bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.656+0000 7f203f86b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 0x56406136ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.656+0000 7f203f86b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 0x56406136f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.656+0000 7f203f86b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56406126c090 con 0x5640613967d0 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.656+0000 7f203f86b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56406126bd30 con 0x5640613477d0 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.656+0000 7f203f86b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640612312a0 con 0x564061391750 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.656+0000 7f203f86b740 10 client.0 _ll_register_callbacks cb 0x564061267050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.656+0000 7f2036d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 0x56406136f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.656+0000 7f2036d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 0x56406136f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40892/0 (socket says 172.21.15.47:40892) 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.656+0000 7f2036d9d640 1 -- 172.21.15.47:0/764801215 learned_addr learned my addr 172.21.15.47:0/764801215 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f203659c640 1 --2- 172.21.15.47:0/764801215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640613477d0 0x564061377bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f2035d9b640 1 --2- 172.21.15.47:0/764801215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 0x56406136ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f2036d9d640 1 -- 172.21.15.47:0/764801215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 msgr2=0x56406136ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f2036d9d640 1 --2- 172.21.15.47:0/764801215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 0x56406136ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f2036d9d640 1 -- 172.21.15.47:0/764801215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640613477d0 msgr2=0x564061377bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f2036d9d640 1 --2- 172.21.15.47:0/764801215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640613477d0 0x564061377bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f2036d9d640 1 -- 172.21.15.47:0/764801215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2038161130 con 0x5640613967d0 2024-03-23T09:51:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f2036d9d640 1 --2- 172.21.15.47:0/764801215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 0x56406136f2a0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f203005f530 tx=0x7f2030057860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4175a93821c6f0e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f20157fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:51:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f20157fa640 1 -- 172.21.15.47:0/764801215 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203005b0d0 con 0x5640613967d0 2024-03-23T09:51:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f20157fa640 1 -- 172.21.15.47:0/764801215 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f203005b230 con 0x5640613967d0 2024-03-23T09:51:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f203f86b740 1 -- 172.21.15.47:0/764801215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564060fa13c0 con 0x5640613967d0 2024-03-23T09:51:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f20157fa640 1 -- 172.21.15.47:0/764801215 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f203005f5d0 con 0x5640613967d0 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f203f86b740 10 client.9378 fetch_fsmap learned FSMap version 80 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f203f86b740 1 -- 172.21.15.47:0/764801215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x5640612ac0b0 con 0x5640613967d0 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f20157fa640 1 -- 172.21.15.47:0/764801215 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564060fa13c0 con 0x5640613967d0 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f20157fa640 1 -- 172.21.15.47:0/764801215 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 80) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f203005d430 con 0x5640613967d0 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f203f86b740 10 client.9378 fetch_fsmap finished waiting for FSMap version 80 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f203f86b740 10 client.9378 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f203f86b740 20 client.9378 populate_metadata read hostname 'smithi047' 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f203f86b740 1 --2- 172.21.15.47:0/764801215 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x564061356ce0 0x5640613570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f203f86b740 4 client.9378 mds_command: new command op to 9234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.657+0000 7f203f86b740 1 -- 172.21.15.47:0/764801215 --> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564060fb26c0 con 0x564061356ce0 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.658+0000 7f203659c640 1 --2- 172.21.15.47:0/764801215 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x564061356ce0 0x5640613570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.658+0000 7f203659c640 1 --2- 172.21.15.47:0/764801215 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x564061356ce0 0x5640613570c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b40270b1c8626f4a server_cookie=6bd957ca64c1b6e8 in_seq=0 out_seq=0 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.658+0000 7f20157fa640 10 client.9378 ms_handle_connect on v2:172.21.15.47:6832/724782441 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.658+0000 7f20157fa640 1 -- 172.21.15.47:0/764801215 <== mds.0 v2:172.21.15.47:6832/724782441 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564060fb26c0 con 0x564061356ce0 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.658+0000 7f20157fa640 10 client.9378 handle_command_reply: tid=0 2024-03-23T09:51:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.658+0000 7f203f86b740 1 client.9378 shutdown 2024-03-23T09:51:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.659+0000 7f203f86b740 1 -- 172.21.15.47:0/764801215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 msgr2=0x56406136f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.659+0000 7f203f86b740 1 --2- 172.21.15.47:0/764801215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 0x56406136f2a0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f203005f530 tx=0x7f2030057860 comp rx=0 tx=0).stop 2024-03-23T09:51:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.659+0000 7f203f86b740 1 -- 172.21.15.47:0/764801215 shutdown_connections 2024-03-23T09:51:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.659+0000 7f203f86b740 1 --2- 172.21.15.47:0/764801215 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x564061356ce0 0x5640613570c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.659+0000 7f203f86b740 1 --2- 172.21.15.47:0/764801215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 0x56406136f2a0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.659+0000 7f203f86b740 1 --2- 172.21.15.47:0/764801215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 0x56406136ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.659+0000 7f20157fa640 0 client.9378 ms_handle_reset on v2:172.21.15.47:6832/724782441 2024-03-23T09:51:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.659+0000 7f203f86b740 1 --2- 172.21.15.47:0/764801215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640613477d0 0x564061377bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.659+0000 7f203f86b740 1 -- 172.21.15.47:0/764801215 >> 172.21.15.47:0/764801215 conn(0x564061343c70 msgr2=0x564061347fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.659+0000 7f203f86b740 1 -- 172.21.15.47:0/764801215 shutdown_connections 2024-03-23T09:51:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.659+0000 7f203f86b740 1 -- 172.21.15.47:0/764801215 wait complete. 2024-03-23T09:51:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.659+0000 7f203f86b740 20 client.9378 trim_cache size 0 max 16384 2024-03-23T09:51:45.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.674+0000 7f203f86b740 1 Processor -- start 2024-03-23T09:51:45.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.674+0000 7f203f86b740 1 -- start start 2024-03-23T09:51:45.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203f86b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 0x56406136ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203f86b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 0x564061349a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203f86b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640613967d0 0x564061349fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203f86b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564061396660 con 0x5640613477d0 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203f86b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56406127e510 con 0x5640613967d0 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203f86b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x56406136b330 con 0x564061391750 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 0x56406136ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 0x56406136ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40908/0 (socket says 172.21.15.47:40908) 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203659c640 1 -- 172.21.15.47:0/2022061901 learned_addr learned my addr 172.21.15.47:0/2022061901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f2035d9b640 1 --2- 172.21.15.47:0/2022061901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 0x564061349a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f2036d9d640 1 --2- 172.21.15.47:0/2022061901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640613967d0 0x564061349fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203659c640 1 -- 172.21.15.47:0/2022061901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 msgr2=0x564061349a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203659c640 1 --2- 172.21.15.47:0/2022061901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 0x564061349a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203659c640 1 -- 172.21.15.47:0/2022061901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640613967d0 msgr2=0x564061349fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203659c640 1 --2- 172.21.15.47:0/2022061901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640613967d0 0x564061349fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203659c640 1 -- 172.21.15.47:0/2022061901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56406124e430 con 0x5640613477d0 2024-03-23T09:51:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f203659c640 1 --2- 172.21.15.47:0/2022061901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 0x56406136ed60 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f2028010620 tx=0x7f2028016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e557acae28b8b1ae server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f202cff9640 1 -- 172.21.15.47:0/2022061901 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028029070 con 0x5640613477d0 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f202cff9640 1 -- 172.21.15.47:0/2022061901 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2028016e50 con 0x5640613477d0 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.675+0000 7f202cff9640 1 -- 172.21.15.47:0/2022061901 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028004030 con 0x5640613477d0 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.676+0000 7f203f86b740 1 -- 172.21.15.47:0/2022061901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 msgr2=0x56406136ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.676+0000 7f203f86b740 1 --2- 172.21.15.47:0/2022061901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 0x56406136ed60 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f2028010620 tx=0x7f2028016000 comp rx=0 tx=0).stop 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.676+0000 7f203f86b740 1 -- 172.21.15.47:0/2022061901 shutdown_connections 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.676+0000 7f203f86b740 1 --2- 172.21.15.47:0/2022061901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640613967d0 0x564061349fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.676+0000 7f203f86b740 1 --2- 172.21.15.47:0/2022061901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564061391750 0x564061349a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.676+0000 7f203f86b740 1 --2- 172.21.15.47:0/2022061901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613477d0 0x56406136ed60 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.676+0000 7f203f86b740 1 -- 172.21.15.47:0/2022061901 >> 172.21.15.47:0/2022061901 conn(0x564061343c70 msgr2=0x5640613581b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.676+0000 7f203f86b740 1 -- 172.21.15.47:0/2022061901 shutdown_connections 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.676+0000 7f203f86b740 1 -- 172.21.15.47:0/2022061901 wait complete. 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.676+0000 7f203f86b740 1 -- start start 2024-03-23T09:51:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.677+0000 7f203f86b740 1 Processor -- start 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.677+0000 7f203f86b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640613477d0 0x5640613f4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.677+0000 7f203f86b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564061391750 0x5640613a7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.677+0000 7f203f86b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 0x5640613f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.677+0000 7f203f86b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640613f8cb0 con 0x5640613967d0 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.677+0000 7f203f86b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640613a7740 con 0x564061391750 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.677+0000 7f203f86b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640613f4970 con 0x5640613477d0 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.677+0000 7f203f86b740 10 client.0 _ll_register_callbacks cb 0x7f20200092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f2036d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 0x5640613f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f2036d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 0x5640613f80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40918/0 (socket says 172.21.15.47:40918) 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f2036d9d640 1 -- 172.21.15.47:0/3060979751 learned_addr learned my addr 172.21.15.47:0/3060979751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f203659c640 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640613477d0 0x5640613f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f2035d9b640 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564061391750 0x5640613a7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f2036d9d640 1 -- 172.21.15.47:0/3060979751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640613477d0 msgr2=0x5640613f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f2036d9d640 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640613477d0 0x5640613f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f2036d9d640 1 -- 172.21.15.47:0/3060979751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564061391750 msgr2=0x5640613a7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f2036d9d640 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564061391750 0x5640613a7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f2036d9d640 1 -- 172.21.15.47:0/3060979751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202800e040 con 0x5640613967d0 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f2035d9b640 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564061391750 0x5640613a7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f2036d9d640 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 0x5640613f80a0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f2030053c20 tx=0x7f2030086bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5f8a114b479d9d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f20157fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f20157fa640 1 -- 172.21.15.47:0/3060979751 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203005f7d0 con 0x5640613967d0 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f20157fa640 1 -- 172.21.15.47:0/3060979751 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2030086c40 con 0x5640613967d0 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f203f86b740 1 -- 172.21.15.47:0/3060979751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f202003f5a0 con 0x5640613967d0 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f20157fa640 1 -- 172.21.15.47:0/3060979751 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f203000e040 con 0x5640613967d0 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f203f86b740 10 client.9384 fetch_fsmap learned FSMap version 80 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f203f86b740 1 -- 172.21.15.47:0/3060979751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x564061273b40 con 0x5640613967d0 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.678+0000 7f20157fa640 1 -- 172.21.15.47:0/3060979751 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202003f5a0 con 0x5640613967d0 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f20157fa640 1 -- 172.21.15.47:0/3060979751 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 80) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2030053510 con 0x5640613967d0 2024-03-23T09:51:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 10 client.9384 fetch_fsmap finished waiting for FSMap version 80 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 10 client.9384 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 20 client.9384 populate_metadata read hostname 'smithi047' 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x56406139dee0 0x564061349a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 4 client.9384 mds_command: new command op to 9234 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 1 -- 172.21.15.47:0/3060979751 --> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564061005d10 con 0x56406139dee0 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203659c640 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x56406139dee0 0x564061349a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203659c640 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x56406139dee0 0x564061349a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7527264ce5ec7b6 server_cookie=2741ecfc9554620e in_seq=0 out_seq=0 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f20157fa640 10 client.9384 ms_handle_connect on v2:172.21.15.47:6832/724782441 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f20157fa640 1 -- 172.21.15.47:0/3060979751 <== mds.0 v2:172.21.15.47:6832/724782441 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x564061005d10 con 0x56406139dee0 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f20157fa640 10 client.9384 handle_command_reply: tid=0 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 1 client.9384 shutdown 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 1 -- 172.21.15.47:0/3060979751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 msgr2=0x5640613f80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 0x5640613f80a0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f2030053c20 tx=0x7f2030086bd0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 1 -- 172.21.15.47:0/3060979751 shutdown_connections 2024-03-23T09:51:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x56406139dee0 0x564061349a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640613967d0 0x5640613f80a0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564061391750 0x5640613a7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f20157fa640 0 client.9384 ms_handle_reset on v2:172.21.15.47:6832/724782441 2024-03-23T09:51:45.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 1 --2- 172.21.15.47:0/3060979751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640613477d0 0x5640613f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 1 -- 172.21.15.47:0/3060979751 >> 172.21.15.47:0/3060979751 conn(0x564061343c70 msgr2=0x5640613779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:45.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.679+0000 7f203f86b740 1 -- 172.21.15.47:0/3060979751 shutdown_connections 2024-03-23T09:51:45.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203f86b740 1 -- 172.21.15.47:0/3060979751 wait complete. 2024-03-23T09:51:45.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203f86b740 20 client.9384 trim_cache size 0 max 16384 2024-03-23T09:51:45.686 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:51:45.686 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:51:45.687 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:51:45.687 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 9234, 2024-03-23T09:51:45.687 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:51:45.687 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:51:45.687 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:51:45.687 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 5.2832196619999996, 2024-03-23T09:51:45.687 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 80, 2024-03-23T09:51:45.687 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 114, 2024-03-23T09:51:45.687 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 114, 2024-03-23T09:51:45.687 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 9.6658655420000006 2024-03-23T09:51:45.687 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:51:45.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203ccfc640 1 -- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f201806ac00 msgr2=0x7f201806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203ccfc640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f201806ac00 0x7f201806d0c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2028006fd0 tx=0x7f2028008040 comp rx=0 tx=0).stop 2024-03-23T09:51:45.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203ccfc640 1 -- 172.21.15.47:0/849195736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2038160cb0 msgr2=0x7f203815d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203ccfc640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2038160cb0 0x7f203815d760 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f203000c390 tx=0x7f203000c850 comp rx=0 tx=0).stop 2024-03-23T09:51:45.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203ccfc640 1 -- 172.21.15.47:0/849195736 shutdown_connections 2024-03-23T09:51:45.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203ccfc640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f201806ac00 0x7f201806d0c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203ccfc640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2038160cb0 0x7f203815d760 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203ccfc640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203815cb40 0x7f203815d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203ccfc640 1 --2- 172.21.15.47:0/849195736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20380b8300 0x7f2038155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203ccfc640 1 -- 172.21.15.47:0/849195736 >> 172.21.15.47:0/849195736 conn(0x7f203806b160 msgr2=0x7f2038157170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:45.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203ccfc640 1 -- 172.21.15.47:0/849195736 shutdown_connections 2024-03-23T09:51:45.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.680+0000 7f203ccfc640 1 -- 172.21.15.47:0/849195736 wait complete. 2024-03-23T09:51:45.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:51:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.792+0000 7fd77d92e640 1 Processor -- start 2024-03-23T09:51:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.792+0000 7fd77d92e640 1 -- start start 2024-03-23T09:51:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.793+0000 7fd77d92e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd7780cd6b0 0x7fd7780cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.793+0000 7fd77d92e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd7780d1420 0x7fd7780d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.793+0000 7fd77d92e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7780d09d0 con 0x7fd77815cc40 2024-03-23T09:51:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.793+0000 7fd77d92e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7780ce080 con 0x7fd7780d1420 2024-03-23T09:51:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.793+0000 7fd77d92e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7780b5a80 con 0x7fd7780cd6b0 2024-03-23T09:51:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.793+0000 7fd776ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd77815cc40 0x7fd778160750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47880/0 (socket says 172.21.15.47:47880) 2024-03-23T09:51:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.793+0000 7fd776ffd640 1 -- 172.21.15.47:0/2094052474 learned_addr learned my addr 172.21.15.47:0/2094052474 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.793+0000 7fd7777fe640 1 --2- 172.21.15.47:0/2094052474 >> v2:172.21.15.47:3300/0 conn(0x7fd7780d1420 0x7fd7780d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.793+0000 7fd7777fe640 1 -- 172.21.15.47:0/2094052474 >> v2:172.21.15.175:3300/0 conn(0x7fd7780cd6b0 msgr2=0x7fd7780cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.793+0000 7fd7777fe640 1 --2- 172.21.15.47:0/2094052474 >> v2:172.21.15.175:3300/0 conn(0x7fd7780cd6b0 0x7fd7780cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.793+0000 7fd7777fe640 1 -- 172.21.15.47:0/2094052474 >> v1:172.21.15.47:6789/0 conn(0x7fd77815cc40 legacy=0x7fd778160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.793+0000 7fd7777fe640 1 -- 172.21.15.47:0/2094052474 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77815d020 con 0x7fd7780d1420 2024-03-23T09:51:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.793+0000 7fd7767fc640 1 -- 172.21.15.47:0/2094052474 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104213987 0 0) 0x7fd7780d09d0 con 0x7fd77815cc40 2024-03-23T09:51:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.794+0000 7fd7777fe640 1 --2- 172.21.15.47:0/2094052474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7780d1420 0x7fd7780d1830 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fd7780cd2f0 tx=0x7fd77002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8061393cdc66d037 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.794+0000 7fd7767fc640 1 -- 172.21.15.47:0/2094052474 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77002cc80 con 0x7fd7780d1420 2024-03-23T09:51:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.794+0000 7fd7767fc640 1 -- 172.21.15.47:0/2094052474 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77002cde0 con 0x7fd7780d1420 2024-03-23T09:51:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.794+0000 7fd7767fc640 1 -- 172.21.15.47:0/2094052474 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd770034660 con 0x7fd7780d1420 2024-03-23T09:51:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.795+0000 7fd77d92e640 1 -- 172.21.15.47:0/2094052474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7780d1420 msgr2=0x7fd7780d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.795+0000 7fd77d92e640 1 --2- 172.21.15.47:0/2094052474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7780d1420 0x7fd7780d1830 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fd7780cd2f0 tx=0x7fd77002bed0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.795+0000 7fd77d92e640 1 -- 172.21.15.47:0/2094052474 shutdown_connections 2024-03-23T09:51:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.795+0000 7fd77d92e640 1 --2- 172.21.15.47:0/2094052474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7780d1420 0x7fd7780d1830 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.795+0000 7fd77d92e640 1 --2- 172.21.15.47:0/2094052474 >> v2:172.21.15.175:3300/0 conn(0x7fd7780cd6b0 0x7fd7780cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.795+0000 7fd77d92e640 1 -- 172.21.15.47:0/2094052474 >> 172.21.15.47:0/2094052474 conn(0x7fd77806b160 msgr2=0x7fd77806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.795+0000 7fd77d92e640 1 -- 172.21.15.47:0/2094052474 shutdown_connections 2024-03-23T09:51:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.795+0000 7fd77d92e640 1 -- 172.21.15.47:0/2094052474 wait complete. 2024-03-23T09:51:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.796+0000 7fd77d92e640 1 Processor -- start 2024-03-23T09:51:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.796+0000 7fd77d92e640 1 -- start start 2024-03-23T09:51:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd77d92e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7780cd6b0 0x7fd77815e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd77d92e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7780d1420 0x7fd7780c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd77d92e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd77815cc40 0x7fd77814fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd77d92e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780c9ef0 con 0x7fd7780cd6b0 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd77d92e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780cacf0 con 0x7fd7780d1420 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd77d92e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7781f22f0 con 0x7fd77815cc40 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd776ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7780cd6b0 0x7fd77815e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd776ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7780cd6b0 0x7fd77815e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40930/0 (socket says 172.21.15.47:40930) 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd776ffd640 1 -- 172.21.15.47:0/2240384809 learned_addr learned my addr 172.21.15.47:0/2240384809 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd76ffff640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7780d1420 0x7fd7780c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd7777fe640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd77815cc40 0x7fd77814fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd776ffd640 1 -- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd77815cc40 msgr2=0x7fd77814fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd776ffd640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd77815cc40 0x7fd77814fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd776ffd640 1 -- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7780d1420 msgr2=0x7fd7780c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd776ffd640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7780d1420 0x7fd7780c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd776ffd640 1 -- 172.21.15.47:0/2240384809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd770009a70 con 0x7fd7780cd6b0 2024-03-23T09:51:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.797+0000 7fd776ffd640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7780cd6b0 0x7fd77815e190 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fd76800c430 tx=0x7fd76800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.798+0000 7fd774ff9640 1 -- 172.21.15.47:0/2240384809 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7680074e0 con 0x7fd7780cd6b0 2024-03-23T09:51:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.798+0000 7fd774ff9640 1 -- 172.21.15.47:0/2240384809 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd768007660 con 0x7fd7780cd6b0 2024-03-23T09:51:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.798+0000 7fd77d92e640 1 -- 172.21.15.47:0/2240384809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7780ca0e0 con 0x7fd7780cd6b0 2024-03-23T09:51:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.798+0000 7fd774ff9640 1 -- 172.21.15.47:0/2240384809 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd768004a20 con 0x7fd7780cd6b0 2024-03-23T09:51:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.798+0000 7fd77d92e640 1 -- 172.21.15.47:0/2240384809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7780ca6a0 con 0x7fd7780cd6b0 2024-03-23T09:51:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.799+0000 7fd77d92e640 1 -- 172.21.15.47:0/2240384809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd778072360 con 0x7fd7780cd6b0 2024-03-23T09:51:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.799+0000 7fd774ff9640 1 -- 172.21.15.47:0/2240384809 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd768004b80 con 0x7fd7780cd6b0 2024-03-23T09:51:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.799+0000 7fd774ff9640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd75806ac00 0x7fd75806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.800+0000 7fd774ff9640 1 -- 172.21.15.47:0/2240384809 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7680890d0 con 0x7fd7780cd6b0 2024-03-23T09:51:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.800+0000 7fd76ffff640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd75806ac00 0x7fd75806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.800+0000 7fd76ffff640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd75806ac00 0x7fd75806d0c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd7780b5420 tx=0x7fd760009110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:45.804+0000 7fd774ff9640 1 -- 172.21.15.47:0/2240384809 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd768054210 con 0x7fd7780cd6b0 2024-03-23T09:51:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.019+0000 7fd77d92e640 1 -- 172.21.15.47:0/2240384809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd7780d0be0 con 0x7fd7780cd6b0 2024-03-23T09:51:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.019+0000 7fd774ff9640 1 -- 172.21.15.47:0/2240384809 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd768057830 con 0x7fd7780cd6b0 2024-03-23T09:51:46.020 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:51:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.021+0000 7fd77d92e640 1 -- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd75806ac00 msgr2=0x7fd75806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.021+0000 7fd77d92e640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd75806ac00 0x7fd75806d0c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd7780b5420 tx=0x7fd760009110 comp rx=0 tx=0).stop 2024-03-23T09:51:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.021+0000 7fd77d92e640 1 -- 172.21.15.47:0/2240384809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7780cd6b0 msgr2=0x7fd77815e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.021+0000 7fd77d92e640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7780cd6b0 0x7fd77815e190 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fd76800c430 tx=0x7fd76800c830 comp rx=0 tx=0).stop 2024-03-23T09:51:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.021+0000 7fd77d92e640 1 -- 172.21.15.47:0/2240384809 shutdown_connections 2024-03-23T09:51:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.021+0000 7fd77d92e640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd75806ac00 0x7fd75806d0c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.021+0000 7fd77d92e640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd77815cc40 0x7fd77814fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.021+0000 7fd77d92e640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7780d1420 0x7fd7780c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.021+0000 7fd77d92e640 1 --2- 172.21.15.47:0/2240384809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7780cd6b0 0x7fd77815e190 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.021+0000 7fd77d92e640 1 -- 172.21.15.47:0/2240384809 >> 172.21.15.47:0/2240384809 conn(0x7fd77806b160 msgr2=0x7fd7780b75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.021+0000 7fd77d92e640 1 -- 172.21.15.47:0/2240384809 shutdown_connections 2024-03-23T09:51:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:46.021+0000 7fd77d92e640 1 -- 172.21.15.47:0/2240384809 wait complete. 2024-03-23T09:51:46.032 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:51:46.032 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:51:46.032 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:51:46.033 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:51:46.033 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:46.033 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:51:46.033 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:51:46.033 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:51:46.033 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:51:46.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:46.033 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:51:46.048 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:51:46.048 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:51:46.048 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:51:46.048 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:46.048 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42032sec preferred_lft 42032sec 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:51:46.049 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:46.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:46.051 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:51:46.114 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:51:46.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:46.116 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:51:46.171 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:51:46.171 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:51:46.171 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:51:46.171 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:51:46.171 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:51:46.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:46.172 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:51:46.172 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:51:46.172 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:51:46.172 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:51:46.172 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:51:46.172 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:51:46.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:46.266 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:51:46.280 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:51:46.280 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:51:46.280 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:51:46.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:46.281 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:51:46.347 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:51:46.347 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:51:46.348 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:46.402 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:51:46.403 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:51:46.403 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:46.457 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:51:46.524 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:51:46.524 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:51:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:51:46.590 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:51:46.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:46.591 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:51:46.645 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:51:46.645 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:51:46.688 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:51:46.714 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:51:46.714 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:51:46.773 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:51:46.773+0000 7f3634239480 -1 init, newargv = 0x55fde78d9fc0 newargc=15 2024-03-23T09:51:46.773 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[79015]: starting ceph client 2024-03-23T09:51:46.777 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[79015]: starting fuse 2024-03-23T09:51:46.782 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:51:46.784 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:51:46.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:46.784 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:51:46.838 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:51:46.839 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:51:46.839 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:51:46.839 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:51:46.839 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:51:46.839 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:51:46.839 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:51:46.839 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:46.839 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:51:46.839 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:51:46.839 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:51:46.839 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:51:46.839 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:46.840 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:51:46.841 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:51:46.955 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.79015.asok 2024-03-23T09:51:47.141 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.79015.asok 2024-03-23T09:51:47.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:47.141 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.79015.asok status 2024-03-23T09:51:47.277 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:51:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:51:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:51:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:51:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:51:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:51:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:51:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "79015", 2024-03-23T09:51:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:51:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:51:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:51:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 9399, 2024-03-23T09:51:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:51:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:51:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:51:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 9399 2024-03-23T09:51:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:51:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:51:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:51:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2976013034 2024-03-23T09:51:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2976013034 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.9399 192.168.0.1:0/2976013034", 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/2976013034", 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 80, 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 114, 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:51:47.280 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:51:47.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:47.285 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:51:47.285 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:47.300 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:51:47.300 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:47.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:47.301 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:47.368 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:51:47.368 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:51:47.368 DEBUG:teuthology.orchestra.run.smithi047:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:51:47.368 DEBUG:teuthology.orchestra.run.smithi047:> parent = os.path.join(mount_point, "parent") 2024-03-23T09:51:47.369 DEBUG:teuthology.orchestra.run.smithi047:> os.mkdir(parent) 2024-03-23T09:51:47.369 DEBUG:teuthology.orchestra.run.smithi047:> flushed = os.path.join(parent, "flushed") 2024-03-23T09:51:47.369 DEBUG:teuthology.orchestra.run.smithi047:> os.mkdir(flushed) 2024-03-23T09:51:47.369 DEBUG:teuthology.orchestra.run.smithi047:> for f in ["alpha", "bravo", "charlie"]: 2024-03-23T09:51:47.369 DEBUG:teuthology.orchestra.run.smithi047:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-03-23T09:51:47.369 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:51:47.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:47.659 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-23T09:51:47.723 INFO:teuthology.orchestra.run.smithi047.stdout:./ 2024-03-23T09:51:47.723 INFO:teuthology.orchestra.run.smithi047.stdout:./parent 2024-03-23T09:51:47.723 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/flushed 2024-03-23T09:51:47.723 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/flushed/charlie 2024-03-23T09:51:47.723 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/flushed/alpha 2024-03-23T09:51:47.723 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/flushed/bravo 2024-03-23T09:51:47.723 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:51:47.724 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:51:47.724 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:51:47.724 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:47.724 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-23T09:51:47.724 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:51:47.824 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:51:48.014 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:51:48.014 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:51:48.014 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:51:48.014 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:48.014 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-23T09:51:48.014 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:51:48.115 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627776 2024-03-23T09:51:48.305 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:51:48.305 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:51:48.305 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:51:48.305 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:48.305 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-23T09:51:48.305 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:51:48.407 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627777 2024-03-23T09:51:48.596 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:51:48.596 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:51:48.596 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:51:48.597 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:48.597 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-23T09:51:48.597 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:51:48.699 INFO:teuthology.orchestra.run.smithi047.stdout:1099511628280 2024-03-23T09:51:48.888 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:51:48.888 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:51:48.888 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:51:48.888 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:48.889 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-23T09:51:48.889 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:51:48.990 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627778 2024-03-23T09:51:49.178 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:51:49.178 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:51:49.179 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:51:49.179 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:49.179 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-03-23T09:51:49.179 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:51:49.280 INFO:teuthology.orchestra.run.smithi047.stdout:1099511628279 2024-03-23T09:51:49.469 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:51:49.470 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:51:49.525 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:51:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.666+0000 7f9eca36d640 1 Processor -- start 2024-03-23T09:51:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.666+0000 7f9eca36d640 1 -- start start 2024-03-23T09:51:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.667+0000 7f9eca36d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ec415ca90 0x7f9ec4160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.667+0000 7f9eca36d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9ec40b8350 0x7f9ec40cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.667+0000 7f9eca36d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec40d11e0 con 0x7f9ec40cdc40 2024-03-23T09:51:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.667+0000 7f9eca36d640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec40ce0d0 con 0x7f9ec40b8350 2024-03-23T09:51:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.667+0000 7f9eca36d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40d2680 con 0x7f9ec415ca90 2024-03-23T09:51:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.667+0000 7f9ec890a640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9ec40cdc40 0x7f9ec40b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47892/0 (socket says 172.21.15.47:47892) 2024-03-23T09:51:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.667+0000 7f9ec890a640 1 -- 172.21.15.47:0/1735108623 learned_addr learned my addr 172.21.15.47:0/1735108623 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.667+0000 7f9ec3fff640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ec415ca90 0x7f9ec4160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.667+0000 7f9ec37fe640 1 --2- 172.21.15.47:0/1735108623 >> v2:172.21.15.175:3300/0 conn(0x7f9ec40b8350 0x7f9ec40cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.667+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684874241 0 0) 0x7f9ec40d11e0 con 0x7f9ec40cdc40 2024-03-23T09:51:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.667+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb0003520 con 0x7f9ec40cdc40 2024-03-23T09:51:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2045731979 0 0) 0x7f9eb0003520 con 0x7f9ec40cdc40 2024-03-23T09:51:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ec415ca90 msgr2=0x7f9ec4160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ec415ca90 0x7f9ec4160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 >> v2:172.21.15.175:3300/0 conn(0x7f9ec40b8350 msgr2=0x7f9ec40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 --2- 172.21.15.47:0/1735108623 >> v2:172.21.15.175:3300/0 conn(0x7f9ec40b8350 0x7f9ec40cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec4160f60 con 0x7f9ec40cdc40 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec3fff640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ec415ca90 0x7f9ec4160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f9ebc0019f0 con 0x7f9ec40cdc40 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 >> v1:172.21.15.47:6789/0 conn(0x7f9ec40cdc40 legacy=0x7f9ec40b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eb0003970 0x7f9eb0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ec415ca90 0x7f9eb00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb0004d00 0x7f9eb0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40ce0d0 con 0x7f9eb0003970 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40d2680 con 0x7f9eb0004d00 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb000b980 con 0x7f9ec415ca90 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec37fe640 1 --2- 172.21.15.47:0/1735108623 >> v2:172.21.15.175:3300/0 conn(0x7f9ec40b8350 0x7f9ec40cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:51:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec3fff640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eb0003970 0x7f9eb0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec890a640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb0004d00 0x7f9eb0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec37fe640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ec415ca90 0x7f9eb00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec3fff640 1 -- 172.21.15.47:0/1735108623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ec415ca90 msgr2=0x7f9eb00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec3fff640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ec415ca90 0x7f9eb00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec3fff640 1 -- 172.21.15.47:0/1735108623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb0004d00 msgr2=0x7f9eb0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec3fff640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb0004d00 0x7f9eb0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec3fff640 1 -- 172.21.15.47:0/1735108623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac009400 con 0x7f9eb0003970 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.668+0000 7f9ec37fe640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ec415ca90 0x7f9eb00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.669+0000 7f9ec3fff640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eb0003970 0x7f9eb0003d90 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009c90 tx=0x7f9eb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24008fe6b1cd761 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.669+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac02c8f0 con 0x7f9eb0003970 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.669+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac02ca50 con 0x7f9eb0003970 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.669+0000 7f9ec2ffd640 1 -- 172.21.15.47:0/1735108623 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac02ce00 con 0x7f9eb0003970 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.669+0000 7f9eca36d640 1 -- 172.21.15.47:0/1735108623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eb0003970 msgr2=0x7f9eb0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.669+0000 7f9eca36d640 1 --2- 172.21.15.47:0/1735108623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eb0003970 0x7f9eb0003d90 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009c90 tx=0x7f9eb00057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.669+0000 7f9ec890a640 1 -- 172.21.15.47:0/1735108623 reap_dead start 2024-03-23T09:51:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.670+0000 7f9eca36d640 1 -- 172.21.15.47:0/1735108623 shutdown_connections 2024-03-23T09:51:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.670+0000 7f9eca36d640 1 -- 172.21.15.47:0/1735108623 >> 172.21.15.47:0/1735108623 conn(0x7f9ec40659d0 msgr2=0x7f9ec4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.670+0000 7f9eca36d640 1 -- 172.21.15.47:0/1735108623 shutdown_connections 2024-03-23T09:51:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.670+0000 7f9eca36d640 1 -- 172.21.15.47:0/1735108623 wait complete. 2024-03-23T09:51:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.671+0000 7f9eca36d640 1 Processor -- start 2024-03-23T09:51:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.671+0000 7f9eca36d640 1 -- start start 2024-03-23T09:51:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9eca36d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb0003970 0x7f9ec415a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9eca36d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ec4153800 0x7f9ec40b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9eca36d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ec4153d50 0x7f9ec40caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9eca36d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40cc3a0 con 0x7f9ec4153d50 2024-03-23T09:51:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9eca36d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec415b000 con 0x7f9ec4153800 2024-03-23T09:51:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9eca36d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40cc890 con 0x7f9eb0003970 2024-03-23T09:51:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec890a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ec4153d50 0x7f9ec40caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec890a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ec4153d50 0x7f9ec40caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40954/0 (socket says 172.21.15.47:40954) 2024-03-23T09:51:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec890a640 1 -- 172.21.15.47:0/3991054289 learned_addr learned my addr 172.21.15.47:0/3991054289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec37fe640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ec4153800 0x7f9ec40b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec3fff640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb0003970 0x7f9ec415a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec890a640 1 -- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb0003970 msgr2=0x7f9ec415a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec890a640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb0003970 0x7f9ec415a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec890a640 1 -- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ec4153800 msgr2=0x7f9ec40b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec890a640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ec4153800 0x7f9ec40b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec890a640 1 -- 172.21.15.47:0/3991054289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec4159780 con 0x7f9ec4153d50 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec890a640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ec4153d50 0x7f9ec40caa60 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc000f80 tx=0x7f9ebc007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec0ff9640 1 -- 172.21.15.47:0/3991054289 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc017070 con 0x7f9ec4153d50 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec0ff9640 1 -- 172.21.15.47:0/3991054289 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc00eb00 con 0x7f9ec4153d50 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9eca36d640 1 -- 172.21.15.47:0/3991054289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec40cc5b0 con 0x7f9ec4153d50 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.672+0000 7f9ec0ff9640 1 -- 172.21.15.47:0/3991054289 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc00ecb0 con 0x7f9ec4153d50 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.673+0000 7f9eca36d640 1 -- 172.21.15.47:0/3991054289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec415a880 con 0x7f9ec4153d50 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.673+0000 7f9eca36d640 1 -- 172.21.15.47:0/3991054289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec415aa70 con 0x7f9ec4153d50 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.674+0000 7f9ec0ff9640 1 -- 172.21.15.47:0/3991054289 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9ebc013420 con 0x7f9ec4153d50 2024-03-23T09:51:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.674+0000 7f9ec0ff9640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9ea406ac30 0x7f9ea406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.674+0000 7f9ec0ff9640 1 -- 172.21.15.47:0/3991054289 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ebc088c00 con 0x7f9ec4153d50 2024-03-23T09:51:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.674+0000 7f9ec3fff640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9ea406ac30 0x7f9ea406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.675+0000 7f9ec3fff640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9ea406ac30 0x7f9ea406d0f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9eac036000 tx=0x7f9eac030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.679+0000 7f9ec0ff9640 1 -- 172.21.15.47:0/3991054289 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9ebc08f080 con 0x7f9ec4153d50 2024-03-23T09:51:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.858+0000 7f9eca36d640 1 -- 172.21.15.47:0/3991054289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ec4072360 con 0x7f9ec4153d50 2024-03-23T09:51:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.860+0000 7f9ec0ff9640 1 -- 172.21.15.47:0/3991054289 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16059 (secure 0 0 0) 0x7f9ebc053f20 con 0x7f9ec4153d50 2024-03-23T09:51:49.860 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:49.860 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":114,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:51:41.397088+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:51:38.679890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-23T09:51:39.396299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:51:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.861+0000 7f9eca36d640 1 -- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9ea406ac30 msgr2=0x7f9ea406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.861+0000 7f9eca36d640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9ea406ac30 0x7f9ea406d0f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9eac036000 tx=0x7f9eac030000 comp rx=0 tx=0).stop 2024-03-23T09:51:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.861+0000 7f9eca36d640 1 -- 172.21.15.47:0/3991054289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ec4153d50 msgr2=0x7f9ec40caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.861+0000 7f9eca36d640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ec4153d50 0x7f9ec40caa60 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc000f80 tx=0x7f9ebc007bc0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.861+0000 7f9eca36d640 1 -- 172.21.15.47:0/3991054289 shutdown_connections 2024-03-23T09:51:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.861+0000 7f9eca36d640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f9ea406ac30 0x7f9ea406d0f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.861+0000 7f9eca36d640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ec4153d50 0x7f9ec40caa60 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.861+0000 7f9eca36d640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ec4153800 0x7f9ec40b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.861+0000 7f9eca36d640 1 --2- 172.21.15.47:0/3991054289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb0003970 0x7f9ec415a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.861+0000 7f9eca36d640 1 -- 172.21.15.47:0/3991054289 >> 172.21.15.47:0/3991054289 conn(0x7f9ec40659d0 msgr2=0x7f9ec415e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.862+0000 7f9eca36d640 1 -- 172.21.15.47:0/3991054289 shutdown_connections 2024-03-23T09:51:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.862+0000 7f9eca36d640 1 -- 172.21.15.47:0/3991054289 wait complete. 2024-03-23T09:51:49.873 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:51:38.679890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:51:39.396299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:51:49.873 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:51:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.976+0000 7f6b9782c640 1 Processor -- start 2024-03-23T09:51:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.976+0000 7f6b9782c640 1 -- start start 2024-03-23T09:51:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b9782c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b900cde10 0x7f6b90160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b9782c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6b900b8350 0x7f6b900d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b9782c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b90161630 con 0x7f6b9015c990 2024-03-23T09:51:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b9782c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b900d1030 con 0x7f6b900cde10 2024-03-23T09:51:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b9782c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b900cfe10 con 0x7f6b900b8350 2024-03-23T09:51:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b94dc7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b900cde10 0x7f6b90160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b955c8640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6b9015c990 0x7f6b900cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47894/0 (socket says 172.21.15.47:47894) 2024-03-23T09:51:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b955c8640 1 -- 172.21.15.47:0/1517647560 learned_addr learned my addr 172.21.15.47:0/1517647560 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b95dc9640 1 --2- 172.21.15.47:0/1517647560 >> v2:172.21.15.175:3300/0 conn(0x7f6b900b8350 0x7f6b900d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b94dc7640 1 -- 172.21.15.47:0/1517647560 >> v2:172.21.15.175:3300/0 conn(0x7f6b900b8350 msgr2=0x7f6b900d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b94dc7640 1 --2- 172.21.15.47:0/1517647560 >> v2:172.21.15.175:3300/0 conn(0x7f6b900b8350 0x7f6b900d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b94dc7640 1 -- 172.21.15.47:0/1517647560 >> v1:172.21.15.47:6789/0 conn(0x7f6b9015c990 legacy=0x7f6b900cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b94dc7640 1 -- 172.21.15.47:0/1517647560 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b900b5b60 con 0x7f6b900cde10 2024-03-23T09:51:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.977+0000 7f6b87fff640 1 -- 172.21.15.47:0/1517647560 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852672073 0 0) 0x7f6b90161630 con 0x7f6b9015c990 2024-03-23T09:51:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.978+0000 7f6b94dc7640 1 --2- 172.21.15.47:0/1517647560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b900cde10 0x7f6b90160690 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f6b900d0fa0 tx=0x7f6b8002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e35ac3b52a6ee66 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.978+0000 7f6b87fff640 1 -- 172.21.15.47:0/1517647560 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8002e070 con 0x7f6b900cde10 2024-03-23T09:51:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.978+0000 7f6b87fff640 1 -- 172.21.15.47:0/1517647560 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b800315a0 con 0x7f6b900cde10 2024-03-23T09:51:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.978+0000 7f6b87fff640 1 -- 172.21.15.47:0/1517647560 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80031930 con 0x7f6b900cde10 2024-03-23T09:51:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.978+0000 7f6b9782c640 1 -- 172.21.15.47:0/1517647560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b900cde10 msgr2=0x7f6b90160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.978+0000 7f6b9782c640 1 --2- 172.21.15.47:0/1517647560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b900cde10 0x7f6b90160690 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f6b900d0fa0 tx=0x7f6b8002b840 comp rx=0 tx=0).stop 2024-03-23T09:51:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.979+0000 7f6b9782c640 1 -- 172.21.15.47:0/1517647560 shutdown_connections 2024-03-23T09:51:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.979+0000 7f6b9782c640 1 --2- 172.21.15.47:0/1517647560 >> v2:172.21.15.175:3300/0 conn(0x7f6b900b8350 0x7f6b900d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.979+0000 7f6b9782c640 1 --2- 172.21.15.47:0/1517647560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b900cde10 0x7f6b90160690 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.979+0000 7f6b9782c640 1 -- 172.21.15.47:0/1517647560 >> 172.21.15.47:0/1517647560 conn(0x7f6b900659d0 msgr2=0x7f6b90065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.979+0000 7f6b9782c640 1 -- 172.21.15.47:0/1517647560 shutdown_connections 2024-03-23T09:51:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.979+0000 7f6b9782c640 1 -- 172.21.15.47:0/1517647560 wait complete. 2024-03-23T09:51:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.980+0000 7f6b9782c640 1 Processor -- start 2024-03-23T09:51:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.980+0000 7f6b9782c640 1 -- start start 2024-03-23T09:51:49.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.980+0000 7f6b9782c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b900b8350 0x7f6b900b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.980+0000 7f6b9782c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b900cde10 0x7f6b901537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.980+0000 7f6b9782c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b9015c990 0x7f6b90155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.980+0000 7f6b9782c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9015a8f0 con 0x7f6b900b8350 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.980+0000 7f6b9782c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900cd090 con 0x7f6b9015c990 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.980+0000 7f6b9782c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90159600 con 0x7f6b900cde10 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b955c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b900b8350 0x7f6b900b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b955c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b900b8350 0x7f6b900b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40976/0 (socket says 172.21.15.47:40976) 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b955c8640 1 -- 172.21.15.47:0/516786533 learned_addr learned my addr 172.21.15.47:0/516786533 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b94dc7640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b900cde10 0x7f6b901537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b95dc9640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b9015c990 0x7f6b90155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b955c8640 1 -- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b900cde10 msgr2=0x7f6b901537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b955c8640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b900cde10 0x7f6b901537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b955c8640 1 -- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b9015c990 msgr2=0x7f6b90155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b955c8640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b9015c990 0x7f6b90155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b955c8640 1 -- 172.21.15.47:0/516786533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b80009400 con 0x7f6b900b8350 2024-03-23T09:51:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b955c8640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b900b8350 0x7f6b900b49d0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00c500 tx=0x7f6b8c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b85ffb640 1 -- 172.21.15.47:0/516786533 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c007560 con 0x7f6b900b8350 2024-03-23T09:51:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b85ffb640 1 -- 172.21.15.47:0/516786533 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c0047e0 con 0x7f6b900b8350 2024-03-23T09:51:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b9782c640 1 -- 172.21.15.47:0/516786533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9015f130 con 0x7f6b900b8350 2024-03-23T09:51:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.981+0000 7f6b85ffb640 1 -- 172.21.15.47:0/516786533 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c004b90 con 0x7f6b900b8350 2024-03-23T09:51:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.982+0000 7f6b9782c640 1 -- 172.21.15.47:0/516786533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b90159b30 con 0x7f6b900b8350 2024-03-23T09:51:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.982+0000 7f6b9782c640 1 -- 172.21.15.47:0/516786533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b90072360 con 0x7f6b900b8350 2024-03-23T09:51:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.983+0000 7f6b85ffb640 1 -- 172.21.15.47:0/516786533 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6b8c004cf0 con 0x7f6b900b8350 2024-03-23T09:51:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.983+0000 7f6b85ffb640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b7406ac00 0x7f6b7406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.983+0000 7f6b85ffb640 1 -- 172.21.15.47:0/516786533 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b8c088690 con 0x7f6b900b8350 2024-03-23T09:51:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.983+0000 7f6b94dc7640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b7406ac00 0x7f6b7406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.984+0000 7f6b94dc7640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b7406ac00 0x7f6b7406d0c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6b80036000 tx=0x7f6b8003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:49.988+0000 7f6b85ffb640 1 -- 172.21.15.47:0/516786533 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6b8c0539b0 con 0x7f6b900b8350 2024-03-23T09:51:50.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.168+0000 7f6b9782c640 1 -- 172.21.15.47:0/516786533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b90066be0 con 0x7f6b900b8350 2024-03-23T09:51:50.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.168+0000 7f6b85ffb640 1 -- 172.21.15.47:0/516786533 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b8c018020 con 0x7f6b900b8350 2024-03-23T09:51:50.169 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:51:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.170+0000 7f6b9782c640 1 -- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b7406ac00 msgr2=0x7f6b7406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.170+0000 7f6b9782c640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b7406ac00 0x7f6b7406d0c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6b80036000 tx=0x7f6b8003d040 comp rx=0 tx=0).stop 2024-03-23T09:51:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.170+0000 7f6b9782c640 1 -- 172.21.15.47:0/516786533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b900b8350 msgr2=0x7f6b900b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.170+0000 7f6b9782c640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b900b8350 0x7f6b900b49d0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00c500 tx=0x7f6b8c00c9c0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.170+0000 7f6b9782c640 1 -- 172.21.15.47:0/516786533 shutdown_connections 2024-03-23T09:51:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.170+0000 7f6b9782c640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6b7406ac00 0x7f6b7406d0c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.170+0000 7f6b9782c640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b9015c990 0x7f6b90155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.170+0000 7f6b9782c640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b900cde10 0x7f6b901537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.170+0000 7f6b9782c640 1 --2- 172.21.15.47:0/516786533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b900b8350 0x7f6b900b49d0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.170+0000 7f6b9782c640 1 -- 172.21.15.47:0/516786533 >> 172.21.15.47:0/516786533 conn(0x7f6b900659d0 msgr2=0x7f6b9015bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.170+0000 7f6b9782c640 1 -- 172.21.15.47:0/516786533 shutdown_connections 2024-03-23T09:51:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.170+0000 7f6b9782c640 1 -- 172.21.15.47:0/516786533 wait complete. 2024-03-23T09:51:50.181 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:51:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.285+0000 7f7d6137d640 1 Processor -- start 2024-03-23T09:51:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.285+0000 7f7d6137d640 1 -- start start 2024-03-23T09:51:50.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d6137d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7d5c15c990 0x7f7d5c160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d6137d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d5c0b8350 0x7f7d5c0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d6137d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5c0cdc40 0x7f7d5c0b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d6137d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5c0ce0b0 con 0x7f7d5c0b8350 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d6137d640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5c161630 con 0x7f7d5c15c990 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d6137d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c0d1030 con 0x7f7d5c0cdc40 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d53fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d5c0b8350 0x7f7d5c0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d53fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d5c0b8350 0x7f7d5c0cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40988/0 (socket says 172.21.15.47:40988) 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d53fff640 1 -- 172.21.15.47:0/1635584590 learned_addr learned my addr 172.21.15.47:0/1635584590 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d53fff640 1 -- 172.21.15.47:0/1635584590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5c0cdc40 msgr2=0x7f7d5c0b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d53fff640 1 --2- 172.21.15.47:0/1635584590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5c0cdc40 0x7f7d5c0b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d53fff640 1 -- 172.21.15.47:0/1635584590 >> v2:172.21.15.175:3300/0 conn(0x7f7d5c15c990 msgr2=0x7f7d5c160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d5affd640 1 --2- 172.21.15.47:0/1635584590 >> v2:172.21.15.175:3300/0 conn(0x7f7d5c15c990 0x7f7d5c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d53fff640 1 --2- 172.21.15.47:0/1635584590 >> v2:172.21.15.175:3300/0 conn(0x7f7d5c15c990 0x7f7d5c160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.286+0000 7f7d53fff640 1 -- 172.21.15.47:0/1635584590 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c0cfe10 con 0x7f7d5c0b8350 2024-03-23T09:51:50.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.287+0000 7f7d53fff640 1 --2- 172.21.15.47:0/1635584590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5c0b8350 0x7f7d5c0cd700 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0b8b30 tx=0x7f7d4402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3475ab3560dde11 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.287+0000 7f7d5a7fc640 1 -- 172.21.15.47:0/1635584590 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d440317e0 con 0x7f7d5c0b8350 2024-03-23T09:51:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.287+0000 7f7d5a7fc640 1 -- 172.21.15.47:0/1635584590 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d44031940 con 0x7f7d5c0b8350 2024-03-23T09:51:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.287+0000 7f7d5a7fc640 1 -- 172.21.15.47:0/1635584590 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44031cd0 con 0x7f7d5c0b8350 2024-03-23T09:51:50.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.288+0000 7f7d6137d640 1 -- 172.21.15.47:0/1635584590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5c0b8350 msgr2=0x7f7d5c0cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.288+0000 7f7d6137d640 1 --2- 172.21.15.47:0/1635584590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5c0b8350 0x7f7d5c0cd700 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0b8b30 tx=0x7f7d4402b880 comp rx=0 tx=0).stop 2024-03-23T09:51:50.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.288+0000 7f7d6137d640 1 -- 172.21.15.47:0/1635584590 shutdown_connections 2024-03-23T09:51:50.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.288+0000 7f7d6137d640 1 --2- 172.21.15.47:0/1635584590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5c0cdc40 0x7f7d5c0b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.288+0000 7f7d6137d640 1 --2- 172.21.15.47:0/1635584590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5c0b8350 0x7f7d5c0cd700 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.288+0000 7f7d6137d640 1 --2- 172.21.15.47:0/1635584590 >> v2:172.21.15.175:3300/0 conn(0x7f7d5c15c990 0x7f7d5c160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.288+0000 7f7d6137d640 1 -- 172.21.15.47:0/1635584590 >> 172.21.15.47:0/1635584590 conn(0x7f7d5c0659d0 msgr2=0x7f7d5c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.288+0000 7f7d6137d640 1 -- 172.21.15.47:0/1635584590 shutdown_connections 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.288+0000 7f7d6137d640 1 -- 172.21.15.47:0/1635584590 wait complete. 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.289+0000 7f7d6137d640 1 Processor -- start 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.289+0000 7f7d6137d640 1 -- start start 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d6137d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5c0b8350 0x7f7d5c0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d6137d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d5c0cdc40 0x7f7d5c157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d6137d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5c15c990 0x7f7d5c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d6137d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c0cd020 con 0x7f7d5c0b8350 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d6137d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c158740 con 0x7f7d5c0cdc40 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d6137d640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c158a20 con 0x7f7d5c15c990 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d5affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5c0b8350 0x7f7d5c0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d5affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5c0b8350 0x7f7d5c0ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40990/0 (socket says 172.21.15.47:40990) 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d5affd640 1 -- 172.21.15.47:0/1701929714 learned_addr learned my addr 172.21.15.47:0/1701929714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d5b7fe640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5c15c990 0x7f7d5c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d53fff640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d5c0cdc40 0x7f7d5c157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d5affd640 1 -- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5c15c990 msgr2=0x7f7d5c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d5affd640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5c15c990 0x7f7d5c0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d5affd640 1 -- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d5c0cdc40 msgr2=0x7f7d5c157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d5affd640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d5c0cdc40 0x7f7d5c157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d5affd640 1 -- 172.21.15.47:0/1701929714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d440094d0 con 0x7f7d5c0b8350 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d5affd640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5c0b8350 0x7f7d5c0ccae0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00c7f0 tx=0x7f7d4c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d537fe640 1 -- 172.21.15.47:0/1701929714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c004220 con 0x7f7d5c0b8350 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d537fe640 1 -- 172.21.15.47:0/1701929714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c0043a0 con 0x7f7d5c0b8350 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d6137d640 1 -- 172.21.15.47:0/1701929714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c158d00 con 0x7f7d5c0b8350 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d537fe640 1 -- 172.21.15.47:0/1701929714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c007860 con 0x7f7d5c0b8350 2024-03-23T09:51:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.290+0000 7f7d6137d640 1 -- 172.21.15.47:0/1701929714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5c1592c0 con 0x7f7d5c0b8350 2024-03-23T09:51:50.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.291+0000 7f7d6137d640 1 -- 172.21.15.47:0/1701929714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5c072360 con 0x7f7d5c0b8350 2024-03-23T09:51:50.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.292+0000 7f7d537fe640 1 -- 172.21.15.47:0/1701929714 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7d4c0079c0 con 0x7f7d5c0b8350 2024-03-23T09:51:50.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.292+0000 7f7d537fe640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7d3c06ac00 0x7f7d3c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.292+0000 7f7d537fe640 1 -- 172.21.15.47:0/1701929714 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d4c089120 con 0x7f7d5c0b8350 2024-03-23T09:51:50.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.292+0000 7f7d53fff640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7d3c06ac00 0x7f7d3c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.293+0000 7f7d53fff640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7d3c06ac00 0x7f7d3c06d0c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7d44036000 tx=0x7f7d4403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:50.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.296+0000 7f7d537fe640 1 -- 172.21.15.47:0/1701929714 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7d4c054300 con 0x7f7d5c0b8350 2024-03-23T09:51:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.474+0000 7f7d6137d640 1 -- 172.21.15.47:0/1701929714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d5c066be0 con 0x7f7d5c0b8350 2024-03-23T09:51:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.475+0000 7f7d537fe640 1 -- 172.21.15.47:0/1701929714 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d4c057920 con 0x7f7d5c0b8350 2024-03-23T09:51:50.475 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:51:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.476+0000 7f7d6137d640 1 -- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7d3c06ac00 msgr2=0x7f7d3c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.476+0000 7f7d6137d640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7d3c06ac00 0x7f7d3c06d0c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7d44036000 tx=0x7f7d4403d040 comp rx=0 tx=0).stop 2024-03-23T09:51:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.476+0000 7f7d6137d640 1 -- 172.21.15.47:0/1701929714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5c0b8350 msgr2=0x7f7d5c0ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.476+0000 7f7d6137d640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5c0b8350 0x7f7d5c0ccae0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00c7f0 tx=0x7f7d4c00ccb0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.476+0000 7f7d6137d640 1 -- 172.21.15.47:0/1701929714 shutdown_connections 2024-03-23T09:51:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.476+0000 7f7d6137d640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7d3c06ac00 0x7f7d3c06d0c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.476+0000 7f7d6137d640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5c15c990 0x7f7d5c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.476+0000 7f7d6137d640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d5c0cdc40 0x7f7d5c157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.476+0000 7f7d6137d640 1 --2- 172.21.15.47:0/1701929714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5c0b8350 0x7f7d5c0ccae0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.476+0000 7f7d6137d640 1 -- 172.21.15.47:0/1701929714 >> 172.21.15.47:0/1701929714 conn(0x7f7d5c0659d0 msgr2=0x7f7d5c0d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.477+0000 7f7d6137d640 1 -- 172.21.15.47:0/1701929714 shutdown_connections 2024-03-23T09:51:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.477+0000 7f7d6137d640 1 -- 172.21.15.47:0/1701929714 wait complete. 2024-03-23T09:51:50.488 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:51:50.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.591+0000 7fc0c1a66640 1 Processor -- start 2024-03-23T09:51:50.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.592+0000 7fc0c1a66640 1 -- start start 2024-03-23T09:51:50.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.592+0000 7fc0c1a66640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc0bc0cde10 0x7fc0bc160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.592+0000 7fc0c1a66640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0bc0b8350 0x7fc0bc0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.592+0000 7fc0c1a66640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0bc161630 con 0x7fc0bc15c990 2024-03-23T09:51:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.592+0000 7fc0c1a66640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0bc0d1030 con 0x7fc0bc0b8350 2024-03-23T09:51:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.592+0000 7fc0c1a66640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0bc0cfe10 con 0x7fc0bc0cde10 2024-03-23T09:51:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.592+0000 7fc0bbfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0bc0b8350 0x7fc0bc0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.592+0000 7fc0bbfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0bc0b8350 0x7fc0bc0d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40994/0 (socket says 172.21.15.47:40994) 2024-03-23T09:51:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.592+0000 7fc0bb7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc0bc15c990 0x7fc0bc0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47896/0 (socket says 172.21.15.47:47896) 2024-03-23T09:51:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.592+0000 7fc0bbfff640 1 -- 172.21.15.47:0/2715014569 learned_addr learned my addr 172.21.15.47:0/2715014569 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.593+0000 7fc0baffd640 1 --2- 172.21.15.47:0/2715014569 >> v2:172.21.15.175:3300/0 conn(0x7fc0bc0cde10 0x7fc0bc160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.593+0000 7fc0bbfff640 1 -- 172.21.15.47:0/2715014569 >> v2:172.21.15.175:3300/0 conn(0x7fc0bc0cde10 msgr2=0x7fc0bc160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.593+0000 7fc0bbfff640 1 --2- 172.21.15.47:0/2715014569 >> v2:172.21.15.175:3300/0 conn(0x7fc0bc0cde10 0x7fc0bc160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.593+0000 7fc0bbfff640 1 -- 172.21.15.47:0/2715014569 >> v1:172.21.15.47:6789/0 conn(0x7fc0bc15c990 legacy=0x7fc0bc0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.593+0000 7fc0ba7fc640 1 -- 172.21.15.47:0/2715014569 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601145056 0 0) 0x7fc0bc161630 con 0x7fc0bc15c990 2024-03-23T09:51:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.593+0000 7fc0bbfff640 1 -- 172.21.15.47:0/2715014569 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc0b5b60 con 0x7fc0bc0b8350 2024-03-23T09:51:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.594+0000 7fc0bbfff640 1 --2- 172.21.15.47:0/2715014569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0bc0b8350 0x7fc0bc0d14c0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc08ac70 tx=0x7fc0b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6ad4b1bfbc48d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.594+0000 7fc0ba7fc640 1 -- 172.21.15.47:0/2715014569 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b402ca80 con 0x7fc0bc0b8350 2024-03-23T09:51:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.594+0000 7fc0ba7fc640 1 -- 172.21.15.47:0/2715014569 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b402cbe0 con 0x7fc0bc0b8350 2024-03-23T09:51:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.594+0000 7fc0ba7fc640 1 -- 172.21.15.47:0/2715014569 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b40343f0 con 0x7fc0bc0b8350 2024-03-23T09:51:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.594+0000 7fc0c1a66640 1 -- 172.21.15.47:0/2715014569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0bc0b8350 msgr2=0x7fc0bc0d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.594+0000 7fc0c1a66640 1 --2- 172.21.15.47:0/2715014569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0bc0b8350 0x7fc0bc0d14c0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc08ac70 tx=0x7fc0b402bee0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.594+0000 7fc0c1a66640 1 -- 172.21.15.47:0/2715014569 shutdown_connections 2024-03-23T09:51:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.594+0000 7fc0c1a66640 1 --2- 172.21.15.47:0/2715014569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0bc0b8350 0x7fc0bc0d14c0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.594+0000 7fc0c1a66640 1 --2- 172.21.15.47:0/2715014569 >> v2:172.21.15.175:3300/0 conn(0x7fc0bc0cde10 0x7fc0bc160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.594+0000 7fc0c1a66640 1 -- 172.21.15.47:0/2715014569 >> 172.21.15.47:0/2715014569 conn(0x7fc0bc0659d0 msgr2=0x7fc0bc065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:50.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.594+0000 7fc0c1a66640 1 -- 172.21.15.47:0/2715014569 shutdown_connections 2024-03-23T09:51:50.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.594+0000 7fc0c1a66640 1 -- 172.21.15.47:0/2715014569 wait complete. 2024-03-23T09:51:50.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.595+0000 7fc0c1a66640 1 Processor -- start 2024-03-23T09:51:50.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.596+0000 7fc0c1a66640 1 -- start start 2024-03-23T09:51:50.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.596+0000 7fc0c1a66640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0bc0b8350 0x7fc0bc158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.596+0000 7fc0c1a66640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0bc0cde10 0x7fc0bc159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.596+0000 7fc0c1a66640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0bc15c990 0x7fc0bc15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.596+0000 7fc0c1a66640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc1f22e0 con 0x7fc0bc15c990 2024-03-23T09:51:50.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.596+0000 7fc0c1a66640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc15b2d0 con 0x7fc0bc0b8350 2024-03-23T09:51:50.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.596+0000 7fc0c1a66640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc1589f0 con 0x7fc0bc0cde10 2024-03-23T09:51:50.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.596+0000 7fc0bbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0bc15c990 0x7fc0bc15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.596+0000 7fc0bbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0bc15c990 0x7fc0bc15a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41004/0 (socket says 172.21.15.47:41004) 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.596+0000 7fc0bbfff640 1 -- 172.21.15.47:0/4046325237 learned_addr learned my addr 172.21.15.47:0/4046325237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.596+0000 7fc0bb7fe640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0bc0b8350 0x7fc0bc158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.596+0000 7fc0baffd640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0bc0cde10 0x7fc0bc159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.597+0000 7fc0bbfff640 1 -- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0bc0cde10 msgr2=0x7fc0bc159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.597+0000 7fc0bbfff640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0bc0cde10 0x7fc0bc159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.597+0000 7fc0bbfff640 1 -- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0bc0b8350 msgr2=0x7fc0bc158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.597+0000 7fc0bbfff640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0bc0b8350 0x7fc0bc158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.597+0000 7fc0bbfff640 1 -- 172.21.15.47:0/4046325237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b4009b30 con 0x7fc0bc15c990 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.597+0000 7fc0bb7fe640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0bc0b8350 0x7fc0bc158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.597+0000 7fc0bbfff640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0bc15c990 0x7fc0bc15a770 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4009f40 tx=0x7fc0b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.597+0000 7fc0a3fff640 1 -- 172.21.15.47:0/4046325237 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4009510 con 0x7fc0bc15c990 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.597+0000 7fc0a3fff640 1 -- 172.21.15.47:0/4046325237 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b403e450 con 0x7fc0bc15c990 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.597+0000 7fc0c1a66640 1 -- 172.21.15.47:0/4046325237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0bc158cd0 con 0x7fc0bc15c990 2024-03-23T09:51:50.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.597+0000 7fc0a3fff640 1 -- 172.21.15.47:0/4046325237 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b403e5e0 con 0x7fc0bc15c990 2024-03-23T09:51:50.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.597+0000 7fc0c1a66640 1 -- 172.21.15.47:0/4046325237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0bc159290 con 0x7fc0bc15c990 2024-03-23T09:51:50.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.598+0000 7fc0c1a66640 1 -- 172.21.15.47:0/4046325237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0bc072360 con 0x7fc0bc15c990 2024-03-23T09:51:50.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.598+0000 7fc0a3fff640 1 -- 172.21.15.47:0/4046325237 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc0b402cd50 con 0x7fc0bc15c990 2024-03-23T09:51:50.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.599+0000 7fc0a3fff640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc0a806ac00 0x7fc0a806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.599+0000 7fc0a3fff640 1 -- 172.21.15.47:0/4046325237 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0b40aba50 con 0x7fc0bc15c990 2024-03-23T09:51:50.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.599+0000 7fc0bb7fe640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc0a806ac00 0x7fc0a806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.599+0000 7fc0bb7fe640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc0a806ac00 0x7fc0a806d0c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac000f60 tx=0x7fc0ac007c30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:50.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.603+0000 7fc0a3fff640 1 -- 172.21.15.47:0/4046325237 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc0b4076cc0 con 0x7fc0bc15c990 2024-03-23T09:51:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.780+0000 7fc0c1a66640 1 -- 172.21.15.47:0/4046325237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0bc066be0 con 0x7fc0bc15c990 2024-03-23T09:51:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.781+0000 7fc0a3fff640 1 -- 172.21.15.47:0/4046325237 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc0b407a2e0 con 0x7fc0bc15c990 2024-03-23T09:51:50.781 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:51:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.783+0000 7fc0c1a66640 1 -- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc0a806ac00 msgr2=0x7fc0a806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.783+0000 7fc0c1a66640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc0a806ac00 0x7fc0a806d0c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac000f60 tx=0x7fc0ac007c30 comp rx=0 tx=0).stop 2024-03-23T09:51:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.783+0000 7fc0c1a66640 1 -- 172.21.15.47:0/4046325237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0bc15c990 msgr2=0x7fc0bc15a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.783+0000 7fc0c1a66640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0bc15c990 0x7fc0bc15a770 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4009f40 tx=0x7fc0b4004a40 comp rx=0 tx=0).stop 2024-03-23T09:51:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.783+0000 7fc0c1a66640 1 -- 172.21.15.47:0/4046325237 shutdown_connections 2024-03-23T09:51:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.783+0000 7fc0c1a66640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fc0a806ac00 0x7fc0a806d0c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.783+0000 7fc0c1a66640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0bc15c990 0x7fc0bc15a770 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.783+0000 7fc0c1a66640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0bc0cde10 0x7fc0bc159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.783+0000 7fc0c1a66640 1 --2- 172.21.15.47:0/4046325237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0bc0b8350 0x7fc0bc158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.783+0000 7fc0c1a66640 1 -- 172.21.15.47:0/4046325237 >> 172.21.15.47:0/4046325237 conn(0x7fc0bc0659d0 msgr2=0x7fc0bc0b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.783+0000 7fc0c1a66640 1 -- 172.21.15.47:0/4046325237 shutdown_connections 2024-03-23T09:51:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.783+0000 7fc0c1a66640 1 -- 172.21.15.47:0/4046325237 wait complete. 2024-03-23T09:51:50.794 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:51:50.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.895+0000 7f3bd7d3e640 1 Processor -- start 2024-03-23T09:51:50.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.895+0000 7f3bd7d3e640 1 -- start start 2024-03-23T09:51:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd7d3e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3bd00cf690 0x7f3bd00b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd7d3e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd015d0e0 con 0x7f3bd015cd00 2024-03-23T09:51:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd7d3e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd00d1540 con 0x7f3bd00cf210 2024-03-23T09:51:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd7d3e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd00d1820 con 0x7f3bd00cf690 2024-03-23T09:51:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd5ada640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3bd015cd00 0x7f3bd00d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47910/0 (socket says 172.21.15.47:47910) 2024-03-23T09:51:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd5ada640 1 -- 172.21.15.47:0/522607341 learned_addr learned my addr 172.21.15.47:0/522607341 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762670312 0 0) 0x7f3bd015d0e0 con 0x7f3bd015cd00 2024-03-23T09:51:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd62db640 1 --2- 172.21.15.47:0/522607341 >> v2:172.21.15.175:3300/0 conn(0x7f3bd00cf690 0x7f3bd00b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bb8003520 con 0x7f3bd015cd00 2024-03-23T09:51:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1037804600 0 0) 0x7f3bb8003520 con 0x7f3bd015cd00 2024-03-23T09:51:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 >> v2:172.21.15.175:3300/0 conn(0x7f3bd00cf690 msgr2=0x7f3bd00b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd4ad8640 1 --2- 172.21.15.47:0/522607341 >> v2:172.21.15.175:3300/0 conn(0x7f3bd00cf690 0x7f3bd00b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 >> v1:172.21.15.175:6789/0 conn(0x7f3bd00cf210 legacy=0x7f3bd00b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00d1b00 con 0x7f3bd015cd00 2024-03-23T09:51:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.896+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f3bc8001670 con 0x7f3bd015cd00 2024-03-23T09:51:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 >> v1:172.21.15.47:6789/0 conn(0x7f3bd015cd00 legacy=0x7f3bd00d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd4ad8640 1 --2- 172.21.15.47:0/522607341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bb8003970 0x7f3bb8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd4ad8640 1 --2- 172.21.15.47:0/522607341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb80047c0 0x7f3bb8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd4ad8640 1 --2- 172.21.15.47:0/522607341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb8005180 0x7f3bb800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00d1820 con 0x7f3bb8003970 2024-03-23T09:51:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb800c120 con 0x7f3bb80047c0 2024-03-23T09:51:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb800c2e0 con 0x7f3bb8005180 2024-03-23T09:51:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd5ada640 1 --2- 172.21.15.47:0/522607341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bb8003970 0x7f3bb8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd52d9640 1 --2- 172.21.15.47:0/522607341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb80047c0 0x7f3bb8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd62db640 1 --2- 172.21.15.47:0/522607341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb8005180 0x7f3bb800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd5ada640 1 -- 172.21.15.47:0/522607341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb8005180 msgr2=0x7f3bb800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd5ada640 1 --2- 172.21.15.47:0/522607341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb8005180 0x7f3bb800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd5ada640 1 -- 172.21.15.47:0/522607341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb80047c0 msgr2=0x7f3bb8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd5ada640 1 --2- 172.21.15.47:0/522607341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb80047c0 0x7f3bb8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd5ada640 1 -- 172.21.15.47:0/522607341 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb8003520 con 0x7f3bb8003970 2024-03-23T09:51:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd62db640 1 -- 172.21.15.47:0/522607341 reap_dead start 2024-03-23T09:51:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.897+0000 7f3bd52d9640 1 --2- 172.21.15.47:0/522607341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb80047c0 0x7f3bb8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.898+0000 7f3bd5ada640 1 --2- 172.21.15.47:0/522607341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bb8003970 0x7f3bb8003d90 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800c240 tx=0x7f3bb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea331463d63691a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.898+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8005ad0 con 0x7f3bb8003970 2024-03-23T09:51:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.898+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc8005c30 con 0x7f3bb8003970 2024-03-23T09:51:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.898+0000 7f3bd4ad8640 1 -- 172.21.15.47:0/522607341 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc802ca90 con 0x7f3bb8003970 2024-03-23T09:51:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.898+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/522607341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bb8003970 msgr2=0x7f3bb8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.898+0000 7f3bd7d3e640 1 --2- 172.21.15.47:0/522607341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bb8003970 0x7f3bb8003d90 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800c240 tx=0x7f3bb80057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.898+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/522607341 shutdown_connections 2024-03-23T09:51:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.898+0000 7f3bd7d3e640 1 --2- 172.21.15.47:0/522607341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bb8003970 0x7f3bb8003d90 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.898+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/522607341 >> 172.21.15.47:0/522607341 conn(0x7f3bd006b160 msgr2=0x7f3bd006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.898+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/522607341 shutdown_connections 2024-03-23T09:51:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.898+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/522607341 wait complete. 2024-03-23T09:51:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.900+0000 7f3bd7d3e640 1 Processor -- start 2024-03-23T09:51:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.900+0000 7f3bd7d3e640 1 -- start start 2024-03-23T09:51:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.900+0000 7f3bd7d3e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb8003970 0x7f3bd00b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.900+0000 7f3bd7d3e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bd0142910 0x7f3bd00b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.900+0000 7f3bd7d3e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd01f5c50 0x7f3bd01f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.900+0000 7f3bd7d3e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd01f6640 con 0x7f3bd0142910 2024-03-23T09:51:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.900+0000 7f3bd7d3e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00b4d00 con 0x7f3bd01f5c50 2024-03-23T09:51:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.900+0000 7f3bd7d3e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00b4fe0 con 0x7f3bb8003970 2024-03-23T09:51:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.900+0000 7f3bd52d9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bd0142910 0x7f3bd00b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.900+0000 7f3bd52d9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bd0142910 0x7f3bd00b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41022/0 (socket says 172.21.15.47:41022) 2024-03-23T09:51:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.900+0000 7f3bd52d9640 1 -- 172.21.15.47:0/1073243594 learned_addr learned my addr 172.21.15.47:0/1073243594 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.901+0000 7f3bd5ada640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb8003970 0x7f3bd00b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.901+0000 7f3bd62db640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd01f5c50 0x7f3bd01f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.901+0000 7f3bd52d9640 1 -- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb8003970 msgr2=0x7f3bd00b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.901+0000 7f3bd52d9640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb8003970 0x7f3bd00b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.901+0000 7f3bd52d9640 1 -- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd01f5c50 msgr2=0x7f3bd01f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.901+0000 7f3bd52d9640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd01f5c50 0x7f3bd01f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.901+0000 7f3bd52d9640 1 -- 172.21.15.47:0/1073243594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00cab20 con 0x7f3bd0142910 2024-03-23T09:51:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.901+0000 7f3bd52d9640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bd0142910 0x7f3bd00b4440 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400e480 tx=0x7f3bc400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.901+0000 7f3bce7fc640 1 -- 172.21.15.47:0/1073243594 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4017070 con 0x7f3bd0142910 2024-03-23T09:51:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.901+0000 7f3bce7fc640 1 -- 172.21.15.47:0/1073243594 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc40052e0 con 0x7f3bd0142910 2024-03-23T09:51:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.901+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/1073243594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd00cae00 con 0x7f3bd0142910 2024-03-23T09:51:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.901+0000 7f3bce7fc640 1 -- 172.21.15.47:0/1073243594 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4005490 con 0x7f3bd0142910 2024-03-23T09:51:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.901+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/1073243594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd00b4980 con 0x7f3bd0142910 2024-03-23T09:51:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.902+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/1073243594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd0072360 con 0x7f3bd0142910 2024-03-23T09:51:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.903+0000 7f3bce7fc640 1 -- 172.21.15.47:0/1073243594 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3bc4007020 con 0x7f3bd0142910 2024-03-23T09:51:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.903+0000 7f3bce7fc640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3bbc06ac00 0x7f3bbc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.903+0000 7f3bce7fc640 1 -- 172.21.15.47:0/1073243594 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bc40888d0 con 0x7f3bd0142910 2024-03-23T09:51:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.903+0000 7f3bd5ada640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3bbc06ac00 0x7f3bbc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.904+0000 7f3bd5ada640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3bbc06ac00 0x7f3bbc06d0c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8034000 tx=0x7f3bc8009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:50.907+0000 7f3bce7fc640 1 -- 172.21.15.47:0/1073243594 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3bc4053b40 con 0x7f3bd0142910 2024-03-23T09:51:51.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.098+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/1073243594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3bd0066be0 con 0x7f3bd0142910 2024-03-23T09:51:51.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.098+0000 7f3bce7fc640 1 -- 172.21.15.47:0/1073243594 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v114) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3bc4057160 con 0x7f3bd0142910 2024-03-23T09:51:51.099 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:51:51.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.100+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3bbc06ac00 msgr2=0x7f3bbc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.100+0000 7f3bd7d3e640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3bbc06ac00 0x7f3bbc06d0c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8034000 tx=0x7f3bc8009030 comp rx=0 tx=0).stop 2024-03-23T09:51:51.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.100+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/1073243594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bd0142910 msgr2=0x7f3bd00b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.100+0000 7f3bd7d3e640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bd0142910 0x7f3bd00b4440 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400e480 tx=0x7f3bc400e940 comp rx=0 tx=0).stop 2024-03-23T09:51:51.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.100+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/1073243594 shutdown_connections 2024-03-23T09:51:51.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.100+0000 7f3bd7d3e640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3bbc06ac00 0x7f3bbc06d0c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.100+0000 7f3bd7d3e640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd01f5c50 0x7f3bd01f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.100+0000 7f3bd7d3e640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bd0142910 0x7f3bd00b4440 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.100+0000 7f3bd7d3e640 1 --2- 172.21.15.47:0/1073243594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb8003970 0x7f3bd00b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.100+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/1073243594 >> 172.21.15.47:0/1073243594 conn(0x7f3bd006b160 msgr2=0x7f3bd00b75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:51.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.100+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/1073243594 shutdown_connections 2024-03-23T09:51:51.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.100+0000 7f3bd7d3e640 1 -- 172.21.15.47:0/1073243594 wait complete. 2024-03-23T09:51:51.111 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:51:51.111 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:51:51.126 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:51:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.266+0000 7fb83b8ab640 1 Processor -- start 2024-03-23T09:51:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.267+0000 7fb83b8ab640 1 -- start start 2024-03-23T09:51:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.267+0000 7fb83b8ab640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb83415c9a0 0x7fb83406b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.267+0000 7fb83b8ab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb83406bd40 0x7fb83415d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.267+0000 7fb83b8ab640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83406c1a0 con 0x7fb83415d770 2024-03-23T09:51:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.267+0000 7fb83b8ab640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8340b1530 con 0x7fb83415c9a0 2024-03-23T09:51:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.267+0000 7fb83b8ab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83415df60 con 0x7fb83406bd40 2024-03-23T09:51:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.267+0000 7fb839647640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb83415c9a0 0x7fb83406b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.267+0000 7fb839647640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb83415c9a0 0x7fb83406b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41030/0 (socket says 172.21.15.47:41030) 2024-03-23T09:51:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.267+0000 7fb839647640 1 -- 172.21.15.47:0/146232055 learned_addr learned my addr 172.21.15.47:0/146232055 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.268+0000 7fb838e46640 1 --2- 172.21.15.47:0/146232055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb83406bd40 0x7fb83415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.268+0000 7fb839647640 1 -- 172.21.15.47:0/146232055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb83406bd40 msgr2=0x7fb83415d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.268+0000 7fb839647640 1 --2- 172.21.15.47:0/146232055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb83406bd40 0x7fb83415d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.268+0000 7fb839647640 1 -- 172.21.15.47:0/146232055 >> v1:172.21.15.175:6789/0 conn(0x7fb83415d770 legacy=0x7fb8340ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.268+0000 7fb839647640 1 -- 172.21.15.47:0/146232055 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8341606e0 con 0x7fb83415c9a0 2024-03-23T09:51:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.268+0000 7fb838e46640 1 --2- 172.21.15.47:0/146232055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb83406bd40 0x7fb83415d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.268+0000 7fb839647640 1 --2- 172.21.15.47:0/146232055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb83415c9a0 0x7fb83406b800 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009870 tx=0x7fb82c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6f8d7caba952d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.269+0000 7fb82bfff640 1 -- 172.21.15.47:0/146232055 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c030840 con 0x7fb83415c9a0 2024-03-23T09:51:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.269+0000 7fb82bfff640 1 -- 172.21.15.47:0/146232055 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c0309a0 con 0x7fb83415c9a0 2024-03-23T09:51:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.269+0000 7fb82bfff640 1 -- 172.21.15.47:0/146232055 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c030d30 con 0x7fb83415c9a0 2024-03-23T09:51:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.269+0000 7fb83b8ab640 1 -- 172.21.15.47:0/146232055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb83415c9a0 msgr2=0x7fb83406b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.269+0000 7fb83b8ab640 1 --2- 172.21.15.47:0/146232055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb83415c9a0 0x7fb83406b800 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009870 tx=0x7fb82c02b980 comp rx=0 tx=0).stop 2024-03-23T09:51:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.269+0000 7fb83b8ab640 1 -- 172.21.15.47:0/146232055 shutdown_connections 2024-03-23T09:51:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.269+0000 7fb83b8ab640 1 --2- 172.21.15.47:0/146232055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb83406bd40 0x7fb83415d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.269+0000 7fb83b8ab640 1 --2- 172.21.15.47:0/146232055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb83415c9a0 0x7fb83406b800 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.269+0000 7fb83b8ab640 1 -- 172.21.15.47:0/146232055 >> 172.21.15.47:0/146232055 conn(0x7fb8340d0670 msgr2=0x7fb8340d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.269+0000 7fb83b8ab640 1 -- 172.21.15.47:0/146232055 shutdown_connections 2024-03-23T09:51:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.269+0000 7fb83b8ab640 1 -- 172.21.15.47:0/146232055 wait complete. 2024-03-23T09:51:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.270+0000 7fb83b8ab640 1 Processor -- start 2024-03-23T09:51:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.270+0000 7fb83b8ab640 1 -- start start 2024-03-23T09:51:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.271+0000 7fb83b8ab640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb83406bd40 0x7fb8340c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.271+0000 7fb83b8ab640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb83415c9a0 0x7fb8340c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.271+0000 7fb83b8ab640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb83415d770 0x7fb83406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.271+0000 7fb83b8ab640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340ca1c0 con 0x7fb83406bd40 2024-03-23T09:51:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.271+0000 7fb83b8ab640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340ad300 con 0x7fb83415c9a0 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.271+0000 7fb83b8ab640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340ad5e0 con 0x7fb83415d770 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.271+0000 7fb839647640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb83406bd40 0x7fb8340c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.271+0000 7fb839647640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb83406bd40 0x7fb8340c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41044/0 (socket says 172.21.15.47:41044) 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.271+0000 7fb839647640 1 -- 172.21.15.47:0/3172828926 learned_addr learned my addr 172.21.15.47:0/3172828926 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.272+0000 7fb839647640 1 -- 172.21.15.47:0/3172828926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb83415d770 msgr2=0x7fb83406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.272+0000 7fb839647640 1 --2- 172.21.15.47:0/3172828926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb83415d770 0x7fb83406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.272+0000 7fb839647640 1 -- 172.21.15.47:0/3172828926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb83415c9a0 msgr2=0x7fb8340c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.272+0000 7fb839647640 1 --2- 172.21.15.47:0/3172828926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb83415c9a0 0x7fb8340c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.272+0000 7fb839647640 1 -- 172.21.15.47:0/3172828926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c0094d0 con 0x7fb83406bd40 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.272+0000 7fb839647640 1 --2- 172.21.15.47:0/3172828926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb83406bd40 0x7fb8340c9b20 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fb82c000c00 tx=0x7fb82c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.272+0000 7fb829ffb640 1 -- 172.21.15.47:0/3172828926 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c039c90 con 0x7fb83406bd40 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.272+0000 7fb829ffb640 1 -- 172.21.15.47:0/3172828926 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c039df0 con 0x7fb83406bd40 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.272+0000 7fb83b8ab640 1 -- 172.21.15.47:0/3172828926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8340ad8c0 con 0x7fb83406bd40 2024-03-23T09:51:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.272+0000 7fb829ffb640 1 -- 172.21.15.47:0/3172828926 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c043640 con 0x7fb83406bd40 2024-03-23T09:51:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.272+0000 7fb83b8ab640 1 -- 172.21.15.47:0/3172828926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8340c8570 con 0x7fb83406bd40 2024-03-23T09:51:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.273+0000 7fb83b8ab640 1 -- 172.21.15.47:0/3172828926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8340672f0 con 0x7fb83406bd40 2024-03-23T09:51:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.274+0000 7fb829ffb640 1 -- 172.21.15.47:0/3172828926 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb82c0437a0 con 0x7fb83406bd40 2024-03-23T09:51:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.274+0000 7fb829ffb640 1 --2- 172.21.15.47:0/3172828926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb81806ac00 0x7fb81806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.274+0000 7fb829ffb640 1 -- 172.21.15.47:0/3172828926 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb82c03e070 con 0x7fb83406bd40 2024-03-23T09:51:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.274+0000 7fb838e46640 1 --2- 172.21.15.47:0/3172828926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb81806ac00 0x7fb81806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.275+0000 7fb838e46640 1 --2- 172.21.15.47:0/3172828926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb81806ac00 0x7fb81806d0c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb824006fd0 tx=0x7fb824008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.278+0000 7fb829ffb640 1 -- 172.21.15.47:0/3172828926 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb82c077620 con 0x7fb83406bd40 2024-03-23T09:51:51.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.458+0000 7fb83b8ab640 1 -- 172.21.15.47:0/3172828926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb83415d030 con 0x7fb83406bd40 2024-03-23T09:51:51.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.459+0000 7fb829ffb640 1 -- 172.21.15.47:0/3172828926 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16059 (secure 0 0 0) 0x7fb82c07ac40 con 0x7fb83406bd40 2024-03-23T09:51:51.460 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:51.460 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":114,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:51:41.397088+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:51:38.679890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-23T09:51:39.396299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:51:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.461+0000 7fb83b8ab640 1 -- 172.21.15.47:0/3172828926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb81806ac00 msgr2=0x7fb81806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.461+0000 7fb83b8ab640 1 --2- 172.21.15.47:0/3172828926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb81806ac00 0x7fb81806d0c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb824006fd0 tx=0x7fb824008040 comp rx=0 tx=0).stop 2024-03-23T09:51:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.461+0000 7fb83b8ab640 1 -- 172.21.15.47:0/3172828926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb83406bd40 msgr2=0x7fb8340c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.461+0000 7fb83b8ab640 1 --2- 172.21.15.47:0/3172828926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb83406bd40 0x7fb8340c9b20 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fb82c000c00 tx=0x7fb82c004730 comp rx=0 tx=0).stop 2024-03-23T09:51:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.461+0000 7fb83b8ab640 1 -- 172.21.15.47:0/3172828926 shutdown_connections 2024-03-23T09:51:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.461+0000 7fb83b8ab640 1 --2- 172.21.15.47:0/3172828926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fb81806ac00 0x7fb81806d0c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.461+0000 7fb83b8ab640 1 --2- 172.21.15.47:0/3172828926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb83415d770 0x7fb83406f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.461+0000 7fb83b8ab640 1 --2- 172.21.15.47:0/3172828926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb83415c9a0 0x7fb8340c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.461+0000 7fb83b8ab640 1 --2- 172.21.15.47:0/3172828926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb83406bd40 0x7fb8340c9b20 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.461+0000 7fb83b8ab640 1 -- 172.21.15.47:0/3172828926 >> 172.21.15.47:0/3172828926 conn(0x7fb8340d0670 msgr2=0x7fb834098a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:51.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.461+0000 7fb83b8ab640 1 -- 172.21.15.47:0/3172828926 shutdown_connections 2024-03-23T09:51:51.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.461+0000 7fb83b8ab640 1 -- 172.21.15.47:0/3172828926 wait complete. 2024-03-23T09:51:51.472 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:51:38.679890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:51:39.396299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:51:51.473 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:51:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.576+0000 7f761384b640 1 Processor -- start 2024-03-23T09:51:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.576+0000 7f761384b640 1 -- start start 2024-03-23T09:51:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.577+0000 7f761384b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f760c0cde10 0x7f760c160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.577+0000 7f761384b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c0b8350 0x7f760c0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.577+0000 7f761384b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f760c161630 con 0x7f760c15c990 2024-03-23T09:51:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.577+0000 7f761384b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f760c0d1030 con 0x7f760c0cde10 2024-03-23T09:51:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.577+0000 7f761384b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760c0cfe10 con 0x7f760c0b8350 2024-03-23T09:51:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.577+0000 7f76115e7640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f760c15c990 0x7f760c0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47922/0 (socket says 172.21.15.47:47922) 2024-03-23T09:51:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.577+0000 7f76115e7640 1 -- 172.21.15.47:0/1477521191 learned_addr learned my addr 172.21.15.47:0/1477521191 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755108863 0 0) 0x7f760c161630 con 0x7f760c15c990 2024-03-23T09:51:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75f4003520 con 0x7f760c15c990 2024-03-23T09:51:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f7610de6640 1 --2- 172.21.15.47:0/1477521191 >> v2:172.21.15.175:3300/0 conn(0x7f760c0cde10 0x7f760c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f7611de8640 1 --2- 172.21.15.47:0/1477521191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c0b8350 0x7f760c0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3718097142 0 0) 0x7f75f4003520 con 0x7f760c15c990 2024-03-23T09:51:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c0b8350 msgr2=0x7f760c0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 --2- 172.21.15.47:0/1477521191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c0b8350 0x7f760c0d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 >> v2:172.21.15.175:3300/0 conn(0x7f760c0cde10 msgr2=0x7f760c160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 --2- 172.21.15.47:0/1477521191 >> v2:172.21.15.175:3300/0 conn(0x7f760c0cde10 0x7f760c160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c0b5b60 con 0x7f760c15c990 2024-03-23T09:51:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f76040019f0 con 0x7f760c15c990 2024-03-23T09:51:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 >> v1:172.21.15.47:6789/0 conn(0x7f760c15c990 legacy=0x7f760c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 --2- 172.21.15.47:0/1477521191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f4003970 0x7f75f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 --2- 172.21.15.47:0/1477521191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75f40047c0 0x7f75f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 --2- 172.21.15.47:0/1477521191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c0b8350 0x7f75f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c0d1030 con 0x7f75f4003970 2024-03-23T09:51:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c0cfe10 con 0x7f75f40047c0 2024-03-23T09:51:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f400ba30 con 0x7f760c0b8350 2024-03-23T09:51:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f76115e7640 1 --2- 172.21.15.47:0/1477521191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f4003970 0x7f75f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f7610de6640 1 --2- 172.21.15.47:0/1477521191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75f40047c0 0x7f75f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f7611de8640 1 --2- 172.21.15.47:0/1477521191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c0b8350 0x7f75f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f76115e7640 1 -- 172.21.15.47:0/1477521191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c0b8350 msgr2=0x7f75f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f76115e7640 1 --2- 172.21.15.47:0/1477521191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c0b8350 0x7f75f4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f76115e7640 1 -- 172.21.15.47:0/1477521191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75f40047c0 msgr2=0x7f75f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f76115e7640 1 --2- 172.21.15.47:0/1477521191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75f40047c0 0x7f75f4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.578+0000 7f76115e7640 1 -- 172.21.15.47:0/1477521191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f4003520 con 0x7f75f4003970 2024-03-23T09:51:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.579+0000 7f7611de8640 1 --2- 172.21.15.47:0/1477521191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c0b8350 0x7f75f4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.579+0000 7f76115e7640 1 --2- 172.21.15.47:0/1477521191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f4003970 0x7f75f4003d90 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f760400a810 tx=0x7f75f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3323a13703cc76a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.579+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604034630 con 0x7f75f4003970 2024-03-23T09:51:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.579+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760402f070 con 0x7f75f4003970 2024-03-23T09:51:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.579+0000 7f760bfff640 1 -- 172.21.15.47:0/1477521191 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76040349d0 con 0x7f75f4003970 2024-03-23T09:51:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.580+0000 7f761384b640 1 -- 172.21.15.47:0/1477521191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f4003970 msgr2=0x7f75f4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.580+0000 7f761384b640 1 --2- 172.21.15.47:0/1477521191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f4003970 0x7f75f4003d90 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f760400a810 tx=0x7f75f40057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.580+0000 7f7611de8640 1 -- 172.21.15.47:0/1477521191 reap_dead start 2024-03-23T09:51:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.580+0000 7f761384b640 1 -- 172.21.15.47:0/1477521191 shutdown_connections 2024-03-23T09:51:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.580+0000 7f761384b640 1 -- 172.21.15.47:0/1477521191 >> 172.21.15.47:0/1477521191 conn(0x7f760c0659d0 msgr2=0x7f760c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.580+0000 7f761384b640 1 -- 172.21.15.47:0/1477521191 shutdown_connections 2024-03-23T09:51:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.580+0000 7f761384b640 1 -- 172.21.15.47:0/1477521191 wait complete. 2024-03-23T09:51:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.581+0000 7f761384b640 1 Processor -- start 2024-03-23T09:51:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.581+0000 7f761384b640 1 -- start start 2024-03-23T09:51:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f761384b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f4003970 0x7f760c15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f761384b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760c15e770 0x7f760c150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f761384b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c150a90 0x7f760c153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f761384b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c1529b0 con 0x7f75f4003970 2024-03-23T09:51:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f761384b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c1601d0 con 0x7f760c15e770 2024-03-23T09:51:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f761384b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760c1541f0 con 0x7f760c150a90 2024-03-23T09:51:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f76115e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f4003970 0x7f760c15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f76115e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f4003970 0x7f760c15fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41058/0 (socket says 172.21.15.47:41058) 2024-03-23T09:51:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f76115e7640 1 -- 172.21.15.47:0/2237489618 learned_addr learned my addr 172.21.15.47:0/2237489618 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f7611de8640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c150a90 0x7f760c153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f7610de6640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760c15e770 0x7f760c150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f76115e7640 1 -- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c150a90 msgr2=0x7f760c153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f76115e7640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c150a90 0x7f760c153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f76115e7640 1 -- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760c15e770 msgr2=0x7f760c150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f76115e7640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760c15e770 0x7f760c150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.582+0000 7f76115e7640 1 -- 172.21.15.47:0/2237489618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c15dc90 con 0x7f75f4003970 2024-03-23T09:51:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.583+0000 7f76115e7640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f4003970 0x7f760c15fc90 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f760400a810 tx=0x7f760c0d1170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.583+0000 7f7609ffb640 1 -- 172.21.15.47:0/2237489618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76040356d0 con 0x7f75f4003970 2024-03-23T09:51:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.583+0000 7f7609ffb640 1 -- 172.21.15.47:0/2237489618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7604035830 con 0x7f75f4003970 2024-03-23T09:51:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.583+0000 7f761384b640 1 -- 172.21.15.47:0/2237489618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760c15df70 con 0x7f75f4003970 2024-03-23T09:51:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.583+0000 7f7609ffb640 1 -- 172.21.15.47:0/2237489618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604035ba0 con 0x7f75f4003970 2024-03-23T09:51:51.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.583+0000 7f761384b640 1 -- 172.21.15.47:0/2237489618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f760c1536b0 con 0x7f75f4003970 2024-03-23T09:51:51.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.584+0000 7f761384b640 1 -- 172.21.15.47:0/2237489618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760c072360 con 0x7f75f4003970 2024-03-23T09:51:51.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.584+0000 7f7609ffb640 1 -- 172.21.15.47:0/2237489618 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7604031030 con 0x7f75f4003970 2024-03-23T09:51:51.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.585+0000 7f7609ffb640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75f806ac30 0x7f75f806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.585+0000 7f7609ffb640 1 -- 172.21.15.47:0/2237489618 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76040abb00 con 0x7f75f4003970 2024-03-23T09:51:51.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.585+0000 7f7610de6640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75f806ac30 0x7f75f806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.585+0000 7f7610de6640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75f806ac30 0x7f75f806d0f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f760c0c9950 tx=0x7f7600008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:51.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.589+0000 7f7609ffb640 1 -- 172.21.15.47:0/2237489618 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7604076c60 con 0x7f75f4003970 2024-03-23T09:51:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.767+0000 7f761384b640 1 -- 172.21.15.47:0/2237489618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f760c15daf0 con 0x7f75f4003970 2024-03-23T09:51:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.767+0000 7f7609ffb640 1 -- 172.21.15.47:0/2237489618 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f760c15daf0 con 0x7f75f4003970 2024-03-23T09:51:51.768 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:51:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.769+0000 7f761384b640 1 -- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75f806ac30 msgr2=0x7f75f806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.769+0000 7f761384b640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75f806ac30 0x7f75f806d0f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f760c0c9950 tx=0x7f7600008040 comp rx=0 tx=0).stop 2024-03-23T09:51:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.769+0000 7f761384b640 1 -- 172.21.15.47:0/2237489618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f4003970 msgr2=0x7f760c15fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.769+0000 7f761384b640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f4003970 0x7f760c15fc90 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f760400a810 tx=0x7f760c0d1170 comp rx=0 tx=0).stop 2024-03-23T09:51:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.769+0000 7f761384b640 1 -- 172.21.15.47:0/2237489618 shutdown_connections 2024-03-23T09:51:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.769+0000 7f761384b640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75f806ac30 0x7f75f806d0f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.769+0000 7f761384b640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c150a90 0x7f760c153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.769+0000 7f761384b640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760c15e770 0x7f760c150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.769+0000 7f761384b640 1 --2- 172.21.15.47:0/2237489618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f4003970 0x7f760c15fc90 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.769+0000 7f761384b640 1 -- 172.21.15.47:0/2237489618 >> 172.21.15.47:0/2237489618 conn(0x7f760c0659d0 msgr2=0x7f760c0b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.769+0000 7f761384b640 1 -- 172.21.15.47:0/2237489618 shutdown_connections 2024-03-23T09:51:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.769+0000 7f761384b640 1 -- 172.21.15.47:0/2237489618 wait complete. 2024-03-23T09:51:51.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:51:51.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.884+0000 7fa348cf3640 1 Processor -- start 2024-03-23T09:51:51.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.884+0000 7fa348cf3640 1 -- start start 2024-03-23T09:51:51.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa348cf3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3440b8350 0x7fa344160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa348cf3640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa344161630 con 0x7fa3440cde10 2024-03-23T09:51:51.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa348cf3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3440d1030 con 0x7fa34415c990 2024-03-23T09:51:51.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa348cf3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3440cfe10 con 0x7fa3440b8350 2024-03-23T09:51:51.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa33bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa3440cde10 0x7fa3440d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47934/0 (socket says 172.21.15.47:47934) 2024-03-23T09:51:51.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa33bfff640 1 -- 172.21.15.47:0/297543347 learned_addr learned my addr 172.21.15.47:0/297543347 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa342d9d640 1 --2- 172.21.15.47:0/297543347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3440b8350 0x7fa344160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa341d9b640 1 -- 172.21.15.47:0/297543347 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016270975 0 0) 0x7fa344161630 con 0x7fa3440cde10 2024-03-23T09:51:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa341d9b640 1 -- 172.21.15.47:0/297543347 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa330003520 con 0x7fa3440cde10 2024-03-23T09:51:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa341d9b640 1 -- 172.21.15.47:0/297543347 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860251936 0 0) 0x7fa3440d1030 con 0x7fa34415c990 2024-03-23T09:51:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa341d9b640 1 -- 172.21.15.47:0/297543347 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa344161630 con 0x7fa34415c990 2024-03-23T09:51:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa341d9b640 1 -- 172.21.15.47:0/297543347 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4094812062 0 0) 0x7fa330003520 con 0x7fa3440cde10 2024-03-23T09:51:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa342d9d640 1 -- 172.21.15.47:0/297543347 >> v1:172.21.15.175:6789/0 conn(0x7fa34415c990 legacy=0x7fa3440cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa342d9d640 1 -- 172.21.15.47:0/297543347 >> v1:172.21.15.47:6789/0 conn(0x7fa3440cde10 legacy=0x7fa3440d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.885+0000 7fa342d9d640 1 -- 172.21.15.47:0/297543347 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3440b5b60 con 0x7fa3440b8350 2024-03-23T09:51:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.886+0000 7fa342d9d640 1 --2- 172.21.15.47:0/297543347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3440b8350 0x7fa344160690 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009dc0 tx=0x7fa33c02bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb0658c02f43aab6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.886+0000 7fa341d9b640 1 -- 172.21.15.47:0/297543347 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c001670 con 0x7fa3440b8350 2024-03-23T09:51:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.886+0000 7fa341d9b640 1 -- 172.21.15.47:0/297543347 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33c02c880 con 0x7fa3440b8350 2024-03-23T09:51:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.886+0000 7fa341d9b640 1 -- 172.21.15.47:0/297543347 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c02cbf0 con 0x7fa3440b8350 2024-03-23T09:51:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.887+0000 7fa348cf3640 1 -- 172.21.15.47:0/297543347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3440b8350 msgr2=0x7fa344160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.887+0000 7fa348cf3640 1 --2- 172.21.15.47:0/297543347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3440b8350 0x7fa344160690 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009dc0 tx=0x7fa33c02bd50 comp rx=0 tx=0).stop 2024-03-23T09:51:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.887+0000 7fa348cf3640 1 -- 172.21.15.47:0/297543347 shutdown_connections 2024-03-23T09:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.887+0000 7fa348cf3640 1 --2- 172.21.15.47:0/297543347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3440b8350 0x7fa344160690 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.887+0000 7fa348cf3640 1 -- 172.21.15.47:0/297543347 >> 172.21.15.47:0/297543347 conn(0x7fa3440659d0 msgr2=0x7fa344065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.887+0000 7fa348cf3640 1 -- 172.21.15.47:0/297543347 shutdown_connections 2024-03-23T09:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.887+0000 7fa348cf3640 1 -- 172.21.15.47:0/297543347 wait complete. 2024-03-23T09:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.888+0000 7fa348cf3640 1 Processor -- start 2024-03-23T09:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.888+0000 7fa348cf3640 1 -- start start 2024-03-23T09:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa348cf3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3440b8350 0x7fa344153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa348cf3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3440cde10 0x7fa3441549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa348cf3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa34415c990 0x7fa344154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa348cf3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa344154280 con 0x7fa3440b8350 2024-03-23T09:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa348cf3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3440cb2d0 con 0x7fa3440cde10 2024-03-23T09:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa348cf3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa344155550 con 0x7fa34415c990 2024-03-23T09:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa34259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3440b8350 0x7fa344153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa34259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3440b8350 0x7fa344153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41074/0 (socket says 172.21.15.47:41074) 2024-03-23T09:51:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa34259c640 1 -- 172.21.15.47:0/1763767526 learned_addr learned my addr 172.21.15.47:0/1763767526 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa33bfff640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3440cde10 0x7fa3441549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa342d9d640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa34415c990 0x7fa344154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa34259c640 1 -- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa34415c990 msgr2=0x7fa344154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa34259c640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa34415c990 0x7fa344154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa34259c640 1 -- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3440cde10 msgr2=0x7fa3441549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa34259c640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3440cde10 0x7fa3441549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.889+0000 7fa34259c640 1 -- 172.21.15.47:0/1763767526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c009a70 con 0x7fa3440b8350 2024-03-23T09:51:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.890+0000 7fa34259c640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3440b8350 0x7fa344153d40 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fa33400c430 tx=0x7fa33400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.890+0000 7fa33affd640 1 -- 172.21.15.47:0/1763767526 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3340052d0 con 0x7fa3440b8350 2024-03-23T09:51:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.890+0000 7fa33affd640 1 -- 172.21.15.47:0/1763767526 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa334005450 con 0x7fa3440b8350 2024-03-23T09:51:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.890+0000 7fa348cf3640 1 -- 172.21.15.47:0/1763767526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa344155830 con 0x7fa3440b8350 2024-03-23T09:51:51.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.890+0000 7fa33affd640 1 -- 172.21.15.47:0/1763767526 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa334003830 con 0x7fa3440b8350 2024-03-23T09:51:51.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.890+0000 7fa348cf3640 1 -- 172.21.15.47:0/1763767526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa344155df0 con 0x7fa3440b8350 2024-03-23T09:51:51.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.891+0000 7fa348cf3640 1 -- 172.21.15.47:0/1763767526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa344072360 con 0x7fa3440b8350 2024-03-23T09:51:51.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.891+0000 7fa33affd640 1 -- 172.21.15.47:0/1763767526 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa334003990 con 0x7fa3440b8350 2024-03-23T09:51:51.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.892+0000 7fa33affd640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa32406ac00 0x7fa32406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:51.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.892+0000 7fa33affd640 1 -- 172.21.15.47:0/1763767526 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3340888c0 con 0x7fa3440b8350 2024-03-23T09:51:51.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.892+0000 7fa33bfff640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa32406ac00 0x7fa32406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:51.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.892+0000 7fa33bfff640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa32406ac00 0x7fa32406d0c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa34415b870 tx=0x7fa32c001690 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:51.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:51.896+0000 7fa33affd640 1 -- 172.21.15.47:0/1763767526 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa334053b50 con 0x7fa3440b8350 2024-03-23T09:51:52.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.074+0000 7fa348cf3640 1 -- 172.21.15.47:0/1763767526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa344066be0 con 0x7fa3440b8350 2024-03-23T09:51:52.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.074+0000 7fa33affd640 1 -- 172.21.15.47:0/1763767526 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa334017020 con 0x7fa3440b8350 2024-03-23T09:51:52.075 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:51:52.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.076+0000 7fa348cf3640 1 -- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa32406ac00 msgr2=0x7fa32406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.076+0000 7fa348cf3640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa32406ac00 0x7fa32406d0c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa34415b870 tx=0x7fa32c001690 comp rx=0 tx=0).stop 2024-03-23T09:51:52.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.076+0000 7fa348cf3640 1 -- 172.21.15.47:0/1763767526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3440b8350 msgr2=0x7fa344153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.076+0000 7fa348cf3640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3440b8350 0x7fa344153d40 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fa33400c430 tx=0x7fa33400c8f0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.076+0000 7fa348cf3640 1 -- 172.21.15.47:0/1763767526 shutdown_connections 2024-03-23T09:51:52.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.077+0000 7fa348cf3640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa32406ac00 0x7fa32406d0c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.077+0000 7fa348cf3640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa34415c990 0x7fa344154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.077+0000 7fa348cf3640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3440cde10 0x7fa3441549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.077+0000 7fa348cf3640 1 --2- 172.21.15.47:0/1763767526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3440b8350 0x7fa344153d40 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.077+0000 7fa348cf3640 1 -- 172.21.15.47:0/1763767526 >> 172.21.15.47:0/1763767526 conn(0x7fa3440659d0 msgr2=0x7fa3440cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:52.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.077+0000 7fa348cf3640 1 -- 172.21.15.47:0/1763767526 shutdown_connections 2024-03-23T09:51:52.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.077+0000 7fa348cf3640 1 -- 172.21.15.47:0/1763767526 wait complete. 2024-03-23T09:51:52.087 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:51:52.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.191+0000 7f6454669640 1 Processor -- start 2024-03-23T09:51:52.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.191+0000 7f6454669640 1 -- start start 2024-03-23T09:51:52.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.192+0000 7f6454669640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f644c0b81f0 0x7f644c0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.192+0000 7f6454669640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f644c0b1b20 0x7f644c0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.192+0000 7f6454669640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644c160d60 con 0x7f644c15ff60 2024-03-23T09:51:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.192+0000 7f6454669640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f644c160340 con 0x7f644c0b1b20 2024-03-23T09:51:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.192+0000 7f6454669640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f644c0b8680 con 0x7f644c0b81f0 2024-03-23T09:51:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.192+0000 7f6452c06640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f644c0b1b20 0x7f644c0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.192+0000 7f6452c06640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f644c0b1b20 0x7f644c0d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41086/0 (socket says 172.21.15.47:41086) 2024-03-23T09:51:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.192+0000 7f6452c06640 1 -- 172.21.15.47:0/3911780600 learned_addr learned my addr 172.21.15.47:0/3911780600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.193+0000 7f6451c04640 1 --2- 172.21.15.47:0/3911780600 >> v2:172.21.15.175:3300/0 conn(0x7f644c0b81f0 0x7f644c0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.193+0000 7f6452c06640 1 -- 172.21.15.47:0/3911780600 >> v2:172.21.15.175:3300/0 conn(0x7f644c0b81f0 msgr2=0x7f644c0d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.193+0000 7f6452c06640 1 --2- 172.21.15.47:0/3911780600 >> v2:172.21.15.175:3300/0 conn(0x7f644c0b81f0 0x7f644c0d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.193+0000 7f6452c06640 1 -- 172.21.15.47:0/3911780600 >> v1:172.21.15.175:6789/0 conn(0x7f644c15ff60 legacy=0x7f644c0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.193+0000 7f6452c06640 1 -- 172.21.15.47:0/3911780600 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c160710 con 0x7f644c0b1b20 2024-03-23T09:51:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.193+0000 7f6452c06640 1 --2- 172.21.15.47:0/3911780600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c0b1b20 0x7f644c0d0f20 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f644c0cefc0 tx=0x7f644402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943f1269a0a9b29a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:52.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.194+0000 7f6451403640 1 -- 172.21.15.47:0/3911780600 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64440317e0 con 0x7f644c0b1b20 2024-03-23T09:51:52.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.194+0000 7f6451403640 1 -- 172.21.15.47:0/3911780600 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6444031940 con 0x7f644c0b1b20 2024-03-23T09:51:52.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.194+0000 7f6451403640 1 -- 172.21.15.47:0/3911780600 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444031cd0 con 0x7f644c0b1b20 2024-03-23T09:51:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.194+0000 7f6454669640 1 -- 172.21.15.47:0/3911780600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c0b1b20 msgr2=0x7f644c0d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.194+0000 7f6454669640 1 --2- 172.21.15.47:0/3911780600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c0b1b20 0x7f644c0d0f20 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f644c0cefc0 tx=0x7f644402b880 comp rx=0 tx=0).stop 2024-03-23T09:51:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.194+0000 7f6454669640 1 -- 172.21.15.47:0/3911780600 shutdown_connections 2024-03-23T09:51:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.194+0000 7f6454669640 1 --2- 172.21.15.47:0/3911780600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c0b1b20 0x7f644c0d0f20 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.194+0000 7f6454669640 1 --2- 172.21.15.47:0/3911780600 >> v2:172.21.15.175:3300/0 conn(0x7f644c0b81f0 0x7f644c0d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.194+0000 7f6454669640 1 -- 172.21.15.47:0/3911780600 >> 172.21.15.47:0/3911780600 conn(0x7f644c065df0 msgr2=0x7f644c066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.194+0000 7f6454669640 1 -- 172.21.15.47:0/3911780600 shutdown_connections 2024-03-23T09:51:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.194+0000 7f6454669640 1 -- 172.21.15.47:0/3911780600 wait complete. 2024-03-23T09:51:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.195+0000 7f6454669640 1 Processor -- start 2024-03-23T09:51:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.195+0000 7f6454669640 1 -- start start 2024-03-23T09:51:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6454669640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f644c0b1b20 0x7f644c0cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6454669640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644c15ff60 0x7f644c15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6454669640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c0cbfd0 0x7f644c0b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6454669640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0a9250 con 0x7f644c0cbfd0 2024-03-23T09:51:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6454669640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0a9510 con 0x7f644c15ff60 2024-03-23T09:51:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6454669640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0a97f0 con 0x7f644c0b1b20 2024-03-23T09:51:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6452c06640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c0cbfd0 0x7f644c0b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6452c06640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c0cbfd0 0x7f644c0b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41100/0 (socket says 172.21.15.47:41100) 2024-03-23T09:51:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6452c06640 1 -- 172.21.15.47:0/3284780765 learned_addr learned my addr 172.21.15.47:0/3284780765 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6452405640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f644c0b1b20 0x7f644c0cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6452c06640 1 -- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f644c0b1b20 msgr2=0x7f644c0cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6452c06640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f644c0b1b20 0x7f644c0cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6452c06640 1 -- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644c15ff60 msgr2=0x7f644c15a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.196+0000 7f6451c04640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644c15ff60 0x7f644c15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.197+0000 7f6452c06640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644c15ff60 0x7f644c15a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.197+0000 7f6452c06640 1 -- 172.21.15.47:0/3284780765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64440094d0 con 0x7f644c0cbfd0 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.197+0000 7f6452c06640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c0cbfd0 0x7f644c0b34b0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f64440090b0 tx=0x7f64440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.197+0000 7f644affd640 1 -- 172.21.15.47:0/3284780765 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444032940 con 0x7f644c0cbfd0 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.197+0000 7f644affd640 1 -- 172.21.15.47:0/3284780765 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6444032ac0 con 0x7f644c0cbfd0 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.197+0000 7f644affd640 1 -- 172.21.15.47:0/3284780765 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444032e70 con 0x7f644c0cbfd0 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.197+0000 7f6454669640 1 -- 172.21.15.47:0/3284780765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644c0a9ad0 con 0x7f644c0cbfd0 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.197+0000 7f6454669640 1 -- 172.21.15.47:0/3284780765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644c0aa090 con 0x7f644c0cbfd0 2024-03-23T09:51:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.198+0000 7f6454669640 1 -- 172.21.15.47:0/3284780765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644c0672f0 con 0x7f644c0cbfd0 2024-03-23T09:51:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.198+0000 7f644affd640 1 -- 172.21.15.47:0/3284780765 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f644404d3f0 con 0x7f644c0cbfd0 2024-03-23T09:51:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.199+0000 7f644affd640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f643806ac50 0x7f643806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.199+0000 7f644affd640 1 -- 172.21.15.47:0/3284780765 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f64440ab220 con 0x7f644c0cbfd0 2024-03-23T09:51:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.199+0000 7f6452405640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f643806ac50 0x7f643806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.200+0000 7f6452405640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f643806ac50 0x7f643806d110 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f643c007dc0 tx=0x7f643c0072f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:52.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.203+0000 7f644affd640 1 -- 172.21.15.47:0/3284780765 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f64440764a0 con 0x7f644c0cbfd0 2024-03-23T09:51:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.383+0000 7f6454669640 1 -- 172.21.15.47:0/3284780765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f644c0b7600 con 0x7f644c0cbfd0 2024-03-23T09:51:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.383+0000 7f644affd640 1 -- 172.21.15.47:0/3284780765 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6444079ac0 con 0x7f644c0cbfd0 2024-03-23T09:51:52.384 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:51:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.385+0000 7f6454669640 1 -- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f643806ac50 msgr2=0x7f643806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.385+0000 7f6454669640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f643806ac50 0x7f643806d110 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f643c007dc0 tx=0x7f643c0072f0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.385+0000 7f6454669640 1 -- 172.21.15.47:0/3284780765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c0cbfd0 msgr2=0x7f644c0b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.385+0000 7f6454669640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c0cbfd0 0x7f644c0b34b0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f64440090b0 tx=0x7f64440046c0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.385+0000 7f6454669640 1 -- 172.21.15.47:0/3284780765 shutdown_connections 2024-03-23T09:51:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.385+0000 7f6454669640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f643806ac50 0x7f643806d110 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.385+0000 7f6454669640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c0cbfd0 0x7f644c0b34b0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.385+0000 7f6454669640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644c15ff60 0x7f644c15a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.385+0000 7f6454669640 1 --2- 172.21.15.47:0/3284780765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f644c0b1b20 0x7f644c0cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.385+0000 7f6454669640 1 -- 172.21.15.47:0/3284780765 >> 172.21.15.47:0/3284780765 conn(0x7f644c065df0 msgr2=0x7f644c0d1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.385+0000 7f6454669640 1 -- 172.21.15.47:0/3284780765 shutdown_connections 2024-03-23T09:51:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.385+0000 7f6454669640 1 -- 172.21.15.47:0/3284780765 wait complete. 2024-03-23T09:51:52.396 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:51:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.497+0000 7fd5c7f2e640 1 Processor -- start 2024-03-23T09:51:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.497+0000 7fd5c7f2e640 1 -- start start 2024-03-23T09:51:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.498+0000 7fd5c7f2e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c00ce8e0 0x7fd5c0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.498+0000 7fd5c7f2e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5c00cb6c0 0x7fd5c00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.498+0000 7fd5c7f2e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c00cbb20 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.498+0000 7fd5c7f2e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c00b1ff0 con 0x7fd5c00cb6c0 2024-03-23T09:51:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.498+0000 7fd5c7f2e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00cf200 con 0x7fd5c00ce8e0 2024-03-23T09:51:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.498+0000 7fd5c54c9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5c00cb6c0 0x7fd5c00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.498+0000 7fd5c54c9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5c00cb6c0 0x7fd5c00cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41106/0 (socket says 172.21.15.47:41106) 2024-03-23T09:51:52.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.498+0000 7fd5c54c9640 1 -- 172.21.15.47:0/521356527 learned_addr learned my addr 172.21.15.47:0/521356527 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:52.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.499+0000 7fd5c5cca640 1 --2- 172.21.15.47:0/521356527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c00ce8e0 0x7fd5c0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.499+0000 7fd5c54c9640 1 -- 172.21.15.47:0/521356527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c00ce8e0 msgr2=0x7fd5c0069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:52.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.499+0000 7fd5c54c9640 1 --2- 172.21.15.47:0/521356527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c00ce8e0 0x7fd5c0069ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.499+0000 7fd5c54c9640 1 -- 172.21.15.47:0/521356527 >> v1:172.21.15.175:6789/0 conn(0x7fd5c00c9ba0 legacy=0x7fd5c00cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.499+0000 7fd5c54c9640 1 -- 172.21.15.47:0/521356527 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c006a2c0 con 0x7fd5c00cb6c0 2024-03-23T09:51:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.500+0000 7fd5c54c9640 1 --2- 172.21.15.47:0/521356527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c00cb6c0 0x7fd5c00cecc0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009870 tx=0x7fd5b402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f28a3f8cda83297 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.500+0000 7fd5c4cc8640 1 -- 172.21.15.47:0/521356527 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4030840 con 0x7fd5c00cb6c0 2024-03-23T09:51:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.500+0000 7fd5c4cc8640 1 -- 172.21.15.47:0/521356527 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b40309a0 con 0x7fd5c00cb6c0 2024-03-23T09:51:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.500+0000 7fd5c4cc8640 1 -- 172.21.15.47:0/521356527 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4030d30 con 0x7fd5c00cb6c0 2024-03-23T09:51:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.500+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/521356527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c00cb6c0 msgr2=0x7fd5c00cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.500+0000 7fd5c7f2e640 1 --2- 172.21.15.47:0/521356527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c00cb6c0 0x7fd5c00cecc0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009870 tx=0x7fd5b402b980 comp rx=0 tx=0).stop 2024-03-23T09:51:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.500+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/521356527 shutdown_connections 2024-03-23T09:51:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.500+0000 7fd5c7f2e640 1 --2- 172.21.15.47:0/521356527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c00cb6c0 0x7fd5c00cecc0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.500+0000 7fd5c7f2e640 1 --2- 172.21.15.47:0/521356527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c00ce8e0 0x7fd5c0069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.500+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/521356527 >> 172.21.15.47:0/521356527 conn(0x7fd5c0098a00 msgr2=0x7fd5c00b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.500+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/521356527 shutdown_connections 2024-03-23T09:51:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.500+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/521356527 wait complete. 2024-03-23T09:51:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.501+0000 7fd5c7f2e640 1 Processor -- start 2024-03-23T09:51:52.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.501+0000 7fd5c7f2e640 1 -- start start 2024-03-23T09:51:52.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.502+0000 7fd5c7f2e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c00c9ba0 0x7fd5c0159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.502+0000 7fd5c7f2e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5c00cb6c0 0x7fd5c00c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.502+0000 7fd5c7f2e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c00ce8e0 0x7fd5c00c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.502+0000 7fd5c7f2e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00c7820 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.502+0000 7fd5c7f2e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00c4650 con 0x7fd5c00cb6c0 2024-03-23T09:51:52.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.502+0000 7fd5c7f2e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00c4370 con 0x7fd5c00ce8e0 2024-03-23T09:51:52.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.502+0000 7fd5c5cca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c00c9ba0 0x7fd5c0159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.502+0000 7fd5c5cca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c00c9ba0 0x7fd5c0159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41110/0 (socket says 172.21.15.47:41110) 2024-03-23T09:51:52.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.502+0000 7fd5c5cca640 1 -- 172.21.15.47:0/1636104748 learned_addr learned my addr 172.21.15.47:0/1636104748 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.502+0000 7fd5c54c9640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5c00cb6c0 0x7fd5c00c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.502+0000 7fd5c64cb640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c00ce8e0 0x7fd5c00c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.503+0000 7fd5c5cca640 1 -- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c00ce8e0 msgr2=0x7fd5c00c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.503+0000 7fd5c5cca640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c00ce8e0 0x7fd5c00c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.503+0000 7fd5c5cca640 1 -- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5c00cb6c0 msgr2=0x7fd5c00c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.503+0000 7fd5c5cca640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5c00cb6c0 0x7fd5c00c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.503+0000 7fd5c5cca640 1 -- 172.21.15.47:0/1636104748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b40094d0 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.503+0000 7fd5c64cb640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c00ce8e0 0x7fd5c00c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.503+0000 7fd5c5cca640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c00c9ba0 0x7fd5c0159040 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800d5d0 tx=0x7fd5b800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.503+0000 7fd5be7fc640 1 -- 172.21.15.47:0/1636104748 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b80125f0 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.503+0000 7fd5be7fc640 1 -- 172.21.15.47:0/1636104748 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b8012770 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.503+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/1636104748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c00c7160 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.503+0000 7fd5be7fc640 1 -- 172.21.15.47:0/1636104748 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8012b20 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.503+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/1636104748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c01596a0 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.504+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/1636104748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c0066be0 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.504+0000 7fd5be7fc640 1 -- 172.21.15.47:0/1636104748 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd5b8012c80 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.505+0000 7fd5be7fc640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5ac06ac00 0x7fd5ac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.505+0000 7fd5be7fc640 1 -- 172.21.15.47:0/1636104748 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd5b80897c0 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.505+0000 7fd5c54c9640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5ac06ac00 0x7fd5ac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.506+0000 7fd5c54c9640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5ac06ac00 0x7fd5ac06d0c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4036000 tx=0x7fd5b4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:52.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.509+0000 7fd5be7fc640 1 -- 172.21.15.47:0/1636104748 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd5b80549f0 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.700+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/1636104748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd5c00c5270 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.700+0000 7fd5be7fc640 1 -- 172.21.15.47:0/1636104748 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v115) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd5b8058010 con 0x7fd5c00c9ba0 2024-03-23T09:51:52.700 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:51:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.702+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5ac06ac00 msgr2=0x7fd5ac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.702+0000 7fd5c7f2e640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5ac06ac00 0x7fd5ac06d0c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4036000 tx=0x7fd5b4008030 comp rx=0 tx=0).stop 2024-03-23T09:51:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.702+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/1636104748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c00c9ba0 msgr2=0x7fd5c0159040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.702+0000 7fd5c7f2e640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c00c9ba0 0x7fd5c0159040 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800d5d0 tx=0x7fd5b800da90 comp rx=0 tx=0).stop 2024-03-23T09:51:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.702+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/1636104748 shutdown_connections 2024-03-23T09:51:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.702+0000 7fd5c7f2e640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5ac06ac00 0x7fd5ac06d0c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.702+0000 7fd5c7f2e640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c00ce8e0 0x7fd5c00c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.702+0000 7fd5c7f2e640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5c00cb6c0 0x7fd5c00c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.702+0000 7fd5c7f2e640 1 --2- 172.21.15.47:0/1636104748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c00c9ba0 0x7fd5c0159040 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.702+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/1636104748 >> 172.21.15.47:0/1636104748 conn(0x7fd5c0098a00 msgr2=0x7fd5c006d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.702+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/1636104748 shutdown_connections 2024-03-23T09:51:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.702+0000 7fd5c7f2e640 1 -- 172.21.15.47:0/1636104748 wait complete. 2024-03-23T09:51:52.712 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:51:52.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:52.713 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:52.738 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[79015]: fuse finished with error 0 and tester_r 0 2024-03-23T09:51:52.739 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:51:52.775 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:51:52.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:52.776 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:51:52.831 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T09:51:52.831 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-03-23T09:51:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.973+0000 7fe4ef230640 1 Processor -- start 2024-03-23T09:51:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.973+0000 7fe4ef230640 1 -- start start 2024-03-23T09:51:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.974+0000 7fe4ef230640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e80cde10 0x7fe4e8160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.974+0000 7fe4ef230640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e8161630 con 0x7fe4e815c990 2024-03-23T09:51:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.974+0000 7fe4ef230640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e80d1030 con 0x7fe4e80b8350 2024-03-23T09:51:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.974+0000 7fe4ef230640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e80cfe10 con 0x7fe4e80cde10 2024-03-23T09:51:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.974+0000 7fe4ecfcc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe4e815c990 0x7fe4e80cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47950/0 (socket says 172.21.15.47:47950) 2024-03-23T09:51:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.974+0000 7fe4ecfcc640 1 -- 172.21.15.47:0/3636148384 learned_addr learned my addr 172.21.15.47:0/3636148384 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:52.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802241304 0 0) 0x7fe4e8161630 con 0x7fe4e815c990 2024-03-23T09:51:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4d0003520 con 0x7fe4e815c990 2024-03-23T09:51:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/3636148384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e80cde10 0x7fe4e8160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2401642963 0 0) 0x7fe4d0003520 con 0x7fe4e815c990 2024-03-23T09:51:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e80cde10 msgr2=0x7fe4e8160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 --2- 172.21.15.47:0/3636148384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e80cde10 0x7fe4e8160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 >> v1:172.21.15.175:6789/0 conn(0x7fe4e80b8350 legacy=0x7fe4e80d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e80b5b60 con 0x7fe4e815c990 2024-03-23T09:51:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fe4e00019f0 con 0x7fe4e815c990 2024-03-23T09:51:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/3636148384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e80cde10 0x7fe4e8160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 >> v1:172.21.15.47:6789/0 conn(0x7fe4e815c990 legacy=0x7fe4e80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 --2- 172.21.15.47:0/3636148384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d00039a0 0x7fe4d0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 --2- 172.21.15.47:0/3636148384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d00047c0 0x7fe4d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 --2- 172.21.15.47:0/3636148384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e80cde10 0x7fe4d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e80cfe10 con 0x7fe4d00047c0 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d00054c0 con 0x7fe4d00039a0 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d0005680 con 0x7fe4e80cde10 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/3636148384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d00047c0 0x7fe4d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/3636148384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d00039a0 0x7fe4d0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4ed7cd640 1 --2- 172.21.15.47:0/3636148384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e80cde10 0x7fe4d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e7fff640 1 -- 172.21.15.47:0/3636148384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e80cde10 msgr2=0x7fe4d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/3636148384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e80cde10 0x7fe4d000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e7fff640 1 -- 172.21.15.47:0/3636148384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d00039a0 msgr2=0x7fe4d0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/3636148384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d00039a0 0x7fe4d0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.975+0000 7fe4e7fff640 1 -- 172.21.15.47:0/3636148384 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d0003520 con 0x7fe4d00047c0 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.976+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/3636148384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d00047c0 0x7fe4d0004c40 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0098a0 tx=0x7fe4d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57952420229a7355 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.976+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc004230 con 0x7fe4d00047c0 2024-03-23T09:51:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.976+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4dc004390 con 0x7fe4d00047c0 2024-03-23T09:51:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.976+0000 7fe4e77fe640 1 -- 172.21.15.47:0/3636148384 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc003830 con 0x7fe4d00047c0 2024-03-23T09:51:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.977+0000 7fe4ef230640 1 -- 172.21.15.47:0/3636148384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d00047c0 msgr2=0x7fe4d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.977+0000 7fe4ef230640 1 --2- 172.21.15.47:0/3636148384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d00047c0 0x7fe4d0004c40 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0098a0 tx=0x7fe4d00057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.977+0000 7fe4ed7cd640 1 -- 172.21.15.47:0/3636148384 reap_dead start 2024-03-23T09:51:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.977+0000 7fe4ef230640 1 -- 172.21.15.47:0/3636148384 shutdown_connections 2024-03-23T09:51:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.977+0000 7fe4ef230640 1 -- 172.21.15.47:0/3636148384 >> 172.21.15.47:0/3636148384 conn(0x7fe4e80659d0 msgr2=0x7fe4e8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.977+0000 7fe4ef230640 1 -- 172.21.15.47:0/3636148384 shutdown_connections 2024-03-23T09:51:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.977+0000 7fe4ef230640 1 -- 172.21.15.47:0/3636148384 wait complete. 2024-03-23T09:51:52.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.978+0000 7fe4ef230640 1 Processor -- start 2024-03-23T09:51:52.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.978+0000 7fe4ef230640 1 -- start start 2024-03-23T09:51:52.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4ef230640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d00047c0 0x7fe4e8153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4ef230640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4e8154750 0x7fe4e81f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4ef230640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e8153f70 0x7fe4e81f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4ef230640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e81525a0 con 0x7fe4e8154750 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4ef230640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e8152880 con 0x7fe4d00047c0 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4ef230640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e8152b60 con 0x7fe4e8153f70 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4e7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4e8154750 0x7fe4e81f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4e7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4e8154750 0x7fe4e81f15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41122/0 (socket says 172.21.15.47:41122) 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4e7fff640 1 -- 172.21.15.47:0/3766166121 learned_addr learned my addr 172.21.15.47:0/3766166121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4ed7cd640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e8153f70 0x7fe4e81f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d00047c0 0x7fe4e8153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4e7fff640 1 -- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e8153f70 msgr2=0x7fe4e81f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e8153f70 0x7fe4e81f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4e7fff640 1 -- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d00047c0 msgr2=0x7fe4e8153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d00047c0 0x7fe4e8153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.979+0000 7fe4e7fff640 1 -- 172.21.15.47:0/3766166121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e8152e40 con 0x7fe4e8154750 2024-03-23T09:51:52.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.980+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4e8154750 0x7fe4e81f15a0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0098a0 tx=0x7fe4dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.980+0000 7fe4e57fa640 1 -- 172.21.15.47:0/3766166121 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc030630 con 0x7fe4e8154750 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.980+0000 7fe4e57fa640 1 -- 172.21.15.47:0/3766166121 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4dc030790 con 0x7fe4e8154750 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.980+0000 7fe4ef230640 1 -- 172.21.15.47:0/3766166121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e8153120 con 0x7fe4e8154750 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.980+0000 7fe4e57fa640 1 -- 172.21.15.47:0/3766166121 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc030b00 con 0x7fe4e8154750 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.980+0000 7fe4ef230640 1 -- 172.21.15.47:0/3766166121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e81f2020 con 0x7fe4e8154750 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.981+0000 7fe4e57fa640 1 -- 172.21.15.47:0/3766166121 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe4dc030c60 con 0x7fe4e8154750 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.981+0000 7fe4e57fa640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe4d406ac00 0x7fe4d406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.982+0000 7fe4e57fa640 1 -- 172.21.15.47:0/3766166121 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe4dc0ab170 con 0x7fe4e8154750 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.982+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe4d406ac00 0x7fe4d406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.982+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe4d406ac00 0x7fe4d406d0c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0001b10 tx=0x7fe4e0006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.983+0000 7fe4f1d9b740 1 Processor -- start 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.983+0000 7fe4f1d9b740 1 -- start start 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.983+0000 7fe4f1d9b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 0x5584ccc00c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.983+0000 7fe4f1d9b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbd0850 0x5584ccbfdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.983+0000 7fe4f1d9b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 0x5584ccbfdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.983+0000 7fe4f1d9b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ccb40dd0 con 0x5584ccbd43b0 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.983+0000 7fe4f1d9b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ccb3e560 con 0x5584ccbd0850 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.983+0000 7fe4f1d9b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584ccb39e50 con 0x5584ccc00830 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.983+0000 7fe4ed7cd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 0x5584ccbfdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.983+0000 7fe4ed7cd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 0x5584ccbfdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41132/0 (socket says 172.21.15.47:41132) 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.983+0000 7fe4ed7cd640 1 -- 172.21.15.47:0/4010399563 learned_addr learned my addr 172.21.15.47:0/4010399563 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.984+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/4010399563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbd0850 0x5584ccbfdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.984+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/4010399563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 0x5584ccc00c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.984+0000 7fe4ed7cd640 1 -- 172.21.15.47:0/4010399563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 msgr2=0x5584ccc00c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.984+0000 7fe4ed7cd640 1 --2- 172.21.15.47:0/4010399563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 0x5584ccc00c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.984+0000 7fe4ed7cd640 1 -- 172.21.15.47:0/4010399563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbd0850 msgr2=0x5584ccbfdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.984+0000 7fe4ed7cd640 1 --2- 172.21.15.47:0/4010399563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbd0850 0x5584ccbfdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.984+0000 7fe4ed7cd640 1 -- 172.21.15.47:0/4010399563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584ccb01470 con 0x5584ccbd43b0 2024-03-23T09:51:52.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.984+0000 7fe4ed7cd640 1 --2- 172.21.15.47:0/4010399563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 0x5584ccbfdfe0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80cd210 tx=0x7fe4d8007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7960e8b4ad19a812 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:52.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.984+0000 7fe4ce7fc640 1 -- 172.21.15.47:0/4010399563 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80014a0 con 0x5584ccbd43b0 2024-03-23T09:51:52.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.984+0000 7fe4ce7fc640 1 -- 172.21.15.47:0/4010399563 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d8001600 con 0x5584ccbd43b0 2024-03-23T09:51:52.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.984+0000 7fe4ce7fc640 1 -- 172.21.15.47:0/4010399563 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8001990 con 0x5584ccbd43b0 2024-03-23T09:51:52.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.985+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/4010399563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 msgr2=0x5584ccbfdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.985+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/4010399563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 0x5584ccbfdfe0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80cd210 tx=0x7fe4d8007c80 comp rx=0 tx=0).stop 2024-03-23T09:51:52.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.985+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/4010399563 shutdown_connections 2024-03-23T09:51:52.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.985+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/4010399563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 0x5584ccbfdfe0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.985+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/4010399563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbd0850 0x5584ccbfdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.985+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/4010399563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 0x5584ccc00c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.985+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/4010399563 >> 172.21.15.47:0/4010399563 conn(0x5584ccbd2ac0 msgr2=0x5584ccc1b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:52.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.985+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/4010399563 shutdown_connections 2024-03-23T09:51:52.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.985+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/4010399563 wait complete. 2024-03-23T09:51:52.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.985+0000 7fe4f1d9b740 1 -- start start 2024-03-23T09:51:52.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.986+0000 7fe4f1d9b740 1 Processor -- start 2024-03-23T09:51:52.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.986+0000 7fe4f1d9b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbd43b0 0x5584ccbfaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.986+0000 7fe4f1d9b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 0x5584ccc03dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.986+0000 7fe4f1d9b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbf97d0 0x5584ccc03630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.986+0000 7fe4f1d9b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ccaf8280 con 0x5584ccbf97d0 2024-03-23T09:51:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.986+0000 7fe4f1d9b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ccaf7f20 con 0x5584ccbd43b0 2024-03-23T09:51:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.986+0000 7fe4f1d9b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584ccabd3c0 con 0x5584ccc00830 2024-03-23T09:51:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.986+0000 7fe4f1d9b740 10 client.0 _ll_register_callbacks cb 0x5584ccb0f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:51:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.986+0000 7fe4ed7cd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbf97d0 0x5584ccc03630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.986+0000 7fe4ed7cd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbf97d0 0x5584ccc03630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41134/0 (socket says 172.21.15.47:41134) 2024-03-23T09:51:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.986+0000 7fe4ed7cd640 1 -- 172.21.15.47:0/1807645015 learned_addr learned my addr 172.21.15.47:0/1807645015 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.986+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/1807645015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbd43b0 0x5584ccbfaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.986+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/1807645015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 0x5584ccc03dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4ed7cd640 1 -- 172.21.15.47:0/1807645015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 msgr2=0x5584ccc03dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4ed7cd640 1 --2- 172.21.15.47:0/1807645015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 0x5584ccc03dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4ed7cd640 1 -- 172.21.15.47:0/1807645015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbd43b0 msgr2=0x5584ccbfaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4ed7cd640 1 --2- 172.21.15.47:0/1807645015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbd43b0 0x5584ccbfaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4ed7cd640 1 -- 172.21.15.47:0/1807645015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e80d1030 con 0x5584ccbf97d0 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4ed7cd640 1 --2- 172.21.15.47:0/1807645015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbf97d0 0x5584ccc03630 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8000c00 tx=0x7fe4d8001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e126c571ed0442 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4c77fe640 1 -- 172.21.15.47:0/1807645015 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d801f850 con 0x5584ccbf97d0 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4c77fe640 1 -- 172.21.15.47:0/1807645015 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe4d801f9b0 con 0x5584ccbf97d0 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/1807645015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584ccb63690 con 0x5584ccbf97d0 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4c77fe640 1 -- 172.21.15.47:0/1807645015 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4d801fb70 con 0x5584ccbf97d0 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4f1d9b740 10 client.9477 fetch_fsmap learned FSMap version 80 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/1807645015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x5584ccb384d0 con 0x5584ccbf97d0 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4c77fe640 1 -- 172.21.15.47:0/1807645015 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5584ccb63690 con 0x5584ccbf97d0 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4c77fe640 1 -- 172.21.15.47:0/1807645015 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 80) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe4d801fcd0 con 0x5584ccbf97d0 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4f1d9b740 10 client.9477 fetch_fsmap finished waiting for FSMap version 80 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4f1d9b740 10 client.9477 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4f1d9b740 20 client.9477 populate_metadata read hostname 'smithi047' 2024-03-23T09:51:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.987+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/1807645015 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x5584ccc2b4a0 0x5584ccc01150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.988+0000 7fe4f1d9b740 4 client.9477 mds_command: new command op to 9234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.988+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/1807645015 --> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584ccb2beb0 con 0x5584ccc2b4a0 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.988+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/1807645015 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x5584ccc2b4a0 0x5584ccc01150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.988+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/1807645015 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x5584ccc2b4a0 0x5584ccc01150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8aa130c08ec03ca server_cookie=5fda8b110a57907b in_seq=0 out_seq=0 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.988+0000 7fe4c77fe640 10 client.9477 ms_handle_connect on v2:172.21.15.47:6832/724782441 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.988+0000 7fe4c77fe640 1 -- 172.21.15.47:0/1807645015 <== mds.0 v2:172.21.15.47:6832/724782441 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5584ccb2beb0 con 0x5584ccc2b4a0 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.988+0000 7fe4c77fe640 10 client.9477 handle_command_reply: tid=0 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.988+0000 7fe4f1d9b740 1 client.9477 shutdown 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.989+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/1807645015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbf97d0 msgr2=0x5584ccc03630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.989+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/1807645015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbf97d0 0x5584ccc03630 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8000c00 tx=0x7fe4d8001190 comp rx=0 tx=0).stop 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.989+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/1807645015 shutdown_connections 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.989+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/1807645015 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x5584ccc2b4a0 0x5584ccc01150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.989+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/1807645015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbf97d0 0x5584ccc03630 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.989+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/1807645015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 0x5584ccc03dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.989+0000 7fe4c77fe640 0 client.9477 ms_handle_reset on v2:172.21.15.47:6832/724782441 2024-03-23T09:51:52.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.989+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/1807645015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbd43b0 0x5584ccbfaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:52.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.989+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/1807645015 >> 172.21.15.47:0/1807645015 conn(0x5584ccbd0850 msgr2=0x5584ccc04440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:52.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.989+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/1807645015 shutdown_connections 2024-03-23T09:51:52.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.989+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/1807645015 wait complete. 2024-03-23T09:51:52.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:52.989+0000 7fe4f1d9b740 20 client.9477 trim_cache size 0 max 16384 2024-03-23T09:51:53.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.001+0000 7fe4f1d9b740 1 Processor -- start 2024-03-23T09:51:53.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.001+0000 7fe4f1d9b740 1 -- start start 2024-03-23T09:51:53.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.002+0000 7fe4f1d9b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 0x5584ccbd73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:53.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.002+0000 7fe4f1d9b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbf97d0 0x5584ccbfbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:53.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.002+0000 7fe4f1d9b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 0x5584ccc7c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:53.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.002+0000 7fe4f1d9b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ccb38f50 con 0x5584ccbd43b0 2024-03-23T09:51:53.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.002+0000 7fe4f1d9b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ccc03d30 con 0x5584ccbf97d0 2024-03-23T09:51:53.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.002+0000 7fe4f1d9b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584ccc29cc0 con 0x5584ccc00830 2024-03-23T09:51:53.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.002+0000 7fe4ecfcc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 0x5584ccbd73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:53.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.002+0000 7fe4ecfcc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 0x5584ccbd73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41148/0 (socket says 172.21.15.47:41148) 2024-03-23T09:51:53.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.002+0000 7fe4ecfcc640 1 -- 172.21.15.47:0/1420908607 learned_addr learned my addr 172.21.15.47:0/1420908607 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:53.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.002+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/1420908607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbf97d0 0x5584ccbfbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:53.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.002+0000 7fe4ed7cd640 1 --2- 172.21.15.47:0/1420908607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 0x5584ccc7c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:53.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4ecfcc640 1 -- 172.21.15.47:0/1420908607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 msgr2=0x5584ccc7c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:53.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/1420908607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 0x5584ccc7c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4ecfcc640 1 -- 172.21.15.47:0/1420908607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbf97d0 msgr2=0x5584ccbfbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:53.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/1420908607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbf97d0 0x5584ccbfbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4ecfcc640 1 -- 172.21.15.47:0/1420908607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584ccb6c9e0 con 0x5584ccbd43b0 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4ed7cd640 1 --2- 172.21.15.47:0/1420908607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 0x5584ccc7c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/1420908607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 0x5584ccbd73f0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0007b50 tx=0x7fe4e000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537152659bed1e07 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4ceffd640 1 -- 172.21.15.47:0/1420908607 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00281f0 con 0x5584ccbd43b0 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4ceffd640 1 -- 172.21.15.47:0/1420908607 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e0028350 con 0x5584ccbd43b0 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4ceffd640 1 -- 172.21.15.47:0/1420908607 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00154b0 con 0x5584ccbd43b0 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/1420908607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 msgr2=0x5584ccbd73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/1420908607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 0x5584ccbd73f0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0007b50 tx=0x7fe4e000c000 comp rx=0 tx=0).stop 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/1420908607 shutdown_connections 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/1420908607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccc00830 0x5584ccc7c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/1420908607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbf97d0 0x5584ccbfbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/1420908607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccbd43b0 0x5584ccbd73f0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/1420908607 >> 172.21.15.47:0/1420908607 conn(0x5584ccbd0850 msgr2=0x5584ccc03440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/1420908607 shutdown_connections 2024-03-23T09:51:53.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.003+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/1420908607 wait complete. 2024-03-23T09:51:53.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.004+0000 7fe4f1d9b740 1 -- start start 2024-03-23T09:51:53.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.004+0000 7fe4f1d9b740 1 Processor -- start 2024-03-23T09:51:53.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4f1d9b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccbd43b0 0x5584ccc6afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:53.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4f1d9b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbf97d0 0x5584ccc10fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:53.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4f1d9b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccc00830 0x5584ccc11510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:53.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4f1d9b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ccbf8500 con 0x5584ccc00830 2024-03-23T09:51:53.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4f1d9b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ccc7a0b0 con 0x5584ccbf97d0 2024-03-23T09:51:53.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4f1d9b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584ccc6b520 con 0x5584ccbd43b0 2024-03-23T09:51:53.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4f1d9b740 10 client.0 _ll_register_callbacks cb 0x7fe4d8018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:51:53.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4ed7cd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccc00830 0x5584ccc11510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:53.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4ed7cd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccc00830 0x5584ccc11510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41160/0 (socket says 172.21.15.47:41160) 2024-03-23T09:51:53.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4ed7cd640 1 -- 172.21.15.47:0/3948988951 learned_addr learned my addr 172.21.15.47:0/3948988951 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:53.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccbd43b0 0x5584ccc6afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:53.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbf97d0 0x5584ccc10fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:53.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4ed7cd640 1 -- 172.21.15.47:0/3948988951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccbd43b0 msgr2=0x5584ccc6afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:53.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4ed7cd640 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccbd43b0 0x5584ccc6afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4ed7cd640 1 -- 172.21.15.47:0/3948988951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbf97d0 msgr2=0x5584ccc10fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:53.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4ed7cd640 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbf97d0 0x5584ccc10fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4ed7cd640 1 -- 172.21.15.47:0/3948988951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e80b5b60 con 0x5584ccc00830 2024-03-23T09:51:53.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.005+0000 7fe4e7fff640 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbf97d0 0x5584ccc10fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:53.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4ed7cd640 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccc00830 0x5584ccc11510 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8005180 tx=0x7fe4d800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3eae06512e0f3e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:53.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:51:53.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4c77fe640 1 -- 172.21.15.47:0/3948988951 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8001d80 con 0x5584ccc00830 2024-03-23T09:51:53.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4c77fe640 1 -- 172.21.15.47:0/3948988951 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe4d801fa60 con 0x5584ccc00830 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/3948988951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4d8001600 con 0x5584ccc00830 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4c77fe640 1 -- 172.21.15.47:0/3948988951 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4d801fc20 con 0x5584ccc00830 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4f1d9b740 10 client.9483 fetch_fsmap learned FSMap version 80 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/3948988951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x5584ccada5c0 con 0x5584ccc00830 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4c77fe640 1 -- 172.21.15.47:0/3948988951 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d8001600 con 0x5584ccc00830 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4c77fe640 1 -- 172.21.15.47:0/3948988951 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 80) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe4d8039410 con 0x5584ccc00830 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4f1d9b740 10 client.9483 fetch_fsmap finished waiting for FSMap version 80 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4f1d9b740 10 client.9483 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4f1d9b740 20 client.9483 populate_metadata read hostname 'smithi047' 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x5584ccc39890 0x5584ccc79200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4f1d9b740 4 client.9483 mds_command: new command op to 9234 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/3948988951 --> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5584cc82d4e0 con 0x5584ccc39890 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.006+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x5584ccc39890 0x5584ccc79200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.007+0000 7fe4c77fe640 10 client.9483 ms_handle_connect on v2:172.21.15.47:6832/724782441 2024-03-23T09:51:53.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.007+0000 7fe4ecfcc640 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x5584ccc39890 0x5584ccc79200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95759e3663f8041b server_cookie=190ef1b8e403e5d5 in_seq=0 out_seq=0 2024-03-23T09:51:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.175+0000 7fe4c77fe640 1 -- 172.21.15.47:0/3948988951 <== mds.0 v2:172.21.15.47:6832/724782441 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5584cc82d4e0 con 0x5584ccc39890 2024-03-23T09:51:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.175+0000 7fe4c77fe640 10 client.9483 handle_command_reply: tid=0 2024-03-23T09:51:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.175+0000 7fe4f1d9b740 1 client.9483 shutdown 2024-03-23T09:51:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/3948988951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccc00830 msgr2=0x5584ccc11510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccc00830 0x5584ccc11510 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8005180 tx=0x7fe4d800c000 comp rx=0 tx=0).stop 2024-03-23T09:51:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/3948988951 shutdown_connections 2024-03-23T09:51:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.47:6832/724782441,v1:172.21.15.47:6834/724782441] conn(0x5584ccc39890 0x5584ccc79200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584ccc00830 0x5584ccc11510 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584ccbf97d0 0x5584ccc10fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4c77fe640 0 client.9483 ms_handle_reset on v2:172.21.15.47:6832/724782441 2024-03-23T09:51:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4f1d9b740 1 --2- 172.21.15.47:0/3948988951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5584ccbd43b0 0x5584ccc6afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/3948988951 >> 172.21.15.47:0/3948988951 conn(0x5584ccbd0850 msgr2=0x5584ccc043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:53.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/3948988951 shutdown_connections 2024-03-23T09:51:53.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4f1d9b740 1 -- 172.21.15.47:0/3948988951 wait complete. 2024-03-23T09:51:53.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4f1d9b740 20 client.9483 trim_cache size 0 max 16384 2024-03-23T09:51:53.178 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:51:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: "message": "", 2024-03-23T09:51:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0 2024-03-23T09:51:53.178 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:51:53.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4ef230640 1 -- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe4d406ac00 msgr2=0x7fe4d406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:53.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4ef230640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe4d406ac00 0x7fe4d406d0c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0001b10 tx=0x7fe4e0006040 comp rx=0 tx=0).stop 2024-03-23T09:51:53.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4ef230640 1 -- 172.21.15.47:0/3766166121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4e8154750 msgr2=0x7fe4e81f15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:53.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.176+0000 7fe4ef230640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4e8154750 0x7fe4e81f15a0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0098a0 tx=0x7fe4dc004730 comp rx=0 tx=0).stop 2024-03-23T09:51:53.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.177+0000 7fe4ef230640 1 -- 172.21.15.47:0/3766166121 shutdown_connections 2024-03-23T09:51:53.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.177+0000 7fe4ef230640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe4d406ac00 0x7fe4d406d0c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.177+0000 7fe4ef230640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4e8153f70 0x7fe4e81f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.177+0000 7fe4ef230640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4e8154750 0x7fe4e81f15a0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.177+0000 7fe4ef230640 1 --2- 172.21.15.47:0/3766166121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d00047c0 0x7fe4e8153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.177+0000 7fe4ef230640 1 -- 172.21.15.47:0/3766166121 >> 172.21.15.47:0/3766166121 conn(0x7fe4e80659d0 msgr2=0x7fe4e80d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:53.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.177+0000 7fe4ef230640 1 -- 172.21.15.47:0/3766166121 shutdown_connections 2024-03-23T09:51:53.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.177+0000 7fe4ef230640 1 -- 172.21.15.47:0/3766166121 wait complete. 2024-03-23T09:51:53.187 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:51:53.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.288+0000 7fca7293e640 1 Processor -- start 2024-03-23T09:51:53.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.288+0000 7fca7293e640 1 -- start start 2024-03-23T09:51:53.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.288+0000 7fca7293e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fca6c15cd00 0x7fca6c0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:53.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.288+0000 7fca7293e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca6c0cf920 0x7fca6c0b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:53.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.288+0000 7fca7293e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca6c15d0e0 con 0x7fca6c0d1020 2024-03-23T09:51:53.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.288+0000 7fca7293e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca6c0d1540 con 0x7fca6c15cd00 2024-03-23T09:51:53.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.288+0000 7fca7293e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c0d1820 con 0x7fca6c0cf920 2024-03-23T09:51:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.289+0000 7fca6bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fca6c15cd00 0x7fca6c0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.289+0000 7fca6bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fca6c15cd00 0x7fca6c0d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41176/0 (socket says 172.21.15.47:41176) 2024-03-23T09:51:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.289+0000 7fca6bfff640 1 -- 172.21.15.47:0/3811411481 learned_addr learned my addr 172.21.15.47:0/3811411481 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.289+0000 7fca70edb640 1 --2- 172.21.15.47:0/3811411481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca6c0cf920 0x7fca6c0b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.289+0000 7fca6bfff640 1 -- 172.21.15.47:0/3811411481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca6c0cf920 msgr2=0x7fca6c0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.289+0000 7fca6bfff640 1 --2- 172.21.15.47:0/3811411481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca6c0cf920 0x7fca6c0b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.289+0000 7fca6bfff640 1 -- 172.21.15.47:0/3811411481 >> v1:172.21.15.175:6789/0 conn(0x7fca6c0d1020 legacy=0x7fca6c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.289+0000 7fca6bfff640 1 -- 172.21.15.47:0/3811411481 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c0d1b00 con 0x7fca6c15cd00 2024-03-23T09:51:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.289+0000 7fca70edb640 1 --2- 172.21.15.47:0/3811411481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca6c0cf920 0x7fca6c0b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.290+0000 7fca6bfff640 1 --2- 172.21.15.47:0/3811411481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c15cd00 0x7fca6c0d0ae0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009870 tx=0x7fca5c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4f5f90c0d9e831 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.290+0000 7fca6affd640 1 -- 172.21.15.47:0/3811411481 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0317c0 con 0x7fca6c15cd00 2024-03-23T09:51:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.290+0000 7fca6affd640 1 -- 172.21.15.47:0/3811411481 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5c031920 con 0x7fca6c15cd00 2024-03-23T09:51:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.290+0000 7fca6affd640 1 -- 172.21.15.47:0/3811411481 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c031cb0 con 0x7fca6c15cd00 2024-03-23T09:51:53.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.290+0000 7fca7293e640 1 -- 172.21.15.47:0/3811411481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c15cd00 msgr2=0x7fca6c0d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:53.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.290+0000 7fca7293e640 1 --2- 172.21.15.47:0/3811411481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c15cd00 0x7fca6c0d0ae0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009870 tx=0x7fca5c02b860 comp rx=0 tx=0).stop 2024-03-23T09:51:53.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.290+0000 7fca7293e640 1 -- 172.21.15.47:0/3811411481 shutdown_connections 2024-03-23T09:51:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.290+0000 7fca7293e640 1 --2- 172.21.15.47:0/3811411481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca6c0cf920 0x7fca6c0b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.290+0000 7fca7293e640 1 --2- 172.21.15.47:0/3811411481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c15cd00 0x7fca6c0d0ae0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.290+0000 7fca7293e640 1 -- 172.21.15.47:0/3811411481 >> 172.21.15.47:0/3811411481 conn(0x7fca6c06b160 msgr2=0x7fca6c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.290+0000 7fca7293e640 1 -- 172.21.15.47:0/3811411481 shutdown_connections 2024-03-23T09:51:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.290+0000 7fca7293e640 1 -- 172.21.15.47:0/3811411481 wait complete. 2024-03-23T09:51:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.292+0000 7fca7293e640 1 Processor -- start 2024-03-23T09:51:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.292+0000 7fca7293e640 1 -- start start 2024-03-23T09:51:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.292+0000 7fca7293e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c0cf920 0x7fca6c14bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.292+0000 7fca7293e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c0d1020 0x7fca6c15e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.292+0000 7fca7293e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca6c15cd00 0x7fca6c15eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.292+0000 7fca7293e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c1498b0 con 0x7fca6c0cf920 2024-03-23T09:51:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.292+0000 7fca7293e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c0cc9a0 con 0x7fca6c0d1020 2024-03-23T09:51:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.292+0000 7fca7293e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c0ccc80 con 0x7fca6c15cd00 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca6bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c0cf920 0x7fca6c14bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca6bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c0cf920 0x7fca6c14bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41180/0 (socket says 172.21.15.47:41180) 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca6bfff640 1 -- 172.21.15.47:0/2258272110 learned_addr learned my addr 172.21.15.47:0/2258272110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca70edb640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca6c15cd00 0x7fca6c15eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca6b7fe640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c0d1020 0x7fca6c15e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca6bfff640 1 -- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca6c15cd00 msgr2=0x7fca6c15eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca6bfff640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca6c15cd00 0x7fca6c15eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca6bfff640 1 -- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c0d1020 msgr2=0x7fca6c15e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca6bfff640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c0d1020 0x7fca6c15e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca6bfff640 1 -- 172.21.15.47:0/2258272110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c0094d0 con 0x7fca6c0cf920 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca70edb640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca6c15cd00 0x7fca6c15eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca6b7fe640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c0d1020 0x7fca6c15e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca6bfff640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c0cf920 0x7fca6c14bc20 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fca5c000c00 tx=0x7fca5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca68ff9640 1 -- 172.21.15.47:0/2258272110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0329a0 con 0x7fca6c0cf920 2024-03-23T09:51:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca68ff9640 1 -- 172.21.15.47:0/2258272110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5c032b20 con 0x7fca6c0cf920 2024-03-23T09:51:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca7293e640 1 -- 172.21.15.47:0/2258272110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6c1491f0 con 0x7fca6c0cf920 2024-03-23T09:51:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca68ff9640 1 -- 172.21.15.47:0/2258272110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0453f0 con 0x7fca6c0cf920 2024-03-23T09:51:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.293+0000 7fca7293e640 1 -- 172.21.15.47:0/2258272110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca6c14c160 con 0x7fca6c0cf920 2024-03-23T09:51:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.294+0000 7fca7293e640 1 -- 172.21.15.47:0/2258272110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca6c072360 con 0x7fca6c0cf920 2024-03-23T09:51:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.295+0000 7fca68ff9640 1 -- 172.21.15.47:0/2258272110 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fca5c04d3f0 con 0x7fca6c0cf920 2024-03-23T09:51:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.295+0000 7fca68ff9640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca5806ac00 0x7fca5806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.295+0000 7fca68ff9640 1 -- 172.21.15.47:0/2258272110 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fca5c0ab3f0 con 0x7fca6c0cf920 2024-03-23T09:51:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.295+0000 7fca6b7fe640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca5806ac00 0x7fca5806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.296+0000 7fca6b7fe640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca5806ac00 0x7fca5806d0c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fca6c14f900 tx=0x7fca60009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:53.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.299+0000 7fca68ff9640 1 -- 172.21.15.47:0/2258272110 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fca5c076510 con 0x7fca6c0cf920 2024-03-23T09:51:53.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.517+0000 7fca7293e640 1 -- 172.21.15.47:0/2258272110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fca6c066be0 con 0x7fca6c0cf920 2024-03-23T09:51:53.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.517+0000 7fca68ff9640 1 -- 172.21.15.47:0/2258272110 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) v1 ==== 53+0+73 (secure 0 0 0) 0x7fca5c079b30 con 0x7fca6c0cf920 2024-03-23T09:51:53.517 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:51:53.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.519+0000 7fca7293e640 1 -- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca5806ac00 msgr2=0x7fca5806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.519+0000 7fca7293e640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca5806ac00 0x7fca5806d0c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fca6c14f900 tx=0x7fca60009040 comp rx=0 tx=0).stop 2024-03-23T09:51:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.519+0000 7fca7293e640 1 -- 172.21.15.47:0/2258272110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c0cf920 msgr2=0x7fca6c14bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.519+0000 7fca7293e640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c0cf920 0x7fca6c14bc20 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fca5c000c00 tx=0x7fca5c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.519+0000 7fca7293e640 1 -- 172.21.15.47:0/2258272110 shutdown_connections 2024-03-23T09:51:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.519+0000 7fca7293e640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fca5806ac00 0x7fca5806d0c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.519+0000 7fca7293e640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca6c15cd00 0x7fca6c15eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.519+0000 7fca7293e640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c0d1020 0x7fca6c15e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.519+0000 7fca7293e640 1 --2- 172.21.15.47:0/2258272110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c0cf920 0x7fca6c14bc20 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.519+0000 7fca7293e640 1 -- 172.21.15.47:0/2258272110 >> 172.21.15.47:0/2258272110 conn(0x7fca6c06b160 msgr2=0x7fca6c15b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.519+0000 7fca7293e640 1 -- 172.21.15.47:0/2258272110 shutdown_connections 2024-03-23T09:51:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:53.519+0000 7fca7293e640 1 -- 172.21.15.47:0/2258272110 wait complete. 2024-03-23T09:51:53.530 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:51:53.530 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:51:53.530 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:51:53.530 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:51:53.530 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:53.530 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:51:53.530 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:51:53.530 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:51:53.530 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:51:53.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:53.531 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:51:53.546 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:51:53.546 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:51:53.546 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:51:53.546 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:53.546 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:51:53.546 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:53.546 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:51:53.546 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:51:53.546 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42025sec preferred_lft 42025sec 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:51:53.547 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:51:53.548 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:51:53.548 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:51:53.548 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:53.548 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:51:53.548 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:53.548 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:51:53.548 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:51:53.548 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:51:53.548 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:53.548 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:51:53.548 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:51:53.548 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:51:53.548 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:51:53.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:53.549 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:51:53.611 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:51:53.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:53.612 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:51:53.668 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:51:53.668 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:51:53.668 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:51:53.668 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:51:53.668 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:51:53.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:53.669 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:51:53.669 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:51:53.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:51:53.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:51:53.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:51:53.669 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:51:53.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:53.763 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:51:53.777 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:51:53.777 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:51:53.777 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:51:53.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:53.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:51:53.843 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:51:53.843 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:51:53.843 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:53.898 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:51:53.898 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:51:53.899 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:53.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:51:54.020 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:51:54.020 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:51:54.086 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:51:54.087 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:51:54.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:54.087 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:51:54.142 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:51:54.143 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:51:54.185 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:51:54.211 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:51:54.211 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:51:54.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:51:54.269+0000 7f3921a3f480 -1 init, newargv = 0x56479c9861b0 newargc=15 2024-03-23T09:51:54.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[80131]: starting ceph client 2024-03-23T09:51:54.274 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[80131]: starting fuse 2024-03-23T09:51:54.278 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:51:54.279 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:51:54.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:54.279 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:51:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:51:54.334 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:51:54.335 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:54.336 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:51:54.336 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:51:54.336 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:51:54.336 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:51:54.336 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:51:54.336 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:51:54.336 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:51:54.336 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:51:54.336 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:54.336 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:51:54.336 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:51:54.451 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.80131.asok 2024-03-23T09:51:54.637 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.80131.asok 2024-03-23T09:51:54.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:54.637 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.80131.asok status 2024-03-23T09:51:54.773 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:51:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:51:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:51:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:51:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:51:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:51:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:51:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "80131", 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 9495, 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 9495 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1337984290 2024-03-23T09:51:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1337984290 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.9495 192.168.0.1:0/1337984290", 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/1337984290", 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 80, 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 116, 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:51:54.775 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:51:54.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:54.781 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:51:54.781 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:54.796 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:51:54.796 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:54.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:54.796 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:54.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:54.862 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-03-23T09:51:54.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:54.920 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-03-23T09:51:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:8+0 records in 2024-03-23T09:51:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:8+0 records out 2024-03-23T09:51:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0325343 s, 258 MB/s 2024-03-23T09:51:55.011 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:51:55.011 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:51:55.011 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:51:55.011 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:55.011 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-23T09:51:55.011 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:51:55.074 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627779 2024-03-23T09:51:55.263 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:51:55.263 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:51:55.263 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:51:55.263 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:51:55.263 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-23T09:51:55.263 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:51:55.365 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627780 2024-03-23T09:51:55.554 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:51:55.554 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:51:55.610 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:51:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.751+0000 7fe0fe884640 1 Processor -- start 2024-03-23T09:51:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.751+0000 7fe0fe884640 1 -- start start 2024-03-23T09:51:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.752+0000 7fe0fe884640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe0f80cf8a0 0x7fe0f8167b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.752+0000 7fe0fe884640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f80d0ee0 con 0x7fe0f815cb40 2024-03-23T09:51:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.752+0000 7fe0fe884640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f81616f0 con 0x7fe0f80b8300 2024-03-23T09:51:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.752+0000 7fe0fe884640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f80b5a10 con 0x7fe0f80cf8a0 2024-03-23T09:51:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.752+0000 7fe0f7fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe0f815cb40 0x7fe0f8160770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56764/0 (socket says 172.21.15.47:56764) 2024-03-23T09:51:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.752+0000 7fe0f7fff640 1 -- 172.21.15.47:0/3356295077 learned_addr learned my addr 172.21.15.47:0/3356295077 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.752+0000 7fe0fce21640 1 --2- 172.21.15.47:0/3356295077 >> v2:172.21.15.175:3300/0 conn(0x7fe0f80cf8a0 0x7fe0f8167b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.752+0000 7fe0f6ffd640 1 -- 172.21.15.47:0/3356295077 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170877509 0 0) 0x7fe0f80d0ee0 con 0x7fe0f815cb40 2024-03-23T09:51:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.752+0000 7fe0f6ffd640 1 -- 172.21.15.47:0/3356295077 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0e0003520 con 0x7fe0f815cb40 2024-03-23T09:51:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.752+0000 7fe0fce21640 1 -- 172.21.15.47:0/3356295077 >> v1:172.21.15.175:6789/0 conn(0x7fe0f80b8300 legacy=0x7fe0f80cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.752+0000 7fe0fce21640 1 -- 172.21.15.47:0/3356295077 >> v1:172.21.15.47:6789/0 conn(0x7fe0f815cb40 legacy=0x7fe0f8160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.752+0000 7fe0f6ffd640 1 -- 172.21.15.47:0/3356295077 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2652192861 0 0) 0x7fe0e0003520 con 0x7fe0f815cb40 2024-03-23T09:51:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.752+0000 7fe0fce21640 1 -- 172.21.15.47:0/3356295077 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f8160e80 con 0x7fe0f80cf8a0 2024-03-23T09:51:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.753+0000 7fe0fce21640 1 --2- 172.21.15.47:0/3356295077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f80cf8a0 0x7fe0f8167b00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009dc0 tx=0x7fe0f002be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5971bc5d0e720b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.753+0000 7fe0f6ffd640 1 -- 172.21.15.47:0/3356295077 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8001670 con 0x7fe0f80cf8a0 2024-03-23T09:51:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.753+0000 7fe0f6ffd640 1 -- 172.21.15.47:0/3356295077 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f002c930 con 0x7fe0f80cf8a0 2024-03-23T09:51:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.753+0000 7fe0f6ffd640 1 -- 172.21.15.47:0/3356295077 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f002cca0 con 0x7fe0f80cf8a0 2024-03-23T09:51:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.754+0000 7fe0fe884640 1 -- 172.21.15.47:0/3356295077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f80cf8a0 msgr2=0x7fe0f8167b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.754+0000 7fe0fe884640 1 --2- 172.21.15.47:0/3356295077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f80cf8a0 0x7fe0f8167b00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009dc0 tx=0x7fe0f002be00 comp rx=0 tx=0).stop 2024-03-23T09:51:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.754+0000 7fe0fe884640 1 -- 172.21.15.47:0/3356295077 shutdown_connections 2024-03-23T09:51:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.754+0000 7fe0fe884640 1 --2- 172.21.15.47:0/3356295077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f80cf8a0 0x7fe0f8167b00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.754+0000 7fe0fe884640 1 -- 172.21.15.47:0/3356295077 >> 172.21.15.47:0/3356295077 conn(0x7fe0f806b160 msgr2=0x7fe0f806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.754+0000 7fe0fe884640 1 -- 172.21.15.47:0/3356295077 shutdown_connections 2024-03-23T09:51:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.754+0000 7fe0fe884640 1 -- 172.21.15.47:0/3356295077 wait complete. 2024-03-23T09:51:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.755+0000 7fe0fe884640 1 Processor -- start 2024-03-23T09:51:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.755+0000 7fe0fe884640 1 -- start start 2024-03-23T09:51:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fe884640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f80b8300 0x7fe0f81f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fe884640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f80cf8a0 0x7fe0f81efb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fe884640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0f815cb40 0x7fe0f81f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fe884640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f81f6c00 con 0x7fe0f815cb40 2024-03-23T09:51:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fe884640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f81f2c60 con 0x7fe0f80b8300 2024-03-23T09:51:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fe884640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f81f2680 con 0x7fe0f80cf8a0 2024-03-23T09:51:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fce21640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0f815cb40 0x7fe0f81f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fce21640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0f815cb40 0x7fe0f81f5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36954/0 (socket says 172.21.15.47:36954) 2024-03-23T09:51:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fce21640 1 -- 172.21.15.47:0/1608361317 learned_addr learned my addr 172.21.15.47:0/1608361317 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0f7fff640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f80b8300 0x7fe0f81f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0f77fe640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f80cf8a0 0x7fe0f81efb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fce21640 1 -- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f80cf8a0 msgr2=0x7fe0f81efb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fce21640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f80cf8a0 0x7fe0f81efb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fce21640 1 -- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f80b8300 msgr2=0x7fe0f81f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fce21640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f80b8300 0x7fe0f81f30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0fce21640 1 -- 172.21.15.47:0/1608361317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f0009a70 con 0x7fe0f815cb40 2024-03-23T09:51:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.756+0000 7fe0f7fff640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f80b8300 0x7fe0f81f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.757+0000 7fe0fce21640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0f815cb40 0x7fe0f81f5cd0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fe0f002c410 tx=0x7fe0f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.757+0000 7fe0f4ff9640 1 -- 172.21.15.47:0/1608361317 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0007000 con 0x7fe0f815cb40 2024-03-23T09:51:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.757+0000 7fe0f4ff9640 1 -- 172.21.15.47:0/1608361317 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f0009450 con 0x7fe0f815cb40 2024-03-23T09:51:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.757+0000 7fe0fe884640 1 -- 172.21.15.47:0/1608361317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f81f3630 con 0x7fe0f815cb40 2024-03-23T09:51:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.757+0000 7fe0f4ff9640 1 -- 172.21.15.47:0/1608361317 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f003f420 con 0x7fe0f815cb40 2024-03-23T09:51:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.757+0000 7fe0fe884640 1 -- 172.21.15.47:0/1608361317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f81f6640 con 0x7fe0f815cb40 2024-03-23T09:51:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.758+0000 7fe0fe884640 1 -- 172.21.15.47:0/1608361317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f8072360 con 0x7fe0f815cb40 2024-03-23T09:51:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.758+0000 7fe0f4ff9640 1 -- 172.21.15.47:0/1608361317 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe0f003f580 con 0x7fe0f815cb40 2024-03-23T09:51:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.759+0000 7fe0f4ff9640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe0e406ac00 0x7fe0e406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.759+0000 7fe0f4ff9640 1 -- 172.21.15.47:0/1608361317 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe0f00abed0 con 0x7fe0f815cb40 2024-03-23T09:51:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.759+0000 7fe0f7fff640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe0e406ac00 0x7fe0e406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.759+0000 7fe0f7fff640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe0e406ac00 0x7fe0e406d0c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80011e0 tx=0x7fe0e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.763+0000 7fe0f4ff9640 1 -- 172.21.15.47:0/1608361317 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe0f0077190 con 0x7fe0f815cb40 2024-03-23T09:51:55.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.942+0000 7fe0fe884640 1 -- 172.21.15.47:0/1608361317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0f81f2e10 con 0x7fe0f815cb40 2024-03-23T09:51:55.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.943+0000 7fe0f4ff9640 1 -- 172.21.15.47:0/1608361317 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16213 (secure 0 0 0) 0x7fe0f81f2e10 con 0x7fe0f815cb40 2024-03-23T09:51:55.943 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:55.944 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":116,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:51:53.059965+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:51:38.679890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-23T09:51:39.396299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:51:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.945+0000 7fe0fe884640 1 -- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe0e406ac00 msgr2=0x7fe0e406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.945+0000 7fe0fe884640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe0e406ac00 0x7fe0e406d0c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80011e0 tx=0x7fe0e8009040 comp rx=0 tx=0).stop 2024-03-23T09:51:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.945+0000 7fe0fe884640 1 -- 172.21.15.47:0/1608361317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0f815cb40 msgr2=0x7fe0f81f5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.945+0000 7fe0fe884640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0f815cb40 0x7fe0f81f5cd0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fe0f002c410 tx=0x7fe0f0004a40 comp rx=0 tx=0).stop 2024-03-23T09:51:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.945+0000 7fe0fe884640 1 -- 172.21.15.47:0/1608361317 shutdown_connections 2024-03-23T09:51:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.945+0000 7fe0fe884640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe0e406ac00 0x7fe0e406d0c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.945+0000 7fe0fe884640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0f815cb40 0x7fe0f81f5cd0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.945+0000 7fe0fe884640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f80cf8a0 0x7fe0f81efb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.945+0000 7fe0fe884640 1 --2- 172.21.15.47:0/1608361317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f80b8300 0x7fe0f81f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.945+0000 7fe0fe884640 1 -- 172.21.15.47:0/1608361317 >> 172.21.15.47:0/1608361317 conn(0x7fe0f806b160 msgr2=0x7fe0f80c6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.945+0000 7fe0fe884640 1 -- 172.21.15.47:0/1608361317 shutdown_connections 2024-03-23T09:51:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:55.945+0000 7fe0fe884640 1 -- 172.21.15.47:0/1608361317 wait complete. 2024-03-23T09:51:55.956 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:51:38.679890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:51:39.396299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:51:55.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:51:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.061+0000 7f915fcbd640 1 Processor -- start 2024-03-23T09:51:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.062+0000 7f915fcbd640 1 -- start start 2024-03-23T09:51:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.062+0000 7f915fcbd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f915815cc40 0x7f91580b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.062+0000 7f915fcbd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91580ce9c0 con 0x7f91580cf9b0 2024-03-23T09:51:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.062+0000 7f915fcbd640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915806b5c0 con 0x7f915806b000 2024-03-23T09:51:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.062+0000 7f915fcbd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91580cebf0 con 0x7f915815cc40 2024-03-23T09:51:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.062+0000 7f915e25a640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f91580cf9b0 0x7f9158098a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56780/0 (socket says 172.21.15.47:56780) 2024-03-23T09:51:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.062+0000 7f915e25a640 1 -- 172.21.15.47:0/2289409058 learned_addr learned my addr 172.21.15.47:0/2289409058 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915da59640 1 --2- 172.21.15.47:0/2289409058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f915815cc40 0x7f91580b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314344777 0 0) 0x7f91580ce9c0 con 0x7f91580cf9b0 2024-03-23T09:51:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9140003520 con 0x7f91580cf9b0 2024-03-23T09:51:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2948051841 0 0) 0x7f9140003520 con 0x7f91580cf9b0 2024-03-23T09:51:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f915815cc40 msgr2=0x7f91580b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 --2- 172.21.15.47:0/2289409058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f915815cc40 0x7f91580b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 >> v1:172.21.15.175:6789/0 conn(0x7f915806b000 legacy=0x7f91580caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91580ceed0 con 0x7f91580cf9b0 2024-03-23T09:51:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f91480019f0 con 0x7f91580cf9b0 2024-03-23T09:51:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 >> v1:172.21.15.47:6789/0 conn(0x7f91580cf9b0 legacy=0x7f9158098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 --2- 172.21.15.47:0/2289409058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91400039a0 0x7f9140003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 --2- 172.21.15.47:0/2289409058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9140004350 0x7f9140004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 --2- 172.21.15.47:0/2289409058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f915815cc40 0x7f9140005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91580cebf0 con 0x7f91400039a0 2024-03-23T09:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914000b980 con 0x7f9140004350 2024-03-23T09:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914000bb40 con 0x7f915815cc40 2024-03-23T09:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915da59640 1 --2- 172.21.15.47:0/2289409058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91400039a0 0x7f9140003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915e25a640 1 --2- 172.21.15.47:0/2289409058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f915815cc40 0x7f9140005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915d258640 1 --2- 172.21.15.47:0/2289409058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9140004350 0x7f9140004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915da59640 1 -- 172.21.15.47:0/2289409058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f915815cc40 msgr2=0x7f9140005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915da59640 1 --2- 172.21.15.47:0/2289409058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f915815cc40 0x7f9140005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915da59640 1 -- 172.21.15.47:0/2289409058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9140004350 msgr2=0x7f9140004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915da59640 1 --2- 172.21.15.47:0/2289409058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9140004350 0x7f9140004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.063+0000 7f915da59640 1 -- 172.21.15.47:0/2289409058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9140003520 con 0x7f91400039a0 2024-03-23T09:51:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.064+0000 7f915da59640 1 --2- 172.21.15.47:0/2289409058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91400039a0 0x7f9140003d80 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f9150009ac0 tx=0x7f91400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3d38093162941c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.064+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915002c6b0 con 0x7f91400039a0 2024-03-23T09:51:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.064+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915002c810 con 0x7f91400039a0 2024-03-23T09:51:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.064+0000 7f915ca57640 1 -- 172.21.15.47:0/2289409058 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915002cba0 con 0x7f91400039a0 2024-03-23T09:51:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.065+0000 7f915fcbd640 1 -- 172.21.15.47:0/2289409058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91400039a0 msgr2=0x7f9140003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.065+0000 7f915fcbd640 1 --2- 172.21.15.47:0/2289409058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91400039a0 0x7f9140003d80 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f9150009ac0 tx=0x7f91400057f0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.065+0000 7f915e25a640 1 -- 172.21.15.47:0/2289409058 reap_dead start 2024-03-23T09:51:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.065+0000 7f915fcbd640 1 -- 172.21.15.47:0/2289409058 shutdown_connections 2024-03-23T09:51:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.065+0000 7f915fcbd640 1 -- 172.21.15.47:0/2289409058 >> 172.21.15.47:0/2289409058 conn(0x7f91580b2140 msgr2=0x7f91580b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.065+0000 7f915fcbd640 1 -- 172.21.15.47:0/2289409058 shutdown_connections 2024-03-23T09:51:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.065+0000 7f915fcbd640 1 -- 172.21.15.47:0/2289409058 wait complete. 2024-03-23T09:51:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.066+0000 7f915fcbd640 1 Processor -- start 2024-03-23T09:51:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.066+0000 7f915fcbd640 1 -- start start 2024-03-23T09:51:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.067+0000 7f915fcbd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91400039a0 0x7f91580c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.067+0000 7f915fcbd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91581564b0 0x7f91580c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.067+0000 7f915fcbd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91580cb530 0x7f91580cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915fcbd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915815d020 con 0x7f91580cb530 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915fcbd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91580cbef0 con 0x7f91581564b0 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915fcbd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9158156930 con 0x7f91400039a0 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915e25a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91580cb530 0x7f91580cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915e25a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91580cb530 0x7f91580cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36986/0 (socket says 172.21.15.47:36986) 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915e25a640 1 -- 172.21.15.47:0/1456671169 learned_addr learned my addr 172.21.15.47:0/1456671169 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915da59640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91400039a0 0x7f91580c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915d258640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91581564b0 0x7f91580c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915e25a640 1 -- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91400039a0 msgr2=0x7f91580c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915e25a640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91400039a0 0x7f91580c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915e25a640 1 -- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91581564b0 msgr2=0x7f91580c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915e25a640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91581564b0 0x7f91580c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915e25a640 1 -- 172.21.15.47:0/1456671169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91580c5b30 con 0x7f91580cb530 2024-03-23T09:51:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915e25a640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91580cb530 0x7f91580cb9b0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f9158071b40 tx=0x7f914800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f91567fc640 1 -- 172.21.15.47:0/1456671169 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914800ec10 con 0x7f91580cb530 2024-03-23T09:51:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915fcbd640 1 -- 172.21.15.47:0/1456671169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91580c5e10 con 0x7f91580cb530 2024-03-23T09:51:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f91567fc640 1 -- 172.21.15.47:0/1456671169 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914800ed90 con 0x7f91580cb530 2024-03-23T09:51:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f91567fc640 1 -- 172.21.15.47:0/1456671169 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148013660 con 0x7f91580cb530 2024-03-23T09:51:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.068+0000 7f915fcbd640 1 -- 172.21.15.47:0/1456671169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915815d270 con 0x7f91580cb530 2024-03-23T09:51:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.069+0000 7f915fcbd640 1 -- 172.21.15.47:0/1456671169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9158066be0 con 0x7f91580cb530 2024-03-23T09:51:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.070+0000 7f91567fc640 1 -- 172.21.15.47:0/1456671169 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f91480137c0 con 0x7f91580cb530 2024-03-23T09:51:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.070+0000 7f91567fc640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f914406ac30 0x7f914406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.070+0000 7f91567fc640 1 -- 172.21.15.47:0/1456671169 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f91480897e0 con 0x7f91580cb530 2024-03-23T09:51:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.070+0000 7f915da59640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f914406ac30 0x7f914406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.071+0000 7f915da59640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f914406ac30 0x7f914406d0f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9150009ac0 tx=0x7f9150030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:56.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.074+0000 7f91567fc640 1 -- 172.21.15.47:0/1456671169 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9148054990 con 0x7f91580cb530 2024-03-23T09:51:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.255+0000 7f915fcbd640 1 -- 172.21.15.47:0/1456671169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91580b2fe0 con 0x7f91580cb530 2024-03-23T09:51:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.255+0000 7f91567fc640 1 -- 172.21.15.47:0/1456671169 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9148057fb0 con 0x7f91580cb530 2024-03-23T09:51:56.257 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:51:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.257+0000 7f915fcbd640 1 -- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f914406ac30 msgr2=0x7f914406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.257+0000 7f915fcbd640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f914406ac30 0x7f914406d0f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9150009ac0 tx=0x7f9150030000 comp rx=0 tx=0).stop 2024-03-23T09:51:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.257+0000 7f915fcbd640 1 -- 172.21.15.47:0/1456671169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91580cb530 msgr2=0x7f91580cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.257+0000 7f915fcbd640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91580cb530 0x7f91580cb9b0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f9158071b40 tx=0x7f914800e530 comp rx=0 tx=0).stop 2024-03-23T09:51:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.257+0000 7f915fcbd640 1 -- 172.21.15.47:0/1456671169 shutdown_connections 2024-03-23T09:51:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.257+0000 7f915fcbd640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f914406ac30 0x7f914406d0f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.257+0000 7f915fcbd640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91580cb530 0x7f91580cb9b0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.257+0000 7f915fcbd640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91581564b0 0x7f91580c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.257+0000 7f915fcbd640 1 --2- 172.21.15.47:0/1456671169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91400039a0 0x7f91580c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.257+0000 7f915fcbd640 1 -- 172.21.15.47:0/1456671169 >> 172.21.15.47:0/1456671169 conn(0x7f91580b2140 msgr2=0x7f915806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.257+0000 7f915fcbd640 1 -- 172.21.15.47:0/1456671169 shutdown_connections 2024-03-23T09:51:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.257+0000 7f915fcbd640 1 -- 172.21.15.47:0/1456671169 wait complete. 2024-03-23T09:51:56.268 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:51:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.371+0000 7f517324a640 1 Processor -- start 2024-03-23T09:51:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.371+0000 7f517324a640 1 -- start start 2024-03-23T09:51:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.372+0000 7f517324a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f516c15ce10 0x7f516c0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.372+0000 7f517324a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516c161600 con 0x7f516c15ca30 2024-03-23T09:51:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.372+0000 7f517324a640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516c161040 con 0x7f516c0b85d0 2024-03-23T09:51:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.372+0000 7f517324a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516c160e50 con 0x7f516c15ce10 2024-03-23T09:51:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.372+0000 7f5170fe6640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f516c15ca30 0x7f516c160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56794/0 (socket says 172.21.15.47:56794) 2024-03-23T09:51:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.372+0000 7f5170fe6640 1 -- 172.21.15.47:0/498034022 learned_addr learned my addr 172.21.15.47:0/498034022 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.373+0000 7f516bfff640 1 --2- 172.21.15.47:0/498034022 >> v2:172.21.15.175:3300/0 conn(0x7f516c15ce10 0x7f516c0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.373+0000 7f516b7fe640 1 -- 172.21.15.47:0/498034022 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168002790 0 0) 0x7f516c161600 con 0x7f516c15ca30 2024-03-23T09:51:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.373+0000 7f516b7fe640 1 -- 172.21.15.47:0/498034022 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5154003520 con 0x7f516c15ca30 2024-03-23T09:51:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.373+0000 7f516bfff640 1 -- 172.21.15.47:0/498034022 >> v1:172.21.15.175:6789/0 conn(0x7f516c0b85d0 legacy=0x7f516c0d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.373+0000 7f516bfff640 1 -- 172.21.15.47:0/498034022 >> v1:172.21.15.47:6789/0 conn(0x7f516c15ca30 legacy=0x7f516c160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.373+0000 7f516bfff640 1 -- 172.21.15.47:0/498034022 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c161320 con 0x7f516c15ce10 2024-03-23T09:51:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.374+0000 7f516bfff640 1 --2- 172.21.15.47:0/498034022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c15ce10 0x7f516c0cd730 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f51600097a0 tx=0x7f516002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10c2eac196793347 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.374+0000 7f516b7fe640 1 -- 172.21.15.47:0/498034022 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160030610 con 0x7f516c15ce10 2024-03-23T09:51:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.374+0000 7f516b7fe640 1 -- 172.21.15.47:0/498034022 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5160030770 con 0x7f516c15ce10 2024-03-23T09:51:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.374+0000 7f516b7fe640 1 -- 172.21.15.47:0/498034022 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160030b00 con 0x7f516c15ce10 2024-03-23T09:51:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.374+0000 7f517324a640 1 -- 172.21.15.47:0/498034022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c15ce10 msgr2=0x7f516c0cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.374+0000 7f517324a640 1 --2- 172.21.15.47:0/498034022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c15ce10 0x7f516c0cd730 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f51600097a0 tx=0x7f516002b960 comp rx=0 tx=0).stop 2024-03-23T09:51:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.374+0000 7f517324a640 1 -- 172.21.15.47:0/498034022 shutdown_connections 2024-03-23T09:51:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.374+0000 7f517324a640 1 --2- 172.21.15.47:0/498034022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c15ce10 0x7f516c0cd730 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.374+0000 7f517324a640 1 -- 172.21.15.47:0/498034022 >> 172.21.15.47:0/498034022 conn(0x7f516c06b160 msgr2=0x7f516c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.375+0000 7f517324a640 1 -- 172.21.15.47:0/498034022 shutdown_connections 2024-03-23T09:51:56.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.375+0000 7f517324a640 1 -- 172.21.15.47:0/498034022 wait complete. 2024-03-23T09:51:56.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.376+0000 7f517324a640 1 Processor -- start 2024-03-23T09:51:56.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.376+0000 7f517324a640 1 -- start start 2024-03-23T09:51:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.376+0000 7f517324a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c0b85d0 0x7f516c0b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.376+0000 7f517324a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c15ca30 0x7f516c158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.376+0000 7f517324a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c15ce10 0x7f516c147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.376+0000 7f517324a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0b7b40 con 0x7f516c15ca30 2024-03-23T09:51:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.376+0000 7f517324a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0b7e20 con 0x7f516c15ce10 2024-03-23T09:51:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.376+0000 7f517324a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0b8100 con 0x7f516c0b85d0 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f516bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c15ca30 0x7f516c158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f516bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c15ca30 0x7f516c158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37000/0 (socket says 172.21.15.47:37000) 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f516bfff640 1 -- 172.21.15.47:0/3939621198 learned_addr learned my addr 172.21.15.47:0/3939621198 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f5170fe6640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c0b85d0 0x7f516c0b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f51717e7640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c15ce10 0x7f516c147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f516bfff640 1 -- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c0b85d0 msgr2=0x7f516c0b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f516bfff640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c0b85d0 0x7f516c0b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f516bfff640 1 -- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c15ce10 msgr2=0x7f516c147690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f516bfff640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c15ce10 0x7f516c147690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f516bfff640 1 -- 172.21.15.47:0/3939621198 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160009400 con 0x7f516c15ca30 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f5170fe6640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c0b85d0 0x7f516c0b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f516bfff640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c15ca30 0x7f516c158050 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f516002bf70 tx=0x7f5160004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f51697fa640 1 -- 172.21.15.47:0/3939621198 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516003e040 con 0x7f516c15ca30 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f51697fa640 1 -- 172.21.15.47:0/3939621198 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5160039cb0 con 0x7f516c15ca30 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f51697fa640 1 -- 172.21.15.47:0/3939621198 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160044630 con 0x7f516c15ca30 2024-03-23T09:51:56.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f517324a640 1 -- 172.21.15.47:0/3939621198 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516c0b68a0 con 0x7f516c15ca30 2024-03-23T09:51:56.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.377+0000 7f517324a640 1 -- 172.21.15.47:0/3939621198 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c0b6e60 con 0x7f516c15ca30 2024-03-23T09:51:56.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.378+0000 7f517324a640 1 -- 172.21.15.47:0/3939621198 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516c072360 con 0x7f516c15ca30 2024-03-23T09:51:56.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.379+0000 7f51697fa640 1 -- 172.21.15.47:0/3939621198 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5160043390 con 0x7f516c15ca30 2024-03-23T09:51:56.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.379+0000 7f51697fa640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515806ac30 0x7f515806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.379+0000 7f51697fa640 1 -- 172.21.15.47:0/3939621198 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f51600ab5b0 con 0x7f516c15ca30 2024-03-23T09:51:56.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.379+0000 7f5170fe6640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515806ac30 0x7f515806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.380+0000 7f5170fe6640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515806ac30 0x7f515806d0f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5164001560 tx=0x7f5164009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:56.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.383+0000 7f51697fa640 1 -- 172.21.15.47:0/3939621198 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f51600766f0 con 0x7f516c15ca30 2024-03-23T09:51:56.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.563+0000 7f517324a640 1 -- 172.21.15.47:0/3939621198 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f516c066be0 con 0x7f516c15ca30 2024-03-23T09:51:56.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.564+0000 7f51697fa640 1 -- 172.21.15.47:0/3939621198 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5160079d10 con 0x7f516c15ca30 2024-03-23T09:51:56.564 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:51:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.566+0000 7f517324a640 1 -- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515806ac30 msgr2=0x7f515806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.566+0000 7f517324a640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515806ac30 0x7f515806d0f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5164001560 tx=0x7f5164009040 comp rx=0 tx=0).stop 2024-03-23T09:51:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.566+0000 7f517324a640 1 -- 172.21.15.47:0/3939621198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c15ca30 msgr2=0x7f516c158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.566+0000 7f517324a640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c15ca30 0x7f516c158050 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f516002bf70 tx=0x7f5160004730 comp rx=0 tx=0).stop 2024-03-23T09:51:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.566+0000 7f517324a640 1 -- 172.21.15.47:0/3939621198 shutdown_connections 2024-03-23T09:51:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.566+0000 7f517324a640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f515806ac30 0x7f515806d0f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.566+0000 7f517324a640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f516c15ce10 0x7f516c147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.566+0000 7f517324a640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f516c15ca30 0x7f516c158050 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.566+0000 7f517324a640 1 --2- 172.21.15.47:0/3939621198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f516c0b85d0 0x7f516c0b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.566+0000 7f517324a640 1 -- 172.21.15.47:0/3939621198 >> 172.21.15.47:0/3939621198 conn(0x7f516c06b160 msgr2=0x7f516c15a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.566+0000 7f517324a640 1 -- 172.21.15.47:0/3939621198 shutdown_connections 2024-03-23T09:51:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.566+0000 7f517324a640 1 -- 172.21.15.47:0/3939621198 wait complete. 2024-03-23T09:51:56.577 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:51:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.680+0000 7f458b95c640 1 Processor -- start 2024-03-23T09:51:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.680+0000 7f458b95c640 1 -- start start 2024-03-23T09:51:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.681+0000 7f458b95c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f45840b8350 0x7f4584160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.681+0000 7f458b95c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4584161630 con 0x7f458415c990 2024-03-23T09:51:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.681+0000 7f458b95c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45840d1030 con 0x7f45840cde10 2024-03-23T09:51:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.681+0000 7f458b95c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45840cfe10 con 0x7f45840b8350 2024-03-23T09:51:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.681+0000 7f45896f8640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f458415c990 0x7f45840cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56804/0 (socket says 172.21.15.47:56804) 2024-03-23T09:51:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.681+0000 7f45896f8640 1 -- 172.21.15.47:0/4142678183 learned_addr learned my addr 172.21.15.47:0/4142678183 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.681+0000 7f4589ef9640 1 --2- 172.21.15.47:0/4142678183 >> v2:172.21.15.175:3300/0 conn(0x7f45840b8350 0x7f4584160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.681+0000 7f4583fff640 1 -- 172.21.15.47:0/4142678183 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668406355 0 0) 0x7f4584161630 con 0x7f458415c990 2024-03-23T09:51:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.681+0000 7f4583fff640 1 -- 172.21.15.47:0/4142678183 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f456c003520 con 0x7f458415c990 2024-03-23T09:51:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.681+0000 7f4589ef9640 1 -- 172.21.15.47:0/4142678183 >> v1:172.21.15.175:6789/0 conn(0x7f45840cde10 legacy=0x7f45840d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.681+0000 7f4589ef9640 1 -- 172.21.15.47:0/4142678183 >> v1:172.21.15.47:6789/0 conn(0x7f458415c990 legacy=0x7f45840cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.682+0000 7f4583fff640 1 -- 172.21.15.47:0/4142678183 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 872601770 0 0) 0x7f456c003520 con 0x7f458415c990 2024-03-23T09:51:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.682+0000 7f4589ef9640 1 -- 172.21.15.47:0/4142678183 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45840b5b60 con 0x7f45840b8350 2024-03-23T09:51:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.682+0000 7f4589ef9640 1 --2- 172.21.15.47:0/4142678183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45840b8350 0x7f4584160690 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f457c0097a0 tx=0x7f457c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a034ea5de33c490c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.683+0000 7f4583fff640 1 -- 172.21.15.47:0/4142678183 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45740019f0 con 0x7f45840b8350 2024-03-23T09:51:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.683+0000 7f4583fff640 1 -- 172.21.15.47:0/4142678183 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457c02e050 con 0x7f45840b8350 2024-03-23T09:51:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.683+0000 7f4583fff640 1 -- 172.21.15.47:0/4142678183 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c030730 con 0x7f45840b8350 2024-03-23T09:51:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.683+0000 7f458b95c640 1 -- 172.21.15.47:0/4142678183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45840b8350 msgr2=0x7f4584160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.683+0000 7f458b95c640 1 --2- 172.21.15.47:0/4142678183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45840b8350 0x7f4584160690 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f457c0097a0 tx=0x7f457c02b790 comp rx=0 tx=0).stop 2024-03-23T09:51:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.683+0000 7f458b95c640 1 -- 172.21.15.47:0/4142678183 shutdown_connections 2024-03-23T09:51:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.683+0000 7f458b95c640 1 --2- 172.21.15.47:0/4142678183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45840b8350 0x7f4584160690 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.683+0000 7f458b95c640 1 -- 172.21.15.47:0/4142678183 >> 172.21.15.47:0/4142678183 conn(0x7f45840659d0 msgr2=0x7f4584065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.683+0000 7f458b95c640 1 -- 172.21.15.47:0/4142678183 shutdown_connections 2024-03-23T09:51:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.683+0000 7f458b95c640 1 -- 172.21.15.47:0/4142678183 wait complete. 2024-03-23T09:51:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.684+0000 7f458b95c640 1 Processor -- start 2024-03-23T09:51:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.684+0000 7f458b95c640 1 -- start start 2024-03-23T09:51:56.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f458b95c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45840b8350 0x7f4584155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f458b95c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45840cde10 0x7f45840cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f458b95c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f458415c990 0x7f4584153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f458b95c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4584159870 con 0x7f45840b8350 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f458b95c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4584159b50 con 0x7f45840cde10 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f458b95c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4584159e30 con 0x7f458415c990 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f45896f8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45840b8350 0x7f4584155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f45896f8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45840b8350 0x7f4584155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37014/0 (socket says 172.21.15.47:37014) 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f45896f8640 1 -- 172.21.15.47:0/2515148410 learned_addr learned my addr 172.21.15.47:0/2515148410 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f4588ef7640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45840cde10 0x7f45840cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f4589ef9640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f458415c990 0x7f4584153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f45896f8640 1 -- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f458415c990 msgr2=0x7f4584153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f45896f8640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f458415c990 0x7f4584153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f45896f8640 1 -- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45840cde10 msgr2=0x7f45840cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f45896f8640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45840cde10 0x7f45840cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.685+0000 7f45896f8640 1 -- 172.21.15.47:0/2515148410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c009400 con 0x7f45840b8350 2024-03-23T09:51:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.686+0000 7f45896f8640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45840b8350 0x7f4584155dd0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f45740018b0 tx=0x7f457400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:56.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.686+0000 7f4581ffb640 1 -- 172.21.15.47:0/2515148410 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574012400 con 0x7f45840b8350 2024-03-23T09:51:56.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.686+0000 7f4581ffb640 1 -- 172.21.15.47:0/2515148410 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4574012560 con 0x7f45840b8350 2024-03-23T09:51:56.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.686+0000 7f458b95c640 1 -- 172.21.15.47:0/2515148410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458415a110 con 0x7f45840b8350 2024-03-23T09:51:56.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.686+0000 7f4581ffb640 1 -- 172.21.15.47:0/2515148410 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574012990 con 0x7f45840b8350 2024-03-23T09:51:56.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.686+0000 7f458b95c640 1 -- 172.21.15.47:0/2515148410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4584159500 con 0x7f45840b8350 2024-03-23T09:51:56.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.687+0000 7f458b95c640 1 -- 172.21.15.47:0/2515148410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4584072360 con 0x7f45840b8350 2024-03-23T09:51:56.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.688+0000 7f4581ffb640 1 -- 172.21.15.47:0/2515148410 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4574007070 con 0x7f45840b8350 2024-03-23T09:51:56.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.688+0000 7f4581ffb640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f457006ac00 0x7f457006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.688+0000 7f4581ffb640 1 -- 172.21.15.47:0/2515148410 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f45740895e0 con 0x7f45840b8350 2024-03-23T09:51:56.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.688+0000 7f4588ef7640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f457006ac00 0x7f457006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.689+0000 7f4588ef7640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f457006ac00 0x7f457006d0c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f45840cf080 tx=0x7f4578009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:56.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.692+0000 7f4581ffb640 1 -- 172.21.15.47:0/2515148410 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4574054780 con 0x7f45840b8350 2024-03-23T09:51:56.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.873+0000 7f458b95c640 1 -- 172.21.15.47:0/2515148410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4584066be0 con 0x7f45840b8350 2024-03-23T09:51:56.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.873+0000 7f4581ffb640 1 -- 172.21.15.47:0/2515148410 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4574057da0 con 0x7f45840b8350 2024-03-23T09:51:56.874 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:51:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.876+0000 7f458b95c640 1 -- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f457006ac00 msgr2=0x7f457006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.876+0000 7f458b95c640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f457006ac00 0x7f457006d0c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f45840cf080 tx=0x7f4578009040 comp rx=0 tx=0).stop 2024-03-23T09:51:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.876+0000 7f458b95c640 1 -- 172.21.15.47:0/2515148410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45840b8350 msgr2=0x7f4584155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.876+0000 7f458b95c640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45840b8350 0x7f4584155dd0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f45740018b0 tx=0x7f457400a430 comp rx=0 tx=0).stop 2024-03-23T09:51:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.876+0000 7f458b95c640 1 -- 172.21.15.47:0/2515148410 shutdown_connections 2024-03-23T09:51:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.876+0000 7f458b95c640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f457006ac00 0x7f457006d0c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.876+0000 7f458b95c640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f458415c990 0x7f4584153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.876+0000 7f458b95c640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45840cde10 0x7f45840cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.876+0000 7f458b95c640 1 --2- 172.21.15.47:0/2515148410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45840b8350 0x7f4584155dd0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.876+0000 7f458b95c640 1 -- 172.21.15.47:0/2515148410 >> 172.21.15.47:0/2515148410 conn(0x7f45840659d0 msgr2=0x7f45840b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.876+0000 7f458b95c640 1 -- 172.21.15.47:0/2515148410 shutdown_connections 2024-03-23T09:51:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.876+0000 7f458b95c640 1 -- 172.21.15.47:0/2515148410 wait complete. 2024-03-23T09:51:56.887 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:51:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.988+0000 7f2893a26640 1 Processor -- start 2024-03-23T09:51:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.988+0000 7f2893a26640 1 -- start start 2024-03-23T09:51:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.989+0000 7f2893a26640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f288c15d0a0 0x7f288c0d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.989+0000 7f2893a26640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f288c0ce090 con 0x7f288c15ccc0 2024-03-23T09:51:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.989+0000 7f2893a26640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f288c0d0f00 con 0x7f288c06b210 2024-03-23T09:51:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.989+0000 7f2893a26640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f288c06c0e0 con 0x7f288c15d0a0 2024-03-23T09:51:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.989+0000 7f2890fc1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f288c15d0a0 0x7f288c0d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.989+0000 7f28917c2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f288c15ccc0 0x7f288c0cd980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56816/0 (socket says 172.21.15.47:56816) 2024-03-23T09:51:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.989+0000 7f28917c2640 1 -- 172.21.15.47:0/4115125958 learned_addr learned my addr 172.21.15.47:0/4115125958 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:51:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.989+0000 7f2890fc1640 1 -- 172.21.15.47:0/4115125958 >> v1:172.21.15.175:6789/0 conn(0x7f288c06b210 legacy=0x7f288c0b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.989+0000 7f2890fc1640 1 -- 172.21.15.47:0/4115125958 >> v1:172.21.15.47:6789/0 conn(0x7f288c15ccc0 legacy=0x7f288c0cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.989+0000 7f2890fc1640 1 -- 172.21.15.47:0/4115125958 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c0ce2d0 con 0x7f288c15d0a0 2024-03-23T09:51:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.989+0000 7f2883fff640 1 -- 172.21.15.47:0/4115125958 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 664757823 0 0) 0x7f2888001110 con 0x7f288c15ccc0 2024-03-23T09:51:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.990+0000 7f2890fc1640 1 --2- 172.21.15.47:0/4115125958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f288c15d0a0 0x7f288c0d1c30 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f287c0097a0 tx=0x7f287c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42e79773d9abd97 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.990+0000 7f2883fff640 1 -- 172.21.15.47:0/4115125958 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c030610 con 0x7f288c15d0a0 2024-03-23T09:51:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.990+0000 7f2883fff640 1 -- 172.21.15.47:0/4115125958 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c030770 con 0x7f288c15d0a0 2024-03-23T09:51:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.990+0000 7f2883fff640 1 -- 172.21.15.47:0/4115125958 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c030b00 con 0x7f288c15d0a0 2024-03-23T09:51:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.991+0000 7f2893a26640 1 -- 172.21.15.47:0/4115125958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f288c15d0a0 msgr2=0x7f288c0d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.991+0000 7f2893a26640 1 --2- 172.21.15.47:0/4115125958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f288c15d0a0 0x7f288c0d1c30 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f287c0097a0 tx=0x7f287c02b960 comp rx=0 tx=0).stop 2024-03-23T09:51:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.991+0000 7f2893a26640 1 -- 172.21.15.47:0/4115125958 shutdown_connections 2024-03-23T09:51:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.991+0000 7f2893a26640 1 --2- 172.21.15.47:0/4115125958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f288c15d0a0 0x7f288c0d1c30 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.991+0000 7f2893a26640 1 -- 172.21.15.47:0/4115125958 >> 172.21.15.47:0/4115125958 conn(0x7f288c0cf430 msgr2=0x7f288c0cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.991+0000 7f2893a26640 1 -- 172.21.15.47:0/4115125958 shutdown_connections 2024-03-23T09:51:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.991+0000 7f2893a26640 1 -- 172.21.15.47:0/4115125958 wait complete. 2024-03-23T09:51:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.992+0000 7f2893a26640 1 Processor -- start 2024-03-23T09:51:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.992+0000 7f2893a26640 1 -- start start 2024-03-23T09:51:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.992+0000 7f2893a26640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f288c06b210 0x7f288c0b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.992+0000 7f2893a26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f288c15ccc0 0x7f288c0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.992+0000 7f2893a26640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f288c15d0a0 0x7f288c0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2893a26640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c1455c0 con 0x7f288c15ccc0 2024-03-23T09:51:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2893a26640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c1452e0 con 0x7f288c06b210 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2893a26640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288c1f5a20 con 0x7f288c15d0a0 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2890fc1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f288c15ccc0 0x7f288c0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2890fc1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f288c15ccc0 0x7f288c0cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37038/0 (socket says 172.21.15.47:37038) 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2890fc1640 1 -- 172.21.15.47:0/2364205145 learned_addr learned my addr 172.21.15.47:0/2364205145 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f28917c2640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f288c06b210 0x7f288c0b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2891fc3640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f288c15d0a0 0x7f288c0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2890fc1640 1 -- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f288c15d0a0 msgr2=0x7f288c0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2890fc1640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f288c15d0a0 0x7f288c0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2890fc1640 1 -- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f288c06b210 msgr2=0x7f288c0b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2890fc1640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f288c06b210 0x7f288c0b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2890fc1640 1 -- 172.21.15.47:0/2364205145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c009400 con 0x7f288c15ccc0 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f28917c2640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f288c06b210 0x7f288c0b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2890fc1640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f288c15ccc0 0x7f288c0cac70 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f287c0098c0 tx=0x7f287c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2881ffb640 1 -- 172.21.15.47:0/2364205145 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c039c10 con 0x7f288c15ccc0 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2881ffb640 1 -- 172.21.15.47:0/2364205145 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c03f040 con 0x7f288c15ccc0 2024-03-23T09:51:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2893a26640 1 -- 172.21.15.47:0/2364205145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288c1f5d00 con 0x7f288c15ccc0 2024-03-23T09:51:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.993+0000 7f2881ffb640 1 -- 172.21.15.47:0/2364205145 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c0433f0 con 0x7f288c15ccc0 2024-03-23T09:51:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.994+0000 7f2893a26640 1 -- 172.21.15.47:0/2364205145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288c0b4e20 con 0x7f288c15ccc0 2024-03-23T09:51:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.995+0000 7f2893a26640 1 -- 172.21.15.47:0/2364205145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288c072360 con 0x7f288c15ccc0 2024-03-23T09:51:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.995+0000 7f2881ffb640 1 -- 172.21.15.47:0/2364205145 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f287c043550 con 0x7f288c15ccc0 2024-03-23T09:51:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.995+0000 7f2881ffb640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f287006ac00 0x7f287006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.995+0000 7f2881ffb640 1 -- 172.21.15.47:0/2364205145 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f287c0abeb0 con 0x7f288c15ccc0 2024-03-23T09:51:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.996+0000 7f28917c2640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f287006ac00 0x7f287006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.996+0000 7f28917c2640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f287006ac00 0x7f287006d0c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2888000f60 tx=0x7f2888007a50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:56.999+0000 7f2881ffb640 1 -- 172.21.15.47:0/2364205145 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f287c077050 con 0x7f288c15ccc0 2024-03-23T09:51:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.194+0000 7f2893a26640 1 -- 172.21.15.47:0/2364205145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f288c1f5880 con 0x7f288c15ccc0 2024-03-23T09:51:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.194+0000 7f2881ffb640 1 -- 172.21.15.47:0/2364205145 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v116) v1 ==== 80+0+0 (secure 0 0 0) 0x7f288c1f5880 con 0x7f288c15ccc0 2024-03-23T09:51:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:51:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.196+0000 7f2893a26640 1 -- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f287006ac00 msgr2=0x7f287006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.196+0000 7f2893a26640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f287006ac00 0x7f287006d0c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2888000f60 tx=0x7f2888007a50 comp rx=0 tx=0).stop 2024-03-23T09:51:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.196+0000 7f2893a26640 1 -- 172.21.15.47:0/2364205145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f288c15ccc0 msgr2=0x7f288c0cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.196+0000 7f2893a26640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f288c15ccc0 0x7f288c0cac70 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f287c0098c0 tx=0x7f287c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.196+0000 7f2893a26640 1 -- 172.21.15.47:0/2364205145 shutdown_connections 2024-03-23T09:51:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.196+0000 7f2893a26640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f287006ac00 0x7f287006d0c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.196+0000 7f2893a26640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f288c15d0a0 0x7f288c0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.196+0000 7f2893a26640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f288c15ccc0 0x7f288c0cac70 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.196+0000 7f2893a26640 1 --2- 172.21.15.47:0/2364205145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f288c06b210 0x7f288c0b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.196+0000 7f2893a26640 1 -- 172.21.15.47:0/2364205145 >> 172.21.15.47:0/2364205145 conn(0x7f288c0cf430 msgr2=0x7f288c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:57.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.196+0000 7f2893a26640 1 -- 172.21.15.47:0/2364205145 shutdown_connections 2024-03-23T09:51:57.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.196+0000 7f2893a26640 1 -- 172.21.15.47:0/2364205145 wait complete. 2024-03-23T09:51:57.208 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:51:57.208 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:51:57.223 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:51:57.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.364+0000 7fad33d4e640 1 Processor -- start 2024-03-23T09:51:57.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.364+0000 7fad33d4e640 1 -- start start 2024-03-23T09:51:57.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad33d4e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad2c15ca90 0x7fad2c160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad33d4e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fad2c0b8350 0x7fad2c0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad33d4e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fad2c0cdc40 0x7fad2c0b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad33d4e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2c0ce0d0 con 0x7fad2c0cdc40 2024-03-23T09:51:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad33d4e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2c0d11e0 con 0x7fad2c0b8350 2024-03-23T09:51:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad33d4e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c160f60 con 0x7fad2c15ca90 2024-03-23T09:51:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad322eb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fad2c0cdc40 0x7fad2c0b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad322eb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fad2c0cdc40 0x7fad2c0b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37042/0 (socket says 172.21.15.47:37042) 2024-03-23T09:51:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad322eb640 1 -- 172.21.15.47:0/2575559899 learned_addr learned my addr 172.21.15.47:0/2575559899 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad312e9640 1 --2- 172.21.15.47:0/2575559899 >> v2:172.21.15.175:3300/0 conn(0x7fad2c0b8350 0x7fad2c0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad31aea640 1 --2- 172.21.15.47:0/2575559899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad2c15ca90 0x7fad2c160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad322eb640 1 -- 172.21.15.47:0/2575559899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad2c15ca90 msgr2=0x7fad2c160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad322eb640 1 --2- 172.21.15.47:0/2575559899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad2c15ca90 0x7fad2c160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad322eb640 1 -- 172.21.15.47:0/2575559899 >> v2:172.21.15.175:3300/0 conn(0x7fad2c0b8350 msgr2=0x7fad2c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad322eb640 1 --2- 172.21.15.47:0/2575559899 >> v2:172.21.15.175:3300/0 conn(0x7fad2c0b8350 0x7fad2c0cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.365+0000 7fad322eb640 1 -- 172.21.15.47:0/2575559899 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c161240 con 0x7fad2c0cdc40 2024-03-23T09:51:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.366+0000 7fad322eb640 1 --2- 172.21.15.47:0/2575559899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2c0cdc40 0x7fad2c0b4440 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0b5540 tx=0x7fad2802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a0f3d5e550bfa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.366+0000 7fad30ae8640 1 -- 172.21.15.47:0/2575559899 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad280317e0 con 0x7fad2c0cdc40 2024-03-23T09:51:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.366+0000 7fad30ae8640 1 -- 172.21.15.47:0/2575559899 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad28031940 con 0x7fad2c0cdc40 2024-03-23T09:51:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.366+0000 7fad30ae8640 1 -- 172.21.15.47:0/2575559899 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28031cd0 con 0x7fad2c0cdc40 2024-03-23T09:51:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.367+0000 7fad33d4e640 1 -- 172.21.15.47:0/2575559899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2c0cdc40 msgr2=0x7fad2c0b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.367+0000 7fad33d4e640 1 --2- 172.21.15.47:0/2575559899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2c0cdc40 0x7fad2c0b4440 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0b5540 tx=0x7fad2802b880 comp rx=0 tx=0).stop 2024-03-23T09:51:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.367+0000 7fad33d4e640 1 -- 172.21.15.47:0/2575559899 shutdown_connections 2024-03-23T09:51:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.367+0000 7fad33d4e640 1 --2- 172.21.15.47:0/2575559899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2c0cdc40 0x7fad2c0b4440 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.367+0000 7fad33d4e640 1 --2- 172.21.15.47:0/2575559899 >> v2:172.21.15.175:3300/0 conn(0x7fad2c0b8350 0x7fad2c0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.367+0000 7fad33d4e640 1 --2- 172.21.15.47:0/2575559899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad2c15ca90 0x7fad2c160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.367+0000 7fad33d4e640 1 -- 172.21.15.47:0/2575559899 >> 172.21.15.47:0/2575559899 conn(0x7fad2c0659d0 msgr2=0x7fad2c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.367+0000 7fad33d4e640 1 -- 172.21.15.47:0/2575559899 shutdown_connections 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.367+0000 7fad33d4e640 1 -- 172.21.15.47:0/2575559899 wait complete. 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.368+0000 7fad33d4e640 1 Processor -- start 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.368+0000 7fad33d4e640 1 -- start start 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad33d4e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2c0b8350 0x7fad2c0b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad33d4e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad2c0cdc40 0x7fad2c14be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad33d4e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad2c15ca90 0x7fad2c0c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad33d4e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c14c3c0 con 0x7fad2c0b8350 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad33d4e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0cae80 con 0x7fad2c15ca90 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad33d4e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0c9fd0 con 0x7fad2c0cdc40 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad31aea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2c0b8350 0x7fad2c0b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad31aea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2c0b8350 0x7fad2c0b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37046/0 (socket says 172.21.15.47:37046) 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad31aea640 1 -- 172.21.15.47:0/4257219705 learned_addr learned my addr 172.21.15.47:0/4257219705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad312e9640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad2c0cdc40 0x7fad2c14be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad322eb640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad2c15ca90 0x7fad2c0c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad31aea640 1 -- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad2c0cdc40 msgr2=0x7fad2c14be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad31aea640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad2c0cdc40 0x7fad2c14be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad31aea640 1 -- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad2c15ca90 msgr2=0x7fad2c0c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad31aea640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad2c15ca90 0x7fad2c0c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad31aea640 1 -- 172.21.15.47:0/4257219705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280094d0 con 0x7fad2c0b8350 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad312e9640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad2c0cdc40 0x7fad2c14be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad31aea640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2c0b8350 0x7fad2c0b64b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fad24007ef0 tx=0x7fad2400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.369+0000 7fad227fc640 1 -- 172.21.15.47:0/4257219705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24014620 con 0x7fad2c0b8350 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.370+0000 7fad227fc640 1 -- 172.21.15.47:0/4257219705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad24014780 con 0x7fad2c0b8350 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.370+0000 7fad33d4e640 1 -- 172.21.15.47:0/4257219705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2c0ca270 con 0x7fad2c0b8350 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.370+0000 7fad227fc640 1 -- 172.21.15.47:0/4257219705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24014b30 con 0x7fad2c0b8350 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.370+0000 7fad33d4e640 1 -- 172.21.15.47:0/4257219705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c0ca830 con 0x7fad2c0b8350 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.371+0000 7fad33d4e640 1 -- 172.21.15.47:0/4257219705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2c072360 con 0x7fad2c0b8350 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.371+0000 7fad227fc640 1 -- 172.21.15.47:0/4257219705 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fad24014c90 con 0x7fad2c0b8350 2024-03-23T09:51:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.371+0000 7fad227fc640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fad1006ac00 0x7fad1006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.371+0000 7fad227fc640 1 -- 172.21.15.47:0/4257219705 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fad24089d20 con 0x7fad2c0b8350 2024-03-23T09:51:57.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.371+0000 7fad312e9640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fad1006ac00 0x7fad1006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.372+0000 7fad312e9640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fad1006ac00 0x7fad1006d0c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fad2c15a910 tx=0x7fad1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:57.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.376+0000 7fad227fc640 1 -- 172.21.15.47:0/4257219705 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fad24054ec0 con 0x7fad2c0b8350 2024-03-23T09:51:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.559+0000 7fad33d4e640 1 -- 172.21.15.47:0/4257219705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad2c066be0 con 0x7fad2c0b8350 2024-03-23T09:51:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.560+0000 7fad227fc640 1 -- 172.21.15.47:0/4257219705 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16213 (secure 0 0 0) 0x7fad240584e0 con 0x7fad2c0b8350 2024-03-23T09:51:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:51:57.560 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":116,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:51:53.059965+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:51:38.679890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-23T09:51:39.396299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:51:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.562+0000 7fad33d4e640 1 -- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fad1006ac00 msgr2=0x7fad1006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.562+0000 7fad33d4e640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fad1006ac00 0x7fad1006d0c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fad2c15a910 tx=0x7fad1c008040 comp rx=0 tx=0).stop 2024-03-23T09:51:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.562+0000 7fad33d4e640 1 -- 172.21.15.47:0/4257219705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2c0b8350 msgr2=0x7fad2c0b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.562+0000 7fad33d4e640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2c0b8350 0x7fad2c0b64b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fad24007ef0 tx=0x7fad2400e4b0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.562+0000 7fad33d4e640 1 -- 172.21.15.47:0/4257219705 shutdown_connections 2024-03-23T09:51:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.562+0000 7fad33d4e640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fad1006ac00 0x7fad1006d0c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.562+0000 7fad33d4e640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad2c15ca90 0x7fad2c0c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.562+0000 7fad33d4e640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad2c0cdc40 0x7fad2c14be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.562+0000 7fad33d4e640 1 --2- 172.21.15.47:0/4257219705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2c0b8350 0x7fad2c0b64b0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.562+0000 7fad33d4e640 1 -- 172.21.15.47:0/4257219705 >> 172.21.15.47:0/4257219705 conn(0x7fad2c0659d0 msgr2=0x7fad2c15f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.562+0000 7fad33d4e640 1 -- 172.21.15.47:0/4257219705 shutdown_connections 2024-03-23T09:51:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.562+0000 7fad33d4e640 1 -- 172.21.15.47:0/4257219705 wait complete. 2024-03-23T09:51:57.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:51:38.679890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:51:39.396299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:51:57.573 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:51:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.677+0000 7faa228c4640 1 Processor -- start 2024-03-23T09:51:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.677+0000 7faa228c4640 1 -- start start 2024-03-23T09:51:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.677+0000 7faa228c4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faa1c0ae270 0x7faa1c0d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.677+0000 7faa228c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa1c0ce3e0 0x7faa1c0ac530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.677+0000 7faa228c4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c0af870 con 0x7faa1c0b0d20 2024-03-23T09:51:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.677+0000 7faa228c4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c0afac0 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.677+0000 7faa228c4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c0afda0 con 0x7faa1c0ae270 2024-03-23T09:51:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.678+0000 7faa20e61640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa1c0ce3e0 0x7faa1c0ac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.678+0000 7faa20e61640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa1c0ce3e0 0x7faa1c0ac530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37050/0 (socket says 172.21.15.47:37050) 2024-03-23T09:51:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.678+0000 7faa20e61640 1 -- 172.21.15.47:0/2720747960 learned_addr learned my addr 172.21.15.47:0/2720747960 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.678+0000 7faa1bfff640 1 --2- 172.21.15.47:0/2720747960 >> v2:172.21.15.175:3300/0 conn(0x7faa1c0ae270 0x7faa1c0d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.678+0000 7faa20e61640 1 -- 172.21.15.47:0/2720747960 >> v2:172.21.15.175:3300/0 conn(0x7faa1c0ae270 msgr2=0x7faa1c0d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.678+0000 7faa20e61640 1 --2- 172.21.15.47:0/2720747960 >> v2:172.21.15.175:3300/0 conn(0x7faa1c0ae270 0x7faa1c0d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.678+0000 7faa20e61640 1 -- 172.21.15.47:0/2720747960 >> v1:172.21.15.175:6789/0 conn(0x7faa1c0b0d20 legacy=0x7faa1c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.678+0000 7faa20e61640 1 -- 172.21.15.47:0/2720747960 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c0b0080 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.678+0000 7faa1bfff640 1 --2- 172.21.15.47:0/2720747960 >> v2:172.21.15.175:3300/0 conn(0x7faa1c0ae270 0x7faa1c0d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:51:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.679+0000 7faa20e61640 1 --2- 172.21.15.47:0/2720747960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c0ce3e0 0x7faa1c0ac530 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7faa1c06fe60 tx=0x7faa1402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ff22fe14ab7408 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.679+0000 7faa1affd640 1 -- 172.21.15.47:0/2720747960 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1402cb20 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.679+0000 7faa1affd640 1 -- 172.21.15.47:0/2720747960 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1402cc80 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.679+0000 7faa1affd640 1 -- 172.21.15.47:0/2720747960 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140343f0 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.679+0000 7faa228c4640 1 -- 172.21.15.47:0/2720747960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c0ce3e0 msgr2=0x7faa1c0ac530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.679+0000 7faa228c4640 1 --2- 172.21.15.47:0/2720747960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c0ce3e0 0x7faa1c0ac530 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7faa1c06fe60 tx=0x7faa1402bf80 comp rx=0 tx=0).stop 2024-03-23T09:51:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.679+0000 7faa228c4640 1 -- 172.21.15.47:0/2720747960 shutdown_connections 2024-03-23T09:51:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.679+0000 7faa228c4640 1 --2- 172.21.15.47:0/2720747960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c0ce3e0 0x7faa1c0ac530 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.679+0000 7faa228c4640 1 --2- 172.21.15.47:0/2720747960 >> v2:172.21.15.175:3300/0 conn(0x7faa1c0ae270 0x7faa1c0d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.679+0000 7faa228c4640 1 -- 172.21.15.47:0/2720747960 >> 172.21.15.47:0/2720747960 conn(0x7faa1c098a00 msgr2=0x7faa1c0cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.679+0000 7faa228c4640 1 -- 172.21.15.47:0/2720747960 shutdown_connections 2024-03-23T09:51:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.679+0000 7faa228c4640 1 -- 172.21.15.47:0/2720747960 wait complete. 2024-03-23T09:51:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.680+0000 7faa228c4640 1 Processor -- start 2024-03-23T09:51:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.681+0000 7faa228c4640 1 -- start start 2024-03-23T09:51:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.681+0000 7faa228c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1c0ae270 0x7faa1c156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.681+0000 7faa228c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa1c0b0d20 0x7faa1c0c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.681+0000 7faa228c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c0ce3e0 0x7faa1c0c2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.681+0000 7faa228c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0c90d0 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.681+0000 7faa228c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0c2b90 con 0x7faa1c0ae270 2024-03-23T09:51:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.681+0000 7faa228c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0c8030 con 0x7faa1c0b0d20 2024-03-23T09:51:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.681+0000 7faa20e61640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c0ce3e0 0x7faa1c0c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.681+0000 7faa20e61640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c0ce3e0 0x7faa1c0c2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37062/0 (socket says 172.21.15.47:37062) 2024-03-23T09:51:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.681+0000 7faa20e61640 1 -- 172.21.15.47:0/3040984450 learned_addr learned my addr 172.21.15.47:0/3040984450 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.682+0000 7faa1bfff640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1c0ae270 0x7faa1c156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.682+0000 7faa20e61640 1 -- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa1c0b0d20 msgr2=0x7faa1c0c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.682+0000 7faa20e61640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa1c0b0d20 0x7faa1c0c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.682+0000 7faa20e61640 1 -- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1c0ae270 msgr2=0x7faa1c156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.682+0000 7faa20e61640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1c0ae270 0x7faa1c156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.682+0000 7faa20e61640 1 -- 172.21.15.47:0/3040984450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa14009b30 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.682+0000 7faa1bfff640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1c0ae270 0x7faa1c156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.682+0000 7faa20e61640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c0ce3e0 0x7faa1c0c2650 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7faa14036040 tx=0x7faa14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.682+0000 7faa18ff9640 1 -- 172.21.15.47:0/3040984450 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14007e70 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.682+0000 7faa18ff9640 1 -- 172.21.15.47:0/3040984450 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14009510 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.682+0000 7faa228c4640 1 -- 172.21.15.47:0/3040984450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c0c8310 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.682+0000 7faa18ff9640 1 -- 172.21.15.47:0/3040984450 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1403e690 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.682+0000 7faa228c4640 1 -- 172.21.15.47:0/3040984450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c0c88d0 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.683+0000 7faa228c4640 1 -- 172.21.15.47:0/3040984450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c0672f0 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.683+0000 7faa18ff9640 1 -- 172.21.15.47:0/3040984450 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7faa1403e7f0 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.684+0000 7faa18ff9640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faa0806ac00 0x7faa0806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.684+0000 7faa18ff9640 1 -- 172.21.15.47:0/3040984450 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7faa140abf20 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.684+0000 7faa1bfff640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faa0806ac00 0x7faa0806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.685+0000 7faa1bfff640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faa0806ac00 0x7faa0806d0c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7faa0c006fd0 tx=0x7faa0c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.688+0000 7faa18ff9640 1 -- 172.21.15.47:0/3040984450 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7faa140770c0 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.867+0000 7faa228c4640 1 -- 172.21.15.47:0/3040984450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa1c0c99f0 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.867+0000 7faa18ff9640 1 -- 172.21.15.47:0/3040984450 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa14043020 con 0x7faa1c0ce3e0 2024-03-23T09:51:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:51:57.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.869+0000 7faa228c4640 1 -- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faa0806ac00 msgr2=0x7faa0806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.869+0000 7faa228c4640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faa0806ac00 0x7faa0806d0c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7faa0c006fd0 tx=0x7faa0c008040 comp rx=0 tx=0).stop 2024-03-23T09:51:57.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.869+0000 7faa228c4640 1 -- 172.21.15.47:0/3040984450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c0ce3e0 msgr2=0x7faa1c0c2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.869+0000 7faa228c4640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c0ce3e0 0x7faa1c0c2650 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7faa14036040 tx=0x7faa14004a40 comp rx=0 tx=0).stop 2024-03-23T09:51:57.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.869+0000 7faa228c4640 1 -- 172.21.15.47:0/3040984450 shutdown_connections 2024-03-23T09:51:57.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.869+0000 7faa228c4640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7faa0806ac00 0x7faa0806d0c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.869+0000 7faa228c4640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c0ce3e0 0x7faa1c0c2650 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.869+0000 7faa228c4640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa1c0b0d20 0x7faa1c0c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.869+0000 7faa228c4640 1 --2- 172.21.15.47:0/3040984450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1c0ae270 0x7faa1c156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.869+0000 7faa228c4640 1 -- 172.21.15.47:0/3040984450 >> 172.21.15.47:0/3040984450 conn(0x7faa1c098a00 msgr2=0x7faa1c0aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:57.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.869+0000 7faa228c4640 1 -- 172.21.15.47:0/3040984450 shutdown_connections 2024-03-23T09:51:57.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.869+0000 7faa228c4640 1 -- 172.21.15.47:0/3040984450 wait complete. 2024-03-23T09:51:57.880 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:51:57.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.983+0000 7f6ae0d99640 1 Processor -- start 2024-03-23T09:51:57.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.983+0000 7f6ae0d99640 1 -- start start 2024-03-23T09:51:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.984+0000 7f6ae0d99640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6adc15ca30 0x7f6adc160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.984+0000 7f6ae0d99640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6adc15ce10 0x7f6adc0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.984+0000 7f6ae0d99640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6adc0b85d0 0x7f6adc0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.984+0000 7f6ae0d99640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6adc161600 con 0x7f6adc15ce10 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.984+0000 7f6ae0d99640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6adc161040 con 0x7f6adc15ca30 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.984+0000 7f6ae0d99640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc161320 con 0x7f6adc0b85d0 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.984+0000 7f6ad3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6adc15ce10 0x7f6adc0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.984+0000 7f6ad3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6adc15ce10 0x7f6adc0cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37080/0 (socket says 172.21.15.47:37080) 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.984+0000 7f6ad3fff640 1 -- 172.21.15.47:0/1753270395 learned_addr learned my addr 172.21.15.47:0/1753270395 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.985+0000 7f6adad9d640 1 --2- 172.21.15.47:0/1753270395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6adc0b85d0 0x7f6adc0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.985+0000 7f6ada59c640 1 --2- 172.21.15.47:0/1753270395 >> v2:172.21.15.175:3300/0 conn(0x7f6adc15ca30 0x7f6adc160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.985+0000 7f6ad3fff640 1 -- 172.21.15.47:0/1753270395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6adc0b85d0 msgr2=0x7f6adc0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.985+0000 7f6ad3fff640 1 --2- 172.21.15.47:0/1753270395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6adc0b85d0 0x7f6adc0cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.985+0000 7f6ad3fff640 1 -- 172.21.15.47:0/1753270395 >> v2:172.21.15.175:3300/0 conn(0x7f6adc15ca30 msgr2=0x7f6adc160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.985+0000 7f6ad3fff640 1 --2- 172.21.15.47:0/1753270395 >> v2:172.21.15.175:3300/0 conn(0x7f6adc15ca30 0x7f6adc160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.985+0000 7f6ad3fff640 1 -- 172.21.15.47:0/1753270395 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc0d0dd0 con 0x7f6adc15ce10 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.985+0000 7f6ada59c640 1 --2- 172.21.15.47:0/1753270395 >> v2:172.21.15.175:3300/0 conn(0x7f6adc15ca30 0x7f6adc160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.985+0000 7f6ad3fff640 1 --2- 172.21.15.47:0/1753270395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6adc15ce10 0x7f6adc0cd730 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f6adc06c1a0 tx=0x7f6ac402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee58f7f06ca2b1d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.986+0000 7f6ad9d9b640 1 -- 172.21.15.47:0/1753270395 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4031870 con 0x7f6adc15ce10 2024-03-23T09:51:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.986+0000 7f6ad9d9b640 1 -- 172.21.15.47:0/1753270395 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac40319d0 con 0x7f6adc15ce10 2024-03-23T09:51:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.986+0000 7f6ad9d9b640 1 -- 172.21.15.47:0/1753270395 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4031d60 con 0x7f6adc15ce10 2024-03-23T09:51:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.986+0000 7f6ae0d99640 1 -- 172.21.15.47:0/1753270395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6adc15ce10 msgr2=0x7f6adc0cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.986+0000 7f6ae0d99640 1 --2- 172.21.15.47:0/1753270395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6adc15ce10 0x7f6adc0cd730 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f6adc06c1a0 tx=0x7f6ac402b910 comp rx=0 tx=0).stop 2024-03-23T09:51:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.986+0000 7f6ae0d99640 1 -- 172.21.15.47:0/1753270395 shutdown_connections 2024-03-23T09:51:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.986+0000 7f6ae0d99640 1 --2- 172.21.15.47:0/1753270395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6adc0b85d0 0x7f6adc0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.986+0000 7f6ae0d99640 1 --2- 172.21.15.47:0/1753270395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6adc15ce10 0x7f6adc0cd730 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.986+0000 7f6ae0d99640 1 --2- 172.21.15.47:0/1753270395 >> v2:172.21.15.175:3300/0 conn(0x7f6adc15ca30 0x7f6adc160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.986+0000 7f6ae0d99640 1 -- 172.21.15.47:0/1753270395 >> 172.21.15.47:0/1753270395 conn(0x7f6adc06b160 msgr2=0x7f6adc06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.986+0000 7f6ae0d99640 1 -- 172.21.15.47:0/1753270395 shutdown_connections 2024-03-23T09:51:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.986+0000 7f6ae0d99640 1 -- 172.21.15.47:0/1753270395 wait complete. 2024-03-23T09:51:57.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.988+0000 7f6ae0d99640 1 Processor -- start 2024-03-23T09:51:57.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.988+0000 7f6ae0d99640 1 -- start start 2024-03-23T09:51:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.988+0000 7f6ae0d99640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6adc0b85d0 0x7f6adc15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.988+0000 7f6ae0d99640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6adc15ca30 0x7f6adc153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.988+0000 7f6ae0d99640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6adc15ce10 0x7f6adc0c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.988+0000 7f6ae0d99640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0cb370 con 0x7f6adc0b85d0 2024-03-23T09:51:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.988+0000 7f6ae0d99640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc1494d0 con 0x7f6adc15ce10 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.988+0000 7f6ae0d99640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc15e1c0 con 0x7f6adc15ca30 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.988+0000 7f6ada59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6adc0b85d0 0x7f6adc15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.988+0000 7f6ada59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6adc0b85d0 0x7f6adc15dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37082/0 (socket says 172.21.15.47:37082) 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.988+0000 7f6ada59c640 1 -- 172.21.15.47:0/2475908957 learned_addr learned my addr 172.21.15.47:0/2475908957 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.989+0000 7f6ada59c640 1 -- 172.21.15.47:0/2475908957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6adc15ca30 msgr2=0x7f6adc153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.989+0000 7f6ada59c640 1 --2- 172.21.15.47:0/2475908957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6adc15ca30 0x7f6adc153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.989+0000 7f6ada59c640 1 -- 172.21.15.47:0/2475908957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6adc15ce10 msgr2=0x7f6adc0c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.989+0000 7f6ada59c640 1 --2- 172.21.15.47:0/2475908957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6adc15ce10 0x7f6adc0c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.989+0000 7f6ada59c640 1 -- 172.21.15.47:0/2475908957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac40094d0 con 0x7f6adc0b85d0 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.989+0000 7f6ada59c640 1 --2- 172.21.15.47:0/2475908957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6adc0b85d0 0x7f6adc15dbe0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00c7f0 tx=0x7f6acc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.989+0000 7f6ad2ffd640 1 -- 172.21.15.47:0/2475908957 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc007a30 con 0x7f6adc0b85d0 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.989+0000 7f6ad2ffd640 1 -- 172.21.15.47:0/2475908957 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc007b90 con 0x7f6adc0b85d0 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.989+0000 7f6ae0d99640 1 -- 172.21.15.47:0/2475908957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc0cad20 con 0x7f6adc0b85d0 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.989+0000 7f6ad2ffd640 1 -- 172.21.15.47:0/2475908957 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc00ce40 con 0x7f6adc0b85d0 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.989+0000 7f6ae0d99640 1 -- 172.21.15.47:0/2475908957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc154680 con 0x7f6adc0b85d0 2024-03-23T09:51:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.990+0000 7f6ae0d99640 1 -- 172.21.15.47:0/2475908957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6adc072360 con 0x7f6adc0b85d0 2024-03-23T09:51:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.990+0000 7f6ad2ffd640 1 -- 172.21.15.47:0/2475908957 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6acc0113f0 con 0x7f6adc0b85d0 2024-03-23T09:51:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.991+0000 7f6ad2ffd640 1 --2- 172.21.15.47:0/2475908957 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6abc06ac30 0x7f6abc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.991+0000 7f6ad2ffd640 1 -- 172.21.15.47:0/2475908957 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6acc0894c0 con 0x7f6adc0b85d0 2024-03-23T09:51:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.991+0000 7f6ad3fff640 1 --2- 172.21.15.47:0/2475908957 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6abc06ac30 0x7f6abc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.992+0000 7f6ad3fff640 1 --2- 172.21.15.47:0/2475908957 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6abc06ac30 0x7f6abc06d0f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4036000 tx=0x7f6ac4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:57.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:57.995+0000 7f6ad2ffd640 1 -- 172.21.15.47:0/2475908957 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6acc0546f0 con 0x7f6adc0b85d0 2024-03-23T09:51:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.173+0000 7f6ae0d99640 1 -- 172.21.15.47:0/2475908957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6adc071f40 con 0x7f6adc0b85d0 2024-03-23T09:51:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.174+0000 7f6ad2ffd640 1 -- 172.21.15.47:0/2475908957 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6acc057d10 con 0x7f6adc0b85d0 2024-03-23T09:51:58.175 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:51:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.176+0000 7f6ae0d99640 1 -- 172.21.15.47:0/2475908957 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6abc06ac30 msgr2=0x7f6abc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.176+0000 7f6ae0d99640 1 --2- 172.21.15.47:0/2475908957 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6abc06ac30 0x7f6abc06d0f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4036000 tx=0x7f6ac4030000 comp rx=0 tx=0).stop 2024-03-23T09:51:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.176+0000 7f6ae0d99640 1 -- 172.21.15.47:0/2475908957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6adc0b85d0 msgr2=0x7f6adc15dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.176+0000 7f6ae0d99640 1 --2- 172.21.15.47:0/2475908957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6adc0b85d0 0x7f6adc15dbe0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00c7f0 tx=0x7f6acc00cbf0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.176+0000 7f6ae0d99640 1 -- 172.21.15.47:0/2475908957 shutdown_connections 2024-03-23T09:51:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.176+0000 7f6ae0d99640 1 --2- 172.21.15.47:0/2475908957 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6abc06ac30 0x7f6abc06d0f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.176+0000 7f6ae0d99640 1 --2- 172.21.15.47:0/2475908957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6adc15ce10 0x7f6adc0c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.176+0000 7f6ae0d99640 1 --2- 172.21.15.47:0/2475908957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6adc15ca30 0x7f6adc153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.176+0000 7f6ae0d99640 1 --2- 172.21.15.47:0/2475908957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6adc0b85d0 0x7f6adc15dbe0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.176+0000 7f6ae0d99640 1 -- 172.21.15.47:0/2475908957 >> 172.21.15.47:0/2475908957 conn(0x7f6adc06b160 msgr2=0x7f6adc154940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.176+0000 7f6ae0d99640 1 -- 172.21.15.47:0/2475908957 shutdown_connections 2024-03-23T09:51:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.176+0000 7f6ae0d99640 1 -- 172.21.15.47:0/2475908957 wait complete. 2024-03-23T09:51:58.187 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:51:58.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.291+0000 7f7e49626640 1 Processor -- start 2024-03-23T09:51:58.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.291+0000 7f7e49626640 1 -- start start 2024-03-23T09:51:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.291+0000 7f7e49626640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e4415ca30 0x7f7e44160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.291+0000 7f7e49626640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7e4415ce10 0x7f7e440cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.291+0000 7f7e49626640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e440b85d0 0x7f7e440cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.291+0000 7f7e49626640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e44161600 con 0x7f7e440b85d0 2024-03-23T09:51:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.291+0000 7f7e49626640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e44161040 con 0x7f7e4415ce10 2024-03-23T09:51:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.291+0000 7f7e49626640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e44161320 con 0x7f7e4415ca30 2024-03-23T09:51:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.292+0000 7f7e437fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e440b85d0 0x7f7e440cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.292+0000 7f7e437fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e440b85d0 0x7f7e440cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37086/0 (socket says 172.21.15.47:37086) 2024-03-23T09:51:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.292+0000 7f7e437fe640 1 -- 172.21.15.47:0/2924621557 learned_addr learned my addr 172.21.15.47:0/2924621557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.292+0000 7f7e42ffd640 1 --2- 172.21.15.47:0/2924621557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e4415ca30 0x7f7e44160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:58.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.292+0000 7f7e3bfff640 1 --2- 172.21.15.47:0/2924621557 >> v2:172.21.15.175:3300/0 conn(0x7f7e4415ce10 0x7f7e440cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.292+0000 7f7e437fe640 1 -- 172.21.15.47:0/2924621557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e4415ca30 msgr2=0x7f7e44160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.292+0000 7f7e437fe640 1 --2- 172.21.15.47:0/2924621557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e4415ca30 0x7f7e44160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.292+0000 7f7e437fe640 1 -- 172.21.15.47:0/2924621557 >> v2:172.21.15.175:3300/0 conn(0x7f7e4415ce10 msgr2=0x7f7e440cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.292+0000 7f7e437fe640 1 --2- 172.21.15.47:0/2924621557 >> v2:172.21.15.175:3300/0 conn(0x7f7e4415ce10 0x7f7e440cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.292+0000 7f7e437fe640 1 -- 172.21.15.47:0/2924621557 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e440d0dd0 con 0x7f7e440b85d0 2024-03-23T09:51:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.292+0000 7f7e3bfff640 1 --2- 172.21.15.47:0/2924621557 >> v2:172.21.15.175:3300/0 conn(0x7f7e4415ce10 0x7f7e440cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.293+0000 7f7e437fe640 1 --2- 172.21.15.47:0/2924621557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e440b85d0 0x7f7e440cdc70 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f7e440b5550 tx=0x7f7e3c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62830bab89195b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.293+0000 7f7e427fc640 1 -- 172.21.15.47:0/2924621557 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c02ca70 con 0x7f7e440b85d0 2024-03-23T09:51:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.293+0000 7f7e427fc640 1 -- 172.21.15.47:0/2924621557 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3c02cbd0 con 0x7f7e440b85d0 2024-03-23T09:51:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.293+0000 7f7e427fc640 1 -- 172.21.15.47:0/2924621557 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c0343f0 con 0x7f7e440b85d0 2024-03-23T09:51:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.293+0000 7f7e49626640 1 -- 172.21.15.47:0/2924621557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e440b85d0 msgr2=0x7f7e440cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.293+0000 7f7e49626640 1 --2- 172.21.15.47:0/2924621557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e440b85d0 0x7f7e440cdc70 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f7e440b5550 tx=0x7f7e3c02bed0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.293+0000 7f7e49626640 1 -- 172.21.15.47:0/2924621557 shutdown_connections 2024-03-23T09:51:58.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.293+0000 7f7e49626640 1 --2- 172.21.15.47:0/2924621557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e440b85d0 0x7f7e440cdc70 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.293+0000 7f7e49626640 1 --2- 172.21.15.47:0/2924621557 >> v2:172.21.15.175:3300/0 conn(0x7f7e4415ce10 0x7f7e440cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.293+0000 7f7e49626640 1 --2- 172.21.15.47:0/2924621557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e4415ca30 0x7f7e44160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.293+0000 7f7e49626640 1 -- 172.21.15.47:0/2924621557 >> 172.21.15.47:0/2924621557 conn(0x7f7e4406b160 msgr2=0x7f7e4406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:58.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.293+0000 7f7e49626640 1 -- 172.21.15.47:0/2924621557 shutdown_connections 2024-03-23T09:51:58.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.294+0000 7f7e49626640 1 -- 172.21.15.47:0/2924621557 wait complete. 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.295+0000 7f7e49626640 1 Processor -- start 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.295+0000 7f7e49626640 1 -- start start 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.295+0000 7f7e49626640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e440b85d0 0x7f7e44147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.295+0000 7f7e49626640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e4415ca30 0x7f7e441534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.295+0000 7f7e49626640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e4415ce10 0x7f7e44153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.295+0000 7f7e49626640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e441546a0 con 0x7f7e4415ca30 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.295+0000 7f7e49626640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e441582a0 con 0x7f7e440b85d0 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.295+0000 7f7e49626640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e441543c0 con 0x7f7e4415ce10 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.295+0000 7f7e3bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e4415ca30 0x7f7e441534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e3bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e4415ca30 0x7f7e441534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37096/0 (socket says 172.21.15.47:37096) 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e3bfff640 1 -- 172.21.15.47:0/1210893437 learned_addr learned my addr 172.21.15.47:0/1210893437 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e42ffd640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e440b85d0 0x7f7e44147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e437fe640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e4415ce10 0x7f7e44153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e3bfff640 1 -- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e4415ce10 msgr2=0x7f7e44153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e3bfff640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e4415ce10 0x7f7e44153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e3bfff640 1 -- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e440b85d0 msgr2=0x7f7e44147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e3bfff640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e440b85d0 0x7f7e44147ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e3bfff640 1 -- 172.21.15.47:0/1210893437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c009b30 con 0x7f7e4415ca30 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e42ffd640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e440b85d0 0x7f7e44147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e437fe640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e4415ce10 0x7f7e44153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e3bfff640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e4415ca30 0x7f7e441534d0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00c7f0 tx=0x7f7e2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e3b7fe640 1 -- 172.21.15.47:0/1210893437 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c007820 con 0x7f7e4415ca30 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e3b7fe640 1 -- 172.21.15.47:0/1210893437 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c007980 con 0x7f7e4415ca30 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e49626640 1 -- 172.21.15.47:0/1210893437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e44157c50 con 0x7f7e4415ca30 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e3b7fe640 1 -- 172.21.15.47:0/1210893437 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c007d10 con 0x7f7e4415ca30 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.296+0000 7f7e49626640 1 -- 172.21.15.47:0/1210893437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e44148220 con 0x7f7e4415ca30 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.297+0000 7f7e49626640 1 -- 172.21.15.47:0/1210893437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e44072360 con 0x7f7e4415ca30 2024-03-23T09:51:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.298+0000 7f7e3b7fe640 1 -- 172.21.15.47:0/1210893437 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7e2c024480 con 0x7f7e4415ca30 2024-03-23T09:51:58.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.299+0000 7f7e3b7fe640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e2406ac00 0x7f7e2406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:58.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.299+0000 7f7e3b7fe640 1 -- 172.21.15.47:0/1210893437 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7e2c089b90 con 0x7f7e4415ca30 2024-03-23T09:51:58.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.299+0000 7f7e42ffd640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e2406ac00 0x7f7e2406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:58.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.299+0000 7f7e42ffd640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e2406ac00 0x7f7e2406d0c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7e34006fd0 tx=0x7f7e34008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:58.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.302+0000 7f7e3b7fe640 1 -- 172.21.15.47:0/1210893437 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7e2c054d10 con 0x7f7e4415ca30 2024-03-23T09:51:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.483+0000 7f7e49626640 1 -- 172.21.15.47:0/1210893437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e44066be0 con 0x7f7e4415ca30 2024-03-23T09:51:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.483+0000 7f7e3b7fe640 1 -- 172.21.15.47:0/1210893437 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e2c01f0a0 con 0x7f7e4415ca30 2024-03-23T09:51:58.483 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:51:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.485+0000 7f7e49626640 1 -- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e2406ac00 msgr2=0x7f7e2406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.485+0000 7f7e49626640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e2406ac00 0x7f7e2406d0c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7e34006fd0 tx=0x7f7e34008040 comp rx=0 tx=0).stop 2024-03-23T09:51:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.485+0000 7f7e49626640 1 -- 172.21.15.47:0/1210893437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e4415ca30 msgr2=0x7f7e441534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.485+0000 7f7e49626640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e4415ca30 0x7f7e441534d0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00c7f0 tx=0x7f7e2c00ccb0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.485+0000 7f7e49626640 1 -- 172.21.15.47:0/1210893437 shutdown_connections 2024-03-23T09:51:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.485+0000 7f7e49626640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e2406ac00 0x7f7e2406d0c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.485+0000 7f7e49626640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e4415ce10 0x7f7e44153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.485+0000 7f7e49626640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e4415ca30 0x7f7e441534d0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.485+0000 7f7e49626640 1 --2- 172.21.15.47:0/1210893437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e440b85d0 0x7f7e44147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.485+0000 7f7e49626640 1 -- 172.21.15.47:0/1210893437 >> 172.21.15.47:0/1210893437 conn(0x7f7e4406b160 msgr2=0x7f7e4414b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.485+0000 7f7e49626640 1 -- 172.21.15.47:0/1210893437 shutdown_connections 2024-03-23T09:51:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.485+0000 7f7e49626640 1 -- 172.21.15.47:0/1210893437 wait complete. 2024-03-23T09:51:58.496 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:51:58.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.598+0000 7f90310ea640 1 Processor -- start 2024-03-23T09:51:58.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.598+0000 7f90310ea640 1 -- start start 2024-03-23T09:51:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.598+0000 7f90310ea640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f902c15cd00 0x7f902c0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.598+0000 7f90310ea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f902c0d1020 0x7f902c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.599+0000 7f90310ea640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f902c15d0e0 con 0x7f902c0cf750 2024-03-23T09:51:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.599+0000 7f90310ea640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f902c0cfbe0 con 0x7f902c15cd00 2024-03-23T09:51:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.599+0000 7f90310ea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0d1540 con 0x7f902c0d1020 2024-03-23T09:51:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.599+0000 7f902ad9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f902c15cd00 0x7f902c0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.599+0000 7f902ad9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f902c15cd00 0x7f902c0d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37098/0 (socket says 172.21.15.47:37098) 2024-03-23T09:51:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.599+0000 7f902ad9d640 1 -- 172.21.15.47:0/2117085097 learned_addr learned my addr 172.21.15.47:0/2117085097 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.599+0000 7f9023fff640 1 --2- 172.21.15.47:0/2117085097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f902c0d1020 0x7f902c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.599+0000 7f902ad9d640 1 -- 172.21.15.47:0/2117085097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f902c0d1020 msgr2=0x7f902c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:51:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.599+0000 7f902ad9d640 1 --2- 172.21.15.47:0/2117085097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f902c0d1020 0x7f902c0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.599+0000 7f902ad9d640 1 -- 172.21.15.47:0/2117085097 >> v1:172.21.15.175:6789/0 conn(0x7f902c0cf750 legacy=0x7f902c0b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.599+0000 7f902ad9d640 1 -- 172.21.15.47:0/2117085097 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c0d1820 con 0x7f902c15cd00 2024-03-23T09:51:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.600+0000 7f902ad9d640 1 --2- 172.21.15.47:0/2117085097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f902c15cd00 0x7f902c0d0ae0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f901c009870 tx=0x7f901c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a281ca5b642ad79 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.600+0000 7f902a59c640 1 -- 172.21.15.47:0/2117085097 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c031640 con 0x7f902c15cd00 2024-03-23T09:51:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.600+0000 7f902a59c640 1 -- 172.21.15.47:0/2117085097 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901c0317a0 con 0x7f902c15cd00 2024-03-23T09:51:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.600+0000 7f902a59c640 1 -- 172.21.15.47:0/2117085097 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c031b30 con 0x7f902c15cd00 2024-03-23T09:51:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.600+0000 7f90310ea640 1 -- 172.21.15.47:0/2117085097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f902c15cd00 msgr2=0x7f902c0d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.600+0000 7f90310ea640 1 --2- 172.21.15.47:0/2117085097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f902c15cd00 0x7f902c0d0ae0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f901c009870 tx=0x7f901c02b8f0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.600+0000 7f90310ea640 1 -- 172.21.15.47:0/2117085097 shutdown_connections 2024-03-23T09:51:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.600+0000 7f90310ea640 1 --2- 172.21.15.47:0/2117085097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f902c0d1020 0x7f902c0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.600+0000 7f90310ea640 1 --2- 172.21.15.47:0/2117085097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f902c15cd00 0x7f902c0d0ae0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.600+0000 7f90310ea640 1 -- 172.21.15.47:0/2117085097 >> 172.21.15.47:0/2117085097 conn(0x7f902c06b160 msgr2=0x7f902c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.600+0000 7f90310ea640 1 -- 172.21.15.47:0/2117085097 shutdown_connections 2024-03-23T09:51:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.601+0000 7f90310ea640 1 -- 172.21.15.47:0/2117085097 wait complete. 2024-03-23T09:51:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.602+0000 7f90310ea640 1 Processor -- start 2024-03-23T09:51:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.602+0000 7f90310ea640 1 -- start start 2024-03-23T09:51:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.602+0000 7f90310ea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f902c0cf750 0x7f902c1591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.602+0000 7f90310ea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f902c0d1020 0x7f902c1f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.602+0000 7f90310ea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f902c15cd00 0x7f902c0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.602+0000 7f90310ea640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c1f3b90 con 0x7f902c0d1020 2024-03-23T09:51:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.602+0000 7f90310ea640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0ca190 con 0x7f902c0cf750 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.602+0000 7f90310ea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902c1f3410 con 0x7f902c15cd00 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f9023fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f902c0d1020 0x7f902c1f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f9023fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f902c0d1020 0x7f902c1f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37104/0 (socket says 172.21.15.47:37104) 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f9023fff640 1 -- 172.21.15.47:0/641154610 learned_addr learned my addr 172.21.15.47:0/641154610 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f902b59e640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f902c15cd00 0x7f902c0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f902ad9d640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f902c0cf750 0x7f902c1591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f9023fff640 1 -- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f902c15cd00 msgr2=0x7f902c0b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f9023fff640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f902c15cd00 0x7f902c0b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f9023fff640 1 -- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f902c0cf750 msgr2=0x7f902c1591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f9023fff640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f902c0cf750 0x7f902c1591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f9023fff640 1 -- 172.21.15.47:0/641154610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c0094d0 con 0x7f902c0d1020 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f9023fff640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f902c0d1020 0x7f902c1f2ed0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f901400d800 tx=0x7f901400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f90237fe640 1 -- 172.21.15.47:0/641154610 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901400b6c0 con 0x7f902c0d1020 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f90237fe640 1 -- 172.21.15.47:0/641154610 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901400b840 con 0x7f902c0d1020 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f90237fe640 1 -- 172.21.15.47:0/641154610 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901400bbf0 con 0x7f902c0d1020 2024-03-23T09:51:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f90310ea640 1 -- 172.21.15.47:0/641154610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902c0c9bd0 con 0x7f902c0d1020 2024-03-23T09:51:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.603+0000 7f90310ea640 1 -- 172.21.15.47:0/641154610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902c1f37b0 con 0x7f902c0d1020 2024-03-23T09:51:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.604+0000 7f90310ea640 1 -- 172.21.15.47:0/641154610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902c072360 con 0x7f902c0d1020 2024-03-23T09:51:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.605+0000 7f90237fe640 1 -- 172.21.15.47:0/641154610 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f901400bd50 con 0x7f902c0d1020 2024-03-23T09:51:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.605+0000 7f90237fe640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f900c06ac00 0x7f900c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:51:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.605+0000 7f90237fe640 1 -- 172.21.15.47:0/641154610 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f90140898c0 con 0x7f902c0d1020 2024-03-23T09:51:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.605+0000 7f902ad9d640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f900c06ac00 0x7f900c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:51:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.606+0000 7f902ad9d640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f900c06ac00 0x7f900c06d0c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f901c036000 tx=0x7f901c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:51:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.609+0000 7f90237fe640 1 -- 172.21.15.47:0/641154610 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9014054af0 con 0x7f902c0d1020 2024-03-23T09:51:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.800+0000 7f90310ea640 1 -- 172.21.15.47:0/641154610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f902c1f3610 con 0x7f902c0d1020 2024-03-23T09:51:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.801+0000 7f90237fe640 1 -- 172.21.15.47:0/641154610 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v116) v1 ==== 80+0+0 (secure 0 0 0) 0x7f902c1f3610 con 0x7f902c0d1020 2024-03-23T09:51:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-03-23T09:51:58.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.803+0000 7f90310ea640 1 -- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f900c06ac00 msgr2=0x7f900c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:58.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.803+0000 7f90310ea640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f900c06ac00 0x7f900c06d0c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f901c036000 tx=0x7f901c03c040 comp rx=0 tx=0).stop 2024-03-23T09:51:58.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.803+0000 7f90310ea640 1 -- 172.21.15.47:0/641154610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f902c0d1020 msgr2=0x7f902c1f2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:51:58.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.803+0000 7f90310ea640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f902c0d1020 0x7f902c1f2ed0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f901400d800 tx=0x7f901400dcc0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.803+0000 7f90310ea640 1 -- 172.21.15.47:0/641154610 shutdown_connections 2024-03-23T09:51:58.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.803+0000 7f90310ea640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f900c06ac00 0x7f900c06d0c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.803+0000 7f90310ea640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f902c15cd00 0x7f902c0b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.803+0000 7f90310ea640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f902c0d1020 0x7f902c1f2ed0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.803+0000 7f90310ea640 1 --2- 172.21.15.47:0/641154610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f902c0cf750 0x7f902c1591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:51:58.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.803+0000 7f90310ea640 1 -- 172.21.15.47:0/641154610 >> 172.21.15.47:0/641154610 conn(0x7f902c06b160 msgr2=0x7f902c15db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:51:58.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.803+0000 7f90310ea640 1 -- 172.21.15.47:0/641154610 shutdown_connections 2024-03-23T09:51:58.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:51:58.803+0000 7f90310ea640 1 -- 172.21.15.47:0/641154610 wait complete. 2024-03-23T09:51:58.814 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:51:58.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:51:58.814 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:51:58.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[80131]: fuse finished with error 0 and tester_r 0 2024-03-23T09:51:58.841 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:52:04.942 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:52:04.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:04.943 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:52:04.961 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:52:04.961 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:52:05.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:52:05.019 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:05.019 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:52:05.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:05.019 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:52:05.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:52:05.076 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-23T09:52:05.077 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:52:05.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.230+0000 7f2ff0651640 1 Processor -- start 2024-03-23T09:52:05.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.230+0000 7f2ff0651640 1 -- start start 2024-03-23T09:52:05.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.231+0000 7f2ff0651640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2fe80cd6b0 0x7f2fe80cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.231+0000 7f2ff0651640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe80d09d0 con 0x7f2fe815cc40 2024-03-23T09:52:05.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.231+0000 7f2ff0651640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe80b5a80 con 0x7f2fe80d1420 2024-03-23T09:52:05.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.231+0000 7f2ff0651640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe80ce080 con 0x7f2fe80cd6b0 2024-03-23T09:52:05.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.231+0000 7f2fee3ed640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2fe815cc40 0x7f2fe8160750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32834/0 (socket says 172.21.15.47:32834) 2024-03-23T09:52:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.231+0000 7f2fee3ed640 1 -- 172.21.15.47:0/3560781747 learned_addr learned my addr 172.21.15.47:0/3560781747 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.231+0000 7f2fedbec640 1 --2- 172.21.15.47:0/3560781747 >> v2:172.21.15.175:3300/0 conn(0x7f2fe80cd6b0 0x7f2fe80cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.231+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991551658 0 0) 0x7f2fe80d09d0 con 0x7f2fe815cc40 2024-03-23T09:52:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.231+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fd4003520 con 0x7f2fe815cc40 2024-03-23T09:52:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.231+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4211958496 0 0) 0x7f2fd4003520 con 0x7f2fe815cc40 2024-03-23T09:52:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.231+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 >> v2:172.21.15.175:3300/0 conn(0x7f2fe80cd6b0 msgr2=0x7f2fe80cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.231+0000 7f2fed3eb640 1 --2- 172.21.15.47:0/3560781747 >> v2:172.21.15.175:3300/0 conn(0x7f2fe80cd6b0 0x7f2fe80cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.231+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 >> v1:172.21.15.175:6789/0 conn(0x7f2fe80d1420 legacy=0x7f2fe80d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe815d020 con 0x7f2fe815cc40 2024-03-23T09:52:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f2fe4001670 con 0x7f2fe815cc40 2024-03-23T09:52:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 >> v1:172.21.15.47:6789/0 conn(0x7f2fe815cc40 legacy=0x7f2fe8160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fed3eb640 1 --2- 172.21.15.47:0/3560781747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fd4003970 0x7f2fd4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fed3eb640 1 --2- 172.21.15.47:0/3560781747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fd40047c0 0x7f2fd4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fed3eb640 1 --2- 172.21.15.47:0/3560781747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fd4005180 0x7f2fd400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe80ce080 con 0x7f2fd4003970 2024-03-23T09:52:05.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd400c020 con 0x7f2fd40047c0 2024-03-23T09:52:05.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd400c1e0 con 0x7f2fd4005180 2024-03-23T09:52:05.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fee3ed640 1 --2- 172.21.15.47:0/3560781747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fd4003970 0x7f2fd4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fedbec640 1 --2- 172.21.15.47:0/3560781747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fd40047c0 0x7f2fd4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2feebee640 1 --2- 172.21.15.47:0/3560781747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fd4005180 0x7f2fd400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fee3ed640 1 -- 172.21.15.47:0/3560781747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fd4005180 msgr2=0x7f2fd400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:05.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fee3ed640 1 --2- 172.21.15.47:0/3560781747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fd4005180 0x7f2fd400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fee3ed640 1 -- 172.21.15.47:0/3560781747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fd40047c0 msgr2=0x7f2fd4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:05.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fee3ed640 1 --2- 172.21.15.47:0/3560781747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fd40047c0 0x7f2fd4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2fee3ed640 1 -- 172.21.15.47:0/3560781747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd4003520 con 0x7f2fd4003970 2024-03-23T09:52:05.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2feebee640 1 --2- 172.21.15.47:0/3560781747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fd4005180 0x7f2fd400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:05.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.232+0000 7f2feebee640 1 -- 172.21.15.47:0/3560781747 reap_dead start 2024-03-23T09:52:05.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.233+0000 7f2fee3ed640 1 --2- 172.21.15.47:0/3560781747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fd4003970 0x7f2fd4003d90 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c240 tx=0x7f2fd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71c04d8a497ef47 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:05.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.233+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4005ad0 con 0x7f2fd4003970 2024-03-23T09:52:05.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.233+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe4005c30 con 0x7f2fd4003970 2024-03-23T09:52:05.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.233+0000 7f2fed3eb640 1 -- 172.21.15.47:0/3560781747 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe402caf0 con 0x7f2fd4003970 2024-03-23T09:52:05.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.234+0000 7f2ff0651640 1 -- 172.21.15.47:0/3560781747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fd4003970 msgr2=0x7f2fd4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.234+0000 7f2ff0651640 1 --2- 172.21.15.47:0/3560781747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fd4003970 0x7f2fd4003d90 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c240 tx=0x7f2fd40057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.234+0000 7f2ff0651640 1 -- 172.21.15.47:0/3560781747 shutdown_connections 2024-03-23T09:52:05.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.234+0000 7f2ff0651640 1 --2- 172.21.15.47:0/3560781747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fd4003970 0x7f2fd4003d90 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.234+0000 7f2ff0651640 1 -- 172.21.15.47:0/3560781747 >> 172.21.15.47:0/3560781747 conn(0x7f2fe806b160 msgr2=0x7f2fe806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:05.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.234+0000 7f2ff0651640 1 -- 172.21.15.47:0/3560781747 shutdown_connections 2024-03-23T09:52:05.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.234+0000 7f2ff0651640 1 -- 172.21.15.47:0/3560781747 wait complete. 2024-03-23T09:52:05.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.235+0000 7f2ff0651640 1 Processor -- start 2024-03-23T09:52:05.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.235+0000 7f2ff0651640 1 -- start start 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2ff0651640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fd4003970 0x7f2fe8150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2ff0651640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe81540d0 0x7f2fe80c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2ff0651640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe80c8b20 0x7f2fe8151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2ff0651640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe8154970 con 0x7f2fd4003970 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2ff0651640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe80c6150 con 0x7f2fe81540d0 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2ff0651640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe80c7e30 con 0x7f2fe80c8b20 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2fee3ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fd4003970 0x7f2fe8150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2fee3ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fd4003970 0x7f2fe8150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59328/0 (socket says 172.21.15.47:59328) 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2fee3ed640 1 -- 172.21.15.47:0/1925154124 learned_addr learned my addr 172.21.15.47:0/1925154124 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2fedbec640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe81540d0 0x7f2fe80c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2feebee640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe80c8b20 0x7f2fe8151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2fee3ed640 1 -- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe80c8b20 msgr2=0x7f2fe8151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2fee3ed640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe80c8b20 0x7f2fe8151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2fee3ed640 1 -- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe81540d0 msgr2=0x7f2fe80c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2fee3ed640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe81540d0 0x7f2fe80c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2fee3ed640 1 -- 172.21.15.47:0/1925154124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe8154690 con 0x7f2fd4003970 2024-03-23T09:52:05.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2fee3ed640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fd4003970 0x7f2fe8150910 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c240 tx=0x7f2fe402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:05.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2fdeffd640 1 -- 172.21.15.47:0/1925154124 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe403c040 con 0x7f2fd4003970 2024-03-23T09:52:05.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2fdeffd640 1 -- 172.21.15.47:0/1925154124 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe402ccc0 con 0x7f2fd4003970 2024-03-23T09:52:05.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2ff0651640 1 -- 172.21.15.47:0/1925154124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe80c8040 con 0x7f2fd4003970 2024-03-23T09:52:05.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.236+0000 7f2fdeffd640 1 -- 172.21.15.47:0/1925154124 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40072b0 con 0x7f2fd4003970 2024-03-23T09:52:05.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.237+0000 7f2ff0651640 1 -- 172.21.15.47:0/1925154124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe80c8600 con 0x7f2fd4003970 2024-03-23T09:52:05.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.237+0000 7f2ff0651640 1 -- 172.21.15.47:0/1925154124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe8072360 con 0x7f2fd4003970 2024-03-23T09:52:05.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.238+0000 7f2fdeffd640 1 -- 172.21.15.47:0/1925154124 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2fe4032070 con 0x7f2fd4003970 2024-03-23T09:52:05.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.238+0000 7f2fdeffd640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2fcc06ac00 0x7f2fcc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.238+0000 7f2fdeffd640 1 -- 172.21.15.47:0/1925154124 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2fe40aa7d0 con 0x7f2fd4003970 2024-03-23T09:52:05.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.239+0000 7f2fedbec640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2fcc06ac00 0x7f2fcc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.239+0000 7f2fedbec640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2fcc06ac00 0x7f2fcc06d0c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80b8020 tx=0x7f2fd8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:05.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.243+0000 7f2fdeffd640 1 -- 172.21.15.47:0/1925154124 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2fe4075990 con 0x7f2fd4003970 2024-03-23T09:52:05.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.459+0000 7f2ff0651640 1 -- 172.21.15.47:0/1925154124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2fe80c8790 con 0x7f2fd4003970 2024-03-23T09:52:05.459 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:52:05.460+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:52:05.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.473+0000 7f2fdeffd640 1 -- 172.21.15.47:0/1925154124 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2fe4046070 con 0x7f2fd4003970 2024-03-23T09:52:05.473 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:52:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.476+0000 7f2ff0651640 1 -- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2fcc06ac00 msgr2=0x7f2fcc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.476+0000 7f2ff0651640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2fcc06ac00 0x7f2fcc06d0c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80b8020 tx=0x7f2fd8009190 comp rx=0 tx=0).stop 2024-03-23T09:52:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.476+0000 7f2ff0651640 1 -- 172.21.15.47:0/1925154124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fd4003970 msgr2=0x7f2fe8150910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.476+0000 7f2ff0651640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fd4003970 0x7f2fe8150910 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c240 tx=0x7f2fe402e070 comp rx=0 tx=0).stop 2024-03-23T09:52:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.476+0000 7f2ff0651640 1 -- 172.21.15.47:0/1925154124 shutdown_connections 2024-03-23T09:52:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.476+0000 7f2ff0651640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f2fcc06ac00 0x7f2fcc06d0c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.476+0000 7f2ff0651640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe80c8b20 0x7f2fe8151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.476+0000 7f2ff0651640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe81540d0 0x7f2fe80c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.476+0000 7f2ff0651640 1 --2- 172.21.15.47:0/1925154124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fd4003970 0x7f2fe8150910 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.476+0000 7f2ff0651640 1 -- 172.21.15.47:0/1925154124 >> 172.21.15.47:0/1925154124 conn(0x7f2fe806b160 msgr2=0x7f2fe80c99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.476+0000 7f2ff0651640 1 -- 172.21.15.47:0/1925154124 shutdown_connections 2024-03-23T09:52:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.476+0000 7f2ff0651640 1 -- 172.21.15.47:0/1925154124 wait complete. 2024-03-23T09:52:05.477 INFO:tasks.ceph.mds.c.smithi047.stderr: -7652> 2024-03-23T09:51:42.013+0000 7f4068e8e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:52:05.488 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T09:52:05.490 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T09:52:05.508 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:52:05.538 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-03-23T09:52:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2024-03-23T09:52:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T09:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T09:52:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock profile = high_recovery_ops 2024-03-23T09:52:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2024-03-23T09:52:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: bdev async discard = True 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: bdev enable discard = True 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore allocator = bitmap 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 1/20 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 1/20 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 4/10 2024-03-23T09:52:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2024-03-23T09:52:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T09:52:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: # 1m isn't quite enough 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: mon_down_mkfs_grace = 2m 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_filestore_osds = false 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T09:52:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = "" 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds balancer = 20 2024-03-23T09:52:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2024-03-23T09:52:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = True 2024-03-23T09:52:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2024-03-23T09:52:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = False 2024-03-23T09:52:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-03-23T09:52:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-03-23T09:52:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-03-23T09:52:05.548 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2024-03-23T09:52:05.548 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2024-03-23T09:52:05.548 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T09:52:05.548 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T09:52:05.549 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.550 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 600 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock profile = high_recovery_ops 2024-03-23T09:52:05.551 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.552 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: bdev async discard = True 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: bdev enable discard = True 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore allocator = bitmap 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore block size = 96636764160 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore fsck on mount = True 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluefs = 1/20 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluestore = 1/20 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 20 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: debug rocksdb = 4/10 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd full ratio = 0.9 2024-03-23T09:52:05.553 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd nearfull ratio = 0.8 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: osd failsafe full ratio = 0.95 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = bluestore 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:05.554 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T09:52:05.555 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: # 1m isn't quite enough 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: mon_down_mkfs_grace = 2m 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_filestore_osds = false 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: mon op complaint time = 120 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: client mount timeout = 600 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: debug client = 20 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = "" 2024-03-23T09:52:05.556 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout:[mds] 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds = 20 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds balancer = 20 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug frag = True 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug scatterstat = True 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout: mds op complaint time = 180 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout: mds verify scatter = False 2024-03-23T09:52:05.557 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-03-23T09:52:05.558 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-03-23T09:52:05.558 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-03-23T09:52:05.561 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T09:52:05.601 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T09:52:05.648 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-03-23T09:52:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2024-03-23T09:52:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T09:52:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2024-03-23T09:52:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2024-03-23T09:52:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2024-03-23T09:52:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2024-03-23T09:52:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2024-03-23T09:52:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2024-03-23T09:52:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2024-03-23T09:52:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T09:52:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2024-03-23T09:52:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T09:52:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T09:52:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2024-03-23T09:52:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2024-03-23T09:52:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2024-03-23T09:52:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2024-03-23T09:52:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2024-03-23T09:52:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2024-03-23T09:52:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-03-23T09:52:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:52:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.652 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2024-03-23T09:52:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2024-03-23T09:52:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2024-03-23T09:52:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2024-03-23T09:52:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock profile = high_recovery_ops 2024-03-23T09:52:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: bdev async discard = True 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: bdev enable discard = True 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore allocator = bitmap 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 1/20 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 1/20 2024-03-23T09:52:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 4/10 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T09:52:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: # 1m isn't quite enough 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: mon_down_mkfs_grace = 2m 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_filestore_osds = false 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2024-03-23T09:52:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = "" 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2024-03-23T09:52:05.658 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2024-03-23T09:52:05.659 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2024-03-23T09:52:05.659 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2024-03-23T09:52:05.659 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2024-03-23T09:52:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2024-03-23T09:52:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds balancer = 20 2024-03-23T09:52:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2024-03-23T09:52:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = False 2024-03-23T09:52:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2024-03-23T09:52:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = False 2024-03-23T09:52:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-03-23T09:52:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-03-23T09:52:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-03-23T09:52:05.660 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2024-03-23T09:52:05.660 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2024-03-23T09:52:05.660 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T09:52:05.660 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2024-03-23T09:52:05.660 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.660 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2024-03-23T09:52:05.660 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.660 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2024-03-23T09:52:05.660 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.661 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2024-03-23T09:52:05.662 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 600 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock profile = high_recovery_ops 2024-03-23T09:52:05.663 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2024-03-23T09:52:05.664 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: bdev async discard = True 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: bdev enable discard = True 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore allocator = bitmap 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore block size = 96636764160 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore fsck on mount = True 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluefs = 1/20 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluestore = 1/20 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 20 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: debug rocksdb = 4/10 2024-03-23T09:52:05.665 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T09:52:05.666 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd full ratio = 0.9 2024-03-23T09:52:05.666 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd nearfull ratio = 0.8 2024-03-23T09:52:05.666 INFO:teuthology.orchestra.run.smithi175.stdout: osd failsafe full ratio = 0.95 2024-03-23T09:52:05.666 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = bluestore 2024-03-23T09:52:05.666 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-03-23T09:52:05.666 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.666 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2024-03-23T09:52:05.666 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:05.666 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2024-03-23T09:52:05.666 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-03-23T09:52:05.666 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-03-23T09:52:05.666 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:05.666 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:05.667 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2024-03-23T09:52:05.667 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.667 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2024-03-23T09:52:05.667 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:05.667 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-03-23T09:52:05.667 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2024-03-23T09:52:05.667 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-03-23T09:52:05.667 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2024-03-23T09:52:05.667 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2024-03-23T09:52:05.667 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:05.667 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: # 1m isn't quite enough 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: mon_down_mkfs_grace = 2m 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.668 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_filestore_osds = false 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout: mon op complaint time = 120 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout: client mount timeout = 600 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout: debug client = 20 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = "" 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2024-03-23T09:52:05.669 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout:[mds] 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds = 20 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds balancer = 20 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug frag = True 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug scatterstat = False 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout: mds op complaint time = 180 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout: mds verify scatter = False 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-03-23T09:52:05.670 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-03-23T09:52:05.671 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-03-23T09:52:05.672 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-03-23T09:52:05.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.753+0000 7fdbab5d6f40 1 Processor -- start 2024-03-23T09:52:05.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.753+0000 7fdbab5d6f40 1 -- start start 2024-03-23T09:52:05.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.753+0000 7fdbab5d6f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f65f314a0 0x558f65fa5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.753+0000 7fdbab5d6f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f65fa1810 0x558f65fa45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.753+0000 7fdbab5d6f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f65fbcba0 0x558f65fb4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.753+0000 7fdbab5d6f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f65e2fb60 con 0x558f65fa1810 2024-03-23T09:52:05.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.753+0000 7fdbab5d6f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f65e2f800 con 0x558f65fbcba0 2024-03-23T09:52:05.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.753+0000 7fdbab5d6f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f65e12520 con 0x558f65f314a0 2024-03-23T09:52:05.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.753+0000 7fdba88d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f65fa1810 0x558f65fa45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdba88d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f65fa1810 0x558f65fa45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59360/0 (socket says 172.21.15.47:59360) 2024-03-23T09:52:05.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdba88d2640 1 -- 172.21.15.47:0/876473188 learned_addr learned my addr 172.21.15.47:0/876473188 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:05.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdba98d4640 1 --2- 172.21.15.47:0/876473188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f65fbcba0 0x558f65fb4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdba90d3640 1 --2- 172.21.15.47:0/876473188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f65f314a0 0x558f65fa5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdba88d2640 1 -- 172.21.15.47:0/876473188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f65f314a0 msgr2=0x558f65fa5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdba88d2640 1 --2- 172.21.15.47:0/876473188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f65f314a0 0x558f65fa5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdba88d2640 1 -- 172.21.15.47:0/876473188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f65fbcba0 msgr2=0x558f65fb4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdba88d2640 1 --2- 172.21.15.47:0/876473188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f65fbcba0 0x558f65fb4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdba88d2640 1 -- 172.21.15.47:0/876473188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba0005d20 con 0x558f65fa1810 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdba88d2640 1 --2- 172.21.15.47:0/876473188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f65fa1810 0x558f65fa45e0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fdba0044910 tx=0x7fdba004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4072944442fef05c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdb9b7fe640 1 -- 172.21.15.47:0/876473188 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0052b10 con 0x558f65fa1810 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdb9b7fe640 1 -- 172.21.15.47:0/876473188 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba0052c90 con 0x558f65fa1810 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdb9b7fe640 1 -- 172.21.15.47:0/876473188 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba005c410 con 0x558f65fa1810 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.754+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/876473188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f65fa1810 msgr2=0x558f65fa45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.755+0000 7fdbab5d6f40 1 --2- 172.21.15.47:0/876473188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f65fa1810 0x558f65fa45e0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fdba0044910 tx=0x7fdba004c000 comp rx=0 tx=0).stop 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.755+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/876473188 shutdown_connections 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.755+0000 7fdbab5d6f40 1 --2- 172.21.15.47:0/876473188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f65fbcba0 0x558f65fb4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.755+0000 7fdbab5d6f40 1 --2- 172.21.15.47:0/876473188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f65fa1810 0x558f65fa45e0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.755+0000 7fdbab5d6f40 1 --2- 172.21.15.47:0/876473188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f65f314a0 0x558f65fa5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.755+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/876473188 >> 172.21.15.47:0/876473188 conn(0x558f65f12e40 msgr2=0x558f65fbb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.755+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/876473188 shutdown_connections 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.755+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/876473188 wait complete. 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.755+0000 7fdbab5d6f40 1 Processor -- start 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.755+0000 7fdbab5d6f40 1 -- start start 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdbab5d6f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f65f314a0 0x558f65fbb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdbab5d6f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f65fa1810 0x558f65fa54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdbab5d6f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f65fbcba0 0x558f65fa5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdbab5d6f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f65f30530 con 0x558f65fa1810 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdbab5d6f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f65f95560 con 0x558f65f314a0 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdbab5d6f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f65f2fbd0 con 0x558f65fbcba0 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdba88d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f65fa1810 0x558f65fa54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdba88d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f65fa1810 0x558f65fa54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59372/0 (socket says 172.21.15.47:59372) 2024-03-23T09:52:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdba88d2640 1 -- 172.21.15.47:0/2272026345 learned_addr learned my addr 172.21.15.47:0/2272026345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdba90d3640 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f65f314a0 0x558f65fbb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdba98d4640 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f65fbcba0 0x558f65fa5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdba88d2640 1 -- 172.21.15.47:0/2272026345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f65fbcba0 msgr2=0x558f65fa5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdba88d2640 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f65fbcba0 0x558f65fa5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdba88d2640 1 -- 172.21.15.47:0/2272026345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f65f314a0 msgr2=0x558f65fbb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdba88d2640 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f65f314a0 0x558f65fbb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.756+0000 7fdba88d2640 1 -- 172.21.15.47:0/2272026345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f65e58dc0 con 0x558f65fa1810 2024-03-23T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.757+0000 7fdba88d2640 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f65fa1810 0x558f65fa54c0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fdba005f040 tx=0x7fdba0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.757+0000 7fdb9a7fc640 1 -- 172.21.15.47:0/2272026345 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0065650 con 0x558f65fa1810 2024-03-23T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.757+0000 7fdb9a7fc640 1 -- 172.21.15.47:0/2272026345 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba0052b20 con 0x558f65fa1810 2024-03-23T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.757+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/2272026345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558f65e3ec20 con 0x558f65fa1810 2024-03-23T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.757+0000 7fdb9a7fc640 1 -- 172.21.15.47:0/2272026345 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba00659d0 con 0x558f65fa1810 2024-03-23T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.757+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/2272026345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f65e32060 con 0x558f65fa1810 2024-03-23T09:52:05.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.758+0000 7fdb9a7fc640 1 -- 172.21.15.47:0/2272026345 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdba0059070 con 0x558f65fa1810 2024-03-23T09:52:05.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.758+0000 7fdb9a7fc640 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdb8c069610 0x7fdb8c06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.759+0000 7fdb9a7fc640 1 -- 172.21.15.47:0/2272026345 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fdba00ca0b0 con 0x558f65fa1810 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.759+0000 7fdba90d3640 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdb8c069610 0x7fdb8c06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.759+0000 7fdbab5d6f40 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x558f65f172b0 0x558f65f17700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.759+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/2272026345 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x558f65f17c40 con 0x558f65f172b0 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.759+0000 7fdba98d4640 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x558f65f172b0 0x558f65f17700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.759+0000 7fdba98d4640 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x558f65f172b0 0x558f65f17700 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.759+0000 7fdba90d3640 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdb8c069610 0x7fdb8c06bad0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c000f60 tx=0x7fdb9c001770 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.762+0000 7fdba98d4640 1 -- 172.21.15.47:0/2272026345 <== osd.3 v2:172.21.15.47:6808/37683 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v117'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fdba400a9d0 con 0x558f65f172b0 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.762+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/2272026345 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x558f65f172b0 msgr2=0x558f65f17700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.762+0000 7fdbab5d6f40 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x558f65f172b0 0x558f65f17700 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.762+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/2272026345 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdb8c069610 msgr2=0x7fdb8c06bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.762+0000 7fdbab5d6f40 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fdb8c069610 0x7fdb8c06bad0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c000f60 tx=0x7fdb9c001770 comp rx=0 tx=0).stop 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.762+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/2272026345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f65fa1810 msgr2=0x558f65fa54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.762+0000 7fdbab5d6f40 1 --2- 172.21.15.47:0/2272026345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f65fa1810 0x558f65fa54c0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fdba005f040 tx=0x7fdba0002fb0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.762+0000 7fdba98d4640 1 -- 172.21.15.47:0/2272026345 reap_dead start 2024-03-23T09:52:05.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.762+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/2272026345 shutdown_connections 2024-03-23T09:52:05.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.762+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/2272026345 >> 172.21.15.47:0/2272026345 conn(0x558f65f12e40 msgr2=0x558f65fbba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:05.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.762+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/2272026345 shutdown_connections 2024-03-23T09:52:05.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.762+0000 7fdbab5d6f40 1 -- 172.21.15.47:0/2272026345 wait complete. 2024-03-23T09:52:05.765 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-23T09:52:05.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.909+0000 7f40bbb38640 1 Processor -- start 2024-03-23T09:52:05.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.910+0000 7f40bbb38640 1 -- start start 2024-03-23T09:52:05.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.910+0000 7f40bbb38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b40cb360 0x7f40b4161000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.910+0000 7f40bbb38640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f40b40b0810 0x7f40b40b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.910+0000 7f40bbb38640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b40d0810 con 0x7f40b40caf80 2024-03-23T09:52:05.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.910+0000 7f40bbb38640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b40b1200 con 0x7f40b40b0810 2024-03-23T09:52:05.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.910+0000 7f40bbb38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b4160b70 con 0x7f40b40cb360 2024-03-23T09:52:05.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.911+0000 7f40ba0d5640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f40b40b0810 0x7f40b40b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.911+0000 7f40b90d3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b40cb360 0x7f40b4161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.911+0000 7f40b90d3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b40cb360 0x7f40b4161000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:55554/0 (socket says 172.21.15.47:55554) 2024-03-23T09:52:05.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.911+0000 7f40b90d3640 1 -- 172.21.15.47:0/216149316 learned_addr learned my addr 172.21.15.47:0/216149316 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:05.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.911+0000 7f40b90d3640 1 -- 172.21.15.47:0/216149316 >> v2:172.21.15.175:3300/0 conn(0x7f40b40b0810 msgr2=0x7f40b40b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:05.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.911+0000 7f40b90d3640 1 --2- 172.21.15.47:0/216149316 >> v2:172.21.15.175:3300/0 conn(0x7f40b40b0810 0x7f40b40b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.911+0000 7f40b90d3640 1 -- 172.21.15.47:0/216149316 >> v1:172.21.15.175:6789/0 conn(0x7f40b40caf80 legacy=0x7f40b40cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.911+0000 7f40b90d3640 1 -- 172.21.15.47:0/216149316 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b40cef80 con 0x7f40b40cb360 2024-03-23T09:52:05.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.912+0000 7f40b90d3640 1 --2- 172.21.15.47:0/216149316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b40cb360 0x7f40b4161000 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009870 tx=0x7f40a402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5dee0197f1c8b34 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:05.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.912+0000 7f40b88d2640 1 -- 172.21.15.47:0/216149316 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4030840 con 0x7f40b40cb360 2024-03-23T09:52:05.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.912+0000 7f40b88d2640 1 -- 172.21.15.47:0/216149316 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a40309a0 con 0x7f40b40cb360 2024-03-23T09:52:05.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.912+0000 7f40b88d2640 1 -- 172.21.15.47:0/216149316 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4030d30 con 0x7f40b40cb360 2024-03-23T09:52:05.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.912+0000 7f40bbb38640 1 -- 172.21.15.47:0/216149316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b40cb360 msgr2=0x7f40b4161000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.912+0000 7f40bbb38640 1 --2- 172.21.15.47:0/216149316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b40cb360 0x7f40b4161000 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009870 tx=0x7f40a402b980 comp rx=0 tx=0).stop 2024-03-23T09:52:05.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.913+0000 7f40bbb38640 1 -- 172.21.15.47:0/216149316 shutdown_connections 2024-03-23T09:52:05.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.913+0000 7f40bbb38640 1 --2- 172.21.15.47:0/216149316 >> v2:172.21.15.175:3300/0 conn(0x7f40b40b0810 0x7f40b40b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.913+0000 7f40bbb38640 1 --2- 172.21.15.47:0/216149316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b40cb360 0x7f40b4161000 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.913+0000 7f40bbb38640 1 -- 172.21.15.47:0/216149316 >> 172.21.15.47:0/216149316 conn(0x7f40b4098a00 msgr2=0x7f40b40cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:05.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.913+0000 7f40bbb38640 1 -- 172.21.15.47:0/216149316 shutdown_connections 2024-03-23T09:52:05.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.913+0000 7f40bbb38640 1 -- 172.21.15.47:0/216149316 wait complete. 2024-03-23T09:52:05.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.914+0000 7f40bbb38640 1 Processor -- start 2024-03-23T09:52:05.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.914+0000 7f40bbb38640 1 -- start start 2024-03-23T09:52:05.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.914+0000 7f40bbb38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b40b0810 0x7f40b40c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.914+0000 7f40bbb38640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b40caf80 0x7f40b41f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.914+0000 7f40bbb38640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b40cb360 0x7f40b40c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.914+0000 7f40bbb38640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b40c73a0 con 0x7f40b40caf80 2024-03-23T09:52:05.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.914+0000 7f40bbb38640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b40c7680 con 0x7f40b40cb360 2024-03-23T09:52:05.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.914+0000 7f40bbb38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b40ac6d0 con 0x7f40b40b0810 2024-03-23T09:52:05.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40b90d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b40caf80 0x7f40b41f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40b90d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b40caf80 0x7f40b41f1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59376/0 (socket says 172.21.15.47:59376) 2024-03-23T09:52:05.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40b90d3640 1 -- 172.21.15.47:0/4251892082 learned_addr learned my addr 172.21.15.47:0/4251892082 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:05.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40b98d4640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b40b0810 0x7f40b40c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40ba0d5640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b40cb360 0x7f40b40c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40b90d3640 1 -- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b40b0810 msgr2=0x7f40b40c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40b90d3640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b40b0810 0x7f40b40c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40b90d3640 1 -- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b40cb360 msgr2=0x7f40b40c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:05.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40b90d3640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b40cb360 0x7f40b40c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:05.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40b90d3640 1 -- 172.21.15.47:0/4251892082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a40094d0 con 0x7f40b40caf80 2024-03-23T09:52:05.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40b90d3640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b40caf80 0x7f40b41f1cb0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009870 tx=0x7f40a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:05.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40aa7fc640 1 -- 172.21.15.47:0/4251892082 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4039c90 con 0x7f40b40caf80 2024-03-23T09:52:05.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40aa7fc640 1 -- 172.21.15.47:0/4251892082 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a4039df0 con 0x7f40b40caf80 2024-03-23T09:52:05.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40bbb38640 1 -- 172.21.15.47:0/4251892082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b40ac9b0 con 0x7f40b40caf80 2024-03-23T09:52:05.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40aa7fc640 1 -- 172.21.15.47:0/4251892082 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4043640 con 0x7f40b40caf80 2024-03-23T09:52:05.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.915+0000 7f40bbb38640 1 -- 172.21.15.47:0/4251892082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b41f21f0 con 0x7f40b40caf80 2024-03-23T09:52:05.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.916+0000 7f40bbb38640 1 -- 172.21.15.47:0/4251892082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b40ac530 con 0x7f40b40caf80 2024-03-23T09:52:05.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.917+0000 7f40aa7fc640 1 -- 172.21.15.47:0/4251892082 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f40a40437a0 con 0x7f40b40caf80 2024-03-23T09:52:05.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.917+0000 7f40aa7fc640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806ac00 0x7f409806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:05.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.917+0000 7f40aa7fc640 1 -- 172.21.15.47:0/4251892082 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f40a403e070 con 0x7f40b40caf80 2024-03-23T09:52:05.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.917+0000 7f40b98d4640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806ac00 0x7f409806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:05.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.918+0000 7f40b98d4640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806ac00 0x7f409806d0c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f40ac000f60 tx=0x7f40ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:05.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:05.922+0000 7f40aa7fc640 1 -- 172.21.15.47:0/4251892082 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f40b40ac530 con 0x7f40b40caf80 2024-03-23T09:52:06.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.141+0000 7f40bbb38640 1 -- 172.21.15.47:0/4251892082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f40b40ac530 con 0x7f40b40caf80 2024-03-23T09:52:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.474+0000 7f40aa7fc640 1 -- 172.21.15.47:0/4251892082 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v82) v1 ==== 162+0+0 (secure 0 0 0) 0x7f40b40ac530 con 0x7f40b40caf80 2024-03-23T09:52:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-23T09:52:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.477+0000 7f40bbb38640 1 -- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806ac00 msgr2=0x7f409806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.477+0000 7f40bbb38640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806ac00 0x7f409806d0c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f40ac000f60 tx=0x7f40ac009040 comp rx=0 tx=0).stop 2024-03-23T09:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.477+0000 7f40bbb38640 1 -- 172.21.15.47:0/4251892082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b40caf80 msgr2=0x7f40b41f1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.477+0000 7f40bbb38640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b40caf80 0x7f40b41f1cb0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009870 tx=0x7f40a4004730 comp rx=0 tx=0).stop 2024-03-23T09:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.477+0000 7f40bbb38640 1 -- 172.21.15.47:0/4251892082 shutdown_connections 2024-03-23T09:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.477+0000 7f40bbb38640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f409806ac00 0x7f409806d0c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.477+0000 7f40bbb38640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b40cb360 0x7f40b40c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.477+0000 7f40bbb38640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b40caf80 0x7f40b41f1cb0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.477+0000 7f40bbb38640 1 --2- 172.21.15.47:0/4251892082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b40b0810 0x7f40b40c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.477+0000 7f40bbb38640 1 -- 172.21.15.47:0/4251892082 >> 172.21.15.47:0/4251892082 conn(0x7f40b4098a00 msgr2=0x7f40b406e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.477+0000 7f40bbb38640 1 -- 172.21.15.47:0/4251892082 shutdown_connections 2024-03-23T09:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.477+0000 7f40bbb38640 1 -- 172.21.15.47:0/4251892082 wait complete. 2024-03-23T09:52:06.491 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:52:06.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.593+0000 7f0b2d7fb640 1 Processor -- start 2024-03-23T09:52:06.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.593+0000 7f0b2d7fb640 1 -- start start 2024-03-23T09:52:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.594+0000 7f0b2d7fb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0b2815ca90 0x7f0b28160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.594+0000 7f0b2d7fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b280cde10 0x7f0b280d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.594+0000 7f0b2d7fb640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b280d11e0 con 0x7f0b280b8350 2024-03-23T09:52:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.594+0000 7f0b2d7fb640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b28160f60 con 0x7f0b2815ca90 2024-03-23T09:52:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.594+0000 7f0b2d7fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b28161240 con 0x7f0b280cde10 2024-03-23T09:52:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.594+0000 7f0b26ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0b2815ca90 0x7f0b28160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.594+0000 7f0b26ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0b2815ca90 0x7f0b28160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59384/0 (socket says 172.21.15.47:59384) 2024-03-23T09:52:06.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.594+0000 7f0b26ffd640 1 -- 172.21.15.47:0/625914243 learned_addr learned my addr 172.21.15.47:0/625914243 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:06.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.594+0000 7f0b277fe640 1 --2- 172.21.15.47:0/625914243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b280cde10 0x7f0b280d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:06.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.594+0000 7f0b26ffd640 1 -- 172.21.15.47:0/625914243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b280cde10 msgr2=0x7f0b280d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:06.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.594+0000 7f0b26ffd640 1 --2- 172.21.15.47:0/625914243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b280cde10 0x7f0b280d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.595+0000 7f0b26ffd640 1 -- 172.21.15.47:0/625914243 >> v1:172.21.15.175:6789/0 conn(0x7f0b280b8350 legacy=0x7f0b280cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:06.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.595+0000 7f0b26ffd640 1 -- 172.21.15.47:0/625914243 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b28161520 con 0x7f0b2815ca90 2024-03-23T09:52:06.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.595+0000 7f0b26ffd640 1 --2- 172.21.15.47:0/625914243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b2815ca90 0x7f0b28160710 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009870 tx=0x7f0b1802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f02163f07fd4e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.595+0000 7f0b267fc640 1 -- 172.21.15.47:0/625914243 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18031640 con 0x7f0b2815ca90 2024-03-23T09:52:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.595+0000 7f0b267fc640 1 -- 172.21.15.47:0/625914243 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b180317a0 con 0x7f0b2815ca90 2024-03-23T09:52:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.595+0000 7f0b267fc640 1 -- 172.21.15.47:0/625914243 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18031b30 con 0x7f0b2815ca90 2024-03-23T09:52:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.596+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/625914243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b2815ca90 msgr2=0x7f0b28160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.596+0000 7f0b2d7fb640 1 --2- 172.21.15.47:0/625914243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b2815ca90 0x7f0b28160710 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009870 tx=0x7f0b1802b8f0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.596+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/625914243 shutdown_connections 2024-03-23T09:52:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.596+0000 7f0b2d7fb640 1 --2- 172.21.15.47:0/625914243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b280cde10 0x7f0b280d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.596+0000 7f0b2d7fb640 1 --2- 172.21.15.47:0/625914243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b2815ca90 0x7f0b28160710 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.596+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/625914243 >> 172.21.15.47:0/625914243 conn(0x7f0b280659d0 msgr2=0x7f0b28065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.596+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/625914243 shutdown_connections 2024-03-23T09:52:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.596+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/625914243 wait complete. 2024-03-23T09:52:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.597+0000 7f0b2d7fb640 1 Processor -- start 2024-03-23T09:52:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.597+0000 7f0b2d7fb640 1 -- start start 2024-03-23T09:52:06.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b2d7fb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b280b8350 0x7f0b2815acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:06.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b2d7fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b280cde10 0x7f0b280b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:06.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b2d7fb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b2815ca90 0x7f0b2814edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:06.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b2d7fb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b280b4b30 con 0x7f0b2815ca90 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b2d7fb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b281589e0 con 0x7f0b280b8350 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b2d7fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b280b4410 con 0x7f0b280cde10 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b277fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b2815ca90 0x7f0b2814edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b277fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b2815ca90 0x7f0b2814edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59396/0 (socket says 172.21.15.47:59396) 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b277fe640 1 -- 172.21.15.47:0/3586299441 learned_addr learned my addr 172.21.15.47:0/3586299441 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b1ffff640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b280cde10 0x7f0b280b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b26ffd640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b280b8350 0x7f0b2815acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b277fe640 1 -- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b280cde10 msgr2=0x7f0b280b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b277fe640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b280cde10 0x7f0b280b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b277fe640 1 -- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b280b8350 msgr2=0x7f0b2815acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b277fe640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b280b8350 0x7f0b2815acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b277fe640 1 -- 172.21.15.47:0/3586299441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b180094d0 con 0x7f0b2815ca90 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b26ffd640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b280b8350 0x7f0b2815acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.598+0000 7f0b277fe640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b2815ca90 0x7f0b2814edb0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007e10 tx=0x7f0b2000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.599+0000 7f0b1f7fe640 1 -- 172.21.15.47:0/3586299441 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20014620 con 0x7f0b2815ca90 2024-03-23T09:52:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.599+0000 7f0b1f7fe640 1 -- 172.21.15.47:0/3586299441 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b200147a0 con 0x7f0b2815ca90 2024-03-23T09:52:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.599+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/3586299441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b280b3b70 con 0x7f0b2815ca90 2024-03-23T09:52:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.599+0000 7f0b1f7fe640 1 -- 172.21.15.47:0/3586299441 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20014b50 con 0x7f0b2815ca90 2024-03-23T09:52:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.599+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/3586299441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b280b4130 con 0x7f0b2815ca90 2024-03-23T09:52:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.600+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/3586299441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b28072360 con 0x7f0b2815ca90 2024-03-23T09:52:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.600+0000 7f0b1f7fe640 1 -- 172.21.15.47:0/3586299441 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0b20014cb0 con 0x7f0b2815ca90 2024-03-23T09:52:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.601+0000 7f0b1f7fe640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b0806ac00 0x7f0b0806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.601+0000 7f0b1f7fe640 1 -- 172.21.15.47:0/3586299441 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0b20089c60 con 0x7f0b2815ca90 2024-03-23T09:52:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.601+0000 7f0b26ffd640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b0806ac00 0x7f0b0806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.601+0000 7f0b26ffd640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b0806ac00 0x7f0b0806d0c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0b18036000 tx=0x7f0b1803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.605+0000 7f0b1f7fe640 1 -- 172.21.15.47:0/3586299441 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0b20054e40 con 0x7f0b2815ca90 2024-03-23T09:52:06.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.821+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/3586299441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b28066be0 con 0x7f0b2815ca90 2024-03-23T09:52:06.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.821+0000 7f0b1f7fe640 1 -- 172.21.15.47:0/3586299441 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f0b20058460 con 0x7f0b2815ca90 2024-03-23T09:52:06.822 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:06.822 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":83,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:51:40.395539+0000","modified":"2024-03-23T09:52:06.475714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9570},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9570":{"gid":9570,"name":"c","rank":0,"incarnation":83,"state":"up:replay","state_seq":1,"addr":"172.21.15.47:6834/1868958629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1868958629},{"type":"v1","addr":"172.21.15.47:6834","nonce":1868958629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-23T09:52:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 83 2024-03-23T09:52:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.823+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b0806ac00 msgr2=0x7f0b0806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.823+0000 7f0b2d7fb640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b0806ac00 0x7f0b0806d0c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0b18036000 tx=0x7f0b1803c040 comp rx=0 tx=0).stop 2024-03-23T09:52:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.823+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/3586299441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b2815ca90 msgr2=0x7f0b2814edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.823+0000 7f0b2d7fb640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b2815ca90 0x7f0b2814edb0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007e10 tx=0x7f0b2000e4b0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.823+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/3586299441 shutdown_connections 2024-03-23T09:52:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.823+0000 7f0b2d7fb640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0b0806ac00 0x7f0b0806d0c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.823+0000 7f0b2d7fb640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b2815ca90 0x7f0b2814edb0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.823+0000 7f0b2d7fb640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b280cde10 0x7f0b280b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.823+0000 7f0b2d7fb640 1 --2- 172.21.15.47:0/3586299441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b280b8350 0x7f0b2815acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.823+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/3586299441 >> 172.21.15.47:0/3586299441 conn(0x7f0b280659d0 msgr2=0x7f0b281571a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.823+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/3586299441 shutdown_connections 2024-03-23T09:52:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:06.823+0000 7f0b2d7fb640 1 -- 172.21.15.47:0/3586299441 wait complete. 2024-03-23T09:52:07.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:51:40.395539+0000', 'modified': '2024-03-23T09:52:06.475714+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9570}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9570': {'gid': 9570, 'name': 'c', 'rank': 0, 'incarnation': 83, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.47:6834/1868958629', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 1868958629}, {'type': 'v1', 'addr': '172.21.15.47:6834', 'nonce': 1868958629}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:52:07.237 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9570:up:replay 2024-03-23T09:52:08.238 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:52:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.351+0000 7f015ebd2640 1 Processor -- start 2024-03-23T09:52:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.351+0000 7f015ebd2640 1 -- start start 2024-03-23T09:52:08.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.351+0000 7f015ebd2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01580b8300 0x7f01580cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:08.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.351+0000 7f015ebd2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01580cf6d0 0x7f01580d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:08.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.351+0000 7f015ebd2640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01580d0ee0 con 0x7f015815cb40 2024-03-23T09:52:08.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.351+0000 7f015ebd2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01580b5a10 con 0x7f01580cf6d0 2024-03-23T09:52:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.351+0000 7f015ebd2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01580cfb20 con 0x7f01580b8300 2024-03-23T09:52:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.351+0000 7f015d16f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01580cf6d0 0x7f01580d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.352+0000 7f015d16f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01580cf6d0 0x7f01580d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59408/0 (socket says 172.21.15.47:59408) 2024-03-23T09:52:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.352+0000 7f015d16f640 1 -- 172.21.15.47:0/4189985575 learned_addr learned my addr 172.21.15.47:0/4189985575 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.352+0000 7f0157fff640 1 --2- 172.21.15.47:0/4189985575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01580b8300 0x7f01580cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.352+0000 7f015d16f640 1 -- 172.21.15.47:0/4189985575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01580b8300 msgr2=0x7f01580cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.352+0000 7f015d16f640 1 --2- 172.21.15.47:0/4189985575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01580b8300 0x7f01580cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.352+0000 7f015d16f640 1 -- 172.21.15.47:0/4189985575 >> v1:172.21.15.175:6789/0 conn(0x7f015815cb40 legacy=0x7f0158160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.352+0000 7f015d16f640 1 -- 172.21.15.47:0/4189985575 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01581616f0 con 0x7f01580cf6d0 2024-03-23T09:52:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.353+0000 7f015d16f640 1 --2- 172.21.15.47:0/4189985575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01580cf6d0 0x7f01580d1370 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f0150009e50 tx=0x7f015002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b21d23a865d0eab server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.353+0000 7f01577fe640 1 -- 172.21.15.47:0/4189985575 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015002cb00 con 0x7f01580cf6d0 2024-03-23T09:52:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.353+0000 7f01577fe640 1 -- 172.21.15.47:0/4189985575 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015002cc60 con 0x7f01580cf6d0 2024-03-23T09:52:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.353+0000 7f01577fe640 1 -- 172.21.15.47:0/4189985575 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500343f0 con 0x7f01580cf6d0 2024-03-23T09:52:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.353+0000 7f015ebd2640 1 -- 172.21.15.47:0/4189985575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01580cf6d0 msgr2=0x7f01580d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.353+0000 7f015ebd2640 1 --2- 172.21.15.47:0/4189985575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01580cf6d0 0x7f01580d1370 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f0150009e50 tx=0x7f015002bf60 comp rx=0 tx=0).stop 2024-03-23T09:52:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.353+0000 7f015ebd2640 1 -- 172.21.15.47:0/4189985575 shutdown_connections 2024-03-23T09:52:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.353+0000 7f015ebd2640 1 --2- 172.21.15.47:0/4189985575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01580cf6d0 0x7f01580d1370 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.353+0000 7f015ebd2640 1 --2- 172.21.15.47:0/4189985575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01580b8300 0x7f01580cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.353+0000 7f015ebd2640 1 -- 172.21.15.47:0/4189985575 >> 172.21.15.47:0/4189985575 conn(0x7f015806b160 msgr2=0x7f015806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.353+0000 7f015ebd2640 1 -- 172.21.15.47:0/4189985575 shutdown_connections 2024-03-23T09:52:08.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.353+0000 7f015ebd2640 1 -- 172.21.15.47:0/4189985575 wait complete. 2024-03-23T09:52:08.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.355+0000 7f015ebd2640 1 Processor -- start 2024-03-23T09:52:08.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.355+0000 7f015ebd2640 1 -- start start 2024-03-23T09:52:08.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.355+0000 7f015ebd2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01580b8300 0x7f01580cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f015ebd2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01580cf6d0 0x7f01580ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f015ebd2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015815cb40 0x7f0158153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f015ebd2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01580cb3b0 con 0x7f01580cf6d0 2024-03-23T09:52:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f015ebd2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01580ca900 con 0x7f015815cb40 2024-03-23T09:52:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f015ebd2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01580cab10 con 0x7f01580b8300 2024-03-23T09:52:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f0157fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01580cf6d0 0x7f01580ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f0157fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01580cf6d0 0x7f01580ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59412/0 (socket says 172.21.15.47:59412) 2024-03-23T09:52:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f0157fff640 1 -- 172.21.15.47:0/217280902 learned_addr learned my addr 172.21.15.47:0/217280902 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f015c96e640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01580b8300 0x7f01580cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f015d16f640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015815cb40 0x7f0158153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f0157fff640 1 -- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01580b8300 msgr2=0x7f01580cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f0157fff640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01580b8300 0x7f01580cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:08.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f0157fff640 1 -- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015815cb40 msgr2=0x7f0158153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:08.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f0157fff640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015815cb40 0x7f0158153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:08.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f0157fff640 1 -- 172.21.15.47:0/217280902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0150009b30 con 0x7f01580cf6d0 2024-03-23T09:52:08.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f015c96e640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01580b8300 0x7f01580cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:08.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f015d16f640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015815cb40 0x7f0158153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:08.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f0157fff640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01580cf6d0 0x7f01580ca3c0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f014400e8b0 tx=0x7f014400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:08.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f01557fa640 1 -- 172.21.15.47:0/217280902 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014400c940 con 0x7f01580cf6d0 2024-03-23T09:52:08.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f01557fa640 1 -- 172.21.15.47:0/217280902 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014400caa0 con 0x7f01580cf6d0 2024-03-23T09:52:08.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f015ebd2640 1 -- 172.21.15.47:0/217280902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01580cadf0 con 0x7f01580cf6d0 2024-03-23T09:52:08.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f01557fa640 1 -- 172.21.15.47:0/217280902 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014400ce30 con 0x7f01580cf6d0 2024-03-23T09:52:08.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.356+0000 7f015ebd2640 1 -- 172.21.15.47:0/217280902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01581f3880 con 0x7f01580cf6d0 2024-03-23T09:52:08.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.357+0000 7f015ebd2640 1 -- 172.21.15.47:0/217280902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0158072360 con 0x7f01580cf6d0 2024-03-23T09:52:08.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.358+0000 7f01557fa640 1 -- 172.21.15.47:0/217280902 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0144013470 con 0x7f01580cf6d0 2024-03-23T09:52:08.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.358+0000 7f01557fa640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f013c06ac00 0x7f013c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:08.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.358+0000 7f01557fa640 1 -- 172.21.15.47:0/217280902 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0144089560 con 0x7f01580cf6d0 2024-03-23T09:52:08.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.359+0000 7f015c96e640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f013c06ac00 0x7f013c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:08.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.359+0000 7f015c96e640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f013c06ac00 0x7f013c06d0c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f01581f5a70 tx=0x7f014c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:08.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.363+0000 7f01557fa640 1 -- 172.21.15.47:0/217280902 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0144054740 con 0x7f01580cf6d0 2024-03-23T09:52:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.581+0000 7f015ebd2640 1 -- 172.21.15.47:0/217280902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01581f36e0 con 0x7f01580cf6d0 2024-03-23T09:52:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.581+0000 7f01557fa640 1 -- 172.21.15.47:0/217280902 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f01581f36e0 con 0x7f01580cf6d0 2024-03-23T09:52:08.581 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":85,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:51:40.395539+0000","modified":"2024-03-23T09:52:08.482217+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9570},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9570":{"gid":9570,"name":"c","rank":0,"incarnation":83,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.47:6834/1868958629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1868958629},{"type":"v1","addr":"172.21.15.47:6834","nonce":1868958629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-23T09:52:08.582 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 85 2024-03-23T09:52:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.583+0000 7f015ebd2640 1 -- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f013c06ac00 msgr2=0x7f013c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.583+0000 7f015ebd2640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f013c06ac00 0x7f013c06d0c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f01581f5a70 tx=0x7f014c009040 comp rx=0 tx=0).stop 2024-03-23T09:52:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.583+0000 7f015ebd2640 1 -- 172.21.15.47:0/217280902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01580cf6d0 msgr2=0x7f01580ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.583+0000 7f015ebd2640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01580cf6d0 0x7f01580ca3c0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f014400e8b0 tx=0x7f014400ed70 comp rx=0 tx=0).stop 2024-03-23T09:52:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.583+0000 7f015ebd2640 1 -- 172.21.15.47:0/217280902 shutdown_connections 2024-03-23T09:52:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.583+0000 7f015ebd2640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f013c06ac00 0x7f013c06d0c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.583+0000 7f015ebd2640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015815cb40 0x7f0158153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.583+0000 7f015ebd2640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01580cf6d0 0x7f01580ca3c0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.583+0000 7f015ebd2640 1 --2- 172.21.15.47:0/217280902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01580b8300 0x7f01580cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.583+0000 7f015ebd2640 1 -- 172.21.15.47:0/217280902 >> 172.21.15.47:0/217280902 conn(0x7f015806b160 msgr2=0x7f01580cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.583+0000 7f015ebd2640 1 -- 172.21.15.47:0/217280902 shutdown_connections 2024-03-23T09:52:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:08.583+0000 7f015ebd2640 1 -- 172.21.15.47:0/217280902 wait complete. 2024-03-23T09:52:08.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:51:40.395539+0000', 'modified': '2024-03-23T09:52:08.482217+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9570}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9570': {'gid': 9570, 'name': 'c', 'rank': 0, 'incarnation': 83, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.47:6834/1868958629', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 1868958629}, {'type': 'v1', 'addr': '172.21.15.47:6834', 'nonce': 1868958629}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:52:08.997 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9570:up:rejoin 2024-03-23T09:52:09.102 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:52:09.105 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:52:09.999 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:52:10.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.111+0000 7fa726366640 1 Processor -- start 2024-03-23T09:52:10.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.112+0000 7fa726366640 1 -- start start 2024-03-23T09:52:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.112+0000 7fa726366640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72015cbe0 0x7fa7200ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.112+0000 7fa726366640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7200b1400 con 0x7fa7200cad40 2024-03-23T09:52:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.112+0000 7fa726366640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7200b16e0 con 0x7fa7200b0de0 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.112+0000 7fa726366640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200cedb0 con 0x7fa72015cbe0 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.112+0000 7fa71f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa7200cad40 0x7fa7200b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32858/0 (socket says 172.21.15.47:32858) 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.112+0000 7fa71f7fe640 1 -- 172.21.15.47:0/389603884 learned_addr learned my addr 172.21.15.47:0/389603884 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841378782 0 0) 0x7fa7200b1400 con 0x7fa7200cad40 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71ffff640 1 --2- 172.21.15.47:0/389603884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72015cbe0 0x7fa7200ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa708003520 con 0x7fa7200cad40 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2310772614 0 0) 0x7fa708003520 con 0x7fa7200cad40 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72015cbe0 msgr2=0x7fa7200ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 --2- 172.21.15.47:0/389603884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72015cbe0 0x7fa7200ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 >> v1:172.21.15.175:6789/0 conn(0x7fa7200b0de0 legacy=0x7fa7200cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7200cf090 con 0x7fa7200cad40 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fa714001670 con 0x7fa7200cad40 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 >> v1:172.21.15.47:6789/0 conn(0x7fa7200cad40 legacy=0x7fa7200b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 --2- 172.21.15.47:0/389603884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7080039a0 0x7fa708003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 --2- 172.21.15.47:0/389603884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72015cbe0 0x7fa7080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 --2- 172.21.15.47:0/389603884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa708004d00 0x7fa708005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200cedb0 con 0x7fa7080039a0 2024-03-23T09:52:10.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70800bab0 con 0x7fa708004d00 2024-03-23T09:52:10.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70800bc70 con 0x7fa72015cbe0 2024-03-23T09:52:10.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71ffff640 1 --2- 172.21.15.47:0/389603884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7080039a0 0x7fa708003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa71f7fe640 1 --2- 172.21.15.47:0/389603884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72015cbe0 0x7fa7080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.113+0000 7fa724903640 1 --2- 172.21.15.47:0/389603884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa708004d00 0x7fa708005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.114+0000 7fa71ffff640 1 -- 172.21.15.47:0/389603884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72015cbe0 msgr2=0x7fa7080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.114+0000 7fa71ffff640 1 --2- 172.21.15.47:0/389603884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72015cbe0 0x7fa7080047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.114+0000 7fa71ffff640 1 -- 172.21.15.47:0/389603884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa708004d00 msgr2=0x7fa708005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.114+0000 7fa71ffff640 1 --2- 172.21.15.47:0/389603884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa708004d00 0x7fa708005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.114+0000 7fa71ffff640 1 -- 172.21.15.47:0/389603884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa708003520 con 0x7fa7080039a0 2024-03-23T09:52:10.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.114+0000 7fa71ffff640 1 --2- 172.21.15.47:0/389603884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7080039a0 0x7fa708003dc0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fa710009ac0 tx=0x7fa7080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e93636b427d2dc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:10.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.114+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71002c600 con 0x7fa7080039a0 2024-03-23T09:52:10.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.115+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71002c760 con 0x7fa7080039a0 2024-03-23T09:52:10.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.115+0000 7fa71effd640 1 -- 172.21.15.47:0/389603884 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71002caf0 con 0x7fa7080039a0 2024-03-23T09:52:10.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.115+0000 7fa726366640 1 -- 172.21.15.47:0/389603884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7080039a0 msgr2=0x7fa708003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.115+0000 7fa726366640 1 --2- 172.21.15.47:0/389603884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7080039a0 0x7fa708003dc0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fa710009ac0 tx=0x7fa7080057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.115+0000 7fa724903640 1 -- 172.21.15.47:0/389603884 reap_dead start 2024-03-23T09:52:10.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.115+0000 7fa726366640 1 -- 172.21.15.47:0/389603884 shutdown_connections 2024-03-23T09:52:10.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.115+0000 7fa726366640 1 -- 172.21.15.47:0/389603884 >> 172.21.15.47:0/389603884 conn(0x7fa7200d06f0 msgr2=0x7fa7200d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:10.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.115+0000 7fa726366640 1 -- 172.21.15.47:0/389603884 shutdown_connections 2024-03-23T09:52:10.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.115+0000 7fa726366640 1 -- 172.21.15.47:0/389603884 wait complete. 2024-03-23T09:52:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.116+0000 7fa726366640 1 Processor -- start 2024-03-23T09:52:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.117+0000 7fa726366640 1 -- start start 2024-03-23T09:52:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.117+0000 7fa726366640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7080039a0 0x7fa7200b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.117+0000 7fa726366640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72014aee0 0x7fa72014d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.117+0000 7fa726366640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa72014dca0 0x7fa72014b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.117+0000 7fa726366640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200b5cd0 con 0x7fa7080039a0 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.117+0000 7fa726366640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72014e150 con 0x7fa72014dca0 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.117+0000 7fa726366640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200b4d40 con 0x7fa72014aee0 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.117+0000 7fa71ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7080039a0 0x7fa7200b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.117+0000 7fa71ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7080039a0 0x7fa7200b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59432/0 (socket says 172.21.15.47:59432) 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.117+0000 7fa71ffff640 1 -- 172.21.15.47:0/3693203381 learned_addr learned my addr 172.21.15.47:0/3693203381 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa71f7fe640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72014aee0 0x7fa72014d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa724903640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa72014dca0 0x7fa72014b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa71ffff640 1 -- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72014aee0 msgr2=0x7fa72014d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa71ffff640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72014aee0 0x7fa72014d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa71ffff640 1 -- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa72014dca0 msgr2=0x7fa72014b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa71ffff640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa72014dca0 0x7fa72014b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa71ffff640 1 -- 172.21.15.47:0/3693203381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7200ca1b0 con 0x7fa7080039a0 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa71f7fe640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72014aee0 0x7fa72014d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa724903640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa72014dca0 0x7fa72014b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa71ffff640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7080039a0 0x7fa7200b5790 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fa71002c1a0 tx=0x7fa710004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa71cff9640 1 -- 172.21.15.47:0/3693203381 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710003810 con 0x7fa7080039a0 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa71cff9640 1 -- 172.21.15.47:0/3693203381 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa710003970 con 0x7fa7080039a0 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa726366640 1 -- 172.21.15.47:0/3693203381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7200ca490 con 0x7fa7080039a0 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa71cff9640 1 -- 172.21.15.47:0/3693203381 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710003ce0 con 0x7fa7080039a0 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.118+0000 7fa726366640 1 -- 172.21.15.47:0/3693203381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7200b5270 con 0x7fa7080039a0 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.119+0000 7fa726366640 1 -- 172.21.15.47:0/3693203381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7200b5cd0 con 0x7fa7080039a0 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.120+0000 7fa71cff9640 1 -- 172.21.15.47:0/3693203381 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa710034440 con 0x7fa7080039a0 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.120+0000 7fa71cff9640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa70c06ac00 0x7fa70c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.120+0000 7fa71cff9640 1 -- 172.21.15.47:0/3693203381 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa7100ab2a0 con 0x7fa7080039a0 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.120+0000 7fa71f7fe640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa70c06ac00 0x7fa70c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.121+0000 7fa71f7fe640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa70c06ac00 0x7fa70c06d0c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa714000c00 tx=0x7fa714009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:10.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.124+0000 7fa71cff9640 1 -- 172.21.15.47:0/3693203381 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa7200b5cd0 con 0x7fa7080039a0 2024-03-23T09:52:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.340+0000 7fa726366640 1 -- 172.21.15.47:0/3693203381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7200b5cd0 con 0x7fa7080039a0 2024-03-23T09:52:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.340+0000 7fa71cff9640 1 -- 172.21.15.47:0/3693203381 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fa7200b5cd0 con 0x7fa7080039a0 2024-03-23T09:52:10.341 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:10.341 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":86,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:51:40.395539+0000","modified":"2024-03-23T09:52:09.483913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9570},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9570":{"gid":9570,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6834/1868958629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1868958629},{"type":"v1","addr":"172.21.15.47:6834","nonce":1868958629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-23T09:52:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 86 2024-03-23T09:52:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.342+0000 7fa726366640 1 -- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa70c06ac00 msgr2=0x7fa70c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.342+0000 7fa726366640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa70c06ac00 0x7fa70c06d0c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa714000c00 tx=0x7fa714009040 comp rx=0 tx=0).stop 2024-03-23T09:52:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.342+0000 7fa726366640 1 -- 172.21.15.47:0/3693203381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7080039a0 msgr2=0x7fa7200b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.342+0000 7fa726366640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7080039a0 0x7fa7200b5790 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fa71002c1a0 tx=0x7fa710004730 comp rx=0 tx=0).stop 2024-03-23T09:52:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.342+0000 7fa726366640 1 -- 172.21.15.47:0/3693203381 shutdown_connections 2024-03-23T09:52:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.342+0000 7fa726366640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa70c06ac00 0x7fa70c06d0c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.342+0000 7fa726366640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa72014dca0 0x7fa72014b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.342+0000 7fa726366640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa72014aee0 0x7fa72014d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.342+0000 7fa726366640 1 --2- 172.21.15.47:0/3693203381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7080039a0 0x7fa7200b5790 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.342+0000 7fa726366640 1 -- 172.21.15.47:0/3693203381 >> 172.21.15.47:0/3693203381 conn(0x7fa7200d06f0 msgr2=0x7fa7200cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.342+0000 7fa726366640 1 -- 172.21.15.47:0/3693203381 shutdown_connections 2024-03-23T09:52:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.342+0000 7fa726366640 1 -- 172.21.15.47:0/3693203381 wait complete. 2024-03-23T09:52:10.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:51:40.395539+0000', 'modified': '2024-03-23T09:52:09.483913+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9570}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9570': {'gid': 9570, 'name': 'c', 'rank': 0, 'incarnation': 83, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.47:6834/1868958629', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 1868958629}, {'type': 'v1', 'addr': '172.21.15.47:6834', 'nonce': 1868958629}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:52:10.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:52:10.756 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T09:52:10.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.860+0000 7f5b195e7640 1 Processor -- start 2024-03-23T09:52:10.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.860+0000 7f5b195e7640 1 -- start start 2024-03-23T09:52:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.861+0000 7f5b195e7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b140cc020 0x7f5b1415c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.861+0000 7f5b195e7640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b140cf030 con 0x7f5b14098a00 2024-03-23T09:52:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.861+0000 7f5b195e7640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b140cfdf0 con 0x7f5b14099ad0 2024-03-23T09:52:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.861+0000 7f5b195e7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140cf270 con 0x7f5b140cc020 2024-03-23T09:52:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.861+0000 7f5b12ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5b14098a00 0x7f5b14098de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32864/0 (socket says 172.21.15.47:32864) 2024-03-23T09:52:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.861+0000 7f5b12ffd640 1 -- 172.21.15.47:0/384355998 learned_addr learned my addr 172.21.15.47:0/384355998 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.861+0000 7f5b137fe640 1 --2- 172.21.15.47:0/384355998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b140cc020 0x7f5b1415c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.861+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643851311 0 0) 0x7f5b0c001110 con 0x7f5b14098a00 2024-03-23T09:52:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b00003520 con 0x7f5b14098a00 2024-03-23T09:52:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3272585953 0 0) 0x7f5b00003520 con 0x7f5b14098a00 2024-03-23T09:52:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b140cc020 msgr2=0x7f5b1415c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 --2- 172.21.15.47:0/384355998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b140cc020 0x7f5b1415c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 >> v1:172.21.15.175:6789/0 conn(0x7f5b14099ad0 legacy=0x7f5b140ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b140cf550 con 0x7f5b14098a00 2024-03-23T09:52:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f5b0c001800 con 0x7f5b14098a00 2024-03-23T09:52:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 >> v1:172.21.15.47:6789/0 conn(0x7f5b14098a00 legacy=0x7f5b14098de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 --2- 172.21.15.47:0/384355998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b000039a0 0x7f5b00003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 --2- 172.21.15.47:0/384355998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b140cc020 0x7f5b000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 --2- 172.21.15.47:0/384355998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b00004d00 0x7f5b00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140cf270 con 0x7f5b000039a0 2024-03-23T09:52:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0000b9b0 con 0x7f5b00004d00 2024-03-23T09:52:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0000bb70 con 0x7f5b140cc020 2024-03-23T09:52:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/384355998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b000039a0 0x7f5b00003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b137fe640 1 --2- 172.21.15.47:0/384355998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b00004d00 0x7f5b00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b127fc640 1 --2- 172.21.15.47:0/384355998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b140cc020 0x7f5b000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b12ffd640 1 -- 172.21.15.47:0/384355998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b140cc020 msgr2=0x7f5b000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/384355998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b140cc020 0x7f5b000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b12ffd640 1 -- 172.21.15.47:0/384355998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b00004d00 msgr2=0x7f5b00005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/384355998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b00004d00 0x7f5b00005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.862+0000 7f5b12ffd640 1 -- 172.21.15.47:0/384355998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b00003520 con 0x7f5b000039a0 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.863+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/384355998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b000039a0 0x7f5b00003d80 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00c240 tx=0x7f5b000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abf16bda3e03a6c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.863+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c005ad0 con 0x7f5b000039a0 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.863+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c005c50 con 0x7f5b000039a0 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.863+0000 7f5b11ffb640 1 -- 172.21.15.47:0/384355998 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c02c810 con 0x7f5b000039a0 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.864+0000 7f5b195e7640 1 -- 172.21.15.47:0/384355998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b000039a0 msgr2=0x7f5b00003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.864+0000 7f5b195e7640 1 --2- 172.21.15.47:0/384355998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b000039a0 0x7f5b00003d80 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00c240 tx=0x7f5b000057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.864+0000 7f5b137fe640 1 -- 172.21.15.47:0/384355998 reap_dead start 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.864+0000 7f5b195e7640 1 -- 172.21.15.47:0/384355998 shutdown_connections 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.864+0000 7f5b195e7640 1 -- 172.21.15.47:0/384355998 >> 172.21.15.47:0/384355998 conn(0x7f5b140b21d0 msgr2=0x7f5b140b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.864+0000 7f5b195e7640 1 -- 172.21.15.47:0/384355998 shutdown_connections 2024-03-23T09:52:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.864+0000 7f5b195e7640 1 -- 172.21.15.47:0/384355998 wait complete. 2024-03-23T09:52:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.865+0000 7f5b195e7640 1 Processor -- start 2024-03-23T09:52:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.865+0000 7f5b195e7640 1 -- start start 2024-03-23T09:52:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b195e7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b000039a0 0x7f5b140c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b195e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b1415c530 0x7f5b1415e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b195e7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b1415c9b0 0x7f5b1415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b195e7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140c9740 con 0x7f5b1415c530 2024-03-23T09:52:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b195e7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140c8ac0 con 0x7f5b1415c9b0 2024-03-23T09:52:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b195e7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140c8da0 con 0x7f5b000039a0 2024-03-23T09:52:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b127fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b1415c530 0x7f5b1415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b127fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b1415c530 0x7f5b1415e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59438/0 (socket says 172.21.15.47:59438) 2024-03-23T09:52:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b127fc640 1 -- 172.21.15.47:0/2041780704 learned_addr learned my addr 172.21.15.47:0/2041780704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b000039a0 0x7f5b140c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b137fe640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b1415c9b0 0x7f5b1415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b127fc640 1 -- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b000039a0 msgr2=0x7f5b140c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b127fc640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b000039a0 0x7f5b140c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b127fc640 1 -- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b1415c9b0 msgr2=0x7f5b1415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b127fc640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b1415c9b0 0x7f5b1415ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b127fc640 1 -- 172.21.15.47:0/2041780704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b140c9080 con 0x7f5b1415c530 2024-03-23T09:52:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b137fe640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b1415c9b0 0x7f5b1415ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b127fc640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b1415c530 0x7f5b1415e910 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00f780 tx=0x7f5afc00fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.866+0000 7f5b0b7fe640 1 -- 172.21.15.47:0/2041780704 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00fe90 con 0x7f5b1415c530 2024-03-23T09:52:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.867+0000 7f5b0b7fe640 1 -- 172.21.15.47:0/2041780704 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc0052e0 con 0x7f5b1415c530 2024-03-23T09:52:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.867+0000 7f5b195e7640 1 -- 172.21.15.47:0/2041780704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b140c9360 con 0x7f5b1415c530 2024-03-23T09:52:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.867+0000 7f5b0b7fe640 1 -- 172.21.15.47:0/2041780704 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00d730 con 0x7f5b1415c530 2024-03-23T09:52:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.867+0000 7f5b195e7640 1 -- 172.21.15.47:0/2041780704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b140c7f40 con 0x7f5b1415c530 2024-03-23T09:52:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.868+0000 7f5b0b7fe640 1 -- 172.21.15.47:0/2041780704 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5afc00d8b0 con 0x7f5b1415c530 2024-03-23T09:52:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.868+0000 7f5b0b7fe640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5af406ac00 0x7f5af406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.868+0000 7f5b0b7fe640 1 -- 172.21.15.47:0/2041780704 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5afc0893c0 con 0x7f5b1415c530 2024-03-23T09:52:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.869+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5af406ac00 0x7f5af406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.869+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5af406ac00 0x7f5af406d0c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c034000 tx=0x7f5b0c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.870+0000 7f5b1c237740 1 Processor -- start 2024-03-23T09:52:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.870+0000 7f5b1c237740 1 -- start start 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.870+0000 7f5b1c237740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 0x55d491cbad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.870+0000 7f5b1c237740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cb97d0 0x55d491cbb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.870+0000 7f5b1c237740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cb5c70 0x55d491ce5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.870+0000 7f5b1c237740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d491c269b0 con 0x55d491d03750 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.870+0000 7f5b1c237740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d491c24140 con 0x55d491cb5c70 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.870+0000 7f5b1c237740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d491c1fa30 con 0x55d491cb97d0 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 0x55d491cbad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 0x55d491cbad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59452/0 (socket says 172.21.15.47:59452) 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b12ffd640 1 -- 172.21.15.47:0/3961779921 learned_addr learned my addr 172.21.15.47:0/3961779921 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b127fc640 1 --2- 172.21.15.47:0/3961779921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cb97d0 0x55d491cbb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b137fe640 1 --2- 172.21.15.47:0/3961779921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cb5c70 0x55d491ce5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b12ffd640 1 -- 172.21.15.47:0/3961779921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cb97d0 msgr2=0x55d491cbb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/3961779921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cb97d0 0x55d491cbb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b12ffd640 1 -- 172.21.15.47:0/3961779921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cb5c70 msgr2=0x55d491ce5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/3961779921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cb5c70 0x55d491ce5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b12ffd640 1 -- 172.21.15.47:0/3961779921 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d491bac2f0 con 0x55d491d03750 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/3961779921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 0x55d491cbad80 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c02c3b0 tx=0x7f5b0c007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b932434c41cc6b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b08ff9640 1 -- 172.21.15.47:0/3961779921 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c049070 con 0x55d491d03750 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b08ff9640 1 -- 172.21.15.47:0/3961779921 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c02de30 con 0x55d491d03750 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.871+0000 7f5b08ff9640 1 -- 172.21.15.47:0/3961779921 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c03c040 con 0x55d491d03750 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.872+0000 7f5b1c237740 1 -- 172.21.15.47:0/3961779921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 msgr2=0x55d491cbad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.872+0000 7f5b1c237740 1 --2- 172.21.15.47:0/3961779921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 0x55d491cbad80 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c02c3b0 tx=0x7f5b0c007000 comp rx=0 tx=0).stop 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.872+0000 7f5b1c237740 1 -- 172.21.15.47:0/3961779921 shutdown_connections 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.872+0000 7f5b1c237740 1 --2- 172.21.15.47:0/3961779921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cb5c70 0x55d491ce5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.872+0000 7f5b1c237740 1 --2- 172.21.15.47:0/3961779921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cb97d0 0x55d491cbb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.872+0000 7f5b1c237740 1 --2- 172.21.15.47:0/3961779921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 0x55d491cbad80 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.872+0000 7f5b1c237740 1 -- 172.21.15.47:0/3961779921 >> 172.21.15.47:0/3961779921 conn(0x55d491cb7ee0 msgr2=0x55d491d00a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.872+0000 7f5b1c237740 1 -- 172.21.15.47:0/3961779921 shutdown_connections 2024-03-23T09:52:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.872+0000 7f5b1c237740 1 -- 172.21.15.47:0/3961779921 wait complete. 2024-03-23T09:52:10.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.872+0000 7f5b1c237740 1 -- start start 2024-03-23T09:52:10.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.873+0000 7f5b1c237740 1 Processor -- start 2024-03-23T09:52:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.873+0000 7f5b1c237740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cb97d0 0x55d491d085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.873+0000 7f5b1c237740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 0x55d491d08b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.873+0000 7f5b1c237740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cdc640 0x55d491d06270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.873+0000 7f5b1c237740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d491bde090 con 0x55d491d03750 2024-03-23T09:52:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.873+0000 7f5b1c237740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d491bddd30 con 0x55d491cdc640 2024-03-23T09:52:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.873+0000 7f5b1c237740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d491ba32a0 con 0x55d491cb97d0 2024-03-23T09:52:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.873+0000 7f5b1c237740 10 client.0 _ll_register_callbacks cb 0x55d491bd9050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:52:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.873+0000 7f5b127fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 0x55d491d08b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.873+0000 7f5b127fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 0x55d491d08b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59468/0 (socket says 172.21.15.47:59468) 2024-03-23T09:52:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.873+0000 7f5b127fc640 1 -- 172.21.15.47:0/237522562 learned_addr learned my addr 172.21.15.47:0/237522562 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b137fe640 1 --2- 172.21.15.47:0/237522562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cdc640 0x55d491d06270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b127fc640 1 -- 172.21.15.47:0/237522562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cb97d0 msgr2=0x55d491d085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b127fc640 1 --2- 172.21.15.47:0/237522562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cb97d0 0x55d491d085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b127fc640 1 -- 172.21.15.47:0/237522562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cdc640 msgr2=0x55d491d06270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b127fc640 1 --2- 172.21.15.47:0/237522562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cdc640 0x55d491d06270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b127fc640 1 -- 172.21.15.47:0/237522562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b00003520 con 0x55d491d03750 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b137fe640 1 --2- 172.21.15.47:0/237522562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cdc640 0x55d491d06270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b127fc640 1 --2- 172.21.15.47:0/237522562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 0x55d491d08b10 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0050a0 tx=0x7f5afc05c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f864e02b6c032fc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5af1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5af1ffb640 1 -- 172.21.15.47:0/237522562 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc05e0c0 con 0x55d491d03750 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5af1ffb640 1 -- 172.21.15.47:0/237522562 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5afc05e220 con 0x55d491d03750 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b1c237740 1 -- 172.21.15.47:0/237522562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4919133c0 con 0x55d491d03750 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5af1ffb640 1 -- 172.21.15.47:0/237522562 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5afc060130 con 0x55d491d03750 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b1c237740 10 client.9609 fetch_fsmap learned FSMap version 86 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b1c237740 1 -- 172.21.15.47:0/237522562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55d491c1e0b0 con 0x55d491d03750 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5af1ffb640 1 -- 172.21.15.47:0/237522562 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d4919133c0 con 0x55d491d03750 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5af1ffb640 1 -- 172.21.15.47:0/237522562 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 86) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5afc012070 con 0x55d491d03750 2024-03-23T09:52:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b1c237740 10 client.9609 fetch_fsmap finished waiting for FSMap version 86 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b1c237740 10 client.9609 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b1c237740 20 client.9609 populate_metadata read hostname 'smithi047' 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b1c237740 1 --2- 172.21.15.47:0/237522562 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55d491d1bb20 0x55d491d1df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b1c237740 4 client.9609 mds_command: new command op to 9570 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.874+0000 7f5b1c237740 1 -- 172.21.15.47:0/237522562 --> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4919246c0 con 0x55d491d1bb20 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.875+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/237522562 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55d491d1bb20 0x55d491d1df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.875+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/237522562 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55d491d1bb20 0x55d491d1df10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf511f10505b3db4 server_cookie=c25d5ab0f467e060 in_seq=0 out_seq=0 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.875+0000 7f5af1ffb640 10 client.9609 ms_handle_connect on v2:172.21.15.47:6832/1868958629 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.875+0000 7f5af1ffb640 1 -- 172.21.15.47:0/237522562 <== mds.0 v2:172.21.15.47:6832/1868958629 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d4919246c0 con 0x55d491d1bb20 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.875+0000 7f5af1ffb640 10 client.9609 handle_command_reply: tid=0 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.875+0000 7f5b1c237740 1 client.9609 shutdown 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.875+0000 7f5b1c237740 1 -- 172.21.15.47:0/237522562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 msgr2=0x55d491d08b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.875+0000 7f5b1c237740 1 --2- 172.21.15.47:0/237522562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 0x55d491d08b10 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0050a0 tx=0x7f5afc05c0c0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.875+0000 7f5b1c237740 1 -- 172.21.15.47:0/237522562 shutdown_connections 2024-03-23T09:52:10.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.875+0000 7f5b1c237740 1 --2- 172.21.15.47:0/237522562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cdc640 0x55d491d06270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.875+0000 7f5b1c237740 1 --2- 172.21.15.47:0/237522562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491d03750 0x55d491d08b10 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.875+0000 7f5b1c237740 1 --2- 172.21.15.47:0/237522562 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55d491d1bb20 0x55d491d1df10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.875+0000 7f5b1c237740 1 --2- 172.21.15.47:0/237522562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cb97d0 0x55d491d085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.876+0000 7f5af1ffb640 0 client.9609 ms_handle_reset on v2:172.21.15.47:6832/1868958629 2024-03-23T09:52:10.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.876+0000 7f5b1c237740 1 -- 172.21.15.47:0/237522562 >> 172.21.15.47:0/237522562 conn(0x55d491cb5c70 msgr2=0x55d491ce3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:10.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.876+0000 7f5b1c237740 1 -- 172.21.15.47:0/237522562 shutdown_connections 2024-03-23T09:52:10.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.876+0000 7f5b1c237740 1 -- 172.21.15.47:0/237522562 wait complete. 2024-03-23T09:52:10.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.876+0000 7f5b1c237740 20 client.9609 trim_cache size 0 max 16384 2024-03-23T09:52:10.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.891+0000 7f5b1c237740 1 Processor -- start 2024-03-23T09:52:10.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.891+0000 7f5b1c237740 1 -- start start 2024-03-23T09:52:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b1c237740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cb97d0 0x55d491cbb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b1c237740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cdc640 0x55d491cbbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b1c237740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491d03750 0x55d491d19920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b1c237740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d491d060a0 con 0x55d491cb97d0 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b1c237740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d491cff840 con 0x55d491d03750 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b1c237740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d491cb7640 con 0x55d491cdc640 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cb97d0 0x55d491cbb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cb97d0 0x55d491cbb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59480/0 (socket says 172.21.15.47:59480) 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b12ffd640 1 -- 172.21.15.47:0/2599686046 learned_addr learned my addr 172.21.15.47:0/2599686046 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b137fe640 1 --2- 172.21.15.47:0/2599686046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491d03750 0x55d491d19920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b127fc640 1 --2- 172.21.15.47:0/2599686046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cdc640 0x55d491cbbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b12ffd640 1 -- 172.21.15.47:0/2599686046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cdc640 msgr2=0x55d491cbbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/2599686046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cdc640 0x55d491cbbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b12ffd640 1 -- 172.21.15.47:0/2599686046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491d03750 msgr2=0x55d491d19920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/2599686046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491d03750 0x55d491d19920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b12ffd640 1 -- 172.21.15.47:0/2599686046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d491bc0430 con 0x55d491cb97d0 2024-03-23T09:52:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b137fe640 1 --2- 172.21.15.47:0/2599686046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491d03750 0x55d491d19920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b127fc640 1 --2- 172.21.15.47:0/2599686046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cdc640 0x55d491cbbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.892+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/2599686046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cb97d0 0x55d491cbb890 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c02c3b0 tx=0x7f5b0c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee36ba97dd517e6c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.893+0000 7f5b097fa640 1 -- 172.21.15.47:0/2599686046 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c049040 con 0x55d491cb97d0 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.893+0000 7f5b097fa640 1 -- 172.21.15.47:0/2599686046 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c04a040 con 0x55d491cb97d0 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.893+0000 7f5b097fa640 1 -- 172.21.15.47:0/2599686046 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c03c040 con 0x55d491cb97d0 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.893+0000 7f5b1c237740 1 -- 172.21.15.47:0/2599686046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cb97d0 msgr2=0x55d491cbb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.893+0000 7f5b1c237740 1 --2- 172.21.15.47:0/2599686046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cb97d0 0x55d491cbb890 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c02c3b0 tx=0x7f5b0c040000 comp rx=0 tx=0).stop 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.893+0000 7f5b1c237740 1 -- 172.21.15.47:0/2599686046 shutdown_connections 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.893+0000 7f5b1c237740 1 --2- 172.21.15.47:0/2599686046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491d03750 0x55d491d19920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.893+0000 7f5b1c237740 1 --2- 172.21.15.47:0/2599686046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491cdc640 0x55d491cbbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.893+0000 7f5b1c237740 1 --2- 172.21.15.47:0/2599686046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cb97d0 0x55d491cbb890 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.893+0000 7f5b1c237740 1 -- 172.21.15.47:0/2599686046 >> 172.21.15.47:0/2599686046 conn(0x55d491cb5c70 msgr2=0x55d491cb9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.893+0000 7f5b1c237740 1 -- 172.21.15.47:0/2599686046 shutdown_connections 2024-03-23T09:52:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.893+0000 7f5b1c237740 1 -- 172.21.15.47:0/2599686046 wait complete. 2024-03-23T09:52:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.894+0000 7f5b1c237740 1 -- start start 2024-03-23T09:52:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.894+0000 7f5b1c237740 1 Processor -- start 2024-03-23T09:52:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.894+0000 7f5b1c237740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cb97d0 0x55d491d66bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b1c237740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cdc640 0x55d491cbb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b1c237740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491d03750 0x55d491cbbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b1c237740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d491d03160 con 0x55d491cdc640 2024-03-23T09:52:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b1c237740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d491d07f40 con 0x55d491cb97d0 2024-03-23T09:52:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b1c237740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d491d67100 con 0x55d491d03750 2024-03-23T09:52:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b1c237740 10 client.0 _ll_register_callbacks cb 0x7f5b000044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:52:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b127fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cdc640 0x55d491cbb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cb97d0 0x55d491d66bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b127fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cdc640 0x55d491cbb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59494/0 (socket says 172.21.15.47:59494) 2024-03-23T09:52:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b127fc640 1 -- 172.21.15.47:0/3898300140 learned_addr learned my addr 172.21.15.47:0/3898300140 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b137fe640 1 --2- 172.21.15.47:0/3898300140 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491d03750 0x55d491cbbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b127fc640 1 -- 172.21.15.47:0/3898300140 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491d03750 msgr2=0x55d491cbbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b127fc640 1 --2- 172.21.15.47:0/3898300140 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491d03750 0x55d491cbbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b127fc640 1 -- 172.21.15.47:0/3898300140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cb97d0 msgr2=0x55d491d66bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b127fc640 1 --2- 172.21.15.47:0/3898300140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cb97d0 0x55d491d66bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b127fc640 1 -- 172.21.15.47:0/3898300140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d491bac2f0 con 0x55d491cdc640 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b127fc640 1 --2- 172.21.15.47:0/3898300140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cdc640 0x55d491cbb890 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0896a0 tx=0x7f5afc05e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ccaddc1822287a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5af1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5af1ffb640 1 -- 172.21.15.47:0/3898300140 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0581d0 con 0x55d491cdc640 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5af1ffb640 1 -- 172.21.15.47:0/3898300140 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5afc0890d0 con 0x55d491cdc640 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b1c237740 1 -- 172.21.15.47:0/3898300140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b0c02de30 con 0x55d491cdc640 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5af1ffb640 1 -- 172.21.15.47:0/3898300140 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5afc0605c0 con 0x55d491cdc640 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b1c237740 10 client.9615 fetch_fsmap learned FSMap version 86 2024-03-23T09:52:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.895+0000 7f5b1c237740 1 -- 172.21.15.47:0/3898300140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55d491be5b40 con 0x55d491cdc640 2024-03-23T09:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5af1ffb640 1 -- 172.21.15.47:0/3898300140 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c02de30 con 0x55d491cdc640 2024-03-23T09:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5af1ffb640 1 -- 172.21.15.47:0/3898300140 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 86) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5afc07d030 con 0x55d491cdc640 2024-03-23T09:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5b1c237740 10 client.9615 fetch_fsmap finished waiting for FSMap version 86 2024-03-23T09:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5b1c237740 10 client.9615 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5b1c237740 20 client.9615 populate_metadata read hostname 'smithi047' 2024-03-23T09:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5b1c237740 1 --2- 172.21.15.47:0/3898300140 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55d491ce9a50 0x55d491d02040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5b1c237740 4 client.9615 mds_command: new command op to 9570 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5b1c237740 1 -- 172.21.15.47:0/3898300140 --> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d491977d10 con 0x55d491ce9a50 2024-03-23T09:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/3898300140 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55d491ce9a50 0x55d491d02040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5af1ffb640 10 client.9615 ms_handle_connect on v2:172.21.15.47:6832/1868958629 2024-03-23T09:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5b12ffd640 1 --2- 172.21.15.47:0/3898300140 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55d491ce9a50 0x55d491d02040 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8385a494e7dbe2e0 server_cookie=b4af41cbe9572f7f in_seq=0 out_seq=0 2024-03-23T09:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5af1ffb640 1 -- 172.21.15.47:0/3898300140 <== mds.0 v2:172.21.15.47:6832/1868958629 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55d491977d10 con 0x55d491ce9a50 2024-03-23T09:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5af1ffb640 10 client.9615 handle_command_reply: tid=0 2024-03-23T09:52:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5b1c237740 1 client.9615 shutdown 2024-03-23T09:52:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5b1c237740 1 -- 172.21.15.47:0/3898300140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cdc640 msgr2=0x55d491cbb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.896+0000 7f5b1c237740 1 --2- 172.21.15.47:0/3898300140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cdc640 0x55d491cbb890 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0896a0 tx=0x7f5afc05e050 comp rx=0 tx=0).stop 2024-03-23T09:52:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b1c237740 1 -- 172.21.15.47:0/3898300140 shutdown_connections 2024-03-23T09:52:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b1c237740 1 --2- 172.21.15.47:0/3898300140 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55d491ce9a50 0x55d491d02040 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b1c237740 1 --2- 172.21.15.47:0/3898300140 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d491d03750 0x55d491cbbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5af1ffb640 0 client.9615 ms_handle_reset on v2:172.21.15.47:6832/1868958629 2024-03-23T09:52:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b1c237740 1 --2- 172.21.15.47:0/3898300140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d491cdc640 0x55d491cbb890 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b1c237740 1 --2- 172.21.15.47:0/3898300140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d491cb97d0 0x55d491d66bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b1c237740 1 -- 172.21.15.47:0/3898300140 >> 172.21.15.47:0/3898300140 conn(0x55d491cb5c70 msgr2=0x55d491d05a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b1c237740 1 -- 172.21.15.47:0/3898300140 shutdown_connections 2024-03-23T09:52:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b1c237740 1 -- 172.21.15.47:0/3898300140 wait complete. 2024-03-23T09:52:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b1c237740 20 client.9615 trim_cache size 0 max 16384 2024-03-23T09:52:10.905 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:52:10.905 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:52:10.905 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:52:10.905 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 9570, 2024-03-23T09:52:10.905 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:52:10.905 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:52:10.905 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:52:10.905 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 4.4184773420000001, 2024-03-23T09:52:10.905 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 86, 2024-03-23T09:52:10.905 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 117, 2024-03-23T09:52:10.905 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 117, 2024-03-23T09:52:10.905 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 5.3867129509999998 2024-03-23T09:52:10.906 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:52:10.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b195e7640 1 -- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5af406ac00 msgr2=0x7f5af406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b195e7640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5af406ac00 0x7f5af406d0c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c034000 tx=0x7f5b0c009030 comp rx=0 tx=0).stop 2024-03-23T09:52:10.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b195e7640 1 -- 172.21.15.47:0/2041780704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b1415c530 msgr2=0x7f5b1415e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:10.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b195e7640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b1415c530 0x7f5b1415e910 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00f780 tx=0x7f5afc00fc40 comp rx=0 tx=0).stop 2024-03-23T09:52:10.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b195e7640 1 -- 172.21.15.47:0/2041780704 shutdown_connections 2024-03-23T09:52:10.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b195e7640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f5af406ac00 0x7f5af406d0c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b195e7640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b1415c9b0 0x7f5b1415ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b195e7640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b1415c530 0x7f5b1415e910 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b195e7640 1 --2- 172.21.15.47:0/2041780704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b000039a0 0x7f5b140c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:10.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b195e7640 1 -- 172.21.15.47:0/2041780704 >> 172.21.15.47:0/2041780704 conn(0x7f5b140b21d0 msgr2=0x7f5b140afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:10.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b195e7640 1 -- 172.21.15.47:0/2041780704 shutdown_connections 2024-03-23T09:52:10.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:10.897+0000 7f5b195e7640 1 -- 172.21.15.47:0/2041780704 wait complete. 2024-03-23T09:52:10.908 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:52:11.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.008+0000 7fbac8a38640 1 Processor -- start 2024-03-23T09:52:11.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.008+0000 7fbac8a38640 1 -- start start 2024-03-23T09:52:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.009+0000 7fbac8a38640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbac4099a40 0x7fbac40c4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.009+0000 7fbac8a38640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac40cb1d0 con 0x7fbac40ceca0 2024-03-23T09:52:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.009+0000 7fbac8a38640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac415e230 con 0x7fbac4099410 2024-03-23T09:52:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.009+0000 7fbac8a38640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac40ce2b0 con 0x7fbac4099a40 2024-03-23T09:52:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.009+0000 7fbac2d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbac4099a40 0x7fbac40c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.009+0000 7fbac2d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbac4099a40 0x7fbac40c4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59498/0 (socket says 172.21.15.47:59498) 2024-03-23T09:52:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.009+0000 7fbac2d9d640 1 -- 172.21.15.47:0/1694297884 learned_addr learned my addr 172.21.15.47:0/1694297884 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.010+0000 7fbac2d9d640 1 -- 172.21.15.47:0/1694297884 >> v1:172.21.15.175:6789/0 conn(0x7fbac4099410 legacy=0x7fbac40cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.010+0000 7fbac2d9d640 1 -- 172.21.15.47:0/1694297884 >> v1:172.21.15.47:6789/0 conn(0x7fbac40ceca0 legacy=0x7fbac40cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:11.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.010+0000 7fbac2d9d640 1 -- 172.21.15.47:0/1694297884 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac40ce590 con 0x7fbac4099a40 2024-03-23T09:52:11.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.010+0000 7fbac2d9d640 1 --2- 172.21.15.47:0/1694297884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac4099a40 0x7fbac40c4230 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009e50 tx=0x7fbabc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce405168884365c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:11.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.010+0000 7fbac1d9b640 1 -- 172.21.15.47:0/1694297884 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc02ca80 con 0x7fbac4099a40 2024-03-23T09:52:11.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.011+0000 7fbac1d9b640 1 -- 172.21.15.47:0/1694297884 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbabc02cbe0 con 0x7fbac4099a40 2024-03-23T09:52:11.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.011+0000 7fbac1d9b640 1 -- 172.21.15.47:0/1694297884 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc0343f0 con 0x7fbac4099a40 2024-03-23T09:52:11.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.011+0000 7fbac8a38640 1 -- 172.21.15.47:0/1694297884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac4099a40 msgr2=0x7fbac40c4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:11.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.011+0000 7fbac8a38640 1 --2- 172.21.15.47:0/1694297884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac4099a40 0x7fbac40c4230 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009e50 tx=0x7fbabc02bee0 comp rx=0 tx=0).stop 2024-03-23T09:52:11.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.011+0000 7fbac8a38640 1 -- 172.21.15.47:0/1694297884 shutdown_connections 2024-03-23T09:52:11.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.011+0000 7fbac8a38640 1 --2- 172.21.15.47:0/1694297884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac4099a40 0x7fbac40c4230 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:11.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.011+0000 7fbac8a38640 1 -- 172.21.15.47:0/1694297884 >> 172.21.15.47:0/1694297884 conn(0x7fbac40b21d0 msgr2=0x7fbac40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:11.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.011+0000 7fbac8a38640 1 -- 172.21.15.47:0/1694297884 shutdown_connections 2024-03-23T09:52:11.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.011+0000 7fbac8a38640 1 -- 172.21.15.47:0/1694297884 wait complete. 2024-03-23T09:52:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.013+0000 7fbac8a38640 1 Processor -- start 2024-03-23T09:52:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.013+0000 7fbac8a38640 1 -- start start 2024-03-23T09:52:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.013+0000 7fbac8a38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac4099410 0x7fbac41e5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.013+0000 7fbac8a38640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac4099a40 0x7fbac41e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.013+0000 7fbac8a38640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac40ceca0 0x7fbac4157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.013+0000 7fbac8a38640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac41e4760 con 0x7fbac4099a40 2024-03-23T09:52:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.013+0000 7fbac8a38640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac40c9eb0 con 0x7fbac40ceca0 2024-03-23T09:52:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.013+0000 7fbac8a38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac41e6160 con 0x7fbac4099410 2024-03-23T09:52:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.013+0000 7fbab9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac4099a40 0x7fbac41e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.013+0000 7fbab9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac4099a40 0x7fbac41e4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59514/0 (socket says 172.21.15.47:59514) 2024-03-23T09:52:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.013+0000 7fbab9d9b640 1 -- 172.21.15.47:0/2052368423 learned_addr learned my addr 172.21.15.47:0/2052368423 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.014+0000 7fbac2d9d640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac40ceca0 0x7fbac4157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.014+0000 7fbac259c640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac4099410 0x7fbac41e5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.014+0000 7fbab9d9b640 1 -- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac4099410 msgr2=0x7fbac41e5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.014+0000 7fbab9d9b640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac4099410 0x7fbac41e5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.014+0000 7fbab9d9b640 1 -- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac40ceca0 msgr2=0x7fbac4157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.014+0000 7fbab9d9b640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac40ceca0 0x7fbac4157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.014+0000 7fbab9d9b640 1 -- 172.21.15.47:0/2052368423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc009b30 con 0x7fbac4099a40 2024-03-23T09:52:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.014+0000 7fbab9d9b640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac4099a40 0x7fbac41e4b80 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00b3e0 tx=0x7fbaac00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.014+0000 7fbabb7fe640 1 -- 172.21.15.47:0/2052368423 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac011830 con 0x7fbac4099a40 2024-03-23T09:52:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.014+0000 7fbabb7fe640 1 -- 172.21.15.47:0/2052368423 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac0119b0 con 0x7fbac4099a40 2024-03-23T09:52:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.014+0000 7fbac8a38640 1 -- 172.21.15.47:0/2052368423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac41e6930 con 0x7fbac4099a40 2024-03-23T09:52:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.014+0000 7fbabb7fe640 1 -- 172.21.15.47:0/2052368423 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac011d60 con 0x7fbac4099a40 2024-03-23T09:52:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.014+0000 7fbac8a38640 1 -- 172.21.15.47:0/2052368423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac41e4380 con 0x7fbac4099a40 2024-03-23T09:52:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.015+0000 7fbac8a38640 1 -- 172.21.15.47:0/2052368423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac41e48a0 con 0x7fbac4099a40 2024-03-23T09:52:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.015+0000 7fbabb7fe640 1 -- 172.21.15.47:0/2052368423 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbaac01a400 con 0x7fbac4099a40 2024-03-23T09:52:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.016+0000 7fbabb7fe640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbaa406ac30 0x7fbaa406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.016+0000 7fbabb7fe640 1 -- 172.21.15.47:0/2052368423 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fbaac089450 con 0x7fbac4099a40 2024-03-23T09:52:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.016+0000 7fbac259c640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbaa406ac30 0x7fbaa406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.017+0000 7fbac259c640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbaa406ac30 0x7fbaa406d0f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbab4000f60 tx=0x7fbab4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.020+0000 7fbabb7fe640 1 -- 172.21.15.47:0/2052368423 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbac41e48a0 con 0x7fbac4099a40 2024-03-23T09:52:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.236+0000 7fbac8a38640 1 -- 172.21.15.47:0/2052368423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbac41e48a0 con 0x7fbac4099a40 2024-03-23T09:52:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.237+0000 7fbabb7fe640 1 -- 172.21.15.47:0/2052368423 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbac41e48a0 con 0x7fbac4099a40 2024-03-23T09:52:11.237 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:52:11.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.239+0000 7fbac8a38640 1 -- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbaa406ac30 msgr2=0x7fbaa406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:11.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.239+0000 7fbac8a38640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbaa406ac30 0x7fbaa406d0f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbab4000f60 tx=0x7fbab4009040 comp rx=0 tx=0).stop 2024-03-23T09:52:11.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.239+0000 7fbac8a38640 1 -- 172.21.15.47:0/2052368423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac4099a40 msgr2=0x7fbac41e4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:11.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.239+0000 7fbac8a38640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac4099a40 0x7fbac41e4b80 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00b3e0 tx=0x7fbaac00b8a0 comp rx=0 tx=0).stop 2024-03-23T09:52:11.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.239+0000 7fbac8a38640 1 -- 172.21.15.47:0/2052368423 shutdown_connections 2024-03-23T09:52:11.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.239+0000 7fbac8a38640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbaa406ac30 0x7fbaa406d0f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:11.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.239+0000 7fbac8a38640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac40ceca0 0x7fbac4157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:11.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.239+0000 7fbac8a38640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac4099a40 0x7fbac41e4b80 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:11.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.239+0000 7fbac8a38640 1 --2- 172.21.15.47:0/2052368423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac4099410 0x7fbac41e5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:11.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.239+0000 7fbac8a38640 1 -- 172.21.15.47:0/2052368423 >> 172.21.15.47:0/2052368423 conn(0x7fbac40b21d0 msgr2=0x7fbac406f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:11.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.239+0000 7fbac8a38640 1 -- 172.21.15.47:0/2052368423 shutdown_connections 2024-03-23T09:52:11.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:11.239+0000 7fbac8a38640 1 -- 172.21.15.47:0/2052368423 wait complete. 2024-03-23T09:52:11.249 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:52:11.249 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:52:11.249 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:52:11.250 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:52:11.250 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:11.250 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:52:11.250 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:52:11.250 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:52:11.250 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:52:11.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:11.250 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42007sec preferred_lft 42007sec 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:11.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:11.267 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:52:11.330 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:52:11.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:11.331 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:52:11.387 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:52:11.387 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:52:11.387 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:52:11.388 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:52:11.388 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:52:11.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:11.388 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:52:11.388 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:52:11.388 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:52:11.388 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:52:11.388 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:52:11.388 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:52:11.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:11.482 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:52:11.497 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:52:11.497 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:52:11.497 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:52:11.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:11.497 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:52:11.563 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:52:11.563 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:52:11.563 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:11.618 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:52:11.619 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:52:11.619 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:11.673 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:52:11.740 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:52:11.740 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:52:11.805 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:52:11.806 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:52:11.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:11.807 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:52:11.861 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:52:11.861 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:52:11.903 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:52:11.929 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:52:11.929 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:52:11.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:52:11.989+0000 7f9b30873480 -1 init, newargv = 0x55874e8a2ec0 newargc=15 2024-03-23T09:52:11.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[81556]: starting ceph client 2024-03-23T09:52:11.994 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[81556]: starting fuse 2024-03-23T09:52:11.996 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:52:11.997 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:52:11.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:11.998 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:52:12.052 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:52:12.052 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:52:12.054 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:12.054 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:52:12.054 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:52:12.054 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:52:12.054 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:52:12.054 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:52:12.054 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:52:12.054 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:52:12.054 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:52:12.054 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:12.054 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:52:12.054 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:12.169 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.81556.asok 2024-03-23T09:52:12.356 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.81556.asok 2024-03-23T09:52:12.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:12.356 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.81556.asok status 2024-03-23T09:52:12.479 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:52:12.478+0000 7fefb746b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:52:12.494 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "81556", 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 9630, 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:52:12.495 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 9630 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1350227814 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1350227814 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.9630 192.168.0.1:0/1350227814", 2024-03-23T09:52:12.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/1350227814", 2024-03-23T09:52:12.497 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:52:12.497 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 86, 2024-03-23T09:52:12.497 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 117, 2024-03-23T09:52:12.497 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:52:12.497 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:52:12.497 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:52:12.497 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:52:12.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:12.503 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:52:12.503 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:12.518 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:52:12.518 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:12.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:12.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:12.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:12.585 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-23T09:52:12.648 INFO:teuthology.orchestra.run.smithi047.stdout:./ 2024-03-23T09:52:12.648 INFO:teuthology.orchestra.run.smithi047.stdout:./parent 2024-03-23T09:52:12.648 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/flushed 2024-03-23T09:52:12.648 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/flushed/charlie 2024-03-23T09:52:12.648 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/flushed/alpha 2024-03-23T09:52:12.649 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/unflushed 2024-03-23T09:52:12.649 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/unflushed/jfile 2024-03-23T09:52:12.649 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:12.650 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:12.650 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:12.650 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:12.650 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-23T09:52:12.650 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:12.751 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:52:12.940 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:12.940 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:12.940 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:12.940 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:12.940 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-23T09:52:12.940 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:13.042 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627776 2024-03-23T09:52:13.231 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:13.232 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:13.232 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:13.232 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:13.232 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-23T09:52:13.232 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:13.335 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627777 2024-03-23T09:52:13.524 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:13.524 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:13.524 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:13.524 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:13.524 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-23T09:52:13.524 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:13.627 INFO:teuthology.orchestra.run.smithi047.stdout:1099511628280 2024-03-23T09:52:13.815 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:13.816 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:13.816 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:13.816 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:13.816 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-23T09:52:13.816 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:13.919 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627778 2024-03-23T09:52:14.107 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:14.108 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:14.108 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:14.108 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:14.108 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-23T09:52:14.108 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:14.210 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627779 2024-03-23T09:52:14.399 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:14.399 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:14.400 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:14.400 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:14.400 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-23T09:52:14.400 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:14.503 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627780 2024-03-23T09:52:14.692 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:52:14.692 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:52:14.747 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:52:14.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.889+0000 7f4bdeb16640 1 Processor -- start 2024-03-23T09:52:14.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.890+0000 7f4bdeb16640 1 -- start start 2024-03-23T09:52:14.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.890+0000 7f4bdeb16640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4bd80cde10 0x7f4bd8167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:14.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.890+0000 7f4bdeb16640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd80d11e0 con 0x7f4bd815ca90 2024-03-23T09:52:14.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.890+0000 7f4bdeb16640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd80d2680 con 0x7f4bd80b8350 2024-03-23T09:52:14.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.890+0000 7f4bdeb16640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd8160f60 con 0x7f4bd80cde10 2024-03-23T09:52:14.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.890+0000 7f4bdd0b3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4bd80cde10 0x7f4bd8167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:14.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.890+0000 7f4bdc8b2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4bd815ca90 0x7f4bd8160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39888/0 (socket says 172.21.15.47:39888) 2024-03-23T09:52:14.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.890+0000 7f4bdd0b3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4bd80cde10 0x7f4bd8167aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35276/0 (socket says 172.21.15.47:35276) 2024-03-23T09:52:14.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.890+0000 7f4bdc8b2640 1 -- 172.21.15.47:0/3129426261 learned_addr learned my addr 172.21.15.47:0/3129426261 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:14.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.891+0000 7f4bdd0b3640 1 -- 172.21.15.47:0/3129426261 >> v1:172.21.15.175:6789/0 conn(0x7f4bd80b8350 legacy=0x7f4bd80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:14.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.891+0000 7f4bdd0b3640 1 -- 172.21.15.47:0/3129426261 >> v1:172.21.15.47:6789/0 conn(0x7f4bd815ca90 legacy=0x7f4bd8160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:14.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.891+0000 7f4bdd0b3640 1 -- 172.21.15.47:0/3129426261 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd8161240 con 0x7f4bd80cde10 2024-03-23T09:52:14.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.891+0000 7f4bd77fe640 1 -- 172.21.15.47:0/3129426261 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252754758 0 0) 0x7f4bd80d11e0 con 0x7f4bd815ca90 2024-03-23T09:52:14.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.891+0000 7f4bdd0b3640 1 --2- 172.21.15.47:0/3129426261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd80cde10 0x7f4bd8167aa0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009d90 tx=0x7f4bd002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b52e8f71353aba server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:14.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.892+0000 7f4bd77fe640 1 -- 172.21.15.47:0/3129426261 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd002cbd0 con 0x7f4bd80cde10 2024-03-23T09:52:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.892+0000 7f4bd77fe640 1 -- 172.21.15.47:0/3129426261 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd002cd30 con 0x7f4bd80cde10 2024-03-23T09:52:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.892+0000 7f4bd77fe640 1 -- 172.21.15.47:0/3129426261 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd002cea0 con 0x7f4bd80cde10 2024-03-23T09:52:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.892+0000 7f4bdeb16640 1 -- 172.21.15.47:0/3129426261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd80cde10 msgr2=0x7f4bd8167aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.892+0000 7f4bdeb16640 1 --2- 172.21.15.47:0/3129426261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd80cde10 0x7f4bd8167aa0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009d90 tx=0x7f4bd002be20 comp rx=0 tx=0).stop 2024-03-23T09:52:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.892+0000 7f4bdeb16640 1 -- 172.21.15.47:0/3129426261 shutdown_connections 2024-03-23T09:52:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.892+0000 7f4bdeb16640 1 --2- 172.21.15.47:0/3129426261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd80cde10 0x7f4bd8167aa0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.892+0000 7f4bdeb16640 1 -- 172.21.15.47:0/3129426261 >> 172.21.15.47:0/3129426261 conn(0x7f4bd80659d0 msgr2=0x7f4bd8065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.892+0000 7f4bdeb16640 1 -- 172.21.15.47:0/3129426261 shutdown_connections 2024-03-23T09:52:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.892+0000 7f4bdeb16640 1 -- 172.21.15.47:0/3129426261 wait complete. 2024-03-23T09:52:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.893+0000 7f4bdeb16640 1 Processor -- start 2024-03-23T09:52:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.893+0000 7f4bdeb16640 1 -- start start 2024-03-23T09:52:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.894+0000 7f4bdeb16640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd80b8350 0x7f4bd8153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.894+0000 7f4bdeb16640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bd80cde10 0x7f4bd815d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.894+0000 7f4bdeb16640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd815ca90 0x7f4bd815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.894+0000 7f4bdeb16640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd8158590 con 0x7f4bd815ca90 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.894+0000 7f4bdeb16640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80b26f0 con 0x7f4bd80b8350 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.894+0000 7f4bdeb16640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80b4cf0 con 0x7f4bd80cde10 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.894+0000 7f4bdd0b3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd815ca90 0x7f4bd815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.894+0000 7f4bdd0b3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd815ca90 0x7f4bd815d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35286/0 (socket says 172.21.15.47:35286) 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.894+0000 7f4bdd0b3640 1 -- 172.21.15.47:0/4212691151 learned_addr learned my addr 172.21.15.47:0/4212691151 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bdc8b2640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd80b8350 0x7f4bd8153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bd7fff640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bd80cde10 0x7f4bd815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bdd0b3640 1 -- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bd80cde10 msgr2=0x7f4bd815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bdd0b3640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bd80cde10 0x7f4bd815d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bdd0b3640 1 -- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd80b8350 msgr2=0x7f4bd8153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bdd0b3640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd80b8350 0x7f4bd8153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bdd0b3640 1 -- 172.21.15.47:0/4212691151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0009a70 con 0x7f4bd815ca90 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bd7fff640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bd80cde10 0x7f4bd815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bdd0b3640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd815ca90 0x7f4bd815d6b0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009eb0 tx=0x7f4bd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:14.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bd57fa640 1 -- 172.21.15.47:0/4212691151 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd002cea0 con 0x7f4bd815ca90 2024-03-23T09:52:14.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bd57fa640 1 -- 172.21.15.47:0/4212691151 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd003f420 con 0x7f4bd815ca90 2024-03-23T09:52:14.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bd57fa640 1 -- 172.21.15.47:0/4212691151 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd003f7b0 con 0x7f4bd815ca90 2024-03-23T09:52:14.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bdeb16640 1 -- 172.21.15.47:0/4212691151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd80b4fd0 con 0x7f4bd815ca90 2024-03-23T09:52:14.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.895+0000 7f4bdeb16640 1 -- 172.21.15.47:0/4212691151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd8158ac0 con 0x7f4bd815ca90 2024-03-23T09:52:14.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.896+0000 7f4bdeb16640 1 -- 172.21.15.47:0/4212691151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd8072360 con 0x7f4bd815ca90 2024-03-23T09:52:14.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.897+0000 7f4bd57fa640 1 -- 172.21.15.47:0/4212691151 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4bd003f910 con 0x7f4bd815ca90 2024-03-23T09:52:14.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.897+0000 7f4bd57fa640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4bbc06ac30 0x7f4bbc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:14.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.897+0000 7f4bd57fa640 1 -- 172.21.15.47:0/4212691151 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4bd00abdb0 con 0x7f4bd815ca90 2024-03-23T09:52:14.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.898+0000 7f4bdc8b2640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4bbc06ac30 0x7f4bbc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:14.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.898+0000 7f4bdc8b2640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4bbc06ac30 0x7f4bbc06d0f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc000f60 tx=0x7f4bcc007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:14.902+0000 7f4bd57fa640 1 -- 172.21.15.47:0/4212691151 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4bd00b0070 con 0x7f4bd815ca90 2024-03-23T09:52:15.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.083+0000 7f4bdeb16640 1 -- 172.21.15.47:0/4212691151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bd8066be0 con 0x7f4bd815ca90 2024-03-23T09:52:15.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.084+0000 7f4bd57fa640 1 -- 172.21.15.47:0/4212691151 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16340 (secure 0 0 0) 0x7f4bd0076de0 con 0x7f4bd815ca90 2024-03-23T09:52:15.085 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:15.085 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":117,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:52:05.460401+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:51:38.679890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-23T09:51:39.396299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6834/724782441":"2024-03-24T09:52:05.460377+0000","172.21.15.47:6832/724782441":"2024-03-24T09:52:05.460377+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:52:15.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.086+0000 7f4bdeb16640 1 -- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4bbc06ac30 msgr2=0x7f4bbc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.086+0000 7f4bdeb16640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4bbc06ac30 0x7f4bbc06d0f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc000f60 tx=0x7f4bcc007b50 comp rx=0 tx=0).stop 2024-03-23T09:52:15.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.086+0000 7f4bdeb16640 1 -- 172.21.15.47:0/4212691151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd815ca90 msgr2=0x7f4bd815d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.086+0000 7f4bdeb16640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd815ca90 0x7f4bd815d6b0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009eb0 tx=0x7f4bd0004a40 comp rx=0 tx=0).stop 2024-03-23T09:52:15.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.086+0000 7f4bdeb16640 1 -- 172.21.15.47:0/4212691151 shutdown_connections 2024-03-23T09:52:15.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.086+0000 7f4bdeb16640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4bbc06ac30 0x7f4bbc06d0f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.086+0000 7f4bdeb16640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd815ca90 0x7f4bd815d6b0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.086+0000 7f4bdeb16640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bd80cde10 0x7f4bd815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.086+0000 7f4bdeb16640 1 --2- 172.21.15.47:0/4212691151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd80b8350 0x7f4bd8153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.086+0000 7f4bdeb16640 1 -- 172.21.15.47:0/4212691151 >> 172.21.15.47:0/4212691151 conn(0x7f4bd80659d0 msgr2=0x7f4bd815bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:15.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.086+0000 7f4bdeb16640 1 -- 172.21.15.47:0/4212691151 shutdown_connections 2024-03-23T09:52:15.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.086+0000 7f4bdeb16640 1 -- 172.21.15.47:0/4212691151 wait complete. 2024-03-23T09:52:15.098 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:51:38.679890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:51:39.396299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:52:15.098 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:52:15.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.202+0000 7f67753ea640 1 Processor -- start 2024-03-23T09:52:15.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.202+0000 7f67753ea640 1 -- start start 2024-03-23T09:52:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.203+0000 7f67753ea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f677015c990 0x7f6770160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.203+0000 7f67753ea640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f67700b8350 0x7f67700cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.203+0000 7f67753ea640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6770161630 con 0x7f67700cdc40 2024-03-23T09:52:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.203+0000 7f67753ea640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67700ce0b0 con 0x7f67700b8350 2024-03-23T09:52:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.203+0000 7f67753ea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67700d1030 con 0x7f677015c990 2024-03-23T09:52:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.203+0000 7f676e7fc640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f67700b8350 0x7f67700cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.203+0000 7f676effd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f677015c990 0x7f6770160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.203+0000 7f676e7fc640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f67700b8350 0x7f67700cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:52100/0 (socket says 172.21.15.47:52100) 2024-03-23T09:52:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.203+0000 7f676e7fc640 1 -- 172.21.15.47:0/2669659346 learned_addr learned my addr 172.21.15.47:0/2669659346 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.204+0000 7f676e7fc640 1 -- 172.21.15.47:0/2669659346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f677015c990 msgr2=0x7f6770160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.204+0000 7f676e7fc640 1 --2- 172.21.15.47:0/2669659346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f677015c990 0x7f6770160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.204+0000 7f676e7fc640 1 -- 172.21.15.47:0/2669659346 >> v1:172.21.15.175:6789/0 conn(0x7f67700cdc40 legacy=0x7f67700b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.204+0000 7f676e7fc640 1 -- 172.21.15.47:0/2669659346 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67700cfe10 con 0x7f67700b8350 2024-03-23T09:52:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.204+0000 7f676effd640 1 --2- 172.21.15.47:0/2669659346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f677015c990 0x7f6770160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:52:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.205+0000 7f676e7fc640 1 --2- 172.21.15.47:0/2669659346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67700b8350 0x7f67700cd700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6758009870 tx=0x7f675802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bd0f538d7e56870 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.205+0000 7f676dffb640 1 -- 172.21.15.47:0/2669659346 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758031640 con 0x7f67700b8350 2024-03-23T09:52:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.205+0000 7f676dffb640 1 -- 172.21.15.47:0/2669659346 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67580317a0 con 0x7f67700b8350 2024-03-23T09:52:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.205+0000 7f676dffb640 1 -- 172.21.15.47:0/2669659346 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758031b30 con 0x7f67700b8350 2024-03-23T09:52:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.205+0000 7f67753ea640 1 -- 172.21.15.47:0/2669659346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67700b8350 msgr2=0x7f67700cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.205+0000 7f67753ea640 1 --2- 172.21.15.47:0/2669659346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67700b8350 0x7f67700cd700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6758009870 tx=0x7f675802b8f0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.205+0000 7f67753ea640 1 -- 172.21.15.47:0/2669659346 shutdown_connections 2024-03-23T09:52:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.205+0000 7f67753ea640 1 --2- 172.21.15.47:0/2669659346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67700b8350 0x7f67700cd700 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.205+0000 7f67753ea640 1 --2- 172.21.15.47:0/2669659346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f677015c990 0x7f6770160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.205+0000 7f67753ea640 1 -- 172.21.15.47:0/2669659346 >> 172.21.15.47:0/2669659346 conn(0x7f67700659d0 msgr2=0x7f6770065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.205+0000 7f67753ea640 1 -- 172.21.15.47:0/2669659346 shutdown_connections 2024-03-23T09:52:15.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.205+0000 7f67753ea640 1 -- 172.21.15.47:0/2669659346 wait complete. 2024-03-23T09:52:15.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.207+0000 7f67753ea640 1 Processor -- start 2024-03-23T09:52:15.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.207+0000 7f67753ea640 1 -- start start 2024-03-23T09:52:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.207+0000 7f67753ea640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700b8350 0x7f6770157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.207+0000 7f67753ea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67700cdc40 0x7f67700b7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.207+0000 7f67753ea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015c990 0x7f67700b7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.207+0000 7f67753ea640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67701581c0 con 0x7f67700cdc40 2024-03-23T09:52:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.207+0000 7f67753ea640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6770159af0 con 0x7f677015c990 2024-03-23T09:52:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.207+0000 7f67753ea640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67700caaa0 con 0x7f67700b8350 2024-03-23T09:52:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f676e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67700cdc40 0x7f67700b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f676e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67700cdc40 0x7f67700b7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35300/0 (socket says 172.21.15.47:35300) 2024-03-23T09:52:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f676e7fc640 1 -- 172.21.15.47:0/3665404942 learned_addr learned my addr 172.21.15.47:0/3665404942 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f676effd640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700b8350 0x7f6770157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f676f7fe640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015c990 0x7f67700b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f676e7fc640 1 -- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700b8350 msgr2=0x7f6770157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f676e7fc640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700b8350 0x7f6770157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f676e7fc640 1 -- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015c990 msgr2=0x7f67700b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f676e7fc640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015c990 0x7f67700b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f676e7fc640 1 -- 172.21.15.47:0/3665404942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6768009400 con 0x7f67700cdc40 2024-03-23T09:52:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f676e7fc640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67700cdc40 0x7f67700b7010 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f6758000c00 tx=0x7f67580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f67677fe640 1 -- 172.21.15.47:0/3665404942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675803a930 con 0x7f67700cdc40 2024-03-23T09:52:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f67677fe640 1 -- 172.21.15.47:0/3665404942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675803aa90 con 0x7f67700cdc40 2024-03-23T09:52:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f67753ea640 1 -- 172.21.15.47:0/3665404942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67580094d0 con 0x7f67700cdc40 2024-03-23T09:52:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f67677fe640 1 -- 172.21.15.47:0/3665404942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675803ae40 con 0x7f67700cdc40 2024-03-23T09:52:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.208+0000 7f67753ea640 1 -- 172.21.15.47:0/3665404942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67700cb060 con 0x7f67700cdc40 2024-03-23T09:52:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.209+0000 7f67753ea640 1 -- 172.21.15.47:0/3665404942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6770072360 con 0x7f67700cdc40 2024-03-23T09:52:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.210+0000 7f67677fe640 1 -- 172.21.15.47:0/3665404942 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f675802f050 con 0x7f67700cdc40 2024-03-23T09:52:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.210+0000 7f67677fe640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f675006ac00 0x7f675006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.210+0000 7f67677fe640 1 -- 172.21.15.47:0/3665404942 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f67580ab0a0 con 0x7f67700cdc40 2024-03-23T09:52:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.211+0000 7f676effd640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f675006ac00 0x7f675006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.211+0000 7f676effd640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f675006ac00 0x7f675006d0c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6768009cd0 tx=0x7f6768009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.214+0000 7f67677fe640 1 -- 172.21.15.47:0/3665404942 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6758076210 con 0x7f67700cdc40 2024-03-23T09:52:15.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.396+0000 7f67753ea640 1 -- 172.21.15.47:0/3665404942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6770066be0 con 0x7f67700cdc40 2024-03-23T09:52:15.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.396+0000 7f67677fe640 1 -- 172.21.15.47:0/3665404942 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6758079830 con 0x7f67700cdc40 2024-03-23T09:52:15.397 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:52:15.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.398+0000 7f67753ea640 1 -- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f675006ac00 msgr2=0x7f675006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.398+0000 7f67753ea640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f675006ac00 0x7f675006d0c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6768009cd0 tx=0x7f6768009190 comp rx=0 tx=0).stop 2024-03-23T09:52:15.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.399+0000 7f67753ea640 1 -- 172.21.15.47:0/3665404942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67700cdc40 msgr2=0x7f67700b7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.399+0000 7f67753ea640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67700cdc40 0x7f67700b7010 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f6758000c00 tx=0x7f67580046c0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.399+0000 7f67753ea640 1 -- 172.21.15.47:0/3665404942 shutdown_connections 2024-03-23T09:52:15.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.399+0000 7f67753ea640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f675006ac00 0x7f675006d0c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.399+0000 7f67753ea640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015c990 0x7f67700b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.399+0000 7f67753ea640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67700cdc40 0x7f67700b7010 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.399+0000 7f67753ea640 1 --2- 172.21.15.47:0/3665404942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700b8350 0x7f6770157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.399+0000 7f67753ea640 1 -- 172.21.15.47:0/3665404942 >> 172.21.15.47:0/3665404942 conn(0x7f67700659d0 msgr2=0x7f6770065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:15.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.399+0000 7f67753ea640 1 -- 172.21.15.47:0/3665404942 shutdown_connections 2024-03-23T09:52:15.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.399+0000 7f67753ea640 1 -- 172.21.15.47:0/3665404942 wait complete. 2024-03-23T09:52:15.410 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:52:15.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.514+0000 7f91152d5640 1 Processor -- start 2024-03-23T09:52:15.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.514+0000 7f91152d5640 1 -- start start 2024-03-23T09:52:15.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.514+0000 7f91152d5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9110161020 0x7f91100b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.514+0000 7f91152d5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f911006bf10 0x7f91100ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.514+0000 7f91152d5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91100cdad0 con 0x7f9110161400 2024-03-23T09:52:15.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.514+0000 7f91152d5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91100b03c0 con 0x7f9110161020 2024-03-23T09:52:15.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.514+0000 7f91152d5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91100b06a0 con 0x7f911006bf10 2024-03-23T09:52:15.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.515+0000 7f910effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9110161020 0x7f91100b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.515+0000 7f910effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9110161020 0x7f91100b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35310/0 (socket says 172.21.15.47:35310) 2024-03-23T09:52:15.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.515+0000 7f910effd640 1 -- 172.21.15.47:0/3761760213 learned_addr learned my addr 172.21.15.47:0/3761760213 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:15.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.515+0000 7f910f7fe640 1 --2- 172.21.15.47:0/3761760213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f911006bf10 0x7f91100ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.515+0000 7f910effd640 1 -- 172.21.15.47:0/3761760213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f911006bf10 msgr2=0x7f91100ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:15.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.515+0000 7f910effd640 1 --2- 172.21.15.47:0/3761760213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f911006bf10 0x7f91100ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.515+0000 7f910effd640 1 -- 172.21.15.47:0/3761760213 >> v1:172.21.15.175:6789/0 conn(0x7f9110161400 legacy=0x7f911006b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.515+0000 7f910effd640 1 -- 172.21.15.47:0/3761760213 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100b0980 con 0x7f9110161020 2024-03-23T09:52:15.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.516+0000 7f910effd640 1 --2- 172.21.15.47:0/3761760213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9110161020 0x7f91100b1180 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f9100009870 tx=0x7f910002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2d51e10a03947c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:15.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.516+0000 7f910dffb640 1 -- 172.21.15.47:0/3761760213 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910002f840 con 0x7f9110161020 2024-03-23T09:52:15.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.516+0000 7f910dffb640 1 -- 172.21.15.47:0/3761760213 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910002f9a0 con 0x7f9110161020 2024-03-23T09:52:15.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.516+0000 7f910dffb640 1 -- 172.21.15.47:0/3761760213 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910002fd30 con 0x7f9110161020 2024-03-23T09:52:15.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.516+0000 7f91152d5640 1 -- 172.21.15.47:0/3761760213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9110161020 msgr2=0x7f91100b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.516+0000 7f91152d5640 1 --2- 172.21.15.47:0/3761760213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9110161020 0x7f91100b1180 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f9100009870 tx=0x7f910002ba20 comp rx=0 tx=0).stop 2024-03-23T09:52:15.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.516+0000 7f91152d5640 1 -- 172.21.15.47:0/3761760213 shutdown_connections 2024-03-23T09:52:15.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.516+0000 7f91152d5640 1 --2- 172.21.15.47:0/3761760213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f911006bf10 0x7f91100ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.516+0000 7f91152d5640 1 --2- 172.21.15.47:0/3761760213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9110161020 0x7f91100b1180 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.516+0000 7f91152d5640 1 -- 172.21.15.47:0/3761760213 >> 172.21.15.47:0/3761760213 conn(0x7f91100d0670 msgr2=0x7f91100d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:15.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.516+0000 7f91152d5640 1 -- 172.21.15.47:0/3761760213 shutdown_connections 2024-03-23T09:52:15.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.516+0000 7f91152d5640 1 -- 172.21.15.47:0/3761760213 wait complete. 2024-03-23T09:52:15.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.518+0000 7f91152d5640 1 Processor -- start 2024-03-23T09:52:15.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.518+0000 7f91152d5640 1 -- start start 2024-03-23T09:52:15.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f91152d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f911006bf10 0x7f91100ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f91152d5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9110161020 0x7f91101f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f91152d5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9110161400 0x7f91100b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f91152d5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91101536e0 con 0x7f9110161020 2024-03-23T09:52:15.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f91152d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100ca630 con 0x7f911006bf10 2024-03-23T09:52:15.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f91152d5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91101f6420 con 0x7f9110161400 2024-03-23T09:52:15.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f910e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9110161020 0x7f91101f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f910e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9110161020 0x7f91101f5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35320/0 (socket says 172.21.15.47:35320) 2024-03-23T09:52:15.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f910e7fc640 1 -- 172.21.15.47:0/1448771207 learned_addr learned my addr 172.21.15.47:0/1448771207 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:15.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f910effd640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f911006bf10 0x7f91100ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f910f7fe640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9110161400 0x7f91100b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f910e7fc640 1 -- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9110161400 msgr2=0x7f91100b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f910e7fc640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9110161400 0x7f91100b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f910e7fc640 1 -- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f911006bf10 msgr2=0x7f91100ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f910e7fc640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f911006bf10 0x7f91100ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f910e7fc640 1 -- 172.21.15.47:0/1448771207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91000094d0 con 0x7f9110161020 2024-03-23T09:52:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f910e7fc640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9110161020 0x7f91101f5ee0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f910400c430 tx=0x7f910400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f90f77fe640 1 -- 172.21.15.47:0/1448771207 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91040052d0 con 0x7f9110161020 2024-03-23T09:52:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f90f77fe640 1 -- 172.21.15.47:0/1448771207 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9104005430 con 0x7f9110161020 2024-03-23T09:52:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f90f77fe640 1 -- 172.21.15.47:0/1448771207 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104003830 con 0x7f9110161020 2024-03-23T09:52:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f91152d5640 1 -- 172.21.15.47:0/1448771207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91101f6610 con 0x7f9110161020 2024-03-23T09:52:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.519+0000 7f91152d5640 1 -- 172.21.15.47:0/1448771207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91101f6bd0 con 0x7f9110161020 2024-03-23T09:52:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.520+0000 7f91152d5640 1 -- 172.21.15.47:0/1448771207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9110153820 con 0x7f9110161020 2024-03-23T09:52:15.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.521+0000 7f90f77fe640 1 -- 172.21.15.47:0/1448771207 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9104003990 con 0x7f9110161020 2024-03-23T09:52:15.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.521+0000 7f90f77fe640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90fc06ac00 0x7f90fc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.521+0000 7f90f77fe640 1 -- 172.21.15.47:0/1448771207 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9104088850 con 0x7f9110161020 2024-03-23T09:52:15.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.521+0000 7f910effd640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90fc06ac00 0x7f90fc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.522+0000 7f910effd640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90fc06ac00 0x7f90fc06d0c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9100036000 tx=0x7f9100008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:15.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.525+0000 7f90f77fe640 1 -- 172.21.15.47:0/1448771207 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9110153820 con 0x7f9110161020 2024-03-23T09:52:15.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.707+0000 7f91152d5640 1 -- 172.21.15.47:0/1448771207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9110153820 con 0x7f9110161020 2024-03-23T09:52:15.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.707+0000 7f90f77fe640 1 -- 172.21.15.47:0/1448771207 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9104017020 con 0x7f9110161020 2024-03-23T09:52:15.708 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:52:15.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.709+0000 7f91152d5640 1 -- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90fc06ac00 msgr2=0x7f90fc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.710+0000 7f91152d5640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90fc06ac00 0x7f90fc06d0c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9100036000 tx=0x7f9100008030 comp rx=0 tx=0).stop 2024-03-23T09:52:15.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.710+0000 7f91152d5640 1 -- 172.21.15.47:0/1448771207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9110161020 msgr2=0x7f91101f5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.710+0000 7f91152d5640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9110161020 0x7f91101f5ee0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f910400c430 tx=0x7f910400c8f0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.710+0000 7f91152d5640 1 -- 172.21.15.47:0/1448771207 shutdown_connections 2024-03-23T09:52:15.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.710+0000 7f91152d5640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90fc06ac00 0x7f90fc06d0c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.710+0000 7f91152d5640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9110161400 0x7f91100b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.710+0000 7f91152d5640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9110161020 0x7f91101f5ee0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.710+0000 7f91152d5640 1 --2- 172.21.15.47:0/1448771207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f911006bf10 0x7f91100ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.710+0000 7f91152d5640 1 -- 172.21.15.47:0/1448771207 >> 172.21.15.47:0/1448771207 conn(0x7f91100d0670 msgr2=0x7f911006f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:15.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.710+0000 7f91152d5640 1 -- 172.21.15.47:0/1448771207 shutdown_connections 2024-03-23T09:52:15.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.710+0000 7f91152d5640 1 -- 172.21.15.47:0/1448771207 wait complete. 2024-03-23T09:52:15.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:52:15.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.825+0000 7f57ed93e640 1 Processor -- start 2024-03-23T09:52:15.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.825+0000 7f57ed93e640 1 -- start start 2024-03-23T09:52:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57ed93e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80cdcb0 0x7f57e80b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57ed93e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f57e80d2020 0x7f57e80b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57ed93e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e80b72f0 con 0x7f57e815cb50 2024-03-23T09:52:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57ed93e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e80b7e70 con 0x7f57e80d2020 2024-03-23T09:52:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57ed93e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e815cf30 con 0x7f57e80cdcb0 2024-03-23T09:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e6ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f57e815cb50 0x7f57e80cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39890/0 (socket says 172.21.15.47:39890) 2024-03-23T09:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e6ffd640 1 -- 172.21.15.47:0/4191081635 learned_addr learned my addr 172.21.15.47:0/4191081635 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142160070 0 0) 0x7f57e80b72f0 con 0x7f57e815cb50 2024-03-23T09:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57d4003520 con 0x7f57e815cb50 2024-03-23T09:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e77fe640 1 --2- 172.21.15.47:0/4191081635 >> v2:172.21.15.175:3300/0 conn(0x7f57e80d2020 0x7f57e80b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e67fc640 1 --2- 172.21.15.47:0/4191081635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80cdcb0 0x7f57e80b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1976994478 0 0) 0x7f57d4003520 con 0x7f57e815cb50 2024-03-23T09:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80cdcb0 msgr2=0x7f57e80b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e5ffb640 1 --2- 172.21.15.47:0/4191081635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80cdcb0 0x7f57e80b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 >> v2:172.21.15.175:3300/0 conn(0x7f57e80d2020 msgr2=0x7f57e80b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e5ffb640 1 --2- 172.21.15.47:0/4191081635 >> v2:172.21.15.175:3300/0 conn(0x7f57e80d2020 0x7f57e80b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e80b75d0 con 0x7f57e815cb50 2024-03-23T09:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f57e0001670 con 0x7f57e815cb50 2024-03-23T09:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 >> v1:172.21.15.47:6789/0 conn(0x7f57e815cb50 legacy=0x7f57e80cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.826+0000 7f57e5ffb640 1 --2- 172.21.15.47:0/4191081635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80cdcb0 0x7f57d4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.827+0000 7f57e5ffb640 1 --2- 172.21.15.47:0/4191081635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57d4003ef0 0x7f57d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.827+0000 7f57e5ffb640 1 --2- 172.21.15.47:0/4191081635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57d4004d00 0x7f57d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.827+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80b7e70 con 0x7f57d4003ef0 2024-03-23T09:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.827+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e815cf30 con 0x7f57d4004d00 2024-03-23T09:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.827+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d400b9b0 con 0x7f57e80cdcb0 2024-03-23T09:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.827+0000 7f57e67fc640 1 --2- 172.21.15.47:0/4191081635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57d4003ef0 0x7f57d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.827+0000 7f57e6ffd640 1 --2- 172.21.15.47:0/4191081635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80cdcb0 0x7f57d4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.827+0000 7f57e77fe640 1 --2- 172.21.15.47:0/4191081635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57d4004d00 0x7f57d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.827+0000 7f57e67fc640 1 -- 172.21.15.47:0/4191081635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80cdcb0 msgr2=0x7f57d4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:15.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.827+0000 7f57e67fc640 1 --2- 172.21.15.47:0/4191081635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80cdcb0 0x7f57d4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.827+0000 7f57e67fc640 1 -- 172.21.15.47:0/4191081635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57d4004d00 msgr2=0x7f57d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:15.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.827+0000 7f57e67fc640 1 --2- 172.21.15.47:0/4191081635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57d4004d00 0x7f57d4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.827+0000 7f57e67fc640 1 -- 172.21.15.47:0/4191081635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d4003520 con 0x7f57d4003ef0 2024-03-23T09:52:15.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.828+0000 7f57e67fc640 1 --2- 172.21.15.47:0/4191081635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57d4003ef0 0x7f57d40047c0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009ac0 tx=0x7f57d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce238ed648627171 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:15.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.828+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d002c6b0 con 0x7f57d4003ef0 2024-03-23T09:52:15.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.828+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d002c810 con 0x7f57d4003ef0 2024-03-23T09:52:15.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.828+0000 7f57e5ffb640 1 -- 172.21.15.47:0/4191081635 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d002cba0 con 0x7f57d4003ef0 2024-03-23T09:52:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.828+0000 7f57ed93e640 1 -- 172.21.15.47:0/4191081635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57d4003ef0 msgr2=0x7f57d40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.828+0000 7f57ed93e640 1 --2- 172.21.15.47:0/4191081635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57d4003ef0 0x7f57d40047c0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009ac0 tx=0x7f57d40057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.828+0000 7f57e77fe640 1 -- 172.21.15.47:0/4191081635 reap_dead start 2024-03-23T09:52:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.828+0000 7f57ed93e640 1 -- 172.21.15.47:0/4191081635 shutdown_connections 2024-03-23T09:52:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.828+0000 7f57ed93e640 1 -- 172.21.15.47:0/4191081635 >> 172.21.15.47:0/4191081635 conn(0x7f57e8065df0 msgr2=0x7f57e8066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.828+0000 7f57ed93e640 1 -- 172.21.15.47:0/4191081635 shutdown_connections 2024-03-23T09:52:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.828+0000 7f57ed93e640 1 -- 172.21.15.47:0/4191081635 wait complete. 2024-03-23T09:52:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.830+0000 7f57ed93e640 1 Processor -- start 2024-03-23T09:52:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.830+0000 7f57ed93e640 1 -- start start 2024-03-23T09:52:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.830+0000 7f57ed93e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57d4003ef0 0x7f57e815f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.830+0000 7f57ed93e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e80cc4d0 0x7f57e8142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.830+0000 7f57ed93e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80cc950 0x7f57e8142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.830+0000 7f57ed93e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e81f3210 con 0x7f57e80cc4d0 2024-03-23T09:52:15.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.830+0000 7f57ed93e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80cb9f0 con 0x7f57d4003ef0 2024-03-23T09:52:15.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.830+0000 7f57ed93e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80cbcd0 con 0x7f57e80cc950 2024-03-23T09:52:15.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57e67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e80cc4d0 0x7f57e8142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57e67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e80cc4d0 0x7f57e8142740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35330/0 (socket says 172.21.15.47:35330) 2024-03-23T09:52:15.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57e67fc640 1 -- 172.21.15.47:0/4017135296 learned_addr learned my addr 172.21.15.47:0/4017135296 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:15.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57e6ffd640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57d4003ef0 0x7f57e815f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57e77fe640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80cc950 0x7f57e8142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57e67fc640 1 -- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80cc950 msgr2=0x7f57e8142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57e67fc640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80cc950 0x7f57e8142c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57e67fc640 1 -- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57d4003ef0 msgr2=0x7f57e815f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:15.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57e67fc640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57d4003ef0 0x7f57e815f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:15.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57e67fc640 1 -- 172.21.15.47:0/4017135296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e80cbfb0 con 0x7f57e80cc4d0 2024-03-23T09:52:15.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57e6ffd640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57d4003ef0 0x7f57e815f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:15.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57e67fc640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e80cc4d0 0x7f57e8142740 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f57d002c250 tx=0x7f57d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:15.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57dffff640 1 -- 172.21.15.47:0/4017135296 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d0003810 con 0x7f57e80cc4d0 2024-03-23T09:52:15.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57dffff640 1 -- 172.21.15.47:0/4017135296 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d0003970 con 0x7f57e80cc4d0 2024-03-23T09:52:15.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57ed93e640 1 -- 172.21.15.47:0/4017135296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e80cb6a0 con 0x7f57e80cc4d0 2024-03-23T09:52:15.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57dffff640 1 -- 172.21.15.47:0/4017135296 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d0003ce0 con 0x7f57e80cc4d0 2024-03-23T09:52:15.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.831+0000 7f57ed93e640 1 -- 172.21.15.47:0/4017135296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e80d13a0 con 0x7f57e80cc4d0 2024-03-23T09:52:15.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.832+0000 7f57ed93e640 1 -- 172.21.15.47:0/4017135296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e8072360 con 0x7f57e80cc4d0 2024-03-23T09:52:15.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.833+0000 7f57dffff640 1 -- 172.21.15.47:0/4017135296 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f57d0047450 con 0x7f57e80cc4d0 2024-03-23T09:52:15.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.833+0000 7f57dffff640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f57c806ac30 0x7f57c806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:15.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.833+0000 7f57dffff640 1 -- 172.21.15.47:0/4017135296 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f57d00ab5c0 con 0x7f57e80cc4d0 2024-03-23T09:52:15.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.833+0000 7f57e6ffd640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f57c806ac30 0x7f57c806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:15.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.834+0000 7f57e6ffd640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f57c806ac30 0x7f57c806d0f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f57e8152550 tx=0x7f57e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:15.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:15.837+0000 7f57dffff640 1 -- 172.21.15.47:0/4017135296 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f57d00767c0 con 0x7f57e80cc4d0 2024-03-23T09:52:16.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.015+0000 7f57ed93e640 1 -- 172.21.15.47:0/4017135296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57e80cb830 con 0x7f57e80cc4d0 2024-03-23T09:52:16.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.016+0000 7f57dffff640 1 -- 172.21.15.47:0/4017135296 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57d00044e0 con 0x7f57e80cc4d0 2024-03-23T09:52:16.016 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:52:16.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.018+0000 7f57ed93e640 1 -- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f57c806ac30 msgr2=0x7f57c806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.018+0000 7f57ed93e640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f57c806ac30 0x7f57c806d0f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f57e8152550 tx=0x7f57e0009040 comp rx=0 tx=0).stop 2024-03-23T09:52:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.018+0000 7f57ed93e640 1 -- 172.21.15.47:0/4017135296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e80cc4d0 msgr2=0x7f57e8142740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.018+0000 7f57ed93e640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e80cc4d0 0x7f57e8142740 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f57d002c250 tx=0x7f57d0004730 comp rx=0 tx=0).stop 2024-03-23T09:52:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.018+0000 7f57ed93e640 1 -- 172.21.15.47:0/4017135296 shutdown_connections 2024-03-23T09:52:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.018+0000 7f57ed93e640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f57c806ac30 0x7f57c806d0f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.018+0000 7f57ed93e640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57e80cc950 0x7f57e8142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.018+0000 7f57ed93e640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e80cc4d0 0x7f57e8142740 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.018+0000 7f57ed93e640 1 --2- 172.21.15.47:0/4017135296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57d4003ef0 0x7f57e815f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.018+0000 7f57ed93e640 1 -- 172.21.15.47:0/4017135296 >> 172.21.15.47:0/4017135296 conn(0x7f57e8065df0 msgr2=0x7f57e80b6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.018+0000 7f57ed93e640 1 -- 172.21.15.47:0/4017135296 shutdown_connections 2024-03-23T09:52:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.018+0000 7f57ed93e640 1 -- 172.21.15.47:0/4017135296 wait complete. 2024-03-23T09:52:16.029 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:52:16.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.131+0000 7f7c70755640 1 Processor -- start 2024-03-23T09:52:16.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.131+0000 7f7c70755640 1 -- start start 2024-03-23T09:52:16.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.132+0000 7f7c70755640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7c6815cc40 0x7f7c68160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.132+0000 7f7c70755640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c68160c90 0x7f7c680cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.132+0000 7f7c70755640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c680d09d0 con 0x7f7c680cdbf0 2024-03-23T09:52:16.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.132+0000 7f7c70755640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c680ce0a0 con 0x7f7c6815cc40 2024-03-23T09:52:16.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.132+0000 7f7c70755640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680b5a80 con 0x7f7c68160c90 2024-03-23T09:52:16.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.132+0000 7f7c6ecf2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7c680cdbf0 0x7f7c680b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39902/0 (socket says 172.21.15.47:39902) 2024-03-23T09:52:16.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.132+0000 7f7c6ecf2640 1 -- 172.21.15.47:0/877180216 learned_addr learned my addr 172.21.15.47:0/877180216 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:16.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6e4f1640 1 --2- 172.21.15.47:0/877180216 >> v2:172.21.15.175:3300/0 conn(0x7f7c6815cc40 0x7f7c68160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191441418 0 0) 0x7f7c680d09d0 con 0x7f7c680cdbf0 2024-03-23T09:52:16.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c50003520 con 0x7f7c680cdbf0 2024-03-23T09:52:16.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6dcf0640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c68160c90 0x7f7c680cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1095854468 0 0) 0x7f7c50003520 con 0x7f7c680cdbf0 2024-03-23T09:52:16.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c68160c90 msgr2=0x7f7c680cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:16.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c68160c90 0x7f7c680cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 >> v2:172.21.15.175:3300/0 conn(0x7f7c6815cc40 msgr2=0x7f7c68160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:16.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 --2- 172.21.15.47:0/877180216 >> v2:172.21.15.175:3300/0 conn(0x7f7c6815cc40 0x7f7c68160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6815d020 con 0x7f7c680cdbf0 2024-03-23T09:52:16.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f7c580019f0 con 0x7f7c680cdbf0 2024-03-23T09:52:16.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 >> v1:172.21.15.47:6789/0 conn(0x7f7c680cdbf0 legacy=0x7f7c680b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c68160c90 0x7f7c50003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c50003ef0 0x7f7c500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c50004d00 0x7f7c50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680ce0a0 con 0x7f7c50003ef0 2024-03-23T09:52:16.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680b5a80 con 0x7f7c50004d00 2024-03-23T09:52:16.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5000b9d0 con 0x7f7c68160c90 2024-03-23T09:52:16.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6dcf0640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c50003ef0 0x7f7c500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6e4f1640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c68160c90 0x7f7c50003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6dcf0640 1 -- 172.21.15.47:0/877180216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c68160c90 msgr2=0x7f7c50003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:16.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6dcf0640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c68160c90 0x7f7c50003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6ecf2640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c50004d00 0x7f7c50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6dcf0640 1 -- 172.21.15.47:0/877180216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c50004d00 msgr2=0x7f7c50005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:16.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6dcf0640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c50004d00 0x7f7c50005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6dcf0640 1 -- 172.21.15.47:0/877180216 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c50003520 con 0x7f7c50003ef0 2024-03-23T09:52:16.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6e4f1640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c68160c90 0x7f7c50003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:16.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.133+0000 7f7c6ecf2640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c50004d00 0x7f7c50005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:16.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.134+0000 7f7c6dcf0640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c50003ef0 0x7f7c500047c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00ba30 tx=0x7f7c500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0d7b60960bc14d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:16.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.134+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c032640 con 0x7f7c50003ef0 2024-03-23T09:52:16.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.134+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c0327a0 con 0x7f7c50003ef0 2024-03-23T09:52:16.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.134+0000 7f7c6d4ef640 1 -- 172.21.15.47:0/877180216 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c032b30 con 0x7f7c50003ef0 2024-03-23T09:52:16.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.135+0000 7f7c70755640 1 -- 172.21.15.47:0/877180216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c50003ef0 msgr2=0x7f7c500047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.135+0000 7f7c70755640 1 --2- 172.21.15.47:0/877180216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c50003ef0 0x7f7c500047c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00ba30 tx=0x7f7c500057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.135+0000 7f7c6ecf2640 1 -- 172.21.15.47:0/877180216 reap_dead start 2024-03-23T09:52:16.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.135+0000 7f7c70755640 1 -- 172.21.15.47:0/877180216 shutdown_connections 2024-03-23T09:52:16.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.135+0000 7f7c70755640 1 -- 172.21.15.47:0/877180216 >> 172.21.15.47:0/877180216 conn(0x7f7c6806b160 msgr2=0x7f7c6806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:16.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.135+0000 7f7c70755640 1 -- 172.21.15.47:0/877180216 shutdown_connections 2024-03-23T09:52:16.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.135+0000 7f7c70755640 1 -- 172.21.15.47:0/877180216 wait complete. 2024-03-23T09:52:16.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.136+0000 7f7c70755640 1 Processor -- start 2024-03-23T09:52:16.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.136+0000 7f7c70755640 1 -- start start 2024-03-23T09:52:16.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c70755640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c50003ef0 0x7f7c68150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c70755640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c680c8120 0x7f7c6815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c70755640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c680c85a0 0x7f7c6815f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c70755640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c68151280 con 0x7f7c680c8120 2024-03-23T09:52:16.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c70755640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6815e340 con 0x7f7c680c85a0 2024-03-23T09:52:16.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c70755640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6815e620 con 0x7f7c50003ef0 2024-03-23T09:52:16.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c6dcf0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c680c8120 0x7f7c6815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c6dcf0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c680c8120 0x7f7c6815f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35360/0 (socket says 172.21.15.47:35360) 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c6dcf0640 1 -- 172.21.15.47:0/3150300272 learned_addr learned my addr 172.21.15.47:0/3150300272 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c6e4f1640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c50003ef0 0x7f7c68150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c6ecf2640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c680c85a0 0x7f7c6815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c6dcf0640 1 -- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c50003ef0 msgr2=0x7f7c68150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c6dcf0640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c50003ef0 0x7f7c68150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c6dcf0640 1 -- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c680c85a0 msgr2=0x7f7c6815f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c6dcf0640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c680c85a0 0x7f7c6815f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c6dcf0640 1 -- 172.21.15.47:0/3150300272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6815e900 con 0x7f7c680c8120 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c6e4f1640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c50003ef0 0x7f7c68150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.137+0000 7f7c6dcf0640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c680c8120 0x7f7c6815f3e0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c02bec0 tx=0x7f7c5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.138+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3150300272 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c004220 con 0x7f7c680c8120 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.138+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3150300272 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c0043a0 con 0x7f7c680c8120 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.138+0000 7f7c70755640 1 -- 172.21.15.47:0/3150300272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6815ebe0 con 0x7f7c680c8120 2024-03-23T09:52:16.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.138+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3150300272 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c03ba30 con 0x7f7c680c8120 2024-03-23T09:52:16.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.138+0000 7f7c70755640 1 -- 172.21.15.47:0/3150300272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c680c7bd0 con 0x7f7c680c8120 2024-03-23T09:52:16.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.139+0000 7f7c70755640 1 -- 172.21.15.47:0/3150300272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c680c7d90 con 0x7f7c680c8120 2024-03-23T09:52:16.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.139+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3150300272 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7c5c03bb90 con 0x7f7c680c8120 2024-03-23T09:52:16.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.139+0000 7f7c66ffd640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7c5406ac30 0x7f7c5406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.140+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3150300272 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7c5c0abaa0 con 0x7f7c680c8120 2024-03-23T09:52:16.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.140+0000 7f7c6e4f1640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7c5406ac30 0x7f7c5406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.140+0000 7f7c6e4f1640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7c5406ac30 0x7f7c5406d0f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7c681f5b10 tx=0x7f7c60008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:16.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.144+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3150300272 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7c5c076c80 con 0x7f7c680c8120 2024-03-23T09:52:16.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.335+0000 7f7c70755640 1 -- 172.21.15.47:0/3150300272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7c680c7d90 con 0x7f7c680c8120 2024-03-23T09:52:16.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.335+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3150300272 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v117) v1 ==== 80+0+120 (secure 0 0 0) 0x7f7c5c036080 con 0x7f7c680c8120 2024-03-23T09:52:16.336 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6834/724782441 2024-03-24T09:52:05.460377+0000 2024-03-23T09:52:16.336 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6832/724782441 2024-03-24T09:52:05.460377+0000 2024-03-23T09:52:16.336 INFO:teuthology.orchestra.run.smithi047.stderr:listed 2 entries 2024-03-23T09:52:16.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.337+0000 7f7c70755640 1 -- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7c5406ac30 msgr2=0x7f7c5406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.337+0000 7f7c70755640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7c5406ac30 0x7f7c5406d0f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7c681f5b10 tx=0x7f7c60008040 comp rx=0 tx=0).stop 2024-03-23T09:52:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.337+0000 7f7c70755640 1 -- 172.21.15.47:0/3150300272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c680c8120 msgr2=0x7f7c6815f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.337+0000 7f7c70755640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c680c8120 0x7f7c6815f3e0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c02bec0 tx=0x7f7c5c004730 comp rx=0 tx=0).stop 2024-03-23T09:52:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.337+0000 7f7c70755640 1 -- 172.21.15.47:0/3150300272 shutdown_connections 2024-03-23T09:52:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.337+0000 7f7c70755640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7c5406ac30 0x7f7c5406d0f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.337+0000 7f7c70755640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c680c85a0 0x7f7c6815f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.337+0000 7f7c70755640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c680c8120 0x7f7c6815f3e0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.337+0000 7f7c70755640 1 --2- 172.21.15.47:0/3150300272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c50003ef0 0x7f7c68150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.337+0000 7f7c70755640 1 -- 172.21.15.47:0/3150300272 >> 172.21.15.47:0/3150300272 conn(0x7f7c6806b160 msgr2=0x7f7c680b7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.337+0000 7f7c70755640 1 -- 172.21.15.47:0/3150300272 shutdown_connections 2024-03-23T09:52:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.337+0000 7f7c70755640 1 -- 172.21.15.47:0/3150300272 wait complete. 2024-03-23T09:52:16.348 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:52:16.348 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:52:16.363 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:52:16.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.503+0000 7f40ab555640 1 Processor -- start 2024-03-23T09:52:16.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.503+0000 7f40ab555640 1 -- start start 2024-03-23T09:52:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.504+0000 7f40ab555640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a40b42b0 0x7f40a4164450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.504+0000 7f40ab555640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a40ccd00 con 0x7f40a4143010 2024-03-23T09:52:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.504+0000 7f40ab555640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a40ccfe0 con 0x7f40a4142a90 2024-03-23T09:52:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.504+0000 7f40ab555640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40b27c0 con 0x7f40a40b42b0 2024-03-23T09:52:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.504+0000 7f40a8af0640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f40a4143010 0x7f40a40b3ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39912/0 (socket says 172.21.15.47:39912) 2024-03-23T09:52:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.504+0000 7f40a8af0640 1 -- 172.21.15.47:0/1042435534 learned_addr learned my addr 172.21.15.47:0/1042435534 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f40a9af2640 1 --2- 172.21.15.47:0/1042435534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a40b42b0 0x7f40a4164450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287607426 0 0) 0x7f40a40ccd00 con 0x7f40a4143010 2024-03-23T09:52:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4090003520 con 0x7f40a4143010 2024-03-23T09:52:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 332041382 0 0) 0x7f4090003520 con 0x7f40a4143010 2024-03-23T09:52:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a40b42b0 msgr2=0x7f40a4164450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 --2- 172.21.15.47:0/1042435534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a40b42b0 0x7f40a4164450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 >> v1:172.21.15.175:6789/0 conn(0x7f40a4142a90 legacy=0x7f40a40cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a40cd2c0 con 0x7f40a4143010 2024-03-23T09:52:16.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f4094001670 con 0x7f40a4143010 2024-03-23T09:52:16.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 >> v1:172.21.15.47:6789/0 conn(0x7f40a4143010 legacy=0x7f40a40b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 --2- 172.21.15.47:0/1042435534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a40b42b0 0x7f40900039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 --2- 172.21.15.47:0/1042435534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4090003f20 0x7f40900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 --2- 172.21.15.47:0/1042435534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090004d00 0x7f4090005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40b27c0 con 0x7f4090003f20 2024-03-23T09:52:16.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409000ba60 con 0x7f4090004d00 2024-03-23T09:52:16.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409000de30 con 0x7f40a40b42b0 2024-03-23T09:52:16.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f40a8af0640 1 --2- 172.21.15.47:0/1042435534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4090003f20 0x7f40900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f40a92f1640 1 --2- 172.21.15.47:0/1042435534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a40b42b0 0x7f40900039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f40a9af2640 1 --2- 172.21.15.47:0/1042435534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090004d00 0x7f4090005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f40a8af0640 1 -- 172.21.15.47:0/1042435534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a40b42b0 msgr2=0x7f40900039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f40a8af0640 1 --2- 172.21.15.47:0/1042435534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a40b42b0 0x7f40900039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f40a8af0640 1 -- 172.21.15.47:0/1042435534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090004d00 msgr2=0x7f4090005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f40a8af0640 1 --2- 172.21.15.47:0/1042435534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090004d00 0x7f4090005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f40a8af0640 1 -- 172.21.15.47:0/1042435534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4090003520 con 0x7f4090003f20 2024-03-23T09:52:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.505+0000 7f40a9af2640 1 --2- 172.21.15.47:0/1042435534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090004d00 0x7f4090005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.506+0000 7f40a8af0640 1 --2- 172.21.15.47:0/1042435534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4090003f20 0x7f40900047c0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f409400c240 tx=0x7f40900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb105a5b70f9be05 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.506+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094005ad0 con 0x7f4090003f20 2024-03-23T09:52:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.506+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4094005c30 con 0x7f4090003f20 2024-03-23T09:52:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.506+0000 7f409bfff640 1 -- 172.21.15.47:0/1042435534 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409402caf0 con 0x7f4090003f20 2024-03-23T09:52:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.507+0000 7f40ab555640 1 -- 172.21.15.47:0/1042435534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4090003f20 msgr2=0x7f40900047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.507+0000 7f40ab555640 1 --2- 172.21.15.47:0/1042435534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4090003f20 0x7f40900047c0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f409400c240 tx=0x7f40900057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.507+0000 7f40a9af2640 1 -- 172.21.15.47:0/1042435534 reap_dead start 2024-03-23T09:52:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.507+0000 7f40ab555640 1 -- 172.21.15.47:0/1042435534 shutdown_connections 2024-03-23T09:52:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.507+0000 7f40ab555640 1 -- 172.21.15.47:0/1042435534 >> 172.21.15.47:0/1042435534 conn(0x7f40a406a310 msgr2=0x7f40a40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.507+0000 7f40ab555640 1 -- 172.21.15.47:0/1042435534 shutdown_connections 2024-03-23T09:52:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.507+0000 7f40ab555640 1 -- 172.21.15.47:0/1042435534 wait complete. 2024-03-23T09:52:16.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.508+0000 7f40ab555640 1 Processor -- start 2024-03-23T09:52:16.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.508+0000 7f40ab555640 1 -- start start 2024-03-23T09:52:16.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40ab555640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090003f20 0x7f40a415c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40ab555640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a40ae850 0x7f40a40aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40ab555640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4139bd0 0x7f40a413a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40ab555640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40af210 con 0x7f40a40ae850 2024-03-23T09:52:16.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40ab555640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a4143a20 con 0x7f4090003f20 2024-03-23T09:52:16.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40ab555640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a415cde0 con 0x7f40a4139bd0 2024-03-23T09:52:16.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40a8af0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a40ae850 0x7f40a40aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40a8af0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a40ae850 0x7f40a40aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35380/0 (socket says 172.21.15.47:35380) 2024-03-23T09:52:16.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40a8af0640 1 -- 172.21.15.47:0/52421953 learned_addr learned my addr 172.21.15.47:0/52421953 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:16.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40a92f1640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090003f20 0x7f40a415c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40a9af2640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4139bd0 0x7f40a413a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40a8af0640 1 -- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4139bd0 msgr2=0x7f40a413a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40a8af0640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4139bd0 0x7f40a413a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40a8af0640 1 -- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090003f20 msgr2=0x7f40a415c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40a8af0640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090003f20 0x7f40a415c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40a8af0640 1 -- 172.21.15.47:0/52421953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a413a970 con 0x7f40a40ae850 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.509+0000 7f40a8af0640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a40ae850 0x7f40a40aecd0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f4094000c00 tx=0x7f4094001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.510+0000 7f4099ffb640 1 -- 172.21.15.47:0/52421953 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409403c040 con 0x7f40a40ae850 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.510+0000 7f4099ffb640 1 -- 172.21.15.47:0/52421953 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409402cce0 con 0x7f40a40ae850 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.510+0000 7f40ab555640 1 -- 172.21.15.47:0/52421953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a413ac50 con 0x7f40a40ae850 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.510+0000 7f4099ffb640 1 -- 172.21.15.47:0/52421953 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940072b0 con 0x7f40a40ae850 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.510+0000 7f40ab555640 1 -- 172.21.15.47:0/52421953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a41436b0 con 0x7f40a40ae850 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.511+0000 7f40ab555640 1 -- 172.21.15.47:0/52421953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a4073480 con 0x7f40a40ae850 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.511+0000 7f4099ffb640 1 -- 172.21.15.47:0/52421953 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4094032070 con 0x7f40a40ae850 2024-03-23T09:52:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.512+0000 7f4099ffb640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f408806ac00 0x7f408806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.512+0000 7f4099ffb640 1 -- 172.21.15.47:0/52421953 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f40940aa740 con 0x7f40a40ae850 2024-03-23T09:52:16.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.512+0000 7f40a92f1640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f408806ac00 0x7f408806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.512+0000 7f40a92f1640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f408806ac00 0x7f408806d0c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f40a00017e0 tx=0x7f40a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:16.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.516+0000 7f4099ffb640 1 -- 172.21.15.47:0/52421953 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4094075940 con 0x7f40a40ae850 2024-03-23T09:52:16.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.695+0000 7f40ab555640 1 -- 172.21.15.47:0/52421953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40a4066be0 con 0x7f40a40ae850 2024-03-23T09:52:16.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.696+0000 7f4099ffb640 1 -- 172.21.15.47:0/52421953 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16340 (secure 0 0 0) 0x7f4094078f60 con 0x7f40a40ae850 2024-03-23T09:52:16.696 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:16.696 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":117,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:52:05.460401+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:51:38.679890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-23T09:51:39.396299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6834/724782441":"2024-03-24T09:52:05.460377+0000","172.21.15.47:6832/724782441":"2024-03-24T09:52:05.460377+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:52:16.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.698+0000 7f40ab555640 1 -- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f408806ac00 msgr2=0x7f408806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.698+0000 7f40ab555640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f408806ac00 0x7f408806d0c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f40a00017e0 tx=0x7f40a0009040 comp rx=0 tx=0).stop 2024-03-23T09:52:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.698+0000 7f40ab555640 1 -- 172.21.15.47:0/52421953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a40ae850 msgr2=0x7f40a40aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.698+0000 7f40ab555640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a40ae850 0x7f40a40aecd0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f4094000c00 tx=0x7f4094001a00 comp rx=0 tx=0).stop 2024-03-23T09:52:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.698+0000 7f40ab555640 1 -- 172.21.15.47:0/52421953 shutdown_connections 2024-03-23T09:52:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.698+0000 7f40ab555640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f408806ac00 0x7f408806d0c0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.698+0000 7f40ab555640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40a4139bd0 0x7f40a413a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.698+0000 7f40ab555640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a40ae850 0x7f40a40aecd0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.698+0000 7f40ab555640 1 --2- 172.21.15.47:0/52421953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090003f20 0x7f40a415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.698+0000 7f40ab555640 1 -- 172.21.15.47:0/52421953 >> 172.21.15.47:0/52421953 conn(0x7f40a406a310 msgr2=0x7f40a40af930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.698+0000 7f40ab555640 1 -- 172.21.15.47:0/52421953 shutdown_connections 2024-03-23T09:52:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.698+0000 7f40ab555640 1 -- 172.21.15.47:0/52421953 wait complete. 2024-03-23T09:52:16.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:51:38.679890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:51:39.396299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:52:16.710 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:52:16.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.813+0000 7f76ea1fb640 1 Processor -- start 2024-03-23T09:52:16.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.813+0000 7f76ea1fb640 1 -- start start 2024-03-23T09:52:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.814+0000 7f76ea1fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76e4161020 0x7f76e40b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.814+0000 7f76ea1fb640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f76e406bf10 0x7f76e40ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.814+0000 7f76ea1fb640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e40cdad0 con 0x7f76e4161400 2024-03-23T09:52:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.814+0000 7f76ea1fb640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e40b03c0 con 0x7f76e406bf10 2024-03-23T09:52:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.814+0000 7f76ea1fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e40b06a0 con 0x7f76e4161020 2024-03-23T09:52:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.814+0000 7f76e3fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f76e406bf10 0x7f76e40ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.814+0000 7f76e37fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76e4161020 0x7f76e40b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.814+0000 7f76e3fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f76e406bf10 0x7f76e40ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:52194/0 (socket says 172.21.15.47:52194) 2024-03-23T09:52:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.815+0000 7f76e37fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76e4161020 0x7f76e40b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:42576/0 (socket says 172.21.15.47:42576) 2024-03-23T09:52:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.815+0000 7f76e3fff640 1 -- 172.21.15.47:0/293214129 learned_addr learned my addr 172.21.15.47:0/293214129 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.815+0000 7f76e37fe640 1 -- 172.21.15.47:0/293214129 >> v2:172.21.15.175:3300/0 conn(0x7f76e406bf10 msgr2=0x7f76e40ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.815+0000 7f76e37fe640 1 --2- 172.21.15.47:0/293214129 >> v2:172.21.15.175:3300/0 conn(0x7f76e406bf10 0x7f76e40ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.815+0000 7f76e37fe640 1 -- 172.21.15.47:0/293214129 >> v1:172.21.15.175:6789/0 conn(0x7f76e4161400 legacy=0x7f76e406b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.815+0000 7f76e37fe640 1 -- 172.21.15.47:0/293214129 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e40b0980 con 0x7f76e4161020 2024-03-23T09:52:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.816+0000 7f76e37fe640 1 --2- 172.21.15.47:0/293214129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76e4161020 0x7f76e40b1180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f76d40097a0 tx=0x7f76d402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccd834545949c089 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.816+0000 7f76e27fc640 1 -- 172.21.15.47:0/293214129 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4030610 con 0x7f76e4161020 2024-03-23T09:52:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.816+0000 7f76e27fc640 1 -- 172.21.15.47:0/293214129 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d4030770 con 0x7f76e4161020 2024-03-23T09:52:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.816+0000 7f76e27fc640 1 -- 172.21.15.47:0/293214129 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4030b00 con 0x7f76e4161020 2024-03-23T09:52:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.816+0000 7f76ea1fb640 1 -- 172.21.15.47:0/293214129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76e4161020 msgr2=0x7f76e40b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.816+0000 7f76ea1fb640 1 --2- 172.21.15.47:0/293214129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76e4161020 0x7f76e40b1180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f76d40097a0 tx=0x7f76d402b960 comp rx=0 tx=0).stop 2024-03-23T09:52:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.816+0000 7f76ea1fb640 1 -- 172.21.15.47:0/293214129 shutdown_connections 2024-03-23T09:52:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.816+0000 7f76ea1fb640 1 --2- 172.21.15.47:0/293214129 >> v2:172.21.15.175:3300/0 conn(0x7f76e406bf10 0x7f76e40ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.816+0000 7f76ea1fb640 1 --2- 172.21.15.47:0/293214129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76e4161020 0x7f76e40b1180 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.816+0000 7f76ea1fb640 1 -- 172.21.15.47:0/293214129 >> 172.21.15.47:0/293214129 conn(0x7f76e40d0670 msgr2=0x7f76e40d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.816+0000 7f76ea1fb640 1 -- 172.21.15.47:0/293214129 shutdown_connections 2024-03-23T09:52:16.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.816+0000 7f76ea1fb640 1 -- 172.21.15.47:0/293214129 wait complete. 2024-03-23T09:52:16.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.817+0000 7f76ea1fb640 1 Processor -- start 2024-03-23T09:52:16.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.818+0000 7f76ea1fb640 1 -- start start 2024-03-23T09:52:16.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.818+0000 7f76ea1fb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76e406bf10 0x7f76e40b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.818+0000 7f76ea1fb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76e4161020 0x7f76e40cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.818+0000 7f76ea1fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76e4161400 0x7f76e409d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.818+0000 7f76ea1fb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e40c8710 con 0x7f76e406bf10 2024-03-23T09:52:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.818+0000 7f76ea1fb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e40c7b90 con 0x7f76e4161020 2024-03-23T09:52:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.818+0000 7f76ea1fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e40c7e70 con 0x7f76e4161400 2024-03-23T09:52:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.818+0000 7f76e37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76e406bf10 0x7f76e40b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76e37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76e406bf10 0x7f76e40b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35384/0 (socket says 172.21.15.47:35384) 2024-03-23T09:52:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76e37fe640 1 -- 172.21.15.47:0/1271980008 learned_addr learned my addr 172.21.15.47:0/1271980008 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76e3fff640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76e4161400 0x7f76e409d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76e2ffd640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76e4161020 0x7f76e40cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76e37fe640 1 -- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76e4161400 msgr2=0x7f76e409d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76e37fe640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76e4161400 0x7f76e409d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76e37fe640 1 -- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76e4161020 msgr2=0x7f76e40cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76e37fe640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76e4161020 0x7f76e40cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76e37fe640 1 -- 172.21.15.47:0/1271980008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d4009400 con 0x7f76e406bf10 2024-03-23T09:52:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76e37fe640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76e406bf10 0x7f76e40b42d0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f76d402bf70 tx=0x7f76d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76cbfff640 1 -- 172.21.15.47:0/1271980008 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d403e040 con 0x7f76e406bf10 2024-03-23T09:52:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76cbfff640 1 -- 172.21.15.47:0/1271980008 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d4039cb0 con 0x7f76e406bf10 2024-03-23T09:52:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76ea1fb640 1 -- 172.21.15.47:0/1271980008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e40c8150 con 0x7f76e406bf10 2024-03-23T09:52:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76cbfff640 1 -- 172.21.15.47:0/1271980008 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4044630 con 0x7f76e406bf10 2024-03-23T09:52:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.819+0000 7f76ea1fb640 1 -- 172.21.15.47:0/1271980008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e40b5e50 con 0x7f76e406bf10 2024-03-23T09:52:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.820+0000 7f76ea1fb640 1 -- 172.21.15.47:0/1271980008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e40672f0 con 0x7f76e406bf10 2024-03-23T09:52:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.821+0000 7f76cbfff640 1 -- 172.21.15.47:0/1271980008 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f76d4043390 con 0x7f76e406bf10 2024-03-23T09:52:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.821+0000 7f76cbfff640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f76d006ac00 0x7f76d006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.821+0000 7f76cbfff640 1 -- 172.21.15.47:0/1271980008 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f76d40ab590 con 0x7f76e406bf10 2024-03-23T09:52:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.821+0000 7f76e2ffd640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f76d006ac00 0x7f76d006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.822+0000 7f76e2ffd640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f76d006ac00 0x7f76d006d0c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f76e40c7570 tx=0x7f76d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:16.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:16.825+0000 7f76cbfff640 1 -- 172.21.15.47:0/1271980008 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f76d4076710 con 0x7f76e406bf10 2024-03-23T09:52:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.008+0000 7f76ea1fb640 1 -- 172.21.15.47:0/1271980008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76e40ca950 con 0x7f76e406bf10 2024-03-23T09:52:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.008+0000 7f76cbfff640 1 -- 172.21.15.47:0/1271980008 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76d4079d30 con 0x7f76e406bf10 2024-03-23T09:52:17.009 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:52:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.010+0000 7f76ea1fb640 1 -- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f76d006ac00 msgr2=0x7f76d006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.010+0000 7f76ea1fb640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f76d006ac00 0x7f76d006d0c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f76e40c7570 tx=0x7f76d8009040 comp rx=0 tx=0).stop 2024-03-23T09:52:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.010+0000 7f76ea1fb640 1 -- 172.21.15.47:0/1271980008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76e406bf10 msgr2=0x7f76e40b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.010+0000 7f76ea1fb640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76e406bf10 0x7f76e40b42d0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f76d402bf70 tx=0x7f76d4004730 comp rx=0 tx=0).stop 2024-03-23T09:52:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.010+0000 7f76ea1fb640 1 -- 172.21.15.47:0/1271980008 shutdown_connections 2024-03-23T09:52:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.010+0000 7f76ea1fb640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f76d006ac00 0x7f76d006d0c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.010+0000 7f76ea1fb640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76e4161400 0x7f76e409d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.010+0000 7f76ea1fb640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76e4161020 0x7f76e40cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.010+0000 7f76ea1fb640 1 --2- 172.21.15.47:0/1271980008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76e406bf10 0x7f76e40b42d0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.010+0000 7f76ea1fb640 1 -- 172.21.15.47:0/1271980008 >> 172.21.15.47:0/1271980008 conn(0x7f76e40d0670 msgr2=0x7f76e406e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.010+0000 7f76ea1fb640 1 -- 172.21.15.47:0/1271980008 shutdown_connections 2024-03-23T09:52:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.010+0000 7f76ea1fb640 1 -- 172.21.15.47:0/1271980008 wait complete. 2024-03-23T09:52:17.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:52:17.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.125+0000 7f24fdc19640 1 Processor -- start 2024-03-23T09:52:17.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.126+0000 7f24fdc19640 1 -- start start 2024-03-23T09:52:17.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.126+0000 7f24fdc19640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f24f80d23a0 0x7f24f80d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.126+0000 7f24fdc19640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f80d10a0 con 0x7f24f815cee0 2024-03-23T09:52:17.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.126+0000 7f24fdc19640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f80d1300 con 0x7f24f80b8300 2024-03-23T09:52:17.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.126+0000 7f24fdc19640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f80d15e0 con 0x7f24f80d23a0 2024-03-23T09:52:17.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.126+0000 7f24f6ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f24f80d23a0 0x7f24f80d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.126+0000 7f24f77fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f24f815cee0 0x7f24f80d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39922/0 (socket says 172.21.15.47:39922) 2024-03-23T09:52:17.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.126+0000 7f24f77fe640 1 -- 172.21.15.47:0/1291356467 learned_addr learned my addr 172.21.15.47:0/1291356467 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:17.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.127+0000 7f24f6ffd640 1 -- 172.21.15.47:0/1291356467 >> v1:172.21.15.175:6789/0 conn(0x7f24f80b8300 legacy=0x7f24f80b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.127+0000 7f24f6ffd640 1 -- 172.21.15.47:0/1291356467 >> v1:172.21.15.47:6789/0 conn(0x7f24f815cee0 legacy=0x7f24f80d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.127+0000 7f24f6ffd640 1 -- 172.21.15.47:0/1291356467 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f80d18c0 con 0x7f24f80d23a0 2024-03-23T09:52:17.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.127+0000 7f24f6ffd640 1 --2- 172.21.15.47:0/1291356467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24f80d23a0 0x7f24f80d2870 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0097a0 tx=0x7f24ec02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1798914fe75fc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:17.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.128+0000 7f24f67fc640 1 -- 172.21.15.47:0/1291356467 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec030710 con 0x7f24f80d23a0 2024-03-23T09:52:17.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.128+0000 7f24f67fc640 1 -- 172.21.15.47:0/1291356467 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec02e050 con 0x7f24f80d23a0 2024-03-23T09:52:17.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.128+0000 7f24f67fc640 1 -- 172.21.15.47:0/1291356467 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec030a90 con 0x7f24f80d23a0 2024-03-23T09:52:17.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.128+0000 7f24fdc19640 1 -- 172.21.15.47:0/1291356467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24f80d23a0 msgr2=0x7f24f80d2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.128+0000 7f24fdc19640 1 --2- 172.21.15.47:0/1291356467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24f80d23a0 0x7f24f80d2870 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0097a0 tx=0x7f24ec02b790 comp rx=0 tx=0).stop 2024-03-23T09:52:17.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.128+0000 7f24fdc19640 1 -- 172.21.15.47:0/1291356467 shutdown_connections 2024-03-23T09:52:17.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.128+0000 7f24fdc19640 1 --2- 172.21.15.47:0/1291356467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24f80d23a0 0x7f24f80d2870 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.128+0000 7f24fdc19640 1 -- 172.21.15.47:0/1291356467 >> 172.21.15.47:0/1291356467 conn(0x7f24f8065df0 msgr2=0x7f24f8066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:17.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.128+0000 7f24fdc19640 1 -- 172.21.15.47:0/1291356467 shutdown_connections 2024-03-23T09:52:17.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.128+0000 7f24fdc19640 1 -- 172.21.15.47:0/1291356467 wait complete. 2024-03-23T09:52:17.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.129+0000 7f24fdc19640 1 Processor -- start 2024-03-23T09:52:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.130+0000 7f24fdc19640 1 -- start start 2024-03-23T09:52:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.130+0000 7f24fdc19640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24f80b8300 0x7f24f81f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.130+0000 7f24fdc19640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24f80d23a0 0x7f24f80cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.130+0000 7f24fdc19640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24f815cee0 0x7f24f80cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.130+0000 7f24fdc19640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f81f3580 con 0x7f24f80b8300 2024-03-23T09:52:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.130+0000 7f24fdc19640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80c9ce0 con 0x7f24f815cee0 2024-03-23T09:52:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.130+0000 7f24fdc19640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80c9fc0 con 0x7f24f80d23a0 2024-03-23T09:52:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.130+0000 7f24f77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24f80b8300 0x7f24f81f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.130+0000 7f24f77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24f80b8300 0x7f24f81f3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35406/0 (socket says 172.21.15.47:35406) 2024-03-23T09:52:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.130+0000 7f24f77fe640 1 -- 172.21.15.47:0/2947194952 learned_addr learned my addr 172.21.15.47:0/2947194952 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.130+0000 7f24f7fff640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24f815cee0 0x7f24f80cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.130+0000 7f24f6ffd640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24f80d23a0 0x7f24f80cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.131+0000 7f24f77fe640 1 -- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24f80d23a0 msgr2=0x7f24f80cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.131+0000 7f24f77fe640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24f80d23a0 0x7f24f80cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.131+0000 7f24f77fe640 1 -- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24f815cee0 msgr2=0x7f24f80cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.131+0000 7f24f77fe640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24f815cee0 0x7f24f80cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.131+0000 7f24f77fe640 1 -- 172.21.15.47:0/2947194952 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec009400 con 0x7f24f80b8300 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.131+0000 7f24f7fff640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24f815cee0 0x7f24f80cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.131+0000 7f24f6ffd640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24f80d23a0 0x7f24f80cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.131+0000 7f24f77fe640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24f80b8300 0x7f24f81f3040 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f24e800c5f0 tx=0x7f24e800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.131+0000 7f24dffff640 1 -- 172.21.15.47:0/2947194952 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e80074e0 con 0x7f24f80b8300 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.131+0000 7f24dffff640 1 -- 172.21.15.47:0/2947194952 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e8007640 con 0x7f24f80b8300 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.131+0000 7f24fdc19640 1 -- 172.21.15.47:0/2947194952 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f80ca2a0 con 0x7f24f80b8300 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.131+0000 7f24dffff640 1 -- 172.21.15.47:0/2947194952 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8004a00 con 0x7f24f80b8300 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.131+0000 7f24fdc19640 1 -- 172.21.15.47:0/2947194952 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f80ca860 con 0x7f24f80b8300 2024-03-23T09:52:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.132+0000 7f24fdc19640 1 -- 172.21.15.47:0/2947194952 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f8072360 con 0x7f24f80b8300 2024-03-23T09:52:17.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.132+0000 7f24dffff640 1 -- 172.21.15.47:0/2947194952 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f24e8004b60 con 0x7f24f80b8300 2024-03-23T09:52:17.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.133+0000 7f24dffff640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f24e406ac30 0x7f24e406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.133+0000 7f24dffff640 1 -- 172.21.15.47:0/2947194952 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f24e8088750 con 0x7f24f80b8300 2024-03-23T09:52:17.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.133+0000 7f24f6ffd640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f24e406ac30 0x7f24e406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.134+0000 7f24f6ffd640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f24e406ac30 0x7f24e406d0f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f24ec034000 tx=0x7f24ec02f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:17.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.137+0000 7f24dffff640 1 -- 172.21.15.47:0/2947194952 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f24e8053820 con 0x7f24f80b8300 2024-03-23T09:52:17.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.316+0000 7f24fdc19640 1 -- 172.21.15.47:0/2947194952 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f24f80caa80 con 0x7f24f80b8300 2024-03-23T09:52:17.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.316+0000 7f24dffff640 1 -- 172.21.15.47:0/2947194952 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7f24e8056e40 con 0x7f24f80b8300 2024-03-23T09:52:17.316 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:52:17.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.318+0000 7f24fdc19640 1 -- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f24e406ac30 msgr2=0x7f24e406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.318+0000 7f24fdc19640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f24e406ac30 0x7f24e406d0f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f24ec034000 tx=0x7f24ec02f000 comp rx=0 tx=0).stop 2024-03-23T09:52:17.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.318+0000 7f24fdc19640 1 -- 172.21.15.47:0/2947194952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24f80b8300 msgr2=0x7f24f81f3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.318+0000 7f24fdc19640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24f80b8300 0x7f24f81f3040 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f24e800c5f0 tx=0x7f24e800cab0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.318+0000 7f24fdc19640 1 -- 172.21.15.47:0/2947194952 shutdown_connections 2024-03-23T09:52:17.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.318+0000 7f24fdc19640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f24e406ac30 0x7f24e406d0f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.318+0000 7f24fdc19640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24f815cee0 0x7f24f80cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.318+0000 7f24fdc19640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24f80d23a0 0x7f24f80cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.318+0000 7f24fdc19640 1 --2- 172.21.15.47:0/2947194952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24f80b8300 0x7f24f81f3040 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.318+0000 7f24fdc19640 1 -- 172.21.15.47:0/2947194952 >> 172.21.15.47:0/2947194952 conn(0x7f24f8065df0 msgr2=0x7f24f80661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:17.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.318+0000 7f24fdc19640 1 -- 172.21.15.47:0/2947194952 shutdown_connections 2024-03-23T09:52:17.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.318+0000 7f24fdc19640 1 -- 172.21.15.47:0/2947194952 wait complete. 2024-03-23T09:52:17.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:52:17.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.432+0000 7fd5c47fb640 1 Processor -- start 2024-03-23T09:52:17.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.432+0000 7fd5c47fb640 1 -- start start 2024-03-23T09:52:17.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.433+0000 7fd5c47fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc0cec50 0x7fd5bc0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.433+0000 7fd5c47fb640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd5bc0b05f0 0x7fd5bc0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.433+0000 7fd5c47fb640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc0cb210 con 0x7fd5bc15ac40 2024-03-23T09:52:17.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.433+0000 7fd5c47fb640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5bc0cbac0 con 0x7fd5bc0b05f0 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.433+0000 7fd5c47fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0cf610 con 0x7fd5bc0cec50 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c2597640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd5bc15ac40 0x7fd5bc0cbee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39934/0 (socket says 172.21.15.47:39934) 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c2597640 1 -- 172.21.15.47:0/2708013054 learned_addr learned my addr 172.21.15.47:0/2708013054 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1d96640 1 --2- 172.21.15.47:0/2708013054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc0cec50 0x7fd5bc0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c2d98640 1 --2- 172.21.15.47:0/2708013054 >> v2:172.21.15.175:3300/0 conn(0x7fd5bc0b05f0 0x7fd5bc0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783156828 0 0) 0x7fd5bc0cb210 con 0x7fd5bc15ac40 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5a8003520 con 0x7fd5bc15ac40 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1759669626 0 0) 0x7fd5a8003520 con 0x7fd5bc15ac40 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc0cec50 msgr2=0x7fd5bc0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 --2- 172.21.15.47:0/2708013054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc0cec50 0x7fd5bc0cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 >> v2:172.21.15.175:3300/0 conn(0x7fd5bc0b05f0 msgr2=0x7fd5bc0b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 --2- 172.21.15.47:0/2708013054 >> v2:172.21.15.175:3300/0 conn(0x7fd5bc0b05f0 0x7fd5bc0b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc0cc5f0 con 0x7fd5bc15ac40 2024-03-23T09:52:17.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fd5b8001670 con 0x7fd5bc15ac40 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c2d98640 1 --2- 172.21.15.47:0/2708013054 >> v2:172.21.15.175:3300/0 conn(0x7fd5bc0b05f0 0x7fd5bc0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1d96640 1 --2- 172.21.15.47:0/2708013054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc0cec50 0x7fd5bc0cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 >> v1:172.21.15.47:6789/0 conn(0x7fd5bc15ac40 legacy=0x7fd5bc0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 --2- 172.21.15.47:0/2708013054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc0cec50 0x7fd5a8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 --2- 172.21.15.47:0/2708013054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a8003eb0 0x7fd5a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 --2- 172.21.15.47:0/2708013054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5a8004d00 0x7fd5a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0cbac0 con 0x7fd5a8004d00 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0cf610 con 0x7fd5a8003eb0 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a800b980 con 0x7fd5bc0cec50 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c2d98640 1 --2- 172.21.15.47:0/2708013054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5a8004d00 0x7fd5a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c2597640 1 --2- 172.21.15.47:0/2708013054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc0cec50 0x7fd5a8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.434+0000 7fd5c1d96640 1 --2- 172.21.15.47:0/2708013054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a8003eb0 0x7fd5a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.435+0000 7fd5c2d98640 1 -- 172.21.15.47:0/2708013054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc0cec50 msgr2=0x7fd5a8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.435+0000 7fd5c2d98640 1 --2- 172.21.15.47:0/2708013054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc0cec50 0x7fd5a8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.435+0000 7fd5c2d98640 1 -- 172.21.15.47:0/2708013054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a8003eb0 msgr2=0x7fd5a80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.435+0000 7fd5c2d98640 1 --2- 172.21.15.47:0/2708013054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a8003eb0 0x7fd5a80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.435+0000 7fd5c2d98640 1 -- 172.21.15.47:0/2708013054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac009400 con 0x7fd5a8004d00 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.435+0000 7fd5c2d98640 1 --2- 172.21.15.47:0/2708013054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5a8004d00 0x7fd5a8005180 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400e350 tx=0x7fd5a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdf84199939638 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.435+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b402ce00 con 0x7fd5a8004d00 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.435+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b400f040 con 0x7fd5a8004d00 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.435+0000 7fd5c1595640 1 -- 172.21.15.47:0/2708013054 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4034640 con 0x7fd5a8004d00 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.436+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2708013054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5a8004d00 msgr2=0x7fd5a8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.436+0000 7fd5c47fb640 1 --2- 172.21.15.47:0/2708013054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5a8004d00 0x7fd5a8005180 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400e350 tx=0x7fd5a80057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.436+0000 7fd5c2d98640 1 -- 172.21.15.47:0/2708013054 reap_dead start 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.436+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2708013054 shutdown_connections 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.436+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2708013054 >> 172.21.15.47:0/2708013054 conn(0x7fd5bc0d0670 msgr2=0x7fd5bc0d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.436+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2708013054 shutdown_connections 2024-03-23T09:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.436+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2708013054 wait complete. 2024-03-23T09:52:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.437+0000 7fd5c47fb640 1 Processor -- start 2024-03-23T09:52:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.437+0000 7fd5c47fb640 1 -- start start 2024-03-23T09:52:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c47fb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a8004d00 0x7fd5bc0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c47fb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc0af410 0x7fd5bc0c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c47fb640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc0c6580 0x7fd5bc0c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c47fb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0c8180 con 0x7fd5bc0af410 2024-03-23T09:52:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c47fb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0c4980 con 0x7fd5a8004d00 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c47fb640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0af870 con 0x7fd5bc0c6580 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c1d96640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc0af410 0x7fd5bc0c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c1d96640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc0af410 0x7fd5bc0c4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35432/0 (socket says 172.21.15.47:35432) 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c1d96640 1 -- 172.21.15.47:0/2711122045 learned_addr learned my addr 172.21.15.47:0/2711122045 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c1d96640 1 -- 172.21.15.47:0/2711122045 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc0c6580 msgr2=0x7fd5bc0c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c1d96640 1 --2- 172.21.15.47:0/2711122045 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc0c6580 0x7fd5bc0c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c1d96640 1 -- 172.21.15.47:0/2711122045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a8004d00 msgr2=0x7fd5bc0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c1d96640 1 --2- 172.21.15.47:0/2711122045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a8004d00 0x7fd5bc0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.438+0000 7fd5c1d96640 1 -- 172.21.15.47:0/2711122045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc0c4c20 con 0x7fd5bc0af410 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.439+0000 7fd5c1d96640 1 --2- 172.21.15.47:0/2711122045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc0af410 0x7fd5bc0c4440 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac011a90 tx=0x7fd5ac011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.439+0000 7fd5b2ffd640 1 -- 172.21.15.47:0/2711122045 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac00fb90 con 0x7fd5bc0af410 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.439+0000 7fd5b2ffd640 1 -- 172.21.15.47:0/2711122045 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ac00fcf0 con 0x7fd5bc0af410 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.439+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2711122045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc0c4f00 con 0x7fd5bc0af410 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.439+0000 7fd5b2ffd640 1 -- 172.21.15.47:0/2711122045 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac004260 con 0x7fd5bc0af410 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.439+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2711122045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc0c86b0 con 0x7fd5bc0af410 2024-03-23T09:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.440+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2711122045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5bc0c8870 con 0x7fd5bc0af410 2024-03-23T09:52:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.440+0000 7fd5b2ffd640 1 -- 172.21.15.47:0/2711122045 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd5ac003610 con 0x7fd5bc0af410 2024-03-23T09:52:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.441+0000 7fd5b2ffd640 1 --2- 172.21.15.47:0/2711122045 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5a006ac00 0x7fd5a006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.441+0000 7fd5b2ffd640 1 -- 172.21.15.47:0/2711122045 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd5ac089b50 con 0x7fd5bc0af410 2024-03-23T09:52:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.441+0000 7fd5c2597640 1 --2- 172.21.15.47:0/2711122045 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5a006ac00 0x7fd5a006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.441+0000 7fd5c2597640 1 --2- 172.21.15.47:0/2711122045 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5a006ac00 0x7fd5a006d0c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8007dd0 tx=0x7fd5b80014b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.445+0000 7fd5b2ffd640 1 -- 172.21.15.47:0/2711122045 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd5ac054b70 con 0x7fd5bc0af410 2024-03-23T09:52:17.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.622+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2711122045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5bc0c5140 con 0x7fd5bc0af410 2024-03-23T09:52:17.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.622+0000 7fd5b2ffd640 1 -- 172.21.15.47:0/2711122045 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5ac022070 con 0x7fd5bc0af410 2024-03-23T09:52:17.623 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:52:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.624+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2711122045 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5a006ac00 msgr2=0x7fd5a006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.624+0000 7fd5c47fb640 1 --2- 172.21.15.47:0/2711122045 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5a006ac00 0x7fd5a006d0c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8007dd0 tx=0x7fd5b80014b0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.624+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2711122045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc0af410 msgr2=0x7fd5bc0c4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.624+0000 7fd5c47fb640 1 --2- 172.21.15.47:0/2711122045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc0af410 0x7fd5bc0c4440 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac011a90 tx=0x7fd5ac011f50 comp rx=0 tx=0).stop 2024-03-23T09:52:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.624+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2711122045 shutdown_connections 2024-03-23T09:52:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.624+0000 7fd5c47fb640 1 --2- 172.21.15.47:0/2711122045 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd5a006ac00 0x7fd5a006d0c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.624+0000 7fd5c47fb640 1 --2- 172.21.15.47:0/2711122045 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc0c6580 0x7fd5bc0c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.624+0000 7fd5c47fb640 1 --2- 172.21.15.47:0/2711122045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc0af410 0x7fd5bc0c4440 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.624+0000 7fd5c47fb640 1 --2- 172.21.15.47:0/2711122045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a8004d00 0x7fd5bc0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.624+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2711122045 >> 172.21.15.47:0/2711122045 conn(0x7fd5bc0d0670 msgr2=0x7fd5bc0cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:17.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.624+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2711122045 shutdown_connections 2024-03-23T09:52:17.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.624+0000 7fd5c47fb640 1 -- 172.21.15.47:0/2711122045 wait complete. 2024-03-23T09:52:17.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:52:17.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.737+0000 7f7a7e73c640 1 Processor -- start 2024-03-23T09:52:17.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.737+0000 7f7a7e73c640 1 -- start start 2024-03-23T09:52:17.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a7e73c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a780cf210 0x7f7a780cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a7e73c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a780b5a90 0x7f7a780b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a7e73c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780cfbd0 con 0x7f7a7815cd00 2024-03-23T09:52:17.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a7e73c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a7815d0e0 con 0x7f7a780b5a90 2024-03-23T09:52:17.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a7e73c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780d1540 con 0x7f7a780cf210 2024-03-23T09:52:17.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a7ccd9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a780b5a90 0x7f7a780b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a7ccd9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a780b5a90 0x7f7a780b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35448/0 (socket says 172.21.15.47:35448) 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a7ccd9640 1 -- 172.21.15.47:0/2693513497 learned_addr learned my addr 172.21.15.47:0/2693513497 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a777fe640 1 --2- 172.21.15.47:0/2693513497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a780cf210 0x7f7a780cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a7ccd9640 1 -- 172.21.15.47:0/2693513497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a780cf210 msgr2=0x7f7a780cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a7ccd9640 1 --2- 172.21.15.47:0/2693513497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a780cf210 0x7f7a780cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a7ccd9640 1 -- 172.21.15.47:0/2693513497 >> v1:172.21.15.175:6789/0 conn(0x7f7a7815cd00 legacy=0x7f7a780d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a7ccd9640 1 -- 172.21.15.47:0/2693513497 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a780d1820 con 0x7f7a780b5a90 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.738+0000 7f7a777fe640 1 --2- 172.21.15.47:0/2693513497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a780cf210 0x7f7a780cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.739+0000 7f7a7ccd9640 1 --2- 172.21.15.47:0/2693513497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a780b5a90 0x7f7a780b5f20 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009e50 tx=0x7f7a7002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da89d2f432cd88d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.739+0000 7f7a76ffd640 1 -- 172.21.15.47:0/2693513497 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7002c9e0 con 0x7f7a780b5a90 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.739+0000 7f7a76ffd640 1 -- 172.21.15.47:0/2693513497 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7002cb40 con 0x7f7a780b5a90 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.739+0000 7f7a76ffd640 1 -- 172.21.15.47:0/2693513497 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a700343f0 con 0x7f7a780b5a90 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.740+0000 7f7a7e73c640 1 -- 172.21.15.47:0/2693513497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a780b5a90 msgr2=0x7f7a780b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.740+0000 7f7a7e73c640 1 --2- 172.21.15.47:0/2693513497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a780b5a90 0x7f7a780b5f20 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009e50 tx=0x7f7a7002be40 comp rx=0 tx=0).stop 2024-03-23T09:52:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.740+0000 7f7a7e73c640 1 -- 172.21.15.47:0/2693513497 shutdown_connections 2024-03-23T09:52:17.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.740+0000 7f7a7e73c640 1 --2- 172.21.15.47:0/2693513497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a780b5a90 0x7f7a780b5f20 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.740+0000 7f7a7e73c640 1 --2- 172.21.15.47:0/2693513497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a780cf210 0x7f7a780cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.740+0000 7f7a7e73c640 1 -- 172.21.15.47:0/2693513497 >> 172.21.15.47:0/2693513497 conn(0x7f7a7806b160 msgr2=0x7f7a7806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:17.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.740+0000 7f7a7e73c640 1 -- 172.21.15.47:0/2693513497 shutdown_connections 2024-03-23T09:52:17.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.740+0000 7f7a7e73c640 1 -- 172.21.15.47:0/2693513497 wait complete. 2024-03-23T09:52:17.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.741+0000 7f7a7e73c640 1 Processor -- start 2024-03-23T09:52:17.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.741+0000 7f7a7e73c640 1 -- start start 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a7e73c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a780b5a90 0x7f7a781f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a7e73c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a780cf210 0x7f7a780c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a7e73c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7815cd00 0x7f7a780c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a7e73c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780c8fd0 con 0x7f7a780b5a90 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a7e73c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780c7f30 con 0x7f7a7815cd00 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a7e73c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780c8210 con 0x7f7a780cf210 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a77fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a780b5a90 0x7f7a781f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a77fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a780b5a90 0x7f7a781f1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35460/0 (socket says 172.21.15.47:35460) 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a77fff640 1 -- 172.21.15.47:0/3013511914 learned_addr learned my addr 172.21.15.47:0/3013511914 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a777fe640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a780cf210 0x7f7a780c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a7ccd9640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7815cd00 0x7f7a780c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a77fff640 1 -- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a780cf210 msgr2=0x7f7a780c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a77fff640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a780cf210 0x7f7a780c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a77fff640 1 -- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7815cd00 msgr2=0x7f7a780c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a77fff640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7815cd00 0x7f7a780c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a77fff640 1 -- 172.21.15.47:0/3013511914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a70009b30 con 0x7f7a780b5a90 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a777fe640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a780cf210 0x7f7a780c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a7ccd9640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7815cd00 0x7f7a780c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a77fff640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a780b5a90 0x7f7a781f1be0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00b3f0 tx=0x7f7a6c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a74ff9640 1 -- 172.21.15.47:0/3013511914 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c011830 con 0x7f7a780b5a90 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a74ff9640 1 -- 172.21.15.47:0/3013511914 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6c011990 con 0x7f7a780b5a90 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a7e73c640 1 -- 172.21.15.47:0/3013511914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a780c84f0 con 0x7f7a780b5a90 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.742+0000 7f7a74ff9640 1 -- 172.21.15.47:0/3013511914 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c011d40 con 0x7f7a780b5a90 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.743+0000 7f7a7e73c640 1 -- 172.21.15.47:0/3013511914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a780c8ab0 con 0x7f7a780b5a90 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.743+0000 7f7a7e73c640 1 -- 172.21.15.47:0/3013511914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a78072360 con 0x7f7a780b5a90 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.744+0000 7f7a74ff9640 1 -- 172.21.15.47:0/3013511914 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7a6c01a400 con 0x7f7a780b5a90 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.744+0000 7f7a74ff9640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a6406ac00 0x7f7a6406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.744+0000 7f7a74ff9640 1 -- 172.21.15.47:0/3013511914 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7a6c0891a0 con 0x7f7a780b5a90 2024-03-23T09:52:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.744+0000 7f7a777fe640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a6406ac00 0x7f7a6406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.745+0000 7f7a777fe640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a6406ac00 0x7f7a6406d0c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7a78152cc0 tx=0x7f7a68008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:17.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.748+0000 7f7a74ff9640 1 -- 172.21.15.47:0/3013511914 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7a6c054380 con 0x7f7a780b5a90 2024-03-23T09:52:17.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.940+0000 7f7a7e73c640 1 -- 172.21.15.47:0/3013511914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7a78066be0 con 0x7f7a780b5a90 2024-03-23T09:52:17.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.940+0000 7f7a74ff9640 1 -- 172.21.15.47:0/3013511914 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v117) v1 ==== 80+0+120 (secure 0 0 0) 0x7f7a6c0579a0 con 0x7f7a780b5a90 2024-03-23T09:52:17.941 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6834/724782441 2024-03-24T09:52:05.460377+0000 2024-03-23T09:52:17.941 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6832/724782441 2024-03-24T09:52:05.460377+0000 2024-03-23T09:52:17.941 INFO:teuthology.orchestra.run.smithi047.stderr:listed 2 entries 2024-03-23T09:52:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.942+0000 7f7a7e73c640 1 -- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a6406ac00 msgr2=0x7f7a6406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.942+0000 7f7a7e73c640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a6406ac00 0x7f7a6406d0c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7a78152cc0 tx=0x7f7a68008040 comp rx=0 tx=0).stop 2024-03-23T09:52:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.942+0000 7f7a7e73c640 1 -- 172.21.15.47:0/3013511914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a780b5a90 msgr2=0x7f7a781f1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.942+0000 7f7a7e73c640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a780b5a90 0x7f7a781f1be0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00b3f0 tx=0x7f7a6c00b8b0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.942+0000 7f7a7e73c640 1 -- 172.21.15.47:0/3013511914 shutdown_connections 2024-03-23T09:52:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.942+0000 7f7a7e73c640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7a6406ac00 0x7f7a6406d0c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.942+0000 7f7a7e73c640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7815cd00 0x7f7a780c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.942+0000 7f7a7e73c640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a780cf210 0x7f7a780c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.942+0000 7f7a7e73c640 1 --2- 172.21.15.47:0/3013511914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a780b5a90 0x7f7a781f1be0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.942+0000 7f7a7e73c640 1 -- 172.21.15.47:0/3013511914 >> 172.21.15.47:0/3013511914 conn(0x7f7a7806b160 msgr2=0x7f7a780d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.942+0000 7f7a7e73c640 1 -- 172.21.15.47:0/3013511914 shutdown_connections 2024-03-23T09:52:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:17.942+0000 7f7a7e73c640 1 -- 172.21.15.47:0/3013511914 wait complete. 2024-03-23T09:52:17.953 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:52:17.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:17.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:17.978 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[81556]: fuse finished with error 0 and tester_r 0 2024-03-23T09:52:17.980 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:52:24.082 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:52:24.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:24.082 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:52:24.101 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:52:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.253+0000 7f935c163640 1 Processor -- start 2024-03-23T09:52:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.254+0000 7f935c163640 1 -- start start 2024-03-23T09:52:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.254+0000 7f935c163640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f93540b8300 0x7f93540cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.254+0000 7f935c163640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f93540cf6d0 0x7f93540d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.255+0000 7f935c163640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93540d0ee0 con 0x7f935415cb40 2024-03-23T09:52:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.255+0000 7f935c163640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93540b5a10 con 0x7f93540cf6d0 2024-03-23T09:52:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.255+0000 7f935c163640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93540cfb60 con 0x7f93540b8300 2024-03-23T09:52:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.255+0000 7f935a700640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f93540cf6d0 0x7f93540d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.255+0000 7f935a700640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f93540cf6d0 0x7f93540d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35464/0 (socket says 172.21.15.47:35464) 2024-03-23T09:52:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.255+0000 7f935a700640 1 -- 172.21.15.47:0/3237486960 learned_addr learned my addr 172.21.15.47:0/3237486960 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.255+0000 7f93596fe640 1 --2- 172.21.15.47:0/3237486960 >> v2:172.21.15.175:3300/0 conn(0x7f93540b8300 0x7f93540cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.255+0000 7f935a700640 1 -- 172.21.15.47:0/3237486960 >> v2:172.21.15.175:3300/0 conn(0x7f93540b8300 msgr2=0x7f93540cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.255+0000 7f935a700640 1 --2- 172.21.15.47:0/3237486960 >> v2:172.21.15.175:3300/0 conn(0x7f93540b8300 0x7f93540cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.255+0000 7f935a700640 1 -- 172.21.15.47:0/3237486960 >> v1:172.21.15.175:6789/0 conn(0x7f935415cb40 legacy=0x7f9354160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.255+0000 7f935a700640 1 -- 172.21.15.47:0/3237486960 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93541616f0 con 0x7f93540cf6d0 2024-03-23T09:52:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.255+0000 7f93596fe640 1 --2- 172.21.15.47:0/3237486960 >> v2:172.21.15.175:3300/0 conn(0x7f93540b8300 0x7f93540cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:24.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.256+0000 7f935a700640 1 --2- 172.21.15.47:0/3237486960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93540cf6d0 0x7f93540d1370 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f9344009e50 tx=0x7f934402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333cf9ab03636449 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.256+0000 7f9358efd640 1 -- 172.21.15.47:0/3237486960 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934402ca80 con 0x7f93540cf6d0 2024-03-23T09:52:24.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.256+0000 7f9358efd640 1 -- 172.21.15.47:0/3237486960 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934402cbe0 con 0x7f93540cf6d0 2024-03-23T09:52:24.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.256+0000 7f9358efd640 1 -- 172.21.15.47:0/3237486960 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440343f0 con 0x7f93540cf6d0 2024-03-23T09:52:24.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.257+0000 7f935c163640 1 -- 172.21.15.47:0/3237486960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93540cf6d0 msgr2=0x7f93540d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.257+0000 7f935c163640 1 --2- 172.21.15.47:0/3237486960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93540cf6d0 0x7f93540d1370 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f9344009e50 tx=0x7f934402bee0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.257+0000 7f935c163640 1 -- 172.21.15.47:0/3237486960 shutdown_connections 2024-03-23T09:52:24.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.257+0000 7f935c163640 1 --2- 172.21.15.47:0/3237486960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93540cf6d0 0x7f93540d1370 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.257+0000 7f935c163640 1 --2- 172.21.15.47:0/3237486960 >> v2:172.21.15.175:3300/0 conn(0x7f93540b8300 0x7f93540cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.257+0000 7f935c163640 1 -- 172.21.15.47:0/3237486960 >> 172.21.15.47:0/3237486960 conn(0x7f935406b160 msgr2=0x7f935406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:24.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.257+0000 7f935c163640 1 -- 172.21.15.47:0/3237486960 shutdown_connections 2024-03-23T09:52:24.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.257+0000 7f935c163640 1 -- 172.21.15.47:0/3237486960 wait complete. 2024-03-23T09:52:24.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.258+0000 7f935c163640 1 Processor -- start 2024-03-23T09:52:24.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.258+0000 7f935c163640 1 -- start start 2024-03-23T09:52:24.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935c163640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93540b8300 0x7f935415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935c163640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93540cf6d0 0x7f93540cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935c163640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f935415cb40 0x7f93540cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935c163640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93540c9ae0 con 0x7f935415cb40 2024-03-23T09:52:24.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935c163640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93540c9dc0 con 0x7f93540b8300 2024-03-23T09:52:24.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935c163640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93540ca0a0 con 0x7f93540cf6d0 2024-03-23T09:52:24.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935a700640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f935415cb40 0x7f93540cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935a700640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f935415cb40 0x7f93540cb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35476/0 (socket says 172.21.15.47:35476) 2024-03-23T09:52:24.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935a700640 1 -- 172.21.15.47:0/1053549820 learned_addr learned my addr 172.21.15.47:0/1053549820 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:24.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f9359eff640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93540b8300 0x7f935415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f93596fe640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93540cf6d0 0x7f93540cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935a700640 1 -- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93540cf6d0 msgr2=0x7f93540cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935a700640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93540cf6d0 0x7f93540cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935a700640 1 -- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93540b8300 msgr2=0x7f935415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935a700640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93540b8300 0x7f935415bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.259+0000 7f935a700640 1 -- 172.21.15.47:0/1053549820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344009b30 con 0x7f935415cb40 2024-03-23T09:52:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.260+0000 7f935a700640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f935415cb40 0x7f93540cb0c0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f9344036040 tx=0x7f9344004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.260+0000 7f93527fc640 1 -- 172.21.15.47:0/1053549820 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934403e450 con 0x7f935415cb40 2024-03-23T09:52:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.260+0000 7f93527fc640 1 -- 172.21.15.47:0/1053549820 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934403e5d0 con 0x7f935415cb40 2024-03-23T09:52:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.260+0000 7f93527fc640 1 -- 172.21.15.47:0/1053549820 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934403e980 con 0x7f935415cb40 2024-03-23T09:52:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.260+0000 7f935c163640 1 -- 172.21.15.47:0/1053549820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93540ca380 con 0x7f935415cb40 2024-03-23T09:52:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.260+0000 7f935c163640 1 -- 172.21.15.47:0/1053549820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935415c1c0 con 0x7f935415cb40 2024-03-23T09:52:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.261+0000 7f935c163640 1 -- 172.21.15.47:0/1053549820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9354072360 con 0x7f935415cb40 2024-03-23T09:52:24.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.261+0000 7f93527fc640 1 -- 172.21.15.47:0/1053549820 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f934402cd50 con 0x7f935415cb40 2024-03-23T09:52:24.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.262+0000 7f93527fc640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f934006ac00 0x7f934006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.262+0000 7f93527fc640 1 -- 172.21.15.47:0/1053549820 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f93440abe60 con 0x7f935415cb40 2024-03-23T09:52:24.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.262+0000 7f9359eff640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f934006ac00 0x7f934006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.263+0000 7f9359eff640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f934006ac00 0x7f934006d0c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f934c006fd0 tx=0x7f934c009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.266+0000 7f93527fc640 1 -- 172.21.15.47:0/1053549820 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9344077180 con 0x7f935415cb40 2024-03-23T09:52:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.482+0000 7f935c163640 1 -- 172.21.15.47:0/1053549820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93540ca8a0 con 0x7f935415cb40 2024-03-23T09:52:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.482+0000 7f93527fc640 1 -- 172.21.15.47:0/1053549820 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f93540ca8a0 con 0x7f935415cb40 2024-03-23T09:52:24.483 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:24.484 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":86,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:51:40.395539+0000","modified":"2024-03-23T09:52:09.483913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9570},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9570":{"gid":9570,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6834/1868958629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1868958629},{"type":"v1","addr":"172.21.15.47:6834","nonce":1868958629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-23T09:52:24.484 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 86 2024-03-23T09:52:24.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.484+0000 7f935c163640 1 -- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f934006ac00 msgr2=0x7f934006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.484+0000 7f935c163640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f934006ac00 0x7f934006d0c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f934c006fd0 tx=0x7f934c009060 comp rx=0 tx=0).stop 2024-03-23T09:52:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.484+0000 7f935c163640 1 -- 172.21.15.47:0/1053549820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f935415cb40 msgr2=0x7f93540cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.484+0000 7f935c163640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f935415cb40 0x7f93540cb0c0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f9344036040 tx=0x7f9344004a40 comp rx=0 tx=0).stop 2024-03-23T09:52:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.484+0000 7f935c163640 1 -- 172.21.15.47:0/1053549820 shutdown_connections 2024-03-23T09:52:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.484+0000 7f935c163640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f934006ac00 0x7f934006d0c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.484+0000 7f935c163640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f935415cb40 0x7f93540cb0c0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.484+0000 7f935c163640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93540cf6d0 0x7f93540cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.484+0000 7f935c163640 1 --2- 172.21.15.47:0/1053549820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93540b8300 0x7f935415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.484+0000 7f935c163640 1 -- 172.21.15.47:0/1053549820 >> 172.21.15.47:0/1053549820 conn(0x7f935406b160 msgr2=0x7f9354157310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.484+0000 7f935c163640 1 -- 172.21.15.47:0/1053549820 shutdown_connections 2024-03-23T09:52:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.484+0000 7f935c163640 1 -- 172.21.15.47:0/1053549820 wait complete. 2024-03-23T09:52:24.496 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-03-23T09:52:24.615 INFO:teuthology.orchestra.run.smithi047.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-03-23T09:52:24.616 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-03-23T09:52:24.616 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-03-23T09:52:24.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.655+0000 7f4f59a4af40 1 Processor -- start 2024-03-23T09:52:24.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.655+0000 7f4f59a4af40 1 -- start start 2024-03-23T09:52:24.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f59a4af40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c9cb8ec140 0x55c9cb8de150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f59a4af40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9cb8eca60 0x55c9cb846a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f59a4af40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c9cb8eed20 0x55c9cb97fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f59a4af40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb75fb60 con 0x55c9cb8ec140 2024-03-23T09:52:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f59a4af40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb75f800 con 0x55c9cb8eca60 2024-03-23T09:52:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f59a4af40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb742520 con 0x55c9cb8eed20 2024-03-23T09:52:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f56e12640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c9cb8ec140 0x55c9cb8de150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f56e12640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c9cb8ec140 0x55c9cb8de150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39656/0 (socket says 172.21.15.47:39656) 2024-03-23T09:52:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f56e12640 1 -- 172.21.15.47:0/1719617237 learned_addr learned my addr 172.21.15.47:0/1719617237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f56611640 1 --2- 172.21.15.47:0/1719617237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9cb8eca60 0x55c9cb846a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f57613640 1 --2- 172.21.15.47:0/1719617237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c9cb8eed20 0x55c9cb97fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f56e12640 1 -- 172.21.15.47:0/1719617237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c9cb8eed20 msgr2=0x55c9cb97fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f56e12640 1 --2- 172.21.15.47:0/1719617237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c9cb8eed20 0x55c9cb97fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f56e12640 1 -- 172.21.15.47:0/1719617237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9cb8eca60 msgr2=0x55c9cb846a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f56e12640 1 --2- 172.21.15.47:0/1719617237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9cb8eca60 0x55c9cb846a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f56e12640 1 -- 172.21.15.47:0/1719617237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c005de0 con 0x55c9cb8ec140 2024-03-23T09:52:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f57613640 1 --2- 172.21.15.47:0/1719617237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c9cb8eed20 0x55c9cb97fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.656+0000 7f4f56611640 1 --2- 172.21.15.47:0/1719617237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9cb8eca60 0x55c9cb846a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.657+0000 7f4f56e12640 1 --2- 172.21.15.47:0/1719617237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c9cb8ec140 0x55c9cb8de150 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800b2e0 tx=0x7f4f4800b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea745b8ad07e1829 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.657+0000 7f4f5560f640 1 -- 172.21.15.47:0/1719617237 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48014070 con 0x55c9cb8ec140 2024-03-23T09:52:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.657+0000 7f4f5560f640 1 -- 172.21.15.47:0/1719617237 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4800d500 con 0x55c9cb8ec140 2024-03-23T09:52:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.657+0000 7f4f5560f640 1 -- 172.21.15.47:0/1719617237 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4800d870 con 0x55c9cb8ec140 2024-03-23T09:52:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.657+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1719617237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c9cb8ec140 msgr2=0x55c9cb8de150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.657+0000 7f4f59a4af40 1 --2- 172.21.15.47:0/1719617237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c9cb8ec140 0x55c9cb8de150 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800b2e0 tx=0x7f4f4800b7a0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.657+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1719617237 shutdown_connections 2024-03-23T09:52:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.657+0000 7f4f59a4af40 1 --2- 172.21.15.47:0/1719617237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c9cb8eed20 0x55c9cb97fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.657+0000 7f4f59a4af40 1 --2- 172.21.15.47:0/1719617237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9cb8eca60 0x55c9cb846a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.657+0000 7f4f59a4af40 1 --2- 172.21.15.47:0/1719617237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c9cb8ec140 0x55c9cb8de150 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.657+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1719617237 >> 172.21.15.47:0/1719617237 conn(0x55c9cb842e40 msgr2=0x55c9cb8dfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.657+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1719617237 shutdown_connections 2024-03-23T09:52:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.657+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1719617237 wait complete. 2024-03-23T09:52:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.658+0000 7f4f59a4af40 1 Processor -- start 2024-03-23T09:52:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.658+0000 7f4f59a4af40 1 -- start start 2024-03-23T09:52:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f59a4af40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9cb8ec140 0x55c9cb9862e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f59a4af40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c9cb8eca60 0x55c9cb845d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f59a4af40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c9cb8eed20 0x55c9cb97ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f59a4af40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb8eb720 con 0x55c9cb8eca60 2024-03-23T09:52:24.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f59a4af40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb8eb160 con 0x55c9cb8ec140 2024-03-23T09:52:24.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f59a4af40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb8eb440 con 0x55c9cb8eed20 2024-03-23T09:52:24.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f56611640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c9cb8eca60 0x55c9cb845d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f56611640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c9cb8eca60 0x55c9cb845d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39668/0 (socket says 172.21.15.47:39668) 2024-03-23T09:52:24.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f56611640 1 -- 172.21.15.47:0/1941984862 learned_addr learned my addr 172.21.15.47:0/1941984862 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:24.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f56e12640 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9cb8ec140 0x55c9cb9862e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f57613640 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c9cb8eed20 0x55c9cb97ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f56611640 1 -- 172.21.15.47:0/1941984862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c9cb8eed20 msgr2=0x55c9cb97ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f56611640 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c9cb8eed20 0x55c9cb97ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f56611640 1 -- 172.21.15.47:0/1941984862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9cb8ec140 msgr2=0x55c9cb9862e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f56611640 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9cb8ec140 0x55c9cb9862e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f56611640 1 -- 172.21.15.47:0/1941984862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f48001c10 con 0x55c9cb8eca60 2024-03-23T09:52:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f56611640 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c9cb8eca60 0x55c9cb845d80 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c000c00 tx=0x7f4f4c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f3ffff640 1 -- 172.21.15.47:0/1941984862 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c052d20 con 0x55c9cb8eca60 2024-03-23T09:52:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f3ffff640 1 -- 172.21.15.47:0/1941984862 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c052e80 con 0x55c9cb8eca60 2024-03-23T09:52:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1941984862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c9cb76ec20 con 0x55c9cb8eca60 2024-03-23T09:52:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f3ffff640 1 -- 172.21.15.47:0/1941984862 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c05c630 con 0x55c9cb8eca60 2024-03-23T09:52:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.659+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1941984862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9cb762060 con 0x55c9cb8eca60 2024-03-23T09:52:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.661+0000 7f4f3ffff640 1 -- 172.21.15.47:0/1941984862 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4f4c059050 con 0x55c9cb8eca60 2024-03-23T09:52:24.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:52:24.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.661+0000 7f4f3ffff640 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4f440695e0 0x7f4f4406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.661+0000 7f4f3ffff640 1 -- 172.21.15.47:0/1941984862 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4f4c0c9dd0 con 0x55c9cb8eca60 2024-03-23T09:52:24.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.661+0000 7f4f56e12640 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4f440695e0 0x7f4f4406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.661+0000 7f4f59a4af40 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55c9cb8da040 0x55c9cb97c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.661+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1941984862 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:1 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v8 -- 0x55c9cb8462c0 con 0x55c9cb8da040 2024-03-23T09:52:24.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.661+0000 7f4f57613640 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55c9cb8da040 0x55c9cb97c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f57613640 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55c9cb8da040 0x55c9cb97c950 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f56e12640 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4f440695e0 0x7f4f4406baa0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4f48000c00 tx=0x7f4f48011040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f57613640 1 -- 172.21.15.47:0/1941984862 <== osd.0 v2:172.21.15.47:6816/37679 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4f50010700 con 0x55c9cb8da040 2024-03-23T09:52:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-03-23T09:52:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1941984862 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55c9cb8da040 msgr2=0x55c9cb97c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f59a4af40 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55c9cb8da040 0x55c9cb97c950 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1941984862 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4f440695e0 msgr2=0x7f4f4406baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f59a4af40 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4f440695e0 0x7f4f4406baa0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4f48000c00 tx=0x7f4f48011040 comp rx=0 tx=0).stop 2024-03-23T09:52:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1941984862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c9cb8eca60 msgr2=0x55c9cb845d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f59a4af40 1 --2- 172.21.15.47:0/1941984862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c9cb8eca60 0x55c9cb845d80 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c000c00 tx=0x7f4f4c002fb0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f57613640 1 -- 172.21.15.47:0/1941984862 reap_dead start 2024-03-23T09:52:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1941984862 shutdown_connections 2024-03-23T09:52:24.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1941984862 >> 172.21.15.47:0/1941984862 conn(0x55c9cb842e40 msgr2=0x55c9cb846a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:24.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1941984862 shutdown_connections 2024-03-23T09:52:24.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.662+0000 7f4f59a4af40 1 -- 172.21.15.47:0/1941984862 wait complete. 2024-03-23T09:52:24.671 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-03-23T09:52:24.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.750+0000 7f4bca60bf40 1 Processor -- start 2024-03-23T09:52:24.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.750+0000 7f4bca60bf40 1 -- start start 2024-03-23T09:52:24.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bca60bf40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d98ff74e10 0x55d99002d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bca60bf40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d990047140 0x55d98ffa20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bca60bf40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d990047a60 0x55d98ffa2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bca60bf40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98febab60 con 0x55d990047a60 2024-03-23T09:52:24.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bca60bf40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98feba800 con 0x55d98ff74e10 2024-03-23T09:52:24.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bca60bf40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d98fe9d520 con 0x55d990047140 2024-03-23T09:52:24.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc7d0a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d990047a60 0x55d98ffa2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc7d0a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d990047a60 0x55d98ffa2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39686/0 (socket says 172.21.15.47:39686) 2024-03-23T09:52:24.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc7d0a640 1 -- 172.21.15.47:0/3154725750 learned_addr learned my addr 172.21.15.47:0/3154725750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:24.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc7d0a640 1 -- 172.21.15.47:0/3154725750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d990047140 msgr2=0x55d98ffa20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:24.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc7509640 1 --2- 172.21.15.47:0/3154725750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d98ff74e10 0x55d99002d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc6d08640 1 --2- 172.21.15.47:0/3154725750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d990047140 0x55d98ffa20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc7d0a640 1 --2- 172.21.15.47:0/3154725750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d990047140 0x55d98ffa20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc7d0a640 1 -- 172.21.15.47:0/3154725750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d98ff74e10 msgr2=0x55d99002d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:24.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc7d0a640 1 --2- 172.21.15.47:0/3154725750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d98ff74e10 0x55d99002d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc7d0a640 1 -- 172.21.15.47:0/3154725750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc0005de0 con 0x55d990047a60 2024-03-23T09:52:24.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc7509640 1 --2- 172.21.15.47:0/3154725750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d98ff74e10 0x55d99002d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:24.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc6d08640 1 --2- 172.21.15.47:0/3154725750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d990047140 0x55d98ffa20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc7d0a640 1 --2- 172.21.15.47:0/3154725750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d990047a60 0x55d98ffa2600 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8002660 tx=0x7f4bb8002b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423b9c12a619df62 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc5d06640 1 -- 172.21.15.47:0/3154725750 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800f710 con 0x55d990047a60 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc5d06640 1 -- 172.21.15.47:0/3154725750 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb800f890 con 0x55d990047a60 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.751+0000 7f4bc5d06640 1 -- 172.21.15.47:0/3154725750 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800fc40 con 0x55d990047a60 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.752+0000 7f4bca60bf40 1 -- 172.21.15.47:0/3154725750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d990047a60 msgr2=0x55d98ffa2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.752+0000 7f4bca60bf40 1 --2- 172.21.15.47:0/3154725750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d990047a60 0x55d98ffa2600 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8002660 tx=0x7f4bb8002b20 comp rx=0 tx=0).stop 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.752+0000 7f4bca60bf40 1 -- 172.21.15.47:0/3154725750 shutdown_connections 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.752+0000 7f4bca60bf40 1 --2- 172.21.15.47:0/3154725750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d990047a60 0x55d98ffa2600 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.752+0000 7f4bca60bf40 1 --2- 172.21.15.47:0/3154725750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d990047140 0x55d98ffa20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.752+0000 7f4bca60bf40 1 --2- 172.21.15.47:0/3154725750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d98ff74e10 0x55d99002d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.752+0000 7f4bca60bf40 1 -- 172.21.15.47:0/3154725750 >> 172.21.15.47:0/3154725750 conn(0x55d98ff9de40 msgr2=0x55d99003a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.752+0000 7f4bca60bf40 1 -- 172.21.15.47:0/3154725750 shutdown_connections 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.752+0000 7f4bca60bf40 1 -- 172.21.15.47:0/3154725750 wait complete. 2024-03-23T09:52:24.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.752+0000 7f4bca60bf40 1 Processor -- start 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.753+0000 7f4bca60bf40 1 -- start start 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.753+0000 7f4bca60bf40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d98ff74e10 0x55d99003c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.753+0000 7f4bca60bf40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d990047140 0x55d99002bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.753+0000 7f4bca60bf40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d990047a60 0x55d990028810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.753+0000 7f4bca60bf40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99003cf50 con 0x55d98ff74e10 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.753+0000 7f4bca60bf40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98ff9e510 con 0x55d990047140 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.753+0000 7f4bca60bf40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d99002af20 con 0x55d990047a60 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.753+0000 7f4bc7509640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d98ff74e10 0x55d99003c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.753+0000 7f4bc7509640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d98ff74e10 0x55d99003c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39688/0 (socket says 172.21.15.47:39688) 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.753+0000 7f4bc7509640 1 -- 172.21.15.47:0/2549585646 learned_addr learned my addr 172.21.15.47:0/2549585646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.754+0000 7f4bc7509640 1 -- 172.21.15.47:0/2549585646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d990047a60 msgr2=0x55d990028810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.754+0000 7f4bc7509640 1 --2- 172.21.15.47:0/2549585646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d990047a60 0x55d990028810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.754+0000 7f4bc7509640 1 -- 172.21.15.47:0/2549585646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d990047140 msgr2=0x55d99002bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.754+0000 7f4bc7509640 1 --2- 172.21.15.47:0/2549585646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d990047140 0x55d99002bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.754+0000 7f4bc7509640 1 -- 172.21.15.47:0/2549585646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb8006d90 con 0x55d98ff74e10 2024-03-23T09:52:24.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.754+0000 7f4bc7509640 1 --2- 172.21.15.47:0/2549585646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d98ff74e10 0x55d99003c3e0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f4bc001f040 tx=0x7f4bc0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.754+0000 7f4bc4d04640 1 -- 172.21.15.47:0/2549585646 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0067660 con 0x55d98ff74e10 2024-03-23T09:52:24.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.754+0000 7f4bc4d04640 1 -- 172.21.15.47:0/2549585646 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc00677e0 con 0x55d98ff74e10 2024-03-23T09:52:24.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.754+0000 7f4bca60bf40 1 -- 172.21.15.47:0/2549585646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d98fec9c20 con 0x55d98ff74e10 2024-03-23T09:52:24.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.754+0000 7f4bc4d04640 1 -- 172.21.15.47:0/2549585646 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0067b90 con 0x55d98ff74e10 2024-03-23T09:52:24.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.754+0000 7f4bca60bf40 1 -- 172.21.15.47:0/2549585646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d98febd060 con 0x55d98ff74e10 2024-03-23T09:52:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.755+0000 7f4bc4d04640 1 -- 172.21.15.47:0/2549585646 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4bc0067cf0 con 0x55d98ff74e10 2024-03-23T09:52:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.756+0000 7f4bc4d04640 1 --2- 172.21.15.47:0/2549585646 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ba80695e0 0x7f4ba806baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.756+0000 7f4bc4d04640 1 -- 172.21.15.47:0/2549585646 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4bc00cca50 con 0x55d98ff74e10 2024-03-23T09:52:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.756+0000 7f4bca60bf40 1 --2- 172.21.15.47:0/2549585646 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55d99003c920 0x55d990028100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.756+0000 7f4bca60bf40 1 -- 172.21.15.47:0/2549585646 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v8 -- 0x55d99003ed10 con 0x55d99003c920 2024-03-23T09:52:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.756+0000 7f4bc6d08640 1 --2- 172.21.15.47:0/2549585646 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ba80695e0 0x7f4ba806baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.756+0000 7f4bc7d0a640 1 --2- 172.21.15.47:0/2549585646 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55d99003c920 0x55d990028100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.756+0000 7f4bc7d0a640 1 --2- 172.21.15.47:0/2549585646 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55d99003c920 0x55d990028100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.756+0000 7f4bc6d08640 1 --2- 172.21.15.47:0/2549585646 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ba80695e0 0x7f4ba806baa0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55d98ffa16b0 tx=0x7f4bb0005570 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.757+0000 7f4bc7d0a640 1 -- 172.21.15.47:0/2549585646 <== osd.3 v2:172.21.15.47:6808/37683 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f4bb8018040 con 0x55d99003c920 2024-03-23T09:52:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.757+0000 7f4bca60bf40 1 -- 172.21.15.47:0/2549585646 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55d99003c920 msgr2=0x55d990028100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.757+0000 7f4bca60bf40 1 --2- 172.21.15.47:0/2549585646 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55d99003c920 0x55d990028100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.757+0000 7f4bca60bf40 1 -- 172.21.15.47:0/2549585646 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ba80695e0 msgr2=0x7f4ba806baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.757+0000 7f4bca60bf40 1 --2- 172.21.15.47:0/2549585646 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4ba80695e0 0x7f4ba806baa0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55d98ffa16b0 tx=0x7f4bb0005570 comp rx=0 tx=0).stop 2024-03-23T09:52:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.757+0000 7f4bca60bf40 1 -- 172.21.15.47:0/2549585646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d98ff74e10 msgr2=0x55d99003c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.757+0000 7f4bca60bf40 1 --2- 172.21.15.47:0/2549585646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d98ff74e10 0x55d99003c3e0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f4bc001f040 tx=0x7f4bc0002fb0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.757+0000 7f4bc7d0a640 1 -- 172.21.15.47:0/2549585646 reap_dead start 2024-03-23T09:52:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.757+0000 7f4bca60bf40 1 -- 172.21.15.47:0/2549585646 shutdown_connections 2024-03-23T09:52:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.757+0000 7f4bca60bf40 1 -- 172.21.15.47:0/2549585646 >> 172.21.15.47:0/2549585646 conn(0x55d98ff9de40 msgr2=0x55d99003e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.757+0000 7f4bca60bf40 1 -- 172.21.15.47:0/2549585646 shutdown_connections 2024-03-23T09:52:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.757+0000 7f4bca60bf40 1 -- 172.21.15.47:0/2549585646 wait complete. 2024-03-23T09:52:24.764 INFO:teuthology.orchestra.run.smithi047.stdout:mytag123 2024-03-23T09:52:24.764 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-03-23T09:52:24.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.844+0000 7f0cd2c42f40 1 Processor -- start 2024-03-23T09:52:24.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.844+0000 7f0cd2c42f40 1 -- start start 2024-03-23T09:52:24.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0cd2c42f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ef67da3e10 0x55ef67e6f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0cd2c42f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ef67e76140 0x55ef67e6ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0cd2c42f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef67e78d20 0x55ef67e5e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0cd2c42f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef67ce9b60 con 0x55ef67e76140 2024-03-23T09:52:24.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0cd2c42f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef67ce9800 con 0x55ef67e78d20 2024-03-23T09:52:24.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0cd2c42f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef67ccc520 con 0x55ef67da3e10 2024-03-23T09:52:24.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0ccf7a0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ef67e76140 0x55ef67e6ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0ccf7a0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ef67e76140 0x55ef67e6ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39704/0 (socket says 172.21.15.47:39704) 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0ccf7a0640 1 -- 172.21.15.47:0/533975436 learned_addr learned my addr 172.21.15.47:0/533975436 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0ccffa1640 1 --2- 172.21.15.47:0/533975436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ef67da3e10 0x55ef67e6f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0cd07a2640 1 --2- 172.21.15.47:0/533975436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef67e78d20 0x55ef67e5e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0ccf7a0640 1 -- 172.21.15.47:0/533975436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ef67da3e10 msgr2=0x55ef67e6f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0ccf7a0640 1 --2- 172.21.15.47:0/533975436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ef67da3e10 0x55ef67e6f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0ccf7a0640 1 -- 172.21.15.47:0/533975436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef67e78d20 msgr2=0x55ef67e5e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0ccf7a0640 1 --2- 172.21.15.47:0/533975436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef67e78d20 0x55ef67e5e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0ccf7a0640 1 -- 172.21.15.47:0/533975436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc000a100 con 0x55ef67e76140 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.845+0000 7f0ccf7a0640 1 --2- 172.21.15.47:0/533975436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ef67e76140 0x55ef67e6ef60 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8001d80 tx=0x7f0cb8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b3cfd8e757e3c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.846+0000 7f0cce79e640 1 -- 172.21.15.47:0/533975436 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8014070 con 0x55ef67e76140 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.846+0000 7f0cce79e640 1 -- 172.21.15.47:0/533975436 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb8008030 con 0x55ef67e76140 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.846+0000 7f0cce79e640 1 -- 172.21.15.47:0/533975436 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb800f400 con 0x55ef67e76140 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.846+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/533975436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ef67e76140 msgr2=0x55ef67e6ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.846+0000 7f0cd2c42f40 1 --2- 172.21.15.47:0/533975436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ef67e76140 0x55ef67e6ef60 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8001d80 tx=0x7f0cb8001e70 comp rx=0 tx=0).stop 2024-03-23T09:52:24.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.846+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/533975436 shutdown_connections 2024-03-23T09:52:24.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.846+0000 7f0cd2c42f40 1 --2- 172.21.15.47:0/533975436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef67e78d20 0x55ef67e5e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.846+0000 7f0cd2c42f40 1 --2- 172.21.15.47:0/533975436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ef67e76140 0x55ef67e6ef60 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.846+0000 7f0cd2c42f40 1 --2- 172.21.15.47:0/533975436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ef67da3e10 0x55ef67e6f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.846+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/533975436 >> 172.21.15.47:0/533975436 conn(0x55ef67dcce40 msgr2=0x55ef67dcd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:24.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.846+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/533975436 shutdown_connections 2024-03-23T09:52:24.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.846+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/533975436 wait complete. 2024-03-23T09:52:24.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.847+0000 7f0cd2c42f40 1 Processor -- start 2024-03-23T09:52:24.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.847+0000 7f0cd2c42f40 1 -- start start 2024-03-23T09:52:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.847+0000 7f0cd2c42f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ef67da3e10 0x55ef67deae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.847+0000 7f0cd2c42f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef67e76140 0x55ef67e6a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.847+0000 7f0cd2c42f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ef67e78d20 0x55ef67e6ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.847+0000 7f0cd2c42f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef67e6e3e0 con 0x55ef67da3e10 2024-03-23T09:52:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.847+0000 7f0cd2c42f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef67e6b5c0 con 0x55ef67e76140 2024-03-23T09:52:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.847+0000 7f0cd2c42f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef67e68ad0 con 0x55ef67e78d20 2024-03-23T09:52:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0ccffa1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ef67da3e10 0x55ef67deae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0ccffa1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ef67da3e10 0x55ef67deae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39708/0 (socket says 172.21.15.47:39708) 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0ccffa1640 1 -- 172.21.15.47:0/3451882456 learned_addr learned my addr 172.21.15.47:0/3451882456 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0ccf7a0640 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef67e76140 0x55ef67e6a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0cd07a2640 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ef67e78d20 0x55ef67e6ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0ccffa1640 1 -- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ef67e78d20 msgr2=0x55ef67e6ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0ccffa1640 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ef67e78d20 0x55ef67e6ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0ccffa1640 1 -- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef67e76140 msgr2=0x55ef67e6a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0ccffa1640 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef67e76140 0x55ef67e6a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0ccffa1640 1 -- 172.21.15.47:0/3451882456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb800a040 con 0x55ef67da3e10 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0ccffa1640 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ef67da3e10 0x55ef67deae00 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000a3f0 tx=0x7f0cc0003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0ccd79c640 1 -- 172.21.15.47:0/3451882456 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc005d820 con 0x55ef67da3e10 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0ccd79c640 1 -- 172.21.15.47:0/3451882456 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc0061070 con 0x55ef67da3e10 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0ccd79c640 1 -- 172.21.15.47:0/3451882456 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc005db80 con 0x55ef67da3e10 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/3451882456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ef67cf8c20 con 0x55ef67da3e10 2024-03-23T09:52:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.848+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/3451882456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef67cec060 con 0x55ef67da3e10 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.850+0000 7f0ccd79c640 1 -- 172.21.15.47:0/3451882456 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0cc005dce0 con 0x55ef67da3e10 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.850+0000 7f0ccd79c640 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cb00695e0 0x7f0cb006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.850+0000 7f0ccd79c640 1 -- 172.21.15.47:0/3451882456 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0cc0063070 con 0x55ef67da3e10 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.850+0000 7f0ccf7a0640 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cb00695e0 0x7f0cb006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.850+0000 7f0cd2c42f40 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55ef67f18440 0x55ef67f1a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.850+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/3451882456 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:1 17.19 17:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v8 -- 0x55ef67f1ad70 con 0x55ef67f18440 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.850+0000 7f0cd07a2640 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55ef67f18440 0x55ef67f1a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0ccf7a0640 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cb00695e0 0x7f0cb006baa0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8003040 tx=0x7f0cb8009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0cd07a2640 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55ef67f18440 0x55ef67f1a830 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0cd07a2640 1 -- 172.21.15.47:0/3451882456 <== osd.5 v2:172.21.15.175:6816/38161 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f0cc8012040 con 0x55ef67f18440 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55ef67f18440 msgr2=0x55ef67f1a830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0cd2c42f40 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55ef67f18440 0x55ef67f1a830 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cb00695e0 msgr2=0x7f0cb006baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0cd2c42f40 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f0cb00695e0 0x7f0cb006baa0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8003040 tx=0x7f0cb8009000 comp rx=0 tx=0).stop 2024-03-23T09:52:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/3451882456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ef67da3e10 msgr2=0x55ef67deae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0cd2c42f40 1 --2- 172.21.15.47:0/3451882456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ef67da3e10 0x55ef67deae00 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000a3f0 tx=0x7f0cc0003330 comp rx=0 tx=0).stop 2024-03-23T09:52:24.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0cd07a2640 1 -- 172.21.15.47:0/3451882456 reap_dead start 2024-03-23T09:52:24.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/3451882456 shutdown_connections 2024-03-23T09:52:24.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/3451882456 >> 172.21.15.47:0/3451882456 conn(0x55ef67dcce40 msgr2=0x55ef67dcd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:24.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/3451882456 shutdown_connections 2024-03-23T09:52:24.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.851+0000 7f0cd2c42f40 1 -- 172.21.15.47:0/3451882456 wait complete. 2024-03-23T09:52:24.858 INFO:teuthology.orchestra.run.smithi047.stdout:mytag123 2024-03-23T09:52:24.858 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-03-23T09:52:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.939+0000 7f376b03cf40 1 Processor -- start 2024-03-23T09:52:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.939+0000 7f376b03cf40 1 -- start start 2024-03-23T09:52:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.939+0000 7f376b03cf40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f610d54a0 0x555f610bb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.939+0000 7f376b03cf40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f61145810 0x555f610d3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.939+0000 7f376b03cf40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f61160ba0 0x555f610d3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.939+0000 7f376b03cf40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f610384d0 con 0x555f61145810 2024-03-23T09:52:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.939+0000 7f376b03cf40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f60fd3b60 con 0x555f610d54a0 2024-03-23T09:52:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.939+0000 7f376b03cf40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f60fd3800 con 0x555f61160ba0 2024-03-23T09:52:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.939+0000 7f3767a89640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f61145810 0x555f610d3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.940+0000 7f3767a89640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f61145810 0x555f610d3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39718/0 (socket says 172.21.15.47:39718) 2024-03-23T09:52:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.940+0000 7f3767a89640 1 -- 172.21.15.47:0/885390594 learned_addr learned my addr 172.21.15.47:0/885390594 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.940+0000 7f3768a8b640 1 --2- 172.21.15.47:0/885390594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f61160ba0 0x555f610d3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.940+0000 7f376828a640 1 --2- 172.21.15.47:0/885390594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f610d54a0 0x555f610bb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.940+0000 7f3767a89640 1 -- 172.21.15.47:0/885390594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f61160ba0 msgr2=0x555f610d3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.940+0000 7f3767a89640 1 --2- 172.21.15.47:0/885390594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f61160ba0 0x555f610d3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.940+0000 7f3767a89640 1 -- 172.21.15.47:0/885390594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f610d54a0 msgr2=0x555f610bb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.940+0000 7f3767a89640 1 --2- 172.21.15.47:0/885390594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f610d54a0 0x555f610bb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.940+0000 7f3767a89640 1 -- 172.21.15.47:0/885390594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3750005d20 con 0x555f61145810 2024-03-23T09:52:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.940+0000 7f3767a89640 1 --2- 172.21.15.47:0/885390594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f61145810 0x555f610d3720 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f375001d030 tx=0x7f375004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59eec43e2d72186a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.940+0000 7f3766a87640 1 -- 172.21.15.47:0/885390594 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37500529b0 con 0x555f61145810 2024-03-23T09:52:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.940+0000 7f3766a87640 1 -- 172.21.15.47:0/885390594 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3750052b10 con 0x555f61145810 2024-03-23T09:52:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.940+0000 7f3766a87640 1 -- 172.21.15.47:0/885390594 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750052e80 con 0x555f61145810 2024-03-23T09:52:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.941+0000 7f376b03cf40 1 -- 172.21.15.47:0/885390594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f61145810 msgr2=0x555f610d3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.941+0000 7f376b03cf40 1 --2- 172.21.15.47:0/885390594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f61145810 0x555f610d3720 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f375001d030 tx=0x7f375004c000 comp rx=0 tx=0).stop 2024-03-23T09:52:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.941+0000 7f376b03cf40 1 -- 172.21.15.47:0/885390594 shutdown_connections 2024-03-23T09:52:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.941+0000 7f376b03cf40 1 --2- 172.21.15.47:0/885390594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f61160ba0 0x555f610d3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.941+0000 7f376b03cf40 1 --2- 172.21.15.47:0/885390594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f61145810 0x555f610d3720 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.941+0000 7f376b03cf40 1 --2- 172.21.15.47:0/885390594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f610d54a0 0x555f610bb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.941+0000 7f376b03cf40 1 -- 172.21.15.47:0/885390594 >> 172.21.15.47:0/885390594 conn(0x555f610b6e40 msgr2=0x555f61140e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.941+0000 7f376b03cf40 1 -- 172.21.15.47:0/885390594 shutdown_connections 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.941+0000 7f376b03cf40 1 -- 172.21.15.47:0/885390594 wait complete. 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.941+0000 7f376b03cf40 1 Processor -- start 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.941+0000 7f376b03cf40 1 -- start start 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376b03cf40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f610d54a0 0x555f611487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376b03cf40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f61145810 0x555f610ba510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376b03cf40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f61160ba0 0x555f610baa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376b03cf40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f60fb6520 con 0x555f610d54a0 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376b03cf40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f61148360 con 0x555f61160ba0 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376b03cf40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f61143ec0 con 0x555f61145810 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376828a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f610d54a0 0x555f611487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376828a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f610d54a0 0x555f611487b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39732/0 (socket says 172.21.15.47:39732) 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376828a640 1 -- 172.21.15.47:0/3251698097 learned_addr learned my addr 172.21.15.47:0/3251698097 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f3767a89640 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f61145810 0x555f610ba510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f3768a8b640 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f61160ba0 0x555f610baa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376828a640 1 -- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f61145810 msgr2=0x555f610ba510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376828a640 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555f61145810 0x555f610ba510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376828a640 1 -- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f61160ba0 msgr2=0x555f610baa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376828a640 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f61160ba0 0x555f610baa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376828a640 1 -- 172.21.15.47:0/3251698097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f60fe2c20 con 0x555f610d54a0 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.942+0000 7f376828a640 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f610d54a0 0x555f611487b0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f376000df30 tx=0x7f376000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.943+0000 7f3765a85640 1 -- 172.21.15.47:0/3251698097 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600032d0 con 0x555f610d54a0 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.943+0000 7f3765a85640 1 -- 172.21.15.47:0/3251698097 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376000a040 con 0x555f610d54a0 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.943+0000 7f376b03cf40 1 -- 172.21.15.47:0/3251698097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555f60fd6060 con 0x555f610d54a0 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.943+0000 7f3765a85640 1 -- 172.21.15.47:0/3251698097 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600016c0 con 0x555f610d54a0 2024-03-23T09:52:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.943+0000 7f376b03cf40 1 -- 172.21.15.47:0/3251698097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f60fd5ac0 con 0x555f610d54a0 2024-03-23T09:52:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.944+0000 7f3765a85640 1 -- 172.21.15.47:0/3251698097 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3760001820 con 0x555f610d54a0 2024-03-23T09:52:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.944+0000 7f3765a85640 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f37480695e0 0x7f374806baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.944+0000 7f3765a85640 1 -- 172.21.15.47:0/3251698097 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f3760009050 con 0x555f610d54a0 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.945+0000 7f3767a89640 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f37480695e0 0x7f374806baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.945+0000 7f376b03cf40 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x555f61148cf0 0x555f611fb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.945+0000 7f376b03cf40 1 -- 172.21.15.47:0/3251698097 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v8 -- 0x555f612071b0 con 0x555f61148cf0 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.945+0000 7f3768a8b640 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x555f61148cf0 0x555f611fb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.945+0000 7f3767a89640 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f37480695e0 0x7f374806baa0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3750000c00 tx=0x7f375004f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.945+0000 7f3768a8b640 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x555f61148cf0 0x555f611fb100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.946+0000 7f3768a8b640 1 -- 172.21.15.47:0/3251698097 <== osd.6 v2:172.21.15.175:6808/38160 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f3758004680 con 0x555f61148cf0 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.946+0000 7f376b03cf40 1 -- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x555f61148cf0 msgr2=0x555f611fb100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.946+0000 7f376b03cf40 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x555f61148cf0 0x555f611fb100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.946+0000 7f376b03cf40 1 -- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f37480695e0 msgr2=0x7f374806baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.946+0000 7f376b03cf40 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f37480695e0 0x7f374806baa0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3750000c00 tx=0x7f375004f000 comp rx=0 tx=0).stop 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.946+0000 7f376b03cf40 1 -- 172.21.15.47:0/3251698097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f610d54a0 msgr2=0x555f611487b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.946+0000 7f376b03cf40 1 --2- 172.21.15.47:0/3251698097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f610d54a0 0x555f611487b0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f376000df30 tx=0x7f376000b650 comp rx=0 tx=0).stop 2024-03-23T09:52:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.946+0000 7f3768a8b640 1 -- 172.21.15.47:0/3251698097 reap_dead start 2024-03-23T09:52:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.946+0000 7f376b03cf40 1 -- 172.21.15.47:0/3251698097 shutdown_connections 2024-03-23T09:52:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.946+0000 7f376b03cf40 1 -- 172.21.15.47:0/3251698097 >> 172.21.15.47:0/3251698097 conn(0x555f610b6e40 msgr2=0x555f610bb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.946+0000 7f376b03cf40 1 -- 172.21.15.47:0/3251698097 shutdown_connections 2024-03-23T09:52:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:24.946+0000 7f376b03cf40 1 -- 172.21.15.47:0/3251698097 wait complete. 2024-03-23T09:52:24.952 INFO:teuthology.orchestra.run.smithi047.stdout:mytag123 2024-03-23T09:52:24.953 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T09:52:24.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-03-23T09:52:25.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.095+0000 7fd2dfe38640 1 Processor -- start 2024-03-23T09:52:25.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.095+0000 7fd2dfe38640 1 -- start start 2024-03-23T09:52:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2dfe38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2d80cde40 0x7fd2d8167880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2dfe38640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d80d25e0 con 0x7fd2d815ce10 2024-03-23T09:52:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2dfe38640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d8161600 con 0x7fd2d815ca30 2024-03-23T09:52:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2dfe38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8160e50 con 0x7fd2d80cde40 2024-03-23T09:52:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2dd3d3640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd2d815ce10 0x7fd2d80cd730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56750/0 (socket says 172.21.15.47:56750) 2024-03-23T09:52:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2dd3d3640 1 -- 172.21.15.47:0/2662067967 learned_addr learned my addr 172.21.15.47:0/2662067967 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2de3d5640 1 --2- 172.21.15.47:0/2662067967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2d80cde40 0x7fd2d8167880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2dcbd2640 1 -- 172.21.15.47:0/2662067967 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254937775 0 0) 0x7fd2d80d25e0 con 0x7fd2d815ce10 2024-03-23T09:52:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2dcbd2640 1 -- 172.21.15.47:0/2662067967 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2c4003520 con 0x7fd2d815ce10 2024-03-23T09:52:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2dcbd2640 1 -- 172.21.15.47:0/2662067967 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210440350 0 0) 0x7fd2d8161600 con 0x7fd2d815ca30 2024-03-23T09:52:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2dcbd2640 1 -- 172.21.15.47:0/2662067967 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2d80d25e0 con 0x7fd2d815ca30 2024-03-23T09:52:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2de3d5640 1 -- 172.21.15.47:0/2662067967 >> v1:172.21.15.175:6789/0 conn(0x7fd2d815ca30 legacy=0x7fd2d8160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2dcbd2640 1 -- 172.21.15.47:0/2662067967 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 646863592 0 0) 0x7fd2c4003520 con 0x7fd2d815ce10 2024-03-23T09:52:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2de3d5640 1 -- 172.21.15.47:0/2662067967 >> v1:172.21.15.47:6789/0 conn(0x7fd2d815ce10 legacy=0x7fd2d80cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.096+0000 7fd2de3d5640 1 -- 172.21.15.47:0/2662067967 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d8161040 con 0x7fd2d80cde40 2024-03-23T09:52:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.097+0000 7fd2de3d5640 1 --2- 172.21.15.47:0/2662067967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2d80cde40 0x7fd2d8167880 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009dc0 tx=0x7fd2d002bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6dde4ea684af40d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.097+0000 7fd2dcbd2640 1 -- 172.21.15.47:0/2662067967 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8001670 con 0x7fd2d80cde40 2024-03-23T09:52:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.097+0000 7fd2dcbd2640 1 -- 172.21.15.47:0/2662067967 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d002c910 con 0x7fd2d80cde40 2024-03-23T09:52:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.097+0000 7fd2dcbd2640 1 -- 172.21.15.47:0/2662067967 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d002cc80 con 0x7fd2d80cde40 2024-03-23T09:52:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.098+0000 7fd2dfe38640 1 -- 172.21.15.47:0/2662067967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2d80cde40 msgr2=0x7fd2d8167880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.098+0000 7fd2dfe38640 1 --2- 172.21.15.47:0/2662067967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2d80cde40 0x7fd2d8167880 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009dc0 tx=0x7fd2d002bde0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.098+0000 7fd2dfe38640 1 -- 172.21.15.47:0/2662067967 shutdown_connections 2024-03-23T09:52:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.098+0000 7fd2dfe38640 1 --2- 172.21.15.47:0/2662067967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2d80cde40 0x7fd2d8167880 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.098+0000 7fd2dfe38640 1 -- 172.21.15.47:0/2662067967 >> 172.21.15.47:0/2662067967 conn(0x7fd2d806b160 msgr2=0x7fd2d806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.098+0000 7fd2dfe38640 1 -- 172.21.15.47:0/2662067967 shutdown_connections 2024-03-23T09:52:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.098+0000 7fd2dfe38640 1 -- 172.21.15.47:0/2662067967 wait complete. 2024-03-23T09:52:25.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.099+0000 7fd2dfe38640 1 Processor -- start 2024-03-23T09:52:25.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.099+0000 7fd2dfe38640 1 -- start start 2024-03-23T09:52:25.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2dfe38640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2d80cde40 0x7fd2d81579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2dfe38640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2d815ca30 0x7fd2d8159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2dfe38640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2d815ce10 0x7fd2d814ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2dfe38640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8158cf0 con 0x7fd2d815ce10 2024-03-23T09:52:25.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2dfe38640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8159e90 con 0x7fd2d815ca30 2024-03-23T09:52:25.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2dfe38640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8159110 con 0x7fd2d80cde40 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2de3d5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2d815ce10 0x7fd2d814ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2de3d5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2d815ce10 0x7fd2d814ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39738/0 (socket says 172.21.15.47:39738) 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2de3d5640 1 -- 172.21.15.47:0/1761853413 learned_addr learned my addr 172.21.15.47:0/1761853413 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2ddbd4640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2d80cde40 0x7fd2d81579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2dd3d3640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2d815ca30 0x7fd2d8159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2de3d5640 1 -- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2d80cde40 msgr2=0x7fd2d81579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2de3d5640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2d80cde40 0x7fd2d81579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2de3d5640 1 -- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2d815ca30 msgr2=0x7fd2d8159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2de3d5640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2d815ca30 0x7fd2d8159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2de3d5640 1 -- 172.21.15.47:0/1761853413 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d0009a70 con 0x7fd2d815ce10 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.100+0000 7fd2de3d5640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2d815ce10 0x7fd2d814ae10 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fd2d002c3f0 tx=0x7fd2d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.101+0000 7fd2ce7fc640 1 -- 172.21.15.47:0/1761853413 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0007000 con 0x7fd2d815ce10 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.101+0000 7fd2ce7fc640 1 -- 172.21.15.47:0/1761853413 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d0009450 con 0x7fd2d815ce10 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.101+0000 7fd2dfe38640 1 -- 172.21.15.47:0/1761853413 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d814b350 con 0x7fd2d815ce10 2024-03-23T09:52:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.101+0000 7fd2ce7fc640 1 -- 172.21.15.47:0/1761853413 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d003e650 con 0x7fd2d815ce10 2024-03-23T09:52:25.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.101+0000 7fd2dfe38640 1 -- 172.21.15.47:0/1761853413 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d8158350 con 0x7fd2d815ce10 2024-03-23T09:52:25.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.103+0000 7fd2ce7fc640 1 -- 172.21.15.47:0/1761853413 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd2d003b050 con 0x7fd2d815ce10 2024-03-23T09:52:25.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.103+0000 7fd2ce7fc640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd2bc06ac30 0x7fd2bc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.103+0000 7fd2ce7fc640 1 -- 172.21.15.47:0/1761853413 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd2d00ab620 con 0x7fd2d815ce10 2024-03-23T09:52:25.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.103+0000 7fd2ddbd4640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd2bc06ac30 0x7fd2bc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.104+0000 7fd2ddbd4640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd2bc06ac30 0x7fd2bc06d0f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40011e0 tx=0x7fd2d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.104+0000 7fd2e29ac740 1 Processor -- start 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.104+0000 7fd2e29ac740 1 -- start start 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2e29ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40a0830 0x55ada40a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2e29ac740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada4070850 0x55ada409daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2e29ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 0x55ada409dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2e29ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3fe0dd0 con 0x55ada40743b0 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2e29ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3fde560 con 0x55ada40a0830 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2e29ac740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ada3fd9e50 con 0x55ada4070850 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2de3d5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 0x55ada409dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2de3d5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 0x55ada409dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39742/0 (socket says 172.21.15.47:39742) 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2de3d5640 1 -- 172.21.15.47:0/2914334526 learned_addr learned my addr 172.21.15.47:0/2914334526 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2dd3d3640 1 --2- 172.21.15.47:0/2914334526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada4070850 0x55ada409daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2ddbd4640 1 --2- 172.21.15.47:0/2914334526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40a0830 0x55ada40a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2de3d5640 1 -- 172.21.15.47:0/2914334526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada4070850 msgr2=0x55ada409daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2de3d5640 1 --2- 172.21.15.47:0/2914334526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada4070850 0x55ada409daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2de3d5640 1 -- 172.21.15.47:0/2914334526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40a0830 msgr2=0x55ada40a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2de3d5640 1 --2- 172.21.15.47:0/2914334526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40a0830 0x55ada40a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2de3d5640 1 -- 172.21.15.47:0/2914334526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ada3fa1470 con 0x55ada40743b0 2024-03-23T09:52:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2de3d5640 1 --2- 172.21.15.47:0/2914334526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 0x55ada409dfe0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00a9af0 tx=0x7fd2d007abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b6b67e48358c0b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2bb7fe640 1 -- 172.21.15.47:0/2914334526 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d007e2f0 con 0x55ada40743b0 2024-03-23T09:52:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2bb7fe640 1 -- 172.21.15.47:0/2914334526 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d00828b0 con 0x55ada40743b0 2024-03-23T09:52:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.105+0000 7fd2bb7fe640 1 -- 172.21.15.47:0/2914334526 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0076bd0 con 0x55ada40743b0 2024-03-23T09:52:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.106+0000 7fd2e29ac740 1 -- 172.21.15.47:0/2914334526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 msgr2=0x55ada409dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.106+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/2914334526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 0x55ada409dfe0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00a9af0 tx=0x7fd2d007abc0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.106+0000 7fd2e29ac740 1 -- 172.21.15.47:0/2914334526 shutdown_connections 2024-03-23T09:52:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.106+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/2914334526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 0x55ada409dfe0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.106+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/2914334526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada4070850 0x55ada409daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.106+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/2914334526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40a0830 0x55ada40a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.106+0000 7fd2e29ac740 1 -- 172.21.15.47:0/2914334526 >> 172.21.15.47:0/2914334526 conn(0x55ada4072ac0 msgr2=0x55ada40bb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.106+0000 7fd2e29ac740 1 -- 172.21.15.47:0/2914334526 shutdown_connections 2024-03-23T09:52:25.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.106+0000 7fd2e29ac740 1 -- 172.21.15.47:0/2914334526 wait complete. 2024-03-23T09:52:25.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.107+0000 7fd2e29ac740 1 -- start start 2024-03-23T09:52:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.107+0000 7fd2e29ac740 1 Processor -- start 2024-03-23T09:52:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2e29ac740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40743b0 0x55ada40baec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2e29ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 0x55ada40b8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2e29ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40b8b90 0x55ada408b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2e29ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3f98280 con 0x55ada40a0830 2024-03-23T09:52:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2e29ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3f97f20 con 0x55ada40b8b90 2024-03-23T09:52:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2e29ac740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ada3f5d3c0 con 0x55ada40743b0 2024-03-23T09:52:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2e29ac740 10 client.0 _ll_register_callbacks cb 0x55ada3faf380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:52:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2dd3d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 0x55ada40b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2dd3d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 0x55ada40b8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39750/0 (socket says 172.21.15.47:39750) 2024-03-23T09:52:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2dd3d3640 1 -- 172.21.15.47:0/3081801341 learned_addr learned my addr 172.21.15.47:0/3081801341 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2de3d5640 1 --2- 172.21.15.47:0/3081801341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40b8b90 0x55ada408b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2ddbd4640 1 --2- 172.21.15.47:0/3081801341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40743b0 0x55ada40baec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2dd3d3640 1 -- 172.21.15.47:0/3081801341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40743b0 msgr2=0x55ada40baec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2dd3d3640 1 --2- 172.21.15.47:0/3081801341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40743b0 0x55ada40baec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2dd3d3640 1 -- 172.21.15.47:0/3081801341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40b8b90 msgr2=0x55ada408b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2dd3d3640 1 --2- 172.21.15.47:0/3081801341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40b8b90 0x55ada408b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2dd3d3640 1 -- 172.21.15.47:0/3081801341 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d8158350 con 0x55ada40a0830 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2dd3d3640 1 --2- 172.21.15.47:0/3081801341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 0x55ada40b8650 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8000c00 tx=0x7fd2c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df635c98eb1e84db server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2b8ff9640 1 -- 172.21.15.47:0/3081801341 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8015660 con 0x55ada40a0830 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2b8ff9640 1 -- 172.21.15.47:0/3081801341 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd2c80157c0 con 0x55ada40a0830 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2e29ac740 1 -- 172.21.15.47:0/3081801341 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ada4003690 con 0x55ada40a0830 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2b8ff9640 1 -- 172.21.15.47:0/3081801341 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2c8015980 con 0x55ada40a0830 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2e29ac740 10 client.9744 fetch_fsmap learned FSMap version 86 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.108+0000 7fd2e29ac740 1 -- 172.21.15.47:0/3081801341 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55ada3fd84d0 con 0x55ada40a0830 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.109+0000 7fd2b8ff9640 1 -- 172.21.15.47:0/3081801341 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ada4003690 con 0x55ada40a0830 2024-03-23T09:52:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.109+0000 7fd2b8ff9640 1 -- 172.21.15.47:0/3081801341 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 86) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd2c8015ae0 con 0x55ada40a0830 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.109+0000 7fd2e29ac740 10 client.9744 fetch_fsmap finished waiting for FSMap version 86 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.109+0000 7fd2e29ac740 10 client.9744 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.109+0000 7fd2e29ac740 20 client.9744 populate_metadata read hostname 'smithi047' 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.109+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/3081801341 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55ada408da10 0x55ada408ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.109+0000 7fd2e29ac740 4 client.9744 mds_command: new command op to 9570 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.109+0000 7fd2e29ac740 1 -- 172.21.15.47:0/3081801341 --> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ada3fcbeb0 con 0x55ada408da10 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.109+0000 7fd2ddbd4640 1 --2- 172.21.15.47:0/3081801341 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55ada408da10 0x55ada408ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.109+0000 7fd2b8ff9640 10 client.9744 ms_handle_connect on v2:172.21.15.47:6832/1868958629 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.109+0000 7fd2ddbd4640 1 --2- 172.21.15.47:0/3081801341 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55ada408da10 0x55ada408ddf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16b00ff3a70ed175 server_cookie=19668df0d247b65f in_seq=0 out_seq=0 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2b8ff9640 1 -- 172.21.15.47:0/3081801341 <== mds.0 v2:172.21.15.47:6832/1868958629 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ada3fcbeb0 con 0x55ada408da10 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2b8ff9640 10 client.9744 handle_command_reply: tid=0 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2e29ac740 1 client.9744 shutdown 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2e29ac740 1 -- 172.21.15.47:0/3081801341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 msgr2=0x55ada40b8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/3081801341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 0x55ada40b8650 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8000c00 tx=0x7fd2c8005f90 comp rx=0 tx=0).stop 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2e29ac740 1 -- 172.21.15.47:0/3081801341 shutdown_connections 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/3081801341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40b8b90 0x55ada408b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/3081801341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 0x55ada40b8650 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/3081801341 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55ada408da10 0x55ada408ddf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/3081801341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40743b0 0x55ada40baec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2b8ff9640 0 client.9744 ms_handle_reset on v2:172.21.15.47:6832/1868958629 2024-03-23T09:52:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2e29ac740 1 -- 172.21.15.47:0/3081801341 >> 172.21.15.47:0/3081801341 conn(0x55ada4070850 msgr2=0x55ada40beb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2e29ac740 1 -- 172.21.15.47:0/3081801341 shutdown_connections 2024-03-23T09:52:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2e29ac740 1 -- 172.21.15.47:0/3081801341 wait complete. 2024-03-23T09:52:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.110+0000 7fd2e29ac740 20 client.9744 trim_cache size 0 max 16384 2024-03-23T09:52:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.123+0000 7fd2e29ac740 1 Processor -- start 2024-03-23T09:52:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.123+0000 7fd2e29ac740 1 -- start start 2024-03-23T09:52:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.123+0000 7fd2e29ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40743b0 0x55ada409d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.123+0000 7fd2e29ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 0x55ada409dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.123+0000 7fd2e29ac740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40b8b90 0x55ada40beb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.123+0000 7fd2e29ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3f62fa0 con 0x55ada40a0830 2024-03-23T09:52:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.123+0000 7fd2e29ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada409d390 con 0x55ada40743b0 2024-03-23T09:52:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.123+0000 7fd2e29ac740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ada40a4720 con 0x55ada40b8b90 2024-03-23T09:52:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.123+0000 7fd2dd3d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 0x55ada409dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.123+0000 7fd2dd3d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 0x55ada409dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39752/0 (socket says 172.21.15.47:39752) 2024-03-23T09:52:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.123+0000 7fd2dd3d3640 1 -- 172.21.15.47:0/4022903959 learned_addr learned my addr 172.21.15.47:0/4022903959 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.124+0000 7fd2dd3d3640 1 -- 172.21.15.47:0/4022903959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40b8b90 msgr2=0x55ada40beb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.124+0000 7fd2dd3d3640 1 --2- 172.21.15.47:0/4022903959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40b8b90 0x55ada40beb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.124+0000 7fd2dd3d3640 1 -- 172.21.15.47:0/4022903959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40743b0 msgr2=0x55ada409d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.124+0000 7fd2ddbd4640 1 --2- 172.21.15.47:0/4022903959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40743b0 0x55ada409d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.124+0000 7fd2dd3d3640 1 --2- 172.21.15.47:0/4022903959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40743b0 0x55ada409d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.124+0000 7fd2dd3d3640 1 -- 172.21.15.47:0/4022903959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ada400c9e0 con 0x55ada40a0830 2024-03-23T09:52:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.124+0000 7fd2dd3d3640 1 --2- 172.21.15.47:0/4022903959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 0x55ada409dce0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8000ec0 tx=0x7fd2c800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1659f271fd10c388 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.124+0000 7fd2bbfff640 1 -- 172.21.15.47:0/4022903959 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8014410 con 0x55ada40a0830 2024-03-23T09:52:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.124+0000 7fd2bbfff640 1 -- 172.21.15.47:0/4022903959 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c8028070 con 0x55ada40a0830 2024-03-23T09:52:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.124+0000 7fd2bbfff640 1 -- 172.21.15.47:0/4022903959 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8014770 con 0x55ada40a0830 2024-03-23T09:52:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.124+0000 7fd2e29ac740 1 -- 172.21.15.47:0/4022903959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 msgr2=0x55ada409dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.125+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/4022903959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 0x55ada409dce0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8000ec0 tx=0x7fd2c800c000 comp rx=0 tx=0).stop 2024-03-23T09:52:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.125+0000 7fd2e29ac740 1 -- 172.21.15.47:0/4022903959 shutdown_connections 2024-03-23T09:52:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.125+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/4022903959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40b8b90 0x55ada40beb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.125+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/4022903959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40a0830 0x55ada409dce0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.125+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/4022903959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40743b0 0x55ada409d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.125+0000 7fd2e29ac740 1 -- 172.21.15.47:0/4022903959 >> 172.21.15.47:0/4022903959 conn(0x55ada4070850 msgr2=0x55ada40bacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.125+0000 7fd2e29ac740 1 -- 172.21.15.47:0/4022903959 shutdown_connections 2024-03-23T09:52:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.125+0000 7fd2e29ac740 1 -- 172.21.15.47:0/4022903959 wait complete. 2024-03-23T09:52:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.125+0000 7fd2e29ac740 1 -- start start 2024-03-23T09:52:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.125+0000 7fd2e29ac740 1 Processor -- start 2024-03-23T09:52:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2e29ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 0x55ada4105210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2e29ac740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40a0830 0x55ada40b9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2e29ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40b8b90 0x55ada409a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2e29ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada409b270 con 0x55ada40743b0 2024-03-23T09:52:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2e29ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada40b2230 con 0x55ada40b8b90 2024-03-23T09:52:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2e29ac740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ada40ca390 con 0x55ada40a0830 2024-03-23T09:52:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2e29ac740 10 client.0 _ll_register_callbacks cb 0x7fd2d0076800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2ddbd4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 0x55ada4105210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2ddbd4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 0x55ada4105210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39768/0 (socket says 172.21.15.47:39768) 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2ddbd4640 1 -- 172.21.15.47:0/1644521313 learned_addr learned my addr 172.21.15.47:0/1644521313 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2dd3d3640 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40a0830 0x55ada40b9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2de3d5640 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40b8b90 0x55ada409a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2ddbd4640 1 -- 172.21.15.47:0/1644521313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40a0830 msgr2=0x55ada40b9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2ddbd4640 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40a0830 0x55ada40b9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2ddbd4640 1 -- 172.21.15.47:0/1644521313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40b8b90 msgr2=0x55ada409a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2ddbd4640 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40b8b90 0x55ada409a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2ddbd4640 1 -- 172.21.15.47:0/1644521313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ada3fd84d0 con 0x55ada40743b0 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.126+0000 7fd2de3d5640 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40b8b90 0x55ada409a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2ddbd4640 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 0x55ada4105210 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x55ada40cbc00 tx=0x7fd2d4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d566830b280a63 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2b8ff9640 1 -- 172.21.15.47:0/1644521313 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4026be0 con 0x55ada40743b0 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2b8ff9640 1 -- 172.21.15.47:0/1644521313 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd2d4026d40 con 0x55ada40743b0 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2e29ac740 1 -- 172.21.15.47:0/1644521313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ada408c2a0 con 0x55ada40743b0 2024-03-23T09:52:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2b8ff9640 1 -- 172.21.15.47:0/1644521313 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2d40035f0 con 0x55ada40743b0 2024-03-23T09:52:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2e29ac740 10 client.9750 fetch_fsmap learned FSMap version 86 2024-03-23T09:52:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2e29ac740 1 -- 172.21.15.47:0/1644521313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55ada3f7a5c0 con 0x55ada40743b0 2024-03-23T09:52:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2b8ff9640 1 -- 172.21.15.47:0/1644521313 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ada408c2a0 con 0x55ada40743b0 2024-03-23T09:52:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2b8ff9640 1 -- 172.21.15.47:0/1644521313 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 86) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd2d4003750 con 0x55ada40743b0 2024-03-23T09:52:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2e29ac740 10 client.9750 fetch_fsmap finished waiting for FSMap version 86 2024-03-23T09:52:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2e29ac740 10 client.9750 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-03-23T09:52:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2e29ac740 20 client.9750 populate_metadata read hostname 'smithi047' 2024-03-23T09:52:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55ada40ca820 0x55ada40cac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2e29ac740 4 client.9750 mds_command: new command op to 9570 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T09:52:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2e29ac740 1 -- 172.21.15.47:0/1644521313 --> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ada40bbeb0 con 0x55ada40ca820 2024-03-23T09:52:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.127+0000 7fd2dd3d3640 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55ada40ca820 0x55ada40cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.128+0000 7fd2b8ff9640 10 client.9750 ms_handle_connect on v2:172.21.15.47:6832/1868958629 2024-03-23T09:52:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.128+0000 7fd2dd3d3640 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55ada40ca820 0x55ada40cac00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31e079aa4926213f server_cookie=ca207110073f3e72 in_seq=0 out_seq=0 2024-03-23T09:52:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.138+0000 7fd2b8ff9640 1 -- 172.21.15.47:0/1644521313 <== mds.0 v2:172.21.15.47:6832/1868958629 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ada40bbeb0 con 0x55ada40ca820 2024-03-23T09:52:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.138+0000 7fd2b8ff9640 10 client.9750 handle_command_reply: tid=0 2024-03-23T09:52:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.138+0000 7fd2e29ac740 1 client.9750 shutdown 2024-03-23T09:52:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.138+0000 7fd2e29ac740 1 -- 172.21.15.47:0/1644521313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 msgr2=0x55ada4105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.138+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 0x55ada4105210 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x55ada40cbc00 tx=0x7fd2d4011000 comp rx=0 tx=0).stop 2024-03-23T09:52:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.138+0000 7fd2e29ac740 1 -- 172.21.15.47:0/1644521313 shutdown_connections 2024-03-23T09:52:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.138+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ada40b8b90 0x55ada409a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.138+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.47:6832/1868958629,v1:172.21.15.47:6834/1868958629] conn(0x55ada40ca820 0x55ada40cac00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.138+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55ada40a0830 0x55ada40b9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.138+0000 7fd2e29ac740 1 --2- 172.21.15.47:0/1644521313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ada40743b0 0x55ada4105210 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.138+0000 7fd2b8ff9640 0 client.9750 ms_handle_reset on v2:172.21.15.47:6832/1868958629 2024-03-23T09:52:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2e29ac740 1 -- 172.21.15.47:0/1644521313 >> 172.21.15.47:0/1644521313 conn(0x55ada4070850 msgr2=0x55ada408b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2e29ac740 1 -- 172.21.15.47:0/1644521313 shutdown_connections 2024-03-23T09:52:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2e29ac740 1 -- 172.21.15.47:0/1644521313 wait complete. 2024-03-23T09:52:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2e29ac740 20 client.9750 trim_cache size 0 max 16384 2024-03-23T09:52:25.142 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:52:25.142 INFO:teuthology.orchestra.run.smithi047.stdout: "message": "", 2024-03-23T09:52:25.142 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0 2024-03-23T09:52:25.142 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:52:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2dfe38640 1 -- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd2bc06ac30 msgr2=0x7fd2bc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2dfe38640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd2bc06ac30 0x7fd2bc06d0f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40011e0 tx=0x7fd2d4009040 comp rx=0 tx=0).stop 2024-03-23T09:52:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2dfe38640 1 -- 172.21.15.47:0/1761853413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2d815ce10 msgr2=0x7fd2d814ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2dfe38640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2d815ce10 0x7fd2d814ae10 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fd2d002c3f0 tx=0x7fd2d0004a40 comp rx=0 tx=0).stop 2024-03-23T09:52:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2dfe38640 1 -- 172.21.15.47:0/1761853413 shutdown_connections 2024-03-23T09:52:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2dfe38640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd2bc06ac30 0x7fd2bc06d0f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2dfe38640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2d815ce10 0x7fd2d814ae10 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2dfe38640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2d815ca30 0x7fd2d8159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2dfe38640 1 --2- 172.21.15.47:0/1761853413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2d80cde40 0x7fd2d81579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2dfe38640 1 -- 172.21.15.47:0/1761853413 >> 172.21.15.47:0/1761853413 conn(0x7fd2d806b160 msgr2=0x7fd2d806b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2dfe38640 1 -- 172.21.15.47:0/1761853413 shutdown_connections 2024-03-23T09:52:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.139+0000 7fd2dfe38640 1 -- 172.21.15.47:0/1761853413 wait complete. 2024-03-23T09:52:25.151 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:52:25.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.252+0000 7f1371be7640 1 Processor -- start 2024-03-23T09:52:25.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.252+0000 7f1371be7640 1 -- start start 2024-03-23T09:52:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f1371be7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f136c0cf8a0 0x7f136c167b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f1371be7640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c0d0ee0 con 0x7f136c15cb40 2024-03-23T09:52:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f1371be7640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c1616f0 con 0x7f136c0b8300 2024-03-23T09:52:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f1371be7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0b5a10 con 0x7f136c0cf8a0 2024-03-23T09:52:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f136b7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f136c15cb40 0x7f136c160770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56756/0 (socket says 172.21.15.47:56756) 2024-03-23T09:52:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f136b7fe640 1 -- 172.21.15.47:0/3669226403 learned_addr learned my addr 172.21.15.47:0/3669226403 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f136bfff640 1 --2- 172.21.15.47:0/3669226403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f136c0cf8a0 0x7f136c167b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f136a7fc640 1 -- 172.21.15.47:0/3669226403 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997357736 0 0) 0x7f136c0d0ee0 con 0x7f136c15cb40 2024-03-23T09:52:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f136a7fc640 1 -- 172.21.15.47:0/3669226403 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1354003520 con 0x7f136c15cb40 2024-03-23T09:52:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f136a7fc640 1 -- 172.21.15.47:0/3669226403 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819527088 0 0) 0x7f136c1616f0 con 0x7f136c0b8300 2024-03-23T09:52:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f136a7fc640 1 -- 172.21.15.47:0/3669226403 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f136c0d0ee0 con 0x7f136c0b8300 2024-03-23T09:52:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f136a7fc640 1 -- 172.21.15.47:0/3669226403 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1904673656 0 0) 0x7f1354003520 con 0x7f136c15cb40 2024-03-23T09:52:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f136bfff640 1 -- 172.21.15.47:0/3669226403 >> v1:172.21.15.175:6789/0 conn(0x7f136c0b8300 legacy=0x7f136c0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f136bfff640 1 -- 172.21.15.47:0/3669226403 >> v1:172.21.15.47:6789/0 conn(0x7f136c15cb40 legacy=0x7f136c160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.253+0000 7f136bfff640 1 -- 172.21.15.47:0/3669226403 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c160e80 con 0x7f136c0cf8a0 2024-03-23T09:52:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.254+0000 7f136bfff640 1 --2- 172.21.15.47:0/3669226403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f136c0cf8a0 0x7f136c167b00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1364009df0 tx=0x7f136402bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aed09e89c9713266 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.254+0000 7f136a7fc640 1 -- 172.21.15.47:0/3669226403 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c001670 con 0x7f136c0cf8a0 2024-03-23T09:52:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.254+0000 7f136a7fc640 1 -- 172.21.15.47:0/3669226403 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136402c910 con 0x7f136c0cf8a0 2024-03-23T09:52:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.254+0000 7f136a7fc640 1 -- 172.21.15.47:0/3669226403 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136402cc80 con 0x7f136c0cf8a0 2024-03-23T09:52:25.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.255+0000 7f1371be7640 1 -- 172.21.15.47:0/3669226403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f136c0cf8a0 msgr2=0x7f136c167b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.255+0000 7f1371be7640 1 --2- 172.21.15.47:0/3669226403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f136c0cf8a0 0x7f136c167b00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1364009df0 tx=0x7f136402bde0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.255+0000 7f1371be7640 1 -- 172.21.15.47:0/3669226403 shutdown_connections 2024-03-23T09:52:25.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.255+0000 7f1371be7640 1 --2- 172.21.15.47:0/3669226403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f136c0cf8a0 0x7f136c167b00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.255+0000 7f1371be7640 1 -- 172.21.15.47:0/3669226403 >> 172.21.15.47:0/3669226403 conn(0x7f136c06b160 msgr2=0x7f136c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.255+0000 7f1371be7640 1 -- 172.21.15.47:0/3669226403 shutdown_connections 2024-03-23T09:52:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.255+0000 7f1371be7640 1 -- 172.21.15.47:0/3669226403 wait complete. 2024-03-23T09:52:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.256+0000 7f1371be7640 1 Processor -- start 2024-03-23T09:52:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.256+0000 7f1371be7640 1 -- start start 2024-03-23T09:52:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f1371be7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f136c0b8300 0x7f136c0cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f1371be7640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f136c0cf8a0 0x7f136c1f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f1371be7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c15cb40 0x7f136c0cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f1371be7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c15d5f0 con 0x7f136c15cb40 2024-03-23T09:52:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f1371be7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0cb910 con 0x7f136c0b8300 2024-03-23T09:52:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f1371be7640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136c1f6f70 con 0x7f136c0cf8a0 2024-03-23T09:52:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f136bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c15cb40 0x7f136c0cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f136bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c15cb40 0x7f136c0cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39772/0 (socket says 172.21.15.47:39772) 2024-03-23T09:52:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f136bfff640 1 -- 172.21.15.47:0/287074099 learned_addr learned my addr 172.21.15.47:0/287074099 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f136b7fe640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f136c0b8300 0x7f136c0cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f136affd640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f136c0cf8a0 0x7f136c1f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f136bfff640 1 -- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f136c0cf8a0 msgr2=0x7f136c1f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f136bfff640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f136c0cf8a0 0x7f136c1f6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f136bfff640 1 -- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f136c0b8300 msgr2=0x7f136c0cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f136bfff640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f136c0b8300 0x7f136c0cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f136bfff640 1 -- 172.21.15.47:0/287074099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1364009a70 con 0x7f136c15cb40 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f136affd640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f136c0cf8a0 0x7f136c1f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.257+0000 7f136bfff640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c15cb40 0x7f136c0cbd50 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f1364009ee0 tx=0x7f1364004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.258+0000 7f1353fff640 1 -- 172.21.15.47:0/287074099 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1364009450 con 0x7f136c15cb40 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.258+0000 7f1353fff640 1 -- 172.21.15.47:0/287074099 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1364034d00 con 0x7f136c15cb40 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.258+0000 7f1371be7640 1 -- 172.21.15.47:0/287074099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c1f7250 con 0x7f136c15cb40 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.258+0000 7f1353fff640 1 -- 172.21.15.47:0/287074099 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136403f660 con 0x7f136c15cb40 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.258+0000 7f1371be7640 1 -- 172.21.15.47:0/287074099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c15d280 con 0x7f136c15cb40 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.259+0000 7f1371be7640 1 -- 172.21.15.47:0/287074099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136c072360 con 0x7f136c15cb40 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.259+0000 7f1353fff640 1 -- 172.21.15.47:0/287074099 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f136403f7c0 con 0x7f136c15cb40 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.259+0000 7f1353fff640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f135806ac00 0x7f135806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.259+0000 7f1353fff640 1 -- 172.21.15.47:0/287074099 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f13640abd10 con 0x7f136c15cb40 2024-03-23T09:52:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.260+0000 7f136b7fe640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f135806ac00 0x7f135806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.260+0000 7f136b7fe640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f135806ac00 0x7f135806d0c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f135c000fd0 tx=0x7f135c001690 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.264+0000 7f1353fff640 1 -- 172.21.15.47:0/287074099 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1364077030 con 0x7f136c15cb40 2024-03-23T09:52:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.482+0000 7f1371be7640 1 -- 172.21.15.47:0/287074099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f136c066be0 con 0x7f136c15cb40 2024-03-23T09:52:25.484 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:52:25.483+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:52:25.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.493+0000 7f1353fff640 1 -- 172.21.15.47:0/287074099 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1364043090 con 0x7f136c15cb40 2024-03-23T09:52:25.495 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:52:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.495+0000 7f1371be7640 1 -- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f135806ac00 msgr2=0x7f135806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.496+0000 7f1371be7640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f135806ac00 0x7f135806d0c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f135c000fd0 tx=0x7f135c001690 comp rx=0 tx=0).stop 2024-03-23T09:52:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.496+0000 7f1371be7640 1 -- 172.21.15.47:0/287074099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c15cb40 msgr2=0x7f136c0cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.496+0000 7f1371be7640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c15cb40 0x7f136c0cbd50 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f1364009ee0 tx=0x7f1364004a40 comp rx=0 tx=0).stop 2024-03-23T09:52:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.496+0000 7f1371be7640 1 -- 172.21.15.47:0/287074099 shutdown_connections 2024-03-23T09:52:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.496+0000 7f1371be7640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f135806ac00 0x7f135806d0c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.496+0000 7f1371be7640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c15cb40 0x7f136c0cbd50 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.496+0000 7f1371be7640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f136c0cf8a0 0x7f136c1f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.496+0000 7f1371be7640 1 --2- 172.21.15.47:0/287074099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f136c0b8300 0x7f136c0cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.496+0000 7f1371be7640 1 -- 172.21.15.47:0/287074099 >> 172.21.15.47:0/287074099 conn(0x7f136c06b160 msgr2=0x7f136c0b5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.496+0000 7f1371be7640 1 -- 172.21.15.47:0/287074099 shutdown_connections 2024-03-23T09:52:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.496+0000 7f1371be7640 1 -- 172.21.15.47:0/287074099 wait complete. 2024-03-23T09:52:25.498 INFO:tasks.ceph.mds.c.smithi047.stderr: -2111> 2024-03-23T09:52:12.478+0000 7fefb746b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:52:25.509 DEBUG:teuthology.orchestra.run.smithi047:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-03-23T09:52:25.523 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:52:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.548+0000 7f351d4f8280 1 -- start start 2024-03-23T09:52:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.548+0000 7f351d4f8280 1 Processor -- start 2024-03-23T09:52:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.548+0000 7f351d4f8280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb97c8ec00 0x55cb97d28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.548+0000 7f351d4f8280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb97c8f800 0x55cb97d28b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.548+0000 7f351d4f8280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97c8fc00 0x55cb97d29080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.548+0000 7f351d4f8280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb97d20a00 con 0x55cb97c8fc00 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.548+0000 7f351d4f8280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb97d20780 con 0x55cb97c8f800 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.548+0000 7f351d4f8280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb97d20640 con 0x55cb97c8ec00 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.548+0000 7f351c49f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97c8fc00 0x55cb97d29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.548+0000 7f351c49f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97c8fc00 0x55cb97d29080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39798/0 (socket says 172.21.15.47:39798) 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.548+0000 7f351c49f640 1 -- 172.21.15.47:0/3484689195 learned_addr learned my addr 172.21.15.47:0/3484689195 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f351b49d640 1 --2- 172.21.15.47:0/3484689195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb97c8f800 0x55cb97d28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f351bc9e640 1 --2- 172.21.15.47:0/3484689195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb97c8ec00 0x55cb97d28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f351c49f640 1 -- 172.21.15.47:0/3484689195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb97c8ec00 msgr2=0x55cb97d28580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f351c49f640 1 --2- 172.21.15.47:0/3484689195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb97c8ec00 0x55cb97d28580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f351c49f640 1 -- 172.21.15.47:0/3484689195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb97c8f800 msgr2=0x55cb97d28b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f351c49f640 1 --2- 172.21.15.47:0/3484689195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb97c8f800 0x55cb97d28b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f351c49f640 1 -- 172.21.15.47:0/3484689195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb97cb49c0 con 0x55cb97c8fc00 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f351bc9e640 1 --2- 172.21.15.47:0/3484689195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb97c8ec00 0x55cb97d28580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f351c49f640 1 --2- 172.21.15.47:0/3484689195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97c8fc00 0x55cb97d29080 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x55cb98aa0c90 tx=0x55cb98b56060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a28234539ae9f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f3519499640 1 -- 172.21.15.47:0/3484689195 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb97d53080 con 0x55cb97c8fc00 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f3519499640 1 -- 172.21.15.47:0/3484689195 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb98a54300 con 0x55cb97c8fc00 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f3519499640 1 -- 172.21.15.47:0/3484689195 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb97d53600 con 0x55cb97c8fc00 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f351d4f8280 1 -- 172.21.15.47:0/3484689195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb97cb4d00 con 0x55cb97c8fc00 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f351d4f8280 1 -- 172.21.15.47:0/3484689195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55cb97cb49c0 con 0x55cb97c8fc00 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f351d4f8280 4 waiting for MDS map... 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f3519499640 1 -- 172.21.15.47:0/3484689195 <== mon.0 v2:172.21.15.47:3300/0 4 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x55cb97cba380 con 0x55cb97c8fc00 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f3519499640 1 -- 172.21.15.47:0/3484689195 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 87) v1 ==== 2573+0+0 (secure 0 0 0) 0x55cb97ce0f00 con 0x55cb97c8fc00 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.549+0000 7f351d4f8280 4 Got MDS map 87 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.550+0000 7f351d4f8280 10 main: JournalTool::main 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.550+0000 7f351d4f8280 4 main: JournalTool: connecting to RADOS... 2024-03-23T09:52:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.550+0000 7f351d4f8280 1 Processor -- start 2024-03-23T09:52:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.550+0000 7f351d4f8280 1 -- start start 2024-03-23T09:52:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351d4f8280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97e01c00 0x55cb97d29600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351d4f8280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98ba0000 0x55cb97d29b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351d4f8280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb98ba0400 0x55cb97d2a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351d4f8280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb97cbfe00 con 0x55cb97e01c00 2024-03-23T09:52:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351d4f8280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb97d20780 con 0x55cb98ba0400 2024-03-23T09:52:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351d4f8280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb97d20a00 con 0x55cb98ba0000 2024-03-23T09:52:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351b49d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97e01c00 0x55cb97d29600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351b49d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97e01c00 0x55cb97d29600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39806/0 (socket says 172.21.15.47:39806) 2024-03-23T09:52:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351b49d640 1 -- 172.21.15.47:0/4016357293 learned_addr learned my addr 172.21.15.47:0/4016357293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351bc9e640 1 --2- 172.21.15.47:0/4016357293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98ba0000 0x55cb97d29b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351b49d640 1 -- 172.21.15.47:0/4016357293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98ba0000 msgr2=0x55cb97d29b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351b49d640 1 --2- 172.21.15.47:0/4016357293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98ba0000 0x55cb97d29b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351b49d640 1 -- 172.21.15.47:0/4016357293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb98ba0400 msgr2=0x55cb97d2a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351b49d640 1 --2- 172.21.15.47:0/4016357293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb98ba0400 0x55cb97d2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351b49d640 1 -- 172.21.15.47:0/4016357293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb97cb4b60 con 0x55cb97e01c00 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351bc9e640 1 --2- 172.21.15.47:0/4016357293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98ba0000 0x55cb97d29b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f351b49d640 1 --2- 172.21.15.47:0/4016357293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97e01c00 0x55cb97d29600 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x55cb98baa0f0 tx=0x55cb98b58c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decbbc2728300ef1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f3517495640 1 -- 172.21.15.47:0/4016357293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb97d53080 con 0x55cb97e01c00 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f3517495640 1 -- 172.21.15.47:0/4016357293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb98a54a80 con 0x55cb97e01c00 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.551+0000 7f3517495640 1 -- 172.21.15.47:0/4016357293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb97d53600 con 0x55cb97e01c00 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.552+0000 7f351d4f8280 1 -- 172.21.15.47:0/4016357293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97e01c00 msgr2=0x55cb97d29600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.552+0000 7f351d4f8280 1 --2- 172.21.15.47:0/4016357293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97e01c00 0x55cb97d29600 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x55cb98baa0f0 tx=0x55cb98b58c00 comp rx=0 tx=0).stop 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.552+0000 7f351d4f8280 1 -- 172.21.15.47:0/4016357293 shutdown_connections 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.552+0000 7f351d4f8280 1 --2- 172.21.15.47:0/4016357293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb98ba0400 0x55cb97d2a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.552+0000 7f351d4f8280 1 --2- 172.21.15.47:0/4016357293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98ba0000 0x55cb97d29b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.552+0000 7f351d4f8280 1 --2- 172.21.15.47:0/4016357293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97e01c00 0x55cb97d29600 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.552+0000 7f351d4f8280 1 -- 172.21.15.47:0/4016357293 >> 172.21.15.47:0/4016357293 conn(0x55cb97e01800 msgr2=0x55cb97d79000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.552+0000 7f351d4f8280 1 -- 172.21.15.47:0/4016357293 shutdown_connections 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.552+0000 7f351d4f8280 1 -- 172.21.15.47:0/4016357293 wait complete. 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.552+0000 7f351d4f8280 1 Processor -- start 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.552+0000 7f351d4f8280 1 -- start start 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351d4f8280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb97e01800 0x55cb97d2b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351d4f8280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba1000 0x55cb97d2ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351d4f8280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98ba0c00 0x55cb97d2a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351d4f8280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb97d20500 con 0x55cb98ba1000 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351d4f8280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb97d203c0 con 0x55cb97e01800 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351d4f8280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb97d20280 con 0x55cb98ba0c00 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351bc9e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba1000 0x55cb97d2ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351bc9e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba1000 0x55cb97d2ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39820/0 (socket says 172.21.15.47:39820) 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351bc9e640 1 -- 172.21.15.47:0/2396544357 learned_addr learned my addr 172.21.15.47:0/2396544357 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351b49d640 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb97e01800 0x55cb97d2b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351b49d640 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98ba0c00 0x55cb97d2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351bc9e640 1 -- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98ba0c00 msgr2=0x55cb97d2a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351bc9e640 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98ba0c00 0x55cb97d2a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351bc9e640 1 -- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb97e01800 msgr2=0x55cb97d2b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351bc9e640 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb97e01800 0x55cb97d2b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351bc9e640 1 -- 172.21.15.47:0/2396544357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb97cb4820 con 0x55cb98ba1000 2024-03-23T09:52:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351b49d640 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb97e01800 0x55cb97d2b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.553+0000 7f351bc9e640 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba1000 0x55cb97d2ac00 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x55cb98baade0 tx=0x55cb98b58d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.554+0000 7f3515c92640 1 -- 172.21.15.47:0/2396544357 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb97d53080 con 0x55cb98ba1000 2024-03-23T09:52:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.554+0000 7f3515c92640 1 -- 172.21.15.47:0/2396544357 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb97cee180 con 0x55cb98ba1000 2024-03-23T09:52:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.554+0000 7f3515c92640 1 -- 172.21.15.47:0/2396544357 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb97d53600 con 0x55cb98ba1000 2024-03-23T09:52:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.554+0000 7f351d4f8280 1 -- 172.21.15.47:0/2396544357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cb97cb44e0 con 0x55cb98ba1000 2024-03-23T09:52:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.554+0000 7f351d4f8280 1 -- 172.21.15.47:0/2396544357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb97cb41a0 con 0x55cb98ba1000 2024-03-23T09:52:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.554+0000 7f351d4f8280 4 main: JournalTool: resolving pool 16 2024-03-23T09:52:25.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.555+0000 7f3515c92640 1 -- 172.21.15.47:0/2396544357 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x55cb97cff080 con 0x55cb98ba1000 2024-03-23T09:52:25.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.555+0000 7f3515c92640 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55cb98bc4c00 0x55cb97d2b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.555+0000 7f3515c92640 1 -- 172.21.15.47:0/2396544357 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x55cb97cba540 con 0x55cb98ba1000 2024-03-23T09:52:25.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.555+0000 7f351d4f8280 4 main: JournalTool: creating IoCtx.. 2024-03-23T09:52:25.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.555+0000 7f351d4f8280 4 main: Executing for rank 0 2024-03-23T09:52:25.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.555+0000 7f351b49d640 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55cb98bc4c00 0x55cb97d2b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.556+0000 7f351d4f8280 1 -- start start 2024-03-23T09:52:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.556+0000 7f351d4f8280 1 Processor -- start 2024-03-23T09:52:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.556+0000 7f351b49d640 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55cb98bc4c00 0x55cb97d2b700 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55cb98baa0f0 tx=0x55cb98a25020 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.556+0000 7f351d4f8280 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba0400 0x55cb97d2a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.556+0000 7f351d4f8280 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb98ba0000 0x55cb97d29b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.556+0000 7f351d4f8280 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98bc5000 0x55cb97d29600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.556+0000 7f351d4f8280 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb97d203c0 con 0x55cb98ba0400 2024-03-23T09:52:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.556+0000 7f351d4f8280 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb97d20a00 con 0x55cb98ba0000 2024-03-23T09:52:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.556+0000 7f351d4f8280 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb97d20780 con 0x55cb98bc5000 2024-03-23T09:52:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351c49f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba0400 0x55cb97d2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351c49f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba0400 0x55cb97d2a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39830/0 (socket says 172.21.15.47:39830) 2024-03-23T09:52:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351c49f640 1 -- 172.21.15.47:0/1008502376 learned_addr learned my addr 172.21.15.47:0/1008502376 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351bc9e640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb98ba0000 0x55cb97d29b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351b49d640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98bc5000 0x55cb97d29600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351c49f640 1 -- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98bc5000 msgr2=0x55cb97d29600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351c49f640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98bc5000 0x55cb97d29600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351c49f640 1 -- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb98ba0000 msgr2=0x55cb97d29b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351c49f640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb98ba0000 0x55cb97d29b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351c49f640 1 -- 172.21.15.47:0/1008502376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb98bf8000 con 0x55cb98ba0400 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351c49f640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba0400 0x55cb97d2a100 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x55cb98aa10b0 tx=0x55cb98b56120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38a798542393672 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f3512c8c640 1 -- 172.21.15.47:0/1008502376 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb97d531e0 con 0x55cb98ba0400 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f3512c8c640 1 -- 172.21.15.47:0/1008502376 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb98a54300 con 0x55cb98ba0400 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f3512c8c640 1 -- 172.21.15.47:0/1008502376 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb97d52f20 con 0x55cb98ba0400 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb98bf44e0 con 0x55cb98ba0400 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55cb98bf5d40 con 0x55cb98ba0400 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f351d4f8280 4 waiting for MDS map... 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f3512c8c640 1 -- 172.21.15.47:0/1008502376 <== mon.0 v2:172.21.15.47:3300/0 4 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x55cb97cbaa80 con 0x55cb98ba0400 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.557+0000 7f3512c8c640 1 -- 172.21.15.47:0/1008502376 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 87) v1 ==== 2573+0+0 (secure 0 0 0) 0x55cb98054300 con 0x55cb98ba0400 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.558+0000 7f351d4f8280 4 Got MDS map 87 2024-03-23T09:52:25.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.558+0000 7f351d4f8280 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55cb98bc5400 0x55cb9805e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.558+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55cb98bc5800 con 0x55cb98bc5400 2024-03-23T09:52:25.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.558+0000 7f351bc9e640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55cb98bc5400 0x55cb9805e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.558+0000 7f351bc9e640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55cb98bc5400 0x55cb9805e000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=914168f4f5a0165c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.562+0000 7f351bc9e640 1 -- 172.21.15.47:0/1008502376 <== osd.0 v2:172.21.15.47:6816/37679 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v118'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55cb98b40b40 con 0x55cb98bc5400 2024-03-23T09:52:25.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.562+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55cb98bc5c00 con 0x55cb98bc5400 2024-03-23T09:52:25.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.563+0000 7f351bc9e640 1 -- 172.21.15.47:0/1008502376 <== osd.0 v2:172.21.15.47:6816/37679 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v118'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55cb98b40b40 con 0x55cb98bc5400 2024-03-23T09:52:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:writing EResetJournal entry 2024-03-23T09:52:25.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.563+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55cb9804d400 con 0x55cb98bc5400 2024-03-23T09:52:25.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.563+0000 7f351d4f8280 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55cb9804d800 0x55cb9805e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.563+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55cb9804dc00 con 0x55cb9804d800 2024-03-23T09:52:25.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.563+0000 7f351d4f8280 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55cb98070000 0x55cb9805eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.563+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55cb98070400 con 0x55cb98070000 2024-03-23T09:52:25.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.563+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55cb98070800 con 0x55cb9804d800 2024-03-23T09:52:25.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.563+0000 7f351d4f8280 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55cb98070c00 0x55cb9805f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.563+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55cb98071000 con 0x55cb98070c00 2024-03-23T09:52:25.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.563+0000 7f351d4f8280 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55cb98071400 0x55cb9805f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.563+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55cb98071800 con 0x55cb98071400 2024-03-23T09:52:25.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.564+0000 7f351c49f640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55cb98070000 0x55cb9805eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.564+0000 7f351b49d640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55cb9804d800 0x55cb9805e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.564+0000 7f351bc9e640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55cb98070c00 0x55cb9805f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.564+0000 7f351b49d640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55cb98071400 0x55cb9805f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.564+0000 7f351c49f640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55cb98070000 0x55cb9805eb00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=f2fd651a67ff948b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.564+0000 7f351bc9e640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55cb98070c00 0x55cb9805f080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=fa5def027079312c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.565+0000 7f351b49d640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55cb9804d800 0x55cb9805e580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=453c67476c50bec9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.565+0000 7f351b49d640 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55cb98071400 0x55cb9805f600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=5f74169c7f22073f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.565+0000 7f351bc9e640 1 -- 172.21.15.47:0/1008502376 <== osd.0 v2:172.21.15.47:6816/37679 3 ==== osd_op_reply(3 200.00000001 [delete] v118'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55cb98b40b40 con 0x55cb98bc5400 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.565+0000 7f351c49f640 1 -- 172.21.15.47:0/1008502376 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== osd_op_reply(5 200.00000003 [delete] v118'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55cb98b40d80 con 0x55cb98070000 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.566+0000 7f351bc9e640 1 -- 172.21.15.47:0/1008502376 <== osd.3 v2:172.21.15.47:6808/37683 1 ==== osd_op_reply(7 200.00000005 [delete] v118'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55cb98b40b40 con 0x55cb98070c00 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.566+0000 7f351b49d640 1 -- 172.21.15.47:0/1008502376 <== osd.5 v2:172.21.15.175:6816/38161 1 ==== osd_op_reply(4 200.00000002 [delete] v118'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55cb98b418c0 con 0x55cb9804d800 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.566+0000 7f3511489640 1 -- 172.21.15.47:0/1008502376 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55cb98ba1c00 con 0x55cb98bc5400 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.567+0000 7f351b49d640 1 -- 172.21.15.47:0/1008502376 <== osd.5 v2:172.21.15.175:6816/38161 2 ==== osd_op_reply(6 200.00000004 [delete] v118'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55cb98b418c0 con 0x55cb9804d800 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.567+0000 7f351bc9e640 1 -- 172.21.15.47:0/1008502376 <== osd.0 v2:172.21.15.47:6816/37679 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v118'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55cb98b40b40 con 0x55cb98bc5400 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351b49d640 1 -- 172.21.15.47:0/1008502376 <== osd.7 v2:172.21.15.175:6824/38174 1 ==== osd_op_reply(8 200.00000006 [delete] v118'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55cb98b418c0 con 0x55cb98071400 2024-03-23T09:52:25.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55cb98bc5400 msgr2=0x55cb9805e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55cb98bc5400 0x55cb9805e000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55cb98070000 msgr2=0x55cb9805eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55cb98070000 0x55cb9805eb00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55cb98070c00 msgr2=0x55cb9805f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55cb98070c00 0x55cb9805f080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55cb9804d800 msgr2=0x55cb9805e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55cb9804d800 0x55cb9805e580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55cb98071400 msgr2=0x55cb9805f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351b49d640 1 -- 172.21.15.47:0/1008502376 reap_dead start 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55cb98071400 0x55cb9805f600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351b49d640 1 -- 172.21.15.47:0/1008502376 reap_dead start 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba0400 msgr2=0x55cb97d2a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba0400 0x55cb97d2a100 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x55cb98aa10b0 tx=0x55cb98b56120 comp rx=0 tx=0).stop 2024-03-23T09:52:25.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 shutdown_connections 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 --2- 172.21.15.47:0/1008502376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba0400 0x55cb97d2a100 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 >> 172.21.15.47:0/1008502376 conn(0x55cb98ba0800 msgr2=0x55cb97d79800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 shutdown_connections 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.569+0000 7f351d4f8280 1 -- 172.21.15.47:0/1008502376 wait complete. 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 -- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55cb98bc4c00 msgr2=0x55cb97d2b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55cb98bc4c00 0x55cb97d2b700 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55cb98baa0f0 tx=0x55cb98a25020 comp rx=0 tx=0).stop 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 -- 172.21.15.47:0/2396544357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba1000 msgr2=0x55cb97d2ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba1000 0x55cb97d2ac00 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x55cb98baade0 tx=0x55cb98b58d20 comp rx=0 tx=0).stop 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 -- 172.21.15.47:0/2396544357 shutdown_connections 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55cb98bc4c00 0x55cb97d2b700 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb98ba0c00 0x55cb97d2a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb98ba1000 0x55cb97d2ac00 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 --2- 172.21.15.47:0/2396544357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb97e01800 0x55cb97d2b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 -- 172.21.15.47:0/2396544357 >> 172.21.15.47:0/2396544357 conn(0x55cb97e01c00 msgr2=0x55cb97d79000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 -- 172.21.15.47:0/2396544357 shutdown_connections 2024-03-23T09:52:25.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 -- 172.21.15.47:0/2396544357 wait complete. 2024-03-23T09:52:25.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 -- 172.21.15.47:0/3484689195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97c8fc00 msgr2=0x55cb97d29080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 --2- 172.21.15.47:0/3484689195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97c8fc00 0x55cb97d29080 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x55cb98aa0c90 tx=0x55cb98b56060 comp rx=0 tx=0).stop 2024-03-23T09:52:25.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 -- 172.21.15.47:0/3484689195 shutdown_connections 2024-03-23T09:52:25.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 --2- 172.21.15.47:0/3484689195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb97c8fc00 0x55cb97d29080 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 --2- 172.21.15.47:0/3484689195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb97c8f800 0x55cb97d28b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 --2- 172.21.15.47:0/3484689195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cb97c8ec00 0x55cb97d28580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 -- 172.21.15.47:0/3484689195 >> 172.21.15.47:0/3484689195 conn(0x55cb97c8f000 msgr2=0x55cb97d78800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 -- 172.21.15.47:0/3484689195 shutdown_connections 2024-03-23T09:52:25.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.570+0000 7f351d4f8280 1 -- 172.21.15.47:0/3484689195 wait complete. 2024-03-23T09:52:25.578 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.069235, result: writing EResetJournal entry 2024-03-23T09:52:25.578 DEBUG:teuthology.orchestra.run.smithi047:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-03-23T09:52:25.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.655+0000 7f867ae38740 1 -- start start 2024-03-23T09:52:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.655+0000 7f867ae38740 1 Processor -- start 2024-03-23T09:52:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.655+0000 7f867ae38740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa58ec0c00 0x55fa58f5bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.655+0000 7f867ae38740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa58ec1800 0x55fa58f5b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.655+0000 7f867ae38740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa58ec1c00 0x55fa58f5b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.655+0000 7f867ae38740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa58f52640 con 0x55fa58ec1800 2024-03-23T09:52:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.655+0000 7f867ae38740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa58f52500 con 0x55fa58ec0c00 2024-03-23T09:52:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.655+0000 7f867ae38740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fa58f523c0 con 0x55fa58ec1c00 2024-03-23T09:52:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.655+0000 7f8678533640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa58ec1800 0x55fa58f5b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.655+0000 7f8678533640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa58ec1800 0x55fa58f5b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39842/0 (socket says 172.21.15.47:39842) 2024-03-23T09:52:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.655+0000 7f8678533640 1 -- 172.21.15.47:0/2695123303 learned_addr learned my addr 172.21.15.47:0/2695123303 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f8678d34640 1 --2- 172.21.15.47:0/2695123303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa58ec0c00 0x55fa58f5bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f8679535640 1 --2- 172.21.15.47:0/2695123303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa58ec1c00 0x55fa58f5b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f8678533640 1 -- 172.21.15.47:0/2695123303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa58ec1c00 msgr2=0x55fa58f5b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f8678533640 1 --2- 172.21.15.47:0/2695123303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa58ec1c00 0x55fa58f5b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f8678533640 1 -- 172.21.15.47:0/2695123303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa58ec0c00 msgr2=0x55fa58f5bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f8678533640 1 --2- 172.21.15.47:0/2695123303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa58ec0c00 0x55fa58f5bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f8678533640 1 -- 172.21.15.47:0/2695123303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa58ee69c0 con 0x55fa58ec1800 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f8678533640 1 --2- 172.21.15.47:0/2695123303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa58ec1800 0x55fa58f5b600 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x55fa59ceb2c0 tx=0x55fa59cccc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c8b3910be8c983 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f867652f640 1 -- 172.21.15.47:0/2695123303 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fa58f83080 con 0x55fa58ec1800 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f867652f640 1 -- 172.21.15.47:0/2695123303 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fa58f1e900 con 0x55fa58ec1800 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f867652f640 1 -- 172.21.15.47:0/2695123303 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fa58f83600 con 0x55fa58ec1800 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f867ae38740 1 -- 172.21.15.47:0/2695123303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa58ee6d00 con 0x55fa58ec1800 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f867ae38740 1 -- 172.21.15.47:0/2695123303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55fa58ee69c0 con 0x55fa58ec1800 2024-03-23T09:52:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f867ae38740 4 waiting for MDS map... 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f867652f640 1 -- 172.21.15.47:0/2695123303 <== mon.0 v2:172.21.15.47:3300/0 4 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x55fa58f08c40 con 0x55fa58ec1800 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f867652f640 1 -- 172.21.15.47:0/2695123303 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 87) v1 ==== 2573+0+0 (secure 0 0 0) 0x55fa58f11200 con 0x55fa58ec1800 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.656+0000 7f867ae38740 4 Got MDS map 87 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.657+0000 7f867ae38740 4 datascan.main: Using metadata pool output 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.657+0000 7f867ae38740 4 datascan.main: connecting to RADOS... 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.657+0000 7f867ae38740 1 Processor -- start 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.657+0000 7f867ae38740 1 -- start start 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.657+0000 7f867ae38740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa59ca9c00 0x55fa58f5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f867ae38740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa59de0000 0x55fa58f5a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f867ae38740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa59de0400 0x55fa58f5c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f867ae38740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa58f52280 con 0x55fa59de0400 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f867ae38740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa58f52a00 con 0x55fa59ca9c00 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f867ae38740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fa58f52780 con 0x55fa59de0000 2024-03-23T09:52:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f8678533640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa59de0400 0x55fa58f5c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f8678533640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa59de0400 0x55fa58f5c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39844/0 (socket says 172.21.15.47:39844) 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f8678533640 1 -- 172.21.15.47:0/970321580 learned_addr learned my addr 172.21.15.47:0/970321580 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f8679535640 1 --2- 172.21.15.47:0/970321580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa59ca9c00 0x55fa58f5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f8678d34640 1 --2- 172.21.15.47:0/970321580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa59de0000 0x55fa58f5a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f8678533640 1 -- 172.21.15.47:0/970321580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa59de0000 msgr2=0x55fa58f5a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f8678533640 1 --2- 172.21.15.47:0/970321580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa59de0000 0x55fa58f5a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f8678533640 1 -- 172.21.15.47:0/970321580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa59ca9c00 msgr2=0x55fa58f5ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f8678533640 1 --2- 172.21.15.47:0/970321580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa59ca9c00 0x55fa58f5ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f8678533640 1 -- 172.21.15.47:0/970321580 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa58ee6b60 con 0x55fa59de0400 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f8678533640 1 --2- 172.21.15.47:0/970321580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa59de0400 0x55fa58f5c100 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x55fa59ceb8c0 tx=0x55fa59cccd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb062573d4f49046 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f867452b640 1 -- 172.21.15.47:0/970321580 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fa58f831e0 con 0x55fa59de0400 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f867452b640 1 -- 172.21.15.47:0/970321580 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fa59cc4000 con 0x55fa59de0400 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.658+0000 7f867452b640 1 -- 172.21.15.47:0/970321580 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fa58f82f20 con 0x55fa59de0400 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.659+0000 7f867ae38740 1 -- 172.21.15.47:0/970321580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa59de0400 msgr2=0x55fa58f5c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.659+0000 7f867ae38740 1 --2- 172.21.15.47:0/970321580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa59de0400 0x55fa58f5c100 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x55fa59ceb8c0 tx=0x55fa59cccd20 comp rx=0 tx=0).stop 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.659+0000 7f867ae38740 1 -- 172.21.15.47:0/970321580 shutdown_connections 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.659+0000 7f867ae38740 1 --2- 172.21.15.47:0/970321580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa59de0400 0x55fa58f5c100 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.659+0000 7f867ae38740 1 --2- 172.21.15.47:0/970321580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa59de0000 0x55fa58f5a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.659+0000 7f867ae38740 1 --2- 172.21.15.47:0/970321580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa59ca9c00 0x55fa58f5ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.659+0000 7f867ae38740 1 -- 172.21.15.47:0/970321580 >> 172.21.15.47:0/970321580 conn(0x55fa59ca9800 msgr2=0x55fa58fa9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.659+0000 7f867ae38740 1 -- 172.21.15.47:0/970321580 shutdown_connections 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.659+0000 7f867ae38740 1 -- 172.21.15.47:0/970321580 wait complete. 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.659+0000 7f867ae38740 1 Processor -- start 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.659+0000 7f867ae38740 1 -- start start 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f867ae38740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa59ca9800 0x55fa58f5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f867ae38740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa59de1000 0x55fa58f5a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f867ae38740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa59de0c00 0x55fa58f5c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f867ae38740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa58f523c0 con 0x55fa59ca9800 2024-03-23T09:52:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f867ae38740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa58f52140 con 0x55fa59de1000 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f867ae38740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fa58f52000 con 0x55fa59de0c00 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8679535640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa59ca9800 0x55fa58f5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8679535640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa59ca9800 0x55fa58f5ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39858/0 (socket says 172.21.15.47:39858) 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 learned_addr learned my addr 172.21.15.47:0/2103910002 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa59de0c00 msgr2=0x55fa58f5c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8678533640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa59de0c00 0x55fa58f5c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8679535640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa59de0c00 0x55fa58f5c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8678d34640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa59de1000 0x55fa58f5a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa59de1000 msgr2=0x55fa58f5a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8679535640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa59de1000 0x55fa58f5a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa58ee6d00 con 0x55fa59ca9800 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8679535640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa59ca9800 0x55fa58f5ab00 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x55fa59ca56b0 tx=0x55fa59ca6c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8672d28640 1 -- 172.21.15.47:0/2103910002 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fa58f831e0 con 0x55fa59ca9800 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8672d28640 1 -- 172.21.15.47:0/2103910002 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fa58f1e480 con 0x55fa59ca9800 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fa58ee69c0 con 0x55fa59ca9800 2024-03-23T09:52:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.660+0000 7f8672d28640 1 -- 172.21.15.47:0/2103910002 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fa58f82f20 con 0x55fa59ca9800 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.661+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa58ee6b60 con 0x55fa59ca9800 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.661+0000 7f867ae38740 4 datascan.init: resolving metadata pool 16 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.662+0000 7f8672d28640 1 -- 172.21.15.47:0/2103910002 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x55fa58f26840 con 0x55fa59ca9800 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.662+0000 7f8672d28640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55fa59df0c00 0x55fa5921c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.662+0000 7f8672d28640 1 -- 172.21.15.47:0/2103910002 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x55fa58f08a80 con 0x55fa59ca9800 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.662+0000 7f867ae38740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.662+0000 7f867ae38740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.662+0000 7f867ae38740 4 datascan.main: opening data pool 'cephfs_data' 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.662+0000 7f867ae38740 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55fa59de0800 0x55fa58f5c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.662+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59de0400 con 0x55fa59de0800 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.662+0000 7f8678d34640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55fa59df0c00 0x55fa5921c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.662+0000 7f8678533640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55fa59de0800 0x55fa58f5c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.663+0000 7f8678d34640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55fa59df0c00 0x55fa5921c000 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x55fa58fbc6f0 tx=0x55fa59ca7da0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.663+0000 7f8678533640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55fa59de0800 0x55fa58f5c680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.663+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.7 v2:172.21.15.175:6824/38174 1 ==== osd_op_reply(1 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0800 2024-03-23T09:52:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.663+0000 7f8673d2a640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55fa59df1c00 0x55fa5921c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.663+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923a000 con 0x55fa59df1c00 2024-03-23T09:52:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.664+0000 7f8679535640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55fa59df1c00 0x55fa5921c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.664+0000 7f8679535640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55fa59df1c00 0x55fa5921c580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.664+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== osd_op_reply(2 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59df1c00 2024-03-23T09:52:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.664+0000 7f867452b640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55fa5923a800 0x55fa5921cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.664+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923ac00 con 0x55fa5923a800 2024-03-23T09:52:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.664+0000 7f8678d34640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55fa5923a800 0x55fa5921cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.664+0000 7f8678d34640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55fa5923a800 0x55fa5921cb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.665+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.3 v2:172.21.15.47:6808/37683 1 ==== osd_op_reply(3 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa5923a800 2024-03-23T09:52:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.665+0000 7f8673d2a640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55fa5923a400 0x55fa5921d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.665+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b400 con 0x55fa5923a400 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.665+0000 7f8678533640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55fa5923a400 0x55fa5921d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.665+0000 7f8678533640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55fa5923a400 0x55fa5921d080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.665+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 1 ==== osd_op_reply(4 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.665+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b000 con 0x55fa59de0800 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.666+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.7 v2:172.21.15.175:6824/38174 2 ==== osd_op_reply(5 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0800 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.666+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b800 con 0x55fa5923a800 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.666+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.3 v2:172.21.15.47:6808/37683 2 ==== osd_op_reply(6 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa5923a800 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.666+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59de1c00 con 0x55fa5923a400 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.666+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 2 ==== osd_op_reply(7 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.666+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:8 17.38 17:1c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923bc00 con 0x55fa59de0800 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.667+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.7 v2:172.21.15.175:6824/38174 3 ==== osd_op_reply(8 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0800 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.667+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:9 17.4 17:20000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59de1800 con 0x55fa5923a400 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.667+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 3 ==== osd_op_reply(9 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.667+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:10 17.24 17:24000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b400 con 0x55fa5923a800 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.667+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.3 v2:172.21.15.47:6808/37683 3 ==== osd_op_reply(10 [pgnls start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa5923a800 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.667+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:11 17.14 17:28000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59ca9400 con 0x55fa5923a800 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.667+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.3 v2:172.21.15.47:6808/37683 4 ==== osd_op_reply(11 [pgnls start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa5923a800 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.667+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:12 17.34 17:2c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59de1400 con 0x55fa5923a400 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.668+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 4 ==== osd_op_reply(12 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.668+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:13 17.c 17:30000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923ac00 con 0x55fa59de0800 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.668+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.7 v2:172.21.15.175:6824/38174 4 ==== osd_op_reply(13 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0800 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.668+0000 7f8673d2a640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55fa59de0400 0x55fa5921d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.668+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:14 17.2c 17:34000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59256000 con 0x55fa59de0400 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.668+0000 7f8679535640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55fa59de0400 0x55fa5921d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.669+0000 7f8679535640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55fa59de0400 0x55fa5921d600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.669+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.4 v2:172.21.15.175:6800/38159 1 ==== osd_op_reply(14 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59de0400 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.669+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:15 17.1c 17:38000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b800 con 0x55fa59df1c00 2024-03-23T09:52:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.669+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.1 v2:172.21.15.47:6824/37680 2 ==== osd_op_reply(15 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59df1c00 2024-03-23T09:52:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.669+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:16 17.3c 17:3c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59256400 con 0x55fa5923a400 2024-03-23T09:52:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.669+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 5 ==== osd_op_reply(16 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.669+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:17 17.2 17:40000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b400 con 0x55fa59de0400 2024-03-23T09:52:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.670+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.4 v2:172.21.15.175:6800/38159 2 ==== osd_op_reply(17 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59de0400 2024-03-23T09:52:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.670+0000 7f8673d2a640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55fa59256800 0x55fa5921db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.670+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:18 17.22 17:44000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59256c00 con 0x55fa59256800 2024-03-23T09:52:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.670+0000 7f8678d34640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55fa59256800 0x55fa5921db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.670+0000 7f8678d34640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55fa59256800 0x55fa5921db80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.671+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.5 v2:172.21.15.175:6816/38161 1 ==== osd_op_reply(18 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa59256800 2024-03-23T09:52:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.671+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:19 17.12 17:48000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59df1800 con 0x55fa5923a800 2024-03-23T09:52:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.671+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.3 v2:172.21.15.47:6808/37683 5 ==== osd_op_reply(19 [pgnls start_epoch 118 out=81b] v117'22 uv22 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa5923a800 2024-03-23T09:52:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.671+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:20 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59de0000 con 0x55fa5923a800 2024-03-23T09:52:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.671+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.3 v2:172.21.15.47:6808/37683 6 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa5923a800 2024-03-23T09:52:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.671+0000 7f867ae38740 10 datascan.operator(): handling object 1099511627778.0 2024-03-23T09:52:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.671+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:21 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923a000 con 0x55fa5923a800 2024-03-23T09:52:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.672+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.3 v2:172.21.15.47:6808/37683 7 ==== osd_op_reply(21 10000000002.00000000 [call] v118'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa5923a800 2024-03-23T09:52:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.672+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:22 17.32 17:4c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b800 con 0x55fa59de0800 2024-03-23T09:52:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.673+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.7 v2:172.21.15.175:6824/38174 5 ==== osd_op_reply(22 [pgnls start_epoch 118 out=49b] v116'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0800 2024-03-23T09:52:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.673+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:23 17.a 17:50000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923a000 con 0x55fa59de0800 2024-03-23T09:52:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.673+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.7 v2:172.21.15.175:6824/38174 6 ==== osd_op_reply(23 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0800 2024-03-23T09:52:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.673+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:24 17.2a 17:54000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59257000 con 0x55fa5923a800 2024-03-23T09:52:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.673+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.3 v2:172.21.15.47:6808/37683 8 ==== osd_op_reply(24 [pgnls start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa5923a800 2024-03-23T09:52:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.673+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:25 17.1a 17:58000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923ac00 con 0x55fa5923a400 2024-03-23T09:52:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.674+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 6 ==== osd_op_reply(25 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.674+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:26 17.3a 17:5c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59256c00 con 0x55fa59df1c00 2024-03-23T09:52:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.674+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.1 v2:172.21.15.47:6824/37680 3 ==== osd_op_reply(26 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59df1c00 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.674+0000 7f867452b640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55fa59de0000 0x55fa5921ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.674+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:27 17.6 17:60000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59df1800 con 0x55fa59de0000 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.674+0000 7f8678533640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55fa59de0000 0x55fa5921ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.675+0000 7f8678533640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55fa59de0000 0x55fa5921ec00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.675+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.6 v2:172.21.15.175:6808/38160 1 ==== osd_op_reply(27 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0000 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.675+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:28 17.26 17:64000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59256000 con 0x55fa5923a400 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.675+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 7 ==== osd_op_reply(28 [pgnls start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.675+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:29 17.16 17:68000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59ca9400 con 0x55fa59256800 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.676+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.5 v2:172.21.15.175:6816/38161 2 ==== osd_op_reply(29 [pgnls start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa59256800 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.676+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:30 17.36 17:6c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59256400 con 0x55fa5923a400 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.676+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 8 ==== osd_op_reply(30 [pgnls start_epoch 118 out=81b] v116'33 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.676+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:31 17.36 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b400 con 0x55fa5923a400 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.676+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 9 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.676+0000 7f867ae38740 10 datascan.operator(): handling object 1099511628279.0 2024-03-23T09:52:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.676+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:32 17.36 17:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b000 con 0x55fa5923a400 2024-03-23T09:52:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.677+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 10 ==== osd_op_reply(32 100000001f7.00000000 [call] v118'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.677+0000 7f867ae38740 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55fa5923bc00 0x55fa58f5cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.677+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:33 17.e 17:70000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59de1c00 con 0x55fa5923bc00 2024-03-23T09:52:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.677+0000 7f8679535640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55fa5923bc00 0x55fa58f5cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.678+0000 7f8679535640 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55fa5923bc00 0x55fa58f5cc00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.678+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.2 v2:172.21.15.47:6800/37678 1 ==== osd_op_reply(33 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa5923bc00 2024-03-23T09:52:25.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.678+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:34 17.2e 17:74000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59257400 con 0x55fa59de0000 2024-03-23T09:52:25.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.678+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.6 v2:172.21.15.175:6808/38160 2 ==== osd_op_reply(34 [pgnls start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0000 2024-03-23T09:52:25.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.678+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:35 17.1e 17:78000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b800 con 0x55fa59df1c00 2024-03-23T09:52:25.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.678+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.1 v2:172.21.15.47:6824/37680 4 ==== osd_op_reply(35 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59df1c00 2024-03-23T09:52:25.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.678+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:36 17.3e 17:7c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59257800 con 0x55fa59de0400 2024-03-23T09:52:25.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.679+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.4 v2:172.21.15.175:6800/38159 3 ==== osd_op_reply(36 [pgnls start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59de0400 2024-03-23T09:52:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.679+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:37 17.1 17:80000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923a000 con 0x55fa5923a400 2024-03-23T09:52:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.679+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 11 ==== osd_op_reply(37 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.679+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:38 17.21 17:84000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59257c00 con 0x55fa5923a400 2024-03-23T09:52:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.679+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 12 ==== osd_op_reply(38 [pgnls start_epoch 118 out=49b] v116'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.679+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:39 17.11 17:88000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923ac00 con 0x55fa5923a800 2024-03-23T09:52:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.680+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.3 v2:172.21.15.47:6808/37683 9 ==== osd_op_reply(39 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa5923a800 2024-03-23T09:52:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.680+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:40 17.31 17:8c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b400 con 0x55fa59df1c00 2024-03-23T09:52:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.680+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.1 v2:172.21.15.47:6824/37680 5 ==== osd_op_reply(40 [pgnls start_epoch 118 out=81b] v116'15 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59df1c00 2024-03-23T09:52:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.680+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:41 17.31 17:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59de1800 con 0x55fa59df1c00 2024-03-23T09:52:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.680+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.1 v2:172.21.15.47:6824/37680 6 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59df1c00 2024-03-23T09:52:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.680+0000 7f867ae38740 10 datascan.operator(): handling object 1099511627780.1 2024-03-23T09:52:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.680+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:42 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59de1400 con 0x55fa59de0000 2024-03-23T09:52:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.681+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.6 v2:172.21.15.175:6808/38160 3 ==== osd_op_reply(42 10000000004.00000000 [call] v118'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0000 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.682+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b800 con 0x55fa5923a800 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.682+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.3 v2:172.21.15.47:6808/37683 10 ==== osd_op_reply(43 [pgnls start_epoch 118 out=49b] v116'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa5923a800 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.682+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b000 con 0x55fa59de0400 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.682+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.4 v2:172.21.15.175:6800/38159 4 ==== osd_op_reply(44 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59de0400 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.682+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59256000 con 0x55fa59256800 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.683+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.5 v2:172.21.15.175:6816/38161 3 ==== osd_op_reply(45 [pgnls start_epoch 118 out=81b] v117'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa59256800 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.683+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:46 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b400 con 0x55fa59256800 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.683+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.5 v2:172.21.15.175:6816/38161 4 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa59256800 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.683+0000 7f867ae38740 10 datascan.operator(): handling object 1099511628280.0 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.683+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:47 17.19 17:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59256c00 con 0x55fa59256800 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.684+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.5 v2:172.21.15.175:6816/38161 5 ==== osd_op_reply(47 100000001f8.00000000 [call] v118'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa59256800 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.684+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:48 17.39 17:9c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59257800 con 0x55fa59256800 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.685+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.5 v2:172.21.15.175:6816/38161 6 ==== osd_op_reply(48 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa59256800 2024-03-23T09:52:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.685+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:49 17.5 17:a0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59256400 con 0x55fa59de0000 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.685+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.6 v2:172.21.15.175:6808/38160 4 ==== osd_op_reply(49 [pgnls start_epoch 118 out=81b] v118'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0000 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.685+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:50 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59de1c00 con 0x55fa59de0000 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.686+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.6 v2:172.21.15.175:6808/38160 5 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0000 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.686+0000 7f867ae38740 10 datascan.operator(): handling object 1099511627780.0 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.686+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:51 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923ac00 con 0x55fa59de0000 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.687+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.6 v2:172.21.15.175:6808/38160 6 ==== osd_op_reply(51 10000000004.00000000 [call] v118'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0000 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.687+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:52 17.25 17:a4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b800 con 0x55fa59df1c00 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.687+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.1 v2:172.21.15.47:6824/37680 7 ==== osd_op_reply(52 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59df1c00 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.687+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59df1800 con 0x55fa59256800 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.687+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.5 v2:172.21.15.175:6816/38161 7 ==== osd_op_reply(53 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa59256800 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.687+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59274000 con 0x55fa59de0400 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.688+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.4 v2:172.21.15.175:6800/38159 5 ==== osd_op_reply(54 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59de0400 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.688+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59276000 con 0x55fa59de0800 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.688+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.7 v2:172.21.15.175:6824/38174 7 ==== osd_op_reply(55 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0800 2024-03-23T09:52:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.688+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59274400 con 0x55fa59de0800 2024-03-23T09:52:25.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.689+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.7 v2:172.21.15.175:6824/38174 8 ==== osd_op_reply(56 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0800 2024-03-23T09:52:25.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.689+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59276400 con 0x55fa59de0800 2024-03-23T09:52:25.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.689+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.7 v2:172.21.15.175:6824/38174 9 ==== osd_op_reply(57 [pgnls start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0800 2024-03-23T09:52:25.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.689+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59274800 con 0x55fa59de0800 2024-03-23T09:52:25.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.689+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.7 v2:172.21.15.175:6824/38174 10 ==== osd_op_reply(58 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0800 2024-03-23T09:52:25.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.689+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59276800 con 0x55fa59de0400 2024-03-23T09:52:25.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.689+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.4 v2:172.21.15.175:6800/38159 6 ==== osd_op_reply(59 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59de0400 2024-03-23T09:52:25.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.689+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59274c00 con 0x55fa59de0400 2024-03-23T09:52:25.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.690+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.4 v2:172.21.15.175:6800/38159 7 ==== osd_op_reply(60 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59de0400 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.690+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59276c00 con 0x55fa5923a800 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.690+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.3 v2:172.21.15.47:6808/37683 11 ==== osd_op_reply(61 [pgnls start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa5923a800 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.690+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59275000 con 0x55fa59de0800 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.690+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.7 v2:172.21.15.175:6824/38174 11 ==== osd_op_reply(62 [pgnls start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0800 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.690+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59277000 con 0x55fa5923a400 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.691+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 13 ==== osd_op_reply(63 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.691+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59275400 con 0x55fa59de0400 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.691+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.4 v2:172.21.15.175:6800/38159 8 ==== osd_op_reply(64 [pgnls start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59de0400 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.691+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59277400 con 0x55fa59de0400 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.691+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.4 v2:172.21.15.175:6800/38159 9 ==== osd_op_reply(65 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59de0400 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.691+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59275800 con 0x55fa5923bc00 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.691+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.2 v2:172.21.15.47:6800/37678 2 ==== osd_op_reply(66 [pgnls start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa5923bc00 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.691+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59277800 con 0x55fa5923a400 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.692+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 14 ==== osd_op_reply(67 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.692+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59275c00 con 0x55fa59de0400 2024-03-23T09:52:25.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.692+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.4 v2:172.21.15.175:6800/38159 10 ==== osd_op_reply(68 [pgnls start_epoch 118 out=49b] v116'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59de0400 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.692+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b000 con 0x55fa59de0000 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.693+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.6 v2:172.21.15.175:6808/38160 7 ==== osd_op_reply(69 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0000 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.693+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59276800 con 0x55fa59256800 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.693+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 <== osd.5 v2:172.21.15.175:6816/38161 8 ==== osd_op_reply(70 [pgnls start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cfc0 con 0x55fa59256800 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.693+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa5923b800 con 0x55fa59df1c00 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.693+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.1 v2:172.21.15.47:6824/37680 8 ==== osd_op_reply(71 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59df1c00 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.693+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59276000 con 0x55fa59de0400 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f8679535640 1 -- 172.21.15.47:0/2103910002 <== osd.4 v2:172.21.15.175:6800/38159 11 ==== osd_op_reply(72 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cd80 con 0x55fa59de0400 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867452b640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59de1800 con 0x55fa59de0000 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.6 v2:172.21.15.175:6808/38160 8 ==== osd_op_reply(73 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa59de0000 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f8673d2a640 1 -- 172.21.15.47:0/2103910002 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x55fa59277c00 con 0x55fa5923a400 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f8678533640 1 -- 172.21.15.47:0/2103910002 <== osd.0 v2:172.21.15.47:6816/37679 15 ==== osd_op_reply(74 [pgnls start_epoch 118 out=49b] v116'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fa59d8cb40 con 0x55fa5923a400 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55fa5923a400 msgr2=0x55fa5921d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x55fa5923a400 0x55fa5921d080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55fa59df1c00 msgr2=0x55fa5921c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x55fa59df1c00 0x55fa5921c580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55fa5923bc00 msgr2=0x55fa58f5cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x55fa5923bc00 0x55fa58f5cc00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55fa5923a800 msgr2=0x55fa5921cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x55fa5923a800 0x55fa5921cb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 reap_dead start 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55fa59de0400 msgr2=0x55fa5921d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x55fa59de0400 0x55fa5921d600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 reap_dead start 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55fa59256800 msgr2=0x55fa5921db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x55fa59256800 0x55fa5921db80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55fa59de0000 msgr2=0x55fa5921ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x55fa59de0000 0x55fa5921ec00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55fa59de0800 msgr2=0x55fa58f5c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x55fa59de0800 0x55fa58f5c680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55fa59df0c00 msgr2=0x55fa5921c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x55fa59df0c00 0x55fa5921c000 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x55fa58fbc6f0 tx=0x55fa59ca7da0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa59ca9800 msgr2=0x55fa58f5ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f867ae38740 1 --2- 172.21.15.47:0/2103910002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa59ca9800 0x55fa58f5ab00 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x55fa59ca56b0 tx=0x55fa59ca6c00 comp rx=0 tx=0).stop 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.694+0000 7f8678d34640 1 -- 172.21.15.47:0/2103910002 reap_dead start 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.695+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 shutdown_connections 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.695+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 >> 172.21.15.47:0/2103910002 conn(0x55fa59ca9c00 msgr2=0x55fa58fa9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.695+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 shutdown_connections 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.695+0000 7f867ae38740 1 -- 172.21.15.47:0/2103910002 wait complete. 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.695+0000 7f867ae38740 1 -- 172.21.15.47:0/2695123303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa58ec1800 msgr2=0x55fa58f5b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.695+0000 7f867ae38740 1 --2- 172.21.15.47:0/2695123303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa58ec1800 0x55fa58f5b600 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x55fa59ceb2c0 tx=0x55fa59cccc60 comp rx=0 tx=0).stop 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.695+0000 7f867ae38740 1 -- 172.21.15.47:0/2695123303 shutdown_connections 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.695+0000 7f867ae38740 1 --2- 172.21.15.47:0/2695123303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fa58ec1c00 0x55fa58f5b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.695+0000 7f867ae38740 1 --2- 172.21.15.47:0/2695123303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa58ec1800 0x55fa58f5b600 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.695+0000 7f867ae38740 1 --2- 172.21.15.47:0/2695123303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa58ec0c00 0x55fa58f5bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.695+0000 7f867ae38740 1 -- 172.21.15.47:0/2695123303 >> 172.21.15.47:0/2695123303 conn(0x55fa58ec1000 msgr2=0x55fa58fa8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.695+0000 7f867ae38740 1 -- 172.21.15.47:0/2695123303 shutdown_connections 2024-03-23T09:52:25.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.695+0000 7f867ae38740 1 -- 172.21.15.47:0/2695123303 wait complete. 2024-03-23T09:52:25.704 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.125815, result: 2024-03-23T09:52:25.704 DEBUG:teuthology.orchestra.run.smithi047:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-03-23T09:52:25.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.778+0000 7fd475039740 1 -- start start 2024-03-23T09:52:25.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.778+0000 7fd475039740 1 Processor -- start 2024-03-23T09:52:25.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.779+0000 7fd475039740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7054c00 0x5569d70efb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.779+0000 7fd475039740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7055800 0x5569d70ef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.779+0000 7fd475039740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7055c00 0x5569d70ef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.779+0000 7fd475039740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d70e6640 con 0x5569d7055c00 2024-03-23T09:52:25.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.779+0000 7fd475039740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d70e6500 con 0x5569d7055800 2024-03-23T09:52:25.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.779+0000 7fd475039740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569d70e63c0 con 0x5569d7054c00 2024-03-23T09:52:25.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.779+0000 7fd473877640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7055c00 0x5569d70ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.779+0000 7fd473877640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7055c00 0x5569d70ef080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39874/0 (socket says 172.21.15.47:39874) 2024-03-23T09:52:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.779+0000 7fd473877640 1 -- 172.21.15.47:0/4063087514 learned_addr learned my addr 172.21.15.47:0/4063087514 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.779+0000 7fd473076640 1 --2- 172.21.15.47:0/4063087514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7054c00 0x5569d70efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.779+0000 7fd472875640 1 --2- 172.21.15.47:0/4063087514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7055800 0x5569d70ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.779+0000 7fd473877640 1 -- 172.21.15.47:0/4063087514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7054c00 msgr2=0x5569d70efb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd473877640 1 --2- 172.21.15.47:0/4063087514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7054c00 0x5569d70efb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd473877640 1 -- 172.21.15.47:0/4063087514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7055800 msgr2=0x5569d70ef600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd473877640 1 --2- 172.21.15.47:0/4063087514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7055800 0x5569d70ef600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd473877640 1 -- 172.21.15.47:0/4063087514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569d707a9c0 con 0x5569d7055c00 2024-03-23T09:52:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd473877640 1 --2- 172.21.15.47:0/4063087514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7055c00 0x5569d70ef080 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x5569d7e75320 tx=0x5569d7e86c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f90264b43461321 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd470871640 1 -- 172.21.15.47:0/4063087514 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569d7117600 con 0x5569d7055c00 2024-03-23T09:52:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd470871640 1 -- 172.21.15.47:0/4063087514 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5569d7f1e000 con 0x5569d7055c00 2024-03-23T09:52:25.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd475039740 1 -- 172.21.15.47:0/4063087514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569d707ab60 con 0x5569d7055c00 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd475039740 1 -- 172.21.15.47:0/4063087514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5569d707a820 con 0x5569d7055c00 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd475039740 4 waiting for MDS map... 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd470871640 1 -- 172.21.15.47:0/4063087514 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569d71171e0 con 0x5569d7055c00 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd470871640 1 -- 172.21.15.47:0/4063087514 <== mon.0 v2:172.21.15.47:3300/0 4 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x5569d709cc40 con 0x5569d7055c00 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd470871640 1 -- 172.21.15.47:0/4063087514 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 87) v1 ==== 2573+0+0 (secure 0 0 0) 0x5569d70a5800 con 0x5569d7055c00 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd475039740 4 Got MDS map 87 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd475039740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd475039740 4 datascan.main: Using metadata pool output 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.780+0000 7fd475039740 4 datascan.main: connecting to RADOS... 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.781+0000 7fd475039740 1 Processor -- start 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.781+0000 7fd475039740 1 -- start start 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.781+0000 7fd475039740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7f29000 0x5569d70eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.781+0000 7fd475039740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7f29400 0x5569d70ee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.781+0000 7fd475039740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7f29800 0x5569d70f0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.781+0000 7fd475039740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d70e6280 con 0x5569d7f29800 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.781+0000 7fd475039740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d70e6500 con 0x5569d7f29400 2024-03-23T09:52:25.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.781+0000 7fd475039740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569d70e6140 con 0x5569d7f29000 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd472875640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7f29800 0x5569d70f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd472875640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7f29800 0x5569d70f0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39888/0 (socket says 172.21.15.47:39888) 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd472875640 1 -- 172.21.15.47:0/166953980 learned_addr learned my addr 172.21.15.47:0/166953980 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd472875640 1 --2- 172.21.15.47:0/166953980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7f29000 0x5569d70eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd473076640 1 --2- 172.21.15.47:0/166953980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7f29400 0x5569d70ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd472875640 1 -- 172.21.15.47:0/166953980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7f29000 msgr2=0x5569d70eeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd472875640 1 --2- 172.21.15.47:0/166953980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7f29000 0x5569d70eeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd472875640 1 -- 172.21.15.47:0/166953980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7f29400 msgr2=0x5569d70ee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd472875640 1 --2- 172.21.15.47:0/166953980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7f29400 0x5569d70ee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd472875640 1 -- 172.21.15.47:0/166953980 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569d707a4e0 con 0x5569d7f29800 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd472875640 1 --2- 172.21.15.47:0/166953980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7f29800 0x5569d70f0100 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x5569d7f7c660 tx=0x5569d7f3acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d19738aef7df3de server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd46e86d640 1 -- 172.21.15.47:0/166953980 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569d7117080 con 0x5569d7f29800 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd46e86d640 1 -- 172.21.15.47:0/166953980 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5569d7f1e480 con 0x5569d7f29800 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd46e86d640 1 -- 172.21.15.47:0/166953980 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569d7117600 con 0x5569d7f29800 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd475039740 1 -- 172.21.15.47:0/166953980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7f29800 msgr2=0x5569d70f0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.782+0000 7fd475039740 1 --2- 172.21.15.47:0/166953980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7f29800 0x5569d70f0100 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x5569d7f7c660 tx=0x5569d7f3acc0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.783+0000 7fd475039740 1 -- 172.21.15.47:0/166953980 shutdown_connections 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.783+0000 7fd475039740 1 --2- 172.21.15.47:0/166953980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7f29800 0x5569d70f0100 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.783+0000 7fd475039740 1 --2- 172.21.15.47:0/166953980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7f29400 0x5569d70ee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.783+0000 7fd475039740 1 --2- 172.21.15.47:0/166953980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7f29000 0x5569d70eeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.783+0000 7fd475039740 1 -- 172.21.15.47:0/166953980 >> 172.21.15.47:0/166953980 conn(0x5569d7f28c00 msgr2=0x5569d713d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.783+0000 7fd475039740 1 -- 172.21.15.47:0/166953980 shutdown_connections 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.783+0000 7fd475039740 1 -- 172.21.15.47:0/166953980 wait complete. 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.783+0000 7fd475039740 1 Processor -- start 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.783+0000 7fd475039740 1 -- start start 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd475039740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7f29000 0x5569d70eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd475039740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7f29400 0x5569d70ee580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd475039740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7f29800 0x5569d70f0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd475039740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d70e6000 con 0x5569d7f29000 2024-03-23T09:52:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd475039740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d70e7040 con 0x5569d7f29800 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd475039740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569d70e7180 con 0x5569d7f29400 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd472875640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7f29000 0x5569d70eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd472875640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7f29000 0x5569d70eeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39894/0 (socket says 172.21.15.47:39894) 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 learned_addr learned my addr 172.21.15.47:0/630479540 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd472875640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7f29800 0x5569d70f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd473076640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7f29400 0x5569d70ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7f29400 msgr2=0x5569d70ee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd472875640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7f29400 0x5569d70ee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7f29800 msgr2=0x5569d70f0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd472875640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7f29800 0x5569d70f0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569d707ad00 con 0x5569d7f29000 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd473076640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7f29400 0x5569d70ee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd472875640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7f29000 0x5569d70eeb00 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x5569d7f7c990 tx=0x5569d7f3aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd46d06a640 1 -- 172.21.15.47:0/630479540 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569d71171e0 con 0x5569d7f29000 2024-03-23T09:52:25.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd46d06a640 1 -- 172.21.15.47:0/630479540 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5569d7f1e180 con 0x5569d7f29000 2024-03-23T09:52:25.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5569d707a4e0 con 0x5569d7f29000 2024-03-23T09:52:25.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd46d06a640 1 -- 172.21.15.47:0/630479540 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569d7116dc0 con 0x5569d7f29000 2024-03-23T09:52:25.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569d707a9c0 con 0x5569d7f29000 2024-03-23T09:52:25.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.784+0000 7fd475039740 4 datascan.init: resolving metadata pool 16 2024-03-23T09:52:25.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.786+0000 7fd46d06a640 1 -- 172.21.15.47:0/630479540 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x5569d70bb080 con 0x5569d7f29000 2024-03-23T09:52:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.786+0000 7fd46d06a640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x5569d7f29c00 0x5569d73be000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.786+0000 7fd46d06a640 1 -- 172.21.15.47:0/630479540 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x5569d709ca80 con 0x5569d7f29000 2024-03-23T09:52:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.786+0000 7fd475039740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-23T09:52:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.786+0000 7fd475039740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-03-23T09:52:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.786+0000 7fd475039740 4 datascan.main: opening data pool 'cephfs_data' 2024-03-23T09:52:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.786+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x5569d73c8000 0x5569d70f0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.786+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c8400 con 0x5569d73c8000 2024-03-23T09:52:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.786+0000 7fd473076640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x5569d7f29c00 0x5569d73be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.786+0000 7fd472875640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x5569d73c8000 0x5569d70f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.787+0000 7fd472875640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x5569d73c8000 0x5569d70f0680 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.787+0000 7fd473076640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x5569d7f29c00 0x5569d73be000 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x5569d7150780 tx=0x5569d7f3ade0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.787+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv35 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.787+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x5569d73c8800 0x5569d70f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.787+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c8c00 con 0x5569d73c8800 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.787+0000 7fd473877640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x5569d73c8800 0x5569d70f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.788+0000 7fd473877640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x5569d73c8800 0x5569d70f0c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.788+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.5 v2:172.21.15.175:6816/38161 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73c8800 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.788+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x5569d73c9000 0x5569d70f1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.788+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d6000 con 0x5569d73c9000 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.788+0000 7fd473076640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x5569d73c9000 0x5569d70f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.789+0000 7fd473076640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x5569d73c9000 0x5569d70f1180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.789+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.789+0000 7fd46e06c640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x5569d73d7800 0x5569d73be580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.789+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d7c00 con 0x5569d73d7800 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.789+0000 7fd472875640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x5569d73d7800 0x5569d73be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.789+0000 7fd472875640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x5569d73d7800 0x5569d73be580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.790+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73d7800 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.790+0000 7fd46e86d640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x5569d73e4000 0x5569d73beb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.790+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e4400 con 0x5569d73e4000 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.790+0000 7fd473877640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x5569d73e4000 0x5569d73beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.790+0000 7fd473877640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x5569d73e4000 0x5569d73beb00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.790+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.790+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e4c00 con 0x5569d73c8000 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.791+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.791+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e4800 con 0x5569d73c9000 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.791+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.791+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5000 con 0x5569d73e4000 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.791+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.792+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c8c00 con 0x5569d73c8000 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.792+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.792+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5400 con 0x5569d73c9000 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.792+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.792+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d7f28800 con 0x5569d73c8000 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.792+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.792+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c9c00 con 0x5569d73e4000 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.793+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.793+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d7f28400 con 0x5569d73e4000 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.793+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 4 ==== osd_op_reply(13 [pgnls-filter start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.793+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d6000 con 0x5569d73c8000 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.793+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.793+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e4800 con 0x5569d73c9000 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.793+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 4 ==== osd_op_reply(15 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.793+0000 7fd46e06c640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x5569d73d7c00 0x5569d73bf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.794+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d7e35400 con 0x5569d73d7c00 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.794+0000 7fd473076640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x5569d73d7c00 0x5569d73bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.794+0000 7fd473076640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x5569d73d7c00 0x5569d73bf080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.794+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.4 v2:172.21.15.175:6800/38159 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73d7c00 2024-03-23T09:52:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.794+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5400 con 0x5569d73d7800 2024-03-23T09:52:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.795+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.1 v2:172.21.15.47:6824/37680 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73d7800 2024-03-23T09:52:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.795+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e4c00 con 0x5569d73c8000 2024-03-23T09:52:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.795+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 6 ==== osd_op_reply(18 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.795+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c8400 con 0x5569d73d7c00 2024-03-23T09:52:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.795+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.4 v2:172.21.15.175:6800/38159 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73d7c00 2024-03-23T09:52:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.795+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e4400 con 0x5569d73c8800 2024-03-23T09:52:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.796+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.5 v2:172.21.15.175:6816/38161 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73c8800 2024-03-23T09:52:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.796+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c8c00 con 0x5569d73e4000 2024-03-23T09:52:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.796+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 118 out=81b] v118'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.796+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d6400 con 0x5569d73c9000 2024-03-23T09:52:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.796+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.796+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f8000 con 0x5569d73d7800 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.797+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.1 v2:172.21.15.47:6824/37680 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73d7800 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.797+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5000 con 0x5569d73e4000 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.797+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.797+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f8400 con 0x5569d73c8000 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.797+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.797+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5800 con 0x5569d73c9000 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.797+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 6 ==== osd_op_reply(26 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.797+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f8800 con 0x5569d73e4000 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.798+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 7 ==== osd_op_reply(27 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.798+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5c00 con 0x5569d73c8000 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.798+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 8 ==== osd_op_reply(28 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.798+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f8c00 con 0x5569d73c9000 2024-03-23T09:52:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.798+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 7 ==== osd_op_reply(29 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.798+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d7f28400 con 0x5569d73c8000 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.798+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 9 ==== osd_op_reply(30 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.798+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f9000 con 0x5569d73e4000 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.799+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 8 ==== osd_op_reply(31 [pgnls-filter start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.799+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d7f28800 con 0x5569d73e4000 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.799+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 9 ==== osd_op_reply(32 [pgnls-filter start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.799+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e4400 con 0x5569d73c8000 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.799+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 10 ==== osd_op_reply(33 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.799+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c9c00 con 0x5569d73c9000 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.799+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 8 ==== osd_op_reply(34 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.800+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5000 con 0x5569d73d7c00 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.800+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.4 v2:172.21.15.175:6800/38159 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73d7c00 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.800+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c8c00 con 0x5569d73d7800 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.800+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.1 v2:172.21.15.47:6824/37680 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73d7800 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.800+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e4800 con 0x5569d73c8000 2024-03-23T09:52:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.800+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 11 ==== osd_op_reply(37 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.800+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c8400 con 0x5569d73d7c00 2024-03-23T09:52:25.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.801+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.4 v2:172.21.15.175:6800/38159 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73d7c00 2024-03-23T09:52:25.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.801+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5800 con 0x5569d73c8800 2024-03-23T09:52:25.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.801+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.5 v2:172.21.15.175:6816/38161 3 ==== osd_op_reply(39 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73c8800 2024-03-23T09:52:25.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.801+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d7e35400 con 0x5569d73e4000 2024-03-23T09:52:25.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.801+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 118 out=49b] v118'23 uv23 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.801+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d6400 con 0x5569d73c9000 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.802+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 9 ==== osd_op_reply(41 [pgnls-filter start_epoch 118 out=49b] v116'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.802+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:42 17.a 17:50000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f8800 con 0x5569d73c9000 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.802+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 10 ==== osd_op_reply(42 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.802+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:43 17.2a 17:54000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d6000 con 0x5569d73e4000 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.802+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 11 ==== osd_op_reply(43 [pgnls-filter start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.802+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:44 17.1a 17:58000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f8000 con 0x5569d73c8000 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.802+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 12 ==== osd_op_reply(44 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.802+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:45 17.3a 17:5c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f8c00 con 0x5569d73d7800 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.803+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.1 v2:172.21.15.47:6824/37680 5 ==== osd_op_reply(45 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73d7800 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.803+0000 7fd46e06c640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x5569d73f8400 0x5569d73bf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.803+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:46 17.6 17:60000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f9400 con 0x5569d73f8400 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.803+0000 7fd472875640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x5569d73f8400 0x5569d73bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.803+0000 7fd472875640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x5569d73f8400 0x5569d73bf600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.803+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.6 v2:172.21.15.175:6808/38160 1 ==== osd_op_reply(46 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73f8400 2024-03-23T09:52:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.804+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:47 17.26 17:64000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f8800 con 0x5569d73c8000 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.804+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 13 ==== osd_op_reply(47 [pgnls-filter start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.804+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:48 17.16 17:68000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f9800 con 0x5569d73c8800 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.804+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.5 v2:172.21.15.175:6816/38161 4 ==== osd_op_reply(48 [pgnls-filter start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73c8800 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.804+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:49 17.36 17:6c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f9c00 con 0x5569d73c8000 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.804+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 14 ==== osd_op_reply(49 [pgnls-filter start_epoch 118 out=81b] v118'34 uv34 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.804+0000 7fd475039740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.804+0000 7fd475039740 10 datascan.operator(): handling object 100000001f7.0 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.804+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:50 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d6800 con 0x5569d73c8000 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.805+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 15 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) v8 ==== 416+0+194 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.805+0000 7fd475039740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.805+0000 7fd475039740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.805+0000 7fd475039740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.805+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:51 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d6c00 con 0x5569d73e4000 2024-03-23T09:52:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.805+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 12 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.805+0000 7fd475039740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.805+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x5569d73d7000 0x5569d73bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.805+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:52 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d7400 con 0x5569d73d7000 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.805+0000 7fd473877640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x5569d73d7000 0x5569d73bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd473877640 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x5569d73d7000 0x5569d73bfb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.2 v2:172.21.15.47:6800/37678 1 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+524 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73d7000 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd475039740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd475039740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd475039740 20 datascan.get_frag_of: frag is * 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:53 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5800 con 0x5569d73e4000 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 13 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd475039740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd475039740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:54 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c8c00 con 0x5569d73e4000 2024-03-23T09:52:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 14 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd475039740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd475039740 20 datascan.read_dentry: bravo_head not found in result 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd475039740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.806+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:55 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d7f28800 con 0x5569d73e4000 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.807+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 15 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v118'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.807+0000 7fd475039740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.807+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d7f28400 con 0x5569d73d7000 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.808+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.2 v2:172.21.15.47:6800/37678 2 ==== osd_op_reply(56 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73d7000 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.808+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d6400 con 0x5569d73f8400 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.808+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.6 v2:172.21.15.175:6808/38160 2 ==== osd_op_reply(57 [pgnls-filter start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73f8400 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.808+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5400 con 0x5569d73d7800 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.808+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.1 v2:172.21.15.47:6824/37680 6 ==== osd_op_reply(58 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73d7800 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.808+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c9c00 con 0x5569d73d7c00 2024-03-23T09:52:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.809+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.4 v2:172.21.15.175:6800/38159 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73d7c00 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.809+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e4c00 con 0x5569d73c8000 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.809+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 16 ==== osd_op_reply(60 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.809+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c8400 con 0x5569d73c8000 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.809+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 17 ==== osd_op_reply(61 [pgnls-filter start_epoch 118 out=49b] v116'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.809+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5400 con 0x5569d73e4000 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.809+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 16 ==== osd_op_reply(62 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.809+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5000 con 0x5569d73d7800 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.810+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.1 v2:172.21.15.47:6824/37680 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 118 out=49b] v116'15 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73d7800 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.810+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e4c00 con 0x5569d73e4000 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.810+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 118 out=49b] v116'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.810+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f9400 con 0x5569d73d7c00 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.810+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.4 v2:172.21.15.175:6800/38159 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73d7c00 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.810+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5c00 con 0x5569d73c8800 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.811+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.5 v2:172.21.15.175:6816/38161 5 ==== osd_op_reply(66 [pgnls-filter start_epoch 118 out=49b] v118'24 uv24 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73c8800 2024-03-23T09:52:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.811+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f8800 con 0x5569d73c8800 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.811+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.5 v2:172.21.15.175:6816/38161 6 ==== osd_op_reply(67 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73c8800 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.811+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e4400 con 0x5569d73f8400 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.812+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.6 v2:172.21.15.175:6808/38160 3 ==== osd_op_reply(68 [pgnls-filter start_epoch 118 out=49b] v118'15 uv15 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73f8400 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.812+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f9c00 con 0x5569d73d7800 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.812+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.1 v2:172.21.15.47:6824/37680 8 ==== osd_op_reply(69 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73d7800 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.812+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e4800 con 0x5569d73c8800 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.812+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.5 v2:172.21.15.175:6816/38161 7 ==== osd_op_reply(70 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73c8800 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.812+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f9000 con 0x5569d73d7c00 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.813+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.4 v2:172.21.15.175:6800/38159 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73d7c00 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.813+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c8400 con 0x5569d73c9000 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.813+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.813+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f9800 con 0x5569d73c9000 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.813+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.813+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d6800 con 0x5569d73c9000 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.814+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 13 ==== osd_op_reply(74 [pgnls-filter start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.814+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f8000 con 0x5569d73c9000 2024-03-23T09:52:25.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.814+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.814+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d6400 con 0x5569d73d7c00 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.814+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.4 v2:172.21.15.175:6800/38159 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73d7c00 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.814+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f8c00 con 0x5569d73d7c00 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.815+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.4 v2:172.21.15.175:6800/38159 9 ==== osd_op_reply(77 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73d7c00 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.815+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c9c00 con 0x5569d73e4000 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.815+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.3 v2:172.21.15.47:6808/37683 18 ==== osd_op_reply(78 [pgnls-filter start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73e4000 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.815+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5800 con 0x5569d73c9000 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.815+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.7 v2:172.21.15.175:6824/38174 15 ==== osd_op_reply(79 [pgnls-filter start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73c9000 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.815+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c8400 con 0x5569d73c8000 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.816+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 18 ==== osd_op_reply(80 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.816+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5400 con 0x5569d73d7c00 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.816+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.4 v2:172.21.15.175:6800/38159 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73d7c00 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.816+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73c8c00 con 0x5569d73d7c00 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.816+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.4 v2:172.21.15.175:6800/38159 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73d7c00 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.816+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e4c00 con 0x5569d73d7000 2024-03-23T09:52:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.816+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.2 v2:172.21.15.47:6800/37678 3 ==== osd_op_reply(83 [pgnls-filter start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73d7000 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.816+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d6800 con 0x5569d73c8000 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.817+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 19 ==== osd_op_reply(84 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.817+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73e5c00 con 0x5569d73d7c00 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.817+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.4 v2:172.21.15.175:6800/38159 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 118 out=49b] v116'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73d7c00 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.817+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73d7400 con 0x5569d73f8400 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.817+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.6 v2:172.21.15.175:6808/38160 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73f8400 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.817+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d7f28800 con 0x5569d73c8800 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.818+0000 7fd473877640 1 -- 172.21.15.47:0/630479540 <== osd.5 v2:172.21.15.175:6816/38161 8 ==== osd_op_reply(87 [pgnls-filter start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18d80 con 0x5569d73c8800 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.818+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f9400 con 0x5569d73d7800 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.818+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.1 v2:172.21.15.47:6824/37680 9 ==== osd_op_reply(88 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73d7800 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.818+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d7f28400 con 0x5569d73d7c00 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.818+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 <== osd.4 v2:172.21.15.175:6800/38159 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18fc0 con 0x5569d73d7c00 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.818+0000 7fd46e06c640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d73f9000 con 0x5569d73f8400 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.6 v2:172.21.15.175:6808/38160 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73f8400 2024-03-23T09:52:25.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd46e86d640 1 -- 172.21.15.47:0/630479540 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5569d7414000 con 0x5569d73c8000 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd472875640 1 -- 172.21.15.47:0/630479540 <== osd.0 v2:172.21.15.47:6816/37679 20 ==== osd_op_reply(91 [pgnls-filter start_epoch 118 out=49b] v116'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5569d7f18b40 con 0x5569d73c8000 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x5569d73c8000 msgr2=0x5569d70f0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x5569d73c8000 0x5569d70f0680 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x5569d73d7800 msgr2=0x5569d73be580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x5569d73d7800 0x5569d73be580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x5569d73d7000 msgr2=0x5569d73bfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x5569d73d7000 0x5569d73bfb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x5569d73e4000 msgr2=0x5569d73beb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x5569d73e4000 0x5569d73beb00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 reap_dead start 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x5569d73d7c00 msgr2=0x5569d73bf080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x5569d73d7c00 0x5569d73bf080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 reap_dead start 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x5569d73c8800 msgr2=0x5569d70f0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x5569d73c8800 0x5569d70f0c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x5569d73f8400 msgr2=0x5569d73bf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x5569d73f8400 0x5569d73bf600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x5569d73c9000 msgr2=0x5569d70f1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x5569d73c9000 0x5569d70f1180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x5569d7f29c00 msgr2=0x5569d73be000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x5569d7f29c00 0x5569d73be000 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x5569d7150780 tx=0x5569d7f3ade0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7f29000 msgr2=0x5569d70eeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 --2- 172.21.15.47:0/630479540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7f29000 0x5569d70eeb00 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x5569d7f7c990 tx=0x5569d7f3aea0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd473076640 1 -- 172.21.15.47:0/630479540 reap_dead start 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 shutdown_connections 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 >> 172.21.15.47:0/630479540 conn(0x5569d7f28c00 msgr2=0x5569d713d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 shutdown_connections 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.819+0000 7fd475039740 1 -- 172.21.15.47:0/630479540 wait complete. 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.820+0000 7fd475039740 1 -- 172.21.15.47:0/4063087514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7055c00 msgr2=0x5569d70ef080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.820+0000 7fd475039740 1 --2- 172.21.15.47:0/4063087514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7055c00 0x5569d70ef080 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x5569d7e75320 tx=0x5569d7e86c60 comp rx=0 tx=0).stop 2024-03-23T09:52:25.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.820+0000 7fd475039740 1 -- 172.21.15.47:0/4063087514 shutdown_connections 2024-03-23T09:52:25.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.820+0000 7fd475039740 1 --2- 172.21.15.47:0/4063087514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569d7055c00 0x5569d70ef080 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.820+0000 7fd475039740 1 --2- 172.21.15.47:0/4063087514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569d7055800 0x5569d70ef600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.820+0000 7fd475039740 1 --2- 172.21.15.47:0/4063087514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5569d7054c00 0x5569d70efb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.820+0000 7fd475039740 1 -- 172.21.15.47:0/4063087514 >> 172.21.15.47:0/4063087514 conn(0x5569d7055000 msgr2=0x5569d713c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.820+0000 7fd475039740 1 -- 172.21.15.47:0/4063087514 shutdown_connections 2024-03-23T09:52:25.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.820+0000 7fd475039740 1 -- 172.21.15.47:0/4063087514 wait complete. 2024-03-23T09:52:25.834 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.129626, result: 2024-03-23T09:52:25.834 DEBUG:teuthology.orchestra.run.smithi047:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-03-23T09:52:25.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.909+0000 7f97a9848740 1 -- start start 2024-03-23T09:52:25.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.909+0000 7f97a9848740 1 Processor -- start 2024-03-23T09:52:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a9848740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558514e0c00 0x55585157a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a9848740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558514e1800 0x55585157ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a9848740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558514e1c00 0x55585157b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a9848740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555851572a00 con 0x5558514e0c00 2024-03-23T09:52:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a9848740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555851572780 con 0x5558514e1c00 2024-03-23T09:52:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a9848740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x555851572640 con 0x5558514e1800 2024-03-23T09:52:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a77de640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558514e0c00 0x55585157a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a77de640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558514e0c00 0x55585157a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39926/0 (socket says 172.21.15.47:39926) 2024-03-23T09:52:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a77de640 1 -- 172.21.15.47:0/149063338 learned_addr learned my addr 172.21.15.47:0/149063338 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/149063338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558514e1800 0x55585157ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a77de640 1 -- 172.21.15.47:0/149063338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558514e1800 msgr2=0x55585157ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/149063338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558514e1c00 0x55585157b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a77de640 1 --2- 172.21.15.47:0/149063338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558514e1800 0x55585157ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a77de640 1 -- 172.21.15.47:0/149063338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558514e1c00 msgr2=0x55585157b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a77de640 1 --2- 172.21.15.47:0/149063338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558514e1c00 0x55585157b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a77de640 1 -- 172.21.15.47:0/149063338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558515069c0 con 0x5558514e0c00 2024-03-23T09:52:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/149063338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558514e1800 0x55585157ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:25.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.910+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/149063338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558514e1c00 0x55585157b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.911+0000 7f97a77de640 1 --2- 172.21.15.47:0/149063338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558514e0c00 0x55585157a580 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x5558522f5320 tx=0x5558523a6060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6c56fb5bd48a1e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.911+0000 7f97a4fd9640 1 -- 172.21.15.47:0/149063338 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558515a3600 con 0x5558514e0c00 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.911+0000 7f97a4fd9640 1 -- 172.21.15.47:0/149063338 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55585153e480 con 0x5558514e0c00 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.911+0000 7f97a4fd9640 1 -- 172.21.15.47:0/149063338 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558515a31e0 con 0x5558514e0c00 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.911+0000 7f97a9848740 1 -- 172.21.15.47:0/149063338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555851506b60 con 0x5558514e0c00 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.911+0000 7f97a9848740 1 -- 172.21.15.47:0/149063338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555851506820 con 0x5558514e0c00 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.911+0000 7f97a9848740 4 waiting for MDS map... 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.911+0000 7f97a4fd9640 1 -- 172.21.15.47:0/149063338 <== mon.0 v2:172.21.15.47:3300/0 4 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x555851528c40 con 0x5558514e0c00 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.911+0000 7f97a4fd9640 1 -- 172.21.15.47:0/149063338 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 87) v1 ==== 2573+0+0 (secure 0 0 0) 0x555851531800 con 0x5558514e0c00 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.911+0000 7f97a9848740 4 Got MDS map 87 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.911+0000 7f97a9848740 4 datascan.main: Using metadata pool output 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.911+0000 7f97a9848740 4 datascan.main: connecting to RADOS... 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.912+0000 7f97a9848740 1 Processor -- start 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.912+0000 7f97a9848740 1 -- start start 2024-03-23T09:52:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.912+0000 7f97a9848740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558523b1000 0x55585157b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.912+0000 7f97a9848740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558523b1400 0x55585157bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.912+0000 7f97a9848740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558523b1800 0x55585157c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.912+0000 7f97a9848740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585150fe00 con 0x5558523b1800 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.912+0000 7f97a9848740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555851572780 con 0x5558523b1000 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.912+0000 7f97a9848740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x555851572500 con 0x5558523b1400 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.912+0000 7f97a6fdd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558523b1800 0x55585157c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.912+0000 7f97a6fdd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558523b1800 0x55585157c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39928/0 (socket says 172.21.15.47:39928) 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.912+0000 7f97a6fdd640 1 -- 172.21.15.47:0/1629790902 learned_addr learned my addr 172.21.15.47:0/1629790902 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.912+0000 7f97a77de640 1 --2- 172.21.15.47:0/1629790902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558523b1400 0x55585157bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.912+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/1629790902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558523b1000 0x55585157b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a6fdd640 1 -- 172.21.15.47:0/1629790902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558523b1400 msgr2=0x55585157bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/1629790902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558523b1400 0x55585157bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a6fdd640 1 -- 172.21.15.47:0/1629790902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558523b1000 msgr2=0x55585157b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/1629790902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558523b1000 0x55585157b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a6fdd640 1 -- 172.21.15.47:0/1629790902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558515064e0 con 0x5558523b1800 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a77de640 1 --2- 172.21.15.47:0/1629790902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558523b1400 0x55585157bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/1629790902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558523b1000 0x55585157b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/1629790902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558523b1800 0x55585157c100 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x5558523c48a0 tx=0x5558523c6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef06d05394b1e8b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a2fd5640 1 -- 172.21.15.47:0/1629790902 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558515a3600 con 0x5558523b1800 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a2fd5640 1 -- 172.21.15.47:0/1629790902 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55585153e900 con 0x5558523b1800 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a2fd5640 1 -- 172.21.15.47:0/1629790902 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558515a31e0 con 0x5558523b1800 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a9848740 1 -- 172.21.15.47:0/1629790902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558523b1800 msgr2=0x55585157c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a9848740 1 --2- 172.21.15.47:0/1629790902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558523b1800 0x55585157c100 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x5558523c48a0 tx=0x5558523c6cc0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a9848740 1 -- 172.21.15.47:0/1629790902 shutdown_connections 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a9848740 1 --2- 172.21.15.47:0/1629790902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558523b1800 0x55585157c100 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a9848740 1 --2- 172.21.15.47:0/1629790902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558523b1400 0x55585157bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a9848740 1 --2- 172.21.15.47:0/1629790902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558523b1000 0x55585157b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a9848740 1 -- 172.21.15.47:0/1629790902 >> 172.21.15.47:0/1629790902 conn(0x5558523b0c00 msgr2=0x5558515c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a9848740 1 -- 172.21.15.47:0/1629790902 shutdown_connections 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.913+0000 7f97a9848740 1 -- 172.21.15.47:0/1629790902 wait complete. 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.914+0000 7f97a9848740 1 Processor -- start 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.914+0000 7f97a9848740 1 -- start start 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.914+0000 7f97a9848740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558523b1000 0x55585157d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.914+0000 7f97a9848740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558523b1400 0x55585157cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.914+0000 7f97a9848740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558523b1800 0x55585157c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.914+0000 7f97a9848740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558515723c0 con 0x5558523b1000 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.914+0000 7f97a9848740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555851572280 con 0x5558523b1400 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.914+0000 7f97a9848740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x555851572140 con 0x5558523b1800 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a7fdf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558523b1000 0x55585157d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a7fdf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558523b1000 0x55585157d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39940/0 (socket says 172.21.15.47:39940) 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 learned_addr learned my addr 172.21.15.47:0/756972849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a77de640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558523b1400 0x55585157cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558523b1800 0x55585157c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558523b1800 msgr2=0x55585157c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558523b1800 0x55585157c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558523b1400 msgr2=0x55585157cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558523b1400 0x55585157cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558515069c0 con 0x5558523b1000 2024-03-23T09:52:25.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558523b1000 0x55585157d180 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x5558522f4a80 tx=0x5558523c6e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a17d2640 1 -- 172.21.15.47:0/756972849 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558515a3080 con 0x5558523b1000 2024-03-23T09:52:25.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a17d2640 1 -- 172.21.15.47:0/756972849 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5558522b2300 con 0x5558523b1000 2024-03-23T09:52:25.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555851506b60 con 0x5558523b1000 2024-03-23T09:52:25.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a17d2640 1 -- 172.21.15.47:0/756972849 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558515a3600 con 0x5558523b1000 2024-03-23T09:52:25.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558515061a0 con 0x5558523b1000 2024-03-23T09:52:25.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.915+0000 7f97a9848740 4 datascan.init: resolving metadata pool 16 2024-03-23T09:52:25.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.916+0000 7f97a17d2640 1 -- 172.21.15.47:0/756972849 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x555851547340 con 0x5558523b1000 2024-03-23T09:52:25.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.917+0000 7f97a17d2640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x555852424800 0x55585157d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.917+0000 7f97a17d2640 1 -- 172.21.15.47:0/756972849 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x555851528a80 con 0x5558523b1000 2024-03-23T09:52:25.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.917+0000 7f97a9848740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.917+0000 7f97a9848740 4 datascan.main: resolving metadata pool 16 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.917+0000 7f97a9848740 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x555852424c00 0x55585157c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.917+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852425000 con 0x555852424c00 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.917+0000 7f97a77de640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x555852424800 0x55585157d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.917+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x555852424c00 0x55585157c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.917+0000 7f97a77de640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x555852424800 0x55585157d700 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x5558523c4090 tx=0x555852277140 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.917+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x555852424c00 0x55585157c100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.918+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 1 ==== osd_op_reply(1 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.918+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x555852424400 0x55585184e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.918+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852424000 con 0x555852424400 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.918+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x555852424400 0x55585184e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.918+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x555852424400 0x55585184e000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.918+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 1 ==== osd_op_reply(2 [pgnls start_epoch 118 out=177b] v117'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.918+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852424000 con 0x555852424c00 2024-03-23T09:52:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.919+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 2 ==== osd_op_reply(3 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.919+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852424000 con 0x555852424c00 2024-03-23T09:52:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.919+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 3 ==== osd_op_reply(4 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.919+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x555852424000 0x55585184e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.919+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852425000 con 0x555852424000 2024-03-23T09:52:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.919+0000 7f97a77de640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x555852424000 0x55585184e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.919+0000 7f97a77de640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x555852424000 0x55585184e580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.920+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 1 ==== osd_op_reply(5 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.920+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852425000 con 0x555852424c00 2024-03-23T09:52:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.920+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 4 ==== osd_op_reply(6 [pgnls start_epoch 118 out=149b] v118'41 uv41 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.920+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x555852425000 0x55585184eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.920+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5558523b1c00 con 0x555852425000 2024-03-23T09:52:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.920+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x555852425000 0x55585184eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.921+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x555852425000 0x55585184eb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.921+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.5 v2:172.21.15.175:6816/38161 1 ==== osd_op_reply(7 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852425000 2024-03-23T09:52:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.921+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5558523b1c00 con 0x555852424400 2024-03-23T09:52:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.921+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 2 ==== osd_op_reply(8 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.921+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x5558523b1c00 0x55585184f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.921+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x5558522af400 con 0x5558523b1c00 2024-03-23T09:52:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.922+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x5558523b1c00 0x55585184f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.922+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x5558523b1c00 0x55585184f080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.922+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.4 v2:172.21.15.175:6800/38159 1 ==== osd_op_reply(9 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396d80 con 0x5558523b1c00 2024-03-23T09:52:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.922+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x5558522af400 0x55585184f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.922+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872000 con 0x5558522af400 2024-03-23T09:52:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.922+0000 7f97a77de640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x5558522af400 0x55585184f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.923+0000 7f97a77de640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x5558522af400 0x55585184f600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.923+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.6 v2:172.21.15.175:6808/38160 1 ==== osd_op_reply(10 [pgnls start_epoch 118 out=81b] v117'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555852396fc0 con 0x5558522af400 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.923+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872000 con 0x555852424400 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.923+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 3 ==== osd_op_reply(11 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.924+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872000 con 0x555852424000 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.924+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 2 ==== osd_op_reply(12 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.924+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872000 con 0x5558522af400 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.924+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.6 v2:172.21.15.175:6808/38160 2 ==== osd_op_reply(13 [pgnls start_epoch 118 out=77b] v117'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x555852396fc0 con 0x5558522af400 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.924+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872000 con 0x555852424000 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.924+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 3 ==== osd_op_reply(14 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.924+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872000 con 0x555852424400 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.925+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 4 ==== osd_op_reply(15 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.925+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x555851872000 0x55585184fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.925+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872400 con 0x555851872000 2024-03-23T09:52:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.925+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x555851872000 0x55585184fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.925+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x555851872000 0x55585184fb80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.926+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.7 v2:172.21.15.175:6824/38174 1 ==== osd_op_reply(16 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555851872000 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.926+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872400 con 0x555852424400 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.926+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 5 ==== osd_op_reply(17 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.926+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872400 con 0x555852425000 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.926+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.5 v2:172.21.15.175:6816/38161 2 ==== osd_op_reply(18 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852425000 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.926+0000 7f97a6fdd640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x555851872400 0x555851850100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.926+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x555851872400 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.926+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x555851872400 0x555851850100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.927+0000 7f97a7fdf640 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x555851872400 0x555851850100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.927+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== osd_op_reply(19 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396d80 con 0x555851872400 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.927+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x5558523b1c00 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.927+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.4 v2:172.21.15.175:6800/38159 2 ==== osd_op_reply(20 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396d80 con 0x5558523b1c00 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.927+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x555852424c00 2024-03-23T09:52:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.927+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 5 ==== osd_op_reply(21 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.927+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x555852424c00 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.928+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 6 ==== osd_op_reply(22 [pgnls start_epoch 118 out=97b] v117'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.928+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x5558522af400 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.928+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.6 v2:172.21.15.175:6808/38160 3 ==== osd_op_reply(23 [pgnls start_epoch 118 out=74b] v113'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x555852396fc0 con 0x5558522af400 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.928+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x555852424400 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.928+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 6 ==== osd_op_reply(24 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.928+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x555852425000 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.929+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.5 v2:172.21.15.175:6816/38161 3 ==== osd_op_reply(25 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852425000 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.929+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x555851872400 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.929+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.1 v2:172.21.15.47:6824/37680 2 ==== osd_op_reply(26 [pgnls start_epoch 118 out=98b] v117'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x555852396d80 con 0x555851872400 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.929+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x555851872400 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.929+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.1 v2:172.21.15.47:6824/37680 3 ==== osd_op_reply(27 [pgnls start_epoch 118 out=76b] v118'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x555852396d80 con 0x555851872400 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.929+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x555852425000 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.929+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.5 v2:172.21.15.175:6816/38161 4 ==== osd_op_reply(28 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852425000 2024-03-23T09:52:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.929+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x555852424000 2024-03-23T09:52:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.930+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 4 ==== osd_op_reply(29 [pgnls start_epoch 118 out=105b] v118'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.930+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x555852424400 2024-03-23T09:52:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.930+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 7 ==== osd_op_reply(30 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.930+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x555852424c00 2024-03-23T09:52:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.930+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 7 ==== osd_op_reply(31 [pgnls start_epoch 118 out=73b] v118'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.930+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x555852425000 2024-03-23T09:52:25.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.931+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.5 v2:172.21.15.175:6816/38161 5 ==== osd_op_reply(32 [pgnls start_epoch 118 out=101b] v118'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x555852396b40 con 0x555852425000 2024-03-23T09:52:25.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.931+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-03-23T09:52:25.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.931+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852425400 con 0x555852424400 2024-03-23T09:52:25.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.931+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.931+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-03-23T09:52:25.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.931+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852425800 con 0x555852424400 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.931+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.931+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.931+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852425c00 con 0x555852424400 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187e000 con 0x555852424400 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187e400 con 0x555852424400 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-03-23T09:52:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187e800 con 0x555852424c00 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 8 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.932+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187ec00 con 0x555852424400 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.933+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.933+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.933+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:40 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187f000 con 0x5558522af400 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.933+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.6 v2:172.21.15.175:6808/38160 4 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x555852396fc0 con 0x5558522af400 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.933+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.933+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.933+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-03-23T09:52:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.933+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:41 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187f400 con 0x555852424400 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.933+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.933+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.933+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187e000 con 0x555852424c00 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187f800 con 0x555852424400 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-23T09:52:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852425400 con 0x555851872400 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.1 v2:172.21.15.47:6824/37680 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x555852396d80 con 0x555851872400 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.934+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852425800 con 0x555852424000 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.935+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 5 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.935+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.935+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852425c00 con 0x555852424000 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.935+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 6 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.935+0000 7f97a9848740 20 datascan.scan_links: injected ino detected 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.935+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.935+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.935+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-03-23T09:52:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.935+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.935+0000 7f97a9848740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.935+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872c00 con 0x555852425000 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.936+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.5 v2:172.21.15.175:6816/38161 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x555852396b40 con 0x555852425000 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.936+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424c00 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.936+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 10 ==== osd_op_reply(48 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.936+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424400 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.936+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 16 ==== osd_op_reply(49 [pgnls start_epoch 118 out=177b] v117'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.936+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424c00 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.936+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 11 ==== osd_op_reply(50 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.936+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424c00 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.937+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 12 ==== osd_op_reply(51 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.937+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424000 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.937+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 7 ==== osd_op_reply(52 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.937+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424c00 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.937+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 13 ==== osd_op_reply(53 [pgnls start_epoch 118 out=149b] v118'41 uv41 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.937+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852425000 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.937+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.5 v2:172.21.15.175:6816/38161 7 ==== osd_op_reply(54 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852425000 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.937+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424400 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.938+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 17 ==== osd_op_reply(55 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.938+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x5558523b1c00 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.938+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.4 v2:172.21.15.175:6800/38159 3 ==== osd_op_reply(56 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396d80 con 0x5558523b1c00 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.938+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x5558522af400 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.938+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.6 v2:172.21.15.175:6808/38160 5 ==== osd_op_reply(57 [pgnls start_epoch 118 out=81b] v117'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555852396fc0 con 0x5558522af400 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.938+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424400 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.939+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 18 ==== osd_op_reply(58 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.939+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424000 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.939+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 8 ==== osd_op_reply(59 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.939+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x5558522af400 2024-03-23T09:52:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.939+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.6 v2:172.21.15.175:6808/38160 6 ==== osd_op_reply(60 [pgnls start_epoch 118 out=77b] v117'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x555852396fc0 con 0x5558522af400 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.939+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424000 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.940+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 9 ==== osd_op_reply(61 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.940+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424400 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.940+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 19 ==== osd_op_reply(62 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.940+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555851872000 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.940+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.7 v2:172.21.15.175:6824/38174 2 ==== osd_op_reply(63 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555851872000 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.940+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424400 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.940+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 20 ==== osd_op_reply(64 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.940+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852425000 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.941+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.5 v2:172.21.15.175:6816/38161 8 ==== osd_op_reply(65 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852425000 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.941+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555851872400 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.941+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.1 v2:172.21.15.47:6824/37680 5 ==== osd_op_reply(66 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396d80 con 0x555851872400 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.941+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x5558523b1c00 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.941+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.4 v2:172.21.15.175:6800/38159 4 ==== osd_op_reply(67 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396d80 con 0x5558523b1c00 2024-03-23T09:52:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.941+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424c00 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.941+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 14 ==== osd_op_reply(68 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.941+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424c00 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.942+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 15 ==== osd_op_reply(69 [pgnls start_epoch 118 out=97b] v117'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.942+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x5558522af400 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.942+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.6 v2:172.21.15.175:6808/38160 7 ==== osd_op_reply(70 [pgnls start_epoch 118 out=74b] v113'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x555852396fc0 con 0x5558522af400 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.942+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424400 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.942+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 21 ==== osd_op_reply(71 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.942+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852425000 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.943+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.5 v2:172.21.15.175:6816/38161 9 ==== osd_op_reply(72 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852425000 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.943+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555851872400 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.943+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.1 v2:172.21.15.47:6824/37680 6 ==== osd_op_reply(73 [pgnls start_epoch 118 out=98b] v117'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x555852396d80 con 0x555851872400 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.943+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555851872400 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.943+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.1 v2:172.21.15.47:6824/37680 7 ==== osd_op_reply(74 [pgnls start_epoch 118 out=76b] v118'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x555852396d80 con 0x555851872400 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.943+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852425000 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.944+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.5 v2:172.21.15.175:6816/38161 10 ==== osd_op_reply(75 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396b40 con 0x555852425000 2024-03-23T09:52:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.944+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424000 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.944+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 10 ==== osd_op_reply(76 [pgnls start_epoch 118 out=105b] v118'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.944+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424400 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.944+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 22 ==== osd_op_reply(77 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.944+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852424c00 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.944+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 16 ==== osd_op_reply(78 [pgnls start_epoch 118 out=73b] v118'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.944+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872800 con 0x555852425000 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.945+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.5 v2:172.21.15.175:6816/38161 11 ==== osd_op_reply(79 [pgnls start_epoch 118 out=101b] v118'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x555852396b40 con 0x555852425000 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.945+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.945+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187e800 con 0x555852424400 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.945+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 23 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.945+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.945+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187e000 con 0x555852424400 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.945+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 24 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.945+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-03-23T09:52:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.945+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187fc00 con 0x555852424400 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.945+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 25 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.945+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.945+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873c00 con 0x555852424400 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 26 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873800 con 0x555852424400 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 27 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873400 con 0x555852424c00 2024-03-23T09:52:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 17 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851873000 con 0x555852424400 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 28 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.946+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:87 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872c00 con 0x5558522af400 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.6 v2:172.21.15.175:6808/38160 8 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x555852396fc0 con 0x5558522af400 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:88 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187e400 con 0x555852424400 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 29 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187ec00 con 0x555852424c00 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.0 v2:172.21.15.47:6816/37679 18 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396b40 con 0x555852424c00 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-03-23T09:52:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.947+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187f400 con 0x555852424400 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.948+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.2 v2:172.21.15.47:6800/37678 30 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396d80 con 0x555852424400 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.948+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.948+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.948+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.948+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187f800 con 0x555851872400 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.948+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.1 v2:172.21.15.47:6824/37680 8 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x555852396d80 con 0x555851872400 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.948+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.948+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.948+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.948+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187e800 con 0x555852424000 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.948+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 11 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.948+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.948+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187e000 con 0x555852424000 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 12 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 20 datascan.scan_links: injected ino detected 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852425400 con 0x555852425000 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 <== osd.5 v2:172.21.15.175:6816/38161 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x555852396b40 con 0x555852425000 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-03-23T09:52:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 10 datascan.scan_links: processing 1 injected_inos 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187f000 con 0x555852424000 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 13 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 20 datascan.scan_links: first is now 2 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.949+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x55585187e800 con 0x555852424000 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.950+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.3 v2:172.21.15.47:6808/37683 14 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v118'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555852396fc0 con 0x555852424000 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.950+0000 7f97a9848740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.950+0000 7f97a9848740 10 datascan.scan_links: updating inotable 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.950+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555851872c00 con 0x555851872400 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.951+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.1 v2:172.21.15.47:6824/37680 9 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x555852396d80 con 0x555851872400 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.951+0000 7f97a9848740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.951+0000 7f97a9848740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-03-23T09:52:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.951+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- osd_op(unknown.0.0:98 16.13 16:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852425800 con 0x555851872400 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.952+0000 7f97a7fdf640 1 -- 172.21.15.47:0/756972849 <== osd.1 v2:172.21.15.47:6824/37680 10 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v118'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x555852396d80 con 0x555851872400 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.952+0000 7f97a9848740 10 datascan.scan_links: updating snaptable 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.952+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- osd_op(unknown.0.0:99 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) v8 -- 0x555852425c00 con 0x5558522af400 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a77de640 1 -- 172.21.15.47:0/756972849 <== osd.6 v2:172.21.15.175:6808/38160 9 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x555852396fc0 con 0x5558522af400 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x555852424c00 msgr2=0x55585157c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x555852424c00 0x55585157c100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x555851872400 msgr2=0x555851850100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x555851872400 0x555851850100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x555852424400 msgr2=0x55585184e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x555852424400 0x55585184e000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x555852424000 msgr2=0x55585184e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x555852424000 0x55585184e580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x5558523b1c00 msgr2=0x55585184f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 reap_dead start 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x5558523b1c00 0x55585184f080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x555852425000 msgr2=0x55585184eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x555852425000 0x55585184eb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x5558522af400 msgr2=0x55585184f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x5558522af400 0x55585184f600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x555851872000 msgr2=0x55585184fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x555851872000 0x55585184fb80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x555852424800 msgr2=0x55585157d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x555852424800 0x55585157d700 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x5558523c4090 tx=0x555852277140 comp rx=0 tx=0).stop 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558523b1000 msgr2=0x55585157d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 --2- 172.21.15.47:0/756972849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558523b1000 0x55585157d180 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x5558522f4a80 tx=0x5558523c6e40 comp rx=0 tx=0).stop 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 reap_dead start 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a6fdd640 1 -- 172.21.15.47:0/756972849 reap_dead start 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 shutdown_connections 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 >> 172.21.15.47:0/756972849 conn(0x5558523b0c00 msgr2=0x5558515c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 shutdown_connections 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/756972849 wait complete. 2024-03-23T09:52:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/149063338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558514e0c00 msgr2=0x55585157a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:25.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 --2- 172.21.15.47:0/149063338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558514e0c00 0x55585157a580 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x5558522f5320 tx=0x5558523a6060 comp rx=0 tx=0).stop 2024-03-23T09:52:25.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 -- 172.21.15.47:0/149063338 shutdown_connections 2024-03-23T09:52:25.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.953+0000 7f97a9848740 1 --2- 172.21.15.47:0/149063338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5558514e1c00 0x55585157b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.954+0000 7f97a9848740 1 --2- 172.21.15.47:0/149063338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5558514e1800 0x55585157ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.954+0000 7f97a9848740 1 --2- 172.21.15.47:0/149063338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558514e0c00 0x55585157a580 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:25.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.954+0000 7f97a9848740 1 -- 172.21.15.47:0/149063338 >> 172.21.15.47:0/149063338 conn(0x5558514e1000 msgr2=0x5558515c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:25.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.954+0000 7f97a9848740 1 -- 172.21.15.47:0/149063338 shutdown_connections 2024-03-23T09:52:25.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:25.954+0000 7f97a9848740 1 -- 172.21.15.47:0/149063338 wait complete. 2024-03-23T09:52:25.974 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.140070, result: 2024-03-23T09:52:25.975 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T09:52:26.000 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T09:52:26.048 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-03-23T09:52:26.048 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2024-03-23T09:52:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T09:52:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T09:52:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2024-03-23T09:52:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2024-03-23T09:52:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2024-03-23T09:52:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2024-03-23T09:52:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2024-03-23T09:52:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2024-03-23T09:52:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2024-03-23T09:52:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2024-03-23T09:52:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T09:52:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T09:52:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2024-03-23T09:52:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T09:52:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2024-03-23T09:52:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2024-03-23T09:52:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2024-03-23T09:52:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock profile = high_recovery_ops 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2024-03-23T09:52:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2024-03-23T09:52:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: bdev async discard = True 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: bdev enable discard = True 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore allocator = bitmap 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 1/20 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 1/20 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 4/10 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2024-03-23T09:52:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2024-03-23T09:52:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T09:52:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: # 1m isn't quite enough 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: mon_down_mkfs_grace = 2m 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_filestore_osds = false 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = "" 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds balancer = 20 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = False 2024-03-23T09:52:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2024-03-23T09:52:26.061 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = True 2024-03-23T09:52:26.061 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-03-23T09:52:26.061 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-03-23T09:52:26.061 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-03-23T09:52:26.061 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2024-03-23T09:52:26.061 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2024-03-23T09:52:26.061 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T09:52:26.061 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T09:52:26.062 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.063 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 600 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock profile = high_recovery_ops 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.064 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2024-03-23T09:52:26.065 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: bdev async discard = True 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: bdev enable discard = True 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore allocator = bitmap 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore block size = 96636764160 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore fsck on mount = True 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluefs = 1/20 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluestore = 1/20 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 20 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: debug rocksdb = 4/10 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd full ratio = 0.9 2024-03-23T09:52:26.066 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd nearfull ratio = 0.8 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: osd failsafe full ratio = 0.95 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = bluestore 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-03-23T09:52:26.067 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: # 1m isn't quite enough 2024-03-23T09:52:26.068 INFO:teuthology.orchestra.run.smithi175.stdout: mon_down_mkfs_grace = 2m 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_filestore_osds = false 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: mon op complaint time = 120 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: client mount timeout = 600 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: debug client = 20 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = "" 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-03-23T09:52:26.069 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout:[mds] 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds = 20 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds balancer = 20 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug frag = True 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug scatterstat = False 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout: mds op complaint time = 180 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout: mds verify scatter = True 2024-03-23T09:52:26.070 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-03-23T09:52:26.071 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-03-23T09:52:26.071 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-03-23T09:52:26.072 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T09:52:26.114 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T09:52:26.162 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-03-23T09:52:26.178 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2024-03-23T09:52:26.178 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T09:52:26.178 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2024-03-23T09:52:26.178 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.178 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2024-03-23T09:52:26.178 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.178 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2024-03-23T09:52:26.179 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.180 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2024-03-23T09:52:26.181 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock profile = high_recovery_ops 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.182 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: bdev async discard = True 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: bdev enable discard = True 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore allocator = bitmap 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 1/20 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 1/20 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 4/10 2024-03-23T09:52:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2024-03-23T09:52:26.184 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T09:52:26.185 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: # 1m isn't quite enough 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: mon_down_mkfs_grace = 2m 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_filestore_osds = false 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2024-03-23T09:52:26.186 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = "" 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds balancer = 20 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2024-03-23T09:52:26.187 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = True 2024-03-23T09:52:26.188 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2024-03-23T09:52:26.188 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = True 2024-03-23T09:52:26.188 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-03-23T09:52:26.188 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-03-23T09:52:26.188 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-03-23T09:52:26.189 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2024-03-23T09:52:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2024-03-23T09:52:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T09:52:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2024-03-23T09:52:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T09:52:26.190 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2024-03-23T09:52:26.191 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = f31804be-685c-4b3b-b2be-155afa9f5d13 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 600 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock profile = high_recovery_ops 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2024-03-23T09:52:26.193 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: bdev async discard = True 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: bdev enable discard = True 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore allocator = bitmap 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore block size = 96636764160 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore fsck on mount = True 2024-03-23T09:52:26.194 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluefs = 1/20 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluestore = 1/20 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 20 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: debug rocksdb = 4/10 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd full ratio = 0.9 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd nearfull ratio = 0.8 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: osd failsafe full ratio = 0.95 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = bluestore 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2024-03-23T09:52:26.195 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2024-03-23T09:52:26.196 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: # 1m isn't quite enough 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: mon_down_mkfs_grace = 2m 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_filestore_osds = false 2024-03-23T09:52:26.197 INFO:teuthology.orchestra.run.smithi175.stdout: mon op complaint time = 120 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout: client mount timeout = 600 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout: debug client = 20 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = "" 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2024-03-23T09:52:26.198 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout:[mds] 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds = 20 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds balancer = 20 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug frag = True 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug scatterstat = True 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout: mds op complaint time = 180 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout: mds verify scatter = True 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-03-23T09:52:26.199 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-03-23T09:52:26.200 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-23T09:52:26.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.331+0000 7fa97a6d5640 1 Processor -- start 2024-03-23T09:52:26.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.331+0000 7fa97a6d5640 1 -- start start 2024-03-23T09:52:26.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa97a6d5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa97415c990 0x7fa974160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:26.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa97a6d5640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa974161630 con 0x7fa9740cde10 2024-03-23T09:52:26.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa97a6d5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9740d1030 con 0x7fa9740b8350 2024-03-23T09:52:26.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa97a6d5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9740cfe10 con 0x7fa97415c990 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa978c72640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa9740cde10 0x7fa9740d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56776/0 (socket says 172.21.15.47:56776) 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa978c72640 1 -- 172.21.15.47:0/3997946135 learned_addr learned my addr 172.21.15.47:0/3997946135 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa973fff640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa97415c990 0x7fa974160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749388688 0 0) 0x7fa974161630 con 0x7fa9740cde10 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa960003520 con 0x7fa9740cde10 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340117679 0 0) 0x7fa9740d1030 con 0x7fa9740b8350 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa974161630 con 0x7fa9740b8350 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4118587294 0 0) 0x7fa960003520 con 0x7fa9740cde10 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa97415c990 msgr2=0x7fa974160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa972ffd640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa97415c990 0x7fa974160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 >> v1:172.21.15.175:6789/0 conn(0x7fa9740b8350 legacy=0x7fa9740cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9740b5b60 con 0x7fa9740cde10 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fa96c0019f0 con 0x7fa9740cde10 2024-03-23T09:52:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.332+0000 7fa973fff640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa97415c990 0x7fa974160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 >> v1:172.21.15.47:6789/0 conn(0x7fa9740cde10 legacy=0x7fa9740d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa972ffd640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa960003b30 0x7fa960003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa972ffd640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa97415c990 0x7fa9600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa972ffd640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa960004d00 0x7fa960005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9740cfe10 con 0x7fa960004d00 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96000ba60 con 0x7fa960003b30 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96000bc20 con 0x7fa97415c990 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa973fff640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa960003b30 0x7fa960003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa978c72640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa960004d00 0x7fa960005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa9737fe640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa97415c990 0x7fa9600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa978c72640 1 -- 172.21.15.47:0/3997946135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa97415c990 msgr2=0x7fa9600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa978c72640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa97415c990 0x7fa9600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa978c72640 1 -- 172.21.15.47:0/3997946135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa960003b30 msgr2=0x7fa960003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa978c72640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa960003b30 0x7fa960003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa978c72640 1 -- 172.21.15.47:0/3997946135 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c009400 con 0x7fa960004d00 2024-03-23T09:52:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.333+0000 7fa973fff640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa960003b30 0x7fa960003f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:52:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.334+0000 7fa978c72640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa960004d00 0x7fa960005180 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fa97408bd50 tx=0x7fa9600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c290dcb261164b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.334+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c034630 con 0x7fa960004d00 2024-03-23T09:52:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.334+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c02f070 con 0x7fa960004d00 2024-03-23T09:52:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.334+0000 7fa972ffd640 1 -- 172.21.15.47:0/3997946135 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0349f0 con 0x7fa960004d00 2024-03-23T09:52:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.334+0000 7fa97a6d5640 1 -- 172.21.15.47:0/3997946135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa960004d00 msgr2=0x7fa960005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.334+0000 7fa97a6d5640 1 --2- 172.21.15.47:0/3997946135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa960004d00 0x7fa960005180 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fa97408bd50 tx=0x7fa9600057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.334+0000 7fa978c72640 1 -- 172.21.15.47:0/3997946135 reap_dead start 2024-03-23T09:52:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.334+0000 7fa97a6d5640 1 -- 172.21.15.47:0/3997946135 shutdown_connections 2024-03-23T09:52:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.334+0000 7fa97a6d5640 1 -- 172.21.15.47:0/3997946135 >> 172.21.15.47:0/3997946135 conn(0x7fa9740659d0 msgr2=0x7fa974065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.335+0000 7fa97a6d5640 1 -- 172.21.15.47:0/3997946135 shutdown_connections 2024-03-23T09:52:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.335+0000 7fa97a6d5640 1 -- 172.21.15.47:0/3997946135 wait complete. 2024-03-23T09:52:26.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.336+0000 7fa97a6d5640 1 Processor -- start 2024-03-23T09:52:26.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.336+0000 7fa97a6d5640 1 -- start start 2024-03-23T09:52:26.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.336+0000 7fa97a6d5640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa960004d00 0x7fa974158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:26.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa97a6d5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa97415ae60 0x7fa974156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:26.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa97a6d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa97415f8b0 0x7fa97415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:26.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa97a6d5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa974159a70 con 0x7fa97415ae60 2024-03-23T09:52:26.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa97a6d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa974160280 con 0x7fa97415f8b0 2024-03-23T09:52:26.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa97a6d5640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa974158df0 con 0x7fa960004d00 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa9737fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa97415ae60 0x7fa974156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa9737fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa97415ae60 0x7fa974156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39952/0 (socket says 172.21.15.47:39952) 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa973fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa960004d00 0x7fa974158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa9737fe640 1 -- 172.21.15.47:0/1150064227 learned_addr learned my addr 172.21.15.47:0/1150064227 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa978c72640 1 --2- 172.21.15.47:0/1150064227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa97415f8b0 0x7fa97415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa9737fe640 1 -- 172.21.15.47:0/1150064227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa960004d00 msgr2=0x7fa974158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa9737fe640 1 --2- 172.21.15.47:0/1150064227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa960004d00 0x7fa974158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa9737fe640 1 -- 172.21.15.47:0/1150064227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa97415f8b0 msgr2=0x7fa97415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa9737fe640 1 --2- 172.21.15.47:0/1150064227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa97415f8b0 0x7fa97415fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa9737fe640 1 -- 172.21.15.47:0/1150064227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9741590d0 con 0x7fa97415ae60 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa9737fe640 1 --2- 172.21.15.47:0/1150064227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa97415ae60 0x7fa974156080 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fa96400f7d0 tx=0x7fa96400fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa970ff9640 1 -- 172.21.15.47:0/1150064227 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96400fe80 con 0x7fa97415ae60 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa970ff9640 1 -- 172.21.15.47:0/1150064227 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9640052e0 con 0x7fa97415ae60 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa97a6d5640 1 -- 172.21.15.47:0/1150064227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9741593b0 con 0x7fa97415ae60 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa970ff9640 1 -- 172.21.15.47:0/1150064227 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96400d970 con 0x7fa97415ae60 2024-03-23T09:52:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.337+0000 7fa97a6d5640 1 -- 172.21.15.47:0/1150064227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97415b2e0 con 0x7fa97415ae60 2024-03-23T09:52:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.338+0000 7fa97a6d5640 1 -- 172.21.15.47:0/1150064227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa974072360 con 0x7fa97415ae60 2024-03-23T09:52:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.339+0000 7fa970ff9640 1 -- 172.21.15.47:0/1150064227 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa96400dad0 con 0x7fa97415ae60 2024-03-23T09:52:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.339+0000 7fa970ff9640 1 --2- 172.21.15.47:0/1150064227 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa95406ac00 0x7fa95406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.339+0000 7fa970ff9640 1 -- 172.21.15.47:0/1150064227 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa9640892c0 con 0x7fa97415ae60 2024-03-23T09:52:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.339+0000 7fa973fff640 1 --2- 172.21.15.47:0/1150064227 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa95406ac00 0x7fa95406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.340+0000 7fa973fff640 1 --2- 172.21.15.47:0/1150064227 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa95406ac00 0x7fa95406d0c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa9740b2bf0 tx=0x7fa95c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:26.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.344+0000 7fa970ff9640 1 -- 172.21.15.47:0/1150064227 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa9640544f0 con 0x7fa97415ae60 2024-03-23T09:52:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:26.563+0000 7fa97a6d5640 1 -- 172.21.15.47:0/1150064227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fa974066be0 con 0x7fa97415ae60 2024-03-23T09:52:27.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.501+0000 7fa970ff9640 1 -- 172.21.15.47:0/1150064227 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v89) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa964057b10 con 0x7fa97415ae60 2024-03-23T09:52:27.502 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-23T09:52:27.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.504+0000 7fa97a6d5640 1 -- 172.21.15.47:0/1150064227 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa95406ac00 msgr2=0x7fa95406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:27.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.504+0000 7fa97a6d5640 1 --2- 172.21.15.47:0/1150064227 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa95406ac00 0x7fa95406d0c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa9740b2bf0 tx=0x7fa95c009190 comp rx=0 tx=0).stop 2024-03-23T09:52:27.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.504+0000 7fa97a6d5640 1 -- 172.21.15.47:0/1150064227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa97415ae60 msgr2=0x7fa974156080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:27.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.504+0000 7fa97a6d5640 1 --2- 172.21.15.47:0/1150064227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa97415ae60 0x7fa974156080 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fa96400f7d0 tx=0x7fa96400fc90 comp rx=0 tx=0).stop 2024-03-23T09:52:27.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.505+0000 7fa97a6d5640 1 -- 172.21.15.47:0/1150064227 shutdown_connections 2024-03-23T09:52:27.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.505+0000 7fa97a6d5640 1 --2- 172.21.15.47:0/1150064227 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fa95406ac00 0x7fa95406d0c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.505+0000 7fa97a6d5640 1 --2- 172.21.15.47:0/1150064227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa97415f8b0 0x7fa97415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.505+0000 7fa97a6d5640 1 --2- 172.21.15.47:0/1150064227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa97415ae60 0x7fa974156080 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.505+0000 7fa97a6d5640 1 --2- 172.21.15.47:0/1150064227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa960004d00 0x7fa974158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.505+0000 7fa97a6d5640 1 -- 172.21.15.47:0/1150064227 >> 172.21.15.47:0/1150064227 conn(0x7fa9740659d0 msgr2=0x7fa97415b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:27.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.505+0000 7fa97a6d5640 1 -- 172.21.15.47:0/1150064227 shutdown_connections 2024-03-23T09:52:27.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.505+0000 7fa97a6d5640 1 -- 172.21.15.47:0/1150064227 wait complete. 2024-03-23T09:52:27.520 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:52:27.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.627+0000 7f7d5d116640 1 Processor -- start 2024-03-23T09:52:27.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.627+0000 7f7d5d116640 1 -- start start 2024-03-23T09:52:27.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d5d116640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5815ca90 0x7f7d58160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:27.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d5d116640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d580cde10 0x7f7d580d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:27.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d5d116640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d580d11e0 con 0x7f7d580b8350 2024-03-23T09:52:27.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d5d116640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d58160f60 con 0x7f7d580cde10 2024-03-23T09:52:27.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d5d116640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d58161240 con 0x7f7d5815ca90 2024-03-23T09:52:27.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d5759e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d580cde10 0x7f7d580d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:27.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d5759e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d580cde10 0x7f7d580d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39966/0 (socket says 172.21.15.47:39966) 2024-03-23T09:52:27.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d5759e640 1 -- 172.21.15.47:0/2113079183 learned_addr learned my addr 172.21.15.47:0/2113079183 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:27.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d5659c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7d580b8350 0x7f7d580cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56786/0 (socket says 172.21.15.47:56786) 2024-03-23T09:52:27.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d56d9d640 1 --2- 172.21.15.47:0/2113079183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5815ca90 0x7f7d58160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:27.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d5759e640 1 -- 172.21.15.47:0/2113079183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5815ca90 msgr2=0x7f7d58160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:27.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d5759e640 1 --2- 172.21.15.47:0/2113079183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5815ca90 0x7f7d58160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d5759e640 1 -- 172.21.15.47:0/2113079183 >> v1:172.21.15.47:6789/0 conn(0x7f7d580b8350 legacy=0x7f7d580cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:27.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d5759e640 1 -- 172.21.15.47:0/2113079183 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58161520 con 0x7f7d580cde10 2024-03-23T09:52:27.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.628+0000 7f7d55d9b640 1 -- 172.21.15.47:0/2113079183 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795847301 0 0) 0x7f7d580d11e0 con 0x7f7d580b8350 2024-03-23T09:52:27.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.629+0000 7f7d5759e640 1 --2- 172.21.15.47:0/2113079183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d580cde10 0x7f7d580d1670 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009e50 tx=0x7f7d5002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d61b3333cff3f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:27.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.629+0000 7f7d55d9b640 1 -- 172.21.15.47:0/2113079183 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5002ca70 con 0x7f7d580cde10 2024-03-23T09:52:27.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.629+0000 7f7d55d9b640 1 -- 172.21.15.47:0/2113079183 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5002cbd0 con 0x7f7d580cde10 2024-03-23T09:52:27.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.629+0000 7f7d55d9b640 1 -- 172.21.15.47:0/2113079183 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d500343f0 con 0x7f7d580cde10 2024-03-23T09:52:27.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.630+0000 7f7d5d116640 1 -- 172.21.15.47:0/2113079183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d580cde10 msgr2=0x7f7d580d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:27.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.630+0000 7f7d5d116640 1 --2- 172.21.15.47:0/2113079183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d580cde10 0x7f7d580d1670 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009e50 tx=0x7f7d5002bed0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.630+0000 7f7d5d116640 1 -- 172.21.15.47:0/2113079183 shutdown_connections 2024-03-23T09:52:27.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.630+0000 7f7d5d116640 1 --2- 172.21.15.47:0/2113079183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d580cde10 0x7f7d580d1670 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.630+0000 7f7d5d116640 1 --2- 172.21.15.47:0/2113079183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d5815ca90 0x7f7d58160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.630+0000 7f7d5d116640 1 -- 172.21.15.47:0/2113079183 >> 172.21.15.47:0/2113079183 conn(0x7f7d580659d0 msgr2=0x7f7d58065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:27.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.630+0000 7f7d5d116640 1 -- 172.21.15.47:0/2113079183 shutdown_connections 2024-03-23T09:52:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.630+0000 7f7d5d116640 1 -- 172.21.15.47:0/2113079183 wait complete. 2024-03-23T09:52:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.631+0000 7f7d5d116640 1 Processor -- start 2024-03-23T09:52:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.631+0000 7f7d5d116640 1 -- start start 2024-03-23T09:52:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.632+0000 7f7d5d116640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d580b8350 0x7f7d58149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.632+0000 7f7d5d116640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d580cde10 0x7f7d580b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.632+0000 7f7d5d116640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d5815ca90 0x7f7d580b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.632+0000 7f7d5d116640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5815a640 con 0x7f7d580b8350 2024-03-23T09:52:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.632+0000 7f7d5d116640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d581498a0 con 0x7f7d5815ca90 2024-03-23T09:52:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.632+0000 7f7d5d116640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d581595a0 con 0x7f7d580cde10 2024-03-23T09:52:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.632+0000 7f7d56d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d580b8350 0x7f7d58149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.632+0000 7f7d56d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d580b8350 0x7f7d58149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39970/0 (socket says 172.21.15.47:39970) 2024-03-23T09:52:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.632+0000 7f7d56d9d640 1 -- 172.21.15.47:0/2825783692 learned_addr learned my addr 172.21.15.47:0/2825783692 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.632+0000 7f7d5659c640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d580cde10 0x7f7d580b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.632+0000 7f7d5759e640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d5815ca90 0x7f7d580b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.633+0000 7f7d56d9d640 1 -- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d580cde10 msgr2=0x7f7d580b7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.633+0000 7f7d56d9d640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d580cde10 0x7f7d580b7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.633+0000 7f7d56d9d640 1 -- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d5815ca90 msgr2=0x7f7d580b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.633+0000 7f7d56d9d640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d5815ca90 0x7f7d580b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.633+0000 7f7d56d9d640 1 -- 172.21.15.47:0/2825783692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d50009b30 con 0x7f7d580b8350 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.633+0000 7f7d5659c640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d580cde10 0x7f7d580b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.633+0000 7f7d5759e640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d5815ca90 0x7f7d580b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.633+0000 7f7d56d9d640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d580b8350 0x7f7d58149360 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800d800 tx=0x7f7d4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.633+0000 7f7d3f7fe640 1 -- 172.21.15.47:0/2825783692 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4800b6c0 con 0x7f7d580b8350 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.633+0000 7f7d3f7fe640 1 -- 172.21.15.47:0/2825783692 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4800b820 con 0x7f7d580b8350 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.633+0000 7f7d3f7fe640 1 -- 172.21.15.47:0/2825783692 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4800bbb0 con 0x7f7d580b8350 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.633+0000 7f7d5d116640 1 -- 172.21.15.47:0/2825783692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d58159880 con 0x7f7d580b8350 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.633+0000 7f7d5d116640 1 -- 172.21.15.47:0/2825783692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d58159e40 con 0x7f7d580b8350 2024-03-23T09:52:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.634+0000 7f7d5d116640 1 -- 172.21.15.47:0/2825783692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d58072360 con 0x7f7d580b8350 2024-03-23T09:52:27.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.634+0000 7f7d3f7fe640 1 -- 172.21.15.47:0/2825783692 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7d4800bd10 con 0x7f7d580b8350 2024-03-23T09:52:27.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.635+0000 7f7d3f7fe640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7d4406ac00 0x7f7d4406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:27.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.635+0000 7f7d3f7fe640 1 -- 172.21.15.47:0/2825783692 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f7d480899b0 con 0x7f7d580b8350 2024-03-23T09:52:27.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.635+0000 7f7d5659c640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7d4406ac00 0x7f7d4406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:27.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.636+0000 7f7d5659c640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7d4406ac00 0x7f7d4406d0c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7d580b3ff0 tx=0x7f7d4c007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:27.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.639+0000 7f7d3f7fe640 1 -- 172.21.15.47:0/2825783692 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7d48054ab0 con 0x7f7d580b8350 2024-03-23T09:52:27.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.855+0000 7f7d5d116640 1 -- 172.21.15.47:0/2825783692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d58066be0 con 0x7f7d580b8350 2024-03-23T09:52:27.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.855+0000 7f7d3f7fe640 1 -- 172.21.15.47:0/2825783692 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f7d480580d0 con 0x7f7d580b8350 2024-03-23T09:52:27.856 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:27.856 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":90,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:51:40.395539+0000","modified":"2024-03-23T09:52:27.502548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"c","rank":0,"incarnation":90,"state":"up:replay","state_seq":1,"addr":"172.21.15.47:6834/3501786111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3501786111},{"type":"v1","addr":"172.21.15.47:6834","nonce":3501786111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-23T09:52:27.856 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 90 2024-03-23T09:52:27.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.857+0000 7f7d5d116640 1 -- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7d4406ac00 msgr2=0x7f7d4406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:27.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.857+0000 7f7d5d116640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7d4406ac00 0x7f7d4406d0c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7d580b3ff0 tx=0x7f7d4c007ca0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.857+0000 7f7d5d116640 1 -- 172.21.15.47:0/2825783692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d580b8350 msgr2=0x7f7d58149360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:27.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.857+0000 7f7d5d116640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d580b8350 0x7f7d58149360 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800d800 tx=0x7f7d4800dcc0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.857+0000 7f7d5d116640 1 -- 172.21.15.47:0/2825783692 shutdown_connections 2024-03-23T09:52:27.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.857+0000 7f7d5d116640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7d4406ac00 0x7f7d4406d0c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.857+0000 7f7d5d116640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d5815ca90 0x7f7d580b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.857+0000 7f7d5d116640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d580cde10 0x7f7d580b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.857+0000 7f7d5d116640 1 --2- 172.21.15.47:0/2825783692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d580b8350 0x7f7d58149360 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:27.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.857+0000 7f7d5d116640 1 -- 172.21.15.47:0/2825783692 >> 172.21.15.47:0/2825783692 conn(0x7f7d580659d0 msgr2=0x7f7d5815bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:27.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.857+0000 7f7d5d116640 1 -- 172.21.15.47:0/2825783692 shutdown_connections 2024-03-23T09:52:27.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:27.857+0000 7f7d5d116640 1 -- 172.21.15.47:0/2825783692 wait complete. 2024-03-23T09:52:28.271 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:51:40.395539+0000', 'modified': '2024-03-23T09:52:27.502548+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9762}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9762': {'gid': 9762, 'name': 'c', 'rank': 0, 'incarnation': 90, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.47:6834/3501786111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 3501786111}, {'type': 'v1', 'addr': '172.21.15.47:6834', 'nonce': 3501786111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:52:28.271 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9762:up:replay 2024-03-23T09:52:29.272 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:52:29.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.382+0000 7f1527a82640 1 Processor -- start 2024-03-23T09:52:29.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.382+0000 7f1527a82640 1 -- start start 2024-03-23T09:52:29.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.383+0000 7f1527a82640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f15200b8350 0x7f15200cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.383+0000 7f1527a82640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15200cdc40 0x7f15200d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.383+0000 7f1527a82640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15200ce090 con 0x7f152015ca90 2024-03-23T09:52:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.383+0000 7f1527a82640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15200d11e0 con 0x7f15200cdc40 2024-03-23T09:52:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.383+0000 7f1527a82640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1520160f60 con 0x7f15200b8350 2024-03-23T09:52:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.383+0000 7f152601f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15200cdc40 0x7f15200d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.383+0000 7f152601f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15200cdc40 0x7f15200d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39980/0 (socket says 172.21.15.47:39980) 2024-03-23T09:52:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.383+0000 7f152601f640 1 -- 172.21.15.47:0/3395915273 learned_addr learned my addr 172.21.15.47:0/3395915273 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.384+0000 7f152501d640 1 --2- 172.21.15.47:0/3395915273 >> v2:172.21.15.175:3300/0 conn(0x7f15200b8350 0x7f15200cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.384+0000 7f152601f640 1 -- 172.21.15.47:0/3395915273 >> v2:172.21.15.175:3300/0 conn(0x7f15200b8350 msgr2=0x7f15200cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.384+0000 7f152601f640 1 --2- 172.21.15.47:0/3395915273 >> v2:172.21.15.175:3300/0 conn(0x7f15200b8350 0x7f15200cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.384+0000 7f152601f640 1 -- 172.21.15.47:0/3395915273 >> v1:172.21.15.175:6789/0 conn(0x7f152015ca90 legacy=0x7f1520160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.384+0000 7f152601f640 1 -- 172.21.15.47:0/3395915273 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1520161240 con 0x7f15200cdc40 2024-03-23T09:52:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.384+0000 7f152501d640 1 --2- 172.21.15.47:0/3395915273 >> v2:172.21.15.175:3300/0 conn(0x7f15200b8350 0x7f15200cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:29.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.384+0000 7f152601f640 1 --2- 172.21.15.47:0/3395915273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15200cdc40 0x7f15200d1670 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f1520160670 tx=0x7f151802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7941f8fdd2ee97 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:29.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.385+0000 7f152481c640 1 -- 172.21.15.47:0/3395915273 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151802c9f0 con 0x7f15200cdc40 2024-03-23T09:52:29.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.385+0000 7f152481c640 1 -- 172.21.15.47:0/3395915273 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151802cb50 con 0x7f15200cdc40 2024-03-23T09:52:29.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.385+0000 7f152481c640 1 -- 172.21.15.47:0/3395915273 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15180343f0 con 0x7f15200cdc40 2024-03-23T09:52:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.385+0000 7f1527a82640 1 -- 172.21.15.47:0/3395915273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15200cdc40 msgr2=0x7f15200d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.385+0000 7f1527a82640 1 --2- 172.21.15.47:0/3395915273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15200cdc40 0x7f15200d1670 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f1520160670 tx=0x7f151802be50 comp rx=0 tx=0).stop 2024-03-23T09:52:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.385+0000 7f1527a82640 1 -- 172.21.15.47:0/3395915273 shutdown_connections 2024-03-23T09:52:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.385+0000 7f1527a82640 1 --2- 172.21.15.47:0/3395915273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15200cdc40 0x7f15200d1670 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.385+0000 7f1527a82640 1 --2- 172.21.15.47:0/3395915273 >> v2:172.21.15.175:3300/0 conn(0x7f15200b8350 0x7f15200cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.385+0000 7f1527a82640 1 -- 172.21.15.47:0/3395915273 >> 172.21.15.47:0/3395915273 conn(0x7f15200659d0 msgr2=0x7f1520065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.385+0000 7f1527a82640 1 -- 172.21.15.47:0/3395915273 shutdown_connections 2024-03-23T09:52:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.385+0000 7f1527a82640 1 -- 172.21.15.47:0/3395915273 wait complete. 2024-03-23T09:52:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.386+0000 7f1527a82640 1 Processor -- start 2024-03-23T09:52:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.386+0000 7f1527a82640 1 -- start start 2024-03-23T09:52:29.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.387+0000 7f1527a82640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200b8350 0x7f15200b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:29.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.387+0000 7f1527a82640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15200cdc40 0x7f15200b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:29.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.387+0000 7f1527a82640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f152015ca90 0x7f15200b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:29.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.387+0000 7f1527a82640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1520158870 con 0x7f15200cdc40 2024-03-23T09:52:29.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.387+0000 7f1527a82640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15200b28e0 con 0x7f152015ca90 2024-03-23T09:52:29.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.387+0000 7f1527a82640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15200b2bc0 con 0x7f15200b8350 2024-03-23T09:52:29.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.387+0000 7f152501d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15200cdc40 0x7f15200b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:29.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.387+0000 7f152501d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15200cdc40 0x7f15200b6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39996/0 (socket says 172.21.15.47:39996) 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.387+0000 7f152501d640 1 -- 172.21.15.47:0/1054389941 learned_addr learned my addr 172.21.15.47:0/1054389941 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f152601f640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f152015ca90 0x7f15200b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f152501d640 1 -- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200b8350 msgr2=0x7f15200b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f152581e640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200b8350 0x7f15200b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f152501d640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200b8350 0x7f15200b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f152501d640 1 -- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f152015ca90 msgr2=0x7f15200b7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f152501d640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f152015ca90 0x7f15200b7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f152501d640 1 -- 172.21.15.47:0/1054389941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518009b30 con 0x7f15200cdc40 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f152601f640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f152015ca90 0x7f15200b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f152581e640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200b8350 0x7f15200b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f152501d640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15200cdc40 0x7f15200b6ce0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f151000d7d0 tx=0x7f151000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f15167fc640 1 -- 172.21.15.47:0/1054389941 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510004030 con 0x7f15200cdc40 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f15167fc640 1 -- 172.21.15.47:0/1054389941 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1510004190 con 0x7f15200cdc40 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f1527a82640 1 -- 172.21.15.47:0/1054389941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15200b2ea0 con 0x7f15200cdc40 2024-03-23T09:52:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f15167fc640 1 -- 172.21.15.47:0/1054389941 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510004520 con 0x7f15200cdc40 2024-03-23T09:52:29.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.388+0000 7f1527a82640 1 -- 172.21.15.47:0/1054389941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152015b580 con 0x7f15200cdc40 2024-03-23T09:52:29.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.389+0000 7f1527a82640 1 -- 172.21.15.47:0/1054389941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1520072360 con 0x7f15200cdc40 2024-03-23T09:52:29.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.389+0000 7f15167fc640 1 -- 172.21.15.47:0/1054389941 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f15100096a0 con 0x7f15200cdc40 2024-03-23T09:52:29.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.390+0000 7f15167fc640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f150406ac00 0x7f150406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:29.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.390+0000 7f15167fc640 1 -- 172.21.15.47:0/1054389941 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f1510089640 con 0x7f15200cdc40 2024-03-23T09:52:29.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.390+0000 7f152581e640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f150406ac00 0x7f150406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:29.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.391+0000 7f152581e640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f150406ac00 0x7f150406d0c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f151c003eb0 tx=0x7f151c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:29.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.394+0000 7f15167fc640 1 -- 172.21.15.47:0/1054389941 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1510054730 con 0x7f15200cdc40 2024-03-23T09:52:29.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.615+0000 7f1527a82640 1 -- 172.21.15.47:0/1054389941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1520066be0 con 0x7f15200cdc40 2024-03-23T09:52:29.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.616+0000 7f15167fc640 1 -- 172.21.15.47:0/1054389941 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f1510057d50 con 0x7f15200cdc40 2024-03-23T09:52:29.616 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:29.616 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":92,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:51:40.395539+0000","modified":"2024-03-23T09:52:29.506769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"c","rank":0,"incarnation":90,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.47:6834/3501786111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3501786111},{"type":"v1","addr":"172.21.15.47:6834","nonce":3501786111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-23T09:52:29.616 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 92 2024-03-23T09:52:29.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.617+0000 7f1527a82640 1 -- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f150406ac00 msgr2=0x7f150406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:29.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.617+0000 7f1527a82640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f150406ac00 0x7f150406d0c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f151c003eb0 tx=0x7f151c009040 comp rx=0 tx=0).stop 2024-03-23T09:52:29.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.617+0000 7f1527a82640 1 -- 172.21.15.47:0/1054389941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15200cdc40 msgr2=0x7f15200b6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:29.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.617+0000 7f1527a82640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15200cdc40 0x7f15200b6ce0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f151000d7d0 tx=0x7f151000dc90 comp rx=0 tx=0).stop 2024-03-23T09:52:29.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.617+0000 7f1527a82640 1 -- 172.21.15.47:0/1054389941 shutdown_connections 2024-03-23T09:52:29.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.617+0000 7f1527a82640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f150406ac00 0x7f150406d0c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:29.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.617+0000 7f1527a82640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f152015ca90 0x7f15200b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.617+0000 7f1527a82640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15200cdc40 0x7f15200b6ce0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.617+0000 7f1527a82640 1 --2- 172.21.15.47:0/1054389941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200b8350 0x7f15200b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.617+0000 7f1527a82640 1 -- 172.21.15.47:0/1054389941 >> 172.21.15.47:0/1054389941 conn(0x7f15200659d0 msgr2=0x7f152015e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.618+0000 7f1527a82640 1 -- 172.21.15.47:0/1054389941 shutdown_connections 2024-03-23T09:52:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:29.618+0000 7f1527a82640 1 -- 172.21.15.47:0/1054389941 wait complete. 2024-03-23T09:52:30.031 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:51:40.395539+0000', 'modified': '2024-03-23T09:52:29.506769+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9762}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9762': {'gid': 9762, 'name': 'c', 'rank': 0, 'incarnation': 90, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.47:6834/3501786111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 3501786111}, {'type': 'v1', 'addr': '172.21.15.47:6834', 'nonce': 3501786111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:52:30.032 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9762:up:rejoin 2024-03-23T09:52:31.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:52:31.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.146+0000 7f995dba9640 1 Processor -- start 2024-03-23T09:52:31.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.146+0000 7f995dba9640 1 -- start start 2024-03-23T09:52:31.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.147+0000 7f995dba9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f995815cb50 0x7f99580b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.147+0000 7f995dba9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f99580cd9e0 0x7f99580b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.147+0000 7f995dba9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99580b72f0 con 0x7f99580cd5a0 2024-03-23T09:52:31.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.147+0000 7f995dba9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99580b7e70 con 0x7f99580cd9e0 2024-03-23T09:52:31.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.147+0000 7f995dba9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995815cf30 con 0x7f995815cb50 2024-03-23T09:52:31.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.147+0000 7f99577fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f995815cb50 0x7f99580b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.147+0000 7f9956ffd640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f99580cd5a0 0x7f99580d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:44834/0 (socket says 172.21.15.47:44834) 2024-03-23T09:52:31.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.147+0000 7f9956ffd640 1 -- 172.21.15.47:0/2621536803 learned_addr learned my addr 172.21.15.47:0/2621536803 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:31.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.147+0000 7f9957fff640 1 --2- 172.21.15.47:0/2621536803 >> v2:172.21.15.175:3300/0 conn(0x7f99580cd9e0 0x7f99580b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.148+0000 7f99567fc640 1 -- 172.21.15.47:0/2621536803 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598289863 0 0) 0x7f99580b72f0 con 0x7f99580cd5a0 2024-03-23T09:52:31.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.148+0000 7f99567fc640 1 -- 172.21.15.47:0/2621536803 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9940003520 con 0x7f99580cd5a0 2024-03-23T09:52:31.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.148+0000 7f99577fe640 1 -- 172.21.15.47:0/2621536803 >> v2:172.21.15.175:3300/0 conn(0x7f99580cd9e0 msgr2=0x7f99580b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:31.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.148+0000 7f99577fe640 1 --2- 172.21.15.47:0/2621536803 >> v2:172.21.15.175:3300/0 conn(0x7f99580cd9e0 0x7f99580b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.148+0000 7f99577fe640 1 -- 172.21.15.47:0/2621536803 >> v1:172.21.15.175:6789/0 conn(0x7f99580cd5a0 legacy=0x7f99580d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.148+0000 7f99577fe640 1 -- 172.21.15.47:0/2621536803 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99580b75d0 con 0x7f995815cb50 2024-03-23T09:52:31.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.149+0000 7f99577fe640 1 --2- 172.21.15.47:0/2621536803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f995815cb50 0x7f99580b84a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f99480097a0 tx=0x7f994802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8a547aed378c4b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:31.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.149+0000 7f99567fc640 1 -- 172.21.15.47:0/2621536803 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948030610 con 0x7f995815cb50 2024-03-23T09:52:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.149+0000 7f99567fc640 1 -- 172.21.15.47:0/2621536803 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9948030770 con 0x7f995815cb50 2024-03-23T09:52:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.149+0000 7f99567fc640 1 -- 172.21.15.47:0/2621536803 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948030ae0 con 0x7f995815cb50 2024-03-23T09:52:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.150+0000 7f995dba9640 1 -- 172.21.15.47:0/2621536803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f995815cb50 msgr2=0x7f99580b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.150+0000 7f995dba9640 1 --2- 172.21.15.47:0/2621536803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f995815cb50 0x7f99580b84a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f99480097a0 tx=0x7f994802b960 comp rx=0 tx=0).stop 2024-03-23T09:52:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.150+0000 7f995dba9640 1 -- 172.21.15.47:0/2621536803 shutdown_connections 2024-03-23T09:52:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.150+0000 7f995dba9640 1 --2- 172.21.15.47:0/2621536803 >> v2:172.21.15.175:3300/0 conn(0x7f99580cd9e0 0x7f99580b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.150+0000 7f995dba9640 1 --2- 172.21.15.47:0/2621536803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f995815cb50 0x7f99580b84a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.150+0000 7f995dba9640 1 -- 172.21.15.47:0/2621536803 >> 172.21.15.47:0/2621536803 conn(0x7f9958065df0 msgr2=0x7f9958066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.150+0000 7f995dba9640 1 -- 172.21.15.47:0/2621536803 shutdown_connections 2024-03-23T09:52:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.150+0000 7f995dba9640 1 -- 172.21.15.47:0/2621536803 wait complete. 2024-03-23T09:52:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.151+0000 7f995dba9640 1 Processor -- start 2024-03-23T09:52:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.151+0000 7f995dba9640 1 -- start start 2024-03-23T09:52:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f995dba9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99580cd5a0 0x7f995814b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f995dba9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99580cd9e0 0x7f99580b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f995dba9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f995815cb50 0x7f99581498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f995dba9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99580b3770 con 0x7f99580cd9e0 2024-03-23T09:52:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f995dba9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995814bfd0 con 0x7f99580cd5a0 2024-03-23T09:52:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f995dba9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995814c2b0 con 0x7f995815cb50 2024-03-23T09:52:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f9956ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99580cd9e0 0x7f99580b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f9956ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99580cd9e0 0x7f99580b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40010/0 (socket says 172.21.15.47:40010) 2024-03-23T09:52:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f9956ffd640 1 -- 172.21.15.47:0/2358963200 learned_addr learned my addr 172.21.15.47:0/2358963200 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f99577fe640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99580cd5a0 0x7f995814b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f9957fff640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f995815cb50 0x7f99581498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f9956ffd640 1 -- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f995815cb50 msgr2=0x7f99581498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f9956ffd640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f995815cb50 0x7f99581498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f9956ffd640 1 -- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99580cd5a0 msgr2=0x7f995814b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f9956ffd640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99580cd5a0 0x7f995814b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.152+0000 7f9956ffd640 1 -- 172.21.15.47:0/2358963200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9948009400 con 0x7f99580cd9e0 2024-03-23T09:52:31.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.153+0000 7f9956ffd640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99580cd9e0 0x7f99580b3230 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f994c00c410 tx=0x7f994c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:31.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.153+0000 7f993ffff640 1 -- 172.21.15.47:0/2358963200 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c005ce0 con 0x7f99580cd9e0 2024-03-23T09:52:31.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.153+0000 7f993ffff640 1 -- 172.21.15.47:0/2358963200 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994c005e40 con 0x7f99580cd9e0 2024-03-23T09:52:31.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.153+0000 7f995dba9640 1 -- 172.21.15.47:0/2358963200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995814c590 con 0x7f99580cd9e0 2024-03-23T09:52:31.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.153+0000 7f993ffff640 1 -- 172.21.15.47:0/2358963200 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c0072f0 con 0x7f99580cd9e0 2024-03-23T09:52:31.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.153+0000 7f995dba9640 1 -- 172.21.15.47:0/2358963200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99581f58e0 con 0x7f99580cd9e0 2024-03-23T09:52:31.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.154+0000 7f995dba9640 1 -- 172.21.15.47:0/2358963200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9958072360 con 0x7f99580cd9e0 2024-03-23T09:52:31.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.154+0000 7f993ffff640 1 -- 172.21.15.47:0/2358963200 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f994c007510 con 0x7f99580cd9e0 2024-03-23T09:52:31.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.155+0000 7f993ffff640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f994406ac00 0x7f994406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.155+0000 7f993ffff640 1 -- 172.21.15.47:0/2358963200 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f994c088b50 con 0x7f99580cd9e0 2024-03-23T09:52:31.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.155+0000 7f99577fe640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f994406ac00 0x7f994406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.156+0000 7f99577fe640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f994406ac00 0x7f994406d0c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9948036000 tx=0x7f9948008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:31.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.159+0000 7f993ffff640 1 -- 172.21.15.47:0/2358963200 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f994c053c80 con 0x7f99580cd9e0 2024-03-23T09:52:31.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.376+0000 7f995dba9640 1 -- 172.21.15.47:0/2358963200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9958066be0 con 0x7f99580cd9e0 2024-03-23T09:52:31.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.377+0000 7f993ffff640 1 -- 172.21.15.47:0/2358963200 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f994c0572a0 con 0x7f99580cd9e0 2024-03-23T09:52:31.377 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:31.378 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":93,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:51:40.395539+0000","modified":"2024-03-23T09:52:30.511498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"c","rank":0,"incarnation":90,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6834/3501786111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3501786111},{"type":"v1","addr":"172.21.15.47:6834","nonce":3501786111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-23T09:52:31.378 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 93 2024-03-23T09:52:31.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.379+0000 7f995dba9640 1 -- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f994406ac00 msgr2=0x7f994406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.379+0000 7f995dba9640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f994406ac00 0x7f994406d0c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9948036000 tx=0x7f9948008030 comp rx=0 tx=0).stop 2024-03-23T09:52:31.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.379+0000 7f995dba9640 1 -- 172.21.15.47:0/2358963200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99580cd9e0 msgr2=0x7f99580b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.379+0000 7f995dba9640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99580cd9e0 0x7f99580b3230 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f994c00c410 tx=0x7f994c00c8d0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.379+0000 7f995dba9640 1 -- 172.21.15.47:0/2358963200 shutdown_connections 2024-03-23T09:52:31.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.379+0000 7f995dba9640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f994406ac00 0x7f994406d0c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.379+0000 7f995dba9640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f995815cb50 0x7f99581498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.379+0000 7f995dba9640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99580cd9e0 0x7f99580b3230 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.379+0000 7f995dba9640 1 --2- 172.21.15.47:0/2358963200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99580cd5a0 0x7f995814b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.379+0000 7f995dba9640 1 -- 172.21.15.47:0/2358963200 >> 172.21.15.47:0/2358963200 conn(0x7f9958065df0 msgr2=0x7f99580d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:31.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.379+0000 7f995dba9640 1 -- 172.21.15.47:0/2358963200 shutdown_connections 2024-03-23T09:52:31.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.379+0000 7f995dba9640 1 -- 172.21.15.47:0/2358963200 wait complete. 2024-03-23T09:52:31.499 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-03-23T09:52:31.499+0000 7fcc7e3c6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:52:31.794 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 93, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T09:51:40.395539+0000', 'modified': '2024-03-23T09:52:30.511498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9762}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9762': {'gid': 9762, 'name': 'c', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.47:6834/3501786111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 3501786111}, {'type': 'v1', 'addr': '172.21.15.47:6834', 'nonce': 3501786111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T09:52:31.794 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T09:52:31.794 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T09:52:31.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.900+0000 7ff879fb1640 1 Processor -- start 2024-03-23T09:52:31.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.900+0000 7ff879fb1640 1 -- start start 2024-03-23T09:52:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff879fb1640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff874098a00 0x7ff874098de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff879fb1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740b15f0 0x7ff87406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff879fb1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8740cf320 con 0x7ff8740caca0 2024-03-23T09:52:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff879fb1640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87415e180 con 0x7ff874098a00 2024-03-23T09:52:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff879fb1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740ceac0 con 0x7ff8740b15f0 2024-03-23T09:52:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff873fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff8740caca0 0x7ff8740cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56796/0 (socket says 172.21.15.47:56796) 2024-03-23T09:52:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff873fff640 1 -- 172.21.15.47:0/235333976 learned_addr learned my addr 172.21.15.47:0/235333976 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605712525 0 0) 0x7ff8740cf320 con 0x7ff8740caca0 2024-03-23T09:52:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff872ffd640 1 --2- 172.21.15.47:0/235333976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740b15f0 0x7ff87406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff8737fe640 1 --2- 172.21.15.47:0/235333976 >> v2:172.21.15.175:3300/0 conn(0x7ff874098a00 0x7ff874098de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff85c003520 con 0x7ff8740caca0 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3365730626 0 0) 0x7ff85c003520 con 0x7ff8740caca0 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740b15f0 msgr2=0x7ff87406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.901+0000 7ff8727fc640 1 --2- 172.21.15.47:0/235333976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740b15f0 0x7ff87406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 >> v2:172.21.15.175:3300/0 conn(0x7ff874098a00 msgr2=0x7ff874098de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8727fc640 1 --2- 172.21.15.47:0/235333976 >> v2:172.21.15.175:3300/0 conn(0x7ff874098a00 0x7ff874098de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff874099320 con 0x7ff8740caca0 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7ff86c0019f0 con 0x7ff8740caca0 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 >> v1:172.21.15.47:6789/0 conn(0x7ff8740caca0 legacy=0x7ff8740cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8727fc640 1 --2- 172.21.15.47:0/235333976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff85c003970 0x7ff85c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8727fc640 1 --2- 172.21.15.47:0/235333976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740b15f0 0x7ff85c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8727fc640 1 --2- 172.21.15.47:0/235333976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff85c004d00 0x7ff85c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87415e180 con 0x7ff85c003970 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740ceac0 con 0x7ff85c004d00 2024-03-23T09:52:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c00b980 con 0x7ff8740b15f0 2024-03-23T09:52:31.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8737fe640 1 --2- 172.21.15.47:0/235333976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff85c003970 0x7ff85c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff873fff640 1 --2- 172.21.15.47:0/235333976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff85c004d00 0x7ff85c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff872ffd640 1 --2- 172.21.15.47:0/235333976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740b15f0 0x7ff85c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8737fe640 1 -- 172.21.15.47:0/235333976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740b15f0 msgr2=0x7ff85c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:31.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8737fe640 1 --2- 172.21.15.47:0/235333976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8740b15f0 0x7ff85c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8737fe640 1 -- 172.21.15.47:0/235333976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff85c004d00 msgr2=0x7ff85c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:31.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8737fe640 1 --2- 172.21.15.47:0/235333976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff85c004d00 0x7ff85c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.902+0000 7ff8737fe640 1 -- 172.21.15.47:0/235333976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c003520 con 0x7ff85c003970 2024-03-23T09:52:31.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.903+0000 7ff8737fe640 1 --2- 172.21.15.47:0/235333976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff85c003970 0x7ff85c003d90 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7ff864009ac0 tx=0x7ff85c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73973f884f3cf5c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:31.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.903+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86402c710 con 0x7ff85c003970 2024-03-23T09:52:31.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.903+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86402c870 con 0x7ff85c003970 2024-03-23T09:52:31.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.903+0000 7ff8727fc640 1 -- 172.21.15.47:0/235333976 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86402cc00 con 0x7ff85c003970 2024-03-23T09:52:31.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.903+0000 7ff879fb1640 1 -- 172.21.15.47:0/235333976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff85c003970 msgr2=0x7ff85c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.903+0000 7ff879fb1640 1 --2- 172.21.15.47:0/235333976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff85c003970 0x7ff85c003d90 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7ff864009ac0 tx=0x7ff85c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.903+0000 7ff873fff640 1 -- 172.21.15.47:0/235333976 reap_dead start 2024-03-23T09:52:31.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.903+0000 7ff879fb1640 1 -- 172.21.15.47:0/235333976 shutdown_connections 2024-03-23T09:52:31.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.903+0000 7ff879fb1640 1 -- 172.21.15.47:0/235333976 >> 172.21.15.47:0/235333976 conn(0x7ff8740b2140 msgr2=0x7ff8740b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:31.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.903+0000 7ff879fb1640 1 -- 172.21.15.47:0/235333976 shutdown_connections 2024-03-23T09:52:31.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.903+0000 7ff879fb1640 1 -- 172.21.15.47:0/235333976 wait complete. 2024-03-23T09:52:31.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.905+0000 7ff879fb1640 1 Processor -- start 2024-03-23T09:52:31.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.905+0000 7ff879fb1640 1 -- start start 2024-03-23T09:52:31.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.905+0000 7ff879fb1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85c003970 0x7ff8741f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.905+0000 7ff879fb1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87415b730 0x7ff874151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.905+0000 7ff879fb1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87406f0d0 0x7ff87406f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.905+0000 7ff879fb1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87406fa90 con 0x7ff87406f0d0 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.905+0000 7ff879fb1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87415d420 con 0x7ff87415b730 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.905+0000 7ff879fb1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff874152040 con 0x7ff85c003970 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff873fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87406f0d0 0x7ff87406f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff873fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87406f0d0 0x7ff87406f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40026/0 (socket says 172.21.15.47:40026) 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff873fff640 1 -- 172.21.15.47:0/483798620 learned_addr learned my addr 172.21.15.47:0/483798620 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff8737fe640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85c003970 0x7ff8741f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff872ffd640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87415b730 0x7ff874151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff873fff640 1 -- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85c003970 msgr2=0x7ff8741f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff873fff640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85c003970 0x7ff8741f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff873fff640 1 -- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87415b730 msgr2=0x7ff874151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff873fff640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87415b730 0x7ff874151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff873fff640 1 -- 172.21.15.47:0/483798620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87415bb90 con 0x7ff87406f0d0 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff872ffd640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87415b730 0x7ff874151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff873fff640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87406f0d0 0x7ff87406f550 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7ff874071bd0 tx=0x7ff86c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff85bfff640 1 -- 172.21.15.47:0/483798620 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c007a80 con 0x7ff87406f0d0 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff85bfff640 1 -- 172.21.15.47:0/483798620 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c00ee20 con 0x7ff87406f0d0 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff879fb1640 1 -- 172.21.15.47:0/483798620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8741f2830 con 0x7ff87406f0d0 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff85bfff640 1 -- 172.21.15.47:0/483798620 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c014670 con 0x7ff87406f0d0 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.906+0000 7ff879fb1640 1 -- 172.21.15.47:0/483798620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8741f2df0 con 0x7ff87406f0d0 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.908+0000 7ff85bfff640 1 -- 172.21.15.47:0/483798620 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff86c0147d0 con 0x7ff87406f0d0 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.908+0000 7ff85bfff640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff86006ac30 0x7ff86006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.908+0000 7ff85bfff640 1 -- 172.21.15.47:0/483798620 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff86c089120 con 0x7ff87406f0d0 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.909+0000 7ff8737fe640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff86006ac30 0x7ff86006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.909+0000 7ff8737fe640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff86006ac30 0x7ff86006d0f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff864036000 tx=0x7ff864030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.909+0000 7ff87cc04740 1 Processor -- start 2024-03-23T09:52:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.909+0000 7ff87cc04740 1 -- start start 2024-03-23T09:52:31.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff87cc04740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fcefff0750 0x55fceffa7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff87cc04740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 0x55fceffa82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff87cc04740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fceffa2c70 0x55fceffd2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff87cc04740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceff139b0 con 0x55fceffa67d0 2024-03-23T09:52:31.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff87cc04740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceff11140 con 0x55fceffa2c70 2024-03-23T09:52:31.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff87cc04740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fceff0ca30 con 0x55fcefff0750 2024-03-23T09:52:31.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff872ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 0x55fceffa82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff872ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 0x55fceffa82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40038/0 (socket says 172.21.15.47:40038) 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff872ffd640 1 -- 172.21.15.47:0/4189511830 learned_addr learned my addr 172.21.15.47:0/4189511830 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff8737fe640 1 --2- 172.21.15.47:0/4189511830 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fcefff0750 0x55fceffa7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff872ffd640 1 -- 172.21.15.47:0/4189511830 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fcefff0750 msgr2=0x55fceffa7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff872ffd640 1 --2- 172.21.15.47:0/4189511830 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fcefff0750 0x55fceffa7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff872ffd640 1 -- 172.21.15.47:0/4189511830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fceffa2c70 msgr2=0x55fceffd2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff873fff640 1 --2- 172.21.15.47:0/4189511830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fceffa2c70 0x55fceffd2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.910+0000 7ff872ffd640 1 --2- 172.21.15.47:0/4189511830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fceffa2c70 0x55fceffd2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.911+0000 7ff872ffd640 1 -- 172.21.15.47:0/4189511830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcefe992f0 con 0x55fceffa67d0 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.911+0000 7ff873fff640 1 --2- 172.21.15.47:0/4189511830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fceffa2c70 0x55fceffd2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.911+0000 7ff872ffd640 1 --2- 172.21.15.47:0/4189511830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 0x55fceffa82c0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7ff868009b80 tx=0x7ff868009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f9911b836654e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.911+0000 7ff8597fa640 1 -- 172.21.15.47:0/4189511830 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868013e70 con 0x55fceffa67d0 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.911+0000 7ff8597fa640 1 -- 172.21.15.47:0/4189511830 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff868009c70 con 0x55fceffa67d0 2024-03-23T09:52:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.911+0000 7ff8597fa640 1 -- 172.21.15.47:0/4189511830 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868016460 con 0x55fceffa67d0 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.911+0000 7ff87cc04740 1 -- 172.21.15.47:0/4189511830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 msgr2=0x55fceffa82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.911+0000 7ff87cc04740 1 --2- 172.21.15.47:0/4189511830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 0x55fceffa82c0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7ff868009b80 tx=0x7ff868009190 comp rx=0 tx=0).stop 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.912+0000 7ff87cc04740 1 -- 172.21.15.47:0/4189511830 shutdown_connections 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.912+0000 7ff87cc04740 1 --2- 172.21.15.47:0/4189511830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fceffa2c70 0x55fceffd2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.912+0000 7ff87cc04740 1 --2- 172.21.15.47:0/4189511830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 0x55fceffa82c0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.912+0000 7ff87cc04740 1 --2- 172.21.15.47:0/4189511830 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fcefff0750 0x55fceffa7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.912+0000 7ff87cc04740 1 -- 172.21.15.47:0/4189511830 >> 172.21.15.47:0/4189511830 conn(0x55fceffa4ee0 msgr2=0x55fceffeda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.912+0000 7ff87cc04740 1 -- 172.21.15.47:0/4189511830 shutdown_connections 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.912+0000 7ff87cc04740 1 -- 172.21.15.47:0/4189511830 wait complete. 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.912+0000 7ff87cc04740 1 -- start start 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.912+0000 7ff87cc04740 1 Processor -- start 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff87cc04740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fceffa67d0 0x55fceffc9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff87cc04740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcefff0750 0x55fceffeecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff87cc04740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 0x55fcefff2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff87cc04740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcefecb090 con 0x55fceffef210 2024-03-23T09:52:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff87cc04740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcefecad30 con 0x55fcefff0750 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff87cc04740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fcefe902a0 con 0x55fceffa67d0 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff87cc04740 10 client.0 _ll_register_callbacks cb 0x55fcefec6050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff873fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 0x55fcefff2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff873fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 0x55fcefff2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40054/0 (socket says 172.21.15.47:40054) 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff873fff640 1 -- 172.21.15.47:0/1842145882 learned_addr learned my addr 172.21.15.47:0/1842145882 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff873fff640 1 -- 172.21.15.47:0/1842145882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fceffa67d0 msgr2=0x55fceffc9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff873fff640 1 --2- 172.21.15.47:0/1842145882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fceffa67d0 0x55fceffc9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff872ffd640 1 --2- 172.21.15.47:0/1842145882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcefff0750 0x55fceffeecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff873fff640 1 -- 172.21.15.47:0/1842145882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcefff0750 msgr2=0x55fceffeecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff873fff640 1 --2- 172.21.15.47:0/1842145882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcefff0750 0x55fceffeecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.913+0000 7ff873fff640 1 -- 172.21.15.47:0/1842145882 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8680097b0 con 0x55fceffef210 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff873fff640 1 --2- 172.21.15.47:0/1842145882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 0x55fcefff2d40 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7ff86c087590 tx=0x7ff86c058010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b36cf5789fd6c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff8527fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:52:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff8527fc640 1 -- 172.21.15.47:0/1842145882 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c05fec0 con 0x55fceffef210 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff8527fc640 1 -- 172.21.15.47:0/1842145882 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff86c060020 con 0x55fceffef210 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff87cc04740 1 -- 172.21.15.47:0/1842145882 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcefc003c0 con 0x55fceffef210 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff8527fc640 1 -- 172.21.15.47:0/1842145882 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff86c087170 con 0x55fceffef210 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff87cc04740 10 client.9855 fetch_fsmap learned FSMap version 93 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff87cc04740 1 -- 172.21.15.47:0/1842145882 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55fceff0b0b0 con 0x55fceffef210 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff8527fc640 1 -- 172.21.15.47:0/1842145882 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fcefc003c0 con 0x55fceffef210 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff8527fc640 1 -- 172.21.15.47:0/1842145882 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 93) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff86c06fb70 con 0x55fceffef210 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff87cc04740 10 client.9855 fetch_fsmap finished waiting for FSMap version 93 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff87cc04740 10 client.9855 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff87cc04740 20 client.9855 populate_metadata read hostname 'smithi047' 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff87cc04740 1 --2- 172.21.15.47:0/1842145882 >> [v2:172.21.15.47:6832/3501786111,v1:172.21.15.47:6834/3501786111] conn(0x55fceffc9c50 0x55fceffca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff87cc04740 4 client.9855 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff87cc04740 1 -- 172.21.15.47:0/1842145882 --> [v2:172.21.15.47:6832/3501786111,v1:172.21.15.47:6834/3501786111] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcefc116c0 con 0x55fceffc9c50 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.914+0000 7ff8737fe640 1 --2- 172.21.15.47:0/1842145882 >> [v2:172.21.15.47:6832/3501786111,v1:172.21.15.47:6834/3501786111] conn(0x55fceffc9c50 0x55fceffca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff8737fe640 1 --2- 172.21.15.47:0/1842145882 >> [v2:172.21.15.47:6832/3501786111,v1:172.21.15.47:6834/3501786111] conn(0x55fceffc9c50 0x55fceffca030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d74922d0aa81bf server_cookie=bd9e84eeedde1a5 in_seq=0 out_seq=0 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff8527fc640 10 client.9855 ms_handle_connect on v2:172.21.15.47:6832/3501786111 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff8527fc640 1 -- 172.21.15.47:0/1842145882 <== mds.0 v2:172.21.15.47:6832/3501786111 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fcefc116c0 con 0x55fceffc9c50 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff8527fc640 10 client.9855 handle_command_reply: tid=0 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff87cc04740 1 client.9855 shutdown 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff87cc04740 1 -- 172.21.15.47:0/1842145882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 msgr2=0x55fcefff2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff87cc04740 1 --2- 172.21.15.47:0/1842145882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 0x55fcefff2d40 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7ff86c087590 tx=0x7ff86c058010 comp rx=0 tx=0).stop 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff87cc04740 1 -- 172.21.15.47:0/1842145882 shutdown_connections 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff87cc04740 1 --2- 172.21.15.47:0/1842145882 >> [v2:172.21.15.47:6832/3501786111,v1:172.21.15.47:6834/3501786111] conn(0x55fceffc9c50 0x55fceffca030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff87cc04740 1 --2- 172.21.15.47:0/1842145882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 0x55fcefff2d40 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff8527fc640 0 client.9855 ms_handle_reset on v2:172.21.15.47:6832/3501786111 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff87cc04740 1 --2- 172.21.15.47:0/1842145882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcefff0750 0x55fceffeecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff87cc04740 1 --2- 172.21.15.47:0/1842145882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fceffa67d0 0x55fceffc9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff87cc04740 1 -- 172.21.15.47:0/1842145882 >> 172.21.15.47:0/1842145882 conn(0x55fceffa2c70 msgr2=0x55fceffa6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff87cc04740 1 -- 172.21.15.47:0/1842145882 shutdown_connections 2024-03-23T09:52:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.915+0000 7ff87cc04740 1 -- 172.21.15.47:0/1842145882 wait complete. 2024-03-23T09:52:31.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.916+0000 7ff87cc04740 20 client.9855 trim_cache size 0 max 16384 2024-03-23T09:52:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.931+0000 7ff87cc04740 1 Processor -- start 2024-03-23T09:52:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.931+0000 7ff87cc04740 1 -- start start 2024-03-23T09:52:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff87cc04740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fceffa67d0 0x55fceffee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff87cc04740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 0x55fceffa3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff87cc04740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fcefff0750 0x55fceffec880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff87cc04740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf000fc20 con 0x55fceffef210 2024-03-23T09:52:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff87cc04740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcefec6440 con 0x55fceffa67d0 2024-03-23T09:52:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff87cc04740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fceffa4640 con 0x55fcefff0750 2024-03-23T09:52:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff872ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 0x55fceffa3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff872ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 0x55fceffa3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40056/0 (socket says 172.21.15.47:40056) 2024-03-23T09:52:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff872ffd640 1 -- 172.21.15.47:0/4028667458 learned_addr learned my addr 172.21.15.47:0/4028667458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff873fff640 1 --2- 172.21.15.47:0/4028667458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fcefff0750 0x55fceffec880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff872ffd640 1 -- 172.21.15.47:0/4028667458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fcefff0750 msgr2=0x55fceffec880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff8737fe640 1 --2- 172.21.15.47:0/4028667458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fceffa67d0 0x55fceffee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff872ffd640 1 --2- 172.21.15.47:0/4028667458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fcefff0750 0x55fceffec880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff872ffd640 1 -- 172.21.15.47:0/4028667458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fceffa67d0 msgr2=0x55fceffee590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff872ffd640 1 --2- 172.21.15.47:0/4028667458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fceffa67d0 0x55fceffee590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff872ffd640 1 -- 172.21.15.47:0/4028667458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcefead430 con 0x55fceffef210 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff873fff640 1 --2- 172.21.15.47:0/4028667458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fcefff0750 0x55fceffec880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff8737fe640 1 --2- 172.21.15.47:0/4028667458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fceffa67d0 0x55fceffee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff872ffd640 1 --2- 172.21.15.47:0/4028667458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 0x55fceffa3150 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7ff86801a000 tx=0x7ff868011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f7fc5a1dd4b0b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff859ffb640 1 -- 172.21.15.47:0/4028667458 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868024070 con 0x55fceffef210 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff859ffb640 1 -- 172.21.15.47:0/4028667458 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff868013e70 con 0x55fceffef210 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.932+0000 7ff859ffb640 1 -- 172.21.15.47:0/4028667458 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868005990 con 0x55fceffef210 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.933+0000 7ff87cc04740 1 -- 172.21.15.47:0/4028667458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 msgr2=0x55fceffa3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.933+0000 7ff87cc04740 1 --2- 172.21.15.47:0/4028667458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 0x55fceffa3150 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7ff86801a000 tx=0x7ff868011000 comp rx=0 tx=0).stop 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.933+0000 7ff87cc04740 1 -- 172.21.15.47:0/4028667458 shutdown_connections 2024-03-23T09:52:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.933+0000 7ff87cc04740 1 --2- 172.21.15.47:0/4028667458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fcefff0750 0x55fceffec880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.933+0000 7ff87cc04740 1 --2- 172.21.15.47:0/4028667458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffef210 0x55fceffa3150 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.933+0000 7ff87cc04740 1 --2- 172.21.15.47:0/4028667458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fceffa67d0 0x55fceffee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.933+0000 7ff87cc04740 1 -- 172.21.15.47:0/4028667458 >> 172.21.15.47:0/4028667458 conn(0x55fceffa2c70 msgr2=0x55fceffcf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.933+0000 7ff87cc04740 1 -- 172.21.15.47:0/4028667458 shutdown_connections 2024-03-23T09:52:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.933+0000 7ff87cc04740 1 -- 172.21.15.47:0/4028667458 wait complete. 2024-03-23T09:52:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.934+0000 7ff87cc04740 1 -- start start 2024-03-23T09:52:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.934+0000 7ff87cc04740 1 Processor -- start 2024-03-23T09:52:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.934+0000 7ff87cc04740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 0x55fcf0006d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.934+0000 7ff87cc04740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fceffef210 0x55fcf00536b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.934+0000 7ff87cc04740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcefff0750 0x55fceffec880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.934+0000 7ff87cc04740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceffa1d90 con 0x55fceffa67d0 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.934+0000 7ff87cc04740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceffee590 con 0x55fcefff0750 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.934+0000 7ff87cc04740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fceffa1870 con 0x55fceffef210 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.934+0000 7ff87cc04740 10 client.0 _ll_register_callbacks cb 0x7ff868009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.934+0000 7ff8737fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 0x55fcf0006d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8737fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 0x55fcf0006d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40058/0 (socket says 172.21.15.47:40058) 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8737fe640 1 -- 172.21.15.47:0/1232425651 learned_addr learned my addr 172.21.15.47:0/1232425651 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff873fff640 1 --2- 172.21.15.47:0/1232425651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcefff0750 0x55fceffec880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff872ffd640 1 --2- 172.21.15.47:0/1232425651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fceffef210 0x55fcf00536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8737fe640 1 -- 172.21.15.47:0/1232425651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fceffef210 msgr2=0x55fcf00536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8737fe640 1 --2- 172.21.15.47:0/1232425651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fceffef210 0x55fcf00536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8737fe640 1 -- 172.21.15.47:0/1232425651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcefff0750 msgr2=0x55fceffec880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:31.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8737fe640 1 --2- 172.21.15.47:0/1232425651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcefff0750 0x55fceffec880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8737fe640 1 -- 172.21.15.47:0/1232425651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcefe992f0 con 0x55fceffa67d0 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8737fe640 1 --2- 172.21.15.47:0/1232425651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 0x55fcf0006d50 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7ff864037040 tx=0x7ff864004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7adae5038a3aed server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8527fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8527fc640 1 -- 172.21.15.47:0/1232425651 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff864050070 con 0x55fceffa67d0 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8527fc640 1 -- 172.21.15.47:0/1232425651 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff864004310 con 0x55fceffa67d0 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff87cc04740 1 -- 172.21.15.47:0/1232425651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff868009c70 con 0x55fceffa67d0 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8527fc640 1 -- 172.21.15.47:0/1232425651 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8640044d0 con 0x55fceffa67d0 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff87cc04740 10 client.9861 fetch_fsmap learned FSMap version 93 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff87cc04740 1 -- 172.21.15.47:0/1232425651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55fcefed2b40 con 0x55fceffa67d0 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8527fc640 1 -- 172.21.15.47:0/1232425651 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff868009c70 con 0x55fceffa67d0 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.935+0000 7ff8527fc640 1 -- 172.21.15.47:0/1232425651 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 93) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff864003610 con 0x55fceffa67d0 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 10 client.9861 fetch_fsmap finished waiting for FSMap version 93 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 10 client.9861 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 20 client.9861 populate_metadata read hostname 'smithi047' 2024-03-23T09:52:31.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 1 --2- 172.21.15.47:0/1232425651 >> [v2:172.21.15.47:6832/3501786111,v1:172.21.15.47:6834/3501786111] conn(0x55fceffe3440 0x55fceffe3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 4 client.9861 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 1 -- 172.21.15.47:0/1232425651 --> [v2:172.21.15.47:6832/3501786111,v1:172.21.15.47:6834/3501786111] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fcefc64d10 con 0x55fceffe3440 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff872ffd640 1 --2- 172.21.15.47:0/1232425651 >> [v2:172.21.15.47:6832/3501786111,v1:172.21.15.47:6834/3501786111] conn(0x55fceffe3440 0x55fceffe3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff872ffd640 1 --2- 172.21.15.47:0/1232425651 >> [v2:172.21.15.47:6832/3501786111,v1:172.21.15.47:6834/3501786111] conn(0x55fceffe3440 0x55fceffe3820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d335f109a7f2f262 server_cookie=dd9445236c409a89 in_seq=0 out_seq=0 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff8527fc640 10 client.9861 ms_handle_connect on v2:172.21.15.47:6832/3501786111 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff8527fc640 1 -- 172.21.15.47:0/1232425651 <== mds.0 v2:172.21.15.47:6832/3501786111 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55fcefc64d10 con 0x55fceffe3440 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff8527fc640 10 client.9861 handle_command_reply: tid=0 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 1 client.9861 shutdown 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 1 -- 172.21.15.47:0/1232425651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 msgr2=0x55fcf0006d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 1 --2- 172.21.15.47:0/1232425651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 0x55fcf0006d50 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7ff864037040 tx=0x7ff864004730 comp rx=0 tx=0).stop 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 1 -- 172.21.15.47:0/1232425651 shutdown_connections 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 1 --2- 172.21.15.47:0/1232425651 >> [v2:172.21.15.47:6832/3501786111,v1:172.21.15.47:6834/3501786111] conn(0x55fceffe3440 0x55fceffe3820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 1 --2- 172.21.15.47:0/1232425651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcefff0750 0x55fceffec880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 1 --2- 172.21.15.47:0/1232425651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55fceffef210 0x55fcf00536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff8527fc640 0 client.9861 ms_handle_reset on v2:172.21.15.47:6832/3501786111 2024-03-23T09:52:31.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 1 --2- 172.21.15.47:0/1232425651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceffa67d0 0x55fcf0006d50 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 1 -- 172.21.15.47:0/1232425651 >> 172.21.15.47:0/1232425651 conn(0x55fceffa2c70 msgr2=0x55fcefff2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:31.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 1 -- 172.21.15.47:0/1232425651 shutdown_connections 2024-03-23T09:52:31.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.936+0000 7ff87cc04740 1 -- 172.21.15.47:0/1232425651 wait complete. 2024-03-23T09:52:31.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.937+0000 7ff87cc04740 20 client.9861 trim_cache size 0 max 16384 2024-03-23T09:52:31.943 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:52:31.943 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "f31804be-685c-4b3b-b2be-155afa9f5d13", 2024-03-23T09:52:31.943 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-23T09:52:31.943 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 9762, 2024-03-23T09:52:31.943 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-23T09:52:31.943 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-23T09:52:31.944 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-23T09:52:31.944 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 4.4328627330000003, 2024-03-23T09:52:31.944 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 93, 2024-03-23T09:52:31.944 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 118, 2024-03-23T09:52:31.944 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 118, 2024-03-23T09:52:31.944 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 6.4118135619999999 2024-03-23T09:52:31.944 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:52:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.937+0000 7ff879fb1640 1 -- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff86006ac30 msgr2=0x7ff86006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.937+0000 7ff879fb1640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff86006ac30 0x7ff86006d0f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff864036000 tx=0x7ff864030000 comp rx=0 tx=0).stop 2024-03-23T09:52:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.937+0000 7ff879fb1640 1 -- 172.21.15.47:0/483798620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87406f0d0 msgr2=0x7ff87406f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.937+0000 7ff879fb1640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87406f0d0 0x7ff87406f550 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7ff874071bd0 tx=0x7ff86c00e530 comp rx=0 tx=0).stop 2024-03-23T09:52:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.937+0000 7ff879fb1640 1 -- 172.21.15.47:0/483798620 shutdown_connections 2024-03-23T09:52:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.937+0000 7ff879fb1640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff86006ac30 0x7ff86006d0f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.937+0000 7ff879fb1640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff87406f0d0 0x7ff87406f550 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.937+0000 7ff879fb1640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87415b730 0x7ff874151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.937+0000 7ff879fb1640 1 --2- 172.21.15.47:0/483798620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85c003970 0x7ff8741f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.937+0000 7ff879fb1640 1 -- 172.21.15.47:0/483798620 >> 172.21.15.47:0/483798620 conn(0x7ff8740b2140 msgr2=0x7ff8740c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.937+0000 7ff879fb1640 1 -- 172.21.15.47:0/483798620 shutdown_connections 2024-03-23T09:52:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:31.937+0000 7ff879fb1640 1 -- 172.21.15.47:0/483798620 wait complete. 2024-03-23T09:52:31.948 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T09:52:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.048+0000 7f90ca20b640 1 Processor -- start 2024-03-23T09:52:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.048+0000 7f90ca20b640 1 -- start start 2024-03-23T09:52:32.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.049+0000 7f90ca20b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f90c4098a00 0x7f90c4098de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:32.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.049+0000 7f90ca20b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c40cf030 con 0x7f90c4099ad0 2024-03-23T09:52:32.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.049+0000 7f90ca20b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c4099320 con 0x7f90c415c350 2024-03-23T09:52:32.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.049+0000 7f90ca20b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c40cfdf0 con 0x7f90c4098a00 2024-03-23T09:52:32.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.049+0000 7f90c37fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f90c4098a00 0x7f90c4098de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:32.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.049+0000 7f90c2ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f90c4099ad0 0x7f90c40ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56802/0 (socket says 172.21.15.47:56802) 2024-03-23T09:52:32.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.049+0000 7f90c2ffd640 1 -- 172.21.15.47:0/823401218 learned_addr learned my addr 172.21.15.47:0/823401218 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:32.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.050+0000 7f90c37fe640 1 -- 172.21.15.47:0/823401218 >> v1:172.21.15.175:6789/0 conn(0x7f90c415c350 legacy=0x7f90c415c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:32.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.050+0000 7f90c37fe640 1 -- 172.21.15.47:0/823401218 >> v1:172.21.15.47:6789/0 conn(0x7f90c4099ad0 legacy=0x7f90c40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:32.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.050+0000 7f90c37fe640 1 -- 172.21.15.47:0/823401218 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c40cf270 con 0x7f90c4098a00 2024-03-23T09:52:32.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.050+0000 7f90c27fc640 1 -- 172.21.15.47:0/823401218 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238831213 0 0) 0x7f90b8001110 con 0x7f90c4099ad0 2024-03-23T09:52:32.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.050+0000 7f90c37fe640 1 --2- 172.21.15.47:0/823401218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90c4098a00 0x7f90c4098de0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f90b40097a0 tx=0x7f90b402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7967735909bc7a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:32.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.051+0000 7f90c27fc640 1 -- 172.21.15.47:0/823401218 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4030610 con 0x7f90c4098a00 2024-03-23T09:52:32.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.051+0000 7f90c27fc640 1 -- 172.21.15.47:0/823401218 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b4030770 con 0x7f90c4098a00 2024-03-23T09:52:32.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.051+0000 7f90c27fc640 1 -- 172.21.15.47:0/823401218 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4030b00 con 0x7f90c4098a00 2024-03-23T09:52:32.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.051+0000 7f90ca20b640 1 -- 172.21.15.47:0/823401218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90c4098a00 msgr2=0x7f90c4098de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:32.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.051+0000 7f90ca20b640 1 --2- 172.21.15.47:0/823401218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90c4098a00 0x7f90c4098de0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f90b40097a0 tx=0x7f90b402b960 comp rx=0 tx=0).stop 2024-03-23T09:52:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.051+0000 7f90ca20b640 1 -- 172.21.15.47:0/823401218 shutdown_connections 2024-03-23T09:52:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.051+0000 7f90ca20b640 1 --2- 172.21.15.47:0/823401218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90c4098a00 0x7f90c4098de0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.051+0000 7f90ca20b640 1 -- 172.21.15.47:0/823401218 >> 172.21.15.47:0/823401218 conn(0x7f90c40b21d0 msgr2=0x7f90c40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.051+0000 7f90ca20b640 1 -- 172.21.15.47:0/823401218 shutdown_connections 2024-03-23T09:52:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.051+0000 7f90ca20b640 1 -- 172.21.15.47:0/823401218 wait complete. 2024-03-23T09:52:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.052+0000 7f90ca20b640 1 Processor -- start 2024-03-23T09:52:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.052+0000 7f90ca20b640 1 -- start start 2024-03-23T09:52:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.053+0000 7f90ca20b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90c4098a00 0x7f90c4159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.053+0000 7f90ca20b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90c4099ad0 0x7f90c40c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.053+0000 7f90ca20b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90c415c350 0x7f90c40c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.053+0000 7f90ca20b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40c9b30 con 0x7f90c4099ad0 2024-03-23T09:52:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.053+0000 7f90ca20b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40c8bf0 con 0x7f90c4098a00 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.053+0000 7f90ca20b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40c8eb0 con 0x7f90c415c350 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.053+0000 7f90c2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90c4099ad0 0x7f90c40c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.053+0000 7f90c2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90c4099ad0 0x7f90c40c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40062/0 (socket says 172.21.15.47:40062) 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.053+0000 7f90c2ffd640 1 -- 172.21.15.47:0/2338697439 learned_addr learned my addr 172.21.15.47:0/2338697439 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90c2ffd640 1 -- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90c415c350 msgr2=0x7f90c40c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90c3fff640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90c415c350 0x7f90c40c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90c37fe640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90c4098a00 0x7f90c4159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90c2ffd640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90c415c350 0x7f90c40c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90c2ffd640 1 -- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90c4098a00 msgr2=0x7f90c4159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90c2ffd640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90c4098a00 0x7f90c4159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90c2ffd640 1 -- 172.21.15.47:0/2338697439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b4009400 con 0x7f90c4099ad0 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90c37fe640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90c4098a00 0x7f90c4159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90c3fff640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90c415c350 0x7f90c40c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90c2ffd640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90c4099ad0 0x7f90c40c8170 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f90b800c780 tx=0x7f90b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90abfff640 1 -- 172.21.15.47:0/2338697439 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b80072d0 con 0x7f90c4099ad0 2024-03-23T09:52:32.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90abfff640 1 -- 172.21.15.47:0/2338697439 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b8007450 con 0x7f90c4099ad0 2024-03-23T09:52:32.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90ca20b640 1 -- 172.21.15.47:0/2338697439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c40c9190 con 0x7f90c4099ad0 2024-03-23T09:52:32.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90abfff640 1 -- 172.21.15.47:0/2338697439 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8001d10 con 0x7f90c4099ad0 2024-03-23T09:52:32.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.054+0000 7f90ca20b640 1 -- 172.21.15.47:0/2338697439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c40c9750 con 0x7f90c4099ad0 2024-03-23T09:52:32.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.055+0000 7f90ca20b640 1 -- 172.21.15.47:0/2338697439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c406c5f0 con 0x7f90c4099ad0 2024-03-23T09:52:32.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.055+0000 7f90abfff640 1 -- 172.21.15.47:0/2338697439 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f90b80047e0 con 0x7f90c4099ad0 2024-03-23T09:52:32.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.056+0000 7f90abfff640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90b006ac00 0x7f90b006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:32.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.056+0000 7f90abfff640 1 -- 172.21.15.47:0/2338697439 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f90b8088800 con 0x7f90c4099ad0 2024-03-23T09:52:32.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.056+0000 7f90c37fe640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90b006ac00 0x7f90b006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:32.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.057+0000 7f90c37fe640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90b006ac00 0x7f90b006d0c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f90b4036000 tx=0x7f90b403b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:32.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.060+0000 7f90abfff640 1 -- 172.21.15.47:0/2338697439 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f90b8053850 con 0x7f90c4099ad0 2024-03-23T09:52:32.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.279+0000 7f90ca20b640 1 -- 172.21.15.47:0/2338697439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f90c40cf030 con 0x7f90c4099ad0 2024-03-23T09:52:32.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.280+0000 7f90abfff640 1 -- 172.21.15.47:0/2338697439 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v93) v1 ==== 53+0+73 (secure 0 0 0) 0x7f90b8018060 con 0x7f90c4099ad0 2024-03-23T09:52:32.280 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T09:52:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.281+0000 7f90ca20b640 1 -- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90b006ac00 msgr2=0x7f90b006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.281+0000 7f90ca20b640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90b006ac00 0x7f90b006d0c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f90b4036000 tx=0x7f90b403b040 comp rx=0 tx=0).stop 2024-03-23T09:52:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.281+0000 7f90ca20b640 1 -- 172.21.15.47:0/2338697439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90c4099ad0 msgr2=0x7f90c40c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.281+0000 7f90ca20b640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90c4099ad0 0x7f90c40c8170 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f90b800c780 tx=0x7f90b800cc40 comp rx=0 tx=0).stop 2024-03-23T09:52:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.281+0000 7f90ca20b640 1 -- 172.21.15.47:0/2338697439 shutdown_connections 2024-03-23T09:52:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.281+0000 7f90ca20b640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f90b006ac00 0x7f90b006d0c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.281+0000 7f90ca20b640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90c415c350 0x7f90c40c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.281+0000 7f90ca20b640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90c4099ad0 0x7f90c40c8170 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.281+0000 7f90ca20b640 1 --2- 172.21.15.47:0/2338697439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90c4098a00 0x7f90c4159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.281+0000 7f90ca20b640 1 -- 172.21.15.47:0/2338697439 >> 172.21.15.47:0/2338697439 conn(0x7f90c40b21d0 msgr2=0x7f90c415b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.282+0000 7f90ca20b640 1 -- 172.21.15.47:0/2338697439 shutdown_connections 2024-03-23T09:52:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:32.282+0000 7f90ca20b640 1 -- 172.21.15.47:0/2338697439 wait complete. 2024-03-23T09:52:32.292 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T09:52:32.292 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T09:52:32.292 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-23T09:52:32.292 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T09:52:32.293 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:32.293 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T09:52:32.293 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T09:52:32.293 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T09:52:32.293 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:52:32.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:32.293 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-23T09:52:32.308 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T09:52:32.308 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T09:52:32.308 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T09:52:32.308 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:32.308 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-23T09:52:32.308 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 41986sec preferred_lft 41986sec 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T09:52:32.309 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::185a:beff:feec:53de/64 scope link 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 56:34:7d:43:a9:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::5434:7dff:fe43:a9be/64 scope link 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1e:e8:4a:e9:f7:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::1ce8:4aff:fee9:f7bb/64 scope link 2024-03-23T09:52:32.310 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-23T09:52:32.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:32.311 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T09:52:32.374 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:52:32.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:32.376 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-23T09:52:32.431 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-23T09:52:32.431 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T09:52:32.431 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T09:52:32.431 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T09:52:32.431 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T09:52:32.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:32.431 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-23T09:52:32.431 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-23T09:52:32.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T09:52:32.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T09:52:32.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T09:52:32.432 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-23T09:52:32.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:32.526 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:52:32.540 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:52:32.540 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:52:32.541 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:52:32.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:32.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-23T09:52:32.607 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T09:52:32.607 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:52:32.608 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:32.662 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T09:52:32.662 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T09:52:32.663 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:32.717 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:52:32.783 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:52:32.784 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:52:32.850 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:52:32.850 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:52:32.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:32.851 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:52:32.905 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T09:52:32.905 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T09:52:32.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-23T09:52:32.973 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T09:52:32.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T09:52:33.032 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-23T09:52:33.032+0000 7f4df304b480 -1 init, newargv = 0x5558b47c52b0 newargc=15 2024-03-23T09:52:33.032 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[83419]: starting ceph client 2024-03-23T09:52:33.036 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[83419]: starting fuse 2024-03-23T09:52:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T09:52:33.043 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T09:52:33.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:33.043 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-23T09:52:33.098 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-03-23T09:52:33.099 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-23T09:52:33.099 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:33.099 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-23T09:52:33.099 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-23T09:52:33.099 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:33.099 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-23T09:52:33.099 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:33.099 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-23T09:52:33.099 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T09:52:33.099 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-23T09:52:33.099 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T09:52:33.099 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:33.099 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-23T09:52:33.099 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-23T09:52:33.100 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:33.215 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.83419.asok 2024-03-23T09:52:33.401 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.83419.asok 2024-03-23T09:52:33.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:33.401 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.83419.asok status 2024-03-23T09:52:33.539 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:52:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-23T09:52:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T09:52:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T09:52:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-23T09:52:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-23T09:52:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T09:52:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "83419", 2024-03-23T09:52:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-23T09:52:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:52:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-23T09:52:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-23T09:52:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 9876, 2024-03-23T09:52:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 9876 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1975500040 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-23T09:52:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1975500040 2024-03-23T09:52:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-23T09:52:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.9876 192.168.0.1:0/1975500040", 2024-03-23T09:52:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/1975500040", 2024-03-23T09:52:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-23T09:52:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 93, 2024-03-23T09:52:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 118, 2024-03-23T09:52:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-23T09:52:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-23T09:52:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-23T09:52:33.542 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:52:33.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:33.548 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-23T09:52:33.548 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:33.563 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-23T09:52:33.563 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:33.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:33.563 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:33.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:33.630 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-23T09:52:33.693 INFO:teuthology.orchestra.run.smithi047.stdout:./ 2024-03-23T09:52:33.693 INFO:teuthology.orchestra.run.smithi047.stdout:./parent 2024-03-23T09:52:33.694 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/flushed 2024-03-23T09:52:33.694 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/flushed/charlie 2024-03-23T09:52:33.694 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/flushed/alpha 2024-03-23T09:52:33.694 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/flushed/bravo 2024-03-23T09:52:33.694 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/unflushed 2024-03-23T09:52:33.694 INFO:teuthology.orchestra.run.smithi047.stdout:./parent/unflushed/jfile 2024-03-23T09:52:33.694 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:33.694 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:33.694 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:33.694 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:33.694 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-23T09:52:33.694 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:33.794 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-23T09:52:33.984 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:33.984 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:33.984 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:33.984 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:33.984 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-23T09:52:33.984 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:34.086 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627776 2024-03-23T09:52:34.274 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:34.274 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:34.274 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:34.274 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:34.275 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-23T09:52:34.275 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:34.376 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627777 2024-03-23T09:52:34.565 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:34.565 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:34.565 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:34.565 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:34.565 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-23T09:52:34.565 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:34.667 INFO:teuthology.orchestra.run.smithi047.stdout:1099511628280 2024-03-23T09:52:34.856 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:34.856 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:34.856 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:34.857 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:34.857 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-23T09:52:34.857 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:34.958 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627778 2024-03-23T09:52:35.147 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:35.147 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:35.147 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:35.147 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:35.147 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-03-23T09:52:35.147 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:35.249 INFO:teuthology.orchestra.run.smithi047.stdout:1099511628279 2024-03-23T09:52:35.438 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:35.438 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:35.438 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:35.439 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:35.439 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-23T09:52:35.439 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:35.541 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627779 2024-03-23T09:52:35.730 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T09:52:35.730 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-23T09:52:35.731 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-03-23T09:52:35.731 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-23T09:52:35.731 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-23T09:52:35.731 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-23T09:52:35.832 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627780 2024-03-23T09:52:36.021 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T09:52:36.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:52:36.108 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T09:52:36.162 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T09:52:36.163 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:52:36.163 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:52:36.178 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:52:36.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.317+0000 7f056ba42640 1 Processor -- start 2024-03-23T09:52:36.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.318+0000 7f056ba42640 1 -- start start 2024-03-23T09:52:36.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.318+0000 7f056ba42640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f05640b8350 0x7f05640cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.318+0000 7f056ba42640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05640ce090 con 0x7f05640cdc40 2024-03-23T09:52:36.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.318+0000 7f056ba42640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05640d11e0 con 0x7f056415ca90 2024-03-23T09:52:36.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.318+0000 7f056ba42640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0564160f60 con 0x7f05640b8350 2024-03-23T09:52:36.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0569fdf640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f05640cdc40 0x7f05640d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33584/0 (socket says 172.21.15.47:33584) 2024-03-23T09:52:36.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0569fdf640 1 -- 172.21.15.47:0/1643257231 learned_addr learned my addr 172.21.15.47:0/1643257231 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:36.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0568fdd640 1 --2- 172.21.15.47:0/1643257231 >> v2:172.21.15.175:3300/0 conn(0x7f05640b8350 0x7f05640cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919322313 0 0) 0x7f0554001490 con 0x7f05640cdc40 2024-03-23T09:52:36.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f054c003520 con 0x7f05640cdc40 2024-03-23T09:52:36.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1307089137 0 0) 0x7f054c003520 con 0x7f05640cdc40 2024-03-23T09:52:36.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 >> v2:172.21.15.175:3300/0 conn(0x7f05640b8350 msgr2=0x7f05640cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 --2- 172.21.15.47:0/1643257231 >> v2:172.21.15.175:3300/0 conn(0x7f05640b8350 0x7f05640cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 >> v1:172.21.15.175:6789/0 conn(0x7f056415ca90 legacy=0x7f0564160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0564161240 con 0x7f05640cdc40 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f0554001b80 con 0x7f05640cdc40 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 >> v1:172.21.15.47:6789/0 conn(0x7f05640cdc40 legacy=0x7f05640d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 --2- 172.21.15.47:0/1643257231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c0039c0 0x7f054c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 --2- 172.21.15.47:0/1643257231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f054c0047c0 0x7f054c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 --2- 172.21.15.47:0/1643257231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f054c0051d0 0x7f054c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0564160f60 con 0x7f054c0039c0 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c00c120 con 0x7f054c0047c0 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c00c2e0 con 0x7f054c0051d0 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f05697de640 1 --2- 172.21.15.47:0/1643257231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c0039c0 0x7f054c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0569fdf640 1 --2- 172.21.15.47:0/1643257231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f054c0051d0 0x7f054c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f0568fdd640 1 --2- 172.21.15.47:0/1643257231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f054c0047c0 0x7f054c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.319+0000 7f05697de640 1 -- 172.21.15.47:0/1643257231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f054c0051d0 msgr2=0x7f054c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.320+0000 7f05697de640 1 --2- 172.21.15.47:0/1643257231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f054c0051d0 0x7f054c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.320+0000 7f05697de640 1 -- 172.21.15.47:0/1643257231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f054c0047c0 msgr2=0x7f054c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.320+0000 7f05697de640 1 --2- 172.21.15.47:0/1643257231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f054c0047c0 0x7f054c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.320+0000 7f05697de640 1 -- 172.21.15.47:0/1643257231 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c003520 con 0x7f054c0039c0 2024-03-23T09:52:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.320+0000 7f0569fdf640 1 -- 172.21.15.47:0/1643257231 reap_dead start 2024-03-23T09:52:36.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.320+0000 7f05697de640 1 --2- 172.21.15.47:0/1643257231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c0039c0 0x7f054c003de0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f055c001720 tx=0x7f054c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa53411af66a551 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:36.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.320+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c0052d0 con 0x7f054c0039c0 2024-03-23T09:52:36.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.320+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c02e050 con 0x7f054c0039c0 2024-03-23T09:52:36.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.320+0000 7f0563fff640 1 -- 172.21.15.47:0/1643257231 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c003830 con 0x7f054c0039c0 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.321+0000 7f056ba42640 1 -- 172.21.15.47:0/1643257231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c0039c0 msgr2=0x7f054c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.321+0000 7f056ba42640 1 --2- 172.21.15.47:0/1643257231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c0039c0 0x7f054c003de0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f055c001720 tx=0x7f054c0057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.321+0000 7f056ba42640 1 -- 172.21.15.47:0/1643257231 shutdown_connections 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.321+0000 7f056ba42640 1 --2- 172.21.15.47:0/1643257231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c0039c0 0x7f054c003de0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.321+0000 7f056ba42640 1 -- 172.21.15.47:0/1643257231 >> 172.21.15.47:0/1643257231 conn(0x7f05640659d0 msgr2=0x7f0564065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.321+0000 7f056ba42640 1 -- 172.21.15.47:0/1643257231 shutdown_connections 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.321+0000 7f056ba42640 1 -- 172.21.15.47:0/1643257231 wait complete. 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.322+0000 7f056ba42640 1 Processor -- start 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.322+0000 7f056ba42640 1 -- start start 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f056ba42640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f054c0039c0 0x7f0564150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f056ba42640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05640cf200 0x7f05640cf680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f056ba42640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05640cbd50 0x7f05640cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f056ba42640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05640b4780 con 0x7f05640cbd50 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f056ba42640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05640cfbc0 con 0x7f054c0039c0 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f056ba42640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05640b3b00 con 0x7f05640cf200 2024-03-23T09:52:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f0569fdf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05640cbd50 0x7f05640cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f0569fdf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05640cbd50 0x7f05640cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48208/0 (socket says 172.21.15.47:48208) 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f0569fdf640 1 -- 172.21.15.47:0/4225952543 learned_addr learned my addr 172.21.15.47:0/4225952543 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f0568fdd640 1 --2- 172.21.15.47:0/4225952543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05640cf200 0x7f05640cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f0569fdf640 1 -- 172.21.15.47:0/4225952543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05640cf200 msgr2=0x7f05640cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f0569fdf640 1 --2- 172.21.15.47:0/4225952543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05640cf200 0x7f05640cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f0569fdf640 1 -- 172.21.15.47:0/4225952543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f054c0039c0 msgr2=0x7f0564150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f0569fdf640 1 --2- 172.21.15.47:0/4225952543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f054c0039c0 0x7f0564150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f0569fdf640 1 -- 172.21.15.47:0/4225952543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05640b3de0 con 0x7f05640cbd50 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.323+0000 7f0569fdf640 1 --2- 172.21.15.47:0/4225952543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05640cbd50 0x7f05640cc190 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f0564161800 tx=0x7f0554007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.324+0000 7f0561ffb640 1 -- 172.21.15.47:0/4225952543 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055400e9e0 con 0x7f05640cbd50 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.324+0000 7f0561ffb640 1 -- 172.21.15.47:0/4225952543 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055400eb60 con 0x7f05640cbd50 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.324+0000 7f056ba42640 1 -- 172.21.15.47:0/4225952543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05640b40c0 con 0x7f05640cbd50 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.324+0000 7f0561ffb640 1 -- 172.21.15.47:0/4225952543 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554013420 con 0x7f05640cbd50 2024-03-23T09:52:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.324+0000 7f056ba42640 1 -- 172.21.15.47:0/4225952543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0564150fa0 con 0x7f05640cbd50 2024-03-23T09:52:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.325+0000 7f056ba42640 1 -- 172.21.15.47:0/4225952543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0564072360 con 0x7f05640cbd50 2024-03-23T09:52:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.325+0000 7f0561ffb640 1 -- 172.21.15.47:0/4225952543 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0554013580 con 0x7f05640cbd50 2024-03-23T09:52:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.326+0000 7f0561ffb640 1 --2- 172.21.15.47:0/4225952543 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f055006ac00 0x7f055006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.326+0000 7f0561ffb640 1 -- 172.21.15.47:0/4225952543 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f0554089650 con 0x7f05640cbd50 2024-03-23T09:52:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.326+0000 7f05697de640 1 --2- 172.21.15.47:0/4225952543 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f055006ac00 0x7f055006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.326+0000 7f05697de640 1 --2- 172.21.15.47:0/4225952543 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f055006ac00 0x7f055006d0c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f055c033000 tx=0x7f055c02f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.330+0000 7f0561ffb640 1 -- 172.21.15.47:0/4225952543 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f05540546a0 con 0x7f05640cbd50 2024-03-23T09:52:36.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.513+0000 7f056ba42640 1 -- 172.21.15.47:0/4225952543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05640ce090 con 0x7f05640cbd50 2024-03-23T09:52:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.514+0000 7f0561ffb640 1 -- 172.21.15.47:0/4225952543 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f05640ce090 con 0x7f05640cbd50 2024-03-23T09:52:36.515 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:36.515 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":118,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:52:25.484824+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:51:38.679890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-23T09:51:39.396299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6832/1868958629":"2024-03-24T09:52:25.484801+0000","172.21.15.47:6834/724782441":"2024-03-24T09:52:05.460377+0000","172.21.15.47:6834/1868958629":"2024-03-24T09:52:25.484801+0000","172.21.15.47:6832/724782441":"2024-03-24T09:52:05.460377+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:52:36.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.516+0000 7f056ba42640 1 -- 172.21.15.47:0/4225952543 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f055006ac00 msgr2=0x7f055006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.516+0000 7f056ba42640 1 --2- 172.21.15.47:0/4225952543 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f055006ac00 0x7f055006d0c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f055c033000 tx=0x7f055c02f000 comp rx=0 tx=0).stop 2024-03-23T09:52:36.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.516+0000 7f056ba42640 1 -- 172.21.15.47:0/4225952543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05640cbd50 msgr2=0x7f05640cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.516+0000 7f056ba42640 1 --2- 172.21.15.47:0/4225952543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05640cbd50 0x7f05640cc190 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f0564161800 tx=0x7f0554007a80 comp rx=0 tx=0).stop 2024-03-23T09:52:36.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.516+0000 7f056ba42640 1 -- 172.21.15.47:0/4225952543 shutdown_connections 2024-03-23T09:52:36.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.516+0000 7f056ba42640 1 --2- 172.21.15.47:0/4225952543 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f055006ac00 0x7f055006d0c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.516+0000 7f056ba42640 1 --2- 172.21.15.47:0/4225952543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05640cbd50 0x7f05640cc190 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.516+0000 7f056ba42640 1 --2- 172.21.15.47:0/4225952543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05640cf200 0x7f05640cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.516+0000 7f056ba42640 1 --2- 172.21.15.47:0/4225952543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f054c0039c0 0x7f0564150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.516+0000 7f056ba42640 1 -- 172.21.15.47:0/4225952543 >> 172.21.15.47:0/4225952543 conn(0x7f05640659d0 msgr2=0x7f05641571e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:36.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.516+0000 7f056ba42640 1 -- 172.21.15.47:0/4225952543 shutdown_connections 2024-03-23T09:52:36.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.516+0000 7f056ba42640 1 -- 172.21.15.47:0/4225952543 wait complete. 2024-03-23T09:52:36.527 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:51:38.679890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:51:39.396299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:52:36.527 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:52:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.631+0000 7f49329a4640 1 Processor -- start 2024-03-23T09:52:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.631+0000 7f49329a4640 1 -- start start 2024-03-23T09:52:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f49329a4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f492c0cde10 0x7f492c160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f49329a4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f492c0b8350 0x7f492c0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f49329a4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c161630 con 0x7f492c15c990 2024-03-23T09:52:36.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f49329a4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492c0d1030 con 0x7f492c0cde10 2024-03-23T09:52:36.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f49329a4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492c0cfe10 con 0x7f492c0b8350 2024-03-23T09:52:36.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f492b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f492c0cde10 0x7f492c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f492b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f492c0cde10 0x7f492c160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48214/0 (socket says 172.21.15.47:48214) 2024-03-23T09:52:36.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f492b7fe640 1 -- 172.21.15.47:0/1186170505 learned_addr learned my addr 172.21.15.47:0/1186170505 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:36.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f4930f41640 1 --2- 172.21.15.47:0/1186170505 >> v2:172.21.15.175:3300/0 conn(0x7f492c0b8350 0x7f492c0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f492b7fe640 1 -- 172.21.15.47:0/1186170505 >> v2:172.21.15.175:3300/0 conn(0x7f492c0b8350 msgr2=0x7f492c0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:36.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f492b7fe640 1 --2- 172.21.15.47:0/1186170505 >> v2:172.21.15.175:3300/0 conn(0x7f492c0b8350 0x7f492c0d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f492b7fe640 1 -- 172.21.15.47:0/1186170505 >> v1:172.21.15.175:6789/0 conn(0x7f492c15c990 legacy=0x7f492c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:36.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f492b7fe640 1 -- 172.21.15.47:0/1186170505 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c0b5b60 con 0x7f492c0cde10 2024-03-23T09:52:36.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.632+0000 7f4930f41640 1 --2- 172.21.15.47:0/1186170505 >> v2:172.21.15.175:3300/0 conn(0x7f492c0b8350 0x7f492c0d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.633+0000 7f492b7fe640 1 --2- 172.21.15.47:0/1186170505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c0cde10 0x7f492c160690 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f492c0d0fa0 tx=0x7f491c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7183c064bb8b08e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.633+0000 7f492affd640 1 -- 172.21.15.47:0/1186170505 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c0317e0 con 0x7f492c0cde10 2024-03-23T09:52:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.633+0000 7f492affd640 1 -- 172.21.15.47:0/1186170505 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c031940 con 0x7f492c0cde10 2024-03-23T09:52:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.633+0000 7f492affd640 1 -- 172.21.15.47:0/1186170505 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c031cd0 con 0x7f492c0cde10 2024-03-23T09:52:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.634+0000 7f49329a4640 1 -- 172.21.15.47:0/1186170505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c0cde10 msgr2=0x7f492c160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.634+0000 7f49329a4640 1 --2- 172.21.15.47:0/1186170505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c0cde10 0x7f492c160690 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f492c0d0fa0 tx=0x7f491c02b880 comp rx=0 tx=0).stop 2024-03-23T09:52:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.634+0000 7f49329a4640 1 -- 172.21.15.47:0/1186170505 shutdown_connections 2024-03-23T09:52:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.634+0000 7f49329a4640 1 --2- 172.21.15.47:0/1186170505 >> v2:172.21.15.175:3300/0 conn(0x7f492c0b8350 0x7f492c0d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.634+0000 7f49329a4640 1 --2- 172.21.15.47:0/1186170505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c0cde10 0x7f492c160690 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.634+0000 7f49329a4640 1 -- 172.21.15.47:0/1186170505 >> 172.21.15.47:0/1186170505 conn(0x7f492c0659d0 msgr2=0x7f492c065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.634+0000 7f49329a4640 1 -- 172.21.15.47:0/1186170505 shutdown_connections 2024-03-23T09:52:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.634+0000 7f49329a4640 1 -- 172.21.15.47:0/1186170505 wait complete. 2024-03-23T09:52:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.635+0000 7f49329a4640 1 Processor -- start 2024-03-23T09:52:36.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.635+0000 7f49329a4640 1 -- start start 2024-03-23T09:52:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f49329a4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f492c0b8350 0x7f492c150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f49329a4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c0cde10 0x7f492c0b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f49329a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f492c15c990 0x7f492c0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f49329a4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c151010 con 0x7f492c0cde10 2024-03-23T09:52:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f49329a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c15ead0 con 0x7f492c15c990 2024-03-23T09:52:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f49329a4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c15edb0 con 0x7f492c0b8350 2024-03-23T09:52:36.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f492b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c0cde10 0x7f492c0b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f492b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c0cde10 0x7f492c0b6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48222/0 (socket says 172.21.15.47:48222) 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f492b7fe640 1 -- 172.21.15.47:0/899467055 learned_addr learned my addr 172.21.15.47:0/899467055 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f492bfff640 1 --2- 172.21.15.47:0/899467055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f492c0b8350 0x7f492c150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f492b7fe640 1 -- 172.21.15.47:0/899467055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f492c0b8350 msgr2=0x7f492c150ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f492b7fe640 1 --2- 172.21.15.47:0/899467055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f492c0b8350 0x7f492c150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f492b7fe640 1 -- 172.21.15.47:0/899467055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f492c15c990 msgr2=0x7f492c0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f492b7fe640 1 --2- 172.21.15.47:0/899467055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f492c15c990 0x7f492c0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f492b7fe640 1 -- 172.21.15.47:0/899467055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c0094d0 con 0x7f492c0cde10 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f492b7fe640 1 --2- 172.21.15.47:0/899467055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c0cde10 0x7f492c0b6ea0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f491c000c00 tx=0x7f491c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f4928ff9640 1 -- 172.21.15.47:0/899467055 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c032980 con 0x7f492c0cde10 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f4928ff9640 1 -- 172.21.15.47:0/899467055 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c032ae0 con 0x7f492c0cde10 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f4928ff9640 1 -- 172.21.15.47:0/899467055 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c032e90 con 0x7f492c0cde10 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.636+0000 7f49329a4640 1 -- 172.21.15.47:0/899467055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c15f090 con 0x7f492c0cde10 2024-03-23T09:52:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.637+0000 7f49329a4640 1 -- 172.21.15.47:0/899467055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c15f650 con 0x7f492c0cde10 2024-03-23T09:52:36.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.637+0000 7f49329a4640 1 -- 172.21.15.47:0/899467055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c072360 con 0x7f492c0cde10 2024-03-23T09:52:36.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.638+0000 7f4928ff9640 1 -- 172.21.15.47:0/899467055 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f491c0453f0 con 0x7f492c0cde10 2024-03-23T09:52:36.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.638+0000 7f4928ff9640 1 --2- 172.21.15.47:0/899467055 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f491806ac00 0x7f491806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.638+0000 7f4928ff9640 1 -- 172.21.15.47:0/899467055 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f491c0abb30 con 0x7f492c0cde10 2024-03-23T09:52:36.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.639+0000 7f492bfff640 1 --2- 172.21.15.47:0/899467055 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f491806ac00 0x7f491806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.639+0000 7f492bfff640 1 --2- 172.21.15.47:0/899467055 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f491806ac00 0x7f491806d0c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f49200015d0 tx=0x7f4920009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.643+0000 7f4928ff9640 1 -- 172.21.15.47:0/899467055 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f491c076dd0 con 0x7f492c0cde10 2024-03-23T09:52:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.822+0000 7f49329a4640 1 -- 172.21.15.47:0/899467055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f492c15eef0 con 0x7f492c0cde10 2024-03-23T09:52:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.823+0000 7f4928ff9640 1 -- 172.21.15.47:0/899467055 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f492c15eef0 con 0x7f492c0cde10 2024-03-23T09:52:36.823 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:52:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.825+0000 7f49329a4640 1 -- 172.21.15.47:0/899467055 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f491806ac00 msgr2=0x7f491806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.825+0000 7f49329a4640 1 --2- 172.21.15.47:0/899467055 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f491806ac00 0x7f491806d0c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f49200015d0 tx=0x7f4920009040 comp rx=0 tx=0).stop 2024-03-23T09:52:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.825+0000 7f49329a4640 1 -- 172.21.15.47:0/899467055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c0cde10 msgr2=0x7f492c0b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.825+0000 7f49329a4640 1 --2- 172.21.15.47:0/899467055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c0cde10 0x7f492c0b6ea0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f491c000c00 tx=0x7f491c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.825+0000 7f49329a4640 1 -- 172.21.15.47:0/899467055 shutdown_connections 2024-03-23T09:52:36.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.825+0000 7f49329a4640 1 --2- 172.21.15.47:0/899467055 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f491806ac00 0x7f491806d0c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.825+0000 7f49329a4640 1 --2- 172.21.15.47:0/899467055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f492c15c990 0x7f492c0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.825+0000 7f49329a4640 1 --2- 172.21.15.47:0/899467055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c0cde10 0x7f492c0b6ea0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.825+0000 7f49329a4640 1 --2- 172.21.15.47:0/899467055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f492c0b8350 0x7f492c150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.825+0000 7f49329a4640 1 -- 172.21.15.47:0/899467055 >> 172.21.15.47:0/899467055 conn(0x7f492c0659d0 msgr2=0x7f492c156fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:36.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.825+0000 7f49329a4640 1 -- 172.21.15.47:0/899467055 shutdown_connections 2024-03-23T09:52:36.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.825+0000 7f49329a4640 1 -- 172.21.15.47:0/899467055 wait complete. 2024-03-23T09:52:36.836 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:52:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.940+0000 7f43d7500640 1 Processor -- start 2024-03-23T09:52:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.941+0000 7f43d7500640 1 -- start start 2024-03-23T09:52:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.941+0000 7f43d7500640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43d006cc50 0x7f43d0071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.941+0000 7f43d7500640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d0150410 con 0x7f43d006d030 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.941+0000 7f43d7500640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d00acd50 con 0x7f43d00ac6f0 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.941+0000 7f43d7500640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d006b770 con 0x7f43d006cc50 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43d4a9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f43d006d030 0x7f43d00abfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33596/0 (socket says 172.21.15.47:33596) 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43d4a9b640 1 -- 172.21.15.47:0/2815961749 learned_addr learned my addr 172.21.15.47:0/2815961749 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43d529c640 1 --2- 172.21.15.47:0/2815961749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43d006cc50 0x7f43d0071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313948672 0 0) 0x7f43d0150410 con 0x7f43d006d030 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43b8003520 con 0x7f43d006d030 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 601455034 0 0) 0x7f43b8003520 con 0x7f43d006d030 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43d006cc50 msgr2=0x7f43d0071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 --2- 172.21.15.47:0/2815961749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43d006cc50 0x7f43d0071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 >> v1:172.21.15.175:6789/0 conn(0x7f43d00ac6f0 legacy=0x7f43d00b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d0070c90 con 0x7f43d006d030 2024-03-23T09:52:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f43c4001670 con 0x7f43d006d030 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 >> v1:172.21.15.47:6789/0 conn(0x7f43d006d030 legacy=0x7f43d00abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 --2- 172.21.15.47:0/2815961749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43b80039a0 0x7f43b8003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 --2- 172.21.15.47:0/2815961749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43b8004350 0x7f43b8004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 --2- 172.21.15.47:0/2815961749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43d006cc50 0x7f43b8005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d006b770 con 0x7f43b80039a0 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b800b980 con 0x7f43b8004350 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b800bb40 con 0x7f43d006cc50 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43d529c640 1 --2- 172.21.15.47:0/2815961749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43b80039a0 0x7f43b8003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43d4a9b640 1 --2- 172.21.15.47:0/2815961749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43b8004350 0x7f43b8004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.942+0000 7f43d5a9d640 1 --2- 172.21.15.47:0/2815961749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43d006cc50 0x7f43b8005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.943+0000 7f43d529c640 1 -- 172.21.15.47:0/2815961749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43d006cc50 msgr2=0x7f43b8005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.943+0000 7f43d529c640 1 --2- 172.21.15.47:0/2815961749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43d006cc50 0x7f43b8005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.943+0000 7f43d529c640 1 -- 172.21.15.47:0/2815961749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43b8004350 msgr2=0x7f43b8004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.943+0000 7f43d529c640 1 --2- 172.21.15.47:0/2815961749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43b8004350 0x7f43b8004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.943+0000 7f43d529c640 1 -- 172.21.15.47:0/2815961749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b8003520 con 0x7f43b80039a0 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.943+0000 7f43d529c640 1 --2- 172.21.15.47:0/2815961749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43b80039a0 0x7f43b8003d80 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f43c0010350 tx=0x7f43b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beea104ff6047825 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.943+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c002ee20 con 0x7f43b80039a0 2024-03-23T09:52:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.944+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c0032070 con 0x7f43b80039a0 2024-03-23T09:52:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.944+0000 7f43cffff640 1 -- 172.21.15.47:0/2815961749 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0038640 con 0x7f43b80039a0 2024-03-23T09:52:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.944+0000 7f43d7500640 1 -- 172.21.15.47:0/2815961749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43b80039a0 msgr2=0x7f43b8003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.944+0000 7f43d7500640 1 --2- 172.21.15.47:0/2815961749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43b80039a0 0x7f43b8003d80 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f43c0010350 tx=0x7f43b80057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.944+0000 7f43d5a9d640 1 -- 172.21.15.47:0/2815961749 reap_dead start 2024-03-23T09:52:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.944+0000 7f43d7500640 1 -- 172.21.15.47:0/2815961749 shutdown_connections 2024-03-23T09:52:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.944+0000 7f43d7500640 1 -- 172.21.15.47:0/2815961749 >> 172.21.15.47:0/2815961749 conn(0x7f43d00b0bb0 msgr2=0x7f43d00b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.944+0000 7f43d7500640 1 -- 172.21.15.47:0/2815961749 shutdown_connections 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.944+0000 7f43d7500640 1 -- 172.21.15.47:0/2815961749 wait complete. 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.945+0000 7f43d7500640 1 Processor -- start 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.945+0000 7f43d7500640 1 -- start start 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d7500640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43b80039a0 0x7f43d014fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d7500640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43d009abb0 0x7f43d01e3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d7500640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43d014eda0 0x7f43d014f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d7500640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d009cf20 con 0x7f43d009abb0 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d7500640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d009bc80 con 0x7f43d014eda0 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d7500640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d014f750 con 0x7f43b80039a0 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d4a9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43d009abb0 0x7f43d01e3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d4a9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43d009abb0 0x7f43d01e3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48232/0 (socket says 172.21.15.47:48232) 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d4a9b640 1 -- 172.21.15.47:0/805775214 learned_addr learned my addr 172.21.15.47:0/805775214 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d4a9b640 1 -- 172.21.15.47:0/805775214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43b80039a0 msgr2=0x7f43d014fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d529c640 1 --2- 172.21.15.47:0/805775214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43b80039a0 0x7f43d014fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d4a9b640 1 --2- 172.21.15.47:0/805775214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43b80039a0 0x7f43d014fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d4a9b640 1 -- 172.21.15.47:0/805775214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43d014eda0 msgr2=0x7f43d014f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d4a9b640 1 --2- 172.21.15.47:0/805775214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43d014eda0 0x7f43d014f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.946+0000 7f43d4a9b640 1 -- 172.21.15.47:0/805775214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d009b070 con 0x7f43d009abb0 2024-03-23T09:52:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.947+0000 7f43d4a9b640 1 --2- 172.21.15.47:0/805775214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43d009abb0 0x7f43d01e3b90 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f43c4007290 tx=0x7f43c4007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.947+0000 7f43cdffb640 1 -- 172.21.15.47:0/805775214 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400f8e0 con 0x7f43d009abb0 2024-03-23T09:52:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.947+0000 7f43cdffb640 1 -- 172.21.15.47:0/805775214 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c400fa40 con 0x7f43d009abb0 2024-03-23T09:52:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.947+0000 7f43d7500640 1 -- 172.21.15.47:0/805775214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d009b350 con 0x7f43d009abb0 2024-03-23T09:52:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.947+0000 7f43cdffb640 1 -- 172.21.15.47:0/805775214 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400fdf0 con 0x7f43d009abb0 2024-03-23T09:52:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.947+0000 7f43d7500640 1 -- 172.21.15.47:0/805775214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d009b910 con 0x7f43d009abb0 2024-03-23T09:52:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.948+0000 7f43d7500640 1 -- 172.21.15.47:0/805775214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d009be10 con 0x7f43d009abb0 2024-03-23T09:52:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.948+0000 7f43cdffb640 1 -- 172.21.15.47:0/805775214 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f43c401d460 con 0x7f43d009abb0 2024-03-23T09:52:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.949+0000 7f43cdffb640 1 --2- 172.21.15.47:0/805775214 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f43bc06ac30 0x7f43bc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.949+0000 7f43cdffb640 1 -- 172.21.15.47:0/805775214 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f43c4089560 con 0x7f43d009abb0 2024-03-23T09:52:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.949+0000 7f43d529c640 1 --2- 172.21.15.47:0/805775214 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f43bc06ac30 0x7f43bc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.949+0000 7f43d529c640 1 --2- 172.21.15.47:0/805775214 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f43bc06ac30 0x7f43bc06d0f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f43c0010350 tx=0x7f43c0030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:36.953+0000 7f43cdffb640 1 -- 172.21.15.47:0/805775214 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f43c40545c0 con 0x7f43d009abb0 2024-03-23T09:52:37.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.132+0000 7f43d7500640 1 -- 172.21.15.47:0/805775214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43d00672f0 con 0x7f43d009abb0 2024-03-23T09:52:37.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.133+0000 7f43cdffb640 1 -- 172.21.15.47:0/805775214 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43c4057be0 con 0x7f43d009abb0 2024-03-23T09:52:37.133 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:52:37.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.135+0000 7f43d7500640 1 -- 172.21.15.47:0/805775214 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f43bc06ac30 msgr2=0x7f43bc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.135+0000 7f43d7500640 1 --2- 172.21.15.47:0/805775214 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f43bc06ac30 0x7f43bc06d0f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f43c0010350 tx=0x7f43c0030000 comp rx=0 tx=0).stop 2024-03-23T09:52:37.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.135+0000 7f43d7500640 1 -- 172.21.15.47:0/805775214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43d009abb0 msgr2=0x7f43d01e3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.135+0000 7f43d7500640 1 --2- 172.21.15.47:0/805775214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43d009abb0 0x7f43d01e3b90 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f43c4007290 tx=0x7f43c4007750 comp rx=0 tx=0).stop 2024-03-23T09:52:37.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.135+0000 7f43d7500640 1 -- 172.21.15.47:0/805775214 shutdown_connections 2024-03-23T09:52:37.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.135+0000 7f43d7500640 1 --2- 172.21.15.47:0/805775214 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f43bc06ac30 0x7f43bc06d0f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.135+0000 7f43d7500640 1 --2- 172.21.15.47:0/805775214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43d014eda0 0x7f43d014f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.135+0000 7f43d7500640 1 --2- 172.21.15.47:0/805775214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43d009abb0 0x7f43d01e3b90 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.135+0000 7f43d7500640 1 --2- 172.21.15.47:0/805775214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43b80039a0 0x7f43d014fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.135+0000 7f43d7500640 1 -- 172.21.15.47:0/805775214 >> 172.21.15.47:0/805775214 conn(0x7f43d00b0bb0 msgr2=0x7f43d00b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:37.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.135+0000 7f43d7500640 1 -- 172.21.15.47:0/805775214 shutdown_connections 2024-03-23T09:52:37.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.135+0000 7f43d7500640 1 -- 172.21.15.47:0/805775214 wait complete. 2024-03-23T09:52:37.146 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:52:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.249+0000 7eff5e763640 1 Processor -- start 2024-03-23T09:52:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.249+0000 7eff5e763640 1 -- start start 2024-03-23T09:52:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff5e763640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7eff5815ce10 0x7eff580cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff5e763640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7eff580b85d0 0x7eff580cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff5e763640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff58161600 con 0x7eff5815ca30 2024-03-23T09:52:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff5e763640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff58160e50 con 0x7eff580b85d0 2024-03-23T09:52:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff5e763640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff580d25e0 con 0x7eff5815ce10 2024-03-23T09:52:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff57fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7eff5815ca30 0x7eff58160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33600/0 (socket says 172.21.15.47:33600) 2024-03-23T09:52:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff57fff640 1 -- 172.21.15.47:0/3923208694 learned_addr learned my addr 172.21.15.47:0/3923208694 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff5cd00640 1 --2- 172.21.15.47:0/3923208694 >> v2:172.21.15.47:3300/0 conn(0x7eff580b85d0 0x7eff580cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff577fe640 1 --2- 172.21.15.47:0/3923208694 >> v2:172.21.15.175:3300/0 conn(0x7eff5815ce10 0x7eff580cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff5cd00640 1 -- 172.21.15.47:0/3923208694 >> v2:172.21.15.175:3300/0 conn(0x7eff5815ce10 msgr2=0x7eff580cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff5cd00640 1 --2- 172.21.15.47:0/3923208694 >> v2:172.21.15.175:3300/0 conn(0x7eff5815ce10 0x7eff580cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff5cd00640 1 -- 172.21.15.47:0/3923208694 >> v1:172.21.15.47:6789/0 conn(0x7eff5815ca30 legacy=0x7eff58160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff5cd00640 1 -- 172.21.15.47:0/3923208694 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff58161040 con 0x7eff580b85d0 2024-03-23T09:52:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.251+0000 7eff577fe640 1 --2- 172.21.15.47:0/3923208694 >> v2:172.21.15.175:3300/0 conn(0x7eff5815ce10 0x7eff580cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.250+0000 7eff56ffd640 1 -- 172.21.15.47:0/3923208694 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082756316 0 0) 0x7eff58161600 con 0x7eff5815ca30 2024-03-23T09:52:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.251+0000 7eff5cd00640 1 --2- 172.21.15.47:0/3923208694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff580b85d0 0x7eff580cdc70 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7eff58066930 tx=0x7eff5002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cedbf4525bc65a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.251+0000 7eff56ffd640 1 -- 172.21.15.47:0/3923208694 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5002cc80 con 0x7eff580b85d0 2024-03-23T09:52:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.251+0000 7eff56ffd640 1 -- 172.21.15.47:0/3923208694 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5002cde0 con 0x7eff580b85d0 2024-03-23T09:52:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.251+0000 7eff56ffd640 1 -- 172.21.15.47:0/3923208694 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50034660 con 0x7eff580b85d0 2024-03-23T09:52:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.252+0000 7eff5e763640 1 -- 172.21.15.47:0/3923208694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff580b85d0 msgr2=0x7eff580cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.252+0000 7eff5e763640 1 --2- 172.21.15.47:0/3923208694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff580b85d0 0x7eff580cdc70 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7eff58066930 tx=0x7eff5002bed0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.252+0000 7eff5e763640 1 -- 172.21.15.47:0/3923208694 shutdown_connections 2024-03-23T09:52:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.252+0000 7eff5e763640 1 --2- 172.21.15.47:0/3923208694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff580b85d0 0x7eff580cdc70 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.252+0000 7eff5e763640 1 --2- 172.21.15.47:0/3923208694 >> v2:172.21.15.175:3300/0 conn(0x7eff5815ce10 0x7eff580cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.252+0000 7eff5e763640 1 -- 172.21.15.47:0/3923208694 >> 172.21.15.47:0/3923208694 conn(0x7eff5806b160 msgr2=0x7eff5806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.252+0000 7eff5e763640 1 -- 172.21.15.47:0/3923208694 shutdown_connections 2024-03-23T09:52:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.252+0000 7eff5e763640 1 -- 172.21.15.47:0/3923208694 wait complete. 2024-03-23T09:52:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.253+0000 7eff5e763640 1 Processor -- start 2024-03-23T09:52:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.253+0000 7eff5e763640 1 -- start start 2024-03-23T09:52:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff5e763640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff580b85d0 0x7eff58155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff5e763640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff5815ca30 0x7eff58149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff5e763640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff5815ce10 0x7eff5815dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff5e763640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff58156650 con 0x7eff580b85d0 2024-03-23T09:52:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff5e763640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5815e150 con 0x7eff5815ce10 2024-03-23T09:52:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff5e763640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff58154ab0 con 0x7eff5815ca30 2024-03-23T09:52:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff57fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff580b85d0 0x7eff58155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff57fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff580b85d0 0x7eff58155b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48264/0 (socket says 172.21.15.47:48264) 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff57fff640 1 -- 172.21.15.47:0/1625677073 learned_addr learned my addr 172.21.15.47:0/1625677073 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff57fff640 1 -- 172.21.15.47:0/1625677073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff5815ca30 msgr2=0x7eff58149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff57fff640 1 --2- 172.21.15.47:0/1625677073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff5815ca30 0x7eff58149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff57fff640 1 -- 172.21.15.47:0/1625677073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff5815ce10 msgr2=0x7eff5815dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff57fff640 1 --2- 172.21.15.47:0/1625677073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff5815ce10 0x7eff5815dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.254+0000 7eff57fff640 1 -- 172.21.15.47:0/1625677073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff50009a70 con 0x7eff580b85d0 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.255+0000 7eff57fff640 1 --2- 172.21.15.47:0/1625677073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff580b85d0 0x7eff58155b50 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00c4d0 tx=0x7eff4c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.255+0000 7eff54ff9640 1 -- 172.21.15.47:0/1625677073 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c0074e0 con 0x7eff580b85d0 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.255+0000 7eff54ff9640 1 -- 172.21.15.47:0/1625677073 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4c007660 con 0x7eff580b85d0 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.255+0000 7eff5e763640 1 -- 172.21.15.47:0/1625677073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff58154d90 con 0x7eff580b85d0 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.255+0000 7eff54ff9640 1 -- 172.21.15.47:0/1625677073 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c004a20 con 0x7eff580b85d0 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.255+0000 7eff5e763640 1 -- 172.21.15.47:0/1625677073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff58155350 con 0x7eff580b85d0 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.256+0000 7eff5e763640 1 -- 172.21.15.47:0/1625677073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff58072360 con 0x7eff580b85d0 2024-03-23T09:52:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.256+0000 7eff54ff9640 1 -- 172.21.15.47:0/1625677073 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7eff4c004b80 con 0x7eff580b85d0 2024-03-23T09:52:37.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.257+0000 7eff54ff9640 1 --2- 172.21.15.47:0/1625677073 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7eff4406ac00 0x7eff4406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:37.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.257+0000 7eff54ff9640 1 -- 172.21.15.47:0/1625677073 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7eff4c088a10 con 0x7eff580b85d0 2024-03-23T09:52:37.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.257+0000 7eff577fe640 1 --2- 172.21.15.47:0/1625677073 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7eff4406ac00 0x7eff4406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:37.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.257+0000 7eff577fe640 1 --2- 172.21.15.47:0/1625677073 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7eff4406ac00 0x7eff4406d0c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7eff48006fd0 tx=0x7eff48008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:37.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.261+0000 7eff54ff9640 1 -- 172.21.15.47:0/1625677073 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7eff4c053b00 con 0x7eff580b85d0 2024-03-23T09:52:37.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.438+0000 7eff5e763640 1 -- 172.21.15.47:0/1625677073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7eff58071f40 con 0x7eff580b85d0 2024-03-23T09:52:37.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.439+0000 7eff54ff9640 1 -- 172.21.15.47:0/1625677073 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+11 (secure 0 0 0) 0x7eff4c018020 con 0x7eff580b85d0 2024-03-23T09:52:37.439 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:52:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.440+0000 7eff5e763640 1 -- 172.21.15.47:0/1625677073 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7eff4406ac00 msgr2=0x7eff4406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.441+0000 7eff5e763640 1 --2- 172.21.15.47:0/1625677073 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7eff4406ac00 0x7eff4406d0c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7eff48006fd0 tx=0x7eff48008040 comp rx=0 tx=0).stop 2024-03-23T09:52:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.441+0000 7eff5e763640 1 -- 172.21.15.47:0/1625677073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff580b85d0 msgr2=0x7eff58155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.441+0000 7eff5e763640 1 --2- 172.21.15.47:0/1625677073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff580b85d0 0x7eff58155b50 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00c4d0 tx=0x7eff4c00c990 comp rx=0 tx=0).stop 2024-03-23T09:52:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.441+0000 7eff5e763640 1 -- 172.21.15.47:0/1625677073 shutdown_connections 2024-03-23T09:52:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.441+0000 7eff5e763640 1 --2- 172.21.15.47:0/1625677073 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7eff4406ac00 0x7eff4406d0c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.441+0000 7eff5e763640 1 --2- 172.21.15.47:0/1625677073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff5815ce10 0x7eff5815dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.441+0000 7eff5e763640 1 --2- 172.21.15.47:0/1625677073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff5815ca30 0x7eff58149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.441+0000 7eff5e763640 1 --2- 172.21.15.47:0/1625677073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff580b85d0 0x7eff58155b50 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.441+0000 7eff5e763640 1 -- 172.21.15.47:0/1625677073 >> 172.21.15.47:0/1625677073 conn(0x7eff5806b160 msgr2=0x7eff5815b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:37.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.441+0000 7eff5e763640 1 -- 172.21.15.47:0/1625677073 shutdown_connections 2024-03-23T09:52:37.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.441+0000 7eff5e763640 1 -- 172.21.15.47:0/1625677073 wait complete. 2024-03-23T09:52:37.452 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T09:52:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.553+0000 7f454173c640 1 Processor -- start 2024-03-23T09:52:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.554+0000 7f454173c640 1 -- start start 2024-03-23T09:52:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.554+0000 7f454173c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453c15cc40 0x7f453c160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.554+0000 7f454173c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f453c160c90 0x7f453c0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.554+0000 7f454173c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c0d09d0 con 0x7f453c0cdbf0 2024-03-23T09:52:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.554+0000 7f454173c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453c0ce060 con 0x7f453c160c90 2024-03-23T09:52:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.554+0000 7f454173c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0b5a80 con 0x7f453c15cc40 2024-03-23T09:52:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.554+0000 7f453a7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f453c160c90 0x7f453c0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.554+0000 7f453b7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f453c0cdbf0 0x7f453c0b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33612/0 (socket says 172.21.15.47:33612) 2024-03-23T09:52:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.554+0000 7f453b7fe640 1 -- 172.21.15.47:0/76029931 learned_addr learned my addr 172.21.15.47:0/76029931 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.555+0000 7f453affd640 1 --2- 172.21.15.47:0/76029931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453c15cc40 0x7f453c160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.555+0000 7f453a7fc640 1 -- 172.21.15.47:0/76029931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453c15cc40 msgr2=0x7f453c160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.555+0000 7f453a7fc640 1 --2- 172.21.15.47:0/76029931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453c15cc40 0x7f453c160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.555+0000 7f453a7fc640 1 -- 172.21.15.47:0/76029931 >> v1:172.21.15.47:6789/0 conn(0x7f453c0cdbf0 legacy=0x7f453c0b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.555+0000 7f453a7fc640 1 -- 172.21.15.47:0/76029931 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c15d020 con 0x7f453c160c90 2024-03-23T09:52:37.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.555+0000 7f4539ffb640 1 -- 172.21.15.47:0/76029931 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663251595 0 0) 0x7f453c0d09d0 con 0x7f453c0cdbf0 2024-03-23T09:52:37.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.556+0000 7f453a7fc640 1 --2- 172.21.15.47:0/76029931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453c160c90 0x7f453c0cd6b0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f45240097a0 tx=0x7f452402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6688eb63b47455 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:37.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.556+0000 7f4539ffb640 1 -- 172.21.15.47:0/76029931 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524031830 con 0x7f453c160c90 2024-03-23T09:52:37.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.556+0000 7f4539ffb640 1 -- 172.21.15.47:0/76029931 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4524031990 con 0x7f453c160c90 2024-03-23T09:52:37.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.556+0000 7f4539ffb640 1 -- 172.21.15.47:0/76029931 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524031d20 con 0x7f453c160c90 2024-03-23T09:52:37.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.556+0000 7f454173c640 1 -- 172.21.15.47:0/76029931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453c160c90 msgr2=0x7f453c0cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.556+0000 7f454173c640 1 --2- 172.21.15.47:0/76029931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453c160c90 0x7f453c0cd6b0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f45240097a0 tx=0x7f452402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.556+0000 7f454173c640 1 -- 172.21.15.47:0/76029931 shutdown_connections 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.556+0000 7f454173c640 1 --2- 172.21.15.47:0/76029931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453c160c90 0x7f453c0cd6b0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.556+0000 7f454173c640 1 --2- 172.21.15.47:0/76029931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453c15cc40 0x7f453c160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.556+0000 7f454173c640 1 -- 172.21.15.47:0/76029931 >> 172.21.15.47:0/76029931 conn(0x7f453c06b160 msgr2=0x7f453c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.556+0000 7f454173c640 1 -- 172.21.15.47:0/76029931 shutdown_connections 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.556+0000 7f454173c640 1 -- 172.21.15.47:0/76029931 wait complete. 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.558+0000 7f454173c640 1 Processor -- start 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.558+0000 7f454173c640 1 -- start start 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.558+0000 7f454173c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453c0cdbf0 0x7f453c1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.558+0000 7f454173c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453c15cc40 0x7f453c151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.558+0000 7f454173c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f453c160c90 0x7f453c151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.558+0000 7f454173c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c156720 con 0x7f453c15cc40 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.558+0000 7f454173c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0cc190 con 0x7f453c160c90 2024-03-23T09:52:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.558+0000 7f454173c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0cc470 con 0x7f453c0cdbf0 2024-03-23T09:52:37.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.558+0000 7f453a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453c15cc40 0x7f453c151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:37.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.558+0000 7f453a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453c15cc40 0x7f453c151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48280/0 (socket says 172.21.15.47:48280) 2024-03-23T09:52:37.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.558+0000 7f453a7fc640 1 -- 172.21.15.47:0/1523152705 learned_addr learned my addr 172.21.15.47:0/1523152705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:37.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.559+0000 7f453a7fc640 1 -- 172.21.15.47:0/1523152705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453c0cdbf0 msgr2=0x7f453c1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.559+0000 7f453a7fc640 1 --2- 172.21.15.47:0/1523152705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453c0cdbf0 0x7f453c1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.559+0000 7f453a7fc640 1 -- 172.21.15.47:0/1523152705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f453c160c90 msgr2=0x7f453c151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.559+0000 7f453a7fc640 1 --2- 172.21.15.47:0/1523152705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f453c160c90 0x7f453c151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.559+0000 7f453a7fc640 1 -- 172.21.15.47:0/1523152705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4524009400 con 0x7f453c15cc40 2024-03-23T09:52:37.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.559+0000 7f453a7fc640 1 --2- 172.21.15.47:0/1523152705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453c15cc40 0x7f453c151960 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f4524000c00 tx=0x7f45240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:37.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.559+0000 7f45337fe640 1 -- 172.21.15.47:0/1523152705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45240317d0 con 0x7f453c15cc40 2024-03-23T09:52:37.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.559+0000 7f45337fe640 1 -- 172.21.15.47:0/1523152705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452403aa60 con 0x7f453c15cc40 2024-03-23T09:52:37.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.559+0000 7f454173c640 1 -- 172.21.15.47:0/1523152705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453c0cc750 con 0x7f453c15cc40 2024-03-23T09:52:37.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.559+0000 7f45337fe640 1 -- 172.21.15.47:0/1523152705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452403ae10 con 0x7f453c15cc40 2024-03-23T09:52:37.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.559+0000 7f454173c640 1 -- 172.21.15.47:0/1523152705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453c15d590 con 0x7f453c15cc40 2024-03-23T09:52:37.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.560+0000 7f454173c640 1 -- 172.21.15.47:0/1523152705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453c072360 con 0x7f453c15cc40 2024-03-23T09:52:37.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.560+0000 7f45337fe640 1 -- 172.21.15.47:0/1523152705 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f452404d450 con 0x7f453c15cc40 2024-03-23T09:52:37.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.561+0000 7f45337fe640 1 --2- 172.21.15.47:0/1523152705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f451c06ac00 0x7f451c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:37.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.561+0000 7f45337fe640 1 -- 172.21.15.47:0/1523152705 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f45240ab630 con 0x7f453c15cc40 2024-03-23T09:52:37.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.561+0000 7f453affd640 1 --2- 172.21.15.47:0/1523152705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f451c06ac00 0x7f451c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:37.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.562+0000 7f453affd640 1 --2- 172.21.15.47:0/1523152705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f451c06ac00 0x7f451c06d0c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f452c0070b0 tx=0x7f452c007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:37.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.565+0000 7f45337fe640 1 -- 172.21.15.47:0/1523152705 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f45240767b0 con 0x7f453c15cc40 2024-03-23T09:52:37.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.755+0000 7f454173c640 1 -- 172.21.15.47:0/1523152705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f453c15d780 con 0x7f453c15cc40 2024-03-23T09:52:37.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.756+0000 7f45337fe640 1 -- 172.21.15.47:0/1523152705 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v118) v1 ==== 80+0+242 (secure 0 0 0) 0x7f4524035020 con 0x7f453c15cc40 2024-03-23T09:52:37.756 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6832/1868958629 2024-03-24T09:52:25.484801+0000 2024-03-23T09:52:37.756 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6834/724782441 2024-03-24T09:52:05.460377+0000 2024-03-23T09:52:37.757 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6834/1868958629 2024-03-24T09:52:25.484801+0000 2024-03-23T09:52:37.757 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6832/724782441 2024-03-24T09:52:05.460377+0000 2024-03-23T09:52:37.757 INFO:teuthology.orchestra.run.smithi047.stderr:listed 4 entries 2024-03-23T09:52:37.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.758+0000 7f454173c640 1 -- 172.21.15.47:0/1523152705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f451c06ac00 msgr2=0x7f451c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.758+0000 7f454173c640 1 --2- 172.21.15.47:0/1523152705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f451c06ac00 0x7f451c06d0c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f452c0070b0 tx=0x7f452c007040 comp rx=0 tx=0).stop 2024-03-23T09:52:37.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.758+0000 7f454173c640 1 -- 172.21.15.47:0/1523152705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453c15cc40 msgr2=0x7f453c151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:37.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.758+0000 7f454173c640 1 --2- 172.21.15.47:0/1523152705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453c15cc40 0x7f453c151960 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f4524000c00 tx=0x7f45240046c0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.758+0000 7f454173c640 1 -- 172.21.15.47:0/1523152705 shutdown_connections 2024-03-23T09:52:37.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.758+0000 7f454173c640 1 --2- 172.21.15.47:0/1523152705 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f451c06ac00 0x7f451c06d0c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.758+0000 7f454173c640 1 --2- 172.21.15.47:0/1523152705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f453c160c90 0x7f453c151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.758+0000 7f454173c640 1 --2- 172.21.15.47:0/1523152705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f453c15cc40 0x7f453c151960 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.758+0000 7f454173c640 1 --2- 172.21.15.47:0/1523152705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f453c0cdbf0 0x7f453c1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:37.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.758+0000 7f454173c640 1 -- 172.21.15.47:0/1523152705 >> 172.21.15.47:0/1523152705 conn(0x7f453c06b160 msgr2=0x7f453c0b71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:37.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.758+0000 7f454173c640 1 -- 172.21.15.47:0/1523152705 shutdown_connections 2024-03-23T09:52:37.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:37.758+0000 7f454173c640 1 -- 172.21.15.47:0/1523152705 wait complete. 2024-03-23T09:52:37.768 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-23T09:52:37.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:37.769 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T09:52:37.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[83419]: fuse finished with error 0 and tester_r 0 2024-03-23T09:52:37.795 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:52:37.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:37.795 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T09:52:37.880 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:52:37.880 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:52:37.899 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:52:37.899 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:52:37.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:37.899 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-23T09:52:37.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:52:37.956 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-23T09:52:37.957 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:52:37.957 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:52:37.971 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:52:37.971 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:52:37.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:37.971 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-23T09:52:38.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:52:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-23T09:52:38.027 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:52:38.027 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:52:38.044 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:52:38.044 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:52:38.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:38.044 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-23T09:52:38.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:52:38.101 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-23T09:52:38.101 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:52:38.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.201+0000 7ff48f93e640 1 Processor -- start 2024-03-23T09:52:38.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.201+0000 7ff48f93e640 1 -- start start 2024-03-23T09:52:38.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.202+0000 7ff48f93e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff48815cb40 0x7ff488160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.202+0000 7ff48f93e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488160cb0 0x7ff4880d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.202+0000 7ff48f93e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4880d0ee0 con 0x7ff4880b8300 2024-03-23T09:52:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.202+0000 7ff48f93e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4880b5a10 con 0x7ff48815cb40 2024-03-23T09:52:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.202+0000 7ff48f93e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4881616f0 con 0x7ff488160cb0 2024-03-23T09:52:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.202+0000 7ff48d6da640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff48815cb40 0x7ff488160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.202+0000 7ff48d6da640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff48815cb40 0x7ff488160770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48294/0 (socket says 172.21.15.47:48294) 2024-03-23T09:52:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.202+0000 7ff48d6da640 1 -- 172.21.15.47:0/1324488113 learned_addr learned my addr 172.21.15.47:0/1324488113 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.202+0000 7ff48d6da640 1 -- 172.21.15.47:0/1324488113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488160cb0 msgr2=0x7ff4880d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.202+0000 7ff48d6da640 1 --2- 172.21.15.47:0/1324488113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488160cb0 0x7ff4880d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.202+0000 7ff48d6da640 1 -- 172.21.15.47:0/1324488113 >> v1:172.21.15.175:6789/0 conn(0x7ff4880b8300 legacy=0x7ff4880cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.202+0000 7ff48d6da640 1 -- 172.21.15.47:0/1324488113 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff488161130 con 0x7ff48815cb40 2024-03-23T09:52:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.203+0000 7ff48d6da640 1 --2- 172.21.15.47:0/1324488113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff48815cb40 0x7ff488160770 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098a0 tx=0x7ff48002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b35a011ae8e742 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.203+0000 7ff487fff640 1 -- 172.21.15.47:0/1324488113 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480031640 con 0x7ff48815cb40 2024-03-23T09:52:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.203+0000 7ff487fff640 1 -- 172.21.15.47:0/1324488113 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4800317a0 con 0x7ff48815cb40 2024-03-23T09:52:38.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.203+0000 7ff487fff640 1 -- 172.21.15.47:0/1324488113 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480031b30 con 0x7ff48815cb40 2024-03-23T09:52:38.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.204+0000 7ff48f93e640 1 -- 172.21.15.47:0/1324488113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff48815cb40 msgr2=0x7ff488160770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.204+0000 7ff48f93e640 1 --2- 172.21.15.47:0/1324488113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff48815cb40 0x7ff488160770 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098a0 tx=0x7ff48002b8f0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.204+0000 7ff48f93e640 1 -- 172.21.15.47:0/1324488113 shutdown_connections 2024-03-23T09:52:38.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.204+0000 7ff48f93e640 1 --2- 172.21.15.47:0/1324488113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488160cb0 0x7ff4880d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.204+0000 7ff48f93e640 1 --2- 172.21.15.47:0/1324488113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff48815cb40 0x7ff488160770 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.204+0000 7ff48f93e640 1 -- 172.21.15.47:0/1324488113 >> 172.21.15.47:0/1324488113 conn(0x7ff48806b160 msgr2=0x7ff48806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:38.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.204+0000 7ff48f93e640 1 -- 172.21.15.47:0/1324488113 shutdown_connections 2024-03-23T09:52:38.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.204+0000 7ff48f93e640 1 -- 172.21.15.47:0/1324488113 wait complete. 2024-03-23T09:52:38.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.205+0000 7ff48f93e640 1 Processor -- start 2024-03-23T09:52:38.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.205+0000 7ff48f93e640 1 -- start start 2024-03-23T09:52:38.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48f93e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4880b8300 0x7ff4881506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48f93e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48815cb40 0x7ff48815f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48f93e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488160cb0 0x7ff48815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48f93e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff488150c20 con 0x7ff4880b8300 2024-03-23T09:52:38.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48f93e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48815fcd0 con 0x7ff48815cb40 2024-03-23T09:52:38.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48f93e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff488159ad0 con 0x7ff488160cb0 2024-03-23T09:52:38.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48d6da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4880b8300 0x7ff4881506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48d6da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4880b8300 0x7ff4881506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48302/0 (socket says 172.21.15.47:48302) 2024-03-23T09:52:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48d6da640 1 -- 172.21.15.47:0/1741186961 learned_addr learned my addr 172.21.15.47:0/1741186961 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48d6da640 1 -- 172.21.15.47:0/1741186961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488160cb0 msgr2=0x7ff48815a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48d6da640 1 --2- 172.21.15.47:0/1741186961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488160cb0 0x7ff48815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48d6da640 1 -- 172.21.15.47:0/1741186961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48815cb40 msgr2=0x7ff48815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48d6da640 1 --2- 172.21.15.47:0/1741186961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48815cb40 0x7ff48815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48d6da640 1 -- 172.21.15.47:0/1741186961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4800094d0 con 0x7ff4880b8300 2024-03-23T09:52:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.206+0000 7ff48d6da640 1 --2- 172.21.15.47:0/1741186961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4880b8300 0x7ff4881506e0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098a0 tx=0x7ff4800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.207+0000 7ff485ffb640 1 -- 172.21.15.47:0/1741186961 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48003ac60 con 0x7ff4880b8300 2024-03-23T09:52:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.207+0000 7ff485ffb640 1 -- 172.21.15.47:0/1741186961 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48003adc0 con 0x7ff4880b8300 2024-03-23T09:52:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.207+0000 7ff48f93e640 1 -- 172.21.15.47:0/1741186961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff488159db0 con 0x7ff4880b8300 2024-03-23T09:52:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.207+0000 7ff485ffb640 1 -- 172.21.15.47:0/1741186961 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480044630 con 0x7ff4880b8300 2024-03-23T09:52:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.207+0000 7ff48f93e640 1 -- 172.21.15.47:0/1741186961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48815ffb0 con 0x7ff4880b8300 2024-03-23T09:52:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.208+0000 7ff48f93e640 1 -- 172.21.15.47:0/1741186961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff488072360 con 0x7ff4880b8300 2024-03-23T09:52:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.208+0000 7ff485ffb640 1 -- 172.21.15.47:0/1741186961 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff48002f050 con 0x7ff4880b8300 2024-03-23T09:52:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.209+0000 7ff485ffb640 1 --2- 172.21.15.47:0/1741186961 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff47406ac00 0x7ff47406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.209+0000 7ff485ffb640 1 -- 172.21.15.47:0/1741186961 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff4800ab4f0 con 0x7ff4880b8300 2024-03-23T09:52:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.209+0000 7ff48ced9640 1 --2- 172.21.15.47:0/1741186961 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff47406ac00 0x7ff47406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.209+0000 7ff48ced9640 1 --2- 172.21.15.47:0/1741186961 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff47406ac00 0x7ff47406d0c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff4880c6300 tx=0x7ff47c007990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:38.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.213+0000 7ff485ffb640 1 -- 172.21.15.47:0/1741186961 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff480076540 con 0x7ff4880b8300 2024-03-23T09:52:38.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.428+0000 7ff48f93e640 1 -- 172.21.15.47:0/1741186961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff488159930 con 0x7ff4880b8300 2024-03-23T09:52:38.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.428+0000 7ff485ffb640 1 -- 172.21.15.47:0/1741186961 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4738 (secure 0 0 0) 0x7ff488159930 con 0x7ff4880b8300 2024-03-23T09:52:38.429 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:38.429 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":93,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:51:40.395539+0000","modified":"2024-03-23T09:52:30.511498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"c","rank":0,"incarnation":90,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6834/3501786111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3501786111},{"type":"v1","addr":"172.21.15.47:6834","nonce":3501786111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-23T09:52:38.429 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 93 2024-03-23T09:52:38.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.430+0000 7ff48f93e640 1 -- 172.21.15.47:0/1741186961 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff47406ac00 msgr2=0x7ff47406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.430+0000 7ff48f93e640 1 --2- 172.21.15.47:0/1741186961 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff47406ac00 0x7ff47406d0c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff4880c6300 tx=0x7ff47c007990 comp rx=0 tx=0).stop 2024-03-23T09:52:38.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.430+0000 7ff48f93e640 1 -- 172.21.15.47:0/1741186961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4880b8300 msgr2=0x7ff4881506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.430+0000 7ff48f93e640 1 --2- 172.21.15.47:0/1741186961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4880b8300 0x7ff4881506e0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098a0 tx=0x7ff4800046c0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.430+0000 7ff48f93e640 1 -- 172.21.15.47:0/1741186961 shutdown_connections 2024-03-23T09:52:38.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.430+0000 7ff48f93e640 1 --2- 172.21.15.47:0/1741186961 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff47406ac00 0x7ff47406d0c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.430+0000 7ff48f93e640 1 --2- 172.21.15.47:0/1741186961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488160cb0 0x7ff48815a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.430+0000 7ff48f93e640 1 --2- 172.21.15.47:0/1741186961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48815cb40 0x7ff48815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.430+0000 7ff48f93e640 1 --2- 172.21.15.47:0/1741186961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4880b8300 0x7ff4881506e0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.430+0000 7ff48f93e640 1 -- 172.21.15.47:0/1741186961 >> 172.21.15.47:0/1741186961 conn(0x7ff48806b160 msgr2=0x7ff4880c6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:38.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.430+0000 7ff48f93e640 1 -- 172.21.15.47:0/1741186961 shutdown_connections 2024-03-23T09:52:38.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.430+0000 7ff48f93e640 1 -- 172.21.15.47:0/1741186961 wait complete. 2024-03-23T09:52:38.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:52:38.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.542+0000 7f141148b640 1 Processor -- start 2024-03-23T09:52:38.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.542+0000 7f141148b640 1 -- start start 2024-03-23T09:52:38.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.543+0000 7f141148b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c06a3f0 0x7f140c0cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.543+0000 7f141148b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f140c098e10 0x7f140c0ca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.543+0000 7f141148b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140c0c9ed0 con 0x7f140c15cbc0 2024-03-23T09:52:38.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.543+0000 7f141148b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140c0b0d10 con 0x7f140c098e10 2024-03-23T09:52:38.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.543+0000 7f141148b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0b0ff0 con 0x7f140c06a3f0 2024-03-23T09:52:38.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.543+0000 7f140b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f140c098e10 0x7f140c0ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.543+0000 7f140b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f140c098e10 0x7f140c0ca920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48304/0 (socket says 172.21.15.47:48304) 2024-03-23T09:52:38.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.543+0000 7f140b7fe640 1 -- 172.21.15.47:0/4133378933 learned_addr learned my addr 172.21.15.47:0/4133378933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:38.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.544+0000 7f1403fff640 1 --2- 172.21.15.47:0/4133378933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c06a3f0 0x7f140c0cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.544+0000 7f140b7fe640 1 -- 172.21.15.47:0/4133378933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c06a3f0 msgr2=0x7f140c0cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:38.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.544+0000 7f140b7fe640 1 --2- 172.21.15.47:0/4133378933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c06a3f0 0x7f140c0cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.544+0000 7f140b7fe640 1 -- 172.21.15.47:0/4133378933 >> v1:172.21.15.175:6789/0 conn(0x7f140c15cbc0 legacy=0x7f140c069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.544+0000 7f140b7fe640 1 -- 172.21.15.47:0/4133378933 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c0ca120 con 0x7f140c098e10 2024-03-23T09:52:38.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.544+0000 7f140b7fe640 1 --2- 172.21.15.47:0/4133378933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f140c098e10 0x7f140c0ca920 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f1404009e50 tx=0x7f140402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd87fe23205f147d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:38.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.545+0000 7f140a7fc640 1 -- 172.21.15.47:0/4133378933 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140402cb00 con 0x7f140c098e10 2024-03-23T09:52:38.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.545+0000 7f140a7fc640 1 -- 172.21.15.47:0/4133378933 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140402cc60 con 0x7f140c098e10 2024-03-23T09:52:38.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.545+0000 7f140a7fc640 1 -- 172.21.15.47:0/4133378933 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14040343f0 con 0x7f140c098e10 2024-03-23T09:52:38.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.545+0000 7f141148b640 1 -- 172.21.15.47:0/4133378933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f140c098e10 msgr2=0x7f140c0ca920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.545+0000 7f141148b640 1 --2- 172.21.15.47:0/4133378933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f140c098e10 0x7f140c0ca920 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f1404009e50 tx=0x7f140402bf60 comp rx=0 tx=0).stop 2024-03-23T09:52:38.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.545+0000 7f141148b640 1 -- 172.21.15.47:0/4133378933 shutdown_connections 2024-03-23T09:52:38.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.545+0000 7f141148b640 1 --2- 172.21.15.47:0/4133378933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f140c098e10 0x7f140c0ca920 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.545+0000 7f141148b640 1 --2- 172.21.15.47:0/4133378933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c06a3f0 0x7f140c0cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.545+0000 7f141148b640 1 -- 172.21.15.47:0/4133378933 >> 172.21.15.47:0/4133378933 conn(0x7f140c098a00 msgr2=0x7f140c0cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:38.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.545+0000 7f141148b640 1 -- 172.21.15.47:0/4133378933 shutdown_connections 2024-03-23T09:52:38.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.545+0000 7f141148b640 1 -- 172.21.15.47:0/4133378933 wait complete. 2024-03-23T09:52:38.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.546+0000 7f141148b640 1 Processor -- start 2024-03-23T09:52:38.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.546+0000 7f141148b640 1 -- start start 2024-03-23T09:52:38.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f141148b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f140c06a3f0 0x7f140c1f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f141148b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c098e10 0x7f140c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f141148b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f140c15cbc0 0x7f140c0b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f141148b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c1f6f00 con 0x7f140c06a3f0 2024-03-23T09:52:38.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f141148b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0adf40 con 0x7f140c15cbc0 2024-03-23T09:52:38.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f141148b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0af230 con 0x7f140c098e10 2024-03-23T09:52:38.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f140affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f140c06a3f0 0x7f140c1f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f140affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f140c06a3f0 0x7f140c1f69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48314/0 (socket says 172.21.15.47:48314) 2024-03-23T09:52:38.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f140affd640 1 -- 172.21.15.47:0/3680288732 learned_addr learned my addr 172.21.15.47:0/3680288732 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:38.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f1403fff640 1 --2- 172.21.15.47:0/3680288732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c098e10 0x7f140c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f140affd640 1 -- 172.21.15.47:0/3680288732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c098e10 msgr2=0x7f140c159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f140affd640 1 --2- 172.21.15.47:0/3680288732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c098e10 0x7f140c159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f140affd640 1 -- 172.21.15.47:0/3680288732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f140c15cbc0 msgr2=0x7f140c0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.547+0000 7f140affd640 1 --2- 172.21.15.47:0/3680288732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f140c15cbc0 0x7f140c0b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.548+0000 7f140affd640 1 -- 172.21.15.47:0/3680288732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1404009b30 con 0x7f140c06a3f0 2024-03-23T09:52:38.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.548+0000 7f140affd640 1 --2- 172.21.15.47:0/3680288732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f140c06a3f0 0x7f140c1f69c0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00c400 tx=0x7f13fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:38.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.548+0000 7f14037fe640 1 -- 172.21.15.47:0/3680288732 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0052d0 con 0x7f140c06a3f0 2024-03-23T09:52:38.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.548+0000 7f14037fe640 1 -- 172.21.15.47:0/3680288732 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13fc005430 con 0x7f140c06a3f0 2024-03-23T09:52:38.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.548+0000 7f141148b640 1 -- 172.21.15.47:0/3680288732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140c0ae190 con 0x7f140c06a3f0 2024-03-23T09:52:38.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.548+0000 7f14037fe640 1 -- 172.21.15.47:0/3680288732 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc003830 con 0x7f140c06a3f0 2024-03-23T09:52:38.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.548+0000 7f141148b640 1 -- 172.21.15.47:0/3680288732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140c0ae750 con 0x7f140c06a3f0 2024-03-23T09:52:38.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.549+0000 7f141148b640 1 -- 172.21.15.47:0/3680288732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140c0672f0 con 0x7f140c06a3f0 2024-03-23T09:52:38.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.550+0000 7f14037fe640 1 -- 172.21.15.47:0/3680288732 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f13fc003990 con 0x7f140c06a3f0 2024-03-23T09:52:38.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.550+0000 7f14037fe640 1 --2- 172.21.15.47:0/3680288732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13ec06ac00 0x7f13ec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.550+0000 7f14037fe640 1 -- 172.21.15.47:0/3680288732 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f13fc0889d0 con 0x7f140c06a3f0 2024-03-23T09:52:38.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.550+0000 7f1403fff640 1 --2- 172.21.15.47:0/3680288732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13ec06ac00 0x7f13ec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.551+0000 7f1403fff640 1 --2- 172.21.15.47:0/3680288732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13ec06ac00 0x7f13ec06d0c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f13f4006fd0 tx=0x7f13f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:38.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.554+0000 7f14037fe640 1 -- 172.21.15.47:0/3680288732 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f13fc053ad0 con 0x7f140c06a3f0 2024-03-23T09:52:38.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.734+0000 7f141148b640 1 -- 172.21.15.47:0/3680288732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f140c0cc2c0 con 0x7f140c06a3f0 2024-03-23T09:52:38.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.735+0000 7f14037fe640 1 -- 172.21.15.47:0/3680288732 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f13fc0570f0 con 0x7f140c06a3f0 2024-03-23T09:52:38.735 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:38.736 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":118,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:52:25.484824+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:51:38.679890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-23T09:51:39.396299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6832/1868958629":"2024-03-24T09:52:25.484801+0000","172.21.15.47:6834/724782441":"2024-03-24T09:52:05.460377+0000","172.21.15.47:6834/1868958629":"2024-03-24T09:52:25.484801+0000","172.21.15.47:6832/724782441":"2024-03-24T09:52:05.460377+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:52:38.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.737+0000 7f141148b640 1 -- 172.21.15.47:0/3680288732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13ec06ac00 msgr2=0x7f13ec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.737+0000 7f141148b640 1 --2- 172.21.15.47:0/3680288732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13ec06ac00 0x7f13ec06d0c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f13f4006fd0 tx=0x7f13f4008040 comp rx=0 tx=0).stop 2024-03-23T09:52:38.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.737+0000 7f141148b640 1 -- 172.21.15.47:0/3680288732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f140c06a3f0 msgr2=0x7f140c1f69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.737+0000 7f141148b640 1 --2- 172.21.15.47:0/3680288732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f140c06a3f0 0x7f140c1f69c0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00c400 tx=0x7f13fc00c8c0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.737+0000 7f141148b640 1 -- 172.21.15.47:0/3680288732 shutdown_connections 2024-03-23T09:52:38.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.737+0000 7f141148b640 1 --2- 172.21.15.47:0/3680288732 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f13ec06ac00 0x7f13ec06d0c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.737+0000 7f141148b640 1 --2- 172.21.15.47:0/3680288732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f140c15cbc0 0x7f140c0b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.737+0000 7f141148b640 1 --2- 172.21.15.47:0/3680288732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c098e10 0x7f140c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.737+0000 7f141148b640 1 --2- 172.21.15.47:0/3680288732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f140c06a3f0 0x7f140c1f69c0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.737+0000 7f141148b640 1 -- 172.21.15.47:0/3680288732 >> 172.21.15.47:0/3680288732 conn(0x7f140c098a00 msgr2=0x7f140c069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:38.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.737+0000 7f141148b640 1 -- 172.21.15.47:0/3680288732 shutdown_connections 2024-03-23T09:52:38.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.737+0000 7f141148b640 1 -- 172.21.15.47:0/3680288732 wait complete. 2024-03-23T09:52:38.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T09:40:58.380031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T09:51:38.679890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T09:51:39.396299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T09:52:38.748 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T09:52:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.852+0000 7f7e25377640 1 Processor -- start 2024-03-23T09:52:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.852+0000 7f7e25377640 1 -- start start 2024-03-23T09:52:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.853+0000 7f7e25377640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e20098a00 0x7f7e200b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.853+0000 7f7e25377640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7e200cc270 0x7f7e200c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.853+0000 7f7e25377640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e200b1b80 con 0x7f7e20098de0 2024-03-23T09:52:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.853+0000 7f7e25377640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e20099d00 con 0x7f7e200cc270 2024-03-23T09:52:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.853+0000 7f7e25377640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200cc6d0 con 0x7f7e20098a00 2024-03-23T09:52:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.853+0000 7f7e1f7fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7e200cc270 0x7f7e200c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.853+0000 7f7e1effd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e20098a00 0x7f7e200b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.853+0000 7f7e1effd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e20098a00 0x7f7e200b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:49892/0 (socket says 172.21.15.47:49892) 2024-03-23T09:52:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.853+0000 7f7e1f7fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7e200cc270 0x7f7e200c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:37626/0 (socket says 172.21.15.47:37626) 2024-03-23T09:52:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.853+0000 7f7e1effd640 1 -- 172.21.15.47:0/1121720900 learned_addr learned my addr 172.21.15.47:0/1121720900 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.853+0000 7f7e17fff640 1 --1- 172.21.15.47:0/1121720900 >> v1:172.21.15.175:6789/0 conn(0x7f7e20098de0 0x7f7e200c41a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:34544/0 (socket says 172.21.15.47:34544) 2024-03-23T09:52:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.854+0000 7f7e1f7fe640 1 -- 172.21.15.47:0/1121720900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e20098a00 msgr2=0x7f7e200b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.854+0000 7f7e1f7fe640 1 --2- 172.21.15.47:0/1121720900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e20098a00 0x7f7e200b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.854+0000 7f7e1f7fe640 1 -- 172.21.15.47:0/1121720900 >> v1:172.21.15.175:6789/0 conn(0x7f7e20098de0 legacy=0x7f7e200c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.854+0000 7f7e1f7fe640 1 -- 172.21.15.47:0/1121720900 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20099740 con 0x7f7e200cc270 2024-03-23T09:52:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.854+0000 7f7e1effd640 1 --2- 172.21.15.47:0/1121720900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e20098a00 0x7f7e200b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:52:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.855+0000 7f7e1f7fe640 1 --2- 172.21.15.47:0/1121720900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e200cc270 0x7f7e200c48b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009dc0 tx=0x7f7e1802be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=348f81e5a0f4b60d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.855+0000 7f7e1e7fc640 1 -- 172.21.15.47:0/1121720900 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1802cc40 con 0x7f7e200cc270 2024-03-23T09:52:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.855+0000 7f7e1e7fc640 1 -- 172.21.15.47:0/1121720900 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1802cda0 con 0x7f7e200cc270 2024-03-23T09:52:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.855+0000 7f7e1e7fc640 1 -- 172.21.15.47:0/1121720900 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18034660 con 0x7f7e200cc270 2024-03-23T09:52:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.855+0000 7f7e25377640 1 -- 172.21.15.47:0/1121720900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e200cc270 msgr2=0x7f7e200c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.855+0000 7f7e25377640 1 --2- 172.21.15.47:0/1121720900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e200cc270 0x7f7e200c48b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009dc0 tx=0x7f7e1802be90 comp rx=0 tx=0).stop 2024-03-23T09:52:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.855+0000 7f7e25377640 1 -- 172.21.15.47:0/1121720900 shutdown_connections 2024-03-23T09:52:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.855+0000 7f7e25377640 1 --2- 172.21.15.47:0/1121720900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e200cc270 0x7f7e200c48b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.855+0000 7f7e25377640 1 --2- 172.21.15.47:0/1121720900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e20098a00 0x7f7e200b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.855+0000 7f7e25377640 1 -- 172.21.15.47:0/1121720900 >> 172.21.15.47:0/1121720900 conn(0x7f7e200b2140 msgr2=0x7f7e200b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.855+0000 7f7e25377640 1 -- 172.21.15.47:0/1121720900 shutdown_connections 2024-03-23T09:52:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.855+0000 7f7e25377640 1 -- 172.21.15.47:0/1121720900 wait complete. 2024-03-23T09:52:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.856+0000 7f7e25377640 1 Processor -- start 2024-03-23T09:52:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.857+0000 7f7e25377640 1 -- start start 2024-03-23T09:52:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.857+0000 7f7e25377640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e20098a00 0x7f7e200ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.857+0000 7f7e25377640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20098de0 0x7f7e200aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.857+0000 7f7e25377640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e200cc270 0x7f7e200af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.857+0000 7f7e25377640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200adc70 con 0x7f7e20098de0 2024-03-23T09:52:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.857+0000 7f7e25377640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200ca540 con 0x7f7e20098a00 2024-03-23T09:52:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.857+0000 7f7e25377640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200adec0 con 0x7f7e200cc270 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.857+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20098de0 0x7f7e200aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.857+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20098de0 0x7f7e200aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48316/0 (socket says 172.21.15.47:48316) 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.857+0000 7f7e17fff640 1 -- 172.21.15.47:0/1172938397 learned_addr learned my addr 172.21.15.47:0/1172938397 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e1effd640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e20098a00 0x7f7e200ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e1f7fe640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e200cc270 0x7f7e200af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e17fff640 1 -- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e200cc270 msgr2=0x7f7e200af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e17fff640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e200cc270 0x7f7e200af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e17fff640 1 -- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e20098a00 msgr2=0x7f7e200ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e17fff640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e20098a00 0x7f7e200ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e17fff640 1 -- 172.21.15.47:0/1172938397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e100094d0 con 0x7f7e20098de0 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e1f7fe640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e200cc270 0x7f7e200af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e17fff640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20098de0 0x7f7e200aee50 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800c3e0 tx=0x7f7e0800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e177fe640 1 -- 172.21.15.47:0/1172938397 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e080074e0 con 0x7f7e20098de0 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e177fe640 1 -- 172.21.15.47:0/1172938397 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e08007640 con 0x7f7e20098de0 2024-03-23T09:52:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e25377640 1 -- 172.21.15.47:0/1172938397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e18009a70 con 0x7f7e20098de0 2024-03-23T09:52:38.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e177fe640 1 -- 172.21.15.47:0/1172938397 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e080049e0 con 0x7f7e20098de0 2024-03-23T09:52:38.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.858+0000 7f7e25377640 1 -- 172.21.15.47:0/1172938397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e200ae480 con 0x7f7e20098de0 2024-03-23T09:52:38.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.859+0000 7f7e25377640 1 -- 172.21.15.47:0/1172938397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e20066be0 con 0x7f7e20098de0 2024-03-23T09:52:38.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.859+0000 7f7e177fe640 1 -- 172.21.15.47:0/1172938397 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7e08004b40 con 0x7f7e20098de0 2024-03-23T09:52:38.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.860+0000 7f7e177fe640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e0006ac00 0x7f7e0006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:38.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.860+0000 7f7e177fe640 1 -- 172.21.15.47:0/1172938397 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f7e08088de0 con 0x7f7e20098de0 2024-03-23T09:52:38.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.860+0000 7f7e1effd640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e0006ac00 0x7f7e0006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:38.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.861+0000 7f7e1effd640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e0006ac00 0x7f7e0006d0c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009c50 tx=0x7f7e10008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:38.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:38.864+0000 7f7e177fe640 1 -- 172.21.15.47:0/1172938397 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7e08053f60 con 0x7f7e20098de0 2024-03-23T09:52:39.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.042+0000 7f7e25377640 1 -- 172.21.15.47:0/1172938397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e200c6cd0 con 0x7f7e20098de0 2024-03-23T09:52:39.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.043+0000 7f7e177fe640 1 -- 172.21.15.47:0/1172938397 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e08057580 con 0x7f7e20098de0 2024-03-23T09:52:39.043 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-23T09:52:39.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.045+0000 7f7e25377640 1 -- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e0006ac00 msgr2=0x7f7e0006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.045+0000 7f7e25377640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e0006ac00 0x7f7e0006d0c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009c50 tx=0x7f7e10008040 comp rx=0 tx=0).stop 2024-03-23T09:52:39.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.045+0000 7f7e25377640 1 -- 172.21.15.47:0/1172938397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20098de0 msgr2=0x7f7e200aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.045+0000 7f7e25377640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20098de0 0x7f7e200aee50 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800c3e0 tx=0x7f7e0800c8a0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.045+0000 7f7e25377640 1 -- 172.21.15.47:0/1172938397 shutdown_connections 2024-03-23T09:52:39.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.045+0000 7f7e25377640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e0006ac00 0x7f7e0006d0c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.045+0000 7f7e25377640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e200cc270 0x7f7e200af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.045+0000 7f7e25377640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20098de0 0x7f7e200aee50 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.045+0000 7f7e25377640 1 --2- 172.21.15.47:0/1172938397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e20098a00 0x7f7e200ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.045+0000 7f7e25377640 1 -- 172.21.15.47:0/1172938397 >> 172.21.15.47:0/1172938397 conn(0x7f7e200b2140 msgr2=0x7f7e2006e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:39.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.045+0000 7f7e25377640 1 -- 172.21.15.47:0/1172938397 shutdown_connections 2024-03-23T09:52:39.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.045+0000 7f7e25377640 1 -- 172.21.15.47:0/1172938397 wait complete. 2024-03-23T09:52:39.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T09:52:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.160+0000 7f038c036640 1 Processor -- start 2024-03-23T09:52:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.160+0000 7f038c036640 1 -- start start 2024-03-23T09:52:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f038c036640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f038415c990 0x7f0384160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f038c036640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03840cde10 0x7f03840d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f038c036640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0384161630 con 0x7f03840b8350 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f038c036640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03840d1030 con 0x7f03840cde10 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f038c036640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03840cfe10 con 0x7f038415c990 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f038a5d3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03840cde10 0x7f03840d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f038a5d3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03840cde10 0x7f03840d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48322/0 (socket says 172.21.15.47:48322) 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f038a5d3640 1 -- 172.21.15.47:0/2718526535 learned_addr learned my addr 172.21.15.47:0/2718526535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f0389dd2640 1 --2- 172.21.15.47:0/2718526535 >> v2:172.21.15.175:3300/0 conn(0x7f038415c990 0x7f0384160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f038a5d3640 1 -- 172.21.15.47:0/2718526535 >> v2:172.21.15.175:3300/0 conn(0x7f038415c990 msgr2=0x7f0384160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f038a5d3640 1 --2- 172.21.15.47:0/2718526535 >> v2:172.21.15.175:3300/0 conn(0x7f038415c990 0x7f0384160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f038a5d3640 1 -- 172.21.15.47:0/2718526535 >> v1:172.21.15.47:6789/0 conn(0x7f03840b8350 legacy=0x7f03840cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f0388dd0640 1 -- 172.21.15.47:0/2718526535 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564215603 0 0) 0x7f0384161630 con 0x7f03840b8350 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.161+0000 7f038a5d3640 1 -- 172.21.15.47:0/2718526535 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03840b5b60 con 0x7f03840cde10 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.162+0000 7f038a5d3640 1 --2- 172.21.15.47:0/2718526535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03840cde10 0x7f03840d14c0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f03840b8b30 tx=0x7f037c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96ea45018132a1c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.162+0000 7f0388dd0640 1 -- 172.21.15.47:0/2718526535 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c031870 con 0x7f03840cde10 2024-03-23T09:52:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.162+0000 7f0388dd0640 1 -- 172.21.15.47:0/2718526535 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c0319d0 con 0x7f03840cde10 2024-03-23T09:52:39.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.162+0000 7f0388dd0640 1 -- 172.21.15.47:0/2718526535 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c031d60 con 0x7f03840cde10 2024-03-23T09:52:39.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.163+0000 7f038c036640 1 -- 172.21.15.47:0/2718526535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03840cde10 msgr2=0x7f03840d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.163+0000 7f038c036640 1 --2- 172.21.15.47:0/2718526535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03840cde10 0x7f03840d14c0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f03840b8b30 tx=0x7f037c02b910 comp rx=0 tx=0).stop 2024-03-23T09:52:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.163+0000 7f038c036640 1 -- 172.21.15.47:0/2718526535 shutdown_connections 2024-03-23T09:52:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.163+0000 7f038c036640 1 --2- 172.21.15.47:0/2718526535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03840cde10 0x7f03840d14c0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.163+0000 7f038c036640 1 --2- 172.21.15.47:0/2718526535 >> v2:172.21.15.175:3300/0 conn(0x7f038415c990 0x7f0384160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.163+0000 7f038c036640 1 -- 172.21.15.47:0/2718526535 >> 172.21.15.47:0/2718526535 conn(0x7f03840659d0 msgr2=0x7f0384065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.163+0000 7f038c036640 1 -- 172.21.15.47:0/2718526535 shutdown_connections 2024-03-23T09:52:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.163+0000 7f038c036640 1 -- 172.21.15.47:0/2718526535 wait complete. 2024-03-23T09:52:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.164+0000 7f038c036640 1 Processor -- start 2024-03-23T09:52:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.164+0000 7f038c036640 1 -- start start 2024-03-23T09:52:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038c036640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03840b8350 0x7f0384154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038c036640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03840cde10 0x7f03840c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038c036640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f038415c990 0x7f038415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038c036640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03840c83f0 con 0x7f038415c990 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038c036640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03840c73e0 con 0x7f03840cde10 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038c036640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03840c7f60 con 0x7f03840b8350 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038a5d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f038415c990 0x7f038415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038a5d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f038415c990 0x7f038415e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48334/0 (socket says 172.21.15.47:48334) 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038a5d3640 1 -- 172.21.15.47:0/3904917805 learned_addr learned my addr 172.21.15.47:0/3904917805 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f03895d1640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03840cde10 0x7f03840c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f0389dd2640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03840b8350 0x7f0384154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038a5d3640 1 -- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03840b8350 msgr2=0x7f0384154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038a5d3640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03840b8350 0x7f0384154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038a5d3640 1 -- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03840cde10 msgr2=0x7f03840c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038a5d3640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03840cde10 0x7f03840c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038a5d3640 1 -- 172.21.15.47:0/3904917805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037c0094d0 con 0x7f038415c990 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.165+0000 7f038a5d3640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f038415c990 0x7f038415e2f0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f037c02b950 tx=0x7f037c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.166+0000 7f03827fc640 1 -- 172.21.15.47:0/3904917805 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c03ac60 con 0x7f038415c990 2024-03-23T09:52:39.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.166+0000 7f03827fc640 1 -- 172.21.15.47:0/3904917805 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c03adc0 con 0x7f038415c990 2024-03-23T09:52:39.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.166+0000 7f038c036640 1 -- 172.21.15.47:0/3904917805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038415e830 con 0x7f038415c990 2024-03-23T09:52:39.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.166+0000 7f03827fc640 1 -- 172.21.15.47:0/3904917805 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c045690 con 0x7f038415c990 2024-03-23T09:52:39.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.166+0000 7f038c036640 1 -- 172.21.15.47:0/3904917805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03840c7910 con 0x7f038415c990 2024-03-23T09:52:39.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.166+0000 7f038c036640 1 -- 172.21.15.47:0/3904917805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0384072360 con 0x7f038415c990 2024-03-23T09:52:39.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.167+0000 7f03827fc640 1 -- 172.21.15.47:0/3904917805 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f037c0457f0 con 0x7f038415c990 2024-03-23T09:52:39.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.167+0000 7f03827fc640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f037006ac30 0x7f037006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.167+0000 7f03827fc640 1 -- 172.21.15.47:0/3904917805 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f037c02d040 con 0x7f038415c990 2024-03-23T09:52:39.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.168+0000 7f0389dd2640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f037006ac30 0x7f037006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.168+0000 7f0389dd2640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f037006ac30 0x7f037006d0f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f03840b4100 tx=0x7f03740039b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:39.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.172+0000 7f03827fc640 1 -- 172.21.15.47:0/3904917805 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f037c076d80 con 0x7f038415c990 2024-03-23T09:52:39.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:52:39.217 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:52:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.350+0000 7f038c036640 1 -- 172.21.15.47:0/3904917805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03840c8d80 con 0x7f038415c990 2024-03-23T09:52:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.350+0000 7f03827fc640 1 -- 172.21.15.47:0/3904917805 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+11 (secure 0 0 0) 0x7f037c0350a0 con 0x7f038415c990 2024-03-23T09:52:39.351 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-23T09:52:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.353+0000 7f038c036640 1 -- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f037006ac30 msgr2=0x7f037006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.353+0000 7f038c036640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f037006ac30 0x7f037006d0f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f03840b4100 tx=0x7f03740039b0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.353+0000 7f038c036640 1 -- 172.21.15.47:0/3904917805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f038415c990 msgr2=0x7f038415e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.353+0000 7f038c036640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f038415c990 0x7f038415e2f0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f037c02b950 tx=0x7f037c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.353+0000 7f038c036640 1 -- 172.21.15.47:0/3904917805 shutdown_connections 2024-03-23T09:52:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.353+0000 7f038c036640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f037006ac30 0x7f037006d0f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.353+0000 7f038c036640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f038415c990 0x7f038415e2f0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.353+0000 7f038c036640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03840cde10 0x7f03840c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.353+0000 7f038c036640 1 --2- 172.21.15.47:0/3904917805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03840b8350 0x7f0384154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.353+0000 7f038c036640 1 -- 172.21.15.47:0/3904917805 >> 172.21.15.47:0/3904917805 conn(0x7f03840659d0 msgr2=0x7f03840b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.353+0000 7f038c036640 1 -- 172.21.15.47:0/3904917805 shutdown_connections 2024-03-23T09:52:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.353+0000 7f038c036640 1 -- 172.21.15.47:0/3904917805 wait complete. 2024-03-23T09:52:39.364 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T09:52:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.467+0000 7fda91051640 1 Processor -- start 2024-03-23T09:52:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.467+0000 7fda91051640 1 -- start start 2024-03-23T09:52:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.468+0000 7fda91051640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fda8c0cf9c0 0x7fda8c0b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.468+0000 7fda91051640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda8c161570 con 0x7fda8c15cbf0 2024-03-23T09:52:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.468+0000 7fda91051640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda8c160960 con 0x7fda8c0cd140 2024-03-23T09:52:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.468+0000 7fda91051640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8c160c40 con 0x7fda8c0cf9c0 2024-03-23T09:52:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.468+0000 7fda83fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fda8c0cf9c0 0x7fda8c0b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.468+0000 7fda8ad9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fda8c15cbf0 0x7fda8c0cf2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33634/0 (socket says 172.21.15.47:33634) 2024-03-23T09:52:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.468+0000 7fda8ad9d640 1 -- 172.21.15.47:0/183120087 learned_addr learned my addr 172.21.15.47:0/183120087 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.468+0000 7fda83fff640 1 -- 172.21.15.47:0/183120087 >> v1:172.21.15.175:6789/0 conn(0x7fda8c0cd140 legacy=0x7fda8c0d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.468+0000 7fda83fff640 1 -- 172.21.15.47:0/183120087 >> v1:172.21.15.47:6789/0 conn(0x7fda8c15cbf0 legacy=0x7fda8c0cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.468+0000 7fda83fff640 1 -- 172.21.15.47:0/183120087 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c160f20 con 0x7fda8c0cf9c0 2024-03-23T09:52:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.468+0000 7fda8a59c640 1 -- 172.21.15.47:0/183120087 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582857255 0 0) 0x7fda8c161570 con 0x7fda8c15cbf0 2024-03-23T09:52:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.469+0000 7fda83fff640 1 --2- 172.21.15.47:0/183120087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda8c0cf9c0 0x7fda8c0b1df0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fda740097a0 tx=0x7fda7402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca5764a135dc714 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.469+0000 7fda8a59c640 1 -- 172.21.15.47:0/183120087 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74031830 con 0x7fda8c0cf9c0 2024-03-23T09:52:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.469+0000 7fda8a59c640 1 -- 172.21.15.47:0/183120087 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda74031990 con 0x7fda8c0cf9c0 2024-03-23T09:52:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.469+0000 7fda8a59c640 1 -- 172.21.15.47:0/183120087 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74031d20 con 0x7fda8c0cf9c0 2024-03-23T09:52:39.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.469+0000 7fda91051640 1 -- 172.21.15.47:0/183120087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda8c0cf9c0 msgr2=0x7fda8c0b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.470+0000 7fda91051640 1 --2- 172.21.15.47:0/183120087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda8c0cf9c0 0x7fda8c0b1df0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fda740097a0 tx=0x7fda7402b8d0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.470+0000 7fda91051640 1 -- 172.21.15.47:0/183120087 shutdown_connections 2024-03-23T09:52:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.470+0000 7fda91051640 1 --2- 172.21.15.47:0/183120087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda8c0cf9c0 0x7fda8c0b1df0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.470+0000 7fda91051640 1 -- 172.21.15.47:0/183120087 >> 172.21.15.47:0/183120087 conn(0x7fda8c06b160 msgr2=0x7fda8c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.470+0000 7fda91051640 1 -- 172.21.15.47:0/183120087 shutdown_connections 2024-03-23T09:52:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.470+0000 7fda91051640 1 -- 172.21.15.47:0/183120087 wait complete. 2024-03-23T09:52:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.471+0000 7fda91051640 1 Processor -- start 2024-03-23T09:52:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.471+0000 7fda91051640 1 -- start start 2024-03-23T09:52:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda91051640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c0cd140 0x7fda8c15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda91051640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c0cf9c0 0x7fda8c0b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda91051640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda8c15cbf0 0x7fda8c1483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda91051640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c160080 con 0x7fda8c15cbf0 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda91051640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c0b2330 con 0x7fda8c0cd140 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda91051640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c14ed20 con 0x7fda8c0cf9c0 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda8b59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda8c15cbf0 0x7fda8c1483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda8b59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda8c15cbf0 0x7fda8c1483f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48338/0 (socket says 172.21.15.47:48338) 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda8b59e640 1 -- 172.21.15.47:0/3584284770 learned_addr learned my addr 172.21.15.47:0/3584284770 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda8ad9d640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c0cd140 0x7fda8c15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda83fff640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c0cf9c0 0x7fda8c0b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda8b59e640 1 -- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c0cf9c0 msgr2=0x7fda8c0b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda8b59e640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c0cf9c0 0x7fda8c0b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda8b59e640 1 -- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c0cd140 msgr2=0x7fda8c15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda8b59e640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c0cd140 0x7fda8c15fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda8b59e640 1 -- 172.21.15.47:0/3584284770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda74009400 con 0x7fda8c15cbf0 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda8b59e640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda8c15cbf0 0x7fda8c1483f0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fda8400c7b0 tx=0x7fda8400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda837fe640 1 -- 172.21.15.47:0/3584284770 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84004b60 con 0x7fda8c15cbf0 2024-03-23T09:52:39.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda837fe640 1 -- 172.21.15.47:0/3584284770 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda84004cc0 con 0x7fda8c15cbf0 2024-03-23T09:52:39.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda837fe640 1 -- 172.21.15.47:0/3584284770 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8400aa30 con 0x7fda8c15cbf0 2024-03-23T09:52:39.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda91051640 1 -- 172.21.15.47:0/3584284770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda8c0b4300 con 0x7fda8c15cbf0 2024-03-23T09:52:39.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.472+0000 7fda91051640 1 -- 172.21.15.47:0/3584284770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda8c0b48c0 con 0x7fda8c15cbf0 2024-03-23T09:52:39.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.473+0000 7fda91051640 1 -- 172.21.15.47:0/3584284770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda8c072360 con 0x7fda8c15cbf0 2024-03-23T09:52:39.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.474+0000 7fda837fe640 1 -- 172.21.15.47:0/3584284770 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fda8400ab90 con 0x7fda8c15cbf0 2024-03-23T09:52:39.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.474+0000 7fda837fe640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda6c06ac00 0x7fda6c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.474+0000 7fda837fe640 1 -- 172.21.15.47:0/3584284770 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fda84089500 con 0x7fda8c15cbf0 2024-03-23T09:52:39.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.475+0000 7fda8ad9d640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda6c06ac00 0x7fda6c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.475+0000 7fda8ad9d640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda6c06ac00 0x7fda6c06d0c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fda7c000f60 tx=0x7fda7c007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:39.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.478+0000 7fda837fe640 1 -- 172.21.15.47:0/3584284770 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fda84054680 con 0x7fda8c15cbf0 2024-03-23T09:52:39.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.656+0000 7fda91051640 1 -- 172.21.15.47:0/3584284770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda8c066be0 con 0x7fda8c15cbf0 2024-03-23T09:52:39.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.657+0000 7fda837fe640 1 -- 172.21.15.47:0/3584284770 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda84057ca0 con 0x7fda8c15cbf0 2024-03-23T09:52:39.657 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-23T09:52:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.659+0000 7fda91051640 1 -- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda6c06ac00 msgr2=0x7fda6c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.659+0000 7fda91051640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda6c06ac00 0x7fda6c06d0c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fda7c000f60 tx=0x7fda7c007c00 comp rx=0 tx=0).stop 2024-03-23T09:52:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.659+0000 7fda91051640 1 -- 172.21.15.47:0/3584284770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda8c15cbf0 msgr2=0x7fda8c1483f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.659+0000 7fda91051640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda8c15cbf0 0x7fda8c1483f0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fda8400c7b0 tx=0x7fda8400cc70 comp rx=0 tx=0).stop 2024-03-23T09:52:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.659+0000 7fda91051640 1 -- 172.21.15.47:0/3584284770 shutdown_connections 2024-03-23T09:52:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.659+0000 7fda91051640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda6c06ac00 0x7fda6c06d0c0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.659+0000 7fda91051640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda8c15cbf0 0x7fda8c1483f0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.659+0000 7fda91051640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c0cf9c0 0x7fda8c0b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.659+0000 7fda91051640 1 --2- 172.21.15.47:0/3584284770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c0cd140 0x7fda8c15fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.659+0000 7fda91051640 1 -- 172.21.15.47:0/3584284770 >> 172.21.15.47:0/3584284770 conn(0x7fda8c06b160 msgr2=0x7fda8c0d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.659+0000 7fda91051640 1 -- 172.21.15.47:0/3584284770 shutdown_connections 2024-03-23T09:52:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.659+0000 7fda91051640 1 -- 172.21.15.47:0/3584284770 wait complete. 2024-03-23T09:52:39.670 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:52:39.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.770+0000 7fbf619a9640 1 Processor -- start 2024-03-23T09:52:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.770+0000 7fbf619a9640 1 -- start start 2024-03-23T09:52:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf619a9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5c142aa0 0x7fbf5c06d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf619a9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5c06c8c0 con 0x7fbf5c0b3c30 2024-03-23T09:52:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf619a9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5c06cba0 con 0x7fbf5c06c350 2024-03-23T09:52:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf619a9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c06ce80 con 0x7fbf5c142aa0 2024-03-23T09:52:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf5affd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbf5c0b3c30 0x7fbf5c0b4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33648/0 (socket says 172.21.15.47:33648) 2024-03-23T09:52:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf5affd640 1 -- 172.21.15.47:0/3999132466 learned_addr learned my addr 172.21.15.47:0/3999132466 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf5b7fe640 1 --2- 172.21.15.47:0/3999132466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5c142aa0 0x7fbf5c06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834345547 0 0) 0x7fbf5c06c8c0 con 0x7fbf5c0b3c30 2024-03-23T09:52:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf48003520 con 0x7fbf5c0b3c30 2024-03-23T09:52:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232521237 0 0) 0x7fbf5c06cba0 con 0x7fbf5c06c350 2024-03-23T09:52:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf5c06c8c0 con 0x7fbf5c06c350 2024-03-23T09:52:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 58999285 0 0) 0x7fbf48003520 con 0x7fbf5c0b3c30 2024-03-23T09:52:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5c142aa0 msgr2=0x7fbf5c06d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf59ffb640 1 --2- 172.21.15.47:0/3999132466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5c142aa0 0x7fbf5c06d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 >> v1:172.21.15.175:6789/0 conn(0x7fbf5c06c350 legacy=0x7fbf5c0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.771+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c06d160 con 0x7fbf5c0b3c30 2024-03-23T09:52:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fbf4c0019f0 con 0x7fbf5c0b3c30 2024-03-23T09:52:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 >> v1:172.21.15.47:6789/0 conn(0x7fbf5c0b3c30 legacy=0x7fbf5c0b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf59ffb640 1 --2- 172.21.15.47:0/3999132466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf48003b30 0x7fbf48003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf59ffb640 1 --2- 172.21.15.47:0/3999132466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf480047c0 0x7fbf48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf59ffb640 1 --2- 172.21.15.47:0/3999132466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5c142aa0 0x7fbf48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c06ce80 con 0x7fbf48003b30 2024-03-23T09:52:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4800ba60 con 0x7fbf480047c0 2024-03-23T09:52:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4800bc20 con 0x7fbf5c142aa0 2024-03-23T09:52:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf5affd640 1 --2- 172.21.15.47:0/3999132466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf48003b30 0x7fbf48003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf5b7fe640 1 --2- 172.21.15.47:0/3999132466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5c142aa0 0x7fbf48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf5a7fc640 1 --2- 172.21.15.47:0/3999132466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf480047c0 0x7fbf48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf5affd640 1 -- 172.21.15.47:0/3999132466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5c142aa0 msgr2=0x7fbf48005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf5affd640 1 --2- 172.21.15.47:0/3999132466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5c142aa0 0x7fbf48005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf5affd640 1 -- 172.21.15.47:0/3999132466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf480047c0 msgr2=0x7fbf48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf5affd640 1 --2- 172.21.15.47:0/3999132466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf480047c0 0x7fbf48004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.772+0000 7fbf5affd640 1 -- 172.21.15.47:0/3999132466 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf48003520 con 0x7fbf48003b30 2024-03-23T09:52:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.773+0000 7fbf5affd640 1 --2- 172.21.15.47:0/3999132466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf48003b30 0x7fbf48003f30 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00a700 tx=0x7fbf480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6fadc6a8377d8f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.773+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c034630 con 0x7fbf48003b30 2024-03-23T09:52:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.773+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c02f070 con 0x7fbf48003b30 2024-03-23T09:52:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.773+0000 7fbf59ffb640 1 -- 172.21.15.47:0/3999132466 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0349b0 con 0x7fbf48003b30 2024-03-23T09:52:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.773+0000 7fbf619a9640 1 -- 172.21.15.47:0/3999132466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf48003b30 msgr2=0x7fbf48003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.773+0000 7fbf619a9640 1 --2- 172.21.15.47:0/3999132466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf48003b30 0x7fbf48003f30 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00a700 tx=0x7fbf480057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.773+0000 7fbf5b7fe640 1 -- 172.21.15.47:0/3999132466 reap_dead start 2024-03-23T09:52:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.773+0000 7fbf619a9640 1 -- 172.21.15.47:0/3999132466 shutdown_connections 2024-03-23T09:52:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.773+0000 7fbf619a9640 1 -- 172.21.15.47:0/3999132466 >> 172.21.15.47:0/3999132466 conn(0x7fbf5c06a310 msgr2=0x7fbf5c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.773+0000 7fbf619a9640 1 -- 172.21.15.47:0/3999132466 shutdown_connections 2024-03-23T09:52:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.773+0000 7fbf619a9640 1 -- 172.21.15.47:0/3999132466 wait complete. 2024-03-23T09:52:39.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.775+0000 7fbf619a9640 1 Processor -- start 2024-03-23T09:52:39.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.775+0000 7fbf619a9640 1 -- start start 2024-03-23T09:52:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.775+0000 7fbf619a9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf48003b30 0x7fbf5c159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.775+0000 7fbf619a9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5c13fd60 0x7fbf5c0aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.775+0000 7fbf619a9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf5c1401e0 0x7fbf5c0ab230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.775+0000 7fbf619a9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c141410 con 0x7fbf48003b30 2024-03-23T09:52:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.775+0000 7fbf619a9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c159780 con 0x7fbf5c1401e0 2024-03-23T09:52:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.775+0000 7fbf619a9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c140790 con 0x7fbf5c13fd60 2024-03-23T09:52:39.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf48003b30 0x7fbf5c159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf48003b30 0x7fbf5c159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48358/0 (socket says 172.21.15.47:48358) 2024-03-23T09:52:39.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf5affd640 1 -- 172.21.15.47:0/1920007698 learned_addr learned my addr 172.21.15.47:0/1920007698 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:39.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf5b7fe640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf5c1401e0 0x7fbf5c0ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf5a7fc640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5c13fd60 0x7fbf5c0aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf5affd640 1 -- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5c13fd60 msgr2=0x7fbf5c0aacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf5affd640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5c13fd60 0x7fbf5c0aacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf5affd640 1 -- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf5c1401e0 msgr2=0x7fbf5c0ab230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:39.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf5affd640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf5c1401e0 0x7fbf5c0ab230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:39.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf5affd640 1 -- 172.21.15.47:0/1920007698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c140a70 con 0x7fbf48003b30 2024-03-23T09:52:39.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf5affd640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf48003b30 0x7fbf5c159240 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00a700 tx=0x7fbf4c001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:39.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf53fff640 1 -- 172.21.15.47:0/1920007698 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c034640 con 0x7fbf48003b30 2024-03-23T09:52:39.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf53fff640 1 -- 172.21.15.47:0/1920007698 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c0347a0 con 0x7fbf48003b30 2024-03-23T09:52:39.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf619a9640 1 -- 172.21.15.47:0/1920007698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5c140d50 con 0x7fbf48003b30 2024-03-23T09:52:39.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf53fff640 1 -- 172.21.15.47:0/1920007698 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c034b10 con 0x7fbf48003b30 2024-03-23T09:52:39.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.776+0000 7fbf619a9640 1 -- 172.21.15.47:0/1920007698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c142290 con 0x7fbf48003b30 2024-03-23T09:52:39.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.777+0000 7fbf619a9640 1 -- 172.21.15.47:0/1920007698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5c073480 con 0x7fbf48003b30 2024-03-23T09:52:39.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.778+0000 7fbf53fff640 1 -- 172.21.15.47:0/1920007698 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbf4c031030 con 0x7fbf48003b30 2024-03-23T09:52:39.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.778+0000 7fbf53fff640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbf3c06ac30 0x7fbf3c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:39.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.778+0000 7fbf53fff640 1 -- 172.21.15.47:0/1920007698 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fbf4c0ab8f0 con 0x7fbf48003b30 2024-03-23T09:52:39.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.778+0000 7fbf5a7fc640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbf3c06ac30 0x7fbf3c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:39.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.779+0000 7fbf5a7fc640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbf3c06ac30 0x7fbf3c06d0f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0af560 tx=0x7fbf44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:39.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:39.782+0000 7fbf53fff640 1 -- 172.21.15.47:0/1920007698 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbf4c076b20 con 0x7fbf48003b30 2024-03-23T09:52:40.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.000+0000 7fbf619a9640 1 -- 172.21.15.47:0/1920007698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf5c066be0 con 0x7fbf48003b30 2024-03-23T09:52:40.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.000+0000 7fbf53fff640 1 -- 172.21.15.47:0/1920007698 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fbf4c07a140 con 0x7fbf48003b30 2024-03-23T09:52:40.001 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":93,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:51:40.395539+0000","modified":"2024-03-23T09:52:30.511498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"c","rank":0,"incarnation":90,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6834/3501786111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3501786111},{"type":"v1","addr":"172.21.15.47:6834","nonce":3501786111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-23T09:52:40.001 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 93 2024-03-23T09:52:40.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.002+0000 7fbf619a9640 1 -- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbf3c06ac30 msgr2=0x7fbf3c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.002+0000 7fbf619a9640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbf3c06ac30 0x7fbf3c06d0f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0af560 tx=0x7fbf44009040 comp rx=0 tx=0).stop 2024-03-23T09:52:40.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.002+0000 7fbf619a9640 1 -- 172.21.15.47:0/1920007698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf48003b30 msgr2=0x7fbf5c159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.002+0000 7fbf619a9640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf48003b30 0x7fbf5c159240 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00a700 tx=0x7fbf4c001cb0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.002+0000 7fbf619a9640 1 -- 172.21.15.47:0/1920007698 shutdown_connections 2024-03-23T09:52:40.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.002+0000 7fbf619a9640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbf3c06ac30 0x7fbf3c06d0f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.002+0000 7fbf619a9640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf5c1401e0 0x7fbf5c0ab230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.002+0000 7fbf619a9640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5c13fd60 0x7fbf5c0aacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.002+0000 7fbf619a9640 1 --2- 172.21.15.47:0/1920007698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf48003b30 0x7fbf5c159240 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.002+0000 7fbf619a9640 1 -- 172.21.15.47:0/1920007698 >> 172.21.15.47:0/1920007698 conn(0x7fbf5c06a310 msgr2=0x7fbf5c0af670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:40.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.002+0000 7fbf619a9640 1 -- 172.21.15.47:0/1920007698 shutdown_connections 2024-03-23T09:52:40.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.002+0000 7fbf619a9640 1 -- 172.21.15.47:0/1920007698 wait complete. 2024-03-23T09:52:40.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:52:40.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.115+0000 7f3b36327640 1 Processor -- start 2024-03-23T09:52:40.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.115+0000 7f3b36327640 1 -- start start 2024-03-23T09:52:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.116+0000 7f3b36327640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b3015ca90 0x7f3b30160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.116+0000 7f3b36327640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b300cde10 0x7f3b300d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.116+0000 7f3b36327640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b300d11e0 con 0x7f3b300b8350 2024-03-23T09:52:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.116+0000 7f3b36327640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b30160f60 con 0x7f3b300cde10 2024-03-23T09:52:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.116+0000 7f3b36327640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30161240 con 0x7f3b3015ca90 2024-03-23T09:52:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.116+0000 7f3b348c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b300cde10 0x7f3b300d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.116+0000 7f3b348c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b300cde10 0x7f3b300d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48368/0 (socket says 172.21.15.47:48368) 2024-03-23T09:52:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.116+0000 7f3b348c4640 1 -- 172.21.15.47:0/3698896553 learned_addr learned my addr 172.21.15.47:0/3698896553 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.116+0000 7f3b2ffff640 1 --2- 172.21.15.47:0/3698896553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b3015ca90 0x7f3b30160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.116+0000 7f3b348c4640 1 -- 172.21.15.47:0/3698896553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b3015ca90 msgr2=0x7f3b30160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.116+0000 7f3b348c4640 1 --2- 172.21.15.47:0/3698896553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b3015ca90 0x7f3b30160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.116+0000 7f3b348c4640 1 -- 172.21.15.47:0/3698896553 >> v1:172.21.15.175:6789/0 conn(0x7f3b300b8350 legacy=0x7f3b300cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.116+0000 7f3b348c4640 1 -- 172.21.15.47:0/3698896553 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b30161520 con 0x7f3b300cde10 2024-03-23T09:52:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.117+0000 7f3b348c4640 1 --2- 172.21.15.47:0/3698896553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b300cde10 0x7f3b300d1670 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009e50 tx=0x7f3b2802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32ef122d0fa68b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.117+0000 7f3b2effd640 1 -- 172.21.15.47:0/3698896553 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2802c9e0 con 0x7f3b300cde10 2024-03-23T09:52:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.117+0000 7f3b2effd640 1 -- 172.21.15.47:0/3698896553 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2802cb40 con 0x7f3b300cde10 2024-03-23T09:52:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.117+0000 7f3b2effd640 1 -- 172.21.15.47:0/3698896553 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b280343f0 con 0x7f3b300cde10 2024-03-23T09:52:40.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.117+0000 7f3b36327640 1 -- 172.21.15.47:0/3698896553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b300cde10 msgr2=0x7f3b300d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.117+0000 7f3b36327640 1 --2- 172.21.15.47:0/3698896553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b300cde10 0x7f3b300d1670 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009e50 tx=0x7f3b2802be40 comp rx=0 tx=0).stop 2024-03-23T09:52:40.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.117+0000 7f3b36327640 1 -- 172.21.15.47:0/3698896553 shutdown_connections 2024-03-23T09:52:40.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.117+0000 7f3b36327640 1 --2- 172.21.15.47:0/3698896553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b300cde10 0x7f3b300d1670 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.117+0000 7f3b36327640 1 --2- 172.21.15.47:0/3698896553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b3015ca90 0x7f3b30160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.117+0000 7f3b36327640 1 -- 172.21.15.47:0/3698896553 >> 172.21.15.47:0/3698896553 conn(0x7f3b300659d0 msgr2=0x7f3b30065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.118+0000 7f3b36327640 1 -- 172.21.15.47:0/3698896553 shutdown_connections 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.118+0000 7f3b36327640 1 -- 172.21.15.47:0/3698896553 wait complete. 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.119+0000 7f3b36327640 1 Processor -- start 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.119+0000 7f3b36327640 1 -- start start 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b36327640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b300b8350 0x7f3b300c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b36327640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b300cde10 0x7f3b300b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b36327640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b3015ca90 0x7f3b30143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b36327640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b300b7e10 con 0x7f3b300b8350 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b36327640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b300b3c30 con 0x7f3b3015ca90 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b36327640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b300c8590 con 0x7f3b300cde10 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b2ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b300b8350 0x7f3b300c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b2ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b300b8350 0x7f3b300c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48374/0 (socket says 172.21.15.47:48374) 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b2ffff640 1 -- 172.21.15.47:0/2761309020 learned_addr learned my addr 172.21.15.47:0/2761309020 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b2ffff640 1 -- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b300cde10 msgr2=0x7f3b300b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b2f7fe640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b300cde10 0x7f3b300b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b348c4640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b3015ca90 0x7f3b30143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b2ffff640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b300cde10 0x7f3b300b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b2ffff640 1 -- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b3015ca90 msgr2=0x7f3b30143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b2ffff640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b3015ca90 0x7f3b30143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b2ffff640 1 -- 172.21.15.47:0/2761309020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b28009b30 con 0x7f3b300b8350 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b2ffff640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b300b8350 0x7f3b300c8050 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400c7f0 tx=0x7f3b2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b2cff9640 1 -- 172.21.15.47:0/2761309020 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b24004220 con 0x7f3b300b8350 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b2cff9640 1 -- 172.21.15.47:0/2761309020 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b240043a0 con 0x7f3b300b8350 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b36327640 1 -- 172.21.15.47:0/2761309020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b300b3ed0 con 0x7f3b300b8350 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b2cff9640 1 -- 172.21.15.47:0/2761309020 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b24007860 con 0x7f3b300b8350 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.120+0000 7f3b36327640 1 -- 172.21.15.47:0/2761309020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b300b4490 con 0x7f3b300b8350 2024-03-23T09:52:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.121+0000 7f3b36327640 1 -- 172.21.15.47:0/2761309020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b30072360 con 0x7f3b300b8350 2024-03-23T09:52:40.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.122+0000 7f3b2cff9640 1 -- 172.21.15.47:0/2761309020 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3b24007bd0 con 0x7f3b300b8350 2024-03-23T09:52:40.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.122+0000 7f3b2cff9640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b1c06ac30 0x7f3b1c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.122+0000 7f3b2cff9640 1 -- 172.21.15.47:0/2761309020 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f3b240893f0 con 0x7f3b300b8350 2024-03-23T09:52:40.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.122+0000 7f3b2f7fe640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b1c06ac30 0x7f3b1c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.123+0000 7f3b2f7fe640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b1c06ac30 0x7f3b1c06d0f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3b300c89e0 tx=0x7f3b20008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:40.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.126+0000 7f3b2cff9640 1 -- 172.21.15.47:0/2761309020 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3b240544e0 con 0x7f3b300b8350 2024-03-23T09:52:40.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.308+0000 7f3b36327640 1 -- 172.21.15.47:0/2761309020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b300b46b0 con 0x7f3b300b8350 2024-03-23T09:52:40.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.309+0000 7f3b2cff9640 1 -- 172.21.15.47:0/2761309020 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f3b24057b00 con 0x7f3b300b8350 2024-03-23T09:52:40.309 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:40.309 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":118,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:52:25.484824+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:51:38.679890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-23T09:51:39.396299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6832/1868958629":"2024-03-24T09:52:25.484801+0000","172.21.15.47:6834/724782441":"2024-03-24T09:52:05.460377+0000","172.21.15.47:6834/1868958629":"2024-03-24T09:52:25.484801+0000","172.21.15.47:6832/724782441":"2024-03-24T09:52:05.460377+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:52:40.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.310+0000 7f3b36327640 1 -- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b1c06ac30 msgr2=0x7f3b1c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.310+0000 7f3b36327640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b1c06ac30 0x7f3b1c06d0f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3b300c89e0 tx=0x7f3b20008040 comp rx=0 tx=0).stop 2024-03-23T09:52:40.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.311+0000 7f3b36327640 1 -- 172.21.15.47:0/2761309020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b300b8350 msgr2=0x7f3b300c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.311+0000 7f3b36327640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b300b8350 0x7f3b300c8050 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400c7f0 tx=0x7f3b2400ccb0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.311+0000 7f3b36327640 1 -- 172.21.15.47:0/2761309020 shutdown_connections 2024-03-23T09:52:40.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.311+0000 7f3b36327640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f3b1c06ac30 0x7f3b1c06d0f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.311+0000 7f3b36327640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b3015ca90 0x7f3b30143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.311+0000 7f3b36327640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b300cde10 0x7f3b300b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.311+0000 7f3b36327640 1 --2- 172.21.15.47:0/2761309020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b300b8350 0x7f3b300c8050 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.311+0000 7f3b36327640 1 -- 172.21.15.47:0/2761309020 >> 172.21.15.47:0/2761309020 conn(0x7f3b300659d0 msgr2=0x7f3b3015f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:40.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.311+0000 7f3b36327640 1 -- 172.21.15.47:0/2761309020 shutdown_connections 2024-03-23T09:52:40.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.311+0000 7f3b36327640 1 -- 172.21.15.47:0/2761309020 wait complete. 2024-03-23T09:52:40.322 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T09:52:40.322 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:52:40.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.422+0000 7f6c76619640 1 Processor -- start 2024-03-23T09:52:40.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.422+0000 7f6c76619640 1 -- start start 2024-03-23T09:52:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.423+0000 7f6c76619640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6c700b0670 0x7f6c70069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.423+0000 7f6c76619640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c700d01a0 0x7f6c700b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.423+0000 7f6c76619640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c7006a370 con 0x7f6c7015ac10 2024-03-23T09:52:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.423+0000 7f6c76619640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7006a650 con 0x7f6c700b0670 2024-03-23T09:52:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.423+0000 7f6c76619640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7015b150 con 0x7f6c700d01a0 2024-03-23T09:52:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.423+0000 7f6c6ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6c700b0670 0x7f6c70069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.423+0000 7f6c6ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6c700b0670 0x7f6c70069ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48384/0 (socket says 172.21.15.47:48384) 2024-03-23T09:52:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.423+0000 7f6c6ffff640 1 -- 172.21.15.47:0/108316006 learned_addr learned my addr 172.21.15.47:0/108316006 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.423+0000 7f6c6f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6c7015ac10 0x7f6c7015b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33656/0 (socket says 172.21.15.47:33656) 2024-03-23T09:52:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.424+0000 7f6c74bb6640 1 --2- 172.21.15.47:0/108316006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c700d01a0 0x7f6c700b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.424+0000 7f6c6ffff640 1 -- 172.21.15.47:0/108316006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c700d01a0 msgr2=0x7f6c700b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.424+0000 7f6c6ffff640 1 --2- 172.21.15.47:0/108316006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c700d01a0 0x7f6c700b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.424+0000 7f6c6ffff640 1 -- 172.21.15.47:0/108316006 >> v1:172.21.15.47:6789/0 conn(0x7f6c7015ac10 legacy=0x7f6c7015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.424+0000 7f6c6ffff640 1 -- 172.21.15.47:0/108316006 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7015b430 con 0x7f6c700b0670 2024-03-23T09:52:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.424+0000 7f6c6effd640 1 -- 172.21.15.47:0/108316006 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668012197 0 0) 0x7f6c7006a370 con 0x7f6c7015ac10 2024-03-23T09:52:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.424+0000 7f6c6ffff640 1 --2- 172.21.15.47:0/108316006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c700b0670 0x7f6c70069ce0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009870 tx=0x7f6c6002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bc7711f9695b9f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.425+0000 7f6c6effd640 1 -- 172.21.15.47:0/108316006 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60030840 con 0x7f6c700b0670 2024-03-23T09:52:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.425+0000 7f6c6effd640 1 -- 172.21.15.47:0/108316006 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c600309a0 con 0x7f6c700b0670 2024-03-23T09:52:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.425+0000 7f6c6effd640 1 -- 172.21.15.47:0/108316006 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60030d30 con 0x7f6c700b0670 2024-03-23T09:52:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.425+0000 7f6c76619640 1 -- 172.21.15.47:0/108316006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c700b0670 msgr2=0x7f6c70069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.425+0000 7f6c76619640 1 --2- 172.21.15.47:0/108316006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c700b0670 0x7f6c70069ce0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009870 tx=0x7f6c6002b980 comp rx=0 tx=0).stop 2024-03-23T09:52:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.425+0000 7f6c76619640 1 -- 172.21.15.47:0/108316006 shutdown_connections 2024-03-23T09:52:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.425+0000 7f6c76619640 1 --2- 172.21.15.47:0/108316006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c700d01a0 0x7f6c700b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.425+0000 7f6c76619640 1 --2- 172.21.15.47:0/108316006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c700b0670 0x7f6c70069ce0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.425+0000 7f6c76619640 1 -- 172.21.15.47:0/108316006 >> 172.21.15.47:0/108316006 conn(0x7f6c70098a00 msgr2=0x7f6c700cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.425+0000 7f6c76619640 1 -- 172.21.15.47:0/108316006 shutdown_connections 2024-03-23T09:52:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.425+0000 7f6c76619640 1 -- 172.21.15.47:0/108316006 wait complete. 2024-03-23T09:52:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.426+0000 7f6c76619640 1 Processor -- start 2024-03-23T09:52:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.427+0000 7f6c76619640 1 -- start start 2024-03-23T09:52:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.427+0000 7f6c76619640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c700b0670 0x7f6c7014f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.427+0000 7f6c76619640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c700d01a0 0x7f6c700c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.427+0000 7f6c76619640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c7015ac10 0x7f6c700c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.427+0000 7f6c76619640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c700c5460 con 0x7f6c7015ac10 2024-03-23T09:52:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.427+0000 7f6c76619640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c700c48b0 con 0x7f6c700b0670 2024-03-23T09:52:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.427+0000 7f6c76619640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c700c45d0 con 0x7f6c700d01a0 2024-03-23T09:52:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.427+0000 7f6c74bb6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c7015ac10 0x7f6c700c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.427+0000 7f6c74bb6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c7015ac10 0x7f6c700c6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48396/0 (socket says 172.21.15.47:48396) 2024-03-23T09:52:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.427+0000 7f6c74bb6640 1 -- 172.21.15.47:0/217536902 learned_addr learned my addr 172.21.15.47:0/217536902 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c6f7fe640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c700d01a0 0x7f6c700c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c6ffff640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c700b0670 0x7f6c7014f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c74bb6640 1 -- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c700d01a0 msgr2=0x7f6c700c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c74bb6640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c700d01a0 0x7f6c700c3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c74bb6640 1 -- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c700b0670 msgr2=0x7f6c7014f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c74bb6640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c700b0670 0x7f6c7014f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c74bb6640 1 -- 172.21.15.47:0/217536902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c600094d0 con 0x7f6c7015ac10 2024-03-23T09:52:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c6f7fe640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c700d01a0 0x7f6c700c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c6ffff640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c700b0670 0x7f6c7014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c74bb6640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c7015ac10 0x7f6c700c6950 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f6c68007e30 tx=0x7f6c6800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c6cff9640 1 -- 172.21.15.47:0/217536902 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68014620 con 0x7f6c7015ac10 2024-03-23T09:52:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c6cff9640 1 -- 172.21.15.47:0/217536902 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6800ee90 con 0x7f6c7015ac10 2024-03-23T09:52:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c76619640 1 -- 172.21.15.47:0/217536902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c700c4da0 con 0x7f6c7015ac10 2024-03-23T09:52:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c6cff9640 1 -- 172.21.15.47:0/217536902 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680149c0 con 0x7f6c7015ac10 2024-03-23T09:52:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.428+0000 7f6c76619640 1 -- 172.21.15.47:0/217536902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7014fe10 con 0x7f6c7015ac10 2024-03-23T09:52:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.429+0000 7f6c76619640 1 -- 172.21.15.47:0/217536902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c700c55a0 con 0x7f6c7015ac10 2024-03-23T09:52:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.429+0000 7f6c6cff9640 1 -- 172.21.15.47:0/217536902 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6c68014b20 con 0x7f6c7015ac10 2024-03-23T09:52:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.430+0000 7f6c6cff9640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6c5006ac00 0x7f6c5006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.430+0000 7f6c6cff9640 1 -- 172.21.15.47:0/217536902 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f6c68089de0 con 0x7f6c7015ac10 2024-03-23T09:52:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.430+0000 7f6c6ffff640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6c5006ac00 0x7f6c5006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.431+0000 7f6c6ffff640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6c5006ac00 0x7f6c5006d0c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6c60036000 tx=0x7f6c6003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.434+0000 7f6c6cff9640 1 -- 172.21.15.47:0/217536902 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6c700c55a0 con 0x7f6c7015ac10 2024-03-23T09:52:40.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.652+0000 7f6c76619640 1 -- 172.21.15.47:0/217536902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c700c55a0 con 0x7f6c7015ac10 2024-03-23T09:52:40.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.653+0000 7f6c6cff9640 1 -- 172.21.15.47:0/217536902 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f6c700c55a0 con 0x7f6c7015ac10 2024-03-23T09:52:40.653 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:40.653 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":93,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:51:40.395539+0000","modified":"2024-03-23T09:52:30.511498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"c","rank":0,"incarnation":90,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6834/3501786111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3501786111},{"type":"v1","addr":"172.21.15.47:6834","nonce":3501786111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-23T09:52:40.653 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 93 2024-03-23T09:52:40.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.655+0000 7f6c76619640 1 -- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6c5006ac00 msgr2=0x7f6c5006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.655+0000 7f6c76619640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6c5006ac00 0x7f6c5006d0c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6c60036000 tx=0x7f6c6003b040 comp rx=0 tx=0).stop 2024-03-23T09:52:40.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.655+0000 7f6c76619640 1 -- 172.21.15.47:0/217536902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c7015ac10 msgr2=0x7f6c700c6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.655+0000 7f6c76619640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c7015ac10 0x7f6c700c6950 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f6c68007e30 tx=0x7f6c6800e4b0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.655+0000 7f6c76619640 1 -- 172.21.15.47:0/217536902 shutdown_connections 2024-03-23T09:52:40.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.655+0000 7f6c76619640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6c5006ac00 0x7f6c5006d0c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.655+0000 7f6c76619640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c7015ac10 0x7f6c700c6950 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.655+0000 7f6c76619640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c700d01a0 0x7f6c700c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.655+0000 7f6c76619640 1 --2- 172.21.15.47:0/217536902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c700b0670 0x7f6c7014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.655+0000 7f6c76619640 1 -- 172.21.15.47:0/217536902 >> 172.21.15.47:0/217536902 conn(0x7f6c70098a00 msgr2=0x7f6c7015ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:40.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.655+0000 7f6c76619640 1 -- 172.21.15.47:0/217536902 shutdown_connections 2024-03-23T09:52:40.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.655+0000 7f6c76619640 1 -- 172.21.15.47:0/217536902 wait complete. 2024-03-23T09:52:40.666 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:52:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.766+0000 7fee5d58b640 1 Processor -- start 2024-03-23T09:52:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.766+0000 7fee5d58b640 1 -- start start 2024-03-23T09:52:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.767+0000 7fee5d58b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee5815cbc0 0x7fee580cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.767+0000 7fee5d58b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fee5806a3f0 0x7fee580ca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.767+0000 7fee5d58b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee580c9ed0 con 0x7fee58098e10 2024-03-23T09:52:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.767+0000 7fee5d58b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee580b0d10 con 0x7fee5806a3f0 2024-03-23T09:52:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.767+0000 7fee5d58b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee580b0ff0 con 0x7fee5815cbc0 2024-03-23T09:52:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.767+0000 7fee567fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fee58098e10 0x7fee58069ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33664/0 (socket says 172.21.15.47:33664) 2024-03-23T09:52:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.767+0000 7fee567fc640 1 -- 172.21.15.47:0/1491654964 learned_addr learned my addr 172.21.15.47:0/1491654964 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee56ffd640 1 --2- 172.21.15.47:0/1491654964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee5815cbc0 0x7fee580cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee577fe640 1 --2- 172.21.15.47:0/1491654964 >> v2:172.21.15.175:3300/0 conn(0x7fee5806a3f0 0x7fee580ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872749750 0 0) 0x7fee580c9ed0 con 0x7fee58098e10 2024-03-23T09:52:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee44003520 con 0x7fee58098e10 2024-03-23T09:52:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 892986896 0 0) 0x7fee44003520 con 0x7fee58098e10 2024-03-23T09:52:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee5815cbc0 msgr2=0x7fee580cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 --2- 172.21.15.47:0/1491654964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee5815cbc0 0x7fee580cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 >> v2:172.21.15.175:3300/0 conn(0x7fee5806a3f0 msgr2=0x7fee580ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 --2- 172.21.15.47:0/1491654964 >> v2:172.21.15.175:3300/0 conn(0x7fee5806a3f0 0x7fee580ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee580ca120 con 0x7fee58098e10 2024-03-23T09:52:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fee40001670 con 0x7fee58098e10 2024-03-23T09:52:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 >> v1:172.21.15.47:6789/0 conn(0x7fee58098e10 legacy=0x7fee58069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 --2- 172.21.15.47:0/1491654964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee44003970 0x7fee44003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 --2- 172.21.15.47:0/1491654964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee440047c0 0x7fee44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 --2- 172.21.15.47:0/1491654964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee5815cbc0 0x7fee44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee580b0d10 con 0x7fee44003970 2024-03-23T09:52:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee580b0ff0 con 0x7fee440047c0 2024-03-23T09:52:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4400b980 con 0x7fee5815cbc0 2024-03-23T09:52:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee56ffd640 1 --2- 172.21.15.47:0/1491654964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee44003970 0x7fee44003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee567fc640 1 --2- 172.21.15.47:0/1491654964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee440047c0 0x7fee44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee577fe640 1 --2- 172.21.15.47:0/1491654964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee5815cbc0 0x7fee44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee56ffd640 1 -- 172.21.15.47:0/1491654964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee5815cbc0 msgr2=0x7fee44005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee56ffd640 1 --2- 172.21.15.47:0/1491654964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee5815cbc0 0x7fee44005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee56ffd640 1 -- 172.21.15.47:0/1491654964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee440047c0 msgr2=0x7fee44004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee56ffd640 1 --2- 172.21.15.47:0/1491654964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee440047c0 0x7fee44004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.768+0000 7fee56ffd640 1 -- 172.21.15.47:0/1491654964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee44003520 con 0x7fee44003970 2024-03-23T09:52:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.769+0000 7fee56ffd640 1 --2- 172.21.15.47:0/1491654964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee44003970 0x7fee44003d90 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fee50009ac0 tx=0x7fee440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794947f30f01e9e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.769+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5002c6b0 con 0x7fee44003970 2024-03-23T09:52:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.769+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5002c810 con 0x7fee44003970 2024-03-23T09:52:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.769+0000 7fee55ffb640 1 -- 172.21.15.47:0/1491654964 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5002cba0 con 0x7fee44003970 2024-03-23T09:52:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.770+0000 7fee5d58b640 1 -- 172.21.15.47:0/1491654964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee44003970 msgr2=0x7fee44003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.770+0000 7fee5d58b640 1 --2- 172.21.15.47:0/1491654964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee44003970 0x7fee44003d90 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fee50009ac0 tx=0x7fee440057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.770+0000 7fee577fe640 1 -- 172.21.15.47:0/1491654964 reap_dead start 2024-03-23T09:52:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.770+0000 7fee5d58b640 1 -- 172.21.15.47:0/1491654964 shutdown_connections 2024-03-23T09:52:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.770+0000 7fee5d58b640 1 -- 172.21.15.47:0/1491654964 >> 172.21.15.47:0/1491654964 conn(0x7fee58098a00 msgr2=0x7fee580cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.770+0000 7fee5d58b640 1 -- 172.21.15.47:0/1491654964 shutdown_connections 2024-03-23T09:52:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.770+0000 7fee5d58b640 1 -- 172.21.15.47:0/1491654964 wait complete. 2024-03-23T09:52:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.771+0000 7fee5d58b640 1 Processor -- start 2024-03-23T09:52:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.771+0000 7fee5d58b640 1 -- start start 2024-03-23T09:52:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee5d58b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee44003970 0x7fee580c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee5d58b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee580c7f70 0x7fee5814ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee5d58b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee581f2610 0x7fee581f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee5d58b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5814f2c0 con 0x7fee44003970 2024-03-23T09:52:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee5d58b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee580c7ae0 con 0x7fee580c7f70 2024-03-23T09:52:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee5d58b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee580ad9c0 con 0x7fee581f2610 2024-03-23T09:52:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee56ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee44003970 0x7fee580c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee56ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee44003970 0x7fee580c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48414/0 (socket says 172.21.15.47:48414) 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee56ffd640 1 -- 172.21.15.47:0/594625903 learned_addr learned my addr 172.21.15.47:0/594625903 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee577fe640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee581f2610 0x7fee581f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee567fc640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee580c7f70 0x7fee5814ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee56ffd640 1 -- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee581f2610 msgr2=0x7fee581f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee56ffd640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee581f2610 0x7fee581f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee56ffd640 1 -- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee580c7f70 msgr2=0x7fee5814ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee56ffd640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee580c7f70 0x7fee5814ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee56ffd640 1 -- 172.21.15.47:0/594625903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee580ad6e0 con 0x7fee44003970 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee577fe640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee581f2610 0x7fee581f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee567fc640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee580c7f70 0x7fee5814ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee56ffd640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee44003970 0x7fee580c97b0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fee5002c250 tx=0x7fee50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.772+0000 7fee4f7fe640 1 -- 172.21.15.47:0/594625903 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50003810 con 0x7fee44003970 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.773+0000 7fee4f7fe640 1 -- 172.21.15.47:0/594625903 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee50003970 con 0x7fee44003970 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.773+0000 7fee5d58b640 1 -- 172.21.15.47:0/594625903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee580c7490 con 0x7fee44003970 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.773+0000 7fee4f7fe640 1 -- 172.21.15.47:0/594625903 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50003ce0 con 0x7fee44003970 2024-03-23T09:52:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.773+0000 7fee5d58b640 1 -- 172.21.15.47:0/594625903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee581f2fd0 con 0x7fee44003970 2024-03-23T09:52:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.773+0000 7fee5d58b640 1 -- 172.21.15.47:0/594625903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee580672f0 con 0x7fee44003970 2024-03-23T09:52:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.774+0000 7fee4f7fe640 1 -- 172.21.15.47:0/594625903 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fee50047450 con 0x7fee44003970 2024-03-23T09:52:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.774+0000 7fee4f7fe640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3806ac00 0x7fee3806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.774+0000 7fee4f7fe640 1 -- 172.21.15.47:0/594625903 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fee500ab7a0 con 0x7fee44003970 2024-03-23T09:52:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.775+0000 7fee567fc640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3806ac00 0x7fee3806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.775+0000 7fee567fc640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3806ac00 0x7fee3806d0c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fee40000f60 tx=0x7fee40000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:40.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.779+0000 7fee4f7fe640 1 -- 172.21.15.47:0/594625903 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fee50076af0 con 0x7fee44003970 2024-03-23T09:52:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.998+0000 7fee5d58b640 1 -- 172.21.15.47:0/594625903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee5806ab00 con 0x7fee44003970 2024-03-23T09:52:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:40.998+0000 7fee4f7fe640 1 -- 172.21.15.47:0/594625903 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fee500044e0 con 0x7fee44003970 2024-03-23T09:52:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:40.999 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":93,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T09:51:40.395539+0000","modified":"2024-03-23T09:52:30.511498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"c","rank":0,"incarnation":90,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6834/3501786111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3501786111},{"type":"v1","addr":"172.21.15.47:6834","nonce":3501786111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-23T09:52:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 93 2024-03-23T09:52:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.000+0000 7fee5d58b640 1 -- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3806ac00 msgr2=0x7fee3806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.000+0000 7fee5d58b640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3806ac00 0x7fee3806d0c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fee40000f60 tx=0x7fee40000ed0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.000+0000 7fee5d58b640 1 -- 172.21.15.47:0/594625903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee44003970 msgr2=0x7fee580c97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.000+0000 7fee5d58b640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee44003970 0x7fee580c97b0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fee5002c250 tx=0x7fee50004730 comp rx=0 tx=0).stop 2024-03-23T09:52:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.000+0000 7fee5d58b640 1 -- 172.21.15.47:0/594625903 shutdown_connections 2024-03-23T09:52:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.000+0000 7fee5d58b640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fee3806ac00 0x7fee3806d0c0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.000+0000 7fee5d58b640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee581f2610 0x7fee581f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.000+0000 7fee5d58b640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee580c7f70 0x7fee5814ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.000+0000 7fee5d58b640 1 --2- 172.21.15.47:0/594625903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee44003970 0x7fee580c97b0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.000+0000 7fee5d58b640 1 -- 172.21.15.47:0/594625903 >> 172.21.15.47:0/594625903 conn(0x7fee58098a00 msgr2=0x7fee580cc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.000+0000 7fee5d58b640 1 -- 172.21.15.47:0/594625903 shutdown_connections 2024-03-23T09:52:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.000+0000 7fee5d58b640 1 -- 172.21.15.47:0/594625903 wait complete. 2024-03-23T09:52:41.011 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:52:41.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.112+0000 7f83c9619640 1 Processor -- start 2024-03-23T09:52:41.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.112+0000 7f83c9619640 1 -- start start 2024-03-23T09:52:41.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.113+0000 7f83c9619640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83c40cec50 0x7f83c40cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.113+0000 7f83c9619640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f83c40b0b90 0x7f83c40b1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.113+0000 7f83c9619640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c40cc610 con 0x7f83c415a8d0 2024-03-23T09:52:41.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.113+0000 7f83c9619640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c40c41c0 con 0x7f83c40b0b90 2024-03-23T09:52:41.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.113+0000 7f83c9619640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c40cf610 con 0x7f83c40cec50 2024-03-23T09:52:41.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.114+0000 7f83c37fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f83c40b0b90 0x7f83c40b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.114+0000 7f83c27fc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83c40cec50 0x7f83c40cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.114+0000 7f83c37fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f83c40b0b90 0x7f83c40b1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:37730/0 (socket says 172.21.15.47:37730) 2024-03-23T09:52:41.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.114+0000 7f83c37fe640 1 -- 172.21.15.47:0/460273171 learned_addr learned my addr 172.21.15.47:0/460273171 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:41.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.114+0000 7f83c27fc640 1 -- 172.21.15.47:0/460273171 >> v2:172.21.15.175:3300/0 conn(0x7f83c40b0b90 msgr2=0x7f83c40b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:41.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.114+0000 7f83c27fc640 1 --2- 172.21.15.47:0/460273171 >> v2:172.21.15.175:3300/0 conn(0x7f83c40b0b90 0x7f83c40b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.114+0000 7f83c27fc640 1 -- 172.21.15.47:0/460273171 >> v1:172.21.15.175:6789/0 conn(0x7f83c415a8d0 legacy=0x7f83c40cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.114+0000 7f83c27fc640 1 -- 172.21.15.47:0/460273171 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c40b0700 con 0x7f83c40cec50 2024-03-23T09:52:41.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.114+0000 7f83c37fe640 1 --2- 172.21.15.47:0/460273171 >> v2:172.21.15.175:3300/0 conn(0x7f83c40b0b90 0x7f83c40b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:52:41.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.115+0000 7f83c27fc640 1 --2- 172.21.15.47:0/460273171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83c40cec50 0x7f83c40cf0d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f83b80097a0 tx=0x7f83b802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=becbf47a05122bd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:41.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.115+0000 7f83c1ffb640 1 -- 172.21.15.47:0/460273171 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b802e070 con 0x7f83c40cec50 2024-03-23T09:52:41.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.115+0000 7f83c1ffb640 1 -- 172.21.15.47:0/460273171 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b80315a0 con 0x7f83c40cec50 2024-03-23T09:52:41.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.115+0000 7f83c1ffb640 1 -- 172.21.15.47:0/460273171 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8031930 con 0x7f83c40cec50 2024-03-23T09:52:41.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.115+0000 7f83c9619640 1 -- 172.21.15.47:0/460273171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83c40cec50 msgr2=0x7f83c40cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.115+0000 7f83c9619640 1 --2- 172.21.15.47:0/460273171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83c40cec50 0x7f83c40cf0d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f83b80097a0 tx=0x7f83b802b840 comp rx=0 tx=0).stop 2024-03-23T09:52:41.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.115+0000 7f83c9619640 1 -- 172.21.15.47:0/460273171 shutdown_connections 2024-03-23T09:52:41.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.115+0000 7f83c9619640 1 --2- 172.21.15.47:0/460273171 >> v2:172.21.15.175:3300/0 conn(0x7f83c40b0b90 0x7f83c40b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.115+0000 7f83c9619640 1 --2- 172.21.15.47:0/460273171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83c40cec50 0x7f83c40cf0d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.115+0000 7f83c9619640 1 -- 172.21.15.47:0/460273171 >> 172.21.15.47:0/460273171 conn(0x7f83c40d0670 msgr2=0x7f83c40d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:41.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.116+0000 7f83c9619640 1 -- 172.21.15.47:0/460273171 shutdown_connections 2024-03-23T09:52:41.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.116+0000 7f83c9619640 1 -- 172.21.15.47:0/460273171 wait complete. 2024-03-23T09:52:41.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.117+0000 7f83c9619640 1 Processor -- start 2024-03-23T09:52:41.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.117+0000 7f83c9619640 1 -- start start 2024-03-23T09:52:41.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.117+0000 7f83c9619640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83c40b0b90 0x7f83c40b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.117+0000 7f83c9619640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c40cec50 0x7f83c41f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c9619640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83c415a8d0 0x7f83c40aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c9619640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c40c7a70 con 0x7f83c40cec50 2024-03-23T09:52:41.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c9619640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c40aea80 con 0x7f83c40b0b90 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c9619640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c41f0ea0 con 0x7f83c415a8d0 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c40cec50 0x7f83c41f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c40cec50 0x7f83c41f12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48420/0 (socket says 172.21.15.47:48420) 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c27fc640 1 -- 172.21.15.47:0/2346577111 learned_addr learned my addr 172.21.15.47:0/2346577111 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c2ffd640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83c40b0b90 0x7f83c40b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c37fe640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83c415a8d0 0x7f83c40aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c27fc640 1 -- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83c415a8d0 msgr2=0x7f83c40aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c27fc640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83c415a8d0 0x7f83c40aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c27fc640 1 -- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83c40b0b90 msgr2=0x7f83c40b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c27fc640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83c40b0b90 0x7f83c40b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c27fc640 1 -- 172.21.15.47:0/2346577111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b4009b30 con 0x7f83c40cec50 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c2ffd640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83c40b0b90 0x7f83c40b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c27fc640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c40cec50 0x7f83c41f12c0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f83b80098c0 tx=0x7f83b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83ab7fe640 1 -- 172.21.15.47:0/2346577111 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8032840 con 0x7f83c40cec50 2024-03-23T09:52:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83ab7fe640 1 -- 172.21.15.47:0/2346577111 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b80329c0 con 0x7f83c40cec50 2024-03-23T09:52:41.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c9619640 1 -- 172.21.15.47:0/2346577111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b8009400 con 0x7f83c40cec50 2024-03-23T09:52:41.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83ab7fe640 1 -- 172.21.15.47:0/2346577111 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8032d70 con 0x7f83c40cec50 2024-03-23T09:52:41.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.118+0000 7f83c9619640 1 -- 172.21.15.47:0/2346577111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c40ae710 con 0x7f83c40cec50 2024-03-23T09:52:41.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.119+0000 7f83c9619640 1 -- 172.21.15.47:0/2346577111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c40aec10 con 0x7f83c40cec50 2024-03-23T09:52:41.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.120+0000 7f83ab7fe640 1 -- 172.21.15.47:0/2346577111 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f83b804e430 con 0x7f83c40cec50 2024-03-23T09:52:41.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.120+0000 7f83ab7fe640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f83b006ac00 0x7f83b006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.120+0000 7f83ab7fe640 1 -- 172.21.15.47:0/2346577111 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f83b80ab3c0 con 0x7f83c40cec50 2024-03-23T09:52:41.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.120+0000 7f83c2ffd640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f83b006ac00 0x7f83b006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.121+0000 7f83c2ffd640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f83b006ac00 0x7f83b006d0c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f83bc007fd0 tx=0x7f83bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:41.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.125+0000 7f83ab7fe640 1 -- 172.21.15.47:0/2346577111 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f83b8076540 con 0x7f83c40cec50 2024-03-23T09:52:41.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.304+0000 7f83c9619640 1 -- 172.21.15.47:0/2346577111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83c40ce990 con 0x7f83c40cec50 2024-03-23T09:52:41.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.305+0000 7f83ab7fe640 1 -- 172.21.15.47:0/2346577111 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f83b8079b60 con 0x7f83c40cec50 2024-03-23T09:52:41.306 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:41.306 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":118,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:52:25.484824+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-23T09:51:38.679890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-23T09:51:39.396299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6832/1868958629":"2024-03-24T09:52:25.484801+0000","172.21.15.47:6834/724782441":"2024-03-24T09:52:05.460377+0000","172.21.15.47:6834/1868958629":"2024-03-24T09:52:25.484801+0000","172.21.15.47:6832/724782441":"2024-03-24T09:52:05.460377+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:52:41.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.307+0000 7f83c9619640 1 -- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f83b006ac00 msgr2=0x7f83b006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.307+0000 7f83c9619640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f83b006ac00 0x7f83b006d0c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f83bc007fd0 tx=0x7f83bc009040 comp rx=0 tx=0).stop 2024-03-23T09:52:41.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.307+0000 7f83c9619640 1 -- 172.21.15.47:0/2346577111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c40cec50 msgr2=0x7f83c41f12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.307+0000 7f83c9619640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c40cec50 0x7f83c41f12c0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f83b80098c0 tx=0x7f83b80046c0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.307+0000 7f83c9619640 1 -- 172.21.15.47:0/2346577111 shutdown_connections 2024-03-23T09:52:41.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.307+0000 7f83c9619640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f83b006ac00 0x7f83b006d0c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.307+0000 7f83c9619640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83c415a8d0 0x7f83c40aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.307+0000 7f83c9619640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c40cec50 0x7f83c41f12c0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.307+0000 7f83c9619640 1 --2- 172.21.15.47:0/2346577111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83c40b0b90 0x7f83c40b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.307+0000 7f83c9619640 1 -- 172.21.15.47:0/2346577111 >> 172.21.15.47:0/2346577111 conn(0x7f83c40d0670 msgr2=0x7f83c406f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:41.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.308+0000 7f83c9619640 1 -- 172.21.15.47:0/2346577111 shutdown_connections 2024-03-23T09:52:41.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.308+0000 7f83c9619640 1 -- 172.21.15.47:0/2346577111 wait complete. 2024-03-23T09:52:41.319 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T09:52:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.420+0000 7f6e4a39f640 1 Processor -- start 2024-03-23T09:52:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.420+0000 7f6e4a39f640 1 -- start start 2024-03-23T09:52:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.421+0000 7f6e4a39f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415d0a0 0x7f6e440d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.421+0000 7f6e4a39f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e4406b210 0x7f6e4406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.421+0000 7f6e4a39f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e440ce090 con 0x7f6e4415ccc0 2024-03-23T09:52:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.421+0000 7f6e4a39f640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4406c0e0 con 0x7f6e4406b210 2024-03-23T09:52:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.421+0000 7f6e4a39f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4406bc00 con 0x7f6e4415d0a0 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.421+0000 7f6e43fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6e4415ccc0 0x7f6e440cd980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33678/0 (socket says 172.21.15.47:33678) 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.421+0000 7f6e43fff640 1 -- 172.21.15.47:0/3889864750 learned_addr learned my addr 172.21.15.47:0/3889864750 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.421+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1902462246 0 0) 0x7f6e2c001110 con 0x7f6e4415ccc0 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.421+0000 7f6e4893c640 1 --2- 172.21.15.47:0/3889864750 >> v2:172.21.15.175:3300/0 conn(0x7f6e4406b210 0x7f6e4406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.421+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e30003520 con 0x7f6e4415ccc0 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.421+0000 7f6e437fe640 1 --2- 172.21.15.47:0/3889864750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415d0a0 0x7f6e440d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1647822994 0 0) 0x7f6e30003520 con 0x7f6e4415ccc0 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415d0a0 msgr2=0x7f6e440d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 --2- 172.21.15.47:0/3889864750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415d0a0 0x7f6e440d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 >> v2:172.21.15.175:3300/0 conn(0x7f6e4406b210 msgr2=0x7f6e4406b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 --2- 172.21.15.47:0/3889864750 >> v2:172.21.15.175:3300/0 conn(0x7f6e4406b210 0x7f6e4406b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440d0f00 con 0x7f6e4415ccc0 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f6e2c001800 con 0x7f6e4415ccc0 2024-03-23T09:52:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 >> v1:172.21.15.47:6789/0 conn(0x7f6e4415ccc0 legacy=0x7f6e440cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 --2- 172.21.15.47:0/3889864750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e30003970 0x7f6e30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 --2- 172.21.15.47:0/3889864750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415d0a0 0x7f6e300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 --2- 172.21.15.47:0/3889864750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e30004d00 0x7f6e30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4406c0e0 con 0x7f6e30004d00 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4406bc00 con 0x7f6e30003970 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3000b9b0 con 0x7f6e4415d0a0 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e4893c640 1 --2- 172.21.15.47:0/3889864750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e30004d00 0x7f6e30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e43fff640 1 --2- 172.21.15.47:0/3889864750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e30003970 0x7f6e30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e437fe640 1 --2- 172.21.15.47:0/3889864750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415d0a0 0x7f6e300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e4893c640 1 -- 172.21.15.47:0/3889864750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415d0a0 msgr2=0x7f6e300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e4893c640 1 --2- 172.21.15.47:0/3889864750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415d0a0 0x7f6e300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e4893c640 1 -- 172.21.15.47:0/3889864750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e30003970 msgr2=0x7f6e30003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e4893c640 1 --2- 172.21.15.47:0/3889864750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e30003970 0x7f6e30003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.422+0000 7f6e4893c640 1 -- 172.21.15.47:0/3889864750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e30003520 con 0x7f6e30004d00 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.423+0000 7f6e4893c640 1 --2- 172.21.15.47:0/3889864750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e30004d00 0x7f6e30005180 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009fc0 tx=0x7f6e300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f526c4a133d748c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.423+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c02cc60 con 0x7f6e30004d00 2024-03-23T09:52:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.423+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c00f040 con 0x7f6e30004d00 2024-03-23T09:52:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.423+0000 7f6e42ffd640 1 -- 172.21.15.47:0/3889864750 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c034440 con 0x7f6e30004d00 2024-03-23T09:52:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.424+0000 7f6e4a39f640 1 -- 172.21.15.47:0/3889864750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e30004d00 msgr2=0x7f6e30005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.424+0000 7f6e4a39f640 1 --2- 172.21.15.47:0/3889864750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e30004d00 0x7f6e30005180 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009fc0 tx=0x7f6e300057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.424+0000 7f6e4893c640 1 -- 172.21.15.47:0/3889864750 reap_dead start 2024-03-23T09:52:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.424+0000 7f6e4a39f640 1 -- 172.21.15.47:0/3889864750 shutdown_connections 2024-03-23T09:52:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.424+0000 7f6e4a39f640 1 -- 172.21.15.47:0/3889864750 >> 172.21.15.47:0/3889864750 conn(0x7f6e440cf430 msgr2=0x7f6e440cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.424+0000 7f6e4a39f640 1 -- 172.21.15.47:0/3889864750 shutdown_connections 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.424+0000 7f6e4a39f640 1 -- 172.21.15.47:0/3889864750 wait complete. 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.425+0000 7f6e4a39f640 1 Processor -- start 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.425+0000 7f6e4a39f640 1 -- start start 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e4a39f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415f350 0x7f6e440cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e4a39f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e441f3120 0x7f6e44156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e4a39f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e4415f730 0x7f6e441572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e4a39f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440cccf0 con 0x7f6e441f3120 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e4a39f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4415d650 con 0x7f6e4415f730 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e4a39f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4415d930 con 0x7f6e4415f350 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e437fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e441f3120 0x7f6e44156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e437fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e441f3120 0x7f6e44156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48434/0 (socket says 172.21.15.47:48434) 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e437fe640 1 -- 172.21.15.47:0/970740802 learned_addr learned my addr 172.21.15.47:0/970740802 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e4893c640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e4415f730 0x7f6e441572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e43fff640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415f350 0x7f6e440cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e437fe640 1 -- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415f350 msgr2=0x7f6e440cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e437fe640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415f350 0x7f6e440cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e437fe640 1 -- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e4415f730 msgr2=0x7f6e441572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e437fe640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e4415f730 0x7f6e441572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e437fe640 1 -- 172.21.15.47:0/970740802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4415dc10 con 0x7f6e441f3120 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e4893c640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e4415f730 0x7f6e441572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e43fff640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415f350 0x7f6e440cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.426+0000 7f6e437fe640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e441f3120 0x7f6e44156d90 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f6e34011910 tx=0x7f6e34011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.427+0000 7f6e40ff9640 1 -- 172.21.15.47:0/970740802 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34009590 con 0x7f6e441f3120 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.427+0000 7f6e40ff9640 1 -- 172.21.15.47:0/970740802 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3400faf0 con 0x7f6e441f3120 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.427+0000 7f6e4a39f640 1 -- 172.21.15.47:0/970740802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4415def0 con 0x7f6e441f3120 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.427+0000 7f6e40ff9640 1 -- 172.21.15.47:0/970740802 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3400fe60 con 0x7f6e441f3120 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.427+0000 7f6e4a39f640 1 -- 172.21.15.47:0/970740802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4415e410 con 0x7f6e441f3120 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.428+0000 7f6e4a39f640 1 -- 172.21.15.47:0/970740802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e44072360 con 0x7f6e441f3120 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.428+0000 7f6e40ff9640 1 -- 172.21.15.47:0/970740802 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6e34014440 con 0x7f6e441f3120 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.428+0000 7f6e40ff9640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e2406ac00 0x7f6e2406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.429+0000 7f6e40ff9640 1 -- 172.21.15.47:0/970740802 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f6e34089c70 con 0x7f6e441f3120 2024-03-23T09:52:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.429+0000 7f6e43fff640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e2406ac00 0x7f6e2406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.429+0000 7f6e43fff640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e2406ac00 0x7f6e2406d0c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c000fb0 tx=0x7f6e2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.433+0000 7f6e40ff9640 1 -- 172.21.15.47:0/970740802 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6e34054df0 con 0x7f6e441f3120 2024-03-23T09:52:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.650+0000 7f6e4a39f640 1 -- 172.21.15.47:0/970740802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6e4415e0b0 con 0x7f6e441f3120 2024-03-23T09:52:41.651 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-03-23T09:52:41.651+0000 7fefe2f3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T09:52:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.658+0000 7f6e40ff9640 1 -- 172.21.15.47:0/970740802 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v94) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6e34058410 con 0x7f6e441f3120 2024-03-23T09:52:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T09:52:41.660 INFO:tasks.ceph.mds.c.smithi047.stderr: -1364> 2024-03-23T09:52:31.499+0000 7fcc7e3c6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T09:52:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.661+0000 7f6e4a39f640 1 -- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e2406ac00 msgr2=0x7f6e2406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.661+0000 7f6e4a39f640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e2406ac00 0x7f6e2406d0c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c000fb0 tx=0x7f6e2c009040 comp rx=0 tx=0).stop 2024-03-23T09:52:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.661+0000 7f6e4a39f640 1 -- 172.21.15.47:0/970740802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e441f3120 msgr2=0x7f6e44156d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.661+0000 7f6e4a39f640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e441f3120 0x7f6e44156d90 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f6e34011910 tx=0x7f6e34011dd0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.661+0000 7f6e4a39f640 1 -- 172.21.15.47:0/970740802 shutdown_connections 2024-03-23T09:52:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.661+0000 7f6e4a39f640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f6e2406ac00 0x7f6e2406d0c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.661+0000 7f6e4a39f640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e4415f730 0x7f6e441572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.661+0000 7f6e4a39f640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e441f3120 0x7f6e44156d90 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.661+0000 7f6e4a39f640 1 --2- 172.21.15.47:0/970740802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e4415f350 0x7f6e440cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.661+0000 7f6e4a39f640 1 -- 172.21.15.47:0/970740802 >> 172.21.15.47:0/970740802 conn(0x7f6e440cf430 msgr2=0x7f6e4415bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.661+0000 7f6e4a39f640 1 -- 172.21.15.47:0/970740802 shutdown_connections 2024-03-23T09:52:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.661+0000 7f6e4a39f640 1 -- 172.21.15.47:0/970740802 wait complete. 2024-03-23T09:52:41.672 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T09:52:41.685 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-03-23T09:52:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.776+0000 7f4d36189640 1 Processor -- start 2024-03-23T09:52:41.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.776+0000 7f4d36189640 1 -- start start 2024-03-23T09:52:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.777+0000 7f4d36189640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4d3015ce10 0x7f4d300cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.777+0000 7f4d36189640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d300b85d0 0x7f4d300cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.777+0000 7f4d36189640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d30161600 con 0x7f4d3015ca30 2024-03-23T09:52:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.777+0000 7f4d36189640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d30160e50 con 0x7f4d300b85d0 2024-03-23T09:52:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.777+0000 7f4d36189640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d300d25e0 con 0x7f4d3015ce10 2024-03-23T09:52:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.777+0000 7f4d2ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d300b85d0 0x7f4d300cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.777+0000 7f4d2f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4d3015ca30 0x7f4d30160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33694/0 (socket says 172.21.15.47:33694) 2024-03-23T09:52:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.777+0000 7f4d2f7fe640 1 -- 172.21.15.47:0/354604771 learned_addr learned my addr 172.21.15.47:0/354604771 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.777+0000 7f4d2ffff640 1 --2- 172.21.15.47:0/354604771 >> v2:172.21.15.47:3300/0 conn(0x7f4d300b85d0 0x7f4d300cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48466/0 (socket says 172.21.15.47:48466) 2024-03-23T09:52:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.777+0000 7f4d27fff640 1 --2- 172.21.15.47:0/354604771 >> v2:172.21.15.175:3300/0 conn(0x7f4d3015ce10 0x7f4d300cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.778+0000 7f4d2ffff640 1 -- 172.21.15.47:0/354604771 >> v2:172.21.15.175:3300/0 conn(0x7f4d3015ce10 msgr2=0x7f4d300cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.778+0000 7f4d2ffff640 1 --2- 172.21.15.47:0/354604771 >> v2:172.21.15.175:3300/0 conn(0x7f4d3015ce10 0x7f4d300cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.778+0000 7f4d2ffff640 1 -- 172.21.15.47:0/354604771 >> v1:172.21.15.47:6789/0 conn(0x7f4d3015ca30 legacy=0x7f4d30160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.778+0000 7f4d2ffff640 1 -- 172.21.15.47:0/354604771 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d30161040 con 0x7f4d300b85d0 2024-03-23T09:52:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.778+0000 7f4d27fff640 1 --2- 172.21.15.47:0/354604771 >> v2:172.21.15.175:3300/0 conn(0x7f4d3015ce10 0x7f4d300cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.778+0000 7f4d2ffff640 1 --2- 172.21.15.47:0/354604771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d300b85d0 0x7f4d300cdc70 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f4d30066930 tx=0x7f4d2802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e3a2e21a271475 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.778+0000 7f4d2effd640 1 -- 172.21.15.47:0/354604771 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2802cc80 con 0x7f4d300b85d0 2024-03-23T09:52:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.778+0000 7f4d2effd640 1 -- 172.21.15.47:0/354604771 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2802cde0 con 0x7f4d300b85d0 2024-03-23T09:52:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.778+0000 7f4d2effd640 1 -- 172.21.15.47:0/354604771 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28034660 con 0x7f4d300b85d0 2024-03-23T09:52:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.779+0000 7f4d36189640 1 -- 172.21.15.47:0/354604771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d300b85d0 msgr2=0x7f4d300cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.779+0000 7f4d36189640 1 --2- 172.21.15.47:0/354604771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d300b85d0 0x7f4d300cdc70 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f4d30066930 tx=0x7f4d2802bed0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.779+0000 7f4d36189640 1 -- 172.21.15.47:0/354604771 shutdown_connections 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.779+0000 7f4d36189640 1 --2- 172.21.15.47:0/354604771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d300b85d0 0x7f4d300cdc70 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.779+0000 7f4d36189640 1 --2- 172.21.15.47:0/354604771 >> v2:172.21.15.175:3300/0 conn(0x7f4d3015ce10 0x7f4d300cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.779+0000 7f4d36189640 1 -- 172.21.15.47:0/354604771 >> 172.21.15.47:0/354604771 conn(0x7f4d3006b160 msgr2=0x7f4d3006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.779+0000 7f4d36189640 1 -- 172.21.15.47:0/354604771 shutdown_connections 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.779+0000 7f4d36189640 1 -- 172.21.15.47:0/354604771 wait complete. 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.780+0000 7f4d36189640 1 Processor -- start 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.780+0000 7f4d36189640 1 -- start start 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d36189640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d300b85d0 0x7f4d300cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d36189640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d3015ca30 0x7f4d3014ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d36189640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d3015ce10 0x7f4d30150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d36189640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d30152350 con 0x7f4d300b85d0 2024-03-23T09:52:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d36189640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d300c70e0 con 0x7f4d3015ce10 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d36189640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d300c6b80 con 0x7f4d3015ca30 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d2f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d300b85d0 0x7f4d300cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d2f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d300b85d0 0x7f4d300cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48472/0 (socket says 172.21.15.47:48472) 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d2f7fe640 1 -- 172.21.15.47:0/902977773 learned_addr learned my addr 172.21.15.47:0/902977773 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d2f7fe640 1 -- 172.21.15.47:0/902977773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d3015ca30 msgr2=0x7f4d3014ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d2f7fe640 1 --2- 172.21.15.47:0/902977773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d3015ca30 0x7f4d3014ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d2f7fe640 1 -- 172.21.15.47:0/902977773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d3015ce10 msgr2=0x7f4d30150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d2f7fe640 1 --2- 172.21.15.47:0/902977773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d3015ce10 0x7f4d30150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d2f7fe640 1 -- 172.21.15.47:0/902977773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d28009a70 con 0x7f4d300b85d0 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.781+0000 7f4d2f7fe640 1 --2- 172.21.15.47:0/902977773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d300b85d0 0x7f4d300cb4a0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000c500 tx=0x7f4d2000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.782+0000 7f4d2cff9640 1 -- 172.21.15.47:0/902977773 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20005ce0 con 0x7f4d300b85d0 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.782+0000 7f4d2cff9640 1 -- 172.21.15.47:0/902977773 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d20005e40 con 0x7f4d300b85d0 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.782+0000 7f4d36189640 1 -- 172.21.15.47:0/902977773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d30151ab0 con 0x7f4d300b85d0 2024-03-23T09:52:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.782+0000 7f4d2cff9640 1 -- 172.21.15.47:0/902977773 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d200072f0 con 0x7f4d300b85d0 2024-03-23T09:52:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.782+0000 7f4d36189640 1 -- 172.21.15.47:0/902977773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d30152070 con 0x7f4d300b85d0 2024-03-23T09:52:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.783+0000 7f4d36189640 1 -- 172.21.15.47:0/902977773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d30072360 con 0x7f4d300b85d0 2024-03-23T09:52:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.783+0000 7f4d2cff9640 1 -- 172.21.15.47:0/902977773 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4d20007450 con 0x7f4d300b85d0 2024-03-23T09:52:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.783+0000 7f4d2cff9640 1 --2- 172.21.15.47:0/902977773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4d1006ac00 0x7f4d1006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.784+0000 7f4d2cff9640 1 -- 172.21.15.47:0/902977773 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f4d20088910 con 0x7f4d300b85d0 2024-03-23T09:52:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.784+0000 7f4d27fff640 1 --2- 172.21.15.47:0/902977773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4d1006ac00 0x7f4d1006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.784+0000 7f4d27fff640 1 --2- 172.21.15.47:0/902977773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4d1006ac00 0x7f4d1006d0c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4d18006fd0 tx=0x7f4d18008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:41.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:41.788+0000 7f4d2cff9640 1 -- 172.21.15.47:0/902977773 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4d20053af0 con 0x7f4d300b85d0 2024-03-23T09:52:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.005+0000 7f4d36189640 1 -- 172.21.15.47:0/902977773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4d30066be0 con 0x7f4d300b85d0 2024-03-23T09:52:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.661+0000 7f4d2cff9640 1 -- 172.21.15.47:0/902977773 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v95) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4d200170a0 con 0x7f4d300b85d0 2024-03-23T09:52:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.663+0000 7f4d36189640 1 -- 172.21.15.47:0/902977773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4d1006ac00 msgr2=0x7f4d1006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.663+0000 7f4d36189640 1 --2- 172.21.15.47:0/902977773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4d1006ac00 0x7f4d1006d0c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4d18006fd0 tx=0x7f4d18008040 comp rx=0 tx=0).stop 2024-03-23T09:52:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.663+0000 7f4d36189640 1 -- 172.21.15.47:0/902977773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d300b85d0 msgr2=0x7f4d300cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.663+0000 7f4d36189640 1 --2- 172.21.15.47:0/902977773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d300b85d0 0x7f4d300cb4a0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000c500 tx=0x7f4d2000c9c0 comp rx=0 tx=0).stop 2024-03-23T09:52:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.664+0000 7f4d36189640 1 -- 172.21.15.47:0/902977773 shutdown_connections 2024-03-23T09:52:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.664+0000 7f4d36189640 1 --2- 172.21.15.47:0/902977773 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f4d1006ac00 0x7f4d1006d0c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.664+0000 7f4d36189640 1 --2- 172.21.15.47:0/902977773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d3015ce10 0x7f4d30150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.664+0000 7f4d36189640 1 --2- 172.21.15.47:0/902977773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d3015ca30 0x7f4d3014ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:42.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.664+0000 7f4d36189640 1 --2- 172.21.15.47:0/902977773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d300b85d0 0x7f4d300cb4a0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:42.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.664+0000 7f4d36189640 1 -- 172.21.15.47:0/902977773 >> 172.21.15.47:0/902977773 conn(0x7f4d3006b160 msgr2=0x7f4d30155290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:42.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.664+0000 7f4d36189640 1 -- 172.21.15.47:0/902977773 shutdown_connections 2024-03-23T09:52:42.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.664+0000 7f4d36189640 1 -- 172.21.15.47:0/902977773 wait complete. 2024-03-23T09:52:42.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T09:52:42.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.831+0000 7fe8386c4640 1 Processor -- start 2024-03-23T09:52:42.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.831+0000 7fe8386c4640 1 -- start start 2024-03-23T09:52:42.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.832+0000 7fe8386c4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8300cde40 0x7fe8300b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:42.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.832+0000 7fe8386c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe830160780 0x7fe8300d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:42.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.832+0000 7fe8386c4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8300b5a90 con 0x7fe83015cb50 2024-03-23T09:52:42.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.832+0000 7fe8386c4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe830161790 con 0x7fe8300cde40 2024-03-23T09:52:42.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.832+0000 7fe8386c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe830160ef0 con 0x7fe830160780 2024-03-23T09:52:42.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.832+0000 7fe836460640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe83015cb50 0x7fe8300cd730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33700/0 (socket says 172.21.15.47:33700) 2024-03-23T09:52:42.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.832+0000 7fe836460640 1 -- 172.21.15.47:0/172448018 learned_addr learned my addr 172.21.15.47:0/172448018 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:42.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.832+0000 7fe836c61640 1 --2- 172.21.15.47:0/172448018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe830160780 0x7fe8300d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:42.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.832+0000 7fe835c5f640 1 --2- 172.21.15.47:0/172448018 >> v2:172.21.15.175:3300/0 conn(0x7fe8300cde40 0x7fe8300b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:42.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.832+0000 7fe83545e640 1 -- 172.21.15.47:0/172448018 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396210575 0 0) 0x7fe8300b5a90 con 0x7fe83015cb50 2024-03-23T09:52:42.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.832+0000 7fe83545e640 1 -- 172.21.15.47:0/172448018 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe818003520 con 0x7fe83015cb50 2024-03-23T09:52:42.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.833+0000 7fe83545e640 1 -- 172.21.15.47:0/172448018 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2607145694 0 0) 0x7fe818003520 con 0x7fe83015cb50 2024-03-23T09:52:42.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.833+0000 7fe836c61640 1 -- 172.21.15.47:0/172448018 >> v2:172.21.15.175:3300/0 conn(0x7fe8300cde40 msgr2=0x7fe8300b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:42.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.833+0000 7fe836c61640 1 --2- 172.21.15.47:0/172448018 >> v2:172.21.15.175:3300/0 conn(0x7fe8300cde40 0x7fe8300b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:42.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.833+0000 7fe836c61640 1 -- 172.21.15.47:0/172448018 >> v1:172.21.15.47:6789/0 conn(0x7fe83015cb50 legacy=0x7fe8300cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:42.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.833+0000 7fe836c61640 1 -- 172.21.15.47:0/172448018 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8301611d0 con 0x7fe830160780 2024-03-23T09:52:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.833+0000 7fe836c61640 1 --2- 172.21.15.47:0/172448018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe830160780 0x7fe8300d1490 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe8280097a0 tx=0x7fe82802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcba1d180ab323d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.834+0000 7fe83545e640 1 -- 172.21.15.47:0/172448018 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820001a10 con 0x7fe830160780 2024-03-23T09:52:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.834+0000 7fe83545e640 1 -- 172.21.15.47:0/172448018 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82802e050 con 0x7fe830160780 2024-03-23T09:52:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.834+0000 7fe83545e640 1 -- 172.21.15.47:0/172448018 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828030710 con 0x7fe830160780 2024-03-23T09:52:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.834+0000 7fe8386c4640 1 -- 172.21.15.47:0/172448018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe830160780 msgr2=0x7fe8300d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.834+0000 7fe8386c4640 1 --2- 172.21.15.47:0/172448018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe830160780 0x7fe8300d1490 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe8280097a0 tx=0x7fe82802b790 comp rx=0 tx=0).stop 2024-03-23T09:52:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.834+0000 7fe8386c4640 1 -- 172.21.15.47:0/172448018 shutdown_connections 2024-03-23T09:52:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.834+0000 7fe8386c4640 1 --2- 172.21.15.47:0/172448018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe830160780 0x7fe8300d1490 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.834+0000 7fe8386c4640 1 --2- 172.21.15.47:0/172448018 >> v2:172.21.15.175:3300/0 conn(0x7fe8300cde40 0x7fe8300b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.834+0000 7fe8386c4640 1 -- 172.21.15.47:0/172448018 >> 172.21.15.47:0/172448018 conn(0x7fe83006b160 msgr2=0x7fe83006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.834+0000 7fe8386c4640 1 -- 172.21.15.47:0/172448018 shutdown_connections 2024-03-23T09:52:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.834+0000 7fe8386c4640 1 -- 172.21.15.47:0/172448018 wait complete. 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.835+0000 7fe8386c4640 1 Processor -- start 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.835+0000 7fe8386c4640 1 -- start start 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.836+0000 7fe8386c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8300cde40 0x7fe83014fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.836+0000 7fe8386c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe83015cb50 0x7fe83015fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.836+0000 7fe8386c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe830160780 0x7fe830156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.836+0000 7fe8386c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83015e9b0 con 0x7fe8300cde40 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.836+0000 7fe8386c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83015ec90 con 0x7fe83015cb50 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.836+0000 7fe8386c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83015f0b0 con 0x7fe830160780 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.836+0000 7fe836460640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8300cde40 0x7fe83014fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.836+0000 7fe836460640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8300cde40 0x7fe83014fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48476/0 (socket says 172.21.15.47:48476) 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.836+0000 7fe836460640 1 -- 172.21.15.47:0/3632628036 learned_addr learned my addr 172.21.15.47:0/3632628036 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.837+0000 7fe836460640 1 -- 172.21.15.47:0/3632628036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe830160780 msgr2=0x7fe830156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.837+0000 7fe836460640 1 --2- 172.21.15.47:0/3632628036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe830160780 0x7fe830156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.837+0000 7fe836460640 1 -- 172.21.15.47:0/3632628036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe83015cb50 msgr2=0x7fe83015fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.837+0000 7fe836460640 1 --2- 172.21.15.47:0/3632628036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe83015cb50 0x7fe83015fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.837+0000 7fe836460640 1 -- 172.21.15.47:0/3632628036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe828009400 con 0x7fe8300cde40 2024-03-23T09:52:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.837+0000 7fe836460640 1 --2- 172.21.15.47:0/3632628036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8300cde40 0x7fe83014fe60 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fe82000caf0 tx=0x7fe820002060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.837+0000 7fe82effd640 1 -- 172.21.15.47:0/3632628036 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82000a700 con 0x7fe8300cde40 2024-03-23T09:52:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.837+0000 7fe8386c4640 1 -- 172.21.15.47:0/3632628036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83015f390 con 0x7fe8300cde40 2024-03-23T09:52:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.837+0000 7fe82effd640 1 -- 172.21.15.47:0/3632628036 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82000a860 con 0x7fe8300cde40 2024-03-23T09:52:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.837+0000 7fe82effd640 1 -- 172.21.15.47:0/3632628036 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82000ac10 con 0x7fe8300cde40 2024-03-23T09:52:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.837+0000 7fe8386c4640 1 -- 172.21.15.47:0/3632628036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8301600d0 con 0x7fe8300cde40 2024-03-23T09:52:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.838+0000 7fe8386c4640 1 -- 172.21.15.47:0/3632628036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe830072360 con 0x7fe8300cde40 2024-03-23T09:52:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.838+0000 7fe82effd640 1 -- 172.21.15.47:0/3632628036 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe8200070d0 con 0x7fe8300cde40 2024-03-23T09:52:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.839+0000 7fe82effd640 1 --2- 172.21.15.47:0/3632628036 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe81c06ac00 0x7fe81c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.839+0000 7fe82effd640 1 -- 172.21.15.47:0/3632628036 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6025+0+0 (secure 0 0 0) 0x7fe820088c40 con 0x7fe8300cde40 2024-03-23T09:52:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.839+0000 7fe835c5f640 1 --2- 172.21.15.47:0/3632628036 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe81c06ac00 0x7fe81c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.839+0000 7fe835c5f640 1 --2- 172.21.15.47:0/3632628036 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe81c06ac00 0x7fe81c06d0c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe824006a90 tx=0x7fe824006a20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:42.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:42.843+0000 7fe82effd640 1 -- 172.21.15.47:0/3632628036 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe820053c60 con 0x7fe8300cde40 2024-03-23T09:52:43.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.021+0000 7fe8386c4640 1 -- 172.21.15.47:0/3632628036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe83015ef00 con 0x7fe8300cde40 2024-03-23T09:52:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.939+0000 7fe82effd640 1 -- 172.21.15.47:0/3632628036 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe8200180d0 con 0x7fe8300cde40 2024-03-23T09:52:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-03-23T09:52:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.943+0000 7fe8386c4640 1 -- 172.21.15.47:0/3632628036 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe81c06ac00 msgr2=0x7fe81c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.943+0000 7fe8386c4640 1 --2- 172.21.15.47:0/3632628036 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe81c06ac00 0x7fe81c06d0c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe824006a90 tx=0x7fe824006a20 comp rx=0 tx=0).stop 2024-03-23T09:52:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.943+0000 7fe8386c4640 1 -- 172.21.15.47:0/3632628036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8300cde40 msgr2=0x7fe83014fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.944+0000 7fe8386c4640 1 --2- 172.21.15.47:0/3632628036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8300cde40 0x7fe83014fe60 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fe82000caf0 tx=0x7fe820002060 comp rx=0 tx=0).stop 2024-03-23T09:52:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.944+0000 7fe8386c4640 1 -- 172.21.15.47:0/3632628036 shutdown_connections 2024-03-23T09:52:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.944+0000 7fe8386c4640 1 --2- 172.21.15.47:0/3632628036 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe81c06ac00 0x7fe81c06d0c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:43.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.944+0000 7fe8386c4640 1 --2- 172.21.15.47:0/3632628036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe830160780 0x7fe830156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:43.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.944+0000 7fe8386c4640 1 --2- 172.21.15.47:0/3632628036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe83015cb50 0x7fe83015fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:43.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.944+0000 7fe8386c4640 1 --2- 172.21.15.47:0/3632628036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8300cde40 0x7fe83014fe60 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:43.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.944+0000 7fe8386c4640 1 -- 172.21.15.47:0/3632628036 >> 172.21.15.47:0/3632628036 conn(0x7fe83006b160 msgr2=0x7fe8300b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:43.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.944+0000 7fe8386c4640 1 -- 172.21.15.47:0/3632628036 shutdown_connections 2024-03-23T09:52:43.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:43.944+0000 7fe8386c4640 1 -- 172.21.15.47:0/3632628036 wait complete. 2024-03-23T09:52:43.958 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T09:52:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.071+0000 7f75fd4af640 1 Processor -- start 2024-03-23T09:52:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.071+0000 7f75fd4af640 1 -- start start 2024-03-23T09:52:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.072+0000 7f75fd4af640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75f80b1bf0 0x7f75f80af950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.072+0000 7f75fd4af640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f75f80b0990 0x7f75f806a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.072+0000 7f75fd4af640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f80af4c0 con 0x7f75f80cb460 2024-03-23T09:52:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.072+0000 7f75fd4af640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f80aef40 con 0x7f75f80b0990 2024-03-23T09:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.072+0000 7f75fd4af640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80afe90 con 0x7f75f80b1bf0 2024-03-23T09:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.072+0000 7f75f6ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f75f80cb460 0x7f75f80b0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33716/0 (socket says 172.21.15.47:33716) 2024-03-23T09:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.072+0000 7f75f6ffd640 1 -- 172.21.15.47:0/1788464186 learned_addr learned my addr 172.21.15.47:0/1788464186 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.072+0000 7f75f67fc640 1 --2- 172.21.15.47:0/1788464186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75f80b1bf0 0x7f75f80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.072+0000 7f75f77fe640 1 --2- 172.21.15.47:0/1788464186 >> v2:172.21.15.175:3300/0 conn(0x7f75f80b0990 0x7f75f806a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651200375 0 0) 0x7f75f80af4c0 con 0x7f75f80cb460 2024-03-23T09:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e4003520 con 0x7f75f80cb460 2024-03-23T09:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 11220947 0 0) 0x7f75e4003520 con 0x7f75f80cb460 2024-03-23T09:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75f80b1bf0 msgr2=0x7f75f80af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 --2- 172.21.15.47:0/1788464186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75f80b1bf0 0x7f75f80af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 >> v2:172.21.15.175:3300/0 conn(0x7f75f80b0990 msgr2=0x7f75f806a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 --2- 172.21.15.47:0/1788464186 >> v2:172.21.15.175:3300/0 conn(0x7f75f80b0990 0x7f75f806a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f806ac30 con 0x7f75f80cb460 2024-03-23T09:52:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f75e80019f0 con 0x7f75f80cb460 2024-03-23T09:52:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f77fe640 1 --2- 172.21.15.47:0/1788464186 >> v2:172.21.15.175:3300/0 conn(0x7f75f80b0990 0x7f75f806a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:52:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 >> v1:172.21.15.47:6789/0 conn(0x7f75f80cb460 legacy=0x7f75f80b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 --2- 172.21.15.47:0/1788464186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e4003970 0x7f75e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 --2- 172.21.15.47:0/1788464186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75f80b1bf0 0x7f75e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 --2- 172.21.15.47:0/1788464186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e4004d00 0x7f75e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80aef40 con 0x7f75e4003970 2024-03-23T09:52:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80afe90 con 0x7f75e4004d00 2024-03-23T09:52:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e400b980 con 0x7f75f80b1bf0 2024-03-23T09:52:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f6ffd640 1 --2- 172.21.15.47:0/1788464186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e4003970 0x7f75e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f67fc640 1 --2- 172.21.15.47:0/1788464186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75f80b1bf0 0x7f75e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f77fe640 1 --2- 172.21.15.47:0/1788464186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e4004d00 0x7f75e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f6ffd640 1 -- 172.21.15.47:0/1788464186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75f80b1bf0 msgr2=0x7f75e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f6ffd640 1 --2- 172.21.15.47:0/1788464186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75f80b1bf0 0x7f75e40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f6ffd640 1 -- 172.21.15.47:0/1788464186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e4004d00 msgr2=0x7f75e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f6ffd640 1 --2- 172.21.15.47:0/1788464186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e4004d00 0x7f75e4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f6ffd640 1 -- 172.21.15.47:0/1788464186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e4003520 con 0x7f75e4003970 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.073+0000 7f75f77fe640 1 --2- 172.21.15.47:0/1788464186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e4004d00 0x7f75e4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.074+0000 7f75f6ffd640 1 --2- 172.21.15.47:0/1788464186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e4003970 0x7f75e4003d90 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f75e800a860 tx=0x7f75e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd62d8e8d3c2a4e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.074+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8034630 con 0x7f75e4003970 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.074+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e802f070 con 0x7f75e4003970 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.074+0000 7f75f5ffb640 1 -- 172.21.15.47:0/1788464186 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8034a10 con 0x7f75e4003970 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.075+0000 7f75fd4af640 1 -- 172.21.15.47:0/1788464186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e4003970 msgr2=0x7f75e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.075+0000 7f75fd4af640 1 --2- 172.21.15.47:0/1788464186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e4003970 0x7f75e4003d90 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f75e800a860 tx=0x7f75e40057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.075+0000 7f75f77fe640 1 -- 172.21.15.47:0/1788464186 reap_dead start 2024-03-23T09:52:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.075+0000 7f75fd4af640 1 -- 172.21.15.47:0/1788464186 shutdown_connections 2024-03-23T09:52:44.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.075+0000 7f75fd4af640 1 -- 172.21.15.47:0/1788464186 >> 172.21.15.47:0/1788464186 conn(0x7f75f8098a00 msgr2=0x7f75f80b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:44.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.075+0000 7f75fd4af640 1 -- 172.21.15.47:0/1788464186 shutdown_connections 2024-03-23T09:52:44.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.075+0000 7f75fd4af640 1 -- 172.21.15.47:0/1788464186 wait complete. 2024-03-23T09:52:44.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.076+0000 7f75fd4af640 1 Processor -- start 2024-03-23T09:52:44.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.076+0000 7f75fd4af640 1 -- start start 2024-03-23T09:52:44.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75fd4af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e4003970 0x7f75f815d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:44.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75fd4af640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75f80b3230 0x7f75f81f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:44.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75fd4af640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75f80b36b0 0x7f75f81f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:44.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75fd4af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80b3c80 con 0x7f75e4003970 2024-03-23T09:52:44.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75fd4af640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80b3f60 con 0x7f75f80b3230 2024-03-23T09:52:44.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75fd4af640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80b4240 con 0x7f75f80b36b0 2024-03-23T09:52:44.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75f6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e4003970 0x7f75f815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:44.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75f6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e4003970 0x7f75f815d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48494/0 (socket says 172.21.15.47:48494) 2024-03-23T09:52:44.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75f6ffd640 1 -- 172.21.15.47:0/2116553854 learned_addr learned my addr 172.21.15.47:0/2116553854 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:44.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75f67fc640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75f80b3230 0x7f75f81f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:44.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75f77fe640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75f80b36b0 0x7f75f81f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75f6ffd640 1 -- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75f80b36b0 msgr2=0x7f75f81f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75f6ffd640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75f80b36b0 0x7f75f81f1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75f6ffd640 1 -- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75f80b3230 msgr2=0x7f75f81f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75f6ffd640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75f80b3230 0x7f75f81f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75f6ffd640 1 -- 172.21.15.47:0/2116553854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f80b4520 con 0x7f75e4003970 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.077+0000 7f75f67fc640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75f80b3230 0x7f75f81f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.078+0000 7f75f6ffd640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e4003970 0x7f75f815d680 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f75e800a860 tx=0x7f75e8001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.078+0000 7f75ef7fe640 1 -- 172.21.15.47:0/2116553854 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8004e30 con 0x7f75e4003970 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.078+0000 7f75ef7fe640 1 -- 172.21.15.47:0/2116553854 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e80354c0 con 0x7f75e4003970 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.078+0000 7f75fd4af640 1 -- 172.21.15.47:0/2116553854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f80b4800 con 0x7f75e4003970 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.078+0000 7f75ef7fe640 1 -- 172.21.15.47:0/2116553854 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8035850 con 0x7f75e4003970 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.078+0000 7f75fd4af640 1 -- 172.21.15.47:0/2116553854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f8149900 con 0x7f75e4003970 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.079+0000 7f75fd4af640 1 -- 172.21.15.47:0/2116553854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f80b4990 con 0x7f75e4003970 2024-03-23T09:52:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.079+0000 7f75ef7fe640 1 -- 172.21.15.47:0/2116553854 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f75e8031030 con 0x7f75e4003970 2024-03-23T09:52:44.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.079+0000 7f75ef7fe640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75d806ac30 0x7f75d806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:44.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.080+0000 7f75ef7fe640 1 -- 172.21.15.47:0/2116553854 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5587+0+0 (secure 0 0 0) 0x7f75e80ab630 con 0x7f75e4003970 2024-03-23T09:52:44.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.080+0000 7f75f67fc640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75d806ac30 0x7f75d806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:44.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.080+0000 7f75f67fc640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75d806ac30 0x7f75d806d0f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f75f815a120 tx=0x7f75e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:44.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.084+0000 7f75ef7fe640 1 -- 172.21.15.47:0/2116553854 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f75e8076880 con 0x7f75e4003970 2024-03-23T09:52:44.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.266+0000 7f75fd4af640 1 -- 172.21.15.47:0/2116553854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f75f80b4990 con 0x7f75e4003970 2024-03-23T09:52:44.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.944+0000 7f75ef7fe640 1 -- 172.21.15.47:0/2116553854 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) v1 ==== 169+0+0 (secure 0 0 0) 0x7f75e8079ea0 con 0x7f75e4003970 2024-03-23T09:52:44.945 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-03-23T09:52:44.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.948+0000 7f75fd4af640 1 -- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75d806ac30 msgr2=0x7f75d806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:44.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.948+0000 7f75fd4af640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75d806ac30 0x7f75d806d0f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f75f815a120 tx=0x7f75e0008040 comp rx=0 tx=0).stop 2024-03-23T09:52:44.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.948+0000 7f75fd4af640 1 -- 172.21.15.47:0/2116553854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e4003970 msgr2=0x7f75f815d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:44.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.948+0000 7f75fd4af640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e4003970 0x7f75f815d680 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f75e800a860 tx=0x7f75e8001d80 comp rx=0 tx=0).stop 2024-03-23T09:52:44.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.948+0000 7f75fd4af640 1 -- 172.21.15.47:0/2116553854 shutdown_connections 2024-03-23T09:52:44.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.948+0000 7f75fd4af640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f75d806ac30 0x7f75d806d0f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:44.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.948+0000 7f75fd4af640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75f80b36b0 0x7f75f81f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:44.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.948+0000 7f75fd4af640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75f80b3230 0x7f75f81f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:44.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.948+0000 7f75fd4af640 1 --2- 172.21.15.47:0/2116553854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75e4003970 0x7f75f815d680 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:44.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.948+0000 7f75fd4af640 1 -- 172.21.15.47:0/2116553854 >> 172.21.15.47:0/2116553854 conn(0x7f75f8098a00 msgr2=0x7f75f80ac5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:44.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.948+0000 7f75fd4af640 1 -- 172.21.15.47:0/2116553854 shutdown_connections 2024-03-23T09:52:44.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:44.948+0000 7f75fd4af640 1 -- 172.21.15.47:0/2116553854 wait complete. 2024-03-23T09:52:44.961 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-23T09:52:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.067+0000 7fd4add00640 1 Processor -- start 2024-03-23T09:52:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.067+0000 7fd4add00640 1 -- start start 2024-03-23T09:52:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.067+0000 7fd4add00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a806e080 0x7fd4a8159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.067+0000 7fd4add00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4a8070900 0x7fd4a80cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.067+0000 7fd4add00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a8159ac0 con 0x7fd4a815cb30 2024-03-23T09:52:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.067+0000 7fd4add00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a815db20 con 0x7fd4a8070900 2024-03-23T09:52:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.067+0000 7fd4add00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a81594e0 con 0x7fd4a806e080 2024-03-23T09:52:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.068+0000 7fd4a7fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4a8070900 0x7fd4a80cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.068+0000 7fd4a6ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd4a815cb30 0x7fd4a80701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37984/0 (socket says 172.21.15.47:37984) 2024-03-23T09:52:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.068+0000 7fd4a6ffd640 1 -- 172.21.15.47:0/3807609928 learned_addr learned my addr 172.21.15.47:0/3807609928 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.068+0000 7fd4a77fe640 1 --2- 172.21.15.47:0/3807609928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a806e080 0x7fd4a8159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.068+0000 7fd4a7fff640 1 -- 172.21.15.47:0/3807609928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a806e080 msgr2=0x7fd4a8159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.068+0000 7fd4a7fff640 1 --2- 172.21.15.47:0/3807609928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a806e080 0x7fd4a8159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.068+0000 7fd4a7fff640 1 -- 172.21.15.47:0/3807609928 >> v1:172.21.15.47:6789/0 conn(0x7fd4a815cb30 legacy=0x7fd4a80701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.068+0000 7fd4a67fc640 1 -- 172.21.15.47:0/3807609928 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460670568 0 0) 0x7fd4a8159ac0 con 0x7fd4a815cb30 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.068+0000 7fd4a7fff640 1 -- 172.21.15.47:0/3807609928 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a815a490 con 0x7fd4a8070900 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.068+0000 7fd4a77fe640 1 --2- 172.21.15.47:0/3807609928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a806e080 0x7fd4a8159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.069+0000 7fd4a7fff640 1 --2- 172.21.15.47:0/3807609928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a8070900 0x7fd4a80cc4b0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009d90 tx=0x7fd4a002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1dd4b002764eea server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.069+0000 7fd4a67fc640 1 -- 172.21.15.47:0/3807609928 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a002cc50 con 0x7fd4a8070900 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.069+0000 7fd4a67fc640 1 -- 172.21.15.47:0/3807609928 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a002cdb0 con 0x7fd4a8070900 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.069+0000 7fd4a67fc640 1 -- 172.21.15.47:0/3807609928 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0034660 con 0x7fd4a8070900 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.069+0000 7fd4add00640 1 -- 172.21.15.47:0/3807609928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a8070900 msgr2=0x7fd4a80cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.069+0000 7fd4add00640 1 --2- 172.21.15.47:0/3807609928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a8070900 0x7fd4a80cc4b0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009d90 tx=0x7fd4a002bea0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.069+0000 7fd4add00640 1 -- 172.21.15.47:0/3807609928 shutdown_connections 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.069+0000 7fd4add00640 1 --2- 172.21.15.47:0/3807609928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a8070900 0x7fd4a80cc4b0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.069+0000 7fd4add00640 1 --2- 172.21.15.47:0/3807609928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a806e080 0x7fd4a8159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.069+0000 7fd4add00640 1 -- 172.21.15.47:0/3807609928 >> 172.21.15.47:0/3807609928 conn(0x7fd4a806a310 msgr2=0x7fd4a80b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.069+0000 7fd4add00640 1 -- 172.21.15.47:0/3807609928 shutdown_connections 2024-03-23T09:52:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.069+0000 7fd4add00640 1 -- 172.21.15.47:0/3807609928 wait complete. 2024-03-23T09:52:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.071+0000 7fd4add00640 1 Processor -- start 2024-03-23T09:52:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.071+0000 7fd4add00640 1 -- start start 2024-03-23T09:52:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.071+0000 7fd4add00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a806e080 0x7fd4a80ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.071+0000 7fd4add00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a8070900 0x7fd4a80ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.071+0000 7fd4add00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a815cb30 0x7fd4a81f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.071+0000 7fd4add00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a81546b0 con 0x7fd4a8070900 2024-03-23T09:52:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.071+0000 7fd4add00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a8154950 con 0x7fd4a815cb30 2024-03-23T09:52:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.071+0000 7fd4add00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a8154c30 con 0x7fd4a806e080 2024-03-23T09:52:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd4a6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a8070900 0x7fd4a80ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd4a6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a8070900 0x7fd4a80ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40028/0 (socket says 172.21.15.47:40028) 2024-03-23T09:52:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd4a6ffd640 1 -- 172.21.15.47:0/2623303016 learned_addr learned my addr 172.21.15.47:0/2623303016 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd4a77fe640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a806e080 0x7fd4a80ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd4a7fff640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a815cb30 0x7fd4a81f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd4a6ffd640 1 -- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a806e080 msgr2=0x7fd4a80ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd4a6ffd640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a806e080 0x7fd4a80ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd4a6ffd640 1 -- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a815cb30 msgr2=0x7fd4a81f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd4a6ffd640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a815cb30 0x7fd4a81f15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd4a6ffd640 1 -- 172.21.15.47:0/2623303016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a0009a70 con 0x7fd4a8070900 2024-03-23T09:52:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd4a6ffd640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a8070900 0x7fd4a80ad720 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00c430 tx=0x7fd49c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd48ffff640 1 -- 172.21.15.47:0/2623303016 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0074e0 con 0x7fd4a8070900 2024-03-23T09:52:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd48ffff640 1 -- 172.21.15.47:0/2623303016 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c007660 con 0x7fd4a8070900 2024-03-23T09:52:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd4add00640 1 -- 172.21.15.47:0/2623303016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a8154f10 con 0x7fd4a8070900 2024-03-23T09:52:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd48ffff640 1 -- 172.21.15.47:0/2623303016 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c004a20 con 0x7fd4a8070900 2024-03-23T09:52:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.072+0000 7fd4add00640 1 -- 172.21.15.47:0/2623303016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a80a78f0 con 0x7fd4a8070900 2024-03-23T09:52:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.073+0000 7fd4add00640 1 -- 172.21.15.47:0/2623303016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a8073480 con 0x7fd4a8070900 2024-03-23T09:52:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.074+0000 7fd48ffff640 1 -- 172.21.15.47:0/2623303016 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd49c004b80 con 0x7fd4a8070900 2024-03-23T09:52:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.074+0000 7fd48ffff640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd49406ac00 0x7fd49406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.074+0000 7fd48ffff640 1 -- 172.21.15.47:0/2623303016 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fd49c088fe0 con 0x7fd4a8070900 2024-03-23T09:52:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.074+0000 7fd4a77fe640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd49406ac00 0x7fd49406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.075+0000 7fd4a77fe640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd49406ac00 0x7fd49406d0c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd49800a990 tx=0x7fd498008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:45.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.078+0000 7fd48ffff640 1 -- 172.21.15.47:0/2623303016 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd49c0543d0 con 0x7fd4a8070900 2024-03-23T09:52:45.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.266+0000 7fd4add00640 1 -- 172.21.15.47:0/2623303016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd4a8155100 con 0x7fd4a8070900 2024-03-23T09:52:45.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.269+0000 7fd48ffff640 1 -- 172.21.15.47:0/2623303016 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd49c0579f0 con 0x7fd4a8070900 2024-03-23T09:52:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.271+0000 7fd4add00640 1 -- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd49406ac00 msgr2=0x7fd49406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.271+0000 7fd4add00640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd49406ac00 0x7fd49406d0c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd49800a990 tx=0x7fd498008040 comp rx=0 tx=0).stop 2024-03-23T09:52:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.271+0000 7fd4add00640 1 -- 172.21.15.47:0/2623303016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a8070900 msgr2=0x7fd4a80ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.271+0000 7fd4add00640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a8070900 0x7fd4a80ad720 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00c430 tx=0x7fd49c00c8f0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.271+0000 7fd4add00640 1 -- 172.21.15.47:0/2623303016 shutdown_connections 2024-03-23T09:52:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.271+0000 7fd4add00640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd49406ac00 0x7fd49406d0c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.271+0000 7fd4add00640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a815cb30 0x7fd4a81f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.271+0000 7fd4add00640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a8070900 0x7fd4a80ad720 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.271+0000 7fd4add00640 1 --2- 172.21.15.47:0/2623303016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a806e080 0x7fd4a80ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.271+0000 7fd4add00640 1 -- 172.21.15.47:0/2623303016 >> 172.21.15.47:0/2623303016 conn(0x7fd4a806a310 msgr2=0x7fd4a815ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.271+0000 7fd4add00640 1 -- 172.21.15.47:0/2623303016 shutdown_connections 2024-03-23T09:52:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.271+0000 7fd4add00640 1 -- 172.21.15.47:0/2623303016 wait complete. 2024-03-23T09:52:45.282 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-03-23T09:52:45.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.383+0000 7f01cc015640 1 Processor -- start 2024-03-23T09:52:45.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.383+0000 7f01cc015640 1 -- start start 2024-03-23T09:52:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01cc015640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c415ca90 0x7f01c4160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01cc015640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f01c40cde10 0x7f01c40d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01cc015640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c40d11e0 con 0x7f01c40b8350 2024-03-23T09:52:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01cc015640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c4160f60 con 0x7f01c40cde10 2024-03-23T09:52:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01cc015640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4161240 con 0x7f01c415ca90 2024-03-23T09:52:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c95b0640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f01c40b8350 0x7f01c40cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37992/0 (socket says 172.21.15.47:37992) 2024-03-23T09:52:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c95b0640 1 -- 172.21.15.47:0/3086899999 learned_addr learned my addr 172.21.15.47:0/3086899999 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147757405 0 0) 0x7f01c40d11e0 con 0x7f01c40b8350 2024-03-23T09:52:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01b0003520 con 0x7f01c40b8350 2024-03-23T09:52:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01ca5b2640 1 --2- 172.21.15.47:0/3086899999 >> v2:172.21.15.175:3300/0 conn(0x7f01c40cde10 0x7f01c40d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c9db1640 1 --2- 172.21.15.47:0/3086899999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c415ca90 0x7f01c4160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1079673598 0 0) 0x7f01b0003520 con 0x7f01c40b8350 2024-03-23T09:52:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c415ca90 msgr2=0x7f01c4160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 --2- 172.21.15.47:0/3086899999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c415ca90 0x7f01c4160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 >> v2:172.21.15.175:3300/0 conn(0x7f01c40cde10 msgr2=0x7f01c40d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 --2- 172.21.15.47:0/3086899999 >> v2:172.21.15.175:3300/0 conn(0x7f01c40cde10 0x7f01c40d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c4161520 con 0x7f01c40b8350 2024-03-23T09:52:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f01b4001670 con 0x7f01c40b8350 2024-03-23T09:52:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c9db1640 1 --2- 172.21.15.47:0/3086899999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c415ca90 0x7f01c4160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 >> v1:172.21.15.47:6789/0 conn(0x7f01c40b8350 legacy=0x7f01c40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 --2- 172.21.15.47:0/3086899999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b0003970 0x7f01b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 --2- 172.21.15.47:0/3086899999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b00047c0 0x7f01b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 --2- 172.21.15.47:0/3086899999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c415ca90 0x7f01b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4160f60 con 0x7f01b00047c0 2024-03-23T09:52:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4161240 con 0x7f01b0003970 2024-03-23T09:52:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b000bab0 con 0x7f01c415ca90 2024-03-23T09:52:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.384+0000 7f01ca5b2640 1 --2- 172.21.15.47:0/3086899999 >> v2:172.21.15.175:3300/0 conn(0x7f01c40cde10 0x7f01c40d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.385+0000 7f01c95b0640 1 --2- 172.21.15.47:0/3086899999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b00047c0 0x7f01b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.385+0000 7f01c9db1640 1 --2- 172.21.15.47:0/3086899999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b0003970 0x7f01b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.385+0000 7f01ca5b2640 1 --2- 172.21.15.47:0/3086899999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c415ca90 0x7f01b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.385+0000 7f01c95b0640 1 -- 172.21.15.47:0/3086899999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c415ca90 msgr2=0x7f01b0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.385+0000 7f01c95b0640 1 --2- 172.21.15.47:0/3086899999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c415ca90 0x7f01b0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.385+0000 7f01c95b0640 1 -- 172.21.15.47:0/3086899999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b0003970 msgr2=0x7f01b0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.385+0000 7f01c95b0640 1 --2- 172.21.15.47:0/3086899999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b0003970 0x7f01b0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.385+0000 7f01c95b0640 1 -- 172.21.15.47:0/3086899999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b0003520 con 0x7f01b00047c0 2024-03-23T09:52:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.386+0000 7f01c95b0640 1 --2- 172.21.15.47:0/3086899999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b00047c0 0x7f01b0004c40 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c210 tx=0x7f01b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1047daa57c5465 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.386+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4005ad0 con 0x7f01b00047c0 2024-03-23T09:52:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.386+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b4005c30 con 0x7f01b00047c0 2024-03-23T09:52:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.386+0000 7f01c8daf640 1 -- 172.21.15.47:0/3086899999 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b402c8a0 con 0x7f01b00047c0 2024-03-23T09:52:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.386+0000 7f01cc015640 1 -- 172.21.15.47:0/3086899999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b00047c0 msgr2=0x7f01b0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.386+0000 7f01cc015640 1 --2- 172.21.15.47:0/3086899999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b00047c0 0x7f01b0004c40 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c210 tx=0x7f01b00057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.386+0000 7f01ca5b2640 1 -- 172.21.15.47:0/3086899999 reap_dead start 2024-03-23T09:52:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.386+0000 7f01cc015640 1 -- 172.21.15.47:0/3086899999 shutdown_connections 2024-03-23T09:52:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.386+0000 7f01cc015640 1 -- 172.21.15.47:0/3086899999 >> 172.21.15.47:0/3086899999 conn(0x7f01c40659d0 msgr2=0x7f01c4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.386+0000 7f01cc015640 1 -- 172.21.15.47:0/3086899999 shutdown_connections 2024-03-23T09:52:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.386+0000 7f01cc015640 1 -- 172.21.15.47:0/3086899999 wait complete. 2024-03-23T09:52:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.387+0000 7f01cc015640 1 Processor -- start 2024-03-23T09:52:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.387+0000 7f01cc015640 1 -- start start 2024-03-23T09:52:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.388+0000 7f01cc015640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b00047c0 0x7f01c40c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.388+0000 7f01cc015640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c4150db0 0x7f01c41f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.388+0000 7f01cc015640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c41f5e50 0x7f01c41f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.388+0000 7f01cc015640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40c9a90 con 0x7f01b00047c0 2024-03-23T09:52:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.388+0000 7f01cc015640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40c8ee0 con 0x7f01c4150db0 2024-03-23T09:52:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.388+0000 7f01cc015640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c41533e0 con 0x7f01c41f5e50 2024-03-23T09:52:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.388+0000 7f01c9db1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b00047c0 0x7f01c40c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.388+0000 7f01c9db1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b00047c0 0x7f01c40c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40064/0 (socket says 172.21.15.47:40064) 2024-03-23T09:52:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.388+0000 7f01c9db1640 1 -- 172.21.15.47:0/3902046730 learned_addr learned my addr 172.21.15.47:0/3902046730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.388+0000 7f01c95b0640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c4150db0 0x7f01c41f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.388+0000 7f01ca5b2640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c41f5e50 0x7f01c41f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.389+0000 7f01c9db1640 1 -- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c41f5e50 msgr2=0x7f01c41f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.389+0000 7f01c9db1640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c41f5e50 0x7f01c41f62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.389+0000 7f01c9db1640 1 -- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c4150db0 msgr2=0x7f01c41f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.389+0000 7f01c9db1640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c4150db0 0x7f01c41f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.389+0000 7f01c9db1640 1 -- 172.21.15.47:0/3902046730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c41536c0 con 0x7f01b00047c0 2024-03-23T09:52:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.389+0000 7f01c95b0640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c4150db0 0x7f01c41f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.389+0000 7f01ca5b2640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c41f5e50 0x7f01c41f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.389+0000 7f01c9db1640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b00047c0 0x7f01c40c89a0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f01c00115f0 tx=0x7f01c0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.389+0000 7f01ba7fc640 1 -- 172.21.15.47:0/3902046730 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c000c070 con 0x7f01b00047c0 2024-03-23T09:52:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.389+0000 7f01ba7fc640 1 -- 172.21.15.47:0/3902046730 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c0004220 con 0x7f01b00047c0 2024-03-23T09:52:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.389+0000 7f01ba7fc640 1 -- 172.21.15.47:0/3902046730 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c000f6f0 con 0x7f01b00047c0 2024-03-23T09:52:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.389+0000 7f01cc015640 1 -- 172.21.15.47:0/3902046730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c41539a0 con 0x7f01b00047c0 2024-03-23T09:52:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.389+0000 7f01cc015640 1 -- 172.21.15.47:0/3902046730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c4151410 con 0x7f01b00047c0 2024-03-23T09:52:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.390+0000 7f01cc015640 1 -- 172.21.15.47:0/3902046730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c4072360 con 0x7f01b00047c0 2024-03-23T09:52:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.390+0000 7f01ba7fc640 1 -- 172.21.15.47:0/3902046730 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f01c000f850 con 0x7f01b00047c0 2024-03-23T09:52:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.391+0000 7f01ba7fc640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f01a806ac00 0x7f01a806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.391+0000 7f01ba7fc640 1 -- 172.21.15.47:0/3902046730 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f01c00890a0 con 0x7f01b00047c0 2024-03-23T09:52:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.391+0000 7f01c95b0640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f01a806ac00 0x7f01a806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.392+0000 7f01c95b0640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f01a806ac00 0x7f01a806d0c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f01b4034000 tx=0x7f01b4007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.395+0000 7f01ba7fc640 1 -- 172.21.15.47:0/3902046730 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f01c00544d0 con 0x7f01b00047c0 2024-03-23T09:52:45.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.642+0000 7f01cc015640 1 -- 172.21.15.47:0/3902046730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f01c4153b30 con 0x7f01b00047c0 2024-03-23T09:52:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.945+0000 7f01ba7fc640 1 -- 172.21.15.47:0/3902046730 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v525) v1 ==== 144+0+0 (secure 0 0 0) 0x7f01c0057af0 con 0x7f01b00047c0 2024-03-23T09:52:45.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.947+0000 7f01cc015640 1 -- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f01a806ac00 msgr2=0x7f01a806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:45.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.947+0000 7f01cc015640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f01a806ac00 0x7f01a806d0c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f01b4034000 tx=0x7f01b4007000 comp rx=0 tx=0).stop 2024-03-23T09:52:45.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.947+0000 7f01cc015640 1 -- 172.21.15.47:0/3902046730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b00047c0 msgr2=0x7f01c40c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:45.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.947+0000 7f01cc015640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b00047c0 0x7f01c40c89a0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f01c00115f0 tx=0x7f01c0011ab0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.947+0000 7f01cc015640 1 -- 172.21.15.47:0/3902046730 shutdown_connections 2024-03-23T09:52:45.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.947+0000 7f01cc015640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f01a806ac00 0x7f01a806d0c0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.947+0000 7f01cc015640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c41f5e50 0x7f01c41f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.947+0000 7f01cc015640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c4150db0 0x7f01c41f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.947+0000 7f01cc015640 1 --2- 172.21.15.47:0/3902046730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b00047c0 0x7f01c40c89a0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:45.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.947+0000 7f01cc015640 1 -- 172.21.15.47:0/3902046730 >> 172.21.15.47:0/3902046730 conn(0x7f01c40659d0 msgr2=0x7f01c40b76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:45.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.947+0000 7f01cc015640 1 -- 172.21.15.47:0/3902046730 shutdown_connections 2024-03-23T09:52:45.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:45.947+0000 7f01cc015640 1 -- 172.21.15.47:0/3902046730 wait complete. 2024-03-23T09:52:45.959 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-23T09:52:45.959 INFO:tasks.cephfs_test_runner: 2024-03-23T09:52:45.959 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-23T09:52:45.959 INFO:tasks.cephfs_test_runner:Ran 7 tests in 673.686s 2024-03-23T09:52:45.960 INFO:tasks.cephfs_test_runner: 2024-03-23T09:52:45.960 INFO:tasks.cephfs_test_runner:OK 2024-03-23T09:52:45.960 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-23T09:52:45.969 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-23T09:52:45.986 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-23T09:52:45.986 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:52:45.986 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:52:46.002 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:52:46.002 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-23T09:52:46.002 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:52:46.058 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=245 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T09:52:46.058 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:52:46.058 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:52:46.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:52:46.077 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-23T09:52:46.077 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T09:52:46.136 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2382 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T09:52:46.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:46.137 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-23T09:52:46.152 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T09:52:46.152 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T09:52:46.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:46.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T09:52:46.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:46.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T09:52:46.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:46.289 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-03-23T09:52:46.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:46.389 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-03-23T09:52:46.407 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-23T09:52:46.407 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-23T09:52:46.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:46.407 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-23T09:52:46.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:46.479 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-23T09:52:46.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T09:52:46.551 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-03-23T09:52:46.637 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-23T09:52:46.646 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-23T09:52:46.646 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:52:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.751+0000 7f4177c2c640 1 Processor -- start 2024-03-23T09:52:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.751+0000 7f4177c2c640 1 -- start start 2024-03-23T09:52:46.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f4177c2c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41700b3c30 0x7f417006d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:46.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f4177c2c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f417006c350 0x7f41700b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f4177c2c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41700b4550 con 0x7f4170142aa0 2024-03-23T09:52:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f4177c2c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417006c8c0 con 0x7f41700b3c30 2024-03-23T09:52:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f4177c2c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417006cba0 con 0x7f417006c350 2024-03-23T09:52:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f41759c8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41700b3c30 0x7f417006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f41761c9640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4170142aa0 0x7f41700cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38004/0 (socket says 172.21.15.47:38004) 2024-03-23T09:52:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f41759c8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41700b3c30 0x7f417006d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40066/0 (socket says 172.21.15.47:40066) 2024-03-23T09:52:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f41761c9640 1 -- 172.21.15.47:0/3053650831 learned_addr learned my addr 172.21.15.47:0/3053650831 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f41751c7640 1 --2- 172.21.15.47:0/3053650831 >> v2:172.21.15.175:3300/0 conn(0x7f417006c350 0x7f41700b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f41759c8640 1 -- 172.21.15.47:0/3053650831 >> v2:172.21.15.175:3300/0 conn(0x7f417006c350 msgr2=0x7f41700b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f41759c8640 1 --2- 172.21.15.47:0/3053650831 >> v2:172.21.15.175:3300/0 conn(0x7f417006c350 0x7f41700b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f41759c8640 1 -- 172.21.15.47:0/3053650831 >> v1:172.21.15.47:6789/0 conn(0x7f4170142aa0 legacy=0x7f41700cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f41759c8640 1 -- 172.21.15.47:0/3053650831 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417006ce80 con 0x7f41700b3c30 2024-03-23T09:52:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.752+0000 7f41751c7640 1 --2- 172.21.15.47:0/3053650831 >> v2:172.21.15.175:3300/0 conn(0x7f417006c350 0x7f41700b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.753+0000 7f41759c8640 1 --2- 172.21.15.47:0/3053650831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41700b3c30 0x7f417006d960 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f417006ab80 tx=0x7f416c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db1ffd5cb5f2808 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.753+0000 7f41749c6640 1 -- 172.21.15.47:0/3053650831 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c031830 con 0x7f41700b3c30 2024-03-23T09:52:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.753+0000 7f41749c6640 1 -- 172.21.15.47:0/3053650831 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c031990 con 0x7f41700b3c30 2024-03-23T09:52:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.753+0000 7f41749c6640 1 -- 172.21.15.47:0/3053650831 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c031d20 con 0x7f41700b3c30 2024-03-23T09:52:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.754+0000 7f4177c2c640 1 -- 172.21.15.47:0/3053650831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41700b3c30 msgr2=0x7f417006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.754+0000 7f4177c2c640 1 --2- 172.21.15.47:0/3053650831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41700b3c30 0x7f417006d960 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f417006ab80 tx=0x7f416c02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:52:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.754+0000 7f4177c2c640 1 -- 172.21.15.47:0/3053650831 shutdown_connections 2024-03-23T09:52:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.754+0000 7f4177c2c640 1 --2- 172.21.15.47:0/3053650831 >> v2:172.21.15.175:3300/0 conn(0x7f417006c350 0x7f41700b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.754+0000 7f4177c2c640 1 --2- 172.21.15.47:0/3053650831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41700b3c30 0x7f417006d960 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.754+0000 7f4177c2c640 1 -- 172.21.15.47:0/3053650831 >> 172.21.15.47:0/3053650831 conn(0x7f417006a310 msgr2=0x7f41700b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:46.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.754+0000 7f4177c2c640 1 -- 172.21.15.47:0/3053650831 shutdown_connections 2024-03-23T09:52:46.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.754+0000 7f4177c2c640 1 -- 172.21.15.47:0/3053650831 wait complete. 2024-03-23T09:52:46.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.755+0000 7f4177c2c640 1 Processor -- start 2024-03-23T09:52:46.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.755+0000 7f4177c2c640 1 -- start start 2024-03-23T09:52:46.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.755+0000 7f4177c2c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f417006c350 0x7f417013e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:46.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.755+0000 7f4177c2c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41700b3c30 0x7f41700a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:46.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.755+0000 7f4177c2c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4170142aa0 0x7f4170158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:46.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f4177c2c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41700a7b30 con 0x7f417006c350 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f4177c2c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417013f050 con 0x7f4170142aa0 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f4177c2c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417013f330 con 0x7f41700b3c30 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41759c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f417006c350 0x7f417013e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41759c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f417006c350 0x7f417013e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40082/0 (socket says 172.21.15.47:40082) 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41759c8640 1 -- 172.21.15.47:0/3784016508 learned_addr learned my addr 172.21.15.47:0/3784016508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41761c9640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4170142aa0 0x7f4170158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41751c7640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41700b3c30 0x7f41700a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41759c8640 1 -- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41700b3c30 msgr2=0x7f41700a75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41759c8640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41700b3c30 0x7f41700a75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41759c8640 1 -- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4170142aa0 msgr2=0x7f4170158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41759c8640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4170142aa0 0x7f4170158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41759c8640 1 -- 172.21.15.47:0/3784016508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c009400 con 0x7f417006c350 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41759c8640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f417006c350 0x7f417013e9b0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f416c000c00 tx=0x7f416c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:46.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41667fc640 1 -- 172.21.15.47:0/3784016508 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c03aa60 con 0x7f417006c350 2024-03-23T09:52:46.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41667fc640 1 -- 172.21.15.47:0/3784016508 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c03abe0 con 0x7f417006c350 2024-03-23T09:52:46.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f4177c2c640 1 -- 172.21.15.47:0/3784016508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4170139eb0 con 0x7f417006c350 2024-03-23T09:52:46.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f41667fc640 1 -- 172.21.15.47:0/3784016508 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c0443f0 con 0x7f417006c350 2024-03-23T09:52:46.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.756+0000 7f4177c2c640 1 -- 172.21.15.47:0/3784016508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417013f610 con 0x7f417006c350 2024-03-23T09:52:46.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.757+0000 7f4177c2c640 1 -- 172.21.15.47:0/3784016508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4170073480 con 0x7f417006c350 2024-03-23T09:52:46.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.758+0000 7f41667fc640 1 -- 172.21.15.47:0/3784016508 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f416c04d450 con 0x7f417006c350 2024-03-23T09:52:46.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.758+0000 7f41667fc640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f415406ac00 0x7f415406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:46.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.758+0000 7f41667fc640 1 -- 172.21.15.47:0/3784016508 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f416c0ab350 con 0x7f417006c350 2024-03-23T09:52:46.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.758+0000 7f41751c7640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f415406ac00 0x7f415406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:46.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.759+0000 7f41751c7640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f415406ac00 0x7f415406d0c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f417013bdc0 tx=0x7f4160008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:46.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.762+0000 7f41667fc640 1 -- 172.21.15.47:0/3784016508 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f416c0768a0 con 0x7f417006c350 2024-03-23T09:52:46.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.939+0000 7f4177c2c640 1 -- 172.21.15.47:0/3784016508 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41701431a0 con 0x7f415406ac00 2024-03-23T09:52:46.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.942+0000 7f41667fc640 1 -- 172.21.15.47:0/3784016508 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50857 (secure 0 0 0) 0x7f416000c040 con 0x7f415406ac00 2024-03-23T09:52:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:46.943 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:52:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.943+0000 7f4177c2c640 1 -- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f415406ac00 msgr2=0x7f415406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.943+0000 7f4177c2c640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f415406ac00 0x7f415406d0c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f417013bdc0 tx=0x7f4160008040 comp rx=0 tx=0).stop 2024-03-23T09:52:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.943+0000 7f4177c2c640 1 -- 172.21.15.47:0/3784016508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f417006c350 msgr2=0x7f417013e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.943+0000 7f4177c2c640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f417006c350 0x7f417013e9b0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f416c000c00 tx=0x7f416c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:52:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.943+0000 7f4177c2c640 1 -- 172.21.15.47:0/3784016508 shutdown_connections 2024-03-23T09:52:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.943+0000 7f4177c2c640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f415406ac00 0x7f415406d0c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.943+0000 7f4177c2c640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4170142aa0 0x7f4170158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.943+0000 7f4177c2c640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41700b3c30 0x7f41700a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.943+0000 7f4177c2c640 1 --2- 172.21.15.47:0/3784016508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f417006c350 0x7f417013e9b0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.943+0000 7f4177c2c640 1 -- 172.21.15.47:0/3784016508 >> 172.21.15.47:0/3784016508 conn(0x7f417006a310 msgr2=0x7f41700af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.944+0000 7f4177c2c640 1 -- 172.21.15.47:0/3784016508 shutdown_connections 2024-03-23T09:52:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:46.944+0000 7f4177c2c640 1 -- 172.21.15.47:0/3784016508 wait complete. 2024-03-23T09:52:46.955 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-03-23T09:52:46.515972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":294428,"kb_used_data":29128,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754680292,"statfs":{"total":773094113280,"available":772792619008,"internally_reserved":0,"allocated":29827072,"data_stored":23087610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":271568440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390585,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-172,"num_write_kb":-8291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001344"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":266,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-23T09:52:25.491130+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:52:25.491130+0000","last_peered":"2024-03-23T09:52:25.491130+0000","last_clean":"2024-03-23T09:52:25.491130+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:52:25.491130+0000","last_undegraded":"2024-03-23T09:52:25.491130+0000","last_fullsized":"2024-03-23T09:52:25.491130+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574992,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33132,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338708,"statfs":{"total":96636764160,"available":96602836992,"internally_reserved":0,"allocated":2068480,"data_stored":1228661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55200000000000005,"15min":0.52700000000000002},"min":{"1min":0.41399999999999998,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.623,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.309},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.44700000000000001,"5min":0.32200000000000001,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.78900000000000003,"15min":0.85999999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.40799999999999997,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.63,"5min":0.79100000000000004,"15min":0.80400000000000005},"last":0.501},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.63600000000000001,"5min":0.747,"15min":0.81299999999999994},"last":0.433}]},{"osd":2,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55200000000000005},"min":{"1min":0.44500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.78500000000000003,"15min":0.82499999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.622,"5min":0.83099999999999996,"15min":0.83599999999999997},"last":0.52600000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.46000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.61699999999999999,"5min":0.73999999999999999,"15min":0.86199999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.627,"5min":0.77400000000000002,"15min":0.83699999999999997},"last":0.42099999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46700000000000003,"15min":0.48999999999999999},"min":{"1min":0.379,"5min":0.29799999999999999,"15min":0.29599999999999999},"max":{"1min":0.61699999999999999,"5min":0.67500000000000004,"15min":0.78200000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46700000000000003,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.29299999999999998},"max":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":0.80100000000000005},"last":0.45100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.46500000000000002,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":0.69799999999999995,"15min":0.81499999999999995},"last":0.443},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.498,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.63300000000000001,"5min":0.73199999999999998,"15min":0.84999999999999998},"last":0.51400000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.55100000000000005},"min":{"1min":0.34699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66300000000000003,"5min":0.75,"15min":0.874},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51200000000000001,"15min":0.55600000000000005},"min":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69199999999999995,"5min":0.73999999999999999,"15min":0.90200000000000002},"last":0.48699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574992,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":5544,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":5677056,"data_stored":4837750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.53300000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.27200000000000002},"max":{"1min":0.746,"5min":0.77700000000000002,"15min":0.81000000000000005},"last":0.505},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57199999999999995,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.76200000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.311},"max":{"1min":0.77600000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.41299999999999998,"15min":0.32700000000000001},"max":{"1min":0.751,"5min":0.80300000000000005,"15min":0.82999999999999996},"last":0.39700000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.29699999999999999},"max":{"1min":0.69499999999999995,"5min":0.747,"15min":0.81599999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59099999999999997,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58899999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.42399999999999999,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73299999999999998,"5min":0.79000000000000004,"15min":0.81699999999999995},"last":0.64100000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.48199999999999998},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65600000000000003,"5min":0.69299999999999995,"15min":0.72499999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.47599999999999998,"15min":0.51700000000000002},"min":{"1min":0.35899999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.63800000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.71299999999999997}]},{"osd":5,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48099999999999998,"15min":0.505},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.316},"max":{"1min":0.64100000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51200000000000001,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.84399999999999997}]},{"osd":7,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52200000000000002,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.72899999999999998,"5min":0.754,"15min":0.82699999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54100000000000004,"15min":0.56499999999999995},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69299999999999995,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.72899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1495040,"data_stored":644604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54300000000000004,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.68700000000000006,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.39400000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73699999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42199999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69799999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51500000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.55100000000000005},"min":{"1min":0.38600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71999999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.54800000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.748,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.495}]},{"osd":3,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58499999999999996,"15min":0.54700000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70899999999999996,"5min":0.77400000000000002,"15min":0.81599999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.77400000000000002},"last":0.441}]},{"osd":4,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29999999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68500000000000005,"5min":0.73099999999999998,"15min":0.73699999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.497,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.93600000000000005},"last":0.33800000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.495,"15min":0.496},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63400000000000001,"5min":0.747,"15min":0.749},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65000000000000002,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.378}]},{"osd":7,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.54700000000000004},"min":{"1min":0.40999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.78200000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77500000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574993,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1466368,"data_stored":637677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.54600000000000004},"min":{"1min":0.34300000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.84599999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.78600000000000003},"last":0.48199999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.52300000000000002,"15min":0.50700000000000001},"min":{"1min":0.36799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63500000000000001,"5min":0.72899999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.28999999999999998},"max":{"1min":0.68700000000000006,"5min":0.751,"15min":0.77400000000000002},"last":0.38600000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58999999999999997,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.39400000000000002},"max":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.80200000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59699999999999998,"15min":0.57299999999999995},"min":{"1min":0.47699999999999998,"5min":0.39700000000000002,"15min":0.372},"max":{"1min":0.73099999999999998,"5min":0.75600000000000001,"15min":0.82199999999999995},"last":0.57199999999999995}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68300000000000005,"5min":0.751,"15min":1.0369999999999999},"last":0.442},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.58999999999999997,"5min":0.77900000000000003,"15min":0.80300000000000005},"last":0.46999999999999997}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29699999999999999,"15min":0.26500000000000001},"max":{"1min":0.77400000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46300000000000002,"15min":0.48799999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.253},"max":{"1min":0.73099999999999998,"5min":0.73199999999999998,"15min":1.008},"last":0.40400000000000003}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46700000000000003,"15min":0.47199999999999998},"min":{"1min":0.29999999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63200000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.82399999999999995},"last":0.5}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33500000000000002,"15min":0.30099999999999999},"max":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":1.0069999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70799999999999996,"5min":0.77600000000000002,"15min":0.93400000000000005},"last":0.55200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574991,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40756,"kb_used_data":5548,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331084,"statfs":{"total":96636764160,"available":96595030016,"internally_reserved":0,"allocated":5681152,"data_stored":4833359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33900000000000002,"15min":0.33300000000000002},"min":{"1min":0.214,"5min":0.192,"15min":0.183},"max":{"1min":0.52300000000000002,"5min":0.67800000000000005,"15min":1.45},"last":0.314},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35699999999999998,"15min":0.34100000000000003},"min":{"1min":0.23400000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.499,"5min":0.60499999999999998,"15min":1.4099999999999999},"last":0.20699999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.48299999999999998,"5min":0.56699999999999995,"15min":1.323},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.36699999999999999},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.94599999999999995},"last":0.22900000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.42799999999999999,"15min":0.40799999999999997},"min":{"1min":0.314,"5min":0.27100000000000002,"15min":0.251},"max":{"1min":0.55600000000000005,"5min":0.68999999999999995,"15min":1.1619999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.43099999999999999},"min":{"1min":0.313,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.59299999999999997,"5min":0.64900000000000002,"15min":1.2989999999999999},"last":0.371}]},{"osd":4,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.35399999999999998},"max":{"1min":0.85399999999999998,"5min":0.88600000000000001,"15min":1.337},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.41599999999999998,"5min":0.35699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":2.758},"last":0.50900000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.61099999999999999},"min":{"1min":0.49199999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.872,"5min":0.872,"15min":1.276},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.77100000000000002,"5min":0.91200000000000003,"15min":1.895},"last":0.56200000000000006}]},{"osd":6,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45100000000000001,"5min":0.41299999999999998,"15min":0.40699999999999997},"max":{"1min":0.78500000000000003,"5min":0.84199999999999997,"15min":5.4539999999999997},"last":0.504},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.432,"5min":0.41599999999999998,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":0.96499999999999997,"15min":2.8159999999999998},"last":0.52700000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.625,"15min":0.622},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.94899999999999995,"15min":3.7400000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.40300000000000002},"max":{"1min":0.91900000000000004,"5min":0.93600000000000005,"15min":5.4900000000000002},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574992,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":1482752,"data_stored":643993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34300000000000003,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.19500000000000001,"15min":0.191},"max":{"1min":0.51100000000000001,"5min":0.55500000000000005,"15min":0.60399999999999998},"last":0.182},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.22},"max":{"1min":0.50600000000000001,"5min":0.59699999999999998,"15min":0.872},"last":0.25700000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":0.55100000000000005,"5min":0.60899999999999999,"15min":0.72399999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39800000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.20899999999999999},"max":{"1min":0.51500000000000001,"5min":0.61199999999999999,"15min":0.746},"last":0.33600000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41699999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.27600000000000002,"15min":0.22900000000000001},"max":{"1min":0.55100000000000005,"5min":0.65200000000000002,"15min":0.67100000000000004},"last":0.248},{"interface":"front","average":{"1min":0.374,"5min":0.39200000000000002,"15min":0.39700000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":0.503,"5min":0.59199999999999997,"15min":0.72499999999999998},"last":0.27800000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81499999999999995,"5min":0.87,"15min":1.216},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81699999999999995,"5min":0.94699999999999995,"15min":0.99299999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.625,"15min":0.627},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.373},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.95699999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.628},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83099999999999996,"5min":0.98499999999999999,"15min":1.101},"last":0.496}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.97299999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":1.0780000000000001},"last":0.50900000000000001}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":1.155},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.41599999999999998},"max":{"1min":0.81599999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40760,"kb_used_data":5548,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331080,"statfs":{"total":96636764160,"available":96595025920,"internally_reserved":0,"allocated":5681152,"data_stored":4838416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.223,"5min":0.185,"15min":0.185},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38500000000000001,"15min":0.41999999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.20699999999999999},"max":{"1min":0.57799999999999996,"5min":0.59099999999999997,"15min":2.7069999999999999},"last":0.33400000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.35599999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.46400000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.435}]},{"osd":3,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.432,"15min":0.42199999999999999},"min":{"1min":0.29399999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55100000000000005,"5min":0.746,"15min":0.76900000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.378,"5min":0.40400000000000003,"15min":0.44700000000000001},"min":{"1min":0.26100000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.63,"5min":0.73999999999999999,"15min":2.75},"last":0.29299999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.61899999999999999,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.29099999999999998},"max":{"1min":0.83399999999999996,"5min":0.95899999999999996,"15min":0.97299999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.61699999999999999},"min":{"1min":0.42799999999999999,"5min":0.39400000000000002,"15min":0.38600000000000001},"max":{"1min":0.84599999999999997,"5min":0.873,"15min":1.002},"last":0.58499999999999996}]},{"osd":5,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63100000000000001,"15min":0.66000000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35299999999999998},"max":{"1min":1.026,"5min":1.026,"15min":2.778},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.88800000000000001,"15min":1.0649999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64200000000000002,"15min":0.66900000000000004},"min":{"1min":0.44600000000000001,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":2.8210000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.438,"15min":0.38800000000000001},"max":{"1min":0.80600000000000005,"5min":0.93300000000000005,"15min":1.1080000000000001},"last":0.69699999999999995}]},{"osd":7,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.95399999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.43099999999999999,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":2.7989999999999999},"last":0.68500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41336,"kb_used_data":6128,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330504,"statfs":{"total":96636764160,"available":96594436096,"internally_reserved":0,"allocated":6275072,"data_stored":5423150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.34499999999999997,"15min":0.33500000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.184},"max":{"1min":0.52300000000000002,"5min":0.625,"15min":0.625},"last":0.376},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.24099999999999999,"5min":0.22900000000000001,"15min":0.217},"max":{"1min":0.49099999999999999,"5min":0.61699999999999999,"15min":0.64900000000000002},"last":0.39100000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38400000000000001,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.58099999999999996,"5min":0.63,"15min":0.63},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.23799999999999999,"5min":0.22600000000000001,"15min":0.21199999999999999},"max":{"1min":0.497,"5min":0.58199999999999996,"15min":0.59399999999999997},"last":0.434}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39300000000000002,"15min":0.38900000000000001},"min":{"1min":0.30299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.44700000000000001,"15min":0.437},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55800000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.52500000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58699999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67000000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.67700000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.35299999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71799999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.42299999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.94299999999999995,"15min":0.95999999999999996},"last":0.626}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78500000000000003,"5min":0.91900000000000004,"15min":0.93600000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.46600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61899999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.40999999999999998,"15min":0.39300000000000002},"max":{"1min":0.753,"5min":0.85499999999999998,"15min":0.93600000000000005},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73999999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:52:46.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:52:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.057+0000 7f7b27441640 1 Processor -- start 2024-03-23T09:52:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.057+0000 7f7b27441640 1 -- start start 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b27441640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b200ae200 0x7f7b200cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b27441640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b2015e520 0x7f7b2015e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b27441640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2015eeb0 con 0x7f7b200b3b70 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b27441640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b200cb6b0 con 0x7f7b200ae200 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b27441640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2006a6b0 con 0x7f7b2015e520 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b251dd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b200ae200 0x7f7b200cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b251dd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b200ae200 0x7f7b200cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40094/0 (socket says 172.21.15.47:40094) 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b251dd640 1 -- 172.21.15.47:0/1334176835 learned_addr learned my addr 172.21.15.47:0/1334176835 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b249dc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7b200b3b70 0x7f7b2006cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38008/0 (socket says 172.21.15.47:38008) 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b259de640 1 --2- 172.21.15.47:0/1334176835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b2015e520 0x7f7b2015e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b251dd640 1 -- 172.21.15.47:0/1334176835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b2015e520 msgr2=0x7f7b2015e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b251dd640 1 --2- 172.21.15.47:0/1334176835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b2015e520 0x7f7b2015e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b251dd640 1 -- 172.21.15.47:0/1334176835 >> v1:172.21.15.47:6789/0 conn(0x7f7b200b3b70 legacy=0x7f7b2006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b17fff640 1 -- 172.21.15.47:0/1334176835 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919195883 0 0) 0x7f7b10001110 con 0x7f7b200b3b70 2024-03-23T09:52:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.058+0000 7f7b251dd640 1 -- 172.21.15.47:0/1334176835 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b20069d80 con 0x7f7b200ae200 2024-03-23T09:52:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.059+0000 7f7b251dd640 1 --2- 172.21.15.47:0/1334176835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b200ae200 0x7f7b200cb170 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009870 tx=0x7f7b1c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c8676ac6dcba85 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.059+0000 7f7b17fff640 1 -- 172.21.15.47:0/1334176835 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c031640 con 0x7f7b200ae200 2024-03-23T09:52:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.059+0000 7f7b17fff640 1 -- 172.21.15.47:0/1334176835 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c0317a0 con 0x7f7b200ae200 2024-03-23T09:52:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.059+0000 7f7b17fff640 1 -- 172.21.15.47:0/1334176835 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c031b30 con 0x7f7b200ae200 2024-03-23T09:52:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.060+0000 7f7b27441640 1 -- 172.21.15.47:0/1334176835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b200ae200 msgr2=0x7f7b200cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.060+0000 7f7b27441640 1 --2- 172.21.15.47:0/1334176835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b200ae200 0x7f7b200cb170 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009870 tx=0x7f7b1c02b8f0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.060+0000 7f7b27441640 1 -- 172.21.15.47:0/1334176835 shutdown_connections 2024-03-23T09:52:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.060+0000 7f7b27441640 1 --2- 172.21.15.47:0/1334176835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b2015e520 0x7f7b2015e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.060+0000 7f7b27441640 1 --2- 172.21.15.47:0/1334176835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b200ae200 0x7f7b200cb170 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.060+0000 7f7b27441640 1 -- 172.21.15.47:0/1334176835 >> 172.21.15.47:0/1334176835 conn(0x7f7b200ced80 msgr2=0x7f7b200cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.060+0000 7f7b27441640 1 -- 172.21.15.47:0/1334176835 shutdown_connections 2024-03-23T09:52:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.060+0000 7f7b27441640 1 -- 172.21.15.47:0/1334176835 wait complete. 2024-03-23T09:52:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.061+0000 7f7b27441640 1 Processor -- start 2024-03-23T09:52:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.061+0000 7f7b27441640 1 -- start start 2024-03-23T09:52:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b27441640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b200ae200 0x7f7b201463d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b27441640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b200b3b70 0x7f7b20148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b27441640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b2015e520 0x7f7b20149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b27441640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2014cc50 con 0x7f7b2015e520 2024-03-23T09:52:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b27441640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2014cf30 con 0x7f7b200b3b70 2024-03-23T09:52:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b27441640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2014d210 con 0x7f7b200ae200 2024-03-23T09:52:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b259de640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b2015e520 0x7f7b20149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b259de640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b2015e520 0x7f7b20149270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40102/0 (socket says 172.21.15.47:40102) 2024-03-23T09:52:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b259de640 1 -- 172.21.15.47:0/3119124439 learned_addr learned my addr 172.21.15.47:0/3119124439 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b251dd640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b200ae200 0x7f7b201463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b249dc640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b200b3b70 0x7f7b20148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b259de640 1 -- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b200ae200 msgr2=0x7f7b201463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b259de640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b200ae200 0x7f7b201463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b259de640 1 -- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b200b3b70 msgr2=0x7f7b20148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b259de640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b200b3b70 0x7f7b20148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b259de640 1 -- 172.21.15.47:0/3119124439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c0094d0 con 0x7f7b2015e520 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b251dd640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b200ae200 0x7f7b201463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b249dc640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b200b3b70 0x7f7b20148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b259de640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b2015e520 0x7f7b20149270 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800cd70 tx=0x7f7b1800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b15ffb640 1 -- 172.21.15.47:0/3119124439 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18003b90 con 0x7f7b2015e520 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b15ffb640 1 -- 172.21.15.47:0/3119124439 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b18003cf0 con 0x7f7b2015e520 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b15ffb640 1 -- 172.21.15.47:0/3119124439 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18004240 con 0x7f7b2015e520 2024-03-23T09:52:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.062+0000 7f7b27441640 1 -- 172.21.15.47:0/3119124439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2014d4f0 con 0x7f7b2015e520 2024-03-23T09:52:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.063+0000 7f7b27441640 1 -- 172.21.15.47:0/3119124439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2015cd20 con 0x7f7b2015e520 2024-03-23T09:52:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.064+0000 7f7b27441640 1 -- 172.21.15.47:0/3119124439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b20066be0 con 0x7f7b2015e520 2024-03-23T09:52:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.064+0000 7f7b15ffb640 1 -- 172.21.15.47:0/3119124439 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7b180043a0 con 0x7f7b2015e520 2024-03-23T09:52:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.064+0000 7f7b15ffb640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b0406ac00 0x7f7b0406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.064+0000 7f7b15ffb640 1 -- 172.21.15.47:0/3119124439 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f7b18012070 con 0x7f7b2015e520 2024-03-23T09:52:47.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.065+0000 7f7b251dd640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b0406ac00 0x7f7b0406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.065+0000 7f7b251dd640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b0406ac00 0x7f7b0406d0c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c036000 tx=0x7f7b1c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:47.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.069+0000 7f7b15ffb640 1 -- 172.21.15.47:0/3119124439 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7b18054f10 con 0x7f7b2015e520 2024-03-23T09:52:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.243+0000 7f7b27441640 1 -- 172.21.15.47:0/3119124439 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b2015f2f0 con 0x7f7b0406ac00 2024-03-23T09:52:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.245+0000 7f7b15ffb640 1 -- 172.21.15.47:0/3119124439 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50857 (secure 0 0 0) 0x7f7b1c031600 con 0x7f7b0406ac00 2024-03-23T09:52:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:52:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.247+0000 7f7b27441640 1 -- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b0406ac00 msgr2=0x7f7b0406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.247+0000 7f7b27441640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b0406ac00 0x7f7b0406d0c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c036000 tx=0x7f7b1c008030 comp rx=0 tx=0).stop 2024-03-23T09:52:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.247+0000 7f7b27441640 1 -- 172.21.15.47:0/3119124439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b2015e520 msgr2=0x7f7b20149270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.247+0000 7f7b27441640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b2015e520 0x7f7b20149270 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800cd70 tx=0x7f7b1800a660 comp rx=0 tx=0).stop 2024-03-23T09:52:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.247+0000 7f7b27441640 1 -- 172.21.15.47:0/3119124439 shutdown_connections 2024-03-23T09:52:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.247+0000 7f7b27441640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7b0406ac00 0x7f7b0406d0c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.247+0000 7f7b27441640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b2015e520 0x7f7b20149270 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.247+0000 7f7b27441640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b200b3b70 0x7f7b20148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.247+0000 7f7b27441640 1 --2- 172.21.15.47:0/3119124439 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b200ae200 0x7f7b201463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.247+0000 7f7b27441640 1 -- 172.21.15.47:0/3119124439 >> 172.21.15.47:0/3119124439 conn(0x7f7b200ced80 msgr2=0x7f7b20071840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.247+0000 7f7b27441640 1 -- 172.21.15.47:0/3119124439 shutdown_connections 2024-03-23T09:52:47.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.247+0000 7f7b27441640 1 -- 172.21.15.47:0/3119124439 wait complete. 2024-03-23T09:52:47.258 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-03-23T09:52:46.515972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":294428,"kb_used_data":29128,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754680292,"statfs":{"total":773094113280,"available":772792619008,"internally_reserved":0,"allocated":29827072,"data_stored":23087610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":271568440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390585,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-172,"num_write_kb":-8291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001344"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":266,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-23T09:52:25.491130+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:52:25.491130+0000","last_peered":"2024-03-23T09:52:25.491130+0000","last_clean":"2024-03-23T09:52:25.491130+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:52:25.491130+0000","last_undegraded":"2024-03-23T09:52:25.491130+0000","last_fullsized":"2024-03-23T09:52:25.491130+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574992,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33132,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338708,"statfs":{"total":96636764160,"available":96602836992,"internally_reserved":0,"allocated":2068480,"data_stored":1228661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55200000000000005,"15min":0.52700000000000002},"min":{"1min":0.41399999999999998,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.623,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.309},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.44700000000000001,"5min":0.32200000000000001,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.78900000000000003,"15min":0.85999999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.40799999999999997,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.63,"5min":0.79100000000000004,"15min":0.80400000000000005},"last":0.501},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.63600000000000001,"5min":0.747,"15min":0.81299999999999994},"last":0.433}]},{"osd":2,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55200000000000005},"min":{"1min":0.44500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.78500000000000003,"15min":0.82499999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.622,"5min":0.83099999999999996,"15min":0.83599999999999997},"last":0.52600000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.46000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.61699999999999999,"5min":0.73999999999999999,"15min":0.86199999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.627,"5min":0.77400000000000002,"15min":0.83699999999999997},"last":0.42099999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46700000000000003,"15min":0.48999999999999999},"min":{"1min":0.379,"5min":0.29799999999999999,"15min":0.29599999999999999},"max":{"1min":0.61699999999999999,"5min":0.67500000000000004,"15min":0.78200000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46700000000000003,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.29299999999999998},"max":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":0.80100000000000005},"last":0.45100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.46500000000000002,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":0.69799999999999995,"15min":0.81499999999999995},"last":0.443},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.498,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.63300000000000001,"5min":0.73199999999999998,"15min":0.84999999999999998},"last":0.51400000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.55100000000000005},"min":{"1min":0.34699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66300000000000003,"5min":0.75,"15min":0.874},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51200000000000001,"15min":0.55600000000000005},"min":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69199999999999995,"5min":0.73999999999999999,"15min":0.90200000000000002},"last":0.48699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574992,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":5544,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":5677056,"data_stored":4837750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.53300000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.27200000000000002},"max":{"1min":0.746,"5min":0.77700000000000002,"15min":0.81000000000000005},"last":0.505},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57199999999999995,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.76200000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.311},"max":{"1min":0.77600000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.41299999999999998,"15min":0.32700000000000001},"max":{"1min":0.751,"5min":0.80300000000000005,"15min":0.82999999999999996},"last":0.39700000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.29699999999999999},"max":{"1min":0.69499999999999995,"5min":0.747,"15min":0.81599999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59099999999999997,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58899999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.42399999999999999,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73299999999999998,"5min":0.79000000000000004,"15min":0.81699999999999995},"last":0.64100000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.48199999999999998},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65600000000000003,"5min":0.69299999999999995,"15min":0.72499999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.47599999999999998,"15min":0.51700000000000002},"min":{"1min":0.35899999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.63800000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.71299999999999997}]},{"osd":5,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48099999999999998,"15min":0.505},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.316},"max":{"1min":0.64100000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51200000000000001,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.84399999999999997}]},{"osd":7,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52200000000000002,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.72899999999999998,"5min":0.754,"15min":0.82699999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54100000000000004,"15min":0.56499999999999995},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69299999999999995,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.72899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1495040,"data_stored":644604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54300000000000004,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.68700000000000006,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.39400000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73699999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42199999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69799999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51500000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.55100000000000005},"min":{"1min":0.38600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71999999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.54800000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.748,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.495}]},{"osd":3,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58499999999999996,"15min":0.54700000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70899999999999996,"5min":0.77400000000000002,"15min":0.81599999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.77400000000000002},"last":0.441}]},{"osd":4,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29999999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68500000000000005,"5min":0.73099999999999998,"15min":0.73699999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.497,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.93600000000000005},"last":0.33800000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.495,"15min":0.496},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63400000000000001,"5min":0.747,"15min":0.749},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65000000000000002,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.378}]},{"osd":7,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.54700000000000004},"min":{"1min":0.40999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.78200000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77500000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574993,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1466368,"data_stored":637677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.54600000000000004},"min":{"1min":0.34300000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.84599999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.78600000000000003},"last":0.48199999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.52300000000000002,"15min":0.50700000000000001},"min":{"1min":0.36799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63500000000000001,"5min":0.72899999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.28999999999999998},"max":{"1min":0.68700000000000006,"5min":0.751,"15min":0.77400000000000002},"last":0.38600000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58999999999999997,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.39400000000000002},"max":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.80200000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59699999999999998,"15min":0.57299999999999995},"min":{"1min":0.47699999999999998,"5min":0.39700000000000002,"15min":0.372},"max":{"1min":0.73099999999999998,"5min":0.75600000000000001,"15min":0.82199999999999995},"last":0.57199999999999995}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68300000000000005,"5min":0.751,"15min":1.0369999999999999},"last":0.442},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.58999999999999997,"5min":0.77900000000000003,"15min":0.80300000000000005},"last":0.46999999999999997}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29699999999999999,"15min":0.26500000000000001},"max":{"1min":0.77400000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46300000000000002,"15min":0.48799999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.253},"max":{"1min":0.73099999999999998,"5min":0.73199999999999998,"15min":1.008},"last":0.40400000000000003}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46700000000000003,"15min":0.47199999999999998},"min":{"1min":0.29999999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63200000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.82399999999999995},"last":0.5}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33500000000000002,"15min":0.30099999999999999},"max":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":1.0069999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70799999999999996,"5min":0.77600000000000002,"15min":0.93400000000000005},"last":0.55200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574991,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40756,"kb_used_data":5548,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331084,"statfs":{"total":96636764160,"available":96595030016,"internally_reserved":0,"allocated":5681152,"data_stored":4833359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33900000000000002,"15min":0.33300000000000002},"min":{"1min":0.214,"5min":0.192,"15min":0.183},"max":{"1min":0.52300000000000002,"5min":0.67800000000000005,"15min":1.45},"last":0.314},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35699999999999998,"15min":0.34100000000000003},"min":{"1min":0.23400000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.499,"5min":0.60499999999999998,"15min":1.4099999999999999},"last":0.20699999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.48299999999999998,"5min":0.56699999999999995,"15min":1.323},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.36699999999999999},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.94599999999999995},"last":0.22900000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.42799999999999999,"15min":0.40799999999999997},"min":{"1min":0.314,"5min":0.27100000000000002,"15min":0.251},"max":{"1min":0.55600000000000005,"5min":0.68999999999999995,"15min":1.1619999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.43099999999999999},"min":{"1min":0.313,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.59299999999999997,"5min":0.64900000000000002,"15min":1.2989999999999999},"last":0.371}]},{"osd":4,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.35399999999999998},"max":{"1min":0.85399999999999998,"5min":0.88600000000000001,"15min":1.337},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.41599999999999998,"5min":0.35699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":2.758},"last":0.50900000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.61099999999999999},"min":{"1min":0.49199999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.872,"5min":0.872,"15min":1.276},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.77100000000000002,"5min":0.91200000000000003,"15min":1.895},"last":0.56200000000000006}]},{"osd":6,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45100000000000001,"5min":0.41299999999999998,"15min":0.40699999999999997},"max":{"1min":0.78500000000000003,"5min":0.84199999999999997,"15min":5.4539999999999997},"last":0.504},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.432,"5min":0.41599999999999998,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":0.96499999999999997,"15min":2.8159999999999998},"last":0.52700000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.625,"15min":0.622},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.94899999999999995,"15min":3.7400000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.40300000000000002},"max":{"1min":0.91900000000000004,"5min":0.93600000000000005,"15min":5.4900000000000002},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574992,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":1482752,"data_stored":643993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34300000000000003,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.19500000000000001,"15min":0.191},"max":{"1min":0.51100000000000001,"5min":0.55500000000000005,"15min":0.60399999999999998},"last":0.182},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.22},"max":{"1min":0.50600000000000001,"5min":0.59699999999999998,"15min":0.872},"last":0.25700000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":0.55100000000000005,"5min":0.60899999999999999,"15min":0.72399999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39800000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.20899999999999999},"max":{"1min":0.51500000000000001,"5min":0.61199999999999999,"15min":0.746},"last":0.33600000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41699999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.27600000000000002,"15min":0.22900000000000001},"max":{"1min":0.55100000000000005,"5min":0.65200000000000002,"15min":0.67100000000000004},"last":0.248},{"interface":"front","average":{"1min":0.374,"5min":0.39200000000000002,"15min":0.39700000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":0.503,"5min":0.59199999999999997,"15min":0.72499999999999998},"last":0.27800000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81499999999999995,"5min":0.87,"15min":1.216},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81699999999999995,"5min":0.94699999999999995,"15min":0.99299999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.625,"15min":0.627},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.373},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.95699999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.628},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83099999999999996,"5min":0.98499999999999999,"15min":1.101},"last":0.496}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.97299999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":1.0780000000000001},"last":0.50900000000000001}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":1.155},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.41599999999999998},"max":{"1min":0.81599999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40760,"kb_used_data":5548,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331080,"statfs":{"total":96636764160,"available":96595025920,"internally_reserved":0,"allocated":5681152,"data_stored":4838416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.223,"5min":0.185,"15min":0.185},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38500000000000001,"15min":0.41999999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.20699999999999999},"max":{"1min":0.57799999999999996,"5min":0.59099999999999997,"15min":2.7069999999999999},"last":0.33400000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.35599999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.46400000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.435}]},{"osd":3,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.432,"15min":0.42199999999999999},"min":{"1min":0.29399999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55100000000000005,"5min":0.746,"15min":0.76900000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.378,"5min":0.40400000000000003,"15min":0.44700000000000001},"min":{"1min":0.26100000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.63,"5min":0.73999999999999999,"15min":2.75},"last":0.29299999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.61899999999999999,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.29099999999999998},"max":{"1min":0.83399999999999996,"5min":0.95899999999999996,"15min":0.97299999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.61699999999999999},"min":{"1min":0.42799999999999999,"5min":0.39400000000000002,"15min":0.38600000000000001},"max":{"1min":0.84599999999999997,"5min":0.873,"15min":1.002},"last":0.58499999999999996}]},{"osd":5,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63100000000000001,"15min":0.66000000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35299999999999998},"max":{"1min":1.026,"5min":1.026,"15min":2.778},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.88800000000000001,"15min":1.0649999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64200000000000002,"15min":0.66900000000000004},"min":{"1min":0.44600000000000001,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":2.8210000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.438,"15min":0.38800000000000001},"max":{"1min":0.80600000000000005,"5min":0.93300000000000005,"15min":1.1080000000000001},"last":0.69699999999999995}]},{"osd":7,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.95399999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.43099999999999999,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":2.7989999999999999},"last":0.68500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41336,"kb_used_data":6128,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330504,"statfs":{"total":96636764160,"available":96594436096,"internally_reserved":0,"allocated":6275072,"data_stored":5423150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.34499999999999997,"15min":0.33500000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.184},"max":{"1min":0.52300000000000002,"5min":0.625,"15min":0.625},"last":0.376},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.24099999999999999,"5min":0.22900000000000001,"15min":0.217},"max":{"1min":0.49099999999999999,"5min":0.61699999999999999,"15min":0.64900000000000002},"last":0.39100000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38400000000000001,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.58099999999999996,"5min":0.63,"15min":0.63},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.23799999999999999,"5min":0.22600000000000001,"15min":0.21199999999999999},"max":{"1min":0.497,"5min":0.58199999999999996,"15min":0.59399999999999997},"last":0.434}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39300000000000002,"15min":0.38900000000000001},"min":{"1min":0.30299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.44700000000000001,"15min":0.437},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55800000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.52500000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58699999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67000000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.67700000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.35299999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71799999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.42299999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.94299999999999995,"15min":0.95999999999999996},"last":0.626}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78500000000000003,"5min":0.91900000000000004,"15min":0.93600000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.46600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61899999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.40999999999999998,"15min":0.39300000000000002},"max":{"1min":0.753,"5min":0.85499999999999998,"15min":0.93600000000000005},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73999999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:52:47.260 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-23T09:52:47.260 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:52:47.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.360+0000 7f940b59f640 1 Processor -- start 2024-03-23T09:52:47.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.360+0000 7f940b59f640 1 -- start start 2024-03-23T09:52:47.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.361+0000 7f940b59f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940415cb40 0x7f9404160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.361+0000 7f940b59f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f94040b8300 0x7f94040cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.361+0000 7f940b59f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94040d0ee0 con 0x7f9404160cb0 2024-03-23T09:52:47.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.361+0000 7f940b59f640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94040b5a10 con 0x7f94040b8300 2024-03-23T09:52:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.361+0000 7f940b59f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94041616f0 con 0x7f940415cb40 2024-03-23T09:52:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.361+0000 7f9409b3c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9404160cb0 0x7f94040b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38024/0 (socket says 172.21.15.47:38024) 2024-03-23T09:52:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.361+0000 7f9409b3c640 1 -- 172.21.15.47:0/2053820092 learned_addr learned my addr 172.21.15.47:0/2053820092 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.361+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710934329 0 0) 0x7f94040d0ee0 con 0x7f9404160cb0 2024-03-23T09:52:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.361+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93ec003520 con 0x7f9404160cb0 2024-03-23T09:52:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9408b3a640 1 --2- 172.21.15.47:0/2053820092 >> v2:172.21.15.175:3300/0 conn(0x7f94040b8300 0x7f94040cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f940933b640 1 --2- 172.21.15.47:0/2053820092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940415cb40 0x7f9404160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3215381496 0 0) 0x7f93ec003520 con 0x7f9404160cb0 2024-03-23T09:52:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940415cb40 msgr2=0x7f9404160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 --2- 172.21.15.47:0/2053820092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940415cb40 0x7f9404160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 >> v2:172.21.15.175:3300/0 conn(0x7f94040b8300 msgr2=0x7f94040cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 --2- 172.21.15.47:0/2053820092 >> v2:172.21.15.175:3300/0 conn(0x7f94040b8300 0x7f94040cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9404161130 con 0x7f9404160cb0 2024-03-23T09:52:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f93f40019f0 con 0x7f9404160cb0 2024-03-23T09:52:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 >> v1:172.21.15.47:6789/0 conn(0x7f9404160cb0 legacy=0x7f94040b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 --2- 172.21.15.47:0/2053820092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ec003970 0x7f93ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 --2- 172.21.15.47:0/2053820092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940415cb40 0x7f93ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 --2- 172.21.15.47:0/2053820092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec004d00 0x7f93ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94040b5a10 con 0x7f93ec004d00 2024-03-23T09:52:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94041616f0 con 0x7f93ec003970 2024-03-23T09:52:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec00b980 con 0x7f940415cb40 2024-03-23T09:52:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9409b3c640 1 --2- 172.21.15.47:0/2053820092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec004d00 0x7f93ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9408b3a640 1 --2- 172.21.15.47:0/2053820092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940415cb40 0x7f93ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f940933b640 1 --2- 172.21.15.47:0/2053820092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ec003970 0x7f93ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9409b3c640 1 -- 172.21.15.47:0/2053820092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940415cb40 msgr2=0x7f93ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9409b3c640 1 --2- 172.21.15.47:0/2053820092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940415cb40 0x7f93ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9409b3c640 1 -- 172.21.15.47:0/2053820092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ec003970 msgr2=0x7f93ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9409b3c640 1 --2- 172.21.15.47:0/2053820092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ec003970 0x7f93ec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.362+0000 7f9409b3c640 1 -- 172.21.15.47:0/2053820092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec003520 con 0x7f93ec004d00 2024-03-23T09:52:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.363+0000 7f9409b3c640 1 --2- 172.21.15.47:0/2053820092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec004d00 0x7f93ec005180 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f93f400a8d0 tx=0x7f93ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450cb67ff7118e6d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.363+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4034630 con 0x7f93ec004d00 2024-03-23T09:52:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.363+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f402f070 con 0x7f93ec004d00 2024-03-23T09:52:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.363+0000 7f9403fff640 1 -- 172.21.15.47:0/2053820092 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40349d0 con 0x7f93ec004d00 2024-03-23T09:52:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.364+0000 7f940b59f640 1 -- 172.21.15.47:0/2053820092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec004d00 msgr2=0x7f93ec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.364+0000 7f940b59f640 1 --2- 172.21.15.47:0/2053820092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ec004d00 0x7f93ec005180 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f93f400a8d0 tx=0x7f93ec0057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.364+0000 7f9409b3c640 1 -- 172.21.15.47:0/2053820092 reap_dead start 2024-03-23T09:52:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.364+0000 7f940b59f640 1 -- 172.21.15.47:0/2053820092 shutdown_connections 2024-03-23T09:52:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.364+0000 7f940b59f640 1 -- 172.21.15.47:0/2053820092 >> 172.21.15.47:0/2053820092 conn(0x7f940406b160 msgr2=0x7f940406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.364+0000 7f940b59f640 1 -- 172.21.15.47:0/2053820092 shutdown_connections 2024-03-23T09:52:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.364+0000 7f940b59f640 1 -- 172.21.15.47:0/2053820092 wait complete. 2024-03-23T09:52:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.365+0000 7f940b59f640 1 Processor -- start 2024-03-23T09:52:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.365+0000 7f940b59f640 1 -- start start 2024-03-23T09:52:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f940b59f640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940415a840 0x7f9404159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f940b59f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9404159740 0x7f940415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f940b59f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f940415ac20 0x7f940415e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f940b59f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940415d220 con 0x7f9404159740 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f940b59f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940415d500 con 0x7f940415ac20 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f940b59f640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940415d7e0 con 0x7f940415a840 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f9408b3a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9404159740 0x7f940415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f9408b3a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9404159740 0x7f940415e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40108/0 (socket says 172.21.15.47:40108) 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f9408b3a640 1 -- 172.21.15.47:0/2318158828 learned_addr learned my addr 172.21.15.47:0/2318158828 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f940933b640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940415a840 0x7f9404159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f9409b3c640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f940415ac20 0x7f940415e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f9408b3a640 1 -- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940415a840 msgr2=0x7f9404159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f9408b3a640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940415a840 0x7f9404159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f9408b3a640 1 -- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f940415ac20 msgr2=0x7f940415e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f9408b3a640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f940415ac20 0x7f940415e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f9408b3a640 1 -- 172.21.15.47:0/2318158828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940415dac0 con 0x7f9404159740 2024-03-23T09:52:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f9408b3a640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9404159740 0x7f940415e2c0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f93f8011980 tx=0x7f93f8011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f9401ffb640 1 -- 172.21.15.47:0/2318158828 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800faf0 con 0x7f9404159740 2024-03-23T09:52:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f9401ffb640 1 -- 172.21.15.47:0/2318158828 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f800fc50 con 0x7f9404159740 2024-03-23T09:52:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f940b59f640 1 -- 172.21.15.47:0/2318158828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940415dda0 con 0x7f9404159740 2024-03-23T09:52:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.366+0000 7f9401ffb640 1 -- 172.21.15.47:0/2318158828 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8014440 con 0x7f9404159740 2024-03-23T09:52:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.367+0000 7f940b59f640 1 -- 172.21.15.47:0/2318158828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9404160090 con 0x7f9404159740 2024-03-23T09:52:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.368+0000 7f940b59f640 1 -- 172.21.15.47:0/2318158828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9404072360 con 0x7f9404159740 2024-03-23T09:52:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.368+0000 7f9401ffb640 1 -- 172.21.15.47:0/2318158828 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f93f80145a0 con 0x7f9404159740 2024-03-23T09:52:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.369+0000 7f9401ffb640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f93f006ac00 0x7f93f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.369+0000 7f9401ffb640 1 -- 172.21.15.47:0/2318158828 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f93f8089ae0 con 0x7f9404159740 2024-03-23T09:52:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.369+0000 7f940933b640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f93f006ac00 0x7f93f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.369+0000 7f940933b640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f93f006ac00 0x7f93f006d0c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009a80 tx=0x7f93fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.373+0000 7f9401ffb640 1 -- 172.21.15.47:0/2318158828 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f93f8054e70 con 0x7f9404159740 2024-03-23T09:52:47.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.549+0000 7f940b59f640 1 -- 172.21.15.47:0/2318158828 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94040d0b70 con 0x7f93f006ac00 2024-03-23T09:52:47.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.551+0000 7f9401ffb640 1 -- 172.21.15.47:0/2318158828 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50857 (secure 0 0 0) 0x7f93fc0096f0 con 0x7f93f006ac00 2024-03-23T09:52:47.552 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:47.552 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:52:47.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.553+0000 7f940b59f640 1 -- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f93f006ac00 msgr2=0x7f93f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.553+0000 7f940b59f640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f93f006ac00 0x7f93f006d0c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009a80 tx=0x7f93fc008040 comp rx=0 tx=0).stop 2024-03-23T09:52:47.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.553+0000 7f940b59f640 1 -- 172.21.15.47:0/2318158828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9404159740 msgr2=0x7f940415e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.553+0000 7f940b59f640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9404159740 0x7f940415e2c0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f93f8011980 tx=0x7f93f8011e40 comp rx=0 tx=0).stop 2024-03-23T09:52:47.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.553+0000 7f940b59f640 1 -- 172.21.15.47:0/2318158828 shutdown_connections 2024-03-23T09:52:47.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.553+0000 7f940b59f640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f93f006ac00 0x7f93f006d0c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.553+0000 7f940b59f640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f940415ac20 0x7f940415e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.553+0000 7f940b59f640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9404159740 0x7f940415e2c0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.553+0000 7f940b59f640 1 --2- 172.21.15.47:0/2318158828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940415a840 0x7f9404159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.553+0000 7f940b59f640 1 -- 172.21.15.47:0/2318158828 >> 172.21.15.47:0/2318158828 conn(0x7f940406b160 msgr2=0x7f94040cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:47.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.553+0000 7f940b59f640 1 -- 172.21.15.47:0/2318158828 shutdown_connections 2024-03-23T09:52:47.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.553+0000 7f940b59f640 1 -- 172.21.15.47:0/2318158828 wait complete. 2024-03-23T09:52:47.564 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-03-23T09:52:46.515972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":294428,"kb_used_data":29128,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754680292,"statfs":{"total":773094113280,"available":772792619008,"internally_reserved":0,"allocated":29827072,"data_stored":23087610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":271568440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390585,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-172,"num_write_kb":-8291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001344"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":266,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-23T09:52:25.491130+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:52:25.491130+0000","last_peered":"2024-03-23T09:52:25.491130+0000","last_clean":"2024-03-23T09:52:25.491130+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:52:25.491130+0000","last_undegraded":"2024-03-23T09:52:25.491130+0000","last_fullsized":"2024-03-23T09:52:25.491130+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:56:00.079449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574992,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33132,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338708,"statfs":{"total":96636764160,"available":96602836992,"internally_reserved":0,"allocated":2068480,"data_stored":1228661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55200000000000005,"15min":0.52700000000000002},"min":{"1min":0.41399999999999998,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.623,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.309},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.44700000000000001,"5min":0.32200000000000001,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.78900000000000003,"15min":0.85999999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.40799999999999997,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.63,"5min":0.79100000000000004,"15min":0.80400000000000005},"last":0.501},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.63600000000000001,"5min":0.747,"15min":0.81299999999999994},"last":0.433}]},{"osd":2,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55200000000000005},"min":{"1min":0.44500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.78500000000000003,"15min":0.82499999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.622,"5min":0.83099999999999996,"15min":0.83599999999999997},"last":0.52600000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.46000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.61699999999999999,"5min":0.73999999999999999,"15min":0.86199999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.627,"5min":0.77400000000000002,"15min":0.83699999999999997},"last":0.42099999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46700000000000003,"15min":0.48999999999999999},"min":{"1min":0.379,"5min":0.29799999999999999,"15min":0.29599999999999999},"max":{"1min":0.61699999999999999,"5min":0.67500000000000004,"15min":0.78200000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46700000000000003,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.29299999999999998},"max":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":0.80100000000000005},"last":0.45100000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.46500000000000002,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":0.69799999999999995,"15min":0.81499999999999995},"last":0.443},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.498,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.63300000000000001,"5min":0.73199999999999998,"15min":0.84999999999999998},"last":0.51400000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.55100000000000005},"min":{"1min":0.34699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66300000000000003,"5min":0.75,"15min":0.874},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51200000000000001,"15min":0.55600000000000005},"min":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69199999999999995,"5min":0.73999999999999999,"15min":0.90200000000000002},"last":0.48699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574992,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":5544,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":5677056,"data_stored":4837750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.53300000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.27200000000000002},"max":{"1min":0.746,"5min":0.77700000000000002,"15min":0.81000000000000005},"last":0.505},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57199999999999995,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.76200000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.311},"max":{"1min":0.77600000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.41299999999999998,"15min":0.32700000000000001},"max":{"1min":0.751,"5min":0.80300000000000005,"15min":0.82999999999999996},"last":0.39700000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.29699999999999999},"max":{"1min":0.69499999999999995,"5min":0.747,"15min":0.81599999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59099999999999997,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58899999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.42399999999999999,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73299999999999998,"5min":0.79000000000000004,"15min":0.81699999999999995},"last":0.64100000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.48199999999999998},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65600000000000003,"5min":0.69299999999999995,"15min":0.72499999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.47599999999999998,"15min":0.51700000000000002},"min":{"1min":0.35899999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.63800000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.71299999999999997}]},{"osd":5,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48099999999999998,"15min":0.505},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.316},"max":{"1min":0.64100000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51200000000000001,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.84399999999999997}]},{"osd":7,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52200000000000002,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.72899999999999998,"5min":0.754,"15min":0.82699999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54100000000000004,"15min":0.56499999999999995},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69299999999999995,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.72899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1495040,"data_stored":644604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54300000000000004,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.68700000000000006,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.39400000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73699999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42199999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69799999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51500000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.55100000000000005},"min":{"1min":0.38600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71999999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.54800000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.748,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.495}]},{"osd":3,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58499999999999996,"15min":0.54700000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70899999999999996,"5min":0.77400000000000002,"15min":0.81599999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.77400000000000002},"last":0.441}]},{"osd":4,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29999999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68500000000000005,"5min":0.73099999999999998,"15min":0.73699999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.497,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.93600000000000005},"last":0.33800000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.495,"15min":0.496},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63400000000000001,"5min":0.747,"15min":0.749},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65000000000000002,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.378}]},{"osd":7,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.54700000000000004},"min":{"1min":0.40999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.78200000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77500000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574993,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1466368,"data_stored":637677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.54600000000000004},"min":{"1min":0.34300000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.84599999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.78600000000000003},"last":0.48199999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.52300000000000002,"15min":0.50700000000000001},"min":{"1min":0.36799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63500000000000001,"5min":0.72899999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.28999999999999998},"max":{"1min":0.68700000000000006,"5min":0.751,"15min":0.77400000000000002},"last":0.38600000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58999999999999997,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.39400000000000002},"max":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.80200000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59699999999999998,"15min":0.57299999999999995},"min":{"1min":0.47699999999999998,"5min":0.39700000000000002,"15min":0.372},"max":{"1min":0.73099999999999998,"5min":0.75600000000000001,"15min":0.82199999999999995},"last":0.57199999999999995}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68300000000000005,"5min":0.751,"15min":1.0369999999999999},"last":0.442},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.58999999999999997,"5min":0.77900000000000003,"15min":0.80300000000000005},"last":0.46999999999999997}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29699999999999999,"15min":0.26500000000000001},"max":{"1min":0.77400000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46300000000000002,"15min":0.48799999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.253},"max":{"1min":0.73099999999999998,"5min":0.73199999999999998,"15min":1.008},"last":0.40400000000000003}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46700000000000003,"15min":0.47199999999999998},"min":{"1min":0.29999999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63200000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.82399999999999995},"last":0.5}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33500000000000002,"15min":0.30099999999999999},"max":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":1.0069999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70799999999999996,"5min":0.77600000000000002,"15min":0.93400000000000005},"last":0.55200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574991,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40756,"kb_used_data":5548,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331084,"statfs":{"total":96636764160,"available":96595030016,"internally_reserved":0,"allocated":5681152,"data_stored":4833359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33900000000000002,"15min":0.33300000000000002},"min":{"1min":0.214,"5min":0.192,"15min":0.183},"max":{"1min":0.52300000000000002,"5min":0.67800000000000005,"15min":1.45},"last":0.314},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35699999999999998,"15min":0.34100000000000003},"min":{"1min":0.23400000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.499,"5min":0.60499999999999998,"15min":1.4099999999999999},"last":0.20699999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.48299999999999998,"5min":0.56699999999999995,"15min":1.323},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.36699999999999999},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.94599999999999995},"last":0.22900000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.42799999999999999,"15min":0.40799999999999997},"min":{"1min":0.314,"5min":0.27100000000000002,"15min":0.251},"max":{"1min":0.55600000000000005,"5min":0.68999999999999995,"15min":1.1619999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.43099999999999999},"min":{"1min":0.313,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.59299999999999997,"5min":0.64900000000000002,"15min":1.2989999999999999},"last":0.371}]},{"osd":4,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.35399999999999998},"max":{"1min":0.85399999999999998,"5min":0.88600000000000001,"15min":1.337},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.41599999999999998,"5min":0.35699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":2.758},"last":0.50900000000000001}]},{"osd":5,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.61099999999999999},"min":{"1min":0.49199999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.872,"5min":0.872,"15min":1.276},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.77100000000000002,"5min":0.91200000000000003,"15min":1.895},"last":0.56200000000000006}]},{"osd":6,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45100000000000001,"5min":0.41299999999999998,"15min":0.40699999999999997},"max":{"1min":0.78500000000000003,"5min":0.84199999999999997,"15min":5.4539999999999997},"last":0.504},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.432,"5min":0.41599999999999998,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":0.96499999999999997,"15min":2.8159999999999998},"last":0.52700000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.625,"15min":0.622},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.94899999999999995,"15min":3.7400000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.40300000000000002},"max":{"1min":0.91900000000000004,"5min":0.93600000000000005,"15min":5.4900000000000002},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574992,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":1482752,"data_stored":643993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34300000000000003,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.19500000000000001,"15min":0.191},"max":{"1min":0.51100000000000001,"5min":0.55500000000000005,"15min":0.60399999999999998},"last":0.182},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.22},"max":{"1min":0.50600000000000001,"5min":0.59699999999999998,"15min":0.872},"last":0.25700000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":0.55100000000000005,"5min":0.60899999999999999,"15min":0.72399999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39800000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.20899999999999999},"max":{"1min":0.51500000000000001,"5min":0.61199999999999999,"15min":0.746},"last":0.33600000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41699999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.27600000000000002,"15min":0.22900000000000001},"max":{"1min":0.55100000000000005,"5min":0.65200000000000002,"15min":0.67100000000000004},"last":0.248},{"interface":"front","average":{"1min":0.374,"5min":0.39200000000000002,"15min":0.39700000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":0.503,"5min":0.59199999999999997,"15min":0.72499999999999998},"last":0.27800000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81499999999999995,"5min":0.87,"15min":1.216},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81699999999999995,"5min":0.94699999999999995,"15min":0.99299999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.625,"15min":0.627},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.373},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.95699999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.628},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83099999999999996,"5min":0.98499999999999999,"15min":1.101},"last":0.496}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.97299999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":1.0780000000000001},"last":0.50900000000000001}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":1.155},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.41599999999999998},"max":{"1min":0.81599999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40760,"kb_used_data":5548,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331080,"statfs":{"total":96636764160,"available":96595025920,"internally_reserved":0,"allocated":5681152,"data_stored":4838416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.223,"5min":0.185,"15min":0.185},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38500000000000001,"15min":0.41999999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.20699999999999999},"max":{"1min":0.57799999999999996,"5min":0.59099999999999997,"15min":2.7069999999999999},"last":0.33400000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.35599999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.46400000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.435}]},{"osd":3,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.432,"15min":0.42199999999999999},"min":{"1min":0.29399999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55100000000000005,"5min":0.746,"15min":0.76900000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.378,"5min":0.40400000000000003,"15min":0.44700000000000001},"min":{"1min":0.26100000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.63,"5min":0.73999999999999999,"15min":2.75},"last":0.29299999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.61899999999999999,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.29099999999999998},"max":{"1min":0.83399999999999996,"5min":0.95899999999999996,"15min":0.97299999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.61699999999999999},"min":{"1min":0.42799999999999999,"5min":0.39400000000000002,"15min":0.38600000000000001},"max":{"1min":0.84599999999999997,"5min":0.873,"15min":1.002},"last":0.58499999999999996}]},{"osd":5,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63100000000000001,"15min":0.66000000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35299999999999998},"max":{"1min":1.026,"5min":1.026,"15min":2.778},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.88800000000000001,"15min":1.0649999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64200000000000002,"15min":0.66900000000000004},"min":{"1min":0.44600000000000001,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":2.8210000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.438,"15min":0.38800000000000001},"max":{"1min":0.80600000000000005,"5min":0.93300000000000005,"15min":1.1080000000000001},"last":0.69699999999999995}]},{"osd":7,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.95399999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.43099999999999999,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":2.7989999999999999},"last":0.68500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41336,"kb_used_data":6128,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330504,"statfs":{"total":96636764160,"available":96594436096,"internally_reserved":0,"allocated":6275072,"data_stored":5423150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.34499999999999997,"15min":0.33500000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.184},"max":{"1min":0.52300000000000002,"5min":0.625,"15min":0.625},"last":0.376},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.24099999999999999,"5min":0.22900000000000001,"15min":0.217},"max":{"1min":0.49099999999999999,"5min":0.61699999999999999,"15min":0.64900000000000002},"last":0.39100000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38400000000000001,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.58099999999999996,"5min":0.63,"15min":0.63},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.23799999999999999,"5min":0.22600000000000001,"15min":0.21199999999999999},"max":{"1min":0.497,"5min":0.58199999999999996,"15min":0.59399999999999997},"last":0.434}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39300000000000002,"15min":0.38900000000000001},"min":{"1min":0.30299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.44700000000000001,"15min":0.437},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55800000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.52500000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58699999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67000000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.67700000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.35299999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71799999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.42299999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.94299999999999995,"15min":0.95999999999999996},"last":0.626}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78500000000000003,"5min":0.91900000000000004,"15min":0.93600000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.46600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61899999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.40999999999999998,"15min":0.39300000000000002},"max":{"1min":0.753,"5min":0.85499999999999998,"15min":0.93600000000000005},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73999999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:52:47.566 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T09:52:47.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.666+0000 7f1f6fd55640 1 Processor -- start 2024-03-23T09:52:47.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.666+0000 7f1f6fd55640 1 -- start start 2024-03-23T09:52:47.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.667+0000 7f1f6fd55640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1f680cd090 0x7f1f680cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.667+0000 7f1f6fd55640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1f6815ab50 0x7f1f680cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.667+0000 7f1f6fd55640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f680cd9b0 con 0x7f1f680cb6f0 2024-03-23T09:52:47.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.667+0000 7f1f6fd55640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f680cbf50 con 0x7f1f680cd090 2024-03-23T09:52:47.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.667+0000 7f1f6fd55640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f680cc230 con 0x7f1f6815ab50 2024-03-23T09:52:47.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.667+0000 7f1f6daf1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1f680cd090 0x7f1f680cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.667+0000 7f1f6e2f2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1f680cb6f0 0x7f1f68098a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38034/0 (socket says 172.21.15.47:38034) 2024-03-23T09:52:47.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.667+0000 7f1f6daf1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1f680cd090 0x7f1f680cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40122/0 (socket says 172.21.15.47:40122) 2024-03-23T09:52:47.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.667+0000 7f1f6e2f2640 1 -- 172.21.15.47:0/2942847158 learned_addr learned my addr 172.21.15.47:0/2942847158 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:47.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.668+0000 7f1f6d2f0640 1 --2- 172.21.15.47:0/2942847158 >> v2:172.21.15.175:3300/0 conn(0x7f1f6815ab50 0x7f1f680cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.668+0000 7f1f6daf1640 1 -- 172.21.15.47:0/2942847158 >> v2:172.21.15.175:3300/0 conn(0x7f1f6815ab50 msgr2=0x7f1f680cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:47.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.668+0000 7f1f6daf1640 1 --2- 172.21.15.47:0/2942847158 >> v2:172.21.15.175:3300/0 conn(0x7f1f6815ab50 0x7f1f680cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.668+0000 7f1f6daf1640 1 -- 172.21.15.47:0/2942847158 >> v1:172.21.15.47:6789/0 conn(0x7f1f680cb6f0 legacy=0x7f1f68098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.668+0000 7f1f6daf1640 1 -- 172.21.15.47:0/2942847158 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f680cc510 con 0x7f1f680cd090 2024-03-23T09:52:47.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.668+0000 7f1f6caef640 1 -- 172.21.15.47:0/2942847158 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553406644 0 0) 0x7f1f680cd9b0 con 0x7f1f680cb6f0 2024-03-23T09:52:47.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.668+0000 7f1f6d2f0640 1 --2- 172.21.15.47:0/2942847158 >> v2:172.21.15.175:3300/0 conn(0x7f1f6815ab50 0x7f1f680cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:47.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.668+0000 7f1f6daf1640 1 --2- 172.21.15.47:0/2942847158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f680cd090 0x7f1f680cd470 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f1f680cdd10 tx=0x7f1f6002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7638745d8fdf3d09 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:47.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.669+0000 7f1f6caef640 1 -- 172.21.15.47:0/2942847158 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60030610 con 0x7f1f680cd090 2024-03-23T09:52:47.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.669+0000 7f1f6caef640 1 -- 172.21.15.47:0/2942847158 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f60030770 con 0x7f1f680cd090 2024-03-23T09:52:47.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.669+0000 7f1f6caef640 1 -- 172.21.15.47:0/2942847158 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60030b00 con 0x7f1f680cd090 2024-03-23T09:52:47.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.669+0000 7f1f6fd55640 1 -- 172.21.15.47:0/2942847158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f680cd090 msgr2=0x7f1f680cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.669+0000 7f1f6fd55640 1 --2- 172.21.15.47:0/2942847158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f680cd090 0x7f1f680cd470 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f1f680cdd10 tx=0x7f1f6002b960 comp rx=0 tx=0).stop 2024-03-23T09:52:47.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.669+0000 7f1f6fd55640 1 -- 172.21.15.47:0/2942847158 shutdown_connections 2024-03-23T09:52:47.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.669+0000 7f1f6fd55640 1 --2- 172.21.15.47:0/2942847158 >> v2:172.21.15.175:3300/0 conn(0x7f1f6815ab50 0x7f1f680cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.669+0000 7f1f6fd55640 1 --2- 172.21.15.47:0/2942847158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f680cd090 0x7f1f680cd470 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.669+0000 7f1f6fd55640 1 -- 172.21.15.47:0/2942847158 >> 172.21.15.47:0/2942847158 conn(0x7f1f680d06a0 msgr2=0x7f1f680d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.669+0000 7f1f6fd55640 1 -- 172.21.15.47:0/2942847158 shutdown_connections 2024-03-23T09:52:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.669+0000 7f1f6fd55640 1 -- 172.21.15.47:0/2942847158 wait complete. 2024-03-23T09:52:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.670+0000 7f1f6fd55640 1 Processor -- start 2024-03-23T09:52:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.670+0000 7f1f6fd55640 1 -- start start 2024-03-23T09:52:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.671+0000 7f1f6fd55640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f680cb6f0 0x7f1f6814edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.671+0000 7f1f6fd55640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f680cd090 0x7f1f6815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.671+0000 7f1f6fd55640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f6815ab50 0x7f1f6815d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.671+0000 7f1f6fd55640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f680c8af0 con 0x7f1f680cb6f0 2024-03-23T09:52:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.671+0000 7f1f6fd55640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f680c8660 con 0x7f1f680cd090 2024-03-23T09:52:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.671+0000 7f1f6fd55640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6814f310 con 0x7f1f6815ab50 2024-03-23T09:52:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.671+0000 7f1f6daf1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f680cb6f0 0x7f1f6814edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.671+0000 7f1f6daf1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f680cb6f0 0x7f1f6814edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40130/0 (socket says 172.21.15.47:40130) 2024-03-23T09:52:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.671+0000 7f1f6daf1640 1 -- 172.21.15.47:0/982894520 learned_addr learned my addr 172.21.15.47:0/982894520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.671+0000 7f1f6e2f2640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f6815ab50 0x7f1f6815d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.671+0000 7f1f6daf1640 1 -- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f6815ab50 msgr2=0x7f1f6815d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.671+0000 7f1f6d2f0640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f680cd090 0x7f1f6815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.671+0000 7f1f6daf1640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f6815ab50 0x7f1f6815d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.672+0000 7f1f6daf1640 1 -- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f680cd090 msgr2=0x7f1f6815cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.672+0000 7f1f6daf1640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f680cd090 0x7f1f6815cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.672+0000 7f1f6daf1640 1 -- 172.21.15.47:0/982894520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f60009400 con 0x7f1f680cb6f0 2024-03-23T09:52:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.672+0000 7f1f6d2f0640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f680cd090 0x7f1f6815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.672+0000 7f1f6daf1640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f680cb6f0 0x7f1f6814edd0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009890 tx=0x7f1f60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.672+0000 7f1f667fc640 1 -- 172.21.15.47:0/982894520 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6003e040 con 0x7f1f680cb6f0 2024-03-23T09:52:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.672+0000 7f1f667fc640 1 -- 172.21.15.47:0/982894520 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f60039cb0 con 0x7f1f680cb6f0 2024-03-23T09:52:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.672+0000 7f1f667fc640 1 -- 172.21.15.47:0/982894520 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60044630 con 0x7f1f680cb6f0 2024-03-23T09:52:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.672+0000 7f1f6fd55640 1 -- 172.21.15.47:0/982894520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f680c7770 con 0x7f1f680cb6f0 2024-03-23T09:52:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.672+0000 7f1f6fd55640 1 -- 172.21.15.47:0/982894520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f680c7d30 con 0x7f1f680cb6f0 2024-03-23T09:52:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.673+0000 7f1f6fd55640 1 -- 172.21.15.47:0/982894520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f680672f0 con 0x7f1f680cb6f0 2024-03-23T09:52:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.673+0000 7f1f667fc640 1 -- 172.21.15.47:0/982894520 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1f60044790 con 0x7f1f680cb6f0 2024-03-23T09:52:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.674+0000 7f1f667fc640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1f5406ac30 0x7f1f5406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.674+0000 7f1f667fc640 1 -- 172.21.15.47:0/982894520 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f1f600ab590 con 0x7f1f680cb6f0 2024-03-23T09:52:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.674+0000 7f1f6d2f0640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1f5406ac30 0x7f1f5406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.675+0000 7f1f6d2f0640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1f5406ac30 0x7f1f5406d0f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006fd0 tx=0x7f1f5c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:47.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.678+0000 7f1f667fc640 1 -- 172.21.15.47:0/982894520 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1f60076b70 con 0x7f1f680cb6f0 2024-03-23T09:52:47.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.855+0000 7f1f6fd55640 1 -- 172.21.15.47:0/982894520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f6806d2b0 con 0x7f1f680cb6f0 2024-03-23T09:52:47.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.856+0000 7f1f667fc640 1 -- 172.21.15.47:0/982894520 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+12786 (secure 0 0 0) 0x7f1f6007a190 con 0x7f1f680cb6f0 2024-03-23T09:52:47.856 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:47.857 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":121,"fsid":"f31804be-685c-4b3b-b2be-155afa9f5d13","created":"2024-03-23T09:40:46.025553+0000","modified":"2024-03-23T09:52:44.938416+0000","last_up_change":"2024-03-23T09:40:56.508007+0000","last_in_change":"2024-03-23T09:40:48.303352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T09:40:58.380031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b4c6a13c-49aa-4d7e-8b7b-3164ae0d1c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":37679},{"type":"v1","addr":"172.21.15.47:6817","nonce":37679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":37679},{"type":"v1","addr":"172.21.15.47:6819","nonce":37679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":37679},{"type":"v1","addr":"172.21.15.47:6823","nonce":37679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":37679},{"type":"v1","addr":"172.21.15.47:6821","nonce":37679}]},"public_addr":"172.21.15.47:6817/37679","cluster_addr":"172.21.15.47:6819/37679","heartbeat_back_addr":"172.21.15.47:6823/37679","heartbeat_front_addr":"172.21.15.47:6821/37679","state":["exists","up"]},{"osd":1,"uuid":"49f7a329-d653-43d1-9e27-54b0e93f2eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":37680},{"type":"v1","addr":"172.21.15.47:6825","nonce":37680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":37680},{"type":"v1","addr":"172.21.15.47:6827","nonce":37680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":37680},{"type":"v1","addr":"172.21.15.47:6831","nonce":37680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":37680},{"type":"v1","addr":"172.21.15.47:6829","nonce":37680}]},"public_addr":"172.21.15.47:6825/37680","cluster_addr":"172.21.15.47:6827/37680","heartbeat_back_addr":"172.21.15.47:6831/37680","heartbeat_front_addr":"172.21.15.47:6829/37680","state":["exists","up"]},{"osd":2,"uuid":"8d3fec55-1be5-429f-888f-5ae197d8ad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":37678},{"type":"v1","addr":"172.21.15.47:6801","nonce":37678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":37678},{"type":"v1","addr":"172.21.15.47:6803","nonce":37678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":37678},{"type":"v1","addr":"172.21.15.47:6807","nonce":37678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":37678},{"type":"v1","addr":"172.21.15.47:6805","nonce":37678}]},"public_addr":"172.21.15.47:6801/37678","cluster_addr":"172.21.15.47:6803/37678","heartbeat_back_addr":"172.21.15.47:6807/37678","heartbeat_front_addr":"172.21.15.47:6805/37678","state":["exists","up"]},{"osd":3,"uuid":"07e2cb27-0df4-4a86-8e29-3dd1bc03072f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":37683},{"type":"v1","addr":"172.21.15.47:6809","nonce":37683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":37683},{"type":"v1","addr":"172.21.15.47:6811","nonce":37683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":37683},{"type":"v1","addr":"172.21.15.47:6815","nonce":37683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":37683},{"type":"v1","addr":"172.21.15.47:6813","nonce":37683}]},"public_addr":"172.21.15.47:6809/37683","cluster_addr":"172.21.15.47:6811/37683","heartbeat_back_addr":"172.21.15.47:6815/37683","heartbeat_front_addr":"172.21.15.47:6813/37683","state":["exists","up"]},{"osd":4,"uuid":"ee55dc92-68e7-472b-8953-1e67644f8e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":38159},{"type":"v1","addr":"172.21.15.175:6801","nonce":38159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":38159},{"type":"v1","addr":"172.21.15.175:6803","nonce":38159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":38159},{"type":"v1","addr":"172.21.15.175:6807","nonce":38159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":38159},{"type":"v1","addr":"172.21.15.175:6805","nonce":38159}]},"public_addr":"172.21.15.175:6801/38159","cluster_addr":"172.21.15.175:6803/38159","heartbeat_back_addr":"172.21.15.175:6807/38159","heartbeat_front_addr":"172.21.15.175:6805/38159","state":["exists","up"]},{"osd":5,"uuid":"b6112e8d-e0eb-4b16-bff7-465d2013a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":38161},{"type":"v1","addr":"172.21.15.175:6817","nonce":38161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":38161},{"type":"v1","addr":"172.21.15.175:6819","nonce":38161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":38161},{"type":"v1","addr":"172.21.15.175:6823","nonce":38161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":38161},{"type":"v1","addr":"172.21.15.175:6821","nonce":38161}]},"public_addr":"172.21.15.175:6817/38161","cluster_addr":"172.21.15.175:6819/38161","heartbeat_back_addr":"172.21.15.175:6823/38161","heartbeat_front_addr":"172.21.15.175:6821/38161","state":["exists","up"]},{"osd":6,"uuid":"4770eeee-358c-49ba-bf13-f4d5ef6c9e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":38160},{"type":"v1","addr":"172.21.15.175:6809","nonce":38160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":38160},{"type":"v1","addr":"172.21.15.175:6811","nonce":38160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":38160},{"type":"v1","addr":"172.21.15.175:6815","nonce":38160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":38160},{"type":"v1","addr":"172.21.15.175:6813","nonce":38160}]},"public_addr":"172.21.15.175:6809/38160","cluster_addr":"172.21.15.175:6811/38160","heartbeat_back_addr":"172.21.15.175:6815/38160","heartbeat_front_addr":"172.21.15.175:6813/38160","state":["exists","up"]},{"osd":7,"uuid":"fb4de25d-5bae-4e6e-9ece-54f206914efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":38174},{"type":"v1","addr":"172.21.15.175:6825","nonce":38174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":38174},{"type":"v1","addr":"172.21.15.175:6827","nonce":38174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":38174},{"type":"v1","addr":"172.21.15.175:6831","nonce":38174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":38174},{"type":"v1","addr":"172.21.15.175:6829","nonce":38174}]},"public_addr":"172.21.15.175:6825/38174","cluster_addr":"172.21.15.175:6827/38174","heartbeat_back_addr":"172.21.15.175:6831/38174","heartbeat_front_addr":"172.21.15.175:6829/38174","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.451576+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.469073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.506471+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.495155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.387924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.489717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.419825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T09:40:53.453086+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6834/3501786111":"2024-03-24T09:52:41.651585+0000","172.21.15.47:6832/3501786111":"2024-03-24T09:52:41.651585+0000","172.21.15.47:6832/1868958629":"2024-03-24T09:52:25.484801+0000","172.21.15.47:6834/724782441":"2024-03-24T09:52:05.460377+0000","172.21.15.47:6834/1868958629":"2024-03-24T09:52:25.484801+0000","172.21.15.47:6832/724782441":"2024-03-24T09:52:05.460377+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T09:52:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.858+0000 7f1f6fd55640 1 -- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1f5406ac30 msgr2=0x7f1f5406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.858+0000 7f1f6fd55640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1f5406ac30 0x7f1f5406d0f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006fd0 tx=0x7f1f5c008040 comp rx=0 tx=0).stop 2024-03-23T09:52:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.858+0000 7f1f6fd55640 1 -- 172.21.15.47:0/982894520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f680cb6f0 msgr2=0x7f1f6814edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.858+0000 7f1f6fd55640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f680cb6f0 0x7f1f6814edd0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009890 tx=0x7f1f60004730 comp rx=0 tx=0).stop 2024-03-23T09:52:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.858+0000 7f1f6fd55640 1 -- 172.21.15.47:0/982894520 shutdown_connections 2024-03-23T09:52:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.858+0000 7f1f6fd55640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1f5406ac30 0x7f1f5406d0f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.858+0000 7f1f6fd55640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f6815ab50 0x7f1f6815d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.858+0000 7f1f6fd55640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f680cd090 0x7f1f6815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.858+0000 7f1f6fd55640 1 --2- 172.21.15.47:0/982894520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f680cb6f0 0x7f1f6814edd0 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.858+0000 7f1f6fd55640 1 -- 172.21.15.47:0/982894520 >> 172.21.15.47:0/982894520 conn(0x7f1f680d06a0 msgr2=0x7f1f68157f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.858+0000 7f1f6fd55640 1 -- 172.21.15.47:0/982894520 shutdown_connections 2024-03-23T09:52:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:47.858+0000 7f1f6fd55640 1 -- 172.21.15.47:0/982894520 wait complete. 2024-03-23T09:52:48.870 INFO:tasks.ceph:Scrubbing osd.0 2024-03-23T09:52:48.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-23T09:52:48.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.990+0000 7fe291120640 1 Processor -- start 2024-03-23T09:52:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.990+0000 7fe291120640 1 -- start start 2024-03-23T09:52:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe291120640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c15d0c0 0x7fe28c15d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe291120640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe28c0d27f0 0x7fe28c0d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe291120640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe28c0d3160 con 0x7fe28c15a700 2024-03-23T09:52:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe291120640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe28c15aae0 con 0x7fe28c0d27f0 2024-03-23T09:52:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe291120640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c15dbd0 con 0x7fe28c15d0c0 2024-03-23T09:52:48.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe28ad9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe28c15a700 0x7fe28c0cfb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38038/0 (socket says 172.21.15.47:38038) 2024-03-23T09:52:48.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe28ad9d640 1 -- 172.21.15.47:0/3272555354 learned_addr learned my addr 172.21.15.47:0/3272555354 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:48.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe28b59e640 1 --2- 172.21.15.47:0/3272555354 >> v2:172.21.15.175:3300/0 conn(0x7fe28c0d27f0 0x7fe28c0d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:48.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe28a59c640 1 --2- 172.21.15.47:0/3272555354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c15d0c0 0x7fe28c15d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:48.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387221449 0 0) 0x7fe28c0d3160 con 0x7fe28c15a700 2024-03-23T09:52:48.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe278003520 con 0x7fe28c15a700 2024-03-23T09:52:48.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2331369867 0 0) 0x7fe278003520 con 0x7fe28c15a700 2024-03-23T09:52:48.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c15d0c0 msgr2=0x7fe28c15d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:48.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe289d9b640 1 --2- 172.21.15.47:0/3272555354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c15d0c0 0x7fe28c15d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:48.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 >> v2:172.21.15.175:3300/0 conn(0x7fe28c0d27f0 msgr2=0x7fe28c0d2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:48.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe289d9b640 1 --2- 172.21.15.47:0/3272555354 >> v2:172.21.15.175:3300/0 conn(0x7fe28c0d27f0 0x7fe28c0d2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:48.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.991+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c15deb0 con 0x7fe28c15a700 2024-03-23T09:52:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7fe284001670 con 0x7fe28c15a700 2024-03-23T09:52:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe28b59e640 1 --2- 172.21.15.47:0/3272555354 >> v2:172.21.15.175:3300/0 conn(0x7fe28c0d27f0 0x7fe28c0d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:52:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe28a59c640 1 --2- 172.21.15.47:0/3272555354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c15d0c0 0x7fe28c15d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:52:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 >> v1:172.21.15.47:6789/0 conn(0x7fe28c15a700 legacy=0x7fe28c0cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe289d9b640 1 --2- 172.21.15.47:0/3272555354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c15d0c0 0x7fe278003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe289d9b640 1 --2- 172.21.15.47:0/3272555354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe278003eb0 0x7fe2780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe289d9b640 1 --2- 172.21.15.47:0/3272555354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe278004d00 0x7fe278005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c15aae0 con 0x7fe278004d00 2024-03-23T09:52:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c15dbd0 con 0x7fe278003eb0 2024-03-23T09:52:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27800b980 con 0x7fe28c15d0c0 2024-03-23T09:52:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe28b59e640 1 --2- 172.21.15.47:0/3272555354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe278004d00 0x7fe278005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe28ad9d640 1 --2- 172.21.15.47:0/3272555354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c15d0c0 0x7fe278003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe28a59c640 1 --2- 172.21.15.47:0/3272555354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe278003eb0 0x7fe2780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe28b59e640 1 -- 172.21.15.47:0/3272555354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c15d0c0 msgr2=0x7fe278003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe28b59e640 1 --2- 172.21.15.47:0/3272555354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c15d0c0 0x7fe278003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe28b59e640 1 -- 172.21.15.47:0/3272555354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe278003eb0 msgr2=0x7fe2780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe28b59e640 1 --2- 172.21.15.47:0/3272555354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe278003eb0 0x7fe2780047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.992+0000 7fe28b59e640 1 -- 172.21.15.47:0/3272555354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe274009400 con 0x7fe278004d00 2024-03-23T09:52:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.993+0000 7fe28b59e640 1 --2- 172.21.15.47:0/3272555354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe278004d00 0x7fe278005180 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00e350 tx=0x7fe2780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9e31edd74a0a8a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.993+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c02ce00 con 0x7fe278004d00 2024-03-23T09:52:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.993+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c00f040 con 0x7fe278004d00 2024-03-23T09:52:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.993+0000 7fe289d9b640 1 -- 172.21.15.47:0/3272555354 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c034640 con 0x7fe278004d00 2024-03-23T09:52:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.993+0000 7fe291120640 1 -- 172.21.15.47:0/3272555354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe278004d00 msgr2=0x7fe278005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.993+0000 7fe291120640 1 --2- 172.21.15.47:0/3272555354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe278004d00 0x7fe278005180 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00e350 tx=0x7fe2780057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.993+0000 7fe28b59e640 1 -- 172.21.15.47:0/3272555354 reap_dead start 2024-03-23T09:52:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.994+0000 7fe291120640 1 -- 172.21.15.47:0/3272555354 shutdown_connections 2024-03-23T09:52:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.994+0000 7fe291120640 1 -- 172.21.15.47:0/3272555354 >> 172.21.15.47:0/3272555354 conn(0x7fe28c06afd0 msgr2=0x7fe28c06b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.994+0000 7fe291120640 1 -- 172.21.15.47:0/3272555354 shutdown_connections 2024-03-23T09:52:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.994+0000 7fe291120640 1 -- 172.21.15.47:0/3272555354 wait complete. 2024-03-23T09:52:49.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.995+0000 7fe291120640 1 Processor -- start 2024-03-23T09:52:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.995+0000 7fe291120640 1 -- start start 2024-03-23T09:52:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe291120640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe28c1f1de0 0x7fe28c158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe291120640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe28c15e1f0 0x7fe28c0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe291120640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c0b3710 0x7fe28c156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe291120640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c0b3ba0 con 0x7fe28c15e1f0 2024-03-23T09:52:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe291120640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c156670 con 0x7fe28c1f1de0 2024-03-23T09:52:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe291120640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c1f21c0 con 0x7fe28c0b3710 2024-03-23T09:52:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe28a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe28c15e1f0 0x7fe28c0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe28a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe28c15e1f0 0x7fe28c0b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40150/0 (socket says 172.21.15.47:40150) 2024-03-23T09:52:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe28a59c640 1 -- 172.21.15.47:0/223763081 learned_addr learned my addr 172.21.15.47:0/223763081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe28ad9d640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe28c1f1de0 0x7fe28c158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe28b59e640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c0b3710 0x7fe28c156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe28a59c640 1 -- 172.21.15.47:0/223763081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c0b3710 msgr2=0x7fe28c156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe28a59c640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c0b3710 0x7fe28c156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe28a59c640 1 -- 172.21.15.47:0/223763081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe28c1f1de0 msgr2=0x7fe28c158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe28a59c640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe28c1f1de0 0x7fe28c158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe28a59c640 1 -- 172.21.15.47:0/223763081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c0cb180 con 0x7fe28c15e1f0 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.996+0000 7fe28b59e640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe28c0b3710 0x7fe28c156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.997+0000 7fe28a59c640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe28c15e1f0 0x7fe28c0b31d0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fe274011ac0 tx=0x7fe274011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.997+0000 7fe2837fe640 1 -- 172.21.15.47:0/223763081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27400fb50 con 0x7fe28c15e1f0 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.997+0000 7fe2837fe640 1 -- 172.21.15.47:0/223763081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27400fcb0 con 0x7fe28c15e1f0 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.997+0000 7fe291120640 1 -- 172.21.15.47:0/223763081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28c15e680 con 0x7fe28c15e1f0 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.997+0000 7fe2837fe640 1 -- 172.21.15.47:0/223763081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274004020 con 0x7fe28c15e1f0 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.997+0000 7fe291120640 1 -- 172.21.15.47:0/223763081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28c15ec40 con 0x7fe28c15e1f0 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.998+0000 7fe291120640 1 -- 172.21.15.47:0/223763081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe28c0cf000 con 0x7fe28c15e1f0 2024-03-23T09:52:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.998+0000 7fe2837fe640 1 -- 172.21.15.47:0/223763081 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe274004180 con 0x7fe28c15e1f0 2024-03-23T09:52:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.999+0000 7fe2837fe640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe26c06ac00 0x7fe26c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.999+0000 7fe2837fe640 1 -- 172.21.15.47:0/223763081 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fe2740897b0 con 0x7fe28c15e1f0 2024-03-23T09:52:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.999+0000 7fe2837fe640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x7fe26c070ba0 0x7fe26c072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.999+0000 7fe2837fe640 1 -- 172.21.15.47:0/223763081 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe26c073670 con 0x7fe26c070ba0 2024-03-23T09:52:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.999+0000 7fe2837fe640 1 -- 172.21.15.47:0/223763081 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe27408be70 con 0x7fe28c15e1f0 2024-03-23T09:52:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.999+0000 7fe28ad9d640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe26c06ac00 0x7fe26c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.999+0000 7fe28b59e640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x7fe26c070ba0 0x7fe26c072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.999+0000 7fe28b59e640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x7fe26c070ba0 0x7fe26c072fc0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:48.999+0000 7fe28ad9d640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe26c06ac00 0x7fe26c06d0c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe284007e50 tx=0x7fe2840078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.000+0000 7fe2837fe640 1 -- 172.21.15.47:0/223763081 <== osd.0 v2:172.21.15.47:6816/37679 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fe26c073670 con 0x7fe26c070ba0 2024-03-23T09:52:49.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.013+0000 7fe291120640 1 -- 172.21.15.47:0/223763081 --> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe28c0cf620 con 0x7fe26c070ba0 2024-03-23T09:52:49.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.013+0000 7fe2837fe640 1 -- 172.21.15.47:0/223763081 <== osd.0 v2:172.21.15.47:6816/37679 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe27c034660 con 0x7fe26c070ba0 2024-03-23T09:52:49.013 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:52:49.013 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T09:52:49.013 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:52:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.013+0000 7fe291120640 1 -- 172.21.15.47:0/223763081 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x7fe26c070ba0 msgr2=0x7fe26c072fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.013+0000 7fe291120640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.47:6816/37679,v1:172.21.15.47:6817/37679] conn(0x7fe26c070ba0 0x7fe26c072fc0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.013+0000 7fe291120640 1 -- 172.21.15.47:0/223763081 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe26c06ac00 msgr2=0x7fe26c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.013+0000 7fe291120640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fe26c06ac00 0x7fe26c06d0c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe284007e50 tx=0x7fe2840078f0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.013+0000 7fe291120640 1 -- 172.21.15.47:0/223763081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe28c15e1f0 msgr2=0x7fe28c0b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.013+0000 7fe291120640 1 --2- 172.21.15.47:0/223763081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe28c15e1f0 0x7fe28c0b31d0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fe274011ac0 tx=0x7fe274011ec0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.013+0000 7fe28b59e640 1 -- 172.21.15.47:0/223763081 reap_dead start 2024-03-23T09:52:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.013+0000 7fe291120640 1 -- 172.21.15.47:0/223763081 shutdown_connections 2024-03-23T09:52:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.013+0000 7fe291120640 1 -- 172.21.15.47:0/223763081 >> 172.21.15.47:0/223763081 conn(0x7fe28c06afd0 msgr2=0x7fe28c15c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.013+0000 7fe291120640 1 -- 172.21.15.47:0/223763081 shutdown_connections 2024-03-23T09:52:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.013+0000 7fe291120640 1 -- 172.21.15.47:0/223763081 wait complete. 2024-03-23T09:52:49.024 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-23T09:52:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.126+0000 7f8d5b4df640 1 Processor -- start 2024-03-23T09:52:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.126+0000 7f8d5b4df640 1 -- start start 2024-03-23T09:52:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.127+0000 7f8d5b4df640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d540b8350 0x7f8d540cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.127+0000 7f8d5b4df640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d540cdc40 0x7f8d540d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.127+0000 7f8d5b4df640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d540d11e0 con 0x7f8d5415ca90 2024-03-23T09:52:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.127+0000 7f8d5b4df640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540ce0d0 con 0x7f8d540b8350 2024-03-23T09:52:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.127+0000 7f8d5b4df640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d54160f60 con 0x7f8d540cdc40 2024-03-23T09:52:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.127+0000 7f8d5927b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8d5415ca90 0x7f8d54160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38052/0 (socket says 172.21.15.47:38052) 2024-03-23T09:52:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.127+0000 7f8d5927b640 1 -- 172.21.15.47:0/842783874 learned_addr learned my addr 172.21.15.47:0/842783874 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.127+0000 7f8d58a7a640 1 --2- 172.21.15.47:0/842783874 >> v2:172.21.15.47:3300/0 conn(0x7f8d540b8350 0x7f8d540cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.128+0000 7f8d58a7a640 1 -- 172.21.15.47:0/842783874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d540cdc40 msgr2=0x7f8d540d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.128+0000 7f8d59a7c640 1 --2- 172.21.15.47:0/842783874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d540cdc40 0x7f8d540d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.128+0000 7f8d58a7a640 1 --2- 172.21.15.47:0/842783874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d540cdc40 0x7f8d540d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.128+0000 7f8d58a7a640 1 -- 172.21.15.47:0/842783874 >> v1:172.21.15.47:6789/0 conn(0x7f8d5415ca90 legacy=0x7f8d54160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.128+0000 7f8d4bfff640 1 -- 172.21.15.47:0/842783874 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273237883 0 0) 0x7f8d540d11e0 con 0x7f8d5415ca90 2024-03-23T09:52:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.128+0000 7f8d58a7a640 1 -- 172.21.15.47:0/842783874 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d54161240 con 0x7f8d540b8350 2024-03-23T09:52:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.128+0000 7f8d58a7a640 1 --2- 172.21.15.47:0/842783874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d540b8350 0x7f8d540cd700 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f8d440097a0 tx=0x7f8d4402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80df1ea8ed3165f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.129+0000 7f8d4bfff640 1 -- 172.21.15.47:0/842783874 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4402e070 con 0x7f8d540b8350 2024-03-23T09:52:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.129+0000 7f8d4bfff640 1 -- 172.21.15.47:0/842783874 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d440315a0 con 0x7f8d540b8350 2024-03-23T09:52:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.129+0000 7f8d4bfff640 1 -- 172.21.15.47:0/842783874 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44031930 con 0x7f8d540b8350 2024-03-23T09:52:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.129+0000 7f8d5b4df640 1 -- 172.21.15.47:0/842783874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d540b8350 msgr2=0x7f8d540cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.129+0000 7f8d5b4df640 1 --2- 172.21.15.47:0/842783874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d540b8350 0x7f8d540cd700 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f8d440097a0 tx=0x7f8d4402b840 comp rx=0 tx=0).stop 2024-03-23T09:52:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.129+0000 7f8d5b4df640 1 -- 172.21.15.47:0/842783874 shutdown_connections 2024-03-23T09:52:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.129+0000 7f8d5b4df640 1 --2- 172.21.15.47:0/842783874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d540cdc40 0x7f8d540d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.129+0000 7f8d5b4df640 1 --2- 172.21.15.47:0/842783874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d540b8350 0x7f8d540cd700 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.129+0000 7f8d5b4df640 1 -- 172.21.15.47:0/842783874 >> 172.21.15.47:0/842783874 conn(0x7f8d540659d0 msgr2=0x7f8d54065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.129+0000 7f8d5b4df640 1 -- 172.21.15.47:0/842783874 shutdown_connections 2024-03-23T09:52:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.129+0000 7f8d5b4df640 1 -- 172.21.15.47:0/842783874 wait complete. 2024-03-23T09:52:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.130+0000 7f8d5b4df640 1 Processor -- start 2024-03-23T09:52:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.130+0000 7f8d5b4df640 1 -- start start 2024-03-23T09:52:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d5b4df640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d540b8350 0x7f8d540cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d5b4df640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d540cdc40 0x7f8d540b7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d5b4df640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d5415ca90 0x7f8d540b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d5b4df640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540ca540 con 0x7f8d5415ca90 2024-03-23T09:52:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d5b4df640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d541f68c0 con 0x7f8d540b8350 2024-03-23T09:52:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d5b4df640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540c9ba0 con 0x7f8d540cdc40 2024-03-23T09:52:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d59a7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d5415ca90 0x7f8d540b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d59a7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d5415ca90 0x7f8d540b7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40166/0 (socket says 172.21.15.47:40166) 2024-03-23T09:52:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d59a7c640 1 -- 172.21.15.47:0/3203187851 learned_addr learned my addr 172.21.15.47:0/3203187851 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d59a7c640 1 -- 172.21.15.47:0/3203187851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d540cdc40 msgr2=0x7f8d540b7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d59a7c640 1 --2- 172.21.15.47:0/3203187851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d540cdc40 0x7f8d540b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d59a7c640 1 -- 172.21.15.47:0/3203187851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d540b8350 msgr2=0x7f8d540cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d59a7c640 1 --2- 172.21.15.47:0/3203187851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d540b8350 0x7f8d540cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d59a7c640 1 -- 172.21.15.47:0/3203187851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d44009400 con 0x7f8d5415ca90 2024-03-23T09:52:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.131+0000 7f8d59a7c640 1 --2- 172.21.15.47:0/3203187851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d5415ca90 0x7f8d540b7990 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000cc30 tx=0x7f8d5000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.132+0000 7f8d49ffb640 1 -- 172.21.15.47:0/3203187851 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5000ce40 con 0x7f8d5415ca90 2024-03-23T09:52:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.132+0000 7f8d49ffb640 1 -- 172.21.15.47:0/3203187851 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d50003990 con 0x7f8d5415ca90 2024-03-23T09:52:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.132+0000 7f8d5b4df640 1 -- 172.21.15.47:0/3203187851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d540c9e80 con 0x7f8d5415ca90 2024-03-23T09:52:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.132+0000 7f8d49ffb640 1 -- 172.21.15.47:0/3203187851 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50003d20 con 0x7f8d5415ca90 2024-03-23T09:52:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.132+0000 7f8d5b4df640 1 -- 172.21.15.47:0/3203187851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d540cc8a0 con 0x7f8d5415ca90 2024-03-23T09:52:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.133+0000 7f8d5b4df640 1 -- 172.21.15.47:0/3203187851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d54072360 con 0x7f8d5415ca90 2024-03-23T09:52:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.133+0000 7f8d49ffb640 1 -- 172.21.15.47:0/3203187851 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8d50003e80 con 0x7f8d5415ca90 2024-03-23T09:52:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.134+0000 7f8d49ffb640 1 --2- 172.21.15.47:0/3203187851 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d3806ac00 0x7f8d3806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.134+0000 7f8d49ffb640 1 -- 172.21.15.47:0/3203187851 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f8d500894d0 con 0x7f8d5415ca90 2024-03-23T09:52:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.134+0000 7f8d5927b640 1 --2- 172.21.15.47:0/3203187851 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d3806ac00 0x7f8d3806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.134+0000 7f8d5927b640 1 --2- 172.21.15.47:0/3203187851 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d3806ac00 0x7f8d3806d0c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c007fd0 tx=0x7f8d4c007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.138+0000 7f8d49ffb640 1 -- 172.21.15.47:0/3203187851 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8d50054900 con 0x7f8d5415ca90 2024-03-23T09:52:49.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.315+0000 7f8d5b4df640 1 -- 172.21.15.47:0/3203187851 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d540d0e70 con 0x7f8d3806ac00 2024-03-23T09:52:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.316+0000 7f8d49ffb640 1 -- 172.21.15.47:0/3203187851 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8d4c0070d0 con 0x7f8d3806ac00 2024-03-23T09:52:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 0 to deep-scrub 2024-03-23T09:52:49.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.317+0000 7f8d5b4df640 1 -- 172.21.15.47:0/3203187851 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d3806ac00 msgr2=0x7f8d3806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.317+0000 7f8d5b4df640 1 --2- 172.21.15.47:0/3203187851 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d3806ac00 0x7f8d3806d0c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c007fd0 tx=0x7f8d4c007920 comp rx=0 tx=0).stop 2024-03-23T09:52:49.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.317+0000 7f8d5b4df640 1 -- 172.21.15.47:0/3203187851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d5415ca90 msgr2=0x7f8d540b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.318+0000 7f8d5b4df640 1 --2- 172.21.15.47:0/3203187851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d5415ca90 0x7f8d540b7990 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000cc30 tx=0x7f8d5000a5c0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.318+0000 7f8d5b4df640 1 -- 172.21.15.47:0/3203187851 shutdown_connections 2024-03-23T09:52:49.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.318+0000 7f8d5b4df640 1 --2- 172.21.15.47:0/3203187851 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d3806ac00 0x7f8d3806d0c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.318+0000 7f8d5b4df640 1 --2- 172.21.15.47:0/3203187851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d5415ca90 0x7f8d540b7990 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.318+0000 7f8d5b4df640 1 --2- 172.21.15.47:0/3203187851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d540cdc40 0x7f8d540b7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.318+0000 7f8d5b4df640 1 --2- 172.21.15.47:0/3203187851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d540b8350 0x7f8d540cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.318+0000 7f8d5b4df640 1 -- 172.21.15.47:0/3203187851 >> 172.21.15.47:0/3203187851 conn(0x7f8d540659d0 msgr2=0x7f8d54160610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:49.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.318+0000 7f8d5b4df640 1 -- 172.21.15.47:0/3203187851 shutdown_connections 2024-03-23T09:52:49.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.318+0000 7f8d5b4df640 1 -- 172.21.15.47:0/3203187851 wait complete. 2024-03-23T09:52:49.328 INFO:tasks.ceph:Scrubbing osd.1 2024-03-23T09:52:49.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-23T09:52:49.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.434+0000 7f7e26a36640 1 Processor -- start 2024-03-23T09:52:49.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.434+0000 7f7e26a36640 1 -- start start 2024-03-23T09:52:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.434+0000 7f7e26a36640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7e2015a860 0x7f7e20161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e26a36640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e20160bd0 con 0x7f7e200cf8a0 2024-03-23T09:52:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e26a36640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2015dbb0 con 0x7f7e2015d0f0 2024-03-23T09:52:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e26a36640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e200d2790 con 0x7f7e2015a860 2024-03-23T09:52:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e24fd3640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7e200cf8a0 0x7f7e200d0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38054/0 (socket says 172.21.15.47:38054) 2024-03-23T09:52:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e24fd3640 1 -- 172.21.15.47:0/3824591146 learned_addr learned my addr 172.21.15.47:0/3824591146 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1ffff640 1 --2- 172.21.15.47:0/3824591146 >> v2:172.21.15.175:3300/0 conn(0x7f7e2015a860 0x7f7e20161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312857346 0 0) 0x7f7e20160bd0 con 0x7f7e200cf8a0 2024-03-23T09:52:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e08003520 con 0x7f7e200cf8a0 2024-03-23T09:52:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2137940640 0 0) 0x7f7e08003520 con 0x7f7e200cf8a0 2024-03-23T09:52:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 >> v2:172.21.15.175:3300/0 conn(0x7f7e2015a860 msgr2=0x7f7e20161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 --2- 172.21.15.47:0/3824591146 >> v2:172.21.15.175:3300/0 conn(0x7f7e2015a860 0x7f7e20161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 >> v1:172.21.15.175:6789/0 conn(0x7f7e2015d0f0 legacy=0x7f7e200cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2015de90 con 0x7f7e200cf8a0 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f7e180019f0 con 0x7f7e200cf8a0 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1ffff640 1 --2- 172.21.15.47:0/3824591146 >> v2:172.21.15.175:3300/0 conn(0x7f7e2015a860 0x7f7e20161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 >> v1:172.21.15.47:6789/0 conn(0x7f7e200cf8a0 legacy=0x7f7e200d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 --2- 172.21.15.47:0/3824591146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e08003970 0x7f7e08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 --2- 172.21.15.47:0/3824591146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e080047c0 0x7f7e08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 --2- 172.21.15.47:0/3824591146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e08005180 0x7f7e0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200d2790 con 0x7f7e08003970 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0800c020 con 0x7f7e080047c0 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0800c1e0 con 0x7f7e08005180 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.435+0000 7f7e1ffff640 1 --2- 172.21.15.47:0/3824591146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e08003970 0x7f7e08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.436+0000 7f7e1f7fe640 1 --2- 172.21.15.47:0/3824591146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e080047c0 0x7f7e08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.436+0000 7f7e24fd3640 1 --2- 172.21.15.47:0/3824591146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e08005180 0x7f7e0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.436+0000 7f7e1ffff640 1 -- 172.21.15.47:0/3824591146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e08005180 msgr2=0x7f7e0800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.436+0000 7f7e1ffff640 1 --2- 172.21.15.47:0/3824591146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e08005180 0x7f7e0800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.436+0000 7f7e1ffff640 1 -- 172.21.15.47:0/3824591146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e080047c0 msgr2=0x7f7e08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.436+0000 7f7e1ffff640 1 --2- 172.21.15.47:0/3824591146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e080047c0 0x7f7e08004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.436+0000 7f7e1ffff640 1 -- 172.21.15.47:0/3824591146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e08003520 con 0x7f7e08003970 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.436+0000 7f7e1f7fe640 1 --2- 172.21.15.47:0/3824591146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e080047c0 0x7f7e08004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.436+0000 7f7e24fd3640 1 -- 172.21.15.47:0/3824591146 reap_dead start 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.436+0000 7f7e1ffff640 1 --2- 172.21.15.47:0/3824591146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e08003970 0x7f7e08003d90 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009c90 tx=0x7f7e080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe2abf0f583d216 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.437+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1002c8f0 con 0x7f7e08003970 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.437+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1002ca50 con 0x7f7e08003970 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.437+0000 7f7e1effd640 1 -- 172.21.15.47:0/3824591146 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1002ce40 con 0x7f7e08003970 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.437+0000 7f7e26a36640 1 -- 172.21.15.47:0/3824591146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e08003970 msgr2=0x7f7e08003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.437+0000 7f7e26a36640 1 --2- 172.21.15.47:0/3824591146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e08003970 0x7f7e08003d90 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009c90 tx=0x7f7e080057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.437+0000 7f7e26a36640 1 -- 172.21.15.47:0/3824591146 shutdown_connections 2024-03-23T09:52:49.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.437+0000 7f7e26a36640 1 --2- 172.21.15.47:0/3824591146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e08003970 0x7f7e08003d90 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.437+0000 7f7e26a36640 1 -- 172.21.15.47:0/3824591146 >> 172.21.15.47:0/3824591146 conn(0x7f7e200659d0 msgr2=0x7f7e20065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:49.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.437+0000 7f7e26a36640 1 -- 172.21.15.47:0/3824591146 shutdown_connections 2024-03-23T09:52:49.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.437+0000 7f7e26a36640 1 -- 172.21.15.47:0/3824591146 wait complete. 2024-03-23T09:52:49.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.438+0000 7f7e26a36640 1 Processor -- start 2024-03-23T09:52:49.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.439+0000 7f7e26a36640 1 -- start start 2024-03-23T09:52:49.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.439+0000 7f7e26a36640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e08003970 0x7f7e200b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.439+0000 7f7e26a36640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e201f6cb0 0x7f7e201503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.439+0000 7f7e26a36640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e200b5b90 0x7f7e200b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.439+0000 7f7e26a36640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20151f20 con 0x7f7e08003970 2024-03-23T09:52:49.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.439+0000 7f7e26a36640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200b80e0 con 0x7f7e201f6cb0 2024-03-23T09:52:49.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.439+0000 7f7e26a36640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20152150 con 0x7f7e200b5b90 2024-03-23T09:52:49.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.439+0000 7f7e1ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e08003970 0x7f7e200b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.439+0000 7f7e1ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e08003970 0x7f7e200b6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40178/0 (socket says 172.21.15.47:40178) 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.439+0000 7f7e1ffff640 1 -- 172.21.15.47:0/1869997419 learned_addr learned my addr 172.21.15.47:0/1869997419 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.439+0000 7f7e1f7fe640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e201f6cb0 0x7f7e201503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.440+0000 7f7e24fd3640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e200b5b90 0x7f7e200b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.440+0000 7f7e1ffff640 1 -- 172.21.15.47:0/1869997419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e200b5b90 msgr2=0x7f7e200b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.440+0000 7f7e1ffff640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e200b5b90 0x7f7e200b6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.440+0000 7f7e1ffff640 1 -- 172.21.15.47:0/1869997419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e201f6cb0 msgr2=0x7f7e201503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.440+0000 7f7e1ffff640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e201f6cb0 0x7f7e201503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.440+0000 7f7e1ffff640 1 -- 172.21.15.47:0/1869997419 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e200b7b20 con 0x7f7e08003970 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.440+0000 7f7e24fd3640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e200b5b90 0x7f7e200b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.440+0000 7f7e1ffff640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e08003970 0x7f7e200b6c70 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f7e10037040 tx=0x7f7e10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.440+0000 7f7e1cff9640 1 -- 172.21.15.47:0/1869997419 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10034a80 con 0x7f7e08003970 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.440+0000 7f7e1cff9640 1 -- 172.21.15.47:0/1869997419 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e10034be0 con 0x7f7e08003970 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.440+0000 7f7e26a36640 1 -- 172.21.15.47:0/1869997419 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e200b7e00 con 0x7f7e08003970 2024-03-23T09:52:49.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.440+0000 7f7e1cff9640 1 -- 172.21.15.47:0/1869997419 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10004020 con 0x7f7e08003970 2024-03-23T09:52:49.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.440+0000 7f7e26a36640 1 -- 172.21.15.47:0/1869997419 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e200b71b0 con 0x7f7e08003970 2024-03-23T09:52:49.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.441+0000 7f7e26a36640 1 -- 172.21.15.47:0/1869997419 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7e2015d580 con 0x7f7e08003970 2024-03-23T09:52:49.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.441+0000 7f7e1cff9640 1 -- 172.21.15.47:0/1869997419 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7e10004180 con 0x7f7e08003970 2024-03-23T09:52:49.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.442+0000 7f7e1cff9640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e0c06ac00 0x7f7e0c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.442+0000 7f7e1cff9640 1 -- 172.21.15.47:0/1869997419 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f7e100ab400 con 0x7f7e08003970 2024-03-23T09:52:49.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.442+0000 7f7e1cff9640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x7f7e0c070ba0 0x7f7e0c072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.442+0000 7f7e1cff9640 1 -- 172.21.15.47:0/1869997419 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e0c073670 con 0x7f7e0c070ba0 2024-03-23T09:52:49.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.442+0000 7f7e1cff9640 1 -- 172.21.15.47:0/1869997419 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e100adac0 con 0x7f7e08003970 2024-03-23T09:52:49.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.442+0000 7f7e1f7fe640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e0c06ac00 0x7f7e0c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.442+0000 7f7e24fd3640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x7f7e0c070ba0 0x7f7e0c072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.442+0000 7f7e24fd3640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x7f7e0c070ba0 0x7f7e0c072fc0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.443+0000 7f7e1f7fe640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e0c06ac00 0x7f7e0c06d0c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7e14000c00 tx=0x7f7e14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.443+0000 7f7e1cff9640 1 -- 172.21.15.47:0/1869997419 <== osd.1 v2:172.21.15.47:6824/37680 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f7e0c073670 con 0x7f7e0c070ba0 2024-03-23T09:52:49.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.455+0000 7f7e26a36640 1 -- 172.21.15.47:0/1869997419 --> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7e20072360 con 0x7f7e0c070ba0 2024-03-23T09:52:49.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.456+0000 7f7e1cff9640 1 -- 172.21.15.47:0/1869997419 <== osd.1 v2:172.21.15.47:6824/37680 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7e20072360 con 0x7f7e0c070ba0 2024-03-23T09:52:49.456 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:52:49.456 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T09:52:49.456 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:52:49.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.456+0000 7f7e26a36640 1 -- 172.21.15.47:0/1869997419 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x7f7e0c070ba0 msgr2=0x7f7e0c072fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.456+0000 7f7e26a36640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.47:6824/37680,v1:172.21.15.47:6825/37680] conn(0x7f7e0c070ba0 0x7f7e0c072fc0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.456+0000 7f7e26a36640 1 -- 172.21.15.47:0/1869997419 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e0c06ac00 msgr2=0x7f7e0c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.456+0000 7f7e26a36640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f7e0c06ac00 0x7f7e0c06d0c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7e14000c00 tx=0x7f7e14009040 comp rx=0 tx=0).stop 2024-03-23T09:52:49.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.456+0000 7f7e26a36640 1 -- 172.21.15.47:0/1869997419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e08003970 msgr2=0x7f7e200b6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.456+0000 7f7e26a36640 1 --2- 172.21.15.47:0/1869997419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e08003970 0x7f7e200b6c70 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f7e10037040 tx=0x7f7e10004730 comp rx=0 tx=0).stop 2024-03-23T09:52:49.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.456+0000 7f7e24fd3640 1 -- 172.21.15.47:0/1869997419 reap_dead start 2024-03-23T09:52:49.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.456+0000 7f7e26a36640 1 -- 172.21.15.47:0/1869997419 shutdown_connections 2024-03-23T09:52:49.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.456+0000 7f7e26a36640 1 -- 172.21.15.47:0/1869997419 >> 172.21.15.47:0/1869997419 conn(0x7f7e200659d0 msgr2=0x7f7e2015c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:49.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.456+0000 7f7e26a36640 1 -- 172.21.15.47:0/1869997419 shutdown_connections 2024-03-23T09:52:49.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.456+0000 7f7e26a36640 1 -- 172.21.15.47:0/1869997419 wait complete. 2024-03-23T09:52:49.466 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-23T09:52:49.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.568+0000 7f48e944b640 1 Processor -- start 2024-03-23T09:52:49.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.568+0000 7f48e944b640 1 -- start start 2024-03-23T09:52:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.569+0000 7f48e944b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e40d16b0 0x7f48e40d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.569+0000 7f48e944b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f48e40d0fa0 0x7f48e40ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.569+0000 7f48e944b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e41658a0 con 0x7f48e4160ed0 2024-03-23T09:52:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.569+0000 7f48e944b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e40d1420 con 0x7f48e40d0fa0 2024-03-23T09:52:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.569+0000 7f48e944b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40d2820 con 0x7f48e40d16b0 2024-03-23T09:52:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.569+0000 7f48e37fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f48e40d0fa0 0x7f48e40ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.569+0000 7f48e2ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f48e4160ed0 0x7f48e40d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38056/0 (socket says 172.21.15.47:38056) 2024-03-23T09:52:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.569+0000 7f48e2ffd640 1 -- 172.21.15.47:0/4117418973 learned_addr learned my addr 172.21.15.47:0/4117418973 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.570+0000 7f48e37fe640 1 -- 172.21.15.47:0/4117418973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e40d16b0 msgr2=0x7f48e40d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.570+0000 7f48e37fe640 1 --2- 172.21.15.47:0/4117418973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e40d16b0 0x7f48e40d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.570+0000 7f48e37fe640 1 -- 172.21.15.47:0/4117418973 >> v1:172.21.15.47:6789/0 conn(0x7f48e4160ed0 legacy=0x7f48e40d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.570+0000 7f48e37fe640 1 -- 172.21.15.47:0/4117418973 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e4164c90 con 0x7f48e40d0fa0 2024-03-23T09:52:49.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.570+0000 7f48e1ffb640 1 -- 172.21.15.47:0/4117418973 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433259187 0 0) 0x7f48e41658a0 con 0x7f48e4160ed0 2024-03-23T09:52:49.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.570+0000 7f48e37fe640 1 --2- 172.21.15.47:0/4117418973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e40d0fa0 0x7f48e40ced80 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009d90 tx=0x7f48d402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1890759f062f7e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.571+0000 7f48e1ffb640 1 -- 172.21.15.47:0/4117418973 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d402cc50 con 0x7f48e40d0fa0 2024-03-23T09:52:49.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.571+0000 7f48e1ffb640 1 -- 172.21.15.47:0/4117418973 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d402cdb0 con 0x7f48e40d0fa0 2024-03-23T09:52:49.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.571+0000 7f48e1ffb640 1 -- 172.21.15.47:0/4117418973 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4034660 con 0x7f48e40d0fa0 2024-03-23T09:52:49.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.571+0000 7f48e944b640 1 -- 172.21.15.47:0/4117418973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e40d0fa0 msgr2=0x7f48e40ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.571+0000 7f48e944b640 1 --2- 172.21.15.47:0/4117418973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e40d0fa0 0x7f48e40ced80 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009d90 tx=0x7f48d402bea0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.571+0000 7f48e944b640 1 -- 172.21.15.47:0/4117418973 shutdown_connections 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.571+0000 7f48e944b640 1 --2- 172.21.15.47:0/4117418973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e40d0fa0 0x7f48e40ced80 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.571+0000 7f48e944b640 1 --2- 172.21.15.47:0/4117418973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e40d16b0 0x7f48e40d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.571+0000 7f48e944b640 1 -- 172.21.15.47:0/4117418973 >> 172.21.15.47:0/4117418973 conn(0x7f48e40659d0 msgr2=0x7f48e4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.571+0000 7f48e944b640 1 -- 172.21.15.47:0/4117418973 shutdown_connections 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.571+0000 7f48e944b640 1 -- 172.21.15.47:0/4117418973 wait complete. 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.572+0000 7f48e944b640 1 Processor -- start 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.572+0000 7f48e944b640 1 -- start start 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.573+0000 7f48e944b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e40d0fa0 0x7f48e4159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.573+0000 7f48e944b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e40d16b0 0x7f48e40c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.573+0000 7f48e944b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48e4160ed0 0x7f48e415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.573+0000 7f48e944b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e41550c0 con 0x7f48e40d16b0 2024-03-23T09:52:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.573+0000 7f48e944b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e4156320 con 0x7f48e4160ed0 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.573+0000 7f48e944b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e41565c0 con 0x7f48e40d0fa0 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.573+0000 7f48e27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e40d16b0 0x7f48e40c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.573+0000 7f48e27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e40d16b0 0x7f48e40c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40204/0 (socket says 172.21.15.47:40204) 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.573+0000 7f48e27fc640 1 -- 172.21.15.47:0/3492484348 learned_addr learned my addr 172.21.15.47:0/3492484348 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.573+0000 7f48e2ffd640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e40d0fa0 0x7f48e4159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.573+0000 7f48e37fe640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48e4160ed0 0x7f48e415ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.574+0000 7f48e27fc640 1 -- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e40d0fa0 msgr2=0x7f48e4159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.574+0000 7f48e27fc640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e40d0fa0 0x7f48e4159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.574+0000 7f48e27fc640 1 -- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48e4160ed0 msgr2=0x7f48e415ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.574+0000 7f48e27fc640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48e4160ed0 0x7f48e415ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.574+0000 7f48e27fc640 1 -- 172.21.15.47:0/3492484348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d4009a70 con 0x7f48e40d16b0 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.574+0000 7f48e37fe640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48e4160ed0 0x7f48e415ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.574+0000 7f48e27fc640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e40d16b0 0x7f48e40c91f0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f48d800b450 tx=0x7f48d800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.574+0000 7f48cb7fe640 1 -- 172.21.15.47:0/3492484348 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8011890 con 0x7f48e40d16b0 2024-03-23T09:52:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.574+0000 7f48cb7fe640 1 -- 172.21.15.47:0/3492484348 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d80119f0 con 0x7f48e40d16b0 2024-03-23T09:52:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.574+0000 7f48e944b640 1 -- 172.21.15.47:0/3492484348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e415be20 con 0x7f48e40d16b0 2024-03-23T09:52:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.574+0000 7f48cb7fe640 1 -- 172.21.15.47:0/3492484348 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8011da0 con 0x7f48e40d16b0 2024-03-23T09:52:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.574+0000 7f48e944b640 1 -- 172.21.15.47:0/3492484348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e41568a0 con 0x7f48e40d16b0 2024-03-23T09:52:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.575+0000 7f48e944b640 1 -- 172.21.15.47:0/3492484348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e4072360 con 0x7f48e40d16b0 2024-03-23T09:52:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.575+0000 7f48cb7fe640 1 -- 172.21.15.47:0/3492484348 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f48d801a400 con 0x7f48e40d16b0 2024-03-23T09:52:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.576+0000 7f48cb7fe640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f48d006ac00 0x7f48d006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.576+0000 7f48cb7fe640 1 -- 172.21.15.47:0/3492484348 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f48d8089410 con 0x7f48e40d16b0 2024-03-23T09:52:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.576+0000 7f48e2ffd640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f48d006ac00 0x7f48d006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.576+0000 7f48e2ffd640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f48d006ac00 0x7f48d006d0c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f48dc000f60 tx=0x7f48dc007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.580+0000 7f48cb7fe640 1 -- 172.21.15.47:0/3492484348 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f48d80548d0 con 0x7f48e40d16b0 2024-03-23T09:52:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.761+0000 7f48e944b640 1 -- 172.21.15.47:0/3492484348 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f48e4165530 con 0x7f48d006ac00 2024-03-23T09:52:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.762+0000 7f48cb7fe640 1 -- 172.21.15.47:0/3492484348 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f48dc007270 con 0x7f48d006ac00 2024-03-23T09:52:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 1 to deep-scrub 2024-03-23T09:52:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.764+0000 7f48e944b640 1 -- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f48d006ac00 msgr2=0x7f48d006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.764+0000 7f48e944b640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f48d006ac00 0x7f48d006d0c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f48dc000f60 tx=0x7f48dc007c00 comp rx=0 tx=0).stop 2024-03-23T09:52:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.764+0000 7f48e944b640 1 -- 172.21.15.47:0/3492484348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e40d16b0 msgr2=0x7f48e40c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.764+0000 7f48e944b640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e40d16b0 0x7f48e40c91f0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f48d800b450 tx=0x7f48d800b910 comp rx=0 tx=0).stop 2024-03-23T09:52:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.764+0000 7f48e944b640 1 -- 172.21.15.47:0/3492484348 shutdown_connections 2024-03-23T09:52:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.764+0000 7f48e944b640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f48d006ac00 0x7f48d006d0c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.764+0000 7f48e944b640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48e4160ed0 0x7f48e415ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.764+0000 7f48e944b640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e40d16b0 0x7f48e40c91f0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.764+0000 7f48e944b640 1 --2- 172.21.15.47:0/3492484348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e40d0fa0 0x7f48e4159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.764+0000 7f48e944b640 1 -- 172.21.15.47:0/3492484348 >> 172.21.15.47:0/3492484348 conn(0x7f48e40659d0 msgr2=0x7f48e40b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.764+0000 7f48e944b640 1 -- 172.21.15.47:0/3492484348 shutdown_connections 2024-03-23T09:52:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.764+0000 7f48e944b640 1 -- 172.21.15.47:0/3492484348 wait complete. 2024-03-23T09:52:49.775 INFO:tasks.ceph:Scrubbing osd.2 2024-03-23T09:52:49.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-23T09:52:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.879+0000 7f712fc51640 1 Processor -- start 2024-03-23T09:52:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.880+0000 7f712fc51640 1 -- start start 2024-03-23T09:52:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.880+0000 7f712fc51640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71280cf840 0x7f71280cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.880+0000 7f712fc51640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f71280b5a00 0x7f71280b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.880+0000 7f712fc51640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71280b1d90 con 0x7f712815cb50 2024-03-23T09:52:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.880+0000 7f712fc51640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712815cf30 con 0x7f71280b5a00 2024-03-23T09:52:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.880+0000 7f712fc51640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71280d2330 con 0x7f71280cf840 2024-03-23T09:52:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.880+0000 7f712d9ed640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f712815cb50 0x7f71280cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38072/0 (socket says 172.21.15.47:38072) 2024-03-23T09:52:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712d9ed640 1 -- 172.21.15.47:0/166998300 learned_addr learned my addr 172.21.15.47:0/166998300 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44753958 0 0) 0x7f71280b1d90 con 0x7f712815cb50 2024-03-23T09:52:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712e1ee640 1 --2- 172.21.15.47:0/166998300 >> v2:172.21.15.175:3300/0 conn(0x7f71280b5a00 0x7f71280b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712d1ec640 1 --2- 172.21.15.47:0/166998300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71280cf840 0x7f71280cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7110003520 con 0x7f712815cb50 2024-03-23T09:52:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2277584090 0 0) 0x7f7110003520 con 0x7f712815cb50 2024-03-23T09:52:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71280cf840 msgr2=0x7f71280cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 --2- 172.21.15.47:0/166998300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71280cf840 0x7f71280cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 >> v2:172.21.15.175:3300/0 conn(0x7f71280b5a00 msgr2=0x7f71280b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 --2- 172.21.15.47:0/166998300 >> v2:172.21.15.175:3300/0 conn(0x7f71280b5a00 0x7f71280b5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71280d01e0 con 0x7f712815cb50 2024-03-23T09:52:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f7120001690 con 0x7f712815cb50 2024-03-23T09:52:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 >> v1:172.21.15.47:6789/0 conn(0x7f712815cb50 legacy=0x7f71280cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 --2- 172.21.15.47:0/166998300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71100039a0 0x7f7110003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 --2- 172.21.15.47:0/166998300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71100047c0 0x7f7110004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 --2- 172.21.15.47:0/166998300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71280cf840 0x7f7110005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712815cf30 con 0x7f71100039a0 2024-03-23T09:52:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71280d2330 con 0x7f71100047c0 2024-03-23T09:52:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711000b980 con 0x7f71280cf840 2024-03-23T09:52:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712d9ed640 1 --2- 172.21.15.47:0/166998300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71100039a0 0x7f7110003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712d1ec640 1 --2- 172.21.15.47:0/166998300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71100047c0 0x7f7110004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712e1ee640 1 --2- 172.21.15.47:0/166998300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71280cf840 0x7f7110005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712d9ed640 1 -- 172.21.15.47:0/166998300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71280cf840 msgr2=0x7f7110005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712d9ed640 1 --2- 172.21.15.47:0/166998300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71280cf840 0x7f7110005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712d9ed640 1 -- 172.21.15.47:0/166998300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71100047c0 msgr2=0x7f7110004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.881+0000 7f712d9ed640 1 --2- 172.21.15.47:0/166998300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71100047c0 0x7f7110004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.882+0000 7f712d9ed640 1 -- 172.21.15.47:0/166998300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7110003520 con 0x7f71100039a0 2024-03-23T09:52:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.882+0000 7f712d1ec640 1 --2- 172.21.15.47:0/166998300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71100047c0 0x7f7110004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.882+0000 7f712d9ed640 1 --2- 172.21.15.47:0/166998300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71100039a0 0x7f7110003dc0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f712000c210 tx=0x7f71100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586ca452a77f5a4b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.882+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120005ad0 con 0x7f71100039a0 2024-03-23T09:52:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.882+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7120005c30 con 0x7f71100039a0 2024-03-23T09:52:49.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.882+0000 7f712c9eb640 1 -- 172.21.15.47:0/166998300 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712002c870 con 0x7f71100039a0 2024-03-23T09:52:49.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.883+0000 7f712fc51640 1 -- 172.21.15.47:0/166998300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71100039a0 msgr2=0x7f7110003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.883+0000 7f712fc51640 1 --2- 172.21.15.47:0/166998300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71100039a0 0x7f7110003dc0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f712000c210 tx=0x7f71100057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.883+0000 7f712e1ee640 1 -- 172.21.15.47:0/166998300 reap_dead start 2024-03-23T09:52:49.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.883+0000 7f712fc51640 1 -- 172.21.15.47:0/166998300 shutdown_connections 2024-03-23T09:52:49.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.883+0000 7f712fc51640 1 -- 172.21.15.47:0/166998300 >> 172.21.15.47:0/166998300 conn(0x7f7128065df0 msgr2=0x7f7128066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:49.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.883+0000 7f712fc51640 1 -- 172.21.15.47:0/166998300 shutdown_connections 2024-03-23T09:52:49.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.883+0000 7f712fc51640 1 -- 172.21.15.47:0/166998300 wait complete. 2024-03-23T09:52:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.884+0000 7f712fc51640 1 Processor -- start 2024-03-23T09:52:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.884+0000 7f712fc51640 1 -- start start 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712fc51640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71100039a0 0x7f712815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712fc51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71280cc100 0x7f71280cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712fc51640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71280c99a0 0x7f71280c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712fc51640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712815fab0 con 0x7f71280cc100 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712fc51640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71280ca360 con 0x7f71280c99a0 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712fc51640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712815ea10 con 0x7f71100039a0 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712d1ec640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71280cc100 0x7f71280cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712d1ec640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71280cc100 0x7f71280cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40216/0 (socket says 172.21.15.47:40216) 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712d1ec640 1 -- 172.21.15.47:0/3171994245 learned_addr learned my addr 172.21.15.47:0/3171994245 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712e1ee640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71280c99a0 0x7f71280c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712d9ed640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71100039a0 0x7f712815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712d1ec640 1 -- 172.21.15.47:0/3171994245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71100039a0 msgr2=0x7f712815e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712d1ec640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71100039a0 0x7f712815e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712d1ec640 1 -- 172.21.15.47:0/3171994245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71280c99a0 msgr2=0x7f71280c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712d1ec640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71280c99a0 0x7f71280c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712d1ec640 1 -- 172.21.15.47:0/3171994245 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712815ecf0 con 0x7f71280cc100 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712e1ee640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71280c99a0 0x7f71280c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.885+0000 7f712d1ec640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71280cc100 0x7f71280cc580 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f711c011540 tx=0x7f711c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.886+0000 7f71267fc640 1 -- 172.21.15.47:0/3171994245 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c015870 con 0x7f71280cc100 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.886+0000 7f71267fc640 1 -- 172.21.15.47:0/3171994245 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711c0159d0 con 0x7f71280cc100 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.886+0000 7f712fc51640 1 -- 172.21.15.47:0/3171994245 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712815efd0 con 0x7f71280cc100 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.886+0000 7f71267fc640 1 -- 172.21.15.47:0/3171994245 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c015d40 con 0x7f71280cc100 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.886+0000 7f712fc51640 1 -- 172.21.15.47:0/3171994245 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712815f590 con 0x7f71280cc100 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.886+0000 7f712fc51640 1 -- 172.21.15.47:0/3171994245 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f71280d1990 con 0x7f71280cc100 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.887+0000 7f71267fc640 1 -- 172.21.15.47:0/3171994245 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f711c01d400 con 0x7f71280cc100 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.887+0000 7f71267fc640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f711406ac30 0x7f711406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.888+0000 7f71267fc640 1 -- 172.21.15.47:0/3171994245 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f711c089b30 con 0x7f71280cc100 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.888+0000 7f71267fc640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x7f7114070bd0 0x7f7114072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.888+0000 7f71267fc640 1 -- 172.21.15.47:0/3171994245 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71140736a0 con 0x7f7114070bd0 2024-03-23T09:52:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.888+0000 7f71267fc640 1 -- 172.21.15.47:0/3171994245 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f711c089f80 con 0x7f71280cc100 2024-03-23T09:52:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.888+0000 7f712d9ed640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f711406ac30 0x7f711406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.888+0000 7f712e1ee640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x7f7114070bd0 0x7f7114072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.888+0000 7f712e1ee640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x7f7114070bd0 0x7f7114072ff0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.888+0000 7f712d9ed640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f711406ac30 0x7f711406d0f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7120034000 tx=0x7f7120037040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.889+0000 7f71267fc640 1 -- 172.21.15.47:0/3171994245 <== osd.2 v2:172.21.15.47:6800/37678 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f71140736a0 con 0x7f7114070bd0 2024-03-23T09:52:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.901+0000 7f712fc51640 1 -- 172.21.15.47:0/3171994245 --> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f71280cb8b0 con 0x7f7114070bd0 2024-03-23T09:52:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.901+0000 7f71267fc640 1 -- 172.21.15.47:0/3171994245 <== osd.2 v2:172.21.15.47:6800/37678 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f71280cb8b0 con 0x7f7114070bd0 2024-03-23T09:52:49.902 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:52:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T09:52:49.902 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:52:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.902+0000 7f712fc51640 1 -- 172.21.15.47:0/3171994245 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x7f7114070bd0 msgr2=0x7f7114072ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.902+0000 7f712fc51640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.47:6800/37678,v1:172.21.15.47:6801/37678] conn(0x7f7114070bd0 0x7f7114072ff0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.902+0000 7f712fc51640 1 -- 172.21.15.47:0/3171994245 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f711406ac30 msgr2=0x7f711406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.902+0000 7f712fc51640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f711406ac30 0x7f711406d0f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7120034000 tx=0x7f7120037040 comp rx=0 tx=0).stop 2024-03-23T09:52:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.902+0000 7f712fc51640 1 -- 172.21.15.47:0/3171994245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71280cc100 msgr2=0x7f71280cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.902+0000 7f712fc51640 1 --2- 172.21.15.47:0/3171994245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71280cc100 0x7f71280cc580 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f711c011540 tx=0x7f711c011a00 comp rx=0 tx=0).stop 2024-03-23T09:52:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.902+0000 7f712e1ee640 1 -- 172.21.15.47:0/3171994245 reap_dead start 2024-03-23T09:52:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.902+0000 7f712fc51640 1 -- 172.21.15.47:0/3171994245 shutdown_connections 2024-03-23T09:52:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.902+0000 7f712fc51640 1 -- 172.21.15.47:0/3171994245 >> 172.21.15.47:0/3171994245 conn(0x7f7128065df0 msgr2=0x7f71280b6170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.902+0000 7f712fc51640 1 -- 172.21.15.47:0/3171994245 shutdown_connections 2024-03-23T09:52:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:49.902+0000 7f712fc51640 1 -- 172.21.15.47:0/3171994245 wait complete. 2024-03-23T09:52:49.912 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-23T09:52:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.014+0000 7fbd8ad2e640 1 Processor -- start 2024-03-23T09:52:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.014+0000 7fbd8ad2e640 1 -- start start 2024-03-23T09:52:50.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.015+0000 7fbd8ad2e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd84099110 0x7fbd840d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.015+0000 7fbd8ad2e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd84099760 con 0x7fbd8406b090 2024-03-23T09:52:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.015+0000 7fbd8ad2e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd840cf020 con 0x7fbd8415cd40 2024-03-23T09:52:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.015+0000 7fbd8ad2e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd8415d120 con 0x7fbd84099110 2024-03-23T09:52:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.015+0000 7fbd83fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd84099110 0x7fbd840d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.015+0000 7fbd892cb640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbd8406b090 0x7fbd840caca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38088/0 (socket says 172.21.15.47:38088) 2024-03-23T09:52:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.015+0000 7fbd83fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd84099110 0x7fbd840d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40226/0 (socket says 172.21.15.47:40226) 2024-03-23T09:52:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.015+0000 7fbd892cb640 1 -- 172.21.15.47:0/4285809198 learned_addr learned my addr 172.21.15.47:0/4285809198 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.015+0000 7fbd83fff640 1 -- 172.21.15.47:0/4285809198 >> v1:172.21.15.175:6789/0 conn(0x7fbd8415cd40 legacy=0x7fbd84098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.015+0000 7fbd83fff640 1 -- 172.21.15.47:0/4285809198 >> v1:172.21.15.47:6789/0 conn(0x7fbd8406b090 legacy=0x7fbd840caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.015+0000 7fbd837fe640 1 -- 172.21.15.47:0/4285809198 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672753737 0 0) 0x7fbd84099760 con 0x7fbd8406b090 2024-03-23T09:52:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.015+0000 7fbd83fff640 1 -- 172.21.15.47:0/4285809198 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd840cf300 con 0x7fbd84099110 2024-03-23T09:52:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.016+0000 7fbd83fff640 1 --2- 172.21.15.47:0/4285809198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd84099110 0x7fbd840d00c0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fbd700097a0 tx=0x7fbd7002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49365bfbfacae33 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.016+0000 7fbd837fe640 1 -- 172.21.15.47:0/4285809198 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70031830 con 0x7fbd84099110 2024-03-23T09:52:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.016+0000 7fbd837fe640 1 -- 172.21.15.47:0/4285809198 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd70031990 con 0x7fbd84099110 2024-03-23T09:52:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.016+0000 7fbd837fe640 1 -- 172.21.15.47:0/4285809198 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70031d20 con 0x7fbd84099110 2024-03-23T09:52:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.017+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/4285809198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd84099110 msgr2=0x7fbd840d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.017+0000 7fbd8ad2e640 1 --2- 172.21.15.47:0/4285809198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd84099110 0x7fbd840d00c0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fbd700097a0 tx=0x7fbd7002b8d0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.017+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/4285809198 shutdown_connections 2024-03-23T09:52:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.017+0000 7fbd8ad2e640 1 --2- 172.21.15.47:0/4285809198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd84099110 0x7fbd840d00c0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.017+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/4285809198 >> 172.21.15.47:0/4285809198 conn(0x7fbd840b2140 msgr2=0x7fbd840b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.017+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/4285809198 shutdown_connections 2024-03-23T09:52:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.017+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/4285809198 wait complete. 2024-03-23T09:52:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.018+0000 7fbd8ad2e640 1 Processor -- start 2024-03-23T09:52:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.018+0000 7fbd8ad2e640 1 -- start start 2024-03-23T09:52:50.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd8ad2e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd8406b090 0x7fbd8414fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd8ad2e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd84099110 0x7fbd840b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd8ad2e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd8415cd40 0x7fbd84158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd8ad2e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd841f24c0 con 0x7fbd84099110 2024-03-23T09:52:50.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd8ad2e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8414ffb0 con 0x7fbd8406b090 2024-03-23T09:52:50.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd8ad2e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840b4d20 con 0x7fbd8415cd40 2024-03-23T09:52:50.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd83fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd84099110 0x7fbd840b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd83fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd84099110 0x7fbd840b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40242/0 (socket says 172.21.15.47:40242) 2024-03-23T09:52:50.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd83fff640 1 -- 172.21.15.47:0/2082734372 learned_addr learned my addr 172.21.15.47:0/2082734372 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:50.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd88aca640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd8406b090 0x7fbd8414fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd892cb640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd8415cd40 0x7fbd84158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd83fff640 1 -- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd8415cd40 msgr2=0x7fbd84158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd83fff640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd8415cd40 0x7fbd84158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd83fff640 1 -- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd8406b090 msgr2=0x7fbd8414fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd83fff640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd8406b090 0x7fbd8414fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd83fff640 1 -- 172.21.15.47:0/2082734372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd70009400 con 0x7fbd84099110 2024-03-23T09:52:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd83fff640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd84099110 0x7fbd840b3b60 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fbd700098c0 tx=0x7fbd700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd817fa640 1 -- 172.21.15.47:0/2082734372 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7003ca80 con 0x7fbd84099110 2024-03-23T09:52:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd817fa640 1 -- 172.21.15.47:0/2082734372 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7003cbe0 con 0x7fbd84099110 2024-03-23T09:52:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/2082734372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd84151c00 con 0x7fbd84099110 2024-03-23T09:52:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.019+0000 7fbd817fa640 1 -- 172.21.15.47:0/2082734372 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70004030 con 0x7fbd84099110 2024-03-23T09:52:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.020+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/2082734372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd841521c0 con 0x7fbd84099110 2024-03-23T09:52:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.020+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/2082734372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd8406c180 con 0x7fbd84099110 2024-03-23T09:52:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.021+0000 7fbd817fa640 1 -- 172.21.15.47:0/2082734372 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbd70004190 con 0x7fbd84099110 2024-03-23T09:52:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.021+0000 7fbd817fa640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd6806ac00 0x7fbd6806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.021+0000 7fbd817fa640 1 -- 172.21.15.47:0/2082734372 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fbd700ab810 con 0x7fbd84099110 2024-03-23T09:52:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.022+0000 7fbd88aca640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd6806ac00 0x7fbd6806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.022+0000 7fbd88aca640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd6806ac00 0x7fbd6806d0c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbd78006fd0 tx=0x7fbd78009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.025+0000 7fbd817fa640 1 -- 172.21.15.47:0/2082734372 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbd70076c40 con 0x7fbd84099110 2024-03-23T09:52:50.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.204+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/2082734372 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd840cfba0 con 0x7fbd6806ac00 2024-03-23T09:52:50.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.205+0000 7fbd817fa640 1 -- 172.21.15.47:0/2082734372 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbd78009550 con 0x7fbd6806ac00 2024-03-23T09:52:50.205 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 2 to deep-scrub 2024-03-23T09:52:50.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.207+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd6806ac00 msgr2=0x7fbd6806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.207+0000 7fbd8ad2e640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd6806ac00 0x7fbd6806d0c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbd78006fd0 tx=0x7fbd78009060 comp rx=0 tx=0).stop 2024-03-23T09:52:50.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.207+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/2082734372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd84099110 msgr2=0x7fbd840b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.207+0000 7fbd8ad2e640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd84099110 0x7fbd840b3b60 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fbd700098c0 tx=0x7fbd700046c0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.207+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/2082734372 shutdown_connections 2024-03-23T09:52:50.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.207+0000 7fbd8ad2e640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd6806ac00 0x7fbd6806d0c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.207+0000 7fbd8ad2e640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd8415cd40 0x7fbd84158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.207+0000 7fbd8ad2e640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd84099110 0x7fbd840b3b60 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.207+0000 7fbd8ad2e640 1 --2- 172.21.15.47:0/2082734372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd8406b090 0x7fbd8414fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.207+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/2082734372 >> 172.21.15.47:0/2082734372 conn(0x7fbd840b2140 msgr2=0x7fbd8415b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:50.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.207+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/2082734372 shutdown_connections 2024-03-23T09:52:50.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.207+0000 7fbd8ad2e640 1 -- 172.21.15.47:0/2082734372 wait complete. 2024-03-23T09:52:50.218 INFO:tasks.ceph:Scrubbing osd.3 2024-03-23T09:52:50.218 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-23T09:52:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.322+0000 7f45ab4c4640 1 Processor -- start 2024-03-23T09:52:50.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.322+0000 7f45ab4c4640 1 -- start start 2024-03-23T09:52:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.323+0000 7f45ab4c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45a415a860 0x7f45a4161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.323+0000 7f45ab4c4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f45a415d0f0 0x7f45a415d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.323+0000 7f45ab4c4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a4160bd0 con 0x7f45a40cf190 2024-03-23T09:52:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.323+0000 7f45ab4c4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a40d2790 con 0x7f45a415d0f0 2024-03-23T09:52:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.323+0000 7f45ab4c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a415dbb0 con 0x7f45a415a860 2024-03-23T09:52:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.323+0000 7f45a8a5f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f45a415d0f0 0x7f45a415d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.323+0000 7f45a9260640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45a415a860 0x7f45a4161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.324+0000 7f45a9260640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45a415a860 0x7f45a4161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:58726/0 (socket says 172.21.15.47:58726) 2024-03-23T09:52:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.324+0000 7f45a9260640 1 -- 172.21.15.47:0/1834232511 learned_addr learned my addr 172.21.15.47:0/1834232511 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.324+0000 7f45a9260640 1 -- 172.21.15.47:0/1834232511 >> v2:172.21.15.175:3300/0 conn(0x7f45a415d0f0 msgr2=0x7f45a415d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.324+0000 7f45a9260640 1 --2- 172.21.15.47:0/1834232511 >> v2:172.21.15.175:3300/0 conn(0x7f45a415d0f0 0x7f45a415d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.324+0000 7f45a9260640 1 -- 172.21.15.47:0/1834232511 >> v1:172.21.15.175:6789/0 conn(0x7f45a40cf190 legacy=0x7f45a40cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.324+0000 7f45a9260640 1 -- 172.21.15.47:0/1834232511 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a415de90 con 0x7f45a415a860 2024-03-23T09:52:50.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.325+0000 7f45a9260640 1 --2- 172.21.15.47:0/1834232511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45a415a860 0x7f45a4161060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f459c009870 tx=0x7f459c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44623fb30b004f7c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.325+0000 7f45a3fff640 1 -- 172.21.15.47:0/1834232511 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c030840 con 0x7f45a415a860 2024-03-23T09:52:50.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.325+0000 7f45a3fff640 1 -- 172.21.15.47:0/1834232511 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c0309a0 con 0x7f45a415a860 2024-03-23T09:52:50.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.325+0000 7f45a3fff640 1 -- 172.21.15.47:0/1834232511 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c030d30 con 0x7f45a415a860 2024-03-23T09:52:50.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.325+0000 7f45ab4c4640 1 -- 172.21.15.47:0/1834232511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45a415a860 msgr2=0x7f45a4161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.325+0000 7f45ab4c4640 1 --2- 172.21.15.47:0/1834232511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45a415a860 0x7f45a4161060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f459c009870 tx=0x7f459c02b980 comp rx=0 tx=0).stop 2024-03-23T09:52:50.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.325+0000 7f45ab4c4640 1 -- 172.21.15.47:0/1834232511 shutdown_connections 2024-03-23T09:52:50.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.325+0000 7f45ab4c4640 1 --2- 172.21.15.47:0/1834232511 >> v2:172.21.15.175:3300/0 conn(0x7f45a415d0f0 0x7f45a415d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.325+0000 7f45ab4c4640 1 --2- 172.21.15.47:0/1834232511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45a415a860 0x7f45a4161060 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.325+0000 7f45ab4c4640 1 -- 172.21.15.47:0/1834232511 >> 172.21.15.47:0/1834232511 conn(0x7f45a40659d0 msgr2=0x7f45a4065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:50.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.325+0000 7f45ab4c4640 1 -- 172.21.15.47:0/1834232511 shutdown_connections 2024-03-23T09:52:50.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.325+0000 7f45ab4c4640 1 -- 172.21.15.47:0/1834232511 wait complete. 2024-03-23T09:52:50.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.327+0000 7f45ab4c4640 1 Processor -- start 2024-03-23T09:52:50.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.327+0000 7f45ab4c4640 1 -- start start 2024-03-23T09:52:50.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.327+0000 7f45ab4c4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45a40cf190 0x7f45a40c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.327+0000 7f45ab4c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a415a860 0x7f45a40c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.327+0000 7f45ab4c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a415d0f0 0x7f45a40cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.327+0000 7f45ab4c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40c8e50 con 0x7f45a415d0f0 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.327+0000 7f45ab4c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40c7870 con 0x7f45a415a860 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.327+0000 7f45ab4c4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40c7b50 con 0x7f45a40cf190 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.327+0000 7f45a9a61640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a415d0f0 0x7f45a40cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a9a61640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a415d0f0 0x7f45a40cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40254/0 (socket says 172.21.15.47:40254) 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a9a61640 1 -- 172.21.15.47:0/4029292040 learned_addr learned my addr 172.21.15.47:0/4029292040 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a9260640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45a40cf190 0x7f45a40c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a8a5f640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a415a860 0x7f45a40c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a9a61640 1 -- 172.21.15.47:0/4029292040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45a40cf190 msgr2=0x7f45a40c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a9a61640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45a40cf190 0x7f45a40c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a9a61640 1 -- 172.21.15.47:0/4029292040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a415a860 msgr2=0x7f45a40c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a9a61640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a415a860 0x7f45a40c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a9a61640 1 -- 172.21.15.47:0/4029292040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c0094d0 con 0x7f45a415d0f0 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a9260640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45a40cf190 0x7f45a40c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a9a61640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a415d0f0 0x7f45a40cc480 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f459400c7a0 tx=0x7f459400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a1ffb640 1 -- 172.21.15.47:0/4029292040 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594003b70 con 0x7f45a415d0f0 2024-03-23T09:52:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a1ffb640 1 -- 172.21.15.47:0/4029292040 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4594003cf0 con 0x7f45a415d0f0 2024-03-23T09:52:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45ab4c4640 1 -- 172.21.15.47:0/4029292040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a40c7e30 con 0x7f45a415d0f0 2024-03-23T09:52:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45a1ffb640 1 -- 172.21.15.47:0/4029292040 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594003ea0 con 0x7f45a415d0f0 2024-03-23T09:52:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.328+0000 7f45ab4c4640 1 -- 172.21.15.47:0/4029292040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a40c83f0 con 0x7f45a415d0f0 2024-03-23T09:52:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.329+0000 7f45ab4c4640 1 -- 172.21.15.47:0/4029292040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f45a41615a0 con 0x7f45a415d0f0 2024-03-23T09:52:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.330+0000 7f45a1ffb640 1 -- 172.21.15.47:0/4029292040 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f459400aa90 con 0x7f45a415d0f0 2024-03-23T09:52:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.330+0000 7f45a1ffb640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f459006ac00 0x7f459006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.330+0000 7f45a1ffb640 1 -- 172.21.15.47:0/4029292040 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f4594088e40 con 0x7f45a415d0f0 2024-03-23T09:52:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.330+0000 7f45a1ffb640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x7f4590070ba0 0x7f4590072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.330+0000 7f45a1ffb640 1 -- 172.21.15.47:0/4029292040 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4590073670 con 0x7f4590070ba0 2024-03-23T09:52:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.330+0000 7f45a1ffb640 1 -- 172.21.15.47:0/4029292040 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f459408b500 con 0x7f45a415d0f0 2024-03-23T09:52:50.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.330+0000 7f45a9260640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f459006ac00 0x7f459006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.330+0000 7f45a8a5f640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x7f4590070ba0 0x7f4590072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.331+0000 7f45a8a5f640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x7f4590070ba0 0x7f4590072fc0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.331+0000 7f45a9260640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f459006ac00 0x7f459006d0c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f459c036000 tx=0x7f459c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.331+0000 7f45a1ffb640 1 -- 172.21.15.47:0/4029292040 <== osd.3 v2:172.21.15.47:6808/37683 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f4590073670 con 0x7f4590070ba0 2024-03-23T09:52:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.344+0000 7f45ab4c4640 1 -- 172.21.15.47:0/4029292040 --> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f45a4072360 con 0x7f4590070ba0 2024-03-23T09:52:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.344+0000 7f45a1ffb640 1 -- 172.21.15.47:0/4029292040 <== osd.3 v2:172.21.15.47:6808/37683 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f45a4072360 con 0x7f4590070ba0 2024-03-23T09:52:50.344 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:52:50.344 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T09:52:50.345 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:52:50.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.344+0000 7f45ab4c4640 1 -- 172.21.15.47:0/4029292040 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x7f4590070ba0 msgr2=0x7f4590072fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.344+0000 7f45ab4c4640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.47:6808/37683,v1:172.21.15.47:6809/37683] conn(0x7f4590070ba0 0x7f4590072fc0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.344+0000 7f45ab4c4640 1 -- 172.21.15.47:0/4029292040 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f459006ac00 msgr2=0x7f459006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.344+0000 7f45ab4c4640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f459006ac00 0x7f459006d0c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f459c036000 tx=0x7f459c008030 comp rx=0 tx=0).stop 2024-03-23T09:52:50.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.344+0000 7f45ab4c4640 1 -- 172.21.15.47:0/4029292040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a415d0f0 msgr2=0x7f45a40cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.344+0000 7f45ab4c4640 1 --2- 172.21.15.47:0/4029292040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a415d0f0 0x7f45a40cc480 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f459400c7a0 tx=0x7f459400cc60 comp rx=0 tx=0).stop 2024-03-23T09:52:50.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.344+0000 7f45a9a61640 1 -- 172.21.15.47:0/4029292040 reap_dead start 2024-03-23T09:52:50.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.344+0000 7f45ab4c4640 1 -- 172.21.15.47:0/4029292040 shutdown_connections 2024-03-23T09:52:50.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.344+0000 7f45ab4c4640 1 -- 172.21.15.47:0/4029292040 >> 172.21.15.47:0/4029292040 conn(0x7f45a40659d0 msgr2=0x7f45a415e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:50.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.344+0000 7f45ab4c4640 1 -- 172.21.15.47:0/4029292040 shutdown_connections 2024-03-23T09:52:50.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.344+0000 7f45ab4c4640 1 -- 172.21.15.47:0/4029292040 wait complete. 2024-03-23T09:52:50.355 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-23T09:52:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.456+0000 7ff9fd6f3640 1 Processor -- start 2024-03-23T09:52:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.456+0000 7ff9fd6f3640 1 -- start start 2024-03-23T09:52:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.457+0000 7ff9fd6f3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff9f80b0810 0x7ff9f80b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.457+0000 7ff9fd6f3640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f80b11c0 con 0x7ff9f80caf80 2024-03-23T09:52:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.457+0000 7ff9fd6f3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f80d0810 con 0x7ff9f80cb360 2024-03-23T09:52:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.457+0000 7ff9fd6f3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f8160b70 con 0x7ff9f80b0810 2024-03-23T09:52:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.457+0000 7ff9f77fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff9f80b0810 0x7ff9f80b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.457+0000 7ff9f77fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff9f80b0810 0x7ff9f80b0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40262/0 (socket says 172.21.15.47:40262) 2024-03-23T09:52:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.457+0000 7ff9f6ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff9f80caf80 0x7ff9f80cb9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38096/0 (socket says 172.21.15.47:38096) 2024-03-23T09:52:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.457+0000 7ff9f77fe640 1 -- 172.21.15.47:0/4121003039 learned_addr learned my addr 172.21.15.47:0/4121003039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.458+0000 7ff9f77fe640 1 -- 172.21.15.47:0/4121003039 >> v1:172.21.15.175:6789/0 conn(0x7ff9f80cb360 legacy=0x7ff9f8161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.458+0000 7ff9f77fe640 1 -- 172.21.15.47:0/4121003039 >> v1:172.21.15.47:6789/0 conn(0x7ff9f80caf80 legacy=0x7ff9f80cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.458+0000 7ff9f77fe640 1 -- 172.21.15.47:0/4121003039 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f8161710 con 0x7ff9f80b0810 2024-03-23T09:52:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.458+0000 7ff9f5ffb640 1 -- 172.21.15.47:0/4121003039 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436120321 0 0) 0x7ff9f80b11c0 con 0x7ff9f80caf80 2024-03-23T09:52:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.458+0000 7ff9f77fe640 1 --2- 172.21.15.47:0/4121003039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9f80b0810 0x7ff9f80b0c80 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009e50 tx=0x7ff9e802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7beb9cff63a93ea server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.458+0000 7ff9f5ffb640 1 -- 172.21.15.47:0/4121003039 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e802cbb0 con 0x7ff9f80b0810 2024-03-23T09:52:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.459+0000 7ff9f5ffb640 1 -- 172.21.15.47:0/4121003039 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e802cd10 con 0x7ff9f80b0810 2024-03-23T09:52:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.459+0000 7ff9f5ffb640 1 -- 172.21.15.47:0/4121003039 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8034610 con 0x7ff9f80b0810 2024-03-23T09:52:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.459+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/4121003039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9f80b0810 msgr2=0x7ff9f80b0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.459+0000 7ff9fd6f3640 1 --2- 172.21.15.47:0/4121003039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9f80b0810 0x7ff9f80b0c80 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009e50 tx=0x7ff9e802c010 comp rx=0 tx=0).stop 2024-03-23T09:52:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.459+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/4121003039 shutdown_connections 2024-03-23T09:52:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.459+0000 7ff9fd6f3640 1 --2- 172.21.15.47:0/4121003039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9f80b0810 0x7ff9f80b0c80 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.459+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/4121003039 >> 172.21.15.47:0/4121003039 conn(0x7ff9f8098a00 msgr2=0x7ff9f80cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:50.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.459+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/4121003039 shutdown_connections 2024-03-23T09:52:50.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.459+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/4121003039 wait complete. 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.460+0000 7ff9fd6f3640 1 Processor -- start 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.460+0000 7ff9fd6f3640 1 -- start start 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.461+0000 7ff9fd6f3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9f80b0810 0x7ff9f80c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.461+0000 7ff9fd6f3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f80caf80 0x7ff9f8158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.461+0000 7ff9fd6f3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80cb360 0x7ff9f80c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.461+0000 7ff9fd6f3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f8159630 con 0x7ff9f80b0810 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.461+0000 7ff9fd6f3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f81590c0 con 0x7ff9f80caf80 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.461+0000 7ff9fd6f3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80b25a0 con 0x7ff9f80cb360 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.461+0000 7ff9f6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9f80b0810 0x7ff9f80c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.461+0000 7ff9f6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9f80b0810 0x7ff9f80c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40278/0 (socket says 172.21.15.47:40278) 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.461+0000 7ff9f6ffd640 1 -- 172.21.15.47:0/1013250585 learned_addr learned my addr 172.21.15.47:0/1013250585 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.461+0000 7ff9f67fc640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f80caf80 0x7ff9f8158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.461+0000 7ff9f77fe640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80cb360 0x7ff9f80c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.462+0000 7ff9f6ffd640 1 -- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80cb360 msgr2=0x7ff9f80c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.462+0000 7ff9f6ffd640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80cb360 0x7ff9f80c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.462+0000 7ff9f6ffd640 1 -- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f80caf80 msgr2=0x7ff9f8158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.462+0000 7ff9f6ffd640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f80caf80 0x7ff9f8158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.462+0000 7ff9f6ffd640 1 -- 172.21.15.47:0/1013250585 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e8009b30 con 0x7ff9f80b0810 2024-03-23T09:52:50.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.462+0000 7ff9f77fe640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80cb360 0x7ff9f80c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:50.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.462+0000 7ff9f6ffd640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9f80b0810 0x7ff9f80c8370 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c4a0 tx=0x7ff9f000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.462+0000 7ff9ef7fe640 1 -- 172.21.15.47:0/1013250585 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00074e0 con 0x7ff9f80b0810 2024-03-23T09:52:50.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.462+0000 7ff9ef7fe640 1 -- 172.21.15.47:0/1013250585 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f0007640 con 0x7ff9f80b0810 2024-03-23T09:52:50.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.462+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/1013250585 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f80b2880 con 0x7ff9f80b0810 2024-03-23T09:52:50.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.462+0000 7ff9ef7fe640 1 -- 172.21.15.47:0/1013250585 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0004a00 con 0x7ff9f80b0810 2024-03-23T09:52:50.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.462+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/1013250585 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f80b2e40 con 0x7ff9f80b0810 2024-03-23T09:52:50.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.463+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/1013250585 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f80672f0 con 0x7ff9f80b0810 2024-03-23T09:52:50.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.463+0000 7ff9ef7fe640 1 -- 172.21.15.47:0/1013250585 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff9f0004b60 con 0x7ff9f80b0810 2024-03-23T09:52:50.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.464+0000 7ff9ef7fe640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff9d806ac00 0x7ff9d806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.464+0000 7ff9ef7fe640 1 -- 172.21.15.47:0/1013250585 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ff9f0088520 con 0x7ff9f80b0810 2024-03-23T09:52:50.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.464+0000 7ff9f67fc640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff9d806ac00 0x7ff9d806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.464+0000 7ff9f67fc640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff9d806ac00 0x7ff9d806d0c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0007fd0 tx=0x7ff9e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.468+0000 7ff9ef7fe640 1 -- 172.21.15.47:0/1013250585 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff9f00539e0 con 0x7ff9f80b0810 2024-03-23T09:52:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.647+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/1013250585 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9f80cf540 con 0x7ff9d806ac00 2024-03-23T09:52:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.648+0000 7ff9ef7fe640 1 -- 172.21.15.47:0/1013250585 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff9e00050d0 con 0x7ff9d806ac00 2024-03-23T09:52:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 3 to deep-scrub 2024-03-23T09:52:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.650+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff9d806ac00 msgr2=0x7ff9d806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.650+0000 7ff9fd6f3640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff9d806ac00 0x7ff9d806d0c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0007fd0 tx=0x7ff9e0009040 comp rx=0 tx=0).stop 2024-03-23T09:52:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.650+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/1013250585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9f80b0810 msgr2=0x7ff9f80c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.650+0000 7ff9fd6f3640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9f80b0810 0x7ff9f80c8370 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c4a0 tx=0x7ff9f000c960 comp rx=0 tx=0).stop 2024-03-23T09:52:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.650+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/1013250585 shutdown_connections 2024-03-23T09:52:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.650+0000 7ff9fd6f3640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff9d806ac00 0x7ff9d806d0c0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.650+0000 7ff9fd6f3640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80cb360 0x7ff9f80c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.650+0000 7ff9fd6f3640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f80caf80 0x7ff9f8158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.650+0000 7ff9fd6f3640 1 --2- 172.21.15.47:0/1013250585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9f80b0810 0x7ff9f80c8370 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.650+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/1013250585 >> 172.21.15.47:0/1013250585 conn(0x7ff9f8098a00 msgr2=0x7ff9f8160060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.650+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/1013250585 shutdown_connections 2024-03-23T09:52:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.650+0000 7ff9fd6f3640 1 -- 172.21.15.47:0/1013250585 wait complete. 2024-03-23T09:52:50.661 INFO:tasks.ceph:Scrubbing osd.4 2024-03-23T09:52:50.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-23T09:52:50.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.765+0000 7fbd375c3640 1 Processor -- start 2024-03-23T09:52:50.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.765+0000 7fbd375c3640 1 -- start start 2024-03-23T09:52:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.766+0000 7fbd375c3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd3015a850 0x7fbd300cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.766+0000 7fbd375c3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd300cea40 0x7fbd300cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.766+0000 7fbd375c3640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd300cb410 con 0x7fbd3015ac30 2024-03-23T09:52:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.766+0000 7fbd375c3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd300cee90 con 0x7fbd3015a850 2024-03-23T09:52:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.766+0000 7fbd375c3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd300cf0a0 con 0x7fbd300cea40 2024-03-23T09:52:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.766+0000 7fbd3535f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd3015a850 0x7fbd300cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.766+0000 7fbd34b5e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbd3015ac30 0x7fbd300ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38104/0 (socket says 172.21.15.47:38104) 2024-03-23T09:52:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.766+0000 7fbd34b5e640 1 -- 172.21.15.47:0/1031820052 learned_addr learned my addr 172.21.15.47:0/1031820052 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.767+0000 7fbd35b60640 1 --2- 172.21.15.47:0/1031820052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd300cea40 0x7fbd300cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.767+0000 7fbd3535f640 1 -- 172.21.15.47:0/1031820052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd300cea40 msgr2=0x7fbd300cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.767+0000 7fbd3535f640 1 --2- 172.21.15.47:0/1031820052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd300cea40 0x7fbd300cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.767+0000 7fbd3535f640 1 -- 172.21.15.47:0/1031820052 >> v1:172.21.15.47:6789/0 conn(0x7fbd3015ac30 legacy=0x7fbd300ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.767+0000 7fbd3535f640 1 -- 172.21.15.47:0/1031820052 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd300cf380 con 0x7fbd3015a850 2024-03-23T09:52:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.767+0000 7fbd3535f640 1 --2- 172.21.15.47:0/1031820052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd3015a850 0x7fbd300cd0e0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fbd280097a0 tx=0x7fbd2802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32ef3eb92676daf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.768+0000 7fbd27fff640 1 -- 172.21.15.47:0/1031820052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28030610 con 0x7fbd3015a850 2024-03-23T09:52:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.768+0000 7fbd27fff640 1 -- 172.21.15.47:0/1031820052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd28030770 con 0x7fbd3015a850 2024-03-23T09:52:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.768+0000 7fbd27fff640 1 -- 172.21.15.47:0/1031820052 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28030b00 con 0x7fbd3015a850 2024-03-23T09:52:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.768+0000 7fbd375c3640 1 -- 172.21.15.47:0/1031820052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd3015a850 msgr2=0x7fbd300cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.768+0000 7fbd375c3640 1 --2- 172.21.15.47:0/1031820052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd3015a850 0x7fbd300cd0e0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fbd280097a0 tx=0x7fbd2802b960 comp rx=0 tx=0).stop 2024-03-23T09:52:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.768+0000 7fbd375c3640 1 -- 172.21.15.47:0/1031820052 shutdown_connections 2024-03-23T09:52:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.768+0000 7fbd375c3640 1 --2- 172.21.15.47:0/1031820052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd300cea40 0x7fbd300cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.768+0000 7fbd375c3640 1 --2- 172.21.15.47:0/1031820052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd3015a850 0x7fbd300cd0e0 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.768+0000 7fbd375c3640 1 -- 172.21.15.47:0/1031820052 >> 172.21.15.47:0/1031820052 conn(0x7fbd300d06f0 msgr2=0x7fbd300d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.768+0000 7fbd375c3640 1 -- 172.21.15.47:0/1031820052 shutdown_connections 2024-03-23T09:52:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.768+0000 7fbd375c3640 1 -- 172.21.15.47:0/1031820052 wait complete. 2024-03-23T09:52:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.769+0000 7fbd375c3640 1 Processor -- start 2024-03-23T09:52:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.769+0000 7fbd375c3640 1 -- start start 2024-03-23T09:52:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.770+0000 7fbd375c3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd300cea40 0x7fbd301f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.770+0000 7fbd375c3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd3015a850 0x7fbd300b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.770+0000 7fbd375c3640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd3015ac30 0x7fbd300b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.770+0000 7fbd375c3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd300b56a0 con 0x7fbd3015a850 2024-03-23T09:52:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.770+0000 7fbd375c3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd301f6590 con 0x7fbd300cea40 2024-03-23T09:52:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.770+0000 7fbd375c3640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd301f6870 con 0x7fbd3015ac30 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.770+0000 7fbd34b5e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd3015a850 0x7fbd300b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.770+0000 7fbd34b5e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd3015a850 0x7fbd300b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40292/0 (socket says 172.21.15.47:40292) 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.770+0000 7fbd34b5e640 1 -- 172.21.15.47:0/1583062906 learned_addr learned my addr 172.21.15.47:0/1583062906 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.770+0000 7fbd34b5e640 1 -- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd3015ac30 msgr2=0x7fbd300b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.770+0000 7fbd34b5e640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd3015ac30 0x7fbd300b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.770+0000 7fbd34b5e640 1 -- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd300cea40 msgr2=0x7fbd301f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.770+0000 7fbd3535f640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd300cea40 0x7fbd301f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.771+0000 7fbd34b5e640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd300cea40 0x7fbd301f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.771+0000 7fbd34b5e640 1 -- 172.21.15.47:0/1583062906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd28009400 con 0x7fbd3015a850 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.771+0000 7fbd3535f640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd300cea40 0x7fbd301f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.771+0000 7fbd34b5e640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd3015a850 0x7fbd300b32e0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000c400 tx=0x7fbd2000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.771+0000 7fbd25ffb640 1 -- 172.21.15.47:0/1583062906 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20005ce0 con 0x7fbd3015a850 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.771+0000 7fbd25ffb640 1 -- 172.21.15.47:0/1583062906 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd20005e40 con 0x7fbd3015a850 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.771+0000 7fbd25ffb640 1 -- 172.21.15.47:0/1583062906 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20007310 con 0x7fbd3015a850 2024-03-23T09:52:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.771+0000 7fbd375c3640 1 -- 172.21.15.47:0/1583062906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3006ae90 con 0x7fbd3015a850 2024-03-23T09:52:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.771+0000 7fbd375c3640 1 -- 172.21.15.47:0/1583062906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3006b450 con 0x7fbd3015a850 2024-03-23T09:52:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.772+0000 7fbd375c3640 1 -- 172.21.15.47:0/1583062906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbd300cbbd0 con 0x7fbd3015a850 2024-03-23T09:52:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.772+0000 7fbd25ffb640 1 -- 172.21.15.47:0/1583062906 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbd20007470 con 0x7fbd3015a850 2024-03-23T09:52:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.773+0000 7fbd25ffb640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd1406ac00 0x7fbd1406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.773+0000 7fbd25ffb640 1 -- 172.21.15.47:0/1583062906 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fbd20088790 con 0x7fbd3015a850 2024-03-23T09:52:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.773+0000 7fbd25ffb640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x7fbd14070ba0 0x7fbd14072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.773+0000 7fbd25ffb640 1 -- 172.21.15.47:0/1583062906 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd14073670 con 0x7fbd14070ba0 2024-03-23T09:52:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.773+0000 7fbd25ffb640 1 -- 172.21.15.47:0/1583062906 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd2008acf0 con 0x7fbd3015a850 2024-03-23T09:52:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.773+0000 7fbd3535f640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd1406ac00 0x7fbd1406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.773+0000 7fbd35b60640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x7fbd14070ba0 0x7fbd14072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.774+0000 7fbd35b60640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x7fbd14070ba0 0x7fbd14072fc0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.774+0000 7fbd3535f640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd1406ac00 0x7fbd1406d0c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbd28036000 tx=0x7fbd2803b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.774+0000 7fbd25ffb640 1 -- 172.21.15.47:0/1583062906 <== osd.4 v2:172.21.15.175:6800/38159 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fbd14073670 con 0x7fbd14070ba0 2024-03-23T09:52:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.787+0000 7fbd375c3640 1 -- 172.21.15.47:0/1583062906 --> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbd30066be0 con 0x7fbd14070ba0 2024-03-23T09:52:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.787+0000 7fbd25ffb640 1 -- 172.21.15.47:0/1583062906 <== osd.4 v2:172.21.15.175:6800/38159 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbd30066be0 con 0x7fbd14070ba0 2024-03-23T09:52:50.788 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:52:50.788 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T09:52:50.788 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:52:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.788+0000 7fbd375c3640 1 -- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x7fbd14070ba0 msgr2=0x7fbd14072fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.788+0000 7fbd375c3640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:6800/38159,v1:172.21.15.175:6801/38159] conn(0x7fbd14070ba0 0x7fbd14072fc0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.788+0000 7fbd375c3640 1 -- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd1406ac00 msgr2=0x7fbd1406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.788+0000 7fbd375c3640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fbd1406ac00 0x7fbd1406d0c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbd28036000 tx=0x7fbd2803b040 comp rx=0 tx=0).stop 2024-03-23T09:52:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.788+0000 7fbd375c3640 1 -- 172.21.15.47:0/1583062906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd3015a850 msgr2=0x7fbd300b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.788+0000 7fbd375c3640 1 --2- 172.21.15.47:0/1583062906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd3015a850 0x7fbd300b32e0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000c400 tx=0x7fbd2000c8c0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.788+0000 7fbd35b60640 1 -- 172.21.15.47:0/1583062906 reap_dead start 2024-03-23T09:52:50.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.788+0000 7fbd375c3640 1 -- 172.21.15.47:0/1583062906 shutdown_connections 2024-03-23T09:52:50.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.788+0000 7fbd375c3640 1 -- 172.21.15.47:0/1583062906 >> 172.21.15.47:0/1583062906 conn(0x7fbd300d06f0 msgr2=0x7fbd300c6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:50.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.788+0000 7fbd375c3640 1 -- 172.21.15.47:0/1583062906 shutdown_connections 2024-03-23T09:52:50.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.788+0000 7fbd375c3640 1 -- 172.21.15.47:0/1583062906 wait complete. 2024-03-23T09:52:50.798 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-23T09:52:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.900+0000 7fda237d2640 1 Processor -- start 2024-03-23T09:52:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.900+0000 7fda237d2640 1 -- start start 2024-03-23T09:52:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.901+0000 7fda237d2640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fda1c159b60 0x7fda1c0c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.901+0000 7fda237d2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fda1c0cba90 0x7fda1c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.901+0000 7fda237d2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1c0afa80 con 0x7fda1c0c9600 2024-03-23T09:52:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.901+0000 7fda237d2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1c0b1810 con 0x7fda1c0cba90 2024-03-23T09:52:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.901+0000 7fda237d2640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1c0b1af0 con 0x7fda1c159b60 2024-03-23T09:52:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.901+0000 7fda21d6f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fda1c0cba90 0x7fda1c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.901+0000 7fda20d6d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fda1c0c9600 0x7fda1c0cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38106/0 (socket says 172.21.15.47:38106) 2024-03-23T09:52:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.901+0000 7fda20d6d640 1 -- 172.21.15.47:0/3857666623 learned_addr learned my addr 172.21.15.47:0/3857666623 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.901+0000 7fda21d6f640 1 -- 172.21.15.47:0/3857666623 >> v2:172.21.15.175:3300/0 conn(0x7fda1c159b60 msgr2=0x7fda1c0c90c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T09:52:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.901+0000 7fda21d6f640 1 --2- 172.21.15.47:0/3857666623 >> v2:172.21.15.175:3300/0 conn(0x7fda1c159b60 0x7fda1c0c90c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.901+0000 7fda21d6f640 1 -- 172.21.15.47:0/3857666623 >> v1:172.21.15.47:6789/0 conn(0x7fda1c0c9600 legacy=0x7fda1c0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.901+0000 7fda13fff640 1 -- 172.21.15.47:0/3857666623 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378827058 0 0) 0x7fda1c0afa80 con 0x7fda1c0c9600 2024-03-23T09:52:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.901+0000 7fda21d6f640 1 -- 172.21.15.47:0/3857666623 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c0cab80 con 0x7fda1c0cba90 2024-03-23T09:52:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.902+0000 7fda21d6f640 1 --2- 172.21.15.47:0/3857666623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1c0cba90 0x7fda1c15d590 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0b4390 tx=0x7fda1402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8a730104e231f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.903+0000 7fda13fff640 1 -- 172.21.15.47:0/3857666623 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1402cc00 con 0x7fda1c0cba90 2024-03-23T09:52:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.903+0000 7fda13fff640 1 -- 172.21.15.47:0/3857666623 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1402cd60 con 0x7fda1c0cba90 2024-03-23T09:52:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.903+0000 7fda13fff640 1 -- 172.21.15.47:0/3857666623 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14034660 con 0x7fda1c0cba90 2024-03-23T09:52:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.903+0000 7fda237d2640 1 -- 172.21.15.47:0/3857666623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1c0cba90 msgr2=0x7fda1c15d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.903+0000 7fda237d2640 1 --2- 172.21.15.47:0/3857666623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1c0cba90 0x7fda1c15d590 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0b4390 tx=0x7fda1402be50 comp rx=0 tx=0).stop 2024-03-23T09:52:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.904+0000 7fda237d2640 1 -- 172.21.15.47:0/3857666623 shutdown_connections 2024-03-23T09:52:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.904+0000 7fda237d2640 1 --2- 172.21.15.47:0/3857666623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1c0cba90 0x7fda1c15d590 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.904+0000 7fda237d2640 1 --2- 172.21.15.47:0/3857666623 >> v2:172.21.15.175:3300/0 conn(0x7fda1c159b60 0x7fda1c0c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.904+0000 7fda237d2640 1 -- 172.21.15.47:0/3857666623 >> 172.21.15.47:0/3857666623 conn(0x7fda1c069ce0 msgr2=0x7fda1c06a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.904+0000 7fda237d2640 1 -- 172.21.15.47:0/3857666623 shutdown_connections 2024-03-23T09:52:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.904+0000 7fda237d2640 1 -- 172.21.15.47:0/3857666623 wait complete. 2024-03-23T09:52:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.905+0000 7fda237d2640 1 Processor -- start 2024-03-23T09:52:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.905+0000 7fda237d2640 1 -- start start 2024-03-23T09:52:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.905+0000 7fda237d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1c0c9600 0x7fda1c14a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.905+0000 7fda237d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda1c0cba90 0x7fda1c14b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda237d2640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda1c159b60 0x7fda1c0c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda237d2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c154d50 con 0x7fda1c0c9600 2024-03-23T09:52:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda237d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c154f60 con 0x7fda1c0cba90 2024-03-23T09:52:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda237d2640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c155240 con 0x7fda1c159b60 2024-03-23T09:52:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda2156e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1c0c9600 0x7fda1c14a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda2156e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1c0c9600 0x7fda1c14a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40306/0 (socket says 172.21.15.47:40306) 2024-03-23T09:52:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda2156e640 1 -- 172.21.15.47:0/700347699 learned_addr learned my addr 172.21.15.47:0/700347699 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda20d6d640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda1c0cba90 0x7fda1c14b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda2156e640 1 -- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda1c159b60 msgr2=0x7fda1c0c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda21d6f640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda1c159b60 0x7fda1c0c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda2156e640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda1c159b60 0x7fda1c0c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda2156e640 1 -- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda1c0cba90 msgr2=0x7fda1c14b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda2156e640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda1c0cba90 0x7fda1c14b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda2156e640 1 -- 172.21.15.47:0/700347699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda14009a70 con 0x7fda1c0c9600 2024-03-23T09:52:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda20d6d640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda1c0cba90 0x7fda1c14b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda21d6f640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda1c159b60 0x7fda1c0c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda2156e640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1c0c9600 0x7fda1c14a6b0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fda1800b450 tx=0x7fda1800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda11ffb640 1 -- 172.21.15.47:0/700347699 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18011890 con 0x7fda1c0c9600 2024-03-23T09:52:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda11ffb640 1 -- 172.21.15.47:0/700347699 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda18011a10 con 0x7fda1c0c9600 2024-03-23T09:52:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda237d2640 1 -- 172.21.15.47:0/700347699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1c155520 con 0x7fda1c0c9600 2024-03-23T09:52:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.906+0000 7fda11ffb640 1 -- 172.21.15.47:0/700347699 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18011dc0 con 0x7fda1c0c9600 2024-03-23T09:52:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.907+0000 7fda237d2640 1 -- 172.21.15.47:0/700347699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1c155ae0 con 0x7fda1c0c9600 2024-03-23T09:52:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.907+0000 7fda237d2640 1 -- 172.21.15.47:0/700347699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1c06e1c0 con 0x7fda1c0c9600 2024-03-23T09:52:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.908+0000 7fda11ffb640 1 -- 172.21.15.47:0/700347699 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fda1801a400 con 0x7fda1c0c9600 2024-03-23T09:52:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.908+0000 7fda11ffb640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda0006ac00 0x7fda0006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.908+0000 7fda11ffb640 1 -- 172.21.15.47:0/700347699 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fda180894d0 con 0x7fda1c0c9600 2024-03-23T09:52:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.909+0000 7fda20d6d640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda0006ac00 0x7fda0006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.909+0000 7fda20d6d640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda0006ac00 0x7fda0006d0c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0c9510 tx=0x7fda0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:50.913+0000 7fda11ffb640 1 -- 172.21.15.47:0/700347699 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fda18054900 con 0x7fda1c0c9600 2024-03-23T09:52:51.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.093+0000 7fda237d2640 1 -- 172.21.15.47:0/700347699 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fda1c155d00 con 0x7fda0006ac00 2024-03-23T09:52:51.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.093+0000 7fda11ffb640 1 -- 172.21.15.47:0/700347699 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fda1c155d00 con 0x7fda0006ac00 2024-03-23T09:52:51.094 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 4 to deep-scrub 2024-03-23T09:52:51.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.095+0000 7fda237d2640 1 -- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda0006ac00 msgr2=0x7fda0006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.096+0000 7fda237d2640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda0006ac00 0x7fda0006d0c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0c9510 tx=0x7fda0c009040 comp rx=0 tx=0).stop 2024-03-23T09:52:51.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.096+0000 7fda237d2640 1 -- 172.21.15.47:0/700347699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1c0c9600 msgr2=0x7fda1c14a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.096+0000 7fda237d2640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1c0c9600 0x7fda1c14a6b0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fda1800b450 tx=0x7fda1800b910 comp rx=0 tx=0).stop 2024-03-23T09:52:51.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.096+0000 7fda237d2640 1 -- 172.21.15.47:0/700347699 shutdown_connections 2024-03-23T09:52:51.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.096+0000 7fda237d2640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fda0006ac00 0x7fda0006d0c0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.096+0000 7fda237d2640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda1c159b60 0x7fda1c0c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.096+0000 7fda237d2640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda1c0cba90 0x7fda1c14b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.096+0000 7fda237d2640 1 --2- 172.21.15.47:0/700347699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda1c0c9600 0x7fda1c14a6b0 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.096+0000 7fda237d2640 1 -- 172.21.15.47:0/700347699 >> 172.21.15.47:0/700347699 conn(0x7fda1c069ce0 msgr2=0x7fda1c0cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:51.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.096+0000 7fda237d2640 1 -- 172.21.15.47:0/700347699 shutdown_connections 2024-03-23T09:52:51.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.096+0000 7fda237d2640 1 -- 172.21.15.47:0/700347699 wait complete. 2024-03-23T09:52:51.106 INFO:tasks.ceph:Scrubbing osd.5 2024-03-23T09:52:51.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-23T09:52:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.211+0000 7f6255cc4640 1 Processor -- start 2024-03-23T09:52:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.211+0000 7f6255cc4640 1 -- start start 2024-03-23T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.212+0000 7f6255cc4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f625015d0f0 0x7f625015d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.212+0000 7f6255cc4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6250160bd0 con 0x7f62500cf190 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.212+0000 7f6255cc4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f625015dbb0 con 0x7f625015a860 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.212+0000 7f6255cc4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62500d2790 con 0x7f625015d0f0 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.212+0000 7f624ffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f62500cf190 0x7f62500cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38122/0 (socket says 172.21.15.47:38122) 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.212+0000 7f624ffff640 1 -- 172.21.15.47:0/2665820436 learned_addr learned my addr 172.21.15.47:0/2665820436 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.212+0000 7f624effd640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f625015d0f0 0x7f625015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.212+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301695176 0 0) 0x7f6250160bd0 con 0x7f62500cf190 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.212+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6238003520 con 0x7f62500cf190 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1788093954 0 0) 0x7f6238003520 con 0x7f62500cf190 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f625015d0f0 msgr2=0x7f625015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624e7fc640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f625015d0f0 0x7f625015d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 >> v1:172.21.15.175:6789/0 conn(0x7f625015a860 legacy=0x7f6250161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625015de90 con 0x7f62500cf190 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f62480019f0 con 0x7f62500cf190 2024-03-23T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 >> v1:172.21.15.47:6789/0 conn(0x7f62500cf190 legacy=0x7f62500cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624e7fc640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f625015d0f0 0x7f62380039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624e7fc640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6238003ee0 0x7f62380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624e7fc640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6238004d00 0x7f6238005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62500d2790 con 0x7f6238003ee0 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623800b9e0 con 0x7f6238004d00 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623800bba0 con 0x7f625015d0f0 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624effd640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6238003ee0 0x7f62380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624ffff640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6238004d00 0x7f6238005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624f7fe640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f625015d0f0 0x7f62380039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624effd640 1 -- 172.21.15.47:0/2665820436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f625015d0f0 msgr2=0x7f62380039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624effd640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f625015d0f0 0x7f62380039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624effd640 1 -- 172.21.15.47:0/2665820436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6238004d00 msgr2=0x7f6238005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624effd640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6238004d00 0x7f6238005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624effd640 1 -- 172.21.15.47:0/2665820436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6238003520 con 0x7f6238003ee0 2024-03-23T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624f7fe640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f625015d0f0 0x7f62380039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.213+0000 7f624ffff640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6238004d00 0x7f6238005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.214+0000 7f624effd640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6238003ee0 0x7f62380047c0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f6244009ac0 tx=0x7f62380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e678efaa019d03af server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.214+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624402c600 con 0x7f6238003ee0 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.214+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624402c760 con 0x7f6238003ee0 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.214+0000 7f624e7fc640 1 -- 172.21.15.47:0/2665820436 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624402caf0 con 0x7f6238003ee0 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.214+0000 7f6255cc4640 1 -- 172.21.15.47:0/2665820436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6238003ee0 msgr2=0x7f62380047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.214+0000 7f6255cc4640 1 --2- 172.21.15.47:0/2665820436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6238003ee0 0x7f62380047c0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f6244009ac0 tx=0x7f62380057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.215+0000 7f624ffff640 1 -- 172.21.15.47:0/2665820436 reap_dead start 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.215+0000 7f6255cc4640 1 -- 172.21.15.47:0/2665820436 shutdown_connections 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.215+0000 7f6255cc4640 1 -- 172.21.15.47:0/2665820436 >> 172.21.15.47:0/2665820436 conn(0x7f62500659d0 msgr2=0x7f6250065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.215+0000 7f6255cc4640 1 -- 172.21.15.47:0/2665820436 shutdown_connections 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.215+0000 7f6255cc4640 1 -- 172.21.15.47:0/2665820436 wait complete. 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.216+0000 7f6255cc4640 1 Processor -- start 2024-03-23T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.216+0000 7f6255cc4640 1 -- start start 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f6255cc4640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6238003ee0 0x7f62500ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f6255cc4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62501591e0 0x7f62501e75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f6255cc4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62500b45f0 0x7f62500b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f6255cc4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62501f2510 con 0x7f62501591e0 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f6255cc4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62500b4fb0 con 0x7f62500b45f0 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f6255cc4640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62500caa80 con 0x7f6238003ee0 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f624effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62501591e0 0x7f62501e75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f624effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62501591e0 0x7f62501e75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40328/0 (socket says 172.21.15.47:40328) 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f624effd640 1 -- 172.21.15.47:0/314016679 learned_addr learned my addr 172.21.15.47:0/314016679 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f624f7fe640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6238003ee0 0x7f62500ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f624ffff640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62500b45f0 0x7f62500b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f624effd640 1 -- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6238003ee0 msgr2=0x7f62500ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f624effd640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6238003ee0 0x7f62500ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f624effd640 1 -- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62500b45f0 msgr2=0x7f62500b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f624effd640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62500b45f0 0x7f62500b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f624effd640 1 -- 172.21.15.47:0/314016679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62500cad60 con 0x7f62501591e0 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f624effd640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62501591e0 0x7f62501e75e0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f624402c1a0 tx=0x7f6244004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f6237fff640 1 -- 172.21.15.47:0/314016679 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6244003810 con 0x7f62501591e0 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f6237fff640 1 -- 172.21.15.47:0/314016679 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6244003970 con 0x7f62501591e0 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f6255cc4640 1 -- 172.21.15.47:0/314016679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62500cb040 con 0x7f62501591e0 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f6237fff640 1 -- 172.21.15.47:0/314016679 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6244003ce0 con 0x7f62501591e0 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.217+0000 7f6255cc4640 1 -- 172.21.15.47:0/314016679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62501e7b20 con 0x7f62501591e0 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.218+0000 7f6255cc4640 1 -- 172.21.15.47:0/314016679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f625015d550 con 0x7f62501591e0 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.219+0000 7f6237fff640 1 -- 172.21.15.47:0/314016679 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6244034440 con 0x7f62501591e0 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.219+0000 7f6237fff640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f623c06ac00 0x7f623c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.219+0000 7f6237fff640 1 -- 172.21.15.47:0/314016679 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f62440aaf50 con 0x7f62501591e0 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.219+0000 7f6237fff640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x7f623c070ba0 0x7f623c072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.219+0000 7f624f7fe640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f623c06ac00 0x7f623c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.219+0000 7f6237fff640 1 -- 172.21.15.47:0/314016679 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f623c073670 con 0x7f623c070ba0 2024-03-23T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.219+0000 7f6237fff640 1 -- 172.21.15.47:0/314016679 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62440ad610 con 0x7f62501591e0 2024-03-23T09:52:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.220+0000 7f624ffff640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x7f623c070ba0 0x7f623c072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.220+0000 7f624f7fe640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f623c06ac00 0x7f623c06d0c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6240007fd0 tx=0x7f6240009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.220+0000 7f624ffff640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x7f623c070ba0 0x7f623c072fc0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.221+0000 7f6237fff640 1 -- 172.21.15.47:0/314016679 <== osd.5 v2:172.21.15.175:6816/38161 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f623c073670 con 0x7f623c070ba0 2024-03-23T09:52:51.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.233+0000 7f6255cc4640 1 -- 172.21.15.47:0/314016679 --> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6250072360 con 0x7f623c070ba0 2024-03-23T09:52:51.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.234+0000 7f6237fff640 1 -- 172.21.15.47:0/314016679 <== osd.5 v2:172.21.15.175:6816/38161 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6250072360 con 0x7f623c070ba0 2024-03-23T09:52:51.234 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:52:51.234 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T09:52:51.234 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:52:51.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.234+0000 7f6255cc4640 1 -- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x7f623c070ba0 msgr2=0x7f623c072fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.234+0000 7f6255cc4640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:6816/38161,v1:172.21.15.175:6817/38161] conn(0x7f623c070ba0 0x7f623c072fc0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.234+0000 7f6255cc4640 1 -- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f623c06ac00 msgr2=0x7f623c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.234+0000 7f6255cc4640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f623c06ac00 0x7f623c06d0c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6240007fd0 tx=0x7f6240009040 comp rx=0 tx=0).stop 2024-03-23T09:52:51.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.234+0000 7f6255cc4640 1 -- 172.21.15.47:0/314016679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62501591e0 msgr2=0x7f62501e75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.234+0000 7f6255cc4640 1 --2- 172.21.15.47:0/314016679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62501591e0 0x7f62501e75e0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f624402c1a0 tx=0x7f6244004730 comp rx=0 tx=0).stop 2024-03-23T09:52:51.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.234+0000 7f624ffff640 1 -- 172.21.15.47:0/314016679 reap_dead start 2024-03-23T09:52:51.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.234+0000 7f6255cc4640 1 -- 172.21.15.47:0/314016679 shutdown_connections 2024-03-23T09:52:51.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.234+0000 7f6255cc4640 1 -- 172.21.15.47:0/314016679 >> 172.21.15.47:0/314016679 conn(0x7f62500659d0 msgr2=0x7f625015c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:51.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.235+0000 7f6255cc4640 1 -- 172.21.15.47:0/314016679 shutdown_connections 2024-03-23T09:52:51.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.235+0000 7f6255cc4640 1 -- 172.21.15.47:0/314016679 wait complete. 2024-03-23T09:52:51.245 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-23T09:52:51.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.347+0000 7fcc07bb9640 1 Processor -- start 2024-03-23T09:52:51.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.347+0000 7fcc07bb9640 1 -- start start 2024-03-23T09:52:51.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.348+0000 7fcc07bb9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc000afa50 0x7fcc00098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.348+0000 7fcc07bb9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc000d0230 con 0x7fcc00098f40 2024-03-23T09:52:51.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.348+0000 7fcc07bb9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc000ceac0 con 0x7fcc0006b570 2024-03-23T09:52:51.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.348+0000 7fcc07bb9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0006b9a0 con 0x7fcc000afa50 2024-03-23T09:52:51.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.348+0000 7fcc05154640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcc00098f40 0x7fcc0006ae60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38128/0 (socket says 172.21.15.47:38128) 2024-03-23T09:52:51.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.348+0000 7fcc05154640 1 -- 172.21.15.47:0/1179936334 learned_addr learned my addr 172.21.15.47:0/1179936334 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:51.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.348+0000 7fcc05955640 1 --2- 172.21.15.47:0/1179936334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc000afa50 0x7fcc00098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.348+0000 7fcc04953640 1 -- 172.21.15.47:0/1179936334 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631047839 0 0) 0x7fcc000d0230 con 0x7fcc00098f40 2024-03-23T09:52:51.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.348+0000 7fcc04953640 1 -- 172.21.15.47:0/1179936334 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbec003520 con 0x7fcc00098f40 2024-03-23T09:52:51.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.348+0000 7fcc04953640 1 -- 172.21.15.47:0/1179936334 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406359362 0 0) 0x7fcc000ceac0 con 0x7fcc0006b570 2024-03-23T09:52:51.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.348+0000 7fcc05955640 1 -- 172.21.15.47:0/1179936334 >> v1:172.21.15.175:6789/0 conn(0x7fcc0006b570 legacy=0x7fcc000cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.348+0000 7fcc05955640 1 -- 172.21.15.47:0/1179936334 >> v1:172.21.15.47:6789/0 conn(0x7fcc00098f40 legacy=0x7fcc0006ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.348+0000 7fcc05955640 1 -- 172.21.15.47:0/1179936334 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc00157490 con 0x7fcc000afa50 2024-03-23T09:52:51.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.349+0000 7fcc05955640 1 --2- 172.21.15.47:0/1179936334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc000afa50 0x7fcc00098a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0097d0 tx=0x7fcbfc02b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc5c654c3a8e3c04 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.349+0000 7fcc04953640 1 -- 172.21.15.47:0/1179936334 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0001670 con 0x7fcc000afa50 2024-03-23T09:52:51.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.349+0000 7fcc04953640 1 -- 172.21.15.47:0/1179936334 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc02e050 con 0x7fcc000afa50 2024-03-23T09:52:51.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.349+0000 7fcc04953640 1 -- 172.21.15.47:0/1179936334 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc030730 con 0x7fcc000afa50 2024-03-23T09:52:51.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.350+0000 7fcc07bb9640 1 -- 172.21.15.47:0/1179936334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc000afa50 msgr2=0x7fcc00098a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.350+0000 7fcc07bb9640 1 --2- 172.21.15.47:0/1179936334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc000afa50 0x7fcc00098a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0097d0 tx=0x7fcbfc02b730 comp rx=0 tx=0).stop 2024-03-23T09:52:51.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.350+0000 7fcc07bb9640 1 -- 172.21.15.47:0/1179936334 shutdown_connections 2024-03-23T09:52:51.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.350+0000 7fcc07bb9640 1 --2- 172.21.15.47:0/1179936334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc000afa50 0x7fcc00098a00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.350+0000 7fcc07bb9640 1 -- 172.21.15.47:0/1179936334 >> 172.21.15.47:0/1179936334 conn(0x7fcc000b2140 msgr2=0x7fcc000b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:51.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.350+0000 7fcc07bb9640 1 -- 172.21.15.47:0/1179936334 shutdown_connections 2024-03-23T09:52:51.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.350+0000 7fcc07bb9640 1 -- 172.21.15.47:0/1179936334 wait complete. 2024-03-23T09:52:51.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.351+0000 7fcc07bb9640 1 Processor -- start 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.351+0000 7fcc07bb9640 1 -- start start 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc07bb9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc0006b570 0x7fcc001f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc07bb9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc00098f40 0x7fcc000c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc07bb9640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc000afa50 0x7fcc001f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc07bb9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc000ae3e0 con 0x7fcc0006b570 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc07bb9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc000c79c0 con 0x7fcc00098f40 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc07bb9640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001f2ae0 con 0x7fcc000afa50 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc05955640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc0006b570 0x7fcc001f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc05955640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc0006b570 0x7fcc001f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40334/0 (socket says 172.21.15.47:40334) 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc05955640 1 -- 172.21.15.47:0/2193114333 learned_addr learned my addr 172.21.15.47:0/2193114333 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc05154640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc00098f40 0x7fcc000c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc06156640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc000afa50 0x7fcc001f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc05955640 1 -- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc000afa50 msgr2=0x7fcc001f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc05955640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc000afa50 0x7fcc001f3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc05955640 1 -- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc00098f40 msgr2=0x7fcc000c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc05955640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc00098f40 0x7fcc000c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc05955640 1 -- 172.21.15.47:0/2193114333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc009400 con 0x7fcc0006b570 2024-03-23T09:52:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.352+0000 7fcc05955640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc0006b570 0x7fcc001f2500 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009720 tx=0x7fcbfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.353+0000 7fcbf67fc640 1 -- 172.21.15.47:0/2193114333 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc004220 con 0x7fcc0006b570 2024-03-23T09:52:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.353+0000 7fcbf67fc640 1 -- 172.21.15.47:0/2193114333 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc004380 con 0x7fcc0006b570 2024-03-23T09:52:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.353+0000 7fcc07bb9640 1 -- 172.21.15.47:0/2193114333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc001f2dc0 con 0x7fcc0006b570 2024-03-23T09:52:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.353+0000 7fcbf67fc640 1 -- 172.21.15.47:0/2193114333 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc03c8c0 con 0x7fcc0006b570 2024-03-23T09:52:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.353+0000 7fcc07bb9640 1 -- 172.21.15.47:0/2193114333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc001f5560 con 0x7fcc0006b570 2024-03-23T09:52:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.354+0000 7fcc07bb9640 1 -- 172.21.15.47:0/2193114333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc00066be0 con 0x7fcc0006b570 2024-03-23T09:52:51.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.354+0000 7fcbf67fc640 1 -- 172.21.15.47:0/2193114333 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcbfc039040 con 0x7fcc0006b570 2024-03-23T09:52:51.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.354+0000 7fcbf67fc640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcbe406ac00 0x7fcbe406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.355+0000 7fcbf67fc640 1 -- 172.21.15.47:0/2193114333 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fcbfc0aaa20 con 0x7fcc0006b570 2024-03-23T09:52:51.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.355+0000 7fcc05154640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcbe406ac00 0x7fcbe406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.355+0000 7fcc05154640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcbe406ac00 0x7fcbe406d0c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcc0015cef0 tx=0x7fcbf0001990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.359+0000 7fcbf67fc640 1 -- 172.21.15.47:0/2193114333 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcbfc076000 con 0x7fcc0006b570 2024-03-23T09:52:51.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.538+0000 7fcc07bb9640 1 -- 172.21.15.47:0/2193114333 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc0006be70 con 0x7fcbe406ac00 2024-03-23T09:52:51.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.539+0000 7fcbf67fc640 1 -- 172.21.15.47:0/2193114333 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcbf000e990 con 0x7fcbe406ac00 2024-03-23T09:52:51.539 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 5 to deep-scrub 2024-03-23T09:52:51.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.541+0000 7fcc07bb9640 1 -- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcbe406ac00 msgr2=0x7fcbe406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.541+0000 7fcc07bb9640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcbe406ac00 0x7fcbe406d0c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcc0015cef0 tx=0x7fcbf0001990 comp rx=0 tx=0).stop 2024-03-23T09:52:51.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.541+0000 7fcc07bb9640 1 -- 172.21.15.47:0/2193114333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc0006b570 msgr2=0x7fcc001f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.541+0000 7fcc07bb9640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc0006b570 0x7fcc001f2500 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009720 tx=0x7fcbfc0046c0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.541+0000 7fcc07bb9640 1 -- 172.21.15.47:0/2193114333 shutdown_connections 2024-03-23T09:52:51.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.541+0000 7fcc07bb9640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fcbe406ac00 0x7fcbe406d0c0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.541+0000 7fcc07bb9640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc000afa50 0x7fcc001f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.541+0000 7fcc07bb9640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc00098f40 0x7fcc000c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.541+0000 7fcc07bb9640 1 --2- 172.21.15.47:0/2193114333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc0006b570 0x7fcc001f2500 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.541+0000 7fcc07bb9640 1 -- 172.21.15.47:0/2193114333 >> 172.21.15.47:0/2193114333 conn(0x7fcc000b2140 msgr2=0x7fcc000cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:51.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.541+0000 7fcc07bb9640 1 -- 172.21.15.47:0/2193114333 shutdown_connections 2024-03-23T09:52:51.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.541+0000 7fcc07bb9640 1 -- 172.21.15.47:0/2193114333 wait complete. 2024-03-23T09:52:51.552 INFO:tasks.ceph:Scrubbing osd.6 2024-03-23T09:52:51.552 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-23T09:52:51.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.657+0000 7fd026141640 1 Processor -- start 2024-03-23T09:52:51.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.657+0000 7fd026141640 1 -- start start 2024-03-23T09:52:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd026141640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd02015cb50 0x7fd0200b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd026141640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd0200cde40 0x7fd0200d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd026141640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0200b5a90 con 0x7fd020160780 2024-03-23T09:52:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd026141640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd020161790 con 0x7fd0200cde40 2024-03-23T09:52:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd026141640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd020160ef0 con 0x7fd02015cb50 2024-03-23T09:52:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd01ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd0200cde40 0x7fd0200d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd01ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd0200cde40 0x7fd0200d1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40350/0 (socket says 172.21.15.47:40350) 2024-03-23T09:52:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd01ffff640 1 -- 172.21.15.47:0/1428060264 learned_addr learned my addr 172.21.15.47:0/1428060264 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd01effd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd020160780 0x7fd0200cd730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38138/0 (socket says 172.21.15.47:38138) 2024-03-23T09:52:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd01f7fe640 1 --2- 172.21.15.47:0/1428060264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd02015cb50 0x7fd0200b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd01ffff640 1 -- 172.21.15.47:0/1428060264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd02015cb50 msgr2=0x7fd0200b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd01ffff640 1 --2- 172.21.15.47:0/1428060264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd02015cb50 0x7fd0200b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd01ffff640 1 -- 172.21.15.47:0/1428060264 >> v1:172.21.15.47:6789/0 conn(0x7fd020160780 legacy=0x7fd0200cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd01ffff640 1 -- 172.21.15.47:0/1428060264 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0201611d0 con 0x7fd0200cde40 2024-03-23T09:52:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.658+0000 7fd01f7fe640 1 --2- 172.21.15.47:0/1428060264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd02015cb50 0x7fd0200b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.659+0000 7fd01ffff640 1 --2- 172.21.15.47:0/1428060264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0200cde40 0x7fd0200d1490 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fd018009e50 tx=0x7fd01802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2b7d23ef9d610b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.659+0000 7fd01e7fc640 1 -- 172.21.15.47:0/1428060264 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01802cb00 con 0x7fd0200cde40 2024-03-23T09:52:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.659+0000 7fd01e7fc640 1 -- 172.21.15.47:0/1428060264 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01802cc60 con 0x7fd0200cde40 2024-03-23T09:52:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.659+0000 7fd01e7fc640 1 -- 172.21.15.47:0/1428060264 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0180343f0 con 0x7fd0200cde40 2024-03-23T09:52:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.659+0000 7fd026141640 1 -- 172.21.15.47:0/1428060264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0200cde40 msgr2=0x7fd0200d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.659+0000 7fd026141640 1 --2- 172.21.15.47:0/1428060264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0200cde40 0x7fd0200d1490 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fd018009e50 tx=0x7fd01802bf60 comp rx=0 tx=0).stop 2024-03-23T09:52:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.660+0000 7fd026141640 1 -- 172.21.15.47:0/1428060264 shutdown_connections 2024-03-23T09:52:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.660+0000 7fd026141640 1 --2- 172.21.15.47:0/1428060264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0200cde40 0x7fd0200d1490 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.660+0000 7fd026141640 1 --2- 172.21.15.47:0/1428060264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd02015cb50 0x7fd0200b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.660+0000 7fd026141640 1 -- 172.21.15.47:0/1428060264 >> 172.21.15.47:0/1428060264 conn(0x7fd02006b160 msgr2=0x7fd02006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.660+0000 7fd026141640 1 -- 172.21.15.47:0/1428060264 shutdown_connections 2024-03-23T09:52:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.660+0000 7fd026141640 1 -- 172.21.15.47:0/1428060264 wait complete. 2024-03-23T09:52:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.661+0000 7fd026141640 1 Processor -- start 2024-03-23T09:52:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.661+0000 7fd026141640 1 -- start start 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd026141640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0200cde40 0x7fd0200c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd026141640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd02015cb50 0x7fd020158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd026141640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd020160780 0x7fd0201f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd026141640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02015f280 con 0x7fd0200cde40 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd026141640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd020159540 con 0x7fd02015cb50 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd026141640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02015ec30 con 0x7fd020160780 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd01f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0200cde40 0x7fd0200c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd01f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0200cde40 0x7fd0200c9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40366/0 (socket says 172.21.15.47:40366) 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd01f7fe640 1 -- 172.21.15.47:0/724971133 learned_addr learned my addr 172.21.15.47:0/724971133 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd01ffff640 1 --2- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd020160780 0x7fd0201f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd01f7fe640 1 -- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd020160780 msgr2=0x7fd0201f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd01f7fe640 1 --2- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd020160780 0x7fd0201f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd01f7fe640 1 -- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd02015cb50 msgr2=0x7fd020158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd01f7fe640 1 --2- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd02015cb50 0x7fd020158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd01f7fe640 1 -- 172.21.15.47:0/724971133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd018009b30 con 0x7fd0200cde40 2024-03-23T09:52:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd01f7fe640 1 --2- 172.21.15.47:0/724971133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0200cde40 0x7fd0200c9a30 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fd01000e7e0 tx=0x7fd01000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd007fff640 1 -- 172.21.15.47:0/724971133 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010004030 con 0x7fd0200cde40 2024-03-23T09:52:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd007fff640 1 -- 172.21.15.47:0/724971133 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd010004190 con 0x7fd0200cde40 2024-03-23T09:52:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd026141640 1 -- 172.21.15.47:0/724971133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02015ef10 con 0x7fd0200cde40 2024-03-23T09:52:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd007fff640 1 -- 172.21.15.47:0/724971133 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01000c6d0 con 0x7fd0200cde40 2024-03-23T09:52:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.662+0000 7fd026141640 1 -- 172.21.15.47:0/724971133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0201f6650 con 0x7fd0200cde40 2024-03-23T09:52:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.663+0000 7fd026141640 1 -- 172.21.15.47:0/724971133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd020160be0 con 0x7fd0200cde40 2024-03-23T09:52:51.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.664+0000 7fd007fff640 1 -- 172.21.15.47:0/724971133 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd01000c830 con 0x7fd0200cde40 2024-03-23T09:52:51.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.664+0000 7fd007fff640 1 --2- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd00c06ac00 0x7fd00c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.664+0000 7fd007fff640 1 -- 172.21.15.47:0/724971133 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fd010088df0 con 0x7fd0200cde40 2024-03-23T09:52:51.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.664+0000 7fd007fff640 1 --2- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x7fd00c070ba0 0x7fd00c072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.664+0000 7fd007fff640 1 -- 172.21.15.47:0/724971133 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd00c073670 con 0x7fd00c070ba0 2024-03-23T09:52:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.664+0000 7fd01effd640 1 --2- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd00c06ac00 0x7fd00c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.664+0000 7fd007fff640 1 -- 172.21.15.47:0/724971133 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd01008b4b0 con 0x7fd0200cde40 2024-03-23T09:52:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.664+0000 7fd01ffff640 1 --2- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x7fd00c070ba0 0x7fd00c072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.665+0000 7fd01effd640 1 --2- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd00c06ac00 0x7fd00c06d0c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd020157df0 tx=0x7fd0140079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.665+0000 7fd01ffff640 1 --2- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x7fd00c070ba0 0x7fd00c072fc0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.666+0000 7fd007fff640 1 -- 172.21.15.47:0/724971133 <== osd.6 v2:172.21.15.175:6808/38160 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fd00c073670 con 0x7fd00c070ba0 2024-03-23T09:52:51.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.678+0000 7fd026141640 1 -- 172.21.15.47:0/724971133 --> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd0200d0f70 con 0x7fd00c070ba0 2024-03-23T09:52:51.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.679+0000 7fd007fff640 1 -- 172.21.15.47:0/724971133 <== osd.6 v2:172.21.15.175:6808/38160 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd018009510 con 0x7fd00c070ba0 2024-03-23T09:52:51.679 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:52:51.679 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T09:52:51.679 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:52:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.679+0000 7fd026141640 1 -- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x7fd00c070ba0 msgr2=0x7fd00c072fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.679+0000 7fd026141640 1 --2- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:6808/38160,v1:172.21.15.175:6809/38160] conn(0x7fd00c070ba0 0x7fd00c072fc0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.679+0000 7fd026141640 1 -- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd00c06ac00 msgr2=0x7fd00c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.679+0000 7fd026141640 1 --2- 172.21.15.47:0/724971133 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fd00c06ac00 0x7fd00c06d0c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd020157df0 tx=0x7fd0140079c0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.679+0000 7fd026141640 1 -- 172.21.15.47:0/724971133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0200cde40 msgr2=0x7fd0200c9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.679+0000 7fd026141640 1 --2- 172.21.15.47:0/724971133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0200cde40 0x7fd0200c9a30 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fd01000e7e0 tx=0x7fd01000eca0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.679+0000 7fd01ffff640 1 -- 172.21.15.47:0/724971133 reap_dead start 2024-03-23T09:52:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.679+0000 7fd026141640 1 -- 172.21.15.47:0/724971133 shutdown_connections 2024-03-23T09:52:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.679+0000 7fd026141640 1 -- 172.21.15.47:0/724971133 >> 172.21.15.47:0/724971133 conn(0x7fd02006b160 msgr2=0x7fd0200b8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.679+0000 7fd026141640 1 -- 172.21.15.47:0/724971133 shutdown_connections 2024-03-23T09:52:51.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.679+0000 7fd026141640 1 -- 172.21.15.47:0/724971133 wait complete. 2024-03-23T09:52:51.689 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-23T09:52:51.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.791+0000 7fde600db640 1 Processor -- start 2024-03-23T09:52:51.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.791+0000 7fde600db640 1 -- start start 2024-03-23T09:52:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.792+0000 7fde600db640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fde5806a3f0 0x7fde580b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.792+0000 7fde600db640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fde580b2710 0x7fde580cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.792+0000 7fde600db640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde58071db0 con 0x7fde5815cc10 2024-03-23T09:52:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.792+0000 7fde600db640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde58072240 con 0x7fde580b2710 2024-03-23T09:52:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.792+0000 7fde600db640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde580b2eb0 con 0x7fde5806a3f0 2024-03-23T09:52:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.792+0000 7fde5e678640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fde580b2710 0x7fde580cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.792+0000 7fde5e678640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fde580b2710 0x7fde580cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40370/0 (socket says 172.21.15.47:40370) 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.792+0000 7fde5e678640 1 -- 172.21.15.47:0/2046457659 learned_addr learned my addr 172.21.15.47:0/2046457659 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.792+0000 7fde5d676640 1 --2- 172.21.15.47:0/2046457659 >> v2:172.21.15.175:3300/0 conn(0x7fde5806a3f0 0x7fde580b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.792+0000 7fde5e678640 1 -- 172.21.15.47:0/2046457659 >> v2:172.21.15.175:3300/0 conn(0x7fde5806a3f0 msgr2=0x7fde580b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.792+0000 7fde5e678640 1 --2- 172.21.15.47:0/2046457659 >> v2:172.21.15.175:3300/0 conn(0x7fde5806a3f0 0x7fde580b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.792+0000 7fde5e678640 1 -- 172.21.15.47:0/2046457659 >> v1:172.21.15.175:6789/0 conn(0x7fde5815cc10 legacy=0x7fde58069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.792+0000 7fde5e678640 1 -- 172.21.15.47:0/2046457659 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde580714c0 con 0x7fde580b2710 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.793+0000 7fde5e678640 1 --2- 172.21.15.47:0/2046457659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde580b2710 0x7fde580cc510 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fde580d06f0 tx=0x7fde4802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e5fe4d719f8e27 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.793+0000 7fde5ce75640 1 -- 172.21.15.47:0/2046457659 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4802c9f0 con 0x7fde580b2710 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.793+0000 7fde5ce75640 1 -- 172.21.15.47:0/2046457659 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4802cb50 con 0x7fde580b2710 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.793+0000 7fde5ce75640 1 -- 172.21.15.47:0/2046457659 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde480343f0 con 0x7fde580b2710 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.794+0000 7fde600db640 1 -- 172.21.15.47:0/2046457659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde580b2710 msgr2=0x7fde580cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.794+0000 7fde600db640 1 --2- 172.21.15.47:0/2046457659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde580b2710 0x7fde580cc510 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fde580d06f0 tx=0x7fde4802be50 comp rx=0 tx=0).stop 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.794+0000 7fde600db640 1 -- 172.21.15.47:0/2046457659 shutdown_connections 2024-03-23T09:52:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.794+0000 7fde600db640 1 --2- 172.21.15.47:0/2046457659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde580b2710 0x7fde580cc510 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.794+0000 7fde600db640 1 --2- 172.21.15.47:0/2046457659 >> v2:172.21.15.175:3300/0 conn(0x7fde5806a3f0 0x7fde580b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.794+0000 7fde600db640 1 -- 172.21.15.47:0/2046457659 >> 172.21.15.47:0/2046457659 conn(0x7fde580b1410 msgr2=0x7fde580b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.794+0000 7fde600db640 1 -- 172.21.15.47:0/2046457659 shutdown_connections 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.794+0000 7fde600db640 1 -- 172.21.15.47:0/2046457659 wait complete. 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.795+0000 7fde600db640 1 Processor -- start 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.795+0000 7fde600db640 1 -- start start 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde600db640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde5806a3f0 0x7fde580caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde600db640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde580b2710 0x7fde581f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde600db640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde5815cc10 0x7fde58153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde600db640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde580caf90 con 0x7fde5815cc10 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde600db640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5815cff0 con 0x7fde580b2710 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde600db640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5815d2d0 con 0x7fde5806a3f0 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde5e678640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde5815cc10 0x7fde58153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde5e678640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde5815cc10 0x7fde58153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40382/0 (socket says 172.21.15.47:40382) 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde5e678640 1 -- 172.21.15.47:0/2445409788 learned_addr learned my addr 172.21.15.47:0/2445409788 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde5d676640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde580b2710 0x7fde581f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde5de77640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde5806a3f0 0x7fde580caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde5e678640 1 -- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde5806a3f0 msgr2=0x7fde580caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde5e678640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde5806a3f0 0x7fde580caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde5e678640 1 -- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde580b2710 msgr2=0x7fde581f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde5e678640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde580b2710 0x7fde581f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde5e678640 1 -- 172.21.15.47:0/2445409788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde48009b30 con 0x7fde5815cc10 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde5d676640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde580b2710 0x7fde581f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde5de77640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde5806a3f0 0x7fde580caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.796+0000 7fde5e678640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde5815cc10 0x7fde58153170 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fde48009f40 tx=0x7fde48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.797+0000 7fde567fc640 1 -- 172.21.15.47:0/2445409788 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48009510 con 0x7fde5815cc10 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.797+0000 7fde567fc640 1 -- 172.21.15.47:0/2445409788 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4803e450 con 0x7fde5815cc10 2024-03-23T09:52:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.797+0000 7fde600db640 1 -- 172.21.15.47:0/2445409788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5815d5b0 con 0x7fde5815cc10 2024-03-23T09:52:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.797+0000 7fde567fc640 1 -- 172.21.15.47:0/2445409788 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4803e7e0 con 0x7fde5815cc10 2024-03-23T09:52:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.797+0000 7fde600db640 1 -- 172.21.15.47:0/2445409788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5815db70 con 0x7fde5815cc10 2024-03-23T09:52:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.798+0000 7fde600db640 1 -- 172.21.15.47:0/2445409788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde58066be0 con 0x7fde5815cc10 2024-03-23T09:52:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.798+0000 7fde567fc640 1 -- 172.21.15.47:0/2445409788 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fde4803e940 con 0x7fde5815cc10 2024-03-23T09:52:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.798+0000 7fde567fc640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fde4406ac00 0x7fde4406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.799+0000 7fde567fc640 1 -- 172.21.15.47:0/2445409788 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fde480abef0 con 0x7fde5815cc10 2024-03-23T09:52:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.799+0000 7fde5de77640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fde4406ac00 0x7fde4406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.799+0000 7fde5de77640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fde4406ac00 0x7fde4406d0c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fde5000a830 tx=0x7fde50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.803+0000 7fde567fc640 1 -- 172.21.15.47:0/2445409788 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fde48077320 con 0x7fde5815cc10 2024-03-23T09:52:51.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.981+0000 7fde600db640 1 -- 172.21.15.47:0/2445409788 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fde580b2b40 con 0x7fde4406ac00 2024-03-23T09:52:51.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.982+0000 7fde567fc640 1 -- 172.21.15.47:0/2445409788 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fde5000adb0 con 0x7fde4406ac00 2024-03-23T09:52:51.982 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 6 to deep-scrub 2024-03-23T09:52:51.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.983+0000 7fde600db640 1 -- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fde4406ac00 msgr2=0x7fde4406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.983+0000 7fde600db640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fde4406ac00 0x7fde4406d0c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fde5000a830 tx=0x7fde50008040 comp rx=0 tx=0).stop 2024-03-23T09:52:51.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.983+0000 7fde600db640 1 -- 172.21.15.47:0/2445409788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde5815cc10 msgr2=0x7fde58153170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:51.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.983+0000 7fde600db640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde5815cc10 0x7fde58153170 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fde48009f40 tx=0x7fde48004a40 comp rx=0 tx=0).stop 2024-03-23T09:52:51.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.983+0000 7fde600db640 1 -- 172.21.15.47:0/2445409788 shutdown_connections 2024-03-23T09:52:51.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.983+0000 7fde600db640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7fde4406ac00 0x7fde4406d0c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.983+0000 7fde600db640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde5815cc10 0x7fde58153170 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.983+0000 7fde600db640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde580b2710 0x7fde581f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.983+0000 7fde600db640 1 --2- 172.21.15.47:0/2445409788 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde5806a3f0 0x7fde580caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:51.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.983+0000 7fde600db640 1 -- 172.21.15.47:0/2445409788 >> 172.21.15.47:0/2445409788 conn(0x7fde580b1410 msgr2=0x7fde580cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:51.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.984+0000 7fde600db640 1 -- 172.21.15.47:0/2445409788 shutdown_connections 2024-03-23T09:52:51.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:51.984+0000 7fde600db640 1 -- 172.21.15.47:0/2445409788 wait complete. 2024-03-23T09:52:51.994 INFO:tasks.ceph:Scrubbing osd.7 2024-03-23T09:52:51.995 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-23T09:52:52.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.100+0000 7effc2b31640 1 Processor -- start 2024-03-23T09:52:52.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.100+0000 7effc2b31640 1 -- start start 2024-03-23T09:52:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.101+0000 7effc2b31640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effbc15cc40 0x7effbc0b8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.101+0000 7effc2b31640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7effbc0cf880 0x7effbc0d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.101+0000 7effc2b31640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effbc0d0b40 con 0x7effbc0b4340 2024-03-23T09:52:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.101+0000 7effc2b31640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc0b48b0 con 0x7effbc15cc40 2024-03-23T09:52:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.101+0000 7effc2b31640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc0b4b90 con 0x7effbc0cf880 2024-03-23T09:52:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.101+0000 7effc08cd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effbc15cc40 0x7effbc0b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.101+0000 7effbbfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7effbc0b4340 0x7effbc0cf170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38150/0 (socket says 172.21.15.47:38150) 2024-03-23T09:52:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.101+0000 7effbbfff640 1 -- 172.21.15.47:0/1051194860 learned_addr learned my addr 172.21.15.47:0/1051194860 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.102+0000 7effc08cd640 1 -- 172.21.15.47:0/1051194860 >> v2:172.21.15.175:3300/0 conn(0x7effbc0cf880 msgr2=0x7effbc0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.102+0000 7effc08cd640 1 --2- 172.21.15.47:0/1051194860 >> v2:172.21.15.175:3300/0 conn(0x7effbc0cf880 0x7effbc0d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.102+0000 7effc08cd640 1 -- 172.21.15.47:0/1051194860 >> v1:172.21.15.47:6789/0 conn(0x7effbc0b4340 legacy=0x7effbc0cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.102+0000 7effbb7fe640 1 -- 172.21.15.47:0/1051194860 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750754394 0 0) 0x7effbc0d0b40 con 0x7effbc0b4340 2024-03-23T09:52:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.102+0000 7effc08cd640 1 -- 172.21.15.47:0/1051194860 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc0b4e70 con 0x7effbc15cc40 2024-03-23T09:52:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.102+0000 7effc08cd640 1 --2- 172.21.15.47:0/1051194860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc15cc40 0x7effbc0b8250 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7effbc06aac0 tx=0x7effb402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40fc1c5bcb96d30 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.102+0000 7effbb7fe640 1 -- 172.21.15.47:0/1051194860 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4031640 con 0x7effbc15cc40 2024-03-23T09:52:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.102+0000 7effbb7fe640 1 -- 172.21.15.47:0/1051194860 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb40317a0 con 0x7effbc15cc40 2024-03-23T09:52:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.102+0000 7effbb7fe640 1 -- 172.21.15.47:0/1051194860 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4031b30 con 0x7effbc15cc40 2024-03-23T09:52:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.103+0000 7effc2b31640 1 -- 172.21.15.47:0/1051194860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc15cc40 msgr2=0x7effbc0b8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.103+0000 7effc2b31640 1 --2- 172.21.15.47:0/1051194860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc15cc40 0x7effbc0b8250 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7effbc06aac0 tx=0x7effb402b8f0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.103+0000 7effc2b31640 1 -- 172.21.15.47:0/1051194860 shutdown_connections 2024-03-23T09:52:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.103+0000 7effc2b31640 1 --2- 172.21.15.47:0/1051194860 >> v2:172.21.15.175:3300/0 conn(0x7effbc0cf880 0x7effbc0d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.103+0000 7effc2b31640 1 --2- 172.21.15.47:0/1051194860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc15cc40 0x7effbc0b8250 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.103+0000 7effc2b31640 1 -- 172.21.15.47:0/1051194860 >> 172.21.15.47:0/1051194860 conn(0x7effbc0659d0 msgr2=0x7effbc065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.103+0000 7effc2b31640 1 -- 172.21.15.47:0/1051194860 shutdown_connections 2024-03-23T09:52:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.103+0000 7effc2b31640 1 -- 172.21.15.47:0/1051194860 wait complete. 2024-03-23T09:52:52.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.104+0000 7effc2b31640 1 Processor -- start 2024-03-23T09:52:52.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.104+0000 7effc2b31640 1 -- start start 2024-03-23T09:52:52.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.105+0000 7effc2b31640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effbc0b4340 0x7effbc14dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.105+0000 7effc2b31640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc0cf880 0x7effbc0d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.105+0000 7effc2b31640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effbc15cc40 0x7effbc1f5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.105+0000 7effc2b31640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc0b1420 con 0x7effbc0cf880 2024-03-23T09:52:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.105+0000 7effc2b31640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc1f6570 con 0x7effbc15cc40 2024-03-23T09:52:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.105+0000 7effc2b31640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc1f5fb0 con 0x7effbc0b4340 2024-03-23T09:52:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.105+0000 7effbbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc0cf880 0x7effbc0d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.105+0000 7effbbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc0cf880 0x7effbc0d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40404/0 (socket says 172.21.15.47:40404) 2024-03-23T09:52:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.105+0000 7effbbfff640 1 -- 172.21.15.47:0/2795139672 learned_addr learned my addr 172.21.15.47:0/2795139672 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.105+0000 7effc08cd640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effbc0b4340 0x7effbc14dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.105+0000 7effc10ce640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effbc15cc40 0x7effbc1f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.106+0000 7effbbfff640 1 -- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effbc0b4340 msgr2=0x7effbc14dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.106+0000 7effbbfff640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effbc0b4340 0x7effbc14dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.106+0000 7effbbfff640 1 -- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effbc15cc40 msgr2=0x7effbc1f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.106+0000 7effbbfff640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effbc15cc40 0x7effbc1f5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.106+0000 7effbbfff640 1 -- 172.21.15.47:0/2795139672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb4009400 con 0x7effbc0cf880 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.106+0000 7effc08cd640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effbc0b4340 0x7effbc14dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.106+0000 7effc10ce640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effbc15cc40 0x7effbc1f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.106+0000 7effbbfff640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc0cf880 0x7effbc0d1b40 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7effac00c890 tx=0x7effac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.106+0000 7effb97fa640 1 -- 172.21.15.47:0/2795139672 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac00a640 con 0x7effbc0cf880 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.106+0000 7effb97fa640 1 -- 172.21.15.47:0/2795139672 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effac00a7c0 con 0x7effbc0cf880 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.106+0000 7effc2b31640 1 -- 172.21.15.47:0/2795139672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effbc1f6290 con 0x7effbc0cf880 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.106+0000 7effb97fa640 1 -- 172.21.15.47:0/2795139672 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac00ab70 con 0x7effbc0cf880 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.106+0000 7effc2b31640 1 -- 172.21.15.47:0/2795139672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc14d9a0 con 0x7effbc0cf880 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.107+0000 7effc2b31640 1 -- 172.21.15.47:0/2795139672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7effbc0b8790 con 0x7effbc0cf880 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.107+0000 7effb97fa640 1 -- 172.21.15.47:0/2795139672 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7effac00acd0 con 0x7effbc0cf880 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.108+0000 7effb97fa640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7eff9c06ac00 0x7eff9c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.108+0000 7effb97fa640 1 -- 172.21.15.47:0/2795139672 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7effac0889b0 con 0x7effbc0cf880 2024-03-23T09:52:52.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.108+0000 7effb97fa640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x7eff9c070ba0 0x7eff9c072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.108+0000 7effb97fa640 1 -- 172.21.15.47:0/2795139672 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff9c073670 con 0x7eff9c070ba0 2024-03-23T09:52:52.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.108+0000 7effb97fa640 1 -- 172.21.15.47:0/2795139672 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7effac08d2c0 con 0x7effbc0cf880 2024-03-23T09:52:52.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.108+0000 7effc08cd640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7eff9c06ac00 0x7eff9c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.108+0000 7effc10ce640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x7eff9c070ba0 0x7eff9c072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.108+0000 7effc08cd640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7eff9c06ac00 0x7eff9c06d0c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7effb4036000 tx=0x7effb403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:52.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.109+0000 7effc10ce640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x7eff9c070ba0 0x7eff9c072fc0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:52.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.110+0000 7effb97fa640 1 -- 172.21.15.47:0/2795139672 <== osd.7 v2:172.21.15.175:6824/38174 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7eff9c073670 con 0x7eff9c070ba0 2024-03-23T09:52:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.123+0000 7effc2b31640 1 -- 172.21.15.47:0/2795139672 --> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7effbc15d020 con 0x7eff9c070ba0 2024-03-23T09:52:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.123+0000 7effb97fa640 1 -- 172.21.15.47:0/2795139672 <== osd.7 v2:172.21.15.175:6824/38174 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7effa4009a00 con 0x7eff9c070ba0 2024-03-23T09:52:52.124 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-23T09:52:52.124 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T09:52:52.124 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-23T09:52:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.124+0000 7effc2b31640 1 -- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x7eff9c070ba0 msgr2=0x7eff9c072fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.124+0000 7effc2b31640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:6824/38174,v1:172.21.15.175:6825/38174] conn(0x7eff9c070ba0 0x7eff9c072fc0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.124+0000 7effc2b31640 1 -- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7eff9c06ac00 msgr2=0x7eff9c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.124+0000 7effc2b31640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7eff9c06ac00 0x7eff9c06d0c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7effb4036000 tx=0x7effb403c040 comp rx=0 tx=0).stop 2024-03-23T09:52:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.124+0000 7effc2b31640 1 -- 172.21.15.47:0/2795139672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc0cf880 msgr2=0x7effbc0d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.124+0000 7effc2b31640 1 --2- 172.21.15.47:0/2795139672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc0cf880 0x7effbc0d1b40 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7effac00c890 tx=0x7effac00cd50 comp rx=0 tx=0).stop 2024-03-23T09:52:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.124+0000 7effc10ce640 1 -- 172.21.15.47:0/2795139672 reap_dead start 2024-03-23T09:52:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.124+0000 7effc2b31640 1 -- 172.21.15.47:0/2795139672 shutdown_connections 2024-03-23T09:52:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.124+0000 7effc2b31640 1 -- 172.21.15.47:0/2795139672 >> 172.21.15.47:0/2795139672 conn(0x7effbc0659d0 msgr2=0x7effbc0b7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.124+0000 7effc2b31640 1 -- 172.21.15.47:0/2795139672 shutdown_connections 2024-03-23T09:52:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.124+0000 7effc2b31640 1 -- 172.21.15.47:0/2795139672 wait complete. 2024-03-23T09:52:52.134 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-23T09:52:52.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.237+0000 7f42062ce640 1 Processor -- start 2024-03-23T09:52:52.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.237+0000 7f42062ce640 1 -- start start 2024-03-23T09:52:52.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.238+0000 7f42062ce640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f42000d23a0 0x7f42000d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.238+0000 7f42062ce640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f42000b8300 0x7f42000d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.238+0000 7f42062ce640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42000d10a0 con 0x7f420015cee0 2024-03-23T09:52:52.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.238+0000 7f42062ce640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42000d1300 con 0x7f42000b8300 2024-03-23T09:52:52.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.238+0000 7f42062ce640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42000d15e0 con 0x7f42000d23a0 2024-03-23T09:52:52.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.238+0000 7f420486b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f42000b8300 0x7f42000d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.238+0000 7f420486b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f42000b8300 0x7f42000d2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40418/0 (socket says 172.21.15.47:40418) 2024-03-23T09:52:52.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.238+0000 7f420486b640 1 -- 172.21.15.47:0/2542218981 learned_addr learned my addr 172.21.15.47:0/2542218981 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:52.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.238+0000 7f420486b640 1 -- 172.21.15.47:0/2542218981 >> v2:172.21.15.175:3300/0 conn(0x7f42000d23a0 msgr2=0x7f42000d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:52.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.238+0000 7f420486b640 1 --2- 172.21.15.47:0/2542218981 >> v2:172.21.15.175:3300/0 conn(0x7f42000d23a0 0x7f42000d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.238+0000 7f420486b640 1 -- 172.21.15.47:0/2542218981 >> v1:172.21.15.175:6789/0 conn(0x7f420015cee0 legacy=0x7f42000d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:52.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.238+0000 7f420486b640 1 -- 172.21.15.47:0/2542218981 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42000d18c0 con 0x7f42000b8300 2024-03-23T09:52:52.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.239+0000 7f420486b640 1 --2- 172.21.15.47:0/2542218981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42000b8300 0x7f42000d2db0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f42000cf8d0 tx=0x7f41f802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd616aee17e4ca7c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:52.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.239+0000 7f41feffd640 1 -- 172.21.15.47:0/2542218981 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f802ca80 con 0x7f42000b8300 2024-03-23T09:52:52.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.239+0000 7f41feffd640 1 -- 172.21.15.47:0/2542218981 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f802cbe0 con 0x7f42000b8300 2024-03-23T09:52:52.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.239+0000 7f41feffd640 1 -- 172.21.15.47:0/2542218981 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f80343f0 con 0x7f42000b8300 2024-03-23T09:52:52.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.240+0000 7f42062ce640 1 -- 172.21.15.47:0/2542218981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42000b8300 msgr2=0x7f42000d2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.240+0000 7f42062ce640 1 --2- 172.21.15.47:0/2542218981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42000b8300 0x7f42000d2db0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f42000cf8d0 tx=0x7f41f802bee0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.240+0000 7f42062ce640 1 -- 172.21.15.47:0/2542218981 shutdown_connections 2024-03-23T09:52:52.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.240+0000 7f42062ce640 1 --2- 172.21.15.47:0/2542218981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42000b8300 0x7f42000d2db0 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.240+0000 7f42062ce640 1 --2- 172.21.15.47:0/2542218981 >> v2:172.21.15.175:3300/0 conn(0x7f42000d23a0 0x7f42000d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.240+0000 7f42062ce640 1 -- 172.21.15.47:0/2542218981 >> 172.21.15.47:0/2542218981 conn(0x7f4200065df0 msgr2=0x7f4200066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:52.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.240+0000 7f42062ce640 1 -- 172.21.15.47:0/2542218981 shutdown_connections 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.240+0000 7f42062ce640 1 -- 172.21.15.47:0/2542218981 wait complete. 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.241+0000 7f42062ce640 1 Processor -- start 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.241+0000 7f42062ce640 1 -- start start 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f42062ce640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42000b8300 0x7f420014c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f42062ce640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42000d23a0 0x7f420015d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f42062ce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f420015cee0 0x7f42000c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f42062ce640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42001598a0 con 0x7f42000d23a0 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f42062ce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420015e120 con 0x7f420015cee0 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f42062ce640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4200159250 con 0x7f42000b8300 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f41ff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42000d23a0 0x7f420015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f41ff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42000d23a0 0x7f420015d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40432/0 (socket says 172.21.15.47:40432) 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f41ff7fe640 1 -- 172.21.15.47:0/4213401774 learned_addr learned my addr 172.21.15.47:0/4213401774 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f420486b640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f420015cee0 0x7f42000c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f41fffff640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42000b8300 0x7f420014c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f41ff7fe640 1 -- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42000b8300 msgr2=0x7f420014c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f41ff7fe640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42000b8300 0x7f420014c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f41ff7fe640 1 -- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f420015cee0 msgr2=0x7f42000c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f41ff7fe640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f420015cee0 0x7f42000c6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f41ff7fe640 1 -- 172.21.15.47:0/4213401774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f8009b30 con 0x7f42000d23a0 2024-03-23T09:52:52.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f41fffff640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42000b8300 0x7f420014c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:52.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f41ff7fe640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42000d23a0 0x7f420015d660 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f41f000b450 tx=0x7f41f000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:52.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.242+0000 7f41fcff9640 1 -- 172.21.15.47:0/4213401774 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0011890 con 0x7f42000d23a0 2024-03-23T09:52:52.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.243+0000 7f41fcff9640 1 -- 172.21.15.47:0/4213401774 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f00119f0 con 0x7f42000d23a0 2024-03-23T09:52:52.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.243+0000 7f41fcff9640 1 -- 172.21.15.47:0/4213401774 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0011da0 con 0x7f42000d23a0 2024-03-23T09:52:52.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.243+0000 7f42062ce640 1 -- 172.21.15.47:0/4213401774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4200159530 con 0x7f42000d23a0 2024-03-23T09:52:52.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.243+0000 7f42062ce640 1 -- 172.21.15.47:0/4213401774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420015dba0 con 0x7f42000d23a0 2024-03-23T09:52:52.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.244+0000 7f42062ce640 1 -- 172.21.15.47:0/4213401774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4200072360 con 0x7f42000d23a0 2024-03-23T09:52:52.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.244+0000 7f41fcff9640 1 -- 172.21.15.47:0/4213401774 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f41f001a400 con 0x7f42000d23a0 2024-03-23T09:52:52.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.244+0000 7f41fcff9640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f41e006ac30 0x7f41e006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.245+0000 7f41fcff9640 1 -- 172.21.15.47:0/4213401774 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f41f0089330 con 0x7f42000d23a0 2024-03-23T09:52:52.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.245+0000 7f41fffff640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f41e006ac30 0x7f41e006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.245+0000 7f41fffff640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f41e006ac30 0x7f41e006d0f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f41e8006fd0 tx=0x7f41e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:52.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.249+0000 7f41fcff9640 1 -- 172.21.15.47:0/4213401774 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f41f00547f0 con 0x7f42000d23a0 2024-03-23T09:52:52.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.427+0000 7f42062ce640 1 -- 172.21.15.47:0/4213401774 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f42000d1e80 con 0x7f41e006ac30 2024-03-23T09:52:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.428+0000 7f41fcff9640 1 -- 172.21.15.47:0/4213401774 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f41e8009e00 con 0x7f41e006ac30 2024-03-23T09:52:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 7 to deep-scrub 2024-03-23T09:52:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.429+0000 7f42062ce640 1 -- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f41e006ac30 msgr2=0x7f41e006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.429+0000 7f42062ce640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f41e006ac30 0x7f41e006d0f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f41e8006fd0 tx=0x7f41e8008040 comp rx=0 tx=0).stop 2024-03-23T09:52:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.429+0000 7f42062ce640 1 -- 172.21.15.47:0/4213401774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42000d23a0 msgr2=0x7f420015d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.429+0000 7f42062ce640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42000d23a0 0x7f420015d660 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f41f000b450 tx=0x7f41f000b910 comp rx=0 tx=0).stop 2024-03-23T09:52:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.429+0000 7f42062ce640 1 -- 172.21.15.47:0/4213401774 shutdown_connections 2024-03-23T09:52:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.429+0000 7f42062ce640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f41e006ac30 0x7f41e006d0f0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.429+0000 7f42062ce640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f420015cee0 0x7f42000c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.429+0000 7f42062ce640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42000d23a0 0x7f420015d660 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.429+0000 7f42062ce640 1 --2- 172.21.15.47:0/4213401774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42000b8300 0x7f420014c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.430+0000 7f42062ce640 1 -- 172.21.15.47:0/4213401774 >> 172.21.15.47:0/4213401774 conn(0x7f4200065df0 msgr2=0x7f42000d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.430+0000 7f42062ce640 1 -- 172.21.15.47:0/4213401774 shutdown_connections 2024-03-23T09:52:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.430+0000 7f42062ce640 1 -- 172.21.15.47:0/4213401774 wait complete. 2024-03-23T09:52:52.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:52:52.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.541+0000 7f1f091b1640 1 Processor -- start 2024-03-23T09:52:52.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.541+0000 7f1f091b1640 1 -- start start 2024-03-23T09:52:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.542+0000 7f1f091b1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f040cdcb0 0x7f1f040b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.542+0000 7f1f091b1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f040b72f0 con 0x7f1f040d2020 2024-03-23T09:52:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.542+0000 7f1f091b1640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f040b75d0 con 0x7f1f0415cb50 2024-03-23T09:52:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.542+0000 7f1f091b1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f040b7e70 con 0x7f1f040cdcb0 2024-03-23T09:52:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.542+0000 7f1f0359e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1f040d2020 0x7f1f040b5a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38162/0 (socket says 172.21.15.47:38162) 2024-03-23T09:52:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.542+0000 7f1f0359e640 1 -- 172.21.15.47:0/1616984341 learned_addr learned my addr 172.21.15.47:0/1616984341 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:52:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.542+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701715378 0 0) 0x7f1f040b72f0 con 0x7f1f040d2020 2024-03-23T09:52:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.542+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ef0003520 con 0x7f1f040d2020 2024-03-23T09:52:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.542+0000 7f1f0259c640 1 --2- 172.21.15.47:0/1616984341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f040cdcb0 0x7f1f040b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3837873989 0 0) 0x7f1ef0003520 con 0x7f1f040d2020 2024-03-23T09:52:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f040cdcb0 msgr2=0x7f1f040b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f01d9b640 1 --2- 172.21.15.47:0/1616984341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f040cdcb0 0x7f1f040b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 >> v1:172.21.15.175:6789/0 conn(0x7f1f0415cb50 legacy=0x7f1f040cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f040b78b0 con 0x7f1f040d2020 2024-03-23T09:52:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f1ef40019f0 con 0x7f1f040d2020 2024-03-23T09:52:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 >> v1:172.21.15.47:6789/0 conn(0x7f1f040d2020 legacy=0x7f1f040b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f01d9b640 1 --2- 172.21.15.47:0/1616984341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef00039a0 0x7f1ef0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f01d9b640 1 --2- 172.21.15.47:0/1616984341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ef00047c0 0x7f1ef0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f01d9b640 1 --2- 172.21.15.47:0/1616984341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f040cdcb0 0x7f1ef0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f040b7e70 con 0x7f1ef00039a0 2024-03-23T09:52:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef000bac0 con 0x7f1ef00047c0 2024-03-23T09:52:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef000bc80 con 0x7f1f040cdcb0 2024-03-23T09:52:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f02d9d640 1 --2- 172.21.15.47:0/1616984341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef00039a0 0x7f1ef0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f0259c640 1 --2- 172.21.15.47:0/1616984341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ef00047c0 0x7f1ef0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f0359e640 1 --2- 172.21.15.47:0/1616984341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f040cdcb0 0x7f1ef0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f02d9d640 1 -- 172.21.15.47:0/1616984341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f040cdcb0 msgr2=0x7f1ef0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f02d9d640 1 --2- 172.21.15.47:0/1616984341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f040cdcb0 0x7f1ef0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f02d9d640 1 -- 172.21.15.47:0/1616984341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ef00047c0 msgr2=0x7f1ef0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f02d9d640 1 --2- 172.21.15.47:0/1616984341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ef00047c0 0x7f1ef0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f02d9d640 1 -- 172.21.15.47:0/1616984341 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef0003520 con 0x7f1ef00039a0 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.543+0000 7f1f0359e640 1 --2- 172.21.15.47:0/1616984341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f040cdcb0 0x7f1ef0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.544+0000 7f1f02d9d640 1 --2- 172.21.15.47:0/1616984341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef00039a0 0x7f1ef0003dc0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f1efc001720 tx=0x7f1ef00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811f693865ee622d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.544+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0052d0 con 0x7f1ef00039a0 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.544+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc02e050 con 0x7f1ef00039a0 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.544+0000 7f1f01d9b640 1 -- 172.21.15.47:0/1616984341 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0035f0 con 0x7f1ef00039a0 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.544+0000 7f1f091b1640 1 -- 172.21.15.47:0/1616984341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef00039a0 msgr2=0x7f1ef0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.544+0000 7f1f091b1640 1 --2- 172.21.15.47:0/1616984341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef00039a0 0x7f1ef0003dc0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f1efc001720 tx=0x7f1ef00057f0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.544+0000 7f1f0359e640 1 -- 172.21.15.47:0/1616984341 reap_dead start 2024-03-23T09:52:52.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.545+0000 7f1f091b1640 1 -- 172.21.15.47:0/1616984341 shutdown_connections 2024-03-23T09:52:52.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.545+0000 7f1f091b1640 1 -- 172.21.15.47:0/1616984341 >> 172.21.15.47:0/1616984341 conn(0x7f1f04065df0 msgr2=0x7f1f04066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:52.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.545+0000 7f1f091b1640 1 -- 172.21.15.47:0/1616984341 shutdown_connections 2024-03-23T09:52:52.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.545+0000 7f1f091b1640 1 -- 172.21.15.47:0/1616984341 wait complete. 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.546+0000 7f1f091b1640 1 Processor -- start 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.546+0000 7f1f091b1640 1 -- start start 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f091b1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ef00039a0 0x7f1f041f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f091b1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f0415e410 0x7f1f040c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f091b1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f040c6790 0x7f1f040b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f091b1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f041f6860 con 0x7f1f0415e410 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f091b1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f041f58d0 con 0x7f1f040c6790 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f091b1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f040c6c10 con 0x7f1ef00039a0 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f0259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f0415e410 0x7f1f040c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f0259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f0415e410 0x7f1f040c6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40448/0 (socket says 172.21.15.47:40448) 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f0259c640 1 -- 172.21.15.47:0/998388926 learned_addr learned my addr 172.21.15.47:0/998388926 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f02d9d640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ef00039a0 0x7f1f041f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f0359e640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f040c6790 0x7f1f040b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f0259c640 1 -- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ef00039a0 msgr2=0x7f1f041f6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f0259c640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ef00039a0 0x7f1f041f6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f0259c640 1 -- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f040c6790 msgr2=0x7f1f040b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f0259c640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f040c6790 0x7f1f040b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f0259c640 1 -- 172.21.15.47:0/998388926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f040c6ef0 con 0x7f1f0415e410 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1f0259c640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f0415e410 0x7f1f040c6250 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0119a0 tx=0x7f1eec011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1efb7fe640 1 -- 172.21.15.47:0/998388926 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec00f9d0 con 0x7f1f0415e410 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.547+0000 7f1efb7fe640 1 -- 172.21.15.47:0/998388926 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec00fb30 con 0x7f1f0415e410 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.548+0000 7f1f091b1640 1 -- 172.21.15.47:0/998388926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f041f5b20 con 0x7f1f0415e410 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.548+0000 7f1efb7fe640 1 -- 172.21.15.47:0/998388926 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec00fea0 con 0x7f1f0415e410 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.548+0000 7f1f091b1640 1 -- 172.21.15.47:0/998388926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f04146470 con 0x7f1f0415e410 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.548+0000 7f1f091b1640 1 -- 172.21.15.47:0/998388926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f04072360 con 0x7f1f0415e410 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.549+0000 7f1efb7fe640 1 -- 172.21.15.47:0/998388926 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1eec014440 con 0x7f1f0415e410 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.549+0000 7f1efb7fe640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1ee406ac00 0x7f1ee406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.550+0000 7f1efb7fe640 1 -- 172.21.15.47:0/998388926 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f1eec089880 con 0x7f1f0415e410 2024-03-23T09:52:52.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.550+0000 7f1f02d9d640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1ee406ac00 0x7f1ee406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:52:52.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.550+0000 7f1f02d9d640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1ee406ac00 0x7f1ee406d0c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1efc033000 tx=0x7f1efc009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:52:52.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.554+0000 7f1efb7fe640 1 -- 172.21.15.47:0/998388926 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1eec054ca0 con 0x7f1f0415e410 2024-03-23T09:52:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.730+0000 7f1f091b1640 1 -- 172.21.15.47:0/998388926 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f04146690 con 0x7f1ee406ac00 2024-03-23T09:52:52.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.732+0000 7f1efb7fe640 1 -- 172.21.15.47:0/998388926 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50844 (secure 0 0 0) 0x7f1f04146690 con 0x7f1ee406ac00 2024-03-23T09:52:52.733 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:52:52.733 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:52:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.734+0000 7f1f091b1640 1 -- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1ee406ac00 msgr2=0x7f1ee406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.734+0000 7f1f091b1640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1ee406ac00 0x7f1ee406d0c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1efc033000 tx=0x7f1efc009030 comp rx=0 tx=0).stop 2024-03-23T09:52:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.734+0000 7f1f091b1640 1 -- 172.21.15.47:0/998388926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f0415e410 msgr2=0x7f1f040c6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:52:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.734+0000 7f1f091b1640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f0415e410 0x7f1f040c6250 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0119a0 tx=0x7f1eec011da0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.734+0000 7f1f091b1640 1 -- 172.21.15.47:0/998388926 shutdown_connections 2024-03-23T09:52:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.734+0000 7f1f091b1640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f1ee406ac00 0x7f1ee406d0c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.734+0000 7f1f091b1640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f040c6790 0x7f1f040b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.734+0000 7f1f091b1640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f0415e410 0x7f1f040c6250 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.734+0000 7f1f091b1640 1 --2- 172.21.15.47:0/998388926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ef00039a0 0x7f1f041f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:52:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.734+0000 7f1f091b1640 1 -- 172.21.15.47:0/998388926 >> 172.21.15.47:0/998388926 conn(0x7f1f04065df0 msgr2=0x7f1f040b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:52:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.735+0000 7f1f091b1640 1 -- 172.21.15.47:0/998388926 shutdown_connections 2024-03-23T09:52:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:52:52.735+0000 7f1f091b1640 1 -- 172.21.15.47:0/998388926 wait complete. 2024-03-23T09:52:52.746 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-03-23T09:52:52.517207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":278296,"kb_used_data":12960,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754696424,"statfs":{"total":773094113280,"available":772809138176,"internally_reserved":0,"allocated":13271040,"data_stored":6427434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":271568440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.575819"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":269,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-23T09:52:44.951631+0000","last_change":"2024-03-23T09:41:00.087176+0000","last_active":"2024-03-23T09:52:44.951631+0000","last_peered":"2024-03-23T09:52:44.951631+0000","last_clean":"2024-03-23T09:52:44.951631+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:52:44.951631+0000","last_undegraded":"2024-03-23T09:52:44.951631+0000","last_fullsized":"2024-03-23T09:52:44.951631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T09:40:58.719700+0000","last_clean_scrub_stamp":"2024-03-23T09:40:58.719700+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33176,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338664,"statfs":{"total":96636764160,"available":96602791936,"internally_reserved":0,"allocated":2105344,"data_stored":1246360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55200000000000005,"15min":0.52700000000000002},"min":{"1min":0.41399999999999998,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.623,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.44700000000000001,"5min":0.32200000000000001,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.78900000000000003,"15min":0.85999999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.40799999999999997,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.63,"5min":0.79100000000000004,"15min":0.80400000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.63600000000000001,"5min":0.747,"15min":0.81299999999999994},"last":0.47599999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55200000000000005},"min":{"1min":0.44500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.78500000000000003,"15min":0.82499999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.622,"5min":0.83099999999999996,"15min":0.83599999999999997},"last":0.64200000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.46000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.61699999999999999,"5min":0.73999999999999999,"15min":0.86199999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.627,"5min":0.77400000000000002,"15min":0.83699999999999997},"last":0.51900000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46700000000000003,"15min":0.48999999999999999},"min":{"1min":0.379,"5min":0.29799999999999999,"15min":0.29599999999999999},"max":{"1min":0.61699999999999999,"5min":0.67500000000000004,"15min":0.78200000000000003},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46700000000000003,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.29299999999999998},"max":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":0.80100000000000005},"last":0.66200000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.46500000000000002,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":0.69799999999999995,"15min":0.81499999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.498,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.63300000000000001,"5min":0.73199999999999998,"15min":0.84999999999999998},"last":0.63}]},{"osd":6,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.55100000000000005},"min":{"1min":0.34699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66300000000000003,"5min":0.75,"15min":0.874},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51200000000000001,"15min":0.55600000000000005},"min":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69199999999999995,"5min":0.73999999999999999,"15min":0.90200000000000002},"last":0.61899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1507328,"data_stored":655373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.53300000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.27200000000000002},"max":{"1min":0.746,"5min":0.77700000000000002,"15min":0.81000000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57199999999999995,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.55000000000000004}]},{"osd":1,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.311},"max":{"1min":0.77600000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.497},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.41299999999999998,"15min":0.32700000000000001},"max":{"1min":0.751,"5min":0.80300000000000005,"15min":0.82999999999999996},"last":0.63500000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.29699999999999999},"max":{"1min":0.69499999999999995,"5min":0.747,"15min":0.81599999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59099999999999997,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54100000000000004}]},{"osd":3,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.42399999999999999,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73299999999999998,"5min":0.79000000000000004,"15min":0.81699999999999995},"last":0.56599999999999995}]},{"osd":4,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.48199999999999998},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65600000000000003,"5min":0.69299999999999995,"15min":0.72499999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.505,"5min":0.47599999999999998,"15min":0.51700000000000002},"min":{"1min":0.35899999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.63800000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.68500000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48099999999999998,"15min":0.505},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.316},"max":{"1min":0.64100000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51200000000000001,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.623}]},{"osd":7,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52200000000000002,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.72899999999999998,"5min":0.754,"15min":0.82699999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54100000000000004,"15min":0.56499999999999995},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69299999999999995,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1511424,"data_stored":655992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54300000000000004,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.68700000000000006,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.39400000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73699999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.45600000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69799999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.40500000000000003}]},{"osd":2,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.55100000000000005},"min":{"1min":0.38600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71999999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.54800000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.748,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.51800000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58499999999999996,"15min":0.54700000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70899999999999996,"5min":0.77400000000000002,"15min":0.81599999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.77400000000000002},"last":0.5}]},{"osd":4,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29999999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68500000000000005,"5min":0.73099999999999998,"15min":0.73699999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.497,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.93600000000000005},"last":0.38500000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.495,"15min":0.496},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63400000000000001,"5min":0.747,"15min":0.749},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65000000000000002,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33300000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.54700000000000004},"min":{"1min":0.40999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.78200000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77500000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.54900000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32596,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1511424,"data_stored":655992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.54600000000000004},"min":{"1min":0.34300000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.84599999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.78600000000000003},"last":0.49399999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.52300000000000002,"15min":0.50700000000000001},"min":{"1min":0.36799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63500000000000001,"5min":0.72899999999999998,"15min":0.72999999999999998},"last":0.438},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.28999999999999998},"max":{"1min":0.68700000000000006,"5min":0.751,"15min":0.77400000000000002},"last":0.39900000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58999999999999997,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.39400000000000002},"max":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.80200000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59699999999999998,"15min":0.57299999999999995},"min":{"1min":0.47699999999999998,"5min":0.39700000000000002,"15min":0.372},"max":{"1min":0.73099999999999998,"5min":0.75600000000000001,"15min":0.82199999999999995},"last":0.51800000000000002}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68300000000000005,"5min":0.751,"15min":1.0369999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.58999999999999997,"5min":0.77900000000000003,"15min":0.80300000000000005},"last":0.55500000000000005}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29699999999999999,"15min":0.26500000000000001},"max":{"1min":0.77400000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46300000000000002,"15min":0.48799999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.253},"max":{"1min":0.73099999999999998,"5min":0.73199999999999998,"15min":1.008},"last":0.53900000000000003}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46700000000000003,"15min":0.47199999999999998},"min":{"1min":0.29999999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63200000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.82399999999999995},"last":0.47299999999999998}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33500000000000002,"15min":0.30099999999999999},"max":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":1.0069999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70799999999999996,"5min":0.77600000000000002,"15min":0.93400000000000005},"last":0.60299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36692,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1511424,"data_stored":655992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33900000000000002,"15min":0.33300000000000002},"min":{"1min":0.214,"5min":0.192,"15min":0.183},"max":{"1min":0.52300000000000002,"5min":0.67800000000000005,"15min":1.45},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35699999999999998,"15min":0.34100000000000003},"min":{"1min":0.23400000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.499,"5min":0.60499999999999998,"15min":1.4099999999999999},"last":0.20999999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.48299999999999998,"5min":0.56699999999999995,"15min":1.323},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.36699999999999999},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.94599999999999995},"last":0.25900000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.42799999999999999,"15min":0.40799999999999997},"min":{"1min":0.314,"5min":0.27100000000000002,"15min":0.251},"max":{"1min":0.55600000000000005,"5min":0.68999999999999995,"15min":1.1619999999999999},"last":0.378},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.43099999999999999},"min":{"1min":0.313,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.59299999999999997,"5min":0.64900000000000002,"15min":1.2989999999999999},"last":0.32000000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.35399999999999998},"max":{"1min":0.85399999999999998,"5min":0.88600000000000001,"15min":1.337},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.41599999999999998,"5min":0.35699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":2.758},"last":0.48899999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.61099999999999999},"min":{"1min":0.49199999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.872,"5min":0.872,"15min":1.276},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.77100000000000002,"5min":0.91200000000000003,"15min":1.895},"last":0.54800000000000004}]},{"osd":6,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45100000000000001,"5min":0.41299999999999998,"15min":0.40699999999999997},"max":{"1min":0.78500000000000003,"5min":0.84199999999999997,"15min":5.4539999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.432,"5min":0.41599999999999998,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":0.96499999999999997,"15min":2.8159999999999998},"last":0.59299999999999997}]},{"osd":7,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.625,"15min":0.622},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.94899999999999995,"15min":3.7400000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.40300000000000002},"max":{"1min":0.91900000000000004,"5min":0.93600000000000005,"15min":5.4900000000000002},"last":0.60699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36692,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1511424,"data_stored":655992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34300000000000003,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.19500000000000001,"15min":0.191},"max":{"1min":0.51100000000000001,"5min":0.55500000000000005,"15min":0.60399999999999998},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.22},"max":{"1min":0.50600000000000001,"5min":0.59699999999999998,"15min":0.872},"last":0.34499999999999997}]},{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":0.55100000000000005,"5min":0.60899999999999999,"15min":0.72399999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39800000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.20899999999999999},"max":{"1min":0.51500000000000001,"5min":0.61199999999999999,"15min":0.746},"last":0.313}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41699999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.27600000000000002,"15min":0.22900000000000001},"max":{"1min":0.55100000000000005,"5min":0.65200000000000002,"15min":0.67100000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.39200000000000002,"15min":0.39700000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":0.503,"5min":0.59199999999999997,"15min":0.72499999999999998},"last":0.33400000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81499999999999995,"5min":0.87,"15min":1.216},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81699999999999995,"5min":0.94699999999999995,"15min":0.99299999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.625,"15min":0.627},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.373},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.95699999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.628},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83099999999999996,"5min":0.98499999999999999,"15min":1.101},"last":0.51600000000000001}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.97299999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":1.0780000000000001},"last":0.54400000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":1.155},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.41599999999999998},"max":{"1min":0.81599999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36692,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1511424,"data_stored":655992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.223,"5min":0.185,"15min":0.185},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.317},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38500000000000001,"15min":0.41999999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.20699999999999999},"max":{"1min":0.57799999999999996,"5min":0.59099999999999997,"15min":2.7069999999999999},"last":0.28499999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.35599999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.46400000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.378}]},{"osd":3,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.432,"15min":0.42199999999999999},"min":{"1min":0.29399999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55100000000000005,"5min":0.746,"15min":0.76900000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.40400000000000003,"15min":0.44700000000000001},"min":{"1min":0.26100000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.63,"5min":0.73999999999999999,"15min":2.75},"last":0.33600000000000002}]},{"osd":4,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.61899999999999999,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.29099999999999998},"max":{"1min":0.83399999999999996,"5min":0.95899999999999996,"15min":0.97299999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.61699999999999999},"min":{"1min":0.42799999999999999,"5min":0.39400000000000002,"15min":0.38600000000000001},"max":{"1min":0.84599999999999997,"5min":0.873,"15min":1.002},"last":0.49199999999999999}]},{"osd":5,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63100000000000001,"15min":0.66000000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35299999999999998},"max":{"1min":1.026,"5min":1.026,"15min":2.778},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.88800000000000001,"15min":1.0649999999999999},"last":0.46000000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64200000000000002,"15min":0.66900000000000004},"min":{"1min":0.44600000000000001,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":2.8210000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.438,"15min":0.38800000000000001},"max":{"1min":0.80600000000000005,"5min":0.93300000000000005,"15min":1.1080000000000001},"last":0.54800000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.95399999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.43099999999999999,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":2.7989999999999999},"last":0.623}]}]},{"osd":0,"up_from":13,"seq":55834574993,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":2101248,"data_stored":1245741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.34499999999999997,"15min":0.33500000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.184},"max":{"1min":0.52300000000000002,"5min":0.625,"15min":0.625},"last":0.17100000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.24099999999999999,"5min":0.22900000000000001,"15min":0.217},"max":{"1min":0.49099999999999999,"5min":0.61699999999999999,"15min":0.64900000000000002},"last":0.248}]},{"osd":2,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38400000000000001,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.58099999999999996,"5min":0.63,"15min":0.63},"last":0.315},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.23799999999999999,"5min":0.22600000000000001,"15min":0.21199999999999999},"max":{"1min":0.497,"5min":0.58199999999999996,"15min":0.59399999999999997},"last":0.187}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39300000000000002,"15min":0.38900000000000001},"min":{"1min":0.30299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.44700000000000001,"15min":0.437},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55800000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.28599999999999998}]},{"osd":4,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58699999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67000000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.434},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.48099999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.35299999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71799999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.42299999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.94299999999999995,"15min":0.95999999999999996},"last":0.51800000000000002}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78500000000000003,"5min":0.91900000000000004,"15min":0.93600000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.46600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.52900000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61899999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.40999999999999998,"15min":0.39300000000000002},"max":{"1min":0.753,"5min":0.85499999999999998,"15min":0.93600000000000005},"last":0.505},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73999999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:52:52.750 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-23T09:40:58.719700+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=23, tm_hour=9, tm_min=40, tm_sec=58, tm_wday=5, tm_yday=83, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=23, tm_hour=9, tm_min=52, tm_sec=47, tm_wday=5, tm_yday=83, tm_isdst=0) 2024-03-23T09:52:52.750 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-23T09:53:09.371 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:53:09.375 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:53:12.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T09:53:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.865+0000 7f8d4fd55640 1 Processor -- start 2024-03-23T09:53:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.865+0000 7f8d4fd55640 1 -- start start 2024-03-23T09:53:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4fd55640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8d4806e080 0x7f8d48159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4fd55640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d4815cb30 0x7f8d480701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4fd55640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d48159ac0 con 0x7f8d4815cfc0 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4fd55640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d4815db20 con 0x7f8d4815cb30 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4fd55640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d481594e0 con 0x7f8d4806e080 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4d2f0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d4815cb30 0x7f8d480701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4d2f0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d4815cb30 0x7f8d480701f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49918/0 (socket says 172.21.15.47:49918) 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4d2f0640 1 -- 172.21.15.47:0/625719378 learned_addr learned my addr 172.21.15.47:0/625719378 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4daf1640 1 --2- 172.21.15.47:0/625719378 >> v2:172.21.15.175:3300/0 conn(0x7f8d4806e080 0x7f8d48159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4d2f0640 1 -- 172.21.15.47:0/625719378 >> v2:172.21.15.175:3300/0 conn(0x7f8d4806e080 msgr2=0x7f8d48159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4d2f0640 1 --2- 172.21.15.47:0/625719378 >> v2:172.21.15.175:3300/0 conn(0x7f8d4806e080 0x7f8d48159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4d2f0640 1 -- 172.21.15.47:0/625719378 >> v1:172.21.15.175:6789/0 conn(0x7f8d4815cfc0 legacy=0x7f8d480cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4d2f0640 1 -- 172.21.15.47:0/625719378 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4815a490 con 0x7f8d4815cb30 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.866+0000 7f8d4daf1640 1 --2- 172.21.15.47:0/625719378 >> v2:172.21.15.175:3300/0 conn(0x7f8d4806e080 0x7f8d48159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.867+0000 7f8d4d2f0640 1 --2- 172.21.15.47:0/625719378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4815cb30 0x7f8d480701f0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f8d480713b0 tx=0x7f8d3c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3564f3781a89cc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.867+0000 7f8d4caef640 1 -- 172.21.15.47:0/625719378 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c02ee90 con 0x7f8d4815cb30 2024-03-23T09:53:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.867+0000 7f8d4caef640 1 -- 172.21.15.47:0/625719378 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c036440 con 0x7f8d4815cb30 2024-03-23T09:53:12.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.867+0000 7f8d4caef640 1 -- 172.21.15.47:0/625719378 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0367d0 con 0x7f8d4815cb30 2024-03-23T09:53:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.868+0000 7f8d4fd55640 1 -- 172.21.15.47:0/625719378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4815cb30 msgr2=0x7f8d480701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.868+0000 7f8d4fd55640 1 --2- 172.21.15.47:0/625719378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4815cb30 0x7f8d480701f0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f8d480713b0 tx=0x7f8d3c02e0e0 comp rx=0 tx=0).stop 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.868+0000 7f8d4fd55640 1 -- 172.21.15.47:0/625719378 shutdown_connections 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.868+0000 7f8d4fd55640 1 --2- 172.21.15.47:0/625719378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4815cb30 0x7f8d480701f0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.868+0000 7f8d4fd55640 1 --2- 172.21.15.47:0/625719378 >> v2:172.21.15.175:3300/0 conn(0x7f8d4806e080 0x7f8d48159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.868+0000 7f8d4fd55640 1 -- 172.21.15.47:0/625719378 >> 172.21.15.47:0/625719378 conn(0x7f8d4806a310 msgr2=0x7f8d480b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.868+0000 7f8d4fd55640 1 -- 172.21.15.47:0/625719378 shutdown_connections 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.868+0000 7f8d4fd55640 1 -- 172.21.15.47:0/625719378 wait complete. 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.869+0000 7f8d4fd55640 1 Processor -- start 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.869+0000 7f8d4fd55640 1 -- start start 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4fd55640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d4806e080 0x7f8d480b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4fd55640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4815cb30 0x7f8d4814bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4fd55640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d4815cfc0 0x7f8d48158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4fd55640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480af130 con 0x7f8d4815cb30 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4fd55640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4814c6f0 con 0x7f8d4806e080 2024-03-23T09:53:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4fd55640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4814c9d0 con 0x7f8d4815cfc0 2024-03-23T09:53:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4d2f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4815cb30 0x7f8d4814bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4d2f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4815cb30 0x7f8d4814bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49934/0 (socket says 172.21.15.47:49934) 2024-03-23T09:53:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4d2f0640 1 -- 172.21.15.47:0/2047796462 learned_addr learned my addr 172.21.15.47:0/2047796462 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:53:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4daf1640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d4806e080 0x7f8d480b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4e2f2640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d4815cfc0 0x7f8d48158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4d2f0640 1 -- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d4815cfc0 msgr2=0x7f8d48158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4d2f0640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d4815cfc0 0x7f8d48158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4d2f0640 1 -- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d4806e080 msgr2=0x7f8d480b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4d2f0640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d4806e080 0x7f8d480b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4d2f0640 1 -- 172.21.15.47:0/2047796462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c00b770 con 0x7f8d4815cb30 2024-03-23T09:53:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.870+0000 7f8d4e2f2640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d4815cfc0 0x7f8d48158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:53:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.871+0000 7f8d4d2f0640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4815cb30 0x7f8d4814bdf0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c02e120 tx=0x7f8d3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:53:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.871+0000 7f8d467fc640 1 -- 172.21.15.47:0/2047796462 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c034b80 con 0x7f8d4815cb30 2024-03-23T09:53:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.871+0000 7f8d467fc640 1 -- 172.21.15.47:0/2047796462 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c034d00 con 0x7f8d4815cb30 2024-03-23T09:53:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.871+0000 7f8d4fd55640 1 -- 172.21.15.47:0/2047796462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4814ccb0 con 0x7f8d4815cb30 2024-03-23T09:53:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.871+0000 7f8d467fc640 1 -- 172.21.15.47:0/2047796462 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c041680 con 0x7f8d4815cb30 2024-03-23T09:53:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.871+0000 7f8d4fd55640 1 -- 172.21.15.47:0/2047796462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4814c330 con 0x7f8d4815cb30 2024-03-23T09:53:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.872+0000 7f8d4fd55640 1 -- 172.21.15.47:0/2047796462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d48073480 con 0x7f8d4815cb30 2024-03-23T09:53:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.872+0000 7f8d467fc640 1 -- 172.21.15.47:0/2047796462 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8d3c0403e0 con 0x7f8d4815cb30 2024-03-23T09:53:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.873+0000 7f8d467fc640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d3406ac30 0x7f8d3406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.873+0000 7f8d467fc640 1 -- 172.21.15.47:0/2047796462 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f8d3c0ab850 con 0x7f8d4815cb30 2024-03-23T09:53:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.873+0000 7f8d4daf1640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d3406ac30 0x7f8d3406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.874+0000 7f8d4daf1640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d3406ac30 0x7f8d3406d0f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000a990 tx=0x7f8d40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:53:12.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:12.877+0000 7f8d467fc640 1 -- 172.21.15.47:0/2047796462 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8d3c076c70 con 0x7f8d4815cb30 2024-03-23T09:53:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.054+0000 7f8d4fd55640 1 -- 172.21.15.47:0/2047796462 --> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8d480aef30 con 0x7f8d3406ac30 2024-03-23T09:53:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.058+0000 7f8d467fc640 1 -- 172.21.15.47:0/2047796462 <== mgr.4103 v2:172.21.15.175:6832/37590 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50934 (secure 0 0 0) 0x7f8d480aef30 con 0x7f8d3406ac30 2024-03-23T09:53:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:53:13.059 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-23T09:53:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.059+0000 7f8d4fd55640 1 -- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d3406ac30 msgr2=0x7f8d3406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.059+0000 7f8d4fd55640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d3406ac30 0x7f8d3406d0f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000a990 tx=0x7f8d40008040 comp rx=0 tx=0).stop 2024-03-23T09:53:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.059+0000 7f8d4fd55640 1 -- 172.21.15.47:0/2047796462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4815cb30 msgr2=0x7f8d4814bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.059+0000 7f8d4fd55640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4815cb30 0x7f8d4814bdf0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c02e120 tx=0x7f8d3c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.059+0000 7f8d4fd55640 1 -- 172.21.15.47:0/2047796462 shutdown_connections 2024-03-23T09:53:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.059+0000 7f8d4fd55640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f8d3406ac30 0x7f8d3406d0f0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.059+0000 7f8d4fd55640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d4815cfc0 0x7f8d48158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.059+0000 7f8d4fd55640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4815cb30 0x7f8d4814bdf0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.059+0000 7f8d4fd55640 1 --2- 172.21.15.47:0/2047796462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d4806e080 0x7f8d480b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.059+0000 7f8d4fd55640 1 -- 172.21.15.47:0/2047796462 >> 172.21.15.47:0/2047796462 conn(0x7f8d4806a310 msgr2=0x7f8d4815ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:53:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.059+0000 7f8d4fd55640 1 -- 172.21.15.47:0/2047796462 shutdown_connections 2024-03-23T09:53:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.059+0000 7f8d4fd55640 1 -- 172.21.15.47:0/2047796462 wait complete. 2024-03-23T09:53:13.070 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-03-23T09:53:12.521720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":278280,"kb_used_data":12960,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754696440,"statfs":{"total":773094113280,"available":772809154560,"internally_reserved":0,"allocated":13271040,"data_stored":6427434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":271568440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002797"},"pg_stats":[{"pgid":"1.0","version":"121'119","reported_seq":283,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-23T09:52:54.386233+0000","last_change":"2024-03-23T09:52:54.386233+0000","last_active":"2024-03-23T09:52:54.386233+0000","last_peered":"2024-03-23T09:52:54.386233+0000","last_clean":"2024-03-23T09:52:54.386233+0000","last_became_active":"2024-03-23T09:41:00.086231+0000","last_became_peered":"2024-03-23T09:41:00.086231+0000","last_unstale":"2024-03-23T09:52:54.386233+0000","last_undegraded":"2024-03-23T09:52:54.386233+0000","last_fullsized":"2024-03-23T09:52:54.386233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"121'119","last_scrub_stamp":"2024-03-23T09:52:54.386086+0000","last_deep_scrub":"121'119","last_deep_scrub_stamp":"2024-03-23T09:52:54.386086+0000","last_clean_scrub_stamp":"2024-03-23T09:52:54.386086+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:15:08.306718+0000","scrub_duration":0.0047374280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2105344,"data_stored":1246360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55200000000000005,"15min":0.52700000000000002},"min":{"1min":0.41399999999999998,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.623,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.44700000000000001,"5min":0.32200000000000001,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.78900000000000003,"15min":0.85999999999999999},"last":0.49299999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.40799999999999997,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.63,"5min":0.79100000000000004,"15min":0.80400000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.63600000000000001,"5min":0.747,"15min":0.81299999999999994},"last":0.52200000000000002}]},{"osd":2,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55200000000000005},"min":{"1min":0.44500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.78500000000000003,"15min":0.82499999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.622,"5min":0.83099999999999996,"15min":0.83599999999999997},"last":0.42899999999999999}]},{"osd":3,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.46000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.61699999999999999,"5min":0.73999999999999999,"15min":0.86199999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.627,"5min":0.77400000000000002,"15min":0.83699999999999997},"last":0.50700000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46700000000000003,"15min":0.48999999999999999},"min":{"1min":0.379,"5min":0.29799999999999999,"15min":0.29599999999999999},"max":{"1min":0.61699999999999999,"5min":0.67500000000000004,"15min":0.78200000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46700000000000003,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.29299999999999998},"max":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":0.80100000000000005},"last":0.40799999999999997}]},{"osd":5,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.46500000000000002,"15min":0.5},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":0.69799999999999995,"15min":0.81499999999999995},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.498,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.63300000000000001,"5min":0.73199999999999998,"15min":0.84999999999999998},"last":0.442}]},{"osd":6,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.55100000000000005},"min":{"1min":0.34699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66300000000000003,"5min":0.75,"15min":0.874},"last":0.377},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51200000000000001,"15min":0.55600000000000005},"min":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69199999999999995,"5min":0.73999999999999999,"15min":0.90200000000000002},"last":0.313}]}]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1507328,"data_stored":655373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.53300000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.27200000000000002},"max":{"1min":0.746,"5min":0.77700000000000002,"15min":0.81000000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57199999999999995,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69399999999999995}]},{"osd":1,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.311},"max":{"1min":0.77600000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.41299999999999998,"15min":0.32700000000000001},"max":{"1min":0.751,"5min":0.80300000000000005,"15min":0.82999999999999996},"last":0.73799999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.29699999999999999},"max":{"1min":0.69499999999999995,"5min":0.747,"15min":0.81599999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59099999999999997,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.53500000000000003}]},{"osd":3,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.42399999999999999,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73299999999999998,"5min":0.79000000000000004,"15min":0.81699999999999995},"last":0.58199999999999996}]},{"osd":4,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.48199999999999998},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65600000000000003,"5min":0.69299999999999995,"15min":0.72499999999999998},"last":0.434},{"interface":"front","average":{"1min":0.505,"5min":0.47599999999999998,"15min":0.51700000000000002},"min":{"1min":0.35899999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.63800000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.66200000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48099999999999998,"15min":0.505},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.316},"max":{"1min":0.64100000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51200000000000001,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.63600000000000001}]},{"osd":7,"last update":"Sat Mar 23 09:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52200000000000002,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.72899999999999998,"5min":0.754,"15min":0.82699999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54100000000000004,"15min":0.56499999999999995},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69299999999999995,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.50800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1511424,"data_stored":655992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54300000000000004,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.68700000000000006,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.39400000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73699999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52800000000000002}]},{"osd":1,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69799999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.503}]},{"osd":2,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.55100000000000005},"min":{"1min":0.38600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71999999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.54800000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.748,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.63700000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58499999999999996,"15min":0.54700000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70899999999999996,"5min":0.77400000000000002,"15min":0.81599999999999995},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.77400000000000002},"last":0.54100000000000004}]},{"osd":4,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29999999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68500000000000005,"5min":0.73099999999999998,"15min":0.73699999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.497,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.93600000000000005},"last":0.41699999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.495,"15min":0.496},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63400000000000001,"5min":0.747,"15min":0.749},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65000000000000002,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.40400000000000003}]},{"osd":7,"last update":"Sat Mar 23 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.54700000000000004},"min":{"1min":0.40999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.78200000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77500000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32596,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1511424,"data_stored":655992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.54600000000000004},"min":{"1min":0.34300000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.84599999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.78600000000000003},"last":0.48499999999999999}]},{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.52300000000000002,"15min":0.50700000000000001},"min":{"1min":0.36799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63500000000000001,"5min":0.72899999999999998,"15min":0.72999999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.28999999999999998},"max":{"1min":0.68700000000000006,"5min":0.751,"15min":0.77400000000000002},"last":0.38900000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58999999999999997,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.39400000000000002},"max":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.80200000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59699999999999998,"15min":0.57299999999999995},"min":{"1min":0.47699999999999998,"5min":0.39700000000000002,"15min":0.372},"max":{"1min":0.73099999999999998,"5min":0.75600000000000001,"15min":0.82199999999999995},"last":0.55300000000000005}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68300000000000005,"5min":0.751,"15min":1.0369999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.58999999999999997,"5min":0.77900000000000003,"15min":0.80300000000000005},"last":0.40600000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29699999999999999,"15min":0.26500000000000001},"max":{"1min":0.77400000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46300000000000002,"15min":0.48799999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.253},"max":{"1min":0.73099999999999998,"5min":0.73199999999999998,"15min":1.008},"last":0.36599999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46700000000000003,"15min":0.47199999999999998},"min":{"1min":0.29999999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63200000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.443},{"interface":"front","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.82399999999999995},"last":0.57299999999999995}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33500000000000002,"15min":0.30099999999999999},"max":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":1.0069999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70799999999999996,"5min":0.77600000000000002,"15min":0.93400000000000005},"last":0.498}]}]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36692,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1511424,"data_stored":655992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33900000000000002,"15min":0.33300000000000002},"min":{"1min":0.214,"5min":0.192,"15min":0.183},"max":{"1min":0.52300000000000002,"5min":0.67800000000000005,"15min":1.45},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35699999999999998,"15min":0.34100000000000003},"min":{"1min":0.23400000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.499,"5min":0.60499999999999998,"15min":1.4099999999999999},"last":0.51100000000000001}]},{"osd":1,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.48299999999999998,"5min":0.56699999999999995,"15min":1.323},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.36699999999999999},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.94599999999999995},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.42799999999999999,"15min":0.40799999999999997},"min":{"1min":0.314,"5min":0.27100000000000002,"15min":0.251},"max":{"1min":0.55600000000000005,"5min":0.68999999999999995,"15min":1.1619999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.43099999999999999},"min":{"1min":0.313,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.59299999999999997,"5min":0.64900000000000002,"15min":1.2989999999999999},"last":0.52900000000000003}]},{"osd":4,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.35399999999999998},"max":{"1min":0.85399999999999998,"5min":0.88600000000000001,"15min":1.337},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.41599999999999998,"5min":0.35699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":2.758},"last":0.621}]},{"osd":5,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.61099999999999999},"min":{"1min":0.49199999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.872,"5min":0.872,"15min":1.276},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.77100000000000002,"5min":0.91200000000000003,"15min":1.895},"last":0.84699999999999998}]},{"osd":6,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45100000000000001,"5min":0.41299999999999998,"15min":0.40699999999999997},"max":{"1min":0.78500000000000003,"5min":0.84199999999999997,"15min":5.4539999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.432,"5min":0.41599999999999998,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":0.96499999999999997,"15min":2.8159999999999998},"last":0.77700000000000002}]},{"osd":7,"last update":"Sat Mar 23 09:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.625,"15min":0.622},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.94899999999999995,"15min":3.7400000000000002},"last":0.879},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.40300000000000002},"max":{"1min":0.91900000000000004,"5min":0.93600000000000005,"15min":5.4900000000000002},"last":0.83199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36692,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1511424,"data_stored":655992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34300000000000003,"15min":0.34499999999999997},"min":{"1min":0.221,"5min":0.19500000000000001,"15min":0.191},"max":{"1min":0.51100000000000001,"5min":0.55500000000000005,"15min":0.60399999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.22},"max":{"1min":0.50600000000000001,"5min":0.59699999999999998,"15min":0.872},"last":0.46800000000000003}]},{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":0.55100000000000005,"5min":0.60899999999999999,"15min":0.72399999999999998},"last":0.44},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39800000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.20899999999999999},"max":{"1min":0.51500000000000001,"5min":0.61199999999999999,"15min":0.746},"last":0.51000000000000001}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41699999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.27600000000000002,"15min":0.22900000000000001},"max":{"1min":0.55100000000000005,"5min":0.65200000000000002,"15min":0.67100000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.39200000000000002,"15min":0.39700000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":0.503,"5min":0.59199999999999997,"15min":0.72499999999999998},"last":0.45200000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81499999999999995,"5min":0.87,"15min":1.216},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81699999999999995,"5min":0.94699999999999995,"15min":0.99299999999999999},"last":0.72599999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.625,"15min":0.627},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.373},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.95699999999999996},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.628},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83099999999999996,"5min":0.98499999999999999,"15min":1.101},"last":0.73699999999999999}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.97299999999999998},"last":0.747},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":1.0780000000000001},"last":0.79900000000000004}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":1.155},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.41599999999999998},"max":{"1min":0.81599999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36692,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1511424,"data_stored":655992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.223,"5min":0.185,"15min":0.185},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38500000000000001,"15min":0.41999999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.20699999999999999},"max":{"1min":0.57799999999999996,"5min":0.59099999999999997,"15min":2.7069999999999999},"last":0.38100000000000001}]},{"osd":2,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.36599999999999999,"15min":0.35599999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.46400000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.34599999999999997}]},{"osd":3,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.432,"15min":0.42199999999999999},"min":{"1min":0.29399999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55100000000000005,"5min":0.746,"15min":0.76900000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.40400000000000003,"15min":0.44700000000000001},"min":{"1min":0.26100000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.63,"5min":0.73999999999999999,"15min":2.75},"last":0.32500000000000001}]},{"osd":4,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.61899999999999999,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.29099999999999998},"max":{"1min":0.83399999999999996,"5min":0.95899999999999996,"15min":0.97299999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.61699999999999999},"min":{"1min":0.42799999999999999,"5min":0.39400000000000002,"15min":0.38600000000000001},"max":{"1min":0.84599999999999997,"5min":0.873,"15min":1.002},"last":0.72899999999999998}]},{"osd":5,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63100000000000001,"15min":0.66000000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35299999999999998},"max":{"1min":1.026,"5min":1.026,"15min":2.778},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.88800000000000001,"15min":1.0649999999999999},"last":0.58699999999999997}]},{"osd":6,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64200000000000002,"15min":0.66900000000000004},"min":{"1min":0.44600000000000001,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":2.8210000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.438,"15min":0.38800000000000001},"max":{"1min":0.80600000000000005,"5min":0.93300000000000005,"15min":1.1080000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.95399999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.43099999999999999,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":2.7989999999999999},"last":0.65000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37260,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334580,"statfs":{"total":96636764160,"available":96598609920,"internally_reserved":0,"allocated":2101248,"data_stored":1245741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.34499999999999997,"15min":0.33500000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.184},"max":{"1min":0.52300000000000002,"5min":0.625,"15min":0.625},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.24099999999999999,"5min":0.22900000000000001,"15min":0.217},"max":{"1min":0.49099999999999999,"5min":0.61699999999999999,"15min":0.64900000000000002},"last":0.41899999999999998}]},{"osd":2,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38400000000000001,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.58099999999999996,"5min":0.63,"15min":0.63},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.23799999999999999,"5min":0.22600000000000001,"15min":0.21199999999999999},"max":{"1min":0.497,"5min":0.58199999999999996,"15min":0.59399999999999997},"last":0.35699999999999998}]},{"osd":3,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39300000000000002,"15min":0.38900000000000001},"min":{"1min":0.30299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.44700000000000001,"15min":0.437},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55800000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.30099999999999999}]},{"osd":4,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58699999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67000000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.65400000000000003}]},{"osd":5,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.35299999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71799999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.42299999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.94299999999999995,"15min":0.95999999999999996},"last":0.626}]},{"osd":6,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78500000000000003,"5min":0.91900000000000004,"15min":0.93600000000000005},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.46600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Sat Mar 23 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61899999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.40999999999999998,"15min":0.39300000000000002},"max":{"1min":0.753,"5min":0.85499999999999998,"15min":0.93600000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73999999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T09:53:13.072 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-23T09:53:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.171+0000 7ff4ac64a640 1 Processor -- start 2024-03-23T09:53:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.171+0000 7ff4ac64a640 1 -- start start 2024-03-23T09:53:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.172+0000 7ff4ac64a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a415ca30 0x7ff4a4160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.172+0000 7ff4ac64a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff4a40cde40 0x7ff4a40d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.172+0000 7ff4ac64a640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a4161600 con 0x7ff4a415ce10 2024-03-23T09:53:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.172+0000 7ff4ac64a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a4161040 con 0x7ff4a40cde40 2024-03-23T09:53:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.172+0000 7ff4ac64a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a4161320 con 0x7ff4a415ca30 2024-03-23T09:53:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.172+0000 7ff4aabe7640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff4a40cde40 0x7ff4a40d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.172+0000 7ff4aa3e6640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a415ca30 0x7ff4a4160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.172+0000 7ff4aabe7640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff4a40cde40 0x7ff4a40d15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:33728/0 (socket says 172.21.15.47:33728) 2024-03-23T09:53:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.172+0000 7ff4aabe7640 1 -- 172.21.15.47:0/1319678000 learned_addr learned my addr 172.21.15.47:0/1319678000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:53:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.172+0000 7ff4aa3e6640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a415ca30 0x7ff4a4160740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:55886/0 (socket says 172.21.15.47:55886) 2024-03-23T09:53:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.173+0000 7ff4aa3e6640 1 -- 172.21.15.47:0/1319678000 >> v2:172.21.15.175:3300/0 conn(0x7ff4a40cde40 msgr2=0x7ff4a40d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:53:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.173+0000 7ff4aa3e6640 1 --2- 172.21.15.47:0/1319678000 >> v2:172.21.15.175:3300/0 conn(0x7ff4a40cde40 0x7ff4a40d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.173+0000 7ff4aa3e6640 1 -- 172.21.15.47:0/1319678000 >> v1:172.21.15.175:6789/0 conn(0x7ff4a415ce10 legacy=0x7ff4a40cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.173+0000 7ff4aa3e6640 1 -- 172.21.15.47:0/1319678000 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a40d0dd0 con 0x7ff4a415ca30 2024-03-23T09:53:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.173+0000 7ff4aabe7640 1 --2- 172.21.15.47:0/1319678000 >> v2:172.21.15.175:3300/0 conn(0x7ff4a40cde40 0x7ff4a40d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T09:53:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.174+0000 7ff4aa3e6640 1 --2- 172.21.15.47:0/1319678000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a415ca30 0x7ff4a4160740 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0097a0 tx=0x7ff49c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95b51416efa4caf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:53:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.174+0000 7ff4a93e4640 1 -- 172.21.15.47:0/1319678000 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c031830 con 0x7ff4a415ca30 2024-03-23T09:53:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.174+0000 7ff4a93e4640 1 -- 172.21.15.47:0/1319678000 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c031990 con 0x7ff4a415ca30 2024-03-23T09:53:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.174+0000 7ff4a93e4640 1 -- 172.21.15.47:0/1319678000 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c031d20 con 0x7ff4a415ca30 2024-03-23T09:53:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.174+0000 7ff4ac64a640 1 -- 172.21.15.47:0/1319678000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a415ca30 msgr2=0x7ff4a4160740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.174+0000 7ff4ac64a640 1 --2- 172.21.15.47:0/1319678000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a415ca30 0x7ff4a4160740 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0097a0 tx=0x7ff49c02b8d0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.174+0000 7ff4ac64a640 1 -- 172.21.15.47:0/1319678000 shutdown_connections 2024-03-23T09:53:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.174+0000 7ff4ac64a640 1 --2- 172.21.15.47:0/1319678000 >> v2:172.21.15.175:3300/0 conn(0x7ff4a40cde40 0x7ff4a40d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.174+0000 7ff4ac64a640 1 --2- 172.21.15.47:0/1319678000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a415ca30 0x7ff4a4160740 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.174+0000 7ff4ac64a640 1 -- 172.21.15.47:0/1319678000 >> 172.21.15.47:0/1319678000 conn(0x7ff4a406b160 msgr2=0x7ff4a406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:53:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.174+0000 7ff4ac64a640 1 -- 172.21.15.47:0/1319678000 shutdown_connections 2024-03-23T09:53:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.174+0000 7ff4ac64a640 1 -- 172.21.15.47:0/1319678000 wait complete. 2024-03-23T09:53:13.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.176+0000 7ff4ac64a640 1 Processor -- start 2024-03-23T09:53:13.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.176+0000 7ff4ac64a640 1 -- start start 2024-03-23T09:53:13.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.176+0000 7ff4ac64a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4a40cde40 0x7ff4a415f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.176+0000 7ff4ac64a640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a415ca30 0x7ff4a41f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.176+0000 7ff4ac64a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4a415ce10 0x7ff4a41f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.176+0000 7ff4ac64a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40c67a0 con 0x7ff4a40cde40 2024-03-23T09:53:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.176+0000 7ff4ac64a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40c6a80 con 0x7ff4a415ce10 2024-03-23T09:53:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.176+0000 7ff4ac64a640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40c6d60 con 0x7ff4a415ca30 2024-03-23T09:53:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.176+0000 7ff4aa3e6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4a40cde40 0x7ff4a415f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.176+0000 7ff4aa3e6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4a40cde40 0x7ff4a415f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49940/0 (socket says 172.21.15.47:49940) 2024-03-23T09:53:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.176+0000 7ff4aa3e6640 1 -- 172.21.15.47:0/3972984608 learned_addr learned my addr 172.21.15.47:0/3972984608 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:53:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4aabe7640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4a415ce10 0x7ff4a41f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4a9be5640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a415ca30 0x7ff4a41f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4aa3e6640 1 -- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a415ca30 msgr2=0x7ff4a41f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4aa3e6640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a415ca30 0x7ff4a41f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4aa3e6640 1 -- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4a415ce10 msgr2=0x7ff4a41f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4aa3e6640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4a415ce10 0x7ff4a41f6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4aa3e6640 1 -- 172.21.15.47:0/3972984608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff494009b30 con 0x7ff4a40cde40 2024-03-23T09:53:13.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4aabe7640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4a415ce10 0x7ff4a41f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T09:53:13.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4aa3e6640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4a40cde40 0x7ff4a415f8a0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0098c0 tx=0x7ff49c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:53:13.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4a2ffd640 1 -- 172.21.15.47:0/3972984608 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c03a6f0 con 0x7ff4a40cde40 2024-03-23T09:53:13.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4a2ffd640 1 -- 172.21.15.47:0/3972984608 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c03a870 con 0x7ff4a40cde40 2024-03-23T09:53:13.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4a2ffd640 1 -- 172.21.15.47:0/3972984608 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c03ac20 con 0x7ff4a40cde40 2024-03-23T09:53:13.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4ac64a640 1 -- 172.21.15.47:0/3972984608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c009400 con 0x7ff4a40cde40 2024-03-23T09:53:13.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.177+0000 7ff4ac64a640 1 -- 172.21.15.47:0/3972984608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a40c7320 con 0x7ff4a40cde40 2024-03-23T09:53:13.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.178+0000 7ff4ac64a640 1 -- 172.21.15.47:0/3972984608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a4072360 con 0x7ff4a40cde40 2024-03-23T09:53:13.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.179+0000 7ff4a2ffd640 1 -- 172.21.15.47:0/3972984608 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff49c030380 con 0x7ff4a40cde40 2024-03-23T09:53:13.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.179+0000 7ff4a2ffd640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff49006ac00 0x7ff49006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.179+0000 7ff4a2ffd640 1 -- 172.21.15.47:0/3972984608 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ff49c0aaa80 con 0x7ff4a40cde40 2024-03-23T09:53:13.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.179+0000 7ff4a9be5640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff49006ac00 0x7ff49006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.180+0000 7ff4a9be5640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff49006ac00 0x7ff49006d0c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40c8210 tx=0x7ff498009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:53:13.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.183+0000 7ff4a2ffd640 1 -- 172.21.15.47:0/3972984608 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff49c075eb0 con 0x7ff4a40cde40 2024-03-23T09:53:13.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.362+0000 7ff4ac64a640 1 -- 172.21.15.47:0/3972984608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7ff4a4066be0 con 0x7ff4a40cde40 2024-03-23T09:53:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.366+0000 7ff4a2ffd640 1 -- 172.21.15.47:0/3972984608 <== mon.0 v2:172.21.15.47:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff49c075eb0 con 0x7ff4a40cde40 2024-03-23T09:53:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.366+0000 7ff4a2ffd640 1 -- 172.21.15.47:0/3972984608 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7ff49c0350c0 con 0x7ff4a40cde40 2024-03-23T09:53:13.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.369+0000 7ff4ac64a640 1 -- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff49006ac00 msgr2=0x7ff49006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.369+0000 7ff4ac64a640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff49006ac00 0x7ff49006d0c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40c8210 tx=0x7ff498009040 comp rx=0 tx=0).stop 2024-03-23T09:53:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.369+0000 7ff4ac64a640 1 -- 172.21.15.47:0/3972984608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4a40cde40 msgr2=0x7ff4a415f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.369+0000 7ff4ac64a640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4a40cde40 0x7ff4a415f8a0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0098c0 tx=0x7ff49c0046c0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.369+0000 7ff4ac64a640 1 -- 172.21.15.47:0/3972984608 shutdown_connections 2024-03-23T09:53:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.369+0000 7ff4ac64a640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7ff49006ac00 0x7ff49006d0c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.369+0000 7ff4ac64a640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4a415ce10 0x7ff4a41f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.369+0000 7ff4ac64a640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a415ca30 0x7ff4a41f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.369+0000 7ff4ac64a640 1 --2- 172.21.15.47:0/3972984608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4a40cde40 0x7ff4a415f8a0 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.369+0000 7ff4ac64a640 1 -- 172.21.15.47:0/3972984608 >> 172.21.15.47:0/3972984608 conn(0x7ff4a406b160 msgr2=0x7ff4a4156ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:53:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.369+0000 7ff4ac64a640 1 -- 172.21.15.47:0/3972984608 shutdown_connections 2024-03-23T09:53:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.369+0000 7ff4ac64a640 1 -- 172.21.15.47:0/3972984608 wait complete. 2024-03-23T09:53:13.380 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T09:53:13.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T09:53:13.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.480+0000 7f505fd3c640 1 Processor -- start 2024-03-23T09:53:13.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.480+0000 7f505fd3c640 1 -- start start 2024-03-23T09:53:13.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.481+0000 7f505fd3c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f50580cf690 0x7f50580b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.481+0000 7f505fd3c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505815d0e0 con 0x7f50580cf210 2024-03-23T09:53:13.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.481+0000 7f505fd3c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50580d1540 con 0x7f505815cd00 2024-03-23T09:53:13.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.481+0000 7f505fd3c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50580d1820 con 0x7f50580cf690 2024-03-23T09:53:13.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.481+0000 7f505d2d7640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f50580cf210 0x7f50580b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59214/0 (socket says 172.21.15.47:59214) 2024-03-23T09:53:13.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.481+0000 7f505d2d7640 1 -- 172.21.15.47:0/1717072852 learned_addr learned my addr 172.21.15.47:0/1717072852 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-03-23T09:53:13.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.481+0000 7f505e2d9640 1 --2- 172.21.15.47:0/1717072852 >> v2:172.21.15.175:3300/0 conn(0x7f50580cf690 0x7f50580b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.481+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584982076 0 0) 0x7f505815d0e0 con 0x7f50580cf210 2024-03-23T09:53:13.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.481+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5044003520 con 0x7f50580cf210 2024-03-23T09:53:13.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1529779493 0 0) 0x7f5044003520 con 0x7f50580cf210 2024-03-23T09:53:13.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 >> v2:172.21.15.175:3300/0 conn(0x7f50580cf690 msgr2=0x7f50580b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:53:13.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505cad6640 1 --2- 172.21.15.47:0/1717072852 >> v2:172.21.15.175:3300/0 conn(0x7f50580cf690 0x7f50580b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 >> v1:172.21.15.175:6789/0 conn(0x7f505815cd00 legacy=0x7f50580d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50580d1b00 con 0x7f50580cf210 2024-03-23T09:53:13.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1221735819 0 0) 0x7f5048001670 con 0x7f50580cf210 2024-03-23T09:53:13.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 >> v1:172.21.15.47:6789/0 conn(0x7f50580cf210 legacy=0x7f50580b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505cad6640 1 --2- 172.21.15.47:0/1717072852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5044003970 0x7f5044003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505cad6640 1 --2- 172.21.15.47:0/1717072852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50440047c0 0x7f5044004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505cad6640 1 --2- 172.21.15.47:0/1717072852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5044005180 0x7f504400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50580d1820 con 0x7f5044005180 2024-03-23T09:53:13.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504400c120 con 0x7f5044003970 2024-03-23T09:53:13.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504400c2e0 con 0x7f50440047c0 2024-03-23T09:53:13.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505e2d9640 1 --2- 172.21.15.47:0/1717072852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5044005180 0x7f504400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505dad8640 1 --2- 172.21.15.47:0/1717072852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5044003970 0x7f5044003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505d2d7640 1 --2- 172.21.15.47:0/1717072852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50440047c0 0x7f5044004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505e2d9640 1 -- 172.21.15.47:0/1717072852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50440047c0 msgr2=0x7f5044004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:53:13.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505e2d9640 1 --2- 172.21.15.47:0/1717072852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50440047c0 0x7f5044004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505e2d9640 1 -- 172.21.15.47:0/1717072852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5044003970 msgr2=0x7f5044003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505e2d9640 1 --2- 172.21.15.47:0/1717072852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5044003970 0x7f5044003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.482+0000 7f505e2d9640 1 -- 172.21.15.47:0/1717072852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5044003520 con 0x7f5044005180 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.483+0000 7f505e2d9640 1 -- 172.21.15.47:0/1717072852 reap_dead start 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.483+0000 7f505e2d9640 1 --2- 172.21.15.47:0/1717072852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5044005180 0x7f504400b980 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f5054009fc0 tx=0x7f50440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a3accae2beed40 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.483+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505402cc60 con 0x7f5044005180 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.483+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f505400f040 con 0x7f5044005180 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.483+0000 7f505cad6640 1 -- 172.21.15.47:0/1717072852 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054003990 con 0x7f5044005180 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.484+0000 7f505fd3c640 1 -- 172.21.15.47:0/1717072852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5044005180 msgr2=0x7f504400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.484+0000 7f505fd3c640 1 --2- 172.21.15.47:0/1717072852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5044005180 0x7f504400b980 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f5054009fc0 tx=0x7f50440057f0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.484+0000 7f505fd3c640 1 -- 172.21.15.47:0/1717072852 shutdown_connections 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.484+0000 7f505fd3c640 1 --2- 172.21.15.47:0/1717072852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5044005180 0x7f504400b980 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.484+0000 7f505fd3c640 1 -- 172.21.15.47:0/1717072852 >> 172.21.15.47:0/1717072852 conn(0x7f505806b160 msgr2=0x7f505806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:53:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.484+0000 7f505fd3c640 1 -- 172.21.15.47:0/1717072852 shutdown_connections 2024-03-23T09:53:13.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.484+0000 7f505fd3c640 1 -- 172.21.15.47:0/1717072852 wait complete. 2024-03-23T09:53:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.485+0000 7f505fd3c640 1 Processor -- start 2024-03-23T09:53:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.485+0000 7f505fd3c640 1 -- start start 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505fd3c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5044005180 0x7f5058158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505fd3c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50580c72e0 0x7f50581f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505fd3c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f505815d2d0 0x7f505815d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505fd3c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50580b36a0 con 0x7f5044005180 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505fd3c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505815dc90 con 0x7f50580c72e0 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505fd3c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5058159370 con 0x7f505815d2d0 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505dad8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5044005180 0x7f5058158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505dad8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5044005180 0x7f5058158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49966/0 (socket says 172.21.15.47:49966) 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505dad8640 1 -- 172.21.15.47:0/646368186 learned_addr learned my addr 172.21.15.47:0/646368186 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505d2d7640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50580c72e0 0x7f50581f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505e2d9640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f505815d2d0 0x7f505815d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505dad8640 1 -- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f505815d2d0 msgr2=0x7f505815d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505dad8640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f505815d2d0 0x7f505815d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505dad8640 1 -- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50580c72e0 msgr2=0x7f50581f1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505dad8640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50580c72e0 0x7f50581f1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505dad8640 1 -- 172.21.15.47:0/646368186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5058159650 con 0x7f5044005180 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505d2d7640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50580c72e0 0x7f50581f1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505dad8640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5044005180 0x7f5058158c70 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f505000f5f0 tx=0x7f505000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f504e7fc640 1 -- 172.21.15.47:0/646368186 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500052e0 con 0x7f5044005180 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f504e7fc640 1 -- 172.21.15.47:0/646368186 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5050005440 con 0x7f5044005180 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505fd3c640 1 -- 172.21.15.47:0/646368186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5058159930 con 0x7f5044005180 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f504e7fc640 1 -- 172.21.15.47:0/646368186 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050003830 con 0x7f5044005180 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.486+0000 7f505fd3c640 1 -- 172.21.15.47:0/646368186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50580c7760 con 0x7f5044005180 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.487+0000 7f505fd3c640 1 -- 172.21.15.47:0/646368186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5058072360 con 0x7f5044005180 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.488+0000 7f504e7fc640 1 -- 172.21.15.47:0/646368186 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5050003990 con 0x7f5044005180 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.488+0000 7f504e7fc640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f503c06ac00 0x7f503c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.488+0000 7f504e7fc640 1 -- 172.21.15.47:0/646368186 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f5050088ff0 con 0x7f5044005180 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.488+0000 7f505d2d7640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f503c06ac00 0x7f503c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T09:53:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.489+0000 7f505d2d7640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f503c06ac00 0x7f503c06d0c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f50581e6ca0 tx=0x7f5048009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T09:53:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.493+0000 7f504e7fc640 1 -- 172.21.15.47:0/646368186 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5050054420 con 0x7f5044005180 2024-03-23T09:53:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.707+0000 7f505fd3c640 1 -- 172.21.15.47:0/646368186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5058066be0 con 0x7f5044005180 2024-03-23T09:53:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.708+0000 7f504e7fc640 1 -- 172.21.15.47:0/646368186 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f5050057a40 con 0x7f5044005180 2024-03-23T09:53:13.709 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-23T09:53:13.709 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/20684012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":20684012},{"type":"v1","addr":"172.21.15.175:6836","nonce":20684012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1707817799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":1707817799},{"type":"v1","addr":"172.21.15.47:6835","nonce":1707817799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7663,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2766927262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2766927262},{"type":"v1","addr":"172.21.15.175:6837","nonce":2766927262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9990,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/2149329979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2149329979},{"type":"v1","addr":"172.21.15.47:6834","nonce":2149329979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[]} 2024-03-23T09:53:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 95 2024-03-23T09:53:13.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.710+0000 7f505fd3c640 1 -- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f503c06ac00 msgr2=0x7f503c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.710+0000 7f505fd3c640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f503c06ac00 0x7f503c06d0c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f50581e6ca0 tx=0x7f5048009040 comp rx=0 tx=0).stop 2024-03-23T09:53:13.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.710+0000 7f505fd3c640 1 -- 172.21.15.47:0/646368186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5044005180 msgr2=0x7f5058158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T09:53:13.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.710+0000 7f505fd3c640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5044005180 0x7f5058158c70 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f505000f5f0 tx=0x7f505000fab0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.710+0000 7f505fd3c640 1 -- 172.21.15.47:0/646368186 shutdown_connections 2024-03-23T09:53:13.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.710+0000 7f505fd3c640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:6832/37590,v1:172.21.15.175:6833/37590] conn(0x7f503c06ac00 0x7f503c06d0c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.710+0000 7f505fd3c640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f505815d2d0 0x7f505815d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.710+0000 7f505fd3c640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50580c72e0 0x7f50581f1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.710+0000 7f505fd3c640 1 --2- 172.21.15.47:0/646368186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5044005180 0x7f5058158c70 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T09:53:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.710+0000 7f505fd3c640 1 -- 172.21.15.47:0/646368186 >> 172.21.15.47:0/646368186 conn(0x7f505806b160 msgr2=0x7f505815dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T09:53:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.710+0000 7f505fd3c640 1 -- 172.21.15.47:0/646368186 shutdown_connections 2024-03-23T09:53:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-23T09:53:13.710+0000 7f505fd3c640 1 -- 172.21.15.47:0/646368186 wait complete. 2024-03-23T09:53:13.721 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-23T09:53:13.721 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-23T09:53:13.722 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-23T09:53:13.722 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:13.730 INFO:tasks.ceph.mds.a:Stopped 2024-03-23T09:53:13.730 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-23T09:53:13.730 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:13.779 INFO:tasks.ceph.mds.c:Stopped 2024-03-23T09:53:13.779 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-23T09:53:13.779 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:13.793 INFO:tasks.ceph.mds.b:Stopped 2024-03-23T09:53:13.793 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-23T09:53:13.793 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:13.844 INFO:tasks.ceph.mds.d:Stopped 2024-03-23T09:53:13.844 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-23T09:53:13.844 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-23T09:53:13.844 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:13.875 INFO:tasks.ceph.osd.0:Stopped 2024-03-23T09:53:13.875 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-23T09:53:13.875 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:13.945 INFO:tasks.ceph.osd.1:Stopped 2024-03-23T09:53:13.945 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-23T09:53:13.945 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:14.014 INFO:tasks.ceph.osd.2:Stopped 2024-03-23T09:53:14.014 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-23T09:53:14.014 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:14.088 INFO:tasks.ceph.osd.3:Stopped 2024-03-23T09:53:14.088 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-23T09:53:14.088 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:14.116 INFO:tasks.ceph.osd.4:Stopped 2024-03-23T09:53:14.116 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-23T09:53:14.116 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:14.189 INFO:tasks.ceph.osd.5:Stopped 2024-03-23T09:53:14.189 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-23T09:53:14.189 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:14.268 INFO:tasks.ceph.osd.6:Stopped 2024-03-23T09:53:14.269 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-23T09:53:14.269 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:14.344 INFO:tasks.ceph.osd.7:Stopped 2024-03-23T09:53:14.344 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-23T09:53:14.344 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-23T09:53:14.344 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:14.369 INFO:tasks.ceph.mgr.y:Stopped 2024-03-23T09:53:14.369 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-23T09:53:14.369 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:14.413 INFO:tasks.ceph.mgr.x:Stopped 2024-03-23T09:53:14.413 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-23T09:53:14.413 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-23T09:53:14.413 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:14.424 INFO:tasks.ceph.mon.a:Stopped 2024-03-23T09:53:14.425 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-23T09:53:14.425 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:14.465 INFO:tasks.ceph.mon.b:Stopped 2024-03-23T09:53:14.465 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-23T09:53:14.465 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T09:53:14.518 INFO:tasks.ceph.mon.c:Stopped 2024-03-23T09:53:14.518 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-23T09:53:14.519 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-23T09:53:14.552 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:45:40.922490+0000 mds.c (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-03-23T09:53:14.552 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-03-23T09:53:14.553 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-23T09:53:14.625 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-23T09:53:14.699 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-23T09:47:35.694493+0000 mds.c (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-23T09:53:14.699 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:53:14.699 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-23T09:53:14.909 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:53:14.909 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-23T09:53:14.997 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:53:14.997 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-23T09:53:15.047 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:53:15.047 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-23T09:53:15.089 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:53:15.089 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-23T09:53:15.252 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:53:15.252 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-23T09:53:15.367 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:53:15.367 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-23T09:53:15.424 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:53:15.424 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-23T09:53:15.482 INFO:tasks.ceph:Archiving mon data... 2024-03-23T09:53:15.483 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427/data/mon.a.tgz 2024-03-23T09:53:15.483 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-03-23T09:53:15.498 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.MVyOZIyQVr 2024-03-23T09:53:15.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.MVyOZIyQVr 2024-03-23T09:53:15.646 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.MVyOZIyQVr 2024-03-23T09:53:15.729 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.MVyOZIyQVr is 376KB 2024-03-23T09:53:15.824 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.MVyOZIyQVr 2024-03-23T09:53:15.839 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427/data/mon.b.tgz 2024-03-23T09:53:15.839 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-03-23T09:53:15.856 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.XS4t7sjvii 2024-03-23T09:53:15.857 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.XS4t7sjvii 2024-03-23T09:53:16.004 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.XS4t7sjvii 2024-03-23T09:53:16.089 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.XS4t7sjvii is 379KB 2024-03-23T09:53:16.155 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.XS4t7sjvii 2024-03-23T09:53:16.169 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427/data/mon.c.tgz 2024-03-23T09:53:16.170 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-03-23T09:53:16.224 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.cxk8UKtHi6 2024-03-23T09:53:16.224 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.cxk8UKtHi6 2024-03-23T09:53:16.358 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.cxk8UKtHi6 2024-03-23T09:53:16.443 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.cxk8UKtHi6 is 379KB 2024-03-23T09:53:16.508 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.cxk8UKtHi6 2024-03-23T09:53:16.522 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-23T09:53:16.522 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-23T09:53:16.524 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-23T09:53:16.588 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-03-23T09:53:16.588 DEBUG:teuthology.orchestra.run.smithi047:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-23T09:53:16.593 DEBUG:teuthology.orchestra.run.smithi175:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-23T09:53:16.622 INFO:teuthology.orchestra.run.smithi047.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-23T09:53:16.657 INFO:teuthology.orchestra.run.smithi175.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-23T09:53:16.661 INFO:tasks.ceph:Archiving crash dumps... 2024-03-23T09:53:16.663 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427/remote/smithi047/crash 2024-03-23T09:53:16.664 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-23T09:53:16.694 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427/remote/smithi175/crash 2024-03-23T09:53:16.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-23T09:53:16.726 INFO:tasks.ceph:Shutting down logrotate 2024-03-23T09:53:16.727 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:53:16.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T09:53:16.817 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-23T09:53:16.844 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-23T09:53:16.870 INFO:tasks.ceph:Compressing logs... 2024-03-23T09:53:16.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T09:53:16.887 DEBUG:teuthology.orchestra.run.smithi175:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T09:53:20.921 INFO:tasks.ceph:Archiving logs... 2024-03-23T09:53:20.922 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427/remote/smithi047/log 2024-03-23T09:53:20.923 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T09:53:21.996 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427/remote/smithi175/log 2024-03-23T09:53:21.997 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T09:53:22.412 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-23T09:53:22.421 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-23T09:53:22.421 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T09:53:22.424 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T09:53:22.508 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-23T09:53:22.548 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-23T09:53:22.548 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T09:53:22.551 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T09:53:22.565 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-03-23T09:53:22.566 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-03-23T09:53:22.570 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T09:53:22.570 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-03-23T09:53:22.570 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 16 +419us[ +456us] +/- 66ms 2024-03-23T09:53:22.570 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -1089us[-1052us] +/- 76ms 2024-03-23T09:53:22.570 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +539us[ +577us] +/- 68ms 2024-03-23T09:53:22.570 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T09:53:22.572 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T09:53:22.572 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-03-23T09:53:22.572 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 16 +426us[ +468us] +/- 66ms 2024-03-23T09:53:22.572 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -1088us[-1047us] +/- 76ms 2024-03-23T09:53:22.572 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 80 +540us[ +595us] +/- 67ms 2024-03-23T09:53:22.572 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T09:53:22.572 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-23T09:53:22.581 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-23T09:53:22.582 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-23T09:53:22.590 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T09:53:22.712 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T09:53:22.783 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T09:53:22.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:53:22.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T09:53:22.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:53:22.846 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-23T09:53:22.856 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-23T09:53:22.902 INFO:teuthology.task.internal:Duration was 1177.419360 seconds 2024-03-23T09:53:22.903 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-23T09:53:22.912 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-23T09:53:22.912 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T09:53:22.915 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T09:53:22.957 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T09:53:22.958 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T09:53:23.227 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-23T09:53:23.228 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-03-23T09:53:23.228 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T09:53:23.247 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-03-23T09:53:23.247 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T09:53:23.293 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-23T09:53:23.294 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T09:53:23.296 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T09:53:23.362 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-23T09:53:23.362 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T09:53:23.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T09:53:23.505 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-23T09:53:23.515 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-23T09:53:23.515 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T09:53:23.541 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T09:53:23.568 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-23T09:53:23.578 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T09:53:23.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T09:53:23.607 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-03-23T09:53:23.634 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-03-23T09:53:23.650 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T09:53:23.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:53:23.676 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T09:53:23.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T09:53:23.704 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-23T09:53:23.713 INFO:teuthology.task.internal:Transferring archived files... 2024-03-23T09:53:23.714 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427/remote/smithi047 2024-03-23T09:53:23.714 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T09:53:23.756 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427/remote/smithi175 2024-03-23T09:53:23.756 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T09:53:23.790 INFO:teuthology.task.internal:Removing archive directory... 2024-03-23T09:53:23.791 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T09:53:23.793 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T09:53:23.841 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-23T09:53:23.852 INFO:teuthology.task.internal:Not uploading archives. 2024-03-23T09:53:23.852 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-23T09:53:23.861 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-23T09:53:23.861 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T09:53:23.863 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T09:53:23.874 INFO:teuthology.orchestra.run.smithi047.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 09:53 /home/ubuntu/cephtest 2024-03-23T09:53:23.895 INFO:teuthology.orchestra.run.smithi175.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 09:53 /home/ubuntu/cephtest 2024-03-23T09:53:23.896 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-23T09:53:23.906 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-23T09:53:23.926 INFO:teuthology.nuke:Checking targets against current locks 2024-03-23T09:53:23.952 DEBUG:teuthology.nuke:shortname: smithi047 2024-03-23T09:53:23.953 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T09:53:23.976 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 09:24:09.621044', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN4NZx+BuJhDznyGKe/diyX8WDTP3EKrADaXXu6wf534XL6d+2TjpBMY3IxKxezfWgv0a/reLAHWTfEnXFwD10w='} 2024-03-23T09:53:23.979 DEBUG:teuthology.nuke:shortname: smithi175 2024-03-23T09:53:23.980 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T09:53:24.002 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617427', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 09:24:09.622540', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIdxuUty99rrpzpD84B71aZ9IUxYu6J/FBGUB3ehkllQxBfNgtQAITD6ApkLFMCuIWn2rUen4iMnaOezzkYrt+4='} 2024-03-23T09:53:24.032 INFO:teuthology.orchestra.console.smithi047:Power off 2024-03-23T09:53:24.032 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T09:53:24.055 DEBUG:teuthology.orchestra.console.smithi047:power off output: Chassis Power Control: Down/Off 2024-03-23T09:53:24.055 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T09:53:24.070 INFO:teuthology.orchestra.console.smithi175:Power off 2024-03-23T09:53:24.070 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T09:53:24.081 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on 2024-03-23T09:53:24.092 DEBUG:teuthology.orchestra.console.smithi175:power off output: Chassis Power Control: Down/Off 2024-03-23T09:53:24.092 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T09:53:24.116 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2024-03-23T09:53:28.083 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T09:53:28.118 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T09:53:28.192 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on 2024-03-23T09:53:28.226 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2024-03-23T09:53:32.194 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T09:53:32.227 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T09:53:32.303 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is off 2024-03-23T09:53:32.337 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2024-03-23T09:53:32.404 INFO:teuthology.orchestra.console.smithi047:Power off completed 2024-03-23T09:53:36.339 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T09:53:36.448 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is off 2024-03-23T09:53:36.549 INFO:teuthology.orchestra.console.smithi175:Power off completed 2024-03-23T09:53:36.653 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1177.4193596839905 failure_reason: '"2024-03-23T09:47:35.694493+0000 mds.c (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2024-03-23T09:53:36.654 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T09:53:36.738 INFO:teuthology.run:FAIL